Uploading Data with Flatfile

There are currently two ways of uploading data into Accelerate, using our legacy CSV Dataloader or by using our integration with Flatfile.io. For guidance on using the legacy CSV Dataloader, please see our documentation on this here. When selecting how you will be uploading data, there is a checkbox that you can tick which will bypass Flatfile completely if you want to force the usage of the legacy form. This setting will be saved in your browser so that you don't have to keep setting this every time you use the Dataloader.

Flatfile is a data onboarding tool which has been integrated with the Dataloader to greatly improve the user experience. This is done by helping to match up the headers of the uploaded file with the template and showing real-time errors and warnings to prevent as many mistakes as possible from being carried through the Dataloader process.

The validation checks, which are automatically run against your data, range from basic required field checks to ensuring that an external ID or reference can actually be resolved by the Dataloader. For example, it can check if a user email address you entered can be matched against a system user in your Accelerate client.

While it allows you to upload CSV files containing your data, Flatfile also allows you to manually enter data into its UI which can save you time if you are only uploading a small amount of data.

Setting User Dataloader Access

Anyone can access the Dataloader but only an administrator can grant access, to do this the administrator must go into Settings > Users & select the user they wish to have access to the Dataloader.

Within the Dataloader Access section, the relevant Dataloader Type can be selected, and then added to the user profile by selecting “Add Dataloader Type”:

Uploading Data

The first step of uploading data is selecting the type of data that you want to upload from the dropdown menu. If you do not see the type of data that you want to upload, it means it is either unsupported or your user has not been given the permission to upload data of that type.

Once you select the data type, the Next button will be enabled. When it is clicked, the Dataloader will fetch the template for that data type. The Flatfile integration is not available for every data type (namely Ratebooks) at this time, but this will change in the future.

Uploading Data

If Flatfile is not available for the selected type then you will be met with a simple form for selecting a CSV file for upload. If you have selected the wrong type by mistake, then you can return to the previous form by clicking on the Change Data Type button. 

However, if Flatfile has been integrated for the desired type, a form populated by Flatfile for inputting your data will appear. The first page you will see every time this opens is the one for providing the data you want to load in. The legacy Dataloader process only allowed you to upload data by attaching a file directly whereas Flatfile allows you to both upload a file normally and also manually enter data. If you choose to manually edit data, you will skip the header matching process as you will be inserting data directly into the provided template (see Repair Data section).

Matching Headers

If you choose to upload data from a file, then you will be met with a page where you should ensure that the headers in your file match up to the fields in the template. Flatfile will do its best to automatically match up these headers and most of the time it will be able to do so. Flatfile will show you the fields it has matched in a long list - one entry for each field. You can click on the "Confirm mapping" button when you are sure that the field has been matched correctly. You can also change this by clicking on the "Edit" button that appears on a confirmed mapping. You can manually assign a mapping by clicking on the dropdown - there you can select the field in the template that you want that specific column of your data to be mapped to. Once you have confirmed a mapping, Flatfile will store your preferences in your browser which means it will remember your choices.

Note: These choices are stored in your browser and the mappings are specific to that combination of data type and the exact set of headers found in your file. So if you try using a slightly different file, different data type, or different browser, then you will have to confirm these mappings again.

Note: You do not have to confirm any mappings and can immediately proceed to the Repair page so long as each column of data you are uploading can be matched to one of the columns in the template.

Repair Data

On this page, you will be able to review the data that has been uploaded (or entered manually if applicable). You can freely edit any data in the table if you need to make any last-minute adjustments. Once you have fixed any errors with the data, you can click "Continue" and submit the data.

Datahooks

Flatfile has a feature called called Datahooks which are used by the Dataloader to automatically clean and validate data as it is entered and updated. It has two main types of Datahooks and they are Field Hooks and Record Hooks.

Field HooksThese work on entire columns of data when they are first loaded into Flatfile's user interface. Flatfile will split up all the rows in a column into chunks of 1,000 values and pass these chunks to the Dataloader. The Dataloader can then efficiently check these values for you and will then pass back any validation errors for you to view in the Flatfile user interface. You can then remedy these issues quickly without having to cancel everything, re-open your spreadsheet, fix the values, re-export the file and then re-upload it.

Record Hooks

Unlike Field Hooks, these run on rows of data instead of columns. These are run when you edit something you have already entered into the Flatfile interface and allow the Dataloader to update the validation results for that row. For instance, if you initially have a mistake in your uploaded data and you fix it, the user interface will be updated to show that you have now fixed the issue.

If you have any remaining errors in your data, you will not be able to submit just yet. Try filtering out the valid rows by enabling the "Only show rows with problems" option to more easily find rows with validation errors. You will likely need to horizontally scroll to see these errors especially when you are working with large templates like the one for the Relationships data type.

If you have no validation errors remaining, you will be able to submit your data for processing.

After your data has been submitted to the Dataloader, you will be redirected to the following page. Once your data has been fully inserted into your Accelerate client, you and any admins on your client will receive a receipt email which outlines how many rows were successfully inserted and how many failed.