Best Practices for Data Source Size
Data Sources
Data sources, also known as "Lookups", are external sources of data that you upload or connect to TrueContext. You can reference this data in a form to populate answers or answer options. Data sources save typing, reduce errors, and make it easy to provide mobile users with only the relevant, most current data. make external data available to the TrueContext Mobile App. We store Data Sources as tables (rows and columns) in the TrueContext system. This topic describes best practices for organizing your data and explains why we limit the size of Data Sources by default.
Tip:If you’re an Enterprise customer, we recommend that you follow the recommendations for Data Source size before you request an increase to the maximum number of cells and rows. The process that you follow to request an increase ensures that we have an opportunity to help you optimize mobile device performance.
Available on all tiers, with more options on the Advanced and Enterprise tiers:
Contents
Info:The topic Data Sources Overview describes the use of Data Sources in more detail.
Recommendations for Data Source size
We recommend that you keep your Data Sources as small as possible for the following reasons.
- Enhanced user experience
-
-
The larger the Data Source
Data sources, also known as "Lookups", are external sources of data that you upload or connect to TrueContext. You can reference this data in a form to populate answers or answer options. Data sources save typing, reduce errors, and make it easy to provide mobile users with only the relevant, most current data., the longer it takes to download data to the mobile device, especially on older hardware and platforms. TrueContext makes downloads as fast as possible, and smaller Data Sources contribute to an experience of quick access to forms and data. -
The same amount of data downloads faster when split into multiple Data Sources—use multiple, smaller Data Sources rather than one large one. Because we use multiple connections, more than one Data Source can download at the same time.
-
Processing times increase when the TrueContext Mobile App has to search through a large data set to display the data for a specific question in a form.
-
- Tip:A frustrated user might complete an outdated version of a form using data that’s not current. Use smaller Data Sources to give your users quick access to your forms.
- Data integrity
-
-
Your organization must regularly maintain and update the data that’s used in a TrueContext Data Source. Organize your data logically so that it’s easier for you to maintain. For example, it’s easier to maintain 1,000 rows of data than it is to maintain 50,000 rows.
-
Smaller Data Sources make it easier to avoid duplication and to maintain data integrity.
-
For high volumes of data, follow best practices for database normalization. This reduces the amount of data to transfer.
-
Make sure your data model is designed to maximize efficiency.
-
Tip:You can also partition a Data Source (Enterprise tier only) to optimize mobile device performance, but this doesn’t reduce the actual Data Source size.
Why we limit the size of your Data Sources
Data Source size limits are a way to help you optimize mobile device performance. The more data that you load on a device, the longer it takes for the device to reconcile.
The size limits apply when you upload, fetch, or update a Data Source. The system enforces a column and character limit for all tiers. Other size limits vary by tier, as shown in the following table.
| Tier | Maximum number of cells | Maximum number of rows | Maximum number of columns | Maximum number of characters in a cell | Maximum file size |
|---|---|---|---|---|---|
| Essentials | 100,000 | 10,000 | 500 | 32,767 | 500 MB |
| Advanced | 500,000 | 50,000 | |||
| Enterprise | 2,000,000 (default) | 200,000 (default) |
How Data Source size limits work
File size
The system limits the size of manually uploaded and dynamically fetched Data Source files to 500 MB. If your file exceeds the limit, the system blocks you from uploading or using the file.
Rows and cells
The TrueContext system limits the size of each Data Source based on the number of rows and cells (rows multiplied by columns). The system
-
calculates the number of rows and cells
-
compares these numbers to the limits for your tier, and
-
determines whether the Data Source exceeds the size limits.
If there are too many rows or cells, the system uploads the data but cuts off (truncates) any rows and cells that exceed the limit. In this case, the system displays an on-screen message and emails the problem contact email address.
Columns and characters
If a Data Source has more than 500 columns or 32,767 characters in one cell, regardless of tier, the system displays an error message and does not upload any data.
How to resolve a Data Source size limit issue
Remember that large Data Sources can cause a device to take longer to reconcile. This might result in a poor mobile user experience.
To resolve the issue of truncated data, try one of the following options.
-
Reduce the Data Source size by removing columns or rows.
-
Use multiple Data Sources instead of one. For example, you can separate customer data from equipment data.
-
Upgrade to a higher tier (contact your Account Manager or your Customer Success Manager for details).
Size limit increases for Enterprise customers
Tip:We recommend that you follow the recommendations for Data Source size before you request a size limit increase.
If you’re an Enterprise customer, you can contact Support to discuss a size limit increase. Please provide Support with the following information:
-
Are all columns in the data source required? Could you remove some to reduce the size?
-
Will partitions be configured? If yes, how many rows and cells will the largest partition contain?
-
What is the business reason for the cell increase?
-
How often will you need to update the Data Source?
We’ll pass this information along to our development team to evaluate the request and implement the change.