Troubleshooting Data Destinations

Contents

About

After adding or updating a ProntoForms Data DestinationClosed A Data Destination specifies where to send data from a submitted form. You can use Data Destinations to automate data sharing and storage, routing data to a specific service (such as email or cloud storage) in several different formats., it is important to test it thoroughly before using it in production.

Destination Executions

A Data Destination executes each time that a form linked to the data destination is submitted.  When it executes, the Data Destination then attempts to route data to the configured location. Executions (one per destination per form submission) are listed in the web administration portal.

The web portal offers several ways to view the details of a data destination execution, making it easy to determine if filters, connections, and other options are configured correctly. It also clearly lists details of execution failures.

When incorrect configuration results in a failure, it is simple to update the data destination and re-execute failed executions to try sending the data again.

Problem Contact Email Address

A key tool in testing and troubleshooting Data Destinations is ensuring that your team has a Problem Contact Email Address configured. This is the easiest way to gain visibility to data destination execution failures; an email notification will be sent to the provided address. The problem contact email address can be configured through Team SettingsClosed The Team Settings page is the page where an admin can manage their team's account and edit certain information, such as assigning a Problem Contact Email Address, toggling push notifications, and viewing the account's billing information. (under General Settings).

Error Handling

When a data destination fails, ProntoForms automatically retries the destination at set intervals. Most failures correct themselves and require no intervention; however, ProntoForms will automatically send a failure notification email to the Problem Contact Email AddressClosed A Problem Contact Email Address is an email address an Admin defines in their team settings. If an issue occurs, such as network problems that cause a data destination to be inaccessible, ProntoForms sends an email to the Problem Contact Email Address. An Admin can override the team setting and specify a different email address at the FormSpace level. after the third failure (approximately an hour after the first attempt). This can be configured under the Destination Basics tab of the data destination setup. For more information, see the topic Set Up Failure Notification Emails.

Testing Data Destinations

The only way to test a data destination is to submit a form linked to the data destination. While some data destination configuration issues will become obvious when a form is submitted (like if the execution fails), other configuration errors may not be immediately noticeable. Without thorough testing, these errors can make it into production before being spotted. Save yourself production errors and delays by testing your destinations before deploying to your users.

Connection

A connection that is not functional will cause data destination executions to fail. Submit a form linked to the data destination, check that the execution has not failed, and check that every document/notification was successfully delivered to the configured locations/recipients.

Enabling/Disabling Data Destinations

Disable this destination: Enable this option if you wish to create the destination without allowing it to accept form submissions—it can still be linked in forms, but the destination will not execute if forms are submitted.

Note: Disabled data destinations will still appear in the Destinations tab in the form builder, but it will not receive any form submissions. 

Filtering

Ensure that any filtering that is configured functions correctly, and the destination only executes when you want it to. This is particularly important in the case of Custom Filter rules, which use regular expressions, and a single incorrect character can break the expression. Test filter rules by submitting forms in a test environment, testing all answers referenced in filter rules to determine if the rules behave as expected.

Always Test:

  • Answers that should pass the filter rules
  • Answers that should NOT pass the filter rules

You will want to test other things based on your specific filter rules and the use of your form. It is best if questions referenced in filter rules have pre-defined answers (dropdown, radiobutton, multi-select, Yes/No), as it is impossible to predict all potential typos/errors made by mobile users and to build a regular expression to accommodate them.

Uses of DREL

Most fields that you can enter DRELClosed Data Reference Expression Language (DREL) is used to get form data and metadata and add it to a string, such as dates, usernames, or answers to questions in forms. in will show sample text underneath the field as you are configuring it, so make use of this.  For example, if %u[name] is entered in the field, the sample text should list a "sample" username -- if it doesn't, something has been entered incorrectly.

Data Destinations might use DREL in a number of places:

  • Folder Expressions
  • Filename Expressions
  • Email Subject Lines
  • Pushing data to specific fields
  • And more...

Any question referenced in DREL should be marked as "Required" in the form, particularly if it is the only field in the expression; if a filename expression ends up empty, for example, the transmission of data to most destinations will fail. Test that this cannot happen in your forms.

Data Types

Some data destinations push data from a selected ProntoForms field to a specific field in the destination service.  Salesforce Data destinations require this, for example.   Salesforce has validation set on many of their fields -- an email field will only accept a correctly formatted email, and an integer field will only accept numbers.  If data pushed to Salesforce doesn't match the validation, no record will be created in Salesforce.  For these data destinations, and others that push to services that use validation, it is best to ensure that the ProntoForms data type matches the validation.  Be sure to test the form and make sure that users cannot enter anything into these fields that the destination service will not accept.

Viewing Execution Details

Once forms linked to the Data Destination have been submitted, there are a few ways that Admin users can view execution details.

Data Destination Execution Statistics

  1. Hover over Manage Forms, confirm that you are in the desired FormSpace, and select Data Destinations.

    Mouse over the Manage Forms tab and select Data Destinations.

  1. Select a Data Destination.

    Result: The Data Destination Configuration page opens.

    On the right side of the Data Destination Configuration page is an Execution Statistics table. This table shows the state and number of form submissions (limited to the past 30 days to optimize performance).

    The Data Destination Execution Statistics table lists stated executions for that destination for the past 30 days. Select a State to view a list of executions by this Data Destination with that state.

    Active: The number of Data Destination executions in process.

    Successful: The number of Data Destination executions that were successful.

    Skipped: If a Data Destination has been "skipped," a form submission failed to save properly and the Data Destination was not processed. This can occur for a number of reasons; contact ProntoForms support for troubleshooting help.

    Filtered: The number of Data Destination executions with submitted data that did not meet the filtering rules.

    Failed: The number of Data Destination executions that have failed.

  2. Select an execution state to open the Data Destination Executions page that lists the executions with that state. You can then use Search Options to list executions for a different date range and apply other filters. The topic View Data Destination executions provides more details on the various search options.

Form Submission Processing Details

To see more detailed data on each form submission, navigate to the Data & Analytics tab in the main menu, and select All Form Activity. All submitted forms will be listed; if a data destination execution failed there will be a red "X" icon beside the name. Select the form submission reference number to view details.

Example of a failed form submission on the analytics page.

Form Submission Processing

The Form Submission Processing table outlines in order each action performed by the ProntoForms server as it processes the data. Note that this table does not include any information about processing/handling of the data in the mobile app, before the form was submitted.

The standard processing steps are as follows:

  • Save Data: Saves the data to the server. The subsequent steps will not be completed if "Save Data" fails. Contact our support team if this step fails.
  • Resolve Geo Address: Will be skipped if no geolocation was collected by the device and attached to the form.
  • Process Images: reading of images collected with the form. Will be skipped if no images are attached. If this step fails, the subsequent steps can still process.
  • Data Destination: A new row will be listed for each data destination linked to the form.  If one data destination execution fails, the others can still execute.

Symbols: Processing Status

Icon for a successfully processed step- The step has been successfully processed.

- The step has failed. See details for more information.

Icon for a skipped/filtered step- The step has been skipped. This usually happens because the step is not applicable.

    • The "Process Images" step will be skipped if there are no images in the form.
    • A Data Destination execution will be skipped if the form submission does not meet the destination's filter rules.

Troubleshooting Data Destinations

Skipped

A Data Destination execution will only show as "skipped" if the submitted data does not meet filter rules. Either adjust the filter rules, or change your source data so that it will meet filter rules.  Details will be provided about the string that is being filtered.  In the sample below, the question referenced in the filter rule is not returning a value, causing the email data destination to skip executing, and not send an email.

Example of a skipped/filtered data destination in a form submission.

Failed

Occasionally, a data destination will fail due to a glitch or a momentary outage in either ProntoForms or the external service the destination is linked to.  Sometimes simply re-executing the failed execution (see below) will resolve the issue.

The error causing the problem will be listed in the "details" column of the form submissions processing table. The messages listed can vary widely, but are typically reasonably descriptive and make the cause quite clear. One common problem is security solutions such as firewalls blocking ProntoForms IP addresses.  If you are uncertain of the cause, feel free to contact ProntoForms Support and we will look into the details of the error for you.

Re-Executing a Failed Execution

Automatic Re-Executions

If a data destination execution fails, the ProntoForms server will automatically re-attempt until the form submission either executes successfully or is removed from the ProntoForms system (see Form Submission Retention and Deletion for timelines). When a data destination execution fails, only the form submission that failed to process will be re-executed.  Other form submissions successfully processed by the data destination will not re-execute.

The first and second attempts at re-execution occur after 30 minutes.  The server will continue to retry in an "exponential back-off" process (increasingly longer intervals between attempts). This is done in order to avoid burdening a service with many retry transactions from the ProntoForms system.  Subsequent retries occur after 1 hour, 10 hours,  2.8 days, etc.   The retry process can be stopped by resolving the issue in the destination system, selecting the "Cancel automatic retries" button on the form submission, or by deleting the record in the ProntoForms system.  Just like with the initial execution attempt, each time a re-execution fails, an email notification is sent to the “Problem Contact Email” address for the team.

Automatic Re-Executions and the Data Passthrough Feature

When using the Data Passthrough feature, automatic re-executions are limited. Please see the Data Passthrough overview for more information.

Manually Re-Execute a Failed Execution

A re-execution can be initiated manually through the ProntoForms web portal as well.  This can be useful after correcting configuration issues with a data destination, to test if the changes are effective (or if it is simply inconvenient to wait until the next automatic execution attempt).

Navigate to the desired data destination, and click the “Re-execute Failed Executions” button to re-process all failed form submissions associated with the data destination.

Data Destination main options

You can also re-execute the single failed step in an execution; click on the desired form submission name, then mouse over the desired step.  A “re-execute” icon will appear to the right.

"Re-Execute" for a processing step

Set Up Failure Notification Emails

If a form submission fails to reach the data destination after (by default) three attempts, ProntoForms will send an email to the Problem Contact Email Address. You can adjust this default in the Data Destination Basics tab of the destination configuration.

2019-02-19-WP-DestinationFail02.png