Best practices for deploying Inspections
When you deploy Inspections in large projects without considering performance, your users may experience performance degradation. Learn about the possible problems in advance and set up your questionnaires correctly to mitigate the risks.
Capacity planning
Inspections can generate a significant amount of data. Lots of data on the other hand results in slow sync times and it can even overwhelm your database. What are some of the factors that affect storage:
- number of employees (inspectors) (plan also for workforce growth)
- number of questionnaires per day
- number of questions and instances of repeatable question groups
- number of attachments taken (photos, signatures, etc.)
- static images, such as logo
- rules
- custom styles and format
For example:
- 100 inspectors doing 10 questionnaires per day = ~250k questionnaires per year
- If each questionnaire has 10 kB, you're looking at 2.5 GB yearly.
- Factoring in workforce growth, this could be 15-20 GB in 5 years. This might be a significant portion of the database.
Possible problems
What are some of the problems that you may encounter?
- Server problems
- Slow Questionnaire Designer for extremely large questionnaires
- Large amount of data filling up the database
- Client problems
- Slow synchronization
- High memory consumption
- Sluggish experience
Questionnaire size
We recommend a maximum of 200 questions per questionnaire.
- On the server side, creating a questionnaire with several hundreds of questions may lead to performance degradation in the Questionnaire Designer.
- You can even run into hard limits of the backend server; for example, on Salesforce, the maximum size of a text field is 131,072 characters. We are working on ways how to bypass these limitations.
- On the client side, performance can be also negatively affected, particularly on older mobile devices. Obviously, current high-end devices can handle more than low-end phones from several years ago.
Too many images
If you add too many images to a single questionnaire, the mobile app may become sluggish or unresponsive. Some devices may encounter performance degradation after several dozens of images, other can handle 100+ without problems. This depends on a variety of factors, such as device memory, operating system, and image size.
Mitigation strategies:
- In the Questionnaire Designer, set the proper limits on the Properties pane for Image/Media type of questions, such as the maximum count of images per question, the maximum size of the uploaded file, or maximum image resolution.
- Split the questionnaire in two.
- Deploy modern devices with high RAM.
Rapidly increasing database footprint
If you created your Resco Inspections organization some time ago, the default setting used to be that every single answered questionnaire is saved in a self-contained way, i.e., it includes the answers, but also the questions along with any possible custom images (logo), business rules, and styles. This approach ensures that even if the original questionnaire template changes, your answered questionnaires are unaffected.
In scenarios where questionnaires are changing frequently and each questionnaire is answered only a limited number of times, this is a valid approach. However, if your inspectors perform hundreds or thousands of inspections every day, you might discover that your database gets full of redundant data.
Eliminate redundant data
Use the parameter Dependency Level in the properties of a questionnaire template to control this behavior.
- None: Questionnaires are self-contained, at the cost of an increased database footprint.
- Images: In this case, static images of the questionnaire template (such as the logo) are not included in each answer; instead, they are replaced with a reference to a questionnaire template.
- Full: Save only the answers. Static images, questions, rules, and styles are replaced with a reference to a questionnaire template. This option is the default and recommended option for versioned questionnaire templates.
- Using versioned templates means that old template versions gain the archived state. Sometimes, the sync filter for the Questionnaire entity is configured to exclude archived templates. If you attempt to view old answered questionnaires (that use an archived template with medium or high dependency level) in the app in offline mode, you may encounter the Questionnaire template not found! error. In this case, consider modifying the sync filter to make archived templates available in the offline mode.
Template dependency can save 60-80% of the storage in a typical scenario. Savings can be much more extreme, for example, if you use a high-resolution 2MB logo for a short questionnaire, you can save 99%.
To learn which entities and fields are taken from the questionnaire template, see Template dependency.
| Note | Self-contained questionnaires are being deprecated in favor of full template dependency. See Active deprecations for more information. |
External document storage
Another complementary option is saving your images in external cloud document storage, rather than in the database. This behavior is configured on the Properties pane for images in the parameter Storage Entity. Cloud document storage is often much cheaper than databases. Resco Inspections supports third-party document storage by Box, Dropbox, Google Drive, OneDrive, and SharePoint. See Cloud storage for questionnaire images for more information.
Sync: upload takes too long
Inspectors in the field who perform many large inspections regularly may sometimes experience the problem that the upload phase of synchronization takes too long, particularly if the network isn't too good. There are two main configuration settings that can dramatically improve sync performance:
- Eliminate redundant data using template dependency.
- Greatly reduce the number of transferred records by saving all answers from a questionnaire as a single JSON string.
Increasing the dependency level to full as described above can help, because the amount of data for upload is reduced. Additional savings can be accomplished by changing how questionnaire answers are stored and uploaded.
Edit your questionnaire template, click Options, go to the Storage tab and select how should the answers be stored.
- Records: Each answer is transferred as a question record during synchronization.
- Significantly lower sync performance (7-10 times slower than the other options)
- Average database consumption
- Best compatibility with the backend servers and third-party tools for data visualization/reporting
- Huge number of question and question group records (if a questionnaire contains 20 groups with 10 questions, that's 210 records for a single questionnaire)
- JSON: All answers to a questionnaire are transferred in a single field of the questionnaire entity. This is the default option.
- Best sync performance
- Average database consumption
- Average compatibility with the backend servers and third-party tools for data visualization/reporting
- Question and question group records not used
- Compressed JSON: The JSON string is compressed to save some more space.
- Best sync performance
- Best database consumption
- Worst compatibility with the backend servers and third-party tools for data visualization/reporting
- Question and question group records not used
See JSON storage for technical specifics of this approach.
| Note | In release 15.1, we have introduced new, leaner, and configurable JSON formats. The legacy JSON is being phased out. See Active deprecations for more information. |
JSON support
We recommend using JSON storage because it offers tremendous improvements in sync performance. However, can this data still be used? The answer is yes. JSON is a common data format with diverse uses in electronic data interchange (according to English Wikipedia).
- All Resco tools support this format out of the box
- Resco mobile apps
- Resco web app
- Results Viewer
- Mobile reports
- Dynamics and Salesforce
- Results Viewer is a Resco tool that is available for these backend servers as well. It allows you to browse the answers or export them to CSV or Excel file for further processing.
- You can add and view related questionnaires to a Dynamics entity form
- You can add and view related questionnaires to a Salesforce Lightning Record page
- An example of an unsupported function is Advanced Find in Dynamics: you cannot use this function to search in questionnaire answers.
- Other tools
- As it already has been established, JSON format is a popular format for exchanging data and it is supported by many third-party reporting and business tools.
Sync: download takes too long
If your app users (inspectors in the field) experience long download times followed by many deletions, consider setting the Synchronization strategy for the entity Questionnaire to Incremental with Sync Filter.
Slow questionnaire loading time
| Note | This problem may apply to questionnaires created before release 13.1. |
If your complex questionnaire templates include many On Change rules, app users may experience slow performance when they open the questionnaire in the app. This issue affects particularly older devices.
Edit your questionnaire template, click Options pane, go to the Rules tab, and set the Rules Version parameter to Latest, to take advantage of optimized handling of On Change rules during questionnaire loading. To learn more about this parameter, read Rules version.
Transitioning from record-based format
As we have established, the old record-based format causes a lot of trouble with required database space and slower synchronization. If you are just getting started with Resco Inspections, simply select the recommended format for questionnaires and you're set. But how to proceed if you need to do the switch?
Update the questionnaire templates
Start the Questionnaire Designer. On the list of questionnaire templates, do the following:
- Back up your questionnaire templates. Export the templates and keep them safe.
- Enable versioning: Non-versioned questionnaire templates are being deprecated, and they don't support modern storage options. Select your questionnaire template in the designer and click Versioning. Read about the implications.
Edit a questionnaire template that needs transitioning and click Options.
- (Optional) Increase feature version: On the Features tab, set Feature version to "Latest". This allows you to benefit from new features as we introduce them. Normally, there's no reason to use older feature versions unless you are using old versions of Resco Mobile CRM or Questionnaire Player.
- (Optional) Increase rules version: On the Rules tab, set Rules version to "Latest". Normally, there's no reason to use older rules versions unless you are using a 4+ year-old version of Resco Mobile CRM.
- Set up modern storage: On the Storage tab:
- Set Template dependency to "Full".
- Set Answer storage to "JSON".
- Optionally, for maximum storage savings, consider compressed JSON.
- Click Edit to configure your JSON format. See JSON storage for questionnaires for details.
- Optionally, set Entity for storing answered questionnaires to "resco_questionnaireanswer". This will be the preferred format going forward, but it brings a few additional changes, as discussed in the next section.
- (Optional) Disable automatic report: On the Auto report tab, set Generate automatic report to Off. If you are not using the report, disabling saves template space.
Save all changes and publish the questionnaire.
Review sync filter
If you are using inspections with Woodford and Resco Mobile CRM, review the sync filter for the questionnaire entity. Non-versioned and versioned templates have different statuses (active versus published). Make sure published questionnaires are allowed by your sync filter.
See Sync filter examples for inspiration.
You can now test the questionnaire in the app.
Update data processing
Depending on how you process questionnaire data, you might need to adapt your workflow.
- Resco tools, such as the Results Viewer or Questionnaire Player support JSON format out of the box.
- If you are using third-party tools to crunch questionnaire data, you might need to insert an extra step that deserializes JSON answers.
- Any custom code (e.g., in JavaScript or reports) that refers directly to resco_question or resco_questiongroup (when used for questionnaire answers) needs to be modified.
Deal with existing data
What should you do if you have previously collected significant data in record-based format?
- Keep it as is.
- Move data to Microsoft Fabric or other cheaper long-term storage.
- Set up a reasonable retention policy, periodically deleting old data.
- On demand, we offer a tool for converting answers to the new format. Contact Resco support.
Separate entity for questionnaire answers
In the release 17.1, we have introduced the option to store questionnaire answers in a new dedicated entity, resco_questionnaireanswer.
Benefits
- Save database space
- The new entity supports only the latest JSON storage. It does not include fields only used for questionnaire templates or record-based answers.
- Security and synchronization
- Separate entities for questionnaire templates and questionnaire answers allow you to set up better permission schemes. For example, your mobile users may now access all questionnaire templates but only the answers they created. This may result in simpler sync filter rules and faster synchronization.
- Futureproof
- Some future additions to questionnaires may only support the standalone answer entity.
Caveats
- Questionnaire answers are now a standalone entity. Your existing views for the questionnaire entity won't contain answers stored in the new entity. You must create a view for resco_questionnaireanswer and add it to your home screen.
- Resco tools such as the Results Viewer support both storage entities. However, if you have custom tools for processing/reporting, you will have to add an extra step to retrieve data from the new entity.
Enabling new storage entity in existing projects
If you want to enable the new storage entity in an existing app project:
- Edit your app project in Woodford.
- Enable the new entity.
- Create at least one public view for this entity.
- Save all changes and publish the app project.
Edit your questionnaires in the Questionnaire Designer:
- Click Options and set Feature Version to 4 (or higher).
- On the Storage tab, set Template dependency to "Full" and Answer storage to "JSON". Then, you can set the Entity for storing answered questionnaires to "resco_questionnaireanswer".
- Save all changes and publish the questionnaire.