Synchronization
| Synchronization |
|---|
Data synchronization is the process of establishing consistency among data from a source to a target data storage and vice versa and the continuous harmonization of the data over time.
In our case, consistency has to be ensured of the data stored on the CRM server and the data stored on a end user device using Resco Mobile CRM app.
Introduction
When you synchronize your app for the first time, you basically download to the device a subset of the server database. This subset refers to the time instant when it was downloaded. (Or synced last time – hence the name LastSync time. Strictly speaking there is no LastSync time for the database as a whole. Instead, each table has its own LastSync time.)
Once you have the data on the mobile device, you can start using it – view, search, but also create, edit or delete. Of course, the same data items may be changed on the server, too.
The purpose of the synchronization is to
- Download server changes to the client
- Upload client changes to the server
- Resolve eventual conflicts
CRM servers have large databases with a lot of data. Users of the mobile app usually need only a small subset of entities, and they don't need the full set of fields for each record.
Types of synchronization
When talking about data synchronization, we sometimes need to differentiate between its various types.
Full vs incremental
- Full synchronization - Download of all server records to the client. This is used when you launch the mobile app for the first time.
- Incremental synchronization - Client uploads its changes to the server and downloads server changes.
In most cases, full sync is much more resource-intensive than incremental sync. Therefore, full sync should be used only when necessary:
- Full sync is initiated automatically when you change either the server URL or the user name.
- App user can initiate full sync by deleting data (Setup form > Delete Data)
- Woodford admin can force full sync using the Device control tool
- Woodford admin can decide that each change in the app project results in a full sync. (Woodford Configuration parameter Full Sync on Schema Change; in general, we do not recommend enabling this option.)
Partial full sync
Sometimes, the sync engine can decide to full sync only selected entities. This typically happens when a customization update changes the entity definition or when performance reasons dictate that it is better to execute entity full sync instead of inc sync.
During customization update, these are the possible reasons for partial full sync:
- for new entities
- for entities with new fields (If fields are disabled for an entity, full sync is not needed. The disabled column is simply removed from the local database.)
- when the entity Sync Filter has changed.
These are other possible reasons for partial full sync:
- The sync engine uses a module called SyncAnalyzer to answer the question of whether for a given entity it is better to perform inc sync or full sync.
- You can also force full sync for a particular entity using Woodford: edit an app project, select an entity from the Project menu, and set Synchronization to Always Full Sync. (When to use Always Full Sync?)
When there are changes in the user’s security role on the server (for example, the user loses access to an entity), Resco Mobile CRM has no information about this change. Hence, it is the responsibility of the Woodford administrator to enforce full sync in these cases.
Foreground vs background
- In foreground mode, the app displays progress dialog; users must wait until the synchronization finishes
- In background mode, users can use the application freely during synchronization. When both the user and the synchronization process attempt to write to the database at the same time, users will be notified and they have to wait.
When a new customization is available (new app project published from Woodford), the synchronization must run in the foreground.
Manual vs automatic
- Manual synchronization is initiated by the app user by tapping the Sync button, or it can be launched via JavaScript.
- Automatic synchronization can be periodic or triggered by app start or data change.
First synchronization
The very first synchronization is a full synchronization with some additional steps. Client downloads the following:
- Customization
- Data model (empty database tables are created on the client)
- Permissions
- User interface screens
- Business logic
- Entity records as specified by the customization (they are saved to the database)
The result is a kind of equilibrium:
- Client has customization that was valid at the time of the last sync.
- Client has a projection of server data (as defined by the customization) that was valid at the time of the last sync.
Subsequent synchronizations
This equilibrium changes after some time:
- Client changes: User created or edited some data records; some new records were created automatically (for example audit)
- Server changes: Someone modified records on the server.
- Some records might have been changed by both client & server (= conflicts)
- Woodford might have changed the customization.
The next synchronization (incremental) has the task of restoring the equilibrium:
- Upload client changes
- Identify conflicts (based on record modifiedon attribute)
- Resolve conflicts if possible. See Conflict resolution for details.
- Unresolved conflicts are shown in SyncErrors form; respective records are excluded from upload.
- Client changes (except unresolved conflicts) are uploaded to the Server. Records that fail to upload are also shown in SyncErrors form. (Together with the error message.)
- Check for the new customization and try to apply it.
- New customization may change the data model. This is a non-trivial operation and it cannot be performed during a background synchronization (dangerous for user interface), or if there are not-uploaded changes - conflicts or errors (new customization could delete the changed columns).
- In these cases the new customization is ignored and the user will be notified about that. (Next time foreground sync will be forced.)
- If a new customization is available and can be applied => It will be applied.
- Finally, download server changes
Result: A new equilibrium between the client and the server with these exceptions:
- Sync Errors (conflicts, upload failures)
- Possibly waiting customization that could not be applied. (background sync or unuploaded changes)
See Synchronization steps for a detailed technical description of the synchronization process.
Servers involved in synchronization
Resco license server: iservices.resco.net
- license
- external users resolution
- ApplicationVersionCheckService (responsible for update available notifications)
CRM data/customization:
- Power Platform / Dynamics CRM
- CRM4 (deprecated)
- CRM5: Dynamics 2011/2015/365; Online / On premise
- Resco Cloud
- Salesforce
- SharePoint
- DropBox, Box, GoogleDrive, OneDrive
- Azure cloud storage for attachments
Mail servers
See also
- Resco Mobile CRM & data synchronization Blog
- 10 tools to improve your synchronization speed and management Blog