Difference between revisions of "Synchronization"

From Resco's Wiki
Jump to: navigation, search
(See also)
(Full vs incremental)
 
(19 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
{{Wikipedia|Data synchronization}}
 
{{Wikipedia|Data synchronization}}
 +
{{Synchronization TOC}}
 
Data synchronization is the process of establishing consistency among data from a source to a target data storage and vice versa and the continuous harmonization of the data over time.  
 
Data synchronization is the process of establishing consistency among data from a source to a target data storage and vice versa and the continuous harmonization of the data over time.  
  
In our case, consistency has to be ensured of the data stored on the [[CRM]] server and the data stored on a end user device using Resco Mobile CRM app.
+
In our case, consistency has to be ensured of the data stored on the [[CRM]] server and the data stored on a end user device using [[Resco Mobile CRM]] app.
 +
 
 +
== Introduction ==
  
 
When you synchronize your app for the first time, you basically download to the device a subset of the server database. This subset refers to the time instant when it was downloaded. (Or synced last time – hence the name LastSync time. Strictly speaking there is no LastSync time for the database as a whole. Instead, each table has its own LastSync time.)
 
When you synchronize your app for the first time, you basically download to the device a subset of the server database. This subset refers to the time instant when it was downloaded. (Or synced last time – hence the name LastSync time. Strictly speaking there is no LastSync time for the database as a whole. Instead, each table has its own LastSync time.)
Line 12: Line 15:
 
* Upload client changes to the server
 
* Upload client changes to the server
 
* Resolve eventual conflicts
 
* Resolve eventual conflicts
 
 
== What data is stored in the app ==
 
  
 
CRM servers have large databases with a lot of data. Users of the mobile app usually need only a small subset of entities, and they don't need the full set of fields for each record.
 
CRM servers have large databases with a lot of data. Users of the mobile app usually need only a small subset of entities, and they don't need the full set of fields for each record.
Line 28: Line 28:
 
Full synchronization can be forced
 
Full synchronization can be forced
 
* for a new customization (using Woodford)
 
* for a new customization (using Woodford)
* for selected entities (using Woodford)
+
* for selected entities (using Woodford: edit an [[App_projects#Managing_entities|app project]], select an entity from the '''Project''' menu, set '''Synchronization''' to '''Always Full Sync''')
* by the client (Setup form > Delete Data)
+
* by the client ([[Setup#Delete_Data|Setup form > Delete Data]])
 +
 
 +
Full synchronization of an entity is needed when you [[App_projects#Managing_entities|enable an entity]] in Woodford, or [[App_projects#Managing_fields|enable a field]] in an already enabled entity. In these cases, full sync is performed automatically, with no additional user or admin action needed.
 +
 
 +
Full synchronization is also necessary when the [[security model]] is changed, for example when you switch the security role of a user, or modify permissions in a security role, or create a new business unit. Such changes require that the full synchronization is triggered manually by user or forced by admin.
  
 
=== Foreground vs background ===
 
=== Foreground vs background ===
 
* In foreground mode, the app displays progress dialog; users must wait until the synchronization finishes
 
* In foreground mode, the app displays progress dialog; users must wait until the synchronization finishes
 
* In background mode, users can use the application freely during synchronization. When both the user and the synchronization process attempt to write to the database at the same time, users will be notified and they have to wait.
 
* In background mode, users can use the application freely during synchronization. When both the user and the synchronization process attempt to write to the database at the same time, users will be notified and they have to wait.
 +
 +
When a new customization is available (new app project published from Woodford), the synchronization must run in foreground.
  
 
(Settings > Sync Login on/off)
 
(Settings > Sync Login on/off)
Line 39: Line 45:
 
=== Manual vs automatic ===
 
=== Manual vs automatic ===
  
'''Manual''' synchronization is initiated by the app user by tapping the Sync button, or it can be launched via JavaScript.
+
* '''Manual synchronization''' is initiated by the app user by tapping the Sync button, or it can be launched via JavaScript.
 +
* '''[[Automatic synchronization]]''' can be periodic or triggered by app start or data change.
 +
 
 +
=== First synchronization ===
 +
 
 +
The very first synchronization is a full synchronization with some additional steps. Client downloads the following:
 +
# Customization
 +
#* [[Data model]] (empty database tables are created on the client)
 +
#* Permissions
 +
#* User interface screens
 +
#* [[Business logic]]
 +
# Entity records as specified by the customization (they are saved to the database)
 +
 
 +
The result is a kind of equilibrium:
 +
* Client has customization that was valid at the time of the last sync.
 +
* Client has a projection of server data (as defined by the customization) that was valid at the time of the last sync.
  
'''Automatic''' synchronization can be periodic or triggered by app start or data change.
+
=== Subsequent synchronizations ===
  
== Automatic synchronization ==
+
This equilibrium changes after some time:
Automatic synchronization is controlled by application settings. It can be triggered in several ways:
+
* Client changes: User created or edited some data records; some new records were created automatically (for example audit)
 +
* Server changes: Someone modified records on the server.
 +
* Some records might have been changed by both client & server (= conflicts)
 +
* Woodford might have changed the customization.
  
* If AppSettings.AutoSyncAfterStart >= 0 // #seconds to wait before syncing after application start
+
The next synchronization (incremental) has the task of restoring the equilibrium:
* If AppSettings.AutoSyncDelay >= 0 // #seconds since the last sync
 
* If AppSettings.AutoSyncAfterChange >= 0 // #seconds to wait before syncing after data change
 
*: Triggered by record change: Created/Deleted/Changed (when Save button pressed).
 
*: Even NN-record creation/deletion (association/disassociation) triggers sync.
 
  
* If AppSettings.SyncEmailOnSend==true:
+
# Upload client changes
*: Email change triggers email sync (unless sync is already triggered) // Email-only sync = syncing only emails
+
#* Identify conflicts (based on record modifiedon attribute)
 +
#* Resolve conflicts if possible. See [[Conflict resolution]] for details.
 +
#* Unresolved conflicts are shown in SyncErrors form; respective records are excluded from upload.
 +
#* Client changes (except unresolved conflicts) are uploaded to the Server. Records that fail to upload are also shown in SyncErrors form. (Together with the error message.)
 +
# Check for the new customization and try to apply it.
 +
#* New customization may change the data model. This is a non-trivial operation and it cannot be performed during a background synchronization (dangerous for user interface), or if there are not-uploaded changes - conflicts or errors (new customization could delete the changed columns).
 +
#* In these cases the new customization is ignored and the user will be notified about that. (Next time foreground sync will be forced.)
 +
#* If a new customization is available and can be applied => It will be applied.
 +
# Finally, download server changes
  
* When changing online <-> offline mode
+
Result: A new equilibrium between the client and the server with these exceptions:
*: AppSettings.GoingOnlineMode // NoAction | AskUserToSync | WarnUserMustSync | ForceSync
+
* Sync Errors (conflicts, upload failures)
*: AppSettings.GoingOfflineMode // NoAction | AskUserToSync | WarnUserMustSync | ForceSync
+
* Possibly waiting customization that could not be applied. (background sync or unuploaded changes)
  
Conditions for auto-sync:
+
See [[Synchronization steps]] for a detailed technical description of the synchronization process.
* Settings > Sync Login off
 
* Password saved
 
* No sync errors from previous sync
 
  
 
== Servers involved in synchronization ==
 
== Servers involved in synchronization ==
  
Resco license server: iservices.resco.net
+
[[Resco licensing service|Resco license server]]: iservices.resco.net
 
* license
 
* license
* external user resolution
+
* [[external users]] resolution
* ApplicationVersionCheckService // "Update Available"
+
* ApplicationVersionCheckService (responsible for update available notifications)
  
 
CRM data/customization:
 
CRM data/customization:
 
* Dynamics CRM
 
* Dynamics CRM
** CRM4: ~deprecated
+
** CRM4 (deprecated)
 
** CRM5: Dynamics 2011/2015/365; Online / On premise
 
** CRM5: Dynamics 2011/2015/365; Online / On premise
* Resco XRM
+
* [[Resco CRM server]]
 
* Salesforce
 
* Salesforce
 
* (Oracle)
 
* (Oracle)
  
Documents
+
[[Documents]]
 
* SharePoint
 
* SharePoint
 
* DropBox, Box, GoogleDrive, OneDrive
 
* DropBox, Box, GoogleDrive, OneDrive
* (Azure - attachments)
+
* Azure cloud storage for attachments
  
 
Emails
 
Emails
* Exchange
+
* [[Exchange]]
* Gmail
+
* [[Gmail]]
 
 
 
 
  
 
== See also ==
 
== See also ==
* [[Synchronization steps]] - in-depth technical description of the synchronization process
+
* [https://blog.resco.net/2018/08/09/quick-overview-resco-mobile-crm-data-synchronization/ Resco Mobile CRM & data synchronization] {{Badge|Blog}}
* [[Synchronization troubleshooting]] - evaluate performance, recognize common issues
 
* [[Sync Filter]] - define what data from the server is replicated on the mobile app
 
* [[Speed up synchronization]] - optimize synchronization of entity records
 
* [[Document filters]] - optimize synchronization of documents
 
 
 
<!--
 
Use materials in
 
* \\ds01\Staff\_Resco University\CRM Basics, Data Model\Sync\SyncOverview.txt (SharePoint details should be separated into separate chapter)
 
* For inspiration: https://blog.resco.net/2014/11/14/resco-mobile-crm-synchronization/
 
Then add special chapters
 
* SyncDownloaderSetup.txt
 
* LinkedSyncFilter.txt
 
* UploadSample_syncLog.txt
 
* Salesforce specifics - \Salesforce\Resco-Salesforce Synchronization 2019_01-24.pptx
 
  
-->
 
 
[[Category:Core concepts]] [[Category:Synchronization]]
 
[[Category:Core concepts]] [[Category:Synchronization]]

Latest revision as of 08:53, 20 February 2020

Wikipedia logo
Wikipedia has an article on the same subject:
Synchronization

Data synchronization is the process of establishing consistency among data from a source to a target data storage and vice versa and the continuous harmonization of the data over time.

In our case, consistency has to be ensured of the data stored on the CRM server and the data stored on a end user device using Resco Mobile CRM app.

Introduction

When you synchronize your app for the first time, you basically download to the device a subset of the server database. This subset refers to the time instant when it was downloaded. (Or synced last time – hence the name LastSync time. Strictly speaking there is no LastSync time for the database as a whole. Instead, each table has its own LastSync time.)

Once you have the data on the mobile device, you can start using it – view, search, but also create, edit or delete. Of course, the same data items may be changed on the server, too.

The purpose of the synchronization is to

  • Download server changes to the client
  • Upload client changes to the server
  • Resolve eventual conflicts

CRM servers have large databases with a lot of data. Users of the mobile app usually need only a small subset of entities, and they don't need the full set of fields for each record.

Types of synchronization

When talking about data synchronization, we sometimes need to differentiate between its various types.

Full vs incremental

  • Full synchronization - Creating a copy of all server records on the client. This is used when you launch the mobile app for the first time.
  • Incremental synchronization - Server sends its changes to the client; client sends its changes to the server.

Full synchronization can be forced

  • for a new customization (using Woodford)
  • for selected entities (using Woodford: edit an app project, select an entity from the Project menu, set Synchronization to Always Full Sync)
  • by the client (Setup form > Delete Data)

Full synchronization of an entity is needed when you enable an entity in Woodford, or enable a field in an already enabled entity. In these cases, full sync is performed automatically, with no additional user or admin action needed.

Full synchronization is also necessary when the security model is changed, for example when you switch the security role of a user, or modify permissions in a security role, or create a new business unit. Such changes require that the full synchronization is triggered manually by user or forced by admin.

Foreground vs background

  • In foreground mode, the app displays progress dialog; users must wait until the synchronization finishes
  • In background mode, users can use the application freely during synchronization. When both the user and the synchronization process attempt to write to the database at the same time, users will be notified and they have to wait.

When a new customization is available (new app project published from Woodford), the synchronization must run in foreground.

(Settings > Sync Login on/off)

Manual vs automatic

  • Manual synchronization is initiated by the app user by tapping the Sync button, or it can be launched via JavaScript.
  • Automatic synchronization can be periodic or triggered by app start or data change.

First synchronization

The very first synchronization is a full synchronization with some additional steps. Client downloads the following:

  1. Customization
  2. Entity records as specified by the customization (they are saved to the database)

The result is a kind of equilibrium:

  • Client has customization that was valid at the time of the last sync.
  • Client has a projection of server data (as defined by the customization) that was valid at the time of the last sync.

Subsequent synchronizations

This equilibrium changes after some time:

  • Client changes: User created or edited some data records; some new records were created automatically (for example audit)
  • Server changes: Someone modified records on the server.
  • Some records might have been changed by both client & server (= conflicts)
  • Woodford might have changed the customization.

The next synchronization (incremental) has the task of restoring the equilibrium:

  1. Upload client changes
    • Identify conflicts (based on record modifiedon attribute)
    • Resolve conflicts if possible. See Conflict resolution for details.
    • Unresolved conflicts are shown in SyncErrors form; respective records are excluded from upload.
    • Client changes (except unresolved conflicts) are uploaded to the Server. Records that fail to upload are also shown in SyncErrors form. (Together with the error message.)
  2. Check for the new customization and try to apply it.
    • New customization may change the data model. This is a non-trivial operation and it cannot be performed during a background synchronization (dangerous for user interface), or if there are not-uploaded changes - conflicts or errors (new customization could delete the changed columns).
    • In these cases the new customization is ignored and the user will be notified about that. (Next time foreground sync will be forced.)
    • If a new customization is available and can be applied => It will be applied.
  3. Finally, download server changes

Result: A new equilibrium between the client and the server with these exceptions:

  • Sync Errors (conflicts, upload failures)
  • Possibly waiting customization that could not be applied. (background sync or unuploaded changes)

See Synchronization steps for a detailed technical description of the synchronization process.

Servers involved in synchronization

Resco license server: iservices.resco.net

  • license
  • external users resolution
  • ApplicationVersionCheckService (responsible for update available notifications)

CRM data/customization:

  • Dynamics CRM
    • CRM4 (deprecated)
    • CRM5: Dynamics 2011/2015/365; Online / On premise
  • Resco CRM server
  • Salesforce
  • (Oracle)

Documents

  • SharePoint
  • DropBox, Box, GoogleDrive, OneDrive
  • Azure cloud storage for attachments

Emails

See also