Shared records

To ensure that each user of a CRM or other computer systems has access to the right data, most data models work with the concept of ownership. Data can be owned directly by users or by teams or organizations. Ownership is often related to data visibility and access, for example, users generally can access the data they own.

Many backend servers have ways to share particular records with other users or organizations. The exact implementation is backend-specific. Sharing is a mean to make accessible records which are normally invisible to the current user.

To ensure that shared records are properly available also in Resco mobile apps, the synchronization process includes a special step called "SyncSharedEntities" that's responsible for keeping shared records up to date.

Dynamics

In Dynamics CRM, users can share records to others. For example, when user A shares the record R to user B:

• R is added to so-called POA (PrincipalObjectAccess) table. (In this context, principal means a user, or a team, etc.)
• R becomes "visible" to user B.
• R itself was not modified through the act of sharing.

This mechanic can have the following impact on Resco Mobile CRM app:

• Full synchronization: No impact. Shared records are downloaded together with other records the mobile user is entitled to see.
• Incremental synchronization: Newly shared records (that were not modified) won't be downloaded. This is a problem.

To keep shared records updated during incremental sync, Sync Shared Records must be enabled in Woodford configuration. The app repeats the following steps for each normal entity (i.e., not an NN entity):

1. The app fetches the IDs of all records shared since the last sync (from the POA table).
2. If such records exist, the app downloads records that match the SyncFilter.

A similar process also runs for records that were unshared since the last sync.

While performing this sync step, the sync progress bar in the app says "Shared".

Notes on POA:

• This table can be important for Technical Support. This table is known to deteriorate server performance. (Possible server timeouts if POA has multi-million entries.)
• Any fetch executed on some table incorporates implicitly also shared records:

Original fetch

 SELECT * FROM email WHERE condition

is replaced by:

WITH emailSecurity as (SELECT * FROM email UNION SELECT shared emails from POA)
SELECT * FROM emailSecurity WHERE condition

Access teams

Access team is a Dynamics collaboration feature for sharing records to teams of users from different business units. Since release 13.3.1, Resco can track a user's membership in access teams and synchronize all shared records even during incremental sync.

Problem

Resco supported sharing for ages. Technically speaking, the app periodically detects the list of teams the mobile user is a member of. When syncing, the app queries the POA table on the server and downloads records that were newly shared to the mobile user or to any of the user's teams. This algorithm is not bad, but does not detect one situation which typically happens namely with access teams (but not only with them):

• Let's have a team T.
• Let's have a mobile user U. The user U is not a member of the team T.
• Let's have a record R. The user U is not entitled to see this record.
• The record R is shared with team T. Technically, a new POA record is created which connects the record R with the team T.
• Next sync ignores this POA entry as it has no relation to the user U. The record won’t be downloaded to the device. So far OK.
• Now the user U is added to the team T. There is no change in the POA table and the next sync won’t download the record again. And that’s the problem.

In other words, Resco properly interpreted new explicit sharing actions (which involved a new POA entry) but it was not able to detect implicit record sharing due to changing the team definition. Users had to perform a full sync to see implicitly shared records. (Or at least to set up entities of interest for ForcedFullSync.)

Since release 13.3.1 Resco supports implicit record sharing. Each sync now detects new user teams, i.e. teams the mobile user became a member of since the last sync. If there are such teams, the sync downloads all records shared to these users in the past.

Solution

• Synchronization of records shared via access teams has to be enabled in Woodford configuration: Sync Shared for Access Teams.
• The first sync after enabling this feature does not download records shared to access teams.
• Later incremental synchronizations download records shared to all teams that a mobile user acquired since the last sync.

Sync log

This synchronization step saves information to the sync log.

<SharedEntities Recv=1234 [Del=123 Conflicts=12] TotalTim=12345ms />
<FullSync Recv='14719' TotalTime='15828ms' Entitys='2312ms' ManyToMany='578ms' Shared='469ms' MarketingLists='11953ms' />

The information about access teams is available when you enable sync details:

*** SyncSharedEntities: FullSync for 2 new team(s) ***

Salesforce

Sharing is completely transparent to Salesforce fetches. E.g., you can ask for all contact records and you'll get:

• All records to which you are entitled based on your current permissions.
• All records which are shared with you at the moment the fetch was issued.

As a result:

• FullSync returns all suitable records, including all shared records.
• IncSync has problems:
  • It downloads changed records. Newly shared records do not need to be changed. Incremental sync will not download them.
  • Another problem is record unsharing: These records just silently disappear from the view; standard incremental sync has no chance to find out that a record is no longer shared.

Sharing in Salesforce

First of all, not all objects support sharing:

• Object with OWD (Organization Wide Defaults), i.e., globally accessible objects, do not need sharing at all as "every user sees everything".
• Objects on the detail side of Master-Detail relationship do not support sharing as well; access to these records is defined by the access to their master record. (Note: MCRM v13.3 does not check this.)

Records are shared with a specific user or to a public group. Groups are hierarchical - a group can contain both users and other groups. Moreover, users can share records not only to a group but also to its subordinates.

From another point of view, sharing can be manual (by a user action), automatic (based on the rules), or even added programmatically via Apex or SOAP.

Rather than analyzing record access in real-time, Salesforce precalculates access data when configuration changes occur. (Manually invoked via Recalculate button in Sharing Settings.) To store the results of these calculations, Salesforce maintains a separate share table for every object that supports sharing. For example:

• AccountShare for Account (standard) object,
• Brunch_Office__Share for sf_brunch_office__c (custom) object.

Note: This contrasts to Dynamics CRM solution, which maintains a single share table (POA) for all entities.

Resco implementation

Woodford admin can enable objects (entities) for which the sharing should be checked. Select the object from the Project menu and check Sync shared records. This function is available since release 13.3.0.1.

Resco mobile apps detect objects that support sharing in the permission check phase of synchronization. (Permission check is done during the first sync in an app session, and also when the sync updates the customization.)

FullSync

• Normal sync downloads all visible records, included those which are shared at the moment.
• SyncShared module just downloads a list of IDs of shared records and stores them in the local share table. (rs_sf_account_share, rs_sf_brunch_office__c_Share...)
• The app remembers the time when the entity shared records were synced.

IncSync

• The app first downloads record IDs that were shared on the server since the last sync.
• These IDs are added to the local share table and corresponding entity records are downloaded from the server.

Sharing cleanup (SyncUnshared)

• The app explicitly checks the visibility of all known shared records and deletes those not visible from the client device.

(Known shared records = records whose IDs are stored in local share table. Record visibility is checked via an attempt to download record ID. This is done in batches.)

• Eventual conflicts (Server unshared a record, which was edited on the client at the same time) are detected.

The sharing cleanup is carried out always, i.e., also for FullSync. In an ideal world, this would not be needed. However, there can be a considerable time between the entity normal sync and shared sync. (For example, when sync was aborted and the user does not start it again until after several days.) Therefore, the app executes the cleanup always.

Prerequisites

The following entities/properties must be enabled in the app project:

• sf_user.userroleid
• sf_group.relatedid
• sf_groupmember.userorgroupid

Sync Shared Records must be enabled on two locations:

• In project Configuration: Offline Data Sync > Sync Data > Sync Shared Records.
• In entity properties: Sync shared records.

Behavior of the mobile app

During the synchronization, the sync progress bar displays the message "Syncing Shares: ...entity_name...".

Error treatment: Abort breaks the sync. Other errors are logged (prompt "SyncSharedEntities"), sync goes on. An (artificial) example of such a log follows:

<EXCEPTION>16:21:58.135: SyncSharedEntities
NullReferenceException: Object reference not set to an instance of an object.
  at Salesforce.SFSharing.SharedSync(SalesforceService sfService)
  at SyncEngine.SFSharedSync(db)
</EXCEPTION>

Sync log

Sharing summary:

<SyncShared Recv='1234' ApiCalls='15' Conflicts='1' Attachments='3' CheckUnshared='2345ms/12calls' TotalTime='3333ms' />

Errors/warnings:

Sharing initialization failed: No share objects accessible to this user
Sharing initialization failed: Disabled sf_user     // sf_user.userroleid, sf_group, sf_group.relatedid, sf_groupmember, sf_groupmember.userorgroupid
WARN: More than 10000 shared records for entity sf_contact; the rest is ignored

List of implemented detail logs:

GetListOfShareIds(sf_account) -> 314 recs, 234ms                            // FullSync
GetListOfShareIds(sf_account) since 2020-11-13 10:23:36 -> 314 recs, 234ms  // IncSync
12 shared sf_account recs ignored as they are already on the client         // IncSync
Download sf_account shares -> 302 recs / 1234ms[; Saved in 12ms]            // IncSync
<CheckUnshared Entity='sf_account' Deleted='0' Conflicts='1'  ApiCalls='3' DB='233ms' TotalTime='2345ms' />  // Cleanup

Sync summary: As of release 13.3, synchronization of shared records for Salesforce does not put any sharing info into the <Summary> section.

Conflicts

In the context of synchronization of shared records, a conflict means that the server unshared the record and the client edited the same record. In technical terms: Device:Update|Server:AccessRevoked.

A different case, Device:Delete|Server:AccessRevoked is not considered as a conflict: The client record is silently deleted, nothing is logged.

Conflict resolution is defined in entity attributes (Woodford):

• Server Wins (default conflict resolution): the conflict is automatically resolved and logged this way: Device:Update|Server:AccessRevoked -> ServerEntityWins (LocalRecordDeleted)
• In other cases (Device Wins or Manual Action) the conflict must be resolved manually. The sync errors form shows the conflict and sync log contains: Device:Update|Server:AccessRevoked -> CustomHandling

Examples of logged conflicts:

• Conflict: sf_contact ID[1fb67823-8000-f000-0000-882702725352] Name[John Smith]: Device:Update|Server:AccessRevoked -> ServerEntityWins(LocalRecordDeleted)
• Conflict: sf_contact ID[1fb67823-8000-f000-0000-882702725352] Name[John Smith]: Device:Update|Server:AccessRevoked -> CustomHandling

Limits, potential problems

• Master-detail relations. (E.g., Order - OrderItem.): The detail objects do not support sharing. In fact, detail records should be shared automatically with their master record. However, this is not implemented in Resco mobile apps.
• Max 10.000 shared records/entity are supported. This is a performance measure. When checking for unshared records, the app validates the existence of past known shares. However, due to Salesforce limitation on SOQL command length, it is possible to check at most 180 IDs in one web request. For this reason, Resco adopted this limitation.