TDS

Warning

This page describes a function that has not yet been publicly released, or has been released in beta / preview quality. Subject to change.

According to Microsoft documentation, the Tabular Data Stream (TDS) protocol is an application-level protocol used to transfer requests and responses between clients and database server systems.

Resco Mobile CRM can use the TDS protocol, which allows for faster synchronization (compared to the SOAP endpoint).

About TDS

• The protocol is 4 decades old.
• The TDS endpoint for Dataverse has been in preview since May 2020.
• There may be breaking changes in the future.

About TDS-based sync

• Introduced in release 17.1.
• It can be used with Microsoft Dataverse (Power Platform, Dynamics 365) but not for on-premises installations.
• Speeds up the record download phase of synchronization (on average, 3-10 times faster). Typically, downloading records is the longest phase of sync.
• Applies to both full sync and incremental sync.
• Does not support FetchXML options (if you have such options in the Sync Filter, sync falls back to classic endpoint)

Warning

TDS is an experimental feature. You can use it in production, but monitor the performance carefully. If TDS sync fails for a particular user, they can always fall back to the previous sync method - disable TDS in the app's Setup > Use TDS.

Enabling TDS endpoint for your Power Platform environment

• Go to the Power Platform admin center, https://admin.powerplatform.microsoft.com/.
• Select the environment where you want to enable TDS.
• Go to Settings > Products > Features.
• Select Enable TDS endpoint.
• Optionally, you can also enable user-level access control for TDS endpoint. This requires assigning users additional privileges when enabled.
• Click Save.

To manage user-level privileges:

• Go to Settings > Products > Features.
• Select the security role for users who need the TDS endpoint.
• On the Miscellaneous privileges tab, enable Allow user to access TDS endpoint and set the Privilege Level to "Organization".
  
• Save all changes.
• Finally, assign users who need TDS endpoint access to the security role.

See Microsoft documentation for details.

Enabling TDS on Resco side

You have several options how to enable TDS:

• On project level: Edit an app project in Woodford, go to Configuration > Offline Data Sync > Sync Advanced, and enable Use TDS download method.
• On table level: Woodford admins can also set up the behavior for each table in Woodford.
• On user level: App users can enable this feature in the app's setup.

Why is TDS so fast

• Whether you use TDS or FetchXML, SQL server's work is the same.
• TDS is a binary protocol written at times when every byte mattered.
• TDS uses TCP communication, which is faster than HTTP.
• Fetch relies on SOAP protocol, which in turn uses HTTP protocol, which in turn uses TCP.
• Fetch response formatting is complex and redundant and requires compression.

TDS in sync log

A new section in the sync log informs about TDS.

<Downloader CacheSize=1500MB UsedCache=1510MB PausedFor=70%/>
16:59:47 resco_question: 15998 recs, 12281ms, 767ms/1402K/page | Increase page (now 1000)
16:59:47 team: 124182 recs, 9937ms, 236ms/1010K/page
16:59:47 yy_country: 247 recs, 2859ms
...
<TdsReport>
(1) resco_question: TDS does not support fetch options
yy_country: 247 recs, 1160ms + MaxVer:561ms
(2) team: 124182 recs, 7767ms + MaxVer:405ms, pageUsed=57000, 0.066 ms/rec
...
(3) AVG(MaxVer) = 163ms
</TdsReport>
<Summary>
<Settings App='17.1.0.4' Threads='3' …/>
<Results Sent='0' Recv='18332902' Result='Aborted'/>
<Times Total='1866031ms' Prepare='15ms'/>
<FullSync Recv='18332902' TotalTime='1865875ms' Entitys='1865859ms'>
<SyncDownloader CacheSize='1500MB' UsedCache='1510MB' PausedFor='70%'/>
</FullSync>
<Analysis>
<DownloadPerformance>0.1 ms/rec (exceptional)</DownloadPerformance>
(4) <Tip><Tip002>SyncDownloader paused for 70%; consider cache size increase.</Tip002></Tip>
</Analysis>
(5) <ApiCalls Downloader='16' Total='16' />
</Summary>

Notes:

1. The SyncFilter for the resco_question table contained the DisableRowGoal option. As TDS does not support FetchXML options, Resco Mobile CRM switched to standard Fetch communication.
2. TDS uses dynamic paging. Page size is selected so that the result set takes at most 20 MB.
3. TDS does not support SELECT MIN_ACTIVE_ROW_VERSION. We need an extra row fetch to get that info.
4. Ignore this warning for TDS.
5. TDS communication does not count against the API call limit.

Risks, caveats, bottlenecks

The download is too fast. Older devices may have trouble processing and saving the records to the offline database. The cache may become full, and the WAL log size may become large. Use modern devices with enough RAM and free disk space. Increase cache size.