Storage analyzer
Resco Support |
---|
|
Storage analyzer is a collection of tools providing statistical data about the local storage of Resco mobile apps, with a particular focus on database storage and binary documents (blob storage).
Warning | The reports include tips and recommendations, however, they are primarily aimed at technical readers. Consider consulting with your technical support provider or Resco support before performing any configuration changes in production. |
Prerequisites
- Storage analyzer requires app version 14.1. Some reports may require a later version, see below for details.
- Storage analyzer is not available on Resco web apps (because they don't have any storage).
Using storage analyzer
- Go to the Setup/Settings screen, scroll down to the Advanced section, and select StorageAnalyzer.
- Tap Select and choose one of the available reports, or choose all of them.
- Wait until the report is generated.
- Optionally, you can email the generated report; or use the email preview screen to copy text from the reports.
If you have a large database, generating the reports can take significant time. Tap Abort to cancel the report generation.
To clear the results pane, tap Clear Results.
Visual guide: How to generate Storage Analyzer reports in Resco Mobile CRM? Demo
AppData report
This report provides general guidance about the data consumption of Resco mobile apps. Other reports provide more details, especially about the database and blob storage.
Customization = 3074K
WWW (Custom JS) = 2839K
Images (Custom images) = 177K
DB size = 6936M
DB log size = 0K
BlobStore = 1224M
Logs size = 38K
Temp files = 1309K
Other files = 57K
Temp Customization = 734K
---> Totally 8168M
- Customization – refers to the folder that contains all Woodford files created by Resco.
- WWW (Custom JS) – JavaScript Woodford files supplied by the customer.
- Images (Custom images) – Woodford images supplied by the customer.
- DB size – SQLite database file size (represents entity records downloaded from the server)
- DB log size – SQLite WAL log file size (temporary database storage responsible for guaranteeing data integrity in extreme situations, such as power outage)
- BlobStore – Folder containing data stored outside of the database in the file system, i.e., attachments and documents.
- Logs – Various log files maintained by Resco mobile apps. Use the About form to preview their content.
- Temp files – Temporary files. (Mainly customization *.zip files.)
- Other files – Remaining files maintained by Resco mobile apps.
- Temp Customization – Folder with new customization which failed to be applied for any reason - maybe app crashed or was forcefully killed. Will be cleared sooner or later.
Entities report
This report provides basic information about all tables stored in the database.
======= Nonempty Tables =======
campaign: 2722 rows, 43 cols, 390 By/row, 1038K (+indexes ~331K)
column xx_desctriptionofexposures = 277K
Empty columns: entityimageid stageid emailaddress entityimage promotioncodename processid pricelistidTarget pricelistid
email: 3 rows, 16 cols, 679 By/row, 2K (+indexes ~1K)
Empty columns: actualdurationminutes scheduledend scheduledstart
resco_mobilereport: 3 rows, 11 cols, 20126 By/row, 59K (+indexes ~1K)
resco_question: 4741142 rows, 38 cols, 348 By/row, 1614703K (+indexes ~443K) -> !!
Empty columns: resco_icon resco_answerstorage resco_defaultvalue
resco_questiongroup: 1185001 rows, 21 cols, 1264 By/row, 1463507K (+indexes ~382K) -> !!
column resco_rules = 1223101K
Empty columns: resco_description
Total size of data stored in DB = 986M
Note:
DB overhead adds ~40%.
The rest are table indexes - often >50%.
(Internal tables use to take negligible space.)
======= Empty synced entities =======
campaignitem
contract
contractdetail
discount
discounttype
incident
invoice
…
(By removing these entities from customization you could save … API call(s) in each sync) -> !!
In the example above, the campaign entity has 2722 records, 43 columns, and the table record stores on average 390 data bytes. The total size of the data stored in the campaign table is 1038K. Table indexes take up an estimated 331K.
The size occupied by individual table columns is listed only in specific cases:
- If the column takes an important portion of the table space. (Dominant columns such as campaign.xx_desctriptionofexposures, resco_questiongroup.resco_rules in the example.)
- If the column is empty.
Empty tables (tables containing 0 records) are reported together.
The report shows the net size of the data stored in the database, which is often less than ½ of the size of the database file. This is because the database has substantial overhead and also because of indexes that serve for fast data access.
Tips to consider
- Do you need empty table columns? They cause a small penalty both in local database operations and in server fetches.
- Do you need empty entities? Each such entity needs 1 fetch during synchronization.
- Label "-> !!" marks tables whose analysis took a long time (right now "too long" means 1+ second). The reason is usually too many rows or too many columns. (Or an older, slower device.)
- For tables containing too many records: Consider decreasing the row count by designing a stronger sync filter.
BlobStore report
The term BlobStore denotes file storage used for
- Attachments
- Documents
- Entity images
All these items represent potentially large data and are thus stored outside of the database as separate files.
======= BlobStore =======
product(images): 1801 file(s), 5080K
annotation: 3694 file(s), 1144M
Cloud docs: 281 file(s), 71M
salesliteratureitem: 18 file(s), 3644K
---> 1224M totally
16 docs enqueued for download when sync resumes ( SharePoint*3 Box*13 )
56 attachments enqueued for download when sync resumes (112M)
Explanation:
- Entity product stores entity images in the BlobStore. A lot of small files, nothing important.
- 3694 attachments (file annotations) take more than 1 GB. Worth investigating.
- 281 small cloud documents, nothing important.
- 18 salesliteratureitem records with an average 200K size.
- The last two lines may appear if the last sync was interrupted before all attachments/documents could be downloaded.
Tips to consider
This report provides a summary of the downloaded items. For example, attachments that were not downloaded because of interrupted sync or because of size limit are not considered here. Other reports provide more info, especially about the attachments and cloud documents.
File annotations report
This report provides detailed information about:
- Entities using annotation attachments,
- Type of data stored in annotations as well as most important file size stats,
- Similar stats are provided also for not downloaded files (if any).
======= File Annotations =======
Annotation counts by related entity:
3 x account
124 x campaign
1 x msdyn_workorder
7 x xx_contract
414 x xx_contractmonitoring
106 x xx_document
1 x resco_chatpost
1303 x resco_questionnaire
1736 x salesorder
---> 3695 Notes totally
======= Analysis: All Notes =======
Downloaded 3694 out of 3695
Jpeg: 278x, AVG=2671K, MAX=7321K, Top10%Avg=4210K
Consider downscaling Jpeg images -> !!
PNG: 2743x, AVG=24K, MAX=125K, Top10%Avg=69K
Reports: 435x, AVG=451K, MAX=466K, Top10%Avg=457K
PDF: 224x, AVG=754K, MAX=6035K, Top10%Avg=3646K
Others: 14x, AVG=77K, MAX=384K
NotDownloaded: 1x, AVG=0K
======= Analysis: Notes.resco_questionnaire =======
Downloaded 1303 out of 1303
Jpeg: 263x, AVG=2687K, MAX=7321K, Top10%Avg=4240K
Consider downscaling Jpeg images -> !!
PNG: 1040x, AVG=40K, MAX=125K, Top10%Avg=79K
======= Analysis: Notes.salesorder =======
Downloaded 1735 out of 1736
Jpeg: 11x, AVG=2379K, MAX=2830K
Consider downscaling Jpeg images -> !!
PNG: 1283x, AVG=11K, MAX=111K, Top10%Avg=30K
Reports: 432x, AVG=451K, MAX=466K, Top10%Avg=457K
PDF: 1x, AVG=172K
Others: 8x, AVG=115K, MAX=384K
NotDownloaded: 1x, AVG=0K
Explanation:
- Attachments are primarily used by questionnaires (1303) and sales orders (1736).
- In general, attachments are small PNG images (2743).
- There are 10x fewer Jpeg images, however, they occupy 10x more space (278*2671 > 700M).
- Even the analyzer found this suspicious and showed warning "Consider downscaling Jpeg images".
- Section "Analysis: All Notes" talks about all attachments.
- Section "Analysis: Notes.resco_questionnaire" talks only about attachments related to questionnaires.
- The row
Jpeg: 263x, AVG=2687K, MAX=7321K, Top10%Avg=4240K
means:
- 263 jpeg images
- Average file size = 2687K (2.6M)
- Maximum file size = 7321K (7.1M)
- Average file size of the largest 10% (highest decile) is 4240K (4.1M)
Tips to consider
- Look for information on which entities and which file types consume most storage.
- Analyze whether it would be useful to decrease either attachment count or their size.
- For example 7MB jpeg images typically use resolution >10MPix and can be often downscaled by as much as 10-50x. Unless you have a specific reason, consider downscaling.
Cloud documents report
This report resembles the annotations report, i.e. it provides detailed information about:
- Entities using documents
- Document types used
- The most important file size stats
Similar stats are provided also for not downloaded files (if any).
======= Cloud Docs =======
Docs counts by related entity:
15 x xx_device_location
419 x xx_vertriebsmediathek
8 x opportunity
---> 442 Cloud docs totally
======= Analysis: All CloudDocs =======
Downloaded 438 out of 442
Jpeg: 221x, AVG=241K, MAX=2143K, Top10%Avg=567K
Consider downscaling Jpeg images -> !!
PNG: 194x, AVG=53K, MAX=304K, Top10%Avg=145K
Others: 23x, AVG=4435K, MAX=15M
NotDownloaded: 4x, AVG=871K, MAX=3432K
======= Analysis: CloudDocs.xx_vertriebsmediathek =======
Downloaded 419 out of 419
Jpeg: 212x, AVG=242K, MAX=2143K, Top10%Avg=564K
Consider downscaling Jpeg images -> !!
PNG: 192x, AVG=52K, MAX=146K, Top10%Avg=136K
Others: 15x, AVG=6800K, MAX=15M
Explanation:
- Documents are primarily used by xx_vertriebsmediathek entity (419).
- In general, most documents are images - either PNG (194) or Jpeg (221).
- Jpeg images are substantially larger than PNG images.
- Even the analyzer found this suspicious and showed warning "Consider downscaling Jpeg images".
- For the meaning of statistical terms (AVG, MAX, Top10%Avg) see the description of FileAnnotationsReport.
LastSyncTime report
Synchronization (among other things) attempts to download server changes for all entities in the client customization. This is done through a set of fetches executed within some time period. Given the fact that these fetches cannot be executed at once, different entities are becoming up-to-date at different times.
This is what this report depicts. For example, the next example shows that the entity account became synced 13 seconds sooner than the entity annotation.
For those who want to dig deeper: The report shows also rowversions which are unlike sync times fully reliable and which should be read as follows: The client imported all account records with rowversion<=561666201.
======= LastSyncTimeReport =======
account 20-05-19 13:34:52, Ver=561666201
annotation 20-05-19 13:35:05, Ver=561668382
appointment 20-05-19 13:35:04, Ver=561668310
businessunit 20-05-19 13:34:51, Ver=557155673
campaign 20-05-19 13:34:52, Ver=561666226
campaignitem 20-05-19 13:39:29, Ver=561711822
Incomplete Entities -> !!
contact
Tips to consider
- If the last sync was aborted, you can encounter a big difference in sync times.
- This report may list also "Incomplete Entities". The sync of these entities was interrupted prematurely because the limit Max Sync Records was reached.
DocuSign report
This report makes sense if you use electronic signatures from DocuSign. It lists technical details about the documents waiting to be signed. This kind of information can be useful for Resco support when investigating problems.
Sign jobs:
SignPending (LocalId: 3b5bf584-2b1d-45e3-a15e-7e71267532db) for [annotation, Signature.png] -> [resco_questionnaire, Q2 Approval]
DownloadPending (ServerId: 8da6a51f-f1e9-420d-a4ed-fb06b72cddb6) for SharePoint doc img_210312_103233.jpg -> [opportunity, Standort #2]
Explanation
Document signing is a complex process consisting of several steps:
- Document is uploaded to the signing server.
- Document is signed by the server.
- Signed document is downloaded to the client, where it replaces the original document.
In the above example:
- Signature.png stored in an annotation record attached to a questionnaire: Signing process initiated.
- Snapshot attached to an opportunity record: Signed file waiting on the server for download.
Database internals report
A rather technical report showing the details of the database engine used.
======= DbInternalsReport =======
LibVersion= Sqlite 3.8.8.1
DbSize= 7102060K
WalSize= 97844K
RegisterCollation() -> OK
Selected PRAGMAs:
freelist_count= 6005
page_size= 4096
cache_size= 20000 pages
fullfsync= 0
CompileTimeOptions: ENABLE_COLUMN_METADATA HAS_CODEC SYSTEM_MALLOC THREADSAFE=1 SQLITE_THREADSAFE=1
*** MemoryInfo ***
SQLITE_STATUS_MEMORY_USED=87882288/87892888
…
Explanation:
- WalSize refers to the size of the database log file.
- RegisterCollation reports an eventual error when NOCASE collation (case-insensitive sorting) is registered with SQLite engine. (“OK” is the expected result.)
- See SQLite documentation for more details.
Requires Resco Mobile CRM version 15.1 or later. This report gives an overview of document location records. In addition to the basic stats, the report may contain various warnings/tips as shown in the following example.
======= Analysis of sharepointdocumentlocation entity =======
sharepointdocumentlocation entity contains enabled fields that are not required. Extra fields may slow down sync, especially for large data sets. -> !!
Root locations (sharepointdocumentlocation records):
Documents at default site 1[4e7ba4ce-1234-e911-a97d-000d3a34a0aa] /appointment
Documents at default site 1[c5d1fa02-1234-e911-a981-000d3a34a0aa] /opportunity
Documents at default site 1[72c8eeae-1234-e911-a986-000d3a34a0aa] /contact
Documents at default site 1[5b1dc708-1234-e911-a979-000d3a34a14b] /competitor
Documents at default site 1[ace73084-1234-e911-a979-000d3a34a14b] /ext_deliverypricelist
Documents at default site 1[48630f4b-1234-e911-a976-000d3a34ae50] /ext_otheractivities
Documents at default site 1[06bc6cc2-1234-e911-a977-000d3a34ae50] /account
Documents at default site 1[31b16a5b-1234-e911-a979-000d3a34ae50] /lead
Documents at default site 1[74adfea6-1234-e911-a979-000d3a34ae50] /ext_businessopportunity
Documents at default site 1[84c84969-1234-e911-a979-000d3a34afa9] /ext_phonecall
Documents at default site 1[3992a56b-1234-eb11-a813-000d3a3be299] /product
Counts of locations pointing to CRM RECORDS IN OFFLINE (LOCAL) DB:
account * 129
appointment * 14106
competitor * 5
contact * 2
ext_deliverypricelist * 1
ext_otheractivities * 5914
product * 2
---> 20159 totally
(Tip: Looks like too many locations synced / files downloaded. Check if they are necessary.)
Counts of locations pointing to CRM RECORDS NOT IN OFFLINE (LOCAL) DB:
account * 1947
appointment * 166192
contact * 86
ext_phonecall * 2 (entity not in the customization)
ext_businessopportunity * 18
ext_otheractivities * 63854
lead * 646
opportunity * 121
---> 232866 totally
(Tip: These records increase sync overhead. Remove them via better SyncFilter.)
Explanation:
- Extra entity fields occupy unneeded storage and slow down sync, especially if a large number of locations are used. Woodford by default selects all required sharepointdocumentlocation fields; no other field is needed.
- Every CRM entity using SharePoint documents needs to have defined so-called root location.
- Every CRM record with SharePoint documents has one or more locations, where documents are stored on SharePoint.
- FullSync queries SharePoint in batches of 100 locations/call. A high number of locations means, a) many server requests, and b) potentially too many files downloaded. The case shown in the example looks like overkill, hence the warning.
- Unknown locations point either to entities not included in the customization or to records that were not downloaded to the client, i.e. excluded by the sync filter. The recommendation (tip) says that the sync filter for sharepointdocumentlocation entity should be adjusted to the customization to avoid unnecessary sync overhead.
Questionnaire statistics
This report introduced in release 17.0 offers more insights into the questionnaires and their local storage consumption in the Resco Mobile CRM. The report includes:
- Number of templates and data questionnaires (answers), as well as their storage needs
- Template statistics by storage type, template dependency, versioning
- Largest data questionnaires
- Largest templates (used/unused)
- Largest columns for templates/data questionnaires
- SyncDownloader stats highlights
See also
For a statistical overview of server storage (Resco Cloud), visit the Admin Console.