Join the Power Platform Masterclass #2 starting April 23

Storage analyzer: Difference between revisions

Jump to navigation Jump to search

Storage analyzer (view source)

Revision as of 13:24, 2 June 2021

9,151 bytes added ,  2 June 2021
→‎Entities report
Line 49: Line 49:
== Entities report ==
== Entities report ==


This report provides basic information about all tables stored in the database.


<syntaxhighlight lang="text">
    ======= Nonempty Tables =======
campaign: 2722 rows, 43 cols, 1038K
column xx_desctriptionofexposures = 277K
Empty columns:  entityimageid stageid emailaddress entityimage promotioncodename processid pricelistidTarget pricelistid
email: 3 rows, 16 cols, 2K
Empty columns:  actualdurationminutes scheduledend scheduledstart
resco_mobilereport: 3 rows, 11 cols, 59K
resco_question: 4741142 rows, 38 cols, 1614703K -> !!
Empty columns:  resco_icon resco_answerstorage resco_defaultvalue
resco_questiongroup: 1185001 rows, 21 cols, 1463507K -> !!
column resco_rules = 1223101K
Empty columns:  resco_description
    ======= Empty Tables =======
campaignitem
contract
contractdetail
discount
discounttype
</syntaxhighlight>
In the example above, the campaign entity has 2722 records, 43 columns, and the total size of the stored data is 1038K. Note that the real data consumption is higher because of the database overhead and also because the space occupied by table indexes is not counted.
The size occupied by individual table columns is listed only in specific cases:
* If the column takes an important portion of the table space. (Dominant columns such as campaign.xx_desctriptionofexposures, resco_questiongroup.resco_rules in the example.)
* If the column is empty.
Empty tables (tables containing 0 records) are reported together.
=== Tips to consider ===
* Do you need empty table columns? Their cause a small penalty both in local database operations and in server fetches.
* Do you need empty tables? They usually represent entities and each such entity needs 1 fetch during synchronization.
* Label "-> !!" marks tables whose analysis took a long time (right now "too long" means 1+ second). The reason is usually too many rows or too many columns. (Or an older, slower device.)
* For tables containing too many records: Can't you decrease the row count by designing a stronger [[sync filter]]?
== BlobStore report ==
The term BlobStore denotes file storage used for
* Attachments
* Documents
* Entity images
All these items represent potentially large data and are thus stored outside of the database as separate files.
<syntaxhighlight lang="text">
======= BlobStore =======
product(images): 1801 file(s), 5080K
annotation: 3694 file(s), 1144M
Cloud docs: 281 file(s), 71M
salesliteratureitem: 18 file(s), 3644K
---> 1224M totally
</syntaxhighlight>
Explanation:
* Entity product stores entity images in the BlobStore. A lot of small files, nothing important.
* 3694 attachments (file annotations) take more than 1 GB. Worth investigating.
* 281 small cloud documents, nothing important.
* 18 salesliteratureitem records with an average 200K size.
=== Tips to consider ===
This report provides a summary of the downloaded items. For example, attachments that were not downloaded because of interrupted sync or because of size limit are not considered here. Other reports provide more info, especially about the attachments and cloud documents.
== File annotations report ==
This report provides detailed information about:
* Entities using annotation attachments,
* Type of data stored in annotations as well as most important file size stats,
* Similar stats are provided also for not downloaded files (if any).
<syntaxhighlight lang="text">
======= File Annotations =======
Annotation counts by related entity:
3 x account
124 x campaign
1 x msdyn_workorder
7 x xx_contract
414 x xx_contractmonitoring
106 x xx_document
1 x resco_chatpost
1303 x resco_questionnaire
1736 x salesorder
---> 3695 Notes totally
======= Analysis: All Notes =======
Downloaded 3694 out of 3695
Jpeg: 278x, AVG=2671K, MAX=7321K, Top10%Avg=4210K
Consider downscaling Jpeg images -> !!
PNG: 2743x, AVG=24K, MAX=125K, Top10%Avg=69K
Reports: 435x, AVG=451K, MAX=466K, Top10%Avg=457K
PDF: 224x, AVG=754K, MAX=6035K, Top10%Avg=3646K
Others: 14x, AVG=77K, MAX=384K
NotDownloaded: 1x, AVG=0K
======= Analysis: Notes.resco_questionnaire =======
Downloaded 1303 out of 1303
Jpeg: 263x, AVG=2687K, MAX=7321K, Top10%Avg=4240K
Consider downscaling Jpeg images -> !!
PNG: 1040x, AVG=40K, MAX=125K, Top10%Avg=79K
======= Analysis: Notes.salesorder =======
Downloaded 1735 out of 1736
Jpeg: 11x, AVG=2379K, MAX=2830K
Consider downscaling Jpeg images -> !!
PNG: 1283x, AVG=11K, MAX=111K, Top10%Avg=30K
Reports: 432x, AVG=451K, MAX=466K, Top10%Avg=457K
PDF: 1x, AVG=172K
Others: 8x, AVG=115K, MAX=384K
NotDownloaded: 1x, AVG=0K
</syntaxhighlight>
Explanation:
* Attachments are primarily used by questionnaires (1303) and sales orders (1736).
* In general, attachments are small PNG images (2743).
* There are 10x fewer Jpeg images, however, they occupy 10x more space (278*2671 > 700M).
* Even the analyzer found this suspicious and showed warning "Consider downscaling Jpeg images".
* Section "Analysis: All Notes" talks about all attachments.
* Section "Analysis: Notes.resco_questionnaire" talks only about attachments related to questionnaires.
* The row <code>Jpeg: 263x, AVG=2687K, MAX=7321K, Top10%Avg=4240K</code> means:
:* 263 jpeg images
:* Average file size = 2687K (2.6M)
:* Maximum file size = 7321K (7.1M)
:* Average file size of the largest 10% (highest decile) is 4240K (4.1M)
=== Tips to consider ===
* Look for information on which entities and which file types consume most storage.
* Analyze whether it would be useful to decrease either attachment count or their size.
* For example 7MB jpeg images typically use resolution >10MPix and can be often downscaled by as much as 10-50x. Unless you have a specific reason, consider downscaling.
== Cloud documents report ==
This report resembles the annotations report, i.e. it provides detailed information about:
* Entities using documents
* Document types used
* The most important file size stats
Similar stats are provided also for not downloaded files (if any).
<syntaxhighlight lang="text">
    ======= Cloud Docs =======
Docs counts by related entity:
15 x xx_device_location
419 x xx_vertriebsmediathek
8 x opportunity
---> 442 Cloud docs totally
======= Analysis: All CloudDocs =======
Downloaded 438 out of 442
Jpeg: 221x, AVG=241K, MAX=2143K, Top10%Avg=567K
Consider downscaling Jpeg images -> !!
PNG: 194x, AVG=53K, MAX=304K, Top10%Avg=145K
Others: 23x, AVG=4435K, MAX=15M
NotDownloaded: 4x, AVG=871K, MAX=3432K
======= Analysis: CloudDocs.xx_vertriebsmediathek =======
Downloaded 419 out of 419
Jpeg: 212x, AVG=242K, MAX=2143K, Top10%Avg=564K
  Consider downscaling Jpeg images -> !!
PNG: 192x, AVG=52K, MAX=146K, Top10%Avg=136K
Others: 15x, AVG=6800K, MAX=15M
</syntaxhighlight>
Explanation:
* Attachments are primarily used by xx_vertriebsmediathek entity (419).
* In general most attachments are images - either PNG (194) or Jpeg (221).
* Jpeg images are substantially larger than PNG images.
* Even the analyzer found this suspicious and showed warning "Consider downscaling Jpeg images".
* For the meaning of statistical terms (AVG, MAX, Top10%Avg) see the description of FileAnnotationsReport.
== LastSyncTime report ==
[[Synchronization]] (among other things) attempts to download server changes for all entities in the client customization. This is done through a set of fetches executed within some time period. Given the fact that these fetches cannot be executed at once, different entities are becoming up-to-date at different times.
This is what this report depicts. For example, the next example shows that the entity account became synced 13 seconds sooner than the entity annotation.
For those who want to dig deeper: The report shows also rowversions which are unlike sync times fully reliable and which should be read as follows: The client imported all account records with rowversion<=561666201.
<syntaxhighlight lang="text">
    ======= LastSyncTimeReport =======
    account  20-05-19 13:34:52, Ver=561666201
    annotation  20-05-19 13:35:05, Ver=561668382
    appointment  20-05-19 13:35:04, Ver=561668310
    businessunit  20-05-19 13:34:51, Ver=557155673
    campaign  20-05-19 13:34:52, Ver=561666226
    campaignitem  20-05-19 13:39:29, Ver=561711822
Incomplete Entities -> !!
contact
</syntaxhighlight>
=== Tips to consider ===
* If the last sync was aborted, you can encounter a big difference in sync times.
* This report may list also "Incomplete Entities". The sync of these entities was interrupted prematurely because the limit '''[[Configuration#Sync_Data|Max Sync Records]]''' was reached.
== Database internals report ==
A rather technical report showing the details of the database engine used.
<syntaxhighlight lang="text">
    ======= DbInternalsReport =======
LibVersion= Sqlite 3.8.8.1
DbSize= 7102060K
WalSize= 0K
Unused pages= 6005
page_size= 4096
cache_size= 20000 pages
fullfsync= 0
CompileTimeOptions: ENABLE_COLUMN_METADATA HAS_CODEC SYSTEM_MALLOC THREADSAFE=1 SQLITE_THREADSAFE=1
*** MemoryInfo ***
SQLITE_STATUS_MEMORY_USED=87882304/88149272
</syntaxhighlight>


[[Category:Support]]
[[Category:Support]]
Retrieved from "https://docs.resco.net/wiki/Special:MobileDiff/4760"

Navigation menu