[
https://issues.apache.org/jira/browse/HDDS-17079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siyao Meng updated HDDS-17079:
------------------------------
Summary: Reduce OM startup GC by streaming snapshot local data YAMLs for
the version graph (was: Skip SST file lists when loading snapshot local data
YAMLs for the snapshot version graph)
> Reduce OM startup GC by streaming snapshot local data YAMLs for the version
> graph
> ---------------------------------------------------------------------------------
>
> Key: HDDS-17079
> URL: https://issues.apache.org/jira/browse/HDDS-17079
> Project: Apache Ozone
> Issue Type: Sub-task
> Components: Ozone Manager, Snapshot
> Reporter: Siyao Meng
> Assignee: Siyao Meng
> Priority: Major
> Labels: pull-request-available
>
> h3. Problem
> To build its snapshot version graph, OmSnapshotLocalDataManager only needs
> each YAML's snapshotId, previousSnapshotId, version and the
> previousSnapshotVersion of each version. Every load still builds all
> SstFileInfo entries, which make up almost the whole file, and then throws
> them away. With many snapshots this is most of the cost of OM startup, even
> after HDDS-17078 (parse each YAML once at startup).
> h3. Analysis
> Timing one 9 MB YAML from a cluster with about 19k snapshots (one version,
> about 36.7k SstFileInfo entries), G1 with a 900 MB young generation:
> ||Load||1 thread, per file||8 threads, 64 files||
> |Full load|0.153 s| |
> |Node filter (drop the sstFiles node before construction)|0.088 s|1.6 to 3.0
> s, GC 46 to 60% of wall, old gen peak 4 GB|
> |Streaming parser events|0.067 s|0.6 s, GC 1% of wall, old gen peak 9 MB|
> Dropping the sstFiles node before construction is not enough: the whole node
> tree is still composed first. With several loads in flight these trees get
> promoted, and on a large cluster startup became GC bound.
> h3. Proposed fix
> Add OmSnapshotLocalDataYaml#loadWithoutSstFiles, which reads the four fields
> above straight from the snakeyaml parser events and skips everything else, so
> no node tree or SstFileInfo is built. The returned object has empty SST file
> lists, no transaction info and no checksum. OmSnapshotLocalDataManager uses
> it only in init and in the previous snapshot walk. All other reads and every
> write still use the full serializer.
> h3. Compatibility
> The on disk format is unchanged. New code reads YAMLs written by existing
> releases, and nothing new is written.
> One behavior change: startup still parses every event, so files that are not
> valid YAML are still skipped as before. But invalid values in other fields
> (for example inside SST file entries) are no longer detected at startup; they
> show up on the next full load of that snapshot.
> h3. Testing
> Added TestOmSnapshotLocalDataYaml#testLoadWithoutSstFiles. A YAML written by
> the regular serializer loads with the same snapshot ids, versions and
> previousSnapshotVersion values as a full load, with empty SST file lists, and
> a YAML without snapshotId is rejected.
> Related to HDDS-17078; independent change, the speedups combine.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
