[
https://issues.apache.org/jira/browse/HBASE-22899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rabi Kumar K C reassigned HBASE-22899:
--------------------------------------
Assignee: Rabi Kumar K C
> logging improvements for snapshot operations w/large manifests
> --------------------------------------------------------------
>
> Key: HBASE-22899
> URL: https://issues.apache.org/jira/browse/HBASE-22899
> Project: HBase
> Issue Type: Improvement
> Components: snapshots
> Reporter: Sean Busbey
> Assignee: Rabi Kumar K C
> Priority: Minor
> Labels: beginner
>
> working through some issues with snapshotting in the presence of a
> substantial (millions-of) hfiles.
> log messages about the state of creating or restoring span multiple hundreds
> of MB of logs, so more context in individual lines would make it easier to
> orient myself.
> RestoreSnapshotHelper has a few different stanzas that essentially all say
> "I'm creating an HFileLink for this thing". e.g.
> {code}
> for (SnapshotRegionManifest.FamilyFiles familyFiles:
> manifest.getFamilyFilesList()) {
> Path familyDir = new Path(regionDir,
> familyFiles.getFamilyName().toStringUtf8());
> for (SnapshotRegionManifest.StoreFile storeFile:
> familyFiles.getStoreFilesList()) {
> LOG.info("Adding HFileLink " + storeFile.getName() + " to table=" +
> tableName);
> restoreStoreFile(familyDir, snapshotRegionInfo, storeFile,
> createBackRefs);
> }
> }
> {code}
> If would be helpful if
> 1) these stanzas offered something in the logged text to tell them apart (I
> think the coincidentally differ in log level, but that's esoteric and easy to
> lose over time)
> 2) it would be nice if we included the snapshot name in the log message so I
> can more easily pull out all log lines related to the snapshot
> 3) these stanzas indicated "progress" of sorts by giving how many storefiles
> are present and where we are in that list.
> sort of like this bit we log when creating the snapshot manifest:
> {code}
> // 2.2. iterate through all the store's files and create "references".
> for (int i = 0, sz = storeFiles.size(); i < sz; i++) {
> HStoreFile storeFile = storeFiles.get(i);
> monitor.rethrowException();
> // create "reference" to this store file.
> LOG.debug("Adding reference for file (" + (i+1) + "/" + sz + "): " +
> storeFile.getPath());
> visitor.storeFile(regionData, familyData, storeFile.getFileInfo());
> }
> {code}
> And speaking of the above bit in SnapshotManifest, it would be nice if it
> included the name of the snapshot in the log message so it's easier to pull
> out all log lines related to the snapshot.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
