[
https://issues.apache.org/jira/browse/HDDS-9299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17765962#comment-17765962
]
Attila Doroszlai commented on HDDS-9299:
----------------------------------------
Breakdown of time elapsed per method:
{code}
Tests run: 164, Failures: 0, Errors: 0, Skipped: 9, Time elapsed: 1,149.131 s -
in org.apache.hadoop.ozone.om.TestOmSnapshot
TestOmSnapshot.testCreateSnapshotMissingMandatoryParams Time elapsed: 26.482 s
TestOmSnapshot.testNonExistentBucket Time elapsed: 0.013 s
TestOmSnapshot.testSnapDiffWithKeyRenamesRecreationAndDelete Time elapsed:
5.412 s
TestOmSnapshot.testSnapshotDiffWhenOmRestart Time elapsed: 10.58 s
TestOmSnapshot skipped
TestOmSnapshot.testDeleteSnapshotFailure Time elapsed: 1.162 s
TestOmSnapshot.testSnapshotQuotaHandling Time elapsed: 1.195 s
TestOmSnapshot.testSnapshotDiffWhenOmRestartAndReportIsPartiallyFetched Time
elapsed: 12.113 s
TestOmSnapshot.testSnapDiffMultipleBuckets Time elapsed: 3.239 s
TestOmSnapshot.testSnapDiffCancelFailureResponses Time elapsed: 2.317 s
TestOmSnapshot.testDeleteSnapshotTwice Time elapsed: 1.122 s
TestOmSnapshot.testCompactionDagDisableForSnapshotMetadata Time elapsed: 1.09 s
TestOmSnapshot.testSnapDiffWith2RenamesAndDelete Time elapsed: 4.178 s
TestOmSnapshot.testSnapDiffWithNoEffectiveRename Time elapsed: 3.13 s
TestOmSnapshot.testSnapDiffNonExistentUrl Time elapsed: 2.121 s
TestOmSnapshot.testSnapDiffHandlingReclaimWithPreviousUse Time elapsed: 5.188 s
TestOmSnapshot.testListSnapshotDiffWithInvalidParameters Time elapsed: 2.177 s
TestOmSnapshot skipped
TestOmSnapshot skipped
TestOmSnapshot.testListKey Time elapsed: 9.265 s
TestOmSnapshot.testSnapDiffReclaimWithKeyRename Time elapsed: 3.12 s
TestOmSnapshot.testSnapDiffWithDirectory Time elapsed: 3.14 s
TestOmSnapshot.testSnapDiffReclaimWithDeferredKeyDeletion Time elapsed: 5.184 s
TestOmSnapshot.testSnapDiffReclaimWithKeyRecreation Time elapsed: 8.273 s
TestOmSnapshot.testBucketDeleteIfSnapshotExists Time elapsed: 1.179 s
TestOmSnapshot.testListDeleteKey Time elapsed: 2.252 s
TestOmSnapshot skipped
TestOmSnapshot.testListAddNewKey Time elapsed: 2.28 s
TestOmSnapshot.testSnapDiff Time elapsed: 12.49 s
TestOmSnapshot skipped
TestOmSnapshot.testDayWeekMonthSnapshotCreationAndExpiration Time elapsed:
136.08 s
TestOmSnapshot.testSnapDiffHandlingReclaimWithLatestUse Time elapsed: 5.158 s
TestOmSnapshot.testListKeyOnEmptyBucket Time elapsed: 1.121 s
TestOmSnapshot.testSnapdiffWithObjectMetaModification Time elapsed: 3.168 s
TestOmSnapshot.testSnapDiffCancel Time elapsed: 2.16 s
TestOmSnapshot.checkKey Time elapsed: 2.24 s
TestOmSnapshot.testSnapDiffNoSnapshot Time elapsed: 1.078 s
TestOmSnapshot.testSnapDiffWithKeyOverwrite Time elapsed: 3.165 s
TestOmSnapshot.testDeleteSnapshotMissingMandatoryParams Time elapsed: 1.084 s
TestOmSnapshot.testSnapDiffMissingMandatoryParams Time elapsed: 2.116 s
TestOmSnapshot.testSnapshotOpensWithDisabledAutoCompaction Time elapsed: 1.137
s
TestOmSnapshot.testCreateSnapshotMissingMandatoryParams Time elapsed: 24.917 s
TestOmSnapshot.testNonExistentBucket Time elapsed: 0.01 s
TestOmSnapshot.testSnapDiffWithKeyRenamesRecreationAndDelete Time elapsed:
5.296 s
TestOmSnapshot.testSnapshotDiffWhenOmRestart Time elapsed: 8.904 s
TestOmSnapshot skipped
TestOmSnapshot.testDeleteSnapshotFailure Time elapsed: 1.107 s
TestOmSnapshot.testSnapshotQuotaHandling Time elapsed: 1.088 s
TestOmSnapshot.testSnapshotDiffWhenOmRestartAndReportIsPartiallyFetched Time
elapsed: 11.048 s
TestOmSnapshot.testSnapDiffMultipleBuckets Time elapsed: 3.208 s
TestOmSnapshot.testSnapDiffCancelFailureResponses Time elapsed: 2.158 s
TestOmSnapshot.testDeleteSnapshotTwice Time elapsed: 1.089 s
TestOmSnapshot.testCompactionDagDisableForSnapshotMetadata Time elapsed: 1.066
s
TestOmSnapshot.testSnapDiffWith2RenamesAndDelete Time elapsed: 4.149 s
TestOmSnapshot.testSnapDiffWithNoEffectiveRename Time elapsed: 3.115 s
TestOmSnapshot.testSnapDiffNonExistentUrl Time elapsed: 2.1 s
TestOmSnapshot.testSnapDiffHandlingReclaimWithPreviousUse Time elapsed: 5.142 s
TestOmSnapshot.testListSnapshotDiffWithInvalidParameters Time elapsed: 2.139 s
TestOmSnapshot.testSnapDiffWithDirRename Time elapsed: 3.096 s
TestOmSnapshot.testSnapdiffWithFilesystemDirectoryRenameOperation Time
elapsed: 3.435 s
TestOmSnapshot.testListKey Time elapsed: 9.069 s
TestOmSnapshot.testSnapDiffReclaimWithKeyRename Time elapsed: 3.139 s
TestOmSnapshot.testSnapDiffWithDirectory Time elapsed: 3.115 s
TestOmSnapshot.testSnapDiffReclaimWithDeferredKeyDeletion Time elapsed: 5.154 s
TestOmSnapshot.testSnapDiffReclaimWithKeyRecreation Time elapsed: 8.243 s
TestOmSnapshot.testBucketDeleteIfSnapshotExists Time elapsed: 1.176 s
TestOmSnapshot.testListDeleteKey Time elapsed: 2.234 s
TestOmSnapshot.testSnapdiffWithFilesystemDirectoryMoveOperation Time elapsed:
3.198 s
TestOmSnapshot.testListAddNewKey Time elapsed: 2.291 s
TestOmSnapshot.testSnapDiff Time elapsed: 12.426 s
TestOmSnapshot.testSnapdiffWithFilesystemCreate Time elapsed: 3.205 s
TestOmSnapshot.testDayWeekMonthSnapshotCreationAndExpiration Time elapsed:
129.842 s
TestOmSnapshot.testSnapDiffHandlingReclaimWithLatestUse Time elapsed: 5.138 s
TestOmSnapshot.testListKeyOnEmptyBucket Time elapsed: 1.129 s
TestOmSnapshot.testSnapdiffWithObjectMetaModification Time elapsed: 3.106 s
TestOmSnapshot.testSnapDiffCancel Time elapsed: 2.143 s
TestOmSnapshot.checkKey Time elapsed: 2.261 s
TestOmSnapshot.testSnapDiffNoSnapshot Time elapsed: 1.066 s
TestOmSnapshot.testSnapDiffWithKeyOverwrite Time elapsed: 3.159 s
TestOmSnapshot.testDeleteSnapshotMissingMandatoryParams Time elapsed: 1.065 s
TestOmSnapshot.testSnapDiffMissingMandatoryParams Time elapsed: 2.104 s
TestOmSnapshot.testSnapshotOpensWithDisabledAutoCompaction Time elapsed: 1.117
s
TestOmSnapshot.testCreateSnapshotMissingMandatoryParams Time elapsed: 25.604 s
TestOmSnapshot.testNonExistentBucket Time elapsed: 0.007 s
TestOmSnapshot.testSnapDiffWithKeyRenamesRecreationAndDelete Time elapsed:
5.201 s
TestOmSnapshot.testSnapshotDiffWhenOmRestart Time elapsed: 8.533 s
TestOmSnapshot skipped
TestOmSnapshot.testDeleteSnapshotFailure Time elapsed: 1.091 s
TestOmSnapshot.testSnapshotQuotaHandling Time elapsed: 1.09 s
TestOmSnapshot.testSnapshotDiffWhenOmRestartAndReportIsPartiallyFetched Time
elapsed: 10.823 s
TestOmSnapshot.testSnapDiffMultipleBuckets Time elapsed: 3.212 s
TestOmSnapshot.testSnapDiffCancelFailureResponses Time elapsed: 2.132 s
TestOmSnapshot.testDeleteSnapshotTwice Time elapsed: 1.089 s
TestOmSnapshot.testCompactionDagDisableForSnapshotMetadata Time elapsed: 1.066
s
TestOmSnapshot.testSnapDiffWith2RenamesAndDelete Time elapsed: 4.125 s
TestOmSnapshot.testSnapDiffWithNoEffectiveRename Time elapsed: 3.104 s
TestOmSnapshot.testSnapDiffNonExistentUrl Time elapsed: 2.118 s
TestOmSnapshot.testSnapDiffHandlingReclaimWithPreviousUse Time elapsed: 5.133 s
TestOmSnapshot.testListSnapshotDiffWithInvalidParameters Time elapsed: 2.156 s
TestOmSnapshot.testSnapDiffWithDirRename Time elapsed: 3.078 s
TestOmSnapshot.testSnapdiffWithFilesystemDirectoryRenameOperation Time
elapsed: 3.165 s
TestOmSnapshot.testListKey Time elapsed: 8.898 s
TestOmSnapshot.testSnapDiffReclaimWithKeyRename Time elapsed: 3.104 s
TestOmSnapshot.testSnapDiffWithDirectory Time elapsed: 3.123 s
TestOmSnapshot.testSnapDiffReclaimWithDeferredKeyDeletion Time elapsed: 5.139 s
TestOmSnapshot.testSnapDiffReclaimWithKeyRecreation Time elapsed: 8.22 s
TestOmSnapshot.testBucketDeleteIfSnapshotExists Time elapsed: 1.162 s
TestOmSnapshot.testListDeleteKey Time elapsed: 2.232 s
TestOmSnapshot.testSnapdiffWithFilesystemDirectoryMoveOperation Time elapsed:
3.174 s
TestOmSnapshot.testListAddNewKey Time elapsed: 2.288 s
TestOmSnapshot.testSnapDiff Time elapsed: 12.417 s
TestOmSnapshot.testSnapdiffWithFilesystemCreate Time elapsed: 3.162 s
TestOmSnapshot.testDayWeekMonthSnapshotCreationAndExpiration Time elapsed:
128.463 s
TestOmSnapshot.testSnapDiffHandlingReclaimWithLatestUse Time elapsed: 5.159 s
TestOmSnapshot.testListKeyOnEmptyBucket Time elapsed: 1.126 s
TestOmSnapshot.testSnapdiffWithObjectMetaModification Time elapsed: 3.099 s
TestOmSnapshot.testSnapDiffCancel Time elapsed: 2.168 s
TestOmSnapshot.checkKey Time elapsed: 2.192 s
TestOmSnapshot.testSnapDiffNoSnapshot Time elapsed: 1.071 s
TestOmSnapshot.testSnapDiffWithKeyOverwrite Time elapsed: 3.149 s
TestOmSnapshot.testDeleteSnapshotMissingMandatoryParams Time elapsed: 1.065 s
TestOmSnapshot.testSnapDiffMissingMandatoryParams Time elapsed: 2.098 s
TestOmSnapshot.testSnapshotOpensWithDisabledAutoCompaction Time elapsed: 1.104
s
TestOmSnapshot.testCreateSnapshotMissingMandatoryParams Time elapsed: 24.976 s
TestOmSnapshot.testNonExistentBucket Time elapsed: 0.005 s
TestOmSnapshot.testSnapDiffWithKeyRenamesRecreationAndDelete Time elapsed:
5.19 s
TestOmSnapshot.testSnapshotDiffWhenOmRestart Time elapsed: 8.389 s
TestOmSnapshot skipped
TestOmSnapshot.testDeleteSnapshotFailure Time elapsed: 1.116 s
TestOmSnapshot.testSnapshotQuotaHandling Time elapsed: 1.079 s
TestOmSnapshot.testSnapshotDiffWhenOmRestartAndReportIsPartiallyFetched Time
elapsed: 11.046 s
TestOmSnapshot.testSnapDiffMultipleBuckets Time elapsed: 3.239 s
TestOmSnapshot.testSnapDiffCancelFailureResponses Time elapsed: 2.125 s
TestOmSnapshot.testDeleteSnapshotTwice Time elapsed: 1.088 s
TestOmSnapshot.testCompactionDagDisableForSnapshotMetadata Time elapsed: 1.07 s
TestOmSnapshot.testSnapDiffWith2RenamesAndDelete Time elapsed: 4.118 s
TestOmSnapshot.testSnapDiffWithNoEffectiveRename Time elapsed: 3.103 s
TestOmSnapshot.testSnapDiffNonExistentUrl Time elapsed: 2.087 s
TestOmSnapshot.testSnapDiffHandlingReclaimWithPreviousUse Time elapsed: 5.125 s
TestOmSnapshot.testListSnapshotDiffWithInvalidParameters Time elapsed: 2.168 s
TestOmSnapshot skipped
TestOmSnapshot.testSnapdiffWithFilesystemDirectoryRenameOperation Time
elapsed: 3.195 s
TestOmSnapshot.testListKey Time elapsed: 8.899 s
TestOmSnapshot.testSnapDiffReclaimWithKeyRename Time elapsed: 3.111 s
TestOmSnapshot.testSnapDiffWithDirectory Time elapsed: 3.104 s
TestOmSnapshot.testSnapDiffReclaimWithDeferredKeyDeletion Time elapsed: 5.135 s
TestOmSnapshot.testSnapDiffReclaimWithKeyRecreation Time elapsed: 8.232 s
TestOmSnapshot.testBucketDeleteIfSnapshotExists Time elapsed: 1.149 s
TestOmSnapshot.testListDeleteKey Time elapsed: 2.225 s
TestOmSnapshot.testSnapdiffWithFilesystemDirectoryMoveOperation Time elapsed:
3.191 s
TestOmSnapshot.testListAddNewKey Time elapsed: 2.267 s
TestOmSnapshot.testSnapDiff Time elapsed: 12.398 s
TestOmSnapshot.testSnapdiffWithFilesystemCreate Time elapsed: 3.179 s
TestOmSnapshot.testDayWeekMonthSnapshotCreationAndExpiration Time elapsed:
124.417 s
TestOmSnapshot.testSnapDiffHandlingReclaimWithLatestUse Time elapsed: 5.135 s
TestOmSnapshot.testListKeyOnEmptyBucket Time elapsed: 1.142 s
TestOmSnapshot.testSnapdiffWithObjectMetaModification Time elapsed: 3.135 s
TestOmSnapshot.testSnapDiffCancel Time elapsed: 2.157 s
TestOmSnapshot.checkKey Time elapsed: 2.194 s
TestOmSnapshot.testSnapDiffNoSnapshot Time elapsed: 1.067 s
TestOmSnapshot.testSnapDiffWithKeyOverwrite Time elapsed: 3.148 s
TestOmSnapshot.testDeleteSnapshotMissingMandatoryParams Time elapsed: 1.074 s
TestOmSnapshot.testSnapDiffMissingMandatoryParams Time elapsed: 2.098 s
TestOmSnapshot.testSnapshotOpensWithDisabledAutoCompaction Time elapsed: 1.108
s
{code}
Single slowest test case is {{testDayWeekMonthSnapshotCreationAndExpiration}}
(added in HDDS-8378), takes 130s in each 4 parameterized runs. Disabling it
saves ~8 minutes.
> Speed up TestOmSnapshot
> -----------------------
>
> Key: HDDS-9299
> URL: https://issues.apache.org/jira/browse/HDDS-9299
> Project: Apache Ozone
> Issue Type: Task
> Components: test
> Affects Versions: 1.4.0
> Reporter: Attila Doroszlai
> Assignee: Swaminathan Balachandran
> Priority: Critical
>
> HDDS-8378 and HDDS-7798 increased the time required for {{TestOmSnapshot}}
> around 4x:
> {code:title=before HDDS-8378}
> Tests run: 75, Failures: 0, Errors: 0, Skipped: 8, Time elapsed: 282.134 s -
> in org.apache.hadoop.ozone.om.TestOmSnapshot
> {code}
> {code:title=before HDDS-7798}
> Tests run: 78, Failures: 0, Errors: 0, Skipped: 5, Time elapsed: 611.18 s -
> in org.apache.hadoop.ozone.om.TestOmSnapshot
> {code}
> {code:title=now}
> Tests run: 156, Failures: 0, Errors: 0, Skipped: 9, Time elapsed: 1,038.297 s
> - in org.apache.hadoop.ozone.om.TestOmSnapshot
> {code}
> CC [~smeng], [~dteng]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]