[
https://issues.apache.org/jira/browse/HDFS-11065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15623700#comment-15623700
]
Mingliang Liu commented on HDFS-11065:
--------------------------------------
The patch looks good to me overall. +1 pending on Jenkins after the following
comments are addressed/considered:
# The code can be simplified.
{code}
598 } catch (Throwable t) {
599 LOG.info("Got expected exception ", t);
600 assertTrue(t instanceof QuotaByStorageTypeExceededException);
601 assertThat(t.toString(),
602 is(allOf(containsString("Quota by storage type"),
603 containsString("DISK on path"),
604 containsString(testDir.toString()))));
{code}
We can only catch the {{QuotaByStorageTypeExceededException}} exception, log
the exception and assert its message. All other exceptions will, by default,
make the test fail. Or else the log message is confusing {{ LOG.info("Got
expected exception ", t);}} as other exceptions are not expected here; and we
will lose the real exception stack if the LOG level is WARN and above. The code
will be simpler and cleaner IHMO:
{code}
598 } catch (QuotaByStorageTypeExceededException e) {
599 LOG.info("Got expected exception ", e);
601 assertThat(e.toString(),
602 is(allOf(containsString("Quota by storage type"),
603 containsString("DISK on path"),
604 containsString(testDir.toString()))));
{code}
This also works for another test case.
# I know this is not related to your test, but the test case is calling
{{refreshClusterState()}} even it has independent cluster for each test case.
Should we reuse the cluster (and refresh if needed) across multiple test cases?
On my local test MBP, it took about 30 seconds to finish all the tests. We can
address this in separate JIRAs if it's complex to the code change.
> Add space quota tests for heterogenous storages
> -----------------------------------------------
>
> Key: HDFS-11065
> URL: https://issues.apache.org/jira/browse/HDFS-11065
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs
> Reporter: Xiaobing Zhou
> Assignee: Xiaobing Zhou
> Labels: fs, test
> Attachments: HDFS-11065.000.patch
>
>
> There are some cases of space quota that need to be tested for heterogenous
> storages. It's quite important to ensure space quota works well in modern
> clusters typically built out of diversified storage structures. This proposes
> adding new tests to cover the following scenarios.
> # Tests space quota for storage policy = WARM
> # Tests if changing replication factor results in copying file as quota
> doesn't exceed
> # Tests if clear quota per heterogenous storage doesnt result in clearing
> quota for another storage
> The following cases have been covered.
> # Tests space quota for storage policy = ALL_SSD, covered by
> TestQuotaByStorageType#testQuotaByStorageTypeWithFileCreateAllSSD
> # Tests if overall space quota exceeds even if particular storage space quota
> is available, covered by
> TestQuotaByStorageType#testQuotaByStorageTypeAndTraditionalQuotaException3
> # Tests spaceQuota for storage policy = Cold, covered by
> TestQuotaByStorageType#testQuotaByStorageTypeWithTraditionalQuota
> # Tests if quota exceeds for DISK storage even if overall space quota is
> available, covered by
> TestQuotaByStorageType#testQuotaByStorageTypeAndTraditionalQuotaException2
> # Tests space quota with append operation, covered by
> TestQuotaByStorageType#testQuotaByStorageTypeWithFileCreateAppend
> # Sanity Test : Checks if copy command fails if quota is exceeded, covered by
> TestQuotaByStorageType#testQuotaByStorageTypeParent*Child*
> # Tests space quota with remove operation, covered by
> TestQuotaByStorageType#testQuotaByStorageTypeWithFileCreateDelete
> # Tests space quota with Snapshot operation, covered by
> TestQuotaByStorageType#testQuotaByStorageTypeWithSnapshot
> # Tests space quota with truncate operation, coveved by
> TestQuotaByStorageType#testQuotaByStorageTypeWithFileCreateTruncate
> # Tests space quota remains valid even with Namenode restart, covered by
> TestQuotaByStorageType#testQuotaByStorageTypePersistenceInEditLog and
> TestQuotaByStorageType#testQuotaByStorageTypePersistenceInFsImage
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
