[
https://issues.apache.org/jira/browse/HDFS-13528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16465381#comment-16465381
]
Yiqun Lin edited comment on HDFS-13528 at 5/7/18 3:04 AM:
----------------------------------------------------------
It's a good catch! I can reproduce this case
{noformat}
2018-05-07 10:48:32,075
[org.apache.hadoop.hdfs.server.federation.router.RouterQuotaUpdateService-0]
ERROR router.RouterQuotaUpdateService
(RouterQuotaUpdateService.java:periodicInvoke(115)) - Quota cache updated error.
org.apache.hadoop.hdfs.protocol.NSQuotaExceededException: The NameSpace quota
(directories and files) is exceeded: quota=3 file count=4
at
org.apache.hadoop.hdfs.server.federation.router.RouterQuotaUsage.verifyNamespaceQuota(RouterQuotaUsage.java:79)
at
org.apache.hadoop.hdfs.server.federation.router.RouterRpcServer.getLocationsForPath(RouterRpcServer.java:2271)
at
org.apache.hadoop.hdfs.server.federation.router.RouterRpcServer.getLocationsForPath(RouterRpcServer.java:2233)
at
org.apache.hadoop.hdfs.server.federation.router.Quota.getQuotaRemoteLocations(Quota.java:202)
at
org.apache.hadoop.hdfs.server.federation.router.Quota.getValidQuotaLocations(Quota.java:113)
at
org.apache.hadoop.hdfs.server.federation.router.Quota.getQuotaUsage(Quota.java:94)
at
org.apache.hadoop.hdfs.server.federation.router.RouterQuotaUpdateService.periodicInvoke(RouterQuotaUpdateService.java:88)
at
org.apache.hadoop.hdfs.server.federation.router.PeriodicService$1.run(PeriodicService.java:178)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
{noformat}
[~dibyendu_hadoop], I'm +1 for your fix way, but have another two comments:
* We can do the additional try-catch for mount table update behavior in loop,
so that we can avoid potential exception. At least, one mount table updated
error, won't affect others.
* It's would be good to add a unit test in {{TestRouterQuota}}.
was (Author: linyiqun):
It's a good catch! I can reproduce this case
{noformat}
2018-05-07 10:48:32,075
[org.apache.hadoop.hdfs.server.federation.router.RouterQuotaUpdateService-0]
ERROR router.RouterQuotaUpdateService
(RouterQuotaUpdateService.java:periodicInvoke(115)) - Quota cache updated error.
org.apache.hadoop.hdfs.protocol.NSQuotaExceededException: The NameSpace quota
(directories and files) is exceeded: quota=3 file count=4
at
org.apache.hadoop.hdfs.server.federation.router.RouterQuotaUsage.verifyNamespaceQuota(RouterQuotaUsage.java:79)
at
org.apache.hadoop.hdfs.server.federation.router.RouterRpcServer.getLocationsForPath(RouterRpcServer.java:2271)
at
org.apache.hadoop.hdfs.server.federation.router.RouterRpcServer.getLocationsForPath(RouterRpcServer.java:2233)
at
org.apache.hadoop.hdfs.server.federation.router.Quota.getQuotaRemoteLocations(Quota.java:202)
at
org.apache.hadoop.hdfs.server.federation.router.Quota.getValidQuotaLocations(Quota.java:113)
at
org.apache.hadoop.hdfs.server.federation.router.Quota.getQuotaUsage(Quota.java:94)
at
org.apache.hadoop.hdfs.server.federation.router.RouterQuotaUpdateService.periodicInvoke(RouterQuotaUpdateService.java:88)
at
org.apache.hadoop.hdfs.server.federation.router.PeriodicService$1.run(PeriodicService.java:178)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
{noformat}
[~dibyendu_hadoop], I'm +1 for your fix way, but have another two comments:
* We can do the additional try-catch for mount table update behavior in loop,
so that we can avoid potential exception.
* It's would be good to add a unit test in {{TestRouterQuota}}.
> RBF: If a directory exceeds quota limit then quota usage is not refreshed for
> other mount entries
> --------------------------------------------------------------------------------------------------
>
> Key: HDFS-13528
> URL: https://issues.apache.org/jira/browse/HDFS-13528
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Dibyendu Karmakar
> Assignee: Dibyendu Karmakar
> Priority: Major
>
> If quota limit is exceeded, RouterQuotaUpdateService#periodicInvoke is
> getting QuotaExceededException and it is not updating the quota usage for
> rest of the mount table entries.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
