[ https://issues.apache.org/jira/browse/HDFS-13528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16465381#comment-16465381 ]
Yiqun Lin edited comment on HDFS-13528 at 5/7/18 3:04 AM: ---------------------------------------------------------- It's a good catch! I can reproduce this case {noformat} 2018-05-07 10:48:32,075 [org.apache.hadoop.hdfs.server.federation.router.RouterQuotaUpdateService-0] ERROR router.RouterQuotaUpdateService (RouterQuotaUpdateService.java:periodicInvoke(115)) - Quota cache updated error. org.apache.hadoop.hdfs.protocol.NSQuotaExceededException: The NameSpace quota (directories and files) is exceeded: quota=3 file count=4 at org.apache.hadoop.hdfs.server.federation.router.RouterQuotaUsage.verifyNamespaceQuota(RouterQuotaUsage.java:79) at org.apache.hadoop.hdfs.server.federation.router.RouterRpcServer.getLocationsForPath(RouterRpcServer.java:2271) at org.apache.hadoop.hdfs.server.federation.router.RouterRpcServer.getLocationsForPath(RouterRpcServer.java:2233) at org.apache.hadoop.hdfs.server.federation.router.Quota.getQuotaRemoteLocations(Quota.java:202) at org.apache.hadoop.hdfs.server.federation.router.Quota.getValidQuotaLocations(Quota.java:113) at org.apache.hadoop.hdfs.server.federation.router.Quota.getQuotaUsage(Quota.java:94) at org.apache.hadoop.hdfs.server.federation.router.RouterQuotaUpdateService.periodicInvoke(RouterQuotaUpdateService.java:88) at org.apache.hadoop.hdfs.server.federation.router.PeriodicService$1.run(PeriodicService.java:178) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at {noformat} [~dibyendu_hadoop], I'm +1 for your fix way, but have another two comments: * We can do the additional try-catch for mount table update behavior in loop, so that we can avoid potential exception. At least, one mount table updated error, won't affect others. * It's would be good to add a unit test in {{TestRouterQuota}}. was (Author: linyiqun): It's a good catch! I can reproduce this case {noformat} 2018-05-07 10:48:32,075 [org.apache.hadoop.hdfs.server.federation.router.RouterQuotaUpdateService-0] ERROR router.RouterQuotaUpdateService (RouterQuotaUpdateService.java:periodicInvoke(115)) - Quota cache updated error. org.apache.hadoop.hdfs.protocol.NSQuotaExceededException: The NameSpace quota (directories and files) is exceeded: quota=3 file count=4 at org.apache.hadoop.hdfs.server.federation.router.RouterQuotaUsage.verifyNamespaceQuota(RouterQuotaUsage.java:79) at org.apache.hadoop.hdfs.server.federation.router.RouterRpcServer.getLocationsForPath(RouterRpcServer.java:2271) at org.apache.hadoop.hdfs.server.federation.router.RouterRpcServer.getLocationsForPath(RouterRpcServer.java:2233) at org.apache.hadoop.hdfs.server.federation.router.Quota.getQuotaRemoteLocations(Quota.java:202) at org.apache.hadoop.hdfs.server.federation.router.Quota.getValidQuotaLocations(Quota.java:113) at org.apache.hadoop.hdfs.server.federation.router.Quota.getQuotaUsage(Quota.java:94) at org.apache.hadoop.hdfs.server.federation.router.RouterQuotaUpdateService.periodicInvoke(RouterQuotaUpdateService.java:88) at org.apache.hadoop.hdfs.server.federation.router.PeriodicService$1.run(PeriodicService.java:178) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at {noformat} [~dibyendu_hadoop], I'm +1 for your fix way, but have another two comments: * We can do the additional try-catch for mount table update behavior in loop, so that we can avoid potential exception. * It's would be good to add a unit test in {{TestRouterQuota}}. > RBF: If a directory exceeds quota limit then quota usage is not refreshed for > other mount entries > -------------------------------------------------------------------------------------------------- > > Key: HDFS-13528 > URL: https://issues.apache.org/jira/browse/HDFS-13528 > Project: Hadoop HDFS > Issue Type: Sub-task > Reporter: Dibyendu Karmakar > Assignee: Dibyendu Karmakar > Priority: Major > > If quota limit is exceeded, RouterQuotaUpdateService#periodicInvoke is > getting QuotaExceededException and it is not updating the quota usage for > rest of the mount table entries. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org