[
https://issues.apache.org/jira/browse/HDFS-13488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16457493#comment-16457493
]
Yiqun Lin commented on HDFS-13488:
----------------------------------
Only some comments for the UT:
* line54. Use \{{ {@link RBFConfigKeys#DFS_ROUTER_CLIENT_REJECT_OVERLOAD}}} to
replace {{DFS_ROUTER_CLIENT_REJECT_OVERLOAD}}.
* line59. LOG instance got incorrectly.
* line130. The test will be failed sometimes in this line. Stack info got from
my local:
{noformat}
java.lang.AssertionError: expected:<0> but was:<2>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at org.junit.Assert.assertEquals(Assert.java:542)
at
org.apache.hadoop.hdfs.server.federation.router.TestRouterClientRejectOverload.testOverloaded(TestRouterClientRejectOverload.java:231)
at
org.apache.hadoop.hdfs.server.federation.router.TestRouterClientRejectOverload.testOverloaded(TestRouterClientRejectOverload.java:168)
at
org.apache.hadoop.hdfs.server.federation.router.TestRouterClientRejectOverload.testOverloaded(TestRouterClientRejectOverload.java:160)
at
org.apache.hadoop.hdfs.server.federation.router.TestRouterClientRejectOverload.testOverloadControl(TestRouterClientRejectOverload.java:130)
{noformat}
Even though we don't simulate the subcluster, there will be also a chance the
minicluster is busy and lead the Router overloaded. Maybe we should use the
client thread number {{4}} as the requests number. {{testOverloaded(0, 0,
address, clientConf, 4);}} At least, we can ensure the Router won't be
overloaded if the thread number is enough.
* line140: This place also have a chance being failed. The failure:
{noformat}
java.lang.AssertionError: Expected <=4 but was 6
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at
org.apache.hadoop.hdfs.server.federation.router.TestRouterClientRejectOverload.testOverloaded(TestRouterClientRejectOverload.java:237)
at
org.apache.hadoop.hdfs.server.federation.router.TestRouterClientRejectOverload.testOverloaded(TestRouterClientRejectOverload.java:169)
at
org.apache.hadoop.hdfs.server.federation.router.TestRouterClientRejectOverload.testOverloadControl(TestRouterClientRejectOverload.java:140)
{noformat}
If the minicluster is running very slowly, all the subsequent requests (10-4=6)
cannot be handled. So {{testOverloaded(4, 6);}} will be better here.
* line172: {{submitting 10 requests}} should be updated.
> RBF: Reject requests when a Router is overloaded
> ------------------------------------------------
>
> Key: HDFS-13488
> URL: https://issues.apache.org/jira/browse/HDFS-13488
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Íñigo Goiri
> Assignee: Íñigo Goiri
> Priority: Major
> Attachments: HDFS-13488.000.patch, HDFS-13488.001.patch,
> HDFS-13488.002.patch, HDFS-13488.003.patch
>
>
> A Router might be overloaded when handling special cases (e.g. a slow
> subcluster). The Router could reject the requests and the client could try
> with another Router. We should leverage the Standby mechanism for this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
