[
https://issues.apache.org/jira/browse/HDFS-13873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16717750#comment-16717750
]
Chen Liang commented on HDFS-13873:
-----------------------------------
I guess there are (at least) two directions: estimate-based and timeout-based.
Timeout-based is simple: if a call has stuck in the queue for too long (> X
sec), reject it. The upside is simple, while the downside is that some resource
gets wasted: client wait has already happened, the server queue slot also has
been occupied for X sec.
Estimate-based is basically, upon seeing the request, Observer make a guess
whether this will wait for too long, if so, reject at this point of time.
Upside is that no wait or queuing, immediate rejection. Downside is that the
estimate better be correct and reasonable, and leaving no wholes for bad client
to leverage...
We can have some abstractions to allow different reject policies to be plugged
in, and we can even have a combination of both: on seeing the requests, make an
estimate, but even if the estimate is inaccurate and the requests pass, timeout
still makes sure that the requests won't stay in the queue indefinitely. For
now, we can have both Konstantin and Chao's strategies combined to start with.
> ObserverNode should reject read requests when it is too far behind.
> -------------------------------------------------------------------
>
> Key: HDFS-13873
> URL: https://issues.apache.org/jira/browse/HDFS-13873
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client, namenode
> Affects Versions: HDFS-12943
> Reporter: Konstantin Shvachko
> Assignee: Chao Sun
> Priority: Major
>
> Add a server-side threshold for ObserverNode to reject read requests when it
> is too far behind.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
