[
https://issues.apache.org/jira/browse/HDFS-14258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16762841#comment-16762841
]
BELUGA BEHR commented on HDFS-14258:
------------------------------------
[~elgoiri] I am flattered that you call my work "fancy." :) Which section of
code are you speaking to specifically?
My intent was to remove the looping sleep calls and to use the latest tools
(concurrent package) to do this in a more action-driven way, instead of
constant polling.
I updated the unit tests to allow for a PASS. There is a subtle difference
with this proposal over how the current implementation works and this is why
the unit tests didn't pass (and I accidentally left out the updated version,
sorry).
In the unlikely scenario that a configuration change is made in the middle of
operation (re-configuration), the current code in trunk will simply update a
"max thread" variable to the new cap and exit. All new threads will fail to
create if the cap is lowered, and eventually, running threads will complete and
the total number of threads will drop below the cap.
In this new proposed implementation, the thread changing the cap value will
block for all threads currently queued to do work, and then continue blocking
until enough running threads complete that the number of threads is below the
new cap. If the number of threads does not come under the cap within 30s, it
gives up and a warning is logged. I personally prefer this setup because it is
more opaque to the operator. If I set the maximum number of threads to X, but
then I still see X+n threads running, because they have not yet completed their
work, I am going to be confused as to the reason for this. With this proposed
implementation, an operator will see that applying the new cap was not
successful, the reason for the failure, and will know why there are currently
more than X threads running.
> Introduce Java Concurrent Package To DataXceiverServer Class
> ------------------------------------------------------------
>
> Key: HDFS-14258
> URL: https://issues.apache.org/jira/browse/HDFS-14258
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Affects Versions: 3.2.0
> Reporter: BELUGA BEHR
> Assignee: BELUGA BEHR
> Priority: Minor
> Attachments: HDFS-14258.1.patch, HDFS-14258.2.patch
>
>
> * Use Java concurrent package to replace current facilities in
> {{DataXceiverServer}}.
> * A little bit of extra clean up
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
