[
https://issues.apache.org/jira/browse/HDFS-13935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhoushulin updated HDFS-13935:
------------------------------
Affects Version/s: 3.1.1
> The timeout used in “setReadTimeout()” should be configurable
> -------------------------------------------------------------
>
> Key: HDFS-13935
> URL: https://issues.apache.org/jira/browse/HDFS-13935
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 3.1.1
> Reporter: zhoushulin
> Priority: Major
>
> The timeout used in "setReadTimeout" in "Fetcher.java" and
> "JobEndNotifier.java" in MapReduce is configurable as follows:
> {code:title="/org/apache/hadoop/mapreduce/task/reduce/Fetcher.java"}
> class Fetcher<K,V> extends Thread {
> ...
> this.readTimeout =
> job.getInt(MRJobConfig.SHUFFLE_READ_TIMEOUT, DEFAULT_READ_TIMEOUT);
> ...
> private void setupShuffleConnection(String encHash) {
> // put url hash into http header
> connection.addRequestProperty(
> SecureShuffleUtils.HTTP_HEADER_URL_HASH, encHash);
> // set the read timeout
> connection.setReadTimeout(readTimeout);
> // put shuffle version into http header
> connection.addRequestProperty(ShuffleHeader.HTTP_HEADER_NAME,
> ShuffleHeader.DEFAULT_HTTP_HEADER_NAME);
> connection.addRequestProperty(ShuffleHeader.HTTP_HEADER_VERSION,
> ShuffleHeader.DEFAULT_HTTP_HEADER_VERSION);
> }
> ...
> }
> {code}
> {code:title="/org/apache/hadoop/mapreduce/v2/app/JobEndNotifier.java"}
> public class JobEndNotifier implements Configurable {
> ...
> timeout = conf.getInt(JobContext.MR_JOB_END_NOTIFICATION_TIMEOUT,
> JobContext.DEFAULT_MR_JOB_END_NOTIFICATION_TIMEOUT);
> ...
> protected boolean notifyURLOnce() {
> ...
> conn.setReadTimeout(timeout);
> ...
> }
> ...
> }
> {code}
> The similar situations happen in "DFSZKFailoverController.java" in HDFS.
> However, the "setReadTimeout()" in "FederationUtil.java",
> "CredentialBasedAccessTokenProvider.java", and
> "ConfRefreshTokenBasedAccessTokenProvider.java" call constants as their
> argument. For example:
> {code:title="/org/apache/hadoop/hdfs/server/federation/router/FederationUtil.java"}
> public final class FederationUtil {
> ...
> public static JSONArray getJmx(String beanQuery, String webAddress) {
> ...
> URLConnection conn = jmxURL.openConnection();
> conn.setConnectTimeout(5 * 1000);
> ...
> }
> ...
> }
> {code}
> I think these constant should be also configurable for different requirements
> and environments.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
