[
https://issues.apache.org/jira/browse/YARN-1666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13901142#comment-13901142
]
Xuan Gong commented on YARN-1666:
---------------------------------
bq. It doesn't seem to apply against latest trunk anymore. Please update.
updated
bq. Fix the file-name constants in YarnConfiguration to be consistent.
change YARN_SITE_XML_FILE to YARN_SITE_CONFIGURATION_FILE, and
YARN_DEFAULT_XML_FILE to YARN_DEFAULT_CONFIGURATION_FILE.
bq. FileSystemBasedConfigurationProvider.getConfiguration(): Let's always throw
exceptions instead of returning nulls in some cases.
Added.
bq. LocalConfigurationProvider: We need to first find the location of the XML
file in the classpath if it is one of RM_CONFIGURATION_FILES, right?
changed
bq. In AdminService, where you use new Configuration(), should you use new
Configuration(false)?
It is fine. Because we will load related configuration later by using
addResource(Configuration) which will reload and overwrite all the properties.
bq. I think we can simply get rid of the LocalConfigurationProvider instance
checks everywhere now.
Yes, we can do that.
bq. inFile -> includesFile and exFile -> excludesFile
DONE
bq. Make both the above as class-fields and use it in refreshNodes method too.
DONE
bq. disableHostsFileReader() should also use the remote-conf provider?
Yes, changed.
bq. HostsFileReader: Change the constructor to not need both the file-names as
well as the streams
I think that we still need it. In the constructor, we will call refresh API
which will print out which hosts will be included or excluded from which
include file or exclude file. In that case, we need give fileNames.
Also, I make several other changes:
* Move logic that loads CapacityScheduler.xml from AdminService#refreshQueues
to CapacityScheduler#reinitiate().
* Add empty yarn-site.xml and hadoop-policy.xml under
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources
to make tests pass. Because right now, LocalConfigurationProvider will load
those configuration file from classPath now.
* At NodeListManager, we will check whether filename is empty or null. If it
is, we will give a null as InputStream. Because both LocalConfigurationProvider
or FSBasedConfigurationProvider will throw an exception when they are trying to
getInputStream by giving a empty or null fileName. But at NodeListManager, it
is allowed to give such value for fileName, it will simply disabled the
HostsFileReader. So, before we actually create the inputstream, we'd better to
do such checks.
> Make admin refreshNodes work across RM failover
> -----------------------------------------------
>
> Key: YARN-1666
> URL: https://issues.apache.org/jira/browse/YARN-1666
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Xuan Gong
> Assignee: Xuan Gong
> Attachments: YARN-1666.1.patch, YARN-1666.2.patch, YARN-1666.2.patch,
> YARN-1666.3.patch, YARN-1666.4.patch, YARN-1666.4.patch, YARN-1666.5.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
