[
https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16208089#comment-16208089
]
Kuhu Shukla commented on YARN-7244:
-----------------------------------
Thank you [~sunilg] for the comments!
bq. Do you think is it better to have a setter and update
AuxiliaryLocalPathHandler to AuxServices rather than changing AuxServices ctor.
The constructor change makes sure that we always initialize the pathHandler
which seems safer to me.
bq. AuxiliaryLocalPathHandler could be in org.apache.hadoop.yarn.server.api?
any reasons to move to api?
You are right. This needs to be in server apis.
bq. All apis in AuxiliaryLocalPathHandlerImpl could have Override annotation.
Will do.
bq. Does ContainerManagerImpl need to have a getAuxiliaryLocalPathHandler ?
I added the getter to assist any testing in the futur and made it package
private. I can mark it as VisibleForTesting or take it out , either way would
be fine.
> ShuffleHandler is not aware of disks that are added
> ---------------------------------------------------
>
> Key: YARN-7244
> URL: https://issues.apache.org/jira/browse/YARN-7244
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Kuhu Shukla
> Assignee: Kuhu Shukla
> Attachments: YARN-7244.001.patch, YARN-7244.002.patch,
> YARN-7244.003.patch, YARN-7244.004.patch, YARN-7244.005.patch,
> YARN-7244.006.patch, YARN-7244.007.patch, YARN-7244.008.patch,
> YARN-7244.009.patch, YARN-7244.010.patch
>
>
> The ShuffleHandler permanently remembers the list of "good" disks on NM
> startup. If disks later are added to the node then map tasks will start using
> them but the ShuffleHandler will not be aware of them. The end result is that
> the data cannot be shuffled from the node leading to fetch failures and
> re-runs of the map tasks.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
