[
https://issues.apache.org/jira/browse/YARN-9699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16934948#comment-16934948
]
Peter Bacsko commented on YARN-9699:
------------------------------------
[~jiwq] [~Prabhu Joseph] [~shuzirra] [~sunilg]
I uploaded a POC that contains an idea about how to map an FS config to CS.
Basic idea is this:
- Re-use as much as possible from FS, because you get the parsed FS config &
queue structure for free
- Use {{AllocationConfiguration}} where necessary (ACLs, some defaults)
- Then walk the queue hieararchy recursively to generate CS config
There are feature gaps between the two schedulers, so not everything can be
converted in a straightforward manner. At various places, the code throws
exception. We can decide later whether we should display a warning instead
(this is probably what [~leftnoteasy] would prefer).
I also included a sample {{fair-scheduler.xml}}, the path is hard coded in the
main class. When you run it, a capacity-scheduler.xml and yarn-site.xml is
printed to the console. It already converts quite a few things.
As I said it's a POC - unfinished, no tests, TODOs. But it's good enough to
have an idea about the direction.
I would really appreciate some thoughts, feedbacks! Thanks.
> Migration tool that help to generate CS configs based on FS
> -----------------------------------------------------------
>
> Key: YARN-9699
> URL: https://issues.apache.org/jira/browse/YARN-9699
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Wanqiang Ji
> Assignee: Gergely Pollak
> Priority: Major
> Attachments: FS_to_CS_migration_POC.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
