[
https://issues.apache.org/jira/browse/YARN-8071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16477863#comment-16477863
]
Jason Lowe commented on YARN-8071:
----------------------------------
Thanks for updating the patch!
bq. after further consideration, I put up a new patch that preserves the
current behavior. Not worth risking a compatibility issue for this.
Yeah, I agree. I tried following the discussion in MAPREDUCE-3068 where it
appears the behavior originated, and it's confusing. It sounded like at one
point the proposal was to override instead of append, but then it somehow
shifted into only appending. Anyway it's been this way since the beginning of
YARN, and this JIRA is really only about the ability to specify environment
variable settings as separate properties, not about changing the semantics of
how values are combined with pre-existing values. That can be tackled in a
separate JIRA.
Patch looks good overall. Curious, why was testPrependDistcache modified to no
longer pass an empty set? The modifications don't appear to be relevant or
necessary.
> Add ability to specify nodemanager environment variables individually
> ---------------------------------------------------------------------
>
> Key: YARN-8071
> URL: https://issues.apache.org/jira/browse/YARN-8071
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
> Affects Versions: 3.0.0
> Reporter: Jim Brennan
> Assignee: Jim Brennan
> Priority: Major
> Attachments: YARN-8071.001.patch, YARN-8071.002.patch,
> YARN-8071.003.patch
>
>
> YARN-6830 describes a problem where environment variables that contain commas
> cannot be specified via {{-Dmapreduce.map.env}}.
> For example:
> {{-Dmapreduce.map.env="MODE=bar,IMAGE_NAME=foo,MOUNTS=/tmp/foo,/tmp/bar"}}
> will set {{MOUNTS}} to {{/tmp/foo}}
> In that Jira, [~aw] suggested that we change the API to provide a way to
> specify environment variables individually, the same way that Spark does.
> {quote}Rather than fight with a regex why not redefine the API instead?
>
> -Dmapreduce.map.env.MODE=bar
> -Dmapreduce.map.env.IMAGE_NAME=foo
> -Dmapreduce.map.env.MOUNTS=/tmp/foo,/tmp/bar
> ...
> e.g, mapreduce.map.env.[foo]=bar gets turned into foo=bar
> This greatly simplifies the input validation needed and makes it clear what
> is actually being defined.
> {quote}
> The mapreduce properties were dealt with in [MAPREDUCE-7069]. This Jira will
> address the YARN properties.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
