[jira] [Commented] (YARN-10420) Update CS MappingRule documentation with the new format and features
[ https://issues.apache.org/jira/browse/YARN-10420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17224086#comment-17224086 ] Szilard Nemeth commented on YARN-10420: --- Hi [~pbacsko], Thanks for working on this. Latest patch LGTM, committed to trunk. Thanks [~adam.antal] for the reviews. > Update CS MappingRule documentation with the new format and features > > > Key: YARN-10420 > URL: https://issues.apache.org/jira/browse/YARN-10420 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Gergely Pollak >Assignee: Peter Bacsko >Priority: Major > Attachments: YARN-10420-001.patch, YARN-10420-002.patch, > YARN-10420-003.patch, YARN-10420-004.patch, YARN-10420-005.patch, > YARN-10420-006.patch > > > Update the upstream documentation with the new changes. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10420) Update CS MappingRule documentation with the new format and features
[
https://issues.apache.org/jira/browse/YARN-10420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220764#comment-17220764
]
Hadoop QA commented on YARN-10420:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 28m
32s{color} | | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | | {color:green} No case conflicting files found. {color} |
| {color:blue}0{color} | {color:blue} markdownlint {color} | {color:blue} 0m
0s{color} | | {color:blue} markdownlint was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | | {color:green} The patch does not contain any @author tags.
{color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
53s{color} | | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
23s{color} | | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
36m 11s{color} | | {color:green} branch has no errors when building and
testing our client artifacts. {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
12s{color} | | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} blanks {color} | {color:green} 0m
0s{color} | | {color:green} The patch has no blanks issues. {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
19s{color} | | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 48s{color} | | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Other Tests {color} || ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s{color} | | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 80m 49s{color} |
| {color:black}{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/250/artifact/out/Dockerfile
|
| JIRA Issue | YARN-10420 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/13014131/YARN-10420-006.patch |
| Optional Tests | dupname asflicense mvnsite markdownlint |
| uname | Linux 01c669426150 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / 12c908c827c80ea37ed5207a65831d0b699c8381 |
| Max. process+thread count | 422 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site |
| Console output |
https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/250/console |
| versions | git=2.17.1 maven=3.6.0 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
> Update CS MappingRule documentation with the new format and features
>
>
> Key: YARN-10420
> URL: https://issues.apache.org/jira/browse/YARN-10420
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Gergely Pollak
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: YARN-10420-001.patch, YARN-10420-002.patch,
> YARN-10420-003.patch, YARN-10420-004.patch, YARN-10420-005.patch,
> YARN-10420-006.patch
>
>
> Update the upstream documentation with the new changes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10420) Update CS MappingRule documentation with the new format and features
[ https://issues.apache.org/jira/browse/YARN-10420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220722#comment-17220722 ] Peter Bacsko commented on YARN-10420: - [~snemeth] could you review patch v6? Thanks. > Update CS MappingRule documentation with the new format and features > > > Key: YARN-10420 > URL: https://issues.apache.org/jira/browse/YARN-10420 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Gergely Pollak >Assignee: Peter Bacsko >Priority: Major > Attachments: YARN-10420-001.patch, YARN-10420-002.patch, > YARN-10420-003.patch, YARN-10420-004.patch, YARN-10420-005.patch, > YARN-10420-006.patch > > > Update the upstream documentation with the new changes. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10420) Update CS MappingRule documentation with the new format and features
[ https://issues.apache.org/jira/browse/YARN-10420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213142#comment-17213142 ] Szilard Nemeth commented on YARN-10420: --- Hi [~pbacsko] , Thumbs up for this patch, very thorough and precise, straightforward documentation. Especially liked the part with "Converting the old mapping rule format to the new one". Adding some comments over the others you already received: 1. It can be adjusted more flexibly compared to the legacy mapping rule evaluator what happens when the evaluation stops and a given rule doesn't match. Changing this to: "Compared to the legacy mapping rule evaluator, what exactly happens when the evaluation stops and a given rule doesn't match can be adjusted more flexibly." sound more natural to me, but I'm not a native speaker of course. 2. Nit: "Doesn't" and "Does not" are both used in your descriptions. I would rewrite all to the latter. 3. Nit: Many of them can be expressed as a "custom" placement policy as you will soon see This might be better: "Many of them can be expressed as a "custom" placement policy as you will see soon." 4. Two typos for search string: "Places the application into the queue hiearchy" Hierarchy is mistyped here. 5. Nit: "because you can also append or preprend extra string" --> Could be changed to: "because you can also append or preprend an extra string" (Note the "an"). 6. Typo in search string: "If the default placemene fails", (typo in word 'placement') > Update CS MappingRule documentation with the new format and features > > > Key: YARN-10420 > URL: https://issues.apache.org/jira/browse/YARN-10420 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Gergely Pollak >Assignee: Peter Bacsko >Priority: Major > Attachments: YARN-10420-001.patch, YARN-10420-002.patch, > YARN-10420-003.patch, YARN-10420-004.patch, YARN-10420-005.patch > > > Update the upstream documentation with the new changes. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10420) Update CS MappingRule documentation with the new format and features
[ https://issues.apache.org/jira/browse/YARN-10420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17212411#comment-17212411 ] Adam Antal commented on YARN-10420: --- Thanks for the patch [~pbacsko]. I'll attach my reply inline. 1. Ok, let's not touch it then. 2. Can we check what happens, and document it as well? I think users would be also interested in that. 3. Can we also add this to the document? 4,5,6. Ok, got it, thanks. bq. "If the target queue doesn't exist or and it cannot be created..." - you propose "and" but that would mean that we always try to create a non-existing queue, which is not the case in CS. Under regular parents, queues cannot be created dynamically and CS doesn't even try. Therefore "or" is more appropriate here. Thanks for the clarification. I suggest to add this to the doc, because I didn't know that "cannot be created" is what you've illustrated as an example. Something like "If the target queue doesn't exist or cannot be created (e.g. under regular parents) ..." For all the other points, I'm fine. > Update CS MappingRule documentation with the new format and features > > > Key: YARN-10420 > URL: https://issues.apache.org/jira/browse/YARN-10420 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Gergely Pollak >Assignee: Peter Bacsko >Priority: Major > Attachments: YARN-10420-001.patch, YARN-10420-002.patch, > YARN-10420-003.patch, YARN-10420-004.patch, YARN-10420-005.patch > > > Update the upstream documentation with the new changes. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10420) Update CS MappingRule documentation with the new format and features
[
https://issues.apache.org/jira/browse/YARN-10420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211051#comment-17211051
]
Hadoop QA commented on YARN-10420:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
17s{color} | | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | | {color:green} No case conflicting files found. {color} |
| {color:blue}0{color} | {color:blue} markdownlint {color} | {color:blue} 0m
0s{color} | | {color:blue} markdownlint was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | | {color:green} The patch does not contain any @author tags.
{color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m
39s{color} | | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
22s{color} | | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
37m 37s{color} | | {color:green} branch has no errors when building and
testing our client artifacts. {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
12s{color} | | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} blanks {color} | {color:green} 0m
0s{color} | | {color:green} The patch has no blanks issues. {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
16s{color} | | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 45s{color} | | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Other Tests {color} || ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 55m 52s{color} |
| {color:black}{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/224/artifact/out/Dockerfile
|
| JIRA Issue | YARN-10420 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/13013318/YARN-10420-005.patch |
| Optional Tests | dupname asflicense mvnsite markdownlint |
| uname | Linux dfadc07df735 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / 518a212cfffdfb7baabd16e2b69db292fdd15dbc |
| Max. process+thread count | 341 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site |
| Console output |
https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/224/console |
| versions | git=2.17.1 maven=3.6.0 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
> Update CS MappingRule documentation with the new format and features
>
>
> Key: YARN-10420
> URL: https://issues.apache.org/jira/browse/YARN-10420
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Gergely Pollak
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: YARN-10420-001.patch, YARN-10420-002.patch,
> YARN-10420-003.patch, YARN-10420-004.patch, YARN-10420-005.patch
>
>
> Update the upstream documentation with the new changes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10420) Update CS MappingRule documentation with the new format and features
[
https://issues.apache.org/jira/browse/YARN-10420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209438#comment-17209438
]
Hadoop QA commented on YARN-10420:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
41s{color} | | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | | {color:green} No case conflicting files found. {color} |
| {color:blue}0{color} | {color:blue} markdownlint {color} | {color:blue} 0m
0s{color} | | {color:blue} markdownlint was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | | {color:green} The patch does not contain any @author tags.
{color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 30m
55s{color} | | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
28s{color} | | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
49m 28s{color} | | {color:green} branch has no errors when building and
testing our client artifacts. {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
15s{color} | | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
23s{color} | | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
17m 44s{color} | | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Other Tests {color} || ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
40s{color} | | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 70m 55s{color} |
| {color:black}{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/216/artifact/out/Dockerfile
|
| JIRA Issue | YARN-10420 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/13013181/YARN-10420-004.patch |
| Optional Tests | dupname asflicense mvnsite markdownlint |
| uname | Linux ada8bdb7db7e 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / 16aea11c945c84936984e80241dcdd4a0d4b7f58 |
| Max. process+thread count | 308 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site |
| Console output |
https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/216/console |
| versions | git=2.17.1 maven=3.6.0 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
> Update CS MappingRule documentation with the new format and features
>
>
> Key: YARN-10420
> URL: https://issues.apache.org/jira/browse/YARN-10420
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Gergely Pollak
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: YARN-10420-001.patch, YARN-10420-002.patch,
> YARN-10420-003.patch, YARN-10420-004.patch
>
>
> Update the upstream documentation with the new changes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10420) Update CS MappingRule documentation with the new format and features
[
https://issues.apache.org/jira/browse/YARN-10420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209437#comment-17209437
]
Hadoop QA commented on YARN-10420:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
25s{color} | | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | | {color:green} No case conflicting files found. {color} |
| {color:blue}0{color} | {color:blue} markdownlint {color} | {color:blue} 0m
0s{color} | | {color:blue} markdownlint was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | | {color:green} The patch does not contain any @author tags.
{color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 32m
16s{color} | | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
28s{color} | | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
50m 43s{color} | | {color:green} branch has no errors when building and
testing our client artifacts. {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
15s{color} | | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
23s{color} | | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
18m 25s{color} | | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Other Tests {color} || ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
41s{color} | | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 72m 40s{color} |
| {color:black}{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/215/artifact/out/Dockerfile
|
| JIRA Issue | YARN-10420 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/13013179/YARN-10420-004.patch |
| Optional Tests | dupname asflicense mvnsite markdownlint |
| uname | Linux 344aeb6e73f0 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / 16aea11c945c84936984e80241dcdd4a0d4b7f58 |
| Max. process+thread count | 309 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site |
| Console output |
https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/215/console |
| versions | git=2.17.1 maven=3.6.0 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
> Update CS MappingRule documentation with the new format and features
>
>
> Key: YARN-10420
> URL: https://issues.apache.org/jira/browse/YARN-10420
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Gergely Pollak
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: YARN-10420-001.patch, YARN-10420-002.patch,
> YARN-10420-003.patch, YARN-10420-004.patch
>
>
> Update the upstream documentation with the new changes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10420) Update CS MappingRule documentation with the new format and features
[ https://issues.apache.org/jira/browse/YARN-10420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209398#comment-17209398 ] Peter Bacsko commented on YARN-10420: - I accidentally removed patch v3, so re-uploaded it, together with the newest version, patch v4. > Update CS MappingRule documentation with the new format and features > > > Key: YARN-10420 > URL: https://issues.apache.org/jira/browse/YARN-10420 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Gergely Pollak >Assignee: Peter Bacsko >Priority: Major > Attachments: YARN-10420-001.patch, YARN-10420-002.patch, > YARN-10420-003.patch, YARN-10420-004.patch > > > Update the upstream documentation with the new changes. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10420) Update CS MappingRule documentation with the new format and features
[ https://issues.apache.org/jira/browse/YARN-10420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17208834#comment-17208834 ] Peter Bacsko commented on YARN-10420: - BTW if the sentence is too long & misleading, prposals are welcome to make it easier to comprehend. > Update CS MappingRule documentation with the new format and features > > > Key: YARN-10420 > URL: https://issues.apache.org/jira/browse/YARN-10420 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Gergely Pollak >Assignee: Peter Bacsko >Priority: Major > Attachments: YARN-10420-001.patch, YARN-10420-002.patch, > YARN-10420-003.patch > > > Update the upstream documentation with the new changes. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10420) Update CS MappingRule documentation with the new format and features
[
https://issues.apache.org/jira/browse/YARN-10420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17208783#comment-17208783
]
Peter Bacsko commented on YARN-10420:
-
Some more:
"If the target queue doesn't exist or and it cannot be created..." - you
propose "and" but that would mean that we always try to create a non-existing
queue, which is not the case in CS. Under regular parents, queues cannot be
created dynamically and CS doesn't even try. Therefore "or" is more appropriate
here.
""It can be a managed parent in order to have userName leaf created
automatically, but..." - you propose "and" but this does not make sense. The
point is that regardless of the type of the parent, it has to exist ie. it has
to be defined statically in \{{capacity-scheduler.xml}}.
""can" should be "shouldn't?"" -- no, "can" is the proper word here.
> Update CS MappingRule documentation with the new format and features
>
>
> Key: YARN-10420
> URL: https://issues.apache.org/jira/browse/YARN-10420
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Gergely Pollak
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: YARN-10420-001.patch, YARN-10420-002.patch,
> YARN-10420-003.patch
>
>
> Update the upstream documentation with the new changes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10420) Update CS MappingRule documentation with the new format and features
[ https://issues.apache.org/jira/browse/YARN-10420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17208721#comment-17208721 ] Peter Bacsko commented on YARN-10420: - [~adam.antal] some answers: 1. The "application lifetime" part already existed before, I just moved it. Don't really want to touch that part. 2. "Will the rule fails if the create flag is set for a non-managed queue? What is the expected behaviour? " -> I don't think anything happens. Maybe a warning message, but I'm not sure. 3. "Are these mapping rules FS or CS specific?" - the phrase "mapping rule" is CS specific. In Fair Scheduler, it's called "placement rules". From the context and the table below, this should be fairly obvious to the reader. 4. I'll try to incorporate your suggestions regarding the wording. 5. "The asterisk is not displayed correctly on markdown, but is generated correctly by mvn site". We might try to use the HTML escape code, I'll check this out. > Update CS MappingRule documentation with the new format and features > > > Key: YARN-10420 > URL: https://issues.apache.org/jira/browse/YARN-10420 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Gergely Pollak >Assignee: Peter Bacsko >Priority: Major > Attachments: YARN-10420-001.patch, YARN-10420-002.patch, > YARN-10420-003.patch > > > Update the upstream documentation with the new changes. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10420) Update CS MappingRule documentation with the new format and features
[
https://issues.apache.org/jira/browse/YARN-10420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17208690#comment-17208690
]
Adam Antal commented on YARN-10420:
---
Thanks for the patch [~pbacsko]. Awesome patch, thanks for the effort on this!
If you don't mind I take the initiative to read through the doc, as I wasn't
much involved in the process and I checked whether I could comprehend it.
Some questions:
- It's a bit strange that the two configuration options in the Queue lifetime
for applications chapter are different:
one is {{yarn.scheduler.capacity..maximum-application-lifetime}}
and the other is
{{yarn.scheduler.capacity.root..default-application-lifetime}}.
Is there any specific reason why one is started with root and the other is
not? I think it may be a mistake as in the doc it says "This feature can be set
at any level in the queue hierarchy.".
- Will the rule fails if the create flag is set for a non-managed queue? What
is the expected behaviour? (Maybe we can add some details to the create flag
part)
- Can you use multiple {{setDefaultQueue}} rule? (Maybe we can add some
details to the doc.)
- "In this table, you can see how to rewrite the old, colon-separated rules to
the new format." <-Are these mapping rules FS or CS specific? Can we add that?
Nits:
- You can use the {{```json}} syntax for the block codes to have json syntax
highlight (it is used for xml in this page).
- The Example does not show a rule where the "create" flag is set (either to
true or false). I think we should add one for the sake of completeness.
Other nits with inlined suggestions:
- "The {{CapacityScheduler}} supports the following parameters to _configure_
lifetime of an application"
- "When the evaluation stops and what happens if a rule doesn't match can be
adjusted more flexibly compared to the legacy mapping rule evaluator." -> "It
can be adjusted more flexibly compared to the legacy mapping rule evaluator
what happens when the evaluation stops and a given rule doesn't match."
- "In case of user, primaryGroup, primaryGroupUser, secondaryGroup,
secondaryGroupUser -,- _policies_ this tells the engine where the matching
queue should be looked for."
- "If the target queue doesn't exist -or- _and_ it cannot be created, it
defines a fallback action. Valid values are skip, reject and placeDefault."
- "placeDefault: place the application to the default queue -"- _`_
root.default _`_ -"- (unless it's overridden to something else)." <- Use `
instead of "
- "Places the application into the default queue root.default -but this can be
changed.- _or its overwritten value._"
- "It can be a managed parent in order to have userName leaf created
automatically, -but- _and_ ..." <- "can" should be "shouldn't"?
- "The custom placement policy can describe other policies with the appropriate
variable placeholders _(see below)_ ."
Note:
- The asterisk is not displayed correctly on markdown, but is generated
correctly by {{mvn site}}. I wonder if we can do something about it. I know
that is common to link the markdown from the GH repository instead of using the
generated documentation from the website, so it would be nice if the markdown
could display this correctly.
If you anything is not clear, please let me know and I try to clarify what I
didn't understand.
> Update CS MappingRule documentation with the new format and features
>
>
> Key: YARN-10420
> URL: https://issues.apache.org/jira/browse/YARN-10420
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Gergely Pollak
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: YARN-10420-001.patch, YARN-10420-002.patch,
> YARN-10420-003.patch
>
>
> Update the upstream documentation with the new changes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10420) Update CS MappingRule documentation with the new format and features
[
https://issues.apache.org/jira/browse/YARN-10420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17208097#comment-17208097
]
Hadoop QA commented on YARN-10420:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
27s{color} | | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | | {color:green} No case conflicting files found. {color} |
| {color:blue}0{color} | {color:blue} markdownlint {color} | {color:blue} 0m
0s{color} | | {color:blue} markdownlint was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | | {color:green} The patch does not contain any @author tags.
{color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 30m
44s{color} | | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
28s{color} | | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
50m 20s{color} | | {color:green} branch has no errors when building and
testing our client artifacts. {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
16s{color} | | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
26s{color} | | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
19m 6s{color} | | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Other Tests {color} || ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
41s{color} | | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 73m 1s{color} |
| {color:black}{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/211/artifact/out/Dockerfile
|
| JIRA Issue | YARN-10420 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/13012993/YARN-10420-003.patch |
| Optional Tests | dupname asflicense mvnsite markdownlint |
| uname | Linux 944f647ddfd6 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / 43b0c0b0546d79a1ab4df5fe02310d007f6bc178 |
| Max. process+thread count | 340 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site |
| Console output |
https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/211/console |
| versions | git=2.17.1 maven=3.6.0 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
> Update CS MappingRule documentation with the new format and features
>
>
> Key: YARN-10420
> URL: https://issues.apache.org/jira/browse/YARN-10420
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Gergely Pollak
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: YARN-10420-001.patch, YARN-10420-002.patch,
> YARN-10420-003.patch
>
>
> Update the upstream documentation with the new changes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10420) Update CS MappingRule documentation with the new format and features
[
https://issues.apache.org/jira/browse/YARN-10420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17208033#comment-17208033
]
Hadoop QA commented on YARN-10420:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 28m
25s{color} | | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | | {color:green} No case conflicting files found. {color} |
| {color:blue}0{color} | {color:blue} markdownlint {color} | {color:blue} 0m
0s{color} | | {color:blue} markdownlint was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | | {color:green} The patch does not contain any @author tags.
{color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
34s{color} | | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
26s{color} | | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
33m 52s{color} | | {color:green} branch has no errors when building and
testing our client artifacts. {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
15s{color} | | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
18s{color} | | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} |
[/whitespace-eol.txt|https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/210/artifact/out/whitespace-eol.txt]
| {color:red} The patch has 1 line(s) that end in whitespace. Use git apply
--whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 47s{color} | | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Other Tests {color} || ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
34s{color} | | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 77m 53s{color} |
| {color:black}{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/210/artifact/out/Dockerfile
|
| JIRA Issue | YARN-10420 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/13012986/YARN-10420-002.patch |
| Optional Tests | dupname asflicense mvnsite markdownlint |
| uname | Linux c5173e910476 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / 7d4bcb312bd2052d4fba880c405fcae08361965d |
| Max. process+thread count | 421 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site |
| Console output |
https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/210/console |
| versions | git=2.17.1 maven=3.6.0 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
> Update CS MappingRule documentation with the new format and features
>
>
> Key: YARN-10420
> URL: https://issues.apache.org/jira/browse/YARN-10420
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Gergely Pollak
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: YARN-10420-001.patch, YARN-10420-002.patch
>
>
> Update the upstream documentation with the new changes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
