[jira] [Commented] (YARN-10687) Add option to disable/enable free disk space checking and percentage checking for full and not-full disks
[ https://issues.apache.org/jira/browse/YARN-10687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17300390#comment-17300390 ] Qi Zhu commented on YARN-10687: --- Thanks [~Jim_Brennan] for last check. > Add option to disable/enable free disk space checking and percentage checking > for full and not-full disks > - > > Key: YARN-10687 > URL: https://issues.apache.org/jira/browse/YARN-10687 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 3.2.2, 3.4.0 >Reporter: Qi Zhu >Assignee: Qi Zhu >Priority: Major > Attachments: YARN-10687.001.patch, YARN-10687.002.patch, > YARN-10687.003.patch, YARN-10687.004.patch > > > Now the two option: > max-disk-utilization-per-disk-percentage > min-free-space-per-disk-mb > for full/not full disk check are all enabled always, i think it's more > reasonable to enable or disable this, and default will be all enabled. > > In our clusters, when the disk is so huge we want to use > min-free-space-per-disk-mb. > In our clusters, when the disk is so small we want to use > max-disk-utilization-per-disk-percentage. > > We should make this more reasonable and not confused. > > cc [~pbacsko] [~Jim_Brennan] [~ebadger] [~gandras] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10687) Add option to disable/enable free disk space checking and percentage checking for full and not-full disks
[ https://issues.apache.org/jira/browse/YARN-10687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17300369#comment-17300369 ] Jim Brennan commented on YARN-10687: Thanks for the updates [~zhuqi]! I am +1 on patch 004. I will commit this later today. > Add option to disable/enable free disk space checking and percentage checking > for full and not-full disks > - > > Key: YARN-10687 > URL: https://issues.apache.org/jira/browse/YARN-10687 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 3.2.2, 3.4.0 >Reporter: Qi Zhu >Assignee: Qi Zhu >Priority: Major > Attachments: YARN-10687.001.patch, YARN-10687.002.patch, > YARN-10687.003.patch, YARN-10687.004.patch > > > Now the two option: > max-disk-utilization-per-disk-percentage > min-free-space-per-disk-mb > for full/not full disk check are all enabled always, i think it's more > reasonable to enable or disable this, and default will be all enabled. > > In our clusters, when the disk is so huge we want to use > min-free-space-per-disk-mb. > In our clusters, when the disk is so small we want to use > max-disk-utilization-per-disk-percentage. > > We should make this more reasonable and not confused. > > cc [~pbacsko] [~Jim_Brennan] [~ebadger] [~gandras] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10687) Add option to disable/enable free disk space checking and percentage checking for full and not-full disks
[
https://issues.apache.org/jira/browse/YARN-10687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17300240#comment-17300240
]
Hadoop QA commented on YARN-10687:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
41s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} No case conflicting files
found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch does not contain any
@author tags. {color} |
| {color:green}+1{color} | {color:green} {color} | {color:green} 0m 0s{color}
| {color:green}test4tests{color} | {color:green} The patch appears to include 1
new or modified test files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
43s{color} | {color:blue}{color} | {color:blue} Maven dependency ordering for
branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
23s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
6s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
50s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
46s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
54s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
18m 2s{color} | {color:green}{color} | {color:green} branch has no errors when
building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
30s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
41s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 28m
49s{color} | {color:blue}{color} | {color:blue} Both FindBugs and SpotBugs are
enabled, using SpotBugs. {color} |
| {color:green}+1{color} | {color:green} spotbugs {color} | {color:green} 5m
38s{color} | {color:green}{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue}{color} | {color:blue} Maven dependency ordering for
patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
54s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
28s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
28s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
54s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
54s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
45s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
36s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch has no whitespace
issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green}{color} | {color:green} The patch has no ill-formed
XML file. {color} |
| {color:green}+1{color} | {color:gree
[jira] [Commented] (YARN-10687) Add option to disable/enable free disk space checking and percentage checking for full and not-full disks
[ https://issues.apache.org/jira/browse/YARN-10687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17300155#comment-17300155 ] Qi Zhu commented on YARN-10687: --- Fixed the remaining checkstyle in latest patch. > Add option to disable/enable free disk space checking and percentage checking > for full and not-full disks > - > > Key: YARN-10687 > URL: https://issues.apache.org/jira/browse/YARN-10687 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 3.2.2, 3.4.0 >Reporter: Qi Zhu >Assignee: Qi Zhu >Priority: Major > Attachments: YARN-10687.001.patch, YARN-10687.002.patch, > YARN-10687.003.patch, YARN-10687.004.patch > > > Now the two option: > max-disk-utilization-per-disk-percentage > min-free-space-per-disk-mb > for full/not full disk check are all enabled always, i think it's more > reasonable to enable or disable this, and default will be all enabled. > > In our clusters, when the disk is so huge we want to use > min-free-space-per-disk-mb. > In our clusters, when the disk is so small we want to use > max-disk-utilization-per-disk-percentage. > > We should make this more reasonable and not confused. > > cc [~pbacsko] [~Jim_Brennan] [~ebadger] [~gandras] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10687) Add option to disable/enable free disk space checking and percentage checking for full and not-full disks
[
https://issues.apache.org/jira/browse/YARN-10687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17300076#comment-17300076
]
Hadoop QA commented on YARN-10687:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
41s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} No case conflicting files
found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch does not contain any
@author tags. {color} |
| {color:green}+1{color} | {color:green} {color} | {color:green} 0m 0s{color}
| {color:green}test4tests{color} | {color:green} The patch appears to include 1
new or modified test files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
2s{color} | {color:blue}{color} | {color:blue} Maven dependency ordering for
branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m
10s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
6s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
47s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
45s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
54s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
20m 1s{color} | {color:green}{color} | {color:green} branch has no errors when
building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
28s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
38s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 31m
4s{color} | {color:blue}{color} | {color:blue} Both FindBugs and SpotBugs are
enabled, using SpotBugs. {color} |
| {color:green}+1{color} | {color:green} spotbugs {color} | {color:green} 6m
1s{color} | {color:green}{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue}{color} | {color:blue} Maven dependency ordering for
patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
2s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
54s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 9m
54s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
55s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
55s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 50s{color} |
{color:orange}https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/787/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt{color}
| {color:orange} hadoop-yarn-project/hadoop-yarn: The patch generated 2 new +
238 unchanged - 0 fixed = 240 total (was 238) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
35s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch has no whitespace
issues. {
[jira] [Commented] (YARN-10687) Add option to disable/enable free disk space checking and percentage checking for full and not-full disks
[ https://issues.apache.org/jira/browse/YARN-10687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17300019#comment-17300019 ] Qi Zhu commented on YARN-10687: --- Thanks a lot [~Jim_Brennan] for patient review. I have fixed it in latest patch, and added the unit test.:D > Add option to disable/enable free disk space checking and percentage checking > for full and not-full disks > - > > Key: YARN-10687 > URL: https://issues.apache.org/jira/browse/YARN-10687 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 3.2.2, 3.4.0 >Reporter: Qi Zhu >Assignee: Qi Zhu >Priority: Major > Attachments: YARN-10687.001.patch, YARN-10687.002.patch, > YARN-10687.003.patch > > > Now the two option: > max-disk-utilization-per-disk-percentage > min-free-space-per-disk-mb > for full/not full disk check are all enabled always, i think it's more > reasonable to enable or disable this, and default will be all enabled. > > In our clusters, when the disk is so huge we want to use > min-free-space-per-disk-mb. > In our clusters, when the disk is so small we want to use > max-disk-utilization-per-disk-percentage. > > We should make this more reasonable and not confused. > > cc [~pbacsko] [~Jim_Brennan] [~ebadger] [~gandras] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10687) Add option to disable/enable free disk space checking and percentage checking for full and not-full disks
[
https://issues.apache.org/jira/browse/YARN-10687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17299827#comment-17299827
]
Jim Brennan commented on YARN-10687:
Thanks for updating [~zhuqi]! Can you please add a unit test to
TestDirectoryCollection? Also, did you see [~gandras]'s suggestion about
changing the property names? e.g., {{disk-utilization-percentage.enabled}}?
> Add option to disable/enable free disk space checking and percentage checking
> for full and not-full disks
> -
>
> Key: YARN-10687
> URL: https://issues.apache.org/jira/browse/YARN-10687
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.2.2, 3.4.0
>Reporter: Qi Zhu
>Assignee: Qi Zhu
>Priority: Major
> Attachments: YARN-10687.001.patch, YARN-10687.002.patch
>
>
> Now the two option:
> max-disk-utilization-per-disk-percentage
> min-free-space-per-disk-mb
> for full/not full disk check are all enabled always, i think it's more
> reasonable to enable or disable this, and default will be all enabled.
>
> In our clusters, when the disk is so huge we want to use
> min-free-space-per-disk-mb.
> In our clusters, when the disk is so small we want to use
> max-disk-utilization-per-disk-percentage.
>
> We should make this more reasonable and not confused.
>
> cc [~pbacsko] [~Jim_Brennan] [~ebadger] [~gandras]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10687) Add option to disable/enable free disk space checking and percentage checking for full and not-full disks
[
https://issues.apache.org/jira/browse/YARN-10687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17299786#comment-17299786
]
Hadoop QA commented on YARN-10687:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
16s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} No case conflicting files
found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch does not contain any
@author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red}{color} | {color:red} The patch doesn't appear to
include any new or modified tests. Please justify why no new tests are needed
for this patch. Also please list what manual steps were performed to verify
this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
48s{color} | {color:blue}{color} | {color:blue} Maven dependency ordering for
branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 23m
38s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
20s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
18s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
41s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
29s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
19m 43s{color} | {color:green}{color} | {color:green} branch has no errors when
building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
5s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
14s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 29m
30s{color} | {color:blue}{color} | {color:blue} Both FindBugs and SpotBugs are
enabled, using SpotBugs. {color} |
| {color:green}+1{color} | {color:green} spotbugs {color} | {color:green} 5m
30s{color} | {color:green}{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
20s{color} | {color:blue}{color} | {color:blue} Maven dependency ordering for
patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
50s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
10s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 9m
10s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
12s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
12s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
38s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
21s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch has no whitespace
issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color}
[jira] [Commented] (YARN-10687) Add option to disable/enable free disk space checking and percentage checking for full and not-full disks
[ https://issues.apache.org/jira/browse/YARN-10687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17299649#comment-17299649 ] Qi Zhu commented on YARN-10687: --- [~Jim_Brennan] [~gandras] Updated the fix in latest patch. Thanks. > Add option to disable/enable free disk space checking and percentage checking > for full and not-full disks > - > > Key: YARN-10687 > URL: https://issues.apache.org/jira/browse/YARN-10687 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 3.2.2, 3.4.0 >Reporter: Qi Zhu >Assignee: Qi Zhu >Priority: Major > Attachments: YARN-10687.001.patch, YARN-10687.002.patch > > > Now the two option: > max-disk-utilization-per-disk-percentage > min-free-space-per-disk-mb > for full/not full disk check are all enabled always, i think it's more > reasonable to enable or disable this, and default will be all enabled. > > In our clusters, when the disk is so huge we want to use > min-free-space-per-disk-mb. > In our clusters, when the disk is so small we want to use > max-disk-utilization-per-disk-percentage. > > We should make this more reasonable and not confused. > > cc [~pbacsko] [~Jim_Brennan] [~ebadger] [~gandras] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10687) Add option to disable/enable free disk space checking and percentage checking for full and not-full disks
[
https://issues.apache.org/jira/browse/YARN-10687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17299616#comment-17299616
]
Qi Zhu commented on YARN-10687:
---
Thanks a lot [~Jim_Brennan] for review and confirm.
I will update your valid suggestions:
"we should also update the descriptions for the threshold properties to
indicate that they only apply when the corresponding {{enabled}} property is
true."
in next patch.
> Add option to disable/enable free disk space checking and percentage checking
> for full and not-full disks
> -
>
> Key: YARN-10687
> URL: https://issues.apache.org/jira/browse/YARN-10687
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.2.2, 3.4.0
>Reporter: Qi Zhu
>Assignee: Qi Zhu
>Priority: Major
> Attachments: YARN-10687.001.patch
>
>
> Now the two option:
> max-disk-utilization-per-disk-percentage
> min-free-space-per-disk-mb
> for full/not full disk check are all enabled always, i think it's more
> reasonable to enable or disable this, and default will be all enabled.
>
> In our clusters, when the disk is so huge we want to use
> min-free-space-per-disk-mb.
> In our clusters, when the disk is so small we want to use
> max-disk-utilization-per-disk-percentage.
>
> We should make this more reasonable and not confused.
>
> cc [~pbacsko] [~Jim_Brennan] [~ebadger] [~gandras]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10687) Add option to disable/enable free disk space checking and percentage checking for full and not-full disks
[
https://issues.apache.org/jira/browse/YARN-10687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17299613#comment-17299613
]
Jim Brennan commented on YARN-10687:
Thanks [~zhuqi]! These are not strictly needed, because the values can be set
to effectively enable/disable each threshold. But I agree that having these
makes the configuration options easier to understand. In addition the the name
changes recommended by [~gandras], we should also update the descriptions for
the threshold properties to indicate that they only apply when the
corresponding {{enabled}} property is true.
> Add option to disable/enable free disk space checking and percentage checking
> for full and not-full disks
> -
>
> Key: YARN-10687
> URL: https://issues.apache.org/jira/browse/YARN-10687
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.2.2, 3.4.0
>Reporter: Qi Zhu
>Assignee: Qi Zhu
>Priority: Major
> Attachments: YARN-10687.001.patch
>
>
> Now the two option:
> max-disk-utilization-per-disk-percentage
> min-free-space-per-disk-mb
> for full/not full disk check are all enabled always, i think it's more
> reasonable to enable or disable this, and default will be all enabled.
>
> In our clusters, when the disk is so huge we want to use
> min-free-space-per-disk-mb.
> In our clusters, when the disk is so small we want to use
> max-disk-utilization-per-disk-percentage.
>
> We should make this more reasonable and not confused.
>
> cc [~pbacsko] [~Jim_Brennan] [~ebadger] [~gandras]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10687) Add option to disable/enable free disk space checking and percentage checking for full and not-full disks
[
https://issues.apache.org/jira/browse/YARN-10687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17299447#comment-17299447
]
Hadoop QA commented on YARN-10687:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 34m
37s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} No case conflicting files
found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch does not contain any
@author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red}{color} | {color:red} The patch doesn't appear to
include any new or modified tests. Please justify why no new tests are needed
for this patch. Also please list what manual steps were performed to verify
this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
50s{color} | {color:blue}{color} | {color:blue} Maven dependency ordering for
branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 23m
23s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
48s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
18s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
40s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
31s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
19m 43s{color} | {color:green}{color} | {color:green} branch has no errors when
building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
4s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
16s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 29m
25s{color} | {color:blue}{color} | {color:blue} Both FindBugs and SpotBugs are
enabled, using SpotBugs. {color} |
| {color:green}+1{color} | {color:green} spotbugs {color} | {color:green} 5m
26s{color} | {color:green}{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
20s{color} | {color:blue}{color} | {color:blue} Maven dependency ordering for
patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
51s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
21s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 9m
21s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
18s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
18s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
37s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
20s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch has no whitespace
issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color}
[jira] [Commented] (YARN-10687) Add option to disable/enable free disk space checking and percentage checking for full and not-full disks
[ https://issues.apache.org/jira/browse/YARN-10687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17299387#comment-17299387 ] Qi Zhu commented on YARN-10687: --- Thanks a lot [~gandras] for review, very valid suggestion to me. I will update this in next patch.:D > Add option to disable/enable free disk space checking and percentage checking > for full and not-full disks > - > > Key: YARN-10687 > URL: https://issues.apache.org/jira/browse/YARN-10687 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 3.2.2, 3.4.0 >Reporter: Qi Zhu >Assignee: Qi Zhu >Priority: Major > Attachments: YARN-10687.001.patch > > > Now the two option: > max-disk-utilization-per-disk-percentage > min-free-space-per-disk-mb > for full/not full disk check are all enabled always, i think it's more > reasonable to enable or disable this, and default will be all enabled. > > In our clusters, when the disk is so huge we want to use > min-free-space-per-disk-mb. > In our clusters, when the disk is so small we want to use > max-disk-utilization-per-disk-percentage. > > We should make this more reasonable and not confused. > > cc [~pbacsko] [~Jim_Brennan] [~ebadger] [~gandras] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10687) Add option to disable/enable free disk space checking and percentage checking for full and not-full disks
[ https://issues.apache.org/jira/browse/YARN-10687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17299386#comment-17299386 ] Andras Gyori commented on YARN-10687: - Thank you [~zhuqi] for the patch. I am by no means expert on this area, but it does not seem uniform to me either. I would, however choose different names for these newly introduced properties, something more descriptive like: * diskUtilizationThresholdEnabled ** Enables the disk utilisation percentage threshold for disk health checker * diskFreeSpaceThresholdEnabled ** Enables the minimum disk free space threshold for disk health checker Currently, the disk-free-space-enabled is quite misleading (it is the threshold we are enabling). Also, I think the disk-utilization-percentage.enabled convention is more frequently used in the configuration, than the disk-utilization-percentage-enable. > Add option to disable/enable free disk space checking and percentage checking > for full and not-full disks > - > > Key: YARN-10687 > URL: https://issues.apache.org/jira/browse/YARN-10687 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 3.2.2, 3.4.0 >Reporter: Qi Zhu >Assignee: Qi Zhu >Priority: Major > Attachments: YARN-10687.001.patch > > > Now the two option: > max-disk-utilization-per-disk-percentage > min-free-space-per-disk-mb > for full/not full disk check are all enabled always, i think it's more > reasonable to enable or disable this, and default will be all enabled. > > In our clusters, when the disk is so huge we want to use > min-free-space-per-disk-mb. > In our clusters, when the disk is so small we want to use > max-disk-utilization-per-disk-percentage. > > We should make this more reasonable and not confused. > > cc [~pbacsko] [~Jim_Brennan] [~ebadger] [~gandras] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10687) Add option to disable/enable free disk space checking and percentage checking for full and not-full disks
[ https://issues.apache.org/jira/browse/YARN-10687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17299336#comment-17299336 ] Qi Zhu commented on YARN-10687: --- [~pbacsko] [~Jim_Brennan] [~ebadger] [~gandras] If you could help review this? Thanks.:D > Add option to disable/enable free disk space checking and percentage checking > for full and not-full disks > - > > Key: YARN-10687 > URL: https://issues.apache.org/jira/browse/YARN-10687 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 3.2.2, 3.4.0 >Reporter: Qi Zhu >Assignee: Qi Zhu >Priority: Major > Attachments: YARN-10687.001.patch > > > Now the two option: > max-disk-utilization-per-disk-percentage > min-free-space-per-disk-mb > for full/not full disk check are all enabled always, i think it's more > reasonable to enable or disable this, and default will be all enabled. > > In our clusters, when the disk is so huge we want to use > min-free-space-per-disk-mb. > In our clusters, when the disk is so small we want to use > max-disk-utilization-per-disk-percentage. > > We should make this more reasonable and not confused. > > cc [~pbacsko] [~Jim_Brennan] [~ebadger] [~gandras] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
