[jira] [Commented] (YARN-9355) RMContainerRequestor#makeRemoteRequest has confusing log message
[
https://issues.apache.org/jira/browse/YARN-9355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17070652#comment-17070652
]
Yousef Abu-Salah commented on YARN-9355:
I'm so sorry! You can just take me off. I didn't mean to take tasks meant for
others, but I'm relatively new to this scene. You are more than welcome to have
this. Thanks. [~sahuja]
> RMContainerRequestor#makeRemoteRequest has confusing log message
>
>
> Key: YARN-9355
> URL: https://issues.apache.org/jira/browse/YARN-9355
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Szilard Nemeth
>Assignee: Yousef Abu-Salah
>Priority: Trivial
> Labels: newbie, newbie++
>
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor#makeRemoteRequest
> has this log:
> {code:java}
> if (ask.size() > 0 || release.size() > 0) {
> LOG.info("getResources() for " + applicationId + ":" + " ask="
> + ask.size() + " release= " + release.size() + " newContainers="
> + allocateResponse.getAllocatedContainers().size()
> + " finishedContainers=" + numCompletedContainers
> + " resourcelimit=" + availableResources + " knownNMs="
> + clusterNmCount);
> }
> {code}
> The reason why "getResources()" is printed because
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator#getResources
> invokes makeRemoteRequest. This is not too informative and error-prone as
> name of getResources could change over time and the log will be outdated.
> Moreover, it's not a good idea to print a method name from a method below the
> current one in the stack.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6886) AllocationFileLoaderService.loadQueue() should validate that setting do not conflict with parent
[ https://issues.apache.org/jira/browse/YARN-6886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17070644#comment-17070644 ] Yousef Abu-Salah commented on YARN-6886: It assumes that the value for each child cannot exceed that of its parent. > AllocationFileLoaderService.loadQueue() should validate that setting do not > conflict with parent > > > Key: YARN-6886 > URL: https://issues.apache.org/jira/browse/YARN-6886 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 3.0.0-alpha4 >Reporter: Daniel Templeton >Assignee: Yousef Abu-Salah >Priority: Minor > Labels: newbie > Attachments: YARN-6886.001.patch > > > Some settings, like policy, are limited by the queue's parent queue's > configuration. We should check those settings when we load the file. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6886) AllocationFileLoaderService.loadQueue() should validate that setting do not conflict with parent
[ https://issues.apache.org/jira/browse/YARN-6886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yousef Abu-Salah updated YARN-6886: --- Attachment: (was: YARN-6886.001.diff) > AllocationFileLoaderService.loadQueue() should validate that setting do not > conflict with parent > > > Key: YARN-6886 > URL: https://issues.apache.org/jira/browse/YARN-6886 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 3.0.0-alpha4 >Reporter: Daniel Templeton >Assignee: Yousef Abu-Salah >Priority: Minor > Labels: newbie > Attachments: YARN-6886.001.patch > > > Some settings, like policy, are limited by the queue's parent queue's > configuration. We should check those settings when we load the file. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6551) Validate SLS input
[ https://issues.apache.org/jira/browse/YARN-6551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yousef Abu-Salah updated YARN-6551: --- Attachment: (was: YARN-6551.001.patch) > Validate SLS input > -- > > Key: YARN-6551 > URL: https://issues.apache.org/jira/browse/YARN-6551 > Project: Hadoop YARN > Issue Type: Bug > Components: scheduler-load-simulator >Reporter: Yufei Gu >Assignee: Yousef Abu-Salah >Priority: Major > Labels: newbie > Attachments: YARN-6551.002.patch > > > SLS takes three different input formats, SLS, RUMEN, and SYN. Some values > needs to be validated, e.g. node number cannot be negative. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6551) Validate SLS input
[ https://issues.apache.org/jira/browse/YARN-6551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17070517#comment-17070517 ] Yousef Abu-Salah commented on YARN-6551: org.apache.hadoop.yarn.sls.SLSRunner.validateConfig(Configuration) makes inefficient use of keySet iterator instead of entrySet iterator At SLSRunner.java:keySet iterator instead of entrySet iterator At SLSRunner.java:[line 229]" In this method 1. The key set is used for comparison to members of another data structure. 2. The entry set is strings that do not have any context without the key. Therefore the keys need to be accessed as a set for the iteration and then the entries need to be considered as per the specific key they belong to. > Validate SLS input > -- > > Key: YARN-6551 > URL: https://issues.apache.org/jira/browse/YARN-6551 > Project: Hadoop YARN > Issue Type: Bug > Components: scheduler-load-simulator >Reporter: Yufei Gu >Assignee: Yousef Abu-Salah >Priority: Major > Labels: newbie > Attachments: YARN-6551.002.patch > > > SLS takes three different input formats, SLS, RUMEN, and SYN. Some values > needs to be validated, e.g. node number cannot be negative. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6551) Validate SLS input
[ https://issues.apache.org/jira/browse/YARN-6551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yousef Abu-Salah updated YARN-6551: --- Attachment: (was: YARN-6551.001.diff) > Validate SLS input > -- > > Key: YARN-6551 > URL: https://issues.apache.org/jira/browse/YARN-6551 > Project: Hadoop YARN > Issue Type: Bug > Components: scheduler-load-simulator >Reporter: Yufei Gu >Assignee: Yousef Abu-Salah >Priority: Major > Labels: newbie > Attachments: YARN-6551.002.patch > > > SLS takes three different input formats, SLS, RUMEN, and SYN. Some values > needs to be validated, e.g. node number cannot be negative. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6425) Move out FS state dump code out of method update()
[
https://issues.apache.org/jira/browse/YARN-6425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17070048#comment-17070048
]
Yousef Abu-Salah commented on YARN-6425:
*Justification for no new tests*:
* The logging dump was moved outside of the read lock. All the existing unit
tests for this method behave correctly which mean that the code operates
correctly
*Manual Steps Performed to make this Patch*:
* All the unit tests in the package '
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair' were run against
this code change. This was done in NetBeans.
The output of the run of unit tests is in the folder I shared with you and is
called 'YARN-6425_unit_test_run
** '[^YARN-6425_unit_test_run.docx]
> Move out FS state dump code out of method update()
> --
>
> Key: YARN-6425
> URL: https://issues.apache.org/jira/browse/YARN-6425
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 2.9.0, 3.0.0-alpha2
>Reporter: Yufei Gu
>Assignee: Yousef Abu-Salah
>Priority: Major
> Labels: newbie++
> Attachments: YARN-6425.001.patch, YARN-6425_unit_test_run.docx
>
>
> Better to move out FS state dump code out of update()
> {code}
> if (LOG.isDebugEnabled()) {
> if (--updatesToSkipForDebug < 0) {
> updatesToSkipForDebug = UPDATE_DEBUG_FREQUENCY;
> dumpSchedulerState();
> }
> }
> {code}
> And, after that we should distinct between update call and update thread
> duration like before YARN-6112.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6425) Move out FS state dump code out of method update()
[
https://issues.apache.org/jira/browse/YARN-6425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yousef Abu-Salah updated YARN-6425:
---
Attachment: YARN-6425_unit_test_run.docx
> Move out FS state dump code out of method update()
> --
>
> Key: YARN-6425
> URL: https://issues.apache.org/jira/browse/YARN-6425
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 2.9.0, 3.0.0-alpha2
>Reporter: Yufei Gu
>Assignee: Yousef Abu-Salah
>Priority: Major
> Labels: newbie++
> Attachments: YARN-6425.001.patch, YARN-6425_unit_test_run.docx
>
>
> Better to move out FS state dump code out of update()
> {code}
> if (LOG.isDebugEnabled()) {
> if (--updatesToSkipForDebug < 0) {
> updatesToSkipForDebug = UPDATE_DEBUG_FREQUENCY;
> dumpSchedulerState();
> }
> }
> {code}
> And, after that we should distinct between update call and update thread
> duration like before YARN-6112.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6425) Move out FS state dump code out of method update()
[
https://issues.apache.org/jira/browse/YARN-6425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yousef Abu-Salah updated YARN-6425:
---
Attachment: YARN-6425.001.patch
> Move out FS state dump code out of method update()
> --
>
> Key: YARN-6425
> URL: https://issues.apache.org/jira/browse/YARN-6425
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 2.9.0, 3.0.0-alpha2
>Reporter: Yufei Gu
>Assignee: Yousef Abu-Salah
>Priority: Major
> Labels: newbie++
> Attachments: YARN-6425.001.patch
>
>
> Better to move out FS state dump code out of update()
> {code}
> if (LOG.isDebugEnabled()) {
> if (--updatesToSkipForDebug < 0) {
> updatesToSkipForDebug = UPDATE_DEBUG_FREQUENCY;
> dumpSchedulerState();
> }
> }
> {code}
> And, after that we should distinct between update call and update thread
> duration like before YARN-6112.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6425) Move out FS state dump code out of method update()
[
https://issues.apache.org/jira/browse/YARN-6425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yousef Abu-Salah updated YARN-6425:
---
Attachment: (was: YARN-6425.001.patch)
> Move out FS state dump code out of method update()
> --
>
> Key: YARN-6425
> URL: https://issues.apache.org/jira/browse/YARN-6425
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 2.9.0, 3.0.0-alpha2
>Reporter: Yufei Gu
>Assignee: Yousef Abu-Salah
>Priority: Major
> Labels: newbie++
> Attachments: YARN-6425.001.patch
>
>
> Better to move out FS state dump code out of update()
> {code}
> if (LOG.isDebugEnabled()) {
> if (--updatesToSkipForDebug < 0) {
> updatesToSkipForDebug = UPDATE_DEBUG_FREQUENCY;
> dumpSchedulerState();
> }
> }
> {code}
> And, after that we should distinct between update call and update thread
> duration like before YARN-6112.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6886) AllocationFileLoaderService.loadQueue() should validate that setting do not conflict with parent
[ https://issues.apache.org/jira/browse/YARN-6886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17069195#comment-17069195 ] Yousef Abu-Salah commented on YARN-6886: [~templedf] it seems that the method SchedulingPolicy.isChildPolicyAllowed is implemented in the classes that inherit SchedulingPolicy, which solves the bug already. > AllocationFileLoaderService.loadQueue() should validate that setting do not > conflict with parent > > > Key: YARN-6886 > URL: https://issues.apache.org/jira/browse/YARN-6886 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 3.0.0-alpha4 >Reporter: Daniel Templeton >Assignee: Yousef Abu-Salah >Priority: Minor > Labels: newbie > Attachments: YARN-6886.001.diff > > > Some settings, like policy, are limited by the queue's parent queue's > configuration. We should check those settings when we load the file. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-6886) AllocationFileLoaderService.loadQueue() should validate that setting do not conflict with parent
[ https://issues.apache.org/jira/browse/YARN-6886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17069187#comment-17069187 ] Yousef Abu-Salah edited comment on YARN-6886 at 3/28/20, 2:27 AM: -- [~templedf] Would you mind also checking YARN-6551 and YARN-5913 for me? Those are the other two I would love some feedback on. was (Author: ykabusalah): [~ykabusalah] Okay nice. Could you check YARN-6551 and YARN-5913 for me? Those are the other two I would love some feedback on. > AllocationFileLoaderService.loadQueue() should validate that setting do not > conflict with parent > > > Key: YARN-6886 > URL: https://issues.apache.org/jira/browse/YARN-6886 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 3.0.0-alpha4 >Reporter: Daniel Templeton >Assignee: Yousef Abu-Salah >Priority: Minor > Labels: newbie > Attachments: YARN-6886.001.diff > > > Some settings, like policy, are limited by the queue's parent queue's > configuration. We should check those settings when we load the file. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6886) AllocationFileLoaderService.loadQueue() should validate that setting do not conflict with parent
[ https://issues.apache.org/jira/browse/YARN-6886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17069187#comment-17069187 ] Yousef Abu-Salah commented on YARN-6886: [~ykabusalah] Okay nice. Could you check YARN-6551 and YARN-5913 for me? Those are the other two I would love some feedback on. > AllocationFileLoaderService.loadQueue() should validate that setting do not > conflict with parent > > > Key: YARN-6886 > URL: https://issues.apache.org/jira/browse/YARN-6886 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 3.0.0-alpha4 >Reporter: Daniel Templeton >Assignee: Yousef Abu-Salah >Priority: Minor > Labels: newbie > Attachments: YARN-6886.001.diff > > > Some settings, like policy, are limited by the queue's parent queue's > configuration. We should check those settings when we load the file. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6886) AllocationFileLoaderService.loadQueue() should validate that setting do not conflict with parent
[ https://issues.apache.org/jira/browse/YARN-6886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17069183#comment-17069183 ] Yousef Abu-Salah commented on YARN-6886: [~templedf] Has this issue been resolved? I submitted patches for the three I had to do last year but am afraid that all of these have already been patched up. > AllocationFileLoaderService.loadQueue() should validate that setting do not > conflict with parent > > > Key: YARN-6886 > URL: https://issues.apache.org/jira/browse/YARN-6886 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 3.0.0-alpha4 >Reporter: Daniel Templeton >Assignee: Yousef Abu-Salah >Priority: Minor > Labels: newbie > Attachments: YARN-6886.001.diff > > > Some settings, like policy, are limited by the queue's parent queue's > configuration. We should check those settings when we load the file. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10163) Nodes with reused IP's will incorrectly maintain old label
[ https://issues.apache.org/jira/browse/YARN-10163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yousef Abu-Salah reassigned YARN-10163: --- Assignee: Yousef Abu-Salah > Nodes with reused IP's will incorrectly maintain old label > -- > > Key: YARN-10163 > URL: https://issues.apache.org/jira/browse/YARN-10163 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 2.8.5 >Reporter: Brandon Scheller >Assignee: Yousef Abu-Salah >Priority: Minor > > Take a long running cluster, where nodemanagers come and go as the number of > instances scale up and down. This can cause an issue with Yarn nodelabels > where an unlabeled node picks up the label of the previous node with a > matching ip. > In this scenario, node ip's could be reused and a new node could come up > taking the place of a previously labeled node. If the old node was labeled, > but the new node is not labeled, the new node will not send a node label > update and the old nodelabel will remain. This can cause issues as the node > now is mislabeled. > > The fix for this likely is to send node label updates even when a node is > unlabeled. This will allow the resourcemanger to reset the label to empty for > that node. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-9355) RMContainerRequestor#makeRemoteRequest has confusing log message
[
https://issues.apache.org/jira/browse/YARN-9355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yousef Abu-Salah reassigned YARN-9355:
--
Assignee: Yousef Abu-Salah (was: Siddharth Ahuja)
> RMContainerRequestor#makeRemoteRequest has confusing log message
>
>
> Key: YARN-9355
> URL: https://issues.apache.org/jira/browse/YARN-9355
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Szilard Nemeth
>Assignee: Yousef Abu-Salah
>Priority: Trivial
> Labels: newbie, newbie++
>
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor#makeRemoteRequest
> has this log:
> {code:java}
> if (ask.size() > 0 || release.size() > 0) {
> LOG.info("getResources() for " + applicationId + ":" + " ask="
> + ask.size() + " release= " + release.size() + " newContainers="
> + allocateResponse.getAllocatedContainers().size()
> + " finishedContainers=" + numCompletedContainers
> + " resourcelimit=" + availableResources + " knownNMs="
> + clusterNmCount);
> }
> {code}
> The reason why "getResources()" is printed because
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator#getResources
> invokes makeRemoteRequest. This is not too informative and error-prone as
> name of getResources could change over time and the log will be outdated.
> Moreover, it's not a good idea to print a method name from a method below the
> current one in the stack.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10129) Code cleanup in UIv2
[ https://issues.apache.org/jira/browse/YARN-10129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yousef Abu-Salah reassigned YARN-10129: --- Assignee: Yousef Abu-Salah > Code cleanup in UIv2 > > > Key: YARN-10129 > URL: https://issues.apache.org/jira/browse/YARN-10129 > Project: Hadoop YARN > Issue Type: Improvement > Components: yarn-ui-v2 >Affects Versions: 3.2.1 >Reporter: Adam Antal >Assignee: Yousef Abu-Salah >Priority: Minor > Labels: newbie > > In YARN-10029 we found lots of code duplicates in the code of Yarn UIv2. > We should try to get rid of them to create more concise code. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10188) Change file extension to support markdown (Readme)
[ https://issues.apache.org/jira/browse/YARN-10188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yousef Abu-Salah reassigned YARN-10188: --- Assignee: Yousef Abu-Salah > Change file extension to support markdown (Readme) > -- > > Key: YARN-10188 > URL: https://issues.apache.org/jira/browse/YARN-10188 > Project: Hadoop YARN > Issue Type: Improvement > Components: documentation >Affects Versions: 3.1.3 >Reporter: Abhishek Das >Assignee: Yousef Abu-Salah >Priority: Minor > > Change the file extension of README file to README.md for better readability. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5913) Consolidate "resource" and "amResourceRequest" in ApplicationSubmissionContext
[ https://issues.apache.org/jira/browse/YARN-5913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yousef Abu-Salah updated YARN-5913: --- Attachment: (was: 5913.diff) > Consolidate "resource" and "amResourceRequest" in ApplicationSubmissionContext > -- > > Key: YARN-5913 > URL: https://issues.apache.org/jira/browse/YARN-5913 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Reporter: Yufei Gu >Assignee: Yousef Abu-Salah >Priority: Minor > Labels: newbie > Attachments: YARN-5913.001.diff > > > Usage of these two variables overlaps and causes confusion. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6886) AllocationFileLoaderService.loadQueue() should validate that setting do not conflict with parent
[ https://issues.apache.org/jira/browse/YARN-6886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yousef Abu-Salah updated YARN-6886: --- Attachment: (was: YARN-6886.001.diff) > AllocationFileLoaderService.loadQueue() should validate that setting do not > conflict with parent > > > Key: YARN-6886 > URL: https://issues.apache.org/jira/browse/YARN-6886 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 3.0.0-alpha4 >Reporter: Daniel Templeton >Assignee: Yousef Abu-Salah >Priority: Minor > Labels: newbie > Attachments: YARN-6886.001.diff > > > Some settings, like policy, are limited by the queue's parent queue's > configuration. We should check those settings when we load the file. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6886) AllocationFileLoaderService.loadQueue() should validate that setting do not conflict with parent
[ https://issues.apache.org/jira/browse/YARN-6886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yousef Abu-Salah updated YARN-6886: --- Attachment: YARN-6886.001.diff > AllocationFileLoaderService.loadQueue() should validate that setting do not > conflict with parent > > > Key: YARN-6886 > URL: https://issues.apache.org/jira/browse/YARN-6886 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 3.0.0-alpha4 >Reporter: Daniel Templeton >Assignee: Yousef Abu-Salah >Priority: Minor > Labels: newbie > Attachments: YARN-6886.001.diff > > > Some settings, like policy, are limited by the queue's parent queue's > configuration. We should check those settings when we load the file. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6886) AllocationFileLoaderService.loadQueue() should validate that setting do not conflict with parent
[ https://issues.apache.org/jira/browse/YARN-6886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yousef Abu-Salah updated YARN-6886: --- Attachment: 6886.diff > AllocationFileLoaderService.loadQueue() should validate that setting do not > conflict with parent > > > Key: YARN-6886 > URL: https://issues.apache.org/jira/browse/YARN-6886 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 3.0.0-alpha4 >Reporter: Daniel Templeton >Assignee: Yousef Abu-Salah >Priority: Minor > Labels: newbie > > Some settings, like policy, are limited by the queue's parent queue's > configuration. We should check those settings when we load the file. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6886) AllocationFileLoaderService.loadQueue() should validate that setting do not conflict with parent
[ https://issues.apache.org/jira/browse/YARN-6886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yousef Abu-Salah updated YARN-6886: --- Attachment: (was: 6886.diff) > AllocationFileLoaderService.loadQueue() should validate that setting do not > conflict with parent > > > Key: YARN-6886 > URL: https://issues.apache.org/jira/browse/YARN-6886 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 3.0.0-alpha4 >Reporter: Daniel Templeton >Assignee: Yousef Abu-Salah >Priority: Minor > Labels: newbie > > Some settings, like policy, are limited by the queue's parent queue's > configuration. We should check those settings when we load the file. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-6382) Address race condition on TimelineWriter.flush() caused by buffer-sized flush
[ https://issues.apache.org/jira/browse/YARN-6382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yousef Abu-Salah reassigned YARN-6382: -- Assignee: Yousef Abu-Salah (was: Haibo Chen) > Address race condition on TimelineWriter.flush() caused by buffer-sized flush > - > > Key: YARN-6382 > URL: https://issues.apache.org/jira/browse/YARN-6382 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.0.0-alpha2 >Reporter: Haibo Chen >Assignee: Yousef Abu-Salah >Priority: Major > > YARN-6376 fixes the race condition between putEntities() and periodical > flush() by WriterFlushThread in TimelineCollectorManager, or between > putEntities() in different threads. > However, BufferedMutator can have internal size-based flush as well. We need > to address the resulting race condition. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5913) Consolidate "resource" and "amResourceRequest" in ApplicationSubmissionContext
[ https://issues.apache.org/jira/browse/YARN-5913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16917402#comment-16917402 ] Yousef Abu-Salah commented on YARN-5913: Can I get a more descriptive rundown of the problem? > Consolidate "resource" and "amResourceRequest" in ApplicationSubmissionContext > -- > > Key: YARN-5913 > URL: https://issues.apache.org/jira/browse/YARN-5913 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Reporter: Yufei Gu >Priority: Minor > Labels: newbie > > Usage of these two variables overlaps and causes confusion. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6425) Move out FS state dump code out of method update()
[
https://issues.apache.org/jira/browse/YARN-6425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16916842#comment-16916842
]
Yousef Abu-Salah commented on YARN-6425:
Can I take this issue? In addition, could I get a more descriptive definition
of the problem and what needs to be accomplished?
> Move out FS state dump code out of method update()
> --
>
> Key: YARN-6425
> URL: https://issues.apache.org/jira/browse/YARN-6425
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 2.9.0, 3.0.0-alpha2
>Reporter: Yufei Gu
>Priority: Major
> Labels: newbie++
>
> Better to move out FS state dump code out of update()
> {code}
> if (LOG.isDebugEnabled()) {
> if (--updatesToSkipForDebug < 0) {
> updatesToSkipForDebug = UPDATE_DEBUG_FREQUENCY;
> dumpSchedulerState();
> }
> }
> {code}
> And, after that we should distinct between update call and update thread
> duration like before YARN-6112.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-5913) Consolidate "resource" and "amResourceRequest" in ApplicationSubmissionContext
[ https://issues.apache.org/jira/browse/YARN-5913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16916838#comment-16916838 ] Yousef Abu-Salah edited comment on YARN-5913 at 8/27/19 4:21 PM: - Can I take this issue? In addition, could I get a more descriptive definition of the problem and what needs to be accomplished? was (Author: ykabusalah): Can I take this issue? > Consolidate "resource" and "amResourceRequest" in ApplicationSubmissionContext > -- > > Key: YARN-5913 > URL: https://issues.apache.org/jira/browse/YARN-5913 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Reporter: Yufei Gu >Priority: Minor > Labels: newbie > > Usage of these two variables overlaps and causes confusion. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5913) Consolidate "resource" and "amResourceRequest" in ApplicationSubmissionContext
[ https://issues.apache.org/jira/browse/YARN-5913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16916838#comment-16916838 ] Yousef Abu-Salah commented on YARN-5913: Can I take this issue? > Consolidate "resource" and "amResourceRequest" in ApplicationSubmissionContext > -- > > Key: YARN-5913 > URL: https://issues.apache.org/jira/browse/YARN-5913 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Reporter: Yufei Gu >Priority: Minor > Labels: newbie > > Usage of these two variables overlaps and causes confusion. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6046) Documentation correction in YarnApplicationSecurity
[
https://issues.apache.org/jira/browse/YARN-6046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yousef Abu-Salah updated YARN-6046:
---
Attachment: yarn-6046.002.patch
> Documentation correction in YarnApplicationSecurity
> ---
>
> Key: YARN-6046
> URL: https://issues.apache.org/jira/browse/YARN-6046
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Yousef Abu-Salah
>Priority: Trivial
> Labels: newbie
> Attachments: YARN-6046.001.patch, yarn-6046.002.patch
>
>
> Few documentation correction required in
> hadoop-yarn/hadoop-yarn-site/YarnApplicationSecurity.html
> {code}
> 1. Suring AM startup, log in to Kerberos.
> {code}
> {code}
> Don’t. Rely on the lifespan of the
> {code}
> {code}
> renewed automatically; the AM pushes out
> {code}
> {code}
> In an insecure cluster, the application will run as the identity of the
> account of the node manager, typically something such as yarn or mapred. By
> default, the application will access HDFS as that user, with a different home
> directory, and with a different user identified in audit logs and on file
> system owner attributes.
> {code}
> Need to reframe sentence.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
