[jira] [Commented] (YARN-4981) limit the max containers to assign to one application per heartbeart
[ https://issues.apache.org/jira/browse/YARN-4981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15614405#comment-15614405 ] Karthik Kambatla commented on YARN-4981: YARN-5035 adjusts the number of containers allocated to a node based on the CPU and memory load on the node. This, IMO, should address the main request here. I ll close this as being incorporated in the other. > limit the max containers to assign to one application per heartbeart > > > Key: YARN-4981 > URL: https://issues.apache.org/jira/browse/YARN-4981 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 2.5.0, 2.6.4 >Reporter: ChenFolin > Labels: oct16-medium > Fix For: 2.5.0 > > Attachments: YARN-4981.patch > > > When use assignMultiple，and such as set maxAssign=10: > if a job is high CPU util or high CPU load, it may lead to a part of nodes > very busy. and the job may cost a long time. > so , i want to limit the max containers to assign to one application per > heartbeart, it may help to nodes use more uniform. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-4743) FairSharePolicy breaks TimSort assumption
[
https://issues.apache.org/jira/browse/YARN-4743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karthik Kambatla updated YARN-4743:
---
Fix Version/s: 2.9.0
> FairSharePolicy breaks TimSort assumption
> -
>
> Key: YARN-4743
> URL: https://issues.apache.org/jira/browse/YARN-4743
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 3.0.0-alpha2
>Reporter: Zephyr Guo
>Assignee: Zephyr Guo
> Fix For: 2.9.0, 3.0.0-alpha2
>
> Attachments: YARN-4743-v1.patch, YARN-4743-v2.patch,
> YARN-4743-v3.patch, YARN-4743-v4.patch, YARN-4743-v5.patch, timsort.log
>
>
> {code}
> 2016-02-26 14:08:50,821 FATAL
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error in
> handling event type NODE_UPDATE to the scheduler
> java.lang.IllegalArgumentException: Comparison method violates its general
> contract!
> at java.util.TimSort.mergeHi(TimSort.java:868)
> at java.util.TimSort.mergeAt(TimSort.java:485)
> at java.util.TimSort.mergeCollapse(TimSort.java:410)
> at java.util.TimSort.sort(TimSort.java:214)
> at java.util.TimSort.sort(TimSort.java:173)
> at java.util.Arrays.sort(Arrays.java:659)
> at java.util.Collections.sort(Collections.java:217)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSLeafQueue.assignContainer(FSLeafQueue.java:316)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSParentQueue.assignContainer(FSParentQueue.java:240)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.attemptScheduling(FairScheduler.java:1091)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.nodeUpdate(FairScheduler.java:989)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:1185)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:112)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:684)
> at java.lang.Thread.run(Thread.java:745)
> 2016-02-26 14:08:50,822 INFO
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Exiting, bbye..
> {code}
> Actually, this bug found in 2.6.0-cdh5.4.7. {{FairShareComparator}} is not
> transitive.
> We get NaN when memorySize=0 and weight=0.
> {code:title=FairSharePolicy.java}
> useToWeightRatio1 = s1.getResourceUsage().getMemorySize() /
> s1.getWeights().getWeight(ResourceType.MEMORY)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4862) Handle duplicate completed containers in RMNodeImpl
[ https://issues.apache.org/jira/browse/YARN-4862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15614386#comment-15614386 ] Rohith Sharma K S commented on YARN-4862: - I request please hold down committing this patch. Actually I was discovered some scenarios where completedContainers list get never removed which would lead to leak in completedContainers gradually. I will confirm those scenarios. > Handle duplicate completed containers in RMNodeImpl > --- > > Key: YARN-4862 > URL: https://issues.apache.org/jira/browse/YARN-4862 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S > Attachments: 0001-YARN-4862.patch, 0002-YARN-4862.patch, > 0003-YARN-4862.patch > > > As per > [comment|https://issues.apache.org/jira/browse/YARN-4852?focusedCommentId=15209689=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15209689] > from [~sharadag], there should be safe guard for duplicated container status > in RMNodeImpl before creating UpdatedContainerInfo. > Or else in heavily loaded cluster where event processing is gradually slow, > if any duplicated container are sent to RM(may be bug in NM also), there is > significant impact that RMNodImpl always create UpdatedContainerInfo for > duplicated containers. This result in increase in the heap memory and causes > problem like YARN-4852. > This is an optimization for issue kind YARN-4852 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5279) Potential Container leak in NM in preemption flow
[
https://issues.apache.org/jira/browse/YARN-5279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15614378#comment-15614378
]
Rohith Sharma K S commented on YARN-5279:
-
bq. Is it truly the case that YARN-4862 requires this patch?
Not necessarily required. I removed the link. Sorry for confusion.
> Potential Container leak in NM in preemption flow
> -
>
> Key: YARN-5279
> URL: https://issues.apache.org/jira/browse/YARN-5279
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager, resourcemanager
>Reporter: Rohith Sharma K S
>Assignee: Rohith Sharma K S
> Attachments: 0001-YARN-5279.patch
>
>
> In discussion YARN-4862
> [comment|https://issues.apache.org/jira/browse/YARN-4862?focusedCommentId=15341538=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15341538],
> it is observed that there could be a container leak in NodeManager whenever
> container is preempted from RM
> Basically if NM receives same containerId details in {{containersToCleanUp}}
> and {{containersToBeRemovedFromNM}} in the same heartbeat then container
> will never-ever removed in NMContext. Rather NM kills the container of
> containersToCleanup and send back status again to RM. But RM blindly reject
> the status since RMContainer is already removed and it is null.
> I think whenever RMContainer is null, RMNode should be informed to send
> {{containersToBeRemovedFromNM}} so that NM will remove from its context.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4862) Handle duplicate completed containers in RMNodeImpl
[ https://issues.apache.org/jira/browse/YARN-4862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15614369#comment-15614369 ] Rohith Sharma K S commented on YARN-4862: - This patch is independent of YARN-5279. I will remove link and lets handle independently. > Handle duplicate completed containers in RMNodeImpl > --- > > Key: YARN-4862 > URL: https://issues.apache.org/jira/browse/YARN-4862 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S > Attachments: 0001-YARN-4862.patch, 0002-YARN-4862.patch, > 0003-YARN-4862.patch > > > As per > [comment|https://issues.apache.org/jira/browse/YARN-4852?focusedCommentId=15209689=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15209689] > from [~sharadag], there should be safe guard for duplicated container status > in RMNodeImpl before creating UpdatedContainerInfo. > Or else in heavily loaded cluster where event processing is gradually slow, > if any duplicated container are sent to RM(may be bug in NM also), there is > significant impact that RMNodImpl always create UpdatedContainerInfo for > duplicated containers. This result in increase in the heap memory and causes > problem like YARN-4852. > This is an optimization for issue kind YARN-4852 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-4396) Log the trace information on FSAppAttempt#assignContainer
[ https://issues.apache.org/jira/browse/YARN-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated YARN-4396: Summary: Log the trace information on FSAppAttempt#assignContainer (was: Log the trance information on FSAppAttempt#assignContainer) > Log the trace information on FSAppAttempt#assignContainer > - > > Key: YARN-4396 > URL: https://issues.apache.org/jira/browse/YARN-4396 > Project: Hadoop YARN > Issue Type: Improvement > Components: applications, fairscheduler >Affects Versions: 2.7.1 >Reporter: Yiqun Lin >Assignee: Yiqun Lin > Labels: oct16-easy > Attachments: YARN-4396.001.patch, YARN-4396.002.patch, > YARN-4396.003.patch, YARN-4396.004.patch, YARN-4396.005.patch > > > When I configure the yarn.scheduler.fair.locality.threshold.node and > yarn.scheduler.fair.locality.threshold.rack to open this function, I have no > detail info of assigning container's locality. And it's important because it > will lead some delay scheduling and will have an influence on my cluster. If > I know these info, I can adjust param in cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-2995) Enhance UI to show cluster resource utilization of various container types
[ https://issues.apache.org/jira/browse/YARN-2995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15614329#comment-15614329 ] Arun Suresh commented on YARN-2995: --- [~kkaranasos], thanks for updating the patch. Can you confirm the test failure is un-related. Also, Can you maybe paste a screen shot of the new web-UI ? +1 pending the above. > Enhance UI to show cluster resource utilization of various container types > -- > > Key: YARN-2995 > URL: https://issues.apache.org/jira/browse/YARN-2995 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Sriram Rao >Assignee: Konstantinos Karanasos > Attachments: YARN-2995.001.patch, YARN-2995.002.patch > > > This JIRA proposes to extend the Resource manager UI to show how cluster > resources are being used to run *guaranteed start* and *queueable* > containers. For example, a graph that shows over time, the fraction of > running containers that are *guaranteed start* and the fraction of running > containers that are *queueable*. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5773) RM recovery too slow due to LeafQueue#activateApplication()
[
https://issues.apache.org/jira/browse/YARN-5773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15614316#comment-15614316
]
Bibin A Chundatt commented on YARN-5773:
[~varun_saxena]
hadoop.yarn.server.resourcemanager.TestRMRestart - YARN-5548
All other test cases looks dependent on {{getNumActiveApplications}} . So can
we go ahead with 0005.patch
> RM recovery too slow due to LeafQueue#activateApplication()
> ---
>
> Key: YARN-5773
> URL: https://issues.apache.org/jira/browse/YARN-5773
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler, rolling upgrade
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Critical
> Labels: oct16-medium
> Attachments: YARN-5773.0001.patch, YARN-5773.0002.patch,
> YARN-5773.0004.patch, YARN-5773.0005.patch, YARN-5773.0006.patch,
> YARN-5773.003.patch
>
>
> # Submit application 10K application to default queue.
> # All applications are in accepted state
> # Now restart resourcemanager
> For each application recovery {{LeafQueue#activateApplications()}} is
> invoked.Resulting in AM limit check to be done even before Node managers are
> getting registered.
> Total iteration for N application is about {{N(N+1)/2}} for {{10K}}
> application {{5000}} iterations causing time take for Rm to be active
> more than 10 min.
> Since NM resources are not yet added to during recovery we should skip
> {{activateApplicaiton()}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5793) Trim configuration values in DockerLinuxContainerRuntime
[
https://issues.apache.org/jira/browse/YARN-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15614302#comment-15614302
]
Tianyin Xu commented on YARN-5793:
--
[~templedf], I'm puzzled by the Jenkins report. The failed test,
org.apache.hadoop.yarn.server.nodemanager.containermanager.queuing.testKillMultipleOpportunisticContainers,
comes from the {{waitForNMContainerState}} call, which seems have nothing to
do with the patch. Also, I did the test on my local repo with the newest code
base, and the tests pass without any problem.
Is it possible for you to take a look and point me what I'm missing?
Thanks a lot in advance!
~t
> Trim configuration values in DockerLinuxContainerRuntime
>
>
> Key: YARN-5793
> URL: https://issues.apache.org/jira/browse/YARN-5793
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.0.0-alpha1
>Reporter: Tianyin Xu
>Priority: Minor
> Attachments: YARN-5793..patch, YARN-5793.0001.patch
>
>
> The current implementation of {{DockerLinuxContainerRuntime}} does not follow
> the practice of trimming configuration values. This leads to errors if users
> set values containing space or newline.
> see the following YARN commits as reference:
> YARN-3395. FairScheduler: Trim whitespaces when using username for queuename.
> YARN-2869. CapacityScheduler should trim sub queue names when parse
> configuration.
> YARN-2843. Fixed NodeLabelsManager to trim inputs for hosts and labels so as
> to make them work correctly.
> and many other Hadoop/HDFS commits (just list a few):
> HDFS-9708. FSNamesystem.initAuditLoggers() doesn't trim classnames
> HDFS-2799. Trim fs.checkpoint.dir values.
> HADOOP-6578. Configuration should trim whitespace around a lot of value types
> HADOOP-6534. Trim whitespace from directory lists initializing
> Patch is available against trunk
> {code:title=DockerLinuxContainerRuntime.java|borderStyle=solid}
> @@ -219,9 +219,9 @@ public void initialize(Configuration conf)
> dockerClient = new DockerClient(conf);
> allowedNetworks.clear();
> allowedNetworks.addAll(Arrays.asList(
> -
> conf.getStrings(YarnConfiguration.NM_DOCKER_ALLOWED_CONTAINER_NETWORKS,
> +
> conf.getTrimmedStrings(YarnConfiguration.NM_DOCKER_ALLOWED_CONTAINER_NETWORKS,
>
> YarnConfiguration.DEFAULT_NM_DOCKER_ALLOWED_CONTAINER_NETWORKS)));
> -defaultNetwork = conf.get(
> +defaultNetwork = conf.getTrimmed(
> YarnConfiguration.NM_DOCKER_DEFAULT_CONTAINER_NETWORK,
> YarnConfiguration.DEFAULT_NM_DOCKER_DEFAULT_CONTAINER_NETWORK);
>
> @@ -237,7 +237,7 @@ public void initialize(Configuration conf)
>throw new ContainerExecutionException(message);
> }
>
> -privilegedContainersAcl = new AccessControlList(conf.get(
> +privilegedContainersAcl = new AccessControlList(conf.getTrimmed(
> YarnConfiguration.NM_DOCKER_PRIVILEGED_CONTAINERS_ACL,
> YarnConfiguration.DEFAULT_NM_DOCKER_PRIVILEGED_CONTAINERS_ACL));
>}
> @@ -439,7 +439,7 @@ public void launchContainer(ContainerRuntimeContext ctx)
> LOCALIZED_RESOURCES);
> @SuppressWarnings("unchecked")
> List userLocalDirs = ctx.getExecutionAttribute(USER_LOCAL_DIRS);
> -Set capabilities = new HashSet<>(Arrays.asList(conf.getStrings(
> +Set capabilities = new
> HashSet<>(Arrays.asList(conf.getTrimmedStrings(
> YarnConfiguration.NM_DOCKER_CONTAINER_CAPABILITIES,
> YarnConfiguration.DEFAULT_NM_DOCKER_CONTAINER_CAPABILITIES)));
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5794) Fix the asflicense warnings
[
https://issues.apache.org/jira/browse/YARN-5794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15614299#comment-15614299
]
Naganarasimha G R commented on YARN-5794:
-
Thanks for the patch [~linyiqun], +1, will commit this shortly !
> Fix the asflicense warnings
> ---
>
> Key: YARN-5794
> URL: https://issues.apache.org/jira/browse/YARN-5794
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Yiqun Lin
>Assignee: Yiqun Lin
> Attachments: YARN-5794.001.patch
>
>
> There are some asflicense warnings showed when building the jenkins of
> hadoop-common. The warning infos (link address
> https://builds.apache.org/job/PreCommit-HDFS-Build/17336/artifact/patchprocess/patch-asflicense-problems.txt):
> {code}
> Lines that start with ? in the ASF License report indicate files that do
> not have an Apache license header:
> !?
> /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/jt/jquery.jstree.js
> !?
> /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/dt-1.9.4/js/jquery.dataTables.min.js
> !?
> /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/jquery/jquery-ui-1.9.1.custom.min.js
> !?
> /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/jquery/jquery-1.8.2.min.js
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5794) Fix the asflicense warnings
[
https://issues.apache.org/jira/browse/YARN-5794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15614286#comment-15614286
]
Hadoop QA commented on YARN-5794:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
16s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 34s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | YARN-5794 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12835741/YARN-5794.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit xml |
| uname | Linux ed46283a135a 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12
17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 57187fd |
| Default Java | 1.8.0_101 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/13625/testReport/ |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/13625/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Fix the asflicense warnings
> ---
>
> Key: YARN-5794
> URL: https://issues.apache.org/jira/browse/YARN-5794
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Yiqun Lin
>Assignee: Yiqun Lin
> Attachments: YARN-5794.001.patch
>
>
> There are some asflicense warnings showed when building the jenkins of
> hadoop-common. The warning infos (link address
> https://builds.apache.org/job/PreCommit-HDFS-Build/17336/artifact/patchprocess/patch-asflicense-problems.txt):
> {code}
> Lines that start with ? in the ASF License report indicate files that do
> not have an Apache license header:
> !?
> /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/jt/jquery.jstree.js
> !?
>
[jira] [Updated] (YARN-5794) Fix the asflicense warnings
[
https://issues.apache.org/jira/browse/YARN-5794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated YARN-5794:
Summary: Fix the asflicense warnings (was: Adding Apache License Headers
to fix the asflicense warnings)
> Fix the asflicense warnings
> ---
>
> Key: YARN-5794
> URL: https://issues.apache.org/jira/browse/YARN-5794
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Yiqun Lin
>Assignee: Yiqun Lin
> Attachments: YARN-5794.001.patch
>
>
> There are some asflicense warnings showed when building the jenkins of
> hadoop-common. The warning infos (link address
> https://builds.apache.org/job/PreCommit-HDFS-Build/17336/artifact/patchprocess/patch-asflicense-problems.txt):
> {code}
> Lines that start with ? in the ASF License report indicate files that do
> not have an Apache license header:
> !?
> /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/jt/jquery.jstree.js
> !?
> /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/dt-1.9.4/js/jquery.dataTables.min.js
> !?
> /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/jquery/jquery-ui-1.9.1.custom.min.js
> !?
> /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/jquery/jquery-1.8.2.min.js
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5794) Adding Apache License Headers to fix the asflicense warnings
[
https://issues.apache.org/jira/browse/YARN-5794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated YARN-5794:
Attachment: YARN-5794.001.patch
> Adding Apache License Headers to fix the asflicense warnings
>
>
> Key: YARN-5794
> URL: https://issues.apache.org/jira/browse/YARN-5794
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Yiqun Lin
>Assignee: Yiqun Lin
> Attachments: YARN-5794.001.patch
>
>
> There are some asflicense warnings showed when building the jenkins of
> hadoop-common. The warning infos (link address
> https://builds.apache.org/job/PreCommit-HDFS-Build/17336/artifact/patchprocess/patch-asflicense-problems.txt):
> {code}
> Lines that start with ? in the ASF License report indicate files that do
> not have an Apache license header:
> !?
> /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/jt/jquery.jstree.js
> !?
> /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/dt-1.9.4/js/jquery.dataTables.min.js
> !?
> /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/jquery/jquery-ui-1.9.1.custom.min.js
> !?
> /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/jquery/jquery-1.8.2.min.js
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5276) print more info when event queue is blocked
[
https://issues.apache.org/jira/browse/YARN-5276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15614206#comment-15614206
]
Miklos Szegedi commented on YARN-5276:
--
Thank you for the quick reply [~sandflee]!
It is really optional. I was wondering if we could test at least whether the
log has arrived. I could come up with a draft but it uses reflection. It is up
to you, whether you want to do something similar.
{code}
Log log = Mockito.mock(Log.class);
AsyncDispatcher dispatcher = new AsyncDispatcher();
Field logger = AsyncDispatcher.class.getDeclaredField("LOG");
logger.setAccessible(true);
Field modifiers = Field.class.getDeclaredField("modifiers");
modifiers.setAccessible(true);
modifiers.setInt(logger, logger.getModifiers() & ~Modifier.FINAL );
Object oldLog = logger.get(null);
try {
logger.set(null, log);
dispatcher.init(new Configuration());
dispatcher.register(TestEnum.class, new EventHandler() {
@Override
public void handle(Event event) {
try {
Thread.sleep(1500); // As long as 1 events queued
} catch (InterruptedException e) {
}
}
});
dispatcher.start();
for (int i = 0; i < 1; ++i) {
dispatcher.getEventHandler().handle(new
TestEvent(TestEnum.TestEventType));
}
Mockito.verify(log, Mockito.atLeastOnce()).info("event type:
TestEventType, num: 5000, capacity: 1.0");
dispatcher.stop();
} finally {
//... restore logger object
}
{code}
> print more info when event queue is blocked
> ---
>
> Key: YARN-5276
> URL: https://issues.apache.org/jira/browse/YARN-5276
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager, resourcemanager
>Reporter: sandflee
>Assignee: sandflee
> Labels: oct16-easy
> Attachments: YARN-5276.01.patch, YARN-5276.02.patch,
> YARN-5276.03.patch, YARN-5276.04.patch
>
>
> we now see logs like "Size of event-queue is 498000, Size of event-queue is
> 499000" and difficult to know which event flood the queue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3732) Change NodeHeartbeatResponse.java and RegisterNodeManagerResponse.java as abstract classes
[ https://issues.apache.org/jira/browse/YARN-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15614197#comment-15614197 ] Devaraj K commented on YARN-3732: - ASF License warnings are not related to the patch. > Change NodeHeartbeatResponse.java and RegisterNodeManagerResponse.java as > abstract classes > -- > > Key: YARN-3732 > URL: https://issues.apache.org/jira/browse/YARN-3732 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Devaraj K >Assignee: Devaraj K >Priority: Minor > Attachments: YARN-3732-1.patch, YARN-3732-2.patch, YARN-3732.patch > > > All the other protocol record classes are abstract classes. Change > NodeHeartbeatResponse.java and RegisterNodeManagerResponse.java as abstract > classes to make it consistent with other protocol record classes. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5795) FairScheduler set AppMaster vcores didn't work
[
https://issues.apache.org/jira/browse/YARN-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15614196#comment-15614196
]
tangshangwen commented on YARN-5795:
I think allocate method using DOMINANT RESOURCE CALCULATOR, AppMaster should
also be the same.
{code:title=FairScheduler.java|borderStyle=solid}
@Override
public Allocation allocate(ApplicationAttemptId appAttemptId,
List ask, List release,
List blacklistAdditions, List blacklistRemovals) {
// Make sure this application exists
FSAppAttempt application = getSchedulerApp(appAttemptId);
if (application == null) {
LOG.info("Calling allocate on removed " +
"or non existant application " + appAttemptId);
return EMPTY_ALLOCATION;
}
// Sanity check
SchedulerUtils.normalizeRequests(ask, DOMINANT_RESOURCE_CALCULATOR,
clusterResource, minimumAllocation, getMaximumResourceCapability(),
incrAllocation);
..
}
{code}
> FairScheduler set AppMaster vcores didn't work
> --
>
> Key: YARN-5795
> URL: https://issues.apache.org/jira/browse/YARN-5795
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 2.7.1
>Reporter: tangshangwen
>Assignee: tangshangwen
>
> In our cluster, we use the linux container, I would like to increase the
> number of cpu to get more CPU time slice, but it did not take effect， i set
> yarn.app.mapreduce.am.resource.cpu-vcores = 2 ,but i found the
> resourcemanager log
> {noformat}
> [2016-10-27T16:36:37.280 08:00] [INFO]
> resourcemanager.scheduler.SchedulerNode.allocateContainer(SchedulerNode.java
> 153) [ResourceManager Event Processor] : Assigned container
> container_1477059529836_336635_01_01 of capacity
> {noformat}
> Because scheduler.getResourceCalculator() only computes memory
> {code:title=RMAppManager.java|borderStyle=solid}
> private ResourceRequest validateAndCreateResourceRequest(
> ApplicationSubmissionContext submissionContext, boolean isRecovery)
> throws InvalidResourceRequestException {
> ...
> SchedulerUtils.normalizeRequest(amReq,
> scheduler.getResourceCalculator(),
> scheduler.getClusterResource(),
> scheduler.getMinimumResourceCapability(),
> scheduler.getMaximumResourceCapability(),
> scheduler.getMinimumResourceCapability());
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Issue Comment Deleted] (YARN-5795) FairScheduler set AppMaster vcores didn't work
[
https://issues.apache.org/jira/browse/YARN-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
tangshangwen updated YARN-5795:
---
Comment: was deleted
(was: I think if we replace RESOURCE_CALCULATOR with
DOMINANT_RESOURCE_CALCULATOR we should be able to fix the problem
{code:title=FairScheduler.java|borderStyle=solid}
@Override
public ResourceCalculator getResourceCalculator() {
//return RESOURCE_CALCULATOR;
return DOMINANT_RESOURCE_CALCULATOR;
}
{code})
> FairScheduler set AppMaster vcores didn't work
> --
>
> Key: YARN-5795
> URL: https://issues.apache.org/jira/browse/YARN-5795
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 2.7.1
>Reporter: tangshangwen
>Assignee: tangshangwen
>
> In our cluster, we use the linux container, I would like to increase the
> number of cpu to get more CPU time slice, but it did not take effect， i set
> yarn.app.mapreduce.am.resource.cpu-vcores = 2 ,but i found the
> resourcemanager log
> {noformat}
> [2016-10-27T16:36:37.280 08:00] [INFO]
> resourcemanager.scheduler.SchedulerNode.allocateContainer(SchedulerNode.java
> 153) [ResourceManager Event Processor] : Assigned container
> container_1477059529836_336635_01_01 of capacity
> {noformat}
> Because scheduler.getResourceCalculator() only computes memory
> {code:title=RMAppManager.java|borderStyle=solid}
> private ResourceRequest validateAndCreateResourceRequest(
> ApplicationSubmissionContext submissionContext, boolean isRecovery)
> throws InvalidResourceRequestException {
> ...
> SchedulerUtils.normalizeRequest(amReq,
> scheduler.getResourceCalculator(),
> scheduler.getClusterResource(),
> scheduler.getMinimumResourceCapability(),
> scheduler.getMaximumResourceCapability(),
> scheduler.getMinimumResourceCapability());
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-3059) RM web page can not display NM's health report which is healthy
[ https://issues.apache.org/jira/browse/YARN-3059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S reassigned YARN-3059: --- Assignee: (was: Rohith Sharma K S) > RM web page can not display NM's health report which is healthy > --- > > Key: YARN-3059 > URL: https://issues.apache.org/jira/browse/YARN-3059 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 2.5.1 >Reporter: Wang Hao > > If the NM is healthy, it's health report can not display in RM web page. > In function reportHealthStatus of NodeHealthMonitorExecutor, I found that if > the HealthCheckerExitStatus is successful, the output was set to a empty > string in function setHealthStatus. > so，I change the code “setHealthStatus(true, "", now)” to > “setHealthStatus(true,shexec.getOutput(), now)” . Then, the RM web page can > display the NM's health report. > Maybe set the output to a empty string can decrease the data that transfered > between RM and NM. But I think we want to see the health report of NM in some > cases. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-5792) adopt the id prefix for YARN, MR, and DS entities
[ https://issues.apache.org/jira/browse/YARN-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena reassigned YARN-5792: -- Assignee: Varun Saxena > adopt the id prefix for YARN, MR, and DS entities > - > > Key: YARN-5792 > URL: https://issues.apache.org/jira/browse/YARN-5792 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-5355 >Reporter: Sangjin Lee >Assignee: Varun Saxena > > We introduced the entity id prefix to support flexible entity sorting > (YARN-5715). We should adopt the id prefix for YARN entities, MR entities, > and DS entities to take advantage of the id prefix. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5715) introduce entity prefix for return and sort order
[ https://issues.apache.org/jira/browse/YARN-5715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15614157#comment-15614157 ] Rohith Sharma K S commented on YARN-5715: - Thanks [~sjlee0] [~varun_saxena] [~gtCarrera9] for your support moving this to committed state. Let us discuss on reader API in YARN-5585. > introduce entity prefix for return and sort order > - > > Key: YARN-5715 > URL: https://issues.apache.org/jira/browse/YARN-5715 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Sangjin Lee >Assignee: Rohith Sharma K S >Priority: Critical > Labels: oct16-easy > Fix For: YARN-5355 > > Attachments: YARN-5715-YARN-5355.01.patch, > YARN-5715-YARN-5355.02.patch, YARN-5715-YARN-5355.03.patch, > YARN-5715-YARN-5355.04.patch, YARN-5715-YARN-5355.05.patch, > YARN-5715-YARN-5355.06.patch > > > While looking into YARN-5585, we have come across the need to provide a sort > order different than the current entity id order. The current entity id order > returns entities strictly in the lexicographical order, and as such it > returns the earliest entities first. This may not be the most natural return > order. A more natural return/sort order would be from the most recent > entities. > To solve this, we would like to add what we call the "entity prefix" in the > row key for the entity table. It is a number (long) that can be easily > provided by the client on write. In the row key, it would be added before the > entity id itself. > The entity prefix would be considered mandatory. On all writes (including > updates) the correct entity prefix should be set by the client so that the > correct row key is used. The entity prefix needs to be unique only within the > scope of the application and the entity type. > For queries that return a list of entities, the prefix values will be > returned along with the entity id's. Queries that specify the prefix and the > id should be returned quickly using the row key. If the query omits the > prefix but specifies the id (query by id), the query may be less efficient. > This JIRA should add the entity prefix to the entity API and add its handling > to the schema and the write path. The read path will be addressed in > YARN-5585. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5795) FairScheduler set AppMaster vcores didn't work
[
https://issues.apache.org/jira/browse/YARN-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15614149#comment-15614149
]
tangshangwen commented on YARN-5795:
I think if we replace RESOURCE_CALCULATOR with DOMINANT_RESOURCE_CALCULATOR we
should be able to fix the problem
{code:title=FairScheduler.java|borderStyle=solid}
@Override
public ResourceCalculator getResourceCalculator() {
//return RESOURCE_CALCULATOR;
return DOMINANT_RESOURCE_CALCULATOR;
}
{code}
> FairScheduler set AppMaster vcores didn't work
> --
>
> Key: YARN-5795
> URL: https://issues.apache.org/jira/browse/YARN-5795
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 2.7.1
>Reporter: tangshangwen
>Assignee: tangshangwen
>
> In our cluster, we use the linux container, I would like to increase the
> number of cpu to get more CPU time slice, but it did not take effect， i set
> yarn.app.mapreduce.am.resource.cpu-vcores = 2 ,but i found the
> resourcemanager log
> {noformat}
> [2016-10-27T16:36:37.280 08:00] [INFO]
> resourcemanager.scheduler.SchedulerNode.allocateContainer(SchedulerNode.java
> 153) [ResourceManager Event Processor] : Assigned container
> container_1477059529836_336635_01_01 of capacity
> {noformat}
> Because scheduler.getResourceCalculator() only computes memory
> {code:title=RMAppManager.java|borderStyle=solid}
> private ResourceRequest validateAndCreateResourceRequest(
> ApplicationSubmissionContext submissionContext, boolean isRecovery)
> throws InvalidResourceRequestException {
> ...
> SchedulerUtils.normalizeRequest(amReq,
> scheduler.getResourceCalculator(),
> scheduler.getClusterResource(),
> scheduler.getMinimumResourceCapability(),
> scheduler.getMaximumResourceCapability(),
> scheduler.getMinimumResourceCapability());
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4734) Merge branch:YARN-3368 to trunk
[
https://issues.apache.org/jira/browse/YARN-4734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15614142#comment-15614142
]
Sunil G commented on YARN-4734:
---
HI [~aw]
Please access UI at {{/8088/ui2}}. {{/cluster}} is present by
default which will serve old UI in both conditions.
> Merge branch:YARN-3368 to trunk
> ---
>
> Key: YARN-4734
> URL: https://issues.apache.org/jira/browse/YARN-4734
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Wangda Tan
>Assignee: Wangda Tan
> Attachments: YARN-4734.1.patch, YARN-4734.10-NOT_READY.patch,
> YARN-4734.11-NOT_READY.patch, YARN-4734.12-NOT_READY.patch,
> YARN-4734.13.patch, YARN-4734.14.patch, YARN-4734.15.patch,
> YARN-4734.16.patch, YARN-4734.17-rebased.patch, YARN-4734.17.patch,
> YARN-4734.2.patch, YARN-4734.3.patch, YARN-4734.4.patch, YARN-4734.5.patch,
> YARN-4734.6.patch, YARN-4734.7.patch, YARN-4734.8.patch,
> YARN-4734.9-NOT_READY.patch
>
>
> YARN-2928 branch is planned to merge back to trunk shortly, it depends on
> changes of YARN-3368. This JIRA is to track the merging task.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-5795) FairScheduler set AppMaster vcores didn't work
tangshangwen created YARN-5795:
--
Summary: FairScheduler set AppMaster vcores didn't work
Key: YARN-5795
URL: https://issues.apache.org/jira/browse/YARN-5795
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Affects Versions: 2.7.1
Reporter: tangshangwen
Assignee: tangshangwen
In our cluster, we use the linux container, I would like to increase the number
of cpu to get more CPU time slice, but it did not take effect， i set
yarn.app.mapreduce.am.resource.cpu-vcores = 2 ,but i found the resourcemanager
log
{noformat}
[2016-10-27T16:36:37.280 08:00] [INFO]
resourcemanager.scheduler.SchedulerNode.allocateContainer(SchedulerNode.java
153) [ResourceManager Event Processor] : Assigned container
container_1477059529836_336635_01_01 of capacity
{noformat}
Because scheduler.getResourceCalculator() only computes memory
{code:title=RMAppManager.java|borderStyle=solid}
private ResourceRequest validateAndCreateResourceRequest(
ApplicationSubmissionContext submissionContext, boolean isRecovery)
throws InvalidResourceRequestException {
...
SchedulerUtils.normalizeRequest(amReq, scheduler.getResourceCalculator(),
scheduler.getClusterResource(),
scheduler.getMinimumResourceCapability(),
scheduler.getMaximumResourceCapability(),
scheduler.getMinimumResourceCapability());
...
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-4498) Application level node labels stats to be available in REST
[ https://issues.apache.org/jira/browse/YARN-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bibin A Chundatt updated YARN-4498: --- Description: Currently nodelabel stats per application is not available through REST like currently used labels by all live containers, total stats of containers per label for app etc.. CLI and web UI scenarios will be handled separately. was: Currently nodelabel stats per application is not available through REST/CLI/UI like currently used labels by all live containers, total stats of containers per label for app etc.. Will split into multiple task if approach is fine. Summary: Application level node labels stats to be available in REST (was: Application level node labels stats to be available in UI/REST/CLI) > Application level node labels stats to be available in REST > --- > > Key: YARN-4498 > URL: https://issues.apache.org/jira/browse/YARN-4498 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, client, resourcemanager >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Labels: oct16-medium > Attachments: 0001-YARN-4498.patch, YARN-4498.0002.patch, apps.xml > > > Currently nodelabel stats per application is not available through REST like > currently used labels by all live containers, total stats of containers per > label for app etc.. > CLI and web UI scenarios will be handled separately. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4396) Log the trance information on FSAppAttempt#assignContainer
[ https://issues.apache.org/jira/browse/YARN-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15614092#comment-15614092 ] Daniel Templeton commented on YARN-4396: The latest patch looks good to me. +1 If no one has any other comments, I'll commit it on Monday. > Log the trance information on FSAppAttempt#assignContainer > -- > > Key: YARN-4396 > URL: https://issues.apache.org/jira/browse/YARN-4396 > Project: Hadoop YARN > Issue Type: Improvement > Components: applications, fairscheduler >Affects Versions: 2.7.1 >Reporter: Yiqun Lin >Assignee: Yiqun Lin > Labels: oct16-easy > Attachments: YARN-4396.001.patch, YARN-4396.002.patch, > YARN-4396.003.patch, YARN-4396.004.patch, YARN-4396.005.patch > > > When I configure the yarn.scheduler.fair.locality.threshold.node and > yarn.scheduler.fair.locality.threshold.rack to open this function, I have no > detail info of assigning container's locality. And it's important because it > will lead some delay scheduling and will have an influence on my cluster. If > I know these info, I can adjust param in cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5794) Adding Apache License Headers to fix the asflicense warnings
[
https://issues.apache.org/jira/browse/YARN-5794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated YARN-5794:
Description:
There are some asflicense warnings showed when building the jenkins of
hadoop-common. The warning infos (link address
https://builds.apache.org/job/PreCommit-HDFS-Build/17336/artifact/patchprocess/patch-asflicense-problems.txt):
{code}
Lines that start with ? in the ASF License report indicate files that do
not have an Apache license header:
!?
/testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/jt/jquery.jstree.js
!?
/testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/dt-1.9.4/js/jquery.dataTables.min.js
!?
/testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/jquery/jquery-ui-1.9.1.custom.min.js
!?
/testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/jquery/jquery-1.8.2.min.js
{code}
was:
There are some asflicense warnings showed when building the jenkins of
hadoop-common. The warning infos:
{code}
Lines that start with ? in the ASF License report indicate files that do
not have an Apache license header:
!?
/testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/jt/jquery.jstree.js
!?
/testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/dt-1.9.4/js/jquery.dataTables.min.js
!?
/testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/jquery/jquery-ui-1.9.1.custom.min.js
!?
/testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/jquery/jquery-1.8.2.min.js
{code}
> Adding Apache License Headers to fix the asflicense warnings
>
>
> Key: YARN-5794
> URL: https://issues.apache.org/jira/browse/YARN-5794
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Yiqun Lin
>Assignee: Yiqun Lin
>
> There are some asflicense warnings showed when building the jenkins of
> hadoop-common. The warning infos (link address
> https://builds.apache.org/job/PreCommit-HDFS-Build/17336/artifact/patchprocess/patch-asflicense-problems.txt):
> {code}
> Lines that start with ? in the ASF License report indicate files that do
> not have an Apache license header:
> !?
> /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/jt/jquery.jstree.js
> !?
> /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/dt-1.9.4/js/jquery.dataTables.min.js
> !?
> /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/jquery/jquery-ui-1.9.1.custom.min.js
> !?
> /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/jquery/jquery-1.8.2.min.js
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5794) Adding Apache License Headers to fix the asflicense warnings
[
https://issues.apache.org/jira/browse/YARN-5794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15614067#comment-15614067
]
Yiqun Lin commented on YARN-5794:
-
Hi [~ajisakaa], I'd like ping you for some help. Is this mean that I just add
the ASF header in thess related files? Or there is some other way? I haven't
fixed this type issue, so I am not so sure for this. Thanks!
> Adding Apache License Headers to fix the asflicense warnings
>
>
> Key: YARN-5794
> URL: https://issues.apache.org/jira/browse/YARN-5794
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Yiqun Lin
>Assignee: Yiqun Lin
>
> There are some asflicense warnings showed when building the jenkins of
> hadoop-common. The warning infos:
> {code}
> Lines that start with ? in the ASF License report indicate files that do
> not have an Apache license header:
> !?
> /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/jt/jquery.jstree.js
> !?
> /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/dt-1.9.4/js/jquery.dataTables.min.js
> !?
> /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/jquery/jquery-ui-1.9.1.custom.min.js
> !?
> /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/jquery/jquery-1.8.2.min.js
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-5794) Adding Apache License Headers to fix the asflicense warnings
Yiqun Lin created YARN-5794:
---
Summary: Adding Apache License Headers to fix the asflicense
warnings
Key: YARN-5794
URL: https://issues.apache.org/jira/browse/YARN-5794
Project: Hadoop YARN
Issue Type: Bug
Reporter: Yiqun Lin
Assignee: Yiqun Lin
There are some asflicense warnings showed when building the jenkins of
hadoop-common. The warning infos:
{code}
Lines that start with ? in the ASF License report indicate files that do
not have an Apache license header:
!?
/testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/jt/jquery.jstree.js
!?
/testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/dt-1.9.4/js/jquery.dataTables.min.js
!?
/testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/jquery/jquery-ui-1.9.1.custom.min.js
!?
/testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/jquery/jquery-1.8.2.min.js
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-4396) Log the trance information on FSAppAttempt#assignContainer
[
https://issues.apache.org/jira/browse/YARN-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated YARN-4396:
Attachment: YARN-4396.005.patch
Thanks [~templedf] for the comments. Your comments look good to me. Make two
changes in the latets patch:
* Change log level from {{debug}} to {{trace}}.
* Do not print the log when the locality doesn't change.
Thanks for reviewing!
> Log the trance information on FSAppAttempt#assignContainer
> --
>
> Key: YARN-4396
> URL: https://issues.apache.org/jira/browse/YARN-4396
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: applications, fairscheduler
>Affects Versions: 2.7.1
>Reporter: Yiqun Lin
>Assignee: Yiqun Lin
> Labels: oct16-easy
> Attachments: YARN-4396.001.patch, YARN-4396.002.patch,
> YARN-4396.003.patch, YARN-4396.004.patch, YARN-4396.005.patch
>
>
> When I configure the yarn.scheduler.fair.locality.threshold.node and
> yarn.scheduler.fair.locality.threshold.rack to open this function, I have no
> detail info of assigning container's locality. And it's important because it
> will lead some delay scheduling and will have an influence on my cluster. If
> I know these info, I can adjust param in cluster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-2664) Improve RM webapp to expose info about reservations.
[
https://issues.apache.org/jira/browse/YARN-2664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15614018#comment-15614018
]
Hadoop QA commented on YARN-2664:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
8s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 10m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
26s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m
27s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 6m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 7m
52s{color} | {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
51s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 1m
9s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}102m 33s{color}
| {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 6m
23s{color} | {color:red} The patch generated 8 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}205m 46s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA |
| | hadoop.hdfs.server.datanode.TestDirectoryScanner |
| | hadoop.hdfs.server.namenode.TestCheckpoint |
| |
hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager
|
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | YARN-2664 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12819668/YARN-2664.17.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle cc |
| uname | Linux fd2078eb001b 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12
17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 022bf78 |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| mvnsite |
[jira] [Commented] (YARN-3934) Application with large ApplicationSubmissionContext can cause RM to exit when ZK store is used
[
https://issues.apache.org/jira/browse/YARN-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15614013#comment-15614013
]
Dustin Cote commented on YARN-3934:
---
[~templedf] feel free to take it over if you'd like. I'm not going to have a
chance to address this for the foreseeable future.
> Application with large ApplicationSubmissionContext can cause RM to exit when
> ZK store is used
> --
>
> Key: YARN-3934
> URL: https://issues.apache.org/jira/browse/YARN-3934
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Reporter: Ming Ma
>Assignee: Dustin Cote
> Labels: oct16-easy
> Attachments: YARN-3934-1.patch
>
>
> Use the following steps to test.
> 1. Set up ZK as the RM HA store.
> 2. Submit a job that refers to lots of distributed cache files with long HDFS
> path, which will cause the app state size to exceed ZK's max object size
> limit.
> 3. RM can't write to ZK and exit with the following exception.
> {noformat}
> 2015-07-10 22:21:13,002 FATAL
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Received a
> org.apache.hadoop.yarn.server.resourcemanager.RMFatalEvent of type
> STATE_STORE_OP_FAILED. Cause:
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode
> = Session expired
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> at org.apache.zookeeper.ZooKeeper.multiInternal(ZooKeeper.java:935)
> at org.apache.zookeeper.ZooKeeper.multi(ZooKeeper.java:915)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$4.run(ZKRMStateStore.java:944)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$4.run(ZKRMStateStore.java:941)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$ZKAction.runWithCheck(ZKRMStateStore.java:1083)
> {noformat}
> In this case, RM could have rejected the app during submitApplication RPC if
> the size of ApplicationSubmissionContext is too large.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4043) Change logging of warning message : "an attempt to override final parameter:"
[
https://issues.apache.org/jira/browse/YARN-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613983#comment-15613983
]
Hadoop QA commented on YARN-4043:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
7s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
30s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 15m
2s{color} | {color:red} The patch generated 4 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 54m 32s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | YARN-4043 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12749681/warn-msg.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux b6ed3e6540a4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 4df8ed6 |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/13621/testReport/ |
| asflicense |
https://builds.apache.org/job/PreCommit-YARN-Build/13621/artifact/patchprocess/patch-asflicense-problems.txt
|
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/13621/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Change logging of warning message : "an attempt to override final parameter:"
> -
>
> Key: YARN-4043
> URL: https://issues.apache.org/jira/browse/YARN-4043
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Spandan Dutta
>Assignee: Spandan
[jira] [Commented] (YARN-5793) Trim configuration values in DockerLinuxContainerRuntime
[
https://issues.apache.org/jira/browse/YARN-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613975#comment-15613975
]
Hadoop QA commented on YARN-5793:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 17s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
The patch generated 1 new + 4 unchanged - 0 fixed = 5 total (was 4) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 15m 11s{color}
| {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 16m
4s{color} | {color:red} The patch generated 4 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 46m 24s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager
|
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | YARN-5793 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12835705/YARN-5793.0001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 1d10bcc7c169 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12
17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / b62bc2b |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/13622/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
|
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/13622/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/13622/testReport/ |
| asflicense |
https://builds.apache.org/job/PreCommit-YARN-Build/13622/artifact/patchprocess/patch-asflicense-problems.txt
|
| modules | C:
[jira] [Commented] (YARN-3460) Test TestSecureRMRegistryOperations failed with IBM_JAVA JVM
[
https://issues.apache.org/jira/browse/YARN-3460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613955#comment-15613955
]
Hadoop QA commented on YARN-3460:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 12s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry: The patch generated 88
new + 101 unchanged - 4 fixed = 189 total (was 105) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
51s{color} | {color:green} hadoop-yarn-registry in the patch passed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 16m
24s{color} | {color:red} The patch generated 4 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 25s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | YARN-3460 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12835706/YARN-3460.004.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 23c68ab93784 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18
21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 57187fd |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/13623/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-registry.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/13623/testReport/ |
| asflicense |
https://builds.apache.org/job/PreCommit-YARN-Build/13623/artifact/patchprocess/patch-asflicense-problems.txt
|
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/13623/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Test TestSecureRMRegistryOperations failed with IBM_JAVA JVM
>
>
> Key: YARN-3460
> URL:
[jira] [Commented] (YARN-5525) Make log aggregation service class configurable
[
https://issues.apache.org/jira/browse/YARN-5525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613940#comment-15613940
]
Hadoop QA commented on YARN-5525:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
5s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
16s{color} | {color:green} hadoop-yarn-project_hadoop-yarn generated 0 new + 33
unchanged - 1 fixed = 33 total (was 34) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 42s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 56 new + 357 unchanged - 60 fixed = 413 total (was 417) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
57s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
26s{color} | {color:red} hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common
generated 7 new + 4579 unchanged - 0 fixed = 4586 total (was 4579) {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 26s{color}
| {color:red} hadoop-yarn-api in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
17s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 14m 54s{color}
| {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 15m
14s{color} | {color:red} The patch generated 4 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 67m 53s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.conf.TestYarnConfigurationFields |
| |
hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager
|
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | YARN-5525 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12835702/YARN-5525.v4.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux abc3229cd057 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12
17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 28660f5 |
| Default Java | 1.8.0_101 |
| findbugs |
[jira] [Commented] (YARN-5259) Add two metrics at FSOpDurations for doing container assign and completed Performance statistical analysis
[
https://issues.apache.org/jira/browse/YARN-5259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613908#comment-15613908
]
Daniel Templeton commented on YARN-5259:
While I don't think you could do anything too exacting, I think adding tests to
{{TestFairScheduler}} would be a good idea. The more we can do to prevent
things from breaking later down the road, the better off we'll be.
> Add two metrics at FSOpDurations for doing container assign and completed
> Performance statistical analysis
> --
>
> Key: YARN-5259
> URL: https://issues.apache.org/jira/browse/YARN-5259
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: fairscheduler
>Reporter: ChenFolin
> Labels: oct16-easy
> Attachments: YARN-5259-001.patch, YARN-5259-002.patch,
> YARN-5259-003.patch, YARN-5259-004.patch
>
>
> If cluster is slow , we can not know Whether it is caused by container assign
> or completed performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4743) FairSharePolicy breaks TimSort assumption
[
https://issues.apache.org/jira/browse/YARN-4743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613896#comment-15613896
]
Hudson commented on YARN-4743:
--
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10715 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/10715/])
YARN-4743. FairSharePolicy breaks TimSort assumption. (Zephyr Guo and (kasha:
rev 4df8ed63ed93f2542e4b48f521b0cc6624ab59c1)
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/policies/FairSharePolicy.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestSchedulingPolicy.java
> FairSharePolicy breaks TimSort assumption
> -
>
> Key: YARN-4743
> URL: https://issues.apache.org/jira/browse/YARN-4743
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 3.0.0-alpha2
>Reporter: Zephyr Guo
>Assignee: Zephyr Guo
> Fix For: 3.0.0-alpha2
>
> Attachments: YARN-4743-v1.patch, YARN-4743-v2.patch,
> YARN-4743-v3.patch, YARN-4743-v4.patch, YARN-4743-v5.patch, timsort.log
>
>
> {code}
> 2016-02-26 14:08:50,821 FATAL
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error in
> handling event type NODE_UPDATE to the scheduler
> java.lang.IllegalArgumentException: Comparison method violates its general
> contract!
> at java.util.TimSort.mergeHi(TimSort.java:868)
> at java.util.TimSort.mergeAt(TimSort.java:485)
> at java.util.TimSort.mergeCollapse(TimSort.java:410)
> at java.util.TimSort.sort(TimSort.java:214)
> at java.util.TimSort.sort(TimSort.java:173)
> at java.util.Arrays.sort(Arrays.java:659)
> at java.util.Collections.sort(Collections.java:217)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSLeafQueue.assignContainer(FSLeafQueue.java:316)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSParentQueue.assignContainer(FSParentQueue.java:240)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.attemptScheduling(FairScheduler.java:1091)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.nodeUpdate(FairScheduler.java:989)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:1185)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:112)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:684)
> at java.lang.Thread.run(Thread.java:745)
> 2016-02-26 14:08:50,822 INFO
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Exiting, bbye..
> {code}
> Actually, this bug found in 2.6.0-cdh5.4.7. {{FairShareComparator}} is not
> transitive.
> We get NaN when memorySize=0 and weight=0.
> {code:title=FairSharePolicy.java}
> useToWeightRatio1 = s1.getResourceUsage().getMemorySize() /
> s1.getWeights().getWeight(ResourceType.MEMORY)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-2306) Add test for leakage of reservation metrics in fair scheduler
[ https://issues.apache.org/jira/browse/YARN-2306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613894#comment-15613894 ] Hudson commented on YARN-2306: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10715 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/10715/]) YARN-2306. Add test for leakage of reservation metrics in fair (subu: rev b2c4f24c31e73faa8f71d44db5de3aa91e3b7d5e) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java > Add test for leakage of reservation metrics in fair scheduler > - > > Key: YARN-2306 > URL: https://issues.apache.org/jira/browse/YARN-2306 > Project: Hadoop YARN > Issue Type: Bug > Components: fairscheduler >Reporter: Hong Zhiguo >Assignee: Hong Zhiguo >Priority: Minor > Labels: oct16-easy > Fix For: 2.9.0, 3.0.0-alpha2 > > Attachments: YARN-2306-2.patch, YARN-2306-3.patch, > YARN-2306.004.patch, YARN-2306.patch > > > This only applies to fair scheduler. Capacity scheduler is OK. > When appAttempt or node is removed, the metrics for > reservation(reservedContainers, reservedMB, reservedVCores) is not reduced > back. > These are important metrics for administrator. The wrong metrics confuses may > confuse them. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4033) In FairScheduler, parent queues should also display queue status
[ https://issues.apache.org/jira/browse/YARN-4033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613878#comment-15613878 ] Gour Saha commented on YARN-4033: - [~vinodkv] or [~kasha] can you look into the patch and commit it, if it looks good to you as well. > In FairScheduler, parent queues should also display queue status > - > > Key: YARN-4033 > URL: https://issues.apache.org/jira/browse/YARN-4033 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Reporter: Siqi Li >Assignee: Siqi Li > Labels: oct16-easy > Attachments: Screen Shot 2015-08-07 at 2.04.04 PM.png, > YARN-4033.v1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5265) Make HBase configuration for the timeline service configurable
[
https://issues.apache.org/jira/browse/YARN-5265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613877#comment-15613877
]
Hadoop QA commented on YARN-5265:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 4m
16s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
15s{color} | {color:green} YARN-5355 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
24s{color} | {color:green} YARN-5355 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} YARN-5355 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
36s{color} | {color:green} YARN-5355 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
50s{color} | {color:green} YARN-5355 passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
29s{color} | {color:green} YARN-5355 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
16s{color} | {color:green} YARN-5355 passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
26s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
16s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
49s{color} | {color:green} hadoop-yarn-server-timelineservice in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
9s{color} | {color:green} hadoop-yarn-site in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 47m 5s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | YARN-5265 |
| JIRA Patch URL |
[jira] [Commented] (YARN-4033) In FairScheduler, parent queues should also display queue status
[ https://issues.apache.org/jira/browse/YARN-4033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613874#comment-15613874 ] Gour Saha commented on YARN-4033: - This patch still applies cleanly on trunk. I ran all resourcemanager webapp tests and they ran successfully. The failed tests in the report does not seem to be related to this patch. > In FairScheduler, parent queues should also display queue status > - > > Key: YARN-4033 > URL: https://issues.apache.org/jira/browse/YARN-4033 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Reporter: Siqi Li >Assignee: Siqi Li > Labels: oct16-easy > Attachments: Screen Shot 2015-08-07 at 2.04.04 PM.png, > YARN-4033.v1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5435) [Regression] QueueCapacities not being updated for dynamic ReservationQueue
[
https://issues.apache.org/jira/browse/YARN-5435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613864#comment-15613864
]
Hadoop QA commented on YARN-5435:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
56s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 0 new + 18 unchanged - 2 fixed = 18 total (was 20) {color}
|
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 36m 28s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 14m
33s{color} | {color:red} The patch generated 4 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 66m 35s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.reservation.TestCapacitySchedulerPlanFollower
|
| | hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart |
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerDynamicBehavior
|
| | hadoop.yarn.server.resourcemanager.TestReservationSystemWithRMHA |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | YARN-5435 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12835693/YARN-5435.v003.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 7e54fe69ea02 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 5877f20 |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/13616/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/13616/testReport/ |
| asflicense |
https://builds.apache.org/job/PreCommit-YARN-Build/13616/artifact/patchprocess/patch-asflicense-problems.txt
|
| modules | C:
[jira] [Commented] (YARN-3732) Change NodeHeartbeatResponse.java and RegisterNodeManagerResponse.java as abstract classes
[
https://issues.apache.org/jira/browse/YARN-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613860#comment-15613860
]
Hadoop QA commented on YARN-3732:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common:
The patch generated 0 new + 34 unchanged - 13 fixed = 34 total (was 47) {color}
|
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
28s{color} | {color:green} hadoop-yarn-server-common in the patch passed.
{color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 15m
5s{color} | {color:red} The patch generated 4 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 39s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | YARN-3732 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12835704/YARN-3732-2.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux f16cbb7889e5 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29
20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 28660f5 |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/13620/testReport/ |
| asflicense |
https://builds.apache.org/job/PreCommit-YARN-Build/13620/artifact/patchprocess/patch-asflicense-problems.txt
|
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/13620/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Change NodeHeartbeatResponse.java and RegisterNodeManagerResponse.java as
> abstract classes
>
[jira] [Commented] (YARN-4743) FairSharePolicy breaks TimSort assumption
[
https://issues.apache.org/jira/browse/YARN-4743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613851#comment-15613851
]
Zephyr Guo commented on YARN-4743:
--
Thanks [~yufeigu], [~kasha] for reviewing!
> FairSharePolicy breaks TimSort assumption
> -
>
> Key: YARN-4743
> URL: https://issues.apache.org/jira/browse/YARN-4743
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 3.0.0-alpha2
>Reporter: Zephyr Guo
>Assignee: Zephyr Guo
> Fix For: 3.0.0-alpha2
>
> Attachments: YARN-4743-v1.patch, YARN-4743-v2.patch,
> YARN-4743-v3.patch, YARN-4743-v4.patch, YARN-4743-v5.patch, timsort.log
>
>
> {code}
> 2016-02-26 14:08:50,821 FATAL
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error in
> handling event type NODE_UPDATE to the scheduler
> java.lang.IllegalArgumentException: Comparison method violates its general
> contract!
> at java.util.TimSort.mergeHi(TimSort.java:868)
> at java.util.TimSort.mergeAt(TimSort.java:485)
> at java.util.TimSort.mergeCollapse(TimSort.java:410)
> at java.util.TimSort.sort(TimSort.java:214)
> at java.util.TimSort.sort(TimSort.java:173)
> at java.util.Arrays.sort(Arrays.java:659)
> at java.util.Collections.sort(Collections.java:217)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSLeafQueue.assignContainer(FSLeafQueue.java:316)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSParentQueue.assignContainer(FSParentQueue.java:240)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.attemptScheduling(FairScheduler.java:1091)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.nodeUpdate(FairScheduler.java:989)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:1185)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:112)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:684)
> at java.lang.Thread.run(Thread.java:745)
> 2016-02-26 14:08:50,822 INFO
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Exiting, bbye..
> {code}
> Actually, this bug found in 2.6.0-cdh5.4.7. {{FairShareComparator}} is not
> transitive.
> We get NaN when memorySize=0 and weight=0.
> {code:title=FairSharePolicy.java}
> useToWeightRatio1 = s1.getResourceUsage().getMemorySize() /
> s1.getWeights().getWeight(ResourceType.MEMORY)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5276) print more info when event queue is blocked
[
https://issues.apache.org/jira/browse/YARN-5276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613844#comment-15613844
]
Hadoop QA commented on YARN-5276:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
56s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
15s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 16m
42s{color} | {color:red} The patch generated 4 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 33m 44s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | YARN-5276 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12835697/YARN-5276.04.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux f32988622f9b 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18
21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 5877f20 |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/13617/testReport/ |
| asflicense |
https://builds.apache.org/job/PreCommit-YARN-Build/13617/artifact/patchprocess/patch-asflicense-problems.txt
|
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/13617/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> print more info when event queue is blocked
> ---
>
> Key: YARN-5276
> URL: https://issues.apache.org/jira/browse/YARN-5276
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager, resourcemanager
>Reporter: sandflee
>
[jira] [Updated] (YARN-4743) FairSharePolicy breaks TimSort assumption
[
https://issues.apache.org/jira/browse/YARN-4743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karthik Kambatla updated YARN-4743:
---
Labels: (was: oct16-easy)
> FairSharePolicy breaks TimSort assumption
> -
>
> Key: YARN-4743
> URL: https://issues.apache.org/jira/browse/YARN-4743
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 3.0.0-alpha2
>Reporter: Zephyr Guo
>Assignee: Zephyr Guo
> Fix For: 3.0.0-alpha2
>
> Attachments: YARN-4743-v1.patch, YARN-4743-v2.patch,
> YARN-4743-v3.patch, YARN-4743-v4.patch, YARN-4743-v5.patch, timsort.log
>
>
> {code}
> 2016-02-26 14:08:50,821 FATAL
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error in
> handling event type NODE_UPDATE to the scheduler
> java.lang.IllegalArgumentException: Comparison method violates its general
> contract!
> at java.util.TimSort.mergeHi(TimSort.java:868)
> at java.util.TimSort.mergeAt(TimSort.java:485)
> at java.util.TimSort.mergeCollapse(TimSort.java:410)
> at java.util.TimSort.sort(TimSort.java:214)
> at java.util.TimSort.sort(TimSort.java:173)
> at java.util.Arrays.sort(Arrays.java:659)
> at java.util.Collections.sort(Collections.java:217)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSLeafQueue.assignContainer(FSLeafQueue.java:316)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSParentQueue.assignContainer(FSParentQueue.java:240)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.attemptScheduling(FairScheduler.java:1091)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.nodeUpdate(FairScheduler.java:989)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:1185)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:112)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:684)
> at java.lang.Thread.run(Thread.java:745)
> 2016-02-26 14:08:50,822 INFO
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Exiting, bbye..
> {code}
> Actually, this bug found in 2.6.0-cdh5.4.7. {{FairShareComparator}} is not
> transitive.
> We get NaN when memorySize=0 and weight=0.
> {code:title=FairSharePolicy.java}
> useToWeightRatio1 = s1.getResourceUsage().getMemorySize() /
> s1.getWeights().getWeight(ResourceType.MEMORY)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5410) Bootstrap Router module
[
https://issues.apache.org/jira/browse/YARN-5410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613825#comment-15613825
]
Hadoop QA commented on YARN-5410:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
52s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
8s{color} | {color:green} YARN-2915 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
22s{color} | {color:green} YARN-2915 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
41s{color} | {color:green} YARN-2915 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 5m
50s{color} | {color:green} YARN-2915 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
23s{color} | {color:green} YARN-2915 passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server
hadoop-yarn-project {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} YARN-2915 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
51s{color} | {color:green} YARN-2915 passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
44s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 36s{color} | {color:orange} root: The patch generated 1 new + 0 unchanged -
0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 6m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server
hadoop-yarn-project {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
17s{color} | {color:red} hadoop-yarn-server-router in the patch failed. {color}
|
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
13s{color} | {color:green} hadoop-project in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 65m 16s{color}
| {color:red} hadoop-yarn-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
44s{color} | {color:green} hadoop-yarn-server-router in the patch passed.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 3m 49s{color}
| {color:red} hadoop-yarn-project in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}151m 50s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.server.TestMiniYarnClusterNodeUtilization |
| | hadoop.yarn.server.TestContainerManagerSecurity |
| |
[jira] [Updated] (YARN-3460) Test TestSecureRMRegistryOperations failed with IBM_JAVA JVM
[
https://issues.apache.org/jira/browse/YARN-3460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Douglas updated YARN-3460:
Attachment: YARN-3460.004.patch
This change invokes the "login" method, when "commit" is intended:
{noformat}
-boolean commitOk = krb5LoginModule.commit();
+Method methodCommit = kerb5LoginObject.getClass().getMethod("commit");
+boolean commitOk = (Boolean) methodLogin.invoke(kerb5LoginObject);
{noformat}
Updated patch. Can someone test this on an IBM jdk?
> Test TestSecureRMRegistryOperations failed with IBM_JAVA JVM
>
>
> Key: YARN-3460
> URL: https://issues.apache.org/jira/browse/YARN-3460
> Project: Hadoop YARN
> Issue Type: Test
>Affects Versions: 2.6.0, 3.0.0-alpha1
> Environment: $ mvn -version
> Apache Maven 3.2.1 (ea8b2b07643dbb1b84b6d16e1f08391b666bc1e9;
> 2014-02-14T11:37:52-06:00)
> Maven home: /opt/apache-maven-3.2.1
> Java version: 1.7.0, vendor: IBM Corporation
> Java home: /usr/lib/jvm/ibm-java-ppc64le-71/jre
> Default locale: en_US, platform encoding: UTF-8
> OS name: "linux", version: "3.10.0-229.ael7b.ppc64le", arch: "ppc64le",
> family: "unix"
>Reporter: pascal oliva
> Attachments: HADOOP-11810-1.patch, YARN-3460-1.patch,
> YARN-3460-2.patch, YARN-3460-3.patch, YARN-3460.004.patch
>
>
> TestSecureRMRegistryOperations failed with JBM IBM JAVA
> mvn test -X
> -Dtest=org.apache.hadoop.registry.secure.TestSecureRMRegistryOperations
> ModuleTotal Failure Error Skipped
> -
> hadoop-yarn-registry 12 0 12 0
> -
> Total 12 0 12 0
> With
> javax.security.auth.login.LoginException: Bad JAAS configuration:
> unrecognized option: isInitiator
> and
> Bad JAAS configuration: unrecognized option: storeKey
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4857) Add missing default configuration regarding preemption of CapacityScheduler
[
https://issues.apache.org/jira/browse/YARN-4857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613821#comment-15613821
]
Kai Sasaki commented on YARN-4857:
--
[~jlowe] Thanks. Sure I'll do that.
> Add missing default configuration regarding preemption of CapacityScheduler
> ---
>
> Key: YARN-4857
> URL: https://issues.apache.org/jira/browse/YARN-4857
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacity scheduler, documentation
>Reporter: Kai Sasaki
>Assignee: Kai Sasaki
>Priority: Minor
> Labels: documentaion
> Attachments: YARN-4857.01.patch, YARN-4857.02.patch,
> YARN-4857.03.patch, YARN-4857.04.patch
>
>
> {{yarn.resourcemanager.monitor.*}} configurations are missing in
> yarn-default.xml. Since they were documented explicitly by YARN-4492,
> yarn-default.xml can be modified as same.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4500) Missing default config values in yarn-default.xml
[
https://issues.apache.org/jira/browse/YARN-4500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613818#comment-15613818
]
Kai Sasaki commented on YARN-4500:
--
Sure let me check again.
> Missing default config values in yarn-default.xml
> -
>
> Key: YARN-4500
> URL: https://issues.apache.org/jira/browse/YARN-4500
> Project: Hadoop YARN
> Issue Type: Bug
> Components: documentation
>Affects Versions: 2.7.1, 2.6.2
>Reporter: Tianyin Xu
>Assignee: Kai Sasaki
> Labels: oct16-easy
> Attachments: YARN-4500.01.patch
>
>
> The docs
> [yarn-default.xml|https://hadoop.apache.org/docs/r2.7.1/hadoop-yarn/hadoop-yarn-common/yarn-default.xml]
> miss the default values of the following parameters:
> {{yarn.web-proxy.address}}
> {{yarn.ipc.client.factory.class}}
> {{yarn.ipc.server.factory.class}}
> {{yarn.ipc.record.factory.class}}
> Here we go,
> {code:title=YarnConfiguration.java|borderStyle=solid}
> 97 /** Factory to create client IPC classes.*/
> 98 public static final String IPC_CLIENT_FACTORY_CLASS =
> 99 IPC_PREFIX + "client.factory.class";
> 100 public static final String DEFAULT_IPC_CLIENT_FACTORY_CLASS =
> 101 "org.apache.hadoop.yarn.factories.impl.pb.RpcClientFactoryPBImpl";
> 102
> 103 /** Factory to create server IPC classes.*/
> 104 public static final String IPC_SERVER_FACTORY_CLASS =
> 105 IPC_PREFIX + "server.factory.class";
> 106 public static final String DEFAULT_IPC_SERVER_FACTORY_CLASS =
> 107 "org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl";
> 108
> 109 /** Factory to create serializeable records.*/
> 110 public static final String IPC_RECORD_FACTORY_CLASS =
> 111 IPC_PREFIX + "record.factory.class";
> 112 public static final String DEFAULT_IPC_RECORD_FACTORY_CLASS =
> 113 "org.apache.hadoop.yarn.factories.impl.pb.RecordFactoryPBImpl";
> ...
> 1119 /** The address for the web proxy.*/
> 1120 public static final String PROXY_ADDRESS =
> 1121 PROXY_PREFIX + "address";
> 1122 public static final int DEFAULT_PROXY_PORT = 9099;
> 1123 public static final String DEFAULT_PROXY_ADDRESS =
> 1124 "0.0.0.0:" + DEFAULT_PROXY_PORT;
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-4743) FairSharePolicy breaks TimSort assumption
[
https://issues.apache.org/jira/browse/YARN-4743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karthik Kambatla updated YARN-4743:
---
Summary: FairSharePolicy breaks TimSort assumption (was: ResourceManager
crash because TimSort)
> FairSharePolicy breaks TimSort assumption
> -
>
> Key: YARN-4743
> URL: https://issues.apache.org/jira/browse/YARN-4743
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 3.0.0-alpha2
>Reporter: Zephyr Guo
>Assignee: Zephyr Guo
> Labels: oct16-easy
> Fix For: 3.0.0-alpha2
>
> Attachments: YARN-4743-v1.patch, YARN-4743-v2.patch,
> YARN-4743-v3.patch, YARN-4743-v4.patch, YARN-4743-v5.patch, timsort.log
>
>
> {code}
> 2016-02-26 14:08:50,821 FATAL
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error in
> handling event type NODE_UPDATE to the scheduler
> java.lang.IllegalArgumentException: Comparison method violates its general
> contract!
> at java.util.TimSort.mergeHi(TimSort.java:868)
> at java.util.TimSort.mergeAt(TimSort.java:485)
> at java.util.TimSort.mergeCollapse(TimSort.java:410)
> at java.util.TimSort.sort(TimSort.java:214)
> at java.util.TimSort.sort(TimSort.java:173)
> at java.util.Arrays.sort(Arrays.java:659)
> at java.util.Collections.sort(Collections.java:217)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSLeafQueue.assignContainer(FSLeafQueue.java:316)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSParentQueue.assignContainer(FSParentQueue.java:240)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.attemptScheduling(FairScheduler.java:1091)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.nodeUpdate(FairScheduler.java:989)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:1185)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:112)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:684)
> at java.lang.Thread.run(Thread.java:745)
> 2016-02-26 14:08:50,822 INFO
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Exiting, bbye..
> {code}
> Actually, this bug found in 2.6.0-cdh5.4.7. {{FairShareComparator}} is not
> transitive.
> We get NaN when memorySize=0 and weight=0.
> {code:title=FairSharePolicy.java}
> useToWeightRatio1 = s1.getResourceUsage().getMemorySize() /
> s1.getWeights().getWeight(ResourceType.MEMORY)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5793) Trim configuration values in DockerLinuxContainerRuntime
[
https://issues.apache.org/jira/browse/YARN-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tianyin Xu updated YARN-5793:
-
Attachment: YARN-5793.0001.patch
fix the style issue (80 character per line)
the failed test
(org.apache.hadoop.yarn.server.nodemanager.containermanager.queuing.testKillMultipleOpportunisticContainers)
seems has nothing to do with the patch and I've tested on my local repo and it
passed this test successfully.
> Trim configuration values in DockerLinuxContainerRuntime
>
>
> Key: YARN-5793
> URL: https://issues.apache.org/jira/browse/YARN-5793
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.0.0-alpha1
>Reporter: Tianyin Xu
>Priority: Minor
> Attachments: YARN-5793..patch, YARN-5793.0001.patch
>
>
> The current implementation of {{DockerLinuxContainerRuntime}} does not follow
> the practice of trimming configuration values. This leads to errors if users
> set values containing space or newline.
> see the following YARN commits as reference:
> YARN-3395. FairScheduler: Trim whitespaces when using username for queuename.
> YARN-2869. CapacityScheduler should trim sub queue names when parse
> configuration.
> YARN-2843. Fixed NodeLabelsManager to trim inputs for hosts and labels so as
> to make them work correctly.
> and many other Hadoop/HDFS commits (just list a few):
> HDFS-9708. FSNamesystem.initAuditLoggers() doesn't trim classnames
> HDFS-2799. Trim fs.checkpoint.dir values.
> HADOOP-6578. Configuration should trim whitespace around a lot of value types
> HADOOP-6534. Trim whitespace from directory lists initializing
> Patch is available against trunk
> {code:title=DockerLinuxContainerRuntime.java|borderStyle=solid}
> @@ -219,9 +219,9 @@ public void initialize(Configuration conf)
> dockerClient = new DockerClient(conf);
> allowedNetworks.clear();
> allowedNetworks.addAll(Arrays.asList(
> -
> conf.getStrings(YarnConfiguration.NM_DOCKER_ALLOWED_CONTAINER_NETWORKS,
> +
> conf.getTrimmedStrings(YarnConfiguration.NM_DOCKER_ALLOWED_CONTAINER_NETWORKS,
>
> YarnConfiguration.DEFAULT_NM_DOCKER_ALLOWED_CONTAINER_NETWORKS)));
> -defaultNetwork = conf.get(
> +defaultNetwork = conf.getTrimmed(
> YarnConfiguration.NM_DOCKER_DEFAULT_CONTAINER_NETWORK,
> YarnConfiguration.DEFAULT_NM_DOCKER_DEFAULT_CONTAINER_NETWORK);
>
> @@ -237,7 +237,7 @@ public void initialize(Configuration conf)
>throw new ContainerExecutionException(message);
> }
>
> -privilegedContainersAcl = new AccessControlList(conf.get(
> +privilegedContainersAcl = new AccessControlList(conf.getTrimmed(
> YarnConfiguration.NM_DOCKER_PRIVILEGED_CONTAINERS_ACL,
> YarnConfiguration.DEFAULT_NM_DOCKER_PRIVILEGED_CONTAINERS_ACL));
>}
> @@ -439,7 +439,7 @@ public void launchContainer(ContainerRuntimeContext ctx)
> LOCALIZED_RESOURCES);
> @SuppressWarnings("unchecked")
> List userLocalDirs = ctx.getExecutionAttribute(USER_LOCAL_DIRS);
> -Set capabilities = new HashSet<>(Arrays.asList(conf.getStrings(
> +Set capabilities = new
> HashSet<>(Arrays.asList(conf.getTrimmedStrings(
> YarnConfiguration.NM_DOCKER_CONTAINER_CAPABILITIES,
> YarnConfiguration.DEFAULT_NM_DOCKER_CONTAINER_CAPABILITIES)));
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5435) [Regression] QueueCapacities not being updated for dynamic ReservationQueue
[
https://issues.apache.org/jira/browse/YARN-5435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613799#comment-15613799
]
Carlo Curino commented on YARN-5435:
[~seanpo03] what you have makes sense to me, but it is mostly overlapping with
YARN-4003 (which I close as duplicate of this). What you are missing here,
which should be added is the setting (via overriding) of the
{{getAMResourceLimit()}} and {{getUserAMResourceLimit()}}. For
{{ReservationQueue}} (as we discuss YARN-4003) this limit should be set
independently of the local capacity and be set to the value of the parent (as
the reservation capacity will fluctuate heavily, hitting zero at times). Look
at the patch i had in YARN-4003 and import in your one similar functionalities.
Thanks...
> [Regression] QueueCapacities not being updated for dynamic ReservationQueue
> ---
>
> Key: YARN-5435
> URL: https://issues.apache.org/jira/browse/YARN-5435
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler, resourcemanager
>Affects Versions: 2.8.0
>Reporter: Sean Po
>Assignee: Sean Po
> Labels: oct16-easy, regression
> Attachments: YARN-5435.v003.patch, YARN-5435.v1.patch,
> YARN-5435.v2.patch
>
>
> YARN-1707 added dynamic queues (ReservationQueue) to CapacityScheduler. The
> QueueCapacities data structure was added subsequently but is not being
> updated correctly for ReservationQueue. This JIRA tracks the changes required
> to update QueueCapacities of ReservationQueue correctly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4743) ResourceManager crash because TimSort
[
https://issues.apache.org/jira/browse/YARN-4743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613794#comment-15613794
]
Karthik Kambatla commented on YARN-4743:
+1
Checking this in..
> ResourceManager crash because TimSort
> -
>
> Key: YARN-4743
> URL: https://issues.apache.org/jira/browse/YARN-4743
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 3.0.0-alpha2
>Reporter: Zephyr Guo
>Assignee: Zephyr Guo
> Labels: oct16-easy
> Fix For: 3.0.0-alpha2
>
> Attachments: YARN-4743-v1.patch, YARN-4743-v2.patch,
> YARN-4743-v3.patch, YARN-4743-v4.patch, YARN-4743-v5.patch, timsort.log
>
>
> {code}
> 2016-02-26 14:08:50,821 FATAL
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error in
> handling event type NODE_UPDATE to the scheduler
> java.lang.IllegalArgumentException: Comparison method violates its general
> contract!
> at java.util.TimSort.mergeHi(TimSort.java:868)
> at java.util.TimSort.mergeAt(TimSort.java:485)
> at java.util.TimSort.mergeCollapse(TimSort.java:410)
> at java.util.TimSort.sort(TimSort.java:214)
> at java.util.TimSort.sort(TimSort.java:173)
> at java.util.Arrays.sort(Arrays.java:659)
> at java.util.Collections.sort(Collections.java:217)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSLeafQueue.assignContainer(FSLeafQueue.java:316)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSParentQueue.assignContainer(FSParentQueue.java:240)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.attemptScheduling(FairScheduler.java:1091)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.nodeUpdate(FairScheduler.java:989)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:1185)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:112)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:684)
> at java.lang.Thread.run(Thread.java:745)
> 2016-02-26 14:08:50,822 INFO
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Exiting, bbye..
> {code}
> Actually, this bug found in 2.6.0-cdh5.4.7. {{FairShareComparator}} is not
> transitive.
> We get NaN when memorySize=0 and weight=0.
> {code:title=FairSharePolicy.java}
> useToWeightRatio1 = s1.getResourceUsage().getMemorySize() /
> s1.getWeights().getWeight(ResourceType.MEMORY)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4043) Change logging of warning message : "an attempt to override final parameter:"
[ https://issues.apache.org/jira/browse/YARN-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613786#comment-15613786 ] Gour Saha commented on YARN-4043: - [~spandan] I also think we should move this to Hadoop Common. Do you agree? > Change logging of warning message : "an attempt to override final parameter:" > - > > Key: YARN-4043 > URL: https://issues.apache.org/jira/browse/YARN-4043 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Spandan Dutta >Assignee: Spandan Dutta > Attachments: warn-msg.patch, warn-msg.patch > > > In the following > [function|https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java#L2739]. > > When the attr is in the list of final attrs it just outputs this message > without actually updating any resources as per my understanding. > We change this to debug logging. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-3732) Change NodeHeartbeatResponse.java and RegisterNodeManagerResponse.java as abstract classes
[ https://issues.apache.org/jira/browse/YARN-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated YARN-3732: Attachment: YARN-3732-2.patch Updated the patch against trunk. > Change NodeHeartbeatResponse.java and RegisterNodeManagerResponse.java as > abstract classes > -- > > Key: YARN-3732 > URL: https://issues.apache.org/jira/browse/YARN-3732 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Devaraj K >Assignee: Devaraj K >Priority: Minor > Attachments: YARN-3732-1.patch, YARN-3732-2.patch, YARN-3732.patch > > > All the other protocol record classes are abstract classes. Change > NodeHeartbeatResponse.java and RegisterNodeManagerResponse.java as abstract > classes to make it consistent with other protocol record classes. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5793) Trim configuration values in DockerLinuxContainerRuntime
[
https://issues.apache.org/jira/browse/YARN-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613773#comment-15613773
]
Hadoop QA commented on YARN-5793:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 14s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
The patch generated 2 new + 4 unchanged - 0 fixed = 6 total (was 4) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 15m 20s{color}
| {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 16m
11s{color} | {color:red} The patch generated 4 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 46m 43s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager
|
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | YARN-5793 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12835688/YARN-5793..patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux c4bff9e7cbc8 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29
20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 5877f20 |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/13615/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
|
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/13615/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/13615/testReport/ |
| asflicense |
https://builds.apache.org/job/PreCommit-YARN-Build/13615/artifact/patchprocess/patch-asflicense-problems.txt
|
| modules | C:
[jira] [Comment Edited] (YARN-5525) Make log aggregation service class configurable
[ https://issues.apache.org/jira/browse/YARN-5525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613767#comment-15613767 ] Botong Huang edited comment on YARN-5525 at 10/28/16 12:22 AM: --- YARN-5525.v4.patch uploaded. Making AppLogAggregator configurable, rather than LogAggregationService. Moved directory creation logic into AppLogAggregator. Please take a look, thanks! was (Author: botong): Making AppLogAggregator configurable, rather than LogAggregationService. Moved directory creation logic into AppLogAggregator. > Make log aggregation service class configurable > --- > > Key: YARN-5525 > URL: https://issues.apache.org/jira/browse/YARN-5525 > Project: Hadoop YARN > Issue Type: Improvement > Components: log-aggregation >Reporter: Giovanni Matteo Fumarola >Assignee: Botong Huang >Priority: Minor > Labels: oct16-medium > Attachments: YARN-5525.v1.patch, YARN-5525.v2.patch, > YARN-5525.v3.patch, YARN-5525.v4.patch > > > Make the log aggregation class configurable and extensible, so that > alternative log aggregation behaviors like app specific log aggregation > directory, log aggregation format can be implemented and plugged in. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5525) Make log aggregation service class configurable
[ https://issues.apache.org/jira/browse/YARN-5525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Botong Huang updated YARN-5525: --- Attachment: YARN-5525.v4.patch Making AppLogAggregator configurable, rather than LogAggregationService. Moved directory creation logic into AppLogAggregator. > Make log aggregation service class configurable > --- > > Key: YARN-5525 > URL: https://issues.apache.org/jira/browse/YARN-5525 > Project: Hadoop YARN > Issue Type: Improvement > Components: log-aggregation >Reporter: Giovanni Matteo Fumarola >Assignee: Botong Huang >Priority: Minor > Labels: oct16-medium > Attachments: YARN-5525.v1.patch, YARN-5525.v2.patch, > YARN-5525.v3.patch, YARN-5525.v4.patch > > > Make the log aggregation class configurable and extensible, so that > alternative log aggregation behaviors like app specific log aggregation > directory, log aggregation format can be implemented and plugged in. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5265) Make HBase configuration for the timeline service configurable
[ https://issues.apache.org/jira/browse/YARN-5265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613764#comment-15613764 ] Joep Rottinghuis commented on YARN-5265: Uploaded a new rebased patch: YARN-5265-YARN-5355.10.patch > Make HBase configuration for the timeline service configurable > -- > > Key: YARN-5265 > URL: https://issues.apache.org/jira/browse/YARN-5265 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Joep Rottinghuis >Assignee: Joep Rottinghuis > Labels: YARN-5355, oct16-medium > Attachments: ATS v2 cluster deployment v1.png, > YARN-5265-YARN-2928.01.patch, YARN-5265-YARN-2928.02.patch, > YARN-5265-YARN-2928.03.patch, YARN-5265-YARN-2928.04.patch, > YARN-5265-YARN-2928.05.patch, YARN-5265-YARN-5355.06.patch, > YARN-5265-YARN-5355.07.patch, YARN-5265-YARN-5355.08.patch, > YARN-5265-YARN-5355.09.patch, YARN-5265-YARN-5355.10.patch > > > Currently we create "default" HBase configurations, this works as long as the > user places the appropriate configuration on the classpath. > This works fine for a standalone Hadoop cluster. > However, if a user wants to monitor an HBase cluster and has a separate ATS > HBase cluster, then it can become tricky to create the right classpath for > the nodemanagers and still have tasks have their separate configs. > It will be much easier to add a yarn configuration to let cluster admins > configure which HBase to connect to to write ATS metrics to. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5265) Make HBase configuration for the timeline service configurable
[ https://issues.apache.org/jira/browse/YARN-5265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joep Rottinghuis updated YARN-5265: --- Attachment: YARN-5265-YARN-5355.10.patch > Make HBase configuration for the timeline service configurable > -- > > Key: YARN-5265 > URL: https://issues.apache.org/jira/browse/YARN-5265 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Joep Rottinghuis >Assignee: Joep Rottinghuis > Labels: YARN-5355, oct16-medium > Attachments: ATS v2 cluster deployment v1.png, > YARN-5265-YARN-2928.01.patch, YARN-5265-YARN-2928.02.patch, > YARN-5265-YARN-2928.03.patch, YARN-5265-YARN-2928.04.patch, > YARN-5265-YARN-2928.05.patch, YARN-5265-YARN-5355.06.patch, > YARN-5265-YARN-5355.07.patch, YARN-5265-YARN-5355.08.patch, > YARN-5265-YARN-5355.09.patch, YARN-5265-YARN-5355.10.patch > > > Currently we create "default" HBase configurations, this works as long as the > user places the appropriate configuration on the classpath. > This works fine for a standalone Hadoop cluster. > However, if a user wants to monitor an HBase cluster and has a separate ATS > HBase cluster, then it can become tricky to create the right classpath for > the nodemanagers and still have tasks have their separate configs. > It will be much easier to add a yarn configuration to let cluster admins > configure which HBase to connect to to write ATS metrics to. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5276) print more info when event queue is blocked
[ https://issues.apache.org/jira/browse/YARN-5276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sandflee updated YARN-5276: --- Attachment: YARN-5276.04.patch > print more info when event queue is blocked > --- > > Key: YARN-5276 > URL: https://issues.apache.org/jira/browse/YARN-5276 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager, resourcemanager >Reporter: sandflee >Assignee: sandflee > Labels: oct16-easy > Attachments: YARN-5276.01.patch, YARN-5276.02.patch, > YARN-5276.03.patch, YARN-5276.04.patch > > > we now see logs like "Size of event-queue is 498000, Size of event-queue is > 499000" and difficult to know which event flood the queue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5276) print more info when event queue is blocked
[ https://issues.apache.org/jira/browse/YARN-5276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613741#comment-15613741 ] sandflee commented on YARN-5276: Thanks for your comment, the patch just add some log info, I wondered how to test it? > print more info when event queue is blocked > --- > > Key: YARN-5276 > URL: https://issues.apache.org/jira/browse/YARN-5276 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager, resourcemanager >Reporter: sandflee >Assignee: sandflee > Labels: oct16-easy > Attachments: YARN-5276.01.patch, YARN-5276.02.patch, > YARN-5276.03.patch > > > we now see logs like "Size of event-queue is 498000, Size of event-queue is > 499000" and difficult to know which event flood the queue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4467) Shell.checkIsBashSupported swallowed an interrupted exception
[
https://issues.apache.org/jira/browse/YARN-4467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613732#comment-15613732
]
Hadoop QA commented on YARN-4467:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
31s{color} | {color:green} root: The patch generated 0 new + 66 unchanged - 1
fixed = 66 total (was 67) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m
15s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 15m
11s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 81m 56s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | YARN-4467 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12778308/YARN-4467.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 5bc334ab1efc 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12
17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 022bf78 |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/13602/testReport/ |
| modules | C: hadoop-common-project/hadoop-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U: . |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/13602/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org
[jira] [Commented] (YARN-5184) Fix up incompatible changes introduced in YARN-2882 and YARN-5430
[ https://issues.apache.org/jira/browse/YARN-5184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613722#comment-15613722 ] Sangjin Lee commented on YARN-5184: --- The changes are the introduction of abstract getter/setter methods. > Fix up incompatible changes introduced in YARN-2882 and YARN-5430 > - > > Key: YARN-5184 > URL: https://issues.apache.org/jira/browse/YARN-5184 > Project: Hadoop YARN > Issue Type: Bug > Components: api >Affects Versions: 2.9.0 >Reporter: Karthik Kambatla >Assignee: Karthik Kambatla >Priority: Blocker > > YARN-2882 and YARN-5430 broke compatibility by adding abstract methods to a > Public-Stable class - ContainerStatus. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5713) Update jackson from 1.9.13 to 2.x in hadoop-yarn
[ https://issues.apache.org/jira/browse/YARN-5713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613718#comment-15613718 ] Li Lu commented on YARN-5713: - Bumping up the version of jackson appears to have some impact to timeline service 1.x and 2. Maybe we'd like to at least try bumping up the version in YARN-5355 branch for timeline v2 before we actually post this to trunk. [~sjlee0][~jrottinghuis] > Update jackson from 1.9.13 to 2.x in hadoop-yarn > > > Key: YARN-5713 > URL: https://issues.apache.org/jira/browse/YARN-5713 > Project: Hadoop YARN > Issue Type: Improvement > Components: build, timelineserver >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka > Labels: oct16-medium > Attachments: HADOOP-13677.01.patch, HADOOP-13677.02.patch > > > Sub-task of HADOOP-13332. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3359) Recover collector list in RM failed over
[
https://issues.apache.org/jira/browse/YARN-3359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613713#comment-15613713
]
Hadoop QA commented on YARN-3359:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 4m
42s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
44s{color} | {color:green} YARN-5355 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
29s{color} | {color:green} YARN-5355 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
32s{color} | {color:green} YARN-5355 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
4s{color} | {color:green} YARN-5355 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
31s{color} | {color:green} YARN-5355 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
57s{color} | {color:green} YARN-5355 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
40s{color} | {color:green} YARN-5355 passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 13m
3s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 35m 16s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 80m 45s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.server.resourcemanager.TestRMRestart |
| | hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | YARN-3359 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12835682/YARN-3359-YARN-5355.002.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux ef951271128d 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18
21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | YARN-5355 / 513dcf6 |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/13597/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/13597/testReport/ |
| modules | C:
[jira] [Commented] (YARN-5184) Fix up incompatible changes introduced in YARN-2882 and YARN-5430
[ https://issues.apache.org/jira/browse/YARN-5184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613706#comment-15613706 ] Andrew Wang commented on YARN-5184: --- Karthik, could you summarize the incompatibilities from these two JIRAs, and the potential fix? If the changes were made intentionally, then I agree with Sangjin and we can leave them for 3.0.0-alpha2. However, if they were accidental or can be easily fixed to make life easier for downstreams, then I would like to address this in alpha2. > Fix up incompatible changes introduced in YARN-2882 and YARN-5430 > - > > Key: YARN-5184 > URL: https://issues.apache.org/jira/browse/YARN-5184 > Project: Hadoop YARN > Issue Type: Bug > Components: api >Affects Versions: 2.9.0 >Reporter: Karthik Kambatla >Assignee: Karthik Kambatla >Priority: Blocker > > YARN-2882 and YARN-5430 broke compatibility by adding abstract methods to a > Public-Stable class - ContainerStatus. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4907) Make all MockRM#waitForState consistent.
[
https://issues.apache.org/jira/browse/YARN-4907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613702#comment-15613702
]
Daniel Templeton commented on YARN-4907:
Thanks, [~yufeigu]. One comment: your while-if-break statements can be
simplified. For example:
{code}
while (true) {
if (timeWaiting >= TIMEOUT_MS_FOR_CONTAINER_AND_NODE) {
break;
}
{code}
should be
{code}
while (timeWaiting < TIMEOUT_MS_FOR_CONTAINER_AND_NODE) {
{code}
There's another in {{waitForNewAMToLaunchAndRegister()}}.
> Make all MockRM#waitForState consistent.
> -
>
> Key: YARN-4907
> URL: https://issues.apache.org/jira/browse/YARN-4907
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
>Reporter: Yufei Gu
>Assignee: Yufei Gu
> Labels: oct16-medium
> Attachments: YARN-4907.001.patch
>
>
> There are some inconsistencies among these {{waitForState}} in {{MockRM}}:
> 1. Some {{waitForState}} return a boolean while others don't.
> 2. Some {{waitForState}} don't have a timeout, they can wait for ever.
> 3. Some {{waitForState}} use LOG.info and others use {{System.out.println}}
> to print messages.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5435) [Regression] QueueCapacities not being updated for dynamic ReservationQueue
[ https://issues.apache.org/jira/browse/YARN-5435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Po updated YARN-5435: -- Attachment: YARN-5435.v003.patch Thanks [~curino] for the review. This latest patch uses the CSQueueUtils class to inherit properties from the parent queue. It also adds the ability to whitelist labels for inheritance so that we can select just NO_LABEL to inherit. > [Regression] QueueCapacities not being updated for dynamic ReservationQueue > --- > > Key: YARN-5435 > URL: https://issues.apache.org/jira/browse/YARN-5435 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler, resourcemanager >Affects Versions: 2.8.0 >Reporter: Sean Po >Assignee: Sean Po > Labels: oct16-easy, regression > Attachments: YARN-5435.v003.patch, YARN-5435.v1.patch, > YARN-5435.v2.patch > > > YARN-1707 added dynamic queues (ReservationQueue) to CapacityScheduler. The > QueueCapacities data structure was added subsequently but is not being > updated correctly for ReservationQueue. This JIRA tracks the changes required > to update QueueCapacities of ReservationQueue correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5446) Cluster Resource Usage table is not displayed when user clicks on hadoop logo on top of web UI home page
[
https://issues.apache.org/jira/browse/YARN-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613673#comment-15613673
]
Hadoop QA commented on YARN-5446:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 3m
27s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 4m 7s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:7f1ec06 |
| JIRA Issue | YARN-5446 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12822971/YARN-5446-YARN-3368.0002.patch
|
| Optional Tests | asflicense |
| uname | Linux 609301a2d660 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18
21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | YARN-3368 / 29d7106 |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/13612/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Cluster Resource Usage table is not displayed when user clicks on hadoop
> logo on top of web UI home page
> -
>
> Key: YARN-5446
> URL: https://issues.apache.org/jira/browse/YARN-5446
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: webapp
>Reporter: Chen Ge
>Assignee: Akhil PB
> Attachments: YARN-5446-YARN-3368.0001.patch,
> YARN-5446-YARN-3368.0002.patch, screenshot-1.png
>
>
> Under latest 3368, when user clicks hadoop icon in cluster overview page,
> "Cluster Resource Usage By Applications" could not be correctly displayed.
> Following is error description found on website.
> {code}
> donut-chart.js:110 Uncaught TypeError: Cannot read property 'value' of
> undefined(anonymous function) @
> donut-chart.js:110arguments.length.each.value.function.value.textContent @
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4994) Use MiniYARNCluster with try-with-resources in tests
[
https://issues.apache.org/jira/browse/YARN-4994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613670#comment-15613670
]
Hadoop QA commented on YARN-4994:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color}
| {color:red} YARN-4994 does not apply to trunk. Rebase required? Wrong Branch?
See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-4994 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12803191/YARN-4994.07.patch |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/13613/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Use MiniYARNCluster with try-with-resources in tests
>
>
> Key: YARN-4994
> URL: https://issues.apache.org/jira/browse/YARN-4994
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: test
>Affects Versions: 2.7.0
>Reporter: Andras Bokor
>Assignee: Andras Bokor
>Priority: Trivial
> Labels: oct16-easy
> Fix For: 2.7.0
>
> Attachments: HDFS-10287.01.patch, HDFS-10287.02.patch,
> HDFS-10287.03.patch, YARN-4994.04.patch, YARN-4994.05.patch,
> YARN-4994.06.patch, YARN-4994.07.patch
>
>
> In tests MiniYARNCluster is used with the following pattern:
> In try-catch block create a MiniYARNCluster instance and in finally block
> close it.
> [Try-with-resources|https://docs.oracle.com/javase/tutorial/essential/exceptions/tryResourceClose.html]
> is preferred since Java7 instead of the pattern above.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4743) ResourceManager crash because TimSort
[
https://issues.apache.org/jira/browse/YARN-4743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613655#comment-15613655
]
Hadoop QA commented on YARN-4743:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
0s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 35m
54s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 51m 35s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | YARN-4743 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12835683/YARN-4743-v5.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 21ab7a0a2f6c 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12
17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 022bf78 |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/13598/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/13598/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> ResourceManager crash because TimSort
> -
>
> Key: YARN-4743
> URL: https://issues.apache.org/jira/browse/YARN-4743
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 3.0.0-alpha2
>Reporter: Zephyr Guo
>Assignee: Zephyr Guo
> Labels: oct16-easy
> Fix For: 3.0.0-alpha2
>
> Attachments: YARN-4743-v1.patch,
[jira] [Commented] (YARN-5259) Add two metrics at FSOpDurations for doing container assign and completed Performance statistical analysis
[
https://issues.apache.org/jira/browse/YARN-5259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613649#comment-15613649
]
Inigo Goiri commented on YARN-5259:
---
[~templedf], I checked and there are no other unit tests for related to
{{FSOpDurations}} metrics.
We would have to add tests for that; the only related is
{{testPerfMetricsInited()#TestFairScheduler}}.
> Add two metrics at FSOpDurations for doing container assign and completed
> Performance statistical analysis
> --
>
> Key: YARN-5259
> URL: https://issues.apache.org/jira/browse/YARN-5259
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: fairscheduler
>Reporter: ChenFolin
> Labels: oct16-easy
> Attachments: YARN-5259-001.patch, YARN-5259-002.patch,
> YARN-5259-003.patch, YARN-5259-004.patch
>
>
> If cluster is slow , we can not know Whether it is caused by container assign
> or completed performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-3748) Fix volatile variable modification in the scheduler
[ https://issues.apache.org/jira/browse/YARN-3748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated YARN-3748: -- Assignee: Gabor Liptak > Fix volatile variable modification in the scheduler > --- > > Key: YARN-3748 > URL: https://issues.apache.org/jira/browse/YARN-3748 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler, resourcemanager >Reporter: Gabor Liptak >Assignee: Gabor Liptak >Priority: Critical > Labels: oct16-easy > Attachments: YARN-3748.1.patch, YARN-3748.2.patch, YARN-3748.3.patch, > YARN-3748.5.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4269) Log aggregation should not swallow the exception during close()
[
https://issues.apache.org/jira/browse/YARN-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613643#comment-15613643
]
Shane Kumpf commented on YARN-4269:
---
Hey [~lichangleo] - It appears the patch no longer applies cleanly to trunk.
I also see that {{IOUtils#cleanup}} almost meets your needs, but catches the
wrong exception and logs at the wrong priority. Given this, I would prefer an
overloaded version of {{IOUtils#cleanup}} vs {{IOUtils#closeStream}}. You could
pass in the exception class and log priority and refactor the existing
{{IOUtils#cleanup}}. What do you think?
> Log aggregation should not swallow the exception during close()
> ---
>
> Key: YARN-4269
> URL: https://issues.apache.org/jira/browse/YARN-4269
> Project: Hadoop YARN
> Issue Type: Bug
> Components: log-aggregation
>Reporter: Chang Li
>Assignee: Chang Li
> Labels: oct16-easy
> Attachments: YARN-4269.2.patch, YARN-4269.3.patch, YARN-4269.patch
>
>
> the log aggregation thread ignores exception thrown by close(). It shouldn't
> be ignored, since the file content may be missing or partial.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4022) queue not remove from webpage(/cluster/scheduler) when delete queue in xxx-scheduler.xml
[
https://issues.apache.org/jira/browse/YARN-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613638#comment-15613638
]
Daniel Templeton commented on YARN-4022:
Thanks for the patch, [~forrestchen]. This is definitely useful functionality.
It the patch you're using a thread that watches for updates to the
configuration so it can remove unused queues. YARN-5556 is adding the same
capability for fair scheduler, and that patch handles the delete during the
reload of the configuration. Think that approach could work for fair scheduler?
I'd also like to see some unit tests to cover the new functionality.
> queue not remove from webpage(/cluster/scheduler) when delete queue in
> xxx-scheduler.xml
>
>
> Key: YARN-4022
> URL: https://issues.apache.org/jira/browse/YARN-4022
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler, resourcemanager
>Affects Versions: 2.7.1
>Reporter: forrestchen
> Labels: oct16-medium, scheduler
> Attachments: YARN-4022.001.patch, YARN-4022.002.patch,
> YARN-4022.003.patch, YARN-4022.004.patch
>
>
> When I delete an existing queue by modify the xxx-schedule.xml, I can still
> see the queue information block in webpage(/cluster/scheduler) though the
> 'Min Resources' items all become to zero and have no item of 'Max Running
> Applications'.
> I can still submit an application to the deleted queue and the application
> will run using 'root.default' queue instead, but submit to an un-exist queue
> will cause an exception.
> My expectation is the deleted queue will not displayed in webpage and submit
> application to the deleted queue will act just like the queue doesn't exist.
> PS: There's no application running in the queue I delete.
> Some related config in yarn-site.xml:
> {code}
>
> yarn.scheduler.fair.user-as-default-queue
> false
>
>
> yarn.scheduler.fair.allow-undeclared-pools
> false
>
> {code}
> a related question is here:
> http://stackoverflow.com/questions/26488564/hadoop-yarn-why-the-queue-cannot-be-deleted-after-i-revise-my-fair-scheduler-xm
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5713) Update jackson from 1.9.13 to 2.x in hadoop-yarn
[ https://issues.apache.org/jira/browse/YARN-5713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613629#comment-15613629 ] Miklos Szegedi commented on YARN-5713: -- Thank you for the patch [~ajisakaa]! Overall the change looks good to me. Please see my comments below: FileSystemTimelineReaderImpl.java has a new newline at the end that is not needed. JsonSerDeser.java: toJson may throw IOException but does not document it. This is not related to your change but since you already changed the comment, I thought I will mention it. TestAHSWebServices ran nicely to me, when I applied the patch to the latest trunk. > Update jackson from 1.9.13 to 2.x in hadoop-yarn > > > Key: YARN-5713 > URL: https://issues.apache.org/jira/browse/YARN-5713 > Project: Hadoop YARN > Issue Type: Improvement > Components: build, timelineserver >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka > Labels: oct16-medium > Attachments: HADOOP-13677.01.patch, HADOOP-13677.02.patch > > > Sub-task of HADOOP-13332. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4899) Queue metrics of SLS capacity scheduler only activated after app submit to the queue
[
https://issues.apache.org/jira/browse/YARN-4899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613624#comment-15613624
]
Hadoop QA commented on YARN-4899:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
49s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 13s{color} | {color:orange} hadoop-tools/hadoop-sls: The patch generated 5
new + 100 unchanged - 0 fixed = 105 total (was 100) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
54s{color} | {color:green} hadoop-sls in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 46s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | YARN-4899 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12803092/YARN-4899.1.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux d6698c81e32f 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29
20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 9e03ee5 |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/13611/artifact/patchprocess/diff-checkstyle-hadoop-tools_hadoop-sls.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/13611/testReport/ |
| modules | C: hadoop-tools/hadoop-sls U: hadoop-tools/hadoop-sls |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/13611/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Queue metrics of SLS capacity scheduler only activated after app submit to
> the queue
>
>
> Key: YARN-4899
> URL: https://issues.apache.org/jira/browse/YARN-4899
> Project: Hadoop YARN
[jira] [Commented] (YARN-4765) Split TestHBaseTimelineStorage into multiple test classes
[ https://issues.apache.org/jira/browse/YARN-4765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613620#comment-15613620 ] Hudson commented on YARN-4765: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10712 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/10712/]) YARN-4765 Split TestHBaseTimelineStorage into multiple test classes (sjlee: rev 022bf783aa89c1c81374ebef5dba2df95b7563b5) * (delete) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests/src/test/java/org/apache/hadoop/yarn/server/timelineservice/storage/TestHBaseTimelineStorage.java * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests/src/test/java/org/apache/hadoop/yarn/server/timelineservice/storage/TestHBaseTimelineStorageEntities.java * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests/src/test/java/org/apache/hadoop/yarn/server/timelineservice/storage/TestHBaseTimelineStorageApps.java * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests/src/test/java/org/apache/hadoop/yarn/server/timelineservice/storage/DataGeneratorForTest.java > Split TestHBaseTimelineStorage into multiple test classes > - > > Key: YARN-4765 > URL: https://issues.apache.org/jira/browse/YARN-4765 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Varun Saxena >Assignee: Varun Saxena > Labels: YARN-5355, oct16-medium > Fix For: YARN-5355 > > Attachments: YARN-4765-YARN-5355.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-2306) Add test for leakage of reservation metrics in fair scheduler
[ https://issues.apache.org/jira/browse/YARN-2306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613617#comment-15613617 ] Yufei Gu commented on YARN-2306: The test failure is unrelated. > Add test for leakage of reservation metrics in fair scheduler > - > > Key: YARN-2306 > URL: https://issues.apache.org/jira/browse/YARN-2306 > Project: Hadoop YARN > Issue Type: Bug > Components: fairscheduler >Reporter: Hong Zhiguo >Assignee: Hong Zhiguo >Priority: Minor > Labels: oct16-easy > Attachments: YARN-2306-2.patch, YARN-2306-3.patch, > YARN-2306.004.patch, YARN-2306.patch > > > This only applies to fair scheduler. Capacity scheduler is OK. > When appAttempt or node is removed, the metrics for > reservation(reservedContainers, reservedMB, reservedVCores) is not reduced > back. > These are important metrics for administrator. The wrong metrics confuses may > confuse them. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3625) RollingLevelDBTimelineStore Incorrectly Forbids Related Entity in Same Put
[
https://issues.apache.org/jira/browse/YARN-3625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613612#comment-15613612
]
Hadoop QA commented on YARN-3625:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 11s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice:
The patch generated 1 new + 3 unchanged - 2 fixed = 4 total (was 5) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
41s{color} | {color:green} hadoop-yarn-server-applicationhistoryservice in the
patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 15m 34s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | YARN-3625 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12732257/YARN-3625.2.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 5bf9ff857272 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18
21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / f3ac1f4 |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/13610/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/13610/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/13610/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> RollingLevelDBTimelineStore Incorrectly Forbids Related Entity in Same Put
>
[jira] [Updated] (YARN-4022) queue not remove from webpage(/cluster/scheduler) when delete queue in xxx-scheduler.xml
[
https://issues.apache.org/jira/browse/YARN-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Templeton updated YARN-4022:
---
Component/s: fairscheduler
> queue not remove from webpage(/cluster/scheduler) when delete queue in
> xxx-scheduler.xml
>
>
> Key: YARN-4022
> URL: https://issues.apache.org/jira/browse/YARN-4022
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler, resourcemanager
>Affects Versions: 2.7.1
>Reporter: forrestchen
> Labels: oct16-medium, scheduler
> Attachments: YARN-4022.001.patch, YARN-4022.002.patch,
> YARN-4022.003.patch, YARN-4022.004.patch
>
>
> When I delete an existing queue by modify the xxx-schedule.xml, I can still
> see the queue information block in webpage(/cluster/scheduler) though the
> 'Min Resources' items all become to zero and have no item of 'Max Running
> Applications'.
> I can still submit an application to the deleted queue and the application
> will run using 'root.default' queue instead, but submit to an un-exist queue
> will cause an exception.
> My expectation is the deleted queue will not displayed in webpage and submit
> application to the deleted queue will act just like the queue doesn't exist.
> PS: There's no application running in the queue I delete.
> Some related config in yarn-site.xml:
> {code}
>
> yarn.scheduler.fair.user-as-default-queue
> false
>
>
> yarn.scheduler.fair.allow-undeclared-pools
> false
>
> {code}
> a related question is here:
> http://stackoverflow.com/questions/26488564/hadoop-yarn-why-the-queue-cannot-be-deleted-after-i-revise-my-fair-scheduler-xm
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-5790) Address collector life-cycle/persistency issues on the NM side.
[ https://issues.apache.org/jira/browse/YARN-5790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vrushali C resolved YARN-5790. -- Resolution: Duplicate Thanks [~varun_saxena], marking this closed as a duplicate of YARN-3115 > Address collector life-cycle/persistency issues on the NM side. > --- > > Key: YARN-5790 > URL: https://issues.apache.org/jira/browse/YARN-5790 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Vrushali C > > As per discussions in YARN-3359 with [~gtCarrera9], filing jira to think over > and address collector life-cycle/persistency issues on the NM side. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5793) Trim configuration values in DockerLinuxContainerRuntime
[
https://issues.apache.org/jira/browse/YARN-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613604#comment-15613604
]
Tianyin Xu commented on YARN-5793:
--
thanks, Daniel!
> Trim configuration values in DockerLinuxContainerRuntime
>
>
> Key: YARN-5793
> URL: https://issues.apache.org/jira/browse/YARN-5793
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.0.0-alpha1
>Reporter: Tianyin Xu
>Priority: Minor
> Attachments: YARN-5793..patch
>
>
> The current implementation of {{DockerLinuxContainerRuntime}} does not follow
> the practice of trimming configuration values. This leads to errors if users
> set values containing space or newline.
> see the following YARN commits as reference:
> YARN-3395. FairScheduler: Trim whitespaces when using username for queuename.
> YARN-2869. CapacityScheduler should trim sub queue names when parse
> configuration.
> YARN-2843. Fixed NodeLabelsManager to trim inputs for hosts and labels so as
> to make them work correctly.
> and many other Hadoop/HDFS commits (just list a few):
> HDFS-9708. FSNamesystem.initAuditLoggers() doesn't trim classnames
> HDFS-2799. Trim fs.checkpoint.dir values.
> HADOOP-6578. Configuration should trim whitespace around a lot of value types
> HADOOP-6534. Trim whitespace from directory lists initializing
> Patch is available against trunk
> {code:title=DockerLinuxContainerRuntime.java|borderStyle=solid}
> @@ -219,9 +219,9 @@ public void initialize(Configuration conf)
> dockerClient = new DockerClient(conf);
> allowedNetworks.clear();
> allowedNetworks.addAll(Arrays.asList(
> -
> conf.getStrings(YarnConfiguration.NM_DOCKER_ALLOWED_CONTAINER_NETWORKS,
> +
> conf.getTrimmedStrings(YarnConfiguration.NM_DOCKER_ALLOWED_CONTAINER_NETWORKS,
>
> YarnConfiguration.DEFAULT_NM_DOCKER_ALLOWED_CONTAINER_NETWORKS)));
> -defaultNetwork = conf.get(
> +defaultNetwork = conf.getTrimmed(
> YarnConfiguration.NM_DOCKER_DEFAULT_CONTAINER_NETWORK,
> YarnConfiguration.DEFAULT_NM_DOCKER_DEFAULT_CONTAINER_NETWORK);
>
> @@ -237,7 +237,7 @@ public void initialize(Configuration conf)
>throw new ContainerExecutionException(message);
> }
>
> -privilegedContainersAcl = new AccessControlList(conf.get(
> +privilegedContainersAcl = new AccessControlList(conf.getTrimmed(
> YarnConfiguration.NM_DOCKER_PRIVILEGED_CONTAINERS_ACL,
> YarnConfiguration.DEFAULT_NM_DOCKER_PRIVILEGED_CONTAINERS_ACL));
>}
> @@ -439,7 +439,7 @@ public void launchContainer(ContainerRuntimeContext ctx)
> LOCALIZED_RESOURCES);
> @SuppressWarnings("unchecked")
> List userLocalDirs = ctx.getExecutionAttribute(USER_LOCAL_DIRS);
> -Set capabilities = new HashSet<>(Arrays.asList(conf.getStrings(
> +Set capabilities = new
> HashSet<>(Arrays.asList(conf.getTrimmedStrings(
> YarnConfiguration.NM_DOCKER_CONTAINER_CAPABILITIES,
> YarnConfiguration.DEFAULT_NM_DOCKER_CONTAINER_CAPABILITIES)));
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3934) Application with large ApplicationSubmissionContext can cause RM to exit when ZK store is used
[
https://issues.apache.org/jira/browse/YARN-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613602#comment-15613602
]
Daniel Templeton commented on YARN-3934:
Thanks for the patch, [~cotedm]. Looking at the patch, it looks to me like it
handles the state store side of things, but it doesn't do anything to address
the RM side. If the state store throws an exception because the app size is
too big, the RM needs to know that it should reject the application rather than
considering it a critical failure.
> Application with large ApplicationSubmissionContext can cause RM to exit when
> ZK store is used
> --
>
> Key: YARN-3934
> URL: https://issues.apache.org/jira/browse/YARN-3934
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Reporter: Ming Ma
>Assignee: Dustin Cote
> Labels: oct16-easy
> Attachments: YARN-3934-1.patch
>
>
> Use the following steps to test.
> 1. Set up ZK as the RM HA store.
> 2. Submit a job that refers to lots of distributed cache files with long HDFS
> path, which will cause the app state size to exceed ZK's max object size
> limit.
> 3. RM can't write to ZK and exit with the following exception.
> {noformat}
> 2015-07-10 22:21:13,002 FATAL
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Received a
> org.apache.hadoop.yarn.server.resourcemanager.RMFatalEvent of type
> STATE_STORE_OP_FAILED. Cause:
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode
> = Session expired
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> at org.apache.zookeeper.ZooKeeper.multiInternal(ZooKeeper.java:935)
> at org.apache.zookeeper.ZooKeeper.multi(ZooKeeper.java:915)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$4.run(ZKRMStateStore.java:944)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$4.run(ZKRMStateStore.java:941)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$ZKAction.runWithCheck(ZKRMStateStore.java:1083)
> {noformat}
> In this case, RM could have rejected the app during submitApplication RPC if
> the size of ApplicationSubmissionContext is too large.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4572) TestCapacityScheduler#testHeadRoomCalculationWithDRC failing
[
https://issues.apache.org/jira/browse/YARN-4572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613597#comment-15613597
]
Hadoop QA commented on YARN-4572:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
1s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
34s{color} | {color:red} hadoop-yarn-server-resourcemanager in the patch
failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
35s{color} | {color:red} hadoop-yarn-server-resourcemanager in the patch
failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 35s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 20s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 2 new + 180 unchanged - 0 fixed = 182 total (was 180)
{color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
35s{color} | {color:red} hadoop-yarn-server-resourcemanager in the patch
failed. {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
27s{color} | {color:red} hadoop-yarn-server-resourcemanager in the patch
failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 33s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 15m 42s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | YARN-4572 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12784573/YARN-4572.1.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 6587b9d00d4c 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29
20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 022bf78 |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| mvninstall |
https://builds.apache.org/job/PreCommit-YARN-Build/13609/artifact/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| compile |
https://builds.apache.org/job/PreCommit-YARN-Build/13609/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| javac |
https://builds.apache.org/job/PreCommit-YARN-Build/13609/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| checkstyle |
[jira] [Commented] (YARN-3538) TimelineServer doesn't catch/translate all exceptions raised
[
https://issues.apache.org/jira/browse/YARN-3538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613593#comment-15613593
]
Hadoop QA commented on YARN-3538:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
1s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 10s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice:
The patch generated 1 new + 3 unchanged - 0 fixed = 4 total (was 3) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
44s{color} | {color:green} hadoop-yarn-server-applicationhistoryservice in the
patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 38s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | YARN-3538 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12727879/YARN-3538-001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux e8d63988391d 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18
21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 022bf78 |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/13606/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/13606/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/13606/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically
[jira] [Commented] (YARN-4950) configure parallel-tests for yarn-client and yarn-server-resourcemanager
[
https://issues.apache.org/jira/browse/YARN-4950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613595#comment-15613595
]
Hadoop QA commented on YARN-4950:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 12m 26s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 4m 33s{color}
| {color:red} hadoop-yarn-client in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 35m 12s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.server.resourcemanager.TestClientRMService |
| | hadoop.yarn.server.resourcemanager.TestRMAdminService |
| | hadoop.yarn.client.api.impl.TestAMRMProxy |
| | hadoop.yarn.client.TestRMFailover |
| | hadoop.yarn.client.api.impl.TestNMClient |
| | hadoop.yarn.client.TestApplicationClientProtocolOnHA |
| | hadoop.yarn.client.api.impl.TestYarnClient |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | YARN-4950 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12798334/YARN-4950.00.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit xml |
| uname | Linux af0e423bdaa4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 221582c |
| Default Java | 1.8.0_101 |
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/13596/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/13596/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/13596/testReport/ |
|
[jira] [Commented] (YARN-5793) Trim configuration values in DockerLinuxContainerRuntime
[
https://issues.apache.org/jira/browse/YARN-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613590#comment-15613590
]
Daniel Templeton commented on YARN-5793:
+1 pending Jenkins.
> Trim configuration values in DockerLinuxContainerRuntime
>
>
> Key: YARN-5793
> URL: https://issues.apache.org/jira/browse/YARN-5793
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.0.0-alpha1
>Reporter: Tianyin Xu
>Priority: Minor
> Attachments: YARN-5793..patch
>
>
> The current implementation of {{DockerLinuxContainerRuntime}} does not follow
> the practice of trimming configuration values. This leads to errors if users
> set values containing space or newline.
> see the following YARN commits as reference:
> YARN-3395. FairScheduler: Trim whitespaces when using username for queuename.
> YARN-2869. CapacityScheduler should trim sub queue names when parse
> configuration.
> YARN-2843. Fixed NodeLabelsManager to trim inputs for hosts and labels so as
> to make them work correctly.
> and many other Hadoop/HDFS commits (just list a few):
> HDFS-9708. FSNamesystem.initAuditLoggers() doesn't trim classnames
> HDFS-2799. Trim fs.checkpoint.dir values.
> HADOOP-6578. Configuration should trim whitespace around a lot of value types
> HADOOP-6534. Trim whitespace from directory lists initializing
> Patch is available against trunk
> {code:title=DockerLinuxContainerRuntime.java|borderStyle=solid}
> @@ -219,9 +219,9 @@ public void initialize(Configuration conf)
> dockerClient = new DockerClient(conf);
> allowedNetworks.clear();
> allowedNetworks.addAll(Arrays.asList(
> -
> conf.getStrings(YarnConfiguration.NM_DOCKER_ALLOWED_CONTAINER_NETWORKS,
> +
> conf.getTrimmedStrings(YarnConfiguration.NM_DOCKER_ALLOWED_CONTAINER_NETWORKS,
>
> YarnConfiguration.DEFAULT_NM_DOCKER_ALLOWED_CONTAINER_NETWORKS)));
> -defaultNetwork = conf.get(
> +defaultNetwork = conf.getTrimmed(
> YarnConfiguration.NM_DOCKER_DEFAULT_CONTAINER_NETWORK,
> YarnConfiguration.DEFAULT_NM_DOCKER_DEFAULT_CONTAINER_NETWORK);
>
> @@ -237,7 +237,7 @@ public void initialize(Configuration conf)
>throw new ContainerExecutionException(message);
> }
>
> -privilegedContainersAcl = new AccessControlList(conf.get(
> +privilegedContainersAcl = new AccessControlList(conf.getTrimmed(
> YarnConfiguration.NM_DOCKER_PRIVILEGED_CONTAINERS_ACL,
> YarnConfiguration.DEFAULT_NM_DOCKER_PRIVILEGED_CONTAINERS_ACL));
>}
> @@ -439,7 +439,7 @@ public void launchContainer(ContainerRuntimeContext ctx)
> LOCALIZED_RESOURCES);
> @SuppressWarnings("unchecked")
> List userLocalDirs = ctx.getExecutionAttribute(USER_LOCAL_DIRS);
> -Set capabilities = new HashSet<>(Arrays.asList(conf.getStrings(
> +Set capabilities = new
> HashSet<>(Arrays.asList(conf.getTrimmedStrings(
> YarnConfiguration.NM_DOCKER_CONTAINER_CAPABILITIES,
> YarnConfiguration.DEFAULT_NM_DOCKER_CONTAINER_CAPABILITIES)));
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-5793) Trim configuration values in DockerLinuxContainerRuntime
Tianyin Xu created YARN-5793:
Summary: Trim configuration values in DockerLinuxContainerRuntime
Key: YARN-5793
URL: https://issues.apache.org/jira/browse/YARN-5793
Project: Hadoop YARN
Issue Type: Bug
Components: nodemanager
Affects Versions: 3.0.0-alpha1
Reporter: Tianyin Xu
Priority: Minor
The current implementation of {{DockerLinuxContainerRuntime}} does not follow
the practice of trimming configuration values. This leads to errors if users
set values containing space or newline.
see the following YARN commits as reference:
YARN-3395. FairScheduler: Trim whitespaces when using username for queuename.
YARN-2869. CapacityScheduler should trim sub queue names when parse
configuration.
YARN-2843. Fixed NodeLabelsManager to trim inputs for hosts and labels so as to
make them work correctly.
and many other Hadoop/HDFS commits (just list a few):
HDFS-9708. FSNamesystem.initAuditLoggers() doesn't trim classnames
HDFS-2799. Trim fs.checkpoint.dir values.
HADOOP-6578. Configuration should trim whitespace around a lot of value types
HADOOP-6534. Trim whitespace from directory lists initializing
Patch is available against trunk
{code:title=DockerLinuxContainerRuntime.java|borderStyle=solid}
@@ -219,9 +219,9 @@ public void initialize(Configuration conf)
dockerClient = new DockerClient(conf);
allowedNetworks.clear();
allowedNetworks.addAll(Arrays.asList(
-conf.getStrings(YarnConfiguration.NM_DOCKER_ALLOWED_CONTAINER_NETWORKS,
+
conf.getTrimmedStrings(YarnConfiguration.NM_DOCKER_ALLOWED_CONTAINER_NETWORKS,
YarnConfiguration.DEFAULT_NM_DOCKER_ALLOWED_CONTAINER_NETWORKS)));
-defaultNetwork = conf.get(
+defaultNetwork = conf.getTrimmed(
YarnConfiguration.NM_DOCKER_DEFAULT_CONTAINER_NETWORK,
YarnConfiguration.DEFAULT_NM_DOCKER_DEFAULT_CONTAINER_NETWORK);
@@ -237,7 +237,7 @@ public void initialize(Configuration conf)
throw new ContainerExecutionException(message);
}
-privilegedContainersAcl = new AccessControlList(conf.get(
+privilegedContainersAcl = new AccessControlList(conf.getTrimmed(
YarnConfiguration.NM_DOCKER_PRIVILEGED_CONTAINERS_ACL,
YarnConfiguration.DEFAULT_NM_DOCKER_PRIVILEGED_CONTAINERS_ACL));
}
@@ -439,7 +439,7 @@ public void launchContainer(ContainerRuntimeContext ctx)
LOCALIZED_RESOURCES);
@SuppressWarnings("unchecked")
List userLocalDirs = ctx.getExecutionAttribute(USER_LOCAL_DIRS);
-Set capabilities = new HashSet<>(Arrays.asList(conf.getStrings(
+Set capabilities = new
HashSet<>(Arrays.asList(conf.getTrimmedStrings(
YarnConfiguration.NM_DOCKER_CONTAINER_CAPABILITIES,
YarnConfiguration.DEFAULT_NM_DOCKER_CONTAINER_CAPABILITIES)));
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5793) Trim configuration values in DockerLinuxContainerRuntime
[
https://issues.apache.org/jira/browse/YARN-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613588#comment-15613588
]
Daniel Templeton commented on YARN-5793:
Sorry, I jumped the gun. :)
> Trim configuration values in DockerLinuxContainerRuntime
>
>
> Key: YARN-5793
> URL: https://issues.apache.org/jira/browse/YARN-5793
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.0.0-alpha1
>Reporter: Tianyin Xu
>Priority: Minor
> Attachments: YARN-5793..patch
>
>
> The current implementation of {{DockerLinuxContainerRuntime}} does not follow
> the practice of trimming configuration values. This leads to errors if users
> set values containing space or newline.
> see the following YARN commits as reference:
> YARN-3395. FairScheduler: Trim whitespaces when using username for queuename.
> YARN-2869. CapacityScheduler should trim sub queue names when parse
> configuration.
> YARN-2843. Fixed NodeLabelsManager to trim inputs for hosts and labels so as
> to make them work correctly.
> and many other Hadoop/HDFS commits (just list a few):
> HDFS-9708. FSNamesystem.initAuditLoggers() doesn't trim classnames
> HDFS-2799. Trim fs.checkpoint.dir values.
> HADOOP-6578. Configuration should trim whitespace around a lot of value types
> HADOOP-6534. Trim whitespace from directory lists initializing
> Patch is available against trunk
> {code:title=DockerLinuxContainerRuntime.java|borderStyle=solid}
> @@ -219,9 +219,9 @@ public void initialize(Configuration conf)
> dockerClient = new DockerClient(conf);
> allowedNetworks.clear();
> allowedNetworks.addAll(Arrays.asList(
> -
> conf.getStrings(YarnConfiguration.NM_DOCKER_ALLOWED_CONTAINER_NETWORKS,
> +
> conf.getTrimmedStrings(YarnConfiguration.NM_DOCKER_ALLOWED_CONTAINER_NETWORKS,
>
> YarnConfiguration.DEFAULT_NM_DOCKER_ALLOWED_CONTAINER_NETWORKS)));
> -defaultNetwork = conf.get(
> +defaultNetwork = conf.getTrimmed(
> YarnConfiguration.NM_DOCKER_DEFAULT_CONTAINER_NETWORK,
> YarnConfiguration.DEFAULT_NM_DOCKER_DEFAULT_CONTAINER_NETWORK);
>
> @@ -237,7 +237,7 @@ public void initialize(Configuration conf)
>throw new ContainerExecutionException(message);
> }
>
> -privilegedContainersAcl = new AccessControlList(conf.get(
> +privilegedContainersAcl = new AccessControlList(conf.getTrimmed(
> YarnConfiguration.NM_DOCKER_PRIVILEGED_CONTAINERS_ACL,
> YarnConfiguration.DEFAULT_NM_DOCKER_PRIVILEGED_CONTAINERS_ACL));
>}
> @@ -439,7 +439,7 @@ public void launchContainer(ContainerRuntimeContext ctx)
> LOCALIZED_RESOURCES);
> @SuppressWarnings("unchecked")
> List userLocalDirs = ctx.getExecutionAttribute(USER_LOCAL_DIRS);
> -Set capabilities = new HashSet<>(Arrays.asList(conf.getStrings(
> +Set capabilities = new
> HashSet<>(Arrays.asList(conf.getTrimmedStrings(
> YarnConfiguration.NM_DOCKER_CONTAINER_CAPABILITIES,
> YarnConfiguration.DEFAULT_NM_DOCKER_CONTAINER_CAPABILITIES)));
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-2306) Add test for leakage of reservation metrics in fair scheduler
[
https://issues.apache.org/jira/browse/YARN-2306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613582#comment-15613582
]
Hadoop QA commented on YARN-2306:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 36m 39s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 53m 13s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.server.resourcemanager.TestRMHA |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | YARN-2306 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12835675/YARN-2306.004.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 8e3b32dcd3da 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 79ae78d |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/13590/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/13590/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/13590/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Add test for leakage of reservation metrics in fair scheduler
> -
>
> Key: YARN-2306
> URL:
[jira] [Commented] (YARN-4093) Encapsulate additional group information in the AM to RM heartbeat
[ https://issues.apache.org/jira/browse/YARN-4093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613581#comment-15613581 ] Gour Saha commented on YARN-4093: - Hi [~rgrandl], your patch does not apply to trunk anymore. Please provide a new rebased patch. Also, please look at YARN-3926 to see if it covers your usecases. > Encapsulate additional group information in the AM to RM heartbeat > -- > > Key: YARN-4093 > URL: https://issues.apache.org/jira/browse/YARN-4093 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, yarn >Affects Versions: 2.7.1 >Reporter: Robert Grandl >Assignee: Robert Grandl > Labels: patch > Attachments: AllocateRequest_extension.docx, YARN-4093.patch, > YARN-4093.v2.patch > > > In this JIRA we propose to enhance the AM RM protocol with a new message > which encapsulates additional information about group of tasks. The RM > scheduler will benefit of the additional information to take better decisions > at the scheduling time. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5793) Trim configuration values in DockerLinuxContainerRuntime
[
https://issues.apache.org/jira/browse/YARN-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613577#comment-15613577
]
Tianyin Xu commented on YARN-5793:
--
patch is attached, and tested on my local repo. perhaps wait for Hudson's
saying.
> Trim configuration values in DockerLinuxContainerRuntime
>
>
> Key: YARN-5793
> URL: https://issues.apache.org/jira/browse/YARN-5793
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.0.0-alpha1
>Reporter: Tianyin Xu
>Priority: Minor
> Attachments: YARN-5793..patch
>
>
> The current implementation of {{DockerLinuxContainerRuntime}} does not follow
> the practice of trimming configuration values. This leads to errors if users
> set values containing space or newline.
> see the following YARN commits as reference:
> YARN-3395. FairScheduler: Trim whitespaces when using username for queuename.
> YARN-2869. CapacityScheduler should trim sub queue names when parse
> configuration.
> YARN-2843. Fixed NodeLabelsManager to trim inputs for hosts and labels so as
> to make them work correctly.
> and many other Hadoop/HDFS commits (just list a few):
> HDFS-9708. FSNamesystem.initAuditLoggers() doesn't trim classnames
> HDFS-2799. Trim fs.checkpoint.dir values.
> HADOOP-6578. Configuration should trim whitespace around a lot of value types
> HADOOP-6534. Trim whitespace from directory lists initializing
> Patch is available against trunk
> {code:title=DockerLinuxContainerRuntime.java|borderStyle=solid}
> @@ -219,9 +219,9 @@ public void initialize(Configuration conf)
> dockerClient = new DockerClient(conf);
> allowedNetworks.clear();
> allowedNetworks.addAll(Arrays.asList(
> -
> conf.getStrings(YarnConfiguration.NM_DOCKER_ALLOWED_CONTAINER_NETWORKS,
> +
> conf.getTrimmedStrings(YarnConfiguration.NM_DOCKER_ALLOWED_CONTAINER_NETWORKS,
>
> YarnConfiguration.DEFAULT_NM_DOCKER_ALLOWED_CONTAINER_NETWORKS)));
> -defaultNetwork = conf.get(
> +defaultNetwork = conf.getTrimmed(
> YarnConfiguration.NM_DOCKER_DEFAULT_CONTAINER_NETWORK,
> YarnConfiguration.DEFAULT_NM_DOCKER_DEFAULT_CONTAINER_NETWORK);
>
> @@ -237,7 +237,7 @@ public void initialize(Configuration conf)
>throw new ContainerExecutionException(message);
> }
>
> -privilegedContainersAcl = new AccessControlList(conf.get(
> +privilegedContainersAcl = new AccessControlList(conf.getTrimmed(
> YarnConfiguration.NM_DOCKER_PRIVILEGED_CONTAINERS_ACL,
> YarnConfiguration.DEFAULT_NM_DOCKER_PRIVILEGED_CONTAINERS_ACL));
>}
> @@ -439,7 +439,7 @@ public void launchContainer(ContainerRuntimeContext ctx)
> LOCALIZED_RESOURCES);
> @SuppressWarnings("unchecked")
> List userLocalDirs = ctx.getExecutionAttribute(USER_LOCAL_DIRS);
> -Set capabilities = new HashSet<>(Arrays.asList(conf.getStrings(
> +Set capabilities = new
> HashSet<>(Arrays.asList(conf.getTrimmedStrings(
> YarnConfiguration.NM_DOCKER_CONTAINER_CAPABILITIES,
> YarnConfiguration.DEFAULT_NM_DOCKER_CONTAINER_CAPABILITIES)));
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4896) ProportionalPreemptionPolicy needs to handle AMResourcePercentage per partition
[
https://issues.apache.org/jira/browse/YARN-4896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613579#comment-15613579
]
Hadoop QA commented on YARN-4896:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 23s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 5 new + 458 unchanged - 3 fixed = 463 total (was 461)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 36m
31s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 52m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | YARN-4896 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12817102/YARN-4896.0003.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 6ef4ea8a3a67 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18
21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 79ae78d |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/13592/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/13592/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/13592/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> ProportionalPreemptionPolicy needs to handle AMResourcePercentage per
> partition
>
[jira] [Commented] (YARN-4994) Use MiniYARNCluster with try-with-resources in tests
[ https://issues.apache.org/jira/browse/YARN-4994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613574#comment-15613574 ] Jonathan Hung commented on YARN-4994: - [~boky01], it seems the latest patch does not apply on trunk anymore, can you rebase? Thanks! > Use MiniYARNCluster with try-with-resources in tests > > > Key: YARN-4994 > URL: https://issues.apache.org/jira/browse/YARN-4994 > Project: Hadoop YARN > Issue Type: Improvement > Components: test >Affects Versions: 2.7.0 >Reporter: Andras Bokor >Assignee: Andras Bokor >Priority: Trivial > Labels: oct16-easy > Fix For: 2.7.0 > > Attachments: HDFS-10287.01.patch, HDFS-10287.02.patch, > HDFS-10287.03.patch, YARN-4994.04.patch, YARN-4994.05.patch, > YARN-4994.06.patch, YARN-4994.07.patch > > > In tests MiniYARNCluster is used with the following pattern: > In try-catch block create a MiniYARNCluster instance and in finally block > close it. > [Try-with-resources|https://docs.oracle.com/javase/tutorial/essential/exceptions/tryResourceClose.html] > is preferred since Java7 instead of the pattern above. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-2876) In Fair Scheduler, JMX and Scheduler UI display wrong maxResource info for subqueues
[ https://issues.apache.org/jira/browse/YARN-2876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613575#comment-15613575 ] Ray Chiang commented on YARN-2876: -- Another quick note. With this patch applied to trunk, I get consistent test failures: Failed tests: TestFairScheduler.testChildMaxResources:504 App 1 is not running with the correct number of containers expected:<2> but was:<3> TestFairScheduler.testMoveWouldViolateMaxResourcesConstraints Expected exception: org.apache.hadoop.yarn.exceptions.YarnException > In Fair Scheduler, JMX and Scheduler UI display wrong maxResource info for > subqueues > > > Key: YARN-2876 > URL: https://issues.apache.org/jira/browse/YARN-2876 > Project: Hadoop YARN > Issue Type: Bug > Components: fairscheduler, resourcemanager >Reporter: Siqi Li >Assignee: Siqi Li > Labels: oct16-easy > Attachments: YARN-2876.v1.patch, YARN-2876.v2.patch, > YARN-2876.v3.patch, YARN-2876.v4.patch, screenshot-1.png > > > If a subqueue doesn't have a maxResource set in fair-scheduler.xml, JMX and > Scheduler UI will display the entire cluster capacity as its maxResource > instead of its parent queue's maxResource. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5793) Trim configuration values in DockerLinuxContainerRuntime
[
https://issues.apache.org/jira/browse/YARN-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613568#comment-15613568
]
Daniel Templeton commented on YARN-5793:
Makes sense to me. Wanna go ahead with posting those changes as a patch?
> Trim configuration values in DockerLinuxContainerRuntime
>
>
> Key: YARN-5793
> URL: https://issues.apache.org/jira/browse/YARN-5793
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.0.0-alpha1
>Reporter: Tianyin Xu
>Priority: Minor
> Attachments: YARN-5793..patch
>
>
> The current implementation of {{DockerLinuxContainerRuntime}} does not follow
> the practice of trimming configuration values. This leads to errors if users
> set values containing space or newline.
> see the following YARN commits as reference:
> YARN-3395. FairScheduler: Trim whitespaces when using username for queuename.
> YARN-2869. CapacityScheduler should trim sub queue names when parse
> configuration.
> YARN-2843. Fixed NodeLabelsManager to trim inputs for hosts and labels so as
> to make them work correctly.
> and many other Hadoop/HDFS commits (just list a few):
> HDFS-9708. FSNamesystem.initAuditLoggers() doesn't trim classnames
> HDFS-2799. Trim fs.checkpoint.dir values.
> HADOOP-6578. Configuration should trim whitespace around a lot of value types
> HADOOP-6534. Trim whitespace from directory lists initializing
> Patch is available against trunk
> {code:title=DockerLinuxContainerRuntime.java|borderStyle=solid}
> @@ -219,9 +219,9 @@ public void initialize(Configuration conf)
> dockerClient = new DockerClient(conf);
> allowedNetworks.clear();
> allowedNetworks.addAll(Arrays.asList(
> -
> conf.getStrings(YarnConfiguration.NM_DOCKER_ALLOWED_CONTAINER_NETWORKS,
> +
> conf.getTrimmedStrings(YarnConfiguration.NM_DOCKER_ALLOWED_CONTAINER_NETWORKS,
>
> YarnConfiguration.DEFAULT_NM_DOCKER_ALLOWED_CONTAINER_NETWORKS)));
> -defaultNetwork = conf.get(
> +defaultNetwork = conf.getTrimmed(
> YarnConfiguration.NM_DOCKER_DEFAULT_CONTAINER_NETWORK,
> YarnConfiguration.DEFAULT_NM_DOCKER_DEFAULT_CONTAINER_NETWORK);
>
> @@ -237,7 +237,7 @@ public void initialize(Configuration conf)
>throw new ContainerExecutionException(message);
> }
>
> -privilegedContainersAcl = new AccessControlList(conf.get(
> +privilegedContainersAcl = new AccessControlList(conf.getTrimmed(
> YarnConfiguration.NM_DOCKER_PRIVILEGED_CONTAINERS_ACL,
> YarnConfiguration.DEFAULT_NM_DOCKER_PRIVILEGED_CONTAINERS_ACL));
>}
> @@ -439,7 +439,7 @@ public void launchContainer(ContainerRuntimeContext ctx)
> LOCALIZED_RESOURCES);
> @SuppressWarnings("unchecked")
> List userLocalDirs = ctx.getExecutionAttribute(USER_LOCAL_DIRS);
> -Set capabilities = new HashSet<>(Arrays.asList(conf.getStrings(
> +Set capabilities = new
> HashSet<>(Arrays.asList(conf.getTrimmedStrings(
> YarnConfiguration.NM_DOCKER_CONTAINER_CAPABILITIES,
> YarnConfiguration.DEFAULT_NM_DOCKER_CONTAINER_CAPABILITIES)));
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5130) Mark ContainerStatus and NodeReport as evolving
[
https://issues.apache.org/jira/browse/YARN-5130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613569#comment-15613569
]
Sangjin Lee commented on YARN-5130:
---
How about using this JIRA to address the incompatibility introduced by
YARN-3866 and YARN-4293 for 2.8.0? We can add default implementations of these
methods (that simply throw an {{UnsupportedOperationException}}). That would
preserve the stability of the class while allowing those features to work
correctly still.
IMO, this should be a blocker for 2.8.0. Let me know what you think.
> Mark ContainerStatus and NodeReport as evolving
> ---
>
> Key: YARN-5130
> URL: https://issues.apache.org/jira/browse/YARN-5130
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: yarn
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Gergely Novák
>Priority: Minor
> Labels: oct16-medium
> Attachments: YARN-5130.001.patch
>
>
> It turns out that slider won't build as the {{ContainerStatus}} and
> {{NodeReport}} classes have added more abstract methods, so breaking the mock
> objects.
> While it is everyone's freedom to change things, these classes are both tagged
> {code}
> @Public
> @Stable
> {code}
> Given they aren't stable, can someone mark them as {{@Evolving}}? That way
> when downstream code breaks, we can be less disappointed
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
