[jira] [Commented] (YARN-2442) ResourceManager JMX UI does not give HA State
[ https://issues.apache.org/jira/browse/YARN-2442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16389214#comment-16389214 ] Rohith Sharma K S commented on YARN-2442: - test failure is unrelated to the patch, YARN-5240 is exiting jira id. > ResourceManager JMX UI does not give HA State > - > > Key: YARN-2442 > URL: https://issues.apache.org/jira/browse/YARN-2442 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.5.0, 2.6.0, 2.7.0 >Reporter: Nishan Shetty >Assignee: Rohith Sharma K S >Priority: Major > Labels: oct16-easy > Attachments: 0001-YARN-2442.patch, YARN-2442.02.patch > > > ResourceManager JMX UI can show the haState (INITIALIZING, ACTIVE, STANDBY, > STOPPED) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-8007) Support specifying placement constraint for task containers in SLS
[ https://issues.apache.org/jira/browse/YARN-8007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16389181#comment-16389181 ] Jiandan Yang edited comment on YARN-8007 at 3/7/18 7:43 AM: - Because currently MR specify placement constraints in the form of data locality, there is no need to specify placement constraint in MR job. I plan to implement specifying placement constraints in *generic/stream type* of SYNTH JSON format (introduced by YARN-7732) . [~cheersyang] [~leftnoteasy] Please feel free to share your opinions, thanks! was (Author: yangjiandan): Because currently MR specify placement constraints in the form of data locality, there is no need to specify placement constraint in MR job. I plan to implement specifying placement constraints in *generic/stream type* app in SLS introduced by YARN-7732. [~cheersyang] [~leftnoteasy] Please feel free to share your opinions, thanks! > Support specifying placement constraint for task containers in SLS > -- > > Key: YARN-8007 > URL: https://issues.apache.org/jira/browse/YARN-8007 > Project: Hadoop YARN > Issue Type: Sub-task > Components: scheduler-load-simulator >Reporter: Jiandan Yang >Assignee: Jiandan Yang >Priority: Major > > YARN-6592 introduces placement constraint. Currently SLS does not support > specify placement constraint. > In order to help better perf test, we should be able to support specify > placement for containers in sls configuration. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-8007) Support specifying placement constraint for task containers in SLS
[ https://issues.apache.org/jira/browse/YARN-8007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16389181#comment-16389181 ] Jiandan Yang edited comment on YARN-8007 at 3/7/18 7:37 AM: - Because currently MR specify placement constraints in the form of data locality, there is no need to specify placement constraint in MR job. I plan to implement specifying placement constraints in *generic/stream type* app in SLS introduced by YARN-7732. [~cheersyang] [~leftnoteasy] Please feel free to share your opinions, thanks! was (Author: yangjiandan): Because currently MR specify placement constraints in the form of data locality, there is no need to specify placement constraint in MR job. I plan to implement specifying placement constraints in *stream type* app in SLS introduced by YARN-7732. [~cheersyang] [~leftnoteasy] Please feel free to share your opinions, thanks! > Support specifying placement constraint for task containers in SLS > -- > > Key: YARN-8007 > URL: https://issues.apache.org/jira/browse/YARN-8007 > Project: Hadoop YARN > Issue Type: Sub-task > Components: scheduler-load-simulator >Reporter: Jiandan Yang >Assignee: Jiandan Yang >Priority: Major > > YARN-6592 introduces placement constraint. Currently SLS does not support > specify placement constraint. > In order to help better perf test, we should be able to support specify > placement for containers in sls configuration. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8007) Support specifying placement constraint for task containers in SLS
[ https://issues.apache.org/jira/browse/YARN-8007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16389181#comment-16389181 ] Jiandan Yang commented on YARN-8007: - Because currently MR specify placement constraints in the form of data locality, there is no need to specify placement constraint in MR job. I plan to implement specifying placement constraints in *stream type* app in SLS introduced by YARN-7732. [~cheersyang] [~leftnoteasy] Please feel free to share your opinions, thanks! > Support specifying placement constraint for task containers in SLS > -- > > Key: YARN-8007 > URL: https://issues.apache.org/jira/browse/YARN-8007 > Project: Hadoop YARN > Issue Type: Sub-task > Components: scheduler-load-simulator >Reporter: Jiandan Yang >Assignee: Jiandan Yang >Priority: Major > > YARN-6592 introduces placement constraint. Currently SLS does not support > specify placement constraint. > In order to help better perf test, we should be able to support specify > placement for containers in sls configuration. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8003) Backport the code structure changes in YARN-7346 to branch-2
[ https://issues.apache.org/jira/browse/YARN-8003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16389156#comment-16389156 ] Rohith Sharma K S commented on YARN-8003: - +lgtm.. committing shortly > Backport the code structure changes in YARN-7346 to branch-2 > > > Key: YARN-8003 > URL: https://issues.apache.org/jira/browse/YARN-8003 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.10.0 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-8003-branch-2.00.patch > > > As discussed in YARN-7346, we want to keep the ATSv2 source code structure in > branch-2 close to that in trunk, in order to ease any future backport. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7952) Find a way to persist the log aggregation status
[ https://issues.apache.org/jira/browse/YARN-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-7952: Attachment: YARN-7952.3.patch > Find a way to persist the log aggregation status > > > Key: YARN-7952 > URL: https://issues.apache.org/jira/browse/YARN-7952 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Xuan Gong >Assignee: Xuan Gong >Priority: Major > Attachments: YARN-7952-poc.patch, YARN-7952.1.patch, > YARN-7952.2.patch, YARN-7952.3.patch, YARN-7952.3.patch > > > In MAPREDUCE-6415, we have created a CLI to har the aggregated logs, and In > YARN-4946: RM should write out Aggregated Log Completion file flag next to > logs, we have a discussion on how we can get the log aggregation status: make > a client call to RM or get it directly from the Distributed file system(HDFS). > No matter which approach we would like to choose, we need to figure out a way > to persist the log aggregation status first. This ticket is used to track the > working progress for this purpose. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8007) Support specifying placement constraint for task containers in SLS
[ https://issues.apache.org/jira/browse/YARN-8007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiandan Yang updated YARN-8007: Issue Type: Sub-task (was: Task) Parent: YARN-5065 > Support specifying placement constraint for task containers in SLS > -- > > Key: YARN-8007 > URL: https://issues.apache.org/jira/browse/YARN-8007 > Project: Hadoop YARN > Issue Type: Sub-task > Components: scheduler-load-simulator >Reporter: Jiandan Yang >Assignee: Jiandan Yang >Priority: Major > > YARN-6592 introduces placement constraint. Currently SLS does not support > specify placement constraint. > In order to help better perf test, we should be able to support specify > placement for containers in sls configuration. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-7986) ATSv2 REST API queries do not return results for uppercase application tags
[ https://issues.apache.org/jira/browse/YARN-7986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charan Hebri reassigned YARN-7986: -- Assignee: Charan Hebri > ATSv2 REST API queries do not return results for uppercase application tags > --- > > Key: YARN-7986 > URL: https://issues.apache.org/jira/browse/YARN-7986 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Charan Hebri >Assignee: Charan Hebri >Priority: Critical > > When applications are submitted to YARN with application tags, the tags are > converted to lowercase. This can be seen on the old/new UI. But using the > original tags for ATSv2 REST API queries do not return results as they expect > the query url to have the tags in lowercase. > This is additional work for the client because each tag needs to be > lowercased before running a query. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7986) ATSv2 REST API queries do not return results for uppercase application tags
[ https://issues.apache.org/jira/browse/YARN-7986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16389056#comment-16389056 ] Charan Hebri commented on YARN-7986: [~rohithsharma] yes will assign to myself. > ATSv2 REST API queries do not return results for uppercase application tags > --- > > Key: YARN-7986 > URL: https://issues.apache.org/jira/browse/YARN-7986 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Charan Hebri >Priority: Critical > > When applications are submitted to YARN with application tags, the tags are > converted to lowercase. This can be seen on the old/new UI. But using the > original tags for ATSv2 REST API queries do not return results as they expect > the query url to have the tags in lowercase. > This is additional work for the client because each tag needs to be > lowercased before running a query. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8007) Support specifying placement constraint for task containers in SLS
Jiandan Yang created YARN-8007: --- Summary: Support specifying placement constraint for task containers in SLS Key: YARN-8007 URL: https://issues.apache.org/jira/browse/YARN-8007 Project: Hadoop YARN Issue Type: Task Components: scheduler-load-simulator Reporter: Jiandan Yang Assignee: Jiandan Yang YARN-6592 introduces placement constraint. Currently SLS does not support specify placement constraint. In order to help better perf test, we should be able to support specify placement for containers in sls configuration. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7952) Find a way to persist the log aggregation status
[ https://issues.apache.org/jira/browse/YARN-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-7952: Attachment: YARN-7952.3.patch > Find a way to persist the log aggregation status > > > Key: YARN-7952 > URL: https://issues.apache.org/jira/browse/YARN-7952 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Xuan Gong >Assignee: Xuan Gong >Priority: Major > Attachments: YARN-7952-poc.patch, YARN-7952.1.patch, > YARN-7952.2.patch, YARN-7952.3.patch > > > In MAPREDUCE-6415, we have created a CLI to har the aggregated logs, and In > YARN-4946: RM should write out Aggregated Log Completion file flag next to > logs, we have a discussion on how we can get the log aggregation status: make > a client call to RM or get it directly from the Distributed file system(HDFS). > No matter which approach we would like to choose, we need to figure out a way > to persist the log aggregation status first. This ticket is used to track the > working progress for this purpose. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-8000) Yarn Service: component instance name shows up as component name in container record
[ https://issues.apache.org/jira/browse/YARN-8000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388259#comment-16388259 ] Gour Saha edited comment on YARN-8000 at 3/7/18 5:40 AM: - [~billie.rinaldi], very good point. The UI2 does not use the API response directly and relies on what we dump into ATS. I checked that the instance name is correctly getting set to COMPONENT_INSTANCE_NAME when it gets pushed to ATS. [~csingh], it is still a good idea to test this patch in your test cluster and ensure that the "Component Instance Information" page in UI2 shows correct data? /cc [~sunilg] was (Author: gsaha): [~billie.rinaldi], very good point. The UI2 does not use the API response directly and relies on what we dump into ATS. I checked that the instance name is correctly getting set to COMPONENT_INSTANCE_NAME when it gets pushed to ATS. [~csingh], it is still a good idea to test this patch in your test cluster and ensure that the "Component Instance Information" page loads correctly in UI2? /cc [~sunilg] > Yarn Service: component instance name shows up as component name in container > record > - > > Key: YARN-8000 > URL: https://issues.apache.org/jira/browse/YARN-8000 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Chandni Singh >Assignee: Chandni Singh >Priority: Major > Attachments: YARN-8000.001.patch, YARN-8000.002.patch, > YARN-8000.003.patch, YARN-8000.004.patch > > > Yarn Service: component instance name shows up as component name in container > record -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7213) [Umbrella] Test and validate HBase-2.0.x with Atsv2
[
https://issues.apache.org/jira/browse/YARN-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16389031#comment-16389031
]
genericqa commented on YARN-7213:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 6s{color}
| {color:red} YARN-7213 does not apply to trunk. Rebase required? Wrong Branch?
See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-7213 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12898997/YARN-7213.prelim.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/19909/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> [Umbrella] Test and validate HBase-2.0.x with Atsv2
> ---
>
> Key: YARN-7213
> URL: https://issues.apache.org/jira/browse/YARN-7213
> Project: Hadoop YARN
> Issue Type: Task
>Reporter: Rohith Sharma K S
>Assignee: Rohith Sharma K S
>Priority: Major
> Attachments: YARN-7213.prelim.patch, YARN-7213.prelim.patch,
> YARN-7213.wip.patch
>
>
> Hbase-2.0.x officially support hadoop-alpha compilations. And also they are
> getting ready for Hadoop-beta release so that HBase can release their
> versions compatible with Hadoop-beta. So, this JIRA is to keep track of
> HBase-2.0 integration issues.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-7946) Update TimelineServerV2 doc as per YARN-7919
[ https://issues.apache.org/jira/browse/YARN-7946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S reassigned YARN-7946: --- Assignee: Rohith Sharma K S > Update TimelineServerV2 doc as per YARN-7919 > > > Key: YARN-7946 > URL: https://issues.apache.org/jira/browse/YARN-7946 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S >Priority: Major > > Post YARN-7919, document need to be updated for co processor jar name and > other related details if any. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7946) Update TimelineServerV2 doc as per YARN-7919
[ https://issues.apache.org/jira/browse/YARN-7946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S updated YARN-7946: Issue Type: Sub-task (was: Bug) Parent: YARN-7213 > Update TimelineServerV2 doc as per YARN-7919 > > > Key: YARN-7946 > URL: https://issues.apache.org/jira/browse/YARN-7946 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S >Priority: Major > > Post YARN-7919, document need to be updated for co processor jar name and > other related details if any. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8006) Make Hbase-2 profile as default for YARN-7055 branch
Rohith Sharma K S created YARN-8006: --- Summary: Make Hbase-2 profile as default for YARN-7055 branch Key: YARN-8006 URL: https://issues.apache.org/jira/browse/YARN-8006 Project: Hadoop YARN Issue Type: Sub-task Reporter: Rohith Sharma K S Assignee: Rohith Sharma K S In last weekly call folks discussed that we should have separate branch with hbase-2 as profile by default. Trunk default profile is hbase-1 which runs all the tests under hbase-1 profile. But for hbase-2 profile tests are not running. As per the discussion, lets keep YARN-7055 branch for hbase-2 profile as default. Any server side patches can be given to this branch as well which runs tests for hbase-2 profile. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-7212) [Atsv2] TimelineSchemaCreator fails to create flowrun table causes RegionServer down!
[
https://issues.apache.org/jira/browse/YARN-7212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohith Sharma K S resolved YARN-7212.
-
Resolution: Done
Definitely NO. I am closing this JIRA as done as part of YARN-7346
> [Atsv2] TimelineSchemaCreator fails to create flowrun table causes
> RegionServer down!
> -
>
> Key: YARN-7212
> URL: https://issues.apache.org/jira/browse/YARN-7212
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Rohith Sharma K S
>Priority: Major
>
> *Hbase-2.0* officially support *hadoop-alpha* compilations. So I was trying
> to build and test with HBase-2.0. But table schema creation fails and causes
> RegionServer to shutdown with following error
> {noformat}
> Caused by: java.lang.NoSuchMethodError:
> org.apache.hadoop.hbase.Tag.asList([BII)Ljava/util/List;
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowScanner.getCurrentAggOp(FlowScanner.java:250)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowScanner.nextInternal(FlowScanner.java:226)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowScanner.next(FlowScanner.java:145)
> at
> org.apache.hadoop.hbase.regionserver.StoreFlusher.performFlush(StoreFlusher.java:132)
> at
> org.apache.hadoop.hbase.regionserver.DefaultStoreFlusher.flushSnapshot(DefaultStoreFlusher.java:75)
> at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:973)
> at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2252)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2672)
> {noformat}
> Since HBase-2.0 community is ready to release Hadoop-3.x compatible versions,
> ATSv2 also need to support HBase-2.0 versions. For this, we need to take up a
> task of test and validate HBase-2.0 issues!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7346) Add a profile to allow optional compilation for ATSv2 with HBase-2.0
[ https://issues.apache.org/jira/browse/YARN-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16389011#comment-16389011 ] Rohith Sharma K S commented on YARN-7346: - thanks [~haibochen] for creating YARN-8003. I will take a look at it. > Add a profile to allow optional compilation for ATSv2 with HBase-2.0 > > > Key: YARN-7346 > URL: https://issues.apache.org/jira/browse/YARN-7346 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Ted Yu >Assignee: Haibo Chen >Priority: Major > Fix For: 3.2.0 > > Attachments: YARN-7346.00.patch, YARN-7346.01.patch, > YARN-7346.02.patch, YARN-7346.03-incremental.patch, YARN-7346.03.patch, > YARN-7346.04-incremental.patch, YARN-7346.04.patch, YARN-7346.05.patch, > YARN-7346.06.patch, YARN-7346.07.patch, YARN-7346.08-incremental.patch, > YARN-7346.08.patch, YARN-7346.09.patch, YARN-7346.10.patch, > YARN-7346.11.patch, YARN-7346.prelim1.patch, YARN-7346.prelim2.patch, > YARN-7581.prelim.patch, > hadoop-yarn-server-timelineservice-hbase-server-1-findbugsXml.xml, > hadoop-yarn-server-timelineservice-hbase-server-1-javadoc-report.txt, > hadoop-yarn-server-timelineservice-hbase-server-2-findbugsXml.xml, > hadoop-yarn-server-timelineservice-hbase-server-2-javadoc-report.txt > > > When compiling hadoop-yarn-server-timelineservice-hbase against 2.0.0-alpha3, > I got the following errors: > [https://pastebin.com/Ms4jYEVB] > This issue is to fix the compilation errors. > The scope of the Jira is to add a profile to allow optional compilation for > ATSv2 with HBase2.0. The default compilation for trunk will still be for > hbase 1.2.6. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7657) Queue Mapping could provide options to provide 'user' specific auto-created queues under a specified group parent queue
[
https://issues.apache.org/jira/browse/YARN-7657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388784#comment-16388784
]
genericqa commented on YARN-7657:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 20s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
7s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 31s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 21 new + 110 unchanged - 0 fixed = 131 total (was 110)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 9s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 65m
7s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}112m 44s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f |
| JIRA Issue | YARN-7657 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12913276/YARN-7657.3.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 236c516bbd58 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18
11:55:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 346caa2 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/19907/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/19907/testReport/ |
| Max. process+thread count | 799 (vs. ulimit of 1) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
[jira] [Commented] (YARN-7677) Docker image cannot set HADOOP_CONF_DIR
[
https://issues.apache.org/jira/browse/YARN-7677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388772#comment-16388772
]
Eric Yang commented on YARN-7677:
-
+1 works on my system.
> Docker image cannot set HADOOP_CONF_DIR
> ---
>
> Key: YARN-7677
> URL: https://issues.apache.org/jira/browse/YARN-7677
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Eric Badger
>Assignee: Jim Brennan
>Priority: Major
> Attachments: YARN-7677.001.patch, YARN-7677.002.patch,
> YARN-7677.003.patch, YARN-7677.004.patch, YARN-7677.005.patch,
> YARN-7677.006.patch, YARN-7677.007.patch
>
>
> Currently, {{HADOOP_CONF_DIR}} is being put into the task environment whether
> it's set by the user or not. It completely bypasses the whitelist and so
> there is no way for a task to not have {{HADOOP_CONF_DIR}} set. This causes
> problems in the Docker use case where Docker containers will set up their own
> environment and have their own {{HADOOP_CONF_DIR}} preset in the image
> itself.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8000) Yarn Service: component instance name shows up as component name in container record
[
https://issues.apache.org/jira/browse/YARN-8000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388762#comment-16388762
]
genericqa commented on YARN-8000:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 24s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
15s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 20s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 2 new + 19 unchanged - 1 fixed = 21 total (was 20) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 12s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
40s{color} | {color:green} hadoop-yarn-services-core in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
32s{color} | {color:green} hadoop-yarn-services-api in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
20s{color} | {color:green} hadoop-yarn-site in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
36s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 76m 37s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce
[jira] [Created] (YARN-8005) Add unit tests for queue priority with dominant resource calculator
Sumana Sathish created YARN-8005: Summary: Add unit tests for queue priority with dominant resource calculator Key: YARN-8005 URL: https://issues.apache.org/jira/browse/YARN-8005 Project: Hadoop YARN Issue Type: Bug Reporter: Sumana Sathish Assignee: Wangda Tan -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8004) Add unit tests for inter queue preemption for dominant resource calculator
Sumana Sathish created YARN-8004: Summary: Add unit tests for inter queue preemption for dominant resource calculator Key: YARN-8004 URL: https://issues.apache.org/jira/browse/YARN-8004 Project: Hadoop YARN Issue Type: Bug Components: yarn Reporter: Sumana Sathish Assignee: Wangda Tan -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7523) Introduce description and version field in Service record
[
https://issues.apache.org/jira/browse/YARN-7523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388728#comment-16388728
]
genericqa commented on YARN-7523:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
50s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 12 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 43s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
26s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 7s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 2 new + 53 unchanged - 0 fixed = 55 total (was 53) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 39s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
55s{color} | {color:green} hadoop-yarn-services-core in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
30s{color} | {color:green} hadoop-yarn-services-api in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
14s{color} | {color:green} hadoop-yarn-site in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
31s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 72m 19s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f |
| JIRA Issue | YARN-7523 |
| JIRA Patch URL |
[jira] [Commented] (YARN-4781) Support intra-queue preemption for fairness ordering policy.
[ https://issues.apache.org/jira/browse/YARN-4781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388701#comment-16388701 ] Eric Payne commented on YARN-4781: -- bq.I have a patch coded up if you would be interested. For reference, uploading patch V001 > Support intra-queue preemption for fairness ordering policy. > > > Key: YARN-4781 > URL: https://issues.apache.org/jira/browse/YARN-4781 > Project: Hadoop YARN > Issue Type: Sub-task > Components: scheduler >Reporter: Wangda Tan >Assignee: Wangda Tan >Priority: Major > Attachments: YARN-4781.001.patch > > > We introduced fairness queue policy since YARN-3319, which will let large > applications make progresses and not starve small applications. However, if a > large application takes the queue’s resources, and containers of the large > app has long lifespan, small applications could still wait for resources for > long time and SLAs cannot be guaranteed. > Instead of wait for application release resources on their own, we need to > preempt resources of queue with fairness policy enabled. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-4781) Support intra-queue preemption for fairness ordering policy.
[ https://issues.apache.org/jira/browse/YARN-4781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-4781: - Attachment: YARN-4781.001.patch > Support intra-queue preemption for fairness ordering policy. > > > Key: YARN-4781 > URL: https://issues.apache.org/jira/browse/YARN-4781 > Project: Hadoop YARN > Issue Type: Sub-task > Components: scheduler >Reporter: Wangda Tan >Assignee: Wangda Tan >Priority: Major > Attachments: YARN-4781.001.patch > > > We introduced fairness queue policy since YARN-3319, which will let large > applications make progresses and not starve small applications. However, if a > large application takes the queue’s resources, and containers of the large > app has long lifespan, small applications could still wait for resources for > long time and SLAs cannot be guaranteed. > Instead of wait for application release resources on their own, we need to > preempt resources of queue with fairness policy enabled. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7891) LogAggregationIndexedFileController should support HAR file
[ https://issues.apache.org/jira/browse/YARN-7891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388686#comment-16388686 ] Wangda Tan commented on YARN-7891: -- Thanks [~xgong] for confirmation, +1 to latest patch. Will commit by tomorrow if no objections. > LogAggregationIndexedFileController should support HAR file > --- > > Key: YARN-7891 > URL: https://issues.apache.org/jira/browse/YARN-7891 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Xuan Gong >Assignee: Xuan Gong >Priority: Major > Attachments: YARN-7891.1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7626) Allow regular expression matching in container-executor.cfg for devices and named docker volumes mount
[ https://issues.apache.org/jira/browse/YARN-7626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388685#comment-16388685 ] Wangda Tan commented on YARN-7626: -- Thanks [~miklos.szeg...@cloudera.com]/[~Zian Chen], will commit the patch by tomorrow if no objections. > Allow regular expression matching in container-executor.cfg for devices and > named docker volumes mount > -- > > Key: YARN-7626 > URL: https://issues.apache.org/jira/browse/YARN-7626 > Project: Hadoop YARN > Issue Type: New Feature >Reporter: Zian Chen >Assignee: Zian Chen >Priority: Major > Attachments: YARN-7626.001.patch, YARN-7626.002.patch, > YARN-7626.003.patch, YARN-7626.004.patch, YARN-7626.005.patch, > YARN-7626.006.patch, YARN-7626.007.patch, YARN-7626.008.patch, > YARN-7626.009.patch, YARN-7626.010.patch, YARN-7626.011.patch > > > Currently when we config some of the GPU devices related fields (like ) in > container-executor.cfg, these fields are generated based on different driver > versions or GPU device names. We want to enable regular expression matching > so that user don't need to manually set up these fields when config > container-executor.cfg, -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8000) Yarn Service: component instance name shows up as component name in container record
[ https://issues.apache.org/jira/browse/YARN-8000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandni Singh updated YARN-8000: Attachment: YARN-8000.004.patch > Yarn Service: component instance name shows up as component name in container > record > - > > Key: YARN-8000 > URL: https://issues.apache.org/jira/browse/YARN-8000 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Chandni Singh >Assignee: Chandni Singh >Priority: Major > Attachments: YARN-8000.001.patch, YARN-8000.002.patch, > YARN-8000.003.patch, YARN-8000.004.patch > > > Yarn Service: component instance name shows up as component name in container > record -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7657) Queue Mapping could provide options to provide 'user' specific auto-created queues under a specified group parent queue
[ https://issues.apache.org/jira/browse/YARN-7657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388649#comment-16388649 ] Suma Shivaprasad commented on YARN-7657: Attached patch with some UTS fixed which were failing. > Queue Mapping could provide options to provide 'user' specific auto-created > queues under a specified group parent queue > --- > > Key: YARN-7657 > URL: https://issues.apache.org/jira/browse/YARN-7657 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler >Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad >Priority: Major > Attachments: YARN-7657.1.patch, YARN-7657.2.patch, YARN-7657.3.patch > > > Current Queue-Mapping only provides %user as an option for 'user' specific > queues as u:%user:%user. We can also support %user with group as > 'g:makerting-group:marketing.%user' and user specific queues can be > automatically created under a group queue in this case. > cc [~leftnoteasy] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7657) Queue Mapping could provide options to provide 'user' specific auto-created queues under a specified group parent queue
[ https://issues.apache.org/jira/browse/YARN-7657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated YARN-7657: --- Attachment: YARN-7657.3.patch > Queue Mapping could provide options to provide 'user' specific auto-created > queues under a specified group parent queue > --- > > Key: YARN-7657 > URL: https://issues.apache.org/jira/browse/YARN-7657 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler >Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad >Priority: Major > Attachments: YARN-7657.1.patch, YARN-7657.2.patch, YARN-7657.3.patch > > > Current Queue-Mapping only provides %user as an option for 'user' specific > queues as u:%user:%user. We can also support %user with group as > 'g:makerting-group:marketing.%user' and user specific queues can be > automatically created under a group queue in this case. > cc [~leftnoteasy] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7523) Introduce description and version field in Service record
[ https://issues.apache.org/jira/browse/YARN-7523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandni Singh updated YARN-7523: Attachment: YARN-7523.004.patch > Introduce description and version field in Service record > - > > Key: YARN-7523 > URL: https://issues.apache.org/jira/browse/YARN-7523 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Gour Saha >Assignee: Chandni Singh >Priority: Major > Fix For: yarn-native-services > > Attachments: YARN-7523.001.patch, YARN-7523.002.patch, > YARN-7523.003.patch, YARN-7523.004.patch > > > YARN-7512 would need version field in Service record. It would be good to > introduce a description field also to allow service owners to capture some > details which can be used to display in Service catalog as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7677) Docker image cannot set HADOOP_CONF_DIR
[
https://issues.apache.org/jira/browse/YARN-7677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388632#comment-16388632
]
Jason Lowe commented on YARN-7677:
--
Thanks for updating the patch!
+1 lgtm. I'll commit this tomorrow if there are no objections.
> Docker image cannot set HADOOP_CONF_DIR
> ---
>
> Key: YARN-7677
> URL: https://issues.apache.org/jira/browse/YARN-7677
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Eric Badger
>Assignee: Jim Brennan
>Priority: Major
> Attachments: YARN-7677.001.patch, YARN-7677.002.patch,
> YARN-7677.003.patch, YARN-7677.004.patch, YARN-7677.005.patch,
> YARN-7677.006.patch, YARN-7677.007.patch
>
>
> Currently, {{HADOOP_CONF_DIR}} is being put into the task environment whether
> it's set by the user or not. It completely bypasses the whitelist and so
> there is no way for a task to not have {{HADOOP_CONF_DIR}} set. This causes
> problems in the Docker use case where Docker containers will set up their own
> environment and have their own {{HADOOP_CONF_DIR}} preset in the image
> itself.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7346) Add a profile to allow optional compilation for ATSv2 with HBase-2.0
[
https://issues.apache.org/jira/browse/YARN-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388617#comment-16388617
]
Haibo Chen commented on YARN-7346:
--
{quote}maybe we should have a jira to track figuring out how we make sure the
hbase shaded client is usable for Hadoop in this specific context
{quote}
YARN-5071 is where we have talked about some issues in this category.
> Add a profile to allow optional compilation for ATSv2 with HBase-2.0
>
>
> Key: YARN-7346
> URL: https://issues.apache.org/jira/browse/YARN-7346
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Ted Yu
>Assignee: Haibo Chen
>Priority: Major
> Fix For: 3.2.0
>
> Attachments: YARN-7346.00.patch, YARN-7346.01.patch,
> YARN-7346.02.patch, YARN-7346.03-incremental.patch, YARN-7346.03.patch,
> YARN-7346.04-incremental.patch, YARN-7346.04.patch, YARN-7346.05.patch,
> YARN-7346.06.patch, YARN-7346.07.patch, YARN-7346.08-incremental.patch,
> YARN-7346.08.patch, YARN-7346.09.patch, YARN-7346.10.patch,
> YARN-7346.11.patch, YARN-7346.prelim1.patch, YARN-7346.prelim2.patch,
> YARN-7581.prelim.patch,
> hadoop-yarn-server-timelineservice-hbase-server-1-findbugsXml.xml,
> hadoop-yarn-server-timelineservice-hbase-server-1-javadoc-report.txt,
> hadoop-yarn-server-timelineservice-hbase-server-2-findbugsXml.xml,
> hadoop-yarn-server-timelineservice-hbase-server-2-javadoc-report.txt
>
>
> When compiling hadoop-yarn-server-timelineservice-hbase against 2.0.0-alpha3,
> I got the following errors:
> [https://pastebin.com/Ms4jYEVB]
> This issue is to fix the compilation errors.
> The scope of the Jira is to add a profile to allow optional compilation for
> ATSv2 with HBase2.0. The default compilation for trunk will still be for
> hbase 1.2.6.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7999) Docker launch fails when user private filecache directory is missing
[
https://issues.apache.org/jira/browse/YARN-7999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388606#comment-16388606
]
Jason Lowe commented on YARN-7999:
--
The unit test failure is unrelated. It passes for me locally with the patch
applied. Also TestContainerManager#testContainerUpgradeRollbackDueToFailure
uses DefaultContainerExecutor, so it does not execute this code path at all.
> Docker launch fails when user private filecache directory is missing
>
>
> Key: YARN-7999
> URL: https://issues.apache.org/jira/browse/YARN-7999
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: Eric Yang
>Assignee: Jason Lowe
>Priority: Major
> Attachments: YARN-7999.001.patch, YARN-7999.002.patch
>
>
> Docker container is failing to launch in trunk. The root cause is:
> {code}
> [COMPINSTANCE sleeper-1 : container_1520032931921_0001_01_20]:
> [2018-03-02 23:26:09.196]Exception from container-launch.
> Container id: container_1520032931921_0001_01_20
> Exit code: 29
> Exception message: image: hadoop/centos:latest is trusted in hadoop registry.
> Could not determine real path of mount
> '/tmp/hadoop-yarn/nm-local-dir/usercache/hbase/filecache'
> Could not determine real path of mount
> '/tmp/hadoop-yarn/nm-local-dir/usercache/hbase/filecache'
> Invalid docker mount
> '/tmp/hadoop-yarn/nm-local-dir/usercache/hbase/filecache:/tmp/hadoop-yarn/nm-local-dir/usercache/hbase/filecache',
> realpath=/tmp/hadoop-yarn/nm-local-dir/usercache/hbase/filecache
> Error constructing docker command, docker error code=12, error
> message='Invalid docker mount'
> Shell output: main : command provided 4
> main : run as user is hbase
> main : requested yarn user is hbase
> Creating script paths...
> Creating local dirs...
> [2018-03-02 23:26:09.240]Diagnostic message from attempt 0 : [2018-03-02
> 23:26:09.240]
> [2018-03-02 23:26:09.240]Container exited with a non-zero exit code 29.
> [2018-03-02 23:26:39.278]Could not find
> nmPrivate/application_1520032931921_0001/container_1520032931921_0001_01_20//container_1520032931921_0001_01_20.pid
> in any of the directories
> [COMPONENT sleeper]: Failed 11 times, exceeded the limit - 10. Shutting down
> now...
> {code}
> The filecache cant not be mounted because it doesn't exist.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7346) Add a profile to allow optional compilation for ATSv2 with HBase-2.0
[ https://issues.apache.org/jira/browse/YARN-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388603#comment-16388603 ] Haibo Chen commented on YARN-7346: -- Filed YARN-8003 to change the code structure in branch-2. > Add a profile to allow optional compilation for ATSv2 with HBase-2.0 > > > Key: YARN-7346 > URL: https://issues.apache.org/jira/browse/YARN-7346 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Ted Yu >Assignee: Haibo Chen >Priority: Major > Fix For: 3.2.0 > > Attachments: YARN-7346.00.patch, YARN-7346.01.patch, > YARN-7346.02.patch, YARN-7346.03-incremental.patch, YARN-7346.03.patch, > YARN-7346.04-incremental.patch, YARN-7346.04.patch, YARN-7346.05.patch, > YARN-7346.06.patch, YARN-7346.07.patch, YARN-7346.08-incremental.patch, > YARN-7346.08.patch, YARN-7346.09.patch, YARN-7346.10.patch, > YARN-7346.11.patch, YARN-7346.prelim1.patch, YARN-7346.prelim2.patch, > YARN-7581.prelim.patch, > hadoop-yarn-server-timelineservice-hbase-server-1-findbugsXml.xml, > hadoop-yarn-server-timelineservice-hbase-server-1-javadoc-report.txt, > hadoop-yarn-server-timelineservice-hbase-server-2-findbugsXml.xml, > hadoop-yarn-server-timelineservice-hbase-server-2-javadoc-report.txt > > > When compiling hadoop-yarn-server-timelineservice-hbase against 2.0.0-alpha3, > I got the following errors: > [https://pastebin.com/Ms4jYEVB] > This issue is to fix the compilation errors. > The scope of the Jira is to add a profile to allow optional compilation for > ATSv2 with HBase2.0. The default compilation for trunk will still be for > hbase 1.2.6. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8003) Backport the code structure changes in YARN-7346 to branch-2
[
https://issues.apache.org/jira/browse/YARN-8003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388596#comment-16388596
]
genericqa commented on YARN-8003:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 15m
28s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
42s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
49s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
0s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
37s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
9s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
23s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
20s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
15s{color} | {color:green} hadoop-project in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
16s{color} | {color:green} hadoop-assemblies in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
26s{color} | {color:green} hadoop-yarn-server-timelineservice-hbase-client in
the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
22s{color} | {color:green} hadoop-yarn-server-timelineservice-hbase-server in
the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
21s{color} | {color:green} hadoop-yarn-server-timelineservice-hbase-server-1 in
the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 6m
43s{color} | {color:green} hadoop-yarn-server-timelineservice-hbase-tests in
the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
35s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 72m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8ad3ec6 |
| JIRA Issue | YARN-8003 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12913264/YARN-8003-branch-2.00.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient xml findbugs checkstyle |
| uname | Linux 53e978708b39 4.4.0-64-generic #85-Ubuntu SMP Mon Feb 20
11:50:30 UTC 2017 x86_64 x86_64 x86_64
[jira] [Commented] (YARN-7523) Introduce description and version field in Service record
[
https://issues.apache.org/jira/browse/YARN-7523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388540#comment-16388540
]
genericqa commented on YARN-7523:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
27s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 12 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
9s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 58s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
54s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 1s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 2 new + 53 unchanged - 0 fixed = 55 total (was 53) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 25s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
37s{color} | {color:green} hadoop-yarn-services-core in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
26s{color} | {color:green} hadoop-yarn-services-api in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
14s{color} | {color:green} hadoop-yarn-site in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 66m 29s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f |
| JIRA Issue | YARN-7523 |
|
[jira] [Updated] (YARN-8003) Backport the code structure changes in YARN-7346 to branch-2
[ https://issues.apache.org/jira/browse/YARN-8003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-8003: - Attachment: YARN-8003-branch-2.00.patch > Backport the code structure changes in YARN-7346 to branch-2 > > > Key: YARN-8003 > URL: https://issues.apache.org/jira/browse/YARN-8003 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.10.0 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-8003-branch-2.00.patch > > > As discussed in YARN-7346, we want to keep the ATSv2 source code structure in > branch-2 close to that in trunk, in order to ease any future backport. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8003) Backport the code structure changes in YARN-7346 to branch-2
[ https://issues.apache.org/jira/browse/YARN-8003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388510#comment-16388510 ] Haibo Chen commented on YARN-8003: -- The patch is based on YARN-7346.11.patch with all the hbase profile bits in pom.xml removed. > Backport the code structure changes in YARN-7346 to branch-2 > > > Key: YARN-8003 > URL: https://issues.apache.org/jira/browse/YARN-8003 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.10.0 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-8003-branch-2.00.patch > > > As discussed in YARN-7346, we want to keep the ATSv2 source code structure in > branch-2 close to that in trunk, in order to ease any future backport. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7581) HBase filters are not constructed correctly in ATSv2
[
https://issues.apache.org/jira/browse/YARN-7581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388498#comment-16388498
]
genericqa commented on YARN-7581:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
30s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
8m 57s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 9s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-client:
The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 33s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
34s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-client
generated 8 new + 0 unchanged - 0 fixed = 8 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
20s{color} | {color:green} hadoop-yarn-server-timelineservice-hbase-client in
the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 40m 1s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs |
module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-client
|
| | Found reliance on default encoding in
org.apache.hadoop.yarn.server.timelineservice.storage.reader.ApplicationEntityReader.constructFilterListBasedOnFields(Set):in
org.apache.hadoop.yarn.server.timelineservice.storage.reader.ApplicationEntityReader.constructFilterListBasedOnFields(Set):
new String(byte[]) At ApplicationEntityReader.java:[line 310] |
| | Found reliance on default encoding in
org.apache.hadoop.yarn.server.timelineservice.storage.reader.ApplicationEntityReader.updateFilterForConfsAndMetricsToRetrieve(FilterList,
Set):in
[jira] [Updated] (YARN-7523) Introduce description and version field in Service record
[ https://issues.apache.org/jira/browse/YARN-7523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandni Singh updated YARN-7523: Attachment: YARN-7523.003.patch > Introduce description and version field in Service record > - > > Key: YARN-7523 > URL: https://issues.apache.org/jira/browse/YARN-7523 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Gour Saha >Assignee: Chandni Singh >Priority: Major > Fix For: yarn-native-services > > Attachments: YARN-7523.001.patch, YARN-7523.002.patch, > YARN-7523.003.patch > > > YARN-7512 would need version field in Service record. It would be good to > introduce a description field also to allow service owners to capture some > details which can be used to display in Service catalog as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7581) HBase filters are not constructed correctly in ATSv2
[ https://issues.apache.org/jira/browse/YARN-7581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-7581: - Attachment: YARN-7581.00.patch > HBase filters are not constructed correctly in ATSv2 > > > Key: YARN-7581 > URL: https://issues.apache.org/jira/browse/YARN-7581 > Project: Hadoop YARN > Issue Type: Sub-task > Components: ATSv2 >Affects Versions: 3.0.0-beta1 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-7581.00.patch > > > Post YARN-7346, > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters() and > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters() > start to fail when hbase.profile is set to 2.0) > *Error Message* > [ERROR] Failures: > [ERROR] > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters:1266 > expected:<2> but was:<0> > [ERROR] > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters:1523 > expected:<1> but was:<0> -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7581) HBase filters are not constructed correctly in ATSv2
[ https://issues.apache.org/jira/browse/YARN-7581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-7581: - Attachment: (was: YARN-7581.prelim.patch) > HBase filters are not constructed correctly in ATSv2 > > > Key: YARN-7581 > URL: https://issues.apache.org/jira/browse/YARN-7581 > Project: Hadoop YARN > Issue Type: Sub-task > Components: ATSv2 >Affects Versions: 3.0.0-beta1 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-7581.00.patch > > > Post YARN-7346, > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters() and > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters() > start to fail when hbase.profile is set to 2.0) > *Error Message* > [ERROR] Failures: > [ERROR] > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters:1266 > expected:<2> but was:<0> > [ERROR] > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters:1523 > expected:<1> but was:<0> -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7581) HBase filters are not constructed correctly in ATSv2
[ https://issues.apache.org/jira/browse/YARN-7581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-7581: - Summary: HBase filters are not constructed correctly in ATSv2 (was: ATSv2 does not construct HBase filters correctly in HBase 2.0) > HBase filters are not constructed correctly in ATSv2 > > > Key: YARN-7581 > URL: https://issues.apache.org/jira/browse/YARN-7581 > Project: Hadoop YARN > Issue Type: Sub-task > Components: ATSv2 >Affects Versions: 3.0.0-beta1 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-7581.prelim.patch > > > Post YARN-7346, > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters() and > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters() > start to fail when hbase.profile is set to 2.0) > *Error Message* > [ERROR] Failures: > [ERROR] > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters:1266 > expected:<2> but was:<0> > [ERROR] > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters:1523 > expected:<1> but was:<0> -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8003) Backport the code structure changes in YARN-7346 to branch-2
[ https://issues.apache.org/jira/browse/YARN-8003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-8003: - Issue Type: Sub-task (was: Improvement) Parent: YARN-7213 > Backport the code structure changes in YARN-7346 to branch-2 > > > Key: YARN-8003 > URL: https://issues.apache.org/jira/browse/YARN-8003 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.10.0 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > > As discussed in YARN-7346, we want to keep the ATSv2 source code structure in > branch-2 close to that in trunk, in order to ease any future backport. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8003) Backport the code structure changes in YARN-7346 to branch-2
Haibo Chen created YARN-8003: Summary: Backport the code structure changes in YARN-7346 to branch-2 Key: YARN-8003 URL: https://issues.apache.org/jira/browse/YARN-8003 Project: Hadoop YARN Issue Type: Improvement Affects Versions: 2.10.0 Reporter: Haibo Chen Assignee: Haibo Chen As discussed in YARN-7346, we want to keep the ATSv2 source code structure in branch-2 close to that in trunk, in order to ease any future backport. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7953) [GQ] Data structures for federation global queues calculations
[ https://issues.apache.org/jira/browse/YARN-7953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388325#comment-16388325 ] Carlo Curino commented on YARN-7953: [~asuresh] I like this suggestion, and from our offline convo I know you are looking into it, please let me know if it looks promising once you tested the ideas. Since the FederationQueue objects I have here are transformed in YARN-7403 and YARN-7834 in other objects for algorithmic calculations, this should be pretty doable in terms of the rest of the YARN-7402 work items. Small caveats: # The reason I had initially not used QueueMetrics is that I saw them being broken/off often in live clusters, so I thought they were maintained a bit sloppily. If we can assure they are correct and consistent I think it is good to have them. # Also we should validate whether the polling of QueueMetrics is for performance (might be better due to already maintained objects and the delta protocol, but want to make sure). # The other advantage of the FedQueue objects was the fact that were very easy to build tests by constructing scenarios in .json. If we can do the same for QueueMetrics and the alike, I think it should be good. > [GQ] Data structures for federation global queues calculations > -- > > Key: YARN-7953 > URL: https://issues.apache.org/jira/browse/YARN-7953 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Carlo Curino >Assignee: Carlo Curino >Priority: Major > Attachments: YARN-7953.v1.patch > > > This Jira tracks data structures and helper classes used by the core > algorithms of YARN-7402 umbrella Jira (currently YARN-7403, and YARN-7834). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7875) AttributeStore for store and recover attributes
[ https://issues.apache.org/jira/browse/YARN-7875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388289#comment-16388289 ] Bibin A Chundatt commented on YARN-7875: [~sunilg]/[~naganarasimha...@apache.org]/[~cheersyang] Attaching store implementation for NodeAttribute based in YARN-7988. Please help in review. > AttributeStore for store and recover attributes > --- > > Key: YARN-7875 > URL: https://issues.apache.org/jira/browse/YARN-7875 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt >Priority: Major > Attachments: YARN-7875-WIP.patch, YARN-7875-YARN-3409.001.patch, > YARN-7875-YARN-3409.002.patch > > > Similar to NodeLabelStore need to support NodeAttributeStore for persisting > attributes mapping to Nodes. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7875) AttributeStore for store and recover attributes
[ https://issues.apache.org/jira/browse/YARN-7875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bibin A Chundatt updated YARN-7875: --- Attachment: YARN-7875-YARN-3409.002.patch > AttributeStore for store and recover attributes > --- > > Key: YARN-7875 > URL: https://issues.apache.org/jira/browse/YARN-7875 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt >Priority: Major > Attachments: YARN-7875-WIP.patch, YARN-7875-YARN-3409.001.patch, > YARN-7875-YARN-3409.002.patch > > > Similar to NodeLabelStore need to support NodeAttributeStore for persisting > attributes mapping to Nodes. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8000) Yarn Service: component instance name shows up as component name in container record
[ https://issues.apache.org/jira/browse/YARN-8000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388259#comment-16388259 ] Gour Saha commented on YARN-8000: - [~billie.rinaldi], very good point. The UI2 does not use the API response directly and relies on what we dump into ATS. I checked that the instance name is correctly getting set to COMPONENT_INSTANCE_NAME when it gets pushed to ATS. [~csingh], it is still a good idea to test this patch in your test cluster and ensure that the "Component Instance Information" page loads correctly in UI2? /cc [~sunilg] > Yarn Service: component instance name shows up as component name in container > record > - > > Key: YARN-8000 > URL: https://issues.apache.org/jira/browse/YARN-8000 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Chandni Singh >Assignee: Chandni Singh >Priority: Major > Attachments: YARN-8000.001.patch, YARN-8000.002.patch, > YARN-8000.003.patch > > > Yarn Service: component instance name shows up as component name in container > record -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7581) ATSv2 does not construct HBase filters correctly in HBase 2.0
[ https://issues.apache.org/jira/browse/YARN-7581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-7581: - Description: Post YARN-7346, TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters() and TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters() start to fail when hbase.profile is set to 2.0) *Error Message* [ERROR] Failures: [ERROR] TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters:1266 expected:<2> but was:<0> [ERROR] TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters:1523 expected:<1> but was:<0> was: TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters() and TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters() start to fail after we upgrade HBase to 2.0-alpha4 (To reproduce locally, apply YARN-7581.00.patch that is attached in YARN-7346 and run the atsv2 unit tests) *Error Message* [ERROR] Failures: [ERROR] TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters:1266 expected:<2> but was:<0> [ERROR] TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters:1523 expected:<1> but was:<0> > ATSv2 does not construct HBase filters correctly in HBase 2.0 > - > > Key: YARN-7581 > URL: https://issues.apache.org/jira/browse/YARN-7581 > Project: Hadoop YARN > Issue Type: Sub-task > Components: ATSv2 >Affects Versions: 3.0.0-beta1 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-7581.prelim.patch > > > Post YARN-7346, > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters() and > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters() > start to fail when hbase.profile is set to 2.0) > *Error Message* > [ERROR] Failures: > [ERROR] > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters:1266 > expected:<2> but was:<0> > [ERROR] > TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters:1523 > expected:<1> but was:<0> -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7523) Introduce description and version field in Service record
[ https://issues.apache.org/jira/browse/YARN-7523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388252#comment-16388252 ] Chandni Singh commented on YARN-7523: - I have missed this file: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/yarn-service/YarnServiceAPI.md I will upload a new patch that includes changes to that as well. > Introduce description and version field in Service record > - > > Key: YARN-7523 > URL: https://issues.apache.org/jira/browse/YARN-7523 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Gour Saha >Assignee: Chandni Singh >Priority: Major > Fix For: yarn-native-services > > Attachments: YARN-7523.001.patch, YARN-7523.002.patch > > > YARN-7512 would need version field in Service record. It would be good to > introduce a description field also to allow service owners to capture some > details which can be used to display in Service catalog as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7212) [Atsv2] TimelineSchemaCreator fails to create flowrun table causes RegionServer down!
[
https://issues.apache.org/jira/browse/YARN-7212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388246#comment-16388246
]
Haibo Chen commented on YARN-7212:
--
[~rohithsharma] Is this still going to be an issue post YARN-7346?
> [Atsv2] TimelineSchemaCreator fails to create flowrun table causes
> RegionServer down!
> -
>
> Key: YARN-7212
> URL: https://issues.apache.org/jira/browse/YARN-7212
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Rohith Sharma K S
>Priority: Major
>
> *Hbase-2.0* officially support *hadoop-alpha* compilations. So I was trying
> to build and test with HBase-2.0. But table schema creation fails and causes
> RegionServer to shutdown with following error
> {noformat}
> Caused by: java.lang.NoSuchMethodError:
> org.apache.hadoop.hbase.Tag.asList([BII)Ljava/util/List;
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowScanner.getCurrentAggOp(FlowScanner.java:250)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowScanner.nextInternal(FlowScanner.java:226)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowScanner.next(FlowScanner.java:145)
> at
> org.apache.hadoop.hbase.regionserver.StoreFlusher.performFlush(StoreFlusher.java:132)
> at
> org.apache.hadoop.hbase.regionserver.DefaultStoreFlusher.flushSnapshot(DefaultStoreFlusher.java:75)
> at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:973)
> at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2252)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2672)
> {noformat}
> Since HBase-2.0 community is ready to release Hadoop-3.x compatible versions,
> ATSv2 also need to support HBase-2.0 versions. For this, we need to take up a
> task of test and validate HBase-2.0 issues!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7935) Expose container's hostname to applications running within the docker container
[
https://issues.apache.org/jira/browse/YARN-7935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388237#comment-16388237
]
Shane Kumpf commented on YARN-7935:
---
Thanks for the patch [~suma.shivaprasad]!
{quote}However there is change in --hostName behaviour in "host" mode. Earlier
the --hostName was being set to YARN defined hostname(generated from
container_id) which is incorrect in host network mode. In this patch, it is set
to the NM's host name which is the correct host name. Although docker seems to
be ignoring the --hostName set in "host" Mode in linux and promotes the
/etc/hosts file from the host, this is not the case in docker on Mac where it
is honoring the value passed in --hostName
{quote}
I had overlooked this comment previously. There was some discussion on which is
the appropriate hostname to use in YARN-7797. Ultimately, it's a trade off. We
had decided there was value in honoring the container_id based hostname, even
in "host" networking mode, for validating Registry DNS, but it seems that
decision may be incompatible with the need here. Given "host" networking mode
is really most appropriate for testing, I guess I don't have a problem with
changing the behavior to use the NM hostname. /cc [~billie.rinaldi]
> Expose container's hostname to applications running within the docker
> container
> ---
>
> Key: YARN-7935
> URL: https://issues.apache.org/jira/browse/YARN-7935
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
>Reporter: Suma Shivaprasad
>Assignee: Suma Shivaprasad
>Priority: Major
> Attachments: YARN-7935.1.patch, YARN-7935.2.patch, YARN-7935.3.patch
>
>
> Some applications have a need to bind to the container's hostname (like
> Spark) which is different from the NodeManager's hostname(NM_HOST which is
> available as an env during container launch) when launched through Docker
> runtime. The container's hostname can be exposed to applications via an env
> CONTAINER_HOSTNAME. Another potential candidate is the container's IP but
> this can be addressed in a separate jira.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7523) Introduce description and version field in Service record
[
https://issues.apache.org/jira/browse/YARN-7523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388226#comment-16388226
]
genericqa commented on YARN-7523:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 12 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
6s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 20s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 13s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications: The patch generated 2
new + 53 unchanged - 0 fixed = 55 total (was 53) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 36s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
31s{color} | {color:green} hadoop-yarn-services-core in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
21s{color} | {color:green} hadoop-yarn-services-api in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 46m 11s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f |
| JIRA Issue | YARN-7523 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12912841/YARN-7523.002.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 848f38b197cc 4.4.0-64-generic #85-Ubuntu SMP Mon Feb 20
11:50:30 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 7060725 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
[jira] [Commented] (YARN-7346) Add a profile to allow optional compilation for ATSv2 with HBase-2.0
[ https://issues.apache.org/jira/browse/YARN-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388220#comment-16388220 ] Vrushali C commented on YARN-7346: -- Hi [~haibochen] [~rohithsharma] Let's keep the same source code structure in branch-2 but we need not add support for compilation with hbase 2.x in branch-2. This will keep things consistent while generating patches. I am of the opinion we do not need to support hbase 2.x on branch-2. > Add a profile to allow optional compilation for ATSv2 with HBase-2.0 > > > Key: YARN-7346 > URL: https://issues.apache.org/jira/browse/YARN-7346 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Ted Yu >Assignee: Haibo Chen >Priority: Major > Fix For: 3.2.0 > > Attachments: YARN-7346.00.patch, YARN-7346.01.patch, > YARN-7346.02.patch, YARN-7346.03-incremental.patch, YARN-7346.03.patch, > YARN-7346.04-incremental.patch, YARN-7346.04.patch, YARN-7346.05.patch, > YARN-7346.06.patch, YARN-7346.07.patch, YARN-7346.08-incremental.patch, > YARN-7346.08.patch, YARN-7346.09.patch, YARN-7346.10.patch, > YARN-7346.11.patch, YARN-7346.prelim1.patch, YARN-7346.prelim2.patch, > YARN-7581.prelim.patch, > hadoop-yarn-server-timelineservice-hbase-server-1-findbugsXml.xml, > hadoop-yarn-server-timelineservice-hbase-server-1-javadoc-report.txt, > hadoop-yarn-server-timelineservice-hbase-server-2-findbugsXml.xml, > hadoop-yarn-server-timelineservice-hbase-server-2-javadoc-report.txt > > > When compiling hadoop-yarn-server-timelineservice-hbase against 2.0.0-alpha3, > I got the following errors: > [https://pastebin.com/Ms4jYEVB] > This issue is to fix the compilation errors. > The scope of the Jira is to add a profile to allow optional compilation for > ATSv2 with HBase2.0. The default compilation for trunk will still be for > hbase 1.2.6. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7999) Docker launch fails when user private filecache directory is missing
[
https://issues.apache.org/jira/browse/YARN-7999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388199#comment-16388199
]
genericqa commented on YARN-7999:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
38s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
26m 33s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 12s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 20m 43s{color}
| {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 60m 30s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.nodemanager.containermanager.TestContainerManager |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f |
| JIRA Issue | YARN-7999 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12913224/YARN-7999.002.patch |
| Optional Tests | asflicense compile cc mvnsite javac unit |
| uname | Linux 060089a5ba6a 4.4.0-116-generic #140-Ubuntu SMP Mon Feb 12
21:23:04 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / e6f99e2 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/19901/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/19901/testReport/ |
| Max. process+thread count | 461 (vs. ulimit of 1) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/19901/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Docker launch fails when user private filecache directory is missing
>
>
> Key: YARN-7999
> URL: https://issues.apache.org/jira/browse/YARN-7999
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: Eric Yang
>
[jira] [Commented] (YARN-7346) Add a profile to allow optional compilation for ATSv2 with HBase-2.0
[
https://issues.apache.org/jira/browse/YARN-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388182#comment-16388182
]
Sean Busbey commented on YARN-7346:
---
{quote}
What I was trying to say is that, In the context of ATSv2, NMs and RM act as
hbase clients that write data to HBase, so the hbase-client dependency code
will be running inside NM/RM processes. If hadoop 2 depends on, say dependency
X of version N and hbase-client 2.0 depends on X of version N+2 , then
hbase-client 2.0 code will be running with X of version N inside NM/RM. This
could potentially break if hbase-client 2.0 relies on something new that only
exists in version N+2.
{quote}
Oh for sure. dependency convergence is always a concern. maybe we should have a
jira to track figuring out how we make sure the hbase shaded client is usable
for Hadoop in this specific context.
> Add a profile to allow optional compilation for ATSv2 with HBase-2.0
>
>
> Key: YARN-7346
> URL: https://issues.apache.org/jira/browse/YARN-7346
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Ted Yu
>Assignee: Haibo Chen
>Priority: Major
> Fix For: 3.2.0
>
> Attachments: YARN-7346.00.patch, YARN-7346.01.patch,
> YARN-7346.02.patch, YARN-7346.03-incremental.patch, YARN-7346.03.patch,
> YARN-7346.04-incremental.patch, YARN-7346.04.patch, YARN-7346.05.patch,
> YARN-7346.06.patch, YARN-7346.07.patch, YARN-7346.08-incremental.patch,
> YARN-7346.08.patch, YARN-7346.09.patch, YARN-7346.10.patch,
> YARN-7346.11.patch, YARN-7346.prelim1.patch, YARN-7346.prelim2.patch,
> YARN-7581.prelim.patch,
> hadoop-yarn-server-timelineservice-hbase-server-1-findbugsXml.xml,
> hadoop-yarn-server-timelineservice-hbase-server-1-javadoc-report.txt,
> hadoop-yarn-server-timelineservice-hbase-server-2-findbugsXml.xml,
> hadoop-yarn-server-timelineservice-hbase-server-2-javadoc-report.txt
>
>
> When compiling hadoop-yarn-server-timelineservice-hbase against 2.0.0-alpha3,
> I got the following errors:
> [https://pastebin.com/Ms4jYEVB]
> This issue is to fix the compilation errors.
> The scope of the Jira is to add a profile to allow optional compilation for
> ATSv2 with HBase2.0. The default compilation for trunk will still be for
> hbase 1.2.6.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7346) Add a profile to allow optional compilation for ATSv2 with HBase-2.0
[ https://issues.apache.org/jira/browse/YARN-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388159#comment-16388159 ] Haibo Chen commented on YARN-7346: -- [~rohithsharma] Do you think it is worthwhile to have the same source code structure in branch-2 to ease any backport of fixes that also apply to hbase-1.2.6? If so, we can work on that in a separate jira. > Add a profile to allow optional compilation for ATSv2 with HBase-2.0 > > > Key: YARN-7346 > URL: https://issues.apache.org/jira/browse/YARN-7346 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Ted Yu >Assignee: Haibo Chen >Priority: Major > Fix For: 3.2.0 > > Attachments: YARN-7346.00.patch, YARN-7346.01.patch, > YARN-7346.02.patch, YARN-7346.03-incremental.patch, YARN-7346.03.patch, > YARN-7346.04-incremental.patch, YARN-7346.04.patch, YARN-7346.05.patch, > YARN-7346.06.patch, YARN-7346.07.patch, YARN-7346.08-incremental.patch, > YARN-7346.08.patch, YARN-7346.09.patch, YARN-7346.10.patch, > YARN-7346.11.patch, YARN-7346.prelim1.patch, YARN-7346.prelim2.patch, > YARN-7581.prelim.patch, > hadoop-yarn-server-timelineservice-hbase-server-1-findbugsXml.xml, > hadoop-yarn-server-timelineservice-hbase-server-1-javadoc-report.txt, > hadoop-yarn-server-timelineservice-hbase-server-2-findbugsXml.xml, > hadoop-yarn-server-timelineservice-hbase-server-2-javadoc-report.txt > > > When compiling hadoop-yarn-server-timelineservice-hbase against 2.0.0-alpha3, > I got the following errors: > [https://pastebin.com/Ms4jYEVB] > This issue is to fix the compilation errors. > The scope of the Jira is to add a profile to allow optional compilation for > ATSv2 with HBase2.0. The default compilation for trunk will still be for > hbase 1.2.6. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7346) Add a profile to allow optional compilation for ATSv2 with HBase-2.0
[ https://issues.apache.org/jira/browse/YARN-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388154#comment-16388154 ] Haibo Chen commented on YARN-7346: -- What I was trying to say is that, In the context of ATSv2, NMs and RM act as hbase clients that write data to HBase, so the hbase-client dependency code will be running inside NM/RM processes. If hadoop 2 depends on, say dependency X of version N and hbase-client 2.0 depends on X of version N+2 , then hbase-client 2.0 code will be running with X of version N inside NM/RM. This could potentially break if hbase-client 2.0 relies on something new that only exists in version N+2. This is independent of the HBase -Hadoop backing storage compatibility issue, which is covered by the HBase support matrix. Sorry for the confusion. > Add a profile to allow optional compilation for ATSv2 with HBase-2.0 > > > Key: YARN-7346 > URL: https://issues.apache.org/jira/browse/YARN-7346 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Ted Yu >Assignee: Haibo Chen >Priority: Major > Fix For: 3.2.0 > > Attachments: YARN-7346.00.patch, YARN-7346.01.patch, > YARN-7346.02.patch, YARN-7346.03-incremental.patch, YARN-7346.03.patch, > YARN-7346.04-incremental.patch, YARN-7346.04.patch, YARN-7346.05.patch, > YARN-7346.06.patch, YARN-7346.07.patch, YARN-7346.08-incremental.patch, > YARN-7346.08.patch, YARN-7346.09.patch, YARN-7346.10.patch, > YARN-7346.11.patch, YARN-7346.prelim1.patch, YARN-7346.prelim2.patch, > YARN-7581.prelim.patch, > hadoop-yarn-server-timelineservice-hbase-server-1-findbugsXml.xml, > hadoop-yarn-server-timelineservice-hbase-server-1-javadoc-report.txt, > hadoop-yarn-server-timelineservice-hbase-server-2-findbugsXml.xml, > hadoop-yarn-server-timelineservice-hbase-server-2-javadoc-report.txt > > > When compiling hadoop-yarn-server-timelineservice-hbase against 2.0.0-alpha3, > I got the following errors: > [https://pastebin.com/Ms4jYEVB] > This issue is to fix the compilation errors. > The scope of the Jira is to add a profile to allow optional compilation for > ATSv2 with HBase2.0. The default compilation for trunk will still be for > hbase 1.2.6. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7998) RM crashes with NPE during recovering if ACL configuration was changed
[
https://issues.apache.org/jira/browse/YARN-7998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388146#comment-16388146
]
genericqa commented on YARN-7998:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
52s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 31s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 35s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 5 new + 142 unchanged - 0 fixed = 147 total (was 142)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 28s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 73m 56s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}129m 13s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerSchedulingRequestUpdate
|
| |
hadoop.yarn.server.resourcemanager.scheduler.TestSchedulingWithAllocationRequestId
|
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestIncreaseAllocationExpirer
|
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing |
| | hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f |
| JIRA Issue | YARN-7998 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12913213/YARN-7998.003.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux ea664b44c8ba 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / e6f99e2 |
| maven | version: Apache Maven 3.3.9 |
| Default Java |
[jira] [Commented] (YARN-7523) Introduce description and version field in Service record
[ https://issues.apache.org/jira/browse/YARN-7523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388137#comment-16388137 ] Billie Rinaldi commented on YARN-7523: -- This change looks okay to me, but it seems like something went wrong with the precommit build. I reran the build and will also try testing the patch. > Introduce description and version field in Service record > - > > Key: YARN-7523 > URL: https://issues.apache.org/jira/browse/YARN-7523 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Gour Saha >Assignee: Chandni Singh >Priority: Major > Fix For: yarn-native-services > > Attachments: YARN-7523.001.patch, YARN-7523.002.patch > > > YARN-7512 would need version field in Service record. It would be good to > introduce a description field also to allow service owners to capture some > details which can be used to display in Service catalog as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8000) Yarn Service: component instance name shows up as component name in container record
[ https://issues.apache.org/jira/browse/YARN-8000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388125#comment-16388125 ] Billie Rinaldi commented on YARN-8000: -- There's an example in this file that uses component_name: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/yarn-service/YarnServiceAPI.md Could you verify that the UI isn't using the component_name field of the Container record? I did not see a place where it was using this, but it would be good to make sure the UI works properly after this change. Thanks, [~csingh]. I'll also test out the patch. > Yarn Service: component instance name shows up as component name in container > record > - > > Key: YARN-8000 > URL: https://issues.apache.org/jira/browse/YARN-8000 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Chandni Singh >Assignee: Chandni Singh >Priority: Major > Attachments: YARN-8000.001.patch, YARN-8000.002.patch, > YARN-8000.003.patch > > > Yarn Service: component instance name shows up as component name in container > record -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7652) Handle AM register requests asynchronously in FederationInterceptor
[
https://issues.apache.org/jira/browse/YARN-7652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388086#comment-16388086
]
Botong Huang commented on YARN-7652:
Hi [~Sammi], yes please push it to next release. Thanks!
> Handle AM register requests asynchronously in FederationInterceptor
> ---
>
> Key: YARN-7652
> URL: https://issues.apache.org/jira/browse/YARN-7652
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: amrmproxy, federation
>Affects Versions: 2.9.0, 3.0.0
>Reporter: Subru Krishnan
>Assignee: Botong Huang
>Priority: Major
>
> We (cc [~goiri]/[~botong]) observed that the {{FederationInterceptor}} in
> {{AMRMProxy}} (and consequently the AM) is blocked if the _StateStore_ has
> outdated info about a _SubCluster_. This is because we handle AM register
> requests synchronously. This jira proposes to move to async similar to how we
> operate with allocate invocations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7346) Add a profile to allow optional compilation for ATSv2 with HBase-2.0
[ https://issues.apache.org/jira/browse/YARN-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388083#comment-16388083 ] Sean Busbey commented on YARN-7346: --- [the hbase community support matrix on JDK versions also lists Java 8 as required for HBase 2.0|http://hbase.apache.org/book.html#java] > Add a profile to allow optional compilation for ATSv2 with HBase-2.0 > > > Key: YARN-7346 > URL: https://issues.apache.org/jira/browse/YARN-7346 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Ted Yu >Assignee: Haibo Chen >Priority: Major > Fix For: 3.2.0 > > Attachments: YARN-7346.00.patch, YARN-7346.01.patch, > YARN-7346.02.patch, YARN-7346.03-incremental.patch, YARN-7346.03.patch, > YARN-7346.04-incremental.patch, YARN-7346.04.patch, YARN-7346.05.patch, > YARN-7346.06.patch, YARN-7346.07.patch, YARN-7346.08-incremental.patch, > YARN-7346.08.patch, YARN-7346.09.patch, YARN-7346.10.patch, > YARN-7346.11.patch, YARN-7346.prelim1.patch, YARN-7346.prelim2.patch, > YARN-7581.prelim.patch, > hadoop-yarn-server-timelineservice-hbase-server-1-findbugsXml.xml, > hadoop-yarn-server-timelineservice-hbase-server-1-javadoc-report.txt, > hadoop-yarn-server-timelineservice-hbase-server-2-findbugsXml.xml, > hadoop-yarn-server-timelineservice-hbase-server-2-javadoc-report.txt > > > When compiling hadoop-yarn-server-timelineservice-hbase against 2.0.0-alpha3, > I got the following errors: > [https://pastebin.com/Ms4jYEVB] > This issue is to fix the compilation errors. > The scope of the Jira is to add a profile to allow optional compilation for > ATSv2 with HBase2.0. The default compilation for trunk will still be for > hbase 1.2.6. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7346) Add a profile to allow optional compilation for ATSv2 with HBase-2.0
[
https://issues.apache.org/jira/browse/YARN-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388071#comment-16388071
]
Rohith Sharma K S commented on YARN-7346:
-
This would become a risk of back porting into branch-2 since HBase-2 claimed to
compile on java8 only. I do see in hbase-env.sh as a note that 1.8+ required.
{code}
# The java implementation to use. Java 1.8+ required.
# export JAVA_HOME=/usr/java/jdk1.8.0/
{code}
Lets drop the idea of supporting HBase-2 in branch-2. cc :/ [~vrushalic]
> Add a profile to allow optional compilation for ATSv2 with HBase-2.0
>
>
> Key: YARN-7346
> URL: https://issues.apache.org/jira/browse/YARN-7346
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Ted Yu
>Assignee: Haibo Chen
>Priority: Major
> Fix For: 3.2.0
>
> Attachments: YARN-7346.00.patch, YARN-7346.01.patch,
> YARN-7346.02.patch, YARN-7346.03-incremental.patch, YARN-7346.03.patch,
> YARN-7346.04-incremental.patch, YARN-7346.04.patch, YARN-7346.05.patch,
> YARN-7346.06.patch, YARN-7346.07.patch, YARN-7346.08-incremental.patch,
> YARN-7346.08.patch, YARN-7346.09.patch, YARN-7346.10.patch,
> YARN-7346.11.patch, YARN-7346.prelim1.patch, YARN-7346.prelim2.patch,
> YARN-7581.prelim.patch,
> hadoop-yarn-server-timelineservice-hbase-server-1-findbugsXml.xml,
> hadoop-yarn-server-timelineservice-hbase-server-1-javadoc-report.txt,
> hadoop-yarn-server-timelineservice-hbase-server-2-findbugsXml.xml,
> hadoop-yarn-server-timelineservice-hbase-server-2-javadoc-report.txt
>
>
> When compiling hadoop-yarn-server-timelineservice-hbase against 2.0.0-alpha3,
> I got the following errors:
> [https://pastebin.com/Ms4jYEVB]
> This issue is to fix the compilation errors.
> The scope of the Jira is to add a profile to allow optional compilation for
> ATSv2 with HBase2.0. The default compilation for trunk will still be for
> hbase 1.2.6.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7346) Add a profile to allow optional compilation for ATSv2 with HBase-2.0
[
https://issues.apache.org/jira/browse/YARN-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388060#comment-16388060
]
Sean Busbey commented on YARN-7346:
---
{quote}
I think it is worth mentioning that the 'support' we are talking about in this
context refers to running HBase 2.0 client with hadoop2 jars/dependencies
inside NM/RM. Sean Busbey Does the hadoop support matrix apply to that specific
context?
{quote}
I don't understand the question, could you rephrase? that 'support' matrix is
just about what the Apache HBase community recommends wrt versions of Hadoop to
use as a backing storage layer for running an HBase instance. It impacts what
they'll answer questions about without pushing back on.
> Add a profile to allow optional compilation for ATSv2 with HBase-2.0
>
>
> Key: YARN-7346
> URL: https://issues.apache.org/jira/browse/YARN-7346
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Ted Yu
>Assignee: Haibo Chen
>Priority: Major
> Fix For: 3.2.0
>
> Attachments: YARN-7346.00.patch, YARN-7346.01.patch,
> YARN-7346.02.patch, YARN-7346.03-incremental.patch, YARN-7346.03.patch,
> YARN-7346.04-incremental.patch, YARN-7346.04.patch, YARN-7346.05.patch,
> YARN-7346.06.patch, YARN-7346.07.patch, YARN-7346.08-incremental.patch,
> YARN-7346.08.patch, YARN-7346.09.patch, YARN-7346.10.patch,
> YARN-7346.11.patch, YARN-7346.prelim1.patch, YARN-7346.prelim2.patch,
> YARN-7581.prelim.patch,
> hadoop-yarn-server-timelineservice-hbase-server-1-findbugsXml.xml,
> hadoop-yarn-server-timelineservice-hbase-server-1-javadoc-report.txt,
> hadoop-yarn-server-timelineservice-hbase-server-2-findbugsXml.xml,
> hadoop-yarn-server-timelineservice-hbase-server-2-javadoc-report.txt
>
>
> When compiling hadoop-yarn-server-timelineservice-hbase against 2.0.0-alpha3,
> I got the following errors:
> [https://pastebin.com/Ms4jYEVB]
> This issue is to fix the compilation errors.
> The scope of the Jira is to add a profile to allow optional compilation for
> ATSv2 with HBase2.0. The default compilation for trunk will still be for
> hbase 1.2.6.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7999) Docker launch fails when user private filecache directory is missing
[
https://issues.apache.org/jira/browse/YARN-7999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388054#comment-16388054
]
Jason Lowe commented on YARN-7999:
--
Thanks for the review, Shane! I updated the patch to fix the warnings and also
fix some issues with handling frees around the file cache directory name.
> Docker launch fails when user private filecache directory is missing
>
>
> Key: YARN-7999
> URL: https://issues.apache.org/jira/browse/YARN-7999
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: Eric Yang
>Assignee: Jason Lowe
>Priority: Major
> Attachments: YARN-7999.001.patch, YARN-7999.002.patch
>
>
> Docker container is failing to launch in trunk. The root cause is:
> {code}
> [COMPINSTANCE sleeper-1 : container_1520032931921_0001_01_20]:
> [2018-03-02 23:26:09.196]Exception from container-launch.
> Container id: container_1520032931921_0001_01_20
> Exit code: 29
> Exception message: image: hadoop/centos:latest is trusted in hadoop registry.
> Could not determine real path of mount
> '/tmp/hadoop-yarn/nm-local-dir/usercache/hbase/filecache'
> Could not determine real path of mount
> '/tmp/hadoop-yarn/nm-local-dir/usercache/hbase/filecache'
> Invalid docker mount
> '/tmp/hadoop-yarn/nm-local-dir/usercache/hbase/filecache:/tmp/hadoop-yarn/nm-local-dir/usercache/hbase/filecache',
> realpath=/tmp/hadoop-yarn/nm-local-dir/usercache/hbase/filecache
> Error constructing docker command, docker error code=12, error
> message='Invalid docker mount'
> Shell output: main : command provided 4
> main : run as user is hbase
> main : requested yarn user is hbase
> Creating script paths...
> Creating local dirs...
> [2018-03-02 23:26:09.240]Diagnostic message from attempt 0 : [2018-03-02
> 23:26:09.240]
> [2018-03-02 23:26:09.240]Container exited with a non-zero exit code 29.
> [2018-03-02 23:26:39.278]Could not find
> nmPrivate/application_1520032931921_0001/container_1520032931921_0001_01_20//container_1520032931921_0001_01_20.pid
> in any of the directories
> [COMPONENT sleeper]: Failed 11 times, exceeded the limit - 10. Shutting down
> now...
> {code}
> The filecache cant not be mounted because it doesn't exist.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7999) Docker launch fails when user private filecache directory is missing
[
https://issues.apache.org/jira/browse/YARN-7999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Lowe updated YARN-7999:
-
Attachment: YARN-7999.002.patch
> Docker launch fails when user private filecache directory is missing
>
>
> Key: YARN-7999
> URL: https://issues.apache.org/jira/browse/YARN-7999
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: Eric Yang
>Assignee: Jason Lowe
>Priority: Major
> Attachments: YARN-7999.001.patch, YARN-7999.002.patch
>
>
> Docker container is failing to launch in trunk. The root cause is:
> {code}
> [COMPINSTANCE sleeper-1 : container_1520032931921_0001_01_20]:
> [2018-03-02 23:26:09.196]Exception from container-launch.
> Container id: container_1520032931921_0001_01_20
> Exit code: 29
> Exception message: image: hadoop/centos:latest is trusted in hadoop registry.
> Could not determine real path of mount
> '/tmp/hadoop-yarn/nm-local-dir/usercache/hbase/filecache'
> Could not determine real path of mount
> '/tmp/hadoop-yarn/nm-local-dir/usercache/hbase/filecache'
> Invalid docker mount
> '/tmp/hadoop-yarn/nm-local-dir/usercache/hbase/filecache:/tmp/hadoop-yarn/nm-local-dir/usercache/hbase/filecache',
> realpath=/tmp/hadoop-yarn/nm-local-dir/usercache/hbase/filecache
> Error constructing docker command, docker error code=12, error
> message='Invalid docker mount'
> Shell output: main : command provided 4
> main : run as user is hbase
> main : requested yarn user is hbase
> Creating script paths...
> Creating local dirs...
> [2018-03-02 23:26:09.240]Diagnostic message from attempt 0 : [2018-03-02
> 23:26:09.240]
> [2018-03-02 23:26:09.240]Container exited with a non-zero exit code 29.
> [2018-03-02 23:26:39.278]Could not find
> nmPrivate/application_1520032931921_0001/container_1520032931921_0001_01_20//container_1520032931921_0001_01_20.pid
> in any of the directories
> [COMPONENT sleeper]: Failed 11 times, exceeded the limit - 10. Shutting down
> now...
> {code}
> The filecache cant not be mounted because it doesn't exist.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7346) Add a profile to allow optional compilation for ATSv2 with HBase-2.0
[
https://issues.apache.org/jira/browse/YARN-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388027#comment-16388027
]
Haibo Chen commented on YARN-7346:
--
[~rohithsharma] I am getting compilation errors due to java version.
{code:java}
[ERROR]
/Users/haibochen/Documents/Apache/hadoop/ats/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-server/hadoop-yarn-server-timelineservice-hbase-server-2/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/common/HBaseTimelineServerUtils.java:[155,35]
static interface method invocations are not supported in -source 1.7
[ERROR] (use -source 8 or higher to enable static interface method invocations)
[ERROR]
/Users/haibochen/Documents/Apache/hadoop/ats/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-server/hadoop-yarn-server-timelineservice-hbase-server-2/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/flow/FlowRunCoprocessor.java:[75,23]
incompatible types:
java.util.Optional
cannot be converted to
java.util.Optional{code}
HBase 2 needs java 8, but we only do java 7 in branch-2.
> Add a profile to allow optional compilation for ATSv2 with HBase-2.0
>
>
> Key: YARN-7346
> URL: https://issues.apache.org/jira/browse/YARN-7346
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Ted Yu
>Assignee: Haibo Chen
>Priority: Major
> Fix For: 3.2.0
>
> Attachments: YARN-7346.00.patch, YARN-7346.01.patch,
> YARN-7346.02.patch, YARN-7346.03-incremental.patch, YARN-7346.03.patch,
> YARN-7346.04-incremental.patch, YARN-7346.04.patch, YARN-7346.05.patch,
> YARN-7346.06.patch, YARN-7346.07.patch, YARN-7346.08-incremental.patch,
> YARN-7346.08.patch, YARN-7346.09.patch, YARN-7346.10.patch,
> YARN-7346.11.patch, YARN-7346.prelim1.patch, YARN-7346.prelim2.patch,
> YARN-7581.prelim.patch,
> hadoop-yarn-server-timelineservice-hbase-server-1-findbugsXml.xml,
> hadoop-yarn-server-timelineservice-hbase-server-1-javadoc-report.txt,
> hadoop-yarn-server-timelineservice-hbase-server-2-findbugsXml.xml,
> hadoop-yarn-server-timelineservice-hbase-server-2-javadoc-report.txt
>
>
> When compiling hadoop-yarn-server-timelineservice-hbase against 2.0.0-alpha3,
> I got the following errors:
> [https://pastebin.com/Ms4jYEVB]
> This issue is to fix the compilation errors.
> The scope of the Jira is to add a profile to allow optional compilation for
> ATSv2 with HBase2.0. The default compilation for trunk will still be for
> hbase 1.2.6.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7346) Add a profile to allow optional compilation for ATSv2 with HBase-2.0
[ https://issues.apache.org/jira/browse/YARN-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388019#comment-16388019 ] Haibo Chen commented on YARN-7346: -- I think it is worth mentioning that the 'support' we are talking about in this context refers to running HBase 2.0 client with hadoop2 jars/dependencies inside NM/RM. [~busbey] Does the hadoop support matrix apply to that specific context? > Add a profile to allow optional compilation for ATSv2 with HBase-2.0 > > > Key: YARN-7346 > URL: https://issues.apache.org/jira/browse/YARN-7346 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Ted Yu >Assignee: Haibo Chen >Priority: Major > Fix For: 3.2.0 > > Attachments: YARN-7346.00.patch, YARN-7346.01.patch, > YARN-7346.02.patch, YARN-7346.03-incremental.patch, YARN-7346.03.patch, > YARN-7346.04-incremental.patch, YARN-7346.04.patch, YARN-7346.05.patch, > YARN-7346.06.patch, YARN-7346.07.patch, YARN-7346.08-incremental.patch, > YARN-7346.08.patch, YARN-7346.09.patch, YARN-7346.10.patch, > YARN-7346.11.patch, YARN-7346.prelim1.patch, YARN-7346.prelim2.patch, > YARN-7581.prelim.patch, > hadoop-yarn-server-timelineservice-hbase-server-1-findbugsXml.xml, > hadoop-yarn-server-timelineservice-hbase-server-1-javadoc-report.txt, > hadoop-yarn-server-timelineservice-hbase-server-2-findbugsXml.xml, > hadoop-yarn-server-timelineservice-hbase-server-2-javadoc-report.txt > > > When compiling hadoop-yarn-server-timelineservice-hbase against 2.0.0-alpha3, > I got the following errors: > [https://pastebin.com/Ms4jYEVB] > This issue is to fix the compilation errors. > The scope of the Jira is to add a profile to allow optional compilation for > ATSv2 with HBase2.0. The default compilation for trunk will still be for > hbase 1.2.6. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7346) Add a profile to allow optional compilation for ATSv2 with HBase-2.0
[ https://issues.apache.org/jira/browse/YARN-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387996#comment-16387996 ] Haibo Chen commented on YARN-7346: -- Thanks [~busbey] for the info. I'll proceed to work on cherry-picking the patch to branch-2. > Add a profile to allow optional compilation for ATSv2 with HBase-2.0 > > > Key: YARN-7346 > URL: https://issues.apache.org/jira/browse/YARN-7346 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Ted Yu >Assignee: Haibo Chen >Priority: Major > Fix For: 3.2.0 > > Attachments: YARN-7346.00.patch, YARN-7346.01.patch, > YARN-7346.02.patch, YARN-7346.03-incremental.patch, YARN-7346.03.patch, > YARN-7346.04-incremental.patch, YARN-7346.04.patch, YARN-7346.05.patch, > YARN-7346.06.patch, YARN-7346.07.patch, YARN-7346.08-incremental.patch, > YARN-7346.08.patch, YARN-7346.09.patch, YARN-7346.10.patch, > YARN-7346.11.patch, YARN-7346.prelim1.patch, YARN-7346.prelim2.patch, > YARN-7581.prelim.patch, > hadoop-yarn-server-timelineservice-hbase-server-1-findbugsXml.xml, > hadoop-yarn-server-timelineservice-hbase-server-1-javadoc-report.txt, > hadoop-yarn-server-timelineservice-hbase-server-2-findbugsXml.xml, > hadoop-yarn-server-timelineservice-hbase-server-2-javadoc-report.txt > > > When compiling hadoop-yarn-server-timelineservice-hbase against 2.0.0-alpha3, > I got the following errors: > [https://pastebin.com/Ms4jYEVB] > This issue is to fix the compilation errors. > The scope of the Jira is to add a profile to allow optional compilation for > ATSv2 with HBase2.0. The default compilation for trunk will still be for > hbase 1.2.6. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Issue Comment Deleted] (YARN-7346) Add a profile to allow optional compilation for ATSv2 with HBase-2.0
[
https://issues.apache.org/jira/browse/YARN-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Haibo Chen updated YARN-7346:
-
Comment: was deleted
(was: {quote}Does HBase-2.0 officially claims their default Hadoop version is
3.0.0?
{quote}
[~yuzhih...@gmail.com], [~mdrob] Is this the case? Specifically, what we will
be effectively doing in branch-2 is running HBase 2.0 client with hadoop2 jars.)
> Add a profile to allow optional compilation for ATSv2 with HBase-2.0
>
>
> Key: YARN-7346
> URL: https://issues.apache.org/jira/browse/YARN-7346
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Ted Yu
>Assignee: Haibo Chen
>Priority: Major
> Fix For: 3.2.0
>
> Attachments: YARN-7346.00.patch, YARN-7346.01.patch,
> YARN-7346.02.patch, YARN-7346.03-incremental.patch, YARN-7346.03.patch,
> YARN-7346.04-incremental.patch, YARN-7346.04.patch, YARN-7346.05.patch,
> YARN-7346.06.patch, YARN-7346.07.patch, YARN-7346.08-incremental.patch,
> YARN-7346.08.patch, YARN-7346.09.patch, YARN-7346.10.patch,
> YARN-7346.11.patch, YARN-7346.prelim1.patch, YARN-7346.prelim2.patch,
> YARN-7581.prelim.patch,
> hadoop-yarn-server-timelineservice-hbase-server-1-findbugsXml.xml,
> hadoop-yarn-server-timelineservice-hbase-server-1-javadoc-report.txt,
> hadoop-yarn-server-timelineservice-hbase-server-2-findbugsXml.xml,
> hadoop-yarn-server-timelineservice-hbase-server-2-javadoc-report.txt
>
>
> When compiling hadoop-yarn-server-timelineservice-hbase against 2.0.0-alpha3,
> I got the following errors:
> [https://pastebin.com/Ms4jYEVB]
> This issue is to fix the compilation errors.
> The scope of the Jira is to add a profile to allow optional compilation for
> ATSv2 with HBase2.0. The default compilation for trunk will still be for
> hbase 1.2.6.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7346) Add a profile to allow optional compilation for ATSv2 with HBase-2.0
[
https://issues.apache.org/jira/browse/YARN-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387988#comment-16387988
]
Haibo Chen commented on YARN-7346:
--
{quote}Does HBase-2.0 officially claims their default Hadoop version is 3.0.0?
{quote}
[~yuzhih...@gmail.com], [~mdrob] Is this the case? Specifically, what we will
be effectively doing in branch-2 is running HBase 2.0 client with hadoop2 jars.
> Add a profile to allow optional compilation for ATSv2 with HBase-2.0
>
>
> Key: YARN-7346
> URL: https://issues.apache.org/jira/browse/YARN-7346
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Ted Yu
>Assignee: Haibo Chen
>Priority: Major
> Fix For: 3.2.0
>
> Attachments: YARN-7346.00.patch, YARN-7346.01.patch,
> YARN-7346.02.patch, YARN-7346.03-incremental.patch, YARN-7346.03.patch,
> YARN-7346.04-incremental.patch, YARN-7346.04.patch, YARN-7346.05.patch,
> YARN-7346.06.patch, YARN-7346.07.patch, YARN-7346.08-incremental.patch,
> YARN-7346.08.patch, YARN-7346.09.patch, YARN-7346.10.patch,
> YARN-7346.11.patch, YARN-7346.prelim1.patch, YARN-7346.prelim2.patch,
> YARN-7581.prelim.patch,
> hadoop-yarn-server-timelineservice-hbase-server-1-findbugsXml.xml,
> hadoop-yarn-server-timelineservice-hbase-server-1-javadoc-report.txt,
> hadoop-yarn-server-timelineservice-hbase-server-2-findbugsXml.xml,
> hadoop-yarn-server-timelineservice-hbase-server-2-javadoc-report.txt
>
>
> When compiling hadoop-yarn-server-timelineservice-hbase against 2.0.0-alpha3,
> I got the following errors:
> [https://pastebin.com/Ms4jYEVB]
> This issue is to fix the compilation errors.
> The scope of the Jira is to add a profile to allow optional compilation for
> ATSv2 with HBase2.0. The default compilation for trunk will still be for
> hbase 1.2.6.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7346) Add a profile to allow optional compilation for ATSv2 with HBase-2.0
[ https://issues.apache.org/jira/browse/YARN-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387981#comment-16387981 ] Sean Busbey commented on YARN-7346: --- HBase 2.0 does not default to Hadoop 3. [our hadoop support matrix|http://hbase.apache.org/book.html#basic.prerequisites] as of today still lists the Hadoop 3 line as "Not Tested (NT)" which is how the community labels things that are experimental. > Add a profile to allow optional compilation for ATSv2 with HBase-2.0 > > > Key: YARN-7346 > URL: https://issues.apache.org/jira/browse/YARN-7346 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Ted Yu >Assignee: Haibo Chen >Priority: Major > Fix For: 3.2.0 > > Attachments: YARN-7346.00.patch, YARN-7346.01.patch, > YARN-7346.02.patch, YARN-7346.03-incremental.patch, YARN-7346.03.patch, > YARN-7346.04-incremental.patch, YARN-7346.04.patch, YARN-7346.05.patch, > YARN-7346.06.patch, YARN-7346.07.patch, YARN-7346.08-incremental.patch, > YARN-7346.08.patch, YARN-7346.09.patch, YARN-7346.10.patch, > YARN-7346.11.patch, YARN-7346.prelim1.patch, YARN-7346.prelim2.patch, > YARN-7581.prelim.patch, > hadoop-yarn-server-timelineservice-hbase-server-1-findbugsXml.xml, > hadoop-yarn-server-timelineservice-hbase-server-1-javadoc-report.txt, > hadoop-yarn-server-timelineservice-hbase-server-2-findbugsXml.xml, > hadoop-yarn-server-timelineservice-hbase-server-2-javadoc-report.txt > > > When compiling hadoop-yarn-server-timelineservice-hbase against 2.0.0-alpha3, > I got the following errors: > [https://pastebin.com/Ms4jYEVB] > This issue is to fix the compilation errors. > The scope of the Jira is to add a profile to allow optional compilation for > ATSv2 with HBase2.0. The default compilation for trunk will still be for > hbase 1.2.6. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8002) Support NOT_SELF and ALL namespace types for allocation tag
[
https://issues.apache.org/jira/browse/YARN-8002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387967#comment-16387967
]
genericqa commented on YARN-8002:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
54s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 26s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
34s{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api in
trunk has 1 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
18s{color} | {color:green} hadoop-yarn-project/hadoop-yarn: The patch generated
0 new + 47 unchanged - 1 fixed = 47 total (was 48) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 33s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
41s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 68m
15s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
40s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}146m 29s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f |
| JIRA Issue | YARN-8002 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12913196/YARN-8002.002.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 74ae031864be 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18
11:55:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 12ecb55 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC1 |
|
[jira] [Comment Edited] (YARN-7999) Docker launch fails when user private filecache directory is missing
[
https://issues.apache.org/jira/browse/YARN-7999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387937#comment-16387937
]
Shane Kumpf edited comment on YARN-7999 at 3/6/18 3:34 PM:
---
Thanks for the patch [~jlowe]! I still haven't been able to recreate this issue
yet, so I think [~eyang] will need to validate if this fixes the issue for him.
Regarding a different approach, my first thought was to check for the existence
of source directory in {{DockerLinuxContainerRuntime}} prior to adding the bind
mount to the docker run command. If the directory doesn't exist, does it even
make sense to request that bind mount?
However, it seems normalize_mounts is also called on each of the paths for the
ro-mounts and rw-mounts in {{container-executor.cfg}}, so if the
usercache/_user_/filecache is listed as a read-only or read-write mount, we'd
still run into the error. The current documentation recommends configuring the
ro-mounts and rw-mounts in {{container-executor.cfg}} to be the
{{nm-local-dir}} root, versus the individual directories under the
{{nm-local-dirs}}. Configuring {{container-executor.cfg}} in this manner would
avoid this issue. IMO, attempting to enumerate every user's
usercache/_user_/filecache directory in {{container-executor.cfg}} sounds like
an administrative nightmare, but I guess at the same time we should allow that
scenario if someone wants to do it.
Given this, I think your approach to ensuring the directory exists prior to
launch is the way to go to address this edge case.
Regarding review, the cc warning seems valid.
was (Author: shaneku...@gmail.com):
Thanks for the patch [~jlowe]! I still haven't been able to recreate this issue
yet, so I think [~eyang] will need to validate if this fixes the issue for him.
Regarding a different approach, my first thought was to check for the existence
of source directory in {{DockerLinuxContainerRuntime}} prior to adding the bind
mount to the docker run command. If the directory doesn't exist, does it even
make sense to request that bind mount?
However, it seems normalize_mounts is also called on each of the paths for the
ro-mounts and rw-mounts in {{container-executor.cfg}}, so if the
usercache/_user_/filecache is listed as a read-only or read-write mount, we'd
still run into the error. The current documentation recommends configuring the
ro-mounts and rw-mounts in {{container-executor.cfg}} to be the
{{nm-local-dir}} root, versus the individual directories under the
{{nm-local-dirs}}. Configuring {{container-executor.cfg}} in this manner would
avoid this issue. IMO, attempting to enumerate every user's
usercache/_user_/filecache directory in {{container-executor.cfg}} sounds like
an administrative nightmare, but I guess at the same time we should allow that
scenario if someone wants to do it.
Given this, I think your approach to ensuring the directory exists prior to
launch is the way to go to address this edge case.
Regarding review, the cc warning seems valid and there is a missing free for
the filecache_dir in the case where the all of the filecache_dir's can be
determined and the mkdirs succeed.
> Docker launch fails when user private filecache directory is missing
>
>
> Key: YARN-7999
> URL: https://issues.apache.org/jira/browse/YARN-7999
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: Eric Yang
>Assignee: Jason Lowe
>Priority: Major
> Attachments: YARN-7999.001.patch
>
>
> Docker container is failing to launch in trunk. The root cause is:
> {code}
> [COMPINSTANCE sleeper-1 : container_1520032931921_0001_01_20]:
> [2018-03-02 23:26:09.196]Exception from container-launch.
> Container id: container_1520032931921_0001_01_20
> Exit code: 29
> Exception message: image: hadoop/centos:latest is trusted in hadoop registry.
> Could not determine real path of mount
> '/tmp/hadoop-yarn/nm-local-dir/usercache/hbase/filecache'
> Could not determine real path of mount
> '/tmp/hadoop-yarn/nm-local-dir/usercache/hbase/filecache'
> Invalid docker mount
> '/tmp/hadoop-yarn/nm-local-dir/usercache/hbase/filecache:/tmp/hadoop-yarn/nm-local-dir/usercache/hbase/filecache',
> realpath=/tmp/hadoop-yarn/nm-local-dir/usercache/hbase/filecache
> Error constructing docker command, docker error code=12, error
> message='Invalid docker mount'
> Shell output: main : command provided 4
> main : run as user is hbase
> main : requested yarn user is hbase
> Creating script paths...
> Creating local dirs...
> [2018-03-02 23:26:09.240]Diagnostic message from attempt 0 : [2018-03-02
> 23:26:09.240]
> [2018-03-02 23:26:09.240]Container exited with a non-zero exit code 29.
> [2018-03-02 23:26:39.278]Could not find
>
[jira] [Commented] (YARN-7346) Add a profile to allow optional compilation for ATSv2 with HBase-2.0
[ https://issues.apache.org/jira/browse/YARN-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387957#comment-16387957 ] Rohith Sharma K S commented on YARN-7346: - This is really a good point to be discussed. Does HBase-2.0 officially claims their default Hadoop version is 3.0.0? I just saw looking HBase-2.0-beta release and found that hadoop-2.7.3 jars in HBase/lib folder. > Add a profile to allow optional compilation for ATSv2 with HBase-2.0 > > > Key: YARN-7346 > URL: https://issues.apache.org/jira/browse/YARN-7346 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Ted Yu >Assignee: Haibo Chen >Priority: Major > Fix For: 3.2.0 > > Attachments: YARN-7346.00.patch, YARN-7346.01.patch, > YARN-7346.02.patch, YARN-7346.03-incremental.patch, YARN-7346.03.patch, > YARN-7346.04-incremental.patch, YARN-7346.04.patch, YARN-7346.05.patch, > YARN-7346.06.patch, YARN-7346.07.patch, YARN-7346.08-incremental.patch, > YARN-7346.08.patch, YARN-7346.09.patch, YARN-7346.10.patch, > YARN-7346.11.patch, YARN-7346.prelim1.patch, YARN-7346.prelim2.patch, > YARN-7581.prelim.patch, > hadoop-yarn-server-timelineservice-hbase-server-1-findbugsXml.xml, > hadoop-yarn-server-timelineservice-hbase-server-1-javadoc-report.txt, > hadoop-yarn-server-timelineservice-hbase-server-2-findbugsXml.xml, > hadoop-yarn-server-timelineservice-hbase-server-2-javadoc-report.txt > > > When compiling hadoop-yarn-server-timelineservice-hbase against 2.0.0-alpha3, > I got the following errors: > [https://pastebin.com/Ms4jYEVB] > This issue is to fix the compilation errors. > The scope of the Jira is to add a profile to allow optional compilation for > ATSv2 with HBase2.0. The default compilation for trunk will still be for > hbase 1.2.6. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-2442) ResourceManager JMX UI does not give HA State
[
https://issues.apache.org/jira/browse/YARN-2442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387954#comment-16387954
]
genericqa commented on YARN-2442:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
27s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 15s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 27s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 5 new + 36 unchanged - 0 fixed = 41 total (was 36) {color}
|
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 7s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 67m 39s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}111m 25s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.metrics.TestSystemMetricsPublisher |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f |
| JIRA Issue | YARN-2442 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12913203/YARN-2442.02.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 89d4d11c352a 4.4.0-64-generic #85-Ubuntu SMP Mon Feb 20
11:50:30 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 12ecb55 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/19899/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| unit |
[jira] [Commented] (YARN-7935) Expose container's hostname to applications running within the docker container
[
https://issues.apache.org/jira/browse/YARN-7935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387944#comment-16387944
]
Jason Lowe commented on YARN-7935:
--
Why is this new environment variable only being set in the Docker case? The
application being launched may unconditionally be relying on this YARN
environment variable, and it won't be there in the non-Docker case. All of the
other YARN-provided environment variables are always set regardless of
container type, and I don't see why this should be an exception.
Catching NullPointerException is often a code smell, especially when the code
being wrapped by it is itself checking for nulls. I'd much rather this code
handle what it is expecting to potentially be null rather than catch NPE. This
smells like accommodating poor unit test mocking, and typically I'd much rather
see the mocks be updated than workarounds in the real code, especially if that
workaround is catching NPE.
{code:java}
private String getDockerNetworkType(Container container) {
String network;
try {
network = container.getLaunchContext().getEnvironment().get(
"YARN_CONTAINER_RUNTIME_DOCKER_CONTAINER_NETWORK");
if (network == null || network.isEmpty()) {
network = defaultNetwork;
}
} catch (NullPointerException e) {
network = defaultNetwork;
}
return network;
}
{code}
> Expose container's hostname to applications running within the docker
> container
> ---
>
> Key: YARN-7935
> URL: https://issues.apache.org/jira/browse/YARN-7935
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
>Reporter: Suma Shivaprasad
>Assignee: Suma Shivaprasad
>Priority: Major
> Attachments: YARN-7935.1.patch, YARN-7935.2.patch, YARN-7935.3.patch
>
>
> Some applications have a need to bind to the container's hostname (like
> Spark) which is different from the NodeManager's hostname(NM_HOST which is
> available as an env during container launch) when launched through Docker
> runtime. The container's hostname can be exposed to applications via an env
> CONTAINER_HOSTNAME. Another potential candidate is the container's IP but
> this can be addressed in a separate jira.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7999) Docker launch fails when user private filecache directory is missing
[
https://issues.apache.org/jira/browse/YARN-7999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387937#comment-16387937
]
Shane Kumpf commented on YARN-7999:
---
Thanks for the patch [~jlowe]! I still haven't been able to recreate this issue
yet, so I think [~eyang] will need to validate if this fixes the issue for him.
Regarding a different approach, my first thought was to check for the existence
of source directory in {{DockerLinuxContainerRuntime}} prior to adding the bind
mount to the docker run command. If the directory doesn't exist, does it even
make sense to request that bind mount?
However, it seems normalize_mounts is also called on each of the paths for the
ro-mounts and rw-mounts in {{container-executor.cfg}}, so if the
usercache/_user_/filecache is listed as a read-only or read-write mount, we'd
still run into the error. The current documentation recommends configuring the
ro-mounts and rw-mounts in {{container-executor.cfg}} to be the
{{nm-local-dir}} root, versus the individual directories under the
{{nm-local-dirs}}. Configuring {{container-executor.cfg}} in this manner would
avoid this issue. IMO, attempting to enumerate every user's
usercache/_user_/filecache directory in {{container-executor.cfg}} sounds like
an administrative nightmare, but I guess at the same time we should allow that
scenario if someone wants to do it.
Given this, I think your approach to ensuring the directory exists prior to
launch is the way to go to address this edge case.
Regarding review, the cc warning seems valid and there is a missing free for
the filecache_dir in the case where the all of the filecache_dir's can be
determined and the mkdirs succeed.
> Docker launch fails when user private filecache directory is missing
>
>
> Key: YARN-7999
> URL: https://issues.apache.org/jira/browse/YARN-7999
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: Eric Yang
>Assignee: Jason Lowe
>Priority: Major
> Attachments: YARN-7999.001.patch
>
>
> Docker container is failing to launch in trunk. The root cause is:
> {code}
> [COMPINSTANCE sleeper-1 : container_1520032931921_0001_01_20]:
> [2018-03-02 23:26:09.196]Exception from container-launch.
> Container id: container_1520032931921_0001_01_20
> Exit code: 29
> Exception message: image: hadoop/centos:latest is trusted in hadoop registry.
> Could not determine real path of mount
> '/tmp/hadoop-yarn/nm-local-dir/usercache/hbase/filecache'
> Could not determine real path of mount
> '/tmp/hadoop-yarn/nm-local-dir/usercache/hbase/filecache'
> Invalid docker mount
> '/tmp/hadoop-yarn/nm-local-dir/usercache/hbase/filecache:/tmp/hadoop-yarn/nm-local-dir/usercache/hbase/filecache',
> realpath=/tmp/hadoop-yarn/nm-local-dir/usercache/hbase/filecache
> Error constructing docker command, docker error code=12, error
> message='Invalid docker mount'
> Shell output: main : command provided 4
> main : run as user is hbase
> main : requested yarn user is hbase
> Creating script paths...
> Creating local dirs...
> [2018-03-02 23:26:09.240]Diagnostic message from attempt 0 : [2018-03-02
> 23:26:09.240]
> [2018-03-02 23:26:09.240]Container exited with a non-zero exit code 29.
> [2018-03-02 23:26:39.278]Could not find
> nmPrivate/application_1520032931921_0001/container_1520032931921_0001_01_20//container_1520032931921_0001_01_20.pid
> in any of the directories
> [COMPONENT sleeper]: Failed 11 times, exceeded the limit - 10. Shutting down
> now...
> {code}
> The filecache cant not be mounted because it doesn't exist.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7346) Add a profile to allow optional compilation for ATSv2 with HBase-2.0
[ https://issues.apache.org/jira/browse/YARN-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387930#comment-16387930 ] Haibo Chen commented on YARN-7346: -- [~rohithsharma] While resolving the conflicts and thinking about this more, do we want to support branch-2 with HBase-2.0 given that HBase 2.0 takes hadoop 3.0 as a dependency? The HBase 2.0 with branch-2 combo might be something no others will try to test and maintain, which means we, ATSv2 community, needs to take care of that by ourselves. Just to keep the two branches (branch-2 and trunk) close, we can file a Jira to change branch-2 so that it has the same file structure of trunk without the conditional compilation bits. Thoughts? CC [~vrushalic] > Add a profile to allow optional compilation for ATSv2 with HBase-2.0 > > > Key: YARN-7346 > URL: https://issues.apache.org/jira/browse/YARN-7346 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Ted Yu >Assignee: Haibo Chen >Priority: Major > Fix For: 3.2.0 > > Attachments: YARN-7346.00.patch, YARN-7346.01.patch, > YARN-7346.02.patch, YARN-7346.03-incremental.patch, YARN-7346.03.patch, > YARN-7346.04-incremental.patch, YARN-7346.04.patch, YARN-7346.05.patch, > YARN-7346.06.patch, YARN-7346.07.patch, YARN-7346.08-incremental.patch, > YARN-7346.08.patch, YARN-7346.09.patch, YARN-7346.10.patch, > YARN-7346.11.patch, YARN-7346.prelim1.patch, YARN-7346.prelim2.patch, > YARN-7581.prelim.patch, > hadoop-yarn-server-timelineservice-hbase-server-1-findbugsXml.xml, > hadoop-yarn-server-timelineservice-hbase-server-1-javadoc-report.txt, > hadoop-yarn-server-timelineservice-hbase-server-2-findbugsXml.xml, > hadoop-yarn-server-timelineservice-hbase-server-2-javadoc-report.txt > > > When compiling hadoop-yarn-server-timelineservice-hbase against 2.0.0-alpha3, > I got the following errors: > [https://pastebin.com/Ms4jYEVB] > This issue is to fix the compilation errors. > The scope of the Jira is to add a profile to allow optional compilation for > ATSv2 with HBase2.0. The default compilation for trunk will still be for > hbase 1.2.6. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7998) RM crashes with NPE during recovering if ACL configuration was changed
[ https://issues.apache.org/jira/browse/YARN-7998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387924#comment-16387924 ] Oleksandr Shevchenko commented on YARN-7998: Fixed whitespaces [^YARN-7998.003.patch] > RM crashes with NPE during recovering if ACL configuration was changed > -- > > Key: YARN-7998 > URL: https://issues.apache.org/jira/browse/YARN-7998 > Project: Hadoop YARN > Issue Type: Bug > Components: fairscheduler, resourcemanager >Affects Versions: 3.0.0 >Reporter: Oleksandr Shevchenko >Priority: Major > Attachments: YARN-7998.000.patch, YARN-7998.001.patch, > YARN-7998.002.patch, YARN-7998.003.patch > > > RM crashes with NPE during failover because ACL configurations were changed > as a result we no longer have a rights to submit an application to a queue. > Scenario: > # Submit an application > # Change ACL configuration for a queue that accepted the application so that > an owner of the application will no longer have a rights to submit this > application. > # Restart RM. > As a result, we get NPE: > 2018-02-27 18:14:00,968 INFO org.apache.hadoop.service.AbstractService: > Service ResourceManager failed in state STARTED; cause: > java.lang.NullPointerException > java.lang.NullPointerException > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.addApplicationAttempt(FairScheduler.java:738) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:1286) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:116) > at > org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl$AttemptRecoveredTransition.transition(RMAppAttemptImpl.java:1098) > at > org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl$AttemptRecoveredTransition.transition(RMAppAttemptImpl.java:1044) > at > org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7998) RM crashes with NPE during recovering if ACL configuration was changed
[ https://issues.apache.org/jira/browse/YARN-7998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleksandr Shevchenko updated YARN-7998: --- Attachment: YARN-7998.003.patch > RM crashes with NPE during recovering if ACL configuration was changed > -- > > Key: YARN-7998 > URL: https://issues.apache.org/jira/browse/YARN-7998 > Project: Hadoop YARN > Issue Type: Bug > Components: fairscheduler, resourcemanager >Affects Versions: 3.0.0 >Reporter: Oleksandr Shevchenko >Priority: Major > Attachments: YARN-7998.000.patch, YARN-7998.001.patch, > YARN-7998.002.patch, YARN-7998.003.patch > > > RM crashes with NPE during failover because ACL configurations were changed > as a result we no longer have a rights to submit an application to a queue. > Scenario: > # Submit an application > # Change ACL configuration for a queue that accepted the application so that > an owner of the application will no longer have a rights to submit this > application. > # Restart RM. > As a result, we get NPE: > 2018-02-27 18:14:00,968 INFO org.apache.hadoop.service.AbstractService: > Service ResourceManager failed in state STARTED; cause: > java.lang.NullPointerException > java.lang.NullPointerException > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.addApplicationAttempt(FairScheduler.java:738) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:1286) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:116) > at > org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl$AttemptRecoveredTransition.transition(RMAppAttemptImpl.java:1098) > at > org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl$AttemptRecoveredTransition.transition(RMAppAttemptImpl.java:1044) > at > org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7998) RM crashes with NPE during recovering if ACL configuration was changed
[
https://issues.apache.org/jira/browse/YARN-7998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387905#comment-16387905
]
genericqa commented on YARN-7998:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
44s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 57s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 27s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 5 new + 142 unchanged - 0 fixed = 147 total (was 142)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 5 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 34s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 68m
49s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}112m 2s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f |
| JIRA Issue | YARN-7998 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12913194/YARN-7998.002.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 723e1eff4992 4.4.0-64-generic #85-Ubuntu SMP Mon Feb 20
11:50:30 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 12ecb55 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/19897/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/19897/artifact/out/whitespace-eol.txt
|
| Test Results |
[jira] [Updated] (YARN-2442) ResourceManager JMX UI does not give HA State
[ https://issues.apache.org/jira/browse/YARN-2442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S updated YARN-2442: Attachment: YARN-2442.02.patch > ResourceManager JMX UI does not give HA State > - > > Key: YARN-2442 > URL: https://issues.apache.org/jira/browse/YARN-2442 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.5.0, 2.6.0, 2.7.0 >Reporter: Nishan Shetty >Assignee: Rohith Sharma K S >Priority: Major > Labels: oct16-easy > Attachments: 0001-YARN-2442.patch, YARN-2442.02.patch > > > ResourceManager JMX UI can show the haState (INITIALIZING, ACTIVE, STANDBY, > STOPPED) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7998) RM crashes with NPE during recovering if ACL configuration was changed
[ https://issues.apache.org/jira/browse/YARN-7998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleksandr Shevchenko updated YARN-7998: --- Component/s: fairscheduler > RM crashes with NPE during recovering if ACL configuration was changed > -- > > Key: YARN-7998 > URL: https://issues.apache.org/jira/browse/YARN-7998 > Project: Hadoop YARN > Issue Type: Bug > Components: fairscheduler, resourcemanager >Affects Versions: 3.0.0 >Reporter: Oleksandr Shevchenko >Priority: Major > Attachments: YARN-7998.000.patch, YARN-7998.001.patch, > YARN-7998.002.patch > > > RM crashes with NPE during failover because ACL configurations were changed > as a result we no longer have a rights to submit an application to a queue. > Scenario: > # Submit an application > # Change ACL configuration for a queue that accepted the application so that > an owner of the application will no longer have a rights to submit this > application. > # Restart RM. > As a result, we get NPE: > 2018-02-27 18:14:00,968 INFO org.apache.hadoop.service.AbstractService: > Service ResourceManager failed in state STARTED; cause: > java.lang.NullPointerException > java.lang.NullPointerException > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.addApplicationAttempt(FairScheduler.java:738) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:1286) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:116) > at > org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl$AttemptRecoveredTransition.transition(RMAppAttemptImpl.java:1098) > at > org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl$AttemptRecoveredTransition.transition(RMAppAttemptImpl.java:1044) > at > org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8002) Support NOT_SELF and ALL namespace types for allocation tag
[
https://issues.apache.org/jira/browse/YARN-8002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387727#comment-16387727
]
Weiwei Yang commented on YARN-8002:
---
Oops, the UT failure was because I forgot to update
{{LocalAllocationTagsManager}}, uploaded v2 patch to fix this, along with minor
checkstyle and java doc issues.
Hi [~asuresh], [~leftnoteasy], [~kkaranasos], could you help to review this
patch?
> Support NOT_SELF and ALL namespace types for allocation tag
> ---
>
> Key: YARN-8002
> URL: https://issues.apache.org/jira/browse/YARN-8002
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
>Priority: Major
> Attachments: YARN-8002.001.patch, YARN-8002.002.patch
>
>
> This is a continua task after YARN-7972, YARN-7972 adds support to specify
> tags with namespace SELF and APP_ID, like following
> * self/
> * app-id//
> this task is to track the work to support 2 of remaining namespace types
> *NOT_SELF* & *ALL* (we'll support app-label later),
> * not-self/
> * all/
> this will require a bit refactoring in {{AllocationTagsManager}} as it needs
> to do some proper aggregation on tags for multiple apps.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8002) Support NOT_SELF and ALL namespace types for allocation tag
[
https://issues.apache.org/jira/browse/YARN-8002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weiwei Yang updated YARN-8002:
--
Attachment: YARN-8002.002.patch
> Support NOT_SELF and ALL namespace types for allocation tag
> ---
>
> Key: YARN-8002
> URL: https://issues.apache.org/jira/browse/YARN-8002
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
>Priority: Major
> Attachments: YARN-8002.001.patch, YARN-8002.002.patch
>
>
> This is a continua task after YARN-7972, YARN-7972 adds support to specify
> tags with namespace SELF and APP_ID, like following
> * self/
> * app-id//
> this task is to track the work to support 2 of remaining namespace types
> *NOT_SELF* & *ALL* (we'll support app-label later),
> * not-self/
> * all/
> this will require a bit refactoring in {{AllocationTagsManager}} as it needs
> to do some proper aggregation on tags for multiple apps.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7998) RM crashes with NPE during recovering if ACL configuration was changed
[ https://issues.apache.org/jira/browse/YARN-7998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleksandr Shevchenko updated YARN-7998: --- Attachment: YARN-7998.002.patch > RM crashes with NPE during recovering if ACL configuration was changed > -- > > Key: YARN-7998 > URL: https://issues.apache.org/jira/browse/YARN-7998 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 3.0.0 >Reporter: Oleksandr Shevchenko >Priority: Major > Attachments: YARN-7998.000.patch, YARN-7998.001.patch, > YARN-7998.002.patch > > > RM crashes with NPE during failover because ACL configurations were changed > as a result we no longer have a rights to submit an application to a queue. > Scenario: > # Submit an application > # Change ACL configuration for a queue that accepted the application so that > an owner of the application will no longer have a rights to submit this > application. > # Restart RM. > As a result, we get NPE: > 2018-02-27 18:14:00,968 INFO org.apache.hadoop.service.AbstractService: > Service ResourceManager failed in state STARTED; cause: > java.lang.NullPointerException > java.lang.NullPointerException > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.addApplicationAttempt(FairScheduler.java:738) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:1286) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:116) > at > org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl$AttemptRecoveredTransition.transition(RMAppAttemptImpl.java:1098) > at > org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl$AttemptRecoveredTransition.transition(RMAppAttemptImpl.java:1044) > at > org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7998) RM crashes with NPE during recovering if ACL configuration was changed
[ https://issues.apache.org/jira/browse/YARN-7998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387712#comment-16387712 ] Oleksandr Shevchenko commented on YARN-7998: Added minor changes [^YARN-7998.002.patch] > RM crashes with NPE during recovering if ACL configuration was changed > -- > > Key: YARN-7998 > URL: https://issues.apache.org/jira/browse/YARN-7998 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 3.0.0 >Reporter: Oleksandr Shevchenko >Priority: Major > Attachments: YARN-7998.000.patch, YARN-7998.001.patch, > YARN-7998.002.patch > > > RM crashes with NPE during failover because ACL configurations were changed > as a result we no longer have a rights to submit an application to a queue. > Scenario: > # Submit an application > # Change ACL configuration for a queue that accepted the application so that > an owner of the application will no longer have a rights to submit this > application. > # Restart RM. > As a result, we get NPE: > 2018-02-27 18:14:00,968 INFO org.apache.hadoop.service.AbstractService: > Service ResourceManager failed in state STARTED; cause: > java.lang.NullPointerException > java.lang.NullPointerException > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.addApplicationAttempt(FairScheduler.java:738) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:1286) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:116) > at > org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl$AttemptRecoveredTransition.transition(RMAppAttemptImpl.java:1098) > at > org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl$AttemptRecoveredTransition.transition(RMAppAttemptImpl.java:1044) > at > org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7952) Find a way to persist the log aggregation status
[
https://issues.apache.org/jira/browse/YARN-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387707#comment-16387707
]
Xuan Gong commented on YARN-7952:
-
Thanks for the review. [~leftnoteasy]
bq. It looks like pullLock should be a readLock and updateLocker should be a
writeLock since pull doesn't change internal data, correct?
No, it is not. Let me try to explain this. For one application in one NM, we
only have one AppLogAggregator which would do the log aggregation which means
it would call updateLogAggregationStatus. And we should allow different
AppLogAggregator to update their own log aggregation status. In this case, I
think that the readLock can be used here.
But for the pull call, we need to make sure when we do the pull, there is no
update. (or when we do update, we should block pull)
bq .For overall workflow of this class, IMO it should be:
This is right.(a. When log aggregation starts, application will be added to
{{NMLogAggregationStatusTracker}} with timestamp.)
This is not correct (b. When RM acknowledges application finished, application
will be removed from the {{NMLogAggregationStatusTracker) Because as I
described in option One, currently, RM will not figure out the log aggregation
status automatically by itself until some CLI or web API call happens. So, RM
will not know whether the status is final immediately that is why NM would
check and remove it periodically without RM.}}
This is correct.(c. If configured timeout reaches, application will be removed
from the {{NMLogAggregationStatusTracker}}.)
bq. Since the currentTime always almost equal to updateTime. The if statement
should be false, correct? And I think if application is null in NM side, we
should not add it to the track, correct?
This if is mainly for handling the mis-order log aggregation update call. So,
first of all, the currentTime should be always larger than updateTime. If this
is the mis-order log aggregation update call, and it is out of timeOut period,
we should ignore it.
bq. I'm not sure if we should ever change the {{lastModifiedTime}}. IIUC, the
RM will timeout log aggregation status after configured timeout. What's the
purpose of updating the {{lastModifiedTime}}? Will this cause OOM issue
potentially?
This check is also used to handle the mis-order log aggregation update call. We
should always handle the latest update call. If the call is from previous, we
need to ignore it.
bq. the {{long updateTime}} parameter is not necessary, can always use
System.currentTime instead.
We need, See my previous comment.
For other re-name comment, i will handle them together after your next comment.
> Find a way to persist the log aggregation status
>
>
> Key: YARN-7952
> URL: https://issues.apache.org/jira/browse/YARN-7952
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Xuan Gong
>Assignee: Xuan Gong
>Priority: Major
> Attachments: YARN-7952-poc.patch, YARN-7952.1.patch, YARN-7952.2.patch
>
>
> In MAPREDUCE-6415, we have created a CLI to har the aggregated logs, and In
> YARN-4946: RM should write out Aggregated Log Completion file flag next to
> logs, we have a discussion on how we can get the log aggregation status: make
> a client call to RM or get it directly from the Distributed file system(HDFS).
> No matter which approach we would like to choose, we need to figure out a way
> to persist the log aggregation status first. This ticket is used to track the
> working progress for this purpose.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-7952) Find a way to persist the log aggregation status
[
https://issues.apache.org/jira/browse/YARN-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387707#comment-16387707
]
Xuan Gong edited comment on YARN-7952 at 3/6/18 12:52 PM:
--
Thanks for the review. [~leftnoteasy]
{quote}It looks like pullLock should be a readLock and updateLocker should be a
writeLock since pull doesn't change internal data, correct?
{quote}
No, it is not. Let me try to explain this. For one application in one NM, we
only have one AppLogAggregator which would do the log aggregation which means
it would call updateLogAggregationStatus. And we should allow different
AppLogAggregator to update their own log aggregation status. In this case, I
think that the readLock can be used here.
But for the pull call, we need to make sure when we do the pull, there is no
update. (or when we do update, we should block pull)
bq. For overall workflow of this class, IMO it should be:
This is right.(a. When log aggregation starts, application will be added to
{{NMLogAggregationStatusTracker}} with timestamp.)
This is not correct (b. When RM acknowledges application finished, application
will be removed from the {{NMLogAggregationStatusTracker) Because as I
described in option One, currently, RM will not figure out the log aggregation
status automatically by itself until some CLI or web API call happens. So, RM
will not know whether the status is final immediately that is why NM would
check and remove it periodically without RM.}}
This is correct.(c. If configured timeout reaches, application will be removed
from the {{NMLogAggregationStatusTracker}}.)
{quote} Since the currentTime always almost equal to updateTime. The if
statement should be false, correct? And I think if application is null in NM
side, we should not add it to the track, correct?
{quote}
This if is mainly for handling the mis-order log aggregation update call. So,
first of all, the currentTime should be always larger than updateTime. If this
is the mis-order log aggregation update call, and it is out of timeOut period,
we should ignore it.
{quote} I'm not sure if we should ever change the {{lastModifiedTime}}. IIUC,
the RM will timeout log aggregation status after configured timeout. What's the
purpose of updating the {{lastModifiedTime}}? Will this cause OOM issue
potentially?
{quote}
This check is also used to handle the mis-order log aggregation update call. We
should always handle the latest update call. If the call is from previous, we
need to ignore it.
{quote} the {{long updateTime}} parameter is not necessary, can always use
System.currentTime instead.
{quote}
We need, See my previous comment.
For other re-name comment, i will handle them together after your next comment.
was (Author: xgong):
Thanks for the review. [~leftnoteasy]
bq. It looks like pullLock should be a readLock and updateLocker should be a
writeLock since pull doesn't change internal data, correct?
No, it is not. Let me try to explain this. For one application in one NM, we
only have one AppLogAggregator which would do the log aggregation which means
it would call updateLogAggregationStatus. And we should allow different
AppLogAggregator to update their own log aggregation status. In this case, I
think that the readLock can be used here.
But for the pull call, we need to make sure when we do the pull, there is no
update. (or when we do update, we should block pull)
bq .For overall workflow of this class, IMO it should be:
This is right.(a. When log aggregation starts, application will be added to
{{NMLogAggregationStatusTracker}} with timestamp.)
This is not correct (b. When RM acknowledges application finished, application
will be removed from the {{NMLogAggregationStatusTracker) Because as I
described in option One, currently, RM will not figure out the log aggregation
status automatically by itself until some CLI or web API call happens. So, RM
will not know whether the status is final immediately that is why NM would
check and remove it periodically without RM.}}
This is correct.(c. If configured timeout reaches, application will be removed
from the {{NMLogAggregationStatusTracker}}.)
bq. Since the currentTime always almost equal to updateTime. The if statement
should be false, correct? And I think if application is null in NM side, we
should not add it to the track, correct?
This if is mainly for handling the mis-order log aggregation update call. So,
first of all, the currentTime should be always larger than updateTime. If this
is the mis-order log aggregation update call, and it is out of timeOut period,
we should ignore it.
bq. I'm not sure if we should ever change the {{lastModifiedTime}}. IIUC, the
RM will timeout log aggregation status after configured timeout. What's the
purpose of updating the {{lastModifiedTime}}? Will this cause OOM issue
potentially?
This
[jira] [Commented] (YARN-8001) Newly created Yarn application ID lost after RM failover
[ https://issues.apache.org/jira/browse/YARN-8001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387624#comment-16387624 ] Rohith Sharma K S commented on YARN-8001: - In such case, YARN Client resubmit the application to new RM with new application id. To the user, there is NO impact. Submitted application start running and succeeds. You should be seeing a log in client something like "Re-submit application with the same ApplicationSubmissionContext" What is the problem are you faced with this kind of restart? Doesn't your application resubmitted and succeeded? > Newly created Yarn application ID lost after RM failover > > > Key: YARN-8001 > URL: https://issues.apache.org/jira/browse/YARN-8001 > Project: Hadoop YARN > Issue Type: Bug > Components: RM >Affects Versions: 2.7.3, 2.9.0 >Reporter: shanyu zhao >Priority: Major > > I’ve seen a problem in Hadoop 2.7.3 where the newly submitted yarn > application was lost after a RM failover. It looks like when handling > Application submission, RM does not write it to the state-store (We are using > zookeeper based state store) immediately before it respond to the client. But > later it failed over to another RM and all write call to the state store > failed. The new RM recovers state from the state-store, and this app is lost. > > The symptom is error message at client side claiming a previously submitted > application ID does not exist: > 2018-02-22 14:54:50,258 [JobControl] WARN > org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider - > Invocation returned exception on [rm1] : > org.apache.hadoop.yarn.exceptions.ApplicationNotFoundException: Application > with id 'application_1519310222933_0160' doesn't exist in RM. Please check > that the job submission was successful. > > This is a timeline excerpted from the resource manager logs: > 2018-02-22 14:54:06.7685260 headnode1 Storing application with id > application_1519310222933_0160 > 2018-02-22 14:54:06.7685660 headnode1 > application_1519310222933_0160 State change from NEW to NEW_SAVING > 2018-02-22 14:54:17.8924760 headnode1 Transitioning to standby state > 2018-02-22 14:54:30.3951160 headnode0 Transitioning to active state -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8002) Support NOT_SELF and ALL namespace types for allocation tag
[
https://issues.apache.org/jira/browse/YARN-8002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387490#comment-16387490
]
genericqa commented on YARN-8002:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
47s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
52s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 50s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
14s{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api in
trunk has 1 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
39s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 10s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 2 new + 47 unchanged - 1 fixed = 49 total (was 48) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 3s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
34s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
28s{color} | {color:red}
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager
generated 1 new + 4 unchanged - 0 fixed = 5 total (was 4) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
39s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 75m 32s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
32s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}147m 32s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.scheduler.constraint.TestPlacementConstraintsUtil
|
| |
hadoop.yarn.server.resourcemanager.scheduler.constraint.TestPlacementProcessor |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f |
| JIRA Issue | YARN-8002 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12913159/YARN-8002.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux
