[jira] [Commented] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15268123#comment-15268123 ] Sangjin Lee commented on MAPREDUCE-6688: The latest patch (v2.02) looks good. I'll wait for jenkins to come back. > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Affects Versions: YARN-2928 >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > MAPREDUCE-6688-YARN-2928.02.patch, MAPREDUCE-6688-YARN-2928.03.patch, > MAPREDUCE-6688-YARN-2928.v2.01.patch, MAPREDUCE-6688-YARN-2928.v2.02.patch, > YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[
https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15268108#comment-15268108
]
Hadoop QA commented on MAPREDUCE-6688:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 12m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 23s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
8s {color} | {color:green} YARN-2928 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 30s
{color} | {color:green} YARN-2928 passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 40s
{color} | {color:green} YARN-2928 passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s {color} | {color:green} YARN-2928 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 30s
{color} | {color:green} YARN-2928 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
45s {color} | {color:green} YARN-2928 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 15s
{color} | {color:red}
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
in YARN-2928 has 2 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 49s
{color} | {color:green} YARN-2928 passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 58s
{color} | {color:green} YARN-2928 passed with JDK v1.7.0_95 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 25s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 25s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 37s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 37s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 31s
{color} | {color:red} hadoop-mapreduce-project/hadoop-mapreduce-client: patch
generated 30 new + 711 unchanged - 32 fixed = 741 total (was 743) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 23s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
40s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
52s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 41s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 51s
{color} | {color:green} hadoop-mapreduce-client-core in the patch passed with
JDK v1.8.0_91. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m 10s
{color} | {color:green} hadoop-mapreduce-client-app in the patch passed with
JDK v1.8.0_91. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 108m 3s {color}
| {color:red} hadoop-mapreduce-client-jobclient in the patch failed with JDK
v1.8.0_91. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 15s
{color} | {color:green} hadoop-mapreduce-client-core in the patch passed with
JDK v1.7.0_95. {color} |
| {color:green}+1{color} |
[jira] [Commented] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15268084#comment-15268084 ] Varun Saxena commented on MAPREDUCE-6688: - Fixed the issues pointed out by Sangjin. Converted configs put to a sync call. > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Affects Versions: YARN-2928 >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > MAPREDUCE-6688-YARN-2928.02.patch, MAPREDUCE-6688-YARN-2928.03.patch, > MAPREDUCE-6688-YARN-2928.v2.01.patch, MAPREDUCE-6688-YARN-2928.v2.02.patch, > YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated MAPREDUCE-6688: Status: Patch Available (was: Open) > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Affects Versions: YARN-2928 >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > MAPREDUCE-6688-YARN-2928.02.patch, MAPREDUCE-6688-YARN-2928.03.patch, > MAPREDUCE-6688-YARN-2928.v2.01.patch, MAPREDUCE-6688-YARN-2928.v2.02.patch, > YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated MAPREDUCE-6688: Attachment: (was: MAPREDUCE-6688-YARN-2928.v2.02.patch) > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Affects Versions: YARN-2928 >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > MAPREDUCE-6688-YARN-2928.02.patch, MAPREDUCE-6688-YARN-2928.03.patch, > MAPREDUCE-6688-YARN-2928.v2.01.patch, MAPREDUCE-6688-YARN-2928.v2.02.patch, > YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated MAPREDUCE-6688: Attachment: MAPREDUCE-6688-YARN-2928.v2.02.patch > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Affects Versions: YARN-2928 >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > MAPREDUCE-6688-YARN-2928.02.patch, MAPREDUCE-6688-YARN-2928.03.patch, > MAPREDUCE-6688-YARN-2928.v2.01.patch, MAPREDUCE-6688-YARN-2928.v2.02.patch, > YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated MAPREDUCE-6688: Status: Open (was: Patch Available) > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Affects Versions: YARN-2928 >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > MAPREDUCE-6688-YARN-2928.02.patch, MAPREDUCE-6688-YARN-2928.03.patch, > MAPREDUCE-6688-YARN-2928.v2.01.patch, MAPREDUCE-6688-YARN-2928.v2.02.patch, > YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15268078#comment-15268078 ] Varun Saxena commented on MAPREDUCE-6688: - Thanks [~gtCarrera9]. Yeah, checked Jenkins now. It seems to be working. For some reason last nights' build was not invoked. Anyways will submit for the new patch. > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Affects Versions: YARN-2928 >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > MAPREDUCE-6688-YARN-2928.02.patch, MAPREDUCE-6688-YARN-2928.03.patch, > MAPREDUCE-6688-YARN-2928.v2.01.patch, MAPREDUCE-6688-YARN-2928.v2.02.patch, > YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated MAPREDUCE-6688: Status: Patch Available (was: Open) > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Affects Versions: YARN-2928 >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > MAPREDUCE-6688-YARN-2928.02.patch, MAPREDUCE-6688-YARN-2928.03.patch, > MAPREDUCE-6688-YARN-2928.v2.01.patch, MAPREDUCE-6688-YARN-2928.v2.02.patch, > YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated MAPREDUCE-6688: Status: Open (was: Patch Available) > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Affects Versions: YARN-2928 >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > MAPREDUCE-6688-YARN-2928.02.patch, MAPREDUCE-6688-YARN-2928.03.patch, > MAPREDUCE-6688-YARN-2928.v2.01.patch, MAPREDUCE-6688-YARN-2928.v2.02.patch, > YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated MAPREDUCE-6688: Status: Patch Available (was: Open) > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Affects Versions: YARN-2928 >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > MAPREDUCE-6688-YARN-2928.02.patch, MAPREDUCE-6688-YARN-2928.03.patch, > MAPREDUCE-6688-YARN-2928.v2.01.patch, MAPREDUCE-6688-YARN-2928.v2.02.patch, > YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated MAPREDUCE-6688: Attachment: MAPREDUCE-6688-YARN-2928.v2.02.patch > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Affects Versions: YARN-2928 >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > MAPREDUCE-6688-YARN-2928.02.patch, MAPREDUCE-6688-YARN-2928.03.patch, > MAPREDUCE-6688-YARN-2928.v2.01.patch, MAPREDUCE-6688-YARN-2928.v2.02.patch, > YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated MAPREDUCE-6688: Status: Open (was: Patch Available) > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Affects Versions: YARN-2928 >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > MAPREDUCE-6688-YARN-2928.02.patch, MAPREDUCE-6688-YARN-2928.03.patch, > MAPREDUCE-6688-YARN-2928.v2.01.patch, YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15268074#comment-15268074 ] Li Lu commented on MAPREDUCE-6688: -- I can see there are active precommit MR builds on jenkins. I can help you to try kick Jenkins if needed. > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Affects Versions: YARN-2928 >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > MAPREDUCE-6688-YARN-2928.02.patch, MAPREDUCE-6688-YARN-2928.03.patch, > MAPREDUCE-6688-YARN-2928.v2.01.patch, YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15268070#comment-15268070 ] Varun Saxena commented on MAPREDUCE-6688: - Thanks [~sjlee0] for the review. Will fix and update a patch shortly. However there seems to be some issue with MAPREDUCE build. Nothing is getting invoked. > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Affects Versions: YARN-2928 >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > MAPREDUCE-6688-YARN-2928.02.patch, MAPREDUCE-6688-YARN-2928.03.patch, > MAPREDUCE-6688-YARN-2928.v2.01.patch, YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6686) Add a way to download the job config from the mapred CLI
[
https://issues.apache.org/jira/browse/MAPREDUCE-6686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15268056#comment-15268056
]
Hadoop QA commented on MAPREDUCE-6686:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 13s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 29s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
54s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 35s
{color} | {color:green} trunk passed with JDK v1.8.0_92 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 45s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 58s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
34s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} trunk passed with JDK v1.8.0_92 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 39s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 32s
{color} | {color:green} the patch passed with JDK v1.8.0_92 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 32s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 43s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 43s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 22s
{color} | {color:red} hadoop-mapreduce-project/hadoop-mapreduce-client: patch
generated 1 new + 37 unchanged - 1 fixed = 38 total (was 38) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 55s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
52s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 28s
{color} | {color:green} the patch passed with JDK v1.8.0_92 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 53s
{color} | {color:green} hadoop-mapreduce-client-core in the patch passed with
JDK v1.8.0_92. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 99m 14s {color}
| {color:red} hadoop-mapreduce-client-jobclient in the patch failed with JDK
v1.8.0_92. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 21s
{color} | {color:green} hadoop-mapreduce-client-core in the patch passed with
JDK v1.7.0_95. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 120m 46s
{color} | {color:red} hadoop-mapreduce-client-jobclient in the patch failed
with JDK v1.7.0_95. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
|
[jira] [Commented] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15268040#comment-15268040 ] Varun Saxena commented on MAPREDUCE-6688: - bq. If the data is too big then a blocking call may delay the job launching process. Seems it would not. Other components interact with JobHistoryEventHandler via a queue. JobHistoryEventHandler fetches events from this queue and then processes them. bq. I'm leaning slightly towards the blocking call, as the config is arguably more important information. Ok. Lets go with sync then. I was frankly thinking of configs to be async put so that we carry on and process other events. But you can say better to publish configs then not have them at all esp when we are publishing them from a single place. If we consider some events to be important and hence a sync, should we do a retry on exception ? Probably the only advantage we gain right now is that they get pushed immediately and hence would have lesser chance of not being sent on stop or worse still not being missed on a crash. > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Affects Versions: YARN-2928 >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > MAPREDUCE-6688-YARN-2928.02.patch, MAPREDUCE-6688-YARN-2928.03.patch, > MAPREDUCE-6688-YARN-2928.v2.01.patch, YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6526) Remove usage of metrics v1 from hadoop-mapreduce
[ https://issues.apache.org/jira/browse/MAPREDUCE-6526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15267955#comment-15267955 ] Hudson commented on MAPREDUCE-6526: --- FAILURE: Integrated in Hadoop-trunk-Commit #9703 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/9703/]) MAPREDUCE-6526. Remove usage of metrics v1 from hadoop-mapreduce. (aajisaka: rev 4ee4e5ca2b8488459d2231dd1de8ed44dd656d5c) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapred/LocalJobRunnerMetrics.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/Shuffle.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/ShuffleClientMetrics.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapred/LocalJobRunner.java > Remove usage of metrics v1 from hadoop-mapreduce > > > Key: MAPREDUCE-6526 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6526 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Akira AJISAKA >Assignee: Akira AJISAKA >Priority: Blocker > Fix For: 3.0.0 > > Attachments: MAPREDUCE-6526.00.patch, MAPREDUCE-6526.01.patch, > MAPREDUCE-6526.02.patch, MAPREDUCE-6526.03.patch > > > LocalJobRunnerMetrics and ShuffleClientMetrics are still using metrics v1. We > should remove these metrics or rewrite them to use metrics v2. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6526) Remove usage of metrics v1 from hadoop-mapreduce
[ https://issues.apache.org/jira/browse/MAPREDUCE-6526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA updated MAPREDUCE-6526: - Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Committed this to trunk. Thanks Andrew for reviewing. > Remove usage of metrics v1 from hadoop-mapreduce > > > Key: MAPREDUCE-6526 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6526 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Akira AJISAKA >Assignee: Akira AJISAKA >Priority: Blocker > Fix For: 3.0.0 > > Attachments: MAPREDUCE-6526.00.patch, MAPREDUCE-6526.01.patch, > MAPREDUCE-6526.02.patch, MAPREDUCE-6526.03.patch > > > LocalJobRunnerMetrics and ShuffleClientMetrics are still using metrics v1. We > should remove these metrics or rewrite them to use metrics v2. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6526) Remove usage of metrics v1 from hadoop-mapreduce
[ https://issues.apache.org/jira/browse/MAPREDUCE-6526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA updated MAPREDUCE-6526: - Release Note: LocalJobRunnerMetrics and ShuffleClientMetrics were updated to use Hadoop Metrics V2 framework. > Remove usage of metrics v1 from hadoop-mapreduce > > > Key: MAPREDUCE-6526 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6526 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Akira AJISAKA >Assignee: Akira AJISAKA >Priority: Blocker > Attachments: MAPREDUCE-6526.00.patch, MAPREDUCE-6526.01.patch, > MAPREDUCE-6526.02.patch, MAPREDUCE-6526.03.patch > > > LocalJobRunnerMetrics and ShuffleClientMetrics are still using metrics v1. We > should remove these metrics or rewrite them to use metrics v2. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6526) Remove usage of metrics v1 from hadoop-mapreduce
[
https://issues.apache.org/jira/browse/MAPREDUCE-6526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15267898#comment-15267898
]
Hadoop QA commented on MAPREDUCE-6526:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
54s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 24s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 43s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 56s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
50s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 41s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 27s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 21s
{color} | {color:green}
hadoop-mapreduce-project_hadoop-mapreduce-client-jdk1.8.0_91 with JDK v1.8.0_91
generated 0 new + 346 unchanged - 16 fixed = 346 total (was 362) {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 27s
{color} | {color:green} hadoop-mapreduce-client in the patch passed with JDK
v1.8.0_91. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 40s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m 1s
{color} | {color:green}
hadoop-mapreduce-project_hadoop-mapreduce-client-jdk1.7.0_95 with JDK v1.7.0_95
generated 0 new + 350 unchanged - 17 fixed = 350 total (was 367) {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 40s
{color} | {color:green} hadoop-mapreduce-client in the patch passed with JDK
v1.7.0_95. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 22s
{color} | {color:red} hadoop-mapreduce-project/hadoop-mapreduce-client: patch
generated 1 new + 74 unchanged - 1 fixed = 75 total (was 75) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 48s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 59s
{color} | {color:green} hadoop-mapreduce-client-core in the patch passed
[jira] [Commented] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15267877#comment-15267877 ] Li Lu commented on MAPREDUCE-6688: -- Actually I was thinking about using the async post vs. the synced version, too. One concern for using the blocking call is the potential size of the data we post. If the data is too big then a blocking call may delay the job launching process. Given the fact that we're limiting the size of each configs, I'm fine with either one but the blocking one seems to have stronger guarantees. I agree that we need to clean up all our sync/asycn post API calls and probably have some guidance on using them. Let's update the non-refactoring version and push this in. > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Affects Versions: YARN-2928 >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > MAPREDUCE-6688-YARN-2928.02.patch, MAPREDUCE-6688-YARN-2928.03.patch, > MAPREDUCE-6688-YARN-2928.v2.01.patch, YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6684) High contention on scanning of user directory under immediate_done in Job History Server
[
https://issues.apache.org/jira/browse/MAPREDUCE-6684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15267823#comment-15267823
]
Hadoop QA commented on MAPREDUCE-6684:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 10s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
1s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 17s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 17s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 23s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
33s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 14s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 15s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 15s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 15s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 16s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 16s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 21s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 10s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 5m 52s
{color} | {color:green} hadoop-mapreduce-client-hs in the patch passed with JDK
v1.8.0_91. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 6m 9s
{color} | {color:green} hadoop-mapreduce-client-hs in the patch passed with JDK
v1.7.0_95. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 28s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:cf2ee45 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12801818/mapreduce6684.001.patch
|
| JIRA Issue | MAPREDUCE-6684 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 4ccdab9d0c8d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 9e8411d |
| Default Java | 1.7.0_95 |
| Multi-JDK
[jira] [Commented] (MAPREDUCE-6526) Remove usage of metrics v1 from hadoop-mapreduce
[
https://issues.apache.org/jira/browse/MAPREDUCE-6526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15267790#comment-15267790
]
Hadoop QA commented on MAPREDUCE-6526:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 11m 57s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 36s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
0s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 26s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 38s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 54s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
47s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 41s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 3s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 27s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 22s
{color} | {color:green}
hadoop-mapreduce-project_hadoop-mapreduce-client-jdk1.8.0_91 with JDK v1.8.0_91
generated 0 new + 346 unchanged - 16 fixed = 346 total (was 362) {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 27s
{color} | {color:green} hadoop-mapreduce-client in the patch passed with JDK
v1.8.0_91. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 37s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m 0s
{color} | {color:green}
hadoop-mapreduce-project_hadoop-mapreduce-client-jdk1.7.0_95 with JDK v1.7.0_95
generated 0 new + 350 unchanged - 17 fixed = 350 total (was 367) {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 37s
{color} | {color:green} hadoop-mapreduce-client in the patch passed with JDK
v1.7.0_95. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 22s
{color} | {color:red} hadoop-mapreduce-project/hadoop-mapreduce-client: patch
generated 1 new + 74 unchanged - 1 fixed = 75 total (was 75) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
21s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 5s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 46s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 1m 55s {color}
| {color:red} hadoop-mapreduce-client-core in the patch failed with JDK
[jira] [Commented] (MAPREDUCE-6526) Remove usage of metrics v1 from hadoop-mapreduce
[ https://issues.apache.org/jira/browse/MAPREDUCE-6526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15267721#comment-15267721 ] Andrew Wang commented on MAPREDUCE-6526: Still LGTM :) +1 > Remove usage of metrics v1 from hadoop-mapreduce > > > Key: MAPREDUCE-6526 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6526 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Akira AJISAKA >Assignee: Akira AJISAKA >Priority: Blocker > Attachments: MAPREDUCE-6526.00.patch, MAPREDUCE-6526.01.patch, > MAPREDUCE-6526.02.patch, MAPREDUCE-6526.03.patch > > > LocalJobRunnerMetrics and ShuffleClientMetrics are still using metrics v1. We > should remove these metrics or rewrite them to use metrics v2. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6526) Remove usage of metrics v1 from hadoop-mapreduce
[
https://issues.apache.org/jira/browse/MAPREDUCE-6526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira AJISAKA updated MAPREDUCE-6526:
-
Attachment: MAPREDUCE-6526.03.patch
Thanks Andrew for the review! I forgot to fix the following checkstyle issue:
{code:title=ShuffleClientMetrics.java}
public void inputBytes(long numBytes) {
this.numBytes.incr(numBytes);
}
{code}
so I've updated the patch. Would you review it?
> Remove usage of metrics v1 from hadoop-mapreduce
>
>
> Key: MAPREDUCE-6526
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6526
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
>Reporter: Akira AJISAKA
>Assignee: Akira AJISAKA
>Priority: Blocker
> Attachments: MAPREDUCE-6526.00.patch, MAPREDUCE-6526.01.patch,
> MAPREDUCE-6526.02.patch, MAPREDUCE-6526.03.patch
>
>
> LocalJobRunnerMetrics and ShuffleClientMetrics are still using metrics v1. We
> should remove these metrics or rewrite them to use metrics v2.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6537) Include hadoop-pipes examples in the release tarball
[ https://issues.apache.org/jira/browse/MAPREDUCE-6537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15267685#comment-15267685 ] Hudson commented on MAPREDUCE-6537: --- FAILURE: Integrated in Hadoop-trunk-Commit #9702 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/9702/]) MAPREDUCE-6537. Include hadoop-pipes examples in the release tarball. (wang: rev 9e8411d0a630cabbfe7493763c172ff6754e51be) * hadoop-assemblies/src/main/resources/assemblies/hadoop-tools.xml > Include hadoop-pipes examples in the release tarball > > > Key: MAPREDUCE-6537 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6537 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: pipes >Affects Versions: 2.8.0 >Reporter: Allen Wittenauer >Assignee: Kai Sasaki >Priority: Blocker > Fix For: 2.8.0 > > Attachments: HADOOP-12381.00.patch > > > Hadoop pipes examples are built but never packaged. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6537) Include hadoop-pipes examples in the release tarball
[ https://issues.apache.org/jira/browse/MAPREDUCE-6537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated MAPREDUCE-6537: --- Affects Version/s: (was: 3.0.0) 2.8.0 > Include hadoop-pipes examples in the release tarball > > > Key: MAPREDUCE-6537 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6537 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: pipes >Affects Versions: 2.8.0 >Reporter: Allen Wittenauer >Assignee: Kai Sasaki >Priority: Blocker > Fix For: 2.8.0 > > Attachments: HADOOP-12381.00.patch > > > Hadoop pipes examples are built but never packaged. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6537) Include hadoop-pipes examples in the release tarball
[ https://issues.apache.org/jira/browse/MAPREDUCE-6537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated MAPREDUCE-6537: --- Resolution: Fixed Fix Version/s: 2.8.0 Target Version/s: 2.8.0 (was: 3.0.0) Status: Resolved (was: Patch Available) This affected branch-2 too, so I committed back through branch-2.8. Thanks again Kai for the patch, Allen for finding this issue. > Include hadoop-pipes examples in the release tarball > > > Key: MAPREDUCE-6537 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6537 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: pipes >Affects Versions: 2.8.0 >Reporter: Allen Wittenauer >Assignee: Kai Sasaki >Priority: Blocker > Fix For: 2.8.0 > > Attachments: HADOOP-12381.00.patch > > > Hadoop pipes examples are built but never packaged. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[
https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15267638#comment-15267638
]
Sangjin Lee commented on MAPREDUCE-6688:
Agreeing with [~gtCarrera9], I'd like to propose that we proceed with the
version without refactoring for now. What do others think?
I took a look at that patch (v.2.01), and had a few comments.
(JobHistoryEventHandler.java)
- l.1129-1130: The id and type are set twice (once in {{createJobEntity()}},
another here). I think these second set calls should be removed.
- l.1140-1145: if a single key/value exceeds the limit, it will do a spurious
{{putEntities()}} call with empty entities unnecessarily; I think there should
be an additional check to see if there is anything to send
Did we reach a conclusion on whether these should be {{putEntities()}}
(blocking) or {{putEntitiesAsync()}}? I'm leaning slightly towards the blocking
call, as the config is arguably more important information than periodic metric
updates for example. I don't think we have established a firm rule on what
should be sync and async, however, and we might need to revisit it as a whole
later.
> Store job configurations in Timeline Service v2
> ---
>
> Key: MAPREDUCE-6688
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster
>Affects Versions: YARN-2928
>Reporter: Junping Du
>Assignee: Varun Saxena
> Labels: yarn-2928-1st-milestone
> Attachments: MAPREDUCE-6688-YARN-2928.01.patch,
> MAPREDUCE-6688-YARN-2928.02.patch, MAPREDUCE-6688-YARN-2928.03.patch,
> MAPREDUCE-6688-YARN-2928.v2.01.patch, YARN-3959-YARN-2928.01.patch
>
>
> We already have configuration field in HBase schema for application entity.
> We need to make sure AM write it out when it get launched.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6537) Include hadoop-pipes examples in the release tarball
[ https://issues.apache.org/jira/browse/MAPREDUCE-6537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated MAPREDUCE-6537: --- Summary: Include hadoop-pipes examples in the release tarball (was: hadoop pipes examples aren't in the mvn package tar ball) > Include hadoop-pipes examples in the release tarball > > > Key: MAPREDUCE-6537 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6537 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: pipes >Affects Versions: 3.0.0 >Reporter: Allen Wittenauer >Assignee: Kai Sasaki >Priority: Blocker > Attachments: HADOOP-12381.00.patch > > > Hadoop pipes examples are built but never packaged. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6537) hadoop pipes examples aren't in the mvn package tar ball
[ https://issues.apache.org/jira/browse/MAPREDUCE-6537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15267636#comment-15267636 ] Andrew Wang commented on MAPREDUCE-6537: Tested the before and after, LGTM. Will commit shortly, thanks for the contribution [~lewuathe]! > hadoop pipes examples aren't in the mvn package tar ball > > > Key: MAPREDUCE-6537 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6537 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: pipes >Affects Versions: 3.0.0 >Reporter: Allen Wittenauer >Assignee: Kai Sasaki >Priority: Blocker > Attachments: HADOOP-12381.00.patch > > > Hadoop pipes examples are built but never packaged. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6526) Remove usage of metrics v1 from hadoop-mapreduce
[ https://issues.apache.org/jira/browse/MAPREDUCE-6526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15267590#comment-15267590 ] Andrew Wang commented on MAPREDUCE-6526: +1 LGTM, thanks Akira! > Remove usage of metrics v1 from hadoop-mapreduce > > > Key: MAPREDUCE-6526 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6526 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Akira AJISAKA >Assignee: Akira AJISAKA >Priority: Blocker > Attachments: MAPREDUCE-6526.00.patch, MAPREDUCE-6526.01.patch, > MAPREDUCE-6526.02.patch > > > LocalJobRunnerMetrics and ShuffleClientMetrics are still using metrics v1. We > should remove these metrics or rewrite them to use metrics v2. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6686) Add a way to download the job config from the mapred CLI
[
https://issues.apache.org/jira/browse/MAPREDUCE-6686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Kanter updated MAPREDUCE-6686:
-
Attachment: MAPREDUCE-6686.002.patch
The 002 patch changes the message as per what Harsh suggested.
The test failures are unrelated. The checkstyle is due to the {{CLI#run}}
method being too long, but that's how it already is.
> Add a way to download the job config from the mapred CLI
>
>
> Key: MAPREDUCE-6686
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6686
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: client
>Affects Versions: 2.9.0
>Reporter: Robert Kanter
>Assignee: Robert Kanter
> Attachments: MAPREDUCE-6686.001.patch, MAPREDUCE-6686.002.patch
>
>
> It would be convenient if there was a way to easily grab the job
> configuration via the CLI instead of having to find and go to the specific
> HDFS location to grab it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15267557#comment-15267557 ] Li Lu commented on MAPREDUCE-6688: -- bq. So I was of the opinion let us first go with what is best for current code and then refactor later, if required, based on decision taken. Thoughts ? Sure, let's take that approach. Thanks! > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Affects Versions: YARN-2928 >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > MAPREDUCE-6688-YARN-2928.02.patch, MAPREDUCE-6688-YARN-2928.03.patch, > MAPREDUCE-6688-YARN-2928.v2.01.patch, YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6684) High contention on scanning of user directory under immediate_done in Job History Server
[
https://issues.apache.org/jira/browse/MAPREDUCE-6684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Haibo Chen updated MAPREDUCE-6684:
--
Status: Patch Available (was: Open)
> High contention on scanning of user directory under immediate_done in Job
> History Server
>
>
> Key: MAPREDUCE-6684
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6684
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 2.7.0
>Reporter: Haibo Chen
>Assignee: Haibo Chen
>Priority: Critical
> Attachments: jhs-jstacks-service-monitor-running.tar.gz,
> jhs-jstacks-service-monitor-stopped.tar.gz, mapreduce6684.001.patch
>
>
> HistoryFileManager.scanIntermediateDirectory() in JHS acquires a lock on each
> user directory it tries to scan (move or delete files under the user
> directory as necessary). This method is called in a thread in JobHistory that
> performs periodical scanning of intermediate directory, and can also be
> called by web server threads for each Web API call made by a JHS client. In
> cases where there are many concurrent Web API calls/connections to JHS, all
> but one thread are blocked on the lock on the user directory. Eventually,
> client connects will time out, but the threads in JHS will not be killed and
> leave a lot of TCP connections in CLOSE_WAIT state.
> {noformat}
> [systest@vb1120 ~]$ sudo netstat -nap | grep 63729 | sort -k 4
> tcp0 0 10.17.202.19:10020 0.0.0.0:*
> LISTEN 63729/java
> tcp0 0 10.17.202.19:10020 10.17.198.30:33010
> ESTABLISHED 63729/java
> tcp0 0 10.17.202.19:10020 10.17.200.30:33980
> ESTABLISHED 63729/java
> tcp0 0 10.17.202.19:10020 10.17.202.10:59625
> ESTABLISHED 63729/java
> tcp0 0 10.17.202.19:10020 10.17.202.13:35765
> ESTABLISHED 63729/java
> tcp0 0 10.17.202.19:10033 0.0.0.0:*
> LISTEN 63729/java
> tcp0 0 10.17.202.19:19888 0.0.0.0:*
> LISTEN 63729/java
> tcp0 0 10.17.202.19:19888 10.17.198.30:35103
> ESTABLISHED 63729/java
> tcp 277 0 10.17.202.19:19888 10.17.198.30:43670
> ESTABLISHED 63729/java
> tcp0 0 10.17.202.19:19888 10.17.198.30:45453
> ESTABLISHED 63729/java
> tcp 277 0 10.17.202.19:19888 10.17.198.30:49184
> ESTABLISHED 63729/java
> tcp1 0 10.17.202.19:19888 10.17.202.13:49992
> CLOSE_WAIT 63729/java
> tcp 261 0 10.17.202.19:19888 10.17.202.13:52703
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52707
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52708
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52710
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52714
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52723
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52726
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52727
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52739
> CLOSE_WAIT 63729/java
> tcp 261 0 10.17.202.19:19888 10.17.202.13:52749
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52753
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52757
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52760
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52820
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52827
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52829
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52831
> CLOSE_WAIT 63729/java
> tcp
[jira] [Commented] (MAPREDUCE-6684) High contention on scanning of user directory under immediate_done in Job History Server
[
https://issues.apache.org/jira/browse/MAPREDUCE-6684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15267466#comment-15267466
]
Haibo Chen commented on MAPREDUCE-6684:
---
Thanks a lot for you all's comments. Uploading a patch according to the
solution suggested by Jason. Will come to this discussion if future improvement
of HistoryFileManager is necessarily needed.
> High contention on scanning of user directory under immediate_done in Job
> History Server
>
>
> Key: MAPREDUCE-6684
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6684
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 2.7.0
>Reporter: Haibo Chen
>Assignee: Haibo Chen
>Priority: Critical
> Attachments: jhs-jstacks-service-monitor-running.tar.gz,
> jhs-jstacks-service-monitor-stopped.tar.gz, mapreduce6684.001.patch
>
>
> HistoryFileManager.scanIntermediateDirectory() in JHS acquires a lock on each
> user directory it tries to scan (move or delete files under the user
> directory as necessary). This method is called in a thread in JobHistory that
> performs periodical scanning of intermediate directory, and can also be
> called by web server threads for each Web API call made by a JHS client. In
> cases where there are many concurrent Web API calls/connections to JHS, all
> but one thread are blocked on the lock on the user directory. Eventually,
> client connects will time out, but the threads in JHS will not be killed and
> leave a lot of TCP connections in CLOSE_WAIT state.
> {noformat}
> [systest@vb1120 ~]$ sudo netstat -nap | grep 63729 | sort -k 4
> tcp0 0 10.17.202.19:10020 0.0.0.0:*
> LISTEN 63729/java
> tcp0 0 10.17.202.19:10020 10.17.198.30:33010
> ESTABLISHED 63729/java
> tcp0 0 10.17.202.19:10020 10.17.200.30:33980
> ESTABLISHED 63729/java
> tcp0 0 10.17.202.19:10020 10.17.202.10:59625
> ESTABLISHED 63729/java
> tcp0 0 10.17.202.19:10020 10.17.202.13:35765
> ESTABLISHED 63729/java
> tcp0 0 10.17.202.19:10033 0.0.0.0:*
> LISTEN 63729/java
> tcp0 0 10.17.202.19:19888 0.0.0.0:*
> LISTEN 63729/java
> tcp0 0 10.17.202.19:19888 10.17.198.30:35103
> ESTABLISHED 63729/java
> tcp 277 0 10.17.202.19:19888 10.17.198.30:43670
> ESTABLISHED 63729/java
> tcp0 0 10.17.202.19:19888 10.17.198.30:45453
> ESTABLISHED 63729/java
> tcp 277 0 10.17.202.19:19888 10.17.198.30:49184
> ESTABLISHED 63729/java
> tcp1 0 10.17.202.19:19888 10.17.202.13:49992
> CLOSE_WAIT 63729/java
> tcp 261 0 10.17.202.19:19888 10.17.202.13:52703
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52707
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52708
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52710
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52714
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52723
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52726
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52727
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52739
> CLOSE_WAIT 63729/java
> tcp 261 0 10.17.202.19:19888 10.17.202.13:52749
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52753
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52757
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52760
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52820
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52827
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888
[jira] [Updated] (MAPREDUCE-6684) High contention on scanning of user directory under immediate_done in Job History Server
[
https://issues.apache.org/jira/browse/MAPREDUCE-6684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Haibo Chen updated MAPREDUCE-6684:
--
Attachment: mapreduce6684.001.patch
> High contention on scanning of user directory under immediate_done in Job
> History Server
>
>
> Key: MAPREDUCE-6684
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6684
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 2.7.0
>Reporter: Haibo Chen
>Assignee: Haibo Chen
>Priority: Critical
> Attachments: jhs-jstacks-service-monitor-running.tar.gz,
> jhs-jstacks-service-monitor-stopped.tar.gz, mapreduce6684.001.patch
>
>
> HistoryFileManager.scanIntermediateDirectory() in JHS acquires a lock on each
> user directory it tries to scan (move or delete files under the user
> directory as necessary). This method is called in a thread in JobHistory that
> performs periodical scanning of intermediate directory, and can also be
> called by web server threads for each Web API call made by a JHS client. In
> cases where there are many concurrent Web API calls/connections to JHS, all
> but one thread are blocked on the lock on the user directory. Eventually,
> client connects will time out, but the threads in JHS will not be killed and
> leave a lot of TCP connections in CLOSE_WAIT state.
> {noformat}
> [systest@vb1120 ~]$ sudo netstat -nap | grep 63729 | sort -k 4
> tcp0 0 10.17.202.19:10020 0.0.0.0:*
> LISTEN 63729/java
> tcp0 0 10.17.202.19:10020 10.17.198.30:33010
> ESTABLISHED 63729/java
> tcp0 0 10.17.202.19:10020 10.17.200.30:33980
> ESTABLISHED 63729/java
> tcp0 0 10.17.202.19:10020 10.17.202.10:59625
> ESTABLISHED 63729/java
> tcp0 0 10.17.202.19:10020 10.17.202.13:35765
> ESTABLISHED 63729/java
> tcp0 0 10.17.202.19:10033 0.0.0.0:*
> LISTEN 63729/java
> tcp0 0 10.17.202.19:19888 0.0.0.0:*
> LISTEN 63729/java
> tcp0 0 10.17.202.19:19888 10.17.198.30:35103
> ESTABLISHED 63729/java
> tcp 277 0 10.17.202.19:19888 10.17.198.30:43670
> ESTABLISHED 63729/java
> tcp0 0 10.17.202.19:19888 10.17.198.30:45453
> ESTABLISHED 63729/java
> tcp 277 0 10.17.202.19:19888 10.17.198.30:49184
> ESTABLISHED 63729/java
> tcp1 0 10.17.202.19:19888 10.17.202.13:49992
> CLOSE_WAIT 63729/java
> tcp 261 0 10.17.202.19:19888 10.17.202.13:52703
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52707
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52708
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52710
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52714
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52723
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52726
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52727
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52739
> CLOSE_WAIT 63729/java
> tcp 261 0 10.17.202.19:19888 10.17.202.13:52749
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52753
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52757
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52760
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52820
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52827
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52829
> CLOSE_WAIT 63729/java
> tcp 256 0 10.17.202.19:19888 10.17.202.13:52831
> CLOSE_WAIT 63729/java
> tcp
[jira] [Commented] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15267403#comment-15267403 ] Varun Saxena commented on MAPREDUCE-6688: - [~sjlee0], it seems build is stuck waiting for an executor. This may take a while. I will be available 6 hours later. I do have a patch ready without the refactoring though in case we cannot reach a consensus on this patch. I have uploaded that patch as well. Its named as MAPREDUCE-6688-YARN-2928.v2.01.patch I have reuploaded my previous patch at the last so that it is taken up when build is executed. > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Affects Versions: YARN-2928 >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > MAPREDUCE-6688-YARN-2928.02.patch, MAPREDUCE-6688-YARN-2928.03.patch, > MAPREDUCE-6688-YARN-2928.v2.01.patch, YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated MAPREDUCE-6688: Attachment: MAPREDUCE-6688-YARN-2928.03.patch > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Affects Versions: YARN-2928 >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > MAPREDUCE-6688-YARN-2928.02.patch, MAPREDUCE-6688-YARN-2928.03.patch, > MAPREDUCE-6688-YARN-2928.v2.01.patch, YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated MAPREDUCE-6688: Attachment: MAPREDUCE-6688-YARN-2928.03.patch > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Affects Versions: YARN-2928 >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > MAPREDUCE-6688-YARN-2928.02.patch, MAPREDUCE-6688-YARN-2928.v2.01.patch, > YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated MAPREDUCE-6688: Attachment: (was: MAPREDUCE-6688-YARN-2928.03.patch) > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Affects Versions: YARN-2928 >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > MAPREDUCE-6688-YARN-2928.02.patch, MAPREDUCE-6688-YARN-2928.v2.01.patch, > YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated MAPREDUCE-6688: Attachment: (was: MAPREDUCE-6688-YARN-2928.03.patch) > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Affects Versions: YARN-2928 >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > MAPREDUCE-6688-YARN-2928.02.patch, MAPREDUCE-6688-YARN-2928.v2.01.patch, > YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated MAPREDUCE-6688: Attachment: MAPREDUCE-6688-YARN-2928.v2.01.patch > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Affects Versions: YARN-2928 >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > MAPREDUCE-6688-YARN-2928.02.patch, MAPREDUCE-6688-YARN-2928.v2.01.patch, > YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated MAPREDUCE-6688: Attachment: MAPREDUCE-6688-YARN-2928.03.patch While doing self review found that in JobSubmittedEvent class, I could move entity object creation inside not null check for jobConf. New patch contains only this minor change. > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Affects Versions: YARN-2928 >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > MAPREDUCE-6688-YARN-2928.02.patch, MAPREDUCE-6688-YARN-2928.03.patch, > YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15267236#comment-15267236 ] Varun Saxena commented on MAPREDUCE-6688: - Updated a patch fixing checkstyle issues. This patch contains refactoring. > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Affects Versions: YARN-2928 >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > MAPREDUCE-6688-YARN-2928.02.patch, YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated MAPREDUCE-6688: Status: Patch Available (was: Open) > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Affects Versions: YARN-2928 >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > MAPREDUCE-6688-YARN-2928.02.patch, YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated MAPREDUCE-6688: Attachment: MAPREDUCE-6688-YARN-2928.02.patch > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Affects Versions: YARN-2928 >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > MAPREDUCE-6688-YARN-2928.02.patch, YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated MAPREDUCE-6688: Status: Open (was: Patch Available) > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Affects Versions: YARN-2928 >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
