[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[
https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368300#comment-16368300
]
genericqa commented on YARN-7919:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
30s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 19 new or modified test
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
19s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
50s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
40s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
57s{color} | {color:green} branch-2 passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project hadoop-assemblies
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
44s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
34s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
59s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 44s{color} | {color:orange} root: The patch generated 2 new + 11 unchanged -
0 fixed = 13 total (was 11) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project hadoop-assemblies
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
22s{color} | {color:green} hadoop-project in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
22s{color} | {color:green} hadoop-assemblies in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
44s{color} | {color:green} hadoop-yarn-server-timelineservice-hbase in the
patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
35s{color} | {color:green} hadoop-yarn-server-timelineservice-hbase-common in
the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m
9s{color} | {color:green} hadoop-yarn-server-timelineservice-hbase-tests in the
patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
33s{color} | {color:green}
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[
https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368259#comment-16368259
]
genericqa commented on YARN-7919:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
28s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
1s{color} | {color:green} The patch appears to include 19 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
20s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 58s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project hadoop-assemblies
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
31s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 49s{color} | {color:orange} root: The patch generated 1 new + 29 unchanged -
9 fixed = 30 total (was 38) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
9s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
8m 38s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project hadoop-assemblies
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
15s{color} | {color:green} hadoop-project in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
17s{color} | {color:green} hadoop-assemblies in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
36s{color} | {color:green} hadoop-yarn-server-timelineservice-hbase in the
patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
27s{color} | {color:green}
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[ https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368229#comment-16368229 ] Rohith Sharma K S commented on YARN-7919: - +1 lgtm for latest patch pending jenkins > Split timelineservice-hbase module to make YARN-7346 easier > --- > > Key: YARN-7919 > URL: https://issues.apache.org/jira/browse/YARN-7919 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineservice >Affects Versions: 3.0.0 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-7919.00.patch, YARN-7919.01.patch, > YARN-7919.02.patch, YARN-7919.03.patch, YARN-7919.04.patch, YARN-7919.05.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[ https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368221#comment-16368221 ] Haibo Chen commented on YARN-7919: -- I will work on a branch-2 patch in the meantime and upload a separate patch later. > Split timelineservice-hbase module to make YARN-7346 easier > --- > > Key: YARN-7919 > URL: https://issues.apache.org/jira/browse/YARN-7919 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineservice >Affects Versions: 3.0.0 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-7919.00.patch, YARN-7919.01.patch, > YARN-7919.02.patch, YARN-7919.03.patch, YARN-7919.04.patch, YARN-7919.05.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[ https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368220#comment-16368220 ] Haibo Chen commented on YARN-7919: -- Thanks [~rohithsharma] for the review and debugging, I have updated the patch to address the issue you found out. > Split timelineservice-hbase module to make YARN-7346 easier > --- > > Key: YARN-7919 > URL: https://issues.apache.org/jira/browse/YARN-7919 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineservice >Affects Versions: 3.0.0 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-7919.00.patch, YARN-7919.01.patch, > YARN-7919.02.patch, YARN-7919.03.patch, YARN-7919.04.patch, YARN-7919.05.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[ https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368150#comment-16368150 ] Rohith Sharma K S commented on YARN-7919: - Debugged a bit and found that _hadoop-yarn-server-timelineservice-hbase-server_ code doesn't exist in co-processor jar. Co-processor jar generated is same as _timelineservice-hbase-common_ jar. > Split timelineservice-hbase module to make YARN-7346 easier > --- > > Key: YARN-7919 > URL: https://issues.apache.org/jira/browse/YARN-7919 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineservice >Affects Versions: 3.0.0 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-7919.00.patch, YARN-7919.01.patch, > YARN-7919.02.patch, YARN-7919.03.patch, YARN-7919.04.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[
https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368126#comment-16368126
]
Rohith Sharma K S commented on YARN-7919:
-
[~haibo.chen] With fresh deployment for 04 patch, I see an error while creating
flowrun table. I am guessing something issue with fat jar generation.
{noformat}
2018-02-17 12:34:55,458 INFO [main] flow.FlowRunTableRW:
CoprocessorJarPath=file:/Users/rsharmaks/Cluster/atsv2/hbase/lib/hadoop-yarn-server-timelineservice-hbase-coprocessor-3.2.0-SNAPSHOT.jar
2018-02-17 12:34:55,541 WARN [main] storage.TimelineSchemaCreator: Skip and
continue on: org.apache.hadoop.hbase.DoNotRetryIOException: Class
org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
cannot be loaded Set hbase.table.sanity.checks to false at conf or table
descriptor if you want to bypass sanity checks
at
org.apache.hadoop.hbase.master.HMaster.warnOrThrowExceptionForFailure(HMaster.java:1754)
at
org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1615)
at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1541)
at
org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:464)
at
org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:55682)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2196)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
at
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
at java.lang.Thread.run(Thread.java:748)
{noformat}
> Split timelineservice-hbase module to make YARN-7346 easier
> ---
>
> Key: YARN-7919
> URL: https://issues.apache.org/jira/browse/YARN-7919
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineservice
>Affects Versions: 3.0.0
>Reporter: Haibo Chen
>Assignee: Haibo Chen
>Priority: Major
> Attachments: YARN-7919.00.patch, YARN-7919.01.patch,
> YARN-7919.02.patch, YARN-7919.03.patch, YARN-7919.04.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[ https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368107#comment-16368107 ] Rohith Sharma K S commented on YARN-7919: - thanks [~vrushalic]! I am doing another round of testing with this patch. I will commit it once I finish sanity testing! > Split timelineservice-hbase module to make YARN-7346 easier > --- > > Key: YARN-7919 > URL: https://issues.apache.org/jira/browse/YARN-7919 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineservice >Affects Versions: 3.0.0 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-7919.00.patch, YARN-7919.01.patch, > YARN-7919.02.patch, YARN-7919.03.patch, YARN-7919.04.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[ https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368099#comment-16368099 ] Vrushali C commented on YARN-7919: -- +1 to the overall approach. I haven't had a lot of time unfortunately to look through it in as much detail as I would have liked to, but so far it looks good to me. thanks [~haibochen] and [~rohithsharma] > Split timelineservice-hbase module to make YARN-7346 easier > --- > > Key: YARN-7919 > URL: https://issues.apache.org/jira/browse/YARN-7919 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineservice >Affects Versions: 3.0.0 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-7919.00.patch, YARN-7919.01.patch, > YARN-7919.02.patch, YARN-7919.03.patch, YARN-7919.04.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[ https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16367808#comment-16367808 ] Rohith Sharma K S commented on YARN-7919: - thanks [~vrushalic] for pitching in. As per your suggestion I tried to build for Hbase-2 and found jcoding issue :-) Let us know your feedback on 04 patch apart from jcoding issue, there are couple of nits exist but those can be handled in YARN-7346 JIRA. I will commit 04 patch this weekend if no more objections from you. > Split timelineservice-hbase module to make YARN-7346 easier > --- > > Key: YARN-7919 > URL: https://issues.apache.org/jira/browse/YARN-7919 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineservice >Affects Versions: 3.0.0 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-7919.00.patch, YARN-7919.01.patch, > YARN-7919.02.patch, YARN-7919.03.patch, YARN-7919.04.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[ https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16367772#comment-16367772 ] Vrushali C commented on YARN-7919: -- Thanks [~haibochen] for working on this patch! This will help resolve our path for working with two hbase versions going forward. Appreciate the effort. As I suggested in the weekly call yesterday, I am trying out the hbase compilation with 2.x .I see that Rohith has already reported the jcoding issue. For completeness, I will also paste in the build error for the coprocessor. I think we can deal with this version issue during the conditional compilation work later via profiles. > Split timelineservice-hbase module to make YARN-7346 easier > --- > > Key: YARN-7919 > URL: https://issues.apache.org/jira/browse/YARN-7919 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineservice >Affects Versions: 3.0.0 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-7919.00.patch, YARN-7919.01.patch, > YARN-7919.02.patch, YARN-7919.03.patch, YARN-7919.04.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[ https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16367769#comment-16367769 ] Haibo Chen commented on YARN-7919: -- Yes, doing the jcoding fix in YARN-7346 sounds good to me. > Split timelineservice-hbase module to make YARN-7346 easier > --- > > Key: YARN-7919 > URL: https://issues.apache.org/jira/browse/YARN-7919 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineservice >Affects Versions: 3.0.0 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-7919.00.patch, YARN-7919.01.patch, > YARN-7919.02.patch, YARN-7919.03.patch, YARN-7919.04.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[ https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16367727#comment-16367727 ] Rohith Sharma K S commented on YARN-7919: - [~haibo.chen] Given this issue happens to be for conditional compilation with Hbase-2, would you recommend to commit this current patch first which we have verified in hbase-1.2.6 since patch size is much bigger? Later while doing YARN-7346, lets worry more about this jcoding dependency issue? > Split timelineservice-hbase module to make YARN-7346 easier > --- > > Key: YARN-7919 > URL: https://issues.apache.org/jira/browse/YARN-7919 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineservice >Affects Versions: 3.0.0 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-7919.00.patch, YARN-7919.01.patch, > YARN-7919.02.patch, YARN-7919.03.patch, YARN-7919.04.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[ https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16367669#comment-16367669 ] Haibo Chen commented on YARN-7919: -- YARN currently does not have jcodings as a direct dependency. Copying from Rohith's comment above, this is what the dependency checker complains about [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (depcheck) @ hadoop-yarn-server-timelineservice-hbase-client --- [WARNING] Dependency convergence error for org.jruby.jcodings:jcodings:1.0.18 paths to dependency are: +-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-client:3.2.0-SNAPSHOT +-org.apache.hbase:hbase-client:2.0.0-beta-1 +-org.jruby.jcodings:jcodings:1.0.18 and +-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-client:3.2.0-SNAPSHOT +-org.apache.hbase:hbase-client:2.0.0-beta-1 +-org.jruby.joni:joni:2.1.11 +-org.jruby.jcodings:jcodings:1.0.13 So maven does see different versions. I suspect this is because the checker plugin only analyzes the pom tree without turning to maven dependency mediation mechanism. > Split timelineservice-hbase module to make YARN-7346 easier > --- > > Key: YARN-7919 > URL: https://issues.apache.org/jira/browse/YARN-7919 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineservice >Affects Versions: 3.0.0 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-7919.00.patch, YARN-7919.01.patch, > YARN-7919.02.patch, YARN-7919.03.patch, YARN-7919.04.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[ https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16367604#comment-16367604 ] Josh Elser commented on YARN-7919: -- bq. In this case, hbase depends on jcodings-1.0-18 (directly) and jcodings-1.0.13 (transitively from joni). It is probably not a real problem given both versions are in the same 1.0.x release line. But hadoop always takes the conservative approach. Is there any downside to make hbase-client depend on the exactly same version, 1.0.13, of joni? It is more of a nice-to-have dependency hygiene. Hrm, this is the part that confuses me. HBase defining a dependency on jcoding-1.0.18 should override the transitive 1.0.13 dependency and prevent YARN from seeing it. Is YARN setting a direct dependency on the same version of jcodings for some reason? > Split timelineservice-hbase module to make YARN-7346 easier > --- > > Key: YARN-7919 > URL: https://issues.apache.org/jira/browse/YARN-7919 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineservice >Affects Versions: 3.0.0 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-7919.00.patch, YARN-7919.01.patch, > YARN-7919.02.patch, YARN-7919.03.patch, YARN-7919.04.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[ https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16367594#comment-16367594 ] Haibo Chen commented on YARN-7919: -- Agreed, [~elserj]. My understanding of why enforcer-plugin is enabled in YARN is, it avoids surprises (such as reported in HADOOP-13866) that come from the breaking changes between two versions of the same dependency. In this case, hbase depends on jcodings-1.0-18 (directly) and jcodings-1.0.13 (transitively from joni). It is probably not a real problem given both versions are in the same 1.0.x release line. But hadoop always takes the conservative approach. Is there any downside to make hbase-client depend on the exactly same version, 1.0.13, of joni? It is more of a nice-to-have dependency hygiene. Regarless, we can always work around in hadoop by explicitly overriding the joni version > Split timelineservice-hbase module to make YARN-7346 easier > --- > > Key: YARN-7919 > URL: https://issues.apache.org/jira/browse/YARN-7919 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineservice >Affects Versions: 3.0.0 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-7919.00.patch, YARN-7919.01.patch, > YARN-7919.02.patch, YARN-7919.03.patch, YARN-7919.04.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[ https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16367571#comment-16367571 ] Josh Elser commented on YARN-7919: -- I'm struggling here. I don't understand what the problem is. HBase-2.0 depends on joni 2.1.11 and jcodings 1.0.18. The build failure is due to the enforcer-plugin you configured in YARN. > Split timelineservice-hbase module to make YARN-7346 easier > --- > > Key: YARN-7919 > URL: https://issues.apache.org/jira/browse/YARN-7919 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineservice >Affects Versions: 3.0.0 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-7919.00.patch, YARN-7919.01.patch, > YARN-7919.02.patch, YARN-7919.03.patch, YARN-7919.04.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[
https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16367502#comment-16367502
]
Rohith Sharma K S commented on YARN-7919:
-
{quote}we could either report to and ask HBase community so that they enforce
the same version of jcodings in hbase-client, or override jcodings version
explicitly in timelineservice modules.
{quote}
make sense to me. we can discuss or create a JIRA in HBase community as well. I
will also check with HBase members internally for feedback. cc :/ [~elserj]
{quote}The override is only necessary in hbase 2.0, so the explicit dependency
on jcodings is going to be in a maven profile.
{quote}
By worst case this would be our choice.
> Split timelineservice-hbase module to make YARN-7346 easier
> ---
>
> Key: YARN-7919
> URL: https://issues.apache.org/jira/browse/YARN-7919
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineservice
>Affects Versions: 3.0.0
>Reporter: Haibo Chen
>Assignee: Haibo Chen
>Priority: Major
> Attachments: YARN-7919.00.patch, YARN-7919.01.patch,
> YARN-7919.02.patch, YARN-7919.03.patch, YARN-7919.04.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[
https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16367485#comment-16367485
]
Haibo Chen commented on YARN-7919:
--
Good catch, [~rohithsharma]! The jcodings problem is more of a dependency
divergence rather than a compilation issue, in my opinion. I greped its usage
in hbase-client (2.0) module, here is what I found.
{code:java}
grep "org.jcodings.*" -R hbase-client
hbase-client/src/main/java/org/apache/hadoop/hbase/filter/RegexStringComparator.java:30:import
org.jcodings.Encoding;
hbase-client/src/main/java/org/apache/hadoop/hbase/filter/RegexStringComparator.java:31:import
org.jcodings.EncodingDB;
hbase-client/src/main/java/org/apache/hadoop/hbase/filter/RegexStringComparator.java:32:import
org.jcodings.specific.UTF8Encoding;
Binary file
hbase-client/target/classes/org/apache/hadoop/hbase/filter/RegexStringComparator$JoniRegexEngine.class
matches {code}
Looks like hbase-client relies on Jruby's joni regex engine to do pattern
matching. I am not sure if it is safe to just exclude jcondings to work around
the dependency divergence problem.
Alternatively, we could either report to and ask HBase community so that they
enforce the same version of jcodings in hbase-client, or override jcodings
version explicitly in timelineservice modules. The override is only necessary
in hbase 2.0, so the explicit dependency on jcodings is going to be in a maven
profile. I think we should do both, do the override as a safe work around to
unblock YARN-7346, but also file an HBase issue so that it can be properly
addressed. Thoughts?
Will address your hbase-schema > hbase-common comment when we decide what to do
with the jcondings issue.--
dfasfdas
> Split timelineservice-hbase module to make YARN-7346 easier
> ---
>
> Key: YARN-7919
> URL: https://issues.apache.org/jira/browse/YARN-7919
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineservice
>Affects Versions: 3.0.0
>Reporter: Haibo Chen
>Assignee: Haibo Chen
>Priority: Major
> Attachments: YARN-7919.00.patch, YARN-7919.01.patch,
> YARN-7919.02.patch, YARN-7919.03.patch, YARN-7919.04.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[
https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366904#comment-16366904
]
Rohith Sharma K S commented on YARN-7919:
-
I added jcodings artifact in exclusion list in
hadoop-yarn-server-timelineservice-hbase-client/pom.xml for hbase-client
dependency and able to compile this module in both HBase versions i.e
HBaes-1.2.6 and Hbase-2.0-beta1. And for HBase-2.0-beta1 failed at
hadoop-yarn-server-timelineservice-hbase-server module which is expected as
long we fix YARN-7346.
jcodings-1.0.8.jar used to be in dependent library for hbase-client in
hbase-1.2.6 version. It used to be shipped in timelineservice/lib folder. But
after this change, I do not see jcoding jar in timelineservice/lib folder when
we generate hadoop tar.gz.
My concern is does this jar really required for hbase-client or we can go ahead
excluding this jar for hbase-1.2.6 version! Does it going to cause
functionality impact?
nit : I see that package hadoop-yarn-server-timelineservice-hbase-common name
is given as {{Apache Hadoop YARN TimelineService HBase Schema}}. I think we can
keep name as same as module name i.e {{Apache Hadoop YARN TimelineService HBase
Common}}.
> Split timelineservice-hbase module to make YARN-7346 easier
> ---
>
> Key: YARN-7919
> URL: https://issues.apache.org/jira/browse/YARN-7919
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineservice
>Affects Versions: 3.0.0
>Reporter: Haibo Chen
>Assignee: Haibo Chen
>Priority: Major
> Attachments: YARN-7919.00.patch, YARN-7919.01.patch,
> YARN-7919.02.patch, YARN-7919.03.patch, YARN-7919.04.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[
https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366871#comment-16366871
]
Rohith Sharma K S commented on YARN-7919:
-
[~haibochen] I applied 04 patch and tried to build in default mode and got
succeeded. Later I tried to build by changing hbase version into beta1 with
below properties.
{code}2.0.0-beta-1
3.0.0{code}
This is giving compilation ERROR at
hadoop-yarn-server-timelineservice-hbase-client module.
{code}
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (depcheck) @
hadoop-yarn-server-timelineservice-hbase-client ---
[WARNING]
Dependency convergence error for org.jruby.jcodings:jcodings:1.0.18 paths to
dependency are:
+-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-client:3.2.0-SNAPSHOT
+-org.apache.hbase:hbase-client:2.0.0-beta-1
+-org.jruby.jcodings:jcodings:1.0.18
and
+-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-client:3.2.0-SNAPSHOT
+-org.apache.hbase:hbase-client:2.0.0-beta-1
+-org.jruby.joni:joni:2.1.11
+-org.jruby.jcodings:jcodings:1.0.13
{code}
Is there any additional libraries to be loaded for hbase-2 compilation even or
client package as well?
> Split timelineservice-hbase module to make YARN-7346 easier
> ---
>
> Key: YARN-7919
> URL: https://issues.apache.org/jira/browse/YARN-7919
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineservice
>Affects Versions: 3.0.0
>Reporter: Haibo Chen
>Assignee: Haibo Chen
>Priority: Major
> Attachments: YARN-7919.00.patch, YARN-7919.01.patch,
> YARN-7919.02.patch, YARN-7919.03.patch, YARN-7919.04.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[
https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366173#comment-16366173
]
genericqa commented on YARN-7919:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
23s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 19 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
20s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m
7s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 17m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
7s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 21s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project hadoop-assemblies
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
28s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 13m
27s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
2m 16s{color} | {color:orange} root: The patch generated 1 new + 29 unchanged -
9 fixed = 30 total (was 38) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
11s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 13s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project hadoop-assemblies
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
21s{color} | {color:green} hadoop-project in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
24s{color} | {color:green} hadoop-assemblies in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
50s{color} | {color:green} hadoop-yarn-server-timelineservice-hbase in the
patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
38s{color} | {color:green}
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[
https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366161#comment-16366161
]
genericqa commented on YARN-7919:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
25s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 19 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 6s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project hadoop-assemblies
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
38s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
55s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 47s{color} | {color:orange} root: The patch generated 1 new + 29 unchanged -
9 fixed = 30 total (was 38) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
9s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
8m 45s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project hadoop-assemblies
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
4s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
15s{color} | {color:green} hadoop-project in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
17s{color} | {color:green} hadoop-assemblies in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
39s{color} | {color:green} hadoop-yarn-server-timelineservice-hbase in the
patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
27s{color} | {color:green}
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[
https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365996#comment-16365996
]
genericqa commented on YARN-7919:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color}
| {color:red} YARN-7919 does not apply to trunk. Rebase required? Wrong Branch?
See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-7919 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12910775/YARN-7346.03.patch |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/19708/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Split timelineservice-hbase module to make YARN-7346 easier
> ---
>
> Key: YARN-7919
> URL: https://issues.apache.org/jira/browse/YARN-7919
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineservice
>Affects Versions: 3.0.0
>Reporter: Haibo Chen
>Assignee: Haibo Chen
>Priority: Major
> Attachments: YARN-7919.00.patch, YARN-7919.01.patch,
> YARN-7919.02.patch, YARN-7919.03.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[ https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16364466#comment-16364466 ] Rohith Sharma K S commented on YARN-7919: - bq. Did you see some source files in the src/ directory? I didn't see any java file, but I see folders and subfolders i.e src/main/java/org/apache/hadoop/yarn/server/timelineservice/reader folder > Split timelineservice-hbase module to make YARN-7346 easier > --- > > Key: YARN-7919 > URL: https://issues.apache.org/jira/browse/YARN-7919 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineservice >Affects Versions: 3.0.0 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-7919.00.patch, YARN-7919.01.patch, > YARN-7919.02.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[ https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16364460#comment-16364460 ] Haibo Chen commented on YARN-7919: -- Did you see some source files in the src/ directory? I applied the patch to trunk and but did not see src/ locally. I will address your coprocessor naming comment together with reviews that Vrushali may have. > Split timelineservice-hbase module to make YARN-7346 easier > --- > > Key: YARN-7919 > URL: https://issues.apache.org/jira/browse/YARN-7919 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineservice >Affects Versions: 3.0.0 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-7919.00.patch, YARN-7919.01.patch, > YARN-7919.02.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[
https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16363552#comment-16363552
]
Rohith Sharma K S commented on YARN-7919:
-
thanks haibo for the patch! I verified the co-processor jar in cluster and it
seems perfectly fine.
# With current patch, I see below jars in timelineservice folder.
## My view on this is lets NOT keep
{color:red}hadoop-yarn-server-timelineservice-hbase-server-3.2.0-SNAPSHOT-coprocessor.jar{color}
separately, rather we shall keep *co-processor* jar as
{color:green}hadoop-yarn-server-timelineservice-hbase-server-3.2.0-SNAPSHOT.jar{color}
itself that include common dependency as well since this jar is used in hbase
server only.
{code}
hadoop-yarn-server-timelineservice-3.2.0-SNAPSHOT.jar
hadoop-yarn-server-timelineservice-hbase-client-3.2.0-SNAPSHOT.jar
hadoop-yarn-server-timelineservice-hbase-common-3.2.0-SNAPSHOT.jar
hadoop-yarn-server-timelineservice-hbase-server-3.2.0-SNAPSHOT-coprocessor.jar
hadoop-yarn-server-timelineservice-hbase-server-3.2.0-SNAPSHOT.jar
lib
test
{code}
## After your current patch which generates jar with name co-processor, I was
thinking about naming convention for co-processor jar. Should we name it as
hadoop-yarn-server-timelineservice-hbase-coprocessor-3.2.0-SNAPSHOT.jar instead
of hadoop-yarn-server-timelineservice-hbase-server-3.2.0-SNAPSHOT.jar?
# nit : I do see src folder in inside hadoop-yarn-server-timelineservice-hbase
package structure. Is this only to me or others as well?
{code}
hadoop-yarn-server-timelineservice-hbase-client
hadoop-yarn-server-timelineservice-hbase-common
hadoop-yarn-server-timelineservice-hbase-server
hadoop-yarn-server-timelineservice-hbase.iml
pom.xml
src
target
{code}
> Split timelineservice-hbase module to make YARN-7346 easier
> ---
>
> Key: YARN-7919
> URL: https://issues.apache.org/jira/browse/YARN-7919
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineservice
>Affects Versions: 3.0.0
>Reporter: Haibo Chen
>Assignee: Haibo Chen
>Priority: Major
> Attachments: YARN-7919.00.patch, YARN-7919.01.patch,
> YARN-7919.02.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[
https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16363157#comment-16363157
]
genericqa commented on YARN-7919:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
32s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 19 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 18m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 7s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project hadoop-assemblies
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
27s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 16m
17s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
2m 26s{color} | {color:orange} root: The patch generated 1 new + 29 unchanged -
9 fixed = 30 total (was 38) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
10s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 29s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project hadoop-assemblies
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
21s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
23s{color} | {color:green} hadoop-project in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
24s{color} | {color:green} hadoop-assemblies in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
48s{color} | {color:green} hadoop-yarn-server-timelineservice-hbase in the
patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
37s{color} | {color:green}
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[ https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362746#comment-16362746 ] Haibo Chen commented on YARN-7919: -- Yes, I was working on it with the maven-assembly-plugin. Will upload a new patch after I do some tests > Split timelineservice-hbase module to make YARN-7346 easier > --- > > Key: YARN-7919 > URL: https://issues.apache.org/jira/browse/YARN-7919 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineservice >Affects Versions: 3.0.0 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-7919.00.patch, YARN-7919.01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[ https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362740#comment-16362740 ] Rohith Sharma K S commented on YARN-7919: - I had chat with HBase members [~elserj] offline, he suggested to build a fat timelineservice-hbase-server jar which can be used only by HBase co-processor. This suggestion make sense to me. Lets have code base in modular split, but while building timelineservice-hbase-server jar, build it as fat jar. [~haibochen] would you incorporate this change as well so that we don't break anything backward compatible? > Split timelineservice-hbase module to make YARN-7346 easier > --- > > Key: YARN-7919 > URL: https://issues.apache.org/jira/browse/YARN-7919 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineservice >Affects Versions: 3.0.0 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-7919.00.patch, YARN-7919.01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[
https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362143#comment-16362143
]
Rohith Sharma K S commented on YARN-7919:
-
I did testing as well by applying this patch, some of the observations are
# Good news is that table created with old jar in hbase and Hadoop using new
jars perfectly works fine.
# Sad news is that FlowRunTable creation always excepts co-processor jar to be
present yarn.timeline-service.hbase.coprocessor.jar.hdfs.location location.
This seems to be an issue during upgrade since it add that location in table
descriptor. RS start failing with exception
{code:java}
2018-02-13 12:27:06,239 ERROR [RS_OPEN_REGION-10.200.4.200:16020-1]
handler.OpenRegionHandler: Failed open of
region=prod.timelineservice.flowrun,,1518501821250.1e108e74a1f96a1fc799842720d78f4f.,
starting to roll back the global memstore size.
java.io.FileNotFoundException: File
file:/Users/rsharmaks/Cluster/atsv2/hadoop/share/hadoop/yarn/timelineservice/hadoop-yarn-server-timelineservice-hbase-3.1.0-SNAPSHOT.jar
does not exist
{code}
On otherside, this was added to keep in co-processor jar in shared storage
place so that all the RegionServer can load it dynamically. Given the condition
where this jar is located in local filesystem or in hdfs, then we might need to
provide additional steps to upgrade. Otherwise, RS can't be started.
# Co processors are dynamically loaded after fix YARN-6094. But it is allowed
only one jar file. Seems like HBASE-14548 is *NOT* fixed in HBase-1.2.x release
line. But issue is with after refactoring code, we can't add single
co-processor jar into flow run table. I am not sure is there any other way to
achieve this from HBase-1.2.x release lines. Looks like after this refactoring,
dynamic loading of co processor would be a problem. [~vrushalic] do you have
any other way to solve this?
[~haibochen] did you test this patch in fresh cluster by creating a table? Am I
missing any steps for table creation?
Comments
# hadoop-yarn-server-timelineservice-hbase module as src folder structure. This
can be removed.
# FlowScanner#resetState java doc has params which are not in method. Params
cell currentAggOp collectedButNotEmitted can be removed.
> Split timelineservice-hbase module to make YARN-7346 easier
> ---
>
> Key: YARN-7919
> URL: https://issues.apache.org/jira/browse/YARN-7919
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineservice
>Affects Versions: 3.0.0
>Reporter: Haibo Chen
>Assignee: Haibo Chen
>Priority: Major
> Attachments: YARN-7919.00.patch, YARN-7919.01.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[
https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16361404#comment-16361404
]
Haibo Chen commented on YARN-7919:
--
{quote}Given this split is committed, then modifications for conditional
compilation is from hbase-server module only right?
{quote}
Right, we'd have two versions of hbase-server module, one of which is loaded
depending on the active maven profile.
> Split timelineservice-hbase module to make YARN-7346 easier
> ---
>
> Key: YARN-7919
> URL: https://issues.apache.org/jira/browse/YARN-7919
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineservice
>Affects Versions: 3.0.0
>Reporter: Haibo Chen
>Assignee: Haibo Chen
>Priority: Major
> Attachments: YARN-7919.00.patch, YARN-7919.01.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[ https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16361225#comment-16361225 ] Rohith Sharma K S commented on YARN-7919: - Thanks [~haibochen] for detailed explanation about the patch. I skimmed through the patch and overall split make sense to me. cc:/ [~vrushalic] I spoke to one of the Hbase member about loading multiple jars for co-processor. He said that if all dependencies are in classpath then is fine. May be you can also check with your internal folks. I will go through in details tomorrow and also try to test it in real cluster as well. Given this split is committed, then for conditional compilation is from hbase-server module only right? > Split timelineservice-hbase module to make YARN-7346 easier > --- > > Key: YARN-7919 > URL: https://issues.apache.org/jira/browse/YARN-7919 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineservice >Affects Versions: 3.0.0 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-7919.00.patch, YARN-7919.01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[ https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16361031#comment-16361031 ] Rohith Sharma K S commented on YARN-7919: - Let me try compiling this new patch and will provide a feedback. > Split timelineservice-hbase module to make YARN-7346 easier > --- > > Key: YARN-7919 > URL: https://issues.apache.org/jira/browse/YARN-7919 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineservice >Affects Versions: 3.0.0 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-7919.00.patch, YARN-7919.01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[
https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16361010#comment-16361010
]
genericqa commented on YARN-7919:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
29s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 19 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 11s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project hadoop-assemblies
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
20s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 48s{color} | {color:orange} root: The patch generated 1 new + 29 unchanged -
9 fixed = 30 total (was 38) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
8s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
8m 41s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project hadoop-assemblies
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
15s{color} | {color:green} hadoop-project in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
16s{color} | {color:green} hadoop-assemblies in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
35s{color} | {color:green} hadoop-yarn-server-timelineservice-hbase in the
patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
27s{color} | {color:green}
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[ https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16360868#comment-16360868 ] Haibo Chen commented on YARN-7919: -- The latest patch does not apply to branch-3.1, please apply it to trunk (Applying the patch to branch-3.1 failed and left source code in *hadoop-yarn-server-timelineservice-hbase*). I can upload a patch for branch-3.1 separately once the trunk patch is ready. > Split timelineservice-hbase module to make YARN-7346 easier > --- > > Key: YARN-7919 > URL: https://issues.apache.org/jira/browse/YARN-7919 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineservice >Affects Versions: 3.0.0 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-7919.00.patch, YARN-7919.01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[ https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16360839#comment-16360839 ] Haibo Chen commented on YARN-7919: -- I uploaded a new patch for trunk which should address issues resulted from branch-3.1 cut and the checkstyle/license/findbug issues reported by Jenkins. I also broken down this patch into 9 smaller commits at [my personal fork|https://github.com/haibchen/hadoop/tree/YARN-7919] > Split timelineservice-hbase module to make YARN-7346 easier > --- > > Key: YARN-7919 > URL: https://issues.apache.org/jira/browse/YARN-7919 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineservice >Affects Versions: 3.0.0 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-7919.00.patch, YARN-7919.01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[ https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16360830#comment-16360830 ] Haibo Chen commented on YARN-7919: -- Thanks [~rohithsharma] for the quick review! I believe branch-3.1 was cut right after I uploaded the patch, hence the patch failed to apply. I will update the patch to address that. It is split based on what role of the code is with respect to HBase servers. The code that reads/writes to tables is executed in NodeManager as hbase clients, TimelineSchemaCreator.java is also executed as hbase clients, so they two are grouped together in a module. Then there is FlowRunCoprocessor code that runs with Region Servers, so it is separated into a server module. There is some code that both the client and server module depend on, that defines what the tables are like and what value encoding they have. It is extracted to a common module. This is what it looks like with the patch applied. *hadoop-yarn-server-timelineservice-hbase:* * *hadoop-yarn-server-timelineservice-hbase-client* (code that reads and writes to the tables, plus TimelineSchemaCreator.java) * *hadoop-yarn-server-timelineservice-hbase-common* (code that defines tables, columns, column familys and column prefixes) * *hadoop-yarn-server-timelineservice-hbase-server* (FlowRunCoprocessor.java)** The current Tables, Columns and Column Families contain both the schema definition and the code that reads and writes to the tables, so what I did is extracting their read/write methods into a ColumnRWHelper class that goes into the hbase-client module. I did not see source files remaining in *hadoop-yarn-server-timelineservice-hbase,* not sure if it is caused by the error while you were applying the patch. > Split timelineservice-hbase module to make YARN-7346 easier > --- > > Key: YARN-7919 > URL: https://issues.apache.org/jira/browse/YARN-7919 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineservice >Affects Versions: 3.0.0 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-7919.00.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[ https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16360354#comment-16360354 ] Rohith Sharma K S commented on YARN-7919: - thanks [~haibochen] for quick patch. I got an error while applying patch and resolved it. But compilation failed to me in branch-3.1! One thing I observed is the module *hadoop-yarn-server-timelineservice-hbase* is changed packaging value to *pom* which indicate, there will be no jar generated for this module. It contains only submodules. But I see there are some java source files still remain in this package which can be removed. I have two doubts on packaging existing class names # which module holds TimelineSchemaCreator class? # which module holds FlowRunCoprocessor class? > Split timelineservice-hbase module to make YARN-7346 easier > --- > > Key: YARN-7919 > URL: https://issues.apache.org/jira/browse/YARN-7919 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineservice >Affects Versions: 3.0.0 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > Attachments: YARN-7919.00.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7919) Split timelineservice-hbase module to make YARN-7346 easier
[
https://issues.apache.org/jira/browse/YARN-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359634#comment-16359634
]
genericqa commented on YARN-7919:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
26s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 19 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
0s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 55s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project hadoop-assemblies
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
37s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
51s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 58s{color} | {color:orange} root: The patch generated 62 new + 29 unchanged
- 9 fixed = 91 total (was 38) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
9s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 45s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project hadoop-assemblies
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
28s{color} | {color:red} hadoop-yarn-server-timelineservice-hbase-common in the
patch failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
29s{color} | {color:red} hadoop-yarn-server-timelineservice-hbase-client in the
patch failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
28s{color} | {color:red} hadoop-yarn-server-timelineservice-hbase-server in the
patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
22s{color} | {color:green} hadoop-project in the patch passed. {color} |
| {color:green}+1{color} |
