[jira] [Comment Edited] (YARN-6608) Backport all SLS improvements from trunk to branch-2
[ https://issues.apache.org/jira/browse/YARN-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043967#comment-16043967 ] Carlo Curino edited comment on YARN-6608 at 6/9/17 6:09 AM: [~yufeigu], that is a good idea. In the latest patch I: # fixed Java 1.7 compilation issues. # rebased to branch-2 and included all trunk patches on SLS. # created the following list of changes we backport (by comparing trunk and branch-2 edits that hit SLS, and then see if they were completely or partially backported). Here is what I got: b65100c YARN-6547. fcbdecc YARN-6111. 40e6a85 YARN-6560. 101852c YARN-6535. 5cb6e3e YARN-6473. 2ba9903 HADOOP-14400. (partial) 3082552 YARN-6522. 475f933 YARN-6423. de69d6e YARN-6363. 7e075a50 YARN-6365. 6a5516c YARN-5654. cc938e9 YARN-6326. 1598fd3 YARN-6275. b32ffa2 YARN-4779. 5ed63e3 YARN-4899. 3bcfe3a YARN-5706. 209e805 HADOOP-13506. (partial) de01327 MAPREDUCE-6796. (partial) 5877f20 HADOOP-10075. (partial) 2cc841f HADOOP-13678 (partial) 58ed4fa HADOOP-13341. (partial) 96fa0f8 HADOOP-12709 (partial) 3c5c57a HADOOP-13142. (partial) 63e5412 HDFS-9427. (partial) 9c32f87 HDFS-10266. (partial) 0a74610 HADOOP-11393. (partial) 7381550 HADOOP-12857. (partial) 73b9c7b HADOOP-10787. (partial) 18727c6 HADOOP-12133 (partial) c5afe44 HDFS-8349. (partial) 618ba70 MAPREDUCE-6057. (partial) dce8b9c HADOOP-11637. (partial) f990e9d HADOOP-11346. d989ac0 MAPREDUCE-5972. (partial) Overall I think it looks pretty promising. We have some 15 YARN patches that are backported in their entirety, and a pile of HADOOP/MAPREDUCE patches that are partial. Many of those where .sh / pom.xml fixes, or edits of rumen traces. Tests for SLS/Rumen pass on local box, so looks reasonable, but let's see a full YETUS. was (Author: curino): [~yufeigu], that is a good idea. In the latest patch I: # fixed Java 1.7 compilation issues. # rebased to branch-2 and included all trunk patches on SLS. # created the following list of changes we backport (by comparing trunk and branch-2 edits that hit SLS, and then see if they were completely or partially backported). Here is what I got: b65100c YARN-6547. fcbdecc YARN-6111. 40e6a85 YARN-6560. 101852c YARN-6535. 5cb6e3e YARN-6473. 2ba9903 HADOOP-14400. (partial) 3082552 YARN-6522. 475f933 YARN-6423. de69d6e YARN-6363. 7e075a50 YARN-6365. 6a5516c YARN-5654. cc938e9 YARN-6326. 1598fd3 YARN-6275. b32ffa2 YARN-4779. 5ed63e3 YARN-4899. 3bcfe3a YARN-5706. 209e805 HADOOP-13506. (partial) de01327 MAPREDUCE-6796. (partial) 5877f20 HADOOP-10075. (partial) 2cc841f HADOOP-13678 (partial) 58ed4fa HADOOP-13341. (partial) 96fa0f8 HADOOP-12709 (partial) 3c5c57a HADOOP-13142. (partial) 63e5412 HDFS-9427. (partial) 9c32f87 HDFS-10266. (partial) 0a74610 HADOOP-11393. (partial) 7381550 HADOOP-12857. (partial) 73b9c7b HADOOP-10787. (partial) 18727c6 HADOOP-12133 (partial) c5afe44 HDFS-8349. (partial) 618ba70 MAPREDUCE-6057. (partial) dce8b9c HADOOP-11637. (partial) f990e9d HADOOP-11346. d989ac0 MAPREDUCE-5972. (partial) Note that many of the older partial patches were .sh / pom.xml file edits. > Backport all SLS improvements from trunk to branch-2 > > > Key: YARN-6608 > URL: https://issues.apache.org/jira/browse/YARN-6608 > Project: Hadoop YARN > Issue Type: Improvement >Affects Versions: 2.9.0 >Reporter: Carlo Curino >Assignee: Carlo Curino > Attachments: YARN-6608-branch-2.v0.patch, > YARN-6608-branch-2.v1.patch, YARN-6608-branch-2.v2.patch, > YARN-6608-branch-2.v3.patch > > > The SLS has received lots of attention in trunk, but only some of it made it > back to branch-2. This patch is a "raw" fork-lift of the trunk development > from hadoop-tools/hadoop-sls. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-6608) Backport all SLS improvements from trunk to branch-2
[ https://issues.apache.org/jira/browse/YARN-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043967#comment-16043967 ] Carlo Curino edited comment on YARN-6608 at 6/9/17 6:05 AM: [~yufeigu], that is a good idea. In the latest patch I: # fixed Java 1.7 compilation issues. # rebased to branch-2 and included all trunk patches on SLS. # created the following list of changes we backport (by comparing trunk and branch-2 edits that hit SLS, and then see if they were completely or partially backported). Here is what I got: b65100c YARN-6547. fcbdecc YARN-6111. 40e6a85 YARN-6560. 101852c YARN-6535. 5cb6e3e YARN-6473. 2ba9903 HADOOP-14400. (partial) 3082552 YARN-6522. 475f933 YARN-6423. de69d6e YARN-6363. 7e075a50 YARN-6365. 6a5516c YARN-5654. cc938e9 YARN-6326. 1598fd3 YARN-6275. b32ffa2 YARN-4779. 5ed63e3 YARN-4899. 3bcfe3a YARN-5706. 209e805 HADOOP-13506. (partial) de01327 MAPREDUCE-6796. (partial) 5877f20 HADOOP-10075. (partial) 2cc841f HADOOP-13678 (partial) 58ed4fa HADOOP-13341. (partial) 96fa0f8 HADOOP-12709 (partial) 3c5c57a HADOOP-13142. (partial) 63e5412 HDFS-9427. (partial) 9c32f87 HDFS-10266. (partial) 0a74610 HADOOP-11393. (partial) 7381550 HADOOP-12857. (partial) 73b9c7b HADOOP-10787. (partial) 18727c6 HADOOP-12133 (partial) c5afe44 HDFS-8349. (partial) 618ba70 MAPREDUCE-6057. (partial) dce8b9c HADOOP-11637. (partial) f990e9d HADOOP-11346. d989ac0 MAPREDUCE-5972. (partial) Note that many of the older partial patches were .sh / pom.xml file edits. was (Author: curino): [~yufeigu], that is a good idea. In the latest patch I: # fixed Java 1.7 compilation issues. # rebased to branch-2 and included all trunk patches on SLS. # created the following list of changes we backport (by comparing trunk and branch-2 edits that hit SLS, and then see if they were completely or partially backported). Here is what I got: b65100c YARN-6547. fcbdecc YARN-6111. 40e6a85 YARN-6560. 101852c YARN-6535. 5cb6e3e YARN-6473. 2ba9903 HADOOP-14400. (partial) 3082552 YARN-6522. 475f933 YARN-6423. de69d6e YARN-6363. 7e075a50 YARN-6365. 6a5516c YARN-5654. cc938e9 YARN-6326. 1598fd3 YARN-6275. b32ffa2 YARN-4779. 5ed63e3 YARN-4899. 3bcfe3a YARN-5706. 209e805 HADOOP-13506. (partial) de01327 MAPREDUCE-6796. (partial) 5877f20 HADOOP-10075. (partial) 2cc841f HADOOP-13678 (partial) 58ed4fa HADOOP-13341. (partial) 96fa0f8 HADOOP-12709 (partial) 3c5c57a HADOOP-13142. (partial) 63e5412 HDFS-9427. (partial) 9c32f87 HDFS-10266. (partial) 0a74610 HADOOP-11393. (partial) 7381550 HADOOP-12857. (partial) 73b9c7b HADOOP-10787. (partial) 18727c6 HADOOP-12133 (partial) c5afe44 HDFS-8349. (partial) 618ba70 MAPREDUCE-6057. (partial) dce8b9c HADOOP-11637. (partial) b6fc1f3 HADOOP-11593. (partial) f990e9d HADOOP-11346. d989ac0 MAPREDUCE-5972. (partial) 22cbcd1 HADOOP-10717. (partial) bae5837 HADOOP-10717. (partial) Note that many of the older partial patches were .sh / pom.xml file edits. > Backport all SLS improvements from trunk to branch-2 > > > Key: YARN-6608 > URL: https://issues.apache.org/jira/browse/YARN-6608 > Project: Hadoop YARN > Issue Type: Improvement >Affects Versions: 2.9.0 >Reporter: Carlo Curino >Assignee: Carlo Curino > Attachments: YARN-6608-branch-2.v0.patch, > YARN-6608-branch-2.v1.patch, YARN-6608-branch-2.v2.patch, > YARN-6608-branch-2.v3.patch > > > The SLS has received lots of attention in trunk, but only some of it made it > back to branch-2. This patch is a "raw" fork-lift of the trunk development > from hadoop-tools/hadoop-sls. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-6608) Backport all SLS improvements from trunk to branch-2
[ https://issues.apache.org/jira/browse/YARN-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043967#comment-16043967 ] Carlo Curino edited comment on YARN-6608 at 6/9/17 5:54 AM: [~yufeigu], that is a good idea. In the latest patch I: # fixed Java 1.7 compilation issues. # rebased to branch-2 and included all trunk patches on SLS. # created the following list of changes we backport (by comparing trunk and branch-2 edits that hit SLS, and then see if they were completely or partially backported). Here is what I got: b65100c YARN-6547. fcbdecc YARN-6111. 40e6a85 YARN-6560. 101852c YARN-6535. 5cb6e3e YARN-6473. 2ba9903 HADOOP-14400. (partial) 3082552 YARN-6522. 475f933 YARN-6423. de69d6e YARN-6363. 7e075a50 YARN-6365. 6a5516c YARN-5654. cc938e9 YARN-6326. 1598fd3 YARN-6275. b32ffa2 YARN-4779. 5ed63e3 YARN-4899. 3bcfe3a YARN-5706. 209e805 HADOOP-13506. (partial) de01327 MAPREDUCE-6796. (partial) 5877f20 HADOOP-10075. (partial) 2cc841f HADOOP-13678 (partial) 58ed4fa HADOOP-13341. (partial) 96fa0f8 HADOOP-12709 (partial) 3c5c57a HADOOP-13142. (partial) 63e5412 HDFS-9427. (partial) 9c32f87 HDFS-10266. (partial) 0a74610 HADOOP-11393. (partial) 7381550 HADOOP-12857. (partial) 73b9c7b HADOOP-10787. (partial) 18727c6 HADOOP-12133 (partial) c5afe44 HDFS-8349. (partial) 618ba70 MAPREDUCE-6057. (partial) dce8b9c HADOOP-11637. (partial) b6fc1f3 HADOOP-11593. (partial) f990e9d HADOOP-11346. d989ac0 MAPREDUCE-5972. (partial) 22cbcd1 HADOOP-10717. (partial) bae5837 HADOOP-10717. (partial) Note that many of the older partial patches were .sh / pom.xml file edits. was (Author: curino): [~yufeigu], that is a good idea. In the latest patch I: # fixed Java 1.7 compilation issues. # rebased to branch-2 and included all trunk patches on SLS. # created the following list of changes we backport (by comparing trunk and branch-2 edits that hit SLS, and then see if they were completely or partially backported). Here is what I got: b65100c YARN-6547. fcbdecc YARN-6111. 40e6a85 YARN-6560. 101852c YARN-6535. 5cb6e3e YARN-6473. 2ba9903 HADOOP-14400. (partial) 3082552 YARN-6522. 475f933 YARN-6423. de69d6e YARN-6363. 7e075a50 YARN-6365. 6a5516c YARN-5654. cc938e9 YARN-6326. 1598fd3 YARN-6275. b32ffa2 YARN-4779. 5ed63e3 YARN-4899. 3bcfe3a YARN-5706. 209e805 HADOOP-13506. (partial) de01327 MAPREDUCE-6796. (partial) 5877f20 HADOOP-10075. (partial) 2cc841f HADOOP-13678 (partial) 58ed4fa HADOOP-13341. (partial) 96fa0f8 HADOOP-12709 (partial) 3c5c57a HADOOP-13142. (partial) 63e5412 HDFS-9427. (partial) 9c32f87 HDFS-10266. (partial) 0a74610 HADOOP-11393. (partial) 7381550 HADOOP-12857. (partial) 73b9c7b HADOOP-10787. (partial) 18727c6 HADOOP-12133 (partial) c5afe44 HDFS-8349. (partial) 618ba70 MAPREDUCE-6057. (partial) dce8b9c HADOOP-11637. (partial) b6fc1f3 HADOOP-11593. (partial) f990e9d HADOOP-11346. d989ac0 MAPREDUCE-5972. (partial) 22cbcd1 HADOOP-10717. (partial) bae5837 HADOOP-10717. (partial) 28db28d YARN-1311. (partial) Note that many of the older partial patches were .sh / pom.xml file edits. > Backport all SLS improvements from trunk to branch-2 > > > Key: YARN-6608 > URL: https://issues.apache.org/jira/browse/YARN-6608 > Project: Hadoop YARN > Issue Type: Improvement >Affects Versions: 2.9.0 >Reporter: Carlo Curino >Assignee: Carlo Curino > Attachments: YARN-6608-branch-2.v0.patch, > YARN-6608-branch-2.v1.patch, YARN-6608-branch-2.v2.patch, > YARN-6608-branch-2.v3.patch > > > The SLS has received lots of attention in trunk, but only some of it made it > back to branch-2. This patch is a "raw" fork-lift of the trunk development > from hadoop-tools/hadoop-sls. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-6608) Backport all SLS improvements from trunk to branch-2
[ https://issues.apache.org/jira/browse/YARN-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043967#comment-16043967 ] Carlo Curino edited comment on YARN-6608 at 6/9/17 5:53 AM: [~yufeigu], that is a good idea. In the latest patch I: # fixed Java 1.7 compilation issues. # rebased to branch-2 and included all trunk patches on SLS. # created the following list of changes we backport (by comparing trunk and branch-2 edits that hit SLS, and then see if they were completely or partially backported). Here is what I got: b65100c YARN-6547. fcbdecc YARN-6111. 40e6a85 YARN-6560. 101852c YARN-6535. 5cb6e3e YARN-6473. 2ba9903 HADOOP-14400. (partial) 3082552 YARN-6522. 475f933 YARN-6423. de69d6e YARN-6363. 7e075a50 YARN-6365. 6a5516c YARN-5654. cc938e9 YARN-6326. 1598fd3 YARN-6275. b32ffa2 YARN-4779. 5ed63e3 YARN-4899. 3bcfe3a YARN-5706. 209e805 HADOOP-13506. (partial) de01327 MAPREDUCE-6796. (partial) 5877f20 HADOOP-10075. (partial) 2cc841f HADOOP-13678 (partial) 58ed4fa HADOOP-13341. (partial) 96fa0f8 HADOOP-12709 (partial) 3c5c57a HADOOP-13142. (partial) 63e5412 HDFS-9427. (partial) 9c32f87 HDFS-10266. (partial) 0a74610 HADOOP-11393. (partial) 7381550 HADOOP-12857. (partial) 73b9c7b HADOOP-10787. (partial) 18727c6 HADOOP-12133 (partial) c5afe44 HDFS-8349. (partial) 618ba70 MAPREDUCE-6057. (partial) dce8b9c HADOOP-11637. (partial) b6fc1f3 HADOOP-11593. (partial) f990e9d HADOOP-11346. d989ac0 MAPREDUCE-5972. (partial) 22cbcd1 HADOOP-10717. (partial) bae5837 HADOOP-10717. (partial) 28db28d YARN-1311. (partial) Note that many of the older partial patches were .sh / pom.xml file edits. was (Author: curino): [~yufeigu], that is a good idea. In the latest patch I: # fixed Java 1.7 compilation issues. # rebased to branch-2 and included all trunk patches on SLS. # created the following list of changes we backport (by comparing trunk and branch-2 edits that hit SLS, and then see if they were completely or partially backported). Here is what I got: b65100c YARN-6547. fcbdecc YARN-6111. 40e6a85 YARN-6560. 101852c YARN-6535. 5cb6e3e YARN-6473. 2ba9903 HADOOP-14400. (partial) 3082552 YARN-6522. 475f933 YARN-6423. de69d6e YARN-6363. 7e075a50 YARN-6365. 6a5516c YARN-5654. cc938e9 YARN-6326. 1598fd3 YARN-6275. b32ffa2 YARN-4779. 5ed63e3 YARN-4899. 3bcfe3a YARN-5706. 209e805 HADOOP-13506. (partial) de01327 MAPREDUCE-6796. (partial) 5877f20 HADOOP-10075. (partial) 2cc841f HADOOP-13678 (partial) 58ed4fa HADOOP-13341. (partial) 96fa0f8 HADOOP-12709 (partial) 3c5c57a HADOOP-13142. (partial) 63e5412 HDFS-9427. (partial) 9c32f87 HDFS-10266. (partial) 0a74610 HADOOP-11393. (partial) 7381550 HADOOP-12857. (partial) 73b9c7b HADOOP-10787. (partial) 18727c6 HADOOP-12133 (partial) c5afe44 HDFS-8349. (partial) 618ba70 MAPREDUCE-6057. (partial) dce8b9c HADOOP-11637. (partial) b6fc1f3 HADOOP-11593. (partial) f990e9d HADOOP-11346. d989ac0 MAPREDUCE-5972. (partial) 4625792 YARN-1726. b0c2c85 YARN-1726. 22cbcd1 HADOOP-10717. (partial) bae5837 HADOOP-10717. (partial) 28db28d YARN-1311. (partial) Note that many of the older partial patches were .sh / pom.xml file edits. > Backport all SLS improvements from trunk to branch-2 > > > Key: YARN-6608 > URL: https://issues.apache.org/jira/browse/YARN-6608 > Project: Hadoop YARN > Issue Type: Improvement >Affects Versions: 2.9.0 >Reporter: Carlo Curino >Assignee: Carlo Curino > Attachments: YARN-6608-branch-2.v0.patch, > YARN-6608-branch-2.v1.patch, YARN-6608-branch-2.v2.patch, > YARN-6608-branch-2.v3.patch > > > The SLS has received lots of attention in trunk, but only some of it made it > back to branch-2. This patch is a "raw" fork-lift of the trunk development > from hadoop-tools/hadoop-sls. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-6608) Backport all SLS improvements from trunk to branch-2
[ https://issues.apache.org/jira/browse/YARN-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16035613#comment-16035613 ] Yufei Gu edited comment on YARN-6608 at 6/2/17 11:18 PM: - [~curino], that's a fair point. We cannot set "fix version" to branch-2 in the related JIRAs if it is partially backported. Howevern, we can make a list here like this: - YARN- - YARN- (partial) The benefits is that we can decide whether we could backport the future SLS related JIRAs according to the list. It is OK to backport the JIRA to banch-2 if all its dependencies are in. Of course the another option would be to set "fix version" to branch-2 in related JIRAs. was (Author: yufeigu): [~curino], that's a fair point. We cannot set "fix version" to branch-2 in the related JIRAs if it is partially backported. Howevern, we can make a list here like this: - YARN- - YARN- (partial) The benefits is that we can decide whether we could backport the future SLS related JIRAs according to the list. It is OK to backport the JIRA to banch-2 if all its dependencies are in. Of course the another option would be to set "fix version" to branch-2 in related JIRAs. > Backport all SLS improvements from trunk to branch-2 > > > Key: YARN-6608 > URL: https://issues.apache.org/jira/browse/YARN-6608 > Project: Hadoop YARN > Issue Type: Improvement >Affects Versions: 2.9.0 >Reporter: Carlo Curino >Assignee: Carlo Curino > Attachments: YARN-6608-branch-2.v0.patch, > YARN-6608-branch-2.v1.patch, YARN-6608-branch-2.v2.patch > > > The SLS has received lots of attention in trunk, but only some of it made it > back to branch-2. This patch is a "raw" fork-lift of the trunk development > from hadoop-tools/hadoop-sls. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-6608) Backport all SLS improvements from trunk to branch-2
[
https://issues.apache.org/jira/browse/YARN-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012767#comment-16012767
]
Carlo Curino edited comment on YARN-6608 at 5/16/17 5:25 PM:
-
SLS is providing a basis to construct useful integration tests (YARN-6363,
YARN-6451, YARN-6547). This would be nice to have in branch-2 as well.
Per off-line discussion with [~leftnoteasy] and [~chris.douglas], we are
proposing to forklift back to branch-2 all the improvements that SLS received
in trunk.
The attached patch is basically a {{git checkout trunk --
hadoop-tools/hadoop-sls}} plus simple deletion of {{ResourceSchedulerWrapper}}
(not needed anymore
and not compiling). SLS tests pass locally (expect issue tracked in YARN-6111),
let's see what yetus has to say.
was (Author: curino):
SLS is providing a basis to construct useful integration tests (YARN-6363,
YARN-6451, YARN-6547). This would be nice to have in branch-2 as well.
Per off-line discussion with [~leftnoteasy] and [~chris.douglas], we are
proposing to forklift back to branch-2 all the improvements that SLS received
in trunk.
The attached patch is basically a {{git checkout trunk --
hadoop-tools/hadoop-sls}} plus simple deletion of {{ResourceSchedulerWrapper}}
(not needed anymore
and not compiling). SLS tests pass locally (expect issue tracked in
YARN-6), let's see what yetus has to say.
> Backport all SLS improvements from trunk to branch-2
>
>
> Key: YARN-6608
> URL: https://issues.apache.org/jira/browse/YARN-6608
> Project: Hadoop YARN
> Issue Type: Improvement
>Affects Versions: 2.9.0
>Reporter: Carlo Curino
>Assignee: Carlo Curino
>
> The SLS has received lots of attention in trunk, but only some of it made it
> back to branch-2. This patch is a "raw" fork-lift of the trunk development
> from hadoop-tools/hadoop-sls.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
