>For the first patch, "0b540b025 IMPALA-7128 (part 1) Refactor interfaces for Db, View, Table, Partition", the cherry-pick conflicts is due to the revert of IMPALA-6479 in 2.x. I'm testing branch-2.x with IMPALA-6479 being picked back. Does anyone know why we revert it? (I also comment in the JIRA).
There are test failures. I guess it's the reason. Hopefully, cdh-5.16.1-release already picked up this patch, which provides some pointers :) On Mon, Jan 28, 2019 at 10:51 PM Quanlong Huang <[email protected]> wrote: > Yes, there are two discussion threads before that are relative to this. > One for stopping the cherrypick-2.x-and-test jenkins job: > > https://lists.apache.org/thread.html/2b4b62d4c07661b27a5203618cb0425a429f6460f2eb505acbcd26c6@%3Cdev.impala.apache.org%3E > > The other for removing support for hadoop 2 in master branch: > https://lists.apache.org/thread.html/49f9b68ed3d6d2c0fdee16a877b259922545e4824e1233479227a657@%3Cdev.impala.apache.org%3E > > I'm +1 with the second thread that we only support Hadoop 2 in branch-2.x > and support Hadoop 3 in the master branch to be more focused. I'm also > agree with Paul's concern. It's such a dilemma that if we skip some > commits, things will be harder and harder as we moving forward; if we > cherry-pick, review, and test the commits one by one, branch-2.x will never > catch up the master branch, which is an obstacle if someone (like me) wants > to backport his/her new patch to branch-2.x but waits too long and finally > fogets details of the patch. > > I roughly investigated how other systems deal with multiple branches. The > efforts to backport a patch could be the same for the original patch. It's > not a easy go, so the Hive community declares that > "The decision to port a feature from master to branch-1 is at the > discretion of the contributor and committer. However no features that break > backwards compatibility will be accepted on branch-1." > > I think it's a chance to understand more parts of Impala by learning and > backporting the patches, since they have execellent commit messages and > were strictly reviewed. So I volunteer for the job to move forward the > branch-2.x. Hopes patch authors could give some pointers when I'm blocked! > I'll try approach (b) first and switch to (a) when (b) becomes impossible > after too many commits are skipped. I'll confirm with the author if I think > a patch should be skipped. > > For the first patch, "0b540b025 IMPALA-7128 (part 1) Refactor interfaces > for Db, View, Table, Partition", the cherry-pick conflicts is due to the > revert of IMPALA-6479 in 2.x. I'm testing branch-2.x with IMPALA-6479 being > picked back. Does anyone know why we revert it? (I also comment in the > JIRA). > > On Mon, Jan 28, 2019 at 12:43 PM Philip Zeyliger <[email protected]> > wrote: > >> As for Quanlong's question, I think the answer is however the folks who >> want to do the work prefer to do it. As you noticed in the CDH >> changelists, >> Cloudera's distribution has opted for something more like approach (a), >> choosing to backport individual features. For a while, we were doing >> automation for cherry-picking things automatically, and it got tedious >> enough that we decided to turn it off. >> >> On Sun, Jan 27, 2019 at 7:37 PM Paul Rogers <[email protected]> wrote: >> >> > Hi Quanlong, >> > >> > Thanks for the suggestion. I wonder if there is a third strategy: >> > >> > c) Isolate the Hadoop 2.x/3.x differences into clearly-defined driver >> > layer so that basically all of 3.x can be applied to the 2.x branch. >> Said >> > another way, a single source base can work against either Hadoop 2.x or >> > 3.x, with the build (C++) or runtime (Java) choosing the proper “driver” >> > classes. >> > >> >> We had such a layer for a while, where Impala master could be built >> against >> either Hadoop3 or Hadoop2. We decided to clean it up in commit >> e4ae605b083ab536c68552e37ca3c46f6bff4c76. >> >> commit e4ae605b083ab536c68552e37ca3c46f6bff4c76 >> Author: Fredy Wijaya <[email protected]> >> Date: Thu Jul 12 17:01:13 2018 -0700 >> >> IMPALA-7295: Remove IMPALA_MINICLUSTER_PROFILE=2 >> >> This patch removes the use of IMPALA_MINICLUSTER_PROFILE. The code >> that >> uses IMPALA_MINICLUSTER_PROFILE=2 is removed and it defaults to code >> from >> IMPALA_MINICLUSTER_PROFILE=3. In order to reduce having too many code >> changes in this patch, there is no code change for the shims. The >> shims >> for IMPALA_MINICLUSTER_PROFILE=3 automatically become the default >> implementation. >> >> Testing: >> - Ran core and exhaustive tests >> >> Change-Id: Iba4a81165b3d2012dc04d4115454372c41e39f08 >> Reviewed-on: http://gerrit.cloudera.org:8080/10940 >> Reviewed-by: Impala Public Jenkins < >> [email protected]> >> Tested-by: Impala Public Jenkins <[email protected]> >> >
