>>For the first patch, "0b540b025 IMPALA-7128 (part 1) Refactor interfaces for Db, View, Table, Partition", the cherry-pick conflicts is due to the revert of IMPALA-6479 in 2.x. I'm testing branch-2.x with IMPALA-6479 being picked back. Does anyone know why we revert it? (I also comment in the JIRA). > >There are test failures. I guess it's the reason. Hopefully, cdh-5.16.1-release already picked up this patch, which provides some pointers :)
I fix the test failures and create a review at https://gerrit.cloudera.org/c/12292/ Waiting for Jenkins maintenance to finish and then run a GVO. Hopes someone can join and have a look! On Tue, Jan 29, 2019 at 7:39 AM Quanlong Huang <huangquanl...@gmail.com> wrote: > >For the first patch, "0b540b025 IMPALA-7128 (part 1) Refactor interfaces > for Db, View, Table, Partition", the cherry-pick conflicts is due to the > revert of IMPALA-6479 in 2.x. I'm testing branch-2.x with IMPALA-6479 being > picked back. Does anyone know why we revert it? (I also comment in the > JIRA). > > There are test failures. I guess it's the reason. Hopefully, > cdh-5.16.1-release already picked up this patch, which provides some > pointers :) > > On Mon, Jan 28, 2019 at 10:51 PM Quanlong Huang <huangquanl...@gmail.com> > wrote: > >> Yes, there are two discussion threads before that are relative to this. >> One for stopping the cherrypick-2.x-and-test jenkins job: >> >> https://lists.apache.org/thread.html/2b4b62d4c07661b27a5203618cb0425a429f6460f2eb505acbcd26c6@%3Cdev.impala.apache.org%3E >> >> The other for removing support for hadoop 2 in master branch: >> https://lists.apache.org/thread.html/49f9b68ed3d6d2c0fdee16a877b259922545e4824e1233479227a657@%3Cdev.impala.apache.org%3E >> >> I'm +1 with the second thread that we only support Hadoop 2 in branch-2.x >> and support Hadoop 3 in the master branch to be more focused. I'm also >> agree with Paul's concern. It's such a dilemma that if we skip some >> commits, things will be harder and harder as we moving forward; if we >> cherry-pick, review, and test the commits one by one, branch-2.x will never >> catch up the master branch, which is an obstacle if someone (like me) wants >> to backport his/her new patch to branch-2.x but waits too long and finally >> fogets details of the patch. >> >> I roughly investigated how other systems deal with multiple branches. The >> efforts to backport a patch could be the same for the original patch. It's >> not a easy go, so the Hive community declares that >> "The decision to port a feature from master to branch-1 is at the >> discretion of the contributor and committer. However no features that break >> backwards compatibility will be accepted on branch-1." >> >> I think it's a chance to understand more parts of Impala by learning and >> backporting the patches, since they have execellent commit messages and >> were strictly reviewed. So I volunteer for the job to move forward the >> branch-2.x. Hopes patch authors could give some pointers when I'm blocked! >> I'll try approach (b) first and switch to (a) when (b) becomes impossible >> after too many commits are skipped. I'll confirm with the author if I think >> a patch should be skipped. >> >> For the first patch, "0b540b025 IMPALA-7128 (part 1) Refactor interfaces >> for Db, View, Table, Partition", the cherry-pick conflicts is due to the >> revert of IMPALA-6479 in 2.x. I'm testing branch-2.x with IMPALA-6479 being >> picked back. Does anyone know why we revert it? (I also comment in the >> JIRA). >> >> On Mon, Jan 28, 2019 at 12:43 PM Philip Zeyliger <phi...@cloudera.com> >> wrote: >> >>> As for Quanlong's question, I think the answer is however the folks who >>> want to do the work prefer to do it. As you noticed in the CDH >>> changelists, >>> Cloudera's distribution has opted for something more like approach (a), >>> choosing to backport individual features. For a while, we were doing >>> automation for cherry-picking things automatically, and it got tedious >>> enough that we decided to turn it off. >>> >>> On Sun, Jan 27, 2019 at 7:37 PM Paul Rogers <prog...@cloudera.com> >>> wrote: >>> >>> > Hi Quanlong, >>> > >>> > Thanks for the suggestion. I wonder if there is a third strategy: >>> > >>> > c) Isolate the Hadoop 2.x/3.x differences into clearly-defined driver >>> > layer so that basically all of 3.x can be applied to the 2.x branch. >>> Said >>> > another way, a single source base can work against either Hadoop 2.x or >>> > 3.x, with the build (C++) or runtime (Java) choosing the proper >>> “driver” >>> > classes. >>> > >>> >>> We had such a layer for a while, where Impala master could be built >>> against >>> either Hadoop3 or Hadoop2. We decided to clean it up in commit >>> e4ae605b083ab536c68552e37ca3c46f6bff4c76. >>> >>> commit e4ae605b083ab536c68552e37ca3c46f6bff4c76 >>> Author: Fredy Wijaya <fwij...@cloudera.com> >>> Date: Thu Jul 12 17:01:13 2018 -0700 >>> >>> IMPALA-7295: Remove IMPALA_MINICLUSTER_PROFILE=2 >>> >>> This patch removes the use of IMPALA_MINICLUSTER_PROFILE. The code >>> that >>> uses IMPALA_MINICLUSTER_PROFILE=2 is removed and it defaults to code >>> from >>> IMPALA_MINICLUSTER_PROFILE=3. In order to reduce having too many code >>> changes in this patch, there is no code change for the shims. The >>> shims >>> for IMPALA_MINICLUSTER_PROFILE=3 automatically become the default >>> implementation. >>> >>> Testing: >>> - Ran core and exhaustive tests >>> >>> Change-Id: Iba4a81165b3d2012dc04d4115454372c41e39f08 >>> Reviewed-on: http://gerrit.cloudera.org:8080/10940 >>> Reviewed-by: Impala Public Jenkins < >>> impala-public-jenk...@cloudera.com> >>> Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com >>> > >>> >>
