On Mon, Jan 28, 2019 at 11:36 PM Quanlong Huang <huangquanl...@gmail.com> wrote:
> >>For the first patch, "0b540b025 IMPALA-7128 (part 1) Refactor interfaces > for Db, View, Table, Partition", the cherry-pick conflicts is due to the > revert of IMPALA-6479 in 2.x. I'm testing branch-2.x with IMPALA-6479 being > picked back. Does anyone know why we revert it? (I also comment in the > JIRA). > > > >There are test failures. I guess it's the reason. Hopefully, > cdh-5.16.1-release already picked up this patch, which provides some > pointers :) > > I fix the test failures and create a review at > https://gerrit.cloudera.org/c/12292/ > Waiting for Jenkins maintenance to finish and then run a GVO. Hopes someone > can join and have a look! > > On Tue, Jan 29, 2019 at 7:39 AM Quanlong Huang <huangquanl...@gmail.com> > wrote: > > > >For the first patch, "0b540b025 IMPALA-7128 (part 1) Refactor interfaces > > for Db, View, Table, Partition", the cherry-pick conflicts is due to the > > revert of IMPALA-6479 in 2.x. I'm testing branch-2.x with IMPALA-6479 > being > > picked back. Does anyone know why we revert it? (I also comment in the > > JIRA). > It was reverted because it is a compatibility breaking change. We typically try not to introduce such behavioral changes in the same major version line as that can cause upgrade issues. > > > > There are test failures. I guess it's the reason. Hopefully, > > cdh-5.16.1-release already picked up this patch, which provides some > > pointers :) > I work at Cloudera and we've gone through this exercise before. It is annoying to resolve the conflicts, so you can reuse our work and save some time. https://github.com/cloudera/Impala/tree/cdh5-2.12.0_5.16.0 > > > > On Mon, Jan 28, 2019 at 10:51 PM Quanlong Huang <huangquanl...@gmail.com > > > > wrote: > > > >> Yes, there are two discussion threads before that are relative to this. > >> One for stopping the cherrypick-2.x-and-test jenkins job: > >> > >> > https://lists.apache.org/thread.html/2b4b62d4c07661b27a5203618cb0425a429f6460f2eb505acbcd26c6@%3Cdev.impala.apache.org%3E > >> > >> The other for removing support for hadoop 2 in master branch: > >> > https://lists.apache.org/thread.html/49f9b68ed3d6d2c0fdee16a877b259922545e4824e1233479227a657@%3Cdev.impala.apache.org%3E > >> > >> I'm +1 with the second thread that we only support Hadoop 2 in > branch-2.x > >> and support Hadoop 3 in the master branch to be more focused. I'm also > >> agree with Paul's concern. It's such a dilemma that if we skip some > >> commits, things will be harder and harder as we moving forward; if we > >> cherry-pick, review, and test the commits one by one, branch-2.x will > never > >> catch up the master branch, which is an obstacle if someone (like me) > wants > >> to backport his/her new patch to branch-2.x but waits too long and > finally > >> fogets details of the patch. > >> > >> I roughly investigated how other systems deal with multiple branches. > The > >> efforts to backport a patch could be the same for the original patch. > It's > >> not a easy go, so the Hive community declares that > >> "The decision to port a feature from master to branch-1 is at the > >> discretion of the contributor and committer. However no features that > break > >> backwards compatibility will be accepted on branch-1." > >> > >> I think it's a chance to understand more parts of Impala by learning and > >> backporting the patches, since they have execellent commit messages and > >> were strictly reviewed. So I volunteer for the job to move forward the > >> branch-2.x. Hopes patch authors could give some pointers when I'm > blocked! > >> I'll try approach (b) first and switch to (a) when (b) becomes > impossible > >> after too many commits are skipped. I'll confirm with the author if I > think > >> a patch should be skipped. > >> > >> For the first patch, "0b540b025 IMPALA-7128 (part 1) Refactor interfaces > >> for Db, View, Table, Partition", the cherry-pick conflicts is due to the > >> revert of IMPALA-6479 in 2.x. I'm testing branch-2.x with IMPALA-6479 > being > >> picked back. Does anyone know why we revert it? (I also comment in the > >> JIRA). > >> > >> On Mon, Jan 28, 2019 at 12:43 PM Philip Zeyliger <phi...@cloudera.com> > >> wrote: > >> > >>> As for Quanlong's question, I think the answer is however the folks who > >>> want to do the work prefer to do it. As you noticed in the CDH > >>> changelists, > >>> Cloudera's distribution has opted for something more like approach (a), > >>> choosing to backport individual features. For a while, we were doing > >>> automation for cherry-picking things automatically, and it got tedious > >>> enough that we decided to turn it off. > >>> > >>> On Sun, Jan 27, 2019 at 7:37 PM Paul Rogers <prog...@cloudera.com> > >>> wrote: > >>> > >>> > Hi Quanlong, > >>> > > >>> > Thanks for the suggestion. I wonder if there is a third strategy: > >>> > > >>> > c) Isolate the Hadoop 2.x/3.x differences into clearly-defined driver > >>> > layer so that basically all of 3.x can be applied to the 2.x branch. > >>> Said > >>> > another way, a single source base can work against either Hadoop 2.x > or > >>> > 3.x, with the build (C++) or runtime (Java) choosing the proper > >>> “driver” > >>> > classes. > >>> > > >>> > >>> We had such a layer for a while, where Impala master could be built > >>> against > >>> either Hadoop3 or Hadoop2. We decided to clean it up in commit > >>> e4ae605b083ab536c68552e37ca3c46f6bff4c76. > >>> > >>> commit e4ae605b083ab536c68552e37ca3c46f6bff4c76 > >>> Author: Fredy Wijaya <fwij...@cloudera.com> > >>> Date: Thu Jul 12 17:01:13 2018 -0700 > >>> > >>> IMPALA-7295: Remove IMPALA_MINICLUSTER_PROFILE=2 > >>> > >>> This patch removes the use of IMPALA_MINICLUSTER_PROFILE. The code > >>> that > >>> uses IMPALA_MINICLUSTER_PROFILE=2 is removed and it defaults to > code > >>> from > >>> IMPALA_MINICLUSTER_PROFILE=3. In order to reduce having too many > code > >>> changes in this patch, there is no code change for the shims. The > >>> shims > >>> for IMPALA_MINICLUSTER_PROFILE=3 automatically become the default > >>> implementation. > >>> > >>> Testing: > >>> - Ran core and exhaustive tests > >>> > >>> Change-Id: Iba4a81165b3d2012dc04d4115454372c41e39f08 > >>> Reviewed-on: http://gerrit.cloudera.org:8080/10940 > >>> Reviewed-by: Impala Public Jenkins < > >>> impala-public-jenk...@cloudera.com> > >>> Tested-by: Impala Public Jenkins < > impala-public-jenk...@cloudera.com > >>> > > >>> > >> >
