Hi all, The progress is slow but finally we merged the packaging support and various bug fixes to branch-3.4.2. Now it's ready to vote. I'll post a vote thread soon.
Thanks, Quanlong On Tue, Apr 9, 2024 at 7:39 PM Quanlong Huang <huangquanl...@gmail.com> wrote: > The packaging support is now merged to branch-3.4.2. While testing the > packages, I found some issues due to differences between the master and the > 3.4.2 branch. > Here are additional patches that we need for branch-3.4.2: > > http://gerrit.cloudera.org:8080/21259 IMPALA-10262: (Addendum) fix > incorrect paths of shared libs in building the ... > http://gerrit.cloudera.org:8080/21260 IMPALA-9845: Point Maven and Ant > downloads to stable locations > http://gerrit.cloudera.org:8080/21261 IMPALA-10052: Expose daemon health > endpoint for statestore and catalog > http://gerrit.cloudera.org:8080/21262 IMPALA-12288: Add > BUILD_WITH_NO_TESTS option to remove test targets > http://gerrit.cloudera.org:8080/21263 IMPALA-12362: (part-1/4) Refactor > service management scripts. > http://gerrit.cloudera.org:8080/21264 IMPALA-12362: (part-2/4) Optimize > default configurations for packaging module. > http://gerrit.cloudera.org:8080/21265 IMPALA-12979: Avoid using wildcard > in CLASSPATH > > Thanks, > Quanlong > > On Sun, Mar 10, 2024 at 8:49 PM Quanlong Huang <huangquanl...@gmail.com> > wrote: > >> Thank Zihao! These are really helpful! >> >> I just backport them + IMPALA-9572 (to make IMPALA-10257 a clean >> cherry-pick): >> >> - IMPALA-9952: Fix page index filtering for empty pages >> - IMPALA-9572: Fix DCHECK in nested Parquet scanning >> - IMPALA-10257: Relax check for page filtering >> - IMPALA-10310: Fix couldn't skip rows in parquet file on NextRowGroup >> - IMPALA-12565: Fix crash triggered by calling pmod() UDF >> - IMPALA-5476: Fix catalogd restart brings stale metadata >> >> Have ran CORE tests before merging them: >> https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/19977/ >> >> Another patch I want to add for each release is the package support >> (IMPALA-10262). It helps to build DEB/RPM packages to ease users to try >> Impala. There are some non-trivial conflicts that I resolved. Uploaded the >> patch for review: >> >> https://gerrit.cloudera.org/c/21129/ IMPALA-10057: Fix log spew by using >> jars in the classpath >> https://gerrit.cloudera.org/c/21130/ IMPALA-10262: RPM/DEB Packaging >> Support >> >> Note that IMPALA-10057 is needed to reduce some conflicts. Looking for >> reviewers for them. >> >> Thanks, >> Quanlong >> >> On Thu, Mar 7, 2024 at 5:17 PM Zihao Ye <eyiz...@163.com> wrote: >> >>> Thanks to Quanlong for the work on this, it's very valuable for those who >>> can't upgrade their Hive. >>> >>> >>> We backported the following patches to our internal Impala 3.4.0 cluster >>> and they have been running well in production. These patches are small >>> and >>> don't seem to be in the 3.4.2 git log right now, so I suggest considering >>> including these fixes in the upcoming 3.4.2 release: >>> >>> >>> - IMPALA-12565: Fix crash triggered by calling pmod() UDF. >>> - IMPALA-10310: Fix inability to skip rows in a parquet file on >>> NextRowGroup. >>> - IMPALA-9952: Fix page index filtering for empty pages. >>> - IMPALA-10257: Relax check for page filtering. >>> - IMPALA-5476: Fix issue where a catalogd restart results in stale >>> metadata. >>> >>> >>> Thanks, >>> Zihao >>> >>> On 2024/02/08 00:11:19 Quanlong Huang wrote: >>> Hi all, >>> >>> I've created the 3.4.2 branch and submitted a patch for review to fix the >>> build issue: https://gerrit.cloudera.org/c/20995/ >>> >>> Thanks, >>> Quanlong >>> >>> On Mon, Jan 29, 2024 at 10:03 AM Guojing Feng <fg...@gmail.com> wrote: >>> >>> volunteer +1, is neccecary to maintain 3.x releases since its very >>> expensive to update Hive2 to Hive3 for legacy systems. >>> >>> Quanlong Huang <hu...@gmail.com> 于2024年1月27日周六 06:59写道： >>> >>> Hi all, >>> >>> We still have users on the 3.x branch since they can't upgrade their Hive >>> to newer versions. It's reported that the 3.4.1 release is not compilable >>> now (IMPALA-12755). It'd be nice to have a buildable release for 3.4. >>> >>> I propose doing a maintenance release of 3.4.2 to fix the build issue >>> and I >>> volunteer to manage the release of it. It can branch out based on the >>> HEAD >>> of 3.4.1. We can also add some fixes (some are already in 3.x): >>> >>> - IMPALA-11296: Fix infinite loop when reading orc files >>> - IMPALA-11444: Fix wrong results when reading wide rows from ORC >>> - IMPALA-12102: Avoid memory leaks in the handling of JNI exceptions >>> - IMPALA-11406: Fix incorrect duration log for authorization >>> >>> Optional items: >>> >>> - Backport RPM/DEB packaging support (IMPALA-10262) so it's easier to >>> distribute Impala. >>> - Backport script changes to support building on Ubuntu 18.04 and >>> 20.04 >>> (IMPALA-10007). >>> >>> I tried a branch like this: >>> https://github.com/stiga-huang/impala/commits/branch-3.4.2 >>> It passed the CORE tests: >>> https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/19973/ >>> >>> Please speak up and let the community know your ideas, especially any bug >>> fixes that you want to backport to 3.4.2. All feedback is welcome! >>> >>> PS This is not a VOTE thread but only a DISCUSS thread. >>> >>> Thanks, >>> Quanlong >>> >>> >>> >>> >>> >>> >>>
