Thank Zihao! These are really helpful! I just backport them + IMPALA-9572 (to make IMPALA-10257 a clean cherry-pick):
- IMPALA-9952: Fix page index filtering for empty pages - IMPALA-9572: Fix DCHECK in nested Parquet scanning - IMPALA-10257: Relax check for page filtering - IMPALA-10310: Fix couldn't skip rows in parquet file on NextRowGroup - IMPALA-12565: Fix crash triggered by calling pmod() UDF - IMPALA-5476: Fix catalogd restart brings stale metadata Have ran CORE tests before merging them: https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/19977/ Another patch I want to add for each release is the package support (IMPALA-10262). It helps to build DEB/RPM packages to ease users to try Impala. There are some non-trivial conflicts that I resolved. Uploaded the patch for review: https://gerrit.cloudera.org/c/21129/ IMPALA-10057: Fix log spew by using jars in the classpath https://gerrit.cloudera.org/c/21130/ IMPALA-10262: RPM/DEB Packaging Support Note that IMPALA-10057 is needed to reduce some conflicts. Looking for reviewers for them. Thanks, Quanlong On Thu, Mar 7, 2024 at 5:17 PM Zihao Ye <eyiz...@163.com> wrote: > Thanks to Quanlong for the work on this, it's very valuable for those who > can't upgrade their Hive. > > > We backported the following patches to our internal Impala 3.4.0 cluster > and they have been running well in production. These patches are small and > don't seem to be in the 3.4.2 git log right now, so I suggest considering > including these fixes in the upcoming 3.4.2 release: > > > - IMPALA-12565: Fix crash triggered by calling pmod() UDF. > - IMPALA-10310: Fix inability to skip rows in a parquet file on > NextRowGroup. > - IMPALA-9952: Fix page index filtering for empty pages. > - IMPALA-10257: Relax check for page filtering. > - IMPALA-5476: Fix issue where a catalogd restart results in stale > metadata. > > > Thanks, > Zihao > > On 2024/02/08 00:11:19 Quanlong Huang wrote: > Hi all, > > I've created the 3.4.2 branch and submitted a patch for review to fix the > build issue: https://gerrit.cloudera.org/c/20995/ > > Thanks, > Quanlong > > On Mon, Jan 29, 2024 at 10:03 AM Guojing Feng <fg...@gmail.com> wrote: > > volunteer +1, is neccecary to maintain 3.x releases since its very > expensive to update Hive2 to Hive3 for legacy systems. > > Quanlong Huang <hu...@gmail.com> 于2024年1月27日周六 06:59写道： > > Hi all, > > We still have users on the 3.x branch since they can't upgrade their Hive > to newer versions. It's reported that the 3.4.1 release is not compilable > now (IMPALA-12755). It'd be nice to have a buildable release for 3.4. > > I propose doing a maintenance release of 3.4.2 to fix the build issue > and I > volunteer to manage the release of it. It can branch out based on the > HEAD > of 3.4.1. We can also add some fixes (some are already in 3.x): > > - IMPALA-11296: Fix infinite loop when reading orc files > - IMPALA-11444: Fix wrong results when reading wide rows from ORC > - IMPALA-12102: Avoid memory leaks in the handling of JNI exceptions > - IMPALA-11406: Fix incorrect duration log for authorization > > Optional items: > > - Backport RPM/DEB packaging support (IMPALA-10262) so it's easier to > distribute Impala. > - Backport script changes to support building on Ubuntu 18.04 and > 20.04 > (IMPALA-10007). > > I tried a branch like this: > https://github.com/stiga-huang/impala/commits/branch-3.4.2 > It passed the CORE tests: > https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/19973/ > > Please speak up and let the community know your ideas, especially any bug > fixes that you want to backport to 3.4.2. All feedback is welcome! > > PS This is not a VOTE thread but only a DISCUSS thread. > > Thanks, > Quanlong > > > > > > >
