This is an automated email from the ASF dual-hosted git repository.
zhouyuan pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/gluten.git
The following commit(s) were added to refs/heads/main by this push:
new acead70e5e [GLUTEN-6887][VL] Daily Update Velox Version (2026_11_10)
(#13612)
acead70e5e is described below
commit acead70e5eb0f995b39eb5487df7fc6078d3fdf2
Author: Gluten Performance Bot
<[email protected]>
AuthorDate: Wed Nov 11 21:33:33 2026 +0000
[GLUTEN-6887][VL] Daily Update Velox Version (2026_11_10) (#13612)
Upstream Velox's New Commits:
b0538b7662 by XiaoDu, feat(nimble): Add a weighted cluster index with
hierarchical keys (#20024)
5b9931ffa4 by Paul Meng, fix(vector): Slice struct children within their
own size
7cea6ca68c by XiaoDu, refactor(common): Remove identity projection alias
(#20222)
cb87150765 by tanjialiang, refactor(nimble): Hold MetadataBuffer bytes in
PinnedBuffer (#20226)
97a183288d by Maria Basmanova, fix(exec): Mark partial TopNRowNumber steps
explicitly (#20219)
13ed8eb28e by Apurva Kumar, fix(vector): Skip unprojected row children in
encodedVectorCopy
da49491784 by Maria Basmanova, fix(build): Fix the Clang builds
94ccb86a70 by Pedro Pedreira, refactor(expression): Add
ITypedExpr::withInputs
b3abcb448e by Shruti Shivakumar, fix(cudf): Use precomputed function to
cast to 32-bit INTEGER
4f1d8564a2 by Ke, refactor(nimble): Name the fused dictionary read gate
(#20164)
3557fb4755 by Maria Basmanova, feat(exec): Push dynamic filters from vector
search join (#20206)
e4d3d21c6f by Raymond Lin, perf(memory): Make the shared iota table size
configurable (#20031)
8540c64ac7 by Sutou Kouhei, refactor(parquet): Replace Arrow bit utilities
removed in Arrow 26
ebb4e14f6f by Devavret Makkar, feat(cudf): Pass the session QueryConfig to
cuDF functions
2c05f5761b by Matt Gara, fix(cudf): Gate Hive remaining filters for GPU
scans
49f3d32c0e by Richard Barnes, refactor: Use [[likely]] / [[unlikely]]
instead of FOLLY_LIKELY (3/4)
c8ee35cfcd by PRASHANT GOLASH, feat(nimble): Configure FSST decode cache in
Hive scans (#20054)
f251616bb7 by Ke, test(nimble): Cover IS NULL in dictionary-aware read
tests (#20153)
2053f51045 by Maria Basmanova, feat(exec): Support vector indexes on either
join side (#20204)
eeadbff939 by Ke, feat(nimble): Count fuzzer dictionary-aware reads by
filter
cb1a6ae426 by Maria Basmanova, refactor(exec): Clean up RowContainer and
HashTable (#20192)
fa5d6a64ed by PRASHANT GOLASH, test(nimble): Export FSST fixtures for
Presto SQL (#20053)
4cb5ff0eed by Xiaoxuan, Keep one point-lookup iterator per Nimble file
0137ea4386 by Xiaoxuan, perf(nimble): Share stream readers across row
iterators (#20154)
55786e4c44 by joshkang97, feat(row): Allow null rows in CompactRow
deserialization (#20140)
42bca31969 by Ke, test(nimble): Fuzz point lookups through chunk stats
(#20122)
7a876d3dbe by XiaoDu, refactor(nimble): Encode stream-major metadata arrays
with a class (#19946)
7cd676e6b9 by XiaoDu, refactor(nimble): Look up stream offset and size
together (#19944)
a0268765f6 by Maria Basmanova, refactor: Move dynamic filter deduplication
into Driver
a805658108 by tanjialiang, fix(nimble): Accept direct-I/O reads past end of
file (#19986)
5833dc7454 by Maria Basmanova, test: Modernize VectorHasherTest
3f0ad88237 by Maria Basmanova, docs: Limit trailing commas to initializer
lists and enums
5e23f2b0a4 by shanhao-203, perf(nimble): Share Hybrid FlatMap groups across
copies (#20034)
3478a2b0a0 by Maria Basmanova, fix: Store vector index aggregate embeddings
in chunks
e598d39eba by Zhu, Lipeng, perf(functions): Force-inline per-row raw input
in simple aggregates
59fd9c7af6 by XiaoDu, perf(nimble): Share encoding views across
deduplicated stripe-group arrays (#19796)
9b08902ea5 by Zhenyuan Zhao, feat(iceberg): Let a delete file add its own
fileReadOps
3a0f518df7 by Suryadev Sahadevan Rajesh, refactor(nimble): Share loaded
selective reader metadata (#20137)
dc88c915e3 by GALI PREM SAGAR, feat(cudf): Add device primitives for the
Presto page layout
dba0a66a3b by PRASHANT GOLASH, perf(nimble): Prefetch stripe I/O; decode on
demand (#20114)
6540854601 by Ke, test(nimble): Log and widen chunk-stats fuzzer coverage
(#20121)
a921413efd by Rong Ma, test(arrow): Derive ArrowBridgeArrayExportTest from
VectorTestBase
81a75473be by dependabot[bot], build(deps): Bump colord from 2.9.3 to
2.10.0 in /website
f912446bf3 by dependabot[bot], build(deps): Bump svgo from 3.3.3 to 3.3.5
in /website
43ac65f20c by dependabot[bot], build(deps): Bump js-yaml from 3.15.1 to
3.15.2 in /website
df39fd9fbe by RindsSchei225e, refactor(nimble): Move ICompressor into its
own header (#20074)
f5fcc3d142 by RindsSchei225e, refactor(nimble): Move the batch reader out
of fb/ (#20073)
a5a7f94b46 by RindsSchei225e, refactor(nimble): Separate the writer
metadata hook header (#20072)
6fc1fbcc31 by Maria Basmanova, feat: Train vector index codecs with FAISS
k-means
8257715bbb by yun.wu, feat(hive): Add read range columns to the file
connector (#20139)
c666f2863a by Xiaoxuan, refactor(nimble): Rename the projection's barrier
stream mask
aab2fb9499 by xiaodouchen, fix(ucx): Stop test publisher before task abort
ac149631a3 by xiaodouchen, fix(ucx): Prevent early broadcast readers from
hanging
b91e3c3c50 by Richard Barnes, refactor(test): Use std::identity instead of
folly::identity in MapTest
de364f9a44 by Maria Basmanova, fix: Default ivf.nprobe to scanning all
lists (#20147)
e87e0ea705 by rdtr, fix: Throw on Spark ceil and floor overflow in ANSI mode
47f6b11127 by XiaoDu, refactor(tablet): Write the streams of a stripe in
one loop (#20143)
9d24c3add9 by Reema, feat(cudf): Support rank and dense_rank in
TopNRowNumber
196a0d7fd3 by Rong Ma, fix: Zero the inline part of an empty StringView
638c83ee36 by Xiaoxuan, feat(nimble): Deserialize dedup arrays and maps
(#20145)
2e0e2e92dc by Richard Barnes, refactor: Use std::identity instead of
folly::identity
c614ca9de5 by Xiaoxuan, fix(nimble): Project dedup values under flat-map
keys (#20138)
e3c6d6b0b2 by bowen.yang, fix(spark): Fix conv() result for powers of the
target base (#20116)
---
.github/workflows/build_bundle_package.yml | 4 +-
.github/workflows/ch_code_style.yml | 4 +-
.github/workflows/code_format.yml | 4 +-
.github/workflows/delta_spark_ut.yml | 10 ++--
.github/workflows/docker_image.yml | 14 +++---
.github/workflows/flink.yml | 2 +-
.github/workflows/iceberg_spark_ut.yml | 4 +-
.github/workflows/iceberg_upstream_revision.yml | 4 +-
.github/workflows/velox_backend_ansi.yml | 16 +++----
.github/workflows/velox_backend_arm.yml | 2 +-
.github/workflows/velox_backend_cache.yml | 10 ++--
.github/workflows/velox_backend_enhanced.yml | 10 ++--
.github/workflows/velox_backend_x86.yml | 58 ++++++++++++------------
.github/workflows/velox_kafka.yml | 4 +-
.github/workflows/velox_native_build.yml | 2 +-
.github/workflows/velox_nightly.yml | 28 ++++++------
.github/workflows/velox_weekly.yml | 4 +-
cpp/velox/cudf/CudfFunctions.cc | 7 +--
cpp/velox/operators/hashjoin/HashTableBuilder.cc | 42 ++++++-----------
cpp/velox/substrait/SubstraitToVeloxPlan.cc | 1 +
cpp/velox/tests/CudfFunctionsTest.cc | 3 +-
ep/build-velox/src/get-velox.sh | 4 +-
22 files changed, 114 insertions(+), 123 deletions(-)
diff --git a/.github/workflows/build_bundle_package.yml
b/.github/workflows/build_bundle_package.yml
index 1e592f054d..676bb223d8 100644
--- a/.github/workflows/build_bundle_package.yml
+++ b/.github/workflows/build_bundle_package.yml
@@ -41,7 +41,7 @@ on:
jobs:
build-native-lib:
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
container: apache/gluten:vcpkg-almalinux-8-gcc13
steps:
- uses: actions/checkout@v7
@@ -77,7 +77,7 @@ jobs:
build-bundle-package-centos8:
needs: build-native-lib
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
container: centos:8
steps:
- uses: actions/checkout@v7
diff --git a/.github/workflows/ch_code_style.yml
b/.github/workflows/ch_code_style.yml
index 98854aeb3e..7209118c78 100644
--- a/.github/workflows/ch_code_style.yml
+++ b/.github/workflows/ch_code_style.yml
@@ -46,13 +46,13 @@ jobs:
ch-CMake-format-check:
name: CH CMake Format Check
runs-on: ubuntu-latest
- container: ubuntu:22.04
+ container: ubuntu:24.04
steps:
- name: Install tools
run: |
apt update -y
apt install git python3-pip -y
- pip3 install --user cmake-format
+ pip3 install --user --break-system-packages cmake-format
echo "$HOME/.local/bin" >> "$GITHUB_PATH"
- uses: actions/checkout@v7
- name: Check CMake format
diff --git a/.github/workflows/code_format.yml
b/.github/workflows/code_format.yml
index 93584a0715..369efcb0d8 100644
--- a/.github/workflows/code_format.yml
+++ b/.github/workflows/code_format.yml
@@ -31,14 +31,14 @@ jobs:
format-check:
runs-on: ubuntu-latest
- container: ubuntu:22.04
+ container: ubuntu:24.04
steps:
- name: Install tools
run: |
apt update -y
apt install git python3-pip -y
apt install --no-install-recommends -y clang-format-15
- pip3 install cmake_format black pyyaml regex
+ pip3 install --break-system-packages cmake_format black pyyaml regex
update-alternatives --install /usr/bin/clang-format clang-format
"$(command -v clang-format-15)" 15
- uses: actions/checkout@v7
with:
diff --git a/.github/workflows/delta_spark_ut.yml
b/.github/workflows/delta_spark_ut.yml
index 3917605354..b3ccf947e2 100644
--- a/.github/workflows/delta_spark_ut.yml
+++ b/.github/workflows/delta_spark_ut.yml
@@ -181,7 +181,7 @@ jobs:
startsWith(github.event.comment.body, '/delta-test ') ||
startsWith(github.event.comment.body, format('/delta-test{0}',
fromJSON('"\r"'))) ||
startsWith(github.event.comment.body, format('/delta-test{0}',
fromJSON('"\n"')))))
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
permissions:
pull-requests: write
steps:
@@ -203,7 +203,7 @@ jobs:
needs: delta-test-requested
# This workflow always builds its own native lib -- there is no caller to
# provide one, and the `paths:` filter already decided whether the run
happens.
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
container: apache/gluten:vcpkg-almalinux-8-gcc13
steps:
- name: Install CI tools (temporary - until image is rebuilt with
gh/jq/bison/python3.9)
@@ -265,7 +265,7 @@ jobs:
build-gluten-bundle:
needs: build-native-lib-centos-8
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
container: apache/gluten:centos-9-jdk17
steps:
- uses: actions/checkout@v7
@@ -349,7 +349,7 @@ jobs:
# the matrix bounded to a genuinely successful bundle, and stays correct
if the
# upstream jobs' conditions change again.
if: ${{ !cancelled() && needs.build-gluten-bundle.result == 'success' }}
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
container: apache/gluten:centos-9-jdk17
# 350-min safety cap. With 4 forks per shard the per-shard suites run
# 4-at-a-time, so a shard finishes well under this.
@@ -569,7 +569,7 @@ jobs:
# skipped because the bundle build failed -- so a single upstream failure
# doesn't surface as a second, misleading red job.
if: ${{ !cancelled() && needs.delta-spark-test.result != 'skipped' }}
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v7
with:
diff --git a/.github/workflows/docker_image.yml
b/.github/workflows/docker_image.yml
index d87ab1b809..15d3715690 100644
--- a/.github/workflows/docker_image.yml
+++ b/.github/workflows/docker_image.yml
@@ -35,7 +35,7 @@ env:
jobs:
build-ubuntu22-runtime:
if: ${{ startsWith(github.repository, 'apache/') }}
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
steps:
- name: Checkout repository
@@ -60,7 +60,7 @@ jobs:
build-centos-8-runtime:
if: ${{ startsWith(github.repository, 'apache/') }}
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
steps:
- name: Checkout repository
@@ -86,7 +86,7 @@ jobs:
# arm64 only: the image is used by the ARM TPC job, and nothing runs it on
x86.
build-centos-9-runtime:
if: ${{ startsWith(github.repository, 'apache/') }}
- runs-on: ubuntu-22.04-arm
+ runs-on: ubuntu-24.04-arm
steps:
- name: Checkout repository
@@ -111,7 +111,7 @@ jobs:
build-centos-9-jdk17-cuda13_1-cudf:
if: ${{ startsWith(github.repository, 'apache/') }}
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
steps:
- name: "node-cleanup" # by default the free runner does not have enough
disk space
@@ -145,7 +145,7 @@ jobs:
runs-on: ${{ matrix.os }}
strategy:
matrix:
- os: [ ubuntu-22.04, ubuntu-22.04-arm ]
+ os: [ ubuntu-24.04, ubuntu-24.04-arm ]
steps:
- name: Checkout repository
@@ -259,12 +259,12 @@ jobs:
runs-on: ${{ matrix.os }}
strategy:
matrix:
- os: [ ubuntu-22.04, ubuntu-22.04-arm ]
+ os: [ ubuntu-24.04, ubuntu-24.04-arm ]
jdk: [jdk8, jdk17]
# Only centos-9-jdk8 is needed on arm64: the ARM cpp-test-udf-test job
and the default
# dev container (Apple-silicon Macs) use it.
exclude:
- - os: ubuntu-22.04-arm
+ - os: ubuntu-24.04-arm
jdk: jdk17
env:
JDK_VERSION: 1.8.0
diff --git a/.github/workflows/flink.yml b/.github/workflows/flink.yml
index 87df3910a0..be7cd62559 100644
--- a/.github/workflows/flink.yml
+++ b/.github/workflows/flink.yml
@@ -58,7 +58,7 @@ env:
jobs:
flink-test:
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
container: apache/gluten:centos-8-jdk17
steps:
- uses: actions/checkout@v7
diff --git a/.github/workflows/iceberg_spark_ut.yml
b/.github/workflows/iceberg_spark_ut.yml
index f7344d58e1..6f8c8ec1eb 100644
--- a/.github/workflows/iceberg_spark_ut.yml
+++ b/.github/workflows/iceberg_spark_ut.yml
@@ -47,7 +47,7 @@ env:
# The head revision reuses the static native build of velox_backend_x86.yml
when called from it.
jobs:
prepare:
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
outputs:
base_sha: ${{ steps.baseline.outputs.base_sha }}
baseline_key: ${{ steps.baseline.outputs.baseline_key }}
@@ -121,7 +121,7 @@ jobs:
name: iceberg-upstream-gate (3.5)
needs: [prepare, iceberg-upstream-test, iceberg-base-test]
if: always() && !cancelled()
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v7
with:
diff --git a/.github/workflows/iceberg_upstream_revision.yml
b/.github/workflows/iceberg_upstream_revision.yml
index ca71e0efab..4dc933ab52 100644
--- a/.github/workflows/iceberg_upstream_revision.yml
+++ b/.github/workflows/iceberg_upstream_revision.yml
@@ -56,7 +56,7 @@ jobs:
build:
needs: native
if: ${{ !cancelled() && (needs.native.result == 'success' ||
needs.native.result == 'skipped') }}
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
container: ${{ inputs.test_image }}
env:
JAVA_HOME: /usr/lib/jvm/java-17-openjdk
@@ -103,7 +103,7 @@ jobs:
name: iceberg-upstream-test (3.5, ${{ inputs.label }}, ${{ matrix.shard }})
needs: build
if: always() && !cancelled() && needs.build.result == 'success'
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
container: ${{ inputs.test_image }}
strategy:
fail-fast: false
diff --git a/.github/workflows/velox_backend_ansi.yml
b/.github/workflows/velox_backend_ansi.yml
index 9a254da175..05dac2cda1 100644
--- a/.github/workflows/velox_backend_ansi.yml
+++ b/.github/workflows/velox_backend_ansi.yml
@@ -65,7 +65,7 @@ jobs:
(github.event.issue.pull_request &&
(contains(github.event.comment.body, '/ansi-test') ||
contains(github.event.comment.body, '/ansi-analyze')))
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
outputs:
pr_number: ${{ steps.pr-info.outputs.pr_number }}
pr_sha: ${{ steps.pr-info.outputs.pr_sha }}
@@ -111,7 +111,7 @@ jobs:
build-native-lib:
needs: check-comment
if: (inputs.mode || needs.check-comment.outputs.mode) != 'analyze-only'
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v7
with:
@@ -152,7 +152,7 @@ jobs:
spark-test-backends-velox-ansi41:
# Lists check-comment directly so that needs.check-comment.outputs.pr_sha
is set.
needs: [check-comment, build-native-lib]
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
env:
SPARK_TESTING: true
container: apache/gluten:centos-9-jdk17
@@ -221,7 +221,7 @@ jobs:
spark-test-spark-ut-ansi41:
# Lists check-comment directly so that needs.check-comment.outputs.pr_sha
is set.
needs: [check-comment, build-native-lib]
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
env:
SPARK_TESTING: true
container: apache/gluten:centos-9-jdk17
@@ -300,7 +300,7 @@ jobs:
spark-test-backends-velox-ansi-spark40:
# Lists check-comment directly so that needs.check-comment.outputs.pr_sha
is set.
needs: [check-comment, build-native-lib]
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
env:
SPARK_TESTING: true
container: apache/gluten:centos-9-jdk17
@@ -369,7 +369,7 @@ jobs:
spark-test-spark-ut-ansi-spark40:
# Lists check-comment directly so that needs.check-comment.outputs.pr_sha
is set.
needs: [check-comment, build-native-lib]
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
env:
SPARK_TESTING: true
container: apache/gluten:centos-9-jdk17
@@ -453,7 +453,7 @@ jobs:
- spark-test-backends-velox-ansi-spark40
- spark-test-spark-ut-ansi-spark40
if: always() && needs.check-comment.result == 'success' && (inputs.mode ||
needs.check-comment.outputs.mode) != 'analyze-only'
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
permissions:
contents: read
pull-requests: write
@@ -498,7 +498,7 @@ jobs:
analyze-only:
needs: check-comment
if: (inputs.mode || needs.check-comment.outputs.mode) == 'analyze-only'
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
permissions:
contents: read
pull-requests: write
diff --git a/.github/workflows/velox_backend_arm.yml
b/.github/workflows/velox_backend_arm.yml
index fcdec8ca26..abf7114891 100644
--- a/.github/workflows/velox_backend_arm.yml
+++ b/.github/workflows/velox_backend_arm.yml
@@ -55,7 +55,7 @@ jobs:
# Detect which parts of the codebase were modified so downstream jobs can
# skip work that is irrelevant to the change.
detect-changes:
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
outputs:
cpp: ${{ steps.filter.outputs.cpp }}
java: ${{ steps.filter.outputs.java }}
diff --git a/.github/workflows/velox_backend_cache.yml
b/.github/workflows/velox_backend_cache.yml
index b4e3d2169c..29f88422d7 100644
--- a/.github/workflows/velox_backend_cache.yml
+++ b/.github/workflows/velox_backend_cache.yml
@@ -36,7 +36,7 @@ concurrency:
jobs:
cache-native-lib-centos-8-static:
if: ${{ startsWith(github.repository, 'apache/') }}
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
container: apache/gluten:vcpkg-almalinux-8-gcc13
steps:
- uses: actions/checkout@v7
@@ -71,7 +71,7 @@ jobs:
container: apache/gluten:vcpkg-centos-9
strategy:
matrix:
- os: [ ubuntu-22.04, ubuntu-24.04-arm ]
+ os: [ ubuntu-24.04, ubuntu-24.04-arm ]
steps:
- uses: actions/checkout@v7
- name: Get Ccache
@@ -103,7 +103,7 @@ jobs:
runs-on: ${{ matrix.os }}
strategy:
matrix:
- os: [ ubuntu-22.04, ubuntu-24.04-arm ]
+ os: [ ubuntu-24.04, ubuntu-24.04-arm ]
container: apache/gluten:centos-9-jdk8
steps:
- uses: actions/checkout@v7
@@ -135,7 +135,7 @@ jobs:
cache-enhanced-native-lib-centos-8-static:
if: ${{ startsWith(github.repository, 'apache/') }}
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
container: apache/gluten:vcpkg-almalinux-8-gcc13
steps:
- uses: actions/checkout@v7
@@ -169,7 +169,7 @@ jobs:
runs-on: ${{ matrix.os }}
strategy:
matrix:
- os: [ ubuntu-22.04 ]
+ os: [ ubuntu-24.04 ]
steps:
- name: "node-cleanup" # by default the free runner does not have enough
disk space
run: |
diff --git a/.github/workflows/velox_backend_enhanced.yml
b/.github/workflows/velox_backend_enhanced.yml
index 00d5da022b..e58d022ccd 100644
--- a/.github/workflows/velox_backend_enhanced.yml
+++ b/.github/workflows/velox_backend_enhanced.yml
@@ -43,7 +43,7 @@ jobs:
# Detect which parts of the codebase were modified so downstream jobs can
# skip work that is irrelevant to the change.
detect-changes:
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
outputs:
cpp: ${{ steps.filter.outputs.cpp }}
java: ${{ steps.filter.outputs.java }}
@@ -77,7 +77,7 @@ jobs:
build-native-lib-centos-8:
needs: detect-changes
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
container: apache/gluten:vcpkg-almalinux-8-gcc13
steps:
- name: Install CI tools (temporary - until image is rebuilt with
gh/jq/bison/python3.9)
@@ -130,7 +130,7 @@ jobs:
needs.detect-changes.outputs.java == 'true' ||
needs.detect-changes.outputs.shims35 == 'true' ||
needs.detect-changes.outputs.cpp == 'true'
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
container: apache/gluten:centos-8-jdk8
steps:
- uses: actions/checkout@v7
@@ -187,7 +187,7 @@ jobs:
needs.detect-changes.outputs.java == 'true' ||
needs.detect-changes.outputs.shims35 == 'true' ||
needs.detect-changes.outputs.cpp == 'true'
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
container: apache/gluten:centos-8-jdk8
steps:
- uses: actions/checkout@v7
@@ -231,7 +231,7 @@ jobs:
# needs.detect-changes.outputs.java == 'true' ||
# needs.detect-changes.outputs.shims40 == 'true' ||
# needs.detect-changes.outputs.cpp == 'true'
- # runs-on: ubuntu-22.04
+ # runs-on: ubuntu-24.04
# strategy:
# fail-fast: false
# matrix:
diff --git a/.github/workflows/velox_backend_x86.yml
b/.github/workflows/velox_backend_x86.yml
index 06bfbb8b89..5a4a22fcd2 100644
--- a/.github/workflows/velox_backend_x86.yml
+++ b/.github/workflows/velox_backend_x86.yml
@@ -102,7 +102,7 @@ jobs:
# workflow: a fork with Actions enabled would otherwise burn ~1800 runner
# minutes nightly on a signal nobody reads.
if: github.event_name != 'schedule' || github.repository_owner == 'apache'
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
outputs:
full_run: ${{ steps.filter.outputs.full_run }}
cpp: ${{ steps.filter.outputs.cpp }}
@@ -168,7 +168,7 @@ jobs:
build-native-lib-centos-8:
needs: detect-changes
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
container: apache/gluten:vcpkg-almalinux-8-gcc13
steps:
- name: Install CI tools (temporary - until image is rebuilt with
gh/jq/bison/python3.9)
@@ -223,7 +223,7 @@ jobs:
strategy:
fail-fast: false
matrix:
- os: [ "ubuntu:22.04" ]
+ os: [ "ubuntu:24.04" ]
spark: [ "spark-3.4", "spark-3.5", "spark-4.0", "spark-4.1" ]
# PR runs test the primary JDK per Spark line (8 for 3.x, 17 for 4.x --
# after the static excludes below that is 4 combos instead of 8). The
@@ -260,7 +260,7 @@ jobs:
- spark: spark-4.1
java: java-11
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
container: ${{ matrix.os }}
steps:
- uses: actions/checkout@v7
@@ -272,7 +272,7 @@ jobs:
- name: Setup tzdata
run: |
sed -i 's|http://archive|http://eu.archive|g' /etc/apt/sources.list
- if [ "${{ matrix.os }}" = "ubuntu:22.04" ]; then
+ if [ "${{ matrix.os }}" = "ubuntu:24.04" ]; then
apt-get update
TZ="Etc/GMT" DEBIAN_FRONTEND=noninteractive apt-get install -y
tzdata
fi
@@ -292,7 +292,7 @@ jobs:
apt remove openjdk-11* -y
fi
- name: Install Hadoop & Setup HDFS
- if: matrix.os == 'ubuntu:22.04' && matrix.spark == 'spark-3.5' &&
matrix.java == 'java-8'
+ if: matrix.os == 'ubuntu:24.04' && matrix.spark == 'spark-3.5' &&
matrix.java == 'java-8'
shell: bash
run: |
export JAVA_HOME=/usr/lib/jvm/${{ matrix.java }}-openjdk-amd64
@@ -300,7 +300,7 @@ jobs:
install_hadoop
setup_hdfs
- name: Install MinIO
- if: matrix.os == 'ubuntu:22.04' && matrix.spark == 'spark-3.5' &&
matrix.java == 'java-8'
+ if: matrix.os == 'ubuntu:24.04' && matrix.spark == 'spark-3.5' &&
matrix.java == 'java-8'
shell: bash
run: |
export JAVA_HOME=/usr/lib/jvm/${{ matrix.java }}-openjdk-amd64
@@ -323,7 +323,7 @@ jobs:
cd $GITHUB_WORKSPACE/tools/gluten-it
$GITHUB_WORKSPACE/$MVN_CMD clean install -P${{ matrix.spark }} -P${{
matrix.java }}
# Setup S3 JARs after gluten-it build
- if [ "${{ matrix.os }}" = "ubuntu:22.04" ] && \
+ if [ "${{ matrix.os }}" = "ubuntu:24.04" ] && \
[ "${{ matrix.spark }}" = "spark-3.5" ] && \
[ "${{ matrix.java }}" = "java-8" ]; then
source
$GITHUB_WORKSPACE/.github/workflows/util/install-spark-deps.sh
@@ -334,7 +334,7 @@ jobs:
--local --preset=velox --benchmark-type=h --error-on-memleak
--off-heap-size=10g -s=1.0 --threads=16 --iterations=1 \
&& GLUTEN_IT_JVM_ARGS=-Xmx6G sbin/gluten-it.sh queries-compare \
--local --preset=velox --benchmark-type=ds --error-on-memleak
--off-heap-size=10g -s=1.0 --threads=16 --iterations=1
- if [ "${{ matrix.os }}" = "ubuntu:22.04" ] && \
+ if [ "${{ matrix.os }}" = "ubuntu:24.04" ] && \
[ "${{ matrix.spark }}" = "spark-3.5" ] && \
[ "${{ matrix.java }}" = "java-8" ]; then
GLUTEN_IT_JVM_ARGS=-Xmx6G sbin/gluten-it.sh queries-compare \
@@ -390,7 +390,7 @@ jobs:
- spark: spark-4.1
java: java-11
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
container: ${{ matrix.os }}
steps:
- uses: actions/checkout@v7
@@ -459,7 +459,7 @@ jobs:
fail-fast: false
matrix:
spark: [ "spark-3.5" ]
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
steps:
- name: Maximize build disk space
shell: bash
@@ -577,7 +577,7 @@ jobs:
matrix:
spark: [ "spark-3.5" ]
shard: [ 1, 2, 3 ]
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
steps:
- name: Maximize build disk space
shell: bash
@@ -630,7 +630,7 @@ jobs:
spark: [ "spark-3.5" ]
uniffle: [ "0.10.0" ]
hadoop: [ "2.10.2" ]
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
container: apache/gluten:centos-8-jdk8
steps:
- uses: actions/checkout@v7
@@ -687,7 +687,7 @@ jobs:
&& '["celeborn-0.6.3", "celeborn-0.5.4"]'
|| '["celeborn-0.6.3"]') }}
writer: ["sort", "hash"]
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
container: centos:8
steps:
- uses: actions/checkout@v7
@@ -758,7 +758,7 @@ jobs:
needs.detect-changes.outputs.java == 'true' ||
needs.detect-changes.outputs.shims34 == 'true' ||
needs.detect-changes.outputs.cpp == 'true'
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
strategy:
fail-fast: false
matrix:
@@ -829,7 +829,7 @@ jobs:
needs.detect-changes.outputs.java == 'true' ||
needs.detect-changes.outputs.shims34 == 'true' ||
needs.detect-changes.outputs.cpp == 'true'
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
strategy:
fail-fast: false
matrix:
@@ -878,7 +878,7 @@ jobs:
spark-test-spark35:
needs: [detect-changes, build-native-lib-centos-8]
if: needs.detect-changes.outputs.spark35_needed == 'true'
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
strategy:
fail-fast: false
matrix:
@@ -951,7 +951,7 @@ jobs:
# little PR-specific signal. The nightly full_run (or a workflow_dispatch
run)
# still covers it daily.
if: needs.detect-changes.outputs.full_run == 'true'
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
strategy:
fail-fast: false
matrix:
@@ -1014,7 +1014,7 @@ jobs:
spark-test-spark35-slow:
needs: [detect-changes, build-native-lib-centos-8]
if: needs.detect-changes.outputs.spark35_needed == 'true'
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
strategy:
fail-fast: false
matrix:
@@ -1064,7 +1064,7 @@ jobs:
cpp-test-udf-test:
needs: detect-changes
if: needs.detect-changes.outputs.cpp == 'true' ||
needs.detect-changes.outputs.udf == 'true'
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
container: apache/gluten:centos-9-jdk8
steps:
- uses: actions/checkout@v7
@@ -1211,7 +1211,7 @@ jobs:
build-cudf-centos-9:
needs: detect-changes
if: needs.detect-changes.outputs.cpp == 'true'
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
steps:
- name: "node-cleanup" # by default the free runner does not have enough
disk space
run: |
@@ -1273,7 +1273,7 @@ jobs:
if: >-
needs.detect-changes.outputs.java == 'true' ||
needs.detect-changes.outputs.cpp == 'true'
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
container: apache/gluten:centos-8-jdk17
steps:
- uses: actions/checkout@v7
@@ -1288,7 +1288,7 @@ jobs:
needs.detect-changes.outputs.java == 'true' ||
needs.detect-changes.outputs.shims42 == 'true' ||
needs.detect-changes.outputs.cpp == 'true'
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
container: apache/gluten:centos-8-jdk17
steps:
- uses: actions/checkout@v7
@@ -1303,7 +1303,7 @@ jobs:
needs.detect-changes.outputs.java == 'true' ||
needs.detect-changes.outputs.shims40 == 'true' ||
needs.detect-changes.outputs.cpp == 'true'
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
strategy:
fail-fast: false
matrix:
@@ -1369,7 +1369,7 @@ jobs:
needs.detect-changes.outputs.java == 'true' ||
needs.detect-changes.outputs.shims40 == 'true' ||
needs.detect-changes.outputs.cpp == 'true'
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
strategy:
fail-fast: false
matrix:
@@ -1422,7 +1422,7 @@ jobs:
needs.detect-changes.outputs.java == 'true' ||
needs.detect-changes.outputs.shims41 == 'true' ||
needs.detect-changes.outputs.cpp == 'true'
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
strategy:
fail-fast: false
matrix:
@@ -1497,7 +1497,7 @@ jobs:
needs.detect-changes.outputs.java == 'true' ||
needs.detect-changes.outputs.shims41 == 'true' ||
needs.detect-changes.outputs.cpp == 'true'
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
strategy:
fail-fast: false
matrix:
@@ -1551,7 +1551,7 @@ jobs:
needs.detect-changes.outputs.java == 'true' ||
needs.detect-changes.outputs.shims42 == 'true' ||
needs.detect-changes.outputs.cpp == 'true'
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
strategy:
fail-fast: false
matrix:
@@ -1629,7 +1629,7 @@ jobs:
needs.detect-changes.outputs.java == 'true' ||
needs.detect-changes.outputs.shims42 == 'true' ||
needs.detect-changes.outputs.cpp == 'true'
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
strategy:
fail-fast: false
matrix:
diff --git a/.github/workflows/velox_kafka.yml
b/.github/workflows/velox_kafka.yml
index c4821a34a6..0acab52607 100644
--- a/.github/workflows/velox_kafka.yml
+++ b/.github/workflows/velox_kafka.yml
@@ -48,7 +48,7 @@ env:
jobs:
build-native:
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
container: apache/gluten:vcpkg-almalinux-8-gcc13
timeout-minutes: 180
steps:
@@ -97,7 +97,7 @@ jobs:
spark35:
needs: build-native
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
timeout-minutes: 60
env:
KAFKA_BOOTSTRAP_SERVERS: localhost:9092
diff --git a/.github/workflows/velox_native_build.yml
b/.github/workflows/velox_native_build.yml
index b74a8212ff..aed0ab7c92 100644
--- a/.github/workflows/velox_native_build.yml
+++ b/.github/workflows/velox_native_build.yml
@@ -34,7 +34,7 @@ on:
jobs:
build:
# A cache miss must be able to finish, using the same timeout allowance as
main.
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
container: ${{ inputs.image }}
env:
ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION: true
diff --git a/.github/workflows/velox_nightly.yml
b/.github/workflows/velox_nightly.yml
index 3fc4917462..e7a8187fae 100644
--- a/.github/workflows/velox_nightly.yml
+++ b/.github/workflows/velox_nightly.yml
@@ -36,7 +36,7 @@ concurrency:
jobs:
build-native-lib-x86:
if: ${{ startsWith(github.repository, 'apache/') }}
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
container: apache/gluten:vcpkg-almalinux-8-gcc13
steps:
- uses: actions/checkout@v7
@@ -69,7 +69,7 @@ jobs:
build-bundle-package-centos8-jdk8-x86:
if: ${{ startsWith(github.repository, 'apache/') }}
needs: build-native-lib-x86
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
container: centos:8
steps:
- uses: actions/checkout@v7
@@ -106,7 +106,7 @@ jobs:
build-bundle-package-centos8-jdk17-x86:
if: ${{ startsWith(github.repository, 'apache/') }}
needs: build-native-lib-x86
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
container: centos:8
steps:
- uses: actions/checkout@v7
@@ -146,7 +146,7 @@ jobs:
build-bundle-package-centos9-jdk21-x86:
if: ${{ startsWith(github.repository, 'apache/') }}
needs: build-native-lib-x86
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
container: quay.io/centos/centos:stream9
steps:
- uses: actions/checkout@v7
@@ -215,7 +215,7 @@ jobs:
build-bundle-package-centos8-jdk8-arm64:
if: ${{ startsWith(github.repository, 'apache/') }}
needs: build-native-lib-centos-8-arm64
- runs-on: ubuntu-22.04-arm
+ runs-on: ubuntu-24.04-arm
container: centos:8
steps:
- uses: actions/checkout@v7
@@ -252,7 +252,7 @@ jobs:
build-bundle-package-centos8-jdk17-arm64:
if: ${{ startsWith(github.repository, 'apache/') }}
needs: build-native-lib-centos-8-arm64
- runs-on: ubuntu-22.04-arm
+ runs-on: ubuntu-24.04-arm
container: centos:8
steps:
- uses: actions/checkout@v7
@@ -292,7 +292,7 @@ jobs:
build-bundle-package-centos9-jdk21-arm64:
if: ${{ startsWith(github.repository, 'apache/') }}
needs: build-native-lib-centos-8-arm64
- runs-on: ubuntu-22.04-arm
+ runs-on: ubuntu-24.04-arm
container: quay.io/centos/centos:stream9
steps:
- uses: actions/checkout@v7
@@ -329,7 +329,7 @@ jobs:
upload-jdk8-package:
if: ${{ startsWith(github.repository, 'apache/') }}
needs: [build-bundle-package-centos8-jdk8-arm64,
build-bundle-package-centos8-jdk8-x86]
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
steps:
- name: Get current date
id: date
@@ -358,7 +358,7 @@ jobs:
upload-jdk17-package:
if: ${{ startsWith(github.repository, 'apache/') }}
needs: [build-bundle-package-centos8-jdk17-arm64,
build-bundle-package-centos8-jdk17-x86]
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
steps:
- name: Get current date
id: date
@@ -387,7 +387,7 @@ jobs:
upload-jdk21-package:
if: ${{ startsWith(github.repository, 'apache/') }}
needs: [ build-bundle-package-centos9-jdk21-arm64,
build-bundle-package-centos9-jdk21-x86 ]
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
steps:
- name: Get current date
id: date
@@ -417,7 +417,7 @@ jobs:
# Build velox with enhanced features
build-native-lib-enhanced:
if: ${{ startsWith(github.repository, 'apache/') }}
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v7
- name: Get Ccache
@@ -454,7 +454,7 @@ jobs:
build-bundle-enhanced-package-centos8-jdk17:
if: ${{ startsWith(github.repository, 'apache/') }}
needs: build-native-lib-enhanced
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
container: centos:8
steps:
- uses: actions/checkout@v7
@@ -504,7 +504,7 @@ jobs:
spark-test-spark35-smj:
if: ${{ startsWith(github.repository, 'apache/') }}
needs: build-native-lib-x86
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
env:
SPARK_TESTING: true
container: apache/gluten:centos-8-jdk8
@@ -552,7 +552,7 @@ jobs:
spark-test-spark35-slow-smj:
if: ${{ startsWith(github.repository, 'apache/') }}
needs: build-native-lib-x86
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
env:
SPARK_TESTING: true
LANG: C.UTF-8 #TODO remove after image update
diff --git a/.github/workflows/velox_weekly.yml
b/.github/workflows/velox_weekly.yml
index b5b4104d41..695014a860 100644
--- a/.github/workflows/velox_weekly.yml
+++ b/.github/workflows/velox_weekly.yml
@@ -44,7 +44,7 @@ jobs:
matrix:
os: [ "centos:8", "quay.io/centos/centos:stream9" ]
if: ${{ startsWith(github.repository, 'apache/') }}
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
container: ${{ matrix.os }}
steps:
- name: Checkout repository
@@ -122,7 +122,7 @@ jobs:
matrix:
os: [ "ubuntu:22.04", "ubuntu:24.04" ]
if: ${{ startsWith(github.repository, 'apache/') }}
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
container: ${{ matrix.os }}
steps:
- uses: actions/checkout@v7
diff --git a/cpp/velox/cudf/CudfFunctions.cc b/cpp/velox/cudf/CudfFunctions.cc
index b0e8d838c4..180f38537d 100644
--- a/cpp/velox/cudf/CudfFunctions.cc
+++ b/cpp/velox/cudf/CudfFunctions.cc
@@ -75,7 +75,7 @@ class RowConstructorWithNullFunction final : public
CudfFunction {
std::shared_ptr<CudfFunction> rowConstructor_;
};
-bool canEvaluateRowConstructorWithNull(const TypedExprPtr& expr) {
+bool canEvaluateRowConstructorWithNull(const TypedExprPtr& expr, const
facebook::velox::core::QueryConfig& /*config*/) {
return expr->type()->isRow() && !expr->inputs().empty() &&
std::any_of(
expr->inputs().begin(), expr->inputs().end(), [](const auto&
input) { return !input->isConstantKind(); });
@@ -88,8 +88,9 @@ void registerCudfFunctions() {
RowConstructorWithNullCallToSpecialForm::kRowConstructorWithNull,
[](const std::string&,
const TypedExprPtr& expr,
- facebook::velox::memory::MemoryPool* pool) ->
std::shared_ptr<CudfFunction> {
- auto rowConstructor =
facebook::velox::cudf_velox::createCudfFunction("row_constructor", expr, pool);
+ facebook::velox::memory::MemoryPool* pool,
+ const facebook::velox::core::QueryConfig& config) ->
std::shared_ptr<CudfFunction> {
+ auto rowConstructor =
facebook::velox::cudf_velox::createCudfFunction("row_constructor", expr, pool,
config);
VELOX_CHECK_NOT_NULL(rowConstructor, "cuDF row_constructor must be
registered first");
return
std::make_shared<RowConstructorWithNullFunction>(std::move(rowConstructor));
},
diff --git a/cpp/velox/operators/hashjoin/HashTableBuilder.cc
b/cpp/velox/operators/hashjoin/HashTableBuilder.cc
index 6414a11485..676b3b65d2 100644
--- a/cpp/velox/operators/hashjoin/HashTableBuilder.cc
+++ b/cpp/velox/operators/hashjoin/HashTableBuilder.cc
@@ -164,42 +164,30 @@ void HashTableBuilder::setupTable() {
for (int i = numKeys; i < tableType_->size(); ++i) {
dependentTypes.emplace_back(tableType_->childAt(i));
}
+ facebook::velox::exec::BaseHashTable::Options options;
+ options.minTableSizeForParallelJoinBuild = minTableRowsForParallelJoinBuild_;
if (isRightJoin(joinType_) || isFullJoin(joinType_) ||
isRightSemiProjectJoin(joinType_)) {
// Do not ignore null keys.
- uniqueTable_ = facebook::velox::exec::HashTable<false>::createForJoin(
- std::move(keyHashers),
- dependentTypes,
- true, // allowDuplicates
- true, // hasProbedFlag
- false, // hasCountFlag
- minTableRowsForParallelJoinBuild_,
- pool_);
+ options.allowDuplicates = true;
+ options.hasProbedFlag = true;
+ options.hasCountFlag = false;
+ uniqueTable_ =
+
facebook::velox::exec::HashTable<false>::createForJoin(std::move(keyHashers),
dependentTypes, pool_, options);
} else {
// Right semi join needs to tag build rows that were probed.
- const bool needProbedFlag = isRightSemiFilterJoin(joinType_);
- const bool hasCountFlag = facebook::velox::core::isCountingJoin(joinType_);
+ options.allowDuplicates = !dropDuplicates_;
+ options.hasProbedFlag = isRightSemiFilterJoin(joinType_);
+ options.hasCountFlag = facebook::velox::core::isCountingJoin(joinType_);
if (facebook::velox::exec::isLeftNullAwareJoinWithFilter(joinType_,
nullAware_, withFilter_)) {
// We need to check null key rows in build side in case of null-aware
anti
// or left semi project join with filter set.
- uniqueTable_ = facebook::velox::exec::HashTable<false>::createForJoin(
- std::move(keyHashers),
- dependentTypes,
- !dropDuplicates_, // allowDuplicates
- needProbedFlag, // hasProbedFlag
- hasCountFlag, // hasCountFlag
- minTableRowsForParallelJoinBuild_,
- pool_);
+ uniqueTable_ =
+
facebook::velox::exec::HashTable<false>::createForJoin(std::move(keyHashers),
dependentTypes, pool_, options);
} else {
// Ignore null keys
- uniqueTable_ = facebook::velox::exec::HashTable<true>::createForJoin(
- std::move(keyHashers),
- dependentTypes,
- !dropDuplicates_, // allowDuplicates
- needProbedFlag, // hasProbedFlag
- hasCountFlag, // hasCountFlag
- minTableRowsForParallelJoinBuild_,
- pool_,
- bloomFilterPushdownSize_);
+ options.bloomFilterMaxSize =
static_cast<uint64_t>(bloomFilterPushdownSize_);
+ uniqueTable_ =
+
facebook::velox::exec::HashTable<true>::createForJoin(std::move(keyHashers),
dependentTypes, pool_, options);
}
}
analyzeKeys_ = uniqueTable_->hashMode() !=
facebook::velox::exec::BaseHashTable::HashMode::kHash;
diff --git a/cpp/velox/substrait/SubstraitToVeloxPlan.cc
b/cpp/velox/substrait/SubstraitToVeloxPlan.cc
index bc52611bb9..dc50d3808c 100644
--- a/cpp/velox/substrait/SubstraitToVeloxPlan.cc
+++ b/cpp/velox/substrait/SubstraitToVeloxPlan.cc
@@ -1320,6 +1320,7 @@ core::PlanNodePtr
SubstraitToVeloxPlanConverter::toVeloxPlan(
sortingOrders,
rowNumberColumnName,
static_cast<int32_t>(windowGroupLimitRel.limit()),
+ /*partial=*/true,
childNode);
}
diff --git a/cpp/velox/tests/CudfFunctionsTest.cc
b/cpp/velox/tests/CudfFunctionsTest.cc
index df62906c3b..bd1105dea0 100644
--- a/cpp/velox/tests/CudfFunctionsTest.cc
+++ b/cpp/velox/tests/CudfFunctionsTest.cc
@@ -141,9 +141,10 @@ TEST_F(CudfFunctionsTest, rejectsAllLiteralRow) {
RowConstructorWithNullCallToSpecialForm::kRowConstructorWithNull);
EXPECT_FALSE(cudf_velox::canExprRunOnGpu(rowConstructor, nullptr,
pool_.get()));
+ core::QueryConfig queryConfig{{}};
EXPECT_EQ(
cudf_velox::createCudfFunction(
- RowConstructorWithNullCallToSpecialForm::kRowConstructorWithNull,
rowConstructor, pool_.get()),
+ RowConstructorWithNullCallToSpecialForm::kRowConstructorWithNull,
rowConstructor, pool_.get(), queryConfig),
nullptr);
}
diff --git a/ep/build-velox/src/get-velox.sh b/ep/build-velox/src/get-velox.sh
index 502d6643e8..7fb5350a75 100755
--- a/ep/build-velox/src/get-velox.sh
+++ b/ep/build-velox/src/get-velox.sh
@@ -18,8 +18,8 @@ set -exu
CURRENT_DIR=$(cd "$(dirname "$BASH_SOURCE")"; pwd)
VELOX_REPO=https://github.com/IBM/velox.git
-VELOX_BRANCH=dft-2026_11_06
-VELOX_ENHANCED_BRANCH=ibm-2026_11_06
+VELOX_BRANCH=dft-2026_11_10
+VELOX_ENHANCED_BRANCH=ibm-2026_11_10
VELOX_HOME=""
RUN_SETUP_SCRIPT=ON
ENABLE_ENHANCED_FEATURES=OFF
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]