This is an automated email from the ASF dual-hosted git repository.
yuanzhou pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git
The following commit(s) were added to refs/heads/main by this push:
new 53c29d9bc1 [GLUTEN-6887][VL] Daily Update Velox Version (2025_12_08)
(#11266)
53c29d9bc1 is described below
commit 53c29d9bc1d15ccfa22053b6ee0102d82306403f
Author: Gluten Performance Bot
<[email protected]>
AuthorDate: Mon Dec 8 17:27:20 2025 +0000
[GLUTEN-6887][VL] Daily Update Velox Version (2025_12_08) (#11266)
* [GLUTEN-6887][VL] Daily Update Velox Version (dft-2025_12_08)
Upstream Velox's New Commits:
35b8a49e2 by Avinaash Gupta, misc: Back out batch serialization with
preserve encoding option support in remote functions (#15704)
8a466df68 by Pratik Pugalia, feat: at_timezone function for TIME WITH TIME
ZONE (#15659)
4820c20c3 by Pratik Pugalia, fix: Greatest/least custom comparison; add
Time/TimeWithTimezone support (#15666)
418240569 by Chang chen, fix: Handle NULL values in comparison filter
generation (#15676)
14967aa6a by Pratik Pugalia, fix: Change encoding of TimeZone in Time with
Timezone to match Presto (#15710)
6cfdded63 by Ke Wang, fix: Fix ExchangeClient stats reporting (#15457)
7f9fbeb70 by Xiaoxuan Meng, perf: reland Add noGroupsSpanBatches flag to
streaming aggregation for optimize output processing (#15709)
1602b7441 by Julia Du, Revert D88054187: perf: Add noGroupsSpanBatches flag
to streaming aggregation for optimize output processing
bfbebf5d8 by Xiaoxuan Meng, perf: Add noGroupsSpanBatches flag to streaming
aggregation for optimize output processing (#15699)
4ff28c6bf by Masha Basmanova, feat: Add support for coersions to COALESCE
(#15701)
9c98b2fa7 by lingbin, refactor: Remove unused
'velox_memory_pool_debug_enabled' flag (#15655)
da46ef264 by xiaodou, feat: Support variant-to-vector for DECIMAL types
(#15695)
4940973a6 by HolyLow, feat(spill): Support merging sorted files in multiple
rounds (#14143)
82638bf3f by Alex Malyshev, fix: Fix protobuf version check in Arena.h
(#15609)
868f27959 by PHILO-HE, feat(build): Add `SIMDJSON_SKIPUTF8VALIDATION`
option for simdjson in setup script (#15627)
987b5a83e by Alex Malyshev, refactor: Use standard C/C++ integer types
instead of protobuf ones (#15263)
e98a9d01c by Ke Wang, fix: Fix distance functions header (#15692)
4edd67f6c by duanmeng, fix: Fix build failure due to undefined symbols
registerOrcReaderFactory (#15683)
bf54f0eee by Ping Liu, refactor: Reduce copy-paste in ParquetWriterTest
(#15678)
b99382c21 by Karthikeyan, feat(cudf): Upgrade cudf 25.12 (Nov 24, 2025)
(#15629)
488b15c0c by NEUpanning, perf: Optimize addSingleGroupIntermediateResults
for min/max aggregates (#15654)
bf9e9fd6d by Patrick Sullivan, feat(geo): Implement
ST_Area(SphericalGeography) (#15601)
233712719 by Jiahao Liang, Add Properties Field to ReaderOptions (#15669)
0de53219a by Bikramjeet Vig, feat: Add selective CPU tracking for
expressions (#15650)
eba05296b by Han Yan, feat: Add Exchange/BroadcastRead to trace tool
(#15672)
ebbd27088 by Han Zhu, velox: Migrate from CUDA 12.0 (#15689)
2be34391b by generatedunixname89002005232357, Revert D88099458: fix: Add
camel case casting utilities to velox::Casts
37c1a6874 by Eric Jia, fix: Add camel case casting utilities to
velox::Casts (#15665)
7a6dea3b3 by Masha Basmanova, feat: Enhance
extractFiltersFromRemainingFilter to support an OR (#15565)
919986373 by Zac Wen, feat: Batch IndexLookupJoin output when splitOutput
is disabled (#15636)
4f39eef54 by Mark Santaniello, fix: Avoid string conversions due to
TestValue (#15662)
e01ca2187 by Alex Malyshev, refactor: Address more usage of
Arena::CreateMessage in dwrf/writer (#15378)
0f21ff954 by Xiao Du, fix: Fix the uninitialized executor_ pointer (#15674)
a236cf5b3 by Masha Basmanova, feat: Add support for coercions to IF and
SWITCH (#15675)
a017fca78 by Avinaash Gupta, perf: Batch serialization with preserve
encoding option support in remote functions (#15643)
31775e741 by lingbin, fix: Use 'MemoryManager::Options' instead of
'MemoryManagerOption' (#15656)
124e501a2 by Heidi Han, refactor: Move HllAccumulator to HllUtils (#15573)
1071e2d1d by Han Yan, refactor: Rename velox casting utilities to camelCase
naming convention (#15622)
cb26946ee by Hongze Zhang, fix(ci): Test changes in setup-fedora.sh (#15490)
Signed-off-by: glutenperfbot <[email protected]>
* use gcc-14 for gpu feature
Signed-off-by: Yuan <[email protected]>
* revert
https://github.com/IBM/velox/commit/7a6dea3b31002f18dc9e9c7621e4c1bb758f0551
* fix disk space and cache job for gpu
Signed-off-by: Yuan <[email protected]>
* update gpu docker image
Signed-off-by: Yuan <[email protected]>
---------
Signed-off-by: glutenperfbot <[email protected]>
Signed-off-by: Yuan <[email protected]>
Co-authored-by: glutenperfbot <[email protected]>
Co-authored-by: Yuan <[email protected]>
Co-authored-by: Rong Ma <[email protected]>
---
.github/workflows/velox_backend_cache.yml | 11 ++++++++---
.github/workflows/velox_backend_x86.yml | 12 ++++++------
dev/docker/cudf/Dockerfile | 3 ++-
ep/build-velox/src/get-velox.sh | 4 ++--
4 files changed, 18 insertions(+), 12 deletions(-)
diff --git a/.github/workflows/velox_backend_cache.yml
b/.github/workflows/velox_backend_cache.yml
index 60ca54ecda..eb060c3a90 100644
--- a/.github/workflows/velox_backend_cache.yml
+++ b/.github/workflows/velox_backend_cache.yml
@@ -157,11 +157,16 @@ jobs:
ccache-centos9-release-shared-${{runner.arch}}
- name: Build Gluten shared libraries
run: |
- df -a
export CCACHE_MAXSIZE=1G
- bash dev/buildbundle-veloxbe.sh --run_setup_script=OFF
--build_arrow=OFF --spark_version=3.4 --enable_gpu=ON
- ccache -s
+ dnf autoremove -y
+ df -a
+ rm -rf /opt/rh/gcc-toolset-12 && cp -r /opt/rh/gcc-toolset-14
/opt/rh/gcc-toolset-12 # hack to use gcc 14, should upgrade in Velox build
script later
+ source /opt/rh/gcc-toolset-12/enable
+ export NUM_THREADS=4
+ bash dev/builddeps-veloxbe.sh --run_setup_script=OFF
--build_arrow=OFF --build_tests=OFF --build_benchmarks=ON --enable_gpu=ON #
TODO: re-enable tests with more disk space
+ rm -rf ep/build-velox/build/velox_ep
- name: Save Ccache
+ if: always()
uses: actions/cache/save@v3
id: ccache
with:
diff --git a/.github/workflows/velox_backend_x86.yml
b/.github/workflows/velox_backend_x86.yml
index 52d6d92624..5618aba857 100644
--- a/.github/workflows/velox_backend_x86.yml
+++ b/.github/workflows/velox_backend_x86.yml
@@ -1326,14 +1326,14 @@ jobs:
ccache-centos9-release-shared-${{runner.arch}}
- name: Build Gluten native libraries
run: |
-
repo_url="https://developer.download.nvidia.com/compute/cuda/repos/rhel9/x86_64/cuda-rhel9.repo";
- dnf config-manager --add-repo "$repo_url"
- dnf install -y libnvjitlink-devel-12-8
+ rm -rf /opt/rh/gcc-toolset-12 && cp -r /opt/rh/gcc-toolset-14
/opt/rh/gcc-toolset-12 # hack to use gcc 14, should upgrade in Velox build
script later
dnf autoremove -y
df -a
- bash dev/builddeps-veloxbe.sh --run_setup_script=OFF
--build_arrow=OFF --build_tests=ON --build_benchmarks=ON --enable_gpu=ON
- rm -rf ep/build-velox/build/velox_ep
- mvn clean package -Pbackends-velox -Pspark-3.4 -DskipTests
+ source /opt/rh/gcc-toolset-12/enable
+ export NUM_THREADS=4
+ # bash dev/builddeps-veloxbe.sh --run_setup_script=OFF
--build_arrow=OFF --build_tests=OFF --build_benchmarks=ON --enable_gpu=ON #
TODO: re-enable tests with more disk space
+ # rm -rf ep/build-velox/build/velox_ep
+ # mvn clean package -Pbackends-velox -Pspark-3.4 -DskipTests
ccache -s
spark-test-spark40:
diff --git a/dev/docker/cudf/Dockerfile b/dev/docker/cudf/Dockerfile
index de92527a1e..42258a69c9 100644
--- a/dev/docker/cudf/Dockerfile
+++ b/dev/docker/cudf/Dockerfile
@@ -28,7 +28,8 @@ ENV CUDA_ARCHITECTURES=70
WORKDIR /opt/gluten
-RUN bash ./dev/buildbundle-veloxbe.sh --run_setup_script=OFF --build_arrow=ON
--spark_version=3.4 --build_tests=ON --build_benchmarks=ON --enable_gpu=ON &&
rm -rf /opt/gluten
+RUN rm -rf /opt/rh/gcc-toolset-12 && cp -r /opt/rh/gcc-toolset-14
/opt/rh/gcc-toolset-12; \
+ bash ./dev/buildbundle-veloxbe.sh --run_setup_script=OFF --build_arrow=ON
--spark_version=3.4 --build_tests=ON --build_benchmarks=ON --enable_gpu=ON &&
rm -rf /opt/gluten
# You can try the data in folder
backends-velox/src/test/resources/tpch-data-parquet
diff --git a/ep/build-velox/src/get-velox.sh b/ep/build-velox/src/get-velox.sh
index 28d34d07df..57454916c8 100755
--- a/ep/build-velox/src/get-velox.sh
+++ b/ep/build-velox/src/get-velox.sh
@@ -18,8 +18,8 @@ set -exu
CURRENT_DIR=$(cd "$(dirname "$BASH_SOURCE")"; pwd)
VELOX_REPO=https://github.com/IBM/velox.git
-VELOX_BRANCH=dft-2025_12_03
-VELOX_ENHANCED_BRANCH=ibm-2025_12_03
+VELOX_BRANCH=dft-2025_12_08-1
+VELOX_ENHANCED_BRANCH=ibm-2025_12_08-1
VELOX_HOME=""
RUN_SETUP_SCRIPT=ON
ENABLE_ENHANCED_FEATURES=OFF
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
