This is an automated email from the ASF dual-hosted git repository.
zhouyuan pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/gluten.git
The following commit(s) were added to refs/heads/main by this push:
new 4c1a850639 [GLUTEN-6887][VL] Daily Update Velox Version (2026_10_13)
(#13257)
4c1a850639 is described below
commit 4c1a8506396a0626707765413b7f2c58d273cbdb
Author: Gluten Performance Bot
<[email protected]>
AuthorDate: Tue Oct 13 19:39:19 2026 +0100
[GLUTEN-6887][VL] Daily Update Velox Version (2026_10_13) (#13257)
* [GLUTEN-6887][VL] Daily Update Velox Version (dft-2026_10_13)
Upstream Velox's New Commits:
b918c63dc by Apurva Kumar, fix(iceberg): Use split values for identity
partition columns
0382c5b76 by Xiaoxuan, feat(nimble): Add index-backed row and batch
iterators (#19520)
34c0db3d0 by Karthikeyan, refactor(functions): Move date/time unit
arithmetic to a GPU header
aeeaf8816 by macduan, feat(nimble): Add ALPRD bulk reads and encoding views
700554055 by Ke, feat(nimble): Reuse dictionary visitor for numeric reads
(#19254)
081ac3c26 by Pedro Pedreira, refactor(vector): Use fromMapVector and
assertEqualVectors in tests
c22e08eb3 by Minni Mittal, feat(spark): Add initial format_string and
printf support
4eb15050d by Karthikeyan, fix(cudf): Label CudfFromVelox output with its
input type
0adb32470 by Maria Basmanova, fix(type): Delete inherited create() in
derived scalar types (#19526)
9f290056e by Minni Mittal, feat(spark): Add octet_length scalar function
5271a1e6d by Mohammad Linjawi, refactor(spark): Move hashOne overloads to
Hash.h
aa5020aef by Shruti Shivakumar, feat(cudf): Enable streaming final decimal
SUM and AVG
315b76984 by Karthikeyan, feat(cudf): Pass the session QueryConfig to
evaluator creation
b19d44974 by Patrick Wilson, fix(cudf): Wait for probe reads before
releasing hash join state
4831b2dba by PRASHANT GOLASH, feat(nimble): Add FSST EncodingView support
(#19397)
c0225a65f by XiaoDu, feat(nimble): Add IVF RaBitQ FastScan (#19405)
18fa49af5 by PRASHANT GOLASH, refactor(nimble): Factor out driver context
setup for parallel encoding (#19410)
f92d61d1c by Sahil Gandhi, feat(exec): Report experimental fixed-point
execution stats (#19455)
0b3944f98 by Natasha Sehgal, test(functions): Re-enable Poisson CDF parity
fuzzing
1625bdb1a by Sahil Gandhi, perf(exec): Build fixed-point hash tables per
driver (#19454)
e216da572 by XiaoDu, fix(nimble): Honor vector index pinning policy (#19515)
c37a5ce37 by Sahil Gandhi, fix(exec): Reject grouped execution for fixed
points (#19453)
780d58a0c by XiaoDu, feat(nimble): Parallelize IVF query-list scans (#19399)
f5c38f87b by XiaoDu, feat(nimble): Push row filters into vector search
(#19321)
5402a01c0 by Pedro Pedreira, feat(vector): Add
FlatMapVector::fromMapVector() (#19507)
1d2534421 by Xiaoxuan, refactor(nimble): Remove `allowNestedAlpSelection`
52827270a by Suryadev Sahadevan Rajesh, feat(nimble): Auto-select
SubIntSplit child encodings and split boundaries (#19504)
71e04d9e9 by tanjialiang, feat(nimble): Store per stream checksums in
stream trailers
6c9c8bad7 by Pedro Pedreira, fix(vector): Fix flat map copies in
encodedVectorCopy (#19499)
6cbe2d062 by Pedro Pedreira, fix(functions): Respect in-map bits in
element_at on flat maps (#19500)
010931b18 by Pedro Pedreira, feat(vector): Copy between MapVector and
FlatMapVector (#19486)
Signed-off-by: glutenperfbot <[email protected]>
* disable spark-40 tests in enhanced branch to save some credit
Signed-off-by: Yuan <[email protected]>
---------
Signed-off-by: glutenperfbot <[email protected]>
Signed-off-by: Yuan <[email protected]>
Co-authored-by: glutenperfbot <[email protected]>
Co-authored-by: Yuan <[email protected]>
---
.github/workflows/velox_backend_enhanced.yml | 130 +++++++++++++--------------
ep/build-velox/src/get-velox.sh | 4 +-
2 files changed, 67 insertions(+), 67 deletions(-)
diff --git a/.github/workflows/velox_backend_enhanced.yml
b/.github/workflows/velox_backend_enhanced.yml
index e84de81c4e..a38e890066 100644
--- a/.github/workflows/velox_backend_enhanced.yml
+++ b/.github/workflows/velox_backend_enhanced.yml
@@ -229,68 +229,68 @@ jobs:
**/gluten-ut/**/hs_err_*.log
**/gluten-ut/**/core.*
- spark-test-spark40:
- needs: [detect-changes, build-native-lib-centos-8]
- if: >-
- needs.detect-changes.outputs.java == 'true' ||
- needs.detect-changes.outputs.shims40 == 'true' ||
- needs.detect-changes.outputs.cpp == 'true'
- runs-on: ubuntu-22.04
- strategy:
- fail-fast: false
- matrix:
- group: [1, 2, 3]
- env:
- SPARK_TESTING: true
- container: apache/gluten:centos-8-jdk17
- steps:
- - uses: actions/checkout@v7
- - name: Download All Artifacts
- uses: actions/download-artifact@v8
- with:
- name: velox-native-lib-enhanced-centos-8-${{github.sha}}
- path: ./cpp/build/releases/
- - name: Prepare
- run: |
- dnf module -y install python39 && \
- alternatives --set python3 /usr/bin/python3.9 && \
- pip3 install setuptools==77.0.3 && \
- pip3 install pyspark==3.5.5 cython && \
- pip3 install pandas==2.2.3 pyarrow==20.0.0
- - name: Build and Run unit test for Spark 4.0.0 with scala-2.13 (other
tests, group ${{ matrix.group }})
- run: |
- cd $GITHUB_WORKSPACE/
- export SPARK_SCALA_VERSION=2.13
- yum install -y java-17-openjdk-devel
- export JAVA_HOME=/usr/lib/jvm/java-17-openjdk
- export PATH=$JAVA_HOME/bin:$PATH
- java -version
-
TAGS_EXCLUDE="org.apache.spark.tags.ExtendedSQLTest,org.apache.spark.tags.SlowHiveTest,org.apache.gluten.tags.UDFTest,org.apache.gluten.tags.CudfTest,org.apache.gluten.tags.SkipTest"
- if [ "${{ matrix.group }}" = "1" ]; then
- $MVN_CMD clean test -Pspark-4.0 -Pscala-2.13 -Pjava-17
-Pbackends-velox -Piceberg -Pdelta \
- -Pspark-ut
-DargLine="-Dspark.test.home=/opt/shims/spark40/spark_home/"
-DtagsToExclude="$TAGS_EXCLUDE" \
-
-DwildcardSuites="org.apache.spark.sql.streaming,org.apache.spark.GlutenSortShuffleSuite,org.apache.gluten"
- elif [ "${{ matrix.group }}" = "2" ]; then
- $MVN_CMD clean test -Pspark-4.0 -Pscala-2.13 -Pjava-17
-Pbackends-velox -Piceberg -Pdelta \
- -Pspark-ut
-DargLine="-Dspark.test.home=/opt/shims/spark40/spark_home/"
-DtagsToExclude="$TAGS_EXCLUDE" \
-
-DwildcardSuites="org.apache.spark.sql.execution,org.apache.spark.sql.catalyst,org.apache.spark.sql.errors,org.apache.spark.sql.extension"
- else
- $MVN_CMD clean test -Pspark-4.0 -Pscala-2.13 -Pjava-17
-Pbackends-velox -Piceberg -Pdelta \
- -Pspark-ut
-DargLine="-Dspark.test.home=/opt/shims/spark40/spark_home/"
-DtagsToExclude="$TAGS_EXCLUDE" \
-
-DwildcardSuites="org.apache.spark.sql.GlutenSQL,org.apache.spark.sql.Gluten,org.apache.spark.sql.connector,org.apache.spark.sql.sources,org.apache.spark.sql.hive,org.apache.spark.sql.gluten,org.apache.spark.sql.shim"
- fi
- - name: Upload test report
- if: always()
- uses: actions/upload-artifact@v7
- with:
- name: spark-test-spark40-group${{ matrix.group }}-report
- path: '**/surefire-reports/TEST-*.xml'
- - name: Upload unit tests log files
- if: ${{ !success() }}
- uses: actions/upload-artifact@v7
- with:
- name: spark-test-spark40-group${{ matrix.group }}-test-log
- path: |
- **/target/*.log
- **/gluten-ut/**/hs_err_*.log
- **/gluten-ut/**/core.*
+ # spark-test-spark40:
+ # needs: [detect-changes, build-native-lib-centos-8]
+ # if: >-
+ # needs.detect-changes.outputs.java == 'true' ||
+ # needs.detect-changes.outputs.shims40 == 'true' ||
+ # needs.detect-changes.outputs.cpp == 'true'
+ # runs-on: ubuntu-22.04
+ # strategy:
+ # fail-fast: false
+ # matrix:
+ # group: [1, 2, 3]
+ # env:
+ # SPARK_TESTING: true
+ # container: apache/gluten:centos-8-jdk17
+ # steps:
+ # - uses: actions/checkout@v7
+ # - name: Download All Artifacts
+ # uses: actions/download-artifact@v8
+ # with:
+ # name: velox-native-lib-enhanced-centos-8-${{github.sha}}
+ # path: ./cpp/build/releases/
+ # - name: Prepare
+ # run: |
+ # dnf module -y install python39 && \
+ # alternatives --set python3 /usr/bin/python3.9 && \
+ # pip3 install setuptools==77.0.3 && \
+ # pip3 install pyspark==3.5.5 cython && \
+ # pip3 install pandas==2.2.3 pyarrow==20.0.0
+ # - name: Build and Run unit test for Spark 4.0.0 with scala-2.13 (other
tests, group ${{ matrix.group }})
+ # run: |
+ # cd $GITHUB_WORKSPACE/
+ # export SPARK_SCALA_VERSION=2.13
+ # yum install -y java-17-openjdk-devel
+ # export JAVA_HOME=/usr/lib/jvm/java-17-openjdk
+ # export PATH=$JAVA_HOME/bin:$PATH
+ # java -version
+ #
TAGS_EXCLUDE="org.apache.spark.tags.ExtendedSQLTest,org.apache.spark.tags.SlowHiveTest,org.apache.gluten.tags.UDFTest,org.apache.gluten.tags.CudfTest,org.apache.gluten.tags.SkipTest"
+ # if [ "${{ matrix.group }}" = "1" ]; then
+ # $MVN_CMD clean test -Pspark-4.0 -Pscala-2.13 -Pjava-17
-Pbackends-velox -Piceberg -Pdelta \
+ # -Pspark-ut
-DargLine="-Dspark.test.home=/opt/shims/spark40/spark_home/"
-DtagsToExclude="$TAGS_EXCLUDE" \
+ #
-DwildcardSuites="org.apache.spark.sql.streaming,org.apache.spark.GlutenSortShuffleSuite,org.apache.gluten"
+ # elif [ "${{ matrix.group }}" = "2" ]; then
+ # $MVN_CMD clean test -Pspark-4.0 -Pscala-2.13 -Pjava-17
-Pbackends-velox -Piceberg -Pdelta \
+ # -Pspark-ut
-DargLine="-Dspark.test.home=/opt/shims/spark40/spark_home/"
-DtagsToExclude="$TAGS_EXCLUDE" \
+ #
-DwildcardSuites="org.apache.spark.sql.execution,org.apache.spark.sql.catalyst,org.apache.spark.sql.errors,org.apache.spark.sql.extension"
+ # else
+ # $MVN_CMD clean test -Pspark-4.0 -Pscala-2.13 -Pjava-17
-Pbackends-velox -Piceberg -Pdelta \
+ # -Pspark-ut
-DargLine="-Dspark.test.home=/opt/shims/spark40/spark_home/"
-DtagsToExclude="$TAGS_EXCLUDE" \
+ #
-DwildcardSuites="org.apache.spark.sql.GlutenSQL,org.apache.spark.sql.Gluten,org.apache.spark.sql.connector,org.apache.spark.sql.sources,org.apache.spark.sql.hive,org.apache.spark.sql.gluten,org.apache.spark.sql.shim"
+ # fi
+ # - name: Upload test report
+ # if: always()
+ # uses: actions/upload-artifact@v7
+ # with:
+ # name: spark-test-spark40-group${{ matrix.group }}-report
+ # path: '**/surefire-reports/TEST-*.xml'
+ # - name: Upload unit tests log files
+ # if: ${{ !success() }}
+ # uses: actions/upload-artifact@v7
+ # with:
+ # name: spark-test-spark40-group${{ matrix.group }}-test-log
+ # path: |
+ # **/target/*.log
+ # **/gluten-ut/**/hs_err_*.log
+ # **/gluten-ut/**/core.*
diff --git a/ep/build-velox/src/get-velox.sh b/ep/build-velox/src/get-velox.sh
index 71f1610c3b..de4b1063eb 100755
--- a/ep/build-velox/src/get-velox.sh
+++ b/ep/build-velox/src/get-velox.sh
@@ -18,8 +18,8 @@ set -exu
CURRENT_DIR=$(cd "$(dirname "$BASH_SOURCE")"; pwd)
VELOX_REPO=https://github.com/IBM/velox.git
-VELOX_BRANCH=dft-2026_10_10
-VELOX_ENHANCED_BRANCH=ibm-2026_10_10
+VELOX_BRANCH=dft-2026_10_13
+VELOX_ENHANCED_BRANCH=ibm-2026_10_13
VELOX_HOME=""
RUN_SETUP_SCRIPT=ON
ENABLE_ENHANCED_FEATURES=OFF
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
