This is an automated email from the ASF dual-hosted git repository.
zhouyuan pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/gluten.git
The following commit(s) were added to refs/heads/main by this push:
new 7b4c340513 [GLUTEN-6887][VL] Daily Update Velox Version (2026_10_08)
(#13221)
7b4c340513 is described below
commit 7b4c34051350fe057350f8c1f5d69ff0fc24a76e
Author: Gluten Performance Bot
<[email protected]>
AuthorDate: Thu Oct 8 18:16:43 2026 +0100
[GLUTEN-6887][VL] Daily Update Velox Version (2026_10_08) (#13221)
* [GLUTEN-6887][VL] Daily Update Velox Version (dft-2026_10_08)
Upstream Velox's New Commits:
0905bb381 by Ke, feat(nimble): Cap string min/max length in vectorized
stats (#19407)
3f23f2386 by rdtr, feat(sparksql): Add Spark elt function
f35c2bd10 by Ke, fix(selective): Surface per-column decode and decompress
stats in Nimble selective reader (#19440)
d9c1afdd8 by Anshul Baliga, docs(sparksql): Fix directives for Spark
functions that did not cross-link
e10bf045a by Jake Jung, refactor(common): Make getClassName return
std::string_view (#19425)
0f6cc7295 by Devavret Makkar, feat(connector): Expose custom operator
memory pools in connector context
3dbeef9d2 by dependabot[bot], build(deps): Bump shell-quote from 1.8.4 to
1.12.0 in /website
114070257 by dependabot[bot], build(deps): Bump joi from 17.13.3 to 17.13.8
in /website
76f5d1db4 by dependabot[bot], build(deps): Bump brace-expansion from 1.1.18
to 1.1.21 in /website
fcf497710 by dependabot[bot], build(ci): Bump docker/setup-buildx-action
from 4.1.0 to 4.4.1
c70cfcf21 by dependabot[bot], build(deps): Bump fast-uri from 3.1.7 to
3.1.8 in /website
5fa331828 by dependabot[bot], build(deps): Bump compression from 1.8.1 to
1.8.2 in /website
4dc89357d by dependabot[bot], build(deps): Bump source-map-js from 1.2.1 to
1.2.2 in /website
deb55f24a by dependabot[bot], build(deps): Bump proxy-addr from 2.0.7 to
2.0.8 in /website
88afa7397 by dependabot[bot], build(ci): Bump docker/bake-action from 6.9.0
to 7.4.0
dd308d17e by Chandrashekhar Kumar Singh, fix(cursor): Synchronize producer
counts
f4c167b63 by Tina Du, feat(vector): Improve MSVC portability
99cf15ad7 by Denodo Research Labs, fix(build): Add libunwind dependency for
CentOS to fix missing stack traces
e2766e7f7 by Karthikeyan, feat(functions): Make Presto scalar call() bodies
device-compatible
6358057b8 by Joe Abraham, fix(iceberg): Apply row lineage filters after
synthesis
b8d4a029d by Pedro Pedreira, perf(dwio): Share the flat map key index
across reader batches
e9d06933d by Pedro Pedreira, perf(vector): Share FlatMapVector key index
with slices (#19408)
529eceb11 by Pedro Pedreira, fix(benchmark): Correct map_concat benchmark
measurements
264d0ba49 by shanhao-203, fix(nimble): Validate groups and index Default
keys (#19388)
3e696ed2b by Karthikeyan, fix(core): Synchronize QueryCtx release callback
registration
63107bd21 by Krishna Pai, fix(rpc): Stop capturing a structured binding in
RPCStateTest
a3dc50ab0 by Jake Jung, refactor: Build deserializer registry keys as
std::string (#19424)
3e48af039 by Jake Jung, refactor(hive): Take string_view in
serializeHiveFields (#19423)
84e9188f5 by Suryadev Sahadevan Rajesh, feat(nimble): Integrate tuned
SubIntSplit MyRocks layout (#19351)
4b1dde549 by Minhan Cao, fix(prestosql): Register between for BOOLEAN
3e36a647b by Bradley Dice, fix(cudf): Remove call to deleted shared
allocation option
dd682b9b5 by yun.wu, feat(exec): Add planNodeId to ScanBatchEvent
08e7e6fb2 by Krishna Pai, fix(fuzzer): Fix use-after-free in table
evolution fuzzer
892968225 by Maria Basmanova, fix(window): Compute k RANGE frames from
their bound values
7c5fe1741 by Xiaoxuan, docs(nimble): Parallel Encoding and Decoding in
Nimble: Faster Flushes and Reads
754bcb336 by Muhammad Haseeb, fix(cudf): Honor session schema reader options
6c77e68a0 by Karthikeyan, feat(cudf): Count rows from the Parquet footer
for scans with no columns
6dbc54b2e by Chandrashekhar Kumar Singh, fix(cursor): Synchronize injected
errors (#19406)
c594325f5 by Krishna Pai, fix(ci): Run debug tests on selectively built
pull requests
f52a3033a by RindsSchei225e, perf(writer): Generalize a cached Constant
layout to MainlyConstant (#19363)
be2c68a47 by RindsSchei225e, perf(encoding): Fall back to Constant when
MainlyConstant has no uncommon values (#19362)
4c8f62a7e by PRASHANT GOLASH, perf(dwio): Always use shared allocation for
eligible direct reads (#19339)
f5218cfee by Karthikeyan, perf(functions): Resolve from_unixtime's constant
zone once
175f2a655 by Muhammad Haseeb, fix(cudf): Honor direct buffered input shared
allocation
f909feccd by Pedro Pedreira, perf(vector): Build FlatMapVector key index
lazily (#19324)
0c5d1763e by Krishnakanth, fix(exec): Allow a null destroy function in
Accumulator
c5f1e5e0d by Apurva Kumar, fix: Emit partitionDataJson from DV sink commit
message (#19393)
278af3d28 by Bikramjeet Singh Vig, feat(connector): Add structured
exception context (#19284)
8051c46eb by Bikramjeet Singh Vig, feat(exception): Support subsystem-owned
context (#19283)
0a65370ba by Xiaoxuan, perf(nimble): Omit constant Hybrid FlatMap metadata
(#19374)
a202d90c6 by Muhammad Haseeb, fix(cudf): Specify column selection mode to
row group pass construction
ec7951d5d by zhichenxu-meta, feat(rpc): Pace fractional RPC admission
Signed-off-by: glutenperfbot <[email protected]>
* fix gpu code compile due to cudf update
Signed-off-by: Yuan <[email protected]>
* speedup hadoop download
Signed-off-by: Yuan <[email protected]>
---------
Signed-off-by: glutenperfbot <[email protected]>
Signed-off-by: Yuan <[email protected]>
Co-authored-by: glutenperfbot <[email protected]>
Co-authored-by: Yuan <[email protected]>
---
.github/workflows/util/install-spark-deps.sh | 10 +++++++++-
cpp/velox/utils/GpuBufferBatchResizer.cc | 14 +++++++-------
ep/build-velox/src/get-velox.sh | 4 ++--
3 files changed, 18 insertions(+), 10 deletions(-)
diff --git a/.github/workflows/util/install-spark-deps.sh
b/.github/workflows/util/install-spark-deps.sh
index 51036f4ddd..5ecdb6f1c9 100755
--- a/.github/workflows/util/install-spark-deps.sh
+++ b/.github/workflows/util/install-spark-deps.sh
@@ -30,7 +30,15 @@ function install_hadoop() {
apt-get install -y curl tar gzip
local HADOOP_VERSION=3.3.6
- curl -fsSL -o hadoop.tgz
"https://archive.apache.org/dist/hadoop/common/hadoop-${HADOOP_VERSION}/hadoop-${HADOOP_VERSION}.tar.gz";
+ local
url_path="hadoop/common/hadoop-${HADOOP_VERSION}/hadoop-${HADOOP_VERSION}.tar.gz"
+ # archive.apache.org is heavily throttled (a download can take hours), so
prefer the CDN.
+ # Abort the CDN transfer if it stalls below 1MB/s for 60s, then fall back to
the archive,
+ # which also serves versions that have been removed from the CDN.
+ if ! curl -fSL --retry 3 --connect-timeout 30 --speed-limit 1048576
--speed-time 60 \
+ -o hadoop.tgz "https://dlcdn.apache.org/${url_path}";; then
+ echo "Failed to download Hadoop from dlcdn.apache.org, falling back to
archive.apache.org"
+ curl -fSL --retry 3 --connect-timeout 30 -o hadoop.tgz
"https://archive.apache.org/dist/${url_path}";
+ fi
tar -xzf hadoop.tgz --no-same-owner --no-same-permissions
rm -f hadoop.tgz
diff --git a/cpp/velox/utils/GpuBufferBatchResizer.cc
b/cpp/velox/utils/GpuBufferBatchResizer.cc
index 5f56ba80e3..c09ad4ae7c 100644
--- a/cpp/velox/utils/GpuBufferBatchResizer.cc
+++ b/cpp/velox/utils/GpuBufferBatchResizer.cc
@@ -34,8 +34,8 @@
#include <cudf/null_mask.hpp>
#include <cudf/types.hpp>
#include <cudf/utilities/type_dispatcher.hpp>
-#include <rmm/cuda_stream_view.hpp>
#include <rmm/device_buffer.hpp>
+#include <cuda/stream>
using namespace facebook::velox;
@@ -44,7 +44,7 @@ namespace gluten {
namespace {
struct DispatchColumn {
- rmm::cuda_stream_view stream;
+ cuda::stream_ref stream;
rmm::device_async_resource_ref mr;
const std::vector<std::shared_ptr<arrow::Buffer>>& buffers;
const int32_t numRows;
@@ -57,7 +57,7 @@ struct DispatchColumn {
auto mask = cudf::create_null_mask(numRows,
cudf::mask_state::UNINITIALIZED, stream, mr);
auto copySize = std::min(static_cast<size_t>(buffer->size()), mask.size());
- CUDF_CUDA_TRY(cudaMemcpyAsync(mask.data(), buffer->data(), copySize,
cudaMemcpyHostToDevice, stream.value()));
+ CUDF_CUDA_TRY(cudaMemcpyAsync(mask.data(), buffer->data(), copySize,
cudaMemcpyHostToDevice, stream.get()));
return mask;
}
@@ -72,7 +72,7 @@ struct DispatchColumn {
// === Step 2: allocate GPU device buffers and copy ===
rmm::device_buffer dataBuf(values->size(), stream, mr);
CUDF_CUDA_TRY(
- cudaMemcpyAsync(dataBuf.data(), values->data(), values->size(),
cudaMemcpyHostToDevice, stream.value()));
+ cudaMemcpyAsync(dataBuf.data(), values->data(), values->size(),
cudaMemcpyHostToDevice, stream.get()));
auto nullBuf = getMaskBuffer(nulls);
@@ -92,7 +92,7 @@ struct DispatchColumn {
// --- 2. Copy offsets to GPU ---
rmm::device_buffer offsetBuf(offsets->size(), stream, mr);
CUDF_CUDA_TRY(
- cudaMemcpyAsync(offsetBuf.data(), offsets->data(), offsets->size(),
cudaMemcpyHostToDevice, stream.value()));
+ cudaMemcpyAsync(offsetBuf.data(), offsets->data(), offsets->size(),
cudaMemcpyHostToDevice, stream.get()));
// --- 3. Empty null mask (no nulls in offset column) ---
auto nullBuf = cudf::create_null_mask(0, cudf::mask_state::UNALLOCATED,
stream, mr);
@@ -125,8 +125,8 @@ struct DispatchColumn {
auto offsetColumn = getOffsetsColumn(offsets);
rmm::device_buffer chars(valueBuffer->size(), stream, mr);
- CUDF_CUDA_TRY(cudaMemcpyAsync(
- chars.data(), valueBuffer->data_as<uint8_t>(), chars.size(),
cudaMemcpyDefault, stream.value()));
+ CUDF_CUDA_TRY(
+ cudaMemcpyAsync(chars.data(), valueBuffer->data_as<uint8_t>(),
chars.size(), cudaMemcpyDefault, stream.get()));
return cudf::make_strings_column(numRows, std::move(offsetColumn),
std::move(chars), nullCount, std::move(mask));
}
};
diff --git a/ep/build-velox/src/get-velox.sh b/ep/build-velox/src/get-velox.sh
index 256f341ab1..8326404ea7 100755
--- a/ep/build-velox/src/get-velox.sh
+++ b/ep/build-velox/src/get-velox.sh
@@ -18,8 +18,8 @@ set -exu
CURRENT_DIR=$(cd "$(dirname "$BASH_SOURCE")"; pwd)
VELOX_REPO=https://github.com/IBM/velox.git
-VELOX_BRANCH=dft-2026_10_06
-VELOX_ENHANCED_BRANCH=ibm-2026_10_06
+VELOX_BRANCH=dft-2026_10_08
+VELOX_ENHANCED_BRANCH=ibm-2026_10_08
VELOX_HOME=""
RUN_SETUP_SCRIPT=ON
ENABLE_ENHANCED_FEATURES=OFF
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
