This is an automated email from the ASF dual-hosted git repository.
yuanzhou pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git
The following commit(s) were added to refs/heads/main by this push:
new fb7ab0e46f [GLUTEN-6887][VL] Daily Update Velox Version (2026_03_08)
(#11717)
fb7ab0e46f is described below
commit fb7ab0e46fae582ca26c137658afb39228771470
Author: Gluten Performance Bot
<[email protected]>
AuthorDate: Mon Mar 9 08:43:30 2026 +0000
[GLUTEN-6887][VL] Daily Update Velox Version (2026_03_08) (#11717)
* [GLUTEN-6887][VL] Daily Update Velox Version (dft-2026_03_08)
Upstream Velox's New Commits:
d085d37b6 by Ke Wang, feat: Add hive footer speculative IO size config
(#16641)
bb7b75c93 by Yedidya Feldblum, refactor: Use explicit types in
Cursor::write (#16585)
1367a7516 by Zac Wen, misc: Fix typo `duckDbVefifySql` in test parameter
(#16668)
2dff55a1b by Zoltan Arnold Nagy, feat(cudf): Add varchar support to cuDF
min/max aggregation registry (#16651)
ba2f8a491 by Chengcheng Jin, fix(cudf): Fix Cudf spark aggregation function
registeration validation (#16614)
1df51f9df by Raymond Lin, feat(connector): Use named constants for
fileReadOps table identity keys in SplitReader (#16656)
3beb70d97 by Raymond Lin, feat(connector): Add dbName field to
HiveTableHandle and inject table identity into fileReadOps (#16647)
ade18bd8c by Shruti Shivakumar, fix(cudf): Set functionEngine to spark in
ExpressionEvaluatorSelection test (#16666)
fe8703d84 by Jimmy Lu, refactor: Remove VELOX_ENABLE_BACKWARD_COMPATIBILITY
shim for filterPushdownEnabled in HiveTableHandle (#16665)
bf98064e1 by Pratik Pugalia, build: Disable build-metrics workflow due to
defunct conbench service (#16648)
24ac0df5d by Pedro Eugenio Rocha Pedreira, test: Support null map rows in
createMapOfArraysVector() (#16657)
5bf47a4a5 by Kent Yao, feat(spark): Add index lambda support for Spark
filter function (#16334)
a9c30b5d7 by Amit Dutta, fix: Fix crashes, memory leaks, and format string
bugs in NumericHistogramAggregate (#16640)
3b263a79f by Han Yan, refactor: Refactor VectorSerde named serde API from
Kind to string (#16558)
c3061087d by hikey-dj, feat(cuDF): Add GPU batch concatenation for small
cuDF batches (#16201)
85f8d00c7 by Yizhou Yang, feat: Add Spark approx_percentile aggregate
function (#16320)
e4e6dc48c by Simon Krueger, build: Move StringCore include and
computeAndSetIsAscii to cpp (#16638)
a6f53d492 by Shruti Shivakumar, feat(cudf): Apply stream-related fixes to
cuDF (#16572)
e2a7f7dff by Pramod Satya, feat: Add variant comparison with null handling
mode (#15726)
d807c6e60 by Wangyang Guo, perf(dwio): Avoid redundant outputBuffer
clearing (#15998)
48d371d8a by Jimmy Lu, feat: Column extraction pushdown design proposal
(#16627)
885364593 by Ke Wang, feat: Add option to collect column decoding time for
selective readers (#16635)
7bc1be16a by n0r0shi, feat: Support decimal type for the Spark checked_add
and checked_subtract functions (#16302)
0c2b4c488 by Matt Gara, fix(cudf): Fix stream synchronization issue with
CudfTopN (#16431)
f2c269936 by Jimmy Lu, fix: Fix build failure caused by merge conflict
(#16639)
f6b7e8372 by lingbin, feat: Expose 'runningIsBlockedWallNanos' operator
runtime metric (#16021)
4d185d130 by Chengcheng Jin, feat(cudf): Register the functions of specific
engine separately (#16524)
f063ee7d4 by Jialiang Tan, refactor: Rename join terminology to more
generic terms in HiveIndexReader/HiveIndexSource (#16631)
6cea60a45 by Jimmy Lu, refactor: Remove deprecated filterPushdownEnabled
from HiveTableHandle (#16593)
6585e3575 by Andrii Rosa, fix: Sanitize file name parts when generating
file name in HiveDataSink (#16603)
6bbd589e5 by Shakyan Kushwaha, feat: Add current_time Presto function
(#15603)
23b491ec0 by Kent Yao, feat(sparksql): Add Spark-specific sequence function
for integer types (#16339)
13656603a by wuyuhao28, perf: Fix readRegions prefetch incorrectly
submitting non-prefetch loads (#16574)
2042c2a18 by Ke Wang, feat: Add column type info to decompression metrics
for dwrf reader (#16602)
c34853c59 by Krishna Pai, fix: Make document publishing job more robust
(#16619)
3a20d75a1 by Chengcheng Jin, feat(spark): Respect Spark bloom_filter_agg
max num items (#16526)
0f070c60b by Ke Wang, fix: Make cacheable default to false in FileIoContext
and BufferedInput (#16621)
e83521455 by Xiaoxuan Meng, fix: Add IoStatistics tracking for
findCachedRegion and use emplace_back in BufferedInput (#16604)
6b362a5dd by Xiaoxuan Meng, feat: Reland D94871321: [velox] feat: Add cache
lookup and zero-copy APIs to BufferedInput (#16606)
2e0e810b0 by Suryadev Sahadevan Rajesh, refactor(encoding): Merge
EncodingLayoutCapture into EncodingLayout (#16582)
d525b0d7b by Rohit Jain, refactor: Migrate remaining string constants to
string_view (#16455)
a8548d60d by zhli1142015, perf(expr): Remove batch size limit for
FlatNoNulls fast path (#16607)
e4a431e01 by Ping Liu, fix(parquet): Max target file size not working
(#16389)
a4a0ddb1d by Patrick Sullivan, refactor(geo): Use envelope checks to skip
deserialization (#16368)
ee9f5395a by Peter Enescu, feat: Add template information to
createEmptyLike (#16453)
6b9604173 by infvg, test: Revamp Spark cast tests to support ANSI on/off
differentiation (#16058)
d64d78858 by David Goode, test(operator): Add velox mixedunion hive
connector test (#16449)
4514fee81 by Pratik Pugalia, feat: Add IPADDRESS support for Fuzzers
(#16020)
1a3478f06 by Pratik Pugalia, feat: Add HUGEINT key support to
MapUnionSumAggregate (#16196)
296571388 by Pratik Pugalia, feat: Add IPADDRESS support to Arbitrary
aggregate function (#16195)
fb5b29684 by Ping Liu, fix: Iceberg positional delete compare base64
encoded value with raw value (#16592)
Signed-off-by: glutenperfbot <[email protected]>
* fix
Signed-off-by: Yuan <[email protected]>
* fix
Signed-off-by: Yuan <[email protected]>
* fix
Signed-off-by: Yuan <[email protected]>
* fix
Signed-off-by: Yuan <[email protected]>
* fix
Signed-off-by: Yuan <[email protected]>
---------
Signed-off-by: glutenperfbot <[email protected]>
Signed-off-by: Yuan <[email protected]>
Co-authored-by: glutenperfbot <[email protected]>
Co-authored-by: Yuan <[email protected]>
---
cpp/velox/compute/VeloxBackend.cc | 2 +-
cpp/velox/shuffle/VeloxShuffleReader.cc | 2 +-
cpp/velox/substrait/SubstraitToVeloxPlan.cc | 4 +---
cpp/velox/utils/ConfigExtractor.cc | 4 +---
ep/build-velox/src/get-velox.sh | 4 ++--
5 files changed, 6 insertions(+), 10 deletions(-)
diff --git a/cpp/velox/compute/VeloxBackend.cc
b/cpp/velox/compute/VeloxBackend.cc
index 45a64908e1..de9e9385f8 100644
--- a/cpp/velox/compute/VeloxBackend.cc
+++ b/cpp/velox/compute/VeloxBackend.cc
@@ -201,7 +201,7 @@ void VeloxBackend::init(
// serde, for spill
facebook::velox::serializer::presto::PrestoVectorSerde::registerVectorSerde();
}
- if
(!isRegisteredNamedVectorSerde(facebook::velox::VectorSerde::Kind::kPresto)) {
+ if (!isRegisteredNamedVectorSerde("Presto")) {
// RSS shuffle serde.
facebook::velox::serializer::presto::PrestoVectorSerde::registerNamedVectorSerde();
}
diff --git a/cpp/velox/shuffle/VeloxShuffleReader.cc
b/cpp/velox/shuffle/VeloxShuffleReader.cc
index bfe11ef3e9..e5435aee52 100644
--- a/cpp/velox/shuffle/VeloxShuffleReader.cc
+++ b/cpp/velox/shuffle/VeloxShuffleReader.cc
@@ -732,7 +732,7 @@
VeloxRssSortShuffleReaderDeserializer::VeloxRssSortShuffleReaderDeserializer(
rowType_(rowType),
batchSize_(batchSize),
veloxCompressionType_(veloxCompressionType),
- serde_(getNamedVectorSerde(facebook::velox::VectorSerde::Kind::kPresto)),
+ serde_(getNamedVectorSerde("Presto")),
deserializeTime_(deserializeTime) {
serdeOptions_ = {false, veloxCompressionType_};
}
diff --git a/cpp/velox/substrait/SubstraitToVeloxPlan.cc
b/cpp/velox/substrait/SubstraitToVeloxPlan.cc
index 727f4882e1..d71ab12528 100644
--- a/cpp/velox/substrait/SubstraitToVeloxPlan.cc
+++ b/cpp/velox/substrait/SubstraitToVeloxPlan.cc
@@ -1447,8 +1447,6 @@ core::PlanNodePtr
SubstraitToVeloxPlanConverter::toVeloxPlan(const ::substrait::
SubstraitParser::parseColumnTypes(baseSchema, columnTypes);
}
- // Velox requires Filter Pushdown must being enabled.
- bool filterPushdownEnabled = true;
auto names = colNameList;
auto types = veloxTypeList;
@@ -1468,7 +1466,7 @@ core::PlanNodePtr
SubstraitToVeloxPlanConverter::toVeloxPlan(const ::substrait::
}
common::SubfieldFilters subfieldFilters;
tableHandle = std::make_shared<connector::hive::HiveTableHandle>(
- connectorId, "hive_table", filterPushdownEnabled,
std::move(subfieldFilters), remainingFilter, tableSchema);
+ connectorId, "hive_table", std::move(subfieldFilters), remainingFilter,
tableSchema);
// Get assignments and out names.
std::vector<std::string> outNames;
diff --git a/cpp/velox/utils/ConfigExtractor.cc
b/cpp/velox/utils/ConfigExtractor.cc
index 52e42d606d..fe35ce9f8e 100644
--- a/cpp/velox/utils/ConfigExtractor.cc
+++ b/cpp/velox/utils/ConfigExtractor.cc
@@ -292,9 +292,7 @@ std::shared_ptr<facebook::velox::config::ConfigBase>
createHiveConnectorConfig(
conf->get<std::string>(kPrefetchRowGroups, "1");
hiveConfMap[facebook::velox::connector::hive::HiveConfig::kLoadQuantum] =
conf->get<std::string>(kLoadQuantum, "268435456"); // 256M
- auto footerEstimatedSize = conf->get<std::string>(kDirectorySizeGuess,
"32768"); // 32K
-
hiveConfMap[facebook::velox::connector::hive::HiveConfig::kFooterEstimatedSize]
=
- conf->get<std::string>(kFooterEstimatedSize, footerEstimatedSize); // 32K
+
hiveConfMap[facebook::velox::connector::hive::HiveConfig::kFilePreloadThreshold]
=
conf->get<std::string>(kFilePreloadThreshold, "1048576"); // 1M
diff --git a/ep/build-velox/src/get-velox.sh b/ep/build-velox/src/get-velox.sh
index dbfe9f2d5b..558f988e5c 100755
--- a/ep/build-velox/src/get-velox.sh
+++ b/ep/build-velox/src/get-velox.sh
@@ -18,8 +18,8 @@ set -exu
CURRENT_DIR=$(cd "$(dirname "$BASH_SOURCE")"; pwd)
VELOX_REPO=https://github.com/IBM/velox.git
-VELOX_BRANCH=dft-2026_03_03-iceberg
-VELOX_ENHANCED_BRANCH=ibm-2026_03_03
+VELOX_BRANCH=dft-2026_03_08-iceberg
+VELOX_ENHANCED_BRANCH=ibm-2026_03_08
VELOX_HOME=""
RUN_SETUP_SCRIPT=ON
ENABLE_ENHANCED_FEATURES=OFF
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
