This is an automated email from the ASF dual-hosted git repository.
yuanzhou pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git
The following commit(s) were added to refs/heads/main by this push:
new f51b7ab23 [VL] Daily Update Velox Version (2024_03_21) (#5064)
f51b7ab23 is described below
commit f51b7ab230a19fd9ddeab7eb53996c308c38b46a
Author: Gluten Performance Bot
<[email protected]>
AuthorDate: Thu Mar 21 22:21:50 2024 +0800
[VL] Daily Update Velox Version (2024_03_21) (#5064)
Velox changes:
```
64f9618c3 by Wei He, Fix exception thrown in custom result verifier in
WindowFuzzer (9189)
3db2a9ad8 by Nicholas Ormrod, Deshim //folly:dynamic to
//folly/json:dynamic in velox (8931)
43a552715 by Sergey Pershin, Add Task:driverCounts() (9166)
7e8105a0b by rui-mo, Fix gatherFromTimestampBuffer (9170)
5186d0bc9 by joey.ljy, Fix the CMakeLists.txt to sort the cpp files in
alphabetical order (9164)
6f09b8fe5 by Richard Barnes, Remove unused exception parameter from
velox/common/base/tests/StatusTest.cpp (9180)
b8c752abc by Richard Barnes, Enable `-Wmismatched-tags` in velox/PACKAGE
(9181)
2c044edfc by Richard Barnes, Remove unused variables in
velox/tpch/gen/dbgen/text.cpp (9142)
e574117a4 by Richard Barnes, Remove unused variables in
velox/connectors/hive/tests/HiveDataSinkTest.cpp
f99cf2858 by Masha Basmanova, Extract CppToType and CastTypeChecker from
Type.h (9161)
491c46fa3 by Krishna Pai, Fix build on ubuntu and gcc 11 (9163)
469e2da6c by Jacob Wujciak-Jens, Move Linux build, adapters and presto
fuzzer to gha (8734)
443c72dc2 by joey.ljy, Unregister the UDTs after use in ArrayViewTest and
ArrayWriterTest (9175)
35fd4b757 by Ma, Rong, Add make_ym_interval Spark function (8887)
fe0c1fd52 by Jialiang Tan, Threshold based spilling removal (9069)
07d638c1b by 8dukongjian, Fix regr_r2 failure in AggregationFuzzer (9173)
8981498a9 by Masha Basmanova, Replace fromKindToScalerType with
createScalarType (9162)
cd2523fac by Kevin Wilfong, Fix hang in MultiThreadedTaskCursor when Task
fails (9134)
5e077909f by xiaoxmeng, Table scan cleanup (9156)
21491d74a by Kevin Wilfong, Fix NPE caused by testingRunArbitration (9137)
1d95a474a by Kevin Wilfong, Fix bug in PrestoVectorSerializers when
serializing empty Vectors with complex types (9157)
29d0f317d by Wei He, Fix window fuzzer test (9153)
```
Co-authored-by: yan ma <[email protected]>
---
cpp/velox/compute/WholeStageResultIterator.cc | 10 ----------
ep/build-velox/src/get_velox.sh | 2 +-
2 files changed, 1 insertion(+), 11 deletions(-)
diff --git a/cpp/velox/compute/WholeStageResultIterator.cc
b/cpp/velox/compute/WholeStageResultIterator.cc
index 9c4367b72..8d106950d 100644
--- a/cpp/velox/compute/WholeStageResultIterator.cc
+++ b/cpp/velox/compute/WholeStageResultIterator.cc
@@ -42,10 +42,6 @@ const std::string kSpillStrategyDefaultValue = "auto";
const std::string kAggregationSpillEnabled =
"spark.gluten.sql.columnar.backend.velox.aggregationSpillEnabled";
const std::string kJoinSpillEnabled =
"spark.gluten.sql.columnar.backend.velox.joinSpillEnabled";
const std::string kOrderBySpillEnabled =
"spark.gluten.sql.columnar.backend.velox.orderBySpillEnabled";
-const std::string kAggregationSpillMemoryThreshold =
- "spark.gluten.sql.columnar.backend.velox.aggregationSpillMemoryThreshold";
-const std::string kJoinSpillMemoryThreshold =
"spark.gluten.sql.columnar.backend.velox.joinSpillMemoryThreshold";
-const std::string kOrderBySpillMemoryThreshold =
"spark.gluten.sql.columnar.backend.velox.orderBySpillMemoryThreshold";
const std::string kMaxSpillLevel =
"spark.gluten.sql.columnar.backend.velox.maxSpillLevel";
const std::string kMaxSpillFileSize =
"spark.gluten.sql.columnar.backend.velox.maxSpillFileSize";
const std::string kMinSpillRunSize =
"spark.gluten.sql.columnar.backend.velox.minSpillRunSize";
@@ -507,12 +503,6 @@ std::unordered_map<std::string, std::string>
WholeStageResultIterator::getQueryC
std::to_string(veloxCfg_->get<bool>(kJoinSpillEnabled, true));
configs[velox::core::QueryConfig::kOrderBySpillEnabled] =
std::to_string(veloxCfg_->get<bool>(kOrderBySpillEnabled, true));
- configs[velox::core::QueryConfig::kAggregationSpillMemoryThreshold] =
std::to_string(
- veloxCfg_->get<uint64_t>(kAggregationSpillMemoryThreshold, 0)); //
spill only when input doesn't fit
- configs[velox::core::QueryConfig::kJoinSpillMemoryThreshold] =
- std::to_string(veloxCfg_->get<uint64_t>(kJoinSpillMemoryThreshold,
0)); // spill only when input doesn't fit
- configs[velox::core::QueryConfig::kOrderBySpillMemoryThreshold] =
- std::to_string(veloxCfg_->get<uint64_t>(kOrderBySpillMemoryThreshold,
0)); // spill only when input doesn't fit
configs[velox::core::QueryConfig::kMaxSpillLevel] =
std::to_string(veloxCfg_->get<int32_t>(kMaxSpillLevel, 4));
configs[velox::core::QueryConfig::kMaxSpillFileSize] =
std::to_string(veloxCfg_->get<uint64_t>(kMaxSpillFileSize, 20L * 1024
* 1024));
diff --git a/ep/build-velox/src/get_velox.sh b/ep/build-velox/src/get_velox.sh
index c911f381b..0fee29c24 100755
--- a/ep/build-velox/src/get_velox.sh
+++ b/ep/build-velox/src/get_velox.sh
@@ -17,7 +17,7 @@
set -exu
VELOX_REPO=https://github.com/oap-project/velox.git
-VELOX_BRANCH=2024_03_20
+VELOX_BRANCH=2024_03_21
VELOX_HOME=""
#Set on run gluten on HDFS
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
