This is an automated email from the ASF dual-hosted git repository.
taiyangli pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git
from 0745303e49 [GLUTEN-8705][CH] Enable MemorySpillScheduler (#8706)
add 223996e8c8 [GLUTEN-8492][CH] Offload RangeExec (#8518)
No new revisions were added by this update.
Summary of changes:
.../clickhouse/ExtensionTableBuilder.java | 24 ++++
.../datasources/clickhouse/ExtensionTableNode.java | 2 +-
.../gluten/backendsapi/clickhouse/CHBackend.scala | 2 +
.../backendsapi/clickhouse/CHIteratorApi.scala | 8 +-
.../clickhouse/CHSparkPlanExecApi.scala | 3 +-
.../gluten/execution/CHRangeExecTransformer.scala | 107 +++++++++++++++++
.../execution/GlutenRangeExecPartition.scala | 12 +-
...csUpdater.scala => CHRangeMetricsUpdater.scala} | 10 +-
.../GlutenClickHouseMergeTreeWriteOnS3Suite.scala | 2 +-
.../GlutenClickHouseMergeTreeWriteSuite.scala | 6 +-
.../backendsapi/velox/VeloxIteratorApi.scala | 2 +-
.../Parser/RelParsers/ReadRelParser.cpp | 124 +++++++++++++++-----
.../local-engine/Parser/RelParsers/ReadRelParser.h | 13 ++-
.../Parser/RelParsers/StreamKafkaRelParser.cpp | 3 +
.../local-engine/Parser/SerializedPlanParser.cpp | 37 +++---
cpp-ch/local-engine/Storages/SourceFromRange.cpp | 130 +++++++++++++++++++++
.../{SourceFromJavaIter.h => SourceFromRange.h} | 48 +++++---
.../execution/kafka/GlutenKafkaScanSuite.scala | 12 +-
.../gluten/backendsapi/BackendSettingsApi.scala | 1 -
.../apache/gluten/backendsapi/IteratorApi.scala | 4 +-
.../gluten/backendsapi/SparkPlanExecApi.scala | 2 -
.../execution/BasicScanExecTransformer.scala | 4 +-
.../execution/GlutenWholeStageColumnarRDD.scala | 4 +-
.../gluten/execution/WholeStageTransformer.scala | 86 +++++++-------
.../utils/clickhouse/ClickHouseTestSettings.scala | 1 +
.../utils/clickhouse/ClickHouseTestSettings.scala | 1 +
.../gluten/utils/velox/VeloxTestSettings.scala | 1 -
.../utils/clickhouse/ClickHouseTestSettings.scala | 1 +
.../utils/clickhouse/ClickHouseTestSettings.scala | 2 +-
.../sql/execution/GlutenSQLRangeExecSuite.scala | 34 ++++--
30 files changed, 533 insertions(+), 153 deletions(-)
create mode 100644
backends-clickhouse/src/main/scala/org/apache/gluten/execution/CHRangeExecTransformer.scala
copy
backends-velox/src-iceberg/test/scala/org/apache/gluten/execution/VeloxIcebergSuite.scala
=>
backends-clickhouse/src/main/scala/org/apache/gluten/execution/GlutenRangeExecPartition.scala
(76%)
copy
backends-clickhouse/src/main/scala/org/apache/gluten/metrics/{GenerateMetricsUpdater.scala
=> CHRangeMetricsUpdater.scala} (87%)
create mode 100644 cpp-ch/local-engine/Storages/SourceFromRange.cpp
copy cpp-ch/local-engine/Storages/{SourceFromJavaIter.h => SourceFromRange.h}
(51%)
rename gluten-ut/{spark33 =>
test}/src/test/scala/org/apache/spark/sql/execution/GlutenSQLRangeExecSuite.scala
(70%)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
