This is an automated email from the ASF dual-hosted git repository.
github-merge-queue[bot] pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
from ccc780a640 chore: deprecate spark.comet.exec.memoryPool.fraction
(#6163)
add e8eedcb2ad perf: project cached batches by buffer selection, prune on
collated strings (#5543)
No new revisions were added by this update.
Summary of changes:
docs/source/user-guide/latest/in-memory-cache.md | 194 +++++
docs/source/user-guide/latest/index.rst | 1 +
pom.xml | 26 +
spark/pom.xml | 22 +
.../main/scala/org/apache/comet/CometConf.scala | 69 +-
.../org/apache/comet/CometKryoRegistrator.scala | 3 +-
.../arrow/ArrowCachedBatchSerializer.scala | 304 ++++---
.../sql/comet/execution/arrow/CachedBatchIpc.scala | 680 +++++++++++++++
.../org/apache/spark/sql/comet/util/Utils.scala | 83 +-
.../comet/exec/CometInMemoryCacheSuite.scala | 939 ++++++++++++++++++---
.../benchmark/CometInMemoryCacheBenchmark.scala | 436 +++++++++-
.../execution/arrow/CometCachedBatchHelper.scala | 367 ++++++--
12 files changed, 2721 insertions(+), 403 deletions(-)
create mode 100644 docs/source/user-guide/latest/in-memory-cache.md
create mode 100644
spark/src/main/scala/org/apache/spark/sql/comet/execution/arrow/CachedBatchIpc.scala
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
