This is an automated email from the ASF dual-hosted git repository.
dependabot[bot] pushed a change to branch
dependabot/cargo/native/main/object_store-0.14.1
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
omit f8cd5c938e chore(deps): bump object_store from 0.13.2 to 0.14.1 in
/native
add 8324fe97a9 fix: apply 1.11 Iceberg diff changes from #4991 to other
versions (#5020)
add 756ddd7e14 feat: Support Spark levenshtein expression in native
execution (#4105)
add 2e907e5602 feat: native collect_list / array_agg aggregate (#4720)
add 376676c7ab docs: show implementation kind for each expression (#5028)
add c7657745f0 fix: raise CAST_INVALID_INPUT for invalid calendar dates in
ANSI cast to date (#5014)
add 69df0f12b2 feat: add CalendarIntervalType support (#4898)
add 8a9473c569 feat: native randstr implementation compatible with Spark
(#5035)
add 41cb698da8 docs: add blog posts and talks page (#5043)
add 82180b87a6 docs: avoid "not a blocker" comments in reviews for review
skill (#5057)
add 1d3044f35b feat: support interval codegen dispatch for nested values
and native shuffle (#4976)
add 2c05a1598f refactor: group orphan `spark.comet.explain.*` configs
under one prefix (#5026)
add c27eed3398 docs: refresh stale issue references and normalize issue
link format (#5062)
add 6bfd5450af fix: materialize ConstantColumnVector on Comet's
serialize/export paths (#4532)
add 9e9e3b895c fix: make native pow implementation compatible with Spark
(#5033)
add f4ae9e026f refactor: unify shuffle configs under
`spark.comet.shuffle.*` prefix (#4986)
add 5fd68c219a docs: fix references to configuration keys that do not
exist (#5063)
add aebe70ba93 docs: define post-1.0 versioning policy and legacy config
process (#5056)
add 880cecfcac docs: expand tuning guide with performance and memory
configs (#4908)
add cf81ab37a4 test: add isolated and small-batch benchmarks for
columnar-to-row conversion (#5113)
add 1db954d085 fix: resolve Comet jar without hardcoding the SNAPSHOT
qualifier (#5108)
add ce84a24c19 docs: document known correctness issues in the
compatibility guide (#5085)
add e344f7490a docs: update post 1.0.0 roadmap (#5064)
add 038fd6e25c fix: honour Spark's legacy `null IN ()` behavior (#5127)
add 76b9d2eee4 feat: add codegen dispatch fallback for CometCast
incompatible/unsupported cases (including legacy config paths) (#5079)
add a1b022b52a fix: honor fail_on_error in native make_decimal (#5080)
add cde6052c53 fix: work around DataFusion 54.1.0 Parquet page-index
regression (#5132)
add 61b61de68c chore: stop enabling incompatible casts in plan stability
suite (#5139)
add 38c3f8bce3 fix: throw ARITHMETIC_OVERFLOW for Long.MinValue div -1
under ANSI mode (#5084)
add 4b091acc9a feat: support SampleExec natively for sampling without
replacement (#5110)
add 4c3f04805a feat: disable native columnar-to-row conversion by default
(#5114)
add 3c830f4de7 feat: expose Comet version as spark.comet.version runtime
config (#5049)
add 0b2a506083 fix: seed native Parquet scan reader options from session
config (#5107)
add 8782d95331 fix: raise REMAINDER_BY_ZERO for Float/Double under ANSI
mode (#5081)
add 2732fe5fa1 adjust logging levels on frequent callsites (#5155)
add dba2ce49e2 chore: replace deprecated symbol literals with $"col" in
tests (#5168)
add cfd8f1facd test: cover ArrayExists three-valued logic config (#5000)
add a0a5d29010 chore(deps): bump actions/stale from 10.4.0 to 11.0.0
(#5163)
add 2707f214c9 chore: remove Array -> Seq round trips flagged by the 2.13
copy deprecation (#5170)
add bfc57995c5 chore: supply explicit empty argument lists for 2.13 (#5175)
add 87f7070b3a chore: clean up Scala compiler warnings (#5141)
add c4074f0e93 refactor: drop redundant concat layer in single-partition
shuffle (#5004)
add e0de5820f8 fix: round on Int64 with scale <= -19 now overflows
correctly (#5082)
add cda9d29576 fix: match Spark's whitespace trim semantics for casts from
string to boolean, integral, float/double and decimal (#5150)
add d2a61bdf1c fix: disambiguate Iceberg scans that share a
metadata_location (#5180)
add 285a80385d chore: drop deprecated .toIterable from protobuf builder
calls (#5173)
add 1ce9df1e4f feat: native uuid() implementation compatible with Spark
(#5034)
add 0abe9667ba chore: bump Spark 3.5 to 3.5.9 (#5181)
add 60413eb52c feat: support Iceberg metadata columns _pos, _spec, _file,
and _partition (#4752)
add fdd558ae1c chore(deps): bump object_store from 0.13.2 to 0.14.1 in
/native
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (f8cd5c938e)
\
N -- N -- N
refs/heads/dependabot/cargo/native/main/object_store-0.14.1 (fdd558ae1c)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.claude/skills/audit-comet-expression/SKILL.md | 7 +-
.claude/skills/review-comet-pr/SKILL.md | 15 +-
.github/actions/setup-spark-builder/action.yaml | 2 +-
.github/workflows/ci.yml | 6 +-
.github/workflows/iceberg_spark_test_reusable.yml | 2 +-
.github/workflows/pr_build_linux.yml | 3 +
.github/workflows/pr_build_macos.yml | 3 +
.github/workflows/spark_sql_test_reusable.yml | 2 +-
.github/workflows/spark_sql_writer_tests.yml | 2 +-
.github/workflows/stale.yml | 2 +-
README.md | 2 +-
benchmarks/README.md | 7 +-
benchmarks/pyspark/README.md | 6 +-
benchmarks/pyspark/benchmarks/base.py | 2 +-
benchmarks/pyspark/run_all_benchmarks.sh | 23 +-
benchmarks/tpc/engines/comet-iceberg-hashjoin.toml | 2 +-
benchmarks/tpc/engines/comet-iceberg.toml | 2 +-
benchmarks/tpc/infra/docker/Dockerfile.build-comet | 2 +-
dev/ci/compute-changes.py | 2 +-
dev/diffs/3.4.3.diff | 79 +-
dev/diffs/{3.5.8.diff => 3.5.9.diff} | 145 +--
dev/diffs/4.0.2.diff | 62 +-
dev/diffs/4.1.2.diff | 81 +-
dev/diffs/iceberg/1.10.0.diff | 39 +
dev/diffs/iceberg/1.11.0.diff | 14 +
dev/diffs/iceberg/1.8.1.diff | 39 +
dev/diffs/iceberg/1.9.1.diff | 39 +
docs/source/about/blogs-and-talks.md | 41 +
docs/source/about/index.md | 2 +
docs/source/about/versioning_policy.md | 257 +++++-
.../contributor-guide/adding_a_new_expression.md | 27 +-
.../contributor-guide/adding_a_new_operator.md | 4 +-
.../adding_a_new_spark_version.md | 2 +-
.../contributor-guide/benchmark-results/tpc-ds.md | 2 +-
.../contributor-guide/benchmark-results/tpc-h.md | 2 +-
.../source/contributor-guide/benchmarking_macos.md | 4 +-
.../benchmarking_spark_sql_perf.md | 5 +-
docs/source/contributor-guide/bug_triage.md | 13 +-
.../source/contributor-guide/config_conventions.md | 47 +
.../expression-audits/agg_funcs.md | 11 +-
.../expression-audits/array_funcs.md | 6 +-
.../expression-audits/bitwise_funcs.md | 2 +-
.../expression-audits/collection_funcs.md | 8 +-
.../expression-audits/conversion_funcs.md | 16 +-
.../expression-audits/datetime_funcs.md | 4 +-
.../expression-audits/generator_funcs.md | 4 +-
.../expression-audits/json_funcs.md | 4 +-
.../expression-audits/map_funcs.md | 4 +-
.../expression-audits/math_funcs.md | 12 +-
.../expression-audits/misc_funcs.md | 14 +
.../expression-audits/predicate_funcs.md | 4 +-
.../expression-audits/string_funcs.md | 64 +-
.../expression-audits/url_funcs.md | 2 +-
.../expression-audits/window_funcs.md | 2 +-
.../contributor-guide/iceberg-spark-tests.md | 2 +-
docs/source/contributor-guide/jvm_shuffle.md | 12 +-
docs/source/contributor-guide/native_shuffle.md | 34 +-
docs/source/contributor-guide/release_process.md | 28 +-
docs/source/contributor-guide/roadmap.md | 118 ++-
.../s3-credential-provider-design.md | 2 +-
.../contributor-guide/spark_configs_support.md | 34 +-
.../expressions/_category_template/cast.md | 27 +-
.../user-guide/latest/compatibility/index.md | 54 ++
.../user-guide/latest/compatibility/operators.md | 18 +-
.../user-guide/latest/compatibility/scans.md | 16 +-
.../latest/compatibility/spark-versions.md | 2 +-
docs/source/user-guide/latest/datasources.md | 2 +-
docs/source/user-guide/latest/datatypes.md | 20 +-
docs/source/user-guide/latest/expressions.md | 981 +++++++++++----------
docs/source/user-guide/latest/iceberg.md | 4 +-
docs/source/user-guide/latest/index.rst | 1 +
docs/source/user-guide/latest/installation.md | 35 +-
docs/source/user-guide/latest/kubernetes.md | 4 +-
docs/source/user-guide/latest/metrics.md | 2 +-
docs/source/user-guide/latest/migration-guide.md | 63 ++
docs/source/user-guide/latest/operators.md | 43 +-
docs/source/user-guide/latest/pyarrow-udfs.md | 9 +-
docs/source/user-guide/latest/tuning.md | 142 ++-
.../user-guide/latest/understanding-comet-plans.md | 40 +-
native/Cargo.lock | 6 +-
native/Cargo.toml | 4 +-
native/common/src/error.rs | 2 +-
native/core/Cargo.toml | 1 +
.../core/src/execution/expressions/arithmetic.rs | 1 +
native/core/src/execution/expressions/random.rs | 36 +-
native/core/src/execution/jni_api.rs | 25 +-
.../core/src/execution/operators/iceberg_scan.rs | 1 +
native/core/src/execution/operators/mod.rs | 2 +
native/core/src/execution/operators/sample.rs | 224 +++++
native/core/src/execution/planner.rs | 366 +++++++-
.../src/execution/planner/expression_registry.rs | 8 +
.../src/execution/planner/operator_registry.rs | 1 +
native/core/src/execution/serde.rs | 3 +
.../src/parquet/eager_page_index_reader_factory.rs | 193 ++++
native/core/src/parquet/mod.rs | 1 +
native/core/src/parquet/parquet_exec.rs | 134 ++-
native/proto/src/proto/expr.proto | 31 +
native/proto/src/proto/operator.proto | 23 +-
native/proto/src/proto/types.proto | 1 +
native/shuffle/benches/shuffle_writer.rs | 45 +-
.../shuffle/src/partitioners/single_partition.rs | 97 +-
native/shuffle/src/shuffle_writer.rs | 89 +-
native/spark-expr/Cargo.toml | 9 +
native/spark-expr/benches/cast_from_string.rs | 199 +++--
native/spark-expr/benches/cast_string_to_date.rs | 18 +-
.../spark-expr/benches/cast_string_to_timestamp.rs | 204 +++++
native/spark-expr/benches/common/mod.rs | 43 +
native/spark-expr/benches/decimal_div.rs | 1 +
native/spark-expr/benches/round.rs | 87 ++
native/spark-expr/src/agg_funcs/avg_decimal.rs | 9 +-
native/spark-expr/src/comet_scalar_funcs.rs | 28 +-
native/spark-expr/src/conversion_funcs/mod.rs | 1 +
native/spark-expr/src/conversion_funcs/string.rs | 347 ++++++--
native/spark-expr/src/conversion_funcs/trim.rs | 192 ++++
native/spark-expr/src/lib.rs | 6 +-
native/spark-expr/src/math_funcs/div.rs | 33 +-
.../src/math_funcs/internal/make_decimal.rs | 91 +-
native/spark-expr/src/math_funcs/mod.rs | 2 +
native/spark-expr/src/math_funcs/modulo_expr.rs | 547 +++++++++++-
native/spark-expr/src/math_funcs/pow.rs | 314 +++++++
native/spark-expr/src/math_funcs/round.rs | 255 +++++-
.../bernoulli_cell_sampler.rs | 93 ++
.../nondetermenistic_funcs/internal/mersenne.rs | 172 ++++
.../src/nondetermenistic_funcs/internal/mod.rs | 1 +
.../spark-expr/src/nondetermenistic_funcs/mod.rs | 6 +
.../spark-expr/src/nondetermenistic_funcs/rand.rs | 6 +
.../src/nondetermenistic_funcs/randstr.rs | 253 ++++++
.../src/nondetermenistic_funcs/shuffle.rs | 195 +---
.../spark-expr/src/nondetermenistic_funcs/uuid.rs | 290 ++++++
native/spark-expr/src/string_funcs/levenshtein.rs | 485 ++++++++++
native/spark-expr/src/string_funcs/mod.rs | 2 +
pom.xml | 6 +-
.../apache/comet/CometShuffleBlockIterator.java | 2 +-
.../java/org/apache/comet/annotation/Public.java | 47 +
.../apache/comet/cloud/s3/CometS3AccessMode.java | 3 +
.../comet/cloud/s3/CometS3CredentialContext.java | 3 +
.../comet/cloud/s3/CometS3CredentialProvider.java | 3 +
.../apache/comet/cloud/s3/CometS3Credentials.java | 3 +
.../org/apache/comet/vector/CometPlainVector.java | 75 ++
.../shuffle/sort/CometShuffleExternalSorter.java | 9 +-
.../execution/shuffle/CometDiskBlockWriter.java | 11 +-
.../sql/comet/execution/shuffle/SpillWriter.java | 2 +-
.../main/scala/org/apache/comet/CometConf.scala | 130 +--
.../scala/org/apache/comet/CometExecIterator.scala | 6 +-
.../apache/comet/CometSparkSessionExtensions.scala | 24 +-
.../scala/org/apache/comet/DataTypeSupport.scala | 4 +-
.../org/apache/comet/ExtendedExplainInfo.scala | 2 +
.../main/scala/org/apache/comet/GenerateDocs.scala | 195 +++-
.../comet/codegen/CometBatchKernelCodegen.scala | 3 +-
.../codegen/CometBatchKernelCodegenInput.scala | 43 +-
.../codegen/CometBatchKernelCodegenOutput.scala | 15 +-
.../codegen/CometSpecializedGettersDispatch.scala | 6 +-
.../org/apache/comet/expressions/CometCast.scala | 38 +-
.../apache/comet/iceberg/IcebergReflection.scala | 84 ++
.../org/apache/comet/rules/CometExecRule.scala | 42 +-
.../org/apache/comet/rules/CometScanRule.scala | 87 +-
.../org/apache/comet/serde/QueryPlanSerde.scala | 32 +-
.../scala/org/apache/comet/serde/aggregates.scala | 73 +-
.../scala/org/apache/comet/serde/arithmetic.scala | 22 +-
.../scala/org/apache/comet/serde/datetime.scala | 14 +
.../apache/comet/serde/decimalExpressions.scala | 8 +
.../main/scala/org/apache/comet/serde/maps.scala | 12 +
.../main/scala/org/apache/comet/serde/math.scala | 8 -
.../org/apache/comet/serde/nondetermenistic.scala | 28 +-
.../serde/operator/CometIcebergNativeScan.scala | 299 ++++---
.../comet/serde/operator/CometNativeScan.scala | 29 +-
.../apache/comet/serde/operator/CometSink.scala | 18 +-
.../org/apache/comet/serde/operator/package.scala | 2 +-
.../scala/org/apache/comet/serde/predicates.scala | 74 +-
.../scala/org/apache/comet/serde/strings.scala | 32 +-
.../scala/org/apache/comet/serde/unixtime.scala | 4 +-
.../apache/comet/testing/FuzzDataGenerator.scala | 17 +-
.../comet/udf/codegen/CometScalaUDFCodegen.scala | 2 +-
.../scala/org/apache/comet/vector/NativeUtil.scala | 16 +
.../src/main/scala/org/apache/spark/Plugins.scala | 14 +-
.../sql/comet/CometBroadcastExchangeExec.scala | 2 +-
.../spark/sql/comet/CometCsvNativeScanExec.scala | 12 +-
.../sql/comet/CometIcebergNativeScanExec.scala | 8 +-
.../spark/sql/comet/CometLocalTableScanExec.scala | 10 +-
.../sql/comet/CometNativeColumnarToRowExec.scala | 11 +-
.../spark/sql/comet/CometNativeWriteExec.scala | 2 +-
.../apache/spark/sql/comet/CometSampleExec.scala | 128 +++
.../org/apache/spark/sql/comet/CometScanExec.scala | 16 +-
.../apache/spark/sql/comet/CometWindowExec.scala | 6 +-
.../sql/comet/execution/arrow/ArrowWriters.scala | 46 +-
.../shuffle/CometBlockStoreShuffleReader.scala | 2 +-
.../shuffle/CometNativeShuffleWriter.scala | 14 +-
.../shuffle/CometShuffleExchangeExec.scala | 25 +-
.../execution/shuffle/CometShuffleManager.scala | 4 +-
.../shuffle/NativeBatchDecoderIterator.scala | 2 +-
.../org/apache/spark/sql/comet/operators.scala | 42 +-
.../org/apache/spark/sql/comet/util/Utils.scala | 21 +-
...nternalRowShim.scala => CometCollectShim.scala} | 14 +-
...InternalRowShim.scala => CometSampleShim.scala} | 13 +-
.../org/apache/comet/shims/CometTypeShim.scala | 3 +
.../org/apache/comet/shims/CometCollectShim.scala} | 14 +-
.../org/apache/comet/shims/CometSampleShim.scala} | 13 +-
.../org/apache/comet/shims/CometCollectShim.scala} | 14 +-
.../org/apache/comet/shims/CometSampleShim.scala} | 13 +-
.../org/apache/comet/shims/CometCollectShim.scala} | 14 +-
.../org/apache/comet/shims/CometSampleShim.scala} | 13 +-
.../org/apache/comet/serde/CometRandStr.scala | 76 ++
.../org/apache/comet/shims/CometTypeShim.scala | 8 +-
.../apache/comet/shims/Spark4xCometExprShim.scala | 4 +-
.../resources/pyspark/benchmark_pyarrow_udf.py | 2 +-
spark/src/test/resources/pyspark/conftest.py | 10 +-
.../src/test/resources/pyspark/test_pyarrow_udf.py | 6 +-
.../expressions/aggregate/collect_list.sql | 437 +++++++++
.../aggregate/collect_respect_nulls.sql | 71 ++
.../sql-tests/expressions/array/exists.sql | 10 +
.../cast/cast_complex_types_to_string.sql | 40 +-
.../cast/cast_complex_types_to_string_legacy.sql | 25 +-
.../expressions/cast/cast_string_to_date_ansi.sql | 52 ++
.../expressions/cast/cast_string_trim.sql | 71 ++
...nge_partition_jvm.sql => calendar_interval.sql} | 29 +-
.../expressions/datetime/make_dt_interval.sql | 30 +
.../expressions/datetime/make_time_shuffle.sql | 4 +-
.../make_time_shuffle_range_partition_jvm.sql | 2 +-
.../make_time_shuffle_range_partition_native.sql | 2 +-
.../expressions/datetime/make_ym_interval.sql | 30 +
.../sql-tests/expressions/math/arithmetic_ansi.sql | 145 +++
.../resources/sql-tests/expressions/math/pow.sql | 33 +-
.../resources/sql-tests/expressions/misc/uuid.sql | 45 +
.../sql-tests/expressions/misc/uuid_with_seed.sql | 96 ++
.../sql-tests/expressions/string/levenshtein.sql | 30 +-
.../expressions/string/levenshtein_threshold.sql | 60 ++
.../sql-tests/expressions/string/randstr.sql | 95 ++
.../iceberg/metadata_column_partition.sql | 854 ++++++++++++++++++
.../approved-plans-v1_4-spark3_5/q44/extended.txt | 20 +-
.../approved-plans-v1_4-spark3_5/q58/extended.txt | 10 +-
.../approved-plans-v1_4-spark3_5/q67/extended.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q70/extended.txt | 8 +-
.../approved-plans-v1_4-spark3_5/q83/extended.txt | 2 +-
.../approved-plans-v1_4-spark4_0/q14b/extended.txt | 12 +-
.../approved-plans-v1_4-spark4_0/q44/extended.txt | 18 +-
.../approved-plans-v1_4-spark4_0/q54/extended.txt | 10 +-
.../approved-plans-v1_4-spark4_0/q58/extended.txt | 10 +-
.../approved-plans-v1_4-spark4_0/q6/extended.txt | 8 +-
.../q83.ansi/extended.txt | 2 +-
.../approved-plans-v1_4-spark4_1/q33/extended.txt | 2 +-
.../approved-plans-v1_4-spark4_1/q49/extended.txt | 2 +-
.../approved-plans-v1_4-spark4_1/q56/extended.txt | 2 +-
.../approved-plans-v1_4-spark4_1/q60/extended.txt | 2 +-
.../approved-plans-v1_4-spark4_1/q66/extended.txt | 2 +-
.../approved-plans-v1_4/q1/extended.txt | 2 +-
.../approved-plans-v1_4/q10/extended.txt | 10 +-
.../approved-plans-v1_4/q11/extended.txt | 2 +-
.../approved-plans-v1_4/q12/extended.txt | 2 +-
.../approved-plans-v1_4/q13/extended.txt | 4 +-
.../approved-plans-v1_4/q14a/extended.txt | 4 +-
.../approved-plans-v1_4/q14b/extended.txt | 12 +-
.../approved-plans-v1_4/q15/extended.txt | 2 +-
.../approved-plans-v1_4/q16/extended.txt | 2 +-
.../approved-plans-v1_4/q17/extended.txt | 2 +-
.../approved-plans-v1_4/q18/extended.txt | 2 +-
.../approved-plans-v1_4/q19/extended.txt | 2 +-
.../approved-plans-v1_4/q2/extended.txt | 2 +-
.../approved-plans-v1_4/q20/extended.txt | 2 +-
.../approved-plans-v1_4/q21/extended.txt | 2 +-
.../approved-plans-v1_4/q22/extended.txt | 2 +-
.../approved-plans-v1_4/q23a/extended.txt | 4 +-
.../approved-plans-v1_4/q23b/extended.txt | 4 +-
.../approved-plans-v1_4/q24a/extended.txt | 4 +-
.../approved-plans-v1_4/q24b/extended.txt | 4 +-
.../approved-plans-v1_4/q25/extended.txt | 2 +-
.../approved-plans-v1_4/q26/extended.txt | 4 +-
.../approved-plans-v1_4/q27/extended.txt | 4 +-
.../approved-plans-v1_4/q28/extended.txt | 14 +-
.../approved-plans-v1_4/q29/extended.txt | 2 +-
.../approved-plans-v1_4/q3/extended.txt | 2 +-
.../approved-plans-v1_4/q30/extended.txt | 2 +-
.../approved-plans-v1_4/q31/extended.txt | 2 +-
.../approved-plans-v1_4/q32/extended.txt | 4 +-
.../approved-plans-v1_4/q33/extended.txt | 2 +-
.../approved-plans-v1_4/q34/extended.txt | 2 +-
.../approved-plans-v1_4/q35/extended.txt | 10 +-
.../approved-plans-v1_4/q36/extended.txt | 2 +-
.../approved-plans-v1_4/q37/extended.txt | 2 +-
.../approved-plans-v1_4/q38/extended.txt | 2 +-
.../approved-plans-v1_4/q39a/extended.txt | 2 +-
.../approved-plans-v1_4/q39b/extended.txt | 2 +-
.../approved-plans-v1_4/q4/extended.txt | 2 +-
.../approved-plans-v1_4/q40/extended.txt | 2 +-
.../approved-plans-v1_4/q41/extended.txt | 2 +-
.../approved-plans-v1_4/q42/extended.txt | 2 +-
.../approved-plans-v1_4/q43/extended.txt | 2 +-
.../approved-plans-v1_4/q44/extended.txt | 14 +-
.../approved-plans-v1_4/q45/extended.txt | 6 +-
.../approved-plans-v1_4/q46/extended.txt | 2 +-
.../approved-plans-v1_4/q47/extended.txt | 2 +-
.../approved-plans-v1_4/q48/extended.txt | 2 +-
.../approved-plans-v1_4/q49/extended.txt | 2 +-
.../approved-plans-v1_4/q5/extended.txt | 2 +-
.../approved-plans-v1_4/q50/extended.txt | 2 +-
.../approved-plans-v1_4/q51/extended.txt | 2 +-
.../approved-plans-v1_4/q52/extended.txt | 2 +-
.../approved-plans-v1_4/q53/extended.txt | 2 +-
.../approved-plans-v1_4/q54/extended.txt | 10 +-
.../approved-plans-v1_4/q55/extended.txt | 2 +-
.../approved-plans-v1_4/q56/extended.txt | 2 +-
.../approved-plans-v1_4/q57/extended.txt | 2 +-
.../approved-plans-v1_4/q58/extended.txt | 6 +-
.../approved-plans-v1_4/q59/extended.txt | 2 +-
.../approved-plans-v1_4/q6/extended.txt | 8 +-
.../approved-plans-v1_4/q60/extended.txt | 2 +-
.../approved-plans-v1_4/q61/extended.txt | 2 +-
.../approved-plans-v1_4/q62/extended.txt | 2 +-
.../approved-plans-v1_4/q63/extended.txt | 2 +-
.../approved-plans-v1_4/q64/extended.txt | 2 +-
.../approved-plans-v1_4/q65/extended.txt | 2 +-
.../approved-plans-v1_4/q66/extended.txt | 2 +-
.../approved-plans-v1_4/q67/extended.txt | 2 +-
.../approved-plans-v1_4/q68/extended.txt | 2 +-
.../approved-plans-v1_4/q69/extended.txt | 2 +-
.../approved-plans-v1_4/q7/extended.txt | 4 +-
.../approved-plans-v1_4/q70/extended.txt | 2 +-
.../approved-plans-v1_4/q71/extended.txt | 2 +-
.../approved-plans-v1_4/q72/extended.txt | 2 +-
.../approved-plans-v1_4/q73/extended.txt | 2 +-
.../approved-plans-v1_4/q74/extended.txt | 2 +-
.../approved-plans-v1_4/q75/extended.txt | 2 +-
.../approved-plans-v1_4/q76/extended.txt | 2 +-
.../approved-plans-v1_4/q77/extended.txt | 2 +-
.../approved-plans-v1_4/q78/extended.txt | 2 +-
.../approved-plans-v1_4/q79/extended.txt | 2 +-
.../approved-plans-v1_4/q8/extended.txt | 2 +-
.../approved-plans-v1_4/q80/extended.txt | 2 +-
.../approved-plans-v1_4/q81/extended.txt | 2 +-
.../approved-plans-v1_4/q82/extended.txt | 2 +-
.../approved-plans-v1_4/q83/extended.txt | 2 +-
.../approved-plans-v1_4/q84/extended.txt | 2 +-
.../approved-plans-v1_4/q85/extended.txt | 4 +-
.../approved-plans-v1_4/q86/extended.txt | 2 +-
.../approved-plans-v1_4/q87/extended.txt | 2 +-
.../approved-plans-v1_4/q88/extended.txt | 2 +-
.../approved-plans-v1_4/q89/extended.txt | 2 +-
.../approved-plans-v1_4/q9/extended.txt | 22 +-
.../approved-plans-v1_4/q90/extended.txt | 2 +-
.../approved-plans-v1_4/q91/extended.txt | 2 +-
.../approved-plans-v1_4/q92/extended.txt | 4 +-
.../approved-plans-v1_4/q93/extended.txt | 2 +-
.../approved-plans-v1_4/q94/extended.txt | 2 +-
.../approved-plans-v1_4/q95/extended.txt | 2 +-
.../approved-plans-v1_4/q96/extended.txt | 2 +-
.../approved-plans-v1_4/q97/extended.txt | 2 +-
.../approved-plans-v1_4/q98/extended.txt | 2 +-
.../approved-plans-v1_4/q99/extended.txt | 2 +-
.../approved-plans-v2_7-spark3_5/q67a/extended.txt | 4 +-
.../approved-plans-v2_7-spark3_5/q70a/extended.txt | 20 +-
.../approved-plans-v2_7-spark4_0/q14/extended.txt | 12 +-
.../approved-plans-v2_7-spark4_0/q6/extended.txt | 8 +-
.../approved-plans-v2_7-spark4_1/q14a/extended.txt | 4 +-
.../approved-plans-v2_7-spark4_1/q49/extended.txt | 2 +-
.../approved-plans-v2_7-spark4_2/q77a/extended.txt | 26 +-
.../approved-plans-v2_7/q10a/extended.txt | 2 +-
.../approved-plans-v2_7/q11/extended.txt | 2 +-
.../approved-plans-v2_7/q12/extended.txt | 2 +-
.../approved-plans-v2_7/q14/extended.txt | 12 +-
.../approved-plans-v2_7/q14a/extended.txt | 12 +-
.../approved-plans-v2_7/q18a/extended.txt | 2 +-
.../approved-plans-v2_7/q20/extended.txt | 2 +-
.../approved-plans-v2_7/q22/extended.txt | 2 +-
.../approved-plans-v2_7/q22a/extended.txt | 2 +-
.../approved-plans-v2_7/q24/extended.txt | 4 +-
.../approved-plans-v2_7/q27a/extended.txt | 8 +-
.../approved-plans-v2_7/q34/extended.txt | 2 +-
.../approved-plans-v2_7/q35/extended.txt | 10 +-
.../approved-plans-v2_7/q35a/extended.txt | 2 +-
.../approved-plans-v2_7/q36a/extended.txt | 2 +-
.../approved-plans-v2_7/q47/extended.txt | 2 +-
.../approved-plans-v2_7/q49/extended.txt | 2 +-
.../approved-plans-v2_7/q51a/extended.txt | 2 +-
.../approved-plans-v2_7/q57/extended.txt | 2 +-
.../approved-plans-v2_7/q5a/extended.txt | 2 +-
.../approved-plans-v2_7/q6/extended.txt | 8 +-
.../approved-plans-v2_7/q64/extended.txt | 2 +-
.../approved-plans-v2_7/q67a/extended.txt | 2 +-
.../approved-plans-v2_7/q70a/extended.txt | 2 +-
.../approved-plans-v2_7/q72/extended.txt | 2 +-
.../approved-plans-v2_7/q74/extended.txt | 2 +-
.../approved-plans-v2_7/q75/extended.txt | 2 +-
.../approved-plans-v2_7/q77a/extended.txt | 2 +-
.../approved-plans-v2_7/q78/extended.txt | 2 +-
.../approved-plans-v2_7/q80a/extended.txt | 2 +-
.../approved-plans-v2_7/q86a/extended.txt | 2 +-
.../approved-plans-v2_7/q98/extended.txt | 2 +-
.../scala/org/apache/comet/CometCastSuite.scala | 74 +-
.../org/apache/comet/CometCodegenHOFSuite.scala | 14 +
.../scala/org/apache/comet/CometCodegenSuite.scala | 51 +-
.../scala/org/apache/comet/CometConfSuite.scala | 43 +
.../org/apache/comet/CometExpressionSuite.scala | 124 ++-
.../org/apache/comet/CometFuzzTestSuite.scala | 11 +-
.../org/apache/comet/CometIcebergNativeSuite.scala | 670 +++++++++++++-
.../comet/CometIcebergRewriteActionSuite.scala | 9 +-
.../org/apache/comet/CometPublicApiSuite.scala | 109 +++
.../comet/CometSparkSessionExtensionsSuite.scala | 8 +-
.../apache/comet/CometUuidExpressionSuite.scala | 65 ++
.../apache/comet/exec/CometAggregateSuite.scala | 161 +++-
.../comet/exec/CometColumnarShuffleSuite.scala | 40 +-
.../apache/comet/exec/CometExec3_4PlusSuite.scala | 2 +-
.../org/apache/comet/exec/CometExecSuite.scala | 118 ++-
.../org/apache/comet/exec/CometJoinSuite.scala | 2 +-
.../apache/comet/exec/CometNativeReaderSuite.scala | 122 ++-
.../comet/exec/CometNativeShuffleSuite.scala | 32 +-
.../apache/comet/exec/CometWindowExecSuite.scala | 10 +-
.../apache/comet/parquet/ParquetReadSuite.scala | 2 +-
.../apache/comet/rules/CometExecRuleSuite.scala | 2 +-
.../org/apache/comet/vector/NativeUtilSuite.scala | 90 ++
.../scala/org/apache/spark/CometPluginsSuite.scala | 15 +-
.../apache/spark/sql/CometTPCDSQuerySuite.scala | 2 +-
.../spark/sql/CometTPCDSQueryTestSuite.scala | 2 +-
.../org/apache/spark/sql/CometTPCHQuerySuite.scala | 2 +-
.../org/apache/spark/sql/CometTPCQueryBase.scala | 3 +-
.../apache/spark/sql/CometTPCQueryListBase.scala | 2 +-
.../scala/org/apache/spark/sql/CometTestBase.scala | 23 +-
.../src/test/scala/org/apache/spark/sql/TPCH.scala | 123 +--
.../test/scala/org/apache/spark/sql/Tables.scala | 13 +-
.../spark/sql/benchmark/CometBenchmarkBase.scala | 5 +-
.../CometBroadcastHashJoinBenchmark.scala | 3 +-
.../CometBroadcastNestedLoopJoinBenchmark.scala | 3 +-
.../sql/benchmark/CometC2RIsolatedBench.scala | 106 +++
.../benchmark/CometColumnarToRowBenchmark.scala | 76 +-
.../spark/sql/benchmark/CometExecBenchmark.scala | 43 +-
.../sql/benchmark/CometHashJoinBenchmark.scala | 3 +-
.../spark/sql/benchmark/CometReadBenchmark.scala | 2 +-
.../benchmark/CometRegExpExtractBenchmark.scala | 2 +-
.../sql/benchmark/CometShuffleBenchmark.scala | 53 +-
.../benchmark/CometSortMergeJoinBenchmark.scala | 3 +-
.../benchmark/CometStringExpressionBenchmark.scala | 4 +
.../spark/sql/comet/CometPlanStabilitySuite.scala | 8 +-
.../spark/sql/comet/ParquetEncryptionITCase.scala | 2 +-
.../spark/sql/comet/PlanDataInjectorSuite.scala | 90 ++
.../execution/arrow/CometArrowStreamSuite.scala | 33 +-
.../apache/spark/sql/comet/util/UtilsSuite.scala | 108 +++
.../org/apache/spark/sql/ShimCometTestBase.scala | 8 +
.../org/apache/spark/sql/ShimCometTestBase.scala | 8 +
.../org/apache/spark/sql/ShimCometTestBase.scala | 10 +-
437 files changed, 12797 insertions(+), 2640 deletions(-)
rename dev/diffs/{3.5.8.diff => 3.5.9.diff} (97%)
create mode 100644 docs/source/about/blogs-and-talks.md
create mode 100644 docs/source/user-guide/latest/migration-guide.md
create mode 100644 native/core/src/execution/operators/sample.rs
create mode 100644 native/core/src/parquet/eager_page_index_reader_factory.rs
create mode 100644 native/spark-expr/benches/cast_string_to_timestamp.rs
create mode 100644 native/spark-expr/benches/common/mod.rs
create mode 100644 native/spark-expr/benches/round.rs
create mode 100644 native/spark-expr/src/conversion_funcs/trim.rs
create mode 100644 native/spark-expr/src/math_funcs/pow.rs
create mode 100644
native/spark-expr/src/nondetermenistic_funcs/bernoulli_cell_sampler.rs
create mode 100644
native/spark-expr/src/nondetermenistic_funcs/internal/mersenne.rs
create mode 100644 native/spark-expr/src/nondetermenistic_funcs/randstr.rs
create mode 100644 native/spark-expr/src/nondetermenistic_funcs/uuid.rs
create mode 100644 native/spark-expr/src/string_funcs/levenshtein.rs
create mode 100644 spark/src/main/java/org/apache/comet/annotation/Public.java
create mode 100644
spark/src/main/scala/org/apache/spark/sql/comet/CometSampleExec.scala
copy
spark/src/main/spark-3.x/org/apache/comet/shims/{CometInternalRowShim.scala =>
CometCollectShim.scala} (62%)
copy
spark/src/main/spark-3.x/org/apache/comet/shims/{CometInternalRowShim.scala =>
CometSampleShim.scala} (71%)
copy
spark/src/main/{spark-3.x/org/apache/comet/shims/CometInternalRowShim.scala =>
spark-4.0/org/apache/comet/shims/CometCollectShim.scala} (62%)
copy
spark/src/main/{spark-3.x/org/apache/comet/shims/CometInternalRowShim.scala =>
spark-4.0/org/apache/comet/shims/CometSampleShim.scala} (71%)
copy
spark/src/main/{spark-3.x/org/apache/comet/shims/CometInternalRowShim.scala =>
spark-4.1/org/apache/comet/shims/CometCollectShim.scala} (62%)
copy
spark/src/main/{spark-3.x/org/apache/comet/shims/CometInternalRowShim.scala =>
spark-4.1/org/apache/comet/shims/CometSampleShim.scala} (71%)
copy
spark/src/main/{spark-3.x/org/apache/comet/shims/CometInternalRowShim.scala =>
spark-4.2/org/apache/comet/shims/CometCollectShim.scala} (62%)
copy
spark/src/main/{spark-3.x/org/apache/comet/shims/CometInternalRowShim.scala =>
spark-4.2/org/apache/comet/shims/CometSampleShim.scala} (69%)
create mode 100644
spark/src/main/spark-4.x/org/apache/comet/serde/CometRandStr.scala
create mode 100644
spark/src/test/resources/sql-tests/expressions/aggregate/collect_list.sql
create mode 100644
spark/src/test/resources/sql-tests/expressions/aggregate/collect_respect_nulls.sql
create mode 100644
spark/src/test/resources/sql-tests/expressions/cast/cast_string_to_date_ansi.sql
create mode 100644
spark/src/test/resources/sql-tests/expressions/cast/cast_string_trim.sql
copy
spark/src/test/resources/sql-tests/expressions/datetime/{make_time_shuffle_range_partition_jvm.sql
=> calendar_interval.sql} (55%)
create mode 100644 spark/src/test/resources/sql-tests/expressions/misc/uuid.sql
create mode 100644
spark/src/test/resources/sql-tests/expressions/misc/uuid_with_seed.sql
create mode 100644
spark/src/test/resources/sql-tests/expressions/string/levenshtein_threshold.sql
create mode 100644
spark/src/test/resources/sql-tests/expressions/string/randstr.sql
create mode 100644
spark/src/test/resources/sql-tests/iceberg/metadata_column_partition.sql
create mode 100644
spark/src/test/scala/org/apache/comet/CometPublicApiSuite.scala
create mode 100644
spark/src/test/scala/org/apache/comet/CometUuidExpressionSuite.scala
create mode 100644
spark/src/test/scala/org/apache/comet/vector/NativeUtilSuite.scala
create mode 100644
spark/src/test/scala/org/apache/spark/sql/benchmark/CometC2RIsolatedBench.scala
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
