This is an automated email from the ASF dual-hosted git repository.
dependabot[bot] pushed a change to branch
dependabot/cargo/native/main/all-other-cargo-deps-4459ef99e2
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
omit ace48e36f5 chore(deps): bump reqsign-core
add 2732fe5fa1 adjust logging levels on frequent callsites (#5155)
add dba2ce49e2 chore: replace deprecated symbol literals with $"col" in
tests (#5168)
add cfd8f1facd test: cover ArrayExists three-valued logic config (#5000)
add a0a5d29010 chore(deps): bump actions/stale from 10.4.0 to 11.0.0
(#5163)
add 2707f214c9 chore: remove Array -> Seq round trips flagged by the 2.13
copy deprecation (#5170)
add bfc57995c5 chore: supply explicit empty argument lists for 2.13 (#5175)
add 87f7070b3a chore: clean up Scala compiler warnings (#5141)
add c4074f0e93 refactor: drop redundant concat layer in single-partition
shuffle (#5004)
add e0de5820f8 fix: round on Int64 with scale <= -19 now overflows
correctly (#5082)
add cda9d29576 fix: match Spark's whitespace trim semantics for casts from
string to boolean, integral, float/double and decimal (#5150)
add d2a61bdf1c fix: disambiguate Iceberg scans that share a
metadata_location (#5180)
add 285a80385d chore: drop deprecated .toIterable from protobuf builder
calls (#5173)
add 1ce9df1e4f feat: native uuid() implementation compatible with Spark
(#5034)
add 0abe9667ba chore: bump Spark 3.5 to 3.5.9 (#5181)
add 60413eb52c feat: support Iceberg metadata columns _pos, _spec, _file,
and _partition (#4752)
add 5a372406ed chore(deps): bump reqsign-core
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (ace48e36f5)
\
N -- N -- N
refs/heads/dependabot/cargo/native/main/all-other-cargo-deps-4459ef99e2
(5a372406ed)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/actions/setup-spark-builder/action.yaml | 2 +-
.github/workflows/ci.yml | 6 +-
.github/workflows/iceberg_spark_test_reusable.yml | 2 +-
.github/workflows/pr_build_linux.yml | 1 +
.github/workflows/pr_build_macos.yml | 1 +
.github/workflows/spark_sql_test_reusable.yml | 2 +-
.github/workflows/spark_sql_writer_tests.yml | 2 +-
.github/workflows/stale.yml | 2 +-
dev/ci/compute-changes.py | 2 +-
dev/diffs/{3.5.8.diff => 3.5.9.diff} | 68 +-
dev/diffs/iceberg/1.10.0.diff | 14 +
dev/diffs/iceberg/1.11.0.diff | 14 +
dev/diffs/iceberg/1.8.1.diff | 14 +
dev/diffs/iceberg/1.9.1.diff | 14 +
.../expression-audits/misc_funcs.md | 7 +
.../contributor-guide/iceberg-spark-tests.md | 2 +-
docs/source/contributor-guide/native_shuffle.md | 6 +-
.../contributor-guide/spark_configs_support.md | 34 +-
.../expressions/_category_template/cast.md | 27 +-
.../user-guide/latest/compatibility/index.md | 9 +-
.../latest/compatibility/spark-versions.md | 2 +-
docs/source/user-guide/latest/expressions.md | 2 +-
docs/source/user-guide/latest/installation.md | 2 +-
native/Cargo.lock | 5 +-
native/Cargo.toml | 4 +-
native/core/src/execution/expressions/random.rs | 18 +-
.../core/src/execution/operators/iceberg_scan.rs | 1 +
native/core/src/execution/planner.rs | 262 +++++++
.../src/execution/planner/expression_registry.rs | 4 +
native/proto/src/proto/expr.proto | 9 +
native/proto/src/proto/operator.proto | 12 +-
native/shuffle/benches/shuffle_writer.rs | 45 +-
.../shuffle/src/partitioners/single_partition.rs | 97 +--
native/shuffle/src/shuffle_writer.rs | 89 ++-
native/spark-expr/Cargo.toml | 9 +
native/spark-expr/benches/cast_from_string.rs | 199 ++---
native/spark-expr/benches/cast_string_to_date.rs | 18 +-
.../spark-expr/benches/cast_string_to_timestamp.rs | 204 +++++
native/spark-expr/benches/common/mod.rs | 43 ++
native/spark-expr/benches/round.rs | 87 +++
native/spark-expr/src/conversion_funcs/mod.rs | 1 +
native/spark-expr/src/conversion_funcs/string.rs | 220 ++++--
native/spark-expr/src/conversion_funcs/trim.rs | 192 +++++
native/spark-expr/src/math_funcs/round.rs | 255 +++++-
.../nondetermenistic_funcs/internal/mersenne.rs | 172 +++++
.../src/nondetermenistic_funcs/internal/mod.rs | 1 +
.../spark-expr/src/nondetermenistic_funcs/mod.rs | 2 +
.../src/nondetermenistic_funcs/shuffle.rs | 195 +----
.../spark-expr/src/nondetermenistic_funcs/uuid.rs | 290 +++++++
pom.xml | 6 +-
.../main/scala/org/apache/comet/CometConf.scala | 5 +-
.../scala/org/apache/comet/CometExecIterator.scala | 6 +-
.../main/scala/org/apache/comet/GenerateDocs.scala | 71 +-
.../apache/comet/iceberg/IcebergReflection.scala | 84 ++
.../org/apache/comet/rules/CometScanRule.scala | 87 ++-
.../org/apache/comet/serde/QueryPlanSerde.scala | 9 +-
.../org/apache/comet/serde/nondetermenistic.scala | 28 +-
.../serde/operator/CometIcebergNativeScan.scala | 299 +++++---
.../comet/serde/operator/CometNativeScan.scala | 29 +-
.../org/apache/comet/serde/operator/package.scala | 2 +-
.../scala/org/apache/comet/serde/unixtime.scala | 4 +-
.../apache/comet/testing/FuzzDataGenerator.scala | 17 +-
.../sql/comet/CometBroadcastExchangeExec.scala | 2 +-
.../spark/sql/comet/CometCsvNativeScanExec.scala | 12 +-
.../sql/comet/CometIcebergNativeScanExec.scala | 8 +-
.../spark/sql/comet/CometLocalTableScanExec.scala | 6 +-
.../spark/sql/comet/CometNativeWriteExec.scala | 2 +-
.../org/apache/spark/sql/comet/CometScanExec.scala | 16 +-
.../apache/spark/sql/comet/CometWindowExec.scala | 6 +-
.../shuffle/CometBlockStoreShuffleReader.scala | 2 +-
.../shuffle/CometNativeShuffleWriter.scala | 5 +-
.../shuffle/NativeBatchDecoderIterator.scala | 2 +-
.../org/apache/spark/sql/comet/operators.scala | 24 +-
.../org/apache/spark/sql/comet/util/Utils.scala | 2 +-
.../resources/pyspark/benchmark_pyarrow_udf.py | 2 +-
.../src/test/resources/pyspark/test_pyarrow_udf.py | 2 +-
.../sql-tests/expressions/array/exists.sql | 10 +
.../expressions/cast/cast_string_trim.sql | 71 ++
.../resources/sql-tests/expressions/misc/uuid.sql | 45 ++
.../sql-tests/expressions/misc/uuid_with_seed.sql | 96 +++
.../iceberg/metadata_column_partition.sql | 854 +++++++++++++++++++++
.../scala/org/apache/comet/CometCastSuite.scala | 37 +
.../org/apache/comet/CometCodegenHOFSuite.scala | 14 +
.../org/apache/comet/CometExpressionSuite.scala | 42 +-
.../org/apache/comet/CometFuzzTestSuite.scala | 11 +-
.../org/apache/comet/CometIcebergNativeSuite.scala | 670 +++++++++++++++-
.../comet/CometIcebergRewriteActionSuite.scala | 9 +-
.../apache/comet/CometUuidExpressionSuite.scala | 65 ++
.../apache/comet/exec/CometNativeReaderSuite.scala | 2 +-
.../apache/comet/parquet/ParquetReadSuite.scala | 2 +-
.../org/apache/spark/sql/CometTPCQueryBase.scala | 3 +-
.../scala/org/apache/spark/sql/CometTestBase.scala | 4 +-
.../src/test/scala/org/apache/spark/sql/TPCH.scala | 123 +--
.../test/scala/org/apache/spark/sql/Tables.scala | 13 +-
.../spark/sql/benchmark/CometBenchmarkBase.scala | 5 +-
.../CometBroadcastHashJoinBenchmark.scala | 3 +-
.../CometBroadcastNestedLoopJoinBenchmark.scala | 3 +-
.../benchmark/CometColumnarToRowBenchmark.scala | 3 +-
.../spark/sql/benchmark/CometExecBenchmark.scala | 3 +-
.../sql/benchmark/CometHashJoinBenchmark.scala | 3 +-
.../spark/sql/benchmark/CometReadBenchmark.scala | 2 +-
.../benchmark/CometRegExpExtractBenchmark.scala | 2 +-
.../sql/benchmark/CometShuffleBenchmark.scala | 3 +-
.../benchmark/CometSortMergeJoinBenchmark.scala | 3 +-
.../spark/sql/comet/PlanDataInjectorSuite.scala | 90 +++
105 files changed, 4756 insertions(+), 842 deletions(-)
rename dev/diffs/{3.5.8.diff => 3.5.9.diff} (98%)
create mode 100644 native/spark-expr/benches/cast_string_to_timestamp.rs
create mode 100644 native/spark-expr/benches/common/mod.rs
create mode 100644 native/spark-expr/benches/round.rs
create mode 100644 native/spark-expr/src/conversion_funcs/trim.rs
create mode 100644
native/spark-expr/src/nondetermenistic_funcs/internal/mersenne.rs
create mode 100644 native/spark-expr/src/nondetermenistic_funcs/uuid.rs
create mode 100644
spark/src/test/resources/sql-tests/expressions/cast/cast_string_trim.sql
create mode 100644 spark/src/test/resources/sql-tests/expressions/misc/uuid.sql
create mode 100644
spark/src/test/resources/sql-tests/expressions/misc/uuid_with_seed.sql
create mode 100644
spark/src/test/resources/sql-tests/iceberg/metadata_column_partition.sql
create mode 100644
spark/src/test/scala/org/apache/comet/CometUuidExpressionSuite.scala
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
