This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch
dependabot/maven/com.google.protobuf-protobuf-java-3.25.5
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
omit d6d25240 build(deps): bump com.google.protobuf:protobuf-java
add 9dfd6d11 docs: Add more detailed architecture documentation (#922)
add fa275f1c feat: implement scripts for binary release build (#932)
add fb4c78a0 Show reason for falling back to Spark when SMJ with join
condition is not enabled (#956)
add c3023c57 feat: Publish artifacts to maven (#946)
add 7a6f47f8 fix: Normalize NaN and zeros for floating number comparison
(#953)
add 459b2b0c fix: window function range offset should be long instead of
int (#733)
add 8410c711 chore: clarify tarball installation (#959)
add 5b3f7bca fix: CometScanExec on Spark 3.5.2 (#915)
add 50517f6a perf: Optimize decimal precision check in decimal aggregates
(sum and avg) (#952)
add 5663fc25 fix: div and rem by negative zero (#960)
add f31f6ccc Generate changelog for 0.3.0 release (#964)
add 94093f30 chore: fix publish-to-maven script (#966)
add b4de8e0f chore: Update benchmarks results based on 0.3.0-rc1 (#969)
add 22561c4e doc: add documentation interlinks (#975)
add 317a5345 fix: Use the number of rows from underlying arrays instead of
logical row count from RecordBatch (#972)
add a8156b53 chore: update rem expression guide (#976)
add a690e9d6 perf: Remove one redundant CopyExec for SMJ (#962)
add 0ee7df83 chore: Enable additional CreateArray tests (#928)
add f64553b6 chore: fix compatibility guide (#978)
add dcc4a8a6 fix: The spilled_bytes metric of CometSortExec should be size
instead of time (#984)
add 84cccf7d docs: Add notes for IntelliJ code size limits for code
inspections. (#985)
add a1599e2c chore: Update for 0.3.0 release, prepare for 0.4.0
development (#970)
add 18150fb6 chore: Don't transform the HashAggregate to
CometHashAggregate if Comet shuffle is disabled (#991)
add afd28b99 Quality of life fixes for easier hacking (#982)
add 3413397c docs: Update tuning guide (#995)
add b131cc39 feat: Support `GetArrayStructFields` expression (#993)
add 0028f1e1 fix: Fallback to Spark if scan has meta columns (#997)
add 0667c60b chore: Make parquet reader options Comet options instead of
Hadoop options (#968)
add 837c256f docs: Various documentation improvements (#1005)
add d40c802c clarify that Maven central only has jars for Linux (#1009)
add 22613e96 remove legacy comet-spark-shell (#1013)
add abd9f858 fix: Fallback to Spark if named_struct contains duplicate
field names (#1016)
add e146cfa1 chore: Reserve memory for native shuffle writer per partition
(#988)
add 591f45a0 chore: Bump arrow-rs to 53.1.0 and datafusion (#1001)
add 8d097d58 chore: Revert "chore: Reserve memory for native shuffle
writer per partition (#988)" (#1020)
add e3ac6cf9 feat: Implement bloom_filter_agg (#987)
add bd541d66 remove hard-coded version number from Dockerfile (#1025)
add 40336873 chore: Reserve memory for native shuffle writer per partition
(#1022)
add 3df9d5cc fix: Make comet-git-info.properties optional (#1027)
add cb3e9775 perf: Add experimental feature to replace SortMergeJoin with
ShuffledHashJoin (#1007)
add 845b654c chore: Improve error handling when native lib fails to load
(#1000)
add 0e9349fb fix: TopK operator should return correct results on
dictionary column with nulls (#1033)
add d670af7a perf: Cache jstrings during metrics collection (#1029)
add 00cf79b2 chore: Use twox-hash 2.0 xxhash64 oneshot api instead of
custom implementation (#1041)
add b38c361c fix: need default value for getSizeAsMb(EXECUTOR_MEMORY.key)
(#1046)
add 1103a6c1 feat: Support more types with BloomFilterAgg (#1039)
add ac4223c5 chore: Refactor Arrow Array and Schema allocation in
ColumnReader and MetadataColumnReader (#1047)
add 69760a37 minor: Refactor binary expr serde to reduce code duplication
(#1053)
add 3ea2da8d doc: fix K8s links and doc (#1058)
add 7c0c596c chore: Upgrade to DataFusion 43.0.0-rc1 (#1057)
add 562a8772 Refactor UnaryExpr and MathExpr in protobuf (#1056)
add aacf45cc minor: use defaults instead of hard-coding values (#1060)
add d1d57c68 minor: refactor UnaryExpr handling to make code more concise
(#1065)
add 4f2a80d5 feat: Implement CAST from struct to string (#1066)
add 82a9b954 Use official DataFusion 43 release (#1070)
add 676769cd chore: Refactor binary and math expression serde code (#1069)
add 712658e2 feat: Implement CAST between struct types (#1074)
add 9657b75f feat: support array_append (#1072)
add c32bf0c9 chore: Simplify CometShuffleMemoryAllocator to use Spark
unified memory allocator (#1063)
add f3da8449 docs: Update benchmarking.md (#1085)
add 2c832b4a feat: Require offHeap memory to be enabled (always use
unified memory) (#1062)
add 7cec2858 test: Restore one test in CometExecSuite by adding
COMET_SHUFFLE_MODE config (#1087)
add 10ef62ab Add changelog for 0.4.0 (#1089)
add 0c9a403f chore: Prepare for 0.5.0 development (#1090)
add 406ffeff build: Skip installation of spark-integration and fuzz
testing modules (#1091)
add bfd7054c Add hint for finding the GPG key to use when publishing to
maven (#1093)
add 59da6ce7 docs: Update documentation for 0.4.0 release (#1096)
add ca3a5290 fix: Unsigned type related bugs (#1095)
add b64c13df chore: Include first ScanExec batch in metrics (#1105)
add 19dd58d4 chore: Improve CometScan metrics (#1100)
add e6023054 chore: Add custom metric for native shuffle fetching batches
from JVM (#1108)
add 9990b349 feat: support array_insert (#1073)
add 500895df feat: enable decimal to decimal cast of different precision
and scale (#1086)
add 7b1a2904 docs: fix readme FGPA/FPGA typo (#1117)
add 5400fd7c fix: Use RDD partition index (#1112)
add ebdde77f fix: Various metrics bug fixes and improvements (#1111)
add 9b250c43 fix: Don't create CometScanExec for subclasses of
ParquetFileFormat (#1129)
add 95727aa7 fix: Fix metrics regressions (#1132)
add 36a23074 docs: Add more technical detail and new diagram to Comet
plugin overview (#1119)
add 2671e0ca Stop passing Java config map into native createPlan (#1101)
add 8d7bcb84 feat: Improve ScanExec native metrics (#1133)
add 587c29bf chore: Remove unused StringView struct (#1143)
add b95dc1d8 docs: Add some documentation explaining how shuffle works
(#1148)
add 1c6c7a93 test: enable more Spark 4.0 tests (#1145)
add 8d83cc15 chore: Refactor cast to use SparkCastOptions param (#1146)
add 21503ca2 Enable more scenarios in CometExecBenchmark. (#1151)
add 73f14051 chore: Move more expressions from core crate to spark-expr
crate (#1152)
add 5c45fdc9 remove dead code (#1155)
add 2c1a6b9c fix: Spark 4.0-preview1 SPARK-47120 (#1156)
add 49cf0d7f chore: Move string kernels and expressions to spark-expr
crate (#1164)
add 7db9aa67 chore: Move remaining expressions to spark-expr crate + some
minor refactoring (#1165)
add f1d08791 chore: Add ignored tests for reading complex types from
Parquet (#1167)
add b9ac78be feat: Add Spark-compatible implementation of
SchemaAdapterFactory (#1169)
add 46a28db5 fix: Document enabling comet explain plan usage in Spark
(4.0) (#1176)
add 655081b6 test: enabling Spark tests with offHeap requirement (#1177)
add e297d23b feat: Improve shuffle metrics (second attempt) (#1175)
add 8f4a8a54 fix: stddev_pop should not directly return 0.0 when count is
1.0 (#1184)
add ea6d2051 feat: Make native shuffle compression configurable and
respect `spark.shuffle.compress` (#1185)
add 053b7ccc minor: move shuffle classes from common to spark (#1193)
add 639fa2fb minor: refactor decodeBatches to make private in broadcast
exchange (#1195)
add 58dee739 minor: refactor prepare_output so that it does not require an
ExecutionContext (#1194)
add 5432e03e fix: fix missing explanation for then branch in case when
(#1200)
add 103f82f5 minor: remove unused source files (#1202)
add 5d2c909f chore: Upgrade to DataFusion 44.0.0-rc2 (#1154)
add 4f8ce757 feat: add support for array_contains expression (#1163)
add 9320aedc feat: Add a `spark.comet.exec.memoryPool` configuration for
experimenting with various datafusion memory pool setups. (#1021)
add 2e0f00a4 feat: Reenable tests for filtered SMJ anti join (#1211)
add 4333dce3 chore: Add safety check to CometBuffer (#1050)
add 4b56c523 remove unreachable code (#1213)
add 5f1e9983 test: Enable Comet by default except some tests in
SparkSessionExtensionSuite (#1201)
add e39ffa6b extract struct expressions to folders based on spark grouping
(#1216)
add 5c389d1b chore: extract static invoke expressions to folders based on
spark grouping (#1217)
add e72beb1f chore: Follow-on PR to fully enable onheap memory usage
(#1210)
add 74a6a8d5 feat: Move shuffle block decompression and decoding to native
code and add LZ4 & Snappy support (#1192)
add 3f0d442c chore: extract agg_funcs expressions to folders based on
spark grouping (#1224)
add 4cf840f5 extract datetime_funcs expressions to folders based on spark
grouping (#1222)
add 508db064 chore: use datafusion from crates.io (#1232)
add c19202cf chore: extract strings file to `strings_func` like in spark
grouping (#1215)
add fbcf0251 chore: extract predicate_functions expressions to folders
based on spark grouping (#1218)
add ca7b4a8a build(deps): bump protobuf version to 3.21.12 (#1234)
add c6acc9df extract json_funcs expressions to folders based on spark
grouping (#1220)
add 0a68f1c6 test: Enable shuffle by default in Spark tests (#1240)
add e731b6e9 chore: extract hash_funcs expressions to folders based on
spark grouping (#1221)
add be488390 fix: Fall back to Spark for unsupported partition or sort
expressions in window aggregates (#1253)
add d15d0511 perf: Improve query planning to more reliably fall back to
columnar shuffle when native shuffle is not supported (#1209)
add 602d697a build(deps): bump com.google.protobuf:protobuf-java
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (d6d25240)
\
N -- N -- N
refs/heads/dependabot/maven/com.google.protobuf-protobuf-java-3.25.5 (602d697a)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/actions/setup-spark-builder/action.yaml | 2 +-
.github/workflows/docker-publish.yml | 2 +-
.github/workflows/spark_sql_test.yml | 2 +-
.github/workflows/spark_sql_test_ansi.yml | 2 +-
.gitignore | 1 +
LICENSE.txt | 29 +-
Makefile | 16 +
NOTICE.txt | 6 +-
README.md | 27 +-
benchmarks/README.md | 2 +-
bin/comet-spark-shell | 86 -
common/pom.xml | 2 +-
.../src/main/java/org/apache/comet/NativeBase.java | 15 +-
.../org/apache/comet/parquet/ColumnReader.java | 105 +-
.../apache/comet/parquet/ConstantColumnReader.java | 4 +-
.../apache/comet/parquet/MetadataColumnReader.java | 27 +-
.../main/java/org/apache/comet/parquet/Native.java | 6 +-
.../java/org/apache/comet/parquet/ReadOptions.java | 50 +-
.../apache/comet/parquet/RowIndexColumnReader.java | 2 +-
.../org/apache/comet/vector/CometPlainVector.java | 16 +
.../main/scala/org/apache/comet/CometConf.scala | 184 +-
.../src/main/scala/org/apache/comet/package.scala | 78 +-
.../scala/org/apache/comet/vector/NativeUtil.scala | 17 +-
.../execution/shuffle/IpcInputStreamIterator.scala | 127 --
.../org/apache/comet/parquet/TestFileReader.java | 8 +-
dev/changelog/0.3.0.md | 115 ++
dev/changelog/0.4.0.md | 108 +
dev/diffs/3.4.3.diff | 144 +-
dev/diffs/3.5.1.diff | 154 +-
dev/diffs/4.0.0-preview1.diff | 574 ++----
dev/release/README.md | 177 +-
dev/release/build-release-comet.sh | 202 ++
dev/release/comet-rm/Dockerfile | 91 +
.../release/comet-rm/build-comet-native-libs.sh | 39 +-
dev/release/create-tarball.sh | 13 +-
dev/release/publish-to-maven.sh | 164 ++
dev/release/rat_exclude_files.txt | 15 +-
dev/release/verifying-release-candidates.md | 2 +
.../{2024-08-23 => 0.2.0}/tpcds_allqueries.png | Bin
.../tpcds_queries_compare.png | Bin
.../tpcds_queries_speedup_abs.png | Bin
.../{2024-08-23 => 0.2.0}/tpch_allqueries.png | Bin
.../{2024-08-23 => 0.2.0}/tpch_queries_compare.png | Bin
.../tpch_queries_speedup_rel.png | Bin
.../benchmark-results/0.3.0/tpcds_allqueries.png | Bin 0 -> 30139 bytes
.../0.3.0/tpcds_queries_compare.png | Bin 0 -> 46449 bytes
.../0.3.0/tpcds_queries_speedup_abs.png | Bin 0 -> 67006 bytes
.../0.3.0/tpcds_queries_speedup_rel.png | Bin 0 -> 96045 bytes
.../benchmark-results/0.3.0/tpch_allqueries.png | Bin 0 -> 29289 bytes
.../0.3.0/tpch_queries_compare.png | Bin 0 -> 33052 bytes
.../0.3.0/tpch_queries_speedup_abs.png | Bin 0 -> 37680 bytes
.../0.3.0/tpch_queries_speedup_rel.png | Bin 0 -> 44133 bytes
.../benchmark-results/0.4.0/tpcds_allqueries.png | Bin 0 -> 28503 bytes
.../0.4.0/tpcds_queries_compare.png | Bin 0 -> 43094 bytes
.../0.4.0/tpcds_queries_speedup_abs.png | Bin 0 -> 64166 bytes
.../0.4.0/tpcds_queries_speedup_rel.png | Bin 0 -> 90369 bytes
.../benchmark-results/0.4.0/tpch_allqueries.png | Bin 0 -> 29251 bytes
.../0.4.0/tpch_queries_compare.png | Bin 0 -> 34357 bytes
.../0.4.0/tpch_queries_speedup_abs.png | Bin 0 -> 37579 bytes
.../0.4.0/tpch_queries_speedup_rel.png | Bin 0 -> 48051 bytes
.../_static/images/comet-dataflow.excalidraw | 2134 ++++++++++++++++++++
docs/source/_static/images/comet-dataflow.svg | 10 +
.../{2024-08-23 => 0.2.0}/comet-tpcds.json | 0
.../{2024-08-23 => 0.2.0}/comet-tpch.json | 0
.../{2024-08-23 => 0.2.0}/spark-tpcds.json | 0
.../{2024-08-23 => 0.2.0}/spark-tpch.json | 0
.../benchmark-results/0.3.0/comet-tpcds.json | 551 +++++
.../benchmark-results/0.3.0/comet-tpch.json | 158 ++
.../benchmark-results/0.3.0/spark-tpcds.json | 524 +++++
.../benchmark-results/0.3.0/spark-tpch.json | 139 ++
.../benchmark-results/0.4.0/comet-tpcds.json | 755 +++++++
.../benchmark-results/0.4.0/comet-tpch.json | 219 ++
.../benchmark-results/0.4.0/spark-tpcds.json | 742 +++++++
.../benchmark-results/0.4.0/spark-tpch.json | 203 ++
.../datafusion-python-tpch.json | 0
.../contributor-guide/benchmark-results/tpc-ds.md | 26 +-
.../contributor-guide/benchmark-results/tpc-h.md | 25 +-
docs/source/contributor-guide/benchmarking.md | 8 +-
docs/source/contributor-guide/debugging.md | 2 +-
docs/source/contributor-guide/development.md | 20 +-
docs/source/contributor-guide/plugin_overview.md | 121 +-
docs/source/index.rst | 3 +
docs/source/user-guide/compatibility.md | 9 +
docs/source/user-guide/configs.md | 34 +-
docs/source/user-guide/expressions.md | 14 +-
docs/source/user-guide/installation.md | 99 +-
docs/source/user-guide/kubernetes.md | 8 +-
docs/source/user-guide/metrics.md | 66 +
docs/source/user-guide/overview.md | 34 +-
docs/source/user-guide/source.md | 69 +
docs/source/user-guide/tuning.md | 141 +-
docs/templates/compatibility-template.md | 8 +
fuzz-testing/README.md | 6 +-
fuzz-testing/pom.xml | 9 +-
kube/Dockerfile | 2 +-
native/Cargo.lock | 1117 ++++++----
native/Cargo.toml | 38 +-
{spark => native}/README.md | 8 +-
native/core/Cargo.toml | 18 +-
.../benches/{aggregate.rs => bloom_filter_agg.rs} | 136 +-
native/core/benches/hash.rs | 137 --
native/core/benches/parquet_read.rs | 7 +-
native/core/benches/row_columnar.rs | 2 +
native/core/benches/shuffle_writer.rs | 116 +-
native/core/src/common/bit.rs | 6 +-
native/core/src/common/buffer.rs | 50 +-
native/core/src/common/mod.rs | 18 -
native/core/src/data_type.rs | 241 ---
native/core/src/errors.rs | 44 +-
.../src/execution/datafusion/expressions/mod.rs | 46 -
.../execution/datafusion/expressions/strings.rs | 301 ---
.../src/execution/expressions/bloom_filter_agg.rs | 170 ++
.../expressions/bloom_filter_might_contain.rs | 52 +-
.../src/execution/{kernels => expressions}/mod.rs | 9 +-
.../{datafusion => }/expressions/subquery.rs | 23 +-
native/core/src/execution/jni_api.rs | 406 ++--
native/core/src/execution/kernels/hash.rs | 187 --
native/core/src/execution/metrics/utils.rs | 59 +-
native/core/src/execution/mod.rs | 8 +-
native/core/src/execution/operators/copy.rs | 19 +-
.../execution/{datafusion => }/operators/expand.rs | 16 +-
native/core/src/execution/operators/filter.rs | 7 +-
native/core/src/execution/operators/mod.rs | 2 +
native/core/src/execution/operators/scan.rs | 115 +-
.../core/src/execution/{datafusion => }/planner.rs | 711 +++++--
native/core/src/execution/shuffle/list.rs | 4 +-
native/core/src/execution/shuffle/map.rs | 17 +-
native/core/src/execution/shuffle/mod.rs | 4 +
native/core/src/execution/shuffle/row.rs | 15 +-
.../{datafusion => shuffle}/shuffle_writer.rs | 726 +++++--
native/core/src/execution/sort.rs | 2 +-
native/core/src/execution/spark_plan.rs | 98 +
.../src/execution/{datafusion => }/util/mod.rs | 0
.../{datafusion => }/util/spark_bit_array.rs | 110 +-
.../{datafusion => }/util/spark_bloom_filter.rs | 83 +-
native/core/src/execution/utils.rs | 18 -
native/core/src/jvm_bridge/batch_iterator.rs | 5 +
native/core/src/jvm_bridge/comet_metric_node.rs | 1 +
.../src/jvm_bridge/comet_task_memory_manager.rs | 1 +
native/core/src/jvm_bridge/mod.rs | 9 +-
native/core/src/lib.rs | 31 -
native/core/src/parquet/compression.rs | 319 ---
native/core/src/parquet/mod.rs | 29 +-
native/core/src/parquet/mutable_vector.rs | 22 +-
native/core/src/parquet/read/column.rs | 19 +-
native/core/src/parquet/read/levels.rs | 7 +-
native/core/src/parquet/read/mod.rs | 3 -
native/core/src/parquet/read/values.rs | 37 +-
native/core/src/parquet/util/jni_buffer.rs | 98 -
native/core/src/parquet/util/test_common/mod.rs | 3 +-
.../core/src/parquet/util/test_common/page_util.rs | 16 +-
native/proto/src/proto/expr.proto | 230 +--
native/proto/src/proto/operator.proto | 16 +-
native/spark-expr/Cargo.toml | 17 +-
native/{core => spark-expr}/benches/aggregate.rs | 18 +-
native/spark-expr/benches/cast_from_string.rs | 30 +-
native/spark-expr/benches/cast_numeric.rs | 22 +-
native/spark-expr/benches/conditional.rs | 4 +-
.../src/agg_funcs}/avg.rs | 22 +-
.../src/agg_funcs}/avg_decimal.rs | 43 +-
.../src/agg_funcs}/correlation.rs | 34 +-
.../src/agg_funcs}/covariance.rs | 26 +-
.../kernels => spark-expr/src/agg_funcs}/mod.rs | 19 +-
.../src/agg_funcs}/stddev.rs | 32 +-
.../src/agg_funcs}/sum_decimal.rs | 161 +-
.../src/agg_funcs}/variance.rs | 25 +-
.../expressions => spark-expr/src}/bitwise_not.rs | 56 +-
native/spark-expr/src/cast.rs | 695 +++++--
.../src}/checkoverflow.rs | 53 +-
.../src}/comet_scalar_funcs.rs | 14 +-
.../src/datetime_funcs/date_arithmetic.rs | 102 +
native/spark-expr/src/datetime_funcs/date_trunc.rs | 113 ++
native/spark-expr/src/datetime_funcs/hour.rs | 122 ++
native/spark-expr/src/datetime_funcs/minute.rs | 122 ++
.../src/datetime_funcs}/mod.rs | 17 +-
native/spark-expr/src/datetime_funcs/second.rs | 122 ++
.../src/datetime_funcs/timestamp_trunc.rs | 152 ++
.../kernels => spark-expr/src/hash_funcs}/mod.rs | 12 +-
native/spark-expr/src/hash_funcs/murmur3.rs | 280 +++
native/spark-expr/src/hash_funcs/sha2.rs | 76 +
native/spark-expr/src/hash_funcs/utils.rs | 393 ++++
native/spark-expr/src/hash_funcs/xxhash64.rs | 264 +++
native/spark-expr/src/if_expr.rs | 49 +-
.../spark-expr/src/{kernels => json_funcs}/mod.rs | 4 +-
native/spark-expr/src/{ => json_funcs}/to_json.rs | 30 +-
native/spark-expr/src/kernels/mod.rs | 1 +
.../src}/kernels/strings.rs | 124 +-
native/spark-expr/src/lib.rs | 61 +-
native/spark-expr/src/list.rs | 601 +++++-
.../expressions => spark-expr/src}/negative.rs | 44 +-
.../src}/normalize_nan.rs | 45 +-
native/spark-expr/src/predicate_funcs/is_nan.rs | 70 +
.../src/predicate_funcs}/mod.rs | 9 +-
.../src/{regexp.rs => predicate_funcs/rlike.rs} | 27 +-
native/spark-expr/src/scalar_funcs.rs | 165 +-
.../src/scalar_funcs/hash_expressions.rs | 162 --
native/spark-expr/src/scalar_funcs/hex.rs | 2 +-
native/spark-expr/src/schema_adapter.rs | 376 ++++
native/spark-expr/src/spark_hash.rs | 708 -------
.../src/static_invoke/char_varchar_utils}/mod.rs | 7 +-
.../char_varchar_utils/read_side_padding.rs | 76 +
.../src/static_invoke}/mod.rs | 7 +-
.../kernels => spark-expr/src/string_funcs}/mod.rs | 11 +-
native/spark-expr/src/string_funcs/prediction.rs | 137 ++
native/spark-expr/src/string_funcs/string_space.rs | 104 +
native/spark-expr/src/string_funcs/substring.rs | 111 +
.../create_named_struct.rs} | 133 +-
.../src/struct_funcs/get_struct_field.rs | 125 ++
.../src/struct_funcs}/mod.rs | 10 +-
native/spark-expr/src/temporal.rs | 535 -----
.../src}/test_common/file_util.rs | 0
.../src/test_common}/mod.rs | 3 +-
.../expressions => spark-expr/src}/unbound.rs | 21 +-
native/spark-expr/src/utils.rs | 39 +-
native/spark-expr/src/xxhash64.rs | 190 --
pom.xml | 7 +-
rust-toolchain.toml | 2 +-
spark-integration/pom.xml | 9 +-
spark/pom.xml | 2 +-
.../java/org/apache/comet/CometBatchIterator.java | 28 +-
.../shuffle/comet/CometShuffleMemoryAllocator.java | 182 +-
....java => CometShuffleMemoryAllocatorTrait.java} | 21 +-
...r.java => CometTestShuffleMemoryAllocator.java} | 35 +-
.../shuffle/sort/CometShuffleExternalSorter.java | 20 +-
.../execution/shuffle/CometDiskBlockWriter.java | 24 +-
.../sql/comet/execution/shuffle/SpillWriter.java | 16 +-
.../scala/org/apache/comet/CometExecIterator.scala | 98 +-
.../apache/comet/CometSparkSessionExtensions.scala | 274 ++-
spark/src/main/scala/org/apache/comet/Native.scala | 56 +-
.../org/apache/comet/expressions/CometCast.scala | 46 +-
.../org/apache/comet/parquet/ParquetFilters.scala | 10 +-
.../scala/org/apache/comet/rules/RewriteJoin.scala | 70 +
.../org/apache/comet/serde/QueryPlanSerde.scala | 959 ++++-----
.../src/main/scala/org/apache/spark/Plugins.scala | 4 +-
.../sql/comet/CometBroadcastExchangeExec.scala | 24 +-
.../spark/sql/comet/CometCollectLimitExec.scala | 3 +-
.../spark/sql/comet/CometColumnarToRowExec.scala | 209 ++
.../apache/spark/sql/comet/CometExecUtils.scala | 9 +-
.../apache/spark/sql/comet/CometMetricNode.scala | 12 +
.../org/apache/spark/sql/comet/CometScanExec.scala | 34 +-
.../sql/comet/CometTakeOrderedAndProjectExec.scala | 10 +-
.../spark/sql/comet/ZippedPartitionsRDD.scala | 11 +-
.../execution/shuffle/ArrowReaderIterator.scala | 0
.../shuffle/CometBlockStoreShuffleReader.scala | 33 +-
.../execution/shuffle/CometShuffleDependency.scala | 4 +-
.../shuffle/CometShuffleExchangeExec.scala | 50 +-
.../execution/shuffle/CometShuffleManager.scala | 2 +-
.../execution/shuffle/CometShuffledRowRDD.scala | 4 +-
.../shuffle/NativeBatchDecoderIterator.scala | 188 ++
.../sql/comet/execution/shuffle/ShuffleUtils.scala | 31 +-
.../org/apache/spark/sql/comet/operators.scala | 73 +-
.../approved-plans-v1_4-spark3_5/q1/explain.txt | 20 +-
.../approved-plans-v1_4-spark3_5/q1/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q10/explain.txt | 150 +-
.../q10/simplified.txt | 125 +-
.../approved-plans-v1_4-spark3_5/q11/explain.txt | 36 +-
.../q11/simplified.txt | 6 +-
.../approved-plans-v1_4-spark3_5/q12/explain.txt | 16 +-
.../q12/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q13/explain.txt | 22 +-
.../q13/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q14a/explain.txt | 50 +-
.../q14a/simplified.txt | 8 +-
.../approved-plans-v1_4-spark3_5/q14b/explain.txt | 68 +-
.../q14b/simplified.txt | 14 +-
.../approved-plans-v1_4-spark3_5/q15/explain.txt | 18 +-
.../q15/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q16/explain.txt | 110 +-
.../q16/simplified.txt | 85 +-
.../approved-plans-v1_4-spark3_5/q17/explain.txt | 30 +-
.../q17/simplified.txt | 6 +-
.../approved-plans-v1_4-spark3_5/q18/explain.txt | 24 +-
.../q18/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q19/explain.txt | 16 +-
.../q19/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q2/explain.txt | 14 +-
.../approved-plans-v1_4-spark3_5/q2/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q20/explain.txt | 16 +-
.../q20/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q21/explain.txt | 20 +-
.../q21/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q22/explain.txt | 18 +-
.../q22/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q23a/explain.txt | 46 +-
.../q23a/simplified.txt | 10 +-
.../approved-plans-v1_4-spark3_5/q23b/explain.txt | 48 +-
.../q23b/simplified.txt | 10 +-
.../approved-plans-v1_4-spark3_5/q24a/explain.txt | 254 +--
.../q24a/simplified.txt | 128 +-
.../approved-plans-v1_4-spark3_5/q24b/explain.txt | 254 +--
.../q24b/simplified.txt | 128 +-
.../approved-plans-v1_4-spark3_5/q25/explain.txt | 30 +-
.../q25/simplified.txt | 6 +-
.../approved-plans-v1_4-spark3_5/q26/explain.txt | 20 +-
.../q26/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q27/explain.txt | 20 +-
.../q27/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q28/explain.txt | 320 +--
.../q28/simplified.txt | 126 +-
.../approved-plans-v1_4-spark3_5/q29/explain.txt | 38 +-
.../q29/simplified.txt | 8 +-
.../approved-plans-v1_4-spark3_5/q3/explain.txt | 10 +-
.../approved-plans-v1_4-spark3_5/q3/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q30/explain.txt | 22 +-
.../q30/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q31/explain.txt | 42 +-
.../q31/simplified.txt | 8 +-
.../approved-plans-v1_4-spark3_5/q32/explain.txt | 18 +-
.../q32/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q33/explain.txt | 24 +-
.../q33/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q34/explain.txt | 22 +-
.../q34/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q35/explain.txt | 148 +-
.../q35/simplified.txt | 123 +-
.../approved-plans-v1_4-spark3_5/q36/explain.txt | 18 +-
.../q36/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q37/explain.txt | 18 +-
.../q37/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q38/explain.txt | 20 +-
.../q38/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q39a/explain.txt | 36 +-
.../q39a/simplified.txt | 6 +-
.../approved-plans-v1_4-spark3_5/q39b/explain.txt | 36 +-
.../q39b/simplified.txt | 6 +-
.../approved-plans-v1_4-spark3_5/q4/explain.txt | 44 +-
.../approved-plans-v1_4-spark3_5/q4/simplified.txt | 6 +-
.../approved-plans-v1_4-spark3_5/q40/explain.txt | 20 +-
.../q40/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q41/explain.txt | 8 +-
.../q41/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q42/explain.txt | 10 +-
.../q42/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q43/explain.txt | 10 +-
.../q43/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q44/explain.txt | 202 +-
.../q44/simplified.txt | 74 +-
.../approved-plans-v1_4-spark3_5/q45/explain.txt | 124 +-
.../q45/simplified.txt | 93 +-
.../approved-plans-v1_4-spark3_5/q46/explain.txt | 22 +-
.../q46/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q47/explain.txt | 26 +-
.../q47/simplified.txt | 8 +-
.../approved-plans-v1_4-spark3_5/q48/explain.txt | 20 +-
.../q48/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q49/explain.txt | 214 +-
.../q49/simplified.txt | 219 +-
.../approved-plans-v1_4-spark3_5/q5/explain.txt | 204 +-
.../approved-plans-v1_4-spark3_5/q5/simplified.txt | 179 +-
.../approved-plans-v1_4-spark3_5/q50/explain.txt | 20 +-
.../q50/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q51/explain.txt | 163 +-
.../q51/simplified.txt | 120 +-
.../approved-plans-v1_4-spark3_5/q52/explain.txt | 10 +-
.../q52/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q53/explain.txt | 18 +-
.../q53/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q54/explain.txt | 46 +-
.../q54/simplified.txt | 10 +-
.../approved-plans-v1_4-spark3_5/q55/explain.txt | 10 +-
.../q55/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q56/explain.txt | 24 +-
.../q56/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q57/explain.txt | 26 +-
.../q57/simplified.txt | 8 +-
.../approved-plans-v1_4-spark3_5/q58/explain.txt | 28 +-
.../q58/simplified.txt | 6 +-
.../approved-plans-v1_4-spark3_5/q59/explain.txt | 16 +-
.../q59/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q6/explain.txt | 28 +-
.../approved-plans-v1_4-spark3_5/q6/simplified.txt | 6 +-
.../approved-plans-v1_4-spark3_5/q60/explain.txt | 24 +-
.../q60/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q61/explain.txt | 30 +-
.../q61/simplified.txt | 6 +-
.../approved-plans-v1_4-spark3_5/q62/explain.txt | 14 +-
.../q62/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q63/explain.txt | 18 +-
.../q63/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q64/explain.txt | 50 +-
.../q64/simplified.txt | 6 +-
.../approved-plans-v1_4-spark3_5/q65/explain.txt | 20 +-
.../q65/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q66/explain.txt | 22 +-
.../q66/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q67/explain.txt | 116 +-
.../q67/simplified.txt | 75 +-
.../approved-plans-v1_4-spark3_5/q68/explain.txt | 22 +-
.../q68/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q69/explain.txt | 148 +-
.../q69/simplified.txt | 125 +-
.../approved-plans-v1_4-spark3_5/q7/explain.txt | 20 +-
.../approved-plans-v1_4-spark3_5/q7/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q70/explain.txt | 158 +-
.../q70/simplified.txt | 128 +-
.../approved-plans-v1_4-spark3_5/q71/explain.txt | 22 +-
.../q71/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q72/explain.txt | 32 +-
.../q72/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q73/explain.txt | 22 +-
.../q73/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q74/explain.txt | 36 +-
.../q74/simplified.txt | 6 +-
.../approved-plans-v1_4-spark3_5/q75/explain.txt | 40 +-
.../q75/simplified.txt | 6 +-
.../approved-plans-v1_4-spark3_5/q76/explain.txt | 14 +-
.../q76/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q77/explain.txt | 230 +--
.../q77/simplified.txt | 207 +-
.../approved-plans-v1_4-spark3_5/q78/explain.txt | 26 +-
.../q78/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q79/explain.txt | 20 +-
.../q79/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q8/explain.txt | 22 +-
.../approved-plans-v1_4-spark3_5/q8/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q80/explain.txt | 270 +--
.../q80/simplified.txt | 239 +--
.../approved-plans-v1_4-spark3_5/q81/explain.txt | 22 +-
.../q81/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q82/explain.txt | 18 +-
.../q82/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q83/explain.txt | 26 +-
.../q83/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q84/explain.txt | 16 +-
.../q84/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q85/explain.txt | 26 +-
.../q85/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q86/explain.txt | 16 +-
.../q86/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q87/explain.txt | 148 +-
.../q87/simplified.txt | 115 +-
.../approved-plans-v1_4-spark3_5/q88/explain.txt | 68 +-
.../q88/simplified.txt | 16 +-
.../approved-plans-v1_4-spark3_5/q89/explain.txt | 18 +-
.../q89/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q9/explain.txt | 36 +-
.../approved-plans-v1_4-spark3_5/q9/simplified.txt | 12 +-
.../approved-plans-v1_4-spark3_5/q90/explain.txt | 20 +-
.../q90/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q91/explain.txt | 24 +-
.../q91/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q92/explain.txt | 18 +-
.../q92/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q93/explain.txt | 10 +-
.../q93/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q94/explain.txt | 110 +-
.../q94/simplified.txt | 85 +-
.../approved-plans-v1_4-spark3_5/q95/explain.txt | 136 +-
.../q95/simplified.txt | 101 +-
.../approved-plans-v1_4-spark3_5/q96/explain.txt | 12 +-
.../q96/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q97/explain.txt | 16 +-
.../q97/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q98/explain.txt | 20 +-
.../q98/simplified.txt | 6 +-
.../approved-plans-v1_4-spark3_5/q99/explain.txt | 14 +-
.../q99/simplified.txt | 2 +-
.../approved-plans-v1_4-spark4_0/q1/explain.txt | 202 +-
.../approved-plans-v1_4-spark4_0/q1/simplified.txt | 85 +-
.../approved-plans-v1_4-spark4_0/q10/explain.txt | 150 +-
.../q10/simplified.txt | 125 +-
.../approved-plans-v1_4-spark4_0/q11/explain.txt | 334 +--
.../q11/simplified.txt | 172 +-
.../approved-plans-v1_4-spark4_0/q12/explain.txt | 104 +-
.../q12/simplified.txt | 64 +-
.../approved-plans-v1_4-spark4_0/q13/explain.txt | 114 +-
.../q13/simplified.txt | 87 +-
.../approved-plans-v1_4-spark4_0/q14a/explain.txt | 476 ++---
.../q14a/simplified.txt | 323 +--
.../approved-plans-v1_4-spark4_0/q14b/explain.txt | 452 +++--
.../q14b/simplified.txt | 299 +--
.../approved-plans-v1_4-spark4_0/q15/explain.txt | 92 +-
.../q15/simplified.txt | 65 +-
.../approved-plans-v1_4-spark4_0/q16/explain.txt | 108 +-
.../q16/simplified.txt | 85 +-
.../approved-plans-v1_4-spark4_0/q17/explain.txt | 156 +-
.../q17/simplified.txt | 111 +-
.../approved-plans-v1_4-spark4_0/q18/explain.txt | 134 +-
.../q18/simplified.txt | 101 +-
.../approved-plans-v1_4-spark4_0/q19/explain.txt | 98 +-
.../q19/simplified.txt | 71 +-
.../approved-plans-v1_4-spark4_0/q2/explain.txt | 142 +-
.../approved-plans-v1_4-spark4_0/q2/simplified.txt | 44 +-
.../approved-plans-v1_4-spark4_0/q20/explain.txt | 104 +-
.../q20/simplified.txt | 64 +-
.../approved-plans-v1_4-spark4_0/q21/explain.txt | 94 +-
.../q21/simplified.txt | 63 +-
.../approved-plans-v1_4-spark4_0/q22/explain.txt | 94 +-
.../q22/simplified.txt | 67 +-
.../approved-plans-v1_4-spark4_0/q23a/explain.txt | 386 ++--
.../q23a/simplified.txt | 273 +--
.../approved-plans-v1_4-spark4_0/q23b/explain.txt | 480 ++---
.../q23b/simplified.txt | 325 +--
.../approved-plans-v1_4-spark4_0/q24a/explain.txt | 254 +--
.../q24a/simplified.txt | 128 +-
.../approved-plans-v1_4-spark4_0/q24b/explain.txt | 254 +--
.../q24b/simplified.txt | 128 +-
.../approved-plans-v1_4-spark4_0/q25/explain.txt | 156 +-
.../q25/simplified.txt | 111 +-
.../approved-plans-v1_4-spark4_0/q26/explain.txt | 108 +-
.../q26/simplified.txt | 79 +-
.../approved-plans-v1_4-spark4_0/q27/explain.txt | 108 +-
.../q27/simplified.txt | 79 +-
.../approved-plans-v1_4-spark4_0/q28/explain.txt | 350 ++--
.../q28/simplified.txt | 170 +-
.../approved-plans-v1_4-spark4_0/q29/explain.txt | 184 +-
.../q29/simplified.txt | 131 +-
.../approved-plans-v1_4-spark4_0/q3/explain.txt | 62 +-
.../approved-plans-v1_4-spark4_0/q3/simplified.txt | 41 +-
.../approved-plans-v1_4-spark4_0/q30/explain.txt | 226 ++-
.../q30/simplified.txt | 101 +-
.../approved-plans-v1_4-spark4_0/q31/explain.txt | 462 +++--
.../q31/simplified.txt | 204 +-
.../approved-plans-v1_4-spark4_0/q32/explain.txt | 134 +-
.../q32/simplified.txt | 96 +-
.../approved-plans-v1_4-spark4_0/q33/explain.txt | 264 +--
.../q33/simplified.txt | 176 +-
.../approved-plans-v1_4-spark4_0/q34/explain.txt | 22 +-
.../q34/simplified.txt | 4 +-
.../approved-plans-v1_4-spark4_0/q35/explain.txt | 148 +-
.../q35/simplified.txt | 123 +-
.../approved-plans-v1_4-spark4_0/q36/explain.txt | 120 +-
.../q36/simplified.txt | 78 +-
.../approved-plans-v1_4-spark4_0/q37/explain.txt | 18 +-
.../q37/simplified.txt | 4 +-
.../approved-plans-v1_4-spark4_0/q38/explain.txt | 20 +-
.../q38/simplified.txt | 4 +-
.../approved-plans-v1_4-spark4_0/q39a/explain.txt | 232 ++-
.../q39a/simplified.txt | 124 +-
.../approved-plans-v1_4-spark4_0/q39b/explain.txt | 232 ++-
.../q39b/simplified.txt | 124 +-
.../approved-plans-v1_4-spark4_0/q4/explain.txt | 486 ++---
.../approved-plans-v1_4-spark4_0/q4/simplified.txt | 242 +--
.../approved-plans-v1_4-spark4_0/q40/explain.txt | 108 +-
.../q40/simplified.txt | 81 +-
.../approved-plans-v1_4-spark4_0/q41/explain.txt | 8 +-
.../q41/simplified.txt | 2 +-
.../approved-plans-v1_4-spark4_0/q42/explain.txt | 62 +-
.../q42/simplified.txt | 41 +-
.../approved-plans-v1_4-spark4_0/q43/explain.txt | 62 +-
.../q43/simplified.txt | 41 +-
.../approved-plans-v1_4-spark4_0/q44/explain.txt | 277 +--
.../q44/simplified.txt | 132 +-
.../approved-plans-v1_4-spark4_0/q45/explain.txt | 124 +-
.../q45/simplified.txt | 93 +-
.../approved-plans-v1_4-spark4_0/q46/explain.txt | 156 +-
.../q46/simplified.txt | 83 +-
.../approved-plans-v1_4-spark4_0/q47/explain.txt | 204 +-
.../q47/simplified.txt | 89 +-
.../approved-plans-v1_4-spark4_0/q48/explain.txt | 102 +-
.../q48/simplified.txt | 77 +-
.../approved-plans-v1_4-spark4_0/q49/explain.txt | 362 ++--
.../q49/simplified.txt | 249 +--
.../approved-plans-v1_4-spark4_0/q5/explain.txt | 306 +--
.../approved-plans-v1_4-spark4_0/q5/simplified.txt | 196 +-
.../approved-plans-v1_4-spark4_0/q50/explain.txt | 104 +-
.../q50/simplified.txt | 75 +-
.../approved-plans-v1_4-spark4_0/q51/explain.txt | 225 ++-
.../q51/simplified.txt | 140 +-
.../approved-plans-v1_4-spark4_0/q52/explain.txt | 62 +-
.../q52/simplified.txt | 41 +-
.../approved-plans-v1_4-spark4_0/q53/explain.txt | 122 +-
.../q53/simplified.txt | 76 +-
.../approved-plans-v1_4-spark4_0/q54/explain.txt | 264 +--
.../q54/simplified.txt | 200 +-
.../approved-plans-v1_4-spark4_0/q55/explain.txt | 62 +-
.../q55/simplified.txt | 41 +-
.../approved-plans-v1_4-spark4_0/q56/explain.txt | 264 +--
.../q56/simplified.txt | 176 +-
.../approved-plans-v1_4-spark4_0/q57/explain.txt | 204 +-
.../q57/simplified.txt | 89 +-
.../approved-plans-v1_4-spark4_0/q58/explain.txt | 278 +--
.../q58/simplified.txt | 147 +-
.../approved-plans-v1_4-spark4_0/q59/explain.txt | 174 +-
.../q59/simplified.txt | 40 +-
.../approved-plans-v1_4-spark4_0/q6/explain.txt | 194 +-
.../approved-plans-v1_4-spark4_0/q6/simplified.txt | 136 +-
.../approved-plans-v1_4-spark4_0/q60/explain.txt | 264 +--
.../q60/simplified.txt | 176 +-
.../approved-plans-v1_4-spark4_0/q61/explain.txt | 236 +--
.../q61/simplified.txt | 152 +-
.../approved-plans-v1_4-spark4_0/q62/explain.txt | 84 +-
.../q62/simplified.txt | 59 +-
.../approved-plans-v1_4-spark4_0/q63/explain.txt | 122 +-
.../q63/simplified.txt | 76 +-
.../approved-plans-v1_4-spark4_0/q64/explain.txt | 731 +++----
.../q64/simplified.txt | 455 +++--
.../approved-plans-v1_4-spark4_0/q65/explain.txt | 192 +-
.../q65/simplified.txt | 85 +-
.../approved-plans-v1_4-spark4_0/q66/explain.txt | 204 +-
.../q66/simplified.txt | 139 +-
.../approved-plans-v1_4-spark4_0/q67/explain.txt | 130 +-
.../q67/simplified.txt | 80 +-
.../approved-plans-v1_4-spark4_0/q68/explain.txt | 156 +-
.../q68/simplified.txt | 83 +-
.../approved-plans-v1_4-spark4_0/q69/explain.txt | 148 +-
.../q69/simplified.txt | 125 +-
.../approved-plans-v1_4-spark4_0/q7/explain.txt | 108 +-
.../approved-plans-v1_4-spark4_0/q7/simplified.txt | 79 +-
.../approved-plans-v1_4-spark4_0/q70/explain.txt | 192 +-
.../q70/simplified.txt | 133 +-
.../approved-plans-v1_4-spark4_0/q71/explain.txt | 130 +-
.../q71/simplified.txt | 97 +-
.../approved-plans-v1_4-spark4_0/q72/explain.txt | 32 +-
.../q72/simplified.txt | 4 +-
.../approved-plans-v1_4-spark4_0/q73/explain.txt | 22 +-
.../q73/simplified.txt | 4 +-
.../approved-plans-v1_4-spark4_0/q74/explain.txt | 330 +--
.../q74/simplified.txt | 172 +-
.../approved-plans-v1_4-spark4_0/q75/explain.txt | 471 ++---
.../q75/simplified.txt | 52 +-
.../approved-plans-v1_4-spark4_0/q76/explain.txt | 96 +-
.../q76/simplified.txt | 71 +-
.../approved-plans-v1_4-spark4_0/q77/explain.txt | 398 ++--
.../q77/simplified.txt | 249 +--
.../approved-plans-v1_4-spark4_0/q78/explain.txt | 290 +--
.../q78/simplified.txt | 151 +-
.../approved-plans-v1_4-spark4_0/q79/explain.txt | 120 +-
.../q79/simplified.txt | 71 +-
.../approved-plans-v1_4-spark4_0/q8/explain.txt | 134 +-
.../approved-plans-v1_4-spark4_0/q8/simplified.txt | 103 +-
.../approved-plans-v1_4-spark4_0/q80/explain.txt | 402 ++--
.../q80/simplified.txt | 254 +--
.../approved-plans-v1_4-spark4_0/q81/explain.txt | 222 +-
.../q81/simplified.txt | 101 +-
.../approved-plans-v1_4-spark4_0/q82/explain.txt | 18 +-
.../q82/simplified.txt | 4 +-
.../q83.ansi/explain.txt | 264 +--
.../q83.ansi/simplified.txt | 139 +-
.../approved-plans-v1_4-spark4_0/q84/explain.txt | 16 +-
.../q84/simplified.txt | 2 +-
.../approved-plans-v1_4-spark4_0/q85/explain.txt | 144 +-
.../q85/simplified.txt | 109 +-
.../approved-plans-v1_4-spark4_0/q86/explain.txt | 106 +-
.../q86/simplified.txt | 66 +-
.../approved-plans-v1_4-spark4_0/q87/explain.txt | 148 +-
.../q87/simplified.txt | 115 +-
.../approved-plans-v1_4-spark4_0/q88/explain.txt | 68 +-
.../q88/simplified.txt | 16 +-
.../approved-plans-v1_4-spark4_0/q89/explain.txt | 120 +-
.../q89/simplified.txt | 74 +-
.../approved-plans-v1_4-spark4_0/q9/explain.txt | 210 +-
.../approved-plans-v1_4-spark4_0/q9/simplified.txt | 97 +-
.../approved-plans-v1_4-spark4_0/q90/explain.txt | 20 +-
.../q90/simplified.txt | 4 +-
.../approved-plans-v1_4-spark4_0/q91/explain.txt | 140 +-
.../q91/simplified.txt | 101 +-
.../approved-plans-v1_4-spark4_0/q92/explain.txt | 134 +-
.../q92/simplified.txt | 96 +-
.../approved-plans-v1_4-spark4_0/q93/explain.txt | 68 +-
.../q93/simplified.txt | 47 +-
.../approved-plans-v1_4-spark4_0/q94/explain.txt | 108 +-
.../q94/simplified.txt | 85 +-
.../approved-plans-v1_4-spark4_0/q95/explain.txt | 134 +-
.../q95/simplified.txt | 101 +-
.../approved-plans-v1_4-spark4_0/q96/explain.txt | 12 +-
.../q96/simplified.txt | 2 +-
.../approved-plans-v1_4-spark4_0/q97/explain.txt | 92 +-
.../q97/simplified.txt | 73 +-
.../approved-plans-v1_4-spark4_0/q98/explain.txt | 116 +-
.../q98/simplified.txt | 66 +-
.../approved-plans-v1_4-spark4_0/q99/explain.txt | 84 +-
.../q99/simplified.txt | 59 +-
.../approved-plans-v1_4/q1/explain.txt | 20 +-
.../approved-plans-v1_4/q1/simplified.txt | 4 +-
.../approved-plans-v1_4/q10/explain.txt | 150 +-
.../approved-plans-v1_4/q10/simplified.txt | 125 +-
.../approved-plans-v1_4/q11/explain.txt | 36 +-
.../approved-plans-v1_4/q11/simplified.txt | 6 +-
.../approved-plans-v1_4/q12/explain.txt | 16 +-
.../approved-plans-v1_4/q12/simplified.txt | 4 +-
.../approved-plans-v1_4/q13/explain.txt | 22 +-
.../approved-plans-v1_4/q13/simplified.txt | 4 +-
.../approved-plans-v1_4/q14a/explain.txt | 50 +-
.../approved-plans-v1_4/q14a/simplified.txt | 8 +-
.../approved-plans-v1_4/q14b/explain.txt | 68 +-
.../approved-plans-v1_4/q14b/simplified.txt | 14 +-
.../approved-plans-v1_4/q15/explain.txt | 18 +-
.../approved-plans-v1_4/q15/simplified.txt | 4 +-
.../approved-plans-v1_4/q16/explain.txt | 110 +-
.../approved-plans-v1_4/q16/simplified.txt | 85 +-
.../approved-plans-v1_4/q17/explain.txt | 30 +-
.../approved-plans-v1_4/q17/simplified.txt | 6 +-
.../approved-plans-v1_4/q18/explain.txt | 24 +-
.../approved-plans-v1_4/q18/simplified.txt | 4 +-
.../approved-plans-v1_4/q19/explain.txt | 16 +-
.../approved-plans-v1_4/q19/simplified.txt | 2 +-
.../approved-plans-v1_4/q2/explain.txt | 14 +-
.../approved-plans-v1_4/q2/simplified.txt | 2 +-
.../approved-plans-v1_4/q20/explain.txt | 16 +-
.../approved-plans-v1_4/q20/simplified.txt | 4 +-
.../approved-plans-v1_4/q21/explain.txt | 20 +-
.../approved-plans-v1_4/q21/simplified.txt | 4 +-
.../approved-plans-v1_4/q22/explain.txt | 18 +-
.../approved-plans-v1_4/q22/simplified.txt | 4 +-
.../approved-plans-v1_4/q23a/explain.txt | 46 +-
.../approved-plans-v1_4/q23a/simplified.txt | 10 +-
.../approved-plans-v1_4/q23b/explain.txt | 48 +-
.../approved-plans-v1_4/q23b/simplified.txt | 10 +-
.../approved-plans-v1_4/q24a/explain.txt | 254 +--
.../approved-plans-v1_4/q24a/simplified.txt | 128 +-
.../approved-plans-v1_4/q24b/explain.txt | 254 +--
.../approved-plans-v1_4/q24b/simplified.txt | 128 +-
.../approved-plans-v1_4/q25/explain.txt | 30 +-
.../approved-plans-v1_4/q25/simplified.txt | 6 +-
.../approved-plans-v1_4/q26/explain.txt | 20 +-
.../approved-plans-v1_4/q26/simplified.txt | 4 +-
.../approved-plans-v1_4/q27/explain.txt | 20 +-
.../approved-plans-v1_4/q27/simplified.txt | 4 +-
.../approved-plans-v1_4/q28/explain.txt | 320 +--
.../approved-plans-v1_4/q28/simplified.txt | 126 +-
.../approved-plans-v1_4/q29/explain.txt | 38 +-
.../approved-plans-v1_4/q29/simplified.txt | 8 +-
.../approved-plans-v1_4/q3/explain.txt | 10 +-
.../approved-plans-v1_4/q3/simplified.txt | 2 +-
.../approved-plans-v1_4/q30/explain.txt | 22 +-
.../approved-plans-v1_4/q30/simplified.txt | 4 +-
.../approved-plans-v1_4/q31/explain.txt | 42 +-
.../approved-plans-v1_4/q31/simplified.txt | 8 +-
.../approved-plans-v1_4/q32/explain.txt | 18 +-
.../approved-plans-v1_4/q32/simplified.txt | 4 +-
.../approved-plans-v1_4/q33/explain.txt | 24 +-
.../approved-plans-v1_4/q33/simplified.txt | 4 +-
.../approved-plans-v1_4/q34/explain.txt | 22 +-
.../approved-plans-v1_4/q34/simplified.txt | 4 +-
.../approved-plans-v1_4/q35/explain.txt | 148 +-
.../approved-plans-v1_4/q35/simplified.txt | 123 +-
.../approved-plans-v1_4/q36/explain.txt | 18 +-
.../approved-plans-v1_4/q36/simplified.txt | 4 +-
.../approved-plans-v1_4/q37/explain.txt | 18 +-
.../approved-plans-v1_4/q37/simplified.txt | 4 +-
.../approved-plans-v1_4/q38/explain.txt | 20 +-
.../approved-plans-v1_4/q38/simplified.txt | 4 +-
.../approved-plans-v1_4/q39a/explain.txt | 36 +-
.../approved-plans-v1_4/q39a/simplified.txt | 6 +-
.../approved-plans-v1_4/q39b/explain.txt | 36 +-
.../approved-plans-v1_4/q39b/simplified.txt | 6 +-
.../approved-plans-v1_4/q4/explain.txt | 44 +-
.../approved-plans-v1_4/q4/simplified.txt | 6 +-
.../approved-plans-v1_4/q40/explain.txt | 20 +-
.../approved-plans-v1_4/q40/simplified.txt | 4 +-
.../approved-plans-v1_4/q41/explain.txt | 8 +-
.../approved-plans-v1_4/q41/simplified.txt | 2 +-
.../approved-plans-v1_4/q42/explain.txt | 10 +-
.../approved-plans-v1_4/q42/simplified.txt | 2 +-
.../approved-plans-v1_4/q43/explain.txt | 10 +-
.../approved-plans-v1_4/q43/simplified.txt | 2 +-
.../approved-plans-v1_4/q44/explain.txt | 22 +-
.../approved-plans-v1_4/q44/simplified.txt | 8 +-
.../approved-plans-v1_4/q45/explain.txt | 124 +-
.../approved-plans-v1_4/q45/simplified.txt | 93 +-
.../approved-plans-v1_4/q46/explain.txt | 22 +-
.../approved-plans-v1_4/q46/simplified.txt | 4 +-
.../approved-plans-v1_4/q47/explain.txt | 26 +-
.../approved-plans-v1_4/q47/simplified.txt | 8 +-
.../approved-plans-v1_4/q48/explain.txt | 20 +-
.../approved-plans-v1_4/q48/simplified.txt | 4 +-
.../approved-plans-v1_4/q49/explain.txt | 214 +-
.../approved-plans-v1_4/q49/simplified.txt | 219 +-
.../approved-plans-v1_4/q5/explain.txt | 204 +-
.../approved-plans-v1_4/q5/simplified.txt | 179 +-
.../approved-plans-v1_4/q50/explain.txt | 20 +-
.../approved-plans-v1_4/q50/simplified.txt | 4 +-
.../approved-plans-v1_4/q51/explain.txt | 163 +-
.../approved-plans-v1_4/q51/simplified.txt | 120 +-
.../approved-plans-v1_4/q52/explain.txt | 10 +-
.../approved-plans-v1_4/q52/simplified.txt | 2 +-
.../approved-plans-v1_4/q53/explain.txt | 18 +-
.../approved-plans-v1_4/q53/simplified.txt | 4 +-
.../approved-plans-v1_4/q54/explain.txt | 46 +-
.../approved-plans-v1_4/q54/simplified.txt | 10 +-
.../approved-plans-v1_4/q55/explain.txt | 10 +-
.../approved-plans-v1_4/q55/simplified.txt | 2 +-
.../approved-plans-v1_4/q56/explain.txt | 24 +-
.../approved-plans-v1_4/q56/simplified.txt | 4 +-
.../approved-plans-v1_4/q57/explain.txt | 26 +-
.../approved-plans-v1_4/q57/simplified.txt | 8 +-
.../approved-plans-v1_4/q58/explain.txt | 28 +-
.../approved-plans-v1_4/q58/simplified.txt | 6 +-
.../approved-plans-v1_4/q59/explain.txt | 16 +-
.../approved-plans-v1_4/q59/simplified.txt | 2 +-
.../approved-plans-v1_4/q6/explain.txt | 28 +-
.../approved-plans-v1_4/q6/simplified.txt | 6 +-
.../approved-plans-v1_4/q60/explain.txt | 24 +-
.../approved-plans-v1_4/q60/simplified.txt | 4 +-
.../approved-plans-v1_4/q61/explain.txt | 30 +-
.../approved-plans-v1_4/q61/simplified.txt | 6 +-
.../approved-plans-v1_4/q62/explain.txt | 14 +-
.../approved-plans-v1_4/q62/simplified.txt | 2 +-
.../approved-plans-v1_4/q63/explain.txt | 18 +-
.../approved-plans-v1_4/q63/simplified.txt | 4 +-
.../approved-plans-v1_4/q64/explain.txt | 50 +-
.../approved-plans-v1_4/q64/simplified.txt | 6 +-
.../approved-plans-v1_4/q65/explain.txt | 20 +-
.../approved-plans-v1_4/q65/simplified.txt | 4 +-
.../approved-plans-v1_4/q66/explain.txt | 22 +-
.../approved-plans-v1_4/q66/simplified.txt | 4 +-
.../approved-plans-v1_4/q67/explain.txt | 18 +-
.../approved-plans-v1_4/q67/simplified.txt | 4 +-
.../approved-plans-v1_4/q68/explain.txt | 22 +-
.../approved-plans-v1_4/q68/simplified.txt | 4 +-
.../approved-plans-v1_4/q69/explain.txt | 148 +-
.../approved-plans-v1_4/q69/simplified.txt | 125 +-
.../approved-plans-v1_4/q7/explain.txt | 20 +-
.../approved-plans-v1_4/q7/simplified.txt | 4 +-
.../approved-plans-v1_4/q70/explain.txt | 156 +-
.../approved-plans-v1_4/q70/simplified.txt | 126 +-
.../approved-plans-v1_4/q71/explain.txt | 22 +-
.../approved-plans-v1_4/q71/simplified.txt | 4 +-
.../approved-plans-v1_4/q72/explain.txt | 32 +-
.../approved-plans-v1_4/q72/simplified.txt | 4 +-
.../approved-plans-v1_4/q73/explain.txt | 22 +-
.../approved-plans-v1_4/q73/simplified.txt | 4 +-
.../approved-plans-v1_4/q74/explain.txt | 36 +-
.../approved-plans-v1_4/q74/simplified.txt | 6 +-
.../approved-plans-v1_4/q75/explain.txt | 40 +-
.../approved-plans-v1_4/q75/simplified.txt | 6 +-
.../approved-plans-v1_4/q76/explain.txt | 14 +-
.../approved-plans-v1_4/q76/simplified.txt | 2 +-
.../approved-plans-v1_4/q77/explain.txt | 230 +--
.../approved-plans-v1_4/q77/simplified.txt | 207 +-
.../approved-plans-v1_4/q78/explain.txt | 26 +-
.../approved-plans-v1_4/q78/simplified.txt | 4 +-
.../approved-plans-v1_4/q79/explain.txt | 20 +-
.../approved-plans-v1_4/q79/simplified.txt | 4 +-
.../approved-plans-v1_4/q8/explain.txt | 22 +-
.../approved-plans-v1_4/q8/simplified.txt | 4 +-
.../approved-plans-v1_4/q80/explain.txt | 270 +--
.../approved-plans-v1_4/q80/simplified.txt | 239 +--
.../approved-plans-v1_4/q81/explain.txt | 22 +-
.../approved-plans-v1_4/q81/simplified.txt | 4 +-
.../approved-plans-v1_4/q82/explain.txt | 18 +-
.../approved-plans-v1_4/q82/simplified.txt | 4 +-
.../approved-plans-v1_4/q83/explain.txt | 26 +-
.../approved-plans-v1_4/q83/simplified.txt | 4 +-
.../approved-plans-v1_4/q84/explain.txt | 16 +-
.../approved-plans-v1_4/q84/simplified.txt | 2 +-
.../approved-plans-v1_4/q85/explain.txt | 26 +-
.../approved-plans-v1_4/q85/simplified.txt | 4 +-
.../approved-plans-v1_4/q86/explain.txt | 16 +-
.../approved-plans-v1_4/q86/simplified.txt | 4 +-
.../approved-plans-v1_4/q87/explain.txt | 148 +-
.../approved-plans-v1_4/q87/simplified.txt | 115 +-
.../approved-plans-v1_4/q88/explain.txt | 68 +-
.../approved-plans-v1_4/q88/simplified.txt | 16 +-
.../approved-plans-v1_4/q89/explain.txt | 18 +-
.../approved-plans-v1_4/q89/simplified.txt | 4 +-
.../approved-plans-v1_4/q9/explain.txt | 36 +-
.../approved-plans-v1_4/q9/simplified.txt | 12 +-
.../approved-plans-v1_4/q90/explain.txt | 20 +-
.../approved-plans-v1_4/q90/simplified.txt | 4 +-
.../approved-plans-v1_4/q91/explain.txt | 24 +-
.../approved-plans-v1_4/q91/simplified.txt | 4 +-
.../approved-plans-v1_4/q92/explain.txt | 18 +-
.../approved-plans-v1_4/q92/simplified.txt | 4 +-
.../approved-plans-v1_4/q93/explain.txt | 10 +-
.../approved-plans-v1_4/q93/simplified.txt | 2 +-
.../approved-plans-v1_4/q94/explain.txt | 110 +-
.../approved-plans-v1_4/q94/simplified.txt | 85 +-
.../approved-plans-v1_4/q95/explain.txt | 136 +-
.../approved-plans-v1_4/q95/simplified.txt | 101 +-
.../approved-plans-v1_4/q96/explain.txt | 12 +-
.../approved-plans-v1_4/q96/simplified.txt | 2 +-
.../approved-plans-v1_4/q97/explain.txt | 16 +-
.../approved-plans-v1_4/q97/simplified.txt | 4 +-
.../approved-plans-v1_4/q98/explain.txt | 20 +-
.../approved-plans-v1_4/q98/simplified.txt | 6 +-
.../approved-plans-v1_4/q99/explain.txt | 14 +-
.../approved-plans-v1_4/q99/simplified.txt | 2 +-
.../approved-plans-v2_7-spark3_5/q10a/explain.txt | 24 +-
.../q10a/simplified.txt | 4 +-
.../approved-plans-v2_7-spark3_5/q11/explain.txt | 36 +-
.../q11/simplified.txt | 6 +-
.../approved-plans-v2_7-spark3_5/q12/explain.txt | 16 +-
.../q12/simplified.txt | 4 +-
.../approved-plans-v2_7-spark3_5/q14/explain.txt | 68 +-
.../q14/simplified.txt | 14 +-
.../approved-plans-v2_7-spark3_5/q14a/explain.txt | 58 +-
.../q14a/simplified.txt | 10 +-
.../approved-plans-v2_7-spark3_5/q18a/explain.txt | 40 +-
.../q18a/simplified.txt | 4 +-
.../approved-plans-v2_7-spark3_5/q20/explain.txt | 16 +-
.../q20/simplified.txt | 4 +-
.../approved-plans-v2_7-spark3_5/q22/explain.txt | 96 +-
.../q22/simplified.txt | 71 +-
.../approved-plans-v2_7-spark3_5/q22a/explain.txt | 18 +-
.../q22a/simplified.txt | 4 +-
.../approved-plans-v2_7-spark3_5/q24/explain.txt | 266 +--
.../q24/simplified.txt | 130 +-
.../approved-plans-v2_7-spark3_5/q27a/explain.txt | 28 +-
.../q27a/simplified.txt | 4 +-
.../approved-plans-v2_7-spark3_5/q34/explain.txt | 22 +-
.../q34/simplified.txt | 4 +-
.../approved-plans-v2_7-spark3_5/q35/explain.txt | 148 +-
.../q35/simplified.txt | 123 +-
.../approved-plans-v2_7-spark3_5/q35a/explain.txt | 24 +-
.../q35a/simplified.txt | 4 +-
.../approved-plans-v2_7-spark3_5/q36a/explain.txt | 18 +-
.../q36a/simplified.txt | 4 +-
.../approved-plans-v2_7-spark3_5/q47/explain.txt | 26 +-
.../q47/simplified.txt | 8 +-
.../approved-plans-v2_7-spark3_5/q49/explain.txt | 214 +-
.../q49/simplified.txt | 219 +-
.../approved-plans-v2_7-spark3_5/q51a/explain.txt | 315 +--
.../q51a/simplified.txt | 205 +-
.../approved-plans-v2_7-spark3_5/q57/explain.txt | 26 +-
.../q57/simplified.txt | 8 +-
.../approved-plans-v2_7-spark3_5/q5a/explain.txt | 288 +--
.../q5a/simplified.txt | 242 +--
.../approved-plans-v2_7-spark3_5/q6/explain.txt | 28 +-
.../approved-plans-v2_7-spark3_5/q6/simplified.txt | 6 +-
.../approved-plans-v2_7-spark3_5/q64/explain.txt | 50 +-
.../q64/simplified.txt | 6 +-
.../approved-plans-v2_7-spark3_5/q67a/explain.txt | 196 +-
.../q67a/simplified.txt | 155 +-
.../approved-plans-v2_7-spark3_5/q70a/explain.txt | 236 ++-
.../q70a/simplified.txt | 62 +-
.../approved-plans-v2_7-spark3_5/q72/explain.txt | 32 +-
.../q72/simplified.txt | 4 +-
.../approved-plans-v2_7-spark3_5/q74/explain.txt | 36 +-
.../q74/simplified.txt | 6 +-
.../approved-plans-v2_7-spark3_5/q75/explain.txt | 40 +-
.../q75/simplified.txt | 6 +-
.../approved-plans-v2_7-spark3_5/q77a/explain.txt | 314 +--
.../q77a/simplified.txt | 272 +--
.../approved-plans-v2_7-spark3_5/q78/explain.txt | 26 +-
.../q78/simplified.txt | 4 +-
.../approved-plans-v2_7-spark3_5/q80a/explain.txt | 354 ++--
.../q80a/simplified.txt | 300 +--
.../approved-plans-v2_7-spark3_5/q86a/explain.txt | 16 +-
.../q86a/simplified.txt | 4 +-
.../approved-plans-v2_7-spark3_5/q98/explain.txt | 20 +-
.../q98/simplified.txt | 6 +-
.../approved-plans-v2_7-spark4_0/q10a/explain.txt | 24 +-
.../q10a/simplified.txt | 4 +-
.../approved-plans-v2_7-spark4_0/q11/explain.txt | 330 +--
.../q11/simplified.txt | 172 +-
.../approved-plans-v2_7-spark4_0/q12/explain.txt | 104 +-
.../q12/simplified.txt | 64 +-
.../approved-plans-v2_7-spark4_0/q14/explain.txt | 452 +++--
.../q14/simplified.txt | 299 +--
.../approved-plans-v2_7-spark4_0/q14a/explain.txt | 638 +++---
.../q14a/simplified.txt | 446 ++--
.../approved-plans-v2_7-spark4_0/q18a/explain.txt | 592 +++---
.../q18a/simplified.txt | 339 ++--
.../approved-plans-v2_7-spark4_0/q20/explain.txt | 104 +-
.../q20/simplified.txt | 64 +-
.../approved-plans-v2_7-spark4_0/q22/explain.txt | 96 +-
.../q22/simplified.txt | 71 +-
.../approved-plans-v2_7-spark4_0/q22a/explain.txt | 220 +-
.../q22a/simplified.txt | 129 +-
.../approved-plans-v2_7-spark4_0/q24/explain.txt | 266 +--
.../q24/simplified.txt | 130 +-
.../approved-plans-v2_7-spark4_0/q27a/explain.txt | 290 +--
.../q27a/simplified.txt | 173 +-
.../approved-plans-v2_7-spark4_0/q34/explain.txt | 22 +-
.../q34/simplified.txt | 4 +-
.../approved-plans-v2_7-spark4_0/q35/explain.txt | 148 +-
.../q35/simplified.txt | 123 +-
.../approved-plans-v2_7-spark4_0/q35a/explain.txt | 128 +-
.../q35a/simplified.txt | 99 +-
.../approved-plans-v2_7-spark4_0/q36a/explain.txt | 198 +-
.../q36a/simplified.txt | 58 +-
.../approved-plans-v2_7-spark4_0/q47/explain.txt | 204 +-
.../q47/simplified.txt | 89 +-
.../approved-plans-v2_7-spark4_0/q49/explain.txt | 362 ++--
.../q49/simplified.txt | 249 +--
.../approved-plans-v2_7-spark4_0/q51a/explain.txt | 375 ++--
.../q51a/simplified.txt | 167 +-
.../approved-plans-v2_7-spark4_0/q57/explain.txt | 204 +-
.../q57/simplified.txt | 89 +-
.../approved-plans-v2_7-spark4_0/q5a/explain.txt | 388 ++--
.../q5a/simplified.txt | 253 +--
.../approved-plans-v2_7-spark4_0/q6/explain.txt | 194 +-
.../approved-plans-v2_7-spark4_0/q6/simplified.txt | 136 +-
.../approved-plans-v2_7-spark4_0/q64/explain.txt | 731 +++----
.../q64/simplified.txt | 455 +++--
.../approved-plans-v2_7-spark4_0/q67a/explain.txt | 372 ++--
.../q67a/simplified.txt | 244 +--
.../approved-plans-v2_7-spark4_0/q70a/explain.txt | 268 +--
.../q70a/simplified.txt | 95 +-
.../approved-plans-v2_7-spark4_0/q72/explain.txt | 32 +-
.../q72/simplified.txt | 4 +-
.../approved-plans-v2_7-spark4_0/q74/explain.txt | 330 +--
.../q74/simplified.txt | 172 +-
.../approved-plans-v2_7-spark4_0/q75/explain.txt | 471 ++---
.../q75/simplified.txt | 52 +-
.../approved-plans-v2_7-spark4_0/q77a/explain.txt | 480 +++--
.../q77a/simplified.txt | 300 +--
.../approved-plans-v2_7-spark4_0/q78/explain.txt | 290 +--
.../q78/simplified.txt | 151 +-
.../approved-plans-v2_7-spark4_0/q80a/explain.txt | 484 ++---
.../q80a/simplified.txt | 311 +--
.../approved-plans-v2_7-spark4_0/q86a/explain.txt | 184 +-
.../q86a/simplified.txt | 58 +-
.../approved-plans-v2_7-spark4_0/q98/explain.txt | 112 +-
.../q98/simplified.txt | 66 +-
.../approved-plans-v2_7/q10a/explain.txt | 24 +-
.../approved-plans-v2_7/q10a/simplified.txt | 4 +-
.../approved-plans-v2_7/q11/explain.txt | 36 +-
.../approved-plans-v2_7/q11/simplified.txt | 6 +-
.../approved-plans-v2_7/q12/explain.txt | 16 +-
.../approved-plans-v2_7/q12/simplified.txt | 4 +-
.../approved-plans-v2_7/q14/explain.txt | 68 +-
.../approved-plans-v2_7/q14/simplified.txt | 14 +-
.../approved-plans-v2_7/q14a/explain.txt | 58 +-
.../approved-plans-v2_7/q14a/simplified.txt | 10 +-
.../approved-plans-v2_7/q18a/explain.txt | 40 +-
.../approved-plans-v2_7/q18a/simplified.txt | 4 +-
.../approved-plans-v2_7/q20/explain.txt | 16 +-
.../approved-plans-v2_7/q20/simplified.txt | 4 +-
.../approved-plans-v2_7/q22/explain.txt | 96 +-
.../approved-plans-v2_7/q22/simplified.txt | 71 +-
.../approved-plans-v2_7/q22a/explain.txt | 18 +-
.../approved-plans-v2_7/q22a/simplified.txt | 4 +-
.../approved-plans-v2_7/q24/explain.txt | 266 +--
.../approved-plans-v2_7/q24/simplified.txt | 130 +-
.../approved-plans-v2_7/q27a/explain.txt | 28 +-
.../approved-plans-v2_7/q27a/simplified.txt | 4 +-
.../approved-plans-v2_7/q34/explain.txt | 22 +-
.../approved-plans-v2_7/q34/simplified.txt | 4 +-
.../approved-plans-v2_7/q35/explain.txt | 148 +-
.../approved-plans-v2_7/q35/simplified.txt | 123 +-
.../approved-plans-v2_7/q35a/explain.txt | 24 +-
.../approved-plans-v2_7/q35a/simplified.txt | 4 +-
.../approved-plans-v2_7/q36a/explain.txt | 18 +-
.../approved-plans-v2_7/q36a/simplified.txt | 4 +-
.../approved-plans-v2_7/q47/explain.txt | 26 +-
.../approved-plans-v2_7/q47/simplified.txt | 8 +-
.../approved-plans-v2_7/q49/explain.txt | 214 +-
.../approved-plans-v2_7/q49/simplified.txt | 219 +-
.../approved-plans-v2_7/q51a/explain.txt | 315 +--
.../approved-plans-v2_7/q51a/simplified.txt | 205 +-
.../approved-plans-v2_7/q57/explain.txt | 26 +-
.../approved-plans-v2_7/q57/simplified.txt | 8 +-
.../approved-plans-v2_7/q5a/explain.txt | 288 +--
.../approved-plans-v2_7/q5a/simplified.txt | 242 +--
.../approved-plans-v2_7/q6/explain.txt | 28 +-
.../approved-plans-v2_7/q6/simplified.txt | 6 +-
.../approved-plans-v2_7/q64/explain.txt | 50 +-
.../approved-plans-v2_7/q64/simplified.txt | 6 +-
.../approved-plans-v2_7/q67a/explain.txt | 18 +-
.../approved-plans-v2_7/q67a/simplified.txt | 4 +-
.../approved-plans-v2_7/q70a/explain.txt | 234 ++-
.../approved-plans-v2_7/q70a/simplified.txt | 62 +-
.../approved-plans-v2_7/q72/explain.txt | 32 +-
.../approved-plans-v2_7/q72/simplified.txt | 4 +-
.../approved-plans-v2_7/q74/explain.txt | 36 +-
.../approved-plans-v2_7/q74/simplified.txt | 6 +-
.../approved-plans-v2_7/q75/explain.txt | 40 +-
.../approved-plans-v2_7/q75/simplified.txt | 6 +-
.../approved-plans-v2_7/q77a/explain.txt | 314 +--
.../approved-plans-v2_7/q77a/simplified.txt | 272 +--
.../approved-plans-v2_7/q78/explain.txt | 26 +-
.../approved-plans-v2_7/q78/simplified.txt | 4 +-
.../approved-plans-v2_7/q80a/explain.txt | 354 ++--
.../approved-plans-v2_7/q80a/simplified.txt | 300 +--
.../approved-plans-v2_7/q86a/explain.txt | 16 +-
.../approved-plans-v2_7/q86a/simplified.txt | 4 +-
.../approved-plans-v2_7/q98/explain.txt | 20 +-
.../approved-plans-v2_7/q98/simplified.txt | 6 +-
.../scala/org/apache/comet/CometCastSuite.scala | 74 +-
.../org/apache/comet/CometExpressionSuite.scala | 314 ++-
.../scala/org/apache/comet/CometNativeSuite.scala | 53 +
.../apache/comet/exec/CometAggregateSuite.scala | 49 +
.../comet/exec/CometColumnarShuffleSuite.scala | 17 +-
.../org/apache/comet/exec/CometExecSuite.scala | 157 +-
.../org/apache/comet/exec/CometJoinSuite.scala | 8 +-
.../apache/comet/parquet/ParquetReadSuite.scala | 91 +-
.../scala/org/apache/spark/sql/CometTestBase.scala | 10 +-
.../spark/sql/benchmark/CometExecBenchmark.scala | 57 +
.../spark/sql/comet/CometPlanStabilitySuite.scala | 4 +
1072 files changed, 53353 insertions(+), 41547 deletions(-)
delete mode 100755 bin/comet-spark-shell
delete mode 100644
common/src/main/scala/org/apache/spark/sql/comet/execution/shuffle/IpcInputStreamIterator.scala
create mode 100644 dev/changelog/0.3.0.md
create mode 100644 dev/changelog/0.4.0.md
create mode 100755 dev/release/build-release-comet.sh
create mode 100644 dev/release/comet-rm/Dockerfile
copy docs/build.sh => dev/release/comet-rm/build-comet-native-libs.sh (60%)
create mode 100755 dev/release/publish-to-maven.sh
rename docs/source/_static/images/benchmark-results/{2024-08-23 =>
0.2.0}/tpcds_allqueries.png (100%)
rename docs/source/_static/images/benchmark-results/{2024-08-23 =>
0.2.0}/tpcds_queries_compare.png (100%)
rename docs/source/_static/images/benchmark-results/{2024-08-23 =>
0.2.0}/tpcds_queries_speedup_abs.png (100%)
rename docs/source/_static/images/benchmark-results/{2024-08-23 =>
0.2.0}/tpch_allqueries.png (100%)
rename docs/source/_static/images/benchmark-results/{2024-08-23 =>
0.2.0}/tpch_queries_compare.png (100%)
rename docs/source/_static/images/benchmark-results/{2024-08-23 =>
0.2.0}/tpch_queries_speedup_rel.png (100%)
create mode 100644
docs/source/_static/images/benchmark-results/0.3.0/tpcds_allqueries.png
create mode 100644
docs/source/_static/images/benchmark-results/0.3.0/tpcds_queries_compare.png
create mode 100644
docs/source/_static/images/benchmark-results/0.3.0/tpcds_queries_speedup_abs.png
create mode 100644
docs/source/_static/images/benchmark-results/0.3.0/tpcds_queries_speedup_rel.png
create mode 100644
docs/source/_static/images/benchmark-results/0.3.0/tpch_allqueries.png
create mode 100644
docs/source/_static/images/benchmark-results/0.3.0/tpch_queries_compare.png
create mode 100644
docs/source/_static/images/benchmark-results/0.3.0/tpch_queries_speedup_abs.png
create mode 100644
docs/source/_static/images/benchmark-results/0.3.0/tpch_queries_speedup_rel.png
create mode 100644
docs/source/_static/images/benchmark-results/0.4.0/tpcds_allqueries.png
create mode 100644
docs/source/_static/images/benchmark-results/0.4.0/tpcds_queries_compare.png
create mode 100644
docs/source/_static/images/benchmark-results/0.4.0/tpcds_queries_speedup_abs.png
create mode 100644
docs/source/_static/images/benchmark-results/0.4.0/tpcds_queries_speedup_rel.png
create mode 100644
docs/source/_static/images/benchmark-results/0.4.0/tpch_allqueries.png
create mode 100644
docs/source/_static/images/benchmark-results/0.4.0/tpch_queries_compare.png
create mode 100644
docs/source/_static/images/benchmark-results/0.4.0/tpch_queries_speedup_abs.png
create mode 100644
docs/source/_static/images/benchmark-results/0.4.0/tpch_queries_speedup_rel.png
create mode 100644 docs/source/_static/images/comet-dataflow.excalidraw
create mode 100644 docs/source/_static/images/comet-dataflow.svg
rename docs/source/contributor-guide/benchmark-results/{2024-08-23 =>
0.2.0}/comet-tpcds.json (100%)
rename docs/source/contributor-guide/benchmark-results/{2024-08-23 =>
0.2.0}/comet-tpch.json (100%)
rename docs/source/contributor-guide/benchmark-results/{2024-08-23 =>
0.2.0}/spark-tpcds.json (100%)
rename docs/source/contributor-guide/benchmark-results/{2024-08-23 =>
0.2.0}/spark-tpch.json (100%)
create mode 100644
docs/source/contributor-guide/benchmark-results/0.3.0/comet-tpcds.json
create mode 100644
docs/source/contributor-guide/benchmark-results/0.3.0/comet-tpch.json
create mode 100644
docs/source/contributor-guide/benchmark-results/0.3.0/spark-tpcds.json
create mode 100644
docs/source/contributor-guide/benchmark-results/0.3.0/spark-tpch.json
create mode 100644
docs/source/contributor-guide/benchmark-results/0.4.0/comet-tpcds.json
create mode 100644
docs/source/contributor-guide/benchmark-results/0.4.0/comet-tpch.json
create mode 100644
docs/source/contributor-guide/benchmark-results/0.4.0/spark-tpcds.json
create mode 100644
docs/source/contributor-guide/benchmark-results/0.4.0/spark-tpch.json
rename docs/source/contributor-guide/benchmark-results/{2024-08-23 =>
datafusion-40.1.0}/datafusion-python-tpch.json (100%)
create mode 100644 docs/source/user-guide/metrics.md
create mode 100644 docs/source/user-guide/source.md
copy {spark => native}/README.md (72%)
copy native/core/benches/{aggregate.rs => bloom_filter_agg.rs} (52%)
delete mode 100644 native/core/benches/hash.rs
delete mode 100644 native/core/src/data_type.rs
delete mode 100644 native/core/src/execution/datafusion/expressions/mod.rs
delete mode 100644 native/core/src/execution/datafusion/expressions/strings.rs
create mode 100644 native/core/src/execution/expressions/bloom_filter_agg.rs
rename native/core/src/execution/{datafusion =>
}/expressions/bloom_filter_might_contain.rs (83%)
copy native/core/src/execution/{kernels => expressions}/mod.rs (83%)
rename native/core/src/execution/{datafusion => }/expressions/subquery.rs (92%)
delete mode 100644 native/core/src/execution/kernels/hash.rs
rename native/core/src/execution/{datafusion => }/operators/expand.rs (94%)
rename native/core/src/execution/{datafusion => }/planner.rs (79%)
rename native/core/src/execution/{datafusion => shuffle}/shuffle_writer.rs
(70%)
create mode 100644 native/core/src/execution/spark_plan.rs
rename native/core/src/execution/{datafusion => }/util/mod.rs (100%)
rename native/core/src/execution/{datafusion => }/util/spark_bit_array.rs (58%)
rename native/core/src/execution/{datafusion => }/util/spark_bloom_filter.rs
(53%)
delete mode 100644 native/core/src/parquet/compression.rs
delete mode 100644 native/core/src/parquet/util/jni_buffer.rs
rename native/{core => spark-expr}/benches/aggregate.rs (94%)
rename native/{core/src/execution/datafusion/expressions =>
spark-expr/src/agg_funcs}/avg.rs (93%)
rename native/{core/src/execution/datafusion/expressions =>
spark-expr/src/agg_funcs}/avg_decimal.rs (92%)
rename native/{core/src/execution/datafusion/expressions =>
spark-expr/src/agg_funcs}/correlation.rs (88%)
rename native/{core/src/execution/datafusion/expressions =>
spark-expr/src/agg_funcs}/covariance.rs (91%)
copy native/{core/src/execution/kernels => spark-expr/src/agg_funcs}/mod.rs
(72%)
rename native/{core/src/execution/datafusion/expressions =>
spark-expr/src/agg_funcs}/stddev.rs (86%)
rename native/{core/src/execution/datafusion/expressions =>
spark-expr/src/agg_funcs}/sum_decimal.rs (76%)
rename native/{core/src/execution/datafusion/expressions =>
spark-expr/src/agg_funcs}/variance.rs (90%)
rename native/{core/src/execution/datafusion/expressions =>
spark-expr/src}/bitwise_not.rs (79%)
rename native/{core/src/execution/datafusion/expressions =>
spark-expr/src}/checkoverflow.rs (88%)
rename native/{core/src/execution/datafusion/expressions =>
spark-expr/src}/comet_scalar_funcs.rs (93%)
create mode 100644 native/spark-expr/src/datetime_funcs/date_arithmetic.rs
create mode 100644 native/spark-expr/src/datetime_funcs/date_trunc.rs
create mode 100644 native/spark-expr/src/datetime_funcs/hour.rs
create mode 100644 native/spark-expr/src/datetime_funcs/minute.rs
copy native/{core/src/execution/kernels =>
spark-expr/src/datetime_funcs}/mod.rs (72%)
create mode 100644 native/spark-expr/src/datetime_funcs/second.rs
create mode 100644 native/spark-expr/src/datetime_funcs/timestamp_trunc.rs
copy native/{core/src/execution/kernels => spark-expr/src/hash_funcs}/mod.rs
(79%)
create mode 100644 native/spark-expr/src/hash_funcs/murmur3.rs
create mode 100644 native/spark-expr/src/hash_funcs/sha2.rs
create mode 100644 native/spark-expr/src/hash_funcs/utils.rs
create mode 100644 native/spark-expr/src/hash_funcs/xxhash64.rs
copy native/spark-expr/src/{kernels => json_funcs}/mod.rs (95%)
rename native/spark-expr/src/{ => json_funcs}/to_json.rs (95%)
rename native/{core/src/execution => spark-expr/src}/kernels/strings.rs (52%)
rename native/{core/src/execution/datafusion/expressions =>
spark-expr/src}/negative.rs (93%)
rename native/{core/src/execution/datafusion/expressions =>
spark-expr/src}/normalize_nan.rs (88%)
create mode 100644 native/spark-expr/src/predicate_funcs/is_nan.rs
copy native/{core/src/execution/kernels =>
spark-expr/src/predicate_funcs}/mod.rs (91%)
rename native/spark-expr/src/{regexp.rs => predicate_funcs/rlike.rs} (91%)
delete mode 100644 native/spark-expr/src/scalar_funcs/hash_expressions.rs
create mode 100644 native/spark-expr/src/schema_adapter.rs
delete mode 100644 native/spark-expr/src/spark_hash.rs
copy native/{core/src/execution/kernels =>
spark-expr/src/static_invoke/char_varchar_utils}/mod.rs (91%)
create mode 100644
native/spark-expr/src/static_invoke/char_varchar_utils/read_side_padding.rs
copy native/{core/src/execution/kernels =>
spark-expr/src/static_invoke}/mod.rs (91%)
rename native/{core/src/execution/kernels =>
spark-expr/src/string_funcs}/mod.rs (84%)
create mode 100644 native/spark-expr/src/string_funcs/prediction.rs
create mode 100644 native/spark-expr/src/string_funcs/string_space.rs
create mode 100644 native/spark-expr/src/string_funcs/substring.rs
rename native/spark-expr/src/{structs.rs =>
struct_funcs/create_named_struct.rs} (57%)
create mode 100644 native/spark-expr/src/struct_funcs/get_struct_field.rs
rename native/{core/src/execution/datafusion =>
spark-expr/src/struct_funcs}/mod.rs (85%)
delete mode 100644 native/spark-expr/src/temporal.rs
rename native/{core/src/parquet/util =>
spark-expr/src}/test_common/file_util.rs (100%)
rename native/{core/src/execution/datafusion/operators =>
spark-expr/src/test_common}/mod.rs (97%)
rename native/{core/src/execution/datafusion/expressions =>
spark-expr/src}/unbound.rs (86%)
delete mode 100644 native/spark-expr/src/xxhash64.rs
copy
spark/src/main/java/org/apache/spark/shuffle/comet/{TooLargePageException.java
=> CometShuffleMemoryAllocatorTrait.java} (52%)
copy
spark/src/main/java/org/apache/spark/shuffle/comet/{CometShuffleMemoryAllocator.java
=> CometTestShuffleMemoryAllocator.java} (87%)
create mode 100644
spark/src/main/scala/org/apache/comet/rules/RewriteJoin.scala
create mode 100644
spark/src/main/scala/org/apache/spark/sql/comet/CometColumnarToRowExec.scala
rename {common =>
spark}/src/main/scala/org/apache/spark/sql/comet/execution/shuffle/ArrowReaderIterator.scala
(100%)
rename {common =>
spark}/src/main/scala/org/apache/spark/sql/comet/execution/shuffle/CometBlockStoreShuffleReader.scala
(89%)
rename {common =>
spark}/src/main/scala/org/apache/spark/sql/comet/execution/shuffle/CometShuffleDependency.scala
(94%)
rename {common =>
spark}/src/main/scala/org/apache/spark/sql/comet/execution/shuffle/CometShuffledRowRDD.scala
(95%)
create mode 100644
spark/src/main/scala/org/apache/spark/sql/comet/execution/shuffle/NativeBatchDecoderIterator.scala
rename {common =>
spark}/src/main/scala/org/apache/spark/sql/comet/execution/shuffle/ShuffleUtils.scala
(52%)
create mode 100644 spark/src/test/scala/org/apache/comet/CometNativeSuite.scala
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
