andygrove opened a new issue, #6855: URL: https://github.com/apache/datafusion-comet/issues/6855
### What is the problem the feature request solves? The microbenchmarks for columnar-to-row (C2R) and row-to-columnar (R2C) conversion covered nested types thinly: no nulls in nested data, no collections of structs or arrays, no string-element collections in the native R2C bench, and no nested end-to-end case for `CometSparkToColumnarExec`. #6858 closes those gaps (see the list there). Running the extended suites on a 32-thread Linux box (Ryzen 9 7950X3D, JDK 17, `local[1]`, 1M rows, release build) turned up the cases below where Comet is slower than Spark, or where the nested cost is out of proportion. This epic tracks them. ### Where Comet is slower than Spark - [ ] **Spark Parquet scan converted to Arrow (`CometSparkToColumnarExec`) loses to Spark on nested shapes.** Plan is scan, native filter, columnar-to-row. Relative to Spark: nullable `array<string>` 0.7X, `map<string,string>` 0.8X, struct holding `array<string>` 0.8X, 20-field struct 0.8X, depth-3 struct 0.9X, nullable struct 0.9X, flat struct 1.0X. The same plan on the native scan is 1.3-2.4X. Related: #6565, #6720, #6721, #6722 (arrays other than `array<string>` and maps other than `map<string,string>` are not converted at all today). - [ ] **JVM shuffle with 201 partitions is slower than Spark on nested types.** At 1M rows every new shape runs 0.7-0.8X (`array<string>`, `array<struct>`, `array<array<int>>`, `map<string,int>`, `map<int,array<string>>`, structs with strings, nullable structs and arrays). On the fuzz-generated nested schema (1000 rows, 201 partitions) it is far worse: 822 ms vs 94 ms for Spark at max depth 2 and 7,996 ms vs 380 ms at max depth 6. At 5 partitions it is at or near parity. That shape suggests a per-partition fixed cost (builder construction per partition per column) rather than per-row work. Related: #5905. - [ ] **Native columnar-to-row is slow at small batch sizes, and the nested case is worse.** Isolated, no scan, against JVM `rowIterator` + `UnsafeProjection` (a proxy for Spark's path): flat schema 0.4X / 0.3X / 0.1X at batch sizes 8192 / 512 / 32; nested schema 0.9X / 0.7X / 0.2X, which is 1.55 us per row at batch size 32 (about 42 us per batch above the flat case). Small batches come from filters, joins and shuffle reads. Related: #5112, which has the flat numbers. - [ ] **Native C2R is slower than Comet's own JVM C2R at the default batch size** for primitives (84 ms vs 54 ms), dictionary-encoded strings (63 vs 48 ms) and complex nested types (291 vs 268 ms). It still beats Spark in every `CometColumnarToRowBenchmark` group (1.1-2.2X), so this is a tuning question rather than a regression against Spark. - [ ] **Hash-partitioning on nested keys.** Native shuffle on an `array<struct<int,string>>` key is 0.9X and `struct<map<string,int>,int>` keys are 0.9X, with the Spark-shuffle and JVM-shuffle arms at 0.5-0.9X. ### Nested conversion cost (Comet only, no Spark counterpart) - [ ] **`ArrowWriter` nested columns cost 10-30x a flat column, and the row path is 3-6x slower than the columnar path.** ns per row, columnar / rows: int 0.8 / 6.3, `array<string>` 17 / 60, `map<string,string>` 25 / 100, `array<struct<int,string>>` 35 / 96, `array<array<int>>` 48 / 82, `map<string,array<int>>` 60 / 137. Related: #6565, #6721. ### Benchmark follow-ups - [ ] The pre-existing `row_columnar` case `list_conversion/elements_100` takes about 1.6 ms at both 1,000 and 10,000 rows, and bytes written are not monotonic in the row count (141 KB at 1k rows, 31 KB at 10k; the 10%-null variant writes 6 KB at 1k and 1.4 MB at 10k). Either the benchmark data or a data-dependent path is wrong. Do not read the 100-element list numbers until this is understood. - [ ] `CometC2RIsolatedBench` writes no results file (it is excluded from `benchmarks/micro/run.py`), so its numbers are not tracked. Make it a `BenchmarkBase` suite. - [ ] No Spark-comparison benchmark for R2C from a row source (RDD, local relation) with nested types. - [ ] Add a small-batch variant of the nested C2R and Spark-to-Arrow cases to the SQL benchmarks, since the SQL path always uses full batches. ### Method Suites run one at a time on the same machine: `CometColumnarToRowBenchmark`, `CometSparkToColumnarBenchmark`, `CometShuffleBenchmark --nested-only`, `CometArrowWriterBenchmark`, `CometC2RIsolatedBench`, and the criterion `row_columnar` bench. Spark's `Benchmark` reports best time over at least 2 s of iterations after a 2 s warmup; criterion runs were not CPU-pinned except one rerun. Numbers are from a single run per case. ### Related - #5112 native C2R slow for small batches - #5905 shuffle reader and writer performance review - #6565 faster Spark-to-Arrow conversion in `CometSparkToColumnarExec` - #6720, #6721, #6722 follow-ups under #6565 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
