andygrove opened a new pull request, #5381: URL: https://github.com/apache/datafusion-comet/pull/5381
## Which issue does this PR close? N/A. This adds a benchmark for an existing operator, so there is no behavior change to track. ## Rationale for this change `CometExplodeExec` has no benchmark. Generator cost is driven by dimensions that other operator benchmarks do not exercise: how far each input row fans out, whether the generator emits a position column or preserves rows with null and empty arrays, the element type being unnested, and how many columns have to be replicated alongside the generated one. Without numbers for those, there is no way to tell whether a change to the operator helped, and no way to see which shapes Comet declines to convert. ## What changes are included in this PR? A new `CometExplodeBenchmark` with 12 cases over 256K input rows, in four groups: - **Fan-out**: `explode` over `array<bigint>` of length 2, 10, and 100. - **Generator variants**: `explode`, `posexplode`, `explode_outer`, `posexplode_outer`. - **Element type**: `array<bigint>`, `array<string>`, `array<struct<a bigint, b string>>`. - **Carried columns**: `explode` alone against `explode` plus three passthrough columns. Two details worth calling out for reviewers: One in ten rows holds a null array and another one in ten holds an empty array. Without that, `explode_outer` and `explode` would produce identical output and the variant group would be measuring the same query twice. The empty array is built with `slice`, not `array()`, because `array()` types as `array<null>`. Each array column gets its own temp view rather than sharing one wide table, so a case is never charged for scanning an array column it does not read. Reported times are whole-query totals and include the Parquet scan and result transfer, which is noted in the class comment: at fan-out 2 the scan is a large share of the total and the ratio understates the difference between the two implementations. The `Rate` and `Per Row` columns are per input row, not per generated row, since fan-out varies across cases. ## How are these changes tested? Run locally on an Apple M3 Ultra with `local[5]`: ```sh SPARK_GENERATE_BENCHMARK_FILES=1 make benchmark-org.apache.spark.sql.benchmark.CometExplodeBenchmark ``` All 12 cases complete, and none trips the harness warning for a plan that is not fully Comet native, including the `array<struct>` case. Comet is roughly 2x Spark across every case, and flat across fan-out, generator variant, and element type (Best ms, Spark then Comet): | Case | Spark | Comet | Relative | | ---- | ----- | ----- | -------- | | fan-out 2 / 10 / 100 | 20 / 27 / 150 | 10 / 14 / 75 | 2.1X / 1.9X / 2.0X | | explode / posexplode / explode_outer / posexplode_outer | 26 / 25 / 25 / 25 | 14 / 14 / 12 / 13 | 1.9X / 1.8X / 2.1X / 1.9X | | bigint / string / struct | 25 / 32 / 42 | 13 / 18 / 24 | 2.0X / 1.8X / 1.8X | | alone / plus 3 carried columns | 25 / 30 | 12 / 18 | 2.0X / 1.7X | Correctness of the query forms themselves is already covered by `CometGenerateExecSuite`, which tests explode and explode_outer over simple, empty, and null arrays. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
