andygrove commented on PR #5110:
URL:
https://github.com/apache/datafusion-comet/pull/5110#issuecomment-5121440471
The latest commit expands test coverage: ordering preservation when sampling
above a sort, sampling above an aggregate, SQL `TABLESAMPLE`, and empty batch
interleaving in the native operator. It also adds a `Sample` case to
`CometExecBenchmark` comparing against Spark.
Benchmark results (Apple M3 Ultra, OpenJDK 17, Spark 4.1 profile, release
build, 10M rows):
```
OpenJDK 64-Bit Server VM 17.0.17+10 on Mac OS X 26.3.1
Apple M3 Ultra
Sample Exec (fraction 0.05): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
SQL Parquet - Spark 35 38
2 303.9 3.3 1.0X
SQL Parquet - Comet 22 24
2 478.9 2.1 1.6X
Sample Exec (fraction 0.5): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
SQL Parquet - Spark 62 63
1 168.9 5.9 1.0X
SQL Parquet - Comet 36 38
2 289.1 3.5 1.7X
Sample Exec (fraction 0.95): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
SQL Parquet - Spark 48 49
2 220.5 4.5 1.0X
SQL Parquet - Comet 49 51
2 215.8 4.6 1.0X
```
Comet is roughly 1.6x to 1.7x faster at low and medium fractions, and on par
at fraction 0.95, where nearly every row passes and the cost is dominated by
scan and row copying rather than the sampling filter.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
