dwsmith1983 commented on PR #5612: URL: https://github.com/apache/datafusion-comet/pull/5612#issuecomment-5508444878
Ran the benchmark on a 10 core Apple M5 (4P + 6E), comparing this branch against the base commit it sits on. The harness builds the UDFs through `create_comet_physical_fun` and calls `invoke_with_args` with the pattern arriving as a scalar argument on every invoke, which is the native path and the reason the cache exists. Matrix: regexp_extract, regexp_extract_all, and split with a regex delimiter (`[,;|]+`), shared and per worker UDF instances, 1/2/4/8 workers, 512 and 8192 row batches, warm and alternating pattern regimes, 4M rows per cell, two full replicates. Outputs were verified byte identical between main and this PR in every cell. Warm regime, per worker instances (matches real plans, where the pattern is a literal and each task gets its own expression instance): | function | workers | rows/batch | main Mrows/s | this PR Mrows/s | change | |---|--:|--:|--:|--:|--:| | regexp_extract | 1 | 512 | 6.9 | 17.9 | +159% | | regexp_extract | 8 | 512 | 5.8 | 62.8 | +986% | | regexp_extract | 8 | 8192 | 79.3 | 111.9 | +41% | | regexp_extract_all | 8 | 512 | 3.2 | 6.5 | +103% | | split | 8 | 512 | 25.6 | 30.2 | +18% | | split | 8 | 8192 | 31.7 | 31.8 | 0% | Main anti-scales on small batches: 8 threads run slower than 1 because every thread recompiles the pattern per batch and the compiles hammer the allocator. This PR scales near linearly. Allocations per 512 row batch for regexp_extract drop from 988 to 82 (the compile alone is roughly 900 allocations and 0.7 MB). Per batch latency follows the same shape, for example 699us mean / 1271us p99 down to 62us / 106us in the 8 worker 512 row cell. Worst case for the one slot cache, a pattern that alternates on every single invoke: within 2 percent of main across all three functions and both batch sizes, since the miss path pays the same compile main always pays plus an uncontended mutex. Cold first invoke on a fresh instance is also unchanged (for example 415us on main vs 403us here for regexp_extract on 8192 rows). One honest caveat: an artificial control where a single UDF instance is shared across 8 threads simultaneously regresses regexp_extract_all on 8192 row batches by 6 to 29 percent. The threads contend on the shared compiled Regex's internal scratch pool in that setup, while main sidesteps it by compiling privately per batch, which is the same behavior causing the anti-scaling above. That configuration does not occur in Comet since each task deserializes its own plan and gets its own expression instance, and regexp_extract and split win in shared mode anyway. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
