qlong commented on PR #16714:
URL: https://github.com/apache/iceberg/pull/16714#issuecomment-4931101522
@tmater Here is 1 e2e test runs using 1-days Github activity data.
partial delegation is what the current state of PR, full means all cast is done
by Spark. 1 extaction per query.
**c-q05, c-q05b, c-q08 all extract to long, they show biggest performance
gain. c-q09 extract boolan, all other queries extract a string value, they do
not show much difference.**
I think in the long term fast inline cast is worth it at least for numbers,
more so if more then 1 extraction per queries. But I am open to remove the
inline cast and related cross check tests for now if you think the PR is
already too big. If we go that way, we need to keep inline cast for
nanoseconds until spark supports it.
```
Compare: gha-payload-full-delegation-20260709 vs
gha-payload-partial-delegation-20260709 (metric: query_median)
Run A:
/Users/qlong/opensources/variant-conformance-benchmark/results/gha-payload-full-delegation-20260709/gha-payload/timings-variant.csv
Run B:
/Users/qlong/opensources/variant-conformance-benchmark/results/gha-payload-partial-delegation-20260709/gha-payload/timings-variant.csv
query median_A median_B delta_s delta_%
------ -------- -------- ------- -------
c-q01 2.73 2.66 +0.08 +2.9%
c-q04 3.97 3.93 +0.03 +0.9%
c-q05 73.18 65.64 +7.54 +11.5%
c-q05b 3.76 3.33 +0.43 +12.9%
c-q06 4.42 4.34 +0.09 +2.0%
c-q07 4.18 4.17 +0.01 +0.2%
c-q08 3.54 3.29 +0.25 +7.7%
c-q09 4.26 4.30 -0.04 -1.0%
c-q10 4.40 4.26 +0.14 +3.2%
c-q11 4.25 4.26 -0.02 -0.4%
c-q12 3.80 3.96 -0.16 -3.9%
c-q13 3.66 3.65 +0.01 +0.3%
c-q14 4.15 4.10 +0.05 +1.3%
Summary: 13 queries, 13 comparable
Geo-mean delta: +2.8% (Run B faster)
Total (query_median): 120.3s vs 111.9s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
