voonhous opened a new issue, #20626:
URL: https://github.com/apache/hudi/issues/20626

   Part of #20625. Line references are against master `db90a0da`.
   
   ### What
   Add two local benchmarks to `hudi-trino` test scope (package 
`io.trino.plugin.hudi.benchmark`) so the other MOR read work items can be 
measured without a cluster or object store:
   
   - `HudiMergeBufferBenchmark`: one file group's log records through the 
spillable merge map, with the size estimator and serializer from a real 
`HudiTrinoReaderContext`.
   - `HudiMorReadBenchmark`: a local `DistributedQueryRunner` over a generated 
TPC-DS-shaped MOR table (table version 9, 4 log files per file group, MOR 
`date_dim`), with storage behind a wrapper that injects object-store latency 
and counts GET/HEAD/LIST per file type.
   
   ### Exit criteria
   - Classes are named `*Benchmark`, so surefire and CI never run them; nothing 
else in the module changes apart from a short "Benchmarks" section in 
`hudi-trino/README.md` with the run commands and knobs.
   - Each benchmark asserts its results against the generator's expected 
values, so a fast but wrong change fails the run.
   - On master: the merge buffer spills more than 50% of 300k records with an 
estimate above 10 KB/record; scan `_rt` reads `hoodie.properties` once per 
split and the timeline twice per split.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to