Rich-T-kid commented on PR #10591: URL: https://github.com/apache/arrow-rs/pull/10591#issuecomment-5241991303
> something i noticed was the other benches in this file seem to be at the page iterator or array reader level, whilst this new one is pretty much at the top level with the record batch reader api > > is this something to be concerned about, or the behaviour we're trying to benchmark here can only be done via the top level api? @Jefffrey The existing benchmarks measure the time to decode an already dictionary-encoded Parquet page into an Arrow dictionary array. This PR's benchmarks measure converting a plain-encoded (non-dictionary) string column into a dictionary array. I used the top-level API here because it's the less verbose, and mirrors how I'd expect DataFusion to actually use it. That said, it's likely possible to do something similar with the finer-grained APIs. should I update the PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
