alamb commented on PR #8524: URL: https://github.com/apache/arrow-rs/pull/8524#issuecomment-3387309666
The only thing I want to make sure is that this doesn't impact writing performance. The [benchmark](https://github.com/apache/arrow-rs/pull/8524#issuecomment-3383177142) results above seem to suggest there might be. ``` list_primitive/parquet_2 1.00 2.3±0.01ms 908.3 MB/sec 1.14 2.7±0.04ms 794.4 MB/sec ``` However, I tried to reproduce this locally and it looks fine to me. ```shell cargo bench --bench arrow_writer -- "list_primitive/parquet_2" ``` ```shell list_primitive/parquet_2 time: [994.65 µs 1.0011 ms 1.0083 ms] thrpt: [2.0649 GiB/s 2.0797 GiB/s 2.0931 GiB/s] change: time: [+0.0649% +2.0194% +3.9738%] (p = 0.04 < 0.05) thrpt: [-3.8219% -1.9795% -0.0649%] Change within noise threshold. Found 4 outliers among 100 measurements (4.00%) 3 (3.00%) high mild 1 (1.00%) high severe ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
