Abdallah-Afifi commented on issue #10645: URL: https://github.com/apache/arrow-rs/issues/10645#issuecomment-5789994373
I'd like to pick this up, following on from #11004. The decoder already handles any size — `AlpHeader` carries `log_vector_size` and the 3..=15 range is validated on read. Only the writer is fixed: `VECTOR_SIZE` is a `const`, and `StreamingPage::put` uses `values.as_chunks::<VECTOR_SIZE>()`, which needs a compile-time constant. Going runtime means `chunks_exact`, which may cost a little on the default 1024 path everyone is already on. Two questions before I start: 1. Is a small regression on the default path acceptable, or would you rather keep a const-generic fast path for 1024 and use runtime chunking only for other sizes? Happy to measure either way with `parquet/benches/alp.rs`. 2. Where should the setting live — a global `WriterProperties` option, per-column like `set_column_encoding`, or both? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
