[GitHub] [arrow] nevi-me commented on pull request #8223: ARROW-10040: [Rust] Add slice that realigns Buffer
nevi-me commented on pull request #8223: URL: https://github.com/apache/arrow/pull/8223#issuecomment-698963093 Hey @jhorstmann, I haven't had time to look, but maybe I'm confused. What I recall is that I needed a way of converting a Buffer to a `Vec` when I write to Parquet. It might even be that I don't need to do this though, but I'll see when I get time to work on the Arrow Parquet writer some more. I'm closing this PR This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow] nevi-me commented on pull request #8223: ARROW-10040: [Rust] Add slice that realigns Buffer
nevi-me commented on pull request #8223: URL: https://github.com/apache/arrow/pull/8223#issuecomment-696669012 @jhorstmann can I close this PR, and rely on your implementation when ready? Also, do you think we'd be able to use your implementation in `parquet`, as we might need that for iterating through Boolean arrays when converting from Arrow to Parquet. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow] nevi-me commented on pull request #8223: ARROW-10040: [Rust] Add slice that realigns Buffer
nevi-me commented on pull request #8223: URL: https://github.com/apache/arrow/pull/8223#issuecomment-696669012 @jhorstmann can I close this PR, and rely on your implementation when ready? Also, do you think we'd be able to use your implementation in `parquet`, as we might need that for iterating through Boolean arrays when converting from Arrow to Parquet. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow] nevi-me commented on pull request #8223: ARROW-10040: [Rust] Add slice that realigns Buffer
nevi-me commented on pull request #8223: URL: https://github.com/apache/arrow/pull/8223#issuecomment-695347849 @jhorstmann @paddyhoran this is related to the alignment fixes made recently. I noticed while reviewing another PR that we had a limitation on boolean kernels if offsets weren't a multiple of 8. So I've implemented a slice on `Buffer` that copies the buffer and removes the offsets. I expect there to be a minor perf impact, and I only use the above slice method when necessary. It's very likely that my implementation can be improved upon, but I'm at my current limits on bit manipulation. Please see https://godbolt.org/z/bMc5qd for some of the assembly generated. Not happy with the instruction count, and would appreciate some guidance or improvements. Feel free to push any changes directly on this branch. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
