yihua opened a new issue, #902: URL: https://github.com/apache/hudi-rs/issues/902
### Feature Description `append` splits its input into per-partition and per-file chunks with `take_rows`, which gathers the rows through Arrow's `take` kernel. The chunks are contiguous runs of the input, so every append copies its whole input once more (2 GB for a 1M-row table of embeddings) on the single preparation thread before any file is encoded. Proposed: return a zero-copy slice when the requested rows are a contiguous ascending run, and keep `take` for the rest. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
