ShiKaiWi commented on issue #1269: URL: https://github.com/apache/arrow-rs/issues/1269#issuecomment-1469696321
> Checking after each call to [write](https://docs.rs/parquet/latest/parquet/arrow/arrow_writer/struct.ArrowWriter.html#method.write) should be fine. A more sophisticated writer could track the number of written rows and only check once they exceed the max row group size, in practice this is highly unlikely to be make a tangible performance difference 👍. This way is good enough to solve the problem we encounter. > share as much between the sync and async implementations as possible After digging into the code, I find this is the hardest part to implement async writer. I guess your suggested way is the best workaround before the most codes of the sync and async implementation can be shared. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
