NarayanB commented on issue #12416: URL: https://github.com/apache/arrow/issues/12416#issuecomment-1065868719
@wjones127 -> the basename_template doesn't work if using legacy_dataset=False, while using the API pq.write_to_dataset. While checking the code, I see that it throws the exception. The other option is to use ds.write_dataset. I was able to get it working. Pyarrow table has the slice option which implies chunking. I would have expected pq.write_to_dataset returns the filename which would have allowed me to rename it. While using ds.write_dataset, I think the issue is gone with Int64 null partition issue. Not sure how though. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
