thisisnic commented on issue #43303: URL: https://github.com/apache/arrow/issues/43303#issuecomment-2254125102
I think the issue here is that the schema is the most reliable way of controlling the type of any variable, and it's inevitable that there will be issues when a data type has to be inferred and isn't provided explicitly. The undesirable behaviour here might be desirable behaviour for other users. I think the solution provided by @amoeba is pretty solid, and a solution for automating it might be to write a wrapper function which looks up the partitioning columns and then generates a schema from that to then pass into `open_dataset()`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
