barbuz opened a new issue, #34898: URL: https://github.com/apache/arrow/issues/34898
### Describe the enhancement requested ### Motivation I need to store a schema in a permanent way, as I'm building a process where several CSV files with the same structure will be converted to Parquet over time, but inferring the schema of the data from each file does not guarantee consistency. I could use something like pickle to dump a binary version of the schema but this is not very portable, and a human-readable representation would allow to do simple modifications by hand if the need arises (e.g. adding or removing columns). For this reason I'm trying to save the schema in a json file and then to read it and re-build a pyarrow Schema. The first step is easy thanks to each type having a nice string representation, but going backwards is harder as I could not find any way of building a type or a Field from a string that did not break on more complex types such as `timestep[ms, tz=utc]`. I ended up having to implement my own function to parse string representations of types and build the appropriate pyarrow objects. ### Idea The basic idea would be implementing a function that takes a string representation of a type and returns the corresponding pyarrow type object. Other things that could make this process easier would be a `to_dict()` function for Schema that basically builds the dictionary obtained combining the `names` and `types` lists of a Schema, or a `from_string(s)` function that can reverse the `to_string()` function. I can contribute my code to convert strings to type objects, but as I've never contributed to this project before I would like some advice on whether this is something desired and what would be the best way to integrate this functionality with the rest of the code base. ### Component(s) Python -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
