zmrdltl opened a new issue, #5410: URL: https://github.com/apache/arrow-rs/issues/5410
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.** In the process of implementing serialization for various data types to Parquet files using Rust, we encountered limitations due to the private visibility of the encoder module within the Parquet crate. Specifically, the ColumnValueEncoder trait, which is crucial for writing generic functions to handle a diverse set of data types efficiently, is not accessible outside of its module. This limitation prevents the creation of reusable, clean, and modular code, forcing us to duplicate logic for each data type or make less efficient implementations. [related](https://github.com/gluesql/gluesql/pull/1269) **Describe the solution you'd like** We propose making the encoder module public, or at least re-exporting the ColumnValueEncoder trait publicly, to allow developers to write generic functions for data serialization. This change would greatly enhance the usability and flexibility of the Parquet crate, enabling developers to write more abstract, reusable code that can work with any data type supported by Parquet, thereby reducing code duplication and improving the overall developer experience. **Describe alternatives you've considered** An alternative solution would be to provide a set of predefined functions within the Parquet crate itself for handling common serialization tasks. However, this approach lacks the flexibility and extensibility offered by allowing developers to implement their own generic functions. Another alternative would be to fork the Parquet crate and make the necessary modifications, but this is not a sustainable solution as it diverges from the main codebase and requires maintaining the fork. **Additional context** The need for this change was identified while trying to implement a comprehensive solution for writing different types of values to Parquet files. The ability to use GenericColumnWriter and ColumnValueEncoder in a generic context would significantly streamline the process of working with Parquet files in Rust, making it easier to support a wide range of data types without introducing excessive code duplication or complexity. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
