FelixGV edited a comment on pull request #6716: URL: https://github.com/apache/incubator-pinot/pull/6716#issuecomment-808856352
Hi, Have you considered leveraging the [avro-util](https://github.com/linkedin/avro-util) library? There are two parts to this project that could be relevant to Pinot: 1. The [helper](https://github.com/linkedin/avro-util/tree/master/helper) module provides a version-agnostic shim for dealing with Avro. Using this shim would allow Pinot to work with any version of Avro between 1.4 and 1.10, even though the Apache Avro project introduced many breaking API changes throughout these versions. Whichever version is found on the classpath will be dealt with transparently. 2. The [avro-fastserde](https://github.com/linkedin/avro-util/tree/master/avro-fastserde) module is an alternative implementation of Avro serde which does runtime code generation to create an optimized serde for each schema reader/writer pair. It is more CPU-efficient and in some cases more GC-efficient as well. We've noticed 2-10x serde performance increase with it, depending on schemas. This module itself leverages the aforementioned helper module in order to achieve compatibility with Avro 1.4-1.10. Let me know if you have any questions. -F -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
