parthchandra commented on PR #13786:
URL: https://github.com/apache/iceberg/pull/13786#issuecomment-3475084816
@huaxingao PTAL. I've moved _all_ Comet classes out of `iceberg-parquet`.
The code now uses a Factory method that loads a vectorized parquet reader via
the Java `ServiceLoader` interface if a custom reader is configured. Comet now
provides this custom reader for each Spark version supported.
This approach also eliminates the 'bridge' classes.
The only downside is that the `CometVectorizedParquetReader` and the
`CometTypeUtils` are duplicated. Also, Flink will no longer automatically get
Comet support. (But can be configured to use the Comet reader if the jars are
in the path).
Note:
I added `VectorizedParquetReaderFactory.java` in the `iceberg-parquet`
module. This interface defines two methods: `name()` and `createReader(...)`
for pluggable vectorized reader implementations.
Concrete implementations are in the Spark modules.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
