JingsongLi commented on code in PR #9827:
URL: https://github.com/apache/paimon/pull/9827#discussion_r4011468410
##########
pom.xml:
##########
@@ -110,7 +110,7 @@ under the License.
<testcontainers.version>1.19.1</testcontainers.version>
<iceberg.version>1.6.1</iceberg.version>
<hudi.version>0.15.0</hudi.version>
- <parquet.version>1.15.2</parquet.version>
+ <parquet.version>1.16.0</parquet.version>
Review Comment:
[P1] Remove the shared-cache call instead of only upgrading Parquet
Parquet 1.16.0 still calls `ReflectionUtils.newInstance(codecClass, ...)` in
[`CodecFactory.getCodec`](https://github.com/apache/parquet-java/blob/apache-parquet-1.16.0/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/CodecFactory.java#L305).
Its constructor does not switch to `DirectCodecFactory`; that is a separate
factory method. Paimon's vendored `ParquetWriter` also still constructs `new
CodecFactory(...)`. A probe against 1.16.0 leaves `ZstandardCodec` in
`ReflectionUtils.CONSTRUCTOR_CACHE` after `release()`, so a shared Hadoop
loader continues to retain the job-loaded codec class in the reported
session-cluster scenario. Please fix that instantiation path and verify that
the shared cache never gains the job class.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
