deepakpanda93 commented on issue #16950:
URL: https://github.com/apache/hudi/issues/16950#issuecomment-5176975543
Revisiting this one, I think it can be closed. The actionable part has
landed, and the part that has not is intentionally not actionable.
Parquet is pinned per Spark profile in the root pom.xml, and the Spark 4
profiles are already past the fixed version:
Profile | parquet.version | CVE-2025-30065 (fixed in 1.15.1)
-- | -- | --
spark3.3 | 1.12.2 | affected
spark3.4 | 1.12.3 | affected
spark3.5 | 1.13.1 | affected
spark4.0 | 1.15.2 | not affected
spark4.1 | 1.16.0 | not affected
spark4.2 | 1.17.0 | not affected
The non-Spark engine versions are also already at or past 1.15.1:
```trino.parquet.version``` 1.15.1, ```presto.parquet.version``` 1.15.2,
```flink.format.parquet.version``` 1.15.2.
That leaves the Spark 3 profiles, which cannot be bumped. As @yihua noted
while reviewing #13315, the Parquet version has to match what each Spark
version ships, otherwise there are compatibility issues. That is a runtime
constraint rather than a preference: Spark 3.5 puts its own Parquet 1.13.1
classes on the classpath, and #13315 showed the concrete failure when a newer
parquet-avro is mixed in:
```
java.lang.NoClassDefFoundError:
org.apache.parquet.conf.HadoopParquetConfiguration
at
org.apache.parquet.avro.AvroSchemaConverter.<init>(AvroSchemaConverter.java:111)
```
```HadoopParquetConfiguration``` does not exist in 1.13.1, so the Azure
build failed. So the remaining exposure is bounded by the Spark version in use,
and is resolved by moving to a Spark 4 profile rather than by a change in Hudi.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
