PenneyHuang opened a new issue, #16856: URL: https://github.com/apache/iceberg/issues/16856
### Apache Iceberg version
1.11.0 (latest release)
### Query engine
Spark
### Please describe the bug 🐞
Using the official iceberg-spark-runtime-4.1_2.13-1.11.0.jar in a Spark-only
runtime fails when Flink table classes are not present.
FormatModelRegistry eagerly tries to register
org.apache.iceberg.flink.data.FlinkFormatModels during static initialization.
That code references Flink
classes such as org.apache.flink.table.data.RowData.
If those Flink classes are absent, Spark reads fail with:
─ text
java.lang.NoClassDefFoundError: Could not initialize class
org.apache.iceberg.formats.FormatModelRegistry
...
Caused by: java.lang.NoClassDefFoundError:
org/apache/flink/table/data/RowData
at
org.apache.iceberg.flink.data.FlinkFormatModels.register(FlinkFormatModels.java:30)
at
org.apache.iceberg.formats.FormatModelRegistry.registerSupportedFormats(FormatModelRegistry.java:207)
at
org.apache.iceberg.formats.FormatModelRegistry.<clinit>(FormatModelRegistry.java:69)
This happens in the normal Spark reader path (BaseBatchReader), even
after disabling Auron/native execution.
Steps to reproduce
1. Start Spark 4.1
2. Add iceberg-spark-runtime-4.1_2.13-1.11.0.jar
3. Read any Iceberg table in Spark SQL
Expected behavior
A Spark-only runtime should be able to use the Spark runtime jar without
requiring Flink runtime classes.
Notes
This looks similar to #16720, but with missing Flink classes instead of
missing ORC classes.
It seems FormatModelRegistry should skip optional engine-specific
registrations when transitive dependencies are absent.
### Willingness to contribute
- [ ] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from
the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
