itxaiohanglover opened a new pull request, #16890: URL: https://github.com/apache/iceberg/pull/16890
Fixes #16838 ## Problem `ExpireSnapshotsSparkAction` fails with an opaque `NullPointerException` when `TableMetadata.metadataFileLocation()` returns `null`. This can happen when a REST catalog returns an invalid or incomplete table metadata response (e.g. using `metadata_location` instead of the expected `metadata-location` field name). Normal table reads continue to work because Iceberg can discover the current metadata through the table location, but after a successful `remove-snapshots` commit, `ExpireSnapshotsSparkAction` reloads the table and constructs a static table via `BaseSparkAction.newStaticTable()`. Since `metadata.metadataFileLocation()` is `null`, the resulting `BaseTable` is created with a `null` metadata location, which later causes an unexpected `NullPointerException` deep inside the Spark job execution — making the root cause very difficult to identify. ## What changed Added an explicit null check in `BaseSparkAction.newStaticTable(TableMetadata, FileIO)` that throws a descriptive `ValidationException` when `metadata.metadataFileLocation()` is `null`, instead of silently proceeding and failing later with an unrelated NPE. The change was applied consistently across all three Spark version modules (v3.5, v4.0, v4.1). The error message clearly explains that a null metadata file location may indicate an invalid REST catalog response or a missing `metadata-location` field, helping users quickly diagnose the root cause. ## Validation - Verified the fix compiles and the null check triggers a `ValidationException` with a descriptive message instead of an NPE. - Existing `ExpireSnapshots` tests continue to pass since the null check only activates when `metadataFileLocation` is genuinely `null` (which does not occur in normal operation with a correctly configured catalog). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
