linzch3 opened a new pull request, #19048:
URL: https://github.com/apache/iceberg/pull/19048

   Closes #19047
   
   ### Background
   When a manifest list (or any other file) is deleted from GCS but still 
referenced by metadata, reading it fails with an NPE instead of a 
`NotFoundException`:
   
   ```
   java.lang.NullPointerException: Cannot invoke 
"com.google.cloud.storage.Blob.getSize()" because the return value of 
"org.apache.iceberg.gcp.gcs.GCSInputFile.getBlob()" is null
        at 
org.apache.iceberg.gcp.gcs.GCSInputFile.getLength(GCSInputFile.java:59)
        at 
org.apache.iceberg.avro.AvroIterable.newFileReader(AvroIterable.java:102)
   ```
   
   ### Root cause
   `Storage.get(BlobId)` returns `null` for a missing object (that's also what 
`BaseGCSFile.exists()` relies on). `GCSInputFile.getLength()` called 
`getBlob().getSize()` without a null check. #15734 already maps 404s to 
`NotFoundException` in the input streams, but `getLength()` runs before any 
stream is opened (e.g. `AvroIterable.newFileReader`), so the NPE comes first.
   
   ### Fix
   If the blob is missing, `getLength()` throws `NotFoundException("Location 
does not exist: %s", location())`, the same message `GCSExceptionUtil` and 
`S3InputStream` use. Nothing else changes: a known length still skips the 
lookup.
   
   ### Tests
   - New `TestGcsInputFile#getLengthMissingBlob`: `storage.get` returns `null`, 
`exists()` is false and `getLength()` throws `NotFoundException` with the 
location.
   - `./gradlew :iceberg-gcp:test` and `:iceberg-gcp:spotlessCheck` pass 
locally (JDK 21). I didn't run the GCS integration tests 
(`gcp/src/integration`), they need the emulator.
   
   ### Limitations
   - Only the `getLength()` path is covered here. Other callers of `getBlob()` 
(`exists()`) already handle `null`.
   - #16935 tried to improve the error for a missing manifest list at the core 
level and was closed as stale; this one fixes it where the NPE happens instead.
   
   AI disclosure: I used an AI assistant (Claude Opus 5.5) to help write this 
change and the test. I reviewed the code and ran the tests myself.
   
   Happy to adjust anything, e.g. the message wording or moving the check into 
`BaseGCSFile`, lmk.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to