Revanth14 opened a new pull request, #1492: URL: https://github.com/apache/iceberg-go/pull/1492
## Summary Implements REST scan-task payload decoding as part of [#1178](https://github.com/apache/iceberg-go/issues/1178). This converts REST `ScanTasks` responses into `table.FileScanTask` values, including data files, delete files, metrics, partitions, bounds, and residual filters. ## Changes - Adds REST wire types for content files, count maps, bounds, data files, and delete files. These (`RESTContentFile`/`RESTDataFile`/`RESTFileScanTask`/`RESTDeleteFile`) are exported as part of the existing `ScanTasks` response surface introduced as stubs in #1324. - Adds `DecodeScanTasks` with metadata-aware partition and residual decoding. - Decodes partition values using their partition transform result types. - Decodes lower and upper bounds from Java-compatible hexadecimal binary values. - Resolves envelope-local delete-file references. - Classifies positional deletes, equality deletes, and Puffin deletion vectors. - Supports and validates Java's optional `referenced-data-file` field, while deriving the target for deletion vectors when omitted. - Validates malformed metrics, references, content variants, file sizes, record counts, split offsets, and equality IDs. - Adds `Residual` to `table.FileScanTask`. - Extends the existing `FileScanTask` codec with an optional framed residual extension: - New decoders can read legacy payloads. - Existing decoders continue reading the unchanged Avro envelope prefix. Bounds fixtures mirror Java's authoritative `TestContentFileParser`, where values are hexadecimal encodings of raw Iceberg binary values, for example: "lower-bounds":{"keys":[3,4],"values":["01000000","02000000"]} ## Scope This PR implements decoding only. It does not add plan-task fanout, `PlanFiles` orchestration, storage-credential handling, capability enablement, or the fake REST planning server. `DecodeScanTasks` is a standalone building block with no production caller in this PR; `PlanFiles` will invoke it in a follow-up. `ReadTasks` currently continues applying the Scan's original row filter and does not consume per-task residuals. This preserves correctness for scans using their originating `Scan`, but applying task-specific residuals remains follow-up work before decoded tasks can be consumed independently. Delete-file references are scoped to each `ScanTasks` response, so callers must decode each response before combining results. ## Testing go test ./... go test -race ./catalog/rest ./codec ./table go vet ./catalog/rest ./codec ./table git diff --check -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
