fmorillo7694 commented on PR #206: URL: https://github.com/apache/flink-connector-aws/pull/206#issuecomment-5908804066
Follow-up on the last round, applying the same three lenses your comments came from (composition, swallowed conditions, read/write asymmetry) to the rest of the catalog. Three commits on `620fdf5`: **Read/write asymmetry (c66b2e6).** `getTable` surfaced Glue storage metadata as table options (`owner`, `table.input.format`, `table.output.format`, storage descriptor parameters) although nothing on the write path consumes them. Two consequences: a `getTable -> createTable` copy wrote them as stray Glue parameters, and a table whose owner or formats were set out of band (console, crawler) handed unknown options to the connector factory, which rejects them at query time. Options are now exactly what the write path produced minus the catalog's bookkeeping. Same lens on partitions: `listPartitions` truncated a Glue partition whose value count did not match the table's keys into a spec naming a partition that does not exist; it now fails with both counts. New `GlueCatalogCopySymmetryTest`: `getX(a) -> createX(b, result) -> getX(b)` reads back identically for tables, views, databases and functions; `alterTable` with a table's own read result is a no-op; a query plans over a table who se Glue metadata was changed out of band. The fake Glue client gained `UpdateTable`, so `alterTable` now runs in fake mode as well (it was moto-only before). **Swallowed conditions (204c133).** `restoreSchema` silently skipped a column recorded in the declared order but dropped from Glue by another engine; that is still the right behaviour but is now logged. Tests for two more foreign table shapes: partition column duplicated in the storage descriptor, and no storage descriptor at all. **Composition (620fdf5).** Whitespace inside parameterized types (`decimal(10, 2)`, as Athena prints nested types) is accepted. New test composes every lossless leaf type into every container position to depth 3 and asserts Flink -> Glue -> Flink is the identity (about 690 cases), plus foreign spellings with whitespace and mixed case, and empty containers. Gates: 956 unit + 10 moto green, checkstyle and spotless clean. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
