SEPURI-SAI-KRISHNA commented on issue #12968: URL: https://github.com/apache/seatunnel/issues/12968#issuecomment-6386030962
Thanks @SEZ9. The PR is already open as #12969 and it covers exactly what you asked for. `ArrowToSeatunnelRowReaderNullCellTest` in `connector-common` has the three cases: | test | asserts | | --- | --- | | `testNullStructCellReadsAsNull` | a null ROW cell reads as `null` instead of throwing | | `testNullMapCellReadsAsNull` | a null MAP cell reads as `null` instead of an empty map | | `testEmptyMapCellStaysAnEmptyMap` | a genuinely empty map still reads as an empty map | On the populated `MapVector`: yes, the fixture builds one. It writes a real entry through `UnionMapWriter` on an actual `MapVector`, `key().integer().writeInt(1)` and `value().integer().writeInt(10)` on row 0, then `setNull(1)` on row 1 and an empty `startMap()`/`endMap()` pair on row 2. The three rows are serialized with `ArrowStreamWriter` and read back through the production byte array constructor of `ArrowToSeatunnelRowReader`, so the assertions run against the real dispatch rather than against a converter in isolation. That does close the gap you referred to. `ArrowToSeatunnelRowReaderTest` still carries `// todo check map` with the note that the "java api has problems building MapVectors" and that there are "no examples on the official website", pointing at arrow issue 44664. A populated MAP column is now actually written and read in the new class. I left that older test alone rather than rewriting it, so the PR stays on one problem, but the coverage it was missing exists now. The PR also carries the EN and ZH `incompatible-changes.md` entries for the MAP behaviour change, since a null MAP cell now reads as `null` where it used to read as an empty map. @DanielLeens has reviewed it and his conclusion is ready to merge, with one non blocking follow up about a null child field inside a non null struct. I measured that case against Arrow 15.0.1 and it does not reproduce, the details are on the PR. The branch is synced with the latest `dev` and the PR reads mergeable. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
