DanielLeens commented on issue #10886: URL: https://github.com/apache/seatunnel/issues/10886#issuecomment-4456133304
Thanks for the clear reproduction and for narrowing it down to the nested `ROW` path. I checked the current `dev` code, and this does look like a real Paimon sink bug. The problematic path is in `RowConverter.reconvert(...)`: - the top-level method first validates the incoming row against the full sink table schema; - but in the `ROW` branch it recursively calls `reconvert((SeaTunnelRow) row, (SeaTunnelRowType) rowType, sinkTableSchema)` again with the outer table schema; - that means the nested row is still compared against the outer table field list, so a nested `ROW(id, name)` can fail with "source has 2 fields, sink has 3 fields" when the outer table has 3 columns. So this is not just a config issue. Under the normal flow, nested `ROW` fields should be validated against the nested row schema, not the full outer table schema. As a temporary workaround, please avoid writing nested `ROW` columns through the current Paimon sink path, or flatten the nested fields before the sink if that is acceptable for your pipeline. This issue is worth keeping open for follow-up. A focused PR around the nested `ROW` conversion path plus a regression test would be very welcome. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
