sqd opened a new pull request, #17024: URL: https://github.com/apache/iceberg/pull/17024
Currently, the dynamic sink has a bug in that if an incoming record requires both schema evolution (because of additional column) and data conversion (because of type widening), the logics will try to apply the schema evolution first (correct behavior), but when it's doing so, it is not aware of the type widening data conversion that it'll do in the next step. As a result, it tries to evolve the table column to the narrow type and crashes. Concrete example: table schema [id: long], incoming record [id: INT, value: string]. This change fixes the bug, by checking whether a data conversion can be performed in the next step, and skip schema evolution on that column. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
