yittg commented on PR #4541: URL: https://github.com/apache/iceberg/pull/4541#issuecomment-1096431433
@kbendick After diving deeper in Flink, for batch mode jobs, the records emitted to `keyBy` node will be sorted. In iceberg, records written to table with identifier fields will always be distributed with `keyBy` identifier fields. And I believe the sorter is unstable, so the records with same key can be swapped. So here all records will be sorted before being emitted into IcebergStreamWriter. And records with same key can be out of order. So dynamic generated case can break these cases, however, static records can always be sorted in the same way i think. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
