tvalentyn opened a new pull request, #41004: URL: https://github.com/apache/beam/pull/41004
The mocked insert_rows_json call failed rows by their position in the batch (index 0 and 2), assuming they are always 'value1' and 'value5'. However, _StreamToBigQuery assigns each row a random shard key in [0, 500] and applies ReshufflePerKey, so the order in which rows reach the client is not guaranteed. When 'value1' and 'value5' land on the same shard (~0.2% of runs), the rows arrive as [value1, value5, value3], and 'value3' is failed instead of 'value5', which breaks the RETRY_NEVER variant. Identify the rows to fail by value instead of by index. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
