westonpace commented on PR #36499: URL: https://github.com/apache/arrow/pull/36499#issuecomment-1629484424
I've been able to reproduce errors very reliably with this patch (https://github.com/apache/arrow/commit/25163548b6f851990f188db52a69f1fa0a7a4322) The patch accurately simulates what I think our real world constraints are on ordering and multi-threading: * Each input may deliver its next batch at any time (e.g. maybe input 2 batch 0 comes first and maybe input 1 batch 0 comes first) * Within in an input we know that batches will be ordered. (e.g. input 2 batch 5 will always come after input 2 batch 4) * A single input will not call InputReceived reentrantly (e.g. a call to input 2 batch 4 InputReceived will finish before the call to input 2 batch 5 InputReceived begins) * There are no guarantees across inputs (e.g. there is no guarantee input 3 will deliver batch 4 before input 2 delivers batch 100) This commit actually generates a lot of errors in many different asof join node tests. Also, unfortunately, the patch proposed here does not fix all of these errors. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
