chenboat edited a comment on issue #4626: Low level realtime consumer (LLC) got into ERROR state due to thread race condition. URL: https://github.com/apache/incubator-pinot/issues/4626#issuecomment-533645973 Here are the interesting events: 2019-09-13 00:21:26 Thread T2 tries to get the semaphore to build the segment. 2019-09-13 00:21:27 Thread T1 got a Helix CONSUMING-> ONLINE transition message and executed the state transition. As part of the transition, T1 asks T2 to stop but T2 ignores the request because it is acquiring semaphore. 2019-09-13 00:27:34 Thread T2 acquired the semaphore and tried to build the segments (in fact T2 should have checked the _shouldStop flag before building the segments and just stop here). 2019-09-13 00:31:27 Thread T1 warned that T2 did not stop within 10 mins. 2019-09-13 00:31:27 Thread T1 downloaded the segments. 2019-09-13 00:31:43 Thread T2 finished building the segments. Now both T1 and T2 will try to write the segment to its final destination, it results in java.lang.RuntimeException: org.apache.commons.io.FileExistsException -- and in this case its T1 tried to overwrite the data written by T2.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
