jihoonson commented on pull request #9935: URL: https://github.com/apache/druid/pull/9935#issuecomment-634440833
@xiangqiao123, thank you for the contribution! I like the idea to announce segments in batch, but I think the giant lock in TaskLockbox should be handled in a different way, that is, we should make the lock more granular instead of using a giant lock. This is because even if you split the lock for streaming and batch ingestion, the same problem can happen for batch ingestion if you have two or more jobs which write to the same datasource and the overlapping interval. The locking can be more granular by splitting it per datasource and per interval. Please check the thread from https://github.com/apache/druid/pull/6336#issuecomment-450603775 for more details. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
