hellobabygogo commented on issue #5871: fix push supervisor error URL: https://github.com/apache/incubator-druid/pull/5871#issuecomment-442310118 > @hellobabygogo sorry for the delayed review.. I think I still don't fully understand what the fundamental problem is. My questions are: > > 1. task2 looks suspicious to me. Why was task2 started even though task1 was running? > 2. with this PR, the overlord can reject starting the supervisor with a new spec if pendingTaskGroup is not empty. Then, what happens if stream data is continuously ingested and there are always some running and publish pending tasks? Is it possible that this solution can work without pausing stream ingestion? > > Probably #6001 can fix this as well. #6001 is to preserve the order of publishing segments, so that early task always publishes its segments earlier than later tasks. 1. task2 started when task1 is PUBLISHING status. 2. with this PR, if stream data is continuously ingested and there are always some running and publish pending tasks, we shouldn't starting the supervisor with a new spec.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
