jihoonson commented on issue #5871: fix push supervisor error URL: https://github.com/apache/incubator-druid/pull/5871#issuecomment-442572301 @hellobabygogo thanks for checking! Couple of more questions. > 1. task2 started when task1 is PUBLISHING status. I guess you're saying task3. task2 is not supposed to start since it has the same startOffset with that of task1. Any tasks shouldn't have the same startOffsets with others except replicas. Is task2 a replica task of task1? > 2. with this PR, if stream data is continuously ingested and there are always some running and publish pending tasks, we shouldn't starting the supervisor with a new spec. If so, I don't think this is a good solution. In most stream applications, data will be ingested continuously and it's not really desired to pause or stop data ingestion just for updating the supervisor spec. It only adds more operation burdens like pausing streams or waiting for all pending tasks to be published. So, to find a better solution, I think it's important why task1 and task2 started even though they have the same startOffset. Do you have more contexts about this? It would be great if you have overlord logs and tasks logs.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
