rseetham commented on PR #18560: URL: https://github.com/apache/pinot/pull/18560#issuecomment-4528993502
> I see, let's try to root cause this then, basically apart from topic deletion, any other failure could cause the entire table stopped ingestion. This is still a valid fix. All topics in the table stopped ingesting. This process tried to restart ingestion and could not because 1 topic was deleted. I saw the RuntimeException from kafka in the logs. I don't know the initial reason the other topic stopped ingestion and I root cause and find it. But this is still a valid fix. This would've restarted the ingestion for the non-deleted topic. More context on the table. We had kafka offset reset enabled. In theory, all the backfill topics should be paused when they are done ingesting which means this restart process would work because it skips paused topics. The backfill topic was deleted. The other main topic stopped ingesting when the backfill topic was deleted. I think it was because the backfill topic was not paused. I will recreate it and see. We have over 300 tables with the same kafka offset reset setup where the backfill topic is in the config but and this issue happened to only 1 table. I will fix the kafka offset reset feature. This feature is the first time we're using the multi topic features so extensively so I'm discovering a lot of edge cases with multi topics where they don't work as expected which I will fix. This is one of them. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
