dilverse commented on PR #18290:
URL: https://github.com/apache/iceberg/pull/18290#issuecomment-5917288425
Thanks @uros-b, I had missed #18030. I ran it on this PR's harness (default
3s `group.initial.rebalance.delay.ms`, a second Connect worker with the default
`offset.flush.interval.ms`, `TestIntegrationIdleTask`): the unit suite and all
15 integration tests pass. I agree `assign()` is the better fix: the worker's
group has one member and is never reused, so dropping membership removes the
failure class instead of waiting out the join. I'm happy to close this in
favour of #18030 and to contribute the harness change and the idle-task test
there or as a follow-up, since the 0s delay is what hid the bug.
One observation from that run: with #18030 the idle task's first commit took
about 125s. Three rounds timed out ("committed to 0 table(s)") before a
`DataComplete` landed. After assignment, the control consumer is polled only
from `put()`, about once per `offset.flush.interval.ms` on an idle task, so a
`StartCommit` is often answered after the 30s commit timeout. Both PRs share
that path, so it looks like a separate issue.
cc @bryanck @danielcweeks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
