kumarpritam863 commented on issue #14818: URL: https://github.com/apache/iceberg/issues/14818#issuecomment-3644978169
@rainerschamm we cannot increase the worker poll duration as it is in the sync flow with the processing of records and will effect the throughput. Additionally we do not get this error and it may be worth checking the session timeout and heartbeat timeout in your settings. I have already raised a PR to remove the polling in the main flow altogether as we should not put the kafka polling in the main flow and block the workers from processing the records. Although this problem which you described looks like a configuration issue but the PR which I have raised address this issue also as it put the complete control topic processing on a background thread giving freedom to the workers to just poll the source topic and write to the files and the background process keeps the coordinator events ready. PR -> https://github.com/apache/iceberg/pull/14816 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
