Gabor Somogyi created SPARK-28367:
-------------------------------------
Summary: Kafka connector infinite wait because metadata never
updated
Key: SPARK-28367
URL: https://issues.apache.org/jira/browse/SPARK-28367
Project: Spark
Issue Type: Bug
Components: Structured Streaming
Affects Versions: 3.0.0
Reporter: Gabor Somogyi
Spark uses an old and deprecated API named poll(long) which never returns and
stays in live lock if metadata is not updated (for instance when broker
disappears at consumer creation).
I've created a small standalone application to test it and the alternatives:
https://github.com/gaborgsomogyi/kafka-get-assignment
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
