Gabor Somogyi created SPARK-28367:
-------------------------------------

             Summary: Kafka connector infinite wait because metadata never 
updated
                 Key: SPARK-28367
                 URL: https://issues.apache.org/jira/browse/SPARK-28367
             Project: Spark
          Issue Type: Bug
          Components: Structured Streaming
    Affects Versions: 3.0.0
            Reporter: Gabor Somogyi


Spark uses an old and deprecated API named poll(long) which never returns and 
stays in live lock if metadata is not updated (for instance when broker 
disappears at consumer creation).

I've created a small standalone application to test it and the alternatives: 
https://github.com/gaborgsomogyi/kafka-get-assignment




--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

Reply via email to