Gabor Somogyi created SPARK-28367: ------------------------------------- Summary: Kafka connector infinite wait because metadata never updated Key: SPARK-28367 URL: https://issues.apache.org/jira/browse/SPARK-28367 Project: Spark Issue Type: Bug Components: Structured Streaming Affects Versions: 3.0.0 Reporter: Gabor Somogyi
Spark uses an old and deprecated API named poll(long) which never returns and stays in live lock if metadata is not updated (for instance when broker disappears at consumer creation). I've created a small standalone application to test it and the alternatives: https://github.com/gaborgsomogyi/kafka-get-assignment -- This message was sent by Atlassian JIRA (v7.6.14#76016) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org