gaborgsomogyi commented on a change in pull request #29729:
URL: https://github.com/apache/spark/pull/29729#discussion_r488436868
##########
File path:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaOffsetReader.scala
##########
@@ -105,34 +102,16 @@ private[kafka010] class KafkaOffsetReader(
minPartitions.map(_ > numTopicPartitions).getOrElse(false)
}
- private def nextGroupId(): String = {
- groupId = driverGroupIdPrefix + "-" + nextId
- nextId += 1
- groupId
- }
-
override def toString(): String = consumerStrategy.toString
/**
* Closes the connection to Kafka, and cleans up state.
*/
def close(): Unit = {
- if (_consumer != null) uninterruptibleThreadRunner.runUninterruptibly {
stopConsumer() }
+ if (_admin != null) uninterruptibleThreadRunner.runUninterruptibly {
stopAdmin() }
Review comment:
I've created a standalone test app which simulates mass `AdminClient`
usage with interruptions:
https://github.com/gaborgsomogyi/kafka-admin-interruption
I've put this into a loop and not seen any infinite lock so we can say this
has proven Kafka the guys statement. I'm going to remove
`UninterruptibleThread` at the end of this PR in a separate commit which is
easily revertible (just in case if somebody objects it).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
