[ https://issues.apache.org/jira/browse/SPARK-27720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16840487#comment-16840487 ]
Gabor Somogyi commented on SPARK-27720: --------------------------------------- Now I see the places where the consumer usage is not protected with this.synchronized. > ConcurrentModificationException on closing DirectKafkaInputDStream > ------------------------------------------------------------------ > > Key: SPARK-27720 > URL: https://issues.apache.org/jira/browse/SPARK-27720 > Project: Spark > Issue Type: Bug > Components: DStreams > Affects Versions: 2.4.1 > Reporter: ov7a > Priority: Minor > > If a DirectKafkaInputDStream is started in one thread and is being stopped in > another thread (e.g. by shutdown hook) a > java.util.ConcurrentModificationException (KafkaConsumer is not safe for > multi-threaded access) is thrown. > This happens even if "spark.streaming.kafka.consumer.cache.enabled" is set to > "false". > -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org