[
https://issues.apache.org/jira/browse/SPARK-9780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-9780:
-----------------------------------
Assignee: Apache Spark
> In case of invalid initialization of KafkaDirectStream, NPE is thrown
> ---------------------------------------------------------------------
>
> Key: SPARK-9780
> URL: https://issues.apache.org/jira/browse/SPARK-9780
> Project: Spark
> Issue Type: Bug
> Components: Streaming
> Affects Versions: 1.3.1, 1.4.1
> Reporter: Grigory Turunov
> Assignee: Apache Spark
> Priority: Minor
>
> [o.a.s.streaming.kafka.KafkaRDD.scala#L143|https://github.com/apache/spark/blob/master/external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaRDD.scala#L143]
> In initialization of KafkaRDDIterator, there is an addition of
> TaskCompletionListener to the context, which calls close() to the consumer,
> which is not initialized yet (and will be initialized 12 lines after that).
> If something happens in this 12 lines (in my case there was a private
> constructor for valueDecoder), an Exception, which is thrown, triggers
> context.markTaskCompleted() in
> [o.a.s.scheduler.Task.scala#L90|https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/scheduler/Task.scala#L90]
> which throws NullPointerException, when tries to call close() for
> non-initialized consumer.
> This masks original exception - so it is very hard to understand, what is
> happening.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
