;>>>>
>>>>>>> On Mon, Nov 2, 2020 at 8:32 PM Jungtaek Lim <
>>>>>>> kabhwan.opensou...@gmail.com> wrote:
>>>>>>>
>>>>>>>
020 at 8:32 PM Jungtaek Lim <
>>>>>> kabhwan.opensou...@gmail.com> wrote:
>>>>>>
>>>>>>> Which Spark version do you use? There's a known issue on Kafka
>>>>>>> producer pool in Spark 2.x which was fixed in Spark 3
rk 2.x which was fixed in Spark 3.0, so you'd like to
>>>>>> check whether your case is bound to the known issue or not.
>>>>>>
>>>>>> https://issues.apache.org/jira/browse/SPARK-21869
>>>>>>
>>>>>
issue or not.
>>>>>
>>>>> https://issues.apache.org/jira/browse/SPARK-21869
>>>>>
>>>>>
>>>>> On Tue, Nov 3, 2020 at 1:53 AM Eric Beabes
>>>>> wrote:
>>>>>
>>>>>> I
nd to the known issue or not.
>>>>
>>>> https://issues.apache.org/jira/browse/SPARK-21869
>>>>
>>>>
>>>> On Tue, Nov 3, 2020 at 1:53 AM Eric Beabes
>>>> wrote:
>>>>
>>>>> I know this is rela
e, Nov 3, 2020 at 1:53 AM Eric Beabes
>>> wrote:
>>>
>>>> I know this is related to Kafka but it happens during the Spark
>>>> Structured Streaming job that's why I am asking on this mailing list.
>>>>
>>>> Ho
;>
>>> I know this is related to Kafka but it happens during the Spark
>>> Structured Streaming job that's why I am asking on this mailing list.
>>>
>>> How would you debug this or get around this in Spark Structured
>>> Streaming?
job that's why I am asking on this mailing list.
>>
>> How would you debug this or get around this in Spark Structured
>> Streaming? Any tips would be appreciated. Thanks.
>>
>>
>> java.lang.Ill
ng.IllegalStateException: Cannot perform operation after producer
> has been closed at
> org.apache.kafka.clients.producer.KafkaProducer.throwIfProducerClosed(KafkaProducer.java:853)
> at
> org.apache.kafka.clients.producer.KafkaPr
operation after producer
has been closed at
org.apache.kafka.clients.producer.KafkaProducer.throwIfProducerClosed(KafkaProducer.java:853)
at
org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:862)
at
org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:846
10 matches
Mail list logo