Re: Cannot perform operation after producer has been closed

2020-12-09 Thread Gabor Somogyi
;>>>> >>>>>>> On Mon, Nov 2, 2020 at 8:32 PM Jungtaek Lim < >>>>>>> kabhwan.opensou...@gmail.com> wrote: >>>>>>> >>>>>>>

Re: Cannot perform operation after producer has been closed

2020-12-09 Thread Eric Beabes
020 at 8:32 PM Jungtaek Lim < >>>>>> kabhwan.opensou...@gmail.com> wrote: >>>>>> >>>>>>> Which Spark version do you use? There's a known issue on Kafka >>>>>>> producer pool in Spark 2.x which was fixed in Spark 3

Re: Cannot perform operation after producer has been closed

2020-11-20 Thread Gabor Somogyi
rk 2.x which was fixed in Spark 3.0, so you'd like to >>>>>> check whether your case is bound to the known issue or not. >>>>>> >>>>>> https://issues.apache.org/jira/browse/SPARK-21869 >>>>>> >>>>>

Re: Cannot perform operation after producer has been closed

2020-11-19 Thread Eric Beabes
issue or not. >>>>> >>>>> https://issues.apache.org/jira/browse/SPARK-21869 >>>>> >>>>> >>>>> On Tue, Nov 3, 2020 at 1:53 AM Eric Beabes >>>>> wrote: >>>>> >>>>>> I

Re: Cannot perform operation after producer has been closed

2020-11-19 Thread Gabor Somogyi
nd to the known issue or not. >>>> >>>> https://issues.apache.org/jira/browse/SPARK-21869 >>>> >>>> >>>> On Tue, Nov 3, 2020 at 1:53 AM Eric Beabes >>>> wrote: >>>> >>>>> I know this is rela

Re: Cannot perform operation after producer has been closed

2020-11-18 Thread Eric Beabes
e, Nov 3, 2020 at 1:53 AM Eric Beabes >>> wrote: >>> >>>> I know this is related to Kafka but it happens during the Spark >>>> Structured Streaming job that's why I am asking on this mailing list. >>>> >>>> Ho

Re: Cannot perform operation after producer has been closed

2020-11-10 Thread Eric Beabes
;> >>> I know this is related to Kafka but it happens during the Spark >>> Structured Streaming job that's why I am asking on this mailing list. >>> >>> How would you debug this or get around this in Spark Structured >>> Streaming?

Re: Cannot perform operation after producer has been closed

2020-11-10 Thread Eric Beabes
job that's why I am asking on this mailing list. >> >> How would you debug this or get around this in Spark Structured >> Streaming? Any tips would be appreciated. Thanks. >> >> >> java.lang.Ill

Re: Cannot perform operation after producer has been closed

2020-11-02 Thread Jungtaek Lim
ng.IllegalStateException: Cannot perform operation after producer > has been closed at > org.apache.kafka.clients.producer.KafkaProducer.throwIfProducerClosed(KafkaProducer.java:853) > at > org.apache.kafka.clients.producer.KafkaPr

Cannot perform operation after producer has been closed

2020-11-02 Thread Eric Beabes
operation after producer has been closed at org.apache.kafka.clients.producer.KafkaProducer.throwIfProducerClosed(KafkaProducer.java:853) at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:862) at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:846