Re: Probable bug in async commit of Kafka offset in DirectKafkaInputDStream

2022-04-08 Thread Dongjoon Hyun
Thank you, Souvik. Dongjoon. On Thu, Apr 7, 2022 at 10:59 AM Paul, Souvik wrote: > Hi Dongjoon, > > > > Raised the JIRA at https://issues.apache.org/jira/browse/SPARK-38824 > > > > Thanks, > > Souvik > > > > *From:* Dongjoon Hyun > *Sent:* Wednesday, March 30, 2022 4:44 AM > *To:* Paul,

RE: Probable bug in async commit of Kafka offset in DirectKafkaInputDStream

2022-04-07 Thread Paul, Souvik
Hi Dongjoon, Raised the JIRA at https://issues.apache.org/jira/browse/SPARK-38824 Thanks, Souvik From: Dongjoon Hyun Sent: Wednesday, March 30, 2022 4:44 AM To: Paul, Souvik [Engineering] Cc: dev@spark.apache.org Subject: Re: Probable bug in async commit of Kafka offset in

Re: Probable bug in async commit of Kafka offset in DirectKafkaInputDStream

2022-03-29 Thread Dongjoon Hyun
Hi, Souvik Could you file a JIRA issue for that? Thanks, Dongjoon On Thu, Mar 24, 2022 at 11:08 AM Paul, Souvik wrote: > Hi Dev, > > I added a few debug statements at the following lines and found few issues. > > 1. At line 254 of override def compute(validTime: Time): > Option[KafkaRDD[K,