Re: Exactly once kafka connect query

2023-03-16 Thread NITTY BENNY
rs for > all > > of those instances. I was thinking that this may lead to the producer > > exceptions you are seeing but, after double-checking this assumption, > that > > does not appear to be the case. > > > > Would it be possible to share the source code fo

Re: Exactly once kafka connect query

2023-03-15 Thread NITTY BENNY
ion, that > does not appear to be the case. > > Would it be possible to share the source code for your connector and a > reproduction scenario for what you're seeing? That may be easier than > coordinating a call. > > Cheers, > > Chris > > On Tue, Mar 14, 2023 at 6:15

Re: Exactly once kafka connect query

2023-03-14 Thread NITTY BENNY
Hi Chris, Is there any possibility to have a call with you? This is actually blocking our delivery, I actually want to sort with this. Thanks, Nitty On Mon, Mar 13, 2023 at 8:18 PM NITTY BENNY wrote: > Hi Chris, > > I really don't understand why a graceful shutdown will happ

Re: Exactly once kafka connect query

2023-03-13 Thread NITTY BENNY
I called commit and sent an empty list for the current batch to poll() and then when the next file comes in and poll sees new records, I see InvalidProducerEpochException. Please advise me. Thanks, Nitty On Mon, Mar 13, 2023 at 5:33 PM NITTY BENNY wrote: > Hi Chris, > > The d

Re: Exactly once kafka connect query

2023-03-13 Thread NITTY BENNY
o be fenced out by the Connect framework later on. Do you see > messages like "Graceful stop of task failed" in the logs for > your Connect worker? > > Cheers, > > Chris > > On Mon, Mar 13, 2023 at 10:58 AM NITTY BENNY wrote: > > > Hi Chris, > >

Re: Exactly once kafka connect query

2023-03-13 Thread NITTY BENNY
InvalidProducerEpoch Exception and the messages are copied in the previous email. I don't know if this will also be fixed by your bug fix.I am using kafka 3.3.1 version as of now. Thanks, Nitty On Mon, Mar 13, 2023 at 10:47 AM NITTY BENNY wrote: > Hi Chris, > > The below mentio

Re: Exactly once kafka connect query

2023-03-13 Thread NITTY BENNY
Hi Chris, The below mentioned issue is happening for Json connector only. Is there any difference with asn1,binary,csv and json connector? Thanks, Nitty On Mon, Mar 13, 2023 at 9:16 AM NITTY BENNY wrote: > Hi Chris, > > Sorry Chris, I am not able to reproduce the above issue. &

Re: Exactly once kafka connect query

2023-03-13 Thread NITTY BENNY
sk is requesting an abort). > > Regardless, I'll work on a fix for the bug with aborting empty > transactions. Thanks for helping uncover that one! > > Cheers, > > Chris > > On Thu, Mar 9, 2023 at 6:36 PM NITTY BENNY wrote: > > > Hi Chris, > > > > We have a use

Re: Exactly once kafka connect query

2023-03-09 Thread NITTY BENNY
il (see > https://issues.apache.org/jira/browse/KAFKA-14799 for more details). > > Cheers, > > Chris > > > On Wed, Mar 8, 2023 at 6:44 PM NITTY BENNY wrote: > > > Hi Chris, > > > > I am not sure if you are able to see the images I shared

Re: Exactly once kafka connect query

2023-03-08 Thread NITTY BENNY
nContext.abortTransaction(records.get(records.size ()-1)); } else { log.info("without record"); transactionContext.abortTransaction(); } Thanks, Nitty On Wed, Mar 8, 2023 at 11:38 PM NITTY BENNY wrote: > Hi Chris, > Sorry for

Re: Exactly once kafka connect query

2023-03-08 Thread NITTY BENNY
is a validation error that should be > handled by reconfiguring the connector, then the task should throw an > exception instead of aborting the transaction. > > If possible, do you think you could provide a brief code snippet > illustrating what your task is doing that's causing issues? > &

Fwd: Exactly once kafka connect query

2023-03-07 Thread NITTY BENNY
your question about committing or aborting in certain > circumstances, it'd be useful to know more about your use case, since it > may not be necessary to define custom transaction boundaries in your > connector at all. > > Cheers, > > Chris > > > >

Re: Exactly once kafka connect query

2023-03-07 Thread NITTY BENNY
your question about committing or aborting in certain > circumstances, it'd be useful to know more about your use case, since it > may not be necessary to define custom transaction boundaries in your > connector at all. > > Cheers, > > Chris > > > >

Fwd: Exactly once kafka connect query

2023-03-07 Thread NITTY BENNY
n.boundary: "connector" Could you please help me here? Thanks, Nitty On Tue, Mar 7, 2023 at 12:29 AM NITTY BENNY wrote: > Hi Team, > I am trying to implement exactly once behavior in our source connector. Is > there any sample source connector implementation available to have a look > at? > Regards, > Nitty >

Exactly once kafka connect query

2023-03-06 Thread NITTY BENNY
Hi Team, I am trying to implement exactly once behavior in our source connector. Is there any sample source connector implementation available to have a look at? Regards, Nitty