Re: [DISCUSS] KIP-980: Allow creating connectors in a stopped state

Hi Chris, I've updated the KIP to call out the parsing logic and the user expectations explicitly. Thanks again for all your feedback on this KIP! I'll wait for a few more days to see if anyone else has comments before kicking off a vote thread. Thanks, Yash On Thu, Oct 5, 2023 at 10:49 PM Chris

Re: [DISCUSS] KIP-980: Allow creating connectors in a stopped state

Hi Yash, Yeah, I think just hardcoding with JSON-first, properties-second is fine. IMO it's worth calling this out explicitly in the KIP. Apart from that, no further comments. LGTM, thanks for the KIP! Cheers, Chris On Thu, Oct 5, 2023 at 6:23 AM Yash Mayya wrote: > Hi Chris, > > Thanks for

Re: [DISCUSS] KIP-980: Allow creating connectors in a stopped state

Hi Chris, Thanks for all your feedback so far! 3. That's a good question. I was thinking we'd do some "intelligent" parsing internally during the implementation (i.e. essentially your last option - attempting to parse first as one format, then the other) which is why I didn't include any more det

Re: [DISCUSS] KIP-980: Allow creating connectors in a stopped state

Hi Yash, Looking great! Few more thoughts: 1. (Downgraded to nit) I still prefer dot-delimitation but it's not a blocker; thanks for addressing my concerns about the name of the field and how it may be perceived by users. 2. (Addressed) Thanks for looking into this, and sorry it turned out to b

Re: [DISCUSS] KIP-980: Allow creating connectors in a stopped state

Hi Chris, Thanks for the quick follow up and the continued insightful discourse! 1. Fair point on the need to differentiate it from the actual state displayed in the status API, I like the prefix of "initial" to make that differentiation (from your suggested alternatives previously). Regarding th

Re: [DISCUSS] KIP-980: Allow creating connectors in a stopped state

Hi Yash, Thanks for the in-depth discussion! Continuations here: 1. Regarding delimiters (dots vs. underscores), we use dots in connector configs for all runtime-recognized properties, including but not limited to connector.class, tasks.max, key.converter, key.converter.*, and transforms.*.type.

Re: [DISCUSS] KIP-980: Allow creating connectors in a stopped state

Hi Chris, Thanks for taking a look at this KIP! 1. I chose to go with simply "state" as that exact term is already exposed via some of the existing REST API responses and would be one that users are already familiar with (although admittedly something like "initial_state" wouldn't be much of a ju

Re: [DISCUSS] KIP-980: Allow creating connectors in a stopped state

Hi Yash, Thanks for the KIP! Frankly this feels like an oversight in 875 and I'm glad we're closing that loop ASAP. Here are my thoughts: 1. (Nit): IMO "start.state", "initial.state", or "target.state" might be better than just "state" for the field name in the connector creation request. 2. W

Re: [DISCUSS] KIP-980: Allow creating connectors in a stopped state

Hi Ashwin, Thanks for taking a look and sharing your thoughts! 1. Yes, the request / response formats of the two APIs were intentionally made identical for such use-cases. [1] 2. I'm assuming you're referring to retaining the offset / config topic records for a connector when it is deleted by a

Re: [DISCUSS] KIP-980: Allow creating connectors in a stopped state

Thanks Yash! This is very useful for migrating connectors from one cluster to another. I had the following comments/questions 1. Is the offset read using `GET /offsets` api always guaranteed to be in a format accepted by `PATCH /offsets` ? 2. I had to tackle a similar migration situation but the

[DISCUSS] KIP-980: Allow creating connectors in a stopped state

Hi all, I'd like to begin discussion on a KIP to allow creating connectors in a stopped state - https://cwiki.apache.org/confluence/display/KAFKA/KIP-980%3A+Allow+creating+connectors+in+a+stopped+state Thanks, Yash