Hi Satish, I think this qualifies as a blocker. This API has been around for years now and, while we don't document it as not exposing duplicates*, it has come with that implicit contract since its inception. More importantly, it has also never exposed plugins that cannot be used on the worker. This change in behavior not only introduces duplicates*, it causes unreachable plugins to be displayed. With this in mind, it seems to qualify pretty clearly as a regression and we should not put out a release that includes it.
* - Really, these aren't duplicates; rather, they're multiple copies of the same plugin that come from different locations on the worker Best, Chris On Tue, Sep 19, 2023 at 4:31 AM Satish Duggana <[email protected]> wrote: > Hi Greg, > Is this API documented that it does not return duplicate entries? > > Can we also get an opinion from PMC/Committers who have KafkaConnect > expertise on whether this issue is a release blocker? > > If we agree that it is not a release blocker then we can have a > release note clarifying this behaviour and add a reference to the JIRA > that follows up on the possible solutions. > > Thanks, > Satish. > > > On Tue, 19 Sept 2023 at 03:29, Greg Harris <[email protected]> > wrote: > > > > Hey Satish, > > > > After investigating further, I believe that this is a regression, but > > mostly a cosmetic one. > > I don't think there is significant risk of breaking clients with this > > change, but it would be confusing for users, so I'd still like to get > > the fix into the next RC. > > I've opened a PR here: https://github.com/apache/kafka/pull/14398 and > > I'll work to get it merged promptly. > > > > Thanks! > > > > On Mon, Sep 18, 2023 at 11:54 AM Greg Harris <[email protected]> > wrote: > > > > > > Hi Satish, > > > > > > While validating 3.6.0-rc0, I noticed this regression as compared to > > > 3.5.1: https://issues.apache.org/jira/browse/KAFKA-15473 > > > > > > Impact: The `connector-plugins` endpoint lists duplicates which may > > > cause confusion for users, or poor behavior in clients. > > > Using the other REST API endpoints appears unaffected. > > > I'll open a PR for this later today. > > > > > > Thanks, > > > Greg > > > > > > On Thu, Sep 14, 2023 at 11:56 AM Satish Duggana > > > <[email protected]> wrote: > > > > > > > > Thanks Justine for the update. I saw in the morning that these > changes > > > > are pushed to trunk and 3.6. > > > > > > > > ~Satish. > > > > > > > > On Thu, 14 Sept 2023 at 21:54, Justine Olshan > > > > <[email protected]> wrote: > > > > > > > > > > Hi Satish, > > > > > We were able to merge > > > > > https://issues.apache.org/jira/browse/KAFKA-15459 yesterday > > > > > and pick to 3.6. > > > > > > > > > > Hopefully nothing more from me on this release. > > > > > > > > > > Thanks, > > > > > Justine > > > > > > > > > > On Wed, Sep 13, 2023 at 9:51 PM Satish Duggana < > [email protected]> > > > > > wrote: > > > > > > > > > > > Thanks Luke for the update. > > > > > > > > > > > > ~Satish. > > > > > > > > > > > > On Thu, 14 Sept 2023 at 07:29, Luke Chen <[email protected]> > wrote: > > > > > > > > > > > > > > Hi Satish, > > > > > > > > > > > > > > Since this PR: > > > > > > > https://github.com/apache/kafka/pull/14366 only changes the > doc, I've > > > > > > > backported to 3.6 branch. FYI. > > > > > > > > > > > > > > Thanks. > > > > > > > Luke > > > > > > > > > > > > > > On Thu, Sep 14, 2023 at 12:15 AM Justine Olshan > > > > > > > <[email protected]> wrote: > > > > > > > > > > > > > > > Hey Satish -- yes, you are correct. KAFKA-15459 only affects > 3.6. > > > > > > > > PR should be finalized soon. > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Justine > > > > > > > > > > > > > > > > On Wed, Sep 13, 2023 at 1:41 AM Federico Valeri < > [email protected]> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Hi Satish, this is a small documentation fix about ZK to > KRaft > > > > > > > > > migration, that we would like to backport to 3.5 and 3.6 > branches. > > > > > > Are > > > > > > > > > you ok with that? > > > > > > > > > > > > > > > > > > https://github.com/apache/kafka/pull/14366 > > > > > > > > > > > > > > > > > > On Wed, Sep 13, 2023 at 3:13 AM Satish Duggana < > > > > > > [email protected] > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > Thanks David for the quick resolution. > > > > > > > > > > > > > > > > > > > > ~Satish. > > > > > > > > > > > > > > > > > > > > On Tue, 12 Sept 2023 at 22:51, David Arthur > > > > > > > > > > <[email protected]> wrote: > > > > > > > > > > > > > > > > > > > > > > Satish, > > > > > > > > > > > > > > > > > > > > > > KAFKA-15450 is merged to 3.6 (as well as trunk, 3.5, > and 3.4) > > > > > > > > > > > > > > > > > > > > > > Thanks! > > > > > > > > > > > David > > > > > > > > > > > > > > > > > > > > > > On Tue, Sep 12, 2023 at 11:44 AM Ismael Juma < > [email protected]> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > Justine, > > > > > > > > > > > > > > > > > > > > > > > > Probably best to have the conversation in the JIRA > ticket vs > > > > > > the > > > > > > > > > release > > > > > > > > > > > > thread. Generally, we want to only include low risk > bug fixes > > > > > > that > > > > > > > > > are > > > > > > > > > > > > fully compatible in patch releases. > > > > > > > > > > > > > > > > > > > > > > > > Ismael > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Sep 12, 2023 at 7:16 AM Justine Olshan > > > > > > > > > > > > <[email protected]> > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > Thanks Satish. I understand. > > > > > > > > > > > > > Just curious, is this something that could be > added to > > > > > > 3.6.1? It > > > > > > > > > would be > > > > > > > > > > > > > nice to say that hanging transactions are fully > covered in a > > > > > > 3.6 > > > > > > > > > release. > > > > > > > > > > > > > I'm not as familiar with the rules around minor > releases, but > > > > > > > > > adding it > > > > > > > > > > > > > there would give more time to ensure stability. > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > Justine > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Sep 12, 2023 at 5:49 AM Satish Duggana < > > > > > > > > > [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Justine, > > > > > > > > > > > > > > We can skip this change into 3.6 now as it is > not a > > > > > > blocker or > > > > > > > > > > > > > > regression and it involves changes to the API > > > > > > implementation. > > > > > > > > > Let us > > > > > > > > > > > > > > plan to add the gap in the release notes as you > mentioned. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > Satish. > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, 12 Sept 2023 at 04:44, Justine Olshan > > > > > > > > > > > > > > <[email protected]> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hey Satish, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > We just discovered a gap in KIP-890 part 1. We > currently > > > > > > > > don't > > > > > > > > > verify > > > > > > > > > > > > > on > > > > > > > > > > > > > > > txn offset commits, so it is still possible to > have > > > > > > hanging > > > > > > > > > > > > > transactions > > > > > > > > > > > > > > on > > > > > > > > > > > > > > > the consumer offsets partitions. > > > > > > > > > > > > > > > I've opened a jira to wire the verification in > that > > > > > > request. > > > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/KAFKA-15449 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This also isn't a regression, but it would be > nice to > > > > > > have > > > > > > > > > part 1 > > > > > > > > > > > > fully > > > > > > > > > > > > > > > complete. I have opened a PR with the fix: > > > > > > > > > > > > > > > https://github.com/apache/kafka/pull/14370. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I understand if there are concerns about last > minute > > > > > > changes > > > > > > > > > to this > > > > > > > > > > > > > API > > > > > > > > > > > > > > > and we can hold off if that makes the most > sense. > > > > > > > > > > > > > > > If we take that route, I think we should still > keep > > > > > > > > > verification for > > > > > > > > > > > > > the > > > > > > > > > > > > > > > data partitions since it still provides full > protection > > > > > > there > > > > > > > > > and > > > > > > > > > > > > > > improves > > > > > > > > > > > > > > > the transactions experience. We will need to > call out > > > > > > the gap > > > > > > > > > in the > > > > > > > > > > > > > > > release notes for consumer offsets partitions > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Let me know what you think. > > > > > > > > > > > > > > > Justine > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Sep 11, 2023 at 12:29 PM David Arthur > > > > > > > > > > > > > > > <[email protected]> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Another (small) ZK migration issue was > identified. > > > > > > This one > > > > > > > > > isn't a > > > > > > > > > > > > > > > > regression (it has existed since 3.4), but I > think it's > > > > > > > > > reasonable > > > > > > > > > > > > to > > > > > > > > > > > > > > > > include. It's a small configuration check > that could > > > > > > > > > potentially > > > > > > > > > > > > save > > > > > > > > > > > > > > end > > > > > > > > > > > > > > > > users from some headaches down the line. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/KAFKA-15450 > > > > > > > > > > > > > > > > https://github.com/apache/kafka/pull/14367 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I think we can get this one committed to > trunk today. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -David > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sun, Sep 10, 2023 at 7:50 PM Ismael Juma < > > > > > > > > > [email protected]> > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Satish, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > That sounds great. I think we should aim > to only > > > > > > allow > > > > > > > > > blockers > > > > > > > > > > > > > > > > > (regressions, impactful security issues, > etc.) on > > > > > > the 3.6 > > > > > > > > > branch > > > > > > > > > > > > > > until > > > > > > > > > > > > > > > > > 3.6.0 is out. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Ismael > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sat, Sep 9, 2023, 12:20 AM Satish > Duggana < > > > > > > > > > > > > > > [email protected]> > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Ismael, > > > > > > > > > > > > > > > > > > It looks like we will publish RC0 by > 14th Sep. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > Satish. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, 8 Sept 2023 at 19:23, Ismael > Juma < > > > > > > > > > [email protected]> > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Satish, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Do you have a sense of when we'll > publish RC0? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > Ismael > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Sep 8, 2023 at 6:27 AM David > Arthur > > > > > > > > > > > > > > > > > > > <[email protected]> > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Quick update on my two blockers: > KAFKA-15435 is > > > > > > > > > merged to > > > > > > > > > > > > > > trunk and > > > > > > > > > > > > > > > > > > > > cherry-picked to 3.6. I have a PR > open for > > > > > > > > > KAFKA-15441 and > > > > > > > > > > > > > will > > > > > > > > > > > > > > > > > > hopefully > > > > > > > > > > > > > > > > > > > > get it merged today. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -David > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Sep 8, 2023 at 5:26 AM Ivan > Yurchenko < > > > > > > > > > > > > > [email protected]> > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Satish and all, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I wonder if > > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/KAFKA-14993 > > > > > > > > > > > > > > > > > > should be > > > > > > > > > > > > > > > > > > > > > included in the 3.6 release plan. > I'm > > > > > > thinking > > > > > > > > > that when > > > > > > > > > > > > > > > > > > implemented, it > > > > > > > > > > > > > > > > > > > > > would be a small, but still a > change in the > > > > > > RSM > > > > > > > > > contract: > > > > > > > > > > > > > > throw > > > > > > > > > > > > > > > > an > > > > > > > > > > > > > > > > > > > > > exception instead of returning an > empty > > > > > > > > > InputStream. > > > > > > > > > > > > Maybe > > > > > > > > > > > > > it > > > > > > > > > > > > > > > > > should > > > > > > > > > > > > > > > > > > be > > > > > > > > > > > > > > > > > > > > > included right away to save the > migration > > > > > > later? > > > > > > > > > What do > > > > > > > > > > > > > you > > > > > > > > > > > > > > > > think? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > > > > > > > > > > > > > > > Ivan > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Sep 8, 2023, at 02:52, > Satish Duggana > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > Hi Jose, > > > > > > > > > > > > > > > > > > > > > > Thanks for looking into this > issue and > > > > > > > > resolving > > > > > > > > > it > > > > > > > > > > > > with > > > > > > > > > > > > > a > > > > > > > > > > > > > > > > quick > > > > > > > > > > > > > > > > > > fix. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ~Satish. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, 7 Sept 2023 at 21:40, > José Armando > > > > > > > > García > > > > > > > > > > > > Sancio > > > > > > > > > > > > > > > > > > > > > > <[email protected]> > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Satish, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Sep 6, 2023 at 4:58 PM > Satish > > > > > > > > Duggana < > > > > > > > > > > > > > > > > > > > > > [email protected]> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Greg, > > > > > > > > > > > > > > > > > > > > > > > > It seems > > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/KAFKA-14273 > > > > > > > > > > > > > > > > > has > > > > > > > > > > > > > > > > > > > > been > > > > > > > > > > > > > > > > > > > > > > > > there in 3.5.x too. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I also agree that it should be > a blocker > > > > > > for > > > > > > > > > 3.6.0. > > > > > > > > > > > > It > > > > > > > > > > > > > > should > > > > > > > > > > > > > > > > > > have > > > > > > > > > > > > > > > > > > > > > > > been a blocker for those > previous > > > > > > releases. I > > > > > > > > > didn't > > > > > > > > > > > > > fix > > > > > > > > > > > > > > it > > > > > > > > > > > > > > > > > > because, > > > > > > > > > > > > > > > > > > > > > > > unfortunately, I wasn't aware > of the > > > > > > issue > > > > > > > > and > > > > > > > > > jira. > > > > > > > > > > > > > > > > > > > > > > > I'll create a PR with a fix in > case the > > > > > > > > > original > > > > > > > > > > > > author > > > > > > > > > > > > > > > > doesn't > > > > > > > > > > > > > > > > > > > > > respond in time. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Satish, do you agree? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks! > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > > > > -José > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > -David > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > -David > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > -David > > > > > > > > > > > > > > > > > > > > > > > >
