[GitHub] [kafka-site] ableegoldman opened a new pull request, #434: Update my entry to include PMC membership

[GitBox]

ableegoldman opened a new pull request, #434:
URL: https://github.com/apache/kafka-site/pull/434

   cc @mjsax @guozhangwang 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@kafka.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #1126

[Apache Jenkins Server]

See 

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #1125

[Apache Jenkins Server]

See 

[jira] [Resolved] (KAFKA-14104) Perform CRC validation on KRaft Batch Records and Snapshots

[Jason Gustafson (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-14104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Gustafson resolved KAFKA-14104.
-
Resolution: Fixed

> Perform CRC validation on KRaft Batch Records and Snapshots
> ---
>
> Key: KAFKA-14104
> URL: https://issues.apache.org/jira/browse/KAFKA-14104
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 3.2.0
>Reporter: Niket Goel
>Assignee: Niket Goel
>Priority: Blocker
> Fix For: 3.3.0
>
>
> Today we stamp the BatchRecord header with a CRC [1] and verify that CRC 
> before the log is written to disk [2]. The CRC checks should also be verified 
> when the records are read back from disk. The same procedure should be 
> followed for KRaft snapshots as well.
> [1] 
> [https://github.com/apache/kafka/blob/6b76c01cf895db0651e2cdcc07c2c392f00a8ceb/clients/src/main/java/org/apache/kafka/common/record/DefaultRecordBatch.java#L501=]
>  
> [2] 
> [https://github.com/apache/kafka/blob/679e9e0cee67e7d3d2ece204a421ea7da31d73e9/core/src/main/scala/kafka/log/UnifiedLog.scala#L1143]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Hosting Kafka Videos on ASF YouTube channel

[Joe Brockmeier]

Repurpose away. Thanks!

On Mon, Aug 8, 2022 at 4:55 PM Bill Bejeck  wrote:
>
> Hi Joe,
>
> Thanks that works for me. As for you watching the videos, they are about 10 
> minutes each, and you can watch them at 1.5 - 1.75 playback speed.
>
> If it's ok with you, I'm going to repurpose this thread as a voting thread 
> for the videos.
>
> I watched the Kafka Streams videos on 
> https://kafka.apache.org/32/documentation/streams/, and I can confirm they 
> are vendor-neutral.
> The other videos and logo that show up at the end are coming from YouTube, so 
> once move the videos to the ASF channel, that should go away.
>
> +1(binding).
>
> Thanks,
> Bill
>
>
>
> On Mon, Aug 8, 2022 at 9:46 AM Joe Brockmeier  wrote:
>>
>> If we can get a +1 from the PMC on each video that they're happy that
>> the videos are vendor neutral I think we can do that. I'll also need
>> to view them as well. I hope they're not long videos. :-)
>>
>> On Tue, Aug 2, 2022 at 3:38 PM Bill Bejeck  wrote:
>> >
>> > Hi Joe,
>> >
>> > Yes, that is correct.  Sorry, I should have mentioned that in the original 
>> > email.  That is the only video where Tim says that.
>> > The Kafka Streams videos do not mention Confluent.
>> >
>> > We're currently pursuing editing the video to remove the "from Confluent" 
>> > part.
>> > Note that the site also uses the same video on the "quickstart" page, so 
>> > both places will be fixed when editing is completed.
>> >
>> > Can we pursue hosting the Kafka Streams videos for now, then revisit the 
>> > "What is Apache Kafka?" when the editing is done?
>> >
>> > Thanks,
>> > Bill
>> >
>> >
>> > On Tue, Aug 2, 2022 at 3:12 PM Joe Brockmeier  wrote:
>> >>
>> >> Hi Bill,
>> >>
>> >> I'm not sure changing hosting would quite solve the problem. The first
>> >> video I see on this page:
>> >>
>> >> https://kafka.apache.org/intro
>> >>
>> >> Starts with "Hi, I'm Bill Berglund from *Confluent*" rather than "Hi,
>> >> I'm Bill from Apache Kafka" -- so moving to the ASF Youtube channel
>> >> wouldn't completely solve the problem.
>> >>
>> >> On Tue, Aug 2, 2022 at 3:05 PM Bill Bejeck  wrote:
>> >> >
>> >> > Hi,
>> >> >
>> >> > I am an Apache Kafka® committer and PMC member, and I'm working on our 
>> >> > site to address some issues around our embedded videos and branding.
>> >> >
>> >> > The Kafka site has six embedded videos:  
>> >> > https://kafka.apache.org/intro, https://kafka.apache.org/quickstart, 
>> >> > and four videos on https://kafka.apache.org/32/documentation/streams/.
>> >> >
>> >> > The videos are hosted on the Confluent YouTube channel, so the branding 
>> >> > on the video is from Confluent.  Since it's coming from YouTube, 
>> >> > there's no way to change it.
>> >> >
>> >> > Would it be possible to upload these videos to the Apache Foundation 
>> >> > YouTube channel 
>> >> > (https://www.youtube.com/c/TheApacheFoundation/featured)?  Doing this 
>> >> > would automatically change the branding to Apache.
>> >> >
>> >> > Thanks, and I look forward to working with you on this matter.
>> >> >
>> >> > Bill Bejeck
>> >>
>> >>
>> >>
>> >> --
>> >> Joe Brockmeier
>> >> Vice President Marketing & Publicity
>> >> j...@apache.org
>>
>>
>>
>> --
>> Joe Brockmeier
>> Vice President Marketing & Publicity
>> j...@apache.org



-- 
Joe Brockmeier
Vice President Marketing & Publicity
j...@apache.org

Re: Hosting Kafka Videos on ASF YouTube channel

[Bill Bejeck]

Hi Joe,

Thanks that works for me. As for you watching the videos, they are about 10
minutes each, and you can watch them at 1.5 - 1.75 playback speed.

If it's ok with you, I'm going to repurpose this thread as a voting thread
for the videos.

I watched the Kafka Streams videos on
https://kafka.apache.org/32/documentation/streams/, and I can confirm they
are vendor-neutral.
The other videos and logo that show up at the end are coming from YouTube,
so once move the videos to the ASF channel, that should go away.

+1(binding).

Thanks,
Bill



On Mon, Aug 8, 2022 at 9:46 AM Joe Brockmeier  wrote:

> If we can get a +1 from the PMC on each video that they're happy that
> the videos are vendor neutral I think we can do that. I'll also need
> to view them as well. I hope they're not long videos. :-)
>
> On Tue, Aug 2, 2022 at 3:38 PM Bill Bejeck  wrote:
> >
> > Hi Joe,
> >
> > Yes, that is correct.  Sorry, I should have mentioned that in the
> original email.  That is the only video where Tim says that.
> > The Kafka Streams videos do not mention Confluent.
> >
> > We're currently pursuing editing the video to remove the "from
> Confluent" part.
> > Note that the site also uses the same video on the "quickstart" page, so
> both places will be fixed when editing is completed.
> >
> > Can we pursue hosting the Kafka Streams videos for now, then revisit the
> "What is Apache Kafka?" when the editing is done?
> >
> > Thanks,
> > Bill
> >
> >
> > On Tue, Aug 2, 2022 at 3:12 PM Joe Brockmeier  wrote:
> >>
> >> Hi Bill,
> >>
> >> I'm not sure changing hosting would quite solve the problem. The first
> >> video I see on this page:
> >>
> >> https://kafka.apache.org/intro
> >>
> >> Starts with "Hi, I'm Bill Berglund from *Confluent*" rather than "Hi,
> >> I'm Bill from Apache Kafka" -- so moving to the ASF Youtube channel
> >> wouldn't completely solve the problem.
> >>
> >> On Tue, Aug 2, 2022 at 3:05 PM Bill Bejeck  wrote:
> >> >
> >> > Hi,
> >> >
> >> > I am an Apache Kafka® committer and PMC member, and I'm working on
> our site to address some issues around our embedded videos and branding.
> >> >
> >> > The Kafka site has six embedded videos:
> https://kafka.apache.org/intro, https://kafka.apache.org/quickstart, and
> four videos on https://kafka.apache.org/32/documentation/streams/.
> >> >
> >> > The videos are hosted on the Confluent YouTube channel, so the
> branding on the video is from Confluent.  Since it's coming from YouTube,
> there's no way to change it.
> >> >
> >> > Would it be possible to upload these videos to the Apache Foundation
> YouTube channel (https://www.youtube.com/c/TheApacheFoundation/featured)?
> Doing this would automatically change the branding to Apache.
> >> >
> >> > Thanks, and I look forward to working with you on this matter.
> >> >
> >> > Bill Bejeck
> >>
> >>
> >>
> >> --
> >> Joe Brockmeier
> >> Vice President Marketing & Publicity
> >> j...@apache.org
>
>
>
> --
> Joe Brockmeier
> Vice President Marketing & Publicity
> j...@apache.org
>

Re: [DISCISS] KIP-860: Add client-provided option to guard against unintentional replication factor change during partition reassignments

[Vikas Singh]

I personally like the UVE option. It provides options for clients to go
either way, retry or abort. If we do it in AdminClient, then users have to
live with what we have chosen.

> Note this can happen during an RF change too. e.g [1,2,3] => [4,5,6,7] (RF
> change, intermediate set is [1,2,3,4,5,6,7]), and we try to do a
> reassignment to [9,10,11], the logic will compare [4,5,6,7] to [9,10,11].
> In such a situation where one wants to cancel the RF increase and reassign
> again, one first needs to cancel the existing reassignment via the API (no
> special action required despite RF change)

Thanks for the explanation. I did realize this nuance and thus requested to
put that in KIP as it's not mentioned why the choice was made. I am fine if
you choose to not do it in the interest of brevity.

Vikas

On Sun, Aug 7, 2022 at 9:02 AM Stanislav Kozlovski
 wrote:

> Thank you for the reviews.
>
> Vikas,
> > > In the case of an already-reassigning partition being reassigned again,
> the validation compares the targetReplicaSet size of the reassignment to
> the targetReplicaSet size of the new reassignment and throws if those
> differ.
> > Can you add more detail to this, or clarify what is targetReplicaSet (for
> e.g. why not sourceReplicaSet?) and how the target replica set will be
> calculated?
> If a reassignment is ongoing, such that [1,2,3] => [4,5,6] (the replica set
> in Kafka will be [1,2,3,4,5,6] during the reassignment), and you try to
> issue a new reassignment (e.g [7,8,9], Kafka should NOT think that the RF
> of the partition is 6 just because a reassignment is ongoing. Hence, we
> compare [4,5,6]'s length to [7,8,9]
> The targetReplicaSet is a term we use in KIP-455
> <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-455%3A+Create+an+Administrative+API+for+Replica+Reassignment
> >.
> It means the desired replica set that a given reassignment is trying to
> achieve. Here we compare said set of the on-going reassignment to the new
> reassignment.
>
> Note this can happen during an RF change too. e.g [1,2,3] => [4,5,6,7] (RF
> change, intermediate set is [1,2,3,4,5,6,7]), and we try to do a
> reassignment to [9,10,11], the logic will compare [4,5,6,7] to [9,10,11].
> In such a situation where one wants to cancel the RF increase and reassign
> again, one first needs to cancel the existing reassignment via the API (no
> special action required despite RF change)
>
>
> > And what about the reassign partitions CLI? Do we want to expose the
> option there too?
> Yes, this is already present in the KIP if I'm not mistaken. We describe it
> in "Accordingly, the kafka-reassign-partitions.sh tool will be updated to
> allow supplying the new option:"
> I have edited the KIP to contain two clear paragraphs called Admin API and
> CLI now.
>
> Colin,
>
> >  it would be nice for the first paragraph to be a bit more explicit about
> this goal.
> sounds good, updated it with that suggestion.
>
> > client-side forward compatibility
> I was under the assumption that it is not recommended to upgrade clients
> before brokers, but a quick search cleared it up to me that we're pretty
> intentional about allowing that
> <
> https://www.confluent.io/blog/upgrading-apache-kafka-clients-just-got-easier/
> >
> .
> Do you happen to know if we have any policy on client-side forward
> compatibility with regard to such things -- extending "write" APIs (that
> mutate the state) with fields that conditionally limit that modification?
> It seems like a rare use case to me, hence renaming it to something like
> tryDisableReplicationFactorChange may unnecessary impair the API.
>
> Would Admin API documentation that says "this is supported only from
> brokers on version X and above. If not supported, the default behavior (no
> replication factor guards) is applied" be sufficient?
>
> Best,
> Stanislav
>
>
> On Fri, Aug 5, 2022 at 8:32 PM Colin McCabe  wrote:
>
> > Hi Stanislav,
> >
> > Thanks for the KIP. I think this is a nice solution to the problem of not
> > wanting to change the replication factor during reassignments.
> >
> > Just from a writing point of view, it would be nice for the first
> > paragraph to be a bit more explicit about this goal. Maybe lead with
> "Many
> > times, we don't want to change the replication factor of a partition
> during
> > reassignment..." As it is, we're talking about metadata races before
> we've
> > even explained what the goal is that the metadata races are thwarting. :)
> >
> > I like the RPC and command-line format changes; they are well-done and
> > well-written. One thing we do need to spell out, those, is what the
> > behavior is when the server does not support this new option. The
> simplest
> > thing to do would be for the client to throw UnsupportedVersionException
> > with an exception message indicating what the problem is. Then the caller
> > could catch this and re-try the call without the flag (or give up, as
> > appropriate?)
> >
> > The other option is to continue on but 

[VOTE] KIP-854 Separate configuration for producer ID expiry

[Justine Olshan]

Hi all,
I'd like to start a vote for KIP-854: Separate configuration for producer
ID expiry.

KIP:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-854+Separate+configuration+for+producer+ID+expiry
JIRA: https://issues.apache.org/jira/browse/KAFKA-14097
Discussion thread:
https://lists.apache.org/thread/cz9x90883on98k082qd0tskj6yjhox1t

Thanks,
Justine

Re: [DISCUSS] KIP-858: Handle JBOD broker disk failure in KRaft

[Jason Gustafson]

Hi Igor,

Thanks for the KIP. It looks like it's on a good track. I have a few
suggestions to throw into the mix:

1. (nit): Instead of "storage id," maybe we could call it "directory id"?
It seems a little clear since each log dir gets a unique id.
2. Rather than introducing a new RPC to communicate offline directories,
would it be reasonable to add it to BrokerHeartbeat? My thinking is that we
can let broker registration include the complete list of log dirs. The
heartbeat can then be used to communicate online/offline status of each log
dir.
3. I think we have an opportunity to consolidate normal partition
reassignment and log dir reassignment here. Since we are modifying the
`Assignment` to consist of both replica id and storage id, couldn't we make
a similar change to the `AlterPartitionReassignment` API? Basically that
would let us treat all reassignments as moving from one log dir to another.
The case handled by `AlterReplicaLogDir` currently would just become a
special case where the replica id happens to be the same. This might also
let us get rid of all the custom logic surrounding JBOD, such as the
additional fetcher, which has proven difficult to maintain.

Thanks,
Jason

On Tue, Aug 2, 2022 at 2:24 AM Igor Soarez  wrote:

> Hi José,
>
> Thanks for having a look at this KIP and thanks for pointing this out,
> I've had a look at KIP-856.
>
> It's good to see there's some overlap in our proposals. we're both
> proposing:
>
> - Identifying log directories with a UUID
> - Extending the storage tool to ensure each log directory has a UUID
> assigned
> - Expanding the topic partition identity to include the log directory UUID
>
> There were differences in our proposals as to how the UUID is to be
> persisted, but I've changed my proposal to match yours — I think adding
> storage.id to meta.properties makes sense.
>
> --
> Igor
>
>
> On Wed, Jul 27, 2022, at 4:42 PM, José Armando García Sancio wrote:
> > Hi Igor,
> >
> > Thanks for the KIP. Looking forward to this improvement. I'll review
> your KIP.
> >
> > I should mention that I started a discussion thread on KIP-856: KRaft
> > Disk Failure Recovery at
> > https://lists.apache.org/thread/ytv0t18cplwwwqcp77h6vry7on378jzj
> >
> > Both keep introducing similar concepts. For example both KIP introduce
> > a storage uuid that is stored in meta.properties. At first glance
> > there are some minor differences. I suggest that we review each
> > other's KIP so that we can remove these minor differences. What do you
> > think?
> >
> > Thanks!
> > --
> > -José
>

[jira] [Created] (KAFKA-14149) Broken DynamicBrokerReconfigurationTest in 3.2 branch

[Mickael Maison (Jira)]

Mickael Maison created KAFKA-14149:
--

 Summary: Broken DynamicBrokerReconfigurationTest in 3.2 branch
 Key: KAFKA-14149
 URL: https://issues.apache.org/jira/browse/KAFKA-14149
 Project: Kafka
  Issue Type: Bug
Affects Versions: 3.2.2
Reporter: Mickael Maison


The backport of [https://github.com/apache/kafka/pull/12455] does not work in 
3.2. The following tests are failing:

DynamicBrokerReconfigurationTest.testConfigDescribeUsingAdminClient(String).quorum=kraft
DynamicBrokerReconfigurationTest.testConsecutiveConfigChange(String).quorum=kraft
DynamicBrokerReconfigurationTest.testKeyStoreAlter(String).quorum=kraft
DynamicBrokerReconfigurationTest.testLogCleanerConfig(String).quorum=kraft
DynamicBrokerReconfigurationTest.testTrustStoreAlter(String).quorum=kraft
DynamicBrokerReconfigurationTest.testUpdatesUsingConfigProvider(String).quorum=kraft

Caused by :
java.util.concurrent.ExecutionException: 
org.apache.kafka.common.errors.InvalidRequestException: Invalid value 
org.apache.kafka.common.config.ConfigException: Dynamic reconfiguration of 
listeners is not yet supported when using a Raft-based metadata quorum for 
configuration Invalid dynamic configuration



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Hosting Kafka Videos on ASF YouTube channel

[Joe Brockmeier]

If we can get a +1 from the PMC on each video that they're happy that
the videos are vendor neutral I think we can do that. I'll also need
to view them as well. I hope they're not long videos. :-)

On Tue, Aug 2, 2022 at 3:38 PM Bill Bejeck  wrote:
>
> Hi Joe,
>
> Yes, that is correct.  Sorry, I should have mentioned that in the original 
> email.  That is the only video where Tim says that.
> The Kafka Streams videos do not mention Confluent.
>
> We're currently pursuing editing the video to remove the "from Confluent" 
> part.
> Note that the site also uses the same video on the "quickstart" page, so both 
> places will be fixed when editing is completed.
>
> Can we pursue hosting the Kafka Streams videos for now, then revisit the 
> "What is Apache Kafka?" when the editing is done?
>
> Thanks,
> Bill
>
>
> On Tue, Aug 2, 2022 at 3:12 PM Joe Brockmeier  wrote:
>>
>> Hi Bill,
>>
>> I'm not sure changing hosting would quite solve the problem. The first
>> video I see on this page:
>>
>> https://kafka.apache.org/intro
>>
>> Starts with "Hi, I'm Bill Berglund from *Confluent*" rather than "Hi,
>> I'm Bill from Apache Kafka" -- so moving to the ASF Youtube channel
>> wouldn't completely solve the problem.
>>
>> On Tue, Aug 2, 2022 at 3:05 PM Bill Bejeck  wrote:
>> >
>> > Hi,
>> >
>> > I am an Apache Kafka® committer and PMC member, and I'm working on our 
>> > site to address some issues around our embedded videos and branding.
>> >
>> > The Kafka site has six embedded videos:  https://kafka.apache.org/intro, 
>> > https://kafka.apache.org/quickstart, and four videos on 
>> > https://kafka.apache.org/32/documentation/streams/.
>> >
>> > The videos are hosted on the Confluent YouTube channel, so the branding on 
>> > the video is from Confluent.  Since it's coming from YouTube, there's no 
>> > way to change it.
>> >
>> > Would it be possible to upload these videos to the Apache Foundation 
>> > YouTube channel (https://www.youtube.com/c/TheApacheFoundation/featured)?  
>> > Doing this would automatically change the branding to Apache.
>> >
>> > Thanks, and I look forward to working with you on this matter.
>> >
>> > Bill Bejeck
>>
>>
>>
>> --
>> Joe Brockmeier
>> Vice President Marketing & Publicity
>> j...@apache.org



-- 
Joe Brockmeier
Vice President Marketing & Publicity
j...@apache.org

Re: [DISCUSS] KIP-848: The Next Generation of the Consumer Rebalance Protocol

[David Jacot]

Hi all,

I am back from vacation. I will go through and address your comments
in the coming days. Thanks for your feedback.

Cheers,
David

On Wed, Aug 3, 2022 at 10:05 PM Gregory Harris  wrote:
>
> Hey All!
>
> Thanks for the KIP, it's wonderful to see cooperative rebalancing making it
> down the stack!
>
> I had a few questions:
>
> 1. The 'Rejected Alternatives' section describes how member epoch should
> advance in step with the group epoch and assignment epoch values. I think
> that this is a good idea for the reasons described in the KIP. When the
> protocol is incrementally assigning partitions to a worker, what member
> epoch does each incremental assignment use? Are member epochs re-used, and
> a single member epoch can correspond to multiple different (monotonically
> larger) assignments?
>
> 2. Is the Assignor's 'Reason' field opaque to the group coordinator? If
> not, should custom client-side assignor implementations interact with the
> Reason field, and how is its common meaning agreed upon? If so, what is the
> benefit of a distinct Reason field over including such functionality in the
> opaque metadata?
>
> 3. The following is included in the KIP: "Thanks to this, the input of the
> client side assignor is entirely driven by the group coordinator. The
> consumer is no longer responsible for maintaining any state besides its
> assigned partitions." Does this mean that the client-side assignor MAY
> incorporate additional non-Metadata state (such as partition throughput,
> cpu/memory metrics, config topics, etc), or that additional non-Metadata
> state SHOULD NOT be used?
>
> 4. I see that there are separate classes
> for org.apache.kafka.server.group.consumer.PartitionAssignor
> and org.apache.kafka.clients.consumer.PartitionAssignor that seem to
> overlap significantly. Is it possible for these two implementations to be
> unified? This would serve to promote feature parity of server-side and
> client-side assignors, and would also facilitate operational flexibility in
> certain situations. For example, if a server-side assignor has some poor
> behavior and needs a patch, deploying the patched assignor to the client
> and switching one consumer group to a client-side assignor may be faster
> and less risky than patching all of the brokers. With the currently
> proposed distinct APIs, a non-trivial reimplementation would have to be
> assembled, and if the two APIs have diverged significantly, then it is
> possible that a reimplementation would not be possible.
>
> --
> Greg Harris
> gharris1...@gmail.com
> github.com/gharris1727
>
> On Wed, Aug 3, 2022 at 8:39 AM Sagar  wrote:
>
> > Hi Guozhang/David,
> >
> > I created a confluence page to discuss how Connect would need to change
> > based on the new rebalance protocol. Here's the page:
> >
> > https://cwiki.apache.org/confluence/display/KAFKA/%5BDRAFT%5DIntegrating+Kafka+Connect+With+New+Consumer+Rebalance+Protocol
> >
> > It's also pretty longish and I have tried to keep a format similar to
> > KIP-848. Let me know what you think. Also, do you think this should be
> > moved to a separate discussion thread or is this one fine?
> >
> > Thanks!
> > Sagar.
> >
> > On Tue, Jul 26, 2022 at 7:37 AM Sagar  wrote:
> >
> > > Hello Guozhang,
> > >
> > > Thank you so much for the doc on Kafka Streams. Sure, I would do the
> > > analysis and come up with such a document.
> > >
> > > Thanks!
> > > Sagar.
> > >
> > > On Tue, Jul 26, 2022 at 4:47 AM Guozhang Wang 
> > wrote:
> > >
> > >> Hello Sagar,
> > >>
> > >> It would be great if you could come back with some analysis on how to
> > >> implement the Connect side integration with the new protocol; so far
> > >> besides leveraging on the new "protocol type" we did not yet think
> > through
> > >> the Connect side implementations. For Streams here's a draft of
> > >> integration
> > >> plan:
> > >>
> > >>
> > https://docs.google.com/document/d/17PNz2sGoIvGyIzz8vLyJTJTU2rqnD_D9uHJnH9XARjU/edit#heading=h.pdgirmi57dvn
> > >> just FYI for your analysis on Connect.
> > >>
> > >> On Tue, Jul 19, 2022 at 10:48 PM Sagar 
> > wrote:
> > >>
> > >> > Hi David,
> > >> >
> > >> > Thank you for your response. The reason I thought connect can also fit
> > >> into
> > >> > this new scheme is that even today the connect uses a
> > WorkerCoordinator
> > >> > extending from AbstractCoordinator to empower rebalances of
> > >> > tasks/connectors. The WorkerCoordinator sets the protocolType() to
> > >> connect
> > >> > and uses the metadata() method by plumbing into
> > >> JoinGroupRequestProtocol.
> > >> >
> > >> > I think the changes to support connect would be similar at a high
> > level
> > >> to
> > >> > the changes in streams mainly because of the Client side assignors
> > being
> > >> > used in both. At an implementation level, we might need to make a lot
> > of
> > >> > changes to get onto this new assignment protocol like enhancing the
> > >> > JoinGroup request/response and SyncGroup and using
> > >> 

[jira] [Created] (KAFKA-14148) Outdated doc for reset-offsets options

[K8sCat (Jira)]

K8sCat created KAFKA-14148:
--

 Summary: Outdated doc for reset-offsets options
 Key: KAFKA-14148
 URL: https://issues.apache.org/jira/browse/KAFKA-14148
 Project: Kafka
  Issue Type: Bug
  Components: tools
Reporter: K8sCat
 Attachments: image-2022-08-08-19-19-34-873.png

!image-2022-08-08-19-19-34-873.png!

--by-period should be --by-duration, and --to-offset show be added



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (KAFKA-14147) Some map objects in KafkaConfigBackingStore grow in size monotonically

[Yash Mayya (Jira)]

Yash Mayya created KAFKA-14147:
--

 Summary: Some map objects in KafkaConfigBackingStore grow in size 
monotonically
 Key: KAFKA-14147
 URL: https://issues.apache.org/jira/browse/KAFKA-14147
 Project: Kafka
  Issue Type: Bug
  Components: KafkaConnect
Reporter: Yash Mayya


Similar to https://issues.apache.org/jira/browse/KAFKA-8869

 

{{deferredTaskUpdates, connectorTaskCountRecords and 
connectorTaskConfigGenerations in KafkaConfigBackingStore }}are never updated 
when a connector is deleted, thus growing monotonically.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (KAFKA-14146) KIP-840: Config file option for MessageReader/MessageFormatter in ConsoleProducer/ConsoleConsumer

[Alexandre Garnier (Jira)]

Alexandre Garnier created KAFKA-14146:
-

 Summary: KIP-840: Config file option for 
MessageReader/MessageFormatter in ConsoleProducer/ConsoleConsumer
 Key: KAFKA-14146
 URL: https://issues.apache.org/jira/browse/KAFKA-14146
 Project: Kafka
  Issue Type: Improvement
  Components: tools
Affects Versions: 3.2.1
Reporter: Alexandre Garnier
Assignee: Alexandre Garnier


Jira for [KIP-840|https://cwiki.apache.org/confluence/x/bBqhD]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)