Exciting! Thanks for driving the release, David.
On Mon, Jan 24, 2022 at 9:04 AM David Jacot wrote:
>
> The Apache Kafka community is pleased to announce the release for
> Apache Kafka 3.1.0.
>
> It is a major release that includes many new features, including:
>
> * Apache Kafka supports Java
PMC.
Congratulations, David!
Gwen Shapira, on behalf of Apache Kafka PMC
nk.
>
> Best,
> David
>
--
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
ver expect the client to carry a persisted client instance id or
> > should
> > > the client be treated as a new instance?
> >
> > The KIP doesn't describe any mechanism for persistence, so I would assume
> > that when you restart the client you get a new UUID. I agree that it
> would
> > be good to spell this out.
> >
> >
> Right, it will not be persisted since a client instance can't be restarted.
>
> Will update the KIP to make this clearer.
>
> /Magnus
>
--
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
can find it
> >>> here
> >>> > >>> https://www.mail-archive.com/dev@kafka.apache.org/msg113373.html
> >>> > >>>
> >>> > >>> On Tue, Dec 8, 2020 at 4:01 PM Omnia Ibrahim <
> >>> o.g.h.ibra...@gmail.com>
> >>> > >>> wrote:
> >>> > >>>
> >>> > >>>> Hi everyone,
> >>> > >>>> I’m proposing a new KIP for MirrorMaker 2 to add the ability to
> >>> control
> >>> > >>>> internal topics naming convention. The proposal details are here
> >>> > >>>>
> >>>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-690%3A+Add+additional+configuration+to+control+MirrorMaker+2+internal+topics+naming+convention
> >>> > >>>>
> >>> > >>>> Please vote in this thread.
> >>> > >>>> Thanks
> >>> > >>>> Omnia
> >>> > >>>>
> >>> > >>>
> >>> > >
> >>>
> >>
>
--
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
I'm supportive of the feature and the interface details discussed in the
discussion thread. I just want to clarify that I'm voting for the last
version discussed in the thread - that includes two phase upgrade and no
breaking changes in 3.0.
+1 (binding)
On Wed, Jun 9, 2021, 5:32 AM Chris
+1 (binding)
Exciting to see this.
On Wed, Jun 9, 2021, 11:29 AM Ismael Juma wrote:
> Hi all,
>
> Consensus was reached in the discussion thread and part of what is proposed
> has to happen by 3.0, so starting the vote for KIP-724:
>
>
>
deck.com/dongjin>*
> > >
> >
>
>
> --
> *Dongjin Lee*
>
> *A hitchhiker in the mathematical world.*
>
>
>
> *github: <http://goog_969573159/>github.com/dongjinleekr
> <https://github.com/dongjinleekr>keybase: https://keybase.io/dongjinleekr
> <https://keybase.io/dongjinleekr>linkedin: kr.linkedin.com/in/dongjinleekr
> <https://kr.linkedin.com/in/dongjinleekr>speakerdeck:
> speakerdeck.com/dongjin
> <https://speakerdeck.com/dongjin>*
>
--
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
t;>>>>> >>> >> On Thu, Dec 10, 2020 at 12:58 PM Mickael Maison <
> >>>>>>>> mickael.mai...@gmail.com> wrote:
> >>>>>>>> >>> >>>
> >>>>>>>> >>> >>> Hi Omnia,
> >>>>>>>> >>> >>>
> >>>>>>>> >>> >>> Thank you for the reply, it makes sense.
> >>>>>>>> >>> >>>
> >>>>>>>> >>> >>> A couple more comments:
> >>>>>>>> >>> >>>
> >>>>>>>> >>> >>> 1) I'm assuming the new interface and default
> >>>>>>>> implementation will be
> >>>>>>>> >>> >>> in the mirror-client project? as the names of some of
> these
> >>>>>>>> topics are
> >>>>>>>> >>> >>> needed by RemoteClusterUtils on the client-side.
> >>>>>>>> >>> >>>
> >>>>>>>> >>> >>> 2) I'm about to open a KIP to specify where the
> >>>>>>>> offset-syncs topic is
> >>>>>>>> >>> >>> created by MM2. In restricted environments, we'd prefer
> MM2
> >>>>>>>> to only
> >>>>>>>> >>> >>> have read access to the source cluster and have the
> >>>>>>>> offset-syncs on
> >>>>>>>> >>> >>> the target cluster. I think allowing to specify the
> cluster
> >>>>>>>> where to
> >>>>>>>> >>> >>> create that topic would be a natural extension of the
> >>>>>>>> interface you
> >>>>>>>> >>> >>> propose here.
> >>>>>>>> >>> >>>
> >>>>>>>> >>> >>> So I wonder if your interface could be named
> >>>>>>>> InternalTopicsPolicy?
> >>>>>>>> >>> >>> That's a bit more generic than InternalTopicNamingPolicy.
> >>>>>>>> That would
> >>>>>>>> >>> >>> also match the configuration setting,
> >>>>>>>> internal.topics.policy.class,
> >>>>>>>> >>> >>> you're proposing.
> >>>>>>>> >>> >>>
> >>>>>>>> >>> >>> Thanks
> >>>>>>>> >>> >>>
> >>>>>>>> >>> >>> On Thu, Dec 3, 2020 at 10:15 PM Omnia Ibrahim <
> >>>>>>>> o.g.h.ibra...@gmail.com> wrote:
> >>>>>>>> >>> >>> >
> >>>>>>>> >>> >>> > Hi Mickael,
> >>>>>>>> >>> >>> > Thanks for your feedback!
> >>>>>>>> >>> >>> > Regards your question about having more configurations,
> I
> >>>>>>>> considered adding
> >>>>>>>> >>> >>> > configuration per each topic however this meant adding
> >>>>>>>> more configurations
> >>>>>>>> >>> >>> > for MM2 which already have so many, also the more
> >>>>>>>> complicated and advanced
> >>>>>>>> >>> >>> > replication pattern you have between clusters the more
> >>>>>>>> configuration lines
> >>>>>>>> >>> >>> > will be added to your MM2 config which isn't going to be
> >>>>>>>> pretty if you
> >>>>>>>> >>> >>> > don't have the same topics names across your clusters.
> >>>>>>>> >>> >>> >
> >>>>>>>> >>> >>> > Also, it added more complexity to the implementation as
> >>>>>>>> MM2 need to
> >>>>>>>> >>> >>> > 1- identify if a topic is checkpoints so we could list
> >>>>>>>> the checkpoints
> >>>>>>>> >>> >>> > topics in MirrorMaker 2 utils as one cluster could have
> X
> >>>>>>>> numbers
> >>>>>>>> >>> >>> > checkpoints topics if it's connected to X clusters, this
> >>>>>>>> is done right now
> >>>>>>>> >>> >>> > by listing any topic with suffix
> `.checkpoints.internal`.
> >>>>>>>> This could be
> >>>>>>>> >>> >>> > done by add `checkpoints.topic.suffix` config but this
> >>>>>>>> would make an
> >>>>>>>> >>> >>> > assumption that checkpoints will always have a suffix
> >>>>>>>> also having a suffix
> >>>>>>>> >>> >>> > means that we may need a separator as well to
> concatenate
> >>>>>>>> this suffix with
> >>>>>>>> >>> >>> > a prefix to identify source cluster name.
> >>>>>>>> >>> >>> > 2- identify if a topic is internal, so it shouldn't be
> >>>>>>>> replicated or track
> >>>>>>>> >>> >>> > checkpoints for it, right now this is relaying on
> >>>>>>>> disallow topics with
> >>>>>>>> >>> >>> > `.internal` suffix to be not replicated and not tracked
> >>>>>>>> in checkpoints but
> >>>>>>>> >>> >>> > with making topics configurable we need a way to define
> >>>>>>>> what is an internal
> >>>>>>>> >>> >>> > topic. This could be done by making using a list of all
> >>>>>>>> internal topics
> >>>>>>>> >>> >>> > have been entered to the configuration.
> >>>>>>>> >>> >>> >
> >>>>>>>> >>> >>> > So having an interface seemed easier and also give more
> >>>>>>>> flexibility for
> >>>>>>>> >>> >>> > users to define their own topics name, define what is
> >>>>>>>> internal topic means,
> >>>>>>>> >>> >>> > how to find checkpoints topics and it will be one line
> >>>>>>>> config for each
> >>>>>>>> >>> >>> > herder, also it more consistence with MM2 code as MM2
> >>>>>>>> config have
> >>>>>>>> >>> >>> > TopicFilter, ReplicationPolicy, GroupFilter, etc as
> >>>>>>>> interface and they can
> >>>>>>>> >>> >>> > be overridden by providing a custom implementation for
> >>>>>>>> them or have some
> >>>>>>>> >>> >>> > config that change their default implementations.
> >>>>>>>> >>> >>> >
> >>>>>>>> >>> >>> > Hope this answer your question. I also updated the KIP
> to
> >>>>>>>> add this to the
> >>>>>>>> >>> >>> > rejected solutions.
> >>>>>>>> >>> >>> >
> >>>>>>>> >>> >>> >
> >>>>>>>> >>> >>> > On Thu, Dec 3, 2020 at 3:19 PM Mickael Maison <
> >>>>>>>> mickael.mai...@gmail.com>
> >>>>>>>> >>> >>> > wrote:
> >>>>>>>> >>> >>> >
> >>>>>>>> >>> >>> > > Hi Omnia,
> >>>>>>>> >>> >>> > >
> >>>>>>>> >>> >>> > > Thanks for the KIP. Indeed being able to configure
> >>>>>>>> MM2's internal
> >>>>>>>> >>> >>> > > topic names would be a nice improvement.
> >>>>>>>> >>> >>> > >
> >>>>>>>> >>> >>> > > Looking at the KIP, I was surprised you propose an
> >>>>>>>> interface to allow
> >>>>>>>> >>> >>> > > users to specify names. Have you considered making
> >>>>>>>> names changeable
> >>>>>>>> >>> >>> > > via configurations? If so, we should definitely
> mention
> >>>>>>>> it in the
> >>>>>>>> >>> >>> > > rejected alternatives as it's the first method that
> >>>>>>>> comes to mind.
> >>>>>>>> >>> >>> > >
> >>>>>>>> >>> >>> > > I understand an interface gives a lot of flexibility
> >>>>>>>> but I'd expect
> >>>>>>>> >>> >>> > > topic names to be relatively simple and known in
> >>>>>>>> advance in most
> >>>>>>>> >>> >>> > > cases.
> >>>>>>>> >>> >>> > >
> >>>>>>>> >>> >>> > > I've not checked all use cases but something like
> below
> >>>>>>>> felt appropriate:
> >>>>>>>> >>> >>> > > clusters = primary,backup
> >>>>>>>> >>> >>> > >
> >>>>>>>> primary->backup.offsets-sync.topic=backup.mytopic-offsets-sync
> >>>>>>>> >>> >>> > >
> >>>>>>>> >>> >>> > > On Tue, Dec 1, 2020 at 3:36 PM Omnia Ibrahim <
> >>>>>>>> o.g.h.ibra...@gmail.com>
> >>>>>>>> >>> >>> > > wrote:
> >>>>>>>> >>> >>> > > >
> >>>>>>>> >>> >>> > > > Hey everyone,
> >>>>>>>> >>> >>> > > > Please take a look at KIP-690:
> >>>>>>>> >>> >>> > > >
> >>>>>>>> >>> >>> > > >
> >>>>>>>> >>> >>> > >
> >>>>>>>>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-690%3A+Add+additional+configuration+to+control+MirrorMaker+2+internal+topics+naming+convention
> >>>>>>>> >>> >>> > > >
> >>>>>>>> >>> >>> > > > Thanks for your feedback and support.
> >>>>>>>> >>> >>> > > >
> >>>>>>>> >>> >>> > > > Omnia
> >>>>>>>> >>> >>> > > >
> >>>>>>>> >>> >>> > >
> >>>>>>>>
> >>>>>>>
>
--
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
t; from myself obviously
>
> -Jason
>
--
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
Gwen Shapira created KAFKA-12674:
Summary: Client failover takes 2-4 seconds on clean broker shutdown
Key: KAFKA-12674
URL: https://issues.apache.org/jira/browse/KAFKA-12674
Project: Kafka
com/apache/kafka/releases/tag/2.8.0-rc2
> > >
> > > * Documentation:
> > > https://kafka.apache.org/28/documentation.html
> > >
> > > * Protocol:
> > > https://kafka.apache.org/28/protocol.html
> > >
> > > * Successful Jenkins builds for the 2.8 branch:
> > > Unit/integration tests:
> > > https://ci-builds.apache.org/job/Kafka/job/kafka/job/2.8/
> > > (still flaky)
> > >
> > > System tests:
> > >
> > > https://jenkins.confluent.io/job/system-test-kafka/job/2.8/6
> > > 0/
> > >
> > >
> > >
> >
> http://confluent-kafka-2-8-system-test-results.s3-us-west-2.amazonaws.com/2021-04-14--001.1618407001--confluentinc--2.8--1b61272d45/report.html
> > >
> > > /**
> > >
> > > Thanks,
> > > John
> > >
> > >
> > >
> >
>
--
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
apache.org/job/Kafka/job/kafka-2.8-jdk8/
>- https://ci-builds.apache.org/job/Kafka/job/kafka-trunk-jdk11/
>
> Let me know if you have questions or comments.
>
> Ismael
>
--
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
Woot!
+1
On Sat, Mar 20, 2021, 10:41 AM Ryanne Dolan wrote:
> Hey y'all, I'm starting the vote on KIP-720, which proposes to deprecate
> the original MirrorMaker in the upcoming 3.0 major release.
>
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-720%3A+Deprecate+MirrorMaker+v1
>
>
rds+and+offsets
>
> This feature should make it possible to implement source connectors with
> exactly-once delivery guarantees, and even allow a wide range of existing
> source connectors to provide exactly-once delivery guarantees with no
> changes required.
>
> Cheers,
>
>
I realize that I'm very very late to a pretty big party, but I was
just looking at the new code (super readable, btw), and noticed that
the inverse API for registerBroker() is called decomissionBroker().
It would be way clearer that we are undoing a registration is we
rename it to
t; KafkaApis framework would handle it like any other request. However, by
> > the
> > > time we are inside the getTopicMetadata call there doesn't seem to be a
> > > clean way to fire off another message that would make its way through
> > > KafkaApis. Perhaps another XManager type object is required?
> > >
> > > Looking for alternative ideas or guidance (or I missed something in the
> > > broker).
> > >
> > > Thank you.
> > >
> > > Knowles
> > >
> >
--
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
parsing of host:port addresses in various configs for Kafka 3.0:
> >
> >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-697%3A+Stricter+parsing+of+addresses+in+configs
> >
> > I'd be grateful for any feedback people may have.
> >
> > Kind regard
to start the voting for KIP-687:
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-687
> > %3A+Automatic+Reloading+of+Security+Store to make the security store
> > reloading automated.
> >
> > Best,
> > Boyang
> >
--
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
+1, Thank you!!!
On Wed, Jan 6, 2021 at 9:29 PM John Roesler wrote:
>
> Hello All,
>
> I'd like to volunteer to be the release manager for our next
> feature release, 2.8.0. If there are no objections, I'll
> send out the release plan soon.
>
> Thanks,
> John R
look and vote if you can.
>
> Best,
> David
--
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
ki, Thorsten Hake, Tom
> Bentley, tswstarplanet, vamossagar12, Vikas Singh, vinoth chandar, Vito
> Jeng, voffcheg109, xakassi, Xavier Léauté, Yuriy Badalyantc, Zach Zhang
>
> We welcome your help and feedback. For more information on how to
> report problems, and to get involved, visit the project website at
> https://kafka.apache.org/
>
> Thank you!
>
>
> Regards,
> Bill Bejeck
--
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
On Fri, Dec 18, 2020, 12:48 PM Colin McCabe wrote:
> On Fri, Dec 18, 2020, at 08:32, Gwen Shapira wrote:
> > Agree. Once we have the basic API we can decide what belongs and what
> > doesn't. I remember requests for broker status, version, and various
> > other things.
>
> > > I'd like to propose a small KIP:
> > >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-700%3A+Add+Describe+Cluster+API
> > >
> > > Please let me know what you think.
> > >
> > > Best,
> > > David
> > >
> >
--
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
for the 2.7 branch:
> Unit/integration tests:
> https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka-2.7-jdk8/detail/kafka-2.7-jdk8/81/
>
> Thanks,
> Bill
--
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
[
https://issues.apache.org/jira/browse/KAFKA-7819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gwen Shapira resolved KAFKA-7819.
-
Fix Version/s: 2.2.0
Resolution: Fixed
Closing since I noticed the PR was merged
erns for testing by making it easier
> to mock AdminService.
>
> I am looking forward to hearing your thoughts on this.
>
> Cheers
> noa
--
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
able+the+strongest+delivery+guarantee+by+default>
>
> Please vote in this mail thread.
>
> Thanks
>
> - Cheng Tan
--
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
cwiki.apache.org/confluence/display/KAFKA/KIP-687%3A+Automatic+Reloading+of+Security+Store
> > > >
> > > > This KIP is trying to deprecate the AlterConfigs API support of updating
> > > > the security store by reloading path in-place, and replace with a
> > > > file-watch mechanism inside the broker. Let me know what you think.
> > > >
> > > > Best,
> > > > Boyang
> > > >
> > >
> >
--
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
ses/tag/2.7.0-rc3
>
> * Documentation:
> https://kafka.apache.org/27/documentation.html
>
> * Protocol:
> https://kafka.apache.org/27/protocol.html
>
> * Successful Jenkins builds for the 2.7 branch:
> Unit/integration tests: (link to follow)
> System tests: (link to follow)
>
> Thanks,
> Bill
--
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
e of accidental deletion. We
> also need to consider that not all installations will be secure and we need
> to
> provide a simpler way to gate keep certain functionalities such as delete
> topic.
>
> Thanks,
> Harsha
>
>
> On Wed, Nov 25, 2020 at 10:25 AM Gwen
amic update of
> delete.topic.enable config.
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-688%3A+Support+dynamic+update+of+delete.topic.enable+config
>
> Please take a look and let me know if you have any feedback.
>
> Thanks
> --
> Prateek Agarwal
--
Gwe
the latest advances on KIP-500, but like
> Gwen, I'm not sure we'd want to move that logic into the Controller.
> This KIP is keeping the metadata creation in AdminManager and just
> making the logic pluggable.
>
> Thanks
>
> On Mon, Nov 16, 2020 at 3:56 PM Gwen Shapira wrote
+1
Thank you for catching this.
On Mon, Nov 16, 2020, 3:22 AM Tom Bentley wrote:
> Hi,
>
> I'd like to start a vote on KIP-681 which proposes to rename a broker
> config to use a more racially neutral term:
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-681
>
Why would the replica placement logic run in the controller rather than in
the AdminManager?
My understanding, and correct me if I got it wrong, is that we are aiming
at better separation of concerns. The controller job will be managing
consensus and consistency of metadata, but creating the
ter Describe.
>
> KIP:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-685%3A+Loosen+permission+for+listing+reassignments
>
> Please, let me know what you think.
>
> Best,
> David
--
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
b8a459842a0fc840c6e40958e7158d29f3f2d6c0d223be80b%40%3Cdev.kafka.apache.org%3E
> [^2]:
> https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/state/internals/NamedCache.java
>
> --
> *Dongjin Lee*
>
> *A hitchhiker in the mathematical world.*
gt;
> Thanks for all the contributions, Sophie!
>
>
> Please join me to congratulate her!
> -Matthias
>
--
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
onnect+REST+API
>
> Thanks
--
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
in the Apache Kafka community.
--
Gwen Shapira
+1 (binding)
Makes sense, thank you.
On Fri, Oct 9, 2020, 2:50 AM Tom Bentley wrote:
> Hi all,
>
> KIP-676 is pretty trivial and the comments on the discussion thread seem to
> be favourable, so I'd like to start a vote on it.
>
>
>
e discover while testing the release)
> will be double-committed. *Please discuss with your reviewer whether your
> PR should go to trunk or to trunk+release so they can merge accordingly.
>
> *Please help us test the release! *
>
> Thanks!
>
> Bill Bejeck
--
Gwen Shapira
Engine
ongjinleekr
> <https://keybase.io/dongjinleekr>linkedin: kr.linkedin.com/in/dongjinleekr
> <https://kr.linkedin.com/in/dongjinleekr>speakerdeck: speakerdeck.com/dongjin
> <https://speakerdeck.com/dongjin>*
--
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
s+with+new+auto-generated+schema
>
> Thanks,
> Anastasia
--
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
be that bad - multi-part messages are well
understood in general.
On Thu, Sep 10, 2020 at 7:51 AM Gwen Shapira wrote:
> There is also another approach (harder to design, but may be easier to use
> and maintain), which is to make Kafka handle large messages better and
> allow users to s
; >
> >
> https://docs.google.com/document/d/1cKBNxPkVVENly9YczYXsVDVWwrCdRq3G_cja5s2QDQg/edit?usp=sharing
> > >
> > > If all that makes sense, I'll create a full fledged KIP document and
> > expand
> > > the idea.
> > >
> > > Thanks,
> > > A.
> > >
> >
> >
> > --
> >
> > Ben Stopford
> >
>
--
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
Congratulations, John!
On Mon, Aug 10, 2020, 1:11 PM Jun Rao wrote:
> Hi, Everyone,
>
> John Roesler has been a Kafka committer since Nov. 5, 2019. He has remained
> active in the community since becoming a committer. It's my pleasure to
> announce that John is now a member of Kafka PMC.
>
>
t; Vito Jeng, Xavier Léauté, xiaodongdu, Zach Zhang, zhaohaidao, zshuo, 阿洋
>
> We welcome your help and feedback. For more information on how to
> report problems, and to get involved, visit the project website at
> https://kafka.apache.org/
>
> Thank you!
>
>
> Regards,
>
> Randall Hauch
>
--
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
/KAFKA/KIP-651+-+Support+PEM+format+for+SSL+certificates+and+private+key
>
>
> Thank you...
>
> Regards,
>
> Rajini
>
--
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
enkins builds for the 2.6 branch:
> Unit/integration tests: https://builds.apache.org/job/kafka-2.6-jdk8/101/
> System tests: (link to follow)
>
>
> Thanks,
> Randall Hauch
>
--
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
y
> pleasure to announce that Mickael is now a member of Kafka PMC.
>
> Congratulations Mickael!
>
> Jun
> on behalf of Apache Kafka PMC
>
--
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
+1 (binding)
Thank you, Sam. It is great to see Trogdor getting the care it deserves.
On Mon, Jun 22, 2020, 1:46 PM Sam Pal wrote:
> Hi all,
>
> I would like to start a vote for KIP-627, which adds metrics about active
> agents and the number of created, running, and done tasks in a Trogdor
>
+1 (binding)
Thank you Xavier!
On Thu, Jun 25, 2020, 3:44 PM Xavier Léauté wrote:
> Hi Everyone,
>
> I would like to initiate the voting process for KIP-629.
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-629%3A+Use+racially+neutral+terms+in+our+codebase
>
> Thank you,
> Xavier
>
fka in the near future. I see there are
> some
> > > > issues
> > > > > > marked as fixed in these two versions
> > > > > > (https://tinyurl.com/ycdpz5cb).
> > > > > > However, I could not find a branch/tag corresponding to these
> > > versions
> > > > in
> > > > > > the github repository (https://github.com/apache/kafka).
> > > > > >
> > > > > > Also, It would be great if someone can help me understand or
> share
> > > any
> > > > > > documentation around the release processes (specifically on when
> we
> > > > > decide
> > > > > > to release a new bug fix version like the ones mentioned above.)
> > > > > >
> > > > > > Thanks,
> > > > > > Sankalp
> > > > > >
> > > > >
> > > >
> > >
> >
>
--
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
Thank you so much for this initiative. Small change, but it makes our
community more inclusive.
Gwen
On Fri, Jun 19, 2020, 6:02 PM Xavier Léauté wrote:
> Hi Everyone,
>
> There are a number of places in our codebase that use racially charged
> terms. I am proposing we update them to use more
failing unit tests that
> have nothing to do with my change. Is there a convention on how to deal
> with these? Should it be something that I try to fix on my branch?
>
> Any thoughts are appreciated.
>
> Thanks,
> Michael
--
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
a connection is attempted?
> >
> > 4) It will be good if we can include two flows with the relationship
> > between various timeouts in the KIP. One with a fixed node like a typical
> > produce/consume request to the leader and another that uses
> > `leastLoadedNode` like a metadata request. Having the comparison between
> > the current and proposed behaviour w.r.t all configurable timeouts (the
> two
> > new connection timeouts, request timeout, api timeout etc.) will be
> useful.
> >
> > Regards,
> >
> > Rajini
> >
>
--
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
s operations to
> protect the Kafka controller:
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-599%3A+Throttle+Create+Topic%2C+Create+Partition+and+Delete+Topic+Operations
>
> Please, let me know what you think.
>
> Cheers,
> David
>
--
Gwen Shapira
Engineering Manag
[
https://issues.apache.org/jira/browse/KAFKA-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gwen Shapira resolved KAFKA-3034.
-
Resolution: Fixed
> kafka.api.PlaintextConsumerTest > testSeek
[
https://issues.apache.org/jira/browse/KAFKA-3035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gwen Shapira resolved KAFKA-3035.
-
Resolution: Fixed
> Transient: kafka.api.PlaintextConsumerTest > testAutoOffsetReset
[
https://issues.apache.org/jira/browse/KAFKA-5034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gwen Shapira resolved KAFKA-5034.
-
Resolution: Won't Do
Assuming this was fixed since there were many iterations on connector
[
https://issues.apache.org/jira/browse/KAFKA-5000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gwen Shapira resolved KAFKA-5000.
-
Resolution: Fixed
Metrics seem like a better indicator than logs.
> Framework should log s
[
https://issues.apache.org/jira/browse/KAFKA-3321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gwen Shapira resolved KAFKA-3321.
-
Resolution: Won't Do
Let's assume this was already resolved or at least the root cause isn't
ce/display/KAFKA/KIP-602%3A+Change+default+value+for+client.dns.lookup
>
> And discussion thread:
>
>
> https://lists.apache.org/thread.html/r0e70d3757267c4158f12c05a4e5ac9eb33f2d11ce99d5878b3b4b3f7%40%3Cdev.kafka.apache.org%3E
>
> --
> Thanks,
> Badai
>
--
Gwen Shapira
ki.apache.org/confluence/display/KAFKA/KIP-612%3A+Ability+to+Limit+Connection+Creation+Rate+on+Brokers
>
> And discussion thread:
>
> https://lists.apache.org/thread.html/r61162661fa307d0bc5c8326818bf223a689c49e1c828c9928ee26969%40%3Cdev.kafka.apache.org%3E
>
> Thanks,
>
&g
ate+commit+records+without+record+metadata
>
> Thanks!
> Mario
>
--
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
d+Metadata+Context+to+MetricsReporter
>
> - Xavier
>
--
Gwen Shapira
m a little worried about backward
> >>> compatibility. And even if the current default is not idea, users can
> >>> still change it.
> >>>
> >>> What I am also wondering thought is, does this only affect MirrorMaker
> >>> or also MirrorMaker 2? A
play/KAFKA/KIP-606%3A+Add+Metadata+Context+to+MetricsReporter
>
> Thank you,
> Xavier
>
--
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
t; brokers anymore since there are no zookeepers.
>
>
> Guozhang
>
> On Mon, May 4, 2020 at 5:33 PM Gwen Shapira wrote:
>
> > +1 for removing MM 1.0 when we cut a breaking release. It is sad to see
> > that we are still investing in it (I just saw a KIP toward improvin
> ZooKeeper-free Kafka code, we will hide it behind an option initially.
> > (We'll have a KIP describing this all in detail soon.)
> >
> > What does everyone think about having Kafka 3.0 come up next after 2.6?
> > Are there any other things we should change in the 2.6 -> 3.0 transition?
> >
> > best,
> > Colin
> >
>
--
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
Hey Boyang,
Sorry if this was already discussed, but I didn't see this as rejected
alternative:
Until now, we always did client side routing - the client itself found the
controller via metadata and directed requests accordingly. Brokers that
were not the controller, rejected those requests.
Kafka purges old messages from both leaders and replicas.
If there was a mistake in the book, can you tell me which chapter and page?
We'll fix it.
Gwen
On Wed, Apr 22, 2020, 7:51 AM Alex Bull wrote:
> Hi, Dear Kafka Developers,
>
> I've read 'Kafka: The Definitive Guide' by Narkhede and
to revisit before releasing new doc version to
the release process wiki.
Gwen
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
On Mon, Apr 13, 2020 at 2:59 PM, Israel Ekpo < israele...@gmail.com > wrote:
>
> Hi Everyone,
>
>
> and taking out the support for reading from STDIN).
>
> Thanks!
>
> On Fri, Mar 27, 2020 at 10:01 AM Rajini Sivaram
> wrote:
> >
> > +1 (binding)
> >
> > Thanks for the KIP, Aneel!
> >
> > Regards,
> >
> > Rajini
> >
> &
+1 (binding), thank you
On Wed, Mar 18, 2020, 2:26 PM Colin McCabe wrote:
> +1 (binding).
>
> Thanks, Aneel.
>
> best,
> Colin
>
> On Wed, Mar 18, 2020, at 00:04, David Jacot wrote:
> > +1 (non-binding), thanks for the KIP!
> >
> > David
> >
> > On Mon, Mar 16, 2020 at 4:06 PM Aneel Nazareth
>
+1 (binding) - thank you
On Mon, Mar 23, 2020, 10:50 AM Sanjana Kaundinya
wrote:
> Hi Everyone,
>
> I’d like to start a vote for KIP-580: Exponential Backoff for Kafka
> Clients. The link to the KIP can be found here:
>
Oh wow, I love this checklist. I don't think we'll have time to create one for
this release, but will be great to track this via JIRA and see if we can get
all those contributed before 2.6...
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
On Mon
+1
Looks great. Thanks for the proposal, Shailesh.
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
On Mon, Mar 09, 2020 at 6:00 AM, Shailesh Panwar < span...@confluent.io > wrote:
>
>
>
> Hi All,
> I would like to st
+1 (binding)
Verified signatures, built jars from source, quickstart passed and local unit
tests all passed.
Thank you for the release Bill!
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
On Sat, Mar 07, 2020 at 8:15 PM, Vahid Hashemian
+1
On Mon, Feb 24, 2020, 2:16 AM David Jacot wrote:
> Hi all,
>
> I would like to start a vote on KIP-570: Add leader epoch in
> StopReplicaRequest
>
> The KIP is here:
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-570%3A+Add+leader+epoch+in+StopReplicaRequest
>
> Thanks,
> David
>
+1 (binding)
Gwen Shapira
Engineering Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
On Mon, Mar 02, 2020 at 8:32 AM, David Jacot < dja...@confluent.io > wrote:
>
>
>
> +1 (non-binding). Thanks for the KIP!
>
>
>
> David
>
>
Gwen Shapira created KAFKA-9631:
---
Summary: MockAdminClient doesn't handle CreateTopics optional
fields
Key: KAFKA-9631
URL: https://issues.apache.org/jira/browse/KAFKA-9631
Project: Kafka
Ah, got it! I am indeed curious why they do this :)
Maybe John can shed more light. But if we can't find a better fix, perhaps the
nice thing to do is really a separate logger, so users who are not worried
about shooting themselves in the foot can make those warnings go away.
Gwen Shapira
erience, but I am not sure it is the job of Apache Kafka
to protect users from their choice of libraries (and as long as those libraries
are OSS, users can fix them). Especially not at the expense of someone who
doesn't use 3rd party libs.
Gwen
Gwen Shapira
Engineering Manager | Confluent
650
supporting Kafka - customers run logs at INFO level
normally, so if we suspect a misconfiguration, we don't want to ask the
customer to change level to DEBUG and bounce the broker. It is time consuming
and can be risky.
*Gwen Shapira*
Product Manager | Confluent
650.450.2760 | @gwenshap
Follow us
Thank you for the KIP. Awesomely cloud-native improvement :)
+1 (binding)
On Tue, Jan 21, 2020, 9:35 AM David Jacot wrote:
> Hi all,
>
> I would like to start a vote on KIP-559: Make the Kafka Protocol Friendlier
> with L7 Proxies.
>
> The KIP is here:
>
>
+1 (binding), this has been an on-going concern. Thank you for
addressing this, Ron.
On Mon, Jan 20, 2020 at 5:22 AM Ron Dagostino wrote:
>
> Hi everyone. I would like to start a vote on KIP-515: Enable ZK
> client to use the new TLS supported authentication.
>
> The KIP is here:
>
you
> Gwen Shapira!
> We'll add a flag to clear all topics by clientId
> It is ‘reset-all-external-topics’
>
> I also want to use regex on the input topic flag to clear all matching
> topics.
>
> On 2020/01/17 19:29:09, Gwen Shapira wrote:
> > Seem like a very nice impro
Seem like a very nice improvement to me. But I have to admit that I
don't understand how this will how - how could you infer the input
topics?
On Thu, Jan 16, 2020 at 10:03 AM Sang wn Lee wrote:
>
> Hello,
>
> Starting this thread to discuss KIP-560:
> wiki link :
>
+1 (binding)
Looks super useful. Thank you.
On Mon, Jan 13, 2020 at 8:16 AM Konstantine Karantasis
wrote:
>
> Hi everyone.
>
> I hope y'all had a nice break. The discussion on KIP-158 seems to have
> wrapped up since last year, so I'd like to open the vote on this KIP.
>
> A reminder that
+1 (binding, re-vote)
On Tue, Jan 14, 2020 at 11:23 AM Colin McCabe wrote:
>
> Hi all,
>
> I'm reposting this since I've been informed that gmail mashed the original
> VOTE thread into a different email thread. Hopefully the different thread
> title will prevent it doing that in this case.
>
Hi everyone,
I'm happy to announce that Colin McCabe, Vahid Hashemian and Manikumar
Reddy are now members of Apache Kafka PMC.
Colin and Manikumar became committers on Sept 2018 and Vahid on Jan
2019. They all contributed many patches, code reviews and participated
in many KIP discussions. We
+1, thanks for driving this
On Fri, Jan 10, 2020 at 2:17 PM Colin McCabe wrote:
>
> Hi all,
>
> I'd like to start the vote on KIP-551: Expose disk read and write metrics.
>
> KIP: https://cwiki.apache.org/confluence/x/sotSC
>
> Discussion thread:
>
Sorry for the super late reply, but since the vote thread showed up,
I've read the KIP and have a concern.
The concern was raised by Matthias Sax earlier, but I didn't see it addressed.
Basically the current iteration of the KIP unifies deserialization
errors with corruption. As was pointed out,
I think you linked to the wrong KIP? Here's a link that worked for me:
https://s.apache.org/ve15g
On Thu, Jan 9, 2020 at 4:24 PM Colin McCabe wrote:
>
> Hi all,
>
> I wrote KIP about deprecating the --zookeeper flag in the administrative
> tools. Take a look if you get a chance:
>
>
Gwen Shapira created KAFKA-9328:
---
Summary: Move MirrorMaker 2.0 documentation to site-docs
Key: KAFKA-9328
URL: https://issues.apache.org/jira/browse/KAFKA-9328
Project: Kafka
Issue Type
e can get the site updated.
>
> Ryanne
>
> On Mon, Dec 23, 2019, 8:38 AM Gwen Shapira wrote:
>
> > There's nothing preventing us from updating the site-docs now to make them
> > match the 2.4 release better (I just opened few JIRA for additional docs).
> >
> > I
a-mirror-maker.sh script is provided which
> replaces and emulates the legacy script.
>
> Accordingly, let's plan to expand the documentation in the next release,
> and then remove the legacy documentation in Phase 3. At present, the
> existing documentation is still accurate.
>
&
Gwen Shapira created KAFKA-9327:
---
Summary: GroupMetadata metrics are not documented
Key: KAFKA-9327
URL: https://issues.apache.org/jira/browse/KAFKA-9327
Project: Kafka
Issue Type: Improvement
1 - 100 of 2807 matches
Mail list logo
