Re: [DISCUSS] KIP-110: Add Codec for ZStandard Compression

*My concern is that this minor improvement (based on the benchmark) over LZ4 does not warrant the work of adding support for a new compression codec to the broker, official clients and horde of 3rd party clients, including upgrade paths, transformations, tests, additional dependencies, etc.* As

Re: [DISCUSS] KIP-117: Add a public AdministrativeClient API for Kafka admin operations

For individual consumer groups, it would be nice if the admin client made it possible to fetch consumer offsets for the entire consumer group. Then we don't have to manually assemble this outside of the admin client interface. On Feb 6, 2017 11:41 AM, "Colin McCabe" wrote: >

KIP-54 voting status?

I joined the dev list after KIP-54 voting started, so unfortunately don't have the old thread to bump. But wanted to check if there was any news on this? >From KAFKA-2273 sounds like there are no outstanding objections to the design, but there also aren't yet enough +1's, so is this just

Re: [DISCUSS] KIP-115: Enforce offsets.topic.replication.factor

As a user, I'd personally rather see this land sooner rather than waiting for a major release... As long as it fails noisily if I don't have enough brokers when it tries to create the topic, then it's easy enough to solve... On Wed, Jan 25, 2017 at 4:22 PM, Ismael Juma wrote:

Fwd: [jira] [Commented] (KAFKA-1207) Launch Kafka from within Apache Mesos

I don't understand why the whole dev list needs to be notified of failed email deliveries. Is there a way to shut these off? -- Forwarded message -- From: postmas...@inn.ru (JIRA) Date: Tue, Jan 17, 2017 at 9:50 PM Subject: [jira] [Commented] (KAFKA-1207)

Okay to remove the deprecated kafka-consumer-offset-checker.sh?

Kafka 9 deprecated kafka-consumer-offset-checker.sh (kafka.tools.ConsumerOffsetChecker) in favor of kafka-consumer-groups.sh (kafka.admin.ConsumerGroupCommand). Since this was deprecated in 9, and the full functionality of the old script appears to be available in the new script, can we

Re: Is this a bug or just unintuitive behavior?

not is, > unfortunately, the more complicated question :( > > -Ewen > > On Thu, Jan 5, 2017 at 1:10 PM, James Cheng <wushuja...@gmail.com> wrote: > > > > > > On Jan 5, 2017, at 12:57 PM, Jeff Widman <j...@netskope.com> wrote: > > > > > > T

Re: Is this a bug or just unintuitive behavior?

ffsets.retention.minutes > from 1 day (1440) to 7 days (10080)...or something similar. I have seen a > handful of scenarios where an outage lasts longer than a day, the offsets > get purged causing the auto.offset.reset to kick in and in the case of > earliest, re-replicating billions of messages. &

Re: [DISCUSS] 0.10.3.0/0.11.0.0 release planning

+1 for major version bump. A good bit of deprecated code I would like to see removed especially on old consumer side plus a few other settings defaults changed such as the brief discussion on mirrormaker options a few months back. Just be good to continue to make the new user experience a lot

Re: [VOTE] Add REST Server to Apache Kafka

-1 As an end-user, while I like the idea in theory, in practice I don't think it's a good idea just yet. Certainly, it'd be useful, enabling things like https://github.com/Landoop/kafka-topics-ui to work without needing anything outside of Kafka core. But there are already enough things in the

Re: [VOTE] KIP-106 - Default unclean.leader.election.enabled True => False

+1 nonbinding. We were bit by this in a production environment. On Wed, Jan 11, 2017 at 11:42 AM, Ian Wrigley wrote: > +1 (non-binding) > > > On Jan 11, 2017, at 11:33 AM, Jay Kreps wrote: > > > > +1 > > > > On Wed, Jan 11, 2017 at 10:56 AM, Ben Stopford

Re: any production deployment of kafka 0.10.2.0

+ Users list On Mar 26, 2017 8:17 AM, "Jianbin Wei" wrote: We are thinking about upgrading our system to 0.10.2.0. Has anybody upgraded his/her system to 0.10.2.0 and any issues? Regards, -- Jianbin

Re: [DISCUSS] KIP-142: Add ListTopicsRequest to efficiently list all the topics in a cluster

In theory this is nice. However, one possibly silly question... personally I've found that most times when I wanted to list the topics, I normally also wanted to know additional metadata such as the leader of the topic. So I just want to doublecheck that there are enough common use cases that

Add me to jira/confluence?

Can someone add me to Jira/confluence for Kafka? Don't use this work email, use j...@jeffwidman.com Thanks!

Re: [VOTE] KIP-107: Add purgeDataBefore() API in AdminClient

re: Purge vs PurgeMessages/Records - I also prefer that it be more explicit about what is being purged. Despite the inconsistency with Fetch/Produce, because it's explicit about what's being purged there shouldn't be additional confusion. Who knows what in the future might need purging? Adding the

Is there already a KIP or JIRA to change auto.create.topics.enable to default to false?

I thought I saw mention somewhere of changing the default of auto.create.topics.enable to false. I searched, but couldn't find anything in JIRA... am I imagining things? Now that there's API support for creating topics, the version bump to 0.11.0 seems like a good time to re-evaluate whether

Re: Is there already a KIP or JIRA to change auto.create.topics.enable to default to false?

enable would > be considered "breaking" I think it would be best to consider this change > as a part of the deprecation and eventual removal of the configuration. > Perhaps 0.11 would be a good timeframe to consider doing that, but it > depends on if the supporting features are

Re: [DISCUSS] KIP-175: Additional '--describe' views for ConsumerGroupCommand

Thanks for the KIP Vahid. I think it'd be useful to have these filters. That said, I also agree with Edo. We don't currently rely on the output, but there's been more than one time when debugging an issue that I notice something amiss when I see all the data at once but if it wasn't present in

Re: Comments on JIRAs

Thanks for bringing this up. I don't know what the solution is, but I agree it would be good to have something. On Jul 13, 2017 6:46 AM, "Tom Bentley" wrote: > The project recently switched from all JIRA events being sent to the dev > mailling list, to just issue

Re: Changing JIRA notifications to remove Single Email Address (dev@kafka.apache.org)

, Matthias J. Sax <matth...@confluent.io> > > wrote: > > > >> Have you considered starting a new mailing list "issues@" that gets the > >> full load of all emails? > >> > >> -Matthias > >> > >> On 6/6/17 7:26 PM, Jeff Widman

Re: Pull-Requests not worked on

On Thu, Jun 8, 2017 at 5:00 PM, Ismael Juma wrote: > One of the reasons why some PRs remain open even if they should be > closed is that we have no way to close PRs ourselves without referencing > them in a commit or asking Apache Infra for help. So, abandoned PRs tend to >

Re: Pull-Requests not worked on

I agree with this. As a new contributor, it was a bit demoralizing to look at all the open PR's and wonder whether when I sent a patch it would just be left to sit in the ether. In other projects I'm involved with, more typically the maintainers go through periodically and close old PR's that

Re: Jira-Spam on Dev-Mailinglist

I'm hugely in favor of this change as well... Although I actually find the Github pull request emails less useful than the jirabot ones since Jira typically has more info when I'm trying to figure out if the issue is relevant to me or not... On Tue, May 30, 2017 at 2:28 PM, Guozhang Wang

Re: Changing JIRA notifications to remove Single Email Address (dev@kafka.apache.org)

rwise we need > to talk to Apache INFRA for it. > > > Guozhang > > > > On Tue, May 30, 2017 at 11:57 PM, Michal Borowiecki < > michal.borowie...@openbet.com> wrote: > >> +1 agree with Jeff, >> >> Michał >> >> On 31/05/17 06:25, Jeff W

Re: [VOTE] KIP-175: Additional '--describe' views for ConsumerGroupCommand

+1 (non-binding) On Sep 28, 2017 12:13 PM, "Vahid S Hashemian" wrote: > I'm bumping this up as it's awaiting one more binding +1, but I'd like to > also mention a recent change to the KIP. > > Since the current DescribeGroup response protocol does not include >

Re: [DISCUSS] KIP-211: Revise Expiration Semantics of Consumer Group Offsets

; cost of extra clutter if there are other consumer groups which are truly > created/used/abandoned on a frequent basis. Being able to set > offsets.retention.minutes on a per groupId basis allows this to also be > covered cleanly, and makes it visible that these groupIds are a special > case. &

Re: [DISCUSS] Kafka 2.0.0 in June 2018

Hi all, > > > > >> > > > > >> I'm starting this discussion early because of the potential > impact. > > > > >> > > > > >> Kafka 1.0.0 was just released and the focus was on achieving the > > > > original > > >

Re: [DISCUSS] KIP-211: Revise Expiration Semantics of Consumer Group Offsets

Any other input on this? Otherwise Vahid what do you think about moving this to a vote? On Tue, Nov 7, 2017 at 2:34 PM, Jeff Widman <j...@jeffwidman.com> wrote: > Any other feedback from folks on KIP-211? > > A prime benefit of this KIP is that it removes the need for the consu

Re: [DISCUSS] KIP-217: Expose a timeout to allow an expired ZK session to be re-created

; > stuck for a while though, maybe a little push would help : > > > > > > https://issues.apache.org/jira/plugins/servlet/mobile# > > > > > issue/ZOOKEEPER-2184 > > > > > > > > > > > > 2) upon recreating the zookeeper object , is it not possible > to > > > > > invalidate > > > > > > the DNS cache so that it resolves the new hostname? > > > > > > > > > > > > 3) could the cluster be down in this situation: one migrates > an > > > > > entire > > > > > > zookeeper cluster to new machines (one by one). The quorum is > > > still > > > > > alive > > > > > > without downtime, but now every broker in a cluster can't > > resolve > > > > > zookeeper > > > > > > at the same time. They all shut down at the same time after > the > > > new > > > > > > time-out setting. > > > > > > > > > > > > Thanks ! > > > > > > Stéphane > > > > > > > > > > > > On 28 Oct. 2017 9:42 am, "Jun Rao" <j...@confluent.io> wrote: > > > > > > > > > > > > > Hi, Everyone, > > > > > > > > > > > > > > We created "KIP-217: Expose a timeout to allow an expired > ZK > > > > > session to > > > > > > be > > > > > > > re-created". > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > > > > > > > 217%3A+Expose+a+timeout+to+allow+an+expired+ZK+session+ > > > > > to+be+re-created > > > > > > > > > > > > > > Please take a look and provide your feedback. > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > Jun > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- *Jeff Widman* jeffwidman.com <http://www.jeffwidman.com/> | 740-WIDMAN-J (943-6265) <><

Re: [DISCUSS] KIP-211: Revise Expiration Semantics of Consumer Group Offsets

)committing duplicate offsets, wasting network/disk I/O. On Mon, Oct 30, 2017 at 3:47 PM, Jeff Widman <j...@jeffwidman.com> wrote: > I support this as the proposed change seems both more intuitive and safer. > > Right now we've essentially hacked this at my day job by bumping the >

Can someone update this KIP's status to "accepted"?

https://cwiki.apache.org/confluence/display/KAFKA/KIP-164-+Add+UnderMinIsrPartitionCount+and+per-partition+UnderMinIsr+metrics the JIRA shows it was shipped in 1.0, but the wiki page lists it as "Discussion" -- *Jeff Widman* jeffwidman.com <http://www.jeffwidman.com/> | 740-WIDMAN-J (943-6265) <><

Re: [VOTE] KIP-214: Add zookeeper.max.in.flight.requests config to the broker

gt; wrote: > > > I'd like to start the vote for KIP-214: Add > > zookeeper.max.in.flight.requests config to the broker > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > > 214%3A+Add+zookeeper.max.in.flight.requests+config+to+the+broker > > > &g

Re: [DISCUSS] KIP-217: Expose a timeout to allow an expired ZK session to be re-created

> Thanks ! > > Stéphane > > > > On 28 Oct. 2017 9:42 am, "Jun Rao" <j...@confluent.io> wrote: > > > > > Hi, Everyone, > > > > > > We created "KIP-217: Expose a timeout to allow an expired ZK > session to

Re: [DISCUSS] KIP-211: Revise Expiration Semantics of Consumer Group Offsets

oup+Offsets > > Your feedback is welcome! > > Thanks. > --Vahid > > > > -- *Jeff Widman* jeffwidman.com <http://www.jeffwidman.com/> | 740-WIDMAN-J (943-6265) <><

Requesting wiki access for creating a KIP

I thought I already had wiki access, but apparently I don't... Can someone give j...@jeffwidman.com proper permissions for creating a new KIP? Thanks! -- *Jeff Widman* jeffwidman.com <http://www.jeffwidman.com/> | 740-WIDMAN-J (943-6265) <><

Re: Requesting wiki access for creating a KIP

; > On 5/6/18 11:12 AM, Jeff Widman wrote: > > I thought I already had wiki access, but apparently I don't... > > > > Can someone give j...@jeffwidman.com proper permissions for creating a > new > > KIP? > > > > Thanks! > > > > -- *Jeff Widman* jeffwidman.com <http://www.jeffwidman.com/> | 740-WIDMAN-J (943-6265) <><

Re: Use of a formatter like Scalafmt

gt; Hi, > > >> > > >> Contributing to Kafka Streams' Scala API, I've been kinda lost on how > > >> should I format my code. > > >> I know formatting is the start of religion wars but personally I have > no > > >> preference at all. I just want consistency acros

Re: [VOTE] KIP-229: DeleteGroups API

+1 (non-binding) On Jan 15, 2018 10:23 AM, "Vahid S Hashemian" wrote: > Happy Monday, > > I believe the concerns on this KIP have been addressed in the current > version of the KIP: > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 229%3A+DeleteGroups+API >

Re: [VOTE] KIP-222 - Add "describe consumer group" to KafkaAdminClient

t; > >> > > > good > > > > > >> > > > > overall. > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > Guozhang > > > > > >> > > > > > > > > > >> > > > > On Tue, Nov 14, 2017 at 10:02 AM, Jorge Esteban Quilcate > > > > Otoya < > > > > > >> > > > > quilcate.jo...@gmail.com> wrote: > > > > > >> > > > > > > > > > >> > > > > > Added. > > > > > >> > > > > > > > > > > >> > > > > > El mar., 14 nov. 2017 a las 19:00, Ted Yu (< > > > > > yuzhih...@gmail.com>) > > > > > >> > > > > > escribió: > > > > > >> > > > > > > > > > > >> > > > > > > Please fill in JIRA number in Status section. > > > > > >> > > > > > > > > > > > >> > > > > > > On Tue, Nov 14, 2017 at 9:57 AM, Jorge Esteban > > Quilcate > > > > > Otoya < > > > > > >> > > > > > > quilcate.jo...@gmail.com> wrote: > > > > > >> > > > > > > > > > > > >> > > > > > > > JIRA issue title updated. > > > > > >> > > > > > > > > > > > > >> > > > > > > > El mar., 14 nov. 2017 a las 18:45, Ted Yu (< > > > > > >> > yuzhih...@gmail.com > > > > > >> > > >) > > > > > >> > > > > > > > escribió: > > > > > >> > > > > > > > > > > > > >> > > > > > > > > Can you fill in JIRA number (KAFKA-6058 > > > > > >> > > > > > > > > <https://issues.apache.org/ > jira/browse/KAFKA-6058 > > >) > > > ? > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > If one JIRA is used for the two additions, > > consider > > > > > updating > > > > > >> > > the > > > > > >> > > > > JIRA > > > > > >> > > > > > > > > title. > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > On Tue, Nov 14, 2017 at 9:04 AM, Jorge Esteban > > > > Quilcate > > > > > >> > Otoya < > > > > > >> > > > > > > > > quilcate.jo...@gmail.com> wrote: > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > Hi all, > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > As I didn't see any further discussion around > > this > > > > > KIP, I'd > > > > > >> > > > like > > > > > >> > > > > to > > > > > >> > > > > > > > start > > > > > >> > > > > > > > > > voting. > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > KIP documentation: > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > https://cwiki.apache.org/confluence/pages/viewpage. > > > > > >> > > > > > > > action?pageId=74686265 > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > Cheers, > > > > > >> > > > > > > > > > Jorge. > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > -- > > > > > >> > > > > -- Guozhang > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > -- > > > > > >> > > -- Guozhang > > > > > >> > > > > > > > >> > > > > > > > > > > > > > > > > -- *Jeff Widman* jeffwidman.com <http://www.jeffwidman.com/> | 740-WIDMAN-J (943-6265) <><

Do the Jackson security vulnerabilities affect Kafka at all?

JSON files, for example the partition-reassignment scripts... So do these CVE's apply to Kafka? If so, it seem the patch is fairly trivial of just upgrading to a newer version of Jackson... should this also be backported to the 1.0.1 release? -- *Jeff Widman* jeffwidman.com <h

Re: Do the Jackson security vulnerabilities affect Kafka at all?

checked trunk and 1.1? They should be using the latest version. > > Ismael > > On Tue, Feb 20, 2018 at 10:38 PM, Jeff Widman <j...@jeffwidman.com> wrote: > > > The Jackson JSON parser library had a couple of CVE's announced: > > 1. CVE-2017-7525 > > 2.

Re: [ANNOUNCE] Apache Kafka 2.0.0 Released

t; > > > > > > > > > > Adem Efe Gencer, Alex D, Alex Dunayevsky, Allen Wang, Andras Beni, > > > > > > > > > > Andy Bryant, Andy Coates, Anna Povzner, Arjun Satish, asutosh936, > > > > > > > > > > Attila Sasvari, bartdevylder, Benedict Jin, Bill Bejeck, Blake > > Miller, > > > > > > > > > > Boyang Chen, cburroughs, Chia-Ping Tsai, Chris Egerton, Colin P. > > > Mccabe, > > > > > > > > > > Colin Patrick McCabe, ConcurrencyPractitioner, Damian Guy, dan > > norwood, > > > > > > > > > > Daniel Shuy, Daniel Wojda, Dark, David Glasser, Debasish Ghosh, > > > Detharon, > > > > > > > > > > Dhruvil Shah, Dmitry Minkovsky, Dong Lin, Edoardo Comar, emmanuel > > > Harel, > > > > > > > > > > Eugene Sevastyanov, Ewen Cheslack-Postava, Fedor Bobin, > > > > fedosov-alexander, > > > > > > > > > > Filipe Agapito, Florian Hussonnois, fredfp, Gilles Degols, gitlw, > > > > Gitomain, > > > > > > > > > > Guangxian, Gunju Ko, Gunnar Morling, Guozhang Wang, hmcl, huxi, > > huxihx, > > > > > > > > > > Igor Kostiakov, Ismael Juma, Jacek Laskowski, Jagadesh Adireddi, > > > > > > > > > > Jarek Rudzinski, Jason Gustafson, Jeff Klukas, Jeremy Custenborder, > > > > > > > > > > Jiangjie (Becket) Qin, Jiangjie Qin, JieFang.He, Jimin Hsieh, Joan > > > > Goyeau, > > > > > > > > > > Joel Hamill, John Roesler, Jon Lee, Jorge Quilcate Otoya, Jun Rao, > > > > > > > > > > Kamal C, khairy, Koen De Groote, Konstantine Karantasis, Lee > Dongjin, > > > > > > > > > > Liju John, Liquan Pei, lisa2lisa, Lucas Wang, Magesh Nandakumar, > > > > > > > > > > Magnus Edenhill, Magnus Reftel, Manikumar Reddy, Manikumar Reddy O, > > > > > > > > > > manjuapu, Mats Julian Olsen, Matthias J. Sax, Max Zheng, maytals, > > > > > > > > > > Michael Arndt, Michael G. Noll, Mickael Maison, nafshartous, Nick > > > > Travers, > > > > > > > > > > nixsticks, Paolo Patierno, parafiend, Patrik Erdes, Radai > Rosenblatt, > > > > > > > > > > Rajini Sivaram, Randall Hauch, ro7m, Robert Yokota, Roman Khlebnov, > > > > > > > > > > Ron Dagostino, Sandor Murakozi, Sasaki Toru, Sean Glover, > > > > > > > > > > Sebastian Bauersfeld, Siva Santhalingam, Stanislav Kozlovski, > > Stephane > > > > > Maarek, > > > > > > > > > > Stuart Perks, Surabhi Dixit, Sönke Liebau, taekyung, tedyu, Thomas > > > > Leplus, > > > > > > > > > > UVN, Vahid Hashemian, Valentino Proietti, Viktor Somogyi, Vitaly > > > Pushkar, > > > > > > > > > > Wladimir Schmidt, wushujames, Xavier Léauté, xin, yaphet, > > > > > > > > > > Yaswanth Kumar, ying-zheng, Yu > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > We welcome your help and feedback. For more information on how to > > > > > > > > > > report problems, and to get involved, visit the project website at > > > > > > > > > > https://kafka.apache.org/ > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thank you! > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > > > > > > > > > > > > > Rajini > > > > > > > > > > > > > -- > > > > *Dongjin Lee* > > > > > > > > *A hitchhiker in the mathematical world.* > > > > > > > > *github: <http://goog_969573159/>github.com/dongjinleekr > > > > <http://github.com/dongjinleekr>linkedin: > > > kr.linkedin.com/in/dongjinleekr > > > > <http://kr.linkedin.com/in/dongjinleekr>slideshare: > > > www.slideshare.net/dongjinleekr > > > > <http://www.slideshare.net/dongjinleekr>* > > > > > > > > > > > > > -- > > Senior Software Engineer, Lightbend, Inc. > > > > <http://lightbend.com> > > > > @seg1o <https://twitter.com/seg1o>, in/seanaglover > > <https://www.linkedin.com/in/seanaglover/> > > > -- *Jeff Widman* jeffwidman.com <http://www.jeffwidman.com/> | 740-WIDMAN-J (943-6265) <><

Re: KIP-250 Add Support for Quorum-based Producer Acknowledgment

nt of P999 is significant with very stable latency. > > Additionally, I have a proposal to achieve a similar data durability > > as with the insync.replicas-based acknowledgment through LEO-based > > leader election. > > > > https://cwiki.apache.org/confluence/display/

Re: [DISCUSS] KIP-211: Revise Expiration Semantics of Consumer Group Offsets

E4Nj- > kjJc7uSVcviKUc=JkzH_2jfSMhCUPMk3rUasrjDAId6xbAEmX7_shSYdU4= > UBu7D2Obulg0fterYxL5m8xrDWkF_O2kGlygTCWsfFc= > > > Your feedback is welcome! > > Thanks. > --Vahid > > > > > > -- *Jeff Widman* jeffwidman.com <http://www.jeffwidman.com/> | 740-WIDMAN-J (943-6265) <><

Re: [DISCUSS] Release Plan for 1.0.1

s such, with the appropriate priority, and b) make sure > anything > > marked for 1.0.1 should really be there. > > > > Once people have taken a pass we can work on a VOTE thread and getting > any > > outstanding PRs reviewed. > > > > -Ewen > > > > > > -- > -- Guozhang > -- *Jeff Widman* jeffwidman.com <http://www.jeffwidman.com/> | 740-WIDMAN-J (943-6265) <><

Re: Kafka KIP meeting on Apr. 9 at 9:00am PDT

Hi, Everyone, > >> > >> We plan to have a Kafka KIP meeting this coming Monday (Apr. 9) at > 9:00am > >> PDT. If you plan to attend but haven't received an invite, please let me > >> know. The following is the agenda. > >> > >> Agenda: > >> KIP-253: Partition expansion > >> > >> Thanks, > >> > >> Jun > >> > >> > >> > >> > >> > > -- *Jeff Widman* jeffwidman.com <http://www.jeffwidman.com/> | 740-WIDMAN-J (943-6265) <><

Re: [DISCUSS] KIP-272: Add API version tag to broker's RequestsPerSec metric

I agree with Allen. Go with the intuitive name, even if it means not deprecating. The impact of breakage here is small since it only breaks monitoring and the folks who watch their dashboards closely are the ones likely to read the release notes carefully and see this change. On Wed, Mar 21,

[DISCUSS] KIP-592: MirrorMaker should replicate topics from earliest

https://cwiki.apache.org/confluence/display/KAFKA/KIP-592%3A+MirrorMaker+should+replicate+topics+from+earliest It's a relatively minor change, only one line of code. :-D -- *Jeff Widman* jeffwidman.com <http://www.jeffwidman.com/> | 740-WIDMAN-J (943-6265) <><

Re: [DISCUSS] KIP-592: MirrorMaker should replicate topics from earliest

vailable? > > > -Matthias > > > On 4/10/20 9:56 PM, Jeff Widman wrote: > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-592%3A+MirrorMaker+should+replicate+topics+from+earliest > > > > It's a relatively minor change, only one line of code. :-D > > > > > > > > -- *Jeff Widman* jeffwidman.com <http://www.jeffwidman.com/> | 740-WIDMAN-J (943-6265) <><

[VOTE] KIP-592: Kafka MirrorMaker 1 should replicate topics from "earliest"

Hi all, I'd like to start a vote on KIP-592: https://cwiki.apache.org/confluence/display/KAFKA/KIP-592%3A+MirrorMaker+1.0+should+replicate+topics+from+earliest Thanks -- *Jeff Widman* jeffwidman.com <http://www.jeffwidman.com/> | 740-WIDMAN-J (943-6265) <><

Re: [DISCUSS] Kafka 3.0

> > is my KIP-604: Remove ZooKeeper Flags from the Administrative > > Tools. > > > > The > > > > > > other is Boyang's KIP-590: Redirect ZK Mutation Protocols to the > > > > > > Controller. I think it's time to start thinking about Ka

Re: [DISCUSS] KIP-592: MirrorMaker should replicate topics from earliest

aking MM2 use "latest" in "legacy mode", and > leave MM1 as it is? (Just thinking out loud.) > > Ryanne > > On Mon, Apr 27, 2020 at 12:39 PM Jeff Widman wrote: > > > Good questions: > > > > > > *I agree that `auto.offset.reset=&quo

[jira] [Commented] (KAFKA-3806) Adjust default values of log.retention.hours and offsets.retention.minutes

[ https://issues.apache.org/jira/browse/KAFKA-3806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15838918#comment-15838918 ] Jeff Widman commented on KAFKA-3806: Too often people not super familiar with Kafka don't realize

[jira] [Commented] (KAFKA-2000) Delete consumer offsets from kafka once the topic is deleted

[ https://issues.apache.org/jira/browse/KAFKA-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832378#comment-15832378 ] Jeff Widman commented on KAFKA-2000: If neither of them is interested, I'm happy to cleanup

[jira] [Comment Edited] (KAFKA-2000) Delete consumer offsets from kafka once the topic is deleted

[ https://issues.apache.org/jira/browse/KAFKA-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832378#comment-15832378 ] Jeff Widman edited comment on KAFKA-2000 at 1/20/17 8:45 PM: - If neither

[jira] [Commented] (KAFKA-4682) Committed offsets should not be deleted if a consumer is still active

[ https://issues.apache.org/jira/browse/KAFKA-4682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832697#comment-15832697 ] Jeff Widman commented on KAFKA-4682: Now that consumers have background heartbeat thread, it should

[jira] [Comment Edited] (KAFKA-4682) Committed offsets should not be deleted if a consumer is still active

[ https://issues.apache.org/jira/browse/KAFKA-4682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832697#comment-15832697 ] Jeff Widman edited comment on KAFKA-4682 at 1/21/17 1:08 AM: - Now

[jira] [Commented] (KAFKA-4517) Remove kafka-consumer-offset-checker.sh script since already deprecated in Kafka 9

[ https://issues.apache.org/jira/browse/KAFKA-4517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836818#comment-15836818 ] Jeff Widman commented on KAFKA-4517: So what are the next actions here since I've got an outstanding

[jira] [Commented] (KAFKA-4668) Mirrormaker should default to auto.offset.reset=earliest

[ https://issues.apache.org/jira/browse/KAFKA-4668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827769#comment-15827769 ] Jeff Widman commented on KAFKA-4668: Also KAFKA-3848 > Mirrormaker should defa

[jira] [Created] (KAFKA-4668) Mirrormaker should default to auto.offset.reset=earliest

Jeff Widman created KAFKA-4668: -- Summary: Mirrormaker should default to auto.offset.reset=earliest Key: KAFKA-4668 URL: https://issues.apache.org/jira/browse/KAFKA-4668 Project: Kafka Issue

[jira] [Commented] (KAFKA-2410) Implement "Auto Topic Creation" client side and remove support from Broker side

[ https://issues.apache.org/jira/browse/KAFKA-2410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15824382#comment-15824382 ] Jeff Widman commented on KAFKA-2410: I strongly support removing it from the Broker. But does it need

[jira] [Commented] (KAFKA-3853) Report offsets for empty groups in ConsumerGroupCommand

[ https://issues.apache.org/jira/browse/KAFKA-3853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828620#comment-15828620 ] Jeff Widman commented on KAFKA-3853: Thanks again for this! Really looking forward to it. > Rep

[jira] [Commented] (KAFKA-4095) When a topic is deleted and then created with the same name, 'committed' offsets are not reset

[ https://issues.apache.org/jira/browse/KAFKA-4095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15889093#comment-15889093 ] Jeff Widman commented on KAFKA-4095: Wasn't this solved by KAFKA-2000? At least for the new consumers

[jira] [Commented] (KAFKA-2076) Add an API to new consumer to allow user get high watermark of partitions.

[ https://issues.apache.org/jira/browse/KAFKA-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15657823#comment-15657823 ] Jeff Widman commented on KAFKA-2076: It'd be great to see this move forward. > Add an API to

[jira] [Commented] (KAFKA-1941) Timeout connections in the clients

[ https://issues.apache.org/jira/browse/KAFKA-1941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15674679#comment-15674679 ] Jeff Widman commented on KAFKA-1941: Shouldn't this be closed? I think it was handled in KAFKA-1282

[jira] [Comment Edited] (KAFKA-2076) Add an API to new consumer to allow user get high watermark of partitions.

[ https://issues.apache.org/jira/browse/KAFKA-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15658812#comment-15658812 ] Jeff Widman edited comment on KAFKA-2076 at 11/12/16 2:22 AM: -- Thanks

[jira] [Comment Edited] (KAFKA-2076) Add an API to new consumer to allow user get high watermark of partitions.

[ https://issues.apache.org/jira/browse/KAFKA-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15658812#comment-15658812 ] Jeff Widman edited comment on KAFKA-2076 at 11/12/16 2:21 AM: -- Thanks

[jira] [Commented] (KAFKA-2076) Add an API to new consumer to allow user get high watermark of partitions.

[ https://issues.apache.org/jira/browse/KAFKA-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15658812#comment-15658812 ] Jeff Widman commented on KAFKA-2076: Thanks [~becket_qin], BTW great presentation at the Kafka meetup

[jira] [Commented] (KAFKA-4254) Questionable handling of unknown partitions in KafkaProducer

[ https://issues.apache.org/jira/browse/KAFKA-4254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15575850#comment-15575850 ] Jeff Widman commented on KAFKA-4254: Does this affect 10.0 or just 10.1? > Questionable handl

[jira] [Commented] (KAFKA-3853) Report offsets for empty groups in ConsumerGroupCommand

[ https://issues.apache.org/jira/browse/KAFKA-3853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736740#comment-15736740 ] Jeff Widman commented on KAFKA-3853: Any idea if this will land in 0.10.2? > Report offsets for em

[jira] [Commented] (KAFKA-3135) Unexpected delay before fetch response transmission

[ https://issues.apache.org/jira/browse/KAFKA-3135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15750082#comment-15750082 ] Jeff Widman commented on KAFKA-3135: Can the tags on this issue be updated to note that it applies

[jira] [Created] (KAFKA-4519) Delete old unused branches in git repo

Jeff Widman created KAFKA-4519: -- Summary: Delete old unused branches in git repo Key: KAFKA-4519 URL: https://issues.apache.org/jira/browse/KAFKA-4519 Project: Kafka Issue Type: Task

[jira] [Created] (KAFKA-4517) Remove shell scripts deprecated in Kafka 9

Jeff Widman created KAFKA-4517: -- Summary: Remove shell scripts deprecated in Kafka 9 Key: KAFKA-4517 URL: https://issues.apache.org/jira/browse/KAFKA-4517 Project: Kafka Issue Type: Task

[jira] [Updated] (KAFKA-4517) Remove kafka-consumer-offset-checker.sh script since already deprecated in Kafka 9

[ https://issues.apache.org/jira/browse/KAFKA-4517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Widman updated KAFKA-4517: --- Summary: Remove kafka-consumer-offset-checker.sh script since already deprecated in Kafka 9

[jira] [Updated] (KAFKA-4517) Remove kafka-consumer-offset-checker.sh script since deprecated in Kafka 9

[ https://issues.apache.org/jira/browse/KAFKA-4517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Widman updated KAFKA-4517: --- Summary: Remove kafka-consumer-offset-checker.sh script since deprecated in Kafka 9 (was: Remove

[jira] [Created] (KAFKA-4621) Fetch Request V3 docs list max_bytes twice

Jeff Widman created KAFKA-4621: -- Summary: Fetch Request V3 docs list max_bytes twice Key: KAFKA-4621 URL: https://issues.apache.org/jira/browse/KAFKA-4621 Project: Kafka Issue Type: Bug

[jira] [Commented] (KAFKA-4621) Fetch Request V3 docs list max_bytes twice

[ https://issues.apache.org/jira/browse/KAFKA-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15819844#comment-15819844 ] Jeff Widman commented on KAFKA-4621: Yeah, I saw that... perhaps the descriptions could also

[jira] [Commented] (KAFKA-1817) AdminUtils.createTopic vs kafka-topics.sh --create with partitions

[ https://issues.apache.org/jira/browse/KAFKA-1817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15803174#comment-15803174 ] Jeff Widman commented on KAFKA-1817: So can this be closed? > AdminUtils.createTopic vs ka

[jira] [Commented] (KAFKA-3135) Unexpected delay before fetch response transmission

[ https://issues.apache.org/jira/browse/KAFKA-3135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15788686#comment-15788686 ] Jeff Widman commented on KAFKA-3135: It's not currently a critical issue for my company. Typically

[jira] [Commented] (KAFKA-3297) More optimally balanced partition assignment strategy (new consumer)

[ https://issues.apache.org/jira/browse/KAFKA-3297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786986#comment-15786986 ] Jeff Widman commented on KAFKA-3297: [~ewencp] KIP-54 tackles both sticky re-assignments and a more

[jira] [Commented] (KAFKA-242) Subsequent calls of ConsumerConnector.createMessageStreams cause Consumer offset to be incorrect

[ https://issues.apache.org/jira/browse/KAFKA-242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15765835#comment-15765835 ] Jeff Widman commented on KAFKA-242: --- Does this bug still exist in 0.10? > Subsequent ca

[jira] [Commented] (KAFKA-2196) remove roundrobin identical topic constraint in consumer coordinator

[ https://issues.apache.org/jira/browse/KAFKA-2196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15766440#comment-15766440 ] Jeff Widman commented on KAFKA-2196: What version did this land in? > remove roundrobin identi

[jira] [Commented] (KAFKA-2019) RoundRobinAssignor clusters by consumer

[ https://issues.apache.org/jira/browse/KAFKA-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15766498#comment-15766498 ] Jeff Widman commented on KAFKA-2019: As noted in KIP-54 (https://cwiki.apache.org/confluence/display

[jira] [Commented] (KAFKA-2172) Round-robin partition assignment strategy too restrictive

[ https://issues.apache.org/jira/browse/KAFKA-2172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15766445#comment-15766445 ] Jeff Widman commented on KAFKA-2172: Should this be marked as resolved? KAFKA-2196 added

[jira] [Commented] (KAFKA-3297) More optimally balanced partition assignment strategy (new consumer)

[ https://issues.apache.org/jira/browse/KAFKA-3297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15766465#comment-15766465 ] Jeff Widman commented on KAFKA-3297: Was this KIP ever voted on? I see there's only a handful

[jira] [Comment Edited] (KAFKA-3297) More optimally balanced partition assignment strategy (new consumer)

[ https://issues.apache.org/jira/browse/KAFKA-3297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15766465#comment-15766465 ] Jeff Widman edited comment on KAFKA-3297 at 12/21/16 8:38 AM: -- Is this being

[jira] [Commented] (KAFKA-2331) Kafka does not spread partitions in a topic among all consumers evenly

[ https://issues.apache.org/jira/browse/KAFKA-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15766509#comment-15766509 ] Jeff Widman commented on KAFKA-2331: Isn't this what round robin partitioning strategy was trying

[jira] [Commented] (KAFKA-3853) Report offsets for empty groups in ConsumerGroupCommand

[ https://issues.apache.org/jira/browse/KAFKA-3853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15759588#comment-15759588 ] Jeff Widman commented on KAFKA-3853: Did the KIP get the missing vote required to pass? > Rep

[jira] [Commented] (KAFKA-2273) Add rebalance with a minimal number of reassignments to server-defined strategy list

[ https://issues.apache.org/jira/browse/KAFKA-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15768710#comment-15768710 ] Jeff Widman commented on KAFKA-2273: What was the outcome of the vote? > Add rebalance with a mini

[jira] [Commented] (KAFKA-4858) Long topic names created using old kafka-topics.sh can prevent newer brokers from joining any ISRs

[ https://issues.apache.org/jira/browse/KAFKA-4858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15929421#comment-15929421 ] Jeff Widman commented on KAFKA-4858: I'd say if you patch it so that replication doesn't break, that's

[jira] [Comment Edited] (KAFKA-3042) updateIsr should stop after failed several times due to zkVersion issue

[ https://issues.apache.org/jira/browse/KAFKA-3042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959479#comment-15959479 ] Jeff Widman edited comment on KAFKA-3042 at 4/6/17 6:16 PM: We hit

[jira] [Commented] (KAFKA-3042) updateIsr should stop after failed several times due to zkVersion issue

[ https://issues.apache.org/jira/browse/KAFKA-3042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959479#comment-15959479 ] Jeff Widman commented on KAFKA-3042: We hit this on 0.10.0.1. Root cause was a really long zookeeper

[jira] [Commented] (KAFKA-3042) updateIsr should stop after failed several times due to zkVersion issue

[ https://issues.apache.org/jira/browse/KAFKA-3042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15975603#comment-15975603 ] Jeff Widman commented on KAFKA-3042: [~lindong] thanks for the offer to help and sorry for the slow

[jira] [Commented] (KAFKA-4858) Long topic names created using old kafka-topics.sh can prevent newer brokers from joining any ISRs

[ https://issues.apache.org/jira/browse/KAFKA-4858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903928#comment-15903928 ] Jeff Widman commented on KAFKA-4858: My understanding was that the general expectation was that you

[jira] [Updated] (KAFKA-4790) Kafka cannot recover after a disk full

[ https://issues.apache.org/jira/browse/KAFKA-4790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Widman updated KAFKA-4790: --- Fix Version/s: (was: 0.10.2.1) > Kafka cannot recover after a disk f

[jira] [Commented] (KAFKA-4858) Long topic names created using old kafka-topics.sh can prevent newer brokers from joining any ISRs

[ https://issues.apache.org/jira/browse/KAFKA-4858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905917#comment-15905917 ] Jeff Widman commented on KAFKA-4858: I saw the PR ignores those topics... is this a silent failure

[jira] [Updated] (KAFKA-2273) KIP-54: Add rebalance with a minimal number of reassignments to server-defined strategy list

[ https://issues.apache.org/jira/browse/KAFKA-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Widman updated KAFKA-2273: --- Fix Version/s: 0.11.0.0 > KIP-54: Add rebalance with a minimal number of reassignments to >

[jira] [Commented] (KAFKA-2273) KIP-54: Add rebalance with a minimal number of reassignments to server-defined strategy list

[ https://issues.apache.org/jira/browse/KAFKA-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15898678#comment-15898678 ] Jeff Widman commented on KAFKA-2273: This KIP was accepted, so I added it to the 0.11 milestone >

[jira] [Commented] (KAFKA-4677) Avoid unnecessary task movement across threads during rebalance

[ https://issues.apache.org/jira/browse/KAFKA-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890935#comment-15890935 ] Jeff Widman commented on KAFKA-4677: Is KIP-54 related since it also implements a sticky partition

[jira] [Resolved] (KAFKA-5299) MirrorMaker with New.consumer doesn't consume message from multiple topics whitelisted

[ https://issues.apache.org/jira/browse/KAFKA-5299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Widman resolved KAFKA-5299. Resolution: Cannot Reproduce > MirrorMaker with New.consumer doesn't consume message from multi

  1   2   >