Re: [VOTE] KIP-55: Secure quotas for authenticated users

2016-06-09 Thread Tom Crayford
+1 (non-binding) On Wed, Jun 8, 2016 at 8:56 PM, Rajini Sivaram wrote: > I would like to initiate the vote for KIP-55. > > The KIP details are here: KIP-55: Secure quotas for authenticated users > < >

Re: [VOTE] KIP-4 Create Topics Schema

2016-06-20 Thread Tom Crayford
+1 (non-binding) On Mon, Jun 20, 2016 at 8:07 PM, Harsha wrote: > +1 (binding) > -Harsha > > On Mon, Jun 20, 2016, at 11:33 AM, Ismael Juma wrote: > > +1 (binding) > > > > On Mon, Jun 20, 2016 at 8:27 PM, Dana Powers > > wrote: > > > > > +1 -- thanks for

Re: [VOTE] KIP-35: Retrieve protocol version - Round 2

2016-04-21 Thread Tom Crayford
+1 (non binding) On Friday, 22 April 2016, Jason Gustafson wrote: > +1 (non-binding) > > On Thu, Apr 21, 2016 at 5:26 PM, Ewen Cheslack-Postava > > wrote: > > > +1 > > > > On Thu, Apr 21, 2016 at 5:25 PM, Guozhang Wang

Default log.roll.ms to the default retention or the retention in that topic

2016-05-20 Thread Tom Crayford
ll request for this change. Thanks Tom Crayford Heroku Kafka

Re: [ANNOUCE] Apache Kafka 0.10.0.0 Released

2016-05-24 Thread Tom Crayford
Stepanov, Ishita Mandhan, > > > Ismael Juma, Jaikiran Pai, Jakub Nowak, James Cheng, Jason Gustafson, > > > Jay Kreps, Jeff Klukas, Jeremy Custenborder, Jesse Anderson, jholoman, > > > Jiangjie Qin, Jin Xing, jinxing, Jonathan Bond, Jun Rao, Ján Koščo, > > > Kaufman

Re: [DISCUSS] KIP-58 - Make Log Compaction Point Configurable

2016-05-20 Thread Tom Crayford
ally really care. The current defaults seem to be doing well here (barring the above note about log compaction size), and any future changes here should keep this up. Thanks Tom Crayford Heroku Kafka On Fri, May 20, 2016 at 4:48 AM, Jay Kreps <j...@confluent.io> wrote: > Hey Gwen, > &g

Re: [VOTE] 0.10.0.0 RC4

2016-05-12 Thread Tom Crayford
We've started running our usual suite of performance tests against Kafka 0.10.0.0 RC. These tests orchestrate multiple consumer/producer machines to run a fairly normal mixed workload of producers and consumers (each producer/consumer are just instances of kafka's inbuilt consumer/producer perf

Re: [VOTE] 0.10.0.0 RC4

2016-05-12 Thread Tom Crayford
Yep, confirm. On Thu, May 12, 2016 at 9:37 PM, Gwen Shapira <g...@confluent.io> wrote: > Just to confirm: > You tested both versions with plain text and saw no performance drop? > > > On Thu, May 12, 2016 at 1:26 PM, Tom Crayford <tcrayf...@heroku.com> > wrote:

Perf producer/consumers for compacted topics

2016-05-17 Thread Tom Crayford
or should do there? We're happy writing the code here, and want to continue contributing back, I'd just love a hand thinking about what perf tests for compacted topics should look like. Thanks Tom Crayford Heroku Kafka

Re: [VOTE] 0.10.0.0 RC4

2016-05-15 Thread Tom Crayford
> https://github.com/apache/kafka/blob/0.10.0/docs/upgrade.html#L67 > > Please feel free to suggest improvements. > > Thanks, > Ismael > > On Sun, May 15, 2016 at 6:39 PM, Tom Crayford <tcrayf...@heroku.com> > wrote: > > > I've been digging into this some m

Re: [VOTE] 0.10.0.0 RC4

2016-05-15 Thread Tom Crayford
this kind of change, you can skip the JIRA, just prefix the PR title with > `MINOR:`. > > Thanks, > Ismael > > On Sun, May 15, 2016 at 9:17 PM, Tom Crayford <tcrayf...@heroku.com> > wrote: > > > How about this? > > > > Note: Due to the additional

Re: [VOTE] 0.10.0.0 RC4

2016-05-13 Thread Tom Crayford
Ok, I don't seem to be able to file a new Jira issue at all. Can somebody check my permissions on Jira? My user is `tcrayford-heroku` Tom Crayford Heroku Kafka On Fri, May 13, 2016 at 12:24 AM, Jun Rao <j...@confluent.io> wrote: > Tom, > > We don't have a CSV metrics reporter

Re: [VOTE] 0.10.0.0 RC4

2016-05-13 Thread Tom Crayford
ontributor in JIRA and you should be able to file a ticket now. > > Thanks, > Ismael > > On Fri, May 13, 2016 at 12:17 PM, Tom Crayford <tcrayf...@heroku.com> > wrote: > > > Ok, I don't seem to be able to file a new Jira issue at all. Can somebody > > check my permi

Re: Perf producer/consumers for compacted topics

2016-05-18 Thread Tom Crayford
y-percent > kafka.log:type=LogCleaner,name=max-buffer-utilization-percent > kafka.log:type=LogCleaner,name=max-clean-time-secs > kafka.log:type=LogCleanerManager,name=max-dirty-percent > > Manikumar > > On Tue, May 17, 2016 at 8:45 PM, Tom Crayford <tcrayf...@heroku.com>

Re: [VOTE] 0.10.0.0 RC4

2016-05-13 Thread Tom Crayford
n general. > > Thanks, > > Jiangjie (Becket) Qin > > On Fri, May 13, 2016 at 6:19 AM, Tom Crayford <tcrayf...@heroku.com> > wrote: > > > Ismael, > > > > Thanks. I'm writing up an issue with some new findings since yesterday > > right now. > >

Re: [VOTE] 0.10.0.0 RC4

2016-05-12 Thread Tom Crayford
> If you have more of them, could you create a jira and attach your findings > there? Yep. I only wrote the email because JIRA was in lockdown mode and I couldn't create new issues. > > Thanks, > > Jun > > > > On Thu, May 12, 2016 at 1:26 PM, Tom Crayford <tcray

Re: [VOTE] 0.10.0.0 RC4

2016-05-15 Thread Tom Crayford
; > wrote: > >> > >> > Tom, > >> > > >> > Maybe it is mentioned and I missed. I am wondering if you see > performance > >> > degradation on the consumer side when TLS is used? This could help us > >> > understand whether the

[VOTE] KIP-58 - Make Log Compaction Point Configurable

2016-05-25 Thread Tom Crayford
t add more tuning". I can't however, come up with any better mechanism (that doesn't require tuning at all) without gross interactions with consumer offset storage, so I remain a +1 here. Thanks Tom Crayford Heroku Kafka On Wednesday, 25 May 2016, Ewen Cheslack-Postava <e...@confluent.io &

Re: Kafka server contains messages but consumer cannot receive any(Producer used compression.type = snappy)

2016-05-25 Thread Tom Crayford
Which consumer are you using? Can you see it connecting to the broker in the broker logs? I'd recommend putting your configs for producer, consumer and broker in a reply to assist debugging. Also please attach any relevant code or log files. Thanks Tom Crayford Heroku Kafka On Wednesday, 25 May

Re: kafka broker is dropping the messages after acknowledging librdkafka

2016-08-11 Thread Tom Crayford
Are you running with unclean leader election on? Are you setting min in sync replicas at all? Can you attach controller and any other logs from the brokers you have? They would be crucial in debugging this kind of issue. Thanks Tom Crayford Heroku Kafka On Thursday, 11 August 2016, Mazhar

Re: [DISCUSS] Optimise memory used by replication process by using adaptive fetch message size

2016-07-21 Thread Tom Crayford
etchResponse and FetchResponseSend (in case you want some pointers to some code). I may be missing something here, but there seems to be a deeper issue here, Tom Crayford Heroku Kafka On Thu, Jul 21, 2016 at 10:49 AM, Andrey L. Neporada < anepor...@yandex-team.ru> wrote: > Hi all! > > We not

Re: [VOTE] KIP-55: Secure quotas for authenticated users

2016-07-07 Thread Tom Crayford
+1 (non-binding) On Thursday, 7 July 2016, Rajini Sivaram wrote: > I would like to initiate voting for KIP-55 ( > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-55%3A+Secure+Quotas+for+Authenticated+Users > ). > Since the KIP has changed quite a lot since

Re: [VOTE] 0.10.0.1 RC2

2016-08-05 Thread Tom Crayford
Heroku has tested this using the same performance testing setup we used to evaluate the impact of 0.9 -> 0.10 (see https://engineering. heroku.com/blogs/2016-05-27-apache-kafka-010-evaluating- performance-in-distributed-systems/). We see no issues at all with them, so +1 (non-binding) from here.

Re: [DISCUSS] KIP-72 Allow Sizing Incoming Request Queue in Bytes

2016-08-05 Thread Tom Crayford
This makes good sense to me, and seems to have very low amounts of downside with large amounts of upside. +1 On Thursday, 4 August 2016, radai wrote: > Hello, > > I'd like to initiate a discussion about > https://cwiki.apache.org/confluence/display/KAFKA/KIP- >

Re: [DISCUSS] KIP-55: Secure quotas for authenticated users

2016-08-09 Thread Tom Crayford
Seeing as voting passed on this, can somebody with access update the wiki? Is there code for this KIP in a PR somewhere that needs merging? Thanks Tom Crayford Heroku Kafka On Friday, 1 July 2016, Rajini Sivaram <rajinisiva...@googlemail.com> wrote: > Thank you, Jun. > > Hi all,

Re: [VOTE] 0.10.2.0 RC0

2017-02-03 Thread Tom Crayford
present in 0.10.2.0 RC0 To quote KIP-13 on this matter: > Note that in case of producers, we should be careful about retaining references to large Message bodies because we could easily exhaust broker memory if parking hundreds of requests. Tom Crayford On Thu, Feb 2, 2017 at 8:01 PM, Math

Re: [VOTE] KIP-98: Exactly Once Delivery and Transactional Messaging

2017-02-02 Thread Tom Crayford
-1 (non-binding) I've been slow at keeping up with the KIP and the discussion thread. This is an exciting and quite complex new feature, which provides great new functionality. There's a thing I noticed missing from the KIP that's present in the google doc - the doc talks about ACLs for

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

2017-02-02 Thread Tom Crayford
I said this in the voting thread, but can the authors include a section about new ACLs if there are going to be ACLs for TransactionalId. It's mentioned in the google doc, but I think new ACLs should be in a KIP directly. On Thu, Feb 2, 2017 at 2:42 PM, Ismael Juma wrote: >

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

2017-02-06 Thread Tom Crayford
I think the updated wiki page makes sense with respect to ACLs, there seems to be little potential for abuse there (other than the noted and known issues). I am going to note that this is a major complexity increase for Kafka, and that I'm concerned about performance impact (the JVM is quite…

Re: [DISCUSS] KIP-118: Drop Support for Java 7 in Kafka 0.11

2017-02-04 Thread Tom Crayford
This is great, thanks. On Sat, Feb 4, 2017 at 12:22 PM, Dongjin Lee wrote: > It also makes sense to me. Thanks for your great work, Ismael. > > Add to this, it seems like there should be some following work on utilizing > some parts of the code by applying Java 8's features.

Re: [VOTE] KIP-98: Exactly Once Delivery and Transactional Messaging

2017-02-10 Thread Tom Crayford
od point. During the discussion, it was agreed that > > changes > > > > to > > > > > public interfaces (message format, protocol, ACLs, etc.) would be > > > copied > > > > to > > > > > the wiki once the things had settled down, but it looks like that > > &g

Re: [VOTE] KIP-118: Drop Support for Java 7 in Kafka 0.11

2017-02-09 Thread Tom Crayford
+1 (non-binding) On Thu, Feb 9, 2017 at 6:37 PM, Apurva Mehta wrote: > +1 (non-binding) > > On Thu, Feb 9, 2017 at 10:33 AM, Colin McCabe wrote: > > > +1 (non-binding) > > > > > > > > On Thu, Feb 9, 2017, at 07:31, Ismael Juma wrote: > > > Hi everyone,

Re: [kafka-clients] [VOTE] 0.10.2.0 RC2

2017-02-16 Thread Tom Crayford
+1 (non-binding) I didn't explicitly state my voting status in my previous comment, sorry. On Thu, Feb 16, 2017 at 1:59 PM, Rajini Sivaram wrote: > +1 (non-binding) > > Ran quick start and some security tests on binary, checked source build and > tests. > > Thank you,

Re: [VOTE] 0.10.2.0 RC2

2017-02-15 Thread Tom Crayford
Heroku tested this with our usual round of performance benchmarks, and there seem to be no notable regressions in this RC that we can see (for a sample on earlier regressions we found using these benchmarks during the 0.10.0.0 release,

Re: [VOTE] KIP-74: Add FetchResponse size limit in bytes

2016-08-19 Thread Tom Crayford
+1 (non binding) On Fri, Aug 19, 2016 at 6:20 AM, Manikumar Reddy wrote: > +1 (non-binding) > > This feature help us control memory footprint and allows consumer to > progress on fetching large messages. > > On Fri, Aug 19, 2016 at 10:32 AM, Gwen Shapira

Re: Change Apache Kafka properties dynamically without restarting servers using Archaius

2016-08-16 Thread Tom Crayford
ny consumer lag > This is simply the number of client objects you have running. It's under your control. Note that "did not result in lost or duplicate message" is not entirely possible in the presence of failure. You have to pick one and optimize for that (Kafka's docs often refer to

Re: Prevent broker from leading topic partitions

2016-08-22 Thread Tom Crayford
Hi, I don't think I understand *why* you need this. Kafka is by default a distributed HA system that balances data and leadership over nodes. Why do you need to change this? You could accomplish something like this with mirror maker, that may make more sense. Thanks Tom Crayford Heroku Kafka

Re: [VOTE] KIP-119: Drop Support for Scala 2.10 in Kafka 0.11

2017-02-28 Thread Tom Crayford
+1 (non-binding) On Tue, 28 Feb 2017 at 11:42, Molnár Bálint wrote: > +1 > > 2017-02-28 12:17 GMT+01:00 Dongjin Lee : > > > > > > > +1. > > > > > > > > Best, > > > > Dongjin > > > > > > -- > > > > > > > > > > Dongjin Lee > > > > > > > > > > > >

Re: Plans to improve SSL performance in Kafka, for 0.10.x?

2016-09-05 Thread Tom Crayford
typical for most Kafka usage. That's for both produce and consume traffic. We do see increase amounts of garbage collection under high consumer load with SSL enabled, but nothing too horrible (brokers are well below the SLA of GC to keep their zookeeper sessions alive). Thanks Tom Crayford Heroku

Re: Plans to improve SSL performance in Kafka, for 0.10.x?

2016-09-06 Thread Tom Crayford
On Tue, Sep 6, 2016 at 12:07 PM, Ismael Juma <ism...@juma.me.uk> wrote: > On Tue, Sep 6, 2016 at 11:35 AM, Tom Crayford <tcrayf...@heroku.com> > wrote: > > > Our performance tests were run with the JVM producer and consumer, and we > > didn't notice any real

Re: Plans to improve SSL performance in Kafka, for 0.10.x?

2016-09-06 Thread Tom Crayford
ons for the slowness. >> >> >> Thanks for all your inputs. I was trying to avoid coming up with a >> reproducer which closely matches our application usage scenario (given our >> tight delivery timelines for the product), but from what you say and a >> re

Re: [DISCUSS] Fault injection tests for Kafka

2016-10-05 Thread Tom Crayford
I did some stuff like this recently with simple calls to `tc` (samples that I used were in the README for https://github.com/tylertreat/comcast). The only notable bug I found so far is that if you cut all the kafka nodes entirely off from zookeeper for say, 60 seconds, then reconnect them, the

Re: [VOTE] 0.10.1.0 RC2

2016-10-14 Thread Tom Crayford
Heroku has tested this using the same performance testing setup we used to evaluate the impact of 0.9 -> 0.10 (see https:// engineering.heroku.com/blogs/2016-05-27-apache-kafka-010- evaluating-performance-in-distributed-systems/). We see no issues at all with them on RC2. On Thu, Oct 13, 2016 at

Re: [VOTE] KIP-106 - Default unclean.leader.election.enabled True => False

2017-01-11 Thread Tom Crayford
+1 (non-binding) On Wed, Jan 11, 2017 at 11:12 PM, Stevo Slavić wrote: > +1 (non-binding) > > On Thu, Jan 12, 2017 at 12:11 AM, Guozhang Wang > wrote: > > > +1 > > > > On Wed, Jan 11, 2017 at 12:09 PM, Jeff Widman wrote: > > > > > +1

Re: [VOTE] KIP-96 - Add per partition metrics for in-sync and replica count

2016-12-01 Thread Tom Crayford
+1 (non-binding) On Thu, Dec 1, 2016 at 12:11 AM, Apurva Mehta wrote: > +1 (non-binding) > > On Wed, Nov 30, 2016 at 2:00 PM, Jason Gustafson > wrote: > > > +1. Thanks for the KIP! > > > > On Wed, Nov 30, 2016 at 1:47 PM, Gwen Shapira

Re: [DISCUSS] KIP-109: Old Consumer Deprecation

2017-01-05 Thread Tom Crayford
It's not time for voting, but a huge +1 from me. On Thu, Jan 5, 2017 at 8:25 PM, Vahid S Hashemian wrote: > Thanks Ismael. I added that to the KIP. > > > > > From: Ismael Juma > To: dev@kafka.apache.org > Date: 01/05/2017 12:16 PM >

Re: [VOTE] KIP-103: Separation of Internal and External traffic

2017-01-06 Thread Tom Crayford
+1 (non-binding) On Fri, Jan 6, 2017 at 6:58 PM, Colin McCabe wrote: > Looks good. +1 (non-binding). > > What do you think about changing "protocol label" to "listener key"? > > best, > Colin > > > On Fri, Jan 6, 2017, at 09:23, Neha Narkhede wrote: > > +1 > > > > On Fri,

Re: [DISCUSS] KIP-106 - Change Default unclean.leader.election.enabled from True to False

2017-01-03 Thread Tom Crayford
+1. We've been running it in production for a long time and it's the right default. On Tue, Jan 3, 2017 at 7:17 PM, Ismael Juma wrote: > Thanks for the KIP, +1 from me. > > Ismael > > On 3 Jan 2017 6:54 pm, "Ben Stopford" wrote: > > > Hi All > > > > Please

Re: 0.10.2.0 release planning

2017-01-04 Thread Tom Crayford
Thanks for running the release and being so transparent around plans, it's a great boon to the community. Looking at the plan ( https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+0.10.2.0), I see KIP-4 and KIP-35 under "Open Issues", but not under "Planned KIP Content". Are those KIPs

Re: 0.10.2.0 release planning

2017-01-04 Thread Tom Crayford
ue to be open after > 0.10.2.0 is released. We haven't made much progress on that front during > this release cycle, but I hope we'll do better for the next release cycle. > > Ismael > > On Wed, Jan 4, 2017 at 4:08 PM, Tom Crayford <tcrayf...@heroku.com> wrote: > > > Thank

Re: [VOTE] Vote for KIP-101 - Leader Epochs

2017-01-04 Thread Tom Crayford
+1 On Wed, Jan 4, 2017 at 5:28 PM, Gwen Shapira wrote: > +1 - thanks for tackling those old and painful bugs! > > On Wed, Jan 4, 2017 at 9:24 AM, Ben Stopford wrote: > > Hi All > > > > We’re having some problems with this thread being subsumed by the >

Re: [DISCUSS] ACL operations

2017-04-14 Thread Tom Crayford
Colin, Reminder that ACLs don't just apply to topics, but also to consumer groups and cluster operations. It seems like having two sets of APIs, one of which is (topics + acls) and one of which is just acls is more complex than just having acls. On Fri, Apr 14, 2017 at 12:17 AM, Colin McCabe

Re: [DISCUSS] 0.10.3.0/0.11.0.0 release planning

2017-03-01 Thread Tom Crayford
+1 (non-binding) On Tue, Feb 28, 2017 at 6:56 PM, Apurva Mehta wrote: > +1 (non-binding) for 0.11.0 > > I do agree with Ismael's point that exactly-once should go through one > release of stabilization before bumping the version to 1.0. > > Thanks, > Apurva > > On Mon, Feb

Re: Is Kafka client 0.10.2.1 backward compatible

2017-06-28 Thread Tom Crayford
Kafka promises one thing and one thing only for backwards compatability, which is that brokers with newer versions will always support older clients. The inverse: old brokers with new clients is not true. On Wed, Jun 28, 2017 at 6:18 AM, saurabh mimani wrote: > I am

Re: [DISCUSS] KIP-143: Controller Health Metrics

2017-04-27 Thread Tom Crayford
d it to make it clear. > > Regarding locks, they are removed by KAFKA-5028, as you say. So, if I > understand correctly, you are suggesting an event processing rate metric > with event type as a tag? Onur and Jun, what do you think? > > Ismael > > On Thu, Apr 27, 2017 a

Re: [DISCUSS] KIP-143: Controller Health Metrics

2017-04-27 Thread Tom Crayford
does it spend processing any individual message from the queue, broken down by message type"). These are minor improvements though, the addition of more metrics to the controller is already going to be very helpful. Thanks Tom Crayford Heroku Kafka On Thu, Apr 27, 2017 at 3:10 PM, Ismael

Re: [VOTE] 0.11.0.0 RC0

2017-06-19 Thread Tom Crayford
ature to be perfect in this release. We expect to test this this week though. Given that the blockers fixed between RC0 and RC1 haven't changed much in the areas we tested, I think the positive results here still apply. Thanks Tom Crayford Heroku Kafka On Thu, Jun 8, 2017 at 2:55 PM, Ismael Juma

Re: [VOTE] 0.11.0.0 RC1

2017-06-21 Thread Tom Crayford
/apache/kafka/pull/3398 (the change is very small). Happy to make a jira as well, if that makes sense. Thanks Tom Crayford Heroku Kafka On Tue, Jun 20, 2017 at 8:32 PM, Vahid S Hashemian < vahidhashem...@us.ibm.com> wrote: > Hi Ismael, > > Thanks for running the release. &g

Re: [VOTE] 0.11.0.0 RC1

2017-06-21 Thread Tom Crayford
e though, since it > doesn't really affect the usability of the feature any way. > > Thanks, > Apurva > > On Wed, Jun 21, 2017 at 11:12 AM, Tom Crayford <tcrayf...@heroku.com> > wrote: > > > Hi there, > > > > I'm -1 (non-binding) on shipping this RC. > >

Re: [VOTE] 0.11.0.0 RC1

2017-06-22 Thread Tom Crayford
and shouldn't delay a release documented somewhere? Thanks Tom Crayford Heroku Kafka On Thu, Jun 22, 2017 at 4:45 PM, Ismael Juma <isma...@gmail.com> wrote: > Hi Tom, > > We are going to do another RC to include Apurva's significant performance > improvement when transactions are

Re: [VOTE] 0.11.0.0 RC2

2017-06-23 Thread Tom Crayford
, this is shaping up to be a great release. We're going to continue some further testing, but right now are heading towards a +1. Thanks Tom Crayford Heroku Kafka On Fri, Jun 23, 2017 at 2:36 AM, Ismael Juma <ism...@juma.me.uk> wrote: > A quick note on notable changes since r

Re: [VOTE] 0.11.0.0 RC0

2017-06-22 Thread Tom Crayford
and workloads, but it does change significantly with what hardware you're using. Because our performance tests run across cloud instances where there might be contention and variability, we typically do a multiple runs on different clusters with each setup before reporting results. Thanks Tom Crayford

Re: KIP-162: Enable topic deletion by default

2017-05-26 Thread Tom Crayford
+1 (non-binding) On Fri, May 26, 2017 at 3:38 PM, Damian Guy wrote: > +1 > Also agree with what Ismael said. > > On Fri, 26 May 2017 at 15:26 Ismael Juma wrote: > > > Thanks for the KIP, sounds good to me. One comment: not sure we need to > add > > the

Re: [VOTE] 0.11.0.0 RC1

2017-06-19 Thread Tom Crayford
Is there a summary of which blockers were fixed in RC0 somewhere? On Mon, Jun 19, 2017 at 1:41 PM, Eno Thereska wrote: > +1 (non-binding) passes Kafka Streams tests. > > Thanks, > Eno > > On 19 Jun 2017, at 06:49, Magnus Edenhill wrote: > > > > +1

Re: [VOTE] KIP-143: Controller Health Metrics

2017-05-05 Thread Tom Crayford
+1 (non-binding) On Fri, 5 May 2017 at 00:28, Michael André Pearce < michael.andre.pea...@me.com> wrote: > +1 , really good to see better operational visibility being added to the > broker. > > Sent from my iPhone > > > On 5 May 2017, at 03:34, Ismael Juma wrote: > > > > Hi

[jira] [Commented] (KAFKA-3894) Log Cleaner thread crashes and never restarts

2016-06-23 Thread Tom Crayford (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346463#comment-15346463 ] Tom Crayford commented on KAFKA-3894: - (disclaimer: I work with Tim) It feels like there are a few

[jira] [Commented] (KAFKA-3894) Log Cleaner thread crashes and never restarts

2016-07-14 Thread Tom Crayford (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15376858#comment-15376858 ] Tom Crayford commented on KAFKA-3894: - Jun: #4 seems potentially very complex to me. It also doesn't

[jira] [Commented] (KAFKA-3894) LogCleaner thread crashes if not even one segment can fit in the offset map

2016-08-11 Thread Tom Crayford (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15417675#comment-15417675 ] Tom Crayford commented on KAFKA-3894: - Hi Jun, We're probably going to start on b. for now. I think

[jira] [Created] (KAFKA-3976) Kafka Controller gets stuck, potentially due to zookeeper session id reuse

2016-07-19 Thread Tom Crayford (JIRA)
Tom Crayford created KAFKA-3976: --- Summary: Kafka Controller gets stuck, potentially due to zookeeper session id reuse Key: KAFKA-3976 URL: https://issues.apache.org/jira/browse/KAFKA-3976 Project

[jira] [Created] (KAFKA-3955) Kafka log recovery doesn't truncate logs on non-monotonic offsets, leading to failed broker boot

2016-07-12 Thread Tom Crayford (JIRA)
Tom Crayford created KAFKA-3955: --- Summary: Kafka log recovery doesn't truncate logs on non-monotonic offsets, leading to failed broker boot Key: KAFKA-3955 URL: https://issues.apache.org/jira/browse/KAFKA-3955

[jira] [Created] (KAFKA-3933) Kafka OOM During Log Recovery Due to Leaked Native Memory

2016-07-07 Thread Tom Crayford (JIRA)
Tom Crayford created KAFKA-3933: --- Summary: Kafka OOM During Log Recovery Due to Leaked Native Memory Key: KAFKA-3933 URL: https://issues.apache.org/jira/browse/KAFKA-3933 Project: Kafka Issue

[jira] [Commented] (KAFKA-3933) Kafka OOM During Log Recovery Due to Leaked Native Memory

2016-07-07 Thread Tom Crayford (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366329#comment-15366329 ] Tom Crayford commented on KAFKA-3933: - That makes sense to me. Thanks for the pointer. > Kafka

[jira] [Created] (KAFKA-3937) Kafka Clients Leak Native Memory For Longer Than Needed With Compressed Messages

2016-07-08 Thread Tom Crayford (JIRA)
Tom Crayford created KAFKA-3937: --- Summary: Kafka Clients Leak Native Memory For Longer Than Needed With Compressed Messages Key: KAFKA-3937 URL: https://issues.apache.org/jira/browse/KAFKA-3937 Project

[jira] [Commented] (KAFKA-3933) Kafka OOM During Log Recovery Due to Leaked Native Memory

2016-07-08 Thread Tom Crayford (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367625#comment-15367625 ] Tom Crayford commented on KAFKA-3933: - Ismael: I've pushed the PR here: https://github.com/apache

[jira] [Assigned] (KAFKA-3933) Kafka OOM During Log Recovery Due to Leaked Native Memory

2016-07-07 Thread Tom Crayford (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom Crayford reassigned KAFKA-3933: --- Assignee: Tom Crayford > Kafka OOM During Log Recovery Due to Leaked Native Mem

[jira] [Commented] (KAFKA-3933) Kafka OOM During Log Recovery Due to Leaked Native Memory

2016-07-07 Thread Tom Crayford (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366721#comment-15366721 ] Tom Crayford commented on KAFKA-3933: - Done. I hope to work on a fix tomorrow. > Kafka OOM During

[jira] [Created] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

2016-08-24 Thread Tom Crayford (JIRA)
Tom Crayford created KAFKA-4084: --- Summary: automated leader rebalance causes replication downtime for clusters with too many partitions Key: KAFKA-4084 URL: https://issues.apache.org/jira/browse/KAFKA-4084

[jira] [Created] (KAFKA-4441) Kafka Monitoring is incorrect during rapid topic creation and deletion

2016-11-24 Thread Tom Crayford (JIRA)
Tom Crayford created KAFKA-4441: --- Summary: Kafka Monitoring is incorrect during rapid topic creation and deletion Key: KAFKA-4441 URL: https://issues.apache.org/jira/browse/KAFKA-4441 Project: Kafka

[jira] [Created] (KAFKA-4596) KIP-73 rebalance throttling breaks on plans for specific partitions

2017-01-05 Thread Tom Crayford (JIRA)
Tom Crayford created KAFKA-4596: --- Summary: KIP-73 rebalance throttling breaks on plans for specific partitions Key: KAFKA-4596 URL: https://issues.apache.org/jira/browse/KAFKA-4596 Project: Kafka

[jira] [Created] (KAFKA-5973) ShutdownableThread catching errors can lead to partial hard to diagnose broker failure

2017-09-25 Thread Tom Crayford (JIRA)
Tom Crayford created KAFKA-5973: --- Summary: ShutdownableThread catching errors can lead to partial hard to diagnose broker failure Key: KAFKA-5973 URL: https://issues.apache.org/jira/browse/KAFKA-5973