Re: [DISCUSS] (KAFKA-9806) authorize cluster operation when creating internal topics

2020-04-13 Thread Paolo Moriello
Right, the problem in this case is that restoring ACLs to a correct configuration does not fix the problem, because the internal topics remains in a bad state. For instance: 1) user sets insufficient cluster level ACLs (now brokers are not able to communicate) 2) user consumes for the first

Re: [DISCUSS] (KAFKA-9806) authorize cluster operation when creating internal topics

2020-04-09 Thread Paolo Moriello
:08, Paolo Moriello wrote: > > Hi Colin, > > > > Thanks for your interest in this. I agree with you, this change could > break > > compatibility. However, changing the source principal is non trivial in > > this case. In fact, here the problem is not in the internal t

Re: [DISCUSS] (KAFKA-9806) authorize cluster operation when creating internal topics

2020-04-07 Thread Paolo Moriello
rget how we do this exactly, > but we do have some way of doing it). > > best, > Colin > > > On Mon, Apr 6, 2020, at 02:56, Paolo Moriello wrote: > > Hello everybody, > > > > I've opened a Jira to fix a bug on creation of internal topics. This &

[DISCUSS] (KAFKA-9806) authorize cluster operation when creating internal topics

2020-04-06 Thread Paolo Moriello
Hello everybody, I've opened a Jira to fix a bug on creation of internal topics. This happens when the topics are created under insufficient ACLs: eg. __consumer_offset is created but subsequent updateMetadata and leaderIsr requests fail; the topic is than in an inconsistent state and it is

[jira] [Created] (KAFKA-9806) Unable to consume when __consumer_offsets is created under insufficient cluster ACLs

2020-04-02 Thread Paolo Moriello (Jira)
Paolo Moriello created KAFKA-9806: - Summary: Unable to consume when __consumer_offsets is created under insufficient cluster ACLs Key: KAFKA-9806 URL: https://issues.apache.org/jira/browse/KAFKA-9806

Re: [DISCUSS] KIP-579: new exception on min.insync.replicas > replication.factor

2020-04-02 Thread Paolo Moriello
at 9:30 AM, Ismael Juma wrote: > > > > Hi Paolo, > > > > Thanks for the KIP. Why would one want to set min.isr to be higher than > > replication factor even in that case? Mickael's suggestion seems better > to > > me. > > > > Ismael > > > >

Re: [VOTE] KIP-579: new exception on min.insync.replicas > replication.factor

2020-04-02 Thread Paolo Moriello
topic creation seems better. > > On Tue, Mar 31, 2020 at 6:17 AM Paolo Moriello > wrote: > > > Hello, > > > > Thanks to everybody who has given feedback. I've incorporated the > > suggestions and think that this is now ready for a vote. > > > > KIP 579:

[VOTE] KIP-579: new exception on min.insync.replicas > replication.factor

2020-03-31 Thread Paolo Moriello
Hello, Thanks to everybody who has given feedback. I've incorporated the suggestions and think that this is now ready for a vote. KIP 579: https://cwiki.apache.org/confluence/display/KAFKA/KIP-579%3A+new+exception+on+min.insync.replicas+%3E+replication.factor PR:

Re: [DISCUSS] KIP-579: new exception on min.insync.replicas > replication.factor

2020-03-30 Thread Paolo Moriello
e feel free to > consider it. > > Regards, > M. MAnna > > On Mon, 30 Mar 2020 at 12:55, Paolo Moriello > wrote: > > > Hi, > > > > Any other feedback on this before we start the vote? > > > > Paolo > > > > On Fri, 13 Mar 2020 at 1

Re: [DISCUSS] KIP-579: new exception on min.insync.replicas > replication.factor

2020-03-30 Thread Paolo Moriello
Hi, Any other feedback on this before we start the vote? Paolo On Fri, 13 Mar 2020 at 17:28, Paolo Moriello wrote: > Hi Mickael, > > Thanks for your interest in this. The main motivation to NOT make topic > creation fail when this mismatch happens is because at the moment it i

Re: [DISCUSS] KIP-579: new exception on min.insync.replicas > replication.factor

2020-03-13 Thread Paolo Moriello
s where a user would > want to create such a topic. I'm guessing it's probably always by > mistake. > > Let's see what other people think but I think it's worth checking what > needs to be done if we wanted to prevent topics with bogus configs > > On Fri, Mar 13, 202

[DISCUSS] KIP-579: new exception on min.insync.replicas > replication.factor

2020-03-13 Thread Paolo Moriello
Hi, Following this Jira ticket (https://issues.apache.org/jira/browse/KAFKA-4680), I've created a proposal ( https://cwiki.apache.org/confluence/display/KAFKA/KIP-579%3A+new+exception+on+min.insync.replicas+%3E+replication.factor) to add a new exception/error to be used on min.insync.replicas >

Create KIP permissions

2020-03-12 Thread Paolo Moriello
Hi, My name is Paolo Moriello, I'd like to get permissions to create KIPs. My wiki id is: paolomoriello. Thanks, Paolo

[DISCUSS] (KAFKA-9693) Kafka latency spikes caused by log segment flush on roll

2020-03-10 Thread Paolo Moriello
Feb 2020 at 14:42, Paolo Moriello wrote: > Hello, > > > I'm performing an investigation on Kafka latency. During my analysis I was > able to reproduce a scenario in which Kafka latency repeatedly spikes at > constant frequency, for small amounts of time. > > In my tests

[jira] [Created] (KAFKA-9693) Kafka latency spikes caused by log segment flush on roll

2020-03-10 Thread Paolo Moriello (Jira)
Paolo Moriello created KAFKA-9693: - Summary: Kafka latency spikes caused by log segment flush on roll Key: KAFKA-9693 URL: https://issues.apache.org/jira/browse/KAFKA-9693 Project: Kafka

[DISCUSS] KAFKA-4680: min.insync.replica can be set > replication factor

2020-02-27 Thread Paolo Moriello
Hello, I'd like to take up this Jira ticket . This is an old ticket, marked as a Kafka bug. Before moving forward, I'd like to open a discussion on what would be the best approach to take on when doing the validation, as well as discuss about

Jira access

2020-02-27 Thread Paolo Moriello
Hello, Can you please add me as a contributor to Jira? My Jira username: paolomoriello Thanks, Paolo

Kafka latency spikes analysis

2020-02-18 Thread Paolo Moriello
Hello, I'm performing an investigation on Kafka latency. During my analysis I was able to reproduce a scenario in which Kafka latency repeatedly spikes at constant frequency, for small amounts of time. In my tests, in particular, latency could spike every ~2 minutes (dependently on the