Build failed in Jenkins: kafka-trunk-jdk11 #710

2019-07-22 Thread Apache Jenkins Server
See Changes: [jason] KAFKA-8454; Add Java AdminClient Interface (KIP-476) (#7087) [jason] KAFKA-8678; Fix leave group protocol bug in throttling and error --

Re: JIRA and KIP contributor permissions

2019-07-22 Thread Matthias J. Sax
Hi Alexandre, I added you to the list of contributors in JIRA, so you can self-assign ticket. However, I did not find any corresponding wiki. Note, that both are independent accounts and you might need to create a wiki account first (and share you ID so we can grant write permission). -Matthias

Jenkins build is back to normal : kafka-2.2-jdk8 #154

2019-07-22 Thread Apache Jenkins Server
See

Build failed in Jenkins: kafka-trunk-jdk8 #3807

2019-07-22 Thread Apache Jenkins Server
See Changes: [jason] KAFKA-8454; Add Java AdminClient Interface (KIP-476) (#7087) [jason] KAFKA-8678; Fix leave group protocol bug in throttling and error --

Re: Stopping All Tasks When a New Connector Added

2019-07-22 Thread Liu Luying
Hi Konstantine, My username is Echolly, with the full name of Luying Liu. Best, Luying Liu From: Konstantine Karantasis Sent: Monday, July 22, 2019 9:30 To: dev@kafka.apache.org Subject: Re: Stopping All Tasks When a New Connector Added Liu feel free to share

Re: Jira Cleanup

2019-07-22 Thread Guozhang Wang
Thanks Sönke! I just made a quick pass on those tickets as well I think your assessments are right. Guozhang On Fri, Jul 19, 2019 at 4:09 AM Sönke Liebau wrote: > All, > > I left a few comments on some old but still open jiras in an attempt to > clean up a little bit. > > Since probably no

Re: [DISCUSS] KIP-444: Augment Metrics for Kafka Streams

2019-07-22 Thread Guozhang Wang
Thanks everyone for your inputs, I've updated the wiki page accordingly. @Bruno: please let me know if you have any further thoughts per my replies above. Guozhang On Mon, Jul 22, 2019 at 6:30 PM Guozhang Wang wrote: > Thanks Boyang, > > I've thought about exposing time via metrics in

Re: [DISCUSS] KIP-444: Augment Metrics for Kafka Streams

2019-07-22 Thread Guozhang Wang
Thanks Boyang, I've thought about exposing time via metrics in Streams. The tricky part though is which layer of time we should expose: right now we have task-level and partition-level stream time (what you suggested), and also some processor internally maintain their own observed time. Today we

Re: [DISCUSS] KIP-447: Producer scalability for exactly once semantics

2019-07-22 Thread Guozhang Wang
On Sat, Jul 20, 2019 at 9:50 AM Boyang Chen wrote: > Thank you Guozhang for the suggestion! I would normally prefer naming a > flag corresponding to its functionality. Seems to me `isolation_level` > makes us another hop on information track. > > Fair enough, let's use a separate flag name then

[jira] [Resolved] (KAFKA-8678) LeaveGroup request getErrorResponse is incorrect on throttle time and error setting

2019-07-22 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-8678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-8678. Resolution: Fixed Fix Version/s: 2.3.1 2.2.2 > LeaveGroup

[jira] [Resolved] (KAFKA-8454) Add Java AdminClient interface

2019-07-22 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-8454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-8454. Resolution: Fixed Fix Version/s: 2.4.0 > Add Java AdminClient interface >

Re: KIP-382 + Kafka Streams Question

2019-07-22 Thread Ryanne Dolan
Hello Adam, thanks for the questions. Yes my organization uses Streams, and yes you can use Streams with MM2/KIP-382, though perhaps not in the way you are describing. The architecture you mention is more "active/standby" than "active/active" IMO. The "secondary" cluster is not being used until a

Re: [DISCUSS] KIP-490: log when consumer groups lose a message because offset has been deleted

2019-07-22 Thread Jose M
Hi Colin, Thanks a lot for your feedback. Please note that I only propose to log when a message is lost this for a set of consumer groups, not as default behaviour for all consumer groups. But in fact, I agree with you that to log a line per message expired can be quite lot, and that is not the

JIRA and KIP contributor permissions

2019-07-22 Thread Alexandre Dupriez
Hello Community, In order to start contributing to Apache Kafka project, could I please request contributor access to JIRA and be granted write permissions to the Kafka wiki? JIRA username: adupriez Committer email: alexandre.dupr...@amazon.com Thank you in advance, Alexandre

KIP-382 + Kafka Streams Question

2019-07-22 Thread Adam Bellemare
Hi Ryanne I have a quick question for you about Active+Active replication and Kafka Streams. First, does your org /do you use Kafka Streams? If not then I think this conversation can end here. ;) Secondly, and for the broader Kafka Dev group - what happens if I want to use Active+Active

[jira] [Created] (KAFKA-8696) Clean up Sum/Count/Total metrics

2019-07-22 Thread John Roesler (JIRA)
John Roesler created KAFKA-8696: --- Summary: Clean up Sum/Count/Total metrics Key: KAFKA-8696 URL: https://issues.apache.org/jira/browse/KAFKA-8696 Project: Kafka Issue Type: Improvement

Re: [VOTE] KIP-488: Clean up Sum,Count,Total Metrics

2019-07-22 Thread John Roesler
Hello all, Thanks for the thoughtful consideration on this KIP. The vote has passed with 3 binding (Guozhang, Bill, and Matthias) votes and 2 non-binding ones (Ryanne and myself). I'll update the PR and submit it for review shortly! Thanks again, -John On Thu, Jul 18, 2019 at 7:55 PM Matthias

Fwd: [DISCUSS] KIP-492 Add java security providers in Kafka Security config

2019-07-22 Thread Sandeep Mopuri
Hi Rajini, Thanks for raising the above questions. Please find the replies below On Wed, Jul 17, 2019 at 2:49 AM Rajini Sivaram wrote: > Hi Sandeep, > > Thanks for the KIP. A few questions below: > >1. Is the main use case for this KIP adding security providers for SSL? >If

Re: [DISCUSS] KIP-490: log when consumer groups lose a message because offset has been deleted

2019-07-22 Thread Colin McCabe
Hi Jose, One issue that I see here is that the number of log messages could be huge. I've seen people create tens of thousands of consumer groups. People can also have settings that create pretty small log files. A message per log file per group could be quite a lot of messages. A log

Re: [VOTE] KIP-455: Create an Administrative API for Replica Reassignment

2019-07-22 Thread Colin McCabe
Hi all, With three non-binding +1 votes from Viktor Somogyi-Vass, Robert Barrett, and George Li, and 3 binding +1 votes from Gwen Shapira, Jason Gustafson, and myself, the vote passes. Thanks, everyone! best, Colin On Fri, Jul 19, 2019, at 08:55, Robert Barrett wrote: > +1 (non-binding).

Re: [DISCUSS] KIP-466: Add support for List serialization and deserialization

2019-07-22 Thread Development
Hey Matthias, It looks a little confusing, but I don’t have enough expertise to judge on the configuration placement. If you think, it is fine I’ll go ahead with this approach. Best, Daniyar Yeralin > On Jul 19, 2019, at 5:49 PM, Matthias J. Sax wrote: > > Good point. > > I guess the

[jira] [Created] (KAFKA-8695) Metrics UnderReplicated and UnderMinSir are diverging when configuration is inconsistent

2019-07-22 Thread Alexandre Dupriez (JIRA)
Alexandre Dupriez created KAFKA-8695: Summary: Metrics UnderReplicated and UnderMinSir are diverging when configuration is inconsistent Key: KAFKA-8695 URL: https://issues.apache.org/jira/browse/KAFKA-8695

Re: Stopping All Tasks When a New Connector Added

2019-07-22 Thread Konstantine Karantasis
Liu feel free to share your jira account id on a separate email, so one of the committers can add you to the project. Then you or someone else will be able to assign this ticket to you. I'll review the fix some time this week. Thanks! Konstantine On Mon, Jul 22, 2019 at 5:13 AM Liu Luying

Re: [DISCUSS] KIP-490: log when consumer groups lose a message because offset has been deleted

2019-07-22 Thread Jose M
Hello, I didn't get any feedback on this small KIP-490 . In summary, I propose a way to be noticed when messages are being removed due to retention policy, without

Metrics UnderReplicated and UnderMinSir are diverging when configuration is inconsistent

2019-07-22 Thread Alexandre Dupriez
Hello Community, I noticed Kafka allows the replication factor of a topic and "min.insync.replicas" to be set such that "min.insync.replicas" > the topic's replication factor. As a consequences, the JMX beans kafka.cluster:type=Partition,name=UnderReplicated and

Build failed in Jenkins: kafka-trunk-jdk11 #709

2019-07-22 Thread Apache Jenkins Server
See Changes: [rajinisivaram] MINOR: kafkatest - adding whitelist for interbroker sasl configs (#7093) -- Started by an SCM change [EnvInject] - Loading node environment

Build failed in Jenkins: kafka-trunk-jdk8 #3806

2019-07-22 Thread Apache Jenkins Server
See Changes: [rajinisivaram] MINOR: kafkatest - adding whitelist for interbroker sasl configs (#7093) -- Started by an SCM change [EnvInject] - Loading node environment