Re: [DISCUSS] KIP-382: MirrorMaker 2.0

2018-10-19 Thread Ryanne Dolan
Thanks Harsha. Done. On Fri, Oct 19, 2018 at 1:03 AM Harsha Chintalapani wrote: > Ryanne, >Makes sense. Can you please add this under rejected alternatives so > that everyone has context on why it wasn’t picked. > > Thanks, > Harsha > On Oct 18, 2018, 8:02 AM -0700, Ryanne Dolan , >

[jira] [Created] (KAFKA-7523) TransformerSupplier/ProcessorSupplier enhancements

2018-10-19 Thread Paul Whalen (JIRA)
Paul Whalen created KAFKA-7523: -- Summary: TransformerSupplier/ProcessorSupplier enhancements Key: KAFKA-7523 URL: https://issues.apache.org/jira/browse/KAFKA-7523 Project: Kafka Issue Type:

RE: [DISCUSS] KIP-383 Pluggable interface for SSL Factory

2018-10-19 Thread Harsha Chintalapani
SslFactory is not a public interface for others to use.  EchoServer is internal testing. We should make these as proposed in rejected alternatives to SslFactory and DefaultSslFactory. I don’t see any one using a internal class as public API. -Harsha On Oct 19, 2018, 3:47 PM -0700, Pellerin,

RE: [DISCUSS] KIP-383 Pluggable interface for SSL Factory

2018-10-19 Thread Pellerin, Clement
>> Can you explain why calling SslFactory and DefaultSslFactory cause any >> issues. When you say "calling", I guess you mean "naming". Renaming SslFactory will only cause backwards compatibility issues for applications that refer to it directly. EchoServer is an example, but maybe that is

Re: Java cluster vs Scala cluster

2018-10-19 Thread M. Manna
Not sure I got your question ? Kafka is written mainly in Scala with additional admin functionality and Client API available in Java. There are some subtle differences but they are not specific to Kafka? What is the context of your question? On Fri, 19 Oct 2018 at 21:32, wrote: > Hi, > what

RE: [DISCUSS] KIP-383 Pluggable interface for SSL Factory

2018-10-19 Thread Harsha Chintalapani
Hi,      Overall the KIP looks good to me. "Ideally, the interface should be called SslFactory and the built-in implementation should be called DefaultSslFactory. This was rejected to improve backwards compatibility for applications that call the SslFactory directly.” Can you explain why

Re: [VOTE] KIP-376: Implement AutoClosable on appropriate classes that want to be used in a try-with-resource statement

2018-10-19 Thread Yishun Guan
Hi, since now we have three binding votes(Colin, Matthias, Harsha), if there are no other opinion, I will go ahead conclude this voting thread and accept this KIP. Thanks! - Yishun On Thu, Oct 18, 2018 at 9:10 PM Harsha Chintalapani wrote: > > LGTM. +1 (binding) > > -- > Harsha > > > On Oct 18,

RE: [DISCUSS] KIP-383 Pluggable interface for SSL Factory

2018-10-19 Thread Pellerin, Clement
I have updated the KIP to use a default constructor in the pluggable SSL Factory implementation. I also changed the name of the config to ssl.sslfactory.class and fixed a typo in the constant names. I would like your feedback on this version of the KIP. -Original Message- From:

Re: Throwing away prefetched records optimisation.

2018-10-19 Thread Zahari Dichev
Hi there Matthias, Very useful thoughts indeed. I have considered the exact same approach but what worries me a bit is that I do not think that will certainly solve the issue. Imagine the following situation. You have one consumer that is quite slow so lets say it call poll every 5 seconds, while

Java cluster vs Scala cluster

2018-10-19 Thread abeceda4
Hi, what is difference between class cluster written in java and class cluster written in scala.   Thanks

Re: Frequent under-replicated partitions

2018-10-19 Thread Suman B N
Rate of ingestion is not 150-200rps. Its 150k-200k rps. On Fri, Oct 19, 2018 at 11:12 PM Suman B N wrote: > Team, > We have been observing some partitions being under-replicated. Broker > version 0.10.2.1. Below actions were carried out but in vain: > >- Tried restarting nodes. >- Tried

Re: [VOTE] KIP-228 Negative record timestamp support

2018-10-19 Thread Konstantin Chukhlomin
bump On Tue, Jun 12, 2018 at 1:48 PM Bill Bejeck wrote: > +1 > > Thanks, > Bill > > On Sun, Jun 10, 2018 at 5:32 PM Ted Yu wrote: > > > +1 > > > > On Sun, Jun 10, 2018 at 2:17 PM, Matthias J. Sax > > wrote: > > > > > +1 (binding) > > > > > > Thanks for the KIP. > > > > > > > > > -Matthias > >

Re: [DISCUSS] KIP-307: Allow to define custom processor names with KStreams DSL

2018-10-19 Thread Matthias J. Sax
What is the status of this KIP? -Matthias On 7/19/18 5:17 PM, Guozhang Wang wrote: > Hello Florian, > > Sorry for being late... Found myself keep apologizing for late replies > these days. But I do want to push this KIP's progress forward as I see it > very important and helpful feature for

Re: [DISCUSSION] KIP-300: Add Windowed KTable API in StreamsBuilder

2018-10-19 Thread Matthias J. Sax
Boyang, what is the status of this KIP? -Matthias On 6/17/18 9:21 PM, Guozhang Wang wrote: > Thanks for the KIP Boyang, I made a pass over the KIP and the PR and have > some comments: > > 1. About the public API, I agree with Matthias that we can consider > exposing the `innerDeserializer` and

Re: [DISCUSS] KIP-262 Metadata should include the number of state stores for task

2018-10-19 Thread Matthias J. Sax
Any thought on my last email about discarding this KIP? -Matthias On 9/14/18 11:44 AM, Matthias J. Sax wrote: > Hi, > > we recently had a discussion on a different ticket to reduce the size of > the metadata we need to send: > https://issues.apache.org/jira/browse/KAFKA-7149 > > It seems,

Re: [VOTE] KIP-334 Include partitions in exceptions raised during consumer record deserialization/validation

2018-10-19 Thread Matthias J. Sax
I am not 100% familiar with the details of the consumer code, however I tend to disagree with: > There's no difference between the two cases -- if (and only if) the message > is corrupt, it can't be deserialized. If (and only if) it can't be > deserialized, it is corrupt. Assume that a user

Frequent under-replicated partitions

2018-10-19 Thread Suman B N
Team, We have been observing some partitions being under-replicated. Broker version 0.10.2.1. Below actions were carried out but in vain: - Tried restarting nodes. - Tried increasing replica fetcher threads. Recommend ideal replica fetcher threads for a 20 node cluster with 150-200rps

Re: [DISCUSS]KIP-216: IQ should throw different exceptions for different errors

2018-10-19 Thread Matthias J. Sax
Glad to have you back Vito :) Some follow up thoughts: - the current `InvalidStateStoreException` is documents as being sometimes retry-able. From the JavaDocs: > These exceptions may be transient [...] Hence, it is valid to backoff and > retry when handling this exception. I am wondering

Re: [DISCUSS] KIP-379: Multiple Consumer Group Management

2018-10-19 Thread Suman B N
This eases debugging metadata information of consumer groups and offsets in case of client hungs which we have been facing frequently. +1 from me. Well done Alex! -Suman On Fri, Oct 19, 2018 at 8:36 PM Vahid Hashemian wrote: > Thanks for proposing the KIP. Looks good to me overall. > > I agree

[VOTE] KIP-380: Detect outdated control requests and bounced brokers using broker generation

2018-10-19 Thread Patrick Huang
Hi All, I would like to call for a vote on KIP-380: https://cwiki.apache.org/confluence/display/KAFKA/KIP-380%3A+Detect+outdated+control+requests+and+bounced+brokers+using+broker+generation Here is the discussion thread:

Re: Throwing away prefetched records optimisation.

2018-10-19 Thread Matthias J. Sax
Just my 2 cents. I am not 100% sure if we would need to change the consumer for this. While I still think, that KIP-349 might be valuable, it seems to be complementary/orthogonal to the issue discussed here. For Kafka Streams, we have a related scenario and what Kafka Streams does is, to add its

RE: Request for contributor permissions

2018-10-19 Thread Erica Parker
I am looking for some Kafka Developers if you know anyone who is interested please reach out! Erica Parker | Senior Technical Recruiter INT Technologies – A Nationally Recognized Veteran Owned Business C: (720) 483-7059 W: (720)735-4829 Stay connected with us: Ask your INT representative about

Re: Request for contributor permissions

2018-10-19 Thread Matthias J. Sax
Done. On 10/18/18 6:51 PM, Stone Huang wrote: > JIRA ID: godisren > Cwiki ID: Ren Jr Huang > > Thanks in advance! > signature.asc Description: OpenPGP digital signature

Re: [VOTE] KIP-369 Alternative Partitioner to Support "Always Round-Robin" Selection

2018-10-19 Thread Matthias J. Sax
People a currently quite busy with 2.1.0 and 2.0.1 releases. That's why the vote might go slow. Just keep bumping the vote thread on a weekly bases... Thanks for your patients. -Matthias On 10/19/18 1:08 AM, M. Manna wrote: > Since this has gone quiet, could I prequest 1 more vote here - if

Re: [DISCUSS] KIP-378: Enable Dependency Injection for Kafka Streams handlers

2018-10-19 Thread Matthias J. Sax
I just realized that there is not Jira for this KIP. Can you please create one? All KIPs need to have a Jira for tracking. Thanks. On 10/10/18 6:51 AM, Damian Guy wrote: > Hi Wladimir, > > Of the two approaches in the KIP - i feel the second approach is cleaner. > However, am i correct in

Re: [DISCUSS] KIP-351: Add --under-min-isr option to describe topics command

2018-10-19 Thread Kevin Lu
Bumping this as I have added some additional details. This change will require adding a "--bootstrap-server" flag to identify the current broker/cluster configured "min.insync.replicas". Regards, Kevin On Fri, Oct 12, 2018 at 4:19 PM Kevin Lu wrote: > Hi All, > > After some feedback, I have

Re: [DISCUSS] KIP-379: Multiple Consumer Group Management

2018-10-19 Thread Vahid Hashemian
Thanks for proposing the KIP. Looks good to me overall. I agree with Jason's suggestion that it would be best to keep the current output format when a single '--group' is present. Because otherwise, there would be an impact to users who rely on the current output format. Also, starting with a

[jira] [Created] (KAFKA-7522) As a client of kafka-streams library I want to provide custom error handlers programmatically

2018-10-19 Thread Yevhen Tsyba (JIRA)
Yevhen Tsyba created KAFKA-7522: --- Summary: As a client of kafka-streams library I want to provide custom error handlers programmatically Key: KAFKA-7522 URL: https://issues.apache.org/jira/browse/KAFKA-7522

[jira] [Created] (KAFKA-7521) [kafka-streams-scala_2.11] Foreach results in StackOverflowError

2018-10-19 Thread Bogdan Iordache (JIRA)
Bogdan Iordache created KAFKA-7521: -- Summary: [kafka-streams-scala_2.11] Foreach results in StackOverflowError Key: KAFKA-7521 URL: https://issues.apache.org/jira/browse/KAFKA-7521 Project: Kafka

apache/kafka

2018-10-19 Thread abeceda4
Hi, first thank you for amazing software kafka. I am studying its code and i need help to identify design patterns, can you help me please ?   Thanks   Mrkvica

[jira] [Created] (KAFKA-7520) Adding possibility to configure versions in Mirror Maker ducktape test

2018-10-19 Thread Andras Katona (JIRA)
Andras Katona created KAFKA-7520: Summary: Adding possibility to configure versions in Mirror Maker ducktape test Key: KAFKA-7520 URL: https://issues.apache.org/jira/browse/KAFKA-7520 Project: Kafka

Re: [VOTE] KIP-369 Alternative Partitioner to Support "Always Round-Robin" Selection

2018-10-19 Thread M. Manna
Since this has gone quiet, could I prequest 1 more vote here - if anyone thinks it's worth doing? Thanks, On Wed, 3 Oct 2018 at 16:14, M. Manna wrote: > Thanks for reminding me about the "Binding" vote Bill. I remember some > people with non-binding vote, so jumped the gun a bit too early. >

Re: [DISCUSS] KIP-379: Multiple Consumer Group Management

2018-10-19 Thread Jason Gustafson
Hi Alex, Thanks for the KIP. I think it makes sense, especially since most of the group apis are intended for batching anyway. The only questions I have are about compatibility. For example, the csv format for resetting offsets is changed, so will we continue to support the old format? Also, if

Re: [DISCUSS] KIP-382: MirrorMaker 2.0

2018-10-19 Thread Harsha Chintalapani
Ryanne,        Makes sense. Can you please add this under rejected alternatives so that everyone has context on why it  wasn’t picked. Thanks, Harsha On Oct 18, 2018, 8:02 AM -0700, Ryanne Dolan , wrote: > Harsha, concerning uReplicator specifically, the project is a major > inspiration for