Re: [VOTE] KIP-75 - Add per-connector Converters

+1 (binding) On 15 Aug 2016 7:21 pm, "Ewen Cheslack-Postava" wrote: > I would like to initiate the voting process for KIP-75: > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 75+-+Add+per-connector+Converters > > I'll kick things off with a +1 (binding). > > -- >

Re: Do not log value of configs that Kafka doesn't recognize

Note that we already mask password related config values[1]. The issue is that it only happens after parsing, and the output of `logUnused` is based on the `originals` map, which has the values before parsing: public void logUnused() { for (String key : unused()) log.warn("The

Re: Do not log value of configs that Kafka doesn't recognize

rstand, this looks more > like a bug where it unintentionally ended up logging these values. Do you > want me to file a JIRA for this? > > -Jaikiran > > > > On Wednesday 17 August 2016 06:09 PM, Ismael Juma wrote: > >> Note that we already mask password related con

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

n the voting thread, but can the authors include a section >> > about new ACLs if there are going to be ACLs for TransactionalId. It's >> > mentioned in the google doc, but I think new ACLs should be in a KIP >> > directly. >> > >> >> On Thu, Feb 2,

Re: [DISCUSS] KIP-117: Add a public AdministrativeClient API for Kafka admin operations

Hi Colin, Thanks for the KIP, great to make progress on this. I have some initial comments, will post more later: 1. We have KafkaProducer that implements the Producer interface and KafkaConsumer that implements the Consumer interface. Maybe we could have KafkaAdminClient that implements the

Re: Trying to understand design decision about producer ack and min.insync.replicas

I'd be in favour too. Ismael On 3 Feb 2017 7:33 am, "Ewen Cheslack-Postava" wrote: > On Thu, Feb 2, 2017 at 11:21 PM, James Cheng wrote: > > > Ewen, > > > > Ah right, that's a good point. > > > > My initial reaction to your examples was that "well,

Re: [DISCUSS] KIP-110: Add Codec for ZStandard Compression

d. > > > > If you have more opinion to this issue, don't hesitate to send me as a > > message. > > > > Best, > > Dongjin > > > > [^1]: see: Record#compressionType. > > [^2]: There is similar routine in Message.scala. But after KAFKA-4390, > &g

Re: [DISCUSS] KIP-110: Add Codec for ZStandard Compression

s > now used in ConsoleConsumer only. I think this class should be replaced but > since it is a separated topic, I will send another message for this issue. > [^3]: commit 642da2f (2011.8.2). > [^4]: commit c51b940. > [^5]: commit 547cced. > [^6]: commit 37356bf. > > On Thu, J

Re: [DISCUSS] KIP-115: Enforce offsets.topic.replication.factor

y problem of offsets.topic.replication.factor > > not > > > > > being > > > > > > enforced but only in testing, integration tests, it was almost > > > > > > unpredictable when offse

Re: [DISCUSS] KIP-117: Add a public AdministrativeClient API for Kafka admin operations

Hi all, I think it's good that we have discussed a number of API that would make sense in the AdminClient. Having done that, I think we should now narrow the focus of this KIP to a small set of methods to get us started. Once we have an AdminClient in the codebase, we can propose subsequent KIPs

Re: [DISCUSS] KIP-117: Add a public AdministrativeClient API for Kafka admin operations

hole generalized > class of possibly completed actions, but the flip side is maybe I'm just a > working guy who needs to list his kafka topics but doesn't know Rust, take > pity on me! :-) > > Nit picking aside, super excited to see us progress on this. > > -Jay > > O

Re: [DISCUSS] KIP-117: Add a public AdministrativeClient API for Kafka admin operations

Hi Jay, Thanks for the feedback. Comments inline. On Tue, Feb 7, 2017 at 8:18 PM, Jay Kreps wrote: > >- I think it would be good to not use "get" as the prefix for things >making remote calls. We've tried to avoid the java getter convention >entirely (see code

Re: Live-lock between consumer thread and heartbeat thread

ava:266) > at org.apache.kafka.clients.consumer.internals.AbstractCoordina > tor$HeartbeatThread.run(AbstractCoordinator.java:865) > - locked <0x0005c0acefc8> (a org.apache.kafka.clients.consu > mer.internals.ConsumerCoordinator) > > Hope this helps, if you

Re: Live-lock between consumer thread and heartbeat thread

Sorry, I replied a little too fast. It is true that my original suggestion would not work and I agree that it would be better to copy the map in `commitSync` (that's why I suggested it in my email). Please file a JIRA. However, that doesn't solve your immediate issue. More inline. On Thu, Feb 2,

Re: Live-lock between consumer thread and heartbeat thread

Hi Jan, Do you have stacktraces showing the issue? That would help. Also, if you can test 0.10.1.1, which is the latest stable release, that would be even better. From looking at the code briefly, I don't see where the consumer is locking on the received offsets map, so not sure what would cause

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

ich satisfies no one. > > > -Jason > > On Wed, Feb 1, 2017 at 4:14 PM, Guozhang Wang <wangg...@gmail.com> wrote: > > > Ismael, thanks for your feedbacks. Replied inline. > > > > On Wed, Feb 1, 2017 at 8:03 AM, Ismael Juma <ism...@juma.me.uk> wrote

Re: [VOTE] KIP-98: Exactly Once Delivery and Transactional Messaging

Hi Tom, That is a good point. During the discussion, it was agreed that changes to public interfaces (message format, protocol, ACLs, etc.) would be copied to the wiki once the things had settled down, but it looks like that hasn't been done yet. I agree that it makes sense to do it before people

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

Hi all, A few comments follow: 1. The document states "inter-broker communications will be increased by M * N * P round trips per sec. We need to conduct some system performance test to make sure this additional inter-broker traffic would not largely impact the broker cluster". Has this testing

Re: [DISCUSS] KIP-117: Add a public AdministrativeClient API for Kafka admin operations

Hi Colin, I was thinking about the API and the fact that the AdminClient will be used by a variety of somewhat unrelated things (unlike the Consumer and Producer), so I think we should consider an approach where not all the methods are at the top-level. One idea is that you could perform

Re: [DISCUSS] KIP-84: Support SASL/SCRAM mechanisms

ail.com> wrote: > Thanks, Ismael. Just curious, why does it not make sense to do bcrypt > it in the context of SCRAM? > > On Mon, Jan 23, 2017 at 3:54 PM, Ismael Juma <ism...@juma.me.uk> wrote: > > > Hi Roger, > > > > SCRAM uses the PBKDF2 mechanism, here's

[DISCUSS] KIP-118: Drop Support for Java 7 in Kafka 0.11

Hi all, I have posted a KIP for dropping support for Java 7 in Kafka 0.11: https://cwiki.apache.org/confluence/display/KAFKA/KIP-118%3A+Drop+Support+for+Java+7+in+Kafka+0.11 Most people were supportive when we last discussed the topic[1], but there were a few concerns. I believe the following

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

Comments inline. On Thu, Feb 2, 2017 at 6:28 PM, Jason Gustafson wrote: > Took me a while to remember why we didn't do this. The timestamp that is > included at the message set level is the max timestamp of all messages in > the message set as is the case in the current

KIP-119: Drop Support for Scala 2.10 in Kafka 0.11

Hi all, I have posted a KIP for dropping support for Scala 2.10 in Kafka 0.11: https://cwiki.apache.org/confluence/display/KAFKA/KIP-119%3A+Drop+Support+for+Scala+2.10+in+Kafka+0.11 Please take a look. Your feedback is appreciated. Thanks, Ismael

Re: KIP-119: Drop Support for Scala 2.10 in Kafka 0.11

ething along the lines > of: > > "Kafka's policy is to support the 2 most recently released versions of Java > and Scala at a given time. When a new version becomes available, the > supported versions will be updated in the next major release of Kafka." > > > O

Re: [DISCUS] consuming messages is polling - is there a reason? new KIP for poll?

On Thu, Feb 2, 2017 at 4:41 PM, radai wrote: > also - i dont think you need to shorten fetch.max.wait.ms to get lower > delays - you could still configure a relatively long fetch.max.wait.ms and > have the broker answer your poll the minute _any_ messags are

Re: [VOTE] KIP-108: Create Topic Policy

/KAFKA/KIP-108%3A+Create+Topic+Policy Hopefully these are uncontroversial, but please let me know if you disagree. Ismael On Wed, Jan 11, 2017 at 11:18 PM, Ismael Juma <ism...@juma.me.uk> wrote: > Thanks to everyone who voted and provided feedback. > > The vote has passed with 4 bi

Re: KafkaException: Message payload is null shutting down broker

Hi Rodrigo, Please file a JIRA so that this can be investigated. Ismael On Mon, Jan 23, 2017 at 1:32 PM, Rodrigo Queiroz Saramago < rodrigo.saram...@zup.com.br> wrote: > Hello, I have a test environment with 3 brokers and 1 zookeeper nodes, in > which clients connect using two-way ssl

Re: [DISCUSS] KIP-84: Support SASL/SCRAM mechanisms

Hi Roger, SCRAM uses the PBKDF2 mechanism, here's a comparison between PBKDF2 and bcrypt: http://security.stackexchange.com/questions/4781/do-any-secu rity-experts-recommend-bcrypt-for-password-storage/6415#6415 It may be worth supporting bcrypt, but not sure it would make sense to do it in the

Re: [DISCUSS] KIP-110: Add Codec for ZStandard Compression

gt; > > Thanks Ewen for the guidance!! > > > > > > Best, > > > Dongjin > > > > > > On Thu, Jan 12, 2017 at 6:44 AM, Ismael Juma <ism...@juma.me.uk> > wrote: > > > > > >> That's a good point Ewen. Dongjin, you could use

Re: [DISCUSS] KIP-115: Enforce offsets.topic.replication.factor

An important question is if this needs to wait for a major release or not. Ismael On Thu, Jan 26, 2017 at 12:19 AM, Ismael Juma <ism...@juma.me.uk> wrote: > +1 from me too. > > Ismael > > On Thu, Jan 26, 2017 at 12:07 AM, Ewen Cheslack-Postava <e...@confluent.io > &

Re: [DISCUSS] KIP-115: Enforce offsets.topic.replication.factor

+1 from me too. Ismael On Thu, Jan 26, 2017 at 12:07 AM, Ewen Cheslack-Postava wrote: > +1 > > Since this is an unusual one, I think it's worth pointing out that the KIP > notes it is really a bug fix, but since it has compatibility implications > the KIP was worth it. It

Re: [VOTE] KIP-115: Enforce offsets.topic.replication.factor

+1 (binding) Ismael On Thu, Jan 26, 2017 at 12:34 AM, Onur Karaman wrote: > I'd like to start the vote for KIP-115: Enforce > offsets.topic.replication.factor > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- >

Re: [VOTE] KIP-74: Add FetchResponse size limit in bytes

Good catch, Colin. +1 to editing the wiki to match the desired behaviour and what was implemented in 0.10.1. Ismael On Sat, Jan 21, 2017 at 12:19 AM, Colin McCabe wrote: > Hi all, > > While looking at some code related to KIP-74, I noticed a slight > discrepancy between the

Re: [ANNOUNCE] Apache Kafka 0.10.2.0 Released

an > Norwood, Dana Powers, dasl, Derrick Or, Dong Lin, Dustin Cote, Edoardo > Comar, Edward Ribeiro, Elias Levy, Emanuele Cesena, Eno Thereska, Ewen > Cheslack-Postava, Flavio Junqueira, fpj, Geoff Anderson, Guozhang Wang, > Gwen Shapira, Hikiko Murakami, Himani Arora, himani1, Hojjat Ja

Re: [ANNOUNCE] Apache Kafka 0.10.2.0 Released

gt;> 0/kafka_2.12-0.10.2.0.tgz > >> (experimental 2.12 artifact) > >> > >> Thanks to the 101 contributors on this release! > >> > >> Akash Sethi, Alex Loddengaard, Alexey Ozeritsky, amethystic, Andrea > >> Cosentino, Andrew Olson, Andrew Stevenson,

Re: [DISCUSS] KIP-82 - Add Record Headers

; On point 2, > > > > > > > > > > Setter/getter typical to properties/headers api’s > > traditionally are map > > > > > styled interfaces and what I believe is most expected > styled > > thus the >

Re: [DISCUSS] KIP-127: Pluggable JAAS LoginModule configuration for SSL

ASL_SSL as it > would be good to let the user configure that regardless (i can do the > jira/pr) > > On Fri, Feb 24, 2017 at 9:47 AM, Ismael Juma <ism...@juma.me.uk> wrote: > > > Hi Christopher, > > > > Thanks for the KIP. I have two comments: > > > > 1. C

Re: [DISCUSS] KIP-127: Pluggable JAAS LoginModule configuration for SSL

Hi Christopher, Thanks for the KIP. I have two comments: 1. Can you please explain in the KIP (maybe in the Rejected Alternatives section) why a PrincipalBuilder is not good enough for your use case? This is typically what people use to customise authentication for the TLS case. 2. You mention

Re: [DISCUSS] KIP-124: Request rate quotas

Thanks for the KIP, Rajini. This is a welcome improvement and the KIP page covers it well. A few comments: 1. Can you expand a bit on the motivation for throttling requests that fail authorization for ClusterAction? Under what scenarios would this help? 2. I think we should rename

Re: [DISCUSS] KIP-124: Request rate quotas

Hi Jay, Regarding 1, I definitely like the simplicity of keeping a single throttle time field in the response. The downside is that the client metrics will be more coarse grained. Regarding 3, we have `leader.imbalance.per.broker.percentage` and `log.cleaner.min.cleanable.ratio`. Ismael On

Re: [VOTE] KIP-98: Exactly Once Delivery and Transactional Messaging

Great work on the proposal and iterating on it based on community feedback. As Jun (and others) said, it's likely that minor changes will happen as the PR is reviewed and additional testing takes place since this is a significant change. I am +1 (binding) on the proposal without optional keys and

Re: [DISCUSS] KIP-82 - Add Record Headers

Hi all, Great to see the progress that has been achieved on this one. :) A few comments regarding the APIs (I'm still reviewing the message format changes): 1. Nit: `getHeaders` in `ProducerRecord` and `ConsumerRecord` should be named `headers` (we avoid the `get` prefix in Kafka) 2. The

[VOTE] KIP-118: Drop Support for Java 7 in Kafka 0.11

Hi everyone, Since everyone in the discuss thread was in favour (10 people responded), I would like to initiate the voting process for KIP-118: Drop Support for Java 7 in Kafka 0.11: https://cwiki.apache.org/confluence/display/KAFKA/KIP-118%3A+Drop+Support+for+Java+7+in+Kafka+0.11 The vote will

Re: KIP-119: Drop Support for Scala 2.10 in Kafka 0.11

So far the feedback is positive, but there weren't many responses. I'll start a vote next week if there are no objections until then. Ismael On Fri, Feb 3, 2017 at 2:30 PM, Ismael Juma <ism...@juma.me.uk> wrote: > Hi all, > > I have posted a KIP for dropping support for Scala 2.1

Re: KIP-121 [VOTE]: Add KStream peek method

+1 (binding) from me. For the record, there were 4 binding +1s (Gwen, Guozhang, Jay and myself). Ismael On Wed, Feb 15, 2017 at 8:24 PM, Steven Schlansker < sschlans...@opentable.com> wrote: > From reading the bylaws it's not entirely clear who closes the vote or how > they > decide when to do

Re: [VOTE] 0.10.2.0 RC2

+1 (non-binding). Verified source and Scala 2.11 binary artifacts, quick start on source artifact and Scala 2.11 binary artifacts. Thanks for managing the release! Ismael On Tue, Feb 14, 2017 at 6:39 PM, Ewen Cheslack-Postava wrote: > Hello Kafka users, developers and

Re: [DISCUSS] KIP-111 : Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

the `channelPrincipal` as a field in `Session` instead of `KafkaPrincipal`. It would be good to understand why this was rejected. Ismael On Thu, Jan 12, 2017 at 7:07 PM, Ismael Juma <ism...@juma.me.uk> wrote: > Hi Mayuresh, > > Thanks for the KIP. A quick comment before I do a more de

Re: [DISCUSS] KIP-114: KTable materialization and improved semantics

Thanks for the KIP, Eno. It seems reasonable on first inspection although others more familiar with Streams may have more to say. :) With regards to "log compaction based on timestamps", it may make sense to do that as its own KIP as interested parties may miss it if it's inside a Streams KIP. It

Re: [VOTE] KIP-74: Add FetchResponse size limit in bytes

Thanks for the KIP. +1 (binding) with the following suggestion: Fetch Request (Version: 3) => replica_id max_wait_time min_bytes response_max_bytes [topics] replica_id => INT32 max_wait_time => INT32 min_bytes => INT32 response_max_bytes => INT32 topics => topic [partitions] topic

Re: [DISCUSS] KIP-74: Add Fetch Response Size Limit in Bytes

Hi Andrey, On Tue, Aug 16, 2016 at 12:58 PM, Andrey L. Neporada < anepor...@yandex-team.ru> wrote: > > BTW, what is the target version of this KIP? Currently the inter-broker > protocol version in KIP is set to 0.11.0-IV0. Do we want to target for 0.11 > or maybe somewhat earlier? > I suggest

Re: [DISCUSS] Time-based releases for Apache Kafka

Hi Jim, Thanks for your feedback. We value the community and we definitely want Kafka to remain a fun and friendly place to participate. Under this proposal, volunteers will still be able to do the work when they can. The benefit is that it is likely to reach users faster since the next release

Re: [DISCUSS] Java 8 as a minimum requirement

> cutting 0.10.1 features moving to 0.11 and giving rough > timeline when that would be released would be ideal. > > Thanks, > Harsha > > On Fri, Jun 17, 2016, at 11:13 AM, Ismael Juma wrote: > > Hi Harsha, > > > > Comments below. &

Re: [DISCUSS] KIP-72 Allow Sizing Incoming Request Queue in Bytes

Hi Radai, Thanks for the proposal. I think it makes sense to be able to limit the request queue by bytes. I haven't made up my mind on whether having both limits is better than having a single one yet, but we probably need to make a call on that before we can start a vote. Just a quick point

[DISCUSS] 0.10.3.0/0.11.0.0 release planning

Hi all, With 0.10.2.0 out of the way, I would like to volunteer to be the release manager for our next time-based release. See https://cwiki.apache.org/c onfluence/display/KAFKA/Time+Based+Release+Plan if you missed previous communication on time-based releases or need a reminder. I put together

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

pleAclAuthorizer with this KIP. > > Thanks, > > Mayuresh > > > > On Mon, Feb 27, 2017 at 10:56 AM, Ismael Juma <ism...@juma.me.uk> wrote: > > > Hi Mayuresh, > > > > Sorry for the delay. The updated KIP states that there is no > compatibility >

Re: [VOTE] KIP-118: Drop Support for Java 7 in Kafka 0.11

updated the relevant wiki pages. Ismael On Thu, Feb 9, 2017 at 3:31 PM, Ismael Juma <ism...@juma.me.uk> wrote: > Hi everyone, > > Since everyone in the discuss thread was in favour (10 people responded), > I would like to initiate the voting process for KIP-118: Drop Support for

[VOTE] KIP-119: Drop Support for Scala 2.10 in Kafka 0.11

Hi everyone, Since the few who responded in the discuss thread were in favour and there were no objections, I would like to initiate the voting process for KIP-119: Drop Support for Scala 2.10 in Kafka 0.11:

Re: [DISCUSS] 0.10.3.0/0.11.0.0 release planning

review. > > Thanks. > --Vahid > > > > > > From: Ismael Juma <ism...@juma.me.uk> > To: dev@kafka.apache.org > Date: 02/27/2017 07:47 PM > Subject:[DISCUSS] 0.10.3.0/0.11.0.0 release planning > Sent by:isma...@gmail.com > > &g

Re: [DISCUSS] KIP-82 - Add Record Headers

Hi Becket, Comments inline. On Sat, Feb 25, 2017 at 10:33 PM, Becket Qin wrote: > > 1. Regarding the mutability. > > I think it would be a big convenience to have headers mutable during > certain stage in the message life cycle for the use cases you mentioned. I > agree

Re: [DISCUSS] 0.10.3.0/0.11.0.0 release planning

fir Manor > > Co-Founder & CTO | Equalum > > Mobile: +972-54-7801286 | Email: ofir.ma...@equalum.io > > On Tue, Feb 28, 2017 at 12:37 PM, Ismael Juma <ism...@juma.me.uk> wrote: > > > Hi Vahid, > > > > Sure, I've added KIP-54. I thought I had done i

Re: [DISCUSS] 0.10.3.0/0.11.0.0 release planning

> > > Hi Ismael, > > > > Thanks for volunteering on the new release. > > > > I think 0.11.0.0 makes a lot of sense given the new big features we are > > intended to include. > > > > Thanks, > > > > Jiangjie (Becket) Qin > > >

Re: [VOTE] KIP-119: Drop Support for Scala 2.10 in Kafka 0.11

Hi Becket, I think there's a misunderstanding. Scala 2.11 has been the recommended version since Kafka 0.9.0.0 as you can see in the downloads page: https://kafka.apache.org/downloads Scala 2.10 is the default build version for the same reason that we recommend people to build with Java 7 when

Re: [DISCUSS] 0.10.3.0/0.11.0.0 release planning

hange the message format, in which case we bump to 0.11.x" in https://cwiki.apache.org/confluence/display/KAFKA/Time+Based+Release+Plan On Tue, Feb 28, 2017 at 3:47 AM, Ismael Juma <ism...@juma.me.uk> wrote: > Hi all, > > With 0.10.2.0 out of the way, I would like to volunteer to

Re: [DISCUSS] 0.10.3.0/0.11.0.0 release planning

I've updated JIRA versions and the KIP wiki page. Ismael On Thu, Mar 2, 2017 at 12:51 AM, Ismael Juma <ism...@juma.me.uk> wrote: > Thanks everyone for the feedback. Since everyone was in favour and we had > covered this particular case in the time-based release plan[1], I

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

Hi Mayuresh, Sorry for the delay. The updated KIP states that there is no compatibility impact, but that doesn't seem right. The fact that we changed the type of Session.principal to `Principal` means that any code that expects it to be `KafkaPrincipal` will break. Either because of declared

[DISCUSS] Remove beta label from the new Java consumer

Hi all, We currently say the following in our documentation: "As of the 0.9.0 release we have added a new Java consumer to replace our existing high-level ZooKeeper-based consumer and low-level consumer APIs. This client is considered beta quality."[1] Since then, Jason and the community have

Re: [DISCUSS] Time-based releases for Apache Kafka

Thanks for putting this together Gwen. I think it sounds reasonable and instead of trying to optimise every aspect of it ahead of time (which is hard, subjective and time-consuming), I am happy to try what is being proposed and tweak based on experience. One thing we should pay particular

Re: [DISCUSS] Remove beta label from the new Java consumer

Hi Jaikiran, Thanks for your feedback. Comments inline. On Wed, Aug 31, 2016 at 5:40 AM, Jaikiran Pai wrote: > Personally, I would be OK if the beta label is removed from it if the dev > team is sure the API isn't going to change. I don't know if that's true or > not

Re: [DISCUSS] KIP-79 - ListOffsetRequest v1 and offsetForTime() method in new consumer.

Thanks for the proposal Becket. Looks good overall, a few comments: ListOffsetResponse => [TopicName [PartitionOffsets]] > PartitionOffsets => Partition ErrorCode Timestamp [Offset] > Partition => int32 > ErrorCode => int16 > Timestamp => int64 > Offset => int It should be int64 for

Re: [DISCUSS] KIP-78: Cluster Id

Hi Dong, Thanks for your feedback. Comments inline. On Thu, Sep 1, 2016 at 7:51 PM, Dong Lin wrote: > > I share the view with Harsha and would like to understand how the current > approach of randomly generating cluster.id compares with the approach of > manually specifying

Re: [DISCUSS] KIP-78: Cluster Id

Dong, Sumit responded to a number of points already, so I will try to be brief. See inline. Also, it may just be possible that we won't reach agreement. In that case, a vote may be a way to figure out if people feel that this proposal adds value in its current form or not. On Mon, Sep 5, 2016

Re: SASL session expiry

Hi Mickael, The Kerberos ticket refresh mechanism is there for new connections, not existing connections. Currently, the suggested approach is to rely on the authorizer to deal with expired credentials. Would this work for you? Ismael On Mon, Sep 5, 2016 at 11:13 AM, Mickael Maison

Re: SASL session expiry

orizers? It sort of expects tight coupling between authenticator and > authorizer, Not sure how an authorizer would deal with certificate expiry > or certificate revocation when using SSL client auth for instance. > > > On Mon, Sep 5, 2016 at 11:20 AM, Ismael Juma <ism...@juma.me.uk>

Re: WARN log message flooding broker logs for a pretty typical SSL setup

Hi Jaikiran, I agree that this is a valid configuration and the log level seems too high given that. The original motivation is explained in the PR: https://github.com/apache/kafka/pull/155/files#diff-fce430ae21a0c98d82da6d4aa551f824L603 That is, help people figure out if client authentication

Re: SASL session expiry

On Mon, Sep 5, 2016 at 3:48 PM, Mickael Maison wrote: > Yes in our case, I can see how we would add the functionality, but I > was wondering if people might be interested to directly have such a > feature in Kafka. At the moment, the authorization logic is only >

Re: [DISCUSS] KIP-78: Cluster Id

Hi Dong, A few clarifications below. On Mon, Sep 5, 2016 at 7:21 PM, Dong Lin wrote: > > I think you are saying that we can stop our discussion and follow simply > take a vote where the majority decides. I don't think this is a good way to > find the best design for a KIP

Re: [DISCUSS] KIP-78: Cluster Id

Hi Dong, Thanks for your reply. See inline. On Mon, Sep 5, 2016 at 11:28 PM, Dong Lin wrote: > Yes, user has the option to manually set the cluster.id by directly > setting > the znode. But the KIP doesn't provide script for doing this. That is correct. And

Re: [ANNOUNCE] New committer: Jason Gustafson

Congratulations Jason! Well-deserved. :) Ismael On Tue, Sep 6, 2016 at 11:25 PM, Neha Narkhede wrote: > The PMC for Apache Kafka has invited Jason Gustafson to join as a > committer and > we are pleased to announce that he has accepted! > > Jason has contributed numerous

[VOTE] KIP-78: Cluster Id

Hi all, I would like to initiate the voting process for KIP-78: Cluster Id: https://cwiki.apache.org/confluence/display/KAFKA/KIP-78%3A+Cluster+Id As explained in the KIP and discussion thread, we see this as a good first step that can serve as a foundation for future improvements. Thanks,

Re: [DISCUSS] KIP-78: Cluster Id

On Mon, Sep 5, 2016 at 10:41 PM, Ismael Juma <ism...@juma.me.uk> wrote: > I will update the KIP to make this point clearer. > Done. Ismael

[VOTE] KIP-78 Cluster Id (second attempt)

Hi all, I would like to (re)initiate[1] the voting process for KIP-78 Cluster Id: https://cwiki.apache.org/confluence/display/KAFKA/KIP-78%3A+Cluster+Id As explained in the KIP and discussion thread, we see this as a good first step that can serve as a foundation for future improvements.

Re: Plans to improve SSL performance in Kafka, for 0.10.x?

On Tue, Sep 6, 2016 at 11:35 AM, Tom Crayford wrote: > Our performance tests were run with the JVM producer and consumer, and we > didn't notice any real slowdown on that side either when enabling SSL. Just > to check the obvious: are your producers and consumers going over

Re: Plans to improve SSL performance in Kafka, for 0.10.x?

Hi Todd, Thanks for sharing your experience enabling TLS in your clusters. Very helpful. One comment below. On Sun, Sep 4, 2016 at 6:28 PM, Todd Palino wrote: > > Right now, we're specifically avoiding moving consume traffic to SSL, due > to the zero copy send issue. Now I've

Re: [VOTE] KIP-78: Cluster Id

> On Tue, Sep 6, 2016 at 4:11 PM, Ismael Juma <ism...@juma.me.uk> wrote: > > > Hi all, > > > > I would like to initiate the voting process for KIP-78: Cluster Id: > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-78%3A+Cluster+Id > > &g

Re: [VOTE] KIP-33 - Add a time based log index

by clients which can be different from the create time of > the > > > log > > > > segment file. 2) On some Linux, the file create time is not > available, > > so > > > > using segment file create time may not always work. > > > > > > > > Do

Re: [DISCUSS] Time-based releases for Apache Kafka

think it > would > > > work well for Kafka too. > > > >> > > > >> Andrew Schofield > > > >> > > > >> > > > >>> From: ofir.ma...@equalum.io > > >

Re: [DISCUSS] KIP-78: Cluster Id

d to change the all the config files and make sure > > every > > > > one > > > > > of > > > > > > them is correct as well as update the ZK metadata. This will > > require > > > a > > > > > > reboot/downtime of the enti

Re: [DISCUSS] KIP-78: Cluster Id

Hi Flavio, Thanks for reviewing the KIP. Comments inline. On Sat, Sep 3, 2016 at 4:48 PM, Flavio Junqueira wrote: > Thanks for the KIP, Ismael, looks great. I have just a couple of comments > and questions: > > - I assume the znode containing the cluster id is persistent. Is

Re: Kafka KIP meeting Aug 30 at 11:00am PST

uot; about it :) > > > On Aug 31, 2016, at 12:34 PM, Ismael Juma <ism...@juma.me.uk> wrote: > > > > Hi Jim, > > > > Thanks for the feedback. More inline. > > > > On Wed, Aug 31, 2016 at 5:19 PM, Jim Jagielski <j...@jagunet.com> wrote: &g

Re: [DISCUSS] KIP-78: Cluster Id

d it's easier to implement on the broker side. > > Jun > > > On Wed, Aug 31, 2016 at 1:24 AM, Ismael Juma <ism...@juma.me.uk> wrote: > > > Thanks for the feedback Guozhang. Comment inline. > > > > On Wed, Aug 31, 2016 at 2:49 AM, Guozhang Wang <wangg.

Re: [DISCUSS] KIP-78: Cluster Id

Thanks for the feedback Guozhang. Comment inline. On Wed, Aug 31, 2016 at 2:49 AM, Guozhang Wang wrote: > About logging / debugging with the cluster id: I think the random UUID > itself may not be very helpful for human-readable debugging information, > and we'd better use

Re: [DISCUSS] Remove beta label from the new Java consumer

nd > > Connect). > > > > -Jason > > > > On Thu, Aug 25, 2016 at 9:57 AM, Gwen Shapira <g...@confluent.io> wrote: > > > > > Makes sense :) > > > > > > On Thu, Aug 25, 2016 at 9:40 AM, Neha Narkhede <n...@confluent.io> > &g

Re: [DISCUSS] KIP-78: Cluster Id

gregation. Perhaps it makes sense to include cluster ID in the > log? For example, as one of the things a broker logs after startup? > And ideally clients would log that as well after successful parsing of > MetadataResponse? > > Gwen > > > On Sat, Aug 27, 2016 at 4:39 AM, Ismael

Re: [DISCUSS] KIP-78: Cluster Id

ld log that as well after successful parsing of > > MetadataResponse? > > > > Gwen > > > > > > On Sat, Aug 27, 2016 at 4:39 AM, Ismael Juma <ism...@juma.me.uk> wrote: > > > Hi all, > > > > > > We've posted "KIP-78: Cluster

Re: Kafka KIP meeting Aug 30 at 11:00am PST

Hi Jim, Thanks for the feedback. More inline. On Wed, Aug 31, 2016 at 5:19 PM, Jim Jagielski wrote: > Just a reminder that these kinds of meetings are, by their very nature, > synchronous and disenfranchise those members of the community unable to > attend. > > It is great

Re: Agenda item for next KIP meeting

Hi Vahid, Sounds good. Jun is unavailable on the day, so I'll send the invite. Any other KIPs that people would like to discuss on the day (Tuesday at 11am PT)? The following KIPs are under discussion and have either been submitted or updated recently: KIP-68: Add a consumed log retention before

Re: [DISCUSS] KIP-79 - ListOffsetRequest v1 and offsetForTime() method in new consumer.

gt; > > it > > > > > > > > > easier for multiple timestamp query. Personally I think > given > > > > that > > > > > > > query > > > > > > > > > multiple timestamps is likely an infrequent operatio

Kafka KIP meeting Sep 13 at 11:00am PST

Hi, Everyone, We plan to have a Kafka KIP meeting this coming Tuesday at 11:00am PST. If you plan to attend but haven't received an invite, please let me know. The following is the tentative agenda. Agenda: KIP-54: Sticky Partition Assignment Strategy KIP-72: Allow Sizing Incoming Request Queue

Re: Kafka KIP meeting Sep 13 at 11:00am PST

Sure, done. Ismael On Mon, Sep 12, 2016 at 5:31 PM, Renu Tewari <tewa...@gmail.com> wrote: > Hi Ismael > Could you please add me to the invite. > > rtew...@linkedin.com > > regards > Renu > > > On Mon, Sep 12, 2016 at 8:39 AM, Ismael Juma <ism..

Re: [VOTE] KIP-78 Cluster Id (second attempt)

t; Thanks for the writeup. +1. > > Jun > > On Tue, Sep 6, 2016 at 7:46 PM, Ismael Juma <ism...@juma.me.uk> wrote: > > > Hi all, > > > > I would like to (re)initiate[1] the voting process for KIP-78 Cluster Id: > > > > https://cwiki.apache.org/conflu

<    1   2   3   4   5   6   7   8   9   10   >