Re: [DISCUSS] KIP-448: Add State Stores Unit Test Support to Kafka Streams Test Utils

Thanks for the KIP. Only one initial comment (Sophie mentioned this already but I want to emphasize on it). You state that > These will be internal classes, so no public API/interface. If this is the case, we don't need a KIP. However, the idea of the original Jira is to actually make those

Re: Request for contribution

Check out the link in section "Developers": https://cwiki.apache.org/confluence/display/KAFKA/Index -Matthias On 4/10/19 1:16 AM, Manish G wrote: > Hi, > > I am Manish, a Java programmer by profession. > > I want to learn and contribute to Kafka. > > Can you please guide me to appropriate

Re: [DISCUSS] KIP-446: Add changelog topic configuration to KTable suppress

I think that the current proposal to add `withLoggingDisabled()` and `withLoggingEnabled(Map)` should be the best option. IMHO there is no reason to add a WARN log. We also don't have a WARN log when people disable logging on regular stores. As Bruno mentioned, this might also lead to data loss,

Re: [VOTE] KIP-339: Create a new IncrementalAlterConfigs API

Sounds good. Thanks Colin! Regards, Rajini On Wed, Apr 10, 2019 at 8:22 PM Colin McCabe wrote: > Hi Rajini, > > That is a good point. We want to keep the "transactionality" of updating > several configs for the same ConfigResource at once. SSL configs are a > good example of this-- it often

Re: [DISCUSS] KIP-439: Deprecate Interface WindowStoreIterator

Thanks for the input. >> Just to clarify the naming conflicts is between the newly added function >> and the old functions that we want to deprecate / remove right? The Yes, the conflict is just fort the existing `fetch()` methods for which we want to change the return type. IMHO, we should not

Kafka KIP Wiki Page not editable with Google Chrome

Hi Does anyone else have a problem editing https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals or creating a KIP using Google Chrome (latest update, on MacOS)? The page hangs forever and only shows an error message. If I do the same in Firefox everything works like a

[jira] [Created] (KAFKA-8216) MockConsumer's poll(Duration timeout) method doesn't wait for timeout

Kevin created KAFKA-8216: Summary: MockConsumer's poll(Duration timeout) method doesn't wait for timeout Key: KAFKA-8216 URL: https://issues.apache.org/jira/browse/KAFKA-8216 Project: Kafka Issue

Re: Kafka Jenkins not showing recent builds in history

Quick update: everything seems to be back to normal. On Mon, Apr 8, 2019 at 9:57 AM Sönke Liebau wrote: > I've opened an INFRA ticket about this [1]. Apparently, it is a known > issue that is solved by a restart of Jenkins. One will be scheduled in the > near future, I'll update the list once

Re: [DISCUSSION] KIP-418: A method-chaining way to branch KStream

Thanks for driving the discussion of this KIP. It seems that everybody agrees that the current branch() method using arrays is not optimal. I had a quick look into the PR and I like the overall proposal. There are some minor things we need to consider. I would recommend the following renaming:

[DISCUSS] KIP-451: Make TopologyTestDriver output iterable

Hi everyone, I would like to start the discussion on this small enhancement of the TopologyTestDriver. https://cwiki.apache.org/confluence/display/KAFKA/KIP-451%3A+Make+TopologyTestDriver+output+iterable Pull request is available at https://github.com/apache/kafka/pull/6556 Any feedback is

Re: [DISCUSS] KIP-450: Sliding Window Aggregations in the DSL

Thanks for the KIP Sophie. Couple of comments: It's a little unclear to me, what public API you propose. It seems you want to add > public class SlidingWindow extends TimeWindow {} and > public class SlidingWindows extends TimeWindows {} // or maybe `extends > Windows` If yes, should we add

Re: Request KIP permissions

Done. On 4/11/19 1:08 AM, Patrik Kleindl wrote: > Hi > Please add me (pkleindl) to the list to write a KIP > Thanks in advance and best regards > Patrik > signature.asc Description: OpenPGP digital signature

Stream caching

Hi guys. I’ve deployed spring cloud application using docker, service is using KStream functionality, and I have problems with /tmp folder. On some instances of application /tmp folder is from 3 to 6 GB. Maybe I missed some property? I will be very grateful for the help. Thanks, best

Request KIP permissions

Hi Please add me (pkleindl) to the list to write a KIP Thanks in advance and best regards Patrik

[jira] [Resolved] (KAFKA-8201) Kafka streams repartitioning topic settings crashing multiple nodes

[ https://issues.apache.org/jira/browse/KAFKA-8201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-8201. Resolution: Not A Bug > Kafka streams repartitioning topic settings crashing multiple

Request KIP Permissions

Hi, Could you please grant me write access to KIP proposals? I intend to make progress on KAFKA-6393. Wiki ID: lukasz (apache.org). Best regards, Lukasz Antoniak

Re: Request KIP Permissions

Done On Thu, Apr 11, 2019 at 8:40 AM Łukasz Antoniak wrote: > Hi, > > Could you please grant me write access to KIP proposals? I intend to make > progress on KAFKA-6393. Wiki ID: lukasz (apache.org). > > Best regards, > Lukasz Antoniak >

[jira] [Resolved] (KAFKA-8182) IllegalStateException in NetworkClient.initiateConnect when handling UnknownHostException thrown from ClusterConnectionStates.connecting

[ https://issues.apache.org/jira/browse/KAFKA-8182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Anderson resolved KAFKA-8182. -- Resolution: Duplicate Resolving as I've confirmed this is a duplicate of KAFKA-7974 and is

[jira] [Created] (KAFKA-8218) IllegalStateException while accessing context in Transformer

Bartłomiej Kępa created KAFKA-8218: -- Summary: IllegalStateException while accessing context in Transformer Key: KAFKA-8218 URL: https://issues.apache.org/jira/browse/KAFKA-8218 Project: Kafka

Re: Kafka KIP Wiki Page not editable with Google Chrome

I don't have issues. Maybe clear the browser cache? -Matthias On 4/11/19 2:40 AM, Patrik Kleindl wrote: > Hi > > Does anyone else have a problem editing > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals > or creating a KIP using Google Chrome (latest update, on

[jira] [Created] (KAFKA-8217) MockConsumer.poll executes pollTasks before checking wakeup flag

Kevin created KAFKA-8217: Summary: MockConsumer.poll executes pollTasks before checking wakeup flag Key: KAFKA-8217 URL: https://issues.apache.org/jira/browse/KAFKA-8217 Project: Kafka Issue Type:

Re: [DISCUSS] KIP-451: Make TopologyTestDriver output iterable

Thanks for the KIP! Overall, this makes sense and can simplify testing. What I am wondering is, why you suggest to return an `Iterable`? Maybe returning an `Iterator` would make more sense? Or a List? Note that the order of emits matters, thus returning a generic `Collection` would not seem to

Re: [DISCUSS] KIP-439: Deprecate Interface WindowStoreIterator

I like the renaming, since it also aligns with our metrics cleanup (KIP-444) which touches upon the store level metrics as well. One question: you seems still suggesting to keep "all" with the current name (and also using a separate metric for it), what's the difference between this one and other

[jira] [Created] (KAFKA-8220) Avoid kicking out members through rebalance timeout

Boyang Chen created KAFKA-8220: -- Summary: Avoid kicking out members through rebalance timeout Key: KAFKA-8220 URL: https://issues.apache.org/jira/browse/KAFKA-8220 Project: Kafka Issue Type:

[jira] [Created] (KAFKA-8219) Add web documentation for static membership

Boyang Chen created KAFKA-8219: -- Summary: Add web documentation for static membership Key: KAFKA-8219 URL: https://issues.apache.org/jira/browse/KAFKA-8219 Project: Kafka Issue Type: Sub-task

Re: [DISCUSS] KIP-429 : Smooth Auto-Scaling for Kafka Streams

Hello Jason, 1. Yeah I think `emigrate` is not a very good term maybe. On the other hand `evicted` is quite similar to `revoked` to me.. I'd like to have another proposal of `onPartitionLost` actually. 2. The interface's default implementation will just be `onPartitionRevoked`, so for user's

Re: [DISCUSS] KIP-439: Deprecate Interface WindowStoreIterator

While working at KIP-444 (https://github.com/apache/kafka/pull/6498) I realized there are a bunch of issues on metric names v.s. function names, e.g. some function named `fetchAll` are actually measure with `fetch`, etc. So in that KIP I proposed to make the function name aligned with metrics

Re: Stream caching

Hello Kohut, Streams's default state.dir location is /tmp/kafka-streams, that may explained it. For more info about state management of Kafka Streams, you can read https://kafka.apache.org/21/documentation/streams/architecture#streams_architecture_state Guozhang On Thu, Apr 11, 2019 at 1:09

Re: [VOTE] KIP-433: Block old clients on brokers

Hi Ying, It looks to me that all the examples given in the KIP can be handled with the existing "message.downconversion.enable" config and by configuring the message format to be the latest: 1. Kafka 8 / 9 / 10 consumer hangs when the message contains message header > ( KAFKA-6739 -

[jira] [Created] (KAFKA-8221) Augment LeaveGroupRequest to batch operation

Boyang Chen created KAFKA-8221: -- Summary: Augment LeaveGroupRequest to batch operation Key: KAFKA-8221 URL: https://issues.apache.org/jira/browse/KAFKA-8221 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-8222) Admin client changes to add ability to batch remove static members

Boyang Chen created KAFKA-8222: -- Summary: Admin client changes to add ability to batch remove static members Key: KAFKA-8222 URL: https://issues.apache.org/jira/browse/KAFKA-8222 Project: Kafka

[jira] [Created] (KAFKA-8223) Deprecate group.initial.rebalance.delay

Boyang Chen created KAFKA-8223: -- Summary: Deprecate group.initial.rebalance.delay Key: KAFKA-8223 URL: https://issues.apache.org/jira/browse/KAFKA-8223 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-8225) handle conflicting static member id

Boyang Chen created KAFKA-8225: -- Summary: handle conflicting static member id Key: KAFKA-8225 URL: https://issues.apache.org/jira/browse/KAFKA-8225 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-8224) Add static member id into Subscription Info for better rebalance behavior

Boyang Chen created KAFKA-8224: -- Summary: Add static member id into Subscription Info for better rebalance behavior Key: KAFKA-8224 URL: https://issues.apache.org/jira/browse/KAFKA-8224 Project: Kafka

[jira] [Resolved] (KAFKA-8214) Handling RecordTooLargeException in the main thread

[ https://issues.apache.org/jira/browse/KAFKA-8214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohan Parthasarathy resolved KAFKA-8214. Resolution: Duplicate > Handling RecordTooLargeException in the main thread >

Build failed in Jenkins: kafka-trunk-jdk8 #3532

See Changes: [github] MINOR: Use generated InitProducerId RPC (#6538) -- [...truncated 2.36 MB...] org.apache.kafka.connect.transforms.TimestampConverterTest >

Proposal to Auto Close Inactive Tickets

Hi Kafka Developers, The Apache Kafka JIRA currently has 2138 open JIRA tickets. As Charlie Munger once said, “Simplicity has a way of improving performance through enabling us to better understand what we are doing.” What are everyone’s thoughts on

Re: [DISCUSS] KIP-451: Make TopologyTestDriver output iterable

Hi Matthias Thanks for the questions. Regarding the return type: Iterable offers the option of being used in a foreach loop directly and it gives you access to the .iterator method, too. (ref: https://www.techiedelight.com/differences-between-iterator-and-iterable-in-java/ ) To return a List

Re: [VOTE] KIP-433: Block old clients on brokers

In general, I support this proposal, but I'd like some more details on what "rejecting API requests" mean? Close the connections? Return some kind of error? Is there a way for the client to know what happened? Is there a way for the admin to know how many clients are rejected? As a nit, the

Re: [DISCUSS] KIP-450: Sliding Window Aggregations in the DSL

Thanks for the comments Guozhang! I've answered your questions below On Tue, Apr 9, 2019 at 4:38 PM Guozhang Wang wrote: > Hi Sophie, > > Thanks for the proposed KIP. I've made a pass over it and here are some > thoughts: > > 1. "The window size is effectively the grace and retention period".

Re: Dynamic window size for aggregations

Hey Rajesh, I do like the idea of customized windows. To make that work, we need to define a rule-based window size generation. In practical sense, fully randomized windows are rare and those are usually addressed as session windows. For calendar based window strategy, the goal is to define a

Re: [DISCUSS] KIP-439: Deprecate Interface WindowStoreIterator

I did not see a reason to rename `all()` to `range()`. `all()` does not take any parameters to limit a range and is a good name IMHO. But I am not married to keep `all()` and if we think we should rename it, too, I am fine with it. Not sure what connection you make to metrics though. Can you

Re: [DISCUSS] KIP-433: Provide client API version to authorizer

@Colin, Thank you for the feedback! I have updated the KIP, added the explanation of why we use API version rather than Kafka version. I will start a vote for this KIP On Fri, Mar 29, 2019 at 9:47 AM Colin McCabe wrote: > Hi Ying, > > That's a fair point. Maybe using API keys directly is

[VOTE] KIP-433: Block old clients on brokers

Hi here, Please vote for https://cwiki.apache.org/confluence/display/KAFKA/KIP-433%3A+Block+old+clients+on+brokers Thank you!

Re: [DISCUSS] KIP-429 : Smooth Auto-Scaling for Kafka Streams

Hello Matthias: Thanks for your review. The background section uses streams assignor as well as the consumer's own stick assignor as examples illustrating the situation, but this KIP is for consumer coordinator itself, and the rest of the paragraph did not talk about Streams any more. If you

Re: [VOTE] KIP-433: Block old clients on brokers

Hi Gwen, Thank you very much for the feedback! I have updated the KIP. "reject request" means return an UNSUPPORTED_VERSION (35) error. Client libraries after 0.10.2 will show the message "The version of API is not supported." Client libraries before 0.10.2 will treat this error as an unknown

Build failed in Jenkins: kafka-trunk-jdk8 #3533

See Changes: [github] MINOR: ConnectionStressWorker: add missing executor shutdown (#6558) [github] MINOR: Remove SubscriptionState.Listener and replace with assignmentId

Use of version attribute in reassignment json of kafka-reassign-partitions.sh

Hi, What is the use/intention of version attribute in kafka-reassign-partitions.sh command ? I see the code(at least version 1.1) just ignores this attribute. > bin/kafka-reassign-partitions.sh --zookeeper localhost:2181 > --reassignment-json-file expand-cluster-reassignment.json --execute