Re: [ANNOUNCE] New committer: Matthias J. Sax

2018-01-15 Thread Michael Noll
Herzlichen Glückwunsch, Matthias. ;-) On Mon, 15 Jan 2018 at 11:38 Viktor Somogyi wrote: > Congrats Matthias! > > On Mon, Jan 15, 2018 at 9:17 AM, Jorge Esteban Quilcate Otoya < > quilcate.jo...@gmail.com> wrote: > > > Congratulations Matthias!! > > > > El lun., 15 ene. 2018 a las 9:08, Boyang

Re: [VOTE] KIP-270 A Scala wrapper library for Kafka Streams

2018-04-16 Thread Michael Noll
+1 (non-binding) Thanks for contributing and shepherding this, Debasish and team! And thanks also to Alexis Seigneurin for the original start of the Scala wrapper API. Best, Michael On Thu, Apr 12, 2018 at 10:24 AM, Damian Guy wrote: > Thanks for the KIP Debasish - +1 binding > > On Wed, 11

Re: KafkaSpout not consuming the first uncommitted offset data from kafka

2017-10-18 Thread Michael Noll
Hi Senthil, you should ask this question in the Apache Storm mailing list. At first sight this looks like a problem with Storm's KafkaSpout implementation, not with Kafka. Best wishes, Michael On Thu, Sep 28, 2017 at 8:47 PM, senthil kumar wrote: > Hi Kafka, > > I have a trident topology i

Re: KIP-457: Add DISCONNECTED state to Kafka Streams

2019-04-17 Thread Michael Noll
Richard, thanks for looking into this! However, I have some concerns. The KIP you created ( https://cwiki.apache.org/confluence/display/KAFKA/KIP-457%3A+Add+DISCONNECTED+status+to+Kafka+Streams) doesn't yet address open questions such as the ones mentioned by Matthias: 1) What is the difference

Re: [ANNOUNCE] New committer: Damian Guy

2017-06-12 Thread Michael Noll
Congratulations, Damian! On Mon, Jun 12, 2017 at 9:07 AM, Molnár Bálint wrote: > Congrats, Damien! > > 2017-06-12 8:44 GMT+02:00 Rajini Sivaram : > > > Congratulations, Damian! > > > > On Sat, Jun 10, 2017 at 12:15 PM, Mickael Maison < > mickael.mai...@gmail.com > > > > > wrote: > > > > > Congra

Re: [VOTE] KIP-138: Change punctuate semantics

2017-09-06 Thread Michael Noll
Thanks, Guozhang. FWIW here's my (non-binding) +1. Sorry for the delay, Gmail decided to mark this thread as spam :roll-eyes:. -Michael On Tue, Sep 5, 2017 at 10:36 PM, Guozhang Wang wrote: > Thanks for your inputs. The main motivation is indeed to achieve > consistency as we use "wall-cloc

Re: Memory consumption of Kafka-examples Kafka-streams around 1.5 GB

2016-06-16 Thread Michael Noll
consumption to be expected on these > test > > > cases? > > > > > > I'd like to run stand-alone from jar and I'll be able to do so by > > excluding > > > test cases. > > > > > > Fyi, on mailing list I see only 1 issue related to Streams and memory > if > > > that's any relevant (KAFKA-3738). > > > > > > Phil > > > > > > > > > > > -- > > -- Guozhang > > > -- Best regards, Michael Noll *Michael G. Noll | Product Manager | Confluent | +1 650.453.5860Download Apache Kafka and Confluent Platform: www.confluent.io/download <http://www.confluent.io/download>*

Re: Complexe Event Processing on top of KafkaStreams

2016-06-16 Thread Michael Noll
Thanks for sharing, Florian! -Michael On Thu, Jun 16, 2016 at 12:32 AM, Guozhang Wang wrote: > Hello Florian, > > Thanks for your interests! As mentioned in our release notes we are > considering to add the SQL support (e.g. using Calcite) on top of Kafka > Streams as a near-term future work. >

Re: Kafka Streams - production use

2016-07-06 Thread Michael Noll
Dirk, we included the note "be careful when using Kafka Streams in production" because Kafka Streams as shipped in Kafka 0.10.0.0 is the first-ever release of Kafka Streams. In practice, users are running Streams applications in a variety of stages -- some are doing pilots or evaluations, some ar

Re: NPE using GenericAvroSerde to deserialize

2016-07-06 Thread Michael Noll
gt; > I'd like to stream the values on topic as GenericRecord. I thought it > should work. Alternatively, guidance on using SpecificAvroSerde would be > very helpful. > -- Best regards, Michael Noll *Michael G. Noll | Product Manager | Confluent | +1 650.453.586

Re: [VOTE] KIP-67: Queryable state for Kafka Streams

2016-07-07 Thread Michael Noll
A+Queryable+state+for+Kafka+Streams > > > > > > > > > > > > > KAFKA-3909 <https://issues.apache.org/jira/browse/KAFKA-3909> is the > > top > > > > level JIRA for this effort. > > > > > > > > Initial PRs for Step

Re: [VOTE] KIP-67: Queryable state for Kafka Streams

2016-07-08 Thread Michael Noll
am Subramanian wrote: > >> > >> +1 > >> > >> On Thu, Jul 7, 2016 at 9:53 AM, Henry Cai > >> wrote: > >> > >>> +1 > >>> > >>> On Thu, Jul 7, 2016 at 6:48 AM, Michael Noll > wrote: > >>> > &g

Re: [VOTE] KIP-67: Queryable state for Kafka Streams

2016-07-08 Thread Michael Noll
ffix, for example). On Fri, Jul 8, 2016 at 6:37 PM, Michael Noll wrote: > Damian, > > about the rolling upgrade comment: An instance A will contact another > instance B by the latter's endpoint, right? So if A has no further > information available than B's host a

Re: [VOTE] KIP-67: Queryable state for Kafka Streams

2016-07-08 Thread Michael Noll
dn't shutdown properly, but then you have another > problem...). > > On Fri, 8 Jul 2016 at 15:26 Michael Noll wrote: > > > I have one further comment about `StreamsConfig.USER_ENDPOINT_CONFIG`. > > > > I think we should consider to not restricting the value of this

Re: [DISCUSS] KIP-66 Kafka Connect Transformers for messages

2016-07-12 Thread Michael Noll
KAFKA-3209 <https://issues.apache.org/jira/browse/KAFKA-3209>. > > >> It’s about capabilities to transform messages in Kafka Connect. > > >> > > >> Some design decisions need to be taken, so please advise me on the > same. > > >> Feel free to express any thoughts or concerns as well. > > >> > > >> Many many thanks to Ewen Cheslack-Postava. > > >> > > >> -Nisarg > > > > > -- Best regards, Michael Noll *Michael G. Noll | Product Manager | Confluent | +1 650.453.5860Download Apache Kafka and Confluent Platform: www.confluent.io/download <http://www.confluent.io/download>*

Re: [VOTE] KIP-67: Queryable state for Kafka Streams

2016-07-12 Thread Michael Noll
State, the bare minimum abstractions that we need to > expose > > are state stores and the location of state store partitions. > > > > For exposing the state stores, the getStore() APIs look good but I think > > for locating the state store partitions, we should go back t

Re: [VOTE] KIP-67: Queryable state for Kafka Streams

2016-07-12 Thread Michael Noll
gt;> > and perform any aggregation or processing yourself. This pattern is > >> > not recommended for all kinds of services/apps, but there are > >> definitely a > >> > category of things where it is a big win and other advanced > applic

Re: Kafka Streams question

2016-07-14 Thread Michael Noll
Poul, to add to what Matthias said: If you are wondering how to manually create a topic, you have basically two options. A. Use Kafka's CLI tools to create the topic "from the outside". # Example $ kafka-topics.sh --create --topic my-custom-toipc --zookeeper localhost:2181 --partitions

Re: Kafka Streams question

2016-07-14 Thread Michael Noll
e of Kafka Streams (v0.10.0.0). -Michael On Thu, Jul 14, 2016 at 10:32 AM, Michael Noll wrote: > Poul, > > to add to what Matthias said: If you are wondering how to manually create > a topic, you have basically two options. > > A. Use Kafka's CLI tools to c

Re: [DISCUSS] KIP-66 Kafka Connect Transformers for messages

2016-07-25 Thread Michael Noll
> > > > >> The original reason why I suggested a set based processing is > > > because > > > > > of the flexibility is provides. The JIRA had a comment by a user > > > > requesting > > > > > a feature that could be achieved with this. > &g

Re: Review Request 34641: Patch for KAFKA-2214

2015-05-26 Thread Michael Noll
and "failed". However, this is arguably more error prone because the log messages can change between Kafka versions (and oftentimes such changes are not prominently advertised, so you only notice this once your deployment script breaks). - Michael Noll O

Re: [ANNOUNCE] New committer: Jiangjie (Becket) Qin

2016-11-07 Thread Michael Noll
Congratulations, Becket! Best wishes, Michael On Thu, Nov 3, 2016 at 5:13 PM, Efe Gencer wrote: > Congratulations, Becket! > > Best, > Efe > > 2016-11-03 11:22 GMT-04:00 Neha Narkhede : > > > Congratulations, Becket! Well done. > > > > On Wed, Nov 2, 2016 at 1:34 PM Eno Thereska > > wrote: > >

Re: Data (re)processing with Kafka (new wiki page)

2016-11-25 Thread Michael Noll
Thanks a lot, Matthias! I have already begun to provide feedback. -Michael On Wed, Nov 23, 2016 at 11:41 PM, Matthias J. Sax wrote: > Hi, > > we added a new wiki page that is supposed to collect data (re)processing > scenario with Kafka: > > https://cwiki.apache.org/confluence/display/KAFKA/

Re: [DISCUSS] KIP-95: Incremental Batch Processing for Kafka Streams

2016-12-01 Thread Michael Noll
> Instead, why would we not have batch.mode=true/false? Matthias already replied in detail, but let me also throw in my thoughts: I'd argue that "there is no batch [mode]" (just like "there is no spoon" for the Matrix fans out there :-P). What would a batch mode look like? It is primarily defin

Re: [VOTE] KIP-94: Session Windows

2016-12-07 Thread Michael Noll
+1 (non-binding) On Wed, Dec 7, 2016 at 1:21 AM, Sriram Subramanian wrote: > +1 (binding) > > On Tue, Dec 6, 2016 at 3:43 PM, Ewen Cheslack-Postava > wrote: > > > +1 binding > > > > -Ewen > > > > On Tue, Dec 6, 2016 at 3:21 PM, Bill Bejeck wrote: > > > > > +1 > > > > > > On Tue, Dec 6, 2016 at

Re: [DISCUSS] KIP-99: Add Global Tables to Kafka Streams

2016-12-07 Thread Michael Noll
> There is no outer-join for GlobalKTables as the tables may be keyed > differently. So you need to use the key from the left side of the join > along with the KeyValueMapper to resolve the right side of the join. This > wont work the other way around. Care to elaborate why it won't work the other

Re: [DISCUSS] KIP-99: Add Global Tables to Kafka Streams

2016-12-07 Thread Michael Noll
-> t2 as in order to > perform the join we need to use a KeyValueMapper to extract the t2 key from > the t1 value. > > Does that make sense? > > Thanks, > Damian > > On Wed, 7 Dec 2016 at 10:44 Michael Noll wrote: > > > > There is no outer-join for Glo

Re: [DISCUSS] KIP-99: Add Global Tables to Kafka Streams

2016-12-08 Thread Michael Noll
gt; Thanks, > Damian > > On Wed, 7 Dec 2016 at 14:31 Michael Noll wrote: > > > Damian, > > > > yes, that makes sense. > > > > But I am still wondering: In your example, there's no prior knowledge > "can > > I map from t1->t2" that

Re: [DISCUSS] KIP-99: Add Global Tables to Kafka Streams

2016-12-09 Thread Michael Noll
he resulting table, > personDevice, what is the key? The person.id ? the device.id? it can't be > both of them. > > Thanks, > Damian > > > > > On Thu, 8 Dec 2016 at 15:51 Michael Noll wrote: > > > The key type returned by both KeyValueMappers (in the cur

Re: [VOTE] KIP-123: Allow per stream/table timestamp extractor

2017-02-28 Thread Michael Noll
Thanks, Jeyhun! I observed that there is still on ongoing conversation in the DISCUSS thread, so perhaps this voting thread was started a bit too early? -Michael On Tue, Feb 28, 2017 at 10:35 AM, Damian Guy wrote: > Thanks for the KIP Jeyhun! > > +1 > > On Tue, 28 Feb 2017 at 08:59 Jeyhun K

Re: [VOTE] KIP-123: Allow per stream/table timestamp extractor

2017-03-01 Thread Michael Noll
+1 (non-binding) Thanks for the KIP! On Wed, Mar 1, 2017 at 1:49 PM, Bill Bejeck wrote: > +1 > > Thanks > Bill > > On Wed, Mar 1, 2017 at 5:06 AM, Eno Thereska > wrote: > > > +1 (non binding). > > > > Thanks > > Eno > > > On 28 Feb 2017, at 17:22, Matthias J. Sax > wrote: > > > > > > +1 > > >

Re: groupBy without auto-repartition topics for Kafka Streams

2017-03-01 Thread Michael Noll
FYI: The difference between `groupBy` (may trigger re-partitioning) vs. `groupByKey` (does not trigger re-partitioning) also applies to: - `map` vs. `mapValues` - `flatMap` vs. `flatMapValues` On Wed, Mar 1, 2017 at 8:15 PM, Damian Guy wrote: > If you use stream.groupByKey() then there will b

Re: [DISCUSS] KIP-120: Cleanup Kafka Streams builder API

2017-03-09 Thread Michael Noll
Thanks for the update, Matthias. +1 to the points 1,2,3,4 you mentioned. Naming is always a tricky subject, but renaming KStreamBuilder to StreamsTopologyBuilder looks ok to me (I would have had a slight preference towards DslTopologyBuilder, but hey.) The most important aspect is, IMHO, what yo

Re: [DISCUSS] KIP-120: Cleanup Kafka Streams builder API

2017-03-14 Thread Michael Noll
I see Jay's point, and I agree with much of it -- notably about being careful which concepts we do and do not expose, depending on which user group / user type is affected. That said, I'm not sure yet whether or not we should get rid of "Topology" (or a similar term) in the DSL. For what it's wor

Re: Re: [DISCUSS] KIP-120: Cleanup Kafka Streams builder API

2017-03-20 Thread Michael Noll
d the topology if provided as a constructor argument. However, > >> especially for DSL (not sure if it would make sense for PAPI), the DSL > >> builder could create the client for the user. > >> > >> Something like this: > >> > >>> KStreamBuilder bu

Re: [DISCUSS] KIP-129: Kafka Streams Exactly-Once Semantics

2017-03-22 Thread Michael Noll
I second Eno's concern regarding the impact of Streams EOS on state stores. > We do a full recovery today and the EOS proposal will not make this any worse. Yes, today we do a full state store recovery under certain failures. However, I think the point (or perhaps: open question) is that, with t

Re: [DISCUSS] KIP-129: Kafka Streams Exactly-Once Semantics

2017-03-22 Thread Michael Noll
http://docs.confluent.io/current/streams/faq.html#handling-corrupted-records-and-deserialization-errors-poison-pill-messages On Wed, Mar 22, 2017 at 3:12 PM, Michael Noll wrote: > I second Eno's concern regarding the impact of Streams EOS on state stores. > > > We do a full

Re: [DISCUSS] KIP-120: Cleanup Kafka Streams builder API

2017-03-22 Thread Michael Noll
Forwarding to kafka-user. -- Forwarded message -- From: Michael Noll Date: Wed, Mar 22, 2017 at 8:48 AM Subject: Re: [DISCUSS] KIP-120: Cleanup Kafka Streams builder API To: dev@kafka.apache.org Matthias, > @Michael: > > You seemed to agree with Jay about not exp

Re: [DISCUSS] KIP-129: Kafka Streams Exactly-Once Semantics

2017-03-24 Thread Michael Noll
n this case. This is similar to how we are introducing > idempotent producer in core kafka with a config to turn it on. There is no > guarantee that the performance of the producer with the config turned on > will be the same although eventually we will like to get to it. > > On W

Re: [VOTE] KIP-129: Kafka Streams Exactly-Once Semantics

2017-03-29 Thread Michael Noll
+1 (non-binding) On Wed, Mar 29, 2017 at 6:38 PM, Sriram Subramanian wrote: > +1 > > On Wed, Mar 29, 2017 at 9:36 AM, Bill Bejeck wrote: > > > +1 (non-binding) > > > > Thanks Matthias, > > Bill > > > > On Wed, Mar 29, 2017 at 12:18 PM, Apurva Mehta > > wrote: > > > > > +1 (non-binding) > > > >

Re: [DISCUSS] KIP-114: KTable materialization and improved semantics

2017-04-11 Thread Michael Noll
t;>>>>>> Again, in either ways the API itself does not "hint" about > > anything > > >>>>>>>>> for > > >>>>>>>>> materializing a KTable or not at all; it is still purely > > determined > >

Re: [DISCUSS] KIP-138: Change punctuate semantics

2017-04-11 Thread Michael Noll
I apologize for the longer email below. To my defense, it started out much shorter. :-) Also, to be super-clear, I am intentionally playing devil's advocate for a number of arguments brought forth in order to help improve this KIP -- I am not implying I necessarily disagree with the arguments. T

Re: [VOTE] KIP-105: Addition of Record Level for Sensors

2017-01-09 Thread Michael Noll
+1 (non-binding) On Fri, Jan 6, 2017 at 6:12 PM, Matthias J. Sax wrote: > +1 > > On 1/6/17 9:09 AM, Neha Narkhede wrote: > > +1 > > > > On Fri, Jan 6, 2017 at 9:04 AM Sriram Subramanian > wrote: > > > >> +1 > >> > >> On Fri, Jan 6, 2017 at 8:40 AM, Bill Bejeck wrote: > >> > >>> +1 > >>> > >>>

Re: [DISCUSS] KIP-114: KTable materialization and improved semantics

2017-01-17 Thread Michael Noll
I think section "Compatibility, Deprecation, and Migration Plan" needs updating. It currently reads "No impact on existing users" but the KIP proposes to remove existing API methods (such as `KTable#foreach()`). -Michael On Tue, Jan 17, 2017 at 10:55 AM, Eno Thereska wrote: > Thanks Damian, a

Re: [DISCUSS] KIP-114: KTable materialization and improved semantics

2017-01-17 Thread Michael Noll
Thanks for the KIP, Eno. In section "Rejected Alternatives" the KIP says: > It is not clear that collapsing 2 abstractions helps. In particular, a KTable models a changelog. > That itself is a useful abstraction. A state store is a materialized view. That’s a distinct abstraction > with parallels

Re: [DISCUSS] KIP-114: KTable materialization and improved semantics

2017-01-17 Thread Michael Noll
> Rename toStream() to toKStream() for consistency. Not sure whether that is really required. We also use `KStreamBuilder#stream()` and `KStreamBuilder#table()`, for example, and don't care about the "K" prefix. On Tue, Jan 17, 2017 at 10:55 AM, Eno Thereska wrote: > Thanks Damian, answers in

Re: Fwd: [DISCUSS] KIP-114: KTable materialization and improved semantics

2017-01-27 Thread Michael Noll
t;>>>> afterwards >>>>> >>>>>> but we have already decided to materialize it, we can replace the >>>>>>>> >>>>>>> internal >>>>>>> >>>>>>>> name with the user's provided

Re: [DISCUSS] KIP-116 - Add State Store Checkpoint Interval Configuration

2017-02-06 Thread Michael Noll
> Starting on another host requires restoring from the earliest offset. Btw, there's a special scenario where a full restore is not required: When the local storage (volume) is being re-used, e.g. when a container uses a storage mount that will be re-used by a new container in case the original o

Re: KIP-121 [Discuss]: Add KStream peek method

2017-02-07 Thread Michael Noll
Many thanks for the KIP and the PR, Steven! My opinion, too, is that we should consider including this. One thing that I would like to see clarified is the difference between the proposed peek() and existing functions map() and foreach(), for instance. My understanding (see also the Java 8 links

Re: [DISCUSS] KIP-116 - Add State Store Checkpoint Interval Configuration

2017-02-09 Thread Michael Noll
Damian, could you elaborate briefly why the default value should be 5 minutes? What are the considerations, assumptions, etc. that go into picking this value? Right now, in the KIP and in this discussion, "5 mins" looks like a magic number to me. :-) -Michael On Thu, Feb 9, 2017 at 11:03 AM,

Re: KIP-121 [VOTE]: Add KStream peek method

2017-02-09 Thread Michael Noll
+1 (non-binding) And thanks again for suggesting and driving this, Steven! :-) Best, Michael On Thu, Feb 9, 2017 at 6:06 AM, Matthias J. Sax wrote: > +1 > > On 2/8/17 4:51 PM, Gwen Shapira wrote: > > +1 (binding) > > > > On Wed, Feb 8, 2017 at 4:45 PM, Steven Schlansker > > wrote: > >> Hi

Re: Streams: TTLCacheStore

2017-02-09 Thread Michael Noll
Thanks for bringing this up, Elias! FYI: A related idea we talked about is that of a timestamped state store (for the lack of a better description), where conceptually every entry would be a tuple of (key, value, timestamp). The use case was slightly different from what Elias describes: here, it

Re: Correct prefetching of data to KTable-like structure on application startup

2017-02-09 Thread Michael Noll
Jan, > - if I don't send any data to a kafka partition for a period longer then the data retention interval, then all data from the partition is wiped out If I interpret your first and second message in this email thread correctly, then you are talking only about your "state topic" here, i.e. th

Re: Review Request 34641: Patch for KAFKA-2214

2015-08-06 Thread Michael Noll
27;t have a strong opinion here as it's IMHO pretty common to ignore the advice/link above. In general the patch is going in the right direction. I only added two minor formatting issues and a clarification request. - Michael Noll On Aug. 5, 2015, 3:19 p.m., Manikumar Reddy O wrote: &g

Re: [VOTE] KIP:71 Enable log compaction and deletion to co-exist

2016-08-15 Thread Michael Noll
+1 (non-binding) On Mon, Aug 15, 2016 at 7:34 PM, Sriram Subramanian wrote: > +1 (binding) > > On Mon, Aug 15, 2016 at 10:27 AM, Ismael Juma wrote: > > > Thanks for the KIP, +1 (binding) > > > > Ismael > > > > On Mon, Aug 15, 2016 at 2:20 PM, Damian Guy > wrote: > > > > > I would like to initi

Re: [VOTE] KIP-77: Improve Kafka Streams Join Semantics

2016-08-31 Thread Michael Noll
+1 (non-binding) On Wed, Aug 31, 2016 at 6:18 AM, Neha Narkhede wrote: > +1 (binding) > > On Tue, Aug 30, 2016 at 5:33 PM Ewen Cheslack-Postava > wrote: > > > +1 (binding) > > > > I think the major gap I notice in the PR is a lack of docs updates to > > notify people of the change. Given these

Re: [VOTE] KIP-63: Unify store and downstream caching in streams

2016-08-31 Thread Michael Noll
+1 (non-binding) On Fri, Aug 26, 2016 at 3:10 PM, Bill Bejeck wrote: > +1 > > On Thu, Aug 25, 2016 at 11:45 AM, Matthias J. Sax > wrote: > > > +1 > > > > On 08/25/2016 04:22 PM, Damian Guy wrote: > > > +1 > > > > > > On Thu, 25 Aug 2016 at 11:57 Eno Thereska > > wrote: > > > > > >> Hi folks, >

Re: [ANNOUNCE] New committer: Jason Gustafson

2016-09-09 Thread Michael Noll
My compliments, Jason -- well deserved! :-) -Michael On Wed, Sep 7, 2016 at 6:49 PM, Grant Henke wrote: > Congratulations and thank you for all of your contributions to Apache > Kafka Jason! > > On Wed, Sep 7, 2016 at 10:12 AM, Mayuresh Gharat < > gharatmayures...@gmail.com > > wrote: > > > c

Re: Help related to KStreams and Stateful event processing.

2016-09-26 Thread Michael Noll
PS: Arvind, I think typically such questions should be sent to the kafka-user mailing list (not to kafka-dev). On Mon, Sep 26, 2016 at 3:41 AM, Matthias J. Sax wrote: > Hi Arvind, > > short answer, Kafka Streams does definitely help you! > > Long answer, Kafka Streams offers two layer to program

Re: Help related to KStreams and Stateful event processing.

2016-09-26 Thread Michael Noll
helps! Michael On Mon, Sep 26, 2016 at 11:02 AM, Michael Noll wrote: > PS: Arvind, I think typically such questions should be sent to the > kafka-user mailing list (not to kafka-dev). > > On Mon, Sep 26, 2016 at 3:41 AM, Matthias J. Sax > wrote: > >> Hi Arvind, &

Re: KStreams / add support for sink processor with dynamic topics

2016-10-20 Thread Michael Noll
Florian, I'd also have a follow-up question: > Actually, the set of subtypes is not known in advance this is why I need to > compute output topics from messages. So the branch method is of any help > in my context. For a number of reasons we recommend pre-creating input, intermediate, and output

Re: [VOTE] KIP-657: Add Customized Kafka Streams Logo

2020-08-19 Thread Michael Noll
Hi all! Great to see we are in the process of creating a cool logo for Kafka Streams. First, I apologize for sharing feedback so late -- I just learned about it today. :-) Here's my *personal, subjective* opinion on the currently two logo candidates for Kafka Streams. TL;DR: Sorry, but I really

Re: [VOTE] KIP-657: Add Customized Kafka Streams Logo

2020-08-19 Thread Michael Noll
o: us...@kafka.apache.org > > Cc: dev@kafka.apache.org > > Subject: Re: [VOTE] KIP-657: Add Customized Kafka Streams Logo > > > > I echo what Michael says here. > > > > Another consideration is that logos are often shrunk (when used on > slides) > > an

Re: [ANNOUNCE] New committer: Rajini Sivaram

2017-04-24 Thread Michael Noll
Congratulations, Rajini! On Mon, Apr 24, 2017 at 11:50 PM, Ismael Juma wrote: > Congrats Rajini! Well-deserved. :) > > Ismael > > On Mon, Apr 24, 2017 at 10:06 PM, Gwen Shapira wrote: > > > The PMC for Apache Kafka has invited Rajini Sivaram as a committer and we > > are pleased to announce tha

Re: [VOTE] KIP-123: Allow per stream/table timestamp extractor

2017-04-25 Thread Michael Noll
Thanks for your work and for driving this, Jeyhun! :-) -Michael On Tue, Apr 25, 2017 at 11:11 PM, Jeyhun Karimov wrote: > Dear all, > > I am closing this vote now. The KIP got accepted with > > +3 binding (Guozhang, Ewen, Gwen) > > Thanks all (especially for Mathias) for guiding me throughout

Re: [ANNOUNCE] New Committer Sriharsha Chintalapani

2015-09-25 Thread Michael Noll
Congratulations, Harsha! :-) -Michael > On 22 Sep 2015, at 06:10, Jun Rao wrote: > > I am pleased to announce that the Apache Kafka PMC has voted to > invite Sriharsha Chintalapani as a committer and Sriharsha has accepted. > > Sriharsha has contributed numerous patches to Kafka. The most si

[jira] [Created] (KAFKA-6871) KStreams Scala API: incorrect Javadocs and misleading parameter name

2018-05-07 Thread Michael Noll (JIRA)
Michael Noll created KAFKA-6871: --- Summary: KStreams Scala API: incorrect Javadocs and misleading parameter name Key: KAFKA-6871 URL: https://issues.apache.org/jira/browse/KAFKA-6871 Project: Kafka

[jira] [Created] (KAFKA-5636) DSL: allow sliding windows to be used directly (i.e. not just implicitly when doing joins)

2017-07-25 Thread Michael Noll (JIRA)
Michael Noll created KAFKA-5636: --- Summary: DSL: allow sliding windows to be used directly (i.e. not just implicitly when doing joins) Key: KAFKA-5636 URL: https://issues.apache.org/jira/browse/KAFKA-5636

[jira] [Commented] (KAFKA-3478) Finer Stream Flow Control

2016-05-30 Thread Michael Noll (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15306409#comment-15306409 ] Michael Noll commented on KAFKA-3478: - Latest documentation link (covers flow con

[jira] [Updated] (KAFKA-3805) Running multiple instances of a Streams app on the same machine results in Java_org_rocksdb_RocksDB_write error

2016-06-08 Thread Michael Noll (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Noll updated KAFKA-3805: Summary: Running multiple instances of a Streams app on the same machine results in

[jira] [Updated] (KAFKA-3478) Finer Stream Flow Control

2016-07-05 Thread Michael Noll (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Noll updated KAFKA-3478: Description: Today we have a event-time based flow control mechanism in order to synchronize

[jira] [Commented] (KAFKA-3973) Investigate feasibility of caching bytes vs. records

2016-07-25 Thread Michael Noll (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15391519#comment-15391519 ] Michael Noll commented on KAFKA-3973: - +1 to having repeatable tests/benchmarks

[jira] [Created] (KAFKA-2214) kafka-reassign-partitions.sh --verify should return non-zero exit codes when reassignment is not completed yet

2015-05-21 Thread Michael Noll (JIRA)
Michael Noll created KAFKA-2214: --- Summary: kafka-reassign-partitions.sh --verify should return non-zero exit codes when reassignment is not completed yet Key: KAFKA-2214 URL: https://issues.apache.org/jira/browse

[jira] [Updated] (KAFKA-2214) kafka-reassign-partitions.sh --verify should return non-zero exit codes when reassignment is not completed yet

2015-05-21 Thread Michael Noll (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Noll updated KAFKA-2214: Description: h4. Background The admin script {{kafka-reassign-partitions.sh}} should integrate

[jira] [Updated] (KAFKA-2214) kafka-reassign-partitions.sh --verify should return non-zero exit codes when reassignment is not completed yet

2015-05-22 Thread Michael Noll (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Noll updated KAFKA-2214: Description: h4. Background The admin script {{kafka-reassign-partitions.sh}} should

[jira] [Created] (KAFKA-4836) Kafka Streams app reset tool: support wildcards for topics

2017-03-03 Thread Michael Noll (JIRA)
Michael Noll created KAFKA-4836: --- Summary: Kafka Streams app reset tool: support wildcards for topics Key: KAFKA-4836 URL: https://issues.apache.org/jira/browse/KAFKA-4836 Project: Kafka Issue

[jira] [Commented] (KAFKA-4875) Kafka streams topic groups and builder.stream API

2017-03-09 Thread Michael Noll (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15902975#comment-15902975 ] Michael Noll commented on KAFKA-4875: - There two things at play here IMHO

[jira] [Updated] (KAFKA-4875) Kafka Streams: topic groups and builder.stream API

2017-03-09 Thread Michael Noll (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Noll updated KAFKA-4875: Summary: Kafka Streams: topic groups and builder.stream API (was: Kafka streams topic groups and

[jira] [Commented] (KAFKA-4835) Allow users control over repartitioning

2017-03-10 Thread Michael Noll (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15904709#comment-15904709 ] Michael Noll commented on KAFKA-4835: - Just to clarify: {quote} However, in our

[jira] [Commented] (KAFKA-4880) Add KTable/GlobalKTable Join to Streams

2017-03-10 Thread Michael Noll (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15904745#comment-15904745 ] Michael Noll commented on KAFKA-4880: - FWIW, I wonder how much interest t

[jira] [Commented] (KAFKA-4609) KTable/KTable join followed by groupBy and aggregate/count can result in incorrect results

2017-03-14 Thread Michael Noll (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15924093#comment-15924093 ] Michael Noll commented on KAFKA-4609: - Care to elaborate [~damianguy]? Why does

[jira] [Updated] (KAFKA-4791) Kafka Streams - unable to add state stores when using wildcard topics on the source

2017-03-21 Thread Michael Noll (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Noll updated KAFKA-4791: Affects Version/s: 0.10.2.0 > Kafka Streams - unable to add state stores when using wildcard top

[jira] [Commented] (KAFKA-4614) Long GC pause harming broker performance which is caused by mmap objects created for OffsetIndex

2017-01-12 Thread Michael Noll (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15820875#comment-15820875 ] Michael Noll commented on KAFKA-4614: - [~kawamuray] Sorry to hear that you ran

[jira] [Commented] (KAFKA-3745) Consider adding join key to ValueJoiner interface

2017-01-12 Thread Michael Noll (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15821054#comment-15821054 ] Michael Noll commented on KAFKA-3745: - I think [~mjsax] meant to say "not a

[jira] [Commented] (KAFKA-3705) Support non-key joining in KTable

2017-01-17 Thread Michael Noll (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826064#comment-15826064 ] Michael Noll commented on KAFKA-3705: - [~jfilipiak]: Now that support for gl

[jira] [Comment Edited] (KAFKA-3705) Support non-key joining in KTable

2017-01-17 Thread Michael Noll (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826064#comment-15826064 ] Michael Noll edited comment on KAFKA-3705 at 1/17/17 1:4

[jira] [Created] (KAFKA-4769) Add Float serializer, deserializer, serde

2017-02-15 Thread Michael Noll (JIRA)
Michael Noll created KAFKA-4769: --- Summary: Add Float serializer, deserializer, serde Key: KAFKA-4769 URL: https://issues.apache.org/jira/browse/KAFKA-4769 Project: Kafka Issue Type

[jira] [Updated] (KAFKA-4769) Add Float serializer, deserializer, serde

2017-02-15 Thread Michael Noll (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Noll updated KAFKA-4769: Status: Patch Available (was: Open) > Add Float serializer, deserializer, se

[jira] [Created] (KAFKA-3411) Streams: rename job.id to application.id

2016-03-16 Thread Michael Noll (JIRA)
Michael Noll created KAFKA-3411: --- Summary: Streams: rename job.id to application.id Key: KAFKA-3411 URL: https://issues.apache.org/jira/browse/KAFKA-3411 Project: Kafka Issue Type: Improvement

[jira] [Updated] (KAFKA-3411) Streams: stop using "job" terminology, rename job.id to application.id

2016-03-16 Thread Michael Noll (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Noll updated KAFKA-3411: Summary: Streams: stop using "job" terminology, rename job.id to application.id (wa

[jira] [Commented] (KAFKA-3153) Serializer/Deserializer Registration and Type inference

2016-03-19 Thread Michael Noll (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15201131#comment-15201131 ] Michael Noll commented on KAFKA-3153: - [~guozhang], [~yasuhiro.matsuda]: What a

[jira] [Created] (KAFKA-3449) Rename filterOut() to filterNot() to achieve better terminology

2016-03-23 Thread Michael Noll (JIRA)
Michael Noll created KAFKA-3449: --- Summary: Rename filterOut() to filterNot() to achieve better terminology Key: KAFKA-3449 URL: https://issues.apache.org/jira/browse/KAFKA-3449 Project: Kafka

[jira] [Updated] (KAFKA-3449) Rename filterOut() to filterNot() to achieve better terminology

2016-03-23 Thread Michael Noll (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Noll updated KAFKA-3449: Fix Version/s: 0.10.0.1 > Rename filterOut() to filterNot() to achieve better terminol

[jira] [Commented] (KAFKA-3511) Provide built-in aggregators sum() and avg() in Kafka Streams DSL

2016-04-11 Thread Michael Noll (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15234770#comment-15234770 ] Michael Noll commented on KAFKA-3511: - Hmm. I'd prefer an API that all

[jira] [Comment Edited] (KAFKA-3511) Provide built-in aggregators sum() and avg() in Kafka Streams DSL

2016-04-11 Thread Michael Noll (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15234770#comment-15234770 ] Michael Noll edited comment on KAFKA-3511 at 4/11/16 9:0

[jira] [Comment Edited] (KAFKA-3511) Provide built-in aggregators sum() and avg() in Kafka Streams DSL

2016-04-11 Thread Michael Noll (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15234770#comment-15234770 ] Michael Noll edited comment on KAFKA-3511 at 4/11/16 9:0

[jira] [Comment Edited] (KAFKA-3511) Provide built-in aggregators sum() and avg() in Kafka Streams DSL

2016-04-11 Thread Michael Noll (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15234770#comment-15234770 ] Michael Noll edited comment on KAFKA-3511 at 4/11/16 9:0

[jira] [Commented] (KAFKA-3511) Provide built-in aggregators sum() and avg() in Kafka Streams DSL

2016-04-11 Thread Michael Noll (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15235113#comment-15235113 ] Michael Noll commented on KAFKA-3511: - PS: As [~guozhang] hinted at, we should

[jira] [Comment Edited] (KAFKA-3511) Provide built-in aggregators sum() and avg() in Kafka Streams DSL

2016-04-11 Thread Michael Noll (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15235113#comment-15235113 ] Michael Noll edited comment on KAFKA-3511 at 4/11/16 1:5

[jira] [Commented] (KAFKA-3499) byte[] should not be used as Map key nor Set member

2016-04-11 Thread Michael Noll (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15235799#comment-15235799 ] Michael Noll commented on KAFKA-3499: - FWIW, we ran into the same problem

  1   2   >