Re: Accessing Topology Builder

ned a previous operator, but what about: > `peek().mapValues().peek()`, will both `peek`s be in the same thread as > well? > > El mar., 25 sept. 2018 a las 23:14, Matthias J. Sax () > escribió: > >> Just for clarification: >> >> `peek()` would run on the same threa

Re: [VOTE] KIP-369 Alternative Partitioner to Support "Always Round-Robin" Selection

+1 (binding) @Abhimanyu: can you please update the table in https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals and add a link to the KIP. Thanks. -Matthias On 9/4/18 9:56 PM, Abhimanyu Nagrath wrote: > +1 > > On Wed, Sep 5, 2018 at 2:39 AM Magesh Nandakumar > wrote:

Re: [DISCUSS]: KIP-230: Name Windowing Joins

18 5:24 PM, Matthias J. Sax wrote: > What is the status of this KIP? > > I also want to point out, that it seems that there is some overlap with > KIP-307 that was proposed recently (I cc'ed Florian who proposed > KIP-307). KIP-230 might even be subsumed. > > I any c

Re: [DISCUSS] KIP-221: Repartition Topic Hints in Streams

feature and would like to > move it forward if no one else is working on it. > > Lei > > On Wed, Jun 20, 2018 at 7:27 PM Matthias J. Sax > wrote: > >> No worries. It's just good to know. It seems that some other people are >> interested to drive this

Re: [VOTE] KIP-232: Detect outdated metadata using leaderEpoch and partitionEpoch

It seems that KIP-320 was accepted. Thus, I am wondering what the status of this KIP is? -Matthias On 7/11/18 10:59 AM, Dong Lin wrote: > Hey Jun, > > Certainly. We can discuss later after KIP-320 settles. > > Thanks! > Dong > > > On Wed, Jul 11, 2018 at 8:54 AM, Jun Rao wrote: > >> Hi, Don

Re: [DISCUSS] KIP-333 Consider a faster form of rebalancing

What is the status of this KIP? I was just catching up and I agree with Becket that it seems a very special use case, that might not be generic enough to be part of Kafka itself. Also, for regular rebalance, as Becket pointed out, catching up should not take very long. Only for longer offline time

Re: [DISCUSS] KIP-280: Enhanced log compaction

Luis, What is the status of this KIP? I tend to agree, that introducing the feature only globally, might be less useful (I would assume that people want to turn it on, on a per-topic basis). As I am not familiar with the corresponding code, I cannot judge the complexity to add topic level configs

Re: [VOTE] KIP-349 Priorities for Source Topics

+1 (binding) As Dongjin pointed out, the community is working on upcoming 2.1 release, and thus it might take some time until people find time to follow up on this an vote. -Matthias On 9/30/18 11:11 AM, n...@afshartous.com wrote: > >> On Sep 30, 2018, at 5:16 AM, Dongjin Lee wrote: >> >> 1.

Re: [DISCUSSION] KIP-376: Implement AutoClosable on appropriate classes that has close()

Closeable is part of `java.io` while AutoClosable is part of `java.lang`. Thus, the second one is more generic. Also, JavaDoc points out that `Closable#close()` must be idempotent while `AutoClosable#close()` can have side effects. Thus, I am not sure atm which one suits better. However, it's a g

Re: [VOTE] KIP-232: Detect outdated metadata using leaderEpoch and partitionEpoch

IP list wiki page and the status has been > updated in the discussion and voting email thread. > > Thanks, > Dong > > On Sun, 30 Sep 2018 at 11:51 AM Matthias J. Sax > wrote: > >> It seems that KIP-320 was accepted. Thus, I am wondering what the status >> of thi

Re: Apache Kafka project charter

I am not aware of anything like this. And I also think, it's difficult to generalize. So far, each feature is discussed on a per-case basis. Because it's hard to draw the boarder line we might be too restrictive or too loose in a "project charter", thus, scaring people from starting KIPs, what wou

Re: Request for contributor permissions

Done. On 9/30/18 11:02 PM, 卓正欣 wrote: > Hello, > > I want to join Kafka contributors, my relative information : > > JIRA ID : saivirtue > cwiki ID : Sam Cho > > Thanks for help! > > --- > signature.asc Description: OpenPGP digital signature

Re: Edit permissions for https://cwiki.apache.org/confluence/display/KAFKA/Ecosystem

Could not find your user -- please provide your wiki user ID so we can grant permissions to edit wiki pages. If you don't have an account yet, you can just create one. -Matthias On 10/2/18 1:33 PM, Bill Mclane wrote: > Hi can you either enable edit permissions for wmcl...@tibco.com >

Re: Re run build of a pull request

We don't know why it fails yet. We are investigating. If anyone knows, please share so we can fix it :) -Matthias On 10/3/18 9:07 AM, Kamal Chandraprakash wrote: > No need to push a commit to run the test case. Type "retest this please" as > comment. > It will again run all the test cases. > >

Re: Edit permissions for https://cwiki.apache.org/confluence/display/KAFKA/Ecosystem

Done On 10/2/18 6:36 PM, Bill Mclane wrote: > User ID should be wmclane > > Thanks Matthias... > > Bill— > > William P. McLane > Messaging Evangelist > TIBCO Software > >> On Oct 2, 2018, at 8:31 PM, Matthias J. Sax wrote: >> >> Could not find

Re: [DISCUSSION] KIP-376: Implement AutoClosable on appropriate classes that has close()

; On Sun, Sep 30, 2018, at 13:19, Matthias J. Sax wrote: >> Closeable is part of `java.io` while AutoClosable is part of >> `java.lang`. Thus, the second one is more generic. Also, JavaDoc points >> out that `Closable#close()` must be idempotent while >> `AutoClosabl

Re: Incremental Cooperative Rebalancing

Thanks for sharing! Excellent write up! -Matthias On 10/2/18 4:22 PM, Konstantine Karantasis wrote: > Hey everyone, > > I'd like to bring to your attention a general design document that was just > published in Apache Kafka's wiki space: > > https://cwiki.apache.org/confluence/display/KAFKA/Inc

Re: Request for contributor permissions

Done On 10/4/18 3:06 AM, 張雅涵 wrote: > Hi, I'd like to request for contributor permission. > > Thank you very much!! > > Jira ID: littleskyqueen > > cwiki ID: 張雅涵 littleskyqueen > > > *Yvonne Chang 張雅涵* > *亦思科技專業資訊服務團隊* > 看見新世代資料庫---*HareDB* > Tel：03-563

Re: integration between pull request and JIRA

IRA > after sending out PR - otherwise he / she may miss comments on the JIRA > (but not on PR). > > On Wed, Sep 6, 2017 at 12:27 PM, Matthias J. Sax > wrote: > >> You can subscribe to single PR if you want, too. (That actually happens, >> when you get tagged or comment on

Re: [DISCUSS] KIP-205: Add getAllKeys() API to ReadOnlyWindowStore

Thanks for driving this and sorry for late response. With release deadline it was pretty busy lately. Can you please add a description for the suggested method, what they are going to return? It's a little unclear to me atm. It would also be helpful to discuss, for which use case each method is u

Re: [DISCUSS] KIP-171: Extend Consumer Group Reset Offset for Stream Application

age otherwise it would be "unrecognized config" blah. >> >> >> If you feel confident about the wiki after discussing about these points, >> please feel free to move on to start a voting thread. Note that we are >> about 3 weeks away from KIP deadline and 4 w

Re: [DISCUSS] KIP-171: Extend Consumer Group Reset Offset for Stream Application

eter` as part of `core` module until we > have `ConsumerGroupCommand` on `client` module as well. > > El vie., 6 oct. 2017 a las 0:05, Matthias J. Sax () > escribió: > >> Jorge, >> >> KIP-198 (that got merged already) overlaps with this KIP. Can you please >>

Re: [DISCUSS] KIP-147: Add missing type parameters to StateStoreSupplier factories and KGroupedStream/Table methods

backward compatibility. >>> >>> Thanks, >>> Damian >>> >>> On Fri, 2 Jun 2017 at 08:20 Michal Borowiecki >>> >> <mailto:michal.borowie...@openbet.com>> wrote: >>> >>> Thanks Matthias

Re: [DISCUSS] KIP-165: Extend Interactive Queries for return latest update timestamp per key

text not be sufficient? >> >> Thanks for correction, this is my bad. >> >> - for backward compatibility, we will also need a new interface and >>> cannot just extend the existing one >> >> I will correct the KIP accordingly. >> >&

Re: Randomly failing tests

Hi, this can happen and we try hard to keep the build stable... If you see any failing tests, please double check if there is a corresponding Jira for those flaky tests and comment on the Jira (to raise awareness), reopen (if it's an older and resolved one) or create a new Jira. Thanks! -Matthi

Re: [VOTE] KIP-171 - Extend Consumer Group Reset Offset for Stream Application

+1 On 9/11/17 3:04 PM, Jorge Esteban Quilcate Otoya wrote: > Hi All, > > It seems that there is no further concern with the KIP-171. > At this point we would like to start the voting process. > > The KIP can be found here: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-171+-+Extend+Co

Re: [DISCUSS] KIP-210: Provide for custom error handling when Kafka Streams fails to produce

Thanks for the KIP. Are you familiar with KIP-161? https://cwiki.apache.org/confluence/display/KAFKA/KIP-161%3A+streams+deserialization+exception+handlers I thinks, we should align the design (parameter naming, return types, class names etc) of KIP-210 to KIP-161 to get a unified user experience

Re: [DISCUSS]: KIP-159: Introducing Rich functions to Streams

n Karimov < >>>> je.kari...@gmail.com >>>>>> >>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Hi all, >>>>>>>>>>> >>>>>>>>>>> Thanks a lot for y

Re: [DISCUSS]: KIP-159: Introducing Rich functions to Streams

reams > [2] > http://search-hadoop.com/m/Kafka/uyzND1kpct22GKKXT1?subj=Re+DISCUSS+KIP+159+Introducing+Rich+functions+to+Streams > [3] > http://search-hadoop.com/m/Kafka/uyzND1G6TGIGKKXT1?subj=Re+DISCUSS+KIP+159+Introducing+Rich+functions+to+Streams > > > On Mon, Oct 23, 2017 at 8:4

Re: [DISCUSS] KIP-213 Support non-key joining in KTable

Thanks a lot for the KIP. Can we please move the discussion to the dev list? Thus, after fixing the KIP collision, just start a new DISCUSS thread. Thx. -Matthias On 10/25/17 4:20 PM, Ted Yu wrote: > Have you seen the email a moment ago from Onur which uses the same KIP > number ? > > Looks l

Re: [DISCUSS]: KIP-159: Introducing Rich functions to Streams

tImpl` is also changed to >> this call. >> >> 2. Add the `task` reference to the impl class, `ProcessorRecordContext`, so >> that it can implement the commit call itself. >> >> 3. In the wiki page, the statement that "However, call to a commit() >> meth

Re: [DISCUSS]: KIP-159: Introducing Rich functions to Streams

n/discussion of KIP/included > jiras please let me know. > > > Cheers, > Jeyhun > > > On Fri 27. Oct 2017 at 12:39, Matthias J. Sax wrote: > >> I am personally still not convinced, that we should add `commit()` at all. >> >> @Guozhang: you created the o

Re: [DISCUSS] KIP-220: Add AdminClient into Kafka Streams' ClientSupplier

I like this KIP. Can you also link to https://issues.apache.org/jira/browse/KAFKA-6126 in the KIP? What I am wondering though: if we start to partially (ie, step by step) replace the existing StreamsKafkaClient with Java AdminClient, don't we need more KIPs? For example, if we use purge-api for in

Re: [DISCUSS]KIP-216: IQ should throw different exceptions for different errors

Thanks for the KIP Vito! I agree with what Guozhang said. The original idea of the Jira was, to give different exceptions for different "recovery" strategies to the user. For example, if a store is currently recreated, a user just need to wait and can query the store later. On the other hand, if

Re: [DISCUSS] KIP-210: Provide for custom error handling when Kafka Streams fails to produce

rd>>>> extends >>>>>>>> Object, ? extends Object>`? >>>>>>>> >>>>>>>> Also, should the handle function include the `RecordMetadata` as >>>>> well in >>>>>>>> case it

Re: [VOTE] KIP-210: Provide for custom error handling when Kafka Streams fails to produce

Yes. A KIP needs 3 binding "+1" to be accepted. You can still work on the PR and get it ready to get merged -- I am quite confident that this KIP will be accepted :) -Matthias On 11/4/17 3:56 PM, Matt Farmer wrote: > Bump! I believe I need two more binding +1's to proceed? > > On Thu, Nov 2, 2

Re: [DISCUSS] KIP-221: Repartition Topic Hints in Streams

Thanks for the KIP. Two comments/questions: - how does this relate to `KStream#through()` ? - do we really need to add new overload or can we just extends exiting options (cf KIP-182); it would be create if we could just extend existing classes like `Produced` instead of adding new methods to "

Re: [DISCUSS] KIP-221: Repartition Topic Hints in Streams

My idea was, to relax the requirement for through() that a topic must be created manually before startup. Thus, if no through() call is made, a (internal) topic is created the same way we do it currently. If one uses `through(String topicName)` we keep the current behavior and require users to cr

Re: [DISCUSS] KIP-221: Repartition Topic Hints in Streams

thias On 11/6/17 4:45 PM, Thomas Becker wrote: > I think this sounds good as well. It's worth clarifying whether topics that > are named by the user but created by streams are considered "internal" topics > also. > > On Sun, 2017-11-05 at 23:02 +0100, Matthias J. Sa

Re: [DISCUSS] KIP-220: Add AdminClient into Kafka Streams' ClientSupplier

well. > > > Guozhang > > > On Sat, Nov 4, 2017 at 9:00 AM, Matthias J. Sax > wrote: > >> I like this KIP. Can you also link to >> https://issues.apache.org/jira/browse/KAFKA-6126 in the KIP? >> >> What I am wondering though: if we start to partiall

Re: [DISCUSS] KIP-213 Support non-key joining in KTable

Jan, thanks a lot for this KIP. I did an initial pass over it, but feel a little lost. Maybe I need to read it more carefully, but atm it's not clear to me at all what algorithm you propose. I think it would be super helpful, to do an example with concrete data that show how records are stored, w

Re: [DISCUSS]: KIP-159: Introducing Rich functions to Streams

. It simplified the >>> overall design of KIP a lot. >>> If it is ok, I would like to start a VOTE thread. >>> >>> Cheers, >>> Jeyhun >>> >>> On Fri, Oct 27, 2017 at 5:28 PM Matthias J. Sax >>> wrote: >>> >>

Re: [VOTE] KIP-159: Introducing Rich functions to Streams

Jan, I understand what you are saying. However, having a RecordContext is super useful for operations that are applied to input topic. Many users requested this feature -- it's much more convenient that falling back to transform() to implement a a filter() for example that want to access some meta

Re: [ANNOUNCE] New committer: Onur Karaman

Congrats!!! On 11/6/17 7:56 PM, Vahid S Hashemian wrote: > Congrats Onur! > > --Vahid > > > > From: Ismael Juma > To: dev@kafka.apache.org > Cc: "us...@kafka.apache.org" > Date: 11/06/2017 10:13 AM > Subject:Re: [ANNOUNCE] New committer: Onur Karaman > Sent by:is

Re: [DISCUSS] KIP-221: Repartition Topic Hints in Streams

om name is actually a requirement. If the goal is to control > repartitioning and tune parallelism, maybe we can just sidestep this issue > altogether by removing the ability to set a different name. > > On Mon, 2017-11-06 at 16:51 +0100, Matthias J. Sax wrote: > > That

Re: [DISCUSS] KIP-222 - Add "describe consumer group" to KafkaAdminClient

The main goal of this KIP is to enable decoupling StreamsResetter from core module. For this case (ie, using AdminClient within StreamsResetter) we get the group.id from the user as command line argument. Thus, I think the KIP is useful without "describe group" command to. I am happy to include "d

Re: [DISCUSS] KIP-213 Support non-key joining in KTable

t Jan >>> >>> On 06.11.2017 21:20, Jan Filipiak wrote: >>> >>>> Will do! Need to do it carefully. One mistake in this detailed approach >>>> and confusion is perfect ;) >>>> Hope I can deliver this week. >>>> >>>>

Re: [DISCUSS] KIP-220: Add AdminClient into Kafka Streams' ClientSupplier

's > API UnsupportedVersionException throwable exceptions. Please share your > thoughts about this. > > 2) we create a one-time NetworkClient upon starting up, send the > ApiVersionsRequest and get the response and do the checking; after that > throw this client away.

Re: [VOTE] KIP-159: Introducing Rich functions to Streams

nd, rather than a direct solution. >>> >>> Perhaps we should continue this discussion in DISCUSS thread. >>> >>> >>> Cheers, >>> Jeyhun >>> >>> >>> On Mon, Nov 6, 2017 at 9:14 PM Jan Filipiak >>> wrote: >>

[DISCUSS] KIP-224: Add configuration parameters `retries` and `retry.backoff.ms` to Streams API

Hi, I want to propose a new KIP to make Streams API more resilient to broker disconnections. https://cwiki.apache.org/confluence/display/KAFKA/KIP-224%3A+Add+configuration+parameters+%60retries%60+and+%60retry.backoff.ms%60+to+Streams+API -Matthias signature.asc Description: OpenPGP digital

Re: [DISCUSS] KIP-224: Add configuration parameters `retries` and `retry.backoff.ms` to Streams API

Matthias, +1 from me. >>>> >>>> >>>> -Bill >>>> >>>> On Thu, Nov 9, 2017 at 8:40 AM, Ted Yu wrote: >>>> >>>>> lgtm >>>>> >>>>> bq. pass both parameter >>>>> >

Re: [DISCUSS] KIP-224: Add configuration parameters `retries` and `retry.backoff.ms` to Streams API

One more change: as parameter is called "retries", default value should be zero (instead of one). -Matthias On 11/10/17 1:14 PM, Matthias J. Sax wrote: > Thanks for the feedback. Typos fixed. > > Damian explained already why we need the new strategy. > > @Kamal: many

Re: [VOTE] KIP-159: Introducing Rich functions to Streams

v >>>> wrote: >>>> >>>> Hi Jan, >>>>> Sorry for late reply. >>>>> >>>>> >>>>> The API Design doesn't look appealing >>>>> >>>>> >>>>> In terms of API desi

Re: [DISCUSS] KIP-210: Provide for custom error handling when Kafka Streams fails to produce

>>> >>>>> On Sat, Nov 4, 2017 at 1:03 PM Matt Farmer wrote: >>>>> >>>>>> Ah. I actually created both of those in the PR and forgot to >> mention >>>> them >>>>>> by name in the KIP! Thanks for pointing out the ov

Re: [DISCUSS] KIP-210: Provide for custom error handling when Kafka Streams fails to produce

to call the handle but Streams should always fail. If we follow this design, the KIP should list all exceptions for which the handler is not called. WDYT? -Matthias On 11/10/17 2:56 PM, Matthias J. Sax wrote: > Just catching up on this KIP. > > One tiny comment: I would prefer to r

Re: [DISCUSS] KIP-147: Add missing type parameters to StateStoreSupplier factories and KGroupedStream/Table methods

I updated this KIP as discarded. On 10/13/17 10:34 AM, Matthias J. Sax wrote: > Hi, > > with KIP-182 being implemented in 1.0 that will be released shortly, I > wanted to follow up on this KIP. It seems, that we don't need it any > longer, as KIP-182 resolves those issue

Re: [DISCUSS] KIP-224: Add configuration parameters `retries` and `retry.backoff.ms` to Streams API

arameter for state lock retries. >> >> Would there ever be a case where you'd need to have them be different >> values? >> >> On Fri, Nov 10, 2017 at 4:17 PM, Ted Yu wrote: >> >>> bq. it would be worth to reuse both parameters for those >>> &

[VOTE] KIP-224: Add configuration parameters `retries` to Streams API

Hi @all, I would like to start the vote for KIP-224: https://cwiki.apache.org/confluence/display/KAFKA/KIP-224%3A+Add+configuration+parameter+%60retries%60+to+Streams+API Thanks a lot! -Matthias signature.asc Description: OpenPGP digital signature

Re: [DISCUSS] KIP-220: Add AdminClient into Kafka Streams' ClientSupplier

e will throw the exception at the first >>> rebalance, but I can still see the value of option 2). Ideally we can >> have >>> some APIs from AdminClient to check API versions but this does not exist >>> today and I do not want to drag too long with growing scope on this KIP, &g

Re: [DISCUSS] KIP-220: Add AdminClient into Kafka Streams' ClientSupplier

lient only. Similar to producer/consumer. Can we add this to the KIP? -Matthias On 11/14/17 2:51 PM, Matthias J. Sax wrote: > Thanks for looking into this into details! > > As mentioned, I would like to keep the check, but if it's too much > overhead, I agree that it's not worth

Re: [VOTE] KIP-159: Introducing Rich functions to Streams

> > Thank you for reconsideration > > Best Jan > > > On 10.11.2017 22:48, Matthias J. Sax wrote: >> I was thinking about the source stream/table idea once more and it seems >> it would not be too hard to implement: >> >> We add two new classes >>

Re: [DISCUSS] KIP-221: Repartition Topic Hints in Streams

oon based on the discussion and inform. >>>> >>>> >>>> Cheers, >>>> Jeyhun >>>> >>>> On Mon, Nov 6, 2017 at 9:18 PM Jan Filipiak >>>> wrote: >>>> >>>>> Sorry for not beeing 100% up to dat

Re: [DISCUSS] KIP-213 Support non-key joining in KTable

Hi, I am just catching up on this discussion and did re-read the KIP and discussion thread. In contrast to you, I prefer the second approach with CombinedKey as return type for the following reasons: 1) the oneToManyJoin() method had less parameter 2) those parameters are easy to understand 3

Re: [VOTE] KIP-159: Introducing Rich functions to Streams

ed > context anymore"? > > > Guozhang > > > On Thu, Nov 16, 2017 at 1:03 PM, Matthias J. Sax > wrote: > >> Any thoughts about my latest proposal? >> >> -Matthias >> >> On 11/10/17 10:02 PM, Jan Filipiak wrote: >>> Hi, >>> >>

Re: [VOTE] KIP-220: Add AdminClient into Kafka Streams' ClientSupplier

+1 On 11/17/17 9:35 AM, Ted Yu wrote: > +1 > > On Fri, Nov 17, 2017 at 9:34 AM, Bill Bejeck wrote: > >> +1 >> >> Thanks, >> Bill >> >> On Fri, Nov 17, 2017 at 12:13 PM, Colin McCabe wrote: >> >>> +1 (non-binding) >>> >>> Colin >>> >>> On Tue, Nov 14, 2017, at 10:02, Guozhang Wang wrote: H

Re: [VOTE] KIP-224: Add configuration parameters `retries` to Streams API

;>>> >>>> -Bill >>>> >>>> On Mon, Nov 13, 2017 at 7:25 PM, Ted Yu wrote: >>>> >>>>> +1 >>>>> >>>>> On Mon, Nov 13, 2017 at 4:20 PM, Matthias J. Sax < >>> matth...@confluent.io> >

Re: [DISCUSS] KIP-210: Provide for custom error handling when Kafka Streams fails to produce

>>> >>> I strongly disagree here. The purpose of this handler isn't *just* to >>> make a decision for streams. There may also be desirable side effects that >>> users wish to cause when production exceptions occur. There may be >>> side-effects that th

Re: [DISCUSS]: KIP-159: Introducing Rich functions to Streams

ed to lay the ground for discussions so we are all on the same > page before chatting more. > > > Guozhang On 11/6/17 1:41 PM, Jeyhun Karimov wrote: > Hi Matthias, > > Thanks a lot for correcting. It is a leftover from the past designs when > punctuate() was not d

Re: [DISCUSS] KIP-213 Support non-key joining in KTable

how would you > formulate this? > > Thanks, > > Jan > > > On 16.11.2017 23:18, Matthias J. Sax wrote: >> Hi, >> >> I am just catching up on this discussion and did re-read the KIP and >> discussion thread. >> >> In contrast to you

Re: [DISCUSS] KIP-213 Support non-key joining in KTable

en the different used types, K0,K1,KO >> should be explains explicitly (all information is there implicitly, but >> one need to think hard to figure it out) >> >> >> I'm probably blind for this. can you help me here? how would you formulate >> this? >

Re: Delete Topic to start Fresh

Topic deletion happens async and there is no guarantee when it will happen. You cannot recreate the topic, until is was deleted. Thus a try-catch-retry strategy would work. Or you check if topic still exists before trying to recreate it. -Matthias On 11/22/17 3:00 AM, Rumani Kumari wrote: > Hi,

Re: [DISCUSS]: KIP-159: Introducing Rich functions to Streams

r even though the >>>> record context() is for change.newValue. I would go as far as applying >>>> the filter before the table processor. Not to just get KIP-159, but >>> because >>>> I think its a side effect of a non ideal topology layout. If

Re: [DISCUSS] KIP-233: Simplify StreamsBuilder#addGlobalStore

Panuwat, Thanks a lot for the KIP! Just one nit: `does not follow provide a good` -> spelling: remove `follow` ? Otherwise, looks good to me. -Matthias On 12/4/17 10:49 AM, Ted Yu wrote: > Looks like you're implying logic similar to this: > > public synchronized GlobalKTable globalTab

Re: [DISCUSS] KIP-221: Repartition Topic Hints in Streams

l decrease the number of overloads but we will need to deprecate > the existing to() and through() methods, perhaps. > I updated the KIP accordingly. > > > Cheers, > Jeyhun > > On Thu, Nov 16, 2017 at 10:21 PM Matthias J. Sax > wrote: > >> @Jan: >> &g

Re: custom ProducerRecord timestamp and compact+delete cleanup

Both use the same timestamp that is record meta data timestamp. KIP-32 behavior was changed later on. -Matthias On 11/27/17 8:00 AM, Michal Hariš wrote: > Hi, > > I've read on the message timestamp(KIP-32) that for time-rolling deletes > use LogAppendTime, but this only applies to delete polic

Re: [DISCUSS] KIP-210: Provide for custom error handling when Kafka Streams fails to produce

ly for the first >>> exception? >> >> I didn't want to invoke the handler in places where the CONTINUE or FAIL >> result would just be ignored. Presumably, after a FAIL has been returned, >> subsequent exceptions are likely to be repeats or noise from my >&

Re: [DISCUSS]: KIP-230: Name Windowing Joins

Matthias, thanks for the KIP. Can you please c&p the corresponding content instead of just putting links? A KIP should be a self-contained Wiki page. Also, if we add a optional config parameter, how would we specify it? Please list all changes to want to apply to `Joined` class. Furthermore, `J

Re: [DISCUSS] KIP-213 Support non-key joining in KTable

. The downside of getting these > nested deeply is probably mitgated by users doing a group by > in the very next step to get rid of A's key again. > > That is what I was able to come up with so far. > Let me know. what you think > > > > > On 22.11.2017 00:14, M

Re: [DISCUSS]: KIP-159: Introducing Rich functions to Streams

eams/kstream/internals/KTableFilter.java#L96 >>> >>> >>> I cannot help you following this. It is really obvious and I am running >>> out of tools for explaining. >>> >>> Thanks for understanding my point to put filter before. Not only &g

Re: [DISCUSS]KIP-216: IQ should throw different exceptions for different errors

Vito, is there any update with regard to this KIP? -Matthias On 11/5/17 6:11 PM, vito jeng wrote: > Thanks, Guozhang and Matthias. Your comments very useful for me. > > I'll update KIP and keep going on. > > > > --- > Vito > > On Sun, Nov 5, 2017 at

Re: [DISCUSS] KIP-165: Extend Interactive Queries for return latest update timestamp per key

Jeyhun, is there any update regarding this KIP? -Matthias On 10/13/17 10:39 AM, Matthias J. Sax wrote: > I wanted to follow up with this KIP. > > One important thing that is missing in this KIP, is the question where > the TS is coming from? ATM, timestamps are not stored

Re: [DISCUSS] KIP-222 - Add "describe consumer group" to KafkaAdminClient

what others thought about this. >>> >>> Cheers, >>> >>> Tom >>> >>> On 6 November 2017 at 22:16, Jorge Esteban Quilcate Otoya < >>> quilcate.jo...@gmail.com> wrote: >>> >>>> Thanks for the feedback! >>>> >>>

Re: [DISCUSS] KIP-210: Provide for custom error handling when Kafka Streams fails to produce

was clear in the KIP, but I can go back and double check my > wording and revise it to try and make it clearer. > > I’ll take a look at doing more work on the KIP and the Pull Request > tomorrow. > > Thanks again! > > On December 4, 2017 at 5:50:33 PM, Matthias J. Sax (matth.

Re: [DISCUSS] KIP-228 Negative record timestamp support

Thanks for the KIP Konstantin. From my understanding, you propose to just remove the negative timestamp check in KafkaProducer and KafkaStreams. If topics are configured with `CreateTime` brokers also write negative timestamps if they are embedded in the message. However, I am not sure about the

Re: [DISCUSS]KIP-216: IQ should throw different exceptions for different errors

--- > Vito > > On Tue, Dec 5, 2017 at 8:05 AM, Matthias J. Sax > wrote: > >> Vito, >> >> is there any update with regard to this KIP? >> >> >> -Matthias >> >> On 11/5/17 6:11 PM, vito jeng wrote: >>>

Re: Interested in becoming committer - Where to fork ?

Thanks for your interest in Kafka! Looking forward to your first PR! :) Development happens via GitHub. Thus, you should clone https://github.com/apache/kafka on GitHub and also open PR on GitHub targeting https://github.com/apache/kafka The Apache repo is the source of truth and it gets mirrored

Re: [VOTE] KIP-210: Provide for custom error handling when Kafka Streams fails to produce

nding +1s. >> >> On November 8, 2017 at 12:26:32 PM, Damian Guy (damian@gmail.com) >> wrote: >> >> +1 (binding) >> >> On Sat, 4 Nov 2017 at 16:50 Matthias J. Sax wrote: >> >>> Yes. A KIP needs 3 binding "+1" to be accepted. &g

Re: KStream: Error Reading and Writing Avro records

Cross posted at SO: https://stackoverflow.com/questions/47712933/kstream-error-reading-and-writing-avro-records I put an answer there. -Matthias On 12/7/17 10:29 PM, Somasundaram Sekar wrote: > I’m trying to write avro record that I read from a topic into another > topic, intentions it to augmen

Re: [DISCUSS] KIP-210: Provide for custom error handling when Kafka Streams fails to produce

he "logger" >> object be created? Note that the implementation of this interface have to >> give a non-param constructor, or as a static field of the class but in that >> case you would not be able to log which instance is throwing this error (we >> may have mult

Re: Reg. beginner issues for a new contributor

Just an add on: Also Connect API and Streams API are written in Java. Not just producer/consumer/admin clients. Thus, Java code base is quite big and not just a small part. -Matthias On 12/14/17 7:28 PM, Gwen Shapira wrote: > Just to clarify, since I wrote the quora answer: Apache Kafka core co

Re: [VOTE] KIP-239 Add queryableStoreName() to GlobalKTable

+1 On 12/15/17 11:56 AM, Ted Yu wrote: > Hi, > Here is the discussion thread: > > http://search-hadoop.com/m/Kafka/uyzND12QnH514pPO9?subj=Re+DISCUSS+KIP+239+Add+queryableStoreName+to+GlobalKTable > > Please vote on this KIP. > > Thanks > signature.asc Description: OpenPGP digital signature

Re: [DISCUSS] KIP-210: Provide for custom error handling when Kafka Streams fails to produce

stomized processing on errors. > > > I think improving the skipped records of all these scenarios itself worth > having another KIP; so I'd suggest we do not drag this KIP-210 into this. > > > Guozhang > > > On Wed, Dec 13, 2017 at 3:45 PM, Matthias J. Sa

[VOTE] KIP-243: Make ProducerConfig and ConsumerConfig constructors public

Hi, I want to propose the following KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-243%3A+Make+ProducerConfig+and+ConsumerConfig+constructors+public This is a rather straight forward change, thus I skip the DISCUSS thread and call for a vote immediately. -Matthias signature.asc

Re: [DISCUSS]: KIP-159: Introducing Rich functions to Streams

;>>> >>>> Initially, I was hesitant to remove information from the context of the >>>> result records (Joins or Aggregations) with the thought that when there >>>> are >>>> unexpected results, the source information would be useful for tracing &g

Re: [VOTE] KIP-220: Add AdminClient into Kafka Streams' ClientSupplier

wrote: >> >>> +1 >>> >>> On Mon, 20 Nov 2017 at 17:52 Gwen Shapira wrote: >>> >>>> +1 >>>> >>>> Make sense. We have a supplier for every other client type :) >>>> >>>> On Fri, Nov 17, 2017 at 1:33 PM

Re: [VOTE] KIP-243: Make ProducerConfig and ConsumerConfig constructors public

;> >>>>> From: Ted Yu >>>>> To: dev@kafka.apache.org >>>>> Date: 12/18/2017 02:45 PM >>>>> Subject:Re: [VOTE] KIP-243: Make ProducerConfig and >>>> ConsumerConfig >>>>> constructors public >

Re: [VOTE] KIP-243: Make ProducerConfig and ConsumerConfig constructors public

;> >>>>> From: Ted Yu >>>>> To: dev@kafka.apache.org >>>>> Date: 12/18/2017 02:45 PM >>>>> Subject:Re: [VOTE] KIP-243: Make ProducerConfig and >>>> ConsumerConfig >>>>> constructors public >

Re: [VOTE] KIP-243: Make ProducerConfig and ConsumerConfig constructors public

e the > public constructor without giving users anything useful to do with the > class. > > What do you think? > > -Jason > > On Wed, Dec 20, 2017 at 5:59 PM, Matthias J. Sax > wrote: > >> It's tailored for internal usage. I think client constructor

Re: Vote for KIP-245: Use Properties instead of StreamsConfig in KafkaStreams constructor

Thanks for the KIP Boyang! I don't have any further comments. +1 from me. @Ted: This is a rather simple KIP, thus, skipping the DISCUSS thread seems ok to me. -Matthias @Boyang: it's recommended to use this format for the subject "[VOTE] KIP-245: ..." Same for DISCUSS threads. People are

Re: KIP-244: Add Record Header support to Kafka Streams

Thanks for the KIP Jorge, As Bill pointed out already, we should be careful with adding new overloads as this contradicts the work done via KIP-182. This KIP also seems to be related to KIP-149 and KIP-159. Are you aware of them? Both have quite long DISCUSS threads, but it might be worth browsin

<    1   2   3   4   5   6   7   8   9   10   >