Re: Please add me to the contributors JIRA list

Please remove me from the contributors' list. I suspect that this is what makes me receive all these emails and I have not participated to this project in about 5 years. My contributions have been reviewed by Guozhang Wang. Thanks, Phil Derome On Sat, Jan 15, 2022 at 6:16 AM Sayantanu Dey

Re: KStream: KTable-KTable leftJoin with key only on RHS of join generates null in joined table

As for Kafka Streams implementation, the if a > null record is received in the downstream operators while there is no > existing record for that key in the materialized view, then it is treated > as a no-op. > > > Guozhang > > > On Wed, Jul 6, 2016 at 3:59 AM, Philippe D

Re: NPE using GenericAvroSerde to deserialize

This item is moved to confluent google group. On Wed, Jul 6, 2016 at 6:25 AM, Philippe Derome <phder...@gmail.com> wrote: > Please ignore until I become quite more specific about my code usage (will > try to recover the same code I used). In the mean time, I managed to > serial

Re: KStream: KTable-KTable leftJoin with key only on RHS of join generates null in joined table

output. We made this > design deliberately just to make sure that "table-table joins are > eventually consistent". This gives a kind of resilience to late arrival of > records that a late arrival in either stream can "update" the join result. > > > Guozhang > &g

Re: NPE using GenericAvroSerde to deserialize

wrote: > Phil, > > > I then specify a Serde (new GenericAvroSerde) as value > > deserializer when outputting to topic via table.to method. > > I suppose that was a typo, and you actually meant "as a value > *serializer*", right? > > > > On Tue, Jul 5

NPE using GenericAvroSerde to deserialize

This is possibly more of a Confluent question as GenericAvroSerde is in Confluent example code base. I have a small Stream application, which creates a KTable with String key and GenericRecord value. I then specify a Serde (new GenericAvroSerde) as value deserializer when outputting to topic via

Re: KStream: KTable-KTable leftJoin with key only on RHS of join generates null in joined table

, Philippe Derome <phder...@gmail.com> wrote: > If we have two streams A and B for which we associate tables tA and tB, > then create a table tC as ta.leftJoin(tB, ) and then we > have a key kB in stream B but never made it to tA nor tC, do we need to > inject a pair (k,v) of (kB, nu

KStream: KTable-KTable leftJoin with key only on RHS of join generates null in joined table

If we have two streams A and B for which we associate tables tA and tB, then create a table tC as ta.leftJoin(tB, ) and then we have a key kB in stream B but never made it to tA nor tC, do we need to inject a pair (k,v) of (kB, null) into resulting change log for tC ? It sounds like it is

Re: KTable.filter usage, memory consumption and materialized view semantics

not enter T2 at all (no null). Ultimately, the code change is simpler and the test results look more intuitive. On Wed, Jun 29, 2016 at 6:55 PM, Philippe Derome <phder...@gmail.com> wrote: > good. > > On Wed, Jun 29, 2016 at 6:44 PM, Guozhang Wang <wangg...@gmail.co

Re: KTable.filter usage, memory consumption and materialized view semantics

nge, I'd prefer to > complete this ticket first, and refactor it when we decided to work on 3911 > later. > > Feel free to link these two tickets though. > > Guozhang > > On Tue, Jun 28, 2016 at 9:47 AM, Philippe Derome <phder...@gmail.com> > wrote: > > &

Re: KTable.filter usage, memory consumption and materialized view semantics

, and sending the {old -> new} pair instead of the new value only) may be over-whelming compared with its potential benefits of reducing the downstream traffic. Guozhang On Sun, Jun 26, 2016 at 8:58 AM, Philippe Derome <phder...@gmail.com> wrote: > Guozhang, > > would you say it's advis

Re: KTable.filter usage, memory consumption and materialized view semantics

o be > materialized (i.e. creating a state store, and sending the {old -> new} > pair instead of the new value only) may be over-whelming compared with its > potential benefits of reducing the downstream traffic. > > Guozhang > > On Sun, Jun 26, 2016 at 8:58 AM, P

Re: [jira] [Commented] (KAFKA-3896) Unstable test KStreamRepartitionJoinTest.shouldCorrectlyRepartitionOnJoinOperations

Never mind sorry. I should research this on my own. On 27 Jun 2016 3:29 p.m., "Philippe Derome" <phder...@gmail.com> wrote: > Guozhang, all on this thread, > > I've been spending a fair amount looking at this problem very indirectly > with my PR for KAFKA-390

Re: [jira] [Commented] (KAFKA-3896) Unstable test KStreamRepartitionJoinTest.shouldCorrectlyRepartitionOnJoinOperations

Guozhang, all on this thread, I've been spending a fair amount looking at this problem very indirectly with my PR for KAFKA-3902. I had 3 commits on the PR and the last one fails on Jenkins for this unit test here: https://github.com/apache/kafka/pull/1556 Additionally, that particular file

Re: KTable.filter usage, memory consumption and materialized view semantics

ributing+Code+Changes#ContributingCodeChanges-PullRequest > > > Guozhang > > > On Sat, Jun 25, 2016 at 8:40 PM, Philippe Derome <phder...@gmail.com> > wrote: > > > I have a 1 liner solution for this in KTableFilter.java and about 5-6 > lines > > changes to existing un

Re: KTable.filter usage, memory consumption and materialized view semantics

, , , , ... > > And the KTable is materialized into a state store with cache on top of it, > then the resulted downstream could be: > > 1}>, 6}> ... > > Instead of > > 1}>, 2}>, 3}>, ... 6}> ... > > So if it is piped to a filter() operator

Re: KTable.filter usage, memory consumption and materialized view semantics

air; > > 2) predicate return false on both, we can optimize and do not send > anything; > > 3) predicate return true on old and false on new, send the key: {old -> > null}; > > 4) predicate return false on old and true on new, send the key: {null -> > new}; >

Re: KTable.filter usage, memory consumption and materialized view semantics

r as well so that > de-duping can be done in this operator and hence the outgoing traffic can > be largely reduced: > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-63:+Unify+store+and+downstream+caching+in+streams > > > Guozhang > > > On Thu, Ju

KTable.filter usage, memory consumption and materialized view semantics

I made a modification of latest Confluent's example UserRegionLambdaExample. See relevant code at end of email. Am I correct in understanding that KTable semantics should be similar to a store-backed cache of a view as (per wikipedia on materialized views) or similar to Oracle's materialized

Re: avro-console-producer issue for Kafka Stream example PageViewRegionLambdaExample

Sorry, this thread belongs to confluent not Kafka.apache.org as it pertains to Confluent's examples. On Thu, Jun 16, 2016 at 2:33 PM, Philippe Derome <phder...@gmail.com> wrote: > I am able to produce Avro from producer for pageview.avsc but not for > userprofile.avsc. Here are two s

Re: [DISCUSS] Java 8 as a minimum requirement

I strongly support motion having difficulty running (Apache Kafka as opposed to Confluent) Stream examples with JDK 8 today. On 16 Jun 2016 4:46 p.m., "Ismael Juma" wrote: > Hi all, > > I would like to start a discussion on making Java 8 a minimum requirement > for Kafka's

avro-console-producer issue for Kafka Stream example PageViewRegionLambdaExample

I am able to produce Avro from producer for pageview.avsc but not for userprofile.avsc. Here are two samples below, first one leaves out Avro optional experience, second one specifies it. Note: I do not register using REST curl calls schemas to schema-registry as I assume that

Re: Memory consumption of Kafka-examples Kafka-streams around 1.5 GB

rprised if that is taking large amount. > > Which two test cases specifically are causing OOMs on your laptop? > > Guozhang > > > On Tue, Jun 14, 2016 at 4:50 PM, Philippe Derome <phder...@gmail.com> > wrote: > > > I am running "mvn test" as per t

Memory consumption of Kafka-examples Kafka-streams around 1.5 GB

I am running "mvn test" as per tip from https://github.com/confluentinc/examples/tree/master/kafka-streams README.MD. This uses embedded Kafka components from test alone (no ZK, Kafka, schema-registry running). I monitor on OSX El Capitan (10.11.5) memory usage and it grows on Java processes from