Re: Re: kafka steams with TimeWindows ,incorrect result

Hi: I don't kown what to do with transform function. and stream is preapred well like this at blew key: 44_14_2018-04-27 value: CountInfo(start=1,active=0,fresh =0) there is amount data like that。 how I aggregate it with peer 1 seconds using transform function？

Re: Re: kafka steams with TimeWindows ,incorrect result

and I checked windowStore interface found it has put method not get method. in one second the stream had sample key and different value in it. and I must update key value which store in it. funk...@live.com From: funk...@live.com Date:

Re: Re: kafka streams with TimeWindows ,incorrect result

the value is defined type which implements Serializer and Deserializer funk...@live.com From: Ted Yu Date: 2018-04-27 16:39 To: users; 杰 杨 Subject: Re: Re: kafka streams with

Re: Re: kafka streams with TimeWindows ,incorrect result

Noticed a typo in streams in subject. Corrected it in this reply. Original message From: 杰 杨 Date: 4/27/18 1:28 AM (GMT-08:00) To: 杰 杨 , users Subject: Re: Re: kafka steams with TimeWindows ,incorrect result and I

Re: Kafka mirror maker help

Sorry I hit send a bit too soon. I was so focused on the systemd part of the email and not the Mirror Maker part. Confluent packages include Mirror Maker but the systemd scripts are setup to use Confluent Replicator rather than Mirror Maker. My apologies. -hans /** * Hans Jespersen, Director

Re: Kafka mirror maker help

Hiya, Saravanan, I saw you emailed my colleague Alex about WMF’s old debian packaging. I’ll reply here. We now use Confluent’s Kafka debian packaging which does not (or did not?) ship with init scripts. We don’t use Sys V init.d scripts anymore either, but use systemd instead. Our systemd

Re: Kafka mirror maker help

I run instances of Mirror Maker as supervisord tasks (http://supervisord.org ). I’d recommend looking into it. In addition to letting you sidestep the service issue, supervisord watches the processes and can auto-restart them if they stop for any reason. — Peter

Kafka-Connect- org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment.

Hi Team, I am new to Kafka. Would like to explore option of using Kafka during data processing. Using Oracle virtual box for practicing Kafka and zookeeper. Below are the steps followed to deploy and test. 1) Start docker-compose 2) Get into Kafka container. Command (docker exec -it

Re: Kafka mirror maker help

The latest Confluent packages now ship with systemd scripts. That is since Confluent Version 4.1 - which included Apache Kafka 1.1 -hans /** * Hans Jespersen, Director Systems Engineering, Confluent Inc. * h...@confluent.io (650)924-2670 */ On Fri, Apr 27, 2018 at 11:15 AM, Andrew Otto

Kafka Streams 1.1.0 - Significant Performance Drop

Hi All, I was trying to switch to the latest version of streams (1.1.0) and started seeing a significant drop in performance of the application. I was using 0.11.0.2 before. After doing some checks I found that the choking point was Rocksdb flush which contributes almost 80% of the CPU time (PFA

Re: Kafka Streams 1.1.0 - Significant Performance Drop

Hi Tony, I'll try to address your questions below: 1. While it's not technically "wrong" to commit for each record, you need to keep in mind that calling commit has resource implications for Kafka Streams. Here's a link to our FAQ describing what happens during a commit

source code location for KSQL

I think KSQL is also open sourced, where is the source code location for KSQL in github? Thanks.

Re: source code location for KSQL

https://github.com/confluentinc/ksql FYI On Fri, Apr 27, 2018 at 4:23 PM, Henry Cai wrote: > I think KSQL is also open sourced, where is the source code location for > KSQL in github? > > Thanks. >

Re: source code location for KSQL

Hey Henry, Here you go https://github.com/confluentinc/ksql Guozhang On Fri, Apr 27, 2018 at 4:23 PM, Henry Cai wrote: > I think KSQL is also open sourced, where is the source code location for > KSQL in github? > > Thanks. > -- -- Guozhang

Re: source code location for KSQL

I guess Ted's 47 minutes faster than me :P On Fri, Apr 27, 2018 at 5:19 PM, Guozhang Wang wrote: > Hey Henry, > > Here you go https://github.com/confluentinc/ksql > > Guozhang > > On Fri, Apr 27, 2018 at 4:23 PM, Henry Cai > wrote: > >> I think

KStreams API Usage

Hi, I am trying to call kafka stream close based on the presence of a value in the output of ValueTransformer.ValueTransformer produces a List Is there a way to avoid the foreach on Kstream and try to get the first value alone? (like streams api method findFirst) private void