Re: Memory consumption of Kafka-examples Kafka-streams around 1.5 GB

2016-06-16 Thread Michael Noll
Phil, I think what you're seeing w.r.t. memory consumption is that the packaging step (`mvn package`) for the Kafka Streams examples under https://github.com/confluentinc/examples also runs the full test suite, and this test suite launches embedded Kafka brokers, ZooKeeper, and Confluent schema re

Re: Memory consumption of Kafka-examples Kafka-streams around 1.5 GB

2016-06-15 Thread Philippe Derome
Guozhang, No two in particular, at first it was simply the last two that the target would choose: SumLambdaIntegrationTest and WordCountLambdaIntegrationTest . I tried another couple to exclude and it was fine as well. There's one Scala test that is included in the run and I run it as well as 7/9

Re: Memory consumption of Kafka-examples Kafka-streams around 1.5 GB

2016-06-15 Thread Guozhang Wang
Hello Phillippe, I used to run the "SimpleBenchmark" on my laptop with 4GB also, and it usually used close to, but less than 1GB. https://www.codatlas.com/github.com/apache/kafka/HEAD/streams/src/test/java/org/apache/kafka/streams/perf/SimpleBenchmark.java Note that I need to bootstrap a real ZK

Memory consumption of Kafka-examples Kafka-streams around 1.5 GB

2016-06-14 Thread Philippe Derome
I am running "mvn test" as per tip from https://github.com/confluentinc/examples/tree/master/kafka-streams README.MD. This uses embedded Kafka components from test alone (no ZK, Kafka, schema-registry running). I monitor on OSX El Capitan (10.11.5) memory usage and it grows on Java processes from