Hi Matthias, I upgraded to 2.0.0 and we're experiencing the same problem. I've posted a new screengrab of a thread profile:
https://imgur.com/a/2wncPHw >From our perspective, it appears something happened after 0.10.2.1 that made >the LRU Cache much slower for our use case. What would you recommend for our >next steps? Jonathan On 2018/11/06 19:22:16, "Matthias J. Sax" <matth...@confluent.io> wrote: > Not sure atm why you see a performance degradation. Would need to dig > into the details. > > However, did you consider to upgrade to 2.0 instead or 0.11? > > Also note that we added a new operator `suppress()` in upcoming 2.1 > release, that allows you to do rate control without caching: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-328%3A+Ability+to+suppress+updates+for+KTables > > Hope this helps. > > > -Matthias > > On 11/6/18 9:49 AM, Jonathan Gordon wrote: > > I have a Kafka Streams app that I'm trying to upgrade from 0.10.2.1 to > > 0.11.0.3 but when I do I notice that CPU goes way up and consumption goes > > down. A thread profile indicates that the most expensive task is during our > > aggregation, fetching from the cache. > > > > Thread profile with caching: > > https://imgur.com/l5VEsC2 > > > > If I disable the cache both performance and consumption are good but we are > > producing every single aggregation modification, which is not what we want. > > > > Thread profile without caching: > > https://imgur.com/a/JK3nkou > > > > I read this thread, which seems relevant e > > > > https://lists.apache.org/thread.html/2b44e74eaec7172b107bcff96861cf8b4837f55a44714f69d033cc2e@%3Cusers.kafka.apache.org%3E > > > > Notably: "Note, that caching was _not_ introduced to reduce the writes to > > RocksDB, but to reduce the write the the changelog topic and to reduce the > > number of records send downstream." > > > > So how can we reduce the number of records sent downstream while > > maintaining the same performance characteristics that we have with caching > > turned off? Or put another way, how can I upgrade my app without taking a > > hit in performance or behavior? > > > > Thanks! > > > >
