@Sachin >>The partition key is very important if you need to run multiple instances of streams application and certain instance processing certain partitions only.
Again, depending on partition key is optional. It's actually a feature enabler, so we can use local state stores to improve throughput. I don't see this as a downside. Regards Sab On 11 Oct 2017 1:44 pm, "Sachin Mittal" <sjmit...@gmail.com> wrote: > Kafka streams has a lower learning curve and if your source data is in > kafka topics it is pretty simple to integrate it with. > It can run like a library inside your main programs. > > So as compared to spark streams > 1. Is much simpler to implement. > 2. Is not much heavy on hardware unlike spark. > > > On the downside > 1. It is not elastic. You need to anticipate before hand on volume of data > you will have. Very difficult to add and reduce topic partitions later on. > 2. The partition key is very important if you need to run multiple > instances of streams application and certain instance processing certain > partitions only. > In case you need aggregation on a different key you may need to > re-partition the data to a new topic and run new streams app against that. > > So yes if you have good idea about your data and if it comes from kafka > and you want to build something quick without much hardware kafka streams > is a way to go. > > We had first tried spark streaming but given hardware limitation and > complexity of fetching data from mongodb we decided kafka streams as way to > go forward. > > Thanks > Sachin > > > > > > On Wed, Oct 11, 2017 at 1:01 PM, Mich Talebzadeh < > mich.talebza...@gmail.com> wrote: > >> Hi, >> >> Has anyone had an experience of using Kafka streams versus Spark? >> >> I am not familiar with Kafka streams concept except that it is a set of >> libraries. >> >> Any feedback will be appreciated. >> >> Regards, >> >> Mich >> >> >> >> LinkedIn * >> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw >> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* >> >> >> >> http://talebzadehmich.wordpress.com >> >> >> *Disclaimer:* Use it at your own risk. Any and all responsibility for >> any loss, damage or destruction of data or any other property which may >> arise from relying on this email's technical content is explicitly >> disclaimed. The author will in no case be liable for any monetary damages >> arising from such loss, damage or destruction. >> >> >> > >
