Avi, thanks for your feedback. We want to enlarge the scope for Streams application and started to collect use cases in the Wiki:
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Streams+Data+%28Re%29Processing+Scenarios Feel free to add there via editing the page or writing a comment. -Matthias On 12/13/16 3:30 PM, Avi Flax wrote: > On 2016-11-28 13:47 (-0500), "Matthias J. Sax" <m...@confluent.io> wrote: >> >> I want to start a discussion about KIP-95: >> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-95%3A+Incremental+Batch+Processing+for+Kafka+Streams >> >> Looking forward to your feedback. > > Hi Matthias, > > I’d just like to share some feedback on this proposal, from the perspective > of a Kafka user rather than developer: > > * Overall, makes a ton of sense and I think it’ll be very useful and will > open up a bunch of additional use cases to Kafka Streams > > * Two use cases that are of particular interest to me: > > * Just last week I created a simple Kafka Streams app (I called it a > “script”) to copy certain records from one topic over to another, > with filtering. I ran the app/script until it reached the end of > the topic, then I manually shut it down with ctrl-c. This worked > fine, but it would have been an even better UX to have specified > the config value `autostop.at` as `eol` and have the process stop > itself at the desired point. That would have required less manual > monitoring on my part. > > * With this new mode I might be able to run Streams apps on AWS Lambda. > I’ve been super-excited about Lambda and similar FaaS services since > their early days, and I’ve been itching to run Kafka Streams apps on > Lambda for since I started using Streams in April or May. Unfortunately, > Lambda functions are limited to 5 minutes per invocation — after 5 > minutes they’re killed. I’m not sure, but I wonder if perhaps this new > autostop.at feature could make it more practical to run a Streams > app on Lambda - the feature seems like it could potentially be adapted > to enable a Streams app to be more generally resilient to being > frequently stopped and started. > > I look forward to seeing progress on this enhancement! > > Thanks, > Avi > > ———— > Software Architect @ Park Assist > We’re hiring! http://tech.parkassist.com/jobs/ >
signature.asc
Description: OpenPGP digital signature
