RE: Kafka Connect or Streams Output to local File System

2016-07-15 Thread David Newberger
afka Connect or Streams Output to local File System Hi David, As Eno said, you can use Kafka Streams to pipe the raw logs as well as those anomaly events to two different topics, and use separate Kafka Connect to read them into two files. An alternative way to use Kafka Streams only, is to use t

Re: Kafka Connect or Streams Output to local File System

2016-07-15 Thread Guozhang Wang
Hi David, As Eno said, you can use Kafka Streams to pipe the raw logs as well as those anomaly events to two different topics, and use separate Kafka Connect to read them into two files. An alternative way to use Kafka Streams only, is to use the `printAsText` operator in the Streams DSL, or you

Re: Kafka Connect or Streams Output to local File System

2016-07-15 Thread Eno Thereska
Hi David, One option would be to first output your info to a topic using Kafka Streams, and then use Connect again (as a sink) to read from the topic and write to a file in the file system. Eno > On 15 Jul 2016, at 08:24, David Newberger > wrote: > > Hello All, > > I'm curious if I can out