Don't push NiFi :) Please don't :) 2017-12-19 16:16 GMT+01:00 Artem Ervits <[email protected]>:
> you're probably better off using Apache Nifi at that point, Kafka Nifi is > better integrated today than Oozie and Kafka. Probably wrong mailing list > to push Nifi but it's a common use case in that realm. > > Thanks > > On Mon, Dec 18, 2017 at 7:04 PM, Serega Sheypak <[email protected]> > wrote: > > > Hi, I know default coordinator functionality, but it's limited (almost) > to > > HDFS. > > Kafka (any other pub/sub or queue like rabbitMQ, whatever-MQ) makes > > integration contract much more flexible. > > I could have traceability, debuggability, transparency, throttling, > > concurrency of oozie and push coordinator job on demand. And I'm not > > limited to strict HDFS path pattern. > > > > > > 2017-12-18 18:14 GMT+01:00 Andras Piros <[email protected]>: > > > > > Hi Serega, > > > > > > not to my knowledge. Would be interested on your use case, though. > > > > > > Would start w/ *Coordinator Input Events / Datasets > > > <https://oozie.apache.org/docs/4.3.0/CoordinatorFunctionalSpec. > > > html#a5._Dataset>* > > > . > > > > > > Andras > > > > > > On Sat, Dec 16, 2017 at 2:54 PM, Serega Sheypak < > > [email protected]> > > > wrote: > > > > > > > Hi, did anyone try to integrate oozie coordinator with kafka? > > > > use case: > > > > > > > > System publishes message to kafka topic (sample message) > > > > - cluster: hdfs://prod-cluster > > > > - path: /my/input/data > > > > - format: avro > > > > > > > > Oozie coordinator listens to kafka topic, consumes message and starts > > > > workflow. > > > > > > > > > >
