Tom Barber created OODT-946:
-------------------------------
Summary: Implement Kafka Messaging System Within OODT
Key: OODT-946
URL: https://issues.apache.org/jira/browse/OODT-946
Project: OODT
Issue Type: Bug
Reporter: Tom Barber
Assignee: Tom Barber
Under the hood of Apache OODT we make use of XMLRPC and/or Apache Avro to help
with messaging and communications.
Our proposal for Apache OODT 2.0 is to replace some of this underlying
communication trafficking with Apache Kafka. In doing so this will allow
multiple components to subscribe to streams and register their interest. It
also allows us to track messages flowing throughout the system allowing for
much improved auditability.
The initial work would be done in the file manager, replacing some of the
internal communications with Kafka based messaging and also then providing an
external interface that other OODT components can subscribe to allowing for
event registration when files are ingested.
Secondly we would look to replace the messaging within the Workflow Manager
with Kafka there by allowing easier tracabilty of the data flowing through it
and again allowing other components to register themselves against it. This
will also make integration of 3rd party applications easier.
Part 3 would introduce Kafka based messaging into the resource manager,
allowing components to be aware of the resources available to them in real time.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
