rdhabalia opened a new issue #1582: Introduce pulsar replicator 
URL: https://github.com/apache/incubator-pulsar/issues/1582
   ## Motivation
   Pulsar already supports geo-replication that persists messages across 
multiple clusters of pulsar instances. Therefore, client can set replication 
clusters for a topic, and pulsar broker internally takes care of replication to 
all the clusters. However, sometimes application may want to replicate the same 
published messages to other external systems which is not part of pulsar-eco 
system such as AWS-Kinesis, DynamoDB. Therefore, right now, client-application 
has to take this extra burden to publish same messages for pulsar and other 
external systems.
   Therefore, it will be useful to introduce server side replication that can 
replicate pulsar messages to external system without client intervention. Also 
server side replication should be extensible which can provide a plugin 
mechanism to add various replicators to support message-replication to 
different external systems.
   ## Requirement
   - Replicate Pulsar message to external system (eg: Kinesis, Kafka, another 
Pulsar cluster)
   - Easy onboarding: client should be able to add replicator configuration 
with CLI/Admin api and it should auto start appropriate replicators. 
   - Isolation from core message bus
   - Pluggable framework and extensible to support multiple external system
   - New connectors should be developed with minimal efforts
   - Operability and monitoring
      - API to control start and stop individual replicator
      - API to get replicator stats
   - Security
      - Replicator framework should provide pluggable mechanism to plugin 
KeyStore implementation that can store and fetch client’s credentials which 
will be required to connect external system.

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:

With regards,
Apache Git Services

Reply via email to