[ 
https://issues.apache.org/jira/browse/KUDU-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18146186#comment-18146186
 ] 

Marton Greber commented on KUDU-2180:
-------------------------------------

Design doc(sent out to the dev list): 
https://docs.google.com/document/d/1Qqm0l3oRUl4BQXyeR_6rTaYOH_bmQKif/edit?usp=sharing&ouid=101270421367316166431&rtpof=true&sd=true
 

> Change data capture (CDC) WAL-tailing support
> ---------------------------------------------
>
>                 Key: KUDU-2180
>                 URL: https://issues.apache.org/jira/browse/KUDU-2180
>             Project: Kudu
>          Issue Type: New Feature
>          Components: api, consensus, master, tablet, tserver
>            Reporter: Mike Percy
>            Assignee: Marton Greber
>            Priority: Major
>              Labels: roadmap-candidate
>
> This is a tracking JIRA for change data capture (CDC) support in Kudu.
> This feature is not trivial and would need a significant amount of design 
> work.
> Here are some high-level considerations and potential requirements / 
> approaches:
> * New CDC Subscription APIs (remote and client)
> * Support for distributed consumers (necessary if subscribing to all cluster 
> changes at scale)
> * Consensus support for permanent non-voters (in order to have a replication 
> target)
> * Ability to retain WAL segments if a CDC subscriber has not yet caught up
> * Ability to re-sync subscribers that have fallen behind the leader's WAL. 
> Required to support a new subscriber joining after the cluster has been 
> running for a long time.
> In order to support re-syncing subscribers that have fallen behind the 
> leader's WAL, we may be able to build a bridge API for a consumer to ingest 
> the results of a tablet copy operation, without having to be a native Kudu 
> tablet server itself.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to