[
https://issues.apache.org/jira/browse/HBASE-28987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17934329#comment-17934329
]
Andor Molnar commented on HBASE-28987:
--------------------------------------
{quote}For me I always believe that, we should try our best to not introduce
more concepts...
{quote}
Yeah, totally agree, but still the changes introduced in the linked patch are
not the end of the world. We might have split into multiple to aid reviewing,
because it only added thousand lines of code for the Continuous Backup itself.
The actual changes in the framework are limited to *ReplicationShource* and
*ReplicationSourceShipper* classes. Rest is - as mentioned - cont. backup logic
and replacing "true" values by "committed" in existing implementors.
Idea is that ReplicationSource gets the replication result and makes decision
whether to persist it. We still need to call logPositionAndCleanOldLogs() to
maintain {*}lastEntryBatch{*}, but will only call processAndClearEntries() if
replicated entries are committed.
Let me think about your other idea.
> Developing a Custom ReplicationEndpoint to Support External Storage
> Integration
> -------------------------------------------------------------------------------
>
> Key: HBASE-28987
> URL: https://issues.apache.org/jira/browse/HBASE-28987
> Project: HBase
> Issue Type: Task
> Components: backup&restore
> Affects Versions: 2.6.0, 3.0.0-alpha-4
> Reporter: Vinayak Hegde
> Assignee: Vinayak Hegde
> Priority: Major
>
> *Develop a Custom Replication Endpoint*
> Implement a custom replication endpoint to support the backup of WALs to
> external storage systems, such as HDFS-compliant storages (including HDFS,
> S3, ADLS, and GCS via respective Hadoop connectors).
> *Support for Bulk-loaded Files*
> Add functionality to back up bulk-loaded files in addition to regular WALs.
> *Ensure Process Durability*
> Ensure the backup process is durable, with no WALs being missed, even in the
> event of issues in the cluster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
