[
https://issues.apache.org/jira/browse/HBASE-28987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17916919#comment-17916919
]
Duo Zhang commented on HBASE-28987:
-----------------------------------
You could still do the filtering when uploading the entire file, or we could
move the filtering to the time when we want to replay them?
The key thing here is that we do not need to tailing the files which are still
being written to.
> Developing a Custom ReplicationEndpoint to Support External Storage
> Integration
> -------------------------------------------------------------------------------
>
> Key: HBASE-28987
> URL: https://issues.apache.org/jira/browse/HBASE-28987
> Project: HBase
> Issue Type: Task
> Components: backup&restore
> Affects Versions: 2.6.0, 3.0.0-alpha-4
> Reporter: Vinayak Hegde
> Assignee: Vinayak Hegde
> Priority: Major
>
> *Develop a Custom Replication Endpoint*
> Implement a custom replication endpoint to support the backup of WALs to
> external storage systems, such as HDFS-compliant storages (including HDFS,
> S3, ADLS, and GCS via respective Hadoop connectors).
> *Support for Bulk-loaded Files*
> Add functionality to back up bulk-loaded files in addition to regular WALs.
> *Ensure Process Durability*
> Ensure the backup process is durable, with no WALs being missed, even in the
> event of issues in the cluster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
