[
https://issues.apache.org/jira/browse/HBASE-29519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18013771#comment-18013771
]
Tak-Lon (Stephen) Wu edited comment on HBASE-29519 at 8/13/25 7:49 PM:
-----------------------------------------------------------------------
I reviewed the current PR, and I'm wondered this blocking upload may even slow
down the WAL edits replication, so either having async upload or a after-all
WAL closed uploaded could solve the blocking issue.
can you help to explain how we will resume the upload for bulkloaded files if
it fail?
When you said in-memory structure after the WAL file is closing (yet closed and
during the COMMIT stage), if the bulkloaded files fail to upload, are we going
to abandon this WAL file and resync/resume from where the WAL replication
starts? the concern is about the replication queue, could this cause another
WAL backlog cascade? with in-memory or store it in some persistent table, what
if the list of bulkloaded files are huge?
was (Author: taklwu):
I reviewed the current PR, and I'm wondered this blocking upload may even slow
down the WAL edits replication, so either having async upload or a after-all
WAL closed uploaded could solve the blocking issue.
can you help to explain how we will resume the upload for bulkloaded files if
it fail? When you said in-memory structure after the WAL file is closing (yet
closed and during the COMMIT stage), if the bulkloaded files fail to upload,
are we going to abandon this WAL file and resync/resume from where the WAL
replication starts? the concern is about the replication queue, could this
cause another WAL backlog cascade?
> Copy Bulkloaded Files in Continuous Backup
> ------------------------------------------
>
> Key: HBASE-29519
> URL: https://issues.apache.org/jira/browse/HBASE-29519
> Project: HBase
> Issue Type: Sub-task
> Components: backup&restore
> Reporter: Vinayak Hegde
> Assignee: Vinayak Hegde
> Priority: Major
> Labels: pull-request-available
>
> Enhance the continuous backup replication endpoint to detect bulkload
> operations and copy their HFiles to the backup location (e.g., S3).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
