anmolnar commented on code in PR #8651: URL: https://github.com/apache/hbase/pull/8651#discussion_r4075659310
########## hbase-website/app/pages/_docs/docs/_mdx/(multi-page)/backup-restore/additional-topics.mdx: ########## @@ -201,14 +243,73 @@ HBase Bulk Load utility. You can only restore on a live HBase cluster because the data must be redistributed to complete the restore operation successfully. +## Technical Details of Continuous Backup and PITR + +Continuous backup uses a dedicated replication peer (`continuous_backup_replication_peer`) with a +custom replication endpoint (`ContinuousBackupReplicationEndpoint`) to stream WAL entries from the +source cluster to the WAL directory. WAL files are organized by date under the WAL directory: + +```text +<wal-directory>/WALs/<YYYY-MM-DD>/<wal-file> +<wal-directory>/bulk-load-files/<bulk-loaded-hfiles> +``` + +WALs are partitioned by date so PITR can efficiently select WAL files covering the requested recovery +interval. + +When a full backup is created with `--continuous-backup-enabled`, HBase performs the following steps: + +1. Sets the replication scope of every column family on the backed-up tables to GLOBAL. +2. Creates or updates the `continuous_backup_replication_peer` to include the specified tables. +3. Records the tables and their continuous backup start timestamps in the `hbase:backup` system table. +4. Takes and exports a full snapshot to the backup directory. + +From that point forward, WAL entries are continuously replicated to the WAL directory. The replication +endpoint groups WAL entries by day and periodically flushes data to avoid large partially-written +files and replication lag. + +### Incremental backups with continuous backup + +When incremental backups are run for tables that have continuous backup enabled, the backup process +reads WALs from the WAL directory instead of collecting them from the source cluster. This eliminates +the need to retain WALs on the source cluster between incremental backup runs and reduces pressure on +the source cluster's log management. The incremental backup command itself is unchanged; the system +detects whether tables are under continuous backup and selects the appropriate WAL source +automatically. + +### Point-in-Time Recovery process + +PITR restores a table to a specific timestamp using the following steps: + +1. **Select a backup**: Locate the most recent valid full or incremental backup completed before the + target time. The backup must have started after continuous backup was enabled for the table. +2. **Restore the backup**: Restore the selected backup image to the target table using the standard + restore process. +3. **Replay WALs**: Run a distributed WAL replay (MapReduce-based WALPlayer) over WAL files from the + WAL directory for the time range between the backup start time and the target timestamp. +4. **Re-apply bulk loads**: Bulk-loaded HFiles discovered in the WAL directory for the same time range + are bulk-loaded into the target table. + +If `--to-datetime` is not specified, PITR restores to the last known safe replication checkpoint, +which is the earliest timestamp across the active RegionServers for which WAL entries before that +point are known to have been persisted to the continuous backup WAL directory. + ## A Warning on File System Growth As a reminder, incremental backups are implemented via retaining the write-ahead logs which HBase primarily uses for data durability. Thus, to ensure that all data needing to be included in a backup is still available in the system, the HBase backup and restore feature -retains all write-ahead logs since the last backup until the next incremental backup is executed. +retains all write-ahead logs since the last backup until the next incremental backup is executed. With continuous backup enabled, +WALs are additionally replicated to the WAL directory. + +WALs in the continuous backup directory must be retained as long as they are required to support the +configured PITR window and the available backup chain. Backup deletion performs validation to prevent +deletion of a backup that would leave a table without a valid PITR recovery path, and removes WALs +that are no longer required. Like HBase Snapshots, this can have an expectedly large impact on the HDFS usage of HBase for high volume tables. Take care in enabling and using the backup and restore feature, specifically with a mind to removing backup sessions when they are not actively being used. +Running incremental backups on a regular schedule reduces WAL retention on the source cluster for tables under continuous backup, +because WALs are read from the WAL directory rather than from the source cluster. Review Comment: I'm a bit confused about this part. In the previous section you mentioned >When incremental backups are run for tables that have continuous backup enabled, the backup process reads WALs from the WAL directory instead of collecting them from the source cluster. This eliminates the need to retain WALs on the source cluster between incremental backup runs and reduces pressure on the source cluster's log management. Which means to me that with Continuous Backup we don't need to keep WAL files in the cluster at all, while the above comment suggests that we need to keep WAL files in the cluster between incrementals. Which one is true? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
