[
https://issues.apache.org/jira/browse/HBASE-30501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Umesh Kumar Kumawat updated HBASE-30501:
----------------------------------------
Description:
LoadIncrementalHFiles automatically splits an HFile that crosses region
boundaries into .bottom and .top files under .tmp and tracks them in an
in-memory queue. If the invocation
fails after splitting but before loading—for example, due to the maximum
HFiles-per-region/family limit—the queue is lost. A new invocation may not
rediscover .tmp files, while the original HFile may already have been deleted.
Update the retry behavior to preserve the original HFile until the split
outputs are successfully loaded, or persist and rediscover split outputs across
invocations. Add tests covering failure after splitting followed by a fresh
LoadIncrementalHFiles retry.
was:
LoadIncrementalHFiles automatically splits an HFile that crosses region
boundaries into .bottom and .top files under .tmp and tracks them in an
in-memory queue. If the invocation
fails after splitting but before loading—for example, due to the maximum
HFiles-per-region/family limit—the queue is lost. A new invocation may not
rediscover .tmp files, while the original HFile may already have been deleted.
Update the retry behavior to preserve the original HFile until the split
outputs are successfully loaded, or persist and rediscover split outputs across
invocations. Add tests covering failure after splitting followed by a fresh
LoadIncrementalHFiles retry.
> HBASE-30501 Preserve original HFiles when splitting for bulk load
> -----------------------------------------------------------------
>
> Key: HBASE-30501
> URL: https://issues.apache.org/jira/browse/HBASE-30501
> Project: HBase
> Issue Type: Bug
> Reporter: Ujjawal Kumar
> Assignee: Ujjawal Kumar
> Priority: Major
> Labels: pull-request-available
>
>
> LoadIncrementalHFiles automatically splits an HFile that crosses region
> boundaries into .bottom and .top files under .tmp and tracks them in an
> in-memory queue. If the invocation
> fails after splitting but before loading—for example, due to the maximum
> HFiles-per-region/family limit—the queue is lost. A new invocation may not
> rediscover .tmp files, while the original HFile may already have been
> deleted.
>
>
>
> Update the retry behavior to preserve the original HFile until the split
> outputs are successfully loaded, or persist and rediscover split outputs
> across invocations. Add tests covering failure after splitting followed by
> a fresh LoadIncrementalHFiles retry.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
