[
https://issues.apache.org/jira/browse/HBASE-28836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17910429#comment-17910429
]
Viraj Jasani commented on HBASE-28836:
--------------------------------------
[~apurtell] [~stoty] [~zhangduo] The changes are backported to all active
branches. It is not reverted yet. Before 2.6 and 2.5 releases, we need to
decide what we need to do with this. As per Istvan, if this is still creating
problem, we will need to revert everywhere because he mentioned that it was
reproducible on master branch. I am still not aware of the root cause but
better to resolve/revert this now to avoid repeated RC attempts with 2.6.2 and
2.5.11 releases.
> Parallelize the archival of compacted files
> --------------------------------------------
>
> Key: HBASE-28836
> URL: https://issues.apache.org/jira/browse/HBASE-28836
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Affects Versions: 2.5.10
> Reporter: Aman Poonia
> Assignee: Aman Poonia
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0-alpha-1
>
>
> While splitting a region in hbase it has to cleanup compacted files for
> bookkeeping.
>
> Currently we do it sequentially and that is good enough because for hdfs as
> it is a fast operation. When we do the same in s3 it becomes a issue. We need
> to paralleize this to make it faster.
> {code:java}
> // code placeholder
> for (File file : toArchive) {
> // if its a file archive it
> try {
> LOG.trace("Archiving {}", file);
> if (file.isFile()) {
> // attempt to archive the file
> if (!resolveAndArchiveFile(baseArchiveDir, file, startTime)) {
> LOG.warn("Couldn't archive " + file + " into backup directory: "
> + baseArchiveDir);
> failures.add(file);
> }
> } else {
> // otherwise its a directory and we need to archive all files
> LOG.trace("{} is a directory, archiving children files", file);
> // so we add the directory name to the one base archive
> Path parentArchiveDir = new Path(baseArchiveDir, file.getName());
> // and then get all the files from that directory and attempt to
> // archive those too
> Collection<File> children = file.getChildren();
> failures.addAll(resolveAndArchive(fs, parentArchiveDir, children,
> start));
> }
> } catch (IOException e) {
> LOG.warn("Failed to archive {}", file, e);
> failures.add(file);
> }
> } {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
