Apache9 commented on a change in pull request #3842:
URL: https://github.com/apache/hbase/pull/3842#discussion_r748684750
##########
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionFileSystem.java
##########
@@ -662,37 +665,77 @@ public Path splitStoreFile(RegionInfo hri, String
familyName, HStoreFile f, byte
LOG.warn("Found an already existing split file for {}. Assuming this is
a recovery.", p);
return p;
}
+ boolean createLinkFile = false;
if (splitPolicy == null ||
!splitPolicy.skipStoreFileRangeCheck(familyName)) {
// Check whether the split row lies in the range of the store file
// If it is outside the range, return directly.
f.initReader();
try {
Cell splitKey = PrivateCellUtil.createFirstOnRow(splitRow);
+ Optional<Cell> lastKey = f.getLastKey();
+ Optional<Cell> firstKey = f.getFirstKey();
if (top) {
//check if larger than last key.
- Optional<Cell> lastKey = f.getLastKey();
// If lastKey is null means storefile is empty.
if (!lastKey.isPresent()) {
return null;
}
if (f.getComparator().compare(splitKey, lastKey.get()) > 0) {
return null;
}
+ if (splitPolicy != null && splitPolicy.isLinkFileEnabledInSplit() &&
firstKey.isPresent()
+ && f.getComparator().compare(splitKey, firstKey.get()) <= 0) {
+ LOG.debug("Will create linkFile for " + f.getPath() + ",
top=true");
+ createLinkFile = true;
+ }
} else {
//check if smaller than first key
- Optional<Cell> firstKey = f.getFirstKey();
// If firstKey is null means storefile is empty.
if (!firstKey.isPresent()) {
return null;
}
if (f.getComparator().compare(splitKey, firstKey.get()) < 0) {
return null;
}
+ if (splitPolicy != null && splitPolicy.isLinkFileEnabledInSplit() &&
lastKey.isPresent()
+ && f.getComparator().compare(splitKey, lastKey.get()) >= 0) {
+ LOG.debug("Will create linkFile for " + f.getPath() + ",
top=false");
+ createLinkFile = true;
+ }
}
} finally {
f.closeStoreFile(f.getCacheConf() != null ?
f.getCacheConf().shouldEvictOnClose() : true);
}
}
+ if (createLinkFile) {
+ // create HFileLink file instead of Reference file for child
+ String hfileName = f.getPath().getName();
+ TableName linkedTable = regionInfoForFs.getTable();
+ String linkedRegion = regionInfoForFs.getEncodedName();
+ try {
+ if (HFileLink.isHFileLink(hfileName)) {
+ Matcher m = LINK_NAME_PATTERN.matcher(hfileName);
+ if (!m.matches()) {
+ throw new IllegalArgumentException(hfileName + " is not a valid
HFileLink name!");
+ }
+ linkedTable = TableName.valueOf(m.group(1), m.group(2));
+ linkedRegion = m.group(3);
+ hfileName = m.group(4);
+ }
+ // must create back reference here
+ HFileLink.create(conf, fs, splitDir, familyName,
hri.getTable().getNameAsString(),
Review comment:
A dumb question, where is the back reference file placed? Under the dir
of the parent region or the sub regions? If it is placed in the parent region,
how to do clean up if we fail in the middle?
##########
File path: hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
##########
@@ -1038,6 +1038,9 @@
public static final String HBASE_REGION_SPLIT_POLICY_KEY =
"hbase.regionserver.region.split.policy";
+ public static final String HBASE_REGION_SPLIT_USE_HFILELINK_ENABLED =
Review comment:
Do we really need this flag? For me I prefer we just do it by default.
You can send an email to the dev list to see if there are any objections, if
not, let's just do it!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
