[jira] [Created] (HBASE-27688) HFile splitting occurs during bulkload, the CREATE_TIME_TS of hfileinfo is 0

[alan.zhao (Jira)]

alan.zhao created HBASE-27688:
---------------------------------

             Summary: HFile splitting occurs during bulkload, the 
CREATE_TIME_TS of hfileinfo is 0
                 Key: HBASE-27688
                 URL: https://issues.apache.org/jira/browse/HBASE-27688
             Project: HBase
          Issue Type: Bug
            Reporter: alan.zhao


If HFile splitting occurs during bulkload, the CREATE_TIME_TS of hfileinfo 
=0,When HFile is copied after splitting, CREATE_TIME_TS of the original file is 
not copied。
{code:java}
##BulkLoadHFilesTool.class 
/**
 * Copy half of an HFile into a new HFile.
 */
private static void copyHFileHalf(Configuration conf, Path inFile, Path outFile,
  Reference reference, ColumnFamilyDescriptor familyDescriptor) throws 
IOException {
  FileSystem fs = inFile.getFileSystem(conf);
  CacheConfig cacheConf = CacheConfig.DISABLED;
  HalfStoreFileReader halfReader = null;
  StoreFileWriter halfWriter = null;
  try {
    。。。
    HFileContext hFileContext = new 
HFileContextBuilder().withCompression(compression)
      .withChecksumType(StoreUtils.getChecksumType(conf))
      
.withBytesPerCheckSum(StoreUtils.getBytesPerChecksum(conf)).withBlockSize(blocksize)
      
.withDataBlockEncoding(familyDescriptor.getDataBlockEncoding()).withIncludesTags(true)
      .build();
// TODO .withCreateTime(EnvironmentEdgeManager.currentTime())      

halfWriter = new StoreFileWriter.Builder(conf, cacheConf, 
fs).withFilePath(outFile)
      .withBloomType(bloomFilterType).withFileContext(hFileContext).build();
    HFileScanner scanner = halfReader.getScanner(false, false, false);
    scanner.seekTo();
    do {
      halfWriter.append(scanner.getCell());
    } while (scanner.next());

    for (Map.Entry<byte[], byte[]> entry : fileInfo.entrySet()) {
      if (shouldCopyHFileMetaKey(entry.getKey())) {
        halfWriter.appendFileInfo(entry.getKey(), entry.getValue());
      }
    }
  } finally {
    。。。
  }
} 


##get lastMajorCompactionTs metric

  lastMajorCompactionTs = this.region.getOldestHfileTs(true);
...
  long now = EnvironmentEdgeManager.currentTime();
  return now - lastMajorCompactionTs;
...
##
public long getOldestHfileTs(boolean majorCompactionOnly) throws IOException {
  long result = Long.MAX_VALUE;
  for (HStore store : stores.values()) {
    Collection<HStoreFile> storeFiles = store.getStorefiles();
   ...
    for (HStoreFile file : storeFiles) {
      StoreFileReader sfReader = file.getReader();
     ...
      result = Math.min(result, reader.getFileContext().getFileCreateTime());
    }
  }
  return result == Long.MAX_VALUE ? 0 : result;
}{code}
 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)