[
https://issues.apache.org/jira/browse/HBASE-27636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
selina.yan updated HBASE-27636:
-------------------------------
Description:
When HFileOutputFormat2 is used to create an hfile, the CREATE_TIME_TS of
hfileinfo will not be assigned when creating an hfileContext, resulting in the
value of lastMajorCompactionAge is the timestamp of the current time.
{code:java}
##HFileOutputFormat2.class
HFileContextBuilder contextBuilder = new HFileContextBuilder()
.withCompression(compression)
.withChecksumType(HStore.getChecksumType(conf))
.withBytesPerCheckSum(HStore.getBytesPerChecksum(conf))
.withBlockSize(blockSize);
##get lastMajorCompactionTs metric
lastMajorCompactionTs = this.region.getOldestHfileTs(true);
...
long now = EnvironmentEdgeManager.currentTime();
return now - lastMajorCompactionTs;
...
##
public long getOldestHfileTs(boolean majorCompactionOnly) throws IOException {
long result = Long.MAX_VALUE;
for (HStore store : stores.values()) {
Collection<HStoreFile> storeFiles = store.getStorefiles();
...
for (HStoreFile file : storeFiles) {
StoreFileReader sfReader = file.getReader();
...
result = Math.min(result, reader.getFileContext().getFileCreateTime());
}
}
return result == Long.MAX_VALUE ? 0 : result;
}{code}
was:
When HFileOutputFormat2 is used to create an hfile, the fileCreateTime will not
be assigned when creating an hfileContext, resulting in the
lastMajorCompactionAge indicator of the region being the timestamp of the
current time after buckload to hbase.
{code:java}
##HFileOutputFormat2.class
HFileContextBuilder contextBuilder = new HFileContextBuilder()
.withCompression(compression)
.withChecksumType(HStore.getChecksumType(conf))
.withBytesPerCheckSum(HStore.getBytesPerChecksum(conf))
.withBlockSize(blockSize);
if (HFile.getFormatVersion(conf) >= HFile.MIN_FORMAT_VERSION_WITH_TAGS) {
contextBuilder.withIncludesTags(true);
}
contextBuilder.withDataBlockEncoding(encoding);
HFileContext hFileContext = contextBuilder.build();
##get lastMajorCompactionTs metric
long lastMajorCompactionTs = 0L;
try {
lastMajorCompactionTs = this.region.getOldestHfileTs(true);
} catch (IOException ioe) {
LOG.error("Could not load HFile info ", ioe);
}
long now = EnvironmentEdgeManager.currentTime();
return now - lastMajorCompactionTs;
}
...
##
public long getOldestHfileTs(boolean majorCompactionOnly) throws IOException {
long result = Long.MAX_VALUE;
for (HStore store : stores.values()) {
Collection<HStoreFile> storeFiles = store.getStorefiles();
...
for (HStoreFile file : storeFiles) {
StoreFileReader sfReader = file.getReader();
...
result = Math.min(result, reader.getFileContext().getFileCreateTime());
}
}
return result == Long.MAX_VALUE ? 0 : result;
}{code}
> The "CREATE_TIME_TS" value of the hfile generated by the HFileOutputFormat2
> class is 0
> --------------------------------------------------------------------------------------
>
> Key: HBASE-27636
> URL: https://issues.apache.org/jira/browse/HBASE-27636
> Project: HBase
> Issue Type: Bug
> Components: hbase-connectors
> Reporter: selina.yan
> Assignee: selina.yan
> Priority: Major
>
> When HFileOutputFormat2 is used to create an hfile, the CREATE_TIME_TS of
> hfileinfo will not be assigned when creating an hfileContext, resulting in
> the value of lastMajorCompactionAge is the timestamp of the current time.
> {code:java}
> ##HFileOutputFormat2.class
> HFileContextBuilder contextBuilder = new HFileContextBuilder()
> .withCompression(compression)
> .withChecksumType(HStore.getChecksumType(conf))
>
> .withBytesPerCheckSum(HStore.getBytesPerChecksum(conf))
> .withBlockSize(blockSize);
> ##get lastMajorCompactionTs metric
> lastMajorCompactionTs = this.region.getOldestHfileTs(true);
> ...
> long now = EnvironmentEdgeManager.currentTime();
> return now - lastMajorCompactionTs;
> ...
> ##
> public long getOldestHfileTs(boolean majorCompactionOnly) throws IOException {
> long result = Long.MAX_VALUE;
> for (HStore store : stores.values()) {
> Collection<HStoreFile> storeFiles = store.getStorefiles();
> ...
> for (HStoreFile file : storeFiles) {
> StoreFileReader sfReader = file.getReader();
> ...
> result = Math.min(result, reader.getFileContext().getFileCreateTime());
> }
> }
> return result == Long.MAX_VALUE ? 0 : result;
> }{code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
