[
https://issues.apache.org/jira/browse/HBASE-3693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13011088#comment-13011088
]
stack commented on HBASE-3693:
------------------------------
I like your patch Liyin. Can I commit it? (I'm nervous about committing your
stuff -- I seem to be jumping the gun).
I'd fix this misspelling: stroeFileNum
Also, why a setModificationTimeStamp? Seems odd being able to set this? It
comes from the fs?
I'd like to get this into 0.90.2.
> isMajorCompaction() check triggers lots of listStatus DFS RPC calls from HBase
> ------------------------------------------------------------------------------
>
> Key: HBASE-3693
> URL: https://issues.apache.org/jira/browse/HBASE-3693
> Project: HBase
> Issue Type: Improvement
> Reporter: Kannan Muthukkaruppan
> Assignee: Liyin Tang
> Attachments: Hbase-3693[r1085306].patch
>
>
> We noticed that there are lots of listStatus calls on the ColumnFamily
> directories within each region, coming from this codepath:
> {code}
> compactionSelection()
> --> isMajorCompaction
> --> getLowestTimestamp()
> --> FileStatus[] stats = fs.listStatus(p);
> {code}
> So on every compactionSelection() we're taking this hit. While not
> immediately an issue, just from log inspection, this accounts for quite a
> large number of RPCs to namenode at the moment and seems like an unnecessary
> load to be sending to the namenode.
> Seems like it would be easy to cache the timestamp for each opened/created
> StoreFile, in memory, in the region server, and avoid going to DFS each time
> for this information.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
