[jira] [Updated] (HBASE-5369) Compaction selection based on the hotness of the HFile's block in the block cache
[ https://issues.apache.org/jira/browse/HBASE-5369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-5369: Component/s: Performance Compaction selection based on the hotness of the HFile's block in the block cache - Key: HBASE-5369 URL: https://issues.apache.org/jira/browse/HBASE-5369 Project: HBase Issue Type: Improvement Components: BlockCache, Compaction, Performance Reporter: Liyin Tang Assignee: Liyin Tang HBase reserves a large set memory for the block cache and the cached blocks will be age out in a LRU fashion. Obviously, we don't want to age out the blocks which are still hot. However, when the compactions are starting, these hot blocks may naturally be invalid. Considering that the block cache has already known which HFiles these hot blocks come from, the compaction selection algorithm could just simply skip compact these HFiles until these block cache become cold. For example, if there is a HFile and 80% of blocks for this HFile is be cached, which means this HFile is really hot, then just skip this HFile during the compaction selection. The percentage of hot blocks should be configured as a high bar to make sure that HBase are still making progress for the compaction. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-5369) Compaction selection based on the hotness of the HFile's block in the block cache
[ https://issues.apache.org/jira/browse/HBASE-5369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5369: - Component/s: Compaction BlockCache Compaction selection based on the hotness of the HFile's block in the block cache - Key: HBASE-5369 URL: https://issues.apache.org/jira/browse/HBASE-5369 Project: HBase Issue Type: Improvement Components: BlockCache, Compaction Reporter: Liyin Tang Assignee: Liyin Tang HBase reserves a large set memory for the block cache and the cached blocks will be age out in a LRU fashion. Obviously, we don't want to age out the blocks which are still hot. However, when the compactions are starting, these hot blocks may naturally be invalid. Considering that the block cache has already known which HFiles these hot blocks come from, the compaction selection algorithm could just simply skip compact these HFiles until these block cache become cold. For example, if there is a HFile and 80% of blocks for this HFile is be cached, which means this HFile is really hot, then just skip this HFile during the compaction selection. The percentage of hot blocks should be configured as a high bar to make sure that HBase are still making progress for the compaction. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-5369) Compaction selection based on the hotness of the HFile's block in the block cache
[ https://issues.apache.org/jira/browse/HBASE-5369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Bautin updated HBASE-5369: -- Description: HBase reserves a large set memory for the block cache and the cached blocks will be age out in a LRU fashion. Obviously, we don't want to age out the blocks which are still hot. However, when the compactions are starting, these hot blocks may naturally be invalid. Considering that the block cache has already known which HFiles these hot blocks come from, the compaction selection algorithm could just simply skip compact these HFiles until these block cache become cold. was: HBase reserves a large set memory for the block cache and the cached blocks will be age out in a LRU fashion. Obviously, we don't want to age out the blocks which are still hot. However, when the compactions are starting, these hot blocks may naturally be invalid. Considering that the block cache has already known which HFiles these hot blocks come from, the compaction selection algorithm could just simply skip compact these HFiles until these block cache become cold. Furthermore, the HBase could compact multiple HFiles into two HFiles. One of them only contains hot blocks which are supposed be cached directly. Compaction selection based on the hotness of the HFile's block in the block cache - Key: HBASE-5369 URL: https://issues.apache.org/jira/browse/HBASE-5369 Project: HBase Issue Type: Improvement Reporter: Liyin Tang Assignee: Liyin Tang HBase reserves a large set memory for the block cache and the cached blocks will be age out in a LRU fashion. Obviously, we don't want to age out the blocks which are still hot. However, when the compactions are starting, these hot blocks may naturally be invalid. Considering that the block cache has already known which HFiles these hot blocks come from, the compaction selection algorithm could just simply skip compact these HFiles until these block cache become cold. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5369) Compaction selection based on the hotness of the HFile's block in the block cache
[ https://issues.apache.org/jira/browse/HBASE-5369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HBASE-5369: -- Description: HBase reserves a large set memory for the block cache and the cached blocks will be age out in a LRU fashion. Obviously, we don't want to age out the blocks which are still hot. However, when the compactions are starting, these hot blocks may naturally be invalid. Considering that the block cache has already known which HFiles these hot blocks come from, the compaction selection algorithm could just simply skip compact these HFiles until these block cache become cold. For example, if there is a HFile and 80% of blocks for this HFile is be cached, which means this HFile is really hot, then just skip this HFile during the compaction selection. The percentage of hot blocks should be configured as a high bar to make sure that HBase are still making progress for the compaction. was: HBase reserves a large set memory for the block cache and the cached blocks will be age out in a LRU fashion. Obviously, we don't want to age out the blocks which are still hot. However, when the compactions are starting, these hot blocks may naturally be invalid. Considering that the block cache has already known which HFiles these hot blocks come from, the compaction selection algorithm could just simply skip compact these HFiles until these block cache become cold. Compaction selection based on the hotness of the HFile's block in the block cache - Key: HBASE-5369 URL: https://issues.apache.org/jira/browse/HBASE-5369 Project: HBase Issue Type: Improvement Reporter: Liyin Tang Assignee: Liyin Tang HBase reserves a large set memory for the block cache and the cached blocks will be age out in a LRU fashion. Obviously, we don't want to age out the blocks which are still hot. However, when the compactions are starting, these hot blocks may naturally be invalid. Considering that the block cache has already known which HFiles these hot blocks come from, the compaction selection algorithm could just simply skip compact these HFiles until these block cache become cold. For example, if there is a HFile and 80% of blocks for this HFile is be cached, which means this HFile is really hot, then just skip this HFile during the compaction selection. The percentage of hot blocks should be configured as a high bar to make sure that HBase are still making progress for the compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
