[jira] [Commented] (HBASE-11195) Potentially improve block locality during major compaction for old regions
[ https://issues.apache.org/jira/browse/HBASE-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951631#comment-16951631 ] jackylau commented on HBASE-11195: -- none of my response :( Does anynone konw that why this sf.isMajorCompaction() condition need in the code ? > Potentially improve block locality during major compaction for old regions > -- > > Key: HBASE-11195 > URL: https://issues.apache.org/jira/browse/HBASE-11195 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.0.0, 0.94.26, 0.98.10, 2.0.0 >Reporter: churro morales >Assignee: churro morales >Priority: Major > Fix For: 1.0.0, 0.98.10, 0.94.27 > > Attachments: HBASE-11195-0.94.patch, HBASE-11195-0.98.patch, > HBASE-11195-0.98.v1.patch, HBASE-11195.patch, HBASE-11195.patch > > > This might be a specific use case. But we have some regions which are no > longer written to (due to the key). Those regions have 1 store file and they > are very old, they haven't been written to in a while. We still use these > regions to read from so locality would be nice. > I propose putting a configuration option: something like > hbase.hstore.min.locality.to.skip.major.compact [between 0 and 1] > such that you can decide whether or not to skip major compaction for an old > region with a single store file. > I'll attach a patch, let me know what you guys think. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-11195) Potentially improve block locality during major compaction for old regions
[ https://issues.apache.org/jira/browse/HBASE-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16949081#comment-16949081 ] jackylau commented on HBASE-11195: -- hi [~churromorales] ，why need this sf.isMajorCompaction() condition in the code ? > Potentially improve block locality during major compaction for old regions > -- > > Key: HBASE-11195 > URL: https://issues.apache.org/jira/browse/HBASE-11195 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.0.0, 0.94.26, 0.98.10, 2.0.0 >Reporter: churro morales >Assignee: churro morales >Priority: Major > Fix For: 1.0.0, 0.98.10, 0.94.27 > > Attachments: HBASE-11195-0.94.patch, HBASE-11195-0.98.patch, > HBASE-11195-0.98.v1.patch, HBASE-11195.patch, HBASE-11195.patch > > > This might be a specific use case. But we have some regions which are no > longer written to (due to the key). Those regions have 1 store file and they > are very old, they haven't been written to in a while. We still use these > regions to read from so locality would be nice. > I propose putting a configuration option: something like > hbase.hstore.min.locality.to.skip.major.compact [between 0 and 1] > such that you can decide whether or not to skip major compaction for an old > region with a single store file. > I'll attach a patch, let me know what you guys think. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-11195) Potentially improve block locality during major compaction for old regions
[ https://issues.apache.org/jira/browse/HBASE-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14367845#comment-14367845 ] Lars Hofhansl commented on HBASE-11195: --- Pretty sure this is wrong. We return true (please run the major compaction) if the current locality index is the min requested one. As it stands we'll now - by default - _always_ compact all the old files, which we did not do before. This looks like a critical issue to me. Potentially improve block locality during major compaction for old regions -- Key: HBASE-11195 URL: https://issues.apache.org/jira/browse/HBASE-11195 Project: HBase Issue Type: Improvement Affects Versions: 1.0.0, 2.0.0, 0.94.26, 0.98.10 Reporter: churro morales Assignee: churro morales Fix For: 1.0.0, 2.0.0, 0.98.10, 0.94.27 Attachments: HBASE-11195-0.94.patch, HBASE-11195-0.98.patch, HBASE-11195.patch, HBASE-11195.patch This might be a specific use case. But we have some regions which are no longer written to (due to the key). Those regions have 1 store file and they are very old, they haven't been written to in a while. We still use these regions to read from so locality would be nice. I propose putting a configuration option: something like hbase.hstore.min.locality.to.skip.major.compact [between 0 and 1] such that you can decide whether or not to skip major compaction for an old region with a single store file. I'll attach a patch, let me know what you guys think. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11195) Potentially improve block locality during major compaction for old regions
[ https://issues.apache.org/jira/browse/HBASE-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14367854#comment-14367854 ] churro morales commented on HBASE-11195: [~lhofhansl] oh my, I don't know how that happened but it looks to me that the 98 patch is incorrect and a quick look at the other patches shows they are correct, I have no idea how this happened. I am so sorry I can get a proper patch to you guys asap. Potentially improve block locality during major compaction for old regions -- Key: HBASE-11195 URL: https://issues.apache.org/jira/browse/HBASE-11195 Project: HBase Issue Type: Improvement Affects Versions: 1.0.0, 2.0.0, 0.94.26, 0.98.10 Reporter: churro morales Assignee: churro morales Fix For: 1.0.0, 2.0.0, 0.98.10, 0.94.27 Attachments: HBASE-11195-0.94.patch, HBASE-11195-0.98.patch, HBASE-11195.patch, HBASE-11195.patch This might be a specific use case. But we have some regions which are no longer written to (due to the key). Those regions have 1 store file and they are very old, they haven't been written to in a while. We still use these regions to read from so locality would be nice. I propose putting a configuration option: something like hbase.hstore.min.locality.to.skip.major.compact [between 0 and 1] such that you can decide whether or not to skip major compaction for an old region with a single store file. I'll attach a patch, let me know what you guys think. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11195) Potentially improve block locality during major compaction for old regions
[ https://issues.apache.org/jira/browse/HBASE-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14367859#comment-14367859 ] churro morales commented on HBASE-11195: do you want me to create a new ticket with a patch, or just add it here? It should be blockLocalityIndex comConf.getMinLocalityToForceCompact() looks to be fine for trunk and 94 after looking at the patches Potentially improve block locality during major compaction for old regions -- Key: HBASE-11195 URL: https://issues.apache.org/jira/browse/HBASE-11195 Project: HBase Issue Type: Improvement Affects Versions: 1.0.0, 2.0.0, 0.94.26, 0.98.10 Reporter: churro morales Assignee: churro morales Fix For: 1.0.0, 2.0.0, 0.98.10, 0.94.27 Attachments: HBASE-11195-0.94.patch, HBASE-11195-0.98.patch, HBASE-11195.patch, HBASE-11195.patch This might be a specific use case. But we have some regions which are no longer written to (due to the key). Those regions have 1 store file and they are very old, they haven't been written to in a while. We still use these regions to read from so locality would be nice. I propose putting a configuration option: something like hbase.hstore.min.locality.to.skip.major.compact [between 0 and 1] such that you can decide whether or not to skip major compaction for an old region with a single store file. I'll attach a patch, let me know what you guys think. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11195) Potentially improve block locality during major compaction for old regions
[ https://issues.apache.org/jira/browse/HBASE-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14367874#comment-14367874 ] Lars Hofhansl commented on HBASE-11195: --- I'm happy to that (new ticket), just wanted to confirm that I did not miss anything. I'll do the fix, and then we'll just release 0.98.12 quickly (right [~apurtell] :) ) Potentially improve block locality during major compaction for old regions -- Key: HBASE-11195 URL: https://issues.apache.org/jira/browse/HBASE-11195 Project: HBase Issue Type: Improvement Affects Versions: 1.0.0, 2.0.0, 0.94.26, 0.98.10 Reporter: churro morales Assignee: churro morales Fix For: 1.0.0, 2.0.0, 0.98.10, 0.94.27 Attachments: HBASE-11195-0.94.patch, HBASE-11195-0.98.patch, HBASE-11195.patch, HBASE-11195.patch This might be a specific use case. But we have some regions which are no longer written to (due to the key). Those regions have 1 store file and they are very old, they haven't been written to in a while. We still use these regions to read from so locality would be nice. I propose putting a configuration option: something like hbase.hstore.min.locality.to.skip.major.compact [between 0 and 1] such that you can decide whether or not to skip major compaction for an old region with a single store file. I'll attach a patch, let me know what you guys think. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11195) Potentially improve block locality during major compaction for old regions
[ https://issues.apache.org/jira/browse/HBASE-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14367880#comment-14367880 ] Lars Hofhansl commented on HBASE-11195: --- Actually lemme apply this to 0.94 (I hadn't before). Potentially improve block locality during major compaction for old regions -- Key: HBASE-11195 URL: https://issues.apache.org/jira/browse/HBASE-11195 Project: HBase Issue Type: Improvement Affects Versions: 1.0.0, 2.0.0, 0.94.26, 0.98.10 Reporter: churro morales Assignee: churro morales Fix For: 1.0.0, 2.0.0, 0.98.10 Attachments: HBASE-11195-0.94.patch, HBASE-11195-0.98.patch, HBASE-11195.patch, HBASE-11195.patch This might be a specific use case. But we have some regions which are no longer written to (due to the key). Those regions have 1 store file and they are very old, they haven't been written to in a while. We still use these regions to read from so locality would be nice. I propose putting a configuration option: something like hbase.hstore.min.locality.to.skip.major.compact [between 0 and 1] such that you can decide whether or not to skip major compaction for an old region with a single store file. I'll attach a patch, let me know what you guys think. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11195) Potentially improve block locality during major compaction for old regions
[ https://issues.apache.org/jira/browse/HBASE-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14368123#comment-14368123 ] Hudson commented on HBASE-11195: FAILURE: Integrated in HBase-0.94-JDK7 #234 (See [https://builds.apache.org/job/HBase-0.94-JDK7/234/]) HBASE-11195 Addendum for TestHeapSize. (larsh: rev 260f2137bdb8b4ae839f5cc285509f34e31a006b) * src/main/java/org/apache/hadoop/hbase/regionserver/Store.java Potentially improve block locality during major compaction for old regions -- Key: HBASE-11195 URL: https://issues.apache.org/jira/browse/HBASE-11195 Project: HBase Issue Type: Improvement Affects Versions: 1.0.0, 2.0.0, 0.94.26, 0.98.10 Reporter: churro morales Assignee: churro morales Fix For: 1.0.0, 2.0.0, 0.98.10, 0.94.27 Attachments: HBASE-11195-0.94.patch, HBASE-11195-0.98.patch, HBASE-11195-0.98.v1.patch, HBASE-11195.patch, HBASE-11195.patch This might be a specific use case. But we have some regions which are no longer written to (due to the key). Those regions have 1 store file and they are very old, they haven't been written to in a while. We still use these regions to read from so locality would be nice. I propose putting a configuration option: something like hbase.hstore.min.locality.to.skip.major.compact [between 0 and 1] such that you can decide whether or not to skip major compaction for an old region with a single store file. I'll attach a patch, let me know what you guys think. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11195) Potentially improve block locality during major compaction for old regions
[ https://issues.apache.org/jira/browse/HBASE-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14368199#comment-14368199 ] Hudson commented on HBASE-11195: SUCCESS: Integrated in HBase-0.94 #1466 (See [https://builds.apache.org/job/HBase-0.94/1466/]) HBASE-11195 Addendum for TestHeapSize. (larsh: rev 260f2137bdb8b4ae839f5cc285509f34e31a006b) * src/main/java/org/apache/hadoop/hbase/regionserver/Store.java Potentially improve block locality during major compaction for old regions -- Key: HBASE-11195 URL: https://issues.apache.org/jira/browse/HBASE-11195 Project: HBase Issue Type: Improvement Affects Versions: 1.0.0, 2.0.0, 0.94.26, 0.98.10 Reporter: churro morales Assignee: churro morales Fix For: 1.0.0, 2.0.0, 0.98.10, 0.94.27 Attachments: HBASE-11195-0.94.patch, HBASE-11195-0.98.patch, HBASE-11195-0.98.v1.patch, HBASE-11195.patch, HBASE-11195.patch This might be a specific use case. But we have some regions which are no longer written to (due to the key). Those regions have 1 store file and they are very old, they haven't been written to in a while. We still use these regions to read from so locality would be nice. I propose putting a configuration option: something like hbase.hstore.min.locality.to.skip.major.compact [between 0 and 1] such that you can decide whether or not to skip major compaction for an old region with a single store file. I'll attach a patch, let me know what you guys think. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11195) Potentially improve block locality during major compaction for old regions
[ https://issues.apache.org/jira/browse/HBASE-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14368096#comment-14368096 ] Hudson commented on HBASE-11195: FAILURE: Integrated in HBase-0.94.27 #52 (See [https://builds.apache.org/job/HBase-0.94.27/52/]) HBASE-11195 Addendum for TestHeapSize. (larsh: rev 260f2137bdb8b4ae839f5cc285509f34e31a006b) * src/main/java/org/apache/hadoop/hbase/regionserver/Store.java Potentially improve block locality during major compaction for old regions -- Key: HBASE-11195 URL: https://issues.apache.org/jira/browse/HBASE-11195 Project: HBase Issue Type: Improvement Affects Versions: 1.0.0, 2.0.0, 0.94.26, 0.98.10 Reporter: churro morales Assignee: churro morales Fix For: 1.0.0, 2.0.0, 0.98.10, 0.94.27 Attachments: HBASE-11195-0.94.patch, HBASE-11195-0.98.patch, HBASE-11195-0.98.v1.patch, HBASE-11195.patch, HBASE-11195.patch This might be a specific use case. But we have some regions which are no longer written to (due to the key). Those regions have 1 store file and they are very old, they haven't been written to in a while. We still use these regions to read from so locality would be nice. I propose putting a configuration option: something like hbase.hstore.min.locality.to.skip.major.compact [between 0 and 1] such that you can decide whether or not to skip major compaction for an old region with a single store file. I'll attach a patch, let me know what you guys think. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11195) Potentially improve block locality during major compaction for old regions
[ https://issues.apache.org/jira/browse/HBASE-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14368185#comment-14368185 ] Hudson commented on HBASE-11195: SUCCESS: Integrated in HBase-0.94-security #579 (See [https://builds.apache.org/job/HBase-0.94-security/579/]) HBASE-11195 Addendum for TestHeapSize. (larsh: rev 260f2137bdb8b4ae839f5cc285509f34e31a006b) * src/main/java/org/apache/hadoop/hbase/regionserver/Store.java Potentially improve block locality during major compaction for old regions -- Key: HBASE-11195 URL: https://issues.apache.org/jira/browse/HBASE-11195 Project: HBase Issue Type: Improvement Affects Versions: 1.0.0, 2.0.0, 0.94.26, 0.98.10 Reporter: churro morales Assignee: churro morales Fix For: 1.0.0, 2.0.0, 0.98.10, 0.94.27 Attachments: HBASE-11195-0.94.patch, HBASE-11195-0.98.patch, HBASE-11195-0.98.v1.patch, HBASE-11195.patch, HBASE-11195.patch This might be a specific use case. But we have some regions which are no longer written to (due to the key). Those regions have 1 store file and they are very old, they haven't been written to in a while. We still use these regions to read from so locality would be nice. I propose putting a configuration option: something like hbase.hstore.min.locality.to.skip.major.compact [between 0 and 1] such that you can decide whether or not to skip major compaction for an old region with a single store file. I'll attach a patch, let me know what you guys think. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11195) Potentially improve block locality during major compaction for old regions
[ https://issues.apache.org/jira/browse/HBASE-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14368079#comment-14368079 ] Andrew Purtell commented on HBASE-11195: Well I was certainly wrong about this being low risk. Potentially improve block locality during major compaction for old regions -- Key: HBASE-11195 URL: https://issues.apache.org/jira/browse/HBASE-11195 Project: HBase Issue Type: Improvement Affects Versions: 1.0.0, 2.0.0, 0.94.26, 0.98.10 Reporter: churro morales Assignee: churro morales Fix For: 1.0.0, 2.0.0, 0.98.10, 0.94.27 Attachments: HBASE-11195-0.94.patch, HBASE-11195-0.98.patch, HBASE-11195-0.98.v1.patch, HBASE-11195.patch, HBASE-11195.patch This might be a specific use case. But we have some regions which are no longer written to (due to the key). Those regions have 1 store file and they are very old, they haven't been written to in a while. We still use these regions to read from so locality would be nice. I propose putting a configuration option: something like hbase.hstore.min.locality.to.skip.major.compact [between 0 and 1] such that you can decide whether or not to skip major compaction for an old region with a single store file. I'll attach a patch, let me know what you guys think. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11195) Potentially improve block locality during major compaction for old regions
[
https://issues.apache.org/jira/browse/HBASE-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14367732#comment-14367732
]
Lars Hofhansl commented on HBASE-11195:
---
Looking at the patch... Isn't this wrong:
{code}
if (blockLocalityIndex comConf.getMinLocalityToForceCompact()) {
if (LOG.isDebugEnabled()) {
LOG.debug(Major compaction triggered on only store + this +
; to make hdfs blocks local, current blockLocalityIndex is
+
blockLocalityIndex + (min +
comConf.getMinLocalityToForceCompact() +
));
}
result = true;
{code}
It seems this should be:
{code}
if (blockLocalityIndex comConf.getMinLocalityToForceCompact()) {
{code}
We want to compact if the current locality index is *less* then what we want it
to be, not more.
Since we default min locality index to 0, we'd always compact these file now,
right? That's bad.
I hope I am missing something.
[~churromorales], [~apurtell].
Potentially improve block locality during major compaction for old regions
--
Key: HBASE-11195
URL: https://issues.apache.org/jira/browse/HBASE-11195
Project: HBase
Issue Type: Improvement
Affects Versions: 1.0.0, 2.0.0, 0.94.26, 0.98.10
Reporter: churro morales
Assignee: churro morales
Fix For: 1.0.0, 2.0.0, 0.98.10, 0.94.27
Attachments: HBASE-11195-0.94.patch, HBASE-11195-0.98.patch,
HBASE-11195.patch, HBASE-11195.patch
This might be a specific use case. But we have some regions which are no
longer written to (due to the key). Those regions have 1 store file and they
are very old, they haven't been written to in a while. We still use these
regions to read from so locality would be nice.
I propose putting a configuration option: something like
hbase.hstore.min.locality.to.skip.major.compact [between 0 and 1]
such that you can decide whether or not to skip major compaction for an old
region with a single store file.
I'll attach a patch, let me know what you guys think.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-11195) Potentially improve block locality during major compaction for old regions
[ https://issues.apache.org/jira/browse/HBASE-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14367947#comment-14367947 ] Hudson commented on HBASE-11195: FAILURE: Integrated in HBase-0.94-security #577 (See [https://builds.apache.org/job/HBase-0.94-security/577/]) HBASE-11195 Potentially improve block locality during major compaction for old regions. (churro morales) (larsh: rev 459fc00ed8ae048efca737c90559a1b0e47bf300) * src/main/java/org/apache/hadoop/hbase/regionserver/Store.java Potentially improve block locality during major compaction for old regions -- Key: HBASE-11195 URL: https://issues.apache.org/jira/browse/HBASE-11195 Project: HBase Issue Type: Improvement Affects Versions: 1.0.0, 2.0.0, 0.94.26, 0.98.10 Reporter: churro morales Assignee: churro morales Fix For: 1.0.0, 2.0.0, 0.98.10, 0.94.27 Attachments: HBASE-11195-0.94.patch, HBASE-11195-0.98.patch, HBASE-11195-0.98.v1.patch, HBASE-11195.patch, HBASE-11195.patch This might be a specific use case. But we have some regions which are no longer written to (due to the key). Those regions have 1 store file and they are very old, they haven't been written to in a while. We still use these regions to read from so locality would be nice. I propose putting a configuration option: something like hbase.hstore.min.locality.to.skip.major.compact [between 0 and 1] such that you can decide whether or not to skip major compaction for an old region with a single store file. I'll attach a patch, let me know what you guys think. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11195) Potentially improve block locality during major compaction for old regions
[ https://issues.apache.org/jira/browse/HBASE-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14367955#comment-14367955 ] Hudson commented on HBASE-11195: FAILURE: Integrated in HBase-0.94-JDK7 #232 (See [https://builds.apache.org/job/HBase-0.94-JDK7/232/]) HBASE-11195 Potentially improve block locality during major compaction for old regions. (churro morales) (larsh: rev 459fc00ed8ae048efca737c90559a1b0e47bf300) * src/main/java/org/apache/hadoop/hbase/regionserver/Store.java Potentially improve block locality during major compaction for old regions -- Key: HBASE-11195 URL: https://issues.apache.org/jira/browse/HBASE-11195 Project: HBase Issue Type: Improvement Affects Versions: 1.0.0, 2.0.0, 0.94.26, 0.98.10 Reporter: churro morales Assignee: churro morales Fix For: 1.0.0, 2.0.0, 0.98.10, 0.94.27 Attachments: HBASE-11195-0.94.patch, HBASE-11195-0.98.patch, HBASE-11195-0.98.v1.patch, HBASE-11195.patch, HBASE-11195.patch This might be a specific use case. But we have some regions which are no longer written to (due to the key). Those regions have 1 store file and they are very old, they haven't been written to in a while. We still use these regions to read from so locality would be nice. I propose putting a configuration option: something like hbase.hstore.min.locality.to.skip.major.compact [between 0 and 1] such that you can decide whether or not to skip major compaction for an old region with a single store file. I'll attach a patch, let me know what you guys think. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11195) Potentially improve block locality during major compaction for old regions
[ https://issues.apache.org/jira/browse/HBASE-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14367895#comment-14367895 ] Lars Hofhansl commented on HBASE-11195: --- Thanks [~churromorales]. Lemme apply this in the sub-task I just created. Potentially improve block locality during major compaction for old regions -- Key: HBASE-11195 URL: https://issues.apache.org/jira/browse/HBASE-11195 Project: HBase Issue Type: Improvement Affects Versions: 1.0.0, 2.0.0, 0.94.26, 0.98.10 Reporter: churro morales Assignee: churro morales Fix For: 1.0.0, 2.0.0, 0.98.10, 0.94.27 Attachments: HBASE-11195-0.94.patch, HBASE-11195-0.98.patch, HBASE-11195-0.98.v1.patch, HBASE-11195.patch, HBASE-11195.patch This might be a specific use case. But we have some regions which are no longer written to (due to the key). Those regions have 1 store file and they are very old, they haven't been written to in a while. We still use these regions to read from so locality would be nice. I propose putting a configuration option: something like hbase.hstore.min.locality.to.skip.major.compact [between 0 and 1] such that you can decide whether or not to skip major compaction for an old region with a single store file. I'll attach a patch, let me know what you guys think. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11195) Potentially improve block locality during major compaction for old regions
[ https://issues.apache.org/jira/browse/HBASE-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14367912#comment-14367912 ] Lars Hofhansl commented on HBASE-11195: --- And no need to be sorry. These things happen, and we all missed it. (I just noticed because I wanted to enable this in our cluster and looked at the code to see what I should set the value to) Potentially improve block locality during major compaction for old regions -- Key: HBASE-11195 URL: https://issues.apache.org/jira/browse/HBASE-11195 Project: HBase Issue Type: Improvement Affects Versions: 1.0.0, 2.0.0, 0.94.26, 0.98.10 Reporter: churro morales Assignee: churro morales Fix For: 1.0.0, 2.0.0, 0.98.10, 0.94.27 Attachments: HBASE-11195-0.94.patch, HBASE-11195-0.98.patch, HBASE-11195-0.98.v1.patch, HBASE-11195.patch, HBASE-11195.patch This might be a specific use case. But we have some regions which are no longer written to (due to the key). Those regions have 1 store file and they are very old, they haven't been written to in a while. We still use these regions to read from so locality would be nice. I propose putting a configuration option: something like hbase.hstore.min.locality.to.skip.major.compact [between 0 and 1] such that you can decide whether or not to skip major compaction for an old region with a single store file. I'll attach a patch, let me know what you guys think. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11195) Potentially improve block locality during major compaction for old regions
[ https://issues.apache.org/jira/browse/HBASE-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14367940#comment-14367940 ] Hudson commented on HBASE-11195: FAILURE: Integrated in HBase-0.94 #1464 (See [https://builds.apache.org/job/HBase-0.94/1464/]) HBASE-11195 Potentially improve block locality during major compaction for old regions. (churro morales) (larsh: rev 459fc00ed8ae048efca737c90559a1b0e47bf300) * src/main/java/org/apache/hadoop/hbase/regionserver/Store.java Potentially improve block locality during major compaction for old regions -- Key: HBASE-11195 URL: https://issues.apache.org/jira/browse/HBASE-11195 Project: HBase Issue Type: Improvement Affects Versions: 1.0.0, 2.0.0, 0.94.26, 0.98.10 Reporter: churro morales Assignee: churro morales Fix For: 1.0.0, 2.0.0, 0.98.10, 0.94.27 Attachments: HBASE-11195-0.94.patch, HBASE-11195-0.98.patch, HBASE-11195-0.98.v1.patch, HBASE-11195.patch, HBASE-11195.patch This might be a specific use case. But we have some regions which are no longer written to (due to the key). Those regions have 1 store file and they are very old, they haven't been written to in a while. We still use these regions to read from so locality would be nice. I propose putting a configuration option: something like hbase.hstore.min.locality.to.skip.major.compact [between 0 and 1] such that you can decide whether or not to skip major compaction for an old region with a single store file. I'll attach a patch, let me know what you guys think. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11195) Potentially improve block locality during major compaction for old regions
[ https://issues.apache.org/jira/browse/HBASE-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14290940#comment-14290940 ] Enis Soztutar commented on HBASE-11195: --- Sorry this fell through the cracks. Thanks for committing to branch-1.0 as well. Potentially improve block locality during major compaction for old regions -- Key: HBASE-11195 URL: https://issues.apache.org/jira/browse/HBASE-11195 Project: HBase Issue Type: Improvement Affects Versions: 1.0.0, 2.0.0, 0.94.26, 0.98.10 Reporter: churro morales Assignee: churro morales Fix For: 1.0.0, 2.0.0, 0.98.10, 0.94.27 Attachments: HBASE-11195-0.94.patch, HBASE-11195-0.98.patch, HBASE-11195.patch, HBASE-11195.patch This might be a specific use case. But we have some regions which are no longer written to (due to the key). Those regions have 1 store file and they are very old, they haven't been written to in a while. We still use these regions to read from so locality would be nice. I propose putting a configuration option: something like hbase.hstore.min.locality.to.skip.major.compact [between 0 and 1] such that you can decide whether or not to skip major compaction for an old region with a single store file. I'll attach a patch, let me know what you guys think. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11195) Potentially improve block locality during major compaction for old regions
[ https://issues.apache.org/jira/browse/HBASE-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14288122#comment-14288122 ] Hudson commented on HBASE-11195: SUCCESS: Integrated in HBase-1.0 #674 (See [https://builds.apache.org/job/HBase-1.0/674/]) HBASE-11195 Potentially improve block locality during major compaction for old regions (apurtell: rev 11dbbcdeee542d73a73b0b1e5c8b85a51b5fe031) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/RatioBasedCompactionPolicy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionConfiguration.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionPolicy.java Potentially improve block locality during major compaction for old regions -- Key: HBASE-11195 URL: https://issues.apache.org/jira/browse/HBASE-11195 Project: HBase Issue Type: Improvement Affects Versions: 1.0.0, 2.0.0, 0.94.26, 0.98.10 Reporter: churro morales Assignee: churro morales Fix For: 1.0.0, 2.0.0, 0.98.10, 0.94.27 Attachments: HBASE-11195-0.94.patch, HBASE-11195-0.98.patch, HBASE-11195.patch, HBASE-11195.patch This might be a specific use case. But we have some regions which are no longer written to (due to the key). Those regions have 1 store file and they are very old, they haven't been written to in a while. We still use these regions to read from so locality would be nice. I propose putting a configuration option: something like hbase.hstore.min.locality.to.skip.major.compact [between 0 and 1] such that you can decide whether or not to skip major compaction for an old region with a single store file. I'll attach a patch, let me know what you guys think. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11195) Potentially improve block locality during major compaction for old regions
[ https://issues.apache.org/jira/browse/HBASE-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14287891#comment-14287891 ] Andrew Purtell commented on HBASE-11195: Since Enis didn't reply I'll take that as assent and push now for this case. Potentially improve block locality during major compaction for old regions -- Key: HBASE-11195 URL: https://issues.apache.org/jira/browse/HBASE-11195 Project: HBase Issue Type: Improvement Affects Versions: 1.0.0, 2.0.0, 0.94.26, 0.98.10 Reporter: churro morales Assignee: churro morales Attachments: HBASE-11195-0.94.patch, HBASE-11195-0.98.patch, HBASE-11195.patch, HBASE-11195.patch This might be a specific use case. But we have some regions which are no longer written to (due to the key). Those regions have 1 store file and they are very old, they haven't been written to in a while. We still use these regions to read from so locality would be nice. I propose putting a configuration option: something like hbase.hstore.min.locality.to.skip.major.compact [between 0 and 1] such that you can decide whether or not to skip major compaction for an old region with a single store file. I'll attach a patch, let me know what you guys think. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11195) Potentially improve block locality during major compaction for old regions
[ https://issues.apache.org/jira/browse/HBASE-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14286586#comment-14286586 ] Nick Dimiduk commented on HBASE-11195: -- By policy, if it went in for 0.98 we must include it in 1.0, yes? Potentially improve block locality during major compaction for old regions -- Key: HBASE-11195 URL: https://issues.apache.org/jira/browse/HBASE-11195 Project: HBase Issue Type: Improvement Affects Versions: 1.0.0, 2.0.0, 0.94.26, 0.98.10 Reporter: churro morales Assignee: churro morales Attachments: HBASE-11195-0.94.patch, HBASE-11195-0.98.patch, HBASE-11195.patch, HBASE-11195.patch This might be a specific use case. But we have some regions which are no longer written to (due to the key). Those regions have 1 store file and they are very old, they haven't been written to in a while. We still use these regions to read from so locality would be nice. I propose putting a configuration option: something like hbase.hstore.min.locality.to.skip.major.compact [between 0 and 1] such that you can decide whether or not to skip major compaction for an old region with a single store file. I'll attach a patch, let me know what you guys think. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11195) Potentially improve block locality during major compaction for old regions
[ https://issues.apache.org/jira/browse/HBASE-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14283604#comment-14283604 ] Hudson commented on HBASE-11195: SUCCESS: Integrated in HBase-0.98 #806 (See [https://builds.apache.org/job/HBase-0.98/806/]) HBASE-11195 Potentially improve block locality during major compaction for old regions (apurtell: rev f7f58f75c95d3647194467833bb56d6bab37472d) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionConfiguration.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/RatioBasedCompactionPolicy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java Potentially improve block locality during major compaction for old regions -- Key: HBASE-11195 URL: https://issues.apache.org/jira/browse/HBASE-11195 Project: HBase Issue Type: Improvement Affects Versions: 1.0.0, 2.0.0, 0.94.26, 0.98.10 Reporter: churro morales Assignee: churro morales Attachments: HBASE-11195-0.94.patch, HBASE-11195-0.98.patch, HBASE-11195.patch, HBASE-11195.patch This might be a specific use case. But we have some regions which are no longer written to (due to the key). Those regions have 1 store file and they are very old, they haven't been written to in a while. We still use these regions to read from so locality would be nice. I propose putting a configuration option: something like hbase.hstore.min.locality.to.skip.major.compact [between 0 and 1] such that you can decide whether or not to skip major compaction for an old region with a single store file. I'll attach a patch, let me know what you guys think. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11195) Potentially improve block locality during major compaction for old regions
[ https://issues.apache.org/jira/browse/HBASE-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14283394#comment-14283394 ] Hudson commented on HBASE-11195: SUCCESS: Integrated in HBase-TRUNK #6038 (See [https://builds.apache.org/job/HBase-TRUNK/6038/]) HBASE-11195 Potentially improve block locality during major compaction for old regions (apurtell: rev f71b980d3715c94478930b614e63250ebb678899) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionPolicy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/RatioBasedCompactionPolicy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionConfiguration.java Potentially improve block locality during major compaction for old regions -- Key: HBASE-11195 URL: https://issues.apache.org/jira/browse/HBASE-11195 Project: HBase Issue Type: Improvement Affects Versions: 1.0.0, 2.0.0, 0.94.26, 0.98.10 Reporter: churro morales Assignee: churro morales Attachments: HBASE-11195-0.94.patch, HBASE-11195-0.98.patch, HBASE-11195.patch, HBASE-11195.patch This might be a specific use case. But we have some regions which are no longer written to (due to the key). Those regions have 1 store file and they are very old, they haven't been written to in a while. We still use these regions to read from so locality would be nice. I propose putting a configuration option: something like hbase.hstore.min.locality.to.skip.major.compact [between 0 and 1] such that you can decide whether or not to skip major compaction for an old region with a single store file. I'll attach a patch, let me know what you guys think. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11195) Potentially improve block locality during major compaction for old regions
[ https://issues.apache.org/jira/browse/HBASE-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14283367#comment-14283367 ] Hudson commented on HBASE-11195: FAILURE: Integrated in HBase-1.1 #91 (See [https://builds.apache.org/job/HBase-1.1/91/]) HBASE-11195 Potentially improve block locality during major compaction for old regions (apurtell: rev 6a1fcf2b85a7d303d9cab92dda4a90a9171cbafc) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionPolicy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionConfiguration.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/RatioBasedCompactionPolicy.java Potentially improve block locality during major compaction for old regions -- Key: HBASE-11195 URL: https://issues.apache.org/jira/browse/HBASE-11195 Project: HBase Issue Type: Improvement Affects Versions: 1.0.0, 2.0.0, 0.94.26, 0.98.10 Reporter: churro morales Assignee: churro morales Attachments: HBASE-11195-0.94.patch, HBASE-11195-0.98.patch, HBASE-11195.patch, HBASE-11195.patch This might be a specific use case. But we have some regions which are no longer written to (due to the key). Those regions have 1 store file and they are very old, they haven't been written to in a while. We still use these regions to read from so locality would be nice. I propose putting a configuration option: something like hbase.hstore.min.locality.to.skip.major.compact [between 0 and 1] such that you can decide whether or not to skip major compaction for an old region with a single store file. I'll attach a patch, let me know what you guys think. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11195) Potentially improve block locality during major compaction for old regions
[ https://issues.apache.org/jira/browse/HBASE-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14283548#comment-14283548 ] Hudson commented on HBASE-11195: SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #768 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/768/]) HBASE-11195 Potentially improve block locality during major compaction for old regions (apurtell: rev f7f58f75c95d3647194467833bb56d6bab37472d) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/RatioBasedCompactionPolicy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionConfiguration.java Potentially improve block locality during major compaction for old regions -- Key: HBASE-11195 URL: https://issues.apache.org/jira/browse/HBASE-11195 Project: HBase Issue Type: Improvement Affects Versions: 1.0.0, 2.0.0, 0.94.26, 0.98.10 Reporter: churro morales Assignee: churro morales Attachments: HBASE-11195-0.94.patch, HBASE-11195-0.98.patch, HBASE-11195.patch, HBASE-11195.patch This might be a specific use case. But we have some regions which are no longer written to (due to the key). Those regions have 1 store file and they are very old, they haven't been written to in a while. We still use these regions to read from so locality would be nice. I propose putting a configuration option: something like hbase.hstore.min.locality.to.skip.major.compact [between 0 and 1] such that you can decide whether or not to skip major compaction for an old region with a single store file. I'll attach a patch, let me know what you guys think. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11195) Potentially improve block locality during major compaction for old regions
[ https://issues.apache.org/jira/browse/HBASE-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14282892#comment-14282892 ] Andrew Purtell commented on HBASE-11195: Committing to 0.98+ today unless objection. Potentially improve block locality during major compaction for old regions -- Key: HBASE-11195 URL: https://issues.apache.org/jira/browse/HBASE-11195 Project: HBase Issue Type: Improvement Affects Versions: 1.0.0, 2.0.0, 0.94.26, 0.98.10 Reporter: churro morales Assignee: churro morales Attachments: HBASE-11195-0.94.patch, HBASE-11195-0.98.patch, HBASE-11195.patch This might be a specific use case. But we have some regions which are no longer written to (due to the key). Those regions have 1 store file and they are very old, they haven't been written to in a while. We still use these regions to read from so locality would be nice. I propose putting a configuration option: something like hbase.hstore.min.locality.to.skip.major.compact [between 0 and 1] such that you can decide whether or not to skip major compaction for an old region with a single store file. I'll attach a patch, let me know what you guys think. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11195) Potentially improve block locality during major compaction for old regions
[ https://issues.apache.org/jira/browse/HBASE-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14280608#comment-14280608 ] Andrew Purtell commented on HBASE-11195: Skimmed the patch, lgtm. It's low risk. This introduces a new tunable hbase.hstore.min.locality.to.skip.major.compact that unless changed from default (0) does not change behavior. I'll commit this later today unless there are additional comments or an objection. Potentially improve block locality during major compaction for old regions -- Key: HBASE-11195 URL: https://issues.apache.org/jira/browse/HBASE-11195 Project: HBase Issue Type: Improvement Affects Versions: 1.0.0, 2.0.0, 0.94.26, 0.98.10 Reporter: churro morales Assignee: churro morales Attachments: HBASE-11195-0.94.patch, HBASE-11195-0.98.patch, HBASE-11195.patch This might be a specific use case. But we have some regions which are no longer written to (due to the key). Those regions have 1 store file and they are very old, they haven't been written to in a while. We still use these regions to read from so locality would be nice. I propose putting a configuration option: something like hbase.hstore.min.locality.to.skip.major.compact [between 0 and 1] such that you can decide whether or not to skip major compaction for an old region with a single store file. I'll attach a patch, let me know what you guys think. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11195) Potentially improve block locality during major compaction for old regions
[
https://issues.apache.org/jira/browse/HBASE-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14280616#comment-14280616
]
Andrew Purtell commented on HBASE-11195:
I think in the refactored else clause of {{if (sf.isMajorCompaction() (cfTtl
== HConstants.FOREVER || oldest cfTtl))}} we should amend the DEBUG level log
to also print the locality, since it's a factor in the decision now. Will make
that trivial change upon commit.
Potentially improve block locality during major compaction for old regions
--
Key: HBASE-11195
URL: https://issues.apache.org/jira/browse/HBASE-11195
Project: HBase
Issue Type: Improvement
Affects Versions: 1.0.0, 2.0.0, 0.94.26, 0.98.10
Reporter: churro morales
Assignee: churro morales
Attachments: HBASE-11195-0.94.patch, HBASE-11195-0.98.patch,
HBASE-11195.patch
This might be a specific use case. But we have some regions which are no
longer written to (due to the key). Those regions have 1 store file and they
are very old, they haven't been written to in a while. We still use these
regions to read from so locality would be nice.
I propose putting a configuration option: something like
hbase.hstore.min.locality.to.skip.major.compact [between 0 and 1]
such that you can decide whether or not to skip major compaction for an old
region with a single store file.
I'll attach a patch, let me know what you guys think.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-11195) Potentially improve block locality during major compaction for old regions
[
https://issues.apache.org/jira/browse/HBASE-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14006606#comment-14006606
]
Ted Yu commented on HBASE-11195:
Trunk patch touches several files. I am not sure if all changes are related to
major compaction.
e.g. this change in ClientSideRegionScanner :
{code}
-if (values == null || values.isEmpty()) {
+if (values.isEmpty()) {
{code}
Potentially improve block locality during major compaction for old regions
--
Key: HBASE-11195
URL: https://issues.apache.org/jira/browse/HBASE-11195
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.19
Reporter: churro morales
Assignee: churro morales
Attachments: HBASE-11195-0.94.patch, HBASE-11195-trunk.patch
This might be a specific use case. But we have some regions which are no
longer written to (due to the key). Those regions have 1 store file and they
are very old, they haven't been written to in a while. We still use these
regions to read from so locality would be nice.
I propose putting a configuration option: something like
hbase.hstore.min.locality.to.skip.major.compact [between 0 and 1]
such that you can decide whether or not to skip major compaction for an old
region with a single store file.
I'll attach a patch, let me know what you guys think.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11195) Potentially improve block locality during major compaction for old regions
[ https://issues.apache.org/jira/browse/HBASE-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14006638#comment-14006638 ] churro morales commented on HBASE-11195: Hi Ted, I must have uploaded the wrong file. Sorry it was done late at night. Please disregard that one, I only touch two files in the patch. I'll upload the correct one. My fault. Potentially improve block locality during major compaction for old regions -- Key: HBASE-11195 URL: https://issues.apache.org/jira/browse/HBASE-11195 Project: HBase Issue Type: Improvement Affects Versions: 0.94.19 Reporter: churro morales Assignee: churro morales Attachments: HBASE-11195-0.94.patch, HBASE-11195-trunk.patch This might be a specific use case. But we have some regions which are no longer written to (due to the key). Those regions have 1 store file and they are very old, they haven't been written to in a while. We still use these regions to read from so locality would be nice. I propose putting a configuration option: something like hbase.hstore.min.locality.to.skip.major.compact [between 0 and 1] such that you can decide whether or not to skip major compaction for an old region with a single store file. I'll attach a patch, let me know what you guys think. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11195) Potentially improve block locality during major compaction for old regions
[ https://issues.apache.org/jira/browse/HBASE-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14006651#comment-14006651 ] churro morales commented on HBASE-11195: The correct patch is: v2. Sorry about that. Potentially improve block locality during major compaction for old regions -- Key: HBASE-11195 URL: https://issues.apache.org/jira/browse/HBASE-11195 Project: HBase Issue Type: Improvement Affects Versions: 0.94.19 Reporter: churro morales Assignee: churro morales Attachments: HBASE-11195-0.94.patch, HBASE-11195-trunk-v2.patch, HBASE-11195-trunk.patch This might be a specific use case. But we have some regions which are no longer written to (due to the key). Those regions have 1 store file and they are very old, they haven't been written to in a while. We still use these regions to read from so locality would be nice. I propose putting a configuration option: something like hbase.hstore.min.locality.to.skip.major.compact [between 0 and 1] such that you can decide whether or not to skip major compaction for an old region with a single store file. I'll attach a patch, let me know what you guys think. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11195) Potentially improve block locality during major compaction for old regions
[ https://issues.apache.org/jira/browse/HBASE-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14005489#comment-14005489 ] Andrew Purtell commented on HBASE-11195: I think this is an interesting idea. Any chance for a patch against trunk? Potentially improve block locality during major compaction for old regions -- Key: HBASE-11195 URL: https://issues.apache.org/jira/browse/HBASE-11195 Project: HBase Issue Type: Improvement Affects Versions: 0.94.19 Reporter: churro morales Assignee: churro morales Attachments: HBASE-11195-0.94.patch This might be a specific use case. But we have some regions which are no longer written to (due to the key). Those regions have 1 store file and they are very old, they haven't been written to in a while. We still use these regions to read from so locality would be nice. I propose putting a configuration option: something like hbase.hstore.min.locality.to.skip.major.compact [between 0 and 1] such that you can decide whether or not to skip major compaction for an old region with a single store file. I'll attach a patch, let me know what you guys think. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11195) Potentially improve block locality during major compaction for old regions
[ https://issues.apache.org/jira/browse/HBASE-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14005628#comment-14005628 ] churro morales commented on HBASE-11195: Hey guys, I'm not too familiar with trunk, but after looking at the code, I came up with something. I don't love it as the getHostname call now becomes a static method of RSRpcServices.java. But its something we can talk and iterate on. Let me know what you folks think. Potentially improve block locality during major compaction for old regions -- Key: HBASE-11195 URL: https://issues.apache.org/jira/browse/HBASE-11195 Project: HBase Issue Type: Improvement Affects Versions: 0.94.19 Reporter: churro morales Assignee: churro morales Attachments: HBASE-11195-0.94.patch This might be a specific use case. But we have some regions which are no longer written to (due to the key). Those regions have 1 store file and they are very old, they haven't been written to in a while. We still use these regions to read from so locality would be nice. I propose putting a configuration option: something like hbase.hstore.min.locality.to.skip.major.compact [between 0 and 1] such that you can decide whether or not to skip major compaction for an old region with a single store file. I'll attach a patch, let me know what you guys think. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11195) Potentially improve block locality during major compaction for old regions
[ https://issues.apache.org/jira/browse/HBASE-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14000828#comment-14000828 ] Jean-Marc Spaggiari commented on HBASE-11195: - Hi Churro, does it apply to Trunk to? Potentially improve block locality during major compaction for old regions -- Key: HBASE-11195 URL: https://issues.apache.org/jira/browse/HBASE-11195 Project: HBase Issue Type: Improvement Affects Versions: 0.94.19 Reporter: churro morales Assignee: churro morales Attachments: HBASE-11195-0.94.patch This might be a specific use case. But we have some regions which are no longer written to (due to the key). Those regions have 1 store file and they are very old, they haven't been written to in a while. We still use these regions to read from so locality would be nice. I propose putting a configuration option: something like hbase.hstore.min.locality.to.skip.major.compact [between 0 and 1] such that you can decide whether or not to skip major compaction for an old region with a single store file. I'll attach a patch, let me know what you guys think. -- This message was sent by Atlassian JIRA (v6.2#6252)
