[jira] [Updated] (HBASE-15480) Bloom Filter check needs to be more efficient for array
[ https://issues.apache.org/jira/browse/HBASE-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Koetke updated HBASE-15480: -- Status: Patch Available (was: In Progress) > Bloom Filter check needs to be more efficient for array > --- > > Key: HBASE-15480 > URL: https://issues.apache.org/jira/browse/HBASE-15480 > Project: HBase > Issue Type: Improvement > Components: Performance >Affects Versions: 1.0.3 >Reporter: Walter Koetke >Assignee: Walter Koetke > Fix For: 1.0.4 > > Attachments: BloomFilterCheckOneByOne.tiff, > HBASE-15480-branch-1.0.patch, HBASE-15480.patch > > > It is currently inefficient to do lots of bloom filter checks. Each check has > overhead like going to the block cache to retrieve the block and recording > metrics. It would be good to have one bloom filter check api that does a > bunch of checks without so much block retrieval and metrics updates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15480) Bloom Filter check needs to be more efficient for array
[ https://issues.apache.org/jira/browse/HBASE-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Koetke updated HBASE-15480: -- Attachment: HBASE-15480.patch Attaching patch file for master > Bloom Filter check needs to be more efficient for array > --- > > Key: HBASE-15480 > URL: https://issues.apache.org/jira/browse/HBASE-15480 > Project: HBase > Issue Type: Improvement > Components: Performance >Affects Versions: 1.0.3 >Reporter: Walter Koetke >Assignee: Walter Koetke > Fix For: 1.0.4 > > Attachments: BloomFilterCheckOneByOne.tiff, > HBASE-15480-branch-1.0.patch, HBASE-15480.patch > > > It is currently inefficient to do lots of bloom filter checks. Each check has > overhead like going to the block cache to retrieve the block and recording > metrics. It would be good to have one bloom filter check api that does a > bunch of checks without so much block retrieval and metrics updates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15556) need extensible ConsistencyControl interface
[ https://issues.apache.org/jira/browse/HBASE-15556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15220363#comment-15220363 ] Walter Koetke commented on HBASE-15556: --- Hold that thought about our custom bit...don't want to attach it here yet. > need extensible ConsistencyControl interface > > > Key: HBASE-15556 > URL: https://issues.apache.org/jira/browse/HBASE-15556 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.0.3 >Reporter: Walter Koetke >Assignee: Walter Koetke > Fix For: 1.0.4 > > Attachments: HBASE-15556-branch-1.0-02.patch, > HBASE-15556-branch-1.0.patch > > > The class MultiVersionConsistencyControl should be abstracted into an > interface ConsistencyControl so it can be extended by a configured custom > implementation class, with MultiVersionConsistencyControl as the default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15556) need extensible ConsistencyControl interface
[ https://issues.apache.org/jira/browse/HBASE-15556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Koetke updated HBASE-15556: -- Attachment: (was: SIMultiVersionConsistencyControl.java) > need extensible ConsistencyControl interface > > > Key: HBASE-15556 > URL: https://issues.apache.org/jira/browse/HBASE-15556 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.0.3 >Reporter: Walter Koetke >Assignee: Walter Koetke > Fix For: 1.0.4 > > Attachments: HBASE-15556-branch-1.0-02.patch, > HBASE-15556-branch-1.0.patch > > > The class MultiVersionConsistencyControl should be abstracted into an > interface ConsistencyControl so it can be extended by a configured custom > implementation class, with MultiVersionConsistencyControl as the default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15556) need extensible ConsistencyControl interface
[ https://issues.apache.org/jira/browse/HBASE-15556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Koetke updated HBASE-15556: -- Attachment: SIMultiVersionConsistencyControl.java Attaching our alternate impl class per previous comment. > need extensible ConsistencyControl interface > > > Key: HBASE-15556 > URL: https://issues.apache.org/jira/browse/HBASE-15556 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.0.3 >Reporter: Walter Koetke >Assignee: Walter Koetke > Fix For: 1.0.4 > > Attachments: HBASE-15556-branch-1.0-02.patch, > HBASE-15556-branch-1.0.patch, SIMultiVersionConsistencyControl.java > > > The class MultiVersionConsistencyControl should be abstracted into an > interface ConsistencyControl so it can be extended by a configured custom > implementation class, with MultiVersionConsistencyControl as the default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15556) need extensible ConsistencyControl interface
[ https://issues.apache.org/jira/browse/HBASE-15556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15220341#comment-15220341 ] Walter Koetke commented on HBASE-15556: --- Sure, Sean, your concern is understood. I can start a discussion on dev@ but in the meantime, this might help: It is not so much that we needed a significantly different implementation. Rather, we just found we were able to improve performance bottlenecks by replacing synchronized code with usage of ConcurrentHashMap and AtomicLong. I'm attaching our 'custom' implementation class (SIMultiVersionConsistencyControl) so you can see what we mean. It seemed 'safer' to make it pluggable but maybe it's cleaner just to propose our changes go directly into the MultiVersionConsistencyControl class. What do you think? > need extensible ConsistencyControl interface > > > Key: HBASE-15556 > URL: https://issues.apache.org/jira/browse/HBASE-15556 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.0.3 >Reporter: Walter Koetke >Assignee: Walter Koetke > Fix For: 1.0.4 > > Attachments: HBASE-15556-branch-1.0-02.patch, > HBASE-15556-branch-1.0.patch > > > The class MultiVersionConsistencyControl should be abstracted into an > interface ConsistencyControl so it can be extended by a configured custom > implementation class, with MultiVersionConsistencyControl as the default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15556) need extensible ConsistencyControl interface
[ https://issues.apache.org/jira/browse/HBASE-15556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Koetke updated HBASE-15556: -- Attachment: HBASE-15556-branch-1.0-02.patch Attaching again with proper patch file name. > need extensible ConsistencyControl interface > > > Key: HBASE-15556 > URL: https://issues.apache.org/jira/browse/HBASE-15556 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.0.3 >Reporter: Walter Koetke >Assignee: Walter Koetke > Fix For: 1.0.4 > > Attachments: HBASE-15556-branch-1.0-02.patch, > HBASE-15556-branch-1.0.patch > > > The class MultiVersionConsistencyControl should be abstracted into an > interface ConsistencyControl so it can be extended by a configured custom > implementation class, with MultiVersionConsistencyControl as the default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15556) need extensible ConsistencyControl interface
[ https://issues.apache.org/jira/browse/HBASE-15556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Koetke updated HBASE-15556: -- Status: Patch Available (was: In Progress) > need extensible ConsistencyControl interface > > > Key: HBASE-15556 > URL: https://issues.apache.org/jira/browse/HBASE-15556 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.0.3 >Reporter: Walter Koetke >Assignee: Walter Koetke > Fix For: 1.0.4 > > Attachments: HBASE-15556-branch-1.0-02.patch, > HBASE-15556-branch-1.0.patch > > > The class MultiVersionConsistencyControl should be abstracted into an > interface ConsistencyControl so it can be extended by a configured custom > implementation class, with MultiVersionConsistencyControl as the default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15556) need extensible ConsistencyControl interface
[ https://issues.apache.org/jira/browse/HBASE-15556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Koetke updated HBASE-15556: -- Status: In Progress (was: Patch Available) > need extensible ConsistencyControl interface > > > Key: HBASE-15556 > URL: https://issues.apache.org/jira/browse/HBASE-15556 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.0.3 >Reporter: Walter Koetke >Assignee: Walter Koetke > Fix For: 1.0.4 > > Attachments: HBASE-15556-branch-1.0.patch > > > The class MultiVersionConsistencyControl should be abstracted into an > interface ConsistencyControl so it can be extended by a configured custom > implementation class, with MultiVersionConsistencyControl as the default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15556) need extensible ConsistencyControl interface
[ https://issues.apache.org/jira/browse/HBASE-15556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Koetke updated HBASE-15556: -- Attachment: (was: HBASE-15556-v2-branch-1.0.patch) > need extensible ConsistencyControl interface > > > Key: HBASE-15556 > URL: https://issues.apache.org/jira/browse/HBASE-15556 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.0.3 >Reporter: Walter Koetke >Assignee: Walter Koetke > Fix For: 1.0.4 > > Attachments: HBASE-15556-branch-1.0.patch > > > The class MultiVersionConsistencyControl should be abstracted into an > interface ConsistencyControl so it can be extended by a configured custom > implementation class, with MultiVersionConsistencyControl as the default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15556) need extensible ConsistencyControl interface
[ https://issues.apache.org/jira/browse/HBASE-15556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Koetke updated HBASE-15556: -- Status: Patch Available (was: In Progress) > need extensible ConsistencyControl interface > > > Key: HBASE-15556 > URL: https://issues.apache.org/jira/browse/HBASE-15556 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.0.3 >Reporter: Walter Koetke >Assignee: Walter Koetke > Fix For: 1.0.4 > > Attachments: HBASE-15556-branch-1.0.patch, > HBASE-15556-v2-branch-1.0.patch > > > The class MultiVersionConsistencyControl should be abstracted into an > interface ConsistencyControl so it can be extended by a configured custom > implementation class, with MultiVersionConsistencyControl as the default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15556) need extensible ConsistencyControl interface
[ https://issues.apache.org/jira/browse/HBASE-15556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Koetke updated HBASE-15556: -- Attachment: HBASE-15556-v2-branch-1.0.patch Attaching V2 of the patch with the suggested changes. > need extensible ConsistencyControl interface > > > Key: HBASE-15556 > URL: https://issues.apache.org/jira/browse/HBASE-15556 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.0.3 >Reporter: Walter Koetke >Assignee: Walter Koetke > Fix For: 1.0.4 > > Attachments: HBASE-15556-branch-1.0.patch, > HBASE-15556-v2-branch-1.0.patch > > > The class MultiVersionConsistencyControl should be abstracted into an > interface ConsistencyControl so it can be extended by a configured custom > implementation class, with MultiVersionConsistencyControl as the default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15480) Bloom Filter check needs to be more efficient for array
[ https://issues.apache.org/jira/browse/HBASE-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15219987#comment-15219987 ] Walter Koetke commented on HBASE-15480: --- To answer Ted's question about the performance improvement, see the attachment BloomFilterCheckOneByOne, screenshot about how much time was going here. All of this went away for us with the change in place. > Bloom Filter check needs to be more efficient for array > --- > > Key: HBASE-15480 > URL: https://issues.apache.org/jira/browse/HBASE-15480 > Project: HBase > Issue Type: Improvement > Components: Performance >Affects Versions: 1.0.3 >Reporter: Walter Koetke >Assignee: Walter Koetke > Fix For: 1.0.4 > > Attachments: BloomFilterCheckOneByOne.tiff, > HBASE-15480-branch-1.0.patch > > > It is currently inefficient to do lots of bloom filter checks. Each check has > overhead like going to the block cache to retrieve the block and recording > metrics. It would be good to have one bloom filter check api that does a > bunch of checks without so much block retrieval and metrics updates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15480) Bloom Filter check needs to be more efficient for array
[ https://issues.apache.org/jira/browse/HBASE-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Koetke updated HBASE-15480: -- Attachment: BloomFilterCheckOneByOne.tiff illustration of poor performance before change > Bloom Filter check needs to be more efficient for array > --- > > Key: HBASE-15480 > URL: https://issues.apache.org/jira/browse/HBASE-15480 > Project: HBase > Issue Type: Improvement > Components: Performance >Affects Versions: 1.0.3 >Reporter: Walter Koetke >Assignee: Walter Koetke > Fix For: 1.0.4 > > Attachments: BloomFilterCheckOneByOne.tiff, > HBASE-15480-branch-1.0.patch > > > It is currently inefficient to do lots of bloom filter checks. Each check has > overhead like going to the block cache to retrieve the block and recording > metrics. It would be good to have one bloom filter check api that does a > bunch of checks without so much block retrieval and metrics updates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15480) Bloom Filter check needs to be more efficient for array
[ https://issues.apache.org/jira/browse/HBASE-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Koetke updated HBASE-15480: -- Status: In Progress (was: Patch Available) > Bloom Filter check needs to be more efficient for array > --- > > Key: HBASE-15480 > URL: https://issues.apache.org/jira/browse/HBASE-15480 > Project: HBase > Issue Type: Improvement > Components: Performance >Affects Versions: 1.0.3 >Reporter: Walter Koetke >Assignee: Walter Koetke > Fix For: 1.0.4 > > Attachments: HBASE-15480-branch-1.0.patch > > > It is currently inefficient to do lots of bloom filter checks. Each check has > overhead like going to the block cache to retrieve the block and recording > metrics. It would be good to have one bloom filter check api that does a > bunch of checks without so much block retrieval and metrics updates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15556) need extensible ConsistencyControl interface
[ https://issues.apache.org/jira/browse/HBASE-15556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Koetke updated HBASE-15556: -- Status: In Progress (was: Patch Available) > need extensible ConsistencyControl interface > > > Key: HBASE-15556 > URL: https://issues.apache.org/jira/browse/HBASE-15556 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.0.3 >Reporter: Walter Koetke >Assignee: Walter Koetke > Fix For: 1.0.4 > > Attachments: HBASE-15556-branch-1.0.patch > > > The class MultiVersionConsistencyControl should be abstracted into an > interface ConsistencyControl so it can be extended by a configured custom > implementation class, with MultiVersionConsistencyControl as the default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15556) need extensible ConsistencyControl interface
[ https://issues.apache.org/jira/browse/HBASE-15556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15216554#comment-15216554 ] Walter Koetke commented on HBASE-15556: --- You are correct, Sean. The HBaseBlockingWaitStrategy is another optimization we came up with but it's not directly related to this Jira, so I will resubmit with this piece removed and with the other feedback incorporated (missing javadoc, annotations, etc.) > need extensible ConsistencyControl interface > > > Key: HBASE-15556 > URL: https://issues.apache.org/jira/browse/HBASE-15556 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.0.3 >Reporter: Walter Koetke >Assignee: Walter Koetke > Fix For: 1.0.4 > > Attachments: HBASE-15556-branch-1.0.patch > > > The class MultiVersionConsistencyControl should be abstracted into an > interface ConsistencyControl so it can be extended by a configured custom > implementation class, with MultiVersionConsistencyControl as the default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15480) Bloom Filter check needs to be more efficient for array
[ https://issues.apache.org/jira/browse/HBASE-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Koetke updated HBASE-15480: -- Attachment: HBASE-15480-branch-1.0.patch Attaching patch file for branch-1.0 > Bloom Filter check needs to be more efficient for array > --- > > Key: HBASE-15480 > URL: https://issues.apache.org/jira/browse/HBASE-15480 > Project: HBase > Issue Type: Improvement > Components: Performance >Affects Versions: 1.0.3 >Reporter: Walter Koetke >Assignee: Walter Koetke > Fix For: 1.0.4 > > Attachments: HBASE-15480-branch-1.0.patch > > > It is currently inefficient to do lots of bloom filter checks. Each check has > overhead like going to the block cache to retrieve the block and recording > metrics. It would be good to have one bloom filter check api that does a > bunch of checks without so much block retrieval and metrics updates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15480) Bloom Filter check needs to be more efficient for array
[ https://issues.apache.org/jira/browse/HBASE-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Koetke updated HBASE-15480: -- Fix Version/s: (was: 1.0.3) 1.0.4 Status: Patch Available (was: In Progress) > Bloom Filter check needs to be more efficient for array > --- > > Key: HBASE-15480 > URL: https://issues.apache.org/jira/browse/HBASE-15480 > Project: HBase > Issue Type: Improvement > Components: Performance >Affects Versions: 1.0.3 >Reporter: Walter Koetke >Assignee: Walter Koetke > Fix For: 1.0.4 > > Attachments: HBASE-15480-branch-1.0.patch > > > It is currently inefficient to do lots of bloom filter checks. Each check has > overhead like going to the block cache to retrieve the block and recording > metrics. It would be good to have one bloom filter check api that does a > bunch of checks without so much block retrieval and metrics updates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (HBASE-15480) Bloom Filter check needs to be more efficient for array
[ https://issues.apache.org/jira/browse/HBASE-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-15480 started by Walter Koetke. - > Bloom Filter check needs to be more efficient for array > --- > > Key: HBASE-15480 > URL: https://issues.apache.org/jira/browse/HBASE-15480 > Project: HBase > Issue Type: Improvement > Components: Performance >Affects Versions: 1.0.3 >Reporter: Walter Koetke >Assignee: Walter Koetke > Fix For: 1.0.3 > > > It is currently inefficient to do lots of bloom filter checks. Each check has > overhead like going to the block cache to retrieve the block and recording > metrics. It would be good to have one bloom filter check api that does a > bunch of checks without so much block retrieval and metrics updates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-15556) need extensible ConsistencyControl interface
[ https://issues.apache.org/jira/browse/HBASE-15556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Koetke reassigned HBASE-15556: - Assignee: Walter Koetke > need extensible ConsistencyControl interface > > > Key: HBASE-15556 > URL: https://issues.apache.org/jira/browse/HBASE-15556 > Project: HBase > Issue Type: Bug >Affects Versions: 1.0.3 >Reporter: Walter Koetke >Assignee: Walter Koetke > Fix For: 1.0.4 > > Attachments: HBASE-15556-branch-1.0.patch > > > The class MultiVersionConsistencyControl should be abstracted into an > interface ConsistencyControl so it can be extended by a configured custom > implementation class, with MultiVersionConsistencyControl as the default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15556) need extensible ConsistencyControl interface
[ https://issues.apache.org/jira/browse/HBASE-15556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15216073#comment-15216073 ] Walter Koetke commented on HBASE-15556: --- Patch file submitted is for branch-1.0. It can be forward ported also with the previously mentioned class name change. > need extensible ConsistencyControl interface > > > Key: HBASE-15556 > URL: https://issues.apache.org/jira/browse/HBASE-15556 > Project: HBase > Issue Type: Bug >Affects Versions: 1.0.3 >Reporter: Walter Koetke > Fix For: 1.0.4 > > Attachments: HBASE-15556-branch-1.0.patch > > > The class MultiVersionConsistencyControl should be abstracted into an > interface ConsistencyControl so it can be extended by a configured custom > implementation class, with MultiVersionConsistencyControl as the default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15556) need extensible ConsistencyControl interface
[ https://issues.apache.org/jira/browse/HBASE-15556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Koetke updated HBASE-15556: -- Attachment: HBASE-15556-branch-1.0.patch > need extensible ConsistencyControl interface > > > Key: HBASE-15556 > URL: https://issues.apache.org/jira/browse/HBASE-15556 > Project: HBase > Issue Type: Bug >Affects Versions: 1.0.3 >Reporter: Walter Koetke > Fix For: 1.0.4 > > Attachments: HBASE-15556-branch-1.0.patch > > > The class MultiVersionConsistencyControl should be abstracted into an > interface ConsistencyControl so it can be extended by a configured custom > implementation class, with MultiVersionConsistencyControl as the default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15556) need extensible ConsistencyControl interface
[ https://issues.apache.org/jira/browse/HBASE-15556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Koetke updated HBASE-15556: -- Status: Patch Available (was: Open) > need extensible ConsistencyControl interface > > > Key: HBASE-15556 > URL: https://issues.apache.org/jira/browse/HBASE-15556 > Project: HBase > Issue Type: Bug >Affects Versions: 1.0.3 >Reporter: Walter Koetke > Fix For: 1.0.4 > > Attachments: HBASE-15556-branch-1.0.patch > > > The class MultiVersionConsistencyControl should be abstracted into an > interface ConsistencyControl so it can be extended by a configured custom > implementation class, with MultiVersionConsistencyControl as the default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15556) need extensible ConsistencyControl interface
[ https://issues.apache.org/jira/browse/HBASE-15556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15216059#comment-15216059 ] Walter Koetke commented on HBASE-15556: --- In order to be able to add small custom extensions to MultiVersionConsistencyControl (which is called MultiVersionConcurrencyControl in versions later than 1.0), we needed to abstract this class into an interface ConsistencyControl and implement it with custom class. The custom class (which for us was called SIMultiVersionConsistencyControl) will not be included in the submitted patch, just the abstraction of the standard class into a pluggable interface. > need extensible ConsistencyControl interface > > > Key: HBASE-15556 > URL: https://issues.apache.org/jira/browse/HBASE-15556 > Project: HBase > Issue Type: Bug >Affects Versions: 1.0.3 >Reporter: Walter Koetke > Fix For: 1.0.4 > > > The class MultiVersionConsistencyControl should be abstracted into an > interface ConsistencyControl so it can be extended by a configured custom > implementation class, with MultiVersionConsistencyControl as the default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15556) need extensible ConsistencyControl interface
Walter Koetke created HBASE-15556: - Summary: need extensible ConsistencyControl interface Key: HBASE-15556 URL: https://issues.apache.org/jira/browse/HBASE-15556 Project: HBase Issue Type: Bug Affects Versions: 1.0.3 Reporter: Walter Koetke Fix For: 1.0.4 The class MultiVersionConsistencyControl should be abstracted into an interface ConsistencyControl so it can be extended by a configured custom implementation class, with MultiVersionConsistencyControl as the default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15546) improvements to consistency
[ https://issues.apache.org/jira/browse/HBASE-15546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15214807#comment-15214807 ] Walter Koetke commented on HBASE-15546: --- Fair enough guys. Sorry. Needed a quick placeholder jira for an extension we have internally at Splice, but I'll create a new Jira that describes it right off the bat. > improvements to consistency > --- > > Key: HBASE-15546 > URL: https://issues.apache.org/jira/browse/HBASE-15546 > Project: HBase > Issue Type: Bug >Affects Versions: 1.0.3 >Reporter: Walter Koetke > > Improvements needed to consistency logic. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15546) improvements to consistency
Walter Koetke created HBASE-15546: - Summary: improvements to consistency Key: HBASE-15546 URL: https://issues.apache.org/jira/browse/HBASE-15546 Project: HBase Issue Type: Bug Affects Versions: 1.0.3 Reporter: Walter Koetke Assignee: Walter Koetke Fix For: 2.0.0, 1.0.4, 1.4.0 Improvements needed to consistency logic. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12148) Remove TimeRangeTracker as point of contention when many threads writing a Store
[ https://issues.apache.org/jira/browse/HBASE-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15209378#comment-15209378 ] Walter Koetke commented on HBASE-12148: --- Lars, can you be more specific about what the problem is with having just this one implementation, as opposed to one that's synchronized and one that is not? Unless there is a mistake in this patched TimeRangeTracker as implemented, the intent is that it would be thread safe and support both usages, with the AtomicReference, the compareAndSet, etc. > Remove TimeRangeTracker as point of contention when many threads writing a > Store > > > Key: HBASE-12148 > URL: https://issues.apache.org/jira/browse/HBASE-12148 > Project: HBase > Issue Type: Sub-task > Components: Performance >Affects Versions: 2.0.0, 0.99.1 >Reporter: stack >Assignee: Walter Koetke > Fix For: 2.0.0, 1.3.0, 0.98.19 > > Attachments: > 0001-In-AtomicUtils-change-updateMin-and-updateMax-to-ret.patch, > 12148.addendum.txt, 12148.txt, 12148.txt, 12148v2.txt, 12148v2.txt, > HBASE-12148-V3.patch, HBASE-12148.txt, HBASE-12148V2.txt, Screen Shot > 2014-10-01 at 3.39.46 PM.png, Screen Shot 2014-10-01 at 3.41.07 PM.png, > TimeRangeTracker.tiff > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-12148) Remove TimeRangeTracker as point of contention when many threads writing a Store
[ https://issues.apache.org/jira/browse/HBASE-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Koetke reassigned HBASE-12148: - Assignee: Walter Koetke (was: John Leach) > Remove TimeRangeTracker as point of contention when many threads writing a > Store > > > Key: HBASE-12148 > URL: https://issues.apache.org/jira/browse/HBASE-12148 > Project: HBase > Issue Type: Sub-task > Components: Performance >Affects Versions: 2.0.0, 0.99.1 >Reporter: stack >Assignee: Walter Koetke > Fix For: 2.0.0, 1.3.0, 0.98.18 > > Attachments: > 0001-In-AtomicUtils-change-updateMin-and-updateMax-to-ret.patch, > 12148.addendum.txt, 12148.txt, 12148.txt, 12148v2.txt, 12148v2.txt, > HBASE-12148.txt, HBASE-12148V2.txt, Screen Shot 2014-10-01 at 3.39.46 PM.png, > Screen Shot 2014-10-01 at 3.41.07 PM.png > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15480) Bloom Filter check needs to be more efficient for array
[ https://issues.apache.org/jira/browse/HBASE-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Koetke updated HBASE-15480: -- Summary: Bloom Filter check needs to be more efficient for array (was: Bloom Filter array access needs to be more efficient) > Bloom Filter check needs to be more efficient for array > --- > > Key: HBASE-15480 > URL: https://issues.apache.org/jira/browse/HBASE-15480 > Project: HBase > Issue Type: Improvement > Components: Performance >Affects Versions: 1.0.3 >Reporter: Walter Koetke >Assignee: Walter Koetke > Fix For: 1.0.3 > > > It is currently inefficient to do lots of bloom filter checks. Each check has > overhead like going to the block cache to retrieve the block and recording > metrics. It would be good to have one bloom filter check api that does a > bunch of checks without so much block retrieval and metrics updates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12148) Remove TimeRangeTracker as point of contention when many threads writing a Store
[ https://issues.apache.org/jira/browse/HBASE-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15200616#comment-15200616 ] Walter Koetke commented on HBASE-12148: --- Attaching another version (HBASE-12148-V3.patch) of the patch John Leach had been working on, which removes synchronization from TimeRangeTracker. Our own internal testing indicated there was lots of contention. > Remove TimeRangeTracker as point of contention when many threads writing a > Store > > > Key: HBASE-12148 > URL: https://issues.apache.org/jira/browse/HBASE-12148 > Project: HBase > Issue Type: Sub-task > Components: Performance >Affects Versions: 2.0.0, 0.99.1 >Reporter: stack >Assignee: Walter Koetke > Fix For: 2.0.0, 1.3.0, 0.98.18 > > Attachments: > 0001-In-AtomicUtils-change-updateMin-and-updateMax-to-ret.patch, > 12148.addendum.txt, 12148.txt, 12148.txt, 12148v2.txt, 12148v2.txt, > HBASE-12148.txt, HBASE-12148V2.txt, Screen Shot 2014-10-01 at 3.39.46 PM.png, > Screen Shot 2014-10-01 at 3.41.07 PM.png > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15480) Bloom Filter array access needs to be more efficient
[ https://issues.apache.org/jira/browse/HBASE-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15200642#comment-15200642 ] Walter Koetke commented on HBASE-15480: --- Will be attaching a patch soon for a change worked on by John Leach. Our internal tests indicated this was a bottleneck. > Bloom Filter array access needs to be more efficient > > > Key: HBASE-15480 > URL: https://issues.apache.org/jira/browse/HBASE-15480 > Project: HBase > Issue Type: Improvement > Components: Performance >Affects Versions: 1.0.3 >Reporter: Walter Koetke >Assignee: Walter Koetke > Fix For: 1.0.3 > > > The bloom filter implementation for an array of items is inefficient. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15480) Bloom Filter array access needs to be more efficient
Walter Koetke created HBASE-15480: - Summary: Bloom Filter array access needs to be more efficient Key: HBASE-15480 URL: https://issues.apache.org/jira/browse/HBASE-15480 Project: HBase Issue Type: Improvement Components: Performance Affects Versions: 1.0.3 Reporter: Walter Koetke Assignee: Walter Koetke Fix For: 1.0.3 The bloom filter implementation for an array of items is inefficient. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12148) Remove TimeRangeTracker as point of contention when many threads writing a Store
[ https://issues.apache.org/jira/browse/HBASE-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Koetke updated HBASE-12148: -- Hadoop Flags: (was: Reviewed) Status: Patch Available (was: Open) Patch file is for 'branch-1.3' > Remove TimeRangeTracker as point of contention when many threads writing a > Store > > > Key: HBASE-12148 > URL: https://issues.apache.org/jira/browse/HBASE-12148 > Project: HBase > Issue Type: Sub-task > Components: Performance >Affects Versions: 0.99.1, 2.0.0 >Reporter: stack >Assignee: Walter Koetke > Fix For: 2.0.0, 1.3.0, 0.98.19 > > Attachments: > 0001-In-AtomicUtils-change-updateMin-and-updateMax-to-ret.patch, > 12148.addendum.txt, 12148.txt, 12148.txt, 12148v2.txt, 12148v2.txt, > HBASE-12148-V3.patch, HBASE-12148.txt, HBASE-12148V2.txt, Screen Shot > 2014-10-01 at 3.39.46 PM.png, Screen Shot 2014-10-01 at 3.41.07 PM.png > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15480) Bloom Filter array access needs to be more efficient
[ https://issues.apache.org/jira/browse/HBASE-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Koetke updated HBASE-15480: -- Description: It is currently inefficient to do lots of bloom filter checks. Each check has overhead like going to the block cache to retrieve the block and recording metrics. It would be good to have one bloom filter check api that does a bunch of checks without so much block retrieval and metrics updates. (was: The bloom filter implementation for an array of items is inefficient.) > Bloom Filter array access needs to be more efficient > > > Key: HBASE-15480 > URL: https://issues.apache.org/jira/browse/HBASE-15480 > Project: HBase > Issue Type: Improvement > Components: Performance >Affects Versions: 1.0.3 >Reporter: Walter Koetke >Assignee: Walter Koetke > Fix For: 1.0.3 > > > It is currently inefficient to do lots of bloom filter checks. Each check has > overhead like going to the block cache to retrieve the block and recording > metrics. It would be good to have one bloom filter check api that does a > bunch of checks without so much block retrieval and metrics updates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12148) Remove TimeRangeTracker as point of contention when many threads writing a Store
[ https://issues.apache.org/jira/browse/HBASE-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Koetke updated HBASE-12148: -- Attachment: HBASE-12148-V3.patch > Remove TimeRangeTracker as point of contention when many threads writing a > Store > > > Key: HBASE-12148 > URL: https://issues.apache.org/jira/browse/HBASE-12148 > Project: HBase > Issue Type: Sub-task > Components: Performance >Affects Versions: 2.0.0, 0.99.1 >Reporter: stack >Assignee: Walter Koetke > Fix For: 2.0.0, 1.3.0, 0.98.18 > > Attachments: > 0001-In-AtomicUtils-change-updateMin-and-updateMax-to-ret.patch, > 12148.addendum.txt, 12148.txt, 12148.txt, 12148v2.txt, 12148v2.txt, > HBASE-12148-V3.patch, HBASE-12148.txt, HBASE-12148V2.txt, Screen Shot > 2014-10-01 at 3.39.46 PM.png, Screen Shot 2014-10-01 at 3.41.07 PM.png > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
