[jira] [Updated] (HBASE-4812) GC benchmarking and analysis tools
[ https://issues.apache.org/jira/browse/HBASE-4812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Gray updated HBASE-4812: - Attachment: GCBench.patch banalyzeGC.py/b is a python script that parses raw GC logs (lots of regex) and has some mechanisms for aggregating and summarizing. It's not done but most of the nastiest bits are completed or near complete. bBlockCacheBench.java/b is the isolated block cache benchmarker. In addition to threading and all that, it uses RandomGenerator to generate various distributions of operations (needs improvements but baseline is there). GC benchmarking and analysis tools -- Key: HBASE-4812 URL: https://issues.apache.org/jira/browse/HBASE-4812 Project: HBase Issue Type: Task Components: io Reporter: Jonathan Gray Assignee: Jonathan Gray Priority: Minor Attachments: GCBench.patch One of the challenges of debugging GC issues is that it can take a long time to reproduce GC pauses. Isolating the LRU block cache and benchmarking it independently can more easily cause measurable and reproducible GC issues. In addition, making sense of the noisy GC logs is difficult. This may not be for commit, but I wanted to share some code and scripts I've written to tackle these problems. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4696) HRegionThriftServer' might have to indefinitely do redirtects
[ https://issues.apache.org/jira/browse/HBASE-4696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Gray updated HBASE-4696: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to trunk. Thanks. HRegionThriftServer' might have to indefinitely do redirtects - Key: HBASE-4696 URL: https://issues.apache.org/jira/browse/HBASE-4696 Project: HBase Issue Type: Improvement Affects Versions: 0.94.0 Reporter: Prakash Khemani Assignee: Jonathan Gray Fix For: 0.94.0 Attachments: HBASE-4696-v1.patch HRegionThriftServer.getRowWithColumnsTs() redirects the request to the correct region server if it has landed on the wrong region-server. With this approach the smart-client will never get a NotServingRegionException and it will never be able to invalidate its cache. It will indefinitely send the request to the wrong region server and the wrong region server will always be redirecting it. Either redirects should be turned off and the client should react to NotServingRegionExceptions. Or somehow a flag should be set in the response telling the client to refresh its cache. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4696) HRegionThriftServer' might have to indefinitely do redirtects
[
https://issues.apache.org/jira/browse/HBASE-4696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Gray updated HBASE-4696:
-
Attachment: HBASE-4696-v1.patch
Adds new redirect configuration option to HRegionThriftServer:
{{hbase.regionserver.thrift.redirect}}
Default is false.
If this is false, then if the current regionserver does not have the specified
region, an exception is thrown rather than auto-re-directing.
This also has another fix where it was using the encoded region name instead of
the full binary region name.
HRegionThriftServer' might have to indefinitely do redirtects
-
Key: HBASE-4696
URL: https://issues.apache.org/jira/browse/HBASE-4696
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.0
Reporter: Prakash Khemani
Assignee: Jonathan Gray
Fix For: 0.94.0
Attachments: HBASE-4696-v1.patch
HRegionThriftServer.getRowWithColumnsTs() redirects the request to the
correct region server if it has landed on the wrong region-server. With this
approach the smart-client will never get a NotServingRegionException and it
will never be able to invalidate its cache. It will indefinitely send the
request to the wrong region server and the wrong region server will always be
redirecting it.
Either redirects should be turned off and the client should react to
NotServingRegionExceptions.
Or somehow a flag should be set in the response telling the client to refresh
its cache.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4696) HRegionThriftServer' might have to indefinitely do redirtects
[ https://issues.apache.org/jira/browse/HBASE-4696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Gray updated HBASE-4696: - Fix Version/s: 0.94.0 Affects Version/s: 0.94.0 Release Note: RS embedded thrift server can have redirecting turned on/off with hbase.regionserver.thrift.redirect (default: false) Status: Patch Available (was: Open) HRegionThriftServer' might have to indefinitely do redirtects - Key: HBASE-4696 URL: https://issues.apache.org/jira/browse/HBASE-4696 Project: HBase Issue Type: Bug Affects Versions: 0.94.0 Reporter: Prakash Khemani Assignee: Jonathan Gray Fix For: 0.94.0 Attachments: HBASE-4696-v1.patch HRegionThriftServer.getRowWithColumnsTs() redirects the request to the correct region server if it has landed on the wrong region-server. With this approach the smart-client will never get a NotServingRegionException and it will never be able to invalidate its cache. It will indefinitely send the request to the wrong region server and the wrong region server will always be redirecting it. Either redirects should be turned off and the client should react to NotServingRegionExceptions. Or somehow a flag should be set in the response telling the client to refresh its cache. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4696) HRegionThriftServer' might have to indefinitely do redirtects
[ https://issues.apache.org/jira/browse/HBASE-4696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Gray updated HBASE-4696: - Issue Type: Improvement (was: Bug) Marking as improvement since this is really further improvement to a new feature rather than a real bug fix. HRegionThriftServer' might have to indefinitely do redirtects - Key: HBASE-4696 URL: https://issues.apache.org/jira/browse/HBASE-4696 Project: HBase Issue Type: Improvement Affects Versions: 0.94.0 Reporter: Prakash Khemani Assignee: Jonathan Gray Fix For: 0.94.0 Attachments: HBASE-4696-v1.patch HRegionThriftServer.getRowWithColumnsTs() redirects the request to the correct region server if it has landed on the wrong region-server. With this approach the smart-client will never get a NotServingRegionException and it will never be able to invalidate its cache. It will indefinitely send the request to the wrong region server and the wrong region server will always be redirecting it. Either redirects should be turned off and the client should react to NotServingRegionExceptions. Or somehow a flag should be set in the response telling the client to refresh its cache. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4528) The put operation can release the rowlock before sync-ing the Hlog
[ https://issues.apache.org/jira/browse/HBASE-4528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Gray updated HBASE-4528: - Resolution: Fixed Release Note: Adds early-lock-release ability so we can do the WAL sync outside of the row lock. Extends MemStore/RWCC to support rollback. Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to trunk. Thanks Dhruba for the awesome work, Thanks Ted and others for all the good reviews. The put operation can release the rowlock before sync-ing the Hlog -- Key: HBASE-4528 URL: https://issues.apache.org/jira/browse/HBASE-4528 Project: HBase Issue Type: Improvement Components: regionserver Reporter: dhruba borthakur Assignee: dhruba borthakur Fix For: 0.94.0 Attachments: 4528-trunk-v9.txt, 4528-trunk.txt, HBASE-4528-Trunk-FINAL.patch, appendNoSync5.txt, appendNoSyncPut1.txt, appendNoSyncPut2.txt, appendNoSyncPut3.txt, appendNoSyncPut4.txt, appendNoSyncPut5.txt, appendNoSyncPut6.txt, appendNoSyncPut7.txt, appendNoSyncPut8.txt This allows for better throughput when there are hot rows. A single row update improves from 100 puts/sec/server to 5000 puts/sec/server. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4528) The put operation can release the rowlock before sync-ing the Hlog
[ https://issues.apache.org/jira/browse/HBASE-4528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Gray updated HBASE-4528: - Attachment: HBASE-4528-Trunk-FINAL.patch Patch being committed. This is dhruba's appendNoSyncPut8.txt rebased on trunk. One small change was needed in TestParallelPut in order to compile. The put operation can release the rowlock before sync-ing the Hlog -- Key: HBASE-4528 URL: https://issues.apache.org/jira/browse/HBASE-4528 Project: HBase Issue Type: Improvement Components: regionserver Reporter: dhruba borthakur Assignee: dhruba borthakur Fix For: 0.94.0 Attachments: HBASE-4528-Trunk-FINAL.patch, appendNoSync5.txt, appendNoSyncPut1.txt, appendNoSyncPut2.txt, appendNoSyncPut3.txt, appendNoSyncPut4.txt, appendNoSyncPut5.txt, appendNoSyncPut6.txt, appendNoSyncPut7.txt, appendNoSyncPut8.txt This allows for better throughput when there are hot rows. A single row update improves from 100 puts/sec/server to 5000 puts/sec/server. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4588) The floating point arithmetic to validate memory allocation configurations need to be done as integers
[ https://issues.apache.org/jira/browse/HBASE-4588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Gray updated HBASE-4588: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to 92 branch and trunk. Thanks Dhruba. The floating point arithmetic to validate memory allocation configurations need to be done as integers -- Key: HBASE-4588 URL: https://issues.apache.org/jira/browse/HBASE-4588 Project: HBase Issue Type: Bug Affects Versions: 0.92.0, 0.94.0 Reporter: Jonathan Gray Assignee: dhruba borthakur Priority: Minor Fix For: 0.92.0 Attachments: configVerify1.txt, configVerify2.txt The floating point arithmetic to validate memory allocation configurations need to be done as integers. On our cluster, we had block cache = 0.6 and memstore = 0.2. It was saying this was 0.8 when it is actually equal. Minor bug but annoying nonetheless. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4641) Block cache can be mistakenly instantiated on Master
[ https://issues.apache.org/jira/browse/HBASE-4641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Gray updated HBASE-4641: - Attachment: HBASE-4641-v1.patch In the HMaster constructor, makes a local copy of the passed-in Configuration and then sets the cache size to 0. This seems like the wrong way to do this. It seems like the only way this happens now is on create table (because HRegion instantiation happens in master at this time). I could set/reset the conf in that place instead? Block cache can be mistakenly instantiated on Master Key: HBASE-4641 URL: https://issues.apache.org/jira/browse/HBASE-4641 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.92.0, 0.94.0 Reporter: Jonathan Gray Assignee: Jonathan Gray Priority: Critical Fix For: 0.92.0 Attachments: HBASE-4641-v1.patch After changes in the block cache instantiation over in HBASE-4422, it looks like the HMaster can now end up with a block cache instantiated. Not a huge deal but prevents the process from shutting down properly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4641) Block cache can be mistakenly instantiated on Master
[ https://issues.apache.org/jira/browse/HBASE-4641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Gray updated HBASE-4641: - Attachment: HBASE-4641-v2.patch This version does the same thing but now does it in the CreateTableHandler which should not be disruptive like the other change. Block cache can be mistakenly instantiated on Master Key: HBASE-4641 URL: https://issues.apache.org/jira/browse/HBASE-4641 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.92.0, 0.94.0 Reporter: Jonathan Gray Assignee: Jonathan Gray Priority: Critical Fix For: 0.92.0 Attachments: HBASE-4641-v1.patch, HBASE-4641-v2.patch After changes in the block cache instantiation over in HBASE-4422, it looks like the HMaster can now end up with a block cache instantiated. Not a huge deal but prevents the process from shutting down properly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4460) Support running an embedded ThriftServer within a RegionServer
[ https://issues.apache.org/jira/browse/HBASE-4460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Gray updated HBASE-4460: - Fix Version/s: 0.94.0 New feature, only going to trunk. Support running an embedded ThriftServer within a RegionServer -- Key: HBASE-4460 URL: https://issues.apache.org/jira/browse/HBASE-4460 Project: HBase Issue Type: New Feature Components: regionserver, thrift Reporter: Jonathan Gray Assignee: Jonathan Gray Fix For: 0.94.0 Attachments: HBASE-4460-v1.patch Rather than a separate process, it can be advantageous in some situations for each RegionServer to embed their own ThriftServer. This allows each embedded ThriftServer to short-circuit any queries that should be executed on the local RS and skip the extra hop. This then enables the building of fat Thrift clients that cache region locations and avoid extra hops all together. This JIRA is just about the embedded ThriftServer. Will open others for the rest. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4609) ThriftServer.getRegionInfo() is expecting old ServerName format, need to use new Addressing class instead
[ https://issues.apache.org/jira/browse/HBASE-4609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Gray updated HBASE-4609: - Attachment: HBASE-4609-v1.patch As advertised. Against trunk. ThriftServer.getRegionInfo() is expecting old ServerName format, need to use new Addressing class instead - Key: HBASE-4609 URL: https://issues.apache.org/jira/browse/HBASE-4609 Project: HBase Issue Type: Bug Components: thrift Affects Versions: 0.92.0, 0.94.0 Reporter: Jonathan Gray Assignee: Jonathan Gray Priority: Minor Fix For: 0.92.0 Attachments: HBASE-4609-v1.patch ThriftServer.getRegionInfo() is expecting the old ServerName that doesn't include start code. Need to fix. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4609) ThriftServer.getRegionInfo() is expecting old ServerName format, need to use new Addressing class instead
[ https://issues.apache.org/jira/browse/HBASE-4609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Gray updated HBASE-4609: - Status: Patch Available (was: Open) ThriftServer.getRegionInfo() is expecting old ServerName format, need to use new Addressing class instead - Key: HBASE-4609 URL: https://issues.apache.org/jira/browse/HBASE-4609 Project: HBase Issue Type: Bug Components: thrift Affects Versions: 0.92.0, 0.94.0 Reporter: Jonathan Gray Assignee: Jonathan Gray Priority: Minor Fix For: 0.92.0 Attachments: HBASE-4609-v1.patch ThriftServer.getRegionInfo() is expecting the old ServerName that doesn't include start code. Need to fix. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4589) CacheOnWrite broken in some cases because it can conflict with evictOnClose
[ https://issues.apache.org/jira/browse/HBASE-4589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Gray updated HBASE-4589: - Status: Patch Available (was: Open) CacheOnWrite broken in some cases because it can conflict with evictOnClose --- Key: HBASE-4589 URL: https://issues.apache.org/jira/browse/HBASE-4589 Project: HBase Issue Type: Bug Components: io Affects Versions: 0.92.0, 0.94.0 Reporter: Jonathan Gray Assignee: Jonathan Gray Priority: Critical Fix For: 0.92.0 Attachments: HBASE-4589-v1.patch Commit of HBASE-4078 added some extra StoreFile verification which just did an open of a StoreFile reader and then closes it, ensuring there's no exception. If evict-on-close is on, which it is by default, this causes all blocks of a file to be evicted even though it's still open. We need to add the boolean into the close call in the way we have booleans for cacheBlocks at some point since we need to make localized decisions in some cases. In lots of places, we can always rely on cacheConf.shouldEvictOnClose() so shouldn't be too burdensome. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4589) CacheOnWrite broken in some cases because it can conflict with evictOnClose
[ https://issues.apache.org/jira/browse/HBASE-4589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Gray updated HBASE-4589: - Attachment: HBASE-4589-v1.patch As advertised. Tests passing. CacheOnWrite broken in some cases because it can conflict with evictOnClose --- Key: HBASE-4589 URL: https://issues.apache.org/jira/browse/HBASE-4589 Project: HBase Issue Type: Bug Components: io Affects Versions: 0.92.0, 0.94.0 Reporter: Jonathan Gray Assignee: Jonathan Gray Priority: Critical Fix For: 0.92.0 Attachments: HBASE-4589-v1.patch Commit of HBASE-4078 added some extra StoreFile verification which just did an open of a StoreFile reader and then closes it, ensuring there's no exception. If evict-on-close is on, which it is by default, this causes all blocks of a file to be evicted even though it's still open. We need to add the boolean into the close call in the way we have booleans for cacheBlocks at some point since we need to make localized decisions in some cases. In lots of places, we can always rely on cacheConf.shouldEvictOnClose() so shouldn't be too burdensome. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3417) CacheOnWrite is using the temporary output path for block names, need to use a more consistent block naming scheme
[ https://issues.apache.org/jira/browse/HBASE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Gray updated HBASE-3417: - Attachment: HBASE-3417-redux-v1.patch Adds a new unit test in TestFromClientSide for testing cache on write and evict on close by looking inside the block cache. This test fails on 92 and trunk without the rest of the changes in this patch (CacheOnWrite is currently completely broken with hfile v2). CacheOnWrite is using the temporary output path for block names, need to use a more consistent block naming scheme -- Key: HBASE-3417 URL: https://issues.apache.org/jira/browse/HBASE-3417 Project: HBase Issue Type: Bug Components: io, regionserver Affects Versions: 0.92.0 Reporter: Jonathan Gray Assignee: Jonathan Gray Priority: Critical Fix For: 0.92.0 Attachments: HBASE-3417-redux-v1.patch, HBASE-3417-v1.patch, HBASE-3417-v2.patch, HBASE-3417-v5.patch Currently the block names used in the block cache are built using the filesystem path. However, for cache on write, the path is a temporary output file. The original COW patch actually made some modifications to block naming stuff to make it more consistent but did not do enough. Should add a separate method somewhere for generating block names using some more easily mocked scheme (rather than just raw path as we generate a random unique file name twice, once for tmp and then again when moved into place). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3417) CacheOnWrite is using the temporary output path for block names, need to use a more consistent block naming scheme
[ https://issues.apache.org/jira/browse/HBASE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Gray updated HBASE-3417: - Release Note: StoreFile naming changes from random ascii longs to [0-9a-f] uuids. The same name is used in the tmp and perm location. This name is what gets used for the block name in the cache. This change is a backwards compatible slow migration, all new files will be named with the new scheme, old files will still be readable. However, once you have new files created in the new scheme, previous installations will fail. was:StoreFile naming changes from random ascii longs to [0-9a-f] uuids. The same name is used in the tmp and perm location. This name is what gets used for the block name in the cache. CacheOnWrite is using the temporary output path for block names, need to use a more consistent block naming scheme -- Key: HBASE-3417 URL: https://issues.apache.org/jira/browse/HBASE-3417 Project: HBase Issue Type: Bug Components: io, regionserver Affects Versions: 0.92.0 Reporter: Jonathan Gray Assignee: Jonathan Gray Priority: Critical Fix For: 0.92.0 Attachments: HBASE-3417-redux-v1.patch, HBASE-3417-v1.patch, HBASE-3417-v2.patch, HBASE-3417-v5.patch Currently the block names used in the block cache are built using the filesystem path. However, for cache on write, the path is a temporary output file. The original COW patch actually made some modifications to block naming stuff to make it more consistent but did not do enough. Should add a separate method somewhere for generating block names using some more easily mocked scheme (rather than just raw path as we generate a random unique file name twice, once for tmp and then again when moved into place). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4469) Avoid top row seek by looking up bloomfilter
[ https://issues.apache.org/jira/browse/HBASE-4469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Gray updated HBASE-4469: - Fix Version/s: 0.94.0 Committed to trunk. Avoid top row seek by looking up bloomfilter Key: HBASE-4469 URL: https://issues.apache.org/jira/browse/HBASE-4469 Project: HBase Issue Type: Improvement Reporter: Liyin Tang Assignee: Liyin Tang Fix For: 0.94.0 Attachments: HBASE-4469_1.patch The problem is that when seeking for the row/col in the hfile, we will go to top of the row in order to check for row delete marker (delete family). However, if the bloomfilter is enabled for the column family, then if a delete family operation is done on a row, the row is already being added to bloomfilter. We can take advantage of this factor to avoid seeking to the top of row. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4469) Avoid top row seek by looking up bloomfilter
[ https://issues.apache.org/jira/browse/HBASE-4469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Gray updated HBASE-4469: - Resolution: Fixed Status: Resolved (was: Patch Available) Got it, thanks Liyin! Nice work! Avoid top row seek by looking up bloomfilter Key: HBASE-4469 URL: https://issues.apache.org/jira/browse/HBASE-4469 Project: HBase Issue Type: Improvement Reporter: Liyin Tang Assignee: Liyin Tang Fix For: 0.94.0 Attachments: HBASE-4469_1.patch The problem is that when seeking for the row/col in the hfile, we will go to top of the row in order to check for row delete marker (delete family). However, if the bloomfilter is enabled for the column family, then if a delete family operation is done on a row, the row is already being added to bloomfilter. We can take advantage of this factor to avoid seeking to the top of row. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4582) Store.java cleanup (failing TestHeapSize and has warnings)
[ https://issues.apache.org/jira/browse/HBASE-4582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Gray updated HBASE-4582: - Attachment: HBASE-4582-v1-branch92.patch As described. TestHeapSize passing here as are the other tests. Store.java cleanup (failing TestHeapSize and has warnings) -- Key: HBASE-4582 URL: https://issues.apache.org/jira/browse/HBASE-4582 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.92.0, 0.94.0 Reporter: Jonathan Gray Assignee: Jonathan Gray Priority: Minor Fix For: 0.92.0 Attachments: HBASE-4582-v1-branch92.patch I broke TestHeapSize on 32-bit when I committed HBASE-4422. Fix it here, remove another boolean variable I forgot to delete, and fix a couple warnings while we are at it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4582) Store.java cleanup (failing TestHeapSize and has warnings)
[ https://issues.apache.org/jira/browse/HBASE-4582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Gray updated HBASE-4582: - Attachment: HBASE-4582-FINAL.patch Final patch, includes removal of one more stale variable and warning over in StoreFile from HBASE-4422. Store.java cleanup (failing TestHeapSize and has warnings) -- Key: HBASE-4582 URL: https://issues.apache.org/jira/browse/HBASE-4582 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.92.0, 0.94.0 Reporter: Jonathan Gray Assignee: Jonathan Gray Priority: Minor Fix For: 0.92.0 Attachments: HBASE-4582-FINAL.patch, HBASE-4582-v1-branch92.patch I broke TestHeapSize on 32-bit when I committed HBASE-4422. Fix it here, remove another boolean variable I forgot to delete, and fix a couple warnings while we are at it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4422) Move block cache parameters and references into single CacheConf class
[ https://issues.apache.org/jira/browse/HBASE-4422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Gray updated HBASE-4422: - Attachment: HBASE-4422-FINAL-branch92.patch Final patch for commit on 92 branch Move block cache parameters and references into single CacheConf class -- Key: HBASE-4422 URL: https://issues.apache.org/jira/browse/HBASE-4422 Project: HBase Issue Type: Improvement Components: io Reporter: Jonathan Gray Assignee: Jonathan Gray Fix For: 0.92.0 Attachments: CacheConfig92-v8.patch, HBASE-4422-FINAL-branch92.patch, HBASE-4422-FINAL-trunk.patch From StoreFile down to HFile, we currently use a boolean argument for each of the various block cache configuration parameters that exist. The number of parameters is going to continue to increase as we look at compressed cache, delta encoding, and more specific L1/L2 configuration. Every new config currently requires changing many constructors because it introduces a new boolean. We should move everything into a single class so that modifications are much less disruptive. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4422) Move block cache parameters and references into single CacheConf class
[ https://issues.apache.org/jira/browse/HBASE-4422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Gray updated HBASE-4422: - Attachment: HBASE-4422-FINAL-trunk.patch Final patch for commit to trunk Move block cache parameters and references into single CacheConf class -- Key: HBASE-4422 URL: https://issues.apache.org/jira/browse/HBASE-4422 Project: HBase Issue Type: Improvement Components: io Reporter: Jonathan Gray Assignee: Jonathan Gray Fix For: 0.92.0 Attachments: CacheConfig92-v8.patch, HBASE-4422-FINAL-branch92.patch, HBASE-4422-FINAL-trunk.patch From StoreFile down to HFile, we currently use a boolean argument for each of the various block cache configuration parameters that exist. The number of parameters is going to continue to increase as we look at compressed cache, delta encoding, and more specific L1/L2 configuration. Every new config currently requires changing many constructors because it introduces a new boolean. We should move everything into a single class so that modifications are much less disruptive. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4422) Move block cache parameters and references into single CacheConf class
[ https://issues.apache.org/jira/browse/HBASE-4422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Gray updated HBASE-4422: - Attachment: CacheConfig92-v8.patch Most recent patch from RB. All tests are passing. I will definitely be pulling this into our internal 92 branch and will be building some other stuff on top of it (compressed cache, etc). It will probably be easiest to integrate the DeltaEncoding stuff with this in place as well. For that reason, I'd be interested in getting it into 92 branch, it should be low risk. Move block cache parameters and references into single CacheConf class -- Key: HBASE-4422 URL: https://issues.apache.org/jira/browse/HBASE-4422 Project: HBase Issue Type: Improvement Components: io Reporter: Jonathan Gray Assignee: Jonathan Gray Fix For: 0.92.0 Attachments: CacheConfig92-v8.patch From StoreFile down to HFile, we currently use a boolean argument for each of the various block cache configuration parameters that exist. The number of parameters is going to continue to increase as we look at compressed cache, delta encoding, and more specific L1/L2 configuration. Every new config currently requires changing many constructors because it introduces a new boolean. We should move everything into a single class so that modifications are much less disruptive. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4488) Store could miss rows during flush
[ https://issues.apache.org/jira/browse/HBASE-4488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Gray updated HBASE-4488: - Resolution: Fixed Fix Version/s: (was: 0.94.0) Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to 92 branch and trunk, marking against 92. Thanks Lars! Store could miss rows during flush -- Key: HBASE-4488 URL: https://issues.apache.org/jira/browse/HBASE-4488 Project: HBase Issue Type: Sub-task Components: regionserver Affects Versions: 0.92.0, 0.94.0 Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.92.0 Attachments: 4488.txt While looking at HBASE-4344 I found that my change HBASE-4241 contains a critical mistake: The while(scanner.next(kvs)) loop is incorrect and might miss the last edits. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4219) Add Per-Column Family Metrics
[ https://issues.apache.org/jira/browse/HBASE-4219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Gray updated HBASE-4219: - Attachment: HBASE-4219-v4.patch This is the latest patch given to me by Nicolas and then rebased on tip of trunk. Going to dig in now on why this is not working with public hadoop. Add Per-Column Family Metrics - Key: HBASE-4219 URL: https://issues.apache.org/jira/browse/HBASE-4219 Project: HBase Issue Type: New Feature Affects Versions: 0.92.0 Reporter: Nicolas Spiegelberg Assignee: David Goode Fix For: 0.92.0 Attachments: 4219-v2.txt, 4219-v3.txt, HBASE-4219-v4.patch, HBASE-4219_percfmetrics_1.patch Right now, we have region server level statistics. However, the read/write flow varies a lot based on the column family involved. We should add dynamic, per column family metrics to JMX so we can track each column family individually. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
