[jira] [Created] (HBASE-5548) Add ability to get a table in the shell
Add ability to get a table in the shell --- Key: HBASE-5548 URL: https://issues.apache.org/jira/browse/HBASE-5548 Project: HBase Issue Type: Improvement Components: shell Reporter: Jesse Yates Fix For: 0.96.0 Currently, all the commands that operate on a table in the shell first have to take the table as name as input. There are two main considerations: * It is annoying to have to write the table name every time, when you should just be able to get a reference to a table * the current implementation is very wasteful - it creates a new HTable for each call (but reuses the connection since it uses the same configuration) We should be able to get a handle to a single HTable and then operate on that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5548) Add ability to get a table in the shell
[ https://issues.apache.org/jira/browse/HBASE-5548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates reassigned HBASE-5548: -- Assignee: Jesse Yates Add ability to get a table in the shell --- Key: HBASE-5548 URL: https://issues.apache.org/jira/browse/HBASE-5548 Project: HBase Issue Type: Improvement Components: shell Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0 Currently, all the commands that operate on a table in the shell first have to take the table as name as input. There are two main considerations: * It is annoying to have to write the table name every time, when you should just be able to get a reference to a table * the current implementation is very wasteful - it creates a new HTable for each call (but reuses the connection since it uses the same configuration) We should be able to get a handle to a single HTable and then operate on that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5548) Add ability to get a table in the shell
[ https://issues.apache.org/jira/browse/HBASE-5548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates updated HBASE-5548: --- Attachment: ruby_HBASE-5528-v0.patch Initial cut at this ticket. Its a little unclean as we don't have a nice way to isolate the java methods from the ruby methods and absolutely zero formatting from the output of the different commands as that is currently handled by the individual 'command' implementations. Add ability to get a table in the shell --- Key: HBASE-5548 URL: https://issues.apache.org/jira/browse/HBASE-5548 Project: HBase Issue Type: Improvement Components: shell Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0 Attachments: ruby_HBASE-5528-v0.patch Currently, all the commands that operate on a table in the shell first have to take the table as name as input. There are two main considerations: * It is annoying to have to write the table name every time, when you should just be able to get a reference to a table * the current implementation is very wasteful - it creates a new HTable for each call (but reuses the connection since it uses the same configuration) We should be able to get a handle to a single HTable and then operate on that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5515) Add a processRow API that supports atomic multiple reads and writes on a row
[ https://issues.apache.org/jira/browse/HBASE-5515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13225921#comment-13225921 ] Scott Chen commented on HBASE-5515: --- @Lars: Yes, 0.96 sounds good to me. Add a processRow API that supports atomic multiple reads and writes on a row Key: HBASE-5515 URL: https://issues.apache.org/jira/browse/HBASE-5515 Project: HBase Issue Type: New Feature Reporter: Scott Chen Assignee: Scott Chen Fix For: 0.96.0 Attachments: HBASE-5515.D2067.1.patch, HBASE-5515.D2067.10.patch, HBASE-5515.D2067.11.patch, HBASE-5515.D2067.12.patch, HBASE-5515.D2067.13.patch, HBASE-5515.D2067.14.patch, HBASE-5515.D2067.15.patch, HBASE-5515.D2067.16.patch, HBASE-5515.D2067.17.patch, HBASE-5515.D2067.18.patch, HBASE-5515.D2067.19.patch, HBASE-5515.D2067.2.patch, HBASE-5515.D2067.20.patch, HBASE-5515.D2067.21.patch, HBASE-5515.D2067.22.patch, HBASE-5515.D2067.3.patch, HBASE-5515.D2067.4.patch, HBASE-5515.D2067.5.patch, HBASE-5515.D2067.6.patch, HBASE-5515.D2067.7.patch, HBASE-5515.D2067.8.patch, HBASE-5515.D2067.9.patch We have modified HRegion.java internally to do some atomic row processing. It will be nice to have a plugable API for this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4435) Add Group By functionality using Coprocessors
[
https://issues.apache.org/jira/browse/HBASE-4435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13225933#comment-13225933
]
lifeng commented on HBASE-4435:
---
6 millis timeout while waiting for channel to be ready for read. ch :
java.nio.channels.SocketChannel
Add Group By functionality using Coprocessors
-
Key: HBASE-4435
URL: https://issues.apache.org/jira/browse/HBASE-4435
Project: HBase
Issue Type: Improvement
Components: coprocessors
Reporter: Nichole Treadway
Priority: Minor
Attachments: HBase-4435.patch
Adds in a Group By -like functionality to HBase, using the Coprocessor
framework.
It provides the ability to group the result set on one or more columns
(groupBy families). It computes statistics (max, min, sum, count, sum of
squares, number missing) for a second column, called the stats column.
To use, I've provided two implementations.
1. In the first, you specify a single group-by column and a stats field:
statsMap = gbc.getStats(tableName, scan, groupByFamily,
groupByQualifier, statsFamily, statsQualifier, statsFieldColumnInterpreter);
The result is a map with the Group By column value (as a String) to a
GroupByStatsValues object. The GroupByStatsValues object has max,min,sum etc.
of the stats column for that group.
2. The second implementation allows you to specify a list of group-by columns
and a stats field. The List of group-by columns is expected to contain lists
of {column family, qualifier} pairs.
statsMap = gbc.getStats(tableName, scan, listOfGroupByColumns,
statsFamily, statsQualifier, statsFieldColumnInterpreter);
The GroupByStatsValues code is adapted from the Solr Stats component.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5520) Support reseek() at RegionScanner
[ https://issues.apache.org/jira/browse/HBASE-5520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-5520: -- Description: reseek() is not supported currently at the RegionScanner level. We can support the same. This is created following the discussion under HBASE-2038 was: seek() reseek() is not supported currently at the RegionScanner level. We can support the same. This is created following the discussion under HBASE-2038 Assignee: ramkrishna.s.vasudevan Summary: Support reseek() at RegionScanner (was: Support seek() reseek() at RegionScanner) Support reseek() at RegionScanner - Key: HBASE-5520 URL: https://issues.apache.org/jira/browse/HBASE-5520 Project: HBase Issue Type: Improvement Components: regionserver Affects Versions: 0.92.0 Reporter: Anoop Sam John Assignee: ramkrishna.s.vasudevan Attachments: HBASE-5520_1.patch reseek() is not supported currently at the RegionScanner level. We can support the same. This is created following the discussion under HBASE-2038 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5549) Master can fail if ZooKeeper session expires
Master can fail if ZooKeeper session expires Key: HBASE-5549 URL: https://issues.apache.org/jira/browse/HBASE-5549 Project: HBase Issue Type: Bug Components: master, zookeeper Affects Versions: 0.96.0 Environment: all Reporter: nkeywal Assignee: nkeywal Priority: Minor There is a retry mechanism in RecoverableZooKeeper, but when the session expires, the whole ZooKeeperWatcher is recreated, hence the retry mechanism does not work in this case. This is why a sleep is needed in TestZooKeeper#testMasterSessionExpired: we need to wait for ZooKeeperWatcher to be recreated before using the connection. This can happen in real life, it can happen when: - master zookeeper starts - zookeeper connection is cut - master enters the retry loop - in the meantime the session expires - the network comes back, the session is recreated - the retries continues, but on the wrong object, hence fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5550) ZooKeeper connection to get the clusterId in the HConnectionImplementation constructor should be removed
ZooKeeper connection to get the clusterId in the HConnectionImplementation constructor should be removed Key: HBASE-5550 URL: https://issues.apache.org/jira/browse/HBASE-5550 Project: HBase Issue Type: Improvement Components: client Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Minor See title. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5551) Some functions should not be used by customer code and must be deprecated in 0.94
Some functions should not be used by customer code and must be deprecated in 0.94 - Key: HBASE-5551 URL: https://issues.apache.org/jira/browse/HBASE-5551 Project: HBase Issue Type: Improvement Affects Versions: 0.92.0 Reporter: nkeywal Assignee: nkeywal Fix For: 0.94.0 They are: HBaseAdmin#getMaster HConnection#getZooKeeperWatcher HConnection#getMaster -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5213) hbase master stop does not bring down backup masters
[ https://issues.apache.org/jira/browse/HBASE-5213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226116#comment-13226116 ] Zhihong Yu commented on HBASE-5213: --- Integrated to TRUNK. Thanks for the patch Gregory. hbase master stop does not bring down backup masters -- Key: HBASE-5213 URL: https://issues.apache.org/jira/browse/HBASE-5213 Project: HBase Issue Type: Bug Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0 Reporter: Gregory Chanan Assignee: Gregory Chanan Priority: Minor Attachments: HBASE-5213-v0-trunk.patch, HBASE-5213-v1-trunk.patch, HBASE-5213-v2-90.patch, HBASE-5213-v2-92.patch, HBASE-5213-v2-trunk.patch Typing hbase master stop produces the following message: stop Start cluster shutdown; Master signals RegionServer shutdown It seems like backup masters should be considered part of the cluster, but they are not brought down by hbase master stop. stop-hbase.sh does correctly bring down the backup masters. The same behavior is observed when a client app makes use of the client API HBaseAdmin.shutdown() http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HBaseAdmin.html#shutdown() -- this isn't too surprising since I think hbase master stop just calls this API. It seems like HBASE-1448 address this; perhaps there was a regression? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5541) Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks
[ https://issues.apache.org/jira/browse/HBASE-5541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226135#comment-13226135 ] Hudson commented on HBASE-5541: --- Integrated in HBase-TRUNK #2675 (See [https://builds.apache.org/job/HBase-TRUNK/2675/]) HBASE-5541 Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks (Revision 1298678) Result = SUCCESS larsh : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks Key: HBASE-5541 URL: https://issues.apache.org/jira/browse/HBASE-5541 Project: HBase Issue Type: Sub-task Components: client, regionserver Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.0 Attachments: 5541-v2.txt, 5541.txt Currently mutateRowsWithLocks holds the row lock while the HLog is sync'ed. Similar to what we do in doMiniBatchPut, we should create the log entry with the lock held, but only sync the HLog after the lock is released, along with rollback logic in case the sync'ing fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5399) Cut the link between the client and the zookeeper ensemble
[ https://issues.apache.org/jira/browse/HBASE-5399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-5399: --- Status: Open (was: Patch Available) Cut the link between the client and the zookeeper ensemble -- Key: HBASE-5399 URL: https://issues.apache.org/jira/browse/HBASE-5399 Project: HBase Issue Type: Improvement Components: client Affects Versions: 0.94.0 Environment: all Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.96.0 Attachments: 5399.v27.patch, 5399.v38.patch, 5399.v39.patch, 5399.v40.patch, 5399.v41.patch, 5399_inprogress.patch, 5399_inprogress.v14.patch, 5399_inprogress.v16.patch, 5399_inprogress.v18.patch, 5399_inprogress.v20.patch, 5399_inprogress.v21.patch, 5399_inprogress.v23.patch, 5399_inprogress.v3.patch, 5399_inprogress.v32.patch, 5399_inprogress.v9.patch The link is often considered as an issue, for various reasons. One of them being that there is a limit on the number of connection that ZK can manage. Stack was suggesting as well to remove the link to master from HConnection. There are choices to be made considering the existing API (that we don't want to break). The first patches I will submit on hadoop-qa should not be committed: they are here to show the progress on the direction taken. ZooKeeper is used for: - public getter, to let the client do whatever he wants, and close ZooKeeper when closing the connection = we have to deprecate this but keep it. - read get master address to create a master = now done with a temporary zookeeper connection - read root location = now done with a temporary zookeeper connection, but questionable. Used in public function locateRegion. To be reworked. - read cluster id = now done once with a temporary zookeeper connection. - check if base done is available = now done once with a zookeeper connection given as a parameter - isTableDisabled/isTableAvailable = public functions, now done with a temporary zookeeper connection. - Called internally from HBaseAdmin and HTable - getCurrentNrHRS(): public function to get the number of region servers and create a pool of thread = now done with a temporary zookeeper connection - Master is used for: - getMaster public getter, as for ZooKeeper = we have to deprecate this but keep it. - isMasterRunning(): public function, used internally by HMerge HBaseAdmin - getHTableDescriptor*: public functions offering access to the master. = we could make them using a temporary master connection as well. Main points are: - hbase class for ZooKeeper; ZooKeeperWatcher is really designed for a strongly coupled architecture ;-). This can be changed, but requires a lot of modifications in these classes (likely adding a class in the middle of the hierarchy, something like that). Anyway, non connected client will always be really slower, because it's a tcp connection, and establishing a tcp connection is slow. - having a link between ZK and all the client seems to make sense for some Use Cases. However, it won't scale if a TCP connection is required for every client - if we move the table descriptor part away from the client, we need to find a new place for it. - we will have the same issue if HBaseAdmin (for both ZK Master), may be we can put a timeout on the connection. That would make the whole system less deterministic however. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5399) Cut the link between the client and the zookeeper ensemble
[ https://issues.apache.org/jira/browse/HBASE-5399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-5399: --- Attachment: 5399.v41.patch This version integrates the last comments + the trunk. Locally, I've got random failures I didn't get 3 days ago. So ley me confirm before committing. Cut the link between the client and the zookeeper ensemble -- Key: HBASE-5399 URL: https://issues.apache.org/jira/browse/HBASE-5399 Project: HBase Issue Type: Improvement Components: client Affects Versions: 0.94.0 Environment: all Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.96.0 Attachments: 5399.v27.patch, 5399.v38.patch, 5399.v39.patch, 5399.v40.patch, 5399.v41.patch, 5399_inprogress.patch, 5399_inprogress.v14.patch, 5399_inprogress.v16.patch, 5399_inprogress.v18.patch, 5399_inprogress.v20.patch, 5399_inprogress.v21.patch, 5399_inprogress.v23.patch, 5399_inprogress.v3.patch, 5399_inprogress.v32.patch, 5399_inprogress.v9.patch The link is often considered as an issue, for various reasons. One of them being that there is a limit on the number of connection that ZK can manage. Stack was suggesting as well to remove the link to master from HConnection. There are choices to be made considering the existing API (that we don't want to break). The first patches I will submit on hadoop-qa should not be committed: they are here to show the progress on the direction taken. ZooKeeper is used for: - public getter, to let the client do whatever he wants, and close ZooKeeper when closing the connection = we have to deprecate this but keep it. - read get master address to create a master = now done with a temporary zookeeper connection - read root location = now done with a temporary zookeeper connection, but questionable. Used in public function locateRegion. To be reworked. - read cluster id = now done once with a temporary zookeeper connection. - check if base done is available = now done once with a zookeeper connection given as a parameter - isTableDisabled/isTableAvailable = public functions, now done with a temporary zookeeper connection. - Called internally from HBaseAdmin and HTable - getCurrentNrHRS(): public function to get the number of region servers and create a pool of thread = now done with a temporary zookeeper connection - Master is used for: - getMaster public getter, as for ZooKeeper = we have to deprecate this but keep it. - isMasterRunning(): public function, used internally by HMerge HBaseAdmin - getHTableDescriptor*: public functions offering access to the master. = we could make them using a temporary master connection as well. Main points are: - hbase class for ZooKeeper; ZooKeeperWatcher is really designed for a strongly coupled architecture ;-). This can be changed, but requires a lot of modifications in these classes (likely adding a class in the middle of the hierarchy, something like that). Anyway, non connected client will always be really slower, because it's a tcp connection, and establishing a tcp connection is slow. - having a link between ZK and all the client seems to make sense for some Use Cases. However, it won't scale if a TCP connection is required for every client - if we move the table descriptor part away from the client, we need to find a new place for it. - we will have the same issue if HBaseAdmin (for both ZK Master), may be we can put a timeout on the connection. That would make the whole system less deterministic however. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5213) hbase master stop does not bring down backup masters
[ https://issues.apache.org/jira/browse/HBASE-5213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226136#comment-13226136 ] Hudson commented on HBASE-5213: --- Integrated in HBase-TRUNK #2675 (See [https://builds.apache.org/job/HBase-TRUNK/2675/]) HBASE-5213 hbase master stop does not bring down backup masters (Gregory) (Revision 1298859) Result = SUCCESS tedyu : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterShutdown.java hbase master stop does not bring down backup masters -- Key: HBASE-5213 URL: https://issues.apache.org/jira/browse/HBASE-5213 Project: HBase Issue Type: Bug Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0 Reporter: Gregory Chanan Assignee: Gregory Chanan Priority: Minor Attachments: HBASE-5213-v0-trunk.patch, HBASE-5213-v1-trunk.patch, HBASE-5213-v2-90.patch, HBASE-5213-v2-92.patch, HBASE-5213-v2-trunk.patch Typing hbase master stop produces the following message: stop Start cluster shutdown; Master signals RegionServer shutdown It seems like backup masters should be considered part of the cluster, but they are not brought down by hbase master stop. stop-hbase.sh does correctly bring down the backup masters. The same behavior is observed when a client app makes use of the client API HBaseAdmin.shutdown() http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HBaseAdmin.html#shutdown() -- this isn't too surprising since I think hbase master stop just calls this API. It seems like HBASE-1448 address this; perhaps there was a regression? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5399) Cut the link between the client and the zookeeper ensemble
[ https://issues.apache.org/jira/browse/HBASE-5399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-5399: --- Status: Patch Available (was: Open) Cut the link between the client and the zookeeper ensemble -- Key: HBASE-5399 URL: https://issues.apache.org/jira/browse/HBASE-5399 Project: HBase Issue Type: Improvement Components: client Affects Versions: 0.94.0 Environment: all Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.96.0 Attachments: 5399.v27.patch, 5399.v38.patch, 5399.v39.patch, 5399.v40.patch, 5399.v41.patch, 5399_inprogress.patch, 5399_inprogress.v14.patch, 5399_inprogress.v16.patch, 5399_inprogress.v18.patch, 5399_inprogress.v20.patch, 5399_inprogress.v21.patch, 5399_inprogress.v23.patch, 5399_inprogress.v3.patch, 5399_inprogress.v32.patch, 5399_inprogress.v9.patch The link is often considered as an issue, for various reasons. One of them being that there is a limit on the number of connection that ZK can manage. Stack was suggesting as well to remove the link to master from HConnection. There are choices to be made considering the existing API (that we don't want to break). The first patches I will submit on hadoop-qa should not be committed: they are here to show the progress on the direction taken. ZooKeeper is used for: - public getter, to let the client do whatever he wants, and close ZooKeeper when closing the connection = we have to deprecate this but keep it. - read get master address to create a master = now done with a temporary zookeeper connection - read root location = now done with a temporary zookeeper connection, but questionable. Used in public function locateRegion. To be reworked. - read cluster id = now done once with a temporary zookeeper connection. - check if base done is available = now done once with a zookeeper connection given as a parameter - isTableDisabled/isTableAvailable = public functions, now done with a temporary zookeeper connection. - Called internally from HBaseAdmin and HTable - getCurrentNrHRS(): public function to get the number of region servers and create a pool of thread = now done with a temporary zookeeper connection - Master is used for: - getMaster public getter, as for ZooKeeper = we have to deprecate this but keep it. - isMasterRunning(): public function, used internally by HMerge HBaseAdmin - getHTableDescriptor*: public functions offering access to the master. = we could make them using a temporary master connection as well. Main points are: - hbase class for ZooKeeper; ZooKeeperWatcher is really designed for a strongly coupled architecture ;-). This can be changed, but requires a lot of modifications in these classes (likely adding a class in the middle of the hierarchy, something like that). Anyway, non connected client will always be really slower, because it's a tcp connection, and establishing a tcp connection is slow. - having a link between ZK and all the client seems to make sense for some Use Cases. However, it won't scale if a TCP connection is required for every client - if we move the table descriptor part away from the client, we need to find a new place for it. - we will have the same issue if HBaseAdmin (for both ZK Master), may be we can put a timeout on the connection. That would make the whole system less deterministic however. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5399) Cut the link between the client and the zookeeper ensemble
[ https://issues.apache.org/jira/browse/HBASE-5399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226140#comment-13226140 ] Hadoop QA commented on HBASE-5399: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12517728/5399.v41.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 30 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1143//console This message is automatically generated. Cut the link between the client and the zookeeper ensemble -- Key: HBASE-5399 URL: https://issues.apache.org/jira/browse/HBASE-5399 Project: HBase Issue Type: Improvement Components: client Affects Versions: 0.94.0 Environment: all Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.96.0 Attachments: 5399.v27.patch, 5399.v38.patch, 5399.v39.patch, 5399.v40.patch, 5399.v41.patch, 5399_inprogress.patch, 5399_inprogress.v14.patch, 5399_inprogress.v16.patch, 5399_inprogress.v18.patch, 5399_inprogress.v20.patch, 5399_inprogress.v21.patch, 5399_inprogress.v23.patch, 5399_inprogress.v3.patch, 5399_inprogress.v32.patch, 5399_inprogress.v9.patch The link is often considered as an issue, for various reasons. One of them being that there is a limit on the number of connection that ZK can manage. Stack was suggesting as well to remove the link to master from HConnection. There are choices to be made considering the existing API (that we don't want to break). The first patches I will submit on hadoop-qa should not be committed: they are here to show the progress on the direction taken. ZooKeeper is used for: - public getter, to let the client do whatever he wants, and close ZooKeeper when closing the connection = we have to deprecate this but keep it. - read get master address to create a master = now done with a temporary zookeeper connection - read root location = now done with a temporary zookeeper connection, but questionable. Used in public function locateRegion. To be reworked. - read cluster id = now done once with a temporary zookeeper connection. - check if base done is available = now done once with a zookeeper connection given as a parameter - isTableDisabled/isTableAvailable = public functions, now done with a temporary zookeeper connection. - Called internally from HBaseAdmin and HTable - getCurrentNrHRS(): public function to get the number of region servers and create a pool of thread = now done with a temporary zookeeper connection - Master is used for: - getMaster public getter, as for ZooKeeper = we have to deprecate this but keep it. - isMasterRunning(): public function, used internally by HMerge HBaseAdmin - getHTableDescriptor*: public functions offering access to the master. = we could make them using a temporary master connection as well. Main points are: - hbase class for ZooKeeper; ZooKeeperWatcher is really designed for a strongly coupled architecture ;-). This can be changed, but requires a lot of modifications in these classes (likely adding a class in the middle of the hierarchy, something like that). Anyway, non connected client will always be really slower, because it's a tcp connection, and establishing a tcp connection is slow. - having a link between ZK and all the client seems to make sense for some Use Cases. However, it won't scale if a TCP connection is required for every client - if we move the table descriptor part away from the client, we need to find a new place for it. - we will have the same issue if HBaseAdmin (for both ZK Master), may be we can put a timeout on the connection. That would make the whole system less deterministic however. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5213) hbase master stop does not bring down backup masters
[ https://issues.apache.org/jira/browse/HBASE-5213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226143#comment-13226143 ] Zhihong Yu commented on HBASE-5213: --- Integrated to 0.94 as well. hbase master stop does not bring down backup masters -- Key: HBASE-5213 URL: https://issues.apache.org/jira/browse/HBASE-5213 Project: HBase Issue Type: Bug Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0 Reporter: Gregory Chanan Assignee: Gregory Chanan Priority: Minor Attachments: HBASE-5213-v0-trunk.patch, HBASE-5213-v1-trunk.patch, HBASE-5213-v2-90.patch, HBASE-5213-v2-92.patch, HBASE-5213-v2-trunk.patch Typing hbase master stop produces the following message: stop Start cluster shutdown; Master signals RegionServer shutdown It seems like backup masters should be considered part of the cluster, but they are not brought down by hbase master stop. stop-hbase.sh does correctly bring down the backup masters. The same behavior is observed when a client app makes use of the client API HBaseAdmin.shutdown() http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HBaseAdmin.html#shutdown() -- this isn't too surprising since I think hbase master stop just calls this API. It seems like HBASE-1448 address this; perhaps there was a regression? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5552) Clean up our jmx view; its a bit of a mess
Clean up our jmx view; its a bit of a mess -- Key: HBASE-5552 URL: https://issues.apache.org/jira/browse/HBASE-5552 Project: HBase Issue Type: Bug Reporter: stack Priority: Blocker Fix before we release 0.92.1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5399) Cut the link between the client and the zookeeper ensemble
[ https://issues.apache.org/jira/browse/HBASE-5399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-5399: --- Status: Patch Available (was: Open) Cut the link between the client and the zookeeper ensemble -- Key: HBASE-5399 URL: https://issues.apache.org/jira/browse/HBASE-5399 Project: HBase Issue Type: Improvement Components: client Affects Versions: 0.94.0 Environment: all Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.96.0 Attachments: 5399.v27.patch, 5399.v38.patch, 5399.v39.patch, 5399.v40.patch, 5399.v41.patch, 5399.v42.patch, 5399_inprogress.patch, 5399_inprogress.v14.patch, 5399_inprogress.v16.patch, 5399_inprogress.v18.patch, 5399_inprogress.v20.patch, 5399_inprogress.v21.patch, 5399_inprogress.v23.patch, 5399_inprogress.v3.patch, 5399_inprogress.v32.patch, 5399_inprogress.v9.patch The link is often considered as an issue, for various reasons. One of them being that there is a limit on the number of connection that ZK can manage. Stack was suggesting as well to remove the link to master from HConnection. There are choices to be made considering the existing API (that we don't want to break). The first patches I will submit on hadoop-qa should not be committed: they are here to show the progress on the direction taken. ZooKeeper is used for: - public getter, to let the client do whatever he wants, and close ZooKeeper when closing the connection = we have to deprecate this but keep it. - read get master address to create a master = now done with a temporary zookeeper connection - read root location = now done with a temporary zookeeper connection, but questionable. Used in public function locateRegion. To be reworked. - read cluster id = now done once with a temporary zookeeper connection. - check if base done is available = now done once with a zookeeper connection given as a parameter - isTableDisabled/isTableAvailable = public functions, now done with a temporary zookeeper connection. - Called internally from HBaseAdmin and HTable - getCurrentNrHRS(): public function to get the number of region servers and create a pool of thread = now done with a temporary zookeeper connection - Master is used for: - getMaster public getter, as for ZooKeeper = we have to deprecate this but keep it. - isMasterRunning(): public function, used internally by HMerge HBaseAdmin - getHTableDescriptor*: public functions offering access to the master. = we could make them using a temporary master connection as well. Main points are: - hbase class for ZooKeeper; ZooKeeperWatcher is really designed for a strongly coupled architecture ;-). This can be changed, but requires a lot of modifications in these classes (likely adding a class in the middle of the hierarchy, something like that). Anyway, non connected client will always be really slower, because it's a tcp connection, and establishing a tcp connection is slow. - having a link between ZK and all the client seems to make sense for some Use Cases. However, it won't scale if a TCP connection is required for every client - if we move the table descriptor part away from the client, we need to find a new place for it. - we will have the same issue if HBaseAdmin (for both ZK Master), may be we can put a timeout on the connection. That would make the whole system less deterministic however. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5399) Cut the link between the client and the zookeeper ensemble
[ https://issues.apache.org/jira/browse/HBASE-5399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-5399: --- Status: Open (was: Patch Available) Cut the link between the client and the zookeeper ensemble -- Key: HBASE-5399 URL: https://issues.apache.org/jira/browse/HBASE-5399 Project: HBase Issue Type: Improvement Components: client Affects Versions: 0.94.0 Environment: all Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.96.0 Attachments: 5399.v27.patch, 5399.v38.patch, 5399.v39.patch, 5399.v40.patch, 5399.v41.patch, 5399.v42.patch, 5399_inprogress.patch, 5399_inprogress.v14.patch, 5399_inprogress.v16.patch, 5399_inprogress.v18.patch, 5399_inprogress.v20.patch, 5399_inprogress.v21.patch, 5399_inprogress.v23.patch, 5399_inprogress.v3.patch, 5399_inprogress.v32.patch, 5399_inprogress.v9.patch The link is often considered as an issue, for various reasons. One of them being that there is a limit on the number of connection that ZK can manage. Stack was suggesting as well to remove the link to master from HConnection. There are choices to be made considering the existing API (that we don't want to break). The first patches I will submit on hadoop-qa should not be committed: they are here to show the progress on the direction taken. ZooKeeper is used for: - public getter, to let the client do whatever he wants, and close ZooKeeper when closing the connection = we have to deprecate this but keep it. - read get master address to create a master = now done with a temporary zookeeper connection - read root location = now done with a temporary zookeeper connection, but questionable. Used in public function locateRegion. To be reworked. - read cluster id = now done once with a temporary zookeeper connection. - check if base done is available = now done once with a zookeeper connection given as a parameter - isTableDisabled/isTableAvailable = public functions, now done with a temporary zookeeper connection. - Called internally from HBaseAdmin and HTable - getCurrentNrHRS(): public function to get the number of region servers and create a pool of thread = now done with a temporary zookeeper connection - Master is used for: - getMaster public getter, as for ZooKeeper = we have to deprecate this but keep it. - isMasterRunning(): public function, used internally by HMerge HBaseAdmin - getHTableDescriptor*: public functions offering access to the master. = we could make them using a temporary master connection as well. Main points are: - hbase class for ZooKeeper; ZooKeeperWatcher is really designed for a strongly coupled architecture ;-). This can be changed, but requires a lot of modifications in these classes (likely adding a class in the middle of the hierarchy, something like that). Anyway, non connected client will always be really slower, because it's a tcp connection, and establishing a tcp connection is slow. - having a link between ZK and all the client seems to make sense for some Use Cases. However, it won't scale if a TCP connection is required for every client - if we move the table descriptor part away from the client, we need to find a new place for it. - we will have the same issue if HBaseAdmin (for both ZK Master), may be we can put a timeout on the connection. That would make the whole system less deterministic however. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5399) Cut the link between the client and the zookeeper ensemble
[ https://issues.apache.org/jira/browse/HBASE-5399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-5399: --- Attachment: 5399.v42.patch Cut the link between the client and the zookeeper ensemble -- Key: HBASE-5399 URL: https://issues.apache.org/jira/browse/HBASE-5399 Project: HBase Issue Type: Improvement Components: client Affects Versions: 0.94.0 Environment: all Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.96.0 Attachments: 5399.v27.patch, 5399.v38.patch, 5399.v39.patch, 5399.v40.patch, 5399.v41.patch, 5399.v42.patch, 5399_inprogress.patch, 5399_inprogress.v14.patch, 5399_inprogress.v16.patch, 5399_inprogress.v18.patch, 5399_inprogress.v20.patch, 5399_inprogress.v21.patch, 5399_inprogress.v23.patch, 5399_inprogress.v3.patch, 5399_inprogress.v32.patch, 5399_inprogress.v9.patch The link is often considered as an issue, for various reasons. One of them being that there is a limit on the number of connection that ZK can manage. Stack was suggesting as well to remove the link to master from HConnection. There are choices to be made considering the existing API (that we don't want to break). The first patches I will submit on hadoop-qa should not be committed: they are here to show the progress on the direction taken. ZooKeeper is used for: - public getter, to let the client do whatever he wants, and close ZooKeeper when closing the connection = we have to deprecate this but keep it. - read get master address to create a master = now done with a temporary zookeeper connection - read root location = now done with a temporary zookeeper connection, but questionable. Used in public function locateRegion. To be reworked. - read cluster id = now done once with a temporary zookeeper connection. - check if base done is available = now done once with a zookeeper connection given as a parameter - isTableDisabled/isTableAvailable = public functions, now done with a temporary zookeeper connection. - Called internally from HBaseAdmin and HTable - getCurrentNrHRS(): public function to get the number of region servers and create a pool of thread = now done with a temporary zookeeper connection - Master is used for: - getMaster public getter, as for ZooKeeper = we have to deprecate this but keep it. - isMasterRunning(): public function, used internally by HMerge HBaseAdmin - getHTableDescriptor*: public functions offering access to the master. = we could make them using a temporary master connection as well. Main points are: - hbase class for ZooKeeper; ZooKeeperWatcher is really designed for a strongly coupled architecture ;-). This can be changed, but requires a lot of modifications in these classes (likely adding a class in the middle of the hierarchy, something like that). Anyway, non connected client will always be really slower, because it's a tcp connection, and establishing a tcp connection is slow. - having a link between ZK and all the client seems to make sense for some Use Cases. However, it won't scale if a TCP connection is required for every client - if we move the table descriptor part away from the client, we need to find a new place for it. - we will have the same issue if HBaseAdmin (for both ZK Master), may be we can put a timeout on the connection. That would make the whole system less deterministic however. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5552) Clean up our jmx view; its a bit of a mess
[ https://issues.apache.org/jira/browse/HBASE-5552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5552: - Attachment: 0.92.0jmx.png This is a mess I for sure helped make. Here is 0.92.0 view. Its: hadoop HBase Master RegionServer ... which is wrong but thats how it is. Should have been hadoop hbase Master RegionServer Clean up our jmx view; its a bit of a mess -- Key: HBASE-5552 URL: https://issues.apache.org/jira/browse/HBASE-5552 Project: HBase Issue Type: Bug Reporter: stack Priority: Blocker Attachments: 0.92.0jmx.png Fix before we release 0.92.1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5520) Support reseek() at RegionScanner
[ https://issues.apache.org/jira/browse/HBASE-5520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-5520: -- Attachment: HBASE-5520_2.patch Updated the patch with requestSeek. Support reseek() at RegionScanner - Key: HBASE-5520 URL: https://issues.apache.org/jira/browse/HBASE-5520 Project: HBase Issue Type: Improvement Components: regionserver Affects Versions: 0.92.0 Reporter: Anoop Sam John Assignee: ramkrishna.s.vasudevan Attachments: HBASE-5520_1.patch, HBASE-5520_2.patch reseek() is not supported currently at the RegionScanner level. We can support the same. This is created following the discussion under HBASE-2038 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5520) Support reseek() at RegionScanner
[ https://issues.apache.org/jira/browse/HBASE-5520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-5520: -- Fix Version/s: 0.96.0 Status: Patch Available (was: Open) Support reseek() at RegionScanner - Key: HBASE-5520 URL: https://issues.apache.org/jira/browse/HBASE-5520 Project: HBase Issue Type: Improvement Components: regionserver Affects Versions: 0.92.0 Reporter: Anoop Sam John Assignee: ramkrishna.s.vasudevan Fix For: 0.96.0 Attachments: HBASE-5520_1.patch, HBASE-5520_2.patch reseek() is not supported currently at the RegionScanner level. We can support the same. This is created following the discussion under HBASE-2038 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5552) Clean up our jmx view; its a bit of a mess
[
https://issues.apache.org/jira/browse/HBASE-5552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226172#comment-13226172
]
stack commented on HBASE-5552:
--
Let me fix formatting:
{code}
hadoop
HBase
Master
RegionServer
{code}
It should be...
{code}
hadoop
hbase
master
regionserver
{code}
Clean up our jmx view; its a bit of a mess
--
Key: HBASE-5552
URL: https://issues.apache.org/jira/browse/HBASE-5552
Project: HBase
Issue Type: Bug
Reporter: stack
Priority: Blocker
Attachments: 0.92.0jmx.png
Fix before we release 0.92.1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5206) Port HBASE-5155 to 0.92 and TRUNK
[ https://issues.apache.org/jira/browse/HBASE-5206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Jindal updated HBASE-5206: --- Attachment: 5206_92_1.patch Port HBASE-5155 to 0.92 and TRUNK - Key: HBASE-5206 URL: https://issues.apache.org/jira/browse/HBASE-5206 Project: HBase Issue Type: Bug Affects Versions: 0.92.2, 0.96.0 Reporter: Zhihong Yu Attachments: 5206_92_1.patch, 5206_trunk_1.patch This JIRA ports HBASE-5155 (ServerShutDownHandler And Disable/Delete should not happen parallely leading to recreation of regions that were deleted) to 0.92 and TRUNK -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5206) Port HBASE-5155 to 0.92 and TRUNK
[ https://issues.apache.org/jira/browse/HBASE-5206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Jindal updated HBASE-5206: --- Attachment: 5206_trunk_1.patch Port HBASE-5155 to 0.92 and TRUNK - Key: HBASE-5206 URL: https://issues.apache.org/jira/browse/HBASE-5206 Project: HBase Issue Type: Bug Affects Versions: 0.92.2, 0.96.0 Reporter: Zhihong Yu Attachments: 5206_92_1.patch, 5206_trunk_1.patch This JIRA ports HBASE-5155 (ServerShutDownHandler And Disable/Delete should not happen parallely leading to recreation of regions that were deleted) to 0.92 and TRUNK -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5206) Port HBASE-5155 to 0.92 and TRUNK
[ https://issues.apache.org/jira/browse/HBASE-5206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Jindal updated HBASE-5206: --- Affects Version/s: 0.96.0 0.92.2 Status: Patch Available (was: Open) Port HBASE-5155 to 0.92 and TRUNK - Key: HBASE-5206 URL: https://issues.apache.org/jira/browse/HBASE-5206 Project: HBase Issue Type: Bug Affects Versions: 0.92.2, 0.96.0 Reporter: Zhihong Yu Attachments: 5206_92_1.patch, 5206_trunk_1.patch This JIRA ports HBASE-5155 (ServerShutDownHandler And Disable/Delete should not happen parallely leading to recreation of regions that were deleted) to 0.92 and TRUNK -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5520) Support reseek() at RegionScanner
[ https://issues.apache.org/jira/browse/HBASE-5520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226177#comment-13226177 ] Hadoop QA commented on HBASE-5520: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12517734/HBASE-5520_2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -123 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 159 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1145//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1145//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1145//console This message is automatically generated. Support reseek() at RegionScanner - Key: HBASE-5520 URL: https://issues.apache.org/jira/browse/HBASE-5520 Project: HBase Issue Type: Improvement Components: regionserver Affects Versions: 0.92.0 Reporter: Anoop Sam John Assignee: ramkrishna.s.vasudevan Fix For: 0.96.0 Attachments: HBASE-5520_1.patch, HBASE-5520_2.patch reseek() is not supported currently at the RegionScanner level. We can support the same. This is created following the discussion under HBASE-2038 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5399) Cut the link between the client and the zookeeper ensemble
[ https://issues.apache.org/jira/browse/HBASE-5399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226180#comment-13226180 ] Hadoop QA commented on HBASE-5399: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12517731/5399.v42.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 30 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -123 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 160 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.TestServerCustomProtocol org.apache.hadoop.hbase.mapreduce.TestImportTsv org.apache.hadoop.hbase.mapred.TestTableMapReduce org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1144//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1144//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1144//console This message is automatically generated. Cut the link between the client and the zookeeper ensemble -- Key: HBASE-5399 URL: https://issues.apache.org/jira/browse/HBASE-5399 Project: HBase Issue Type: Improvement Components: client Affects Versions: 0.94.0 Environment: all Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.96.0 Attachments: 5399.v27.patch, 5399.v38.patch, 5399.v39.patch, 5399.v40.patch, 5399.v41.patch, 5399.v42.patch, 5399_inprogress.patch, 5399_inprogress.v14.patch, 5399_inprogress.v16.patch, 5399_inprogress.v18.patch, 5399_inprogress.v20.patch, 5399_inprogress.v21.patch, 5399_inprogress.v23.patch, 5399_inprogress.v3.patch, 5399_inprogress.v32.patch, 5399_inprogress.v9.patch The link is often considered as an issue, for various reasons. One of them being that there is a limit on the number of connection that ZK can manage. Stack was suggesting as well to remove the link to master from HConnection. There are choices to be made considering the existing API (that we don't want to break). The first patches I will submit on hadoop-qa should not be committed: they are here to show the progress on the direction taken. ZooKeeper is used for: - public getter, to let the client do whatever he wants, and close ZooKeeper when closing the connection = we have to deprecate this but keep it. - read get master address to create a master = now done with a temporary zookeeper connection - read root location = now done with a temporary zookeeper connection, but questionable. Used in public function locateRegion. To be reworked. - read cluster id = now done once with a temporary zookeeper connection. - check if base done is available = now done once with a zookeeper connection given as a parameter - isTableDisabled/isTableAvailable = public functions, now done with a temporary zookeeper connection. - Called internally from HBaseAdmin and HTable - getCurrentNrHRS(): public function to get the number of region servers and create a pool of thread = now done with a temporary zookeeper connection - Master is used for: - getMaster public getter, as for ZooKeeper = we have to deprecate this but keep it. - isMasterRunning(): public function, used internally by HMerge HBaseAdmin - getHTableDescriptor*: public functions offering access to the master. = we could make them using a temporary master connection as well. Main points are: - hbase class for ZooKeeper; ZooKeeperWatcher is really designed for a strongly coupled architecture ;-). This can be changed, but requires a lot of modifications in these classes (likely adding a class in the middle of the hierarchy, something like that). Anyway, non connected client will always be really slower, because it's a tcp connection, and establishing a tcp connection is slow. - having a link between ZK and all the client seems to make sense for some Use Cases. However, it won't scale if a TCP connection is required for every client - if we move the table descriptor part away from the client, we need to find a new place for it. - we will have the same issue if HBaseAdmin (for both ZK Master), may be we can put
[jira] [Commented] (HBASE-5552) Clean up our jmx view; its a bit of a mess
[ https://issues.apache.org/jira/browse/HBASE-5552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226182#comment-13226182 ] stack commented on HBASE-5552: -- As is, our naming is just broke. You can't have instances of different clusters on one machine. Our master bean is not uniquely named so you can't have multiple masters on the one machine ditto regionservers (The rpc servers publish their own bean distingushed by the port they run on which is better only should probably have master or regionserver prefix). The name of our master bean is 'MasterStatistics' though its metrics only (even the operation is a reset on metrics). Doing minimum so can get 0.92.1 in this issue but this stuff needs a revamp. Clean up our jmx view; its a bit of a mess -- Key: HBASE-5552 URL: https://issues.apache.org/jira/browse/HBASE-5552 Project: HBase Issue Type: Bug Reporter: stack Priority: Blocker Attachments: 0.92.0jmx.png Fix before we release 0.92.1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5213) hbase master stop does not bring down backup masters
[ https://issues.apache.org/jira/browse/HBASE-5213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226184#comment-13226184 ] Hudson commented on HBASE-5213: --- Integrated in HBase-0.94 #23 (See [https://builds.apache.org/job/HBase-0.94/23/]) HBASE-5213 hbase master stop does not bring down backup masters (Gregory) (Revision 1298877) Result = SUCCESS tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestMasterShutdown.java hbase master stop does not bring down backup masters -- Key: HBASE-5213 URL: https://issues.apache.org/jira/browse/HBASE-5213 Project: HBase Issue Type: Bug Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0 Reporter: Gregory Chanan Assignee: Gregory Chanan Priority: Minor Attachments: HBASE-5213-v0-trunk.patch, HBASE-5213-v1-trunk.patch, HBASE-5213-v2-90.patch, HBASE-5213-v2-92.patch, HBASE-5213-v2-trunk.patch Typing hbase master stop produces the following message: stop Start cluster shutdown; Master signals RegionServer shutdown It seems like backup masters should be considered part of the cluster, but they are not brought down by hbase master stop. stop-hbase.sh does correctly bring down the backup masters. The same behavior is observed when a client app makes use of the client API HBaseAdmin.shutdown() http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HBaseAdmin.html#shutdown() -- this isn't too surprising since I think hbase master stop just calls this API. It seems like HBASE-1448 address this; perhaps there was a regression? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5552) Clean up our jmx view; its a bit of a mess
[ https://issues.apache.org/jira/browse/HBASE-5552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5552: - Attachment: currentjmxview.png Here is the current 0.92 tip view. Its got the broke org.apache.hbase (which was my suggestion made w/o looking at the context into which I was suggesting... thought our beans pegged at root, not under hadoop which makes the suggestion silly). Clean up our jmx view; its a bit of a mess -- Key: HBASE-5552 URL: https://issues.apache.org/jira/browse/HBASE-5552 Project: HBase Issue Type: Bug Reporter: stack Priority: Blocker Attachments: 0.92.0jmx.png, currentjmxview.png Fix before we release 0.92.1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5552) Clean up our jmx view; its a bit of a mess
[ https://issues.apache.org/jira/browse/HBASE-5552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5552: - Attachment: patchedjmxview.png Here is what this patch does... it makes new beans Master and Regionserver which have Master and RegionServer vitals in them. They sit beside the statistics on each. I think this is what the lads over in hbase-5325 originally suggested and I thought it wrong -- I still do.. but its less egregious that this o.a.h stuff) Clean up our jmx view; its a bit of a mess -- Key: HBASE-5552 URL: https://issues.apache.org/jira/browse/HBASE-5552 Project: HBase Issue Type: Bug Reporter: stack Priority: Blocker Attachments: 0.92.0jmx.png, currentjmxview.png, patchedjmxview.png Fix before we release 0.92.1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5552) Clean up our jmx view; its a bit of a mess
[ https://issues.apache.org/jira/browse/HBASE-5552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5552: - Attachment: 5552.txt Clean up our jmx view; its a bit of a mess -- Key: HBASE-5552 URL: https://issues.apache.org/jira/browse/HBASE-5552 Project: HBase Issue Type: Bug Reporter: stack Priority: Blocker Attachments: 0.92.0jmx.png, 5552.txt, currentjmxview.png, patchedjmxview.png Fix before we release 0.92.1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5399) Cut the link between the client and the zookeeper ensemble
[ https://issues.apache.org/jira/browse/HBASE-5399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226195#comment-13226195 ] nkeywal commented on HBASE-5399: They're all different from the ones I got locally. It could be pure test flakiness. Let's retry. Cut the link between the client and the zookeeper ensemble -- Key: HBASE-5399 URL: https://issues.apache.org/jira/browse/HBASE-5399 Project: HBase Issue Type: Improvement Components: client Affects Versions: 0.94.0 Environment: all Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.96.0 Attachments: 5399.v27.patch, 5399.v38.patch, 5399.v39.patch, 5399.v40.patch, 5399.v41.patch, 5399.v42.patch, 5399.v42.patch, 5399_inprogress.patch, 5399_inprogress.v14.patch, 5399_inprogress.v16.patch, 5399_inprogress.v18.patch, 5399_inprogress.v20.patch, 5399_inprogress.v21.patch, 5399_inprogress.v23.patch, 5399_inprogress.v3.patch, 5399_inprogress.v32.patch, 5399_inprogress.v9.patch The link is often considered as an issue, for various reasons. One of them being that there is a limit on the number of connection that ZK can manage. Stack was suggesting as well to remove the link to master from HConnection. There are choices to be made considering the existing API (that we don't want to break). The first patches I will submit on hadoop-qa should not be committed: they are here to show the progress on the direction taken. ZooKeeper is used for: - public getter, to let the client do whatever he wants, and close ZooKeeper when closing the connection = we have to deprecate this but keep it. - read get master address to create a master = now done with a temporary zookeeper connection - read root location = now done with a temporary zookeeper connection, but questionable. Used in public function locateRegion. To be reworked. - read cluster id = now done once with a temporary zookeeper connection. - check if base done is available = now done once with a zookeeper connection given as a parameter - isTableDisabled/isTableAvailable = public functions, now done with a temporary zookeeper connection. - Called internally from HBaseAdmin and HTable - getCurrentNrHRS(): public function to get the number of region servers and create a pool of thread = now done with a temporary zookeeper connection - Master is used for: - getMaster public getter, as for ZooKeeper = we have to deprecate this but keep it. - isMasterRunning(): public function, used internally by HMerge HBaseAdmin - getHTableDescriptor*: public functions offering access to the master. = we could make them using a temporary master connection as well. Main points are: - hbase class for ZooKeeper; ZooKeeperWatcher is really designed for a strongly coupled architecture ;-). This can be changed, but requires a lot of modifications in these classes (likely adding a class in the middle of the hierarchy, something like that). Anyway, non connected client will always be really slower, because it's a tcp connection, and establishing a tcp connection is slow. - having a link between ZK and all the client seems to make sense for some Use Cases. However, it won't scale if a TCP connection is required for every client - if we move the table descriptor part away from the client, we need to find a new place for it. - we will have the same issue if HBaseAdmin (for both ZK Master), may be we can put a timeout on the connection. That would make the whole system less deterministic however. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5399) Cut the link between the client and the zookeeper ensemble
[ https://issues.apache.org/jira/browse/HBASE-5399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-5399: --- Attachment: 5399.v42.patch Cut the link between the client and the zookeeper ensemble -- Key: HBASE-5399 URL: https://issues.apache.org/jira/browse/HBASE-5399 Project: HBase Issue Type: Improvement Components: client Affects Versions: 0.94.0 Environment: all Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.96.0 Attachments: 5399.v27.patch, 5399.v38.patch, 5399.v39.patch, 5399.v40.patch, 5399.v41.patch, 5399.v42.patch, 5399.v42.patch, 5399_inprogress.patch, 5399_inprogress.v14.patch, 5399_inprogress.v16.patch, 5399_inprogress.v18.patch, 5399_inprogress.v20.patch, 5399_inprogress.v21.patch, 5399_inprogress.v23.patch, 5399_inprogress.v3.patch, 5399_inprogress.v32.patch, 5399_inprogress.v9.patch The link is often considered as an issue, for various reasons. One of them being that there is a limit on the number of connection that ZK can manage. Stack was suggesting as well to remove the link to master from HConnection. There are choices to be made considering the existing API (that we don't want to break). The first patches I will submit on hadoop-qa should not be committed: they are here to show the progress on the direction taken. ZooKeeper is used for: - public getter, to let the client do whatever he wants, and close ZooKeeper when closing the connection = we have to deprecate this but keep it. - read get master address to create a master = now done with a temporary zookeeper connection - read root location = now done with a temporary zookeeper connection, but questionable. Used in public function locateRegion. To be reworked. - read cluster id = now done once with a temporary zookeeper connection. - check if base done is available = now done once with a zookeeper connection given as a parameter - isTableDisabled/isTableAvailable = public functions, now done with a temporary zookeeper connection. - Called internally from HBaseAdmin and HTable - getCurrentNrHRS(): public function to get the number of region servers and create a pool of thread = now done with a temporary zookeeper connection - Master is used for: - getMaster public getter, as for ZooKeeper = we have to deprecate this but keep it. - isMasterRunning(): public function, used internally by HMerge HBaseAdmin - getHTableDescriptor*: public functions offering access to the master. = we could make them using a temporary master connection as well. Main points are: - hbase class for ZooKeeper; ZooKeeperWatcher is really designed for a strongly coupled architecture ;-). This can be changed, but requires a lot of modifications in these classes (likely adding a class in the middle of the hierarchy, something like that). Anyway, non connected client will always be really slower, because it's a tcp connection, and establishing a tcp connection is slow. - having a link between ZK and all the client seems to make sense for some Use Cases. However, it won't scale if a TCP connection is required for every client - if we move the table descriptor part away from the client, we need to find a new place for it. - we will have the same issue if HBaseAdmin (for both ZK Master), may be we can put a timeout on the connection. That would make the whole system less deterministic however. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5399) Cut the link between the client and the zookeeper ensemble
[ https://issues.apache.org/jira/browse/HBASE-5399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-5399: --- Status: Open (was: Patch Available) Cut the link between the client and the zookeeper ensemble -- Key: HBASE-5399 URL: https://issues.apache.org/jira/browse/HBASE-5399 Project: HBase Issue Type: Improvement Components: client Affects Versions: 0.94.0 Environment: all Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.96.0 Attachments: 5399.v27.patch, 5399.v38.patch, 5399.v39.patch, 5399.v40.patch, 5399.v41.patch, 5399.v42.patch, 5399.v42.patch, 5399_inprogress.patch, 5399_inprogress.v14.patch, 5399_inprogress.v16.patch, 5399_inprogress.v18.patch, 5399_inprogress.v20.patch, 5399_inprogress.v21.patch, 5399_inprogress.v23.patch, 5399_inprogress.v3.patch, 5399_inprogress.v32.patch, 5399_inprogress.v9.patch The link is often considered as an issue, for various reasons. One of them being that there is a limit on the number of connection that ZK can manage. Stack was suggesting as well to remove the link to master from HConnection. There are choices to be made considering the existing API (that we don't want to break). The first patches I will submit on hadoop-qa should not be committed: they are here to show the progress on the direction taken. ZooKeeper is used for: - public getter, to let the client do whatever he wants, and close ZooKeeper when closing the connection = we have to deprecate this but keep it. - read get master address to create a master = now done with a temporary zookeeper connection - read root location = now done with a temporary zookeeper connection, but questionable. Used in public function locateRegion. To be reworked. - read cluster id = now done once with a temporary zookeeper connection. - check if base done is available = now done once with a zookeeper connection given as a parameter - isTableDisabled/isTableAvailable = public functions, now done with a temporary zookeeper connection. - Called internally from HBaseAdmin and HTable - getCurrentNrHRS(): public function to get the number of region servers and create a pool of thread = now done with a temporary zookeeper connection - Master is used for: - getMaster public getter, as for ZooKeeper = we have to deprecate this but keep it. - isMasterRunning(): public function, used internally by HMerge HBaseAdmin - getHTableDescriptor*: public functions offering access to the master. = we could make them using a temporary master connection as well. Main points are: - hbase class for ZooKeeper; ZooKeeperWatcher is really designed for a strongly coupled architecture ;-). This can be changed, but requires a lot of modifications in these classes (likely adding a class in the middle of the hierarchy, something like that). Anyway, non connected client will always be really slower, because it's a tcp connection, and establishing a tcp connection is slow. - having a link between ZK and all the client seems to make sense for some Use Cases. However, it won't scale if a TCP connection is required for every client - if we move the table descriptor part away from the client, we need to find a new place for it. - we will have the same issue if HBaseAdmin (for both ZK Master), may be we can put a timeout on the connection. That would make the whole system less deterministic however. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-5552) Clean up our jmx view; its a bit of a mess
[ https://issues.apache.org/jira/browse/HBASE-5552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-5552. -- Resolution: Fixed Fix Version/s: 0.94.0 0.92.1 Assignee: stack Committed small patch to 0.92, 0.94 and trunk. Only person I'm messing up is Benoit I believe (he amended tsdb to read beans in new location... will talk to him) Clean up our jmx view; its a bit of a mess -- Key: HBASE-5552 URL: https://issues.apache.org/jira/browse/HBASE-5552 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Priority: Blocker Fix For: 0.92.1, 0.94.0 Attachments: 0.92.0jmx.png, 5552.txt, currentjmxview.png, patchedjmxview.png Fix before we release 0.92.1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5553) Revisit our jmx view
Revisit our jmx view Key: HBASE-5553 URL: https://issues.apache.org/jira/browse/HBASE-5553 Project: HBase Issue Type: Bug Reporter: stack Fix For: 0.96.0 See HBASE-5552 for some notes. As is, its hard to make sense of what we are publishing and its not amenable to new beans showing up -- it'll be hard to fit them in to give a nice logical jmx view. Fix. Entertain doing this at the singularity. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5325) Expose basic information about the master-status through jmx beans
[ https://issues.apache.org/jira/browse/HBASE-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226200#comment-13226200 ] stack commented on HBASE-5325: -- I took a look. Its broke (I'm to blame I'd say for it being broke). Did basic fixup and committed in hbase-5552. Made new issue to revisit our jmx view. Its always been broke. Expose basic information about the master-status through jmx beans --- Key: HBASE-5325 URL: https://issues.apache.org/jira/browse/HBASE-5325 Project: HBase Issue Type: Improvement Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Minor Fix For: 0.92.1, 0.94.0 Attachments: HBASE-5325.1.patch, HBASE-5325.2.patch, HBASE-5325.3.branch-0.92.patch, HBASE-5325.3.patch, HBASE-5325.wip.patch Similar to the Namenode and Jobtracker, it would be good if the hbase master could expose some information through mbeans. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5551) Some functions should not be used by customer code and must be deprecated in 0.94
[ https://issues.apache.org/jira/browse/HBASE-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-5551: --- Attachment: 5551.092.patch Some functions should not be used by customer code and must be deprecated in 0.94 - Key: HBASE-5551 URL: https://issues.apache.org/jira/browse/HBASE-5551 Project: HBase Issue Type: Improvement Affects Versions: 0.92.0 Reporter: nkeywal Assignee: nkeywal Fix For: 0.94.0 Attachments: 5551.092.patch They are: HBaseAdmin#getMaster HConnection#getZooKeeperWatcher HConnection#getMaster -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5551) Some functions should not be used by customer code and must be deprecated in 0.94
[ https://issues.apache.org/jira/browse/HBASE-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226204#comment-13226204 ] nkeywal commented on HBASE-5551: Patch contains only comments meta info modifications. Done on 0.94 branch. Can be committed. Some functions should not be used by customer code and must be deprecated in 0.94 - Key: HBASE-5551 URL: https://issues.apache.org/jira/browse/HBASE-5551 Project: HBase Issue Type: Improvement Affects Versions: 0.92.0 Reporter: nkeywal Assignee: nkeywal Fix For: 0.94.0 Attachments: 5551.092.patch They are: HBaseAdmin#getMaster HConnection#getZooKeeperWatcher HConnection#getMaster -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5554) hadoop.native.lib config is deprecated
hadoop.native.lib config is deprecated
Key: HBASE-5554
URL: https://issues.apache.org/jira/browse/HBASE-5554
Project: HBase
Issue Type: Task
Reporter: Zhihong Yu
When using HBase shell, we see:
{code}
12/03/09 09:06:58 WARN conf.Configuration: hadoop.native.lib is deprecated.
Instead, use io.native.lib.available
{code}
io.native.lib.available should be used.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5206) Port HBASE-5155 to 0.92 and TRUNK
[ https://issues.apache.org/jira/browse/HBASE-5206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226215#comment-13226215 ] Hadoop QA commented on HBASE-5206: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12517736/5206_trunk_1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 15 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -123 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 159 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat org.apache.hadoop.hbase.mapred.TestTableMapReduce org.apache.hadoop.hbase.TestDrainingServer org.apache.hadoop.hbase.mapreduce.TestImportTsv Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1146//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1146//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1146//console This message is automatically generated. Port HBASE-5155 to 0.92 and TRUNK - Key: HBASE-5206 URL: https://issues.apache.org/jira/browse/HBASE-5206 Project: HBase Issue Type: Bug Affects Versions: 0.92.2, 0.96.0 Reporter: Zhihong Yu Attachments: 5206_92_1.patch, 5206_trunk_1.patch This JIRA ports HBASE-5155 (ServerShutDownHandler And Disable/Delete should not happen parallely leading to recreation of regions that were deleted) to 0.92 and TRUNK -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5551) Some functions should not be used by customer code and must be deprecated in 0.94
[ https://issues.apache.org/jira/browse/HBASE-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226226#comment-13226226 ] Lars Hofhansl commented on HBASE-5551: -- Patch looks good. Should we explain alternatives in the Javadoc? Some functions should not be used by customer code and must be deprecated in 0.94 - Key: HBASE-5551 URL: https://issues.apache.org/jira/browse/HBASE-5551 Project: HBase Issue Type: Improvement Affects Versions: 0.92.0 Reporter: nkeywal Assignee: nkeywal Fix For: 0.94.0 Attachments: 5551.092.patch They are: HBaseAdmin#getMaster HConnection#getZooKeeperWatcher HConnection#getMaster -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5551) Some functions should not be used by customer code and must be deprecated in 0.94
[ https://issues.apache.org/jira/browse/HBASE-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226235#comment-13226235 ] nkeywal commented on HBASE-5551: We expect that all the functions needed are already in HTable or HBaseAdmin. We can detail this in the javadoc of course (but then first in 5399 to ensure that all the info is in the trunk for 0.96). Some functions should not be used by customer code and must be deprecated in 0.94 - Key: HBASE-5551 URL: https://issues.apache.org/jira/browse/HBASE-5551 Project: HBase Issue Type: Improvement Affects Versions: 0.92.0 Reporter: nkeywal Assignee: nkeywal Fix For: 0.94.0 Attachments: 5551.092.patch They are: HBaseAdmin#getMaster HConnection#getZooKeeperWatcher HConnection#getMaster -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5533) Add more metrics to HBase
[
https://issues.apache.org/jira/browse/HBASE-5533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226238#comment-13226238
]
stack commented on HBASE-5533:
--
Line lengths are usually 80 chars in our src code Shaneal.
Minor, assign and declare on the one line the below rather than wait till
constructor?
{code}
+this.counts = new MapMaker().makeComputingMap(new FunctionString,
Counter() {
+ @Override
+ public Counter apply(String input) {
+return new Counter();
+ }
+});
{code}
This looks cool... ExponentiallyDecayingSample
Snapshot looks excellent.
UniformSample looks sweet
On this...
{code}
+final long startTime = System.nanoTime();
{code}
... aren't we getting a currentTimeMillis soon after? Do we want to be doing
all this time getting? We should minimize as much as we can? Check it out I'd
say. Also, there is EdgeEnvironment or EnvironmentEdge that we use for system
things like getting time because we want to have a layer between us and system
especially when testing ... so we can mess things up. You might want to check
it out.
Tests look good.
How does this stuff look for a server under load? Will say tsdb be able to
make sense of it? Hows it look in the ui? I suppose I could just apply the
patch and try it but you might have some pictures laying around.
How did we make it this far w/o this stuff?
Add more metrics to HBase
-
Key: HBASE-5533
URL: https://issues.apache.org/jira/browse/HBASE-5533
Project: HBase
Issue Type: Improvement
Affects Versions: 0.92.0
Reporter: Shaneal Manek
Assignee: Shaneal Manek
Priority: Minor
Attachments: BlockingQueueContention.java, hbase-5533-0.92.patch,
hbase5533-0.92-v2.patch, hbase5533-0.92-v3.patch
To debub/monitor production clusters, there are some more metrics I wish I
had available.
In particular:
- Although the average FS latencies are useful, a 'histogram' of recent
latencies (90% of reads completed in under 100ms, 99% in under 200ms, etc)
would be more useful
- Similar histograms of latencies on common operations (GET, PUT, DELETE)
would be useful
- Counting the number of accesses to each region to detect hotspotting
- Exposing the current number of HLog files
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5533) Add more metrics to HBase
[ https://issues.apache.org/jira/browse/HBASE-5533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5533: - Affects Version/s: (was: 0.92.0) 0.94.0 0.92.2 Marking 0.92.2 because though new facility, its too good to pass up. Also marked it 0.94.0 so lars would take a look and consider it for 0.94.0. Add more metrics to HBase - Key: HBASE-5533 URL: https://issues.apache.org/jira/browse/HBASE-5533 Project: HBase Issue Type: Improvement Affects Versions: 0.92.2, 0.94.0 Reporter: Shaneal Manek Assignee: Shaneal Manek Priority: Minor Attachments: BlockingQueueContention.java, hbase-5533-0.92.patch, hbase5533-0.92-v2.patch, hbase5533-0.92-v3.patch To debub/monitor production clusters, there are some more metrics I wish I had available. In particular: - Although the average FS latencies are useful, a 'histogram' of recent latencies (90% of reads completed in under 100ms, 99% in under 200ms, etc) would be more useful - Similar histograms of latencies on common operations (GET, PUT, DELETE) would be useful - Counting the number of accesses to each region to detect hotspotting - Exposing the current number of HLog files -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5552) Clean up our jmx view; its a bit of a mess
[ https://issues.apache.org/jira/browse/HBASE-5552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226241#comment-13226241 ] Hudson commented on HBASE-5552: --- Integrated in HBase-0.94 #24 (See [https://builds.apache.org/job/HBase-0.94/24/]) HBASE-5552 Clean up our jmx view; its a bit of a mess (Revision 1298922) Result = SUCCESS stack : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPCStatistics.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/metrics/HBaseInfo.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java Clean up our jmx view; its a bit of a mess -- Key: HBASE-5552 URL: https://issues.apache.org/jira/browse/HBASE-5552 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Priority: Blocker Fix For: 0.92.1, 0.94.0 Attachments: 0.92.0jmx.png, 5552.txt, currentjmxview.png, patchedjmxview.png Fix before we release 0.92.1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5399) Cut the link between the client and the zookeeper ensemble
[ https://issues.apache.org/jira/browse/HBASE-5399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226245#comment-13226245 ] Hadoop QA commented on HBASE-5399: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12517743/5399.v42.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 30 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -123 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 160 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.mapreduce.TestImportTsv org.apache.hadoop.hbase.mapred.TestTableMapReduce org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1147//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1147//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1147//console This message is automatically generated. Cut the link between the client and the zookeeper ensemble -- Key: HBASE-5399 URL: https://issues.apache.org/jira/browse/HBASE-5399 Project: HBase Issue Type: Improvement Components: client Affects Versions: 0.94.0 Environment: all Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.96.0 Attachments: 5399.v27.patch, 5399.v38.patch, 5399.v39.patch, 5399.v40.patch, 5399.v41.patch, 5399.v42.patch, 5399.v42.patch, 5399_inprogress.patch, 5399_inprogress.v14.patch, 5399_inprogress.v16.patch, 5399_inprogress.v18.patch, 5399_inprogress.v20.patch, 5399_inprogress.v21.patch, 5399_inprogress.v23.patch, 5399_inprogress.v3.patch, 5399_inprogress.v32.patch, 5399_inprogress.v9.patch The link is often considered as an issue, for various reasons. One of them being that there is a limit on the number of connection that ZK can manage. Stack was suggesting as well to remove the link to master from HConnection. There are choices to be made considering the existing API (that we don't want to break). The first patches I will submit on hadoop-qa should not be committed: they are here to show the progress on the direction taken. ZooKeeper is used for: - public getter, to let the client do whatever he wants, and close ZooKeeper when closing the connection = we have to deprecate this but keep it. - read get master address to create a master = now done with a temporary zookeeper connection - read root location = now done with a temporary zookeeper connection, but questionable. Used in public function locateRegion. To be reworked. - read cluster id = now done once with a temporary zookeeper connection. - check if base done is available = now done once with a zookeeper connection given as a parameter - isTableDisabled/isTableAvailable = public functions, now done with a temporary zookeeper connection. - Called internally from HBaseAdmin and HTable - getCurrentNrHRS(): public function to get the number of region servers and create a pool of thread = now done with a temporary zookeeper connection - Master is used for: - getMaster public getter, as for ZooKeeper = we have to deprecate this but keep it. - isMasterRunning(): public function, used internally by HMerge HBaseAdmin - getHTableDescriptor*: public functions offering access to the master. = we could make them using a temporary master connection as well. Main points are: - hbase class for ZooKeeper; ZooKeeperWatcher is really designed for a strongly coupled architecture ;-). This can be changed, but requires a lot of modifications in these classes (likely adding a class in the middle of the hierarchy, something like that). Anyway, non connected client will always be really slower, because it's a tcp connection, and establishing a tcp connection is slow. - having a link between ZK and all the client seems to make sense for some Use Cases. However, it won't scale if a TCP connection is required for every client - if we move the table descriptor part away from the client, we need to find a new place for it. - we will have the same issue if HBaseAdmin (for both ZK Master), may be we can put a timeout on the connection. That would make the whole system
[jira] [Commented] (HBASE-5552) Clean up our jmx view; its a bit of a mess
[ https://issues.apache.org/jira/browse/HBASE-5552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226249#comment-13226249 ] Todd Lipcon commented on HBASE-5552: Changing the nesting of beans in 0.92 and 0.94 seems like an incompatible change. I'm pretty sure it will screw up our monitoring upon upgrade (Benoit isn't the only one who uses JMX!) Is it possible to have a conf to switch this? Otherwise, should defer to 96. Clean up our jmx view; its a bit of a mess -- Key: HBASE-5552 URL: https://issues.apache.org/jira/browse/HBASE-5552 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Priority: Blocker Fix For: 0.92.1, 0.94.0 Attachments: 0.92.0jmx.png, 5552.txt, currentjmxview.png, patchedjmxview.png Fix before we release 0.92.1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-5552) Clean up our jmx view; its a bit of a mess
[ https://issues.apache.org/jira/browse/HBASE-5552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon reopened HBASE-5552: Clean up our jmx view; its a bit of a mess -- Key: HBASE-5552 URL: https://issues.apache.org/jira/browse/HBASE-5552 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Priority: Blocker Fix For: 0.92.1, 0.94.0 Attachments: 0.92.0jmx.png, 5552.txt, currentjmxview.png, patchedjmxview.png Fix before we release 0.92.1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5552) Clean up our jmx view; its a bit of a mess
[ https://issues.apache.org/jira/browse/HBASE-5552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated HBASE-5552: --- Hadoop Flags: Incompatible change Clean up our jmx view; its a bit of a mess -- Key: HBASE-5552 URL: https://issues.apache.org/jira/browse/HBASE-5552 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Priority: Blocker Fix For: 0.92.1, 0.94.0 Attachments: 0.92.0jmx.png, 5552.txt, currentjmxview.png, patchedjmxview.png Fix before we release 0.92.1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5542) Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()
[ https://issues.apache.org/jira/browse/HBASE-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HBASE-5542: --- Attachment: HBASE-5542.D2217.2.patch sc updated the revision HBASE-5542 [jira] Unify HRegion.mutateRowsWithLocks() and HRegion.processRow(). Reviewers: tedyu, lhofhansl, JIRA Addressed Ted's review comments, Thanks. Sorry for dumping the RowProcessor API almost immediately. After created MultiRowProcessor, I figured out that we don't need the RowProcessor interface anymore. Better to get ride of it earlier. REVISION DETAIL https://reviews.facebook.net/D2217 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java src/main/java/org/apache/hadoop/hbase/regionserver/MultiRowMutationProcessor.java src/main/java/org/apache/hadoop/hbase/regionserver/MultiRowProcessor.java src/main/java/org/apache/hadoop/hbase/coprocessor/RowProcessor.java src/test/java/org/apache/hadoop/hbase/coprocessor/TestProcessRowEndpoint.java Unify HRegion.mutateRowsWithLocks() and HRegion.processRow() Key: HBASE-5542 URL: https://issues.apache.org/jira/browse/HBASE-5542 Project: HBase Issue Type: Improvement Reporter: Scott Chen Assignee: Scott Chen Attachments: HBASE-5542.D2217.1.patch, HBASE-5542.D2217.2.patch mutateRowsWithLocks() does atomic mutations on multiple rows. processRow() does atomic read-modify-writes on a single row. It will be useful to generalize both and have a processRowsWithLocks() that does atomic read-modify-writes on multiple rows. This also helps reduce some redundancy in the codes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5206) Port HBASE-5155 to 0.92 and TRUNK
[
https://issues.apache.org/jira/browse/HBASE-5206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226268#comment-13226268
]
Zhihong Yu commented on HBASE-5206:
---
{code}
+ for(HRegionInfo hri : allRegions.keySet()){
+setEnabledTable(hri);
+ }
{code}
Each setEnabledTable() call results in round-trip to zookeeper.
Can we remember the tables we have enabled so that the number of calls to
setEnabledTable() is reduced ?
For formatting, please insert space between for and (, between ) and {.
Port HBASE-5155 to 0.92 and TRUNK
-
Key: HBASE-5206
URL: https://issues.apache.org/jira/browse/HBASE-5206
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.2, 0.96.0
Reporter: Zhihong Yu
Attachments: 5206_92_1.patch, 5206_trunk_1.patch
This JIRA ports HBASE-5155 (ServerShutDownHandler And Disable/Delete should
not happen parallely leading to recreation of regions that were deleted) to
0.92 and TRUNK
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5542) Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()
[ https://issues.apache.org/jira/browse/HBASE-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226280#comment-13226280 ] Phabricator commented on HBASE-5542: tedyu has accepted the revision HBASE-5542 [jira] Unify HRegion.mutateRowsWithLocks() and HRegion.processRow(). If tests pass. INLINE COMMENTS src/main/java/org/apache/hadoop/hbase/regionserver/MultiRowProcessor.java:31 Should we mention that the rows are inside one region ? REVISION DETAIL https://reviews.facebook.net/D2217 BRANCH 5542-new-file Unify HRegion.mutateRowsWithLocks() and HRegion.processRow() Key: HBASE-5542 URL: https://issues.apache.org/jira/browse/HBASE-5542 Project: HBase Issue Type: Improvement Reporter: Scott Chen Assignee: Scott Chen Attachments: HBASE-5542.D2217.1.patch, HBASE-5542.D2217.2.patch mutateRowsWithLocks() does atomic mutations on multiple rows. processRow() does atomic read-modify-writes on a single row. It will be useful to generalize both and have a processRowsWithLocks() that does atomic read-modify-writes on multiple rows. This also helps reduce some redundancy in the codes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4608) HLog Compression
[ https://issues.apache.org/jira/browse/HBASE-4608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226283#comment-13226283 ] Zhihong Yu commented on HBASE-4608: --- I performed the above procedure again by using two config objects at the beginning of transformFile(). I got same result. HLog Compression Key: HBASE-4608 URL: https://issues.apache.org/jira/browse/HBASE-4608 Project: HBase Issue Type: New Feature Reporter: Li Pi Assignee: Li Pi Fix For: 0.94.0 Attachments: 4608-v19.txt, 4608v1.txt, 4608v13.txt, 4608v13.txt, 4608v14.txt, 4608v15.txt, 4608v16.txt, 4608v17.txt, 4608v18.txt, 4608v5.txt, 4608v6.txt, 4608v7.txt, 4608v8fixed.txt The current bottleneck to HBase write speed is replicating the WAL appends across different datanodes. We can speed up this process by compressing the HLog. Current plan involves using a dictionary to compress table name, region id, cf name, and possibly other bits of repeated data. Also, HLog format may be changed in other ways to produce a smaller HLog. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5520) Support reseek() at RegionScanner
[
https://issues.apache.org/jira/browse/HBASE-5520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226290#comment-13226290
]
Zhihong Yu commented on HBASE-5520:
---
Compilation failed:
{code}
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java:[54,17]
org.apache.hadoop.hbase.coprocessor.TestCoprocessorInterface.CustomScanner is
not abstract and does not override abstract method
reseek(org.apache.hadoop.hbase.KeyValue) in
org.apache.hadoop.hbase.regionserver.RegionScanner
{code}
Support reseek() at RegionScanner
-
Key: HBASE-5520
URL: https://issues.apache.org/jira/browse/HBASE-5520
Project: HBase
Issue Type: Improvement
Components: regionserver
Affects Versions: 0.92.0
Reporter: Anoop Sam John
Assignee: ramkrishna.s.vasudevan
Fix For: 0.96.0
Attachments: HBASE-5520_1.patch, HBASE-5520_2.patch
reseek() is not supported currently at the RegionScanner level. We can
support the same.
This is created following the discussion under HBASE-2038
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5552) Clean up our jmx view; its a bit of a mess
[ https://issues.apache.org/jira/browse/HBASE-5552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226296#comment-13226296 ] stack commented on HBASE-5552: -- Its not incompatible change Todd because we've not had a release w/ these new mbeans yet. On the rename, yeah, over in 'HBASE-5553 Revisit our jmx view', it suggests we'd have to wait till the singularity. Clean up our jmx view; its a bit of a mess -- Key: HBASE-5552 URL: https://issues.apache.org/jira/browse/HBASE-5552 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Priority: Blocker Fix For: 0.92.1, 0.94.0 Attachments: 0.92.0jmx.png, 5552.txt, currentjmxview.png, patchedjmxview.png Fix before we release 0.92.1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4818) HBase Shell - Add support for formatting row keys before output
[
https://issues.apache.org/jira/browse/HBASE-4818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ben West updated HBASE-4818:
Attachment: hbase-4818.patch
Attaching a patch which includes:
1. The ability to specify a custom formatter on the command line
2. A sample custom formatter which reverses the keys before printing them in a
scan
You can use the new formatter by doing {code}
hbase shell --format=Shell::Formatter::ReverseID.new
{code}
We have an existing shell variable (JRUBY_OPTS) which you can set in your
config script to persist your options, as Lars suggested. I'm not sure how to
implement Eran's suggestion of per-table formatters using the command line;
maybe we should deprecate the command line option since it doesn't do anything
anyway and store this in .irbrc.
Also, the reverse ID formatter works by a kind of hack.
I'd like to hear from people more familiar with the shell on how to make this
better.
HBase Shell - Add support for formatting row keys before output
---
Key: HBASE-4818
URL: https://issues.apache.org/jira/browse/HBASE-4818
Project: HBase
Issue Type: Improvement
Components: shell
Reporter: Eran Kampf
Priority: Trivial
Attachments: hbase-4818.patch
Original Estimate: 24h
Remaining Estimate: 24h
As many HBase users use binary row keys rather than strings to optimize
memory consumption displaying an escaped string in the HBase shell isn't
useful (and takes a lot of screen space)
Allowing user to provide a row key formatter as part of the scan\get commands
would allow developers to display the row key in a way thats makes sense for
them.
Example:
scan 'stats', { ROWFORMATTER = MyRowFormatter.new }
The row formatter simply gets the bytes array key and formats it to a string.
Its an easy change tomake with simple monkey-patching of the shell commands
but I would be happy to see it as part of the shell itself.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5552) Clean up our jmx view; its a bit of a mess
[ https://issues.apache.org/jira/browse/HBASE-5552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226308#comment-13226308 ] Todd Lipcon commented on HBASE-5552: Oh, does this not affect the other metrics published at /jmx in the servlet? Sorry for confusion. Clean up our jmx view; its a bit of a mess -- Key: HBASE-5552 URL: https://issues.apache.org/jira/browse/HBASE-5552 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Priority: Blocker Fix For: 0.92.1, 0.94.0 Attachments: 0.92.0jmx.png, 5552.txt, currentjmxview.png, patchedjmxview.png Fix before we release 0.92.1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5552) Clean up our jmx view; its a bit of a mess
[ https://issues.apache.org/jira/browse/HBASE-5552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226309#comment-13226309 ] Hudson commented on HBASE-5552: --- Integrated in HBase-0.92 #323 (See [https://builds.apache.org/job/HBase-0.92/323/]) HBASE-5552 Clean up our jmx view; its a bit of a mess (Revision 1298924) HBASE-5552 Clean up our jmx view; its a bit of a mess (Revision 1298919) Result = SUCCESS stack : Files : * /hbase/branches/0.92/CHANGES.txt stack : Files : * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPCStatistics.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/metrics/HBaseInfo.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java Clean up our jmx view; its a bit of a mess -- Key: HBASE-5552 URL: https://issues.apache.org/jira/browse/HBASE-5552 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Priority: Blocker Fix For: 0.92.1, 0.94.0 Attachments: 0.92.0jmx.png, 5552.txt, currentjmxview.png, patchedjmxview.png Fix before we release 0.92.1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5542) Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()
[ https://issues.apache.org/jira/browse/HBASE-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5542: -- Hadoop Flags: Reviewed Status: Patch Available (was: Open) Unify HRegion.mutateRowsWithLocks() and HRegion.processRow() Key: HBASE-5542 URL: https://issues.apache.org/jira/browse/HBASE-5542 Project: HBase Issue Type: Improvement Reporter: Scott Chen Assignee: Scott Chen Attachments: HBASE-5542.D2217.1.patch, HBASE-5542.D2217.2.patch mutateRowsWithLocks() does atomic mutations on multiple rows. processRow() does atomic read-modify-writes on a single row. It will be useful to generalize both and have a processRowsWithLocks() that does atomic read-modify-writes on multiple rows. This also helps reduce some redundancy in the codes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5542) Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()
[ https://issues.apache.org/jira/browse/HBASE-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5542: -- Fix Version/s: 0.96.0 Unify HRegion.mutateRowsWithLocks() and HRegion.processRow() Key: HBASE-5542 URL: https://issues.apache.org/jira/browse/HBASE-5542 Project: HBase Issue Type: Improvement Reporter: Scott Chen Assignee: Scott Chen Fix For: 0.96.0 Attachments: HBASE-5542.D2217.1.patch, HBASE-5542.D2217.2.patch mutateRowsWithLocks() does atomic mutations on multiple rows. processRow() does atomic read-modify-writes on a single row. It will be useful to generalize both and have a processRowsWithLocks() that does atomic read-modify-writes on multiple rows. This also helps reduce some redundancy in the codes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4818) HBase Shell - Add support for formatting row keys before output
[
https://issues.apache.org/jira/browse/HBASE-4818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226311#comment-13226311
]
Todd Lipcon commented on HBASE-4818:
I think this should be a table property, and refer to a Java class name, rather
than doing it in ruby. Doing it in ruby only helps with shell, but doing it in
Java means we can also use it in the UIs, etc. ACCUMULO-303 is helpful
reference material.
HBase Shell - Add support for formatting row keys before output
---
Key: HBASE-4818
URL: https://issues.apache.org/jira/browse/HBASE-4818
Project: HBase
Issue Type: Improvement
Components: shell
Reporter: Eran Kampf
Priority: Trivial
Attachments: hbase-4818.patch
Original Estimate: 24h
Remaining Estimate: 24h
As many HBase users use binary row keys rather than strings to optimize
memory consumption displaying an escaped string in the HBase shell isn't
useful (and takes a lot of screen space)
Allowing user to provide a row key formatter as part of the scan\get commands
would allow developers to display the row key in a way thats makes sense for
them.
Example:
scan 'stats', { ROWFORMATTER = MyRowFormatter.new }
The row formatter simply gets the bytes array key and formats it to a string.
Its an easy change tomake with simple monkey-patching of the shell commands
but I would be happy to see it as part of the shell itself.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5542) Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()
[ https://issues.apache.org/jira/browse/HBASE-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226316#comment-13226316 ] Hadoop QA commented on HBASE-5542: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12517751/HBASE-5542.D2217.2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1148//console This message is automatically generated. Unify HRegion.mutateRowsWithLocks() and HRegion.processRow() Key: HBASE-5542 URL: https://issues.apache.org/jira/browse/HBASE-5542 Project: HBase Issue Type: Improvement Reporter: Scott Chen Assignee: Scott Chen Fix For: 0.96.0 Attachments: HBASE-5542.D2217.1.patch, HBASE-5542.D2217.2.patch mutateRowsWithLocks() does atomic mutations on multiple rows. processRow() does atomic read-modify-writes on a single row. It will be useful to generalize both and have a processRowsWithLocks() that does atomic read-modify-writes on multiple rows. This also helps reduce some redundancy in the codes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5355) Compressed RPC's for HBase
[ https://issues.apache.org/jira/browse/HBASE-5355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226323#comment-13226323 ] dhruba borthakur commented on HBASE-5355: - From what I understand, the aim of this jira is not to reduce latency of an RPC, but rather to reduce the network bandwidth usage. Ryan's comments makes sense: it is possible that in your setup the additional time taken to compress/decompress adds additional latency for the rpc completion times; but if you enable this on a cluster where the network bandwidth is the bottleneck, then there could be a net-gain in the rpc latency. Todd/Karthik: do you think that the prefix encoding would be a good candidate for this one? Compressed RPC's for HBase -- Key: HBASE-5355 URL: https://issues.apache.org/jira/browse/HBASE-5355 Project: HBase Issue Type: Improvement Components: ipc Affects Versions: 0.89.20100924 Reporter: Karthik Ranganathan Assignee: Karthik Ranganathan Some application need ability to do large batched writes and reads from a remote MR cluster. These eventually get bottlenecked on the network. These results are also pretty compressible sometimes. The aim here is to add the ability to do compressed calls to the server on both the send and receive paths. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5542) Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()
[ https://issues.apache.org/jira/browse/HBASE-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226324#comment-13226324 ] Lars Hofhansl commented on HBASE-5542: -- Let me do a pass through it before you commit. Unify HRegion.mutateRowsWithLocks() and HRegion.processRow() Key: HBASE-5542 URL: https://issues.apache.org/jira/browse/HBASE-5542 Project: HBase Issue Type: Improvement Reporter: Scott Chen Assignee: Scott Chen Fix For: 0.96.0 Attachments: HBASE-5542.D2217.1.patch, HBASE-5542.D2217.2.patch mutateRowsWithLocks() does atomic mutations on multiple rows. processRow() does atomic read-modify-writes on a single row. It will be useful to generalize both and have a processRowsWithLocks() that does atomic read-modify-writes on multiple rows. This also helps reduce some redundancy in the codes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4542) add filter info to slow query logging
[
https://issues.apache.org/jira/browse/HBASE-4542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226335#comment-13226335
]
Phabricator commented on HBASE-4542:
mbautin has accepted the revision [jira] [HBASE-4542] Add filter info to slow
query logging.
REVISION DETAIL
https://reviews.facebook.net/D1539
BRANCH
filterLog
add filter info to slow query logging
-
Key: HBASE-4542
URL: https://issues.apache.org/jira/browse/HBASE-4542
Project: HBase
Issue Type: Improvement
Affects Versions: 0.89.20100924
Reporter: Kannan Muthukkaruppan
Assignee: Madhuwanti Vaidya
Attachments:
0001-jira-HBASE-4542-Add-filter-info-to-slow-query-loggin.patch,
Add-filter-info-to-slow-query-logging-2012-03-06_14_28_13.patch,
D1263.2.patch, D1539.1.patch
Slow query log doesn't report filters in effect.
For example:
{code}
(operationTooSlow): \
{processingtimems:3468,client:10.138.43.206:40035,timeRange:
[0,9223372036854775807],\
starttimems:1317772005821,responsesize:42411, \
class:HRegionServer,table:myTable,families:{CF1:ALL]},\
row:6c3b8efa132f0219b7621ed1e5c8c70b,queuetimems:0,\
method:get,totalColumns:1,maxVersions:1,storeLimit:-1}
{code}
the above would suggest that all columns of myTable:CF1 are being requested
for the given row. But in reality there could be filters in effect (such as
ColumnPrefixFilter, ColumnRangeFilter, TimestampsFilter() etc.). We should
enhance the slow query log to capture report this information.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5552) Clean up our jmx view; its a bit of a mess
[ https://issues.apache.org/jira/browse/HBASE-5552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226341#comment-13226341 ] stack commented on HBASE-5552: -- bq. Oh, does this not affect the other metrics published at /jmx in the servlet? Sorry for confusion. It should, but it doesn't (smile). Its just a renaming of the beans added by HBASE-5325 (0.92.1 is their first airing in a release). I should have been more clear. Clean up our jmx view; its a bit of a mess -- Key: HBASE-5552 URL: https://issues.apache.org/jira/browse/HBASE-5552 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Priority: Blocker Fix For: 0.92.1, 0.94.0 Attachments: 0.92.0jmx.png, 5552.txt, currentjmxview.png, patchedjmxview.png Fix before we release 0.92.1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4818) HBase Shell - Add support for formatting row keys before output
[
https://issues.apache.org/jira/browse/HBASE-4818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226347#comment-13226347
]
Ben West commented on HBASE-4818:
-
Todd: I think since we're using JRuby the formatter can be a java class, right?
You'd just have --format=org.apache
But I guess we could store it as a table property.
(Btw: if the formatters are to be useful outside of shell, we'll need a revamp
of how they work. Right now, it just formats text without much knowledge of
what the text is - we'd probably want to have FormatKey() FormatColumn() etc.
methods. Which is a good idea anyway.)
HBase Shell - Add support for formatting row keys before output
---
Key: HBASE-4818
URL: https://issues.apache.org/jira/browse/HBASE-4818
Project: HBase
Issue Type: Improvement
Components: shell
Reporter: Eran Kampf
Priority: Trivial
Attachments: hbase-4818.patch
Original Estimate: 24h
Remaining Estimate: 24h
As many HBase users use binary row keys rather than strings to optimize
memory consumption displaying an escaped string in the HBase shell isn't
useful (and takes a lot of screen space)
Allowing user to provide a row key formatter as part of the scan\get commands
would allow developers to display the row key in a way thats makes sense for
them.
Example:
scan 'stats', { ROWFORMATTER = MyRowFormatter.new }
The row formatter simply gets the bytes array key and formats it to a string.
Its an easy change tomake with simple monkey-patching of the shell commands
but I would be happy to see it as part of the shell itself.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5313) Restructure hfiles layout for better compression
[ https://issues.apache.org/jira/browse/HBASE-5313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226351#comment-13226351 ] dhruba borthakur commented on HBASE-5313: - I am guessing that initially, we keep this new columnar encoding completely isolated inside a HFileBlock. At table creation time, one can specify that the table be stored in columnar-encoded fashion. A HFile will have information in the FixedFileTrailer that specifies whether the data inside the hfile is in columnar-format. A single HFileBlock will have four column-entity: all the rowkeys will be laid out first, followed by all the cf, followed by all the column names, followed by the timestamps, followed by the memstoreTS, followed by all the values. If 'prefix-encoding' is enabled, then each column-entity will be prefix encoded individually. If compression (lzo, gz, etc) is enabled, the entire HFileBlock will be compressed accordingly. Prefix-encoding will work well for the rowkey entity and the column-family entity. The column name entity may need to be sorted and then prefix encoded. The timestamp entity may need special kind of encoding. One option (suggested by a co-worker Chip Turner) is to take the first timestamp as the base and xor it with each of the following timestamps (thus, zeroing out the common bits) and then storing it. Another option is to find the minimum timestamp in the block and then store diffs from that minimum value. Yet another option is to make Jan-01-2012 as the hbase-epoch and store the difference from that number. Restructure hfiles layout for better compression Key: HBASE-5313 URL: https://issues.apache.org/jira/browse/HBASE-5313 Project: HBase Issue Type: Improvement Components: io Reporter: dhruba borthakur Assignee: dhruba borthakur A HFile block contain a stream of key-values. Can we can organize these kvs on the disk in a better way so that we get much greater compression ratios? One option (thanks Prakash) is to store all the keys in the beginning of the block (let's call this the key-section) and then store all their corresponding values towards the end of the block. This will allow us to not-even decompress the values when we are scanning and skipping over rows in the block. Any other ideas? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5213) hbase master stop does not bring down backup masters
[ https://issues.apache.org/jira/browse/HBASE-5213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226353#comment-13226353 ] Gregory Chanan commented on HBASE-5213: --- Any thoughts regarding 92 or 90? Thanks for the reviews/commit. hbase master stop does not bring down backup masters -- Key: HBASE-5213 URL: https://issues.apache.org/jira/browse/HBASE-5213 Project: HBase Issue Type: Bug Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0 Reporter: Gregory Chanan Assignee: Gregory Chanan Priority: Minor Attachments: HBASE-5213-v0-trunk.patch, HBASE-5213-v1-trunk.patch, HBASE-5213-v2-90.patch, HBASE-5213-v2-92.patch, HBASE-5213-v2-trunk.patch Typing hbase master stop produces the following message: stop Start cluster shutdown; Master signals RegionServer shutdown It seems like backup masters should be considered part of the cluster, but they are not brought down by hbase master stop. stop-hbase.sh does correctly bring down the backup masters. The same behavior is observed when a client app makes use of the client API HBaseAdmin.shutdown() http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HBaseAdmin.html#shutdown() -- this isn't too surprising since I think hbase master stop just calls this API. It seems like HBASE-1448 address this; perhaps there was a regression? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4542) add filter info to slow query logging
[
https://issues.apache.org/jira/browse/HBASE-4542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226356#comment-13226356
]
stack commented on HBASE-4542:
--
+1 on commit (Thanks for taking care of the this Mikhail)
add filter info to slow query logging
-
Key: HBASE-4542
URL: https://issues.apache.org/jira/browse/HBASE-4542
Project: HBase
Issue Type: Improvement
Affects Versions: 0.89.20100924
Reporter: Kannan Muthukkaruppan
Assignee: Madhuwanti Vaidya
Attachments:
0001-jira-HBASE-4542-Add-filter-info-to-slow-query-loggin.patch,
Add-filter-info-to-slow-query-logging-2012-03-06_14_28_13.patch,
D1263.2.patch, D1539.1.patch
Slow query log doesn't report filters in effect.
For example:
{code}
(operationTooSlow): \
{processingtimems:3468,client:10.138.43.206:40035,timeRange:
[0,9223372036854775807],\
starttimems:1317772005821,responsesize:42411, \
class:HRegionServer,table:myTable,families:{CF1:ALL]},\
row:6c3b8efa132f0219b7621ed1e5c8c70b,queuetimems:0,\
method:get,totalColumns:1,maxVersions:1,storeLimit:-1}
{code}
the above would suggest that all columns of myTable:CF1 are being requested
for the given row. But in reality there could be filters in effect (such as
ColumnPrefixFilter, ColumnRangeFilter, TimestampsFilter() etc.). We should
enhance the slow query log to capture report this information.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5213) hbase master stop does not bring down backup masters
[ https://issues.apache.org/jira/browse/HBASE-5213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5213: - Fix Version/s: 0.90.2 Marking as for 0.92.2 (of if a new 0.92.1 RC, will include it in that) hbase master stop does not bring down backup masters -- Key: HBASE-5213 URL: https://issues.apache.org/jira/browse/HBASE-5213 Project: HBase Issue Type: Bug Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0 Reporter: Gregory Chanan Assignee: Gregory Chanan Priority: Minor Fix For: 0.90.2 Attachments: HBASE-5213-v0-trunk.patch, HBASE-5213-v1-trunk.patch, HBASE-5213-v2-90.patch, HBASE-5213-v2-92.patch, HBASE-5213-v2-trunk.patch Typing hbase master stop produces the following message: stop Start cluster shutdown; Master signals RegionServer shutdown It seems like backup masters should be considered part of the cluster, but they are not brought down by hbase master stop. stop-hbase.sh does correctly bring down the backup masters. The same behavior is observed when a client app makes use of the client API HBaseAdmin.shutdown() http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HBaseAdmin.html#shutdown() -- this isn't too surprising since I think hbase master stop just calls this API. It seems like HBASE-1448 address this; perhaps there was a regression? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (HBASE-5542) Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()
[ https://issues.apache.org/jira/browse/HBASE-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226368#comment-13226368 ] Lars Hofhansl edited comment on HBASE-5542 at 3/9/12 7:35 PM: -- Can we let this sit for a bit? I would like to spend some time thinking about this. doMiniBatchPut almost follows the same logic, should be able to unify that code as well. It's not needed for 0.94, so it's not time critical. Thanks for working on this Scott! was (Author: lhofhansl): Can we let this sit for a bit. I would like to spend some time thinking about this. doMiniBatchPut almost follows the same logic, should be able to unify that code as well. It's not needed for 0.94, so it's not time critical. Thanks for working on this Scott! Unify HRegion.mutateRowsWithLocks() and HRegion.processRow() Key: HBASE-5542 URL: https://issues.apache.org/jira/browse/HBASE-5542 Project: HBase Issue Type: Improvement Reporter: Scott Chen Assignee: Scott Chen Fix For: 0.96.0 Attachments: HBASE-5542.D2217.1.patch, HBASE-5542.D2217.2.patch mutateRowsWithLocks() does atomic mutations on multiple rows. processRow() does atomic read-modify-writes on a single row. It will be useful to generalize both and have a processRowsWithLocks() that does atomic read-modify-writes on multiple rows. This also helps reduce some redundancy in the codes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5542) Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()
[ https://issues.apache.org/jira/browse/HBASE-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226368#comment-13226368 ] Lars Hofhansl commented on HBASE-5542: -- Can we let this sit for a bit. I would like to spend some time thinking about this. doMiniBatchPut almost follows the same logic, should be able to unify that code as well. It's not needed for 0.94, so it's not time critical. Thanks for working on this Scott! Unify HRegion.mutateRowsWithLocks() and HRegion.processRow() Key: HBASE-5542 URL: https://issues.apache.org/jira/browse/HBASE-5542 Project: HBase Issue Type: Improvement Reporter: Scott Chen Assignee: Scott Chen Fix For: 0.96.0 Attachments: HBASE-5542.D2217.1.patch, HBASE-5542.D2217.2.patch mutateRowsWithLocks() does atomic mutations on multiple rows. processRow() does atomic read-modify-writes on a single row. It will be useful to generalize both and have a processRowsWithLocks() that does atomic read-modify-writes on multiple rows. This also helps reduce some redundancy in the codes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5542) Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()
[ https://issues.apache.org/jira/browse/HBASE-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226379#comment-13226379 ] stack commented on HBASE-5542: -- I took a look. Patch looks good. I think this kinda work is really important to do -- now while its fresh in everyone's minds whats going on here -- if we're not to let the mess get out of hand. Thanks Scott. Yeah Lars, this is your old stomping ground. Could do w/ a review by you. Unify HRegion.mutateRowsWithLocks() and HRegion.processRow() Key: HBASE-5542 URL: https://issues.apache.org/jira/browse/HBASE-5542 Project: HBase Issue Type: Improvement Reporter: Scott Chen Assignee: Scott Chen Fix For: 0.96.0 Attachments: HBASE-5542.D2217.1.patch, HBASE-5542.D2217.2.patch mutateRowsWithLocks() does atomic mutations on multiple rows. processRow() does atomic read-modify-writes on a single row. It will be useful to generalize both and have a processRowsWithLocks() that does atomic read-modify-writes on multiple rows. This also helps reduce some redundancy in the codes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5213) hbase master stop does not bring down backup masters
[ https://issues.apache.org/jira/browse/HBASE-5213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gregory Chanan updated HBASE-5213: -- Fix Version/s: (was: 0.90.2) 0.92.2 @stack: You wrote 0.92.2 but marked as 0.90.2. Changed to 0.92.2 hbase master stop does not bring down backup masters -- Key: HBASE-5213 URL: https://issues.apache.org/jira/browse/HBASE-5213 Project: HBase Issue Type: Bug Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0 Reporter: Gregory Chanan Assignee: Gregory Chanan Priority: Minor Fix For: 0.92.2 Attachments: HBASE-5213-v0-trunk.patch, HBASE-5213-v1-trunk.patch, HBASE-5213-v2-90.patch, HBASE-5213-v2-92.patch, HBASE-5213-v2-trunk.patch Typing hbase master stop produces the following message: stop Start cluster shutdown; Master signals RegionServer shutdown It seems like backup masters should be considered part of the cluster, but they are not brought down by hbase master stop. stop-hbase.sh does correctly bring down the backup masters. The same behavior is observed when a client app makes use of the client API HBaseAdmin.shutdown() http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HBaseAdmin.html#shutdown() -- this isn't too surprising since I think hbase master stop just calls this API. It seems like HBASE-1448 address this; perhaps there was a regression? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5213) hbase master stop does not bring down backup masters
[ https://issues.apache.org/jira/browse/HBASE-5213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226385#comment-13226385 ] stack commented on HBASE-5213: -- Thanks G. hbase master stop does not bring down backup masters -- Key: HBASE-5213 URL: https://issues.apache.org/jira/browse/HBASE-5213 Project: HBase Issue Type: Bug Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0 Reporter: Gregory Chanan Assignee: Gregory Chanan Priority: Minor Fix For: 0.92.2 Attachments: HBASE-5213-v0-trunk.patch, HBASE-5213-v1-trunk.patch, HBASE-5213-v2-90.patch, HBASE-5213-v2-92.patch, HBASE-5213-v2-trunk.patch Typing hbase master stop produces the following message: stop Start cluster shutdown; Master signals RegionServer shutdown It seems like backup masters should be considered part of the cluster, but they are not brought down by hbase master stop. stop-hbase.sh does correctly bring down the backup masters. The same behavior is observed when a client app makes use of the client API HBaseAdmin.shutdown() http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HBaseAdmin.html#shutdown() -- this isn't too surprising since I think hbase master stop just calls this API. It seems like HBASE-1448 address this; perhaps there was a regression? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5542) Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()
[ https://issues.apache.org/jira/browse/HBASE-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226386#comment-13226386 ] Scott Chen commented on HBASE-5542: --- @Ted: Thanks for the help. I will rebase this one right away. @Lars: I just see your work in HBASE-5541. That's pretty nice. I will double check with it to make sure I am also doing the right thing in this one. Thanks! Unify HRegion.mutateRowsWithLocks() and HRegion.processRow() Key: HBASE-5542 URL: https://issues.apache.org/jira/browse/HBASE-5542 Project: HBase Issue Type: Improvement Reporter: Scott Chen Assignee: Scott Chen Fix For: 0.96.0 Attachments: HBASE-5542.D2217.1.patch, HBASE-5542.D2217.2.patch mutateRowsWithLocks() does atomic mutations on multiple rows. processRow() does atomic read-modify-writes on a single row. It will be useful to generalize both and have a processRowsWithLocks() that does atomic read-modify-writes on multiple rows. This also helps reduce some redundancy in the codes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5542) Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()
[ https://issues.apache.org/jira/browse/HBASE-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226387#comment-13226387 ] Scott Chen commented on HBASE-5542: --- @Stack: Thanks for the review! We will get this done :) Unify HRegion.mutateRowsWithLocks() and HRegion.processRow() Key: HBASE-5542 URL: https://issues.apache.org/jira/browse/HBASE-5542 Project: HBase Issue Type: Improvement Reporter: Scott Chen Assignee: Scott Chen Fix For: 0.96.0 Attachments: HBASE-5542.D2217.1.patch, HBASE-5542.D2217.2.patch mutateRowsWithLocks() does atomic mutations on multiple rows. processRow() does atomic read-modify-writes on a single row. It will be useful to generalize both and have a processRowsWithLocks() that does atomic read-modify-writes on multiple rows. This also helps reduce some redundancy in the codes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4542) add filter info to slow query logging
[
https://issues.apache.org/jira/browse/HBASE-4542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226408#comment-13226408
]
Phabricator commented on HBASE-4542:
mbautin has committed the revision [jira] [HBASE-4542] Add filter info to slow
query logging.
REVISION DETAIL
https://reviews.facebook.net/D1539
COMMIT
https://reviews.facebook.net/rHBASE1299019
add filter info to slow query logging
-
Key: HBASE-4542
URL: https://issues.apache.org/jira/browse/HBASE-4542
Project: HBase
Issue Type: Improvement
Affects Versions: 0.89.20100924
Reporter: Kannan Muthukkaruppan
Assignee: Madhuwanti Vaidya
Attachments:
0001-jira-HBASE-4542-Add-filter-info-to-slow-query-loggin.patch,
Add-filter-info-to-slow-query-logging-2012-03-06_14_28_13.patch,
D1263.2.patch, D1539.1.patch
Slow query log doesn't report filters in effect.
For example:
{code}
(operationTooSlow): \
{processingtimems:3468,client:10.138.43.206:40035,timeRange:
[0,9223372036854775807],\
starttimems:1317772005821,responsesize:42411, \
class:HRegionServer,table:myTable,families:{CF1:ALL]},\
row:6c3b8efa132f0219b7621ed1e5c8c70b,queuetimems:0,\
method:get,totalColumns:1,maxVersions:1,storeLimit:-1}
{code}
the above would suggest that all columns of myTable:CF1 are being requested
for the given row. But in reality there could be filters in effect (such as
ColumnPrefixFilter, ColumnRangeFilter, TimestampsFilter() etc.). We should
enhance the slow query log to capture report this information.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4542) add filter info to slow query logging
[
https://issues.apache.org/jira/browse/HBASE-4542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Bautin updated HBASE-4542:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
add filter info to slow query logging
-
Key: HBASE-4542
URL: https://issues.apache.org/jira/browse/HBASE-4542
Project: HBase
Issue Type: Improvement
Affects Versions: 0.89.20100924
Reporter: Kannan Muthukkaruppan
Assignee: Madhuwanti Vaidya
Attachments:
0001-jira-HBASE-4542-Add-filter-info-to-slow-query-loggin.patch,
Add-filter-info-to-slow-query-logging-2012-03-06_14_28_13.patch,
D1263.2.patch, D1539.1.patch
Slow query log doesn't report filters in effect.
For example:
{code}
(operationTooSlow): \
{processingtimems:3468,client:10.138.43.206:40035,timeRange:
[0,9223372036854775807],\
starttimems:1317772005821,responsesize:42411, \
class:HRegionServer,table:myTable,families:{CF1:ALL]},\
row:6c3b8efa132f0219b7621ed1e5c8c70b,queuetimems:0,\
method:get,totalColumns:1,maxVersions:1,storeLimit:-1}
{code}
the above would suggest that all columns of myTable:CF1 are being requested
for the given row. But in reality there could be filters in effect (such as
ColumnPrefixFilter, ColumnRangeFilter, TimestampsFilter() etc.). We should
enhance the slow query log to capture report this information.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4542) add filter info to slow query logging
[
https://issues.apache.org/jira/browse/HBASE-4542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Bautin updated HBASE-4542:
--
Fix Version/s: 0.94.0
add filter info to slow query logging
-
Key: HBASE-4542
URL: https://issues.apache.org/jira/browse/HBASE-4542
Project: HBase
Issue Type: Improvement
Affects Versions: 0.89.20100924
Reporter: Kannan Muthukkaruppan
Assignee: Madhuwanti Vaidya
Fix For: 0.94.0
Attachments:
0001-jira-HBASE-4542-Add-filter-info-to-slow-query-loggin.patch,
Add-filter-info-to-slow-query-logging-2012-03-06_14_28_13.patch,
D1263.2.patch, D1539.1.patch
Slow query log doesn't report filters in effect.
For example:
{code}
(operationTooSlow): \
{processingtimems:3468,client:10.138.43.206:40035,timeRange:
[0,9223372036854775807],\
starttimems:1317772005821,responsesize:42411, \
class:HRegionServer,table:myTable,families:{CF1:ALL]},\
row:6c3b8efa132f0219b7621ed1e5c8c70b,queuetimems:0,\
method:get,totalColumns:1,maxVersions:1,storeLimit:-1}
{code}
the above would suggest that all columns of myTable:CF1 are being requested
for the given row. But in reality there could be filters in effect (such as
ColumnPrefixFilter, ColumnRangeFilter, TimestampsFilter() etc.). We should
enhance the slow query log to capture report this information.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5292) getsize per-CF metric incorrectly counts compaction related reads as well
[ https://issues.apache.org/jira/browse/HBASE-5292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226412#comment-13226412 ] Mikhail Bautin commented on HBASE-5292: --- @Stack: yes, I will commit this. Sorry it's been a while. getsize per-CF metric incorrectly counts compaction related reads as well -- Key: HBASE-5292 URL: https://issues.apache.org/jira/browse/HBASE-5292 Project: HBase Issue Type: Bug Affects Versions: 0.89.20100924 Reporter: Kannan Muthukkaruppan Attachments: 0001-jira-HBASE-5292-Prevent-counting-getSize-on-compacti.patch, D1527.1.patch, D1527.2.patch, D1527.3.patch, D1527.4.patch, D1617.1.patch The per-CF getsize metric's intent was to track bytes returned (to HBase clients) per-CF. [Note: We already have metrics to track # of HFileBlock's read for compaction vs. non-compaction cases -- e.g., compactionblockreadcnt vs. fsblockreadcnt.] Currently, the getsize metric gets updated for both client initiated Get/Scan operations as well for compaction related reads. The metric is updated in StoreScanner.java:next() when the Scan query matcher returns an INCLUDE* code via a: HRegion.incrNumericMetric(this.metricNameGetsize, copyKv.getLength()); We should not do the above in case of compactions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5292) getsize per-CF metric incorrectly counts compaction related reads as well
[ https://issues.apache.org/jira/browse/HBASE-5292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226416#comment-13226416 ] Phabricator commented on HBASE-5292: mbautin has accepted the revision [jira] [HBASE-5292] Prevent counting getSize on compactions. REVISION DETAIL https://reviews.facebook.net/D1617 BRANCH getsize getsize per-CF metric incorrectly counts compaction related reads as well -- Key: HBASE-5292 URL: https://issues.apache.org/jira/browse/HBASE-5292 Project: HBase Issue Type: Bug Affects Versions: 0.89.20100924 Reporter: Kannan Muthukkaruppan Attachments: 0001-jira-HBASE-5292-Prevent-counting-getSize-on-compacti.patch, D1527.1.patch, D1527.2.patch, D1527.3.patch, D1527.4.patch, D1617.1.patch The per-CF getsize metric's intent was to track bytes returned (to HBase clients) per-CF. [Note: We already have metrics to track # of HFileBlock's read for compaction vs. non-compaction cases -- e.g., compactionblockreadcnt vs. fsblockreadcnt.] Currently, the getsize metric gets updated for both client initiated Get/Scan operations as well for compaction related reads. The metric is updated in StoreScanner.java:next() when the Scan query matcher returns an INCLUDE* code via a: HRegion.incrNumericMetric(this.metricNameGetsize, copyKv.getLength()); We should not do the above in case of compactions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5520) Support reseek() at RegionScanner
[
https://issues.apache.org/jira/browse/HBASE-5520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226414#comment-13226414
]
stack commented on HBASE-5520:
--
{code}
+ if (kv == null) {$
+throw new IllegalArgumentException(Row cannot be null.);$
+ }$
{code}
Should be kv cannot be null?
I don't understand what this means:
{code}
The kv that is required to seek must be given$
+ * explicitly for reseek. Should not be used to seek to a key which may come$
+ * before the current position.$
+ * Note : Recommended to use knowing how seek can be done on different kvs.$
+ * Suggested to seek to row boundaries like start of a row or end of a row.$
+ * Seeking to the middle of a row may lead to inconsistencies across stores.$
{code}
... its probably because I'm slow but I'm sure there will be slow fellows
following behind me. Is it saying you need to create a KV to do this?
How would I know what the next row is in advance? I suppose with the mvcc, you
have some hope of isolating this row... but if we are going for row
boundaries, this patch looks to be missing some heft.
Support reseek() at RegionScanner
-
Key: HBASE-5520
URL: https://issues.apache.org/jira/browse/HBASE-5520
Project: HBase
Issue Type: Improvement
Components: regionserver
Affects Versions: 0.92.0
Reporter: Anoop Sam John
Assignee: ramkrishna.s.vasudevan
Fix For: 0.96.0
Attachments: HBASE-5520_1.patch, HBASE-5520_2.patch
reseek() is not supported currently at the RegionScanner level. We can
support the same.
This is created following the discussion under HBASE-2038
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5542) Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()
[ https://issues.apache.org/jira/browse/HBASE-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HBASE-5542: --- Attachment: HBASE-5542.D2217.3.patch sc updated the revision HBASE-5542 [jira] Unify HRegion.mutateRowsWithLocks() and HRegion.processRow(). Reviewers: tedyu, lhofhansl, JIRA Rebase with HBASE-5541 REVISION DETAIL https://reviews.facebook.net/D2217 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java src/main/java/org/apache/hadoop/hbase/regionserver/MultiRowMutationProcessor.java src/main/java/org/apache/hadoop/hbase/regionserver/MultiRowProcessor.java src/main/java/org/apache/hadoop/hbase/coprocessor/RowProcessor.java src/test/java/org/apache/hadoop/hbase/coprocessor/TestProcessRowEndpoint.java Unify HRegion.mutateRowsWithLocks() and HRegion.processRow() Key: HBASE-5542 URL: https://issues.apache.org/jira/browse/HBASE-5542 Project: HBase Issue Type: Improvement Reporter: Scott Chen Assignee: Scott Chen Fix For: 0.96.0 Attachments: HBASE-5542.D2217.1.patch, HBASE-5542.D2217.2.patch, HBASE-5542.D2217.3.patch mutateRowsWithLocks() does atomic mutations on multiple rows. processRow() does atomic read-modify-writes on a single row. It will be useful to generalize both and have a processRowsWithLocks() that does atomic read-modify-writes on multiple rows. This also helps reduce some redundancy in the codes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5542) Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()
[ https://issues.apache.org/jira/browse/HBASE-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226422#comment-13226422 ] Hadoop QA commented on HBASE-5542: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12517781/HBASE-5542.D2217.3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1149//console This message is automatically generated. Unify HRegion.mutateRowsWithLocks() and HRegion.processRow() Key: HBASE-5542 URL: https://issues.apache.org/jira/browse/HBASE-5542 Project: HBase Issue Type: Improvement Reporter: Scott Chen Assignee: Scott Chen Fix For: 0.96.0 Attachments: HBASE-5542.D2217.1.patch, HBASE-5542.D2217.2.patch, HBASE-5542.D2217.3.patch mutateRowsWithLocks() does atomic mutations on multiple rows. processRow() does atomic read-modify-writes on a single row. It will be useful to generalize both and have a processRowsWithLocks() that does atomic read-modify-writes on multiple rows. This also helps reduce some redundancy in the codes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4542) add filter info to slow query logging
[
https://issues.apache.org/jira/browse/HBASE-4542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-4542:
-
Fix Version/s: 0.96.0
add filter info to slow query logging
-
Key: HBASE-4542
URL: https://issues.apache.org/jira/browse/HBASE-4542
Project: HBase
Issue Type: Improvement
Affects Versions: 0.89.20100924
Reporter: Kannan Muthukkaruppan
Assignee: Madhuwanti Vaidya
Fix For: 0.94.0, 0.96.0
Attachments:
0001-jira-HBASE-4542-Add-filter-info-to-slow-query-loggin.patch,
Add-filter-info-to-slow-query-logging-2012-03-06_14_28_13.patch,
D1263.2.patch, D1539.1.patch
Slow query log doesn't report filters in effect.
For example:
{code}
(operationTooSlow): \
{processingtimems:3468,client:10.138.43.206:40035,timeRange:
[0,9223372036854775807],\
starttimems:1317772005821,responsesize:42411, \
class:HRegionServer,table:myTable,families:{CF1:ALL]},\
row:6c3b8efa132f0219b7621ed1e5c8c70b,queuetimems:0,\
method:get,totalColumns:1,maxVersions:1,storeLimit:-1}
{code}
the above would suggest that all columns of myTable:CF1 are being requested
for the given row. But in reality there could be filters in effect (such as
ColumnPrefixFilter, ColumnRangeFilter, TimestampsFilter() etc.). We should
enhance the slow query log to capture report this information.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4542) add filter info to slow query logging
[
https://issues.apache.org/jira/browse/HBASE-4542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226435#comment-13226435
]
Lars Hofhansl commented on HBASE-4542:
--
Are you going to commit this to 0.94 as well? (trunk is 0.96 now)
add filter info to slow query logging
-
Key: HBASE-4542
URL: https://issues.apache.org/jira/browse/HBASE-4542
Project: HBase
Issue Type: Improvement
Affects Versions: 0.89.20100924
Reporter: Kannan Muthukkaruppan
Assignee: Madhuwanti Vaidya
Fix For: 0.94.0, 0.96.0
Attachments:
0001-jira-HBASE-4542-Add-filter-info-to-slow-query-loggin.patch,
Add-filter-info-to-slow-query-logging-2012-03-06_14_28_13.patch,
D1263.2.patch, D1539.1.patch
Slow query log doesn't report filters in effect.
For example:
{code}
(operationTooSlow): \
{processingtimems:3468,client:10.138.43.206:40035,timeRange:
[0,9223372036854775807],\
starttimems:1317772005821,responsesize:42411, \
class:HRegionServer,table:myTable,families:{CF1:ALL]},\
row:6c3b8efa132f0219b7621ed1e5c8c70b,queuetimems:0,\
method:get,totalColumns:1,maxVersions:1,storeLimit:-1}
{code}
the above would suggest that all columns of myTable:CF1 are being requested
for the given row. But in reality there could be filters in effect (such as
ColumnPrefixFilter, ColumnRangeFilter, TimestampsFilter() etc.). We should
enhance the slow query log to capture report this information.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5074) support checksums in HBase block cache
[ https://issues.apache.org/jira/browse/HBASE-5074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226438#comment-13226438 ] Lars Hofhansl commented on HBASE-5074: -- Actually svn diff -r r1298574:r1298641 gives me a diff of size different from the attached D1521.14.patch (I can't diff the patch files easily, as files are reordered) Mikhail, are you sure D1521.14.patch is the exact committed patch? support checksums in HBase block cache -- Key: HBASE-5074 URL: https://issues.apache.org/jira/browse/HBASE-5074 Project: HBase Issue Type: Improvement Components: regionserver Reporter: dhruba borthakur Assignee: dhruba borthakur Fix For: 0.94.0, 0.96.0 Attachments: 5074-0.94.txt, D1521.1.patch, D1521.1.patch, D1521.10.patch, D1521.10.patch, D1521.10.patch, D1521.10.patch, D1521.10.patch, D1521.11.patch, D1521.11.patch, D1521.12.patch, D1521.12.patch, D1521.13.patch, D1521.13.patch, D1521.14.patch, D1521.14.patch, D1521.2.patch, D1521.2.patch, D1521.3.patch, D1521.3.patch, D1521.4.patch, D1521.4.patch, D1521.5.patch, D1521.5.patch, D1521.6.patch, D1521.6.patch, D1521.7.patch, D1521.7.patch, D1521.8.patch, D1521.8.patch, D1521.9.patch, D1521.9.patch The current implementation of HDFS stores the data in one block file and the metadata(checksum) in another block file. This means that every read into the HBase block cache actually consumes two disk iops, one to the datafile and one to the checksum file. This is a major problem for scaling HBase, because HBase is usually bottlenecked on the number of random disk iops that the storage-hardware offers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5542) Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()
[ https://issues.apache.org/jira/browse/HBASE-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226451#comment-13226451 ] Phabricator commented on HBASE-5542: lhofhansl has commented on the revision HBASE-5542 [jira] Unify HRegion.mutateRowsWithLocks() and HRegion.processRow(). INLINE COMMENTS src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java:4362 Hmm... That means we're now running the coprocessor post host after the region operation closed. It almost seems that the RowProcessor should have three phases: 1. A pre phase (without locks and mvcc, but with the WALEdit). The prephase could abort the operation 2. the op itself 3. a post operation (after locks are released, mvcc is rolled forward) REVISION DETAIL https://reviews.facebook.net/D2217 BRANCH 5542-new-file Unify HRegion.mutateRowsWithLocks() and HRegion.processRow() Key: HBASE-5542 URL: https://issues.apache.org/jira/browse/HBASE-5542 Project: HBase Issue Type: Improvement Reporter: Scott Chen Assignee: Scott Chen Fix For: 0.96.0 Attachments: HBASE-5542.D2217.1.patch, HBASE-5542.D2217.2.patch, HBASE-5542.D2217.3.patch mutateRowsWithLocks() does atomic mutations on multiple rows. processRow() does atomic read-modify-writes on a single row. It will be useful to generalize both and have a processRowsWithLocks() that does atomic read-modify-writes on multiple rows. This also helps reduce some redundancy in the codes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5552) Clean up our jmx view; its a bit of a mess
[ https://issues.apache.org/jira/browse/HBASE-5552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226464#comment-13226464 ] Hudson commented on HBASE-5552: --- Integrated in HBase-0.92-security #102 (See [https://builds.apache.org/job/HBase-0.92-security/102/]) HBASE-5552 Clean up our jmx view; its a bit of a mess (Revision 1298924) HBASE-5552 Clean up our jmx view; its a bit of a mess (Revision 1298919) Result = SUCCESS stack : Files : * /hbase/branches/0.92/CHANGES.txt stack : Files : * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPCStatistics.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/metrics/HBaseInfo.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java Clean up our jmx view; its a bit of a mess -- Key: HBASE-5552 URL: https://issues.apache.org/jira/browse/HBASE-5552 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Priority: Blocker Fix For: 0.92.1, 0.94.0 Attachments: 0.92.0jmx.png, 5552.txt, currentjmxview.png, patchedjmxview.png Fix before we release 0.92.1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5542) Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()
[
https://issues.apache.org/jira/browse/HBASE-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226474#comment-13226474
]
Scott Chen commented on HBASE-5542:
---
@Lars: It seems to me that the major difference in doMiniBatchPut is that it
gets the locks with non-block option and process the locked rows. It remembers
a index of processed row and keeps continuing.
How about making MultiRowProcessor having an atomic option?
{code}
interface MultiRowProcessor {
boolean isAtomic();
Collectionbyte[] rowsToLock();
// need to tell processor which rows to process in the non-atomic mode
process(Collectionbyte[] rowsLocked);
}
{code}
In HRegion, we check if the processor is atomic.
If it is not, we get as many lock as possible and process them in a loop.
{code}
HRegion.processRowsWithLocks(MultiRowProcessor processor) {
while (done) {
if (processor.isAtomic()) {
// get all locks with blocking option
} else {
// get as many locks with non-blocking option
}
processor.process(lockedRows);
if (/*we process all the rows*/) {
done = true;
}
}
}
{code}
What do you think?
Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()
Key: HBASE-5542
URL: https://issues.apache.org/jira/browse/HBASE-5542
Project: HBase
Issue Type: Improvement
Reporter: Scott Chen
Assignee: Scott Chen
Fix For: 0.96.0
Attachments: HBASE-5542.D2217.1.patch, HBASE-5542.D2217.2.patch,
HBASE-5542.D2217.3.patch
mutateRowsWithLocks() does atomic mutations on multiple rows.
processRow() does atomic read-modify-writes on a single row.
It will be useful to generalize both and have a
processRowsWithLocks() that does atomic read-modify-writes on multiple rows.
This also helps reduce some redundancy in the codes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4608) HLog Compression
[
https://issues.apache.org/jira/browse/HBASE-4608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226477#comment-13226477
]
stack commented on HBASE-4608:
--
The TestLRUDictionary test looks like it could be fatter. Looks like you
should be able to throw at it a bunch more combinations. And better
excercising of new BidirectionalLRUMap type. Better to find the issues here
in unit test than
Whats the difference between
{code}
+ public static int hashBytes(byte[] bytes, int offset, int length) {
{code}
and the existing
{code}
public static int hashCode(final byte [] b, final int length) {
{code}
They look to do the same thing? We should remove the new one if so.
We will have a keycontext when we are deserializing? Hows that work?
So we compress at the individual entry level? Why not file at a time? (Sorry
if this has been explained earlier)
Is this right in the WALReader?
{code}
+compression = conf.getBoolean(HConstants.ENABLE_WAL_COMPRESSION, false);
{code}
How does that work if the WAL was written compressed but this flag is false?
We break? Shouldn't this instead be keyed off the entries themselves? Should
it be a sequence file attribute saying this a compressed file?
Do we foresee replication being able to use this facility? Seems like a
natural having it ship compressed entries.
Good stuff.
HLog Compression
Key: HBASE-4608
URL: https://issues.apache.org/jira/browse/HBASE-4608
Project: HBase
Issue Type: New Feature
Reporter: Li Pi
Assignee: Li Pi
Fix For: 0.94.0
Attachments: 4608-v19.txt, 4608v1.txt, 4608v13.txt, 4608v13.txt,
4608v14.txt, 4608v15.txt, 4608v16.txt, 4608v17.txt, 4608v18.txt, 4608v5.txt,
4608v6.txt, 4608v7.txt, 4608v8fixed.txt
The current bottleneck to HBase write speed is replicating the WAL appends
across different datanodes. We can speed up this process by compressing the
HLog. Current plan involves using a dictionary to compress table name, region
id, cf name, and possibly other bits of repeated data. Also, HLog format may
be changed in other ways to produce a smaller HLog.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5292) getsize per-CF metric incorrectly counts compaction related reads as well
[ https://issues.apache.org/jira/browse/HBASE-5292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Bautin updated HBASE-5292: -- Attachment: jira-HBASE-5292-Prevent-counting-getSize-on-compacti-2012-03-09_13_26_52.patch Rebased patch for Hadoop QA testing getsize per-CF metric incorrectly counts compaction related reads as well -- Key: HBASE-5292 URL: https://issues.apache.org/jira/browse/HBASE-5292 Project: HBase Issue Type: Bug Affects Versions: 0.89.20100924 Reporter: Kannan Muthukkaruppan Attachments: 0001-jira-HBASE-5292-Prevent-counting-getSize-on-compacti.patch, D1527.1.patch, D1527.2.patch, D1527.3.patch, D1527.4.patch, D1617.1.patch, jira-HBASE-5292-Prevent-counting-getSize-on-compacti-2012-03-09_13_26_52.patch The per-CF getsize metric's intent was to track bytes returned (to HBase clients) per-CF. [Note: We already have metrics to track # of HFileBlock's read for compaction vs. non-compaction cases -- e.g., compactionblockreadcnt vs. fsblockreadcnt.] Currently, the getsize metric gets updated for both client initiated Get/Scan operations as well for compaction related reads. The metric is updated in StoreScanner.java:next() when the Scan query matcher returns an INCLUDE* code via a: HRegion.incrNumericMetric(this.metricNameGetsize, copyKv.getLength()); We should not do the above in case of compactions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5546) Master assigns region in the original region server when opening region failed
[ https://issues.apache.org/jira/browse/HBASE-5546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5546: - Fix Version/s: (was: 0.92.1) 0.92.2 Master assigns region in the original region server when opening region failed Key: HBASE-5546 URL: https://issues.apache.org/jira/browse/HBASE-5546 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.92.0 Reporter: gaojinchao Priority: Minor Fix For: 0.92.2 Master assigns region in the original region server when RS_ZK_REGION_FAILED_OPEN envent was coming. Maybe we should choose other region server. [2012-03-07 10:14:21,750] [DEBUG] [main-EventThread] [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, region=c70e98bdca98a0657a56436741523053 [2012-03-07 10:14:31,826] [DEBUG] [main-EventThread] [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, region=c70e98bdca98a0657a56436741523053 [2012-03-07 10:14:41,903] [DEBUG] [main-EventThread] [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, region=c70e98bdca98a0657a56436741523053 [2012-03-07 10:14:51,975] [DEBUG] [main-EventThread] [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, region=c70e98bdca98a0657a56436741523053 [2012-03-07 10:15:02,056] [DEBUG] [main-EventThread] [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, region=c70e98bdca98a0657a56436741523053 [2012-03-07 10:15:12,167] [DEBUG] [main-EventThread] [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, region=c70e98bdca98a0657a56436741523053 [2012-03-07 10:15:22,231] [DEBUG] [main-EventThread] [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, region=c70e98bdca98a0657a56436741523053 [2012-03-07 10:15:32,303] [DEBUG] [main-EventThread] [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, region=c70e98bdca98a0657a56436741523053 [2012-03-07 10:15:42,375] [DEBUG] [main-EventThread] [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, region=c70e98bdca98a0657a56436741523053 [2012-03-07 10:15:52,447] [DEBUG] [main-EventThread] [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, region=c70e98bdca98a0657a56436741523053 [2012-03-07 10:16:02,528] [DEBUG] [main-EventThread] [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, region=c70e98bdca98a0657a56436741523053 [2012-03-07 10:16:12,600] [DEBUG] [main-EventThread] [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, region=c70e98bdca98a0657a56436741523053 [2012-03-07 10:16:22,676] [DEBUG] [main-EventThread] [org.apache.hadoop.hbase.master.AssignmentManager 553] Handling transition=RS_ZK_REGION_FAILED_OPEN, server=158-1-130-11,20020,1331108408232, region=c70e98bdca98a0657a56436741523053 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
