[jira] [Commented] (HBASE-7880) HFile Recovery/Rewrite Tool
[ https://issues.apache.org/jira/browse/HBASE-7880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901224#comment-14901224 ] David S. Wang commented on HBASE-7880: -- We should get this in - seems like useful work. > HFile Recovery/Rewrite Tool > --- > > Key: HBASE-7880 > URL: https://issues.apache.org/jira/browse/HBASE-7880 > Project: HBase > Issue Type: New Feature > Components: HFile >Affects Versions: 0.95.2 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Attachments: HBASE-7880-v0.patch > > > Sometimes is useful to have a tool to migrate files from a new version to an > old version (e.g. convert a new XYZ encoded/compressed file to an old > "uncompressed" format) > also it will be useful to been able to recover an hfile from a corrupted > state. (e.g. trailer missing/broken, ...) > The "user" can provide the information about the file (compression & co) and > try to recover as much as possible from the file by reading data blocks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-9369) Add support for 1- and 2-byte integers in OrderedBytes and provide types
[ https://issues.apache.org/jira/browse/HBASE-9369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-9369: - Fix Version/s: (was: 0.95.2) 0.96.0 Add support for 1- and 2-byte integers in OrderedBytes and provide types Key: HBASE-9369 URL: https://issues.apache.org/jira/browse/HBASE-9369 Project: HBase Issue Type: Improvement Reporter: Nick Dimiduk Assignee: He Liangliang Fix For: 0.98.0, 0.96.0 Attachments: HBASE-9369.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9076) Throttle log messages for missing peer cluster
[ https://issues.apache.org/jira/browse/HBASE-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-9076: - Summary: Throttle log messages for missing peer cluster (was: Change ) Throttle log messages for missing peer cluster -- Key: HBASE-9076 URL: https://issues.apache.org/jira/browse/HBASE-9076 Project: HBase Issue Type: Improvement Components: Replication Affects Versions: 0.95.1, 0.94.10 Reporter: David S. Wang Assignee: David S. Wang Priority: Trivial If the replication peer is not available, then RS logs get flooded by messages similar to the following: 2013-07-19 11:55:27,964 INFO org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Getting 0 rs from peer cluster # Indexer_hbaseIndexer 2013-07-19 11:55:28,670 INFO org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Getting 0 rs from peer cluster # Indexer_hbaseIndexer 2013-07-19 11:55:28,966 INFO org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Getting 0 rs from peer cluster # Indexer_hbaseIndexer 2013-07-19 11:55:29,672 INFO org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Getting 0 rs from peer cluster # Indexer_hbaseIndexer 2013-07-19 11:55:29,969 INFO org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Getting 0 rs from peer cluster # Indexer_hbaseIndexer It would be better to change the log level to DEBUG in order to avoid these messages showing up in the default case. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-9076) Change
David S. Wang created HBASE-9076: Summary: Change Key: HBASE-9076 URL: https://issues.apache.org/jira/browse/HBASE-9076 Project: HBase Issue Type: Improvement Components: Replication Affects Versions: 0.94.10, 0.95.1 Reporter: David S. Wang Assignee: David S. Wang Priority: Trivial If the replication peer is not available, then RS logs get flooded by messages similar to the following: 2013-07-19 11:55:27,964 INFO org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Getting 0 rs from peer cluster # Indexer_hbaseIndexer 2013-07-19 11:55:28,670 INFO org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Getting 0 rs from peer cluster # Indexer_hbaseIndexer 2013-07-19 11:55:28,966 INFO org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Getting 0 rs from peer cluster # Indexer_hbaseIndexer 2013-07-19 11:55:29,672 INFO org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Getting 0 rs from peer cluster # Indexer_hbaseIndexer 2013-07-19 11:55:29,969 INFO org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Getting 0 rs from peer cluster # Indexer_hbaseIndexer It would be better to change the log level to DEBUG in order to avoid these messages showing up in the default case. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9076) Throttle log messages for missing peer cluster
[ https://issues.apache.org/jira/browse/HBASE-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-9076: - Status: Patch Available (was: Open) Throttle log messages for missing peer cluster -- Key: HBASE-9076 URL: https://issues.apache.org/jira/browse/HBASE-9076 Project: HBase Issue Type: Improvement Components: Replication Affects Versions: 0.94.10, 0.95.1 Reporter: David S. Wang Assignee: David S. Wang Priority: Trivial Attachments: HBASE-9076.patch If the replication peer is not available, then RS logs get flooded by messages similar to the following: 2013-07-19 11:55:27,964 INFO org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Getting 0 rs from peer cluster # Indexer_hbaseIndexer 2013-07-19 11:55:28,670 INFO org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Getting 0 rs from peer cluster # Indexer_hbaseIndexer 2013-07-19 11:55:28,966 INFO org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Getting 0 rs from peer cluster # Indexer_hbaseIndexer 2013-07-19 11:55:29,672 INFO org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Getting 0 rs from peer cluster # Indexer_hbaseIndexer 2013-07-19 11:55:29,969 INFO org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Getting 0 rs from peer cluster # Indexer_hbaseIndexer It would be better to change the log level to DEBUG in order to avoid these messages showing up in the default case. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9076) Throttle log messages for missing peer cluster
[ https://issues.apache.org/jira/browse/HBASE-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-9076: - Attachment: HBASE-9076.patch Throttle log messages for missing peer cluster -- Key: HBASE-9076 URL: https://issues.apache.org/jira/browse/HBASE-9076 Project: HBase Issue Type: Improvement Components: Replication Affects Versions: 0.95.1, 0.94.10 Reporter: David S. Wang Assignee: David S. Wang Priority: Trivial Attachments: HBASE-9076.patch If the replication peer is not available, then RS logs get flooded by messages similar to the following: 2013-07-19 11:55:27,964 INFO org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Getting 0 rs from peer cluster # Indexer_hbaseIndexer 2013-07-19 11:55:28,670 INFO org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Getting 0 rs from peer cluster # Indexer_hbaseIndexer 2013-07-19 11:55:28,966 INFO org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Getting 0 rs from peer cluster # Indexer_hbaseIndexer 2013-07-19 11:55:29,672 INFO org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Getting 0 rs from peer cluster # Indexer_hbaseIndexer 2013-07-19 11:55:29,969 INFO org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Getting 0 rs from peer cluster # Indexer_hbaseIndexer It would be better to change the log level to DEBUG in order to avoid these messages showing up in the default case. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9076) Throttle log messages for missing peer cluster
[ https://issues.apache.org/jira/browse/HBASE-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-9076: - Resolution: Duplicate Status: Resolved (was: Patch Available) Throttle log messages for missing peer cluster -- Key: HBASE-9076 URL: https://issues.apache.org/jira/browse/HBASE-9076 Project: HBase Issue Type: Improvement Components: Replication Affects Versions: 0.95.1, 0.94.10 Reporter: David S. Wang Assignee: David S. Wang Priority: Trivial Attachments: HBASE-9076.patch If the replication peer is not available, then RS logs get flooded by messages similar to the following: 2013-07-19 11:55:27,964 INFO org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Getting 0 rs from peer cluster # Indexer_hbaseIndexer 2013-07-19 11:55:28,670 INFO org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Getting 0 rs from peer cluster # Indexer_hbaseIndexer 2013-07-19 11:55:28,966 INFO org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Getting 0 rs from peer cluster # Indexer_hbaseIndexer 2013-07-19 11:55:29,672 INFO org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Getting 0 rs from peer cluster # Indexer_hbaseIndexer 2013-07-19 11:55:29,969 INFO org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Getting 0 rs from peer cluster # Indexer_hbaseIndexer It would be better to change the log level to DEBUG in order to avoid these messages showing up in the default case. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8745) Fix src assembly so includes top-level src dir
[ https://issues.apache.org/jira/browse/HBASE-8745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13684191#comment-13684191 ] David S. Wang commented on HBASE-8745: -- [~saint@gmail.com], I see this change applied to trunk, but I don't see this change applied to 0.95, though the JIRA is marked as fixed in 0.95.2. Fix src assembly so includes top-level src dir -- Key: HBASE-8745 URL: https://issues.apache.org/jira/browse/HBASE-8745 Project: HBase Issue Type: Bug Components: build Reporter: stack Assignee: stack Fix For: 0.98.0, 0.95.2 Attachments: 8745.txt Noticed by Dave Wang. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8450) Update hbase-default.xml and general recommendations to better suit current hw, h2, experience, etc.
[ https://issues.apache.org/jira/browse/HBASE-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13658829#comment-13658829 ] David S. Wang commented on HBASE-8450: -- What's the plan to validate any changes that are made here? e.g. throw code onto a cluster of size X with reasonably-modern specs, run YCSB workload A,B,C etc., then change defaults, reboot entire cluster, and run same workloads? That may cover some performance-related areas, but not sure what else to do to validate. Update hbase-default.xml and general recommendations to better suit current hw, h2, experience, etc. Key: HBASE-8450 URL: https://issues.apache.org/jira/browse/HBASE-8450 Project: HBase Issue Type: Task Components: Usability Reporter: stack Priority: Critical Fix For: 0.95.1 Attachments: 8450.txt This is a critical task we need to do before we release; review our defaults. On cursory review, there are configs in hbase-default.xml that no longer have matching code; there are some that should be changed because we know better now and others that should be amended because hardware and deploys are bigger than they used to be. We could also move stuff around so that the must-edit stuff is near the top (zk quorum config. is mid-way down the page) and beef up the descriptions -- especially since these descriptions shine through in the refguide. Lastly, I notice that our tgz does not include an hbase-default.xml other than the one bundled up in the jar. Maybe we should make it more accessible. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8298) desc tablename shorthand for describe tablename, similar to how databases have
[ https://issues.apache.org/jira/browse/HBASE-8298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-8298: - Labels: noob (was: ) desc tablename shorthand for describe tablename, similar to how databases have -- Key: HBASE-8298 URL: https://issues.apache.org/jira/browse/HBASE-8298 Project: HBase Issue Type: Improvement Components: shell Affects Versions: 0.94.2 Reporter: Hari Sekhon Priority: Trivial Labels: noob It would be nice if you could type desc tablename in hbase shell as shorthand for describe tablename similar to how you can in traditional databases. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8026) HBase Shell docs for scan command don't reference VERSIONS
[ https://issues.apache.org/jira/browse/HBASE-8026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-8026: - Labels: noob (was: ) HBase Shell docs for scan command don't reference VERSIONS -- Key: HBASE-8026 URL: https://issues.apache.org/jira/browse/HBASE-8026 Project: HBase Issue Type: Bug Reporter: Jonathan Natkins Labels: noob hbase(main):046:0 help 'scan' Scan a table; pass table name and optionally a dictionary of scanner specifications. Scanner specifications may include one or more of: TIMERANGE, FILTER, LIMIT, STARTROW, STOPROW, TIMESTAMP, MAXLENGTH, or COLUMNS, CACHE VERSIONS should be mentioned somewhere here. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7183) print WARN message if hbase.replication.sizeOfLogQueue is too big
[ https://issues.apache.org/jira/browse/HBASE-7183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-7183: - Labels: noob (was: ) print WARN message if hbase.replication.sizeOfLogQueue is too big - Key: HBASE-7183 URL: https://issues.apache.org/jira/browse/HBASE-7183 Project: HBase Issue Type: Improvement Components: Replication Reporter: Sho Shimauchi Labels: noob A metric hbase.replication.sizeOfLogQueue may become big when replication is delaying. It would be useful if HBase prints WARN log which tells hbase.replication.sizeOfLogQueue is too big. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7314) Can't start REST/Thrift server if HBASE_JMX_OPTS not set
[ https://issues.apache.org/jira/browse/HBASE-7314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13528401#comment-13528401 ] David S. Wang commented on HBASE-7314: -- +1 Can't start REST/Thrift server if HBASE_JMX_OPTS not set Key: HBASE-7314 URL: https://issues.apache.org/jira/browse/HBASE-7314 Project: HBase Issue Type: Bug Components: scripts Affects Versions: 0.96.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Fix For: 0.96.0 Attachments: trunk-7314.patch By default JMX is enabled for REST server and Thrift server. However, if HBASE_JMX_OPTS is not set, and JMX remote access rule is not set, we can't bring up the REST/Thrift server due to JMX remote access rule errors. We need to enhance the hbase script not to enable JMX for REST/Thrift server is HBASE_JMX_OPTS is not set. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6972) HBase Shell deleteall requires column to be defined
[ https://issues.apache.org/jira/browse/HBASE-6972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481877#comment-13481877 ] David S. Wang commented on HBASE-6972: -- +1 on patch. HBase Shell deleteall requires column to be defined Key: HBASE-6972 URL: https://issues.apache.org/jira/browse/HBASE-6972 Project: HBase Issue Type: Bug Components: shell Affects Versions: 0.96.0 Reporter: Ricky Saltzer Assignee: Ricky Saltzer Fix For: 0.96.0 Attachments: HBASE-6972.2.patch, HBASE-6972.patch It appears that the shell does not allow users to delete a row without specifying a column (deleteall). It looks like the deleteall.rb used to pre-define column as nil, making it optional. I've created a patch and confirmed it to be working in standalone mode, I will upload it shortly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6904) In the HBase shell, an error is thrown that states replication-related znodes already exist
[ https://issues.apache.org/jira/browse/HBASE-6904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-6904: - Priority: Minor (was: Major) In the HBase shell, an error is thrown that states replication-related znodes already exist --- Key: HBASE-6904 URL: https://issues.apache.org/jira/browse/HBASE-6904 Project: HBase Issue Type: Bug Components: Replication, Zookeeper Affects Versions: 0.92.0, 0.92.1 Reporter: Aleksandr Shulman Priority: Minor On a replication-enabled cluster, querying the list_peers produces the error lines shown below. It doesn't appear that anything is broken in terms of functionality. Stack trace: hbase(main):001:0 list_peers 12/09/29 14:41:03 ERROR zookeeper.RecoverableZooKeeper: Node /hbase/replication/peers already exists and this is not a retry 12/09/29 14:41:03 ERROR zookeeper.RecoverableZooKeeper: Node /hbase/replication/rs already exists and this is not a retry PEER ID CLUSTER KEY 0 row(s) in 0.4650 seconds -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6182) Make HBase works with jdk1.7
[ https://issues.apache.org/jira/browse/HBASE-6182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13466112#comment-13466112 ] David S. Wang commented on HBASE-6182: -- +1 on this. I've heard performance gains in the range of 20% from Todd on his HBase benchmarks. Make HBase works with jdk1.7 Key: HBASE-6182 URL: https://issues.apache.org/jira/browse/HBASE-6182 Project: HBase Issue Type: Task Reporter: Jimmy Xiang Priority: Critical Fix For: 0.96.0 Attachments: large-tests.log, medium-tests.log jdk1.7 is out for a while. HBase should support it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6682) Bad characters in logs for server names: SplitLogManager: task following PBUF
[ https://issues.apache.org/jira/browse/HBASE-6682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13445530#comment-13445530 ] David S. Wang commented on HBASE-6682: -- +1 Bad characters in logs for server names: SplitLogManager: task following PBUF -- Key: HBASE-6682 URL: https://issues.apache.org/jira/browse/HBASE-6682 Project: HBase Issue Type: Bug Affects Versions: 0.96.0 Reporter: nkeywal Assignee: Aleksandr Shulman Priority: Minor Fix For: 0.96.0 Attachments: before_after_logs, diff_splitLogManager.diff See how the server name is printed: 2012-08-29 14:28:53,567 INFO org.apache.hadoop.hbase.master.SplitLogManager: task /hbase/splitlog/hdfs%3A%2F%2Flocalhost%3A9000%2Fhbase%2F.logs%2Flocalhost%2C60202%2C1346241077569-splitting%2Flocalhost%252C60202%252C1346241077569.1346241967431 entered state PBUF localhost����ޑ�' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5948) Deprecate and remove the Avro gateway
[ https://issues.apache.org/jira/browse/HBASE-5948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13439148#comment-13439148 ] David S. Wang commented on HBASE-5948: -- The patch seems to deprecate in 0.96 without removing it. The original goal was to deprecate in 0.94 and remove in 0.96, correct? Deprecate and remove the Avro gateway - Key: HBASE-5948 URL: https://issues.apache.org/jira/browse/HBASE-5948 Project: HBase Issue Type: Task Affects Versions: 0.96.0 Reporter: Andrew Purtell Assignee: Elliott Clark Priority: Blocker Fix For: 0.96.0 Attachments: HBASE-5948-0.patch, HBASE-5948-1.patch Deprecate the Avro gateway in 0.94. Remove in 0.96. Made a blocker against that release. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5948) Deprecate and remove the Avro gateway
[ https://issues.apache.org/jira/browse/HBASE-5948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13439155#comment-13439155 ] David S. Wang commented on HBASE-5948: -- I should mention that I can provide a patch on top of this one to do the removal for 0.96; then a committer can just backport this patch to 0.94 to fulfill the original intent. I'll file that JIRA with the patch shortly unless I hear objections. Deprecate and remove the Avro gateway - Key: HBASE-5948 URL: https://issues.apache.org/jira/browse/HBASE-5948 Project: HBase Issue Type: Task Affects Versions: 0.96.0 Reporter: Andrew Purtell Assignee: Elliott Clark Priority: Blocker Fix For: 0.96.0 Attachments: HBASE-5948-0.patch, HBASE-5948-1.patch Deprecate the Avro gateway in 0.94. Remove in 0.96. Made a blocker against that release. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5948) Deprecate and remove the Avro gateway
[ https://issues.apache.org/jira/browse/HBASE-5948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13439163#comment-13439163 ] David S. Wang commented on HBASE-5948: -- That's what I get for looking at an outdated tree, and for having too many trees in general. Sorry for the spam. Deprecate and remove the Avro gateway - Key: HBASE-5948 URL: https://issues.apache.org/jira/browse/HBASE-5948 Project: HBase Issue Type: Task Affects Versions: 0.96.0 Reporter: Andrew Purtell Assignee: Elliott Clark Priority: Blocker Fix For: 0.96.0 Attachments: HBASE-5948-0.patch, HBASE-5948-1.patch Deprecate the Avro gateway in 0.94. Remove in 0.96. Made a blocker against that release. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6378) the javadoc of setEnabledTable maybe not describe accurately
[
https://issues.apache.org/jira/browse/HBASE-6378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13438211#comment-13438211
]
David S. Wang commented on HBASE-6378:
--
Actually, I did not make the original patch; I just suggested a grammatical
change.
the javadoc of setEnabledTable maybe not describe accurately
--
Key: HBASE-6378
URL: https://issues.apache.org/jira/browse/HBASE-6378
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.94.0
Reporter: zhou wenjian
Assignee: David S. Wang
Fix For: 0.94.2
Attachments: 6378.patch, HBASE-6378.patch, HBASE-6378-trunk.patch
/**
* Sets the ENABLED state in the cache and deletes the zookeeper node. Fails
* silently if the node is not in enabled in zookeeper
*
* @param tableName
* @throws KeeperException
*/
public void setEnabledTable(final String tableName) throws KeeperException {
setTableState(tableName, TableState.ENABLED);
}
When setEnabledTable occours ,It will update the cache and the zookeeper
node,rather than to delete the zk node.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6538) Remove copy_table.rb script
[ https://issues.apache.org/jira/browse/HBASE-6538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13436887#comment-13436887 ] David S. Wang commented on HBASE-6538: -- Sure, why not. It doesn't work in those releases, either. Remove copy_table.rb script --- Key: HBASE-6538 URL: https://issues.apache.org/jira/browse/HBASE-6538 Project: HBase Issue Type: Task Components: scripts Affects Versions: 0.96.0 Reporter: David S. Wang Assignee: Andrew Wang Priority: Minor Labels: noob Attachments: hbase-6583-1.patch Remove copy_table.rb script as per mailing list discussion. It hasn't been maintained in a while and does not run against any recent HBase release. There is also an MR job to do the same thing that does work. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6593) TestAdmin times out sometimes
[ https://issues.apache.org/jira/browse/HBASE-6593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13436147#comment-13436147 ] David S. Wang commented on HBASE-6593: -- +1 TestAdmin times out sometimes - Key: HBASE-6593 URL: https://issues.apache.org/jira/browse/HBASE-6593 Project: HBase Issue Type: Test Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Fix For: 0.96.0, 0.94.2 Attachments: trunk-6593.patch In TestAdmin#splitTest, individual put is used to prepare the test data. We can group them together so as to avoid possible timeout. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6539) Remove ZooKeeperWatcher from copy_table_desc.rb script
[ https://issues.apache.org/jira/browse/HBASE-6539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-6539: - Description: copy_table_desc.rb Jruby script currently fails with: $ hbase org.jruby.Main copy_tables_desc.rb NameError: cannot load Java class org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper get_proxy_or_package_under_package at org/jruby/javasupport/JavaUtilities.java:54 method_missing at file:/mnt/data/hbase/lib/jruby-complete-1.6.5.jar!/builtin/javasupport/java.rb:51 (root) at copy_tables_desc.rb:35 Just removing the ZooKeeperWatcher import seems to make it work. was:Remove copy_table.rb script as per mailing list discussion. It hasn't been maintained in a while and does not run against any recent HBase release. There is also an MR job to do the same thing that does work. Priority: Trivial (was: Minor) Summary: Remove ZooKeeperWatcher from copy_table_desc.rb script (was: Remove copy_table.rb script) Remove ZooKeeperWatcher from copy_table_desc.rb script -- Key: HBASE-6539 URL: https://issues.apache.org/jira/browse/HBASE-6539 Project: HBase Issue Type: Task Components: scripts Affects Versions: 0.96.0 Reporter: David S. Wang Assignee: David S. Wang Priority: Trivial Labels: noob copy_table_desc.rb Jruby script currently fails with: $ hbase org.jruby.Main copy_tables_desc.rb NameError: cannot load Java class org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper get_proxy_or_package_under_package at org/jruby/javasupport/JavaUtilities.java:54 method_missing at file:/mnt/data/hbase/lib/jruby-complete-1.6.5.jar!/builtin/javasupport/java.rb:51 (root) at copy_tables_desc.rb:35 Just removing the ZooKeeperWatcher import seems to make it work. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6539) Remove ZooKeeperWatcher from copy_table_desc.rb script
[ https://issues.apache.org/jira/browse/HBASE-6539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-6539: - This is a duplicate of HBASE-6525. Please resolve (I don't seem to have the permissions to do so). Remove ZooKeeperWatcher from copy_table_desc.rb script -- Key: HBASE-6539 URL: https://issues.apache.org/jira/browse/HBASE-6539 Project: HBase Issue Type: Task Components: scripts Affects Versions: 0.96.0 Reporter: David S. Wang Assignee: David S. Wang Priority: Trivial Labels: noob copy_table_desc.rb Jruby script currently fails with: $ hbase org.jruby.Main copy_tables_desc.rb NameError: cannot load Java class org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper get_proxy_or_package_under_package at org/jruby/javasupport/JavaUtilities.java:54 method_missing at file:/mnt/data/hbase/lib/jruby-complete-1.6.5.jar!/builtin/javasupport/java.rb:51 (root) at copy_tables_desc.rb:35 Just removing the ZooKeeperWatcher import seems to make it work. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6540) Remove copy_table.rb script
[ https://issues.apache.org/jira/browse/HBASE-6540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13433812#comment-13433812 ] David S. Wang commented on HBASE-6540: -- This is a dup of HBASE-6538, created during the latest JIRA wonkiness. Please resolve accordingly (I don't seem to have the permissions to do so). Remove copy_table.rb script --- Key: HBASE-6540 URL: https://issues.apache.org/jira/browse/HBASE-6540 Project: HBase Issue Type: Task Components: scripts Affects Versions: 0.96.0 Reporter: David S. Wang Assignee: David S. Wang Priority: Minor Labels: noob Remove copy_table.rb script as per mailing list discussion. It hasn't been maintained in a while and does not run against any recent HBase release. There is also an MR job to do the same thing that does work. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6542) Remove copy_table.rb
[ https://issues.apache.org/jira/browse/HBASE-6542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13433814#comment-13433814 ] David S. Wang commented on HBASE-6542: -- This is a dup of HBASE-6538, created during the latest JIRA wonkiness. Please resolve accordingly (I don't seem to have the permissions to do so). Remove copy_table.rb Key: HBASE-6542 URL: https://issues.apache.org/jira/browse/HBASE-6542 Project: HBase Issue Type: Task Components: scripts Affects Versions: 0.96.0 Reporter: David S. Wang Assignee: David S. Wang Priority: Minor Labels: noob Remove copy_table.rb script as per mailing list discussion. It hasn't been maintained in a while and does not run against any recent HBase release. There is also an MR job to do the same thing that does work. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6541) Remove copy_table.rb
[ https://issues.apache.org/jira/browse/HBASE-6541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13433813#comment-13433813 ] David S. Wang commented on HBASE-6541: -- This is a dup of HBASE-6538, created during the latest JIRA wonkiness. Please resolve accordingly (I don't seem to have the permissions to do so). Remove copy_table.rb Key: HBASE-6541 URL: https://issues.apache.org/jira/browse/HBASE-6541 Project: HBase Issue Type: Task Components: scripts Affects Versions: 0.96.0 Reporter: David S. Wang Assignee: David S. Wang Priority: Minor Labels: noob Remove copy_table.rb script as per mailing list discussion. It hasn't been maintained in a while and does not run against any recent HBase release. There is also an MR job to do the same thing that does work. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6543) Remove copy_table.rb
[ https://issues.apache.org/jira/browse/HBASE-6543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13433815#comment-13433815 ] David S. Wang commented on HBASE-6543: -- This is a dup of HBASE-6538, created during the latest JIRA wonkiness. Please resolve accordingly (I don't seem to have the permissions to do so). Remove copy_table.rb Key: HBASE-6543 URL: https://issues.apache.org/jira/browse/HBASE-6543 Project: HBase Issue Type: Task Components: scripts Affects Versions: 0.96.0 Reporter: David S. Wang Assignee: David S. Wang Priority: Minor Labels: noob Remove copy_table.rb script as per mailing list discussion. It hasn't been maintained in a while and does not run against any recent HBase release. There is also an MR job to do the same thing that does work. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6525) bin/replication/copy_tables_desc.rb references non-existent class
David S. Wang created HBASE-6525: Summary: bin/replication/copy_tables_desc.rb references non-existent class Key: HBASE-6525 URL: https://issues.apache.org/jira/browse/HBASE-6525 Project: HBase Issue Type: Bug Components: scripts Affects Versions: 0.94.0, 0.96.0 Reporter: David S. Wang Assignee: David S. Wang Priority: Trivial $ hbase org.jruby.Main copy_tables_desc.rb NameError: cannot load Java class org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper get_proxy_or_package_under_package at org/jruby/javasupport/JavaUtilities.java:54 method_missing at file:/mnt/data/hbase/lib/jruby-complete-1.6.5.jar!/builtin/javasupport/java.rb:51 (root) at copy_tables_desc.rb:35 Removing the line that references the non-existent class seems to make the script work without any visible side-effects. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6525) bin/replication/copy_tables_desc.rb references non-existent class
[ https://issues.apache.org/jira/browse/HBASE-6525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-6525: - Attachment: HBASE-6525.patch One-liner patch to address the issue. bin/replication/copy_tables_desc.rb references non-existent class - Key: HBASE-6525 URL: https://issues.apache.org/jira/browse/HBASE-6525 Project: HBase Issue Type: Bug Components: scripts Affects Versions: 0.94.0, 0.96.0 Reporter: David S. Wang Assignee: David S. Wang Priority: Trivial Labels: noob Attachments: HBASE-6525.patch $ hbase org.jruby.Main copy_tables_desc.rb NameError: cannot load Java class org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper get_proxy_or_package_under_package at org/jruby/javasupport/JavaUtilities.java:54 method_missing at file:/mnt/data/hbase/lib/jruby-complete-1.6.5.jar!/builtin/javasupport/java.rb:51 (root) at copy_tables_desc.rb:35 Removing the line that references the non-existent class seems to make the script work without any visible side-effects. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6525) bin/replication/copy_tables_desc.rb references non-existent class
[ https://issues.apache.org/jira/browse/HBASE-6525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-6525: - Status: Patch Available (was: Open) Attached patch to run through the robot. bin/replication/copy_tables_desc.rb references non-existent class - Key: HBASE-6525 URL: https://issues.apache.org/jira/browse/HBASE-6525 Project: HBase Issue Type: Bug Components: scripts Affects Versions: 0.94.0, 0.96.0 Reporter: David S. Wang Assignee: David S. Wang Priority: Trivial Labels: noob Attachments: HBASE-6525.patch $ hbase org.jruby.Main copy_tables_desc.rb NameError: cannot load Java class org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper get_proxy_or_package_under_package at org/jruby/javasupport/JavaUtilities.java:54 method_missing at file:/mnt/data/hbase/lib/jruby-complete-1.6.5.jar!/builtin/javasupport/java.rb:51 (root) at copy_tables_desc.rb:35 Removing the line that references the non-existent class seems to make the script work without any visible side-effects. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5189) Add metrics to keep track of region-splits in RS
[ https://issues.apache.org/jira/browse/HBASE-5189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13429327#comment-13429327 ] David S. Wang commented on HBASE-5189: -- Matteo, my non-binding opintion is that I think moving to PersistentMetricsTimeVaryingRate would be the right idea here in order to make this metric useful. Add metrics to keep track of region-splits in RS Key: HBASE-5189 URL: https://issues.apache.org/jira/browse/HBASE-5189 Project: HBase Issue Type: Improvement Components: metrics, regionserver Affects Versions: 0.90.5, 0.92.0 Reporter: Mubarak Seyed Assignee: Mubarak Seyed Priority: Minor Labels: noob Fix For: 0.94.0 Attachments: HBASE-5189-persistent.patch, HBASE-5189.trunk.v1.patch, HBASE-5189.trunk.v2.patch For write-heavy workload with region-size 1 GB, region-split is considerably high. We do normally grep the NN log (grep mkdir*.split NN.log | sort | uniq -c) to get the count. I would like to have a counter incremented each time region-split execution succeeds and this counter exposed via the metrics stuff in HBase. - regionSplitSuccessCount - regionSplitFailureCount (will help us to correlate the timestamp range in RS logs across all RS) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5189) Add metrics to keep track of region-splits in RS
[ https://issues.apache.org/jira/browse/HBASE-5189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13429328#comment-13429328 ] David S. Wang commented on HBASE-5189: -- ... for the reason you state (hit return too quickly). Add metrics to keep track of region-splits in RS Key: HBASE-5189 URL: https://issues.apache.org/jira/browse/HBASE-5189 Project: HBase Issue Type: Improvement Components: metrics, regionserver Affects Versions: 0.90.5, 0.92.0 Reporter: Mubarak Seyed Assignee: Mubarak Seyed Priority: Minor Labels: noob Fix For: 0.94.0 Attachments: HBASE-5189-persistent.patch, HBASE-5189.trunk.v1.patch, HBASE-5189.trunk.v2.patch For write-heavy workload with region-size 1 GB, region-split is considerably high. We do normally grep the NN log (grep mkdir*.split NN.log | sort | uniq -c) to get the count. I would like to have a counter incremented each time region-split execution succeeds and this counter exposed via the metrics stuff in HBase. - regionSplitSuccessCount - regionSplitFailureCount (will help us to correlate the timestamp range in RS logs across all RS) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6419) PersistentMetricsTimeVaryingRate gets used for non-time-based metrics (part2 of HBASE-6220)
[ https://issues.apache.org/jira/browse/HBASE-6419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13417255#comment-13417255 ] David S. Wang commented on HBASE-6419: -- +1 on the review. PersistentMetricsTimeVaryingRate gets used for non-time-based metrics (part2 of HBASE-6220) --- Key: HBASE-6419 URL: https://issues.apache.org/jira/browse/HBASE-6419 Project: HBase Issue Type: Improvement Reporter: stack Assignee: Paul Cavallaro Attachments: ServerMetrics_HBASE_6220_Flush_Metrics.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6382) Upgrade Jersey to 1.8 to match Hadoop 1 and 2
[ https://issues.apache.org/jira/browse/HBASE-6382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13416512#comment-13416512 ] David S. Wang commented on HBASE-6382: -- @Jon: I confirmed that Hadoop 1.0 is using Jersey 1.8. See: http://svn.apache.org/viewvc/hadoop/common/branches/branch-1/ivy/libraries.properties?view=log and search for jersey. Upgrade Jersey to 1.8 to match Hadoop 1 and 2 - Key: HBASE-6382 URL: https://issues.apache.org/jira/browse/HBASE-6382 Project: HBase Issue Type: Improvement Components: rest Affects Versions: 0.92.2, 0.96.0, 0.94.2 Reporter: David S. Wang Assignee: David S. Wang Fix For: 0.92.2, 0.96.0, 0.94.1 Attachments: HBASE-6382-trunk.patch Upgrade Jersey dependency from 1.4 to 1.8 to match Hadoop dependencies. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6382) Upgrade Jersey to 1.8 to match Hadoop 1 and 2
[ https://issues.apache.org/jira/browse/HBASE-6382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-6382: - Attachment: HBASE-6382-trunk.patch Upgrade Jersey to 1.8 to match Hadoop 1 and 2 - Key: HBASE-6382 URL: https://issues.apache.org/jira/browse/HBASE-6382 Project: HBase Issue Type: Improvement Components: rest Affects Versions: 0.90.6, 0.92.1, 0.94.0, 0.96.0 Reporter: David S. Wang Assignee: David S. Wang Attachments: HBASE-6382-trunk.patch Upgrade Jersey dependency from 1.4 to 1.8 to match Hadoop dependencies. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6382) Upgrade Jersey to 1.8 to match Hadoop 1 and 2
[ https://issues.apache.org/jira/browse/HBASE-6382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-6382: - Status: Patch Available (was: Open) Trunk patch. Upgrade Jersey to 1.8 to match Hadoop 1 and 2 - Key: HBASE-6382 URL: https://issues.apache.org/jira/browse/HBASE-6382 Project: HBase Issue Type: Improvement Components: rest Affects Versions: 0.94.0, 0.92.1, 0.90.6, 0.96.0 Reporter: David S. Wang Assignee: David S. Wang Attachments: HBASE-6382-trunk.patch Upgrade Jersey dependency from 1.4 to 1.8 to match Hadoop dependencies. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6382) Upgrade Jersey to 1.8 to match Hadoop 1 and 2
[ https://issues.apache.org/jira/browse/HBASE-6382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415422#comment-13415422 ] David S. Wang commented on HBASE-6382: -- Patch passed all REST tests twice using mvn test -Dtest=org.apache.hadoop.hbase.rest.* Output below: [INFO] [INFO] Reactor Summary: [INFO] [INFO] HBase . SUCCESS [2.569s] [INFO] HBase - Common SUCCESS [1.944s] [INFO] HBase - Server SUCCESS [2:19.009s] [INFO] HBase - Integration Tests . SUCCESS [1.545s] [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 2:25.492s [INFO] Finished at: Mon Jul 16 10:32:59 PDT 2012 [INFO] Final Memory: 28M/344M [INFO] I tried to run the whole enchilada, but there are unrelated test failures. Upgrade Jersey to 1.8 to match Hadoop 1 and 2 - Key: HBASE-6382 URL: https://issues.apache.org/jira/browse/HBASE-6382 Project: HBase Issue Type: Improvement Components: rest Affects Versions: 0.90.6, 0.92.1, 0.94.0, 0.96.0 Reporter: David S. Wang Assignee: David S. Wang Attachments: HBASE-6382-trunk.patch Upgrade Jersey dependency from 1.4 to 1.8 to match Hadoop dependencies. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6382) Upgrade Jersey to 1.8 to match Hadoop 1 and 2
[ https://issues.apache.org/jira/browse/HBASE-6382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415687#comment-13415687 ] David S. Wang commented on HBASE-6382: -- I also built and ran the same tests with mvn -Dtest=org.apache.hadoop.hbase.rest.* -Dhadoop.profile=2.0 test to make sure that Hadoop 2 was covered. Same results. Upgrade Jersey to 1.8 to match Hadoop 1 and 2 - Key: HBASE-6382 URL: https://issues.apache.org/jira/browse/HBASE-6382 Project: HBase Issue Type: Improvement Components: rest Affects Versions: 0.90.6, 0.92.1, 0.94.0, 0.96.0 Reporter: David S. Wang Assignee: David S. Wang Attachments: HBASE-6382-trunk.patch Upgrade Jersey dependency from 1.4 to 1.8 to match Hadoop dependencies. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6378) the javadoc of setEnabledTable maybe not describe accurately
[
https://issues.apache.org/jira/browse/HBASE-6378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13413883#comment-13413883
]
David S. Wang commented on HBASE-6378:
--
From the patch:
+ * Sets the ENABLED state in the cache and Creates or force updates an node
to
+ * the ENABLED state for the specified table.
I'd modify the above to be:
+ * Sets the ENABLED state in the cache and creates or force updates a node to
+ * ENABLED state for the specified table.
the javadoc of setEnabledTable maybe not describe accurately
--
Key: HBASE-6378
URL: https://issues.apache.org/jira/browse/HBASE-6378
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.94.0
Reporter: zhou wenjian
Fix For: 0.94.2
Attachments: 6378.patch
/**
* Sets the ENABLED state in the cache and deletes the zookeeper node. Fails
* silently if the node is not in enabled in zookeeper
*
* @param tableName
* @throws KeeperException
*/
public void setEnabledTable(final String tableName) throws KeeperException {
setTableState(tableName, TableState.ENABLED);
}
When setEnabledTable occours ,It will update the cache and the zookeeper
node,rather than to delete the zk node.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6378) the javadoc of setEnabledTable maybe not describe accurately
[
https://issues.apache.org/jira/browse/HBASE-6378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13412843#comment-13412843
]
David S. Wang commented on HBASE-6378:
--
Yeah, this got changed as a result of HBASE-5206, which ported HBASE-5155 to
0.94+. See commit SHA 115b7ed75e274accdce57e2d7266b52dc162df60.
I think we just need to change the comment in 0.94+.
the javadoc of setEnabledTable maybe not describe accurately
--
Key: HBASE-6378
URL: https://issues.apache.org/jira/browse/HBASE-6378
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.94.0
Reporter: zhou wenjian
Fix For: 0.94.1
/**
* Sets the ENABLED state in the cache and deletes the zookeeper node. Fails
* silently if the node is not in enabled in zookeeper
*
* @param tableName
* @throws KeeperException
*/
public void setEnabledTable(final String tableName) throws KeeperException {
setTableState(tableName, TableState.ENABLED);
}
When setEnabledTable occours ,It will update the cache and the zookeeper
node,rather than to delete the zk node.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6382) Upgrade Jersey to 1.8 to match Hadoop 2
David S. Wang created HBASE-6382: Summary: Upgrade Jersey to 1.8 to match Hadoop 2 Key: HBASE-6382 URL: https://issues.apache.org/jira/browse/HBASE-6382 Project: HBase Issue Type: Improvement Components: rest Affects Versions: 0.94.0, 0.92.1, 0.90.6, 0.96.0 Reporter: David S. Wang Assignee: David S. Wang Upgrade Jersey dependency from 1.4 to 1.8 to match Hadoop 2 dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6382) Upgrade Jersey to 1.8 to match Hadoop 1 and 2
[ https://issues.apache.org/jira/browse/HBASE-6382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-6382: - Description: Upgrade Jersey dependency from 1.4 to 1.8 to match Hadoop dependencies. (was: Upgrade Jersey dependency from 1.4 to 1.8 to match Hadoop 2 dependency.) Summary: Upgrade Jersey to 1.8 to match Hadoop 1 and 2 (was: Upgrade Jersey to 1.8 to match Hadoop 2) Upgrade Jersey to 1.8 to match Hadoop 1 and 2 - Key: HBASE-6382 URL: https://issues.apache.org/jira/browse/HBASE-6382 Project: HBase Issue Type: Improvement Components: rest Affects Versions: 0.90.6, 0.92.1, 0.94.0, 0.96.0 Reporter: David S. Wang Assignee: David S. Wang Upgrade Jersey dependency from 1.4 to 1.8 to match Hadoop dependencies. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6382) Upgrade Jersey to 1.8 to match Hadoop 1 and 2
[ https://issues.apache.org/jira/browse/HBASE-6382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13413084#comment-13413084 ] David S. Wang commented on HBASE-6382: -- Title is changed. The patch is trivial; running through unit tests now. I'd like to get this into all versions of HBase if folks think that is good. Upgrade Jersey to 1.8 to match Hadoop 1 and 2 - Key: HBASE-6382 URL: https://issues.apache.org/jira/browse/HBASE-6382 Project: HBase Issue Type: Improvement Components: rest Affects Versions: 0.90.6, 0.92.1, 0.94.0, 0.96.0 Reporter: David S. Wang Assignee: David S. Wang Upgrade Jersey dependency from 1.4 to 1.8 to match Hadoop dependencies. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6382) Upgrade Jersey to 1.8 to match Hadoop 1 and 2
[ https://issues.apache.org/jira/browse/HBASE-6382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13413254#comment-13413254 ] David S. Wang commented on HBASE-6382: -- That's why it's taking a while to run through all of the tests. I don't think it will break anything, but I like to make sure by running the unit tests and issuing some REST commands. But I'll get there. ;) Upgrade Jersey to 1.8 to match Hadoop 1 and 2 - Key: HBASE-6382 URL: https://issues.apache.org/jira/browse/HBASE-6382 Project: HBase Issue Type: Improvement Components: rest Affects Versions: 0.90.6, 0.92.1, 0.94.0, 0.96.0 Reporter: David S. Wang Assignee: David S. Wang Upgrade Jersey dependency from 1.4 to 1.8 to match Hadoop dependencies. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6220) PersistentMetricsTimeVaryingRate gets used for non-time-based metrics
[ https://issues.apache.org/jira/browse/HBASE-6220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13410331#comment-13410331 ] David S. Wang commented on HBASE-6220: -- Apologies for not getting to this earlier. I noticed that the flush metrics were not changed in the patch. Was there a reason for that? PersistentMetricsTimeVaryingRate gets used for non-time-based metrics - Key: HBASE-6220 URL: https://issues.apache.org/jira/browse/HBASE-6220 Project: HBase Issue Type: Bug Components: metrics Affects Versions: 0.96.0 Reporter: David S. Wang Assignee: Paul Cavallaro Priority: Minor Labels: noob Attachments: ServerMetrics_HBASE_6220.patch PersistentMetricsTimeVaryingRate gets used for metrics that are not time-based, leading to confusing names such as avg_time for compaction size, etc. You hav to read the code in order to understand that this is actually referring to bytes, not seconds. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6342) Setup apache jenkins trunk builds with stable and flaky builds
[ https://issues.apache.org/jira/browse/HBASE-6342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13410340#comment-13410340 ] David S. Wang commented on HBASE-6342: -- Thanks for looking at this, Jon. I think both the flaky test lists and script modifications should go into the repository, as they should be treated as first-class changes if we do this. Setup apache jenkins trunk builds with stable and flaky builds -- Key: HBASE-6342 URL: https://issues.apache.org/jira/browse/HBASE-6342 Project: HBase Issue Type: Task Reporter: Jonathan Hsieh HBase trunk builds (and several others) have a set of known flaky tests. We could setup the current jenkins builds, or the hadoopqa builds to run the known good set separately from the flaky set. This would be helpful because it would expose known flaky tests, reduce the noise due to flapping test runs, and potentially improve our discipline with unit tests and CI going forwards. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6362) Enhance test-patch.sh script to recognize images / non-trunk patches
[ https://issues.apache.org/jira/browse/HBASE-6362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13410425#comment-13410425 ] David S. Wang commented on HBASE-6362: -- What will be the behavior if trunk or TRUNK is not specified? Enhance test-patch.sh script to recognize images / non-trunk patches Key: HBASE-6362 URL: https://issues.apache.org/jira/browse/HBASE-6362 Project: HBase Issue Type: Bug Reporter: Zhihong Ted Yu When user uploads logs / images / non-trunk patches, Hadoop QA would complain that the file couldn't be applied as a patch (for trunk). We should make this script smarter by recognizing image files and non-trunk patches. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6362) Enhance test-patch.sh script to recognize images / non-trunk patches
[ https://issues.apache.org/jira/browse/HBASE-6362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13410445#comment-13410445 ] David S. Wang commented on HBASE-6362: -- My thoughts: I like making the branch name the exact name of the branch as it exists in the repository. It would be great to have some sort of guidance message when attaching a file to the JIRA, stating if this is a patch, follow these naming guidelines. Otherwise we will get lots of questions from new folks about what to do. If we have such a message, then we can probably be a bit more strict on what format the name will be in (e.g. only .patch instead of .txt, etc.) if you'd like. Enhance test-patch.sh script to recognize images / non-trunk patches Key: HBASE-6362 URL: https://issues.apache.org/jira/browse/HBASE-6362 Project: HBase Issue Type: Bug Reporter: Zhihong Ted Yu When user uploads logs / images / non-trunk patches, Hadoop QA would complain that the file couldn't be applied as a patch (for trunk). We should make this script smarter by recognizing image files and non-trunk patches. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6362) Enhance test-patch.sh script to recognize images / non-trunk patches
[ https://issues.apache.org/jira/browse/HBASE-6362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13410974#comment-13410974 ] David S. Wang commented on HBASE-6362: -- OK, if we are only filtering for trunk patches vs. everything else, then your approach is fine with me. Enhance test-patch.sh script to recognize images / non-trunk patches Key: HBASE-6362 URL: https://issues.apache.org/jira/browse/HBASE-6362 Project: HBase Issue Type: Bug Reporter: Zhihong Ted Yu When user uploads logs / images / non-trunk patches, Hadoop QA would complain that the file couldn't be applied as a patch (for trunk). We should make this script smarter by recognizing image files and non-trunk patches. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6258) Backport some region splitting fixes into 0.90.7
[ https://issues.apache.org/jira/browse/HBASE-6258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-6258: - Resolution: Won't Fix Status: Resolved (was: Patch Available) There's not enough value here to do the backports into 0.90. Backport some region splitting fixes into 0.90.7 Key: HBASE-6258 URL: https://issues.apache.org/jira/browse/HBASE-6258 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.90.6 Reporter: David S. Wang Assignee: David S. Wang Attachments: HBASE-4816+4881+5189+6158.patch Issue tracking backport of some relatively small region splitting fixes into 0.90.7: HBASE-4816: Regionserver wouldn't go down because split happened exactly at same time we issued bulk user region close call on our way out - fixed in 0.92 HBASE-4881: Unhealthy region is on service caused by rollback of region splitting - fixed in 0.92 HBASE-5189: Add metrics to keep track of region-splits in RS - fixed in 0.94 HBASE-6158: Data loss if the words 'merges' or 'splits' are used as Column Family name - fixed in 0.92 and 0.94 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-6091) Come up with strawman proposal for RC testing matrix
[ https://issues.apache.org/jira/browse/HBASE-6091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang resolved HBASE-6091. -- Resolution: Implemented Come up with strawman proposal for RC testing matrix Key: HBASE-6091 URL: https://issues.apache.org/jira/browse/HBASE-6091 Project: HBase Issue Type: Bug Components: documentation Affects Versions: 0.96.0 Reporter: David S. Wang Assignee: David S. Wang -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6258) Backport some region splitting fixes into 0.90.7
[ https://issues.apache.org/jira/browse/HBASE-6258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399934#comment-13399934 ] David S. Wang commented on HBASE-6258: -- JD, Thanks for the timely comments. Yes, including SplitRequest is an error on my end. I will remove it and apply the few changes that went into it accordingly. Changing the SPLITDIR is from HBASE-6158, which went into 0.92.2 and 0.94.1. I think you bring up a good point, and it sounds like we will have the same problem in those releases as well. Perhaps the original patch needs to be changed to handle this case for the aforementioned releases. I'll take a look at what region splitting tests we have today and add as necessary. It looks like we need to add at least a test for when SPLITDIR is changed. Backport some region splitting fixes into 0.90.7 Key: HBASE-6258 URL: https://issues.apache.org/jira/browse/HBASE-6258 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.90.6 Reporter: David S. Wang Assignee: David S. Wang Attachments: HBASE-4816+4881+5189+6158.patch Issue tracking backport of some relatively small region splitting fixes into 0.90.7: HBASE-4816: Regionserver wouldn't go down because split happened exactly at same time we issued bulk user region close call on our way out - fixed in 0.92 HBASE-4881: Unhealthy region is on service caused by rollback of region splitting - fixed in 0.92 HBASE-5189: Add metrics to keep track of region-splits in RS - fixed in 0.94 HBASE-6158: Data loss if the words 'merges' or 'splits' are used as Column Family name - fixed in 0.92 and 0.94 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6258) Backport some region splitting fixes into 0.90.7
[ https://issues.apache.org/jira/browse/HBASE-6258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399946#comment-13399946 ] David S. Wang commented on HBASE-6258: -- I see this comment from HBASE-6158: https://issues.apache.org/jira/browse/HBASE-6158?focusedCommentId=13288890page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13288890 Is this change incompatible for rolling upgrade? I don't think so. These are temporary work folders which get cleaned up routinely and particularly at Region Server (re)start for each region through HRegion.initialize(). However, a possible side effect could be that merges and splits folders created by pre-upgrade code (which somehow did not get cleaned up during a shutdown) may continue to exist on the file system as the cleanup code would no longer be looking for them. But this is better fixed in the upgrade scripts by deleting these folders if they are found to be a work folder instead of a CF container folder. I'll continue the conversation over on HBASE-6158. Backport some region splitting fixes into 0.90.7 Key: HBASE-6258 URL: https://issues.apache.org/jira/browse/HBASE-6258 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.90.6 Reporter: David S. Wang Assignee: David S. Wang Attachments: HBASE-4816+4881+5189+6158.patch Issue tracking backport of some relatively small region splitting fixes into 0.90.7: HBASE-4816: Regionserver wouldn't go down because split happened exactly at same time we issued bulk user region close call on our way out - fixed in 0.92 HBASE-4881: Unhealthy region is on service caused by rollback of region splitting - fixed in 0.92 HBASE-5189: Add metrics to keep track of region-splits in RS - fixed in 0.94 HBASE-6158: Data loss if the words 'merges' or 'splits' are used as Column Family name - fixed in 0.92 and 0.94 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6158) Data loss if the words 'merges' or 'splits' are used as Column Family name
[ https://issues.apache.org/jira/browse/HBASE-6158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399949#comment-13399949 ] David S. Wang commented on HBASE-6158: -- This change does make upgrade scripts more complex as noted earlier in this JIRA. Having to change the scripts to detect this condition is potentially problematic: if splits or merges was an actual user table before the upgrade, what does the script do in that case? Also, is merges/MERGEDIR actually created anywhere? I don't see any such call in the code in trunk or in earlier releases. Data loss if the words 'merges' or 'splits' are used as Column Family name -- Key: HBASE-6158 URL: https://issues.apache.org/jira/browse/HBASE-6158 Project: HBase Issue Type: Bug Components: master, regionserver Affects Versions: 0.94.0 Reporter: Aditya Kishore Assignee: Aditya Kishore Fix For: 0.92.2, 0.94.1 Attachments: HBASE-6158_94.patch, HBASE-6158_trunk.patch If a table is creates with either 'merges' or 'splits' as one of the Column Family name it can never be flushed to the disk even though the table creation (and data population) succeeds. The reason for this is that these two are used as temporary directory names inside the region folder or merge and splits respectively and hence conflicts with the directories created for CF with same name. A simple fix would be to uses .merges' and .splits as the working folder (patch attached). This will also be consistent with other work folder names. An alternate fix would be to declare these words (and other similar) as reserve words and throw exception when they are used. However, I do find the alternate approach as unnecessarily restrictive. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6258) Backport some region splitting fixes into 0.90.7
David S. Wang created HBASE-6258: Summary: Backport some region splitting fixes into 0.90.7 Key: HBASE-6258 URL: https://issues.apache.org/jira/browse/HBASE-6258 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.90.6 Reporter: David S. Wang Assignee: David S. Wang Issue tracking backport of some relatively small region splitting fixes into 0.90.7: HBASE-4816: Regionserver wouldn't go down because split happened exactly at same time we issued bulk user region close call on our way out - fixed in 0.92 HBASE-4881: Unhealthy region is on service caused by rollback of region splitting - fixed in 0.92 HBASE-5189: Add metrics to keep track of region-splits in RS - fixed in 0.94 HBASE-6158: Data loss if the words 'merges' or 'splits' are used as Column Family name - fixed in 0.92 and 0.94 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6258) Backport some region splitting fixes into 0.90.7
[ https://issues.apache.org/jira/browse/HBASE-6258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-6258: - Status: Patch Available (was: Open) Patch for the backport into 0.90. Backport some region splitting fixes into 0.90.7 Key: HBASE-6258 URL: https://issues.apache.org/jira/browse/HBASE-6258 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.90.6 Reporter: David S. Wang Assignee: David S. Wang Attachments: HBASE-4816+4881+5189+6158.patch Issue tracking backport of some relatively small region splitting fixes into 0.90.7: HBASE-4816: Regionserver wouldn't go down because split happened exactly at same time we issued bulk user region close call on our way out - fixed in 0.92 HBASE-4881: Unhealthy region is on service caused by rollback of region splitting - fixed in 0.92 HBASE-5189: Add metrics to keep track of region-splits in RS - fixed in 0.94 HBASE-6158: Data loss if the words 'merges' or 'splits' are used as Column Family name - fixed in 0.92 and 0.94 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6258) Backport some region splitting fixes into 0.90.7
[ https://issues.apache.org/jira/browse/HBASE-6258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-6258: - Attachment: HBASE-4816+4881+5189+6158.patch Backport some region splitting fixes into 0.90.7 Key: HBASE-6258 URL: https://issues.apache.org/jira/browse/HBASE-6258 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.90.6 Reporter: David S. Wang Assignee: David S. Wang Attachments: HBASE-4816+4881+5189+6158.patch Issue tracking backport of some relatively small region splitting fixes into 0.90.7: HBASE-4816: Regionserver wouldn't go down because split happened exactly at same time we issued bulk user region close call on our way out - fixed in 0.92 HBASE-4881: Unhealthy region is on service caused by rollback of region splitting - fixed in 0.92 HBASE-5189: Add metrics to keep track of region-splits in RS - fixed in 0.94 HBASE-6158: Data loss if the words 'merges' or 'splits' are used as Column Family name - fixed in 0.92 and 0.94 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6220) PersistentMetricsTimeVaryingRate gets used for non-time-based metrics
David S. Wang created HBASE-6220: Summary: PersistentMetricsTimeVaryingRate gets used for non-time-based metrics Key: HBASE-6220 URL: https://issues.apache.org/jira/browse/HBASE-6220 Project: HBase Issue Type: Bug Components: metrics Affects Versions: 0.96.0 Reporter: David S. Wang Priority: Minor PersistentMetricsTimeVaryingRate gets used for metrics that are not time-based, leading to confusing names such as avg_time for compaction size, etc. You hav to read the code in order to understand that this is actually referring to bytes, not seconds. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6190) TestPriorityCompactionQueue throws NPE
[ https://issues.apache.org/jira/browse/HBASE-6190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13291168#comment-13291168 ] David S. Wang commented on HBASE-6190: -- +1 TestPriorityCompactionQueue throws NPE -- Key: HBASE-6190 URL: https://issues.apache.org/jira/browse/HBASE-6190 Project: HBase Issue Type: Test Components: test Affects Versions: 0.90.6 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Fix For: 0.90.7 Attachments: hbase-6190.patch 0.90 only. java.lang.NullPointerException at org.apache.hadoop.hbase.regionserver.HRegion.getRegionId(HRegion.java:622) at org.apache.hadoop.hbase.regionserver.CompactSplitThread.preRequest(CompactSplitThread.java:78) at org.apache.hadoop.hbase.regionserver.PriorityCompactionQueue.addToRegionsInQueue(PriorityCompactionQueue.java:146) at org.apache.hadoop.hbase.regionserver.PriorityCompactionQueue.add(PriorityCompactionQueue.java:188) at org.apache.hadoop.hbase.regionserver.TestPriorityCompactionQueue.addRegion(TestPriorityCompactionQueue.java:79) at org.apache.hadoop.hbase.regionserver.TestPriorityCompactionQueue.testPriorityQueue(TestPriorityCompactionQueue.java:107) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Metho -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6091) Come up with strawman proposal for RC testing matrix
[ https://issues.apache.org/jira/browse/HBASE-6091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13289517#comment-13289517 ] David S. Wang commented on HBASE-6091: -- My wiki name is DavidWang. Come up with strawman proposal for RC testing matrix Key: HBASE-6091 URL: https://issues.apache.org/jira/browse/HBASE-6091 Project: HBase Issue Type: Bug Components: documentation Affects Versions: 0.96.0 Reporter: David S. Wang Assignee: David S. Wang -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6091) Come up with strawman proposal for RC testing matrix
[ https://issues.apache.org/jira/browse/HBASE-6091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13289649#comment-13289649 ] David S. Wang commented on HBASE-6091: -- I posted it at: http://wiki.apache.org/hadoop/ReleaseCandidateTestMatrix and made a link to it from: http://wiki.apache.org/hadoop/Hbase/HowToRelease I had to edit the source a bit as the Apache JIRA is not Atlassian format. What are the next steps here? Come up with strawman proposal for RC testing matrix Key: HBASE-6091 URL: https://issues.apache.org/jira/browse/HBASE-6091 Project: HBase Issue Type: Bug Components: documentation Affects Versions: 0.96.0 Reporter: David S. Wang Assignee: David S. Wang -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6091) Come up with strawman proposal for RC testing matrix
[ https://issues.apache.org/jira/browse/HBASE-6091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13289681#comment-13289681 ] David S. Wang commented on HBASE-6091: -- YCSB is something we use quite a bit and is easy to setup. I don't see the lack of ownership of the tool as something that would preclude it from being used at least for some basic first-level testing. Why don't we have both? I've added LoadTestTool to the matrix. Come up with strawman proposal for RC testing matrix Key: HBASE-6091 URL: https://issues.apache.org/jira/browse/HBASE-6091 Project: HBase Issue Type: Bug Components: documentation Affects Versions: 0.96.0 Reporter: David S. Wang Assignee: David S. Wang -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6091) Come up with strawman proposal for RC testing matrix
[ https://issues.apache.org/jira/browse/HBASE-6091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13289717#comment-13289717 ] David S. Wang commented on HBASE-6091: -- Probably ... IIRC we are using YCSB 0.14. In any case, it's not like every single field in the matrix has to be filled out before an RC is +1'ed; that would make RCs take a very long time indeed. I think leaving them both in is good for now; if it turns out that some rows never get filled out, we can always remove them later. Come up with strawman proposal for RC testing matrix Key: HBASE-6091 URL: https://issues.apache.org/jira/browse/HBASE-6091 Project: HBase Issue Type: Bug Components: documentation Affects Versions: 0.96.0 Reporter: David S. Wang Assignee: David S. Wang -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6091) Come up with strawman proposal for RC testing matrix
[ https://issues.apache.org/jira/browse/HBASE-6091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13289046#comment-13289046 ] David S. Wang commented on HBASE-6091: -- Here's a first draft in Atlassian JIRA format. I can convert this to something else if needed. Release version: (e.g. 0.94.1 RC2) || Category || Reporter || Underlying Hadoop version || Tests run || +1/0/-1 || Results details || || Master || | Log splitting | Dave | Hadoop 2.0.0 | e.g. Configured no distributed log splitting, killed RS | +1 | e.g. looked at logs, verified no data loss | | Distributed log splitting | | | | | | | Load balancer | | | | | | | Regions in transition | | Hadoop 1.0.3 | e.g. unassign region(s) from shell, see that they get reassigned after a while | | e.g. check logs that region was in transition for a little while, not stuck there | || Region server || | Flushes | | | | | | | Compactions | | | | | | | Splits | | | | | | | Block cache | | | | | | || Client || | Java | | | | | | | Thrift | | | | | | | REST | | | | | | | Shell | | | | | | || Tools || | scripts (e.g. graceful stop) | | | | | | | Bulk load | | | | | | | CopyTable | | | | | | | importtsv | | | | | | | hbck | | | | | | || Metrics || || Coprocessors || || Security || | Kerberos authentication | | | | | | | ACL coprocessor | | | | | | | TokenProvider coprocessor | | | | | | || Replication || | Master-slave | | | | | | | Master-master | | | | | | | Cyclic | | | | | | || Compression || | Snappy | | | | | | | LZO | | | | | | || Performance || | YCSB | | | | | | | PerformanceEvaluation | | | | | | || Compatibility || | Client-server compatibility between minor versions | | | | | | | Server-server compatibility between minor versions | | | | | | | Client-server compatibility between major versions -+1 | | | | | | | Server-server compatibility between major versions -+1 | | | | | | | Client-server compatibility between major versions -+1 | | | | | | | Server-server compatibility between major versions -+1 | | | | | | | Manual rolling restart between this version and previous version (minor or major) | | | | | | || Configuration || || System || | TestLoadAndVerify | | | | | | | TestAcidGuarantees | | | | | | | TestConcurrentScanAndPut | | | | | | | Failover to backup master | | | | | | | Kill RS | | | | | | | Kill RS holding .META. | | | | | | | Kill RS holding -ROOT- | | | | | | || New features || | e.g. Snapshots | | | | | | || Documentation || | Review section X of reference guide | | | | | | | Review bundled documentation | | | | | | Come up with strawman proposal for RC testing matrix Key: HBASE-6091 URL: https://issues.apache.org/jira/browse/HBASE-6091 Project: HBase Issue Type: Bug Components: documentation Affects Versions: 0.96.0 Reporter: David S. Wang Assignee: David S. Wang -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6091) Come up with strawman proposal for RC testing matrix
[ https://issues.apache.org/jira/browse/HBASE-6091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13287062#comment-13287062 ] David S. Wang commented on HBASE-6091: -- I signed up for a wiki page account, but cannot seem to create a new page, or at least I could not figure out how to do that. Can I get perms to create and edit my own page on the hbase wiki? Otherwise I can just attach what I have so far in ugly text format. Come up with strawman proposal for RC testing matrix Key: HBASE-6091 URL: https://issues.apache.org/jira/browse/HBASE-6091 Project: HBase Issue Type: Bug Components: documentation Affects Versions: 0.96.0 Reporter: David S. Wang Assignee: David S. Wang -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6091) Come up with strawman proposal for RC testing matrix
[ https://issues.apache.org/jira/browse/HBASE-6091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13285804#comment-13285804 ] David S. Wang commented on HBASE-6091: -- I have some content ready; however is there some format I should post it in (e.g. Confluence wiki, etc.)? Come up with strawman proposal for RC testing matrix Key: HBASE-6091 URL: https://issues.apache.org/jira/browse/HBASE-6091 Project: HBase Issue Type: Bug Components: documentation Affects Versions: 0.96.0 Reporter: David S. Wang Assignee: David S. Wang -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6091) Come up with strawman proposal for RC testing matrix
David S. Wang created HBASE-6091: Summary: Come up with strawman proposal for RC testing matrix Key: HBASE-6091 URL: https://issues.apache.org/jira/browse/HBASE-6091 Project: HBase Issue Type: Bug Components: documentation Affects Versions: 0.96.0 Reporter: David S. Wang Assignee: David S. Wang -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6080) site.xml - adding ReviewBoard to main page left-hand nav
[ https://issues.apache.org/jira/browse/HBASE-6080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13282035#comment-13282035 ] David S. Wang commented on HBASE-6080: -- +1 site.xml - adding ReviewBoard to main page left-hand nav Key: HBASE-6080 URL: https://issues.apache.org/jira/browse/HBASE-6080 Project: HBase Issue Type: Improvement Reporter: Doug Meil Assignee: Doug Meil Priority: Trivial Attachments: src_hbase_6080.patch By request, adding ReviewBoard to left-hand nav on website -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6009) Changes for HBASE-5209 are technically incompatible
[ https://issues.apache.org/jira/browse/HBASE-6009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13279067#comment-13279067 ] David S. Wang commented on HBASE-6009: -- The immediate issue here is that HBASE-5209 was committed in 0.92.1, and that broke compatibility with 0.92.0. I suppose anyone who cares about 0.92 branch has moved to 0.92.1 so that there is no practical hit. You are right that adding a size would be another incompatible change, hence my later comment about let's just not make any more changes until 0.96. :D Anyway in the absence of any changes, I can at least add a release note to 0.92.1 stating this incompatibility with 0.92.0. I'll use this JIRA to track that. Changes for HBASE-5209 are technically incompatible --- Key: HBASE-6009 URL: https://issues.apache.org/jira/browse/HBASE-6009 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.92.1, 0.94.0 Reporter: David S. Wang The additions to add backup masters to ClusterStatus are technically incompatible between clients and servers. Older clients will basically not read the extra bits that the newer server pushes for the backup masters, thus screwing up the serialization for the next blob in the pipe. For the Writable, we can add a total size field for ClusterStatus at the beginning, or we can have start and end markers. I can make a patch for either approach; interested in whatever folks have to suggest. Would be good to get this in soon to limit the damage to 0.92.1 (don't know if we can get this in in time for 0.94.0). Either change will make us forward-compatible starting with when the change goes in, but will not fix the backwards incompatibility, which we will have to mark with a release note as there have already been releases with this change. Hopefully we can do this in a cleaner way when wire compat rolls around in 0.96. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6009) Changes for HBASE-5209 are technically incompatible
[ https://issues.apache.org/jira/browse/HBASE-6009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13279093#comment-13279093 ] David S. Wang commented on HBASE-6009: -- I wouldn't back it out ... I think that would just create another incompatibility event between 0.92.1 and 0.92.2. The original goal was to get this committed for 0.92.0 (which would have avoided this), but I suppose we didn't make it for 0.92.0. I should have said something about not doing this once 0.92.0 came out, so my apologies on that. Changes for HBASE-5209 are technically incompatible --- Key: HBASE-6009 URL: https://issues.apache.org/jira/browse/HBASE-6009 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.92.1, 0.94.0 Reporter: David S. Wang The additions to add backup masters to ClusterStatus are technically incompatible between clients and servers. Older clients will basically not read the extra bits that the newer server pushes for the backup masters, thus screwing up the serialization for the next blob in the pipe. For the Writable, we can add a total size field for ClusterStatus at the beginning, or we can have start and end markers. I can make a patch for either approach; interested in whatever folks have to suggest. Would be good to get this in soon to limit the damage to 0.92.1 (don't know if we can get this in in time for 0.94.0). Either change will make us forward-compatible starting with when the change goes in, but will not fix the backwards incompatibility, which we will have to mark with a release note as there have already been releases with this change. Hopefully we can do this in a cleaner way when wire compat rolls around in 0.96. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6009) Changes for HBASE-5209 are technically incompatible
[ https://issues.apache.org/jira/browse/HBASE-6009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13277810#comment-13277810 ] David S. Wang commented on HBASE-6009: -- I looked at the total size field option for this, starting from the write case. To calculate total size written, you have to know how many bytes were written for each write() call on ClusterStatus, including any objects contained inside it. The DataOutput interface for Writables doesn't have a way to return how many bytes were written to the stream. This is not a problem for primitive types as we can figure that out trivially. Even for somewhat more complicated situations such as modified UTF-8s written with the writeUTF call, the number of written bytes for a String can at least be calculated based on the formula for modified UTF-8 conversion. However, for calls to Object's write functions (e.g. for HRegionLoad), this becomes somewhat more problematic as there is no obvious answer as to how many bytes were written. We could use reflection to grab the fields, but then there is no guarantee that all of the fields of the Object are actually written to the stream when write() is called. So you'd have to introduce some hardcoded way of knowing how much was written for each Object, which is Bad. I'm tempted to say that we shouldn't add any more fields to ClusterStatus or similar APIs until 0.96, when hopefully our wire compatibility efforts will kick in and we can do this in a compatible way without having to jump through hoops. Changes for HBASE-5209 are technically incompatible --- Key: HBASE-6009 URL: https://issues.apache.org/jira/browse/HBASE-6009 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.92.1, 0.94.0 Reporter: David S. Wang The additions to add backup masters to ClusterStatus are technically incompatible between clients and servers. Older clients will basically not read the extra bits that the newer server pushes for the backup masters, thus screwing up the serialization for the next blob in the pipe. For the Writable, we can add a total size field for ClusterStatus at the beginning, or we can have start and end markers. I can make a patch for either approach; interested in whatever folks have to suggest. Would be good to get this in soon to limit the damage to 0.92.1 (don't know if we can get this in in time for 0.94.0). Either change will make us forward-compatible starting with when the change goes in, but will not fix the backwards incompatibility, which we will have to mark with a release note as there have already been releases with this change. Hopefully we can do this in a cleaner way when wire compat rolls around in 0.96. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-6024) Add state of load balancer to cluster status
[ https://issues.apache.org/jira/browse/HBASE-6024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang resolved HBASE-6024. -- Resolution: Duplicate Duplicate of HBASE-5953 as Ted stated. Add state of load balancer to cluster status Key: HBASE-6024 URL: https://issues.apache.org/jira/browse/HBASE-6024 Project: HBase Issue Type: Task Reporter: Gregory Chanan Assignee: Gregory Chanan Priority: Minor Fix For: 0.96.0 There currently does not seem to be anyway to see if the load balancer is actually running or not. synchronousBalanceSwitch and balanceSwitch will return the state, but require possibly turning off/on the balancer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5953) Expose the current state of the balancerSwitch
[ https://issues.apache.org/jira/browse/HBASE-5953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-5953: - Affects Version/s: (was: 0.92.1) (was: 0.94.0) Assignee: Gregory Chanan (was: David S. Wang) Assigning to Greg at his request. Removing 0.92 and 0.94 from fixVersions at least until we have a compatibility story for ClusterStatus (see HBASE-6009 for details). Expose the current state of the balancerSwitch -- Key: HBASE-5953 URL: https://issues.apache.org/jira/browse/HBASE-5953 Project: HBase Issue Type: Improvement Components: master Affects Versions: 0.96.0 Reporter: David S. Wang Assignee: Gregory Chanan Priority: Minor The balancerSwitch as in both 0.90 and 0.92.1 is implemented in such a way that it is impossible to retrieve its value without changing it: /** Turn the load balancer on or off. @param b If true, enable balancer. If false, disable balancer. @return Previous balancer value */ public boolean balanceSwitch(final boolean b); It would be nice to have a way to just get the current state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6009) Changes for HBASE-5209 are technically incompatible
David S. Wang created HBASE-6009: Summary: Changes for HBASE-5209 are technically incompatible Key: HBASE-6009 URL: https://issues.apache.org/jira/browse/HBASE-6009 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.94.0, 0.92.1 Reporter: David S. Wang The additions to add backup masters to ClusterStatus are technically incompatible between clients and servers. Older clients will basically not read the extra bits that the newer server pushes for the backup masters, thus screwing up the serialization for the next blob in the pipe. For the Writable, we can add a total size field for ClusterStatus at the beginning, or we can have start and end markers. I can make a patch for either approach; interested in whatever folks have to suggest. Would be good to get this in soon to limit the damage to 0.92.1 (don't know if we can get this in in time for 0.94.0). Either change will make us forward-compatible starting with when the change goes in, but will not fix the backwards incompatibility, which we will have to mark with a release note as there have already been releases with this change. Hopefully we can do this in a cleaner way when wire compat rolls around in 0.96. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5904) is_enabled from shell returns differently from pre- and post- HBASE-5155
[ https://issues.apache.org/jira/browse/HBASE-5904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13274101#comment-13274101 ] David S. Wang commented on HBASE-5904: -- Stack: That (backing out HBASE-5155 only) is what my patch does. is_enabled from shell returns differently from pre- and post- HBASE-5155 Key: HBASE-5904 URL: https://issues.apache.org/jira/browse/HBASE-5904 Project: HBase Issue Type: Bug Components: zookeeper Affects Versions: 0.90.6 Reporter: David S. Wang Assignee: David S. Wang Priority: Blocker Fix For: 0.90.7 Attachments: HBASE-5904.patch If I launch an hbase shell that uses HBase and ZooKeeper without HBASE-5155, against HBase servers with HBASE-5155, then is_enabled for a table always returns false even if the table is considered enabled by the servers from the logs. If I then do the same thing but with an HBase shell and ZooKeeper with HBASE-5155, then is_enabled returns as expected. If I launch an HBase shell that uses HBase and ZooKeeper without HBASE-5155, against HBase servers also without HBASE-5155, then is_enabled works as you'd expect. But if I then do the same thing but with an HBase shell and ZooKeeper with HBASE-5155, then is_enabled returns false even though the table is considered enabled by the servers from the logs. Additionally, if I then try to enable the table from the HBASE-5155-containing shell, it hangs because the ZooKeeper code waits for the ZNode to be updated with ENABLED in the data field, but what actually happens is that the ZNode gets deleted since the servers are running without HBASE-5155. I think the culprit is that the indication of how a table is considered enabled inside ZooKeeper has changed with HBASE-5155. Before HBASE-5155, a table was considered enabled if the ZNode for it did not exist. After HBASE-5155, a table is considered enabled if the ZNode for it exists and has ENABLED in its data. I think the current code is incompatible when running clients and servers where one side has HBASE-5155 and the other side does not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3899) enhance HBase RPC to support free-ing up server handler threads even if response is not ready
[ https://issues.apache.org/jira/browse/HBASE-3899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-3899: - Attachment: HBASE-3899.patch Backport to 0.90.x branch. enhance HBase RPC to support free-ing up server handler threads even if response is not ready - Key: HBASE-3899 URL: https://issues.apache.org/jira/browse/HBASE-3899 Project: HBase Issue Type: Improvement Components: ipc Reporter: dhruba borthakur Assignee: dhruba borthakur Fix For: 0.92.0 Attachments: HBASE-3899-2.patch, HBASE-3899-amend-v4.patch, HBASE-3899.patch, HBASE-3899.patch, asyncRpc.txt, asyncRpc.txt In the current implementation, the server handler thread picks up an item from the incoming callqueue, processes it and then wraps the response as a Writable and sends it back to the IPC server module. This wastes thread-resources when the thread is blocked for disk IO (transaction logging, read into block cache, etc). It would be nice if we can make the RPC Server Handler threads pick up a call from the IPC queue, hand it over to the application (e.g. HRegion), the application can queue it to be processed asynchronously and send a response back to the IPC server module saying that the response is not ready. The RPC Server Handler thread is now ready to pick up another request from the incoming callqueue. When the queued call is processed by the application, it indicates to the IPC module that the response is now ready to be sent back to the client. The RPC client continues to experience the same behaviour as before. A RPC client is synchronous and blocks till the response arrives. This RPC enhancement allows us to do very powerful things with the RegionServer. In future, we can make enhance the RegionServer's threading model to a message-passing model for better performance. We will not be limited by the number of threads in the RegionServer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3899) enhance HBase RPC to support free-ing up server handler threads even if response is not ready
[ https://issues.apache.org/jira/browse/HBASE-3899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-3899: - Fix Version/s: 0.90.7 Affects Version/s: 0.90.6 Status: Patch Available (was: Reopened) enhance HBase RPC to support free-ing up server handler threads even if response is not ready - Key: HBASE-3899 URL: https://issues.apache.org/jira/browse/HBASE-3899 Project: HBase Issue Type: Improvement Components: ipc Affects Versions: 0.90.6 Reporter: dhruba borthakur Assignee: dhruba borthakur Fix For: 0.90.7, 0.92.0 Attachments: HBASE-3899-2.patch, HBASE-3899-amend-v4.patch, HBASE-3899.patch, HBASE-3899.patch, asyncRpc.txt, asyncRpc.txt In the current implementation, the server handler thread picks up an item from the incoming callqueue, processes it and then wraps the response as a Writable and sends it back to the IPC server module. This wastes thread-resources when the thread is blocked for disk IO (transaction logging, read into block cache, etc). It would be nice if we can make the RPC Server Handler threads pick up a call from the IPC queue, hand it over to the application (e.g. HRegion), the application can queue it to be processed asynchronously and send a response back to the IPC server module saying that the response is not ready. The RPC Server Handler thread is now ready to pick up another request from the incoming callqueue. When the queued call is processed by the application, it indicates to the IPC module that the response is now ready to be sent back to the client. The RPC client continues to experience the same behaviour as before. A RPC client is synchronous and blocks till the response arrives. This RPC enhancement allows us to do very powerful things with the RegionServer. In future, we can make enhance the RegionServer's threading model to a message-passing model for better performance. We will not be limited by the number of threads in the RegionServer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-3899) enhance HBase RPC to support free-ing up server handler threads even if response is not ready
[ https://issues.apache.org/jira/browse/HBASE-3899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang reopened HBASE-3899: -- Would like to get this backported to 0.90.x branch, as HBASE-5973 depends on Delayable which is from this JIRA. enhance HBase RPC to support free-ing up server handler threads even if response is not ready - Key: HBASE-3899 URL: https://issues.apache.org/jira/browse/HBASE-3899 Project: HBase Issue Type: Improvement Components: ipc Affects Versions: 0.90.6 Reporter: dhruba borthakur Assignee: dhruba borthakur Fix For: 0.90.7, 0.92.0 Attachments: HBASE-3899-2.patch, HBASE-3899-amend-v4.patch, HBASE-3899.patch, HBASE-3899.patch, asyncRpc.txt, asyncRpc.txt In the current implementation, the server handler thread picks up an item from the incoming callqueue, processes it and then wraps the response as a Writable and sends it back to the IPC server module. This wastes thread-resources when the thread is blocked for disk IO (transaction logging, read into block cache, etc). It would be nice if we can make the RPC Server Handler threads pick up a call from the IPC queue, hand it over to the application (e.g. HRegion), the application can queue it to be processed asynchronously and send a response back to the IPC server module saying that the response is not ready. The RPC Server Handler thread is now ready to pick up another request from the incoming callqueue. When the queued call is processed by the application, it indicates to the IPC module that the response is now ready to be sent back to the client. The RPC client continues to experience the same behaviour as before. A RPC client is synchronous and blocks till the response arrives. This RPC enhancement allows us to do very powerful things with the RegionServer. In future, we can make enhance the RegionServer's threading model to a message-passing model for better performance. We will not be limited by the number of threads in the RegionServer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3899) enhance HBase RPC to support free-ing up server handler threads even if response is not ready
[ https://issues.apache.org/jira/browse/HBASE-3899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-3899: - Attachment: (was: HBASE-3899.patch) enhance HBase RPC to support free-ing up server handler threads even if response is not ready - Key: HBASE-3899 URL: https://issues.apache.org/jira/browse/HBASE-3899 Project: HBase Issue Type: Improvement Components: ipc Affects Versions: 0.90.6 Reporter: dhruba borthakur Assignee: dhruba borthakur Fix For: 0.90.7, 0.92.0 Attachments: HBASE-3899-2.patch, HBASE-3899-amend-v4.patch, HBASE-3899.patch, asyncRpc.txt, asyncRpc.txt In the current implementation, the server handler thread picks up an item from the incoming callqueue, processes it and then wraps the response as a Writable and sends it back to the IPC server module. This wastes thread-resources when the thread is blocked for disk IO (transaction logging, read into block cache, etc). It would be nice if we can make the RPC Server Handler threads pick up a call from the IPC queue, hand it over to the application (e.g. HRegion), the application can queue it to be processed asynchronously and send a response back to the IPC server module saying that the response is not ready. The RPC Server Handler thread is now ready to pick up another request from the incoming callqueue. When the queued call is processed by the application, it indicates to the IPC module that the response is now ready to be sent back to the client. The RPC client continues to experience the same behaviour as before. A RPC client is synchronous and blocks till the response arrives. This RPC enhancement allows us to do very powerful things with the RegionServer. In future, we can make enhance the RegionServer's threading model to a message-passing model for better performance. We will not be limited by the number of threads in the RegionServer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3899) enhance HBase RPC to support free-ing up server handler threads even if response is not ready
[ https://issues.apache.org/jira/browse/HBASE-3899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-3899: - Attachment: HBASE-3899.patch enhance HBase RPC to support free-ing up server handler threads even if response is not ready - Key: HBASE-3899 URL: https://issues.apache.org/jira/browse/HBASE-3899 Project: HBase Issue Type: Improvement Components: ipc Affects Versions: 0.90.6 Reporter: dhruba borthakur Assignee: dhruba borthakur Fix For: 0.90.7, 0.92.0 Attachments: HBASE-3899-2.patch, HBASE-3899-amend-v4.patch, HBASE-3899.patch, HBASE-3899.patch, asyncRpc.txt, asyncRpc.txt In the current implementation, the server handler thread picks up an item from the incoming callqueue, processes it and then wraps the response as a Writable and sends it back to the IPC server module. This wastes thread-resources when the thread is blocked for disk IO (transaction logging, read into block cache, etc). It would be nice if we can make the RPC Server Handler threads pick up a call from the IPC queue, hand it over to the application (e.g. HRegion), the application can queue it to be processed asynchronously and send a response back to the IPC server module saying that the response is not ready. The RPC Server Handler thread is now ready to pick up another request from the incoming callqueue. When the queued call is processed by the application, it indicates to the IPC module that the response is now ready to be sent back to the client. The RPC client continues to experience the same behaviour as before. A RPC client is synchronous and blocks till the response arrives. This RPC enhancement allows us to do very powerful things with the RegionServer. In future, we can make enhance the RegionServer's threading model to a message-passing model for better performance. We will not be limited by the number of threads in the RegionServer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3899) enhance HBase RPC to support free-ing up server handler threads even if response is not ready
[ https://issues.apache.org/jira/browse/HBASE-3899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-3899: - Status: Open (was: Patch Available) On second thought, I might not need this to backport HBASE-5973. enhance HBase RPC to support free-ing up server handler threads even if response is not ready - Key: HBASE-3899 URL: https://issues.apache.org/jira/browse/HBASE-3899 Project: HBase Issue Type: Improvement Components: ipc Affects Versions: 0.90.6 Reporter: dhruba borthakur Assignee: dhruba borthakur Fix For: 0.90.7, 0.92.0 Attachments: HBASE-3899-2.patch, HBASE-3899-amend-v4.patch, HBASE-3899.patch, HBASE-3899.patch, asyncRpc.txt, asyncRpc.txt In the current implementation, the server handler thread picks up an item from the incoming callqueue, processes it and then wraps the response as a Writable and sends it back to the IPC server module. This wastes thread-resources when the thread is blocked for disk IO (transaction logging, read into block cache, etc). It would be nice if we can make the RPC Server Handler threads pick up a call from the IPC queue, hand it over to the application (e.g. HRegion), the application can queue it to be processed asynchronously and send a response back to the IPC server module saying that the response is not ready. The RPC Server Handler thread is now ready to pick up another request from the incoming callqueue. When the queued call is processed by the application, it indicates to the IPC module that the response is now ready to be sent back to the client. The RPC client continues to experience the same behaviour as before. A RPC client is synchronous and blocks till the response arrives. This RPC enhancement allows us to do very powerful things with the RegionServer. In future, we can make enhance the RegionServer's threading model to a message-passing model for better performance. We will not be limited by the number of threads in the RegionServer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-3899) enhance HBase RPC to support free-ing up server handler threads even if response is not ready
[ https://issues.apache.org/jira/browse/HBASE-3899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang resolved HBASE-3899. -- Resolution: Fixed Fix Version/s: (was: 0.90.7) enhance HBase RPC to support free-ing up server handler threads even if response is not ready - Key: HBASE-3899 URL: https://issues.apache.org/jira/browse/HBASE-3899 Project: HBase Issue Type: Improvement Components: ipc Affects Versions: 0.90.6 Reporter: dhruba borthakur Assignee: dhruba borthakur Fix For: 0.92.0 Attachments: HBASE-3899-2.patch, HBASE-3899-amend-v4.patch, HBASE-3899.patch, asyncRpc.txt, asyncRpc.txt In the current implementation, the server handler thread picks up an item from the incoming callqueue, processes it and then wraps the response as a Writable and sends it back to the IPC server module. This wastes thread-resources when the thread is blocked for disk IO (transaction logging, read into block cache, etc). It would be nice if we can make the RPC Server Handler threads pick up a call from the IPC queue, hand it over to the application (e.g. HRegion), the application can queue it to be processed asynchronously and send a response back to the IPC server module saying that the response is not ready. The RPC Server Handler thread is now ready to pick up another request from the incoming callqueue. When the queued call is processed by the application, it indicates to the IPC module that the response is now ready to be sent back to the client. The RPC client continues to experience the same behaviour as before. A RPC client is synchronous and blocks till the response arrives. This RPC enhancement allows us to do very powerful things with the RegionServer. In future, we can make enhance the RegionServer's threading model to a message-passing model for better performance. We will not be limited by the number of threads in the RegionServer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3899) enhance HBase RPC to support free-ing up server handler threads even if response is not ready
[ https://issues.apache.org/jira/browse/HBASE-3899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-3899: - Attachment: (was: HBASE-3899.patch) enhance HBase RPC to support free-ing up server handler threads even if response is not ready - Key: HBASE-3899 URL: https://issues.apache.org/jira/browse/HBASE-3899 Project: HBase Issue Type: Improvement Components: ipc Affects Versions: 0.90.6 Reporter: dhruba borthakur Assignee: dhruba borthakur Fix For: 0.92.0 Attachments: HBASE-3899-2.patch, HBASE-3899-amend-v4.patch, HBASE-3899.patch, asyncRpc.txt, asyncRpc.txt In the current implementation, the server handler thread picks up an item from the incoming callqueue, processes it and then wraps the response as a Writable and sends it back to the IPC server module. This wastes thread-resources when the thread is blocked for disk IO (transaction logging, read into block cache, etc). It would be nice if we can make the RPC Server Handler threads pick up a call from the IPC queue, hand it over to the application (e.g. HRegion), the application can queue it to be processed asynchronously and send a response back to the IPC server module saying that the response is not ready. The RPC Server Handler thread is now ready to pick up another request from the incoming callqueue. When the queued call is processed by the application, it indicates to the IPC module that the response is now ready to be sent back to the client. The RPC client continues to experience the same behaviour as before. A RPC client is synchronous and blocks till the response arrives. This RPC enhancement allows us to do very powerful things with the RegionServer. In future, we can make enhance the RegionServer's threading model to a message-passing model for better performance. We will not be limited by the number of threads in the RegionServer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-5988) HBase would be blocked writing for a few Seconds
[ https://issues.apache.org/jira/browse/HBASE-5988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang resolved HBASE-5988. -- Resolution: Invalid Please send your questions to the user mailing list: http://hbase.apache.org/mail-lists.html This JIRA is only used for HBase development. HBase would be blocked writing for a few Seconds - Key: HBASE-5988 URL: https://issues.apache.org/jira/browse/HBASE-5988 Project: HBase Issue Type: Bug Reporter: zhangyong we use thrift to access the hbase. but The problems we for a long time. the hbase would be blocked writing for a few seconds. we check the log but find nothing. eachtime, it will effect 48 or 96 request. hbase have 13 regionserver, 3 zookeeper and 3 masterserver hadoop have 16 datanodeserver. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5973) Add ability for potentially long-running IPC calls to abort if client disconnects
[ https://issues.apache.org/jira/browse/HBASE-5973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-5973: - Attachment: HBASE-5973-0.90.txt Passed unit tests locally. Add ability for potentially long-running IPC calls to abort if client disconnects - Key: HBASE-5973 URL: https://issues.apache.org/jira/browse/HBASE-5973 Project: HBase Issue Type: Improvement Components: ipc Affects Versions: 0.90.7, 0.92.1, 0.94.0, 0.96.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Fix For: 0.92.2, 0.96.0, 0.94.1 Attachments: HBASE-5973-0.90.txt, hbase-5973-0.92.txt, hbase-5973-0.94.txt, hbase-5973-0.94.txt, hbase-5973.txt, hbase-5973.txt, hbase-5973.txt We recently had a cluster issue where a user was submitting scanners with a very restrictive filter, and then calling next() with a high scanner caching value. The clients would generally time out the next() call and disconnect, but the IPC kept running looking to fill the requested number of rows. Since this was in the context of MR, the tasks making the calls would retry, and the retries wuld be more likely to time out due to contention with the previous still-running scanner next() call. Eventually, the system spiraled out of control. We should add a hook to the IPC system so that RPC calls can check if the client has already disconnected. In such a case, the next() call could abort processing, given any further work is wasted. I imagine coprocessor endpoints, etc, could make good use of this as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5973) Add ability for potentially long-running IPC calls to abort if client disconnects
[ https://issues.apache.org/jira/browse/HBASE-5973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13273490#comment-13273490 ] David S. Wang commented on HBASE-5973: -- Sure. This is what I saw on my local box: Tests in error: Tests run: 723, Failures: 0, Errors: 1, Skipped: 9 [INFO] Total time: 1:31:50.066s Only test that failed was: Running org.apache.hadoop.hbase.master.TestRestartCluster Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 45.891 sec FAILURE! Test seems flaky as it passed just fine when run individually. Add ability for potentially long-running IPC calls to abort if client disconnects - Key: HBASE-5973 URL: https://issues.apache.org/jira/browse/HBASE-5973 Project: HBase Issue Type: Improvement Components: ipc Affects Versions: 0.90.7, 0.92.1, 0.94.0, 0.96.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Fix For: 0.92.2, 0.96.0, 0.94.1 Attachments: HBASE-5973-0.90.txt, hbase-5973-0.92.txt, hbase-5973-0.94.txt, hbase-5973-0.94.txt, hbase-5973.txt, hbase-5973.txt, hbase-5973.txt We recently had a cluster issue where a user was submitting scanners with a very restrictive filter, and then calling next() with a high scanner caching value. The clients would generally time out the next() call and disconnect, but the IPC kept running looking to fill the requested number of rows. Since this was in the context of MR, the tasks making the calls would retry, and the retries wuld be more likely to time out due to contention with the previous still-running scanner next() call. Eventually, the system spiraled out of control. We should add a hook to the IPC system so that RPC calls can check if the client has already disconnected. In such a case, the next() call could abort processing, given any further work is wasted. I imagine coprocessor endpoints, etc, could make good use of this as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5904) is_enabled from shell returns differently from pre- and post- HBASE-5155
[ https://issues.apache.org/jira/browse/HBASE-5904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13271950#comment-13271950 ] David S. Wang commented on HBASE-5904: -- The patch did not apply because it's against 0.90, which it seems always fails with the QA robot. Anyone care to take a stab at the review? I thought it was a fairly straightforward revert. is_enabled from shell returns differently from pre- and post- HBASE-5155 Key: HBASE-5904 URL: https://issues.apache.org/jira/browse/HBASE-5904 Project: HBase Issue Type: Bug Components: zookeeper Affects Versions: 0.90.6 Reporter: David S. Wang Assignee: David S. Wang Attachments: HBASE-5904.patch If I launch an hbase shell that uses HBase and ZooKeeper without HBASE-5155, against HBase servers with HBASE-5155, then is_enabled for a table always returns false even if the table is considered enabled by the servers from the logs. If I then do the same thing but with an HBase shell and ZooKeeper with HBASE-5155, then is_enabled returns as expected. If I launch an HBase shell that uses HBase and ZooKeeper without HBASE-5155, against HBase servers also without HBASE-5155, then is_enabled works as you'd expect. But if I then do the same thing but with an HBase shell and ZooKeeper with HBASE-5155, then is_enabled returns false even though the table is considered enabled by the servers from the logs. Additionally, if I then try to enable the table from the HBASE-5155-containing shell, it hangs because the ZooKeeper code waits for the ZNode to be updated with ENABLED in the data field, but what actually happens is that the ZNode gets deleted since the servers are running without HBASE-5155. I think the culprit is that the indication of how a table is considered enabled inside ZooKeeper has changed with HBASE-5155. Before HBASE-5155, a table was considered enabled if the ZNode for it did not exist. After HBASE-5155, a table is considered enabled if the ZNode for it exists and has ENABLED in its data. I think the current code is incompatible when running clients and servers where one side has HBASE-5155 and the other side does not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5904) is_enabled from shell returns differently from pre- and post- HBASE-5155
[ https://issues.apache.org/jira/browse/HBASE-5904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13272084#comment-13272084 ] David S. Wang commented on HBASE-5904: -- I thought that was what we agreed upon earlier. At least that is the impression that I got from reading the earlier comments. Do you think there is a way to keep the part of the patch with the actual fix, without having the incompatible ZK behavior? is_enabled from shell returns differently from pre- and post- HBASE-5155 Key: HBASE-5904 URL: https://issues.apache.org/jira/browse/HBASE-5904 Project: HBase Issue Type: Bug Components: zookeeper Affects Versions: 0.90.6 Reporter: David S. Wang Assignee: David S. Wang Attachments: HBASE-5904.patch If I launch an hbase shell that uses HBase and ZooKeeper without HBASE-5155, against HBase servers with HBASE-5155, then is_enabled for a table always returns false even if the table is considered enabled by the servers from the logs. If I then do the same thing but with an HBase shell and ZooKeeper with HBASE-5155, then is_enabled returns as expected. If I launch an HBase shell that uses HBase and ZooKeeper without HBASE-5155, against HBase servers also without HBASE-5155, then is_enabled works as you'd expect. But if I then do the same thing but with an HBase shell and ZooKeeper with HBASE-5155, then is_enabled returns false even though the table is considered enabled by the servers from the logs. Additionally, if I then try to enable the table from the HBASE-5155-containing shell, it hangs because the ZooKeeper code waits for the ZNode to be updated with ENABLED in the data field, but what actually happens is that the ZNode gets deleted since the servers are running without HBASE-5155. I think the culprit is that the indication of how a table is considered enabled inside ZooKeeper has changed with HBASE-5155. Before HBASE-5155, a table was considered enabled if the ZNode for it did not exist. After HBASE-5155, a table is considered enabled if the ZNode for it exists and has ENABLED in its data. I think the current code is incompatible when running clients and servers where one side has HBASE-5155 and the other side does not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5904) is_enabled from shell returns differently from pre- and post- HBASE-5155
[ https://issues.apache.org/jira/browse/HBASE-5904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-5904: - Attachment: HBASE-5904.patch is_enabled from shell returns differently from pre- and post- HBASE-5155 Key: HBASE-5904 URL: https://issues.apache.org/jira/browse/HBASE-5904 Project: HBase Issue Type: Bug Components: zookeeper Affects Versions: 0.90.6 Reporter: David S. Wang If I launch an hbase shell that uses HBase and ZooKeeper without HBASE-5155, against HBase servers with HBASE-5155, then is_enabled for a table always returns false even if the table is considered enabled by the servers from the logs. If I then do the same thing but with an HBase shell and ZooKeeper with HBASE-5155, then is_enabled returns as expected. If I launch an HBase shell that uses HBase and ZooKeeper without HBASE-5155, against HBase servers also without HBASE-5155, then is_enabled works as you'd expect. But if I then do the same thing but with an HBase shell and ZooKeeper with HBASE-5155, then is_enabled returns false even though the table is considered enabled by the servers from the logs. Additionally, if I then try to enable the table from the HBASE-5155-containing shell, it hangs because the ZooKeeper code waits for the ZNode to be updated with ENABLED in the data field, but what actually happens is that the ZNode gets deleted since the servers are running without HBASE-5155. I think the culprit is that the indication of how a table is considered enabled inside ZooKeeper has changed with HBASE-5155. Before HBASE-5155, a table was considered enabled if the ZNode for it did not exist. After HBASE-5155, a table is considered enabled if the ZNode for it exists and has ENABLED in its data. I think the current code is incompatible when running clients and servers where one side has HBASE-5155 and the other side does not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5904) is_enabled from shell returns differently from pre- and post- HBASE-5155
[ https://issues.apache.org/jira/browse/HBASE-5904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-5904: - Attachment: (was: HBASE-5904.patch) is_enabled from shell returns differently from pre- and post- HBASE-5155 Key: HBASE-5904 URL: https://issues.apache.org/jira/browse/HBASE-5904 Project: HBase Issue Type: Bug Components: zookeeper Affects Versions: 0.90.6 Reporter: David S. Wang If I launch an hbase shell that uses HBase and ZooKeeper without HBASE-5155, against HBase servers with HBASE-5155, then is_enabled for a table always returns false even if the table is considered enabled by the servers from the logs. If I then do the same thing but with an HBase shell and ZooKeeper with HBASE-5155, then is_enabled returns as expected. If I launch an HBase shell that uses HBase and ZooKeeper without HBASE-5155, against HBase servers also without HBASE-5155, then is_enabled works as you'd expect. But if I then do the same thing but with an HBase shell and ZooKeeper with HBASE-5155, then is_enabled returns false even though the table is considered enabled by the servers from the logs. Additionally, if I then try to enable the table from the HBASE-5155-containing shell, it hangs because the ZooKeeper code waits for the ZNode to be updated with ENABLED in the data field, but what actually happens is that the ZNode gets deleted since the servers are running without HBASE-5155. I think the culprit is that the indication of how a table is considered enabled inside ZooKeeper has changed with HBASE-5155. Before HBASE-5155, a table was considered enabled if the ZNode for it did not exist. After HBASE-5155, a table is considered enabled if the ZNode for it exists and has ENABLED in its data. I think the current code is incompatible when running clients and servers where one side has HBASE-5155 and the other side does not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5904) is_enabled from shell returns differently from pre- and post- HBASE-5155
[ https://issues.apache.org/jira/browse/HBASE-5904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-5904: - Assignee: David S. Wang Status: Patch Available (was: Open) Revert HBASE-5155. Should apply only to 0.90 branch. is_enabled from shell returns differently from pre- and post- HBASE-5155 Key: HBASE-5904 URL: https://issues.apache.org/jira/browse/HBASE-5904 Project: HBase Issue Type: Bug Components: zookeeper Affects Versions: 0.90.6 Reporter: David S. Wang Assignee: David S. Wang Attachments: HBASE-5904.patch If I launch an hbase shell that uses HBase and ZooKeeper without HBASE-5155, against HBase servers with HBASE-5155, then is_enabled for a table always returns false even if the table is considered enabled by the servers from the logs. If I then do the same thing but with an HBase shell and ZooKeeper with HBASE-5155, then is_enabled returns as expected. If I launch an HBase shell that uses HBase and ZooKeeper without HBASE-5155, against HBase servers also without HBASE-5155, then is_enabled works as you'd expect. But if I then do the same thing but with an HBase shell and ZooKeeper with HBASE-5155, then is_enabled returns false even though the table is considered enabled by the servers from the logs. Additionally, if I then try to enable the table from the HBASE-5155-containing shell, it hangs because the ZooKeeper code waits for the ZNode to be updated with ENABLED in the data field, but what actually happens is that the ZNode gets deleted since the servers are running without HBASE-5155. I think the culprit is that the indication of how a table is considered enabled inside ZooKeeper has changed with HBASE-5155. Before HBASE-5155, a table was considered enabled if the ZNode for it did not exist. After HBASE-5155, a table is considered enabled if the ZNode for it exists and has ENABLED in its data. I think the current code is incompatible when running clients and servers where one side has HBASE-5155 and the other side does not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5904) is_enabled from shell returns differently from pre- and post- HBASE-5155
[ https://issues.apache.org/jira/browse/HBASE-5904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-5904: - Attachment: HBASE-5904.patch is_enabled from shell returns differently from pre- and post- HBASE-5155 Key: HBASE-5904 URL: https://issues.apache.org/jira/browse/HBASE-5904 Project: HBase Issue Type: Bug Components: zookeeper Affects Versions: 0.90.6 Reporter: David S. Wang Attachments: HBASE-5904.patch If I launch an hbase shell that uses HBase and ZooKeeper without HBASE-5155, against HBase servers with HBASE-5155, then is_enabled for a table always returns false even if the table is considered enabled by the servers from the logs. If I then do the same thing but with an HBase shell and ZooKeeper with HBASE-5155, then is_enabled returns as expected. If I launch an HBase shell that uses HBase and ZooKeeper without HBASE-5155, against HBase servers also without HBASE-5155, then is_enabled works as you'd expect. But if I then do the same thing but with an HBase shell and ZooKeeper with HBASE-5155, then is_enabled returns false even though the table is considered enabled by the servers from the logs. Additionally, if I then try to enable the table from the HBASE-5155-containing shell, it hangs because the ZooKeeper code waits for the ZNode to be updated with ENABLED in the data field, but what actually happens is that the ZNode gets deleted since the servers are running without HBASE-5155. I think the culprit is that the indication of how a table is considered enabled inside ZooKeeper has changed with HBASE-5155. Before HBASE-5155, a table was considered enabled if the ZNode for it did not exist. After HBASE-5155, a table is considered enabled if the ZNode for it exists and has ENABLED in its data. I think the current code is incompatible when running clients and servers where one side has HBASE-5155 and the other side does not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5904) is_enabled from shell returns differently from pre- and post- HBASE-5155
[ https://issues.apache.org/jira/browse/HBASE-5904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13269811#comment-13269811 ] David S. Wang commented on HBASE-5904: -- I added a patch off of 0.90 to revert HBASE-5155. I'll throw this on RB as well, but wanted to kick off Hadoop QA bot now. Note that all unit tests for me except for org.apache.hadoop.hbase.TestZooKeeper.testClientSessionExpired, which fails fairly consistently with or without this change. is_enabled from shell returns differently from pre- and post- HBASE-5155 Key: HBASE-5904 URL: https://issues.apache.org/jira/browse/HBASE-5904 Project: HBase Issue Type: Bug Components: zookeeper Affects Versions: 0.90.6 Reporter: David S. Wang Assignee: David S. Wang Attachments: HBASE-5904.patch If I launch an hbase shell that uses HBase and ZooKeeper without HBASE-5155, against HBase servers with HBASE-5155, then is_enabled for a table always returns false even if the table is considered enabled by the servers from the logs. If I then do the same thing but with an HBase shell and ZooKeeper with HBASE-5155, then is_enabled returns as expected. If I launch an HBase shell that uses HBase and ZooKeeper without HBASE-5155, against HBase servers also without HBASE-5155, then is_enabled works as you'd expect. But if I then do the same thing but with an HBase shell and ZooKeeper with HBASE-5155, then is_enabled returns false even though the table is considered enabled by the servers from the logs. Additionally, if I then try to enable the table from the HBASE-5155-containing shell, it hangs because the ZooKeeper code waits for the ZNode to be updated with ENABLED in the data field, but what actually happens is that the ZNode gets deleted since the servers are running without HBASE-5155. I think the culprit is that the indication of how a table is considered enabled inside ZooKeeper has changed with HBASE-5155. Before HBASE-5155, a table was considered enabled if the ZNode for it did not exist. After HBASE-5155, a table is considered enabled if the ZNode for it exists and has ENABLED in its data. I think the current code is incompatible when running clients and servers where one side has HBASE-5155 and the other side does not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5953) Expose the current state of the balancerSwitch
David S. Wang created HBASE-5953: Summary: Expose the current state of the balancerSwitch Key: HBASE-5953 URL: https://issues.apache.org/jira/browse/HBASE-5953 Project: HBase Issue Type: Improvement Components: master Affects Versions: 0.92.1, 0.94.0, 0.96.0 Reporter: David S. Wang Assignee: David S. Wang Priority: Minor The balancerSwitch as in both 0.90 and 0.92.1 is implemented in such a way that it is impossible to retrieve its value without changing it: /** Turn the load balancer on or off. @param b If true, enable balancer. If false, disable balancer. @return Previous balancer value */ public boolean balanceSwitch(final boolean b); It would be nice to have a way to just get the current state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5676) Note that dfs.support.append does not have to be enabled post 1.x
[ https://issues.apache.org/jira/browse/HBASE-5676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang reassigned HBASE-5676: Assignee: David S. Wang Note that dfs.support.append does not have to be enabled post 1.x - Key: HBASE-5676 URL: https://issues.apache.org/jira/browse/HBASE-5676 Project: HBase Issue Type: Task Reporter: Eli Collins Assignee: David S. Wang In Hadoop 1.x (HADOOP-8230) we are going to enable durable sync by default, and remove the dfs.support.append option. What this means for you: - HBase will work out of the box on Hadoop 1.x, no need to tell people to re-configure - You no longer have to enable append (which can result in data loss) to enable HBase support -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5676) Note that dfs.support.append does not have to be enabled post 1.x
[ https://issues.apache.org/jira/browse/HBASE-5676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-5676: - Attachment: HBASE-5676.patch Note that dfs.support.append does not have to be enabled post 1.x - Key: HBASE-5676 URL: https://issues.apache.org/jira/browse/HBASE-5676 Project: HBase Issue Type: Task Affects Versions: 0.96.0 Reporter: Eli Collins Assignee: David S. Wang Attachments: HBASE-5676.patch In Hadoop 1.x (HADOOP-8230) we are going to enable durable sync by default, and remove the dfs.support.append option. What this means for you: - HBase will work out of the box on Hadoop 1.x, no need to tell people to re-configure - You no longer have to enable append (which can result in data loss) to enable HBase support -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5676) Note that dfs.support.append does not have to be enabled post 1.x
[ https://issues.apache.org/jira/browse/HBASE-5676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-5676: - Affects Version/s: 0.96.0 Status: Patch Available (was: Open) Note that dfs.support.append does not have to be enabled post 1.x - Key: HBASE-5676 URL: https://issues.apache.org/jira/browse/HBASE-5676 Project: HBase Issue Type: Task Affects Versions: 0.96.0 Reporter: Eli Collins Assignee: David S. Wang Attachments: HBASE-5676.patch In Hadoop 1.x (HADOOP-8230) we are going to enable durable sync by default, and remove the dfs.support.append option. What this means for you: - HBase will work out of the box on Hadoop 1.x, no need to tell people to re-configure - You no longer have to enable append (which can result in data loss) to enable HBase support -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5676) Note that dfs.support.append does not have to be enabled post 1.x
[ https://issues.apache.org/jira/browse/HBASE-5676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HBASE-5676: - Review is at: https://reviews.apache.org/r/4999/ Note that dfs.support.append does not have to be enabled post 1.x - Key: HBASE-5676 URL: https://issues.apache.org/jira/browse/HBASE-5676 Project: HBase Issue Type: Task Affects Versions: 0.96.0 Reporter: Eli Collins Assignee: David S. Wang Attachments: HBASE-5676.patch In Hadoop 1.x (HADOOP-8230) we are going to enable durable sync by default, and remove the dfs.support.append option. What this means for you: - HBase will work out of the box on Hadoop 1.x, no need to tell people to re-configure - You no longer have to enable append (which can result in data loss) to enable HBase support -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
