[jira] [Created] (HBASE-11067) HBase cannot build againt Hadoop branch-2
Fengdong Yu created HBASE-11067: --- Summary: HBase cannot build againt Hadoop branch-2 Key: HBASE-11067 URL: https://issues.apache.org/jira/browse/HBASE-11067 Project: HBase Issue Type: Bug Components: hadoop2 Affects Versions: 0.98.1 Reporter: Fengdong Yu Assignee: Fengdong Yu Due to HADOOP-10499, HBase cannot build against branch-2, I will upload patch shortly. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11067) HBase cannot build againt Hadoop branch-2
[ https://issues.apache.org/jira/browse/HBASE-11067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fengdong Yu updated HBASE-11067: Attachment: HBASE-11067.patch HBase cannot build againt Hadoop branch-2 - Key: HBASE-11067 URL: https://issues.apache.org/jira/browse/HBASE-11067 Project: HBase Issue Type: Bug Components: hadoop2 Affects Versions: 0.98.1 Reporter: Fengdong Yu Assignee: Fengdong Yu Attachments: HBASE-11067.patch Due to HADOOP-10499, HBase cannot build against branch-2, I will upload patch shortly. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11067) HBase cannot build againt Hadoop branch-2
[ https://issues.apache.org/jira/browse/HBASE-11067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fengdong Yu updated HBASE-11067: Status: Patch Available (was: Open) HBase cannot build againt Hadoop branch-2 - Key: HBASE-11067 URL: https://issues.apache.org/jira/browse/HBASE-11067 Project: HBase Issue Type: Bug Components: hadoop2 Affects Versions: 0.98.1 Reporter: Fengdong Yu Assignee: Fengdong Yu Attachments: HBASE-11067.patch Due to HADOOP-10499, HBase cannot build against branch-2, I will upload patch shortly. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11067) HBase cannot build againt Hadoop branch-2
[ https://issues.apache.org/jira/browse/HBASE-11067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980605#comment-13980605 ] Fengdong Yu commented on HBASE-11067: - Ok, HADOOP-10539 add backward compatible and committed to the trunk, so branch-2 and 'branch-3' in future will be supported. HBase cannot build againt Hadoop branch-2 - Key: HBASE-11067 URL: https://issues.apache.org/jira/browse/HBASE-11067 Project: HBase Issue Type: Bug Components: hadoop2 Affects Versions: 0.98.1 Reporter: Fengdong Yu Attachments: HBASE-11067.patch Due to HADOOP-10499, HBase cannot build against branch-2, I will upload patch shortly. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10621) Unable to grant user permission to namespace
[
https://issues.apache.org/jira/browse/HBASE-10621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13913836#comment-13913836
]
Fengdong Yu commented on HBASE-10621:
-
Looks good.
Unable to grant user permission to namespace
Key: HBASE-10621
URL: https://issues.apache.org/jira/browse/HBASE-10621
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.0
Reporter: Ted Yu
Assignee: Ted Yu
Fix For: 0.98.1, 0.99.0
Attachments: 10621-v1.txt, 10621-v2.txt
In a secure cluster, I tried to grant user 'hrt_1' permission to namespace X
{code}
hbase(main):002:0 grant 'hrt_1', 'W', '@X'
ERROR: no method 'grant' for arguments
(org.apache.hadoop.hbase.protobuf.generated.AccessControlProtos.AccessControlService.BlockingStub,org.jruby.RubyString,org.jruby.java.proxies.ArrayJavaProxy,org.jruby.java.proxies.ArrayJavaProxy)
on Java::OrgApacheHadoopHbaseProtobuf::ProtobufUtil
available overloads:
(org.apache.hadoop.hbase.protobuf.generated.AccessControlProtos.AccessControlService.BlockingInterface,java.lang.String,java.lang.String,org.apache.hadoop.hbase.security.access.Permission.Action[])
(org.apache.hadoop.hbase.protobuf.generated.AccessControlProtos.AccessControlService.BlockingInterface,java.lang.String,org.apache.hadoop.hbase.TableName,byte[],byte[],org.apache.hadoop.hbase.security.access.Permission.Action[])
(org.apache.hadoop.hbase.protobuf.generated.AccessControlProtos.AccessControlService.BlockingInterface,java.lang.String,org.apache.hadoop.hbase.security.access.Permission.Action[])
Here is some help for this command:
Grant users specific rights.
Syntax : grant user permissions [table [column family [column
qualifier]]
permissions is either zero or more letters from the set RWXCA.
READ('R'), WRITE('W'), EXEC('X'), CREATE('C'), ADMIN('A')
For example:
hbase grant 'bobsmith', 'RWXCA'
hbase grant 'bobsmith', 'RW', 't1', 'f1', 'col1'
{code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10047) postScannerFilterRow consumes a lot of CPU in tall table scans
[ https://issues.apache.org/jira/browse/HBASE-10047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1389#comment-1389 ] Fengdong Yu commented on HBASE-10047: - [~lhofhansi] Can you share which tool you used for your profiling? I am also has an REMOTE Application consumed a lot of CPU, I want to profile this App remotely. postScannerFilterRow consumes a lot of CPU in tall table scans -- Key: HBASE-10047 URL: https://issues.apache.org/jira/browse/HBASE-10047 Project: HBase Issue Type: Bug Reporter: Lars Hofhansl Attachments: postScannerFilterRow.png Continuing my profiling quest, I find that in scanning tall table (and filtering everything on the server) a quarter of the time is now spent in the postScannerFilterRow coprocessor hook. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-10047) postScannerFilterRow consumes a lot of CPU in tall table scans
[ https://issues.apache.org/jira/browse/HBASE-10047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13833341#comment-13833341 ] Fengdong Yu commented on HBASE-10047: - [~lhofhansl] Can you share which tool you used for your profiling? I am also has an REMOTE Application consumed a lot of CPU, I want to profile this App remotely. postScannerFilterRow consumes a lot of CPU in tall table scans -- Key: HBASE-10047 URL: https://issues.apache.org/jira/browse/HBASE-10047 Project: HBase Issue Type: Bug Reporter: Lars Hofhansl Attachments: postScannerFilterRow.png Continuing my profiling quest, I find that in scanning tall table (and filtering everything on the server) a quarter of the time is now spent in the postScannerFilterRow coprocessor hook. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-8812) Avoid a wide line on the HMaster webUI if we have many ZooKeeper servers
[ https://issues.apache.org/jira/browse/HBASE-8812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13697547#comment-13697547 ] Fengdong Yu commented on HBASE-8812: So can I close this issue? Avoid a wide line on the HMaster webUI if we have many ZooKeeper servers Key: HBASE-8812 URL: https://issues.apache.org/jira/browse/HBASE-8812 Project: HBase Issue Type: Improvement Components: master Affects Versions: 0.94.8 Reporter: Fengdong Yu Assignee: Fengdong Yu Priority: Minor Attachments: HBASE-8812.patch, screeshot.PNG add a line break for every four zookeeper quorums on the HMaster webUI. I don't think this need a test case. just manual testing is enough. I've tested on my testing cluster. everything works well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8812) Avoid a wide line on the HMaster webUI if we have many ZooKeeper servers
[ https://issues.apache.org/jira/browse/HBASE-8812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fengdong Yu updated HBASE-8812: --- Affects Version/s: 0.94.8 Avoid a wide line on the HMaster webUI if we have many ZooKeeper servers Key: HBASE-8812 URL: https://issues.apache.org/jira/browse/HBASE-8812 Project: HBase Issue Type: Improvement Components: master Affects Versions: 0.94.8 Reporter: Fengdong Yu Assignee: Fengdong Yu Priority: Minor Attachments: HBASE-8812.patch, screeshot.PNG add a line break for every four zookeeper quorums on the HMaster webUI. I don't think this need a test case. just manual testing is enough. I've tested on my testing cluster. everything works well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8812) Avoid a wide line on the HMaster webUI if we have many ZooKeeper servers
[ https://issues.apache.org/jira/browse/HBASE-8812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13698606#comment-13698606 ] Fengdong Yu commented on HBASE-8812: bq. I applied to trunk and 0.95. Lars Hofhansl you want this fixup for 0.94? Thanks for the patch Fengdong Yu Thanks Stack, please also update CHANGES.txt, Thanks. Avoid a wide line on the HMaster webUI if we have many ZooKeeper servers Key: HBASE-8812 URL: https://issues.apache.org/jira/browse/HBASE-8812 Project: HBase Issue Type: Improvement Components: master Affects Versions: 0.94.8 Reporter: Fengdong Yu Assignee: Fengdong Yu Priority: Minor Fix For: 0.98.0, 0.95.2 Attachments: HBASE-8812.patch, screeshot.PNG add a line break for every four zookeeper quorums on the HMaster webUI. I don't think this need a test case. just manual testing is enough. I've tested on my testing cluster. everything works well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8812) Avoid a wide line on the HMaster webUI if we have many ZooKeeper servers
[ https://issues.apache.org/jira/browse/HBASE-8812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13696810#comment-13696810 ] Fengdong Yu commented on HBASE-8812: bq.Mind attaching screen shot of the new UI showing more than 4 ZK servers ? Yes. actually I've applied the patch in our prod cluster. Screen shot was attached. Thanks. Avoid a wide line on the HMaster webUI if we have many ZooKeeper servers Key: HBASE-8812 URL: https://issues.apache.org/jira/browse/HBASE-8812 Project: HBase Issue Type: Improvement Components: master Reporter: Fengdong Yu Assignee: Fengdong Yu Priority: Minor Attachments: HBASE-8812.patch add a line break for every four zookeeper quorums on the HMaster webUI. I don't think this need a test case. just manual testing is enough. I've tested on my testing cluster. everything works well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8812) Avoid a wide line on the HMaster webUI if we have many ZooKeeper servers
[ https://issues.apache.org/jira/browse/HBASE-8812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fengdong Yu updated HBASE-8812: --- Attachment: screeshot.PNG Avoid a wide line on the HMaster webUI if we have many ZooKeeper servers Key: HBASE-8812 URL: https://issues.apache.org/jira/browse/HBASE-8812 Project: HBase Issue Type: Improvement Components: master Reporter: Fengdong Yu Assignee: Fengdong Yu Priority: Minor Attachments: HBASE-8812.patch, screeshot.PNG add a line break for every four zookeeper quorums on the HMaster webUI. I don't think this need a test case. just manual testing is enough. I've tested on my testing cluster. everything works well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8812) Avoid a wide line on the HMaster webUI if we have more zookeeper quorums
Fengdong Yu created HBASE-8812: -- Summary: Avoid a wide line on the HMaster webUI if we have more zookeeper quorums Key: HBASE-8812 URL: https://issues.apache.org/jira/browse/HBASE-8812 Project: HBase Issue Type: Improvement Components: master Reporter: Fengdong Yu Priority: Minor -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8812) Avoid a wide line on the HMaster webUI if we have more zookeeper quorums
[ https://issues.apache.org/jira/browse/HBASE-8812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fengdong Yu updated HBASE-8812: --- Description: add a line break for every four zookeeper quorums on the HMaster webUI. Avoid a wide line on the HMaster webUI if we have more zookeeper quorums Key: HBASE-8812 URL: https://issues.apache.org/jira/browse/HBASE-8812 Project: HBase Issue Type: Improvement Components: master Reporter: Fengdong Yu Priority: Minor add a line break for every four zookeeper quorums on the HMaster webUI. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8812) Avoid a wide line on the HMaster webUI if we have more zookeeper quorums
[ https://issues.apache.org/jira/browse/HBASE-8812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fengdong Yu updated HBASE-8812: --- Status: Patch Available (was: Open) Avoid a wide line on the HMaster webUI if we have more zookeeper quorums Key: HBASE-8812 URL: https://issues.apache.org/jira/browse/HBASE-8812 Project: HBase Issue Type: Improvement Components: master Reporter: Fengdong Yu Priority: Minor Attachments: HBASE-8812.patch add a line break for every four zookeeper quorums on the HMaster webUI. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8812) Avoid a wide line on the HMaster webUI if we have more zookeeper quorums
[ https://issues.apache.org/jira/browse/HBASE-8812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fengdong Yu updated HBASE-8812: --- Attachment: HBASE-8812.patch Avoid a wide line on the HMaster webUI if we have more zookeeper quorums Key: HBASE-8812 URL: https://issues.apache.org/jira/browse/HBASE-8812 Project: HBase Issue Type: Improvement Components: master Reporter: Fengdong Yu Priority: Minor Attachments: HBASE-8812.patch add a line break for every four zookeeper quorums on the HMaster webUI. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8812) Avoid a wide line on the HMaster webUI if we have more zookeeper quorums
[ https://issues.apache.org/jira/browse/HBASE-8812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fengdong Yu updated HBASE-8812: --- Description: add a line break for every four zookeeper quorums on the HMaster webUI. I don't think this need a test case. just manual testing is enough. I've tested on my testing cluster. everything works well. was:add a line break for every four zookeeper quorums on the HMaster webUI. Avoid a wide line on the HMaster webUI if we have more zookeeper quorums Key: HBASE-8812 URL: https://issues.apache.org/jira/browse/HBASE-8812 Project: HBase Issue Type: Improvement Components: master Reporter: Fengdong Yu Priority: Minor Attachments: HBASE-8812.patch add a line break for every four zookeeper quorums on the HMaster webUI. I don't think this need a test case. just manual testing is enough. I've tested on my testing cluster. everything works well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8259) Snapshot backport in 0.94.6 breaks rolling restarts
[
https://issues.apache.org/jira/browse/HBASE-8259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13623274#comment-13623274
]
Fengdong Yu commented on HBASE-8259:
All mirror sites are not updated.
Does there has command to trigger mirror sites update?
Snapshot backport in 0.94.6 breaks rolling restarts
---
Key: HBASE-8259
URL: https://issues.apache.org/jira/browse/HBASE-8259
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.6
Reporter: Jean-Daniel Cryans
Assignee: Matteo Bertozzi
Priority: Blocker
Fix For: 0.94.7, 0.94.6.1
Attachments: HBASE-8259-v0.patch
[~aleksshulman] found with his nifty QA tools that 0.94.6 has an incompatible
change due to HBASE-7360 (Snapshot 0.94 Backport) that breaks rolling
restarts.
RegionTransitionData.write() uses eventType.ordinal() that is the index in
the enum and not the value specified in the enum definition. It means we
can't add new states in the middle of the list. This can be fixed by moving
C_M_SNAPSHOT_TABLE and C_M_RESTORE_SNAPSHOT at the end of the list. Trunk
does the right thing already.
Right now, RIT znodes created with 0.94.6 (or top of the branch) will use the
wrong value for the event type. You will see things like:
{noformat}
2013-04-03 14:57:25,197 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
regionserver:60020-0x13dd1e10dbd0004 Attempting to transition node
70236052/-ROOT- from M_ZK_REGION_OFFLINE to RS_ZK_REGION_OPENING
2013-04-03 14:57:25,201 WARN org.apache.hadoop.hbase.zookeeper.ZKAssign:
regionserver:60020-0x13dd1e10dbd0004 Attempt to transition the unassigned
node for 70236052 from M_ZK_REGION_OFFLINE to RS_ZK_REGION_OPENING failed,
the node existed but was in the state M_SERVER_SHUTDOWN set by the server
192.168.1.112,60020,1365026237977
2013-04-03 14:57:25,201 WARN
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Failed
transition from OFFLINE to OPENING for region=70236052
2013-04-03 14:57:25,201 WARN
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Region was
hijacked? It no longer exists, encodedName=70236052
{noformat}
We should roll a 0.94.6.1 or 0.94.7 as soon this is fixed IMO.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8259) Snapshot backport in 0.94.6 breaks rolling restarts
[
https://issues.apache.org/jira/browse/HBASE-8259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13623322#comment-13623322
]
Fengdong Yu commented on HBASE-8259:
Lars,
you can download it from backup sites, they are all updated.
http://www.eu.apache.org/dist/hbase/
http://www.us.apache.org/dist/hbase/
Snapshot backport in 0.94.6 breaks rolling restarts
---
Key: HBASE-8259
URL: https://issues.apache.org/jira/browse/HBASE-8259
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.6
Reporter: Jean-Daniel Cryans
Assignee: Matteo Bertozzi
Priority: Blocker
Fix For: 0.94.7, 0.94.6.1
Attachments: HBASE-8259-v0.patch
[~aleksshulman] found with his nifty QA tools that 0.94.6 has an incompatible
change due to HBASE-7360 (Snapshot 0.94 Backport) that breaks rolling
restarts.
RegionTransitionData.write() uses eventType.ordinal() that is the index in
the enum and not the value specified in the enum definition. It means we
can't add new states in the middle of the list. This can be fixed by moving
C_M_SNAPSHOT_TABLE and C_M_RESTORE_SNAPSHOT at the end of the list. Trunk
does the right thing already.
Right now, RIT znodes created with 0.94.6 (or top of the branch) will use the
wrong value for the event type. You will see things like:
{noformat}
2013-04-03 14:57:25,197 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
regionserver:60020-0x13dd1e10dbd0004 Attempting to transition node
70236052/-ROOT- from M_ZK_REGION_OFFLINE to RS_ZK_REGION_OPENING
2013-04-03 14:57:25,201 WARN org.apache.hadoop.hbase.zookeeper.ZKAssign:
regionserver:60020-0x13dd1e10dbd0004 Attempt to transition the unassigned
node for 70236052 from M_ZK_REGION_OFFLINE to RS_ZK_REGION_OPENING failed,
the node existed but was in the state M_SERVER_SHUTDOWN set by the server
192.168.1.112,60020,1365026237977
2013-04-03 14:57:25,201 WARN
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Failed
transition from OFFLINE to OPENING for region=70236052
2013-04-03 14:57:25,201 WARN
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Region was
hijacked? It no longer exists, encodedName=70236052
{noformat}
We should roll a 0.94.6.1 or 0.94.7 as soon this is fixed IMO.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8211) Support for NN HA for 0.94
[ https://issues.apache.org/jira/browse/HBASE-8211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13621756#comment-13621756 ] Fengdong Yu commented on HBASE-8211: It will go for commit. Support for NN HA for 0.94 -- Key: HBASE-8211 URL: https://issues.apache.org/jira/browse/HBASE-8211 Project: HBase Issue Type: Bug Components: master, regionserver Affects Versions: 0.94.6 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Fix For: 0.94.7 Attachments: HBase-8211-v1.patch, HBase-8211-v2.patch HBase-8156 is for adding support for retrying non-idempotent operations. This is useful in case NN is suffering from n/w hiccups, etc. This jira is to add similar support for 0.94.x branch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8211) Support for NN HA for 0.94
[ https://issues.apache.org/jira/browse/HBASE-8211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13620642#comment-13620642 ] Fengdong Yu commented on HBASE-8211: bq.increased the log level in sleepWithRetries method in the HBaseSystem class. Can I fix some typo here: increased the log level in sleepBeforeRetry method in the HBaseFileSystem class. but there is a minor difference from HBase-8211-v1.patch: Path rootDir = new Path(TestHBaseFileSystem.conf.get(HConstants.HBASE_DIR)); this looks good. Support for NN HA for 0.94 -- Key: HBASE-8211 URL: https://issues.apache.org/jira/browse/HBASE-8211 Project: HBase Issue Type: Bug Components: master, regionserver Affects Versions: 0.94.6 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Fix For: 0.94.7 Attachments: HBase-8211-v1.patch, HBase-8211-v2.patch HBase-8156 is for adding support for retrying non-idempotent operations. This is useful in case NN is suffering from n/w hiccups, etc. This jira is to add similar support for 0.94.x branch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8211) Support for NN HA for 0.94
[ https://issues.apache.org/jira/browse/HBASE-8211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618564#comment-13618564 ] Fengdong Yu commented on HBASE-8211: Himanshu, Thanks for comments. blockquoteThe idea of HA support is that a client doesn't need to change any url whatsoever, otherwise its not really a failover. /blockquote yes. I read the code again. that's it. blockquoteIf you mean a change in their value because of failover, then it is not required./blockquote for HDFS HA, fs.defaultFS should be configured such as valuehdfs://mycluster/value, so for hbase.rootdir, can we configured such as: valuehdfs://mycluster/hbase/value currently, we just using a specified name node address for hbase.rootdit. Support for NN HA for 0.94 -- Key: HBASE-8211 URL: https://issues.apache.org/jira/browse/HBASE-8211 Project: HBase Issue Type: Bug Components: master, regionserver Affects Versions: 0.94.6 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Fix For: 0.94.7 Attachments: HBase-8211-v1.patch HBase-8156 is for adding support for retrying non-idempotent operations. This is useful in case NN is suffering from n/w hiccups, etc. This jira is to add similar support for 0.94.x branch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8211) Support for NN HA for 0.94
[ https://issues.apache.org/jira/browse/HBASE-8211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618589#comment-13618589 ] Fengdong Yu commented on HBASE-8211: Thanks，I will try. Support for NN HA for 0.94 -- Key: HBASE-8211 URL: https://issues.apache.org/jira/browse/HBASE-8211 Project: HBase Issue Type: Bug Components: master, regionserver Affects Versions: 0.94.6 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Fix For: 0.94.7 Attachments: HBase-8211-v1.patch HBase-8156 is for adding support for retrying non-idempotent operations. This is useful in case NN is suffering from n/w hiccups, etc. This jira is to add similar support for 0.94.x branch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8211) Support for NN HA for 0.94
[ https://issues.apache.org/jira/browse/HBASE-8211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13617931#comment-13617931 ] Fengdong Yu commented on HBASE-8211: Himanshu, I also reviewed the patch, but this just added reties, not support HDFS HA really. Since during Namenode failover, FileSystem URI will be changed, but we always try using the old fs, I don't think this works well. Another, for HDFS HA, fs.defaultFS can be configured as hdfs://mycluster, does this patch support hbase.rootdir configured hdfs://mycluster/hbase ? Support for NN HA for 0.94 -- Key: HBASE-8211 URL: https://issues.apache.org/jira/browse/HBASE-8211 Project: HBase Issue Type: Bug Components: master, regionserver Affects Versions: 0.94.6 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Fix For: 0.94.7 Attachments: HBase-8211-v1.patch HBase-8156 is for adding support for retrying non-idempotent operations. This is useful in case NN is suffering from n/w hiccups, etc. This jira is to add similar support for 0.94.x branch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7251) Avoid flood logs during client disconnect during batch get operation
[
https://issues.apache.org/jira/browse/HBASE-7251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fengdong Yu updated HBASE-7251:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
Avoid flood logs during client disconnect during batch get operation
Key: HBASE-7251
URL: https://issues.apache.org/jira/browse/HBASE-7251
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.2
Reporter: Fengdong Yu
Labels: patch
Fix For: 0.94.4
Attachments: HBASE-7251.patch
Background:
A smart guy in the company want to read data from the HBASE in batch, the
code like the following:(just demonstrate, not runnable):
ListGet gets = new ArrayListGet();
for(int i =0; i n; ++i){
gets.add(some row key here);
if (i % 1 == 0){
Results[] results = htable.get(gets);
//process results here. so delete some code
}
}
Yes, you know that, this guy forgot gets.clear() after each htable.get()
operation in his code.
One region server becomes very slow, and crashed after 30mins becauseof OOM,
but we got 15GB log file.
there are flood logs as following:
ERROR org.apache.hadoop.hbase.regionserver.HRegionServer:
org.apache.hadoop.hbase.ipc.CallerDisconnectedException: Aborting call
multi(org.apache.hadoop.hbase.client.MultiAction@49540d8d), rpc version=1,
client version=29, methodsFingerPrint=-56040613 from 10.1.1.1:57933 after
3980 ms, since caller disconnected
at
org.apache.hadoop.hbase.ipc.HBaseServer$Call.throwExceptionIfCallerDisconnected(HBaseServer.java:436)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:3468)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3425)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3449)
at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4198)
at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4171)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1993)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3410)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1409)
Fix:
Server is stop get but cannot exit from the for loop, so write flood logs
here.
My patch just log one exception log instead of flood logs.
Importantly, server stop processing immediately if client timeout or
disconnect.
Test:
I used this guy's wrong code read data( NO gets.clear() ) from the HBASE,
when it becomes very slow to get results, I pressed ctrl+C, then there is
only ONE CallerDisconnectedException exception log and the server stop
reading immediately, LOG generate the last log entry:
WARN org.apache.hadoop.ipc.HBaseServer: IPC Server handler 1 on 60020 caught
a ClosedChannelException, this means that the server was processing a request
but the client went away. The error message was: null
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7251) Avoid flood logs during client disconnect during batch get operation
[
https://issues.apache.org/jira/browse/HBASE-7251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13508190#comment-13508190
]
Fengdong Yu commented on HBASE-7251:
apply patch failed , because I don't check out from
http://svn.apache.org/repos/asf/hbase/trunk,
I used: svn co http://svn.apache.org/repos/asf/hbase/branches/0.94 hbase-0.94.2
Avoid flood logs during client disconnect during batch get operation
Key: HBASE-7251
URL: https://issues.apache.org/jira/browse/HBASE-7251
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.2
Reporter: Fengdong Yu
Labels: patch
Fix For: 0.94.3
Attachments: HBASE-7251.patch
Background:
A smart guy in the company want to read data from the HBASE in batch, the
code like the following:(just demonstrate, not runnable):
ListGet gets = new ArrayListGet();
for(int i =0; i n; ++i){
gets.add(some row key here);
if (i % 1 == 0){
Results[] results = htable.get(gets);
//process results here. so delete some code
}
}
Yes, you know that, this guy forgot gets.clear() after each htable.get()
operation in his code.
One region server becomes very slow, and crashed after 30mins becauseof OOM,
but we got 15GB log file.
there are flood logs as following:
ERROR org.apache.hadoop.hbase.regionserver.HRegionServer:
org.apache.hadoop.hbase.ipc.CallerDisconnectedException: Aborting call
multi(org.apache.hadoop.hbase.client.MultiAction@49540d8d), rpc version=1,
client version=29, methodsFingerPrint=-56040613 from 10.1.1.1:57933 after
3980 ms, since caller disconnected
at
org.apache.hadoop.hbase.ipc.HBaseServer$Call.throwExceptionIfCallerDisconnected(HBaseServer.java:436)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:3468)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3425)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3449)
at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4198)
at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4171)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1993)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3410)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1409)
Fix:
Server is stop get but cannot exit from the for loop, so write flood logs
here.
My patch just log one exception log instead of flood logs.
Importantly, server stop processing immediately if client timeout or
disconnect.
Test:
I used this guy's wrong code read data( NO gets.clear() ) from the HBASE,
when it becomes very slow to get results, I pressed ctrl+C, then there is
only ONE CallerDisconnectedException exception log and the server stop
reading immediately, LOG generate the last log entry:
WARN org.apache.hadoop.ipc.HBaseServer: IPC Server handler 1 on 60020 caught
a ClosedChannelException, this means that the server was processing a request
but the client went away. The error message was: null
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7251) Avoid flood logs during client disconnect during batch get operation
[
https://issues.apache.org/jira/browse/HBASE-7251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13508185#comment-13508185
]
Fengdong Yu commented on HBASE-7251:
Andrew,
I don't think all ops need this patch, especially write ops, because server
should continue working even if client disconnect for writes.
Avoid flood logs during client disconnect during batch get operation
Key: HBASE-7251
URL: https://issues.apache.org/jira/browse/HBASE-7251
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.2
Reporter: Fengdong Yu
Labels: patch
Fix For: 0.94.3
Attachments: HBASE-7251.patch
Background:
A smart guy in the company want to read data from the HBASE in batch, the
code like the following:(just demonstrate, not runnable):
ListGet gets = new ArrayListGet();
for(int i =0; i n; ++i){
gets.add(some row key here);
if (i % 1 == 0){
Results[] results = htable.get(gets);
//process results here. so delete some code
}
}
Yes, you know that, this guy forgot gets.clear() after each htable.get()
operation in his code.
One region server becomes very slow, and crashed after 30mins becauseof OOM,
but we got 15GB log file.
there are flood logs as following:
ERROR org.apache.hadoop.hbase.regionserver.HRegionServer:
org.apache.hadoop.hbase.ipc.CallerDisconnectedException: Aborting call
multi(org.apache.hadoop.hbase.client.MultiAction@49540d8d), rpc version=1,
client version=29, methodsFingerPrint=-56040613 from 10.1.1.1:57933 after
3980 ms, since caller disconnected
at
org.apache.hadoop.hbase.ipc.HBaseServer$Call.throwExceptionIfCallerDisconnected(HBaseServer.java:436)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:3468)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3425)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3449)
at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4198)
at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4171)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1993)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3410)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1409)
Fix:
Server is stop get but cannot exit from the for loop, so write flood logs
here.
My patch just log one exception log instead of flood logs.
Importantly, server stop processing immediately if client timeout or
disconnect.
Test:
I used this guy's wrong code read data( NO gets.clear() ) from the HBASE,
when it becomes very slow to get results, I pressed ctrl+C, then there is
only ONE CallerDisconnectedException exception log and the server stop
reading immediately, LOG generate the last log entry:
WARN org.apache.hadoop.ipc.HBaseServer: IPC Server handler 1 on 60020 caught
a ClosedChannelException, this means that the server was processing a request
but the client went away. The error message was: null
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
