[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[ https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16201166#comment-16201166 ] Jerry He commented on HBASE-16010: -- I am going to close this ticket and post a patch on HBASE-10367, which will modify the API to decommission region servers (mark the drain and move off the regions). > Put draining function through Admin API > --- > > Key: HBASE-16010 > URL: https://issues.apache.org/jira/browse/HBASE-16010 > Project: HBase > Issue Type: Improvement >Reporter: Jerry He >Assignee: Jerry He >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16010-v3.patch, hbase-16010-v1.patch, > hbase-16010-v2.patch > > > Currently, there is no Amdin API for draining function. Client has to > interact directly with Zookeeper draining node to add and remove draining > servers. > For example, in draining_servers.rb: > {code} > zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config, > "draining_servers", nil) > parentZnode = zkw.drainingZNode > begin > for server in servers > node = ZKUtil.joinZNode(parentZnode, server) > ZKUtil.createAndFailSilent(zkw, node) > end > ensure > zkw.close() > end > {code} > This is not good in cases like secure clusters with protected Zookeeper nodes. > Let's put draining function through Admin API. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[ https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16183340#comment-16183340 ] Jerry He commented on HBASE-16010: -- I'd like to get a closure on this ticket. I am ready to add the 'decommission' logic as suggested by the folks here. Decommission: Mark the region servers as 'draining' so that no regions will be added. Also unload the regions on them. What about 'recommission'? Remove the 'draining' mode. Then load the regions? What regions to load? API accepts a list of regions to region servers? Or only remove the 'draining' mode, and let the balancer take care? > Put draining function through Admin API > --- > > Key: HBASE-16010 > URL: https://issues.apache.org/jira/browse/HBASE-16010 > Project: HBase > Issue Type: Improvement >Reporter: Jerry He >Assignee: Jerry He >Priority: Minor > Fix For: 2.0.0 > > Attachments: hbase-16010-v1.patch, hbase-16010-v2.patch, > HBASE-16010-v3.patch > > > Currently, there is no Amdin API for draining function. Client has to > interact directly with Zookeeper draining node to add and remove draining > servers. > For example, in draining_servers.rb: > {code} > zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config, > "draining_servers", nil) > parentZnode = zkw.drainingZNode > begin > for server in servers > node = ZKUtil.joinZNode(parentZnode, server) > ZKUtil.createAndFailSilent(zkw, node) > end > ensure > zkw.close() > end > {code} > This is not good in cases like secure clusters with protected Zookeeper nodes. > Let's put draining function through Admin API. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[ https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905501#comment-15905501 ] Jerry He commented on HBASE-16010: -- I was trying to give myself more time to think, but forgot to get back to it. Sorry about it. The current patch provides API via master to put region servers in 'drain' mode indicating no regions should be assigned here. The argument is whether this JIRA should include additional work to actually move the regions out from the affected server, again carried out on master, thus providing a true 'decommission' feature, which is not available anywhere yet. A rb script currently does part of the work. I like the idea. If the contributor will extend the work, I will be glad to help. There was concern doing the bulk move of the regions on the master at the moment given the ongoing work in the AM proc. > Put draining function through Admin API > --- > > Key: HBASE-16010 > URL: https://issues.apache.org/jira/browse/HBASE-16010 > Project: HBase > Issue Type: Improvement >Reporter: Jerry He >Assignee: Jerry He >Priority: Minor > Fix For: 2.0.0 > > Attachments: hbase-16010-v1.patch, hbase-16010-v2.patch, > HBASE-16010-v3.patch > > > Currently, there is no Amdin API for draining function. Client has to > interact directly with Zookeeper draining node to add and remove draining > servers. > For example, in draining_servers.rb: > {code} > zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config, > "draining_servers", nil) > parentZnode = zkw.drainingZNode > begin > for server in servers > node = ZKUtil.joinZNode(parentZnode, server) > ZKUtil.createAndFailSilent(zkw, node) > end > ensure > zkw.close() > end > {code} > This is not good in cases like secure clusters with protected Zookeeper nodes. > Let's put draining function through Admin API. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[ https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905034#comment-15905034 ] Ashish Singhi commented on HBASE-16010: --- I see this jira is in reopened state but the patch code is available in master branch. So what's the final conclusion ? > Put draining function through Admin API > --- > > Key: HBASE-16010 > URL: https://issues.apache.org/jira/browse/HBASE-16010 > Project: HBase > Issue Type: Improvement >Reporter: Jerry He >Assignee: Jerry He >Priority: Minor > Fix For: 2.0.0 > > Attachments: hbase-16010-v1.patch, hbase-16010-v2.patch, > HBASE-16010-v3.patch > > > Currently, there is no Amdin API for draining function. Client has to > interact directly with Zookeeper draining node to add and remove draining > servers. > For example, in draining_servers.rb: > {code} > zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config, > "draining_servers", nil) > parentZnode = zkw.drainingZNode > begin > for server in servers > node = ZKUtil.joinZNode(parentZnode, server) > ZKUtil.createAndFailSilent(zkw, node) > end > ensure > zkw.close() > end > {code} > This is not good in cases like secure clusters with protected Zookeeper nodes. > Let's put draining function through Admin API. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[ https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15797412#comment-15797412 ] Appy commented on HBASE-16010: -- bq. There is currently a slight difference between 'drain mode' and 'decommission' from what I understand Do we need to continue with two concepts? I spent good time on this feature yesterday, but never realized that there was a difference. Let's not put users through this misery. +1 on Enis's suggestion - extend work done in this jira with region_mover to build proper decommission API. As for scripts vs shell, let's continue that in HBASE-17370. > Put draining function through Admin API > --- > > Key: HBASE-16010 > URL: https://issues.apache.org/jira/browse/HBASE-16010 > Project: HBase > Issue Type: Improvement >Reporter: Jerry He >Assignee: Jerry He >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16010-v3.patch, hbase-16010-v1.patch, > hbase-16010-v2.patch > > > Currently, there is no Amdin API for draining function. Client has to > interact directly with Zookeeper draining node to add and remove draining > servers. > For example, in draining_servers.rb: > {code} > zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config, > "draining_servers", nil) > parentZnode = zkw.drainingZNode > begin > for server in servers > node = ZKUtil.joinZNode(parentZnode, server) > ZKUtil.createAndFailSilent(zkw, node) > end > ensure > zkw.close() > end > {code} > This is not good in cases like secure clusters with protected Zookeeper nodes. > Let's put draining function through Admin API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[ https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15796881#comment-15796881 ] Jerry He commented on HBASE-16010: -- This is good plan, if I missed it earlier! Ok. let us change the names then. > Put draining function through Admin API > --- > > Key: HBASE-16010 > URL: https://issues.apache.org/jira/browse/HBASE-16010 > Project: HBase > Issue Type: Improvement >Reporter: Jerry He >Assignee: Jerry He >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16010-v3.patch, hbase-16010-v1.patch, > hbase-16010-v2.patch > > > Currently, there is no Amdin API for draining function. Client has to > interact directly with Zookeeper draining node to add and remove draining > servers. > For example, in draining_servers.rb: > {code} > zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config, > "draining_servers", nil) > parentZnode = zkw.drainingZNode > begin > for server in servers > node = ZKUtil.joinZNode(parentZnode, server) > ZKUtil.createAndFailSilent(zkw, node) > end > ensure > zkw.close() > end > {code} > This is not good in cases like secure clusters with protected Zookeeper nodes. > Let's put draining function through Admin API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[ https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15796815#comment-15796815 ] Enis Soztutar commented on HBASE-16010: --- I actually want to fix the drain script, move away from it and implement decommissioning in the master side properly, as does mighty [~larsgeorge]. There are a couple of threads going around in the dev mailing list. HBASE-10367 is the main issue. I was thinking that we can introduce the decommission APIs as is in this jira, and later on follow up with actually moving the region mover to be executed inside the master in the same API call. So in that sense, the API names will be for future use when we will do the actual decommissioning. > Put draining function through Admin API > --- > > Key: HBASE-16010 > URL: https://issues.apache.org/jira/browse/HBASE-16010 > Project: HBase > Issue Type: Improvement >Reporter: Jerry He >Assignee: Matt Warhaftig >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16010-v3.patch, hbase-16010-v1.patch, > hbase-16010-v2.patch > > > Currently, there is no Amdin API for draining function. Client has to > interact directly with Zookeeper draining node to add and remove draining > servers. > For example, in draining_servers.rb: > {code} > zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config, > "draining_servers", nil) > parentZnode = zkw.drainingZNode > begin > for server in servers > node = ZKUtil.joinZNode(parentZnode, server) > ZKUtil.createAndFailSilent(zkw, node) > end > ensure > zkw.close() > end > {code} > This is not good in cases like secure clusters with protected Zookeeper nodes. > Let's put draining function through Admin API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[ https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15796779#comment-15796779 ] Jerry He commented on HBASE-16010: -- bq. Would be great to use Enis's above suggestion. Instead of drainRegionServers*/removeDrainFromRegionServers* in functions and PBs, decommissionRegionServers*/recommissionRegionServers* would have been better. There is currently a slight difference between 'drain mode' and 'decommission' from what I understand. 'decommission' uses the region mover to offload the regions in addition to putting the server on 'drain mode', where the drain API only does the latter. I think we should just provide the drain API as they are in the patch. The java doc of the API clearly stated their purpose. Then we can have a single shell command or script that provides the convenient 'decommission' feature. A subtask was created to do that. How does it sound? [~enis] and [~appy] > Put draining function through Admin API > --- > > Key: HBASE-16010 > URL: https://issues.apache.org/jira/browse/HBASE-16010 > Project: HBase > Issue Type: Improvement >Reporter: Jerry He >Assignee: Matt Warhaftig >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16010-v3.patch, hbase-16010-v1.patch, > hbase-16010-v2.patch > > > Currently, there is no Amdin API for draining function. Client has to > interact directly with Zookeeper draining node to add and remove draining > servers. > For example, in draining_servers.rb: > {code} > zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config, > "draining_servers", nil) > parentZnode = zkw.drainingZNode > begin > for server in servers > node = ZKUtil.joinZNode(parentZnode, server) > ZKUtil.createAndFailSilent(zkw, node) > end > ensure > zkw.close() > end > {code} > This is not good in cases like secure clusters with protected Zookeeper nodes. > Let's put draining function through Admin API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[ https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15795329#comment-15795329 ] Appy commented on HBASE-16010: -- Sorry for late review, was away on vacation. Please consider making following changes in an addendum patch. bq. Instead of draining, we should use the term decommission/recommission I think Would be great to use Enis's above suggestion. Instead of drainRegionServers*/removeDrainFromRegionServers* in functions and PBs, decommissionRegionServers*/recommissionRegionServers* would have been better. bq. ListDrainingRegionServersRequest.newBuilder().build() ListDrainingRegionServersRequest.getDefaultInstance() bq. listDrainingRegionServers(null, req) Why is it null? bq. {noformat} 1702try { 1703 master.checkInitialized(); 1704 List servers = master.listDrainingRegionServers(); 1705 for (ServerName server : servers) { 1706response.addServerName(ProtobufUtil.toServerName(server)); 1707 } 1708} catch (IOException io) { 1709 throw new ServiceException(io); 1710} {noformat} only master.checkInitilized needs exception handling, move others out of the block. ditto for next two functions bq. DrainRegionServersResponse.Builder response = DrainRegionServersResponse.newBuilder(); Directly return DrainRegionServersResponse.getDefaultInstance() in the end. ditto for RemoveDrainFromRegionServersResponse. In testDrainRegionServers(), instead of manually maintaining thread,executor,future, please use admin.createTableAsync() and wait on the future. > Put draining function through Admin API > --- > > Key: HBASE-16010 > URL: https://issues.apache.org/jira/browse/HBASE-16010 > Project: HBase > Issue Type: Improvement >Reporter: Jerry He >Assignee: Matt Warhaftig >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16010-v3.patch, hbase-16010-v1.patch, > hbase-16010-v2.patch > > > Currently, there is no Amdin API for draining function. Client has to > interact directly with Zookeeper draining node to add and remove draining > servers. > For example, in draining_servers.rb: > {code} > zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config, > "draining_servers", nil) > parentZnode = zkw.drainingZNode > begin > for server in servers > node = ZKUtil.joinZNode(parentZnode, server) > ZKUtil.createAndFailSilent(zkw, node) > end > ensure > zkw.close() > end > {code} > This is not good in cases like secure clusters with protected Zookeeper nodes. > Let's put draining function through Admin API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[ https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15779161#comment-15779161 ] Jerry He commented on HBASE-16010: -- I think it should require global admin, similar to the 'balance' operation. > Put draining function through Admin API > --- > > Key: HBASE-16010 > URL: https://issues.apache.org/jira/browse/HBASE-16010 > Project: HBase > Issue Type: Improvement >Reporter: Jerry He >Assignee: Matt Warhaftig >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16010-v3.patch, hbase-16010-v1.patch, > hbase-16010-v2.patch > > > Currently, there is no Amdin API for draining function. Client has to > interact directly with Zookeeper draining node to add and remove draining > servers. > For example, in draining_servers.rb: > {code} > zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config, > "draining_servers", nil) > parentZnode = zkw.drainingZNode > begin > for server in servers > node = ZKUtil.joinZNode(parentZnode, server) > ZKUtil.createAndFailSilent(zkw, node) > end > ensure > zkw.close() > end > {code} > This is not good in cases like secure clusters with protected Zookeeper nodes. > Let's put draining function through Admin API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[ https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15778964#comment-15778964 ] Matt Warhaftig commented on HBASE-16010: {quote} "Matt Warhaftig, would you like to work on the ACL task?" {quote} Would ACL just entail requiring that the user doing the draining functionality has admin (table scope) access. > Put draining function through Admin API > --- > > Key: HBASE-16010 > URL: https://issues.apache.org/jira/browse/HBASE-16010 > Project: HBase > Issue Type: Improvement >Reporter: Jerry He >Assignee: Matt Warhaftig >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16010-v3.patch, hbase-16010-v1.patch, > hbase-16010-v2.patch > > > Currently, there is no Amdin API for draining function. Client has to > interact directly with Zookeeper draining node to add and remove draining > servers. > For example, in draining_servers.rb: > {code} > zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config, > "draining_servers", nil) > parentZnode = zkw.drainingZNode > begin > for server in servers > node = ZKUtil.joinZNode(parentZnode, server) > ZKUtil.createAndFailSilent(zkw, node) > end > ensure > zkw.close() > end > {code} > This is not good in cases like secure clusters with protected Zookeeper nodes. > Let's put draining function through Admin API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[ https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15774194#comment-15774194 ] Hudson commented on HBASE-16010: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2187 (See [https://builds.apache.org/job/HBase-Trunk_matrix/2187/]) HBASE-16010 Put draining function through Admin API (Matt Warhaftig) (jerryjch: rev 992e5717d4e4deeef46836acea323a312b1e0851) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin2.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockNoopMasterServices.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/MasterProtos.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java * (edit) hbase-protocol-shaded/src/main/protobuf/Master.proto > Put draining function through Admin API > --- > > Key: HBASE-16010 > URL: https://issues.apache.org/jira/browse/HBASE-16010 > Project: HBase > Issue Type: Improvement >Reporter: Jerry He >Assignee: Matt Warhaftig >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16010-v3.patch, hbase-16010-v1.patch, > hbase-16010-v2.patch > > > Currently, there is no Amdin API for draining function. Client has to > interact directly with Zookeeper draining node to add and remove draining > servers. > For example, in draining_servers.rb: > {code} > zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config, > "draining_servers", nil) > parentZnode = zkw.drainingZNode > begin > for server in servers > node = ZKUtil.joinZNode(parentZnode, server) > ZKUtil.createAndFailSilent(zkw, node) > end > ensure > zkw.close() > end > {code} > This is not good in cases like secure clusters with protected Zookeeper nodes. > Let's put draining function through Admin API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[ https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15773763#comment-15773763 ] Jerry He commented on HBASE-16010: -- Committed to master. v3 is the committed version with the mentioned minor changes and rebase to master again. [~mwarhaftig], would you like to work on the ACL task? > Put draining function through Admin API > --- > > Key: HBASE-16010 > URL: https://issues.apache.org/jira/browse/HBASE-16010 > Project: HBase > Issue Type: Improvement >Reporter: Jerry He >Assignee: Matt Warhaftig >Priority: Minor > Attachments: hbase-16010-v1.patch, hbase-16010-v2.patch > > > Currently, there is no Amdin API for draining function. Client has to > interact directly with Zookeeper draining node to add and remove draining > servers. > For example, in draining_servers.rb: > {code} > zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config, > "draining_servers", nil) > parentZnode = zkw.drainingZNode > begin > for server in servers > node = ZKUtil.joinZNode(parentZnode, server) > ZKUtil.createAndFailSilent(zkw, node) > end > ensure > zkw.close() > end > {code} > This is not good in cases like secure clusters with protected Zookeeper nodes. > Let's put draining function through Admin API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[ https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15773507#comment-15773507 ] Jerry He commented on HBASE-16010: -- Thanks, [~enis]. Created the subtasks. bq. GetDrainingServers should be ListDrainingServers Make this change too. > Put draining function through Admin API > --- > > Key: HBASE-16010 > URL: https://issues.apache.org/jira/browse/HBASE-16010 > Project: HBase > Issue Type: Improvement >Reporter: Jerry He >Assignee: Matt Warhaftig >Priority: Minor > Attachments: hbase-16010-v1.patch, hbase-16010-v2.patch > > > Currently, there is no Amdin API for draining function. Client has to > interact directly with Zookeeper draining node to add and remove draining > servers. > For example, in draining_servers.rb: > {code} > zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config, > "draining_servers", nil) > parentZnode = zkw.drainingZNode > begin > for server in servers > node = ZKUtil.joinZNode(parentZnode, server) > ZKUtil.createAndFailSilent(zkw, node) > end > ensure > zkw.close() > end > {code} > This is not good in cases like secure clusters with protected Zookeeper nodes. > Let's put draining function through Admin API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[ https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15771486#comment-15771486 ] Enis Soztutar commented on HBASE-16010: --- bq. Let me open a subtask right away and make sure it will be in. Ok, as long as there is a critical issue marked against the versions, and a commitment to fix it, I'll be happy. bq. You think we should combine the two steps into one? I think so, having the draining functionality in a ruby script is not acceptable (for the long term). That is why I was asking for the plans around that. We do not have to address them now, but at least track those in a jira. > Put draining function through Admin API > --- > > Key: HBASE-16010 > URL: https://issues.apache.org/jira/browse/HBASE-16010 > Project: HBase > Issue Type: Improvement >Reporter: Jerry He >Assignee: Matt Warhaftig >Priority: Minor > Attachments: hbase-16010-v1.patch, hbase-16010-v2.patch > > > Currently, there is no Amdin API for draining function. Client has to > interact directly with Zookeeper draining node to add and remove draining > servers. > For example, in draining_servers.rb: > {code} > zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config, > "draining_servers", nil) > parentZnode = zkw.drainingZNode > begin > for server in servers > node = ZKUtil.joinZNode(parentZnode, server) > ZKUtil.createAndFailSilent(zkw, node) > end > ensure > zkw.close() > end > {code} > This is not good in cases like secure clusters with protected Zookeeper nodes. > Let's put draining function through Admin API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[ https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15768204#comment-15768204 ] Jerry He commented on HBASE-16010: -- Hi, [~enis] bq. I think we should have the ACL changes in this patch as well. Otherwise, it will get forgotten and leave a security hole We surely need to get the ACL in. But let's get this JIRA with the protobuf changes in first? Mixing the ACL observers and the protobuf changes will probably bloat the patch and confusing. Let me open a subtask right away and make sure it will be in. Currently, the decommission works this way (i played with it recently.) 1. Put the server in drain mode. 2 Move the regions off with the region mover. You think we should combine the two steps into one? bq. "decommissioning" a server should be integral to the new assignment manager in the sense that the core assignment should be aware of decommissioning servers. I think currently if a server is in drain mode, serverManager/assignment/balancer will skip it as candidate servers. But not sure much about the details. > Put draining function through Admin API > --- > > Key: HBASE-16010 > URL: https://issues.apache.org/jira/browse/HBASE-16010 > Project: HBase > Issue Type: Improvement >Reporter: Jerry He >Assignee: Matt Warhaftig >Priority: Minor > Attachments: hbase-16010-v1.patch, hbase-16010-v2.patch > > > Currently, there is no Amdin API for draining function. Client has to > interact directly with Zookeeper draining node to add and remove draining > servers. > For example, in draining_servers.rb: > {code} > zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config, > "draining_servers", nil) > parentZnode = zkw.drainingZNode > begin > for server in servers > node = ZKUtil.joinZNode(parentZnode, server) > ZKUtil.createAndFailSilent(zkw, node) > end > ensure > zkw.close() > end > {code} > This is not good in cases like secure clusters with protected Zookeeper nodes. > Let's put draining function through Admin API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[ https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15768178#comment-15768178 ] Jerry He commented on HBASE-16010: -- bq. MasterRpcServices already has a ServerName import entry (org.apache.hadoop.hbase.ServerName) and they would conflict. Use HBaseProtos.ServerName > Put draining function through Admin API > --- > > Key: HBASE-16010 > URL: https://issues.apache.org/jira/browse/HBASE-16010 > Project: HBase > Issue Type: Improvement >Reporter: Jerry He >Assignee: Matt Warhaftig >Priority: Minor > Attachments: hbase-16010-v1.patch, hbase-16010-v2.patch > > > Currently, there is no Amdin API for draining function. Client has to > interact directly with Zookeeper draining node to add and remove draining > servers. > For example, in draining_servers.rb: > {code} > zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config, > "draining_servers", nil) > parentZnode = zkw.drainingZNode > begin > for server in servers > node = ZKUtil.joinZNode(parentZnode, server) > ZKUtil.createAndFailSilent(zkw, node) > end > ensure > zkw.close() > end > {code} > This is not good in cases like secure clusters with protected Zookeeper nodes. > Let's put draining function through Admin API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[ https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15767918#comment-15767918 ] Enis Soztutar commented on HBASE-16010: --- I think we should have the ACL changes in this patch as well. Otherwise, it will get forgotten and leave a security hole. Instead of draining, we should use the term decommission/recommission I think. And GetDrainingServers should be ListDrainingServers, or ListDecommissionedServers. This is obviously already broken, but the new API right now only puts the server in "draining" mode and does not do anything else. Is there a plan to bring the actual functionality (of moving regions out of the RS) in the master as well? As I have noted elsewhere, "decommissioning" a server should be integral to the new assignment manager in the sense that the core assignment should be aware of decommissioning servers. [~stack], [~syuanjiang] what do you guys think? Is the current stuff have ways to address that? > Put draining function through Admin API > --- > > Key: HBASE-16010 > URL: https://issues.apache.org/jira/browse/HBASE-16010 > Project: HBase > Issue Type: Improvement >Reporter: Jerry He >Assignee: Matt Warhaftig >Priority: Minor > Attachments: hbase-16010-v1.patch, hbase-16010-v2.patch > > > Currently, there is no Amdin API for draining function. Client has to > interact directly with Zookeeper draining node to add and remove draining > servers. > For example, in draining_servers.rb: > {code} > zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config, > "draining_servers", nil) > parentZnode = zkw.drainingZNode > begin > for server in servers > node = ZKUtil.joinZNode(parentZnode, server) > ZKUtil.createAndFailSilent(zkw, node) > end > ensure > zkw.close() > end > {code} > This is not good in cases like secure clusters with protected Zookeeper nodes. > Let's put draining function through Admin API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[ https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15767600#comment-15767600 ] Matt Warhaftig commented on HBASE-16010: Thanks Jerry, Feel free to fix the first issue and commit. As for the second issue, MasterRpcServices uses the fully qualified classname of {{org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos.ServerName}} because MasterRpcServices already has a {{ServerName}} import entry ({{org.apache.hadoop.hbase.ServerName}}) and they would conflict. > Put draining function through Admin API > --- > > Key: HBASE-16010 > URL: https://issues.apache.org/jira/browse/HBASE-16010 > Project: HBase > Issue Type: Improvement >Reporter: Jerry He >Assignee: Matt Warhaftig >Priority: Minor > Attachments: hbase-16010-v1.patch, hbase-16010-v2.patch > > > Currently, there is no Amdin API for draining function. Client has to > interact directly with Zookeeper draining node to add and remove draining > servers. > For example, in draining_servers.rb: > {code} > zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config, > "draining_servers", nil) > parentZnode = zkw.drainingZNode > begin > for server in servers > node = ZKUtil.joinZNode(parentZnode, server) > ZKUtil.createAndFailSilent(zkw, node) > end > ensure > zkw.close() > end > {code} > This is not good in cases like secure clusters with protected Zookeeper nodes. > Let's put draining function through Admin API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[ https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15766128#comment-15766128 ] Jerry He commented on HBASE-16010: -- Can fix on commit. > Put draining function through Admin API > --- > > Key: HBASE-16010 > URL: https://issues.apache.org/jira/browse/HBASE-16010 > Project: HBase > Issue Type: Improvement >Reporter: Jerry He >Assignee: Matt Warhaftig >Priority: Minor > Attachments: hbase-16010-v1.patch, hbase-16010-v2.patch > > > Currently, there is no Amdin API for draining function. Client has to > interact directly with Zookeeper draining node to add and remove draining > servers. > For example, in draining_servers.rb: > {code} > zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config, > "draining_servers", nil) > parentZnode = zkw.drainingZNode > begin > for server in servers > node = ZKUtil.joinZNode(parentZnode, server) > ZKUtil.createAndFailSilent(zkw, node) > end > ensure > zkw.close() > end > {code} > This is not good in cases like secure clusters with protected Zookeeper nodes. > Let's put draining function through Admin API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[ https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15766066#comment-15766066 ] Jerry He commented on HBASE-16010: -- +1 Some nit. In HBaseAdmin, two places: {noformat} +executeCallable(new MasterCallable(getConnection(), getRpcControllerFactory()) { + @Override + public Void rpcCall() throws ServiceException { +HBaseRpcController controller = rpcControllerFactory.newController(); {noformat} Use getRpcController() instead of creating a new one. In MasterRpcServices, two places: {noformat} + for (org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos.ServerName pbServer : request {noformat} Let's use the short name in the code and add to import. > Put draining function through Admin API > --- > > Key: HBASE-16010 > URL: https://issues.apache.org/jira/browse/HBASE-16010 > Project: HBase > Issue Type: Improvement >Reporter: Jerry He >Assignee: Matt Warhaftig >Priority: Minor > Attachments: hbase-16010-v1.patch, hbase-16010-v2.patch > > > Currently, there is no Amdin API for draining function. Client has to > interact directly with Zookeeper draining node to add and remove draining > servers. > For example, in draining_servers.rb: > {code} > zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config, > "draining_servers", nil) > parentZnode = zkw.drainingZNode > begin > for server in servers > node = ZKUtil.joinZNode(parentZnode, server) > ZKUtil.createAndFailSilent(zkw, node) > end > ensure > zkw.close() > end > {code} > This is not good in cases like secure clusters with protected Zookeeper nodes. > Let's put draining function through Admin API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[ https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15765646#comment-15765646 ] Matt Warhaftig commented on HBASE-16010: [~jinghe], Can you or someone else please review the new patch soon to avoid the need for another rebase later on. Thanks, Matt > Put draining function through Admin API > --- > > Key: HBASE-16010 > URL: https://issues.apache.org/jira/browse/HBASE-16010 > Project: HBase > Issue Type: Improvement >Reporter: Jerry He >Assignee: Matt Warhaftig >Priority: Minor > Attachments: hbase-16010-v1.patch, hbase-16010-v2.patch > > > Currently, there is no Amdin API for draining function. Client has to > interact directly with Zookeeper draining node to add and remove draining > servers. > For example, in draining_servers.rb: > {code} > zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config, > "draining_servers", nil) > parentZnode = zkw.drainingZNode > begin > for server in servers > node = ZKUtil.joinZNode(parentZnode, server) > ZKUtil.createAndFailSilent(zkw, node) > end > ensure > zkw.close() > end > {code} > This is not good in cases like secure clusters with protected Zookeeper nodes. > Let's put draining function through Admin API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[ https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15760183#comment-15760183 ] Hadoop QA commented on HBASE-16010: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 14s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s {color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 3 new or modified test files. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 18s {color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 49s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 16s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 10m 19s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 29s {color} | {color:green} master passed {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 49s {color} | {color:red} hbase-protocol-shaded in master has 24 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 49s {color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s {color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 24s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 16s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 16s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 16s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 10m 44s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 29s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s {color} | {color:red} The patch has 33 line(s) that end in whitespace. Use git apply --whitespace=fix. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 25m 19s {color} | {color:green} Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} | | {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 1m 0s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m 0s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 48s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 26s {color} | {color:green} hbase-protocol-shaded in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 56s {color} | {color:green} hbase-client in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 97m 11s {color} | {color:green} hbase-server in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 41s {color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 166m 37s {color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12843780/hbase-16010-v2.patch | | JIRA Issue | HBASE-16010 | | Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc | | uname | Linux 49010f70ad30 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality |
[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[ https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15687746#comment-15687746 ] Hadoop QA commented on HBASE-16010: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s {color} | {color:blue} Docker mode activated. {color} | | {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s {color} | {color:red} HBASE-16010 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/0.3.0/precommit-patchnames for help. {color} | \\ \\ || Subsystem || Report/Notes || | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12811470/hbase-16010-v1.patch | | JIRA Issue | HBASE-16010 | | Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/4577/console | | Powered by | Apache Yetus 0.3.0 http://yetus.apache.org | This message was automatically generated. > Put draining function through Admin API > --- > > Key: HBASE-16010 > URL: https://issues.apache.org/jira/browse/HBASE-16010 > Project: HBase > Issue Type: Improvement >Reporter: Jerry He >Assignee: Matt Warhaftig >Priority: Minor > Attachments: hbase-16010-v1.patch > > > Currently, there is no Amdin API for draining function. Client has to > interact directly with Zookeeper draining node to add and remove draining > servers. > For example, in draining_servers.rb: > {code} > zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config, > "draining_servers", nil) > parentZnode = zkw.drainingZNode > begin > for server in servers > node = ZKUtil.joinZNode(parentZnode, server) > ZKUtil.createAndFailSilent(zkw, node) > end > ensure > zkw.close() > end > {code} > This is not good in cases like secure clusters with protected Zookeeper nodes. > Let's put draining function through Admin API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[ https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15687741#comment-15687741 ] Jerry He commented on HBASE-16010: -- [~mwarhaftig] Could you do a rebase with the latest master? The master has quite a bit of refactoring on the protobuf classes. > Put draining function through Admin API > --- > > Key: HBASE-16010 > URL: https://issues.apache.org/jira/browse/HBASE-16010 > Project: HBase > Issue Type: Improvement >Reporter: Jerry He >Assignee: Matt Warhaftig >Priority: Minor > Attachments: hbase-16010-v1.patch > > > Currently, there is no Amdin API for draining function. Client has to > interact directly with Zookeeper draining node to add and remove draining > servers. > For example, in draining_servers.rb: > {code} > zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config, > "draining_servers", nil) > parentZnode = zkw.drainingZNode > begin > for server in servers > node = ZKUtil.joinZNode(parentZnode, server) > ZKUtil.createAndFailSilent(zkw, node) > end > ensure > zkw.close() > end > {code} > This is not good in cases like secure clusters with protected Zookeeper nodes. > Let's put draining function through Admin API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[ https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15360411#comment-15360411 ] Jerry He commented on HBASE-16010: -- Patch lgtm. We need a follow-on patch to add ACL to the APIs. > Put draining function through Admin API > --- > > Key: HBASE-16010 > URL: https://issues.apache.org/jira/browse/HBASE-16010 > Project: HBase > Issue Type: Improvement >Reporter: Jerry He >Assignee: Matt Warhaftig >Priority: Minor > Attachments: hbase-16010-v1.patch > > > Currently, there is no Amdin API for draining function. Client has to > interact directly with Zookeeper draining node to add and remove draining > servers. > For example, in draining_servers.rb: > {code} > zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config, > "draining_servers", nil) > parentZnode = zkw.drainingZNode > begin > for server in servers > node = ZKUtil.joinZNode(parentZnode, server) > ZKUtil.createAndFailSilent(zkw, node) > end > ensure > zkw.close() > end > {code} > This is not good in cases like secure clusters with protected Zookeeper nodes. > Let's put draining function through Admin API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[ https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337387#comment-15337387 ] Hadoop QA commented on HBASE-16010: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s {color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 3 new or modified test files. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 3m 20s {color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 51s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 57s {color} | {color:green} master passed with JDK v1.8.0 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 23s {color} | {color:green} master passed with JDK v1.7.0_80 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 7m 34s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m 15s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m 55s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 52s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 24s {color} | {color:green} master passed with JDK v1.8.0 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 4s {color} | {color:green} master passed with JDK v1.7.0_80 {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s {color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 34s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 35s {color} | {color:green} the patch passed with JDK v1.8.0 {color} | | {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 35s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 35s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 24s {color} | {color:green} the patch passed with JDK v1.7.0_80 {color} | | {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 24s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 24s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 7m 34s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 46s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s {color} | {color:red} The patch has 30 line(s) that end in whitespace. Use git apply --whitespace=fix. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 33m 44s {color} | {color:green} Patch does not cause any errors with Hadoop 2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} | | {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m 50s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m 21s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 10s {color} | {color:green} the patch passed with JDK v1.8.0 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 59s {color} | {color:green} the patch passed with JDK v1.7.0_80 {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 20s {color} | {color:green} hbase-protocol in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 1s {color} | {color:green} hbase-client in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 99m 19s {color} | {color:green} hbase-server in the patch passed. {color} | |
[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[ https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15328275#comment-15328275 ] Matt Warhaftig commented on HBASE-16010: {quote} "or was the method supposed to be a List getDrainingRegionServers()? which is what the ServerManager.getDrainingServersList()?" {quote} Yes, you are on top of things Matteo - the method signature was changed from that preliminary draft to return a List of the draining region servers. As of 6/13 the RPC and API methods are written and closely follow the ruby script's logic. Am hoping to wrap up the development in the next 48 hours. > Put draining function through Admin API > --- > > Key: HBASE-16010 > URL: https://issues.apache.org/jira/browse/HBASE-16010 > Project: HBase > Issue Type: Improvement >Reporter: Jerry He >Assignee: Matt Warhaftig >Priority: Minor > > Currently, there is no Amdin API for draining function. Client has to > interact directly with Zookeeper draining node to add and remove draining > servers. > For example, in draining_servers.rb: > {code} > zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config, > "draining_servers", nil) > parentZnode = zkw.drainingZNode > begin > for server in servers > node = ZKUtil.joinZNode(parentZnode, server) > ZKUtil.createAndFailSilent(zkw, node) > end > ensure > zkw.close() > end > {code} > This is not good in cases like secure clusters with protected Zookeeper nodes. > Let's put draining function through Admin API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[ https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15328274#comment-15328274 ] Enis Soztutar commented on HBASE-16010: --- bq. I don't remember anything in the AM or ServerManager that keeps track of regions moving because of the server in the draining list. I think we should have decommissioning mode for servers in the core assignment paths as a concept. Not this patch though, we should do it in the assignment re-work. Both assignment and LB should be aware of decommissioning servers. > Put draining function through Admin API > --- > > Key: HBASE-16010 > URL: https://issues.apache.org/jira/browse/HBASE-16010 > Project: HBase > Issue Type: Improvement >Reporter: Jerry He >Assignee: Matt Warhaftig >Priority: Minor > > Currently, there is no Amdin API for draining function. Client has to > interact directly with Zookeeper draining node to add and remove draining > servers. > For example, in draining_servers.rb: > {code} > zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config, > "draining_servers", nil) > parentZnode = zkw.drainingZNode > begin > for server in servers > node = ZKUtil.joinZNode(parentZnode, server) > ZKUtil.createAndFailSilent(zkw, node) > end > ensure > zkw.close() > end > {code} > This is not good in cases like secure clusters with protected Zookeeper nodes. > Let's put draining function through Admin API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[ https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15327817#comment-15327817 ] Jerry He commented on HBASE-16010: -- Concur with Matteo. We can probably use similar API names as in the ServerManager. We still need to write to ZK node from master, in case one master fails. Then the direct client ZK call is still possible. > Put draining function through Admin API > --- > > Key: HBASE-16010 > URL: https://issues.apache.org/jira/browse/HBASE-16010 > Project: HBase > Issue Type: Improvement >Reporter: Jerry He >Assignee: Matt Warhaftig >Priority: Minor > > Currently, there is no Amdin API for draining function. Client has to > interact directly with Zookeeper draining node to add and remove draining > servers. > For example, in draining_servers.rb: > {code} > zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config, > "draining_servers", nil) > parentZnode = zkw.drainingZNode > begin > for server in servers > node = ZKUtil.joinZNode(parentZnode, server) > ZKUtil.createAndFailSilent(zkw, node) > end > ensure > zkw.close() > end > {code} > This is not good in cases like secure clusters with protected Zookeeper nodes. > Let's put draining function through Admin API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[ https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15327694#comment-15327694 ] Matteo Bertozzi commented on HBASE-16010: - how does the getDrainingRegions() works? doesn't this just turn out to be getOnlineRegions(ServerName)? I don't remember anything in the AM or ServerManager that keeps track of regions moving because of the server in the draining list. or was the method supposed to be a List getDrainingRegionServers()? which is what the ServerManager.getDrainingServersList()? (pretty sure that you already know this, but just to make it clear. we need to also add Master RPCs if we want to add support for secure clusters and ACLs. and we probably need to have a fallback to zk calls if the client is new but the server is old to keep compat, at least in branch-1) > Put draining function through Admin API > --- > > Key: HBASE-16010 > URL: https://issues.apache.org/jira/browse/HBASE-16010 > Project: HBase > Issue Type: Improvement >Reporter: Jerry He >Assignee: Matt Warhaftig >Priority: Minor > > Currently, there is no Amdin API for draining function. Client has to > interact directly with Zookeeper draining node to add and remove draining > servers. > For example, in draining_servers.rb: > {code} > zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config, > "draining_servers", nil) > parentZnode = zkw.drainingZNode > begin > for server in servers > node = ZKUtil.joinZNode(parentZnode, server) > ZKUtil.createAndFailSilent(zkw, node) > end > ensure > zkw.close() > end > {code} > This is not good in cases like secure clusters with protected Zookeeper nodes. > Let's put draining function through Admin API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[ https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15326049#comment-15326049 ] Matt Warhaftig commented on HBASE-16010: Proposed Admin drain methods and signatures: Drain RegionServer: - void drainRegionServer(final ServerName sn); List Draining Regions: - List getDrainingRegions(final ServerName sn); Clear Drain From RegionServer: - void removeDrainFromRegionServer(final ServerName sn); - void removeDrainFromAllRegionServers(); > Put draining function through Admin API > --- > > Key: HBASE-16010 > URL: https://issues.apache.org/jira/browse/HBASE-16010 > Project: HBase > Issue Type: Improvement >Reporter: Jerry He >Assignee: Matt Warhaftig >Priority: Minor > > Currently, there is no Amdin API for draining function. Client has to > interact directly with Zookeeper draining node to add and remove draining > servers. > For example, in draining_servers.rb: > {code} > zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config, > "draining_servers", nil) > parentZnode = zkw.drainingZNode > begin > for server in servers > node = ZKUtil.joinZNode(parentZnode, server) > ZKUtil.createAndFailSilent(zkw, node) > end > ensure > zkw.close() > end > {code} > This is not good in cases like secure clusters with protected Zookeeper nodes. > Let's put draining function through Admin API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)