[jira] [Updated] (HBASE-8710) The balancer shouldn't try balancing one node
[
https://issues.apache.org/jira/browse/HBASE-8710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Kinley updated HBASE-8710:
Attachment: HBASE-8710-3.patch
> The balancer shouldn't try balancing one node
> -
>
> Key: HBASE-8710
> URL: https://issues.apache.org/jira/browse/HBASE-8710
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.1
>Reporter: Jean-Daniel Cryans
>Assignee: James Kinley
>Priority: Minor
> Fix For: 0.96.0
>
> Attachments: HBASE-8710-1.patch, HBASE-8710-2.patch,
> HBASE-8710-3.patch
>
>
> In my logs, testing 0.95.1 RC1, I see:
> {noformat}
> 2013-06-07 17:31:47,377 DEBUG
> [ip-10-20-46-44.novalocal,48569,1370640098134-BalancerChore]
> balancer.StochasticLoadBalancer: Could not find a better load balance plan.
> Tried 3200 different configurations in 27ms, and did not find anything with a
> computed cost less than 25.0
> {noformat}
> Ideally we'd not even try one configuration, let alone 3.2k.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-8710) The balancer shouldn't try balancing one node
[
https://issues.apache.org/jira/browse/HBASE-8710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13781572#comment-13781572
]
James Kinley commented on HBASE-8710:
-
needsBalance() uses an instance of ClusterLoadState to determine how many
servers are active in the cluster. By default,
"hbase.master.loadbalance.bytable" is false and therefore ClusterLoadState has
a global view of the cluster and how many servers there are. If
"hbase.master.loadbalance.bytable" was set to true then ClusterLoadState will
only have a per table view of the cluster. In this case, if the logic was moved
to needsBalance() then the balancer may or may not run for each table, it will
definitely not run when the cluster has a single server, but needsBalance()
will be called for each table. Is this what you would expect?
Is there any reason why ClusterLoadState and ClusterStatus cannot be merged? At
the moment, a new instance of both is created every time the balancer is run
and ClusterStatus is only used by StochasticLoadBalancer. Wouldn't it be better
to have a single ClusterStatus object that also contains load information, and
to call balancer.needsBalance() from HMaster#balance() before
balancer.balanceCluster()?
The two other checks in HMaster#balance() could also be moved into
balancer.needsBalance() - isRegionsInTransition() and
areDeadServersInProgress().
> The balancer shouldn't try balancing one node
> -
>
> Key: HBASE-8710
> URL: https://issues.apache.org/jira/browse/HBASE-8710
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.1
>Reporter: Jean-Daniel Cryans
>Assignee: James Kinley
>Priority: Minor
> Fix For: 0.96.0
>
> Attachments: HBASE-8710-1.patch, HBASE-8710-2.patch
>
>
> In my logs, testing 0.95.1 RC1, I see:
> {noformat}
> 2013-06-07 17:31:47,377 DEBUG
> [ip-10-20-46-44.novalocal,48569,1370640098134-BalancerChore]
> balancer.StochasticLoadBalancer: Could not find a better load balance plan.
> Tried 3200 different configurations in 27ms, and did not find anything with a
> computed cost less than 25.0
> {noformat}
> Ideally we'd not even try one configuration, let alone 3.2k.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-8710) The balancer shouldn't try balancing one node
[
https://issues.apache.org/jira/browse/HBASE-8710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Kinley updated HBASE-8710:
Attachment: HBASE-8710-1.patch
> The balancer shouldn't try balancing one node
> -
>
> Key: HBASE-8710
> URL: https://issues.apache.org/jira/browse/HBASE-8710
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.1
>Reporter: Jean-Daniel Cryans
>Assignee: James Kinley
>Priority: Minor
> Fix For: 0.96.0
>
> Attachments: HBASE-8710-1.patch
>
>
> In my logs, testing 0.95.1 RC1, I see:
> {noformat}
> 2013-06-07 17:31:47,377 DEBUG
> [ip-10-20-46-44.novalocal,48569,1370640098134-BalancerChore]
> balancer.StochasticLoadBalancer: Could not find a better load balance plan.
> Tried 3200 different configurations in 27ms, and did not find anything with a
> computed cost less than 25.0
> {noformat}
> Ideally we'd not even try one configuration, let alone 3.2k.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-8710) The balancer shouldn't try balancing one node
[
https://issues.apache.org/jira/browse/HBASE-8710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Kinley reassigned HBASE-8710:
---
Assignee: James Kinley
> The balancer shouldn't try balancing one node
> -
>
> Key: HBASE-8710
> URL: https://issues.apache.org/jira/browse/HBASE-8710
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.1
>Reporter: Jean-Daniel Cryans
>Assignee: James Kinley
>Priority: Minor
> Fix For: 0.96.0
>
>
> In my logs, testing 0.95.1 RC1, I see:
> {noformat}
> 2013-06-07 17:31:47,377 DEBUG
> [ip-10-20-46-44.novalocal,48569,1370640098134-BalancerChore]
> balancer.StochasticLoadBalancer: Could not find a better load balance plan.
> Tried 3200 different configurations in 27ms, and did not find anything with a
> computed cost less than 25.0
> {noformat}
> Ideally we'd not even try one configuration, let alone 3.2k.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9656) Remove decimal places from Requests Per Second stats
[ https://issues.apache.org/jira/browse/HBASE-9656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Kinley updated HBASE-9656: Attachment: HBASE-9656-1.patch > Remove decimal places from Requests Per Second stats > > > Key: HBASE-9656 > URL: https://issues.apache.org/jira/browse/HBASE-9656 > Project: HBase > Issue Type: Improvement > Components: UI >Affects Versions: 0.95.2 >Reporter: James Kinley >Assignee: James Kinley >Priority: Trivial > Attachments: HBASE-9656-1.patch > > > The Requests Per Second stats on the Master and RegionServer UI pages would > look better without decimal places. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9656) Remove decimal places from Requests Per Second stats
[ https://issues.apache.org/jira/browse/HBASE-9656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Kinley updated HBASE-9656: Status: Patch Available (was: Open) > Remove decimal places from Requests Per Second stats > > > Key: HBASE-9656 > URL: https://issues.apache.org/jira/browse/HBASE-9656 > Project: HBase > Issue Type: Improvement > Components: UI >Affects Versions: 0.95.2 >Reporter: James Kinley >Assignee: James Kinley >Priority: Trivial > Attachments: HBASE-9656-1.patch > > > The Requests Per Second stats on the Master and RegionServer UI pages would > look better without decimal places. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-9656) Remove decimal places from Requests Per Second stats
James Kinley created HBASE-9656: --- Summary: Remove decimal places from Requests Per Second stats Key: HBASE-9656 URL: https://issues.apache.org/jira/browse/HBASE-9656 Project: HBase Issue Type: Improvement Components: UI Affects Versions: 0.95.2 Reporter: James Kinley Assignee: James Kinley Priority: Trivial The Requests Per Second stats on the Master and RegionServer UI pages would look better without decimal places. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8711) Requests count is completely off
[ https://issues.apache.org/jira/browse/HBASE-8711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Kinley updated HBASE-8711: Attachment: HBASE-8711-1.patch Attached patch to make regionserver metric period configurable, and to lower the default period to 5 seconds. > Requests count is completely off > > > Key: HBASE-8711 > URL: https://issues.apache.org/jira/browse/HBASE-8711 > Project: HBase > Issue Type: Bug > Components: UI >Affects Versions: 0.95.1 >Reporter: Jean-Daniel Cryans >Assignee: James Kinley > Fix For: 0.96.0 > > Attachments: HBASE-8711-1.patch, RPS-TEST-1.log, RPS-TEST-2.log > > > I tried 0.95.1 RC1 in standalone, and the requests count in both the master > and RS web UIs are wrong. I haven't dug too much in but it seems too low when > I'm sending load, and it takes >10 seconds to clear up when the cluster > becomes completely idle. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8711) Requests count is completely off
[ https://issues.apache.org/jira/browse/HBASE-8711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Kinley updated HBASE-8711: Attachment: RPS-TEST-1.log RPS-TEST-2.log > Requests count is completely off > > > Key: HBASE-8711 > URL: https://issues.apache.org/jira/browse/HBASE-8711 > Project: HBase > Issue Type: Bug > Components: UI >Affects Versions: 0.95.1 >Reporter: Jean-Daniel Cryans >Assignee: James Kinley > Fix For: 0.96.0 > > Attachments: RPS-TEST-1.log, RPS-TEST-2.log > > > I tried 0.95.1 RC1 in standalone, and the requests count in both the master > and RS web UIs are wrong. I haven't dug too much in but it seems too low when > I'm sending load, and it takes >10 seconds to clear up when the cluster > becomes completely idle. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8711) Requests count is completely off
[
https://issues.apache.org/jira/browse/HBASE-8711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13777370#comment-13777370
]
James Kinley commented on HBASE-8711:
-
There are a couple of things going on here. The first issue is that the point
in time metrics for a region server are only computed every 15 seconds and
therefore requestsPerSecond is actually an average for the previous interval:
{code}
requestsPerSecond = (numberRequestsSinceLastRun / secondsSinceLastRun)
{code}
The client write buffer also confuses things because the writes are delayed
reaching the server, then the server can take up to 15 seconds to include them
in the metrics, and another 15 seconds to clear them down.
The second issue is that the requestCount counter used to compute
requestsPerSecond is incremented for every region server operation (i.e.
including requests for region info and not just for mutations) and is therefore
higher than expected.
I've attached the results of two tests to show whats happening. The same client
was used in both tests to generate 100 req/sec for 60 seconds (6000 writes):
In the first test (RPS-TEST-1.log) the client write buffer was enabled and you
can see that it took 52 seconds before the writes appeared in the metrics, and
because they were all sent at once requestsPerSecond was skewed at 09:07:28
(requestsPerSecond: 602.7862951606452). It took a further 30 seconds before
requestsPerSecond returned to 0. Also, writeRequestsCount was only incremented
once for each flush, and the individual writes were not counted
(tempWriteRequestsCount = 4).
In the second test (RPS-TEST-2.log) the client write buffer was disabled and
you can see that the writes started to appear in the metrics after the next
metric computation was run (15 seconds), and requestsPerSecond hovered at ~170
for the duration of the test. The writes were also counted individually
(tempWriteRequestsCount = 6004).
Both tests show an increase of over 12000 requests for 6000 writes.
Computing requestsPerSecond from the requestCount counter makes sense, but the
interval should at least be configurable and be lower by default (maybe 5
seconds).
> Requests count is completely off
>
>
> Key: HBASE-8711
> URL: https://issues.apache.org/jira/browse/HBASE-8711
> Project: HBase
> Issue Type: Bug
> Components: UI
>Affects Versions: 0.95.1
>Reporter: Jean-Daniel Cryans
>Assignee: James Kinley
> Fix For: 0.96.0
>
>
> I tried 0.95.1 RC1 in standalone, and the requests count in both the master
> and RS web UIs are wrong. I haven't dug too much in but it seems too low when
> I'm sending load, and it takes >10 seconds to clear up when the cluster
> becomes completely idle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6973) Trim trailing whitespace from configuration values
[
https://issues.apache.org/jira/browse/HBASE-6973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13743035#comment-13743035
]
James Kinley commented on HBASE-6973:
-
Can this be closed now that HADOOP-9855 has been resolved?
> Trim trailing whitespace from configuration values
> --
>
> Key: HBASE-6973
> URL: https://issues.apache.org/jira/browse/HBASE-6973
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.1
>Reporter: James Kinley
>Priority: Minor
>
> If the following properties have a whitespace at the end of the value it
> causes problems with Kerberos authentication...
> {code}
>
> hbase.regionserver.keytab.file
> /etc/hbase/conf/hbase.krb5
>
>
> hbase.master.keytab.file
> /etc/hbase/conf/hbase.krb5
>
> {code}
> I'm guessing this can affect other properties as well, so can the values be
> trimmed when read?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8711) Requests count is completely off
[ https://issues.apache.org/jira/browse/HBASE-8711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13736812#comment-13736812 ] James Kinley commented on HBASE-8711: - Hey Elliott. Are you working on this at the moment? If not, I was going to take a look if thats ok with you? > Requests count is completely off > > > Key: HBASE-8711 > URL: https://issues.apache.org/jira/browse/HBASE-8711 > Project: HBase > Issue Type: Bug > Components: UI >Affects Versions: 0.95.1 >Reporter: Jean-Daniel Cryans >Assignee: Elliott Clark > Fix For: 0.95.2 > > > I tried 0.95.1 RC1 in standalone, and the requests count in both the master > and RS web UIs are wrong. I haven't dug too much in but it seems too low when > I'm sending load, and it takes >10 seconds to clear up when the cluster > becomes completely idle. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6973) Trim trailing whitespace from configuration values
[
https://issues.apache.org/jira/browse/HBASE-6973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13734593#comment-13734593
]
James Kinley commented on HBASE-6973:
-
bq. In any case, I think Hadoop should have the backport done on branch-1 so
that HBase does not have to take care of this.
I agree, I think this should be fixed in Hadoop branch-1 too. I've opened
HADOOP-9855 for the backport.
> Trim trailing whitespace from configuration values
> --
>
> Key: HBASE-6973
> URL: https://issues.apache.org/jira/browse/HBASE-6973
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.1
>Reporter: James Kinley
>Priority: Minor
>
> If the following properties have a whitespace at the end of the value it
> causes problems with Kerberos authentication...
> {code}
>
> hbase.regionserver.keytab.file
> /etc/hbase/conf/hbase.krb5
>
>
> hbase.master.keytab.file
> /etc/hbase/conf/hbase.krb5
>
> {code}
> I'm guessing this can affect other properties as well, so can the values be
> trimmed when read?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-7482) Port HBASE-7442 HBase remote CopyTable not working when security enabled to trunk
[
https://issues.apache.org/jira/browse/HBASE-7482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Kinley reassigned HBASE-7482:
---
Assignee: Gary Helmling (was: James Kinley)
> Port HBASE-7442 HBase remote CopyTable not working when security enabled to
> trunk
> -
>
> Key: HBASE-7482
> URL: https://issues.apache.org/jira/browse/HBASE-7482
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Gary Helmling
>Priority: Critical
> Fix For: 0.95.0
>
> Attachments: HBASE-7482-trunk.patch, HBASE-7482-v2.patch
>
>
> Excerpt about the choice of solution from :
> The first option was actually quite messy to implement. {{clusterId}} and
> {{conf}} are fixed in *{{HBaseClient}}* when it's created and cached by
> *{{SecureRpcEngine}}*, so to implement the fix here I would have had to pass
> the different cluster {{confs}} up through *{{HConnectionManager}}* and
> *{{HBaseRPC}}* in order to override the clusterId in
> *{{SecureClient#SecureConnection}}*.
> I've gone with the second option of creating and caching different
> *{{SecureClients}}* for the local and remote clusters in
> *{{SecureRpcEngine}}* - keyed off of the {{clusterId}} instead of the default
> *{{SocketFactory}}*. I think this is a cleaner solution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7482) Port HBASE-7442 HBase remote CopyTable not working when security enabled to trunk
[
https://issues.apache.org/jira/browse/HBASE-7482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13595772#comment-13595772
]
James Kinley commented on HBASE-7482:
-
+1
It's better not having to muck around with the configuration. V2 patch looks
good and copytable works on my secure clusters.
> Port HBASE-7442 HBase remote CopyTable not working when security enabled to
> trunk
> -
>
> Key: HBASE-7482
> URL: https://issues.apache.org/jira/browse/HBASE-7482
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: James Kinley
>Priority: Critical
> Fix For: 0.95.0
>
> Attachments: HBASE-7482-trunk.patch, HBASE-7482-v2.patch
>
>
> Excerpt about the choice of solution from :
> The first option was actually quite messy to implement. {{clusterId}} and
> {{conf}} are fixed in *{{HBaseClient}}* when it's created and cached by
> *{{SecureRpcEngine}}*, so to implement the fix here I would have had to pass
> the different cluster {{confs}} up through *{{HConnectionManager}}* and
> *{{HBaseRPC}}* in order to override the clusterId in
> *{{SecureClient#SecureConnection}}*.
> I've gone with the second option of creating and caching different
> *{{SecureClients}}* for the local and remote clusters in
> *{{SecureRpcEngine}}* - keyed off of the {{clusterId}} instead of the default
> *{{SocketFactory}}*. I think this is a cleaner solution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7482) Port HBASE-7442 HBase remote CopyTable not working when security enabled to trunk
[
https://issues.apache.org/jira/browse/HBASE-7482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Kinley updated HBASE-7482:
Attachment: HBASE-7482-trunk.patch
> Port HBASE-7442 HBase remote CopyTable not working when security enabled to
> trunk
> -
>
> Key: HBASE-7482
> URL: https://issues.apache.org/jira/browse/HBASE-7482
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: James Kinley
>Priority: Critical
> Fix For: 0.95.0
>
> Attachments: HBASE-7482-trunk.patch
>
>
> Excerpt about the choice of solution from :
> The first option was actually quite messy to implement. {{clusterId}} and
> {{conf}} are fixed in *{{HBaseClient}}* when it's created and cached by
> *{{SecureRpcEngine}}*, so to implement the fix here I would have had to pass
> the different cluster {{confs}} up through *{{HConnectionManager}}* and
> *{{HBaseRPC}}* in order to override the clusterId in
> *{{SecureClient#SecureConnection}}*.
> I've gone with the second option of creating and caching different
> *{{SecureClients}}* for the local and remote clusters in
> *{{SecureRpcEngine}}* - keyed off of the {{clusterId}} instead of the default
> *{{SocketFactory}}*. I think this is a cleaner solution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7482) Port HBASE-7442 HBase remote CopyTable not working when security enabled to trunk
[
https://issues.apache.org/jira/browse/HBASE-7482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Kinley updated HBASE-7482:
Attachment: (was: HBASE-7482-trunk.patch)
> Port HBASE-7442 HBase remote CopyTable not working when security enabled to
> trunk
> -
>
> Key: HBASE-7482
> URL: https://issues.apache.org/jira/browse/HBASE-7482
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: James Kinley
>Priority: Critical
> Fix For: 0.95.0
>
>
> Excerpt about the choice of solution from :
> The first option was actually quite messy to implement. {{clusterId}} and
> {{conf}} are fixed in *{{HBaseClient}}* when it's created and cached by
> *{{SecureRpcEngine}}*, so to implement the fix here I would have had to pass
> the different cluster {{confs}} up through *{{HConnectionManager}}* and
> *{{HBaseRPC}}* in order to override the clusterId in
> *{{SecureClient#SecureConnection}}*.
> I've gone with the second option of creating and caching different
> *{{SecureClients}}* for the local and remote clusters in
> *{{SecureRpcEngine}}* - keyed off of the {{clusterId}} instead of the default
> *{{SocketFactory}}*. I think this is a cleaner solution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7482) Port HBASE-7442 HBase remote CopyTable not working when security enabled to trunk
[
https://issues.apache.org/jira/browse/HBASE-7482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Kinley updated HBASE-7482:
Status: Patch Available (was: Open)
> Port HBASE-7442 HBase remote CopyTable not working when security enabled to
> trunk
> -
>
> Key: HBASE-7482
> URL: https://issues.apache.org/jira/browse/HBASE-7482
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: James Kinley
>Priority: Critical
> Fix For: 0.95.0
>
> Attachments: HBASE-7482-trunk.patch
>
>
> Excerpt about the choice of solution from :
> The first option was actually quite messy to implement. {{clusterId}} and
> {{conf}} are fixed in *{{HBaseClient}}* when it's created and cached by
> *{{SecureRpcEngine}}*, so to implement the fix here I would have had to pass
> the different cluster {{confs}} up through *{{HConnectionManager}}* and
> *{{HBaseRPC}}* in order to override the clusterId in
> *{{SecureClient#SecureConnection}}*.
> I've gone with the second option of creating and caching different
> *{{SecureClients}}* for the local and remote clusters in
> *{{SecureRpcEngine}}* - keyed off of the {{clusterId}} instead of the default
> *{{SocketFactory}}*. I think this is a cleaner solution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7482) Port HBASE-7442 HBase remote CopyTable not working when security enabled to trunk
[
https://issues.apache.org/jira/browse/HBASE-7482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13592500#comment-13592500
]
James Kinley commented on HBASE-7482:
-
In addition to the TableMapReduceUtil change, 'hbase.cluster.id' has to be
reset in HConnectionImplementation to ensure that it's HBaseClient instance has
the correct 'clusterId' set and therefore the correct token is selected when
setting up the Connection.
> Port HBASE-7442 HBase remote CopyTable not working when security enabled to
> trunk
> -
>
> Key: HBASE-7482
> URL: https://issues.apache.org/jira/browse/HBASE-7482
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: James Kinley
>Priority: Critical
> Fix For: 0.95.0
>
> Attachments: HBASE-7482-trunk.patch
>
>
> Excerpt about the choice of solution from :
> The first option was actually quite messy to implement. {{clusterId}} and
> {{conf}} are fixed in *{{HBaseClient}}* when it's created and cached by
> *{{SecureRpcEngine}}*, so to implement the fix here I would have had to pass
> the different cluster {{confs}} up through *{{HConnectionManager}}* and
> *{{HBaseRPC}}* in order to override the clusterId in
> *{{SecureClient#SecureConnection}}*.
> I've gone with the second option of creating and caching different
> *{{SecureClients}}* for the local and remote clusters in
> *{{SecureRpcEngine}}* - keyed off of the {{clusterId}} instead of the default
> *{{SocketFactory}}*. I think this is a cleaner solution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7482) Port HBASE-7442 HBase remote CopyTable not working when security enabled to trunk
[
https://issues.apache.org/jira/browse/HBASE-7482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Kinley updated HBASE-7482:
Attachment: HBASE-7482-trunk.patch
> Port HBASE-7442 HBase remote CopyTable not working when security enabled to
> trunk
> -
>
> Key: HBASE-7482
> URL: https://issues.apache.org/jira/browse/HBASE-7482
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: James Kinley
>Priority: Critical
> Fix For: 0.95.0
>
> Attachments: HBASE-7482-trunk.patch
>
>
> Excerpt about the choice of solution from :
> The first option was actually quite messy to implement. {{clusterId}} and
> {{conf}} are fixed in *{{HBaseClient}}* when it's created and cached by
> *{{SecureRpcEngine}}*, so to implement the fix here I would have had to pass
> the different cluster {{confs}} up through *{{HConnectionManager}}* and
> *{{HBaseRPC}}* in order to override the clusterId in
> *{{SecureClient#SecureConnection}}*.
> I've gone with the second option of creating and caching different
> *{{SecureClients}}* for the local and remote clusters in
> *{{SecureRpcEngine}}* - keyed off of the {{clusterId}} instead of the default
> *{{SocketFactory}}*. I think this is a cleaner solution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7482) Port HBASE-7442 HBase remote CopyTable not working when security enabled to trunk
[
https://issues.apache.org/jira/browse/HBASE-7482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13592401#comment-13592401
]
James Kinley commented on HBASE-7482:
-
Hi Gary. Yes, I have 2 secure clusters which I setup for HBASE-7442. I've been
testing a patch this afternoon and had to make an additional change to
HConnectionImplementation to get it to work. I'll attach it for review.
> Port HBASE-7442 HBase remote CopyTable not working when security enabled to
> trunk
> -
>
> Key: HBASE-7482
> URL: https://issues.apache.org/jira/browse/HBASE-7482
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: James Kinley
>Priority: Critical
> Fix For: 0.95.0
>
>
> Excerpt about the choice of solution from :
> The first option was actually quite messy to implement. {{clusterId}} and
> {{conf}} are fixed in *{{HBaseClient}}* when it's created and cached by
> *{{SecureRpcEngine}}*, so to implement the fix here I would have had to pass
> the different cluster {{confs}} up through *{{HConnectionManager}}* and
> *{{HBaseRPC}}* in order to override the clusterId in
> *{{SecureClient#SecureConnection}}*.
> I've gone with the second option of creating and caching different
> *{{SecureClients}}* for the local and remote clusters in
> *{{SecureRpcEngine}}* - keyed off of the {{clusterId}} instead of the default
> *{{SocketFactory}}*. I think this is a cleaner solution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-7482) Port HBASE-7442 HBase remote CopyTable not working when security enabled to trunk
[
https://issues.apache.org/jira/browse/HBASE-7482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Kinley reassigned HBASE-7482:
---
Assignee: James Kinley (was: Gary Helmling)
> Port HBASE-7442 HBase remote CopyTable not working when security enabled to
> trunk
> -
>
> Key: HBASE-7482
> URL: https://issues.apache.org/jira/browse/HBASE-7482
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: James Kinley
>Priority: Critical
> Fix For: 0.95.0
>
>
> Excerpt about the choice of solution from :
> The first option was actually quite messy to implement. {{clusterId}} and
> {{conf}} are fixed in *{{HBaseClient}}* when it's created and cached by
> *{{SecureRpcEngine}}*, so to implement the fix here I would have had to pass
> the different cluster {{confs}} up through *{{HConnectionManager}}* and
> *{{HBaseRPC}}* in order to override the clusterId in
> *{{SecureClient#SecureConnection}}*.
> I've gone with the second option of creating and caching different
> *{{SecureClients}}* for the local and remote clusters in
> *{{SecureRpcEngine}}* - keyed off of the {{clusterId}} instead of the default
> *{{SocketFactory}}*. I think this is a cleaner solution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6626) Add a chapter on HDFS in the troubleshooting section of the HBase reference guide.
[ https://issues.apache.org/jira/browse/HBASE-6626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13580500#comment-13580500 ] James Kinley commented on HBASE-6626: - This is really useful. A couple of suggestions: it might be easier to digest if you describe the various HDFS communication points along the lines of the write path. For example: * Client issues Put to a region server, which writes the edit to the WAL, which appends to HLog in HDFS ** This uses DFSClient ** HLog writes are pipelined to X number of datanodes - based on replication factor ** HLogs are rolled at intervals (60 minutes), which requires interaction with HDFS ** Indicate which timeout and retry options apply here ** ... * Same for HFiles... In addition to this, and the typical error messages, it would also be great to describe what to look out for in certain scenarios. For example: * What happens when a datanode goes bad (you've touched on this already) ** How does this affect a region server? ** What to look out for in the logs? ** What configuration options apply? * What happens when the active namenode goes bad, and what happens during namenode failover? ** A region server needs to communicate with the namenode when it rolls it's WAL, flushes it's store files, etc ** In between, it only communicates with the datanodes to append to the WAL ** What configuration options apply? ** What to look out for in the logs? Thanks, James. > Add a chapter on HDFS in the troubleshooting section of the HBase reference > guide. > -- > > Key: HBASE-6626 > URL: https://issues.apache.org/jira/browse/HBASE-6626 > Project: HBase > Issue Type: Improvement > Components: documentation >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: Doug Meil >Priority: Blocker > Attachments: troubleshooting.txt > > > I looked mainly at the major failure case, but here is what I have: > New sub chapter in the existing chapter "Troubleshooting and Debugging > HBase": "HDFS & HBASE" > 1) HDFS & HBase > 2) Connection related settings > 2.1) Number of retries > 2.2) Timeouts > 3) Log samples > 1) HDFS & HBase > HBase uses HDFS to store its HFile, i.e. the core HBase files and the > Write-Ahead-Logs, i.e. the files that will be used to restore the data after > a crash. > In both cases, the reliability of HBase comes from the fact that HDFS writes > the data to multiple locations. To be efficient, HBase needs the data to be > available locally, hence it's highly recommended to have the HDFS datanode on > the same machines as the HBase Region Servers. > Detailed information on how HDFS works can be found at [1]. > Important features are: > - HBase is a client application of HDFS, i.e. uses the HDFS DFSClient class. > This class can appears in HBase logs with other HDFS client related logs. > - Some HDFS settings are HDFS-server-side, i.e. must be set on the HDFS > side, while some other are HDFS-client-side, i.e. must be set in HBase, while > some other must be set in both places. > - the HDFS writes are pipelined from one datanode to another. When writing, > there are communications between: > - HBase and HDFS namenode, through the HDFS client classes. > - HBase and HDFS datanodes, through the HDFS client classes. > - HDFS datanode between themselves: issues on these communications are in > HDFS logs, not HBase. HDFS writes are always local when possible. As a > consequence, there should not be much write error in HBase Region Servers: > they write to the local datanode. If this datanode can't replicate the > blocks, it will appear in its logs, not in the region servers logs. > - datanodes can be contacted through the ipc.Client interface (once again > this class can shows up in HBase logs) and the data transfer interface > (usually shows up as the DataNode class in the HBase logs). There are on > different ports (defaults being: 50010 and 50020). > - To understand exactly what's going on, you must look that the HDFS log > files as well: HBase logs represent the client side. > - With the default setting, HDFS needs 630s to mark a datanode as dead. For > this reason, this node will still be tried by HBase or by other datanodes > when writing and reading until HDFS definitively decides it's dead. This will > add some extras lines in the logs. This monitoring is performed by the > NameNode. > - The HDFS clients (i.e. HBase using HDFS client code) don't fully rely on > the NameNode, but can mark temporally a node as dead if they had an error > when they tried to use it. > 2) Settings for retries and timeouts > 2.1) Retries > ipc.client.connect.max.retries > Default 10 > Indicates the number of retries a client will make to establish
[jira] [Updated] (HBASE-7442) HBase remote CopyTable not working when security enabled
[
https://issues.apache.org/jira/browse/HBASE-7442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Kinley updated HBASE-7442:
Attachment: HBASE-7442-0.94.patch
It works :)
You're correct, the 0.94 patch is identical (attached). Trunk is different so
this will take me a little longer.
> HBase remote CopyTable not working when security enabled
>
>
> Key: HBASE-7442
> URL: https://issues.apache.org/jira/browse/HBASE-7442
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC, mapreduce, security
>Affects Versions: 0.92.1
>Reporter: James Kinley
> Fix For: 0.92.3, 0.96.0, 0.94.5
>
> Attachments: attempt_201212271546_0001_m_00_0.log,
> HBASE-7442-0.92.1.patch, HBASE-7442-0.94.patch
>
>
> When security is enabled, HBase CopyTable fails with Kerberos exception:
> {code}
> FATAL org.apache.hadoop.ipc.SecureClient: SASL authentication failed. The
> most likely cause is missing or invalid credentials. Consider 'kinit'.
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> {code}
> This is only when copying to remote HBase cluster (using either MRv1 or
> YARN), local copy works fine.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7442) HBase remote CopyTable not working when security enabled
[
https://issues.apache.org/jira/browse/HBASE-7442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13540618#comment-13540618
]
James Kinley commented on HBASE-7442:
-
Agreed, this would be a short term fix and more refactoring work is required to
get it right. I agree with Ted that this work warrants its own JIRA.
> HBase remote CopyTable not working when security enabled
>
>
> Key: HBASE-7442
> URL: https://issues.apache.org/jira/browse/HBASE-7442
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC, mapreduce, security
>Affects Versions: 0.92.1
>Reporter: James Kinley
> Fix For: 0.96.0, 0.94.5
>
> Attachments: attempt_201212271546_0001_m_00_0.log,
> HBASE-7442-0.92.1.patch
>
>
> When security is enabled, HBase CopyTable fails with Kerberos exception:
> {code}
> FATAL org.apache.hadoop.ipc.SecureClient: SASL authentication failed. The
> most likely cause is missing or invalid credentials. Consider 'kinit'.
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> {code}
> This is only when copying to remote HBase cluster (using either MRv1 or
> YARN), local copy works fine.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7442) HBase remote CopyTable not working when security enabled
[
https://issues.apache.org/jira/browse/HBASE-7442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13540472#comment-13540472
]
James Kinley commented on HBASE-7442:
-
Please see attached patch.
The first option was actually quite messy to implement. {{clusterId}} and
{{conf}} are fixed in *{{HBaseClient}}* when it's created and cached by
*{{SecureRpcEngine}}*, so to implement the fix here I would have had to pass
the different cluster {{confs}} up through *{{HConnectionManager}}* and
*{{HBaseRPC}}* in order to override the clusterId in
*{{SecureClient#SecureConnection}}*.
I've gone with the second option of creating and caching different
*{{SecureClients}}* for the local and remote clusters in *{{SecureRpcEngine}}*
- keyed off of the {{clusterId}} instead of the default *{{SocketFactory}}*. I
think this is a cleaner solution.
Please let me know what you think.
> HBase remote CopyTable not working when security enabled
>
>
> Key: HBASE-7442
> URL: https://issues.apache.org/jira/browse/HBASE-7442
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC, mapreduce, security
>Affects Versions: 0.92.1
>Reporter: James Kinley
> Fix For: 0.96.0, 0.94.5
>
> Attachments: attempt_201212271546_0001_m_00_0.log,
> HBASE-7442-0.92.1.patch
>
>
> When security is enabled, HBase CopyTable fails with Kerberos exception:
> {code}
> FATAL org.apache.hadoop.ipc.SecureClient: SASL authentication failed. The
> most likely cause is missing or invalid credentials. Consider 'kinit'.
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> {code}
> This is only when copying to remote HBase cluster (using either MRv1 or
> YARN), local copy works fine.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7442) HBase remote CopyTable not working when security enabled
[
https://issues.apache.org/jira/browse/HBASE-7442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Kinley updated HBASE-7442:
Attachment: HBASE-7442-0.92.1.patch
> HBase remote CopyTable not working when security enabled
>
>
> Key: HBASE-7442
> URL: https://issues.apache.org/jira/browse/HBASE-7442
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC, mapreduce, security
>Affects Versions: 0.92.1
>Reporter: James Kinley
> Fix For: 0.96.0, 0.94.5
>
> Attachments: attempt_201212271546_0001_m_00_0.log,
> HBASE-7442-0.92.1.patch
>
>
> When security is enabled, HBase CopyTable fails with Kerberos exception:
> {code}
> FATAL org.apache.hadoop.ipc.SecureClient: SASL authentication failed. The
> most likely cause is missing or invalid credentials. Consider 'kinit'.
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> {code}
> This is only when copying to remote HBase cluster (using either MRv1 or
> YARN), local copy works fine.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7442) HBase remote CopyTable not working when security enabled
[
https://issues.apache.org/jira/browse/HBASE-7442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13540031#comment-13540031
]
James Kinley commented on HBASE-7442:
-
I noticed that *{{CopyTable#createSubmittableJob}}*, and specifically
*{{TableMapReduceUtil#initCredentials}}* was only acquiring a delegation token
for the local HBase cluster so I added the following to
*{{TableMapReduceUtil#initCredentials}}*:
{code}
String quorumAddress =
job.getConfiguration().get(TableOutputFormat.QUORUM_ADDRESS);
if (quorumAddress != null) {
String[] parts = ZKUtil.transformClusterKey(quorumAddress);
Configuration peerConf = HBaseConfiguration.create(job.getConfiguration());
peerConf.set(HConstants.ZOOKEEPER_QUORUM, parts[0]);
peerConf.set("hbase.zookeeper.client.port", parts[1]);
peerConf.set(HConstants.ZOOKEEPER_ZNODE_PARENT, parts[2]);
User.getCurrent().obtainAuthTokenForJob(peerConf, job);
}
{code}
The user now has the correct tokens:
Cluster 1 ID = 7bb01c84-3c66-49ca-82d2-2f9218f13dad
Cluster 2 ID = d4653a37-83df-4929-b950-e03987c3b4e8
{code}
INFO token.TokenUtil: Obtained token HBASE_AUTH_TOKEN for user
kinley/ad...@kinley.com on cluster 7bb01c84-3c66-49ca-82d2-2f9218f13dad
INFO token.TokenUtil: Obtained token HBASE_AUTH_TOKEN for user
kinley/ad...@kinley.com on cluster d4653a37-83df-4929-b950-e03987c3b4e8
{code}
But when initialising the *{{HTable}}* connection in the mapper, the
*{{TokenSelector}}* in *{{SecureClient#SecureConnection}}* returns the wrong
token from the user's token cache. This is because the token cache is keyed on
{{clusterId}}, and despite the token for the local cluster existing in the
cache ({{key=7bb01c84-3c66-49ca-82d2-2f9218f13dad}}), the {{clusterId}} is
fixed at this point in *{{HBaseClient}}* to the remote cluster's ID
({{key=d4653a37-83df-4929-b950-e03987c3b4e8}}), and it is this ID that is
always passed to the *{{TokenSelector}}*.
I can think of a couple of possible solutions:
1. Either pass the {{clusterId}} into *{{SecureClient#SecureConnection}}* as
part of the *{{ConnectionId}}*. This way it can be used by the
*{{TokenSelector}}* instead of the fixed ID in the parent class
*{{HBaseClient}}*?, or
2. *{{SecureRpcEngine}}* caches multiple clients based on a
*{{SocketFactory}}*, so we could maintain separate clients for both the local
and remote clusters here?
> HBase remote CopyTable not working when security enabled
>
>
> Key: HBASE-7442
> URL: https://issues.apache.org/jira/browse/HBASE-7442
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC, mapreduce, security
>Affects Versions: 0.92.1
>Reporter: James Kinley
> Attachments: attempt_201212271546_0001_m_00_0.log
>
>
> When security is enabled, HBase CopyTable fails with Kerberos exception:
> {code}
> FATAL org.apache.hadoop.ipc.SecureClient: SASL authentication failed. The
> most likely cause is missing or invalid credentials. Consider 'kinit'.
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> {code}
> This is only when copying to remote HBase cluster (using either MRv1 or
> YARN), local copy works fine.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7442) HBase remote CopyTable not working when security enabled
[
https://issues.apache.org/jira/browse/HBASE-7442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Kinley updated HBASE-7442:
Attachment: attempt_201212271546_0001_m_00_0.log
> HBase remote CopyTable not working when security enabled
>
>
> Key: HBASE-7442
> URL: https://issues.apache.org/jira/browse/HBASE-7442
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC, mapreduce, security
>Affects Versions: 0.92.1
>Reporter: James Kinley
> Attachments: attempt_201212271546_0001_m_00_0.log
>
>
> When security is enabled, HBase CopyTable fails with Kerberos exception:
> {code}
> FATAL org.apache.hadoop.ipc.SecureClient: SASL authentication failed. The
> most likely cause is missing or invalid credentials. Consider 'kinit'.
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> {code}
> This is only when copying to remote HBase cluster (using either MRv1 or
> YARN), local copy works fine.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7442) HBase remote CopyTable not working when security enabled
[
https://issues.apache.org/jira/browse/HBASE-7442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13540007#comment-13540007
]
James Kinley commented on HBASE-7442:
-
To give some context for the attached log
({{attempt_201212271546_0001_m_00_0.log}}).
I have setup 2 clusters:
Cluster 1 has two nodes running the following services:
one.kinley.com: HDFS, MRv1
two.kinley.com: HBase, ZK
Cluster 2 has a single node running the following:
three.kinley.com: HDFS, HBase, ZK
Security is enabled on both clusters and all principals are in the same realm
(so cross-realm support is not required).
I'm trying to copy an HBase table from cluster 1 to cluster 2:
{code}
two.kinley.com:~$ hbase org.apache.hadoop.hbase.mapreduce.CopyTable
-Dmapred.child.java.opts="-Dsun.security.krb5.debug=true"
-Dmapred.map.child.log.level=DEBUG --peer.adr=three.kinley.com:2181:/hbase
--new.name=Converged-headers Converged-headers
{code}
Running a local2local CopyTable on cluster 1 and 2 works ok.
> HBase remote CopyTable not working when security enabled
>
>
> Key: HBASE-7442
> URL: https://issues.apache.org/jira/browse/HBASE-7442
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC, mapreduce, security
>Affects Versions: 0.92.1
>Reporter: James Kinley
>
> When security is enabled, HBase CopyTable fails with Kerberos exception:
> {code}
> FATAL org.apache.hadoop.ipc.SecureClient: SASL authentication failed. The
> most likely cause is missing or invalid credentials. Consider 'kinit'.
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> {code}
> This is only when copying to remote HBase cluster (using either MRv1 or
> YARN), local copy works fine.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7442) HBase remote CopyTable not working when security enabled
James Kinley created HBASE-7442:
---
Summary: HBase remote CopyTable not working when security enabled
Key: HBASE-7442
URL: https://issues.apache.org/jira/browse/HBASE-7442
Project: HBase
Issue Type: Bug
Components: IPC/RPC, mapreduce, security
Affects Versions: 0.92.1
Reporter: James Kinley
When security is enabled, HBase CopyTable fails with Kerberos exception:
{code}
FATAL org.apache.hadoop.ipc.SecureClient: SASL authentication failed. The most
likely cause is missing or invalid credentials. Consider 'kinit'.
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException:
No valid credentials provided (Mechanism level: Failed to find any Kerberos
tgt)]
{code}
This is only when copying to remote HBase cluster (using either MRv1 or YARN),
local copy works fine.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7059) Add Region Server ID to metrics
James Kinley created HBASE-7059: --- Summary: Add Region Server ID to metrics Key: HBASE-7059 URL: https://issues.apache.org/jira/browse/HBASE-7059 Project: HBase Issue Type: Improvement Components: metrics, regionserver Affects Versions: 0.92.1 Reporter: James Kinley When running multiple Region Servers on a single node monitoring tools are unable to distinguish between the metrics, because they group them by hostname. E.g. Ganglia shows accumulated metrics for "hbase.regionserver.blockCacheCount" for all Region Servers running on the same node. If a Region Server ID was added to the metric, e.g. "hbase.regionserver001.blockCacheCount", then Ganglia will be able to display the metrics for each Region Server separately. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6973) Trim whitespace from configuration values
[
https://issues.apache.org/jira/browse/HBASE-6973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Kinley updated HBASE-6973:
Description:
If the following properties have a whitespace at the end of the value it causes
problems with Kerberos authentication...
{code}
hbase.regionserver.keytab.file
/etc/hbase/conf/hbase.krb5
hbase.master.keytab.file
/etc/hbase/conf/hbase.krb5
{code}
I'm guessing this can affect other properties as well, so can the values be
trimmed when read?
was:
If the following properties have a whitespace at the end of the value it causes
problems with Kerberos authentication...
hbase.regionserver.keytab.file
/etc/hbase/conf/hbase.krb5
hbase.master.keytab.file
/etc/hbase/conf/hbase.krb5
I'm guessing this can affect other properties as well, so can the values be
trimmed when read?
> Trim whitespace from configuration values
> -
>
> Key: HBASE-6973
> URL: https://issues.apache.org/jira/browse/HBASE-6973
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.1
>Reporter: James Kinley
>Priority: Minor
>
> If the following properties have a whitespace at the end of the value it
> causes problems with Kerberos authentication...
> {code}
>
> hbase.regionserver.keytab.file
> /etc/hbase/conf/hbase.krb5
>
>
> hbase.master.keytab.file
> /etc/hbase/conf/hbase.krb5
>
> {code}
> I'm guessing this can affect other properties as well, so can the values be
> trimmed when read?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6973) Trim whitespace from configuration values
James Kinley created HBASE-6973: --- Summary: Trim whitespace from configuration values Key: HBASE-6973 URL: https://issues.apache.org/jira/browse/HBASE-6973 Project: HBase Issue Type: Bug Affects Versions: 0.92.1 Reporter: James Kinley Priority: Minor If the following properties have a whitespace at the end of the value it causes problems with Kerberos authentication... hbase.regionserver.keytab.file /etc/hbase/conf/hbase.krb5 hbase.master.keytab.file /etc/hbase/conf/hbase.krb5 I'm guessing this can affect other properties as well, so can the values be trimmed when read? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
