[jira] [Commented] (HBASE-5738) Using HBase with HA HDFS requires bogus hardcoded port value
[
https://issues.apache.org/jira/browse/HBASE-5738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13264135#comment-13264135
]
stack commented on HBASE-5738:
--
Shaneal, shall we close this (as per Eli's suggestion above)?
> Using HBase with HA HDFS requires bogus hardcoded port value
>
>
> Key: HBASE-5738
> URL: https://issues.apache.org/jira/browse/HBASE-5738
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 0.92.1, 0.94.0
>Reporter: Shaneal Manek
>Assignee: Shaneal Manek
> Attachments: HBASE-5738.patch
>
>
> When configuring HBase with HDFS HA, we currently have to have the 8020 port
> (regardless of what port HDFS is using for the namenode rpc address) in the
> following property in hbase-site.xml:
> {noformat}
>
> hbase.rootdir
> hdfs://ha-nn-uri:8020/hbase
>
> {noformat}
> Otherwise the master and regionservers will not start.
> The value in the above property should really just be
> "hdfs://ha-nn-uri/hbase" (replace "ha-nn-uri" with your uri and "hbase" with
> the name of the hbase directory in HDFS that you are using, as appropriate).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5738) Using HBase with HA HDFS requires bogus hardcoded port value
[
https://issues.apache.org/jira/browse/HBASE-5738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13260883#comment-13260883
]
Eli Collins commented on HBASE-5738:
I believe this is a dupe of HADOOP-8310.
> Using HBase with HA HDFS requires bogus hardcoded port value
>
>
> Key: HBASE-5738
> URL: https://issues.apache.org/jira/browse/HBASE-5738
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 0.92.1, 0.94.0
>Reporter: Shaneal Manek
>Assignee: Shaneal Manek
> Attachments: HBASE-5738.patch
>
>
> When configuring HBase with HDFS HA, we currently have to have the 8020 port
> (regardless of what port HDFS is using for the namenode rpc address) in the
> following property in hbase-site.xml:
> {noformat}
>
> hbase.rootdir
> hdfs://ha-nn-uri:8020/hbase
>
> {noformat}
> Otherwise the master and regionservers will not start.
> The value in the above property should really just be
> "hdfs://ha-nn-uri/hbase" (replace "ha-nn-uri" with your uri and "hbase" with
> the name of the hbase directory in HDFS that you are using, as appropriate).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5738) Using HBase with HA HDFS requires bogus hardcoded port value
[
https://issues.apache.org/jira/browse/HBASE-5738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13258056#comment-13258056
]
Eli Collins commented on HBASE-5738:
Shaneal,
Does using defaultFS fix the issues?
Looking at the exception..
{noformat}
Caused by: org.apache.hadoop.fs.InvalidPathException: Invalid path
name Wrong FS: hdfs://ha-nn-uri/hbase/-ROOT-/70236052/.logs/hlog.1327624121445,
expected: hdfs://ha-nn-uri:8020
at
org.apache.hadoop.fs.AbstractFileSystem.checkPath(AbstractFileSystem.java:361)
at
org.apache.hadoop.fs.AbstractFileSystem.create(AbstractFileSystem.java:462)
at org.apache.hadoop.fs.FileContext$3.next(FileContext.java:657)
at org.apache.hadoop.fs.FileContext$3.next(FileContext.java:654)
at
org.apache.hadoop.fs.FileContext$FSLinkResolver.resolve(FileContext.java:2319)
at org.apache.hadoop.fs.FileContext.create(FileContext.java:654)
at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:501)
at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:468)
{noformat}
AFS#checkPath is failing because the authority of the FC URI (ha-nn-uri:8020)
doesn't match the authority of the given URI (ha-nn-uri).
Looking at the FC URI, AFS#getUri...
{code}
int port = uri.getPort();
port = (port == -1 ? defaultPort : port);
if (port == -1) { // no port supplied and default port is not specified
return new URI(supportedScheme, authority, "/", null);
}
{code}
So if we don't specify a port when creating the FC (which we don't, we're using
hdfs://ha-nn-uri/hbase) we'll fail to match the authority because we didn't set
a port and we're not using the default port for HDFS (8020).
Seems like a HADOOP bug to me, specifically the path given the AFS checkPath
should be one of the configured NN addresses which will have a port (ie the
address returned by the proxy provider) instead of the logical NN uri (or the
AFS#checkPath should be more lenient if the given path doesn't specify a port).
> Using HBase with HA HDFS requires bogus hardcoded port value
>
>
> Key: HBASE-5738
> URL: https://issues.apache.org/jira/browse/HBASE-5738
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 0.92.1, 0.94.0
>Reporter: Shaneal Manek
>Assignee: Shaneal Manek
> Attachments: HBASE-5738.patch
>
>
> When configuring HBase with HDFS HA, we currently have to have the 8020 port
> (regardless of what port HDFS is using for the namenode rpc address) in the
> following property in hbase-site.xml:
> {noformat}
>
> hbase.rootdir
> hdfs://ha-nn-uri:8020/hbase
>
> {noformat}
> Otherwise the master and regionservers will not start.
> The value in the above property should really just be
> "hdfs://ha-nn-uri/hbase" (replace "ha-nn-uri" with your uri and "hbase" with
> the name of the hbase directory in HDFS that you are using, as appropriate).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5738) Using HBase with HA HDFS requires bogus hardcoded port value
[
https://issues.apache.org/jira/browse/HBASE-5738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13248101#comment-13248101
]
Hadoop QA commented on HBASE-5738:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12521649/HBASE-5738.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified
tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9)
warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
+1 core tests. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1426//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1426//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1426//console
This message is automatically generated.
> Using HBase with HA HDFS requires bogus hardcoded port value
>
>
> Key: HBASE-5738
> URL: https://issues.apache.org/jira/browse/HBASE-5738
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 0.92.1, 0.94.0
>Reporter: Shaneal Manek
>Assignee: Shaneal Manek
> Attachments: HBASE-5738.patch
>
>
> When configuring HBase with HDFS HA, we currently have to have the 8020 port
> (regardless of what port HDFS is using for the namenode rpc address) in the
> following property in hbase-site.xml:
> {noformat}
>
> hbase.rootdir
> hdfs://ha-nn-uri:8020/hbase
>
> {noformat}
> Otherwise the master and regionservers will not start.
> The value in the above property should really just be
> "hdfs://ha-nn-uri/hbase" (replace "ha-nn-uri" with your uri and "hbase" with
> the name of the hbase directory in HDFS that you are using, as appropriate).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5738) Using HBase with HA HDFS requires bogus hardcoded port value
[
https://issues.apache.org/jira/browse/HBASE-5738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13248096#comment-13248096
]
Shaneal Manek commented on HBASE-5738:
--
Thanks Jon - I didn't know about that ticket.
> Using HBase with HA HDFS requires bogus hardcoded port value
>
>
> Key: HBASE-5738
> URL: https://issues.apache.org/jira/browse/HBASE-5738
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 0.92.1, 0.94.0
>Reporter: Shaneal Manek
>Assignee: Shaneal Manek
> Attachments: HBASE-5738.patch
>
>
> When configuring HBase with HDFS HA, we currently have to have the 8020 port
> (regardless of what port HDFS is using for the namenode rpc address) in the
> following property in hbase-site.xml:
> {noformat}
>
> hbase.rootdir
> hdfs://ha-nn-uri:8020/hbase
>
> {noformat}
> Otherwise the master and regionservers will not start.
> The value in the above property should really just be
> "hdfs://ha-nn-uri/hbase" (replace "ha-nn-uri" with your uri and "hbase" with
> the name of the hbase directory in HDFS that you are using, as appropriate).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5738) Using HBase with HA HDFS requires bogus hardcoded port value
[
https://issues.apache.org/jira/browse/HBASE-5738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13248093#comment-13248093
]
Shaneal Manek commented on HBASE-5738:
--
Ah, thanks for the clarification Todd!
I wouldn't have expected it to break anything in 0.20 (since this patch sets
both fs.default.name and fs.defaultFS) - but I'll test it before we continue
any further.
I'll post an update after I've had a chance to see how it works on 0.20 (I'm
using my dev cluster for security testing right now).
> Using HBase with HA HDFS requires bogus hardcoded port value
>
>
> Key: HBASE-5738
> URL: https://issues.apache.org/jira/browse/HBASE-5738
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 0.92.1, 0.94.0
>Reporter: Shaneal Manek
>Assignee: Shaneal Manek
> Attachments: HBASE-5738.patch
>
>
> When configuring HBase with HDFS HA, we currently have to have the 8020 port
> (regardless of what port HDFS is using for the namenode rpc address) in the
> following property in hbase-site.xml:
> {noformat}
>
> hbase.rootdir
> hdfs://ha-nn-uri:8020/hbase
>
> {noformat}
> Otherwise the master and regionservers will not start.
> The value in the above property should really just be
> "hdfs://ha-nn-uri/hbase" (replace "ha-nn-uri" with your uri and "hbase" with
> the name of the hbase directory in HDFS that you are using, as appropriate).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5738) Using HBase with HA HDFS requires bogus hardcoded port value
[
https://issues.apache.org/jira/browse/HBASE-5738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13248083#comment-13248083
]
Todd Lipcon commented on HBASE-5738:
bq. The problem is that the HDFS client changed the fs.default.name property to
fs.defaultFS in HA.
That's not quite right -- the config key change was from 0.20 to 0.21, but the
old config key should still work. So I'm confused why this fixes the problem...
Also, this might break 0.20 (if it doesnt, then I'm suspicious of why this code
is there at all!)
> Using HBase with HA HDFS requires bogus hardcoded port value
>
>
> Key: HBASE-5738
> URL: https://issues.apache.org/jira/browse/HBASE-5738
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 0.92.1, 0.94.0
>Reporter: Shaneal Manek
>Assignee: Shaneal Manek
> Attachments: HBASE-5738.patch
>
>
> When configuring HBase with HDFS HA, we currently have to have the 8020 port
> (regardless of what port HDFS is using for the namenode rpc address) in the
> following property in hbase-site.xml:
> {noformat}
>
> hbase.rootdir
> hdfs://ha-nn-uri:8020/hbase
>
> {noformat}
> Otherwise the master and regionservers will not start.
> The value in the above property should really just be
> "hdfs://ha-nn-uri/hbase" (replace "ha-nn-uri" with your uri and "hbase" with
> the name of the hbase directory in HDFS that you are using, as appropriate).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
