[
https://issues.apache.org/jira/browse/HBASE-8304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13913998#comment-13913998
]
haosdent commented on HBASE-8304:
---------------------------------
Thank you for your reply. For "Why check prefix of desServiceName along with
checking srcFs", I think we should consider these cases:
1. The desFs(HBase) is hdfs://host(default port) while the srcFs(HFile) is
hdfs://host:port
2. The desFs(HBase) is hdfs://nameservices while the srcFs(HFile) is
hdfs://activeNamenode:port or hdfs://standbyNamenode:port
In hadoop 1.2.1, the achievement of canonicalServiceName comes from
{code:java}
/**
* create the service name for a Delegation token
* @param uri of the service * @param defPort is used if the uri lacks a
port * @return the token service, or null if no authority
* @see #buildTokenService(InetSocketAddress)
*/
public static String buildDTServiceName(URI uri, int defPort) {
String authority = uri.getAuthority();
if (authority == null || authority.isEmpty()) {
return null;
}
InetSocketAddress addr = NetUtils.createSocketAddr(authority, defPort);
return buildTokenService(addr).toString(); }
{code}
In hadoop 2.2.0, the achievement of cononicalServiceName comes from
{code:java}
/**
* Get the service name used in the delegation token for the given logical
* HA service.
* @param uri the logical URI of the cluster
* @return the service name
*/
public static Text buildTokenServiceForLogicalUri(URI uri) {
return new Text(HA_DT_SERVICE_PREFIX + uri.getHost());
}
{code}
HA_DT_SERVICE_PREFIX is "ha-hdfs:" here. So I think the cases above could be
covered by the patch. [[email protected]]
> Bulkload fail to remove files if fs.default.name / fs.defaultFS is configured
> without default port.
> ---------------------------------------------------------------------------------------------------
>
> Key: HBASE-8304
> URL: https://issues.apache.org/jira/browse/HBASE-8304
> Project: HBase
> Issue Type: Bug
> Components: HFile, regionserver
> Affects Versions: 0.94.5
> Reporter: Raymond Liu
> Labels: bulkloader
> Attachments: HBASE-8304.patch
>
>
> When fs.default.name or fs.defaultFS in hadoop core-site.xml is configured as
> hdfs://ip, and hbase.rootdir is configured as hdfs://ip:port/hbaserootdir
> where port is the hdfs namenode's default port. the bulkload operation will
> not remove the file in bulk output dir. Store::bulkLoadHfile will think
> hdfs:://ip and hdfs:://ip:port as different filesystem and go with copy
> approaching instead of rename.
> The root cause is that hbase master will rewrite fs.default.name/fs.defaultFS
> according to hbase.rootdir when regionserver started, thus, dest fs uri from
> the hregion will not matching src fs uri passed from client.
> any suggestion what is the best approaching to fix this issue?
> I kind of think that we could check for default port if src uri come without
> port info.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
