[ https://issues.apache.org/jira/browse/HBASE-15860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Wellington Chevreuil updated HBASE-15860: ----------------------------------------- Status: In Progress (was: Patch Available) > Improvements for HBASE-14280 - Fixing Bulkload for HDFS HA Clusters > ------------------------------------------------------------------- > > Key: HBASE-15860 > URL: https://issues.apache.org/jira/browse/HBASE-15860 > Project: HBase > Issue Type: Improvement > Components: util > Affects Versions: 1.0.0 > Reporter: Wellington Chevreuil > Assignee: Wellington Chevreuil > Priority: Minor > Attachments: 0001-HBASE-15860.master.002.patch, > HBASE-15860.master.002.patch, HBASE-15860.master.002.patch, HBASE-15860.patch > > > HBASE-14280 introduced fix for bulkload failures when referring a remote > cluster name service id if "bulkloading" from a HA cluster. > HBASE-14280 solution on *FSHDFSUtils.getNNAddresses* was to invoke > *DFSUtil.getNNServiceRpcAddressesForCluster* instead of > *DFSUtil.getNNServiceRpcAddresses*. This works for hadoop 2.6 and above. > Proposed change here is to use "*DFSUtil.getRpcAddressesForNameserviceId*" > instead, which already returns only addresses for specific nameservice > informed. This is available since hadoop 2.4. > Sample proposal on FSHDFSUtils.getNNAddresses: > ... > {noformat} > String nameServiceId = serviceName.split(":")[1]; > if (dfsUtilClazz == null) { > dfsUtilClazz = Class.forName("org.apache.hadoop.hdfs.DFSUtil"); > } > if (getNNAddressesMethod == null) { > getNNAddressesMethod = > dfsUtilClazz.getMethod("getRpcAddressesForNameserviceId", > Configuration.class, > String.class, String.class); > } > Map<String, InetSocketAddress> nnMap = > (Map<String, InetSocketAddress>) getNNAddressesMethod > .invoke(null, conf, nameServiceId, null); > for (Map.Entry<String, InetSocketAddress> e2 : nnMap.entrySet()) { > InetSocketAddress addr = e2.getValue(); > addresses.add(addr); > } > ... > {noformat} > Will also add test conditions for *FSHDFSUtils.isSameHdfs* to verify scenario > when multiple name service ids are defined. -- This message was sent by Atlassian JIRA (v6.3.4#6332)