[
https://issues.apache.org/jira/browse/HDFS-8078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14515054#comment-14515054
]
Nate Edel commented on HDFS-8078:
---------------------------------
Thanks!
The base case of just appending ipAddr blindly was the existing behavior, and
it's not super. As far as I can tell, the existing code path won't pass in a
null, but will in some cases pass in empty string -- the comment addresses
that. I can update the patch to treat null with "" (as it stands, it will
append the string "null" -- forcing that results in an UnknownHostException
when it tries to open the URI) -- neither option is super, and this change
might mask some bugs (UnknownHostException on "null" being potentially clearer
than a failed connection back to localhost.)
ipAddr.contains(":") is a very basic check that it might be an IPv6 address; we
should never have an ip:port pair at that point. I then call getByName() as the
safest way to validate that it's a legal IPv6 address and convert it to an
Inet6Address (the latter allows us to normalize it if it's in a compressed
format - e.g. [::1] to [0:0:0:0:0:0:0:1] - or there is other valid but
undesirable formatting weirdness.)
getByName will never do a DNS lookup; it uses the same heuristic of "is there a
colon" to assume things are IPv6 literals. I don't see any current cases which
will pass the ipAddr.contains(":") test but not be an IPv6 address, but if
through a bug we got an ip:port combination passed in to DataNodeID as the
ipAddr string, it would generate the UnknownHostException as it would fail to
parse as an IPV6 literal without doing a DNS lookup (e.g.
java.net.UnknownHostException: 127.0.0.1:9000: invalid IPv6 address)
The last behavior might be JDK-dependent, but appears to be the case for recent
Oracle releases on both 1.7 and 1.8 (and on a quick look at the code for
InetAddress.java this is unchanged in OpenJDK.)
Minor patch change coming later this afternoon to address nit and case of null
being passed in.
> HDFS client gets errors trying to to connect to IPv6 DataNode
> -------------------------------------------------------------
>
> Key: HDFS-8078
> URL: https://issues.apache.org/jira/browse/HDFS-8078
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
> Affects Versions: 2.6.0
> Reporter: Nate Edel
> Assignee: Nate Edel
> Labels: ipv6
> Attachments: HDFS-8078.4.patch
>
>
> 1st exception, on put:
> 15/03/23 18:43:18 WARN hdfs.DFSClient: DataStreamer Exception
> java.lang.IllegalArgumentException: Does not contain a valid host:port
> authority: 2401:db00:1010:70ba:face:0:8:0:50010
> at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:212)
> at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164)
> at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:153)
> at
> org.apache.hadoop.hdfs.DFSOutputStream.createSocketForPipeline(DFSOutputStream.java:1607)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1408)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1361)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:588)
> Appears to actually stem from code in DataNodeID which assumes it's safe to
> append together (ipaddr + ":" + port) -- which is OK for IPv4 and not OK for
> IPv6. NetUtils.createSocketAddr( ) assembles a Java URI object, which
> requires the format proto://[2401:db00:1010:70ba:face:0:8:0]:50010
> Currently using InetAddress.getByName() to validate IPv6 (guava
> InetAddresses.forString has been flaky) but could also use our own parsing.
> (From logging this, it seems like a low-enough frequency call that the extra
> object creation shouldn't be problematic, and for me the slight risk of
> passing in bad input that is not actually an IPv4 or IPv6 address and thus
> calling an external DNS lookup is outweighed by getting the address
> normalized and avoiding rewriting parsing.)
> Alternatively, sun.net.util.IPAddressUtil.isIPv6LiteralAddress()
> -------
> 2nd exception (on datanode)
> 15/04/13 13:18:07 ERROR datanode.DataNode:
> dev1903.prn1.facebook.com:50010:DataXceiver error processing unknown
> operation src: /2401:db00:20:7013:face:0:7:0:54152 dst:
> /2401:db00:11:d010:face:0:2f:0:50010
> java.io.EOFException
> at java.io.DataInputStream.readShort(DataInputStream.java:315)
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:58)
> at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:226)
> at java.lang.Thread.run(Thread.java:745)
> Which also comes as client error "-get: 2401 is not an IP string literal."
> This one has existing parsing logic which needs to shift to the last colon
> rather than the first. Should also be a tiny bit faster by using lastIndexOf
> rather than split. Could alternatively use the techniques above.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
