We've had issues with IPv6 (though I haven't had the problem with Crunch
specifically). When we last had the issue it was actually ZK that had
trouble connecting to IPv6.
There are two fixes we've found:
The first is to set surefire to use IPv4
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>-Xmx2048m -Djava.net.preferIPv4Stack=true</argLine>
</configuration>
</plugin>
The second is to change the devices /etc/hosts to be more explicit on
the IPv4 entries so it chose that over the the IPv6.
Obviously the first would probably be preferred for repeatable builds.
On Mon, Mar 31, 2014 at 10:24 AM, Josh Wills <[email protected]> wrote:
> Got it, thanks Gabriel. Will try it again when I'm @ my hotel.
>
> J
>
>
> On Mon, Mar 31, 2014 at 8:23 AM, Gabriel Reid <[email protected]
> >wrote:
>
> > Hi Josh,
> >
> > I've run into things like this before as well when running HBase
> > integration tests. As far as I can understand it, it has something to
> > do with IPv6 addresses being returned when doing a reverse DNS lookup.
> > I think that I've only gotten this intermittently when running things
> > from home, which has made me assume that my ISP was doing something
> > with their DNS setup at the time.
> >
> > Unfortunately, I don't have any suggestions as to a resolution. This
> > problem has always disappeared on its own before I've figured out how
> > to resolve it properly.
> >
> > - Gabriel
> >
> > On Mon, Mar 31, 2014 at 5:13 PM, Josh Wills <[email protected]> wrote:
> > > Hey devs,
> > >
> > > I'm back from a long break and I'm getting an odd exception on my hbase
> > > integration tests running on my mac--
> > >
> > > INFO
> > org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchControlledJob
> > > - java.lang.NumberFormatException: For input string:
> > ":22e5:2aff:fe15:2686"
> > > at
> > >
> >
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> > > at java.lang.Integer.parseInt(Integer.java:449)
> > > at java.lang.Integer.parseInt(Integer.java:499)
> > > at com.sun.jndi.dns.DnsClient.<init>(DnsClient.java:103)
> > > at com.sun.jndi.dns.Resolver.<init>(Resolver.java:44)
> > > at com.sun.jndi.dns.DnsContext.getResolver(DnsContext.java:553)
> > > at com.sun.jndi.dns.DnsContext.c_getAttributes(DnsContext.java:413)
> > > at
> > >
> >
> com.sun.jndi.toolkit.ctx.ComponentDirContext.p_getAttributes(ComponentDirContext.java:213)
> > > at
> > >
> >
> com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.getAttributes(PartialCompositeDirContext.java:121)
> > > at
> > >
> >
> com.sun.jndi.toolkit.url.GenericURLDirContext.getAttributes(GenericURLDirContext.java:85)
> > > at
> > >
> >
> javax.naming.directory.InitialDirContext.getAttributes(InitialDirContext.java:123)
> > > at org.apache.hadoop.net.DNS.reverseDns(DNS.java:84)
> > > at
> > >
> >
> org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.reverseDNS(TableInputFormatBase.java:223)
> > > at
> > >
> >
> org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:189)
> > >
> > > I get it w/both hadoop-1 and hadoop-2-- any clues as to what's going on
> > > here? To be fair, I'm running this from the Cloudera UK office for the
> > > first time evah, and I'm wondering if there's some thing I need to do
> > > differently here.
> > >
> > > Thanks,
> > > J
> > >
> > > --
> > > Director of Data Science
> > > Cloudera <http://www.cloudera.com>
> > > Twitter: @josh_wills <http://twitter.com/josh_wills>
> >
>
