I'm having trouble building accumulo 1.5.0 with the hadoop 2.0 profile. I'm
using the source release accumulo-1.5.0-src.tar.gz from
http://accumulo.apache.org/downloads/ . I'm building on Centos 6.3 with
oracle java 1.7 and maven 3.0.5.
I think I'm running into two separate issues.
First, when building with the default hadoop version for the 2.0 profile
with this command: 'mvn -Dhadoop.profile=2.0
-Dmaven.test.failure.ignore=true clean install'
I get two test failures.
testFileMonitor(org.apache.accumulo.start.classloader.vfs.providers.VfsClassLoaderTest)
Time elapsed: 0.385 sec <<< ERROR!
testGetClass(org.apache.accumulo.start.classloader.vfs.providers.VfsClassLoaderTest)
Time elapsed: 0.084 sec <<< ERROR!
Both errors have the same cause:
java.lang.UnsatisfiedLinkError:
org.apache.hadoop.util.NativeCrc32.nativeVerifyChunkedSums(IILjava/nio/ByteBuffer;ILjava/nio/ByteBuffer;IILjava/lang/String;J)V
at
org.apache.hadoop.util.NativeCrc32.nativeVerifyChunkedSums(Native Method)
at
org.apache.hadoop.util.NativeCrc32.verifyChunkedSums(NativeCrc32.java:57)
at
org.apache.hadoop.util.DataChecksum.verifyChunkedSums(DataChecksum.java:291)
at
org.apache.hadoop.hdfs.RemoteBlockReader2.readNextPacket(RemoteBlockReader2.java:187)
at
org.apache.hadoop.hdfs.RemoteBlockReader2.read(RemoteBlockReader2.java:122)
at
org.apache.hadoop.hdfs.DFSInputStream$ByteArrayStrategy.doRead(DFSInputStream.java:542)
at
org.apache.hadoop.hdfs.DFSInputStream.readBuffer(DFSInputStream.java:594)
at
org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:648)
at
org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:689)
at java.io.DataInputStream.read(DataInputStream.java:149)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
at
org.apache.commons.vfs2.util.MonitorInputStream.read(MonitorInputStream.java:100)
at java.io.FilterInputStream.read(FilterInputStream.java:107)
at org.apache.commons.vfs2.FileUtil.writeContent(FileUtil.java:85)
at org.apache.commons.vfs2.FileUtil.copyContent(FileUtil.java:114)
at
org.apache.commons.vfs2.provider.AbstractFileObject.copyFrom(AbstractFileObject.java:1053)
at
org.apache.commons.vfs2.impl.DefaultFileReplicator.replicateFile(DefaultFileReplicator.java:249)
at
org.apache.commons.vfs2.provider.AbstractFileSystem.doReplicateFile(AbstractFileSystem.java:467)
at
org.apache.commons.vfs2.provider.AbstractFileSystem.replicateFile(AbstractFileSystem.java:423)
at
org.apache.commons.vfs2.provider.zip.ZipFileSystem.<init>(ZipFileSystem.java:61)
at
org.apache.commons.vfs2.provider.jar.JarFileSystem.<init>(JarFileSystem.java:50)
at
org.apache.commons.vfs2.provider.jar.JarFileProvider.doCreateFileSystem(JarFileProvider.java:82)
at
org.apache.commons.vfs2.provider.AbstractLayeredFileProvider.createFileSystem(AbstractLayeredFileProvider.java:89)
at
org.apache.commons.vfs2.impl.DefaultFileSystemManager.createFileSystem(DefaultFileSystemManager.java:914)
at
org.apache.commons.vfs2.impl.DefaultFileSystemManager.createFileSystem(DefaultFileSystemManager.java:933)
at
org.apache.commons.vfs2.impl.VFSClassLoader.addFileObjects(VFSClassLoader.java:153)
at
org.apache.commons.vfs2.impl.VFSClassLoader.<init>(VFSClassLoader.java:116)
at
org.apache.commons.vfs2.impl.VFSClassLoader.<init>(VFSClassLoader.java:98)
at
org.apache.accumulo.start.classloader.vfs.providers.VfsClassLoaderTest.setup(VfsClassLoaderTest.java:58)
... and a bunch more
Second problem: If I try to build with a more recent hadoop release
(2.1.0-beta and up) command: 'mvn -Dhadoop.profile=2.0
-Dhadoop.version=2.1.0-beta -Dmaven.test.failure.ignore=true clean install'
I end up with 140+ test errors. Many are like this:
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:131)
at org.apache.hadoop.security.Groups.<init>(Groups.java:55)
at
org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:182)
at
org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:234)
at
org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:213)
at
org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:662)
at
org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:570)
at
org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:2586)
at
org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:2578)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2444)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:363)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:165)
at
org.apache.accumulo.core.client.mock.MockInstance.getDefaultFileSystem(MockInstance.java:69)
at
org.apache.accumulo.core.client.mock.MockInstance.<init>(MockInstance.java:60)
at
org.apache.accumulo.server.util.CloneTest.testMerge(CloneTest.java:344)
... 25 more (my truncation)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:129)
... 39 more
Caused by: java.lang.UnsatisfiedLinkError:
org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative()V
at
org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative(Native
Method)
at
org.apache.hadoop.security.JniBasedUnixGroupsMapping.<clinit>(JniBasedUnixGroupsMapping.java:49)
at
org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback.<init>(JniBasedUnixGroupsMappingWithFallback.java:38)
... 44 more
Lastly, I can build with no errors if I use the default hadoop-1.0 profile.
Both of my problems seem to boil down to UnsatisfiedLinkErrors related to
the underlying Hadoop dependencies. I can't find much information to help
resolve this other than some general java related discussions saying that
UnsatisfiedLinkErrors usually result from missing native libraries. Can
anyone point me in the right direction? Thanks!
