OK, I see it now. You mean to say the JNI-Snappy which is in org.xerial package works correctly but not Dain's snappy which is in org.iq80 , which is pure Java.
I would try to check all of these after my exams, once again.(Day After tomorrow) I remember that I used 0.3-Incubating version of Dain's snappy and it seemed to work fine. I was able to run Pi Example with it. Now if we are to use the JNI version of snappy it might be slow so instead of using it as a default compressor, we would like to give option of choosing between compression and no-compression? -- thanks and regards, Apurv Verma B. Tech.(CSE) IIT- Ropar On Fri, Feb 24, 2012 at 4:36 AM, Apurv Verma <[email protected]> wrote: > What is buggy in Dain's snappy, can you let me know? > > -- > thanks and regards, > > Apurv Verma > B. Tech.(CSE) > IIT- Ropar > > > > > > > On Thu, Feb 23, 2012 at 5:06 PM, Thomas Jungblut < > [email protected]> wrote: > >> It seems that Dain's snappy is buggy. I tried the Hadoop codec as well as >> directly using streams. However, I just changed it to the xerial version >> in >> POM and it now works correctly. >> >> I will contact dain and try to sort out things. Please update the trunk >> and rebuild. >> Sorry for the inconvenience. >> >> 2012/2/23 Thomas Jungblut <[email protected]> >> >> > Yep, maybe this is the issue. In the Maven POM is the dain version, in >> > impor statements the xerial version. Sorry for the inconsistency. I fix >> it. >> > >> > >> > 2012/2/23 Thomas Jungblut <[email protected]> >> > >> >> Wait a minute, I guess this is the wrong library. >> >> >> >> My intention was to use the https://github.com/dain/snappy lib, since >> >> it is faster than the JNI port. >> >> I investigate it after launch. >> >> >> >> >> >> 2012/2/23 Edward J. Yoon <[email protected]> >> >> >> >>> Hmm, the snappy-0.2.jar is already in lib folder. >> >>> >> >>> edward@slave:~/workspace/hama-trunk$ ls lib/ >> >>> ant-1.7.1.jar commons-configuration-1.6.jar >> >>> hadoop-test-1.0.0.jar jetty-util-6.1.14.jar >> >>> netty-3.2.6.Final.jar zookeeper-3.3.3.jar >> >>> ant-launcher-1.7.1.jar commons-lang-2.6.jar >> >>> jackson-core-asl-1.9.2.jar jsp-2.1-6.1.14.jar >> >>> servlet-api-6.0.32.jar >> >>> avro-1.6.0.jar commons-logging-1.1.1.jar >> >>> jackson-mapper-asl-1.9.2.jar jsp-api-2.1-6.1.14.jar >> >>> slf4j-api-1.5.8.jar >> >>> avro-ipc-1.6.0.jar guava-10.0.1.jar >> >>> jetty-6.1.14.jar junit-4.8.1.jar >> >>> slf4j-log4j12-1.5.8.jar >> >>> commons-cli-1.2.jar hadoop-core-1.0.0.jar >> >>> jetty-annotations-6.1.14.jar log4j-1.2.16.jar snappy-0.2.jar >> >>> >> >>> >> >>> On Thu, Feb 23, 2012 at 4:51 PM, Thomas Jungblut >> >>> <[email protected]> wrote: >> >>> > You have to update your jars, Snappy is a new dependency for >> >>> compression. >> >>> > >> >>> > 2012/2/23 Edward J. Yoon <[email protected]> >> >>> > >> >>> >> I received below error message with pi example. >> >>> >> >> >>> >> 12/02/23 15:26:16 DEBUG sync.ZooKeeperSyncClientImpl: >> leaveBarrier() >> >>> >> at superstep:-1 znode size: (0) znodes:[] >> >>> >> 12/02/23 15:26:16 WARN bsp.GroomServer: Error running child >> >>> >> java.lang.NoClassDefFoundError: org/xerial/snappy/SnappyInputStream >> >>> >> at java.lang.Class.forName0(Native Method) >> >>> >> at java.lang.Class.forName(Class.java:247) >> >>> >> at >> >>> >> >> >>> >> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:819) >> >>> >> at >> >>> >> >> >>> >> org.apache.hama.bsp.message.compress.BSPMessageCompressorFactory.getCompressor(BSPMessageCompressorFactory.java:37) >> >>> >> at >> >>> >> >> >>> >> org.apache.hama.bsp.message.CompressableMessageManager.initCompression(CompressableMessageManager.java:36) >> >>> >> at >> >>> >> >> >>> >> org.apache.hama.bsp.message.AvroMessageManagerImpl.init(AvroMessageManagerImpl.java:60) >> >>> >> at >> org.apache.hama.bsp.BSPPeerImpl.<init>(BSPPeerImpl.java:145) >> >>> >> at >> >>> >> >> >>> >> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:969) >> >>> >> Caused by: java.lang.ClassNotFoundException: >> >>> >> org.xerial.snappy.SnappyInputStream >> >>> >> at java.net.URLClassLoader$1.run(URLClassLoader.java:202) >> >>> >> at java.security.AccessController.doPrivileged(Native >> Method) >> >>> >> at >> java.net.URLClassLoader.findClass(URLClassLoader.java:190) >> >>> >> at java.lang.ClassLoader.loadClass(ClassLoader.java:306) >> >>> >> at >> >>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >> >>> >> at java.lang.ClassLoader.loadClass(ClassLoader.java:247) >> >>> >> >> >>> >> >> >>> >> -- >> >>> >> Best Regards, Edward J. Yoon >> >>> >> @eddieyoon >> >>> >> >> >>> > >> >>> > >> >>> > >> >>> > -- >> >>> > Thomas Jungblut >> >>> > Berlin <[email protected]> >> >>> >> >>> >> >>> >> >>> -- >> >>> Best Regards, Edward J. Yoon >> >>> @eddieyoon >> >>> >> >> >> >> >> >> >> >> -- >> >> Thomas Jungblut >> >> Berlin <[email protected]> >> >> >> > >> > >> > >> > -- >> > Thomas Jungblut >> > Berlin <[email protected]> >> > >> >> >> >> -- >> Thomas Jungblut >> Berlin <[email protected]> >> > >
