I'm now able to use snappy compression by adding export JAVA_LIBRARY_PATH=/usr/lib/hadoop/lib/native/ in my /etc/default/mesos-slave (environment variable for mesos-slave process used by my init.d script)
This envar is propagated to executor Jvm and so taskTracker can find libsnappy.so to use it. Starting using local deployement of cdh4 ... Reading at the source it seams that something could be done using mapred.mesos.executor.directory and mapred.mesos.executor.command to use local hadoop. Le 31/12/2013 15:08, Damien Hardy a écrit : > Hello, > > Happy new year 2014 @mesos users. > > I am trying to get MapReduce cdh4.1.2 running on Mesos. > > Seams working mostly but few things are still problematic. > > * MR1 code is already deployed locally with HDFS is there a way to use > it instead of tar.gz stored on HDFS to be copied locally and untar. > > * If not, using tar.gz distribution of cdh4 seams not supporting > Snappy compression. is there a way to correct it ? > > Best regards, > -- Damien HARDY
signature.asc
Description: OpenPGP digital signature
