On 06/06/2012 07:44 PM, Sid Kumar wrote:
I am able to set it via the API.
Configuration.setBoolean(mapred.output.compress,true). This works!
But the -D from the command line still doesn't work. Any idea what I
may be missing here?
Some additional info - Also when I try running the -D on command line
on a local cluster (pseudo distributed mode) it works, but when I try
it on a fully distributed cluster running jobs from a client machine
it doesn't work. Is there a different way for setting it in this case
- in hadoop-env perhaps?
Thanks
Sid
On Wed, Jun 6, 2012 at 4:06 PM, Sid Kumar <sqlsid...@gmail.com
<mailto:sqlsid...@gmail.com>> wrote:
Mayank,
I dont have a final tag for that property set. I looked at the
mapred-default.xml in the src/mapred folder and that doesn't have
a final tag too. Should I set it explicitly to false?
You should do it explicitly.
You should read the excellent blog post from Lars Francke where he did a
great job explaining parameter by parameter and why is recommendable to
set them to final.
http://gbif.blogspot.com/2011/01/setting-up-hadoop-cluster-part-1-manual.html
Regards
Sid
On Wed, Jun 6, 2012 at 3:50 PM, Mayank Bansal <may...@apache.org
<mailto:may...@apache.org>> wrote:
Check your mapred site xml if these parameters have
<final>true</final>
making final to false should solve your problem.
On Wed, Jun 6, 2012 at 3:41 PM, Sid Kumar <sqlsid...@gmail.com
<mailto:sqlsid...@gmail.com>> wrote:
Hi,
I am trying to override mapred-site.xml (more specifically
mapred.compress.map.output
and mapred.output.compression.
codec) from the command line when I
execute the jar.
I have been using hadoop jar <jarname> <class> -
Dmapred.compress.map.output=true and
-Dmapred.output.compression.codec=org.apache.hadoop.io.SnappyCodec
The above doesnt work as the job.xml for the jar still
uses the default properties and not the one i specify
here. Is there a different approach to override these
properties. I am submitting jobs from a client machine
that has the same version of configuration files as my
cluster.
Thanks
Sid
--
Marcos Luis OrtÃz Valmaseda
Data Engineer&& Sr. System Administrator at UCI
http://marcosluis2186.posterous.com
http://www.linkedin.com/in/marcosluis2186
Twitter: @marcosluis2186
10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS
INFORMATICAS...
CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION
http://www.uci.cu
http://www.facebook.com/universidad.uci
http://www.flickr.com/photos/universidad_uci
