Yes, whatever I name the output directory, the files inside are compressed with the lzo codec.
On Mon, Jul 30, 2012 at 2:52 PM, souri datta <[email protected]>wrote: > James, > I may not have understood your question fully,but did you try renaming the > file w/o the '.gz' ? In hadoop/pig, if you have bz2 extn, files get > compressed.Hence,.. > > On Mon, Jul 30, 2012 at 11:51 PM, James Kebinger <[email protected] > >wrote: > > > Hello, I'm running a pretty simple pig job but despite my best efforts to > > disable compression, the output parts are written in in .lzo_deflate form > > like > > > > "/user/jkebinger/pigresults3.gz/part-r-00000.lzo_deflate" > > > > I'm launching the job like this JAVA_OPTS=-Dmapred.output.compress=false > > PIG_OPTS=-Dmapred.output.compress=false ../pig-0.10.0/bin/pig test.pig > > > > in a vain effort to turn that off. > > > > Can someone please point me in the right direction? > > > > > > The store statement looks like this: > > > > STORE bothcountedp into '/user/jkebinger/pigresults3.gz' USING > > PigStorage(); > > > > (the filename there has varied from no extension to .gz with no effect on > > how the files inside it are treated) > > > > thanks! > > > > -James > > >
