If I'm not mistaken LZO compression better suits when splitting needed, not gzip.
Alex Baranau http://sematext.com On Mon, May 10, 2010 at 3:52 PM, Jeff Zhang <[email protected]> wrote: > What's the format of this file ? gzip can been split. > > > > On Mon, May 10, 2010 at 5:21 AM, Pierre ANCELOT <[email protected]> > wrote: > > Hi folks :) > > I have one big file... I read it with FileInputFormat, this generates > only > > one task and of course, this doesn't get distributed across the cluster > > nodes. > > Should I use an other Input class or do I have a bug in my > implementation? > > > > The desired behavior is one task per line. > > > > Thanks. > > > > > > > > -- > > http://www.neko-consulting.com > > Ego sum quis ego servo > > "Je suis ce que je protège" > > "I am what I protect" > > > > > > -- > Best Regards > > Jeff Zhang >
