NLineInputFormat seems a fit for your need. On Mon, May 10, 2010 at 6:05 AM, Pierre ANCELOT <[email protected]> wrote:
> Simple and pure raw ascii text. One line == one treatment to do. > > > > On Mon, May 10, 2010 at 2:52 PM, Jeff Zhang <[email protected]> wrote: > > > What's the format of this file ? gzip can been split. > > > > > > > > On Mon, May 10, 2010 at 5:21 AM, Pierre ANCELOT <[email protected]> > > wrote: > > > Hi folks :) > > > I have one big file... I read it with FileInputFormat, this generates > > only > > > one task and of course, this doesn't get distributed across the cluster > > > nodes. > > > Should I use an other Input class or do I have a bug in my > > implementation? > > > > > > The desired behavior is one task per line. > > > > > > Thanks. > > > > > > > > > > > > -- > > > http://www.neko-consulting.com > > > Ego sum quis ego servo > > > "Je suis ce que je protège" > > > "I am what I protect" > > > > > > > > > > > -- > > Best Regards > > > > Jeff Zhang > > > > > > -- > http://www.neko-consulting.com > Ego sum quis ego servo > "Je suis ce que je protège" > "I am what I protect" >
