Idea is, I want to share the lines of the file equally between nodes...
On Mon, May 10, 2010 at 3:05 PM, Pierre ANCELOT <[email protected]> wrote: > Simple and pure raw ascii text. One line == one treatment to do. > > > > > On Mon, May 10, 2010 at 2:52 PM, Jeff Zhang <[email protected]> wrote: > >> What's the format of this file ? gzip can been split. >> >> >> >> On Mon, May 10, 2010 at 5:21 AM, Pierre ANCELOT <[email protected]> >> wrote: >> > Hi folks :) >> > I have one big file... I read it with FileInputFormat, this generates >> only >> > one task and of course, this doesn't get distributed across the cluster >> > nodes. >> > Should I use an other Input class or do I have a bug in my >> implementation? >> > >> > The desired behavior is one task per line. >> > >> > Thanks. >> > >> > >> > >> > -- >> > http://www.neko-consulting.com >> > Ego sum quis ego servo >> > "Je suis ce que je protège" >> > "I am what I protect" >> > >> >> >> >> -- >> Best Regards >> >> Jeff Zhang >> > > > > -- > http://www.neko-consulting.com > Ego sum quis ego servo > "Je suis ce que je protège" > "I am what I protect" > > -- http://www.neko-consulting.com Ego sum quis ego servo "Je suis ce que je protège" "I am what I protect"
