RE: Multiple Reducers for Linkdb
Thanks Markus, I will try to run linkdb with multiple Reducer. Regards, Suraj Singh -Original Message- From: Markus Jelsma Sent: Tuesday, 18 December 2018 15:48 To: user@nutch.apache.org Subject: RE: Multiple Reducers for Linkdb Hello Suraj, You can safely run the LinkDB merger with as many reducers as you like. Regards, Markus -Original message- > From:Suraj Singh > Sent: Tuesday 18th December 2018 15:39 > To: user@nutch.apache.org > Subject: Multiple Reducers for Linkdb > > Hello, > > Can we run Linkdb(invertlinks) with multiple reducers? > > I am asking this because by default it runs with just one Reducer and it > takes good amount of time to complete and it keeps on increasing every > subsequent round since the number of Mappers keeps on increasing with every > round. > > Of course we can set the number of reducers to more than one but I am afraid > it will break something in subsequent steps as I am not sure why it was built > to run with just one Reducer considering the overload with every subsequent > round. > > Thanks in Advance. > > Regards, > Suraj Singh > >
RE: Multiple Reducers for Linkdb
Hello Suraj, You can safely run the LinkDB merger with as many reducers as you like. Regards, Markus -Original message- > From:Suraj Singh > Sent: Tuesday 18th December 2018 15:39 > To: user@nutch.apache.org > Subject: Multiple Reducers for Linkdb > > Hello, > > Can we run Linkdb(invertlinks) with multiple reducers? > > I am asking this because by default it runs with just one Reducer and it > takes good amount of time to complete and it keeps on increasing every > subsequent round since the number of Mappers keeps on increasing with every > round. > > Of course we can set the number of reducers to more than one but I am afraid > it will break something in subsequent steps as I am not sure why it was built > to run with just one Reducer considering the overload with every subsequent > round. > > Thanks in Advance. > > Regards, > Suraj Singh > >
Multiple Reducers for Linkdb
Hello, Can we run Linkdb(invertlinks) with multiple reducers? I am asking this because by default it runs with just one Reducer and it takes good amount of time to complete and it keeps on increasing every subsequent round since the number of Mappers keeps on increasing with every round. Of course we can set the number of reducers to more than one but I am afraid it will break something in subsequent steps as I am not sure why it was built to run with just one Reducer considering the overload with every subsequent round. Thanks in Advance. Regards, Suraj Singh
