Well in the segment there is nothing but _temporary... and then after a number of spinwaiting for the last 6 elements it aborts and the segment is empty... I guess everything is left in the tmp files. If only it would abort gracefully and close the segment properly... basically lost 200k URL.
2009/8/19 Doğacan Güney <[email protected]> > On Wed, Aug 19, 2009 at 09:40, MilleBii <[email protected]> wrote: > > > Quite frustrating problem , after 24h fetching properly > > > > fetcher is spinwaiting on a fetchqueue size of 6 and then goes in > aborting > > the fetch. > > I have this problem from time to time and I have no idea why. > > Can anyone help or suggest something to debug this issue. > > > > When this happens, every URL successfully fetched should be stored in the > segment. This part works for you right? Only those last 6 URLs are not > fetched? > > > > > > -- > > -MilleBii- > > > > > > -- > Doğacan Güney > -- -MilleBii-
