On Fri, Jan 20, 2012 at 5:10 PM, Marek Bachmann <[email protected]>wrote:
> Hello again, > > I was inspecting the generator because it doesn't deliver all urls for the > fetcht list from the crawldb even if I set the addDays atribute to a value > much higher than the max fetch intervall. > How much higher was your -addDays arguement value set to? > > As I had a look at the log file I notice that it uses a time stamp which I > don't know: > > Mmmm... perhaps not by coincidence I don't seem to know these TS either... > > Does the generator use another kind of timestamp than unix systems? Or is > something terrible wrong here? > > I would say the latter of the two. If you look at the Generator we get the fetchtime from the CrawlDatum and the curTime by setting it to System.currentTimeMillis(). In all honesty I have no idea how you go these value's. Is this the only thing that looks suspicious in your logs? As far as I am aware, all date's ACROSS THE BOARD with Nutch should be in the sdf format yyyy-MM-dd HH:mm:ss -- *Lewis*
