Thanks Andrez One more thing which i wanted to know was fetching the last modified date is done as a part of content parsing function.
Cant we split the whole thing into two funtions 1)The first function will getmodified date 2) The second function will get content Initiall the first function will be called to get the modified date and compared against crawldb modified date and only if it is different will the second function (which actually fetches the content) will be called Will this be the motive of adpative interval patch? Rgds Prabhu On 2/14/06, Andrzej Bialecki <[EMAIL PROTECTED]> wrote: > > Raghavendra Prabhu wrote: > > Hi > > > > Currently the trunk has provision to setModified time in the crawldatum > > > > This is not currently used . Will this be used in the future ? > > > > > > Yes, it was added in anticipation of the adaptive fetch interval > patches. However, now I think it should probably be stored in the > CrawlDatum metadata. I'm working on this patch, I'll update it soon on > JIRA. > > -- > Best regards, > Andrzej Bialecki <>< > ___. ___ ___ ___ _ _ __________________________________ > [__ || __|__/|__||\/| Information Retrieval, Semantic Web > ___|||__|| \| || | Embedded Unix, System Integration > http://www.sigram.com Contact: info at sigram dot com > > >
