Caffeinate The World wrote:
>
> --- Alexander Barkov <[EMAIL PROTECTED]> wrote:
> > Caffeinate The World wrote:
> > >
> > > if i insert a bunch of URLs into the db via a text file.. and those
> > > urls have no corresponding server command, by default, it should
> > have a
> > > Follow of path and NOT site.
> > >
> > > here is one example. i pull URLs from a sub section of dmoz using
> > the
> > > Href directive. Some of those are site URLs and some are a path of
> > a
> > > site. now, when i pull all the urls from that sub section of dmoz,
> > the
> > > current setting will index thru all the "whole" sites/servers
> > instead
> > > of just the paths.
> > >
> > > if default is to follow path, then the whole site would still get
> > > indexed. i hope that made sense. i brought this topic up about a
> > month
> > > ago and i still see this behavior in -current.
> >
> >
> > When "DeleteNoServer no" is specified, indexer follows whole site
> > if there is no "Server" command for some URL.
>
> i understand this, but is this the correct behavior? if it is, i can't
> think of any situation where it's appropriate.
Probably we'll change this in 3.2.x branch. Currently it requires major
changes in sources. We want to avoid this before stable version release.
___________________________________________
If you want to unsubscribe send "unsubscribe general"
to [EMAIL PROTECTED]
