Manuel, I think that the key here is NOT to merge the results. Assuming that you have enough disc space, my suggestion would be:
1. Do a full site index (-i option), using the -a option to give you a set of .work databases, preferably in a different/sub-folder 2. Do nightly update digs on these databases (remove the -i) 3. Create copies of the .work databases in the live folder 4. Delete the live databases 5. Rename the copied .work databases This avoids having to take the databases off-line while you are digging, and is the only way that I can think of to allow htdig to purge old URLs from the live database. NB steps 3,4,5 may look a little convoluted, but duplicating databases of the size that these will be will take many seconds, by doing things like this the database will be offline for less than 1 second, (typically). All of this should be easy to automate - I do similar things to this with batch files on NT-based servers, but I am sure that the same would be possible with shell scripts on *nix. Hope that was what you were after, Mike NB My apologies Geoff - Outlook was doing its best to confuse me, and succeeded easily. > -----Original Message----- > > My question is how can I reindex just a few pages at once and > merge the > crawled pages with a previously indexed site database? I mean, index > only a few pages that I list and only follow links to site pages that > were not yet indexed. ******************************************************************** This email may contain information which is privileged or confidential. If you are not the intended recipient of this email, please notify the sender immediately and delete it without reading, copying, storing, forwarding or disclosing its contents to any other person Thank you Check us out at http://www.bt.com/consulting ******************************************************************** ------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader's Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_idU88&alloc_id065&op=click _______________________________________________ ht://Dig general mailing list: <[EMAIL PROTECTED]> ht://Dig FAQ: http://htdig.sourceforge.net/FAQ.html List information (subscribe/unsubscribe, etc.) https://lists.sourceforge.net/lists/listinfo/htdig-general
