I have a quick question regarding this..:) Should the urls.txt file contain ALL sites you wish to index or ONLY new ones that you wish to have aspseek index?
I am doing the same thing but my urls.txt file has a listing of all sites I wish updated. I'm concerned the MySQL table that stores this information will become huge with duplicated information. Thanks. -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of Kir Kolyshkin Sent: Monday, October 07, 2002 4:49 AM To: [EMAIL PROTECTED] Subject: Re: [aseek-users] adding URLs via aspseek.conf or from file? There should not be any problem if the URL is already in the database. If you want to actually reindex it now whether it is expired or not, additionally use -a flag to 'index'. Karen Barnes wrote: > Just a question on how aspseek deals with external insertion of URLs. > My > question is this: > > Let's say I have 100 URLs already indexed and available for searching. > Now I get a request to insert this URL: > > http://www.somedomain.com/ > > I now place this URL in an external file I call urls.txt. I then do > this > with index: > > ./index -i -f ./urls.txt > > and finally index it: > > ./index > > BUT what if this URL is already in the database? Will aspseek be > intelligent enough to know this and NOT cause a problem? How will > aspseek deal with this? Will aspseek delete, overwrite, replace, > duplicate it or what? > > I have added this URL in aspseek.conf and that didn't cause a > duplicate > or any problems that I know of, but wasn't sure if I insert it using the > -i -f command whether or not this will mess things up or not. > > Of course it's easy to check if the URL already exists, but I have > about > 4 million URLs currently and receive about 1,000 URL insertion requests > a day so checking is out of the question. > > Help in answering this question would be appreciated, > Karen > > _________________________________________________________________ > Send and receive Hotmail on your mobile device: http://mobile.msn.com > -- [EMAIL PROTECTED] 7551596@ICQ [EMAIL PROTECTED] Dream like you'll live forever...Love like you've never been hurt... Work like you don't need the money...and Dance like nobody is watching! -- Satchel Paige
