We have about 60,000 URLs in total... and only about 2,300 are indexed... I
think that the problem is not really in the DB, i think that it is more of
an indexer problem, stalling at URLs that to not answer, and maybe stopping
in the middle of indexing...
I have let 3 indexers run for about 1 hour and it has indexed only about 200
sites, on a dedicated line (i.e. it is not a bandwidth limitation, i am
sure...
Oh well... hehehe... I am a bit frustrated, but i will try what you
suggested...
On another note, I am still unable to make any Frontend search my index...
No matter what backend, no matter what keyword, i always get 0 found... Can
someone help me? (I dont get any errors, just get 0 found)...
Please help me, aterall, there is no use indexing if i cant read the
index...
-----Original Message-----
From: Shane Wegner
To: Roy Nasser
Cc: '[EMAIL PROTECTED] '
Sent: 08/10/2000 22:15
Subject: Re: UdmSearch: Question: way to compile for more performance
On Sun, Oct 08, 2000 at 06:45:07PM -0300, Roy Nasser wrote:
> Hi,
>
> I can see that it indexes the documents, but it sometimes stalls for
over 2
> minutes on a single URL... We have about 58,000 pending URLs, and it
really
> takes a long time to index...
How many URLs do you have in the database? 2 minutes is a
bit long but try adding an index on url.next_index_time.
Indexer if I recall correctly does select * from url where
next_index_time<=unix_timestamp() limit 32;
So every 32 URLs you will get a pause if you have many URLs
in the database. I don't have it indexed but only have
74,000 URLs being indexed. The pause is about 10 seconds
or so every 32 entries.
Anyways, you can give that a try and see if it helps.
> Is there a special compilation option, or should i pass a special
parameter
> to the indexer for it to fetch like, 5 or 10 URLs simultaneously?
Well there may be gcc options which could improve your
performance. The various -m options come to mind but from
my work with UDMSearch, it's pretty fast and is generally
the database which is the bottleneck.
Cheers,
Shane
--
Shane Wegner: [EMAIL PROTECTED]
http://www.cm.nu/~shane/
PGP: 1024D/FFE3035D
A0ED DAC4 77EC D674 5487
5B5C 4F89 9A4E FFE3 035D
______________
If you want to unsubscribe send "unsubscribe udmsearch"
to [EMAIL PROTECTED]
______________
If you want to unsubscribe send "unsubscribe udmsearch"
to [EMAIL PROTECTED]
