Re: [aseek-users] Extremely slow search if many urls are returned

[Kir Kolyshkin]

Oops, sorry, probably I drank too much coffee yesterday.
'vmstat 1 1' is not enough, smth like 'vmstat 1 10' is needed.

Max Lytvyn wrote:

Here is the 'vmstat 1 1' output:

procs      memory      page                    disks     faults      cpu
r b w     avm    fre  flt  re  pi  po  fr  sr ad0 md0   in   sy  cs us
sy id
1 7 0  415740 232820  335   1   1   1 323 195   0   0  341 2120 153 26
9 65

It was taken while executing a one word ('word') query with 38k returned
ulrs (query took 6 seconds). Some apache requests might have been
processed in parallel.

Best
Max




-----Original Message-----
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]] On Behalf Of Kir Kolyshkin
Sent: Monday, November 25, 2002 2:02 PM
To: [EMAIL PROTECTED]
Subject: Re: [aseek-users] Extremely slow search if many urls are
returned

Can you send output of 'vmstat 1 1' made during execution of query?

Max Lytvyn wrote:

Thanks for reply.

MySQL buffer is set to 128MB.

For the second server, it makes sence to set it to 256Mb.

The system:
Hardware
Server 1: AMD Athlon 1.4GHz, 512MB memory, 60GB IDE, aspseek is the

only

big soft running.
Server 2: Dual AMD Athlon PR2200, 1GB RAM, 210GB Raid, some soft

running

in parallel, but most memory is free.

Software
Server 1: FreeBSD 4.5, MySQL 3.23.40, aspseek from cvs
Server 2: FreeBSD 5.0Current, MySQL 4.01, aspseek 1.2.10 (the latest
release)

The problem is the same on both machines.
BTW, the entire processor load is created by searchd process.
MnogoSearch developers told me that the problem is with slow sorting

of

search results because all documents are have almost the same

relevancy.

Can it be the case for ASPseek?

Best
Max


-----Original Message-----
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]] On Behalf Of Kir

Kolyshkin

Sent: Monday, November 25, 2002 1:10 PM
To: [EMAIL PROTECTED]
Subject: Re: [aseek-users] Extremely slow search if many urls are
returned

Have you increased MySQL's key_buffer_size, as described in FAQ? If

not,

this is a definitely a bottleneck in your case.

Also, please describe your hardware if you want your question to be
answered ;)

Max Lytvyn wrote:

I have a big problem with search speed - queries that contain common
words are very slow. I have an index of 200,000 documents on one server; all files are

plain

html with just title and body text, 3-40kb size. Searching with one word queries that return less than 1000 results are
very fast - about 0.1 sec or even faster. But if any word of a query
matches many urls (e.g. 'word' matches 58000 urls), the search takes
over 5 seconds (up to 25 secs, if several common words are used
together). I had the same problem with MnogoSearch, but in that case query time

was

exponentially dependent on the number of results returned, and queries
with more keywords (and thus fewer urls returned) were faster. In
aspseek the situation is the same, but looks like the search time is
exponentially dependent on the sum of the url all keywords of the

query

return, not on intersection.
Please HELP!!! I'm desperate - the server load reaches 87% - it is
critical.
Best Max