- - - - - - - - - - - - - - - - - - - - - - - - - - - -
Name: Jon
Subject: Setting up Cached Mode w/SearchD
First off... thanks for a great piece of software.
Second, I have crc-multi mode running great with searchD as the current setup.
Third, I have issues when starting to use cacheD.
Setup:
I have three servers. One server is the indexer, another is the cacheD/searchD
and the other is the frontend (search.cgi/mod_dpsearch). All three servers are
running 4.38. When using the dbmode 'crc-multi' everything works great... other
then it is slow. I am using mySQL for the database and it just can't keep up.
Thus, I setup searchD. This speeds things up a lot. search.cgi (DBAddr
searchd://host:port/)->searchD(DBAddr mysql://foo:[EMAIL
PROTECTED]/dbname/?socket=/path/to/mysql.sock&dbmode=crc-multi) ->mysql works
great. It just slows down when I add tons of data. Thus, I have tried setting
up cacheD.
When I get cacheD into the mix I get empty search results. I am using the same
search.cgi config and the searchD gets the request. A transcript of the request
is as follows:
[5789]{05} Connect 63.231.95.97
[5789]{05} 63.231.95.97 Allow by default
[6376]{1000} Waiting for command header
[6376]{1000} Received header cmd=3 len=142
[6376]{1000} Received words len=142
words='q=fedora&cmd=Search&tmplt=search.fp.htm&BrowserCharset=iso-8859-1&IP=xxx&g-lc=en&ExcerptSize=256&ExcerptPadding=40&tmplt=search.fp.htm'
[6376]{1000} Query: fedora [Charset: iso-8859-1]
[6376]{1000} Prepare query: fedora, ltxt:fedora
[6376]{1000} Segment lang:
[6376]{1000} wrd {6}: fedora
[6376]{1000} DpsFind for mysql://foo:[EMAIL
PROTECTED]/dbname/?socket=/path/to/mysql.sock&dbmode=cache&cached=localhost:7001
[6376]{1000} DpsGetWords for mysql://foo:[EMAIL
PROTECTED]/dbname/?socket=/path/to/mysql.sock&dbmode=cache&cached=localhost:7001
[6376]{1000} Can't open/create file /path/to/tree/wrd04ed.i [base.c:233] -- 2
(No such file or directory)
[6376]{1000} Sent total_found packet 24 bytes buf='Total_found=0(0)'
[6376]{1000} Sent WWL packet 95 bytes cmd=8 len=87 nwords=1
[6376]{1000} Sent URLDATA packet 8 bytes cmd=13 len=0
[6376]{1000} Sent words packet 8 bytes cmd=3 len=0 nwords=0
[6376]{1000} Waiting for command header
[6376]{1000} Received header cmd=4 len=0
[6376]{1000} Received goodbye command. Work time: 0.025 sec.
[6376]{1000} Quit
There is nothing in the var/tree dir. The var/ dir looks like the following:
.:
cache cached.pid searchd.pid splitter store tree url
./cache:
./splitter:
./store:
./tree:
./url:
info0000.i info0054.s info005b.i info00ac.s info00b3.i info00ef.s
info0000.s info0055.i info005b.s info00ad.i info00b3.s info00f0.i
info0009.i info0055.s info005c.i info00ad.s info00b4.i info00f0.s
info0009.s info0056.i info005c.s info00ae.i info00b4.s info00f1.i
info002b.i info0056.s info00a8.i info00ae.s info00b5.i info00f1.s
info002b.s info0057.i info00a8.s info00af.i info00b5.s info00f2.i
info002c.i info0057.s info00a9.i info00af.s info00d0.i info00f2.s
info002c.s info0058.i info00a9.s info00b0.i info00d0.s info00f3.i
info002d.i info0058.s info00aa.i info00b0.s info00ed.i info00f3.s
info002d.s info0059.i info00aa.s info00b1.i info00ed.s
info0032.i info0059.s info00ab.i info00b1.s info00ee.i
info0032.s info005a.i info00ab.s info00b2.i info00ee.s
info0054.i info005a.s info00ac.i info00b2.s info00ef.i
The permissions are fine and I have checked to make sure it is not SELinux
getting in the way. Example is as follows:
-rw-r--r-- 1 user user 114604 Mar 15 02:01 info00f3.i
The cacheD gets the request from the searchD. Examples follow:
[5807]{00} [127.0.0.1] Connected. PORT: 157,206
[5807]{31} Wed 15 03:42:36 [5807] Client thread started
[5807]{31} Wed 15 03:42:36 [5807] Client action BYE received.
[5807]{00} [127.0.0.1] Connected. PORT: 157,211
[5807]{32} Wed 15 04:07:05 [5807] Client thread started
[5807]{32} Wed 15 04:07:05 [5807] Client action BYE received.
[5807]{00} [127.0.0.1] Connected. PORT: 157,214
[5807]{33} Wed 15 04:07:10 [5807] Client thread started
[5807]{33} Wed 15 04:07:10 [5807] Client action BYE received.
I have run the indexer as I normally would.. and have tried some new options
that I didn't use on the crc-multi dbmode index. Three example indexer commands
follow:
./indexer -N 3 -U -n 500 -YY -ZZZ -W -H /path/to/indexer.cache.fp.conf
./indexer -N 3 -U -n 500 /path/to/indexer.cache.fp.conf
./indexer -n 500 /path/to/indexer.cache.fp.conf
DBAddr mysql://foo:[EMAIL
PROTECTED]/dbname/?socket=/path/to/mysql.sock&dbmode=cache&cached=localhost:7001
(indexer.cache.fp.conf DBAddr)
./indexer -S /path/to/indexer.cache.fp.conf:
Database statistics
Status Expired Total
-----------------------------
0 39756 39756 Not indexed yet
200 6752 7172 OK
301 9 11 Moved Permanently
302 805 854 Moved Temporarily
304 0 12 Not Modified
403 25 26 Forbidden
404 69 73 Not found
415 116 128 Unsupported Media Type
-----------------------------
Total 47532 48032
And the resulting call to cacheD:
[5807]{35} Wed 15 04:27:27 [5807] Client thread started
[5807]{35} Wed 15 04:27:27 [5807] Client action BYE received.
Both of the indexers run fine. The var/url dir is getting filled up but I am
getting no data in var/tree or anywhere else. There is nothing in var/splitter
so I don't think I need to run splitter or run-splitter. I am out of ideas. It
looks to me like the search.cgi, searchD, cacheD combination is working fine..
there just is no data for the searchD to send back to the search.cgi.
My question is: How do I populate the data that is needed to do searches in the
cached mode? What am I missing?
- - - - - - - - - - - - - - - - - - - - - - - - - - - -
Read the full topic here:
http://www.dataparksearch.org/cgi-bin/simpleforum.cgi?fid=02;post=
