- - - - - - - - - - - - - - - - - - - - - - - - - - - -
Name: takerushi
Subject: Slow Indexing
Hi Maxime,
I have trying to index several sites with dataparksearch, the number showed in
status 0 increase real fast, but the number in status 200 is increase by a very
slow pace, here is an example of my index status
##################################################
Status Expired Total
-----------------------------
0 500 500 Not indexed yet
200 0 77 OK
206 0 1 Partial OK
301 0 2 Moved Permanently
302 0 2 Moved Temporarily
401 0 1 Unauthorized
403 0 2 Forbidden
404 0 3 Not found
406 0 8 Not Acceptable
415 0 5 Unsupported Media Type
500 2 2 Internal Server Error
503 8 9 Service Unavailable
504 6 6 Gateway Timeout
-----------------------------
Total xxxx xxxx
##################################################
My questions are:
1. What is the maximum amount of pages dataparksearch can store and index?
2. Why is the number is status 0 increase much faster than status 200? The page
number showed in status 0 sometime even 4000 times larger than number showed in
status 200?
3. Is there anyway to ask the indexer to stop indexing extra sites that have
not yet been stored into the database (stop increasing the number showed in
status 0), but start indexing the "500 Not Indexed Yet site" as showed in the
above example (start increasing the number of status 200, and make it much
faster than before)?
- - - - - - - - - - - - - - - - - - - - - - - - - - - -
Read the full topic here:
http://www.dataparksearch.org/cgi-bin/simpleforum.cgi?fid=02;post=
