- - - - - - - - - - - - - - - - - - - - - - - - - - - -
Name: seirge
Subject: Re: robots.txt:
The same issue.
here is -v5 output
indexer[13496]: {00} indexer from dpsearch-4.48-mysql started with
search/4.48//etc/indexer.conf'
indexer[13496]: {00} Chinese dictionary with 0 entries
indexer[13496]: {00} Korean dictionary with 0 entries
indexer[13496]: {00} Thai dictionary with 0 entries
indexer[13496]: {00} LogsOnly: no
indexer[13496]: {00} mutexes used: 256
indexer[13496]: {01} DpsOpenCache:
indexer[13496]: {01} Done.
indexer[13496]: {01} Target.body: <NULL>
indexer[13496]: {01} Target.Content-Length: 22905
indexer[13496]: {01} Target.crc32: -300891917
indexer[13496]: {01} Target.DP_ID: 7989746
indexer[13496]: {01} Target.E_URL:
http://www.shopireland.ie/books/detail/000617454X/The-Cardinal-of-the-Kremlin/
indexer[13496]: {01} Target.Hops: 21
indexer[13496]: {01} Target.Pop_Rank: 0.25
indexer[13496]: {01} Target.PrevStatus: 200
indexer[13496]: {01} Target.Since: 1193462786
indexer[13496]: {01} Target.Status: 200
indexer[13496]: {01} Target.URL:
http://www.shopireland.ie/books/detail/000617454X/The-Cardinal-of-the-Kremlin/
indexer[13496]: {01} Target.URL_ID: 1910442282
indexer[13496]: {01} URL:
http://www.shopireland.ie/books/detail/000617454X/The-Cardinal-of-the-Kremlin/
indexer[13496]: {01} Realm Allow wild 'http://www.shopireland.ie/books/detail/*'
indexer[13496]: {01} Allow by default
indexer[13496]: {01} Resolver 0th checking for 70.85.56.41
indexer[13496]: {01} ROBOTS path: /, pathlen:1 URL:
/books/detail/000617454X/The-Cardinal-of-the-Kremlin/ cmd: Disallow
indexer[13496]: {01} Doc.robots.txt: 'Disallow /'
indexer[13496]: {01} Deleting
http://www.shopireland.ie/books/detail/000617454X/The-Cardinal-of-the-Kremlin/
indexer[13496]: {01} Done (17 seconds, 0 documents, 0 bytes, 0.00 Kbytes/sec.)
indexer[13496]: {00} Total 18 seconds, 0 documents, 0 bytes, 0.00 Kbytes/sec,
0.00 sec/doc, 0 bytes/doc.
indexer[13496]: {00} Neo PopRank: 0 documents, 0 pas, 0.00 Kpas/sec, 0.00
sec/doc, 0.00 pas/doc.
It is reindexing attempt. (after updating from 4.42 where robots.txt also
doesn't work but was removed from code by force).
Sites to index are taken from db, 'Robots no' is in indexer.conf as well as for
every domain in srvinfo table.
Thanks for support.
- - - - - - - - - - - - - - - - - - - - - - - - - - - -
Read the full topic here:
http://www.dataparksearch.org/cgi-bin/simpleforum.cgi?fid=02;topic_id=1185818418;page=2
