Problem with -E option with Spider.py
I am having problems getting the -E (exclusion list) option on the command-line working with Spider.py. Although I provide the exclusion file, Spider.py does not appear to add it to the default exclusion list (exclusion.txt) before grabbing data. Using the -V 2 option when starting, I can see a message saying that exclusion.txt is being used but not the one I provided via the -E option. I am running Spider.py directly under Python 2.1 in the bash shell under Cygwin. The version of Spider.py I have is $Id: Spider.py,v 1.37 ... $ which came with Plucker V1.1SR1.exe for Windows. P.S. Could a --version option be put into Spider.py? Regards, Kam-Yung -- -- Soh Kam Yung email: [EMAIL PROTECTED] homepage: http://www.geocities.com/firstspeaker.geo/index.html __ Get your own FREE, personal Netscape Webmail account today at http://webmail.netscape.com/
Re: Problem with -E option with Spider.py
The version of Spider.py I have is $Id: Spider.py,v 1.37 ... $ which came with Plucker V1.1SR1.exe for Windows. P.S. Could a --version option be put into Spider.py? There was a minor update to the Windows package. SR2 is now available from the website (click the same link you did to get SR1 from the downloads page). The -E directive shouldn't "add" the exclusion to the existing exclusionlist.txt, it should simply override it's use. Let me test for a sec to see (running it as root so I can point to a user's exclusion file): plucker-build -E /home/hacker/.plucker/exclusionlist.txt -f /tmp/Bellairs -H "http://www.compleatbellairs.com/pigwidgeon/" --zlib-compression --bpp=4 --maxdepth=2 Working for pluckerdir /root/.plucker ZLib compression turned on Using exclusion list /root/.plucker/exclusionlist.txt Using exclusion list /root/.plucker/exclusionlist.txt Adding extra exclusion list /home/hacker/.plucker/exclusionlist.txt Processing http://www.compleatbellairs.com/pigwidgeon/. Seemed to work here. Are you sure you're putting it in the right place and that your path is right? Paths with spaces should be quoted, such as: -E "C:\Program Files\exclusionlist.txt" Also, make sure your filenames if they are longer than 8.3 use the Windows tilde notation: exclusionlist.txt would be: exclus~1.txt on Windows. Hope that helps. /d
