RE: [Robots] robot in python?

2003-11-26 Thread Sean M. Burke
At 11:47 PM 2003-11-17, SsolSsinclair wrote: Open Source is a project which came into being through a collective effort. Intelligence matching Intelligence. This movement cannot be stopped or prevented, SHORT of ceasing communication of all [resulting in Deaf Silence, and the Elimination of

Re: [Robots] robot in python?

2003-11-26 Thread Nick Arnett
Petter Karlström wrote: Hello all, Nice to see that this list woke up again! :^) And now the list owner finally woke up, too... I hadn't noticed the recent traffic on the list until just now. Are those messages about an address no longer in use going to the whole list? Aghh. I've taken care

Re: [Robots] robot in python?

2003-11-25 Thread Petter Karlström
Me again, Still wonder how to handle logins, though... In case anyone else is interested, and for the archives... John J. Lees ClientCookie module is excellent for handling cookies and other kinds of session data. Here: http://wwwsearch.sourceforge.net/ClientCookie/ cheers /petter

Re: [Robots] robot in python?

2003-11-25 Thread Eric Thompson
Anyone know the best method for simulating session with .net/C#? This maybe built into the framework I don't know. Any suggestions? -E Me again, Still wonder how to handle logins, though... In case anyone else is interested, and for the archives... John J. Lees ClientCookie module is

RE: [Robots] robot in python?

2003-11-25 Thread Erick Thompson
of a cookie container, and make sure that you use that container for every request. HTH Erick -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Eric Thompson Sent: Tuesday, November 25, 2003 12:35 PM To: [EMAIL PROTECTED] Subject: Re: [Robots] robot

RE: [Robots] robot in python?

2003-11-19 Thread SsolSsinclair
developing a crawler for a search engine with Python == You may find that the threads and exceptions in Python more than make up for anything you are missing in Perl. The Python libraries are not as extensive, but that is mostly because they have one of

RE: [Robots] robot in python?

2003-11-19 Thread SsolSsinclair
BUSH CALLS ON SENATE TO RATIFY CYBERCRIME TREATY President Bush has asked the US Senate to ratify the first international cybercrime treaty. Bush called the Council of Europe's controversial treaty an effective tool in the global effort to combat computer-related crime and the only multilateral

RE: [Robots] robot in python?

2003-11-18 Thread SsolSsinclair
from [EMAIL PROTECTED] wrote: I have written crawlers in Perl before, but I wish to try out Python for a hobby project. Has anybody here written a webbot i Python? Verity Ultraseek is a web crawler and search engine written in Python. Portions of it are C or C++ native modules. Ultraseek is a

Re: [Robots] robot in python?

2003-11-17 Thread Walter Underwood
--On Sunday, November 16, 2003 4:23 PM +0100 Petter Karlström [EMAIL PROTECTED] wrote: I have written crawlers in Perl before, but I wish to try out Python for a hobby project. Has anybody here written a webbot i Python? Verity Ultraseek is a web crawler and search engine written in Python.

Re: [Robots] robot in python?

2003-11-17 Thread Alexander Halavais
Walter Underwood wrote: Extracting links using a regular HTML parser works fine, and isn't that much work. One of the major issues in an HTML parser is dealing with all the illegal HTML on the web. It really depends on what you are looking for, and how tolerant of errors you are. For most of