yep,
Actually some tools help to fill common / well identify forms fields
(like email, phone, etc...)...
So it's totally reasonable to think about a robot with some "how to fill
form" rules.
And could be valuable in some cases.
++
On 04/27/2011 03:32 PM, [email protected] wrote:
Hello Fuad,
I can agree that robots should not perform POST requests, but Droids
is a library for building robots, it is not a robot itself.
My suggestion to extends the request types was to widen the scope
where Droids could be useful.
I would find that an unfortunate decision, but that's just my point of view.
Thanks,
Giulio Cesare
On Wed, Apr 27, 2011 at 1:55 PM, Fuad Efendi<[email protected]> wrote:
Hi Giulio, "other request types" - HEAD only, and only if Internet Server supports that (I don't
think more than 1% support, and I don't think it will be performance improvement in an era of sitemaps)...
there is no "Link" request types, and "POST" shouldn't be done by robots - otherwise
which URL will you show in a search results page? You can't show form submission via POST as a search results
-Fuad
-----Original Message-----
From: [email protected] [mailto:[email protected]]
Sent: April-27-11 7:33 AM
To: [email protected]
Subject: About Link and POST requests
Hello,
using Droid library I came to the realization that the current API are not
suitable for handling anything other than GET requests.
This is due mainly to two choices made in the code:
- using the Link interface for collecting and processing pages to process (all
based on simple URI values, both for 'getURI' and 'getTo');
- content being loaded by CrawlingWorker, passing just the URI to the Protocol.
Is there anyone interested in thinking how to change the API to support also
other request types?
Regards,
Giulio Cesare
