Hello Fuad, I can agree that robots should not perform POST requests, but Droids is a library for building robots, it is not a robot itself. My suggestion to extends the request types was to widen the scope where Droids could be useful.
I would find that an unfortunate decision, but that's just my point of view. Thanks, Giulio Cesare On Wed, Apr 27, 2011 at 1:55 PM, Fuad Efendi <[email protected]> wrote: > Hi Giulio, "other request types" - HEAD only, and only if Internet Server > supports that (I don't think more than 1% support, and I don't think it will > be performance improvement in an era of sitemaps)... there is no "Link" > request types, and "POST" shouldn't be done by robots - otherwise which URL > will you show in a search results page? You can't show form submission via > POST as a search results > -Fuad > > -----Original Message----- > From: [email protected] [mailto:[email protected]] > Sent: April-27-11 7:33 AM > To: [email protected] > Subject: About Link and POST requests > > Hello, > > using Droid library I came to the realization that the current API are not > suitable for handling anything other than GET requests. > > This is due mainly to two choices made in the code: > - using the Link interface for collecting and processing pages to process > (all based on simple URI values, both for 'getURI' and 'getTo'); > - content being loaded by CrawlingWorker, passing just the URI to the > Protocol. > > Is there anyone interested in thinking how to change the API to support also > other request types? > > Regards, > > Giulio Cesare > >
