It seems like it's to our advantage to always do a HEAD call, unless it's an initial dig, where it is wastefull... and that the state of persitent_connections is irrelevant to this decision.
Let me try to understand. What you suggest is:
1) killing head_before_get
2) performing HEAD calls only in the incremental dig (either with or without persistent connections)
3) Unlinking the Head before Get mechanism from the persistent connections one
If it is so, it could be good for me (for number 1 I will do what you guys decide). I had not understood it from the earlier messages, sorry.
Even though - personally - I would not kill the attribute because:
1) It could be useful in cases when we don't know whether a document is parsable or not according to the *usual* means of exclusions (that is to say the URL). I know so far we take in consideration only the content-type but maybe in future release we could use other HTTP headers (i.e. cookies, language, etc.) and a pre-emptive head could save time in a initial dig as well.
2) I share the library with ht://Check which massively uses this option as it has to retrieve any document - images too - and a HEAD call could save a lot of time in the initial dig. I'd love to maintain the logic of the net library the more similar possible.
3) Killing the attribute would not avoid us to change the code in order to store information about the retrieval status in the Retriever and Document classes (unless we intend to use some classes variables - which I hate)
I don't have a problem keeping head_before_get, as long as we make the default TRUE.
That's the default.
Please let me know if the Retriever and Document classes changes make sense to you guys and I will modify the code.
Ciao ciao
-Gabriele
--
Gabriele Bartolini: Web Programmer, ht://Dig & IWA/HWG Member, ht://Check maintainer
Current Location: Melbourne, Victoria, Australia
[EMAIL PROTECTED] | http://www.prato.linux.it/~gbartolini | ICQ#129221447
> "Leave every hope, ye who enter!", Dante Alighieri, Divine Comedy, The Inferno
------------------------------------------------------- This SF.net email is sponsored by: SF.net Giveback Program. SourceForge.net hosts over 70,000 Open Source Projects. See the people who have HELPED US provide better services: Click here: http://sourceforge.net/supporters.php _______________________________________________ ht://Dig Developer mailing list: [EMAIL PROTECTED] List information (subscribe/unsubscribe, etc.) https://lists.sourceforge.net/lists/listinfo/htdig-dev
