Re: Wget and HTTP-redirects (302)

2006-02-22 Thread Hrvoje Niksic
Jasmo Hiltula [EMAIL PROTECTED] writes:

 I tried today a easy hax for one problem. It lead me to a situation
 where wget should be able to follow more than 20
 http-redirections. Now wget says 20 redirections exceeded. I
 searched the manual even read the source code trying to find some
 way to alter that number (20) but no luck.

In retr.c:

#define MAX_REDIRECTIONS 20

Increase the size to whatever you need.  However, are you sure that
you really need more than twenty redirections?  Is it possible that
Wget is in fact stuck in a redirection loop?

How many redirections would be enough for what you are diong?


Re: Wget and HTTP-redirects (302)

2006-02-22 Thread Mauro Tortonesi

Jasmo Hiltula wrote:


Hi

I tried today a easy hax for one problem. It lead me to a situation 
where  wget should be able to follow more than 20 http-redirections. 
Now wget  says 20 redirections exceeded. I searched the manual even 
read the  source code trying to find some way to alter that number 
(20) but no luck.


Can i do it? How can i do it?


hi jasmo,

the 20 redirections limit was hardcoded in wget some time ago to prevent 
infinite loops. at the moment the only way to bypass it, is to manually 
modify retr.c to change the MAX_REDIRECTIONS parameter.


a couple of months ago someone actually suggested to add a new 
--max-redirections option to wget, but at that time the proposal was 
rejected as it was considered too low-level for wget.


--
Aequam memento rebus in arduis servare mentem...

Mauro Tortonesi  http://www.tortonesi.com

University of Ferrara - Dept. of Eng.http://www.ing.unife.it
GNU Wget - HTTP/FTP file retrieval tool  http://www.gnu.org/software/wget
Deep Space 6 - IPv6 for Linuxhttp://www.deepspace6.net
Ferrara Linux User Group http://www.ferrara.linux.it



Re: Wget and HTTP-redirects (302)

2006-02-22 Thread Keith Moore
Mauro Tortonesi wrote:

 a couple of months ago someone actually suggested to add a new
 --max-redirections option to wget, but at that time the proposal was
 rejected as it was considered too low-level for wget.

That was me. I can easily resurrect that patch (against the
current sources) if anyone is interested.

KM



Re: Wget and HTTP-redirects (302)

2006-02-22 Thread Jasmo Hiltula
On Wed, 22 Feb 2006 18:43:23 +0200, Hrvoje Niksic [EMAIL PROTECTED]  
wrote:



Jasmo Hiltula [EMAIL PROTECTED] writes:


I tried today a easy hax for one problem. It lead me to a situation
where wget should be able to follow more than 20
http-redirections. Now wget says 20 redirections exceeded. I
searched the manual even read the source code trying to find some
way to alter that number (20) but no luck.


In retr.c:

#define MAX_REDIRECTIONS 20

Increase the size to whatever you need.  However, are you sure that
you really need more than twenty redirections?  Is it possible that
Wget is in fact stuck in a redirection loop?

How many redirections would be enough for what you are diong?


Okay, that info suits me very well. I can compile my own wget with bigger  
number of redirects allowed if i need to. I'm doing something what one  
might consider as stupid - a database cleaning system in PHP (there's  
reason for that :) ) and database is so big that i've to refresh page  
constantly to avoid php timeout. That means my page refreshes itself about  
25-30 times until the work is done.


I was a bit surprised when i was not allowed to change that option.  
Usually this kind of numbers are never hard coded ;) Nothing bad happened  
though, maybe i'll modify my code or compile new wget.


Thank you very much for information.
 -jasmo

--
Jasmo Hiltula