signoff robots
----- Original Message ----- From: "Nick Arnett" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Friday, March 15, 2002 3:48 PM Subject: [Robots] LWP (was RE: Re: better language for writing a Spider ?) > > > > > -----Original Message----- > > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On > > Behalf Of Sean M. Burke > > ... > > > At 10:36 2002-03-14 -0800, Nick Arnett wrote: > > >[with Python] I'm not seeing long, mysterious time-outs as I > > occasionally > > >did with LWP. > > > > I have never run into this problem, but I have a dim memory that > > you may be > > alluding to what is a known bug not with LWP, but with old versions (long > > since fixed in modern Perls and/or CPAN) of the socket libraries in IO::*. > > I'm very diligent about updating, so I doubt if I was seeing an old bug. > What I would see would be a series of time-outs, usually no more than 10 in > a row (I limited re-tries to 10, with increasing delays between them in case > it was a server busy issue). But I should make it clear that the bug > producing the error message out of expat was completely separate. > > > >Following an LWP request through the debugger is a long and convoluted > > >journey... > > > > Are you referring to perl -d, or LWP::Debug? > > Sorry for not specifying. I was using the ActiveState graphical debugger on > Windows, although sometimes the code was actually running on Linux. Same > behavior on both, though. I did give LWP::Debug a shot, but still could see > where the error code was getting introduced. Wish I could recall better > specifics, but it's been a few weeks. As I recall, the server was returning > an error, suggesting that there was something malformed about the request I > sent it, and that error was being mistranslated in the expat DLL... and I > recall having trouble even figuring out where expat got involved in the > mess. > > > Maybe I should write an addendum to "lwpcook.pod" on figuring out what's > > going wrong, when something does go wrong. The current lwpcook really > > needs an overhaul, and once my book /Perl & LWP/ is done (hopefully it'll > > be in press within a few weeks), I hope to send up some big doc > > patches to > > LWP, at the very least revamping lwpcook and then going into each > > class and > > noting in the docs whether a typical user needs to bother knowing about > > it. (E.g., you need to know about HTTP::Response; you do /not/ need to > > know about LWP::Protocol.) > > That would really be good. Examples, examples, examples. I learn by doing, > not by reading, and I think there are a fair number of people like me out > there. > > > In short, if people want to see improvements to LWP, email me and > > say what > > you want done, and I'll either try my hand at implementing it, or > > I'll pass > > it on to someone more capable. > > A re-try mechanism would be terrific. Mine is fairly straightforward. The > parameters are a max number of tries, a delay factor that optionally rises > with each try, and a logging method that details as much as possible about > each failure. The latter is where some work on the internals would be > helpful, to disambiguate error messages as much as possible. Perhaps a > simple way to kick in LWP::Debug with appropriate parameters and log the > results if repeated failures occur? I always want to see exactly what the > outgoing request was and the server's actual response, so I know whether the > request is munged or the server is being difficult... not that that's always > clear. > > Nick > > > -- > This message was sent by the Internet robots and spiders discussion list ([EMAIL PROTECTED]). For list server commands, send "help" in the body of a message to "[EMAIL PROTECTED]". > -- This message was sent by the Internet robots and spiders discussion list ([EMAIL PROTECTED]). For list server commands, send "help" in the body of a message to "[EMAIL PROTECTED]".
