On Tue, Sep 9, 2008 at 3:14 AM, Daniel Stenberg <[EMAIL PROTECTED]> wrote: > On Mon, 8 Sep 2008, Donald Allen wrote: > >> The page I get is what would be obtained if an un-logged-in user went to >> the specified url. Opening that same url in Firefox *does* correctly >> indicate that it is logged in as me and reflects my customizations. > > First, LiveHTTPHeaders is the Firefox plugin everyone who tries these stunts > need. Then you read the capure and replay them as closely as possible using > your tool. > > As you will find out, sites like this use all sorts of funny tricks to > figure out you and to make it hard to automate what you're trying to do. > They tend to use javascripts for redirects and for fiddling with cookies > just to make sure you have a javascript and cookie enabled browser. So you > need to work hard(er) when trying this with non-browsers. > > It's certainly still possible, even without using the browser to get the > first cookie file. But it may take some effort.
I have not been able to retrieve a page with wget as if I were logged in using --load-cookies and Micah's suggestion about 'Accept-Encoding' (there was a typo in his message -- it's 'Accept-Encoding', not 'Accept-Encodings'). I did install livehttpheaders and tried --no-cookies and --header <cookie info from livehttpheaders> and that did work. Some of the cookie info sent by Firefox was a mystery, because it's not in the cookie file. Perhaps that's the crucial difference -- I'm speculating that wget isn't sending quite the same thing as Firefox when --load-cookies is used, because Firefox is adding stuff that isn't in the cookie file. Just a guess. Is there a way to ask wget to print the headers it sends (ala livehttpheaders)? I've looked through the options on the man page and didn't see anything, though I might have missed it. > > -- > > / daniel.haxx.se >
