On Fri, October 27, 2006 17:07, Thomas Tschoellitsch wrote: > I would like to mirror a site that wants authentication on a .aspx > file using a form accepting a combination of username and password.
If you plan to be an official mirror, contact them and ask what kind of mechanism they use :-) > I've tried various things with the http options and I've tried > searching for hints on this but so far haven't come up with something > conclusive. Some websites use complicated mechanims, possibly combined with javascript and use of all kinds of cookies, even cookies inside the javascript. A way to find out what's going on is trough the use of a web debugging proxy like Charles ( http://www.xk72.com/charles/ ). With telnet www.somehost.com 80 or openssl s_client www.somehost.com:443 you can fiddle around to see what gets the job done. Here is a simple example I use on my Linux system: #!/bin/bash # getros.sh version 0.1 # # Shell script to download RosterInfo.pdf from www.specificwebsite.com # to the current directory: # # $ ./getros.sh username:password # # Bugs: Doesn't do any kind of imput checking. # Doesn't give any feedback on errors. # # Copyright (C) 2006 Remko Bolt <[EMAIL PROTECTED]> # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. wget --quiet --output-document=/dev/null --keep-session-cookies --save-cookies session-cookie www.specificwebsite.com wget --quiet --output-document=/dev/null --load-cookies session-cookie --pos t-data="hLoginCount=&hLoginId=&hBrowser=NETSCAPE&hMenuMode=N" www.specificwebsite.com/forBrowser.asp line1="POST /Asp-bin/login/authenticatenormal.asp HTTP/1.1\n" line2="Host: www.specificwebsite.com\n" cookie=`cat session-cookie` pat="#*K " # <-- last character is a tab cookie="${cookie##$pat}" line3="Cookie: ASPSESSIONIDQGQGGHIK="$cookie"\n" line4="Content-Type: application/x-www-form-urlencoded\n" date=`date +%Y%m%d%H%M%S` userid=${1%%:*} password=${1#*:} line7="hLoginCount=&hLoginId=&hMenuMode=N&hClientDate="$date"&dfUserID="$userid"&dfPassword="$password"&image1.x=0&image1.y=0" lenght=${#line7} line56="Content-Length: "$lenght"\n\n" { echo -e $line1$line2$line3$line4$line56$line7"\n\n" sleep 1 } | openssl s_client -bugs -connect www.specific.com:443 &>/dev/null wget --quiet --output-document=RosterInfo.pdf --timestamping --load-cookies session-cookie http://www.specificwebsite.com/asp-nin/Main/RosterInfo/RosterInfo.asp rm klc-session-cookie exit
