Re: client side http
At 7:55 PM -0400 9/11/00, andu wrote: How to handle http proxies? Is this just a question of opening the socket to the proxy and carrying on as before, or is it, as I fear, more torturous? Yes, take a look at my ftp client (http://www.cloud9.net/~undo) which handles proxy, it should be the same. It also shows you how I handled the progress bar and others. Thanks Andu. I already have your ftp stack. A case of stupidity made me forget that it implemented a progress bar. Still, hacking around in the dark with an rfc document and a lot of coffee was interesting (I won't say enjoyable. :)) Please keep up the good work. Cheers Dave Cragg -- _ The LACS Centre (Business English Training Resources) mailto:[EMAIL PROTECTED] http://www.lacscentre.co.uk _ Archives: http://www.mail-archive.com/metacard%40lists.best.com/ Info: http://www.xworlds.com/metacard/mailinglist.htm Please send bug reports to [EMAIL PROTECTED], not this list.
Re: client side http
On Mon, 11 Sep 2000, Dave Cragg wrote: I've been playing with the sockets features in order to try and implement some client side http. I've been doing this because Metacard's built-in http support won't work with some URLs (something to do with the way aliasing is set up on the server, I think). Please try 2.3.2, which is a lot better about these things. It has just been released to beta test and now available for Linux, MacOS, and Win32 in the directory ftp://ftp.metacard.com/MetaCard/2.3.2/ The script below is a crude first attempt at this. It simply loads the data from the specified URL. (I've removed some stuff that was specific to my situation.) So far, it seems to work, even with previously unreachable URLs. Ideally, I'd like to be able to emulate the built-in http features of Metacard, but using sockets. (I've also had a go at the POST method, and it seems to work, but I need to make a more general solution than what I've done so far.) We're actually working on the same thing, as the plan all along was to replace the built-in HTTP routines with an script library that uses sockets to do HTTP, FTP, and other protocols. We welcome your participation in the design, implementation, and testing of this new library. 2.3.2 already has one new feature that will make implementing such a library much easier: a "wait for messages" command which allows event dispatching within a handler. This is useful for building synchronous function calls that still require some event dispatching (e.g., one that does the equivalent of "get url http://somehost/somefile"). The scripts may be useful to anyone else wanting to do the same thing. More importantly, I'd appreciate any feedback from the sockets gurus about whether I'm going in the right direction. Two questions foremost on my mind: In order to show progress of the download (progress bar or whatever), is it necessary to do reads in short steps (as in the script below) or is there another way? The best way to do it is to do a simple "read from socket s with message m" and handle the assembly and processing of the data in the message handler. This does the chunking automatically for you in whatever size chunks the OS is dealing in and so is the most efficient method. How to handle http proxies? Is this just a question of opening the socket to the proxy and carrying on as before, or is it, as I fear, more torturous? There are (at least) two types of proxies: those that you know the address of and those where the proxy is actually specified as a URL of a document which contains the proxy information. MetaCard's built-in HTTP support supports the former, but implementing support for the latter is one of the things that the new script-based library will do. Regards, Scott Any comments appreciated. Cheers Dave Cragg _ The LACS Centre (Business English Training Resources) mailto:[EMAIL PROTECTED] http://www.lacscentre.co.uk _ Scott Raney [EMAIL PROTECTED] http://www.metacard.com MetaCard: You know, there's an easier way to do that... Archives: http://www.mail-archive.com/metacard%40lists.best.com/ Info: http://www.xworlds.com/metacard/mailinglist.htm Please send bug reports to [EMAIL PROTECTED], not this list.
Re: client side http
At 12:02 PM -0600 9/11/00, Scott Raney wrote: On Mon, 11 Sep 2000, Dave Cragg wrote: I've been playing with the sockets features in order to try and implement some client side http. I've been doing this because Metacard's built-in http support won't work with some URLs (something to do with the way aliasing is set up on the server, I think). Please try 2.3.2, which is a lot better about these things. It has just been released to beta test and now available for Linux, MacOS, and Win32 in the directory ftp://ftp.metacard.com/MetaCard/2.3.2/ I just tried it, and you're right. It now works with previously unreachable URLs. Great stuff. Thanks too for the other information. Cheers Dave Cragg _ The LACS Centre (Business English Training Resources) mailto:[EMAIL PROTECTED] http://www.lacscentre.co.uk _ Archives: http://www.mail-archive.com/metacard%40lists.best.com/ Info: http://www.xworlds.com/metacard/mailinglist.htm Please send bug reports to [EMAIL PROTECTED], not this list.
Re: client side http
Hi
I've been playing with the sockets features in order to try and
implement some client side http. I've been doing this because
Metacard's built-in http support won't work with some URLs (something
to do with the way aliasing is set up on the server, I think).
The script below is a crude first attempt at this. It simply loads
the data from the specified URL. (I've removed some stuff that was
specific to my situation.) So far, it seems to work, even with
previously unreachable URLs. Ideally, I'd like to be able to emulate
the built-in http features of Metacard, but using sockets. (I've also
had a go at the POST method, and it seems to work, but I need to make
a more general solution than what I've done so far.)
The scripts may be useful to anyone else wanting to do the same
thing. More importantly, I'd appreciate any feedback from the sockets
gurus about whether I'm going in the right direction.
Two questions foremost on my mind:
In order to show progress of the download (progress bar or whatever),
is it necessary to do reads in short steps (as in the script below)
or is there another way?
That's the way but when you get data it comes in chunks anyway.
Use repeat only if you have to.
How to handle http proxies? Is this just a question of opening the
socket to the proxy and carrying on as before, or is it, as I fear,
more torturous?
Yes, take a look at my ftp client (http://www.cloud9.net/~undo) which handles proxy,
it should be the same.
It also shows you how I handled the progress bar and others.
Any comments appreciated.
Cheers
Dave Cragg
---
local lcSocket, lcPath, lcData, lcHeader
on httpLoad pURL
## remove initial "http://" if present
if "http://" is char 1 to 7 of pURL then
delete char 1 to 7 of pURL
end if
put offset("/",pURL) into tOff
if tOff = 0 then
put pURL into lcSocket
put "" into lcPath
else
put char 1 to (tOff - 1) of pURL into lcSocket
put char tOff to -1 of pURL into lcPath
end if
put numToChar(13) numToChar(10) into CRLF
CRLF is built in you don't need to do the above.
open socket to lcSocket
put "GET" lcPath"HTTP/1.1" CRLF into tString
put "host:" lcSocket CRLF after tString
put CRLF after tString ## blank line is necessary
Put an empty space between header item and value = host: host.com
write tString to socket lcSocket
read from socket lcSocket until CRLF ##line 1
if word 2 of it = 200 then
read from socket lcSocket until CRLF CRLF ##get the header
put it into lcHeader
get lineOffset("Content-Length", lcHeader)
if it 0 then
put it into tLineNum
put word 2 line tLineNum of lcHeader into tLength
put 1024 into tStep
end if
put empty into lcData
put 0 into tCount
put tLength div tStep into tTimes
## read in small steps in order to show a progress bar or other indication
repeat for tTimes
add 1 to tCount
read from socket lcSocket for tStep
put it after lcData
## handle progress bar updating here
end repeat
read from socket lcSocket until empty ## catch last few dribbles
put it after lcData
close socket lcSocket
## do something with lcData here (cache it, return it, or whatever)
else
## crude error handling
put word 2 of it
close socket lcSocket
end if
end httpLoad
---
on socketError pSocket, pErr
##crude error handling
put pSocket ":" pErr
end socketError
--
on socketTimeout pSocket
##crude error handling
put "Timed out"
close socket pSocket
end socketTimeout
--
_
The LACS Centre (Business English Training Resources)
mailto:[EMAIL PROTECTED]
http://www.lacscentre.co.uk
_
Regards, Andu
___
[EMAIL PROTECTED]
Archives: http://www.mail-archive.com/metacard%40lists.best.com/
Info: http://www.xworlds.com/metacard/mailinglist.htm
Please send bug reports to [EMAIL PROTECTED], not this list.
