forgot a 'nip' there, since the item is outside the quotation. should be:
: safer-http-request ( request -- response/f data/f )
[ http-request ] [
nip dup download-failed? [ response>> f ] [ drop f f ] if
] recover ;
examples:
"http://www.google.com"; <get-request> safer-http-request ! =>
response data
"http://www.googlez.com"; <get-request> safer-http-request ! => f f
"http://www.google.com/234"; <get-request> safer-http-request ! =>
404-response f
also, this opens the way for split downloaders, which are generally
faster in network i/o.
ERROR: max<parts ;
: allot-ranges ( max #parts -- pairs )
2dup< [ max<parts ] when
dupd [ /i ] keep iota [ * ] with map
swap suffix dup rest [ 1 - ] map zip ;
: allot-ranges-by-size ( max est-chunk-size -- pairs )
dupd /i allot-ranges ;
: download-in-parts ( url n -- parts )
[ dup http-size ] dip allot-ranges
[ first2 grab-http-chunk nip ] with parallel-map ;
in timing a url download (10 times benchmark) , 'download-in-parts' with
5 parts took about 0.5 the time it took 'download'.
a proper implementation should use streams and append to disk every x bytes.
okay enough spamming this list with code snippets :)
signing off, kobi
On 28/08/2010 14:16, Kobi Lurie wrote:
> hmm,
> some things can fail. 404, bad domain ..
> try to get a response if possible.
>
> : safer-http-request ( request -- response/f data/f )
> [ http-request ] [
> dup download-failed? [ response>> f ] [ drop f f ] if
> ] recover ;
>
> ------------------------------------------------------------------------------
> Sell apps to millions through the Intel(R) Atom(Tm) Developer Program
> Be part of this innovative community and reach millions of netbook users
> worldwide. Take advantage of special opportunities to increase revenue and
> speed time-to-market. Join now, and jumpstart your future.
> http://p.sf.net/sfu/intel-atom-d2d
> _______________________________________________
> Factor-talk mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/factor-talk
>
--
China: stop persecuting Falun Gong!
http://faluninfo.net
URGENT: Innocent people are being persecuted for their belief inside Communist
China.
------------------------------------------------------------------------------
Sell apps to millions through the Intel(R) Atom(Tm) Developer Program
Be part of this innovative community and reach millions of netbook users
worldwide. Take advantage of special opportunities to increase revenue and
speed time-to-market. Join now, and jumpstart your future.
http://p.sf.net/sfu/intel-atom-d2d
_______________________________________________
Factor-talk mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/factor-talk
