Send Beginners mailing list submissions to
[email protected]
To subscribe or unsubscribe via the World Wide Web, visit
http://www.haskell.org/mailman/listinfo/beginners
or, via email, send a message with subject or body 'help' to
[email protected]
You can reach the person managing the list at
[email protected]
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Beginners digest..."
Today's Topics:
1. Download a large file using Network.HTTP (Cedric Fung)
2. Re: Download a large file using Network.HTTP
(Felipe Almeida Lessa)
3. Re: Download a large file using Network.HTTP (Michael Snoyman)
4. Re: Download a large file using Network.HTTP
(Felipe Almeida Lessa)
5. Re: Download a large file using Network.HTTP (Cedric Fung)
6. Re: Download a large file using Network.HTTP (Cedric Fung)
----------------------------------------------------------------------
Message: 1
Date: Mon, 10 Dec 2012 21:34:16 +0800
From: Cedric Fung <[email protected]>
Subject: [Haskell-beginners] Download a large file using Network.HTTP
To: [email protected]
Message-ID:
<cadrduwrrdfwur37s0avon6no7fpifop97kppyst3oypoe6p...@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"
Hi,
Are there any suggestions to download a large file with Haskell? I have
read the docs for Network, Network.HTTP and Network.HTTP.Conduit, but can't
find anything which fit my requirements.
I want to download a large file from an HTTP URL, and show the progress
instantly. Maybe some functions which read HTTP connection and return a
lazy ByteString could do this work?
Though I found a low-level socket lazy package, which seems to work, I just
want a more high level API.
Thanks and regards.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.haskell.org/pipermail/beginners/attachments/20121210/817c5cb6/attachment-0001.htm>
------------------------------
Message: 2
Date: Mon, 10 Dec 2012 11:45:56 -0200
From: Felipe Almeida Lessa <[email protected]>
Subject: Re: [Haskell-beginners] Download a large file using
Network.HTTP
To: Cedric Fung <[email protected]>
Cc: beginners <[email protected]>
Message-ID:
<CANd=OGFGz5CcB5XY9owM73wwmTRZTthU4Wf9krY4BEv7G9=w...@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Hey!
It's probably going to be easier to use http-conduit. As the docs of
httpLbs say,
If you want more power, such as interleaved actions on the
response body during download, you'll need to use 'http' directly.
So you'll need to use the 'http' function. On the response headers
you'll find the total size of the download. With that size in hand,
you may implement a Conduit (from the conduit package)
type TotalSize = Int
findTotalSize :: ResponseHeaders -> Maybe TotalSize
showProgress :: MonadIO m => Maybe TotalSize -> Conduit ByteString
m ByteString
All data will pass through this conduit, so it may just keep track of
how many bytes it has seen already and show the progress while
returning the data itself unaltered. Then you'll just need a Sink
that saves the file (e.g. sinkFile from Data.Conduit.Binary) and to
connect everything, e.g.
res <- http request manager
let mtotalSize = findTotalSize (responseHeaders res)
responseBody res $= showProgress mtotalSize $$ sinkFile "output"
Cheers,
On Mon, Dec 10, 2012 at 11:34 AM, Cedric Fung <[email protected]> wrote:
> Hi,
>
> Are there any suggestions to download a large file with Haskell? I have read
> the docs for Network, Network.HTTP and Network.HTTP.Conduit, but can't find
> anything which fit my requirements.
>
> I want to download a large file from an HTTP URL, and show the progress
> instantly. Maybe some functions which read HTTP connection and return a lazy
> ByteString could do this work?
>
> Though I found a low-level socket lazy package, which seems to work, I just
> want a more high level API.
>
> Thanks and regards.
>
> _______________________________________________
> Beginners mailing list
> [email protected]
> http://www.haskell.org/mailman/listinfo/beginners
>
--
Felipe.
------------------------------
Message: 3
Date: Mon, 10 Dec 2012 15:46:41 +0200
From: Michael Snoyman <[email protected]>
Subject: Re: [Haskell-beginners] Download a large file using
Network.HTTP
To: Cedric Fung <[email protected]>
Cc: beginners <[email protected]>
Message-ID:
<caka2jgjdhe9ypjjwlm6ego+obsx50ewvnx3ev3tz6kccmgt...@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"
Here's an example of printing the total number of bytes consumed using
http-conduit:
import Control.Monad.IO.Class (liftIO)
import qualified Data.ByteString as S
import Data.Conduit
import Data.Conduit.Binary as CB
import Network.HTTP.Conduit
main :: IO ()
main = withManager $ \manager -> do
req <- parseUrl "http://www.yesodweb.com/";
res <- http req manager
responseBody res $$+- printProgress =$ CB.sinkFile "yesodweb.html"
printProgress :: Conduit S.ByteString (ResourceT IO) S.ByteString
printProgress =
loop 0
where
loop len = await >>= maybe (return ()) (\bs -> do
let len' = len + S.length bs
liftIO $ putStrLn $ "Bytes consumed: " ++ show len'
yield bs
loop len')
HTH,
Michael
On Mon, Dec 10, 2012 at 3:34 PM, Cedric Fung <[email protected]> wrote:
> Hi,
>
> Are there any suggestions to download a large file with Haskell? I have
> read the docs for Network, Network.HTTP and Network.HTTP.Conduit, but can't
> find anything which fit my requirements.
>
> I want to download a large file from an HTTP URL, and show the progress
> instantly. Maybe some functions which read HTTP connection and return a
> lazy ByteString could do this work?
>
> Though I found a low-level socket lazy package, which seems to work, I
> just want a more high level API.
>
> Thanks and regards.
>
> _______________________________________________
> Beginners mailing list
> [email protected]
> http://www.haskell.org/mailman/listinfo/beginners
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.haskell.org/pipermail/beginners/attachments/20121210/70d06390/attachment-0001.htm>
------------------------------
Message: 4
Date: Mon, 10 Dec 2012 11:50:02 -0200
From: Felipe Almeida Lessa <[email protected]>
Subject: Re: [Haskell-beginners] Download a large file using
Network.HTTP
To: Michael Snoyman <[email protected]>
Cc: beginners <[email protected]>
Message-ID:
<CANd=ogg027paw671ymokoujomuvtsej6kjmrp48jjkybqdc...@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
And here's the guy who's http-conduit's maintainer =). The only thing
I said that he didn't is that you may take the total size from the
response headers, but you may do this over his code.
Cheers,
On Mon, Dec 10, 2012 at 11:46 AM, Michael Snoyman <[email protected]> wrote:
> Here's an example of printing the total number of bytes consumed using
> http-conduit:
>
> import Control.Monad.IO.Class (liftIO)
> import qualified Data.ByteString as S
> import Data.Conduit
> import Data.Conduit.Binary as CB
> import Network.HTTP.Conduit
>
> main :: IO ()
> main = withManager $ \manager -> do
> req <- parseUrl "http://www.yesodweb.com/";
> res <- http req manager
> responseBody res $$+- printProgress =$ CB.sinkFile "yesodweb.html"
>
> printProgress :: Conduit S.ByteString (ResourceT IO) S.ByteString
> printProgress =
> loop 0
> where
> loop len = await >>= maybe (return ()) (\bs -> do
> let len' = len + S.length bs
> liftIO $ putStrLn $ "Bytes consumed: " ++ show len'
> yield bs
> loop len')
>
>
> HTH,
> Michael
>
>
> On Mon, Dec 10, 2012 at 3:34 PM, Cedric Fung <[email protected]> wrote:
>>
>> Hi,
>>
>> Are there any suggestions to download a large file with Haskell? I have
>> read the docs for Network, Network.HTTP and Network.HTTP.Conduit, but can't
>> find anything which fit my requirements.
>>
>> I want to download a large file from an HTTP URL, and show the progress
>> instantly. Maybe some functions which read HTTP connection and return a lazy
>> ByteString could do this work?
>>
>> Though I found a low-level socket lazy package, which seems to work, I
>> just want a more high level API.
>>
>> Thanks and regards.
>>
>> _______________________________________________
>> Beginners mailing list
>> [email protected]
>> http://www.haskell.org/mailman/listinfo/beginners
>>
>
>
> _______________________________________________
> Beginners mailing list
> [email protected]
> http://www.haskell.org/mailman/listinfo/beginners
>
--
Felipe.
------------------------------
Message: 5
Date: Mon, 10 Dec 2012 21:51:43 +0800
From: Cedric Fung <[email protected]>
Subject: Re: [Haskell-beginners] Download a large file using
Network.HTTP
To: Michael Snoyman <[email protected]>
Cc: beginners <[email protected]>
Message-ID:
<CADrDuWqU5gNefpSaZECQT7EiA_=tj2wp7s+r1eomchyhw4t...@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"
Wow, thanks to Felipe's and Michael's quick response. Haskell community is
so kind.
I will try your suggestions right now!
On Mon, Dec 10, 2012 at 9:46 PM, Michael Snoyman <[email protected]>wrote:
> Here's an example of printing the total number of bytes consumed using
> http-conduit:
>
> import Control.Monad.IO.Class (liftIO)
> import qualified Data.ByteString as S
> import Data.Conduit
> import Data.Conduit.Binary as CB
> import Network.HTTP.Conduit
>
> main :: IO ()
> main = withManager $ \manager -> do
> req <- parseUrl "http://www.yesodweb.com/";
> res <- http req manager
> responseBody res $$+- printProgress =$ CB.sinkFile "yesodweb.html"
>
> printProgress :: Conduit S.ByteString (ResourceT IO) S.ByteString
> printProgress =
> loop 0
> where
> loop len = await >>= maybe (return ()) (\bs -> do
> let len' = len + S.length bs
> liftIO $ putStrLn $ "Bytes consumed: " ++ show len'
> yield bs
> loop len')
>
>
> HTH,
> Michael
>
>
> On Mon, Dec 10, 2012 at 3:34 PM, Cedric Fung <[email protected]> wrote:
>
>> Hi,
>>
>> Are there any suggestions to download a large file with Haskell? I have
>> read the docs for Network, Network.HTTP and Network.HTTP.Conduit, but can't
>> find anything which fit my requirements.
>>
>> I want to download a large file from an HTTP URL, and show the progress
>> instantly. Maybe some functions which read HTTP connection and return a
>> lazy ByteString could do this work?
>>
>> Though I found a low-level socket lazy package, which seems to work, I
>> just want a more high level API.
>>
>> Thanks and regards.
>>
>> _______________________________________________
>> Beginners mailing list
>> [email protected]
>> http://www.haskell.org/mailman/listinfo/beginners
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.haskell.org/pipermail/beginners/attachments/20121210/cc08ee7c/attachment-0001.htm>
------------------------------
Message: 6
Date: Mon, 10 Dec 2012 21:56:20 +0800
From: Cedric Fung <[email protected]>
Subject: Re: [Haskell-beginners] Download a large file using
Network.HTTP
To: Felipe Almeida Lessa <[email protected]>
Cc: beginners <[email protected]>
Message-ID:
<CADrDuWpZwqbf3YjgTi4O_vN-R1Yn295s637C0Eo=-egfyem...@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"
It's so amazing to receive your help. I have just learnt Haskell for about
1 week, wish I could handle it soon. The Haskell community is small (at
least smaller than the Ruby's) but full of kindly people.
On Mon, Dec 10, 2012 at 9:50 PM, Felipe Almeida Lessa <
[email protected]> wrote:
> And here's the guy who's http-conduit's maintainer =). The only thing
> I said that he didn't is that you may take the total size from the
> response headers, but you may do this over his code.
>
> Cheers,
>
> On Mon, Dec 10, 2012 at 11:46 AM, Michael Snoyman <[email protected]>
> wrote:
> > Here's an example of printing the total number of bytes consumed using
> > http-conduit:
> >
> > import Control.Monad.IO.Class (liftIO)
> > import qualified Data.ByteString as S
> > import Data.Conduit
> > import Data.Conduit.Binary as CB
> > import Network.HTTP.Conduit
> >
> > main :: IO ()
> > main = withManager $ \manager -> do
> > req <- parseUrl "http://www.yesodweb.com/";
> > res <- http req manager
> > responseBody res $$+- printProgress =$ CB.sinkFile "yesodweb.html"
> >
> > printProgress :: Conduit S.ByteString (ResourceT IO) S.ByteString
> > printProgress =
> > loop 0
> > where
> > loop len = await >>= maybe (return ()) (\bs -> do
> > let len' = len + S.length bs
> > liftIO $ putStrLn $ "Bytes consumed: " ++ show len'
> > yield bs
> > loop len')
> >
> >
> > HTH,
> > Michael
> >
> >
> > On Mon, Dec 10, 2012 at 3:34 PM, Cedric Fung <[email protected]> wrote:
> >>
> >> Hi,
> >>
> >> Are there any suggestions to download a large file with Haskell? I have
> >> read the docs for Network, Network.HTTP and Network.HTTP.Conduit, but
> can't
> >> find anything which fit my requirements.
> >>
> >> I want to download a large file from an HTTP URL, and show the progress
> >> instantly. Maybe some functions which read HTTP connection and return a
> lazy
> >> ByteString could do this work?
> >>
> >> Though I found a low-level socket lazy package, which seems to work, I
> >> just want a more high level API.
> >>
> >> Thanks and regards.
> >>
> >> _______________________________________________
> >> Beginners mailing list
> >> [email protected]
> >> http://www.haskell.org/mailman/listinfo/beginners
> >>
> >
> >
> > _______________________________________________
> > Beginners mailing list
> > [email protected]
> > http://www.haskell.org/mailman/listinfo/beginners
> >
>
>
>
> --
> Felipe.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.haskell.org/pipermail/beginners/attachments/20121210/d669902b/attachment.htm>
------------------------------
_______________________________________________
Beginners mailing list
[email protected]
http://www.haskell.org/mailman/listinfo/beginners
End of Beginners Digest, Vol 54, Issue 15
*****************************************
