Re: [Haskell-cafe] Stack space overflow in HaskellNet

2011-07-27 Thread Manfred Lotz 
On Tue, 26 Jul 2011 14:17:08 -0700 (PDT)
Donn Cave d...@avvanta.com wrote:

 Quoth Manfred Lotz manfred.l...@arcor.de,
 ...
  I'm not quite sure I understand what you mean. Stack overflow comes
  from this:
  forM_ msgs (\x - fetch con x = print)
 
  If I change it to:
  mapM_  (\x - fetch con x = print) msgs
 
  there is the same stack overflow.
 
 I didn't understand that myself, but neither do I know what might
 be wrong.  One thing to consider is that email messages can be very
 large.  Looking at messages received in the last 10 days I see I
 have one that exceeds your reported stack size, and that isn't
 counting the extra space required for text representation of non
 printing characters etc.  There may be messages that you simply
 can't print.
 
 The HaskellNet IMAP fetch is actually FETCH ... BODY[], i.e.,
 the whole contents of the message.  Normal practice for giant data
 files is to send them as part of a MIME multipart/mixed message,
 and something like the above can proceed with a reasonable chance
 of success if it avoids these attachments by fetching BODY[1]
 (or BODY[1.1], etc. depending on actual structure.)  I just fetched
 the 10Mb message I mentioned above to check the structure, and it
 happened in the blink of an eye - BODY[1] is smaller than the header.
 
 I don't see any support for fetch by part, you might have to hack
 that up yourself.  You may ideally also want to fetch BODYSTRUCTURE,
 but practically I might go out on a limb and predict that you won't
 run into messages where the first part is a multipart/mixed with a
 large attachment - so if the object is just a survivable first part,
 you could live without BODYSTRUCTURE analysis and optimistically
 ask for BODY[1].
 
 Moving on to practical use of email via IMAP, you'd also want to
 be able to fetch and decode the attachments.  At this point, it's
 interesting to return to the question of space requirements.
 
   Donn

The problem seems to lie in the HaskellNet package. If for example I
only fetch a specific message 
   m - fetch con 2092
having a size of some 1.2m then I get the same stack overflow.

If at runtime I specify +RTS -K40M -RTS it works but takes over 40
seconds to fetch the message. 



-- 
Manfred



___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

Re: [Haskell-cafe] Stack space overflow in HaskellNet

2011-07-27 Thread Manfred Lotz 
On Wed, 27 Jul 2011 09:47:52 -0700 (PDT)
Donn Cave d...@avvanta.com wrote:

 Quoth Manfred Lotz manfred.l...@arcor.de,
 ...
  The problem seems to lie in the HaskellNet package. If for example I
  only fetch a specific message 
 m - fetch con 2092
  having a size of some 1.2m then I get the same stack overflow.
 
  If at runtime I specify +RTS -K40M -RTS it works but takes over 40
  seconds to fetch the message. 
 
 That's not so good, but I wouldn't be surprised if it's a natural
 parsing problem, I mean it's just a lot of data to run through a
 Haskell parser.
 

Yep, I agree. Perhaps the library should provide a fetchRaw function to
get the whole message without much parsing. Perhaps I could tell the
package author what the problem is, and he is happy to provide a
solution.


In the end the only thing I need is to get the full message because I
want to feed bogofilter to learn that a message is ham or spam.


For the time being I decided to write my own program to fetch the data
because it is a good exercise for a Haskell beginner as I am.



-- 
Manfred



___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

Re: [Haskell-cafe] Stack space overflow in HaskellNet

2011-07-26 Thread Manfred Lotz 
On Tue, 26 Jul 2011 10:17:22 +0200 (CEST)
Henning Thielemann lemm...@henning-thielemann.de wrote:

 
 On Mon, 25 Jul 2011, Manfred Lotz wrote:
 
  Hi there,
  If I take example imap.hs
 
 
  import System.IO
  import Network.HaskellNet.IMAP
  import Text.Mime
  import qualified Data.ByteString.Char8 as BS
  import Control.Monad
 
  -- the next lines were changed to fit to my local imap server
  imapServer = imap.mail.org
  user = 
  pass = 
 
  main = do
s   con - connectIMAP imapServer
   login con user pass
   mboxes - list con
   mapM print mboxes
 
 This should be mapM_ and 'ghc -Wall' spots this problem since 6.12.


The compiler (7.04) doesn't tell me anything about it.
 
   select con INBOX
   msgs - search con [ALLs]
   mapM_ (\x - print x) (take 4 msgs)
   forM_ (take 4msgs) (\x - fetch con x = print)

I'm not quite sure I understand what you mean. Stack overflow comes
from this:
forM_ msgs (\x - fetch con x = print)

If I change it to:
mapM_  (\x - fetch con x = print) msgs

there is the same stack overflow.



-- 
Thanks,
Manfred




___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

[Haskell-cafe] Stack space overflow in HaskellNet

2011-07-25 Thread Manfred Lotz 
Hi there,
If I take example imap.hs 


import System.IO
import Network.HaskellNet.IMAP
import Text.Mime
import qualified Data.ByteString.Char8 as BS
import Control.Monad

-- the next lines were changed to fit to my local imap server
imapServer = imap.mail.org
user = 
pass = 

main = do
  con - connectIMAP imapServer 
  login con user pass
  mboxes - list con
  mapM print mboxes
  select con INBOX
  msgs - search con [ALLs]
  mapM_ (\x - print x) (take 4 msgs)
  forM_ (take 4msgs) (\x - fetch con x = print)
 

and change the last line (in order to print all messages of the
mailbox) into:
   forM_ msgs (\x - fetch con x = print)

I get 
Stack space overflow: current size 8388608 bytes.
Use `+RTS -Ksize -RTS' to increase it.


Is this something to be fixed in the HaskellNet code, or in the example
code?



-- 
Manfred



___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

Re: [Haskell-cafe] timezone-series and timezone-olson

2011-03-27 Thread Manfred Lotz 
Hi Yitz,
Thanks for answering so quickly.

On Sun, 27 Mar 2011 14:16:50 +0200
Yitzchak Gale g...@sefer.org wrote:

 Hi Manfred,
  runhaskell caltest.hs
  03/27/11\n

 I am copying this response to the Haskell Cafe mailing list.
 
 Manfred Lotz wrote:
  ...I'm trying to figure out how to use your
  packages to get the time in a different timezone.
  Do you have an example how to do that? What I want for example is to
  provide the timezone preferably like this: US/Eastern, and get the
  time information like for example this:
 
  US/Eastern        DST  EDT   2011.03.27 05:20:52
 
  The pieces I like to get are: Summertime indicator, TZ abbrev and
  the date resp. time in that timezone.
 

...

 
 Assuming you have the timezone file that you need - let's say it
 is in the directory /usr/share/zoneinfo as typical for Linux and
 Mac - here is how to code it:
 
 do
   ...
   tzs - getTimeZoneSeriesFromOlsonFile
 /usr/share/zoneinfo/America/New_York let usEastern =
 utcToLocalTime' tzs utc
 
 Then, to format the time as you require, use the formatTime function
 from the time package.
 

I tried it like this:

--snip-

module Main where

import System.Environment
import System.Exit
import System.Locale
import Data.Time
import Data.Time.Clock
import Data.Time.LocalTime
import Data.Time.LocalTime.TimeZone.Olson 
import Data.Time.LocalTime.TimeZone.Series


path = /usr/share/zoneinfo/

showTime t = fmap (formatTime defaultTimeLocale %Z %z %D%n) t

  
getTime :: [Char] - IO LocalTime
getTime tz = do
  tzs - getTimeZoneSeriesFromOlsonFile (path ++ tz)
  utcnow - getCurrentTime
  let tzo = utcToLocalTime' tzs utcnow in return tzo


 
main = let t = getTime US/Eastern
   in showTime 
--snap-

Don't know if this is the proper way to do. In the output I do not get
the timezone information:

 runhaskell caltest.hs
  03/27/11\n

Don't know what I'm doing wrong.


-- 
Manfred



___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe