Re: [Haskell-cafe] Memory corruption issues when using newAlignedPinnedByteArray, GC kicking in?

2012-07-10 Thread Thomas Schilling
I think you should ask this question on the glasgow-haskell-users mailing list: http://www.haskell.org/mailman/listinfo/glasgow-haskell-users On 10 July 2012 18:20, Nicolas Trangez wrote: > All, > > While working on my vector-simd library, I noticed somehow memory I'm > using gets corrupted/overw

[Haskell-cafe] Memory corruption issues when using newAlignedPinnedByteArray, GC kicking in?

2012-07-10 Thread Nicolas Trangez
All, While working on my vector-simd library, I noticed somehow memory I'm using gets corrupted/overwritten. I reworked this into a test case, and would love to get some help on how to fix this. Previously I used some custom FFI calls to C to allocate aligned memory, which yields correct results,

[Haskell-cafe] Memory leak from close fields in conduit

2012-04-11 Thread Michael Snoyman
Hi all, Hideyuki Tanaka alerted me[1] to a memory leak in conduit. Long story short: it appears that Pipe composition leads to collection of a large number of `return ()` actions for unnecessary memory cleanup. We came up with a possible solution: a Finalize type[2]. In both of our testing, this e

Re: [Haskell-cafe] Memory usage with TVar?

2012-02-01 Thread Jake McArthur
First, see this question about space usage on Stack Overflow: http://stackoverflow.com/questions/3254758/memory-footprint-of-haskell-data-types Next, apply this knowledge not only to Ints, but also to tuples and lists. There's your memory usage. - Jake On Wed, Feb 1, 2012 at 10:29 AM, Johan Bri

Re: [Haskell-cafe] Memory usage with TVar?

2012-02-01 Thread Johan Brinch
On Wed, Feb 1, 2012 at 16:29, Johan Brinch wrote: > I'm on a 64-bit machine > where 2'000'000 integers uses 32 MB. 2 * 2'000'000 ;) -- Johan Brinch ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell

[Haskell-cafe] Memory usage with TVar?

2012-02-01 Thread Johan Brinch
Here's the example program: https://gist.github.com/1cbe113d2c79e2fc9d2b When I run the program (which maintains a list inside an STM TVar), I get the following statistics: > ./Test +RTS -s 176,041,728 bytes allocated in the heap 386,794,976 bytes copied during GC 69,180,224 bytes

[Haskell-cafe] Memory management issue in notmuch-haskell bindings

2011-08-16 Thread Ben Gamari
It seems that the notmuch-haskell bindings (version 0.2.2 built against notmuch from git master; passes notmuch-test) aren't dealing with memory management properly. In particular, the attached test code[1] causes talloc to abort. Unfortunately, while the issue is consistently reproducible, it onl

Re: [Haskell-cafe] Memory and Threads - MVars or TVars

2010-07-29 Thread Thomas DuBuisson
On Thu, Jul 29, 2010 at 6:53 AM, Job Vranish wrote: > > You might try pulling downloading the package ('cabal fetch org'  will do > this) and changing the base dependency (to >= 4.1) in the orc.cabal file cabal also has an 'unpack' command for the particularly lazy (me). Ex: cabal unpack orc ;

Re: [Haskell-cafe] Memory and Threads - MVars or TVars

2010-07-29 Thread Job Vranish
You might try pulling downloading the package ('cabal fetch org' will do this) and changing the base dependency (to >= 4.1) in the orc.cabal file and then build it manually (cabal configure && cabal build && cabal install (while in the same directory as the .cabal file)) and see what happens. I d

Re: [Haskell-cafe] Memory and Threads - MVars or TVars

2010-07-29 Thread Ben Millwood
On Thu, Jul 29, 2010 at 3:49 AM, Eitan Goldshtrom wrote: > Perhaps you guys could help me with Cabal now though? I'm > trying to install Orc but it wants base>=4.2 and <=4.3 and I have 4.1 after > installing the latest release of GHC. Cabal won't upgrade the base. It > complains about a dependency

Re: [Haskell-cafe] Memory and Threads - MVars or TVars

2010-07-28 Thread Christopher Done
This could be useful: Beautiful concurrency by Simon Peyton Jones http://research.microsoft.com/en-us/um/people/simonpj/papers/stm/beautiful.pdf On 29 July 2010 02:23, Eitan Goldshtrom wrote: > Hi everyone. I was wondering if someone could just guide me toward some good > information, but if any

Re: [Haskell-cafe] Memory and Threads - MVars or TVars

2010-07-28 Thread Eitan Goldshtrom
Ah! That clears that up a lot. I read the wiki page but something just didn't make full sense about it until you used the word "prevent". I understand that the computer doesn't actually prevent other threads from running -- that would defeat the purpose of the concurrency -- but it helped clear

Re: [Haskell-cafe] Memory and Threads - MVars or TVars

2010-07-28 Thread Job Vranish
Atomic operations are special operations where you don't have to worry about some other process messing with things while the operation is taking place. For a simple example of why atomic operations are important: (taken from: http://en.wikipedia.org/wiki/Linearizability#Non-atomic) The naive, no

[Haskell-cafe] Memory and Threads - MVars or TVars

2010-07-28 Thread Eitan Goldshtrom
Hi everyone. I was wondering if someone could just guide me toward some good information, but if anyone wants to help with a personal explanation I welcome it. I'm trying to write a threaded program and I'm not sure how to manage my memory. I read up on MVars and they make a lot of sense. My re

Re: [Haskell-cafe] memory needed for SAX parsing XML

2010-04-23 Thread Daniil Elovkov
John Lato wrote: Another (additional) approach would be to encapsulate unsafeInterleaveIO within some routine and not let it go out into the wild. lazilyDoWithIO :: IO a -> (a -> b) -> IO b It would use unsafeInterleave internally but catch all IO errors within itself. I wonder if this is a

Re: [Haskell-cafe] memory needed for SAX parsing XML

2010-04-22 Thread John Lato
> Message: 8 > Date: Tue, 20 Apr 2010 12:08:36 +0400 > From: Daniil Elovkov > Subject: Re: [Haskell-cafe] memory needed for SAX parsing XML > To: Haskell-Cafe > Message-ID: <4bcd6104.50...@googlemail.com> > Content-Type: text/plain; charset=ISO-8859-1; format=f

Re: [Haskell-cafe] memory needed for SAX parsing XML

2010-04-20 Thread Daniil Elovkov
Jason Dagit wrote: On Mon, Apr 19, 2010 at 3:01 AM, Daniil Elovkov mailto:daniil.elov...@googlemail.com>> wrote: Hello haskellers! I'm trying to process an xml file with as little footprint as possible. SAX is alright for my case, and I think that's the lightest way possibl

Re: [Haskell-cafe] memory needed for SAX parsing XML

2010-04-19 Thread Jason Dagit
On Mon, Apr 19, 2010 at 3:01 AM, Daniil Elovkov < daniil.elov...@googlemail.com> wrote: > Hello haskellers! > > I'm trying to process an xml file with as little footprint as possible. SAX > is alright for my case, and I think that's the lightest way possible. So, > I'm looking at HaXml.SAX > > I'm

[Haskell-cafe] memory needed for SAX parsing XML

2010-04-19 Thread Daniil Elovkov
Hello haskellers! I'm trying to process an xml file with as little footprint as possible. SAX is alright for my case, and I think that's the lightest way possible. So, I'm looking at HaXml.SAX I'm surprised to see that it takes about 56-60 MB of ram. This seems constant relative to xml file

Re: [Haskell-cafe] Memory-aware Haskell?

2009-12-25 Thread Daniel Fischer
Am Freitag 25 Dezember 2009 15:45:29 schrieb Gwern Branwen: > On Fri, Dec 25, 2009 at 5:14 AM, Svein Ove Aas wrote: > > On Thu, Dec 24, 2009 at 11:38 PM, Roman Cheplyaka wrote: > >> So, let's think what we can do at runtime. Suppose RTS takes the > >> parameter -- upper limit of consumed memory.

Re: [Haskell-cafe] Memory-aware Haskell?

2009-12-25 Thread Gwern Branwen
On Fri, Dec 25, 2009 at 5:14 AM, Svein Ove Aas wrote: > On Thu, Dec 24, 2009 at 11:38 PM, Roman Cheplyaka wrote: >> So, let's think what we can do at runtime. Suppose RTS takes the parameter -- >> upper limit of consumed memory. When it sees that memory consumption is >> close to upper bound, it

Re: [Haskell-cafe] Memory-aware Haskell?

2009-12-25 Thread Svein Ove Aas
On Thu, Dec 24, 2009 at 11:38 PM, Roman Cheplyaka wrote: > So, let's think what we can do at runtime. Suppose RTS takes the parameter -- > upper limit of consumed memory. When it sees that memory consumption is > close to upper bound, it can: > > 1. force garbage collection > This is already imple

Re: [Haskell-cafe] Memory-aware Haskell?

2009-12-24 Thread John Van Enk
This is a problem with partitioned operating systems used in avionics. The airplane computers require certain partitions to exist between programs in both time and space. The space guarantees are most easily enforced by eliminating any dynamic memory allocation once the operating system enters a 'n

[Haskell-cafe] Memory-aware Haskell?

2009-12-24 Thread Roman Cheplyaka
Imagine some system with hard memory bound (e.g. 64M of physical memory, no swap). I don't want some accidental laziness (maybe even not in my code, but in some used package) to crash my program. So, let's think what we can do at runtime. Suppose RTS takes the parameter -- upper limit of consumed

Re: [Haskell-cafe] Memory Leak - Artificial Neural Network

2009-11-05 Thread wren ng thornton
Hector Guilarte wrote: Hi Luke, The code is mainly in Spanish with son parts in English... Thanks for the explanation, I got the idea very well, but now I got some questions about that. How does the Prelude functions for managing lists work? I mean, what does zip, unzip, foldl, foldr, map an

Re: [Haskell-cafe] Memory usage of cabal install

2009-04-27 Thread Krzysztof Kościuszkiewicz
On Mon, Apr 27, 2009 at 02:10:28PM +0100, Duncan Coutts wrote: > If you're using ghc 6.10 then the solution is to update to cabal-install > 0.6.x. If you're quite sure you are using 6.8 then the bug is unknown. > It may still be worth trying upgrading to cabal-install 0.6.x. I've upgraded to caba

Re: [Haskell-cafe] Memory usage of cabal install

2009-04-27 Thread Krzysztof Kościuszkiewicz
On Mon, Apr 27, 2009 at 02:10:28PM +0100, Duncan Coutts wrote: > > [...] > > Increasing verbosity does not help, memory consumption goes up after the > > message "Resolving dependencies..." shows up. > > > > I use ghc 6.8.2 and cabal-install version 0.5.1 using version 1.4.0.1 of > > the Cabal li

Re: [Haskell-cafe] Memory usage of cabal install

2009-04-27 Thread Duncan Coutts
On Mon, 2009-04-27 at 12:56 +0200, Krzysztof Kościuszkiewicz wrote: > Hello Haskell-Café, > > I have a problem with high memory usage of cabal-install. Whenever I > try to install or upgrade a package, cabal manages to consume 1,3G of > memory before I killed it (on a 32-bit machine with 1 GB of

Re: [Haskell-cafe] Memory usage of cabal install

2009-04-27 Thread Krzysztof Skrzętnicki
Jakiej platformy dokładnie dotyczy Twój problem? Proponuję upgrade do najnowszej wersji - można ją ściągnąć ze strony GHC. Pozdrawiam Krzysztof Skrzętnicki 2009/4/27 Krzysztof Kościuszkiewicz : > Hello Haskell-Café, > > I have a problem with high memory usage of cabal-install.  Whenever I > try

[Haskell-cafe] Memory usage of cabal install

2009-04-27 Thread Krzysztof Kościuszkiewicz
Hello Haskell-Café, I have a problem with high memory usage of cabal-install. Whenever I try to install or upgrade a package, cabal manages to consume 1,3G of memory before I killed it (on a 32-bit machine with 1 GB of memory). Increasing verbosity does not help, memory consumption goes up after

Re: [Haskell-cafe] Memory usage when passing arrays in state

2009-03-03 Thread Daniel Fischer
Am Mittwoch, 4. März 2009 02:30 schrieb Tobias Olausson: > Thank you Daniel. > As I understood it DiffArrays are supposed to be faster than the regular They may be supposed to be faster, but they aren't. If you want anything resembling speed, use UArrays, STUArrays, or, if your array elements can

Re: [Haskell-cafe] Memory usage when passing arrays in state

2009-03-03 Thread Ryan Ingram
I've found DiffArrays to be way too slow/memory-hogging for real usage. Since you are in IO already (StateT s IO), you'll probably be better off using a mutable array for a data structure. Some things are still best done in the imperative style. You can be a bit safer by using ST as the bottom m

Re: [Haskell-cafe] Memory usage when passing arrays in state

2009-03-03 Thread Daniel Fischer
Am Mittwoch, 4. März 2009 01:44 schrieb Tobias Olausson: > Hello all. > I am currently implementing an emulation of a CPU, in which the CPU's > RAM is part of the internal state > that is passed around in the program using a state monad. However, the > program performs > unexpectingly bad, and some

Re: [Haskell-cafe] Memory usage when passing arrays in state

2009-03-03 Thread Tobias Olausson
Thank you Daniel. As I understood it DiffArrays are supposed to be faster than the regular Array due to the fact that it doesnt copy the entire Array, but just updates the position that changes, and keeps some kind of "changelog" on the array. But when looking at the statistics for my sample progra

Re: [Haskell-cafe] Memory usage when passing arrays in state

2009-03-03 Thread Daniel Peebles
This may be completely unrelated to your problem, but there's a ticket in the GHC trac saying that DiffArray is unusably slow: http://hackage.haskell.org/trac/ghc/ticket/2727 . It doesn't analyze the cause of the slowness, so it's quite possible that it may be related to GC as in your case. Cheers

[Haskell-cafe] Memory usage when passing arrays in state

2009-03-03 Thread Tobias Olausson
Hello all. I am currently implementing an emulation of a CPU, in which the CPU's RAM is part of the internal state that is passed around in the program using a state monad. However, the program performs unexpectingly bad, and some profiling information makes us believe that the problem is the high

Re: [Haskell-cafe] memory-efficient data type for Netflix data - UArray Int Int vs UArray Int Word8

2009-03-01 Thread Manlio Perillo
Kenneth Hoste ha scritto: Hello, I'm having a go at the Netflix Prize using Haskell. Yes, I'm brave. I kind of have an algorithm in mind that I want to implement using Haskell, but up until now, the main issue has been to find a way to efficiently represent the data... For people who are not

Re[2]: [Haskell-cafe] memory issues

2009-02-28 Thread Bulat Ziganshin
Hello Daniel, Saturday, February 28, 2009, 6:20:09 PM, you wrote: > But they would not be equivalent if stdout has to be locked for each output > operation separately, but a file opened with openFile fp WriteMode was > locked then once and remained so until closed. ghc Handles are locked for eve

Re: [Haskell-cafe] memory issues

2009-02-28 Thread Daniel Fischer
Hello Bulat, Am Samstag, 28. Februar 2009 09:38 schrieb Bulat Ziganshin: > Hello Daniel, > > Saturday, February 28, 2009, 3:10:44 AM, you wrote: > >> print may waste a lot of time, locking stdout for every > >> line printed > > > > hout <- openFile (args!!1) WriteMode > > mapM_ (hPrint hout

Re[2]: [Haskell-cafe] memory issues

2009-02-28 Thread Bulat Ziganshin
Hello Daniel, Saturday, February 28, 2009, 3:10:44 AM, you wrote: >> print may waste a lot of time, locking stdout for every >> line printed > hout <- openFile (args!!1) WriteMode > mapM_ (hPrint hout) $ sort $ blocks content > ? I find hardly any difference, though. no difference. if

Re: [Haskell-cafe] memory issues

2009-02-27 Thread Daniel Fischer
Am Samstag, 28. Februar 2009 00:37 schrieb Bulat Ziganshin: > Hello Daniel, > > Saturday, February 28, 2009, 2:21:31 AM, you wrote: > >> printf "%s" $ unlines $ map (show) (sort $! blocks content) > > > > Bad! > > Use > > mapM_ print $ sort $ blocks content > > are you sure? Tested it. T

Re[2]: [Haskell-cafe] memory issues

2009-02-27 Thread Bulat Ziganshin
Hello Don, Saturday, February 28, 2009, 2:18:37 AM, you wrote: > offset :: !Integer oh yes > And possibly just using {-# UNPACK #-}!Int64 would be ok? i think that it will be even better but main problem is a huge unevaluated thunks. as the last hope, this may be converted to x <- getOffs

Re[2]: [Haskell-cafe] memory issues

2009-02-27 Thread Bulat Ziganshin
Hello Daniel, Saturday, February 28, 2009, 2:21:31 AM, you wrote: >> printf "%s" $ unlines $ map (show) (sort $! blocks content) > Bad! > Use > mapM_ print $ sort $ blocks content are you sure? print may waste a lot of time, locking stdout for every line printed $! is really useless

Re: [Haskell-cafe] memory issues

2009-02-27 Thread Daniel Fischer
Am Freitag, 27. Februar 2009 23:18 schrieb Rogan Creswick: > > \begin{code} > -- Compiled with: > -- $ ghc --make offsetSorter.hs > -- > -- (ghc v. 6.8.2) > -- > -- Run with: > -- $ time ./offsetSorter data/byteOffsets.txt > haskOffsets.txt > -- offsetSorter: out of memory (requested 1048576 bytes

Re: [Haskell-cafe] memory issues

2009-02-27 Thread Don Stewart
bulat.ziganshin: > Hello Rogan, > > Saturday, February 28, 2009, 1:18:47 AM, you wrote: > > > data Block = Block { > > offset::Integer > > , size::Integer > > } deriving (Eq) > > try >!offset::Integer > , !size::Integer > offset :: !Integer And possibly just usi

Re: [Haskell-cafe] memory issues

2009-02-27 Thread Bulat Ziganshin
Hello Rogan, Saturday, February 28, 2009, 1:18:47 AM, you wrote: > data Block = Block { > offset::Integer > , size::Integer > } deriving (Eq) try !offset::Integer , !size::Integer -- Best regards, Bulatmailto:bulat.zigans...@gmail.com __

Re: [Haskell-cafe] memory issues

2009-02-27 Thread Don Stewart
creswick: > On Fri, Feb 27, 2009 at 2:20 PM, Don Stewart wrote: > > creswick: > >> \begin{code} > >> -- Compiled with: > >> -- $ ghc --make offsetSorter.hs > > > > YIKES!! Use the optimizer! > > > >    ghc -O2 --make > > Ah, that did drastically cut the amount of time it takes to run out of > mem

Re: [Haskell-cafe] memory issues

2009-02-27 Thread Rogan Creswick
On Fri, Feb 27, 2009 at 2:20 PM, Don Stewart wrote: > creswick: >> \begin{code} >> -- Compiled with: >> -- $ ghc --make offsetSorter.hs > > YIKES!! Use the optimizer! > >    ghc -O2 --make Ah, that did drastically cut the amount of time it takes to run out of memory (down to 1:23), but unfortunat

Re: [Haskell-cafe] memory issues

2009-02-27 Thread Don Stewart
creswick: > First off, my apologies for breaking etiquette, if/when I do -- I've > only just joined Haskell-cafe, and I'm quite new to Haskell. > > I have recently been trying to process a large data set (the 2.8tb > wikipedia data dump), and also replace my scripting needs with haskell > (needs t

[Haskell-cafe] memory issues

2009-02-27 Thread Rogan Creswick
First off, my apologies for breaking etiquette, if/when I do -- I've only just joined Haskell-cafe, and I'm quite new to Haskell. I have recently been trying to process a large data set (the 2.8tb wikipedia data dump), and also replace my scripting needs with haskell (needs that have previously be

Re: [Haskell-cafe] memory-efficient data type for Netflix data - UArray Int Int vs UArray Int Word8

2009-02-26 Thread Manlio Perillo
Kenneth Hoste ha scritto: [...] However, as I posted yesterday, I've been able to circumvent the issue by rethinking my data type, i.e. using the ~18K movie IDs as key instead of the 480K user IDs, which radically limits the overhead... Well, but what if you really need the original data stru

Re: [Haskell-cafe] memory-efficient data type for Netflix data - UArray Int Int vs UArray Int Word8

2009-02-26 Thread Kenneth Hoste
On Feb 26, 2009, at 13:00 , Manlio Perillo wrote: Kenneth Hoste ha scritto: Hello, I'm having a go at the Netflix Prize using Haskell. Yes, I'm brave. [...] To see if I could efficiently represent the data set in this way, I wrote a small Haskell program (attached) which uses the following

Re: [Haskell-cafe] memory-efficient data type for Netflix data - UArray Int Int vs UArray Int Word8

2009-02-26 Thread Manlio Perillo
Kenneth Hoste ha scritto: Hello, I'm having a go at the Netflix Prize using Haskell. Yes, I'm brave. [...] To see if I could efficiently represent the data set in this way, I wrote a small Haskell program (attached) which uses the following data type: From what I see, to append a new integ

Re: [Haskell-cafe] memory-efficient data type for Netflix data - UArray Int Int vs UArray Int Word8

2009-02-23 Thread wren ng thornton
Kenneth Hoste wrote: Well, I'm using UArray, but I'm willing to consider other suitable containers... As long as they are memory efficient. :-) The typical usage of a UArray will be getting all it's contents, and converting it to a list to easily manipulate (filter, ...). So, maybe another dat

Re: [Haskell-cafe] memory-efficient data type for Netflix data - UArray Int Int vs UArray Int Word8

2009-02-23 Thread Kenneth Hoste
On Feb 23, 2009, at 19:57 , Don Stewart wrote: bos: 2009/2/23 Kenneth Hoste Does anyone know why the Word8 version is not significantly better in terms of memory usage? Yes, because there's a typo on line 413 of Data/Array/Vector/Prim/ BUArr.hs. How's that for service? :-) U

Re: [Haskell-cafe] memory-efficient data type for Netflix data - UArray Int Int vs UArray Int Word8

2009-02-23 Thread Don Stewart
bos: > 2009/2/23 Kenneth Hoste > > > Does anyone know why the Word8 version is not significantly better in > terms > of memory usage? > > > Yes, because there's a typo on line 413 of Data/Array/Vector/Prim/BUArr.hs. > > How's that for service? :-) UArray or UArr? __

Re: [Haskell-cafe] memory-efficient data type for Netflix data - UArray Int Int vs UArray Int Word8

2009-02-23 Thread Bryan O'Sullivan
2009/2/23 Kenneth Hoste > Does anyone know why the Word8 version is not significantly better in terms > of memory usage? > Yes, because there's a typo on line 413 of Data/Array/Vector/Prim/BUArr.hs. How's that for service? :-) ___ Haskell-Cafe mailin

[Haskell-cafe] memory-efficient data type for Netflix data - UArray Int Int vs UArray Int Word8

2009-02-23 Thread Kenneth Hoste
Hello, I'm having a go at the Netflix Prize using Haskell. Yes, I'm brave. I kind of have an algorithm in mind that I want to implement using Haskell, but up until now, the main issue has been to find a way to efficiently represent the data... For people who are not familiar with the Netfl

Re: [Haskell-cafe] Memory

2009-02-18 Thread Jeff Douglas
I'd love to. I'm thinking of starting a blog once I get more experience and familiarity with the language. Jeff On Wed, Feb 18, 2009 at 6:13 PM, Magnus Therning wrote: > On Tue, Feb 17, 2009 at 5:53 AM, Jeff Douglas wrote: >> Thanks Guys, >> >> Not only did I not run optimizations, I misread th

Re: [Haskell-cafe] Memory

2009-02-18 Thread Magnus Therning
On Tue, Feb 17, 2009 at 5:53 AM, Jeff Douglas wrote: > Thanks Guys, > > Not only did I not run optimizations, I misread the profile. It looks > like it was an imaginary problem from the beginning. I guess I should > go through all the profiling documentation more carefully. Please share what you

Re: [Haskell-cafe] Memory

2009-02-16 Thread Jeff Douglas
Thanks Guys, Not only did I not run optimizations, I misread the profile. It looks like it was an imaginary problem from the beginning. I guess I should go through all the profiling documentation more carefully. Jeff On Tue, Feb 17, 2009 at 2:46 PM, Bernie Pope wrote: > > On 17/02/2009, at 3:56

Re: [Haskell-cafe] Memory

2009-02-16 Thread Bernie Pope
On 17/02/2009, at 3:56 PM, Jeff Douglas wrote: Hello All, The kind people at #haskell suggested I come to haskell-cafe for questions about haskell performance issues. I'm new to haskell, and I'm having a hard time understanding how to deal with memory leaks. I've been playing with some networ

Re: [Haskell-cafe] Memory

2009-02-16 Thread Don Stewart
inbuninbu: > Hello All, > > The kind people at #haskell suggested I come to haskell-cafe for > questions about haskell performance issues. > I'm new to haskell, and I'm having a hard time understanding how to > deal with memory leaks. > > I've been playing with some network server examples and I

[Haskell-cafe] Memory

2009-02-16 Thread Jeff Douglas
Hello All, The kind people at #haskell suggested I come to haskell-cafe for questions about haskell performance issues. I'm new to haskell, and I'm having a hard time understanding how to deal with memory leaks. I've been playing with some network server examples and I noticed with each new conne

RE: [Haskell-cafe] Memory efficiency questions for real-time graphics

2008-11-05 Thread Tobias Bexelius
ylvan [mailto:[EMAIL PROTECTED] Sent: den 4 november 2008 20:06 To: [EMAIL PROTECTED] Cc: Tobias Bexelius; haskell-cafe@haskell.org Subject: Re: [Haskell-cafe] Memory efficiency questions for real-time graphics On Mon, Nov 3, 2008 at 3:45 PM, Svein Ove Aas <[EMAIL PROTECTED]> wrote:

Re: [Haskell-cafe] Memory efficiency questions for real-time graphics

2008-11-04 Thread Sebastian Sylvan
On Mon, Nov 3, 2008 at 3:45 PM, Svein Ove Aas <[EMAIL PROTECTED]> wrote: > On Mon, Nov 3, 2008 at 11:31 AM, Tobias Bexelius > <[EMAIL PROTECTED]> wrote: > > Before Direct3D 10, its too costly to read back the updated vertex data > > in every frame, which force you to make this kind of operations o

Re: [Haskell-cafe] Memory efficiency questions for real-time graphics

2008-11-03 Thread Svein Ove Aas
On Mon, Nov 3, 2008 at 11:31 AM, Tobias Bexelius <[EMAIL PROTECTED]> wrote: > Before Direct3D 10, its too costly to read back the updated vertex data > in every frame, which force you to make this kind of operations on the > CPU. > With D3D 10 however, you should use the new Stream-Output stage whi

RE: [Haskell-cafe] Memory efficiency questions for real-time graphics

2008-11-03 Thread Tobias Bexelius
if you can afford a new graphics card and likes Vista, that's the way to go :) /Tobias -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of T Willingham Sent: den 2 november 2008 20:11 To: haskell-cafe@haskell.org Subject: Re: [Haskell-cafe] M

Re: [Haskell-cafe] Memory efficiency questions for real-time graphics

2008-11-02 Thread T Willingham
On Sun, Nov 2, 2008 at 2:13 PM, Don Stewart <[EMAIL PROTECTED]> wrote: > t.r.willingham: >> Take a highly complicated function and apply it to N vertices. Now >> increase N until the framerate is affected. That is where I am. It >> is obvious that any N-sized allocations will cause the framerate

Re: [Haskell-cafe] Memory efficiency questions for real-time graphics

2008-11-02 Thread Don Stewart
t.r.willingham: > On Sat, Nov 1, 2008 at 2:11 PM, Sebastian Sylvan > <[EMAIL PROTECTED]> wrote: > > On Sat, Nov 1, 2008 at 6:57 PM, T Willingham <[EMAIL PROTECTED]> > >> > >> The per-vertex computation is a quite complex time-dependent function > >> applied to the given domain on each update. Yet

Re: [Haskell-cafe] Memory efficiency questions for real-time graphics

2008-11-02 Thread T Willingham
On Sat, Nov 1, 2008 at 2:11 PM, Sebastian Sylvan <[EMAIL PROTECTED]> wrote: > On Sat, Nov 1, 2008 at 6:57 PM, T Willingham <[EMAIL PROTECTED]> >> >> The per-vertex computation is a quite complex time-dependent function >> applied to the given domain on each update. Yet even if it were >> simple, I

Re: [Haskell-cafe] Memory efficiency questions for real-time graphics

2008-11-01 Thread T Willingham
On Tue, Oct 28, 2008 at 3:24 PM, Sebastian Sylvan <[EMAIL PROTECTED]> wrote: > 2008/10/28 T Willingham <[EMAIL PROTECTED]> >> >> To give a context for all of this, I am applying a non-linear >> transformation to an object on every frame. (Note: non-linear, so a >> matrix transform will not suffice

Re: [Haskell-cafe] Memory efficiency questions for real-time graphics

2008-10-28 Thread Sebastian Sylvan
2008/10/28 T Willingham <[EMAIL PROTECTED]> > > As my first Haskell exposure, I've been working through Real World > Haskell. > > I am considering converting some of my C++ graphics libraries to > Haskell. I've done a fair amount of googling on the subject, however > I haven't quite been able to

Re: [Haskell-cafe] Memory efficiency questions for real-time graphics

2008-10-27 Thread Jefferson Heard
By the way, T, feel free to lean on me if you run into any problems. I did something along the lines of what you were describing some time ago, my particular non-linear transform being converting a vertex array to/from polar coordinates and updating in realtime. -- Jeff On Tue, Oct 28, 2008 at 12

Re: [Haskell-cafe] Memory efficiency questions for real-time graphics

2008-10-27 Thread T Willingham
On Mon, Oct 27, 2008 at 11:04 PM, Don Stewart <[EMAIL PROTECTED]> wrote: > It depends on the operations (safe indexing or unsafe indexing). > Being strict or unboxed doesn't determine the safety. OK, that makes sense. This is a huge load off my conscience. I can now dig into Real World Haskell w

Re: [Haskell-cafe] Memory efficiency questions for real-time graphics

2008-10-27 Thread Don Stewart
t.r.willingham: > On Mon, Oct 27, 2008 at 10:07 PM, Don Stewart <[EMAIL PROTECTED]> wrote: > > > > Seems fine. You'll be working at a low level, with strict, mutable, > > unboxed data structures, but that's fine: the machine loves them. > > > > Thanks for the quick reply. One last question -- is

Re: [Haskell-cafe] Memory efficiency questions for real-time graphics

2008-10-27 Thread T Willingham
On Mon, Oct 27, 2008 at 10:07 PM, Don Stewart <[EMAIL PROTECTED]> wrote: > > Seems fine. You'll be working at a low level, with strict, mutable, > unboxed data structures, but that's fine: the machine loves them. > Thanks for the quick reply. One last question -- is it at all possible to segfault

Re: [Haskell-cafe] Memory efficiency questions for real-time graphics

2008-10-27 Thread Don Stewart
t.r.willingham: >As my first Haskell exposure, I've been working through Real World >Haskell. > >I am considering converting some of my C++ graphics libraries to >Haskell. I've done a fair amount of googling on the subject, however >I haven't quite been able to find clear answ

[Haskell-cafe] Memory efficiency questions for real-time graphics

2008-10-27 Thread T Willingham
As my first Haskell exposure, I've been working through Real World Haskell. I am considering converting some of my C++ graphics libraries to Haskell. I've done a fair amount of googling on the subject, however I haven't quite been able to find clear answers to some of following issues. (1) Using

Re: [Haskell-cafe] Memory problems reading a IntMap from a binary file

2008-08-03 Thread Don Stewart
lutzsteens: > Hi, > > I have IntMap String with about 40,000 entries. After saving it to disk > (via Data.Binary) the file is 3.5 Mb small. However if I load it and > save it back again my program needs 180 MB memory. Is there anything I > do wrong or does the map really need that much memory?

[Haskell-cafe] Memory problems reading a IntMap from a binary file

2008-08-03 Thread Ludger Steens
Hi, I have IntMap String with about 40,000 entries. After saving it to disk (via Data.Binary) the file is 3.5 Mb small. However if I load it and save it back again my program needs 180 MB memory. Is there anything I do wrong or does the map really need that much memory? The (simple) program

Re: [Haskell-cafe] Memory profiling

2008-06-16 Thread Chaddaï Fouché
2008/6/16 Pieter Laeremans <[EMAIL PROTECTED]>: > Hi, > > Which tools do you recommand for memory profiling haskell programs > on a *nix system. > I'm using haskell to develop a CGI program/script. > > The application has to be deployed on shared hosting infrastructure. > Since I would like to be

[Haskell-cafe] Memory profiling

2008-06-15 Thread Pieter Laeremans
Hi, Which tools do you recommand for memory profiling haskell programs on a *nix system. I'm using haskell to develop a CGI program/script. The application has to be deployed on shared hosting infrastructure. Since I would like to be a good citizen , I would need to meassure the maximum amount

Re: [Haskell-cafe] Memory allocation in extractor function (newbie question)

2008-04-07 Thread Alexander Kireyev
On Mon, Apr 7, 2008 at 4:16 PM, Yitzchak Gale <[EMAIL PROTECTED]> wrote: > You didn't show us the code for countForPoints. I'll bet you wrote > something like > > countForPoints area ls count points = > sum $ map (countPathsFrom area (count + 1) ls) points > > Unfortunately, the standard sum

Re: [Haskell-cafe] Memory allocation in extractor function (newbie question)

2008-04-07 Thread Yitzchak Gale
Alexander Kireyev wrote: > While trying to write a program for the countPaths Code Jam problem I > ran into what seems to me as a weird behaviour in terms of memory > allocation... > The profiling log (./paths +RTS -P) shows the following time/space > behaviour for them... Hi Alexander, I'm

[Haskell-cafe] Memory allocation in extractor function (newbie question)

2008-04-07 Thread Alexander Kireyev
Hello, While trying to write a program for the countPaths Code Jam problem I ran into what seems to me as a weird behaviour in terms of memory allocation. The task is to count the number of way you can spell a certain "word" by walking some path on a board of letters. Being a newbie I started wi

Re: [Haskell-cafe] Memory Leak Help

2007-11-11 Thread Bertram Felgenhauer
J. Garrett Morris wrote: > Hello, > > I have code which seems to contain a memory leak, but I'm not sure > where it is or what's causing it. Any help would be greatly > appreciated: I see no memory leak in your code, it "just" breaks the garbage collector's heuristics by allocating an awful lot

[Haskell-cafe] Memory Leak Help

2007-11-11 Thread J. Garrett Morris
Hello, I have code which seems to contain a memory leak, but I'm not sure where it is or what's causing it. Any help would be greatly appreciated: The code is: data Ratings = Ratings { movieCount :: Int , movieLookup :: IOUArray Int Word32 , movieRa

Re: [Haskell-cafe] Memory-mapped arrays? (IArray interfaces, slices, and so on)

2007-11-08 Thread David Roundy
On Wed, Nov 07, 2007 at 10:10:16PM +, Jules Bean wrote: > Joel Reymont wrote: > >Is there such a thing as memory-mapped arrays in GHC? > > In principle, there could be an IArray instance to memory-mapped files. > > (There could also be a mutable version, but just the IArray version > would b

Re: [Haskell-cafe] Memory-mapped arrays? (IArray interfaces, slices, and so on)

2007-11-07 Thread Stefan O'Rear
On Wed, Nov 07, 2007 at 10:10:16PM +, Jules Bean wrote: > Joel Reymont wrote: >> Is there such a thing as memory-mapped arrays in GHC? > > In principle, there could be an IArray instance to memory-mapped files. > > (There could also be a mutable version, but just the IArray version would > be

Re: [Haskell-cafe] Memory-mapped arrays? (IArray interfaces, slices, and so on)

2007-11-07 Thread Jules Bean
Joel Reymont wrote: Is there such a thing as memory-mapped arrays in GHC? In principle, there could be an IArray instance to memory-mapped files. (There could also be a mutable version, but just the IArray version would be useful). I noticed just the other day that there are some 'obvious'

Re: [Haskell-cafe] Memory-mapped arrays?

2007-11-07 Thread Don Stewart
joelr1: > Is there such a thing as memory-mapped arrays in GHC? > > I'm looking for something that would let me memory-map a file of > floats and access it as an array. > There's a commented out mmapFile for ByteString in Data.ByteString's source. Use that, and then extract the ForeignPtr from

[Haskell-cafe] Memory-mapped arrays?

2007-11-07 Thread Joel Reymont
Is there such a thing as memory-mapped arrays in GHC? I'm looking for something that would let me memory-map a file of floats and access it as an array. Thanks, Joel -- http://wagerlabs.com ___ Haskell-Cafe mailing list Haskell-Cafe@ha

Re: [Haskell-cafe] Memory leak or wrong use of Array ?

2007-09-14 Thread Brandon S. Allbery KF8NH
On Sep 14, 2007, at 21:35 , L.Guo wrote: Thanks for your advice about thunk, though I do not understand *thunk* very well. Is there any other discriptions about thunk ? A "thunk" is, in general, a piece of code which represents a suspended or delayed action. In Haskell, it represents a laz

Re: [Haskell-cafe] Memory leak or wrong use of Array ?

2007-09-14 Thread L.Guo
Hi Stuart. Thanks for your advice about thunk, though I do not understand *thunk* very well. Is there any other discriptions about thunk ? I have tried the *seq* operation. When input is 10,000,000, the memory still "leak", and there is still a "stack overflow". I changed some mapM_ to sequence

Re: [Haskell-cafe] Memory leak or wrong use of Array ?

2007-09-13 Thread Stuart Cook
On 9/14/07, L.Guo <[EMAIL PROTECTED]> wrote: > Hi MailList Haskell-Cafe: > > I am tring to solve Project Euler problem 70. > And write some code. (will at the end of this mail) > And, I run the code in GHCi. > > The problem is that, when the input is 1,000,000, it works > fine, when the input is up

[Haskell-cafe] Memory leak or wrong use of Array ?

2007-09-13 Thread L.Guo
Hi MailList Haskell-Cafe: I am tring to solve Project Euler problem 70. And write some code. (will at the end of this mail) And, I run the code in GHCi. The problem is that, when the input is 1,000,000, it works fine, when the input is up to 10,000,000, the memory GHCi used increase very fast a

Re: [Haskell-cafe] Memory profiler

2007-07-27 Thread Stefan O'Rear
On Sat, Jul 28, 2007 at 12:11:31AM +0100, Jon Harrop wrote: > Is there a memory profiler for Haskell? Yes. GHC, NHC and HBC all have integrated heap profilers. ghc --make -prof -auto-all ... ./MyProgram +RTS -hc -RTS ./MyProgram +RTS -hm -RTS ./MyProgram +RTS -hd -RTS ./MyProgram +RTS -hy -RTS .

[Haskell-cafe] Memory profiler

2007-07-27 Thread Jon Harrop
Is there a memory profiler for Haskell? -- Dr Jon D Harrop, Flying Frog Consultancy Ltd. OCaml for Scientists http://www.ffconsultancy.com/products/ocaml_for_scientists/?e ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/ma

Re: [Haskell-cafe] memory usage in repeated reading of an external program's output

2007-06-21 Thread Andrea Rossato
On Thu, Jun 21, 2007 at 01:36:16PM -0700, Bryan O'Sullivan wrote: > Andrea Rossato wrote: > > > Still I do not understand you reference to the leak problem. Could you > > please elaborate a bit? > > The runProcess function returns a ProcessHandle. If you don't call > waitForProcess on that h

  1   2   >