[Haskell-cafe] Memory corruption issues when using newAlignedPinnedByteArray, GC kicking in?

2012-07-10 Thread Nicolas Trangez
All, While working on my vector-simd library, I noticed somehow memory I'm using gets corrupted/overwritten. I reworked this into a test case, and would love to get some help on how to fix this. Previously I used some custom FFI calls to C to allocate aligned memory, which yields correct

Re: [Haskell-cafe] Memory corruption issues when using newAlignedPinnedByteArray, GC kicking in?

2012-07-10 Thread Thomas Schilling
I think you should ask this question on the glasgow-haskell-users mailing list: http://www.haskell.org/mailman/listinfo/glasgow-haskell-users On 10 July 2012 18:20, Nicolas Trangez nico...@incubaid.com wrote: All, While working on my vector-simd library, I noticed somehow memory I'm using

[Haskell-cafe] Memory leak from close fields in conduit

2012-04-11 Thread Michael Snoyman
Hi all, Hideyuki Tanaka alerted me[1] to a memory leak in conduit. Long story short: it appears that Pipe composition leads to collection of a large number of `return ()` actions for unnecessary memory cleanup. We came up with a possible solution: a Finalize type[2]. In both of our testing, this

[Haskell-cafe] Memory usage with TVar?

2012-02-01 Thread Johan Brinch
Here's the example program: https://gist.github.com/1cbe113d2c79e2fc9d2b When I run the program (which maintains a list inside an STM TVar), I get the following statistics: ./Test +RTS -s 176,041,728 bytes allocated in the heap 386,794,976 bytes copied during GC 69,180,224 bytes

Re: [Haskell-cafe] Memory usage with TVar?

2012-02-01 Thread Johan Brinch
On Wed, Feb 1, 2012 at 16:29, Johan Brinch brin...@gmail.com wrote: I'm on a 64-bit machine where 2'000'000 integers uses 32 MB. 2 * 2'000'000 ;) -- Johan Brinch ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org

Re: [Haskell-cafe] Memory usage with TVar?

2012-02-01 Thread Jake McArthur
First, see this question about space usage on Stack Overflow: http://stackoverflow.com/questions/3254758/memory-footprint-of-haskell-data-types Next, apply this knowledge not only to Ints, but also to tuples and lists. There's your memory usage. - Jake On Wed, Feb 1, 2012 at 10:29 AM, Johan

[Haskell-cafe] Memory management issue in notmuch-haskell bindings

2011-08-16 Thread Ben Gamari
It seems that the notmuch-haskell bindings (version 0.2.2 built against notmuch from git master; passes notmuch-test) aren't dealing with memory management properly. In particular, the attached test code[1] causes talloc to abort. Unfortunately, while the issue is consistently reproducible, it

Re: [Haskell-cafe] Memory and Threads - MVars or TVars

2010-07-29 Thread Christopher Done
This could be useful: Beautiful concurrency by Simon Peyton Jones http://research.microsoft.com/en-us/um/people/simonpj/papers/stm/beautiful.pdf On 29 July 2010 02:23, Eitan Goldshtrom thesource...@gmail.com wrote: Hi everyone. I was wondering if someone could just guide me toward some good

Re: [Haskell-cafe] Memory and Threads - MVars or TVars

2010-07-29 Thread Ben Millwood
On Thu, Jul 29, 2010 at 3:49 AM, Eitan Goldshtrom thesource...@gmail.com wrote: Perhaps you guys could help me with Cabal now though? I'm trying to install Orc but it wants base=4.2 and =4.3 and I have 4.1 after installing the latest release of GHC. Cabal won't upgrade the base. It complains

Re: [Haskell-cafe] Memory and Threads - MVars or TVars

2010-07-29 Thread Job Vranish
You might try pulling downloading the package ('cabal fetch org' will do this) and changing the base dependency (to = 4.1) in the orc.cabal file and then build it manually (cabal configure cabal build cabal install (while in the same directory as the .cabal file)) and see what happens. I don't

Re: [Haskell-cafe] Memory and Threads - MVars or TVars

2010-07-29 Thread Thomas DuBuisson
On Thu, Jul 29, 2010 at 6:53 AM, Job Vranish job.vran...@gmail.com wrote: You might try pulling downloading the package ('cabal fetch org'  will do this) and changing the base dependency (to = 4.1) in the orc.cabal file cabal also has an 'unpack' command for the particularly lazy (me). Ex:

[Haskell-cafe] Memory and Threads - MVars or TVars

2010-07-28 Thread Eitan Goldshtrom
Hi everyone. I was wondering if someone could just guide me toward some good information, but if anyone wants to help with a personal explanation I welcome it. I'm trying to write a threaded program and I'm not sure how to manage my memory. I read up on MVars and they make a lot of sense. My

Re: [Haskell-cafe] Memory and Threads - MVars or TVars

2010-07-28 Thread Job Vranish
Atomic operations are special operations where you don't have to worry about some other process messing with things while the operation is taking place. For a simple example of why atomic operations are important: (taken from: http://en.wikipedia.org/wiki/Linearizability#Non-atomic) The naive,

Re: [Haskell-cafe] Memory and Threads - MVars or TVars

2010-07-28 Thread Eitan Goldshtrom
Ah! That clears that up a lot. I read the wiki page but something just didn't make full sense about it until you used the word prevent. I understand that the computer doesn't actually prevent other threads from running -- that would defeat the purpose of the concurrency -- but it helped clear

Re: [Haskell-cafe] memory needed for SAX parsing XML

2010-04-23 Thread Daniil Elovkov
John Lato wrote: Another (additional) approach would be to encapsulate unsafeInterleaveIO within some routine and not let it go out into the wild. lazilyDoWithIO :: IO a - (a - b) - IO b It would use unsafeInterleave internally but catch all IO errors within itself. I wonder if this is a

Re: [Haskell-cafe] memory needed for SAX parsing XML

2010-04-22 Thread John Lato
Message: 8 Date: Tue, 20 Apr 2010 12:08:36 +0400 From: Daniil Elovkov daniil.elov...@googlemail.com Subject: Re: [Haskell-cafe] memory needed for SAX parsing XML To: Haskell-Cafe haskell-cafe@haskell.org Message-ID: 4bcd6104.50...@googlemail.com Content-Type: text/plain; charset=ISO-8859-1

Re: [Haskell-cafe] memory needed for SAX parsing XML

2010-04-20 Thread Daniil Elovkov
Jason Dagit wrote: On Mon, Apr 19, 2010 at 3:01 AM, Daniil Elovkov daniil.elov...@googlemail.com mailto:daniil.elov...@googlemail.com wrote: Hello haskellers! I'm trying to process an xml file with as little footprint as possible. SAX is alright for my case, and I think that's

[Haskell-cafe] memory needed for SAX parsing XML

2010-04-19 Thread Daniil Elovkov
Hello haskellers! I'm trying to process an xml file with as little footprint as possible. SAX is alright for my case, and I think that's the lightest way possible. So, I'm looking at HaXml.SAX I'm surprised to see that it takes about 56-60 MB of ram. This seems constant relative to xml file

Re: [Haskell-cafe] memory needed for SAX parsing XML

2010-04-19 Thread Jason Dagit
On Mon, Apr 19, 2010 at 3:01 AM, Daniil Elovkov daniil.elov...@googlemail.com wrote: Hello haskellers! I'm trying to process an xml file with as little footprint as possible. SAX is alright for my case, and I think that's the lightest way possible. So, I'm looking at HaXml.SAX I'm

Re: [Haskell-cafe] Memory-aware Haskell?

2009-12-25 Thread Svein Ove Aas
On Thu, Dec 24, 2009 at 11:38 PM, Roman Cheplyaka r...@ro-che.info wrote: So, let's think what we can do at runtime. Suppose RTS takes the parameter -- upper limit of consumed memory. When it sees that memory consumption is close to upper bound, it can: 1. force garbage collection This is

Re: [Haskell-cafe] Memory-aware Haskell?

2009-12-25 Thread Gwern Branwen
On Fri, Dec 25, 2009 at 5:14 AM, Svein Ove Aas svein@aas.no wrote: On Thu, Dec 24, 2009 at 11:38 PM, Roman Cheplyaka r...@ro-che.info wrote: So, let's think what we can do at runtime. Suppose RTS takes the parameter -- upper limit of consumed memory. When it sees that memory consumption is

Re: [Haskell-cafe] Memory-aware Haskell?

2009-12-25 Thread Daniel Fischer
Am Freitag 25 Dezember 2009 15:45:29 schrieb Gwern Branwen: On Fri, Dec 25, 2009 at 5:14 AM, Svein Ove Aas svein@aas.no wrote: On Thu, Dec 24, 2009 at 11:38 PM, Roman Cheplyaka r...@ro-che.info wrote: So, let's think what we can do at runtime. Suppose RTS takes the parameter -- upper

[Haskell-cafe] Memory-aware Haskell?

2009-12-24 Thread Roman Cheplyaka
Imagine some system with hard memory bound (e.g. 64M of physical memory, no swap). I don't want some accidental laziness (maybe even not in my code, but in some used package) to crash my program. So, let's think what we can do at runtime. Suppose RTS takes the parameter -- upper limit of consumed

Re: [Haskell-cafe] Memory-aware Haskell?

2009-12-24 Thread John Van Enk
This is a problem with partitioned operating systems used in avionics. The airplane computers require certain partitions to exist between programs in both time and space. The space guarantees are most easily enforced by eliminating any dynamic memory allocation once the operating system enters a

Re: [Haskell-cafe] Memory Leak - Artificial Neural Network

2009-11-05 Thread wren ng thornton
Hector Guilarte wrote: Hi Luke, The code is mainly in Spanish with son parts in English... Thanks for the explanation, I got the idea very well, but now I got some questions about that. How does the Prelude functions for managing lists work? I mean, what does zip, unzip, foldl, foldr, map

[Haskell-cafe] Memory usage of cabal install

2009-04-27 Thread Krzysztof Kościuszkiewicz
Hello Haskell-Café, I have a problem with high memory usage of cabal-install. Whenever I try to install or upgrade a package, cabal manages to consume 1,3G of memory before I killed it (on a 32-bit machine with 1 GB of memory). Increasing verbosity does not help, memory consumption goes up

Re: [Haskell-cafe] Memory usage of cabal install

2009-04-27 Thread Krzysztof Skrzętnicki
Jakiej platformy dokładnie dotyczy Twój problem? Proponuję upgrade do najnowszej wersji - można ją ściągnąć ze strony GHC. Pozdrawiam Krzysztof Skrzętnicki 2009/4/27 Krzysztof Kościuszkiewicz k.kosciuszkiew...@gmail.com: Hello Haskell-Café, I have a problem with high memory usage of

Re: [Haskell-cafe] Memory usage of cabal install

2009-04-27 Thread Duncan Coutts
On Mon, 2009-04-27 at 12:56 +0200, Krzysztof Kościuszkiewicz wrote: Hello Haskell-Café, I have a problem with high memory usage of cabal-install. Whenever I try to install or upgrade a package, cabal manages to consume 1,3G of memory before I killed it (on a 32-bit machine with 1 GB of

Re: [Haskell-cafe] Memory usage of cabal install

2009-04-27 Thread Krzysztof Kościuszkiewicz
On Mon, Apr 27, 2009 at 02:10:28PM +0100, Duncan Coutts wrote: [...] Increasing verbosity does not help, memory consumption goes up after the message Resolving dependencies... shows up. I use ghc 6.8.2 and cabal-install version 0.5.1 using version 1.4.0.1 of the Cabal library. The

Re: [Haskell-cafe] Memory usage of cabal install

2009-04-27 Thread Krzysztof Kościuszkiewicz
On Mon, Apr 27, 2009 at 02:10:28PM +0100, Duncan Coutts wrote: If you're using ghc 6.10 then the solution is to update to cabal-install 0.6.x. If you're quite sure you are using 6.8 then the bug is unknown. It may still be worth trying upgrading to cabal-install 0.6.x. I've upgraded to

[Haskell-cafe] Memory usage when passing arrays in state

2009-03-03 Thread Tobias Olausson
Hello all. I am currently implementing an emulation of a CPU, in which the CPU's RAM is part of the internal state that is passed around in the program using a state monad. However, the program performs unexpectingly bad, and some profiling information makes us believe that the problem is the high

Re: [Haskell-cafe] Memory usage when passing arrays in state

2009-03-03 Thread Daniel Peebles
This may be completely unrelated to your problem, but there's a ticket in the GHC trac saying that DiffArray is unusably slow: http://hackage.haskell.org/trac/ghc/ticket/2727 . It doesn't analyze the cause of the slowness, so it's quite possible that it may be related to GC as in your case.

Re: [Haskell-cafe] Memory usage when passing arrays in state

2009-03-03 Thread Tobias Olausson
Thank you Daniel. As I understood it DiffArrays are supposed to be faster than the regular Array due to the fact that it doesnt copy the entire Array, but just updates the position that changes, and keeps some kind of changelog on the array. But when looking at the statistics for my sample

Re: [Haskell-cafe] Memory usage when passing arrays in state

2009-03-03 Thread Daniel Fischer
Am Mittwoch, 4. März 2009 01:44 schrieb Tobias Olausson: Hello all. I am currently implementing an emulation of a CPU, in which the CPU's RAM is part of the internal state that is passed around in the program using a state monad. However, the program performs unexpectingly bad, and some

Re: [Haskell-cafe] Memory usage when passing arrays in state

2009-03-03 Thread Ryan Ingram
I've found DiffArrays to be way too slow/memory-hogging for real usage. Since you are in IO already (StateT s IO), you'll probably be better off using a mutable array for a data structure. Some things are still best done in the imperative style. You can be a bit safer by using ST as the bottom

Re: [Haskell-cafe] Memory usage when passing arrays in state

2009-03-03 Thread Daniel Fischer
Am Mittwoch, 4. März 2009 02:30 schrieb Tobias Olausson: Thank you Daniel. As I understood it DiffArrays are supposed to be faster than the regular They may be supposed to be faster, but they aren't. If you want anything resembling speed, use UArrays, STUArrays, or, if your array elements

Re: [Haskell-cafe] memory-efficient data type for Netflix data - UArray Int Int vs UArray Int Word8

2009-03-01 Thread Manlio Perillo
Kenneth Hoste ha scritto: Hello, I'm having a go at the Netflix Prize using Haskell. Yes, I'm brave. I kind of have an algorithm in mind that I want to implement using Haskell, but up until now, the main issue has been to find a way to efficiently represent the data... For people who are

Re[2]: [Haskell-cafe] memory issues

2009-02-28 Thread Bulat Ziganshin
Hello Daniel, Saturday, February 28, 2009, 3:10:44 AM, you wrote: print may waste a lot of time, locking stdout for every line printed hout - openFile (args!!1) WriteMode mapM_ (hPrint hout) $ sort $ blocks content ? I find hardly any difference, though. no difference. if handle

Re: [Haskell-cafe] memory issues

2009-02-28 Thread Daniel Fischer
Hello Bulat, Am Samstag, 28. Februar 2009 09:38 schrieb Bulat Ziganshin: Hello Daniel, Saturday, February 28, 2009, 3:10:44 AM, you wrote: print may waste a lot of time, locking stdout for every line printed hout - openFile (args!!1) WriteMode mapM_ (hPrint hout) $ sort $

Re[2]: [Haskell-cafe] memory issues

2009-02-28 Thread Bulat Ziganshin
Hello Daniel, Saturday, February 28, 2009, 6:20:09 PM, you wrote: But they would not be equivalent if stdout has to be locked for each output operation separately, but a file opened with openFile fp WriteMode was locked then once and remained so until closed. ghc Handles are locked for every

Re: [Haskell-cafe] memory issues

2009-02-27 Thread Don Stewart
bulat.ziganshin: Hello Rogan, Saturday, February 28, 2009, 1:18:47 AM, you wrote: data Block = Block { offset::Integer , size::Integer } deriving (Eq) try !offset::Integer , !size::Integer offset :: !Integer And possibly just using {-# UNPACK

Re: [Haskell-cafe] memory issues

2009-02-27 Thread Daniel Fischer
Am Samstag, 28. Februar 2009 00:37 schrieb Bulat Ziganshin: Hello Daniel, Saturday, February 28, 2009, 2:21:31 AM, you wrote: printf %s $ unlines $ map (show) (sort $! blocks content) Bad! Use mapM_ print $ sort $ blocks content are you sure? Tested it. The printf %s is

Re: [Haskell-cafe] memory-efficient data type for Netflix data - UArray Int Int vs UArray Int Word8

2009-02-26 Thread Manlio Perillo
Kenneth Hoste ha scritto: Hello, I'm having a go at the Netflix Prize using Haskell. Yes, I'm brave. [...] To see if I could efficiently represent the data set in this way, I wrote a small Haskell program (attached) which uses the following data type: From what I see, to append a new

Re: [Haskell-cafe] memory-efficient data type for Netflix data - UArray Int Int vs UArray Int Word8

2009-02-26 Thread Kenneth Hoste
On Feb 26, 2009, at 13:00 , Manlio Perillo wrote: Kenneth Hoste ha scritto: Hello, I'm having a go at the Netflix Prize using Haskell. Yes, I'm brave. [...] To see if I could efficiently represent the data set in this way, I wrote a small Haskell program (attached) which uses the following

Re: [Haskell-cafe] memory-efficient data type for Netflix data - UArray Int Int vs UArray Int Word8

2009-02-26 Thread Manlio Perillo
Kenneth Hoste ha scritto: [...] However, as I posted yesterday, I've been able to circumvent the issue by rethinking my data type, i.e. using the ~18K movie IDs as key instead of the 480K user IDs, which radically limits the overhead... Well, but what if you really need the original data

[Haskell-cafe] memory-efficient data type for Netflix data - UArray Int Int vs UArray Int Word8

2009-02-23 Thread Kenneth Hoste
Hello, I'm having a go at the Netflix Prize using Haskell. Yes, I'm brave. I kind of have an algorithm in mind that I want to implement using Haskell, but up until now, the main issue has been to find a way to efficiently represent the data... For people who are not familiar with the

Re: [Haskell-cafe] memory-efficient data type for Netflix data - UArray Int Int vs UArray Int Word8

2009-02-23 Thread Bryan O'Sullivan
2009/2/23 Kenneth Hoste kenneth.ho...@ugent.be Does anyone know why the Word8 version is not significantly better in terms of memory usage? Yes, because there's a typo on line 413 of Data/Array/Vector/Prim/BUArr.hs. How's that for service? :-) ___

Re: [Haskell-cafe] memory-efficient data type for Netflix data - UArray Int Int vs UArray Int Word8

2009-02-23 Thread Don Stewart
bos: 2009/2/23 Kenneth Hoste kenneth.ho...@ugent.be Does anyone know why the Word8 version is not significantly better in terms of memory usage? Yes, because there's a typo on line 413 of Data/Array/Vector/Prim/BUArr.hs. How's that for service? :-) UArray or UArr?

Re: [Haskell-cafe] memory-efficient data type for Netflix data - UArray Int Int vs UArray Int Word8

2009-02-23 Thread Kenneth Hoste
On Feb 23, 2009, at 19:57 , Don Stewart wrote: bos: 2009/2/23 Kenneth Hoste kenneth.ho...@ugent.be Does anyone know why the Word8 version is not significantly better in terms of memory usage? Yes, because there's a typo on line 413 of Data/Array/Vector/Prim/ BUArr.hs. How's

Re: [Haskell-cafe] memory-efficient data type for Netflix data - UArray Int Int vs UArray Int Word8

2009-02-23 Thread wren ng thornton
Kenneth Hoste wrote: Well, I'm using UArray, but I'm willing to consider other suitable containers... As long as they are memory efficient. :-) The typical usage of a UArray will be getting all it's contents, and converting it to a list to easily manipulate (filter, ...). So, maybe another

Re: [Haskell-cafe] Memory

2009-02-18 Thread Magnus Therning
On Tue, Feb 17, 2009 at 5:53 AM, Jeff Douglas inbuni...@gmail.com wrote: Thanks Guys, Not only did I not run optimizations, I misread the profile. It looks like it was an imaginary problem from the beginning. I guess I should go through all the profiling documentation more carefully. Please

Re: [Haskell-cafe] Memory

2009-02-18 Thread Jeff Douglas
I'd love to. I'm thinking of starting a blog once I get more experience and familiarity with the language. Jeff On Wed, Feb 18, 2009 at 6:13 PM, Magnus Therning mag...@therning.org wrote: On Tue, Feb 17, 2009 at 5:53 AM, Jeff Douglas inbuni...@gmail.com wrote: Thanks Guys, Not only did I not

[Haskell-cafe] Memory

2009-02-16 Thread Jeff Douglas
Hello All, The kind people at #haskell suggested I come to haskell-cafe for questions about haskell performance issues. I'm new to haskell, and I'm having a hard time understanding how to deal with memory leaks. I've been playing with some network server examples and I noticed with each new

Re: [Haskell-cafe] Memory

2009-02-16 Thread Don Stewart
inbuninbu: Hello All, The kind people at #haskell suggested I come to haskell-cafe for questions about haskell performance issues. I'm new to haskell, and I'm having a hard time understanding how to deal with memory leaks. I've been playing with some network server examples and I noticed

Re: [Haskell-cafe] Memory

2009-02-16 Thread Bernie Pope
On 17/02/2009, at 3:56 PM, Jeff Douglas wrote: Hello All, The kind people at #haskell suggested I come to haskell-cafe for questions about haskell performance issues. I'm new to haskell, and I'm having a hard time understanding how to deal with memory leaks. I've been playing with some

Re: [Haskell-cafe] Memory

2009-02-16 Thread Jeff Douglas
Thanks Guys, Not only did I not run optimizations, I misread the profile. It looks like it was an imaginary problem from the beginning. I guess I should go through all the profiling documentation more carefully. Jeff On Tue, Feb 17, 2009 at 2:46 PM, Bernie Pope bj...@csse.unimelb.edu.au wrote:

RE: [Haskell-cafe] Memory efficiency questions for real-time graphics

2008-11-05 Thread Tobias Bexelius
[mailto:[EMAIL PROTECTED] Sent: den 4 november 2008 20:06 To: [EMAIL PROTECTED] Cc: Tobias Bexelius; haskell-cafe@haskell.org Subject: Re: [Haskell-cafe] Memory efficiency questions for real-time graphics On Mon, Nov 3, 2008 at 3:45 PM, Svein Ove Aas [EMAIL PROTECTED] wrote: On Mon, Nov 3

Re: [Haskell-cafe] Memory efficiency questions for real-time graphics

2008-11-04 Thread Sebastian Sylvan
On Mon, Nov 3, 2008 at 3:45 PM, Svein Ove Aas [EMAIL PROTECTED] wrote: On Mon, Nov 3, 2008 at 11:31 AM, Tobias Bexelius [EMAIL PROTECTED] wrote: Before Direct3D 10, its too costly to read back the updated vertex data in every frame, which force you to make this kind of operations on the

Re: [Haskell-cafe] Memory efficiency questions for real-time graphics

2008-11-03 Thread Svein Ove Aas
On Mon, Nov 3, 2008 at 11:31 AM, Tobias Bexelius [EMAIL PROTECTED] wrote: Before Direct3D 10, its too costly to read back the updated vertex data in every frame, which force you to make this kind of operations on the CPU. With D3D 10 however, you should use the new Stream-Output stage which is

Re: [Haskell-cafe] Memory efficiency questions for real-time graphics

2008-11-02 Thread T Willingham
On Sat, Nov 1, 2008 at 2:11 PM, Sebastian Sylvan [EMAIL PROTECTED] wrote: On Sat, Nov 1, 2008 at 6:57 PM, T Willingham [EMAIL PROTECTED] The per-vertex computation is a quite complex time-dependent function applied to the given domain on each update. Yet even if it were simple, I would still

Re: [Haskell-cafe] Memory efficiency questions for real-time graphics

2008-11-02 Thread Don Stewart
t.r.willingham: On Sat, Nov 1, 2008 at 2:11 PM, Sebastian Sylvan [EMAIL PROTECTED] wrote: On Sat, Nov 1, 2008 at 6:57 PM, T Willingham [EMAIL PROTECTED] The per-vertex computation is a quite complex time-dependent function applied to the given domain on each update. Yet even if it were

Re: [Haskell-cafe] Memory efficiency questions for real-time graphics

2008-11-02 Thread T Willingham
On Sun, Nov 2, 2008 at 2:13 PM, Don Stewart [EMAIL PROTECTED] wrote: t.r.willingham: Take a highly complicated function and apply it to N vertices. Now increase N until the framerate is affected. That is where I am. It is obvious that any N-sized allocations will cause the framerate to

Re: [Haskell-cafe] Memory efficiency questions for real-time graphics

2008-11-01 Thread T Willingham
On Tue, Oct 28, 2008 at 3:24 PM, Sebastian Sylvan [EMAIL PROTECTED] wrote: 2008/10/28 T Willingham [EMAIL PROTECTED] To give a context for all of this, I am applying a non-linear transformation to an object on every frame. (Note: non-linear, so a matrix transform will not suffice.) Any

Re: [Haskell-cafe] Memory efficiency questions for real-time graphics

2008-10-28 Thread Sebastian Sylvan
2008/10/28 T Willingham [EMAIL PROTECTED] As my first Haskell exposure, I've been working through Real World Haskell. I am considering converting some of my C++ graphics libraries to Haskell. I've done a fair amount of googling on the subject, however I haven't quite been able to find

[Haskell-cafe] Memory efficiency questions for real-time graphics

2008-10-27 Thread T Willingham
As my first Haskell exposure, I've been working through Real World Haskell. I am considering converting some of my C++ graphics libraries to Haskell. I've done a fair amount of googling on the subject, however I haven't quite been able to find clear answers to some of following issues. (1)

Re: [Haskell-cafe] Memory efficiency questions for real-time graphics

2008-10-27 Thread Don Stewart
t.r.willingham: As my first Haskell exposure, I've been working through Real World Haskell. I am considering converting some of my C++ graphics libraries to Haskell. I've done a fair amount of googling on the subject, however I haven't quite been able to find clear answers

Re: [Haskell-cafe] Memory efficiency questions for real-time graphics

2008-10-27 Thread T Willingham
On Mon, Oct 27, 2008 at 10:07 PM, Don Stewart [EMAIL PROTECTED] wrote: Seems fine. You'll be working at a low level, with strict, mutable, unboxed data structures, but that's fine: the machine loves them. Thanks for the quick reply. One last question -- is it at all possible to segfault with

Re: [Haskell-cafe] Memory efficiency questions for real-time graphics

2008-10-27 Thread Don Stewart
t.r.willingham: On Mon, Oct 27, 2008 at 10:07 PM, Don Stewart [EMAIL PROTECTED] wrote: Seems fine. You'll be working at a low level, with strict, mutable, unboxed data structures, but that's fine: the machine loves them. Thanks for the quick reply. One last question -- is it at all

Re: [Haskell-cafe] Memory efficiency questions for real-time graphics

2008-10-27 Thread T Willingham
On Mon, Oct 27, 2008 at 11:04 PM, Don Stewart [EMAIL PROTECTED] wrote: It depends on the operations (safe indexing or unsafe indexing). Being strict or unboxed doesn't determine the safety. OK, that makes sense. This is a huge load off my conscience. I can now dig into Real World Haskell

Re: [Haskell-cafe] Memory efficiency questions for real-time graphics

2008-10-27 Thread Jefferson Heard
By the way, T, feel free to lean on me if you run into any problems. I did something along the lines of what you were describing some time ago, my particular non-linear transform being converting a vertex array to/from polar coordinates and updating in realtime. -- Jeff On Tue, Oct 28, 2008 at

[Haskell-cafe] Memory problems reading a IntMap from a binary file

2008-08-03 Thread Ludger Steens
Hi, I have IntMap String with about 40,000 entries. After saving it to disk (via Data.Binary) the file is 3.5 Mb small. However if I load it and save it back again my program needs 180 MB memory. Is there anything I do wrong or does the map really need that much memory? The (simple) program

Re: [Haskell-cafe] Memory problems reading a IntMap from a binary file

2008-08-03 Thread Don Stewart
lutzsteens: Hi, I have IntMap String with about 40,000 entries. After saving it to disk (via Data.Binary) the file is 3.5 Mb small. However if I load it and save it back again my program needs 180 MB memory. Is there anything I do wrong or does the map really need that much memory?

Re: [Haskell-cafe] Memory profiling

2008-06-16 Thread Chaddaï Fouché
2008/6/16 Pieter Laeremans [EMAIL PROTECTED]: Hi, Which tools do you recommand for memory profiling haskell programs on a *nix system. I'm using haskell to develop a CGI program/script. The application has to be deployed on shared hosting infrastructure. Since I would like to be a good

[Haskell-cafe] Memory profiling

2008-06-15 Thread Pieter Laeremans
Hi, Which tools do you recommand for memory profiling haskell programs on a *nix system. I'm using haskell to develop a CGI program/script. The application has to be deployed on shared hosting infrastructure. Since I would like to be a good citizen , I would need to meassure the maximum amount

[Haskell-cafe] Memory allocation in extractor function (newbie question)

2008-04-07 Thread Alexander Kireyev
Hello, While trying to write a program for the countPaths Code Jam problem I ran into what seems to me as a weird behaviour in terms of memory allocation. The task is to count the number of way you can spell a certain word by walking some path on a board of letters. Being a newbie I started

Re: [Haskell-cafe] Memory allocation in extractor function (newbie question)

2008-04-07 Thread Alexander Kireyev
On Mon, Apr 7, 2008 at 4:16 PM, Yitzchak Gale [EMAIL PROTECTED] wrote: You didn't show us the code for countForPoints. I'll bet you wrote something like countForPoints area ls count points = sum $ map (countPathsFrom area (count + 1) ls) points Unfortunately, the standard sum function

[Haskell-cafe] Memory Leak Help

2007-11-11 Thread J. Garrett Morris
Hello, I have code which seems to contain a memory leak, but I'm not sure where it is or what's causing it. Any help would be greatly appreciated: The code is: data Ratings = Ratings { movieCount :: Int , movieLookup :: IOUArray Int Word32 ,

Re: [Haskell-cafe] Memory Leak Help

2007-11-11 Thread Bertram Felgenhauer
J. Garrett Morris wrote: Hello, I have code which seems to contain a memory leak, but I'm not sure where it is or what's causing it. Any help would be greatly appreciated: I see no memory leak in your code, it just breaks the garbage collector's heuristics by allocating an awful lot of

Re: [Haskell-cafe] Memory-mapped arrays? (IArray interfaces, slices, and so on)

2007-11-08 Thread David Roundy
On Wed, Nov 07, 2007 at 10:10:16PM +, Jules Bean wrote: Joel Reymont wrote: Is there such a thing as memory-mapped arrays in GHC? In principle, there could be an IArray instance to memory-mapped files. (There could also be a mutable version, but just the IArray version would be

[Haskell-cafe] Memory-mapped arrays?

2007-11-07 Thread Joel Reymont
Is there such a thing as memory-mapped arrays in GHC? I'm looking for something that would let me memory-map a file of floats and access it as an array. Thanks, Joel -- http://wagerlabs.com ___ Haskell-Cafe mailing list

Re: [Haskell-cafe] Memory-mapped arrays?

2007-11-07 Thread Don Stewart
joelr1: Is there such a thing as memory-mapped arrays in GHC? I'm looking for something that would let me memory-map a file of floats and access it as an array. There's a commented out mmapFile for ByteString in Data.ByteString's source. Use that, and then extract the ForeignPtr from the

Re: [Haskell-cafe] Memory-mapped arrays? (IArray interfaces, slices, and so on)

2007-11-07 Thread Jules Bean
Joel Reymont wrote: Is there such a thing as memory-mapped arrays in GHC? In principle, there could be an IArray instance to memory-mapped files. (There could also be a mutable version, but just the IArray version would be useful). I noticed just the other day that there are some 'obvious'

Re: [Haskell-cafe] Memory-mapped arrays? (IArray interfaces, slices, and so on)

2007-11-07 Thread Stefan O'Rear
On Wed, Nov 07, 2007 at 10:10:16PM +, Jules Bean wrote: Joel Reymont wrote: Is there such a thing as memory-mapped arrays in GHC? In principle, there could be an IArray instance to memory-mapped files. (There could also be a mutable version, but just the IArray version would be

Re: [Haskell-cafe] Memory leak or wrong use of Array ?

2007-09-14 Thread L.Guo
Hi Stuart. Thanks for your advice about thunk, though I do not understand *thunk* very well. Is there any other discriptions about thunk ? I have tried the *seq* operation. When input is 10,000,000, the memory still leak, and there is still a stack overflow. I changed some mapM_ to sequence .

Re: [Haskell-cafe] Memory leak or wrong use of Array ?

2007-09-14 Thread Brandon S. Allbery KF8NH
On Sep 14, 2007, at 21:35 , L.Guo wrote: Thanks for your advice about thunk, though I do not understand *thunk* very well. Is there any other discriptions about thunk ? A thunk is, in general, a piece of code which represents a suspended or delayed action. In Haskell, it represents a lazy

[Haskell-cafe] Memory leak or wrong use of Array ?

2007-09-13 Thread L.Guo
Hi MailList Haskell-Cafe: I am tring to solve Project Euler problem 70. And write some code. (will at the end of this mail) And, I run the code in GHCi. The problem is that, when the input is 1,000,000, it works fine, when the input is up to 10,000,000, the memory GHCi used increase very fast

[Haskell-cafe] Memory profiler

2007-07-27 Thread Jon Harrop
Is there a memory profiler for Haskell? -- Dr Jon D Harrop, Flying Frog Consultancy Ltd. OCaml for Scientists http://www.ffconsultancy.com/products/ocaml_for_scientists/?e ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org

Re: [Haskell-cafe] Memory profiler

2007-07-27 Thread Stefan O'Rear
On Sat, Jul 28, 2007 at 12:11:31AM +0100, Jon Harrop wrote: Is there a memory profiler for Haskell? Yes. GHC, NHC and HBC all have integrated heap profilers. ghc --make -prof -auto-all ... ./MyProgram +RTS -hc -RTS ./MyProgram +RTS -hm -RTS ./MyProgram +RTS -hd -RTS ./MyProgram +RTS -hy -RTS

[Haskell-cafe] memory usage in repeated reading of an external program's output

2007-06-21 Thread Andrea Rossato
Hello, I have this very simple program that executes an external program, reads its output and prints it (the program is date). The readings is done with pipes. The problem is that memory usage constantly increases over time. Profiling does not show garbage collection of any sort. File

Re: [Haskell-cafe] memory usage in repeated reading of an external program's output

2007-06-21 Thread Dougal Stanton
On 21/06/07, Andrea Rossato [EMAIL PROTECTED] wrote: runComLoop :: String - IO () runComLoop command = do (r,w) - createPipe wh - fdToHandle w hSetBuffering wh LineBuffering p - runProcess command [] Nothing Nothing Nothing (Just wh) (Just wh) rh - fdToHandle r

Re: [Haskell-cafe] memory usage in repeated reading of an external program's output

2007-06-21 Thread Brandon S. Allbery KF8NH
On Jun 21, 2007, at 6:40 , Andrea Rossato wrote: I have this very simple program that executes an external program, reads its output and prints it (the program is date). The readings is done with pipes. The problem is that memory usage constantly increases over time. Profiling does not show

Re: [Haskell-cafe] memory usage in repeated reading of an external program's output

2007-06-21 Thread Andrea Rossato
On Thu, Jun 21, 2007 at 08:18:23AM -0400, Brandon S. Allbery KF8NH wrote: On Jun 21, 2007, at 6:40 , Andrea Rossato wrote: I have this very simple program that executes an external program, reads its output and prints it (the program is date). The readings is done with pipes. The

Re: [Haskell-cafe] memory usage in repeated reading of an external program's output

2007-06-21 Thread Bryan O'Sullivan
Andrea Rossato wrote: Now I'm going to profile for memory usage: I've seen that some GC happens if you are patient enough. Yes, the process will hit a steady state of a few megabytes of heap after a short time. By the way, your program leaks ProcessHandles. b

Re: [Haskell-cafe] memory usage in repeated reading of an external program's output

2007-06-21 Thread Bryan O'Sullivan
Andrea Rossato wrote: Still I do not understand you reference to the leak problem. Could you please elaborate a bit? The runProcess function returns a ProcessHandle. If you don't call waitForProcess on that handle, you'll leak those handles. On Unix-like systems, this means you'll

Re: [Haskell-cafe] memory usage in repeated reading of an external program's output

2007-06-21 Thread Andrea Rossato
On Thu, Jun 21, 2007 at 01:36:16PM -0700, Bryan O'Sullivan wrote: Andrea Rossato wrote: Still I do not understand you reference to the leak problem. Could you please elaborate a bit? The runProcess function returns a ProcessHandle. If you don't call waitForProcess on that handle,

[Haskell-cafe] Memory leak in streaming parser

2007-04-03 Thread Oren Ben-Kiki
I refactored the code and uploaded a new version to Hackage (YamlReference-0.2). It is cleaner now and much more efficient. It is still leaking memory though. The profiler hints at bindReply as the culprit retaining State and Reply objects, but it isn't clear to me why the code would do that.

Re: [Haskell-cafe] Memory leak in streaming parser

2007-04-02 Thread Malcolm Wallace
Oren Ben-Kiki [EMAIL PROTECTED] wrote: I just created an initial version of a streaming parser. This parser is intended to serve as a reference parser for the YAML spec. An observation about your state setter functions, e.g. setDecision :: String - State - State setDecision decision state

Re: [Haskell-cafe] Memory leak in streaming parser

2007-04-02 Thread Oren Ben-Kiki
On Mon, 2007-04-02 at 13:54 +0100, Malcolm Wallace wrote: An observation about your state setter functions, ... You can shorten your code considerably by using the standard named-field update syntax for exactly this task: setDecision :: String - State - State setDecision decision state =

[Haskell-cafe] Memory leak in streaming parser

2007-04-01 Thread Oren Ben-Kiki
I just created an initial version of a streaming parser. This parser is intended to serve as a reference parser for the YAML spec. Efficiency isn't the main concern; the priority was to have it use the BNF productions from the spec without any changes (ok, with tiny, minor, insignificant changes

Re: [Haskell-cafe] memory, garbage collection and other newbie's issues

2006-10-22 Thread Andrea Rossato
Hello! On Sun, Oct 22, 2006 at 12:27:05AM +0400, Bulat Ziganshin wrote: as Udo said, it should be better to evaluate thunks just when they are created, by using proper 'seq' calls. While I understand why you and Udo are right, still it is difficult for me to related this discussion to my code.

  1   2   >