Re: Memory Consumption of Large Sequences

2009-02-03 Thread Keith Bennett
Mark - Not a problem. I didn't take it that way at all. - Keith Hope I didn't offend with my rather sharp reply -- I meant to be clear but I realized later that it could be taken as sounding annoyed!  The other folks did a great job of explaining the situation with a more friendly tone

Re: Memory Consumption of Large Sequences

2009-02-02 Thread Christian Vest Hansen
The trick with these listish things is to not calculate the tail until you need it, and to throw away the head when you're done with it. On Mon, Feb 2, 2009 at 7:06 PM, Keith Bennett keithrbenn...@gmail.com wrote: All - I'm curious to know how to minimize memory consumption in Clojure for

Re: Memory Consumption of Large Sequences

2009-02-02 Thread Keith Bennett
On Feb 2, 1:41 pm, Paul Barry pauljbar...@gmail.com wrote: Ketih, I think what you have done, at least at the JVM level, is create 100,000 lists, with basically each list containing an element and a pointer to the next element.  Because one list points to the next, none of them can be

Re: Memory Consumption of Large Sequences

2009-02-02 Thread Keith Bennett
Paul - Clojure definitely has its benefits, but in terms of memory footprint, Java appears to be *much* more economical, unless elements can be discarded shortly after use as Christian describes, in which case it's much *less* economical. In a Java ArrayList, only a single ArrayList object is

Re: Memory Consumption of Large Sequences

2009-02-02 Thread Chouser
On Mon, Feb 2, 2009 at 4:48 PM, Keith Bennett keithrbenn...@gmail.com wrote: Clojure definitely has its benefits, but in terms of memory footprint, Java appears to be *much* more economical It's probably worth being careful to separate the different parts of Java and Clojure. Clojure code

Re: Memory Consumption of Large Sequences

2009-02-02 Thread Mark H.
On Feb 2, 10:06 am, Keith Bennett keithrbenn...@gmail.com wrote: I'm curious to know how to minimize memory consumption in Clojure for large lists.  I did the following test in repl: (def d ()) (dotimes [_ 10] (def d (cons x d))) Let me translate this into pseudocode for you: Make d

Re: Memory Consumption of Large Sequences

2009-02-02 Thread Luc Prefontaine
Paul, I can understand the concerns about memory footprint if you work in a restricted environment (lack of physical memory or other system resources) or if your memory load is very high. However, these things are less common these days with the kind of hardware we can buy compared to what we

Re: Memory Consumption of Large Sequences

2009-02-02 Thread Luc Prefontaine
Paul sorry for the mistake, these emails are a pain to follow sometimes, Keith, It's up to you if you prefer to slice the Clojure features one by one down the bone marrow, as for myself I used a different approach to ramp up with Clojure ASAP The need to get down to implementation details came

Re: Memory Consumption of Large Sequences

2009-02-02 Thread Mark H.
On Feb 2, 5:32 pm, Keith Bennett keithrbenn...@gmail.com wrote: I am just now learning Clojure, and for me, understanding what's going on underneath the surface helps me understand how to use the language properly.  As I said previously, the amount of memory consumed by a list will very

Re: Memory Consumption of Large Sequences

2009-02-02 Thread Mark H.
On Feb 2, 5:32 pm, Keith Bennett keithrbenn...@gmail.com wrote: I have a lot of work to do to learn how to think in functional programming.  These kinds of discussions are very helpful. A picky point -- lazy sequences aren't really a functional programming thing (although restricting side