Re: Laziness madness
Hi, Am 02.03.2009 um 06:06 schrieb max3000: Here is the code that gave me trouble: (map #(add-watch % watcher callback-fn) all-agents) This was not executing. I had to change it to the below expression: (doseq [agent all-labor-agents] (add-watch agent total-labor-agent callback-fn)) This second expression seems less elegant than the map above. The unhelpful answer: Do it the right way! Neither map nor for are looping constructs. The (hopefully) more helpful answer: The second expression looks much more elegant than (dorun (map ...)). Let me explain why I think so. The map produces a list of nils, which you throw away. This is waste. I come from Germany and here it's a big deal with recycling and resource saving. So I find such a waste deeply ugly. On the other hand doseq just iterates eager through the sequence evaluating the body and finally returning nil. Emphasising the difference between functional code and code living from side-effects is a very strong point of Clojure. Laurent already gave some hints to recognise side-effecting functions. So whenever you have such a smell, look for the do* functions, doall, doseq, do itself, ... There were already many answers and good ideas (check usage of lazy return values) in this thread. But I thought, one should also show that doseq is not ugly. Sincerely Meikel smime.p7s Description: S/MIME cryptographic signature
Re: Laziness madness
On Mar 2, 3:29 am, Mark Volkmann r.mark.volkm...@gmail.com wrote: On Sun, Mar 1, 2009 at 11:06 PM, max3000 maxime.lar...@gmail.com wrote: Hi, I find the laziness in clojure very hard to wrap my head around. I understand the idea and it's probably nice in theory. However, in real life it doesn't seem really useful beyond hardcore mathematical problems. I felt the same way at first. I think it would help if the group shared some common, non-mathematical cases, where laziness is helpful. I'll share this one which is described near the end of the section on Input/Output athttp://www.ociweb.com/mark/clojure/article.html#IO. The line-seq function takes a java.io.BufferedReader and returns a lazy sequence of all the lines of text in it. The significance of returning a lazy sequence is that it doesn't really read all of the lines when it is called. That could consume too much memory. Instead it reads a line each time one is requested from the lazy sequence. Another way to think about this is that it allows writing functions that return all the data a caller might want without having to know ahead of time how much each caller might want. It may be the case that a caller just wants the first piece of data or three pieces or all of them. As the writer of the function that returns the data, you don't have to care about that. -- R. Mark Volkmann Object Computing, Inc. I also struggled (still struggle) with laziness. My main objection was that lazy/non-lazy sequences is not a hidden abstraction - ie you really need to know if what you are asking for is actually being produced. I am slowly changing my mind. I now consider every seq operation to be lazy, which means I only need to think about the cases where I map for side effects. I really like the above suggestion that the compiler warn about lazy computation that is thrown away. Another very subtle (at least I think so) point about lazyness is that you need to be VERY careful about using refs in a lazy seq. Because the work for creating a seq is deferred, the dereference will happen later than what you possibly expect. I'd also like to see a warning in the compiler that says Don't use refs inside lazy seqs Cheers, Brad --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Laziness madness
In August, I was also struggling with laziness. As I went along my learning curve, I realized that much of this fog in my mind came up because I was still taking mutability of data and side effects as normal by habit. I had not spliced my thoughts while toggling between Java and Clojure, something I was doing a lot at the beginning. Once I hammered in my brains that everything in Clojure is immutable by default while coding, it cleared up things. I could now deal with the distinction between desired side effects and deferred results (laziness). Now side effects in my code are exceptions and this helps circumvent the laziness gotchas. Today it's now integrated in my mind and I do not face such problems anymore. I still do not know why the two were linked in my mind except for the fact that I was really messed up with the right-left assignment syndrome and this probably increased confusion regarding laziness : Our brains is suppose to be like plasticine but as we grow older it gets a bit dryer :))) Luc On Mon, 2009-03-02 at 08:55 -0800, Bradbev wrote: On Mar 2, 3:29 am, Mark Volkmann r.mark.volkm...@gmail.com wrote: On Sun, Mar 1, 2009 at 11:06 PM, max3000 maxime.lar...@gmail.com wrote: Hi, I find the laziness in clojure very hard to wrap my head around. I understand the idea and it's probably nice in theory. However, in real life it doesn't seem really useful beyond hardcore mathematical problems. I felt the same way at first. I think it would help if the group shared some common, non-mathematical cases, where laziness is helpful. I'll share this one which is described near the end of the section on Input/Output athttp://www.ociweb.com/mark/clojure/article.html#IO. The line-seq function takes a java.io.BufferedReader and returns a lazy sequence of all the lines of text in it. The significance of returning a lazy sequence is that it doesn't really read all of the lines when it is called. That could consume too much memory. Instead it reads a line each time one is requested from the lazy sequence. Another way to think about this is that it allows writing functions that return all the data a caller might want without having to know ahead of time how much each caller might want. It may be the case that a caller just wants the first piece of data or three pieces or all of them. As the writer of the function that returns the data, you don't have to care about that. -- R. Mark Volkmann Object Computing, Inc. I also struggled (still struggle) with laziness. My main objection was that lazy/non-lazy sequences is not a hidden abstraction - ie you really need to know if what you are asking for is actually being produced. I am slowly changing my mind. I now consider every seq operation to be lazy, which means I only need to think about the cases where I map for side effects. I really like the above suggestion that the compiler warn about lazy computation that is thrown away. Another very subtle (at least I think so) point about lazyness is that you need to be VERY careful about using refs in a lazy seq. Because the work for creating a seq is deferred, the dereference will happen later than what you possibly expect. I'd also like to see a warning in the compiler that says Don't use refs inside lazy seqs Cheers, Brad -- Luc Préfontaine Off.:(514) 993-0320 Fax.:(514) 993-0325 Armageddon was yesterday, today we have a real problem... --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Laziness madness
I felt the same way at first. I think it would help if the group shared some common, non-mathematical cases, where laziness is helpful. I've been using multiple resultset-seq to collect together matching data from different databases and stream it on through some existing Java code. It is fast, and memory efficient. I don't have to even think about releasing memory or moving resultset cursors along. My algorithm can look ahead in the cached sequences without losing rows, and thanks to the common seq abstraction I can change the head of the pipeline to pump unit-test data through without changing the rest of the code. It is awesome. -- Dave --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Laziness madness
Hi, I find the laziness in clojure very hard to wrap my head around. I understand the idea and it's probably nice in theory. However, in real life it doesn't seem really useful beyond hardcore mathematical problems. Case in point, I just spent 2 hours debugging a piece of code (shown below) that seemed simple enough. This is the 3rd time this week that I've lost substantial time to laziness. I'm pretty pissed to tell the truth and I find myself wrapping things in doseq more and more just to be sure. I rarely use 'for' anymore, what's the point? Here is the code that gave me trouble: (map #(add-watch % watcher callback-fn) all-agents) This was not executing. I had to change it to the below expression: (doseq [agent all-labor-agents] (add-watch agent total-labor-agent callback-fn)) This second expression seems less elegant than the map above. Why doesn't clojure realize that an add-watch really should actually loop over all-agents? Why is it that Java calls are not made in similar expressions? Is laziness so useful that we should waste time investigating and fixing errors like this? Sure, there could be special constructs for laziness when we really need it. However, clojure shouldn't default to it IMO. At this point, laziness is a nice concept but it feels somewhat removed from reality to tell the truth. Of course I want to iterate over my collection when I'm doing an add-watch! What am I missing? Thanks, Max --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Laziness madness
doseq is the idiomatic way to write imperative code segments like this (add-watch generates a side-effect). Nevertheless, I too love using map for stuff like that. All you need to do is surround the map with dorun (or doall if you need the return value) and it will force the computation. see: (doc dorun) and (doc doall) On Mar 2, 7:06 am, max3000 maxime.lar...@gmail.com wrote: Hi, I find the laziness in clojure very hard to wrap my head around. I understand the idea and it's probably nice in theory. However, in real life it doesn't seem really useful beyond hardcore mathematical problems. Case in point, I just spent 2 hours debugging a piece of code (shown below) that seemed simple enough. This is the 3rd time this week that I've lost substantial time to laziness. I'm pretty pissed to tell the truth and I find myself wrapping things in doseq more and more just to be sure. I rarely use 'for' anymore, what's the point? Here is the code that gave me trouble: (map #(add-watch % watcher callback-fn) all-agents) This was not executing. I had to change it to the below expression: (doseq [agent all-labor-agents] (add-watch agent total-labor-agent callback-fn)) This second expression seems less elegant than the map above. Why doesn't clojure realize that an add-watch really should actually loop over all-agents? Why is it that Java calls are not made in similar expressions? Is laziness so useful that we should waste time investigating and fixing errors like this? Sure, there could be special constructs for laziness when we really need it. However, clojure shouldn't default to it IMO. At this point, laziness is a nice concept but it feels somewhat removed from reality to tell the truth. Of course I want to iterate over my collection when I'm doing an add-watch! What am I missing? Thanks, Max --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Laziness madness
Remember that you can force lazy sequences if you need to as you are with doseq with doall (retains head) and dorun (does not) as well. You probably want (dorun (map #(add-watch % watcher callback-fn) all-agents)) I think it's pretty clear here what's going on. Your code needs side-effects. In Clojure side-effect code tends to stands out. I sympathize with your frustration, I remember getting excited about Lisp macros but spending many many long hours trying to understand how to write even a simple one. Similarly I think lazy sequences are worth the initial frustration ;) On Mon, Mar 2, 2009 at 12:06 AM, max3000 maxime.lar...@gmail.com wrote: Hi, I find the laziness in clojure very hard to wrap my head around. I understand the idea and it's probably nice in theory. However, in real life it doesn't seem really useful beyond hardcore mathematical problems. Case in point, I just spent 2 hours debugging a piece of code (shown below) that seemed simple enough. This is the 3rd time this week that I've lost substantial time to laziness. I'm pretty pissed to tell the truth and I find myself wrapping things in doseq more and more just to be sure. I rarely use 'for' anymore, what's the point? Here is the code that gave me trouble: (map #(add-watch % watcher callback-fn) all-agents) This was not executing. I had to change it to the below expression: (doseq [agent all-labor-agents] (add-watch agent total-labor-agent callback-fn)) This second expression seems less elegant than the map above. Why doesn't clojure realize that an add-watch really should actually loop over all-agents? Why is it that Java calls are not made in similar expressions? Is laziness so useful that we should waste time investigating and fixing errors like this? Sure, there could be special constructs for laziness when we really need it. However, clojure shouldn't default to it IMO. At this point, laziness is a nice concept but it feels somewhat removed from reality to tell the truth. Of course I want to iterate over my collection when I'm doing an add-watch! What am I missing? Thanks, Max --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Laziness madness
Thanks for the pointers. I was indeed able to make it work with dorun. ;) Still, I'm not sure I understand what all the fuss is about regarding laziness. I'll take your word for it for now but I hope to grock it eventually. Thanks! Max On Mar 2, 12:30 am, David Nolen dnolen.li...@gmail.com wrote: Remember that you can force lazy sequences if you need to as you are with doseq with doall (retains head) and dorun (does not) as well. You probably want (dorun (map #(add-watch % watcher callback-fn) all-agents)) I think it's pretty clear here what's going on. Your code needs side-effects. In Clojure side-effect code tends to stands out. I sympathize with your frustration, I remember getting excited about Lisp macros but spending many many long hours trying to understand how to write even a simple one. Similarly I think lazy sequences are worth the initial frustration ;) On Mon, Mar 2, 2009 at 12:06 AM, max3000 maxime.lar...@gmail.com wrote: Hi, I find the laziness in clojure very hard to wrap my head around. I understand the idea and it's probably nice in theory. However, in real life it doesn't seem really useful beyond hardcore mathematical problems. Case in point, I just spent 2 hours debugging a piece of code (shown below) that seemed simple enough. This is the 3rd time this week that I've lost substantial time to laziness. I'm pretty pissed to tell the truth and I find myself wrapping things in doseq more and more just to be sure. I rarely use 'for' anymore, what's the point? Here is the code that gave me trouble: (map #(add-watch % watcher callback-fn) all-agents) This was not executing. I had to change it to the below expression: (doseq [agent all-labor-agents] (add-watch agent total-labor-agent callback-fn)) This second expression seems less elegant than the map above. Why doesn't clojure realize that an add-watch really should actually loop over all-agents? Why is it that Java calls are not made in similar expressions? Is laziness so useful that we should waste time investigating and fixing errors like this? Sure, there could be special constructs for laziness when we really need it. However, clojure shouldn't default to it IMO. At this point, laziness is a nice concept but it feels somewhat removed from reality to tell the truth. Of course I want to iterate over my collection when I'm doing an add-watch! What am I missing? Thanks, Max --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Laziness madness
Hello, The way I think about it is am I in a portion of code that does pure functional stuff, or am I doing side effects ? A pure functional stuff resembles a computation, but not the storage part, or the part that gets the stuff from the outside (that is, no input / output stuff). doing side effects is when your function not only responds to a call to it by reading its input data and computing a new one that it returns, but also when it reads the input data, does some stuff to change the world (by accessing functions that will permanently change the world), and also (maybe) return a value. In this case, generally, comes with the non pure stuff the need to control the flow of execution of your program. That is : function A must be called before function B, and the effects of A on the world must have been realized before function B can run. In your case, you're calling add-watch which is clearly (to me) a side-effect function because it adds watchers to a long list of watchees by modifying them in place (side-effect). Generally speaking, if this talk about side-effect free function is still a little bit unclear to you, here are some smells to guess if a given function is pure or not : - does the function return a value (different from nil) ? If no - probably a function with side effect .. or a function that really does nothing ! - does the function return newly computed values ? If no - probably a function with side effect .. or a function that really does nothing ! (please note that the inverse is not true : you can't say that a function is pure just because it returns a new computed value of something) I would like to add a third rule to this list, but currently you will not encounter it for a lot of cases, yet it is correct in a number of case and I think it applies correctly to those cases : - does the function implementation use calls to functions ending with a ! ( e.g. set!) : those functions are performing side effects by permanently changing the value of something in the world. HTH, -- Laurent 2009/3/2 max3000 maxime.lar...@gmail.com Hi, I find the laziness in clojure very hard to wrap my head around. I understand the idea and it's probably nice in theory. However, in real life it doesn't seem really useful beyond hardcore mathematical problems. Case in point, I just spent 2 hours debugging a piece of code (shown below) that seemed simple enough. This is the 3rd time this week that I've lost substantial time to laziness. I'm pretty pissed to tell the truth and I find myself wrapping things in doseq more and more just to be sure. I rarely use 'for' anymore, what's the point? Here is the code that gave me trouble: (map #(add-watch % watcher callback-fn) all-agents) This was not executing. I had to change it to the below expression: (doseq [agent all-labor-agents] (add-watch agent total-labor-agent callback-fn)) This second expression seems less elegant than the map above. Why doesn't clojure realize that an add-watch really should actually loop over all-agents? Why is it that Java calls are not made in similar expressions? Is laziness so useful that we should waste time investigating and fixing errors like this? Sure, there could be special constructs for laziness when we really need it. However, clojure shouldn't default to it IMO. At this point, laziness is a nice concept but it feels somewhat removed from reality to tell the truth. Of course I want to iterate over my collection when I'm doing an add-watch! What am I missing? Thanks, Max --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Laziness madness
Quite a frequent occurrence, welcome to the club! One simple check that the Clojure compiler could make is to produce a warning if any lazy result is discarded. I think that would catch 90% of the problems and give the user an immediate clue what to investigate. Well it sounds simple, but I have no idea where to start implementing it. To me the big win of laziness is dealing with large collections, which would otherwise break. Regards, Tim. On Mar 2, 4:06 pm, max3000 maxime.lar...@gmail.com wrote: Hi, I find the laziness in clojure very hard to wrap my head around. I understand the idea and it's probably nice in theory. However, in real life it doesn't seem really useful beyond hardcore mathematical problems. Case in point, I just spent 2 hours debugging a piece of code (shown below) that seemed simple enough. This is the 3rd time this week that I've lost substantial time to laziness. I'm pretty pissed to tell the truth and I find myself wrapping things in doseq more and more just to be sure. I rarely use 'for' anymore, what's the point? Here is the code that gave me trouble: (map #(add-watch % watcher callback-fn) all-agents) This was not executing. I had to change it to the below expression: (doseq [agent all-labor-agents] (add-watch agent total-labor-agent callback-fn)) This second expression seems less elegant than the map above. Why doesn't clojure realize that an add-watch really should actually loop over all-agents? Why is it that Java calls are not made in similar expressions? Is laziness so useful that we should waste time investigating and fixing errors like this? Sure, there could be special constructs for laziness when we really need it. However, clojure shouldn't default to it IMO. At this point, laziness is a nice concept but it feels somewhat removed from reality to tell the truth. Of course I want to iterate over my collection when I'm doing an add-watch! What am I missing? Thanks, Max --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Laziness madness
Laziness hit home for me in Clojure when I noticed that one of our Java library functions, looking for a Collection, had happily accepted a Clojure (map ...) object which was going to expand a small set of integers into much larger objects. Because Clojure deferred the work, they were only generated when they were actually needed, and saved memory the rest of the time. Thanks, Stu On Mon, Mar 2, 2009 at 1:17 AM, Laurent PETIT laurent.pe...@gmail.comwrote: Hello, The way I think about it is am I in a portion of code that does pure functional stuff, or am I doing side effects ? A pure functional stuff resembles a computation, but not the storage part, or the part that gets the stuff from the outside (that is, no input / output stuff). doing side effects is when your function not only responds to a call to it by reading its input data and computing a new one that it returns, but also when it reads the input data, does some stuff to change the world (by accessing functions that will permanently change the world), and also (maybe) return a value. In this case, generally, comes with the non pure stuff the need to control the flow of execution of your program. That is : function A must be called before function B, and the effects of A on the world must have been realized before function B can run. In your case, you're calling add-watch which is clearly (to me) a side-effect function because it adds watchers to a long list of watchees by modifying them in place (side-effect). Generally speaking, if this talk about side-effect free function is still a little bit unclear to you, here are some smells to guess if a given function is pure or not : - does the function return a value (different from nil) ? If no - probably a function with side effect .. or a function that really does nothing ! - does the function return newly computed values ? If no - probably a function with side effect .. or a function that really does nothing ! (please note that the inverse is not true : you can't say that a function is pure just because it returns a new computed value of something) I would like to add a third rule to this list, but currently you will not encounter it for a lot of cases, yet it is correct in a number of case and I think it applies correctly to those cases : - does the function implementation use calls to functions ending with a ! ( e.g. set!) : those functions are performing side effects by permanently changing the value of something in the world. HTH, -- Laurent 2009/3/2 max3000 maxime.lar...@gmail.com Hi, I find the laziness in clojure very hard to wrap my head around. I understand the idea and it's probably nice in theory. However, in real life it doesn't seem really useful beyond hardcore mathematical problems. Case in point, I just spent 2 hours debugging a piece of code (shown below) that seemed simple enough. This is the 3rd time this week that I've lost substantial time to laziness. I'm pretty pissed to tell the truth and I find myself wrapping things in doseq more and more just to be sure. I rarely use 'for' anymore, what's the point? Here is the code that gave me trouble: (map #(add-watch % watcher callback-fn) all-agents) This was not executing. I had to change it to the below expression: (doseq [agent all-labor-agents] (add-watch agent total-labor-agent callback-fn)) This second expression seems less elegant than the map above. Why doesn't clojure realize that an add-watch really should actually loop over all-agents? Why is it that Java calls are not made in similar expressions? Is laziness so useful that we should waste time investigating and fixing errors like this? Sure, there could be special constructs for laziness when we really need it. However, clojure shouldn't default to it IMO. At this point, laziness is a nice concept but it feels somewhat removed from reality to tell the truth. Of course I want to iterate over my collection when I'm doing an add-watch! What am I missing? Thanks, Max --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
