Re: Poll for a new name for clojure eclipse plugin 'clojure-dev'
There's going to be a solar eclipse in a couple of weeks visible from lots of places with great names: Varanasi, Darjeeling (has a j in it!) and Surat, for example. An alignment of three cosmic bodies is a syzygy. Many eclipse cycles seem to have neat names: http://www.phys.uu.nl/~vgent/eclipse/eclipsecycles.htm On Sat, Jul 11, 2009 at 1:48 PM, beatle...@gmail.combeatle...@gmail.com wrote: I like Corona as well. Maybe taking it one little step further and turn it into Clorona, which has a more clojurish sound to it. On Jun 24, 1:47 am, Rayne disciplera...@gmail.com wrote: I vote Corona. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Incanter classpath difficulties
On Mon, Jun 22, 2009 at 2:41 AM, liebke lie...@gmail.com wrote: Hi Anand, Try changing the INCANTER_HOME variable in the clj script to the absolute path to the incanter directory, instead of a relative path, like ./ or ../ Let me know if that doesn't work. Hi David, Didn't work, unfortunately. I can still import the required class directly, but (use '(incanter core)) fails. Thanks again, Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Incanter classpath difficulties
Hi David, no worries, thanks for trying. I'll let you know if I find a solution. Cheers Anand On Mon, Jun 22, 2009 at 1:41 PM, liebke lie...@gmail.com wrote: Anand, you've stumped me! You can import the class directly, but you can't load incanter.core because it can't import the class? I'm not sure what would cause that behavior, and I can't reproduce it. Whenever I encounter weird errors like this, I try rebooting. You might also try cloning incanter in a fresh directory and see it if has the same behavior. Sorry, I can't be more helpful. good luck, David On Jun 22, 4:43 am, Anand Patil anand.prabhakar.pa...@gmail.com wrote: On Mon, Jun 22, 2009 at 2:41 AM, liebke lie...@gmail.com wrote: Hi Anand, Try changing the INCANTER_HOME variable in the clj script to the absolute path to the incanter directory, instead of a relative path, like ./ or ../ Let me know if that doesn't work. Hi David, Didn't work, unfortunately. I can still import the required class directly, but (use '(incanter core)) fails. Thanks again, Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Incanter classpath difficulties
OK, figured it out... the problem was I had copied incanter.clj to /Library/Java/Extensions. When I removed that copy, everything worked as expected. Weird, but I'm glad it's not happening anymore. Anand On Mon, Jun 22, 2009 at 1:52 PM, Anand Patil anand.prabhakar.pa...@gmail.com wrote: Hi David, no worries, thanks for trying. I'll let you know if I find a solution. Cheers Anand On Mon, Jun 22, 2009 at 1:41 PM, liebke lie...@gmail.com wrote: Anand, you've stumped me! You can import the class directly, but you can't load incanter.core because it can't import the class? I'm not sure what would cause that behavior, and I can't reproduce it. Whenever I encounter weird errors like this, I try rebooting. You might also try cloning incanter in a fresh directory and see it if has the same behavior. Sorry, I can't be more helpful. good luck, David On Jun 22, 4:43 am, Anand Patil anand.prabhakar.pa...@gmail.com wrote: On Mon, Jun 22, 2009 at 2:41 AM, liebke lie...@gmail.com wrote: Hi Anand, Try changing the INCANTER_HOME variable in the clj script to the absolute path to the incanter directory, instead of a relative path, like ./ or ../ Let me know if that doesn't work. Hi David, Didn't work, unfortunately. I can still import the required class directly, but (use '(incanter core)) fails. Thanks again, Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Clojure in Computing in Science and Engineering
On Sat, Jun 20, 2009 at 11:29 AM, Jon Harrop j...@ffconsultancy.com wrote: The Task Parallel Library. It uses concurrent wait-free work-stealing queues to provide an efficient implementation of work items than can spawn other work items with automatic load balancing on shared memory machines. Cilk uses the same technology (well worth a look if you haven't already seen it!). That makes it easy to write efficient parallel algorithms in any .NET language. In particular, it can sustain billions of work items (as opposed to thousands of threads or processes) and the time taken to spawn is ~30,000x faster than forking a process. Extremely useful stuff! Sounds similar to ForkJoin, which Rich pointed out to me a while ago: http://www.ibm.com/developerworks/java/library/j-jtp11137.html Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Incanter classpath difficulties
Hi David and all, Just trying to get up and running with Incanter, on a Mac with the github head. The clj script doesn't find part of parallel colt: (head-mac bin) ./clj Clojure 1.1.0-alpha-SNAPSHOT user= (use '(incanter core)) java.lang.NoClassDefFoundError: cern/colt/matrix/tdouble/impl/DenseColDoubleMatrix2D (internal.clj:19) user= (import cern.colt.matrix.tdouble.impl.DenseColDoubleMatrix2D) java.lang.ClassNotFoundException: cern.colt.matrix.tdouble.impl.DenseColDoubleMatrix2D (NO_SOURCE_FILE:2) user= When I change INCANTER_HOME=. to INCANTER_HOME=.. in the clj script, I can import the required class but still not use Incanter: (head-mac bin) ./clj Clojure 1.1.0-alpha-SNAPSHOT user= (import cern.colt.matrix.tdouble.impl.DenseColDoubleMatrix2D) cern.colt.matrix.tdouble.impl.DenseColDoubleMatrix2D user= (use '(incanter core)) java.lang.NoClassDefFoundError: cern/colt/matrix/tdouble/impl/DenseColDoubleMatrix2D (internal.clj:19) user= The problem seems to be in the Matrix class: user= (import incanter.Matrix) java.lang.NoClassDefFoundError: cern/colt/matrix/tdouble/impl/DenseColDoubleMatrix2D (NO_SOURCE_FILE:3) Thanks in advance for any help, Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Clojure in Computing in Science and Engineering
On Thu, Jun 18, 2009 at 5:48 PM, Konrad Hinsen konrad.hin...@laposte.netwrote: On 18.06.2009, at 16:47, psf wrote: That is funny... I put a little Clojure plug into the article entitled Trailblazing with Roadrunner in the very same issue of CiSE. Great minds think alike ;-) On 18.06.2009, at 18:16, Michel Salim wrote: Really neat -- hopefully there will be a follow-up that demonstrates Clojure's concurrency features (perhaps contrasting it with Haskell). The problem is that neither one is particularly well suited for the majority of scientific applications, which work best on distributed- memory machines. Of course this may change with the increasing number of cores-per-processor, shared-memory SMP may become fashionable again even for number crunching. Anyway, if anyone has a scientific (in the widest possible sense) application that exploits Clojure's currency, contact me if you want to write about it! Great article, Konrad. Thanks for giving us something to email to colleagues! I've got something bubbling away on the back burner, but will let the suspense build up a bit ;). Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Feedback on new persistentmatrix datatype
On Sun, May 17, 2009 at 1:15 PM, Konrad Hinsen konrad.hin...@laposte.netwrote: On 16.05.2009, at 15:53, aperotte wrote: Yes Anand, I'm worried about that. What I think the solution should be is to allow mutability in the implementation of algorithms in the java back end for the reasons you mentioned, but a clean immutable interface on the clojure side. When users are faced with serious memory limitations, though, that could be a problem. The question is whether that sacrifice is worth the cleanliness of immutability. A detailed explanation is given in Philip Wadler's paper Monads for functional programming (downloadable from http:// homepages.inf.ed.ac.uk/wadler/topics/monads.html). The principle is to have a specialized version of the state monad in which the state is the array. The only way to get an array into the state is by creating a new one immediately as a state monad value (meaning there is no way to access the array otherwise). It can then be modified using a set of monadic expressions that can be combined arbitrarily. They guarantee that updates happen in the specified order without permitting any other access to the array, meaning that the updates can be made destructively without causing any problems. Huh, sounds like just the thing. Security is going to be especially difficult in Clojure, though. For example, say I stuck the array into a ref from within the monad, then carried on overwriting it. A consumer who gets the array out of the ref from another thread might get it while I'm still mutating it. Granted that would be a pretty stupid thing for me to do, but it would be nice to guarantee immutability to consumers. Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Feedback on new persistentmatrix datatype
On Mon, May 18, 2009 at 10:54 AM, Konrad Hinsen konrad.hin...@laposte.netwrote: On May 18, 2009, at 11:21, Anand Patil wrote: Huh, sounds like just the thing. Security is going to be especially difficult in Clojure, though. For example, say I stuck the array into a ref from within the monad, then carried on overwriting it. A consumer who gets the array out of the ref from another thread might get it while I'm still mutating it. Here is an example of how this might look, using 1D Java arrays for simplicity: (def an-array (initalize-array Integer/TYPE 10 [_ (set-element 0 1) x (get-element 0) _ (set-element 1 (inc x)])) I understand better now, thanks. That seems pretty much ideal to me. Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Feedback on new persistentmatrix datatype
On Sat, May 9, 2009 at 7:09 PM, aperotte apero...@gmail.com wrote: It shouldn't be a problem to maintain immutability and also perform a cross/cartesian product. I'm not sure I understand the problem. It was a pretty bad example... what I meant was, in scientific computing, people often have to take a deep breath and mutate because of speed or memory limitations. It would be nice to be able to 'unlock' the immutability somehow if you really need to. Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Feedback on new persistentmatrix datatype
On Sat, May 2, 2009 at 11:19 PM, aperotte apero...@gmail.com wrote: Hello everyone, I just uploaded some of my work on a new datatype for clojure to a git repository. http://github.com/aperotte/persistentmatrix A bit of the rationale and motivation for the datatype is described on the github page. I basically wanted to create a datastructure for manipulating large amounts of data efficiently and in a human friendly way in clojure. Its main features are: 1. Immutability 2. Primitive Support 3. N-Dimensional – Arbitrary number and size of dimensions (ie. ability to create a 4×3×5x6 datastructure) 4. N-Dimensional units (ie. ability to create a 10×10 matrix with 2×1 units to represent complex numbers) 5. Fast submatrix views via structural sharing (ie. constant time slicing, transposing, and other data manipulations) 6. Maintenance of both deep and superficial dimensionality (ie. slicing a 4×3×5x6 along the 3rd dimension will yield a datastructure with a superficial dimensionality of 3 and a deep dimensionality of 4) 7. Axis and element labeling and label indexing (ie. ability to label axes and elements of an axis with strings or any arbitrary object) 8. Implementing many of the clojure interfaces and thereby automatically inheriting much of the functionality of the standard library for data structures. I would welcome any feedback. Also, if anyone is interested in working together to accelerate its development, that would be welcome too! Great work! I'm glad to see some numerical advances in Clojure. I've got a couple of suggestions: first, you might want to follow numpy and call n-dimensional arrays 'arrays', and reserve 'matrix' for 2d arrays. Or not... Second, immutability is definitely the right default, but it would be nice to be able to create nonstandard arrays from Clojure somehow. For example, say I wanted to compute the matrix (f (- x y)) over the Cartesian product of vectors x and y. Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: March 20th 2009 Rich Hickey Appreciation Day!
Clojure's great, thanks very much for making it available supporting it! Anand On Fri, Mar 20, 2009 at 6:26 PM, Rayne disciplera...@gmail.com wrote: I Anthony Simpson, with the support of fellow Clojurists hereby declare March 20th, the first day of spring, Rich Hickey appreciation day! Rich Hickey has certainly done a lot for us, making this wonderful language and continuing to take his time to work on it. He is dedicated and he wants to bring Clojure along with it's users to heights that Lisp has never been before. In just some 2 years, Rich has gathered together a vibrant and large community of users and contributors who believe in Clojure's future immensely . One such contributor who believes in Clojure enough to write an entire book on it! Clojure gains more attention and support everyday. I believe in the bright future that Rich Hickey believes exists for Clojure. I believe that with this community, and with such a wonderful creator as Rich Hickey, Clojure will achieve it's goals and meet the destiny that Rich is writing for it. I thank you Rich Hicky for all your work on Clojure. I thank you for all the time you've spent building this community and giving us one of the most awesome languages that have existed. Thank you for caring about us enough to listen to the community before making big changes, I believe in Clojure and I will be here watching it evolve with you! If you would like to thank Rich Hickey for all he has done for us, you can post in this thread, or tell him yourself in the #Clojure IRC channel. :) March 20th 2009 Rich Hickey Appreciation Day -Rayne --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Synchronous watches
Excellent point. Yes, get-watches seems like a better option.
Anand
On Thu, Mar 5, 2009 at 3:33 AM, Timothy Pratley timothyprat...@gmail.comwrote:
I forgot to mention the reason I don't feel it should be an error to
remove a non-existent watch
user= (dissoc {:a 1 :b 2} :c)
{:a 1, :b 2}
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Clojure group.
To post to this group, send email to clojure@googlegroups.com
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---
Re: Synchronous watches
I've managed to create weird race conditions by adding watches while actions
are running. Are there guarantees about the scheduling of actions and
adding/ removing watches? If I add or remove a watch on an agent while an
action is running, what happens?
So to summarize, my feedback that still may be sensible consists of this
question and a request for watches that respond to errors.
Thanks, Anand
On Thu, Mar 5, 2009 at 9:34 AM, Anand Patil anand.prabhakar.pa...@gmail.com
wrote:
Excellent point. Yes, get-watches seems like a better option.
Anand
On Thu, Mar 5, 2009 at 3:33 AM, Timothy Pratley
timothyprat...@gmail.comwrote:
I forgot to mention the reason I don't feel it should be an error to
remove a non-existent watch
user= (dissoc {:a 1 :b 2} :c)
{:a 1, :b 2}
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Clojure group.
To post to this group, send email to clojure@googlegroups.com
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---
Re: Metadata on functions
On Wed, Mar 4, 2009 at 12:50 PM, Konrad Hinsen konrad.hin...@laposte.netwrote: Rich, is there a reason why metadata is explicitly disabled on function objects? I find myself wanting to put metadata on functions frequently. It seems even more important for functions than for anything else, given that there is no way to inspect a function object at all, it's a complete black box. So I looked at the implementation to see if it would be easy to add metadata handling, and found to my surprise that function objects already have a metadata pointer but that setting it is disabled. Two situations where I think metadata on functions would be very useful: - To put on a type tag. If functions are to be first-class data objects, it should be possible to have multimethods dispatching on a function argument. My concrete use case is stream-generating functions with different interfaces that I would like to use with a common interface defined as a multimethod. - For implementing automatic derivatives, it would be useful to attach a derivative function to a mathematical function such as log or sqrt. Konrad. +1. Also, are there any plans to allow function introspection in the future? Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Synchronous watches
On Wed, Mar 4, 2009 at 10:35 AM, Anand Patil anand.prabhakar.pa...@gmail.com wrote: Hi Rich, I like the flexibility of the new watches, but I'm missing a way to watch for errors. Currently, if an agent's action results in an error its watchers aren't even triggered. Thanks, Anand Also, (remove-watch a key) fails to raise an error if a doesn't have any watches associated with key. Anand On Feb 27, 2:57 pm, Rich Hickey richhic...@gmail.com wrote: I've added (back)synchronouswatches(svn 1309+), which used to exist for agents, now for all reference types. (defn add-watch Experimental. Adds a watch function to an agent/atom/var/ref reference. The watch fn must be a fn of 4 args: a key, the reference, its old-state, its new-state. Whenever the reference's state might have been changed, any registeredwatcheswill have their functions called. The watch fn will be called synchronously, on the agent's thread if an agent, before any pending sends if agent or ref. Note that an atom's or ref's state may have changed again prior to the fn call, so use old-state rather than derefing the reference. Note also that watch fns may be called from multiple threads simultaneously. Var watchers are triggered only by root binding changes, not thread-local set!s [reference key fn] ...) (defn remove-watch [reference key]...) I've left in add/remove-watcher for now, in order to avoid disruption. It is defined in terms of add-watch, demonstrating add-watch is the more flexible core function. My plan is to remove add-watcher/remove- watcher, since you can build your own such variants. Highlights are: - Deterministic times when callbacks will occur. - Provision of before/after state - you determine what constitutes an interesting change. - Arbitrary key will be passed back to callback. Please try them out for your latest Cells-like and other reactive programs. I'd like to move this out of the experimental category. Feedback and questions welcome, Rich --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: So what do we want for a Cells-alike ?
On Mar 3, 1:04 pm, Rich Hickey richhic...@gmail.com wrote:
I think it is important to embrace asynchrony and concurrency as
Clojure does. Any Cells-for-Clojure that presumes the world is a
single synchronous chain of events would be a misfit. The whole notion
of a 'current step' is suspect.
I'd appreciate it if my asynchronous lazy cells got a look, as I've
given them a few revisions now. They're at
http://github.com/onyin/lazy-agent/tree.
They don't have any global counter, but the diamond pattern is no
problem for them. As noted in an earlier thread, my def-cell macro is
not as nice as it could be, but it wouldn't be hard for someone who is
good at macros to write a quick wrapper.
The readme:
Implements two types of agent-based 'cells' for Clojure: lazy agents
and oblivious agents. These complement the auto-agents available in
Clojure Contrib. Both allow for concurrent cell updates with
respectably efficient scheduling and avoid unnecessarily repeating
cell updates.
If you deref a lazy cell, you'll see a map: {:value xxx :status
yyy}. :status may be :needs-update, :updating, :up-to-date
or :oblivious. If a cell is up-to-date or oblivious, :value gives the
value of the cell.
When a lazy agent's ancestor changes, its value changes to {:value
nil :status :needs-update} but it does not compute its new value until
it receives a message instructing it to do so. To send the update
message to a group of agents, do (update a b c d e). To send the
update message and wait for the values, do (evaluate a b c d e).
Oblivious agents are even lazier than lazy agents. When an oblivious
agent is up-to-date, its status is :oblivious. If an ancestor
subsequently changes, the oblivious agent will not do anything. It
needs to receive a 'force-need-update' message to change state to
{:value nil :status :needs-update}, but then it starts watching its
parents for changes like a lazy agent until it computes.
Anand
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Clojure group.
To post to this group, send email to clojure@googlegroups.com
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---
Re: So what do we want for a Cells-alike ?
On Mar 3, 3:22 pm, Raffael Cavallaro raffaelcavall...@gmail.com wrote: On Mar 3, 8:04 am, Rich Hickey richhic...@gmail.com wrote: If B and C depend on A in such a way that their values must be coordinated in order to be valid and consistent, then maybe they shouldn't independently depend on A, instead a transaction should fire on changes to A, which in turn updates B and C. Then any downstream consumers of B and C could never see inconsistent values, all without a notion of 'current step'. Certainly you can and should hide the notion of current step for *most* purposes. Using transactions is certainly a good way to do it; but you must expose it in some way or you won't be able to express the full range of dataflow models. Typically: ;; pseudocode (defmodel my-model (a :independent) (b (fn [a] (+ (* a 12) 3))) (c (fn [a] (+ a 7))) (d (fn [c b] (* c d 3.2 Here the step is implicit, as each reference to another cell in the functions above really means the value of that cell at the *same* step/time/data pulse. However: 1. failing to deal with this issue in the *implementation* of a dataflow library leads to the wrong semantics; the absence of this notion or any other means of dealing with it, (such as a transaction as you suggest) commonly leads to bugs in cells-alikes, including at least one clojure implementation. 2. there are certain types of models where one needs to be able to refer to previous pulses/steps. These include reflexive update functions, (where the value of a cell depends on its own value at some previous step/time/pulse), and time lags. With these types of dependencies, there must be some way to explicitly refer to step or pulse: ;pseudocode with time lag ((defmodel my-model (a :independent) (b (fn [a] (previous a 3))) ;; i.e., how many steps back (c (fn [a] (+ a 7))) (d (fn [c d] (* c d 3.2 You can certainly hide the most common case (i.e., everything is at the same step/pulse), but it has to be there internally or the semantics will be wrong, and it has to be exposed in some way or you won't be able to express many interesting types of models in a simple, direct way. Laziness can work like a step counter, too. When you update a group of the lazy cells in my previous post, you cause a single data pulse. Cells don't update until all of their parents have reported in, ie reached the current step. Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Making agents able to temporarily yield place on thread pool
On Mar 2, 1:16 am, MikeM michael.messini...@invista.com wrote: Not sure if I understand what you need, but could you build on the existing capability to send to the current agent: (send *agent* ...) ? You could have the agent send to itself, then exit the function with some work left to do that would be restarted on the next go-around. I could get away with sending to b with instructions to send back to a... the yield suggestion was just syntactic sugar for doing that. Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Making agents able to temporarily yield place on thread pool
On Mar 1, 10:58 pm, Rich Hickey richhic...@gmail.com wrote: On Mar 1, 2009, at 1:53 PM, Anand Patil wrote: I've made futures use the Agents' CachedThreadPool, which should prevent the thread pool exhaustion (svn 1316). You shouldn't worry about the expense of the threads, that's why there's a pool. Yield as you've described it is definitely not going to happen. Rich, If you'll bear with me a bit longer- what if I set breadth=1000 and depth=1000 in the program above. The thread pool would have to grow by thousands to avoid a deadlock, whereas with something like yield it could stay alive using only the standard, fixed-size thread pool. Anand Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Making agents able to temporarily yield place on thread pool
On Mar 2, 12:48 pm, Rich Hickey richhic...@gmail.com wrote: On Mar 2, 2009, at 5:15 AM, Anand Patil wrote: On Mar 1, 10:58 pm, Rich Hickey richhic...@gmail.com wrote: On Mar 1, 2009, at 1:53 PM, Anand Patil wrote: I've made futures use the Agents' CachedThreadPool, which should prevent the thread pool exhaustion (svn 1316). You shouldn't worry about the expense of the threads, that's why there's a pool. Yield as you've described it is definitely not going to happen. Rich, If you'll bear with me a bit longer- what if I set breadth=1000 and depth=1000 in the program above. The thread pool would have to grow by thousands to avoid a deadlock, whereas with something like yield it could stay alive using only the standard, fixed-size thread pool. I understand the problem and there is already a solution - it's called the Fork/Join framework, and you can use it easily from Clojure: http://www.ibm.com/developerworks/java/library/j-jtp11137.htmlhttp://gee.cs.oswego.edu/dl/concurrency-interest/index.html Good to know- I'll have a look at Fork/Join. Thanks, Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Making agents able to temporarily yield place on thread pool
On Mar 2, 2:34 pm, Anand Patil anand.prabhakar.pa...@gmail.com wrote: On Mar 2, 12:48 pm, Rich Hickey richhic...@gmail.com wrote: On Mar 2, 2009, at 5:15 AM, Anand Patil wrote: On Mar 1, 10:58 pm, Rich Hickey richhic...@gmail.com wrote: If you'll bear with me a bit longer- what if I set breadth=1000 and depth=1000 in the program above. The thread pool would have to grow by thousands to avoid a deadlock, whereas with something like yield it could stay alive using only the standard, fixed-size thread pool. I understand the problem and there is already a solution - it's called the Fork/Join framework, and you can use it easily from Clojure: http://www.ibm.com/developerworks/java/library/j-jtp11137.htmlhttp://... Good to know- I'll have a look at Fork/Join. OK, ForkJoin really does solve exactly that problem. Nice. Out of curiosity, what's your vision for how ForkJoin would ideally interoperate with Clojure's own agents and futures? Specifically, would they share a thread pool, and would the availability of ForkJoin actually change the behavior of agents and futures at all? Thanks, Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Making agents able to temporarily yield place on thread pool
Hi all, My concurrent dabblings in Clojure have been a real pleasure so far. In terms of concurrency, it's just in a completely different league from any other language I've tried. However, I think agents could be made even more friendly by allowing them to temporarily surrender their place in the thread pool pending another computation. Specifically, I'm thinking of a function 'yield' called as (yield other-agent msg-fn args) from within agent actions. If agent a yields to agent b, a gets off the thread pool and msg is sent to b with given args. When b finishes executing the message, the yield call returns b's new value, and a gets back on the thread pool. I _think_ the language could prevent deadlocks by: - making agents not take any messages while they are 'yielded' - keeping a 'yield stack' and checking that no cycles happen. Here are two use cases for yield: 1) Say an 'auto-agent' cell needs to run an expensive, parallelizable computation to update its value. Without yield you could do this by dispatching the action with send-off, or by splitting the cell up into multiple cells, each handling a chunk of the computation. Yield would be nicer than both: send-off spawns an expensive kernel thread (as I understand it) and breaking up the cell complicates the dataflow model unnecessarily. 2) Currently this program, which creates a tree of dependent futures, deadlocks on my 8-core mac pro: (def breadth 4) (defn with-deref [fun] (fn [ x] (apply fun (map deref x (defn future-tree [depth] (if (= depth 0) (future 1) (let [new-futures (map future-tree (repeat breadth (- depth 1)))] (future (apply (with-deref +) new-futures) ; WARNING: This deadlocks on 8-core Mac Pro! (future-tree 4) I'm guessing it deadlocks because the thread pool fills up with futures that aren't ready to go, but it feels like the language should be able to avoid the deadlock without requiring the programmer to think about how many threads are actually in the thread pool. If futures were based on agents, which yielded when deref-ing other futures, the deadlock wouldn't happen. I have no idea how hard it would be to implement yield, as I have very little practice with Java. Thoughts? Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Synchronous watches
Hi Rich, On Feb 27, 2:57 pm, Rich Hickey richhic...@gmail.com wrote: I've added (back) synchronous watches (svn 1309+), which used to exist for agents, now for all reference types. (defn add-watch Experimental. Adds a watch function to an agent/atom/var/ref reference. The watch fn must be a fn of 4 args: a key, the reference, its old-state, its new-state. Could you say more about the key? Who gets to decide what it is, and what does it mean? Thanks, Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Synchronous watches
Never mind, I get it now. Thanks, Anand On Feb 27, 4:32 pm, Anand Patil anand.prabhakar.pa...@gmail.com wrote: Hi Rich, On Feb 27, 2:57 pm, Rich Hickey richhic...@gmail.com wrote: I've added (back) synchronous watches (svn 1309+), which used to exist for agents, now for all reference types. (defn add-watch Experimental. Adds a watch function to an agent/atom/var/ref reference. The watch fn must be a fn of 4 args: a key, the reference, its old-state, its new-state. Could you say more about the key? Who gets to decide what it is, and what does it mean? Thanks, Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Richer 'partial'
Hi all, I could use a version of 'partial' that would allow me to: - Partially apply a function to any of its arguments, not just the first one - 'Unapply' a partially-applied function from one of its arguments. Is any such thing already available? Thanks, Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Richer 'partial'
Thanks for the responses, guys. - Partially apply a function to any of its arguments, not just the first one That's already the case, haven't you made a little test ? I meant I want to apply it out of sequence, sorry. You can easily partially apply to other arguments by doing this: #(fred %1 some-arg %2 other-arg). That makes sense, thanks, though it's not any easier to unapply than partial. I guess I could make a new 'partial-like' function that returns code to apply a function, with nil's in unapplied argument slots... ? Not possible as far as I know. Could you please explain use cases you have in mind for such a feature ? Sure, it's the lazy cells I've been working on. When a cell's parent changes, I don't want it to compute right away- I want it to register the fact that the parent has changed, switch to an 'unevaluated' state, and 'unapply' its childrens' update functions on itself. I won't explain how I'm doing it right now for fear of muddying the waters, but it feels overcomplicated and un-idiomatic. Thanks, Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Richer 'partial'
Thanks for the responses, guys. - Partially apply a function to any of its arguments, not just the first one That's already the case, haven't you made a little test ? I meant I want to apply it out of sequence, sorry. You can easily partially apply to other arguments by doing this: #(fred %1 some-arg %2 other-arg). That makes sense, thanks, though it's not any easier to unapply than partial. I guess I could make a new 'partial-like' function that returns code to apply a function, with nil's in unapplied argument slots... ? Not possible as far as I know. Could you please explain use cases you have in mind for such a feature ? Sure, it's the lazy cells I've been working on. When a cell's parent changes, I don't want it to compute right away- I want it to recognize the new value, but switch to an un-evaluated state. Since the cell has no value, it would be nice if its children could just un-apply their update functions on it. I won't explain how I'm doing it right now for fear of muddying the waters, but it feels overcomplicated and un-idiomatic. Thanks, Anand On Feb 26, 4:00 pm, Laurent PETIT laurent.pe...@gmail.com wrote: 2009/2/26 Jeffrey Straszheim straszheimjeff...@gmail.com partial is a currying function. It can be provided any number of parameter(s), but it is always behaves sequentially from start to finish. That is what currying *is*. Ah, I thought currying / uncurrying what the term reserved for this operation (as I remember from Haskell) that changes the arity of functions ( transforming and untransforming a function of n parameters into a function of a tuple of n elements) ? You can easily partially apply to other arguments by doing this: #(fred %1 some-arg %2 other-arg). partial could not easily support unapplying it. On Thu, Feb 26, 2009 at 10:24 AM, Anand Patil anand.prabhakar.pa...@gmail.com wrote: Hi all, I could use a version of 'partial' that would allow me to: - Partially apply a function to any of its arguments, not just the first one - 'Unapply' a partially-applied function from one of its arguments. Is any such thing already available? Thanks, Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Richer 'partial'
Sorry, I thought I had pushed 'stop' in time to stop the first response. On Feb 26, 4:16 pm, Anand Patil anand.prabhakar.pa...@gmail.com wrote: Thanks for the responses, guys. - Partially apply a function to any of its arguments, not just the first one That's already the case, haven't you made a little test ? I meant I want to apply it out of sequence, sorry. You can easily partially apply to other arguments by doing this: #(fred %1 some-arg %2 other-arg). That makes sense, thanks, though it's not any easier to unapply than partial. I guess I could make a new 'partial-like' function that returns code to apply a function, with nil's in unapplied argument slots... ? Not possible as far as I know. Could you please explain use cases you have in mind for such a feature ? Sure, it's the lazy cells I've been working on. When a cell's parent changes, I don't want it to compute right away- I want it to recognize the new value, but switch to an un-evaluated state. Since the cell has no value, it would be nice if its children could just un-apply their update functions on it. I won't explain how I'm doing it right now for fear of muddying the waters, but it feels overcomplicated and un-idiomatic. Thanks, Anand On Feb 26, 4:00 pm, Laurent PETIT laurent.pe...@gmail.com wrote: 2009/2/26 Jeffrey Straszheim straszheimjeff...@gmail.com partial is a currying function. It can be provided any number of parameter(s), but it is always behaves sequentially from start to finish. That is what currying *is*. Ah, I thought currying / uncurrying what the term reserved for this operation (as I remember from Haskell) that changes the arity of functions ( transforming and untransforming a function of n parameters into a function of a tuple of n elements) ? You can easily partially apply to other arguments by doing this: #(fred %1 some-arg %2 other-arg). partial could not easily support unapplying it. On Thu, Feb 26, 2009 at 10:24 AM, Anand Patil anand.prabhakar.pa...@gmail.com wrote: Hi all, I could use a version of 'partial' that would allow me to: - Partially apply a function to any of its arguments, not just the first one - 'Unapply' a partially-applied function from one of its arguments. Is any such thing already available? Thanks, Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Richer 'partial'
On Feb 26, 4:41 pm, mikel mev...@mac.com wrote: Other people have explained currying and partial application, and why it doesn't normally spply the feature you want. I'd be interested in reading about this if you know of a link. Normally, in a lnaguage that supplies partial application, the way to achieve the effect you want is to use combinators that change the order of arguments. OK, that sounds like a clean functional way to do it. Thanks. Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: How to get rid of reflection warning from countdown latch?
Thanks Miʃel, On Feb 23, 10:15 pm, Michel Salim michel.syl...@gmail.com wrote: What's the object on which .countDown is called? You need to find where it's first declared and give it a type annotation. It's created here: let [ latch (java.util.concurrent.CountDownLatch. n) ... Sorry if I'm being dense, but do you mean I should annotate it in the arguments list of the function where it's used, like so: (defn unlatching-watcher [#^java.util.concurrent.CountDownLatch latch cell cell-updated?] A watcher function that decrements a latch when a cell updates. (if cell-updated? (if (not (:updating @cell)) (.countDown latch or annotate in when it's actually created? If the latter, what's the syntax for that? Thanks, Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: How to get rid of reflection warning from countdown latch?
On Feb 24, 1:32 pm, Michel Salim michel.syl...@gmail.com wrote: On Tue, Feb 24, 2009 at 4:44 AM, Anand Patil anand.prabhakar.pa...@gmail.com wrote: Thanks Miʃel, On Feb 23, 10:15 pm, Michel Salim michel.syl...@gmail.com wrote: What's the object on which .countDown is called? You need to find where it's first declared and give it a type annotation. It's created here: let [ latch (java.util.concurrent.CountDownLatch. n) ... Sorry if I'm being dense, but do you mean I should annotate it in the arguments list of the function where it's used, like so: (defn unlatching-watcher [#^java.util.concurrent.CountDownLatch latch cell cell-updated?] A watcher function that decrements a latch when a cell updates. (if cell-updated? (if (not (:updating @cell)) (.countDown latch or annotate in when it's actually created? If the latter, what's the syntax for that? I was unclear in my hint, apologies. You need a type hint in the function that takes a CountDownLatch, otherwise Clojure would not know what it is until you try to call a method on it (and it has to do reflection) Annotating in the arguments list, as you showed in the example, is fine. This way, you only annotate once for the entire function (I didn't know you can do (.method #^classname foo) before, actually, that was neat) Thanks for the help! Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
How to get rid of reflection warning from countdown latch?
Hi all, I'm getting Reflection warning, line: 150 - reference to field countDown can't be resolved. from (if cell-updated? (if (not (:updating @cell)) (.countDown latch Is there any way to get rid of it? Thanks, Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: How to get rid of reflection warning from countdown latch?
This blind stab worked... is it correct? (if cell-updated? (if (not (:updating @cell)) (.countDown #^java.util.concurrent.CountDownLatch latch Thanks, Anand On Feb 23, 5:20 pm, Anand Patil anand.prabhakar.pa...@gmail.com wrote: Hi all, I'm getting Reflection warning, line: 150 - reference to field countDown can't be resolved. from (if cell-updated? (if (not (:updating @cell)) (.countDown latch Is there any way to get rid of it? Thanks, Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Feedback sought on yet another cells implementation!
Hi Tim, thanks for the feedback!
On Feb 21, 11:16 am, Timothy Pratley timothyprat...@gmail.com wrote:
(1) auto-agents by SS in contrib has a more convenient syntax [maybe
you can mimic it]
Agreed, it is nicer. At the moment 'def-cell' is already a stretch for
me, but maybe someday I'll get there. :)
(2) I can add a watcher to one of your cells - it wont do anything
unless evaluate is called. I can see the advantage of not calculating
needlessly, but if it is not going to be updated dynamically it is a
bit misleading to allow watchers.
The cell changes to {:needs-update true} as soon as one of its
ancestors changes (plus the time it takes for the messages to
propagate). Watchers could make use of that information, no?
(3) I don't see the need for agents and watcher for lazy cells, after
all can't they just be formula which are evaluated/updated in the
correct order? I have a feeling this would be more efficient (seeing
that is your goal).
It's the concurrency. If you have a cell setup like
(def a (agent 0))
(def-cell b fn [a])
(def-cell c fn [a])
the agent-based implementation allows b and c can update concurrently,
whereas if they were just formulas they would of course have to update
sequentially. Or do you have some clever use of pmap in mind?
Either way, I'm planning on using this for cells whose update
functions involve reasonably heavy number-crunching, so I'm hoping
that the concurrency will be worth the overhead it requires.
However the overhead in my implementation is admittedly substantial...
I couldn't figure out a way to keep the number of updates to a minimum
yet keep the updates concurrent that required less than two watchers
per cell.
(4) Both lazy and watchable cells are useful.
Good point... and come to think of it, there's no reason why these
cells shouldn't be able to mix with Stuart Sierra's auto-agents, given
that they're both just agents with watchers.
(5) id? could be just: instance? IDeref x unless I'm missing
something
Thanks for the tip!
(6) case macro - not used in the code, and doesn't cond suffice?
You're right, as of version two case is officially cruft, and anyway
cond does the same job. No problem, it was good macro practice. :-)
That was very helpful, I appreciate it! Please pass along any other
thoughts on this.
Anand
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Clojure group.
To post to this group, send email to clojure@googlegroups.com
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---
Re: Feedback sought on yet another cells implementation!
Hi all,
My second attempt is lazy-cells.clj in this group's files. The cells
are actual agents rather than ugly maps, and all the information about
parents is hidden in watchers.
If you change a cell, the cells that depend on it all change their
values to {:needs-update true}. You can subsequently tell cells a b c
d e to update by calling (update a b d c e), which will change their
values to {:needs-update true :updating true} and eventually an actual
value. To wait for and get the results, do (evaluate a b c d e).
The updates should (hopefully) be as economical as in the first
version.
Thanks,
Anand
On Feb 19, 6:46 pm, Anand Patil anand.prabhakar.pa...@gmail.com
wrote:
Hi all,
I would really appreciate some comments and criticism on
mycellsimplementation, which I've uploaded to this group's files as
dataflow.clj . My goal was to fully exploit the available concurrency,
but not do any unnecessary cell evaluations.
My solution is lazy rather than push-based; if you change a ref
thatcellsdepend on, nothing happens to thecellsuntil you actually ask
for their values. If you call
(evaluate [cell1 cell2 cell3 ...])
first cell1, cell2, cell3, ... and all their ancestors have their
values set to {:updating}, and the rootcellsof the computation (all
ancestors of cell1 etc. whose parents are non-cell identities like
refs) are identified. Then, messages are sent to the rootcells
telling them to recompute their values.
Each time a cell p is able to compute its value, it sends a message to
each of its children c that needs to update. The message extends c's
value, which is a map, with the key and value {p (p's value)}. If c
has received reports from all its cell parents, it retrieves values of
its non-cell parents, computes its value, and reports to its children.
When cell1, cell2, cell3 ... are all able to compute, the computation
is over. You can optionally call
(sync-evaluate [cell1 cell2 cell3 ...])
which blocks until the computation finishes using a Java countdown
latch.
This approach (I think) guarantees that each cell recomputes at most
once per computation. This helps certain models perform much better
than with push-basedcells. For example, think about m generations of
ncellseach, where every cell in a generation is a parent of every
cell in the next generation. Say all the rootcellsdepend on a single
ref. If you change the value of the ref, all nmcellsneed to update.
If you do it with pushes, you first have to update all the rootcells,
for n updates. Then each root cell pushes to all of its children, for
n^2 updates. You end up doing waaay more than nm updates.
You could update the generations sequentially to get back to good
performance... but I feel like my approach will incur less overhead
than finding all of the generations.
The example in the testing section of dataflow.clj is less dramatic.
The dependency graph, with - pointing from parents to child, is:
a b - c
a c - d
a - e
c e - f
The roots are a and b, and each update function takes 1s.
With myimplementation, it runs in 3s with 6 updates:
- a and b update together
- c and e update together
- d and f update together
whereas if you were using a push-based scheme I think it would take 4s
with 9 updates. It would take 4s because:
- a and b update together
- c has to update twice in sequence, once in response to a and once in
response to b
- d and f have to update together in response to c's final update.
Myimplementationruns the test, but:
- It's pretty ugly,
- I'd eventually like it to be fast am sure it's nowhere near its
potential,
- I feel like I'm not taking full advantage of the language.
After the basic idea gets some scrutiny, I'd like to include a system
for handling exceptions. I'll also make thecellsmaintain caches
keyed by their non-cell ancestors. This will save a lot of
coordination overhead in addition to actual computation because if you
evaluate a cell twice in a row it won't have to bother its cell
ancestors at all the second time.
I haven't given much thought to circular dependencies, because I don't
currently need them.
Thanks very much!
Anand
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Clojure group.
To post to this group, send email to clojure@googlegroups.com
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---
Feedback sought on yet another cells implementation!
Hi all,
I would really appreciate some comments and criticism on my cells
implementation, which I've uploaded to this group's files as
dataflow.clj . My goal was to fully exploit the available concurrency,
but not do any unnecessary cell evaluations.
My solution is lazy rather than push-based; if you change a ref that
cells depend on, nothing happens to the cells until you actually ask
for their values. If you call
(evaluate [cell1 cell2 cell3 ...])
first cell1, cell2, cell3, ... and all their ancestors have their
values set to {:updating}, and the root cells of the computation (all
ancestors of cell1 etc. whose parents are non-cell identities like
refs) are identified. Then, messages are sent to the root cells
telling them to recompute their values.
Each time a cell p is able to compute its value, it sends a message to
each of its children c that needs to update. The message extends c's
value, which is a map, with the key and value {p (p's value)}. If c
has received reports from all its cell parents, it retrieves values of
its non-cell parents, computes its value, and reports to its children.
When cell1, cell2, cell3 ... are all able to compute, the computation
is over. You can optionally call
(sync-evaluate [cell1 cell2 cell3 ...])
which blocks until the computation finishes using a Java countdown
latch.
This approach (I think) guarantees that each cell recomputes at most
once per computation. This helps certain models perform much better
than with push-based cells. For example, think about m generations of
n cells each, where every cell in a generation is a parent of every
cell in the next generation. Say all the root cells depend on a single
ref. If you change the value of the ref, all nm cells need to update.
If you do it with pushes, you first have to update all the root cells,
for n updates. Then each root cell pushes to all of its children, for
n^2 updates. You end up doing waaay more than nm updates.
You could update the generations sequentially to get back to good
performance... but I feel like my approach will incur less overhead
than finding all of the generations.
The example in the testing section of dataflow.clj is less dramatic.
The dependency graph, with - pointing from parents to child, is:
a b - c
a c - d
a - e
c e - f
The roots are a and b, and each update function takes 1s.
With my implementation, it runs in 3s with 6 updates:
- a and b update together
- c and e update together
- d and f update together
whereas if you were using a push-based scheme I think it would take 4s
with 9 updates. It would take 4s because:
- a and b update together
- c has to update twice in sequence, once in response to a and once in
response to b
- d and f have to update together in response to c's final update.
My implementation runs the test, but:
- It's pretty ugly,
- I'd eventually like it to be fast am sure it's nowhere near its
potential,
- I feel like I'm not taking full advantage of the language.
After the basic idea gets some scrutiny, I'd like to include a system
for handling exceptions. I'll also make the cells maintain caches
keyed by their non-cell ancestors. This will save a lot of
coordination overhead in addition to actual computation because if you
evaluate a cell twice in a row it won't have to bother its cell
ancestors at all the second time.
I haven't given much thought to circular dependencies, because I don't
currently need them.
Thanks very much!
Anand
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Clojure group.
To post to this group, send email to clojure@googlegroups.com
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---
Pmapping force over treed delays
Hi all, Say I have a collection of delays, some of which need to get others' values in order to compute (there are no cyclic dependencies). If I just pmap force over the entire collection, do I run the risk of filling up a thread pool with operations that aren't ready to go yet? Would it be better to represent the collection as a tree-seq and then pmap force over that? Many thanks, Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Discarding entire transactions
Hello again, In my application, I'll frequently want to quickly discard all the changes made during a transaction involving many refs. I don't want to force the refs to roll back to their values at the beginning of the transaction, I just want to end the transaction immediately and skip the write stage; updates made in other transactions should still 'stick'. What's the most efficient way to do this? Thanks again! Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Discarding entire transactions
On Feb 8, 7:29 pm, Shawn Hoover shawn.hoo...@gmail.com wrote: On Sun, Feb 8, 2009 at 12:23 PM, Anand Patil anand.prabhakar.pa...@gmail.com wrote: Hello again, In my application, I'll frequently want to quickly discard all the changes made during a transaction involving many refs. I don't want to force the refs to roll back to their values at the beginning of the transaction, I just want to end the transaction immediately and skip the write stage; updates made in other transactions should still 'stick'. What's the most efficient way to do this? Thanks again! Anand You can abort a transaction by throwing an exception, but in that case all refs automatically retain their pre-transaction values. You can't partially roll back a transaction, because by definition all changes to refs in a transaction occur or none do. By the way, what do you mean by other transactions? - Transactions running on other threads? Aborting in one thread does not affect refs in other threads. - Transactions that completed earlier in time on the same thread? Those values are not affected by aborting a later transaction. - dosync calling dosync on the same on the same thread? Aborting from an inner dosync aborts the whole stack of dosyncs in that thread. I think your first response covers my case. Here's an example of what I mean: (def a (ref 1)) (def b (ref 1)) ; Do these concurrently, either from separate agents or using pmap (dosync (commute b error-throwing-fn a)) (dosync (commute a + @b)) I want to have the option to abort the first transaction without rolling back the second. Based on what you said, if the first transaction is aborted but the second is not then no matter what a's value will be 2 and b's value will be 1, correct? Thanks for the replies, Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Possible bug in preduce
On Feb 3, 11:09 pm, Rich Hickey richhic...@gmail.com wrote: On Feb 3, 4:43 pm, Anand Patil anand.prabhakar.pa...@gmail.com wrote: No, it's not. as the docs for preduce say:http://clojure.org/api#preduce Also note that (f base an-element) might be performed many times in fact, an arbitrary number of times depending on how many parallel tasks are fired up. So, the function with base value should produce an identity, e.g. (+ 0 x) and (* 1 x), and your q with 0 does not. Ah, thanks. I interpreted the docs as meaning preduce might do this: (q (q (q 2 0) (q 3 0)) (q 1 0)) meaning (f base an-element) is performed many times, but just once per an-element. I didn't understand that f with base needed to be an identity. Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Stupid Java questions
I agree don't mean to complain. I'm just flagging the issues to make it easier for whoever goes through the docs after clojure 1.0 is tagged. Anand On Feb 4, 9:45 am, Zak Wilson zak.wil...@gmail.com wrote: The namespace is correct on clojure.org/api, but there it doesn't mention that it has a dependency that isn't included with Clojure. Clojure has been evolving very quickly, and sometimes the website doesn't keep up. It might be nice if somebody could take charge of making sure the site is up to date, but right now I'd rather keep being thrilled by the fact that the feature I find myself needing today was just added yesterday. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Stupid Java questions
Hi all, I'm using the system java on Mac OS Leopard, and confused about how to get the parallel library working. I've got the necessary jar file on my classpath: sihpc03:clojure anand$ echo $CLASSPATH /usr/local/clojure:/Library/Java/Extensions sihpc03:clojure anand$ ls /Library/Java/Extensions/ clojure.jar jline-0.9.94.jar jsr166y.jar libsvnjavahl-1.jnilib libsvnjavahl.jnilib and I include it when I start up clojure: sihpc03:clojure anand$ java -server -cp jline-0.9.94.jar:jsr166y.jar:clojure.jar jline.ConsoleRunner clojure.lang.Repl However, when I try to load parallel.clj from the clojure source directory, a problem happens: user= (load-file parallel.clj) java.lang.ClassNotFoundException: jsr166y.forkjoin.ParallelArray (parallel.clj:0) How can I fix that? Another question: in the example on the clojure webpage, the load-file command is (load-file src/parallel.clj). To make that work do I have to have clojure's source directory on my classpath, or my regular path, or are the sources available from inside the jar file somehow? I'm sure the answers are available from Java documentation somewhere, but the fact that I'm working from closure adds a thick enough layer of confusion that I considered this worth posting... hope I'm not taking advantage. Thanks, Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Stupid Java questions
OK, I get it: - parallel.clj writes into the namespace 'clojure.parallel, not plain 'parallel as written on clojure.org/other_libraries - parallel.clj needs to be on my path, not my classpath. That wasn't so bad, but It'll be easier if the examples on the website were brought up to date. Cheers, Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Possible bug in preduce
Hi all, Messing around with preduce at the REPL I saw this: user= (defn q [sofar new] (do (print new sofar\n) (+ (+ 1 new) sofar))) #'user/q user= (reduce q 0 [1 2 3]) 1 0 2 2 3 5 9 user= (preduce q 0 [1 2 3]) 3 2 6 1 8 It looks like preduce takes its arguments in the opposite order from reduce, but of course with this fn it shouldn't make any difference: user= (defn q [new sofar] (do (print new sofar\n) (+ (+ 1 new) sofar))) #'user/q user= (preduce q 0 [1 2 3]) 2 3 1 6 8 Looks like it's using 3 where it should compute (q 3). This is a bug, correct? Thanks, Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Stupid Java questions
Thanks Zak, With the other jar I could load parallel.clj without errors, but I wasn't able to refer to the parallel namespace as on clojure.org/ other_libraries, nor was preduce present in the user namespace: user= (load-file parallel.clj) nil user= (refer 'parallel) java.lang.Exception: No namespace: parallel (NO_SOURCE_FILE:0) user= preduce java.lang.Exception: Unable to resolve symbol: preduce in this context (NO_SOURCE_FILE:0) Also, the word 'jsr166y.jar' on clojure.org/other_libraries should probably point to the working jar. Thanks, Anand On Feb 3, 6:22 pm, Zak Wilson zak.wil...@gmail.com wrote: I had similar results when I compiled jsr166y myself. There's a jar in the group's files that is known to work. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
definline example?
Hi all, I'd like to repeat my request for an example of definline usage; I feel I've tried everything. user= (definline f [x] x) java.lang.ExceptionInInitializerError (NO_SOURCE_FILE:1) user= (definline [x] x) java.lang.NullPointerException (NO_SOURCE_FILE:2) user= (definline (f [x] x)) java.lang.NullPointerException (NO_SOURCE_FILE:3) user= (definline (defmacro f [x] x)) java.lang.NullPointerException (NO_SOURCE_FILE:4) user= (definline f) java.lang.NullPointerException (NO_SOURCE_FILE:12) Thanks, Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: definline example?
On Jan 25, 5:27 pm, Chouser chou...@gmail.com wrote: On Sun, Jan 25, 2009 at 8:01 AM, Chouser chou...@gmail.com wrote: On Sun, Jan 25, 2009 at 7:06 AM, Anand Patil anand.prabhakar.pa...@gmail.com wrote: I'd like to repeat my request for an example of definline usage; I feel I've tried everything. I looked into this yesterday, and it seems to me that definline has been broken since SVN rev 1065. I don't know why the usages of it in core.clj don't cause errors at compile time, but they don't seem to work quite right. I wrote a new definline that seems to work correctly: (defmacro definline Experimental - like defmacro, except defines a named function whose body is the expansion, calls to which may be expanded inline as if it were a macro. Cannot be used with variadic () args. [name decl] (let [[pre-args [args expr]] (split-with (comp not vector?) decl)] `(do (defn ~name ~...@pre-args ~args ~(apply (eval (list `fn args expr)) args)) (alter-meta! (var ~name) assoc :inline (fn ~args ~expr)) (var ~name Using definline work, and return the Var, just like defn does: user= (definline inc2 returns x plus 2 [x] `(+ 2 ~x)) #'user/inc2 Terrific, thanks very much! Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Request for code examples: definline and 'full' macro syntax
Hi all, Does anyone have, or know where I can find, simple working examples showing the use of: - definline - defmacro with all the fields, (defmacro name doc-string? attr-map? ([params*] body) + attr-map?) I get doc-string, but don't understand: - attr-map - (...) + attr-map - the precise meaning of params* ... does it just mean that there are a variable number of them? Does it mean the same thing in clojure as in the docstrig? Many thanks, Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: How to make function that derefs vars and refs?
On Jan 19, 1:05 am, Stuart Sierra the.stuart.sie...@gmail.com wrote: On Jan 18, 6:56 pm, Anand Patil anand.prabhakar.pa...@gmail.com wrote: Would it make any sense to make @ polymorphic so that @x return x's value when x is a var rather than raising an error? Actually, it already is: user (def x 5) #'user/x user (deref (var x)) 5 user @(var x) 5 @x is just shorthand for (deref x), which resolves to (deref 5), which doesn't make sense. Stuart, Sorry, I was not being clear. Why not just let (deref 5) return 5? I would consider that syntactic sugar, as it would make it easier to deref a mixed vector of refs, constants, agents and atoms without keeping track of which indices correspond to (ref, agent, atom)s and which to constants. Thanks, Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: How to make function that derefs vars and refs?
On Jan 19, 1:25 pm, Chouser chou...@gmail.com wrote: On Mon, Jan 19, 2009 at 4:07 AM, Anand Patil anand.prabhakar.pa...@gmail.com wrote: Sorry, I was not being clear. Why not just let (deref 5) return 5? I would consider that syntactic sugar, as it would make it easier to deref a mixed vector of refs, constants, agents and atoms without keeping track of which indices correspond to (ref, agent, atom)s and which to constants. You can do this yourself quite easily: (defn my-deref [x] (if (instance? clojure.lang.IRef x) @x x)) (my-deref 5) == 5 (my-deref (var filter)) == #core$filter__3586 clojure.core$filter__3...@1e1be92 (my-deref filter) == #core$filter__3586 clojure.core$filter__3...@1e1be92 (my-deref (agent 99)) == 99 I don't know if the builtin 'deref' should do this -- it seems like it might make things more confusing rather then less. --Chouser Thanks Chouser, I'll take your word for it use a specialized function. Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: How to make function that derefs vars and refs?
Timothy, Stephen and Stuart, Thanks for three very informative and understandable responses! It's very encouraging to see that such support is available when approaching a new language. I look forward to further exploration of Clojure. The var/value/symbol relationship continues to be a bit slippery for me but I think I get it. The compiler's differing behaviors when presented with vars vs. identities makes sense now that I think about it. Thanks also for the class-checking syntax, which will come in very handy. Would it make any sense to make @ polymorphic so that @x return x's value when x is a var rather than raising an error? Anand On Jan 18, 3:21 pm, Stuart Sierra the.stuart.sie...@gmail.com wrote: Hi Anand, You don't normally need to dereference vars because the compiler does it for you. user (def x 5) #'user/x This creates a var bound to the symbol x. But now, whenever you type x the compiler will automatically look up the var and return its value. user x 5 So when you type (var? x) it's resolved to (var? 5). You're asking is 5 a var? and the answer is no. If you want to get at the var object itself, you can use the special form (var x) user (var x) #'user/x Note the #' prefix -- that means you're looking at a var object and not the symbol user/x. user (var? (var x)) true user (var-get (var x)) 5 I think deref worked on vars in early versions of Clojure, but it doesn't now. If you're just getting started with Clojure, you can probably ignore vars. You will rarely have any need to work with vars directly unless you're doing some complicated metaprogramming. -Stuart Sierra On Jan 18, 5:11 am, anand.prabhakar.patil anand.prabhakar.pa...@gmail.com wrote: Hi all, I'm new to Java, Lisp and Clojure, so please be patient. I'm trying to make a function that behaves as follows: (my-deref x) returns x if x is a var, or @x if x is a ref. However none of the obvious options seem to be working. - First, it seems from the api documentation that @x will work fine even if x is a var. I don't understand the following: user= (def x 5) #'user/x user= @x java.lang.IncompatibleClassChangeError (NO_SOURCE_FILE:0) user= (var? x) false or user= (def x) #'user/x user= (var? x) false Why isn't x a var? - Second, I remember having seen a ref? function analogous to var? in the online documentation, but it doesn't seem to actualy exist: user= (ref? x) java.lang.Exception: Unable to resolve symbol: ref? in this context (NO_SOURCE_FILE:13) Thanks in advance, Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
