Re: Clojure in Computing in Science and Engineering
On Sat, Jun 20, 2009 at 11:29 AM, Jon Harrop j...@ffconsultancy.com wrote: The Task Parallel Library. It uses concurrent wait-free work-stealing queues to provide an efficient implementation of work items than can spawn other work items with automatic load balancing on shared memory machines. Cilk uses the same technology (well worth a look if you haven't already seen it!). That makes it easy to write efficient parallel algorithms in any .NET language. In particular, it can sustain billions of work items (as opposed to thousands of threads or processes) and the time taken to spawn is ~30,000x faster than forking a process. Extremely useful stuff! Sounds similar to ForkJoin, which Rich pointed out to me a while ago: http://www.ibm.com/developerworks/java/library/j-jtp11137.html Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Clojure in Computing in Science and Engineering
On Sunday 21 June 2009 02:44:02 Kyle Schaffrick wrote: On Sat, 20 Jun 2009 11:29:44 +0100 Jon Harrop j...@ffconsultancy.com wrote: The Task Parallel Library. It uses concurrent wait-free work-stealing queues to provide an efficient implementation of work items than can spawn other work items with automatic load balancing on shared memory machines. Cilk uses the same technology (well worth a look if you haven't already seen it!). That makes it easy to write efficient parallel algorithms in any .NET language. In particular, it can sustain billions of work items (as opposed to thousands of threads or processes) and the time taken to spawn is ~30,000x faster than forking a process. Extremely useful stuff! Interesting. It strikes me that it's called task parallel library, while it sounds a lot like Intel Threading Building Blocks, which is a sort of STL-style quasi-functional template library for *data*-parallel algorithms; the stuff people like to write with Fortran, OpenMP and friends. I had not looked at Intel's offering because it does not (AFAIK) support accurate garbage collection. Also, it is worth noting that there is no difference between data and task parallelism in a genuine functional language. It uses a work-stealing thread-pool scheduler as well, atop which stuff like parallel maps and reductions are implemented as templates. You can create your own work items and stick them in the scheduler by hand, but the useful bits are actually the prefab algorithms, IMO. Right. If you hand-roll your C++ very carefully then you can get decent performance but I was disappointed with the performance of the template libraries and quality of g++ back in 2004 and have never used C++ since. I've heard good things about D but, IMHO, it fails to cash in on lots of language features. Most notably FP. The tunable/pluggable slicing strategies, built on the standard iterator concepts, are particularly interesting way to give you full control of work unit granularity, without having to know too much about the innards of the algorithms themselves. Yes. The nice thing about using a functional language is that you can easily pass two functions, one to solve the problem and the other describing the complexity of the first as a function of its inputs. If necessary, you can augment your data structures with extra information to make it efficient to compute complexities. Then you can use the new complexity function to intelligently parallelize your algorithms into tasks such that the (roughly-constant) overhead of spawning tasks is guaranteed to be small compared to the work done by the task and, hence, you can greatly increase the probability of getting a speedup compared to ordinary chunking. Moreover, there is great benefit in sharing the same load balancing system. For example, if you write a parallel .NET program where tasks invoke the Intel MKL then the MKL uses independent parallelization and that renders your performance unpredictable at best and awful at worst. -- Dr Jon Harrop, Flying Frog Consultancy Ltd. http://www.ffconsultancy.com/?e --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Clojure in Computing in Science and Engineering
On Sunday 21 June 2009 08:55:54 Anand Patil wrote: Sounds similar to ForkJoin, which Rich pointed out to me a while ago: http://www.ibm.com/developerworks/java/library/j-jtp11137.html Yes. I believe the main difference is that the TPL does not block because there is no join operation. -- Dr Jon Harrop, Flying Frog Consultancy Ltd. http://www.ffconsultancy.com/?e --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Why different behaviour?
Why are these different? user #^String hello ; Evaluation aborted. (def a hello) #'user/a user #^String a hello --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Why different behaviour?
On Jun 21, 2009, at 9:51 AM, Ratandeep Ratti wrote:
Why are these different?
user #^String hello
; Evaluation aborted.
#^ is a reader macro for applying metadata to the next object read.
The metadata applied is always a map. If a sequence of characters
(chars) appears after #^: #^chars, it is interpreted as a
shorthand notation for #^{:tag chars}.
The reader can only apply metadata to objects that implement the Java
interface IMeta. When I try your example, I get:
user= #^String hello
java.lang.IllegalArgumentException: Metadata can only be applied to
IMetas
While it might be useful to apply metadata to strings, we can't
because Clojure strings are Java Strings and Java Strings are final
and don't implement IMeta.
(def a hello)
#'user/a
user #^String a
hello
Here the reader was asked to apply the map {:tag String} as metadata
to the symbol a as it was read in. Symbol does implement IMeta so that
succeeds.
For understanding reader macros, it can be helpful to use with-in-
str and read to capture exactly what read returns before eval gets
hold of it:
user= (def b (with-in-str #^String a (read)))
#'user/b
user= (class b)
clojure.lang.Symbol
user= b
a
user= ^b
{:tag String}
Here, we def the var named user/b to be the value the reader returns
when it reads #^String a. We see that it's a Symbol whose name is a
and whose metadata is {:tag String}.
In a normal read-eval-print operation (or for definitions, in a read-
compile-store operation), the compiler can use the :tag information
on the symbol a to produce better code to manipulate the value it
represents in expressions.
--Steve
smime.p7s
Description: S/MIME cryptographic signature
Incanter classpath difficulties
Hi David and all, Just trying to get up and running with Incanter, on a Mac with the github head. The clj script doesn't find part of parallel colt: (head-mac bin) ./clj Clojure 1.1.0-alpha-SNAPSHOT user= (use '(incanter core)) java.lang.NoClassDefFoundError: cern/colt/matrix/tdouble/impl/DenseColDoubleMatrix2D (internal.clj:19) user= (import cern.colt.matrix.tdouble.impl.DenseColDoubleMatrix2D) java.lang.ClassNotFoundException: cern.colt.matrix.tdouble.impl.DenseColDoubleMatrix2D (NO_SOURCE_FILE:2) user= When I change INCANTER_HOME=. to INCANTER_HOME=.. in the clj script, I can import the required class but still not use Incanter: (head-mac bin) ./clj Clojure 1.1.0-alpha-SNAPSHOT user= (import cern.colt.matrix.tdouble.impl.DenseColDoubleMatrix2D) cern.colt.matrix.tdouble.impl.DenseColDoubleMatrix2D user= (use '(incanter core)) java.lang.NoClassDefFoundError: cern/colt/matrix/tdouble/impl/DenseColDoubleMatrix2D (internal.clj:19) user= The problem seems to be in the Matrix class: user= (import incanter.Matrix) java.lang.NoClassDefFoundError: cern/colt/matrix/tdouble/impl/DenseColDoubleMatrix2D (NO_SOURCE_FILE:3) Thanks in advance for any help, Anand --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Incanter classpath difficulties
On Jun 21, 5:01 pm, Anand Patil anand.prabhakar.pa...@gmail.com wrote: Hi David and all, Just trying to get up and running with Incanter, on a Mac with the github head. The clj script doesn't find part of parallel colt: (head-mac bin) ./clj Clojure 1.1.0-alpha-SNAPSHOT Hello, I think the script is intended to be run as bin/clj, from the incanter directory. Have you done other modifications to the file (apart from the changes to INCANTER_HOME you mention)? I can reproduce the issues you report: when I use the original script (with INCANTER_HOME=.) as ./clj in the bin directory, clojure doesn't start at all (jline.jar and clojure.jar are not found), when I set INCANTER_HOME=.. everything seems to be ok. The tests above where conducted on a fresh git clone git://github.com/liebke/incanter.git on Leopard using java 1.5.0_19. Carlos --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Incanter classpath difficulties
In my previou message I wrote I can reproduce the issues you report where I meant to say I _can't_ reproduce the issues you report Sorry for the noise. Carlos --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
clojure loader script
here's a ruby script which configures a java classpath against a bunch of clojure projects and starts clojure http://github.com/mccraigmccraig/clojure-load.rb e.g. clojure -l clojure-json -l clojure-http-client -l clojureql - it assumes it's repo is in the same dir as the clojure repo and other project repos - it assumes the [emergent?] standard repo layout : - clojure source in : project/project.jar || project/src/clj || project/src - additional jars in : project/lib/*.jar --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: problems with regex
On Jun 21, 2009, at 4:34 PM, Alpinweis wrote: I am trying some regex in Clojure and I have problems using regex literals. Don't know why the following doesn't seem to work: user= (re-seq #\\W+ the quick brown fox) () user= (re-seq #\\w+ the quick brown fox) () user= (re-seq #\\s the quick brown fox) () user= (re-seq #\\S the quick brown fox) () The string that forms the regex literal is not subject to the usual string escaping. You don't need to (and should not) double the backslash. Is there any detailed info on using regex literals in Clojure? It appears the official doc at http://clojure.org/reader doesn't make this clear. It would be good to fix that. Here's some info: http://items.sjbach.com/406/clojures-new-regex-syntax And the Google search: http://www.google.com/search?q=clojure+regex+literal --Steve smime.p7s Description: S/MIME cryptographic signature
jar handling
Hi, Does clojure have any way to handle jar loading without having to specify it in command line? I'm looking for something like groovy, where if you place a jar file in ~/.groovy/lib. it's available to any groovy code. Thanks -- Omnem crede diem tibi diluxisse supremum. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
passing clojure functions into a java program.
I can't seem to figure out why java can't find the clojure class in the following example, where f is a clojure function (defn cascading-function [f] (proxy [BaseOperation Function] [(Integer. 1) (Fields. (into-array Comparable [line]))] (operate [flowProcess functionCall] (let [clojureData (. (Tuple. (. (. functionCall getArguments) getTuple)) getString 1) result (hadoop-function f clojureData) resultTuple (Tuple.)] (. resultTuple add result) (. (. functionCall getOutputCollector) add resultTuple) cascading function returns a java object that implements Function (from the cascading framework) and extends BaseOperation, and it implements the operate method. The error that I am seeing is a ClassNotFoundException for whatever I pass in as f. For some more context, I am trying to wrap the cascading framework in clojure, (www.cascading.org) The internal driver/framework code is in Java, and I am using clojure to wrap the user hooks for defining operations and making jobs. This wrapping mainly involves tedious creation of objects, one of the objects is this Function object, and I am trying to proxy Function and make it close over (closure) clojure functions so that I can define the actual logic in a nice (clojure) way. (If i implement the Function interface as an object in Java, everything seems to run fine) Java (framework code) - Clojure (wrapper over making jobs) - Java (an operation, implements Function interface) (defn cascading-function [] (FunctionSample.)) what I want is Java (framework code) - Clojure (wrapper over making jobs) - Clojure (operation) but the final part is not working so well. cascading-function is called by clojure code which wraps/builds up objects for use by Java/cascading framework. Does anyone know what I might be doing wrong? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: jar handling
In Java 6 you can do a wildcard for jar files in a directory: java -cp /opt/jars/*:. clojure.main this will find all the jar files in /opt/jars/ and put them on the classpath. On Sun, Jun 21, 2009 at 9:10 PM, Wilson MacGyverwmacgy...@gmail.com wrote: Hi, Does clojure have any way to handle jar loading without having to specify it in command line? I'm looking for something like groovy, where if you place a jar file in ~/.groovy/lib. it's available to any groovy code. Thanks -- Omnem crede diem tibi diluxisse supremum. -- Chris Wilson christopher.j.wil...@gmail.com --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: passing clojure functions into a java program.
On Jun 21, 2009, at 11:07 PM, timshawn wrote: I can't seem to figure out why java can't find the clojure class in the following example, where f is a clojure function (defn cascading-function [f] (proxy [BaseOperation Function] [(Integer. 1) (Fields. (into-array Comparable [line]))] (operate [flowProcess functionCall] (let [clojureData (. (Tuple. (. (. functionCall getArguments) getTuple)) getString 1) result (hadoop-function f clojureData) resultTuple (Tuple.)] (. resultTuple add result) (. (. functionCall getOutputCollector) add resultTuple) cascading function returns a java object that implements Function (from the cascading framework) and extends BaseOperation, and it implements the operate method. The error that I am seeing is a ClassNotFoundException for whatever I pass in as f. The only place f appears is as the argument to hadoop-function. What's that expecting for arguments? In general if you can come up with a small runnable example that shows the problem (with as few dependencies as possible), that's the easiest way for interested folks to help. Showing the relevant portion of the stack trace for the exception is also likely to be helpful. You can get the stack trace for the most recent exception caught by the REPL with (.printStackTrace *e) (see also clojure.contrib.repl-utils for convenience functions around that.) --Steve smime.p7s Description: S/MIME cryptographic signature
Re: Question/Problem re (set! *warn-on-reflection* true)
On Jun 21, 2009, at 11:17 PM, arasoft wrote: When I enter the following function into the REPL it compiles and works without problems: (defn harmonic-number [n prec] (reduce + (map #(with-precision prec (/ 1 (bigdec %))) (range 1 (inc n ) After (set! *warn-on-reflection* true), in a normal REPL I get: Reflection warning, NO_SOURCE_PATH:3 - call to java.math.MathContext ctor can't be resolved. You can give the compiler a type hint on prec to avoid the reflection: (defn harmonic-number [n #^Integer prec] (reduce + (map #(with-precision prec (/ 1 (bigdec %))) (range 1 (inc n ) In Enclojure, it's worse: CompilerException java.lang.ClassCastException: java.io.OutputStreamWriter cannot be cast to java.io.PrintWriter (NO_SOURCE_FILE:8) (At least) some versions of Enclojure are binding *err* to an OutputStreamWriter rather than a PrintWriter. Clojure's default binding is to a PrintWriter and some of Clojure expects a PrintWriter. I consider binding *err* to an OutputStreamWriter to be a bug. I think the Enclojure developers know about it, so if there's a version of Enclojure you haven't upgraded to, I recommend trying that. If you're using the latest version, I suggest taking a look at their bug list (if it's public) and/or letting them know about it. --Steve smime.p7s Description: S/MIME cryptographic signature
Re: passing clojure functions into a java program.
Hi Stephen, Thanks for the note. Yea that first post was kind of unwieldy. hadoop-function takes in f and data, and does some preprocessing before applying f to the processed data. (I tried it by just using f directly and taking hadoop-function out, and same thing happens) One of the problems that I am having is that I'm having a hard time to reproduce this in a smaller manner. Everything works out fine in the repl, but when the framework/runtime starts up, it throws a ClassNotFoundException It's trying to find:cascading$cascading_function__44$fn__46 When I look in my target directory/exploded jar, it has this cascading$cascading_function__38$fn__40.class On Jun 21, 8:36 pm, Stephen C. Gilardi squee...@mac.com wrote: On Jun 21, 2009, at 11:07 PM, timshawn wrote: I can't seem to figure out why java can't find the clojure class in the following example, where f is a clojure function (defn cascading-function [f] (proxy [BaseOperation Function] [(Integer. 1) (Fields. (into-array Comparable [line]))] (operate [flowProcess functionCall] (let [clojureData (. (Tuple. (. (. functionCall getArguments) getTuple)) getString 1) result (hadoop-function f clojureData) resultTuple (Tuple.)] (. resultTuple add result) (. (. functionCall getOutputCollector) add resultTuple) cascading function returns a java object that implements Function (from the cascading framework) and extends BaseOperation, and it implements the operate method. The error that I am seeing is a ClassNotFoundException for whatever I pass in as f. The only place f appears is as the argument to hadoop-function. What's that expecting for arguments? In general if you can come up with a small runnable example that shows the problem (with as few dependencies as possible), that's the easiest way for interested folks to help. Showing the relevant portion of the stack trace for the exception is also likely to be helpful. You can get the stack trace for the most recent exception caught by the REPL with (.printStackTrace *e) (see also clojure.contrib.repl-utils for convenience functions around that.) --Steve smime.p7s 3KViewDownload --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: passing clojure functions into a java program.
On Jun 22, 2009, at 12:29 AM, timshawn wrote: It's trying to find:cascading$cascading_function__44$fn__46 When I look in my target directory/exploded jar, it has this cascading$cascading_function__38$fn__40.class Could there be a classes directory that's in the classpath of the repl, but not of the launched environment. I would try rebuilding/ recompiling the jar file cleanly from scratch and then using it from a repl with your classpath carefully controlled so you're sure you're only using what's in the jar. java -cp clojure.jar:mystuff.jar clojure.main --Steve smime.p7s Description: S/MIME cryptographic signature
sorted-map-by sample call
Hi folks,
I had some fortune with the sorted-by function:
1:11 user= (sort-by (fn [e] (second e)) [[1 99] [3 4] [5 6] [7 8]])
([3 4] [5 6] [7 8] [1 99])
so I thought I'd have a go with sorted-map-by also:
1:13 user= (doc sorted-map-by)
-
clojure.core/sorted-map-by
([comparator keyvals])
keyval = key val
Returns a new sorted map with supplied mappings, using the supplied
comparator.
1:14 user= (sorted-map-by (fn [c] c) 1 2)
{1 2}
1:15 user= (sorted-map-by (fn [c] c) 1 2 3)
java.lang.IllegalArgumentException: No value supplied for key: 3
(repl-1:15)
1:16 user= (sorted-map-by (fn [c] c) 1 2 3 4)
java.lang.RuntimeException: java.lang.IllegalArgumentException: Wrong
number of args passed to: user$eval--85$fn (repl-1:16)
I expected line 1:15 to fail because I didn't have matching
quantities of keys and values, but I don't know what I've done wrong
in 1:16. I searched the google group discussions for sorted map by,
but I didn't understand what I found. Would a compassionate soul guide
me to where I went wrong?
Kev
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~--~~~~--~~--~--~---
Re: sorted-map-by sample call
On Jun 22, 2009, at 1:23 AM, kkw wrote:
I had some fortune with the sorted-by function:
1:11 user= (sort-by (fn [e] (second e)) [[1 99] [3 4] [5 6] [7 8]])
([3 4] [5 6] [7 8] [1 99])
You were using the form of sort-by that accepts a keyfn, not the one
where you also provide a comparator.
so I thought I'd have a go with sorted-map-by also:
1:13 user= (doc sorted-map-by)
-
clojure.core/sorted-map-by
([comparator keyvals])
keyval = key val
Returns a new sorted map with supplied mappings, using the supplied
comparator.
1:14 user= (sorted-map-by (fn [c] c) 1 2)
{1 2}
No comparisons were necessary, so your comparator function wasn't
called.
1:16 user= (sorted-map-by (fn [c] c) 1 2 3 4)
java.lang.RuntimeException: java.lang.IllegalArgumentException: Wrong
number of args passed to: user$eval--85$fn (repl-1:16)
sorted-map-by called your comparator, but comparators take two
arguments and your function only accepts one.
--Steve
smime.p7s
Description: S/MIME cryptographic signature
