Re: Clojure for large programs
No worries. I have the book on my shelf. The first version. But thanks for making me aware of the second version. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Clojure for large programs
I still have to do my personal large scale project in Clojure, but I would like to share my thoughts so far. (10 years ago I implemented a 60k Common Lisp project; I never worked on more than 5k Clojure code so far; the C++ and Java projects I was involved in reached 800k to 1M lines of code). I think one important ingredient for large scale Clojure projects would be the literate programming approach that Tim Daly described some time ago: https://groups.google.com/group/clojure/browse_thread/thread/460417fe45f314c3?hl=de In addition I think midje is a very good unit testing framework. I am currently working on a maven archetype that makes the construction of projects with literate programming and midje easy. The toy project that I am thinking of implementing in order to test the large scale characteristics of Clojure is the ERP like system described in Java Modeling In Color With UML: http://www.amazon.com/Java-Modeling-Color-UML-Enterprise/dp/013011510X/ The biggest trouble in larger Clojure projects that deal with large data graphs that I have is with the lack of a data schema like a Java class definition or a SQL DDL schema definition or an xml schema. I just want to know how data structures have to look like even before instantiating the first instance of it. In addition a reflection like query mechanism and other meta data like validation rules would be helpful. I for myself came to the conclusion that Clojure is not made for large nested data structures on its own. I personally feel much better with a combined approach like the Functional Relational Programming approach recently mentioned here on the group: https://groups.google.com/group/clojure/browse_frm/thread/ba3da253f6358ac9?hl=de http://web.mac.com/ben_moseley/frp/frp.html I think one could use either SQL (e.g. an embedded h2sql database) or something like datalog: http://code.google.com/p/clojure-contrib/wiki/DatalogOverview to describe entities and their relations. Those entities would be maps in clojure with *few*! key value pairs. The nice characteristics of Clojure as a pure functional language that deals with immutable data structures and is easy to test would be kept. You could use SQL or datalog as a query language to link the smaller structures together, the entities, together. You could even write a runtime documentation feature that would display a graphical representation of an E/R like diagram. Clojure would be the pure functional aspect of the FRP approach and the data structures would be handled by the relational algebra part. I believe in what Alan Perlis said: It is better to have 100 functions operate on one data structure than 10 functions on 10 data structures. That's the reason why I have my difficulties with Python. Nevertheless for large connected data graphs I think something like a data-schema is needed. Clojure would still follow its approach to only deal with maps, but there is a descriptive meta-data level in addition that explains the connection between those maps. I would agree to what was said elsewhere: the Clojure community has to come up with idioms on how to deal with large scale projects. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Clojure for large programs
This is an unfinished thought: I think that the Single-Level-of- Abstraction (SLA) principle promoted in OO needs to have a prominent place in functional programming, too! Each function should talk about the problem in its level of abstraction, e.g. in its language. Functions related to the same level of abstraction can be put into a package. The problem of course is to understand what a level of abstraction is. Therefore this thought is yet unfinished. Just a parallel I came across some time ago. I prefer designing software along features. A lot of people have different ideas of what a feature is. I came across a good way to identify features in the already mentioned book Java Modeling in Color with UML. They use a language template action the result by|for|of|to a(n) object e.g. Calculate the total of a sale or Calculate the total purchases by a customer. The key here was to find a language template. Language is at the core of this concept. I feel that language may be also the core to find out a level of abstraction. A rule of thumb for me was in the past to look for groups of functions that I start to add a name prefix to, e.g. excel-number-cell, excel- format, ... I put them in a package...excel. In general I am very reluctant to creating namespaces. I often feel that name spaces are as much in my way to create software as static type systems are. A certain fraction of my brain is constantly involved in thinking about which grouping should I use instead of thinking about the problem and the level of abstraction at hand. By the way a similar experience is between using C++ and using a language with a garbage collection. A fraction of your brain is constantly busy with thinking about memory allocation (extremely low level of abstraction) instead of the level of abstraction you should think about. Perhaps somebody else has already thought further into that direction and I would be happy to take over your learnings :) -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Clojure for large programs
Thanks for your feed-back. I already have RDF/OWL in my tool-kit. I am only not sure if an ERP like system should be modeled along those lines. But I did not put enough thought in that direction yet. Would you base an ERP like system on top of RDF/OWL? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Question about data structures and encapsulation
Thanks a lot for the link to the paper about FRP! My personal thinking is going 90% in the same direction that the paper describes. I am happy to see that somebody else did the hard work of writing it down :) Is anybody aware of an implementation of such an approach for Clojure? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Wisdom sought for functional iterative processing
Ah, sorry, perhaps I misunderstand you, but if you use multimethods
(defmulti) in Clojure you do not need to attach methods to anything.
The defmulti will allow you dispatch-on-type based on the key which
is present in the map (e.g. if :delete is present or if :insert is
present).
The nice thing about multimethods the lisp style is that they are
additive, e.g. you do not have to change anything in your op
structure.
On Jun 15, 4:38 am, Matthew Phillips mattp...@gmail.com wrote:
On Jun 14, 12:05 pm, gaz jones gareth.e.jo...@gmail.com wrote:
if i was writing the java i would probably do a tell dont ask
refactoring so that the operations had an applyTo method:
ListItem items = initialItems ();
for (Op op : operations)
{
op.applyTo(items);
}
not sure what your op data structure is, but i would image you could
translate that to the clojure code also -- is that reasonable?
I think you're right that the above is a better OO design, but that
was really just a Java translation of the sort of think I'd like to do
in Clojure (minus the mutable list).
The ops in this case have abstract meaning (they are [source target
delta] tuples describing list transformations), and attaching
methods to them (presumably as fn's) in this case would be weird I
think, as well as requiring a switch from tuples to maps or records.
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
Re: Reading clojure code of larger domain specific projects
Thanks, that sound good! I'll have a look at the clojars project and yours once it is in the right state for that. On 10 Mai, 18:04, Paul deGrandis paul.degran...@gmail.com wrote: I'm also working on a project to augment clojars called clopi (Clojure Package Index), that will let you filter results based on project metrics (for example, line count), project activity, etc. It also displays the dependency information (the default rank score for searching), allowing you mostly to answer the question Which library should I choose? but also can help you find possible business applications (edge/external-nodes in the graph, ie: projects which nothing else depend upon). I'll ping you when it's live. Paul -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Reading clojure code of larger domain specific projects
Hello list, I have a question that perhaps may be relevant for more people. I strongly believe that reading code of other people is an undervalued discipline of all developers. Typically it just happens as a side effect of working in a project with other people. Like that a style of development evolves in a programming language community. You may think of projects written by other people what you like (well done, poorly done), but I believe that it is always beneficial to read code written by other people. I've done that in C++ and Java quite a bit (ACE framework, TAO orb, STLport, Java Swing libraries, Java Spring libraries, Apache Commons libraries, JBoss SEAM, ...). I am writing programs in Common Lisp since 1995, but up to now I never worked in Lisp projects with more than me being involved. There are definitely many well written Lisp projects out there and books like PAIP may definitely help, too, but I was wondering if there are any larger domain specific open-source projects written in Clojure out there that you would recommend for reading as some sort of best practice guide? I was thinking about leiningen or cake, but I would prefer projects that are closer to fulfilling a business purpose than a technical purpose like a build system. If the project then also would have a good documentation then that would be perfect :) Any suggestions from your side? Thanks, Christian -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Reading clojure code of larger domain specific projects
Many thanks for all of your contributions so far! I definitely will (and already have to some extent) go through the recommended links. One additional project I came across that looks interesting is midje: https://github.com/marick/Midje/wiki I see the point of having a forum to discuss best solutions, but that is a different matter from what I am talking about. Code looks different if you look at some lines of code or if you look at projects that have a relevant size (more than 100k lines of code) and where more than 10 people have worked on over an extended period of time. In addition the reason for looking for a project that is less technical and more domain specific is that typically developers have a very good understanding of how a technical solution should look like. They have the requirements already in their heads, because they are the final audience, the end-users. In a domain specific project the requirements come from outside and the business domain is typically not clear from the start. The project has to develop ways on how to deal with the evolution of the understanding/insight into the problem domain. You get a balanced view across different styles by looking at several projects of relevant size. Different teams find different solutions to similar problems. Projects that are not open-source but would seem to fall in the category that I am looking for are FlightCaster and TheDeadline: http://www.infoq.com/articles/deadline-clojure-appengine Do you know of any projects of that type that are open-source to be able to analyse the code? Many thanks! Christian -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Declarative Data Models
My comment is unrelated to Clojure but related to the general topic of avoiding redundancy. Please have a look at InfoQ's Dan Haywood's Domain-Driven Design Using Naked Objects: http://www.infoq.com/articles/haywood-ddd-no The book is definitely worth a read! Naked Object's (now called Apache Isis http://incubator.apache.org/isis/) whole purpose is to concentrate the domain knowledge (including the meta data!) in the domain model and thus avoid the typical redundancies you find across the different application layers. Here is an answer to one of my questions to their mailing list to find out how to query their central meta data infrastructure: http://mail-archives.apache.org/mod_mbox/incubator-isis-dev/201104.mbox/browser One of my ideas (don't know when and if I will ever come around to pursue that) would be to create an implementation of that meta data interface in Clojure. That way the whole feature set of the framework would be available for developing applications in Clojure. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Declarative Data Models
Sorry, the link to their mailing list is here: http://mail-archives.apache.org/mod_mbox/incubator-isis-dev/201104.mbox/BANLkTim+9061Se1mPLK3=wymmj0qkdu...@mail.gmail.com -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: swank-cdt: Using Slime with the Clojure Debugging Toolkit
I am using maven directly to work with clojure and I had the problem
with the tools.jar, too. I resolved it by adding the following block
to my pom.xml:
profiles
profile
iddefault-tools.jar/id
activation
property
namejava.vendor/name
valueSun Microsystems Inc./value
/property
/activation
dependencies
dependency
groupIdcom.sun/groupId
artifactIdtools/artifactId
version1.6.0/version
scopesystem/scope
systemPath${java.home}/../lib/tools.jar/systemPath
/dependency
/dependencies
/profile
/profiles
After that it is still complaining: warning: unabled to add tools.jar
to classpath. This may cause CDT initialization to fail.
But this does not seem to cause a problem.
Just wanted to share this bit of information with you,
Christian
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
Re: closed maps / reducing runtime errors due to mistyped keywords
Thanks for that recommendation, I definitely like that proposal! I guess the drawback is then that destructuring will not be safe. I have to think some more time about it. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: closed maps / reducing runtime errors due to mistyped keywords
Thanks for all of your answers, but I think for my use cases the approach proposed by Justin looks most promising. I'll implement my own closed map. Another idea that come to my mind was that a closed map is actually a Java Bean without behaviour. Perhaps the closed map could be implemented by dynamically creating an immutable bean at run-time somehow along the lines of this article: http://blogs.sun.com/adventures/entry/dynamic_java_beans The advantage would be that other JVM languages could more easily take advantage of such objects, e.g. rule engines like drools or complex even processing engines like esper. Are there any disadvantages you would see? I would imagine that clojure's compiler already has most of what is needed for such an approach. Could somebody give me a good entry point where to start my analysis of the clojure internals to create dynamically beans at runtime? In the past clojure seems to have had a gen-and-load-class feature, which disappeared. I could not find references that would explain the reasons why it was taken out. The following post from 2008 basically would already do what I have in mind but it uses the gen-and-load-class feature: http://groups.google.com/group/clojure/browse_thread/thread/79e4cb66a05c8ce9 -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
closed maps / reducing runtime errors due to mistyped keywords
I am taking up a discussion from 2010: https://groups.google.com/group/clojure/browse_frm/thread/60dff89149c3d2e6/ I would prefer if it would be possible to define closed maps, e.g. maps that allow only a certain set of keywords, both for get and set (like in assoc, assoc-in, update-in, ...). It would be nice if there would be an option on defrecord to mark the map as closed. I imagine that this would have some runtime penalty, because for every access it has to be verified if the key is a valid one. I understand that. Perhaps in that case the language could react differently based on the value of a dynamic var (*check-closed-defrecrod-p*) that can be set when the unit tests are run and could be false by default. I guess this topic was already discussed several times. Could somebody point me to previous discussions and their outcome? I've found in addition the following thread where the idea of safe maps was discussed to some extent: https://groups.google.com/group/clojure/browse_frm/thread/134642cc76de17f7/ Thanks, Christian -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Emacs `align' function customization for Clojure
I am replying to this thread from January this year: https://groups.google.com/group/clojure/browse_frm/thread/47c388127e0da3ef I am not certain if my solution fulfills the aesthetic requirements of clojure hackers, but I use one of my old C/C++ marcros for aligning equal signs to align at commas (see below). As commas in clojure are whitespace you can put them in front of the clauses you want to be aligned, e.g. in let binding forms. You select the region you want the alignment to happen and press C-= and the full block gets nicely aligned. I hope this may be useful for other people, too. -- snip start -- (defun align-equals (start end) make the first assignment operator on each line line up vertically (interactive *r) (save-excursion (let ((indent 0)) (narrow-to-region start end) (beginning-of-buffer) (while (not (eobp)) (if (find-assignment) (progn (exchange-point-and-mark) (setq indent (max indent (current-column))) (delete-horizontal-space) (insert ))) (forward-line 1)) (beginning-of-buffer) (while (not (eobp)) (if (find-assignment) (indent-to-column (1+ (- indent (- (mark) (point)) (forward-line 1))) (widen))) (defun find-assignment () (if (re-search-forward [^=!]=\\|\\+=\\|-=\\|\\*=\\|/=\\|=\\||=\\|\\^=\\|=\\|=\ \|:=\\|:\\|, (save-excursion (end-of-line) (point)) t) (progn (goto-char (match-beginning 0)) (if (looking-at .==) nil (if (looking-at \\+=\\|-=\\|\\*=\\|/=\\|=\\||=\\|\\^=\\|=\\|=\ \|:=\\|:\\|,) (set-mark (match-end 0)) (forward-char 1) (set-mark (1+ (point (delete-horizontal-space) t)) nil)) (global-set-key [(control =)] 'align-equals) -- snip end -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Generating cross recursive lazy sequences in clojure
Hi all, I am continuing on my path to explore clojure in more detail and am trying to implement the following haskell algorithm in clojure: http://www.csse.monash.edu.au/~lloyd/tildeFP/Haskell/1998/Edit01/ Even after trying several different approaches and investing several hours I do not seem to get a handle on the problem. Clojure seems to be lazy, but not lazy enough for that algorithm, because functions are always called eagerly. I tried combinations of returning closures or using delays, but up to now none of my versions are working. Could anybody on the list give me a hint on how to approach that problem? Thanks, Christian -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Generating cross recursive lazy sequences in clojure
I understand now the problem. Clojure is really not lazy enough :) I was forgetting that clojure evaluates its function arguments eagerly like lisp and not lazily like haskell. I have to wrap the function arguments that should be evaluated lazily into closures (fn [] value). Once I have a solution to the clojure implementation of the mentioned algorithm I'll post it here. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Generating cross recursive lazy sequences in clojure
Finally! I have a solution. You can have a look at it here: https://gist.github.com/889354/ I would like to hear comments about how to do it better or in a more idiomatic clojure way. Especially I am uncertain about my use of binding and the top level declares. I needed the ability to reference the symbols from the mutually recursive sequences and in addition the symbols needed to have dynamic extent. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Clojure in Small Pieces -- Literate Clojure
Hello,
I only had time now to look at this.
I am running Ubuntu 10.04 LTS Lucid Lynx and on my machine running the
tangle program had always the effect that the clojure.pamphlet file was
reduced to file size 0.
Looking ad the man page for the open system call (man 2 open) says that
there is no version of the open system call with only one parameter.
I've modified the tangle program and now its working.
I've attached the patch.
I cannot imagine that I am the only one with that problem. Am I?
Thanks,
Christian
daly wrote:
On Wed, 2011-03-23 at 00:31 -0700, Christian Schuhegger wrote:
Does a public repository exist (e.g. on github) where people could
clone the repository and potentially contribute?
There is no repository. Everything (including all of the runnable
source) is in a single book. To build a running Clojure REPL and
a PDF copy of the book do:
wget http://literatesoftware.com/clojure.pamphlet
wget http://literatesoftware.com/tangle.c
(the tangle.c source is also in the book so you could extract
it from the pamphlet file instead)
wget http://literatesoftware.com/clojure.pdf
(not really necessary as the next step will create the pdf
but you might want to read it first)
gcc -o tangle tangle.c
tangle clojure.pamphlet MakefileMakefile
make
You end up with a running Clojure REPL and a PDF of the book.
To make changes just modify the clojure.pamphlet file and type
make
You can send me your complete changed version or do
diff -Naur clojure.pamphlet.original clojure.pamphlet.newclojure.patch
The basic philosophy of literate programming is that you
are writing for other people, not for the machine. This book
is an attempt to make the Clojure source code readable by
more people. What we are trying to do is start from the ideas
and move to the code so people understand why the code does
what it does.
Tim Daly
--
Christian Schuhegger
http://www.el-chef.de/
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en--- orig/tangle.c 2011-02-16 15:18:40.0 +0100
+++ tangle.c 2011-03-25 19:37:30.035951116 +0100
@@ -3,6 +3,8 @@
#include string.h
#include sys/stat.h
#include sys/mman.h
+#include sys/types.h
+#include fcntl.h
#define DEBUG 0
@@ -134,7 +136,7 @@
perror(Usage: tangle filename chunkname);
exit(-1);
}
- fd = open(argv[1]);
+ fd = open(argv[1], O_RDONLY);
if (fd == -1) {
perror(Error opening file for reading);
exit(-2);
Deep recursion, continuation passing style, trampolining and memoization
Hello all, I've implemented the levenshtein measure in clojure and am quite happy with it except that I run into stack overflows if I hand over long strings. My current solution looks like this: -- snip start -- (declare levenshtein-rec) (declare change-cost) (defn levenshtein Compute Levenshtein distance between 2 strings [a b] (binding [change-cost (fn [x y] (if (= x y) 0 1)) levenshtein-rec (memoize (fn [x y] (cond (empty? x) (count y) (empty? y) (count x) :default (min (+ 1 (levenshtein-rec (rest x) y)) ; deletion (+ 1 (levenshtein-rec x (rest y))) ; addition (+ (change-cost (first x) (first y)) (levenshtein-rec (rest x) (rest y))) ; change )) ) ) ] (levenshtein-rec a b))) ;;;(levenshtein kitten sitting) ;;;(time (levenshtein (apply str (repeat 300 kitten )) (apply str (repeat 300 sitting -- snip end -- Now what I've done to overcome that limitation was to transform the function from above manually into continuation passing style (CPS), adapted it to be ready for trampolining (return in certain places functions instead of evaluating them) and added memoization. The result is extremely ugly and can be seen here: https://gist.github.com/880873 I did not look yet into the clojure libraries for automatic CPS transformation, but I am nearly certain that these libraries will not take care of adapting the result to be ready for trampolining and definitely will not take care of memoization. Would somebody on the list have a proposal on how to create such a solution simpler and nicer? Especially how would you apply memoization easily in CPS style? I've found an article on the web where somebody did something in that direction in F#: http://www.patrickdewane.com/2009/02/continuation-passing-style-and-memoization-part-2.html Many thanks and best regards, Christian -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Deep recursion, continuation passing style, trampolining and memoization
Actually Mark is right. My point is not about the Levenshtein distance. I've even found a quite nice and concise implementation here on the list a few weeks ago: generic (works for any seq) levenshtein distance https://groups.google.com/group/clojure/browse_frm/thread/c5da3ac1b6704eda My point is about what the subject says: deep recursion, CPS, trampolining and memoization. My question is: how to do that in idiomatic clojure in general? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: intersection and union on sequences/lists
You're right. Your version does what I want. Actually I've just seen that common lisps behaviour in the case of duplicates is undefined: http://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node152.html I was certain that it behaved the way your intersect behaves. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
intersection and union on sequences/lists
Hi all, I am coming from common lisp and was wondering if clojure supports intersection and union not only on sets, but also on normal sequences/ lists? I just did a google search but without a result. For the moment I'm writing those functions myself. Thanks for any hints, Christian -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Implementing let-over-lambda chapter 7 badger network for sorting
Hello,
after my first attempts with clojure I needed to evaluate the clojure
results against pure Java results.
First to make my report complete here are my OS and JVM details. I run
my tests on an Ubuntu 64bit 10.04 on Intel i7 3GHz hardware with a
custom compiled kernel using the BFS scheduler.
-- snip start --
Linux i7 2.6.34-custom-201005231602 #1 SMP PREEMPT Sun May 23 16:06:01
CEST 2010 x86_64 GNU/Linux
java version 1.6.0_20
Java(TM) SE Runtime Environment (build 1.6.0_20-b02)
Java HotSpot(TM) 64-Bit Server VM (build 16.3-b01, mixed mode)
-Xmx512m -Xms512m -server -XX:+DisableExplicitGC -XX:+AggressiveOpts -
XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=60 -
XX:ParallelGCThreads=5 -XX:MaxGCPauseMillis=10 -XX:+UseParNewGC -XX:
+CMSParallelRemarkEnabled -XX:+CMSParallelSurvivorRemarkEnabled -XX:
+UseNUMA -XX:+UseTLAB -XX:+BindGCTaskThreadsToCPUs -XX:
+UseGCTaskAffinity -XX:-DontCompileHugeMethods
-- snip end --
What I did is to convert my clojure program into a Java source code
generator that create a sort implementation like this:
-- snip start --
import java.util.Comparator;
public class BatcherSortNetwork50 {
private static T void swap(T[] input, int i1, int i2) {
T tmp = input[i1];input[i1] = input[i2];input[i2] = tmp;
}
private static T void cas(T[] input, int i1, int i2, ComparatorT
comparator) {
if(1 == comparator.compare(input[i1],input[i2])) swap(input, i1,
i2);
}
public static T T[] sort(T[] input, ComparatorT comparator) {
cas(input, 0, 32, comparator);
cas(input, 1, 33, comparator);
cas(input, 2, 34, comparator);
...
-- snip end --
In addition I use the benchmark code from ellipticgroup described
here:
http://ellipticgroup.com/html/benchmarkingArticle.html
http://www.ibm.com/developerworks/java/library/j-benchmark1.html
http://www.ibm.com/developerworks/library/j-benchmark2/index.html
The test always works the same:
1) initially I create an array of random items with length 50.
2) then I iterate over the following block:
3a) clone the array
3b) sort the array
For all tests I use the same initial random array, e.g. all sort
implementations have the same piece of work to perform.
The measurements show that the cloning of the array takes roughly
60ns (e.g. this is negligible).
The Arrays.sort implementation needs 2.628us.
The batcher sort implementation needs 2.895us.
All in all the two versions are more or less equal. But that was not
satisfactory, because in the end I wanted to get a performance
improvement and not an implementation that is more or less on par with
the provided generic implementation.
Therefore I modified the clojure source code generator in such a way
that it creates C code. I compared the performance of the qsort libc
implementation against my batcher implementation. The code in C looks
like this:
-- snip start --
static int compare_doubles (const double *x,const double *y) {
if (*x *y)
return 1;
else if (*x *y)
return -1;
else
return 0;
}
static void batcher_sort_50(double a[]) {
double tmp;
if(1 == compare_doubles((a[0]),(a[32]))) {tmp = a[0]; a[0] =
a[32]; a[32]=a[0];}
if(1 == compare_doubles((a[1]),(a[33]))) {tmp = a[1]; a[1] =
a[33]; a[33]=a[1];}
...
-- snip end --
The C code is not 100% comparable to the Java code, because in C I
only sort double arrays and in Java I compare Object
arrays. Nevertheless the results look much more promising:
qsort : 2.880us
batcher sort: 0.315us
Finally! I have my factor of 10 performance improvement that I was
looking for. The qsort implementation seems to be on the same level as
the Java Arrays.sort() implementation but the C version of my batcher
sort is much faster. I would be interested to hear about ideas on how
to explain why the Java batcher version cannot keep up with the C
version?
Also interesting to note:
Initially my Java implementation of the batcher sort looked the same
as the C version, e.g. I did not call the cas() method but had one
large message doing everything inline. In that case my batcher sort
was a factor of 10 slower than the Arrays.sort() implementation and
when looking at the output of -XX:+PrintCompilation I saw a message:
COMPILE SKIPPED: out of nodes parsing method
The reason why my first Java version was slower had something to do
with not compiling the method, even if I explicitly said
-XX:-DontCompileHugeMethods. No idea what happened here.
I used the Sun Studio collect/analyzer pair to identify the root cause
why the Clojure version cannot keep up with the native Java
implementation. I saw several calls to the reflection API in the
different call stacks even if *warn-on-reflection* did not warn me
about anything in my code. I have to dig deeper here.
What follows is the complete output of the ellipticgroup benchmark
classes plus my Callable implementations for the different tests. If
anybody is interested I can
Re: Implementing let-over-lambda chapter 7 badger network for sorting
I understand the differenece between Object vs doubles and in fact I do not want to point out that C is faster than Java (which it is not) but I want to see the 10:1 speed improvement between my Batcher sort implementation and the standard sort. The ellipticgroup Benchmark class is very soffisticated in making sure that the results are really meaningful. Please have a look at the two developerworks articles mentioned above plus the additional material given on the ellipticgroup web site. In addition to what the Benchmark class does (to double check so to say) I made sure that all code is compiled in advance before the measuring part begins by looking at the output of -XX:+PrintCompilation. Therefore the measurement is only about the sorting itself and not about JIT compilation. My question is: what do I have to do in Java in order to see the 10:1 speed improvement that should be possible? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Implementing let-over-lambda chapter 7 badger network for sorting
Hi all, just out of interest I've done the tests with double arrays in Java. The results for the Arrays.sort are 1.060us. Now it's becoming interesting. The results for the Batcher sort are: 803.784ns. Already an improvement! Finally I've tried once again to do the Batcher double sort in the same way as I did in C with the inlining of all statements in one method and the results are: 274.880ns!! The same league as the C version :) This time I did not get the COMPILE SKIPPED: out of nodes parsing method message when watching the compiler output. I would have guessed that the JVM would do by itself that kind of inlining, but it obviously does not. This inlining has the biggest effect of all. I'll try to find out how to create an inlining version that does not cause this COMPILE SKIPPED. For completeness here are the results of the Benchmark class: -- batcherDoubleSort: action statistics: first = 10.576 us, mean = 803.784 ns (CI deltas: -369.781 ps, +397.860 ps), sd = 1.720 us (CI deltas: -229.938 ns, +371.988 ns) WARNING: SD VALUES MAY BE INACCURATE -- --the action statistics were calculated from block statistics --each block measured 128 task executions --the user says that task internally performs m = 1 actions --then the number of actions per block measurement is a = 128 --block statistics: mean = 1.029 s (CI deltas: -473.320 us, +509.260 us), sd = 1.946 ms (CI deltas: -260.145 us, +420.856 us) --the forumla used to convert block statistics to action statistics (mean scales as 1/a, sd scales as 1/sqrt(a)) assumes that the action execution times are iid -- --each confidence interval (CI) is reported as either +- deltas from the point estimate, or as a closed interval ([x, y]) --each confidence interval has confidence level = 0.95 -- --sd results have unknown validity (the environmental noise test was skipped) -- --action sd values ALMOST CERTAINLY GROSSLY INFLATED by outliers --they cause at least 94.50279412648933% of the measured VARIANCE according to a equi-valued outlier model --model quantities: a = 128.0, muB = 1.028844119133, sigmaB = 0.0019464056773121957, muA = 8.037844680729166E-7, sigmaA = 1.7203958167093106E-6, tMin = 0.0, muGMin = 4.018922340364583E-7, sigmaG = 1.0047305850911458E-7, cMax1 = 229870, cMax2 = 66440, cMax = 66440, cOutMin = 66440, varOutMin = 3.5802336876801615E-6, muG(cOutMin) = 4.018925748069688E-7, U(cOutMin) = 8.14454192152E-6 -- -- javaDoubleSort: action statistics: first = 1.337 us, mean = 1.060 us (CI deltas: -774.553 ps, +952.287 ps), sd = 3.829 us (CI deltas: -703.088 ns, +1.507 us) WARNING: execution times have mild outliers, SD VALUES MAY BE INACCURATE -- --the action statistics were calculated from block statistics --each block measured 128 task executions --the user says that task internally performs m = 1 actions --then the number of actions per block measurement is a = 128 --block statistics: mean = 1.356 s (CI deltas: -991.428 us, +1.219 ms), sd = 4.332 ms (CI deltas: -795.453 us, +1.705 ms) --the forumla used to convert block statistics to action statistics (mean scales as 1/a, sd scales as 1/sqrt(a)) assumes that the action execution times are iid -- --each confidence interval (CI) is reported as either +- deltas from the point estimate, or as a closed interval ([x, y]) --each confidence interval has confidence level = 0.95 -- --EXECUTION TIMES APPEAR TO HAVE OUTLIERS --this was determined using the boxplot algorithm with median = 1.356 s, interquantileRange = 5.429 ms --1 are mild (on the high side): #59 = 1.373 s -- --sd results have unknown validity (the environmental noise test was skipped) -- --action sd values ALMOST CERTAINLY GROSSLY INFLATED by outliers --they cause at least 98.00343339546181% of the measured VARIANCE according to a equi-valued outlier model --model quantities: a = 128.0, muB = 1.356377725948, sigmaB = 0.004331502593009262, muA = 1.0596700983984375E-6, sigmaA = 3.828543570304954E-6, tMin = 0.0, muGMin = 5.298350491992187E-7, sigmaG = 1.3245876229980468E-7, cMax1 = 91174, cMax2 = 24081, cMax = 24081, cOutMin = 24081, varOutMin = 1.838732058970935E-5, muG(cOutMin) = 5.298421637517654E-7, U(cOutMin) = 2.869228377114931E-5 -- -- batcherDoubleSortInlining: action statistics: first = 9.769 us, mean = 274.880 ns (CI deltas: -32.818 ps, +54.994 ps), sd = 372.099 ns (CI deltas: -125.295 ns, +165.081 ns) WARNING: EXECUTION TIMES HAVE EXTREME OUTLIERS, SD VALUES MAY BE INACCURATE -- --the action statistics were calculated from block statistics --each block measured 512 task executions --the user says that task
maven compilation of a clojure project hangs due to still running thread pools
Hi all, I just had a lengthy debugging session behind me to find out why maven hangs when trying to execute the clojure:compile target of the clojure-maven-plugin. I found out that the problem does not lie in maven but in clojure (or in my code if you want to see it that way). My code executes some expressions that cause the clojure thread pools to be started. These thread pools stay alive after the compilation phase is over. My intent would be that such objects are only created when running the program but not when compiling it. The only reference where I found this problem discussed is from 2009-06-01: http://code.google.com/p/clojure/issues/detail?id=120 but I did not find any follow up or recommendation on how to deal with such issues? What is the current recommended way on how to deal with this situation? Many thanks, Christian -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
