Re: Closures eat permgen?
Ken~ The CMS settings are best for low latency applications or applications that want more predictable pause times. They can cause a drop in throughput for scientific computing or for Extract-Transform-Load style programs. CMS is not the default for historical reasons. The permgen sweeping is needed for programs that dynamically load and unload classes. Because this is a somewhat uncommon use case in the world of java, they are not the default. Matt On Thu, Nov 18, 2010 at 3:07 AM, Ken Wesson kwess...@gmail.com wrote: With those settings, it does not OOME even with 'eval'. It looks like those settings allow infinite sequential function creation without leaks. I did not closely evaluate GC performance. However I must suspect there to be some reasons why those settings aren't the default. I wouldn't mind knowing what those reasons are. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.comclojure%2bunsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Closures eat permgen?
Ken~ CMS (Concurrent Mark Sweep) is part of a multi-stage generational GC. It is the newest GC in a released version of the JVM (the G1 GC not having been released yet). With the below settings, the young gen is divided into Eden and two survivor spaces. The survivor spaces act as generations for young objects before they are tenured into the old gen. The concurrent mark sweep is only used for the old generation and is far more than a simple mark-sweep GC. If you are curious I can provide a somewhat more detailed explanation of the different collectors and phases, but I can assure you that these settings are very good defaults for high performance systems. Matt On Wed, Nov 17, 2010 at 10:37 PM, Ken Wesson kwess...@gmail.com wrote: On Wed, Nov 17, 2010 at 10:12 PM, Matt Fowles matt.fow...@gmail.com wrote: Ken~ Not sure what jvm args you are running with, but not all GC settings will sweep or clear the permgen. You should try it with: -XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled Maybe I will. (Use a mark-sweep GC instead of the generational one that all newer JVMs use by default, though? That will really hurt GC performance.) -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.comclojure%2bunsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: how to pass lazy seq to function for processing without keeping the head?
rb~
I believe that local variables are nulled after their last usage to prevent
exactly this behavior. In fact, you can find code like:
Object dummy(Object o, Object dummy) { return o; }
call_stuff(dummy(var, var = null));
In the Clojure source to accomplish this.
Matt
On Wed, Oct 27, 2010 at 4:12 PM, rb ruzsa.bal...@gmail.com wrote:
On Oct 27, 4:36 pm, Chris Maier christopher.ma...@gmail.com wrote:
On Wed, Oct 27, 2010 at 10:18 AM, rb ruzsa.bal...@gmail.com wrote:
P.s. I still don't understand though why the 'calls' argument to the
store-calls function is not a held reference to the head of the lazy
seq...
I think (and someone please correct me if I'm wrong) that the head of
'calls' isn't being retained because you recur on (rest calls),
replacing your hold on the head with the next item downstream. Now
nobody's holding on to the head, so it gets GC'd. Keep recurring, and
you just move down the sequence, always dropping what you were holding
onto, thus allowing it all to be GC'd.
Chris
Yes, this is quite clear in the case of the loop construct:
(loop [calls calls
conn nil
stmt nil
year nil
month nil]
(if-let [c (first calls)]
...
(recur (rest calls) conn stmt year month
The 'calls' variable bound in the loop is clearly not retained.
What bothers me is the function argument:
(defn store-calls
[source calls]
...
Here the lazy seq coming in as the second arg is bound to a local
variable named 'calls'. I would think this counts as retainment of the
head of the lazy seq (for the lifetime of the function), but it
apparently doesn't.
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with
your first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.comclojure%2bunsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
Re: precise numbers
Felix~ You are correct that the sequence of numbers 0.9 0.99 0.999 ... asymptotically approaches 1; however, the number 0.... (with an infinite number of 9s) is equal to 1. The formal proof of this is fairly tricky as the definition of the real number is usually done as an equivalence class of Cauchy sequences; a simplified version of the proof can be thought of as follows: For any two real numbers *a* and *b* there exists an infinite number of real numbers *c *such that *a c b*. However, there do not exist any numbers between 0.9... and 1, thus they must be same number. As it turns out, it took mathematicians a long time to nail down formally exactly what we naively think of as numbers. Matt On Wed, Oct 13, 2010 at 6:27 PM, Felix H. Dahlke f...@ubercode.de wrote: On 13/10/10 22:28, David Sletten wrote: On Oct 12, 2010, at 5:44 PM, Brian Hurt wrote: For example, in base 10, 1/3 * 3 = 0.9... It may seem counterintuitive, but that statement is perfectly true. 1 = 0.... That's a good test of how well you understand infinity. I'm clearly not a mathematician, but doesn't 0.9... asymptotically approach 1, i.e. never reaching it? How is that the same as 1? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Is it possible in theory to write/modify a Clojure compiler that doesn't
All~ There was a presentation at the JVM language summit about this exact topic. http://wiki.jvmlangsummit.com/Mixed_language_project_compilation_in_Eclipse:_Java_and_Groovy http://wiki.jvmlangsummit.com/Mixed_language_project_compilation_in_Eclipse:_Java_and_GroovyIn fact, the person was looking for more people to help contribute implementations for other languages to work together on this. Matt On Sun, Aug 29, 2010 at 5:44 PM, Robert McIntyre r...@mit.edu wrote: I don't think two pases is enough. What if a clojure file uses reflection (with macros of course) on a compiled java file to generate classes, sort of like how lancet works but generating classes instead of functions? And then those classes are used from other clojure and java files. Oh, and then another clojure file does the same reflection based junk to make more classes, from the classes that were generated from the first clojure file. These classes are, of course, used by other java and clojure files. In this case the only way I can think to compile everything (and sadly I'm doing something like this on my current project in lieu of a better way) is to: try to compile all the java files first (fail at some), try to compile all the clojure files (fail at some), try to compile all the java files (fail at some, but a little bit less than the first time), repeat until bytecode-level quiescence. The clojure compiler and java compiler will at most have to be invoked something like n times where n is the total number of source files in the project in certain pathological conditions as above. However, no matter what crazy stuff they're doing, as long as there no circular dependencies I think I can prove that this process will always make progress and converge to a steady state. I can make a concrete example of this condition if anyone's interested (at least I think I can!) As wasteful as it sounds is it really that bad? If the compilers only tried files that weren't already compiled and had minimal startup time then this sort of oscillation can be continued until quiescence is reached and will only take around the same time as single pass over all files. I've screwed around with this a lot and think that it would be a triumph if clojure and java could be intermixed freely. Then, there are no barriers to rewriting just one piece of a tightly interconnected java program in clojure for greater clarity/efficiency. You can just replace one java file with one clojure file, and use a java-clojure aware compiler instead of pure javac. I dream of the day where you could do this in a streamlined way in eclipse, so that everyone else on the project can focus on the concepts behind the program, not trivial minutiae like compile order. --Robert McIntyre On Sun, Aug 29, 2010 at 4:46 PM, Meikel Brandmeyer m...@kotka.de wrote: Hi, Am 28.08.2010 um 19:09 schrieb Michał Marczyk: I'm sure I'm missing lots of things, but I'd love to know which, so -- please let me know. :-) In fact, your two-pass scenario is probably the best you can get, since you can define arbitrary classes in arbitrary namespaces. (Whether this is advisable is a different question!) So any compiler trying to translate a classname to a defining namespace must fail in general. This could only be remedied by following a convention of one class per namespace which is quite a restriction. Just scanning the source files will also fail, because classes might be generated by macros. And macros might to depend on arbitrary functions defined in the file. So they can only be expanded by loading the file and executing the functions. Hence you cannot discover class generation in general. So I would suspect that your two pass scenario is a strict limit. Sincerely Meikel -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.comclojure%2bunsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.comclojure%2bunsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your
Re: Today's clojure trick question
Armando~
Libraries that target JRE 1.2 compatibility, will not call
`Boolean.valueOf(var)` internally. Instead they will call `new
Boolean(var)`. If they return those results to modern java code, it will
unbox correctly into either true or false and thus work as expected in
constructs of the form:
if (old_library_call()) {
// handle true
} else {
// handle false
}
If clojure's `if` form does not correctly unbox Booleans in the same manner,
then that really should be viewed as a bug in Clojure. As these libraries
are not likely to change to using the newer `Boolean.valueOf()` since that
break compatibility. Yes, they could create their own static factory for
Booleans and call that instead, but I don't think consumers of the library
are going to want to figure out (the hard way) that they need to hack their
library to use it from Clojure.
Matt
On Wed, Aug 18, 2010 at 4:07 PM, Armando Blancas
armando_blan...@yahoo.comwrote:
I don't see what the concern may be. Can you elaborate?
On Aug 18, 10:04 am, Matt Fowles matt.fow...@gmail.com wrote:
All~
Boolean.valueOf() was added in 1.4. While that seems ancient, some older
libraries use 'new Boolean()' because they maintain 1.2 compatibility.
It
seems like Clojure should take more care when it unboxes Booleans...
Matt
On Wed, Aug 18, 2010 at 12:57 PM, Nicolas Oury nicolas.o...@gmail.com
wrote:
I am not an expert. Is it possible on some JDK to put a breakpoint on
Boolean constructor and look at the stack?
Or you can't put a breakpoint on standard library?
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with
your first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.comclojure%2bunsubscr...@googlegroups.com
clojure%2bunsubscr...@googlegroups.comclojure%252bunsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en- Hide quoted text -
- Show quoted text -
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with
your first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.comclojure%2bunsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
Re: Today's clojure trick question
All~ Boolean.valueOf() was added in 1.4. While that seems ancient, some older libraries use 'new Boolean()' because they maintain 1.2 compatibility. It seems like Clojure should take more care when it unboxes Booleans... Matt On Wed, Aug 18, 2010 at 12:57 PM, Nicolas Oury nicolas.o...@gmail.comwrote: I am not an expert. Is it possible on some JDK to put a breakpoint on Boolean constructor and look at the stack? Or you can't put a breakpoint on standard library? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.comclojure%2bunsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Generics in PersistentXXX classes
Alex~ There is a project on github that does exactly this. http://github.com/krukow/clj-ds I don't know much about the current state of it, but I have plans in the next month or so to try it out at work. http://github.com/krukow/clj-dsMatt On Wed, Aug 4, 2010 at 6:10 AM, Alex Tkachman alex.tkach...@gmail.comwrote: Hi! Are there any plans to generify Clojure's collections like PersistentHashMap etc. to make them usable for Java/Scala/Groovy++ people? Does it make sense at all? If the only issue is resources I am volunteering to help. Best regards Alex -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.comclojure%2bunsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: starting and getting the most out of concurrent processes
Lee~ Doug Lea (of java.util.Concurrent fame) designed a framework called Fork/Join which is made to separate the idea of work from threads so that things can be parallelized out as much as possible. I don't know clojure that well, but would guess that it has some support for it and that you would see the best parallel fan out if you use the framework instead of explicitly managing your thread counts like this. Matt On Wed, Aug 4, 2010 at 2:51 PM, Lee Spector lspec...@hampshire.edu wrote: On Aug 4, 2010, at 2:37 PM, Armando Blancas wrote: Sorry, you had mentioned 1.1. The hint should be #^Callable Making this change, along with Mark's suggested change to burn-via-futures (except that I had to change the inner dorun to doall), I get the results below. It doesn't look to me like there's any difference with the explicit thread pool. -Lee 16 cores: 4 sequential burns: Elapsed time: 2005.169 msecs 4 sequential burns: Elapsed time: 1471.785 msecs 4 sequential burns: Elapsed time: 1289.797 msecs 4 burns via pmap: Elapsed time: 524.613 msecs 4 burns via pmap: Elapsed time: 506.69 msecs 4 burns via pmap: Elapsed time: 457.318 msecs 4 burns via futures: Elapsed time: 439.053 msecs 4 burns via futures: Elapsed time: 428.6 msecs 4 burns via futures: Elapsed time: 469.888 msecs 4 burns via agents: Elapsed time: 487.441 msecs 4 burns via agents: Elapsed time: 472.364 msecs 4 burns via agents: Elapsed time: 475.687 msecs 4 burns via a thread pool: Elapsed time: 496.032 msecs 4 burns via a thread pool: Elapsed time: 475.124 msecs 4 burns via a thread pool: Elapsed time: 451.277 msecs 16 sequential burns: Elapsed time: 6192.356 msecs 16 sequential burns: Elapsed time: 6255.118 msecs 16 sequential burns: Elapsed time: 6341.482 msecs 16 burns via pmap: Elapsed time: 963.27 msecs 16 burns via pmap: Elapsed time: 933.341 msecs 16 burns via pmap: Elapsed time: 963.068 msecs 16 burns via futures: Elapsed time: 959.335 msecs 16 burns via futures: Elapsed time: 909.289 msecs 16 burns via futures: Elapsed time: 945.639 msecs 16 burns via agents: Elapsed time: 948.955 msecs 16 burns via agents: Elapsed time: 887.656 msecs 16 burns via agents: Elapsed time: 965.753 msecs 16 burns via a thread pool: Elapsed time: 902.119 msecs 16 burns via a thread pool: Elapsed time: 942.239 msecs 16 burns via a thread pool: Elapsed time: 968.304 msecs 48 sequential burns: Elapsed time: 17280.549 msecs 48 sequential burns: Elapsed time: 16962.004 msecs 48 sequential burns: Elapsed time: 17222.746 msecs 48 burns via pmap: Elapsed time: 2856.71 msecs 48 burns via pmap: Elapsed time: 2751.755 msecs 48 burns via pmap: Elapsed time: 2517.552 msecs 48 burns via futures: Elapsed time: 2471.599 msecs 48 burns via futures: Elapsed time: 2488.509 msecs 48 burns via futures: Elapsed time: 2497.917 msecs 48 burns via agents: Elapsed time: 2532.471 msecs 48 burns via agents: Elapsed time: 2492.998 msecs 48 burns via agents: Elapsed time: 2475.815 msecs 48 burns via a thread pool: Elapsed time: 2491.58 msecs 48 burns via a thread pool: Elapsed time: 2483.124 msecs 48 burns via a thread pool: Elapsed time: 2475.017 msecs 48 cores: -- Lee Spector, Professor of Computer Science School of Cognitive Science, Hampshire College 893 West Street, Amherst, MA 01002-3359 lspec...@hampshire.edu, http://hampshire.edu/lspector/ Phone: 413-559-5352, Fax: 413-559-5438 Check out Genetic Programming and Evolvable Machines: http://www.springer.com/10710 - http://gpemjournal.blogspot.com/ -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.comclojure%2bunsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
