Re: Enhanced Primitive Support Syntax
Yes, very important difference that. Clojure will *not* return incorrect results on overflow (Java will) but it might throw an exception. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Enhanced Primitive Support Syntax
On Mon, Jan 17, 2011 at 3:10 AM, chris cnuern...@gmail.com wrote: Is it insane to suggest that perhaps clojure should work with scala such that we can write both languages in the same file? A lot of reasons for which it is not possible: - it would mean coordinating two implementations/implementers. - it would prevent to go to platform for which there is no support in the other language. - A type checker would not be really happy to deal with a lot of Object - Object functions... - it would be ugly Having a bit of (optional) type inference for performance and compile-time safety in Clojure could be interesting though. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Enhanced Primitive Support Syntax
Yeah yeah! http://www.google.com/search?q=lisp+type+inference Chris On Jan 17, 5:55 am, nicolas.o...@gmail.com nicolas.o...@gmail.com wrote: On Mon, Jan 17, 2011 at 3:10 AM, chris cnuern...@gmail.com wrote: Is it insane to suggest that perhaps clojure should work with scala such that we can write both languages in the same file? A lot of reasons for which it is not possible: - it would mean coordinating two implementations/implementers. - it would prevent to go to platform for which there is no support in the other language. - A type checker would not be really happy to deal with a lot of Object - Object functions... - it would be ugly Having a bit of (optional) type inference for performance and compile-time safety in Clojure could be interesting though. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Enhanced Primitive Support Syntax
Hi Stuart, On 2011-01-15, at 4:06 PM, Stuart Halloway wrote: In my experience, errors are the problem and we should be avoiding them, almost at all costs. This debate always starts by conflating three things into two, and then goes downhill from there. :-( It isn't (a) safe/slow vs. (b) unsafe/fast. That's how us outsiders are left to look at it. It is (a) unsafe/incorrect value on overflow/fastest/unifiable* vs. (b) safe/error on overflow/fast/unifiable vs. (c) safe/promoting on overflow/slow/not-unifiable *unifiable: able to deliver same semantics for primitives and objects This doesn't really help me understand your argument. It looks to me as though Clojure is trying to steer itself through the middle of something. The trouble is that I don't know where the edges of the middle are. Maybe it is just a documentation problem. But I'd also suggest that there's a bit of a sales job necessary here. We have thought about this quite a bit, Nobody doubts that, certainly I don't. And I'm not trying to minimise or dismiss what you've done. And I'm not claiming that I've thought about it better or more or deeper. But I do have concerns and I don't see them being addressed, and I'd like it if they weren't minimised either. Maybe my concerns are completely addressed. Maybe not. I don't know, and I'd like to be convinced. and an argument from one axis only (e.g safe/unsafe) that doesn't even mention some of the other axes is not likely to be persuasive. Would be more interesting to see a new axis we haven't thought of... Numerical correctness, for some of us, is an overwhelming issue. This is purely from experience... bad experience... 30+ years of bad experience in my case :-) From my point of view, the approach Clojure is taking isn't persuasive, not to say it couldn't be made persuasive. I think I did add what might be considered an additional axis. Syntax. Specifically what annotations are needed and for what purpose. I don't think this should be dismissed out of hand. Cheers, Bob Stu Stuart Halloway Clojure/core http://clojure.com -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en Bob Hutchison Recursive Design Inc. http://www.recursive.ca/ weblog: http://xampl.com/so -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Enhanced Primitive Support Syntax
On Jan 16, 6:18 pm, Sean Corfield seancorfi...@gmail.com wrote: On Sun, Jan 16, 2011 at 3:50 PM, Jason Wolfe ja...@w01fe.com wrote: Moreover, we do not need to redefine the class at run-time. A simple way to do this: when you compile a function with arithmetic operations, concatenate bytecode for two versions: essentially, one with the unprimed (exception-throwing) ops and one with the primed (overflowing) ones. Now, replace the exceptions in the first path with local jumps into the second path. There are probably ways to get around generating two copies of the bytecode in advance, and making such a unilateral shift to inefficient semantics, but I don't know enough about JVM internals to say for sure. The problem is that if you have an arbitrary form that can operate entirely in primitives (some loop/recur perhaps) and you allow primitives to magically convert to Objects in that code, then the entire piece of code has to handle both primitives AND Objects and every single sub-form must be capable of handling primitives as input AND Objects as input and returning primitives if possible... You can't have automatic promotion to Object from primitive and expect any reasonable code to be generated that can maintain primitive performance across arbitrary expressions. Either everything can work with Objects - and you lose performance - or everything must be able to work within primitives (and at most throw exceptions) and remain performant. I think you can. Let me elaborate on my simplistic example. Compile the code for a function twice -- once where everything works within primitives, and once where everything works with Objects -- and concatenate the bytecode together. Start by running the primitive version. On overflow, jump into the corresponding spot in the object version (after some patching up / boxing) -- rather than throwing an exception. If no overflow happens, you run *exactly* the same bytecode as the current unprimed ops. (I'm not sure how this would interact with JIT compared to exceptions, though). As soon as overflow happens, you run code as if all of the operations were primed, until function call exit. I believe this is related to the approach used by TraceMonkey, Firefox's Javascript compiler: http://hacks.mozilla.org/2009/07/tracemonkey-overview/ Anyway, I'll leave this be now; I just wanted to mention the idea since I hadn't seen it discussed. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Enhanced Primitive Support Syntax
On Mon, Jan 17, 2011 at 1:05 PM, Bob Hutchison hutch-li...@recursive.ca wrote: Numerical correctness, for some of us, is an overwhelming issue. This is purely from experience... bad experience... 30+ years of bad experience in my case :-) From my point of view, the approach Clojure is taking isn't persuasive, not to say it couldn't be made persuasive. Under the current proposal, you should never get an incorrect answer. You might get an error, though. It's a subtle difference, but this is the main reason why the developers don't see it as a correctness issue. If your program runs, and gives you back an answer, it will be correct. If it crashes, you convert to the overflow version of arithmetic, or typecast some of your numbers to bigints, and you'll get the right answer. I think a lot of the argument from both sides boils down to how much you fear runtime crashes. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Enhanced Primitive Support Syntax
On Jan 17, 3:17 pm, Jason Wolfe ja...@w01fe.com wrote: I think you can. Let me elaborate on my simplistic example. Compile the code for a function twice -- once where everything works within primitives, and once where everything works with Objects -- and concatenate the bytecode together. Start by running the primitive version. On overflow, jump into the corresponding spot in the object version (after some patching up / boxing) -- rather than throwing an exception. If no overflow happens, you run *exactly* the same bytecode as the current unprimed ops. (I'm not sure how this would interact with JIT compared to exceptions, though). As soon as overflow happens, you run code as if all of the operations were primed, until function call exit. I'm not sure the JVM allows methods to do that; it might be possible though. This doesn't address the issue that a method has to decide up front whether it is going to return a primitive or an object. I suppose you could compile two versions of each function, but brings up two issues: First, there would be massive code bloat, which might hurt the ability of the code to be JITed well. Second, if a function is hinted to return a primitive, it would almost certainly be an error to return something that isn't a primitive. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Enhanced Primitive Support Syntax
On Jan 17, 3:24 pm, Brian Goslinga quickbasicg...@gmail.com wrote: On Jan 17, 3:17 pm, Jason Wolfe ja...@w01fe.com wrote: I think you can. Let me elaborate on my simplistic example. Compile the code for a function twice -- once where everything works within primitives, and once where everything works with Objects -- and concatenate the bytecode together. Start by running the primitive version. On overflow, jump into the corresponding spot in the object version (after some patching up / boxing) -- rather than throwing an exception. If no overflow happens, you run *exactly* the same bytecode as the current unprimed ops. (I'm not sure how this would interact with JIT compared to exceptions, though). As soon as overflow happens, you run code as if all of the operations were primed, until function call exit. I'm not sure the JVM allows methods to do that; it might be possible though. I believe it's possible, as long as the bytecodes are combined into a single method body (due to the local jump restriction). This doesn't address the issue that a method has to decide up front whether it is going to return a primitive or an object. I suppose you could compile two versions of each function, but brings up two issues: First, there would be massive code bloat, which might hurt the ability of the code to be JITed well. This is an issue. I believe it might be avoidable with some tricks, but I don't know enough about JVM internals to say for sure. (I can elaborate on ideas if requested). Second, if a function is hinted to return a primitive, it would almost certainly be an error to return something that isn't a primitive. Correct. As I mentioned earlier, if you hint the return as long, we would always return a long (truncating/throwing if necessary). This is only talking about behavior *within* a function; the external interface of the function is not up for interpretation. And, taking a step back, what people seem to be concerned with is long semantics being inferred when that's not what they want. If the user explicitly declares the return value to be a long, this concern would no longer apply. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Enhanced Primitive Support Syntax
I'll also add that type inference wouldn't solve the problem, it would just move the pain the the design of the type system and details relating to it. The type system would probably be at least as complex as Java generics to be something worthwhile if you do the type inferencing for perf primarily and type checking incidentally; if you want type checking primarily it would most likely be even more complicated. The changes in 1.3 don't really make the language more complicated; in contrast, type inferencing would. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Enhanced Primitive Support Syntax
On 16 January 2011 05:35, Jason Wolfe ja...@w01fe.com wrote: (a) unsafe/incorrect value on overflow/fastest/unifiable* vs. (b) safe/error on overflow/fast/unifiable vs. (c) safe/promoting on overflow/slow/not-unifiable If I understand correctly, the issue with auto-promotion is that we have to box the output of an operation even if it turns out to fit in a long, since the compiler must specify whether the result will be long or Object. This is probably a stupid question, but since Clojure is already testing for overflow in (b) to throw an exception, would it be possible to jump into an alternative (c)-type compilation of the function just-in-time to promote on overflow instead? It seems like this could achieve the performance of (b) while still allowing for auto-promotion (albeit perhaps with a performance hit in that case, code bloat, compromises about how many versions to compile, etc.). When Clojure compiles your function, it emits JVM bytecode for a new class which is then loaded by the classloader. That JVM bytecode defines a function (well, a method as far as the JVM is concerned) which returns either a primitive type or Object. Your suggestion would involve redifining the class while it is executing. That's not possible on the JVM. Even if it were possible -- your function now returns Object instead of long. But the variable that the result of your function is about to be assigned to is a long, because that's what your function used to be defined to return, and the next bytecode operation in the calling code is the one that subtracts a primitive long from a primitive long. Now what? Fundamentally, Clojure has to contend with the fact that the JVM as a platform distinguishes between primitives and Objects. The bytecode operations which the Clojure compiler emits have to differ based on that distinction. Essentially, the distinction cannot be magicked away, because JVM bytecode is going to statically enforce that we're either working with a (long or double) or a (Long or BigInt or BigDecimal or whatever), and never the twain shall meet. So if we ever want to be able to access the speed of the primitive bytecode operations, then the primitive/Object distinction has to leak into Clojure. (Or we have to redefine or move away from the JVM. Obviously not really an option, but I mention it to point out that the reason Clojure has to make this decision is that it's a hosted language. That has a lot of benefits; this is one of the trade-offs we have to contend with in return.) I think everyone agrees that it's important to make the speed of the primitive bytecode operations available in Clojure (whether or not it's the default), so that rules out the option of always doing auto-promotion. I think it's probably also agreed that allowing unchecked overflow is not good (at least, no one seems to be arguing for it). So we're left with option (b) and a choice about the default behaviour of the core library functions. If we default to boxing and treating everything as an Object then we get the nice comfy numeric tower that we never have to worry about, but the default case suffers in performance. Otherwise, we default to primitive, and accept that if we're dealing with numbers which might get bigger than Long.MAX_VALUE, then we might need to explicitly use a BigInt to get contagion, or use an operator like +' which will always deal with Objects. By choosing to make speed the default preference in the core library functions, I suppose there's more for Clojure programmers to think about, because whenever you're dealing with numbers, you need to have in the back of your mind the question of whether this might ever need to be bigger than Long.MAX_VALUE, and so whether you might need +' instead of +. Then again, how often do you write code that might be doing maths with numbers that big and not realise it? For that matter, how often do you write code that might be doing maths with numbers that big and not spend time thinking carefully about its performance anyway? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Enhanced Primitive Support Syntax
Then again, how often do you write code that might be doing maths with numbers that big and not realise it? For that matter, how often do you write code that might be doing maths with numbers that big and not spend time thinking carefully about its performance anyway? This reminds me of a little joke: Cooper: Honey, will you please - what are the odds of the Russians attacking on a Thursday night? --The Man with One Red Shoe (1985) Having heard from various posters how reasonable it all is, and from Rich now necessary [1], I still have a feeling that the implications and consequences of the new semantics aren't well understood, and if anything, any points of real concerned are downplayed. I'll second Mark's concerns, above. Like others, I don't like to annotate and postquote my way to the semantics I want, but that may be the easy part. But I hope I'm wrong; I guess we'll see. [1] (in this thread http://groups.google.com/group/clojure/browse_thread/thread/c8c850595c91cc11/8a4eee5ac4eab3f9?lnk=gstq=autopromotion#8a4eee5ac4eab3f9) -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Enhanced Primitive Support Syntax
(a) unsafe/incorrect value on overflow/fastest/unifiable* vs. (b) safe/error on overflow/fast/unifiable vs. (c) safe/promoting on overflow/slow/not-unifiable If I understand correctly, the issue with auto-promotion is that we have to box the output of an operation even if it turns out to fit in a long, since the compiler must specify whether the result will be long or Object. This is probably a stupid question, but since Clojure is already testing for overflow in (b) to throw an exception, would it be possible to jump into an alternative (c)-type compilation of the function just-in-time to promote on overflow instead? It seems like this could achieve the performance of (b) while still allowing for auto-promotion (albeit perhaps with a performance hit in that case, code bloat, compromises about how many versions to compile, etc.). When Clojure compiles your function, it emits JVM bytecode for a new class which is then loaded by the classloader. That JVM bytecode defines a function (well, a method as far as the JVM is concerned) which returns either a primitive type or Object. Your suggestion would involve redifining the class while it is executing. That's not possible on the JVM. Even if it were possible -- your function now returns Object instead of long. If I understand correctly, the return type of your function is not up for interpretation. It is Object if undeclared, and primitive only if declared -- so that is not a problem (if you ask for long, that's what you'll get). I am only talking about behavior within a single function call -- no calling code needs to change. Moreover, we do not need to redefine the class at run-time. A simple way to do this: when you compile a function with arithmetic operations, concatenate bytecode for two versions: essentially, one with the unprimed (exception-throwing) ops and one with the primed (overflowing) ones. Now, replace the exceptions in the first path with local jumps into the second path. There are probably ways to get around generating two copies of the bytecode in advance, and making such a unilateral shift to inefficient semantics, but I don't know enough about JVM internals to say for sure. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Enhanced Primitive Support Syntax
On Sun, Jan 16, 2011 at 3:50 PM, Jason Wolfe ja...@w01fe.com wrote: Moreover, we do not need to redefine the class at run-time. A simple way to do this: when you compile a function with arithmetic operations, concatenate bytecode for two versions: essentially, one with the unprimed (exception-throwing) ops and one with the primed (overflowing) ones. Now, replace the exceptions in the first path with local jumps into the second path. There are probably ways to get around generating two copies of the bytecode in advance, and making such a unilateral shift to inefficient semantics, but I don't know enough about JVM internals to say for sure. The problem is that if you have an arbitrary form that can operate entirely in primitives (some loop/recur perhaps) and you allow primitives to magically convert to Objects in that code, then the entire piece of code has to handle both primitives AND Objects and every single sub-form must be capable of handling primitives as input AND Objects as input and returning primitives if possible... You can't have automatic promotion to Object from primitive and expect any reasonable code to be generated that can maintain primitive performance across arbitrary expressions. Either everything can work with Objects - and you lose performance - or everything must be able to work within primitives (and at most throw exceptions) and remain performant. -- Sean A Corfield -- (904) 302-SEAN Railo Technologies, Inc. -- http://getrailo.com/ An Architect's View -- http://corfield.org/ If you're not annoying somebody, you're not really alive. -- Margaret Atwood -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Enhanced Primitive Support Syntax
Is it insane to suggest that perhaps clojure should work with scala such that we can write both languages in the same file? Use scala to do you strongly typed work and things where you are really concerned that auto-promotion. Let the language made for helping a programmer lots of information about his/her code (and perhaps overspecify the solution a gross amount) be used where that is necessary. Look, annotating shit in a clojure file to ensure you don't have a crapload of boxing and inference going in is a large PITA. So in areas where you know you are going to need to do lots of things like that, why wouldn't you use scala? Specifying the type of everything going on is obviously, for a lot of cases, grossly overspecifying the problem. But what if we could just write scala code in a clojure file or vice versa? It seems that you would bypass a lot of the odder 'improvements' to clojure for expert users and just be better off. Obviously one of the answers is 'if you think it is a good idea then you do it' and I don't have the time. But I know this: 1. Meta programming is an insanely powerful tool for compressing code. 2. Type inference is insanely powerful for producing programs that do 'exactly' what the programming said they should. 3. Clojure-in-java is fast and thus clojure-in-scala would be fast, no language additions required. And probably 30-50% shorter but perhaps not more than that. 4. Type annotations are as bad as C programming in terms of verbosity and to write fast code you need them. A little inference would make a large difference. Types in a lisp-like language suck. There isn't a way around it. Lets use a powerful type-inference tool where appropriate and ditch them completely where it is inappropriate. You want a hover-over or tooltip to tell you what type something is producing or taking (or to guarantee details about what you are doing)? Use scala. You want to write extremely compressed code in a way that is very easy to change and specifies as little of the answer to the problem as possible (thus giving a lot of leeway for implementation and extension)? Use clojure. Chris -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Enhanced Primitive Support Syntax
On Jan 15, 2:40 am, Armando Blancas armando_blan...@yahoo.com wrote: They used to give you compile switches for that kind of stuff, not hope and wholesome wishes. Seems like every performance improvements makes the language more complex, uglier or both. I don't feel strongly about integer limits at all and am always surprised when this comes up. I did scientific programming on a 32 bit platform for several years and never met anyone who hit big problems with fixed size integers. For illustration, Long.max is: 9 223 372 036 854 775 807 which is so much bigger than I was used to. I know encryption requires BigInteger but I have yet to see a native clojure encryption library. It would help people like me understand the debate if some mainstream examples of applications requiring (seamless) BigInteger support could be identified. Saul -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Enhanced Primitive Support Syntax
I'm going to re-organise this a bit… On Jan 14, 2:40 pm, Stuart Sierra the.stuart.sie...@gmail.com wrote: Debatable it is, endlessly. ;) So Clojure committers made a choice. Hopefully, they have made a choice that has: I agree that they've made a choice, and I really don't want to be too critical here. However, since Clojure 1.3 is still in an alpha stage, maybe this discussion can still contribute something. Maybe what I'm saying is that I want to stay constructive and that there's maybe still time to be constructive. * a small positive effect (better performance with less effort) on a majority of users * a small negative effect (worse performance, extra effort) on a minority These goals are good, but I don't know that the approach taken achieves them. In my experience, errors are the problem and we should be avoiding them, almost at all costs. Numbers are confusing to people. Numbers approximated on a computer are far more confusing. How many times do you see threads discussing how a compiler is buggy because it can't divide two numbers and get the right answer? I've been doing this stuff for years and I can come up with an awful lot of amusing and/or horribly nasty examples. But I don't think this needs to be re-established. Given my experience I *strongly* lean towards not making a 'mistake' due to compiler optimisations. In other words, I'd be very annoyed, and I'd expect others to be annoyed too, if a numerical error was introduced to one of my programs because of an unexpected, silent, compiler optimisation. Secondly, Clojure has already established that we will use type annotations to signal to the compiler what's what. When we annotate, we are relaxing our requirements on the compiler to not make a mistake by assuming that responsibility ourselves. I would suggest the following: 1) if there's type annotation on both values of, say, an addition, then the optimised version can be used. If there isn't, or the compiler isn't sure, then use safe operations. 2) if the compiler isn't cooperating (because it isn't sure what's going on) we should be helping it by again assuming the responsibility of being right and marking the operator, say with a tick. And yes, this likely has problems too. I'm not saying that this is an issue with easy solutions. We're heading for a hodgepodge of annotation purposes, some for optimisation, some for correctness (and one of these days I'll mention what I think of the @/deref thing :-) And now we're pretty much guaranteed ugly code no matter what. Though I'd prefer no ugliness, I'd trade ugly code for speed, but I'd rather not for correctness. And there's a practical problem with mixed annotation purposes. If you want to track down a bug you can't just remove all annotations temporarily. You'd have to remove some and add others. Not looking forward to that. Maybe a macro: make-this-safe could be written. Hmm. Maybe a 'defn-safe' would be something to think about??? This is also the kind of thing that you just can't fix later. Imagine how we'll feel in ten or twenty years about this decision. -S On 2011-01-14, at 8:40 PM, Armando Blancas wrote: They used to give you compile switches for that kind of stuff, not hope and wholesome wishes. Seems like every performance improvements makes the language more complex, uglier or both. Compiler switches were/are problematic too, but at least they are explicit and have to be *added*. Cheers, Bob Bob Hutchison Recursive Design Inc. http://www.recursive.ca/ weblog: http://xampl.com/so -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Enhanced Primitive Support Syntax
Hi, just for the record: from what I have done in the past I wholeheartedly agree with Bob. Of course the developers of Clojure have the final say on this. It's just my 2ct. Kind regards, Stefan -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Enhanced Primitive Support Syntax
Bob Hutchison said: In other words, I'd be very annoyed, and I'd expect others to be annoyed too, if a numerical error was introduced to one of my programs because of an unexpected, silent, compiler optimisation. Just to be clear, Clojure 1.3-alpha does not introduce numerical errors, unless you explicitly ask for them; it throws a RuntimeException - which I guess is analogous to it being a dynamically-typed language and throwing RuntimeExceptions to signal type errors. user= (* 1000 1000) ArithmeticException integer overflow clojure.lang.Numbers.throwIntOverflow (Numbers.java:1583) user= (*' 1000 1000) 100N user= (* 1000 1000N) 100N user= (unchecked-multiply 1000 1000) 1864712049423024128 -- Dave -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Enhanced Primitive Support Syntax
On Jan 15, 2011, at 8:29 AM, Saul Hazledine wrote: It would help people like me understand the debate if some mainstream examples of applications requiring (seamless) BigInteger support could be identified. I doubt that many will consider this mainstream, but I evolve programs using genetic programming techniques and I've found that in this context BigIntegers can arise in all sorts of unexpected and weird and wonderful and sometimes adaptive ways. I still haven't figured out exactly what the 1.3 changes will mean for this work -- maybe it'll be fine or even better -- but I've liked not having to think about integer sizes much at all previously (as in Common Lisp, where the handling of complex numbers is also nice). -Lee -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Enhanced Primitive Support Syntax
I think this is looking at the situation backwards. I don't want BigInts, why should I want *them*? Nor longs or whatever for that matter. What I want is a numerical tower and a language that can handle it correctly, without blowing up, and if possible with clean code. I can type 9223372036854775807 or something bigger in a spreadsheet and get 9.22337203685478E+018, and take it from there. Now, I don't know if spreadsheet programmers *want* doubles, but something correct (if less precise) must be done depending on what *I, the user* wants to do. The spreadsheet example may be a useful one. Clojure is a language you could write a spreadsheet in. Would you want to use a spreadsheet written entirely in a JVM language that did not provide access to the primitives? Now we face the choice of putting limits on what our users can do, use a different set of operators, or decide that we want BigInts. Again, this isn't about wanting bigints, that's a red herring. Part of the problem is the complex, confusing, and sometimes quite mysterious Type System that's been creeping into Clojure for the sake of performance. Clojure is not getting a type system. nor is the behavior in 1.3 complex. It can be confusing, because it is addressing a multifaceted problem, and it certainly is mysterious, because (1) we haven't spent enough effort documenting it, and (2) lots of people have misdocumented it. I'll make a documentation update higher priority; hopefully that will help. Another is the conflicted attitude of being an untyped language: a kind of guilty pleasure with the remorse it brings of all those reflective calls and boxing/unboxing, whose negative effects on performance supposedly makes the language lose credibility. The Clojure design process is not about achieving credibility, it is about solving problems. Credibility has followed, and will continue to follow, to the extent that Clojure solves problems well. Stu Stuart Halloway Clojure/core http://clojure.com -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Enhanced Primitive Support Syntax
On Sat, Jan 15, 2011 at 1:06 PM, Stuart Halloway stuart.hallo...@gmail.com wrote: We have thought about this quite a bit, and an argument from one axis only (e.g safe/unsafe) that doesn't even mention some of the other axes is not likely to be persuasive. Would be more interesting to see a new axis we haven't thought of... Here's an axis that hasn't gotten much discussion: How evident is the behavior of Clojure code, and what features help and hinder this clarity? Clojure is a dynamically typed language, which means that generally speaking, it is not obvious what the type of a given variable is, since there aren't annotations immediately prior to the variable telling you what it must be. Similarly, a Clojure IDE does not offer any way to to hover over a variable and see what the type is. There is a lot of freedom that comes with this, but the cost is that a dynamic programmer must be careful to document in some way what kinds of things are acceptable inputs, and what kinds of promises are made of the outputs. The compiler can't check this, so it's up to the programmer. As Clojure programmers, we take on the responsibility of tracking a certain amount of unseen information that isn't readily evident from the code itself, but there's a limit to how much responsibility programmers can take on before programs become brittle, so new features should take this axis into account. Primitives are especially problematic because there is no good way to determine whether something is a primitive or not. Consider the following interactions in the 1.1 REPL: user (type 1) java.lang.Integer user (type (int 1)) java.lang.Integer Any features involving primitives should be assessed from the standpoint that it is extremely difficult to know from looking at code whether something is a primitive or not. Many of the new features (e.g., static functions can now return primitives, literals are primitives, but numbers that get stored in collections or cross certain kinds of function boundaries are not), means that you'll frequently end up with a mixture of primitives and non-primitives, and it won't always be obvious which is which. When designing math operators that behave one way for longs and another for bigints, one question that needs to be asked is: How apparent will it be whether a variable represents a long or a bigint? If it's not apparent, how will the programmer know which behavior to expect? Is there any tooling that can help make this more discoverable? One possibility is that Clojure programmers will need to evolve ways to track this information, perhaps by explicitly commenting in code whether a function can gracefully handle both longs and bigints. On the other hand, there's already a history in Clojure and similar languages of just documenting certain vars as numbers without needing to get more precise than that, so this could be a painful transition for many programmers who are not used to thinking about specifying their numeric types in greater detail than that. Because it's difficult to do typeflow analysis within a dynamically-typed language as Clojure, this clarity axis also comes into play when thinking about what sorts of burdens are going to be placed on library developers. As a case in point, I developed the expt function in clojure.contrib.math because I was surprised when I first came to Clojure that no generic exponentiation operator existed in the language. The expt in contrib handles all of Clojure's numeric types seamlessly. But what am I supposed to do with expt in Clojure 1.3? New expectations are being created with the new model -- some people will expect expt with primitives to return primitives; some will expect computation with longs to return bigints when necessary, since exponentiation frequently overflows. Do I need to provide an expt and expt' function to make both camps happy? (For that matter, is there even a way to overload expt for both primitive longs and primitive doubles, or do I need to make separate expt-long and expt-double functions?) Are we going to see a proliferation of variations for all mathematical functions once we start going down this road? This is an axis I think about a lot, and I hope this is something that the Clojure dev team is carefully considering as well. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Enhanced Primitive Support Syntax
On Sat, Jan 15, 2011 at 9:40 PM, Mark Engelberg mark.engelb...@gmail.com wrote: On Sat, Jan 15, 2011 at 1:06 PM, Stuart Halloway stuart.hallo...@gmail.com wrote: We have thought about this quite a bit, and an argument from one axis only (e.g safe/unsafe) that doesn't even mention some of the other axes is not likely to be persuasive. Would be more interesting to see a new axis we haven't thought of... Here's an axis that hasn't gotten much discussion: How evident is the behavior of Clojure code, and what features help and hinder this clarity? [massive snip] This is an axis I think about a lot, and I hope this is something that the Clojure dev team is carefully considering as well. +1 to all of that. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Enhanced Primitive Support Syntax
On Jan 15, 2011, at 4:04 PM, Stuart Halloway wrote: I'll make a documentation update higher priority; hopefully that will help. This should help. I feel like the discussion is going in circles because there's no single, official source that summarizes exactly what is happening with numerics in 1.3. (I know about http://www.assembla.com/wiki/show/clojure/Enhanced_Primitive_Support, but it's terse and a bit confusing.) -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Enhanced Primitive Support Syntax
(a) unsafe/incorrect value on overflow/fastest/unifiable* vs. (b) safe/error on overflow/fast/unifiable vs. (c) safe/promoting on overflow/slow/not-unifiable If I understand correctly, the issue with auto-promotion is that we have to box the output of an operation even if it turns out to fit in a long, since the compiler must specify whether the result will be long or Object. This is probably a stupid question, but since Clojure is already testing for overflow in (b) to throw an exception, would it be possible to jump into an alternative (c)-type compilation of the function just-in-time to promote on overflow instead? It seems like this could achieve the performance of (b) while still allowing for auto-promotion (albeit perhaps with a performance hit in that case, code bloat, compromises about how many versions to compile, etc.). For what it's worth, I'm personally happy with the approach of the current alpha. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Enhanced Primitive Support Syntax
Debatable it is, endlessly. ;) So Clojure committers made a choice. Hopefully, they have made a choice that has: * a small positive effect (better performance with less effort) on a majority of users * a small negative effect (worse performance, extra effort) on a minority -S -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Enhanced Primitive Support Syntax
They used to give you compile switches for that kind of stuff, not hope and wholesome wishes. Seems like every performance improvements makes the language more complex, uglier or both. On Jan 14, 2:40 pm, Stuart Sierra the.stuart.sie...@gmail.com wrote: Debatable it is, endlessly. ;) So Clojure committers made a choice. Hopefully, they have made a choice that has: * a small positive effect (better performance with less effort) on a majority of users * a small negative effect (worse performance, extra effort) on a minority -S -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Enhanced Primitive Support Syntax
Hi all, I've been reading about the changes to Clojure that are occurring with regards to improving performance of arithmetic operations. For example, + no longer auto promotes and +' has been introduced as an auto-promoting addition operator. The idea of introducing additional syntax is a bit odd to me, so I had thought that it could be solved by setting an environmental variable for the namespace and the use of macros. For example, maybe set *enhanced-primitive-support* true or false will determine the behavior of the operators (+, -, etc.) for an entire namespace. If a user wants a different set of behaviors for a block of code in that namespaces, they could wrap the code in a macro called something like enhanced-primitive-support or old-primitive-support: (enhanced-primitive-support (+ a b) ) I was wondering why an approach like this was not taken? Thanks, RJ -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Enhanced Primitive Support Syntax
The goal of primitive math is better performance in the common case. The implementation makes the assumptions that Java long is big enough for nearly all cases, and that auto-promotion to BigInteger (and the resulting performance hit) is rarely desirable. The + function still promotes, it just promotes everything to long or double. If you want BigInts or BigDecimals, just insert one of them into the calculation, and everything else will get promoted to BigInt or BigDecimal. The only difference between + and +' is that +' will promote to BigInt if you overflow a long, whereas + will thrown an exception. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Enhanced Primitive Support Syntax
On Thu, Jan 13, 2011 at 7:28 PM, Stuart Sierra the.stuart.sie...@gmail.com wrote: The goal of primitive math is better performance in the common case. Of course, this better performance is not needed in the common case, IMO, but only in hotspots that do number crunching, where people already optimize using primitive locals, coercion, and unchecked-foo. The implementation makes the assumptions that Java long is big enough for nearly all cases It also makes the assumption that the Java long is as fast as native arithmetic. Which, on a lot of 32-bit hardware, it won't be. and that auto-promotion to BigInteger (and the resulting performance hit) is rarely desirable. Debatable. I, for one, prefer to have unadorned arithmetic be correct at the expense of a little speed, while still having a way to get the speed in performance-critical parts of my code. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
