Re: Mysterious ClassFormatError after simple code change.
On Mon, Jul 6, 2009 at 7:50 PM, Richard Newman holyg...@gmail.com wrote: Since it's not apparently a simple bug in my function above, but something about a combination of that version of that function and some other part of my code, I can't think of a way to track the cause down short of the very tedious method of commenting out functions or replacing them with dummy functions (if they're called by the above function) and seeing which one has the property that commenting it out or dummy-izing it makes the error go away. That will probably take all afternoon, unfortunately; there are dozens of functions and over 1600 lines of code in that source file. Have you tried simpler things like splitting the offending function into a separate namespace, or seeing what happens with (or without) AOT compilation? I didn't get around to that because I accidentally fixed the bug somehow. I moved a few functions that I realized were general purpose over to a separate utils project and recompiled everything. The source file with the problem contained some of the functions I moved, and now loads correctly. It looks like one of those functions conflicted in some way with the newer version of the function posted to this thread, though I'm baffled as to how or why. None were directly called by it, or otherwise directly referenced by it, nor vice versa. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Mysterious ClassFormatError after simple code change.
On Mon, Jul 6, 2009 at 11:25 PM, John Harrop jharrop...@gmail.com wrote: On Mon, Jul 6, 2009 at 7:50 PM, Richard Newman holyg...@gmail.com wrote: Have you tried simpler things like splitting the offending function into a separate namespace, or seeing what happens with (or without) AOT compilation? I didn't get around to that because I accidentally fixed the bug somehow. I moved a few functions that I realized were general purpose over to a separate utils project and recompiled everything. The source file with the problem contained some of the functions I moved, and now loads correctly. It looks like one of those functions conflicted in some way with the newer version of the function posted to this thread, though I'm baffled as to how or why. None were directly called by it, or otherwise directly referenced by it, nor vice versa. And now it's mysteriously back after I implemented a few more functions. I can't find any obvious errors in any of the new code. Weirdly, if I copy the entire source file, paste it at the REPL, and hit enter, the error does not occur! Somehow, code that is treated as valid when compiled a function at a time is treated as invalid when compiled all at once. That pretty much proves it's an implementation bug, since the same code can't both be buggy and be fine at the same time. :( --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Mysterious ClassFormatError after simple code change.
On Jul 7, 2009, at 5:51 AM, John Harrop wrote: Somehow, code that is treated as valid when compiled a function at a time is treated as invalid when compiled all at once. That pretty much proves it's an implementation bug, since the same code can't both be buggy and be fine at the same time. :( It sounds a lot like you're running into limits on the size or count of something in a class file: http://java.sun.com/docs/books/jvms/second_edition/html/ClassFile.doc.html (section 4.10 limitations of the JVM) If that's true, I think Clojure should be detecting the problem and reporting it in a way that's helpful rather than generating a bad class file. Are you able and willing to make the entire Clojure source file that's failing for you available so it's feasible to track down the problem? --Steve smime.p7s Description: S/MIME cryptographic signature
Re: Mysterious ClassFormatError after simple code change.
On Tue, Jul 7, 2009 at 9:30 AM, Stephen C. Gilardi squee...@mac.com wrote: On Jul 7, 2009, at 5:51 AM, John Harrop wrote: Somehow, code that is treated as valid when compiled a function at a time is treated as invalid when compiled all at once. That pretty much proves it's an implementation bug, since the same code can't both be buggy and be fine at the same time. :( It sounds a lot like you're running into limits on the size or count of something in a class file: Shouldn't be. I'm not using :gen-class or similar, so the only classes generated should be ones for the individual functions, and the largest one of those works, so is not exceeding any kind of limit. Furthermore, the clojure documentation does not state any limits on function size or similar, so implies that there are none. Last but not least, a change to a single function definition's interior triggered the error. If the error was: * too large a function, a much larger function elsewhere in the file would fail; * too many names or other things in the namespace, changing a single thing's interior without adding more top-level things would not fail; * just about anything else, either something else would have failed earlier or things would have stayed working, one or the other. That compiling the functions one by one in the REPL, in the same namespace, exceeds no limits is also worthy of note. It can't be a namespace size limit then, nor can any of the individual functions be exceeding a limit. In fact, the only thing different is that a load script of sorts has to be compiled for a load-file or build, which evaluates the definitions and other instructions in order. If that's hitting some sort of limit, the ability to execute exactly the same things in the same sequence at the REPL proves that that limit can be circumvented under the hood, simply by having load-file more faithfully emulate pasting everything into a REPL set to the appropriate namespace and hitting enter than it apparently currently does. Is it making the load script into a single huge static method perhaps? If so, having it break such things up under the hood into several smaller ones with a driver that calls them all in sequence would fix it, or even having it break up the -init class into -init, -init2, -init3 and so forth if necessary and making these invoke one another in chains. There's really no reason I can think of for anything like this to even be affecting the user. If that's true, I think Clojure should be detecting the problem and reporting it in a way that's helpful rather than generating a bad class file. That much I'd agree with. Are you able and willing to make the entire Clojure source file that's failing for you available so it's feasible to track down the problem? It's probably going to end up GPL'd eventually -- if I can get it to actually work and stay working -- but it's not presently in a releasable state. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Mysterious ClassFormatError after simple code change.
On Sun, Jul 5, 2009 at 3:51 PM, John Harropjharrop...@gmail.com wrote: This is frankly quite baffling. The changes to the function are innocent from a large-literal or pretty much any other perspective. Both your functions load fine for me without the rest of your code. Are there type hints on the return values of any of the functions called in your examples? --Chouser --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Mysterious ClassFormatError after simple code change.
On Mon, Jul 6, 2009 at 8:53 AM, Chouser chou...@gmail.com wrote: On Sun, Jul 5, 2009 at 3:51 PM, John Harropjharrop...@gmail.com wrote: This is frankly quite baffling. The changes to the function are innocent from a large-literal or pretty much any other perspective. Both your functions load fine for me without the rest of your code. Are there type hints on the return values of any of the functions called in your examples? Not that I know of. Called macros above are let-print; called functions are cons, map, factor-term, rest, make-product*, concat, and subexpressions-of-product. All are called by both versions; cons, map, rest, and concat are the clojure.core versions and let-print is just (defmacro let-print [bindings body] `(let ~bindings ~@(map (fn [x#] `(println ~x#)) (take-nth 2 bindings)) ~...@body)) which I find handy sometimes when debugging, because I can just slap a -print at the end of a let to add some debugging dumps to the console output. (I also have eval-print and intend to add loop-print to dump loop variables every time around the loop.) I had been debugging the above function, which work led to the change that triggered the new problem. Interestingly, changing the let-print back to let alters the error message, specifically, the cryptic number after unknown constant tag, but has no other effect. The functions factor-term, make-product*, and subexpressions-of-product have all been working for ages, and have not changed recently. None type-hints anything, return value or otherwise. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Mysterious ClassFormatError after simple code change.
(defn- subexpressions-of-sum** [[n p] terms] (let-print [sum (cons '+ (map #(factor-term % n p) terms)) prod (rest (make-product* n p))] (cons sum (map #(cons '* (cons sum (rest %))) (concat prod (subexpressions-of-product prod)) I look at the above, and something stupid just entered into my thought. What happened when this situation occurs (cons sum ())? I hope that's not possible with your code. Regards, Emeka On Sun, Jul 5, 2009 at 6:01 AM, John Harrop jharrop...@gmail.com wrote: I had this: (defn- subexpressions-of-sum** [[n p] terms] (let-print [sum (cons '+ (map #(factor-term % n p) terms)) prod (rest (make-product* n p))] (concat [sum] (subexpressions-of-product (cons sum prod) in a source file with other definitions. Load-file worked. I then changed it to this: (defn- subexpressions-of-sum** [[n p] terms] (let-print [sum (cons '+ (map #(factor-term % n p) terms)) prod (rest (make-product* n p))] (cons sum (map #(cons '* (cons sum (rest %))) (concat prod (subexpressions-of-product prod)) and got: #CompilerException java.lang.ClassFormatError: Unknown constant tag 32 in class file com/mycompany/myfile$eval__14598 (NO_SOURCE_FILE:0) when I tried to do a load-file. That function definition was the ONLY thing I changed, but I'm at a loss to find any kind of error in it. Delimiters balance, all of the referenced functions exist, basically there's nothing wrong. The full exception trace, which required evaluating (.printStackTrace (.getCause *e)) at the repl, is: java.lang.ClassFormatError: Unknown constant tag 32 in class file com/mycompany/myfile$eval__14598 at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:621) at java.lang.ClassLoader.defineClass(ClassLoader.java:466) at clojure.lang.DynamicClassLoader.defineClass(DynamicClassLoader.java:42) at clojure.lang.Compiler$FnExpr.getCompiledClass(Compiler.java:3417) at clojure.lang.Compiler$FnExpr.eval(Compiler.java:3428) at clojure.lang.Compiler.eval(Compiler.java:4531) at clojure.core$eval__3990.invoke(core.clj:1728) at clojure.main$repl__5813$read_eval_print__5825.invoke(main.clj:176) at clojure.main$repl__5813.doInvoke(main.clj:193) at clojure.lang.RestFn.invoke(RestFn.java:548) at org.enclojure.repl.main$create_clojure_repl__53$repl_thread_fn__55.invoke(main.clj:96) at clojure.lang.AFn.run(AFn.java:37) at java.lang.Thread.run(Thread.java:619) It does not point to any line of my source file. Perhaps the rewritten version of the function provokes a compiler bug? If there is a known bug that would cause this, let me know of the known workaround. If there is an error in the second version of my function, let me know. (It has intentionally different semantics from the first version, so that's not an error in and of itself.) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Mysterious ClassFormatError after simple code change.
On Mon, Jul 6, 2009 at 3:58 PM, Emeka emekami...@gmail.com wrote: (defn- subexpressions-of-sum** [[n p] terms] (let-print [sum (cons '+ (map #(factor-term % n p) terms)) prod (rest (make-product* n p))] (cons sum (map #(cons '* (cons sum (rest %))) (concat prod (subexpressions-of-product prod)) I look at the above, and something stupid just entered into my thought. What happened when this situation occurs (cons sum ())? I hope that's not possible with your code. It shouldn't be. Even if it is, there'd just be some (* sum) in the results, which is inefficient but not illegitimate. I don't see any way that this could be the cause of the ClassFormatError, in particular. Frankly, I am starting to suspect a bug in clojure. I'm using the stable 1.0, not the bleeding edge stuff. Whether the occurrence of the ClassFormatError itself is down to a subtle error in my code (so subtle no-one can spot it) or an error in clojure, its failure to point to a line in my source file as the erroneous one is in and of itself a bug, I'd argue. Since it's not apparently a simple bug in my function above, but something about a combination of that version of that function and some other part of my code, I can't think of a way to track the cause down short of the very tedious method of commenting out functions or replacing them with dummy functions (if they're called by the above function) and seeing which one has the property that commenting it out or dummy-izing it makes the error go away. That will probably take all afternoon, unfortunately; there are dozens of functions and over 1600 lines of code in that source file. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Mysterious ClassFormatError after simple code change.
Since it's not apparently a simple bug in my function above, but something about a combination of that version of that function and some other part of my code, I can't think of a way to track the cause down short of the very tedious method of commenting out functions or replacing them with dummy functions (if they're called by the above function) and seeing which one has the property that commenting it out or dummy-izing it makes the error go away. That will probably take all afternoon, unfortunately; there are dozens of functions and over 1600 lines of code in that source file. Have you tried simpler things like splitting the offending function into a separate namespace, or seeing what happens with (or without) AOT compilation? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Mysterious ClassFormatError after simple code change.
On Jul 5, 2009, at 2:01 AM, John Harrop wrote: and got: #CompilerException java.lang.ClassFormatError: Unknown constant tag 32 in class file com/mycompany/myfile$eval__14598 (NO_SOURCE_FILE:0) Are there large literals elsewhere in the same namespace? Here's some info from a previous report of this error: http://groups.google.com/group/clojure/browse_frm/thread/e556434a382de814/f8183c88db8fa257?lnk=gstq=oct+18+2008#f8183c88db8fa257 --Steve smime.p7s Description: S/MIME cryptographic signature
Re: Mysterious ClassFormatError after simple code change.
I'm not doing any funny things with load-string. The largest literal at this time is [2 3 5 7 11 13 17 19 23 29 31 37 41 43 47 53 59 61 67 71 73 79 83 89 97 101 103 107 109 113 127 131 137 139 149 151 157 163 167 173 179 181 191 193 197 199 211 223 227 229 233 239 241 251 257 263 269 271 277 281 283 293 307 311 313 317 331 337 347 349 353 359 367 373 379 383 389 397 401 409 419 421 431 433 439 443 449 457 461 463 467 479 487 491 499 503 509 521 523 541 547 557 563 569 571 577 587 593 599 601 607 613 617 619 631 641 643 647 653 659 661 673 677 683 691 701 709 719 727 733 739 743 751 757 761 769 773 787 797 809 811 821 823 827 829 839 853 857 859 863 877 881 883 887 907 911 919 929 937 941 947 953 967 971 977 983 991 997] and has been in there a while without causing problems. The function definition in my original post is the ONLY change between working and not-working. With the first version, the file loads consistently. With the second version, it fails consistently. The error is definitely in that function and not elsewhere, or else it is in the implementation of load-file, the compiler, or even Java or the JVM. The JVM is 1.6.0_something, I forget what. It's pretty current. The literal generated in the post you linked to took a list of four items and doubled it up 13 times, for a grand total of 2^15 = 32768 items. There's nothing that large in that function, nor anything in the second version of it that would expand (say, via a macro) into something that large from a number being changed. This is frankly quite baffling. The changes to the function are innocent from a large-literal or pretty much any other perspective. On 7/5/09, Stephen C. Gilardi squee...@mac.com wrote: On Jul 5, 2009, at 2:01 AM, John Harrop wrote: and got: #CompilerException java.lang.ClassFormatError: Unknown constant tag 32 in class file com/mycompany/myfile$eval__14598 (NO_SOURCE_FILE:0) Are there large literals elsewhere in the same namespace? Here's some info from a previous report of this error: http://groups.google.com/group/clojure/browse_frm/thread/e556434a382de814/f8183c88db8fa257?lnk=gstq=oct+18+2008#f8183c88db8fa257 --Steve --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
