Re: Clojure deployment questions w.r.t. jars, clojure source files, compiled class files
Excellent answers and I usually find a use for extra wrenches. I'm still confused about when anybody actually calls the (compile) function, any more tips here? Or is it being done for me by leiningen? On Wed, Jan 8, 2014 at 1:54 AM, Softaddicts lprefonta...@softaddicts.cawrote: To complement my previous post and in relation to this one, we use (if-not *compile-file* ...) to wrap expressions that cannot be AOT compiied (like loading configuration or connecting to external resources which you do not want to do at compile time but at run time only:) This confused a lot of people in the past trying to use AOT. For many the compilation phase and code execution are perceived as a single step. When working in the REPL you do not need to be aware of this. With AOT it's important to be aware of the difference when writing code in dev. Luc P. On Tue, Jan 7, 2014 at 4:26 PM, Dave Tenny dave.te...@gmail.com wrote: 1) When and under what circumstances projects are compiled in source versus .class form? Most Clojure projects ship in source form (and are therefore compiled to bytecode on demand as they are loaded). 2) Why there is no project.clj in the org.clojure jar file? It's built with Maven. As are most of the Clojure contrib libraries too, although some are now starting to sport project.clj files to make local development easier. In the last round of development, I added Leiningen support to clojure.java.jdbc to make it easier to jack in with Emacs and test the code. It still uses Maven for primary testing (on build.clojure.org) and packaging - and Clojure plus its contrib libraries are hosted on Maven Central (where they are retrieved primarily by Leiningen into other Clojure projects). 3) When the clojure 'compile' function comes into play in your typical clojure project deployments? (vs. :aot targets or other leiningen deployment techniques). At World Singles, we AOT compile very little of our code. We only AOT namespaces that generate Java-compatible classes for situations where we must be natively callable from Java (e.g., we have a log4j appender written in Clojure). Within that AOT-compiled code, we require namespaces and resolve symbols dynamically (at runtime) to bind the Java-called code to the rest of our code base. Part of the reason is for the flexibility that source deployments provide: the ability to REPL into a live, running process and reload code from updated source files without needing to stop the world, for example. If you're relatively new to Clojure, I'd recommend completely ignoring the whole compilation thing unless you specifically need to generate natively callable code for Java to Clojure interop. In case anyone is interested, our pattern for bridging from the AOT-compiled namespace to the rest of the code base tends to look like this: (def the-symbol (delay (do (require 'the.namespace) (resolve (symbol the.namespace/the-symbol) and then: ... (@the-symbol arg1 arg2) ... Our AOT-compiled layer is deliberately minimal and serves only to provide the Java-callable API so the rest of our code can be developed and tested in our normal interactive, incremental way. -- Sean A Corfield -- (904) 302-SEAN An Architect's View -- http://corfield.org/ World Singles, LLC. -- http://worldsingles.com/ Perfection is the enemy of the good. -- Gustave Flaubert, French realist novelist (1821-1880) -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- Softaddictslprefonta...@softaddicts.ca sent by ibisMail from my ipad! -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to a topic in the Google Groups Clojure group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojure/sU0r00i7Ls0/unsubscribe. To unsubscribe from
Re: Clojure deployment questions w.r.t. jars, clojure source files, compiled class files
Look at the compile fn at the bottom: https://github.com/technomancy/leiningen/blob/master/src/leiningen/compile.clj you will find your answer :) Luc P. Excellent answers and I usually find a use for extra wrenches. I'm still confused about when anybody actually calls the (compile) function, any more tips here? Or is it being done for me by leiningen? On Wed, Jan 8, 2014 at 1:54 AM, Softaddicts lprefonta...@softaddicts.cawrote: To complement my previous post and in relation to this one, we use (if-not *compile-file* ...) to wrap expressions that cannot be AOT compiied (like loading configuration or connecting to external resources which you do not want to do at compile time but at run time only:) This confused a lot of people in the past trying to use AOT. For many the compilation phase and code execution are perceived as a single step. When working in the REPL you do not need to be aware of this. With AOT it's important to be aware of the difference when writing code in dev. Luc P. On Tue, Jan 7, 2014 at 4:26 PM, Dave Tenny dave.te...@gmail.com wrote: 1) When and under what circumstances projects are compiled in source versus .class form? Most Clojure projects ship in source form (and are therefore compiled to bytecode on demand as they are loaded). 2) Why there is no project.clj in the org.clojure jar file? It's built with Maven. As are most of the Clojure contrib libraries too, although some are now starting to sport project.clj files to make local development easier. In the last round of development, I added Leiningen support to clojure.java.jdbc to make it easier to jack in with Emacs and test the code. It still uses Maven for primary testing (on build.clojure.org) and packaging - and Clojure plus its contrib libraries are hosted on Maven Central (where they are retrieved primarily by Leiningen into other Clojure projects). 3) When the clojure 'compile' function comes into play in your typical clojure project deployments? (vs. :aot targets or other leiningen deployment techniques). At World Singles, we AOT compile very little of our code. We only AOT namespaces that generate Java-compatible classes for situations where we must be natively callable from Java (e.g., we have a log4j appender written in Clojure). Within that AOT-compiled code, we require namespaces and resolve symbols dynamically (at runtime) to bind the Java-called code to the rest of our code base. Part of the reason is for the flexibility that source deployments provide: the ability to REPL into a live, running process and reload code from updated source files without needing to stop the world, for example. If you're relatively new to Clojure, I'd recommend completely ignoring the whole compilation thing unless you specifically need to generate natively callable code for Java to Clojure interop. In case anyone is interested, our pattern for bridging from the AOT-compiled namespace to the rest of the code base tends to look like this: (def the-symbol (delay (do (require 'the.namespace) (resolve (symbol the.namespace/the-symbol) and then: ... (@the-symbol arg1 arg2) ... Our AOT-compiled layer is deliberately minimal and serves only to provide the Java-callable API so the rest of our code can be developed and tested in our normal interactive, incremental way. -- Sean A Corfield -- (904) 302-SEAN An Architect's View -- http://corfield.org/ World Singles, LLC. -- http://worldsingles.com/ Perfection is the enemy of the good. -- Gustave Flaubert, French realist novelist (1821-1880) -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- Softaddictslprefonta...@softaddicts.ca sent by ibisMail from my ipad! -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at
Clojure deployment questions w.r.t. jars, clojure source files, compiled class files
So if I sample some clojure jars in my local maven .m2 directory, most of the jar files have only clojure code, and a project.clj. If I look at the org.clojure project there, it has many java class files, what appear to be corresponding clojure source files, and *no* project.clj. With only a little experiencing on some single AOT compiled modules, I wondered 1) When and under what circumstances projects are compiled in source versus .class form? 2) Why there is no project.clj in the org.clojure jar file? 3) When the clojure 'compile' function comes into play in your typical clojure project deployments? (vs. :aot targets or other leiningen deployment techniques). Any pointers or enlightenment on the above questions appreciated. I read the clojure compilation page, but I haven't quite put together what that's telling me about *how* to comjpile a file and gen-class a class, versus why I see all these clojure .m2 jar files with no classes whatsoever. Thanks, Dave -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Clojure deployment questions w.r.t. jars, clojure source files, compiled class files
AOT in my experience is a little dicey and complicated. On Tue, Jan 7, 2014 at 7:26 PM, Dave Tenny dave.te...@gmail.com wrote: So if I sample some clojure jars in my local maven .m2 directory, most of the jar files have only clojure code, and a project.clj. If I look at the org.clojure project there, it has many java class files, what appear to be corresponding clojure source files, and *no* project.clj. With only a little experiencing on some single AOT compiled modules, I wondered 1) When and under what circumstances projects are compiled in source versus .class form? In general, you always want to distribute libs as maven artifacts with source included and no AOT compilation. Adding in compiled classes means also adding a dependency on whatever clojure version was used to compile those classes. The effects of this lead to often confounding errors. As an extra caveat, AOT currently does not make it easy to separate that project's class files from every other deps generated class files, so any project artifact will have extra classfiles in it. My company has grappled with separating these about five different ways in order to not distribute source, and there is a ticket to address it, but it looks like it'll persist. 2) Why there is no project.clj in the org.clojure jar file? Clojure itself isn't built with leiningen, hence no project.clj. 3) When the clojure 'compile' function comes into play in your typical clojure project deployments? (vs. :aot targets or other leiningen deployment techniques). Any pointers or enlightenment on the above questions appreciated. I read the clojure compilation page, but I haven't quite put together what that's telling me about *how* to comjpile a file and gen-class a class, versus why I see all these clojure .m2 jar files with no classes whatsoever. Thanks, Dave -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Clojure deployment questions w.r.t. jars, clojure source files, compiled class files
On Tue, Jan 7, 2014 at 4:26 PM, Dave Tenny dave.te...@gmail.com wrote: 1) When and under what circumstances projects are compiled in source versus .class form? Most Clojure projects ship in source form (and are therefore compiled to bytecode on demand as they are loaded). 2) Why there is no project.clj in the org.clojure jar file? It's built with Maven. As are most of the Clojure contrib libraries too, although some are now starting to sport project.clj files to make local development easier. In the last round of development, I added Leiningen support to clojure.java.jdbc to make it easier to jack in with Emacs and test the code. It still uses Maven for primary testing (on build.clojure.org) and packaging - and Clojure plus its contrib libraries are hosted on Maven Central (where they are retrieved primarily by Leiningen into other Clojure projects). 3) When the clojure 'compile' function comes into play in your typical clojure project deployments? (vs. :aot targets or other leiningen deployment techniques). At World Singles, we AOT compile very little of our code. We only AOT namespaces that generate Java-compatible classes for situations where we must be natively callable from Java (e.g., we have a log4j appender written in Clojure). Within that AOT-compiled code, we require namespaces and resolve symbols dynamically (at runtime) to bind the Java-called code to the rest of our code base. Part of the reason is for the flexibility that source deployments provide: the ability to REPL into a live, running process and reload code from updated source files without needing to stop the world, for example. If you're relatively new to Clojure, I'd recommend completely ignoring the whole compilation thing unless you specifically need to generate natively callable code for Java to Clojure interop. In case anyone is interested, our pattern for bridging from the AOT-compiled namespace to the rest of the code base tends to look like this: (def the-symbol (delay (do (require 'the.namespace) (resolve (symbol the.namespace/the-symbol) and then: ... (@the-symbol arg1 arg2) ... Our AOT-compiled layer is deliberately minimal and serves only to provide the Java-callable API so the rest of our code can be developed and tested in our normal interactive, incremental way. -- Sean A Corfield -- (904) 302-SEAN An Architect's View -- http://corfield.org/ World Singles, LLC. -- http://worldsingles.com/ Perfection is the enemy of the good. -- Gustave Flaubert, French realist novelist (1821-1880) -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Clojure deployment questions w.r.t. jars, clojure source files, compiled class files
Just to throw a wrench in this thread :))) a) We AOT our stuff to avoid shipping our sourde code to customer sites (:aot [regex...]), this requirement is the basis for what follows. b) We filter classes using lein to remove from our artifacts classes from other libs so only our AOT code is shipped in our jar files (:jar-exclusions [regex...]). This prevents mixing compiled classes of third party in our own jars which I agree can be pretty confusing (from which jar comes this compiled class ? Euuuh) c) We are strict on version control and obviously will not ship Clojure code compiled with different versions of Clojure, this is a recipe for disaster when there are feature changes in Clojure. We follow Orson's advice here, We will sell no wine before its time. We skipped some Clojure releases in the past and we moved to 1.5.1 by mid 2013. d) Most of the source code shipped comes from the open source libs we build upon. We can deploy site patches as source code and get them applied at app startup time when we need quick fixes in prod but they are not part of the build (otherwise they would not be called patches :) e) We also have Clojure data/code stored as part of the configuration and loaded on demand, it's been mixing fine with our AOT compiled aritfacts. e) We use an integrated nrepl console embedded in our apps and attach to it. We can investigate live in prod, apply patches live albeit with some limitations but as far as I can see AOT is not bitting us here. Restrictions have more to do with the app life cycle which we are improving along the way. f) We obvisously do integration tests using AOT compiled artifacts while dev testing is not to avoid time travels to stone age (edit compile restart hell) In our design most of the stuff is done in our supervisor process and each supervisor auto configures itself differently using plugins according to the local configuration and it has to run 24/7. I think we kept most of the dynamic benefits while meeting the goal in a). We have been using AOT for 4 years now. It's not at as hard as people think it is but you need a minimal understanding of the underlying machinery and if you come from a non Java background I agree 100%, it's not easy to find a path between the sand traps to get you to putt in the hole from dev to prod. End of the wrench throwing :))) Luc P. AOT in my experience is a little dicey and complicated. On Tue, Jan 7, 2014 at 7:26 PM, Dave Tenny dave.te...@gmail.com wrote: So if I sample some clojure jars in my local maven .m2 directory, most of the jar files have only clojure code, and a project.clj. If I look at the org.clojure project there, it has many java class files, what appear to be corresponding clojure source files, and *no* project.clj. With only a little experiencing on some single AOT compiled modules, I wondered 1) When and under what circumstances projects are compiled in source versus .class form? In general, you always want to distribute libs as maven artifacts with source included and no AOT compilation. Adding in compiled classes means also adding a dependency on whatever clojure version was used to compile those classes. The effects of this lead to often confounding errors. As an extra caveat, AOT currently does not make it easy to separate that project's class files from every other deps generated class files, so any project artifact will have extra classfiles in it. My company has grappled with separating these about five different ways in order to not distribute source, and there is a ticket to address it, but it looks like it'll persist. 2) Why there is no project.clj in the org.clojure jar file? Clojure itself isn't built with leiningen, hence no project.clj. 3) When the clojure 'compile' function comes into play in your typical clojure project deployments? (vs. :aot targets or other leiningen deployment techniques). Any pointers or enlightenment on the above questions appreciated. I read the clojure compilation page, but I haven't quite put together what that's telling me about *how* to comjpile a file and gen-class a class, versus why I see all these clojure .m2 jar files with no classes whatsoever. Thanks, Dave -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to
Re: Clojure deployment questions w.r.t. jars, clojure source files, compiled class files
To complement my previous post and in relation to this one, we use (if-not *compile-file* ...) to wrap expressions that cannot be AOT compiied (like loading configuration or connecting to external resources which you do not want to do at compile time but at run time only:) This confused a lot of people in the past trying to use AOT. For many the compilation phase and code execution are perceived as a single step. When working in the REPL you do not need to be aware of this. With AOT it's important to be aware of the difference when writing code in dev. Luc P. On Tue, Jan 7, 2014 at 4:26 PM, Dave Tenny dave.te...@gmail.com wrote: 1) When and under what circumstances projects are compiled in source versus .class form? Most Clojure projects ship in source form (and are therefore compiled to bytecode on demand as they are loaded). 2) Why there is no project.clj in the org.clojure jar file? It's built with Maven. As are most of the Clojure contrib libraries too, although some are now starting to sport project.clj files to make local development easier. In the last round of development, I added Leiningen support to clojure.java.jdbc to make it easier to jack in with Emacs and test the code. It still uses Maven for primary testing (on build.clojure.org) and packaging - and Clojure plus its contrib libraries are hosted on Maven Central (where they are retrieved primarily by Leiningen into other Clojure projects). 3) When the clojure 'compile' function comes into play in your typical clojure project deployments? (vs. :aot targets or other leiningen deployment techniques). At World Singles, we AOT compile very little of our code. We only AOT namespaces that generate Java-compatible classes for situations where we must be natively callable from Java (e.g., we have a log4j appender written in Clojure). Within that AOT-compiled code, we require namespaces and resolve symbols dynamically (at runtime) to bind the Java-called code to the rest of our code base. Part of the reason is for the flexibility that source deployments provide: the ability to REPL into a live, running process and reload code from updated source files without needing to stop the world, for example. If you're relatively new to Clojure, I'd recommend completely ignoring the whole compilation thing unless you specifically need to generate natively callable code for Java to Clojure interop. In case anyone is interested, our pattern for bridging from the AOT-compiled namespace to the rest of the code base tends to look like this: (def the-symbol (delay (do (require 'the.namespace) (resolve (symbol the.namespace/the-symbol) and then: ... (@the-symbol arg1 arg2) ... Our AOT-compiled layer is deliberately minimal and serves only to provide the Java-callable API so the rest of our code can be developed and tested in our normal interactive, incremental way. -- Sean A Corfield -- (904) 302-SEAN An Architect's View -- http://corfield.org/ World Singles, LLC. -- http://worldsingles.com/ Perfection is the enemy of the good. -- Gustave Flaubert, French realist novelist (1821-1880) -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- Softaddictslprefonta...@softaddicts.ca sent by ibisMail from my ipad! -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
