Question: How do you organize large clojure projects?
I've been happy using a monorepo with boot, and a build.boot with many building blocks that can be mixed and matched for many deployables (uberjars in my case). In my build.boot I define my internal blocks, but also defs for external deps like eg postgres and grpc. That way I have one place for these groups of dependencies to manage, and can merge them into many of my own building blocks in a flexible and consistent way. There are no checkouts, multiple outputs and consistent usage of dependencies. If I need different release management/deployment process for different deployables of my own I tend to use different release branches by deployable that my ci/CD setup triggers from. Hope that helps! -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [ANN] 1.10.0-beta5
I have run versions 0.2.6 and 0.3.3 of the Eastwood Clojure lint tool on 84
projects in its "crucible" set of projects that we use for testing
Eastwood. You can find the list at [2].
I tried comparing the output for these pairs of version combinations of
(Eastwood version, Clojure version, JDK version):
(0.2.6, 1.9.0, Oracle 1.8.0_192) compared with (0.2.6, 1.10.0-beta5, Oracle
1.8.0_192)
(0.2.6, 1.9.0, OpenJDK 11.0.1) compared with (0.2.6, 1.10.0-beta5, OpenJDK
11.0.1)
(0.3.3, 1.9.0, Oracle 1.8.0_192) compared with (0.3.3, 1.10.0-beta5, Oracle
1.8.0_192)
(0.3.3, 1.9.0, OpenJDK 11.0.1) compared with (0.3.3, 1.10.0-beta5, OpenJDK
11.0.1)
All test runs were run on an Ubuntu Linux 18.04.1 system in a VM on my
Intel x86_64 based Mac laptop.
The only differences I found were:
+ differing line numbers of Clojure source code that appeared in stack
traces, which is expected in a new Clojure release
+ fewer deprecation warnings when linting Clojure namespace
clojure.java.browse-ui, because of intentional changes made for Clojure
1.10.0 to eliminate the use of deprecated APIs [3]
+ a couple of reflection warnings present in namespace clojure.main are
gone. There was no change in the Java interop calls, so perhaps this is
because of changes made in how the Clojure compiler resolves Java method
calls.
I didn't see anything more than 3 to 4 percent difference in run times
across all of the version combinations above.
Andy
[1] https://github.com/jonase/eastwood
[2]
https://github.com/jonase/eastwood/tree/master/crucible/project-clj-files
[3]
https://github.com/clojure/clojure/commit/fb916808669ef65dce5dfe58e23d4a902253ca55
On Tue, Nov 6, 2018 at 5:42 AM Alex Miller wrote:
> 1.10.0-beta5 is now available.
>
> You can try it with clj using:
>
> clj -Sdeps '{:deps {org.clojure/clojure {:mvn/version
> "1.10.0-beta5"}}}'
>
> Changes in 1.10.0-beta5:
>
>
>- In addition to prior methods of extension, values can now extend
>protocols by adding metadata where keys are fully-qualified symbols naming
>protocol functions and values are function implementations. Protocol
>implementations are checked first for direct definitions (defrecord,
>deftype, reify), then metadata definitions, then external extensions
>(extend, extend-type, extend-protocol). datafy has been updated to use this
>mechanism.
>- `symbol` can now be passed vars or keywords to obtain the
>corresponding symbol
>- CLJ-2420 error
>reporting enhancements - more refined phase reporting, new
>clojure.main/ex-triage split out of clojure.main/ex-str, execution errors
>now report the top *user* line in the stack trace omitting frames from
>core, enhancements to providing file and line via meta on a form
>- CLJ-2425 add java 11
>javadoc url
>- CLJ-2424 fix test bug
>from CLJ-2417
>
> You can read the full 1.10 changelog here:
> https://github.com/clojure/clojure/blob/master/changes.md
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Question: How do you organize large clojure projects?
This is somewhat of a retrospective -- so please bear with me. I've had the
privilege of working on a clojure project for a couple of years now, and have
accumulated some 15-20k lines of clojure code. I'm taking a little time to
look back over what has worked for me and what hasn't in terms of code/project
organization -- and *I'd love to know what has worked for other people (or
hasn't)* for similarly large projects.
I knew my project was going to grow to at least as much code as it has now at
the start, and my domain problem was fairly well-defined. From the very
beginning I organized my code into many (15-20) different clojure 'projects'
using lein. Rather than organizing code into these projects *by function
area*, I found myself organizing code into projects by their *dependencies*.
So any code that used libraries X,Y,Z went into a project that declared those
dependencies -- even if a function made sense in a different namespace by name
-- if it needed deps that I already had in another project, I moved the
function to that project. For example, in the Java GIS world, if you do
anything with swing components, you can easily pull in 100s of MBs of
dependencies. My project involves GIS work both server-side rest-apis, but
it's also nice to pull up a quick swing component showing data on a map for
debugging etc. I don't have these things in the same project even though there
is some library overlap because the GUI deps are just too many.
I think for me on my project, the only reason to separate anything into
'projects' is for reuse based on dependencies -- i.e., to use a function/set of
functions I've written again, what's the minimal amount of deps to pull in.
It's worked well for me in that sense -- I'm able to create a new 'main'
project, include libraries that I want from my project, and not pull in 2,000
dependencies unless I need it. The dependencies I'm mostly concerned with here
are Java deps and not clojure deps. I've found a relatively small core set of
clojure deps I almost always want available to me (specter, timbre, core.async)
-- though even still I have a utility library project where I have a hard rule
of zero dependencies (for basic macros like (ignore-exception body-forms)).
I've used lein's checkouts and managed dependencies fairly successfully though
I still forget to lein install everything before I lein uberjar my final
delivery artifacts and end up debugging old code before realizing what happened
(and my way of installing everything is a bash for loop :-)).
I've found this approach somewhat tedious and have been wondering if there's a
better way -- and am very curious what others do.
What I've been playing around with lately is a different concept for my own
code organization:
- What if all my clojure code could go in one place, or one project? (Even if
it ended up being 20k+ lines of code)
- What if namespaces contained their required dependencies in their metadata?
- What if upon namespace creation, a namespace's dependencies were
automatically added to the classpath?
- What if functions declared in a namespace could also declare additional
dependencies? These would be added to the classpath upon first invocation of
the function. This is great for my seldom used functions that need many
dependencies -- code could live in the namespace that is matches its function
instead of squirreled away in a project just to match its deps.
I have written a basic library that does these things, and am currently trying
it out on a small scale. Concerns I've already been trying to address:
- Dynamically adding things to the classpath is generally considered a /bad/
thing to do.
- My code reads all pom.xmls on the classpath to determine what libraries
are already on the classpath -- and does not re-add libraries that are already
on the classpath. I think this alone takes care of a lot of issues with
dynamically adding deps to the classpath (multiple versions of libraries on the
classpath).
- Production deployments should not need to calculate classpaths dynamically
- This technique should be able to be used whether the classpath is
precomputed (i.e., for production), or dynamically during development.
Example namespace loaded deps:
(ns test
{:dependencies 'com.taoensso/timbre "4.10.0"}
(:require [taoensso.timbre :as timbre
:refer [info debug error warn spy]]))
-or-
(ns test
{:deps '{[[com.taoensso/timbre {:mvn/version "4.10.0"]]}}}
(:require [taoensso.timbre :as timbre
:refer [info debug error warn spy]]))
Now these namespace deps would be loaded dynamically by aliasing the ns macro
for development with one that loads deps dynamically. In production, the
metadata is simply attached to the namespace per normal use of the ns form, no
deps added dynamically.
(defn-deps test-fn-deps
"Test Function with optional deps"
{:dependencies '[[diffit "1.0.0"]
Re: [ANN] High Performance Numerics Compiler
Wow, awesome! On Monday, 12 November 2018 07:46:14 UTC-8, Chris Nuernberger wrote: > > Clojurians, we are happy to put forth our work bringing the TVM compiler > and infrastructure to clojure. It has backends for ARM, Intel, OpenCV, > Cuda, OpenGL, Vulkan, ROCm, and more. > > This system is currently getting state of the art performance several deep > learning kernels and I demonstrate it getting great performance (usually > beating by a sound margin but not always) as compared to hand optimized > code in OpenCV. > > Utilizing years of experience writing doing HPC and GPGPU programming, we > researched the best way to integrate clojure at a low level with one of the > major NN toolkits. We didn't want to sit on top of them alone because we > believe the underlying technology is useful in a broader context and we > believe that Clojure should be a first class citizen in these ecosystems. > > So we spent roughly the last 6 months working on and off in infrastructure > and support systems and you saw an example of the result earlier this year > with the opencv bindings. > > So, with no further ado: > > 1. Light description of the overall compiler architecture and theory: > http://techascent.com/blog/high-performance-compilers.html > > 2. Example of using the compiler: > http://techascent.com/blog/tvm-for-the-win.html > > 3. Github project (with lots of links to more material): > https://github.com/tech-ascent/tvm-clj > > Happy Monday from snowy Boulder, Colorado :-) > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Request to review my CLJS using core.async
Do you really need core.async for this? Seems like you can just normally call the service to get the token once at the beginning and then just go in a normal loop of calling for the next message -> handling message -> repeat. I'm not sure there's any reason to use core.async in ClojureScript other then if you were doing a bunch of concurrent async IO. On Monday, 12 November 2018 05:46:17 UTC-8, Kashyap CK wrote: > > Hi all, > > I am attempting to use core.async to poll a service - > https://gist.github.com/ckkashyap/c8423dcfc3a3f28b67e18ae76cc13f53 > Broadly, I need to hit the service endpoint with a secret to get a token > and subsequently use the token to poll for messages. > > I'd appreciate any feedback on the approach - particularly around > 1. Error handling > 2. Use of channels > > Is there some project out there I could take a look at to learn the > right/good way to do this? > > Regards, > Kashyap > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
