Re: Utility libraries and dependency hygiene
puzzler wrote: I decided it would be a bad idea to include rhizome directly in instaparse's dependencies. Nevertheless, it made sense to enable the visualize function *provided* rhizome was already in the user's dependencies. I prefer to avoid these kinds of load-time tricks, as they break easily when anything else is involved in loading code, e.g. an IDE or app server. Fortunately, Rhizome is driven by data. If you can provide functions that return the data structures that Rhizome expects, then Instaparse users can easily invoke Rhizome themselves to get the visualizations. If this is not possible, I would suggest providing the visualization tools as a separate library release which depends on both Instaparse and Rhizome. This might make sense anyway, if they are intended only as a development-time tool. -S -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Utility libraries and dependency hygiene
This thread came up right around the time I was considering adding a dependency on rhizome to instaparse to make it easy to visualize the parse trees. Based on the discussion here, I decided it would be a bad idea to include rhizome directly in instaparse's dependencies. Nevertheless, it made sense to enable the visualize function *provided* rhizome was already in the user's dependencies. More specifically, it will use whichever version of rhizome they choose to put in their dependencies, thus a version conflict is not an issue. I found a technique to achieve this effect, and wanted to report on it here. First, towards the top of the file, after the namespace declaration, I included the following: (try (require '[rhizome.viz :as r]) (catch Exception e (require '[instaparse.viz-not-found :as r]))) This sets things up so r/ refers either to the rhizome.viz namespace, or if it is not installed, my dummy instaparse.viz-not-found namespace. The instaparse.viz-not-found contains stubs for all the functions from rhizome that I use; the stubs simply throw a friendly error message saying to add rhizome to the project's dependencies. -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Utility libraries and dependency hygiene
Using the decomposition I made earlier, I would refine my analysis a bit further and say: - the original problem lies mostly within the programming model: it's about resolving dependencies when reusing and composing deployable units, and how to avoid complexity and conflicts. - part of the problem also lies within the runtime model: as of yet Clojure core is not expressive enough to support the use of two versions of the same library at runtime. - the proposed solution belongs to the design paradigm: some form of heuristics on the number of dependencies, copying code, and the sense of having some sort of layer of utility libraries. So in fact the problem and proposed solution were already spread across all these domains to start with. Let's continue with more facts while summarizing what's already been discussed: - there is a difference between managing and resolving dependencies during development, and doing the same for preparing deployable units; lein uberjar and google closure dependency management and compilation/minification are great examples. - talking about leingingen and automating code copying/renaming is searching for a solution within the programming model. - talking about classloaders and using different versions at runtime is searching for a solution within the runtime model. - the homoiconic and dynamic nature of Clojure means that programming and runtime models are intimately related: classes are generated and loaded at runtime, and dependencies can be resolved at runtime (e.g. pomegranate). - not everyone and every host language use leiningen, so the solution cannot be about leiningen only, but the latter can certainly facilitate the process of applying a solution. If we were to organize possible solutions at each level: 1. design paradigm: what the original post proposed, i.e. heuristic on dependencies, manually copying code, and the sense of having some sort of layer of utility libraries. 2. programming model: automate the copying and/or re-namespacing of code during the creation of deployable units, e.g. some form of :copy/:copy-as directives within the ns form; and/or we could bring version specification into the ns form, although ns has no knowledge of deployment units, so some form of mapping needs to be done and/or we need some cross-platform reification of deployment units (e.g. some generalization of jars, wars, ears, osgi bundles, modules in jigsaw, minified scripts, etc.). 3. runtime model: depends largely on the host language: classloaders in java, jigsaw later, closure minification in clojurescript, but that's a programming model solution. I'm sure there's more. And a combination of all these solutions certainly makes sense. We can start at level 1 with the original proposed solution, but the pressure of moving into solutions at level 2 is already felt in this discussion. Moving into level 3 solutions later on should be the natural continuation, and that should be present in our minds while working on other levels. -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Utility libraries and dependency hygiene
At the risk of making a slight strawman here, I agree this issue has the potential to drag on Clojure adoption - but in the opposite way to what you describe. At the root of it, is the Clojure ecosystem a dependencies are bad system, or a dependencies are good system? The former encourages roll-your-own, but reduces code sharing and re-use The latter promotes sharing and building upon the work of others, but requires more advanced dependency management tooling As great as the language is, having the general advice be don't have dependencies in your libraries seems like a step backwards to me. I would always rather spend an hour finding a documented, tested existing library, than spend 20 minutes creating my own duplicate of that work. I hope this doesn't come over as too accusatory, if the cost of dependencies truly is higher than the cost of duplicating work/code then perhaps we need to try and make the former easier. Glen On Tuesday, 14 May 2013 13:19:15 UTC+1, Dave Kincaid wrote: This thread seems to have gotten way off track and I think that Stuart made a very important point in the original post that's getting lost. It would help all of us out if library authors stopped making their libraries dependent on 10+ other libraries. This issue does have the potential to really drag on Clojure adoption. Just try to maintain your own local repository for your projects and you'll see what I mean the first time you need to add a Clojure dependency. On Tuesday, May 14, 2013 4:22:59 AM UTC-5, Phillip Lord wrote: Stuart Sierra ma...@stuartsierra.com writes: On Tue, May 14, 2013 at 3:25 AM, Phil Hagelberg ph...@hagelb.org wrote: It's really not difficult to do if you limit yourself to Clojure since Clojure namespaces are first-class and easy to manipulate at run-time. We implemneted a prototype of this in under two hours at a Seajure meeting a while back: https://github.com/technomancy/metaverse However, it's significantly more difficult to do for arbitrary Java bytecode. That's cool, and it will work for the simple case of libraries A and B depending on different versions of C. But it still breaks down in more complex cases: e.g. if I want to share data between A and B using a protocol or type defined in C, and there are 2 incompatible versions of C. Even ClassLoaders can't help you there - I'm not aware of any solution. Automatically, no, but the solution would be to use something akin to an adaptor. The two versions of C would be manipulated to be in different namespaces; now you just have two libraries, so the task of plumbing them together remains the same. To be honest, though, this is unlikely; after all, if you are using A and B, and they are using C *as a utility*, my feeling is that C shouldn't really be in their public interface. If C *is* in their public interface, then again, you need adaptors. Or you can fork A and/or B, fix them to use the same version! Phil -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Utility libraries and dependency hygiene
As long as we remember that not everyone is using Leiningen and/or Maven. There are other build and dependency systems out there, so any change to dependency management tooling will have to cover all of them. There are already popular libraries that don't work outside of Leiningen, but that's a topic for another thread... On Wednesday, May 15, 2013 3:17:59 AM UTC-5, Glen Mailer wrote: At the risk of making a slight strawman here, I agree this issue has the potential to drag on Clojure adoption - but in the opposite way to what you describe. At the root of it, is the Clojure ecosystem a dependencies are bad system, or a dependencies are good system? The former encourages roll-your-own, but reduces code sharing and re-use The latter promotes sharing and building upon the work of others, but requires more advanced dependency management tooling As great as the language is, having the general advice be don't have dependencies in your libraries seems like a step backwards to me. I would always rather spend an hour finding a documented, tested existing library, than spend 20 minutes creating my own duplicate of that work. I hope this doesn't come over as too accusatory, if the cost of dependencies truly is higher than the cost of duplicating work/code then perhaps we need to try and make the former easier. Glen On Tuesday, 14 May 2013 13:19:15 UTC+1, Dave Kincaid wrote: This thread seems to have gotten way off track and I think that Stuart made a very important point in the original post that's getting lost. It would help all of us out if library authors stopped making their libraries dependent on 10+ other libraries. This issue does have the potential to really drag on Clojure adoption. Just try to maintain your own local repository for your projects and you'll see what I mean the first time you need to add a Clojure dependency. On Tuesday, May 14, 2013 4:22:59 AM UTC-5, Phillip Lord wrote: Stuart Sierra ma...@stuartsierra.com writes: On Tue, May 14, 2013 at 3:25 AM, Phil Hagelberg ph...@hagelb.org wrote: It's really not difficult to do if you limit yourself to Clojure since Clojure namespaces are first-class and easy to manipulate at run-time. We implemneted a prototype of this in under two hours at a Seajure meeting a while back: https://github.com/technomancy/metaverse However, it's significantly more difficult to do for arbitrary Java bytecode. That's cool, and it will work for the simple case of libraries A and B depending on different versions of C. But it still breaks down in more complex cases: e.g. if I want to share data between A and B using a protocol or type defined in C, and there are 2 incompatible versions of C. Even ClassLoaders can't help you there - I'm not aware of any solution. Automatically, no, but the solution would be to use something akin to an adaptor. The two versions of C would be manipulated to be in different namespaces; now you just have two libraries, so the task of plumbing them together remains the same. To be honest, though, this is unlikely; after all, if you are using A and B, and they are using C *as a utility*, my feeling is that C shouldn't really be in their public interface. If C *is* in their public interface, then again, you need adaptors. Or you can fork A and/or B, fix them to use the same version! Phil -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Utility libraries and dependency hygiene
2013/5/15 Dave Kincaid kincaid.d...@gmail.com: As long as we remember that not everyone is using Leiningen and/or Maven. There are other build and dependency systems out there, so any change to dependency management tooling will have to cover all of them. There are already popular libraries that don't work outside of Leiningen, but that's a topic for another thread... Hello, What are the popular Clojure (not ClojureScript) libraries that don't work outside a maven-style-backed-central-public-repository? On Wednesday, May 15, 2013 3:17:59 AM UTC-5, Glen Mailer wrote: At the risk of making a slight strawman here, I agree this issue has the potential to drag on Clojure adoption - but in the opposite way to what you describe. At the root of it, is the Clojure ecosystem a dependencies are bad system, or a dependencies are good system? The former encourages roll-your-own, but reduces code sharing and re-use The latter promotes sharing and building upon the work of others, but requires more advanced dependency management tooling As great as the language is, having the general advice be don't have dependencies in your libraries seems like a step backwards to me. I would always rather spend an hour finding a documented, tested existing library, than spend 20 minutes creating my own duplicate of that work. I hope this doesn't come over as too accusatory, if the cost of dependencies truly is higher than the cost of duplicating work/code then perhaps we need to try and make the former easier. Glen On Tuesday, 14 May 2013 13:19:15 UTC+1, Dave Kincaid wrote: This thread seems to have gotten way off track and I think that Stuart made a very important point in the original post that's getting lost. It would help all of us out if library authors stopped making their libraries dependent on 10+ other libraries. This issue does have the potential to really drag on Clojure adoption. Just try to maintain your own local repository for your projects and you'll see what I mean the first time you need to add a Clojure dependency. On Tuesday, May 14, 2013 4:22:59 AM UTC-5, Phillip Lord wrote: Stuart Sierra ma...@stuartsierra.com writes: On Tue, May 14, 2013 at 3:25 AM, Phil Hagelberg ph...@hagelb.org wrote: It's really not difficult to do if you limit yourself to Clojure since Clojure namespaces are first-class and easy to manipulate at run-time. We implemneted a prototype of this in under two hours at a Seajure meeting a while back: https://github.com/technomancy/metaverse However, it's significantly more difficult to do for arbitrary Java bytecode. That's cool, and it will work for the simple case of libraries A and B depending on different versions of C. But it still breaks down in more complex cases: e.g. if I want to share data between A and B using a protocol or type defined in C, and there are 2 incompatible versions of C. Even ClassLoaders can't help you there - I'm not aware of any solution. Automatically, no, but the solution would be to use something akin to an adaptor. The two versions of C would be manipulated to be in different namespaces; now you just have two libraries, so the task of plumbing them together remains the same. To be honest, though, this is unlikely; after all, if you are using A and B, and they are using C *as a utility*, my feeling is that C shouldn't really be in their public interface. If C *is* in their public interface, then again, you need adaptors. Or you can fork A and/or B, fix them to use the same version! Phil -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit
Re: Utility libraries and dependency hygiene
That seems orthogonal to the problem the loader is trying to solve. -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Utility libraries and dependency hygiene
One that I encountered last week was Midje. In trying to work through all of its dependencies I discovered that Pomegranate is also dependent on Maven, but I think I was able to work around that by adding 4 or 5 Maven libraries into our repository as dependencies (not ideal, but it looked like it would work). But I still wasn't able to get Midje working due some assumptions about Leiningen. On Wednesday, May 15, 2013 8:13:19 AM UTC-5, Laurent PETIT wrote: 2013/5/15 Dave Kincaid kincai...@gmail.com javascript:: As long as we remember that not everyone is using Leiningen and/or Maven. There are other build and dependency systems out there, so any change to dependency management tooling will have to cover all of them. There are already popular libraries that don't work outside of Leiningen, but that's a topic for another thread... Hello, What are the popular Clojure (not ClojureScript) libraries that don't work outside a maven-style-backed-central-public-repository? On Wednesday, May 15, 2013 3:17:59 AM UTC-5, Glen Mailer wrote: At the risk of making a slight strawman here, I agree this issue has the potential to drag on Clojure adoption - but in the opposite way to what you describe. At the root of it, is the Clojure ecosystem a dependencies are bad system, or a dependencies are good system? The former encourages roll-your-own, but reduces code sharing and re-use The latter promotes sharing and building upon the work of others, but requires more advanced dependency management tooling As great as the language is, having the general advice be don't have dependencies in your libraries seems like a step backwards to me. I would always rather spend an hour finding a documented, tested existing library, than spend 20 minutes creating my own duplicate of that work. I hope this doesn't come over as too accusatory, if the cost of dependencies truly is higher than the cost of duplicating work/code then perhaps we need to try and make the former easier. Glen -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Utility libraries and dependency hygiene
take this opportunity to ask everyone to help us avoid the dependency mess that Common Lisp has gotten into, where there are over a dozen such convenience libraries[1]. Are Common Lispers actively suffering under this problem? With the emergence of QuickLisp, CL dependency problems seem to have been smoothed over. adding to the dependencies of your library, you increase the likelihood of dependency conflicts for consumers. Agreed. But is the solution to strive for zero-dependencies? That seems extreme. How should we view contrib libraries? Should we avoid depending on them too? From my perspective, I try to minimize dependencies, but if I need a library then I use it. The Clojure community is very flexible, yet pragmatic. Maybe a better solution to zero-dependency is a suite of common libraries driven by the community, that is not affiliated with Core and contrib? I don't know, just a thought. It would lay somewhere between the more strict contrib and the wild-west model of NPM and JS micro-libs. -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Utility libraries and dependency hygiene
I think this discussion would greatly benefit from some hammock development time ;) The original post was quite precise in its scope, but we quickly side-stepped into related issues, so it feels like we have lost some focus. On the other hand I think many of us get the feeling we can't provide some satisfactory answer by just narrowing our discussion to the original proposed scope. We sense there's a bigger picture here and that's why we easily get carried away. So since the first step in hammock development is to state the problem, understand it, provide facts, context, constraints, trade-offs, etc. I propose we just do that. Let's start by naming the related areas that are at play here: 1. The design paradigm: How to design libraries with the right level of granularity and dependencies. How to make dependency layers (e.g. core libraries, then common utility libraries, etc.). This is the level where this discussion started, but it quickly went into the other complementary aspects below. 2. The programming model: How we logically organize the code into elements such as packages and namespaces. How we physically bundle the code into deployable units such as jars, wars, bundles, etc. The tools and api for defining and composing units, defining and resolving dependencies, such as leiningen, maven, ejb, spring, etc. 3. The runtime model: How the deployable units are discovered and loaded. Whether they can be loaded at boot time or dynamically at runtime (e.g. hot-deploy). Whether dependencies can also be resolved at runtime. Whether separate versions can coexists at runtime. That's just one way to decompose the problem into related areas, inspired by [1], and there are certainly other ways to decompose the problem. So the above is just a start that may provide more context to this discussion. Of particular interest is that fact that since Clojure is a hosted language, each host language has its own story to tell about 1,2, and 3. In the case of Java we're talking jars, classloaders, osgi, jigsaw in jdk9, etc. In JavaScript we're talking about namespaces, script loading, google closure dependency management, etc. I don't know about CLR but they certainly have their own story. So the key point here is that, while building the Clojure story on modularity, we need to be aware of how it maps to each host language. And if we were to better see the trade-offs we make, it could be a good exercise to come up with the ideal story for Clojure regardless of host languages, and then compare. My guess is that we'll discover we don't need much on the Clojure side of things, but that the real challenge is in how to integrate with the OO-twisted way of doing things in each host language. Which brings me to this: I think there's a window of opportunity between now and jigsaw in jdk9, because we won't be able to ignore their story. Rather than wait for it to be complete, it may be wise to be proactive and see what they're up to. I say this because by providing some input into the relevant JSRs we could avoid a stronger impedance mismatch between the Java/OO way of doing things, and the FP/Clojure way. For example: will the way of generating/loading classes at runtime be made easier? Will jigsaw make it easier to load separate versions of Clojure? At boot time or at runtime? Are interfaces the best and only way to define services, or would some form of duck typing be supported? Out of all the JVM languages Clojure is probably the one that has the most interesting things to say on FP best practices. Other languages are either hybrids, and therefore lack motivation for purity, or don't have the weight of the Clojure community. Finally, I can't help but think about Rich's presentation on the Language of Systems [2] and how this discussion fits into it. It seems to me there's some overlapping somewhere... Anyway, how about a wiki page we could use to keep some sort of blackboard on how to better state the problem, provide facts, context, constraints, trade-offs, etc.? Then we can each have our own separate and private hammock time to hammer this down. [1] http://www.kirkk.com/modularity/ [2] http://www.youtube.com/watch?v=ROor6_NGIWU -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit
Re: Utility libraries and dependency hygiene
Timothy Baldridge writes: Neither of these snarky answers solve the problem. I just spent an entire week updating modules from version of a library that expected a seq of maps to one that expected map of seqs of maps. The very nature of data implies a format. If that format changes you have a compatibility issue no fancy namespace system can solve. That is quite orthogonal to do with the topic at hand. Bridging two different libraries is often going to take some data munging. The fact that the two different libraries happen to have the same artifact-id and group-id is incidental once you start treating code as values. In fact, a better namespacing system would actually make it much easier to deal with because it would make it possible for both libraries to exist at the same time, much in the same way persistent data structures allow old versions of a vector to remain stable in the presence of change. -Phil pgpvl8UETak1n.pgp Description: PGP signature
Re: Utility libraries and dependency hygiene
On Wed, May 15, 2013 at 11:35 PM, Michael Fogus mefo...@gmail.com wrote: Are Common Lispers actively suffering under this problem? I certainly suffered from it back when I used Common Lisp. Every library was written in its own dialect of CL based on a different set of these utilities. It made it hard to read code written by anyone else. -S -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Utility libraries and dependency hygiene
I did not intend to say that libraries should never have any dependencies at all. (Perhaps zero was too strong in my original post.) I only ask that libraries try to avoid any **unnecessary** dependencies. Many utility libraries fall into this category, especially if you only depend on them for 1 or 2 functions. Given the dependency management tools we have right now, this will make life easier for users of your library. Thanks, -S On Mon, May 13, 2013 at 9:29 AM, Stuart Sierra the.stuart.sie...@gmail.comwrote: Based on a recent thread about utility libraries, I would like to take this opportunity to ask everyone to help us avoid the dependency mess that Common Lisp has gotten into, where there are over a dozen such convenience libraries[1]. By all means, use these libraries in your *applications* if you find them useful. But please don't make your *libraries* depend on them unless you really need to. Doing this will help application developers who want to use your library. For example, if my application depends on libraries A, B, and C, I might end up with transitive dependencies on three different utility libraries. If I want to add library D which depends on an incompatible version of one of those utilities, I'm stuck. By adding to the dependencies of your library, you increase the likelihood of dependency conflicts for consumers. The ideal number of dependencies for a library release (not counting Clojure itself) is zero. Obviously, use common sense. If your library relies on critical functionality in another library, then make the dependency. But if you can get by without the dependency (even if that means copying some of those utility functions into your own code and making them private) then you will make life easier for consumers of your library. Thanks, and happy coding. -S [1]: http://cliki.net/Convenience%20library -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to a topic in the Google Groups Clojure group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojure/WuS31RSiz_A/unsubscribe?hl=en. To unsubscribe from this group and all its topics, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Utility libraries and dependency hygiene
On May 15, 2013, at 8:28 AM, Dave Kincaid kincaid.d...@gmail.com wrote: One that I encountered last week was Midje. In trying to work through all of its dependencies I discovered that Pomegranate is also dependent on Maven, but I think I was able to work around that by adding 4 or 5 Maven libraries into our repository as dependencies (not ideal, but it looked like it would work). But I still wasn't able to get Midje working due some assumptions about Leiningen. This will be fixed in 1.6. I'm taking the minimize dependencies advice to heart for those libraries that Midje doesn't use much of. Latest book: /Functional Programming for the Object-Oriented Programmer/ https://leanpub.com/fp-oo -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Utility libraries and dependency hygiene
Hello, 2013/5/16 Stuart Sierra m...@stuartsierra.com: On Wed, May 15, 2013 at 11:35 PM, Michael Fogus mefo...@gmail.com wrote: Are Common Lispers actively suffering under this problem? I certainly suffered from it back when I used Common Lisp. Every library was written in its own dialect of CL based on a different set of these utilities. It made it hard to read code written by anyone else. How is the advice of each library re-creating for itself little utility functions, again and again, going to address the specific concern of made it hard to read code written by anyone else ? -S -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Utility libraries and dependency hygiene
On Thu, May 16, 2013 at 8:51 AM, Laurent PETIT laurent.pe...@gmail.comwrote: How is the advice of each library re-creating for itself little utility functions, again and again, going to address the specific concern of made it hard to read code written by anyone else ? If the functions are in the same file or project they're easier to find when you're browsing the code. -S -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Utility libraries and dependency hygiene
I must admit that I haven't read every response in this thread but my gut feeling is that a lot of the I just need a couple of functions situations would be mitigated by making non-core devs feel more welcome to suggesting and contributing modular contrib ideas. I hate to bring up process because I think there is more discussion to be had that is valuable and completely unrelated. I know why we have new contrib, but old contrib seemed to inspire smaller contributions. — Sent via Mobile On Wed, May 15, 2013 at 6:58 PM, Stuart Sierra m...@stuartsierra.com wrote: On Thu, May 16, 2013 at 8:51 AM, Laurent PETIT laurent.pe...@gmail.comwrote: How is the advice of each library re-creating for itself little utility functions, again and again, going to address the specific concern of made it hard to read code written by anyone else ? If the functions are in the same file or project they're easier to find when you're browsing the code. -S -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Utility libraries and dependency hygiene
Stuart Sierra m...@stuartsierra.com writes: On Tue, May 14, 2013 at 3:25 AM, Phil Hagelberg p...@hagelb.org wrote: It's really not difficult to do if you limit yourself to Clojure since Clojure namespaces are first-class and easy to manipulate at run-time. We implemneted a prototype of this in under two hours at a Seajure meeting a while back: https://github.com/technomancy/metaverse However, it's significantly more difficult to do for arbitrary Java bytecode. That's cool, and it will work for the simple case of libraries A and B depending on different versions of C. But it still breaks down in more complex cases: e.g. if I want to share data between A and B using a protocol or type defined in C, and there are 2 incompatible versions of C. Even ClassLoaders can't help you there - I'm not aware of any solution. Automatically, no, but the solution would be to use something akin to an adaptor. The two versions of C would be manipulated to be in different namespaces; now you just have two libraries, so the task of plumbing them together remains the same. To be honest, though, this is unlikely; after all, if you are using A and B, and they are using C *as a utility*, my feeling is that C shouldn't really be in their public interface. If C *is* in their public interface, then again, you need adaptors. Or you can fork A and/or B, fix them to use the same version! Phil -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Utility libraries and dependency hygiene
This thread seems to have gotten way off track and I think that Stuart made a very important point in the original post that's getting lost. It would help all of us out if library authors stopped making their libraries dependent on 10+ other libraries. This issue does have the potential to really drag on Clojure adoption. Just try to maintain your own local repository for your projects and you'll see what I mean the first time you need to add a Clojure dependency. On Tuesday, May 14, 2013 4:22:59 AM UTC-5, Phillip Lord wrote: Stuart Sierra ma...@stuartsierra.com javascript: writes: On Tue, May 14, 2013 at 3:25 AM, Phil Hagelberg ph...@hagelb.orgjavascript: wrote: It's really not difficult to do if you limit yourself to Clojure since Clojure namespaces are first-class and easy to manipulate at run-time. We implemneted a prototype of this in under two hours at a Seajure meeting a while back: https://github.com/technomancy/metaverse However, it's significantly more difficult to do for arbitrary Java bytecode. That's cool, and it will work for the simple case of libraries A and B depending on different versions of C. But it still breaks down in more complex cases: e.g. if I want to share data between A and B using a protocol or type defined in C, and there are 2 incompatible versions of C. Even ClassLoaders can't help you there - I'm not aware of any solution. Automatically, no, but the solution would be to use something akin to an adaptor. The two versions of C would be manipulated to be in different namespaces; now you just have two libraries, so the task of plumbing them together remains the same. To be honest, though, this is unlikely; after all, if you are using A and B, and they are using C *as a utility*, my feeling is that C shouldn't really be in their public interface. If C *is* in their public interface, then again, you need adaptors. Or you can fork A and/or B, fix them to use the same version! Phil -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Utility libraries and dependency hygiene
It may be pragmatic currently but *manually* copying code is a bad solution. Not depending on libraries is not a good solution either. The premise that there are only 3 levels of composition - clojure - library - application - I think is wrong, even if it works and is useful in some cases. Don't depend on things you don't need to. But don't turn this approach into dogma and give up on better answers. I think there are 3 problems here: 1. Possible library* *conflict - this is primarily a *Naming* conflict - i.e. currently you can only have one version because they are named the same. 2. Data incompatibility - this is incompatibility when passing data across interface boundaries. 3. Number of dependency versions - but only if you need to care about size. Library versioning is currently another example of place oriented programming. i.e same global name, different content = problem. Dave On Tuesday, 14 May 2013 22:19:15 UTC+10, Dave Kincaid wrote: This thread seems to have gotten way off track and I think that Stuart made a very important point in the original post that's getting lost. It would help all of us out if library authors stopped making their libraries dependent on 10+ other libraries. This issue does have the potential to really drag on Clojure adoption. Just try to maintain your own local repository for your projects and you'll see what I mean the first time you need to add a Clojure dependency. On Tuesday, May 14, 2013 4:22:59 AM UTC-5, Phillip Lord wrote: Stuart Sierra ma...@stuartsierra.com writes: On Tue, May 14, 2013 at 3:25 AM, Phil Hagelberg ph...@hagelb.org wrote: It's really not difficult to do if you limit yourself to Clojure since Clojure namespaces are first-class and easy to manipulate at run-time. We implemneted a prototype of this in under two hours at a Seajure meeting a while back: https://github.com/technomancy/metaverse However, it's significantly more difficult to do for arbitrary Java bytecode. That's cool, and it will work for the simple case of libraries A and B depending on different versions of C. But it still breaks down in more complex cases: e.g. if I want to share data between A and B using a protocol or type defined in C, and there are 2 incompatible versions of C. Even ClassLoaders can't help you there - I'm not aware of any solution. Automatically, no, but the solution would be to use something akin to an adaptor. The two versions of C would be manipulated to be in different namespaces; now you just have two libraries, so the task of plumbing them together remains the same. To be honest, though, this is unlikely; after all, if you are using A and B, and they are using C *as a utility*, my feeling is that C shouldn't really be in their public interface. If C *is* in their public interface, then again, you need adaptors. Or you can fork A and/or B, fix them to use the same version! Phil -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Utility libraries and dependency hygiene
Yes, by all means please just copy-n-paste out of https://github.com/runa-dev/kits if it simplifies your dependency tree. On Sun, May 12, 2013 at 8:53 PM, Dave Kincaid kincaid.d...@gmail.comwrote: Thanks for this, Stuart. I hope it's not too late. As one who has spent the last couple weeks spinning up a new project that uses its own local Ivy repository I've been feeling this pain first hand. The number of dependencies I have had to add just for a few Clojure libraries has been quite painful. I attributed it to how young the language is that these utility libraries haven't had time to consolidate. On Sunday, May 12, 2013 6:29:28 PM UTC-5, Stuart Sierra wrote: Based on a recent thread about utility libraries, I would like to take this opportunity to ask everyone to help us avoid the dependency mess that Common Lisp has gotten into, where there are over a dozen such convenience libraries[1]. By all means, use these libraries in your *applications* if you find them useful. But please don't make your *libraries* depend on them unless you really need to. Doing this will help application developers who want to use your library. For example, if my application depends on libraries A, B, and C, I might end up with transitive dependencies on three different utility libraries. If I want to add library D which depends on an incompatible version of one of those utilities, I'm stuck. By adding to the dependencies of your library, you increase the likelihood of dependency conflicts for consumers. The ideal number of dependencies for a library release (not counting Clojure itself) is zero. Obviously, use common sense. If your library relies on critical functionality in another library, then make the dependency. But if you can get by without the dependency (even if that means copying some of those utility functions into your own code and making them private) then you will make life easier for consumers of your library. Thanks, and happy coding. -S [1]: http://cliki.net/Convenience%**20libraryhttp://cliki.net/Convenience%20library -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Utility libraries and dependency hygiene
On Mon, May 13, 2013 at 12:55 PM, Mikera mike.r.anderson...@gmail.comwrote: Maybe we could try to develop towards some kind of lightweight dependency loading system that avoids this problem? I believe lightweight dependency loading system is an oxymoron. Either you A) design a new module format and try to get everyone to follow it (OSGI) or B) build an ad-hoc solution that tries to guess at the right behavior (JBoss). Either way, application developers still have a mess to deal with, just with an added layer of complexity. If someone out there can fix this problem once and for all, I will buy you a drink. But you can't bill me for all the drinks you'll need along the way. ;) -S -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Utility libraries and dependency hygiene
Hi, Am Montag, 13. Mai 2013 10:35:14 UTC+2 schrieb Stuart Sierra: I believe lightweight dependency loading system is an oxymoron. Either you A) design a new module format and try to get everyone to follow it (OSGI) or B) build an ad-hoc solution that tries to guess at the right behavior (JBoss). Either way, application developers still have a mess to deal with, just with an added layer of complexity. If someone out there can fix this problem once and for all, I will buy you a drink. But you can't bill me for all the drinks you'll need along the way. ;) Write a tool which rebases the dependency in your namespace tree. Et voila. No nifty class loader magic necessary. As long as you always use *ns* to get hold of the namespace name, this should work. Meikel -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Utility libraries and dependency hygiene
Are dependencies the potential problem, or are utility belt dependencies the issue here? If its the latter, then should we be trying to break down the utility belt into groups of related functionality, which can be stable and focused. Promoting zero-dependency libraries just means re-inventing a bunch of wheels, I believe node.js has this right by making dependencies cheap and easy and promoting publishing reusable libraries with only a few functions in. Not being able to load multiple versions of the same dependency make this slightly harder, but the general goal appears to me to be a good one. Glen -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Utility libraries and dependency hygiene
On Mon, May 13, 2013 at 1:42 AM, Meikel Brandmeyer (kotarak) m...@kotka.de wrote: Hi, Am Montag, 13. Mai 2013 10:35:14 UTC+2 schrieb Stuart Sierra: I believe lightweight dependency loading system is an oxymoron. Either you A) design a new module format and try to get everyone to follow it (OSGI) or B) build an ad-hoc solution that tries to guess at the right behavior (JBoss). Either way, application developers still have a mess to deal with, just with an added layer of complexity. If someone out there can fix this problem once and for all, I will buy you a drink. But you can't bill me for all the drinks you'll need along the way. ;) Write a tool which rebases the dependency in your namespace tree. Et voila. No nifty class loader magic necessary. As long as you always use *ns* to get hold of the namespace name, this should work. In Java-land this tool is called Jar Jar Links (http://code.google.com/p/jarjar/). -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Utility libraries and dependency hygiene
Angel Java Lopez writes: I guess it could be difficult to implement such feature in Java/Clojure It's really not difficult to do if you limit yourself to Clojure since Clojure namespaces are first-class and easy to manipulate at run-time. We implemneted a prototype of this in under two hours at a Seajure meeting a while back: https://github.com/technomancy/metaverse However, it's significantly more difficult to do for arbitrary Java bytecode. -Phil pgpKcBOJ394eb.pgp Description: PGP signature
Re: Utility libraries and dependency hygiene
I'm guilty of this... But I really don't know what to do I still find myself thinking... Hmmm this library provides about 50% of what I want... This library has 2 functions that I like And I don't like the way something is implemented here... If only i can combine this, this, and this... Oh screw it! I'm going to write my own ;) Any suggestions? -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Utility libraries and dependency hygiene
On Tue, May 14, 2013 at 3:25 AM, Phil Hagelberg p...@hagelb.org wrote: It's really not difficult to do if you limit yourself to Clojure since Clojure namespaces are first-class and easy to manipulate at run-time. We implemneted a prototype of this in under two hours at a Seajure meeting a while back: https://github.com/technomancy/metaverse However, it's significantly more difficult to do for arbitrary Java bytecode. That's cool, and it will work for the simple case of libraries A and B depending on different versions of C. But it still breaks down in more complex cases: e.g. if I want to share data between A and B using a protocol or type defined in C, and there are 2 incompatible versions of C. Even ClassLoaders can't help you there - I'm not aware of any solution. -S -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Utility libraries and dependency hygiene
Stuart Sierra writes: That's cool, and it will work for the simple case of libraries A and B depending on different versions of C. But it still breaks down in more complex cases: e.g. if I want to share data between A and B using a protocol or type defined in C, and there are 2 incompatible versions of C. Even ClassLoaders can't help you there - I'm not aware of any solution. One of many great reasons not to use protocols and records. =) -Phil pgpUUtqCHKWCP.pgp Description: PGP signature
Re: Utility libraries and dependency hygiene
Or a reason to integrate via data, not api or type. On Monday, May 13, 2013 10:54:26 PM UTC+2, Phil Hagelberg wrote: Stuart Sierra writes: That's cool, and it will work for the simple case of libraries A and B depending on different versions of C. But it still breaks down in more complex cases: e.g. if I want to share data between A and B using a protocol or type defined in C, and there are 2 incompatible versions of C. Even ClassLoaders can't help you there - I'm not aware of any solution. One of many great reasons not to use protocols and records. =) -Phil -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Utility libraries and dependency hygiene
Neither of these snarky answers solve the problem. I just spent an entire week updating modules from version of a library that expected a seq of maps to one that expected map of seqs of maps. The very nature of data implies a format. If that format changes you have a compatibility issue no fancy namespace system can solve. Timothy On May 13, 2013 4:08 PM, fmj...@gmail.com wrote: Or a reason to integrate via data, not api or type. On Monday, May 13, 2013 10:54:26 PM UTC+2, Phil Hagelberg wrote: Stuart Sierra writes: That's cool, and it will work for the simple case of libraries A and B depending on different versions of C. But it still breaks down in more complex cases: e.g. if I want to share data between A and B using a protocol or type defined in C, and there are 2 incompatible versions of C. Even ClassLoaders can't help you there - I'm not aware of any solution. One of many great reasons not to use protocols and records. =) -Phil -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Utility libraries and dependency hygiene
Only a lateral quick comment: Node.js + NPM (its package manager) has a nice version management. If your module A needs D version 0.1, and module B needs D version 0.2, no problem. Both versions are added, and A loads version 0.1, meanwhile B loads version 0.2. The trick is the search path for modules: each module has its own subfolder for dependencies I guess it could be difficult to implement such feature in Java/Clojure Angel Java Lopez @ajlopez On Sun, May 12, 2013 at 8:29 PM, Stuart Sierra the.stuart.sie...@gmail.comwrote: Based on a recent thread about utility libraries, I would like to take this opportunity to ask everyone to help us avoid the dependency mess that Common Lisp has gotten into, where there are over a dozen such convenience libraries[1]. By all means, use these libraries in your *applications* if you find them useful. But please don't make your *libraries* depend on them unless you really need to. Doing this will help application developers who want to use your library. For example, if my application depends on libraries A, B, and C, I might end up with transitive dependencies on three different utility libraries. If I want to add library D which depends on an incompatible version of one of those utilities, I'm stuck. By adding to the dependencies of your library, you increase the likelihood of dependency conflicts for consumers. The ideal number of dependencies for a library release (not counting Clojure itself) is zero. Obviously, use common sense. If your library relies on critical functionality in another library, then make the dependency. But if you can get by without the dependency (even if that means copying some of those utility functions into your own code and making them private) then you will make life easier for consumers of your library. Thanks, and happy coding. -S [1]: http://cliki.net/Convenience%20library -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Utility libraries and dependency hygiene
Isolated dependency loading is not possible in the JVM without complex ClassLoader-based schemes like OSGI, which come with their own set of problems. -S On Mon, May 13, 2013 at 9:45 AM, Angel Java Lopez ajlopez2...@gmail.comwrote: Node.js + NPM (its package manager) has a nice version management. If your module A needs D version 0.1, and module B needs D version 0.2, no problem. Both versions are added, and A loads version 0.1, meanwhile B loads version 0.2. The trick is the search path for modules: each module has its own subfolder for dependencies I guess it could be difficult to implement such feature in Java/Clojure -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Utility libraries and dependency hygiene
On Monday, 13 May 2013 09:20:19 UTC+8, Stuart Sierra wrote: Isolated dependency loading is not possible in the JVM without complex ClassLoader-based schemes like OSGI, which come with their own set of problems. -S Hi Stuart, That's true for compiled Java classes on the classpath, but I don't see why it needs to be true for JVM languages like Clojure that are already loading / compiling most code at runtime. In effect we already have a complex ClassLoader based scheme. Maybe we could try to develop towards some kind of lightweight dependency loading system that avoids this problem? A lot of utility functions are so small that it wouldn't matter if you loaded them many times. It might even improve performance if the JIT is able to optimise different instantiations of functions based on how they are being used by different parts of the code base. The alternatives aren't pretty: 1. We do a lot of cut and paste coding. OK in small doses, but not going to scale. 2. We get into dependency deadlocks with different libraries using different versions 3. Everyone develops their own utility libraries independently and manages against this 4. Everyone co-ordinates their dependency updates (yeah, like that's going to happen...) 5. We wait 10 years until Java develops a decent module system that we can use I'm personally feeling this problem a lot: the move towards small composable libraries in Clojure is IMHO the right direction, but I'm concerned that it's making the dependency management problem more complex and we haven't (yet) got the tools to handle this. The nightmare scenario is that we end up with thousands of individually cool and useful libraries, but no practical way to make them work together. I touched on a few of these themes with my post The Environment as a Value, which would be one way to solve a lot of this via isolation of different versions of code: https://groups.google.com/forum/?fromgroups=#!topic/clojure-dev/S8BawG7nzJA -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Utility libraries and dependency hygiene
Thanks for this, Stuart. I hope it's not too late. As one who has spent the last couple weeks spinning up a new project that uses its own local Ivy repository I've been feeling this pain first hand. The number of dependencies I have had to add just for a few Clojure libraries has been quite painful. I attributed it to how young the language is that these utility libraries haven't had time to consolidate. On Sunday, May 12, 2013 6:29:28 PM UTC-5, Stuart Sierra wrote: Based on a recent thread about utility libraries, I would like to take this opportunity to ask everyone to help us avoid the dependency mess that Common Lisp has gotten into, where there are over a dozen such convenience libraries[1]. By all means, use these libraries in your *applications* if you find them useful. But please don't make your *libraries* depend on them unless you really need to. Doing this will help application developers who want to use your library. For example, if my application depends on libraries A, B, and C, I might end up with transitive dependencies on three different utility libraries. If I want to add library D which depends on an incompatible version of one of those utilities, I'm stuck. By adding to the dependencies of your library, you increase the likelihood of dependency conflicts for consumers. The ideal number of dependencies for a library release (not counting Clojure itself) is zero. Obviously, use common sense. If your library relies on critical functionality in another library, then make the dependency. But if you can get by without the dependency (even if that means copying some of those utility functions into your own code and making them private) then you will make life easier for consumers of your library. Thanks, and happy coding. -S [1]: http://cliki.net/Convenience%20library -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
