Hello Duncan, In my eyes, this proposal looks like some sort of generalization of Stackage; and one further use case is "special purpose" collection. My big question: how composable are these collections really? I can't put two collections with conflicting versions together (or can I? Do I union?); and is there any point to having a collection without versions in it? (If Cabal syntax is extended to support depending on collections as well as packages, yes?)
The classic use-case for package collections is deployment settings, ala Stack, or even Cargo lockfiles / Bundler Gemfile.lock (versioned collections). In all these use-cases package collections are treated as non-compositional things. http://doc.crates.io/guide.html http://bundler.io/v1.7/rationale.html#checking-your-code-into-version-control Libraries (compositional) do NOT publish lockfiles: only executables (non-compositional) DO. Re the file format, it seems fine; suitable for the lockfile use-case and the Stackage use-case. Less sure about the unioning semantics. Edward Excerpts from Duncan Coutts's message of 2015-07-14 05:52:46 -0700: > Hi folks, > > I'd like to get feedback on a spec/proposal for distributing package > collections via hackage. This is currently somewhere beyond vapourware > but certainly not a fait accompli and hopefully it is at an appropriate > point to get feedback. > > The basic idea is that package collections are: > * useful (IMHO, one of the top two solutions to dependency hell, > alongside nix-style package management); and > * just as we distribute packages via hackage, we should also be > able to easily distribute package collections. > > One would then use them with tools like cabal and stack. Distributing > via hackage (both in the sense of the format/protocol and in the sense > of the central community hackage instance) seems natural, and allows > taking advantage of much of the infrastructure we have for packages > already like: > * existing user accounts and management infrastructure on the > hackage website > * allowing anyone to host collections on their own servers, just > as they can host their own package archives currently (either as > static file sets or with smart servers) > * low barrier for distribution, potentially encouraging more > collections to be created potentially covering more use cases > * security infrastructure (currently in alpha) > * automatic mirroring (currently in alpha) > > Two obvious examples are stackage-lts and stackage-nightly but if we > lower the barrier for distribution then there may well be many more. For > example, the existing Linux distros put a lot of effort into selecting > and maintain package collections, and some of these collections could be > distributed via hackage. In fast several Linux distributions already use > Hackage's "distro" feature to advertise which versions of packages are > provided by that distro. One can also imagine special-purpose > collections, and there's probably cases we've not thought of yet. > > Package collections are different things from packages, not like "meta > packages" that one gets in some package systems. A package collection at > it's simplest is just a set of source package identifiers (ie > names-version pairs). Like packages, package collections have names and > versions and are immutable once distributed. > > The intention is that users can configure their tool to use > collection(s), either by nailing down a specific collection version, or > by not specifying a version it would default to the latest version of > the named collection. (But the specific behaviour is up to the tool) > > Use cases: > > * versioned collections. For some collections the policy by which > it's defined naturally uses meaningful versions. > * daily collections. These can have a date-form version number > imposed on them. > * "live" "rolling" collections. These could have a simple > monotonic increasing version with no particular meaning > attached. For such collections, clients might be configured to > use the latest (by not specifying a version), but it's always > possible to pick a specific revision. > * special-purpose collections. Not necessarily collections aiming > to cover a large number of common packages, but aiming to cover > some application area, or related stack of packages (e.g. some > of the web frameworks). > * negative collections. Collections of packages you may > specifically want to avoid (e.g. deprecated by their authors, or > known-broken). Using such collections would rely on clients that > can be configured to treat it negatively. > > Specifics: > > A package collection specifies a set of source package ids (id being > name-version pair). It also optionally specifies a (partial) flag > assignment for any package name. > > The collection does not specify how tools should treat them. That is, a > collection does not specify if it should be treated as a strong or a > soft constraint, inclusive or exclusive, positive or negative. Such > things are completely up to the client's policy and configuration. > Similarly for flag assignments, collections do not specify whether tools > should interpret these as strong or soft constraints. > > Syntax: > > Package collection names and versions exactly follow those of package > names (but they live in a different namespace). For example, > "stackage-lts-2.9", or "deprecated-343" (the latter being a "rolling" > collection with a meaningless monotonically increasing version). > > A collection distributed in the archive format is just a text file with > one entry per line, such as: > > foo-1.0 > foo-1.1 > bar >= 3 && < 4 > bar +this -that > > So each line can be one of: > * a simple package id > * a package version range, using Cabal version range syntax > * a package name with a flag assignment, + for on, - for off > > The interpretation of the above is that: > * both foo-1.0 and foo-1.1 are in the collection (ie union not > intersection) > * all versions of bar between 3 and 4 are in the collection > * the package bar has flag 'this' as True, and flag 'that' as > False > > Of course for some collections the policy is that only one version of > any package is included, but this is a policy question and the format > itself does not impose this constraint. > > Hackage archive format: > > collection files live under a different prefix from package tarballs > (but are still considered part of the archive) and are named after the > collection id. The collection files are not compressed (but of course > http clients and servers can negotiate transport compression). The > collection files are not included in nor listed in the existing > 00-index.tar.gz, but there's other json format metadata for a client to > enumerate the available collections and versions. And like with package > tarballs, a client that wants a specific collection version can > construct the url and fetch it directly. > > Security: > > The hackage security system that's currently in alpha testing can easily > be extended to cover collections, similarly to how it covers package > tarballs. > > Misc notes: > > There is no requirement that a hackage-format repo containing > collections be closed. That is, the collections may refer to packages > not in that archive. This could be useful for private hackage repos that > host a small number of private packages, but also host collections that > refer both to the private packages and public ones from the community > central hackage. The resolution of package names is done by the clients, > and some clients may be configured to union/overlay multiple repos. > > On the other hand, for the central community hackage it may be sensible > to enforce a policy that the collections it distributes be closed (ie > refer only to packages distributed via hackage). > > Questions: > > Is this sufficiently flexible to fully cover the obvious use cases? Are > there any interesting use cases that are excluded? > > Anything else? > > > Duncan > _______________________________________________ cabal-devel mailing list cabal-devel@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/cabal-devel
