Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-18 Thread Neil Van Dyke

Matthew Flatt wrote at 06/18/2013 07:59 AM:

In principle, you should add a versioned dependency on racket to
indicate that the package does not work with version 5.3.4, and so
users of v5.3.4 should get an earlier revision of the package.


Just a general comment... For production use, I try not to have 
libraries push people to new versions of Racket without good reason.


Despite my general awareness and practice, I was actually bit by this 
problem recently.  A consulting client who used one of my open source 
libraries in production had a problem that I had addressed in a new 
version of the library.  But it turned out that an interim version of 
the library added a dependency on some new Racket feature, and client's 
production setup would be disrupted if they had to move Racket versions 
(increases unknowns).  I ended up forking my own library for the client, 
and merging the fix into their fork.


In the more distant past, I think I've also had at least one situation 
in which I made a Racket (or PLT Scheme) library version use some new 
Racket feature, then in a subsequent version I reversed this change, so 
that it could work with an older Racket version.


This doesn't have to be a big encumbrance -- I do adopt new Racket 
features when they are win (e.g., submodules, define-logger), but I try 
to make small efforts to avoid requiring newer Racket versions than I 
really need, since pushing people to new Racket versions does seem to 
cause problems in practice.


Neil V.

_
 Racket Developers list:
 http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-17 Thread Michael Wilber
I vote for this change too.

I've been letting my planet packages sit in planet for too long now.
This change provides a bit nicer upgrade path. :)

Vincent St-Amour stamo...@ccs.neu.edu writes:
 I vote for this change.

 I'm happy to change my packages (which make more sense as
 single-collection packages anyway).

 If I understand correctly, this would also have the advantage of making
 a lot of github repositories (including most of mine) installable as
 packages automatically, no change required. That sounds like a huge win
 to me.

 Vincent



 At Fri, 14 Jun 2013 07:07:26 -0700,
 Matthew Flatt wrote:

 I think more people need to speak up on this question --- particularly
 authors of existing packages, since the current proposal necessitates
 an update to each existing package.

 The proposal is to make single-package collections the default:

  * If a directory used as a package has no info.rkt file, then it is
treated as a single-collection package.

The single collection's name is the same as the package name (which
tends to be the directory name, but it depends on how you install
the package).

  * If a directory used as a package has an info.rkt file, but
info.rkt doesn't explicitly say that the package is
multi-collection, then it's still a single-collection package.

The info.rkt file might supply a name for the single collection,
instead of leaving it to the package name; supplying a name would be
a requirement for ring-0 packages.

 For each existing package, the author would need to add a line to the
 package's info.rkt to indicate that it is a multi-collection package
 (or change the layout to single-collection mode, with the caveat that
 the package won't work with v5.3.4).

 Any more votes for/against?

 _
   Racket Developers list:
   http://lists.racket-lang.org/dev
 _
   Racket Developers list:
   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-17 Thread Greg Hendershott
I'm still thinking that I'll keep my existing multi-collection
packages as multi, to preserve compatibility with 5.3.4. Only because,
although my packages don't have many users, I'll err on the side of
their convenience.

But if someone else did want to change from multi to single:

1. Philosophical: Wouldn't this be bad to do without renaming the
package, since the spirit of the package system versioning is, never
break backward compatibility?  Or, is it acceptable, now, since this
is technically still in beta?

2. Practical: Would `raco pkg update' handle such a multi-single
transform?  Or should packagers instruct users to (a) remove and (b)
install?

I'm asking in case it would help to have the answer(s) ready for doc
or FAQ purposes.


On Sat, Jun 15, 2013 at 11:19 AM, Matthew Flatt mfl...@cs.utah.edu wrote:
 At Fri, 14 Jun 2013 21:14:58 -0400, Greg Hendershott wrote:
 I just want to be clear what I need to do to
 keep compatibility with 5.3.4 for existing packages. If that means
 adding something to info.rkt to say, yeah, I'm still multi, I may do
 that.

 Yes, that's exactly what will be required for a package to work with
 both v5.3.4 and after the change: keep the package multi-collection,
 and explicitly declare it as such by adding a line to info.rkt.

 For the new info.rkt field, should it be

   (define multi-collection? #t)

 ?

_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-16 Thread Greg Hendershott
On Sat, Jun 15, 2013 at 11:19 AM, Matthew Flatt mfl...@cs.utah.edu wrote:
 For the new info.rkt field, should it be

   (define multi-collection? #t)

 ?

That seems fine.
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-15 Thread Matthew Flatt
At Fri, 14 Jun 2013 21:14:58 -0400, Greg Hendershott wrote:
 I just want to be clear what I need to do to
 keep compatibility with 5.3.4 for existing packages. If that means
 adding something to info.rkt to say, yeah, I'm still multi, I may do
 that.

Yes, that's exactly what will be required for a package to work with
both v5.3.4 and after the change: keep the package multi-collection,
and explicitly declare it as such by adding a line to info.rkt.

For the new info.rkt field, should it be

  (define multi-collection? #t)

?

_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-14 Thread Matthew Flatt
I think more people need to speak up on this question --- particularly
authors of existing packages, since the current proposal necessitates
an update to each existing package.

The proposal is to make single-package collections the default:

 * If a directory used as a package has no info.rkt file, then it is
   treated as a single-collection package.

   The single collection's name is the same as the package name (which
   tends to be the directory name, but it depends on how you install
   the package).

 * If a directory used as a package has an info.rkt file, but
   info.rkt doesn't explicitly say that the package is
   multi-collection, then it's still a single-collection package.

   The info.rkt file might supply a name for the single collection,
   instead of leaving it to the package name; supplying a name would be
   a requirement for ring-0 packages.

For each existing package, the author would need to add a line to the
package's info.rkt to indicate that it is a multi-collection package
(or change the layout to single-collection mode, with the caveat that
the package won't work with v5.3.4).

Any more votes for/against?

_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-14 Thread Carl Eastlund
I vote for this change.  I'll happily update my package in order to make it
easier for others to contribute new ones.

Carl Eastlund


On Fri, Jun 14, 2013 at 10:07 AM, Matthew Flatt mfl...@cs.utah.edu wrote:

 I think more people need to speak up on this question --- particularly
 authors of existing packages, since the current proposal necessitates
 an update to each existing package.

 The proposal is to make single-package collections the default:

  * If a directory used as a package has no info.rkt file, then it is
treated as a single-collection package.

The single collection's name is the same as the package name (which
tends to be the directory name, but it depends on how you install
the package).

  * If a directory used as a package has an info.rkt file, but
info.rkt doesn't explicitly say that the package is
multi-collection, then it's still a single-collection package.

The info.rkt file might supply a name for the single collection,
instead of leaving it to the package name; supplying a name would be
a requirement for ring-0 packages.

 For each existing package, the author would need to add a line to the
 package's info.rkt to indicate that it is a multi-collection package
 (or change the layout to single-collection mode, with the caveat that
 the package won't work with v5.3.4).

 Any more votes for/against?

 _
   Racket Developers list:
   http://lists.racket-lang.org/dev

_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-14 Thread Stephen Bloch

 I think more people need to speak up on this question --- particularly
 authors of existing packages, since the current proposal necessitates
 an update to each existing package.
 
 The proposal is to make single-package collections the default:

Fine by me: the only package I've contributed is single-collection anyway. And 
this does sound like it makes the common case easy.


Stephen Bloch
sbl...@adelphi.edu

_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-14 Thread Greg Hendershott
Oh. I thought the proposal was that packages would remain multi
collection by default, and you'd add something like (define
single-collection name) in info.rkt to opt for single.  And the work
for current packagers would be if they wanted to change from multi to
single.

But I'm fine either way. I just want to be clear what I need to do to
keep compatibility with 5.3.4 for existing packages. If that means
adding something to info.rkt to say, yeah, I'm still multi, I may do
that. But I would really enjoy using single for any new ones.



On Fri, Jun 14, 2013 at 10:07 AM, Matthew Flatt mfl...@cs.utah.edu wrote:
 I think more people need to speak up on this question --- particularly
 authors of existing packages, since the current proposal necessitates
 an update to each existing package.

 The proposal is to make single-package collections the default:

  * If a directory used as a package has no info.rkt file, then it is
treated as a single-collection package.

The single collection's name is the same as the package name (which
tends to be the directory name, but it depends on how you install
the package).

  * If a directory used as a package has an info.rkt file, but
info.rkt doesn't explicitly say that the package is
multi-collection, then it's still a single-collection package.

The info.rkt file might supply a name for the single collection,
instead of leaving it to the package name; supplying a name would be
a requirement for ring-0 packages.

 For each existing package, the author would need to add a line to the
 package's info.rkt to indicate that it is a multi-collection package
 (or change the layout to single-collection mode, with the caveat that
 the package won't work with v5.3.4).

 Any more votes for/against?

 _
   Racket Developers list:
   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-14 Thread Neil Van Dyke
I've been hesitant to comment on any of this, for three reasons: (1) 
I've read the new package system documentation on at least 3 separate 
occasions, and -- perhaps because I'm biased by having already formed 
some ideas about where I'd like things to go -- I've had trouble 
understanding the rationale behind a lot of the decisions; (2) I'm not 
familiar enough with DrRacket internals to know whether there's some 
unusual barrier to keeping things simpler than it seems like people have 
been talking about, so I can't help with that; and (3) I'm confused 
about what's already decided and what's open to discussion, and suspect 
it would take many more hours to understand.


Since I'm heavily invested in Racket packages but lost in the current 
discussion (sorry), all I can offer right now are some comments that 
start from the concepts of the original PLaneT, in case anyone who 
understands the new system sees anything of interest in this quick brain 
dump...


For all of my packages, as well as any package I can imagine, I think 
that the original PLaneT got many things right or close to right:


* Some kind of unique package-name controlled by a developer.

* Multiple versions for a given package-name (I'll call them 
package-versions in this email) can be installed, and there is some 
version selection mechanism.


* Files from a package-version end up grouped together in the directory 
structure, specific to that package-version, and certainly not mixed 
into directories with files from package-versions of different 
package-names.


* Flat namespace (let's ignore the PLaneT package-owner part for now), 
without attempt to name packages according to some topical ontology.


* Metadata in info.rkt.

I was expecting to use this original PLaneT as a starting point, and 
evolve it in ways like the following...


* In addition to the (planet ...) require-specs, package-versions also 
can come from http:, https:, and git: URLs.  (github: would also 
be OK.)  Each such URL would identify trees or a tarball.  Then we see 
how people choose the PLaneT server vs. HTTP vs. Git over time.


* Simple web-of-trust package-version public-key signing of 
package-versions (e.g., URLs plus hashes of contents), to start with, 
perhaps initially with only centralized repository for signatures.  Soon 
build distributed web-of-trust, plus multiple repositories so 
organizations have option to keep their signatures separate.  Build 
mechanisms atop that, including advancing the state of the art.


* Maybe improve the version-selection and compatibility support.  
Investigate whether it's worthwhile to separate out the 
backward-compatibility information from the static package-version 
distribution (and especially from the version number), or whether in 
practice there are simpler ways that are satisfactory.


* Maybe a facility in info.rkt to provide aliases for require specs.  
Otherwise, people writing nontrivial multi-file code that uses other 
packages from PLaneT/whatever end up having to make wrapper modules so 
that we don't goof our require-specs and accidentally depend on multiple 
package-versions for the same package-name.  Note that, with URLs, these 
aliases *might* be the only actual package-name construct in the 
HTTP/Git system as distinct from URL similarities of package-versions.  
This info might be implicit in a package-version's info.rkt's 
reference to a previous package-version, perhaps coming from an 
assertion of compatibility info.  This might be simpler than it might 
sound, but it has some interesting implications, including for forking 
and web-of-trust.)


* Automate and simplify releasing in general.  With PLaneT, it's been 
not-unusual for even core Racket developers to avoid releasing some 
add-on code to PLaneT, perhaps because the clerical stuff was a 
headache.  For the old PLaneT, I was simplifying this with McFly, but 
with new a package mechanism, I would start with that and then ask what 
clerical parts still need help.  (For example, if doing development in 
an SCM repository that's accessed directly via require-specs, then 
releasing a package-version might consist mainly of adding a tag/label.  
planet-lang.org's directory might even update automatically, given info 
about a previously-released package-version of the same package-name.)


* Use submodule support to support single-file packages, at least for 
the HTTP/Git package-versions.  (module+ info ...).  It seems from 
Emacs history that some people really like the single-file module, it 
lowers barriers, and now submodules give us an easy way to finally do it.


* Do whatever is necessary to avoid blocking the program for 
few/several-minutes while documentation is reformatted, when requiring 
an uncached package-version.  Maybe even moving it to an async process 
that's run when idle (Unix nice?) would work.


* To put it vaguely: keep things simple in most cases, but don't 
dumb-down in practically restrictive ways, and 

Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-07 Thread Greg Hendershott
 I am *very* strongly in favor of this -- I'd rather have
 single-collection packages than multi-collection packages, if forced
 to choose. I'm very glad that you and Laurent have done the work here.

 I'd be happy to update all of my packages.  Currently, of my 9
 packages on pkg.racket-lang.org, 8 are single collection, and 1
 splices into the existing `data` collection (and has a second
 collection to work around a now-fixed limitation.

Likewise I'd be happy to update all of mine.


As I thought about this, I realized I have what feels like a _really_
dumb question. I realize maybe I don't have a crisp handle on
collection, library, module, and subdirectory.

A single collection can have multiple modules, correct?  At what point
does it become multiple collections -- simply because some of the
modules are in subdirs?

In other words:

  my-collection/
subidr1/
subdir2/

Is that 1 collection or 3?  And why?

Is the answer related to whether there's an info.rkt in a subdir?

The discussion at http://docs.racket-lang.org/guide/module-basics.html
doesn't clear this up for me (unless I'm overlooking something on
multiple re-reads). 6.1.1. has an example of modules organized in
subdirs. This feels like one collection but the term hasn't been
introduced yet.  6.1.2 starts talking about library collections, and
says A collection is a set of installed library modules. But it
doesn't say modules only in the same directory, and it doesn't
really explain at what point it would be considered separate sets of
modules, i.e. two collections instead of one.

Again, I feel like I'm being dense and will probably be embarrassed
when I hear the answer, but ... there. I asked.
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-07 Thread Jay McCarthy
On Fri, Jun 7, 2013 at 2:49 PM, Greg Hendershott
greghendersh...@gmail.com wrote:
 I am *very* strongly in favor of this -- I'd rather have
 single-collection packages than multi-collection packages, if forced
 to choose. I'm very glad that you and Laurent have done the work here.

 I'd be happy to update all of my packages.  Currently, of my 9
 packages on pkg.racket-lang.org, 8 are single collection, and 1
 splices into the existing `data` collection (and has a second
 collection to work around a now-fixed limitation.

 Likewise I'd be happy to update all of mine.


 As I thought about this, I realized I have what feels like a _really_
 dumb question. I realize maybe I don't have a crisp handle on
 collection, library, module, and subdirectory.

(require a/b/c x/y/z)

A collection is something that comes first in a require: a and x.
A library normally means module, but might be a general term for code I share.
A module path is the full path: a/b/c and x/y/z.
A module is a file: a/b/c.rkt and x/y/z.rkt
A module has a 'short name': 'c and 'z (but you normally don't think about it.)
A subdirectory is where a file might be and shows up in the module path.
A package is a set of modules.

Some documentation might refer to subdirectories as sub collections.

This whole debate has been about whether a SINGLE package can have
MANY modules from DIFFERENT collections.

Can package P have module a/b/c AND x/y/z?
Does package P have to have a collection P?

The current package system puts no restriction on what modules a
package can have in it.

The proposed patch supported vigorously by Sam is to restrict some
packages so that the can only have modules from one collection. Since
this restriction is only on some packages, it doesn't take power from
the system. And some people believe that this restriction is more
convenient, because they do not have to come up with a name for P or a
directory for it. There is a second debate on the point of whether the
collection name of a restricted package P must be P.

Jay

 A single collection can have multiple modules, correct?  At what point
 does it become multiple collections -- simply because some of the
 modules are in subdirs?

 In other words:

   my-collection/
 subidr1/
 subdir2/

 Is that 1 collection or 3?  And why?

 Is the answer related to whether there's an info.rkt in a subdir?

 The discussion at http://docs.racket-lang.org/guide/module-basics.html
 doesn't clear this up for me (unless I'm overlooking something on
 multiple re-reads). 6.1.1. has an example of modules organized in
 subdirs. This feels like one collection but the term hasn't been
 introduced yet.  6.1.2 starts talking about library collections, and
 says A collection is a set of installed library modules. But it
 doesn't say modules only in the same directory, and it doesn't
 really explain at what point it would be considered separate sets of
 modules, i.e. two collections instead of one.

 Again, I feel like I'm being dense and will probably be embarrassed
 when I hear the answer, but ... there. I asked.
 _
   Racket Developers list:
   http://lists.racket-lang.org/dev



--
Jay McCarthy j...@cs.byu.edu
Assistant Professor / Brigham Young University
http://faculty.cs.byu.edu/~jay

The glory of God is Intelligence - DC 93
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-07 Thread Greg Hendershott
Thank you for the thorough explanation.

Also, I'm having the duh moment I predicted.

A collection may have modules in subdirs and still be just one collection.

The use case for a multi-collection package is when you have
collections A and B that you want to be packaged and installed
together -- as top-level A and B collections. (Although you make A
and B into subdirs of a package dir when preparing a multi-collection
package -- even though you push them down for that prep -- when
installed they'll be hoisted back up side by side with other
collections.)


On Fri, Jun 7, 2013 at 5:04 PM, Jay McCarthy jay.mccar...@gmail.com wrote:
 On Fri, Jun 7, 2013 at 2:49 PM, Greg Hendershott
 greghendersh...@gmail.com wrote:
 I am *very* strongly in favor of this -- I'd rather have
 single-collection packages than multi-collection packages, if forced
 to choose. I'm very glad that you and Laurent have done the work here.

 I'd be happy to update all of my packages.  Currently, of my 9
 packages on pkg.racket-lang.org, 8 are single collection, and 1
 splices into the existing `data` collection (and has a second
 collection to work around a now-fixed limitation.

 Likewise I'd be happy to update all of mine.


 As I thought about this, I realized I have what feels like a _really_
 dumb question. I realize maybe I don't have a crisp handle on
 collection, library, module, and subdirectory.

 (require a/b/c x/y/z)

 A collection is something that comes first in a require: a and x.
 A library normally means module, but might be a general term for code I 
 share.
 A module path is the full path: a/b/c and x/y/z.
 A module is a file: a/b/c.rkt and x/y/z.rkt
 A module has a 'short name': 'c and 'z (but you normally don't think about 
 it.)
 A subdirectory is where a file might be and shows up in the module path.
 A package is a set of modules.

 Some documentation might refer to subdirectories as sub collections.

 This whole debate has been about whether a SINGLE package can have
 MANY modules from DIFFERENT collections.

 Can package P have module a/b/c AND x/y/z?
 Does package P have to have a collection P?

 The current package system puts no restriction on what modules a
 package can have in it.

 The proposed patch supported vigorously by Sam is to restrict some
 packages so that the can only have modules from one collection. Since
 this restriction is only on some packages, it doesn't take power from
 the system. And some people believe that this restriction is more
 convenient, because they do not have to come up with a name for P or a
 directory for it. There is a second debate on the point of whether the
 collection name of a restricted package P must be P.

 Jay

 A single collection can have multiple modules, correct?  At what point
 does it become multiple collections -- simply because some of the
 modules are in subdirs?

 In other words:

   my-collection/
 subidr1/
 subdir2/

 Is that 1 collection or 3?  And why?

 Is the answer related to whether there's an info.rkt in a subdir?

 The discussion at http://docs.racket-lang.org/guide/module-basics.html
 doesn't clear this up for me (unless I'm overlooking something on
 multiple re-reads). 6.1.1. has an example of modules organized in
 subdirs. This feels like one collection but the term hasn't been
 introduced yet.  6.1.2 starts talking about library collections, and
 says A collection is a set of installed library modules. But it
 doesn't say modules only in the same directory, and it doesn't
 really explain at what point it would be considered separate sets of
 modules, i.e. two collections instead of one.

 Again, I feel like I'm being dense and will probably be embarrassed
 when I hear the answer, but ... there. I asked.
 _
   Racket Developers list:
   http://lists.racket-lang.org/dev



 --
 Jay McCarthy j...@cs.byu.edu
 Assistant Professor / Brigham Young University
 http://faculty.cs.byu.edu/~jay

 The glory of God is Intelligence - DC 93
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-06 Thread Laurent
Great! Thank you very much.

On Wed, Jun 5, 2013 at 11:04 PM, Matthew Flatt mfl...@cs.utah.edu wrote:

 More generally, I hope I haven't come across as being firmly opposed to
 the idea of single-collection packages. I intended to come across as
 being opposed to implementing the idea myself. :)


I wanted to give it a stab, but didn't have much time. Plus you did it way
better than I could have done anyway.


 Some other the details:

  * A package's mode is recorded in the installed-package table.
Otherwise, a linked package could switch modes just because the
package directory's content changes, which would be difficult to
keep in sync with the low-level table of links.


Does it handle the  case where user A installs B's single-collection
package,
and at some point B changes its package to a multi-collection package,
triggering the updating of A's installation?

As Jay pointed out to me, putting the collection name in the info.rkt
 file --- instead of using the package name as the collection name ---
 makes the package content work the same even when the package is
 renamed. Otherwise, you have to know whether the package is
 single-collection or multi-collection to know whether changing the
 package's name changes its content.

 It's debatable whether single-collection or multi-collection would be a
 better default, but I favor multi-collection mode.


What I'd like is to have single-collection being the default so that the
average Joe can even not care about including a info.rkt file, and
expect the collection's name to be just the directory's name. Less burden
on the user - more packages.
I believe most users will not even (try to) understand the concept of multi-
collection packages. To them a package is just a bunch of source files
and that's all.

Then to create a simple package, Joe only has to create a directory,
put his my-file.rkt in it, and he's done!

If at some point Joe thinks about having a different name for his package
and for his collection, it should be intuitive to him that he should say so
somewhere (i.e. in the info.rkt file).

So here is a demo patch attached to precise what I mean (without test,
would have taken me way too much time). Because it considers that
single-collections are the default, it is backward incompatible.
If info.rkt exists, it looks for 'multi-collection, and otherwise looks for
the
'collection-name string.
If info.rkt does not exist, it creates it and gives the 'collection-name
the name of the package by default.
(This of course fails if the directory is read-only, but since it's just a
useful default it should not be a problem.
This should at least work for all downloaded archives.)

Laurent


lib.rkt.patch
Description: Binary data
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-06 Thread Matthew Flatt
At Thu, 6 Jun 2013 13:36:38 +0200, Laurent wrote:
  Some other the details:
 
   * A package's mode is recorded in the installed-package table.
 Otherwise, a linked package could switch modes just because the
 package directory's content changes, which would be difficult to
 keep in sync with the low-level table of links.
 
 
 Does it handle the  case where user A installs B's single-collection
 package,
 and at some point B changes its package to a multi-collection package,
 triggering the updating of A's installation?

Yes.


 What I'd like is to have single-collection being the default [...]
 
 So here is a demo patch attached to precise what I mean (without
 test, would have taken me way too much time). Because it considers
 that single-collections are the default, it is backward incompatible.
 If info.rkt exists, it looks for 'multi-collection, and otherwise
 looks for the 'collection-name string.

I could go along with this, as long as (most?) everyone agrees, and as
long as package authors are willing to update existing packages.

If we go that way, then I'd characterize a single-collection package
without 'single-collection' in info.rkt as a low-quality package, but
a low-quality package is a fine starting point for a high-quality
package.


 If info.rkt does not exist, it creates it and gives the 'collection-name
 the name of the package by default.

That doesn't seem like a good idea to me. As you've noted, there can be
problems with writing extra files. The collection name could be instead
computed in `pkg-single-collection', which would treat the absence of
an info.rkt to mean single-collection mode, right?


Some calls to `pkg-single-collection' would need to change to pass in
a package name to use instead of the `dir' name, since the `dir'
argument is sometimes a temporary directory that is used to stage a
package. That's an easy change.

_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-06 Thread Sam Tobin-Hochstadt
On Thu, Jun 6, 2013 at 10:00 AM, Matthew Flatt mfl...@cs.utah.edu wrote:


 What I'd like is to have single-collection being the default [...]

 So here is a demo patch attached to precise what I mean (without
 test, would have taken me way too much time). Because it considers
 that single-collections are the default, it is backward incompatible.
 If info.rkt exists, it looks for 'multi-collection, and otherwise
 looks for the 'collection-name string.

 I could go along with this, as long as (most?) everyone agrees, and as
 long as package authors are willing to update existing packages.

I am *very* strongly in favor of this -- I'd rather have
single-collection packages than multi-collection packages, if forced
to choose. I'm very glad that you and Laurent have done the work here.

I'd be happy to update all of my packages.  Currently, of my 9
packages on pkg.racket-lang.org, 8 are single collection, and 1
splices into the existing `data` collection (and has a second
collection to work around a now-fixed limitation.

In the interests of data, I looked at all of the packages on
pkg.racket-lang.org marked as recently updated (6 of them) as well as
the first 10 in alphabetical order. Of these, all but 3 have a single
collection, 1 has some test code in the `tests` collection, 1 has an
additional `example` collection, and 1 has an additional `attic`
collection that doesn't have any .rkt files to avoid the code there
ever being compiled. I think this suggests that single-collection is
the right default.

 If we go that way, then I'd characterize a single-collection package
 without 'single-collection' in info.rkt as a low-quality package, but
 a low-quality package is a fine starting point for a high-quality
 package.

Would this characterization be something reflected in any of the
software, or just a recommendation we make to packagers?

Sam
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-06 Thread Laurent
 If info.rkt does not exist, it creates it and gives the 'collection-name
  the name of the package by default.

 That doesn't seem like a good idea to me. As you've noted, there can be
 problems with writing extra files. The collection name could be instead
 computed in `pkg-single-collection', which would treat the absence of
 an info.rkt to mean single-collection mode, right?


 Some calls to `pkg-single-collection' would need to change to pass in
 a package name to use instead of the `dir' name, since the `dir'
 argument is sometimes a temporary directory that is used to stage a
 package. That's an easy change.


If the original package/directory name can be retrieved at any time, then
yes, it's a much better idea (actually I had started this way, but didn't
know how to retrieve the name at any time).

Laurent
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-06 Thread Matthew Flatt
At Thu, 6 Jun 2013 10:17:28 -0400, Sam Tobin-Hochstadt wrote:
  If we go that way, then I'd characterize a single-collection package
  without 'single-collection' in info.rkt as a low-quality package, but
  a low-quality package is a fine starting point for a high-quality
  package.
 
 Would this characterization be something reflected in any of the
 software, or just a recommendation we make to packagers?

Mostly a recommendation, but it might correspond to a software-enforced
constraint on ring-0 packages (for which DrDr runs tests).

_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-06 Thread Jay McCarthy
On Thu, Jun 6, 2013 at 8:17 AM, Sam Tobin-Hochstadt sa...@ccs.neu.edu wrote:
 On Thu, Jun 6, 2013 at 10:00 AM, Matthew Flatt mfl...@cs.utah.edu wrote:


 What I'd like is to have single-collection being the default [...]

 So here is a demo patch attached to precise what I mean (without
 test, would have taken me way too much time). Because it considers
 that single-collections are the default, it is backward incompatible.
 If info.rkt exists, it looks for 'multi-collection, and otherwise
 looks for the 'collection-name string.

 I could go along with this, as long as (most?) everyone agrees, and as
 long as package authors are willing to update existing packages.

 I am *very* strongly in favor of this -- I'd rather have
 single-collection packages than multi-collection packages, if forced
 to choose. I'm very glad that you and Laurent have done the work here.

The main problem with this is that it brings in internal linking where
code directly refers to other packages (implementations) and not
modules (interfaces) as the default. This means we will see code like
(require jays-awesome-data-structure) rather than (require
data/awesome). I think this is bad because it makes it harder to deal
with forking, maintainers abandoning their code, splitting packages
into packages that just depend on 'jays-awesome-data-structure, rather
than implementing it internally, etc. Obviously you can still deal
with those things, but if the path of evolution we imagine starts off
with single-package-with-package/collection-name-shared, then most
packaged collections will have a package as their name.

Jay

--
Jay McCarthy j...@cs.byu.edu
Assistant Professor / Brigham Young University
http://faculty.cs.byu.edu/~jay

The glory of God is Intelligence - DC 93
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-06 Thread Sam Tobin-Hochstadt
On Thu, Jun 6, 2013 at 1:42 PM, Jay McCarthy jay.mccar...@gmail.com wrote:

 I am *very* strongly in favor of this -- I'd rather have
 single-collection packages than multi-collection packages, if forced
 to choose. I'm very glad that you and Laurent have done the work here.

 The main problem with this is that it brings in internal linking where
 code directly refers to other packages (implementations) and not
 modules (interfaces) as the default. This means we will see code like
 (require jays-awesome-data-structure) rather than (require
 data/awesome). I think this is bad because it makes it harder to deal
 with forking, maintainers abandoning their code, splitting packages
 into packages that just depend on 'jays-awesome-data-structure, rather
 than implementing it internally, etc. Obviously you can still deal
 with those things, but if the path of evolution we imagine starts off
 with single-package-with-package/collection-name-shared, then most
 packaged collections will have a package as their name.

First, I don't see why single-collection packages make things harder
to deal with re-implementing a package.  Imagine that I make
`disassemble` a single-collection package, and someone depends on it,
and then I abandon my code.  You can easily create a new package that
ships that collection.  I think that places the burden correctly --
it's easy to create packages initially, and handling abandoned
packages takes a little more work.  In JavaScript, where there's a
flat namespace, this works -- the Zepto library just provides the same
names as jQuery, even though they're different implementations and not
generic names.

Second, I think this is already happening.  Almost every package on
`pkg.racket-lang.org` provides a new collection which is the primary
code for the package, and which is named the same as the package.

Sam
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-06 Thread Jay McCarthy
On Thu, Jun 6, 2013 at 12:03 PM, Sam Tobin-Hochstadt sa...@ccs.neu.edu wrote:
 On Thu, Jun 6, 2013 at 1:42 PM, Jay McCarthy jay.mccar...@gmail.com wrote:

 I am *very* strongly in favor of this -- I'd rather have
 single-collection packages than multi-collection packages, if forced
 to choose. I'm very glad that you and Laurent have done the work here.

 The main problem with this is that it brings in internal linking where
 code directly refers to other packages (implementations) and not
 modules (interfaces) as the default. This means we will see code like
 (require jays-awesome-data-structure) rather than (require
 data/awesome). I think this is bad because it makes it harder to deal
 with forking, maintainers abandoning their code, splitting packages
 into packages that just depend on 'jays-awesome-data-structure, rather
 than implementing it internally, etc. Obviously you can still deal
 with those things, but if the path of evolution we imagine starts off
 with single-package-with-package/collection-name-shared, then most
 packaged collections will have a package as their name.

 First, I don't see why single-collection packages make things harder
 to deal with re-implementing a package.  Imagine that I make
 `disassemble` a single-collection package, and someone depends on it,
 and then I abandon my code.  You can easily create a new package that
 ships that collection.  I think that places the burden correctly --
 it's easy to create packages initially, and handling abandoned
 packages takes a little more work.  In JavaScript, where there's a
 flat namespace, this works -- the Zepto library just provides the same
 names as jQuery, even though they're different implementations and not
 generic names.

The problem is that if you use the name samths-disassemble then even
in twenty years, my implementation uses the collection
samths-disassemble and not disassemble.

Jay


--
Jay McCarthy j...@cs.byu.edu
Assistant Professor / Brigham Young University
http://faculty.cs.byu.edu/~jay

The glory of God is Intelligence - DC 93
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-06 Thread Laurent
On Thu, Jun 6, 2013 at 7:42 PM, Jay McCarthy jay.mccar...@gmail.com wrote:

 On Thu, Jun 6, 2013 at 8:17 AM, Sam Tobin-Hochstadt sa...@ccs.neu.edu
 wrote:
  On Thu, Jun 6, 2013 at 10:00 AM, Matthew Flatt mfl...@cs.utah.edu
 wrote:
 
 
  What I'd like is to have single-collection being the default [...]
 
  So here is a demo patch attached to precise what I mean (without
  test, would have taken me way too much time). Because it considers
  that single-collections are the default, it is backward incompatible.
  If info.rkt exists, it looks for 'multi-collection, and otherwise
  looks for the 'collection-name string.
 
  I could go along with this, as long as (most?) everyone agrees, and as
  long as package authors are willing to update existing packages.
 
  I am *very* strongly in favor of this -- I'd rather have
  single-collection packages than multi-collection packages, if forced
  to choose. I'm very glad that you and Laurent have done the work here.

 The main problem with this is that it brings in internal linking where
 code directly refers to other packages (implementations) and not
 modules (interfaces) as the default. This means we will see code like
 (require jays-awesome-data-structure) rather than (require
 data/awesome). I think this is bad because it makes it harder to deal
 with forking, maintainers abandoning their code, splitting packages
 into packages that just depend on 'jays-awesome-data-structure, rather
 than implementing it internally, etc. Obviously you can still deal
 with those things, but if the path of evolution we imagine starts off
 with single-package-with-package/collection-name-shared, then most
 packaged collections will have a package as their name.


Ok, I see your point.
Let me try to make a use case here:
User A writes package P on Github at A/P.
A creates a PLaneT package also named P.
User B forks P to B/P.
Since the package name P is already taken, you expect B to name his
package B-P.
(But I think you make this point sufficiently clear in the docs that this
should be very seldom.)

So on installation, you have a collection name B-P, which is bad,
because you cannot replace A's P directly by B's P, simply by
uninstalling A's and installing B's.

Now, the thing is you'd like B's collection to be name P also, right?
(which means only one could be installed at the same time, otherwise we
get back to the same problem, whatever the option we choose.)

Now if we really want to have P as the default collection name for B's P,
we could also probably use B's Github repo name. Is it retrievable from
the downloaded package?
We would then use this if available, or the package/directory name
otherwise (I guess non-repo installations will be rare anyway).

But then people would still write
$ raco pkg install jay-awesome-package
but would then need to write
(require awesome)

which is, in most cases, quite counter-intuitive: you'd need to state it
clearly in the docs of the package, but you would still have dyslexic
users
asking why it doesn't work (like I have with you, btw, with
slideshow-latex).
And this will probably occur much more often than the case above,
because you have many more users of packages than developers, and the
default should be intuitive for the largest part.

Laurent
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-06 Thread Sean McBeth
What if the differentiation between User A and User B's Package P were
encoded in the version number, instead of the name. Semantically, that's
what we're dealing with, two different versions of the same package.
Directly after a fork, the packages would be Package P, version A.1.15, and
Package P, version B.0.1;


On Thu, Jun 6, 2013 at 2:31 PM, Laurent laurent.ors...@gmail.com wrote:




 On Thu, Jun 6, 2013 at 7:42 PM, Jay McCarthy jay.mccar...@gmail.comwrote:

 On Thu, Jun 6, 2013 at 8:17 AM, Sam Tobin-Hochstadt sa...@ccs.neu.edu
 wrote:
  On Thu, Jun 6, 2013 at 10:00 AM, Matthew Flatt mfl...@cs.utah.edu
 wrote:
 
 
  What I'd like is to have single-collection being the default [...]
 
  So here is a demo patch attached to precise what I mean (without
  test, would have taken me way too much time). Because it considers
  that single-collections are the default, it is backward incompatible.
  If info.rkt exists, it looks for 'multi-collection, and otherwise
  looks for the 'collection-name string.
 
  I could go along with this, as long as (most?) everyone agrees, and as
  long as package authors are willing to update existing packages.

 
  I am *very* strongly in favor of this -- I'd rather have
  single-collection packages than multi-collection packages, if forced
  to choose. I'm very glad that you and Laurent have done the work here.

 The main problem with this is that it brings in internal linking where
 code directly refers to other packages (implementations) and not
 modules (interfaces) as the default. This means we will see code like
 (require jays-awesome-data-structure) rather than (require
 data/awesome). I think this is bad because it makes it harder to deal
 with forking, maintainers abandoning their code, splitting packages
 into packages that just depend on 'jays-awesome-data-structure, rather
 than implementing it internally, etc. Obviously you can still deal
 with those things, but if the path of evolution we imagine starts off
 with single-package-with-package/collection-name-shared, then most
 packaged collections will have a package as their name.


 Ok, I see your point.
 Let me try to make a use case here:
 User A writes package P on Github at A/P.
 A creates a PLaneT package also named P.
 User B forks P to B/P.
 Since the package name P is already taken, you expect B to name his
 package B-P.
 (But I think you make this point sufficiently clear in the docs that this
 should be very seldom.)

 So on installation, you have a collection name B-P, which is bad,
 because you cannot replace A's P directly by B's P, simply by
 uninstalling A's and installing B's.

 Now, the thing is you'd like B's collection to be name P also, right?
 (which means only one could be installed at the same time, otherwise we
 get back to the same problem, whatever the option we choose.)

 Now if we really want to have P as the default collection name for B's P,
 we could also probably use B's Github repo name. Is it retrievable from
 the downloaded package?
 We would then use this if available, or the package/directory name
 otherwise (I guess non-repo installations will be rare anyway).

 But then people would still write
 $ raco pkg install jay-awesome-package
 but would then need to write
 (require awesome)

 which is, in most cases, quite counter-intuitive: you'd need to state it
 clearly in the docs of the package, but you would still have dyslexic
 users
 asking why it doesn't work (like I have with you, btw, with
 slideshow-latex).
 And this will probably occur much more often than the case above,
 because you have many more users of packages than developers, and the
 default should be intuitive for the largest part.

 Laurent



 _
   Racket Developers list:
   http://lists.racket-lang.org/dev


_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-06 Thread Sam Tobin-Hochstadt
Can't this be alleviated by the guidance on naming that the docs already
provide?

Sam
On Jun 6, 2013 2:30 PM, Jay McCarthy jay.mccar...@gmail.com wrote:

 On Thu, Jun 6, 2013 at 12:03 PM, Sam Tobin-Hochstadt sa...@ccs.neu.edu
 wrote:
  On Thu, Jun 6, 2013 at 1:42 PM, Jay McCarthy jay.mccar...@gmail.com
 wrote:
 
  I am *very* strongly in favor of this -- I'd rather have
  single-collection packages than multi-collection packages, if forced
  to choose. I'm very glad that you and Laurent have done the work here.
 
  The main problem with this is that it brings in internal linking where
  code directly refers to other packages (implementations) and not
  modules (interfaces) as the default. This means we will see code like
  (require jays-awesome-data-structure) rather than (require
  data/awesome). I think this is bad because it makes it harder to deal
  with forking, maintainers abandoning their code, splitting packages
  into packages that just depend on 'jays-awesome-data-structure, rather
  than implementing it internally, etc. Obviously you can still deal
  with those things, but if the path of evolution we imagine starts off
  with single-package-with-package/collection-name-shared, then most
  packaged collections will have a package as their name.
 
  First, I don't see why single-collection packages make things harder
  to deal with re-implementing a package.  Imagine that I make
  `disassemble` a single-collection package, and someone depends on it,
  and then I abandon my code.  You can easily create a new package that
  ships that collection.  I think that places the burden correctly --
  it's easy to create packages initially, and handling abandoned
  packages takes a little more work.  In JavaScript, where there's a
  flat namespace, this works -- the Zepto library just provides the same
  names as jQuery, even though they're different implementations and not
  generic names.

 The problem is that if you use the name samths-disassemble then even
 in twenty years, my implementation uses the collection
 samths-disassemble and not disassemble.

 Jay


 --
 Jay McCarthy j...@cs.byu.edu
 Assistant Professor / Brigham Young University
 http://faculty.cs.byu.edu/~jay

 The glory of God is Intelligence - DC 93

_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-06 Thread Jay McCarthy
On Thu, Jun 6, 2013 at 12:31 PM, Laurent laurent.ors...@gmail.com wrote:



 On Thu, Jun 6, 2013 at 7:42 PM, Jay McCarthy jay.mccar...@gmail.com wrote:

 On Thu, Jun 6, 2013 at 8:17 AM, Sam Tobin-Hochstadt sa...@ccs.neu.edu
 wrote:
  On Thu, Jun 6, 2013 at 10:00 AM, Matthew Flatt mfl...@cs.utah.edu
  wrote:
 
 
  What I'd like is to have single-collection being the default [...]
 
  So here is a demo patch attached to precise what I mean (without
  test, would have taken me way too much time). Because it considers
  that single-collections are the default, it is backward incompatible.
  If info.rkt exists, it looks for 'multi-collection, and otherwise
  looks for the 'collection-name string.
 
  I could go along with this, as long as (most?) everyone agrees, and as
  long as package authors are willing to update existing packages.
 
  I am *very* strongly in favor of this -- I'd rather have
  single-collection packages than multi-collection packages, if forced
  to choose. I'm very glad that you and Laurent have done the work here.

 The main problem with this is that it brings in internal linking where
 code directly refers to other packages (implementations) and not
 modules (interfaces) as the default. This means we will see code like
 (require jays-awesome-data-structure) rather than (require
 data/awesome). I think this is bad because it makes it harder to deal
 with forking, maintainers abandoning their code, splitting packages
 into packages that just depend on 'jays-awesome-data-structure, rather
 than implementing it internally, etc. Obviously you can still deal
 with those things, but if the path of evolution we imagine starts off
 with single-package-with-package/collection-name-shared, then most
 packaged collections will have a package as their name.


 Ok, I see your point.
 Let me try to make a use case here:
 User A writes package P on Github at A/P.
 A creates a PLaneT package also named P.
 User B forks P to B/P.
 Since the package name P is already taken, you expect B to name his
 package B-P.
 (But I think you make this point sufficiently clear in the docs that this
 should be very seldom.)

 So on installation, you have a collection name B-P, which is bad,
 because you cannot replace A's P directly by B's P, simply by
 uninstalling A's and installing B's.

 Now, the thing is you'd like B's collection to be name P also, right?
 (which means only one could be installed at the same time, otherwise we
 get back to the same problem, whatever the option we choose.)

 Now if we really want to have P as the default collection name for B's P,
 we could also probably use B's Github repo name. Is it retrievable from
 the downloaded package?
 We would then use this if available, or the package/directory name
 otherwise (I guess non-repo installations will be rare anyway).

 But then people would still write
 $ raco pkg install jay-awesome-package
 but would then need to write
 (require awesome)

 which is, in most cases, quite counter-intuitive: you'd need to state it
 clearly in the docs of the package, but you would still have dyslexic
 users
 asking why it doesn't work (like I have with you, btw, with
 slideshow-latex).
 And this will probably occur much more often than the case above,
 because you have many more users of packages than developers, and the
 default should be intuitive for the largest part.

You are correct that I see the separation of package-name and
code-to-require as a big feature. You also correct that right now this
can be a bit painful where it's not clear how they are connected.
However, I see that as a consequence of where we are in development.

My goal is to have a large xrefed documentation site of all ring-0
packages (we are close to this) so you can look up awesome or
slideshow-latex and then require it and have racket/DrRacket give you
a pop up that says, Do you want to install package blah to get
this? (I believe DrRacket does this now and we have xrepl that does
it too). Since ring-0 guarantees no conflicts, there is always a
unique answer to the require-package question.

In other words, my goal is to have only package authors ever concerned
with (a) their package names and (b) which package they are even
getting their modules from. UNLESS, you want to use ring-1 (untested)
or ring-2 (conflicting) packages---since using those rings is
inconvenient, it is incentivizes authors to remove conflicts and test
their code to get to ring-0. (In other words, I don't want any ring-1
or 2 packages.)

Jay


--
Jay McCarthy j...@cs.byu.edu
Assistant Professor / Brigham Young University
http://faculty.cs.byu.edu/~jay

The glory of God is Intelligence - DC 93
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-05 Thread Matthew Flatt
At Tue, 4 Jun 2013 07:41:53 -0400, Eli Barzilay wrote:
 Yesterday, Jay McCarthy wrote:
  and you should deal with the non-proof of concept method of
  specifying it in, for instance, the info file, which is now package
  info AND collect info.
 
 This shouldn't be a problem

At Tue, 4 Jun 2013 10:32:12 +0200, Laurent wrote:
 As Greg suggested, the single-collection package would have only one
 info.rkt file (which is simpler for the user[*]), and would contain both
 the info for the package and for the collection.

I agree with both of you.

My intent in echoing Jay's comment about info.rkt was to point out
areas that need attention to fill out the patch --- not to suggest that
it isn't possible to make things work.

More generally, I hope I haven't come across as being firmly opposed to
the idea of single-collection packages. I intended to come across as
being opposed to implementing the idea myself. :)

But it seems that I haven't convinced anyone to take on the
implementation task (understandably), and I've arrived back to a point
where I want single-collection packages (in the ongoing
collection-splitting experiment). So, the enclosed patch is a
suggestion on how to have single-collection packages.


With the patch, a single-collection package must have an info.rkt
file containing a definition of `single-collection' as a string, and
the string is used as the package's collection name. That is, you don't
have to know from the outside what kind of package you're installing;
it's up to the package author to choose single-collection mode or
multi-collection mode within the package's implementation. A package in
either mode can be upgraded to the other mode. If you install a package
with `--link' and you want to change modes, the only available
upgrade process (at the moment) is to remove and re-install the
package.

Some other the details:

 * A package's mode is recorded in the installed-package table.
   Otherwise, a linked package could switch modes just because the
   package directory's content changes, which would be difficult to
   keep in sync with the low-level table of links.

 * Both modes are handled in-place instead of trying to reshape one
   mode's directory structure to the other.

 * In a single-collection package, info.rkt doubles as the package's
   info.rkt and the collection's info.rkt.

 * .plt-format files cannot encode single-collection packages. It's a
   deprecated format, anyway.

I experimented with various ways of distinguishing single-collection
packages from multi-collection packages. An info.rkt definition seems
best among the things I tried, at least. (A discarded experiment was to
treat a directory a single-collection package when it has any .rkt
file other than info.rkt.)

As Jay pointed out to me, putting the collection name in the info.rkt
file --- instead of using the package name as the collection name ---
makes the package content work the same even when the package is
renamed. Otherwise, you have to know whether the package is
single-collection or multi-collection to know whether changing the
package's name changes its content.

It's debatable whether single-collection or multi-collection would be a
better default, but I favor multi-collection mode.

We can try different ways of distinguishing single-collection package
from multi-collection packages by changing the implementation of
`pkg-single-collection', as long as we stick to the distinction being
inside the package instead of outside.


0001-raco-pkg-support-single-collection-packages.patch
Description: Binary data
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-04 Thread Laurent
On Mon, Jun 3, 2013 at 7:36 PM, Matthew Flatt mfl...@cs.utah.edu wrote:

 The simplification of not handling links also seems like a big weakness
 to me. I think one of the best things about Jay's design is that I
 don't have to change the way I think about a package when moving
 between link and non-link modes (which makes it much easier to work
 with a GitHub repository, for example). I particularly encourage you to
 think about what should happen when a user needs to transition a GitHub
 repo from a representing a single-collection package to a
 multiple-collection package --- and how that interacts with links as
 well as non-link, GitHub-based installations of the package.


Actually I use collection links instead of package links for my
single-collection packages, which are Github clones. It kind of makes
sense. (One more homogeneous possibility would be to add a `raco pkg
install --link-single my-collect/package` command that is just a synonym
for `raco link my-collect/package`. Actually I would much prefer to have
`raco pkg install --link` work like `raco link` and have an additional
`raco pkg install --link-multi` command instead.)

As Greg suggested, the single-collection package would have only one
info.rkt file (which is simpler for the user[*]), and would contain both
the info for the package and for the collection. As they seem to have
mutually-exclusive information, it should be ok. (But one would have to be
sure that this will remain the case in the future but I think it should
hold. I think it should be ok too, see at the bottom of the email.)
E.g., dependencies of a single collection fit well in the collection's
info.rkt I think (meaning these are the dependencies of the collection).
When installing the package with 'dir, the collection gets its own subdir,
and the info.rkt file just needs to be copied in the package directory.

When changing from a single-collection to a multi-collection package, you'd
have to do (I think):
- collect-unlink the current package,
- move the collection to a subdirectory (along with the info.rkt file),
- create a info.rkt file in the package root, and move the parts of the
collection's info.rkt that are package-specific to this new file,
- package-link the new package.

Since I don't expect people (other than the Racket dev) to do that often
(or even at all for most of them), I don't see that's a problem (it's not
even a big deal).
When someone wants to do that, it means he already has some good knowledge
about collections and packages (since he wants to build a multi-collection
package), and thus it should not be a problem either.

You could generalize the principle:
- When all you need is a file, just use a file. It should have all the
necessary information (i.e., why not have a 'info' submodule? E.g. to deal
with dependencies)
- If you need several files, create a folder, put your files in there, and
add a collection-specific file (info.rkt)
- if you need several collections (a package), create a new folder, put
your collections in there, add a file that is specific to the collection of
collections (the package)
- if you need several packages (a repository?), create a new folder, put
your packages in there, etc.
- etc.

The info.rkt files that are closer to the root are more general, and more
specific info.rkt files should have precedence in case there is a conflict
(like in CSS for example).

I expect the first two cases to be overwhelmingly dominant, and thus they
should be easier to achieve.

Laurent

[*] I think a good tool is one that asks for Minimum User Effort, and if
the user wants to do something simple, it should be simple to achieve.
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-04 Thread Eli Barzilay
Yesterday, Jay McCarthy wrote:
 and you should deal with the non-proof of concept method of
 specifying it in, for instance, the info file, which is now package
 info AND collect info.

This shouldn't be a problem -- *most* of the information in the info
files that we have now is really information that applies to packages.
The only thing that I could find that doesn't is compile-omit-paths.
In fact, as I said in the past, the creation of a new info file is the
thing that is problematic.

What if, for example, there is a `name' entry in both a package info
file and in (one of) its collections?  The first one would supposedly
be used when talking about the package and the second one would be
used in setup to indicate the compiled collection...  Or what happens
with the `version' entry in some of the existing collections?  My
guess is that the answer to both is that there shouldn't be entries
that exist in both info files (eg, the current `version' fields in the
collection info files should go away) -- and that shows that there
shouldn't be ambiguity problems.


Yesterday, Matthew Flatt wrote:
 
 Meanwhile, as we've started leaning on the package system and looked
 at how to fit our current collections into it, I think we're running
 into the same problem at another level: if web-server-lib and
 web-server-doc are both packages in a single web-server
 repository, and if they have to be separate directories, then we
 have yet another layer of directories that affect repository layout.

Yes, that's related, and part of my motivation to avoid that level of
splitting.  I think that telling people that they should start their
hacking work with

  git clone pltgit:pkg/web-server
  edit web-server/web-server-lib/web-server/web-server.rkt

is making it obvious that there's some problem here...


 Maybe there's a more general answer to the directory-layout
 problem that would neatly cover single-collection packages.

I originally thought (/hoped) that you would be able to place packages
at any level.  Going back to the info files -- this is what happens
now, roughly -- you just specify an info file anywhere in the tree and
it gets picked up as something very close to packages.  (For example,
the game info files, or drr tools in nested directories.)


Three hours ago, Laurent wrote:
 
 Actually I use collection links instead of package links for my
 single-collection packages, which are Github clones. It kind of
 makes sense.  (One more homogeneous possibility would be to add a
 `raco pkg install --link-single my-collect/package` command that
 is just a synonym for `raco link my-collect/package`.

Yes, that's what I thought should happen -- that installed packages
get linked as a single collection when they have a single collection.


 Actually I would much prefer to have `raco pkg install --link` work
 like `raco link` and have an additional `raco pkg install
 --link-multi` command instead.)

+173, with --root being used for both.


 - When all you need is a file, just use a file. It should have all
 the necessary information (i.e., why not have a 'info' submodule?
 E.g. to deal with dependencies)

(This would be nice, but won't work now, since there's no way to
require toplevel files.  But that's fine since it corresponds to the
fact that you can't have a repository without a directory to hold it.
So I view an `in-url' thing is the way for this super-cheap code
distribution.)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-03 Thread Eli Barzilay
Yesterday, Laurent wrote:
 On Sun, Jun 2, 2013 at 1:47 PM, Eli Barzilay e...@barzilay.org wrote:
 
 To clarify, because of reasons that I won't go into on the list,
 the actual chances of me getting this implemented (and of such a
 change being accepted) are pretty much in the area of slim to
 none.  
 
 That's a bummer. At first sight I'd have thought that it would have
 just been a matter of creating a directory with the same name as the
 package, and then decompressing the package into that directory.

At the lower level, packages use links, and raco link's default mode
of work is at the collection level.  There is obviously a need for
packages that correspond to collection roots, but I wanted to see the
default being the same as the raco-link default (which fits most uses
that don't come out of the current main tree).


 TBH, the chances of implementing an `in-url' are low too, but
 they're probably higher than going into the package system.
 
 That would be nice, but I'd much much prefer to see the
 single-collection package thing implemented.

Well, the reason I think that plain collection packages are important
is it will simplify dealing with packages, and lower the investment
threshold that you need to publish a package.  Along the same line,
there should also be a way to put out just a single file (or very few)
as a first step where I make some code public but with absolutely
minimum effort on the publisher's side.  See for example the
Processing thread, where Sean said that people who do these kind of
thing don't even know what source control is, let alone packages --
so the existence of such a crowd is a perfect reason for a single-file
thing being another important point on the code distribution spectrum.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!

_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-03 Thread Laurent
On Mon, Jun 3, 2013 at 2:44 PM, Eli Barzilay e...@barzilay.org wrote:

 Yesterday, Laurent wrote:
  On Sun, Jun 2, 2013 at 1:47 PM, Eli Barzilay e...@barzilay.org wrote:
 
  To clarify, because of reasons that I won't go into on the list,
  the actual chances of me getting this implemented (and of such a
  change being accepted) are pretty much in the area of slim to
  none.
 
  That's a bummer. At first sight I'd have thought that it would have
  just been a matter of creating a directory with the same name as the
  package, and then decompressing the package into that directory.

 At the lower level, packages use links, and raco link's default mode
 of work is at the collection level.  There is obviously a need for
 packages that correspond to collection roots, but I wanted to see the
 default being the same as the raco-link default (which fits most uses
 that don't come out of the current main tree).


Not sure I was clear, or I don't exactly understand what you mean.
And maybe I'm missing some important point here.

But the more I think about it, the less I see why this
double-directory thing should be the default behavior of the new package
system (I can understand why it's useful for plt packages though).
Even Carl's mischief package doesn't make use of collections like
that.
Jen's this-and-that does, though, but then every collection of his
package must have a name that is different from any existing collection.
(And actually I think that's a bad idea, and that he should split his
package into several single-collection packages −unless they are
tightly related− because I don't want to install all his collections if I
want
to only try one of them, unless there is a way to install only one
collection of an existing package?)

For example, my personal racket project tree is arranged somewhat like
that:
racket
├── project-B
│   └── main.rkt
├── project-A
│   └── main.rkt
├── not-yet(?)a-project-C
├── my-collects
│   ├── slideshow
│   ├── scribble
│   ├── racket
│   └── data
└── misc
├── try-this.rkt
└── try-that.rkt

project-A and project-B are git clones.
my-collects contains all my extensions to racket's collections,
plus some more generally useful collections.

For my personal use, I just do:
  raco link project-A project-B my-collects
and all is fine.
I tend to believe it's a quite common tree for racket users.

In my-collects, I have names that would collide with racket's
collections,
so instead of having:
(require orseau/racket/slideshow)
as in the former package system, I would now have
(require some-funny-name/slideshow)
after renaming my-collects to some-funny-name to make sure it does not
collide with anything else.

... and since I don't want to invent a funny name for each extension of a
racket collection I do, I prefer to put them all in a single package and
collection.

Now if I were to follow the new package system, I'd need to change my
tree to:
racket
├── misc
│   ├── try-that.rkt
│   └── try-this.rkt
├── some-funny-name
│   └── some-funny-name
│   ├── data
│   ├── racket
│   ├── scribble
│   └── slideshow
├── not-yet(?)a-project-D
├── project-A
│   └── project-A
│   └── main.rkt
└── project-B
└── project-B
└── main.rkt

which is a mess. And I would also need to change my git repos to
match that structure, which looks awkward.
/But/ I don't care if the tree of the packages I *install* look like that.

So what I would like is to keep my tree like the first one, but the
package system can still install like the second one.

My proposal was that for single-collection packages, I could keep
my tree like the first one, my git clones like project-A/main.rkt, and
tell the package manager (probably in the info.rkt file) that it is a
single-collection package so that when it installs it in the
installed-package-directory-that-I-will-not-look-into it generates a tree
like the second one.

 Well, the reason I think that plain collection packages are important
 is it will simplify dealing with packages, and lower the investment
 threshold that you need to publish a package.  Along the same line,
 there should also be a way to put out just a single file (or very few)
 as a first step where I make some code public but with absolutely
 minimum effort on the publisher's side.  See for example the
 Processing thread, where Sean said that people who do these kind of
 thing don't even know what source control is, let alone packages --
 so the existence of such a crowd is a perfect reason for a single-file
 thing being another important point on the code distribution spectrum.


I entirely agree with that, and I think it is of utmost importance for a
programming language (and system) to make sure that its users can
easily share code. Probably all of you agree with that, but I want to
emphasize that it's really important to put as much effort as required
into that.

Don't get me wrong, I really like the steps forward taken with the new
direction of 

Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-03 Thread Eli Barzilay
(I completely agree with you, so I'll take it off-line.)


30 minutes ago, Laurent wrote:
 On Mon, Jun 3, 2013 at 2:44 PM, Eli Barzilay e...@barzilay.org wrote:
 
 Yesterday, Laurent wrote:
  On Sun, Jun 2, 2013 at 1:47 PM, Eli Barzilay e...@barzilay.org wrote:
 
      To clarify, because of reasons that I won't go into on the list,
      the actual chances of me getting this implemented (and of such a
      change being accepted) are pretty much in the area of slim to
      none.  
 
  That's a bummer. At first sight I'd have thought that it would have
  just been a matter of creating a directory with the same name as the
  package, and then decompressing the package into that directory.

 At the lower level, packages use links, and raco link's default mode
 of work is at the collection level.  There is obviously a need for
 packages that correspond to collection roots, but I wanted to see the
 default being the same as the raco-link default (which fits most uses
 that don't come out of the current main tree).
 
 Not sure I was clear, or I don't exactly understand what you mean.
 And maybe I'm missing some important point here.
 
 But the more I think about it, the less I see why this
 double-directory thing should be the default behavior of the new package
 system (I can understand why it's useful for plt packages though).
 Even Carl's mischief package doesn't make use of collections like
 that.
 Jen's this-and-that does, though, but then every collection of his
 package must have a name that is different from any existing collection.
 (And actually I think that's a bad idea, and that he should split his
 package into several single-collection packages −unless they are
 tightly related− because I don't want to install all his collections if I want
 to only try one of them, unless there is a way to install only one
 collection of an existing package?)
 
 For example, my personal racket project tree is arranged somewhat like
 that:
 racket
 ├── project-B
 │   └── main.rkt
 ├── project-A
 │   └── main.rkt
 ├── not-yet(?)a-project-C
 ├── my-collects
 │   ├── slideshow
 │   ├── scribble
 │   ├── racket
 │   └── data
 └── misc
     ├── try-this.rkt
     └── try-that.rkt
 
 project-A and project-B are git clones.
 my-collects contains all my extensions to racket's collections,
 plus some more generally useful collections.
 
 For my personal use, I just do:
   raco link project-A project-B my-collects
 and all is fine.
 I tend to believe it's a quite common tree for racket users.
 
 In my-collects, I have names that would collide with racket's collections,
 so instead of having:
 (require orseau/racket/slideshow)
 as in the former package system, I would now have
 (require some-funny-name/slideshow)
 after renaming my-collects to some-funny-name to make sure it does not
 collide with anything else.
 
 ... and since I don't want to invent a funny name for each extension of a
 racket collection I do, I prefer to put them all in a single package and
 collection.
 
 Now if I were to follow the new package system, I'd need to change my
 tree to:
 racket
 ├── misc
 │   ├── try-that.rkt
 │   └── try-this.rkt
 ├── some-funny-name
 │   └── some-funny-name
 │   ├── data
 │   ├── racket
 │   ├── scribble
 │   └── slideshow
 ├── not-yet(?)a-project-D
 ├── project-A
 │   └── project-A
 │   └── main.rkt
 └── project-B
     └── project-B
     └── main.rkt
 
 which is a mess. And I would also need to change my git repos to
 match that structure, which looks awkward.
 /But/ I don't care if the tree of the packages I *install* look like that.
 
 So what I would like is to keep my tree like the first one, but the
 package system can still install like the second one.
 
 My proposal was that for single-collection packages, I could keep
 my tree like the first one, my git clones like project-A/main.rkt, and
 tell the package manager (probably in the info.rkt file) that it is a
 single-collection package so that when it installs it in the
 installed-package-directory-that-I-will-not-look-into it generates a tree
 like the second one.
 
 Well, the reason I think that plain collection packages are important
 is it will simplify dealing with packages, and lower the investment
 threshold that you need to publish a package.  Along the same line,
 there should also be a way to put out just a single file (or very few)
 as a first step where I make some code public but with absolutely
 minimum effort on the publisher's side.  See for example the
 Processing thread, where Sean said that people who do these kind of
 thing don't even know what source control is, let alone packages --
 so the existence of such a crowd is a perfect reason for a single-file
 thing being another important point on the code distribution spectrum.
 
 I entirely agree with that, and I think it is of utmost importance for a
 programming 

Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-03 Thread Laurent
Here is a patch for a proof of concept (file /collects/pkg/lib.rkt).

The modifications are minimal as I had expected, but obviously I only have
a very narrow view of the package system, so probably something does not
work properly.
In particular, I changed only for the 'dir type, and did not touch the
'link type.
Since the 'github type and others use the 'dir type once the archive is
downloaded, I think it should work as expected for those too.

I've tested on an old 5.3.4.10--2013-05-13(96acfb0/a) [3m], but made the
patch with the latest in git, and it still works with the old version.
(I was about to test it with the latest nightly, but its installation
failed, see the mailing list)

To try it, in the collects/pkg directory:
$ patch lib.rkt lib.rkt.patch
$ sudo raco setup -D pkg

Then create a dummy package:
$ cd /tmp
$ mkdir dummy
$ echo '#lang racket\n(displayln It works!)'  dummy/main.rkt
$ touch dummy/single-collect

The file single-collect is here to signal that this package is a
single-collection package (it's not the right way to do it, but it's a
proof of concept).

Install the package:
$ raco pkg install -t dir dummy

(You should see This is a single-collection package as the first line of
the log)

Then try it:
$ racket
 (require dummy)
It works!

Or is this patch way too naive for some reason?

Laurent



On Mon, Jun 3, 2013 at 4:37 PM, Eli Barzilay e...@barzilay.org wrote:

 (I completely agree with you, so I'll take it off-line.)


 30 minutes ago, Laurent wrote:
  On Mon, Jun 3, 2013 at 2:44 PM, Eli Barzilay e...@barzilay.org wrote:
 
  Yesterday, Laurent wrote:
   On Sun, Jun 2, 2013 at 1:47 PM, Eli Barzilay e...@barzilay.org
 wrote:
  
   To clarify, because of reasons that I won't go into on the
 list,
   the actual chances of me getting this implemented (and of such
 a
   change being accepted) are pretty much in the area of slim to
   none.
  
   That's a bummer. At first sight I'd have thought that it would have
   just been a matter of creating a directory with the same name as
 the
   package, and then decompressing the package into that directory.
 
  At the lower level, packages use links, and raco link's default
 mode
  of work is at the collection level.  There is obviously a need for
  packages that correspond to collection roots, but I wanted to see the
  default being the same as the raco-link default (which fits most uses
  that don't come out of the current main tree).
 
  Not sure I was clear, or I don't exactly understand what you mean.
  And maybe I'm missing some important point here.
 
  But the more I think about it, the less I see why this
  double-directory thing should be the default behavior of the new package
  system (I can understand why it's useful for plt packages though).
  Even Carl's mischief package doesn't make use of collections like
  that.
  Jen's this-and-that does, though, but then every collection of his
  package must have a name that is different from any existing collection.
  (And actually I think that's a bad idea, and that he should split his
  package into several single-collection packages −unless they are
  tightly related− because I don't want to install all his collections if
 I want
  to only try one of them, unless there is a way to install only one
  collection of an existing package?)
 
  For example, my personal racket project tree is arranged somewhat like
  that:
  racket
  ├── project-B
  │   └── main.rkt
  ├── project-A
  │   └── main.rkt
  ├── not-yet(?)a-project-C
  ├── my-collects
  │   ├── slideshow
  │   ├── scribble
  │   ├── racket
  │   └── data
  └── misc
  ├── try-this.rkt
  └── try-that.rkt
 
  project-A and project-B are git clones.
  my-collects contains all my extensions to racket's collections,
  plus some more generally useful collections.
 
  For my personal use, I just do:
raco link project-A project-B my-collects
  and all is fine.
  I tend to believe it's a quite common tree for racket users.
 
  In my-collects, I have names that would collide with racket's
 collections,
  so instead of having:
  (require orseau/racket/slideshow)
  as in the former package system, I would now have
  (require some-funny-name/slideshow)
  after renaming my-collects to some-funny-name to make sure it does not
  collide with anything else.
 
  ... and since I don't want to invent a funny name for each extension of a
  racket collection I do, I prefer to put them all in a single package and
  collection.
 
  Now if I were to follow the new package system, I'd need to change my
  tree to:
  racket
  ├── misc
  │   ├── try-that.rkt
  │   └── try-this.rkt
  ├── some-funny-name
  │   └── some-funny-name
  │   ├── data
  │   ├── racket
  │   ├── scribble
  │   └── slideshow
  ├── not-yet(?)a-project-D
  ├── project-A
  │   └── project-A
  │   └── main.rkt
  └── project-B
  └── project-B
  └── main.rkt
 
  which 

Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-03 Thread Jay McCarthy
Oh, and test cases would go here:

https://github.com/plt/racket/blob/master/collects/tests/pkg/tests-create.rkt

and

https://github.com/plt/racket/blob/master/collects/tests/pkg/tests-install.rkt

On Mon, Jun 3, 2013 at 10:44 AM, Jay McCarthy jay.mccar...@gmail.com wrote:
 You would need to update the metadata calculation, such as

 https://github.com/plt/racket/blob/master/collects/pkg/lib.rkt#L154

 and you should deal with the non-proof of concept method of specifying
 it in, for instance, the info file, which is now package info AND
 collect info.

 Finally, you have to deal with how to say what the name of the
 collection is, because it can't be derived only from the source,
 because we want to be able to have

 package-from-20130203.tgz

 have collections other than package-from-20130203 in it.

 Jay


 On Mon, Jun 3, 2013 at 10:37 AM, Laurent laurent.ors...@gmail.com wrote:
 Here is a patch for a proof of concept (file /collects/pkg/lib.rkt).

 The modifications are minimal as I had expected, but obviously I only have a
 very narrow view of the package system, so probably something does not work
 properly.
 In particular, I changed only for the 'dir type, and did not touch the 'link
 type.
 Since the 'github type and others use the 'dir type once the archive is
 downloaded, I think it should work as expected for those too.

 I've tested on an old 5.3.4.10--2013-05-13(96acfb0/a) [3m], but made the
 patch with the latest in git, and it still works with the old version.
 (I was about to test it with the latest nightly, but its installation
 failed, see the mailing list)

 To try it, in the collects/pkg directory:
 $ patch lib.rkt lib.rkt.patch
 $ sudo raco setup -D pkg

 Then create a dummy package:
 $ cd /tmp
 $ mkdir dummy
 $ echo '#lang racket\n(displayln It works!)'  dummy/main.rkt
 $ touch dummy/single-collect

 The file single-collect is here to signal that this package is a
 single-collection package (it's not the right way to do it, but it's a proof
 of concept).

 Install the package:
 $ raco pkg install -t dir dummy

 (You should see This is a single-collection package as the first line of
 the log)

 Then try it:
 $ racket
 (require dummy)
 It works!

 Or is this patch way too naive for some reason?

 Laurent



 On Mon, Jun 3, 2013 at 4:37 PM, Eli Barzilay e...@barzilay.org wrote:

 (I completely agree with you, so I'll take it off-line.)


 30 minutes ago, Laurent wrote:
  On Mon, Jun 3, 2013 at 2:44 PM, Eli Barzilay e...@barzilay.org wrote:
 
  Yesterday, Laurent wrote:
   On Sun, Jun 2, 2013 at 1:47 PM, Eli Barzilay e...@barzilay.org
  wrote:
  
   To clarify, because of reasons that I won't go into on the
  list,
   the actual chances of me getting this implemented (and of such
  a
   change being accepted) are pretty much in the area of slim to
   none.
  
   That's a bummer. At first sight I'd have thought that it would
  have
   just been a matter of creating a directory with the same name as
  the
   package, and then decompressing the package into that directory.
 
  At the lower level, packages use links, and raco link's default
  mode
  of work is at the collection level.  There is obviously a need for
  packages that correspond to collection roots, but I wanted to see
  the
  default being the same as the raco-link default (which fits most
  uses
  that don't come out of the current main tree).
 
  Not sure I was clear, or I don't exactly understand what you mean.
  And maybe I'm missing some important point here.
 
  But the more I think about it, the less I see why this
  double-directory thing should be the default behavior of the new package
  system (I can understand why it's useful for plt packages though).
  Even Carl's mischief package doesn't make use of collections like
  that.
  Jen's this-and-that does, though, but then every collection of his
  package must have a name that is different from any existing collection.
  (And actually I think that's a bad idea, and that he should split his
  package into several single-collection packages −unless they are
  tightly related− because I don't want to install all his collections if
  I want
  to only try one of them, unless there is a way to install only one
  collection of an existing package?)
 
  For example, my personal racket project tree is arranged somewhat like
  that:
  racket
  ├── project-B
  │   └── main.rkt
  ├── project-A
  │   └── main.rkt
  ├── not-yet(?)a-project-C
  ├── my-collects
  │   ├── slideshow
  │   ├── scribble
  │   ├── racket
  │   └── data
  └── misc
  ├── try-this.rkt
  └── try-that.rkt
 
  project-A and project-B are git clones.
  my-collects contains all my extensions to racket's collections,
  plus some more generally useful collections.
 
  For my personal use, I just do:
raco link project-A project-B my-collects
  and all is fine.
  I tend to believe it's a quite common tree for racket users.
 

Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-03 Thread Jay McCarthy
You would need to update the metadata calculation, such as

https://github.com/plt/racket/blob/master/collects/pkg/lib.rkt#L154

and you should deal with the non-proof of concept method of specifying
it in, for instance, the info file, which is now package info AND
collect info.

Finally, you have to deal with how to say what the name of the
collection is, because it can't be derived only from the source,
because we want to be able to have

package-from-20130203.tgz

have collections other than package-from-20130203 in it.

Jay


On Mon, Jun 3, 2013 at 10:37 AM, Laurent laurent.ors...@gmail.com wrote:
 Here is a patch for a proof of concept (file /collects/pkg/lib.rkt).

 The modifications are minimal as I had expected, but obviously I only have a
 very narrow view of the package system, so probably something does not work
 properly.
 In particular, I changed only for the 'dir type, and did not touch the 'link
 type.
 Since the 'github type and others use the 'dir type once the archive is
 downloaded, I think it should work as expected for those too.

 I've tested on an old 5.3.4.10--2013-05-13(96acfb0/a) [3m], but made the
 patch with the latest in git, and it still works with the old version.
 (I was about to test it with the latest nightly, but its installation
 failed, see the mailing list)

 To try it, in the collects/pkg directory:
 $ patch lib.rkt lib.rkt.patch
 $ sudo raco setup -D pkg

 Then create a dummy package:
 $ cd /tmp
 $ mkdir dummy
 $ echo '#lang racket\n(displayln It works!)'  dummy/main.rkt
 $ touch dummy/single-collect

 The file single-collect is here to signal that this package is a
 single-collection package (it's not the right way to do it, but it's a proof
 of concept).

 Install the package:
 $ raco pkg install -t dir dummy

 (You should see This is a single-collection package as the first line of
 the log)

 Then try it:
 $ racket
 (require dummy)
 It works!

 Or is this patch way too naive for some reason?

 Laurent



 On Mon, Jun 3, 2013 at 4:37 PM, Eli Barzilay e...@barzilay.org wrote:

 (I completely agree with you, so I'll take it off-line.)


 30 minutes ago, Laurent wrote:
  On Mon, Jun 3, 2013 at 2:44 PM, Eli Barzilay e...@barzilay.org wrote:
 
  Yesterday, Laurent wrote:
   On Sun, Jun 2, 2013 at 1:47 PM, Eli Barzilay e...@barzilay.org
  wrote:
  
   To clarify, because of reasons that I won't go into on the
  list,
   the actual chances of me getting this implemented (and of such
  a
   change being accepted) are pretty much in the area of slim to
   none.
  
   That's a bummer. At first sight I'd have thought that it would
  have
   just been a matter of creating a directory with the same name as
  the
   package, and then decompressing the package into that directory.
 
  At the lower level, packages use links, and raco link's default
  mode
  of work is at the collection level.  There is obviously a need for
  packages that correspond to collection roots, but I wanted to see
  the
  default being the same as the raco-link default (which fits most
  uses
  that don't come out of the current main tree).
 
  Not sure I was clear, or I don't exactly understand what you mean.
  And maybe I'm missing some important point here.
 
  But the more I think about it, the less I see why this
  double-directory thing should be the default behavior of the new package
  system (I can understand why it's useful for plt packages though).
  Even Carl's mischief package doesn't make use of collections like
  that.
  Jen's this-and-that does, though, but then every collection of his
  package must have a name that is different from any existing collection.
  (And actually I think that's a bad idea, and that he should split his
  package into several single-collection packages −unless they are
  tightly related− because I don't want to install all his collections if
  I want
  to only try one of them, unless there is a way to install only one
  collection of an existing package?)
 
  For example, my personal racket project tree is arranged somewhat like
  that:
  racket
  ├── project-B
  │   └── main.rkt
  ├── project-A
  │   └── main.rkt
  ├── not-yet(?)a-project-C
  ├── my-collects
  │   ├── slideshow
  │   ├── scribble
  │   ├── racket
  │   └── data
  └── misc
  ├── try-this.rkt
  └── try-that.rkt
 
  project-A and project-B are git clones.
  my-collects contains all my extensions to racket's collections,
  plus some more generally useful collections.
 
  For my personal use, I just do:
raco link project-A project-B my-collects
  and all is fine.
  I tend to believe it's a quite common tree for racket users.
 
  In my-collects, I have names that would collide with racket's
  collections,
  so instead of having:
  (require orseau/racket/slideshow)
  as in the former package system, I would now have
  (require some-funny-name/slideshow)
  after renaming my-collects to some-funny-name to make 

Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-03 Thread Matthew Flatt
At Mon, 3 Jun 2013 18:37:43 +0200, Laurent wrote:
 Here is a patch for a proof of concept (file /collects/pkg/lib.rkt).

 The modifications are minimal as I had expected, but obviously I only have
 a very narrow view of the package system, so probably something does not
 work properly.
 In particular, I changed only for the 'dir type, and did not touch the
 'link type.
 Since the 'github type and others use the 'dir type once the archive is
 downloaded, I think it should work as expected for those too.

Thanks --- I appreciate your willingness to contribute!

Speaking for myself, I'd like to see a full implementation rather than
a proof of concept. In this case, I've implemented the idea once
before, and so I know it's basically possible. But I didn't like how
the details worked out in my approach, so I'm hoping that someone will
find a better approach to the details. Unfortunately, that really
requires getting into all the details.

I'll echo Jay's observation that this patch's implementation leaves a
single-collection package's installation without an info.rkt file,
which is needed for information like package-level dependencies.

The simplification of not handling links also seems like a big weakness
to me. I think one of the best things about Jay's design is that I
don't have to change the way I think about a package when moving
between link and non-link modes (which makes it much easier to work
with a GitHub repository, for example). I particularly encourage you to
think about what should happen when a user needs to transition a GitHub
repo from a representing a single-collection package to a
multiple-collection package --- and how that interacts with links as
well as non-link, GitHub-based installations of the package.


Meanwhile, as we've started leaning on the package system and looked at
how to fit our current collections into it, I think we're running into
the same problem at another level: if web-server-lib and
web-server-doc are both packages in a single web-server repository,
and if they have to be separate directories, then we have yet another
layer of directories that affect repository layout. Maybe there's a
more general answer to the directory-layout problem that would neatly
cover single-collection packages.

_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-03 Thread Greg Hendershott
 and you should deal with the non-proof of concept method of specifying
 it in, for instance, the info file, which is now package info AND
 collect info.

 Finally, you have to deal with how to say what the name of the
 collection is, because it can't be derived only from the source,

Perhaps could handle both with one info.rkt item:

;; info.rkt
#lang setup/infotab
(define single-collection-package collection-name)
;; I am a single collection package and I am named this.
;; Else if absent assume status quo subdir collection(s)

Speaking of which, it's not just the extra subdir. The existence of
two info.rkt files was confusing (to me) at first, although it made
sense in hindsight.

Now that I'm accustomed it to the extra subdir isn't a huge annoyance,
but I do find myself feeling like Sheldon Cooper on TBBT whenever I
type triples like collect/collect/collect.rkt.
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-02 Thread Laurent
Ah, that's cool. Looking forward to it!
And the in-url thing would be useful indeed for gists for example.

Laurent


On Thu, May 30, 2013 at 8:32 PM, Eli Barzilay e...@barzilay.org wrote:

 Yes, I really want to try and get to look into doing this.  The thing
 is that multi-collection libraries are going to be a common case for
 plt packages (which will get pulled out from the main repository), but
 the single-collection ones are going to be the much more popular case
 elsewhere.

 (And I also still dream on doing some `url-in' to require a single
 file, since that's also an important mode of quick sharing.  But
 that's another story.)


 10 minutes ago, Jay McCarthy wrote:
  The Racket package system doesn't support packages that aren't
  collection roots. Eli has said that he wants to implement such a
  feature, but it is not available today.
 
  Jay
 
  On Thu, May 30, 2013 at 8:29 AM, Laurent laurent.ors...@gmail.com
 wrote:
   I'm willing to upgrade my packages for the new PLaneT system, but one
 thing
   that pushes me back is the fact that I need to create a directory for
 my
   package and a subdirectory for the collection(s).
  
   But most of my packages are (and will probably be) single-collection
   packages, and it hurts my logic to have to modify my directory tree
 for that
   purpose.
  
   I know there have been several discussions already about this, but
 perhaps
   there is a simple fix:
   Maybe there could be a `single-collection?' entry in the info.rkt file
   which, when true, would tell `raco pkg install' to create itself a
 directory
   for the package that has the same name as the collection it contains?

 --
   ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
 http://barzilay.org/   Maze is Life!

_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-02 Thread Eli Barzilay
To clarify, because of reasons that I won't go into on the list, the
actual chances of me getting this implemented (and of such a change
being accepted) are pretty much in the area of slim to none.  TBH,
the chances of implementing an `in-url' are low too, but they're
probably higher than going into the package system.


40 minutes ago, Laurent wrote:
 Ah, that's cool. Looking forward to it!
 And the in-url thing would be useful indeed for gists for example.
 
 Laurent
 
 On Thu, May 30, 2013 at 8:32 PM, Eli Barzilay e...@barzilay.org wrote:
 
 Yes, I really want to try and get to look into doing this.  The
 thing is that multi-collection libraries are going to be a
 common case for plt packages (which will get pulled out from the
 main repository), but the single-collection ones are going to be
 the much more popular case elsewhere.

 (And I also still dream on doing some `url-in' to require a
 single file, since that's also an important mode of quick
 sharing.  But that's another story.)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!

_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-02 Thread Laurent
On Sun, Jun 2, 2013 at 1:47 PM, Eli Barzilay e...@barzilay.org wrote:

 To clarify, because of reasons that I won't go into on the list, the
 actual chances of me getting this implemented (and of such a change
 being accepted) are pretty much in the area of slim to none.


That's a bummer. At first sight I'd have thought that it would have just
been a matter of creating a directory with the same name as the package,
and then decompressing the package into that directory.

But I guess things are always way more complicated than at first sight...


 TBH,
 the chances of implementing an `in-url' are low too, but they're
 probably higher than going into the package system.


That would be nice, but I'd much much prefer to see the single-collection
package thing implemented.

Laurent
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


[racket-dev] PLaneT(2): Single vs multi-collection packages

2013-05-30 Thread Laurent
I'm willing to upgrade my packages for the new PLaneT system, but one thing
that pushes me back is the fact that I need to create a directory for my
package and a subdirectory for the collection(s).

But most of my packages are (and will probably be) single-collection
packages, and it hurts my logic to have to modify my directory tree for
that purpose.

I know there have been several discussions already about this, but perhaps
there is a simple fix:
Maybe there could be a `single-collection?' entry in the info.rkt file
which, when true, would tell `raco pkg install' to create itself a
directory for the package that has the same name as the collection it
contains?

Laurent
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-05-30 Thread Jay McCarthy
The Racket package system doesn't support packages that aren't
collection roots. Eli has said that he wants to implement such a
feature, but it is not available today.

Jay

On Thu, May 30, 2013 at 8:29 AM, Laurent laurent.ors...@gmail.com wrote:
 I'm willing to upgrade my packages for the new PLaneT system, but one thing
 that pushes me back is the fact that I need to create a directory for my
 package and a subdirectory for the collection(s).

 But most of my packages are (and will probably be) single-collection
 packages, and it hurts my logic to have to modify my directory tree for that
 purpose.

 I know there have been several discussions already about this, but perhaps
 there is a simple fix:
 Maybe there could be a `single-collection?' entry in the info.rkt file
 which, when true, would tell `raco pkg install' to create itself a directory
 for the package that has the same name as the collection it contains?

 Laurent


 _
   Racket Developers list:
   http://lists.racket-lang.org/dev




-- 
Jay McCarthy j...@cs.byu.edu
Assistant Professor / Brigham Young University
http://faculty.cs.byu.edu/~jay

The glory of God is Intelligence - DC 93
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-05-30 Thread Eli Barzilay
Yes, I really want to try and get to look into doing this.  The thing
is that multi-collection libraries are going to be a common case for
plt packages (which will get pulled out from the main repository), but
the single-collection ones are going to be the much more popular case
elsewhere.

(And I also still dream on doing some `url-in' to require a single
file, since that's also an important mode of quick sharing.  But
that's another story.)


10 minutes ago, Jay McCarthy wrote:
 The Racket package system doesn't support packages that aren't
 collection roots. Eli has said that he wants to implement such a
 feature, but it is not available today.
 
 Jay
 
 On Thu, May 30, 2013 at 8:29 AM, Laurent laurent.ors...@gmail.com wrote:
  I'm willing to upgrade my packages for the new PLaneT system, but one thing
  that pushes me back is the fact that I need to create a directory for my
  package and a subdirectory for the collection(s).
 
  But most of my packages are (and will probably be) single-collection
  packages, and it hurts my logic to have to modify my directory tree for that
  purpose.
 
  I know there have been several discussions already about this, but perhaps
  there is a simple fix:
  Maybe there could be a `single-collection?' entry in the info.rkt file
  which, when true, would tell `raco pkg install' to create itself a directory
  for the package that has the same name as the collection it contains?

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev