Re: How to have Perl 5 and Perl 6 cohabitate nicely on CPAN
On Wed, Apr 14, 2010 at 09:59:32PM -0400, Jesse Vincent wrote: On Wed, Apr 14, 2010 at 09:53:45PM -0400, David Golden wrote: On Wed, Apr 14, 2010 at 8:53 PM, Adam Kennedy a...@ali.as wrote: I've been meaning to propose for a while that for CPAN Perl 5 we drop support for everything other than tar.gz (we can talk about the need for zip files later) and encourage a switch to something like D/DA/DAGOLDEN/Foo-Bar-1.23.cpan +1 (reserving the option to bikeshed about the extension) -1 Forcing an extension makes all sorts of tools that previously just worked with a tarball suddenly start to fail. -1 It's also the wrong place to encode version information. Tim.
Re: How to have Perl 5 and Perl 6 cohabitate nicely on CPAN
On Thu, Apr 15, 2010 at 10:14 AM, Tim Bunce tim.bu...@pobox.com wrote: On Wed, Apr 14, 2010 at 09:59:32PM -0400, Jesse Vincent wrote: Forcing an extension makes all sorts of tools that previously just worked with a tarball suddenly start to fail. We've dealt with it in the past. However, I'm inclined to think that going that way means an entirely new client that is designed for .cpan files. This is more akin to PPM, except non binary. The question in my mind is what this gains us beyond double-clickability for Windows. (Which is a non-trivial benefit -- e.g. see a *.cpan distribution on a web page, click it to install in Strawberry Perl). It might let us rethink some of the painful conventions of distribution tarballs today. It's also the wrong place to encode version information. Putting version info in the tarball name was necessary to avoid duplicate uploads, I suspect. It *is* a terrible place to be encoding version information, I agree. David
Re: How to have Perl 5 and Perl 6 cohabitate nicely on CPAN
This has already existed for about a year or more. See pip's .p5i file (which I think cpanminus also supports). Adam K On Fri, Apr 16, 2010 at 12:56 AM, Curtis Jewell lists.perl.cpan-work...@csjewell.fastmail.us wrote: Why don't we make the .cpan/.c6pan file (.c6pan) a JSON file, generated by PAUSE once the tarball is uploaded, with whatever information is required (a reference to the tarball, what version of perl is required, etc.) in order, when that file is passed to a CPAN client, to download and install the tarball. I don't see why a double-clickable file (or what-have-you equivalent) necessarily has to be the distribution tarball? That way, the tarball can keep its extension... best of both worlds yet?
Re: How to have Perl 5 and Perl 6 cohabitate nicely on CPAN
Larry points out that the shibboleth for Perl 6 code is: : No package statements. Instead, you'll see module statements. That ought to be a pretty reasonable way for the indexer to know it's Perl 6 code in the absence of other markers. I do think that xdg's plan to have an at-upload-time flag for hey, this is perl 6 which tells PAUSE not to even _try_ looking at the tarball as a Perl 5 module makes tons of sense. And I really like having Perl 6 dists split out into /perl6/ inside an author directory. It keeps the author tree together as one coherent community while making it really easy for people browsing the hierarchy to understand what's what. -Jesse
Re: How to have Perl 5 and Perl 6 cohabitate nicely on CPAN
# from Jesse Vincent # on Thursday 15 April 2010 09:24: -1 ... Forcing an extension makes all sorts of tools that previously just worked with a tarball suddenly start to fail. Which tools are going to just work on a perl 6 tarball? If it's a generic tar tool, it should still just work regardless of the extension. If it's assuming a perl5 dist, it is going to fail in amazing ways. Then we'll have perl6 authors getting spurious mail from bots about how their code doesn't run in perl5? # from Tim Bunce -1 ... It's also the wrong place to encode version information. But perl6 is not a version of perl5. Larry points out that the shibboleth for Perl 6 code is: : No package statements. Instead, you'll see module statements. That requires examining the contents and not just the path, and doesn't solve the naming conflict on the CPAN filesystem. And I really like having Perl 6 dists split out into /perl6/ inside an author directory. I really dislike this. Again, tools that assume perl5 dists are going to suddenly be assaulted by perl6 dists and it is going to leave us with a kludgy mess. Anything that uses `find -name '*.tar.gz'` will need special rules to skip perl6 directories. It keeps the author tree together as one coherent community while making it really easy for people browsing the hierarchy to understand what's what. I'm all for that, but if we're not going to have a replica of the existing tree rooted at /perl6/, then the next best thing would be to have a new file extension. It's easy enough to add a file extension to existing code which can handle perl6 dists, requires no immediate plumbing on PAUSE, and it solves the filename conflict between an author's perl5/perl6 dists e.g. E/EW/EWILHELM/Foo-Bar-v1.2.0.tar.gz | E/EW/EWILHELM/Foo-Bar-v1.2.0.p6d. If we're going to decide that a naming convention is the quick-fix place to start, let's use one that isn't going to cause trouble for existing tools. The file extension convention has the benefit of also giving meaning to the filename outside of the CPAN tree, which opens the possibility of having smarter tools later. --Eric -- If you only know how to use a hammer, every problem begins to look like a nail. --Richard B. Johnson --- http://scratchcomputing.com ---
Re: How to have Perl 5 and Perl 6 cohabitate nicely on CPAN
On Thu, Apr 15, 2010 at 08:56:04AM -0600, Curtis Jewell wrote: wrote: On Thu, Apr 15, 2010 at 10:14 AM, Tim Bunce tim.bu...@pobox.com wrote: On Wed, Apr 14, 2010 at 09:59:32PM -0400, Jesse Vincent wrote: Forcing an extension makes all sorts of tools that previously just worked with a tarball suddenly start to fail. We've dealt with it in the past. However, I'm inclined to think that going that way means an entirely new client that is designed for .cpan files. This is more akin to PPM, except non binary. The question in my mind is what this gains us beyond double-clickability for Windows. (Which is a non-trivial benefit -- e.g. see a *.cpan distribution on a web page, click it to install in Strawberry Perl). It might let us rethink some of the painful conventions of distribution tarballs today. (Idea suddenly came to mind. If not workable, just take apart and reassemble until it does work.) Why don't we make the .cpan/.c6pan file (.c6pan) a JSON file, generated by PAUSE once the tarball is uploaded, with whatever information is required (a reference to the tarball, what version of perl is required, etc.) in order, when that file is passed to a CPAN client, to download and install the tarball. This is essentially how PPM works. The PPD file points to a tarball, via an extension to the OSD file format. It would be easier to generate an installation configuration file, supported by a new installer that can cope with the double-click install of several OSes these days, than changing the package archive format. Changing PAUSE/CPAN to require a new package archive is likely to have a detrimental effect on Perl as a whole, as older installers will then be unable to install anything new. Upgrading is not always easy for some organisations (I know of one still using 5.6.1!). Mind you, if it means adding even more files to the filesystem, we come back to the discussion about rsync :( Cheers, Barbie. -- Birmingham Perl Mongers http://birmingham.pm.org Memoirs Of A Roadie http://barbie.missbarbell.co.uk CPAN Testers Blog http://blog.cpantesters.org YAPC Conference Surveys http://yapc-surveys.org
RE: How to have Perl 5 and Perl 6 cohabitate nicely on CPAN
On Thu, 15 Apr 2010, Barbie wrote: On Thu, Apr 15, 2010 at 08:56:04AM -0600, Curtis Jewell wrote: Why don't we make the .cpan/.c6pan file (.c6pan) a JSON file, generated by PAUSE once the tarball is uploaded, with whatever information is required (a reference to the tarball, what version of perl is required, etc.) in order, when that file is passed to a CPAN client, to download and install the tarball. This is essentially how PPM works. The PPD file points to a tarball, via an extension to the OSD file format. It would be easier to generate an installation configuration file, supported by a new installer that can cope with the double-click install of several OSes these days, than changing the package archive format. PPM4 actually supports putting the PPD file inside the tarball. When the tarball is downloaded with a .ppmx extension, then the client knows to extract the .ppd from inside the file and install everything else according to this .ppd file. The big problem with this scheme is though that is doesn't allow you to install prerequisites using this mechanism: You can double-click on a .ppmx file in a web browser and it can be downloaded and installed automatically. The ppm installer however no longer knows where the file came from, so it cannot look for prerequisites in the same place. This places the burden of installing prerequisites on the user, making the mechanism less useful than originally expected. One way around this would be to embed the file location inside the tarball itself, but that doesn't work if the tarballs may be mirrored to multiple locations. It would be nice if browsers would include the download URL in an alternate data stream (where possible, e.g. NTFS), but it looks like they only store the Internet.Zone information... Cheers, -Jan
RE: How to have Perl 5 and Perl 6 cohabitate nicely on CPAN
On Thu, 15 Apr 2010, Barbie wrote: On Thu, Apr 15, 2010 at 12:50:29PM -0700, Jan Dubois wrote: PPM4 actually supports putting the PPD file inside the tarball. When the tarball is downloaded with a .ppmx extension, then the client knows to extract the .ppd from inside the file and install everything else according to this .ppd file. The big problem with this scheme is though that is doesn't allow you to install prerequisites using this mechanism: You can double-click on a .ppmx file in a web browser and it can be downloaded and installed automatically. The ppm installer however no longer knows where the file came from, so it cannot look for prerequisites in the same place. The last time I used PPM it was able to support multiple repositories, in the same way apt or yum does. I also thought it was able to install prerequisites from these if a local ppd wasn't found. I'd certainly like to see a double-click installer work like that :) Yes, it certainly downloads prerequisites from the configured repositories. I was talking about downloading .ppmx files from some random website that you didn't configure first as a repository in the PPM client. The double- clicked file is not able to download additional prerequisites from that random website it came from itself. Think private/internal repository of a bunch of interdependent modules. Once you have to configure the web site as a repo there isn't much advantage of installing modules via the browser compare to installing them with the PPM client: ppm repo add random http://random.example.com/repo ppm install my-random-module Cheers, -Jan
Re: How to have Perl 5 and Perl 6 cohabitate nicely on CPAN
I'm pretty sure that's true for all such systems, including things like .war files. Even if we split, say, Strawberry from ActivePerl, what if you have multiple installations of ActivePerl? Adam K On Fri, Apr 16, 2010 at 8:56 AM, Jan Dubois j...@activestate.com wrote: On Thu, 15 Apr 2010, Barbie wrote: On Thu, Apr 15, 2010 at 02:58:12PM -0700, Jan Dubois wrote: Yes, it certainly downloads prerequisites from the configured repositories. I was talking about downloading .ppmx files from some random website that you didn't configure first as a repository in the PPM client. The double- clicked file is not able to download additional prerequisites from that random website it came from itself. Think private/internal repository of a bunch of interdependent modules. Got it. However, I was thinking the initial file (ppmx) would essentially be a stand-alone download, the installer would then revert to the predefined repos to download the prerequisites. Yes, that's how it works. In PPM's case there are only a few repos that try to keep up with CPAN, so knowing where the original repo was may be significant. This is not important for modules on CPAN. It is important for modules whose prerequisites are *not* on CPAN and only exist in the same location as the original ppmx file. The way around this is probably to implement a protocol handler like ppm://random.example.com/repo/my-random-module.ppmx That way the browser would invoke the protocol handler with the full URL ppm install ppm://random.example.com/repo/my-random-module.ppmx instead of downloading the tarball via HTTP itself and then just passing a local filename to the installer: ppm install /tmp/downloads/my-random-module.ppmx Implementing a protocol handler for the major operating systems is somewhat more involved than just setting up a file extension/MIME type mapping though. However, if we're downloading from CPAN, then it would be reasonable to try and grab prereqs from a predefined set of CPAN mirrors. This all is not an issue if the extension mechanism only has to work for CPAN and can ignore the DarkPAN. One other issue that has been ignored so far is that neither the file extension nor the protocol scheme will work particular well if you have multiple Perl installations on your machine. You cannot direct the automatic download/install to a particular instance. Cheers, -Jan
Re: How to have Perl 5 and Perl 6 cohabitate nicely on CPAN
--- On Wed, 14/4/10, Tim Bunce tim.bu...@pobox.com wrote: No need for a radio-box if PAUSE auto-detects from the META file. Tim. Agreed, but you know you're going to see this and variants thereof: requires: perl: 6,0 Should that be a rejection for unknown format? I would think so. Better to fail early than have junk spread throughout the CPAN. Cheers, Ovid -- Buy the book - http://www.oreilly.com/catalog/perlhks/ Tech blog- http://blogs.perl.org/users/ovid/ Twitter - http://twitter.com/OvidPerl Official Perl 6 Wiki - http://www.perlfoundation.org/perl6
Re: How to have Perl 5 and Perl 6 cohabitate nicely on CPAN
On Apr 14, 2010, at 6:48 AM, Tim Bunce wrote: If we separate Perl 5 and Perl 6 distributions within the file system, we can distinguish between them easily and without relying on indexes, meta data files or other elements of complexity. True, but is that *really* a significant need? Why bother? If we do not, then we need a different naming convention for the tar files. Otherwise how can we upload FooBar-1.45.tar.gz that works for perl5 and also FooBar-1.45.tar.gz that is written in perl6 The perl5 version may work with perl6, but no visa-versa. I think there needs to be some separation in the file space in order to do this. Using a directory perl6/ and controlling that via PAUSE seems a bit more reliable and consistent than expecting users to use a particular convention not to confuse files. We also thought that it might be relatively easy for to add radio-boxes to the PAUSE upload page to indicate Perl 5 or Perl 6 and to place Perl 6 distributions into the proper directory without it having to be specified explicitly. (Whether it *must* be specified as one or the other, which would require updates to Perl 5 tools like CPAN::Uploader I will defer to Andreas' judgment.) No need for a radio-box if PAUSE auto-detects from the META file. I think there are arguments both for and against. For example what to do if PAUSE cannot detect ? Graham.
Re: How to have Perl 5 and Perl 6 cohabitate nicely on CPAN
On Apr 14, 2010, at 8:55 AM, Graham Barr wrote: On Apr 14, 2010, at 6:48 AM, Tim Bunce wrote: If we separate Perl 5 and Perl 6 distributions within the file system, we can distinguish between them easily and without relying on indexes, meta data files or other elements of complexity. True, but is that *really* a significant need? Why bother? If we do not, then we need a different naming convention for the tar files. Otherwise how can we upload FooBar-1.45.tar.gz that works for perl5 and also FooBar-1.45.tar.gz that is written in perl6 The perl5 version may work with perl6, but no visa-versa. I think there needs to be some separation in the file space in order to do this. Using a directory perl6/ and controlling that via PAUSE seems a bit more reliable and consistent than expecting users to use a particular convention not to confuse files. I remember bringing this up in one of the panel discussions at OSCON following the decision to go with Perl6 as it will prove to be problematic, especially given the limitations in the namespace, and I think they will need to be very separate. I haven't kept up with P6 development and the module conventions, but I had thought this particular issue had been solved with requiring 'Perl6::' or something similar to distinguish P5 from P6 modules. Briefly looking at the P6 synopsis for CPAN ( http://perlcabal.org/syn/S22.html ) still gives me the impression that there will be a version prefix and that cpan6 is still in development, both of which still seem like very good ideas. We also thought that it might be relatively easy for to add radio-boxes to the PAUSE upload page to indicate Perl 5 or Perl 6 and to place Perl 6 distributions into the proper directory without it having to be specified explicitly. (Whether it *must* be specified as one or the other, which would require updates to Perl 5 tools like CPAN::Uploader I will defer to Andreas' judgment.) No need for a radio-box if PAUSE auto-detects from the META file. I think there are arguments both for and against. For example what to do if PAUSE cannot detect ? In a post-P6 world, I'd say if it can't be detected, it should be rejected since it will be an important, if not essential, piece of information for PAUSE. e.
Re: How to have Perl 5 and Perl 6 cohabitate nicely on CPAN
* Adam Kennedy a...@ali.as [2010-04-14T20:53:15] Alternative option, when we switch to Perl 6 also take the opportunity to switch to a specific named extension. D/DA/DAGOLDEN/Foo-Bar-1.23.c6z I haven't wanted to get into this thread, but I will say that I had the same idea as Adam, and didn't think it was crazy. -- rjbs
Re: How to have Perl 5 and Perl 6 cohabitate nicely on CPAN
On Wed, Apr 14, 2010 at 8:53 PM, Adam Kennedy a...@ali.as wrote: I've been meaning to propose for a while that for CPAN Perl 5 we drop support for everything other than tar.gz (we can talk about the need for zip files later) and encourage a switch to something like D/DA/DAGOLDEN/Foo-Bar-1.23.cpan +1 (reserving the option to bikeshed about the extension) Extending the idea to .pm6, .pl6, etc. would seem useful as well (sadly). David
How to have Perl 5 and Perl 6 cohabitate nicely on CPAN
I'm writing up my notes from an extensive IRC conversation with Jesse Vincent. I offered to post it to cpan-workers for feedback. He and I agree that there are significant benefits to the community to using a single repository for both Perl 5 and Perl 6 distributions. We worked through some differences of opinion in various ways this could be accomplished harmoniously and came up with what we think is a reasonable approach using much of the existing infrastructure. We agreed on several things: * We want to avoid bikeshed discussions of CPAN/PAUSE 2.0 * We want to use the existing PAUSE/CPAN author credentials for both Perl 5 and Perl 6 * We want to use the existing PAUSE upload infrastructure with as few changes as possible * We want to consider uploading and indexing as separate issues [1] * We want to keep the existing package index files for Perl 5 only * We want to distinguish between Perl 5 and Perl 6 distributions within the file system I will explain the last point briefly as it was the crux of our discussion. While the 02packages file is a useful index of things that are Perl 5 modules, it does not include developer distributions, outdated distributions, distributions without modules, etc. It is a subset of Perl 5 things, not complete index. Any tool that wants to look at Perl 5 things that aren't in the 02packages file is forced to look at the only other index available -- the file system. If we separate Perl 5 and Perl 6 distributions within the file system, we can distinguish between them easily and without relying on indexes, meta data files or other elements of complexity. Our proposal is for Perl 6 modules to be uploaded into a 'perl6' subdirectory of a CPAN author's directory like so: ...D/DA/DAGOLDEN/perl6/Foo-Bar-1.23.tar.gz PAUSE already supports uploading to subdirectories today, so the functionality exists NOW without any changes to PAUSE. It requires a *convention* of authors uploading their Perl 6 distributions to the right place and Jesse is looking into who is writing the Perl 6 uploader client (and will encourage them to join cpan-workers). If uploads are automated with tools, it would be a fairly simple matter to have the tools follow the convention. We also thought that it might be relatively easy for to add radio-boxes to the PAUSE upload page to indicate Perl 5 or Perl 6 and to place Perl 6 distributions into the proper directory without it having to be specified explicitly. (Whether it *must* be specified as one or the other, which would require updates to Perl 5 tools like CPAN::Uploader I will defer to Andreas' judgment.) This seemed like a very easy way to let Perl 5 and Perl 6 co-habitate nicely on CPAN, without any change to the existing infrastructure. It makes it easier (though not foolproof) for PAUSE or other indexers like the ones for CP5XXXAN or BackPAN to differentiate Perl 5 and Perl 6 distributions. [2] We considered whether we should encourage new Perl 5 uploads to be put in a 'perl5' subdirectory but decided that historical precedent was too strong and it would be a point of unnecessary controversy. [1] There will, of course, eventually be a need down the line for Perl 6 package/distribution indexing, but that is true regardless of where/how files are uploaded. [2] Indexers may still need to double check inside things in case they were put in the wrong place. -- David
Re: How to have Perl 5 and Perl 6 cohabitate nicely on CPAN
# from David Golden # on Tuesday 13 April 2010 10:48: Our proposal is for Perl 6 modules to be uploaded into a 'perl6' subdirectory of a CPAN author's directory like so: ...D/DA/DAGOLDEN/perl6/Foo-Bar-1.23.tar.gz This would require changes to any tools which think that D/DA/DAGOLDEN/*/ are all simple subdirectories with no special conventions attached to them. I would prefer that the perl6/ part of the tree be created a few levels up - e.g. perl6/authors/id/D/DA/DAGOLDEN/. Aside from being able to start uploading without changing PAUSE, what is gained by putting the perl6/ directory in this deep, and how many other things are going to have to permanently bear the complexity of having this one special directory land in the middle of the existing filesystem? Further: where (if they exist at all) will the various symlinks go for perl6 dists? I'm thinking that rooting a completely new tree at /perl6/ might also make it possible to repurpose perl5 dist tools for handling perl6 code by simply appending '/perl6' to the base mirror uri. (Assuming that we make the perl6 tree look substantially similar to the existing one.) PAUSE already supports uploading to subdirectories today, so the functionality exists NOW without any changes to PAUSE. It requires a *convention* of authors uploading their Perl 6 distributions to the right place ... ...add radio-boxes to the PAUSE upload page to indicate Perl 5 or Perl 6 and to place Perl 6 distributions into the proper directory without it having to be specified explicitly. If you're changing the upload tool and indexer, it would seem that making them auto-detect perl6-ness (via the metafile?) would be less error-prone for users. I wonder: is it possible/feasible to run a second instance of PAUSE as a perl6-mode without extensive changes? Presumably, requiring a metafile should put the indexer into a flow where it isn't trying to do anything perl5-specific, so then you just need some new tables? Or, a perl6-only fork/rewrite of it if someone is so inclined. --Eric -- software: a hypothetical exercise which happens to compile. --- http://scratchcomputing.com ---
