When panda was first introduced as the 'adopted' module or package
manager for rakudo, a file containing the meta information for all the
modules, located at "http://ecosystem-api.p6c.org/projects.json";, was
established.
At the time, the developers said that the meta information file was
designed to be agnostic of the package manager.
Consequently, it was possible to study the ecosystem of modules by
analysing the meta information. I did this with a module citation index
to see which modules were important for the system. The most interesting
result was how JSON::Tiny was replaced over time by JSON::Fast. This is
actually quite useful for a new developer who wants to know what is the
most popular implementation without having to investigate all modules in
depth.
To be clear, a file containing the meta information about modules is not
the same as specifying where the modules should or should not be located.
Effectively, an Ecosystem began to develop for modules, and the
ecosystem is agnostic of location or package manager.
Recently, cpan6 came on line and its developers began encouraging module
developers to move modules to cpan6. (I was not aware of this migration
until recently, nor have I seen any discussion about whether such a
migration is a good idea.)
But it turned out that cpan6 requires another module meta file, (located
at
https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/cpan.json)
I discovered that that the two lists appear to be different, with
JSON::Fast moving off "projects.json" onto 'cpan.json'. This of course
broke my ModuleCitation system.
So, in order to modify the ModuleCitation system, I wanted to know how
to find whether there are more lists, and whether there would be some
transparency about the whole ecosystem.
There was a discussion on IRC perl6, but it was between only three
participants, including zef's developer, and there did not seem to be
any attempt to discuss the issue of coordination or what would be good
for the perl6 ecosystem. I was simply told that this "... was the way
things were intended", even "before perl6 was invented".
It seems to me the health and development of the perl6 module ecosystem
is a topic that should be discussed and a consensus reached about the
way things should develop. There are a number of innovations in the way
module meta information is specified (including authors and revisions)
to overcome problems faced in the past.
However, there is no consensus - from what I have discovered so far -
about how meta information about perl6 modules is shared.
It seems to me that everyone should be able to find out where all public
perl6 modules are located and how to obtain their meta data, which
includes dependencies, authors, and code locations.
Not all modules will be public, but that is a decision to be made by the
developers. Not all the modules will be located in the same location or
accessed in the same way, eg. on cpan6 or on a git repository.
However, if a developer wants the module to be made public, then there
should be a single way of doing this, not multiple ways of doing this.
If there are multiple ways of registering a module, one for each package
manager, then package manager developers will be in a privileged
position about which modules to accept as public.
It seems to me that a single canonical list kept by the perl6 community
would be the best option.
In the future, it may become necessary for health tests to be performed
on modules, for example, to detect viruses, malware, or the like. If
this does become necessary, then the community can itself decide how to
handle all modules, eg, adding a meta data attribute that a module has
passed a health test.
By "community", I mean the way in which perl6 itself is developed. It is
not a free for all, but it is not restrictive. It is simply an effort
with coordination. I am arguing here for a level of coordination about
the meta information about modules.
I am actually not fundamentally against the existence of multiple meta
files (which is the case now). It think multiple files containing
effectively the same information is less effective, and more dangerous
to the community, than a single file.
But if there are to be multiple files, at the very least, there should
be some public place (such as a note on the modules.perl6.org site)
where all the lists are themselves listed - a sort of meta meta list.
What seems to be the case at the present is that there are two lists,
which a privileged number of perl6 gurus know about, but that
information is not generally available. There is also no coordination
between the lists - that I am aware of.
What is to prevent a situation in which the meta data in each list gets
out of sync in some way? It is not unreasonable to presume that this
could lead to incompatible dependencies being created and for modules
that work on one list, not working on another.
finanalyst.
