Hi Tom. My thanks for the thoughtful reply.
Doubly so since complete silence would have been quite a bummer.
It's late, but let's see...
My objective is to determine whether to continue working on Perl 6,
and if so, in which of the multitude of possible directions to push.
What is most likely to attract good developers to Perl 6 implementation?
I know of two perl cases which can be used to help test hypotheses.
Pugs managed to attract, at its peak a few years ago, tens of active
developers. And Moose has an active community now. So what might make
Perl 6 development more like Moose or Pugs? Or like the many healthy
similar projects in other languages.
Perl is increasingly unlike other popular languages. It is singular among
the top 10+ most used languages in still having only a single
implementation.
It is unusual in not having both JVM and CLR backends, usually with at least
one being production hardened. Most of the others also have lots of little
experimental projects - pyjamas, etc. Most of the others have JITs. Both
ruby
and python are managing transitions to non-backwardly compatible revisions.
I'm not seeing anything remotely like this level of activity in perl.
And almost everything I do hear of, seems to occur in the context of Perl 6.
Perhaps I'm just not reading the right blogs. The perl community's level of
investment in Perl 6 seems comparable to that which other communities
put into
their 'large but non-mainstream' projects. That is, not much. A handful of
people.
Hmm. Perhaps that's not quite true. Ruby's YARV wasn't perhaps that much
larger at its core, though community support was broader, and it was more
tightly integrated with the main branch language. But it was also a much
more modest endeavor of course. And they had the p6 parrot experience as a
cautionary tale, and so tried to avoid sanctioning a particular
implementation too early.
I was struck by "A language intended to supplant an already highly useful
language". Supplant? Some form of p6 currently seems the only likely path
by which perl5 will end up on other vm's, no? Not for any inherent reason.
The p5 opcodes are in better shape now than they've ever been. One could
certainly imagine doing a javascript implementation of the p5 vm for
instance. But, I've not seen signs of anything like that happening. It's
other languages which are supplanting p5, not p6. P6 seems p5's last
chance at non-death.
My impression of perl5's status is one of 'our engines have flamed out,
we've gone ballistic, and we don't have much in the way of wings'. A few
languages have managed to be multi-decade zombies, COBOL, Common Lisp,
Prolog. But I don't think p5 is different enough from its competitors to
achieve living deadness. Moose provides a bit of wings, but not enough to
change the picture. Perl seems to be living off its installed base for
now, which will get burned through in just a few years.
So a first strawman opportunity might be to address the impression that
"p6 is
a completely different language than p5 - p5 code won't run on anything p6".
Replacing it with "p6 is the mechanism by which the perl community is
attempting
to do what every other major language has already managed - to grow
beyond its
original C implementation." Without much success to date, because that C
implementation is rather unmaintainable, preventing incremental change, and
hitting that, a really enormous change was attempted. And except around
pugs,
there hasn't been the community support needed to carry it through. That
"p6" and "parrot" are closely associated in peoples' minds hasn't helped.
You suggest a couple of good issues to consider - slow, throwaway, and
agile.
Rakudo/parrot is indeed currently slow. My policy is to generally avoid
commenting on parrot. I note however that at least some parrot folk consider
it an open question whether parrot will eventually become acceptably fast. I
pursued elf because, with pugs in mothballs, I didn't want the perl6 egg to
depend on a parrot basket.
Elf has two backends, a perl5 and a common lisp one. In development, elfp5
generally self-compiles in about 10 seconds (though it takes 3 self-compiles
to bootstrap). Fibonacci on elfp5 runs several times slower than on p5, and
elfp5 sacrifices p6 correctness for speed. Speed which future compiler
smarts may or may not permit improvement on, or even retention if/when
correctness becomes more of a priority. Nonetheless, it's currently faster
than parrot at what it does (parrot has been focused more on language
coverage and correctness than on speed). ElfCL is closer to spec. Being a
real compiler, it can afford to be. Though compilation takes longer. Fib in
naive CL is 10x faster than p5. With p6 boxing, multimethod arithmetic, etc,
elfcl runs fib several times slower than p5. But note that creates a story
for eventually running faster than p5. If the compiler notices '+' hasn't
been lexically rebound, removes some .Bool calls, and unboxes some integers
(which admittedly may take type declarations), elfcl is back to being faster
than p5. Which CL generally is. Regexps included.
I don't know how fast STD/mildew/smop is.
A hypothetical STD/viv/???/Moose implementation would of course take more
time to compile than p5. The perl .pmc mechanism could be used (as Moose
can?) to separate compiletime from runtime. The resulting code, if one
tolerated some non-specness, could be roughly Moose speed. Which is somewhat
slower than raw p5, but is obviously something people are happy with. So a
STD/viv/???/Moose solution seems potentially viable in many of the places a
Moose solution is.
So yes, slow is an issue. Perl 6 in general will tend to have a longer
compile time, but potentially much faster runtimes. Especially when one
pragma's away full p6 support to permit a tight mapping between p6 and the
underlying system.
So another strawman opportunity is applications that require some subset of
perl running faster than p5. For example, regexps on CL. And maybe on
recent javascript (webkit). Anything best written using multimethods.
Things with a lot of functions which can be compiled together. Things which
jit does well on. Doing all this with p6 code would be easiest, but one
could hypothetically do it with p5, wrapped in p6 or not. The gotcha is many
such needs are addressed in a p5 context by using a wrapped C library. Aside
from regexps, I'm not sure what possibilities avoid this filter. Thoughts?
Ok, so that's speed.
Concern for throwaway-ness seems to decompose into two issues: "code churn
will make my code's existence transient"; and "the big picture will
evolve in a
way such that no one ever cares that my code was written". With perception
being more important than reality.
Current Moose and past pugs suggest churn doesn't discourage people from
contributing. Which is critical for elf, because the p6 compiler one
writes in any existing language, including our partial p6 implementations,
is not the p6 compiler you would write, in p6, given a working full p6
implementation. Everything about elf is throwaway. It's a bootstrap tool.
Any large project written in p6 now is almost by definition throw away code.
If you don't mind the "I'd have architected this differently if I had full
p6", then throwing away code seems to only matter to the extent one has to
waste time getting it to run, fighting implementation bugs. All the other
effort, much of development cost, isn't thrown away. If you instead do care,
for instance, "I can't write the p6 compiler I'd like to write on top of elf
or rakudo, so I'm not going to do anything", then yes, one has a problem.
Hmm. But with pugs, where it was clear there would be much churn, there was
also rapid activity and progress, which might place the churn in a different
perspective. And I don't know either how much actual Moose churn has
happened, nor how people perceive it. With rakudo, there _did_ seem to be a
"this won't churn" perception, which I've never understood, but which seemed
potentially motivating. So... I dunno.
The concern that 'no one will ever care' is more problematic. The belief
that rakudo/parrot will become a/the production p6 has motivated its
developers, discouraged others from working on other things. And the belief
that it won't, has discouraged others from working on it. For p6 in general,
there being neither a project manager, nor broad community involvement,
there
is no "sanctioned" implementation effort. So all efforts have the 'this may
dead end and no one will ever care' burden. The Perl 6 language itself has
it. Hmm. *Perl* has it. Manifesting as the large number of good people
leaving perl for other languages.
I had hoped the story for elf would end up "The test suite is correct for
Perl 6 as currently defined. Here is elf. The parser correctly parses the
test suite. That gives you a solid place to stand. Elf is usably fast for
development. It's written in p6. It bootstraps. It has a backend(s) with
the clear potential to run full p6 fast. Christmas will be here as soon as
you write enough p6 to make it so".
Getting the correct parse part turned out to be problematic. At least given
my declining motivation in the absence of community support.
The elf xmas story attempted to address the 'no one will ever care' by
removing project uncertainty, providing a clear development path which could
be clearly traveled with existing tools.
Other ways to address it, include incremental improvements on top of p5.
One can view them as backports, or instead as the small steps which would
have happened years ago if perl core bugs hadn't prevented the incremental
development of p6.
I'm unsure what other options exist. Thoughts?
So that's throwaway-ness.
For agile, there are also several components. There's "it works, and it's
easy to make progress". That's true for elf, and may even be true for
rakudo now, as one can now extend it with p6. But there's also "and I can
use the improvements in production". That's Moose, but nothing p6.
The first is advertising. The second requires code.
STD, the engine of which is written in p5, could be CPANed. So one could
do p6-flavor Moose. And write superficially p6 code in p5. There's always
the issue of the perl core being crufty, but Moose is managing that now.
Other code, let's see... There's an old but fast regex engine which might
be CPAN-ified, but it sat for several years without anyone caring, so it's
not clear anyone would now. I'm not sure what other candidates exist.
P6ish methods on autoboxed objects perhaps.
My understanding is efforts to do p6 on perl5 have always bogged down on
perl5 core bugs. But Moose has managed. So maybe past efforts have bogged
down only because they had very small teams, which would get frustrated and
punt. So... there might be something here. A p6ish standard library of
classes might be done in the context of the existing Moose community.
Thoughts?
So that's agile.
And I'll end this very long note there.
My thanks again for your thoughts. I appreciate it.
Mitchell
_______________________________________________
Boston-pm mailing list
[email protected]
http://mail.pm.org/mailman/listinfo/boston-pm
