David: I think splitting PDL out into multiple modules is a good idea, provided that it is also maintained as a bundle. To some degree this has started to happen -- there are many PDL:: modules in CPAN that are not part of the PDL distro -- but sometime in the not-too-distant future someone needs to look over the actual included modules and do some judicious trimming.
PDL has suffered from several marketing problems, the single largest of which is the difficulty of installation. For years, "something" has worked out of the box from CPAN, but that PGPLOT graphics front- end is always a bear to install, and PDL is practically (to most) useless without the ability to generate graphs. I think that is likely to get fixed RSN. LL_Array looks like a nice start at a stripped down core for PDL -- which I believe is how PDL itself got started. If the author manages to address the same generality issues that Christian and Karl and Tuomas tackled in PDL 1.x (and that led to the development of PP) then it could start to take off. PDL's great strength has been the PP core, which let PDL grow rapidly into a full data reduction environment. The PP core is also PDL's great weakness -- it is quite clever, and in fact much cleverer than it has to be given the functionality we all have settled on. There are still hooks all over for explicit threading and for computed dataflow - but I don't think computed dataflow is likely ever to be implemented. The PP core is arguably *too* clever: nobody seems to understand it fully these days, which is part of why we don't (yet) have multicore PDL. That is a storm cloud on the horizon -- while PP is fabulous it needs a major overhaul, and we don't seem to have an appropriately amazing person available to dive into it for 1-2 months. (PP overhaul might make a nice GSoC project, if someone wants to be a mentor...) Er, just my $0.02 rant for the evening... On Oct 28, 2009, at 10:09 PM, David Mertens wrote: > I don't know if anybody has pointed this out before, but on Monday > night I stumbled across Numeric::LL_Array. It seems to be a very > new project, but in principle is very similar to PDL. > > I contacted the author and asked him how his package compared with > PDL; he said he didn't know because he could never get PDL to > install. I got the impression that he was turned off from PDL by > the dependency requirement and the footprint, and after the initial > installation attempt failed he started on his own project. He seems > to be very enthusiastic about it. > > The goals of Numeric::LL_Array seem to be a bit different from PDL's > aims, though it's not clear because the author does not explicitly > state them. The module seems to focus on compact storage, small > installation footprint, no external dependencies, and quick binary > operations. Initially PDL focused on compact storage and quick > operations, but it has expanded and I would say that it now aims to > be a complete scientific numerical and analysis suite, complete with > visualization and PDL::PP for hand-optimized routines and links to > external libraries. We are not so concerned about the size of our > footprint or the external dependencies. > > Anyway, it's definitely a project worth following. If nothing else, > our docs should point to it as an alternative to PDL in case any of > our readers are looking for a smaller, simpler solution. However, I > think there's a lesson we could learn form this: We might consider > breaking PDL into small individual modules, and then create a meta- > distribution on CPAN called PDL-Full that requires all the small > packages. This way, PDL could compete head-to-head with > Numeric:LL_Array if somebody simply wanted to install the PDL-Core, > but would aim to match the full capabilities of IDL or Matlab, which > most of us see as our real 'competition.' > > Thoughts? > David > > P.S. I saw a paper comparing Numpy, PDL, hand-rolled C code, and > plain Perl and Python code for computing a numerical integral. > Plain old Python and Perl were terribly slow, but Python had two > distinct numerical libraries, Numpy and something else. I was > jealous. So I don't think it's necessarily bad that Perl has a > second numerical data processing project springing into existence. > _______________________________________________ > Perldl mailing list > [email protected] > http://mailman.jach.hawaii.edu/mailman/listinfo/perldl _______________________________________________ Perldl mailing list [email protected] http://mailman.jach.hawaii.edu/mailman/listinfo/perldl
