Hey everyone, Although I am very interested to see this happen, I have not done anything for it. However, just today I was hunting around Software Carpentry's web page and found that they are interested in a Perl backend for iPython <http://software-carpentry.org/pages/create.html> (scroll to the bottom of the list). In case you have not heard of Software Carpentry, it has become a go-to resource for grad students to learn Python. Their interest in adding Perl is unexpected, and worth contributing to, I think.
Don't get me wrong: I'd like to eventually create a stand-alone pure-Perl application. However, I think that iPython integration is both a suitable goal and a suitable intermediate step towards the pure-Perl application. Zaki, what would be the must useful thing we can do to help with your work? David On Wed, Jun 18, 2014 at 3:39 PM, Chris Marshall <[email protected]> wrote: > On Wed, Jun 18, 2014 at 3:10 PM, Zakariyya Mughal <[email protected]> > wrote: > > On 2014-06-17 at 14:30:23 -0400, Chris Marshall wrote: > >> On Tue, Jun 17, 2014 at 1:13 PM, Zakariyya Mughal < > [email protected]> wrote: > >> > You can see in the second link that the architecture is made up of two > >> > parts: the frontend (the interface the user sees) and the language > >> > kernel (what evaluates expressions, returns data, and provides > services > >> > like completion). They communicate with each other over ZeroMQ. It is > a > >> > bit unnecessarily complex, but it allows for using multiple clients > with > >> > different capabilities at the same time. > >> > >> I like the layered architecture but it would be nice if we could > >> have a base implementation without adding a complex, 3rd party > >> networking and concurrency library to the mix. Any chance of > >> a simple framework that could be extended to full 0MQ features > >> if required? > > > > Yes, this is definitely possible. My design separates the wire protocol > > from the message format, so a simpler protocol is possible for > > communicating with just a single client. > > Sounds good. Is the implementation based on a plugin type > approach? > > > Another possibility is to fix Alien::ZMQ on Win32 (either by fixing the > > compilation or work on providing something pre-compiled from > > <http://zeromq.org/distro:microsoft-windows>). Every other platform > > has packages which work. I have some experience with > > Alien packages, so the latter shouldn't be too difficult > > Removing the hard dependency for 0MQ for the basic implementation > is preferred. There are a large number of partially implemented > Alien modules that only sort of work (usually if you happen to use > the OS/platform of the developer). I've been working to update the > Alien manifesto to be more usable: > > > http://blogs.perl.org/Fusers/chris_marshall/2013/12/a-framework-for-alien-modules-the-alien2-manifesto.html > > But getting traction and agreement has been slow/difficult. Lots of > differing opinions. :-) Fixing Alien:XXX for all the libraries that PDL > builds with is a definite goal to improve portability. > > > My current design requires IO::Async which seems rather portable > > by the results on CPAN Testers. > > One thing that often happens with portability for windows platforms > is that a module sounds great and tests pass *but* if you look at > the details it is possible that many of the key features actually don't > work for windows so if an implementation requires those features, > the result is non-portable (doesn't work) to windows. Looking at the > test output it appears that the usual suspects are missing for > IO::Async: signals and fork. > > >> > What I'm working on now is the language kernel (repo here > >> > <https://github.com/zmughal/p5-Devel-IPerl>). Right now I've just > got a > >> > prompt working — I need to connect it to Devel::REPL to support > >> > evaluation and completion events. I don't want to give any estimates, > >> > but I don't think that getting to an alpha version will take long. > >> > >> Do you have to hardwire in the REPL or is there a way to just > >> provide an IN and OUT handles for any REPL? Mentioning this > >> since I'm thinking to refactor pdl2 from Devel::REPL to Reply > >> which is lighter weight and cleaner for shell type applications. > > > > No, it isn't hardwired. All I'm doing is creating a Term::ReadLine > > subclass that feeds in commands from a string, so it should be > > compatible with anything that uses ReadLine including Reply or even > > Devel::Trepan. > > Ok. I would like to have the console interface (completion, syntax > coloring, ...) be generic enough to be usable with the GUI front end > as well to avoid duplication of code. > > > As a sidenote, as far as I can tell, IPython Notebook isn't attempting > > to deal with excessive output, so you can crash the browser by executing > > an infinite loop that quickly prints out lots of data. I will probably > > try to deal with those problems later as an unresponsive REPL is > > unacceptable (this actually leads to losing code). I think it will have > > to be dealt with both on the kernel side and frontend side. Just > > thinking ahead. > > Definitely something to be avoided. --Chris > > > Cheers > > - Zaki Mughal > > > >> > >> > The main reason I'm working with IPython is because it has a spec and > >> > coding to a spec is easier than coming up with everything from > scratch. ;-) > >> > >> Definitely. > >> > >> >> As for the IDE/workbook style, maybe we could start with something > >> >> like the LCD of the current implementations: iPython, Matlab, Maple, > >> >> Mathematica, Spyder(?),... Once this interface is architected, the > >> >> next step would be to implement it. At this point, it would be > possible > >> >> to hack in an iPython version but that would probably have > portability > >> >> problems and only of use for Python + PDL users which is a step > >> >> backwards from making PDL easy to install/use. > >> >> > >> >> --Chris > >> >> > >> >> On Tue, Jun 17, 2014 at 7:05 AM, David Mertens < > [email protected]> wrote: > >> >> > Hey everyone, > >> >> > > >> >> > This and another recent conversation prompted me to dust off > >> >> > App::Prima::REPL last night. I was in the middle of a refactoring > effort > >> >> > when I left it off, so I hammered through that last night. The > only obvious > >> >> > difference between what's on Github and what's on CPAN is the > handling of > >> >> > the output window, but a more important refactorization underlies > that > >> >> > difference. I feel a lot better about it. > >> >> > > >> >> > It seems that we could pretty easily move forward on two > independent fronts. > >> >> > Perl has some very nice web frameworks, but since the iPython code > is > >> >> > already available, we could hook into iPython's framework for the > first cut > >> >> > of the web stuff. If we later want a pure-Perl solution, we could > build a > >> >> > Perl web front end that could be swapped out for the iPython one. > >> >> > > >> >> > Then we'd have to get the GUI side working. One hard part will be > the math > >> >> > typesetting, but I have a shortcut in mind that we can try using. > Another > >> >> > hard part will be changing the workflow and layout to mimic > iPython instead > >> >> > of Matlab. That will take a bit of study, and it may be better to > write a > >> >> > different GUI app (called perhaps iperl) rather than try to fold > this > >> >> > functionality into the current GUI (prima-repl). > >> > > >> > Hmm, after looking at the App::Prima::REPL code, I don't see a > >> > structured output format. I was wondering if you've considered looking > >> > at the .ipynb format for serialisation. It's just JSON and seems > >> > extensible (even though the frequent use of data: URIs feels wrong to > >> > me). Using this format means that all notebooks can be viewed online > >> > using <http://nbviewer.ipython.org/>. Unfortunately, there is no easy > >> > way to add POD formatting to that site instead of Markdown, but I can > >> > think of some workarounds. > >> > > >> > There's a schema for the JSON here: < > https://github.com/ipython/ipython/blob/master/IPython/nbformat/v3/v3.withref.json > >. > >> > > >> > Cheers, > >> > - Zaki Mughal > >> > > >> > > >> >> > > >> >> > I suspect that anybody who wants to get involved can begin by > downloading > >> >> > and digging into the notebook software. I wonder if the Julia > bindings might > >> >> > serve as a good reference? > >> >> > > >> >> > David > >> >> > > >> >> > > >> >> > On Mon, Jun 16, 2014 at 8:32 AM, Chris Marshall < > [email protected]> > >> >> > wrote: > >> >> >> > >> >> >> Very interesting discussion so far. A focus on PDL > >> >> >> development for me as release manager has been to > >> >> >> improve the portability and buildability of PDL across > >> >> >> all major perl platforms (windows, macosx, and unix/linux/bsd/*). > >> >> >> > >> >> >> We've made steady progress but once PDL is installed > >> >> >> the user might ask "Now what?". It would be nice to > >> >> >> have a clear and simple answer for that. (In addition to > >> >> >> the use case of supporting better scientific development > >> >> >> and collaboration). > >> >> >> > >> >> >> The good news is that we have two key pieces already > >> >> >> available that could be a foundation for iPDL: > >> >> >> > >> >> >> (1) Interactive PDL shells (perldl, pdl2) > >> >> >> > >> >> >> We've already made a start at integrating multiple GUI > >> >> >> toolkit event loops. Stalled for now but I think we know > >> >> >> what is needed. > >> >> >> > >> >> >> (2) Prima and Prima::OpenGL > >> >> >> > >> >> >> This gives us a baseline, *extremely* portable GUI > >> >> >> toolkit to build on. We could use other toolkits but > >> >> >> it is really difficult to beat the portability of Prima as > >> >> >> a powerful GUI for perl. In a sense it is a little known > >> >> >> super-power perl module. :-) > >> >> >> > >> >> >> NOTE: I explicitly call out Prima::OpenGL because > >> >> >> I think for high performance and portable graphics and > >> >> >> realtime visualization, OpenGL is now the default > >> >> >> standard---even including GPU compute shaders in > >> >> >> the latest version. > >> >> >> > >> >> >> I'm sure there are some other ideas but, like the PDL3 > >> >> >> development discussions, I think the best approach is > >> >> >> to KISS as much as possible. Avoiding outside toolkits > >> >> >> and libraries where possible is a win for portability, > >> >> >> especially for non-unix-ish platforms such as windows. > >> >> >> > >> >> >> --Chris > >> >> >> > >> >> >> > >> >> >> On Mon, Jun 16, 2014 at 7:10 AM, Paul Goodall > >> >> >> <[email protected]> wrote: > >> >> >> > Hi David, Craig, > >> >> >> > > >> >> >> > I’d be happy to help with this - I should have spare time in > between > >> >> >> > projects to contribute to it. > >> >> >> > > >> >> >> > Personally, I don’t think it would be a bad thing for PDL to be > more > >> >> >> > accessible to the general community. Typically when I explain > to others > >> >> >> > that I use PDL, I’m met with a blank face, prompting for an > explanation. > >> >> >> > It > >> >> >> > would be nice if PDL were to be recognised as a desirable skill > in the > >> >> >> > same > >> >> >> > way that Python is (particularly, for example, in job interview > >> >> >> > situations). > >> >> >> > It is a shame that more people don’t know about/have the power > of PDL at > >> >> >> > their fingertips :-) > >> >> >> > > >> >> >> > Paul > >> >> >> > > >> >> >> > > >> >> >> > On 13 Jun 2014, at 18:12, David Mertens < > [email protected]> > >> >> >> > wrote: > >> >> >> > > >> >> >> > Paul, > >> >> >> > > >> >> >> > To clarify, the notebooks that you mention in your link have > two key > >> >> >> > features. First, they provide online sharing, so it is very > easy to show > >> >> >> > your colleagues some ideas and calculations. Your colleagues can > >> >> >> > probably > >> >> >> > even try manipulating the data in their browser, if it's fancy > enough. > >> >> >> > Second, they provide means for (1) writing code, (2) writing > prose, (3) > >> >> >> > typesetting math, and (4) embedding media such as pictures. > They are, in > >> >> >> > essence, Mathematica clones for their respective languages. > >> >> >> > > >> >> >> > PDL does not have an equivalent to this sort of tool. I wrote a > >> >> >> > rudimentary > >> >> >> > offline GUI data analysis program called App::Prima::REPL, but > that was > >> >> >> > more > >> >> >> > targeted at the Matlab audience, not the Mathematica audience. > It was > >> >> >> > also a > >> >> >> > giant pile of spaghetti, and I got stalled partway through a > refactoring > >> >> >> > effort. It is not document focused, but rather tab focused. > There is an > >> >> >> > API > >> >> >> > for building our own custom tabs, but it's really more of a > programmer's > >> >> >> > tool, not a scientists log book. > >> >> >> > > >> >> >> > I have lately found myself doing a lot of thinking in LyX, then > >> >> >> > programming > >> >> >> > in Perl. I would really like if there was some way for me to > combine all > >> >> >> > of > >> >> >> > that into a single document, much like the notebooks that you > mention. > >> >> >> > However, my programming time has lately been dedicated to other > projects > >> >> >> > (especially, this last week, polishing off some final work on > >> >> >> > PDL::Graphics::Prima for a forthcoming release). > >> >> >> > > >> >> >> > If you are interested in helping, please let me know. I'd love > to work > >> >> >> > with > >> >> >> > somebody on this. :-) > >> >> >> > > >> >> >> > David > >> >> >> > > >> >> >> > > >> >> >> > On Fri, Jun 13, 2014 at 12:32 PM, Craig DeForest > >> >> >> > <[email protected]> > >> >> >> > wrote: > >> >> >> >> > >> >> >> >> I wouldn't say there's an online notebook viewer so much a > powerful > >> >> >> >> toolkit to build one. David Mertens recently implemented > >> >> >> >> PDL::Graphics::Prima, which is an object framework that can be > used to > >> >> >> >> construct interactive notebooks very simply and quickly. For > example, > >> >> >> >> you > >> >> >> >> can generate a plot object and connect it to a PDL, and very > easily > >> >> >> >> update > >> >> >> >> the plot as the PDL evolves - or autogenerate/autoupdate plots > as you > >> >> >> >> carry > >> >> >> >> out a calculation. > >> >> >> >> > >> >> >> >> That is sort of in keeping with the PDL "style" -- our niche > seems to > >> >> >> >> be > >> >> >> >> powerful tools that are expert-friendly, rather than polished > packages. > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> On Jun 13, 2014, at 9:27 AM, Paul Goodall > >> >> >> >> <[email protected]> wrote: > >> >> >> >> > >> >> >> >> > Hi, > >> >> >> >> > > >> >> >> >> > Apologies if this has a very obvious answer, but does PDL > have an > >> >> >> >> > equivalent to the online notebook viewers available to the > likes of > >> >> >> >> > Python, > >> >> >> >> > Ruby and (even) Julia? > >> >> >> >> > http://nbviewer.ipython.org > >> >> >> >> > > >> >> >> >> > I’d really like to make use of this ‘IPDL’ if it exists. > >> >> >> >> > > >> >> >> >> > Thanks, > >> >> >> >> > Paul > >> >> > > >> >> > > >> >> > > >> >> > > >> >> > -- > >> >> > "Debugging is twice as hard as writing the code in the first > place. > >> >> > Therefore, if you write the code as cleverly as possible, you > are, > >> >> > by definition, not smart enough to debug it." -- Brian Kernighan > -- "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." -- Brian Kernighan
_______________________________________________ Perldl mailing list [email protected] http://mailman.jach.hawaii.edu/mailman/listinfo/perldl
