On Wed, Jun 18, 2014 at 3:10 PM, Zakariyya Mughal <[email protected]> wrote: > On 2014-06-17 at 14:30:23 -0400, Chris Marshall wrote: >> On Tue, Jun 17, 2014 at 1:13 PM, Zakariyya Mughal <[email protected]> >> wrote: >> > You can see in the second link that the architecture is made up of two >> > parts: the frontend (the interface the user sees) and the language >> > kernel (what evaluates expressions, returns data, and provides services >> > like completion). They communicate with each other over ZeroMQ. It is a >> > bit unnecessarily complex, but it allows for using multiple clients with >> > different capabilities at the same time. >> >> I like the layered architecture but it would be nice if we could >> have a base implementation without adding a complex, 3rd party >> networking and concurrency library to the mix. Any chance of >> a simple framework that could be extended to full 0MQ features >> if required? > > Yes, this is definitely possible. My design separates the wire protocol > from the message format, so a simpler protocol is possible for > communicating with just a single client.
Sounds good. Is the implementation based on a plugin type approach? > Another possibility is to fix Alien::ZMQ on Win32 (either by fixing the > compilation or work on providing something pre-compiled from > <http://zeromq.org/distro:microsoft-windows>). Every other platform > has packages which work. I have some experience with > Alien packages, so the latter shouldn't be too difficult Removing the hard dependency for 0MQ for the basic implementation is preferred. There are a large number of partially implemented Alien modules that only sort of work (usually if you happen to use the OS/platform of the developer). I've been working to update the Alien manifesto to be more usable: http://blogs.perl.org/Fusers/chris_marshall/2013/12/a-framework-for-alien-modules-the-alien2-manifesto.html But getting traction and agreement has been slow/difficult. Lots of differing opinions. :-) Fixing Alien:XXX for all the libraries that PDL builds with is a definite goal to improve portability. > My current design requires IO::Async which seems rather portable > by the results on CPAN Testers. One thing that often happens with portability for windows platforms is that a module sounds great and tests pass *but* if you look at the details it is possible that many of the key features actually don't work for windows so if an implementation requires those features, the result is non-portable (doesn't work) to windows. Looking at the test output it appears that the usual suspects are missing for IO::Async: signals and fork. >> > What I'm working on now is the language kernel (repo here >> > <https://github.com/zmughal/p5-Devel-IPerl>). Right now I've just got a >> > prompt working — I need to connect it to Devel::REPL to support >> > evaluation and completion events. I don't want to give any estimates, >> > but I don't think that getting to an alpha version will take long. >> >> Do you have to hardwire in the REPL or is there a way to just >> provide an IN and OUT handles for any REPL? Mentioning this >> since I'm thinking to refactor pdl2 from Devel::REPL to Reply >> which is lighter weight and cleaner for shell type applications. > > No, it isn't hardwired. All I'm doing is creating a Term::ReadLine > subclass that feeds in commands from a string, so it should be > compatible with anything that uses ReadLine including Reply or even > Devel::Trepan. Ok. I would like to have the console interface (completion, syntax coloring, ...) be generic enough to be usable with the GUI front end as well to avoid duplication of code. > As a sidenote, as far as I can tell, IPython Notebook isn't attempting > to deal with excessive output, so you can crash the browser by executing > an infinite loop that quickly prints out lots of data. I will probably > try to deal with those problems later as an unresponsive REPL is > unacceptable (this actually leads to losing code). I think it will have > to be dealt with both on the kernel side and frontend side. Just > thinking ahead. Definitely something to be avoided. --Chris > Cheers > - Zaki Mughal > >> >> > The main reason I'm working with IPython is because it has a spec and >> > coding to a spec is easier than coming up with everything from scratch. ;-) >> >> Definitely. >> >> >> As for the IDE/workbook style, maybe we could start with something >> >> like the LCD of the current implementations: iPython, Matlab, Maple, >> >> Mathematica, Spyder(?),... Once this interface is architected, the >> >> next step would be to implement it. At this point, it would be possible >> >> to hack in an iPython version but that would probably have portability >> >> problems and only of use for Python + PDL users which is a step >> >> backwards from making PDL easy to install/use. >> >> >> >> --Chris >> >> >> >> On Tue, Jun 17, 2014 at 7:05 AM, David Mertens <[email protected]> >> >> wrote: >> >> > Hey everyone, >> >> > >> >> > This and another recent conversation prompted me to dust off >> >> > App::Prima::REPL last night. I was in the middle of a refactoring effort >> >> > when I left it off, so I hammered through that last night. The only >> >> > obvious >> >> > difference between what's on Github and what's on CPAN is the handling >> >> > of >> >> > the output window, but a more important refactorization underlies that >> >> > difference. I feel a lot better about it. >> >> > >> >> > It seems that we could pretty easily move forward on two independent >> >> > fronts. >> >> > Perl has some very nice web frameworks, but since the iPython code is >> >> > already available, we could hook into iPython's framework for the first >> >> > cut >> >> > of the web stuff. If we later want a pure-Perl solution, we could build >> >> > a >> >> > Perl web front end that could be swapped out for the iPython one. >> >> > >> >> > Then we'd have to get the GUI side working. One hard part will be the >> >> > math >> >> > typesetting, but I have a shortcut in mind that we can try using. >> >> > Another >> >> > hard part will be changing the workflow and layout to mimic iPython >> >> > instead >> >> > of Matlab. That will take a bit of study, and it may be better to write >> >> > a >> >> > different GUI app (called perhaps iperl) rather than try to fold this >> >> > functionality into the current GUI (prima-repl). >> > >> > Hmm, after looking at the App::Prima::REPL code, I don't see a >> > structured output format. I was wondering if you've considered looking >> > at the .ipynb format for serialisation. It's just JSON and seems >> > extensible (even though the frequent use of data: URIs feels wrong to >> > me). Using this format means that all notebooks can be viewed online >> > using <http://nbviewer.ipython.org/>. Unfortunately, there is no easy >> > way to add POD formatting to that site instead of Markdown, but I can >> > think of some workarounds. >> > >> > There's a schema for the JSON here: >> > <https://github.com/ipython/ipython/blob/master/IPython/nbformat/v3/v3.withref.json>. >> > >> > Cheers, >> > - Zaki Mughal >> > >> > >> >> > >> >> > I suspect that anybody who wants to get involved can begin by >> >> > downloading >> >> > and digging into the notebook software. I wonder if the Julia bindings >> >> > might >> >> > serve as a good reference? >> >> > >> >> > David >> >> > >> >> > >> >> > On Mon, Jun 16, 2014 at 8:32 AM, Chris Marshall <[email protected]> >> >> > wrote: >> >> >> >> >> >> Very interesting discussion so far. A focus on PDL >> >> >> development for me as release manager has been to >> >> >> improve the portability and buildability of PDL across >> >> >> all major perl platforms (windows, macosx, and unix/linux/bsd/*). >> >> >> >> >> >> We've made steady progress but once PDL is installed >> >> >> the user might ask "Now what?". It would be nice to >> >> >> have a clear and simple answer for that. (In addition to >> >> >> the use case of supporting better scientific development >> >> >> and collaboration). >> >> >> >> >> >> The good news is that we have two key pieces already >> >> >> available that could be a foundation for iPDL: >> >> >> >> >> >> (1) Interactive PDL shells (perldl, pdl2) >> >> >> >> >> >> We've already made a start at integrating multiple GUI >> >> >> toolkit event loops. Stalled for now but I think we know >> >> >> what is needed. >> >> >> >> >> >> (2) Prima and Prima::OpenGL >> >> >> >> >> >> This gives us a baseline, *extremely* portable GUI >> >> >> toolkit to build on. We could use other toolkits but >> >> >> it is really difficult to beat the portability of Prima as >> >> >> a powerful GUI for perl. In a sense it is a little known >> >> >> super-power perl module. :-) >> >> >> >> >> >> NOTE: I explicitly call out Prima::OpenGL because >> >> >> I think for high performance and portable graphics and >> >> >> realtime visualization, OpenGL is now the default >> >> >> standard---even including GPU compute shaders in >> >> >> the latest version. >> >> >> >> >> >> I'm sure there are some other ideas but, like the PDL3 >> >> >> development discussions, I think the best approach is >> >> >> to KISS as much as possible. Avoiding outside toolkits >> >> >> and libraries where possible is a win for portability, >> >> >> especially for non-unix-ish platforms such as windows. >> >> >> >> >> >> --Chris >> >> >> >> >> >> >> >> >> On Mon, Jun 16, 2014 at 7:10 AM, Paul Goodall >> >> >> <[email protected]> wrote: >> >> >> > Hi David, Craig, >> >> >> > >> >> >> > I’d be happy to help with this - I should have spare time in between >> >> >> > projects to contribute to it. >> >> >> > >> >> >> > Personally, I don’t think it would be a bad thing for PDL to be more >> >> >> > accessible to the general community. Typically when I explain to >> >> >> > others >> >> >> > that I use PDL, I’m met with a blank face, prompting for an >> >> >> > explanation. >> >> >> > It >> >> >> > would be nice if PDL were to be recognised as a desirable skill in >> >> >> > the >> >> >> > same >> >> >> > way that Python is (particularly, for example, in job interview >> >> >> > situations). >> >> >> > It is a shame that more people don’t know about/have the power of >> >> >> > PDL at >> >> >> > their fingertips :-) >> >> >> > >> >> >> > Paul >> >> >> > >> >> >> > >> >> >> > On 13 Jun 2014, at 18:12, David Mertens <[email protected]> >> >> >> > wrote: >> >> >> > >> >> >> > Paul, >> >> >> > >> >> >> > To clarify, the notebooks that you mention in your link have two key >> >> >> > features. First, they provide online sharing, so it is very easy to >> >> >> > show >> >> >> > your colleagues some ideas and calculations. Your colleagues can >> >> >> > probably >> >> >> > even try manipulating the data in their browser, if it's fancy >> >> >> > enough. >> >> >> > Second, they provide means for (1) writing code, (2) writing prose, >> >> >> > (3) >> >> >> > typesetting math, and (4) embedding media such as pictures. They >> >> >> > are, in >> >> >> > essence, Mathematica clones for their respective languages. >> >> >> > >> >> >> > PDL does not have an equivalent to this sort of tool. I wrote a >> >> >> > rudimentary >> >> >> > offline GUI data analysis program called App::Prima::REPL, but that >> >> >> > was >> >> >> > more >> >> >> > targeted at the Matlab audience, not the Mathematica audience. It was >> >> >> > also a >> >> >> > giant pile of spaghetti, and I got stalled partway through a >> >> >> > refactoring >> >> >> > effort. It is not document focused, but rather tab focused. There is >> >> >> > an >> >> >> > API >> >> >> > for building our own custom tabs, but it's really more of a >> >> >> > programmer's >> >> >> > tool, not a scientists log book. >> >> >> > >> >> >> > I have lately found myself doing a lot of thinking in LyX, then >> >> >> > programming >> >> >> > in Perl. I would really like if there was some way for me to combine >> >> >> > all >> >> >> > of >> >> >> > that into a single document, much like the notebooks that you >> >> >> > mention. >> >> >> > However, my programming time has lately been dedicated to other >> >> >> > projects >> >> >> > (especially, this last week, polishing off some final work on >> >> >> > PDL::Graphics::Prima for a forthcoming release). >> >> >> > >> >> >> > If you are interested in helping, please let me know. I'd love to >> >> >> > work >> >> >> > with >> >> >> > somebody on this. :-) >> >> >> > >> >> >> > David >> >> >> > >> >> >> > >> >> >> > On Fri, Jun 13, 2014 at 12:32 PM, Craig DeForest >> >> >> > <[email protected]> >> >> >> > wrote: >> >> >> >> >> >> >> >> I wouldn't say there's an online notebook viewer so much a powerful >> >> >> >> toolkit to build one. David Mertens recently implemented >> >> >> >> PDL::Graphics::Prima, which is an object framework that can be used >> >> >> >> to >> >> >> >> construct interactive notebooks very simply and quickly. For >> >> >> >> example, >> >> >> >> you >> >> >> >> can generate a plot object and connect it to a PDL, and very easily >> >> >> >> update >> >> >> >> the plot as the PDL evolves - or autogenerate/autoupdate plots as >> >> >> >> you >> >> >> >> carry >> >> >> >> out a calculation. >> >> >> >> >> >> >> >> That is sort of in keeping with the PDL "style" -- our niche seems >> >> >> >> to >> >> >> >> be >> >> >> >> powerful tools that are expert-friendly, rather than polished >> >> >> >> packages. >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> On Jun 13, 2014, at 9:27 AM, Paul Goodall >> >> >> >> <[email protected]> wrote: >> >> >> >> >> >> >> >> > Hi, >> >> >> >> > >> >> >> >> > Apologies if this has a very obvious answer, but does PDL have an >> >> >> >> > equivalent to the online notebook viewers available to the likes >> >> >> >> > of >> >> >> >> > Python, >> >> >> >> > Ruby and (even) Julia? >> >> >> >> > http://nbviewer.ipython.org >> >> >> >> > >> >> >> >> > I’d really like to make use of this ‘IPDL’ if it exists. >> >> >> >> > >> >> >> >> > Thanks, >> >> >> >> > Paul >> >> > >> >> > >> >> > >> >> > >> >> > -- >> >> > "Debugging is twice as hard as writing the code in the first place. >> >> > Therefore, if you write the code as cleverly as possible, you are, >> >> > by definition, not smart enough to debug it." -- Brian Kernighan _______________________________________________ Perldl mailing list [email protected] http://mailman.jach.hawaii.edu/mailman/listinfo/perldl
