Re: [O] asynchronous code evaluation
Johannes Rainer johannes.rai...@gmail.com writes: hi all! I have some pretty calculation intense R code in a code block and when I execute the code (C-c C-c) my emacs freezes and waits for the code to finish. Is there a way to evaluate the code asynchronously, so that I could edit the file (obviously not the code block itself) while the code is being evaluated? Hi Jo, This is not currently possible, but it would certainly be a nice feature and is something that has been requested before. This would require some general Org-mode support before any language specific code is written (ideally no language specific code would be required). For anyone who feels like tackling such a project the first step would be to look at `org-babel-comint-with-output' and `org-babel-eval'. I'm not sure what the best way is to wait until the external process returns to handle the code block results, I'd guess we'd either want to pass closures to those functions, or somehow yield while the external process runs. Some careful thought of the ramifications of asynchronous code block evaluation may be required. E.g., would every piece of code which calls a code block (from C-c C-c to :var resolution) need to be updated to call the code block in an asynchronous manner? Best, -- Eric Schulte https://cs.unm.edu/~eschulte PGP: 0x614CA05D
Re: [O] asynchronous code evaluation
On Wed, Sep 04, 2013 at 07:27:01AM +0200, Johannes Rainer wrote: hi all! I have some pretty calculation intense R code in a code block and when I execute the code (C-c C-c) my emacs freezes and waits for the code to finish. Is there a way to evaluate the code asynchronously, so that I could edit the file (obviously not the code block itself) while the code is being evaluated? Not sure that is a possibility yet. For the moment you could try asynchronous export as a workaround. That said, I am curious why you are knowingly tying a computationally intensive (possibly buggy, during development i.e.) process with your editor. I have to work on some fairly computationally intensive analysis with Python. I always keep them in separate modules and scripts run from the shell. Hope this helps, -- Suvayu Open source is the future. It sets us free.
Re: [O] asynchronous code evaluation
On Wed, Sep 4, 2013 at 10:00 AM, Suvayu Ali fatkasuvayu+li...@gmail.comwrote: On Wed, Sep 04, 2013 at 07:27:01AM +0200, Johannes Rainer wrote: hi all! I have some pretty calculation intense R code in a code block and when I execute the code (C-c C-c) my emacs freezes and waits for the code to finish. Is there a way to evaluate the code asynchronously, so that I could edit the file (obviously not the code block itself) while the code is being evaluated? Not sure that is a possibility yet. For the moment you could try asynchronous export as a workaround. yes, I will do that. I just realized that I have to set ess-ask-for-ess-directory to nil, otherwise the async export hangs because it tries to ask where to start R. That said, I am curious why you are knowingly tying a computationally intensive (possibly buggy, during development i.e.) process with your editor. I have to work on some fairly computationally intensive analysis with Python. I always keep them in separate modules and scripts run from the shell. well, I'm using emacs/org for my data analyses in R. I thus combine documentation (i.e. the conclusions drawn from e.g. plots created in R) and the R code to perform the analysis in my org file. Since I'm analysing high throughput data some tasks to handle the data are quite time consuming. cheers, jo Hope this helps, -- Suvayu Open source is the future. It sets us free. -- Johannes Rainer, PhD Applied Bioinformatics Group, Division Molecular Pathophysiology, Biocenter, Medical University Innsbruck, Innrain 80/82 II, 6020 Innsbruck, Austria and Tyrolean Cancer Research Institute Innrain 66, 6020 Innsbruck, Austria Tel.: +43 (0)512 9003 70961 Email: johannes.rai...@i-med.ac.at johannes.rai...@tcri.at URL: http://bioinfo.i-med.ac.at
Re: [O] asynchronous code evaluation
On Wed, Sep 04, 2013 at 03:23:03PM +0200, Johannes Rainer wrote: On Wed, Sep 4, 2013 at 10:00 AM, Suvayu Ali fatkasuvayu+li...@gmail.comwrote: That said, I am curious why you are knowingly tying a computationally intensive (possibly buggy, during development i.e.) process with your editor. I have to work on some fairly computationally intensive analysis with Python. I always keep them in separate modules and scripts run from the shell. well, I'm using emacs/org for my data analyses in R. I thus combine documentation (i.e. the conclusions drawn from e.g. plots created in R) and the R code to perform the analysis in my org file. Since I'm analysing high throughput data some tasks to handle the data are quite time consuming. Okay, in that case I would try to achieve the asynchronicity in R. I think you would have better luck that way. -- Suvayu Open source is the future. It sets us free.
Re: [O] asynchronous code evaluation
Johannes Rainer johannes.rainer at gmail.com writes: hi all! I have some pretty calculation intense R code in a code block and when I execute the code (C-c C-c) my emacs freezes and waits for the code to finish. Is there a way to evaluate the code asynchronously, so that I could edit the file (obviously not the code block itself) while the code is being evaluated? thanks, jo Try this: C-c ' in the R src block. Then use C-c C-c (ess-eval-region-or-function-or-paragraph-and-step) to execute the long running code from the ESS edit buffer. You may need to select the region if there are several paragraphs, or delete any blank lines. == Another alternative is to use ravel. https://github.com/chasberry/orgmode-accessories Create a knitr or Sweave doc from the org doc or a suitable subtree of the computationally intensive parts. Then you can knit() the doc from the R session and switch back to the buffer you want to edit. Also, you might want to turn on caching in the knitr doc to enable a quick restart when you log in again.
Re: [O] asynchronous code evaluation
On 09/04/2013 09:23 AM, Johannes Rainer wrote: well, I'm using emacs/org for my data analyses in R. I thus combine documentation (i.e. the conclusions drawn from e.g. plots created in R) and the R code to perform the analysis in my org file. Since I'm analysing high throughput data some tasks to handle the data are quite time consuming. In the past, I have solved that problem by adding a makefile in the directory, which produces a separate fragment to be consumed/included by the org file. Make based on the updated data, publish based on the static outuput of the make process. If you let the sim run in the background while you edit, you'll likely get to a point, whilst tweaking or such, that you have dispatched several long-running jobs in parallel, and only care about the last result. That's probably counterproductive. :) - Allen S. Rout
Re: [O] asynchronous code evaluation
Hi Johannes, Johannes Rainer johannes.rai...@i-med.ac.at writes: well, I'm using emacs/org for my data analyses in R. I thus combine documentation (i.e. the conclusions drawn from e.g. plots created in R) and the R code to perform the analysis in my org file. Since I'm analysing high throughput data some tasks to handle the data are quite time consuming. A variation on Allen's Makefile approach, I structure my (Python) code to be idempotent. The real code never (re)runs unless its output either doesn't exist or is older than the input. Normally when org runs the code's higher level interface it's essentially a no-op and very fast. If something does change it's usually because I'm doing development and will be running the code outside of org enough that it's reached its no-op state before I go back into emacs to refresh/edit the org document. Also, I keep things synchronous to avoid having to multiple, competing processes running at once. What I want is to take this approach and generalize into a more formal workflow system. One which handles caching files and in-memory objects in a versioned store to provide data provenance and idempotent running in a general sense. There's already been some work on this (bein, pyutilib workflow and sumatra to name some Python oriented ones) but so far nothing that just fits. I'm in the middle of trying my hand at something better, but it's not yet usable. -Brett. pgpgHPRVolmV1.pgp Description: PGP signature
[O] asynchronous code evaluation
hi all! I have some pretty calculation intense R code in a code block and when I execute the code (C-c C-c) my emacs freezes and waits for the code to finish. Is there a way to evaluate the code asynchronously, so that I could edit the file (obviously not the code block itself) while the code is being evaluated? thanks, jo
