Re: [O] babel and long-running computations

2014-04-20 Thread John Kitchin
You are describing what I see as a need to run processes asynchronously. We use the Torque queue system for this. It takes care of scheduling and running jobs on a cluster. On Thu, Apr 17, 2014 at 6:52 AM, Christoph Groth wrote: > Hi Julian, > > > Generally, org-babel is suited for long running

Re: [O] babel and long-running computations

2014-04-20 Thread John Kitchin
On Sat, Apr 19, 2014 at 5:49 PM, Christoph Groth wrote: > The following message is a courtesy copy of an article > that has been posted to gmane.emacs.orgmode as well. > > Thank you, John, for your detailed reply. > > > we routinely do this, in the following way. We run jobs that may take > > up t

Re: [O] babel and long-running computations

2014-04-19 Thread Christoph Groth
Thank you, John, for your detailed reply. > we routinely do this, in the following way. We run jobs that may take > up to a week to finish, and they are usually run on a cluster. Our > setup relies on the following behavior for a script. > > 1. you can run the script anytime you want, and it can t

Re: [O] babel and long-running computations

2014-04-18 Thread John Kitchin
we routinely do this, in the following way. We run jobs that may take up to a week to finish, and they are usually run on a cluster. Our setup relies on the following behavior for a script. 1. you can run the script anytime you want, and it can tell the state of the calculation by some means. If t

Re: [O] babel and long-running computations

2014-04-17 Thread Nick Dokos
Christoph Groth writes: > Eric Schulte writes: > >> Currently Babel does not have support for long-running computations, >> however this feature has been discussed and there are previous >> messages on the mailing list and even some comments in the code >> outlining a potential implementation st

Re: [O] babel and long-running computations

2014-04-17 Thread Christoph Groth
Eric Schulte writes: > Currently Babel does not have support for long-running computations, > however this feature has been discussed and there are previous > messages on the mailing list and even some comments in the code > outlining a potential implementation strategy. So this may change > if/

Re: [O] babel and long-running computations

2014-04-17 Thread Eric Schulte
Christoph Groth writes: > Thorsten Jolitz writes: > >>> I’d love to hear about any frameworks or workflows that fulfill these >>> requirements. >> >> here is a link about that topic: >> >> ,--- >> | http://draketo.de/english/emacs/parallel-babel >> `--

Re: [O] babel and long-running computations

2014-04-17 Thread Eric S Fraga
On Thursday, 17 Apr 2014 at 11:52, Christoph Groth wrote: [...] > - Not freezing the editor during the execution of a task This is a problem but I solve this by running multiple instances of emacs. The difficulty arises if you wish to edit the same file in more than one emacs... but if you can

Re: [O] babel and long-running computations

2014-04-17 Thread Waldemar Quevedo
Hey Christoph, Not sure if this would help you, but I've playing with the idea of extracting some of the functionality that Org babel provides and created a small tool for doing reproducible runs using Org mode syntax: https://gist.github.com/wallyqs/10989253 It is based on the Org ruby parser imp

Re: [O] babel and long-running computations

2014-04-17 Thread Christoph Groth
Thorsten Jolitz writes: >> I’d love to hear about any frameworks or workflows that fulfill these >> requirements. > > here is a link about that topic: > > ,--- > | http://draketo.de/english/emacs/parallel-babel > `---

Re: [O] babel and long-running computations

2014-04-17 Thread Thorsten Jolitz
Christoph Groth writes: > I’d love to hear about any frameworks or workflows that fulfill these > requirements. here is a link about that topic: ,--- | http://draketo.de/english/emacs/parallel-babel `--- --

Re: [O] babel and long-running computations

2014-04-17 Thread Christoph Groth
Hi Julian, > Generally, org-babel is suited for long running computations. Its > caching can help you avoid rerunning code chunks. And long runtime does > not conflict with the idea of reproducible research, it just may be not > very comfortable for the user. I agree of course that it’s general

Re: [O] babel and long-running computations

2014-04-17 Thread Julian Gehring
Hi Christoph, Generally, org-babel is suited for long running computations. Its caching can help you avoid rerunning code chunks. And long runtime does not conflict with the idea of reproducible research, it just may be not very comfortable for the user. In my experience, one can avoid your pro

[O] babel and long-running computations

2014-04-17 Thread Christoph Groth
Dear all, I’m considering the use of org-babel for “reproducible science”. Is anyone successfully using org-babel for research that involves calculations that typically take a few minutes up to several hours on a single CPU (i.e. up to a few minutes on a cluster)? It seems to me that currently o