Re: [R] multi day R run crashed - is there a log
Oh dear. No there isn't. I do a lot of very long runs, and have learned to write out intermediate steps. Depending on what you are doing, saving the RData file periodically may be appropriate, or writing out a csv file of results so far, or even just printing something to the output file (if you are using batch). There may well be something more elegant, but these solutions have all worked well for me in various situations. Sarah On Wed, Aug 25, 2010 at 3:46 AM, Martin Tomko martin.to...@geo.uzh.ch wrote: Dear all, I am using an R 2.10 installation on a Windows 203 server that I have no control over. After a multi-day run I found that it was terminated/crashed. Is there any log kept by R where I could see whether something/what happened? The same process has been run beofre on a smaller dataset (also at least a day of computing) without problems. Thanks Martin -- Sarah Goslee http://www.functionaldiversity.org __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] multi day R run crashed - is there a log
Hi Sarah, thank you very much for your answer. I have been spitting things out on screen, but unfortunately I have not run it as a batch log, but in the interactive window, so when the GUI crashed I was left without trace... I guess I should google how to run it from a batch. I should also explore using RData, I have ony been using csv files and tables so far, it seems that can also bring some added performance. As the main output of my process is a matrix, I would really need to append to a matrix after each iteration. I have identified the write.table append parameter-based solution, but that would only append rows. Is there a way to slowly grow a matrix in both directions, meaning append columns as well (it is a big distance matrix). Thanks a lot for your help, Cheers Martin On 8/25/2010 1:56 PM, Sarah Goslee wrote: Oh dear. No there isn't. I do a lot of very long runs, and have learned to write out intermediate steps. Depending on what you are doing, saving the RData file periodically may be appropriate, or writing out a csv file of results so far, or even just printing something to the output file (if you are using batch). There may well be something more elegant, but these solutions have all worked well for me in various situations. Sarah On Wed, Aug 25, 2010 at 3:46 AM, Martin Tomkomartin.to...@geo.uzh.ch wrote: Dear all, I am using an R 2.10 installation on a Windows 203 server that I have no control over. After a multi-day run I found that it was terminated/crashed. Is there any log kept by R where I could see whether something/what happened? The same process has been run beofre on a smaller dataset (also at least a day of computing) without problems. Thanks Martin -- Martin Tomko Postdoctoral Research Assistant Geographic Information Systems Division Department of Geography University of Zurich - Irchel Winterthurerstr. 190 CH-8057 Zurich, Switzerland email: martin.to...@geo.uzh.ch site: http://www.geo.uzh.ch/~mtomko mob:+41-788 629 558 tel:+41-44-6355256 fax:+41-44-6356848 __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] multi day R run crashed - is there a log
Hi, On Wed, Aug 25, 2010 at 8:11 AM, Martin Tomko martin.to...@geo.uzh.ch wrote: Hi Sarah, thank you very much for your answer. I have been spitting things out on screen, but unfortunately I have not run it as a batch log, but in the interactive window, so when the GUI crashed I was left without trace... I guess I should google how to run it from a batch. I have no idea how to do that in Windows, but I'm sure it's possible. :) That way the things written to the screen will be saved in a file instead. I should also explore using RData, I have ony been using csv files and tables so far, it seems that can also bring some added performance. If you need to save *everything*, RData is what you get when you use save(), or when you close a session and choose y to saving the data. It can be read in using load(). That's one way to be able to pick up where you left off. As the main output of my process is a matrix, I would really need to append to a matrix after each iteration. I have identified the write.table append parameter-based solution, but that would only append rows. Is there a way to slowly grow a matrix in both directions, meaning append columns as well (it is a big distance matrix). AFAIK, you can't append columns that way because of the way text files are written to disk. You'd need to rewrite the whole thing, or possibly write it out in lower triangular format with NA values as padding (assuming it's a symmetric distance). Or for that matter, you could just write it out as a really long vector, and turn it back into a matrix later if you need to read the saved file in after a crash. I'd recommend saving whatever variables are needed so that you can pick up exactly where you left off, if possible. Much nicer to pick up 12 hours in than to start over from the beginning. Not R, but I just finished a 5-week batch job. You can bet that I put a lot of thought into incremental save points and how to resume after an unexpected halt! Sarah -- Sarah Goslee http://www.functionaldiversity.org __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] multi day R run crashed - is there a log
Hi , thanks, the lower.tri idea is I guess the best way. Will try that. Cheers Martin On 8/25/2010 2:21 PM, Sarah Goslee wrote: Hi, On Wed, Aug 25, 2010 at 8:11 AM, Martin Tomkomartin.to...@geo.uzh.ch wrote: Hi Sarah, thank you very much for your answer. I have been spitting things out on screen, but unfortunately I have not run it as a batch log, but in the interactive window, so when the GUI crashed I was left without trace... I guess I should google how to run it from a batch. I have no idea how to do that in Windows, but I'm sure it's possible. :) That way the things written to the screen will be saved in a file instead. I should also explore using RData, I have ony been using csv files and tables so far, it seems that can also bring some added performance. If you need to save *everything*, RData is what you get when you use save(), or when you close a session and choose y to saving the data. It can be read in using load(). That's one way to be able to pick up where you left off. As the main output of my process is a matrix, I would really need to append to a matrix after each iteration. I have identified the write.table append parameter-based solution, but that would only append rows. Is there a way to slowly grow a matrix in both directions, meaning append columns as well (it is a big distance matrix). AFAIK, you can't append columns that way because of the way text files are written to disk. You'd need to rewrite the whole thing, or possibly write it out in lower triangular format with NA values as padding (assuming it's a symmetric distance). Or for that matter, you could just write it out as a really long vector, and turn it back into a matrix later if you need to read the saved file in after a crash. I'd recommend saving whatever variables are needed so that you can pick up exactly where you left off, if possible. Much nicer to pick up 12 hours in than to start over from the beginning. Not R, but I just finished a 5-week batch job. You can bet that I put a lot of thought into incremental save points and how to resume after an unexpected halt! Sarah -- Martin Tomko Postdoctoral Research Assistant Geographic Information Systems Division Department of Geography University of Zurich - Irchel Winterthurerstr. 190 CH-8057 Zurich, Switzerland email: martin.to...@geo.uzh.ch site: http://www.geo.uzh.ch/~mtomko mob:+41-788 629 558 tel:+41-44-6355256 fax:+41-44-6356848 __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
