Re: [R] Step Function Freezing R
Can you think of any systemic changes that might interefere with R besides Symantec EndPoint and LiveUpdate? I have removed those programs and allocated more memory to R, but it is still way too slow. On Nov 13, 10:45 pm, J Dougherty j...@surewest.net wrote: On Friday 13 November 2009 07:17:28 am Jgabriel wrote: I can't fully answer all these questions, but I'll do my best - There have not been any updates of Windows, and I did not update R during the period, although I did reinstall it after the problem started. There have been no changes to Norton or any other software that uses system resources in that way. The one thing I can think of is that I installed a program called Digitizer (creates data tables/ csds from visually analyzing line charts) around the same time it started freezing. I have completely uninstalled it and deleted all related files. The problem should not be with the data, which is fine. I am running Windows XP Professional and Excel 2007. I have allowed the process to run overnight on two separate occasions. The first time it was completely frozen and there was no progress. Last night it actually made progress, but the fact remains that a process that used to take an hour at the most still has not even halfway completed after over 8 hours. I even installed extra RAM on the system so that there is more than when the process used to work. I agree that it is probably a change in software, but I can't figure out what has changed or what I can do about it. OK, from what you are saying, it seems clear the problem is a system problem rather than an R issue. MS issues patches every month, so there may not have been upgrades, but there could still be systemic changes. As regards R, did you update R? How big is the data table? Has it grown? Did you or someone else alter the available memory to R by an environment setting? Have you searched Memory in R help and manuals? Is this computer yours, or is it used by others who may have altered settings? These are all questions that may be pertinent. Good luck. JWDougherty __ r-h...@r-project.org mailing listhttps://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guidehttp://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.- Hide quoted text - - Show quoted text - __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Step Function Freezing R
I can't fully answer all these questions, but I'll do my best - There have not been any updates of Windows, and I did not update R during the period, although I did reinstall it after the problem started. There have been no changes to Norton or any other software that uses system resources in that way. The one thing I can think of is that I installed a program called Digitizer (creates data tables/ csds from visually analyzing line charts) around the same time it started freezing. I have completely uninstalled it and deleted all related files. The problem should not be with the data, which is fine. I am running Windows XP Professional and Excel 2007. I have allowed the process to run overnight on two separate occasions. The first time it was completely frozen and there was no progress. Last night it actually made progress, but the fact remains that a process that used to take an hour at the most still has not even halfway completed after over 8 hours. I even installed extra RAM on the system so that there is more than when the process used to work. I agree that it is probably a change in software, but I can't figure out what has changed or what I can do about it. Thanks for your help, JGS On Nov 12, 7:05 pm, J Dougherty j...@surewest.net wrote: There are some points of information you should include. One, have there been recent updates of Windows that may have altered system behaviour? Two, have you updated R during the same period? Three, have you installed software that uses system resources that were available to R before - e.g. has Norton or whatever you use to guard your system upgraded? Four, what versions of Windows and Excel are you running? Five, have you inspected your data for corruption? Six, what changes (data entry, editing, recoding, etc.) have you done that might coincide with the changed behaviour? Seven, have you allowed the process to simply run - say overnight? Nine, how big is you database? In essence, your problem sounds like system problem. The most likely cause is a change in software, but without some really clear information on changes on your system that could potentially have caused this, you are unlikely to find it easily. One possibility is that a change in software has resulted in smaller available resource space. JWDougherty On Thursday 12 November 2009 09:05:34 am Jgabriel wrote: I have a question that might not be kosher here, but I'm running out of options and need some help. Basically using the function step is freezingR. I am running a model that includes a number of interactions on a large data set with a number of dummy variables representing whether an event occurred or not. Here's a simplified example - fit = lm(a~b*(c+d+e)*(f+g+h)*(i+j+k)) regs = step(fit) at this point R freezes and alternates between Not Responding and Running in Windows task manager. It wasn't always this way. I was able to run the same model on the same dataset a couple weeks ago and while it took the step function 20-30 minutes to compute the best AIC and didn't exclude all the insignificant variables/interactions, it served its purpose. Even after uninstalling and reinstalling R and RExcel (which I used to load the data set), the problem persists. I tried it on a smaller data set and still no luck. This is more of a computing issue, so if anyone can point me to a forum better equipped to help me out, I will owe you my eternal gratitude. I hate to fill up space with such an uninspired question, but like I said I'm running out of options. Thanks, JGS __ r-h...@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.htmland provide commented, minimal, self-contained, reproducible code. __ r-h...@r-project.org mailing listhttps://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guidehttp://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Step Function Freezing R
Jgabriel wrote: I can't fully answer all these questions, but I'll do my best - There have not been any updates of Windows, and I did not update R during the period, although I did reinstall it after the problem started. There have been no changes to Norton or any other software that uses system resources in that way. The one thing I can think of is that I installed a program called Digitizer (creates data tables/ csds from visually analyzing line charts) around the same time it started freezing. I have completely uninstalled it and deleted all related files. The problem should not be with the data, which is fine. I am running Windows XP Professional and Excel 2007. I have allowed the process to run overnight on two separate occasions. The first time it was completely frozen and there was no progress. Last night it actually made progress, but the fact remains that a process that used to take an hour at the most still has not even halfway completed after over 8 hours. I even installed extra RAM on the system Aha! Please ask your local IT experts to check the RAM and if that is fine ask them to proceed. I have already seen too many machines with RAM problems during the last years. It is hardly an R problem. Uwe Ligges so that there is more than when the process used to work. I agree that it is probably a change in software, but I can't figure out what has changed or what I can do about it. Thanks for your help, JGS On Nov 12, 7:05 pm, J Dougherty j...@surewest.net wrote: There are some points of information you should include. One, have there been recent updates of Windows that may have altered system behaviour? Two, have you updated R during the same period? Three, have you installed software that uses system resources that were available to R before - e.g. has Norton or whatever you use to guard your system upgraded? Four, what versions of Windows and Excel are you running? Five, have you inspected your data for corruption? Six, what changes (data entry, editing, recoding, etc.) have you done that might coincide with the changed behaviour? Seven, have you allowed the process to simply run - say overnight? Nine, how big is you database? In essence, your problem sounds like system problem. The most likely cause is a change in software, but without some really clear information on changes on your system that could potentially have caused this, you are unlikely to find it easily. One possibility is that a change in software has resulted in smaller available resource space. JWDougherty On Thursday 12 November 2009 09:05:34 am Jgabriel wrote: I have a question that might not be kosher here, but I'm running out of options and need some help. Basically using the function step is freezingR. I am running a model that includes a number of interactions on a large data set with a number of dummy variables representing whether an event occurred or not. Here's a simplified example - fit = lm(a~b*(c+d+e)*(f+g+h)*(i+j+k)) regs = step(fit) at this point R freezes and alternates between Not Responding and Running in Windows task manager. It wasn't always this way. I was able to run the same model on the same dataset a couple weeks ago and while it took the step function 20-30 minutes to compute the best AIC and didn't exclude all the insignificant variables/interactions, it served its purpose. Even after uninstalling and reinstalling R and RExcel (which I used to load the data set), the problem persists. I tried it on a smaller data set and still no luck. This is more of a computing issue, so if anyone can point me to a forum better equipped to help me out, I will owe you my eternal gratitude. I hate to fill up space with such an uninspired question, but like I said I'm running out of options. Thanks, JGS __ r-h...@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.htmland provide commented, minimal, self-contained, reproducible code. __ r-h...@r-project.org mailing listhttps://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guidehttp://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Step Function Freezing R
Alright, here's the update: the IT guy insists that the RAM is not the problem. RAM, according to him, is pretty black and white. There should be enough RAM on the system for the function to work, and he was the one who installed the extra RAM in the first place. Could there be another RAM issue besides the amount? Is there some special way in which R uses RAM that could be causing the issue? He also wiped the computer and rebuilt the whole system, to no avail. He's getting pretty tired of my problems, to say the least. You say you've seen a lot of RAM problems over the years, can you help me out? On Nov 13, 1:15 pm, Uwe Ligges lig...@statistik.tu-dortmund.de wrote: Jgabriel wrote: I can't fully answer all these questions, but I'll do my best - There have not been any updates of Windows, and I did not update R during the period, although I did reinstall it after the problem started. There have been no changes to Norton or any other software that uses system resources in that way. The one thing I can think of is that I installed a program called Digitizer (creates data tables/ csds from visually analyzing line charts) around the same time it started freezing. I have completely uninstalled it and deleted all related files. The problem should not be with the data, which is fine. I am running Windows XP Professional and Excel 2007. I have allowed the process to run overnight on two separate occasions. The first time it was completely frozen and there was no progress. Last night it actually made progress, but the fact remains that a process that used to take an hour at the most still has not even halfway completed after over 8 hours. I even installed extra RAM on the system Aha! Please ask your local IT experts to check the RAM and if that is fine ask them to proceed. I have already seen too many machines with RAM problems during the last years. It is hardly an R problem. Uwe Ligges so that there is more than when the process used to work. I agree that it is probably a change in software, but I can't figure out what has changed or what I can do about it. Thanks for your help, JGS On Nov 12, 7:05 pm, J Dougherty j...@surewest.net wrote: There are some points of information you should include. One, have there been recent updates of Windows that may have altered system behaviour? Two, have you updated R during the same period? Three, have you installed software that uses system resources that were available to R before - e.g. has Norton or whatever you use to guard your system upgraded? Four, what versions of Windows and Excel are you running? Five, have you inspected your data for corruption? Six, what changes (data entry, editing, recoding, etc.) have you done that might coincide with the changed behaviour? Seven, have you allowed the process to simply run - say overnight? Nine, how big is you database? In essence, your problem sounds like system problem. The most likely cause is a change in software, but without some really clear information on changes on your system that could potentially have caused this, you are unlikely to find it easily. One possibility is that a change in software has resulted in smaller available resource space. JWDougherty On Thursday 12 November 2009 09:05:34 am Jgabriel wrote: I have a question that might not be kosher here, but I'm running out of options and need some help. Basically using the function step is freezingR. I am running a model that includes a number of interactions on a large data set with a number of dummy variables representing whether an event occurred or not. Here's a simplified example - fit = lm(a~b*(c+d+e)*(f+g+h)*(i+j+k)) regs = step(fit) at this point R freezes and alternates between Not Responding and Running in Windows task manager. It wasn't always this way. I was able to run the same model on the same dataset a couple weeks ago and while it took the step function 20-30 minutes to compute the best AIC and didn't exclude all the insignificant variables/interactions, it served its purpose. Even after uninstalling and reinstalling R and RExcel (which I used to load the data set), the problem persists. I tried it on a smaller data set and still no luck. This is more of a computing issue, so if anyone can point me to a forum better equipped to help me out, I will owe you my eternal gratitude. I hate to fill up space with such an uninspired question, but like I said I'm running out of options. Thanks, JGS __ r-h...@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.htmlandprovide commented, minimal, self-contained, reproducible code. __ r-h...@r-project.org mailing
Re: [R] Step Function Freezing R
On Friday 13 November 2009 07:17:28 am Jgabriel wrote: I can't fully answer all these questions, but I'll do my best - There have not been any updates of Windows, and I did not update R during the period, although I did reinstall it after the problem started. There have been no changes to Norton or any other software that uses system resources in that way. The one thing I can think of is that I installed a program called Digitizer (creates data tables/ csds from visually analyzing line charts) around the same time it started freezing. I have completely uninstalled it and deleted all related files. The problem should not be with the data, which is fine. I am running Windows XP Professional and Excel 2007. I have allowed the process to run overnight on two separate occasions. The first time it was completely frozen and there was no progress. Last night it actually made progress, but the fact remains that a process that used to take an hour at the most still has not even halfway completed after over 8 hours. I even installed extra RAM on the system so that there is more than when the process used to work. I agree that it is probably a change in software, but I can't figure out what has changed or what I can do about it. OK, from what you are saying, it seems clear the problem is a system problem rather than an R issue. MS issues patches every month, so there may not have been upgrades, but there could still be systemic changes. As regards R, did you update R? How big is the data table? Has it grown? Did you or someone else alter the available memory to R by an environment setting? Have you searched Memory in R help and manuals? Is this computer yours, or is it used by others who may have altered settings? These are all questions that may be pertinent. Good luck. JWDougherty __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Step Function Freezing R
I have a question that might not be kosher here, but I'm running out of options and need some help. Basically using the function step is freezing R. I am running a model that includes a number of interactions on a large data set with a number of dummy variables representing whether an event occurred or not. Here's a simplified example - fit = lm(a~b*(c+d+e)*(f+g+h)*(i+j+k)) regs = step(fit) at this point R freezes and alternates between Not Responding and Running in Windows task manager. It wasn't always this way. I was able to run the same model on the same dataset a couple weeks ago and while it took the step function 20-30 minutes to compute the best AIC and didn't exclude all the insignificant variables/interactions, it served its purpose. Even after uninstalling and reinstalling R and RExcel (which I used to load the data set), the problem persists. I tried it on a smaller data set and still no luck. This is more of a computing issue, so if anyone can point me to a forum better equipped to help me out, I will owe you my eternal gratitude. I hate to fill up space with such an uninspired question, but like I said I'm running out of options. Thanks, JGS __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Step Function Freezing R
There are some points of information you should include. One, have there been recent updates of Windows that may have altered system behaviour? Two, have you updated R during the same period? Three, have you installed software that uses system resources that were available to R before - e.g. has Norton or whatever you use to guard your system upgraded? Four, what versions of Windows and Excel are you running? Five, have you inspected your data for corruption? Six, what changes (data entry, editing, recoding, etc.) have you done that might coincide with the changed behaviour? Seven, have you allowed the process to simply run - say overnight? Nine, how big is you database? In essence, your problem sounds like system problem. The most likely cause is a change in software, but without some really clear information on changes on your system that could potentially have caused this, you are unlikely to find it easily. One possibility is that a change in software has resulted in smaller available resource space. JWDougherty On Thursday 12 November 2009 09:05:34 am Jgabriel wrote: I have a question that might not be kosher here, but I'm running out of options and need some help. Basically using the function step is freezing R. I am running a model that includes a number of interactions on a large data set with a number of dummy variables representing whether an event occurred or not. Here's a simplified example - fit = lm(a~b*(c+d+e)*(f+g+h)*(i+j+k)) regs = step(fit) at this point R freezes and alternates between Not Responding and Running in Windows task manager. It wasn't always this way. I was able to run the same model on the same dataset a couple weeks ago and while it took the step function 20-30 minutes to compute the best AIC and didn't exclude all the insignificant variables/interactions, it served its purpose. Even after uninstalling and reinstalling R and RExcel (which I used to load the data set), the problem persists. I tried it on a smaller data set and still no luck. This is more of a computing issue, so if anyone can point me to a forum better equipped to help me out, I will owe you my eternal gratitude. I hate to fill up space with such an uninspired question, but like I said I'm running out of options. Thanks, JGS __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
