Re: [ccp4bb] step refine speed of wincoot - a little drawback of the new interruptible_fit_protein() function in wincoot
Thank you for the comment. You are correct that there are being backups made in the newer interruptable fit function (they were't previously in the uninterruptable fitting). I wasnt thinking about that (*). This may (especially on windows) slow down things a great deal as we write and gzip these files as well. To note here, gzip may be slow on windows as is writing files. In the pre-release versions I disabled the gzip on windows by default (and in general made the compression of backup files optional). This may speed up things (in certain circumstances). I hope this clarifies things a bit... B (*) It's debatable if we want backups here or not. If you can and want to interrupt the fitting you may want to go back one step, which requires a backup. On the other side this will write a lot of backup files which is slowing thing down by writing and compressing files, as well as producing a lot of backup files. Discuss... (or Paul and I just decide). Hi Xiaopeng, and those who are using the new interruptible fit_protein... or stepped_refine... in wincoot 0.6.1: I just took a look at wincoot 0.6.1's fitting.py file (WinCoot\share\coot\python\fitting.py). It seems that in the old fit_protein() and stepped_refine_protein() functions, the backup was programmed to be turned off during the cycles when it steps through the protein. However when using the Stepped refine... from the Extensions menu, the program starts to spit out tons of backup files. I then checked the extension.py. It turns out that these menu items are calling the new interruptible_fit_protein() function instead of the old uninterruptible ones. Some further experiment showed that this behavior was dependent on the backup state variable of the model being refined. When the backup state is 1 (the default value?), these new interruptible fitting functions will save a compressed backup file on each step. So, in order to save time when using the new interruptible protein fitting functions, one can type the following command in the python script window to tweak the backup state to 0: make_backup(imol) #to save a backup turn_off_backup(imol) where imol is the number associated with the pdb file being fitted. Do not forget to turn it back on after the fitting: turn_on_backup(imol) To check the current model's backup state: backup_state(imol) Zhijie -- From: Xiaopeng Hu huxp...@mail.sysu.edu.cn Sent: Tuesday, March 29, 2011 9:02 AM To: Zhijie Li zhijie...@utoronto.ca Subject: Re: [ccp4bb] step refine speed of wincoot Dear Zhijie, Thanks for there suggestions. I tried to turn off backup (with script command) and it did help although not much. Is there any way to add this perference to setting of coot/wincoot? best xiaopeng - 原始邮件 - 发件人: Zhijie Li zhijie...@utoronto.ca 收件人: Xiaopeng Hu huxp...@mail.sysu.edu.cn 发送时间: 星期二, 2011年 3 月 29日 下午 7:46:05 主题: Re: [ccp4bb] step refine speed of wincoot Hi, If you feel the refinement to be too slow, you may turn off the smooth centering (in preferences) or change the centering steps to a smaller number to save unnecessary graphical calculations. To go extreme, you may even remove the real time display commands in the scripts - also a way to test if the difference observed is due to different graphical efficiency. Reducing the size of the displayed map also helps. The other thing you may need to consider is that coot/wincoot will save a backup file each time it updates the model, which means on each step of the refinement you have a new file generated (take a look at the coot-backup directory when doing stepped refinement). If your windows disk is terribly fragmented then sure you will spend a lot of time on writing these files. The other thing is, windows has a different file caching mechanism than linux, this can also cause a huge difference when a few hundred small temporary files are queued for writing. My impression is that both the ext2/3 file system and the way linux handles caching are more efficient for this kind of situations. You may try deleting the files in wincoot\coot-backup periodically and defragmenting that partition. Making a virtual disk in the RAM to put your backup directory there could be something to experiment on too. Zhijie -- From: Xiaopeng Hu huxp...@mail.sysu.edu.cn Sent: Sunday, March 20, 2011 9:22 AM To: CCP4BB@JISCMAIL.AC.UK Subject: [ccp4bb] step refine speed of wincoot Dear all, I found the step refine speed of wincoot is much slower than that of linux coot (with the same pc). Is it normal or I need to configure something with the wincoot? best, Xiaopeng Hu
Re: [ccp4bb] step refine speed of wincoot - a little drawback of the new interruptible_fit_protein() function in wincoot
Hi Bernhard, I realized this when comparing the functions yesterday. Exactly like what you said, when a user interrupts the process, what he/she wants at the moment is most likely to be going back a few residues instead of starting all over again. I am quite happy to have both interruptible and uninterruptible functions to choose from. Thanks for the effort. But maybe it would make the impatient ones happier by presenting both versions in the Extension menu? Besides speed, one other advantage I can see for the old function is that by having a simpler structure, it is way easier for the users to make their own modifications. May I beg you not to remove it from future versions? Regards, Zhijie -- From: Bernhard C. Lohkamp bernh...@chem.gla.ac.uk Sent: Thursday, March 31, 2011 1:11 PM To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] step refine speed of wincoot - a little drawback of the new interruptible_fit_protein() function in wincoot Thank you for the comment. You are correct that there are being backups made in the newer interruptable fit function (they were't previously in the uninterruptable fitting). I wasnt thinking about that (*). This may (especially on windows) slow down things a great deal as we write and gzip these files as well. To note here, gzip may be slow on windows as is writing files. In the pre-release versions I disabled the gzip on windows by default (and in general made the compression of backup files optional). This may speed up things (in certain circumstances). I hope this clarifies things a bit... B (*) It's debatable if we want backups here or not. If you can and want to interrupt the fitting you may want to go back one step, which requires a backup. On the other side this will write a lot of backup files which is slowing thing down by writing and compressing files, as well as producing a lot of backup files. Discuss... (or Paul and I just decide). Hi Xiaopeng, and those who are using the new interruptible fit_protein... or stepped_refine... in wincoot 0.6.1: I just took a look at wincoot 0.6.1's fitting.py file (WinCoot\share\coot\python\fitting.py). It seems that in the old fit_protein() and stepped_refine_protein() functions, the backup was programmed to be turned off during the cycles when it steps through the protein. However when using the Stepped refine... from the Extensions menu, the program starts to spit out tons of backup files. I then checked the extension.py. It turns out that these menu items are calling the new interruptible_fit_protein() function instead of the old uninterruptible ones. Some further experiment showed that this behavior was dependent on the backup state variable of the model being refined. When the backup state is 1 (the default value?), these new interruptible fitting functions will save a compressed backup file on each step. So, in order to save time when using the new interruptible protein fitting functions, one can type the following command in the python script window to tweak the backup state to 0: make_backup(imol) #to save a backup turn_off_backup(imol) where imol is the number associated with the pdb file being fitted. Do not forget to turn it back on after the fitting: turn_on_backup(imol) To check the current model's backup state: backup_state(imol) Zhijie -- From: Xiaopeng Hu huxp...@mail.sysu.edu.cn Sent: Tuesday, March 29, 2011 9:02 AM To: Zhijie Li zhijie...@utoronto.ca Subject: Re: [ccp4bb] step refine speed of wincoot Dear Zhijie, Thanks for there suggestions. I tried to turn off backup (with script command) and it did help although not much. Is there any way to add this perference to setting of coot/wincoot? best xiaopeng - 原始邮件 - 发件人: Zhijie Li zhijie...@utoronto.ca 收件人: Xiaopeng Hu huxp...@mail.sysu.edu.cn 发送时间: 星期二, 2011年 3 月 29日 下午 7:46:05 主题: Re: [ccp4bb] step refine speed of wincoot Hi, If you feel the refinement to be too slow, you may turn off the smooth centering (in preferences) or change the centering steps to a smaller number to save unnecessary graphical calculations. To go extreme, you may even remove the real time display commands in the scripts - also a way to test if the difference observed is due to different graphical efficiency. Reducing the size of the displayed map also helps. The other thing you may need to consider is that coot/wincoot will save a backup file each time it updates the model, which means on each step of the refinement you have a new file generated (take a look at the coot-backup directory when doing stepped refinement). If your windows disk is terribly fragmented then sure you will spend a lot of time on writing these files. The other thing is, windows has a different file caching mechanism than linux, this can also cause a huge difference when a few hundred small temporary files are queued for writing. My impression is that both the ext2
Re: [ccp4bb] step refine speed of wincoot - a little drawback of the new interruptible_fit_protein() function in wincoot
Hi Xiaopeng, and those who are using the new interruptible fit_protein... or stepped_refine... in wincoot 0.6.1: I just took a look at wincoot 0.6.1's fitting.py file (WinCoot\share\coot\python\fitting.py). It seems that in the old fit_protein() and stepped_refine_protein() functions, the backup was programmed to be turned off during the cycles when it steps through the protein. However when using the Stepped refine... from the Extensions menu, the program starts to spit out tons of backup files. I then checked the extension.py. It turns out that these menu items are calling the new interruptible_fit_protein() function instead of the old uninterruptible ones. Some further experiment showed that this behavior was dependent on the backup state variable of the model being refined. When the backup state is 1 (the default value?), these new interruptible fitting functions will save a compressed backup file on each step. So, in order to save time when using the new interruptible protein fitting functions, one can type the following command in the python script window to tweak the backup state to 0: make_backup(imol) #to save a backup turn_off_backup(imol) where imol is the number associated with the pdb file being fitted. Do not forget to turn it back on after the fitting: turn_on_backup(imol) To check the current model's backup state: backup_state(imol) Zhijie -- From: Xiaopeng Hu huxp...@mail.sysu.edu.cn Sent: Tuesday, March 29, 2011 9:02 AM To: Zhijie Li zhijie...@utoronto.ca Subject: Re: [ccp4bb] step refine speed of wincoot Dear Zhijie, Thanks for there suggestions. I tried to turn off backup (with script command) and it did help although not much. Is there any way to add this perference to setting of coot/wincoot? best xiaopeng - 原始邮件 - 发件人: Zhijie Li zhijie...@utoronto.ca 收件人: Xiaopeng Hu huxp...@mail.sysu.edu.cn 发送时间: 星期二, 2011年 3 月 29日 下午 7:46:05 主题: Re: [ccp4bb] step refine speed of wincoot Hi, If you feel the refinement to be too slow, you may turn off the smooth centering (in preferences) or change the centering steps to a smaller number to save unnecessary graphical calculations. To go extreme, you may even remove the real time display commands in the scripts - also a way to test if the difference observed is due to different graphical efficiency. Reducing the size of the displayed map also helps. The other thing you may need to consider is that coot/wincoot will save a backup file each time it updates the model, which means on each step of the refinement you have a new file generated (take a look at the coot-backup directory when doing stepped refinement). If your windows disk is terribly fragmented then sure you will spend a lot of time on writing these files. The other thing is, windows has a different file caching mechanism than linux, this can also cause a huge difference when a few hundred small temporary files are queued for writing. My impression is that both the ext2/3 file system and the way linux handles caching are more efficient for this kind of situations. You may try deleting the files in wincoot\coot-backup periodically and defragmenting that partition. Making a virtual disk in the RAM to put your backup directory there could be something to experiment on too. Zhijie -- From: Xiaopeng Hu huxp...@mail.sysu.edu.cn Sent: Sunday, March 20, 2011 9:22 AM To: CCP4BB@JISCMAIL.AC.UK Subject: [ccp4bb] step refine speed of wincoot Dear all, I found the step refine speed of wincoot is much slower than that of linux coot (with the same pc). Is it normal or I need to configure something with the wincoot? best, Xiaopeng Hu
