Re: control CPU usage
Jiang Fung Wong wrote: Dear All, Thank you for the information. I think I've some idea what the problem is about after seeing the replies. More information about my system and my script PIII 1Ghz, 512MB RAM, Windows XP SP3 The script monitors global input using PyHook, and calculates on the information collected from the events to output some numbers. Based on the numbers, the script then performs some automation using SendKeys module. here is the memory usage: firefox.exe, 69MB, 109MB svchost.exe, 26MB, 17MB pythonw.exe, 22MB, 17MB searchindexer.exe, 16MB, 19MB My first guess is that the script calculated for too long time after receiving an event before propagating it to the default handler, resulting the system to be non-responsive. I will try to implement the calculation part in another thread. Then the separate will have 100% CPU usage, hope the task scheduling of Windows works in my favour. (You top-posted this message, putting the whole stream out of order. So I deleted the history.) All my assumptions about your environment are now invalid. You don't have a CPU-bound application, you have a Windows application with event loop. Further, you're using SendKeys to generate a keystroke to the other process. So there are many things that could be affecting your latency, and all my previous guesses are useless. Adding threads to your application will probably slow the system down much more. You need to find out what your present problem is before complicating it. You haven't really described the problem. You say the system is unresponsive, but you made it that way by creating a global hook; a notoriously inefficient mechanism. That global hook inserts code into every process in the system, and you've got a pretty low-end environment to begin with. So what's the real problem, and how severe is it? And how will you measure improvement? The Task manager numbers are probably irrelevant. My first question is whether the pyHook event is calling the SendKeys function directly (after your lengthy calculation) or whether there are other events firing off in between. If it's all being done in the one event, then measure its time, and gather some statistics (min time, max time, average...). The task manager has far too simplistic visibility to be useful for this purpose. What else is this application doing when it's waiting for a pyHook call? Whose event loop implementation are you using? And the program you're trying to control -- is there perhaps another way in? DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: control CPU usage
On Sep 20, 6:24 pm, Dave Angel da...@ieee.org wrote: Jiang Fung Wong wrote: Dear All, Thank you for the information. I think I've some idea what the problem is about after seeing the replies. More information about my system and my script PIII 1Ghz, 512MB RAM, Windows XP SP3 The script monitors global input using PyHook, and calculates on the information collected from the events to output some numbers. Based on the numbers, the script then performs some automation using SendKeys module. here is the memory usage: firefox.exe, 69MB, 109MB svchost.exe, 26MB, 17MB pythonw.exe, 22MB, 17MB searchindexer.exe, 16MB, 19MB My first guess is that the script calculated for too long time after receiving an event before propagating it to the default handler, resulting the system to be non-responsive. I will try to implement the calculation part in another thread. Then the separate will have 100% CPU usage, hope the task scheduling of Windows works in my favour. (You top-posted this message, putting the whole stream out of order. So I deleted the history.) All my assumptions about your environment are now invalid. You don't have a CPU-bound application, you have a Windows application with event loop. Further, you're using SendKeys to generate a keystroke to the other process. So there are many things that could be affecting your latency, and all my previous guesses are useless. Adding threads to your application will probably slow the system down much more. You need to find out what your present problem is before complicating it. You haven't really described the problem. You say the system is unresponsive, but you made it that way by creating a global hook; a notoriously inefficient mechanism. That global hook inserts code into every process in the system, and you've got a pretty low-end environment to begin with. So what's the real problem, and how severe is it? And how will you measure improvement? The Task manager numbers are probably irrelevant. My first question is whether the pyHook event is calling the SendKeys function directly (after your lengthy calculation) or whether there are other events firing off in between. If it's all being done in the one event, then measure its time, and gather some statistics (min time, max time, average...). The task manager has far too simplistic visibility to be useful for this purpose. What else is this application doing when it's waiting for a pyHook call? Whose event loop implementation are you using? And the program you're trying to control -- is there perhaps another way in? DaveA Hi, Sorry I wasn't sure how to use Google groups to post a msg to the newsgroup, I used Gmail to write my previous reply. What you and the other guy have provided me isn't useless. Now I understand the non- responsiveness may not be caused by high CPU usage, as the OS, be it Windows or Linux, has a way to prioritize the tasks. This is a vital clue to me. By not responsive, I mean, for some time, the mouse pointer is not moving smoothly, to such extent that I can't do anything with the mouse. It's like playing a multi-player game on a connection with a lot of lag. It's not caused by global hook, because it happens under certain condition, i.e. when fpa.ProcessEvent(word) is computing. I included my main script for your reference. Comments: (1) The automation method tc.Auto() is slow, but it doesn't cause any problem, because the user would wait for the automation to finish, before he continues to do something. (2) all other methods invoked are fast, except fpa.ProcessEvent(word) (this information is obtained from profiling). It is this method that causes 100% CPU usage. I'm planning to move this method to a separate thread, so that OnEvent(event) can finish executing, while the separate thread goes on to finish its calculation. Is this a good idea? import pyHook import TypingAnalyzer import GUI def OnEvent(event): if hasattr(event, Key) and event.Ascii == 9 and event.Key == Tab and event.Injected == 0 and event.Alt == 0: tc.Auto() return False else: recognized = rk.ProcessEvent(event) if recognized: tc.MatchChar(recognized) paragraph = rc.ProcessEvent(recognized) if paragraph: for word in paragraph: fpa.ProcessEvent(word) return True hm = pyHook.HookManager() hm.MouseAllButtonsDown = OnEvent hm.KeyDown = OnEvent hm.HookMouse() hm.HookKeyboard() rk = TypingAnalyzer.ReadKey() rc = TypingAnalyzer.ReadChar() fpa = TypingAnalyzer.Analysis() tc = TypingAnalyzer.Automation(fpa) if __name__ == '__main__': app = GUI.AWAApp() app.MainLoop() Thank you for your attention. -- http://mail.python.org/mailman/listinfo/python-list
Re: control CPU usage
kakarukeys wrote: On Sep 20, 6:24 pm, Dave Angel da...@ieee.org wrote: Jiang Fung Wong wrote: Dear All, Thank you for the information. I think I've some idea what the problem is about after seeing the replies. More information about my system and my script PIII 1Ghz, 512MB RAM, Windows XP SP3 The script monitors global input using PyHook, and calculates on the information collected from the events to output some numbers. Based on the numbers, the script then performs some automation using SendKeys module. here is the memory usage: firefox.exe, 69MB, 109MB svchost.exe, 26MB, 17MB pythonw.exe, 22MB, 17MB searchindexer.exe, 16MB, 19MB My first guess is that the script calculated for too long time after receiving an event before propagating it to the default handler, resulting the system to be non-responsive. I will try to implement the calculation part in another thread. Then the separate will have 100% CPU usage, hope the task scheduling of Windows works in my favour. (You top-posted this message, putting the whole stream out of order. So I deleted the history.) All my assumptions about your environment are now invalid. You don't have a CPU-bound application, you have a Windows application with event loop. Further, you're using SendKeys to generate a keystroke to the other process. So there are many things that could be affecting your latency, and all my previous guesses are useless. Adding threads to your application will probably slow the system down much more. You need to find out what your present problem is before complicating it. You haven't really described the problem. You say the system is unresponsive, but you made it that way by creating a global hook; a notoriously inefficient mechanism. That global hook inserts code into every process in the system, and you've got a pretty low-end environment to begin with. So what's the real problem, and how severe is it? And how will you measure improvement? The Task manager numbers are probably irrelevant. My first question is whether the pyHook event is calling the SendKeys function directly (after your lengthy calculation) or whether there are other events firing off in between. If it's all being done in the one event, then measure its time, and gather some statistics (min time, max time, average...). The task manager has far too simplistic visibility to be useful for this purpose. What else is this application doing when it's waiting for a pyHook call? Whose event loop implementation are you using? And the program you're trying to control -- is there perhaps another way in? DaveA Hi, Sorry I wasn't sure how to use Google groups to post a msg to the newsgroup, I used Gmail to write my previous reply. What you and the other guy have provided me isn't useless. Now I understand the non- responsiveness may not be caused by high CPU usage, as the OS, be it Windows or Linux, has a way to prioritize the tasks. This is a vital clue to me. By not responsive, I mean, for some time, the mouse pointer is not moving smoothly, to such extent that I can't do anything with the mouse. It's like playing a multi-player game on a connection with a lot of lag. It's not caused by global hook, because it happens under certain condition, i.e. when fpa.ProcessEvent(word) is computing. I included my main script for your reference. Comments: (1) The automation method tc.Auto() is slow, but it doesn't cause any problem, because the user would wait for the automation to finish, before he continues to do something. (2) all other methods invoked are fast, except fpa.ProcessEvent(word) (this information is obtained from profiling). It is this method that causes 100% CPU usage. I'm planning to move this method to a separate thread, so that OnEvent(event) can finish executing, while the separate thread goes on to finish its calculation. Is this a good idea? import pyHook import TypingAnalyzer import GUI def OnEvent(event): if hasattr(event, Key) and event.Ascii == 9 and event.Key == Tab and event.Injected == 0 and event.Alt == 0: tc.Auto() return False else: recognized = rk.ProcessEvent(event) if recognized: tc.MatchChar(recognized) paragraph = rc.ProcessEvent(recognized) if paragraph: for word in paragraph: fpa.ProcessEvent(word) return True hm = pyHook.HookManager() hm.MouseAllButtonsDown = OnEvent hm.KeyDown = OnEvent hm.HookMouse() hm.HookKeyboard() rk = TypingAnalyzer.ReadKey() rc = TypingAnalyzer.ReadChar() fpa = TypingAnalyzer.Analysis() tc = TypingAnalyzer.Automation(fpa) if __name__ == '__main__': app = GUI.AWAApp() app.MainLoop() Thank you for your attention. I can't readily comment on your code, since it's entirely
Re: control CPU usage
On Sep 20, 10:57 pm, Dave Angel da...@ieee.org wrote: kakarukeys wrote: On Sep 20, 6:24 pm, Dave Angel da...@ieee.org wrote: Jiang Fung Wong wrote: Dear All, Thank you for the information. I think I've some idea what the problem is about after seeing the replies. More information about my system and my script PIII 1Ghz, 512MB RAM, Windows XP SP3 The script monitors global input using PyHook, and calculates on the information collected from the events to output some numbers. Based on the numbers, the script then performs some automation using SendKeys module. here is the memory usage: firefox.exe, 69MB, 109MB svchost.exe, 26MB, 17MB pythonw.exe, 22MB, 17MB searchindexer.exe, 16MB, 19MB My first guess is that the script calculated for too long time after receiving an event before propagating it to the default handler, resulting the system to be non-responsive. I will try to implement the calculation part in another thread. Then the separate will have 100% CPU usage, hope the task scheduling of Windows works in my favour. (You top-posted this message, putting the whole stream out of order. So I deleted the history.) All my assumptions about your environment are now invalid. You don't have a CPU-bound application, you have a Windows application with event loop. Further, you're using SendKeys to generate a keystroke to the other process. So there are many things that could be affecting your latency, and all my previous guesses are useless. Adding threads to your application will probably slow the system down much more. You need to find out what your present problem is before complicating it. You haven't really described the problem. You say the system is unresponsive, but you made it that way by creating a global hook; a notoriously inefficient mechanism. That global hook inserts code into every process in the system, and you've got a pretty low-end environment to begin with. So what's the real problem, and how severe is it? And how will you measure improvement? The Task manager numbers are probably irrelevant. My first question is whether the pyHook event is calling the SendKeys function directly (after your lengthy calculation) or whether there are other events firing off in between. If it's all being done in the one event, then measure its time, and gather some statistics (min time, max time, average...). The task manager has far too simplistic visibility to be useful for this purpose. What else is this application doing when it's waiting for a pyHook call? Whose event loop implementation are you using? And the program you're trying to control -- is there perhaps another way in? DaveA Hi, Sorry I wasn't sure how to use Google groups to post a msg to the newsgroup, I used Gmail to write my previous reply. What you and the other guy have provided me isn't useless. Now I understand the non- responsiveness may not be caused by high CPU usage, as the OS, be it Windows or Linux, has a way to prioritize the tasks. This is a vital clue to me. By not responsive, I mean, for some time, the mouse pointer is not moving smoothly, to such extent that I can't do anything with the mouse. It's like playing a multi-player game on a connection with a lot of lag. It's not caused by global hook, because it happens under certain condition, i.e. when fpa.ProcessEvent(word) is computing. I included my main script for your reference. Comments: (1) The automation method tc.Auto() is slow, but it doesn't cause any problem, because the user would wait for the automation to finish, before he continues to do something. (2) all other methods invoked are fast, except fpa.ProcessEvent(word) (this information is obtained from profiling). It is this method that causes 100% CPU usage. I'm planning to move this method to a separate thread, so that OnEvent(event) can finish executing, while the separate thread goes on to finish its calculation. Is this a good idea? import pyHook import TypingAnalyzer import GUI def OnEvent(event): if hasattr(event, Key) and event.Ascii == 9 and event.Key == Tab and event.Injected == 0 and event.Alt == 0: tc.Auto() return False else: recognized = rk.ProcessEvent(event) if recognized: tc.MatchChar(recognized) paragraph = rc.ProcessEvent(recognized) if paragraph: for word in paragraph: fpa.ProcessEvent(word) return True hm = pyHook.HookManager() hm.MouseAllButtonsDown = OnEvent hm.KeyDown = OnEvent hm.HookMouse() hm.HookKeyboard() rk = TypingAnalyzer.ReadKey() rc = TypingAnalyzer.ReadChar() fpa = TypingAnalyzer.Analysis() tc = TypingAnalyzer.Automation(fpa) if __name__ == '__main__': app =
Re: control CPU usage
kakarukeys wrote: On Sep 20, 10:57 pm, Dave Angel da...@ieee.org wrote: kakarukeys wrote: On Sep 20, 6:24 pm, Dave Angel da...@ieee.org wrote: Jiang Fung Wong wrote: Dear All, Thank you for the information. I think I've some idea what the problem is about after seeing the replies. More information about my system and my script PIII 1Ghz, 512MB RAM, Windows XP SP3 The script monitors global input using PyHook, and calculates on the information collected from the events to output some numbers. Based on the numbers, the script then performs some automation using SendKeys module. here is the memory usage: firefox.exe, 69MB, 109MB svchost.exe, 26MB, 17MB pythonw.exe, 22MB, 17MB searchindexer.exe, 16MB, 19MB My first guess is that the script calculated for too long time after receiving an event before propagating it to the default handler, resulting the system to be non-responsive. I will try to implement the calculation part in another thread. Then the separate will have 100% CPU usage, hope the task scheduling of Windows works in my favour. (You top-posted this message, putting the whole stream out of order. So I deleted the history.) All my assumptions about your environment are now invalid. You don't have a CPU-bound application, you have a Windows application with event loop. Further, you're using SendKeys to generate a keystroke to the other process. So there are many things that could be affecting your latency, and all my previous guesses are useless. Adding threads to your application will probably slow the system down much more. You need to find out what your present problem is before complicating it. You haven't really described the problem. You say the system is unresponsive, but you made it that way by creating a global hook; a notoriously inefficient mechanism. That global hook inserts code into every process in the system, and you've got a pretty low-end environment to begin with. So what's the real problem, and how severe is it? And how will you measure improvement? The Task manager numbers are probably irrelevant. My first question is whether the pyHook event is calling the SendKeys function directly (after your lengthy calculation) or whether there are other events firing off in between. If it's all being done in the one event, then measure its time, and gather some statistics (min time, max time, average...). The task manager has far too simplistic visibility to be useful for this purpose. What else is this application doing when it's waiting for a pyHook call? Whose event loop implementation are you using? And the program you're trying to control -- is there perhaps another way in? DaveA Hi, Sorry I wasn't sure how to use Google groups to post a msg to the newsgroup, I used Gmail to write my previous reply. What you and the other guy have provided me isn't useless. Now I understand the non- responsiveness may not be caused by high CPU usage, as the OS, be it Windows or Linux, has a way to prioritize the tasks. This is a vital clue to me. By not responsive, I mean, for some time, the mouse pointer is not moving smoothly, to such extent that I can't do anything with the mouse. It's like playing a multi-player game on a connection with a lot of lag. It's not caused by global hook, because it happens under certain condition, i.e. when fpa.ProcessEvent(word) is computing. I included my main script for your reference. Comments: (1) The automation method tc.Auto() is slow, but it doesn't cause any problem, because the user would wait for the automation to finish, before he continues to do something. (2) all other methods invoked are fast, except fpa.ProcessEvent(word) (this information is obtained from profiling). It is this method that causes 100% CPU usage. I'm planning to move this method to a separate thread, so that OnEvent(event) can finish executing, while the separate thread goes on to finish its calculation. Is this a good idea? import pyHook import TypingAnalyzer import GUI def OnEvent(event): if hasattr(event, Key) and event.Ascii =9 and event.Key == Tab and event.Injected =0 and event.Alt == 0: tc.Auto() return False else: recognized =k.ProcessEvent(event) if recognized: tc.MatchChar(recognized) paragraph =c.ProcessEvent(recognized) if paragraph: for word in paragraph: fpa.ProcessEvent(word) return True hm =yHook.HookManager() hm.MouseAllButtonsDown =nEvent hm.KeyDown =nEvent hm.HookMouse() hm.HookKeyboard() rk =ypingAnalyzer.ReadKey() rc =ypingAnalyzer.ReadChar() fpa =TypingAnalyzer.Analysis() tc =ypingAnalyzer.Automation(fpa) if __name__
control CPU usage
Hi, When I am running a loop for a long time, calculating heavily, the CPU usage is at 100%, making the comp not so responsive. Is there a way to control the CPU usage at say 80%? putting a time.sleep(0.x) doesn't seem to help although CPU usage level is reduced, but it's unstable. Regards, W.J.F. -- http://mail.python.org/mailman/listinfo/python-list
Re: control CPU usage
On Sep 19, 9:17 am, kakarukeys kakaruk...@gmail.com wrote: Hi, When I am running a loop for a long time, calculating heavily, the CPU usage is at 100%, making the comp not so responsive. Is there a way to control the CPU usage at say 80%? putting a time.sleep(0.x) doesn't seem to help although CPU usage level is reduced, but it's unstable. Regards, W.J.F. If you are on linux, you can use the 'nice' command. It will still take 100%, but will give it up to other processes that need to use the cpu. nice -n 19 script In windows, I guess you could change the priority of the process in the Task Manager. Not sure how to do it programatically. ~Sean -- http://mail.python.org/mailman/listinfo/python-list
Re: control CPU usage
kakarukeys wrote: Hi, When I am running a loop for a long time, calculating heavily, the CPU usage is at 100%, making the comp not so responsive. Is there a way to control the CPU usage at say 80%? putting a time.sleep(0.x) doesn't seem to help although CPU usage level is reduced, but it's unstable. Regards, W.J.F. Controlling a task's scheduling is most definitely OS-dependent., so you need to say what OS you're running on. And whether it's a multi-core and or duo processor. In Windows, there is a generic way to tell the system that you want to give a boost to whatever task has the user focus (generally the top-window on the desktop). On some versions, that's the default, on others, it's not. You change it from Control Panel. I'd have to go look to tell you what applet, but I don't even know if you're on Windows. In addition, a program can adjust its own priority, much the way the Unix 'nice' command works. You'd use the Win32 library for that. And as you already tried, you can add sleep() operations to your application. But if you're looking at the task list in the Windows Task Manager, you aren't necessarily going to see what you apparently want. There's no way to programmatically tell the system to use a certain percentage for a given task. If there's nothing else to do, then a low priority task is still going to get nearly 100% of the CPU. Good thing. But even if there are other things to do, the scheduling is a complex interaction between what kinds of work the various processes have been doing lately, how much memory load they have, and what priority they're assigned. If you just want other processes to be responsive when they've got the focus, you may want to make that global setting. But you may need to better define responsive and unstable. DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: control CPU usage
Dear All, Thank you for the information. I think I've some idea what the problem is about after seeing the replies. More information about my system and my script PIII 1Ghz, 512MB RAM, Windows XP SP3 The script monitors global input using PyHook, and calculates on the information collected from the events to output some numbers. Based on the numbers, the script then performs some automation using SendKeys module. here is the memory usage: firefox.exe, 69MB, 109MB svchost.exe, 26MB, 17MB pythonw.exe, 22MB, 17MB searchindexer.exe, 16MB, 19MB My first guess is that the script calculated for too long time after receiving an event before propagating it to the default handler, resulting the system to be non-responsive. I will try to implement the calculation part in another thread. Then the separate will have 100% CPU usage, hope the task scheduling of Windows works in my favour. On Sun, Sep 20, 2009 at 5:22 AM, Dave Angel da...@ieee.org wrote: kakarukeys wrote: Hi, When I am running a loop for a long time, calculating heavily, the CPU usage is at 100%, making the comp not so responsive. Is there a way to control the CPU usage at say 80%? putting a time.sleep(0.x) doesn't seem to help although CPU usage level is reduced, but it's unstable. Regards, W.J.F. Controlling a task's scheduling is most definitely OS-dependent., so you need to say what OS you're running on. And whether it's a multi-core and or duo processor. In Windows, there is a generic way to tell the system that you want to give a boost to whatever task has the user focus (generally the top-window on the desktop). On some versions, that's the default, on others, it's not. You change it from Control Panel. I'd have to go look to tell you what applet, but I don't even know if you're on Windows. In addition, a program can adjust its own priority, much the way the Unix 'nice' command works. You'd use the Win32 library for that. And as you already tried, you can add sleep() operations to your application. But if you're looking at the task list in the Windows Task Manager, you aren't necessarily going to see what you apparently want. There's no way to programmatically tell the system to use a certain percentage for a given task. If there's nothing else to do, then a low priority task is still going to get nearly 100% of the CPU. Good thing. But even if there are other things to do, the scheduling is a complex interaction between what kinds of work the various processes have been doing lately, how much memory load they have, and what priority they're assigned. If you just want other processes to be responsive when they've got the focus, you may want to make that global setting. But you may need to better define responsive and unstable. DaveA -- http://mail.python.org/mailman/listinfo/python-list
