Re: [Webware-discuss] RuntimeError: dictionary changed size during iteration
My tasks do not access the sessions. (Why would they?) As it is a new behaviour, (I was using 0.9) a look at the changes could help... Could it be the scheduler instances sharing some global task lists? On Fri, Dec 7, 2012 at 2:38 PM, Christoph Zwerschke c...@online.de wrote: Am 07.12.2012 12:16, schrieb Sophana K: Yes, I'm creating a Scheduler instance. I didn't know I was supposed to use the one from the application. It's ok to create your own instance. But then the two schedulers will run as two parallel threads which needs more ressources and you can get these race conditions if both schedulers have tasks which access the session store. It's also convenient to use the application.taskManager() because it's started and stopped automatically together with the application. Is it normal that a scheduler instance automatically does session sweeping? Shouldn't the application create the task explicitely? But that's excactly how it works. The application creates its own scheduler, and then adds just one task, the one which is used for session sweeping. You can replace that task with your own or add more tasks if you like. I have now fixed my code to use the existing scheduler. Shouldn't you remove the previous fixes then? The fixes are still helpful in case anybody changes the session store for whatever maybe valid reasons. It's still unclear to me what caused your problems: Do your own tasks modify the session store? Then that's the explanation, because you ran them in a parallel scheduler. If your own tasks don't change the session, then you must continue searching for the error cause. It looks like the session sweeper thread somehow was duplicated, but how this happens is unclear to me without seeing your code. If you just create your own Scheduler instance, this should not happen. -- Christoph -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d ___ Webware-discuss mailing list Webware-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/webware-discuss -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d___ Webware-discuss mailing list Webware-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/webware-discuss
Re: [Webware-discuss] RuntimeError: dictionary changed size during iteration
Am 08.12.2012 09:51, schrieb Sophana K: My tasks do not access the sessions. (Why would they?) As it is a new behaviour, (I was using 0.9) a look at the changes could help... In 0.9 and 1.0 the session sweeper loops run over the keys list, not over the dictionaries, simply because older Python versions did not support that. So the problem with duplicate session sweepers might well have existed, you just didn't notice it. Could it be the scheduler instances sharing some global task lists? I thought about that too, because it would be an explanation, but no, they don't. One explanation for the duplicate session sweeping could be that your tasks inherit from WebKit.Tasks.SessionTask instead of TaskKit.Task. Or that you inherit from Application and run __init__ twice. To debug the problem, you can put some print statement in the __init__() and run() methods of SessionTask and see when it is instantiated (should be instantiated once at the start) and how often it is run (should run once every 10 minutes, depending on your SessionTimeout). -- Christoph -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d ___ Webware-discuss mailing list Webware-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/webware-discuss
Re: [Webware-discuss] RuntimeError: dictionary changed size during iteration
Yes, I'm creating a Scheduler instance. I didn't know I was supposed to use the one from the application. Is it normal that a scheduler instance automatically does session sweeping? Shouldn't the application create the task explicitely? I have now fixed my code to use the existing scheduler. Shouldn't you remove the previous fixes then? Thanks for the hint. On Thu, Dec 6, 2012 at 12:23 PM, Christoph Zwerschke c...@online.de wrote: Am 06.12.2012 10:59, schrieb Sophana K: I have some tasks scheduled every 30m and 24h. I did nothing special about the session sweeper. Are they related to taskKit? The session sweeper thread is started by the Application instance using its own instance of the TaskKit scheduler, available as Application.taskManager(). Do you use that scheduler instance or do you use your own? To me it looks as if you're either running the session sweeper twice or one of your tasks is doing something with the session store of the application. Otherwise you shouldn't see that error. -- Christoph -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d ___ Webware-discuss mailing list Webware-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/webware-discuss -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d___ Webware-discuss mailing list Webware-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/webware-discuss
Re: [Webware-discuss] RuntimeError: dictionary changed size during iteration
Am 07.12.2012 12:16, schrieb Sophana K: Yes, I'm creating a Scheduler instance. I didn't know I was supposed to use the one from the application. It's ok to create your own instance. But then the two schedulers will run as two parallel threads which needs more ressources and you can get these race conditions if both schedulers have tasks which access the session store. It's also convenient to use the application.taskManager() because it's started and stopped automatically together with the application. Is it normal that a scheduler instance automatically does session sweeping? Shouldn't the application create the task explicitely? But that's excactly how it works. The application creates its own scheduler, and then adds just one task, the one which is used for session sweeping. You can replace that task with your own or add more tasks if you like. I have now fixed my code to use the existing scheduler. Shouldn't you remove the previous fixes then? The fixes are still helpful in case anybody changes the session store for whatever maybe valid reasons. It's still unclear to me what caused your problems: Do your own tasks modify the session store? Then that's the explanation, because you ran them in a parallel scheduler. If your own tasks don't change the session, then you must continue searching for the error cause. It looks like the session sweeper thread somehow was duplicated, but how this happens is unclear to me without seeing your code. If you just create your own Scheduler instance, this should not happen. -- Christoph -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d ___ Webware-discuss mailing list Webware-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/webware-discuss
Re: [Webware-discuss] RuntimeError: dictionary changed size during iteration
I have some tasks scheduled every 30m and 24h. I did nothing special about the session sweeper. Are they related to taskKit? On Fri, Nov 30, 2012 at 6:28 PM, Christoph Zwerschke c...@online.de wrote: Am 30.11.2012 18:01, schrieb Sophana K: Isn't it strange nobody had this problem before? Seems that there isn't a lot of webware 1.1 applications in production. Don't you think? That problem should not appear in practice unless you have somehow two session sweeper tasks running at the same time. Are you doing anything special in this regard? I'm running Webware 1.1 apps with many users which use sessions heavily, and I've never seen this issue. -- Christoph (Sorry, my mailer just sent an empty reply.) -- Keep yourself connected to Go Parallel: TUNE You got it built. Now make it sing. Tune shows you how. http://goparallel.sourceforge.net ___ Webware-discuss mailing list Webware-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/webware-discuss -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d___ Webware-discuss mailing list Webware-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/webware-discuss
Re: [Webware-discuss] RuntimeError: dictionary changed size during iteration
It seems the problem has now moved to another place. Should I try the trunk? File /home/.../Webware-1.1/WebKit/Tasks/SessionTask.py, line 13, in run self._sessionstore.cleanStaleSessions(self) File /home/.../Webware-1.1/WebKit/SessionDynamicStore.py, line 256, in cleanStaleSessions self._memoryStore.cleanStaleSessions(task) File /home/.../Webware-1.1/WebKit/SessionStore.py, line 202, in cleanStaleSessions for key in self: RuntimeError: dictionary changed size during iteration On Wed, Nov 28, 2012 at 12:56 PM, Christoph Zwerschke c...@online.dewrote: Am 28.11.2012 12:10, schrieb Sophana K: Look like another thread could be changing the _memoryStore during this iteration. Should I change to for key in list(self._memoryStore): Yes, this or for key in self._memoryStore.keys() should fix it. I have already committed this as a fix to the trunk. -- Chris -- Keep yourself connected to Go Parallel: INSIGHTS What's next for parallel hardware, programming and related areas? Interviews and blogs by thought leaders keep you ahead of the curve. http://goparallel.sourceforge.net ___ Webware-discuss mailing list Webware-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/webware-discuss -- Keep yourself connected to Go Parallel: TUNE You got it built. Now make it sing. Tune shows you how. http://goparallel.sourceforge.net___ Webware-discuss mailing list Webware-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/webware-discuss
Re: [Webware-discuss] RuntimeError: dictionary changed size during iteration
Hi Sophana, hi Christoph, the webware for python is used in two companies, on 4 Servers, I work for. It is running very stable. But however we use a self-implemented session. Thats why it seems, I didn't mentioned this problem to Christoph. But our claims, we have to the session is a little bit bigger one that the one from webware has and it was easier to build an own, derived one. However, Webkit is pretty stable, there is no need not to take it, because Christoph very cares about all. Thank you Christoph btw. for all your help and support everytime so far! Best Regards, Tobias Am 30.11.2012 18:01, schrieb Sophana K: Thanks Isn't it strange nobody had this problem before? Seems that there isn't a lot of webware 1.1 applications in production. Don't you think? Sophana On Fri, Nov 30, 2012 at 5:36 PM, Christoph Zwerschke c...@online.de mailto:c...@online.de wrote: Am 30.11.2012 13:48, schrieb Sophana K: It seems the problem has now moved to another place. Should I try the trunk? File /home/.../Webware-1.1/WebKit/Tasks/SessionTask.py, line 13, in run self._sessionstore.cleanStaleSessions(self) File /home/.../Webware-1.1/WebKit/SessionDynamicStore.py, line 256, in cleanStaleSessions self._memoryStore.cleanStaleSessions(task) File /home/.../Webware-1.1/WebKit/SessionStore.py, line 202, in cleanStaleSessions for key in self: RuntimeError: dictionary changed size during iteration Same problem. It's now also fixed in the trunk. -- Christoph -- Keep yourself connected to Go Parallel: TUNE You got it built. Now make it sing. Tune shows you how. http://goparallel.sourceforge.net ___ Webware-discuss mailing list Webware-discuss@lists.sourceforge.net mailto:Webware-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/webware-discuss -- Keep yourself connected to Go Parallel: TUNE You got it built. Now make it sing. Tune shows you how. http://goparallel.sourceforge.net ___ Webware-discuss mailing list Webware-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/webware-discuss -- Keep yourself connected to Go Parallel: TUNE You got it built. Now make it sing. Tune shows you how. http://goparallel.sourceforge.net___ Webware-discuss mailing list Webware-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/webware-discuss
Re: [Webware-discuss] RuntimeError: dictionary changed size during iteration
Am 30.11.2012 18:01, schrieb Sophana K: Isn't it strange nobody had this problem before? Seems that there isn't a lot of webware 1.1 applications in production. Don't you think? T -- Keep yourself connected to Go Parallel: TUNE You got it built. Now make it sing. Tune shows you how. http://goparallel.sourceforge.net ___ Webware-discuss mailing list Webware-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/webware-discuss
Re: [Webware-discuss] RuntimeError: dictionary changed size during iteration
Am 30.11.2012 18:01, schrieb Sophana K: Isn't it strange nobody had this problem before? Seems that there isn't a lot of webware 1.1 applications in production. Don't you think? That problem should not appear in practice unless you have somehow two session sweeper tasks running at the same time. Are you doing anything special in this regard? I'm running Webware 1.1 apps with many users which use sessions heavily, and I've never seen this issue. -- Christoph (Sorry, my mailer just sent an empty reply.) -- Keep yourself connected to Go Parallel: TUNE You got it built. Now make it sing. Tune shows you how. http://goparallel.sourceforge.net ___ Webware-discuss mailing list Webware-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/webware-discuss
Re: [Webware-discuss] RuntimeError: dictionary changed size during iteration
Am 28.11.2012 12:10, schrieb Sophana K: Look like another thread could be changing the _memoryStore during this iteration. Should I change to for key in list(self._memoryStore): Yes, this or for key in self._memoryStore.keys() should fix it. I have already committed this as a fix to the trunk. -- Chris -- Keep yourself connected to Go Parallel: INSIGHTS What's next for parallel hardware, programming and related areas? Interviews and blogs by thought leaders keep you ahead of the curve. http://goparallel.sourceforge.net ___ Webware-discuss mailing list Webware-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/webware-discuss
