Re: [Webware-discuss] RuntimeError: dictionary changed size during iteration
My tasks do not access the sessions. (Why would they?) As it is a new behaviour, (I was using 0.9) a look at the changes could help... Could it be the scheduler instances sharing some global task lists? On Fri, Dec 7, 2012 at 2:38 PM, Christoph Zwerschke c...@online.de wrote: Am 07.12.2012 12:16, schrieb Sophana K: Yes, I'm creating a Scheduler instance. I didn't know I was supposed to use the one from the application. It's ok to create your own instance. But then the two schedulers will run as two parallel threads which needs more ressources and you can get these race conditions if both schedulers have tasks which access the session store. It's also convenient to use the application.taskManager() because it's started and stopped automatically together with the application. Is it normal that a scheduler instance automatically does session sweeping? Shouldn't the application create the task explicitely? But that's excactly how it works. The application creates its own scheduler, and then adds just one task, the one which is used for session sweeping. You can replace that task with your own or add more tasks if you like. I have now fixed my code to use the existing scheduler. Shouldn't you remove the previous fixes then? The fixes are still helpful in case anybody changes the session store for whatever maybe valid reasons. It's still unclear to me what caused your problems: Do your own tasks modify the session store? Then that's the explanation, because you ran them in a parallel scheduler. If your own tasks don't change the session, then you must continue searching for the error cause. It looks like the session sweeper thread somehow was duplicated, but how this happens is unclear to me without seeing your code. If you just create your own Scheduler instance, this should not happen. -- Christoph -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d ___ Webware-discuss mailing list Webware-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/webware-discuss -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d___ Webware-discuss mailing list Webware-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/webware-discuss
Re: [Webware-discuss] RuntimeError: dictionary changed size during iteration
Yes, I'm creating a Scheduler instance. I didn't know I was supposed to use the one from the application. Is it normal that a scheduler instance automatically does session sweeping? Shouldn't the application create the task explicitely? I have now fixed my code to use the existing scheduler. Shouldn't you remove the previous fixes then? Thanks for the hint. On Thu, Dec 6, 2012 at 12:23 PM, Christoph Zwerschke c...@online.de wrote: Am 06.12.2012 10:59, schrieb Sophana K: I have some tasks scheduled every 30m and 24h. I did nothing special about the session sweeper. Are they related to taskKit? The session sweeper thread is started by the Application instance using its own instance of the TaskKit scheduler, available as Application.taskManager(). Do you use that scheduler instance or do you use your own? To me it looks as if you're either running the session sweeper twice or one of your tasks is doing something with the session store of the application. Otherwise you shouldn't see that error. -- Christoph -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d ___ Webware-discuss mailing list Webware-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/webware-discuss -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d___ Webware-discuss mailing list Webware-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/webware-discuss
Re: [Webware-discuss] RuntimeError: dictionary changed size during iteration
I have some tasks scheduled every 30m and 24h. I did nothing special about the session sweeper. Are they related to taskKit? On Fri, Nov 30, 2012 at 6:28 PM, Christoph Zwerschke c...@online.de wrote: Am 30.11.2012 18:01, schrieb Sophana K: Isn't it strange nobody had this problem before? Seems that there isn't a lot of webware 1.1 applications in production. Don't you think? That problem should not appear in practice unless you have somehow two session sweeper tasks running at the same time. Are you doing anything special in this regard? I'm running Webware 1.1 apps with many users which use sessions heavily, and I've never seen this issue. -- Christoph (Sorry, my mailer just sent an empty reply.) -- Keep yourself connected to Go Parallel: TUNE You got it built. Now make it sing. Tune shows you how. http://goparallel.sourceforge.net ___ Webware-discuss mailing list Webware-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/webware-discuss -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d___ Webware-discuss mailing list Webware-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/webware-discuss
Re: [Webware-discuss] RuntimeError: dictionary changed size during iteration
It seems the problem has now moved to another place. Should I try the trunk? File /home/.../Webware-1.1/WebKit/Tasks/SessionTask.py, line 13, in run self._sessionstore.cleanStaleSessions(self) File /home/.../Webware-1.1/WebKit/SessionDynamicStore.py, line 256, in cleanStaleSessions self._memoryStore.cleanStaleSessions(task) File /home/.../Webware-1.1/WebKit/SessionStore.py, line 202, in cleanStaleSessions for key in self: RuntimeError: dictionary changed size during iteration On Wed, Nov 28, 2012 at 12:56 PM, Christoph Zwerschke c...@online.dewrote: Am 28.11.2012 12:10, schrieb Sophana K: Look like another thread could be changing the _memoryStore during this iteration. Should I change to for key in list(self._memoryStore): Yes, this or for key in self._memoryStore.keys() should fix it. I have already committed this as a fix to the trunk. -- Chris -- Keep yourself connected to Go Parallel: INSIGHTS What's next for parallel hardware, programming and related areas? Interviews and blogs by thought leaders keep you ahead of the curve. http://goparallel.sourceforge.net ___ Webware-discuss mailing list Webware-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/webware-discuss -- Keep yourself connected to Go Parallel: TUNE You got it built. Now make it sing. Tune shows you how. http://goparallel.sourceforge.net___ Webware-discuss mailing list Webware-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/webware-discuss
[Webware-discuss] RuntimeError: dictionary changed size during iteration
Hi I just upgraded to webware-1.1, and get some errors (about every day) in the session dynamic store intervalSweep method. Here is the traceback File /home/sophana/src/env1/Webware-1.1/WebKit/Tasks/SessionTask.py, line 13, in run self._sessionstore.cleanStaleSessions(self) File /home/sophana/src/env1/Webware-1.1/WebKit/SessionDynamicStore.py, line 264, in cleanStaleSessions self.intervalSweep() File /home/sophana/src/env1/Webware-1.1/WebKit/SessionDynamicStore.py, line 288, in intervalSweep for key in self._memoryStore: RuntimeError: dictionary changed size during iteration Look like another thread could be changing the _memoryStore during this iteration. Should I change to for key in list(self._memoryStore): ? Regards Sophana -- Keep yourself connected to Go Parallel: INSIGHTS What's next for parallel hardware, programming and related areas? Interviews and blogs by thought leaders keep you ahead of the curve. http://goparallel.sourceforge.net___ Webware-discuss mailing list Webware-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/webware-discuss
Re: [Webware-discuss] webware gevent based appServer
I will give a try again with webware 1.1 It works, but not for a long time... python3 doesn't look like a priority for me. (I admit I had some hard time with unicode...) Backward compatibility for webkit is the most important IMHO. About naming convention, I don't care that much. I'm not using plugins. A little bit of TaskKit (not a big deal) Migrating to more modern server architectures is a big plus for me: more scalability, comet, etc... If it is possible to just use WebKit compatibility in order to integrate more easily with other python libraries, it would be great. Sophana On Wed, Sep 19, 2012 at 8:07 PM, Christoph Zwerschke c...@online.de wrote: Am 19.09.2012 18:13, schrieb NoRaGen: It would be nice to have the option to upgrade to a python3.x Version of Webware for later purpose. Definitely, that has also been one of my goals for a new version. Compatiblity should be given so, that API remains the same, but using python3.x capability. This should be possible, I think. API changes would very bad at all. The problem here is that Webware development started even before Python 2, so the old API is very old-fashioned concerning naming conventions, use of getters and setters etc. My idea was to add a tool like py2to3 that can convert an old style Webware app to a new style app. Another idea would be to allow both styles. E.g. request could become a property, but if you call the request object, then it returns self, so if you use request as a getter function, it would still work. We already discussed this in the past. Method names could be changed from camelCase to PEP8 style, but the old names could be kept as aliases. However we use many modules where still no python3 port is existing. For now, switching to higher python version, would still not effective for us. Yes, e.g. ReportLab is still one of the roadblockers. -- Chris -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Webware-discuss mailing list Webware-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/webware-discuss -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://ad.doubleclick.net/clk;258768047;13503038;j? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Webware-discuss mailing list Webware-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/webware-discuss
[Webware-discuss] webware gevent based appServer
Hi I'm using webware for my website since 2007. I'm using webware 1.0, because webware 1.1 didn't work correctly for me (I don't remember why) I'm currently trying to debug a freeze problem that occur every 1 to 3 weeks. The python appServer process is completely frozen, and must be killed with -9. It doesn't respond to any signal, so I can't dump its stack, and really don't know the cause strace shows that python is waiting on a futex. I still have some actions to do in my investigations like: - rebooting the server which is up since 620 days... - separating the custom radius server I made which is launched in the same process in several new threads. (I didn't know about the GIL at that time...) - reverting to webware 0.9 I intend to use gevent for the radius server, because I like it. I would also like to use comet services for the web application, which brings me to my question: I've seen that there is a WSGI adapter that connects to the appServer. Is it possible to wrap a webware application in a real WSGI handler? This would allow to use webware applications under modern servers like gevent (gunicorn) and why not google app engine. The threaded app server is no more adapted to modern web techniques. A gevent based appServer would be so great! libraries like gevent-zeromq and gevent-socketio would be so nice to have. Gevent is able to wrap standard python libraries (monkey patching) and make standard code greenlet compatible, so I'm not sure the wsgi wrapper would be complicated to do. I have started to look inside the appServer. I don't know where to start. There are quite some code that initializes different modules and things like that. In gevent, the idea is to spawn a greenlet for every new transaction. I'm not sure there is a pool of workers. Thanks for reading. -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Webware-discuss mailing list Webware-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/webware-discuss
