Re: ANNOUNCE: Gazest 0.3.7 -- a Pylons powered wiki
On 10/18/07, Yannick Gingras [EMAIL PROTECTED] wrote:
Greetings Pyloneers,
I'm am please to announce the release of Gazest 0.3.7, a Pylons
powered wiki.
[snip]
Both the idea and the demo looks cool!
I tried it using easy_install, but failed with python-openid version
conflicted:
gazest-god-mode
Traceback (most recent call last):
File /home/number5/bin/gazest-god-mode, line 5, in ?
from pkg_resources import load_entry_point
File
/home/number5/lib/python2.4/site-packages/setuptools-0.6c7-py2.4.egg/pkg_resources.py,
line 2564, in ?
for dist in working_set.resolve(
File
/home/number5/lib/python2.4/site-packages/setuptools-0.6c7-py2.4.egg/pkg_resources.py,
line 528, in resolve
raise VersionConflict(dist,req) # XXX put more info here
pkg_resources.VersionConflict: (python-openid
2.0.1(/home/number5/lib/python2.4/site-packages/python_openid-
2.0.1-py2.4.egg), Requirement.parse('python-openid=1.1,=1.2'))
Didn't try bleeding edge yet, git looks scary ;)
BTW: Is there any docs available now?
--
simple is good
http://brucewang.net
http://twitter.com/number5
http://www.linkist.com
skype: number5
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
pylons-discuss group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/pylons-discuss?hl=en
-~--~~~~--~~--~--~---
Re: ANNOUNCE: Gazest 0.3.7 -- a Pylons powered wiki
Bruce Wang [EMAIL PROTECTED] writes: Both the idea and the demo looks cool! Thanks I tried it using easy_install, but failed with python-openid version conflicted: [sniped] This is quite annoying and I'm note sure how to solve the problem. Maybe someone in here have an idea. Your python-openid is too recent. If you don't have it installed, setuptools will pick a suitable version; but if you have a too recent version installed, it will just say that it's too recent and it won't fetch an older version. This is really pissing me off because I wanted to postpone the switch to Alchemy 0.4 a bit but everyone and their dog is going to have Alchemy 0.4 on disk now that it's stable. My only solution would be to kick the global openid and to install it in a workingenv for the apps that needs it. The instructions to install Gazest in a workingenv are included in the tarball. Didn't try bleeding edge yet, git looks scary ;) Git is more scary than Mercurial but really not much. I use both all the time: I love them both and I could not pick a definitive winner. The Python web community seems to standardize on Mercurial; this is a really good choice but I don't want to switch until Maris release a patched versions of hg.setuptools. The current version will bork setuptools when it's outside of a Mercurial repo. No doubt that you'd hate me if I was to ask you to install a plugin that renders your setuptool unusable. I have a patched version in here: http://ygingras.net/files/hg.setuptools-0.1.2.tar.gz BTW: Is there any docs available now? There is some doc in the tarball but not much. I imagine there will be a wiki for the user doc in the near future. I'm a bit out of my app slots at webfaction but I'm working on a new hosting deal. -- Yannick Gingras --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: Reducing pylons app memory usage?
you should see if the Apache configuration you are using defines a value for the ThreadStackSize directive. Thanks for the hint. I am still to make my experiments, but here is initial summary for this thread from my point of view. It is highly suspected, that significant part of the RAM consumption is caused by the thread stacks and thread-local data. Therefore some tuning here is likely to improve things. The following paths are to be explored: 1) Keep using embedded mod_wsgi (and apache worker mode), but: - experiment with ThreadStackSize, trying to reduce it from default 8MB - tune worker threads count and processess count (standard apache worker parameters) - check whether there are any unnecessary apache modules loaded - as a safety measure, set some MaxRequestsPerChild 2) Consider (and test) mod_wsgi daemon mode. Main advantage over embedded mode is that it will allow to tune wsgi threads (count, stack size...) separately from main apache threads. 3) Serve content using standalone paste, either directly, or using reverse proxy [server:main] threadpool_nworkers = 5 to tune worker threads count (default is 10). In such case, nginx is to be explored as an alternative for apache (to serve the static content and perform reverse proxying) (I leave out the idea of using fastcgi/scgi with flup, I do not think it has any advantages over 2) and 3) in my case, similarly I leave out the idea of standalone cherrypy as I do not know about any advantages over paste) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: Reducing pylons app memory usage?
On 10/17/07, Graham Dumpleton [EMAIL PROTECTED] wrote: Note that in general the argument that lighttpd and nginx are better because they are quicker in serving static files is totally meaningless in the context of Python web applications, as the real work is being done in the Python application and that along with any database access will be your bottleneck. Unless your web application is specifically being used as a mechanism for working with large numbers of very large distinct static media files, you will not really gain anything from using lighttpd or nginx as static files isn't the bottleneck. All you will possibly do is just make your setup and configuration more complicated than it needs to be. That's not true. nginx or lighttpd are far better than Apache + mod_proxy by any metric, at least with Apache 2.0.x on FreeBSD, Mac OS X, or Linux (other platforms I have not tried myself). Also, unless you're using lots of mod_rewrite, nginx isn't any more complicated than Apache. In fact, I find it much easier because you don't have to figure out what you have to turn off. In an environment with only 96 mb available, using a web server that uses forks or threads is a really bad choice anyway. That pretty much rules Apache 2.0.x out. -bob --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: Reducing pylons app memory usage?
On 10/17/07, Ian Bicking [EMAIL PROTECTED] wrote: Ben Bangert wrote: On Oct 17, 2007, at 2:24 PM, Christoph Haas wrote: Why am I still confused by the top columns after 15 years of UNIX usage. Yes, the virtual usage is around 100 MB while the resident size is at about 18 MB. Of course, since most of this ram use is from the libraries, it means it will increase very little should you run multiple Pylons apps in a single process. You can do this with the ini file to tell Paste to load multiple apps, and even host them on different virtual hosts, that'll save you some ram if you're running multiple Pylons apps on the same machine. It'd be hot if you could load up Pylons and then fork for each app, sharing a bunch of the libraries. You can't. But such a thing is not impossible. Mmm... a whole entry point: [app:otherprocapp] use = egg:DeploySubprocess app = config:%(here)s/myapp.ini # maybe also: add_libraries = /path/to/some/virtualenv-or-workingenv/ It'd fork, use that add_libraries to do a site.addsitedir(), and maybe start up with flup's scgi or fcgi server, with s/fcgi_app on the unforked site, forwarding any requests over, and doing a little process management (just passing over signals, so that you can kill both as a unit by killing the parent). I'm guessing 200 lines of code. Who wants to write it? Note you won't notice the memory savings unless you look closely; it's hard to see shared memory. Surprisingly hard. But much of the memory will be shared between the master process and the forked subprocess. But any of those pages are going to get written to any time you touch any of those objects due to the ref count changing. If you actually those libraries, the savings should disappear pretty fast. Doing this for C code makes more sense since you don't increase the ref count on functions in C when you call them, but I think most modern OS' already share C libraries across processes using a copy-on-write strategy. -bob --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: connection reset on file upload
Finally I found the source of the problem. In case of a file upload the responsible controller is called immediately, even before the file is completely uploaded. The execution of the controller instructions is stalled until the file is uploaded completely if - request.POST['data_file'] is accessed - a template is rendered If however a string response is sent via * return this is my string response * no delay occurs. So in this case the controller sends a response even before the browser has uploaded the file. This is the reason for the connection reset message. def upload(self, environ): return this is my string response reproduces the problem on a file upload form. As its is imho valid controller code the error should not occur but the controller should be stalled at/before the return statement. regards, /ch --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: Pylons curve and surface fitting web site tutorial complete
On Oct 18, 11:24 am, zunzun [EMAIL PROTECTED] wrote: I have completed work on a new Pylons curve and surface fitting web site tutorial, the URL is; http://wiki.pylonshq.com/display/pylonscommunity/Pylons+Curve+and+Sur... James Phillips Thanks for the tutorial, I found it interesting. Almost inspired me to write my first pylons app, if I could just find the time! Regards Gerard --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: Pylons without Myghty
On 10/21/07, Stou Sandalski [EMAIL PROTECTED] wrote: I am running 0.9.6, after some tinkering I found out it was ToscaWidgets that was causing the dependency, sorry. Is there something out there a bit 'newer' or 'cleaner' than tw that someone can recommend? At least something that doesn't depend on all those packages from PEAK that are 3 years old (PyProtocols, etc.) The non-ToscaWidgets way is plain ol' FormEncode but then you're going lower level. ToscaWidgets has been positioning itself as the high-end solution; I don't know of any other that works with Pylons. So the real question is, when will ToscaWidgets modernize its dependencies? Looking at TW's setup.py, the only ancient dependency is RuleDispatch. http://svn.turbogears.org/projects/ToscaWidgets/trunk/setup.py I recall a discussion about wanting to improve RuleDispatch and spin it off, but apparently this has not been done yet and there are insufficient volunteers for it. I'm stunned that ToscaWidgets would depend on Myghty by default, especially since it's spun off from TurboGears which prefers Genshi/Kid. Maybe they did this as a courtesy to Pylons 0.9.4 and they could change it fairly easily? Though looking at setup.py, I don't see how Mako is strictly required unless you invoke the Mako extra. Though I'm not sure how you'd install an extra with easy_install anyway. -- Mike Orr [EMAIL PROTECTED] --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: cleaner ToscaWidgets
On 10/21/07, Max Ischenko [EMAIL PROTECTED] wrote: Hello Stou, On 10/22/07, Stou Sandalski [EMAIL PROTECTED] wrote: I am running 0.9.6, after some tinkering I found out it was ToscaWidgets that was causing the dependency, sorry. Is there something out there a bit 'newer' or 'cleaner' than tw that someone can recommend? At least something that doesn't depend on all those packages from PEAK that are 3 years old (PyProtocols, etc.) I am desperately looking for such a a thing myself. Alas, the best approach I found so far is write forms in plain HTML (Mako) and use formencode validators + stock @validate Pylons decorator. So, to toscawidget or not to toscawidget? I'm about to write some forms soon. I was going to learn TW because I assumed it would be better in the long term when I add more Javascript features, and better to start with it now than to rewrite my forms in it later. Is TW really that bad? I don't care about a few extra dependencies as long as I know who's using them. -- Mike Orr [EMAIL PROTECTED] --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: cleaner ToscaWidgets
Hello Mike, On 10/22/07, Mike Orr [EMAIL PROTECTED] wrote: So, to toscawidget or not to toscawidget? I'm about to write some forms soon. I was going to learn TW because I assumed it would be better in the long term when I add more Javascript features, and better to start with it now than to rewrite my forms in it later. Is TW really that bad? I don't care about a few extra dependencies as long as I know who's using them. You 've got to use it for a while to make a judgement. It was a fine idea and I don't want to spread FUD. My problems with TW were: - rough (random bugs) - hard to use (basically no docs) - too complicated an implementation - development stalled -- Max Ischenko http://www.developers.org.ua/ -- Ukrainian software developers community http://maxischenko.in.ua/ -- my blog (Russian) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: Pylons without Myghty
Mike Orr wrote: On 10/21/07, Stou Sandalski [EMAIL PROTECTED] wrote: I am running 0.9.6, after some tinkering I found out it was ToscaWidgets that was causing the dependency, sorry. Is there something out there a bit 'newer' or 'cleaner' than tw that someone can recommend? At least something that doesn't depend on all those packages from PEAK that are 3 years old (PyProtocols, etc.) Out of curiosity, is there any inherent problems in packages 3 years old? The non-ToscaWidgets way is plain ol' FormEncode but then you're going lower level. ToscaWidgets has been positioning itself as the high-end solution; I don't know of any other that works with Pylons. So the real question is, when will ToscaWidgets modernize its dependencies? Yeah, I've been planning to drop the ruledispatch dependency long ago since it has caused more problems than it has solved (specially due to its C speedups). I've just comitted a TODO file [1] as a kind of roadmap for things I'd like to get done before releasing a final stable 1.0. Looking at TW's setup.py, the only ancient dependency is RuleDispatch. http://svn.turbogears.org/projects/ToscaWidgets/trunk/setup.py I recall a discussion about wanting to improve RuleDispatch and spin it off, but apparently this has not been done yet and there are insufficient volunteers for it. I'm stunned that ToscaWidgets would depend on Myghty by default, especially since it's spun off from TurboGears which prefers Genshi/Kid. Maybe they did this as a courtesy to Pylons 0.9.4 and they could change it fairly easily? TW doesn't depend on Mighty. However, I've nailed the problem and worked around it in newest trunk: Apparently Pylons declares a python.templating.engines entry point, pylonsmighty, which TW tries to load when initialized [2]. This will cause a DistributionError if Myghty is not installed. Though looking at setup.py, I don't see how Mako is strictly required unless you invoke the Mako extra. Though I'm not sure how you'd install an extra with easy_install anyway. Alberto [1] http://trac.turbogears.org/browser/projects/ToscaWidgets/trunk/TODO.txt [2] Actually, it tries to load all entrypoints by default on init time. This should be fixed so they're loaded JIT to prevent this kind of errors. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: ANNOUNCE: Gazest 0.3.7 -- a Pylons powered wiki
On 10/22/07, Yannick Gingras [EMAIL PROTECTED] wrote: This is really pissing me off because I wanted to postpone the switch to Alchemy 0.4 a bit but everyone and their dog is going to have Alchemy 0.4 on disk now that it's stable. I've been diverting sqlalchemy list messages to a folder so I missed the release. I've updated the introduction to SQLAlchemy 0.4 for people in a hurry. There's still a link to the 0.3 version for those maintaining existing apps. http://wiki.pylonshq.com/pages/editpage.action?pageId=10518754 My application is still on 0.4 beta 6, which is working fine. I assume 0.4 final has not introduced any new bugs that would impact Pylons applications. Does anybody have reason to believe the contrary? One improvement in 0.4 final is that engine_from_config correctly converts 'false' and '0' values to False rather than True, to conform with the Pylons/Paste standard. Pylons' default configuration seems to prefer 'true' and 'false' nowadays. -- Mike Orr [EMAIL PROTECTED] --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: cleaner ToscaWidgets
Max Ischenko wrote: Hello Mike, On 10/22/07, *Mike Orr* [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: So, to toscawidget or not to toscawidget? I'm about to write some forms soon. I was going to learn TW because I assumed it would be better in the long term when I add more Javascript features, and better to start with it now than to rewrite my forms in it later. Is TW really that bad? I don't care about a few extra dependencies as long as I know who's using them. I'm obviously biased in regards to this so I hope a happy user chimes in instead of giving my opinion. :) You 've got to use it for a while to make a judgement. It was a fine idea and I don't want to spread FUD. My problems with TW were: - rough (random bugs) I'd really appreciate some tickets at TG's trac to nail them. - hard to use (basically no docs) Fair enough. This is something I would really have liked to solve long ago but seems like some sort of mental blockade of mine. I've tried my best to personally give support in TW's mailing list to anyone needing help (secretly hoping users would fill the docs gap :) but experience has shown it's no substitute for real documentation. I hope to have time to change this soon. - too complicated an implementation. Ironically enough one of the reasons I spun TW out of TG's widgets was to simplify the implementation. I suspect the use of RuleDispatch greatly contributes to this perception, this is something I plan to change too. [1] - development stalled ?? [2] Regards, Alberto [1] http://tinyurl.com/263mfy [2] http://tinyurl.com/ysk68z --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: cleaner ToscaWidgets
On 10/22/07, Alberto Valverde [EMAIL PROTECTED] wrote: - hard to use (basically no docs) Fair enough. This is something I would really have liked to solve long ago but seems like some sort of mental blockade of mine. I've tried my best to personally give support in TW's mailing list to anyone needing help (secretly hoping users would fill the docs gap :) but experience has shown it's no substitute for real documentation. I hope to have time to change this soon. The lack of documentation has been the main reason I haven't stepped into the the ToscaWidgets learning curve yet. If you know me, you know that documentation is my forte, so I'll be writing at least a Pylons-ToscaWidgets howto and a FormEncode-ToscaWidgets comparision once I know enough to do it. Alberto, is the ToscaWidgets trunk kept reasonably stable? Meaning, Cheetah has a rule that all changes have to pass the unittests before being checked in, and backward compatibility is generally maintained, so that the trunk is almost always more usable and less buggy than the last release. Pylons, SQLAlchemy, and Supervisor seem to have a similar attitude because I've had good luck with their trunks. Does ToscaWidgets keep a clean trunk too? Is it safe enough for application development, knowing that we may have to change our API calls occasionally? Are there any anticipated changes which will change the API? -- Mike Orr [EMAIL PROTECTED] --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: cleaner ToscaWidgets
Mike Orr wrote: On 10/22/07, Alberto Valverde [EMAIL PROTECTED] wrote: - hard to use (basically no docs) Fair enough. This is something I would really have liked to solve long ago but seems like some sort of mental blockade of mine. I've tried my best to personally give support in TW's mailing list to anyone needing help (secretly hoping users would fill the docs gap :) but experience has shown it's no substitute for real documentation. I hope to have time to change this soon. The lack of documentation has been the main reason I haven't stepped into the the ToscaWidgets learning curve yet. If you know me, you know that documentation is my forte, so I'll be writing at least a Pylons-ToscaWidgets howto and a FormEncode-ToscaWidgets comparision once I know enough to do it. Your SQLAlchemy 0.4 tutorial made me brave enough to make the jump from 0.3 :) I'd really appreciate any effort in the docs area. Thanks! Alberto, is the ToscaWidgets trunk kept reasonably stable? Meaning, Cheetah has a rule that all changes have to pass the unittests before being checked in, and backward compatibility is generally maintained, so that the trunk is almost always more usable and less buggy than the last release. Pylons, SQLAlchemy, and Supervisor seem to have a similar attitude because I've had good luck with their trunks. Does ToscaWidgets keep a clean trunk too? I try my best to keep it that way. Unit tests always pass before anything is committed and there haven't been any major API changes since I first released it. Keep in mind TW didn't start from scratch but evolved from TG's widgets. trying to mimic their API as much as possible (except some small improvements and new functionallity, namely being able to mix and match templating engines). What I'm trying to say with this is that most API experiments already took place before TW was born. Is it safe enough for application development, knowing that we may have to change our API calls occasionally? Yes. TW hasn't even got a stable release yet so the trunk is as stable as it can be. All eggs that have been packaged are really snapshots of the trunk and experiments take place in separate branches before being merged and announced in the mailing list. Today I merged a breaking change which aims to reduce all the boiler plate needed to configure TW in Pylons as described by James' An Alternative ToscaWidgets Setup with Mako [1] (which, BTW, I should update, as soon as confluence mails me my lost passwd, to reflect the changes...). However, this change will not affect existing widgets, only the way TW's middleware is initialized. Are there any anticipated changes which will change the API? I expect a little breakage in the public API to take place before 1.0 is out since I want to remove the RuleDispatch dependency. However, I don't think it will affect many users (if any at all) since it uses an obscure feature (adapt_value()) involving generic functions which I haven't seen in the wild. I might also rename the package name to tw and move the widget packages inside the tw namespace (ie: tw.forms, tw.mochikit, etc...) to avoid the redundant from toscawidgets.widgets import foo. This should be easily aliased and DeprecationWarned though... Alberto [1] http://wiki.pylonshq.com/x/B4A1 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: UNS: Re: cleaner ToscaWidgets
Philip Cooper wrote: Alberto Valverde at about 10/22/07 2:07 PM said: Max Ischenko wrote: Hello Mike, On 10/22/07, *Mike Orr* [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: So, to toscawidget or not to toscawidget? I'm about to write some forms soon. I'm obviously biased in regards to this so I hope a happy user chimes in instead of giving my opinion. :) Wrks 4 me. Turbogears is not great on documentation of widgets but it's a not better than starting from scratch. Things like date picker and tinyMCE are not things I really want to recreate. I've started http://www.openvest.com/trac/wiki/ToscawidgetsInPylons for my own edification. More to come later. - hard to use (basically no docs) Pretty easy to use. Everytime someone writes a doc, it seems they are already super-users so they jump right into creating a custom widget. Too advanced if you don't know what magic is behind a basic widget. Understanding how DataGrid works makes the whole tw world make sense IMHO. Glad to hear this. :) So maybe I should reduce the initial scope of the docs I have in mind and not dive directly into custom widgets but concentrate on how existing widgets are used? Alberto --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: Reducing pylons app memory usage?
On Oct 22, 10:28 pm, Bob Ippolito [EMAIL PROTECTED] wrote: On 10/17/07, Graham Dumpleton [EMAIL PROTECTED] wrote: Note that in general the argument that lighttpd and nginx are better because they are quicker in serving static files is totally meaningless in the context of Python web applications, as the real work is being done in the Python application and that along with any database access will be your bottleneck. Unless your web application is specifically being used as a mechanism for working with large numbers of very large distinct static media files, you will not really gain anything from using lighttpd or nginx as static files isn't the bottleneck. All you will possibly do is just make your setup and configuration more complicated than it needs to be. That's not true. nginx or lighttpd are far better than Apache + mod_proxy by any metric, at least with Apache 2.0.x on FreeBSD, Mac OS X, or Linux (other platforms I have not tried myself). Also, unless you're using lots of mod_rewrite, nginx isn't any more complicated than Apache. In fact, I find it much easier because you don't have to figure out what you have to turn off. In an environment with only 96 mb available, using a web server that uses forks or threads is a really bad choice anyway. That pretty much rules Apache 2.0.x out. The complexity I am referring to is not in configuring nginx, but that you also have to install and configure some separate supervisor system which starts up and ensures that your backend process is running. You also need to integrate that into the system startup scripts whereas web servers such as Apache on prepackaged Linux distributions generally have that. This is extra software dependencies and extra work to setup. Although it may be manageable if you are running a single web application, it becomes a lot more work if you want to run a mix of web applications, or multiple sites, all in distinct processes as you may then need separate supervisor configurations and startup scripts for each. Because Apache when using appropriate modules can itself act as the supervisor for the applications, it all becomes somewhat simpler as the configuration is all in one spot and all the startup scripts become irrelevant as that would already exist for Apache. BTW, Apache/mod_proxy or Apache/fastcgi/flup solutions are not the fastest way of supporting WSGI applications with Apache and better performing, easier to setup solutions exist that don't need horrible rewrite rules. So, comparing to those solutions doesn't give the full picture. Also, as I keep trying to put out, speed of request handling and proxying is generally not the problem as your bottleneck is more often than not going to be in your Python web application or in the way it interacts with a database. Thus, for most it would be far more important to use a system which has less dependencies on third party packages and is easier to setup. True, the answer may not always be Apache, especially, if forced to use a memory constrained environment more tailored to PHP web application hosting. In some respects though this points out an issue that Python web developers still have yet to come to grips with. That is that web hosting solutions offered by companies tend to be tailored to PHP programming which has a model which throws away stuff at the end of each request. This is in contrast to Python where everything generally persists and so applications are fatter. This isn't helped by Pythons batteries included mentality as all that does is cause additional memory bloat due to a lot of dead unused code being imported that you don't really need. If Python folks expect to work within the constrained environments offered for PHP web applications, then the large frameworks have to find ways of cutting down on memory use somehow. Graham --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: Reducing pylons app memory usage?
On Oct 22, 10:12 pm, Marcin Kasperski [EMAIL PROTECTED] wrote: 2) Consider (and test) mod_wsgi daemon mode. Main advantage over embedded mode is that it will allow to tune wsgi threads (count, stack size...) separately from main apache threads. Noting though that mod_wsgi daemon mode doesn't allow tuning of stack size on a process group basis yet. Only way of tuning it at the moment would be to use 'ulimit -s' within the environment of Apache itself. Ie., set it from the Apache 'envars' startup file. Graham --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: Pylons without Myghty
On Oct 22, 12:52 pm, Alberto Valverde [EMAIL PROTECTED] wrote: Mike Orr wrote: On 10/21/07, Stou Sandalski [EMAIL PROTECTED] wrote: I am running 0.9.6, after some tinkering I found out it was ToscaWidgets that was causing the dependency, sorry. Is there something out there a bit 'newer' or 'cleaner' than tw that someone can recommend? At least something that doesn't depend on all those packages from PEAK that are 3 years old (PyProtocols, etc.) Out of curiosity, is there any inherent problems in packages 3 years old? No, not really. Except for the TW ones requiring almost manual install (somehow I have never been able to simply do easy_install -U ToscaWidgets)... I always had to do an easy_install on those packages by hand, that is Google for the toscawidgets download directory and then use that for easy_install PyProtocols, etc. TW doesn't depend on Mighty. However, I've nailed the problem and worked around it in newest trunk: Apparently Pylons declares a python.templating.engines entry point, pylonsmighty, which TW tries to load when initialized [2]. This will cause a DistributionError if Myghty is not installed. snip [2] Actually, it tries to load all entrypoints by default on init time. This should be fixed so they're loaded JIT to prevent this kind of errors. Yes that's what I noticed but I couldn't figure out what to do about it. Regards, Stou --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: Reducing pylons app memory usage?
On Thu, 2007-10-18 at 12:18 +0200, Marcin Kasperski wrote: I am watching with interest efforts to port WSGI to nginx which show up on some lists. Nevertheless, it seems it will take some time... Actually I believe Manlio's implementation is usable right now for WSGI 2.0, with WSGI 1.0 support slated for the next release. http://hg.mperillo.ath.cx/nginx/mod_wsgi/ Regardless, WSGI support isn't required to run Pylons under Nginx. I personally use plain HTTP proxying and am quite satisfied. Regards, Cliff --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: Reducing pylons app memory usage?
On Oct 23, 10:53 am, Cliff Wells [EMAIL PROTECTED] wrote: On Thu, 2007-10-18 at 12:18 +0200, Marcin Kasperski wrote: I am watching with interest efforts to port WSGI to nginx which show up on some lists. Nevertheless, it seems it will take some time... Actually I believe Manlio's implementation is usable right now for WSGI 2.0, with WSGI 1.0 support slated for the next release. http://hg.mperillo.ath.cx/nginx/mod_wsgi/ Except that there is no such thing as WSGI 2.0. The only version of WSGI that exists is 1.0. What they have done is taken once core idea which had been put up for future discussion for a WSGI 2.0 and implemented that. What has been done should not be called WSGI 2.0 as there isn't a guarantee that WSGI 2.0 will ever even be created or that it will look like what they have implemented. For further information about the ideas that have been put up for a successor to WSGI 1.0, then see: http://www.wsgi.org/wsgi/WSGI_2.0 Graham --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: Reducing pylons app memory usage?
On Mon, 2007-10-22 at 23:24 +, Graham Dumpleton wrote: The complexity I am referring to is not in configuring nginx, but that you also have to install and configure some separate supervisor system which starts up and ensures that your backend process is running. I think this argument doesn't really hold. Either you have to install a supervisor process or you don't. If you do have to, then you arguably need to install one for Apache as well, so the complexity is equivalent. If you don't have to (I've had Pylons and TurboGears sites up for months without restarts or supervisors), then once again the complexity is even. You also need to integrate that into the system startup scripts whereas web servers such as Apache on prepackaged Linux distributions generally have that. If by integrate you mean add a line or two to /etc/rc.local (or your distro's equivalent), then I'll agree with you, but simultaneously fail to see your point. This is extra software dependencies and extra work to setup. I'll take this added complexity, subtract the complexity of reducing Apache's memory footprint and still come out with an overall savings in complexity. I'll agree that Apache can be made more efficient (ignoring the threading vs async debate) but doing so is far more complicated and error-prone than installing Nginx and a supervisor. Apache starts out big and must be pruned down, Nginx starts out small and pretty much stays there regardless. Although it may be manageable if you are running a single web application, it becomes a lot more work if you want to run a mix of web applications, or multiple sites, all in distinct processes as you may then need separate supervisor configurations and startup scripts for each. Since the OP mentioned 96MB of RAM, I'll assume he's running single web app. Unless he requires mod_svn or some other esoteric module, there's simply no way Apache can win here. Because Apache when using appropriate modules can itself act as the supervisor for the applications, it all becomes somewhat simpler as the configuration is all in one spot and all the startup scripts become irrelevant as that would already exist for Apache. I know this is difficult for an Apache expert to grok, but configuring Apache is everything except easy to understand. Some of this is due to Apache's rather large feature-set, but some of it is also due to Apache's rather weird and limited configuration syntax. Also, as I keep trying to put out, speed of request handling and proxying is generally not the problem as your bottleneck is more often than not going to be in your Python web application or in the way it interacts with a database. Thus, for most it would be far more important to use a system which has less dependencies on third party packages and is easier to setup. In general I agree, but if Apache is sucking down 1/10th or more of your memory (and with 96MB, that's certainly likely), then it will certainly reduce overall system performance significantly as disk caching will suffer and you'll be forced to swap much sooner. True, the answer may not always be Apache, especially, if forced to use a memory constrained environment more tailored to PHP web application hosting. I generally find Apache preferable to Nginx when one or more of the following conditions is present: 1) Some esoteric feature of Apache is needed. 2) The customer/user is an Apache expert. 3) I don't care about the installation and hope someone else has to maintain it. If Python folks expect to work within the constrained environments offered for PHP web applications, then the large frameworks have to find ways of cutting down on memory use somehow. I'm not sure the blame is being aimed in the right direction here. I don't think PHP is what Pylons is competing with and I have a definite concern that tailoring it to run in a $7/month hosting account would require too deep of cuts. RoR is much more of a direct competitor, and has been noted, Pylons uses a fraction of the RAM RoR does (not to mention CPU). Quite frankly, were I consulting with a customer in the same position as the OP, what I'd be suggesting is they quit trying to run their site in a toy environment. I think a 96MB VPS is fine for testing, but not a realistic deployment environment for much more than a personal blog or photo album. As an aside, my Pylons (paste+cherrypy) app for Breve only uses around 48MB after a couple months of uptime (there are four Pylons backends load balanced by Nginx): hosting ~ # ps -U twisty -o rss,vsz,command RSSVSZ COMMAND 48000 132720 /usr/bin/python /var/www/virtual/twisty-industries.com//bin/paster serve breve1.ini 48312 133264 /usr/bin/python /var/www/virtual/twisty-industries.com//bin/paster serve breve2.ini 46024 130860 /usr/bin/python /var/www/virtual/twisty-industries.com//bin/paster serve breve3.ini 47920 132860 /usr/bin/python
Re: cleaner ToscaWidgets
Hello Alberto, On 10/22/07, Alberto Valverde [EMAIL PROTECTED] wrote: - too complicated an implementation. Ironically enough one of the reasons I spun TW out of TG's widgets was to simplify the implementation. I suspect the use of RuleDispatch greatly contributes to this perception, this is something I plan to change too. [1] - development stalled ?? [2] [1] http://tinyurl.com/263mfy [2] http://tinyurl.com/ysk68z Thanks a lot for your reply; it's very helpful. It looks like I should put aside a few hours and re-evaluate TW to have an up-to-date opinion. Hopefully Mike Orr could write some Pylons HOWTO by then. ;) Regards, Max. -- Max Ischenko http://www.developers.org.ua/ -- Ukrainian software developers community http://maxischenko.in.ua/ -- my blog (Russian) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: UNS: Re: cleaner ToscaWidgets
On 10/23/07, Mike Orr [EMAIL PROTECTED] wrote: Glad to hear this. :) So maybe I should reduce the initial scope of the docs I have in mind and not dive directly into custom widgets but concentrate on how existing widgets are used? Yes. Think in terms of somebody who has used another Python form library or has handcoded all their forms, and just wants to use the standard widgets at first. They'll be curious about any special widgets TW provides such as a date picker, WSIWYG editor, combo box, enhanced file uploader, etc -- things they've had to struggle with or do without in their other libraries. They don't want to build custom widgets yet; they just want to build forms that work. I'd say that even tutorial for a simple form (2-5 fields, nothing fancy) would work if it highlights how TW is different from other approaches, like handcoding, and what it can give you. -- Max Ischenko http://www.developers.org.ua/ -- Ukrainian software developers community http://maxischenko.in.ua/ -- my blog (Russian) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: Reducing pylons app memory usage?
On Oct 23, 11:54 am, Cliff Wells [EMAIL PROTECTED] wrote: On Mon, 2007-10-22 at 23:24 +, Graham Dumpleton wrote: The complexity I am referring to is not in configuring nginx, but that you also have to install and configure some separate supervisor system which starts up and ensures that your backend process is running. I think this argument doesn't really hold. Either you have to install a supervisor process or you don't. If you do have to, then you arguably need to install one for Apache as well, so the complexity is equivalent. If you don't have to (I've had Pylons and TurboGears sites up for months without restarts or supervisors), then once again the complexity is even. You do not need a separate supervisor if you use mod_python, mod_wsgi or mod_fastcgi. In the case of mod_python and embedded mode of mod_wsgi the standard Apache child process supervisor does it all for you. For mod_wsgi daemon mode the mod_wsgi implementation makes use of the Apache runtime library supervisor directly, the same one used by Apache to manage its child processes. For mod_fastcgi it contains its own supervisor implementation. In all cases the supervisor is a part of Apache or provided by the Apache module in question, there is no need to install a separate supervisor system. It is only when you use mod_proxy to a back end web server, or use fastcgi in external process mode that you need to install a separate supervisor. Thus the supervisor comes for free if you don't restrict yourself to using mod_proxy to a back end web server. You also need to integrate that into the system startup scripts whereas web servers such as Apache on prepackaged Linux distributions generally have that. If by integrate you mean add a line or two to /etc/rc.local (or your distro's equivalent), then I'll agree with you, but simultaneously fail to see your point. And that may well be fine if you know your way around UNIX system administration. More and more you are getting people coming to Python web application programming who know nothing about UNIX system administration, so if getting things to work simply means adding a line or two in the Apache configuration to get it all going, it is much better than having to install a separate supervisor system and ensure that the system starts it all up on a reboot. An Apache installation on the other hand is already going to be setup to start on boot. Using periodic cron jobs to check if an application is running and restart it, or using 404 error document handler scripts to trigger starting of an application like one sees suggested occasionally are not robust solutions. This is extra software dependencies and extra work to setup. I'll take this added complexity, subtract the complexity of reducing Apache's memory footprint and still come out with an overall savings in complexity. Use mod_wsgi daemon mode or mod_fastcgi and the memory footprint of Apache is not the big deal that people think it is as you can limit your web application to run in a single or limited number of threaded processes, thereby getting rid of most of the problems. Yes, this is not much different to mod_proxy in terms of architecture, but you are able to skip the need for a separate supervisor if you want to ensure your application always stays running. You are of course throwing out the good features of Apache in as far as its scalability, but then if you need to do this, scalability is the least of your worries. Whether you use Apache or a backend web server implemented in Python, the fact remains that the bulk of the memory consumption comes from the Python web framework and application. Just the starting memory size for the most popular Python web frameworks range from 5MB up to 15MB and that is before any of the user specific code has been loaded. Even if you use a Python web server backend, on Linux it would appear you are still going to have to contend with the large stack size it allocates to each thread as that is a property of Linux and not Apache, so you still need to configure your system around that, or reduce the numbers of threads you use, something that can also be done with Apache daemon process solutions. But then, why that is an issue I don't know, for reasons mentioned below. Although it may be manageable if you are running a single web application, it becomes a lot more work if you want to run a mix of web applications, or multiple sites, all in distinct processes as you may then need separate supervisor configurations and startup scripts for each. Since the OP mentioned 96MB of RAM, I'll assume he's running single web app. Unless he requires mod_svn or some other esoteric module, there's simply no way Apache can win here. If one doesn't know that running your Python web application embedded within the Apache child processes is not your only option then this is true, but mod_proxy is not the only solution, other options do exist such as mod_wsgi
