Re: advice with configuration/scaling a live Pylons app.
W liście SamDonaldson z dnia czwartek 29 maja 2008: Yes, lighttpd serving static and all dynamic requests forwarded to the paster process talking SCGI. Should I make the switch to nginx as it seems like everybody has something good to say about nginx, and it's a good load balancer. Also, is the fact that I'm only running a single paster process a problem? I remember iwth RoR, mongrel started multiple child processes. RoR is non-threadsafe and uses processes to allow concurrent access. Pylons, on the other hand, uses threads. I don't know if starting multiple processes would give much. -- Paweł Stradomski --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: How did you begin your fun with Pylons?
On May 22, 6:07 pm, Shannon -jj Behrens [EMAIL PROTECTED] wrote: Ben Bangert and I decided to meet for dinner in Berkeley. I got very lost and ended up in Oakland. I finally got to the restaurant an hour late. Ben had already eaten. We had a good talk about Web development. He told me he wanted to write a new Python Web application framework. I tried to talk him out of it. He didn't listen to me ;) I'm glad he did. The TG2 team is happy too :) I tried to offload some mod_perl appserver bottlenecks onto TurboGears when it first came out ( they were on numeric crunching , not Perl's specialty ). That was a complete nightmare. The specs shifted nonstop, I'm sorry. this is what you get with alpha software which is actively defining itself (which is what TG was when we had the pleasure of reading you) and the developer community was one of the least pleasant groups I've ever had the misfortune of working with ( Kevin excluded ). Perhaps a trip to archives [1] to refresh your memory on the tone of some of your posts can shed a light on the reason you felt treated in this way by TG's dev. community. I usually ignore this kind of crap in public forums but today's there's a nice friday-feeling at the office which invites to procrastinate. Sorry but I can't guarantee it'll remain this way hence me being available for the rest of the piss-fest Yours, Alberto [1] http://groups.google.com/group/turbogears/search?q=author%3Ajvanasco%40gmail.comstart=0scoring=d; --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: Pylons Security Advisory - Pylons 0.9.6.2 released
Hi, Just to clarify, will upgrading to routes 1.8 also resolve the problem, and does just upgrading pylons resolve the problem? Thanks, Damian On May 28, 9:31 pm, Ben Bangert [EMAIL PROTECTED] wrote: Pylons Security Advisory Topic: Path traversal bug in default error controller Module: controllers/error.py Announced: 2008-05-15 Credits: Webwise Security Affects: All Pylons releases with Routes 1.7.3 Corrected: Routes 1.7.3 or greater Pylons 0.9.6.2 I. Background The Pylons error.py controller is a default controller created in new Pylons projects. It handles serving media and the default error page for a Pylons application. II. Problem Description The error.py controller uses paste.fileapp to serve the static resources to the browser. The default error.py controller uses os.path.join to combine the id from Routes with the media path. Routes prior to 1.8 double unquoted the PATH_INFO, resulting in FileApp returning files from the filesystem that can be outside of the intended media path directory. III. Impact An attacker can craft URL's which utilize the double escaping to pass in a name to the error.py controller which contains a leading slash thus escaping the intended media path and serving files from any location on the filesystem that the Pylons application has access to. IV. Workaround Any of the following will prevent the file traversal: 1) Upgrade Routes to Routes 1.7.3 (easy_install -U Routes) This is a fix only for the multiple escaping which made it possible to exploit FileApp. 2) Remove the methods 'img' and 'style' from the ErrorController inside controllers/error.py. These methods are only needed to serve the default error media, customizing the error page to render your own template doesn't require these 2 methods. 3) Patch the controllers/error.py controller to import urllib, and then urllib.quote_plus the id values before having them served by the fileapp. V. Solution For new Pylons projects, starting with Pylons 0.9.6.2, the project template will include changes to ensure that the base media path is not escaped. For existing Pylons projects perform one of the following: 1) Update the pylons/error.py to use the StaticURLParser, this requires the following changes: a) Replace the fileapp import at the top with: from paste.urlparser import StaticURLParser b) Replace the img, style and _serve_file methods with the following ones: def img(self, id): Serve Pylons' stock images return self._serve_file(os.path.join(media_path, 'img'), id) def style(self, id): Serve Pylons' stock stylesheets return self._serve_file(os.path.join(media_path, 'style'), id) def _serve_file(self, root, path): Call Paste's FileApp (a WSGI application) to serve the file at the specified path static = StaticURLParser(root) request.environ['PATH_INFO'] = '/%s' % path return static(request.environ, self.start_response) 2) Remove the 'img' and 'style' methods entirely. If the error.py 'document' method is loading a custom error handler, the additional methods to load Pylons media for the default error page is unnecessary. 3) Upgrade to Routes 1.7.3. This will prevent the double unquoting behavior, but is not as secure as option (1) as there is still no additional check that fileapp is being constrainted to the appropriate media directories. smime.p7s 3KDownload --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: advice with configuration/scaling a live Pylons app.
Paweł Stradomski wrote: W liście SamDonaldson z dnia czwartek 29 maja 2008: Yes, lighttpd serving static and all dynamic requests forwarded to the paster process talking SCGI. Should I make the switch to nginx as it seems like everybody has something good to say about nginx, and it's a good load balancer. Also, is the fact that I'm only running a single paster process a problem? I remember iwth RoR, mongrel started multiple child processes. RoR is non-threadsafe and uses processes to allow concurrent access. Pylons, on the other hand, uses threads. I don't know if starting multiple processes would give much. The default server(s) (Paste's and CherryPy's both) use threads, but Pylons is not particularly tied to threads. Multiple processes can definitely speed up your application on multi-processor/multi-core systems. And setups like that shouldn't really hurt on other systems. -- Ian Bicking : [EMAIL PROTECTED] : http://blog.ianbicking.org --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: advice with configuration/scaling a live Pylons app.
2008/5/28 SamDonaldson [EMAIL PROTECTED]: Running ab benchmark tests revealed many requests were failing. Should I be starting MORE Paster processes to handle the load? Is this the right thing to do, or is the Paster (SCGI) process itself multi-threaded to handle such synchronous requests. Say the Paster process is doing some I/O (SQL query through sqlalchemy), would that process block and would other requests wait, or would they get serviced? Hi Sam, According to http://wiki.pylonshq.com/display/pylonscookbook/Pylons+Execution+Analysis+0.9.6 paster is multithreaded. It should be easy to test. Hit a controller action that sleeps for 10 seconds and see if you can still make requests to other actions. I actually need to test for myself as soon as I get to work. As for why you get failed requests, I don't know. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: advice with configuration/scaling a live Pylons app.
2008/5/29 Alex Marandon [EMAIL PROTECTED]: 2008/5/28 SamDonaldson [EMAIL PROTECTED]: Running ab benchmark tests revealed many requests were failing. Should I be starting MORE Paster processes to handle the load? Is this the right thing to do, or is the Paster (SCGI) process itself multi-threaded to handle such synchronous requests. Say the Paster process is doing some I/O (SQL query through sqlalchemy), would that process block and would other requests wait, or would they get serviced? Hi Sam, According to http://wiki.pylonshq.com/display/pylonscookbook/Pylons+Execution+Analysis+0.9.6 paster is multithreaded. It should be easy to test. Hit a controller action that sleeps for 10 seconds and see if you can still make requests to other actions. I actually need to test for myself as soon as I get to work. As for why you get failed requests, I don't know. class FooController(BaseController): def bla(self): return 'Hello World! %s' % time.time() def slow_bla(self): time.sleep(10) return 'Hello slow World!' With something like that, manual testing works just fine (ie: I'm able to hit the action bla as many times as I want while the action slow_bla is still serving). When testing with ab though, I do get failed requests, but I actually get even more failed requests with Apache/mod_wsgi. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: Pylons Security Advisory - Pylons 0.9.6.2 released
On May 29, 2008, at 7:21 AM, Damian wrote: Just to clarify, will upgrading to routes 1.8 also resolve the problem, and does just upgrading pylons resolve the problem? Upgrading Pylons will force Routes to 1.7.3 which is a partial fix. Routes 1.8 upgrade has the same effect. It fixes the immediate vulnerability, but changing your error.py controller to either remove those methods, or use the StaticURLParser is the best way to ensure the problem is resolved. Cheers, Ben smime.p7s Description: S/MIME cryptographic signature
Re: advice with configuration/scaling a live Pylons app.
On May 29, 7:29 pm, Alex Marandon [EMAIL PROTECTED] wrote: 2008/5/29 Alex Marandon [EMAIL PROTECTED]: 2008/5/28 SamDonaldson [EMAIL PROTECTED]: Running ab benchmark tests revealed many requests were failing. Should I be starting MORE Paster processes to handle the load? Is this the right thing to do, or is the Paster (SCGI) process itself multi-threaded to handle such synchronous requests. Say the Paster process is doing some I/O (SQL query through sqlalchemy), would that process block and would other requests wait, or would they get serviced? Hi Sam, According tohttp://wiki.pylonshq.com/display/pylonscookbook/Pylons+Execution+Anal... paster is multithreaded. It should be easy to test. Hit a controller action that sleeps for 10 seconds and see if you can still make requests to other actions. I actually need to test for myself as soon as I get to work. As for why you get failed requests, I don't know. class FooController(BaseController): def bla(self): return 'Hello World! %s' % time.time() def slow_bla(self): time.sleep(10) return 'Hello slow World!' With something like that, manual testing works just fine (ie: I'm able to hit the action bla as many times as I want while the action slow_bla is still serving). When testing with ab though, I do get failed requests, but I actually get even more failed requests with Apache/mod_wsgi. What options are you using to 'ab'? The 'ab' program can give a perception of errors, normally manifesting as lots of exceptions on server side in log file, if you use concurrent requests. This occurs because it only stops when it receives responses for the number of requests you tell it to make. With a request handler that takes a long time, because of a sleep() call, the 'ab' program can actually issue more requests than you tell it to. When it does receive responses for the number you told it, it will simply shutdown the connections for all the additional requests it made which it hasn't yet received a response. The result of this is that on the server side it will see a bunch of closed connections causing failures to be able to write the responses for the additional requests. The end result is that one thinks that the hosting mechanism is having problems, when in fact it is due to how the 'ab' program works. Graham --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: advice with configuration/scaling a live Pylons app.
On Thu, May 29, 2008 at 4:30 PM, Graham Dumpleton [EMAIL PROTECTED] wrote: On May 29, 7:29 pm, Alex Marandon [EMAIL PROTECTED] wrote: 2008/5/29 Alex Marandon [EMAIL PROTECTED]: 2008/5/28 SamDonaldson [EMAIL PROTECTED]: Running ab benchmark tests revealed many requests were failing. Should I be starting MORE Paster processes to handle the load? Is this the right thing to do, or is the Paster (SCGI) process itself multi-threaded to handle such synchronous requests. Say the Paster process is doing some I/O (SQL query through sqlalchemy), would that process block and would other requests wait, or would they get serviced? Hi Sam, According tohttp://wiki.pylonshq.com/display/pylonscookbook/Pylons+Execution+Anal... paster is multithreaded. It should be easy to test. Hit a controller action that sleeps for 10 seconds and see if you can still make requests to other actions. I actually need to test for myself as soon as I get to work. As for why you get failed requests, I don't know. class FooController(BaseController): def bla(self): return 'Hello World! %s' % time.time() def slow_bla(self): time.sleep(10) return 'Hello slow World!' With something like that, manual testing works just fine (ie: I'm able to hit the action bla as many times as I want while the action slow_bla is still serving). When testing with ab though, I do get failed requests, but I actually get even more failed requests with Apache/mod_wsgi. What options are you using to 'ab'? The 'ab' program can give a perception of errors, normally manifesting as lots of exceptions on server side in log file, if you use concurrent requests. This occurs because it only stops when it receives responses for the number of requests you tell it to make. With a request handler that takes a long time, because of a sleep() call, the 'ab' program can actually issue more requests than you tell it to. When it does receive responses for the number you told it, it will simply shutdown the connections for all the additional requests it made which it hasn't yet received a response. The result of this is that on the server side it will see a bunch of closed connections causing failures to be able to write the responses for the additional requests. The end result is that one thinks that the hosting mechanism is having problems, when in fact it is due to how the 'ab' program works. That's akin to the user pressing the Stop button in the browser. I get a connection reset line for that in my Quixote apps but not in my Pylons app. I assumed Pylons just ignored the exceptions because it's not really an error if the client goes away. But in any case, I don't get socket errors in my Pylons error log, and I can't assume that's because nobody ever presses Stop. So if Alex is getting such errors, the question is why the difference in behavior? -- Mike Orr [EMAIL PROTECTED] --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: How did you begin your fun with Pylons?
On May 29, 8:07 am, Alberto Valverde [EMAIL PROTECTED] wrote: Perhaps a trip to archives [1] to refresh your memory on the tone of some of your posts can shed a light on the reason you felt treated in this way by TG's dev. community. Perhaps my tone got increasingly negative as a response to poor treatment. I constantly created patches to bring TG's SqlAlchemy support in line with the SqlObject -- as one was supported and another was 'experiemental'. I would then be derided for trying to instill new design patterns -- which were exact clones of Kevin's 'official' sqlobject work. On top of that, every bit of new functionality I offered and coded to the TG community as a patch set was summarily rejected... then 2-8 months later I would receive an email saying So we realized that we should have gone in this direction... would you mind updating your patch to the new codebase? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
