Re: [web2py] CPU 210%

[Richard Vézina]

Couples of folks seem to work over performance tuning right now, so I
thought bring that thread UP would give some cues

On Tue, Feb 19, 2013 at 12:02 PM, Richard Vézina <
ml.richard.vez...@gmail.com> wrote:

> Ok, find thanks for clarification.
>
> Richard
>
>
> On Tue, Feb 19, 2013 at 11:30 AM, Niphlod  wrote:
>
>> No, it's not a parameter.
>> He was just explaining that pool_size = 1 may be "weird to look at", and
>> to think at it AS "recycle_connection=True".
>>
>> On Tuesday, February 19, 2013 5:16:29 PM UTC+1, Richard wrote:
>>
>>> Hello Michele,
>>>
>>> recycle_connection=True is a web2py connection string parameters? I
>>> didn't see entry in the book, maybe it is not documented yet?!
>>>
>>> Richard
>>>
>>>
>>> On Mon, Feb 18, 2013 at 5:50 PM, Michele Comitini >> > wrote:
>>>
 Niphold explanation is better than anything I could come out with.

 One could ask why pool_size=1 and not pool_size=0 then?
 A pool of one seems logical nonsense, but it works.
 1 means that we keep recycling that same connection that is bound to
 the non-threading process.
 Think "recycle_connection=True" and write it as "pool_size=1"

 mic


 2013/2/18 Richard Vézina :
 > Thanks Niphold for clarification, really appreciate.
 >
 > Richard
 >
 >
 > On Mon, Feb 18, 2013 at 4:17 PM, Niphlod  wrote:
 >>
 >> it "easy". if you run web2py using threads, then pooling is ok,
 since it's
 >> managed in a single process, recycling connections in a pool for
 each new
 >> thread that processes a request, and speeds up things a lot.
 >>
 >> A lot of webserver though use a single process to handle every
 request,
 >> using fork() (gunicorn, uwsgi, and so on) to provide
 concurrency. It
 >> means that there are n processes able to serve up to n requests
 >> concurrently.
 >> In that case, there are no threads involved, so there's no need to
 use a
 >> pool, because every request is handled in a "freshly created" new
 single
 >> process.
 >>
 >> --
 >>
 >> ---
 >> You received this message because you are subscribed to the Google
 Groups
 >> "web2py-users" group.
 >> To unsubscribe from this group and stop receiving emails from it,
 send an
 >> email to web2py+un...@googlegroups.com.

 >> For more options, visit https://groups.google.com/groups/opt_out.
 >>
 >>
 >
 >
 > --
 >
 > ---
 > You received this message because you are subscribed to the Google
 Groups
 > "web2py-users" group.
 > To unsubscribe from this group and stop receiving emails from it,
 send an
 > email to web2py+un...@googlegroups.com.

 > For more options, visit https://groups.google.com/groups/opt_out.
 >
 >

 --

 ---
 You received this message because you are subscribed to the Google
 Groups "web2py-users" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to web2py+un...@googlegroups.com.

 For more options, visit https://groups.google.com/groups/opt_out.



>>> --
>>
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "web2py-users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to web2py+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>>
>
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [web2py] CPU 210%

[Richard Vézina]

Ok, find thanks for clarification.

Richard


On Tue, Feb 19, 2013 at 11:30 AM, Niphlod  wrote:

> No, it's not a parameter.
> He was just explaining that pool_size = 1 may be "weird to look at", and
> to think at it AS "recycle_connection=True".
>
> On Tuesday, February 19, 2013 5:16:29 PM UTC+1, Richard wrote:
>
>> Hello Michele,
>>
>> recycle_connection=True is a web2py connection string parameters? I
>> didn't see entry in the book, maybe it is not documented yet?!
>>
>> Richard
>>
>>
>> On Mon, Feb 18, 2013 at 5:50 PM, Michele Comitini 
>> wrote:
>>
>>> Niphold explanation is better than anything I could come out with.
>>>
>>> One could ask why pool_size=1 and not pool_size=0 then?
>>> A pool of one seems logical nonsense, but it works.
>>> 1 means that we keep recycling that same connection that is bound to
>>> the non-threading process.
>>> Think "recycle_connection=True" and write it as "pool_size=1"
>>>
>>> mic
>>>
>>>
>>> 2013/2/18 Richard Vézina :
>>> > Thanks Niphold for clarification, really appreciate.
>>> >
>>> > Richard
>>> >
>>> >
>>> > On Mon, Feb 18, 2013 at 4:17 PM, Niphlod  wrote:
>>> >>
>>> >> it "easy". if you run web2py using threads, then pooling is ok, since
>>> it's
>>> >> managed in a single process, recycling connections in a pool for each
>>> new
>>> >> thread that processes a request, and speeds up things a lot.
>>> >>
>>> >> A lot of webserver though use a single process to handle every
>>> request,
>>> >> using fork() (gunicorn, uwsgi, and so on) to provide concurrency.
>>> It
>>> >> means that there are n processes able to serve up to n requests
>>> >> concurrently.
>>> >> In that case, there are no threads involved, so there's no need to
>>> use a
>>> >> pool, because every request is handled in a "freshly created" new
>>> single
>>> >> process.
>>> >>
>>> >> --
>>> >>
>>> >> ---
>>> >> You received this message because you are subscribed to the Google
>>> Groups
>>> >> "web2py-users" group.
>>> >> To unsubscribe from this group and stop receiving emails from it,
>>> send an
>>> >> email to web2py+un...@**googlegroups.com.
>>>
>>> >> For more options, visit 
>>> >> https://groups.google.com/**groups/opt_out
>>> .
>>> >>
>>> >>
>>> >
>>> >
>>> > --
>>> >
>>> > ---
>>> > You received this message because you are subscribed to the Google
>>> Groups
>>> > "web2py-users" group.
>>> > To unsubscribe from this group and stop receiving emails from it, send
>>> an
>>> > email to web2py+un...@**googlegroups.com.
>>>
>>> > For more options, visit 
>>> > https://groups.google.com/**groups/opt_out
>>> .
>>> >
>>> >
>>>
>>> --
>>>
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "web2py-users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to web2py+un...@**googlegroups.com.
>>>
>>> For more options, visit 
>>> https://groups.google.com/**groups/opt_out
>>> .
>>>
>>>
>>>
>>  --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "web2py-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] CPU 210%

[Niphlod]

No, it's not a parameter.
He was just explaining that pool_size = 1 may be "weird to look at", and to 
think at it AS "recycle_connection=True". 

On Tuesday, February 19, 2013 5:16:29 PM UTC+1, Richard wrote:
>
> Hello Michele,
>
> recycle_connection=True is a web2py connection string parameters? I didn't 
> see entry in the book, maybe it is not documented yet?!
>
> Richard
>
>
> On Mon, Feb 18, 2013 at 5:50 PM, Michele Comitini 
> 
> > wrote:
>
>> Niphold explanation is better than anything I could come out with.
>>
>> One could ask why pool_size=1 and not pool_size=0 then?
>> A pool of one seems logical nonsense, but it works.
>> 1 means that we keep recycling that same connection that is bound to
>> the non-threading process.
>> Think "recycle_connection=True" and write it as "pool_size=1"
>>
>> mic
>>
>>
>> 2013/2/18 Richard Vézina >:
>> > Thanks Niphold for clarification, really appreciate.
>> >
>> > Richard
>> >
>> >
>> > On Mon, Feb 18, 2013 at 4:17 PM, Niphlod > 
>> wrote:
>> >>
>> >> it "easy". if you run web2py using threads, then pooling is ok, since 
>> it's
>> >> managed in a single process, recycling connections in a pool for each 
>> new
>> >> thread that processes a request, and speeds up things a lot.
>> >>
>> >> A lot of webserver though use a single process to handle every request,
>> >> using fork() (gunicorn, uwsgi, and so on) to provide concurrency. 
>> It
>> >> means that there are n processes able to serve up to n requests
>> >> concurrently.
>> >> In that case, there are no threads involved, so there's no need to use 
>> a
>> >> pool, because every request is handled in a "freshly created" new 
>> single
>> >> process.
>> >>
>> >> --
>> >>
>> >> ---
>> >> You received this message because you are subscribed to the Google 
>> Groups
>> >> "web2py-users" group.
>> >> To unsubscribe from this group and stop receiving emails from it, send 
>> an
>> >> email to web2py+un...@googlegroups.com .
>> >> For more options, visit https://groups.google.com/groups/opt_out.
>> >>
>> >>
>> >
>> >
>> > --
>> >
>> > ---
>> > You received this message because you are subscribed to the Google 
>> Groups
>> > "web2py-users" group.
>> > To unsubscribe from this group and stop receiving emails from it, send 
>> an
>> > email to web2py+un...@googlegroups.com .
>> > For more options, visit https://groups.google.com/groups/opt_out.
>> >
>> >
>>
>> --
>>
>> ---
>> You received this message because you are subscribed to the Google Groups 
>> "web2py-users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to web2py+un...@googlegroups.com .
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] CPU 210%

[Richard Vézina]

Hello Michele,

recycle_connection=True is a web2py connection string parameters? I didn't
see entry in the book, maybe it is not documented yet?!

Richard


On Mon, Feb 18, 2013 at 5:50 PM, Michele Comitini <
michele.comit...@gmail.com> wrote:

> Niphold explanation is better than anything I could come out with.
>
> One could ask why pool_size=1 and not pool_size=0 then?
> A pool of one seems logical nonsense, but it works.
> 1 means that we keep recycling that same connection that is bound to
> the non-threading process.
> Think "recycle_connection=True" and write it as "pool_size=1"
>
> mic
>
>
> 2013/2/18 Richard Vézina :
> > Thanks Niphold for clarification, really appreciate.
> >
> > Richard
> >
> >
> > On Mon, Feb 18, 2013 at 4:17 PM, Niphlod  wrote:
> >>
> >> it "easy". if you run web2py using threads, then pooling is ok, since
> it's
> >> managed in a single process, recycling connections in a pool for each
> new
> >> thread that processes a request, and speeds up things a lot.
> >>
> >> A lot of webserver though use a single process to handle every request,
> >> using fork() (gunicorn, uwsgi, and so on) to provide concurrency. It
> >> means that there are n processes able to serve up to n requests
> >> concurrently.
> >> In that case, there are no threads involved, so there's no need to use a
> >> pool, because every request is handled in a "freshly created" new single
> >> process.
> >>
> >> --
> >>
> >> ---
> >> You received this message because you are subscribed to the Google
> Groups
> >> "web2py-users" group.
> >> To unsubscribe from this group and stop receiving emails from it, send
> an
> >> email to web2py+unsubscr...@googlegroups.com.
> >> For more options, visit https://groups.google.com/groups/opt_out.
> >>
> >>
> >
> >
> > --
> >
> > ---
> > You received this message because you are subscribed to the Google Groups
> > "web2py-users" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to web2py+unsubscr...@googlegroups.com.
> > For more options, visit https://groups.google.com/groups/opt_out.
> >
> >
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "web2py-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] CPU 210%

[Michele Comitini]

Niphold explanation is better than anything I could come out with.

One could ask why pool_size=1 and not pool_size=0 then?
A pool of one seems logical nonsense, but it works.
1 means that we keep recycling that same connection that is bound to
the non-threading process.
Think "recycle_connection=True" and write it as "pool_size=1"

mic


2013/2/18 Richard Vézina :
> Thanks Niphold for clarification, really appreciate.
>
> Richard
>
>
> On Mon, Feb 18, 2013 at 4:17 PM, Niphlod  wrote:
>>
>> it "easy". if you run web2py using threads, then pooling is ok, since it's
>> managed in a single process, recycling connections in a pool for each new
>> thread that processes a request, and speeds up things a lot.
>>
>> A lot of webserver though use a single process to handle every request,
>> using fork() (gunicorn, uwsgi, and so on) to provide concurrency. It
>> means that there are n processes able to serve up to n requests
>> concurrently.
>> In that case, there are no threads involved, so there's no need to use a
>> pool, because every request is handled in a "freshly created" new single
>> process.
>>
>> --
>>
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "web2py-users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to web2py+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "web2py-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] CPU 210%

[Richard Vézina]

Thanks Niphold for clarification, really appreciate.

Richard


On Mon, Feb 18, 2013 at 4:17 PM, Niphlod  wrote:

> it "easy". if you run web2py using threads, then pooling is ok, since it's
> managed in a single process, recycling connections in a pool for each new
> thread that processes a request, and speeds up things a lot.
>
> A lot of webserver though use a single process to handle every request,
> using fork() (gunicorn, uwsgi, and so on) to provide concurrency. It
> means that there are n processes able to serve up to n requests
> concurrently.
> In that case, there are no threads involved, so there's no need to use a
> pool, because every request is handled in a "freshly created" new single
> process.
>
>  --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "web2py-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] CPU 210%

[Niphlod]

it "easy". if you run web2py using threads, then pooling is ok, since it's 
managed in a single process, recycling connections in a pool for each new 
thread that processes a request, and speeds up things a lot.

A lot of webserver though use a single process to handle every request, 
using fork() (gunicorn, uwsgi, and so on) to provide concurrency. It 
means that there are n processes able to serve up to n requests 
concurrently.
In that case, there are no threads involved, so there's no need to use a 
pool, because every request is handled in a "freshly created" new single 
process.

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] CPU 210%

[Richard Vézina]

Hello,

I am trying to optimize speed a bit these days, and read this thread and
Michele "10 rules to speed" and I am mystify about rule 10 : 10. use the
dal pooling machinery, i.e. db=DAL(..., pool_size=1)

About the pool_size parameter is very important to reduce response
time and use an adequate value.
The above pool_size=1 using processes seems weird, but is not: each
request is handled by a process that needs only a single connection no
more,
using threads pool_size needs a value >= to the maximum number of
threads in a web2py instance.
The total number of persistent connections to the db is conn_n =
proc_num x pool_size (with proc_num = number of processes). So if you
have 20 processes (proc_num=20) with pool_size=1:

conn_n = 20 x 1 = 20

If you have 20 threads (proc_num = 1) with pool_size=20:

conn_n = 1 x 20 = 20


I read many time on this Massimo wrote more high is the pool_size the
better will be the performance... But what Michele seems to explain here is
the inverse... Or that only pool_size=1 is enough?!

Is it possible to get clarification here?

Thanks

Richard


On Tue, Apr 17, 2012 at 4:40 PM, Michele Comitini <
michele.comit...@gmail.com> wrote:

> Richard you are right.  We should make a web2py slice out of this.  I
> keep repeating same things over from time to time, I suppose it's the
> age... ;-)
>
> Anyway a put a star on the thread so one day I will write.
>
> Here are 10 system configuration rules I follow for high scalability
> with web2py on multicore machines:
> 1. use processes not threads or use pypy
> 2. use an event based http frontend
> 3. use UWSGI or SCGI or FCGI
> 4. use HTTP keepalive
> 5. treat static content as static i.e. keep it out of the views
> 6. reduce number and size of static files, i.e. sprites and pack js
> and css and aggregate them in a file
> 7. support gzip encoding for files > 1KB
> 8. put the database(s) on a different machine(s) or double the cores
> i.e. for each request have 2 cores at hand
> 9. cache dynamic content leveraging on web2py cache
> 10. use the dal pooling machinery, i.e. db=DAL(..., pool_size=1)
>
> About the pool_size parameter is very important to reduce response
> time and use an adequate value.
> The above pool_size=1 using processes seems weird, but is not: each
> request is handled by a process that needs only a single connection no
> more,
> using threads pool_size needs a value >= to the maximum number of
> threads in a web2py instance.
> The total number of persistent connections to the db is conn_n =
> proc_num x pool_size (with proc_num = number of processes). So if you
> have 20 processes (proc_num=20) with pool_size=1:
>
> conn_n = 20 x 1 = 20
>
> If you have 20 threads (proc_num = 1) with pool_size=20:
>
> conn_n = 1 x 20 = 20
>
> .
>
> mic
>
> Il 17 aprile 2012 21:00, Richard Vézina 
> ha scritto:
> > Michele,
> >
> > I read this thread, not sure if it is not already existing... But it
> start
> > to look like a recipe for speed tuning that could be translate into a
> web2py
> > slice or blog post :)
> >
> > Richard
> >
> >
> > On Tue, Apr 17, 2012 at 2:32 PM, Michele Comitini
> >  wrote:
> >>
> >> What I suggest is use nginx + uwsgi or nginx + scgi or nginx + fcgi.
> >> You will notice big reduction in resource consuption compared to
> >> Apache.
> >> Apache can be taken away.  By the way you can use any of the above
> >> protocols to do balancing over a pool of machines running web2py and
> >> in front you can put nginx as balancer.
> >>
> >> About caching, what is important is using as much as possible the
> >> *static* dir and have the files underneath served by nginx directly.
> >> You must check that expiration times on static objects are correctly
> >> set.  Also use sprites as much as you can.  You will enjoy a big
> >> improvement because after downloading the objects the first time a
> >> client browser will make only requests for the dynamically generated
> >> contents.  Other objects will be taken by the ondisk cache of the
> >> browser. If  the object is expired  in browser cache, but the content
> >> not changed on the server, the request will result only in a 304
> >> answer so little data is exchanged and little computation is required.
> >>
> >> mic
> >>
> >>
> >> Il 17 aprile 2012 19:10, Bruce Wade  ha scritto:
> >> > Currently I just had 1 server running apache mod_wsgi using the same
> >> > configuration as pyramid. However I just got approved for a few grand
> a
> >> > month to spend on server resources, so I am looking at load balancers.
> >> > And I
> >> > will put nginx in front of apache, and also start using a lot more
> >> > caching.
> >> >
> >> >
> >> > On Tue, Apr 17, 2012 at 5:15 AM, Michele Comitini
> >> >  wrote:
> >> >>
> >> >> One more thing make css and js packed + server side gzipped (nginx
> and
> >> >> cherokee can do also gzip caching)
> >> >>
> >> >> mic
> >> >>
> >> >> Il 17 aprile 2012 14:12, Michele Comitini <
> michele.comit...@gmail.com>
> >> >> ha scritto:
> >> >> > If you

Re: [web2py] CPU 210%

[Roberto De Ioris]

> Any ideas what would cause the system to stop working without errors when
> using database sessions?
>

I suggest you to start with a uWSGI config resembling mod_wsgi:

--lazy
--enable-threads

Should be enough, and start with a single process (and a single thread).

If it works (and it should), remove --lazy (in that way you will abuse
fork() cow). And then remove --enable-threads (even if web2py uses threads
in a couple of areas, so i would not remove it)



-- 
Roberto De Ioris
http://unbit.it

Re: [web2py] CPU 210%

[Bruce Wade]

Any ideas what would cause the system to stop working without errors when
using database sessions?

On Wed, Apr 18, 2012 at 4:08 PM, Bruce Wade  wrote:

> Why I have no idea, there were not errors in any of the logs, the site
> would just not load. However the apache one loaded without a problem. as
> soon as I put it to use the FS again it work.
>
>
> On Wed, Apr 18, 2012 at 4:02 PM, Bruno Rocha wrote:
>
>> why?
>>
>>
>> On Wed, Apr 18, 2012 at 7:24 PM, Bruce Wade  wrote:
>>
>>> After changing sessions to use the database instead of the FS I can no
>>> longer use uwsgi
>>>
>>>
>>> On Wed, Apr 18, 2012 at 3:17 PM, Bruce Wade wrote:
>>>
 I am about to give up on uwsgi. Everything was working, and I pulled
 the latest code now it doesn't work, but my other server with the same code
 works.


 On Wed, Apr 18, 2012 at 2:40 PM, Bruce Wade wrote:

> Thanks for the explanation the error was strange because first I was
> getting permission connecting to the database error I updated that then I
> got that error.
>
> Oh well it is all working now. 2 systems (1 apache, 1 nginx + uwsgi)
> on a load balancer going to add a 3rd nginx + uwsgi
> --
> Regards,
> Bruce
>
>
> On Wed, Apr 18, 2012 at 11:08 AM, Roberto De Ioris 
> wrote:
>
>>
>> > Ok got it running, restarted uwsgi not sure what the issue was.
>> >
>> > However following the guides I my static content isn't being loaded:
>> > http://50.116.6.124
>>
>>
>> The error you got earlier, happens when you do not read the body of
>> the
>> request (for example when you have POST data, but you do not read
>> them).
>> This is how nginx works. You can avoid that (if you do not want to
>> read
>> post data) with --post-buffering  in uWSGI. It is a trick to force
>> uWSGI to blindly read the http body independently by your app.
>>
>> Regarding static files, you have to fight with nginx (uWSGI should
>> only
>> manage the python part)
>>
>> Normally working on the location directives and root/alias options is
>> enough
>>
>> >
>> > On Wed, Apr 18, 2012 at 10:46 AM, Bruce Wade 
>> wrote:
>> >
>> >> uwsgi is sure not as easy to get working correctly as apache is.
>> Keep
>> >> getting the following:
>> >> 2012/04/18 10:44:49 [error] 23686#0: *8454 upstream prematurely
>> closed
>> >> connection while reading response header from upstream, client:
>> >> 173.180.212.86, server: youadstage, request: "GET / HTTP/1.1",
>> upstream:
>> >> "uwsgi://127.0.0.1:9001", host: "50.116.6.124"
>> >>
>> >>
>> >> On Wed, Apr 18, 2012 at 9:43 AM, Bruno Rocha
>> >> wrote:
>> >>
>> >>> There are directions for this on book
>> >>> http://web2py.com/books/default/chapter/29/4#session
>> >>>
>> >>> Store session on db or set separate=True
>> >>>
>> >>>
>> >>> On Wed, Apr 18, 2012 at 1:36 PM, Bruno Rocha
>> >>> wrote:
>> >>>
>>  I recommend you to store sessions on db or cache like memcached,
>>  redis... If you are going to run in a load balance, sessions in
>> disk
>>  will
>>  be a problem.
>> 
>> 
>>  On Wed, Apr 18, 2012 at 11:32 AM, Bruce Wade
>>  wrote:
>> 
>> > Currently FS
>> >
>> >
>> > On Wed, Apr 18, 2012 at 7:22 AM, Bruno Rocha
>> > wrote:
>> >
>> >> Are you storing sessions in db or FS?
>> >>
>> >> http://zerp.ly/rochacbruno
>> >> Em 18/04/2012 02:18, "Bruce Wade" 
>> escreveu:
>> >>
>> >> Yeah, I am not sure how well web2py can handle load but I am
>> going
>> >> to
>> >>> find out pretty fast. I finished setting up a load balancer
>> today,
>> >>> going to
>> >>> have 3 servers under it running nginx + uwsgi + web2py a
>> single
>> >>> dedicated
>> >>> DB server for now, but figure I am going to need some slave
>> read
>> >>> only
>> >>> servers. The speed at which web2py allows you to develop
>> >>> applications
>> >>> will surpass any load issues, well unless it could only handle
>> >>> 50,000
>> >>> users/per second then that would be a big issue :D
>> >>>
>> >>> My one concern is how good the sticky sessions on linode's
>> >>> nodebalancer works.
>> >>>
>> >>> Regarding uwsgi I have ready time out issues when using it
>> with
>> >>> web2py.
>> >>>
>> >>> --
>> >>> Regards,
>> >>> Bruce
>> >>>
>> >>> On Tue, Apr 17, 2012 at 9:55 PM, pbreit
>> >>> wrote:
>> >>>
>>  I've never read anything but good things about uwsgi. Django
>> and
>>  Flask both promote it.
>> >>>
>>

Re: [web2py] CPU 210%

[Bruce Wade]

Why I have no idea, there were not errors in any of the logs, the site
would just not load. However the apache one loaded without a problem. as
soon as I put it to use the FS again it work.

On Wed, Apr 18, 2012 at 4:02 PM, Bruno Rocha  wrote:

> why?
>
>
> On Wed, Apr 18, 2012 at 7:24 PM, Bruce Wade  wrote:
>
>> After changing sessions to use the database instead of the FS I can no
>> longer use uwsgi
>>
>>
>> On Wed, Apr 18, 2012 at 3:17 PM, Bruce Wade  wrote:
>>
>>> I am about to give up on uwsgi. Everything was working, and I pulled the
>>> latest code now it doesn't work, but my other server with the same code
>>> works.
>>>
>>>
>>> On Wed, Apr 18, 2012 at 2:40 PM, Bruce Wade wrote:
>>>
 Thanks for the explanation the error was strange because first I was
 getting permission connecting to the database error I updated that then I
 got that error.

 Oh well it is all working now. 2 systems (1 apache, 1 nginx + uwsgi) on
 a load balancer going to add a 3rd nginx + uwsgi
 --
 Regards,
 Bruce


 On Wed, Apr 18, 2012 at 11:08 AM, Roberto De Ioris wrote:

>
> > Ok got it running, restarted uwsgi not sure what the issue was.
> >
> > However following the guides I my static content isn't being loaded:
> > http://50.116.6.124
>
>
> The error you got earlier, happens when you do not read the body of the
> request (for example when you have POST data, but you do not read
> them).
> This is how nginx works. You can avoid that (if you do not want to read
> post data) with --post-buffering  in uWSGI. It is a trick to force
> uWSGI to blindly read the http body independently by your app.
>
> Regarding static files, you have to fight with nginx (uWSGI should only
> manage the python part)
>
> Normally working on the location directives and root/alias options is
> enough
>
> >
> > On Wed, Apr 18, 2012 at 10:46 AM, Bruce Wade 
> wrote:
> >
> >> uwsgi is sure not as easy to get working correctly as apache is.
> Keep
> >> getting the following:
> >> 2012/04/18 10:44:49 [error] 23686#0: *8454 upstream prematurely
> closed
> >> connection while reading response header from upstream, client:
> >> 173.180.212.86, server: youadstage, request: "GET / HTTP/1.1",
> upstream:
> >> "uwsgi://127.0.0.1:9001", host: "50.116.6.124"
> >>
> >>
> >> On Wed, Apr 18, 2012 at 9:43 AM, Bruno Rocha
> >> wrote:
> >>
> >>> There are directions for this on book
> >>> http://web2py.com/books/default/chapter/29/4#session
> >>>
> >>> Store session on db or set separate=True
> >>>
> >>>
> >>> On Wed, Apr 18, 2012 at 1:36 PM, Bruno Rocha
> >>> wrote:
> >>>
>  I recommend you to store sessions on db or cache like memcached,
>  redis... If you are going to run in a load balance, sessions in
> disk
>  will
>  be a problem.
> 
> 
>  On Wed, Apr 18, 2012 at 11:32 AM, Bruce Wade
>  wrote:
> 
> > Currently FS
> >
> >
> > On Wed, Apr 18, 2012 at 7:22 AM, Bruno Rocha
> > wrote:
> >
> >> Are you storing sessions in db or FS?
> >>
> >> http://zerp.ly/rochacbruno
> >> Em 18/04/2012 02:18, "Bruce Wade" 
> escreveu:
> >>
> >> Yeah, I am not sure how well web2py can handle load but I am
> going
> >> to
> >>> find out pretty fast. I finished setting up a load balancer
> today,
> >>> going to
> >>> have 3 servers under it running nginx + uwsgi + web2py a single
> >>> dedicated
> >>> DB server for now, but figure I am going to need some slave
> read
> >>> only
> >>> servers. The speed at which web2py allows you to develop
> >>> applications
> >>> will surpass any load issues, well unless it could only handle
> >>> 50,000
> >>> users/per second then that would be a big issue :D
> >>>
> >>> My one concern is how good the sticky sessions on linode's
> >>> nodebalancer works.
> >>>
> >>> Regarding uwsgi I have ready time out issues when using it with
> >>> web2py.
> >>>
> >>> --
> >>> Regards,
> >>> Bruce
> >>>
> >>> On Tue, Apr 17, 2012 at 9:55 PM, pbreit
> >>> wrote:
> >>>
>  I've never read anything but good things about uwsgi. Django
> and
>  Flask both promote it.
> >>>
> >>>
> >>>
> >>>
> >>> --
> >>> --
> >>> Regards,
> >>> Bruce Wade
> >>> http://ca.linkedin.com/in/brucelwade
> >>> http://www.wadecybertech.com
> >>> http://www.fittraineronline.com - Fitness Personal Trainers
> Online
> >>> http://www.warplydesigned.

Re: [web2py] CPU 210%

[Bruno Rocha]

why?

On Wed, Apr 18, 2012 at 7:24 PM, Bruce Wade  wrote:

> After changing sessions to use the database instead of the FS I can no
> longer use uwsgi
>
>
> On Wed, Apr 18, 2012 at 3:17 PM, Bruce Wade  wrote:
>
>> I am about to give up on uwsgi. Everything was working, and I pulled the
>> latest code now it doesn't work, but my other server with the same code
>> works.
>>
>>
>> On Wed, Apr 18, 2012 at 2:40 PM, Bruce Wade  wrote:
>>
>>> Thanks for the explanation the error was strange because first I was
>>> getting permission connecting to the database error I updated that then I
>>> got that error.
>>>
>>> Oh well it is all working now. 2 systems (1 apache, 1 nginx + uwsgi) on
>>> a load balancer going to add a 3rd nginx + uwsgi
>>> --
>>> Regards,
>>> Bruce
>>>
>>>
>>> On Wed, Apr 18, 2012 at 11:08 AM, Roberto De Ioris wrote:
>>>

 > Ok got it running, restarted uwsgi not sure what the issue was.
 >
 > However following the guides I my static content isn't being loaded:
 > http://50.116.6.124


 The error you got earlier, happens when you do not read the body of the
 request (for example when you have POST data, but you do not read them).
 This is how nginx works. You can avoid that (if you do not want to read
 post data) with --post-buffering  in uWSGI. It is a trick to force
 uWSGI to blindly read the http body independently by your app.

 Regarding static files, you have to fight with nginx (uWSGI should only
 manage the python part)

 Normally working on the location directives and root/alias options is
 enough

 >
 > On Wed, Apr 18, 2012 at 10:46 AM, Bruce Wade 
 wrote:
 >
 >> uwsgi is sure not as easy to get working correctly as apache is. Keep
 >> getting the following:
 >> 2012/04/18 10:44:49 [error] 23686#0: *8454 upstream prematurely
 closed
 >> connection while reading response header from upstream, client:
 >> 173.180.212.86, server: youadstage, request: "GET / HTTP/1.1",
 upstream:
 >> "uwsgi://127.0.0.1:9001", host: "50.116.6.124"
 >>
 >>
 >> On Wed, Apr 18, 2012 at 9:43 AM, Bruno Rocha
 >> wrote:
 >>
 >>> There are directions for this on book
 >>> http://web2py.com/books/default/chapter/29/4#session
 >>>
 >>> Store session on db or set separate=True
 >>>
 >>>
 >>> On Wed, Apr 18, 2012 at 1:36 PM, Bruno Rocha
 >>> wrote:
 >>>
  I recommend you to store sessions on db or cache like memcached,
  redis... If you are going to run in a load balance, sessions in
 disk
  will
  be a problem.
 
 
  On Wed, Apr 18, 2012 at 11:32 AM, Bruce Wade
  wrote:
 
 > Currently FS
 >
 >
 > On Wed, Apr 18, 2012 at 7:22 AM, Bruno Rocha
 > wrote:
 >
 >> Are you storing sessions in db or FS?
 >>
 >> http://zerp.ly/rochacbruno
 >> Em 18/04/2012 02:18, "Bruce Wade" 
 escreveu:
 >>
 >> Yeah, I am not sure how well web2py can handle load but I am
 going
 >> to
 >>> find out pretty fast. I finished setting up a load balancer
 today,
 >>> going to
 >>> have 3 servers under it running nginx + uwsgi + web2py a single
 >>> dedicated
 >>> DB server for now, but figure I am going to need some slave read
 >>> only
 >>> servers. The speed at which web2py allows you to develop
 >>> applications
 >>> will surpass any load issues, well unless it could only handle
 >>> 50,000
 >>> users/per second then that would be a big issue :D
 >>>
 >>> My one concern is how good the sticky sessions on linode's
 >>> nodebalancer works.
 >>>
 >>> Regarding uwsgi I have ready time out issues when using it with
 >>> web2py.
 >>>
 >>> --
 >>> Regards,
 >>> Bruce
 >>>
 >>> On Tue, Apr 17, 2012 at 9:55 PM, pbreit
 >>> wrote:
 >>>
  I've never read anything but good things about uwsgi. Django
 and
  Flask both promote it.
 >>>
 >>>
 >>>
 >>>
 >>> --
 >>> --
 >>> Regards,
 >>> Bruce Wade
 >>> http://ca.linkedin.com/in/brucelwade
 >>> http://www.wadecybertech.com
 >>> http://www.fittraineronline.com - Fitness Personal Trainers
 Online
 >>> http://www.warplydesigned.com
 >>>
 >>>
 >
 >
 > --
 > --
 > Regards,
 > Bruce Wade
 > http://ca.linkedin.com/in/brucelwade
 > http://www.wadecybertech.com
 > http://www.fittraineronline.com - Fitness Personal Trainers
 Online
 > http://www.warplydesigned.com
 >
 >
 
 
  --
 
 

Re: [web2py] CPU 210%

[Bruce Wade]

After changing sessions to use the database instead of the FS I can no
longer use uwsgi

On Wed, Apr 18, 2012 at 3:17 PM, Bruce Wade  wrote:

> I am about to give up on uwsgi. Everything was working, and I pulled the
> latest code now it doesn't work, but my other server with the same code
> works.
>
>
> On Wed, Apr 18, 2012 at 2:40 PM, Bruce Wade  wrote:
>
>> Thanks for the explanation the error was strange because first I was
>> getting permission connecting to the database error I updated that then I
>> got that error.
>>
>> Oh well it is all working now. 2 systems (1 apache, 1 nginx + uwsgi) on a
>> load balancer going to add a 3rd nginx + uwsgi
>> --
>> Regards,
>> Bruce
>>
>>
>> On Wed, Apr 18, 2012 at 11:08 AM, Roberto De Ioris wrote:
>>
>>>
>>> > Ok got it running, restarted uwsgi not sure what the issue was.
>>> >
>>> > However following the guides I my static content isn't being loaded:
>>> > http://50.116.6.124
>>>
>>>
>>> The error you got earlier, happens when you do not read the body of the
>>> request (for example when you have POST data, but you do not read them).
>>> This is how nginx works. You can avoid that (if you do not want to read
>>> post data) with --post-buffering  in uWSGI. It is a trick to force
>>> uWSGI to blindly read the http body independently by your app.
>>>
>>> Regarding static files, you have to fight with nginx (uWSGI should only
>>> manage the python part)
>>>
>>> Normally working on the location directives and root/alias options is
>>> enough
>>>
>>> >
>>> > On Wed, Apr 18, 2012 at 10:46 AM, Bruce Wade 
>>> wrote:
>>> >
>>> >> uwsgi is sure not as easy to get working correctly as apache is. Keep
>>> >> getting the following:
>>> >> 2012/04/18 10:44:49 [error] 23686#0: *8454 upstream prematurely closed
>>> >> connection while reading response header from upstream, client:
>>> >> 173.180.212.86, server: youadstage, request: "GET / HTTP/1.1",
>>> upstream:
>>> >> "uwsgi://127.0.0.1:9001", host: "50.116.6.124"
>>> >>
>>> >>
>>> >> On Wed, Apr 18, 2012 at 9:43 AM, Bruno Rocha
>>> >> wrote:
>>> >>
>>> >>> There are directions for this on book
>>> >>> http://web2py.com/books/default/chapter/29/4#session
>>> >>>
>>> >>> Store session on db or set separate=True
>>> >>>
>>> >>>
>>> >>> On Wed, Apr 18, 2012 at 1:36 PM, Bruno Rocha
>>> >>> wrote:
>>> >>>
>>>  I recommend you to store sessions on db or cache like memcached,
>>>  redis... If you are going to run in a load balance, sessions in disk
>>>  will
>>>  be a problem.
>>> 
>>> 
>>>  On Wed, Apr 18, 2012 at 11:32 AM, Bruce Wade
>>>  wrote:
>>> 
>>> > Currently FS
>>> >
>>> >
>>> > On Wed, Apr 18, 2012 at 7:22 AM, Bruno Rocha
>>> > wrote:
>>> >
>>> >> Are you storing sessions in db or FS?
>>> >>
>>> >> http://zerp.ly/rochacbruno
>>> >> Em 18/04/2012 02:18, "Bruce Wade" 
>>> escreveu:
>>> >>
>>> >> Yeah, I am not sure how well web2py can handle load but I am going
>>> >> to
>>> >>> find out pretty fast. I finished setting up a load balancer
>>> today,
>>> >>> going to
>>> >>> have 3 servers under it running nginx + uwsgi + web2py a single
>>> >>> dedicated
>>> >>> DB server for now, but figure I am going to need some slave read
>>> >>> only
>>> >>> servers. The speed at which web2py allows you to develop
>>> >>> applications
>>> >>> will surpass any load issues, well unless it could only handle
>>> >>> 50,000
>>> >>> users/per second then that would be a big issue :D
>>> >>>
>>> >>> My one concern is how good the sticky sessions on linode's
>>> >>> nodebalancer works.
>>> >>>
>>> >>> Regarding uwsgi I have ready time out issues when using it with
>>> >>> web2py.
>>> >>>
>>> >>> --
>>> >>> Regards,
>>> >>> Bruce
>>> >>>
>>> >>> On Tue, Apr 17, 2012 at 9:55 PM, pbreit
>>> >>> wrote:
>>> >>>
>>>  I've never read anything but good things about uwsgi. Django and
>>>  Flask both promote it.
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> --
>>> >>> Regards,
>>> >>> Bruce Wade
>>> >>> http://ca.linkedin.com/in/brucelwade
>>> >>> http://www.wadecybertech.com
>>> >>> http://www.fittraineronline.com - Fitness Personal Trainers
>>> Online
>>> >>> http://www.warplydesigned.com
>>> >>>
>>> >>>
>>> >
>>> >
>>> > --
>>> > --
>>> > Regards,
>>> > Bruce Wade
>>> > http://ca.linkedin.com/in/brucelwade
>>> > http://www.wadecybertech.com
>>> > http://www.fittraineronline.com - Fitness Personal Trainers Online
>>> > http://www.warplydesigned.com
>>> >
>>> >
>>> 
>>> 
>>>  --
>>> 
>>>  Bruno Rocha
>>>  [http://rochacbruno.com.br]
>>> 
>>> 
>>> >>>
>>> >>>
>>> >>> --
>>> >>>
>>> >>> Bruno Rocha
>>> >>> [http://rochacbruno.com.br]
>>> >>>
>>> >>>
>>> >>
>>> >>
>>> >> --
>>> >> --
>>> >> Regards,
>>> >> Bruce Wad

Re: [web2py] CPU 210%

[Bruce Wade]

I am about to give up on uwsgi. Everything was working, and I pulled the
latest code now it doesn't work, but my other server with the same code
works.

On Wed, Apr 18, 2012 at 2:40 PM, Bruce Wade  wrote:

> Thanks for the explanation the error was strange because first I was
> getting permission connecting to the database error I updated that then I
> got that error.
>
> Oh well it is all working now. 2 systems (1 apache, 1 nginx + uwsgi) on a
> load balancer going to add a 3rd nginx + uwsgi
> --
> Regards,
> Bruce
>
>
> On Wed, Apr 18, 2012 at 11:08 AM, Roberto De Ioris wrote:
>
>>
>> > Ok got it running, restarted uwsgi not sure what the issue was.
>> >
>> > However following the guides I my static content isn't being loaded:
>> > http://50.116.6.124
>>
>>
>> The error you got earlier, happens when you do not read the body of the
>> request (for example when you have POST data, but you do not read them).
>> This is how nginx works. You can avoid that (if you do not want to read
>> post data) with --post-buffering  in uWSGI. It is a trick to force
>> uWSGI to blindly read the http body independently by your app.
>>
>> Regarding static files, you have to fight with nginx (uWSGI should only
>> manage the python part)
>>
>> Normally working on the location directives and root/alias options is
>> enough
>>
>> >
>> > On Wed, Apr 18, 2012 at 10:46 AM, Bruce Wade 
>> wrote:
>> >
>> >> uwsgi is sure not as easy to get working correctly as apache is. Keep
>> >> getting the following:
>> >> 2012/04/18 10:44:49 [error] 23686#0: *8454 upstream prematurely closed
>> >> connection while reading response header from upstream, client:
>> >> 173.180.212.86, server: youadstage, request: "GET / HTTP/1.1",
>> upstream:
>> >> "uwsgi://127.0.0.1:9001", host: "50.116.6.124"
>> >>
>> >>
>> >> On Wed, Apr 18, 2012 at 9:43 AM, Bruno Rocha
>> >> wrote:
>> >>
>> >>> There are directions for this on book
>> >>> http://web2py.com/books/default/chapter/29/4#session
>> >>>
>> >>> Store session on db or set separate=True
>> >>>
>> >>>
>> >>> On Wed, Apr 18, 2012 at 1:36 PM, Bruno Rocha
>> >>> wrote:
>> >>>
>>  I recommend you to store sessions on db or cache like memcached,
>>  redis... If you are going to run in a load balance, sessions in disk
>>  will
>>  be a problem.
>> 
>> 
>>  On Wed, Apr 18, 2012 at 11:32 AM, Bruce Wade
>>  wrote:
>> 
>> > Currently FS
>> >
>> >
>> > On Wed, Apr 18, 2012 at 7:22 AM, Bruno Rocha
>> > wrote:
>> >
>> >> Are you storing sessions in db or FS?
>> >>
>> >> http://zerp.ly/rochacbruno
>> >> Em 18/04/2012 02:18, "Bruce Wade"  escreveu:
>> >>
>> >> Yeah, I am not sure how well web2py can handle load but I am going
>> >> to
>> >>> find out pretty fast. I finished setting up a load balancer today,
>> >>> going to
>> >>> have 3 servers under it running nginx + uwsgi + web2py a single
>> >>> dedicated
>> >>> DB server for now, but figure I am going to need some slave read
>> >>> only
>> >>> servers. The speed at which web2py allows you to develop
>> >>> applications
>> >>> will surpass any load issues, well unless it could only handle
>> >>> 50,000
>> >>> users/per second then that would be a big issue :D
>> >>>
>> >>> My one concern is how good the sticky sessions on linode's
>> >>> nodebalancer works.
>> >>>
>> >>> Regarding uwsgi I have ready time out issues when using it with
>> >>> web2py.
>> >>>
>> >>> --
>> >>> Regards,
>> >>> Bruce
>> >>>
>> >>> On Tue, Apr 17, 2012 at 9:55 PM, pbreit
>> >>> wrote:
>> >>>
>>  I've never read anything but good things about uwsgi. Django and
>>  Flask both promote it.
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> --
>> >>> Regards,
>> >>> Bruce Wade
>> >>> http://ca.linkedin.com/in/brucelwade
>> >>> http://www.wadecybertech.com
>> >>> http://www.fittraineronline.com - Fitness Personal Trainers
>> Online
>> >>> http://www.warplydesigned.com
>> >>>
>> >>>
>> >
>> >
>> > --
>> > --
>> > Regards,
>> > Bruce Wade
>> > http://ca.linkedin.com/in/brucelwade
>> > http://www.wadecybertech.com
>> > http://www.fittraineronline.com - Fitness Personal Trainers Online
>> > http://www.warplydesigned.com
>> >
>> >
>> 
>> 
>>  --
>> 
>>  Bruno Rocha
>>  [http://rochacbruno.com.br]
>> 
>> 
>> >>>
>> >>>
>> >>> --
>> >>>
>> >>> Bruno Rocha
>> >>> [http://rochacbruno.com.br]
>> >>>
>> >>>
>> >>
>> >>
>> >> --
>> >> --
>> >> Regards,
>> >> Bruce Wade
>> >> http://ca.linkedin.com/in/brucelwade
>> >> http://www.wadecybertech.com
>> >> http://www.fittraineronline.com - Fitness Personal Trainers Online
>> >> http://www.warplydesigned.com
>> >>
>> >>
>> >
>> >
>> > --
>> > --
>> > Regards,
>> > Bruce Wade
>> > http://ca.linkedin.com/in/brucelwade
>> > http://www.wa

Re: [web2py] CPU 210%

[Bruce Wade]

Thanks for the explanation the error was strange because first I was
getting permission connecting to the database error I updated that then I
got that error.

Oh well it is all working now. 2 systems (1 apache, 1 nginx + uwsgi) on a
load balancer going to add a 3rd nginx + uwsgi
--
Regards,
Bruce

On Wed, Apr 18, 2012 at 11:08 AM, Roberto De Ioris  wrote:

>
> > Ok got it running, restarted uwsgi not sure what the issue was.
> >
> > However following the guides I my static content isn't being loaded:
> > http://50.116.6.124
>
>
> The error you got earlier, happens when you do not read the body of the
> request (for example when you have POST data, but you do not read them).
> This is how nginx works. You can avoid that (if you do not want to read
> post data) with --post-buffering  in uWSGI. It is a trick to force
> uWSGI to blindly read the http body independently by your app.
>
> Regarding static files, you have to fight with nginx (uWSGI should only
> manage the python part)
>
> Normally working on the location directives and root/alias options is
> enough
>
> >
> > On Wed, Apr 18, 2012 at 10:46 AM, Bruce Wade 
> wrote:
> >
> >> uwsgi is sure not as easy to get working correctly as apache is. Keep
> >> getting the following:
> >> 2012/04/18 10:44:49 [error] 23686#0: *8454 upstream prematurely closed
> >> connection while reading response header from upstream, client:
> >> 173.180.212.86, server: youadstage, request: "GET / HTTP/1.1",
> upstream:
> >> "uwsgi://127.0.0.1:9001", host: "50.116.6.124"
> >>
> >>
> >> On Wed, Apr 18, 2012 at 9:43 AM, Bruno Rocha
> >> wrote:
> >>
> >>> There are directions for this on book
> >>> http://web2py.com/books/default/chapter/29/4#session
> >>>
> >>> Store session on db or set separate=True
> >>>
> >>>
> >>> On Wed, Apr 18, 2012 at 1:36 PM, Bruno Rocha
> >>> wrote:
> >>>
>  I recommend you to store sessions on db or cache like memcached,
>  redis... If you are going to run in a load balance, sessions in disk
>  will
>  be a problem.
> 
> 
>  On Wed, Apr 18, 2012 at 11:32 AM, Bruce Wade
>  wrote:
> 
> > Currently FS
> >
> >
> > On Wed, Apr 18, 2012 at 7:22 AM, Bruno Rocha
> > wrote:
> >
> >> Are you storing sessions in db or FS?
> >>
> >> http://zerp.ly/rochacbruno
> >> Em 18/04/2012 02:18, "Bruce Wade"  escreveu:
> >>
> >> Yeah, I am not sure how well web2py can handle load but I am going
> >> to
> >>> find out pretty fast. I finished setting up a load balancer today,
> >>> going to
> >>> have 3 servers under it running nginx + uwsgi + web2py a single
> >>> dedicated
> >>> DB server for now, but figure I am going to need some slave read
> >>> only
> >>> servers. The speed at which web2py allows you to develop
> >>> applications
> >>> will surpass any load issues, well unless it could only handle
> >>> 50,000
> >>> users/per second then that would be a big issue :D
> >>>
> >>> My one concern is how good the sticky sessions on linode's
> >>> nodebalancer works.
> >>>
> >>> Regarding uwsgi I have ready time out issues when using it with
> >>> web2py.
> >>>
> >>> --
> >>> Regards,
> >>> Bruce
> >>>
> >>> On Tue, Apr 17, 2012 at 9:55 PM, pbreit
> >>> wrote:
> >>>
>  I've never read anything but good things about uwsgi. Django and
>  Flask both promote it.
> >>>
> >>>
> >>>
> >>>
> >>> --
> >>> --
> >>> Regards,
> >>> Bruce Wade
> >>> http://ca.linkedin.com/in/brucelwade
> >>> http://www.wadecybertech.com
> >>> http://www.fittraineronline.com - Fitness Personal Trainers Online
> >>> http://www.warplydesigned.com
> >>>
> >>>
> >
> >
> > --
> > --
> > Regards,
> > Bruce Wade
> > http://ca.linkedin.com/in/brucelwade
> > http://www.wadecybertech.com
> > http://www.fittraineronline.com - Fitness Personal Trainers Online
> > http://www.warplydesigned.com
> >
> >
> 
> 
>  --
> 
>  Bruno Rocha
>  [http://rochacbruno.com.br]
> 
> 
> >>>
> >>>
> >>> --
> >>>
> >>> Bruno Rocha
> >>> [http://rochacbruno.com.br]
> >>>
> >>>
> >>
> >>
> >> --
> >> --
> >> Regards,
> >> Bruce Wade
> >> http://ca.linkedin.com/in/brucelwade
> >> http://www.wadecybertech.com
> >> http://www.fittraineronline.com - Fitness Personal Trainers Online
> >> http://www.warplydesigned.com
> >>
> >>
> >
> >
> > --
> > --
> > Regards,
> > Bruce Wade
> > http://ca.linkedin.com/in/brucelwade
> > http://www.wadecybertech.com
> > http://www.fittraineronline.com - Fitness Personal Trainers Online
> > http://www.warplydesigned.com
> >
>
>
> --
> Roberto De Ioris
> http://unbit.it
>



-- 
-- 
Regards,
Bruce Wade
http://ca.linkedin.com/in/brucelwade
http://www.wadecybertech.com
http://www.fittraineronline.com - Fitness Personal Trainers Online
http://www.warplydesigned.com

Re: [web2py] CPU 210%

[Roberto De Ioris]

> Ok got it running, restarted uwsgi not sure what the issue was.
>
> However following the guides I my static content isn't being loaded:
> http://50.116.6.124


The error you got earlier, happens when you do not read the body of the
request (for example when you have POST data, but you do not read them).
This is how nginx works. You can avoid that (if you do not want to read
post data) with --post-buffering  in uWSGI. It is a trick to force
uWSGI to blindly read the http body independently by your app.

Regarding static files, you have to fight with nginx (uWSGI should only
manage the python part)

Normally working on the location directives and root/alias options is enough

>
> On Wed, Apr 18, 2012 at 10:46 AM, Bruce Wade  wrote:
>
>> uwsgi is sure not as easy to get working correctly as apache is. Keep
>> getting the following:
>> 2012/04/18 10:44:49 [error] 23686#0: *8454 upstream prematurely closed
>> connection while reading response header from upstream, client:
>> 173.180.212.86, server: youadstage, request: "GET / HTTP/1.1", upstream:
>> "uwsgi://127.0.0.1:9001", host: "50.116.6.124"
>>
>>
>> On Wed, Apr 18, 2012 at 9:43 AM, Bruno Rocha
>> wrote:
>>
>>> There are directions for this on book
>>> http://web2py.com/books/default/chapter/29/4#session
>>>
>>> Store session on db or set separate=True
>>>
>>>
>>> On Wed, Apr 18, 2012 at 1:36 PM, Bruno Rocha
>>> wrote:
>>>
 I recommend you to store sessions on db or cache like memcached,
 redis... If you are going to run in a load balance, sessions in disk
 will
 be a problem.


 On Wed, Apr 18, 2012 at 11:32 AM, Bruce Wade
 wrote:

> Currently FS
>
>
> On Wed, Apr 18, 2012 at 7:22 AM, Bruno Rocha
> wrote:
>
>> Are you storing sessions in db or FS?
>>
>> http://zerp.ly/rochacbruno
>> Em 18/04/2012 02:18, "Bruce Wade"  escreveu:
>>
>> Yeah, I am not sure how well web2py can handle load but I am going
>> to
>>> find out pretty fast. I finished setting up a load balancer today,
>>> going to
>>> have 3 servers under it running nginx + uwsgi + web2py a single
>>> dedicated
>>> DB server for now, but figure I am going to need some slave read
>>> only
>>> servers. The speed at which web2py allows you to develop
>>> applications
>>> will surpass any load issues, well unless it could only handle
>>> 50,000
>>> users/per second then that would be a big issue :D
>>>
>>> My one concern is how good the sticky sessions on linode's
>>> nodebalancer works.
>>>
>>> Regarding uwsgi I have ready time out issues when using it with
>>> web2py.
>>>
>>> --
>>> Regards,
>>> Bruce
>>>
>>> On Tue, Apr 17, 2012 at 9:55 PM, pbreit
>>> wrote:
>>>
 I've never read anything but good things about uwsgi. Django and
 Flask both promote it.
>>>
>>>
>>>
>>>
>>> --
>>> --
>>> Regards,
>>> Bruce Wade
>>> http://ca.linkedin.com/in/brucelwade
>>> http://www.wadecybertech.com
>>> http://www.fittraineronline.com - Fitness Personal Trainers Online
>>> http://www.warplydesigned.com
>>>
>>>
>
>
> --
> --
> Regards,
> Bruce Wade
> http://ca.linkedin.com/in/brucelwade
> http://www.wadecybertech.com
> http://www.fittraineronline.com - Fitness Personal Trainers Online
> http://www.warplydesigned.com
>
>


 --

 Bruno Rocha
 [http://rochacbruno.com.br]


>>>
>>>
>>> --
>>>
>>> Bruno Rocha
>>> [http://rochacbruno.com.br]
>>>
>>>
>>
>>
>> --
>> --
>> Regards,
>> Bruce Wade
>> http://ca.linkedin.com/in/brucelwade
>> http://www.wadecybertech.com
>> http://www.fittraineronline.com - Fitness Personal Trainers Online
>> http://www.warplydesigned.com
>>
>>
>
>
> --
> --
> Regards,
> Bruce Wade
> http://ca.linkedin.com/in/brucelwade
> http://www.wadecybertech.com
> http://www.fittraineronline.com - Fitness Personal Trainers Online
> http://www.warplydesigned.com
>


-- 
Roberto De Ioris
http://unbit.it

Re: [web2py] CPU 210%

[Bruce Wade]

Got it work had to change the static path in the config to remove ~*

On Wed, Apr 18, 2012 at 11:00 AM, Bruce Wade  wrote:

> Ok got it running, restarted uwsgi not sure what the issue was.
>
> However following the guides I my static content isn't being loaded:
> http://50.116.6.124
>
>
> On Wed, Apr 18, 2012 at 10:46 AM, Bruce Wade  wrote:
>
>> uwsgi is sure not as easy to get working correctly as apache is. Keep
>> getting the following:
>> 2012/04/18 10:44:49 [error] 23686#0: *8454 upstream prematurely closed
>> connection while reading response header from upstream, client:
>> 173.180.212.86, server: youadstage, request: "GET / HTTP/1.1", upstream:
>> "uwsgi://127.0.0.1:9001", host: "50.116.6.124"
>>
>>
>> On Wed, Apr 18, 2012 at 9:43 AM, Bruno Rocha wrote:
>>
>>> There are directions for this on book
>>> http://web2py.com/books/default/chapter/29/4#session
>>>
>>> Store session on db or set separate=True
>>>
>>>
>>> On Wed, Apr 18, 2012 at 1:36 PM, Bruno Rocha wrote:
>>>
 I recommend you to store sessions on db or cache like memcached,
 redis... If you are going to run in a load balance, sessions in disk will
 be a problem.


 On Wed, Apr 18, 2012 at 11:32 AM, Bruce Wade wrote:

> Currently FS
>
>
> On Wed, Apr 18, 2012 at 7:22 AM, Bruno Rocha wrote:
>
>> Are you storing sessions in db or FS?
>>
>> http://zerp.ly/rochacbruno
>> Em 18/04/2012 02:18, "Bruce Wade"  escreveu:
>>
>> Yeah, I am not sure how well web2py can handle load but I am going to
>>> find out pretty fast. I finished setting up a load balancer today, 
>>> going to
>>> have 3 servers under it running nginx + uwsgi + web2py a single 
>>> dedicated
>>> DB server for now, but figure I am going to need some slave read only
>>> servers. The speed at which web2py allows you to develop applications
>>> will surpass any load issues, well unless it could only handle 50,000
>>> users/per second then that would be a big issue :D
>>>
>>> My one concern is how good the sticky sessions on linode's
>>> nodebalancer works.
>>>
>>> Regarding uwsgi I have ready time out issues when using it with
>>> web2py.
>>>
>>> --
>>> Regards,
>>> Bruce
>>>
>>> On Tue, Apr 17, 2012 at 9:55 PM, pbreit wrote:
>>>
 I've never read anything but good things about uwsgi. Django and
 Flask both promote it.
>>>
>>>
>>>
>>>
>>> --
>>> --
>>> Regards,
>>> Bruce Wade
>>> http://ca.linkedin.com/in/brucelwade
>>> http://www.wadecybertech.com
>>> http://www.fittraineronline.com - Fitness Personal Trainers Online
>>> http://www.warplydesigned.com
>>>
>>>
>
>
> --
> --
> Regards,
> Bruce Wade
> http://ca.linkedin.com/in/brucelwade
> http://www.wadecybertech.com
> http://www.fittraineronline.com - Fitness Personal Trainers Online
> http://www.warplydesigned.com
>
>


 --

 Bruno Rocha
 [http://rochacbruno.com.br]


>>>
>>>
>>> --
>>>
>>> Bruno Rocha
>>> [http://rochacbruno.com.br]
>>>
>>>
>>
>>
>> --
>> --
>> Regards,
>> Bruce Wade
>> http://ca.linkedin.com/in/brucelwade
>> http://www.wadecybertech.com
>> http://www.fittraineronline.com - Fitness Personal Trainers Online
>> http://www.warplydesigned.com
>>
>>
>
>
> --
> --
> Regards,
> Bruce Wade
> http://ca.linkedin.com/in/brucelwade
> http://www.wadecybertech.com
> http://www.fittraineronline.com - Fitness Personal Trainers Online
> http://www.warplydesigned.com
>
>


-- 
-- 
Regards,
Bruce Wade
http://ca.linkedin.com/in/brucelwade
http://www.wadecybertech.com
http://www.fittraineronline.com - Fitness Personal Trainers Online
http://www.warplydesigned.com

Re: [web2py] CPU 210%

[Bruce Wade]

Ok got it running, restarted uwsgi not sure what the issue was.

However following the guides I my static content isn't being loaded:
http://50.116.6.124

On Wed, Apr 18, 2012 at 10:46 AM, Bruce Wade  wrote:

> uwsgi is sure not as easy to get working correctly as apache is. Keep
> getting the following:
> 2012/04/18 10:44:49 [error] 23686#0: *8454 upstream prematurely closed
> connection while reading response header from upstream, client:
> 173.180.212.86, server: youadstage, request: "GET / HTTP/1.1", upstream:
> "uwsgi://127.0.0.1:9001", host: "50.116.6.124"
>
>
> On Wed, Apr 18, 2012 at 9:43 AM, Bruno Rocha wrote:
>
>> There are directions for this on book
>> http://web2py.com/books/default/chapter/29/4#session
>>
>> Store session on db or set separate=True
>>
>>
>> On Wed, Apr 18, 2012 at 1:36 PM, Bruno Rocha wrote:
>>
>>> I recommend you to store sessions on db or cache like memcached,
>>> redis... If you are going to run in a load balance, sessions in disk will
>>> be a problem.
>>>
>>>
>>> On Wed, Apr 18, 2012 at 11:32 AM, Bruce Wade wrote:
>>>
 Currently FS


 On Wed, Apr 18, 2012 at 7:22 AM, Bruno Rocha wrote:

> Are you storing sessions in db or FS?
>
> http://zerp.ly/rochacbruno
> Em 18/04/2012 02:18, "Bruce Wade"  escreveu:
>
> Yeah, I am not sure how well web2py can handle load but I am going to
>> find out pretty fast. I finished setting up a load balancer today, going 
>> to
>> have 3 servers under it running nginx + uwsgi + web2py a single dedicated
>> DB server for now, but figure I am going to need some slave read only
>> servers. The speed at which web2py allows you to develop applications
>> will surpass any load issues, well unless it could only handle 50,000
>> users/per second then that would be a big issue :D
>>
>> My one concern is how good the sticky sessions on linode's
>> nodebalancer works.
>>
>> Regarding uwsgi I have ready time out issues when using it with
>> web2py.
>>
>> --
>> Regards,
>> Bruce
>>
>> On Tue, Apr 17, 2012 at 9:55 PM, pbreit wrote:
>>
>>> I've never read anything but good things about uwsgi. Django and
>>> Flask both promote it.
>>
>>
>>
>>
>> --
>> --
>> Regards,
>> Bruce Wade
>> http://ca.linkedin.com/in/brucelwade
>> http://www.wadecybertech.com
>> http://www.fittraineronline.com - Fitness Personal Trainers Online
>> http://www.warplydesigned.com
>>
>>


 --
 --
 Regards,
 Bruce Wade
 http://ca.linkedin.com/in/brucelwade
 http://www.wadecybertech.com
 http://www.fittraineronline.com - Fitness Personal Trainers Online
 http://www.warplydesigned.com


>>>
>>>
>>> --
>>>
>>> Bruno Rocha
>>> [http://rochacbruno.com.br]
>>>
>>>
>>
>>
>> --
>>
>> Bruno Rocha
>> [http://rochacbruno.com.br]
>>
>>
>
>
> --
> --
> Regards,
> Bruce Wade
> http://ca.linkedin.com/in/brucelwade
> http://www.wadecybertech.com
> http://www.fittraineronline.com - Fitness Personal Trainers Online
> http://www.warplydesigned.com
>
>


-- 
-- 
Regards,
Bruce Wade
http://ca.linkedin.com/in/brucelwade
http://www.wadecybertech.com
http://www.fittraineronline.com - Fitness Personal Trainers Online
http://www.warplydesigned.com

Re: [web2py] CPU 210%

[Bruce Wade]

uwsgi is sure not as easy to get working correctly as apache is. Keep
getting the following:
2012/04/18 10:44:49 [error] 23686#0: *8454 upstream prematurely closed
connection while reading response header from upstream, client:
173.180.212.86, server: youadstage, request: "GET / HTTP/1.1", upstream:
"uwsgi://127.0.0.1:9001", host: "50.116.6.124"

On Wed, Apr 18, 2012 at 9:43 AM, Bruno Rocha  wrote:

> There are directions for this on book
> http://web2py.com/books/default/chapter/29/4#session
>
> Store session on db or set separate=True
>
>
> On Wed, Apr 18, 2012 at 1:36 PM, Bruno Rocha wrote:
>
>> I recommend you to store sessions on db or cache like memcached, redis...
>> If you are going to run in a load balance, sessions in disk will be a
>> problem.
>>
>>
>> On Wed, Apr 18, 2012 at 11:32 AM, Bruce Wade wrote:
>>
>>> Currently FS
>>>
>>>
>>> On Wed, Apr 18, 2012 at 7:22 AM, Bruno Rocha wrote:
>>>
 Are you storing sessions in db or FS?

 http://zerp.ly/rochacbruno
 Em 18/04/2012 02:18, "Bruce Wade"  escreveu:

 Yeah, I am not sure how well web2py can handle load but I am going to
> find out pretty fast. I finished setting up a load balancer today, going 
> to
> have 3 servers under it running nginx + uwsgi + web2py a single dedicated
> DB server for now, but figure I am going to need some slave read only
> servers. The speed at which web2py allows you to develop applications
> will surpass any load issues, well unless it could only handle 50,000
> users/per second then that would be a big issue :D
>
> My one concern is how good the sticky sessions on linode's
> nodebalancer works.
>
> Regarding uwsgi I have ready time out issues when using it with
> web2py.
>
> --
> Regards,
> Bruce
>
> On Tue, Apr 17, 2012 at 9:55 PM, pbreit wrote:
>
>> I've never read anything but good things about uwsgi. Django and
>> Flask both promote it.
>
>
>
>
> --
> --
> Regards,
> Bruce Wade
> http://ca.linkedin.com/in/brucelwade
> http://www.wadecybertech.com
> http://www.fittraineronline.com - Fitness Personal Trainers Online
> http://www.warplydesigned.com
>
>
>>>
>>>
>>> --
>>> --
>>> Regards,
>>> Bruce Wade
>>> http://ca.linkedin.com/in/brucelwade
>>> http://www.wadecybertech.com
>>> http://www.fittraineronline.com - Fitness Personal Trainers Online
>>> http://www.warplydesigned.com
>>>
>>>
>>
>>
>> --
>>
>> Bruno Rocha
>> [http://rochacbruno.com.br]
>>
>>
>
>
> --
>
> Bruno Rocha
> [http://rochacbruno.com.br]
>
>


-- 
-- 
Regards,
Bruce Wade
http://ca.linkedin.com/in/brucelwade
http://www.wadecybertech.com
http://www.fittraineronline.com - Fitness Personal Trainers Online
http://www.warplydesigned.com

Re: [web2py] CPU 210%

[Bruno Rocha]

There are directions for this on book
http://web2py.com/books/default/chapter/29/4#session

Store session on db or set separate=True

On Wed, Apr 18, 2012 at 1:36 PM, Bruno Rocha  wrote:

> I recommend you to store sessions on db or cache like memcached, redis...
> If you are going to run in a load balance, sessions in disk will be a
> problem.
>
>
> On Wed, Apr 18, 2012 at 11:32 AM, Bruce Wade  wrote:
>
>> Currently FS
>>
>>
>> On Wed, Apr 18, 2012 at 7:22 AM, Bruno Rocha wrote:
>>
>>> Are you storing sessions in db or FS?
>>>
>>> http://zerp.ly/rochacbruno
>>> Em 18/04/2012 02:18, "Bruce Wade"  escreveu:
>>>
>>> Yeah, I am not sure how well web2py can handle load but I am going to
 find out pretty fast. I finished setting up a load balancer today, going to
 have 3 servers under it running nginx + uwsgi + web2py a single dedicated
 DB server for now, but figure I am going to need some slave read only
 servers. The speed at which web2py allows you to develop applications
 will surpass any load issues, well unless it could only handle 50,000
 users/per second then that would be a big issue :D

 My one concern is how good the sticky sessions on linode's nodebalancer
 works.

 Regarding uwsgi I have ready time out issues when using it with web2py.

 --
 Regards,
 Bruce

 On Tue, Apr 17, 2012 at 9:55 PM, pbreit  wrote:

> I've never read anything but good things about uwsgi. Django and Flask
> both promote it.




 --
 --
 Regards,
 Bruce Wade
 http://ca.linkedin.com/in/brucelwade
 http://www.wadecybertech.com
 http://www.fittraineronline.com - Fitness Personal Trainers Online
 http://www.warplydesigned.com


>>
>>
>> --
>> --
>> Regards,
>> Bruce Wade
>> http://ca.linkedin.com/in/brucelwade
>> http://www.wadecybertech.com
>> http://www.fittraineronline.com - Fitness Personal Trainers Online
>> http://www.warplydesigned.com
>>
>>
>
>
> --
>
> Bruno Rocha
> [http://rochacbruno.com.br]
>
>


-- 

Bruno Rocha
[http://rochacbruno.com.br]

Re: [web2py] CPU 210%

[Bruno Rocha]

I recommend you to store sessions on db or cache like memcached, redis...
If you are going to run in a load balance, sessions in disk will be a
problem.

On Wed, Apr 18, 2012 at 11:32 AM, Bruce Wade  wrote:

> Currently FS
>
>
> On Wed, Apr 18, 2012 at 7:22 AM, Bruno Rocha wrote:
>
>> Are you storing sessions in db or FS?
>>
>> http://zerp.ly/rochacbruno
>> Em 18/04/2012 02:18, "Bruce Wade"  escreveu:
>>
>> Yeah, I am not sure how well web2py can handle load but I am going to
>>> find out pretty fast. I finished setting up a load balancer today, going to
>>> have 3 servers under it running nginx + uwsgi + web2py a single dedicated
>>> DB server for now, but figure I am going to need some slave read only
>>> servers. The speed at which web2py allows you to develop applications
>>> will surpass any load issues, well unless it could only handle 50,000
>>> users/per second then that would be a big issue :D
>>>
>>> My one concern is how good the sticky sessions on linode's nodebalancer
>>> works.
>>>
>>> Regarding uwsgi I have ready time out issues when using it with web2py.
>>>
>>> --
>>> Regards,
>>> Bruce
>>>
>>> On Tue, Apr 17, 2012 at 9:55 PM, pbreit  wrote:
>>>
 I've never read anything but good things about uwsgi. Django and Flask
 both promote it.
>>>
>>>
>>>
>>>
>>> --
>>> --
>>> Regards,
>>> Bruce Wade
>>> http://ca.linkedin.com/in/brucelwade
>>> http://www.wadecybertech.com
>>> http://www.fittraineronline.com - Fitness Personal Trainers Online
>>> http://www.warplydesigned.com
>>>
>>>
>
>
> --
> --
> Regards,
> Bruce Wade
> http://ca.linkedin.com/in/brucelwade
> http://www.wadecybertech.com
> http://www.fittraineronline.com - Fitness Personal Trainers Online
> http://www.warplydesigned.com
>
>


-- 

Bruno Rocha
[http://rochacbruno.com.br]

Re: [web2py] CPU 210%

[Bruce Wade]

Currently FS

On Wed, Apr 18, 2012 at 7:22 AM, Bruno Rocha  wrote:

> Are you storing sessions in db or FS?
>
> http://zerp.ly/rochacbruno
> Em 18/04/2012 02:18, "Bruce Wade"  escreveu:
>
> Yeah, I am not sure how well web2py can handle load but I am going to find
>> out pretty fast. I finished setting up a load balancer today, going to have
>> 3 servers under it running nginx + uwsgi + web2py a single dedicated DB
>> server for now, but figure I am going to need some slave read only servers.
>> The speed at which web2py allows you to develop applications
>> will surpass any load issues, well unless it could only handle 50,000
>> users/per second then that would be a big issue :D
>>
>> My one concern is how good the sticky sessions on linode's nodebalancer
>> works.
>>
>> Regarding uwsgi I have ready time out issues when using it with web2py.
>>
>> --
>> Regards,
>> Bruce
>>
>> On Tue, Apr 17, 2012 at 9:55 PM, pbreit  wrote:
>>
>>> I've never read anything but good things about uwsgi. Django and Flask
>>> both promote it.
>>
>>
>>
>>
>> --
>> --
>> Regards,
>> Bruce Wade
>> http://ca.linkedin.com/in/brucelwade
>> http://www.wadecybertech.com
>> http://www.fittraineronline.com - Fitness Personal Trainers Online
>> http://www.warplydesigned.com
>>
>>


-- 
-- 
Regards,
Bruce Wade
http://ca.linkedin.com/in/brucelwade
http://www.wadecybertech.com
http://www.fittraineronline.com - Fitness Personal Trainers Online
http://www.warplydesigned.com

Re: [web2py] CPU 210%

[Bruno Rocha]

Are you storing sessions in db or FS?

http://zerp.ly/rochacbruno
Em 18/04/2012 02:18, "Bruce Wade"  escreveu:

> Yeah, I am not sure how well web2py can handle load but I am going to find
> out pretty fast. I finished setting up a load balancer today, going to have
> 3 servers under it running nginx + uwsgi + web2py a single dedicated DB
> server for now, but figure I am going to need some slave read only servers.
> The speed at which web2py allows you to develop applications
> will surpass any load issues, well unless it could only handle 50,000
> users/per second then that would be a big issue :D
>
> My one concern is how good the sticky sessions on linode's nodebalancer
> works.
>
> Regarding uwsgi I have ready time out issues when using it with web2py.
>
> --
> Regards,
> Bruce
>
> On Tue, Apr 17, 2012 at 9:55 PM, pbreit  wrote:
>
>> I've never read anything but good things about uwsgi. Django and Flask
>> both promote it.
>
>
>
>
> --
> --
> Regards,
> Bruce Wade
> http://ca.linkedin.com/in/brucelwade
> http://www.wadecybertech.com
> http://www.fittraineronline.com - Fitness Personal Trainers Online
> http://www.warplydesigned.com
>
>

Re: [web2py] CPU 210%

[Bruce Wade]

LOL sorry I didn't mean programming threads, I mean't form threads, meaning
people posting comments in forms about issues. I can see how that was miss
understood. Honestly I am kind of liking the idea behind event based
servers more then thread based.

On Tue, Apr 17, 2012 at 11:54 PM, Roberto De Ioris  wrote:

>
> > Thanks this is very helpful to know. I am building a system with nginx
> and
> > uwsgi to get the system running on that. I have been avoiding using uwsgi
> > for anything serious because I have been seeing a lot of threads
> reporting
> > problems with it.
>
> ...and you will continue to see that :)
>
> Threads are hard to domate (in-fact most of the time you will go
> multiprocess). The current (1.2) uWSGI threading implementation is pretty
> solid (while 0.9.x are completely broken), but being an ultra-complex part
> i am pretty sure more bugs will pop-up in the future. (and i have to
> thanks Graham Dumpleton [author of mod_wsgi] for pointing me to better
> directions)
>
> By the way, it is pretty strange you found 'threading' support a
> uwsgi-blocker as very few (maybe 3-4, uWSGI included) production-grade
> WSGI servers seriously supports them (with mod_wsgi having the better
> implementation for sure).
>
> In addition to this, a little part of web-related popular libraries are
> not thread-safe, that is why multithreading is not a "too much popular"
> paradigm in python-hosting.
>
>
> --
> Roberto De Ioris
> http://unbit.it
>



-- 
-- 
Regards,
Bruce Wade
http://ca.linkedin.com/in/brucelwade
http://www.wadecybertech.com
http://www.fittraineronline.com - Fitness Personal Trainers Online
http://www.warplydesigned.com

Re: [web2py] CPU 210%

[Roberto De Ioris]

> Thanks this is very helpful to know. I am building a system with nginx and
> uwsgi to get the system running on that. I have been avoiding using uwsgi
> for anything serious because I have been seeing a lot of threads reporting
> problems with it.

...and you will continue to see that :)

Threads are hard to domate (in-fact most of the time you will go
multiprocess). The current (1.2) uWSGI threading implementation is pretty
solid (while 0.9.x are completely broken), but being an ultra-complex part
i am pretty sure more bugs will pop-up in the future. (and i have to
thanks Graham Dumpleton [author of mod_wsgi] for pointing me to better
directions)

By the way, it is pretty strange you found 'threading' support a
uwsgi-blocker as very few (maybe 3-4, uWSGI included) production-grade
WSGI servers seriously supports them (with mod_wsgi having the better
implementation for sure).

In addition to this, a little part of web-related popular libraries are
not thread-safe, that is why multithreading is not a "too much popular"
paradigm in python-hosting.


-- 
Roberto De Ioris
http://unbit.it

Re: [web2py] CPU 210%

[Bruce Wade]

OK when having your web2py app running on multiple servers during load
balancing. What is the best way to handle the error's so I don't have to
constantly lot into each server just to find out the problem?

On Tue, Apr 17, 2012 at 10:17 PM, Bruce Wade  wrote:

> Yeah, I am not sure how well web2py can handle load but I am going to find
> out pretty fast. I finished setting up a load balancer today, going to have
> 3 servers under it running nginx + uwsgi + web2py a single dedicated DB
> server for now, but figure I am going to need some slave read only servers.
> The speed at which web2py allows you to develop applications
> will surpass any load issues, well unless it could only handle 50,000
> users/per second then that would be a big issue :D
>
> My one concern is how good the sticky sessions on linode's nodebalancer
> works.
>
> Regarding uwsgi I have ready time out issues when using it with web2py.
>
> --
> Regards,
> Bruce
>
>
> On Tue, Apr 17, 2012 at 9:55 PM, pbreit  wrote:
>
>> I've never read anything but good things about uwsgi. Django and Flask
>> both promote it.
>
>
>
>
> --
> --
> Regards,
> Bruce Wade
> http://ca.linkedin.com/in/brucelwade
> http://www.wadecybertech.com
> http://www.fittraineronline.com - Fitness Personal Trainers Online
> http://www.warplydesigned.com
>
>


-- 
-- 
Regards,
Bruce Wade
http://ca.linkedin.com/in/brucelwade
http://www.wadecybertech.com
http://www.fittraineronline.com - Fitness Personal Trainers Online
http://www.warplydesigned.com

Re: [web2py] CPU 210%

[Bruce Wade]

Yeah, I am not sure how well web2py can handle load but I am going to find
out pretty fast. I finished setting up a load balancer today, going to have
3 servers under it running nginx + uwsgi + web2py a single dedicated DB
server for now, but figure I am going to need some slave read only servers.
The speed at which web2py allows you to develop applications
will surpass any load issues, well unless it could only handle 50,000
users/per second then that would be a big issue :D

My one concern is how good the sticky sessions on linode's nodebalancer
works.

Regarding uwsgi I have ready time out issues when using it with web2py.

--
Regards,
Bruce

On Tue, Apr 17, 2012 at 9:55 PM, pbreit  wrote:

> I've never read anything but good things about uwsgi. Django and Flask
> both promote it.




-- 
-- 
Regards,
Bruce Wade
http://ca.linkedin.com/in/brucelwade
http://www.wadecybertech.com
http://www.fittraineronline.com - Fitness Personal Trainers Online
http://www.warplydesigned.com

Re: [web2py] CPU 210%

[pbreit]

I've never read anything but good things about uwsgi. Django and Flask both 
promote it.

Re: [web2py] CPU 210%

[Michele Comitini]

Apropos of Bruno...

 my rules are more from a sysadmin point of view, while Bruno has
defined a way to boost performance from the application developer POV.

Bruno rule is simple but effective:
1. move as much as you can out of the models and put it in the modules.
The speed increase can be dramatic.  The details are in some thread of
this forum.

Summary:
Write the app following Bruno's layout.
When deploying follow the 10 rules I wrote.

I am I tired of hearing that framework x performs better than web2py!
Can't be true! ;-)

mic

Il 17 aprile 2012 22:47, Bruno Rocha  ha scritto:
> Or you can just hire Michele to solve all your scaling problems! :)
>
>
> On Tue, Apr 17, 2012 at 5:40 PM, Michele Comitini
>  wrote:
>>
>> Richard you are right.  We should make a web2py slice out of this.  I
>> keep repeating same things over from time to time, I suppose it's the
>> age... ;-)
>>
>> Anyway a put a star on the thread so one day I will write.
>>
>> Here are 10 system configuration rules I follow for high scalability
>> with web2py on multicore machines:
>> 1. use processes not threads or use pypy
>> 2. use an event based http frontend
>> 3. use UWSGI or SCGI or FCGI
>> 4. use HTTP keepalive
>> 5. treat static content as static i.e. keep it out of the views
>> 6. reduce number and size of static files, i.e. sprites and pack js
>> and css and aggregate them in a file
>> 7. support gzip encoding for files > 1KB
>> 8. put the database(s) on a different machine(s) or double the cores
>> i.e. for each request have 2 cores at hand
>> 9. cache dynamic content leveraging on web2py cache
>> 10. use the dal pooling machinery, i.e. db=DAL(..., pool_size=1)
>>
>> About the pool_size parameter is very important to reduce response
>> time and use an adequate value.
>> The above pool_size=1 using processes seems weird, but is not: each
>> request is handled by a process that needs only a single connection no
>> more,
>> using threads pool_size needs a value >= to the maximum number of
>> threads in a web2py instance.
>> The total number of persistent connections to the db is conn_n =
>> proc_num x pool_size (with proc_num = number of processes). So if you
>> have 20 processes (proc_num=20) with pool_size=1:
>>
>> conn_n = 20 x 1 = 20
>>
>> If you have 20 threads (proc_num = 1) with pool_size=20:
>>
>> conn_n = 1 x 20 = 20
>>
>> .
>>
>> mic
>>
>> Il 17 aprile 2012 21:00, Richard Vézina 
>> ha scritto:
>> > Michele,
>> >
>> > I read this thread, not sure if it is not already existing... But it
>> > start
>> > to look like a recipe for speed tuning that could be translate into a
>> > web2py
>> > slice or blog post :)
>> >
>> > Richard
>> >
>> >
>> > On Tue, Apr 17, 2012 at 2:32 PM, Michele Comitini
>> >  wrote:
>> >>
>> >> What I suggest is use nginx + uwsgi or nginx + scgi or nginx + fcgi.
>> >> You will notice big reduction in resource consuption compared to
>> >> Apache.
>> >> Apache can be taken away.  By the way you can use any of the above
>> >> protocols to do balancing over a pool of machines running web2py and
>> >> in front you can put nginx as balancer.
>> >>
>> >> About caching, what is important is using as much as possible the
>> >> *static* dir and have the files underneath served by nginx directly.
>> >> You must check that expiration times on static objects are correctly
>> >> set.  Also use sprites as much as you can.  You will enjoy a big
>> >> improvement because after downloading the objects the first time a
>> >> client browser will make only requests for the dynamically generated
>> >> contents.  Other objects will be taken by the ondisk cache of the
>> >> browser. If  the object is expired  in browser cache, but the content
>> >> not changed on the server, the request will result only in a 304
>> >> answer so little data is exchanged and little computation is required.
>> >>
>> >> mic
>> >>
>> >>
>> >> Il 17 aprile 2012 19:10, Bruce Wade  ha scritto:
>> >> > Currently I just had 1 server running apache mod_wsgi using the same
>> >> > configuration as pyramid. However I just got approved for a few grand
>> >> > a
>> >> > month to spend on server resources, so I am looking at load
>> >> > balancers.
>> >> > And I
>> >> > will put nginx in front of apache, and also start using a lot more
>> >> > caching.
>> >> >
>> >> >
>> >> > On Tue, Apr 17, 2012 at 5:15 AM, Michele Comitini
>> >> >  wrote:
>> >> >>
>> >> >> One more thing make css and js packed + server side gzipped (nginx
>> >> >> and
>> >> >> cherokee can do also gzip caching)
>> >> >>
>> >> >> mic
>> >> >>
>> >> >> Il 17 aprile 2012 14:12, Michele Comitini
>> >> >> 
>> >> >> ha scritto:
>> >> >> > If you are on postgreSQL use a process per request setup, you will
>> >> >> > have a great benefit.  Use cherokee or nginx (with keepalive
>> >> >> > working)
>> >> >> > you will scale smoothly.
>> >> >> >
>> >> >> > Check that you do as much as possible of a page in a single http
>> >> >> > request (i.e. limit ajax load).  Use only one cacheable css and
>> 

Re: [web2py] CPU 210%

[Bruce Wade]

Thanks this is very helpful to know. I am building a system with nginx and
uwsgi to get the system running on that. I have been avoiding using uwsgi
for anything serious because I have been seeing a lot of threads reporting
problems with it.

Michele thanks for the feed back I am working on implementing your 10 steps
and will use the profiler to see where things need to be optimized.

--
Regards,
Bruce


On Tue, Apr 17, 2012 at 1:47 PM, Michele Comitini <
michele.comit...@gmail.com> wrote:

> As always Massimo is also correct on this.
> Do not try to optimize code before the profiler says where to optimize.
> Use the profiler any time you want increase performance of the code
> you wrote, it helps you save much time, since unless you optimize
> every line of code you will end optimizing things that do not impact
> on the overall speed.
>
> mic
>
> Il 17 aprile 2012 21:29, Richard Vézina 
> ha scritto:
> > Massimo post :
> https://groups.google.com/forum/#!topic/web2py/hB3y_6gFIi4
> >
> > Pretty instructive on speed trouble shooting.
> >
> > Richard
> >
> >
> > On Tue, Apr 17, 2012 at 3:02 PM, Ron McOuat 
> wrote:
> >>
> >> I ran across newrelic.com which offers profiling for Python web apps
> >> amongst several other environments. I did a search on the web2py users
> list
> >> and found a few threads about a problem using LIKE % values with
> PostgreSQL
> >> but it appears to have been resolved. It looks like it might help with
> >> identifying the problems areas which helps find the low hanging fruit
> for
> >> caching. Does anyone else have experience with using this tool? You can
> use
> >> it for a 2 week trial, then there is a cost. I am not associated with
> them
> >> and have not used the product in any way, just passing on the fact they
> >> exist.
> >>
> >> Ron
> >>
> >>
> >> On Tuesday, 17 April 2012 10:10:08 UTC-7, Bruce Wade wrote:
> >>>
> >>> Currently I just had 1 server running apache mod_wsgi using the same
> >>> configuration as pyramid. However I just got approved for a few grand a
> >>> month to spend on server resources, so I am looking at load balancers.
> And I
> >>> will put nginx in front of apache, and also start using a lot more
> caching.
> >>>
> >>> On Tue, Apr 17, 2012 at 5:15 AM, Michele Comitini
> >>>  wrote:
> 
>  One more thing make css and js packed + server side gzipped (nginx and
>  cherokee can do also gzip caching)
> 
>  mic
> 
>  Il 17 aprile 2012 14:12, Michele Comitini  >
>  ha scritto:
>  > If you are on postgreSQL use a process per request setup, you will
>  > have a great benefit.  Use cherokee or nginx (with keepalive
> working)
>  > you will scale smoothly.
>  >
>  > Check that you do as much as possible of a page in a single http
>  > request (i.e. limit ajax load).  Use only one cacheable css and
> limit
>  > the number of scripts or aggregate them in a cacheable file.
>  > Check that everything that is cacheable gets cached indeed (use
>  > firebug or chrome dev tools to find out).
>  >
>  > mic
>  >
>  >
>  > Il 17 aprile 2012 14:07, Michele Comitini <
> michele.comit...@gmail.com>
>  > ha scritto:
>  >> What is your architecture?  What do you use as frontend http
> server?
>  >> What protocol: SCGI, UWSGI, FCGI...?
>  >> Are you in a thread per request or process per request setup?
>  >>
>  >> mic
>  >>
>  >>
>  >> Il 17 aprile 2012 08:36, Bruce Wade  ha
>  >> scritto:
>  >>> Yes you are correct plus there was 10,000+ requests a second just
>  >>> hitting
>  >>> the site I think I really need a load balanced. We are getting on
>  >>> average
>  >>> 500-1000 new members a day.
>  >>>
>  >>> On Apr 16, 2012 10:59 PM, "pbreit" 
> wrote:
>  
>   Don't forget you probably spent quite a bit if time tuning your
>   Pyramid
>   app.
>  
>   The best ways to scale are:
>   1) Cache
>   2) Cache
>   3) Cache
>  
>   Web2py makes caching queries super easy.
>  
>   If you are serving a lot of static assets, check out Cloudflare
> for
>   free
>   CDN.
> >>>
> >>>
> >>>
> >>>
> >>> --
> >>> --
> >>> Regards,
> >>> Bruce Wade
> >>> http://ca.linkedin.com/in/brucelwade
> >>> http://www.wadecybertech.com
> >>> http://www.fittraineronline.com - Fitness Personal Trainers Online
> >>> http://www.warplydesigned.com
> >>>
> >
>



-- 
-- 
Regards,
Bruce Wade
http://ca.linkedin.com/in/brucelwade
http://www.wadecybertech.com
http://www.fittraineronline.com - Fitness Personal Trainers Online
http://www.warplydesigned.com

Re: [web2py] CPU 210%

[Michele Comitini]

As always Massimo is also correct on this.
Do not try to optimize code before the profiler says where to optimize.
Use the profiler any time you want increase performance of the code
you wrote, it helps you save much time, since unless you optimize
every line of code you will end optimizing things that do not impact
on the overall speed.

mic

Il 17 aprile 2012 21:29, Richard Vézina 
ha scritto:
> Massimo post : https://groups.google.com/forum/#!topic/web2py/hB3y_6gFIi4
>
> Pretty instructive on speed trouble shooting.
>
> Richard
>
>
> On Tue, Apr 17, 2012 at 3:02 PM, Ron McOuat  wrote:
>>
>> I ran across newrelic.com which offers profiling for Python web apps
>> amongst several other environments. I did a search on the web2py users list
>> and found a few threads about a problem using LIKE % values with PostgreSQL
>> but it appears to have been resolved. It looks like it might help with
>> identifying the problems areas which helps find the low hanging fruit for
>> caching. Does anyone else have experience with using this tool? You can use
>> it for a 2 week trial, then there is a cost. I am not associated with them
>> and have not used the product in any way, just passing on the fact they
>> exist.
>>
>> Ron
>>
>>
>> On Tuesday, 17 April 2012 10:10:08 UTC-7, Bruce Wade wrote:
>>>
>>> Currently I just had 1 server running apache mod_wsgi using the same
>>> configuration as pyramid. However I just got approved for a few grand a
>>> month to spend on server resources, so I am looking at load balancers. And I
>>> will put nginx in front of apache, and also start using a lot more caching.
>>>
>>> On Tue, Apr 17, 2012 at 5:15 AM, Michele Comitini
>>>  wrote:

 One more thing make css and js packed + server side gzipped (nginx and
 cherokee can do also gzip caching)

 mic

 Il 17 aprile 2012 14:12, Michele Comitini 
 ha scritto:
 > If you are on postgreSQL use a process per request setup, you will
 > have a great benefit.  Use cherokee or nginx (with keepalive working)
 > you will scale smoothly.
 >
 > Check that you do as much as possible of a page in a single http
 > request (i.e. limit ajax load).  Use only one cacheable css and limit
 > the number of scripts or aggregate them in a cacheable file.
 > Check that everything that is cacheable gets cached indeed (use
 > firebug or chrome dev tools to find out).
 >
 > mic
 >
 >
 > Il 17 aprile 2012 14:07, Michele Comitini 
 > ha scritto:
 >> What is your architecture?  What do you use as frontend http server?
 >> What protocol: SCGI, UWSGI, FCGI...?
 >> Are you in a thread per request or process per request setup?
 >>
 >> mic
 >>
 >>
 >> Il 17 aprile 2012 08:36, Bruce Wade  ha
 >> scritto:
 >>> Yes you are correct plus there was 10,000+ requests a second just
 >>> hitting
 >>> the site I think I really need a load balanced. We are getting on
 >>> average
 >>> 500-1000 new members a day.
 >>>
 >>> On Apr 16, 2012 10:59 PM, "pbreit"  wrote:
 
  Don't forget you probably spent quite a bit if time tuning your
  Pyramid
  app.
 
  The best ways to scale are:
  1) Cache
  2) Cache
  3) Cache
 
  Web2py makes caching queries super easy.
 
  If you are serving a lot of static assets, check out Cloudflare for
  free
  CDN.
>>>
>>>
>>>
>>>
>>> --
>>> --
>>> Regards,
>>> Bruce Wade
>>> http://ca.linkedin.com/in/brucelwade
>>> http://www.wadecybertech.com
>>> http://www.fittraineronline.com - Fitness Personal Trainers Online
>>> http://www.warplydesigned.com
>>>
>

Re: [web2py] CPU 210%

[Bruno Rocha]

Or you can just hire Michele to solve all your scaling problems! :)

On Tue, Apr 17, 2012 at 5:40 PM, Michele Comitini <
michele.comit...@gmail.com> wrote:

> Richard you are right.  We should make a web2py slice out of this.  I
> keep repeating same things over from time to time, I suppose it's the
> age... ;-)
>
> Anyway a put a star on the thread so one day I will write.
>
> Here are 10 system configuration rules I follow for high scalability
> with web2py on multicore machines:
> 1. use processes not threads or use pypy
> 2. use an event based http frontend
> 3. use UWSGI or SCGI or FCGI
> 4. use HTTP keepalive
> 5. treat static content as static i.e. keep it out of the views
> 6. reduce number and size of static files, i.e. sprites and pack js
> and css and aggregate them in a file
> 7. support gzip encoding for files > 1KB
> 8. put the database(s) on a different machine(s) or double the cores
> i.e. for each request have 2 cores at hand
> 9. cache dynamic content leveraging on web2py cache
> 10. use the dal pooling machinery, i.e. db=DAL(..., pool_size=1)
>
> About the pool_size parameter is very important to reduce response
> time and use an adequate value.
> The above pool_size=1 using processes seems weird, but is not: each
> request is handled by a process that needs only a single connection no
> more,
> using threads pool_size needs a value >= to the maximum number of
> threads in a web2py instance.
> The total number of persistent connections to the db is conn_n =
> proc_num x pool_size (with proc_num = number of processes). So if you
> have 20 processes (proc_num=20) with pool_size=1:
>
> conn_n = 20 x 1 = 20
>
> If you have 20 threads (proc_num = 1) with pool_size=20:
>
> conn_n = 1 x 20 = 20
>
> .
>
> mic
>
> Il 17 aprile 2012 21:00, Richard Vézina 
> ha scritto:
> > Michele,
> >
> > I read this thread, not sure if it is not already existing... But it
> start
> > to look like a recipe for speed tuning that could be translate into a
> web2py
> > slice or blog post :)
> >
> > Richard
> >
> >
> > On Tue, Apr 17, 2012 at 2:32 PM, Michele Comitini
> >  wrote:
> >>
> >> What I suggest is use nginx + uwsgi or nginx + scgi or nginx + fcgi.
> >> You will notice big reduction in resource consuption compared to
> >> Apache.
> >> Apache can be taken away.  By the way you can use any of the above
> >> protocols to do balancing over a pool of machines running web2py and
> >> in front you can put nginx as balancer.
> >>
> >> About caching, what is important is using as much as possible the
> >> *static* dir and have the files underneath served by nginx directly.
> >> You must check that expiration times on static objects are correctly
> >> set.  Also use sprites as much as you can.  You will enjoy a big
> >> improvement because after downloading the objects the first time a
> >> client browser will make only requests for the dynamically generated
> >> contents.  Other objects will be taken by the ondisk cache of the
> >> browser. If  the object is expired  in browser cache, but the content
> >> not changed on the server, the request will result only in a 304
> >> answer so little data is exchanged and little computation is required.
> >>
> >> mic
> >>
> >>
> >> Il 17 aprile 2012 19:10, Bruce Wade  ha scritto:
> >> > Currently I just had 1 server running apache mod_wsgi using the same
> >> > configuration as pyramid. However I just got approved for a few grand
> a
> >> > month to spend on server resources, so I am looking at load balancers.
> >> > And I
> >> > will put nginx in front of apache, and also start using a lot more
> >> > caching.
> >> >
> >> >
> >> > On Tue, Apr 17, 2012 at 5:15 AM, Michele Comitini
> >> >  wrote:
> >> >>
> >> >> One more thing make css and js packed + server side gzipped (nginx
> and
> >> >> cherokee can do also gzip caching)
> >> >>
> >> >> mic
> >> >>
> >> >> Il 17 aprile 2012 14:12, Michele Comitini <
> michele.comit...@gmail.com>
> >> >> ha scritto:
> >> >> > If you are on postgreSQL use a process per request setup, you will
> >> >> > have a great benefit.  Use cherokee or nginx (with keepalive
> working)
> >> >> > you will scale smoothly.
> >> >> >
> >> >> > Check that you do as much as possible of a page in a single http
> >> >> > request (i.e. limit ajax load).  Use only one cacheable css and
> limit
> >> >> > the number of scripts or aggregate them in a cacheable file.
> >> >> > Check that everything that is cacheable gets cached indeed (use
> >> >> > firebug or chrome dev tools to find out).
> >> >> >
> >> >> > mic
> >> >> >
> >> >> >
> >> >> > Il 17 aprile 2012 14:07, Michele Comitini
> >> >> > 
> >> >> > ha scritto:
> >> >> >> What is your architecture?  What do you use as frontend http
> server?
> >> >> >> What protocol: SCGI, UWSGI, FCGI...?
> >> >> >> Are you in a thread per request or process per request setup?
> >> >> >>
> >> >> >> mic
> >> >> >>
> >> >> >>
> >> >> >> Il 17 aprile 2012 08:36, Bruce Wade  ha
> >> >> >> scritto:
> >> >> >>> Yes you are correct plu

Re: [web2py] CPU 210%

[Michele Comitini]

Richard you are right.  We should make a web2py slice out of this.  I
keep repeating same things over from time to time, I suppose it's the
age... ;-)

Anyway a put a star on the thread so one day I will write.

Here are 10 system configuration rules I follow for high scalability
with web2py on multicore machines:
1. use processes not threads or use pypy
2. use an event based http frontend
3. use UWSGI or SCGI or FCGI
4. use HTTP keepalive
5. treat static content as static i.e. keep it out of the views
6. reduce number and size of static files, i.e. sprites and pack js
and css and aggregate them in a file
7. support gzip encoding for files > 1KB
8. put the database(s) on a different machine(s) or double the cores
i.e. for each request have 2 cores at hand
9. cache dynamic content leveraging on web2py cache
10. use the dal pooling machinery, i.e. db=DAL(..., pool_size=1)

About the pool_size parameter is very important to reduce response
time and use an adequate value.
The above pool_size=1 using processes seems weird, but is not: each
request is handled by a process that needs only a single connection no
more,
using threads pool_size needs a value >= to the maximum number of
threads in a web2py instance.
The total number of persistent connections to the db is conn_n =
proc_num x pool_size (with proc_num = number of processes). So if you
have 20 processes (proc_num=20) with pool_size=1:

conn_n = 20 x 1 = 20

If you have 20 threads (proc_num = 1) with pool_size=20:

conn_n = 1 x 20 = 20

.

mic

Il 17 aprile 2012 21:00, Richard Vézina 
ha scritto:
> Michele,
>
> I read this thread, not sure if it is not already existing... But it start
> to look like a recipe for speed tuning that could be translate into a web2py
> slice or blog post :)
>
> Richard
>
>
> On Tue, Apr 17, 2012 at 2:32 PM, Michele Comitini
>  wrote:
>>
>> What I suggest is use nginx + uwsgi or nginx + scgi or nginx + fcgi.
>> You will notice big reduction in resource consuption compared to
>> Apache.
>> Apache can be taken away.  By the way you can use any of the above
>> protocols to do balancing over a pool of machines running web2py and
>> in front you can put nginx as balancer.
>>
>> About caching, what is important is using as much as possible the
>> *static* dir and have the files underneath served by nginx directly.
>> You must check that expiration times on static objects are correctly
>> set.  Also use sprites as much as you can.  You will enjoy a big
>> improvement because after downloading the objects the first time a
>> client browser will make only requests for the dynamically generated
>> contents.  Other objects will be taken by the ondisk cache of the
>> browser. If  the object is expired  in browser cache, but the content
>> not changed on the server, the request will result only in a 304
>> answer so little data is exchanged and little computation is required.
>>
>> mic
>>
>>
>> Il 17 aprile 2012 19:10, Bruce Wade  ha scritto:
>> > Currently I just had 1 server running apache mod_wsgi using the same
>> > configuration as pyramid. However I just got approved for a few grand a
>> > month to spend on server resources, so I am looking at load balancers.
>> > And I
>> > will put nginx in front of apache, and also start using a lot more
>> > caching.
>> >
>> >
>> > On Tue, Apr 17, 2012 at 5:15 AM, Michele Comitini
>> >  wrote:
>> >>
>> >> One more thing make css and js packed + server side gzipped (nginx and
>> >> cherokee can do also gzip caching)
>> >>
>> >> mic
>> >>
>> >> Il 17 aprile 2012 14:12, Michele Comitini 
>> >> ha scritto:
>> >> > If you are on postgreSQL use a process per request setup, you will
>> >> > have a great benefit.  Use cherokee or nginx (with keepalive working)
>> >> > you will scale smoothly.
>> >> >
>> >> > Check that you do as much as possible of a page in a single http
>> >> > request (i.e. limit ajax load).  Use only one cacheable css and limit
>> >> > the number of scripts or aggregate them in a cacheable file.
>> >> > Check that everything that is cacheable gets cached indeed (use
>> >> > firebug or chrome dev tools to find out).
>> >> >
>> >> > mic
>> >> >
>> >> >
>> >> > Il 17 aprile 2012 14:07, Michele Comitini
>> >> > 
>> >> > ha scritto:
>> >> >> What is your architecture?  What do you use as frontend http server?
>> >> >> What protocol: SCGI, UWSGI, FCGI...?
>> >> >> Are you in a thread per request or process per request setup?
>> >> >>
>> >> >> mic
>> >> >>
>> >> >>
>> >> >> Il 17 aprile 2012 08:36, Bruce Wade  ha
>> >> >> scritto:
>> >> >>> Yes you are correct plus there was 10,000+ requests a second just
>> >> >>> hitting
>> >> >>> the site I think I really need a load balanced. We are getting on
>> >> >>> average
>> >> >>> 500-1000 new members a day.
>> >> >>>
>> >> >>> On Apr 16, 2012 10:59 PM, "pbreit"  wrote:
>> >> 
>> >>  Don't forget you probably spent quite a bit if time tuning your
>> >>  Pyramid
>> >>  app.
>> >> 
>> >>  The best ways to scale are:
>> >> 

Re: [web2py] CPU 210%

[Richard Vézina]

Massimo post : https://groups.google.com/forum/#!topic/web2py/hB3y_6gFIi4

Pretty instructive on speed trouble shooting.

Richard

On Tue, Apr 17, 2012 at 3:02 PM, Ron McOuat  wrote:

> I ran across newrelic.com which offers profiling for Python web apps
> amongst several other environments. I did a search on the web2py users list
> and found a few threads about a problem using LIKE % values with PostgreSQL
> but it appears to have been resolved. It looks like it might help with
> identifying the problems areas which helps find the low hanging fruit for
> caching. Does anyone else have experience with using this tool? You can use
> it for a 2 week trial, then there is a cost. I am not associated with them
> and have not used the product in any way, just passing on the fact they
> exist.
>
> Ron
>
>
> On Tuesday, 17 April 2012 10:10:08 UTC-7, Bruce Wade wrote:
>>
>> Currently I just had 1 server running apache mod_wsgi using the same
>> configuration as pyramid. However I just got approved for a few grand a
>> month to spend on server resources, so I am looking at load balancers. And
>> I will put nginx in front of apache, and also start using a lot more
>> caching.
>>
>> On Tue, Apr 17, 2012 at 5:15 AM, Michele Comitini <
>> michele.comit...@gmail.com> wrote:
>>
>>> One more thing make css and js packed + server side gzipped (nginx and
>>> cherokee can do also gzip caching)
>>>
>>> mic
>>>
>>> Il 17 aprile 2012 14:12, Michele Comitini 
>>> ha scritto:
>>> > If you are on postgreSQL use a process per request setup, you will
>>> > have a great benefit.  Use cherokee or nginx (with keepalive working)
>>> > you will scale smoothly.
>>> >
>>> > Check that you do as much as possible of a page in a single http
>>> > request (i.e. limit ajax load).  Use only one cacheable css and limit
>>> > the number of scripts or aggregate them in a cacheable file.
>>> > Check that everything that is cacheable gets cached indeed (use
>>> > firebug or chrome dev tools to find out).
>>> >
>>> > mic
>>> >
>>> >
>>> > Il 17 aprile 2012 14:07, Michele Comitini 
>>> > ha scritto:
>>> >> What is your architecture?  What do you use as frontend http server?
>>> >> What protocol: SCGI, UWSGI, FCGI...?
>>> >> Are you in a thread per request or process per request setup?
>>> >>
>>> >> mic
>>> >>
>>> >>
>>> >> Il 17 aprile 2012 08:36, Bruce Wade  ha
>>> scritto:
>>> >>> Yes you are correct plus there was 10,000+ requests a second just
>>> hitting
>>> >>> the site I think I really need a load balanced. We are getting on
>>> average
>>> >>> 500-1000 new members a day.
>>> >>>
>>> >>> On Apr 16, 2012 10:59 PM, "pbreit"  wrote:
>>> 
>>>  Don't forget you probably spent quite a bit if time tuning your
>>> Pyramid
>>>  app.
>>> 
>>>  The best ways to scale are:
>>>  1) Cache
>>>  2) Cache
>>>  3) Cache
>>> 
>>>  Web2py makes caching queries super easy.
>>> 
>>>  If you are serving a lot of static assets, check out Cloudflare for
>>> free
>>>  CDN.
>>>
>>
>>
>>
>> --
>> --
>> Regards,
>> Bruce Wade
>> http://ca.linkedin.com/in/**brucelwade
>> http://www.wadecybertech.com
>> http://www.fittraineronline.**com  -
>> Fitness Personal Trainers Online
>> http://www.warplydesigned.com
>>
>>

Re: [web2py] CPU 210%

[Ron McOuat]

I ran across newrelic.com which offers profiling for Python web apps 
amongst several other environments. I did a search on the web2py users list 
and found a few threads about a problem using LIKE % values with PostgreSQL 
but it appears to have been resolved. It looks like it might help with 
identifying the problems areas which helps find the low hanging fruit for 
caching. Does anyone else have experience with using this tool? You can use 
it for a 2 week trial, then there is a cost. I am not associated with them 
and have not used the product in any way, just passing on the fact they 
exist.

Ron

On Tuesday, 17 April 2012 10:10:08 UTC-7, Bruce Wade wrote:
>
> Currently I just had 1 server running apache mod_wsgi using the same 
> configuration as pyramid. However I just got approved for a few grand a 
> month to spend on server resources, so I am looking at load balancers. And 
> I will put nginx in front of apache, and also start using a lot more 
> caching. 
>
> On Tue, Apr 17, 2012 at 5:15 AM, Michele Comitini <
> michele.comit...@gmail.com> wrote:
>
>> One more thing make css and js packed + server side gzipped (nginx and
>> cherokee can do also gzip caching)
>>
>> mic
>>
>> Il 17 aprile 2012 14:12, Michele Comitini 
>> ha scritto:
>> > If you are on postgreSQL use a process per request setup, you will
>> > have a great benefit.  Use cherokee or nginx (with keepalive working)
>> > you will scale smoothly.
>> >
>> > Check that you do as much as possible of a page in a single http
>> > request (i.e. limit ajax load).  Use only one cacheable css and limit
>> > the number of scripts or aggregate them in a cacheable file.
>> > Check that everything that is cacheable gets cached indeed (use
>> > firebug or chrome dev tools to find out).
>> >
>> > mic
>> >
>> >
>> > Il 17 aprile 2012 14:07, Michele Comitini 
>> > ha scritto:
>> >> What is your architecture?  What do you use as frontend http server?
>> >> What protocol: SCGI, UWSGI, FCGI...?
>> >> Are you in a thread per request or process per request setup?
>> >>
>> >> mic
>> >>
>> >>
>> >> Il 17 aprile 2012 08:36, Bruce Wade  ha scritto:
>> >>> Yes you are correct plus there was 10,000+ requests a second just 
>> hitting
>> >>> the site I think I really need a load balanced. We are getting on 
>> average
>> >>> 500-1000 new members a day.
>> >>>
>> >>> On Apr 16, 2012 10:59 PM, "pbreit"  wrote:
>> 
>>  Don't forget you probably spent quite a bit if time tuning your 
>> Pyramid
>>  app.
>> 
>>  The best ways to scale are:
>>  1) Cache
>>  2) Cache
>>  3) Cache
>> 
>>  Web2py makes caching queries super easy.
>> 
>>  If you are serving a lot of static assets, check out Cloudflare for 
>> free
>>  CDN.
>>
>
>
>
> -- 
> -- 
> Regards,
> Bruce Wade
> http://ca.linkedin.com/in/brucelwade
> http://www.wadecybertech.com
> http://www.fittraineronline.com - Fitness Personal Trainers Online
> http://www.warplydesigned.com
>
>

Re: [web2py] CPU 210%

[Richard Vézina]

Michele,

I read this thread, not sure if it is not already existing... But it start
to look like a recipe for speed tuning that could be translate into a
web2py slice or blog post :)

Richard

On Tue, Apr 17, 2012 at 2:32 PM, Michele Comitini <
michele.comit...@gmail.com> wrote:

> What I suggest is use nginx + uwsgi or nginx + scgi or nginx + fcgi.
> You will notice big reduction in resource consuption compared to
> Apache.
> Apache can be taken away.  By the way you can use any of the above
> protocols to do balancing over a pool of machines running web2py and
> in front you can put nginx as balancer.
>
> About caching, what is important is using as much as possible the
> *static* dir and have the files underneath served by nginx directly.
> You must check that expiration times on static objects are correctly
> set.  Also use sprites as much as you can.  You will enjoy a big
> improvement because after downloading the objects the first time a
> client browser will make only requests for the dynamically generated
> contents.  Other objects will be taken by the ondisk cache of the
> browser. If  the object is expired  in browser cache, but the content
> not changed on the server, the request will result only in a 304
> answer so little data is exchanged and little computation is required.
>
> mic
>
>
> Il 17 aprile 2012 19:10, Bruce Wade  ha scritto:
> > Currently I just had 1 server running apache mod_wsgi using the same
> > configuration as pyramid. However I just got approved for a few grand a
> > month to spend on server resources, so I am looking at load balancers.
> And I
> > will put nginx in front of apache, and also start using a lot more
> caching.
> >
> >
> > On Tue, Apr 17, 2012 at 5:15 AM, Michele Comitini
> >  wrote:
> >>
> >> One more thing make css and js packed + server side gzipped (nginx and
> >> cherokee can do also gzip caching)
> >>
> >> mic
> >>
> >> Il 17 aprile 2012 14:12, Michele Comitini 
> >> ha scritto:
> >> > If you are on postgreSQL use a process per request setup, you will
> >> > have a great benefit.  Use cherokee or nginx (with keepalive working)
> >> > you will scale smoothly.
> >> >
> >> > Check that you do as much as possible of a page in a single http
> >> > request (i.e. limit ajax load).  Use only one cacheable css and limit
> >> > the number of scripts or aggregate them in a cacheable file.
> >> > Check that everything that is cacheable gets cached indeed (use
> >> > firebug or chrome dev tools to find out).
> >> >
> >> > mic
> >> >
> >> >
> >> > Il 17 aprile 2012 14:07, Michele Comitini  >
> >> > ha scritto:
> >> >> What is your architecture?  What do you use as frontend http server?
> >> >> What protocol: SCGI, UWSGI, FCGI...?
> >> >> Are you in a thread per request or process per request setup?
> >> >>
> >> >> mic
> >> >>
> >> >>
> >> >> Il 17 aprile 2012 08:36, Bruce Wade  ha
> scritto:
> >> >>> Yes you are correct plus there was 10,000+ requests a second just
> >> >>> hitting
> >> >>> the site I think I really need a load balanced. We are getting on
> >> >>> average
> >> >>> 500-1000 new members a day.
> >> >>>
> >> >>> On Apr 16, 2012 10:59 PM, "pbreit"  wrote:
> >> 
> >>  Don't forget you probably spent quite a bit if time tuning your
> >>  Pyramid
> >>  app.
> >> 
> >>  The best ways to scale are:
> >>  1) Cache
> >>  2) Cache
> >>  3) Cache
> >> 
> >>  Web2py makes caching queries super easy.
> >> 
> >>  If you are serving a lot of static assets, check out Cloudflare for
> >>  free
> >>  CDN.
> >
> >
> >
> >
> > --
> > --
> > Regards,
> > Bruce Wade
> > http://ca.linkedin.com/in/brucelwade
> > http://www.wadecybertech.com
> > http://www.fittraineronline.com - Fitness Personal Trainers Online
> > http://www.warplydesigned.com
> >
>

Re: [web2py] CPU 210%

[Michele Comitini]

What I suggest is use nginx + uwsgi or nginx + scgi or nginx + fcgi.
You will notice big reduction in resource consuption compared to
Apache.
Apache can be taken away.  By the way you can use any of the above
protocols to do balancing over a pool of machines running web2py and
in front you can put nginx as balancer.

About caching, what is important is using as much as possible the
*static* dir and have the files underneath served by nginx directly.
You must check that expiration times on static objects are correctly
set.  Also use sprites as much as you can.  You will enjoy a big
improvement because after downloading the objects the first time a
client browser will make only requests for the dynamically generated
contents.  Other objects will be taken by the ondisk cache of the
browser. If  the object is expired  in browser cache, but the content
not changed on the server, the request will result only in a 304
answer so little data is exchanged and little computation is required.

mic


Il 17 aprile 2012 19:10, Bruce Wade  ha scritto:
> Currently I just had 1 server running apache mod_wsgi using the same
> configuration as pyramid. However I just got approved for a few grand a
> month to spend on server resources, so I am looking at load balancers. And I
> will put nginx in front of apache, and also start using a lot more caching.
>
>
> On Tue, Apr 17, 2012 at 5:15 AM, Michele Comitini
>  wrote:
>>
>> One more thing make css and js packed + server side gzipped (nginx and
>> cherokee can do also gzip caching)
>>
>> mic
>>
>> Il 17 aprile 2012 14:12, Michele Comitini 
>> ha scritto:
>> > If you are on postgreSQL use a process per request setup, you will
>> > have a great benefit.  Use cherokee or nginx (with keepalive working)
>> > you will scale smoothly.
>> >
>> > Check that you do as much as possible of a page in a single http
>> > request (i.e. limit ajax load).  Use only one cacheable css and limit
>> > the number of scripts or aggregate them in a cacheable file.
>> > Check that everything that is cacheable gets cached indeed (use
>> > firebug or chrome dev tools to find out).
>> >
>> > mic
>> >
>> >
>> > Il 17 aprile 2012 14:07, Michele Comitini 
>> > ha scritto:
>> >> What is your architecture?  What do you use as frontend http server?
>> >> What protocol: SCGI, UWSGI, FCGI...?
>> >> Are you in a thread per request or process per request setup?
>> >>
>> >> mic
>> >>
>> >>
>> >> Il 17 aprile 2012 08:36, Bruce Wade  ha scritto:
>> >>> Yes you are correct plus there was 10,000+ requests a second just
>> >>> hitting
>> >>> the site I think I really need a load balanced. We are getting on
>> >>> average
>> >>> 500-1000 new members a day.
>> >>>
>> >>> On Apr 16, 2012 10:59 PM, "pbreit"  wrote:
>> 
>>  Don't forget you probably spent quite a bit if time tuning your
>>  Pyramid
>>  app.
>> 
>>  The best ways to scale are:
>>  1) Cache
>>  2) Cache
>>  3) Cache
>> 
>>  Web2py makes caching queries super easy.
>> 
>>  If you are serving a lot of static assets, check out Cloudflare for
>>  free
>>  CDN.
>
>
>
>
> --
> --
> Regards,
> Bruce Wade
> http://ca.linkedin.com/in/brucelwade
> http://www.wadecybertech.com
> http://www.fittraineronline.com - Fitness Personal Trainers Online
> http://www.warplydesigned.com
>

Re: [web2py] CPU 210%

[Richard Vézina]

Notice, that caching query not improve speed that much, at least it didn't
help very much. But if you use to build dict with query for iteration
caching the whole dict creation variable, that help a lot.

The syntax for caching variable is the following :


dblabels_en = cache.ram('dblabels_en',
lambda: dict([(r.table_name + r.column_name, r.column_name_en_ui)
 for r in db().select(db.dict_database.ALL)]),
time_expire=3600)


>From Anthony (somewhere in this thread) :
https://groups.google.com/forum/#!topic/web2py/IfbZWFZv-5s


Richard


On Tue, Apr 17, 2012 at 1:10 PM, Bruce Wade  wrote:

> Currently I just had 1 server running apache mod_wsgi using the same
> configuration as pyramid. However I just got approved for a few grand a
> month to spend on server resources, so I am looking at load balancers. And
> I will put nginx in front of apache, and also start using a lot more
> caching.
>
>
> On Tue, Apr 17, 2012 at 5:15 AM, Michele Comitini <
> michele.comit...@gmail.com> wrote:
>
>> One more thing make css and js packed + server side gzipped (nginx and
>> cherokee can do also gzip caching)
>>
>> mic
>>
>> Il 17 aprile 2012 14:12, Michele Comitini 
>> ha scritto:
>> > If you are on postgreSQL use a process per request setup, you will
>> > have a great benefit.  Use cherokee or nginx (with keepalive working)
>> > you will scale smoothly.
>> >
>> > Check that you do as much as possible of a page in a single http
>> > request (i.e. limit ajax load).  Use only one cacheable css and limit
>> > the number of scripts or aggregate them in a cacheable file.
>> > Check that everything that is cacheable gets cached indeed (use
>> > firebug or chrome dev tools to find out).
>> >
>> > mic
>> >
>> >
>> > Il 17 aprile 2012 14:07, Michele Comitini 
>> > ha scritto:
>> >> What is your architecture?  What do you use as frontend http server?
>> >> What protocol: SCGI, UWSGI, FCGI...?
>> >> Are you in a thread per request or process per request setup?
>> >>
>> >> mic
>> >>
>> >>
>> >> Il 17 aprile 2012 08:36, Bruce Wade  ha scritto:
>> >>> Yes you are correct plus there was 10,000+ requests a second just
>> hitting
>> >>> the site I think I really need a load balanced. We are getting on
>> average
>> >>> 500-1000 new members a day.
>> >>>
>> >>> On Apr 16, 2012 10:59 PM, "pbreit"  wrote:
>> 
>>  Don't forget you probably spent quite a bit if time tuning your
>> Pyramid
>>  app.
>> 
>>  The best ways to scale are:
>>  1) Cache
>>  2) Cache
>>  3) Cache
>> 
>>  Web2py makes caching queries super easy.
>> 
>>  If you are serving a lot of static assets, check out Cloudflare for
>> free
>>  CDN.
>>
>
>
>
> --
> --
> Regards,
> Bruce Wade
> http://ca.linkedin.com/in/brucelwade
> http://www.wadecybertech.com
> http://www.fittraineronline.com - Fitness Personal Trainers Online
> http://www.warplydesigned.com
>
>

Re: [web2py] CPU 210%

[Bruce Wade]

Currently I just had 1 server running apache mod_wsgi using the same
configuration as pyramid. However I just got approved for a few grand a
month to spend on server resources, so I am looking at load balancers. And
I will put nginx in front of apache, and also start using a lot more
caching.

On Tue, Apr 17, 2012 at 5:15 AM, Michele Comitini <
michele.comit...@gmail.com> wrote:

> One more thing make css and js packed + server side gzipped (nginx and
> cherokee can do also gzip caching)
>
> mic
>
> Il 17 aprile 2012 14:12, Michele Comitini 
> ha scritto:
> > If you are on postgreSQL use a process per request setup, you will
> > have a great benefit.  Use cherokee or nginx (with keepalive working)
> > you will scale smoothly.
> >
> > Check that you do as much as possible of a page in a single http
> > request (i.e. limit ajax load).  Use only one cacheable css and limit
> > the number of scripts or aggregate them in a cacheable file.
> > Check that everything that is cacheable gets cached indeed (use
> > firebug or chrome dev tools to find out).
> >
> > mic
> >
> >
> > Il 17 aprile 2012 14:07, Michele Comitini 
> > ha scritto:
> >> What is your architecture?  What do you use as frontend http server?
> >> What protocol: SCGI, UWSGI, FCGI...?
> >> Are you in a thread per request or process per request setup?
> >>
> >> mic
> >>
> >>
> >> Il 17 aprile 2012 08:36, Bruce Wade  ha scritto:
> >>> Yes you are correct plus there was 10,000+ requests a second just
> hitting
> >>> the site I think I really need a load balanced. We are getting on
> average
> >>> 500-1000 new members a day.
> >>>
> >>> On Apr 16, 2012 10:59 PM, "pbreit"  wrote:
> 
>  Don't forget you probably spent quite a bit if time tuning your
> Pyramid
>  app.
> 
>  The best ways to scale are:
>  1) Cache
>  2) Cache
>  3) Cache
> 
>  Web2py makes caching queries super easy.
> 
>  If you are serving a lot of static assets, check out Cloudflare for
> free
>  CDN.
>



-- 
-- 
Regards,
Bruce Wade
http://ca.linkedin.com/in/brucelwade
http://www.wadecybertech.com
http://www.fittraineronline.com - Fitness Personal Trainers Online
http://www.warplydesigned.com

Re: [web2py] CPU 210%

[Michele Comitini]

One more thing make css and js packed + server side gzipped (nginx and
cherokee can do also gzip caching)

mic

Il 17 aprile 2012 14:12, Michele Comitini 
ha scritto:
> If you are on postgreSQL use a process per request setup, you will
> have a great benefit.  Use cherokee or nginx (with keepalive working)
> you will scale smoothly.
>
> Check that you do as much as possible of a page in a single http
> request (i.e. limit ajax load).  Use only one cacheable css and limit
> the number of scripts or aggregate them in a cacheable file.
> Check that everything that is cacheable gets cached indeed (use
> firebug or chrome dev tools to find out).
>
> mic
>
>
> Il 17 aprile 2012 14:07, Michele Comitini 
> ha scritto:
>> What is your architecture?  What do you use as frontend http server?
>> What protocol: SCGI, UWSGI, FCGI...?
>> Are you in a thread per request or process per request setup?
>>
>> mic
>>
>>
>> Il 17 aprile 2012 08:36, Bruce Wade  ha scritto:
>>> Yes you are correct plus there was 10,000+ requests a second just hitting
>>> the site I think I really need a load balanced. We are getting on average
>>> 500-1000 new members a day.
>>>
>>> On Apr 16, 2012 10:59 PM, "pbreit"  wrote:

 Don't forget you probably spent quite a bit if time tuning your Pyramid
 app.

 The best ways to scale are:
 1) Cache
 2) Cache
 3) Cache

 Web2py makes caching queries super easy.

 If you are serving a lot of static assets, check out Cloudflare for free
 CDN.

Re: [web2py] CPU 210%

[Michele Comitini]

If you are on postgreSQL use a process per request setup, you will
have a great benefit.  Use cherokee or nginx (with keepalive working)
you will scale smoothly.

Check that you do as much as possible of a page in a single http
request (i.e. limit ajax load).  Use only one cacheable css and limit
the number of scripts or aggregate them in a cacheable file.
Check that everything that is cacheable gets cached indeed (use
firebug or chrome dev tools to find out).

mic


Il 17 aprile 2012 14:07, Michele Comitini 
ha scritto:
> What is your architecture?  What do you use as frontend http server?
> What protocol: SCGI, UWSGI, FCGI...?
> Are you in a thread per request or process per request setup?
>
> mic
>
>
> Il 17 aprile 2012 08:36, Bruce Wade  ha scritto:
>> Yes you are correct plus there was 10,000+ requests a second just hitting
>> the site I think I really need a load balanced. We are getting on average
>> 500-1000 new members a day.
>>
>> On Apr 16, 2012 10:59 PM, "pbreit"  wrote:
>>>
>>> Don't forget you probably spent quite a bit if time tuning your Pyramid
>>> app.
>>>
>>> The best ways to scale are:
>>> 1) Cache
>>> 2) Cache
>>> 3) Cache
>>>
>>> Web2py makes caching queries super easy.
>>>
>>> If you are serving a lot of static assets, check out Cloudflare for free
>>> CDN.

Re: [web2py] CPU 210%

[Michele Comitini]

What is your architecture?  What do you use as frontend http server?
What protocol: SCGI, UWSGI, FCGI...?
Are you in a thread per request or process per request setup?

mic


Il 17 aprile 2012 08:36, Bruce Wade  ha scritto:
> Yes you are correct plus there was 10,000+ requests a second just hitting
> the site I think I really need a load balanced. We are getting on average
> 500-1000 new members a day.
>
> On Apr 16, 2012 10:59 PM, "pbreit"  wrote:
>>
>> Don't forget you probably spent quite a bit if time tuning your Pyramid
>> app.
>>
>> The best ways to scale are:
>> 1) Cache
>> 2) Cache
>> 3) Cache
>>
>> Web2py makes caching queries super easy.
>>
>> If you are serving a lot of static assets, check out Cloudflare for free
>> CDN.

Re: [web2py] CPU 210%

[Bruce Wade]

Yes you are correct plus there was 10,000+ requests a second just hitting
the site I think I really need a load balanced. We are getting on average
500-1000 new members a day.
On Apr 16, 2012 10:59 PM, "pbreit"  wrote:

> Don't forget you probably spent quite a bit if time tuning your Pyramid
> app.
>
> The best ways to scale are:
> 1) Cache
> 2) Cache
> 3) Cache
>
> Web2py makes caching queries super easy.
>
> If you are serving a lot of static assets, check out Cloudflare for free
> CDN.
>

Re: [web2py] CPU 210%

[pbreit]

Don't forget you probably spent quite a bit if time tuning your Pyramid app.

The best ways to scale are:
1) Cache
2) Cache
3) Cache

Web2py makes caching queries super easy.

If you are serving a lot of static assets, check out Cloudflare for free 
CDN.

Re: [web2py] CPU 210%

[Bruce Wade]

Yeah the site can't even handle half the load the pyramid version handled.
I really hope I can fix this. Just told some of the members to use it but
it buckles it doesn't use much memory but kills the processor.

The load bandwidth out going went from 0.2MB/sec to 1.6MB/s

On Sun, Apr 15, 2012 at 10:33 AM, Bruce Wade  wrote:

> I think it was the server. I rebuilt/installed from scratch OS etc.. the
> server last night and now it seems to be running without problems. I am
> slowing increasing the load to see how well it handles before moving it to
> the new site again.
>
>
> On Sat, Apr 14, 2012 at 8:13 PM, Bruno Rocha wrote:
>
>> We have to see significant part of the code too figure out what is going
>> on...
>>
>> http://zerp.ly/rochacbruno
>> Em 14/04/2012 22:45, "Bruce Wade"  escreveu:
>>
>> Hi,
>>>
>>> So I just launched and have been hit with a tone of traffic and my CPU
>>> on linode is showing 210% usage. Any suggestions?
>>>
>>> --
>>> --
>>> Regards,
>>> Bruce Wade
>>> http://ca.linkedin.com/in/brucelwade
>>> http://www.wadecybertech.com
>>> http://www.fittraineronline.com - Fitness Personal Trainers Online
>>> http://www.warplydesigned.com
>>>
>>>
>
>
> --
> --
> Regards,
> Bruce Wade
> http://ca.linkedin.com/in/brucelwade
> http://www.wadecybertech.com
> http://www.fittraineronline.com - Fitness Personal Trainers Online
> http://www.warplydesigned.com
>
>


-- 
-- 
Regards,
Bruce Wade
http://ca.linkedin.com/in/brucelwade
http://www.wadecybertech.com
http://www.fittraineronline.com - Fitness Personal Trainers Online
http://www.warplydesigned.com

Re: [web2py] CPU 210%

[Bruce Wade]

I think it was the server. I rebuilt/installed from scratch OS etc.. the
server last night and now it seems to be running without problems. I am
slowing increasing the load to see how well it handles before moving it to
the new site again.

On Sat, Apr 14, 2012 at 8:13 PM, Bruno Rocha  wrote:

> We have to see significant part of the code too figure out what is going
> on...
>
> http://zerp.ly/rochacbruno
> Em 14/04/2012 22:45, "Bruce Wade"  escreveu:
>
> Hi,
>>
>> So I just launched and have been hit with a tone of traffic and my CPU on
>> linode is showing 210% usage. Any suggestions?
>>
>> --
>> --
>> Regards,
>> Bruce Wade
>> http://ca.linkedin.com/in/brucelwade
>> http://www.wadecybertech.com
>> http://www.fittraineronline.com - Fitness Personal Trainers Online
>> http://www.warplydesigned.com
>>
>>


-- 
-- 
Regards,
Bruce Wade
http://ca.linkedin.com/in/brucelwade
http://www.wadecybertech.com
http://www.fittraineronline.com - Fitness Personal Trainers Online
http://www.warplydesigned.com

Re: [web2py] CPU 210%

[Bruno Rocha]

We have to see significant part of the code too figure out what is going
on...

http://zerp.ly/rochacbruno
Em 14/04/2012 22:45, "Bruce Wade"  escreveu:

> Hi,
>
> So I just launched and have been hit with a tone of traffic and my CPU on
> linode is showing 210% usage. Any suggestions?
>
> --
> --
> Regards,
> Bruce Wade
> http://ca.linkedin.com/in/brucelwade
> http://www.wadecybertech.com
> http://www.fittraineronline.com - Fitness Personal Trainers Online
> http://www.warplydesigned.com
>
>

[web2py] CPU 210%

[Bruce Wade]

Hi,

So I just launched and have been hit with a tone of traffic and my CPU on
linode is showing 210% usage. Any suggestions?

-- 
-- 
Regards,
Bruce Wade
http://ca.linkedin.com/in/brucelwade
http://www.wadecybertech.com
http://www.fittraineronline.com - Fitness Personal Trainers Online
http://www.warplydesigned.com