Re: [AOLSERVER] AOLserver questions

[Sep Ng]

This is an intriguing solution.  If it were possible to configure ns_pools 
to do that.  I'm not too familiar with how this can be accomplished however.

Regards.

On Friday, March 20, 2015 at 8:54:10 PM UTC+8, Jeff Rogers wrote:
>
> Sep Ng wrote: 
> > Thank you very much for shedding a lot of light into this. 
> > 
> > On Friday, March 20, 2015 at 3:58:19 PM UTC+8, Gustaf Neumann wrote: 
> > 
> >  > Am 20.03.15 um 07:48 schrieb Sep Ng: 
> > 
> > what is hurting you? 
> > 
> >  > We have instances where we'd get a high number of concurrent 
> > users that the requests are getting queued, but when I look at the 
> > logs, there's a > lot of static files being served for each login 
> > page, let alone other pages being served in aolserver.  So, I'm 
> > theorizing that being able to get those > static file requests 
> > pushed into a single thread and free up the connection threads would 
> > help in scalability. 
> > 
> > yes, there is a certain hope, that removing this burden from the 
> > connection threads will improve the situation. Another option to 
> reduce 
> > queuing time is to increase the number of connection threads. 
> > If the bottleneck are slow sql-queries then this pooling stuff will 
> > not help. 
> > 
> > Right now, I do not believe sql queries are the culprit for the 
> > sacalability issues.  I have a better understanding on this now.  I 
> > think the only real issue from implementation stand point is getting the 
> > reverse proxy setup right. 
>
>
> Another thing to try out if you can distinguish static from dynamic by 
> the url pattern is to use [ns_pools] to set up the server so that all 
> static content is served from one threadpool and the slow dynamic pages 
> from a different pool.  These would still be ordinary conn threads (no 
> background delivery), but it would keep the static requests from one 
> user from queuing behind the dynamic pages from a different user.  I 
> haven't completely thought through exactly how this would work, but my 
> first impression is that it would mean longer queuing times for dynamic 
> requests, but more responsive servicing of static ones, so that pages 
> would be slower to start but faster once they started. 
>
> -J 
>
>
>
> --
>  
>
> Dive into the World of Parallel Programming The Go Parallel Website, 
> sponsored 
> by Intel and developed in partnership with Slashdot Media, is your hub for 
> all 
> things parallel software development, from weekly thought leadership blogs 
> to 
> news, videos, case studies, tutorials and more. Take a look and join the 
> conversation now. http://goparallel.sourceforge.net/ 
> ___ 
> aolserver-talk mailing list 
> aolserv...@lists.sourceforge.net  
> https://lists.sourceforge.net/lists/listinfo/aolserver-talk 
>
--
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the 
conversation now. http://goparallel.sourceforge.net/___
aolserver-talk mailing list
aolserver-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/aolserver-talk

Re: [AOLSERVER] AOLserver questions

[Jeff Rogers]

Sep Ng wrote:
> Thank you very much for shedding a lot of light into this.
>
> On Friday, March 20, 2015 at 3:58:19 PM UTC+8, Gustaf Neumann wrote:
>
>  > Am 20.03.15 um 07:48 schrieb Sep Ng:
>
> what is hurting you?
>
>  > We have instances where we'd get a high number of concurrent
> users that the requests are getting queued, but when I look at the
> logs, there's a > lot of static files being served for each login
> page, let alone other pages being served in aolserver.  So, I'm
> theorizing that being able to get those > static file requests
> pushed into a single thread and free up the connection threads would
> help in scalability.
>
> yes, there is a certain hope, that removing this burden from the
> connection threads will improve the situation. Another option to reduce
> queuing time is to increase the number of connection threads.
> If the bottleneck are slow sql-queries then this pooling stuff will
> not help.
>
> Right now, I do not believe sql queries are the culprit for the
> sacalability issues.  I have a better understanding on this now.  I
> think the only real issue from implementation stand point is getting the
> reverse proxy setup right.


Another thing to try out if you can distinguish static from dynamic by 
the url pattern is to use [ns_pools] to set up the server so that all 
static content is served from one threadpool and the slow dynamic pages 
from a different pool.  These would still be ordinary conn threads (no 
background delivery), but it would keep the static requests from one 
user from queuing behind the dynamic pages from a different user.  I 
haven't completely thought through exactly how this would work, but my 
first impression is that it would mean longer queuing times for dynamic 
requests, but more responsive servicing of static ones, so that pages 
would be slower to start but faster once they started.

-J



--
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the 
conversation now. http://goparallel.sourceforge.net/
___
aolserver-talk mailing list
aolserver-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/aolserver-talk

Re: [AOLSERVER] AOLserver questions

[Sep Ng]

Thank you very much for shedding a lot of light into this.

On Friday, March 20, 2015 at 3:58:19 PM UTC+8, Gustaf Neumann wrote:
>
>  > Am 20.03.15 um 07:48 schrieb Sep Ng:
>
>>  what is hurting you?
>>  
>  
> > We have instances where we'd get a high number of concurrent users that 
> the requests are getting queued, but when I look at the logs, there's a > 
> lot of static files being served for each login page, let alone other pages 
> being served in aolserver.  So, I'm theorizing that being able to get those 
> > static file requests pushed into a single thread and free up the 
> connection threads would help in scalability.
>
> yes, there is a certain hope, that removing this burden from the 
> connection threads will improve the situation. Another option to reduce 
> queuing time is to increase the number of connection threads. 
> If the bottleneck are slow sql-queries then this pooling stuff will not 
> help.
>
> Right now, I do not believe sql queries are the culprit for the 
sacalability issues.  I have a better understanding on this now.  I think 
the only real issue from implementation stand point is getting the reverse 
proxy setup right.
 

> Often the first task to determine, what the bottleneck is, can be already 
> be difficult.
> NaviServer has several introspection means for monitoring. The following  
> graph shows queuing times, filter and run times (you won't get
> these numbers from aolserver). The graph (from OpenACS.org) shows
> that queuing time is on that site typically around 0.1 ms, with peaks in 
> the range of 16 ms. This is for example quite useful for determining the 
> right number of running connection threads. naviserver allows to 
> change this number dynamically without restart
>
> [image: weekly graph]
>   
>
This chart is something that would benefit us very much.  NaviServer is 
looking like a target I should be working towards in the future.
 

>
>  > By the way, I've seen in previous posts of yours that the you did 
> switch from aolserver to naviserver.  How big was the change?  What things 
> did > you have to re-write/port to get them running in naviserver?
>
> We did the move of our main site 4 years ago (now we have around 50 
> naviserver sites),
> but i do not have a detailed writeup of the changes. Most of our changes 
> went into OpenACS (download OpenACS 5.8.1, search for NaviServer). 
>
> what comes to my mind is:
> - NaviServer dropped the useless "$conn" argument from several commands
>   (like old: "ns_return $conn 200 text/plain ..." -> "ns_return 200 
> text/plain ..."
> - different modules (e.g. for ssl), different config file
> - more functionality built-in which was as a module under aolserver
>   crypo functions (sha, md5), cache, base-64 encoding, gzip delivery
>   (actually, the "ns_cache" function in naviserver usues a single 
>   command style (ns_cache_eval) and in aolserver subcommand style, 
>   but we added already a compatibility layer to the naviserver source tree
>   which is sufficient for OpenACS
> - no ns_share (use nsv instead)
> - no "ns_set -persistent"
>
> We did not use the latter two, but this comes sometimes up in the mailing 
> lists.
> The move was quite easy for us, but ymmv.
>  
> There seems to be much work to be done and this can't be rolled out 
quickly.  I will have to spend more time on this when the time comes.

 

> -g
>  
--
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the 
conversation now. http://goparallel.sourceforge.net/___
aolserver-talk mailing list
aolserver-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/aolserver-talk

Re: [AOLSERVER] AOLserver questions

[Gustaf Neumann]

> Am 20.03.15 um 07:48 schrieb Sep Ng:

   what is hurting you?

> We have instances where we'd get a high number of concurrent users 
that the requests are getting queued, but when I look at the logs, 
there's a > lot of static files being served for each login page, let 
alone other pages being served in aolserver.  So, I'm theorizing that 
being able to get those > static file requests pushed into a single 
thread and free up the connection threads would help in scalability.


yes, there is a certain hope, that removing this burden from the
connection threads will improve the situation. Another option to reduce
queuing time is to increase the number of connection threads.
If the bottleneck are slow sql-queries then this pooling stuff will not 
help.


Often the first task to determine, what the bottleneck is, can be 
already be difficult.

NaviServer has several introspection means for monitoring. The following
graph shows queuing times, filter and run times (you won't get
these numbers from aolserver). The graph (from OpenACS.org) shows
that queuing time is on that site typically around 0.1 ms, with peaks in
the range of 16 ms. This is for example quite useful for determining the
right number of running connection threads. naviserver allows to
change this number dynamically without restart

weekly graph

> By the way, I've seen in previous posts of yours that the you did 
switch from aolserver to naviserver.  How big was the change?  What 
things did > you have to re-write/port to get them running in naviserver?


We did the move of our main site 4 years ago (now we have around 50 
naviserver sites),

but i do not have a detailed writeup of the changes. Most of our changes
went into OpenACS (download OpenACS 5.8.1, search for NaviServer).

what comes to my mind is:
- NaviServer dropped the useless "$conn" argument from several commands
  (like old: "ns_return $conn 200 text/plain ..." -> "ns_return 200 
text/plain ..."

- different modules (e.g. for ssl), different config file
- more functionality built-in which was as a module under aolserver
  crypo functions (sha, md5), cache, base-64 encoding, gzip delivery
  (actually, the "ns_cache" function in naviserver usues a single
  command style (ns_cache_eval) and in aolserver subcommand style,
  but we added already a compatibility layer to the naviserver source tree
  which is sufficient for OpenACS
- no ns_share (use nsv instead)
- no "ns_set -persistent"

We did not use the latter two, but this comes sometimes up in the 
mailing lists.

The move was quite easy for us, but ymmv.

-g
--
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the 
conversation now. http://goparallel.sourceforge.net/___
aolserver-talk mailing list
aolserver-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/aolserver-talk

Re: [AOLSERVER] AOLserver questions

[Sep Ng]

Thank you once again for your swift response!

On Friday, March 20, 2015 at 2:33:59 PM UTC+8, Gustaf Neumann wrote:
>
>  Am 20.03.15 um 05:47 schrieb Sep Ng:
>  
> Hi Gustaf!  Thank you for the informative response! 
>
>  I've been thinking of moving to NaviServer but I don't know enough about 
> the transition to make that call yet.  Right now, we're on aolserver and 
> so, I'm trying to see what I can do on this platform.  I do not understand 
> why the delivery doesn't work on https out of the box and requires a 
> reverse proxy. 
>  
> bgdelivery takes the socket (file descriptor) of the current connection, 
> but it has no knowledge about SSL. When it hands the file descriptor to the 
> background delivery thread, this can write back to the client just using 
> plain tcl i/o. So, background delivery can certainly write to the 
> file-descriptor, but that won't be accepted by the client trying to decrypt 
> the channel.
>
 
I think that is now making more sense now.  Thanks.  I may have to look 
into this  as well.

>  
>  I suspect the varied client connection is part of the problem and them 
> sitting on the connection threads is hurting us. 
>  
> what is hurting you?
>
 
We have instances where we'd get a high number of concurrent users that the 
requests are getting queued, but when I look at the logs, there's a lot of 
static files being served for each login page, let alone other pages being 
served in aolserver.  So, I'm theorizing that being able to get those 
static file requests pushed into a single thread and free up the connection 
threads would help in scalability.
 

>   However, we do not serve big files on our server so this has me 
> wondering about the benefits of this change.
>  
> whatever big means. connections can "hang" also when writing a few KBs.
>
Interesting. 

>  
>  I'm not certain if aolserver has any facilities for asynchronous file 
> writing and spooling. 
>  
> the writer threads are an extension of naviserver over aolserver
>
>   It seems that I will have to build everything by hand.  I had hoped 
> that simply transferring the thread and having it ns_returnfile would be 
> enough to get a simple form of background delivery going but it doesn't 
> look like that's the case.
>  
> if your site requires https, one cant use bgdelivery without a reverse 
> proxy. 
> otherwise, everything is pre-packaged.
>
> Oh.  We don't use OpenACS as everything here is custom built by me and 
others before me.  So, it's looking like I'm going to have to roll up my 
sleeves and get to work.

By the way, I've seen in previous posts of yours that the you did switch 
from aolserver to naviserver.  How big was the change?  What things did you 
have to re-write/port to get them running in naviserver?

Regards.

 

> -g
>
>  
>  Regards.
>
>
> On Friday, March 20, 2015 at 12:03:52 PM UTC+8, Gustaf Neumann wrote: 
>>
>> Dear Sep, 
>>
>> The question whether it is worth to use asynchronous delivery boils 
>> down to a question of usage pattern and desired scalability. 
>> The general problem with serving (large) resources via 
>> classical aolserver is that a connection thread is unable 
>> to handle other threads for the time span of the delivery. 
>> It is important to understand that the time span of the delivery is 
>> mostly 
>> determined by the client. A client with little processing power 
>> connection 
>> over e.g. a mobile phone can block a connection quite a long time. A 
>> special instance of this is the slow-read attack [2], which is 
>> a special denial-of-service attack. 
>>
>> To serve e.g. 60 concurrent files one would require 60 
>> connection threads. Note that this can happen quite soon when 
>> serving content with several included  resources (images, css, js) 
>> the first time to a client. When the server runs out of connection 
>> threads, the requests are queued, which means that the 
>> the user-perceived runtime of a request is actually queueing 
>> time plus execution time. 
>>
>> Background delivery (as described in [2]) is fully integrated in OpenACS 
>> addresses the problem by delegating output spooling (file deliveries) 
>> to a single thread, which can deliver easily several 100 concurrent 
>> downloads by using Tcl's asynchronous I/O operations. Note that 
>> this works not only for static resources, but as well dynamic 
>> requests (e.g. generating long HTML pages from e.g. a database). 
>> We used this approach with very good success since 2006 
>> in large OpenACS installations (with e.g. 2000 simultaneous 
>> active users; "simultaneous active" means here users who 
>> requested pages within a time interval of 5 secs). 
>>
>> In OpenACS, one can use simply ad_returnfile_background [3] 
>> instead of ad_returnfile to make use of background delivery. 
>>
>> The limitations of background delivery are that (a) it just works for 
>> plain http, and (b) that it works for at most 1024 concurrently open 
>> file handles. We addressed (a) by using a reverse prox

Re: [AOLSERVER] AOLserver questions

[Gustaf Neumann]

Am 20.03.15 um 05:47 schrieb Sep Ng:

Hi Gustaf!  Thank you for the informative response!

I've been thinking of moving to NaviServer but I don't know enough 
about the transition to make that call yet.  Right now, we're on 
aolserver and so, I'm trying to see what I can do on this platform.  I 
do not understand why the delivery doesn't work on https out of the 
box and requires a reverse proxy.
bgdelivery takes the socket (file descriptor) of the current connection, 
but it has no knowledge about SSL. When it hands the file descriptor to 
the background delivery thread, this can write back to the client just 
using plain tcl i/o. So, background delivery can certainly write to the 
file-descriptor, but that won't be accepted by the client trying to 
decrypt the channel.


I suspect the varied client connection is part of the problem and them 
sitting on the connection threads is hurting us.

what is hurting you?
 However, we do not serve big files on our server so this has me 
wondering about the benefits of this change.

whatever big means. connections can "hang" also when writing a few KBs.


I'm not certain if aolserver has any facilities for asynchronous file 
writing and spooling.

the writer threads are an extension of naviserver over aolserver
 It seems that I will have to build everything by hand.  I had hoped 
that simply transferring the thread and having it ns_returnfile would 
be enough to get a simple form of background delivery going but it 
doesn't look like that's the case.
if your site requires https, one cant use bgdelivery without a reverse 
proxy.

otherwise, everything is pre-packaged.

-g


Regards.


On Friday, March 20, 2015 at 12:03:52 PM UTC+8, Gustaf Neumann wrote:

Dear Sep,

The question whether it is worth to use asynchronous delivery boils
down to a question of usage pattern and desired scalability.
The general problem with serving (large) resources via
classical aolserver is that a connection thread is unable
to handle other threads for the time span of the delivery.
It is important to understand that the time span of the delivery
is mostly
determined by the client. A client with little processing power
connection
over e.g. a mobile phone can block a connection quite a long time. A
special instance of this is the slow-read attack [2], which is
a special denial-of-service attack.

To serve e.g. 60 concurrent files one would require 60
connection threads. Note that this can happen quite soon when
serving content with several included  resources (images, css, js)
the first time to a client. When the server runs out of connection
threads, the requests are queued, which means that the
the user-perceived runtime of a request is actually queueing
time plus execution time.

Background delivery (as described in [2]) is fully integrated in
OpenACS
addresses the problem by delegating output spooling (file deliveries)
to a single thread, which can deliver easily several 100 concurrent
downloads by using Tcl's asynchronous I/O operations. Note that
this works not only for static resources, but as well dynamic
requests (e.g. generating long HTML pages from e.g. a database).
We used this approach with very good success since 2006
in large OpenACS installations (with e.g. 2000 simultaneous
active users; "simultaneous active" means here users who
requested pages within a time interval of 5 secs).

In OpenACS, one can use simply ad_returnfile_background [3]
instead of ad_returnfile to make use of background delivery.

The limitations of background delivery are that (a) it just works for
plain http, and (b) that it works for at most 1024 concurrently open
file handles. We addressed (a) by using a reverse proxy in front
of the server, which delivers the files from the backend via https.
The limitation (b) is harder, since it depends on Tcl's usage of the
select()
system call, which allows to wait for events for max. 1024 file
descriptors. Above this limit, it simply crashes. Lifting this limit
in systems like Linux is possible, but requires a privately compiled
libc and linux kernel. You might think, 1024 this is much more
one needs, but we were actually running close to this limit for
lecture casting (video streaming of university lectures).

A better approach is to use NaviServer.'s c-level support.
NaviServer provides lightweight c-implemented
writer-threads using asynchronous  I/O similar to
bg-delivery, but not using select(). The writer threads
works seemless with http and https. As with bgdelivery, a single
writer thread can serve a multitude of concurrent deliveries.
When several writer threads are defined, the load is split up
between these. NaviServer can also serve streaming
HTML (multiple ns_write commands) via writer threads.
It also support static and dynamic gzi

Re: [AOLSERVER] AOLserver questions

[Sep Ng]

Hi Gustaf!  Thank you for the informative response!

I've been thinking of moving to NaviServer but I don't know enough about 
the transition to make that call yet.  Right now, we're on aolserver and 
so, I'm trying to see what I can do on this platform.  I do not understand 
why the delivery doesn't work on https out of the box and requires a 
reverse proxy. 

I suspect the varied client connection is part of the problem and them 
sitting on the connection threads is hurting us.  However, we do not serve 
big files on our server so this has me wondering about the benefits of this 
change.

I'm not certain if aolserver has any facilities for asynchronous file 
writing and spooling.  It seems that I will have to build everything by 
hand.  I had hoped that simply transferring the thread and having it 
ns_returnfile would be enough to get a simple form of background delivery 
going but it doesn't look like that's the case.

Regards.


On Friday, March 20, 2015 at 12:03:52 PM UTC+8, Gustaf Neumann wrote:
>
> Dear Sep, 
>
> The question whether it is worth to use asynchronous delivery boils 
> down to a question of usage pattern and desired scalability. 
> The general problem with serving (large) resources via 
> classical aolserver is that a connection thread is unable 
> to handle other threads for the time span of the delivery. 
> It is important to understand that the time span of the delivery is mostly 
> determined by the client. A client with little processing power connection 
> over e.g. a mobile phone can block a connection quite a long time. A 
> special instance of this is the slow-read attack [2], which is 
> a special denial-of-service attack. 
>
> To serve e.g. 60 concurrent files one would require 60 
> connection threads. Note that this can happen quite soon when 
> serving content with several included  resources (images, css, js) 
> the first time to a client. When the server runs out of connection 
> threads, the requests are queued, which means that the 
> the user-perceived runtime of a request is actually queueing 
> time plus execution time. 
>
> Background delivery (as described in [2]) is fully integrated in OpenACS 
> addresses the problem by delegating output spooling (file deliveries) 
> to a single thread, which can deliver easily several 100 concurrent 
> downloads by using Tcl's asynchronous I/O operations. Note that 
> this works not only for static resources, but as well dynamic 
> requests (e.g. generating long HTML pages from e.g. a database). 
> We used this approach with very good success since 2006 
> in large OpenACS installations (with e.g. 2000 simultaneous 
> active users; "simultaneous active" means here users who 
> requested pages within a time interval of 5 secs). 
>
> In OpenACS, one can use simply ad_returnfile_background [3] 
> instead of ad_returnfile to make use of background delivery. 
>
> The limitations of background delivery are that (a) it just works for 
> plain http, and (b) that it works for at most 1024 concurrently open 
> file handles. We addressed (a) by using a reverse proxy in front 
> of the server, which delivers the files from the backend via https. 
> The limitation (b) is harder, since it depends on Tcl's usage of the 
> select() 
> system call, which allows to wait for events for max. 1024 file 
> descriptors. Above this limit, it simply crashes. Lifting this limit 
> in systems like Linux is possible, but requires a privately compiled 
> libc and linux kernel. You might think, 1024 this is much more 
> one needs, but we were actually running close to this limit for 
> lecture casting (video streaming of university lectures). 
>
> A better approach is to use NaviServer.'s c-level support. 
> NaviServer provides lightweight c-implemented 
> writer-threads using asynchronous  I/O similar to 
> bg-delivery, but not using select(). The writer threads 
> works seemless with http and https. As with bgdelivery, a single 
> writer thread can serve a multitude of concurrent deliveries. 
> When several writer threads are defined, the load is split up 
> between these. NaviServer can also serve streaming 
> HTML (multiple ns_write commands) via writer threads. 
> It also support static and dynamic gzip deliveries see e.g. [3] 
>
> When one uses OpenACS with NaviServer it will automatically use 
> writer-threads when configured. In reference [4] on can see the 
> difference in response time (actually the time duration spent 
> in connection threads) in NaviServer. OpenACS.org runs 
> on NaviServer since Sep 2014. A more detailed discussion 
> of these properties is in [5], all of this is part of NaviServer 4.99.6. 
>
> sorry for the longish reply, 
> -g 
>
> [1] 
>
> http://openacs.org/xowiki/Boost_your_application_performance_to_serve_large_files!
>  
>
> [2] http://en.wikipedia.org/wiki/Denial-of-service_attack#Slow_Read_attack 
> [3] 
>
> http://openacs.org/api-doc/proc-view?proc=ad_returnfile_background&source_p=1 
> [3] http://www.qcode.co.uk/pos

Re: [AOLSERVER] AOLserver questions

[Gustaf Neumann]

Dear Sep,

The question whether it is worth to use asynchronous delivery boils
down to a question of usage pattern and desired scalability.
The general problem with serving (large) resources via
classical aolserver is that a connection thread is unable
to handle other threads for the time span of the delivery.
It is important to understand that the time span of the delivery is mostly
determined by the client. A client with little processing power connection
over e.g. a mobile phone can block a connection quite a long time. A
special instance of this is the slow-read attack [2], which is
a special denial-of-service attack.

To serve e.g. 60 concurrent files one would require 60
connection threads. Note that this can happen quite soon when
serving content with several included  resources (images, css, js)
the first time to a client. When the server runs out of connection
threads, the requests are queued, which means that the
the user-perceived runtime of a request is actually queueing
time plus execution time.

Background delivery (as described in [2]) is fully integrated in OpenACS
addresses the problem by delegating output spooling (file deliveries)
to a single thread, which can deliver easily several 100 concurrent
downloads by using Tcl's asynchronous I/O operations. Note that
this works not only for static resources, but as well dynamic
requests (e.g. generating long HTML pages from e.g. a database).
We used this approach with very good success since 2006
in large OpenACS installations (with e.g. 2000 simultaneous
active users; "simultaneous active" means here users who
requested pages within a time interval of 5 secs).

In OpenACS, one can use simply ad_returnfile_background [3]
instead of ad_returnfile to make use of background delivery.

The limitations of background delivery are that (a) it just works for
plain http, and (b) that it works for at most 1024 concurrently open
file handles. We addressed (a) by using a reverse proxy in front
of the server, which delivers the files from the backend via https.
The limitation (b) is harder, since it depends on Tcl's usage of the 
select()
system call, which allows to wait for events for max. 1024 file
descriptors. Above this limit, it simply crashes. Lifting this limit
in systems like Linux is possible, but requires a privately compiled
libc and linux kernel. You might think, 1024 this is much more
one needs, but we were actually running close to this limit for
lecture casting (video streaming of university lectures).

A better approach is to use NaviServer.'s c-level support.
NaviServer provides lightweight c-implemented
writer-threads using asynchronous  I/O similar to
bg-delivery, but not using select(). The writer threads
works seemless with http and https. As with bgdelivery, a single
writer thread can serve a multitude of concurrent deliveries.
When several writer threads are defined, the load is split up
between these. NaviServer can also serve streaming
HTML (multiple ns_write commands) via writer threads.
It also support static and dynamic gzip deliveries see e.g. [3]

When one uses OpenACS with NaviServer it will automatically use
writer-threads when configured. In reference [4] on can see the
difference in response time (actually the time duration spent
in connection threads) in NaviServer. OpenACS.org runs
on NaviServer since Sep 2014. A more detailed discussion
of these properties is in [5], all of this is part of NaviServer 4.99.6.

sorry for the longish reply,
-g

[1] 
http://openacs.org/xowiki/Boost_your_application_performance_to_serve_large_files!
[2] http://en.wikipedia.org/wiki/Denial-of-service_attack#Slow_Read_attack
[3] 
http://openacs.org/api-doc/proc-view?proc=ad_returnfile_background&source_p=1
[3] http://www.qcode.co.uk/post/121
[4] http://openacs.org/forums/message-view?message_id=4111406
[5] 
https://next-scripting.org/xowiki/docs/misc/naviserver-connthreadqueue/index1

Am 19.03.15 um 07:09 schrieb Sep Ng:
> Hi all,
>
> I've been reading up on aolserver background delivery tricks on 
> OpenACS and I've seen that the patches for the static TCL channel is 
> already in 4.5.1.  In the spirit of improving server performance, I've 
> been wondering if such facility is worth building on the custom app to 
> increase concurrency and scalability.
>
> Most of the time, our aolserver also has to handle incoming requests 
> for multiple jpeg, javascript libraries, and a lot of other things. 
>  Freeing up the connection thread sounds very useful in improving the 
> server scalability so I wanted a little bit of help on getting this to 
> work.
>
> It's been hard trying to wrap my head around using ns_conn channel and 
> what I can actually do with this static TCL thread.  It seems that I 
> should be redefining ns_returnfile to use background delivery.  Could 
> I use it to push a TCL proc that generates given the parameters, the 
> dynamic page to this TCL channel to free up my connections?
>
> Sep

---

Re: [AOLSERVER] AOLserver questions

[Sep Ng]

Hi Alex!

Unfortunately, we do have some tcl code that goes on before the image file 
requests.  So, at the moment, I am thinking of getting those tasks 
offloaded ASAP from the connection threads to serve valuable requests 
instead of the static files.  Scalability has been a sticking point for 
years now and I hope to finally get something working to address this.

Regards.

On Friday, March 20, 2015 at 10:27:32 AM UTC+8, Alex Hisen wrote:
>
>  Hi, Sep. 
>
>  
>
> If you don’t need to run any tcl code when serving these requests and you 
> still want to serve them using AOLserver (as opposed to some other web 
> server or a CDN), you can take advantage of pools to segregate threads for 
> serving these static resources into their own pool. If those threads never 
> allocate a tcl interpreter, they will be an order of magnitude smaller in 
> footprint than your normal threads and so you can just have a lot of them.
>
>  
>
> We have a module that you might find useful that helps with this:
>
> http://aolserver.am.net/code/modules/ampools.adpx
>
>  
>
> -Alex Hisen
>
>  
>
> *From:* Sep Ng [mailto:thejack...@gmail.com ] 
> *Sent:* Thursday, March 19, 2015 5:52 PM
> *To:* aols...@googlegroups.com 
> *Cc:* aolserv...@lists.sourceforge.net 
> *Subject:* Re: [AOLSERVER] AOLserver questions
>
>  
>  
> Generally, what I hope to achieve is that all of these static files will 
> be offloaded into a single connection thread(?)  So, when a request for a 
> static file comes in, I can push it to this sleeping thread and then serve 
> another request while this sleeping thread will look up the image and do 
> ns_returnfile (I guess).  At least, this is how I'm envisioning it right 
> now.
>  
>  
>  
> I don't know if I'm looking at it right or not.  Judging from my readings 
> on Gustaf's work, this is how it would operate.  Feel free to correct me if 
> I'm looking at this totally wrong.
>  
>
> On Friday, March 20, 2015 at 8:42:04 AM UTC+8, Tony Bennett (Brown Paper 
> Tickets) wrote:
>
>  Scheduling isn't needed. I read your question again and I understand 
> what you're looking for.  You're asking for all the javascript and images 
> on a page to be sent in one request correct?  You'll need to find a way to 
> buffer the output and then parse and change the buffer before it's sent.  
> It would be nice to have this be part of ns_register_filter postauth.
>  
> On 3/19/15 5:12 PM, Sep Ng wrote:
>  
>  Thanks for the reply.  I am perhaps confused with all of this.  It seems 
> that if I use the scheduling proc, I can start a thread that runs 
> perpetually and does nothing.  Then, I can use tclthread API to transfer 
> control into this and issue some proc that would perform mutex and serve 
> the file to the current ns_conn details and quit.  Am I thinking this right 
> or am I being stupid? :-)
>  
>
> On Friday, March 20, 2015 at 5:09:20 AM UTC+8, Tony Bennett (Brown Paper 
> Tickets) wrote: 
>
>  Look at the scheduling commands at 
> http://panoptic.com/wiki/aolserver/Tcl_API.  You could make an image 
> processing queue that runs in it's own thread and it won't take up any 
> connections.
>
> Tony
>  
> On 3/18/15 11:09 PM, Sep Ng wrote:
>  
>  Hi all, 
>  
>  
>  
> I've been reading up on aolserver background delivery tricks on OpenACS 
> and I've seen that the patches for the static TCL channel is already in 
> 4.5.1.  In the spirit of improving server performance, I've been wondering 
> if such facility is worth building on the custom app to increase 
> concurrency and scalability.
>  
>  
>  
> Most of the time, our aolserver also has to handle incoming requests for 
> multiple jpeg, javascript libraries, and a lot of other things.  Freeing up 
> the connection thread sounds very useful in improving the server 
> scalability so I wanted a little bit of help on getting this to work.
>  
>  
>  
> It's been hard trying to wrap my head around using ns_conn channel and 
> what I can actually do with this static TCL thread.  It seems that I should 
> be redefining ns_returnfile to use background delivery.  Could I use it to 
> push a TCL proc that generates given the parameters, the dynamic page to 
> this TCL channel to free up my connections?
>  
>  
>  
> Sep
>  
>  
>
> --
>
> Dive into the World of Parallel Programming The Go Parallel Website, sponsored
>
> by Intel and developed in partnership with Slashdot Media, is your 

Re: [AOLSERVER] AOLserver questions

[Alex Hisen]

Hi, Sep.

If you don’t need to run any tcl code when serving these requests and you still 
want to serve them using AOLserver (as opposed to some other web server or a 
CDN), you can take advantage of pools to segregate threads for serving these 
static resources into their own pool. If those threads never allocate a tcl 
interpreter, they will be an order of magnitude smaller in footprint than your 
normal threads and so you can just have a lot of them.

We have a module that you might find useful that helps with this:
http://aolserver.am.net/code/modules/ampools.adpx

-Alex Hisen

From: Sep Ng [mailto:thejackschm...@gmail.com]
Sent: Thursday, March 19, 2015 5:52 PM
To: aolser...@googlegroups.com
Cc: aolserver-talk@lists.sourceforge.net
Subject: Re: [AOLSERVER] AOLserver questions

Generally, what I hope to achieve is that all of these static files will be 
offloaded into a single connection thread(?)  So, when a request for a static 
file comes in, I can push it to this sleeping thread and then serve another 
request while this sleeping thread will look up the image and do ns_returnfile 
(I guess).  At least, this is how I'm envisioning it right now.

I don't know if I'm looking at it right or not.  Judging from my readings on 
Gustaf's work, this is how it would operate.  Feel free to correct me if I'm 
looking at this totally wrong.

On Friday, March 20, 2015 at 8:42:04 AM UTC+8, Tony Bennett (Brown Paper 
Tickets) wrote:
Scheduling isn't needed. I read your question again and I understand what 
you're looking for.  You're asking for all the javascript and images on a page 
to be sent in one request correct?  You'll need to find a way to buffer the 
output and then parse and change the buffer before it's sent.  It would be nice 
to have this be part of ns_register_filter postauth.
On 3/19/15 5:12 PM, Sep Ng wrote:
Thanks for the reply.  I am perhaps confused with all of this.  It seems that 
if I use the scheduling proc, I can start a thread that runs perpetually and 
does nothing.  Then, I can use tclthread API to transfer control into this and 
issue some proc that would perform mutex and serve the file to the current 
ns_conn details and quit.  Am I thinking this right or am I being stupid? :-)

On Friday, March 20, 2015 at 5:09:20 AM UTC+8, Tony Bennett (Brown Paper 
Tickets) wrote:
Look at the scheduling commands at http://panoptic.com/wiki/aolserver/Tcl_API.  
You could make an image processing queue that runs in it's own thread and it 
won't take up any connections.

Tony
On 3/18/15 11:09 PM, Sep Ng wrote:
Hi all,

I've been reading up on aolserver background delivery tricks on OpenACS and 
I've seen that the patches for the static TCL channel is already in 4.5.1.  In 
the spirit of improving server performance, I've been wondering if such 
facility is worth building on the custom app to increase concurrency and 
scalability.

Most of the time, our aolserver also has to handle incoming requests for 
multiple jpeg, javascript libraries, and a lot of other things.  Freeing up the 
connection thread sounds very useful in improving the server scalability so I 
wanted a little bit of help on getting this to work.

It's been hard trying to wrap my head around using ns_conn channel and what I 
can actually do with this static TCL thread.  It seems that I should be 
redefining ns_returnfile to use background delivery.  Could I use it to push a 
TCL proc that generates given the parameters, the dynamic page to this TCL 
channel to free up my connections?

Sep


--

Dive into the World of Parallel Programming The Go Parallel Website, sponsored

by Intel and developed in partnership with Slashdot Media, is your hub for all

things parallel software development, from weekly thought leadership blogs to

news, videos, case studies, tutorials and more. Take a look and join the

conversation now. http://goparallel.sourceforge.net/


___

aolserver-talk mailing list

aolserv...@lists.sourceforge.net<mailto:aolserv...@lists.sourceforge.net>

https://lists.sourceforge.net/lists/listinfo/aolserver-talk


--
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the 
conversation now. http://goparallel.sourceforge.net/___
aolserver-talk mailing list
aolserver-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/aolserver-talk

Re: [AOLSERVER] AOLserver questions

[Sep Ng]

Generally, what I hope to achieve is that all of these static files will be 
offloaded into a single connection thread(?)  So, when a request for a 
static file comes in, I can push it to this sleeping thread and then serve 
another request while this sleeping thread will look up the image and do 
ns_returnfile (I guess).  At least, this is how I'm envisioning it right 
now.

I don't know if I'm looking at it right or not.  Judging from my readings 
on Gustaf's work, this is how it would operate.  Feel free to correct me if 
I'm looking at this totally wrong.

On Friday, March 20, 2015 at 8:42:04 AM UTC+8, Tony Bennett (Brown Paper 
Tickets) wrote:
>
>  Scheduling isn't needed. I read your question again and I understand what 
> you're looking for.  You're asking for all the javascript and images on a 
> page to be sent in one request correct?  You'll need to find a way to 
> buffer the output and then parse and change the buffer before it's sent.  
> It would be nice to have this be part of ns_register_filter postauth.
>
> On 3/19/15 5:12 PM, Sep Ng wrote:
>  
>  Thanks for the reply.  I am perhaps confused with all of this.  It seems 
> that if I use the scheduling proc, I can start a thread that runs 
> perpetually and does nothing.  Then, I can use tclthread API to transfer 
> control into this and issue some proc that would perform mutex and serve 
> the file to the current ns_conn details and quit.  Am I thinking this right 
> or am I being stupid? :-)
>
> On Friday, March 20, 2015 at 5:09:20 AM UTC+8, Tony Bennett (Brown Paper 
> Tickets) wrote: 
>>
>>  Look at the scheduling commands at 
>> http://panoptic.com/wiki/aolserver/Tcl_API.  You could make an image 
>> processing queue that runs in it's own thread and it won't take up any 
>> connections.
>>
>> Tony
>>
>> On 3/18/15 11:09 PM, Sep Ng wrote:
>>  
>> Hi all, 
>>
>>  I've been reading up on aolserver background delivery tricks on OpenACS 
>> and I've seen that the patches for the static TCL channel is already in 
>> 4.5.1.  In the spirit of improving server performance, I've been wondering 
>> if such facility is worth building on the custom app to increase 
>> concurrency and scalability.
>>
>>  Most of the time, our aolserver also has to handle incoming requests 
>> for multiple jpeg, javascript libraries, and a lot of other things. 
>>  Freeing up the connection thread sounds very useful in improving the 
>> server scalability so I wanted a little bit of help on getting this to work.
>>
>>  It's been hard trying to wrap my head around using ns_conn channel and 
>> what I can actually do with this static TCL thread.  It seems that I should 
>> be redefining ns_returnfile to use background delivery.  Could I use it to 
>> push a TCL proc that generates given the parameters, the dynamic page to 
>> this TCL channel to free up my connections?
>>
>>  Sep
>>  
>>  
>> --
>> Dive into the World of Parallel Programming The Go Parallel Website, 
>> sponsored
>> by Intel and developed in partnership with Slashdot Media, is your hub for 
>> all
>> things parallel software development, from weekly thought leadership blogs to
>> news, videos, case studies, tutorials and more. Take a look and join the 
>> conversation now. http://goparallel.sourceforge.net/
>>
>>
>>  
>> ___
>> aolserver-talk mailing 
>> listaolserv...@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/aolserver-talk
>>
>>  
>>   
>  --
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the 
conversation now. http://goparallel.sourceforge.net/___
aolserver-talk mailing list
aolserver-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/aolserver-talk

Re: [AOLSERVER] AOLserver questions

[Tony Bennett (Brown Paper Tickets)]

Scheduling isn't needed. I read your question again and I understand
what you're looking for.  You're asking for all the javascript and
images on a page to be sent in one request correct?  You'll need to find
a way to buffer the output and then parse and change the buffer before
it's sent.  It would be nice to have this be part of ns_register_filter
postauth.

On 3/19/15 5:12 PM, Sep Ng wrote:
> Thanks for the reply.  I am perhaps confused with all of this.  It
> seems that if I use the scheduling proc, I can start a thread that
> runs perpetually and does nothing.  Then, I can use tclthread API to
> transfer control into this and issue some proc that would perform
> mutex and serve the file to the current ns_conn details and quit.  Am
> I thinking this right or am I being stupid? :-)
>
> On Friday, March 20, 2015 at 5:09:20 AM UTC+8, Tony Bennett (Brown
> Paper Tickets) wrote:
>
> Look at the scheduling commands at
> http://panoptic.com/wiki/aolserver/Tcl_API
> .  You could make an
> image processing queue that runs in it's own thread and it won't
> take up any connections.
>
> Tony
>
> On 3/18/15 11:09 PM, Sep Ng wrote:
>> Hi all,
>>
>> I've been reading up on aolserver background delivery tricks on
>> OpenACS and I've seen that the patches for the static TCL channel
>> is already in 4.5.1.  In the spirit of improving server
>> performance, I've been wondering if such facility is worth
>> building on the custom app to increase concurrency and scalability.
>>
>> Most of the time, our aolserver also has to handle incoming
>> requests for multiple jpeg, javascript libraries, and a lot of
>> other things.  Freeing up the connection thread sounds very
>> useful in improving the server scalability so I wanted a little
>> bit of help on getting this to work.
>>
>> It's been hard trying to wrap my head around using ns_conn
>> channel and what I can actually do with this static TCL thread.
>>  It seems that I should be redefining ns_returnfile to use
>> background delivery.  Could I use it to push a TCL proc that
>> generates given the parameters, the dynamic page to this TCL
>> channel to free up my connections?
>>
>> Sep
>>
>>
>> 
>> --
>> Dive into the World of Parallel Programming The Go Parallel Website, 
>> sponsored
>> by Intel and developed in partnership with Slashdot Media, is your hub 
>> for all
>> things parallel software development, from weekly thought leadership 
>> blogs to
>> news, videos, case studies, tutorials and more. Take a look and join the 
>> conversation now. http://goparallel.sourceforge.net/ 
>> 
>>
>>
>> ___
>> aolserver-talk mailing list
>> aolserv...@lists.sourceforge.net 
>> https://lists.sourceforge.net/lists/listinfo/aolserver-talk 
>> 
>

--
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the 
conversation now. http://goparallel.sourceforge.net/___
aolserver-talk mailing list
aolserver-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/aolserver-talk

Re: [AOLSERVER] AOLserver questions

[Sep Ng]

Thanks for the reply.  I am perhaps confused with all of this.  It seems 
that if I use the scheduling proc, I can start a thread that runs 
perpetually and does nothing.  Then, I can use tclthread API to transfer 
control into this and issue some proc that would perform mutex and serve 
the file to the current ns_conn details and quit.  Am I thinking this right 
or am I being stupid? :-)

On Friday, March 20, 2015 at 5:09:20 AM UTC+8, Tony Bennett (Brown Paper 
Tickets) wrote:
>
>  Look at the scheduling commands at 
> http://panoptic.com/wiki/aolserver/Tcl_API.  You could make an image 
> processing queue that runs in it's own thread and it won't take up any 
> connections.
>
> Tony
>
> On 3/18/15 11:09 PM, Sep Ng wrote:
>  
> Hi all, 
>
>  I've been reading up on aolserver background delivery tricks on OpenACS 
> and I've seen that the patches for the static TCL channel is already in 
> 4.5.1.  In the spirit of improving server performance, I've been wondering 
> if such facility is worth building on the custom app to increase 
> concurrency and scalability.
>
>  Most of the time, our aolserver also has to handle incoming requests for 
> multiple jpeg, javascript libraries, and a lot of other things.  Freeing up 
> the connection thread sounds very useful in improving the server 
> scalability so I wanted a little bit of help on getting this to work.
>
>  It's been hard trying to wrap my head around using ns_conn channel and 
> what I can actually do with this static TCL thread.  It seems that I should 
> be redefining ns_returnfile to use background delivery.  Could I use it to 
> push a TCL proc that generates given the parameters, the dynamic page to 
> this TCL channel to free up my connections?
>
>  Sep
>  
>  
> --
> Dive into the World of Parallel Programming The Go Parallel Website, sponsored
> by Intel and developed in partnership with Slashdot Media, is your hub for all
> things parallel software development, from weekly thought leadership blogs to
> news, videos, case studies, tutorials and more. Take a look and join the 
> conversation now. http://goparallel.sourceforge.net/
>
>
>  
> ___
> aolserver-talk mailing listaolserv...@lists.sourceforge.net 
> https://lists.sourceforge.net/lists/listinfo/aolserver-talk
>
>  
>  --
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the 
conversation now. http://goparallel.sourceforge.net/___
aolserver-talk mailing list
aolserver-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/aolserver-talk

Re: [AOLSERVER] AOLserver questions

[Tony Bennett (Brown Paper Tickets)]

Look at the scheduling commands at
http://panoptic.com/wiki/aolserver/Tcl_API.  You could make an image
processing queue that runs in it's own thread and it won't take up any
connections.

Tony

On 3/18/15 11:09 PM, Sep Ng wrote:
> Hi all,
>
> I've been reading up on aolserver background delivery tricks on
> OpenACS and I've seen that the patches for the static TCL channel is
> already in 4.5.1.  In the spirit of improving server performance, I've
> been wondering if such facility is worth building on the custom app to
> increase concurrency and scalability.
>
> Most of the time, our aolserver also has to handle incoming requests
> for multiple jpeg, javascript libraries, and a lot of other things.
>  Freeing up the connection thread sounds very useful in improving the
> server scalability so I wanted a little bit of help on getting this to
> work.
>
> It's been hard trying to wrap my head around using ns_conn channel and
> what I can actually do with this static TCL thread.  It seems that I
> should be redefining ns_returnfile to use background delivery.  Could
> I use it to push a TCL proc that generates given the parameters, the
> dynamic page to this TCL channel to free up my connections?
>
> Sep
>
>
> --
> Dive into the World of Parallel Programming The Go Parallel Website, sponsored
> by Intel and developed in partnership with Slashdot Media, is your hub for all
> things parallel software development, from weekly thought leadership blogs to
> news, videos, case studies, tutorials and more. Take a look and join the 
> conversation now. http://goparallel.sourceforge.net/
>
>
> ___
> aolserver-talk mailing list
> aolserver-talk@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/aolserver-talk

--
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the 
conversation now. http://goparallel.sourceforge.net/___
aolserver-talk mailing list
aolserver-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/aolserver-talk