Thanks for all the ideas. I think folks are right about the backend. I am not sure why it was having problems when before under loads like this it did not have any issues. I wonder if it is because the proxy looks like 1 machine instead of a bunch of different machines (not really sure how all this works).

Any way, the backend is an apache2/php black box. I have managed to increase the php memory as I was seeing many, many Php memory exhausted errors. Now I see very few. I have also increased the number of workers.

Unfortunately, the load has really dropped now that school has started so I am not really able to test it.

Again thanks for all your help.

cheers,

ski

On 12/20/2013 10:07 AM, Robert Hajime Lanning wrote:
Try looking into TIME_WAIT tuning.
/etc/sysctl.conf
#TIME_WAIT tuning
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_fin_timeout = 1
net.ipv4.tcp_syncookies = 1

Then sysctl -p

On 12.20.2013 09:24, Chris Ski Kacoroski wrote:
Hi,

I have put an apache server in front of my primary webserver as a
proxy in preparing to move to a new webserver.  In testing it worked
fine, but under load (we have snow today and everyone it hitting the
website to see if school is open), I am getting all kinds of timeout
errors and service not available pages.  In server-status on the
proxy, it does not look like it is loaded:

Server load: 0.40 0.27 0.23
Total accesses: 99516 - Total Traffic: 2.6 GB
CPU Usage: u4.35 s3.08 cu0 cs0 - .382% CPU load
51.1 requests/sec - 1.4 MB/second - 27.4 kB/request
91 requests currently being processed, 59 idle workers

_KK_KK_KK_KCKKKK_KKKKK_KKKKKKKKKKCKKKKKKKKKKKKKKKK______________
____________KK____________K________KKKK_KK_KK_KKKK_KKKKKKKK_KKCC
KKKKK_KKKKKKW_KKKKKKKK..........................................
................................................................
................................................................
................................................................
................................................................
....................................................

Errors are:
(70007)The timeout specified has expired: AH00957: HTTP: attempt to
connect to 10.1.3.100:80 (webold.nsd.org) failed
AH01114: HTTP: failed to make connection to backend:
AH00959: ap_proxy_connect_backend disabling worker for
(webold.nsd.org) for 0s

Config is:
<VirtualHost *:80>
  ServerAdmin [email protected]

  DocumentRoot /var/www
  <Directory />
    Options FollowSymLinks
    AllowOverride None
  </Directory>
  <Directory /var/www/>
    Options Indexes FollowSymLinks MultiViews
    AllowOverride None
    Order allow,deny
    allow from all
  </Directory>

  ErrorLog ${APACHE_LOG_DIR}/errormap.log

  # Possible values include: debug, info, notice, warn, error, crit,
  # alert, emerg.
  #LogLevel warn rewrite:trace2
  LogLevel warn

  CustomLog ${APACHE_LOG_DIR}/accessmap.log combined

  # rewrite rules for variables after the url, e.g. after the ?
  RewriteEngine On

  # For each page we match the query string in RewriteCond and then map
  # it to the new page/server in RewriteRule.  This version uses an
external
  # file to map key, value pairs.  There is one map for each section of
  # schoolcenter.
  RewriteMap scrapbook txt:/etc/apache2/maps/scrapbook
  RewriteMap docmgr txt:/etc/apache2/maps/docmgr
  RewriteMap dept txt:/etc/apache2/maps/dept

  RewriteCond %{HTTP_HOST} .*nsdproxy.*
  RewriteRule ^/server-status.* http://nsdproxy.nsd.org/server-status

  RewriteCond %{HTTP_HOST} .*schoolcenter.*
  RewriteRule ^/(.*) http://www.nsd.org/$1

  RewriteRule ^.*components/scrapbook
${scrapbook:%{QUERY_STRING}|http://webold.nsd.org/%{REQUEST_URI}?%{QUERY_STRING}}
[END,NC,QSD]
  RewriteRule ^.*components/docmgr
${docmgr:%{QUERY_STRING}|http://webold.nsd.org/%{REQUEST_URI}?%{QUERY_STRING}}
[END,NC,QSD]
  RewriteRule ^.*education/dept
${dept:%{QUERY_STRING}|http://webold.nsd.org/%{REQUEST_URI}?%{QUERY_STRING}}
[END,NC,QSD]


  # these map all of / to http://www.nsd.org so they need to be last
  ProxyPass      /   http://webold.nsd.org/ retry=0 timeout=30
  ProxyPassReverse  /   http://webold.nsd.org/
</VirtualHost>


Any help/ideas are appreciated.

ski


--
"When we try to pick out anything by itself, we find it
 connected to the entire universe"            John Muir

Chris "Ski" Kacoroski, Director of LOPSA, [email protected],
206-501-9803, or ski98033 on most IM services "
_______________________________________________
Discuss mailing list
[email protected]
https://lists.lopsa.org/cgi-bin/mailman/listinfo/discuss
This list provided by the League of Professional System Administrators
http://lopsa.org/

Reply via email to