Hi!
We're currently in the process of migrating our ColdFusion MX 6.1 (32-bit)
servers to ColdFusion 8 (64-bit). When testing our websites without load on the
server, everything is working fine. As soon as we switch to production mode and
let the users surf the sites, the proxy service denies requests and users get
an ugly 500 server error message. The jrun logs however do not show any
problems. We used a standalone-server configuration with CFMX 6.1 and switched
to multiserver with two instances. Immediately after starting Apache, I always
get a few pages delivered, but after roundabout 40-60 seconds, the apache error
logs spawn lots and lots of the following messages:
[Wed Oct 22 18:53:55 2008] [notice] jrApache[22743: 45316] 127.0.0.1:51000
connect failed[122]: 110 110 Connection timed out
[Wed Oct 22 18:53:55 2008] [notice] jrApache[22743: 45316] Couldn't initialize
from remote server, JRun server(s) probably down.
Testing was done by simply changing the Apache port from 80/443 to something
that's not accessible from the internet; nothing else has been touched. The
machine is the very same that used to run CFMX 6.1 without any such trouble
before, though we used the standalone server configuration.
OS is Debian Etch, with libc6 2.7-14 from Debian Lenny (along with the
dependencies). I tried the JRE that came with CF8, as well as jdk1.6.0_10 and
jdk1.5.0_16, neither resolved the issue at hand. Kernel is 2.6.18-6-amd64 SMP,
the machine has currently got 4GB RAM and two AMD Opteron 275 totalling four
cores.
This is my current jvm.config:
java.home=/opt/jrun4/jdk1.5.0_16
java.args=-server -Xmx1024m -Dsun.io.useCanonCaches=false -XX:MaxPermSize=192m
-XX:+UseParallelGC -Xbatch -Djava.awt.headless=true
-Dcoldfusion.rootDir={application.home}/
-Djava.security.policy={application.home}/servers/cfusion/cfusion-ear/cfusion-war/WEB-INF/cfusion/lib/coldfusion.policy
-Djava.security.auth.policy={application.home}/servers/cfusion/cfusion-ear/cfusion-war/WEB-INF/cfusion/lib/neo_jaas.policy
-Duser.timezone=Europe/Berlin
java.ext.dirs={jre.home}/lib/ext
java.library.path={application.home}/servers/cfusion/cfusion-ear/cfusion-war/WEB-INF/cfusion/lib
system.path.first=false
java.user.dir={application.home}/../lib
java.class.path={application.home}/servers/lib,{application.home}/lib,{application.home}/servers/cfusion/cfusion-ear/cfusion-war/WEB-INF/lib
Apache is version 2.2.3-4+etch5; here's the general config for ColdFusion:
# JRun Settings
LoadModule jrun_module /opt/jrun4/lib/wsconfig/1/mod_jrun22.so
<IfModule mod_jrun22.c>
JRunConfig Verbose false
JRunConfig Apialloc false
JRunConfig Ssl false
JRunConfig Ignoresuffixmap false
JRunConfig ProxyRetryInterval 600
JRunConfig ConnectTimeout 5
AddHandler jrun-handler .jsp .jws .cfm .cfml .cfc .cfr .cfswf
</IfModule>
This is my include for the virtual hosts of the first instance WEB-01-CYNAMITE:
<IfModule mod_jrun22.c>
JRunConfig Verbose false
JRunConfig Serverstore "/opt/jrun4/lib/wsconfig/cynamite/jrunserver.store"
JRunConfig Bootstrap 127.0.0.1:51002
Alias /CFIDE
/opt/jrun4/servers/WEB-01-CYNAMITE/cfusion.ear/cfusion.war/CFIDE
<Directory
"/opt/jrun4/servers/WEB-01-CYNAMITE/cfusion.ear/cfusion.war/CFIDE">
AllowOverride All
Order allow,deny
Allow from all
</Directory>
This is my include for the virtual host of the second instance named WEB-01-CIA:
<IfModule mod_jrun22.c>
JRunConfig Verbose false
JRunConfig Serverstore "/opt/jrun4/lib/wsconfig/cia/jrunserver.store"
JRunConfig Bootstrap 127.0.0.1:51000
Alias /CFIDE /opt/jrun4/servers/WEB-01-CIA/cfusion.ear/cfusion.war/CFIDE
<Directory "/opt/jrun4/servers/WEB-01-CIA/cfusion.ear/cfusion.war/CFIDE">
AllowOverride All
Order allow,deny
Allow from all
</Directory>
</IfModule>
The non-instance server has got the following include:
<IfModule mod_jrun22.c>
JRunConfig Serverstore /opt/jrun4/lib/wsconfig/1/jrunserver.store
JRunConfig Bootstrap 127.0.0.1:51020
</IfModule>
I made sure that the jrunserver.store-files are present and accessible; the
ports are correct too (otherwise the whole lot wouldn't be working in the test
scenario). The Proxy-Services are started alright according to the CF-logs.
I edited the ./servers/[instancename]/SERVER-INF/jrun.xml files; this is the
section concerning the ProxyService:
<service class="jrun.servlet.jrpp.JRunProxyService" name="ProxyService">
<attribute name="activeHandlerThreads">80</attribute>
<attribute name="backlog">500</attribute>
<attribute name="deactivated">false</attribute>
<attribute name="interface">*</attribute>
<attribute name="maxHandlerThreads">80</attribute>
<attribute name="minHandlerThreads">10</attribute>
<attribute name="port">51000</attribute>
<attribute name="threadWaitTimeout">300</attribute>
<attribute name="timeout">300</attribute>
<!-- if 0, web server connector loads mappings once at startup -->
<!-- if not specified, default is every 10 seonds -->
<attribute name="mapcheck">0</attribute>
<!-- set this to false for multi-hosted sites -->
<attribute name="cacheRealPath">false</attribute>
<!--
<attribute name="keyStore">{jrun.rootdir}/lib/keystore</attribute>
<attribute name="keyStorePassword">changeit</attribute>
<attribute name="trustStore">{jrun.rootdir}/lib/trustStore</attribute>
<attribute
name="socketFactoryName">jrun.servlet.jrpp.JRunProxySSLServerSocketFactory</attribute>
-->
</service>
Concerning the tuning in the Admin interface, I have tried the "Max
simultaneous template requests" settings with both 50 (I know it's high, but it
has worked fine with CF MX 6.1 to help it over the ramp-up after a restart) and
16, neither changed anything about the issue.
I cannot see that it's a memory issue either - the CFML-code base hasn't
changed. And vmstat doesn't show any memory exhaustion; this is a vmstat 5
which is started just after starting Apache and running until the first error
messages appeared in the webserver log:
# vmstat 5
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
0 0 28184 2152100 67412 360480 1 2 22 89 165 288 4 1 92 3
0 0 28184 2154652 67420 360484 0 0 0 30 268 458 1 0 99 0
0 0 28184 2135116 67428 367752 0 0 0 88 533 1135 8 2 90 0
2 0 28184 2132248 67444 367880 0 0 32 37 310 811 6 0 93 1
0 0 28184 2098932 67724 369540 0 0 280 90 1725 2381 27 2 63 9
0 0 28184 2023000 67952 381332 0 0 1198 261 955 1465 15 2 78 4
1 0 28184 1987232 68064 388624 0 0 711 62 663 859 2 1 95 2
0 0 28184 1909228 68080 388852 0 0 2 385 1037 1300 3 2 82 13
0 0 28184 1867604 68092 389512 0 0 119 65 474 1154 3 1 96 0
1 0 28184 1759428 68104 389592 0 0 2 30 646 1022 2 2 96 1
1 0 28184 1732936 68108 389636 0 0 2 30 859 1007 1 1 97 0
As you may see, there's still a lot of memory free and the CPU is all but
totally idle. There's no swapping and i/o wait is hardly noticeable. So why on
earth do I get so much less performance out of the supposedly far superior CF8
64-bit multi-instance configuration than out of good old CFMX 6.1 running on a
32-bit VM?
If I cannot resolve this soon, I'll have to try with a standalone server
install and hope for better luck with that one. Any help or pointers in the
right direction would be immensely appreciated!
Kind regards
Markus
Jede Stimme zählt, jetzt abstimmen für die besten Games: www.bamaward.de
Computec Media AG
Sitz der Gesellschaft und Registergericht: Fürth (HRB 8818)
Vorstandsmitglieder: Johannes S. Gözalan (Vorsitzender) und Rainer Rosenbusch
Vorsitzender des Aufsichtsrates: Jürg Marquard
Umsatzsteuer-Identifikationsnummer: DE 812 575 276
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to
date
Get the Free Trial
http://ad.doubleclick.net/clk;207172674;29440083;f
Archive:
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:314268
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
