The three mid-tiers we deployed into production in August and September last
year are all 64-bit:
Windows Server 2003 R2 Enterprise x64 Sp2, and Windows Server 2003 Enterprise
x64 Sp2
JDK 1.6.24 x64 (32-bit is also installed)
Tomcat 6.0.32 x64 full install including Service Startup, Native
(runs under a domain account with FULL permissions to the Tomcat 6.0, OpenSSL,
and BMC Software directories)
Mid-Tier 7.6.04 SP1_MDB_08032011+ESC23674+403423+398559 x64 (specified 64-bit
JRE during installation)
All are set to use SSL
Added to Java Options for Tomcat:
-Xincgc
-XX:PermSize=256m
-XX:MaxPermSize=256m
-XX:+UseConcMarkSweepGC
//this enables the low pause concurrent garbage collector
-XX:+UseCompressedOops
//improves performance of 64 bit JDK; Must be on at least 1.6
patch 006 of the JDK to use
Initial memory pool = 1536
Maximum memory pool = 1536
Shutdown = 240
-In <tomcat root>/conf/server.xml change the <connector>
element used for the HTTP and HTTPS SSL connections to Tomcat, add these
attributes:
connectionTimeout="90000"
maxKeepAliveRequests="-1"
acceptCount="100"
maxThreads="500"
We have not seen any performance or load problems, and of course all of our
support staff are now using these mid-tiers (one primarily) for their access to
the ITSM apps. The ONLY problem that we have had with these mid-tiers is with
the ehcache crap that they now use for pre-fetch and caching. These servers
cache all logins, to include mis-typed ones, incorrect case (capitalized first
letter), ones that do not exist at all, and ones that have been deprecated when
support staff are removed. On a ,id-tier startup, or when they feel the need
to re-fetch/re-cache, they throw all of these account names (without passwords)
at the AR Server, which passes them on to the AREA plugin which tries to
resolve them. Or something like that. The result is occasionally a crash of
the AREA plugin (simply stops authenticating - requires killing the
arplugin.exe process to force a restart), but more often it is a spate of
thread deaths on the AR Server, during which time the mid-tier becomes
unresponsive and drops sessions. BMC Support has been useless on this one,
which we reported as early as September 2011 - after going into production in
late August. The most recent thread crashes are as recent as 2 Feb, so the fun
continues. They look like this in the arerror.log (server/user/IP masked) -
these ones definitely triggered by restarting the mid-tier tomcat instance:
Mon Jan 16 02:05:04 2012: AR System server terminated when a signal/exception
was received by the server (ARNOTE 20)
Thread Id: 1240
Version: 7.6.04 SP1 HotFix 01 201107051610 Jul 5 2011 17:17:48
ServerName: xxxxxx
Database: SQL -- SQL Server
Hardware: x86_64
OS: Windows Server 2003
RPC Id: 16071
RPC Call: 36 (GSI)
RPC Queue: 390620
Client: User xxxxxxx from Mid-tier (protocol 18) at IP address
999.999.999.999
Logging On: User
Code: c0000005
Operation: read
Access Addr: 0000000000000000
Stack Begin:
Stack End
Mon Jan 16 02:05:03 2012 390620 : AR System server terminated when a
signal/exception was received by the server (ARNOTE 20)
Mon Jan 16 02:05:03 2012 0xc0000005
Mon Jan 16 02:05:03 2012 390620 : AR System server terminated - fatal error
occurred in ARSERVER (ARNOTE 21)
No, I don't attribute these problems to going 64-bit; I blame 7.6.04.01+ and
ehcache. We were seeing similar thread deaths from when people tried to log in
to our Kinetic Request web using an iCrap or other mobile device - they
capitalize your login name - but we stopped that by adding some javascript to
the login page to force everything to lower case.
Christopher Strauss, Ph.D.
Call Tracking Administration Manager
University of North Texas Computing & IT Center
http://itsm.unt.edu/
From: Action Request System discussion list(ARSList)
[mailto:[email protected]] On Behalf Of Andrew C Goodall
Sent: Tuesday, January 17, 2012 9:36 AM
To: [email protected]
Subject: 32bit Vs.64bit JVM performance
**
I'm soliciting feedback regarding experience observed in the field regarding
migrating from 32bit JVM to 64bit processes - is it worth it?
So we'll be upgrading from 7.5 32bit to 7.6.04 64bit (on Windows 2008), our
major objective is to drastically improve client performance.
Due to BMC R&D response, BMC PS is recommending we stay with the 32bit JVMs -
what are your thoughts?
I don't mind extra CPU overhead if it means better performance to the client.
>From BMC Remedy AR System Server 7.6 Performance Tuning for Business Service
>Management page 20:
Many BMC customers are moving toward 64-bit platforms and running the 64-bit
JVM.
Be aware that the 64-bit JVM has performance overhead (see
http://www.oracle.com/technetwork/java/hotspotfaq-138619.html#64bit_performance).
BMC internal performance stress tests demonstrate that the 32-bit JVM
outperforms the
64-bit JVM by at least 45% in terms of CPU utilization. However, if you need
the 64-bit
JVM, consider using hybrid mode and parallel GC as recommended by Oracle, that
is, -
XX:+UseCompressedOops and -XX:+UseParallelGC. The details and
implications of using hybrid mode and parallel GC are beyond the scope of this
document.
Below is BMC R&D response to BMC PS:
The deployment architect team response was the following:
on 32-bit JVM, recommendation is to leave GC at default.
For the same exact JVM args (minus GC) when 32-bit JVM versus 64-bit
JVM, 32-bit will outperform the 64-bit JVM by about 15% (Sun's official number).
in our load test, when we run the same exact load with tomcat
hosted by 32 versus 64, 32 has about ½ CPU utilization and 25%
less heap usage.
so, when running 32-bit JVM, leave GC at default (unless a specific
GC model has shown better perf) and very safe to leave the same
exact JVM heap config as 32 bit JVM will use less heap than exact
same when running 64.
same token, when running 64-bit JVM, if not full 64-bit addressing
is needed, run in hybrid mode (-XX:+UseCompressedOops); min
perf overhead (better than the 15% Sun published.)
Regards,
Andrew Goodall
Software Engineer 2 | Development Services | jcpenney . www.jcp.com
<http://www.jcp.com/> | 972.431.1518
The information transmitted is intended only for the person or entity to which
it is addressed and
may contain confidential and/or privileged material. If the reader of this
message is not the intended
recipient, you are hereby notified that your access is unauthorized, and any
review, dissemination,
distribution or copying of this message including any attachments is strictly
prohibited. If you are not
the intended recipient, please contact the sender and delete the material from
any computer.
_attend WWRUG12 www.wwrug.com<http://www.wwrug.com> ARSlist: "Where the Answers
Are"_
_______________________________________________________________________________
UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
attend wwrug12 www.wwrug12.com ARSList: "Where the Answers Are"
