Sudheer Vinukonda created TS-3016:
-------------------------------------
Summary: High CPU in 5.0
Key: TS-3016
URL: https://issues.apache.org/jira/browse/TS-3016
Project: Traffic Server
Issue Type: Bug
Components: Core
Reporter: Sudheer Vinukonda
After 5.0 roll out, our production systems are noticing much higher cpu
compared to pre-upgrade ats4 cpu usage.
For example, on some of our hosts running 8 hosts, below is the comparison
before and after the upgade:
{code}
cpu %util: 88.7% (version: ats5)
cpu %util: 70% (version: ats4)
{code}
Running perf top on traffic_server shows most CPU spent in SSL api calls,
initially causing us to suspect any SSL related changes. However, after some
analysis, it looks like the issue might be caused by the changes made in
TS-1365. After reverting the changes in TS-1365, the CPU usage comes back to
pre-upgrade levels again.
{code}
ats5:
------
Samples: 374K of event 'cycles', Event count (approx.): 29126701697
26.44% libcrypto.so.1.0.1e [.] bn_sqr4x_mont
16.26% libcrypto.so.1.0.1e [.] bn_mul_mont <------
6.74% libcrypto.so.1.0.1e [.] bn_mul4x_mont_gather5
4.95% libcrypto.so.1.0.1e [.] BN_usub
1.54% libcrypto.so.1.0.1e [.] sha256_block_data_order
1.37% libcrypto.so.1.0.1e [.] BN_mod_mul_montgomery
80.89 libcrypto.so.1.0.1e
7.09 traffic_server
5.02 [kernel]
2.71 libc-2.12.so
0.59 libpthread-2.12.so
0.42 libssl.so.1.0.1e
0.32 libtsutil.so.5
0.32 [bnx2]
0.02 libstdc++.so.6.0.13
0.02 libpcre.so.0.0.1
0.01 quick_filter.so
0.01 libtcl8.5.so
0.01 librt-2.12.so
0.01 [kernel].vsyscall_fn
0.01 [kernel].vsyscall_1
0 regex_remap.so
0 libresolv-2.12.so
0 libgtlocip.so.1.7.4.B86
0 header_rewrite.so
0 conf_remap.so
0 [sd_mod]
0 [mpt2sas]
0 [kernel].vsyscall_0
0 [jbd]
0 [ipv6]
0 [ext3]
0 [dm_mod]
ats4:
-----
Samples: 249K of event 'cycles', Event count (approx.): 18282595621
39.34% libcrypto.so.1.0.1e [.] bn_sqr4x_mont
10.23% libcrypto.so.1.0.1e [.] bn_mul4x_mont_gather5
6.25% libcrypto.so.1.0.1e [.] bn_mul_mont <------
2.19% libcrypto.so.1.0.1e [.] sha256_block_data_order
2.03% libcrypto.so.1.0.1e [.] BN_usub
1.50% [kernel] [k] find_busiest_group
1.08% libcrypto.so.1.0.1e [.] sha1_block_data_order_ssse3
80.93 libcrypto.so.1.0.1e
6.9 [kernel]
4.96 traffic_server
3 libc-2.12.so
0.84 libpthread-2.12.so
0.81 libssl.so.1.0.1e
0.38 [bnx2]
0.32 libtsutil.so.4
0.02 libtcl8.5.so
0.02 librt-2.12.so
0.02 libpcre.so.0.0.1
0.02 [kernel].vsyscall_fn
0.01 quick_filter.so
0.01 [kernel].vsyscall_1
0.01 [kernel].vsyscall_0
0 regex_remap.so
0 libstdc++.so.6.0.13
0 libresolv-2.12.so
0 header_rewrite.so
0 conf_remap.so
0 [sd_mod]
0 [mpt2sas]
0 [jbd]
0 [ipv6]
0 [ext3]
0 [dm_mod]
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
