[
https://issues.apache.org/jira/browse/TS-3016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14098589#comment-14098589
]
Sudheer Vinukonda commented on TS-3016:
---------------------------------------
[~zwoop] - Apologize, it looks like TS-1365 may not be the issue. I was running
multiple versions and probably misread the data. Unfortunately, I ran out of
the peak hour window and couldn't continue further yesterday. I am continuing
the tests today, will update once I have a more concrete evidence.
> High CPU in 5.0
> ---------------
>
> Key: TS-3016
> URL: https://issues.apache.org/jira/browse/TS-3016
> Project: Traffic Server
> Issue Type: Bug
> Components: Core
> Reporter: Sudheer Vinukonda
> Fix For: 5.1.0
>
>
> After 5.0 roll out, our production systems are noticing much higher cpu
> compared to pre-upgrade ats4 cpu usage.
> For example, on some of our hosts running 8 hosts, below is the comparison
> before and after the upgade:
> {code}
> cpu %util: 88.7% (version: ats5)
> cpu %util: 70% (version: ats4)
> {code}
> Running perf top on traffic_server shows most CPU spent in SSL api calls,
> initially causing us to suspect any SSL related changes. However, after some
> analysis, it looks like the issue might be caused by the changes made in
> TS-1365. After reverting the changes in TS-1365, the CPU usage comes back to
> pre-upgrade levels again.
> {code}
> ats5:
> ------
> Samples: 374K of event 'cycles', Event count (approx.): 29126701697
> 26.44% libcrypto.so.1.0.1e [.] bn_sqr4x_mont
> 16.26% libcrypto.so.1.0.1e [.] bn_mul_mont <------
> 6.74% libcrypto.so.1.0.1e [.] bn_mul4x_mont_gather5
> 4.95% libcrypto.so.1.0.1e [.] BN_usub
> 1.54% libcrypto.so.1.0.1e [.] sha256_block_data_order
> 1.37% libcrypto.so.1.0.1e [.] BN_mod_mul_montgomery
> 80.89 libcrypto.so.1.0.1e
> 7.09 traffic_server
> 5.02 [kernel]
> 2.71 libc-2.12.so
> 0.59 libpthread-2.12.so
> 0.42 libssl.so.1.0.1e
> 0.32 libtsutil.so.5
> 0.32 [bnx2]
> 0.02 libstdc++.so.6.0.13
> 0.02 libpcre.so.0.0.1
> 0.01 quick_filter.so
> 0.01 libtcl8.5.so
> 0.01 librt-2.12.so
> 0.01 [kernel].vsyscall_fn
> 0.01 [kernel].vsyscall_1
> 0 regex_remap.so
> 0 libresolv-2.12.so
> 0 libgtlocip.so.1.7.4.B86
> 0 header_rewrite.so
> 0 conf_remap.so
> 0 [sd_mod]
> 0 [mpt2sas]
> 0 [kernel].vsyscall_0
> 0 [jbd]
> 0 [ipv6]
> 0 [ext3]
> 0 [dm_mod]
> ats4:
> -----
> Samples: 249K of event 'cycles', Event count (approx.): 18282595621
> 39.34% libcrypto.so.1.0.1e [.] bn_sqr4x_mont
> 10.23% libcrypto.so.1.0.1e [.] bn_mul4x_mont_gather5
> 6.25% libcrypto.so.1.0.1e [.] bn_mul_mont <------
> 2.19% libcrypto.so.1.0.1e [.] sha256_block_data_order
> 2.03% libcrypto.so.1.0.1e [.] BN_usub
> 1.50% [kernel] [k] find_busiest_group
> 1.08% libcrypto.so.1.0.1e [.] sha1_block_data_order_ssse3
> 80.93 libcrypto.so.1.0.1e
> 6.9 [kernel]
> 4.96 traffic_server
> 3 libc-2.12.so
> 0.84 libpthread-2.12.so
> 0.81 libssl.so.1.0.1e
> 0.38 [bnx2]
> 0.32 libtsutil.so.4
> 0.02 libtcl8.5.so
> 0.02 librt-2.12.so
> 0.02 libpcre.so.0.0.1
> 0.02 [kernel].vsyscall_fn
> 0.01 quick_filter.so
> 0.01 [kernel].vsyscall_1
> 0.01 [kernel].vsyscall_0
> 0 regex_remap.so
> 0 libstdc++.so.6.0.13
> 0 libresolv-2.12.so
> 0 header_rewrite.so
> 0 conf_remap.so
> 0 [sd_mod]
> 0 [mpt2sas]
> 0 [jbd]
> 0 [ipv6]
> 0 [ext3]
> 0 [dm_mod]
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
