Github user smalenfant commented on the issue:
https://github.com/apache/trafficserver/pull/1050
I've been doing lots of testing and sharing results with @PSUdaemon. I'm
seeing the issue when using fairly big ram_cache with the normal pages under
Centos 6. I'm not using SSL at all. I've been troubleshooting this since I
posted the `ATS blocking at half hour interval` in the traffic server forum.
Test system was multiple Dell R720/730xd with 24 disks and 192GB of memory.
- `CONFIG proxy.config.cache.ram_cache.size INT 103079215104`. => 96GB
Using Centos 6.8 `2.6.32-642.4.2` (version doesn't matter, any variant of
Centos 6.7 and 6.8), I would run HLS streaming to fill up the entire ram_cache.
At some point (few hours) I would start to see system CPU spikes every 22
minutes. If you run a `perf top` during that time, you can see high activity by
`try_to_unmap_anon`.
Then, trying to stop traffic server would take about an hour and there
`try_to_unmap_anon` would show up during the entire time.
Now, I then tested with Centos 7 under the same conditions. I didn't see
the CPU Spikes and didn't experience the traffic server stop issue.
To make this work with Centos 6, I then configured Huge Pages to stop using
anonymous pages. I've set the amount of reserved Huge Pages to 48000 (96GB) and
gave it a surplus of 8000 pages (16GB).
```
vm.nr_hugepages = 48000
vm.nr_overcommit_hugepages = 8000
```
You also have to enable Huge Pages `CONFIG proxy.config.allocator.hugepages
INT 1`.
Reboot the server to ensure the pages gets allocated without being
fragmented (reserved one).
You can then observe the problem is gone (hopefully) and also that the
PageTables size is kept to a reasonable size. (Was >200,000kB before).
```
[root@myserver ~]# cat /proc/meminfo | grep Page
AnonPages: 1176352 kB
PageTables: 9836 kB
AnonHugePages: 0 kB
HugePages_Total: 48651
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 651
```
Not sure how related to madvise, but those are the way to get it working
and keep performance at a good level. Feel free to comment or ask questions.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
