[
https://issues.apache.org/jira/browse/TS-1316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13449797#comment-13449797
]
David Carlin commented on TS-1316:
----------------------------------
Still seeing this issue occasionally crop up even with
proxy.config.cache.min_average_object_size = 8000
No core dump was created.
[Sep 6 15:44:16.271] Server {0x2ae640dc7860} WARNING: cache directory overflow
on '/dev/dm-4' segment 365, purging...
NOTE: Traffic Server received Sig 11: Segmentation fault
/home/y/bin/traffic_server - STACK TRACE:
/lib64/libpthread.so.0[0x332ce0f500]
/home/y/lib64/libtsutil.so.3(ink_atomiclist_popall+0x64)[0x2ae64030bbb4]
/home/y/bin/traffic_server(_ZN16LogBufferManager13flush_buffersEP13LogBufferSink+0x1f)[0x5a6fef]
/home/y/bin/traffic_server(_ZN16LogObjectManager13flush_buffersEv+0x28)[0x5a7ac8]
/home/y/bin/traffic_server(_ZN3Log17flush_thread_mainEPv+0xaa)[0x58fdca]
/home/y/bin/traffic_server(_ZN24LoggingFlushContinuation9mainEventEiPv+0xb)[0x5932bb]
/home/y/bin/traffic_server(_ZN7EThread7executeEv+0xb86)[0x697d36]
/home/y/bin/traffic_server[0x695cb2]
/lib64/libpthread.so.0[0x332ce07851]
/lib64/libc.so.6(clone+0x6d)[0x332cae767d]
> ATS connection refused once cache direntries exhausted
> ------------------------------------------------------
>
> Key: TS-1316
> URL: https://issues.apache.org/jira/browse/TS-1316
> Project: Traffic Server
> Issue Type: Bug
> Components: Cache
> Affects Versions: 3.2.0
> Environment: RHEL 6.2 x86_64
> Reporter: David Carlin
> Fix For: 3.3.1
>
>
> We had a pair of ATS 3.2.0 boxes that stopped passing traffic simultaneously.
> Here are the traffic.out msgs we saw on both boxes:
> [Jun 22 05:53:27.637] Server {0x2b6b6d9da700} WARNING: cache directory
> overflow on '/dev/dm-4' segment 36, purging...
> [Jun 22 05:56:05.542] Server {0x2b6b6d2d3700} WARNING: cache directory
> overflow on '/dev/dm-4' segment 85, purging...
> [Jun 22 05:56:07.434] Server {0x2b6b6d4d5700} WARNING: cache directory
> overflow on '/dev/dm-4' segment 71, purging...
> [Jun 22 05:58:24.743] Server {0x2b6b6d8d9700} WARNING: cache directory
> overflow on '/dev/dm-4' segment 33, purging...
> Those messages went on for a couple minutes, then traffic apparently ceased -
> our monitoring system saw connection refused for port 80 on ATS from then on.
> The connection refused state went on for many hours until ATS was restarted.
> There were no traffic_cop msgs in /var/log/messages indicating that the
> heartbeat failed.
> Here are the relevant ATS settings/stats:
> proxy.process.cache.bytes_total = 190690320384
> proxy.process.cache.direntries.total = 5817752
> proxy.config.cache.min_average_object_size = 32768
> We previously came up with proxy.config.cache.min_average_object_size by
> waiting for the cache to fill and dividing proxy.process.cache.bytes_used by
> proxy.process.cache.direntries.used - which equals about 34KB.
> We're assuming ATS ran out of direntries and it didn't handle this situation
> gracefully. As a possible workaround, I'm going to lower
> proxy.config.cache.min_average_object_size to 24KB.
> Thanks to Bryan Call for helping me troubleshoot this!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
