[
https://issues.apache.org/jira/browse/TS-3032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14109151#comment-14109151
]
Nikolai Gorchilov commented on TS-3032:
---------------------------------------
* traffic_line -r proxy.config.cache.ram_cache.size
{noformat}
1073741824
{noformat}
* traffic_line -s proxy.config.dump_mem_info_frequency -v 60
{noformat}
allocated | in-use | type size | free list name
--------------------|--------------------|------------|----------------------------------
67108864 | 0 | 2097152 |
memory/ioBufAllocator[14]
268435456 | 80740352 | 1048576 |
memory/ioBufAllocator[13]
83886080 | 7340032 | 524288 |
memory/ioBufAllocator[12]
218103808 | 11272192 | 262144 |
memory/ioBufAllocator[11]
88080384 | 9568256 | 131072 |
memory/ioBufAllocator[10]
243269632 | 36831232 | 65536 | memory/ioBufAllocator[9]
2797600768 | 2661744640 | 32768 | memory/ioBufAllocator[8]
443547648 | 184287232 | 16384 | memory/ioBufAllocator[7]
612630528 | 525918208 | 8192 | memory/ioBufAllocator[6]
1208483840 | 905179136 | 4096 | memory/ioBufAllocator[5]
4456448 | 4358144 | 2048 | memory/ioBufAllocator[4]
1179648 | 1078272 | 1024 | memory/ioBufAllocator[3]
9043968 | 9004032 | 512 | memory/ioBufAllocator[2]
4521984 | 1761536 | 256 | memory/ioBufAllocator[1]
39108608 | 1824768 | 128 | memory/ioBufAllocator[0]
120840192 | 249408 | 96 | memory/eventAllocator
10270720 | 7616480 | 80 | memory/mutexAllocator
46039040 | 17172480 | 64 | memory/ioBlockAllocator
28323840 | 20658912 | 48 | memory/ioDataAllocator
55449600 | 40364640 | 240 | memory/ioAllocator
0 | 0 | 384 | memory/socksAllocator
0 | 0 | 128 |
memory/udpReadContAllocator
0 | 0 | 160 |
memory/udpPacketAllocator
105627648 | 71248800 | 672 | memory/netVCAllocator
0 | 0 | 128 |
memory/UDPIOEventAllocator
92160 | 68400 | 720 | memory/sslNetVCAllocator
0 | 0 | 64 | memory/RamCacheLRUEntry
32526336 | 32181984 | 96 |
memory/RamCacheCLFUSEntry
1454080 | 1420640 | 160 | memory/openDirEntry
14336 | 0 | 112 |
memory/migrateToInterimCache
0 | 0 | 48 | memory/evacuationKey
6144 | 0 | 48 | memory/cacheRemoveCont
10936320 | 10875648 | 96 | memory/evacuationBlock
11673600 | 11447040 | 960 | memory/cacheVConnection
0 | 0 | 32 | memory/byteBankAllocator
0 | 0 | 592 |
memory/clusterVCAllocator
0 | 0 | 112 |
memory/inControlAllocator
0 | 0 | 112 |
memory/outControlAllocator
0 | 0 | 48 |
memory/ClusterVConnectionCache::Entry
0 | 0 | 576 |
memory/cacheContAllocator
0 | 0 | 16 |
memory/DNSRequestDataAllocator
1354240 | 33856 | 33856 | memory/dnsBufAllocator
1146880 | 20480 | 1280 | memory/dnsEntryAllocator
24477696 | 73728 | 2304 |
memory/hostDBContAllocator
0 | 0 | 112 |
memory/OneWayTunnelAllocator
168034304 | 47165440 | 2048 | memory/hdrStrHeap
217841664 | 75976704 | 2048 | memory/hdrHeap
229376 | 70400 | 256 |
memory/httpCacheAltAllocator
0 | 0 | 48 |
memory/CongestRequestParamAllocator
0 | 0 | 144 |
memory/CongestionDBContAllocator
0 | 0 | 128 | memory/RemapPluginsAlloc
0 | 0 | 7760 |
memory/httpUpdateSMAllocator
0 | 0 | 48 |
memory/CacheLookupHttpConfigAllocator
13848576 | 9485280 | 224 |
memory/httpServerSessionAllocator
332959744 | 64707968 | 7696 | memory/httpSMAllocator
51781632 | 33138336 | 672 |
memory/httpClientSessionAllocator
0 | 0 | 144 | memory/ObjectReloadCont
0 | 0 | 128 |
memory/socksProxyAllocator
0 | 0 | 96 |
memory/prefetchLockHandlerAllocator
0 | 0 | 320 |
memory/PrefetchBlasterAllocator
0 | 0 | 208 |
memory/prefetchUrlBlaster
0 | 0 | 224 | memory/blasterUrlList
0 | 0 | 192 |
memory/prefetchUrlEntryAllocator
4096 | 0 | 32 |
memory/MIMEFieldSDKHandle
30720 | 0 | 240 | memory/INKVConnAllocator
405504 | 672 | 96 | memory/INKContAllocator
229376 | 224 | 32 | memory/apiHookAllocator
0 | 0 | 592 |
memory/ICPRequestCont_allocator
0 | 0 | 112 |
memory/ICPPeerReadContAllocator
0 | 0 | 432 |
memory/PeerReadDataAllocator
0 | 0 | 496 | memory/FetchSMAllocator
43909120 | 8493056 | 1024 | memory/ArenaBlock
{noformat}
> FATAL: ats_malloc: couldn't allocate XXXXXX bytes
> -------------------------------------------------
>
> Key: TS-3032
> URL: https://issues.apache.org/jira/browse/TS-3032
> Project: Traffic Server
> Issue Type: Bug
> Components: Core
> Affects Versions: 5.0.1
> Reporter: Nikolai Gorchilov
> Assignee: Brian Geffon
> Labels: crash
> Fix For: 5.2.0
>
>
> ATS 5.0.1 under Unbuntu 12.04.4 running happily for days suddenly crashes due
> to memory allocation issue. Happens once or twice a week.
> Server is having plenty of RAM - 128G - out of which 64G+ are free. Nothing
> suspicious in dmesg.
> {noformat}
> FATAL: ats_malloc: couldn't allocate 155648 bytes
> /z/bin/traffic_server - STACK TRACE:
> /z/lib/libtsutil.so.5(+0x1e837)[0x2b6251b3d837]
> /z/lib/libtsutil.so.5(ats_malloc+0x30)[0x2b6251b40c50]
> /z/bin/traffic_server(HdrHeap::coalesce_str_heaps(int)+0x34)[0x62e834]
> /z/bin/traffic_server(http_hdr_clone(HTTPHdrImpl*, HdrHeap*,
> HdrHeap*)+0x8f)[0x62a54f]
> /z/bin/traffic_server(HttpTransactHeaders::copy_header_fields(HTTPHdr*,
> HTTPHdr*, bool, long)+0x1ae)[0x5d08de]
> /z/bin/traffic_server(HttpTransact::build_request(HttpTransact::State*,
> HTTPHdr*, HTTPHdr*, HTTPVersion)+0x5c)[0x5b280c]
> /z/bin/traffic_server(HttpTransact::HandleCacheOpenReadMiss(HttpTransact::State*)+0x2c8)[0x5c2ce8]
> /z/bin/traffic_server(HttpSM::call_transact_and_set_next_state(void
> (*)(HttpTransact::State*))+0x66)[0x58e356]
> /z/bin/traffic_server(HttpSM::state_api_callout(int, void*)+0x343)[0x599c03]
> /z/bin/traffic_server(HttpSM::set_next_state()+0x238)[0x5a0528]
> /z/bin/traffic_server(HttpSM::do_hostdb_lookup()+0x27a)[0x58e84a]
> /z/bin/traffic_server(HttpSM::set_next_state()+0xd48)[0x5a1038]
> /z/bin/traffic_server(HttpSM::state_api_callout(int, void*)+0x343)[0x599c03]
> /z/bin/traffic_server(HttpSM::state_api_callback(int, void*)+0x8a)[0x59c81a]
> /z/bin/traffic_server(TSHttpTxnReenable+0x141)[0x4caa51]
> /z/lib/plugins/x3me_dscp.so(http_txn_hook(tsapi_cont*, TSEvent,
> void*)+0x236)[0x2b626342b508]
> /z/bin/traffic_server(HttpSM::state_api_callout(int, void*)+0x102)[0x5999c2]
> /z/bin/traffic_server(HttpSM::state_cache_open_read(int,
> void*)+0x180)[0x59b070]
> /z/bin/traffic_server(HttpSM::main_handler(int, void*)+0xd8)[0x59ad98]
> /z/bin/traffic_server(HttpCacheSM::state_cache_open_read(int,
> void*)+0x173)[0x57bbb3]
> /z/bin/traffic_server(Cache::open_read(Continuation*, INK_MD5*, HTTPHdr*,
> CacheLookupHttpConfig*, CacheFragType, char*, int)+0x616)[0x6d65a6]
> /z/bin/traffic_server(CacheProcessor::open_read(Continuation*, URL*, bool,
> HTTPHdr*, CacheLookupHttpConfig*, long, CacheFragType)+0xb0)[0x6b1af0]
> /z/bin/traffic_server(HttpCacheSM::open_read(URL*, HTTPHdr*,
> CacheLookupHttpConfig*, long)+0x83)[0x57c2d3]
> /z/bin/traffic_server(HttpSM::do_cache_lookup_and_read()+0xfb)[0x58baeb]
> /z/bin/traffic_server(HttpSM::set_next_state()+0x888)[0x5a0b78]
> /z/bin/traffic_server(HttpSM::state_api_callout(int, void*)+0x343)[0x599c03]
> /z/bin/traffic_server(HttpSM::set_next_state()+0x238)[0x5a0528]
> /z/bin/traffic_server(HttpSM::set_next_state()+0x7e2)[0x5a0ad2]
> /z/bin/traffic_server(HttpSM::state_api_callout(int, void*)+0x343)[0x599c03]
> /z/bin/traffic_server(HttpSM::set_next_state()+0x238)[0x5a0528]
> /z/bin/traffic_server(HttpSM::state_api_callout(int, void*)+0x343)[0x599c03]
> /z/bin/traffic_server(HttpSM::state_api_callback(int, void*)+0x8a)[0x59c81a]
> /z/bin/traffic_server(TSHttpTxnReenable+0x141)[0x4caa51]
> /z/lib/plugins/cacheurl.so(+0x17dc)[0x2b6263a477dc]
> /z/bin/traffic_server(HttpSM::state_api_callout(int, void*)+0x102)[0x5999c2]
> /z/bin/traffic_server(HttpSM::state_api_callback(int, void*)+0x8a)[0x59c81a]
> /z/bin/traffic_server(TSHttpTxnReenable+0x141)[0x4caa51]
> /z/lib/plugins/tslua.so(+0x596f)[0x2b626363396f]
> /z/bin/traffic_server(HttpSM::state_api_callout(int, void*)+0x102)[0x5999c2]
> /z/bin/traffic_server(HttpSM::state_api_callback(int, void*)+0x8a)[0x59c81a]
> /z/bin/traffic_server(TSHttpTxnReenable+0x141)[0x4caa51]
> /z/lib/plugins/stats_over_http.so(+0x1235)[0x2b6263228235]
> /z/bin/traffic_server(HttpSM::state_api_callout(int, void*)+0x102)[0x5999c2]
> /z/bin/traffic_server(HttpSM::set_next_state()+0x238)[0x5a0528]
> /z/bin/traffic_server(HttpSM::state_read_client_request_header(int,
> void*)+0x22b)[0x59270b]
> /z/bin/traffic_server(HttpSM::main_handler(int, void*)+0xd8)[0x59ad98]
> /z/bin/traffic_server[0x714a60]
> /z/bin/traffic_server(NetHandler::mainNetEvent(int, Event*)+0x1ed)[0x7077cd]
> /z/bin/traffic_server(EThread::process_event(Event*, int)+0x91)[0x736111]
> /z/bin/traffic_server(EThread::execute()+0x4fc)[0x736bcc]
> /z/bin/traffic_server[0x7353aa]
> /lib/x86_64-linux-gnu/libpthread.so.0(+0x7e9a)[0x2b6253148e9a]
> /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x2b6253e6873d]
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
