[
https://issues.apache.org/jira/browse/TS-1411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13753327#comment-13753327
]
Yunkai Zhang commented on TS-1411:
----------------------------------
By analyzing the backtraces.txt you providated:
{code}
#0 LogAccess::marshal_mem (
dest=0x2ab54ca34ec0
"pellcheck_5259.js&nq/templates/js/_shared/module/offscreen-bin/compose/modal-cancel-compose_5259.js&nq/templates/js/_shared/module/offscreen-bin/compose/menu-spell-langs_5259.js&nq/templates/js/_share"...,
source=0x2ab54cb8f9a8 <Address 0x2ab54cb8f9a8 out of bounds>, actual_len=1996,
padded_len=2000) at LogAccess.cc:814
#1 0x000000000058c3f0 in LogAccessHttp::marshal_client_req_url_canon
(this=<value optimized out>, buf=<value optimized out>) at LogAccessHttp.cc:305
#2 0x000000000059d5a2 in LogFieldList::marshal (this=<value optimized out>,
lad=0x2ab4f4402a70, buf=0x2ab54ca34e88 "") at LogField.cc:517
{code}
We can find that the root cause of this coredump is the bad *source* address in
LogAccess::marshal_mem(dest, source, actual_len, padded_len):
{code}
void
LogAccess::marshal_mem(char *dest, const char *source, int actual_len, int
padded_len)
{
...
}
----
int
LogAccessHttp::marshal_client_req_url_canon(char *buf)
{
int len = round_strlen(m_client_req_url_canon_len + 1);
if (buf) {
marshal_mem(buf, m_client_req_url_canon_str, m_client_req_url_canon_len,
len);
}
return len;
}
{code}
That is say, the address pointed to by *m_client_req_url_canon_str* is bad.
This issue should be duplicated with
https://issues.apache.org/jira/browse/TS-1757,
I have submitted a workaroud patch to fix this problem, but it has not been
merged into master:
https://issues.apache.org/jira/secure/attachment/12590770/0001-TS-1757-workaround-to-fix-core-at-escapify_url.patch
Please apply this patch, can test again.
> Seg faulting during log rotation
> --------------------------------
>
> Key: TS-1411
> URL: https://issues.apache.org/jira/browse/TS-1411
> Project: Traffic Server
> Issue Type: Bug
> Components: Logging
> Affects Versions: 3.2.0
> Environment: RHEL 6.2 x86_64
> Reporter: David Carlin
> Assignee: Leif Hedstrom
> Fix For: 4.2.0
>
> Attachments: Log rotation segaults.txt, TS-1411 backtraces.txt
>
>
> I've been experiencing some segfaults during log rotation. The sequence of
> events is this.. log rotation occurs, then I get hundreds of dropping log
> buffer error msgs, then the segfault.
> This started occurring when I lengthened the default log format to include
> the unmapped URL and the user agent string:
> %<cqtq> %<ttms> %<chi> %<crc>/%<pssc> %<psql> %<cqhm> %<cquc> %<caun>
> %<phr>/%<pqsn> %<psct> %<xid> %<cquuc> \"%<{User-Agent}cqh>\"
> In terms of frequency, we have a number of boxes and I probably see one of
> these crashed per day since the above change. Logs are rotated every 2 hours.
> I've had other log related segfaults, reported in TS-1330 - these new ones
> seem to have a different cause.
> [Aug 14 21:07:20.002] Server {0x2ae3a8887700} STATUS: The rolled logfile,
> /home/y/logs/trafficserver/error.log_l30.ycs.a4e.yahoo.com.20120814.17h59m50s-20120814.20h00m00s.old,
> was auto-deleted; 3148252 bytes were reclaimed.
> [Aug 14 21:07:42.859] Server {0x2ae3a8887700} STATUS: The rolled logfile,
> /home/y/logs/trafficserver/squid.blog_l30.ycs.a4e.yahoo.com.20120814.18h00m00s-20120814.20h00m00s.old,
> was auto-deleted; 14735520048 bytes were reclaimed.
> [Aug 14 21:07:42.865] Server {0x2ae3a8887700} WARNING: Dropping log buffer,
> can't keep up.
> [Aug 14 21:07:42.865] Server {0x2ae3a8887700} WARNING: Dropping log buffer,
> can't keep up.
> [Aug 14 21:07:42.865] Server {0x2ae3a8887700} WARNING: Dropping log buffer,
> can't keep up.
> [Aug 14 21:07:42.865] Server {0x2ae3a8887700} WARNING: Dropping log buffer,
> can't keep up.
> [Aug 14 21:07:42.865] Server {0x2ae3a8887700} WARNING: Dropping log buffer,
> can't keep up.
> [...]
> [Aug 14 21:07:42.876] Server {0x2ae3a8887700} WARNING: Dropping log buffer,
> can't keep up.
> [Aug 14 21:07:42.876] Server {0x2ae3a8887700} WARNING: Dropping log buffer,
> can't keep up.
> [Aug 14 21:07:42.876] Server {0x2ae3a8887700} WARNING: Dropping log buffer,
> can't keep up.
> [Aug 14 21:07:42.876] Server {0x2ae3a8887700} WARNING: Dropping log buffer,
> can't keep up.
> NOTE: Traffic Server received Sig 11: Segmentation fault
> /home/y/bin/traffic_server - STACK TRACE:
> /lib64/libpthread.so.0[0x383f00f500]
> /home/y/bin/traffic_server(_ZN9LogAccess11marshal_memEPcPKcii+0x48)[0x58a118]
> /home/y/bin/traffic_server(_ZN13LogAccessHttp28marshal_client_req_url_canonEPc+0x20)[0x58c3f0]
> /home/y/bin/traffic_server(_ZN12LogFieldList7marshalEP9LogAccessPc+0x32)[0x59d5a2]
> /home/y/bin/traffic_server(_ZN9LogObject3logEP9LogAccessPc+0x399)[0x5a7ed9]
> /home/y/bin/traffic_server(_ZN3Log6accessEP9LogAccess+0x146)[0x58f506]
> /home/y/bin/traffic_server(_ZN6HttpSM12update_statsEv+0x630)[0x526c50]
> /home/y/bin/traffic_server(_ZN6HttpSM9kill_thisEv+0x928)[0x52b548]
> /home/y/bin/traffic_server(_ZN6HttpSM12main_handlerEiPv+0x198)[0x52b868]
> /home/y/bin/traffic_server(_ZN10HttpTunnel12main_handlerEiPv+0xde)[0x56c3ee]
> /home/y/bin/traffic_server[0x673871]
> /home/y/bin/traffic_server(_Z15write_to_net_ioP10NetHandlerP18UnixNetVConnectionP7EThread+0x847)[0x6756e7]
> /home/y/bin/traffic_server(_ZN10NetHandler12mainNetEventEiP5Event+0x286)[0x66e076]
> /home/y/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0xb4)[0x696ce4]
> /home/y/bin/traffic_server(_ZN7EThread7executeEv+0x4c3)[0x697673]
> /home/y/bin/traffic_server[0x695cb2]
> /lib64/libpthread.so.0[0x383f007851]
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
