[
https://issues.apache.org/jira/browse/TS-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Susan Hinrichs updated TS-2970:
-------------------------------
Attachment: ts-2970.patch
This patch has been tested against 5.0.1. I still need to move my branch up to
master.
It is definitely fixing "a" problem. I'm not sure it is "the" problem that you
are seeing. When setting up the blind tunnel, there is a gap before the
HttpTunnel data structure is ready to move data. The incoming connection read
events were not being turned off. So if you were unlucky in the timing the
Http StateMachine may read any waiting data before the HttpTunnel is setup. In
that case, the read request logic is run again, which is a bad thing.
Fixed this by turning off events on the incoming connection. It appears that
the HttpTunnel startup logic turns the events back on.
I was able to reproduce this problem my netcatting files longer than a http
header size through Traffic Server. In my case they were html files I happened
to have sitting on my test client.
I was seeing (!s->hdr_info.server_request.valid()) assert at the top of
HttpTransact::HandleRequest() go. Not the same as the asserts you reported,
but in a more complicated operational environment, going through the request
setup twice could cause a fair number of problems.
> Multiple crashes when using tr-pass
> -----------------------------------
>
> Key: TS-2970
> URL: https://issues.apache.org/jira/browse/TS-2970
> Project: Traffic Server
> Issue Type: Bug
> Components: Core, HTTP
> Affects Versions: 5.0.0, 5.0.1
> Reporter: Nikolai Gorchilov
> Assignee: Susan Hinrichs
> Fix For: sometime
>
> Attachments: ts-2970.patch
>
>
> ATS 5.0.1 Running under Linux 3+ kernel in TPROXY mode.
> Same configuration runs without a single crash for weeks, but in the moment I
> enable tr-pass (8081:ip-in=127.0.0.1:tr-ful:tr-pass) on server_ports I get
> many crashes in matter of seconds to minutes after starting the ATS:
> {noformat}
> FATAL: HttpSM.cc:2714: failed assert `event == HTTP_TUNNEL_EVENT_DONE`
> /z/bin/traffic_server - STACK TRACE:
> /z/lib/libtsutil.so.5(ink_fatal_die+0x0)[0x2b52382e8067]
> /z/lib/libtsutil.so.5(ink_get_rand()+0x0)[0x2b52382e6b28]
> /z/bin/traffic_server(HttpSM::tunnel_handler(int, void*)+0x148)[0x5cf9aa]
> /z/bin/traffic_server(HttpSM::main_handler(int, void*)+0x333)[0x5cebeb]
> /z/bin/traffic_server(Continuation::handleEvent(int, void*)+0x68)[0x4f089a]
> /z/bin/traffic_server(CheckConnect::handle_connect(int,
> Event*)+0x2f0)[0x74313c]
> /z/bin/traffic_server(Continuation::handleEvent(int, void*)+0x68)[0x4f089a]
> /z/bin/traffic_server[0x743c82]
> /z/bin/traffic_server(write_to_net_io(NetHandler*, UnixNetVConnection*,
> EThread*)+0x389)[0x7449c3]
> /z/bin/traffic_server(write_to_net(NetHandler*, UnixNetVConnection*,
> EThread*)+0x79)[0x744638]
> /z/bin/traffic_server(NetHandler::mainNetEvent(int, Event*)+0x6f3)[0x73dc21]
> /z/bin/traffic_server(Continuation::handleEvent(int, void*)+0x68)[0x4f089a]
> /z/bin/traffic_server(EThread::process_event(Event*, int)+0x11e)[0x7664ca]
> /z/bin/traffic_server(EThread::execute()+0x45b)[0x766aa1]
> /z/bin/traffic_server[0x76598f]
> /lib64/libpthread.so.0(+0x7034)[0x2b52398f4034]
> /lib64/libc.so.6(clone+0x6d)[0x2b523a63eb5d]
> {noformat}
> {noformat}
> FATAL: HttpSM.cc:1687: failed assert `0`
> /z/bin/traffic_server - STACK TRACE:
> /z/lib/libtsutil.so.5(+0x1e837)[0x2b6545caa837]
> /z/lib/libtsutil.so.5(+0x1d51f)[0x2b6545ca951f]
> /z/bin/traffic_server(HttpSM::state_http_server_open(int,
> void*)+0xfd)[0x5a13cd]
> /z/bin/traffic_server(HttpSM::main_handler(int, void*)+0xd8)[0x59ac88]
> /z/bin/traffic_server[0x65313a]
> /z/bin/traffic_server(HostDBContinuation::probeEvent(int,
> Event*)+0x27a)[0x65967a]
> /z/bin/traffic_server(EThread::process_event(Event*, int)+0x91)[0x736071]
> /z/bin/traffic_server(EThread::execute()+0x2fb)[0x73692b]
> /z/bin/traffic_server[0x73530a]
> /lib64/libpthread.so.0(+0x7034)[0x2b65472b0034]
> /lib64/libc.so.6(clone+0x6d)[0x2b6547ffab5d]
> {noformat}
> {noformat}
> FATAL: HttpTunnel.cc:554: failed assert `active == false`
> /z/bin/traffic_server - STACK TRACE:
> /z/lib/libtsutil.so.5(+0x1e837)[0x2b679373e837]
> /z/lib/libtsutil.so.5(+0x1d51f)[0x2b679373d51f]
> /z/bin/traffic_server[0x5d660f]
> /z/bin/traffic_server(HttpSM::kill_this()+0x674)[0x59a974]
> /z/bin/traffic_server(HttpSM::main_handler(int, void*)+0x148)[0x59acf8]
> /z/bin/traffic_server(CheckConnect::handle_connect(int,
> Event*)+0x119)[0x70d659]
> /z/bin/traffic_server(UnixNetVConnection::mainEvent(int,
> Event*)+0x3f0)[0x716640]
> /z/bin/traffic_server(InactivityCop::check_inactivity(int,
> Event*)+0x275)[0x708045]
> /z/bin/traffic_server(EThread::process_event(Event*, int)+0x91)[0x736071]
> /z/bin/traffic_server(EThread::execute()+0x2fb)[0x73692b]
> /z/bin/traffic_server[0x73530a]
> /lib64/libpthread.so.0(+0x7034)[0x2b6794d44034]
> /lib64/libc.so.6(clone+0x6d)[0x2b6795a8eb5d]
> {noformat}
> My configure options other than layout-related are:
> {noformat}
> --with-group=proxy \
> --with-xml=libxml2 \
> --disable-static \
> --disable-static-libts \
> --disable-spdy \
> --enable-interim-cache \
> --enable-tproxy \
> --enable-hwloc \
> --enable-experimental-plugins \
> --enable-example-plugins
> {noformat}
> P.S. Report updated due to some --enable-debug related crashes, that weren't
> connected to tr-pass at all
--
This message was sent by Atlassian JIRA
(v6.2#6252)
