[
https://issues.apache.org/jira/browse/TS-2941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Ivanov updated TS-2941:
------------------------------
Description:
We are seeing lots of RSTs on our edge boxes from HTTPS-enabled clients:
{code}
20:31:42.861752 IP client-ip.56898 > www.linkedin.com.https: Flags [S], seq
2989744105, win 65535, options [mss 1460,nop,wscale 4,nop,nop,TS val 1060727394
ecr 0,sackOK,eol], length 0
20:31:42.861760 IP www.linkedin.com.https > client-ip.56898: Flags [S.], seq
441157858, ack 2989744106, win 14480, options [mss 1460,sackOK,TS val 579955489
ecr 1060727394,nop,wscale 7], length 0
20:31:43.005735 IP client-ip.56898 > www.linkedin.com.https: Flags [.], ack 1,
win 8235, options [nop,nop,TS val 1060727553 ecr 579955489], length 0
20:31:43.012094 IP client-ip.56898 > www.linkedin.com.https: Flags [P.], seq
1:216, ack 1, win 8235, options [nop,nop,TS val 1060727555 ecr 579955489],
length 215
20:31:43.012115 IP www.linkedin.com.https > client-ip.56898: Flags [.], ack
216, win 122, options [nop,nop,TS val 579955639 ecr 1060727555], length 0
20:31:43.012400 IP www.linkedin.com.https > client-ip.56898: Flags [P.], seq
1:186, ack 216, win 122, options [nop,nop,TS val 579955640 ecr 1060727555],
length 185
20:31:43.157903 IP client-ip.56898 > www.linkedin.com.https: Flags [.], ack
186, win 8223, options [nop,nop,TS val 1060727703 ecr 579955640], length 0
20:31:43.157919 IP client-ip.56898 > www.linkedin.com.https: Flags [P.], seq
216:222, ack 186, win 8223, options [nop,nop,TS val 1060727704 ecr 579955640],
length 6
20:31:43.163988 IP client-ip.56898 > www.linkedin.com.https: Flags [P.], seq
222:307, ack 186, win 8223, options [nop,nop,TS val 1060727704 ecr 579955640],
length 85
20:31:43.164096 IP www.linkedin.com.https > client-ip.56898: Flags [.], ack
307, win 122, options [nop,nop,TS val 579955791 ecr 1060727704], length 0
20:31:43.164650 IP client-ip.56898 > www.linkedin.com.https: Flags [.], seq
307:1755, ack 186, win 8223, options [nop,nop,TS val 1060727705 ecr 579955640],
length 1448
20:31:43.164668 IP client-ip.56898 > www.linkedin.com.https: Flags [P.], seq
1755:2472, ack 186, win 8223, options [nop,nop,TS val 1060727705 ecr
579955640], length 717
20:31:43.164677 IP www.linkedin.com.https > client-ip.56898: Flags [.], ack
2472, win 167, options [nop,nop,TS val 579955792 ecr 1060727705], length 0
20:31:43.504019 IP www.linkedin.com.https > client-ip.56898: Flags [P.], seq
186:1167, ack 2472, win 167, options [nop,nop,TS val 579956131 ecr 1060727705],
length 981
20:31:43.504043 IP www.linkedin.com.https > client-ip.56898: Flags [P.], seq
1167:1764, ack 2472, win 167, options [nop,nop,TS val 579956131 ecr
1060727705], length 597
20:31:43.504101 IP www.linkedin.com.https > client-ip.56898: Flags [.], seq
1764:3212, ack 2472, win 167, options [nop,nop,TS val 579956131 ecr
1060727705], length 1448
20:31:43.504118 IP www.linkedin.com.https > client-ip.56898: Flags [.], seq
3212:4660, ack 2472, win 167, options [nop,nop,TS val 579956131 ecr
1060727705], length 1448
20:31:43.504150 IP www.linkedin.com.https > client-ip.56898: Flags [.], seq
4660:6108, ack 2472, win 167, options [nop,nop,TS val 579956132 ecr
1060727705], length 1448
20:31:43.743945 IP client-ip.56898 > www.linkedin.com.https: Flags [.], ack
1167, win 8162, options [nop,nop,TS val 1060728286 ecr 579956131], length 0
20:31:43.743966 IP www.linkedin.com.https > client-ip.56898: Flags [P.], seq
6108:6126, ack 2472, win 167, options [nop,nop,TS val 579956371 ecr
1060728286], length 18
20:31:43.743987 IP client-ip.56898 > www.linkedin.com.https: Flags [.], ack
4660, win 7944, options [nop,nop,TS val 1060728286 ecr 579956131], length 0
20:31:43.743994 IP client-ip.56898 > www.linkedin.com.https: Flags [.], ack
6108, win 8192, options [nop,nop,TS val 1060728286 ecr 579956132], length 0
20:31:43.895889 IP client-ip.56898 > www.linkedin.com.https: Flags [.], ack
6126, win 8190, options [nop,nop,TS val 1060728437 ecr 579956371], length 0
20:31:54.189610 IP www.linkedin.com.https > client-ip.56898: Flags [F.], seq
6126, ack 2472, win 167, options [nop,nop,TS val 579966817 ecr 1060728437],
length 0
20:31:54.305500 IP client-ip.56898 > www.linkedin.com.https: Flags [.], ack
6127, win 8192, options [nop,nop,TS val 1060738819 ecr 579966817], length 0
20:31:54.775695 IP client-ip.56898 > www.linkedin.com.https: Flags [P.], seq
2472:2541, ack 6127, win 8192, options [nop,nop,TS val 1060739277 ecr
579966817], length 69
20:31:54.775732 IP www.linkedin.com.https > client-ip.56898: Flags [R], seq
441163985, win 0, length 0
20:31:54.775739 IP client-ip.56898 > www.linkedin.com.https: Flags [F.], seq
2541, ack 6127, win 8192, options [nop,nop,TS val 1060739277 ecr 579966817],
length 0
20:31:54.775743 IP www.linkedin.com.https > client-ip.56898: Flags [R], seq
441163985, win 0, length 0
{code}
You can see that after {{proxy.config.http.keep_alive_no_activity_timeout_in}}
(10 secs) ATS is closes TCP connection without sending TLS {{close_notify}}
alert which results in client sending it (last part with {{[P.], seq
2472:2541}}) and only then normally closing the connection ({{[F.], seq
2541}}), but because on ATS side socket is already closed OS replies with RST
(twice actually, one for {{2472:2541}} and one for {{2541}}).
>From standard [RFC5246]:
{code}
Unless some other fatal alert has been transmitted, each party is
required to send a close_notify alert before closing the write side
of the connection. The other party MUST respond with a close_notify
alert of its own and close down the connection immediately,
discarding any pending writes. It is not required for the initiator
of the close to wait for the responding close_notify alert before
closing the read side of the connection.
{code}
[RFC5246] http://tools.ietf.org/html/rfc5246#section-7.2.1
PS. Is anyone seeing that behaviour on prod. boxes?
PPS. [~briang] can you take a look at it?
was:
We are seeing lots of RSTs on our edge boxes from HTTPS-enabled clients:
{code}
20:31:42.861752 IP client-ip.56898 > www.linkedin.com.https: Flags [S], seq
2989744105, win 65535, options [mss 1460,nop,wscale 4,nop,nop,TS val 1060727394
ecr 0,sackOK,eol], length 0
20:31:42.861760 IP www.linkedin.com.https > client-ip.56898: Flags [S.], seq
441157858, ack 2989744106, win 14480, options [mss 1460,sackOK,TS val 579955489
ecr 1060727394,nop,wscale 7], length 0
20:31:43.005735 IP client-ip.56898 > www.linkedin.com.https: Flags [.], ack 1,
win 8235, options [nop,nop,TS val 1060727553 ecr 579955489], length 0
20:31:43.012094 IP client-ip.56898 > www.linkedin.com.https: Flags [P.], seq
1:216, ack 1, win 8235, options [nop,nop,TS val 1060727555 ecr 579955489],
length 215
20:31:43.012115 IP www.linkedin.com.https > client-ip.56898: Flags [.], ack
216, win 122, options [nop,nop,TS val 579955639 ecr 1060727555], length 0
20:31:43.012400 IP www.linkedin.com.https > client-ip.56898: Flags [P.], seq
1:186, ack 216, win 122, options [nop,nop,TS val 579955640 ecr 1060727555],
length 185
20:31:43.157903 IP client-ip.56898 > www.linkedin.com.https: Flags [.], ack
186, win 8223, options [nop,nop,TS val 1060727703 ecr 579955640], length 0
20:31:43.157919 IP client-ip.56898 > www.linkedin.com.https: Flags [P.], seq
216:222, ack 186, win 8223, options [nop,nop,TS val 1060727704 ecr 579955640],
length 6
20:31:43.163988 IP client-ip.56898 > www.linkedin.com.https: Flags [P.], seq
222:307, ack 186, win 8223, options [nop,nop,TS val 1060727704 ecr 579955640],
length 85
20:31:43.164096 IP www.linkedin.com.https > client-ip.56898: Flags [.], ack
307, win 122, options [nop,nop,TS val 579955791 ecr 1060727704], length 0
20:31:43.164650 IP client-ip.56898 > www.linkedin.com.https: Flags [.], seq
307:1755, ack 186, win 8223, options [nop,nop,TS val 1060727705 ecr 579955640],
length 1448
20:31:43.164668 IP client-ip.56898 > www.linkedin.com.https: Flags [P.], seq
1755:2472, ack 186, win 8223, options [nop,nop,TS val 1060727705 ecr
579955640], length 717
20:31:43.164677 IP www.linkedin.com.https > client-ip.56898: Flags [.], ack
2472, win 167, options [nop,nop,TS val 579955792 ecr 1060727705], length 0
20:31:43.504019 IP www.linkedin.com.https > client-ip.56898: Flags [P.], seq
186:1167, ack 2472, win 167, options [nop,nop,TS val 579956131 ecr 1060727705],
length 981
20:31:43.504043 IP www.linkedin.com.https > client-ip.56898: Flags [P.], seq
1167:1764, ack 2472, win 167, options [nop,nop,TS val 579956131 ecr
1060727705], length 597
20:31:43.504101 IP www.linkedin.com.https > client-ip.56898: Flags [.], seq
1764:3212, ack 2472, win 167, options [nop,nop,TS val 579956131 ecr
1060727705], length 1448
20:31:43.504118 IP www.linkedin.com.https > client-ip.56898: Flags [.], seq
3212:4660, ack 2472, win 167, options [nop,nop,TS val 579956131 ecr
1060727705], length 1448
20:31:43.504150 IP www.linkedin.com.https > client-ip.56898: Flags [.], seq
4660:6108, ack 2472, win 167, options [nop,nop,TS val 579956132 ecr
1060727705], length 1448
20:31:43.743945 IP client-ip.56898 > www.linkedin.com.https: Flags [.], ack
1167, win 8162, options [nop,nop,TS val 1060728286 ecr 579956131], length 0
20:31:43.743966 IP www.linkedin.com.https > client-ip.56898: Flags [P.], seq
6108:6126, ack 2472, win 167, options [nop,nop,TS val 579956371 ecr
1060728286], length 18
20:31:43.743987 IP client-ip.56898 > www.linkedin.com.https: Flags [.], ack
4660, win 7944, options [nop,nop,TS val 1060728286 ecr 579956131], length 0
20:31:43.743994 IP client-ip.56898 > www.linkedin.com.https: Flags [.], ack
6108, win 8192, options [nop,nop,TS val 1060728286 ecr 579956132], length 0
20:31:43.895889 IP client-ip.56898 > www.linkedin.com.https: Flags [.], ack
6126, win 8190, options [nop,nop,TS val 1060728437 ecr 579956371], length 0
20:31:54.189610 IP www.linkedin.com.https > client-ip.56898: Flags [F.], seq
6126, ack 2472, win 167, options [nop,nop,TS val 579966817 ecr 1060728437],
length 0
20:31:54.305500 IP client-ip.56898 > www.linkedin.com.https: Flags [.], ack
6127, win 8192, options [nop,nop,TS val 1060738819 ecr 579966817], length 0
20:31:54.775695 IP client-ip.56898 > www.linkedin.com.https: Flags [P.], seq
2472:2541, ack 6127, win 8192, options [nop,nop,TS val 1060739277 ecr
579966817], length 69
20:31:54.775732 IP www.linkedin.com.https > client-ip.56898: Flags [R], seq
441163985, win 0, length 0
20:31:54.775739 IP client-ip.56898 > www.linkedin.com.https: Flags [F.], seq
2541, ack 6127, win 8192, options [nop,nop,TS val 1060739277 ecr 579966817],
length 0
20:31:54.775743 IP www.linkedin.com.https > client-ip.56898: Flags [R], seq
441163985, win 0, length 0
{code}
You can see that after {{proxy.config.http.keep_alive_no_activity_timeout_in}}
ATS is close(2)ing TLS connection without sending TLS {{close_notify}} alert
which results in client sending it (last part with {{[P.], seq 2472:2541}}) and
only then normally closing the connection ({{[F.], seq 2541}}), but because on
ATS side socket is already closed OS replies with RST (twice actually, one for
{{2472:2541}} and one for {{2541}}).
>From standard [RFC5246]:
{code}
Unless some other fatal alert has been transmitted, each party is
required to send a close_notify alert before closing the write side
of the connection. The other party MUST respond with a close_notify
alert of its own and close down the connection immediately,
discarding any pending writes. It is not required for the initiator
of the close to wait for the responding close_notify alert before
closing the read side of the connection.
{code}
[RFC5246] http://tools.ietf.org/html/rfc5246#section-7.2.1
PS. Is anyone seeing that behaviour on prod. boxes?
PPS. [~briang] can you take a look at it?
> ATS not closing TLS connection properly
> ---------------------------------------
>
> Key: TS-2941
> URL: https://issues.apache.org/jira/browse/TS-2941
> Project: Traffic Server
> Issue Type: Bug
> Components: SSL
> Reporter: Alexey Ivanov
>
> We are seeing lots of RSTs on our edge boxes from HTTPS-enabled clients:
> {code}
> 20:31:42.861752 IP client-ip.56898 > www.linkedin.com.https: Flags [S], seq
> 2989744105, win 65535, options [mss 1460,nop,wscale 4,nop,nop,TS val
> 1060727394 ecr 0,sackOK,eol], length 0
> 20:31:42.861760 IP www.linkedin.com.https > client-ip.56898: Flags [S.], seq
> 441157858, ack 2989744106, win 14480, options [mss 1460,sackOK,TS val
> 579955489 ecr 1060727394,nop,wscale 7], length 0
> 20:31:43.005735 IP client-ip.56898 > www.linkedin.com.https: Flags [.], ack
> 1, win 8235, options [nop,nop,TS val 1060727553 ecr 579955489], length 0
> 20:31:43.012094 IP client-ip.56898 > www.linkedin.com.https: Flags [P.], seq
> 1:216, ack 1, win 8235, options [nop,nop,TS val 1060727555 ecr 579955489],
> length 215
> 20:31:43.012115 IP www.linkedin.com.https > client-ip.56898: Flags [.], ack
> 216, win 122, options [nop,nop,TS val 579955639 ecr 1060727555], length 0
> 20:31:43.012400 IP www.linkedin.com.https > client-ip.56898: Flags [P.], seq
> 1:186, ack 216, win 122, options [nop,nop,TS val 579955640 ecr 1060727555],
> length 185
> 20:31:43.157903 IP client-ip.56898 > www.linkedin.com.https: Flags [.], ack
> 186, win 8223, options [nop,nop,TS val 1060727703 ecr 579955640], length 0
> 20:31:43.157919 IP client-ip.56898 > www.linkedin.com.https: Flags [P.], seq
> 216:222, ack 186, win 8223, options [nop,nop,TS val 1060727704 ecr
> 579955640], length 6
> 20:31:43.163988 IP client-ip.56898 > www.linkedin.com.https: Flags [P.], seq
> 222:307, ack 186, win 8223, options [nop,nop,TS val 1060727704 ecr
> 579955640], length 85
> 20:31:43.164096 IP www.linkedin.com.https > client-ip.56898: Flags [.], ack
> 307, win 122, options [nop,nop,TS val 579955791 ecr 1060727704], length 0
> 20:31:43.164650 IP client-ip.56898 > www.linkedin.com.https: Flags [.], seq
> 307:1755, ack 186, win 8223, options [nop,nop,TS val 1060727705 ecr
> 579955640], length 1448
> 20:31:43.164668 IP client-ip.56898 > www.linkedin.com.https: Flags [P.], seq
> 1755:2472, ack 186, win 8223, options [nop,nop,TS val 1060727705 ecr
> 579955640], length 717
> 20:31:43.164677 IP www.linkedin.com.https > client-ip.56898: Flags [.], ack
> 2472, win 167, options [nop,nop,TS val 579955792 ecr 1060727705], length 0
> 20:31:43.504019 IP www.linkedin.com.https > client-ip.56898: Flags [P.], seq
> 186:1167, ack 2472, win 167, options [nop,nop,TS val 579956131 ecr
> 1060727705], length 981
> 20:31:43.504043 IP www.linkedin.com.https > client-ip.56898: Flags [P.], seq
> 1167:1764, ack 2472, win 167, options [nop,nop,TS val 579956131 ecr
> 1060727705], length 597
> 20:31:43.504101 IP www.linkedin.com.https > client-ip.56898: Flags [.], seq
> 1764:3212, ack 2472, win 167, options [nop,nop,TS val 579956131 ecr
> 1060727705], length 1448
> 20:31:43.504118 IP www.linkedin.com.https > client-ip.56898: Flags [.], seq
> 3212:4660, ack 2472, win 167, options [nop,nop,TS val 579956131 ecr
> 1060727705], length 1448
> 20:31:43.504150 IP www.linkedin.com.https > client-ip.56898: Flags [.], seq
> 4660:6108, ack 2472, win 167, options [nop,nop,TS val 579956132 ecr
> 1060727705], length 1448
> 20:31:43.743945 IP client-ip.56898 > www.linkedin.com.https: Flags [.], ack
> 1167, win 8162, options [nop,nop,TS val 1060728286 ecr 579956131], length 0
> 20:31:43.743966 IP www.linkedin.com.https > client-ip.56898: Flags [P.], seq
> 6108:6126, ack 2472, win 167, options [nop,nop,TS val 579956371 ecr
> 1060728286], length 18
> 20:31:43.743987 IP client-ip.56898 > www.linkedin.com.https: Flags [.], ack
> 4660, win 7944, options [nop,nop,TS val 1060728286 ecr 579956131], length 0
> 20:31:43.743994 IP client-ip.56898 > www.linkedin.com.https: Flags [.], ack
> 6108, win 8192, options [nop,nop,TS val 1060728286 ecr 579956132], length 0
> 20:31:43.895889 IP client-ip.56898 > www.linkedin.com.https: Flags [.], ack
> 6126, win 8190, options [nop,nop,TS val 1060728437 ecr 579956371], length 0
> 20:31:54.189610 IP www.linkedin.com.https > client-ip.56898: Flags [F.], seq
> 6126, ack 2472, win 167, options [nop,nop,TS val 579966817 ecr 1060728437],
> length 0
> 20:31:54.305500 IP client-ip.56898 > www.linkedin.com.https: Flags [.], ack
> 6127, win 8192, options [nop,nop,TS val 1060738819 ecr 579966817], length 0
> 20:31:54.775695 IP client-ip.56898 > www.linkedin.com.https: Flags [P.], seq
> 2472:2541, ack 6127, win 8192, options [nop,nop,TS val 1060739277 ecr
> 579966817], length 69
> 20:31:54.775732 IP www.linkedin.com.https > client-ip.56898: Flags [R], seq
> 441163985, win 0, length 0
> 20:31:54.775739 IP client-ip.56898 > www.linkedin.com.https: Flags [F.], seq
> 2541, ack 6127, win 8192, options [nop,nop,TS val 1060739277 ecr 579966817],
> length 0
> 20:31:54.775743 IP www.linkedin.com.https > client-ip.56898: Flags [R], seq
> 441163985, win 0, length 0
> {code}
> You can see that after
> {{proxy.config.http.keep_alive_no_activity_timeout_in}} (10 secs) ATS is
> closes TCP connection without sending TLS {{close_notify}} alert which
> results in client sending it (last part with {{[P.], seq 2472:2541}}) and
> only then normally closing the connection ({{[F.], seq 2541}}), but because
> on ATS side socket is already closed OS replies with RST (twice actually, one
> for {{2472:2541}} and one for {{2541}}).
> From standard [RFC5246]:
> {code}
> Unless some other fatal alert has been transmitted, each party is
> required to send a close_notify alert before closing the write side
> of the connection. The other party MUST respond with a close_notify
> alert of its own and close down the connection immediately,
> discarding any pending writes. It is not required for the initiator
> of the close to wait for the responding close_notify alert before
> closing the read side of the connection.
> {code}
> [RFC5246] http://tools.ietf.org/html/rfc5246#section-7.2.1
> PS. Is anyone seeing that behaviour on prod. boxes?
> PPS. [~briang] can you take a look at it?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
