Yesterday I was trying to "pkg image-update" for about 3 hours and it
never completed downloading packages. It was making progress, just
slowly. I needed to shut down, then restarted the process this
morning. It spent about 40 minutes creating a plan before continuing
with the download.
In the process of looking into why it was so slow, I noticed a TCP
state that I've not seen before:
# netstat -f inet -P tcp | grep 72.5.123.21
192.168.1.101.58792 72.5.123.21.80 49640 0 49640 0 LAST_ACK
192.168.1.101.53478 72.5.123.21.80 49640 0 49640 0 LAST_ACK
192.168.1.101.53874 72.5.123.21.80 49640 0 49640 0 LAST_ACK
...
# netstat -f inet -P tcp | grep 72.5.123.21 | nawk '{print $NF}' |
sort | uniq -c
78 LAST_ACK
Are there pkg server or firewall issues that are preventing the
connections from being closed in a timely manner? I can't help but
think that this is causing significant problems on the server side if
connections take too long to fully close.
A bit earlier, I had about 15 connections in a CLOSE_WAIT state as well.
Snooping the network traffic I can see that HTTP connections are timing out.
# snoop -i /tmp/pkts | grep Timeout
58 0.00059 72.5.123.21 -> 192.168.1.101 HTTP HTTP/1.1 408 Request Timeout
251 0.00066 72.5.123.21 -> 192.168.1.101 HTTP HTTP/1.1 408 Request Timeout
262 0.00074 72.5.123.21 -> 192.168.1.101 HTTP HTTP/1.1 408 Request Timeout
309 0.00063 72.5.123.21 -> 192.168.1.101 HTTP HTTP/1.1 408 Request Timeout
320 0.00056 72.5.123.21 -> 192.168.1.101 HTTP HTTP/1.1 408 Request Timeout
# snoop -i /tmp/pkts -p 251 -v | grep port
IP: .... ..0. = not ECN capable transport
TCP: Source port = 80
TCP: Destination port = 64359
# TZ=GMT snoop -i /tmp/pkts -t a port 64359
1 12:01:4.14818 192.168.1.101 -> 72.5.123.21 HTTP C port=64359
2 12:01:4.21296 72.5.123.21 -> 192.168.1.101 HTTP R port=64359
3 12:01:4.21302 192.168.1.101 -> 72.5.123.21 HTTP C port=64359
4 12:01:4.21311 192.168.1.101 -> 72.5.123.21 HTTP POST
/dev/filelist/0 HTTP/1.1
5 12:01:4.28061 72.5.123.21 -> 192.168.1.101 HTTP R port=64359
6 12:01:4.28067 192.168.1.101 -> 72.5.123.21 HTTP
File-Name-0=01b4925bc26e9d9ac6a74a4e8dcc05034c2f80f6
7 12:01:4.34435 72.5.123.21 -> 192.168.1.101 HTTP R port=64359
8 12:01:4.34501 72.5.123.21 -> 192.168.1.101 HTTP HTTP/1.1 408
Request Timeout
9 12:01:4.34506 192.168.1.101 -> 72.5.123.21 HTTP C port=64359
10 12:01:4.34532 72.5.123.21 -> 192.168.1.101 HTTP R port=64359
11 12:01:4.34535 192.168.1.101 -> 72.5.123.21 HTTP C port=64359
12 12:01:36.63898 192.168.1.101 -> 72.5.123.21 HTTP C port=64359
13 12:01:41.27358 192.168.1.101 -> 72.5.123.21 HTTP C port=64359
Notice that it only takes about 60 ms for the Request Timeout (pkt 6
to 8) but takes another 37 seconds to close the connection.
--
Mike Gerdts
http://mgerdts.blogspot.com/
_______________________________________________
pkg-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
