Hi, (sorry for the long mail..)i am having a strange problem with a clients
server witch is sitting behind a obsd3.9 bridge hopefully somebody can shed
some light on it..
The server sits on a DMZ behind a bridge, there is a couple of other machines
behind there to and non of them are having any problem...that we know of.
There seem to be a problem with the connection trough the fw to the client mail
server.
This is what the customer gets in his maillog (Mdeamon):
"
Mon 2006-07-03 13:42:16: Accepting SMTP connection from [xx.xx.xx.xx : 64148]
Mon 2006-07-03 13:42:16: --> 220 foo.net ESMTP Mon, 03 Jul 2006 13:42:16 +0200
Mon 2006-07-03 13:45:26: Error reading from socket!
Mon 2006-07-03 13:45:26: Winsock Error 10054 Connection was reset by the other
side!
Mon 2006-07-03 13:45:26: SMTP session terminated (Bytes in/out: 0/73)
"
This happens a lot and from different hosts..and not all the time But after a
while the mail seems to get trough.
---------------------
>From pf.conf:
#aaa.aaa.aaa.aaa = client server
#Runtime Options
set timeout tcp.first 120
set timeout tcp.established 86400
set limit { states 10000, frags 8000}
set timeout { adaptive.start 3000, adaptive.end 6000 }
set optimization normal
set block-policy drop
set loginterface sis3
set skip on lo0
#Scrub
scrub on $WAN all reassemble tcp
no scrub on $WAN to aaa.aaa.aaa.aaa # Tryed different scrub and no scrub rules
but with the same result..
pass out quick on $WAN proto tcp all flags S/SA
pass in log (all) quick on sis3 inet proto tcp from any to aaa.aaa.aaa.aaa port
= smtp flags S/SA keep state label client_aaa
-----------------------
If i do a "tcpdump -e -n -ttt -vv -i pflog0" i get:
"
Jul 04 11:06:51.960142 rule 88/(match) [uid 0, pid 11358] pass out on sis3:
aaa.aaa.aaa.aaa.25 > bbb.bbb.bbb.bbb.36842: [|tcp] (DF) (ttl 128, id 2665, len
125)
Jul 04 11:06:51.967327 rule 88/(match) [uid 0, pid 11358] pass in on sis3:
bbb.bbb.bbb.bbb.36842 > aaa.aaa.aaa.aaa.25: [|tcp] (DF) [tos 0x20] (ttl 58, id
48736, len 52, bad cksum 0! differs by 696c
Jul 04 11:48:30.655896 rule 88/(match) [uid 0, pid 6857] pass in on sis3:
bbb.bbb.bbb.bbb.2916 > aaa.aaa.aaa.aaa.25: [|tcp] (DF) (ttl 120, id 41051, len
46, bad cksum 0! differs by d64)
Jul 04 11:48:30.656547 rule 88/(match) [uid 0, pid 6857] pass in on sis3:
bbb.bbb.bbb.bbb.2916 > aaa.aaa.aaa.aaa.25: [|tcp] (DF) (ttl 120, id 41052, len
40, bad cksum 0! differs by d69)
Jul 04 11:48:30.656714 rule 88/(match) [uid 0, pid 6857] pass out on sis3:
aaa.aaa.aaa.aaa.25 > bbb.bbb.bbb.bbb.2916: [|tcp] (DF) (ttl 128, id 25485, len
40)
Jul 04 11:48:30.657983 rule 88/(match) [uid 0, pid 6857] pass out on sis3:
aaa.aaa.aaa.aaa.25 > bbb.bbb.bbb.bbb.2916: [|tcp] (DF) (ttl 128, id 25486, len
66)
Jul 04 11:48:30.658131 rule 88/(match) [uid 0, pid 6857] pass out on sis3:
aaa.aaa.aaa.aaa.25 > bbb.bbb.bbb.bbb.2916: [|tcp] (DF) (ttl 128, id 25487, len
40)
Jul 04 11:48:30.678318 rule 88/(match) [uid 0, pid 6857] pass in on sis3:
bbb.bbb.bbb.bbb.2916 > aaa.aaa.aaa.aaa.25: [|tcp] (DF) (ttl 120, id 41053, len
40, bad cksum 0! differs by d68)
"
Bad checksum on incoming packets ?, could that be the problem?
If i do a tcpdump -n -vv -i sis3 host aaa.aaa.aaa.aaa i get:
"
12:05:58.164470 bbb.bbb.bbb.bbb.2860 > aaa.aaa.aaa.aaa.25: S [tcp sum ok]
2701663852:2701663852(0) win 64240 <mss 1460,nop,nop,sackOK> (DF) (ttl 120, id
15292, len 48)
12:05:58.164750 aaa.aaa.aaa.aaa.25 > bbb.bbb.bbb.bbb.2860: S [tcp sum ok]
3645997308:3645997308(0) ack 2701663853 win 17520 <mss 1460,nop,nop,sackOK>
(DF) (ttl 128, id 62899, len 48)
12:05:58.174420 bbb.bbb.bbb.bbb.2860 > aaa.aaa.aaa.aaa.25: . [tcp sum ok]
1:1(0) ack 1 win 64240 (DF) (ttl 120, id 15293, len 40)
12:05:58.199337 aaa.aaa.aaa.aaa.25 > bbb.bbb.bbb.bbb.2860: P 1:74(73) ack 1 win
17520 (DF) (ttl 128, id 62900, len 113)
12:05:58.210634 bbb.bbb.bbb.bbb.2860 > aaa.aaa.aaa.aaa.25: P [tcp sum ok]
1:13(12) ack 74 win 64167 (DF) (ttl 120, id 15302, len 52)
12:05:58.213567 aaa.aaa.aaa.aaa.25 > bbb.bbb.bbb.bbb.2860: P 74:127(53) ack 13
win 17508 (DF) (ttl 128, id 62901, len 93)
12:05:58.393083 bbb.bbb.bbb.bbb.2860 > aaa.aaa.aaa.aaa.25: . [tcp sum ok]
13:13(0) ack 127 win 64114 (DF) (ttl 120, id 15318, len 40)
12:05:58.393378 aaa.aaa.aaa.aaa.25 > bbb.bbb.bbb.bbb.2860: P 127:225(98) ack 13
win 17508 (DF) (ttl 128, id 62902, len 138)
12:05:58.421317 bbb.bbb.bbb.bbb.2860 > aaa.aaa.aaa.aaa.25: P [tcp sum ok]
13:44(31) ack 225 win 64016 (DF) (ttl 120, id 15320, len 71)
12:05:58.427743 aaa.aaa.aaa.aaa.25 > bbb.bbb.bbb.bbb.2860: P [tcp sum ok]
225:260(35) ack 44 win 17477 (DF) (ttl 128, id 62903, len 75)
12:05:58.446798 bbb.bbb.bbb.bbb.2860 > aaa.aaa.aaa.aaa.25: P [tcp sum ok]
44:83(39) ack 260 win 63981 (DF) (ttl 120, id 15321, len 79)
12:05:58.462241 aaa.aaa.aaa.aaa.25 > bbb.bbb.bbb.bbb.2860: P 260:308(48) ack 83
win 17438 (DF) (ttl 128, id 62904, len 88)
12:05:58.477187 bbb.bbb.bbb.bbb.2860 > aaa.aaa.aaa.aaa.25: P [tcp sum ok]
83:89(6) ack 308 win 63933 (DF) (ttl 120, id 15322, len 46)
12:05:58.481407 aaa.aaa.aaa.aaa.25 > bbb.bbb.bbb.bbb.2860: P [tcp sum ok]
308:348(40) ack 89 win 17432 (DF) (ttl 128, id 62905, len 80)
12:05:58.504634 bbb.bbb.bbb.bbb.2860 > aaa.aaa.aaa.aaa.25: . [tcp sum ok]
89:89(0) ack 348 win 63893 (DF) (ttl 120, id 15324, len 40)
12:05:59.066653 bbb.bbb.bbb.bbb.2860 > aaa.aaa.aaa.aaa.25: . 89:1549(1460) ack
348 win 63893 (DF) (ttl 120, id 15437, len 1500)
12:05:59.083808 bbb.bbb.bbb.bbb.2860 > aaa.aaa.aaa.aaa.25: . 1549:3009(1460)
ack 348 win 63893 (DF) (ttl 120, id 15438, len 1500)
12:05:59.084257 aaa.aaa.aaa.aaa.25 > bbb.bbb.bbb.bbb.2860: . [tcp sum ok]
348:348(0) ack 3009 win 17520 (DF) (ttl 128, id 62906, len 40)
12:05:59.100520 bbb.bbb.bbb.bbb.2860 > aaa.aaa.aaa.aaa.25: . 3009:4469(1460)
ack 348 win 63893 (DF) (ttl 120, id 15439, len 1500)
12:05:59.117717 bbb.bbb.bbb.bbb.2860 > aaa.aaa.aaa.aaa.25: . 4469:5929(1460)
ack 348 win 63893 (DF) (ttl 120, id 15440, len 1500)
12:05:59.118190 aaa.aaa.aaa.aaa.25 > bbb.bbb.bbb.bbb.2860: . [tcp sum ok]
348:348(0) ack 5929 win 17520 (DF) (ttl 128, id 62907, len 40)
12:05:59.134145 bbb.bbb.bbb.bbb.2860 > aaa.aaa.aaa.aaa.25: . 5929:7389(1460)
ack 348 win 63893 (DF) (ttl 120, id 15441, len 1500)
12:05:59.141687 bbb.bbb.bbb.bbb.2860 > aaa.aaa.aaa.aaa.25: P 7389:8281(892) ack
348 win 63893 (DF) (ttl 120, id 15442, len 932)
12:05:59.141996 aaa.aaa.aaa.aaa.25 > bbb.bbb.bbb.bbb.2860: . [tcp sum ok]
348:348(0) ack 8281 win 17520 (DF) (ttl 128, id 62908, len 40)
12:05:59.181316 bbb.bbb.bbb.bbb.2860 > aaa.aaa.aaa.aaa.25: . 8281:9741(1460)
ack 348 win 63893 (DF) (ttl 120, id 15445, len 1500)
12:05:59.197660 bbb.bbb.bbb.bbb.2860 > aaa.aaa.aaa.aaa.25: . 9741:11201(1460)
ack 348 win 63893 (DF) (ttl 120, id 15446, len 1500)
12:05:59.198096 aaa.aaa.aaa.aaa.25 > bbb.bbb.bbb.bbb.2860: . [tcp sum ok]
348:348(0) ack 11201 win 17520 (DF) (ttl 128, id 62909, len 40)
12:05:59.214779 bbb.bbb.bbb.bbb.2860 > aaa.aaa.aaa.aaa.25: . 11201:12661(1460)
ack 348 win 63893 (DF) (ttl 120, id 15447, len 1500)
12:05:59.231917 bbb.bbb.bbb.bbb.2860 > aaa.aaa.aaa.aaa.25: . 12661:14121(1460)
ack 348 win 63893 (DF) (ttl 120, id 15448, len 1500)"
If i check with etereal it seems like there is a lot of tcp dup ack and tcp
retransmission..
An thoughts ?
Mvh
Daniel Rapp
