> Hello Robert,
>
> On 14/02/2011, at 16:36, Robert Olson wrote:
>
> > The problem we're seeing is that for a particular test script on the
> > client side, one of the exchanges is failing.
> >
> > Looking at packet traces, I see the client sending a complete
> > request to cherokee. Cherokee sends the request to the compute
> > server, but it appears to be truncating the request one packet shy
> > of finishing it. the compute server then reports a bad parse in
> > response.
> >
> > The problem initially showed up fairly reliably only when both front
> > ends were running. If I killed wackamole on one of them (pushing
> > both IPs over to a single server) the problem vanished.
>
> Did it perform a clean 'three way' close sequence (FIN, FIN+ACK, ACK)?
> The last package might be lost if a RST were sent while the connection
> is being closed.
>
> Cheers!
Here's the last bit of one of the failed exchanges. ml-mds is the frontend, oak
is the compute server. It sure looks like the frontend just decided to close
down the connection; its last packet appears to be the FIN. I unfortunately
don't appear to have saved any subsequent packets. I'll be trying today to
replicate the problem again and get strace output on the frontends as well as
detailed packet traces all around.
Thanks,
--bob
10:48:00.188439 IP oak.mcs.anl.gov.5104 > ml-mds.mcs.anl.gov.42694: . ack 12625
win 243 <nop,nop,timestamp 1364352976 1359990193>
0x0000: 4500 0034 68ac 4000 4006 c142 c005 c860 E..4h.@[email protected]...`
0x0010: c005 c869 13f0 a6c6 7649 376b 6cd7 3bb7 ...i....vI7kl.;.
0x0020: 8010 00f3 8417 0000 0101 080a 5152 5fd0 ............QR_.
0x0030: 510f cdb1 Q...
10:48:00.188591 IP ml-mds.mcs.anl.gov.42694 > oak.mcs.anl.gov.5104: .
12625:14073(1448) ack 1 win 46 <nop,nop,timestamp 1359990193 1364352976>
0x0000: 4500 05dc c7e2 4000 4006 5c64 c005 c869 E.....@.@.\d...i
0x0010: c005 c860 a6c6 13f0 6cd7 3bb7 7649 376b ...`....l.;.vI7k
0x0020: 8010 002e 0d8c 0000 0101 080a 510f cdb1 ............Q...
0x0030: 5152 5fd0 3938 2e35 2e70 6567 2e34 3834 QR_.98.5.peg.484
0x0040: 2530 412b 2b2d 2b66 6967 2537 4332 3039 %0A++-+fig%7C209
0x0050: 382e 352e 7065 672e 3438 3525 3041 2b2b 8.5.peg.485%0A++
0x0060: 2d2b 6669 6725 3743 3230 3938 2e35 2e70 -+fig%7C2098.5.p
0x0070: 6567 2e34 3836 2530 412b 2b2d 2b66 6967 eg.486%0A++-+fig
0x0080: 2537 4332 3039 382e 352e 7065 672e 3438 %7C2098.5.peg.48
0x0090: 3725 3041 2b2b 2d2b 6669 6725 3743 3230 7%0A++-+fig%7C20
0x00a0: 3938 2e35 2e70 6567 2e34 3838 2530 412b 98.5.peg.488%0A+
0x00b0: 2b2d 2b66 6967 2537 4332 3039 382e 352e +-+fig%7C2098.5.
0x00c0: 7065 672e 3438 3925 3041 2b2b 2d2b 6669 peg.489%0A++-+fi
0x00d0: 6725 3743 3230 3938 2e35 2e70 6567 2e34 g%7C2098.5.peg.4
0x00e0: 3925 3041 2b2b 2d2b 6669 6725 3743 3230 9%0A++-+fig%7C20
0x00f0: 3938 2e35 2e70 6567 2e34 3930 2530 412b 98.5.peg.490%0A+
0x0100: 2b2d 2b66 6967 2537 4332 3039 382e 352e +-+fig%7C2098.5.
0x0110: 7065 672e 3439 3125 3041 2b2b 2d2b 6669 peg.491%0A++-+fi
0x0120: 6725 3743 3230 3938 2e35 2e70 6567 2e34 g%7C2098.5.peg.4
0x0130: 3932 2530 412b 2b2d 2b66 6967 2537 4332 92%0A++-+fig%7C2
0x0140: 3039 382e 352e 7065 672e 3439 3325 3041 098.5.peg.493%0A
0x0150: 2b2b 2d2b 6669 6725 3743 3230 3938 2e35 ++-+fig%7C2098.5
0x0160: 2e70 6567 2e34 3934 2530 412b 2b2d 2b66 .peg.494%0A++-+f
0x0170: 6967 2537 4332 3039 382e 352e 7065 672e ig%7C2098.5.peg.
0x0180: 3439 49
10:48:00.188595 IP ml-mds.mcs.anl.gov.42694 > oak.mcs.anl.gov.5104: P
14073:15521(1448) ack 1 win 46 <nop,nop,timestamp 1359990193 1364352976>
0x0000: 4500 05dc c7e3 4000 4006 5c63 c005 c869 E.....@.@.\c...i
0x0010: c005 c860 a6c6 13f0 6cd7 415f 7649 376b ...`....l.A_vI7k
0x0020: 8018 002e 6f84 0000 0101 080a 510f cdb1 ....o.......Q...
0x0030: 5152 5fd0 4332 3039 382e 352e 7065 672e QR_.C2098.5.peg.
0x0040: 3533 3225 3041 2b2b 2d2b 6669 6725 3743 532%0A++-+fig%7C
0x0050: 3230 3938 2e35 2e70 6567 2e35 3333 2530 2098.5.peg.533%0
0x0060: 412b 2b2d 2b66 6967 2537 4332 3039 382e A++-+fig%7C2098.
0x0070: 352e 7065 672e 3533 3425 3041 2b2b 2d2b 5.peg.534%0A++-+
0x0080: 6669 6725 3743 3230 3938 2e35 2e70 6567 fig%7C2098.5.peg
0x0090: 2e35 3335 2530 412b 2b2d 2b66 6967 2537 .535%0A++-+fig%7
0x00a0: 4332 3039 382e 352e 7065 672e 3533 3625 C2098.5.peg.536%
0x00b0: 3041 2b2b 2d2b 6669 6725 3743 3230 3938 0A++-+fig%7C2098
0x00c0: 2e35 2e70 6567 2e35 3337 2530 412b 2b2d .5.peg.537%0A++-
0x00d0: 2b66 6967 2537 4332 3039 382e 352e 7065 +fig%7C2098.5.pe
0x00e0: 672e 3533 3825 3041 2b2b 2d2b 6669 6725 g.538%0A++-+fig%
0x00f0: 3743 3230 3938 2e35 2e70 6567 2e35 3339 7C2098.5.peg.539
0x0100: 2530 412b 2b2d 2b66 6967 2537 4332 3039 %0A++-+fig%7C209
0x0110: 382e 352e 7065 672e 3534 2530 412b 2b2d 8.5.peg.54%0A++-
0x0120: 2b66 6967 2537 4332 3039 382e 352e 7065 +fig%7C2098.5.pe
0x0130: 672e 3534 3025 3041 2b2b 2d2b 6669 6725 g.540%0A++-+fig%
0x0140: 3743 3230 3938 2e35 2e70 6567 2e35 3431 7C2098.5.peg.541
0x0150: 2530 412b 2b2d 2b66 6967 2537 4332 3039 %0A++-+fig%7C209
0x0160: 382e 352e 7065 672e 3534 3225 3041 2b2b 8.5.peg.542%0A++
0x0170: 2d2b 6669 6725 3743 3230 3938 2e35 2e70 -+fig%7C2098.5.p
0x0180: 6567 eg
10:48:00.188684 IP oak.mcs.anl.gov.5104 > ml-mds.mcs.anl.gov.42694: . ack 14073
win 266 <nop,nop,timestamp 1364352976 1359990193>
0x0000: 4500 0034 68ad 4000 4006 c141 c005 c860 E..4h.@[email protected]...`
0x0010: c005 c869 13f0 a6c6 7649 376b 6cd7 415f ...i....vI7kl.A_
0x0020: 8010 010a 7e58 0000 0101 080a 5152 5fd0 ....~X......QR_.
0x0030: 510f cdb1 Q...
10:48:00.188691 IP oak.mcs.anl.gov.5104 > ml-mds.mcs.anl.gov.42694: . ack 15521
win 288 <nop,nop,timestamp 1364352976 1359990193>
0x0000: 4500 0034 68ae 4000 4006 c140 c005 c860 E..4h.@.@..@...`
0x0010: c005 c869 13f0 a6c6 7649 376b 6cd7 4707 ...i....vI7kl.G.
0x0020: 8010 0120 789a 0000 0101 080a 5152 5fd0 ....x.......QR_.
0x0030: 510f cdb1 Q...
10:48:17.186797 IP ml-mds.mcs.anl.gov.42694 > oak.mcs.anl.gov.5104: F
15521:15521(0) ack 1 win 46 <nop,nop,timestamp 1360007195 1364352976>
0x0000: 4500 0034 c7e4 4000 4006 620a c005 c869 E..4..@[email protected]
0x0010: c005 c860 a6c6 13f0 6cd7 4707 7649 376b ...`....l.G.vI7k
0x0020: 8011 002e 3721 0000 0101 080a 5110 101b ....7!......Q...
0x0030: 5152 5fd0 QR_.
10:48:17.189495 IP oak.mcs.anl.gov.5104 > ml-mds.mcs.anl.gov.42694: P
1:337(336) ack 15522 win 288 <nop,nop,timestamp 1364369979 1360007195>
0x0000: 4500 0184 68af 4000 4006 bfef c005 c860 E...h.@.@......`
0x0010: c005 c869 13f0 a6c6 7649 376b 6cd7 4708 ...i....vI7kl.G.
0x0020: 8018 0120 124c 0000 0101 080a 5152 a23b .....L......QR.;
0x0030: 5110 101b 0106 0001 012e 0200 5374 6174 Q...........Stat
0x0040: 7573 3a20 3530 3020 4572 726f 7220 666f us:.500.Error.fo
0x0050: 726d 6174 7469 6e67 2070 6172 616d 6574 rmatting.paramet
0x0060: 6572 732e 0d0a 436f 6e74 656e 742d 5479 ers...Content-Ty
0x0070: 7065 3a20 7465 7874 2f70 6c61 696e 3b20 pe:.text/plain;.
0x0080: 6368 6172 7365 743d 4953 4f2d 3838 3539 charset=ISO-8859
0x0090: 2d31 0d0a 0d0a 5941 4d4c 2045 7272 6f72 -1....YAML.Error
0x00a0: 3a20 5374 7265 616d 2064 6f65 7320 6e6f :.Stream.does.no
0x00b0: 7420 656e 6420 7769 7468 206e 6577 6c69 t.end.with.newli
0x00c0: 6e65 2063 6861 7261 6374 6572 0a20 2020 ne.character....
0x00d0: 436f 6465 3a20 5941 4d4c 5f50 4152 5345 Code:.YAML_PARSE
0x00e0: 5f45 5252 5f4e 4f5f 4649 4e41 4c5f 4e45 _ERR_NO_FINAL_NE
0x00f0: 574c 494e 450a 2020 204c 696e 653a 2030 WLINE....Line:.0
0x0100: 0a20 2020 446f 6375 6d65 6e74 3a20 300a ....Document:.0.
0x0110: 2061 7420 2f76 6f6c 2f72 6173 742d 6263 .at./vol/rast-bc
0x0120: 722f 3230 3130 2d31 3132 342f 6c69 6e75 r/2010-1124/linu
0x0130: 782d 7268 656c 352d 7838 365f 3634 2f6c x-rhel5-x86_64/l
0x0140: 6962 2f70 6572 6c35 2f73 6974 655f 7065 ib/perl5/site_pe
0x0150: 726c 2f35 2e31 322e 322f 5941 4d4c 2e70 rl/5.12.2/YAML.p
0x0160: 6d20 6c69 6e65 2033 360a 0000 0106 0001 m.line.36.......
0x0170: 0000 0000 0103 0001 0008 0000 0000 0000 ................
0x0180: 0000
_______________________________________________
Cherokee mailing list
[email protected]
http://lists.octality.com/listinfo/cherokee
