Hi, I have a weird issue with my iscsi setup that I can't figure out.. I have the feeling this has something to do with the networking, but i can't put my finger on it. Any help would be much appreciated.
My setup is as follows: - 2 servers, both with a bridge on top of a bonded interface. - target running on one of the servers with a dedicated IP (managed with pacemaker for failover). - Centos 5.6 with the latest updates installed. Discovery works perfectly fine on both machines and when I login either one of the two on the target, everything is fine as well. However, as soon as I login the second machine, i start getting "Kernel reported iSCSI connection 1:0 error (1011) state (3)" on both machines every 30s orso. My noop interval.. The target says: session ..... not found? I have tried with both tgtd and ietd, didn't make a difference. a few tcp dumps revealed the connections to the target are made either through the loopback interface if the target is local, or over the bridge interface if it is remote. This is the traffic i see on the appropriate NIC with one initiator logged in: 16:16:54.683206 IP en1-r1-srv3.48924 > en1-r1-tgt.iscsi-target: P 785:833(48) ack 693 win 63 <nop,nop,timestamp 3863324 3684368> 16:16:54.683336 IP en1-r1-tgt.iscsi-target > en1-r1-srv3.48924: P 693:741(48) ack 833 win 62 <nop,nop,timestamp 3714618 3863324> 16:16:54.683423 IP en1-r1-srv3.48924 > en1-r1-tgt.iscsi-target: . ack 741 win 63 <nop,nop,timestamp 3863324 3714618> 16:17:24.683169 IP en1-r1-srv3.48924 > en1-r1-tgt.iscsi-target: P 833:881(48) ack 741 win 63 <nop,nop,timestamp 3893324 3714618> 16:17:24.683287 IP en1-r1-tgt.iscsi-target > en1-r1-srv3.48924: P 741:789(48) ack 881 win 62 <nop,nop,timestamp 3744618 3893324> 16:17:24.683413 IP en1-r1-srv3.48924 > en1-r1-tgt.iscsi-target: . ack 789 win 63 <nop,nop,timestamp 3893324 3744618> 16:17:54.683182 IP en1-r1-srv3.48924 > en1-r1-tgt.iscsi-target: P 881:929(48) ack 789 win 63 <nop,nop,timestamp 3923324 3744618> 16:17:54.683298 IP en1-r1-tgt.iscsi-target > en1-r1-srv3.48924: P 789:837(48) ack 929 win 62 <nop,nop,timestamp 3774618 3923324> 16:17:54.683424 IP en1-r1-srv3.48924 > en1-r1-tgt.iscsi-target: . ack 837 win 63 <nop,nop,timestamp 3923324 3774618> As soon as the other initiator is logged in, this happens: 16:20:40.809550 IP en1-r1-tgt.iscsi-target > en1-r1-srv3.48924: F 1077:1077(0) ack 1169 win 62 <nop,nop,timestamp 3940744 4073328> 16:20:40.852521 IP en1-r1-srv3.48924 > en1-r1-tgt.iscsi-target: . ack 1078 win 63 <nop,nop,timestamp 4089493 3940744> 16:20:54.687962 IP en1-r1-srv3.48924 > en1-r1-tgt.iscsi-target: P 1169:1217(48) ack 1078 win 63 <nop,nop,timestamp 4103328 3940744> 16:20:54.688015 IP en1-r1-tgt.iscsi-target > en1-r1-srv3.48924: R 3327718642:3327718642(0) win 0 16:20:57.455787 IP en1-r1-srv3.48051 > en1-r1-tgt.iscsi-target: S 3654326285:3654326285(0) win 5840 <mss 1460,sackOK,timestamp 4106086 0,nop,wscale 7> 16:20:57.455859 IP en1-r1-tgt.iscsi-target > en1-r1-srv3.48051: S 3626155967:3626155967(0) ack 3654326286 win 5792 <mss 1460,sackOK,timestamp 3957390 4106086,nop,wscale 7> 16:20:57.459047 IP en1-r1-srv3.48051 > en1-r1-tgt.iscsi-target: . ack 1 win 46 <nop,nop,timestamp 4106096 3957390> 16:20:57.700846 IP en1-r1-srv3.48051 > en1-r1-tgt.iscsi-target: P 1:49(48) ack 1 win 46 <nop,nop,timestamp 4106337 3957390> 16:20:57.700891 IP en1-r1-tgt.iscsi-target > en1-r1-srv3.48051: . ack 49 win 46 <nop,nop,timestamp 3957635 4106337> 16:20:57.761029 IP en1-r1-srv3.48051 > en1-r1-tgt.iscsi-target: P 49:205(156) ack 1 win 46 <nop,nop,timestamp 4106337 3957390> 16:20:57.761087 IP en1-r1-tgt.iscsi-target > en1-r1-srv3.48051: . ack 205 win 54 <nop,nop,timestamp 3957696 4106337> 16:20:57.761745 IP en1-r1-tgt.iscsi-target > en1-r1-srv3.48051: P 1:89(88) ack 205 win 54 <nop,nop,timestamp 3957696 4106337> 16:20:57.761858 IP en1-r1-srv3.48051 > en1-r1-tgt.iscsi-target: . ack 89 win 46 <nop,nop,timestamp 4106402 3957696> 16:20:58.011342 IP en1-r1-srv3.48051 > en1-r1-tgt.iscsi-target: P 205:253(48) ack 89 win 46 <nop,nop,timestamp 4106652 3957696> 16:20:58.011381 IP en1-r1-srv3.48051 > en1-r1-tgt.iscsi-target: P 253:265(12) ack 89 win 46 <nop,nop,timestamp 4106652 3957696> 16:20:58.011499 IP en1-r1-tgt.iscsi-target > en1-r1-srv3.48051: . ack 265 win 54 <nop,nop,timestamp 3957946 4106652> 16:20:58.011578 IP en1-r1-tgt.iscsi-target > en1-r1-srv3.48051: P 89:233(144) ack 265 win 54 <nop,nop,timestamp 3957946 4106652> 16:20:58.011640 IP en1-r1-srv3.48051 > en1-r1-tgt.iscsi-target: . ack 233 win 54 <nop,nop,timestamp 4106652 3957946> 16:20:58.261524 IP en1-r1-srv3.48051 > en1-r1-tgt.iscsi-target: P 265:313(48) ack 233 win 54 <nop,nop,timestamp 4106902 3957946> 16:20:58.261565 IP en1-r1-srv3.48051 > en1-r1-tgt.iscsi-target: P 313:425(112) ack 233 win 54 <nop,nop,timestamp 4106902 3957946> 16:20:58.261682 IP en1-r1-tgt.iscsi-target > en1-r1-srv3.48051: . ack 425 win 54 <nop,nop,timestamp 3958196 4106902> 16:20:58.262305 IP en1-r1-tgt.iscsi-target > en1-r1-srv3.48051: P 233:337(104) ack 425 win 54 <nop,nop,timestamp 3958197 4106902> 16:20:58.262419 IP en1-r1-srv3.48051 > en1-r1-tgt.iscsi-target: . ack 337 win 54 <nop,nop,timestamp 4106903 3958197> 16:20:58.512373 IP en1-r1-srv3.48051 > en1-r1-tgt.iscsi-target: P 425:473(48) ack 337 win 54 <nop,nop,timestamp 4107153 3958197> 16:20:58.512412 IP en1-r1-srv3.48051 > en1-r1-tgt.iscsi-target: P 473:785(312) ack 337 win 54 <nop,nop,timestamp 4107153 3958197> 16:20:58.512529 IP en1-r1-tgt.iscsi-target > en1-r1-srv3.48051: . ack 785 win 62 <nop,nop,timestamp 3958447 4107153> 16:20:58.512647 IP en1-r1-tgt.iscsi-target > en1-r1-srv3.48051: P 337:661(324) ack 785 win 62 <nop,nop,timestamp 3958447 4107153> 16:20:58.512730 IP en1-r1-srv3.48051 > en1-r1-tgt.iscsi-target: . ack 661 win 63 <nop,nop,timestamp 4107153 3958447> Not sure what happens here, but for some reason the connections/ sessions are now dropped with every ping/noop... Could it be that having the local connection running over the LO device is causing this? Or am I'm missing something obvious. I tried bonding the initiator to the bridge interface, but that left me unable to connect at all if the target was local.. Does anybody have a clue where I should be looking? Because I'm pretty much at a loss here.. Thanks Arthur -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
