Hi, we are using iSCSI targets as swap devices in our Diskless NFS-Root servers - since Swap over NFS does not work properly at the moment.
The target server is a SUN Unified Storage 7310. The initiators are Debian 6.0.4 based. In our roll out we realized now that after a certain amount of targets the discovery process does not work anymore. We already increased discovery.sendtargets.iscsi.MaxRecvDataSegmentLength to 262144 from the default 32768 - but that did not help. Here the debug output from the discovery command: ---- root@disklesstest.dc1:/etc/iscsi# iscsiadm -d 4 -m discovery -t sendtargets -p 10.20.56.2 -l iscsiadm: ip 10.20.56.2, port 3260, tgpt -1 iscsiadm: Max file limits 1024 1024 iscsiadm: sysfs_attr_get_value: open '/class/iscsi_transport/ iser'/'handle' iscsiadm: sysfs_attr_get_value: new uncached attribute '/sys/class/ iscsi_transport/iser/handle' iscsiadm: sysfs_attr_get_value: add to cache '/sys/class/ iscsi_transport/iser/handle' iscsiadm: sysfs_attr_get_value: cache '/sys/class/iscsi_transport/iser/ handle' with attribute value '18446744072104264112' iscsiadm: sysfs_attr_get_value: open '/class/iscsi_transport/ iser'/'caps' iscsiadm: sysfs_attr_get_value: new uncached attribute '/sys/class/ iscsi_transport/iser/caps' iscsiadm: sysfs_attr_get_value: add to cache '/sys/class/ iscsi_transport/iser/caps' iscsiadm: sysfs_attr_get_value: cache '/sys/class/iscsi_transport/iser/ caps' with attribute value '0x9' iscsiadm: sysfs_attr_get_value: open '/class/iscsi_transport/ tcp'/'handle' iscsiadm: sysfs_attr_get_value: new uncached attribute '/sys/class/ iscsi_transport/tcp/handle' iscsiadm: sysfs_attr_get_value: add to cache '/sys/class/ iscsi_transport/tcp/handle' iscsiadm: sysfs_attr_get_value: cache '/sys/class/iscsi_transport/tcp/ handle' with attribute value '18446744072103851664' iscsiadm: sysfs_attr_get_value: open '/class/iscsi_transport/ tcp'/'caps' iscsiadm: sysfs_attr_get_value: new uncached attribute '/sys/class/ iscsi_transport/tcp/caps' iscsiadm: sysfs_attr_get_value: add to cache '/sys/class/ iscsi_transport/tcp/caps' iscsiadm: sysfs_attr_get_value: cache '/sys/class/iscsi_transport/tcp/ caps' with attribute value '0x39' iscsiadm: sysfs_attr_get_value: open '/class/iscsi_transport/ iser'/'handle' iscsiadm: sysfs_attr_get_value: found in cache '/class/iscsi_transport/ iser/handle' iscsiadm: sysfs_attr_get_value: open '/class/iscsi_transport/ iser'/'caps' iscsiadm: sysfs_attr_get_value: found in cache '/class/iscsi_transport/ iser/caps' iscsiadm: sysfs_attr_get_value: open '/class/iscsi_transport/ tcp'/'handle' iscsiadm: sysfs_attr_get_value: found in cache '/class/iscsi_transport/ tcp/handle' iscsiadm: sysfs_attr_get_value: open '/class/iscsi_transport/ tcp'/'caps' iscsiadm: sysfs_attr_get_value: found in cache '/class/iscsi_transport/ tcp/caps' iscsiadm: starting sendtargets discovery, address 10.20.56.2:3260, iscsiadm: connecting to 10.20.56.2:3260 iscsiadm: connected local port 57632 to 10.20.56.2:3260 iscsiadm: connected to discovery address 10.20.56.2 iscsiadm: login response status 0000 iscsiadm: discovery session to 10.20.56.2:3260 received unexpected opcode 0x3f iscsiadm: disconnecting conn 0x258cde0, fd 3 iscsiadm: connecting to 10.20.56.2:3260 iscsiadm: connected local port 57633 to 10.20.56.2:3260 iscsiadm: connected to discovery address 10.20.56.2 iscsiadm: login response status 0211 iscsiadm: discovery login to 10.20.56.2 rejected: initiator error (02/0b), non-retryable, giving up iscsiadm: disconnecting conn 0x258cde0, fd 3 iscsiadm: Could not perform SendTargets discovery. root@disklesstest.dc1:/etc/iscsi# ---- Using -d 5 we see that after some 100 targets the output stops: ---- <cut> iscsiadm: > TargetName=iqn.2010-06.de.boerse-go:mongo02 iscsiadm: > TargetAddress=10.20.56.2:3260,2 iscsiadm: > TargetAddress=10.20.57.2:3260,3 iscsiadm: > TargetAddress=10.20.81.2:3260,4 iscsiadm: > TargetName=iqn.2010-06.de.boerse-go:mongo03 iscsiadm: > TargetAddress=10.20.56.2:3260,2 iscsiadm: > TargetAddress=10.20.57.2:3260,3 iscsiadm: > TargetAddress=10.20.81.2:3260,4 iscsiadm: > TargetName=iqn.2010-06.de.boerse-go:monitor02 iscsiadm: > TargetAddres iscsiadm: discovery session to 10.20.56.2:3260 received text response, 8192 data bytes, ttt 0x1000000, final 0x0 iscsiadm: sending text pdu with CmdSN 2, exp_statsn 2 iscsiadm: wrote 48 bytes of PDU header iscsiadm: discovery process 10.20.56.2:3260 polling fd 3, timeout in 30.000000 seconds iscsiadm: read 48 bytes of PDU header iscsiadm: read 48 PDU header bytes, opcode 0x3f, dlength 48, data 0x7f3334008010, max 262144 iscsiadm: read 48 bytes of PDU data iscsiadm: discovery session to 10.20.56.2:3260 received unexpected opcode 0x3f iscsiadm: disconnecting conn 0x887de0, fd 3 iscsiadm: connecting to 10.20.56.2:3260 iscsiadm: connected local port 57637 to 10.20.56.2:3260 <cut> ---- We now solved our problem using ---- root@disklesstest.dc1:/etc/iscsi# iscsiadm -m node -T iqn. 2010-06.de.boerse-go:disklesstest -p 10.20.56.2:3260 -o new New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 10.20.56.2,3260,-1 iqn.2010-06.de.boerse-go:disklesstest] added root@disklesstest.dc1:/etc/iscsi# iscsiadm -m node -T iqn. 2010-06.de.boerse-go:disklesstest -p 10.20.56.2:3260 -l Logging in to [iface: default, target: iqn.2010-06.de.boerse- go:disklesstest, portal: 10.20.56.2,3260] Login to [iface: default, target: iqn.2010-06.de.boerse- go:disklesstest, portal: 10.20.56.2,3260]: successful root@disklesstest.dc1:/etc/iscsi# ---- But we believe that the sendtargets discovery should work as well :-( Best regards Rainer -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
