Re: Best way to create multiple TCP flows on 10 Gbps link
I had applied the tuning for my 10g link but didn't see much impact. Actually for me tcp is already line rate with 2/3 threads but iscsi/fio read is around 5.5gbps only - with 3/4 fio threads. Perhaps the bottleneck is somewhere else... Thanks! Sent from my iPhone On Aug 27, 2014, at 8:25 AM, Mark Lehrer m...@knm.org wrote: On Tue, 26 Aug 2014 13:05:11 -0700 Learner learner.st...@gmail.com wrote: How many iscsi and underlying top sessions are u using? If multiple, pls check if all to sessions are being used. Btw, what tuning did u perform to fix Tcp BDP issue? I'm just doing netcat tests to/from /dev/shm at the moment. I wouldn't consider it fixed necessarily, but the info from this link was useful: http://www.kehlet.cx/articles/99.html Thanks, Mark -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi. For more options, visit https://groups.google.com/d/optout.
Re: Best way to create multiple TCP flows on 10 Gbps link
Hi Mike, Thanks for suggestions I think you meant, echo 1 /sys/block/sdX/device/delete I don't see /sys/block/sdX/device/remove in my setup. How do following FIO options look? [default] rw=read size=4g bs=1m ioengine=libaio direct=1 numjobs=1 filename=/dev/sda runtime=360 iodepth=256 Thanks for your time! On Tue, Aug 26, 2014 at 4:49 PM, Michael Christie micha...@cs.wisc.edu wrote: On Aug 26, 2014, at 3:11 PM, Learner learner.st...@gmail.com wrote: Another related observation and some questions; I am using open iscsi on init with IET on trgt over a single 10gbps link There are three ip aliases on each side I have 3 ramdisks exported by IET to init I do iscsi login 3 times, once using each underlying ip address and notice that each iscsi session sees all 3 disks. Is it possible to restrict such that each init only sees one separate disk? There is no iscsi initiator or target setting for this. The default is to show all paths (each /dev/sdx is a path to the same device).. You would have to manually delete some paths by doing echo 1 /sys/block/sdX/device/remove When I run fio on each mounted disk, I see that only two underlying tcp sessions are being used - that limits the perf. Any ideas on how to overcome this? How are you matching sessions with devices? It should just be a matter of running fio on the right devices. If you run: iscsiadm -m session -P 3 you can see how the sdXs match up with sessions/connections. If you run fio to a /dev/sdX from each session, you should be seeing IO to all 3 sessions. Thanks! Sent from my iPhone On Aug 26, 2014, at 12:53 PM, Mark Lehrer m...@knm.org wrote: On Tue, 26 Aug 2014 08:58:46 -0400 Alvin Starr al...@iplink.net wrote: I am trying to achieve10Gbps in my single initiator/single target env. (open-iscsi and IET) On a semi-related note, are there any good guides out there to tuning Linux for maximum single-socket performance? On my 40 gigabit You are likely getting hit by the bandwidth-delay product. Take a look at http://en.wikipedia.org/wiki/Bandwidth-delay_product and http://www.kehlet.cx/articles/99.html Thanks that helped get my netcat transfer up over 500MB/sec using IPoIB. Unfortunately that is still only about 10% of the available bandwidth. I'll keep on tweaking and see how far I can take it. Thanks, Mark -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi. For more options, visit https://groups.google.com/d/optout.
Re: Best way to create multiple TCP flows on 10 Gbps link
I am monitoring with netstat -a...looking at sendq and recvq there for the three iscsi/tcp sessions. Also checked with tcpdump. Thanks! Sent from my iPhone On Aug 26, 2014, at 9:46 PM, Michael Christie micha...@cs.wisc.edu wrote: On Aug 26, 2014, at 6:49 PM, Michael Christie micha...@cs.wisc.edu wrote: On Aug 26, 2014, at 3:11 PM, Learner learner.st...@gmail.com wrote: Another related observation and some questions; I am using open iscsi on init with IET on trgt over a single 10gbps link There are three ip aliases on each side I have 3 ramdisks exported by IET to init I do iscsi login 3 times, once using each underlying ip address and notice that each iscsi session sees all 3 disks. Is it possible to restrict such that each init only sees one separate disk? There is no iscsi initiator or target setting for this. The default is to show all paths (each /dev/sdx is a path to the same device).. You would have to manually delete some paths by doing echo 1 /sys/block/sdX/device/remove When I run fio on each mounted disk, I see that only two underlying tcp sessions are being used - that limits the perf. Any ideas on how to overcome this? How are you matching sessions with devices? It should just be a matter of running fio on the right devices. If you run: iscsiadm -m session -P 3 you can see how the sdXs match up with sessions/connections. If you run fio to a /dev/sdX from each session, you should be seeing IO to all 3 sessions. How are you determining if a session is being used or not? Are you running the iscsiadm -m session --stats command, watching with wireshark/tcpdump or something else? If you have all three IPs on the same subnet, then it is going to be a little more complicated than what I described above. -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi. For more options, visit https://groups.google.com/d/optout.
MC/S support
Hello: Does open-iscsi (version 2.0-870) support MC/S (Multiple connections per iSCSI session)? Can someone please let me know how to enable this? Thanks a lot! --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
