Re: Antw: Re: Best way to create multiple TCP flows on 10 Gbps link
Mike Christie micha...@cs.wisc.edu schrieb am 27.08.2014 um 23:49 in
Nachricht 53fe5276.2060...@cs.wisc.edu:
On 08/27/2014 02:24 AM, Ulrich Windl wrote:
Learner Study learner.st...@gmail.com schrieb am 27.08.2014 um 02:13 in
Nachricht
CAP8+hKW=HApS+=vxeaaibtbbd7yzndu4squt+84se99aglc...@mail.gmail.com:
Hi Mike,
Thanks for suggestions
I think you meant,
echo 1 /sys/block/sdX/device/delete
I don't see /sys/block/sdX/device/remove in my setup.
I'm not sure: Is it echo offline /sys/block/sdX/device/state, echo
scsi
remove-single-device ${host} ${channel} ${id} ${lun} /proc/scsi/scsi, or
echo 1
/sys/class/scsi_device/${host}:${channel}:${id}:${lun}/device/delete
?
To delete a device just do
echo 1 /sys/block/sdX/device/delete
I think the confusing thing is that you don't see a delete in
/sys/block/sdX/device.
You can also do it through proc if it is enabled for your kernel.
No need to offline the device before deleting. The scsi layer will
handle the device state transitions.
--
You received this message because you are subscribed to the Google Groups
open-iscsi group.
To unsubscribe from this group and stop receiving emails from it, send an
email to open-iscsi+unsubscr...@googlegroups.com.
To post to this group, send email to open-iscsi@googlegroups.com.
Visit this group at http://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
open-iscsi group.
To unsubscribe from this group and stop receiving emails from it, send an email
to open-iscsi+unsubscr...@googlegroups.com.
To post to this group, send email to open-iscsi@googlegroups.com.
Visit this group at http://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.
Re: Antw: Re: Best way to create multiple TCP flows on 10 Gbps link
On 08/28/2014 12:59 AM, Ulrich Windl wrote: To delete a device just do echo 1 /sys/block/sdX/device/delete I think the confusing thing is that you don't see a delete in /sys/block/sdX/device. Not sure what you mean. I do: ls /sys/block/sda/device/ block evt_media_change max_sectors rescanstate bsg generic modalias rev subsystem delete iocounterbits modelscsi_device timeout device_blocked iodone_cntpowerscsi_disk type dh_stateioerr_cnt queue_depth scsi_generic uevent driver iorequest_cnt queue_type scsi_levelvendor -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi. For more options, visit https://groups.google.com/d/optout.
Re: Antw: Re: Best way to create multiple TCP flows on 10 Gbps link
On 08/28/2014 11:29 AM, Mike Christie wrote: On 08/28/2014 12:59 AM, Ulrich Windl wrote: To delete a device just do echo 1 /sys/block/sdX/device/delete I think the confusing thing is that you don't see a delete in /sys/block/sdX/device. Not sure what you mean. I do: ls /sys/block/sda/device/ block evt_media_change max_sectors rescanstate bsg generic modalias rev subsystem delete iocounterbits modelscsi_device timeout device_blocked iodone_cntpowerscsi_disk type dh_stateioerr_cnt queue_depth scsi_generic uevent driver iorequest_cnt queue_type scsi_levelvendor Ah, I see. I think depending on the kernel config options used the device symlink might not even be there. -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi. For more options, visit https://groups.google.com/d/optout.
Antw: Re: Best way to create multiple TCP flows on 10 Gbps link
Learner Study learner.st...@gmail.com schrieb am 27.08.2014 um 02:13 in
Nachricht
CAP8+hKW=HApS+=vxeaaibtbbd7yzndu4squt+84se99aglc...@mail.gmail.com:
Hi Mike,
Thanks for suggestions
I think you meant,
echo 1 /sys/block/sdX/device/delete
I don't see /sys/block/sdX/device/remove in my setup.
I'm not sure: Is it echo offline /sys/block/sdX/device/state, echo scsi
remove-single-device ${host} ${channel} ${id} ${lun} /proc/scsi/scsi, or
echo 1 /sys/class/scsi_device/${host}:${channel}:${id}:${lun}/device/delete?
How do following FIO options look?
[default]
rw=read
size=4g
bs=1m
ioengine=libaio
direct=1
numjobs=1
filename=/dev/sda
runtime=360
iodepth=256
Thanks for your time!
On Tue, Aug 26, 2014 at 4:49 PM, Michael Christie micha...@cs.wisc.edu
wrote:
On Aug 26, 2014, at 3:11 PM, Learner learner.st...@gmail.com wrote:
Another related observation and some questions;
I am using open iscsi on init with IET on trgt over a single 10gbps link
There are three ip aliases on each side
I have 3 ramdisks exported by IET to init
I do iscsi login 3 times, once using each underlying ip address and notice
that each iscsi session sees all 3 disks.
Is it possible to restrict such that each init only sees one separate disk?
There is no iscsi initiator or target setting for this. The default is to
show all paths (each /dev/sdx is a path to the same device)..
You would have to manually delete some paths by doing
echo 1 /sys/block/sdX/device/remove
When I run fio on each mounted disk, I see that only two underlying tcp
sessions are being used - that limits the perf.
Any ideas on how to overcome this?
How are you matching sessions with devices? It should just be a matter of
running fio on the right devices. If you run:
iscsiadm -m session -P 3
you can see how the sdXs match up with sessions/connections. If you run fio
to a /dev/sdX from each session, you should be seeing IO to all 3 sessions.
Thanks!
Sent from my iPhone
On Aug 26, 2014, at 12:53 PM, Mark Lehrer m...@knm.org wrote:
On Tue, 26 Aug 2014 08:58:46 -0400 Alvin Starr al...@iplink.net wrote:
I am trying to achieve10Gbps in my single initiator/single target
env. (open-iscsi and IET)
On a semi-related note, are there any good guides out there to
tuning Linux for maximum single-socket performance? On my 40 gigabit
You are likely getting hit by the bandwidth-delay product.
Take a look at http://en.wikipedia.org/wiki/Bandwidth-delay_product
and http://www.kehlet.cx/articles/99.html
Thanks that helped get my netcat transfer up over 500MB/sec using IPoIB.
Unfortunately that is still only about 10% of the available bandwidth.
I'll keep on tweaking and see how far I can take it.
Thanks,
Mark
--
You received this message because you are subscribed to the Google Groups
open-iscsi group.
To unsubscribe from this group and stop receiving emails from it, send an
email to open-iscsi+unsubscr...@googlegroups.com.
To post to this group, send email to open-iscsi@googlegroups.com.
Visit this group at http://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
open-iscsi group.
To unsubscribe from this group and stop receiving emails from it, send an
email to open-iscsi+unsubscr...@googlegroups.com.
To post to this group, send email to open-iscsi@googlegroups.com.
Visit this group at http://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
open-iscsi group.
To unsubscribe from this group and stop receiving emails from it, send an
email to open-iscsi+unsubscr...@googlegroups.com.
To post to this group, send email to open-iscsi@googlegroups.com.
Visit this group at http://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
open-iscsi group.
To unsubscribe from this group and stop receiving emails from it, send an
email to open-iscsi+unsubscr...@googlegroups.com.
To post to this group, send email to open-iscsi@googlegroups.com.
Visit this group at http://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
open-iscsi group.
To unsubscribe from this group and stop receiving emails from it, send an email
to open-iscsi+unsubscr...@googlegroups.com.
To post to this group, send email to open-iscsi@googlegroups.com.
Visit this group at http://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.
Re: Antw: Re: Best way to create multiple TCP flows on 10 Gbps link
On 08/27/2014 02:24 AM, Ulrich Windl wrote:
Learner Study learner.st...@gmail.com schrieb am 27.08.2014 um 02:13 in
Nachricht
CAP8+hKW=HApS+=vxeaaibtbbd7yzndu4squt+84se99aglc...@mail.gmail.com:
Hi Mike,
Thanks for suggestions
I think you meant,
echo 1 /sys/block/sdX/device/delete
I don't see /sys/block/sdX/device/remove in my setup.
I'm not sure: Is it echo offline /sys/block/sdX/device/state, echo scsi
remove-single-device ${host} ${channel} ${id} ${lun} /proc/scsi/scsi, or
echo 1
/sys/class/scsi_device/${host}:${channel}:${id}:${lun}/device/delete?
To delete a device just do
echo 1 /sys/block/sdX/device/delete
You can also do it through proc if it is enabled for your kernel.
No need to offline the device before deleting. The scsi layer will
handle the device state transitions.
--
You received this message because you are subscribed to the Google Groups
open-iscsi group.
To unsubscribe from this group and stop receiving emails from it, send an email
to open-iscsi+unsubscr...@googlegroups.com.
To post to this group, send email to open-iscsi@googlegroups.com.
Visit this group at http://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.
Antw: Re: Best way to create multiple TCP flows on 10 Gbps link
Mark Lehrer m...@knm.org schrieb am 25.08.2014 um 20:58 in Nachricht ximss-10382...@knm.org: I am trying to achieve10Gbps in my single initiator/single target env. (open-iscsi and IET) On a semi-related note, are there any good guides out there to tuning Linux for maximum single-socket performance? On my 40 gigabit setup, I seem to Hi! You are referring to networks sockets, not to CPU sockets, I guess. Have you tried larger packets (if you can control the LAN). I don't know if open iSCSI can do IPv6, but from what I read IPv6 could give better TCP performance. Have you checked interrupt assignments for the NIC? I guess your card is PCIe and it uses one lane? Have you tried (for comparison) to do just a netcat to/from /dev/zero? You have to analyze the groups, hardware, network stack and iSCSI separately, I guess. iSCSI can not do any better than the networks stack, and the network stack cannot do better than the hardware can. Regards, Ulrich hit a wall around 3 gigabits when doing a single TCP socket. To go far above that I need to do multipath, initiator-side RAID, or RDMA. Thanks, Mark -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi. For more options, visit https://groups.google.com/d/optout.
