I understand how rados and ceph work. The host being used for iscsi testing has a direct 10GB link to the cluster which has 6 storage servers each with 12 4TB drives for a total of 72 OSD and about 288TB total size. The journals are writing to disk (not SSD).
The problem I was trying to identify was the bottleneck in the iscsi-RBD chain and I think it is in the TGT-RBD backend since all other combinations appear to perform OK. Bumping the total threads that tgtd uses to service iSCSI commands seems to help some, though. I realize tgtd is a separate project with different devs, but originally I was gathering info on both the initiator and the target side trying to squeeze the most performance from both sides. On Thu, Aug 28, 2014 at 9:24 PM, Gruher, Joseph R <joseph.r.gru...@intel.com > wrote: > Note that librados can access data from all nodes in the cluster while > iSCSI will funnel the data through whatever node or proxy is hosting the > iSCSI target. What does your overall network and storage layout look > like? Number of systems, number and type of disks, how are Ceph journals > set up, etc? > > > > *From:* open-iscsi@googlegroups.com [mailto:open-iscsi@googlegroups.com] *On > Behalf Of *Wyllys Ingersoll > *Sent:* Tuesday, August 26, 2014 2:05 PM > > *To:* open-iscsi@googlegroups.com > *Subject:* Re: iscsi over RBD performance tips? > > > > iscsi performance to a RAM disk iscsi target using the same fio > parameters, yields a 1GB/second throughput for both read and write > operations, compared to about 400(r)/400(w) MB/second using an RBD based > backend over a 10GB link. > > > My fio job file looks like this (change 'randread' to 'randwrite' for the > write test and modify the filename device to switch between the ramdisk and > rbd). > > [default] > rw=randread > size=10g > bs=1m > ioengine=libaio > direct=1 > numjobs=1 > filename=/dev/sdb > runtime=600 > write_bw_log=iscsiread > iodepth=256 > iodepth_batch=256 > > > > -- > You received this message because you are subscribed to the Google Groups > "open-iscsi" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to open-iscsi+unsubscr...@googlegroups.com. > > To post to this group, send email to open-iscsi@googlegroups.com. > Visit this group at http://groups.google.com/group/open-iscsi. > For more options, visit https://groups.google.com/d/optout. > > -- > You received this message because you are subscribed to a topic in the > Google Groups "open-iscsi" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/open-iscsi/UDnEeyzk4jo/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > open-iscsi+unsubscr...@googlegroups.com. > To post to this group, send email to open-iscsi@googlegroups.com. > Visit this group at http://groups.google.com/group/open-iscsi. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi. For more options, visit https://groups.google.com/d/optout.
