Hi Deepak-san, Thank you for your comment. Please see following comments.
>>1) There is a lof of manual work needed here.. like every time the new host >>added.. admin needs to do FC zoning to ensure that LU is visible by the host. Right. Compared to LVMiSCSI driver, proposed driver are needed some manual admin works. >>Also the method you mentioend for refreshing (echo '---' > ...) doesn't work >>reliably across all storage types does it ? The "echo" command is already used in rescan_hosts() at linuxfc.py before connecting new volume to a instance. As you mentioned, whether this command works properly or not depends on storage types. Therefore, admin needs to confirm the command working properly. >>2) In Slide 1-1 .. how ( and who?) ensures that the compute nodes don't step >>on each other is using the LVs ? In other words.. how is it ensured that LV1 >>is not used by compute nodes 1 and 2 at the same time ? In my understanding, Nova can't assign single cinder volume(ex. VOL1) to multiple instances. After attaching the VOL1 to an instance, a status of the VOL1 is changed to "in-use" and user can’t attach the VOL1 to other instances. >>3) In slide 1-2, you show that the LU1 is seen as /dev/sdx on all the nodes.. >>this is wrong.. it can be seen as anything (/dev/sdx on control node, sdn on >>compute 1, sdz on compute 2) so assumign sdx on all nodes is wrong. How does >>this different device names handled.. in short, how does compute node 2 knows >>that LU1 is actually sdn and not sdz (assuming you had > 1 LUs provisioned) Right. Same device name may not be assigned on all nodes. At my proposed driver, admin needs to create PV and VG manually. Therefore, all nodes do not have to recognize the LU1 as /dev/sdx. >>4) What abt multipath ? In most prod env.. the FC storage will be >>multipath'ed.. hence you will actually see sdx and sdy on each node and you >>actually need to use mpathN (which is multipathe'd to sdx anx sdy) device and >>NOT the sd? device to take adv of the customer multipath env. How does the >>nodes know which mpath? device to use and which mpath? device maps to which >>LU on the array ? As I mentioned above, admin creates PV and VG manually at my proposed driver. If a product environment uses multipath, admin can create a PV and VG on top of mpath device, using "pvcreate /dev/mpath/mpathX". >>5) Doesnt this new proposal also causes the compute nodes to be physcially >>connected (via FC) to the array, which means more wiring and need for FC HBA >>on compute nodes. With LVMiSCSI, we don't need FC HBA on compute nodes so you >>are actualluy adding cost of each FC HBA to the compute nodes and slowly >>turning commodity system to non-commodity ;-) (in a way) I think this is depends on a customer's or cloud provider's requirement.(In slide P9) If the requirement is low cost and none FC cloud environment, LVMiSCSI is appropriate driver. If better I/O performance is required, proposed driver or vendor cinder storage driver with FC are appropriate, because these drivers can directly issue I/O to volumes via FC. >>6) Last but not the least... since you are using 1 BIG LU on the array to >>host multiple volumes, you cannot possibly take adv of the premium, efficient >>snapshot/clone/mirroring features of the array, since they are at LU level, >>not at the LV level. LV snapshots have limitations (as mentioned by you in >>other thread) and are always in-efficient compared to array snapshots. Why >>would someone want to use less efficient method when they invested on a >>expensive array ? Right. If user uses array volume directly, user can take adv of efficient snapshot/clone/mirroring features. As I wrote in a reply e-mail to Avishay-san, in OpenStack cloud environment, workloads of storages have been increasing and it is difficult to manage the workloads because every user have a permission to execute storage operations via cinder. In order to use expensive array more efficient, I think it is better to reduce hardware based storage workload by offloading the workload to software based volume operation on a case by case. If we have two drivers in regards to a storage, we can provide volume both way as the situation demands. Ex. As for "Standard" type storage, use proposed software based LVM cinder driver. As for "High performance" type storage, use hardware based cinder driver.(Ex. Higher charge than "Standard" volume) This is one of use-case of my proposed driver. Regards, Mitsuhiro Tanino <[email protected]> HITACHI DATA SYSTEMS c/o Red Hat, 314 Littleton Road, Westford, MA 01886 From: Deepak Shetty [mailto:[email protected]] Sent: Wednesday, May 28, 2014 3:11 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Cinder] Support LVM on a shared LU Mitsuhiro, Few questions that come to my mind based on your proposal 1) There is a lof of manual work needed here.. like every time the new host added.. admin needs to do FC zoning to ensure that LU is visible by the host. Also the method you mentioend for refreshing (echo '---' > ...) doesn't work reliably across all storage types does it ? 2) In Slide 1-1 .. how ( and who?) ensures that the compute nodes don't step on each other is using the LVs ? In other words.. how is it ensured that LV1 is not used by compute nodes 1 and 2 at the same time ? 3) In slide 1-2, you show that the LU1 is seen as /dev/sdx on all the nodes.. this is wrong.. it can be seen as anything (/dev/sdx on control node, sdn on compute 1, sdz on compute 2) so assumign sdx on all nodes is wrong. How does this different device names handled.. in short, how does compute node 2 knows that LU1 is actually sdn and not sdz (assuming you had > 1 LUs provisioned) 4) What abt multipath ? In most prod env.. the FC storage will be multipath'ed.. hence you will actually see sdx and sdy on each node and you actually need to use mpathN (which is multipathe'd to sdx anx sdy) device and NOT the sd? device to take adv of the customer multipath env. How does the nodes know which mpath? device to use and which mpath? device maps to which LU on the array ? 5) Doesnt this new proposal also causes the compute nodes to be physcially connected (via FC) to the array, which means more wiring and need for FC HBA on compute nodes. With LVMiSCSI, we don't need FC HBA on compute nodes so you are actualluy adding cost of each FC HBA to the compute nodes and slowly turning commodity system to non-commodity ;-) (in a way) 6) Last but not the least... since you are using 1 BIG LU on the array to host multiple volumes, you cannot possibly take adv of the premium, efficient snapshot/clone/mirroring features of the array, since they are at LU level, not at the LV level. LV snapshots have limitations (as mentioned by you in other thread) and are always in-efficient compared to array snapshots. Why would someone want to use less efficient method when they invested on a expensive array ? thanx, deepak On Tue, May 20, 2014 at 9:01 PM, Mitsuhiro Tanino <[email protected]<mailto:[email protected]>> wrote: Hello All, I’m proposing a feature of LVM driver to support LVM on a shared LU. The proposed LVM volume driver provides these benefits. - Reduce hardware based storage workload by offloading the workload to software based volume operation. - Provide quicker volume creation and snapshot creation without storage workloads. - Enable cinder to any kinds of shared storage volumes without specific cinder storage driver. - Better I/O performance using direct volume access via Fibre channel. In the attachment pdf, following contents are explained. 1. Detail of Proposed LVM volume driver 1-1. Big Picture 1-2. Administrator preparation 1-3. Work flow of volume creation and attachment 2. Target of Proposed LVM volume driver 3. Comparison of Proposed LVM volume driver Could you review the attachment? Any comments, questions, additional ideas would be appreciated. Also there are blueprints, wiki and patches related to the slide. https://blueprints.launchpad.net/cinder/+spec/lvm-driver-for-shared-storage https://blueprints.launchpad.net/nova/+spec/lvm-driver-for-shared-storage https://wiki.openstack.org/wiki/Cinder/NewLVMbasedDriverForSharedStorageInCinder https://review.openstack.org/#/c/92479/ https://review.openstack.org/#/c/92443/ Regards, Mitsuhiro Tanino <[email protected]<http://[email protected]>> HITACHI DATA SYSTEMS c/o Red Hat, 314 Littleton Road, Westford, MA 01886 _______________________________________________ OpenStack-dev mailing list [email protected]<mailto:[email protected]> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
