Re: [openstack-dev] [Manila] Ceph native driver for manila
Am 04.03.2015 um 15:18 schrieb Csaba Henk: Hi Danny, - Original Message - From: Danny Al-Gaaf danny.al-g...@bisect.de To: Deepak Shetty dpkshe...@gmail.com Cc: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org, ceph-de...@vger.kernel.org Sent: Wednesday, March 4, 2015 3:05:46 PM Subject: Re: [openstack-dev] [Manila] Ceph native driver for manila ... Another level of indirection. I really like the approach of filesystem passthrough ... the only critical question is if virtfs/p9 is still supported in some way (and the question if not: why?). That only seems to be a biggie, isn't it? Yes, it is. We -- Red Hat -- considered a similar, virtfs based driver for GlusterFS but we dropped that plan exactly for virtfs being abandonware. As far as I know it was meant to be a research project, and providing a fairly well working POC it was concluded -- but Deepak knows more of the story. Would like to understand why it was abandoned. I see the need of filesystem passthrough in the area of virtualization. Is there another solution available? Danny __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Manila] Ceph native driver for manila
Am 04.03.2015 um 15:12 schrieb Csaba Henk: Hi Danny, - Original Message - From: Danny Al-Gaaf danny.al-g...@bisect.de To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org, ceph-de...@vger.kernel.org Sent: Sunday, March 1, 2015 3:07:36 PM Subject: Re: [openstack-dev] [Manila] Ceph native driver for manila ... For us security is very critical, as the performance is too. The first solution via ganesha is not what we prefer (to use CephFS via p9 and NFS would not perform that well I guess). The second solution, to use Can you please explain that why does the Ganesha based stack involve 9p? (Maybe I miss something basic, but I don't know.) Sorry, seems that I mixed it up with the p9 case. But the performance is may still an issue if you use NFS on top of CephFS (incl. all the VM layer involved within this setup). For me the question with all these NFS setups is: why should I use NFS on top on CephFS? What is the right to exist of CephFS in this case? I would like to use CephFS directly or via filesystem passthrough. Danny __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Manila] Ceph native driver for manila
Am 04.03.2015 um 05:19 schrieb Deepak Shetty: On Wed, Mar 4, 2015 at 5:10 AM, Danny Al-Gaaf danny.al-g...@bisect.de wrote: Am 03.03.2015 um 19:31 schrieb Deepak Shetty: [...] [...] I was curious to understand. IIUC Neutron provides private and public networks and for VMs to access external CephFS network, the tenant private network needs to be bridged/routed to the external provider network and there are ways neturon achives it. Are you saying that this approach of neutron is insecure ? I don't say neutron itself is insecure. The problem is: we don't want any VM to get access to the ceph public network at all since this would mean access to all MON, OSDs and MDS daemons. If a tenant VM has access to the ceph public net, which is needed to use/mount native cephfs in this VM, one critical issue would be: the client can attack any ceph component via this network. Maybe I misses something, but routing doesn't change this fact. Agree, but there are ways you can restrict the tenant VMs to specific network ports only using neutron security groups and limit what tenant VM can do. On the CephFS side one can use selinux labels to provide addnl level of security for Ceph daemons, where in only certain process can access/modify them, I am just thinking aloud here, i m not sure how well cephfs works with selinux combined. I don't see how neutron security groups would help here. The problem is if a VM has access, in which way ever, to the Ceph network a attacker/user can on one hand attack ALL ceph daemons and on the other also, if there is a bug, crash all daemons and you would lose the complete cluster. SELinux profiles can may help with preventing subvert security or gain privileges it would not help in this case prevent the VM user to crash the cluster. Thinking more, it seems like then you need a solution that goes via the serviceVM approach but provide native CephFS mounts instead of NFS ? Another level of indirection. I really like the approach of filesystem passthrough ... the only critical question is if virtfs/p9 is still supported in some way (and the question if not: why?). Danny __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Manila] Ceph native driver for manila
Am 03.03.2015 um 19:31 schrieb Deepak Shetty: [...] For us security is very critical, as the performance is too. The first solution via ganesha is not what we prefer (to use CephFS via p9 and NFS would not perform that well I guess). The second solution, to use CephFS directly to the VM would be a bad solution from the security point of view since we can't expose the Ceph public network directly to the VMs to prevent all the security issues we discussed already. Is there any place the security issues are captured for the case where VMs access CephFS directly ? No there isn't any place and this is the issue for us. I was curious to understand. IIUC Neutron provides private and public networks and for VMs to access external CephFS network, the tenant private network needs to be bridged/routed to the external provider network and there are ways neturon achives it. Are you saying that this approach of neutron is insecure ? I don't say neutron itself is insecure. The problem is: we don't want any VM to get access to the ceph public network at all since this would mean access to all MON, OSDs and MDS daemons. If a tenant VM has access to the ceph public net, which is needed to use/mount native cephfs in this VM, one critical issue would be: the client can attack any ceph component via this network. Maybe I misses something, but routing doesn't change this fact. Danny __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Manila] Ceph native driver for manila
Am 27.02.2015 um 01:04 schrieb Sage Weil: [sorry for ceph-devel double-post, forgot to include openstack-dev] Hi everyone, The online Ceph Developer Summit is next week[1] and among other things we'll be talking about how to support CephFS in Manila. At a high level, there are basically two paths: We discussed the CephFS Manila topic also on the last Manila Midcycle Meetup (Kilo) [1][2] 2) Native CephFS driver As I currently understand it, - The driver will set up CephFS auth credentials so that the guest VM can mount CephFS directly - The guest VM will need access to the Ceph network. That makes this mainly interesting for private clouds and trusted environments. - The guest is responsible for running 'mount -t ceph ...'. - I'm not sure how we provide the auth credential to the user/guest... The auth credentials need to be handled currently by a application orchestration solution I guess. I see currently no solution on the Manila layer level atm. If Ceph would provide OpenStack Keystone authentication for rados/cephfs instead of CephX, it could be handled via app orch easily. This would perform better than an NFS gateway, but there are several gaps on the security side that make this unusable currently in an untrusted environment: - The CephFS MDS auth credentials currently are _very_ basic. As in, binary: can this host mount or it cannot. We have the auth cap string parsing in place to restrict to a subdirectory (e.g., this tenant can only mount /tenants/foo), but the MDS does not enforce this yet. [medium project to add that] - The same credential could be used directly via librados to access the data pool directly, regardless of what the MDS has to say about the namespace. There are two ways around this: 1- Give each tenant a separate rados pool. This works today. You'd set a directory policy that puts all files created in that subdirectory in that tenant's pool, then only let the client access those rados pools. 1a- We currently lack an MDS auth capability that restricts which clients get to change that policy. [small project] 2- Extend the MDS file layouts to use the rados namespaces so that users can be separated within the same rados pool. [Medium project] 3- Something fancy with MDS-generated capabilities specifying which rados objects clients get to read. This probably falls in the category of research, although there are some papers we've seen that look promising. [big project] Anyway, this leads to a few questions: - Who is interested in using Manila to attach CephFS to guest VMs? - What use cases are you interested? - How important is security in your environment? As you know we (Deutsche Telekom) are may interested to provide shared filesystems via CephFS to VMs instead of e.g. via NFS. We can provide/discuss use cases at CDS. For us security is very critical, as the performance is too. The first solution via ganesha is not what we prefer (to use CephFS via p9 and NFS would not perform that well I guess). The second solution, to use CephFS directly to the VM would be a bad solution from the security point of view since we can't expose the Ceph public network directly to the VMs to prevent all the security issues we discussed already. We discussed during the Midcycle a third option: Mount CephFS directly on the host system and provide the filesystem to the VMs via p9/virtfs. This need nova integration (I will work on a POC patch for this) to setup libvirt config correctly for virtfs. This solve the security issue and the auth key distribution for the VMs, but it may introduces performance issues due to virtfs usage. We have to check what the specific performance impact will be. Currently this is the preferred solution for our use cases. What's still missing in this solution is user/tenant/subtree separation as in the 2th option. But this is needed anyway for CephFS in general. Danny [1] https://etherpad.openstack.org/p/manila-kilo-midcycle-meetup [2] https://etherpad.openstack.org/p/manila-meetup-winter-2015 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] Etherpad for volume replication created ...
Hi Jay, do you have a link to the etherpad? Danny Am 13.02.2015 um 05:54 schrieb Jay S. Bryant: All, Several members of the Cinder team and I were discussing the current state of volume replication while trying to figure out the best way to resolve bug 1383524 [1]. The outcome of the discussion was a decision to hold off on integrating volume replication support for additional drivers. I took notes from the discussion and have put them in the etherpad. We can use that, first thing in L, as a starting point to rework and fix replication support. Please let me know if you have any questions and feel free to update the etherpad with addition thoughts. Thanks! Jay [1] https://bugs.launchpad.net/cinder/+bug/1383524-- Periodic update replication status causing issues __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Manila] Manila virtual midcycle meetup
Am 11.02.2015 um 04:10 schrieb Ben Swartzlander: https://etherpad.openstack.org/p/manila-kilo-midcycle-meetup This is a reminder that the meetup is tomorrow! It will be entirely virtual, so please join the Google Hangout or the phone bridge. The details are in the etherpad. Do you have by any chance an European/German number for the phone bridge? From the list of attendees also a Shanghai number may helps. Danny __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
