Re: [openstack-dev] [Fuel] Ceph Public Network Setting
Zhou, On Wed, Jun 24, 2015 at 8:38 PM Zhou Zheng Sheng / 周征晟 zhengsh...@awcloud.com wrote: Hi, I also notice that in latest ceph-deploy source code, it allows us to define public_network and cluster_network argument when calling 'ceph-deploy new'. It also checks if monitor IP address is in either of the two networks. https://github.com/ceph/ceph-deploy/blob/master/ceph_deploy/new.py#L158 I'm not sure that Ceph monitor operating in the public network is just an artificial requirement, or it's an inherent precondition when they implement Ceph and breaking the precondition might lead to some strange problems. If it's the latter case, when we change Ceph public network, we have to change Ceph monitor IP address configuration at the same time. on 2015/06/25 11:01, Zhou Zheng Sheng / 周征晟 wrote: Hello Andrew, I did some similar modifications in Fuel 6.0 as [1][2]. However in [3] it says Ceph monitor have to run on the public network. It doesn't have to run on fuel public network, ceph's public is required for MONs and OSDs in this case public is because it's expected that the Ceph clients will connect to it. Where cluster is used for intra-OSD communication. We started by assigning Ceph's public to Fuel's management(internal in this scope) and Ceph's cluster to Fuel's stroage In the file deployment/puppet/ceph/manifests/conf.pp, we can see it calls 'ceph-deploy new ${::hostname}:${::internal_address}'. In deployment/puppet/ceph/manifests/mon.pp, it calls 'ceph-deploy mon create ${::hostname}:${::internal_address}'. Thanks, I thought we removed these codes already and where using the class entry points, We will fix these as part of the change over. After we set Ceph public network to OpenStack storage network, Ceph monitor still runs on management network. I can see in QEMU command line, it connects to Ceph via management network IP address. Though this setup does not follow the rule suggested in [3], it probably works because Ceph monitor only provides PG location information, and then QEMU should talk to OSDs directly using the storage network addresses. That's why I didn't take your patches as-is. I meant to setup a Fuel 6.1 environment and test the modifications before I can give an accurate comment. [3] http://ceph.com/docs/master/rados/configuration/network-config-ref/#monitor-ip-tables on 2015/06/25 03:20, Andrew Woodward wrote: Zhou, As mentioned, Please review [1][2]. This is the interface we will support as we implement the advanced networking parts. In your case, just fip the role map in your patch to nailgun and the library interface will remain the same. [1] https://review.openstack.org/#/c/194434/ [2] https://review.openstack.org/#/c/194438/ On Wed, Jun 24, 2015 at 3:21 AM Stanislav Makar sma...@mirantis.com wrote: Hello My five cents :) I also very proud of our 6.1 release Unfortunately we still do not have this separation for ceph public and cluster networks in 6.1. One little hint In 6.1 we have granular deployment ( https://docs.mirantis.com/fuel/fuel-master/release-notes.html#granular-deployment-based-on-pluggable-tasks) and all tasks which are connected with ceph deployment are in https://github.com/stackforge/fuel-library/tree/stable/6.1/deployment/puppet/osnailyfacter/modular/ceph To have your patch working in 6.1 you should hack all these files now If you have question feel free to ask. Thanks. -- All the best, Stanislav Makar skype: makar_stanislav irc: stamak On Wed, Jun 24, 2015 at 12:04 PM, Sergii Golovatiuk sgolovat...@mirantis.com wrote: Hi Zhou, Try Fuel 6.1 where we have a lot of very nice features including 'advanced networking'. Feel free to read release notes [1]. It's one of the most significant releases with many many features and improvements. I am very proud of it. [1] https://docs.mirantis.com/fuel/fuel-master/release-notes.html -- Best regards, Sergii Golovatiuk, Skype #golserge IRC #holser On Wed, Jun 24, 2015 at 11:42 AM, Zhou Zheng Sheng / 周征晟 zhengsh...@awcloud.com wrote: Thank you guys! I'm reading the advanced networking spec and think it's good. I'll try to comment it. It's too epic for me for now. I wrote a patch to move Ceph public network to storage network. It's based on Fuel 6.0, so I'm trying to rebase it to lastest master then maybe ask you for a review. on 2015/06/23 19:21, Igor Kalnitsky wrote: Hello, That makes sense to me. Still, I want to point that we're going to implement advanced networking and with this feature you'll be able to assign every single network role to any network. That means, you'll be able to assign ceph network role to storage, management or whatever-you-want network. Sounds cool, ha? :) Feel free to read a design spec [1]. Thanks, Igor [1]: https://review.openstack.org/#/c/115340/ On Tue, Jun 23, 2015 at 1:13 PM, Zhou Zheng Sheng / 周征晟 zhengsh...@awcloud.com
Re: [openstack-dev] [Fuel] Ceph Public Network Setting
Hi Zhou, Try Fuel 6.1 where we have a lot of very nice features including 'advanced networking'. Feel free to read release notes [1]. It's one of the most significant releases with many many features and improvements. I am very proud of it. [1] https://docs.mirantis.com/fuel/fuel-master/release-notes.html -- Best regards, Sergii Golovatiuk, Skype #golserge IRC #holser On Wed, Jun 24, 2015 at 11:42 AM, Zhou Zheng Sheng / 周征晟 zhengsh...@awcloud.com wrote: Thank you guys! I'm reading the advanced networking spec and think it's good. I'll try to comment it. It's too epic for me for now. I wrote a patch to move Ceph public network to storage network. It's based on Fuel 6.0, so I'm trying to rebase it to lastest master then maybe ask you for a review. on 2015/06/23 19:21, Igor Kalnitsky wrote: Hello, That makes sense to me. Still, I want to point that we're going to implement advanced networking and with this feature you'll be able to assign every single network role to any network. That means, you'll be able to assign ceph network role to storage, management or whatever-you-want network. Sounds cool, ha? :) Feel free to read a design spec [1]. Thanks, Igor [1]: https://review.openstack.org/#/c/115340/ On Tue, Jun 23, 2015 at 1:13 PM, Zhou Zheng Sheng / 周征晟 zhengsh...@awcloud.com wrote: Hi! I notice that in OpenStack deployed by Fuel, Ceph public network is on management network. In some environments, not all NICs of a physical server are 10Gb. Sometimes 1 or 2 among the NICs on a machine may be 1Gb. Usually on this type of machine we assign management network to 1Gb NIC, and storage network to 10Gb NIC. If Ceph public network is with management network, the QEMU accesses Ceph using management network, and the performance is not optimal. In a small deployment, cloud controller and Ceph OSD may be assigned to the same machine, so it would be more effective to keep Ceph client traffic separated from MySQL, RabbitMQ, Pacemaker traffic. Maybe it's better to place Ceph public network on the storage network. Agree? -- Best wishes! Zhou Zheng Sheng, Software Developer Beijing AWcloud Software Co., Ltd. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Best wishes! Zhou Zheng Sheng / 周征晟 Software Developer Beijing AWcloud Software Co., Ltd. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Fuel] Ceph Public Network Setting
Thank you guys! I'm reading the advanced networking spec and think it's good. I'll try to comment it. It's too epic for me for now. I wrote a patch to move Ceph public network to storage network. It's based on Fuel 6.0, so I'm trying to rebase it to lastest master then maybe ask you for a review. on 2015/06/23 19:21, Igor Kalnitsky wrote: Hello, That makes sense to me. Still, I want to point that we're going to implement advanced networking and with this feature you'll be able to assign every single network role to any network. That means, you'll be able to assign ceph network role to storage, management or whatever-you-want network. Sounds cool, ha? :) Feel free to read a design spec [1]. Thanks, Igor [1]: https://review.openstack.org/#/c/115340/ On Tue, Jun 23, 2015 at 1:13 PM, Zhou Zheng Sheng / 周征晟 zhengsh...@awcloud.com wrote: Hi! I notice that in OpenStack deployed by Fuel, Ceph public network is on management network. In some environments, not all NICs of a physical server are 10Gb. Sometimes 1 or 2 among the NICs on a machine may be 1Gb. Usually on this type of machine we assign management network to 1Gb NIC, and storage network to 10Gb NIC. If Ceph public network is with management network, the QEMU accesses Ceph using management network, and the performance is not optimal. In a small deployment, cloud controller and Ceph OSD may be assigned to the same machine, so it would be more effective to keep Ceph client traffic separated from MySQL, RabbitMQ, Pacemaker traffic. Maybe it's better to place Ceph public network on the storage network. Agree? -- Best wishes! Zhou Zheng Sheng, Software Developer Beijing AWcloud Software Co., Ltd. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Best wishes! Zhou Zheng Sheng / 周征晟 Software Developer Beijing AWcloud Software Co., Ltd. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Fuel] Ceph Public Network Setting
Hello My five cents :) I also very proud of our 6.1 release Unfortunately we still do not have this separation for ceph public and cluster networks in 6.1. One little hint In 6.1 we have granular deployment ( https://docs.mirantis.com/fuel/fuel-master/release-notes.html#granular-deployment-based-on-pluggable-tasks) and all tasks which are connected with ceph deployment are in https://github.com/stackforge/fuel-library/tree/stable/6.1/deployment/puppet/osnailyfacter/modular/ceph To have your patch working in 6.1 you should hack all these files now If you have question feel free to ask. Thanks. -- All the best, Stanislav Makar skype: makar_stanislav irc: stamak On Wed, Jun 24, 2015 at 12:04 PM, Sergii Golovatiuk sgolovat...@mirantis.com wrote: Hi Zhou, Try Fuel 6.1 where we have a lot of very nice features including 'advanced networking'. Feel free to read release notes [1]. It's one of the most significant releases with many many features and improvements. I am very proud of it. [1] https://docs.mirantis.com/fuel/fuel-master/release-notes.html -- Best regards, Sergii Golovatiuk, Skype #golserge IRC #holser On Wed, Jun 24, 2015 at 11:42 AM, Zhou Zheng Sheng / 周征晟 zhengsh...@awcloud.com wrote: Thank you guys! I'm reading the advanced networking spec and think it's good. I'll try to comment it. It's too epic for me for now. I wrote a patch to move Ceph public network to storage network. It's based on Fuel 6.0, so I'm trying to rebase it to lastest master then maybe ask you for a review. on 2015/06/23 19:21, Igor Kalnitsky wrote: Hello, That makes sense to me. Still, I want to point that we're going to implement advanced networking and with this feature you'll be able to assign every single network role to any network. That means, you'll be able to assign ceph network role to storage, management or whatever-you-want network. Sounds cool, ha? :) Feel free to read a design spec [1]. Thanks, Igor [1]: https://review.openstack.org/#/c/115340/ On Tue, Jun 23, 2015 at 1:13 PM, Zhou Zheng Sheng / 周征晟 zhengsh...@awcloud.com wrote: Hi! I notice that in OpenStack deployed by Fuel, Ceph public network is on management network. In some environments, not all NICs of a physical server are 10Gb. Sometimes 1 or 2 among the NICs on a machine may be 1Gb. Usually on this type of machine we assign management network to 1Gb NIC, and storage network to 10Gb NIC. If Ceph public network is with management network, the QEMU accesses Ceph using management network, and the performance is not optimal. In a small deployment, cloud controller and Ceph OSD may be assigned to the same machine, so it would be more effective to keep Ceph client traffic separated from MySQL, RabbitMQ, Pacemaker traffic. Maybe it's better to place Ceph public network on the storage network. Agree? -- Best wishes! Zhou Zheng Sheng, Software Developer Beijing AWcloud Software Co., Ltd. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Best wishes! Zhou Zheng Sheng / 周征晟 Software Developer Beijing AWcloud Software Co., Ltd. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Fuel] Ceph Public Network Setting
Thank you Sergii, I read the release notes but didn't find a specific 'Advanced Networking' part. It seems advanced networking is a spec [1] mentioned by Igor in a previous mail. Maybe I overlook something? Anyway, I think 6.1 is a great release, for it solves some major problems in my previous experience using 6.0. [1] https://blueprints.launchpad.net/fuel/+spec/granular-network-functions on 2015/06/24 17:04, Sergii Golovatiuk wrote: Hi Zhou, Try Fuel 6.1 where we have a lot of very nice features including 'advanced networking'. Feel free to read release notes [1]. It's one of the most significant releases with many many features and improvements. I am very proud of it. [1] https://docs.mirantis.com/fuel/fuel-master/release-notes.html -- Best regards, Sergii Golovatiuk, Skype #golserge IRC #holser On Wed, Jun 24, 2015 at 11:42 AM, Zhou Zheng Sheng / 周征晟 zhengsh...@awcloud.com mailto:zhengsh...@awcloud.com wrote: Thank you guys! I'm reading the advanced networking spec and think it's good. I'll try to comment it. It's too epic for me for now. I wrote a patch to move Ceph public network to storage network. It's based on Fuel 6.0, so I'm trying to rebase it to lastest master then maybe ask you for a review. on 2015/06/23 19:21, Igor Kalnitsky wrote: Hello, That makes sense to me. Still, I want to point that we're going to implement advanced networking and with this feature you'll be able to assign every single network role to any network. That means, you'll be able to assign ceph network role to storage, management or whatever-you-want network. Sounds cool, ha? :) Feel free to read a design spec [1]. Thanks, Igor [1]: https://review.openstack.org/#/c/115340/ On Tue, Jun 23, 2015 at 1:13 PM, Zhou Zheng Sheng / 周征晟 zhengsh...@awcloud.com mailto:zhengsh...@awcloud.com wrote: Hi! I notice that in OpenStack deployed by Fuel, Ceph public network is on management network. In some environments, not all NICs of a physical server are 10Gb. Sometimes 1 or 2 among the NICs on a machine may be 1Gb. Usually on this type of machine we assign management network to 1Gb NIC, and storage network to 10Gb NIC. If Ceph public network is with management network, the QEMU accesses Ceph using management network, and the performance is not optimal. In a small deployment, cloud controller and Ceph OSD may be assigned to the same machine, so it would be more effective to keep Ceph client traffic separated from MySQL, RabbitMQ, Pacemaker traffic. Maybe it's better to place Ceph public network on the storage network. Agree? -- Best wishes! Zhou Zheng Sheng, Software Developer Beijing AWcloud Software Co., Ltd. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Best wishes! Zhou Zheng Sheng / 周征晟 Software Developer Beijing AWcloud Software Co., Ltd. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Best wishes! Zhou Zheng Sheng / 周征晟 Software Developer Beijing AWcloud Software Co., Ltd. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Fuel] Ceph Public Network Setting
Hello Andrew, I did some similar modifications in Fuel 6.0 as [1][2]. However in [3] it says Ceph monitor have to run on the public network. In the file deployment/puppet/ceph/manifests/conf.pp, we can see it calls 'ceph-deploy new ${::hostname}:${::internal_address}'. In deployment/puppet/ceph/manifests/mon.pp, it calls 'ceph-deploy mon create ${::hostname}:${::internal_address}'. After we set Ceph public network to OpenStack storage network, Ceph monitor still runs on management network. I also see in QEMU command line, it connects to Ceph via management network IP address. Though this setup does not follow the rule suggested in [3], it probably works because Ceph monitor only provides PG location information, and then QEMU should talk to OSDs directly using the storage network addresses. That's why I didn't take your patches as-is. I meant to setup a Fuel 6.1 environment and test the modifications before I can give an accurate comment. [3] http://ceph.com/docs/master/rados/configuration/network-config-ref/#monitor-ip-tables on 2015/06/25 03:20, Andrew Woodward wrote: Zhou, As mentioned, Please review [1][2]. This is the interface we will support as we implement the advanced networking parts. In your case, just fip the role map in your patch to nailgun and the library interface will remain the same. [1] https://review.openstack.org/#/c/194434/ [2] https://review.openstack.org/#/c/194438/ On Wed, Jun 24, 2015 at 3:21 AM Stanislav Makar sma...@mirantis.com mailto:sma...@mirantis.com wrote: Hello My five cents :) I also very proud of our 6.1 release Unfortunately we still do not have this separation for ceph public and cluster networks in 6.1. One little hint In 6.1 we have granular deployment (https://docs.mirantis.com/fuel/fuel-master/release-notes.html#granular-deployment-based-on-pluggable-tasks) and all tasks which are connected with ceph deployment are in https://github.com/stackforge/fuel-library/tree/stable/6.1/deployment/puppet/osnailyfacter/modular/ceph To have your patch working in 6.1 you should hack all these files now If you have question feel free to ask. Thanks. -- All the best, Stanislav Makar skype: makar_stanislav irc: stamak On Wed, Jun 24, 2015 at 12:04 PM, Sergii Golovatiuk sgolovat...@mirantis.com mailto:sgolovat...@mirantis.com wrote: Hi Zhou, Try Fuel 6.1 where we have a lot of very nice features including 'advanced networking'. Feel free to read release notes [1]. It's one of the most significant releases with many many features and improvements. I am very proud of it. [1] https://docs.mirantis.com/fuel/fuel-master/release-notes.html -- Best regards, Sergii Golovatiuk, Skype #golserge IRC #holser On Wed, Jun 24, 2015 at 11:42 AM, Zhou Zheng Sheng / 周征晟 zhengsh...@awcloud.com mailto:zhengsh...@awcloud.com wrote: Thank you guys! I'm reading the advanced networking spec and think it's good. I'll try to comment it. It's too epic for me for now. I wrote a patch to move Ceph public network to storage network. It's based on Fuel 6.0, so I'm trying to rebase it to lastest master then maybe ask you for a review. on 2015/06/23 19:21, Igor Kalnitsky wrote: Hello, That makes sense to me. Still, I want to point that we're going to implement advanced networking and with this feature you'll be able to assign every single network role to any network. That means, you'll be able to assign ceph network role to storage, management or whatever-you-want network. Sounds cool, ha? :) Feel free to read a design spec [1]. Thanks, Igor [1]: https://review.openstack.org/#/c/115340/ On Tue, Jun 23, 2015 at 1:13 PM, Zhou Zheng Sheng / 周征晟 zhengsh...@awcloud.com mailto:zhengsh...@awcloud.com wrote: Hi! I notice that in OpenStack deployed by Fuel, Ceph public network is on management network. In some environments, not all NICs of a physical server are 10Gb. Sometimes 1 or 2 among the NICs on a machine may be 1Gb. Usually on this type of machine we assign management network to 1Gb NIC, and storage network to 10Gb NIC. If Ceph public network is with management network, the QEMU accesses Ceph using management network, and
Re: [openstack-dev] [Fuel] Ceph Public Network Setting
Hi, I also notice that in latest ceph-deploy source code, it allows us to define public_network and cluster_network argument when calling 'ceph-deploy new'. It also checks if monitor IP address is in either of the two networks. https://github.com/ceph/ceph-deploy/blob/master/ceph_deploy/new.py#L158 I'm not sure that Ceph monitor operating in the public network is just an artificial requirement, or it's an inherent precondition when they implement Ceph and breaking the precondition might lead to some strange problems. If it's the latter case, when we change Ceph public network, we have to change Ceph monitor IP address configuration at the same time. on 2015/06/25 11:01, Zhou Zheng Sheng / 周征晟 wrote: Hello Andrew, I did some similar modifications in Fuel 6.0 as [1][2]. However in [3] it says Ceph monitor have to run on the public network. In the file deployment/puppet/ceph/manifests/conf.pp, we can see it calls 'ceph-deploy new ${::hostname}:${::internal_address}'. In deployment/puppet/ceph/manifests/mon.pp, it calls 'ceph-deploy mon create ${::hostname}:${::internal_address}'. After we set Ceph public network to OpenStack storage network, Ceph monitor still runs on management network. I can see in QEMU command line, it connects to Ceph via management network IP address. Though this setup does not follow the rule suggested in [3], it probably works because Ceph monitor only provides PG location information, and then QEMU should talk to OSDs directly using the storage network addresses. That's why I didn't take your patches as-is. I meant to setup a Fuel 6.1 environment and test the modifications before I can give an accurate comment. [3] http://ceph.com/docs/master/rados/configuration/network-config-ref/#monitor-ip-tables on 2015/06/25 03:20, Andrew Woodward wrote: Zhou, As mentioned, Please review [1][2]. This is the interface we will support as we implement the advanced networking parts. In your case, just fip the role map in your patch to nailgun and the library interface will remain the same. [1] https://review.openstack.org/#/c/194434/ [2] https://review.openstack.org/#/c/194438/ On Wed, Jun 24, 2015 at 3:21 AM Stanislav Makar sma...@mirantis.com mailto:sma...@mirantis.com wrote: Hello My five cents :) I also very proud of our 6.1 release Unfortunately we still do not have this separation for ceph public and cluster networks in 6.1. One little hint In 6.1 we have granular deployment (https://docs.mirantis.com/fuel/fuel-master/release-notes.html#granular-deployment-based-on-pluggable-tasks) and all tasks which are connected with ceph deployment are in https://github.com/stackforge/fuel-library/tree/stable/6.1/deployment/puppet/osnailyfacter/modular/ceph To have your patch working in 6.1 you should hack all these files now If you have question feel free to ask. Thanks. -- All the best, Stanislav Makar skype: makar_stanislav irc: stamak On Wed, Jun 24, 2015 at 12:04 PM, Sergii Golovatiuk sgolovat...@mirantis.com mailto:sgolovat...@mirantis.com wrote: Hi Zhou, Try Fuel 6.1 where we have a lot of very nice features including 'advanced networking'. Feel free to read release notes [1]. It's one of the most significant releases with many many features and improvements. I am very proud of it. [1] https://docs.mirantis.com/fuel/fuel-master/release-notes.html -- Best regards, Sergii Golovatiuk, Skype #golserge IRC #holser On Wed, Jun 24, 2015 at 11:42 AM, Zhou Zheng Sheng / 周征晟 zhengsh...@awcloud.com mailto:zhengsh...@awcloud.com wrote: Thank you guys! I'm reading the advanced networking spec and think it's good. I'll try to comment it. It's too epic for me for now. I wrote a patch to move Ceph public network to storage network. It's based on Fuel 6.0, so I'm trying to rebase it to lastest master then maybe ask you for a review. on 2015/06/23 19:21, Igor Kalnitsky wrote: Hello, That makes sense to me. Still, I want to point that we're going to implement advanced networking and with this feature you'll be able to assign every single network role to any network. That means, you'll be able to assign ceph network role to storage, management or whatever-you-want network. Sounds cool, ha? :) Feel free to read a design spec [1]. Thanks, Igor [1]: https://review.openstack.org/#/c/115340/ On Tue, Jun 23, 2015 at 1:13 PM, Zhou Zheng
Re: [openstack-dev] [Fuel] Ceph Public Network Setting
Zhou, As mentioned, Please review [1][2]. This is the interface we will support as we implement the advanced networking parts. In your case, just fip the role map in your patch to nailgun and the library interface will remain the same. [1] https://review.openstack.org/#/c/194434/ [2] https://review.openstack.org/#/c/194438/ On Wed, Jun 24, 2015 at 3:21 AM Stanislav Makar sma...@mirantis.com wrote: Hello My five cents :) I also very proud of our 6.1 release Unfortunately we still do not have this separation for ceph public and cluster networks in 6.1. One little hint In 6.1 we have granular deployment ( https://docs.mirantis.com/fuel/fuel-master/release-notes.html#granular-deployment-based-on-pluggable-tasks) and all tasks which are connected with ceph deployment are in https://github.com/stackforge/fuel-library/tree/stable/6.1/deployment/puppet/osnailyfacter/modular/ceph To have your patch working in 6.1 you should hack all these files now If you have question feel free to ask. Thanks. -- All the best, Stanislav Makar skype: makar_stanislav irc: stamak On Wed, Jun 24, 2015 at 12:04 PM, Sergii Golovatiuk sgolovat...@mirantis.com wrote: Hi Zhou, Try Fuel 6.1 where we have a lot of very nice features including 'advanced networking'. Feel free to read release notes [1]. It's one of the most significant releases with many many features and improvements. I am very proud of it. [1] https://docs.mirantis.com/fuel/fuel-master/release-notes.html -- Best regards, Sergii Golovatiuk, Skype #golserge IRC #holser On Wed, Jun 24, 2015 at 11:42 AM, Zhou Zheng Sheng / 周征晟 zhengsh...@awcloud.com wrote: Thank you guys! I'm reading the advanced networking spec and think it's good. I'll try to comment it. It's too epic for me for now. I wrote a patch to move Ceph public network to storage network. It's based on Fuel 6.0, so I'm trying to rebase it to lastest master then maybe ask you for a review. on 2015/06/23 19:21, Igor Kalnitsky wrote: Hello, That makes sense to me. Still, I want to point that we're going to implement advanced networking and with this feature you'll be able to assign every single network role to any network. That means, you'll be able to assign ceph network role to storage, management or whatever-you-want network. Sounds cool, ha? :) Feel free to read a design spec [1]. Thanks, Igor [1]: https://review.openstack.org/#/c/115340/ On Tue, Jun 23, 2015 at 1:13 PM, Zhou Zheng Sheng / 周征晟 zhengsh...@awcloud.com wrote: Hi! I notice that in OpenStack deployed by Fuel, Ceph public network is on management network. In some environments, not all NICs of a physical server are 10Gb. Sometimes 1 or 2 among the NICs on a machine may be 1Gb. Usually on this type of machine we assign management network to 1Gb NIC, and storage network to 10Gb NIC. If Ceph public network is with management network, the QEMU accesses Ceph using management network, and the performance is not optimal. In a small deployment, cloud controller and Ceph OSD may be assigned to the same machine, so it would be more effective to keep Ceph client traffic separated from MySQL, RabbitMQ, Pacemaker traffic. Maybe it's better to place Ceph public network on the storage network. Agree? -- Best wishes! Zhou Zheng Sheng, Software Developer Beijing AWcloud Software Co., Ltd. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Best wishes! Zhou Zheng Sheng / 周征晟 Software Developer Beijing AWcloud Software Co., Ltd. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -- Andrew Woodward Mirantis Fuel
Re: [openstack-dev] [Fuel] Ceph Public Network Setting
I notice that in OpenStack deployed by Fuel, Ceph public network is on management network. As I know separating cuph/public and management networks in a scope of 7.0 release. /sv __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Fuel] Ceph Public Network Setting
Hello, That makes sense to me. Still, I want to point that we're going to implement advanced networking and with this feature you'll be able to assign every single network role to any network. That means, you'll be able to assign ceph network role to storage, management or whatever-you-want network. Sounds cool, ha? :) Feel free to read a design spec [1]. Thanks, Igor [1]: https://review.openstack.org/#/c/115340/ On Tue, Jun 23, 2015 at 1:13 PM, Zhou Zheng Sheng / 周征晟 zhengsh...@awcloud.com wrote: Hi! I notice that in OpenStack deployed by Fuel, Ceph public network is on management network. In some environments, not all NICs of a physical server are 10Gb. Sometimes 1 or 2 among the NICs on a machine may be 1Gb. Usually on this type of machine we assign management network to 1Gb NIC, and storage network to 10Gb NIC. If Ceph public network is with management network, the QEMU accesses Ceph using management network, and the performance is not optimal. In a small deployment, cloud controller and Ceph OSD may be assigned to the same machine, so it would be more effective to keep Ceph client traffic separated from MySQL, RabbitMQ, Pacemaker traffic. Maybe it's better to place Ceph public network on the storage network. Agree? -- Best wishes! Zhou Zheng Sheng, Software Developer Beijing AWcloud Software Co., Ltd. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Fuel] Ceph Public Network Setting
Hi! I notice that in OpenStack deployed by Fuel, Ceph public network is on management network. In some environments, not all NICs of a physical server are 10Gb. Sometimes 1 or 2 among the NICs on a machine may be 1Gb. Usually on this type of machine we assign management network to 1Gb NIC, and storage network to 10Gb NIC. If Ceph public network is with management network, the QEMU accesses Ceph using management network, and the performance is not optimal. In a small deployment, cloud controller and Ceph OSD may be assigned to the same machine, so it would be more effective to keep Ceph client traffic separated from MySQL, RabbitMQ, Pacemaker traffic. Maybe it's better to place Ceph public network on the storage network. Agree? -- Best wishes! Zhou Zheng Sheng, Software Developer Beijing AWcloud Software Co., Ltd. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Fuel] Ceph Public Network Setting
As part of moving towards the granular network roles, the roles should be extended as I describe in [1]. I quickly hacked it together, however if you apply both patches you would be able to change your nailgun side to the values you need and when the complete code comes along you will be able to use the same network role mapping to control the placement in future versions. [1] https://bugs.launchpad.net/fuel/+bug/1467700 On Tue, Jun 23, 2015 at 10:07 AM Sergey Vasilenko svasile...@mirantis.com wrote: I notice that in OpenStack deployed by Fuel, Ceph public network is on management network. As I know separating cuph/public and management networks in a scope of 7.0 release. /sv __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -- Andrew Woodward Mirantis Fuel Community Ambassador Ceph Community __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev