Re: [openstack-dev] [tripleo] Default the HA scenario to Ceph

2016-10-13 Thread Paul Dardeau 
RGW does not use Swift. It’s an optional component of Ceph that can provide a 
Swift compliant API (and also S3). But it does not make use of openstack-swift. 
It’s a completely separate implementation written in c++ [1].

[1] https://github.com/ceph/ceph/blob/master/src/rgw/rgw_rest_swift.cc

> On Oct 13, 2016, at 5:36 AM, Shinobu Kinjo  wrote:
> 
> 
> 
> On Wed, Oct 12, 2016 at 9:59 PM, Giulio Fidente  > wrote:
> On 10/12/2016 02:29 PM, Thiago da Silva wrote:
> 
> 
> On 10/12/2016 07:10 AM, Giulio Fidente wrote:
> hi,
> 
> we introduced support for the deployment of Ceph in the liberty
> release so that it could optionally be used as backend for one or more
> of Cinder, Glance, Nova and more recently Gnocchi.
> 
> We used to deploy Ceph MONs on the controller nodes and Ceph OSDs on
> dedicated ceph-storage nodes so a deployment of OpenStack with Ceph
> would need at least 1 more additional node to host a Ceph OSD.
> 
> In our HA scenario the storage backends are configured as follows:
> 
> Glance -> Swift
> Nova (ephemeral) -> Local
> Cinder (persistent) -> LVM (on controllers)
> Gnocchi -> Swift
> 
> The downside of the above configuration is that Cinder volumes can not
> be replicated across the controller nodes and become unavailable if a
> controller fails, while production environments generally expect
> persistent storage to be highly available. Cinder volumes instead
> could even get lost completely in case of a permanent failure of a
> controller.
> 
> With the Newton release and the composable roles we can now deploy
> Ceph OSDs on the compute nodes, removing the requirement we had for an
> additional node to host a Ceph OSD.
> 
> I would like to ask for some feedback on the possibility of deploying
> Ceph by default in the HA scenario and use it as backend for Cinder.
> 
> Also using Swift as backend for Glance and Gnocchi is enough to cover
> the availability issue for the data, but it also means we're storing
> that data on the controller nodes which might or might not be wanted;
> I don't see a strong reason for defaulting them to Ceph, but it might
> make more sense when Ceph is available; feedback about this would be
> appreciated as well.
> I think it would be important to take into account the recently created
> guiding principles [0]:
> 
> "While the software that OpenStack produces has well defined and
> documented APIs, the primary output of OpenStack is software, not API
> definitions. We expect people who say they run “OpenStack” to run the
> software produced by and in the community, rather than alternative
> implementations of the API."
> 
> In the case of Cinder, I think the situation is a bit muddy as LVM is
> not openstack software, and my limited understanding is that LVM is used
> as a reference implementation, but in the case of Swift, I think RGW
> would be considered an 'alternative implementation of the API'.
> 
> Thiago
> 
> hi Thiago,
> 
> sorry if it wasn't clear in my original message but I did not suggest to 
> replace Swift with Ceph RGW.
> 
> Swift would continue to be deployed by default, not RGW.
> 
> RGW utilizes Swift. So Swift has to be there anyway -;
>  
> 
> The feedback I'm asking for is about storing (or not) the Cinder volumes in 
> Ceph for the HA scenario by default, and also store the Glance images and 
> Gnocchi metrics in Ceph or rather keep that data in Swift.
> -- 
> Giulio Fidente
> GPG KEY: 08D733BA
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe 
> 
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev 
> 
> 
> 
> 
> -- 
> Email:
> shin...@linux.com 
> shin...@redhat.com 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] Default the HA scenario to Ceph

2016-10-13 Thread Shinobu Kinjo 
On Wed, Oct 12, 2016 at 9:59 PM, Giulio Fidente  wrote:

> On 10/12/2016 02:29 PM, Thiago da Silva wrote:
>
>>
>>
>> On 10/12/2016 07:10 AM, Giulio Fidente wrote:
>>
>>> hi,
>>>
>>> we introduced support for the deployment of Ceph in the liberty
>>> release so that it could optionally be used as backend for one or more
>>> of Cinder, Glance, Nova and more recently Gnocchi.
>>>
>>> We used to deploy Ceph MONs on the controller nodes and Ceph OSDs on
>>> dedicated ceph-storage nodes so a deployment of OpenStack with Ceph
>>> would need at least 1 more additional node to host a Ceph OSD.
>>>
>>> In our HA scenario the storage backends are configured as follows:
>>>
>>> Glance -> Swift
>>> Nova (ephemeral) -> Local
>>> Cinder (persistent) -> LVM (on controllers)
>>> Gnocchi -> Swift
>>>
>>> The downside of the above configuration is that Cinder volumes can not
>>> be replicated across the controller nodes and become unavailable if a
>>> controller fails, while production environments generally expect
>>> persistent storage to be highly available. Cinder volumes instead
>>> could even get lost completely in case of a permanent failure of a
>>> controller.
>>>
>>> With the Newton release and the composable roles we can now deploy
>>> Ceph OSDs on the compute nodes, removing the requirement we had for an
>>> additional node to host a Ceph OSD.
>>>
>>> I would like to ask for some feedback on the possibility of deploying
>>> Ceph by default in the HA scenario and use it as backend for Cinder.
>>>
>>> Also using Swift as backend for Glance and Gnocchi is enough to cover
>>> the availability issue for the data, but it also means we're storing
>>> that data on the controller nodes which might or might not be wanted;
>>> I don't see a strong reason for defaulting them to Ceph, but it might
>>> make more sense when Ceph is available; feedback about this would be
>>> appreciated as well.
>>>
>> I think it would be important to take into account the recently created
>> guiding principles [0]:
>>
>> "While the software that OpenStack produces has well defined and
>> documented APIs, the primary output of OpenStack is software, not API
>> definitions. We expect people who say they run “OpenStack” to run the
>> software produced by and in the community, rather than alternative
>> implementations of the API."
>>
>> In the case of Cinder, I think the situation is a bit muddy as LVM is
>> not openstack software, and my limited understanding is that LVM is used
>> as a reference implementation, but in the case of Swift, I think RGW
>> would be considered an 'alternative implementation of the API'.
>>
>> Thiago
>>
>
> hi Thiago,
>
> sorry if it wasn't clear in my original message but I did not suggest to
> replace Swift with Ceph RGW.
>
> Swift would continue to be deployed by default, not RGW.
>

RGW utilizes Swift. So Swift has to be there anyway -;


>
> The feedback I'm asking for is about storing (or not) the Cinder volumes
> in Ceph for the HA scenario by default, and also store the Glance images
> and Gnocchi metrics in Ceph or rather keep that data in Swift.
> --
> Giulio Fidente
> GPG KEY: 08D733BA
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Email:
shin...@linux.com
shin...@redhat.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] Default the HA scenario to Ceph

2016-10-12 Thread Pradeep Kilambi 
On Wednesday, October 12, 2016, Emilien Macchi  wrote:

> On Wed, Oct 12, 2016 at 7:10 AM, Giulio Fidente  > wrote:
> > hi,
> >
> > we introduced support for the deployment of Ceph in the liberty release
> so
> > that it could optionally be used as backend for one or more of Cinder,
> > Glance, Nova and more recently Gnocchi.
> >
> > We used to deploy Ceph MONs on the controller nodes and Ceph OSDs on
> > dedicated ceph-storage nodes so a deployment of OpenStack with Ceph would
> > need at least 1 more additional node to host a Ceph OSD.
> >
> > In our HA scenario the storage backends are configured as follows:
> >
> > Glance -> Swift
> > Nova (ephemeral) -> Local
> > Cinder (persistent) -> LVM (on controllers)
> > Gnocchi -> Swift
> >
> > The downside of the above configuration is that Cinder volumes can not be
> > replicated across the controller nodes and become unavailable if a
> > controller fails, while production environments generally expect
> persistent
> > storage to be highly available. Cinder volumes instead could even get
> lost
> > completely in case of a permanent failure of a controller.
> >
> > With the Newton release and the composable roles we can now deploy Ceph
> OSDs
> > on the compute nodes, removing the requirement we had for an additional
> node
> > to host a Ceph OSD.
> >
> > I would like to ask for some feedback on the possibility of deploying
> Ceph
> > by default in the HA scenario and use it as backend for Cinder.
> >
> > Also using Swift as backend for Glance and Gnocchi is enough to cover the
> > availability issue for the data, but it also means we're storing that
> data
> > on the controller nodes which might or might not be wanted; I don't see a
> > strong reason for defaulting them to Ceph, but it might make more sense
> when
> > Ceph is available; feedback about this would be appreciated as well.
> >
> > Finally a shared backend (Ceph) for Nova would allow live migrations but
> > probably decrease performances for the guests in general; so I'd be
> against
> > defaulting Nova to Ceph. Feedback?
>
> +1 on making ceph default backend for nova/glance/cinder/gnocchi.
> I think this is the most common use-case we currently have in our
> deployments AFIK.


 + 1 from me. Ceph is the recommended backend for gnocchi and this will
help a lot with some recent performance issue we have seen.

- Prad



> Also, I'll continue to work on scenarios jobs (scenario002 and
> scenario003 without Ceph to cover other use cases).
>
> > --
> > Giulio Fidente
> > GPG KEY: 08D733BA
> >
> > 
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:
> unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> --
> Emilien Macchi
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] Default the HA scenario to Ceph

2016-10-12 Thread Emilien Macchi 
On Wed, Oct 12, 2016 at 7:10 AM, Giulio Fidente  wrote:
> hi,
>
> we introduced support for the deployment of Ceph in the liberty release so
> that it could optionally be used as backend for one or more of Cinder,
> Glance, Nova and more recently Gnocchi.
>
> We used to deploy Ceph MONs on the controller nodes and Ceph OSDs on
> dedicated ceph-storage nodes so a deployment of OpenStack with Ceph would
> need at least 1 more additional node to host a Ceph OSD.
>
> In our HA scenario the storage backends are configured as follows:
>
> Glance -> Swift
> Nova (ephemeral) -> Local
> Cinder (persistent) -> LVM (on controllers)
> Gnocchi -> Swift
>
> The downside of the above configuration is that Cinder volumes can not be
> replicated across the controller nodes and become unavailable if a
> controller fails, while production environments generally expect persistent
> storage to be highly available. Cinder volumes instead could even get lost
> completely in case of a permanent failure of a controller.
>
> With the Newton release and the composable roles we can now deploy Ceph OSDs
> on the compute nodes, removing the requirement we had for an additional node
> to host a Ceph OSD.
>
> I would like to ask for some feedback on the possibility of deploying Ceph
> by default in the HA scenario and use it as backend for Cinder.
>
> Also using Swift as backend for Glance and Gnocchi is enough to cover the
> availability issue for the data, but it also means we're storing that data
> on the controller nodes which might or might not be wanted; I don't see a
> strong reason for defaulting them to Ceph, but it might make more sense when
> Ceph is available; feedback about this would be appreciated as well.
>
> Finally a shared backend (Ceph) for Nova would allow live migrations but
> probably decrease performances for the guests in general; so I'd be against
> defaulting Nova to Ceph. Feedback?

+1 on making ceph default backend for nova/glance/cinder/gnocchi.
I think this is the most common use-case we currently have in our
deployments AFIK.

Also, I'll continue to work on scenarios jobs (scenario002 and
scenario003 without Ceph to cover other use cases).

> --
> Giulio Fidente
> GPG KEY: 08D733BA
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] Default the HA scenario to Ceph

2016-10-12 Thread Ben Nemec 



On 10/12/2016 06:10 AM, Giulio Fidente wrote:

hi,

we introduced support for the deployment of Ceph in the liberty release
so that it could optionally be used as backend for one or more of
Cinder, Glance, Nova and more recently Gnocchi.

We used to deploy Ceph MONs on the controller nodes and Ceph OSDs on
dedicated ceph-storage nodes so a deployment of OpenStack with Ceph
would need at least 1 more additional node to host a Ceph OSD.

In our HA scenario the storage backends are configured as follows:

Glance -> Swift
Nova (ephemeral) -> Local
Cinder (persistent) -> LVM (on controllers)
Gnocchi -> Swift

The downside of the above configuration is that Cinder volumes can not
be replicated across the controller nodes and become unavailable if a
controller fails, while production environments generally expect
persistent storage to be highly available. Cinder volumes instead could
even get lost completely in case of a permanent failure of a controller.

With the Newton release and the composable roles we can now deploy Ceph
OSDs on the compute nodes, removing the requirement we had for an
additional node to host a Ceph OSD.

I would like to ask for some feedback on the possibility of deploying
Ceph by default in the HA scenario and use it as backend for Cinder.


+1 from me.  It sounds like our current default is inappropriate for an 
HA environment anyway, so if someone is using it they're already broken 
by design.  Hopefully everyone is already setting up Ceph or some other 
shared storage backend in HA so changing the default should be largely a 
non-event.  Obviously we would still need to provide an upgrade path for 
anyone who did deploy the old default (maybe they don't use Cinder and 
don't care if it's HA, for example).




Also using Swift as backend for Glance and Gnocchi is enough to cover
the availability issue for the data, but it also means we're storing
that data on the controller nodes which might or might not be wanted; I
don't see a strong reason for defaulting them to Ceph, but it might make
more sense when Ceph is available; feedback about this would be
appreciated as well.

Finally a shared backend (Ceph) for Nova would allow live migrations but
probably decrease performances for the guests in general; so I'd be
against defaulting Nova to Ceph. Feedback?


Agreed.  It's simple enough for people to set Nova to use Ceph if they 
want, but if people haven't spec'd their compute nodes to handle heavy 
converged Ceph usage I suspect performance would be unacceptable for VMs.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] Default the HA scenario to Ceph

2016-10-12 Thread Giulio Fidente 

On 10/12/2016 02:29 PM, Thiago da Silva wrote:



On 10/12/2016 07:10 AM, Giulio Fidente wrote:

hi,

we introduced support for the deployment of Ceph in the liberty
release so that it could optionally be used as backend for one or more
of Cinder, Glance, Nova and more recently Gnocchi.

We used to deploy Ceph MONs on the controller nodes and Ceph OSDs on
dedicated ceph-storage nodes so a deployment of OpenStack with Ceph
would need at least 1 more additional node to host a Ceph OSD.

In our HA scenario the storage backends are configured as follows:

Glance -> Swift
Nova (ephemeral) -> Local
Cinder (persistent) -> LVM (on controllers)
Gnocchi -> Swift

The downside of the above configuration is that Cinder volumes can not
be replicated across the controller nodes and become unavailable if a
controller fails, while production environments generally expect
persistent storage to be highly available. Cinder volumes instead
could even get lost completely in case of a permanent failure of a
controller.

With the Newton release and the composable roles we can now deploy
Ceph OSDs on the compute nodes, removing the requirement we had for an
additional node to host a Ceph OSD.

I would like to ask for some feedback on the possibility of deploying
Ceph by default in the HA scenario and use it as backend for Cinder.

Also using Swift as backend for Glance and Gnocchi is enough to cover
the availability issue for the data, but it also means we're storing
that data on the controller nodes which might or might not be wanted;
I don't see a strong reason for defaulting them to Ceph, but it might
make more sense when Ceph is available; feedback about this would be
appreciated as well.

I think it would be important to take into account the recently created
guiding principles [0]:

"While the software that OpenStack produces has well defined and
documented APIs, the primary output of OpenStack is software, not API
definitions. We expect people who say they run “OpenStack” to run the
software produced by and in the community, rather than alternative
implementations of the API."

In the case of Cinder, I think the situation is a bit muddy as LVM is
not openstack software, and my limited understanding is that LVM is used
as a reference implementation, but in the case of Swift, I think RGW
would be considered an 'alternative implementation of the API'.

Thiago


hi Thiago,

sorry if it wasn't clear in my original message but I did not suggest to 
replace Swift with Ceph RGW.


Swift would continue to be deployed by default, not RGW.

The feedback I'm asking for is about storing (or not) the Cinder volumes 
in Ceph for the HA scenario by default, and also store the Glance images 
and Gnocchi metrics in Ceph or rather keep that data in Swift.

--
Giulio Fidente
GPG KEY: 08D733BA

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] Default the HA scenario to Ceph

2016-10-12 Thread Thiago da Silva 



On 10/12/2016 07:10 AM, Giulio Fidente wrote:

hi,

we introduced support for the deployment of Ceph in the liberty 
release so that it could optionally be used as backend for one or more 
of Cinder, Glance, Nova and more recently Gnocchi.


We used to deploy Ceph MONs on the controller nodes and Ceph OSDs on 
dedicated ceph-storage nodes so a deployment of OpenStack with Ceph 
would need at least 1 more additional node to host a Ceph OSD.


In our HA scenario the storage backends are configured as follows:

Glance -> Swift
Nova (ephemeral) -> Local
Cinder (persistent) -> LVM (on controllers)
Gnocchi -> Swift

The downside of the above configuration is that Cinder volumes can not 
be replicated across the controller nodes and become unavailable if a 
controller fails, while production environments generally expect 
persistent storage to be highly available. Cinder volumes instead 
could even get lost completely in case of a permanent failure of a 
controller.


With the Newton release and the composable roles we can now deploy 
Ceph OSDs on the compute nodes, removing the requirement we had for an 
additional node to host a Ceph OSD.


I would like to ask for some feedback on the possibility of deploying 
Ceph by default in the HA scenario and use it as backend for Cinder.


Also using Swift as backend for Glance and Gnocchi is enough to cover 
the availability issue for the data, but it also means we're storing 
that data on the controller nodes which might or might not be wanted; 
I don't see a strong reason for defaulting them to Ceph, but it might 
make more sense when Ceph is available; feedback about this would be 
appreciated as well.
I think it would be important to take into account the recently created 
guiding principles [0]:


"While the software that OpenStack produces has well defined and 
documented APIs, the primary output of OpenStack is software, not API 
definitions. We expect people who say they run “OpenStack” to run the 
software produced by and in the community, rather than alternative 
implementations of the API."


In the case of Cinder, I think the situation is a bit muddy as LVM is 
not openstack software, and my limited understanding is that LVM is used 
as a reference implementation, but in the case of Swift, I think RGW 
would be considered an 'alternative implementation of the API'.


Thiago

[0] - 
https://governance.openstack.org/reference/principles.html#openstack-primarily-produces-software


Finally a shared backend (Ceph) for Nova would allow live migrations 
but probably decrease performances for the guests in general; so I'd 
be against defaulting Nova to Ceph. Feedback?



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] Default the HA scenario to Ceph

2016-10-12 Thread Julien Danjou 
On Wed, Oct 12 2016, Giulio Fidente wrote:

> Also using Swift as backend for Glance and Gnocchi is enough to cover the
> availability issue for the data, but it also means we're storing that data on
> the controller nodes which might or might not be wanted; I don't see a strong
> reason for defaulting them to Ceph, but it might make more sense when Ceph is
> available; feedback about this would be appreciated as well.

The Ceph driver of Gnocchi is better than the Swift one, so it'd make
total sense to default to Ceph if Ceph is available.

-- 
Julien Danjou
// Free Software hacker
// https://julien.danjou.info


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [tripleo] Default the HA scenario to Ceph

2016-10-12 Thread Giulio Fidente 

hi,

we introduced support for the deployment of Ceph in the liberty release 
so that it could optionally be used as backend for one or more of 
Cinder, Glance, Nova and more recently Gnocchi.


We used to deploy Ceph MONs on the controller nodes and Ceph OSDs on 
dedicated ceph-storage nodes so a deployment of OpenStack with Ceph 
would need at least 1 more additional node to host a Ceph OSD.


In our HA scenario the storage backends are configured as follows:

Glance -> Swift
Nova (ephemeral) -> Local
Cinder (persistent) -> LVM (on controllers)
Gnocchi -> Swift

The downside of the above configuration is that Cinder volumes can not 
be replicated across the controller nodes and become unavailable if a 
controller fails, while production environments generally expect 
persistent storage to be highly available. Cinder volumes instead could 
even get lost completely in case of a permanent failure of a controller.


With the Newton release and the composable roles we can now deploy Ceph 
OSDs on the compute nodes, removing the requirement we had for an 
additional node to host a Ceph OSD.


I would like to ask for some feedback on the possibility of deploying 
Ceph by default in the HA scenario and use it as backend for Cinder.


Also using Swift as backend for Glance and Gnocchi is enough to cover 
the availability issue for the data, but it also means we're storing 
that data on the controller nodes which might or might not be wanted; I 
don't see a strong reason for defaulting them to Ceph, but it might make 
more sense when Ceph is available; feedback about this would be 
appreciated as well.


Finally a shared backend (Ceph) for Nova would allow live migrations but 
probably decrease performances for the guests in general; so I'd be 
against defaulting Nova to Ceph. Feedback?

--
Giulio Fidente
GPG KEY: 08D733BA

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev