Re: [openstack-dev] [Nova] Handling soft delete for instance rows in a new cells database

2014-11-28 Thread Jay Pipes
On 11/27/2014 04:20 PM, Michael Still wrote: On Fri, Nov 28, 2014 at 2:59 AM, Jay Pipes wrote: On 11/26/2014 04:24 PM, Mike Bayer wrote: Precisely. Why is the RDBMS the thing that is used for archival/audit logging? Why not a NoSQL store or a centralized log facility? All that would be needed

Re: [openstack-dev] [Nova] Handling soft delete for instance rows in a new cells database

2014-11-27 Thread Michael Still
On Fri, Nov 28, 2014 at 2:59 AM, Jay Pipes wrote: > On 11/26/2014 04:24 PM, Mike Bayer wrote: >>> >>> Precisely. Why is the RDBMS the thing that is used for >>> archival/audit logging? Why not a NoSQL store or a centralized log >>> facility? All that would be needed would be for us to standardize

Re: [openstack-dev] [Nova] Handling soft delete for instance rows in a new cells database

2014-11-27 Thread Jay Pipes
On 11/26/2014 04:24 PM, Mike Bayer wrote: Precisely. Why is the RDBMS the thing that is used for archival/audit logging? Why not a NoSQL store or a centralized log facility? All that would be needed would be for us to standardize on the format of the archival record, standardize on the things to

Re: [openstack-dev] [Nova] Handling soft delete for instance rows in a new cells database

2014-11-26 Thread Joshua Harlow
Mike Bayer wrote: Precisely. Why is the RDBMS the thing that is used for archival/audit logging? Why not a NoSQL store or a centralized log facility? All that would be needed would be for us to standardize on the format of the archival record, standardize on the things to provide with the arch

Re: [openstack-dev] [Nova] Handling soft delete for instance rows in a new cells database

2014-11-26 Thread Andrew Laski
On 11/26/2014 03:39 PM, Belmiro Moreira wrote: Hi, my experience is that "soft delete" is important to keep record of deleted instances and its characteristics. In fact in my organization we are obliged to keep these records for several months. However, it would be nice that after few months w

Re: [openstack-dev] [Nova] Handling soft delete for instance rows in a new cells database

2014-11-26 Thread Mike Bayer
> > Precisely. Why is the RDBMS the thing that is used for archival/audit > logging? Why not a NoSQL store or a centralized log facility? All that would > be needed would be for us to standardize on the format of the archival > record, standardize on the things to provide with the archival rec

Re: [openstack-dev] [Nova] Handling soft delete for instance rows in a new cells database

2014-11-26 Thread Belmiro Moreira
Hi, my experience is that "soft delete" is important to keep record of deleted instances and its characteristics. In fact in my organization we are obliged to keep these records for several months. However, it would be nice that after few months we were able to purge the DB with a nova tool. In th

Re: [openstack-dev] [Nova] Handling soft delete for instance rows in a new cells database

2014-11-26 Thread Jay Pipes
On 11/25/2014 09:34 PM, Mike Bayer wrote: On Nov 25, 2014, at 8:15 PM, Ahmed RAHAL wrote: Hi, Le 2014-11-24 17:20, Michael Still a écrit : Heya, This is a new database, so its our big chance to get this right. So, ideas welcome... Some initial proposals: - we do what we do in the current

Re: [openstack-dev] [Nova] Handling soft delete for instance rows in a new cells database

2014-11-26 Thread Andrew Laski
On 11/25/2014 11:54 AM, Solly Ross wrote: I can't comment on other projects, but Nova definitely needs the soft delete in the main nova database. Perhaps not for every table, but there is definitely code in the code base which uses it right now. Search for read_deleted=True if you're curious. J

Re: [openstack-dev] [Nova] Handling soft delete for instance rows in a new cells database

2014-11-25 Thread Mike Bayer
> On Nov 25, 2014, at 8:15 PM, Ahmed RAHAL wrote: > > Hi, > > Le 2014-11-24 17:20, Michael Still a écrit : >> Heya, >> >> This is a new database, so its our big chance to get this right. So, >> ideas welcome... >> >> Some initial proposals: >> >> - we do what we do in the current nova datab

Re: [openstack-dev] [Nova] Handling soft delete for instance rows in a new cells database

2014-11-25 Thread Ahmed RAHAL
Hi, Le 2014-11-24 17:20, Michael Still a écrit : Heya, This is a new database, so its our big chance to get this right. So, ideas welcome... Some initial proposals: - we do what we do in the current nova database -- we have a deleted column, and we set it to true when we delete the instance

Re: [openstack-dev] [Nova] Handling soft delete for instance rows in a new cells database

2014-11-25 Thread Solly Ross
> I can't comment on other projects, but Nova definitely needs the soft > delete in the main nova database. Perhaps not for every table, but > there is definitely code in the code base which uses it right now. > Search for read_deleted=True if you're curious. Just to save people a bit of time, it'

Re: [openstack-dev] [Nova] Handling soft delete for instance rows in a new cells database

2014-11-24 Thread Mike Bayer
> On Nov 24, 2014, at 7:32 PM, Michael Still wrote: > > Interesting. I hadn't seen consistency between the two databases as > trumping doing this less horribly, but it sounds like its more of a > thing that I thought. it really depends on what you need to do. if you need to get a result set of

Re: [openstack-dev] [Nova] Handling soft delete for instance rows in a new cells database

2014-11-24 Thread Michael Still
On Tue, Nov 25, 2014 at 11:14 AM, Mike Bayer wrote: > >> On Nov 24, 2014, at 5:20 PM, Michael Still wrote: >> >> Heya, >> >> Review https://review.openstack.org/#/c/135644/4 proposes the addition >> of a new database for our improved implementation of cells in Nova. >> However, there's an outstan

Re: [openstack-dev] [Nova] Handling soft delete for instance rows in a new cells database

2014-11-24 Thread Mike Bayer
> On Nov 24, 2014, at 5:20 PM, Michael Still wrote: > > Heya, > > Review https://review.openstack.org/#/c/135644/4 proposes the addition > of a new database for our improved implementation of cells in Nova. > However, there's an outstanding question about how to handle soft > delete of rows --

Re: [openstack-dev] [Nova] Handling soft delete for instance rows in a new cells database

2014-11-24 Thread Kevin L. Mitchell
On Tue, 2014-11-25 at 09:20 +1100, Michael Still wrote: > - we do what we do in the current nova database -- we have a deleted > column, and we set it to true when we delete the instance. Actually, current nova uses the oslo.db.sqlalchemy.models.SoftDeleteMixin class, which defines the columns 'd

[openstack-dev] [Nova] Handling soft delete for instance rows in a new cells database

2014-11-24 Thread Michael Still
Heya, Review https://review.openstack.org/#/c/135644/4 proposes the addition of a new database for our improved implementation of cells in Nova. However, there's an outstanding question about how to handle soft delete of rows -- we believe that we need to soft delete for forensic purposes. This i