Re: [openstack-dev] [Glance] Replacing Glance DB code to Oslo DB code.

2013-08-20 Thread Flavio Percoco
On 20/08/13 00:15 -0700, Mark Washenberger wrote: 2) I highly caution folks who think a No-SQL store is a good storage solution for any of the data currently used by Nova, Glance (registry), Cinder (registry), Ceilometer, and Quantum. All of the data stored and

Re: [openstack-dev] [Glance] Replacing Glance DB code to Oslo DB code.

2013-08-20 Thread Johannes Erdfelt
On Tue, Aug 20, 2013, Flavio Percoco fla...@redhat.com wrote: There are a couple of things that would worry me about an hypothetic support for NoSQL but I guess one that I'd consider very critical is migrations. Some could argue asking whether we'd really need them or not - when talking about

Re: [openstack-dev] [Glance] Replacing Glance DB code to Oslo DB code.

2013-08-20 Thread Mark Washenberger
On Tue, Aug 20, 2013 at 3:20 AM, Flavio Percoco fla...@redhat.com wrote: On 20/08/13 00:15 -0700, Mark Washenberger wrote: 2) I highly caution folks who think a No-SQL store is a good storage solution for any of the data currently used by Nova, Glance (registry),

Re: [openstack-dev] [Glance] Replacing Glance DB code to Oslo DB code.

2013-08-20 Thread John Bresnahan
Mark, good thoughts (as usual) On 08/19/2013 09:15 PM, Mark Washenberger wrote: The goal isn't really to replace sqlalchemy completely. Perhaps my problem is that I am not exactly sure what the goals are. Cleanup (BL mixed in the BL seems wrong)? HA or performance (are people hitting limits

Re: [openstack-dev] [Glance] Replacing Glance DB code to Oslo DB code.

2013-08-19 Thread Jay Pipes
On 08/18/2013 11:07 PM, Robert Collins wrote: On 19 August 2013 14:22, Jay Pipes jaypi...@gmail.com wrote: I'm completely with Joshua here - the ORM layer is more often than not a source of bugs and performance issues. If used improperly, yep.

Re: [openstack-dev] [Glance] Replacing Glance DB code to Oslo DB code.

2013-08-19 Thread Jay Pipes
On 08/19/2013 12:56 AM, Joshua Harlow wrote: Another good article from an ex-coworker that keeps on making more and more sense the more projects I get into... http://seldo.com/weblog/2011/08/11/orm_is_an_antipattern Your mileage/opinion though may vary :) I don't disagree with most of that

Re: [openstack-dev] [Glance] Replacing Glance DB code to Oslo DB code.

2013-08-19 Thread Robert Collins
On 19 August 2013 18:35, Jay Pipes jaypi...@gmail.com wrote: http://www.codinghorror.com/blog/2006/06/object-relational-mapping-is-the-vietnam-of-computer-science.html There is no proper use of an ORM. I'm not a super-fan of ORMs, Robert. I'm not sure why you're insisting on taking me down

Re: [openstack-dev] [Glance] Replacing Glance DB code to Oslo DB code.

2013-08-19 Thread Jay Pipes
On 08/18/2013 10:33 PM, Joe Gordon wrote: An alternative I think would be better would be to scrap the use of the SQLAlchemy ORM; keep using the DB engine abstraction support. +1, I am hoping this will provide noticeable performance benefits while being agnostic of what DB

Re: [openstack-dev] [Glance] Replacing Glance DB code to Oslo DB code.

2013-08-19 Thread Jay Pipes
OK, cool. I'm in agreement with your explained storage/logic separation below. Cheers, -jay On 08/19/2013 03:12 AM, Robert Collins wrote: On 19 August 2013 18:35, Jay Pipes jaypi...@gmail.com wrote:

Re: [openstack-dev] [Glance] Replacing Glance DB code to Oslo DB code.

2013-08-19 Thread Flavio Percoco
On 18/08/13 18:47 -0400, Jay Pipes wrote: On 08/18/2013 06:28 PM, Joe Gordon wrote: On Aug 18, 2013 3:58 PM, Jay Pipes jaypi...@gmail.com mailto:jaypi...@gmail.com wrote: On 08/18/2013 03:53 AM, Joshua Harlow wrote: I always just liked SQL as the database abstraction layer ;) On a more

Re: [openstack-dev] [Glance] Replacing Glance DB code to Oslo DB code.

2013-08-19 Thread John Bresnahan
All I'm saying is that we should be careful not to swap one set of problems for another. My 2 cents: I am in agreement with Jay. I am leery of NoSQL being a direct sub in and I fear that this effort can be adding a large workload for little benefit. A somewhat related post:

Re: [openstack-dev] [Glance] Replacing Glance DB code to Oslo DB code.

2013-08-19 Thread Boris Pavlovic
Mark, But for a variety of reasons, I do not consider the general thrust of use oslo db code to be approved. Instead, lets continue to consider features from olso db on a case by case basis, and see what the right resolution is in each case. Absolutely agree with this point (e.g. we removed

Re: [openstack-dev] [Glance] Replacing Glance DB code to Oslo DB code.

2013-08-19 Thread Joshua Harlow
, August 19, 2013 2:12 PM To: OpenStack Development Mailing List openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [Glance] Replacing Glance DB code to Oslo DB code. Mark, But for a variety of reasons, I do not consider the general thrust of use oslo

Re: [openstack-dev] [Glance] Replacing Glance DB code to Oslo DB code.

2013-08-19 Thread Ben Nemec
On 08/19/13 20:34, Joshua Harlow wrote: Just a related question, Oslo 'incubator' db code I think depends on eventlet. This means any code that uses the oslo.db code could/would(?) be dependent on eventlet. Will there be some refactoring there to not require it (useful for projects

Re: [openstack-dev] [Glance] Replacing Glance DB code to Oslo DB code.

2013-08-18 Thread Jay Pipes
On 08/18/2013 03:53 AM, Joshua Harlow wrote: I always just liked SQL as the database abstraction layer ;) On a more serious note I think novas new object model might be a way to go but in all honesty there won't be a one size fits all solution. I just don't think sqlalchemy is that solution

Re: [openstack-dev] [Glance] Replacing Glance DB code to Oslo DB code.

2013-08-18 Thread Joshua Harlow
In my opinion (and just an opinion that I know everyone doesn't share) ORM layers are bulky, restrictive and overly complicate and confuse the reader of the code (code is read more often than written) and require another layer of understanding (a layer is useful if it adds good value, I am not

Re: [openstack-dev] [Glance] Replacing Glance DB code to Oslo DB code.

2013-08-18 Thread Joe Gordon
On Aug 18, 2013 3:58 PM, Jay Pipes jaypi...@gmail.com wrote: On 08/18/2013 03:53 AM, Joshua Harlow wrote: I always just liked SQL as the database abstraction layer ;) On a more serious note I think novas new object model might be a way to go but in all honesty there won't be a one size fits

Re: [openstack-dev] [Glance] Replacing Glance DB code to Oslo DB code.

2013-08-18 Thread Jay Pipes
On 08/18/2013 06:28 PM, Joe Gordon wrote: On Aug 18, 2013 3:58 PM, Jay Pipes jaypi...@gmail.com mailto:jaypi...@gmail.com wrote: On 08/18/2013 03:53 AM, Joshua Harlow wrote: I always just liked SQL as the database abstraction layer ;) On a more serious note I think novas new object

Re: [openstack-dev] [Glance] Replacing Glance DB code to Oslo DB code.

2013-08-18 Thread Joshua Harlow
It would be neat to see what would happen if just the raw models were just used directly. Of course this must be treaded careful since I could see it spreading db logic all over. +1 for turning off deferred loads, I think this encourages and actually hides bugs when lazy loads occur on demand.

Re: [openstack-dev] [Glance] Replacing Glance DB code to Oslo DB code.

2013-08-18 Thread Joshua Harlow
Using an ORM how does the ORM know what attributes u might access (forgive me if this is a documented sqlalchemy pattern/solution). Doesn't it have to give u back the full model since the ORM layer can't predict what u might do with the model object? Sent from my really tiny device... On Aug

Re: [openstack-dev] [Glance] Replacing Glance DB code to Oslo DB code.

2013-08-18 Thread Robert Collins
On 19 August 2013 10:43, Jay Pipes jaypi...@gmail.com wrote: On 08/18/2013 06:08 PM, Joshua Harlow wrote: In my opinion (and just an opinion that I know everyone doesn't share) ORM layers are bulky, restrictive and overly complicate and confuse the reader of the code (code is read more often

Re: [openstack-dev] [Glance] Replacing Glance DB code to Oslo DB code.

2013-08-18 Thread Jay Pipes
On 08/18/2013 07:44 PM, Joshua Harlow wrote: Using an ORM how does the ORM know what attributes u might access (forgive me if this is a documented sqlalchemy pattern/solution). Doesn't it have to give u back the full model since the ORM layer can't predict what u might do with the model

Re: [openstack-dev] [Glance] Replacing Glance DB code to Oslo DB code.

2013-08-18 Thread Robert Collins
On 19 August 2013 14:22, Jay Pipes jaypi...@gmail.com wrote: I'm completely with Joshua here - the ORM layer is more often than not a source of bugs and performance issues. If used improperly, yep.

Re: [openstack-dev] [Glance] Replacing Glance DB code to Oslo DB code.

2013-08-18 Thread Joshua Harlow
It will be I interesting to see how it works out in nova, correct me if I am wrong but nova has even more onion layers than other openstack projects. For ex: Nova compute -unified object model -rpc-conductor-sqlalchemy ORM model-SQL-your db Is nova moving away from the ORM model or is the

Re: [openstack-dev] [Glance] Replacing Glance DB code to Oslo DB code.

2013-08-18 Thread Joshua Harlow
Another good article from an ex-coworker that keeps on making more and more sense the more projects I get into... http://seldo.com/weblog/2011/08/11/orm_is_an_antipattern Your mileage/opinion though may vary :) Sent from my really tiny device... On Aug 18, 2013, at 8:12 PM, Robert Collins

[openstack-dev] [Glance] Replacing Glance DB code to Oslo DB code.

2013-08-16 Thread Victor Sergeyev
Hello All. Glance cores (Mark Washenberger, Flavio Percoco, Iccha Sethi) have some questions about Oslo DB code, and why is it so important to use it instead of custom implementation and so on. As there were a lot of questions it was really hard to answer on all this questions in IRC. So we

Re: [openstack-dev] [Glance] Replacing Glance DB code to Oslo DB code.

2013-08-16 Thread Monty Taylor
On 08/16/2013 09:31 AM, Victor Sergeyev wrote: Hello All. Glance cores (Mark Washenberger, Flavio Percoco, Iccha Sethi) have some questions about Oslo DB code, and why is it so important to use it instead of custom implementation and so on. As there were a lot of questions it was really

Re: [openstack-dev] [Glance] Replacing Glance DB code to Oslo DB code.

2013-08-16 Thread Eric Windisch
On Fri, Aug 16, 2013 at 9:31 AM, Victor Sergeyev vserge...@mirantis.com wrote: Hello All. Glance cores (Mark Washenberger, Flavio Percoco, Iccha Sethi) have some questions about Oslo DB code, and why is it so important to use it instead of custom implementation and so on. As there were a lot

Re: [openstack-dev] [Glance] Replacing Glance DB code to Oslo DB code.

2013-08-16 Thread Flavio Percoco
On 16/08/13 11:42 -0400, Monty Taylor wrote: On 08/16/2013 09:31 AM, Victor Sergeyev wrote: Hello All. Glance cores (Mark Washenberger, Flavio Percoco, Iccha Sethi) have some questions about Oslo DB code, and why is it so important to use it instead of custom implementation and so on. As

Re: [openstack-dev] [Glance] Replacing Glance DB code to Oslo DB code.

2013-08-16 Thread Mark Washenberger
I would prefer to pick and choose which parts of oslo common db code to reuse in glance. Most parts there look great and very useful. However, some parts seem like they would conflict with several goals we have. 1) To improve code sanity, we need to break away from the idea of having one giant db

Re: [openstack-dev] [Glance] Replacing Glance DB code to Oslo DB code.

2013-08-16 Thread Jay Pipes
On 08/16/2013 02:41 PM, Mark Washenberger wrote: I think the issue here for glance is whether or not oslo common code makes it easier or harder to make other planned improvements. In particular, using openstack.common.db.api will make it harder to refactor away from a giant procedural interface