[Openstack] openstack-common PTL candidacy
Hi, I'd like to put my name forward as a candidate for openstack-common PTL. I helped start the project with Jason Kölker in January and wrote the plan we've been following: http://wiki.openstack.org/CommonLibrary Since then, I've been doing reviews, triaging bugs and organizing the blueprints for the project. At the Folsom Design Summit, I lead several sessions on openstack-common related topics. I also wrote the cfg API and worked to ensure its use across OpenStack. Recently, the PPB decided that openstack-common should officially have a PTL and I was appointed to the position in the interim. As elected PTL, I would increase my focus on moving the project forward. My goals for the Grizzly cycle would be: - Finalize the renaming of the project to Oslo - Rename the openstack-common repo to oslo-incubator - Promote the cfg and rpc APIs out of incubation - Agree on a versioning scheme for oslo-* releases - Release the first versions of the oslo-config and oslo-messaging packages - Help with merging services infrastructure, WSGI and DB APIs into the incubator - Help OpenStack projects adopt existing incubating APIs - Clear out some of the unused/stale APIs out of the incubator Like all other PTLs, I would also do my best to encourage new contributors, guide the development of new features, review patches, triage bugs and serve as a contact point for the project. Thanks! Mark ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] openstack-common PTL candidacy
Mark McLoughlin wrote: I'd like to put my name forward as a candidate for openstack-common PTL. As an election official, I confirm that you're eligible to that position. -- Thierry Carrez (ttx) Release Manager, OpenStack ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Openstack-Common] RPC
On 07/10/2012 12:49 PM, Gary Kotton wrote: On 07/10/2012 06:29 PM, Eric Windisch wrote: 2. I based my integration on the patch https://review.openstack.org/#/c/9166/. A number of files were missing. Should this have specifically mentioned the missing files or should the rpc part have taken care of this? Are you concerned that the RPC module itself has dependencies on openstack-common which are not being automatically resolved and included when you run update.py? Thanks, I was reviewing a patch that did the update. There were some files missing. I'll try and check. Ok, it was probably just a failure to 'git add' everything. However, if you guys discover a problem with update.py not copying everything, please report it against openstack-common so we can fix it up. -- Russell Bryant ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [Openstack-Common] RPC
Hi,
I am in the process of integrating the RPC code from OpenStack common
into Quantum. I initially started working with qpid as the backend
implementation. I ran into problems due to the fact that
control_exchange is defined as 'nova'. This is in
quantum/openstack/common/rpc/__init__.py where the following is defined.
cfg.StrOpt('control_exchange',
default='nova',
help='AMQP exchange to connect to if using RabbitMQ or
Qpid'),
When I changed this to 'quantum' it worked. My topic was defined as
'quantum'.
In addition to this I have a few additional questions and or concerns:
1. When we import code from openstack common the test cases for the
modules are not imported (maybe I missed something with running setup).
When the code is copied the imports are updated. It would be nice to
know that the auto tests are also run in the context of the project
importing the code.
2. I based my integration on the patch
https://review.openstack.org/#/c/9166/. A number of files were missing.
Should this have specifically mentioned the missing files or should the
rpc part have taken care of this?
Thanks
Gary
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Openstack-Common] RPC
On 07/10/2012 11:03 AM, Gary Kotton wrote: 1. When we import code from openstack common the test cases for the modules are not imported (maybe I missed something with running setup). When the code is copied the imports are updated. It would be nice to know that the auto tests are also run in the context of the project importing the code. Copying code isn't a long term thing. Anyway, I'm not sure why the tests would help. The unit tests are by design completely isolated to the code being copied in. For what you want, I would write some quantum specific tests that exercise code paths that hit the RPC APIs. 2. I based my integration on the patch https://review.openstack.org/#/c/9166/. A number of files were missing. Should this have specifically mentioned the missing files or should the rpc part have taken care of this? I don't think I understand your question. If you add rpc to openstack-common.conf, and then run the update.py tool in openstack-common, it should pull in everything. -- Russell Bryant ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Openstack-Common] RPC
In addition to this I have a few additional questions and or concerns: 1. When we import code from openstack common the test cases for the modules are not imported (maybe I missed something with running setup). When the code is copied the imports are updated. It would be nice to know that the auto tests are also run in the context of the project importing the code. As Russell said, the tests inside common are intended to ensure independent functionality of the features within common. There should be tests in your own project that test your project's use of RPC. There has been some discussion on having integration tests for common test official openstack projects for compatibility/breakages. You might also want to look at and contribute to the thread best practices for merging common into specific projects. 2. I based my integration on the patch https://review.openstack.org/#/c/9166/. A number of files were missing. Should this have specifically mentioned the missing files or should the rpc part have taken care of this? Are you concerned that the RPC module itself has dependencies on openstack-common which are not being automatically resolved and included when you run update.py? Regards, Eric Windisch ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Openstack-Common] RPC
On 07/10/2012 06:29 PM, Eric Windisch wrote: In addition to this I have a few additional questions and or concerns: 1. When we import code from openstack common the test cases for the modules are not imported (maybe I missed something with running setup). When the code is copied the imports are updated. It would be nice to know that the auto tests are also run in the context of the project importing the code. As Russell said, the tests inside common are intended to ensure independent functionality of the features within common. There should be tests in your own project that test your project's use of RPC. There has been some discussion on having integration tests for common test official openstack projects for compatibility/breakages. You might also want to look at and contribute to the thread best practices for merging common into specific projects. 2. I based my integration on the patch https://review.openstack.org/#/c/9166/. A number of files were missing. Should this have specifically mentioned the missing files or should the rpc part have taken care of this? Are you concerned that the RPC module itself has dependencies on openstack-common which are not being automatically resolved and included when you run update.py? Thanks, I was reviewing a patch that did the update. There were some files missing. I'll try and check. Thanks Gary Regards, Eric Windisch ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Openstack-Common ZeroMQ
The zeromq tests are failing in jenkins. I created bug https://bugs.launchpad.net/openstack-common/+bug/1023060 for this. Anyone with an interest in ZeroMQ support, please help to resolve this bug. Happy Hacking! 7-11 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Openstack-Common ZeroMQ
On Tuesday, July 10, 2012 at 13:24 PM, Jason Kölker wrote: The zeromq tests are failing in jenkins. I created bug https://bugs.launchpad.net/openstack-common/+bug/1023060 for this. Anyone with an interest in ZeroMQ support, please help to resolve this bug. I'm maintaining this code and would love to see it working again. There were already bugs filed for this and changes already in gerrit for review, that once committed, should fix the tests. The bigger issue is getting people to do the reviews... -- Eric Windisch ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Openstack-Common ZeroMQ
The bigger issue is getting people to do the reviews... Here is the link for those that want to help: https://review.openstack.org/#/q/status:open+project:openstack/openstack-common+branch:master+topic:bug/1021459,n,z Regards, Eric Windisch ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Openstack-Common ZeroMQ
On Tue, 2012-07-10 at 13:36 -0400, Eric Windisch wrote: The bigger issue is getting people to do the reviews... Here is the link for those that want to help: https://review.openstack.org/#/q/status:open+project:openstack/openstack-common+branch:master+topic:bug/1021459,n,z Cool rebase those off trunk, and I'll push them in. The problem is that now that pyzmq is installed on the jenkins boxen, it will prevent merging if we revert the skip tests. Happy Hacking! 7-11 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Openstack common
There's a new version of pep8 out today (1.3.1) which fixes a few indentation cases of if statements that were broken in 1.3. Sergio On Sun, Jun 17, 2012 at 9:01 PM, Adrian Smith adr...@17od.com wrote: pep8 1.3 (released 15-Jun) is much stricter about the indentation used on continuation lines. After upgrading we started seeing quite a few instances of E127,E128... E127 continuation line over-indented for visual indent. Adrian On 17 June 2012 17:52, Jay Pipes jaypi...@gmail.com wrote: What version of pep8 are you using? The errors look to be warnings that are no longer printed in more modern versions of pep8... All the best, -jay On 06/17/2012 03:42 AM, Gary Kotton wrote: Hi, Over the weekend patches were made to Quantum to support Pep 1.3. Some of the patches were in the openstack common code. I have opened a bug to address this in the openstack common code (https://bugs.launchpad.net/openstack-common/+bug/1014216) I am currently updating the common code and will hopefully have a patch soon to address this for review. Thanks Gary ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Openstack common
What version of pep8 are you using? The errors look to be warnings that are no longer printed in more modern versions of pep8... All the best, -jay On 06/17/2012 03:42 AM, Gary Kotton wrote: Hi, Over the weekend patches were made to Quantum to support Pep 1.3. Some of the patches were in the openstack common code. I have opened a bug to address this in the openstack common code (https://bugs.launchpad.net/openstack-common/+bug/1014216) I am currently updating the common code and will hopefully have a patch soon to address this for review. Thanks Gary ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Openstack common
pep8 1.3 (released 15-Jun) is much stricter about the indentation used on continuation lines. After upgrading we started seeing quite a few instances of E127,E128... E127 continuation line over-indented for visual indent. Adrian On 17 June 2012 17:52, Jay Pipes jaypi...@gmail.com wrote: What version of pep8 are you using? The errors look to be warnings that are no longer printed in more modern versions of pep8... All the best, -jay On 06/17/2012 03:42 AM, Gary Kotton wrote: Hi, Over the weekend patches were made to Quantum to support Pep 1.3. Some of the patches were in the openstack common code. I have opened a bug to address this in the openstack common code (https://bugs.launchpad.net/openstack-common/+bug/1014216) I am currently updating the common code and will hopefully have a patch soon to address this for review. Thanks Gary ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] openstack-common
Hey,
On Tue, 2012-01-03 at 16:57 +, Mark McLoughlin wrote:
The openstack-common project intends to produce a python library containing
infrastructure code shared by OpenStack projects. The APIs provided by the
project should be high quality, stable, consistent and generally useful.
Jason and I have come up with a bit of a compromise on this front:
http://wiki.openstack.org/CommonLibrary#Incubation
I've appended the text below. We've started with melange:
http://review.openstack.org/3465
While we still think that we should be able to make API compat
guarantees about the openstack-common library, there needs to be a
stepping stone for new APIs which are shared between multiple projects
but aren't yet up to criteria required for inclusion in the library.
This is still evil copy-and-paste, but it's managed evilness :-)
Thanks,
Mark.
== Incubation ==
openstack-common also provides a process for incubating APIs which,
while they are shared between multiple OpenStack projects, have not yet
matured to meet the criteria described above.
APIs which are incubating can be copied into individual openstack
projects from openstack-common using the {{{update.py}}} script
provided. An {{{openstack-common.conf}}} configuration file in the
project describes which modules to copy and where they should be copied
to e.g.
$ git clone .../melange
$ cd melange
$ cat openstack-common.conf
[DEFAULT]
# The list of modules to copy from openstack-common
modules=config,exception,extensions,utils,wsgi
# The base module to hold the copy of openstack.common
base=melange
and then, to copy the code across:
$ cd ../
$ git clone .../openstack-common
$ cd openstack-common
$ python update.py ../melange
Copying the config,exception,extensions,utils,wsgi modules under the melange
module in ../melange
Projects which are using such incubating APIs must avoid ever modifying
their copies of the code. All changes should be made in openstack-common
itself and copied into the project.
Developers making changes to incubating APIs in openstack-common must be
prepared to update the copies in the projects which have previously
imported the code.
Incubation shouldn't be seen as a long term option for any API - it is
merely a stepping stone to inclusion into the openstack-common library
proper.
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] openstack-common
Yippe common code that people can share! Win!
On 1/26/12 8:32 AM, Mark McLoughlin mar...@redhat.com wrote:
Hey,
On Tue, 2012-01-03 at 16:57 +, Mark McLoughlin wrote:
The openstack-common project intends to produce a python library containing
infrastructure code shared by OpenStack projects. The APIs provided by the
project should be high quality, stable, consistent and generally useful.
Jason and I have come up with a bit of a compromise on this front:
http://wiki.openstack.org/CommonLibrary#Incubation
I've appended the text below. We've started with melange:
http://review.openstack.org/3465
While we still think that we should be able to make API compat
guarantees about the openstack-common library, there needs to be a
stepping stone for new APIs which are shared between multiple projects
but aren't yet up to criteria required for inclusion in the library.
This is still evil copy-and-paste, but it's managed evilness :-)
Thanks,
Mark.
== Incubation ==
openstack-common also provides a process for incubating APIs which,
while they are shared between multiple OpenStack projects, have not yet
matured to meet the criteria described above.
APIs which are incubating can be copied into individual openstack
projects from openstack-common using the {{{update.py}}} script
provided. An {{{openstack-common.conf}}} configuration file in the
project describes which modules to copy and where they should be copied
to e.g.
$ git clone .../melange
$ cd melange
$ cat openstack-common.conf
[DEFAULT]
# The list of modules to copy from openstack-common
modules=config,exception,extensions,utils,wsgi
# The base module to hold the copy of openstack.common
base=melange
and then, to copy the code across:
$ cd ../
$ git clone .../openstack-common
$ cd openstack-common
$ python update.py ../melange
Copying the config,exception,extensions,utils,wsgi modules under the melange
module in ../melange
Projects which are using such incubating APIs must avoid ever modifying
their copies of the code. All changes should be made in openstack-common
itself and copied into the project.
Developers making changes to incubating APIs in openstack-common must be
prepared to update the copies in the projects which have previously
imported the code.
Incubation shouldn't be seen as a long term option for any API - it is
merely a stepping stone to inclusion into the openstack-common library
proper.
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] openstack-common
Can't wait for openstack-common to be usable for Quantum as well. Here is
our write-up of code in Quantum that seems generic (and is likely
borrowed from other openstack project):
http://wiki.openstack.org/QuantumOpenstackCommon
Would love to get much of this into openstack-common.
Dan
On Thu, Jan 26, 2012 at 10:58 AM, Joshua Harlow harlo...@yahoo-inc.comwrote:
Yippe common code that people can share! Win!
On 1/26/12 8:32 AM, Mark McLoughlin mar...@redhat.com wrote:
Hey,
On Tue, 2012-01-03 at 16:57 +, Mark McLoughlin wrote:
The openstack-common project intends to produce a python library
containing
infrastructure code shared by OpenStack projects. The APIs provided by
the
project should be high quality, stable, consistent and generally useful.
Jason and I have come up with a bit of a compromise on this front:
http://wiki.openstack.org/CommonLibrary#Incubation
I've appended the text below. We've started with melange:
http://review.openstack.org/3465
While we still think that we should be able to make API compat
guarantees about the openstack-common library, there needs to be a
stepping stone for new APIs which are shared between multiple projects
but aren't yet up to criteria required for inclusion in the library.
This is still evil copy-and-paste, but it's managed evilness :-)
Thanks,
Mark.
== Incubation ==
openstack-common also provides a process for incubating APIs which,
while they are shared between multiple OpenStack projects, have not yet
matured to meet the criteria described above.
APIs which are incubating can be copied into individual openstack
projects from openstack-common using the {{{update.py}}} script
provided. An {{{openstack-common.conf}}} configuration file in the
project describes which modules to copy and where they should be copied
to e.g.
$ git clone .../melange
$ cd melange
$ cat openstack-common.conf
[DEFAULT]
# The list of modules to copy from openstack-common
modules=config,exception,extensions,utils,wsgi
# The base module to hold the copy of openstack.common
base=melange
and then, to copy the code across:
$ cd ../
$ git clone .../openstack-common
$ cd openstack-common
$ python update.py ../melange
Copying the config,exception,extensions,utils,wsgi modules under the
melange module in ../melange
Projects which are using such incubating APIs must avoid ever modifying
their copies of the code. All changes should be made in openstack-common
itself and copied into the project.
Developers making changes to incubating APIs in openstack-common must be
prepared to update the copies in the projects which have previously
imported the code.
Incubation shouldn't be seen as a long term option for any API - it is
merely a stepping stone to inclusion into the openstack-common library
proper.
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
--
~~~
Dan Wendlandt
Nicira Networks: www.nicira.com
twitter: danwendlandt
~~~
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] openstack-common
Openstack-common could be great. There are lots of use cases that make a lot of sense to put in openstack common. Configuration loading, context, some aspects of logging, wsgi middleware, some parts of utils--those seem to me like great opportunities to save time and effort, both writing and reading code, through code reuse. Unfortunately, openstack-common could also be horrible. When a problematic module makes it into openstack-common it gets enshrined in rough consensus and is much harder to change or replace. If there are divergent use cases for that module, no doubt the common implementation of it will be riddled with conditional code paths, with each path only truly being exercised by one project (illusory reuse). If the module frequently has to change, then backwards compatibility requirements will make the code even more complicated and harder to understand and change. The requirement that there is no other API in OpenStack competing for this consensus means that, when a spaghetti-code module ought to be simply replaced or removed from common, fewer people will be willing to do this work because they now have to change the calling code in N projects. We need to think carefully about how we will avoid and address this problem. I have a few proposals. I'm not 100% certain of any of these, but I'd like for you to consider them. 1) The process of adding code to common should require input from every core project. This might take the form of requiring one core member from each to approve the merge. 2) We should plan to be restrictive about what lands in common. We should plan to fight complexity and illusory reuse before something lands in common rather than relying exclusively on iterating on it after it is in common. 3) Rather than forking jkoelker/openstack-common, we should create a new project and merge sections in piecemeal. This creates the opportunity to review each part separately. 4) Above and beyond #2 and #3, our first steps should err on the side of safety. So while I feel that a lot of the modules in the current openstack-common github repo definitely belong there, I am really worried about the web framework and serialization code. That code in particular feels likely to change and likely to have different uses in different projects. As such it will probably bite us with complexity both from backwards compatibility and illusory reuse. Maybe we can refactor and rewrite to avoid these problems, but we should probably do that before adding it to common rather than after. As always, just my 2 cents, but I hope there are others out there who are similarly concerned that openstack-common might not be worth it if we just try to go full-tilt on code reuse. Mark McLoughlin mar...@redhat.com said: Hey, As Jason says - another year, another openstack-common thread! :-) I've just written up the plan Jason and I have for openstack-common: http://wiki.openstack.org/CommonLibrary (also pasted below to make it easier to reply to) I guess what we're trying to do is quickly get this thing into good enough shape to do a first release. Even if that release only contains a handful of APIs, but they meet the criteria below, then we'll have a really solid foundation to build on. Thoughts? Cheers, Mark. The openstack-common project intends to produce a python library containing infrastructure code shared by OpenStack projects. The APIs provided by the project should be high quality, stable, consistent and generally useful. The existence of an API in openstack-common should be indicitative of rough consensus across the project on that API's design. New OpenStack projects should be able to use an API in the library safe in the knowledge that, by doing so, the project is being a good OpenStack citizen and building upon established best practice. To that end, a number of principles should be adhered to when considering any proposed API for openstack-common: 1) The API is generally useful and is a good fit - e.g. it doesn't encode some assumptions specific to the project it originated from, it should follow a style consistent with other openstack-common APIs and should fit generally in a theme like error handling, configuration options, time and date, notifications, WSGI, etc. 2) The API is already in use by a number of OpenStack projects 3) There is a commitment to adopt the API in all other OpenStack projects (where appropriate) and there are no known major blockers to that adoption 4) The API represents the rough consensus across OpenStack projects 5) There is no other API in OpenStack competing for this rough consensus 6) It should be possible for the API to evolve while continuing to maintain backwards compatibility with older versions for a reasonable period - e.g. compatibility with an API deprecated in release N may only be removed in
[Openstack] openstack-common
Hey, As Jason says - another year, another openstack-common thread! :-) I've just written up the plan Jason and I have for openstack-common: http://wiki.openstack.org/CommonLibrary (also pasted below to make it easier to reply to) I guess what we're trying to do is quickly get this thing into good enough shape to do a first release. Even if that release only contains a handful of APIs, but they meet the criteria below, then we'll have a really solid foundation to build on. Thoughts? Cheers, Mark. The openstack-common project intends to produce a python library containing infrastructure code shared by OpenStack projects. The APIs provided by the project should be high quality, stable, consistent and generally useful. The existence of an API in openstack-common should be indicitative of rough consensus across the project on that API's design. New OpenStack projects should be able to use an API in the library safe in the knowledge that, by doing so, the project is being a good OpenStack citizen and building upon established best practice. To that end, a number of principles should be adhered to when considering any proposed API for openstack-common: 1) The API is generally useful and is a good fit - e.g. it doesn't encode some assumptions specific to the project it originated from, it should follow a style consistent with other openstack-common APIs and should fit generally in a theme like error handling, configuration options, time and date, notifications, WSGI, etc. 2) The API is already in use by a number of OpenStack projects 3) There is a commitment to adopt the API in all other OpenStack projects (where appropriate) and there are no known major blockers to that adoption 4) The API represents the rough consensus across OpenStack projects 5) There is no other API in OpenStack competing for this rough consensus 6) It should be possible for the API to evolve while continuing to maintain backwards compatibility with older versions for a reasonable period - e.g. compatibility with an API deprecated in release N may only be removed in release N+2 There have been several attempts at kick-starting openstack-common, each attempt beginning with moving a number of existing APIs from Glance or Nova into a new repository. None of these attempts have quite managed to reach the point where they were ready for other OpenStack projects to depend on the library. This proposal marks the beginning of yet another attempt. The idea is to create a new openstack-common repository, seed it with Jason Kölker's excellent infrastructure from his repository[1] and begin importing individual APIs while applying the principles above during the review process for each proposed API. [1] - https://github.com/jkoelker/openstack-common ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] openstack-common
I'd love to see openstack-common get off the ground, so I'm all in favor of this. One question: why do you feel that you need such strong backwards compatibility? If someone makes a change in openstack-common and makes simultaneous changes in all OpenStack projects to match, isn’t that sufficient? Ewan. -Original Message- From: openstack-bounces+ewan.mellor=citrix@lists.launchpad.net [mailto:openstack-bounces+ewan.mellor=citrix@lists.launchpad.net] On Behalf Of Mark McLoughlin Sent: 03 January 2012 08:58 To: openstack@lists.launchpad.net Cc: Jason Koelker Subject: [Openstack] openstack-common Hey, As Jason says - another year, another openstack-common thread! :-) I've just written up the plan Jason and I have for openstack-common: http://wiki.openstack.org/CommonLibrary (also pasted below to make it easier to reply to) I guess what we're trying to do is quickly get this thing into good enough shape to do a first release. Even if that release only contains a handful of APIs, but they meet the criteria below, then we'll have a really solid foundation to build on. Thoughts? Cheers, Mark. The openstack-common project intends to produce a python library containing infrastructure code shared by OpenStack projects. The APIs provided by the project should be high quality, stable, consistent and generally useful. The existence of an API in openstack-common should be indicitative of rough consensus across the project on that API's design. New OpenStack projects should be able to use an API in the library safe in the knowledge that, by doing so, the project is being a good OpenStack citizen and building upon established best practice. To that end, a number of principles should be adhered to when considering any proposed API for openstack-common: 1) The API is generally useful and is a good fit - e.g. it doesn't encode some assumptions specific to the project it originated from, it should follow a style consistent with other openstack-common APIs and should fit generally in a theme like error handling, configuration options, time and date, notifications, WSGI, etc. 2) The API is already in use by a number of OpenStack projects 3) There is a commitment to adopt the API in all other OpenStack projects (where appropriate) and there are no known major blockers to that adoption 4) The API represents the rough consensus across OpenStack projects 5) There is no other API in OpenStack competing for this rough consensus 6) It should be possible for the API to evolve while continuing to maintain backwards compatibility with older versions for a reasonable period - e.g. compatibility with an API deprecated in release N may only be removed in release N+2 There have been several attempts at kick-starting openstack-common, each attempt beginning with moving a number of existing APIs from Glance or Nova into a new repository. None of these attempts have quite managed to reach the point where they were ready for other OpenStack projects to depend on the library. This proposal marks the beginning of yet another attempt. The idea is to create a new openstack-common repository, seed it with Jason Kölker's excellent infrastructure from his repository[1] and begin importing individual APIs while applying the principles above during the review process for each proposed API. [1] - https://github.com/jkoelker/openstack-common ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] openstack-common
On Tue, 2012-01-03 at 19:54 +, Ewan Mellor wrote: I'd love to see openstack-common get off the ground, so I'm all in favor of this. One question: why do you feel that you need such strong backwards compatibility? If someone makes a change in openstack-common and makes simultaneous changes in all OpenStack projects to match, isn’t that sufficient? No simultaneous change is ever actually simultaneous. We see this all the time with interop between keystone (in particular), nova, and glance. Once openstack-common gets into the picture, the interop problems stand to be significantly worse; if one tiny change is not backwards compatible, you break *everything* that uses openstack-common. The good thing, of course, is that it'll be noticed quickly; the bad thing is that all work gets significantly impeded until the fix(es) go in. Speaking from experience: it is possible to preserve N+2 backwards compatibility while still making major enhancements. It can be a pain in the butt sometimes, but it is doable, and, in cases like openstack-common, I think it is necessary. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] openstack-common
Operationally they'll need to be able to make the change in a way that it can be sequenced. We don't have a concept of simultaneous tied changes. So a the change you describe would need to look like: Land change to openstack-common to add something new Land changes to dependent projects to use that new thing Land change to openstack-common to remove now deprecated thing. As a related note, I'm going to get the current repo moved in to gerrit today or tomorrow. Monty On 01/03/2012 11:54 AM, Ewan Mellor wrote: I'd love to see openstack-common get off the ground, so I'm all in favor of this. One question: why do you feel that you need such strong backwards compatibility? If someone makes a change in openstack-common and makes simultaneous changes in all OpenStack projects to match, isn’t that sufficient? Ewan. -Original Message- From: openstack-bounces+ewan.mellor=citrix@lists.launchpad.net [mailto:openstack-bounces+ewan.mellor=citrix@lists.launchpad.net] On Behalf Of Mark McLoughlin Sent: 03 January 2012 08:58 To: openstack@lists.launchpad.net Cc: Jason Koelker Subject: [Openstack] openstack-common Hey, As Jason says - another year, another openstack-common thread! :-) I've just written up the plan Jason and I have for openstack-common: http://wiki.openstack.org/CommonLibrary (also pasted below to make it easier to reply to) I guess what we're trying to do is quickly get this thing into good enough shape to do a first release. Even if that release only contains a handful of APIs, but they meet the criteria below, then we'll have a really solid foundation to build on. Thoughts? Cheers, Mark. The openstack-common project intends to produce a python library containing infrastructure code shared by OpenStack projects. The APIs provided by the project should be high quality, stable, consistent and generally useful. The existence of an API in openstack-common should be indicitative of rough consensus across the project on that API's design. New OpenStack projects should be able to use an API in the library safe in the knowledge that, by doing so, the project is being a good OpenStack citizen and building upon established best practice. To that end, a number of principles should be adhered to when considering any proposed API for openstack-common: 1) The API is generally useful and is a good fit - e.g. it doesn't encode some assumptions specific to the project it originated from, it should follow a style consistent with other openstack-common APIs and should fit generally in a theme like error handling, configuration options, time and date, notifications, WSGI, etc. 2) The API is already in use by a number of OpenStack projects 3) There is a commitment to adopt the API in all other OpenStack projects (where appropriate) and there are no known major blockers to that adoption 4) The API represents the rough consensus across OpenStack projects 5) There is no other API in OpenStack competing for this rough consensus 6) It should be possible for the API to evolve while continuing to maintain backwards compatibility with older versions for a reasonable period - e.g. compatibility with an API deprecated in release N may only be removed in release N+2 There have been several attempts at kick-starting openstack-common, each attempt beginning with moving a number of existing APIs from Glance or Nova into a new repository. None of these attempts have quite managed to reach the point where they were ready for other OpenStack projects to depend on the library. This proposal marks the beginning of yet another attempt. The idea is to create a new openstack-common repository, seed it with Jason Kölker's excellent infrastructure from his repository[1] and begin importing individual APIs while applying the principles above during the review process for each proposed API. [1] - https://github.com/jkoelker/openstack-common ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] openstack-common
On 01/03/2012 02:54 PM, Ewan Mellor wrote: I'd love to see openstack-common get off the ground, so I'm all in favor of this. One question: why do you feel that you need such strong backwards compatibility? If someone makes a change in openstack-common and makes simultaneous changes in all OpenStack projects to match, isn’t that sufficient? This is somewhat related to the recent thread on gating commits on integration tests. There has to be at least some level of backwards compatibility to ensure that tests don't explode when changes are going in across multiple repos. It's also nice to set expectations for projects that want to use openstack-common, but aren't an openstack core project but perhaps would like to be eventually. -- Russell Bryant ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] openstack-common
On Tue, 2012-01-03 at 19:54 +, Ewan Mellor wrote: I'd love to see openstack-common get off the ground, so I'm all in favor of this. One question: why do you feel that you need such strong backwards compatibility? If someone makes a change in openstack-common and makes simultaneous changes in all OpenStack projects to match, isn’t that sufficient? No, I really don't think it is sufficient. Unless we want packagers to hunt us down with sharp implements :) Each of the projects are separated by an API which we maintain some compatibility around. So, in theory, you can use e.g. Essex Glance with Diablo Nova. If you make it so that Glance requires Essex Common and doesn't work with Diablo Common, but Nova requires Diablo Common and doesn't work with Essex common ... you're screwed. This may not sound like much of a practical concern, but it's exactly why packagers curse the Java and Ruby worlds for their sloppiness. It's really painful. And, IMHO, if we allow it to happen, it will be an even bigger problem with the APIs in openstack-common than the inter-project APIs. Cheers, Mark. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] openstack-common
On Tue, 2012-01-03 at 13:04 -0800, Monty Taylor wrote: Operationally they'll need to be able to make the change in a way that it can be sequenced. We don't have a concept of simultaneous tied changes. So a the change you describe would need to look like: Land change to openstack-common to add something new Land changes to dependent projects to use that new thing Land change to openstack-common to remove now deprecated thing. Yes, but for packaging and release sanity, the removal must come much later. As a related note, I'm going to get the current repo moved in to gerrit today or tomorrow. It's more Jason's call, but I think we're basically asking you to hold off on that for a little while. We may decide to start a new repo. Cheers, Mark. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] openstack-common
On 01/03/2012 01:38 PM, Mark McLoughlin wrote: On Tue, 2012-01-03 at 13:04 -0800, Monty Taylor wrote: Operationally they'll need to be able to make the change in a way that it can be sequenced. We don't have a concept of simultaneous tied changes. So a the change you describe would need to look like: Land change to openstack-common to add something new Land changes to dependent projects to use that new thing Land change to openstack-common to remove now deprecated thing. Yes, but for packaging and release sanity, the removal must come much later. As a related note, I'm going to get the current repo moved in to gerrit today or tomorrow. It's more Jason's call, but I think we're basically asking you to hold off on that for a little while. We may decide to start a new repo. Oh - ok. My bad - I'll learn to read entire threads next time... Let let me know when it's ready to go or if there's anything you want from me helpwise. Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] openstack-common
On Tue, 2012-01-03 at 21:38 +, Mark McLoughlin wrote: As a related note, I'm going to get the current repo moved in to gerrit today or tomorrow. It's more Jason's call, but I think we're basically asking you to hold off on that for a little while. We may decide to start a new repo. I've got Melange going off a non-master branch from my github repo at the moment, so we could start with a clean repo and not have to worry about breaking melange while we figure out the api on our side should look like after surveying the other projects. I'm 50/50 on the blank slate approach. I'm leaning towards just taking what is there as is, and just start refactoring it as needed. Since melange is running off the other branch, we won't have to worry about compat at first and can make sweeping changes. One thing I would like to ask is if we could not use run_tests.sh/run_tests.py in the jenkins job as I'd like to remove it. Currently in the setup.cfg there is a note on how to get the same output (style wise) with nosetests. The only downside is I haven't written a nose venv plugin yet, but if we move it to tox, then that shouldn't be needed anyway. Happy Hacking! 7-11 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] openstack-common
On Tue, 2012-01-03 at 13:49 -0800, Monty Taylor wrote: It's more Jason's call, but I think we're basically asking you to hold off on that for a little while. We may decide to start a new repo. Oh - ok. My bad - I'll learn to read entire threads next time... Let let me know when it's ready to go or if there's anything you want from me helpwise. Sorry, my bad on not keeping up with things ;). If you get a chance go ahead and pull it into gerrit as is, we can handle any changes we want through reviews. (See my note in the other email about preferring to run the tests with nosetests vs run_tests.sh though). Happy Hacking! 7-11 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] openstack-common
On 01/03/2012 02:11 PM, Jason Kölker wrote: On Tue, 2012-01-03 at 21:38 +, Mark McLoughlin wrote: As a related note, I'm going to get the current repo moved in to gerrit today or tomorrow. It's more Jason's call, but I think we're basically asking you to hold off on that for a little while. We may decide to start a new repo. I've got Melange going off a non-master branch from my github repo at the moment, so we could start with a clean repo and not have to worry about breaking melange while we figure out the api on our side should look like after surveying the other projects. I'm 50/50 on the blank slate approach. I'm leaning towards just taking what is there as is, and just start refactoring it as needed. Since melange is running off the other branch, we won't have to worry about compat at first and can make sweeping changes. One thing I would like to ask is if we could not use run_tests.sh/run_tests.py in the jenkins job as I'd like to remove it. Currently in the setup.cfg there is a note on how to get the same output (style wise) with nosetests. The only downside is I haven't written a nose venv plugin yet, but if we move it to tox, then that shouldn't be needed anyway. I'd be for that 1000% and will work with you on making it happen not just in openstack-common, but across the board. I have been wanting to kill run_tests.py since the first day that we imported nova into the code repos. If we can get this tox off the ground, then we're potentially in GREAT shape. BUT - you already have the nose plugin and the pep8 checker done, so consider me sold on that! :) Feature request - can you add an option to tissue that will cause it to output in pylint-compat format? (because we already have a jenkins thing that reads that output. In our jenkins jobs, we do this: pep8 blah | perl -ple 's/: ([WE]\d+)/: [$1]/' pep8.txt Which is a silly filter, but I'd love to have less stupid things in jenkins jobs. Also, have you seen the way that py.test displays tracebacks? Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] openstack-common
-Original Message- From: Mark McLoughlin [mailto:mar...@redhat.com] Sent: 03 January 2012 13:35 To: Ewan Mellor Cc: openstack@lists.launchpad.net; Jason Koelker Subject: RE: [Openstack] openstack-common On Tue, 2012-01-03 at 19:54 +, Ewan Mellor wrote: I'd love to see openstack-common get off the ground, so I'm all in favor of this. One question: why do you feel that you need such strong backwards compatibility? If someone makes a change in openstack-common and makes simultaneous changes in all OpenStack projects to match, isn’t that sufficient? No, I really don't think it is sufficient. Unless we want packagers to hunt us down with sharp implements :) Each of the projects are separated by an API which we maintain some compatibility around. So, in theory, you can use e.g. Essex Glance with Diablo Nova. If you make it so that Glance requires Essex Common and doesn't work with Diablo Common, but Nova requires Diablo Common and doesn't work with Essex common ... you're screwed. Yeah, that's fair enough. I didn't think of the Essex Glance with Diablo Nova case. You'd definitely like to be able to update the supporting libraries first. Cheers, Ewan. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] OpenStack Common
Here's the start of a skeleton project: https://github.com/openstack/openstack-skeleton Fork away. We can use the pull requests for discussion about what's best practice, what isn't, etc... -jay On Tue, Jul 26, 2011 at 8:26 AM, Thierry Carrez thie...@openstack.org wrote: Brian Lamar wrote: I love the idea of having an openstack-common project. However, the prospect of creating such a project is daunting and quite difficult. [...] Thanks for bringing up the subject ! I think there are two types of benefits from this: The first is, like you said, to lower the barrier of entry for new projects. Having a set of proven ways of doing logging, configuration etc. would definitely help in spawning a new project the OpenStack way. The second is to better establish OpenStack as an integrated product made of cooperating components. If people that want to deploy multiple OpenStack components end up having to learn 4 different config file formats and teach their tools to recognize 3 different log formats, it appears like the whole OpenStack thing is non-professional. Choosing the best practice way among the different projects implementations and making it the common way would allow to quickly reap the first type of benefits. The cost of separating it is not so large. Then we can encourage existing projects using different implementations to migrate to the common way, in order to reap the second type as well. There would certainly be delays and exceptions to the common rule, since this part would probably be a bit more painful: the cost of replacing a current (working) system is a lot larger. Cheers, -- Thierry Carrez (ttx) Release Manager, OpenStack ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] OpenStack Common
All, I love the idea of having an openstack-common project. However, the prospect of creating such a project is daunting and quite difficult. It's my belief that standardizing/collecting common logic into a single module will be beneficial to our current code-base and allow for future projects to be created more quickly/easily. Currently the behemoth in the room is OpenStack Compute (aka Nova). The Compute project contains much more code than all other OpenStack projects (combined), and rightly so...virtualization is a pretty darn complex thing to do in a flexible way. This might be why other projects have been spawned to take away some of the logic from a single massive project and place that logic into smaller, more focused projects. However, I would argue that the barrier of entry is too high for this strategy to work. Projects such as Quantum, Burrow, Lunr, and Keystone suffer from the lack of a single cohesive strategy in the following areas: -Logging -Configuration -WSGI -RPC -Database -Testing -Deployment/Distribution of code These are the building blocks which most projects will require, yet every project has to create their own implementations? Sure, it's not going to be easy, and maybe some categories I've labeled won't make the final cut, but I would like to start a conversation with the community as to the feasibility of such an endeavor. This is not the first time such a project has been brought up. Much earlier in the year, a number of people suggested moving lazy loading code into a common project. I would like to think that project died due to complexity rather than the community rejecting the idea. To create a common library of this nature requires a bit of pushing aside one's partisan blinders and the abandonment of ideological entrenchments*, so hopefully this won't deteriorate into a FLAGS vs. argparse flame-war. TLDR: No * - Shamelessly stolen from The West Wing (tm) ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] OpenStack Common
Thats a better direction. Another way to think about this is to consider this an extensible service framework. If we agree on lightweight code wrappers (apis/modules) first with whatever backend each project uses, and then migrate each of those areas to a common implementation, we can inject uniformity and expected levels of instrumentation/capabilities across the projects. Example: log4j extender model... Jan On Jul 25, 2011, at 10:27 AM, Brian Lamar brian.la...@rackspace.com wrote: I don't see it as multiple projects, just as a set of modules: openstack.common.config openstack.common.deployment openstack.common.logging ... etc. These are purely modules which help you create you're own OpenStack-compatible project...all available through a single openstack-common project. -Original Message- From: Glen Campbell glen.campb...@rackspace.com Sent: Monday, July 25, 2011 1:20pm To: Brian Lamar brian.la...@rackspace.com, openstack@lists.launchpad.net openstack@lists.launchpad.net Subject: Re: [Openstack] OpenStack Common Would it better to break it down even further? I.e., instead of trying to put ALL the common code into one project, create mini projects for common-logging, common-configuration, etc. That would permit other projects to adopt what they need, when they need it, rather than trying to integrate the entire common project at once. For example, we're working on converting Glance to use the logging/notification mechanism defined by Nova. The first step in the project was, however, cut and paste notification code from one to the other. That's disturbing to me, because it doubles the amount of effort required to implement changes to the notification system in the future. It would be much better IMHO to have a refactored common-logging module that could be included by multiple projects, with a standard interface each project could rely upon. There's no requirement, then, to implement common-rpc when you integrate common-logging, which lowers the barrier to entry for each project. On 7/25/11 12:11 PM, Brian Lamar brian.la...@rackspace.com wrote: All, I love the idea of having an openstack-common project. However, the prospect of creating such a project is daunting and quite difficult. It's my belief that standardizing/collecting common logic into a single module will be beneficial to our current code-base and allow for future projects to be created more quickly/easily. Currently the behemoth in the room is OpenStack Compute (aka Nova). The Compute project contains much more code than all other OpenStack projects (combined), and rightly so...virtualization is a pretty darn complex thing to do in a flexible way. This might be why other projects have been spawned to take away some of the logic from a single massive project and place that logic into smaller, more focused projects. However, I would argue that the barrier of entry is too high for this strategy to work. Projects such as Quantum, Burrow, Lunr, and Keystone suffer from the lack of a single cohesive strategy in the following areas: -Logging -Configuration -WSGI -RPC -Database -Testing -Deployment/Distribution of code These are the building blocks which most projects will require, yet every project has to create their own implementations? Sure, it's not going to be easy, and maybe some categories I've labeled won't make the final cut, but I would like to start a conversation with the community as to the feasibility of such an endeavor. This is not the first time such a project has been brought up. Much earlier in the year, a number of people suggested moving lazy loading code into a common project. I would like to think that project died due to complexity rather than the community rejecting the idea. To create a common library of this nature requires a bit of pushing aside one's partisan blinders and the abandonment of ideological entrenchments*, so hopefully this won't deteriorate into a FLAGS vs. argparse flame-war. TLDR: No * - Shamelessly stolen from The West Wing (tm) ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp This email may include confidential information. If you received it in error, please delete it. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] OpenStack Common
If by mini-projects you mean small and separate projects, then I don't think that makes sense. All we need for this is a single project that contains submodules that don't contain unnecessary dependencies on other submodules within the common project. So lots of bite size pieces that can be used or not used. Devin On Jul 25, 2011, at 10:20 AM, Glen Campbell wrote: Would it better to break it down even further? I.e., instead of trying to put ALL the common code into one project, create mini projects for common-logging, common-configuration, etc. That would permit other projects to adopt what they need, when they need it, rather than trying to integrate the entire common project at once. For example, we're working on converting Glance to use the logging/notification mechanism defined by Nova. The first step in the project was, however, cut and paste notification code from one to the other. That's disturbing to me, because it doubles the amount of effort required to implement changes to the notification system in the future. It would be much better IMHO to have a refactored common-logging module that could be included by multiple projects, with a standard interface each project could rely upon. There's no requirement, then, to implement common-rpc when you integrate common-logging, which lowers the barrier to entry for each project. On 7/25/11 12:11 PM, Brian Lamar brian.la...@rackspace.com wrote: All, I love the idea of having an openstack-common project. However, the prospect of creating such a project is daunting and quite difficult. It's my belief that standardizing/collecting common logic into a single module will be beneficial to our current code-base and allow for future projects to be created more quickly/easily. Currently the behemoth in the room is OpenStack Compute (aka Nova). The Compute project contains much more code than all other OpenStack projects (combined), and rightly so...virtualization is a pretty darn complex thing to do in a flexible way. This might be why other projects have been spawned to take away some of the logic from a single massive project and place that logic into smaller, more focused projects. However, I would argue that the barrier of entry is too high for this strategy to work. Projects such as Quantum, Burrow, Lunr, and Keystone suffer from the lack of a single cohesive strategy in the following areas: -Logging -Configuration -WSGI -RPC -Database -Testing -Deployment/Distribution of code These are the building blocks which most projects will require, yet every project has to create their own implementations? Sure, it's not going to be easy, and maybe some categories I've labeled won't make the final cut, but I would like to start a conversation with the community as to the feasibility of such an endeavor. This is not the first time such a project has been brought up. Much earlier in the year, a number of people suggested moving lazy loading code into a common project. I would like to think that project died due to complexity rather than the community rejecting the idea. To create a common library of this nature requires a bit of pushing aside one's partisan blinders and the abandonment of ideological entrenchments*, so hopefully this won't deteriorate into a FLAGS vs. argparse flame-war. TLDR: No * - Shamelessly stolen from The West Wing (tm) ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp This email may include confidential information. If you received it in error, please delete it. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] OpenStack Common
+1 On Jul 25, 2011, at 11:59 AM, Devin Carlen devin.car...@gmail.com wrote: If by mini-projects you mean small and separate projects, then I don't think that makes sense. All we need for this is a single project that contains submodules that don't contain unnecessary dependencies on other submodules within the common project. So lots of bite size pieces that can be used or not used. Devin On Jul 25, 2011, at 10:20 AM, Glen Campbell wrote: Would it better to break it down even further? I.e., instead of trying to put ALL the common code into one project, create mini projects for common-logging, common-configuration, etc. That would permit other projects to adopt what they need, when they need it, rather than trying to integrate the entire common project at once. For example, we're working on converting Glance to use the logging/notification mechanism defined by Nova. The first step in the project was, however, cut and paste notification code from one to the other. That's disturbing to me, because it doubles the amount of effort required to implement changes to the notification system in the future. It would be much better IMHO to have a refactored common-logging module that could be included by multiple projects, with a standard interface each project could rely upon. There's no requirement, then, to implement common-rpc when you integrate common-logging, which lowers the barrier to entry for each project. On 7/25/11 12:11 PM, Brian Lamar brian.la...@rackspace.com wrote: All, I love the idea of having an openstack-common project. However, the prospect of creating such a project is daunting and quite difficult. It's my belief that standardizing/collecting common logic into a single module will be beneficial to our current code-base and allow for future projects to be created more quickly/easily. Currently the behemoth in the room is OpenStack Compute (aka Nova). The Compute project contains much more code than all other OpenStack projects (combined), and rightly so...virtualization is a pretty darn complex thing to do in a flexible way. This might be why other projects have been spawned to take away some of the logic from a single massive project and place that logic into smaller, more focused projects. However, I would argue that the barrier of entry is too high for this strategy to work. Projects such as Quantum, Burrow, Lunr, and Keystone suffer from the lack of a single cohesive strategy in the following areas: -Logging -Configuration -WSGI -RPC -Database -Testing -Deployment/Distribution of code These are the building blocks which most projects will require, yet every project has to create their own implementations? Sure, it's not going to be easy, and maybe some categories I've labeled won't make the final cut, but I would like to start a conversation with the community as to the feasibility of such an endeavor. This is not the first time such a project has been brought up. Much earlier in the year, a number of people suggested moving lazy loading code into a common project. I would like to think that project died due to complexity rather than the community rejecting the idea. To create a common library of this nature requires a bit of pushing aside one's partisan blinders and the abandonment of ideological entrenchments*, so hopefully this won't deteriorate into a FLAGS vs. argparse flame-war. TLDR: No * - Shamelessly stolen from The West Wing (tm) ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp This email may include confidential information. If you received it in error, please delete it. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] OpenStack Common
One thing that might be added would be dynamic module and class loading. This has implications for flags/options and help output as well. It is something nova does, and I suspect keystone and others will need to do as well. On Mon, Jul 25, 2011 at 2:59 PM, Devin Carlen devin.car...@gmail.com wrote: If by mini-projects you mean small and separate projects, then I don't think that makes sense. All we need for this is a single project that contains submodules that don't contain unnecessary dependencies on other submodules within the common project. So lots of bite size pieces that can be used or not used. Devin On Jul 25, 2011, at 10:20 AM, Glen Campbell wrote: Would it better to break it down even further? I.e., instead of trying to put ALL the common code into one project, create mini projects for common-logging, common-configuration, etc. That would permit other projects to adopt what they need, when they need it, rather than trying to integrate the entire common project at once. For example, we're working on converting Glance to use the logging/notification mechanism defined by Nova. The first step in the project was, however, cut and paste notification code from one to the other. That's disturbing to me, because it doubles the amount of effort required to implement changes to the notification system in the future. It would be much better IMHO to have a refactored common-logging module that could be included by multiple projects, with a standard interface each project could rely upon. There's no requirement, then, to implement common-rpc when you integrate common-logging, which lowers the barrier to entry for each project. On 7/25/11 12:11 PM, Brian Lamar brian.la...@rackspace.com wrote: All, I love the idea of having an openstack-common project. However, the prospect of creating such a project is daunting and quite difficult. It's my belief that standardizing/collecting common logic into a single module will be beneficial to our current code-base and allow for future projects to be created more quickly/easily. Currently the behemoth in the room is OpenStack Compute (aka Nova). The Compute project contains much more code than all other OpenStack projects (combined), and rightly so...virtualization is a pretty darn complex thing to do in a flexible way. This might be why other projects have been spawned to take away some of the logic from a single massive project and place that logic into smaller, more focused projects. However, I would argue that the barrier of entry is too high for this strategy to work. Projects such as Quantum, Burrow, Lunr, and Keystone suffer from the lack of a single cohesive strategy in the following areas: -Logging -Configuration -WSGI -RPC -Database -Testing -Deployment/Distribution of code These are the building blocks which most projects will require, yet every project has to create their own implementations? Sure, it's not going to be easy, and maybe some categories I've labeled won't make the final cut, but I would like to start a conversation with the community as to the feasibility of such an endeavor. This is not the first time such a project has been brought up. Much earlier in the year, a number of people suggested moving lazy loading code into a common project. I would like to think that project died due to complexity rather than the community rejecting the idea. To create a common library of this nature requires a bit of pushing aside one's partisan blinders and the abandonment of ideological entrenchments*, so hopefully this won't deteriorate into a FLAGS vs. argparse flame-war. TLDR: No * - Shamelessly stolen from The West Wing (tm) ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp This email may include confidential information. If you received it in error, please delete it. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
