Re: [openstack-dev] [Nova] Requirements.txt and optional requirements
As point, we are trying to move away from this model. Having to know the dependencies is a bad experience in general. But with the move to eliminate optional parts of the api, most of these become real dependencies for keystone (a few things will still be optional eg memcache lib). --Morgan Sent via mobile On Jan 30, 2015, at 17:08, Alan Pevec ape...@gmail.com wrote: - Remove this requirement, no optional entries in requirements.txt, a 'deployer' has to know what dependencies the components he wants to use have Keystone is documenting its optional dependencies in test-requirements.txt look for # Optional ... comments in http://git.openstack.org/cgit/openstack/keystone/tree/test-requirements.txt Cheers, Alan __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Requirements.txt and optional requirements
- Remove this requirement, no optional entries in requirements.txt, a 'deployer' has to know what dependencies the components he wants to use have Keystone is documenting its optional dependencies in test-requirements.txt look for # Optional ... comments in http://git.openstack.org/cgit/openstack/keystone/tree/test-requirements.txt Cheers, Alan __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Requirements.txt and optional requirements
Hello again, i submitted a new patch set for this at https://review.openstack.org/#/c/110722/ , looking forward to further reviews. :) Best regards Silvan 2015-01-28 10:19 GMT+01:00 Silvan Kaiser sil...@quobyte.com: Hi All! Thanks for the feedback! I'll remove xattr from the requirements in my change set. Currently i'm working on a workaround to execute 'getfattr' instead of the xattr api call. We can asure getfattr is available via package dependencies of our client who has to be installed either way. I'm also checking out your proposal in parallel, i cannot find any documentation about the 'configuration management manifests', do you mean the puppet manifests? Otherwise, could somebody please give me a pointer to their documentation, etc.? Best regards Silvan 2015-01-27 18:32 GMT+01:00 Jay Pipes jaypi...@gmail.com: On 01/27/2015 09:13 AM, Silvan Kaiser wrote: Am 27.01.2015 um 16:51 schrieb Jay Pipes jaypi...@gmail.com: b) The Glance API image cache can use xattr if SQLite is not desired [1], and Glance does *not* list xattr as a dependency in requirements.txt. Swift also has a dependency on python-xattr [2]. So, this particular Python library is not an unknown by any means. Do you happen to know how Glance handles this if the dep. is not handled in requirements.txt? Yep, it's considered a documentation thing and handled in configuration management manifests... Best, -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject: unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -- *Quobyte* GmbH Boyenstr. 41 - 10115 Berlin-Mitte - Germany +49-30-814 591 800 - www.quobyte.com Amtsgericht Berlin-Charlottenburg, HRB 149012B management board: Dr. Felix Hupfeld, Dr. Björn Kolbeck, Dr. Jan Stender __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Requirements.txt and optional requirements
Hi All! Thanks for the feedback! I'll remove xattr from the requirements in my change set. Currently i'm working on a workaround to execute 'getfattr' instead of the xattr api call. We can asure getfattr is available via package dependencies of our client who has to be installed either way. I'm also checking out your proposal in parallel, i cannot find any documentation about the 'configuration management manifests', do you mean the puppet manifests? Otherwise, could somebody please give me a pointer to their documentation, etc.? Best regards Silvan 2015-01-27 18:32 GMT+01:00 Jay Pipes jaypi...@gmail.com: On 01/27/2015 09:13 AM, Silvan Kaiser wrote: Am 27.01.2015 um 16:51 schrieb Jay Pipes jaypi...@gmail.com: b) The Glance API image cache can use xattr if SQLite is not desired [1], and Glance does *not* list xattr as a dependency in requirements.txt. Swift also has a dependency on python-xattr [2]. So, this particular Python library is not an unknown by any means. Do you happen to know how Glance handles this if the dep. is not handled in requirements.txt? Yep, it's considered a documentation thing and handled in configuration management manifests... Best, -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -- *Quobyte* GmbH Boyenstr. 41 - 10115 Berlin-Mitte - Germany +49-30-814 591 800 - www.quobyte.com Amtsgericht Berlin-Charlottenburg, HRB 149012B management board: Dr. Felix Hupfeld, Dr. Björn Kolbeck, Dr. Jan Stender __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Nova] Requirements.txt and optional requirements
Hello! Do dependencies required only in some contexts belong into requirements.txt? Yesterday we had a short discussion on #openstack-nova regarding how to handle optional requirements. This was triggered by our quobyte nova driver (https://review.openstack.org/#/c/110722/18), who requires xattr, which we therefore added to requirements.txt (as it is provided by the requirements project). Points from the discussion: - If we add this we will be adding every requirement for every component --- this becomes to big. - Remove this requirement, no optional entries in requirements.txt, a 'deployer' has to know what dependencies the components he wants to use have --- Usually he does not know and installation becomes more issue prone - Other (in between) ideas??? Please note that this has some urgency, the change set referenced above has been in review for months and i'm trying to react asap on comments but the deadline is approaching (next week) and if i have to do bigger changes I'd like to know as fast as possible... Best regards SIlvan Kaiser -- -- *Quobyte* GmbH Boyenstr. 41 - 10115 Berlin-Mitte - Germany +49-30-814 591 800 - www.quobyte.com Amtsgericht Berlin-Charlottenburg, HRB 149012B management board: Dr. Felix Hupfeld, Dr. Björn Kolbeck, Dr. Jan Stender __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Requirements.txt and optional requirements
On 01/27/2015 12:18 AM, Silvan Kaiser wrote: Hello! Do dependencies required only in some contexts belong into requirements.txt? Yesterday we had a short discussion on #openstack-nova regarding how to handle optional requirements. This was triggered by our quobyte nova driver (https://review.openstack.org/#/c/110722/18), who requires xattr, which we therefore added to requirements.txt (as it is provided by the requirements project). Points from the discussion: - If we add this we will be adding every requirement for every component --- this becomes to big. - Remove this requirement, no optional entries in requirements.txt, a 'deployer' has to know what dependencies the components he wants to use have --- Usually he does not know and installation becomes more issue prone - Other (in between) ideas??? Please note that this has some urgency, the change set referenced above has been in review for months and i'm trying to react asap on comments but the deadline is approaching (next week) and if i have to do bigger changes I'd like to know as fast as possible... Typically the answer is no. The libvirt volume driver architecture is kind of weird in the fact that it doesn't actually split it's drivers out into separate files, so there can be a file level boundry (including loading) for these things. That being said, in general, optional things are not in requirements.txt. You will notice libvirt isn't even in requirements.txt, because it's not a nova requirement. Optional things should be handled in documentation at this point, and the code should be structured to not fail when that's not installed if it's not needed. But, also, in staring at the code, I'm confused that xattr is used in only 1 place, which is a fall back failure path. Seems like the code could easily be redone to not need it, and get info from mount instead, no? -Sean -- Sean Dague http://dague.net signature.asc Description: OpenPGP digital signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Requirements.txt and optional requirements
On 1/27/2015 2:18 AM, Silvan Kaiser wrote: Hello! Do dependencies required only in some contexts belong into requirements.txt? Yesterday we had a short discussion on #openstack-nova regarding how to handle optional requirements. This was triggered by our quobyte nova driver (https://review.openstack.org/#/c/110722/18), who requires xattr, which we therefore added to requirements.txt (as it is provided by the requirements project). Points from the discussion: - If we add this we will be adding every requirement for every component --- this becomes to big. - Remove this requirement, no optional entries in requirements.txt, a 'deployer' has to know what dependencies the components he wants to use have --- Usually he does not know and installation becomes more issue prone - Other (in between) ideas??? Please note that this has some urgency, the change set referenced above has been in review for months and i'm trying to react asap on comments but the deadline is approaching (next week) and if i have to do bigger changes I'd like to know as fast as possible... Best regards SIlvan Kaiser -- *Quobyte* GmbH Boyenstr. 41 - 10115 Berlin-Mitte - Germany +49-30-814 591 800 - www.quobyte.com http://www.quobyte.com/ Amtsgericht Berlin-Charlottenburg, HRB 149012B management board: Dr. Felix Hupfeld, Dr. Björn Kolbeck, Dr. Jan Stender __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev In my opinion the volume driver is optional and therefore the dependency is optional, it's all based on what the configuration is, the same as which DB or RPC backend you use, which is why those dependencies are in test-requirements.txt. While it's more obvious to a deployer that if you're going to configure Nova to use MySQL you need some MySQL packages to make it work, any deployer that's adding support for this volume driver should also probably be testing their deployment scripts, e.g. chef cookbooks/recipes, and if they haven't written their script correctly they'll find out that it blows up with an ImportError because of a missing xattr. Otherwise, [1]. [1] http://troll.me/images/the-most-interesting-man-in-the-world/i-dont-always-test-my-code-but-when-i-do-i-do-it-in-production.jpg -- Thanks, Matt Riedemann __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Requirements.txt and optional requirements
On 01/27/2015 07:14 AM, Matt Riedemann wrote: On 1/27/2015 2:18 AM, Silvan Kaiser wrote: Hello! Do dependencies required only in some contexts belong into requirements.txt? Yesterday we had a short discussion on #openstack-nova regarding how to handle optional requirements. This was triggered by our quobyte nova driver (https://review.openstack.org/#/c/110722/18), who requires xattr, which we therefore added to requirements.txt (as it is provided by the requirements project). Points from the discussion: - If we add this we will be adding every requirement for every component --- this becomes to big. - Remove this requirement, no optional entries in requirements.txt, a 'deployer' has to know what dependencies the components he wants to use have --- Usually he does not know and installation becomes more issue prone - Other (in between) ideas??? Please note that this has some urgency, the change set referenced above has been in review for months and i'm trying to react asap on comments but the deadline is approaching (next week) and if i have to do bigger changes I'd like to know as fast as possible... Best regards SIlvan Kaiser -- *Quobyte* GmbH Boyenstr. 41 - 10115 Berlin-Mitte - Germany +49-30-814 591 800 - www.quobyte.com http://www.quobyte.com/ Amtsgericht Berlin-Charlottenburg, HRB 149012B management board: Dr. Felix Hupfeld, Dr. Björn Kolbeck, Dr. Jan Stender __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev In my opinion the volume driver is optional and therefore the dependency is optional, it's all based on what the configuration is, the same as which DB or RPC backend you use, which is why those dependencies are in test-requirements.txt. While it's more obvious to a deployer that if you're going to configure Nova to use MySQL you need some MySQL packages to make it work, any deployer that's adding support for this volume driver should also probably be testing their deployment scripts, e.g. chef cookbooks/recipes, and if they haven't written their script correctly they'll find out that it blows up with an ImportError because of a missing xattr. Otherwise, [1]. Couple things... a) I agree with Sean and Matt here that this is an optional dependency and belongs in the deployment documentation and configuration management manifests. b) The Glance API image cache can use xattr if SQLite is not desired [1], and Glance does *not* list xattr as a dependency in requirements.txt. Swift also has a dependency on python-xattr [2]. So, this particular Python library is not an unknown by any means. c) Remember that even if you install python-xattr, that still doesn't mean it will automatically work. You still need to enable a filesystem that supports atime (i.e. noatime must not be set in fstab for the filesystem) [3]. Just an FYI. Best, -jay [1] https://github.com/openstack/glance/blob/master/glance/image_cache/drivers/xattr.py [2] https://github.com/openstack/swift/blob/master/requirements.txt#L11 [3] https://github.com/openstack/glance/blob/master/glance/image_cache/drivers/xattr.py#L23-L24 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Requirements.txt and optional requirements
On 01/27/2015 02:18 AM, Silvan Kaiser wrote: Hello! Do dependencies required only in some contexts belong into requirements.txt? Yesterday we had a short discussion on #openstack-nova regarding how to handle optional requirements. This was triggered by our quobyte nova driver (https://review.openstack.org/#/c/110722/18), who requires xattr, which we therefore added to requirements.txt (as it is provided by the requirements project). Points from the discussion: - If we add this we will be adding every requirement for every component --- this becomes to big. - Remove this requirement, no optional entries in requirements.txt, a 'deployer' has to know what dependencies the components he wants to use have --- Usually he does not know and installation becomes more issue prone - Other (in between) ideas??? Please note that this has some urgency, the change set referenced above has been in review for months and i'm trying to react asap on comments but the deadline is approaching (next week) and if i have to do bigger changes I'd like to know as fast as possible... Best regards SIlvan Kaiser I will just put in another plug for some work I had started around this (and never finished :-( ). https://review.openstack.org/#/c/83150/ (despite the Jenkins results, I'm pretty sure that was working locally for me) Also the discussion at http://lists.openstack.org/pipermail/openstack-dev/2014-February/026976.html There was general consensus on the approach and support has been in pbr for a long time now, but I've never had time to figure out what would be needed to make it play nicely with global requirements. I _think_ that's the only major blocker preventing this from being useful, so if someone wanted to pick it up and run with it that would be awesome. -Ben __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Requirements.txt and optional requirements
On Tue, Jan 27, 2015 at 07:12:29AM -0800, Sean Dague wrote: On 01/27/2015 12:18 AM, Silvan Kaiser wrote: Hello! Do dependencies required only in some contexts belong into requirements.txt? Yesterday we had a short discussion on #openstack-nova regarding how to handle optional requirements. This was triggered by our quobyte nova driver (https://review.openstack.org/#/c/110722/18), who requires xattr, which we therefore added to requirements.txt (as it is provided by the requirements project). Points from the discussion: - If we add this we will be adding every requirement for every component --- this becomes to big. - Remove this requirement, no optional entries in requirements.txt, a 'deployer' has to know what dependencies the components he wants to use have --- Usually he does not know and installation becomes more issue prone - Other (in between) ideas??? Please note that this has some urgency, the change set referenced above has been in review for months and i'm trying to react asap on comments but the deadline is approaching (next week) and if i have to do bigger changes I'd like to know as fast as possible... Typically the answer is no. The libvirt volume driver architecture is kind of weird in the fact that it doesn't actually split it's drivers out into separate files, so there can be a file level boundry (including loading) for these things. Not immediately relevant, but in the L cycle, I'm hoping to finally split the volume.py up into a subdirectory of files. The nova.conf 'volume_drivers' config will be removed by thenm, avoiding the back compat class naming problem we'd have if we split it in Kilo Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Requirements.txt and optional requirements
Am 27.01.2015 um 16:51 schrieb Jay Pipes jaypi...@gmail.com: […snip...] a) I agree with Sean and Matt here that this is an optional dependency and belongs in the deployment documentation and configuration management manifests b) The Glance API image cache can use xattr if SQLite is not desired [1], and Glance does *not* list xattr as a dependency in requirements.txt. Swift also has a dependency on python-xattr [2]. So, this particular Python library is not an unknown by any means. Do you happen to know how Glance handles this if the dep. is not handled in requirements.txt? c) Remember that even if you install python-xattr, that still doesn't mean it will automatically work. You still need to enable a filesystem that supports atime (i.e. noatime must not be set in fstab for the filesystem) [3]. Just an FYI. In our case the Quobyte file system does support that metadata and we use it to verify that a QB volume was mounted. Best regards Silvan Best, -jay [1] https://github.com/openstack/glance/blob/master/glance/image_cache/drivers/xattr.py [2] https://github.com/openstack/swift/blob/master/requirements.txt#L11 [3] https://github.com/openstack/glance/blob/master/glance/image_cache/drivers/xattr.py#L23-L24 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -- *Quobyte* GmbH Boyenstr. 41 - 10115 Berlin-Mitte - Germany +49-30-814 591 800 - www.quobyte.com Amtsgericht Berlin-Charlottenburg, HRB 149012B management board: Dr. Felix Hupfeld, Dr. Björn Kolbeck, Dr. Jan Stender __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Requirements.txt and optional requirements
On 01/27/2015 09:13 AM, Silvan Kaiser wrote: Am 27.01.2015 um 16:51 schrieb Jay Pipes jaypi...@gmail.com: b) The Glance API image cache can use xattr if SQLite is not desired [1], and Glance does *not* list xattr as a dependency in requirements.txt. Swift also has a dependency on python-xattr [2]. So, this particular Python library is not an unknown by any means. Do you happen to know how Glance handles this if the dep. is not handled in requirements.txt? Yep, it's considered a documentation thing and handled in configuration management manifests... Best, -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Requirements.txt and optional requirements
On 01/27/2015 08:14 AM, Daniel P. Berrange wrote: On Tue, Jan 27, 2015 at 07:12:29AM -0800, Sean Dague wrote: On 01/27/2015 12:18 AM, Silvan Kaiser wrote: Hello! Do dependencies required only in some contexts belong into requirements.txt? Yesterday we had a short discussion on #openstack-nova regarding how to handle optional requirements. This was triggered by our quobyte nova driver (https://review.openstack.org/#/c/110722/18), who requires xattr, which we therefore added to requirements.txt (as it is provided by the requirements project). Points from the discussion: - If we add this we will be adding every requirement for every component --- this becomes to big. - Remove this requirement, no optional entries in requirements.txt, a 'deployer' has to know what dependencies the components he wants to use have --- Usually he does not know and installation becomes more issue prone - Other (in between) ideas??? Please note that this has some urgency, the change set referenced above has been in review for months and i'm trying to react asap on comments but the deadline is approaching (next week) and if i have to do bigger changes I'd like to know as fast as possible... Typically the answer is no. The libvirt volume driver architecture is kind of weird in the fact that it doesn't actually split it's drivers out into separate files, so there can be a file level boundry (including loading) for these things. Not immediately relevant, but in the L cycle, I'm hoping to finally split the volume.py up into a subdirectory of files. The nova.conf 'volume_drivers' config will be removed by thenm, avoiding the back compat class naming problem we'd have if we split it in Kilo Woot! awesome. -Sean -- Sean Dague http://dague.net signature.asc Description: OpenPGP digital signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
