Re: [openstack-dev] [Nova] Requirements.txt and optional requirements

2015-01-30 Thread Morgan Fainberg
As point, we are trying to move away from this model. Having to know the 
dependencies is a bad experience in general. But with the move to eliminate 
optional parts of the api, most of these become real dependencies for 
keystone (a few things will still be optional eg memcache lib). 

--Morgan 

Sent via mobile

On Jan 30, 2015, at 17:08, Alan Pevec ape...@gmail.com wrote:

 - Remove this requirement, no optional entries in requirements.txt, a
 'deployer' has to know what dependencies the components he wants to use have
 
 Keystone is documenting its optional dependencies in test-requirements.txt
 look for # Optional ... comments in
 http://git.openstack.org/cgit/openstack/keystone/tree/test-requirements.txt
 
 
 Cheers,
 Alan
 
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova] Requirements.txt and optional requirements

2015-01-30 Thread Alan Pevec
 - Remove this requirement, no optional entries in requirements.txt, a
 'deployer' has to know what dependencies the components he wants to use have

Keystone is documenting its optional dependencies in test-requirements.txt
look for # Optional ... comments in
http://git.openstack.org/cgit/openstack/keystone/tree/test-requirements.txt


Cheers,
Alan

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova] Requirements.txt and optional requirements

2015-01-30 Thread Silvan Kaiser
Hello again,
i submitted a new patch set for this at
https://review.openstack.org/#/c/110722/ ,
looking forward to further reviews. :)
Best regards
Silvan


2015-01-28 10:19 GMT+01:00 Silvan Kaiser sil...@quobyte.com:

 Hi All!
 Thanks for the feedback!

 I'll remove xattr from the requirements in my change set.
 Currently i'm working on a workaround to execute 'getfattr' instead of the
 xattr api call. We can asure getfattr is available via package dependencies
 of our client who has to be installed either way.

 I'm also checking out your proposal in parallel, i cannot find any
 documentation about the 'configuration management manifests', do you mean
 the puppet manifests? Otherwise, could somebody please give me a pointer to
 their documentation, etc.?

 Best regards
 Silvan


 2015-01-27 18:32 GMT+01:00 Jay Pipes jaypi...@gmail.com:

 On 01/27/2015 09:13 AM, Silvan Kaiser wrote:

 Am 27.01.2015 um 16:51 schrieb Jay Pipes jaypi...@gmail.com:
 b) The Glance API image cache can use xattr if SQLite is not
 desired [1], and Glance does *not* list xattr as a dependency in
 requirements.txt. Swift also has a dependency on python-xattr [2].
 So, this particular Python library is not an unknown by any means.

 Do you happen to know how Glance handles this if the dep. is not
 handled in requirements.txt?


 Yep, it's considered a documentation thing and handled in configuration
 management manifests...

 Best,
 -jay


 
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:
 unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 

--
*Quobyte* GmbH
Boyenstr. 41 - 10115 Berlin-Mitte - Germany
+49-30-814 591 800 - www.quobyte.com
Amtsgericht Berlin-Charlottenburg, HRB 149012B
management board: Dr. Felix Hupfeld, Dr. Björn Kolbeck, Dr. Jan Stender
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova] Requirements.txt and optional requirements

2015-01-28 Thread Silvan Kaiser
Hi All!
Thanks for the feedback!

I'll remove xattr from the requirements in my change set.
Currently i'm working on a workaround to execute 'getfattr' instead of the
xattr api call. We can asure getfattr is available via package dependencies
of our client who has to be installed either way.

I'm also checking out your proposal in parallel, i cannot find any
documentation about the 'configuration management manifests', do you mean
the puppet manifests? Otherwise, could somebody please give me a pointer to
their documentation, etc.?

Best regards
Silvan


2015-01-27 18:32 GMT+01:00 Jay Pipes jaypi...@gmail.com:

 On 01/27/2015 09:13 AM, Silvan Kaiser wrote:

 Am 27.01.2015 um 16:51 schrieb Jay Pipes jaypi...@gmail.com:
 b) The Glance API image cache can use xattr if SQLite is not
 desired [1], and Glance does *not* list xattr as a dependency in
 requirements.txt. Swift also has a dependency on python-xattr [2].
 So, this particular Python library is not an unknown by any means.

 Do you happen to know how Glance handles this if the dep. is not
 handled in requirements.txt?


 Yep, it's considered a documentation thing and handled in configuration
 management manifests...

 Best,
 -jay


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


-- 

--
*Quobyte* GmbH
Boyenstr. 41 - 10115 Berlin-Mitte - Germany
+49-30-814 591 800 - www.quobyte.com
Amtsgericht Berlin-Charlottenburg, HRB 149012B
management board: Dr. Felix Hupfeld, Dr. Björn Kolbeck, Dr. Jan Stender
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova] Requirements.txt and optional requirements

2015-01-27 Thread Sean Dague
On 01/27/2015 12:18 AM, Silvan Kaiser wrote:
 Hello!
 Do dependencies required only in some contexts belong into requirements.txt?
 
 Yesterday we had a short discussion on #openstack-nova regarding how to
 handle optional requirements. This was triggered by our quobyte nova
 driver (https://review.openstack.org/#/c/110722/18), who requires xattr,
 which we therefore added to requirements.txt (as it is provided by the
 requirements project).
 
 Points from the discussion:
 - If we add this we will be adding every requirement for every component
 --- this becomes to big.
 - Remove this requirement, no optional entries in requirements.txt, a
 'deployer' has to know what dependencies the components he wants to use have
 --- Usually he does not know and installation becomes more issue prone
 - Other (in between) ideas???
 
 Please note that this has some urgency, the change set referenced above
 has been in review for months and i'm trying to react asap on comments
 but the deadline is approaching (next week) and if i have to do bigger
 changes I'd like to know as fast as possible...

Typically the answer is no. The libvirt volume driver architecture is
kind of weird in the fact that it doesn't actually split it's drivers
out into separate files, so there can be a file level boundry (including
loading) for these things.

That being said, in general, optional things are not in
requirements.txt. You will notice libvirt isn't even in
requirements.txt, because it's not a nova requirement.

Optional things should be handled in documentation at this point, and
the code should be structured to not fail when that's not installed if
it's not needed.

But, also, in staring at the code, I'm confused that xattr is used in
only 1 place, which is a fall back failure path. Seems like the code
could easily be redone to not need it, and get info from mount instead, no?

-Sean

-- 
Sean Dague
http://dague.net



signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova] Requirements.txt and optional requirements

2015-01-27 Thread Matt Riedemann



On 1/27/2015 2:18 AM, Silvan Kaiser wrote:

Hello!
Do dependencies required only in some contexts belong into requirements.txt?

Yesterday we had a short discussion on #openstack-nova regarding how to
handle optional requirements. This was triggered by our quobyte nova
driver (https://review.openstack.org/#/c/110722/18), who requires xattr,
which we therefore added to requirements.txt (as it is provided by the
requirements project).

Points from the discussion:
- If we add this we will be adding every requirement for every component
--- this becomes to big.
- Remove this requirement, no optional entries in requirements.txt, a
'deployer' has to know what dependencies the components he wants to use have
--- Usually he does not know and installation becomes more issue prone
- Other (in between) ideas???

Please note that this has some urgency, the change set referenced above
has been in review for months and i'm trying to react asap on comments
but the deadline is approaching (next week) and if i have to do bigger
changes I'd like to know as fast as possible...

Best regards
SIlvan Kaiser


--
*Quobyte* GmbH
Boyenstr. 41 - 10115 Berlin-Mitte - Germany
+49-30-814 591 800 - www.quobyte.com http://www.quobyte.com/
Amtsgericht Berlin-Charlottenburg, HRB 149012B
management board: Dr. Felix Hupfeld, Dr. Björn Kolbeck, Dr. Jan Stender


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



In my opinion the volume driver is optional and therefore the dependency 
is optional, it's all based on what the configuration is, the same as 
which DB or RPC backend you use, which is why those dependencies are in 
test-requirements.txt.


While it's more obvious to a deployer that if you're going to configure 
Nova to use MySQL you need some MySQL packages to make it work, any 
deployer that's adding support for this volume driver should also 
probably be testing their deployment scripts, e.g. chef 
cookbooks/recipes, and if they haven't written their script correctly 
they'll find out that it blows up with an ImportError because of a 
missing xattr.  Otherwise, [1].


[1] 
http://troll.me/images/the-most-interesting-man-in-the-world/i-dont-always-test-my-code-but-when-i-do-i-do-it-in-production.jpg


--

Thanks,

Matt Riedemann


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova] Requirements.txt and optional requirements

2015-01-27 Thread Jay Pipes

On 01/27/2015 07:14 AM, Matt Riedemann wrote:

On 1/27/2015 2:18 AM, Silvan Kaiser wrote:

Hello!
Do dependencies required only in some contexts belong into
requirements.txt?

Yesterday we had a short discussion on #openstack-nova regarding how to
handle optional requirements. This was triggered by our quobyte nova
driver (https://review.openstack.org/#/c/110722/18), who requires xattr,
which we therefore added to requirements.txt (as it is provided by the
requirements project).

Points from the discussion:
- If we add this we will be adding every requirement for every component
--- this becomes to big.
- Remove this requirement, no optional entries in requirements.txt, a
'deployer' has to know what dependencies the components he wants to
use have
--- Usually he does not know and installation becomes more issue prone
- Other (in between) ideas???

Please note that this has some urgency, the change set referenced above
has been in review for months and i'm trying to react asap on comments
but the deadline is approaching (next week) and if i have to do bigger
changes I'd like to know as fast as possible...

Best regards
SIlvan Kaiser


--
*Quobyte* GmbH
Boyenstr. 41 - 10115 Berlin-Mitte - Germany
+49-30-814 591 800 - www.quobyte.com http://www.quobyte.com/
Amtsgericht Berlin-Charlottenburg, HRB 149012B
management board: Dr. Felix Hupfeld, Dr. Björn Kolbeck, Dr. Jan Stender


__

OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



In my opinion the volume driver is optional and therefore the dependency
is optional, it's all based on what the configuration is, the same as
which DB or RPC backend you use, which is why those dependencies are in
test-requirements.txt.

While it's more obvious to a deployer that if you're going to configure
Nova to use MySQL you need some MySQL packages to make it work, any
deployer that's adding support for this volume driver should also
probably be testing their deployment scripts, e.g. chef
cookbooks/recipes, and if they haven't written their script correctly
they'll find out that it blows up with an ImportError because of a
missing xattr.  Otherwise, [1].


Couple things...

a) I agree with Sean and Matt here that this is an optional dependency 
and belongs in the deployment documentation and configuration management 
manifests.


b) The Glance API image cache can use xattr if SQLite is not desired 
[1], and Glance does *not* list xattr as a dependency in 
requirements.txt. Swift also has a dependency on python-xattr [2]. So, 
this particular Python library is not an unknown by any means.


c) Remember that even if you install python-xattr, that still doesn't 
mean it will automatically work. You still need to enable a filesystem 
that supports atime (i.e. noatime must not be set in fstab for the 
filesystem) [3]. Just an FYI.


Best,
-jay

[1] 
https://github.com/openstack/glance/blob/master/glance/image_cache/drivers/xattr.py

[2] https://github.com/openstack/swift/blob/master/requirements.txt#L11
[3] 
https://github.com/openstack/glance/blob/master/glance/image_cache/drivers/xattr.py#L23-L24


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova] Requirements.txt and optional requirements

2015-01-27 Thread Ben Nemec
On 01/27/2015 02:18 AM, Silvan Kaiser wrote:
 Hello!
 Do dependencies required only in some contexts belong into requirements.txt?
 
 Yesterday we had a short discussion on #openstack-nova regarding how to
 handle optional requirements. This was triggered by our quobyte nova driver
 (https://review.openstack.org/#/c/110722/18), who requires xattr, which we
 therefore added to requirements.txt (as it is provided by the requirements
 project).
 
 Points from the discussion:
 - If we add this we will be adding every requirement for every component
 --- this becomes to big.
 - Remove this requirement, no optional entries in requirements.txt, a
 'deployer' has to know what dependencies the components he wants to use have
 --- Usually he does not know and installation becomes more issue prone
 - Other (in between) ideas???
 
 Please note that this has some urgency, the change set referenced above has
 been in review for months and i'm trying to react asap on comments but the
 deadline is approaching (next week) and if i have to do bigger changes I'd
 like to know as fast as possible...
 
 Best regards
 SIlvan Kaiser

I will just put in another plug for some work I had started around this
(and never finished :-( ).

https://review.openstack.org/#/c/83150/ (despite the Jenkins results,
I'm pretty sure that was working locally for me)

Also the discussion at
http://lists.openstack.org/pipermail/openstack-dev/2014-February/026976.html

There was general consensus on the approach and support has been in pbr
for a long time now, but I've never had time to figure out what would be
needed to make it play nicely with global requirements.  I _think_
that's the only major blocker preventing this from being useful, so if
someone wanted to pick it up and run with it that would be awesome.

-Ben

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova] Requirements.txt and optional requirements

2015-01-27 Thread Daniel P. Berrange
On Tue, Jan 27, 2015 at 07:12:29AM -0800, Sean Dague wrote:
 On 01/27/2015 12:18 AM, Silvan Kaiser wrote:
  Hello!
  Do dependencies required only in some contexts belong into requirements.txt?
  
  Yesterday we had a short discussion on #openstack-nova regarding how to
  handle optional requirements. This was triggered by our quobyte nova
  driver (https://review.openstack.org/#/c/110722/18), who requires xattr,
  which we therefore added to requirements.txt (as it is provided by the
  requirements project).
  
  Points from the discussion:
  - If we add this we will be adding every requirement for every component
  --- this becomes to big.
  - Remove this requirement, no optional entries in requirements.txt, a
  'deployer' has to know what dependencies the components he wants to use have
  --- Usually he does not know and installation becomes more issue prone
  - Other (in between) ideas???
  
  Please note that this has some urgency, the change set referenced above
  has been in review for months and i'm trying to react asap on comments
  but the deadline is approaching (next week) and if i have to do bigger
  changes I'd like to know as fast as possible...
 
 Typically the answer is no. The libvirt volume driver architecture is
 kind of weird in the fact that it doesn't actually split it's drivers
 out into separate files, so there can be a file level boundry (including
 loading) for these things.

Not immediately relevant, but in the L cycle, I'm hoping to finally
split the volume.py up into a subdirectory of files. The nova.conf
'volume_drivers' config will be removed by thenm, avoiding the back
compat class naming problem we'd have if we split it in Kilo

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova] Requirements.txt and optional requirements

2015-01-27 Thread Silvan Kaiser

 Am 27.01.2015 um 16:51 schrieb Jay Pipes jaypi...@gmail.com:
 

[…snip...]

 a) I agree with Sean and Matt here that this is an optional dependency and 
 belongs in the deployment documentation and configuration management manifests
 
 b) The Glance API image cache can use xattr if SQLite is not desired [1], and 
 Glance does *not* list xattr as a dependency in requirements.txt. Swift also 
 has a dependency on python-xattr [2]. So, this particular Python library is 
 not an unknown by any means.
Do you happen to know how Glance handles this if the dep. is not handled in 
requirements.txt?

 
 c) Remember that even if you install python-xattr, that still doesn't mean it 
 will automatically work. You still need to enable a filesystem that supports 
 atime (i.e. noatime must not be set in fstab for the filesystem) [3]. Just an 
 FYI.
In our case the Quobyte file system does support that metadata and we use it to 
verify that a QB volume was mounted.

Best regards
Silvan


 
 Best,
 -jay
 
 [1] 
 https://github.com/openstack/glance/blob/master/glance/image_cache/drivers/xattr.py
 [2] https://github.com/openstack/swift/blob/master/requirements.txt#L11
 [3] 
 https://github.com/openstack/glance/blob/master/glance/image_cache/drivers/xattr.py#L23-L24
 
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


-- 

--
*Quobyte* GmbH
Boyenstr. 41 - 10115 Berlin-Mitte - Germany
+49-30-814 591 800 - www.quobyte.com
Amtsgericht Berlin-Charlottenburg, HRB 149012B
management board: Dr. Felix Hupfeld, Dr. Björn Kolbeck, Dr. Jan Stender

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova] Requirements.txt and optional requirements

2015-01-27 Thread Jay Pipes

On 01/27/2015 09:13 AM, Silvan Kaiser wrote:

Am 27.01.2015 um 16:51 schrieb Jay Pipes jaypi...@gmail.com:
b) The Glance API image cache can use xattr if SQLite is not
desired [1], and Glance does *not* list xattr as a dependency in
requirements.txt. Swift also has a dependency on python-xattr [2].
So, this particular Python library is not an unknown by any means.

Do you happen to know how Glance handles this if the dep. is not
handled in requirements.txt?


Yep, it's considered a documentation thing and handled in configuration 
management manifests...


Best,
-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova] Requirements.txt and optional requirements

2015-01-27 Thread Sean Dague
On 01/27/2015 08:14 AM, Daniel P. Berrange wrote:
 On Tue, Jan 27, 2015 at 07:12:29AM -0800, Sean Dague wrote:
 On 01/27/2015 12:18 AM, Silvan Kaiser wrote:
 Hello!
 Do dependencies required only in some contexts belong into requirements.txt?

 Yesterday we had a short discussion on #openstack-nova regarding how to
 handle optional requirements. This was triggered by our quobyte nova
 driver (https://review.openstack.org/#/c/110722/18), who requires xattr,
 which we therefore added to requirements.txt (as it is provided by the
 requirements project).

 Points from the discussion:
 - If we add this we will be adding every requirement for every component
 --- this becomes to big.
 - Remove this requirement, no optional entries in requirements.txt, a
 'deployer' has to know what dependencies the components he wants to use have
 --- Usually he does not know and installation becomes more issue prone
 - Other (in between) ideas???

 Please note that this has some urgency, the change set referenced above
 has been in review for months and i'm trying to react asap on comments
 but the deadline is approaching (next week) and if i have to do bigger
 changes I'd like to know as fast as possible...

 Typically the answer is no. The libvirt volume driver architecture is
 kind of weird in the fact that it doesn't actually split it's drivers
 out into separate files, so there can be a file level boundry (including
 loading) for these things.
 
 Not immediately relevant, but in the L cycle, I'm hoping to finally
 split the volume.py up into a subdirectory of files. The nova.conf
 'volume_drivers' config will be removed by thenm, avoiding the back
 compat class naming problem we'd have if we split it in Kilo

Woot! awesome.

-Sean

-- 
Sean Dague
http://dague.net



signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev