Re: [openstack-dev] [kolla][kolla-ansible] Proposing Bertrand Lallau for kolla and kolla-ansible core

[Dave Walker]

+1, some great contributions!

Thanks

On 8 May 2017 at 19:11, Kwasniewska, Alicja 
wrote:

> +1 Congrats☺
>
> Regards,
> Alicja Kwasniewska
>
>
>
> *From: *"Vikram Hosakote (vhosakot)" 
> *Reply-To: *"OpenStack Development Mailing List (not for usage
> questions)" 
> *Date: *Monday, May 8, 2017 at 6:54 AM
> *To: *"OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> *Subject: *Re: [openstack-dev] [kolla][kolla-ansible] Proposing Bertrand
> Lallau for kolla and kolla-ansible core
>
>
>
> +1  Great job Bertrand!
>
>
>
> Regards,
>
> Vikram Hosakote
>
> IRC:  vhosakot
>
>
>
> *From: *Michał Jastrzębski 
> *Reply-To: *"OpenStack Development Mailing List (not for usage
> questions)" 
> *Date: *Tuesday, May 02, 2017 at 10:13 PM
> *To: *"OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> *Subject: *[openstack-dev] [kolla][kolla-ansible] Proposing Bertrand
> Lallau for kolla and kolla-ansible core
>
>
>
> Hello,
>
>
>
> It's my pleasure to start another core reviewer vote. Today it's
>
> Bertrand (blallau). Consider this mail my +1 vote. Members of
>
> kolla-ansible and kolla core team, please cast your votes:) Voting
>
> will be open for 2 weeks (until 16th of May).
>
>
>
> I also wanted to say that Bertrand went through our core mentorship
>
> program (if only for few weeks because he did awesome job before too)
>
> :)
>
>
>
> Thank you,
>
> Michal
>
> __
>
> OpenStack Development Mailing List (not for usage questions)
>
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] Proposing duonghq for core

[Dave Walker]

+1, some great contributions.  Looking forward to having Duong on the team.

--
Kind Regards,
Dave Walker

On 15 March 2017 at 19:52, Vikram Hosakote (vhosakot) <vhosa...@cisco.com>
wrote:

> +1  Great job Duong!
>
>
>
> Regards,
>
> Vikram Hosakote
>
> IRC:  vhosakot
>
>
>
> *From: *Michał Jastrzębski <inc...@gmail.com>
> *Reply-To: *"OpenStack Development Mailing List (not for usage
> questions)" <openstack-dev@lists.openstack.org>
> *Date: *Wednesday, March 08, 2017 at 11:21 PM
> *To: *"OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> *Subject: *[openstack-dev] [kolla] Proposing duonghq for core
>
>
>
> Hello,
>
>
>
> I'd like to start voting to include Duong (duonghq) in Kolla and
>
> Kolla-ansible core teams. Voting will be open for 2 weeks (ends at
>
> 21st of March).
>
>
>
> Consider this my +1 vote.
>
>
>
> Cheers,
>
> Michal
>
> __
>
> OpenStack Development Mailing List (not for usage questions)
>
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] Domains support

[Dave Walker]

Try /etc/kolla/config/keystone/domains/keystone.$DOMAIN.conf

Thanks

On 2 February 2017 at 00:20, Christian Tardif <christian.tar...@servinfo.ca>
wrote:

> Will sure give it a try ! And from a kolla perspective, it means that this
> file should go in /etc/kolla/config/domains/keystone.$DOMAIN.conf in
> order to be pushed to the relevant containers ?
> --
>
>
> *Christian Tardif*christian.tar...@servinfo.ca
>
> SVP, pensez à l’environnement avant d’imprimer ce message.
>
>
>
> -- Message d'origine --
> De: "Dave Walker" <em...@daviey.com>
> À: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> Envoyé : 2017-02-01 11:39:15
> Objet : Re: [openstack-dev] [kolla] Domains support
>
> Hi Christian,
>
> I added the domain support, but I didn't document it as well as I should
> have. Apologies!
>
> This is the config I am using to talk to a windows AD server.  Hope this
> helps.
>
> create a domain specific file:
> etc/keystone/domains/keystone.$DOMAIN.conf:
>
> [ldap]
> use_pool = true
> pool_size = 10
> pool_retry_max = 3
> pool_retry_delay = 0.1
> pool_connection_timeout = -1
> pool_connection_lifetime = 600
> use_auth_pool = false
> auth_pool_size = 100
> auth_pool_connection_lifetime = 60
> url = ldap://server1:389,ldap://server2:389
> user = CN=Linux SSSD Kerberos Service Account,CN=Users,DC=example,DC=com
> password = password
> suffix   = dc=example,dc=com
> user_tree_dn = OU=Personnel,OU=Users,OU=
> example,DC=example,DC=com
> user_objectclass = person
> user_filter  = (memberOf=CN=mail,OU=GPO
> Security,OU=Groups,OU=COMPANY,DC=example,DC=com)
> user_id_attribute= sAMAccountName
> user_name_attribute  = sAMAccountName
> user_description_attribute = displayName
> user_mail_attribute  = mail
> user_pass_attribute  =
> user_enabled_attribute   = userAccountControl
> user_enabled_mask= 2
> user_enabled_default = 512
> user_attribute_ignore= password,tenant_id,tenants
> group_tree_dn= OU=GPO Security,OU=Groups,OU=COMPANY,
> DC=example,DC=com
> group_name_attribute = name
> group_id_attribute   = cn
> group_objectclass= group
> group_member_attribute   = member
>
> [identity]
> driver = keystone.identity.backends.ldap.Identity
>
> [assignment]
> driver = keystone.assignment.backends.sql.Assignment
>
> --
> Kind Regards,
> Dave Walker
>
> On 1 February 2017 at 05:03, Christian Tardif <
> christian.tar...@servinfo.ca> wrote:
>
>> Hi,
>>
>> I'm looking for domains support in Kolla. I've searched, but didn't find
>> anything relevant. Could someone point me how to achieve this?
>>
>> What I'm really looking for, in fact, is a decent way or setting auth
>> through LDAP backend while keeping service users (neutron, for example) in
>> the SQL backend. I know that this can be achieved with domains support
>> (leaving default domain on SQL, and another domain for LDAP users. Or maybe
>> there's another of doing this?
>>
>> Thanks,
>> --
>>
>>
>> *Christian Tardif*christian.tar...@servinfo.ca
>>
>> 
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscrib
>> e
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] Domains support

[Dave Walker]

Hi Christian,

I added the domain support, but I didn't document it as well as I should
have. Apologies!

This is the config I am using to talk to a windows AD server.  Hope this
helps.

create a domain specific file:
etc/keystone/domains/keystone.$DOMAIN.conf:

[ldap]
use_pool = true
pool_size = 10
pool_retry_max = 3
pool_retry_delay = 0.1
pool_connection_timeout = -1
pool_connection_lifetime = 600
use_auth_pool = false
auth_pool_size = 100
auth_pool_connection_lifetime = 60
url = ldap://server1:389,ldap://server2:389
user = CN=Linux SSSD Kerberos Service Account,CN=Users,DC=example,DC=com
password = password
suffix   = dc=example,dc=com
user_tree_dn =
OU=Personnel,OU=Users,OU=example,DC=example,DC=com
user_objectclass = person
user_filter  = (memberOf=CN=mail,OU=GPO
Security,OU=Groups,OU=COMPANY,DC=example,DC=com)
user_id_attribute= sAMAccountName
user_name_attribute  = sAMAccountName
user_description_attribute = displayName
user_mail_attribute  = mail
user_pass_attribute  =
user_enabled_attribute   = userAccountControl
user_enabled_mask= 2
user_enabled_default = 512
user_attribute_ignore= password,tenant_id,tenants
group_tree_dn= OU=GPO
Security,OU=Groups,OU=COMPANY,DC=example,DC=com
group_name_attribute = name
group_id_attribute   = cn
group_objectclass= group
group_member_attribute   = member

[identity]
driver = keystone.identity.backends.ldap.Identity

[assignment]
driver = keystone.assignment.backends.sql.Assignment

--
Kind Regards,
Dave Walker

On 1 February 2017 at 05:03, Christian Tardif <christian.tar...@servinfo.ca>
wrote:

> Hi,
>
> I'm looking for domains support in Kolla. I've searched, but didn't find
> anything relevant. Could someone point me how to achieve this?
>
> What I'm really looking for, in fact, is a decent way or setting auth
> through LDAP backend while keeping service users (neutron, for example) in
> the SQL backend. I know that this can be achieved with domains support
> (leaving default domain on SQL, and another domain for LDAP users. Or maybe
> there's another of doing this?
>
> Thanks,
> --
>
>
> *Christian Tardif*christian.tar...@servinfo.ca
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] Vote to add zhubingbing to core team

[Dave Walker]

On 29 November 2016 at 16:21, Michał Jastrzębski  wrote:

> Hello team!
>
> I'd like to propose to add zhubingbing to kolla (and kolla-ansible)
> core teams. He did great job reviewing code during last couple of
> months.
>
> Consider this proposal +1 from me, voting will be open for 1 week
> (until Dec 6) or if we get unanimous agreement (or veto) before.
>
> Regards,
> Michal
>

+1, Great reviews Zhu!  I've also been wanting to try out the designate and
panko work you did.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [requirements][kolla][security] pycrypto vs cryptography

[Dave Walker]

Hey Steve,

All of the credential generation is optional right?  I mean, as far as
kolla is concerned - it doesn't *need* to generate the passwords... If
/etc/kolla/passwords.yml is created outside of kolla-genpwd, then kolla
isn't creating any credentials itself and the algorithm, entropy and policy
is transparent to kolla.

On 8 November 2016 at 21:50, Steven Dake (stdake)  wrote:

> Ok,
>
> Pavo has told me he has exceptions in place for everything related to
> Kolla.  He says as long as we don’t use MD5, he is good to go for a 232
> node deploy with more to follow (assuming Kolla works out of the box at
> that scale - we have only tested 123 node scale).
>
> We do some basic PRNG to generate passwords, and some PKCS#11 (iirc) algos
> to generate passwords, and we also generate some ssh public/private keys.
>
> Hope the security context helps.
>
> Thanks everyone on his thread for providing guidance.  RobC++ on article.
>
> Regards
> -steve
>
>
>
>
> On 11/8/16, 1:46 PM, "Clint Byrum"  wrote:
>
> >Excerpts from Ian Cordasco's message of 2016-11-08 16:11:26 -0500:
> >> Can I ask why FIPS compliance is a requirement for Kolla? This seems
> >> like an odd request for a deployment project.
> >>
> >
> >Guessing it's for the modules that need to communicate securely with
> >OpenStack itself.
> >
> >___
> ___
> >OpenStack Development Mailing List (not for usage questions)
> >Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:
> unsubscribe
> >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Security] XML Attacks and DefusedXML on Global Requirements

[Dave Walker]

On 27 September 2016 at 19:19, Sean Dague <s...@dague.net> wrote:

> On 09/27/2016 01:24 PM, Travis McPeak wrote:
> > There are several attacks (https://pypi.python.org/pypi/defusedxml#id3)
> > that can be performed when XML is parsed from untrusted input.
> > DefusedXML offers safe alternatives to XML parsing libraries but is not
> > currently part of global requirements.
> >
> > I propose adding DefusedXML to global requirements so that projects have
> > an option for safe XML parsing.  Does anybody have any thoughts or
> > objections?
>
> Out of curiosity, are there specific areas of concern in existing
> projects here? Most projects have dropped XML API support.
>
>
Outbound XML datasources which are parsed still used with at least nova
vmware support and multiple cinder drivers.

openstack/ec2-api is still providing an xml api service?

--
Kind Regards,
Dave Walker
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Security] Picking a new tag

[Dave Walker]

On 22 September 2016 at 19:52, Ian Cordasco <sigmaviru...@gmail.com> wrote:

> During the OpenStack Security Project (OSSP) meeting today we
> discussed the fact that some MUAs don't filter the "[Security]" tag
> very well and this causes a bit of an overload for people trying to
> follow the internal workings of the OSSP. We were briefly side-tracked
> trying to come up with a different tag that would be less likely to
> cause false positives with filtering.
>
> This also seemed like a good opportunity to use the mailing list to
> come up with our new tag, since we've had such an atrocious time using
> it in the past.
>
> Some of the suggestions I recall from the meeting include:
>
> - OSSP
> - openstack-sec
>
> I think we'd want to keep "openstack" out of our tag name, so maybe
>
> - sec-project
> - security-project
>
> Thoughts?
>
>
Hi,

I'm not convinced it needs changing.  [security] is a pretty logical topic
tag, and rolls off the keyboard quite easily.

So the real issue is filtering on headers.  Most mail providers do provide
this, and certainly MUA.. however gmail does make it a bit harder.

Mailman wont see all arbitrary "[strings]", but the ones that are added
allows the user to subscript specifically to them.  [security] is one such
tag, which means that OSSP interested parties could subscribe
*specifically* to that tag (and probably [all] for good measure).

However, this does mean that these subscribers have little chance of seeing
any other mail.  What would be better would be to add labels (gmail
terminology) specifically to [security] threads.

Mailman does add an X- field such as:
"X-Topics: foo Security bar"

Sadly you cannot search using the Gmail interface for these fields.. but it
does provide a service to run scripts on mails on regular intervals, which
will allow the desired labelling.

I've written a sample script, and it works.  Go to https://script.google.com
and add the contents of
https://gist.github.com/Daviey/eb61c284b6d3bf6562763db2cb8a7351 .  Click
the clock symbol, and set an hourly interval.

This will mean that all [Security] tagged mails receive an OSSP gmail label.

HTH, let me know if it does.

--
Kind Regards,
Dave Walker
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [security] [salt] Removal of Security and OpenStackSalt project teams from the Big Tent

[Dave Walker]

On 21 September 2016 at 22:41, Kyle Mestery <mest...@mestery.com> wrote:

> On Wed, Sep 21, 2016 at 3:35 PM, Thierry Carrez <thie...@openstack.org>
> wrote:
> > Chivers, Doug wrote:
> >> My concern is with the original wording “The suggested way forward
> there would be to remove the "Security project team"”.
> >>
> >> This seems like a move to instantly reduce investment in OpenStack
> security, because the majority of members of the Security Project are
> corporately funded, which will be significantly impacted by the removal of
> the security project. I have no knowledge over the difference between a
> working group and a project, like everyone else on the project we are
> simply here to contribute to OpenStack security, drive innovation in
> security, deliver documentation like OSSNs, etc, rather than get involved
> in the politics of OpenStack.
> >>
> >> In response to the various questions of why no-one from our project
> noticed that we didn’t have a nomination for the PTL, we assumed that was
> taken care of. Realistically maybe two or three people on the security
> project have the availability to be PTL, one being our current PTL, for all
> the rest of us its simply not a concern until we need to vote.
> >>
> >> On a personal note, reading –dev is unfortunately a lower priority than
> designing architectures, responding to customers and sales teams, closing
> tickets, writing decks and on the afternoon or so I can spend each week,
> working on my upstream projects (this week it was:
> https://review.openstack.org/#/c/357978/5 - thanks to the Barbican team
> for all their work). Possibly this is wrong, but I didn’t sign up as a
> contributor to spend all my spare time reading mailing lists.
> >
> > So while I still think there is a slight disconnect (like, members of
> > the security team are less often involved in other teams) that results
> > in the Security team being more likely to miss the very few process
> > deadlines that apply to them, I'm not convinced it justifies removing
> > the "official" status of the team and make it a workgroup.
> >
> > I privately received information that explains why the PTL was not on
> > top of things during election weeks. With ~60 teams around there will
> > always be one or two that miss and that we must check on. It /always/ is
> > symptomatic of /some/ disconnect. But here I'm not sure it passes the
> > bar of "non-alignment with the community" that would make the Security
> > team unfit to be an official OpenStack team...
> >
> I agree, and in times like this, it's best to use common sense rather
> than trying to have a rule to fit everything into. In this case, Rob
> and the security team have put forth an explanation of what happened,
> I fail to see how removing them after this does anything other than
> foster bad will. I would vote to keep the security team around at this
> point.
>
>
I feel bad quoting policy here... but we do have prior art for this... If
we look at resolution, "2014-11-28 Process for Leaderless Programs"[0], we
have policy for *exactly* this situation.. which should probably have been
the first action rather than considering a new resolution.

For reference:

   1. Programs without a minimum of one eligible candidate are identified
   to the Technical Committee by the Election Officials, as soon as possible
   after the nomination period has expired.
   2. The Technical Committee can appoint a leader to any programs in this
   situation, by mutual agreement of the Technical Committee and the proposed
   appointee.
   3. The appointed leader has all the same obligations and
   responsibilities as a self-nominated elected Program Technical Lead.

[0]
http://governance.openstack.org/resolutions/20141128-elections-process-for-leaderless-programs.html


--
Kind Regards,
Dave Walker
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [security] [salt] Removal of Security and OpenStackSalt project teams from the Big Tent

[Dave Walker]

On 21 September 2016 at 19:20, Chivers, Doug <doug.chiv...@hpe.com> wrote:

> My concern is with the original wording “The suggested way forward there
> would be to remove the "Security project team"”.
>
> This seems like a move to instantly reduce investment in OpenStack
> security, because the majority of members of the Security Project are
> corporately funded, which will be significantly impacted by the removal of
> the security project. I have no knowledge over the difference between a
> working group and a project, like everyone else on the project we are
> simply here to contribute to OpenStack security, drive innovation in
> security, deliver documentation like OSSNs, etc, rather than get involved
> in the politics of OpenStack.
>
> In response to the various questions of why no-one from our project
> noticed that we didn’t have a nomination for the PTL, we assumed that was
> taken care of. Realistically maybe two or three people on the security
> project have the availability to be PTL, one being our current PTL, for all
> the rest of us its simply not a concern until we need to vote.
>
> On a personal note, reading –dev is unfortunately a lower priority than
> designing architectures, responding to customers and sales teams, closing
> tickets, writing decks and on the afternoon or so I can spend each week,
> working on my upstream projects (this week it was:
> https://review.openstack.org/#/c/357978/5 - thanks to the Barbican team
> for all their work). Possibly this is wrong, but I didn’t sign up as a
> contributor to spend all my spare time reading mailing lists.
>
>


Honestly, I can only echo this.  I've been around the OSSP(G) since 2013,
but only really been active in the last 18 months or so.  It's been pretty
clear that when Security moved from a Group to a Project, investment
towards security grew dramatically.

The meetings are well run with real objectives achieved with members
focused on constant outreach to other projects.  For reference, the email
that started this thread was picked up and discussed by some members of the
OSSP within *minutes* of it being sent... and those people were pretty
outraged.

I'm sure it wasn't intended, but the original email could be read as quite
insulting.. "That points to a real disconnect between those teams and the
rest of the community".  I think this is an unfair statement based on
minimal observation of a point of order.

The OSSP spends a significant amount of its time on outreach, which is the
whole underlying principle of the project.  This can be seen with efforts
such as bandit gate coverage, Threat Analysis and OSSN's.

Further, reducing the summit timetable for Security and "have Security be
just a workgroup".. really sends the wrong message about Security being a
first class citizen in OpenStack.

OSSP ticks all the 4 opens, and stating that "The leadership is chosen by
the contributors to the project".. it is convention that a nomination email
is sent to -dev, but that shouldn't be assumed that the contributors have
not considered their leader.

I think people working on the OSSP assumed it would be Rob again, and were
happy with this.  It isn't because of lack of community engagement or
interest IMO.

So.. other than someone failing to nominate for PTL in the time-frame, what
else justifies the statement of "points[ing] to a real disconnect between
those teams and the rest of the community".. or shows that OSSG no longer
meets the 4 opens?

--
Kind Regards,
Dave Walker
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vote][kolla] deprecation for fedora distro support

[Dave Walker]

On 19 September 2016 at 18:40, Jeffrey Zhang <zhang.lei@gmail.com>
wrote:

> Kolla core reviewer team,
>
> Kolla supports multiple Linux distros now, including
>
> * Ubuntu
> * CentOS
> * RHEL
> * Fedora
> * Debian
> * OracleLinux
>
> But only Ubuntu, CentOS, and OracleLinux are widely used and we have
> robust gate to ensure the quality.
>
> For fedora, Kolla hasn't any test for it and nobody reports any bug
> about it( i.e. nobody use fedora as base distro image). We (kolla
> team) also do not have enough resources to support so many Linux
> distros. I prefer to deprecate fedora support now.  This is talked in
> past but inconclusive[0].
>
> Please vote:
>
> 1. Kolla needs support fedora( if so, we need some guys to set up the
> gate and fix all the issues ASAP in O cycle)
> 2. Kolla should deprecate fedora support
>
> [0] http://lists.openstack.org/pipermail/openstack-dev/2016-
> June/098526.html
>
>
>
+1

Honestly, only CentOS and Ubuntu are seriously implemented.. I'd go further
and rip out the rest unless there is a strong effort from people committed
to getting parity.

--
Kind Regards,
Dave Walker
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vote][kolla] Splitting out Ansible into a separate deliverable

[Dave Walker]

Option C

Thanks

On 15 September 2016 at 12:10, Ryan Hallisey  wrote:

> Option c.
>
> - Ryan
>
> > On Sep 15, 2016, at 4:33 AM, Paul Bourke  wrote:
> >
> > c)   Split the repository shortly after tagging 3.0.0 – creating a
> kolla-ansible deliverable for Ocata.
> >
> >> On 15/09/16 07:12, Steven Dake (stdake) wrote:
> >> Core Reviewers:
> >>
> >>
> >>
> >> The facts:
> >>
> >> We have roughly 250 bugs in rc2.  Of those, I suspect over half can just
> >> be closed out as dupes, fixed, wontfix, or the like.
> >>
> >> The core reviewer team has had various discussions around splitting the
> >> repository at various times but has not come to a concrete conclusion
> >> via a vote.
> >>
> >> Once RC1 is tagged, the stable/newton branch will be created
> automatically.
> >>
> >> All rc2 bug fixes will require bug IDs and backports to stable/newton to
> >> enable the ability to manage the release of rc2 and 3.0.0.
> >>
> >> There is an expectation for core reviewers to do the work of backporting
> >> to stable/newton – only our backports team typically does this work –
> >> however during release we really need everyone’s participation.
> >>
> >>
> >>
> >> My understanding of general consensus beliefs:
> >>
> >> We believe splitting out the Ansible implementation into a separate
> >> repository will produce a better outcome for both Kolla-Ansible and
> >> Kolla-Kubernetes
> >>
> >> We have been unable to achieve consensus on the right timing for a repo
> >> split in the past but generally believe the timing is right at some
> >> point between rc1 and Summit or shortly thereafter, if we are to do the
> >> repo split during Newton or very early Ocata.)
> >>
> >>
> >>
> >> This vote is a multiple choice (one choice please) vote.  Feel free to
> >> discuss before making a decision.
> >>
> >>
> >>
> >> Please vote:
> >>
> >> a)   Do not split the repository between rc1 and Summit or shortly
> >> thereafter at all, keeping the Ansible implementation intact in Ocata
> >>
> >> b)   Split the repository shortly after tagging RC1 – creating of a
> >> kolla-ansible deliverable for Ocata.
> >>
> >> c)   Split the repository shortly after tagging 3.0.0 – creating a
> >> kolla-ansible deliverable for Ocata.
> >>
> >>
> >>
> >> Voting is open for 7 days until September 21^st , 2016. Please do not
> >> abstain on this critical vote.  Remember, no veto vote is available in
> >> roll-call votes.  If a majority can’t be reached on any one choice, but
> >> there is a majority around B & C, (which are the same idea, but
> >> different timing) a second vote will be triggered around when to split
> >> the repository.  The implication there is if you vote for b or c, your
> >> voting for a repository split.  If you vote for A you are voting for no
> >> repository split.  I hate to overload voting in this way.  It is only an
> >> optimization to speed things up as execution may need to happen now, or
> >> can be pushed out a month, or may not be needed at this time.
> >>
> >>
> >>
> >> Regards
> >>
> >> -steve
> >>
> >>
> >>
> >> 
> __
> >> OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:
> unsubscribe
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> > 
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:
> unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [requirements][FFE] block graphviz 0.5.0

[Dave Walker]

On 9 September 2016 at 08:42, Tony Breeds <t...@bakeyournoodle.com> wrote:

> On Fri, Sep 09, 2016 at 11:58:29AM +0530, Swapnil Kulkarni (coolsvap)
> wrote:
> > Currently kolla gate jobs are failing due to issues in graphviz 0.5.0.
> [1]
> >
> > We need to unblock the gates by blocking graphviz===0.5.0.
>
> It looks to me that the only package directly affected by this FFE is kolla
>
> Package  : graphviz [graphviz>=0.4.0] (used by 1 projects)
> Also affects : 1 projects
> openstack/kolla   [release:cycle-trailing]
>
> http://codesearch.openstack.org/?q=graphviz=nope=.
> *requirements.txt=
>
> Has some impact on  octavia (doc-requirements) and rpm-packaging (internal
> global-requirements.txt)
>
> So Looks good to me
>
> Yours Tony.
>
>
Looking at the Kolla CI log[0], it doesn't look like Kolla is either at
fault, or can adapt to the issue in Graphviz.

Graphviz released 0.5.0 last night, and it seems to have introduced a
regression.  I've raised this as an issue[1], and made a pull request for a
fix for this.

The change I've proposed to them does allow Kolla tests to pass.  Providing
this, or similar lands - then it is entirely appropriate to blacklist
graphviz===0.5.0 and pick up the fix in the next release of Graphviz.

Thanks

[0]
http://logs.openstack.org/16/367416/1/gate/gate-kolla-python34/64dd4ae/console.html#_2016-09-09_04_42_27_380320
[1] https://github.com/xflr6/graphviz/issues/24

--
Kind Regards,
Dave Walker
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vote][kolla] Core Nomination for Christian Berendt (berendt on irc)

[Dave Walker]

+1

Great stuff Christian

On 8 September 2016 at 03:59, Steven Dake (stdake)  wrote:

> Kolla core reviewer team,
>
>
>
> It is my pleasure to nominate Christian Berendt for the Kolla core team.
>
>
>
> Christian’s output over the last cycle has been fantastic – cracking the
> top 10 reviewer list for the full cycle.  His 30 day stats [1] place him in
> solid 7th position, and considering the size of the core review team we
> have, this is a great accomplishment.  Christian also has solid 60 day
> review stats [2] place him in solid 7th position.  But more importantly
> his reviews are of high quality.  He has great IRC participation as well as
> having implemented all kinds of bug fixes all over the code base.  He
> clearly understands Kolla by the quality of his reviews.
>
>
>
> Consider this nomination a +1 vote from me.
>
>
>
> A +1 vote indicates you are in favor of Christian as a candidate, a 0 vote
> indicates an abstain, and a -1 is a veto.  Voting is open for 7 days until
> September 15th, or a unanimous response is reached or a veto vote
> occurs.  If a unanimous response is reached or a veto occurs, voting will
> close early.
>
>
>
> Regards,
>
> -steve
>
>
>
> [1] http://stackalytics.com/report/contribution/kolla/30
>
> [2] http://stackalytics.com/report/contribution/kolla/60
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] Requirement for bug when reno is present

[Dave Walker]

On 30 August 2016 at 11:42, Paul Bourke <paul.bou...@oracle.com> wrote:

> Kolla,
>
> Do people feel we still want to require a bug-id in the commit message for
> features, when reno notes are present? My understanding is that till now
> we've required people to add bugs for non trivial features in order to
> track them as part of releases. Does/should reno supersede this?
>
> -Paul
>

I'm guess you raised this because my recent comment on a change you did...
but actually, I agree with you.  I don't think it is a good process, but
standardisation is key.

The issue comes around because Kolla wanted to use bugs to track potential
backports to stable/*.  However, I think this is generally overrated and
the Change-ID is suitable for this purpose.

I really hate raising bugs "just because", when realistically many of them
are not bugs and contain one-line style "Add this $feature" bug
description.  It just burns through Launchpad bug numbers, and will likely
never be looked at again.

--
Kind Regards,
Dave Walker
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova][kolla] Live migration, VNC / SPICE address?

[Dave Walker]

Hi,

In Kolla, we are having an issue with Nova's VNC / SPICE address and live
migration.  Currently, we are declaring the IP address for vncserver_listen
on each node (via ansible).  However, when a live migration is performed,
it fails due to this address not being available.

The hack is to switch the vncserver_listen to be 0.0.0.0, but this is
horribly insecure and breaks the network isolation that kolla supports.

Looking at the relevant code, this looks like it should be functional
via VIR_DOMAIN_XML_MIGRATABLE, but it doesn't seem to be working.

Could someone from Nova help us determine the cause?  We are tracking this
as bug 1596724

https://github.com/openstack/nova/blob/04cef3b5d03be3db7efab6896de867fc2cbbd03a/nova/virt/libvirt/driver.py#L5393

https://github.com/openstack/nova/blob/04cef3b5d03be3db7efab6896de867fc2cbbd03a/nova/virt/libvirt/driver.py#L5754

Thanks

--
Kind Regards,
Dave Walker
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tempest][swift][radosgw] Can we please merge the fix for the RFC 7230 violation issue?

[Dave Walker]

On 8 August 2016 at 17:50, Jay Pipes <jaypi...@gmail.com> wrote:

> Tempest devs,
>
> Let me please draw your attention to a LP bug that may not seem
> particularly high priority, but I believe could be resolved easily with a
> patch already proposed.
>
> LP bug 1536251 [1] accurately states that Tempest is actively verifying
> that an OpenStack API call violates RFC 7230.
>
> When a 204 No Content is received, the Content-Length header MUST NOT be
> present.
>
> However, Swift returns a Content-Length header and also an HTTP response
> code of 204 for a request to list containers of a new user (that has no
> containers).
>
> Tempest has been validating this behaviour even though it is a violation
> of RFC 7230:
>
> https://github.com/openstack/tempest/blob/master/tempest/api
> /object_storage/test_account_services.py#L81
>
> RadosGW provides a proxy API that attempts to match the OpenStack Object
> Storage API, backed by Ceph object storage. In order for RadosGW to pass
> RefStack's burden of compatibility, it must pass the Tempest OpenStack
> Object Storage API tests. It currently cannot do so because RadosGW does
> not violate RFC 7230.
>
> The RadosGW developer community does not wish to argue about whether or
> not to make Swift's API comply with RFC 7230. At the same time, they do not
> want to add a configuration option to RadosGW to force the proxy service to
> violate RFC 7230 just to satisfy the RefStack/Tempest API tests.
>
> Therefore, Radoslaw (cc'd) has proposed a patch to Tempest that would
> allow RadosGW's proxy API to meet the RefStack compatibility tests while
> also not violating RFC 7230 and not requiring any change of Swift:
>
> https://review.openstack.org/#/c/272062
>
> I ask Tempest devs to re-review the above patch and consider merging it
> for the sake of collaboration between the OpenStack and Ceph developer
> communities.
>
> Thanks very much!
> -jay
>
> [1] https://bugs.launchpad.net/tempest/+bug/1536251



These sorts of issues aren't just theoretical and following policy for the
sake of it..  Glance had 3 x areas where 200 responses that also included a
Location header (against RFC-2616 §14.30) which totally broke glance when
deployed behind apache+fcgid+flup (the presence of Location, that stack
rewrote it to a 302).

Fun bug btw:
https://bugs.launchpad.net/glance/+bug/1299095

--
Kind Regards,
Dave Walker
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] Binary containers with pip install?

[Dave Walker]

Hi,

The definition of 'source' is quite loose, the way it is currently followed
seems to be OpenStack source - which is not necessarily the project the
Dockerfile is creating.  Ie, mariadb is always installed via binary (and I
don't think this should be changed).

Kolla will (normally) always be able to track other openstack projects
faster than distros can, via source install.  This means that either we
hold the specific project on ice until it is completely ready (likely very
late into the Kolla cycle), or do best effort.

So there are 4 scenarios i can see:
 - Some Dockerfiles output "Sorry, binary isn't supported at this stage"
(or similar).  This means that the deployer has to switch their entire
deployment to type source, meaning that a single project has caused the
user to switch away from binary.  This isn't ideal.
 - Have a single if-logic for the flavor distro (rather than flavor AND
source/binary), so the single logic is built for both scenarios (binary and
source).  The deployer can have a mostly binary build, but have a couple of
hidden source builds (which will be called binary), that can be fixed
if/when support is added.
 - Add support to have per project build type, but this is likely a testing
scenario that cannot be reasonably assured.
 - Allow source installs in binary containers, but track it as a TODO bug
"Please package foo" (Launchpad has great support for linking to other bug
trackers).  Then once this bug is closed, proper binary support can be
resolved.  This has the benefit of 'best-effort' towards binary, with a
clear intent to move across.  It also allows more testing of the binary
parts that are present, with just the source parts as required. (this is my
favourite)

--
Kind Regards,
Dave Walker

On 26 July 2016 at 08:37, Swapnil Kulkarni (coolsvap) <m...@coolsvap.net>
wrote:

> Dear Kollagues,
>
> There has been a detailed conversation between me and Prithiv related
> to [1] and allowing pip install in binary containers.
>
> Till now we have simply skipped the binary containers till we have
> official packages in respective distro repositories. I strongly feel
> we should be following that.
>
> If someone wishes to have a functionality without a package in the
> base distro, there is always an option to build source containers and
> use it with Kolla.
>
> Thoughs welcome?
>
> [1] https://review.openstack.org/#/c/344930/
>
> --
> Best Regards,
> Swapnil Kulkarni
> irc : coolsvap
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla][vote] Applying for stable-follows tag

[Dave Walker]

Hi,

I'm not currently kolla-core, but I am a member of stable-maint-core (cross
project).  I've been pretty involved with Kolla this cycle, some 7 landed
commits and 34 patchsets pushed up.. So I have a good understanding of both
sides of the camp.  I'd be happy to throw my head into the ring for
kolla-stable-maint.

--
Kind Regards,
Dave Walker

On 22 July 2016 at 10:47, Kwasniewska, Alicja <alicja.kwasniew...@intel.com>
wrote:

> +1 too
>
>
>
> *From:* Mauricio Lima [mailto:mauricioli...@gmail.com]
> *Sent:* Tuesday, July 19, 2016 5:29 PM
>
> *To:* OpenStack Development Mailing List (not for usage questions) <
> openstack-dev@lists.openstack.org>
> *Subject:* Re: [openstack-dev] [kolla][vote] Applying for stable-follows
> tag
>
>
>
> +1
>
>
>
> 2016-07-19 12:23 GMT-03:00 Vikram Hosakote (vhosakot) <vhosa...@cisco.com
> >:
>
> +1 sure.
>
>
>
> Regards,
>
> Vikram Hosakote
>
> IRC:  vhosakot
>
>
>
> *From: *Michał Jastrzębski <inc...@gmail.com>
> *Reply-To: *"OpenStack Development Mailing List (not for usage
> questions)" <openstack-dev@lists.openstack.org>
> *Date: *Tuesday, July 19, 2016 at 9:20 AM
> *To: *"OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
>
> *Subject: *Re: [openstack-dev] [kolla][vote] Applying for stable-follows
> tag
>
>
>
> +1 ofc
>
>
>
> On 19 July 2016 at 06:02, Ryan Hallisey <rhall...@redhat.com> wrote:
>
> +1
>
>
>
> -Ryan
>
>
>
> - Original Message -
>
> From: "Jeffrey Zhang" <zhang.lei@gmail.com>
>
> To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
>
> Sent: Monday, July 18, 2016 9:16:09 PM
>
> Subject: Re: [openstack-dev] [kolla][vote] Applying for stable-follows tag
>
>
>
> +1 to apply
>
> I'd like to be the volunteer.
>
>
>
> On Mon, Jul 18, 2016 at 9:04 PM, Swapnil Kulkarni (coolsvap)
>
> <m...@coolsvap.net> wrote:
>
> On Mon, Jul 18, 2016 at 6:23 PM, Paul Bourke <paul.bou...@oracle.com>
> wrote:
>
> Hi Steve,
>
>
>
> +1 to applying. I'll volunteer for the backport team also.
>
>
>
> -Paul
>
>
>
>
>
> On 18/07/16 13:07, Steven Dake (stdake) wrote:
>
>
>
> Hey Koalians,
>
>
>
> I'd like us to consider applying  for the stable follows policy tag.
>
>Full details are here:
>
>
>
>
>
>
> http://github.com/openstack/governance/blob/master/reference/tags/stable_follows-policy.rst
>
>
>
> Because of the magic work we did to make liberty functional, it is
>
> possible that we may not be able to apply for this tag until Liberty
>
> falls into EOL.  Still I personally believe intent matters most, and our
>
> intent has always been for these to be stable low-rate-of-change
>
> no-feature-backport branches.  There are some exceptions I think we
>
> would fit under for the Liberty case, so I think it is worth a shot.
>
>
>
> I'd need 2-4 people to commit to joining the stable backport team for
>
> Kolla reviews specifically.  These folks would be the only folks that
>
> could ACK patches in the stable branch maintenance queue.  Anyone could
>
> continue to submit backport patches as they desire.
>
>
>
> I'll leave voting open for 1 week or until there I a majority (6 core
>
> reviewers) or until there is a unanimous vote.  If there is not, then we
>
> won't apply.  The deadline for this vote is July 25th.
>
>
>
> Thanks!
>
> -steve
>
>
>
>
>
>
>
>
>
> __
>
> OpenStack Development Mailing List (not for usage questions)
>
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
>
> __
>
> OpenStack Development Mailing List (not for usage questions)
>
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> +1 to apply for stable follows policy.
>
> I would like to volunteer for the backport team.
>
>
>
> __
>
> OpenStack Development Mailing List (not for usage questions)
>
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>
> http://list

Re: [openstack-dev] [kolla] Monitoring tooling

[Dave Walker]

So, this is one of the areas i'm currently on with the Sensu work.  I've
been experimenting with a privileged container, which is very similar to
our current kolla_toolbox container.

The agent certainly needs to be in it's own container, with access to be
able to run commands in other namespaces / containers.

On 25 July 2016 at 11:58, Mathias Ewald  wrote:

> Excellent point ... I just checked what happens when running telegraf in a
> container: You'll get paths like
>
> /etc/hostname
> /etc/hosts
> /var/log/kolla
>
> and others as available file systems. I guess it makes no sense at all
> then to containerize monitoring agents ... Sensu is going to have the same
> problems.
>
> Any suggestions?
>
>
> 2016-07-25 9:39 GMT+02:00 Jeffrey Zhang :
>
>> I am open to the choice of tools. But i am worried on thing: how to
>> get all the host disk usage when containerized the monitor tool?
>>
>>
>>
>> On Mon, Jul 25, 2016 at 2:12 PM, Mathias Ewald  wrote:
>> > Understood.
>> >
>> > 2016-07-25 7:34 GMT+02:00 Matthias Runge :
>> >>
>> >> On 25/07/16 06:38, Mathias Ewald wrote:
>> >> > Hi, correct me if wrong please: Isn't gnocchi more targeting cloud
>> >> > tenants rather than cloud ops? If that's the case I wont find info
>> like
>> >> > "controller cpu usage" or "hypervisor memory usage".
>> >> >
>> >> > Cheers
>> >> > Mathias
>> >> >
>> >> Uhm, in this scenario, gnocchi just used as time-series database. It's
>> >> up to you, what you feed in. collectd collects whatever you want and
>> put
>> >> it into your database.
>> >>
>> >> Best,
>> >> Matthias
>> >>
>> >> --
>> >> Matthias Runge 
>> >>
>> >> Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
>> >> Commercial register: Amtsgericht Muenchen, HRB 153243,
>> >> Managing Directors: Charles Cachera, Michael Cunningham,
>> >> Michael O'Neill, Eric Shander
>> >>
>> >>
>> __
>> >> OpenStack Development Mailing List (not for usage questions)
>> >> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>> >
>> >
>> >
>> > --
>> > Mobil: +49 176 10567592
>> > E-Mail: mew...@evoila.de
>> >
>> > evoila GmbH
>> > Wilhelm-Theodor-Römheld-Str. 34
>> > 55130 Mainz
>> > Germany
>> >
>> > Geschäftsführer: Johannes Hiemer
>> >
>> > Amtsgericht Mainz HRB 42719
>> >
>> > Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte
>> > Informationen. Wenn Sie nicht der richtige Adressat sind oder diese
>> E-Mail
>> > irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und
>> > vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte
>> > Weitergabe dieser Mail ist nicht gestattet.
>> >
>> > This e-mail may contain confidential and/or privileged information. If
>> You
>> > are not the intended recipient (or have received this e-mail in error)
>> > please notify the sender immediately and destroy this e-mail. Any
>> > unauthorised copying, disclosure or distribution of the material in this
>> > e-mail is strictly forbidden.
>> >
>> >
>> __
>> > OpenStack Development Mailing List (not for usage questions)
>> > Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>>
>>
>>
>> --
>> Regards,
>> Jeffrey Zhang
>> Blog: http://xcodest.me
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
>
> --
> Mobil: +49 176 10567592
> E-Mail: mew...@evoila.de
>
> evoila GmbH
> Wilhelm-Theodor-Römheld-Str. 34
> 55130 Mainz
> Germany
>
> Geschäftsführer: Johannes Hiemer
>
> Amtsgericht Mainz HRB 42719
>
> Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte
> Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail
> irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und
> vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte
> Weitergabe dieser Mail ist nicht gestattet.
>
> This e-mail may contain confidential and/or privileged information. If You
> are not the intended recipient (or have received this e-mail in error)
> please notify the sender immediately and destroy this e-mail. Any
> unauthorised copying, disclosure or distribution of the material in this
> e-mail is strictly forbidden.
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: 

Re: [openstack-dev] [kolla] Monitoring tooling

[Dave Walker]

Thanks Mathias,

I'm not tied to Sensu.. anything can really fill that gap in my mind.
You've done a good job at outlining the steps involved.  I created a
blueprint with the steps I had in mind[0]

For this cycle, I wanted to keep it simple so it was easily achievable.  I
only planned to have some basic up/down for each node and throw
the performance data on the floor.

I wanted to include the option to include local configs, as json blobs.  Some
of the things I was thinking as local config:
  - daily checkouts, can instances be built with networking
  - remaining resources count (ie, does each subnet have X remaining ip
addresses available)
  - Is Ceph healthy?

So, these things aren't really performance over time interesting.. which
means the intention does differ.  However, I do agree that both stacks
could achieve both objectives.

I've essentially got much of this working locally, but would require about
a day of cleaning up for submission... but if your work can achieve the
objectives above, i'm happy to discontinue... or help make your stack
pluggable.

[0] https://blueprints.launchpad.net/kolla/+spec/sensu

--
Kind Regards,
Dave Walker

On 24 July 2016 at 11:56, Mathias Ewald <mew...@evoila.de> wrote:

> Monitoring is a difficult topic as the number of options regarding the
> toolset and mechanisms are very high. We had some chats about it in IRC
> that discovered even more options than I thought existed :D I believe
> Dave's view on Sensu is generally correct in that Sensu is more directed to
> monitoring in the form of "if X running/working" but of course has the
> ability to transport metrics, too, but lacks the good dashboarding
> capabilities for performance data. One set up I could images is
>
> 1. Sensu Client to collect checks and metrics
> 2. RabbitMQ for transport
> 3. Sensu Server to receive, evaluate, alarm and write metrics to InfluxDB
> 4. Uchiwa as a Dashboard to Sensu
> 5. InfluxDB to store metrics
> 6. Grafana to dashboard metrics
>
> So Sensu could be used as a replacement for (or in addition to) a metrics
> collection daemon like Collectd or what I decided to use: Telegraf. For my
> implementation, this means I will add a parameter to make Telegraf
> optional. This way, someone else may implement the rest of the stack and
> the user can decide which one to use.
>
> What do you think?
>
> Mathias
>
>
>
> 2016-07-23 21:51 GMT+02:00 Stephen Hindle <shin...@llnw.com>:
>
>> My understanding was Sensu could produce metrics ?
>> And Kapacitor can do alerting for the TICK stack stuff mewald is doing...
>> I really don't see them as that different ?
>>
>>
>> On Fri, Jul 22, 2016 at 5:19 PM, Dave Walker <em...@daviey.com> wrote:
>> > Yes, this is my thought.
>> >
>> > The scope of the Sensu work is: "Is this thing working?" (with the
>> reference
>> > being up/down)
>> > But the scope of the Grafana and friends is, "How hard is this working?"
>> > (but no alerting)
>> >
>> > They are certainly complementary However, Sensu can throw data at a
>> > Grafana stack (aiui).. but I fear that is too much to achieve this
>> cycle.
>> >
>> > --
>> > Kind Regards,
>> > Dave Walker
>> >
>> > On 23 July 2016 at 00:11, Fox, Kevin M <kevin@pnnl.gov> wrote:
>> >>
>> >> I think those are two different, complementary things.
>> >>
>> >> One's metrics and the other is monitoring. You probably want both at
>> the
>> >> same time.
>> >>
>> >> Thanks,
>> >> Kevin
>> >> 
>> >> From: Steven Dake (stdake) [std...@cisco.com]
>> >> Sent: Friday, July 22, 2016 3:52 PM
>> >> To: OpenStack Development Mailing List (not for usage questions)
>> >> Subject: Re: [openstack-dev] [kolla] Monitoring tooling
>> >>
>> >> Thanks for pointing that out.  Brain out to lunch today it appears :(
>> >>
>> >> I think choices are a good thing even though they increase our
>> >> implementation footprint.  Anyone opposed to implementing both with
>> >> something in globals.yml like
>> >> monitoring: grafana or
>> >> monitoring: sensu
>> >>
>> >> Comments questions or concerns welcome.
>> >>
>> >> Regards
>> >> -steve
>> >>
>> >> On 7/22/16, 3:42 PM, "Stephen Hindle" <shin...@llnw.com> wrote:
>> >>
>> >> >Don't forget mewalds implementation as well - we now have 2 monitoring
>> >> &g

Re: [openstack-dev] [kolla] Monitoring tooling

[Dave Walker]

Yes, this is my thought.

The scope of the Sensu work is: "Is this thing working?" (with the
reference being up/down)
But the scope of the Grafana and friends is, "How hard is this working?"
(but no alerting)

They are certainly complementary However, Sensu can throw data at a
Grafana stack (aiui).. but I fear that is too much to achieve this cycle.

--
Kind Regards,
Dave Walker

On 23 July 2016 at 00:11, Fox, Kevin M <kevin@pnnl.gov> wrote:

> I think those are two different, complementary things.
>
> One's metrics and the other is monitoring. You probably want both at the
> same time.
>
> Thanks,
> Kevin
> 
> From: Steven Dake (stdake) [std...@cisco.com]
> Sent: Friday, July 22, 2016 3:52 PM
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: Re: [openstack-dev] [kolla] Monitoring tooling
>
> Thanks for pointing that out.  Brain out to lunch today it appears :(
>
> I think choices are a good thing even though they increase our
> implementation footprint.  Anyone opposed to implementing both with
> something in globals.yml like
> monitoring: grafana or
> monitoring: sensu
>
> Comments questions or concerns welcome.
>
> Regards
> -steve
>
> On 7/22/16, 3:42 PM, "Stephen Hindle" <shin...@llnw.com> wrote:
>
> >Don't forget mewalds implementation as well - we now have 2 monitoring
> >options for kolla :-)
> >
> >On Fri, Jul 22, 2016 at 3:15 PM, Steven Dake (stdake) <std...@cisco.com>
> >wrote:
> >> Hi folks,
> >>
> >> At the midcycle we decided to push off implementing Monitoring until
> >>post
> >> Newton.  The rationale for this decision was that the core review team
> >>has
> >> enough on their plates and nobody was super keen to implement any
> >>monitoring
> >> solution given our other priorities.
> >>
> >> Like all good things, communities produce new folks that want to do new
> >> things, and Sensu was proposed as Kolla's monitoring solution (atleast
> >>the
> >> first one).  A developer that has done some good work has shown up to
> >>do the
> >> job as well :)  I have heard good things about Sensu, minus the fact
> >>that it
> >> is implemented in Ruby and I fear it may end up causing our gate a lot
> >>of
> >> hassle.
> >>
> >> https://review.openstack.org/#/c/341861/
> >>
> >>
> >> Anyway I think we can work through the gate problem.
> >>
> >> Does anyone have any better suggestion?  I'd like to unblock Dave's work
> >> which is blocked on a ­2 pending a complete discussion of our monitoring
> >> solution.  Note we may end up implementing more than one down the road ­
> >> Sensu is just where the original interest was.
> >>
> >> Please provide feedback, even if you don't have a preference, whether
> >>your a
> >> core reviewer or not.
> >>
> >> My take is we can merge this work in non-prioirty order, and if it
> >>makes the
> >> end of the cycle fantastic ­ if not, we can release it in Ocatta.
> >>
> >> Regards
> >> -steve
> >>
> >>
> >>
> >>_
> >>_
> >> OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe:
> >>openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >
> >
> >
> >--
> >Stephen Hindle - Senior Systems Engineer
> >480.807.8189 480.807.8189
> >www.limelight.com Delivering Faster Better
> >
> >Join the conversation
> >
> >at Limelight Connect
> >
> >--
> >The information in this message may be confidential.  It is intended
> >solely
> >for
> >the addressee(s).  If you are not the intended recipient, any disclosure,
> >copying or distribution of the message, or any action or omission taken
> >by
> >you
> >in reliance on it, is prohibited and may be unlawful.  Please immediately
> >contact the sender if you have received this message in error.
> >
> >
> >__
> >OpenStack Development Mailing List (not for usage questions)
> >Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> 

Re: [openstack-dev] [kolla] Please start getting in the habit of breaking up containers from ansible changes

[Dave Walker]

On 22 July 2016 at 21:35, Steven Dake (stdake) <std...@cisco.com> wrote:
>
> Hey folks,
>
> I know it doesn't make a lot of sense to break up containers from ansible
changes to people outside the core review team, but for anything with
backport potential, please do so.  We are considering in Occata splitting
the kolla repo into two (kolla = containers & build, kolla-ansible =
playbooks).  I think the timing is right after we branch Kolla Newton, but
I don't want to crater our backport process in the process.  By keeping the
changes separate we can still have a tidy backport experience.
>
> Even for small changes - 2–3 liner, please break them up using
Partial-Bug.
>
> Core reviewers please start enforcing this.
>
> TIA!
> -steve
>

Hi Steve,

Why would this cause a problem in current Master?  As I understand it, you
want to make sure that changes that touch both Dockerfiles and Playbooks
are in isolated commits so they can be backported.  However, this surely
won't be relevant until Newton is cut and Occata is opened, as Newton is
remaining as a single tree.  So, the splitting of commits is only relevant
in Occata+1, where splitting will already be enforced - by the splitting of
the trees in Occata?  I say O+1, as split trees will only start in O.. So
for Newton, O commits will still backport cleanly... as they will be
separated by the nature of the tree split.

Or... Have I horribly misunderstood your push?

With Occata, will kolla and kolla-ansible have common ancestry? As in, will
they both be based on current Master with irrelevant files removed from
each tree?

--
Kind Regards,
Dave Walker
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [kolla][horizon] Out of branch horizon plugins?

[Dave Walker]

Hi,

Whilst writing a Kolla plugin, I noticed some issues with the way Horizon
is configured in Kolla.

Horizon is increasingly embracing a plugin architecture with UI's and
Dashboards being maintained outside of Horizon's tree.

These can be found with the type:horizon-plugin tag in openstack/governance
[0], with 16 projects at current.

This isn't really addressed in Kolla, and is a little awkward to integrate
as the horizon docker image is pure horizon.

Some projects have a tools/register_plugin.sh which performs the grunt
work, where as others require a workflow similar to:

cp /path/to/projects/new/panel openstack_dashboard/local/enabled/
cp /path/to/local/defualt/settings
openstack_dashboard/local/local_settings.d/
cp /path/to/*policy.json openstack_dashboard/conf/
# compress if environment wants it
./manage.py collectstatic
./manage.py compress

(Separately, it would be nice if this was standardised.. but not the topic
of this thread)

It would seem logical to pack all of these into the horizon docker image,
and add a symlink into dashboard/local/enabled via ansible, policy.json and
default settings with some conditionals if enabled_$service... but this
will make the horizon docker image larger and more complicated.

What are your thoughts?

[0]
http://git.openstack.org/cgit/openstack/governance/tree/reference/projects.yaml

--
Kind Regards,
Dave Walker
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [daisycloud-core] Kolla Mitaka requirements supported by CentOS

[Dave Walker]

On 2 July 2016 at 19:42, jason <huzhiji...@gmail.com> wrote:

>
>
> As above table shows, only two (graphviz and Jinja2) are not supported
> by centos currently. As those not supported packages are definitly not
> used by OpenStack as well as Daisy. So basicaly we can use pip to
> install them after installing other packages by yum. But note that
> Jinja2 2.7.2 will be installed as dependency while yum install
> ansible, so we need to using pip to install jinja2 2.8 after that to
> overide the old one. Also note that we must make sure pip is ONLY used
> for installing those two not supported packages.
>
> 

Whilst this appears to be accurate, it would probably not be an appropriate
change for stable/mitaka.  However, it is probably worth checking the
current development focus Master, which will become Newton
(kolla/docker/openstack-base/Dockerfile.j2) and seeing if this is still an
issue.  A bunch of Centos Binary improvements were made this current cycle
to make more use of yum packages[0].

[0]
https://github.com/openstack/kolla/commit/a8f3da204e6d6ae42b30c166d436d74064394a1b

--
Kind Regards,
Dave Walker
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] I'm going to expire open bug reports older than 18 months.

[Dave Walker]

On 24 May 2016 at 18:05, Doug Hellmann <d...@doughellmann.com> wrote:

> Excerpts from Markus Zoeller's message of 2016-05-24 11:00:35 +0200:
> > On 24.05.2016 09:34, Duncan Thomas wrote:
> > > Cinder bugs list was far more manageable once this had been done.
> > >
> > > It is worth sharing the tool for this? I realise it's fairly trivial to
> > > write one, but some standardisation on the comment format etc seems
> > > valuable, particularly for Q/A folks who work between different
> projects.
> >
> > A first draft (without the actual expiring) is at [1]. I'm going to
> > finish it this week. If there is a place in an OpenStack repo, just give
> > me a pointer and I'll push a change.
> >
> > > On 23 May 2016 at 14:02, Markus Zoeller <mzoel...@linux.vnet.ibm.com>
> wrote:
> > >
> > >> TL;DR: Automatic closing of 185 bug reports which are older than 18
> > >> months in the week R-13. Skipping specific bug reports is possible. A
> > >> bug report comment explains the reasons.
> > >> [...]
> >
> > References:
> > [1]
> >
> https://github.com/markuszoeller/openstack/blob/master/scripts/launchpad/expire_old_bug_reports.py
> >
>
> Feel free to submit that to the openstack-infra/release-tools repo. We
> have some other tools in that repo for managing launchpad bugs.
>
> Doug


Rather that blanket expiring old bugs, it might seem better to expire bugs
which are in non-triaged state which haven't had activity for >18 months.
This would seem like a less aggressive approach to closing issues which
haven't managed to reach triaged state and are otherwise stale.

This information is available in the API and i'd be happy to suggest a
review to change to this model if it is agreed.

Thanks

--
Kind Regards,
Dave Walker
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Team blogs

[Dave Walker]

On 9 May 2016 at 23:37, Joshua Harlow <harlo...@fastmail.com> wrote:

> After seeing the amount of summit recaps and the scattered nature of these
> (some on the ML, some on etherpads, some on personal blogs); I am starting
> to wonder if we should again bring up the question of having infra (and I
> guess the foundation?) support/provide a place for team blogs...
>
> I've heard its been discussed before but nothing much ever came of that
> (not enough people?) and I'm wondering if we can restart this kind of
> effort so that teams can show-case what is happening in there project,
> project summit recaps, newer or advertise less used features/functionality
> and so-on. Having such hosted team blogs also makes it easier to have a
> history of information (personal blogs can be deleted...) and I think can
> be quite useful for collaboration/information sharing.
>
> Thoughts?
>
> -Josh
>

The OpenStack Security Project has currently put their blog on github,
using Jekyll.. because of the low barrier of entry and collaborative review
nature of git.
http://openstack-security.github.io

It would be nicer if this workflow could be ported to other teams and was
hosted within openstack-infra and using standard tooling.

--
Kind Regards,
Dave Walker
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron] Re: [stable] Requesting exception for stable/kilo

[Dave Walker]

On 4 May 2016 at 17:55, Sukhdev Kapur <sukhdevka...@gmail.com> wrote:

> Hi Stable Maintainers,
>
> We have a critical bug impacting customers production network. This is a
> minor fix.
> I would like to request an exception so that the customers do not have to
> baby
> sit this patch.
>
> https://review.openstack.org/#/c/309653/
>
> Please allow this to be merged.
>
> Thanks
> -Sukhdev
>
>
Hi,

Thanks for raising this.

I'm currently blocking the 2015.1.4 release on this request.  As it isn't a
clean backport and the 11th hour, I really want a review from neutron-core.

However, this hasn't been forthcoming.  I really need this asap, or we will
have to release without it.

Ps. apologies for the empty reply just now.

--
Kind Regards,
Dave Walker
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron] Re: [stable] Requesting exception for stable/kilo

[Dave Walker]

On 4 May 2016 at 17:55, Sukhdev Kapur  wrote:

> Hi Stable Maintainers,
>
> We have a critical bug impacting customers production network. This is a
> minor fix.
> I would like to request an exception so that the customers do not have to
> baby
> sit this patch.
>
> https://review.openstack.org/#/c/309653/
>
> Please allow this to be merged.
>
> Thanks
> -Sukhdev
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [all][stable] stable/kilo (2015.1.4) Release Freeze

[Dave Walker]

Hi,

I have just frozen pending merges for stable/kilo in preparation for the
next (and final) point release.

It was agreed in Austin that Kilo will be End Of Life'd directly after this
release.

Therefore, any changes not merged during this freeze will be abandoned.

Currently there are 54 changes 'frozen'[0].  Whilst the stable teams will
be trying to help review them during this freeze, any specific changes
required should be called out on this list.

To provide adequate time for landing any exception changes and testing, I'm
currently expecting to release 2015.1.4 on Monday 9th May.

Any questions, please reply here or #openstack-stable

[0] https://review.openstack.org/#/q/status:open+branch:stable/kilo

Thanks

--
Kind Regards,
Dave Walker
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-security] [Security]abandoned OSSNs?

[Dave Walker]

Hi,

I believe 50 and 51 were both assigned to me.  They were closely linked,
but seperate issues.

I wrote 50 up here:
https://review.openstack.org/#/c/200303/2

After discussion in a security meeting, my memory is that it was agreed
that they probably weren't required.

I'd have to pull out the meeting log to be certain, but I'd also continue
them if the mood has now changed.

--
Kind Regards,
Dave Walker

On 11 Apr 2016 16:06, "Clark, Robert Graham" <robert.cl...@hpe.com> wrote:
>
> Thanks Matt, Michael,
>
>
>
> To start with, lets look quickly at the more recent OSSNs that are marked
as work in progress, namely 63,64,65 and 66 – these should all be published
within a week or so.
>
>
>
> Looking further back we have the more difficult OSSNs 50 and 51, I’m not
100% sure what the blockers are on these.  I believe
https://wiki.openstack.org/wiki/OSSN/OSSN-0056 may supersede OSSN-0051 and
is rooted in bug https://bugs.launchpad.net/ossn/+bug/1435530 - it looks to
me like OSSN-0056 was written during a mid-cycle and could be the right one.
>
>
>
> I’m struggling to work out the story behind OSSN-0050 – I’m adding Nathan
Kinder who might be able to shed more light on this.
>
>
>
> -Rob
>
>
>
>
>
>
>
> From: Michael Xin [mailto:michael@rackspace.com]
> Sent: 11 April 2016 15:28
> To: Matt Fischer; OpenStack Development Mailing List (not for usage
questions)
> Subject: Re: [openstack-dev] [Openstack-security] [Security]abandoned
OSSNs?
>
>
>
> Matt:
>
> Thanks for asking this. I forwarded this email to the new email list so
that folks with better knowledge can answer this.
>
>
>
>
>
> Thanks and have a great day.
>
>
>
> Yours,
>
> Michael
>
>
>
>
>
>
-
>
> Michael Xin | Manager, Security Engineering - US
>
> Product Security  |Rackspace Hosting
>
> Office #: 501-7341   or  210-312-7341
>
> Mobile #: 210-284-8674
>
> 5000 Walzem Road, San Antonio, Tx 78218
>
>

>
> Experience fanatical support
>
>
>
> From: Matt Fischer <m...@mattfischer.com>
> Date: Monday, April 11, 2016 at 9:19 AM
> To: "openstack-secur...@lists.openstack.org" <
openstack-secur...@lists.openstack.org>
> Subject: [Openstack-security] abandoned OSSNs?
>
>
>
> Some folks from our security team here asked me to ensure them that our
services were patched for all the OSSNs that are listed here:
https://wiki.openstack.org/wiki/Security_Notes
>
>
>
> Most of these are straight-forward, but there are some OSSNs that have
been allocated an ID but then abandoned. There is no detailed wiki page and
my best google efforts lead me to a possible IRC mention and maybe an
abandoned review. The two specifically are OSSN-50/51.
>
>
>
> So what am I to do with an "abandoned" OSSN? Has it been decided that
there is no issue anymore? These are pretty old if I look at the dates
framing the other OSSNs (49/52), so I assume they aren't urgent. Can we
ignore these? They sound somewhat scary, for example, "keystonemiddleware
can allow access after token revocation" but I have no means to say whether
it affects us or how we can mitigate without more info.
>
>
>
> Thoughts?
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [stable] Proposing Tony Breeds for stable-maint-core

[Dave Walker]

On 18 Mar 2016 20:11, "Matt Riedemann" <mrie...@linux.vnet.ibm.com> wrote:
>
> I'd like to propose tonyb for stable-maint-core.

> Please respond with ack/nack.

+1, Great work Tony.

--
Kind Regards,
Dave Walker
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][infra] revert new gerrit

[Dave Walker]

On 18 March 2016 at 16:21, Andrey Kurilin <akuri...@mirantis.com> wrote:

> > If you haven't tried gertty yet, I highly recommend it.
>
> I have an opened tab with it and I should try it, but I wonder why we need
> gerrit if it a bad thing for a lot of contributors? Is there an alternative
> for it?
> Btw, I'm ready to have local instance of gerrit for myself. But I didn't
> find a way to configure it to use upstream API.
>
> > Someone should make a mobile-native app (android/ios) for gerrit now
> since the new interface is so bad. Hopefully someone somewhere can come up
> with the time for it. (I haven't had the time myself).
>
> It looks like there is an app for android -
> https://play.google.com/store/apps/details?id=com.jbirdvegas.mgerrit ,
> but, unfortunately, I have a phone from Apple and I didn't find anything
> for it.
>
>
I found this to be pretty effective with a friendly upstream.  Last summer
I pushed a change to their upstream to have OpenStack gerrit included in
the default app and it was warmly received  I used to use this app' quite a
bit when commuting.  Would recommend.

--
Kind Regards,
Dave Walker
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [stable][ceilometer][all] stable/kilo 2015.1.3 delayed

[Dave Walker]

On 29 January 2016 at 19:34, gordon chung <g...@live.ca> wrote:

>
> hmm.. that's unfortunate... anything we need to update so this doesn't
> happen again? or just a matter of lesson learned, let's keep an eye out
> next time?
>
> i guess the question is can users wait (a month?) for next release? i'm
> willing to poll operator list (or any list) to query for demand if
> that's easier on your end? if there's very little interest we can defer
> -- i do have a few patches lined up for next kilo release window so i
> would expect another release.
>
> cheers,
>

I'd like to think that in the new world order of proposing tags through
gerrit, rather than direct applying this could be avoided.

When Iapplied the tag locally, the current state of the branch did sdist
successfully.. but when jenkins tried to react to the pushed tag it was
non-buildable. This is yet another reason why directly applying tags
should burn.

Thanks

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [stable][ceilometer][all] stable/kilo 2015.1.3 delayed

[Dave Walker]

On 29 January 2016 at 20:36, Jeremy Stanley <fu...@yuggoth.org> wrote:
> On 2016-01-29 19:34:01 + (+), gordon chung wrote:
>> hmm.. that's unfortunate... anything we need to update so this doesn't
>> happen again? or just a matter of lesson learned, let's keep an eye out
>> next time?
>
> Well, I backported the downloadcache removal to the stable/kilo
> branch after discovering this issue, and while that's too late to
> solve it for 2015.1.3 it will at least no longer prevent a 2015.1.4
> tarball from being built.
>
>> i guess the question is can users wait (a month?) for next release? i'm
>> willing to poll operator list (or any list) to query for demand if
>> that's easier on your end? if there's very little interest we can defer
>> -- i do have a few patches lined up for next kilo release window so i
>> would expect another release.
>
> I'm perfectly okay uploading a tarball I or someone else builds for
> this, as long as it's acceptable to leadership from stable branch
> management, Telemetry and the community at large. Our infrastructure
> exists to make things more consistent and convenient, but it's there
> to serve us and so we shouldn't be slaves to it.

Unless anyone else objects, I'd be really happy if you are willing to
scp a handrolled tarball.

I'm happy to help validate it's pristine-state locally here.

Thanks Jeremy!

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [stable][ceilometer][all] stable/kilo 2015.1.3 delayed

[Dave Walker]

Hi,

As many of you have probably seen, most of the projects targeted for
2015.1.3 release have been tagged and have tarballs ready for release.

However, pip 8 was released around the same time as the tarballs were
attempted to be generated.  Most of the projects are OK with this, but
ceilometer declares pbr!=0.7,<1.0,>=0.6 and then forces an update via
tox.

This has caused a delay across all the projects in announcing the
release.  Jeremy Stanley has been super useful in helping to pin-point
this (and another infra issue).. but now we are stuck.

This leaves us with bit of a pickle, and i'm looking for suggestions
how best we can move forward:

A few ideas are:
- stable/kilo is now fixed, so we could do another patch release which
would succeed (but ugly version numbering)
- rebuuild ceilometer from a separate PyPI mirror which lacks pip>=8 in the pool
- Manually push a tarball generated outside of Jenkins.
- Skip ceilometer for this release.

Any other ideas?

Thanks

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [stable] kilo 2015.1.3 freeze exception request for cve fixes

[Dave Walker]

On 15 January 2016 at 10:01, Thierry Carrez <thie...@openstack.org> wrote:
> Ihar Hrachyshka wrote:
>>
>> +1. CVE fixes obviously should be granted an exception.
>
>
> +1
>

Agreed, I have already +2'd on Gerrit.  Can another core please do the same?

Thanks

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [stable] Call for testing: 2015.1.3 candidate tarballs

[Dave Walker]

Hi all,

We are scheduled to release the next point release of stable Kilo
(2015.1.3).  The current candidates for release are: Ceilometer,
Cinder, Glance, Heat, Horizon, Ironic, Keystone, Neutron,
Neutron-lbaas, Neutron-vpnaas, Nova and Sahara releases on Thursday
Jan 21st 2016.

The current open changes on Gerrit have been frozen to help testing of
the branches prior to release.  Please help test the current
candidates as described below:

Ceilometer
 - Commits since previous release: 19
 - Last built: Sat, 09 Jan 2016 00:58:01 GMT
 - http://tarballs.openstack.org/ceilometer/ceilometer-stable-kilo.tar.gz

Cinder
 - Commits since previous release: 19
 - Last built: Tue, 05 Jan 2016 19:45:36 GMT
 - http://tarballs.openstack.org/cinder/cinder-stable-kilo.tar.gz

Glance
 - Commits since previous release: 7
 - Last built: Wed, 06 Jan 2016 20:46:58 GMT
 - http://tarballs.openstack.org/glance/glance-stable-kilo.tar.gz

Heat
 - Commits since previous release: 32
 - Last built: Tue, 12 Jan 2016 06:14:01 GMT
 - http://tarballs.openstack.org/heat/heat-stable-kilo.tar.gz

Horizon
 - Commits since previous release: 13
 - Last built: Thu, 14 Jan 2016 12:59:37 GMT
 - http://tarballs.openstack.org/horizon/horizon-stable-kilo.tar.gz

Ironic
 - Commits since previous release: 3
 - Last built: Thu, 03 Dec 2015 07:45:38 GMT
 - http://tarballs.openstack.org/ironic/ironic-stable-kilo.tar.gz

Keystone
 - Commits since previous release: 19
 - Last built: Thu, 14 Jan 2016 04:03:08 GMT
 - http://tarballs.openstack.org/keystone/keystone-stable-kilo.tar.gz

Neutron
 - Commits since previous release: 75
 - Last built: Wed, 13 Jan 2016 08:41:11 GMT
 - http://tarballs.openstack.org/neutron/neutron-stable-kilo.tar.gz

Neutron-lbaas
 - Commits since previous release: 6
 - Last built: Thu, 31 Dec 2015 13:21:20 GMT
 - http://tarballs.openstack.org/neutron-lbaas/neutron-lbaas-stable-kilo.tar.gz

Neutron-vpnaas
 - Commits since previous release: 5
 - Last built: Wed, 02 Dec 2015 16:34:11 GMT
 - 
http://tarballs.openstack.org/neutron-vpnaas/neutron-vpnaas-stable-kilo.tar.gz

Nova
 - Commits since previous release: 4
 - Last built: Fri, 18 Dec 2015 03:42:49 GMT
 - http://tarballs.openstack.org/nova/nova-stable-kilo.tar.gz

Sahara
 - Commits since previous release: 3
 - Last built: Wed, 02 Dec 2015 11:15:06 GMT
 - http://tarballs.openstack.org/sahara/sahara-stable-kilo.tar.gz

If you have any questions please direct them to this thread or ping me
(Daviey) on #openstack-stable.

Thanks

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [stable] Making stable maintenance its own OpenStack project team

[Dave Walker]

On 13 November 2015 at 14:10, Thierry Carrez <thie...@openstack.org> wrote:
> So.. quick summary of this thread so far.
>
> (-)
> * Not enough work to warrant a designated "team", now that the work is
> decentralized and the cats are mostly herding themselves
> * The change is unlikely to bring a meaningful improvement to the
> situation, or sudden new resources
>
> (+)
> * An empowered team could tackle new coordination tasks, like engaging
> more directly in converging stable branch rules across teams, or
> producing tools
> * Release management doesn't overlap anymore with stable branch, so
> having them under that PTL is limiting and inefficient
> * Reinforcing the branding (by giving it its own team) may encourage
> more organizations to affect new resources to it
>
> In summary, I think this is worth a try. If the team fails, at least it
> will be on its own rather than as the 5th squeaky wheel of release
> management (where lack of leadership and focus could be rightly blamed
> for failure).
>
> For this to succeed, we need someone to own the effort and push it
> forward, and a number of people caring enough about it to attend regular
> meetings about it and to lurk on #openstack-stable. I'm fine helping the
> team in the spin-off effort but I don't want to lead it (I proved I was
> unable to make it my top priority in the past, so I think the team
> deserves a more focused lead).
>
> Erno has volunteered to lead. Ihar, Alan (Pevec) and myself don't have
> enough time to lead but are happy to help. Anyone else interested to
> join that initial group ? Flavio ? Matt ?
>
> Once we have a list of key members we should set up a meeting to discuss
> the details.

I think this summary is pretty reflective of the thread so far.  I
also agree that I see very little benefit from transitioning the
stable-maint-core team into a recognised project team, I can't imagine
there being any measurable benefit from this.. feels more like a paper
exercise..  however, I don't feel strongly enough to try and stop
this.

Due to limited time, I'd not like to be a driver... but would like to
be part of the seed group of cores.  If this is to happen, it  seems
reasonable to map stable-maint-core -> stable-maint project.

We've traditionally been pretty bad with discussion and engaging with
fellow stable-maint members, so a standing (short) meeting might help
improve this.

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [stable] [infra] How to auto-generate stable release notes

[Dave Walker]

On 25 August 2015 at 10:28, Alexis Lee lx...@hpe.com wrote:
 Thierry Carrez said on Fri, Aug 21, 2015 at 04:56:37PM +0200:
 Tag-every-commit:
 (+) Conveys clearly that every commit is consumable
 (-) Current tooling doesn't support this, we need to write something
 (-) Zillions of tags will make tags ref space a bit unusable by humans

 Time to time tagging:
 (+) Aligned with how we do releases everywhere else
 (-) Makes some commits special
 (-) Making a release still requires someone to care

 Missing anything ?

 Without offering an opinion either way, I'm just wondering how
 tag-every-commit is superior to never tagging? The git SHAs already
 uniquely identify every commit; if you want only those on master, simply
 `git log master`.


 Alexis (lxsli)

Hey Alexis,

The issue with this is deterministic version counting between commits,
allowing distributed additional commits but still keeping the version
counting centralised.  We use pbr to determine version numbers, which
has logic around git tags to determine version numbering.

For example:
$ git clone master # == version 1
$ echo foo  stuff.txt
$ git add stuff
$ git commit stuff.txt -m Daviey's awesome value-add
# # should still == version 1, but without a centralised reference
marker it will be version 2.

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [stable] [infra] How to auto-generate stable release notes

[Dave Walker]

On 21 August 2015 at 11:38, Thierry Carrez thie...@openstack.org wrote:
SNIP
 Since then, replying to another concern about common downstream
 reference points, we moved to tagging everything, then replying to
 Clark's pollution remark, to tag from time to time. That doesn't
 remove the need to *conveniently* ship the best release notes we can
 with every commit. Including them in every code tarball (and relying on
 well-known python sdist commands to generate them for those consuming
 the git tree directly) sounded like the most accessible way to do it,
 which the related thread on the Ops ML confirmed. But then I'm (and
 maybe they are) still open to alternative suggestions...

This is probably a good entry point for my ACTION item from the
cross-project meeting:

I disagree that time to time tagging makes sense in what we are
trying to achieve.  I believe we are in agreement that we want to move
way from co-ordinated releases and treat each commit as an accessible
release.  Therefore, tagging each project at arbitrary times
introduces snowflake releases, rather than the importance being on
each commit being a release.

I agree that this would take away the 'co-ordinated' part of the
release, but still requires release management of each project (unless
the time to time is automated), which we are not sure that each
project will commit to.

If we are treating each commit to be a release, maybe we should just
bite the bullet and enlarge the ref tag length.  I've not done a
comparison of what this would look like, but I believe it to be rare
that people look at the list anyway.  Throwing in a | grep -v
^$RELEASE*, and it becomes as usable as before.  We could also
expunge the tags after the release is no longer supported by upstream.

In my mind, we are then truly treating each commit as a release AND we
benefit from not needing hacky tooling to fake this.

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [requirements] modifying the 'is it packaged' test

[Dave Walker]

On 22 August 2015 at 00:04, Matthew Thode prometheanf...@gentoo.org wrote:
 On 08/21/2015 05:59 PM, Robert Collins wrote:
 On 22 August 2015 at 10:57, Matthew Thode prometheanf...@gentoo.org wrote:
 Packaging for us is fairly easy, but it is annoying to have to add 5-6
 deps each release, (which means we are adding cruft over time).

 We're adding functionality by bringing in existing implementations.
 Surely thats better than reinventing *everything* ?

 -Rob

 totally, more of a minor annoyance :P

A strong reason that requirements was created was to give distros a
voice and avoid incompatible versions, which was more of a problem for
distros than it was for each different service at that point.

I'm not sure that a requirement has ever been not included because it
*wasn't* packaged, but perhaps because it *couldn't* be packaged.  Is
there an example that has caused you to raise this?

The is-it-packaged-test was added at a time where large changes were
happening in OpenStack right up to the (release) wire and cause scary
changes for distros that were tracking the release.  Now, Feature
development has become more mature with the scary stuff being front
loaded, I'm not quite sure this is such a problem.

The release schedule used to document a DepFreeze[0] to avoid nasty
surprises for distros, which used to be at the same point of
FeatureFreeze[1].  This reference seems to have been removed from the
last few cycles, but I would suggest that it could be re-added.

[0] https://wiki.openstack.org/wiki/DepFreeze
[1] https://wiki.openstack.org/wiki/FeatureFreeze

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [stable] [infra] How to auto-generate stable release notes

[Dave Walker]

On 17 August 2015 at 15:59, Jeremy Stanley fu...@yuggoth.org wrote:
 On 2015-08-17 15:46:24 +0200 (+0200), Thierry Carrez wrote:
 [...]
 OSSA: -ZZZ
 For commits that correspond to vulnerability fixes.
 [...]

 I don't think that's going to be feasible. Consider the sequence
 with a public security vulnerability... often the OSSA number isn't
 assigned until after one or more backports have been approved. With
 some careful controls introduced into the VMT process we may be able
 to make sure most of these get updated commit messages before they
 merge, but would still need a plan to solve for the times when
 backported security fixes slip in without an OSSA header in the
 commit message.

Maybe this is a perfect use-case for git-notes?  This means the commit
itself isn't touched and the non-scale git-tag space isn't wasted?

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [keystone] LDAP identity driver with groups from local DB

[Dave Walker]

On 24 July 2015 at 05:00, Julian Edwards bigjo...@gmail.com wrote:
 Hello,

 I am relatively new to Openstack and Keystone so please forgive me any
 crazy misunderstandings here.

 One of the problems with the existing LDAP Identity driver that I see
 is that for group management it needs write access to the LDAP server,
 or requires an LDAP admin to set up groups separately.

 Neither of these are palatable to some larger users with corporate
 LDAP directories, so I'm interested in discussing a solution that
 would get acceptance from core devs.

 My initial thoughts are to create a new driver that would store groups
 and their user memberships in the local keystone database, while
 continuing to rely on LDAP for user authentication. The advantages of
 this would be that the standard UI tools could continue to work for
 group manipulation.  This is somewhat parallel with ephemeral
 federated user group mappings, but that's all done in the json blob
 which is a bit horrible. (I'd like to see that working with a decent
 UI some time, perhaps it is solved in the same way)

 However, one of the other reasons I'm sending this is to gather more
 ideas to solve this. I'd like to hear from anyone in a similar
 position, and anyone with input on how to help.


Hey Julian,

Can I suggest reading this excellent write up by Adam Young?
http://adam.younglogic.com/2013/10/read-only-ldap-in-keystone/

Tl;DR is that the *User* management can come from LDAP via the
Identity driver, but the Project/Tenants and Roles on these come from
the *Assignment* driver via SQL - almost as an overlay.

This would seem to solve the issue you outline?

As a side note, I had a comparable idea for an external AuthN driver
to plug into legacy RBAC systems but this area of Keystone wants to
focus on Federation rather than extending interaction at other levels.
You may fine the thread of interest:
http://lists.openstack.org/pipermail/openstack-dev/2014-October/048639.html

Thanks

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [keystone] LDAP identity driver with groups from local DB

[Dave Walker]

On 24 July 2015 at 15:26, Boris Bobrov bbob...@mirantis.com wrote:
 On Friday 24 July 2015 09:29:32 Dave Walker wrote:
 On 24 July 2015 at 05:00, Julian Edwards bigjo...@gmail.com wrote:
 Tl;DR is that the *User* management can come from LDAP via the
 Identity driver, but the Project/Tenants and Roles on these come from
 the *Assignment* driver via SQL - almost as an overlay.

 This would seem to solve the issue you outline?

 As far as I understand the issue is that corps want to have users in read-only
 LDAP and have an ability to create groups outside of LDAP, in SQL.

 Am I right?

I think as you described can already be done.  There are SQL and LDAP
backends for both Assignment and Identity and a deployment can choose
their own mix, including RO for LDAP.  However, if you have an
enterprise resource / entitlement management system which can map to
OpenStack Assignments, but not exposed via SAML or LDAP - you either
need to mirror in SQL or SOL.

User authentication is pretty solid via the external Identity
management as you can simply use anything that sets REMOTE_USER
upstream in the web server (Kerberos or X.509 makes sense).

The issue I wanted to solve was 'backend' AuthN and AuthZ integration
to external systems.  The current Federation mechanism relies on
getting Assignment (Roles and Groups) from keystone edge service via
SAML assertions.  I needed to keep Keystone stateless, but use
deployment specific plugin to grab User, Project and Role data from a
third-party entitlement system at runtime (via a backend plugin).

The direction of Keystone (based on the thread I linked to and summit
conversations) is to embrace edge Federation more and reduce the
backend integration.  Which makes sense really.. but when hit with
legacy enterprise resource, entitlement and access systems that don't
support SAML, the need to absorb tech' debt is a balance.  Thankfully,
Keystone still makes it quite easy to maintain custom backend drivers.

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [requirements] backlog of requirements changes

[Dave Walker]

On 21 July 2015 at 16:17, Robert Collins robe...@robertcollins.net wrote:
 There seem to be quite a backlog in openstack/requirements.

 http://russellbryant.net/openstack-stats/requirements-reviewers-30.txt

 there are roughly 10x new changes to the number of reviews cores are
 managing to do.

 This worries me, and I'd like to help.

 How can I best do so?


It seems that last year I was silently dropped from requirements-core
and so I've been less interested in providing trivial +1's which IMO
rarely add much value to this project.   Looking at the stats, i'm
still healthily represented.

If i'm re-added, i'll gladly help more with reviews.

Thanks

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Non-responsive upstream libraries (python34 specifically)

[Dave Walker]

On 17 Jul 2015 1:38 pm, Morgan Fainberg morgan.fainb...@gmail.com wrote:

  On Jul 17, 2015, at 04:36, Victor Stinner vstin...@redhat.com wrote:
SNIP
  FYI some colleagues just forked python-ldap in
https://github.com/pyldap/pyldap/ to add Python 3 support. Stay tuned ;-)
 

 I highly recommend contributing to the ldap3 project instead of
continuing the python-ldap work. Ldap3 is (imho) a much better design and
really (if anything) simply lacks a compatibility layer.

 Keystone will be moving to ldap3 (regardless of the compat module) either
this cycle or next.


As a point of reference, openstack/anchor went to ldap3 last week
uneventfully, to achieve py3 support.

There is a pending global-requirements change to bring it into there.

Thanks

--
Kind Regards,
Dave Walker
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Setting epoch in setup.cfg??

[Dave Walker]

On 13 Jul 2015 8:52 pm, Ian Cordasco ian.corda...@rackspace.com wrote:
 On 7/13/15, 03:38, Thierry Carrez thie...@openstack.org wrote:
SNIP
 By counter-productive, I meant: likely to generate more confusion than
 clarity. If you provide an epoch in the version and it doesn't match
 downstream packagers ones, it's hard to rely on it.

 I see what you mean now. The thing is that for Debian/Fedora the epoch
 syntax is different from PEP 440

 For them it's

 [distro-epoch]:[upstream-version][otherstuff]

 PEP 440 epochs are separated by a !, so let's say $(distro) has an epoch
 value of 1 and we choose 2, for glance the version would look ugly but
 would be:

 1:2!11.0.0

SNIP

This would be a problem for at least Ubuntu and Debian as the version
string is specifically not allowed to contain a '!'.

The upstream_version may contain only alphanumerics and the
characters . +- : ~ (full stop, plus, hyphen, colon, tilde) and should
start with a digit. [0]

Therefore, these distro's would need to increment the distro epoch if the
upstream version (without the upstream epoch) is lower than the version
currently in their archives.

I am not sure how rpm based distro's handle '!' in the upstream version.

[0]
https://www.debian.org/doc/debian-policy/ch-controlfields.html#s-f-Version

--
Kind Regards,
Dave Walker
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Setting epoch in setup.cfg??

[Dave Walker]

On 13 July 2015 at 21:58, Ian Cordasco ian.corda...@rackspace.com wrote:
 On 7/13/15, 15:09, Dave Walker em...@daviey.com wrote:
SNIP


Therefore, these distro's would need to increment the distro epoch if the
upstream version (without the upstream epoch) is lower than the version
currently in their archives.
I am not sure how rpm based distro's handle '!' in the upstream version.

 In other words, if Debian/Ubuntu currently have Glance versioned:

   1:2015.1.0

 They'll be doing

   2:8.0.0

 For Liberty?

They already are:
https://launchpad.net/ubuntu/+source/glance/2:11.0.0~b1-0ubuntu2

--
KInd Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Should project name be uppercase or lowercase?

[Dave Walker]

On 7 July 2015 at 13:11, Thierry Carrez thie...@openstack.org wrote:
 Yuiko Takada wrote:
 Hi folks,

 I found the difference between wiki[1] and governance[2].
 wiki says Generally the capitalization of the project team names like
 swift is lowercase.
 But in governance, written as uppercase like Nova, Swift.

 How do you think which should we use uppercase vs lowercase for
 representing project names?

 My personal take on it is that the project names (or the code
 repositories) are lowercase: nova, or openstack/nova.

 The *project team* name is capitalized: the Nova project team.

 I'm sure that Anne has a pretty strong position on that, though :)


This makes entire sense to me.  I assumed that when used
programmatically (or repositroy names) it should be in lowercase
(unless a class name), when in prose it should be uppercase.

For a recent documentation change, I put a project names capitalised
which feels entirely appropriate as it is a proper noun.  However,
review requested that it followed documentation convention[0] of using
lower case so I amended it to follow.

I am interested to hear why the convention in documentation for it to
be lower case.

[0] 
https://wiki.openstack.org/wiki/Documentation/Conventions#Service_and_project_names

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] ALL dependencies pinned in devstack-gate now

[Dave Walker]

On 3 July 2015 at 10:17, Robert Collins robe...@robertcollins.net wrote:
 On 3 July 2015 at 21:01, Thierry Carrez thie...@openstack.org wrote:
SNIP
 - Does that (or should that) also affect stable/kilo and stable/juno ?
 (there is no upper-constraints there)

 No, and since the disruption of a new pbr and associated minimum
 versions throughout stable would be huge, I've no plans to backport
 it. If folk wanted to (e.g. if stable suffers from lots of firedrills
 even with the caps we added last time) I can throw up some patches and
 we can see about it: but its a big effort requiring point releases of
 /everything/ (because of the pbr version issue).


At the moment, it seems that additional point releases need to be
created on-demand - but the situation does seem better than it did a
year ago.  Is it less effort to do this across the board, once for all
stable/* or continue the current process?

Long term, we'll benefit from this on stable/liberty - but defining a
process for the old world is probably useful.

Thanks

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all]deprecating [test-]requirements-PYN.txt

[Dave Walker]

On 29 June 2015 at 04:59, Robert Collins robe...@robertcollins.net wrote:
 Hi, so we're nearly ready to deprecate the python-version-specific
 requirements files. Once we have infra's requirements cross checking
 jobs all copacetic again, we should be able to move forward.

 There isn't a specific spec for this in pbr, and I wanted to get some
 broad input into the manner of the deprecation.
SNIP

Slightly offtopic, but I've noticed that some consumers of bandit[0]
have been creating requirements-bandit.txt.  This is to specify bandit
requirements without requiring the whole test-requirements.txt env to
be installed, to run what is essentially a linting tool.

I'm not sure I like the idea of creating MORE requirements.txt style
files as it pollutes the project root namespace and currently has no
syncing from global-requirements.

I wondered if you had any ideas on how to solve this for bandit usage,
and potentially other projects?

[0] https://wiki.openstack.org/wiki/Security/Projects/Bandit

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder][stable] Cinder client broken in Juno

[Dave Walker]

On 24 June 2015 at 14:34, Jeremy Stanley fu...@yuggoth.org wrote:
 On 2015-06-24 09:01:51 +0300 (+0300), Duncan Thomas wrote:
 The problem there is that it takes (significant) time for the connection
 attempt to time out - every cinder command taking several minutes is not
 acceptable.
 [...]

 Another silly idea from an end user of cinderclient here, but try
 both in parallel and use whichever responds first (abandoning the
 other connection attempt at that point)?

 I just don't buy that we can consider new versions of the client
 requiring existing deployments on still relevant releases to upgrade
 to support it (either to a newer release, a patch, an additional
 config option, some combination of the three). What we're going to
 find instead is some (lots of?) providers updating their
 documentation to tell users that they should stick with an older
 cinderclient release indefinitely, and that's terrible from an
 OpenStack interoperability standpoint.

The clients have always strived to have backwards compatibility, and
this seems to be no different from the other scenarios that have been
encountered.  We've started to get away from that, and it feels like a
mistake.

As Jeremy points out, I fail to see why fallback default behaviour
cannot be used.. Attempt to use version discovery feature, if fail -
fall back to legacy behaviour..

What are the issues associated with this?

Thanks

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Debian already using Python 3.5: please gate on that

[Dave Walker]

On 20 Jun 2015 1:05 pm, Doug Hellmann d...@doughellmann.com wrote:
SNIP

  Whether we want to support 3.4 and 3.5, or just 3.4 and then just 3.5
  is an ecosystem question IMO, not an upstream one. 3.4 and 3.5 are
  very similar when you consider the feature set crossover with 2.7.

 Right, and IIRC that's why we said at the summit that we would rather
 not take the resources (in terms of people and CI servers) to run tests
 against both, yet. Let's get one project fully working on one version of
 python 3, and then we can start thinking about whether we need to
 test multiple versions.

Having information available, even if there is not immediate intent to
support seems like something we should constantly be driving for.  It
should certainly be non-voting, but if resources are limited - perhaps a
periodic job, rather that gate?

 OTOH, if Canonical doesn't release a version of 3.4 that removes the
 core dump bug soon, I will support moving fully to 3.5 or another test
 platform, because that bug is causing us trouble in Oslo still.

s/Canonical/ubuntu

Can you link to the bug? I did a quick search, but couldn't find it quickly.

--
Kind Regards,
Dave Walker
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Dave Walker]

On 17 June 2015 at 17:19, Thierry Carrez thie...@openstack.org wrote:
 Ihar Hrachyshka wrote:
 On 06/17/2015 05:40 PM, Douglas Mendizábal wrote:
 I tend to agree with Thomas that plan D is not ideal.  For one, it
 prevents changes to the stable branch that span multiple CRs, since
 a two patch change would generate two tags and there would be no
 clear indication that the first patch should not be released on its
 own.

 If we will end up with a half-broken product due to merging a patch
 without another one, then those patches should be squashed. Also, I
 wonder how they will pass gate if something is broken. Do you suggest
 that test coverage is incomplete?

 Also remember we are talking about *stable branches* here. The only
 acceptable things there are bugfix backports and CVE fixes. So:

 - there is no case of partially-merged features that span multiple CRs
 - in the rare case of a bugfix needing multiple commits, those should be
 squashed in the stable branch
 - if the fix is so huge that it can't be merged as a single patch, it's
 probably not desirable in the stable branch

 The stable branch should always be usable and releaseable. If it's not,
 you're doing it wrong.


Just to add to this, we have a decent demonstrable history of
squashing commits together that have been required to land
concurrently.  We need to do this when the Gate is wedged, and
requires two or more concurrent backports to unwedge.

Such as: https://review.openstack.org/#/c/163378/

As Ihar points out, the test coverage mandates that we do this.. which
nicely helps us provide some assurance that the branch is constantly
releasable.

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [packaging] Adding packaging as an OpenStack project

[Dave Walker]

On 10 June 2015 at 11:07, Robert Collins robe...@robertcollins.net wrote:
 On 10 June 2015 at 20:12, Matthias Runge mru...@redhat.com wrote:

 Since our software is to be consumed by packages, shouldn't the packages
 project consider itself to be responsible for global requirements? I.e.
 checking, if requirements are packageable, if versions fit, etc.

 I think we welcome input from distribution maintainers on
 global-requirements; especially for new packages.

 But, the responsibility is ultimately a team effort: all the
 components of openstack have to meet the operator/distributor
 co-installability requirement. If one project has a minimum version of
 X, its not possible for other projects to have a max version of  X
 otherwise we're not coinstallable. This works both ways of course.

 In some distros, there are multiple versions of the same package allowed, in
 others, it's forbidden.

 Thats true, but its also a per-distro thing. Within a distro you need
 to be consistent. There's no need for RHEL to match RDO for instance,
 and trying to make that happen across a dozen redistributors in the
 OpenStack context makes no sense at all. We're moving to making our
 ranges as wide as we can to make life easier for anyone that wants to
 pick slightly different versions: we can't assert that it will work,
 but unless we know it doesnt', we won't preclude you trying :)

 -Rob


Just to add some history here, this was *precisely* the problems that
vendors were having - but worse, each openstack project had
conflicting version requirements making it really quite hard for
distro's to centralise package this..

This is why the project, openstack/requirements was created to
centralise the management of this to avoid conflicting version
requirements AND get input back from distro's and vendors.  The
initial core reviewers was seeded by representatives of distro's and
vendors to get their input on viability in distro's.

Vendors and distro's didn't engage as much as they could have (myself
included), which means that they had less input.  It is pretty easy to
get that input back, you just need to review the incoming changes:
https://review.openstack.org/#/q/status:open+project:openstack/requirements+branch:master,n,z

I did start working on a jenkins job to check distro's to see what
version of package was already in releases, but it wasn't really
reliable enough, so I dropped it on the floor.  If you wanted to work
on a jenkins job to provide advise on proposed changesets, I am sure
the infra' team would be supportive.

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Dave Walker]

On 10 June 2015 at 09:53, Thomas Goirand z...@debian.org wrote:
 On 06/05/2015 02:46 PM, Thierry Carrez wrote:
 So.. summarizing the various options again:

 Plan A
 Just drop stable point releases.
 (-) No more release notes
 (-) Lack of reference points to compare installations

 Plan B
 Push date-based tags across supported projects from time to time.
 (-) Encourages to continue using same version across the board
 (-) Almost as much work as making proper releases

 Plan C
 Let projects randomly tag point releases whenever
 (-) Still a bit costly in terms of herding cats

 Plan D
 Drop stable point releases, publish per-commit tarballs
 (-) Requires some infra changes, takes some storage space

 Plans B, C and D also require some release note / changelog generation
 from data maintained *within* the repository.

 Personally I think the objections raised against plan A are valid. I
 like plan D, since it's more like releasing every commit than not
 releasing anymore. I think it's the most honest trade-off. I could go
 with plan C, but I think it's added work for no additional value to the
 user.

 What would be your preferred option ?

 I see no point of doing D. I already don't use tarballs, and those who
 do could as well switch to generating them (how hard is it to run
 python setup.py sdist or git archive?).

 What counts is having a schedule date, where all distros are releasing a
 point release, so we have a common reference point. If that is a fully
 automated process, then great, less work for everyone, and it wont
 change anything from what we had in the past (we can even collectively
 decide for point release dates...).

 Cheers,

 Thomas Goirand (zigo)

This is really one of the things I think we want to get away from...
If *every* stable commit is treated with the seriousness of it
creating a release, lets make every commit a release.

This means that Debian may be using a (micro)patch release newer or
older than a different distro, but the key is that it empowers the
vendors and/or users to select a release cadence that best fits them,
rather than being tied to an arbitrary upstream community wide date.

Yes, this might mean that your cadence might be more or less regular
than an alternative vendor / distribution, but the key is that it
empowers the vendor to meet the needs of their users/customers.

For example, you could select a cadence of rebasing to a release every
6 months - where as another consumer could choose to do one every 6
weeks.  The difference is how much of a jump, at which intervals..
Alternatively, a vendor might choose just to go with stock release +
their own select cherry picked patches from stable/*, which is also a
model that works.

The issue around producing tarballs, is really about having forwards
and backwards verification by means of sha/md5 sums, which is hard to
do when generating your own orig tarball.  Debian, Ubuntu and I
believe Arch have made varying use of 'pristine-tar' - which was an
effort to re-producible tarballs using xdelta to make the sum match.
However, Maintainers seem to be moving away from this now.

When I perform source NEW reviews for Ubuntu Archive, I always check
that getting the source orig tarball can be done with either
get-orig-source (inspecting the generation method) or uscan and then
diff the tarballs with the one included on the upload and the one
generated.  Timestamps (or even shasums) haven't been an important
issue for me, but the actual content and verifiable source is what has
mattered more.

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [packaging] Adding packaging as an OpenStack project

[Dave Walker]

On 10 June 2015 at 15:12, Thomas Goirand z...@debian.org wrote:
 On 06/10/2015 12:25 PM, Dave Walker wrote:
 The initial core reviewers was seeded by representatives of distro's and
 vendors to get their input on viability in distro's.

 Really? James, were you made core on the requirements?

 I once tried to follow the requirements repo, though it moves too fast,
 and sometimes, it takes less than 2 days to get a new thing approved.
 Also, this repository has more than just dependencies, there's a lot of
 noise due to changes changes in projects.txt. I also don't mind any
 upgrade for whatever oslo or client libs.

 I'd love to have an easier way to voice my opinion without having all
 the noise of that repo in my inbox. I'm not sure if there's a solution
 to this problem though.

On the Ubuntu side, I believe Chuck and myself were carrying the flag
(we had +2).  When the openstack/requirements project was created
James was less involved upstream, and two representatives of a distro
ought to have been enough.

An yes, I also found that the flow was too much to keep up with which
is why I tried to automate some of this.  I hoped the burden has
reduced somewhat, as prospective changes now need to do more of the
distro research themselves.

Is the library already packaged in the distros we target (Ubuntu
latest / Fedora latest)?

By adding something to OpenStack global-requirements.txt we are
basically demanding that Linux Distros package this for the next
release of OpenStack. If they already have, great. If not, we should
be cautious of adding it.:ref:`finding-distro-status`[0]

But only so much can be done by non-distro focussed upstream consumers...

[0] 
https://github.com/openstack/requirements/blob/master/README.rst#for-new-requirements

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Dave Walker]

On 10 June 2015 at 17:06, Fox, Kevin M kevin@pnnl.gov wrote:
 So... as an op, without release notes, how am I supposed to figure out the 
 proper upgrade procedure's when you often have to lockstep, in the right 
 order, nova+neutron upgrades (or other project combinations)?

 Thanks,
 Kevin

Hi Kevin,

I suspect there is some confusion here between stable point releases
and major releases.  The major releases are still continuing as
expected with rich release notes.  This is talking about stable patch
releases such as:

https://wiki.openstack.org/wiki/ReleaseNotes/2014.2.3

As you can see there is only notices in Known Issues and Limitations
section of low impact.  I do not believe we've ever required ordering
in the updates of these, as they are supposed to be pretty
conservative changes that shouldn't enforce limitations like that.

HTH

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Dave Walker]

On 8 June 2015 at 16:48, Jeremy Stanley fu...@yuggoth.org wrote:
 On 2015-06-08 10:54:32 +1200 (+1200), Robert Collins wrote:
 On 8 June 2015 at 10:14, Alan Pevec ape...@gmail.com wrote:
  2015-06-06 19:08 GMT+02:00 Ian Cordasco ian.corda...@rackspace.com:
  Not exactly. PBR/OpenStack follow SemVer and 2015.1.0.38 isn't valid
  SemVer (http://semver.org/)
 
  Right, so semver compatible versioning on stable/kilo would be 2015.1.N
  but PBR doesn't support that.

 PBR supports it fine. Whats the issue?

 Having PBR automatically increment the N in 2015.1.N for each patch
 applied on the branch rather than needing it to be tagged.

This breaks the desire of wanting to have a shared version scheme if
consumers add their own local patches via git.  This works fine for
consumers that do not use git for handling their local patches, but
does not support the model of allowing the user to rebase using git.

Perhaps tags ARE superior for this?

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Dave Walker]

On 29 May 2015 at 14:41, Thierry Carrez thie...@openstack.org wrote:
 Hi everyone,

 TL;DR:
 - We propose to stop tagging coordinated point releases (like 2015.1.1)
 - We continue maintaining stable branches as a trusted source of stable
 updates for all projects though

 Long version:

 At the stable branch session in Vancouver we discussed recent
 evolutions in the stable team processes and how to further adapt the
 work of the team in a big tent world.

 One of the key questions there was whether we should continue doing
 stable point releases. Those were basically tags with the same version
 number (2015.1.1) that we would periodically push to the stable
 branches for all projects.

 Those create three problems.

 (1) Projects do not all follow the same versioning, so some projects
 (like Swift) were not part of the stable point releases. More and more
 projects are considering issuing intermediary releases (like Swift
 does), like Ironic. That would result in a variety of version numbers,
 and ultimately less and less projects being able to have a common
 2015.1.1-like version.

 (2) Producing those costs a non-trivial amount of effort on a very small
 team of volunteers, especially with projects caring about stable
 branches in various amounts. We were constantly missing the
 pre-announced dates on those ones. Looks like that effort could be
 better spent improving the stable branches themselves and keeping them
 working.

 (3) The resulting stable point releases are mostly useless. Stable
 branches are supposed to be always usable, and the released version
 did not undergo significantly more testing. Issuing them actually
 discourages people from taking whatever point in stable branches makes
 the most sense for them, testing and deploying that.

 The suggestion we made during that session (and which was approved by
 the session participants) is therefore to just get rid of the stable
 point release concept altogether for non-libraries. That said:

 - we'd still do individual point releases for libraries (for critical
 bugs and security issues), so that you can still depend on a specific
 version there

 - we'd still very much maintain stable branches (and actually focus our
 efforts on that work) to ensure they are a continuous source of safe
 upgrades for users of a given series

 Now we realize that the cross-section of our community which was present
 in that session might not fully represent the consumers of those
 artifacts, which is why we expand the discussion on this mailing-list
 (and soon on the operators ML).

 If you were a consumer of those and will miss them, please explain why.
 In particular, please let us know how consuming that version (which was
 only made available every n months) is significantly better than picking
 your preferred time and get all the current stable branch HEADs at that
 time.

 Thanks in advance for your feedback,

 [1] https://etherpad.openstack.org/p/YVR-relmgt-stable-branch

 --
 Thierry Carrez (ttx)

This is generally my opinion as-well, I always hoped that *every*
commit would be considered a release rather than an arbitrary tagged
date.  This empowers vendors and distributors to create their own
service pack style update on a cadence that suits their requirements
and users, rather than feeling tied to cross-vendor schedule or
feeling bad picking interim commits.

The primary push back on this when we started the stable branches was
a vendor wanting to have known release versions for their customers,
and I don't think we have had comment from that (or all) vendors.  I
hope this is seen as a positive thing, as it really is IMO.

I have a question about still having library releases you mentioned,
as generally - everything in python is a library.  I don't think we
have a definition of what in OpenStack is considered a mere library,
compared to a project that wouldn't have a release.

I also wondered if it might make sense for us to do a better job of
storing metadata of what the shasums of projects used to pass gate for
a given commit - as this might be both useful as a known good state
but also, slightly unrelated, might be helpful in debugging gate
blockages in the future.

Thanks

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Dave Walker]

On 29 May 2015 at 16:25, Matthew Thode prometheanf...@gentoo.org wrote:
SNIP

 for release notes just do git log between commit hashes?


For Ubuntu, I wrote a tool to do just this and generate a Debian style
changelog (later cleaned up quite a lot by adamg!).  It parses the git
log looking for LP bug references and uses the bug title as the
changelog entry, and adds the bug number to the change if there is a
Ubuntu task on the bug tracker.  If there is no bug reference in the
commit, it simply uses the first line of the git commit message.

Seems to work well enough for Ubuntu..

Here is an example of how it presents it
http://changelogs.ubuntu.com/changelogs/pool/main/n/nova/nova_2014.2.3-0ubuntu1/changelog

Let me know if you want a hand with it, as it should be pretty
portable to other distros quite easily.

Thanks

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Dave Walker]

Responses inline.

On 29 May 2015 6:15 pm, Haïkel hgue...@fedoraproject.org wrote:

 2015-05-29 15:41 GMT+02:00 Thierry Carrez thie...@openstack.org:
  Hi everyone,
 
  TL;DR:
  - We propose to stop tagging coordinated point releases (like 2015.1.1)
  - We continue maintaining stable branches as a trusted source of stable
  updates for all projects though
 

 Hi,

 I'm one of the main maintainer of the packages for Fedora/RHEL/CentOS.
 We try to stick as much as possible to upstream (almost zero
 downstream patches),
 and without intermediate releases, it will get difficult.

If you consider *every* commit to be a release, then your life becomes
easier. This is just a case of bumping the SemVer patch version per commit
(as eloquently put by Jeremy).  We even have tooling to automate the
version generation via pbr..

Therefore, you might want to jump from X.X.100 to X.X.200 which would mean
100 commits since the last update.

 I'm personally not fond of this as it will lead to more fragmentation.
 It may encourage
 bad behaviors like shipping downstream patches for bug fixes and CVE
instead
 of collaborating upstream to differentiate themselves.
 For instance, if we had no point-based release, for issues tracking
 purposes, we would
 have to maintain our sets of tags somewhere.

I disagree, each distro already does security patching and whilst I expect
this to still happens, it actually *encourages* upstream first workflow as
you can select a release on your own cadence that includes commits you
need, for your users.

 There's also the release notes issue that has already been mentioned.
 Still continuous release notes won't solve the problem, as you wouldn't
 be able to map these to the actual packages. Will we require operators
 to find from which git commit, the packages were built and then try to
figure
 out which fixes are and are not included?

Can you provide more detail? I'm not understanding the problem.

  Long version:
 
  At the stable branch session in Vancouver we discussed recent
  evolutions in the stable team processes and how to further adapt the
  work of the team in a big tent world.
 
  One of the key questions there was whether we should continue doing
  stable point releases. Those were basically tags with the same version
  number (2015.1.1) that we would periodically push to the stable
  branches for all projects.
 
  Those create three problems.
 
  (1) Projects do not all follow the same versioning, so some projects
  (like Swift) were not part of the stable point releases. More and more
  projects are considering issuing intermediary releases (like Swift
  does), like Ironic. That would result in a variety of version numbers,
  and ultimately less and less projects being able to have a common
  2015.1.1-like version.
 

 And that's actually a pain point to track for these releases in which
 OpenStack branch belong. And this is probably something that needs to
 be resolved.

  (2) Producing those costs a non-trivial amount of effort on a very small
  team of volunteers, especially with projects caring about stable
  branches in various amounts. We were constantly missing the
  pre-announced dates on those ones. Looks like that effort could be
  better spent improving the stable branches themselves and keeping them
  working.
 

 Agreed, but why not switching to a time-based release?
 Regularly, we tag/generate/upload tarballs, this could even be automated.
 As far as I'm concerned, I would be more happy to have more frequent
releases.

  (3) The resulting stable point releases are mostly useless. Stable
  branches are supposed to be always usable, and the released version
  did not undergo significantly more testing. Issuing them actually
  discourages people from taking whatever point in stable branches makes
  the most sense for them, testing and deploying that.
 
  The suggestion we made during that session (and which was approved by
  the session participants) is therefore to just get rid of the stable
  point release concept altogether for non-libraries. That said:
 
  - we'd still do individual point releases for libraries (for critical
  bugs and security issues), so that you can still depend on a specific
  version there
 
  - we'd still very much maintain stable branches (and actually focus our
  efforts on that work) to ensure they are a continuous source of safe
  upgrades for users of a given series
 
  Now we realize that the cross-section of our community which was present
  in that session might not fully represent the consumers of those
  artifacts, which is why we expand the discussion on this mailing-list
  (and soon on the operators ML).
 

 Thanks, I was not able to join this discussion, and that was the kind
 of proposal
 that I was fearing to see happen.

  If you were a consumer of those and will miss them, please explain why.
  In particular, please let us know how consuming that version (which was
  only made available every n months) is significantly 

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Dave Walker]

On 29 May 2015 7:41 pm, Matt Riedemann mrie...@linux.vnet.ibm.com wrote:
SNIP

 This, IMO, is about the only time right now that I see doing point
releases on stable as worthwhile.  In other words, things have been very
touchy in stable for at least the last 6 months, so in the rare moments of
stability with the gate on stable is when I'd cut a release before the next
gate breaker.  You can get some examples of why here:
SNIP

I disagree this would help things, as every commit that lands the gate was
perfectly functional at that time by definition.

It is usually retrospection that the gate falls to pass, with the usual
case being changing of bound direct or indirect dependencies.

If we grab a recent point release tarball, and put it back through the gate
with the same declared requirement bounding - it will still fail (even
though it passed when the release was cut).

--
Kind Regards,
Dave Walker
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [pbr] regular releases, and path to 1.0

[Dave Walker]

On 4 May 2015 at 23:01, Jeremy Stanley fu...@yuggoth.org wrote:
 On 2015-05-05 07:53:20 +1200 (+1200), Robert Collins wrote:
 [...]
 release weekly
 [...]

 I'm fine with releasing weekly when there's something to release,
 but as PBR is somewhat stabilized and relatively tightly scoped I
 _hope_ that we get to the point where we don't have bugs or new
 features in PBR on a weekly basis.

 Cool with the rest of the proposal too.

Hey,

As someone that did track master PBR Master for internal cross-project
builds during the SemanticVersioning ping-pong, I have to agree that
having a core tool that should be pretty static in feature deliverable
be a regular blocker and instigator of build fails is a real pain.

I am not sure that weekly builds provide much in the way of value,
depending on the consumer of the library.  The release cadence would
be too short to really get value out of time base releases.  Is it
expected to assist openstack-infra in being able to plan upgrading?  I
can't see it helping distros or other vendors building derived
versions of OpenStack?  Would this mean that OpenStack projects would
have to start caring about PBR, or can they expected the core
pipelines to continue to work without knowledge of PBR's releases?
What version(s) would stable/* be expected to use?

For sake of argument, what does weekly provide that monthly doesn't?
Or in the opposite direction - why would each commit not be treated as
a release?

As a consumer of PBR, I stopped tracking master because I was
frustrated rebasing, and I had low confidence the next rebase wouldn't
break my entire pipeline in hidden or subtle ways.

The last change I made to PBR took 4 months to get approved, just
idling as unreviewed.  There is *nothing* more demotivating having a
changeset blocked in this status, particularly when it is a simplistic
change that is forcing you to use a derived version for internal usage
causing additional cost of rebasing.  So, what is happening with the
project now to make reviews happen soon enough to make frequent
time-based release useful?

Perhaps it would be useful to spell out some of the API breaking
changes you are planning?  It feels to me that PBR should be pretty
static in the near term... I am not convinced that frequent releases
make API breaking changes easier, as I am not sure a core library like
PBR can just support 1.0 and n-1 - so would each release keep support
for pbr's major and minor?

(PS. I really like PBR)

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [cinder] cinder support matrix, chap support?

[Dave Walker]

Hi,

Recently I have been curious as to which Cinder drivers support
authentication. It seems that only a subset do.  I wondered, is this
something that would be useful on the CinderSupportMatrix wiki page?

Thanks

--
Kind Regards,
Dave Walker
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [pbr] support for 'python setup.py install'

[Dave Walker]

I'm not going to pretend it is graceful... but is there a situation
where _ isn't correct?

$ cat setup.py
#!/usr/bin/env python

from distutils.core import setup as _setup
import os

def new_setup(**attrs):
if pip not in os.environ.get(_):
raise SystemExit(Please use pip to install)
else:
_setup(**attrs)

setup = new_setup

setup(name='foobar',
  version='1.0',
  description='Foobar',
 )


--
Kind Regards,
Dave Walker


On 25 April 2015 at 15:27, Monty Taylor mord...@inaugust.com wrote:
 On 04/25/2015 09:49 AM, Jeremy Stanley wrote:
 On 2015-04-25 12:12:15 +1200 (+1200), Robert Collins wrote:
 [...]
 I'd like to make that a little more official:
  - put it in our docs
  - stop testing python setup.py install.
 [...]

 And emit a clear error message? (Even if that just means updating the
 setup.py boilerplate in the cookiecutter repo and encouraging
 projects to adopt the function.)


 Unfortunately there is no way to detect that you're being run via
 setup.py install instead of pip. What will happen though is that
 easy_install will get triggered  instead of egg_info/pip install ... so
 we _could_ just do something to try to emit a warning around one of the
 easy_install classes ...

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Glance] IRC logging

[Dave Walker]

On 13 January 2015 at 12:32, Kuvaja, Erno kuv...@hp.com wrote:
 I'm heavily against the public logging to the level that I will just leave 
 the channel if that will be enabled. My point is not foul language and I do 
 understand that there could be some benefits out of it. Personally I think we 
 have enough tracked public communication means like ask.openstack.org and the 
 mailing lists. IRC is and has always been real time communications with 
 defined audience.

 I think the major benefits of this defined audience are:
 1) One does not need to express themselves in a way that is for public. ( 
 Misunderstandings can be corrected on the fly if needed. ) There is no need 
 to explain to anyone reading the logs what you actually meant during the 
 conversation month ago.
 2) there is level of confidentiality within that defined audience. ( For 
 example someone not familiar with the processes thinks they have found 
 security vulnerability and comes to the IRC-channel to ask second opinion. 
 Those details are not public and that bug can still be raised and dealt 
 properly. Once the discussion is logged and the logs are publicly available 
 the details are publicly available as well. )
 3) That defined audience does not usually limit content. I have no problem to 
 throw my e-mail address, phone number etc. into the channel, I would not yell 
 them out publicly.

 For me personally the last point is the biggest problem, professionally the 
 second is major concern. I have been using IRC for so long time that I'm not 
 willing to take the risk I can't filter myself on my regular channels. 
 Meetings are different story as there it is defined time and at least I'm on 
 meeting mode that time knowing it will be publicly logged.

 The channels are not locked so anyone can keep a client online and log it for 
 themselves if they feel need for it and lots of people do so. There is just 
 that big barrier having it within the defined group you can see on the 
 channel versus public to anyone.

 As opposed to Cindy's original statement of not wanting to be available 
 off-hours, that's solved already: you can set your client to away or not 
 respond. It's really common on any IRC network that nick is online while user 
 is not or is ignoring that real time outreach by personal preference. No-one 
 will/should take that personally or offensive. Not having bouncer/shell to 
 run your client is as well personal preference, I doubt anyone can claim they 
 could not do it with the options available nowadays.

  - Erno (jokke_) Kuvaja


Hi,

I think these concerns are more based around fear, than any real
merit.  I would suggest that any IRC communication should be treated
as public, and therefore the idea of bouncing around personal contacts
details is pretty poor personal security.  If this is required, then
using private messages would seem to be perfectly suitable.

A user can join any #openstack-* channel, and not necessarily be a
friend of the project.  The concerns about security issues should be
treated as if they have already become public.

It seems that Openstack currently has around 40 non-meeting channels
logged[0] and contrasting with the Ubuntu project, there are some 350
public logged channels[1] - with the logs going back to 2004.  This
has caused little issue over the years.

It would seem logical to introduce project-wide irc logging IMO.  I
*have* found it useful to search through archives of projects, and
find it frustrating when this data is not available.

I really struggle with the idea that contributors of a developer
channel do not consider themselves to be talking in a public forum,
which to me - is the same as being logged.  Without this mindset, the
channel (and project?) merely becomes a cabal developers area.

[0] http://eavesdrop.openstack.org/irclogs/
[1] http://irclogs.ubuntu.com/2015/01/01/

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [stable] Proposal to add Flavio Percoco to stable-maint-core

[Dave Walker]

+2

Flavio seems to have a good understanding of stable branch and has a
good history of reviews.

Thanks.

On 6 January 2015 at 19:32, Adam Gandelman ad...@ubuntu.com wrote:
 Hiya-

 Flavio has been actively involved in stable branch maintenance for as long
 as I can remember, but it looks like his +2 abilities were removed after the
 organizational changes made to the stable maintenance teams.  He has
 expressed interest in continuing on with general stable maintenance and I
 think his proven understanding of branch policies make him a valuable
 contributor. I propose we add him to the stable-maint-core team.

 Cheers,
 Adam

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Fixing the console.log grows forever bug.

[Dave Walker]

On 8 December 2014 at 10:33, Daniel P. Berrange berra...@redhat.com wrote:
 On Sat, Dec 06, 2014 at 04:38:52PM +1100, Tony Breeds wrote:
 Hi All,
 In the most recent team meeting we briefly discussed: [1] where the
 console.log grows indefinitely, eventually causing guest stalls.  I mentioned
 that I was working on a spec to fix this issue.

 My original plan was fairly similar to [2]  In that we'd switch libvirt/qemu 
 to
 using a unix domain socket and write a simple helper to read from that socket
 and write to disk.  That helper would close and reopen the on disk file upon
 receiving a HUP (so logrotate just works).   Life would be good. and we could
 all move on.

 However I was encouraged to investigate fixing this in qemu, such that qemu
 could process the HUP and make life better for all.  This is certainly doable
 and I'm happy[3] to do this work.  I've floated the idea past qemu-devel and
 they seem okay with the idea.  My main concern is in lag and supporting
 qemu/libvirt that can't handle this option.

 As mentioned in my reply on qemu-devel, I think the right long term solution
 for this is to fix it in libvirt. We have a general security goal to remove
 QEMU's ability to open any files whatsoever, instead having it receive all
 host resources as pre-opened file descriptors from libvirt. So what we
 anticipate is a new libvirt daemon for processing logs, virtlogd. Anywhere
 where QEMU currently gets a file to log to (serial devices, and its
 stdout/stderr), it would instead be given a FD that's connected to virtlogd.
 virtlogd would simply write the data out to file  would be able to close
  re-open files to integrate with logrotate.

 For the sake of discussion  I'll lay out my best guess right now on fixing 
 this
 in qemu.

 qemu 2.2.0 /should/ release this year the ETA is 2014-12-09[4] so the fix I'm
 proposing would be available in qemu 2.3.0 which I think will be available in
 June/July 2015.  So we'd be into 'L' development before this fix is available
 and possibly 'M' before the community distros (Fedora and  Ubuntu)[5] include
 and almost certainly longer for Enterprise distros.  Along with the qemu
 development I expect there to be some libvirt development as well but right 
 now
 I don't think that's critical to the feature or this discussion.

 So if that timeline is approximately correct:

 - Can we wait this long to fix the bug?  As opposed to having it squashed in 
 Kilo.
 - What do we do in nova for the next ~12 months while know there isn't a 
 qemu to fix this?
 - Then once there is a qemu that fixes the issue, do we just say 'thou must 
 use
   qemu 2.3.0' or would nova still need to support old and new qemu's ?

 FWIW, by comparison libvirt is on a monthly release schedule, so a fix done in
 libvirt has potential to be available sooner, though obviously there's bigger
 dev work to be done in libvirt for this.

 Regards,
 Daniel

Hey,

This thread started by suggesting having a scheduled task to read from
a unix socket.  I don't think this can really be considered an
acceptable fix, as the guest does indeed lock up when the buffer is
full.

Initially, I proposed a quick fix for this back in 2011 which provided
a config option to enable a kernel level ring buffer via a
non-mainline module called emlog.  This was not merged for
understandable reasons.  (pre gerrit) -
http://bazaar.launchpad.net/~davewalker/nova/832507_with_emlog/revision/1509/nova/virt/libvirt/connection.py

Later that same year, Robie Basak presented a change which introduced
similar logic ringbuffer support in the nova code itself making use of
eventlet. This seems quite a reasonable fix, but there was concern it
might lock-up guests.. https://review.openstack.org/#/c/706/

I think shortly after this, it was pretty widely agreed that fixing
this in Nova is not the correct layer.  Personally, I struggle
thinking qemu or libvirt is right layer either.  I can't think that
treating a console as a flat log file is the best default behavior.

I still quite like the emlog approach, as having a ringbuffer device
type in the kernel provides exactly what we need and is pretty simple
to implement.

Does anyone know if this generic ringbuffer kernel support was
proposed to mainline kernel?

--
Kind Regards,
Dave Walker

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] testtools 1.2.0 release breaks the world

[Dave Walker]

On 15 November 2014 14:25, Sean Dague s...@dague.net wrote:
 Testtools 1.2.0 release apparently broke subunit.run discover --list
 which breaks the way that tempest calls the test chain. No tempest runs
 have passed since it's release.

 https://review.openstack.org/#/c/134705/ is a requirements pin, though I
 think because of grenade this is actually going to have to be laddered
 up from icehouse = juno = master.

 https://review.openstack.org/#/q/I2f8737b44c703c3094d6bbb6580993f86a571934,n,z

 It's probably half a day babysitting getting all the pins in place to
 make the world work again. I'm offline from here on out for the weekend,
 but I'll put a +2/+A on all of these so if someone wants to recheck them
 in the right order to land them, they can get things fixed.

 Also... lets try not to release libraries on Fridays before disappearing
 for the weekend... please. Pretty please.

 -Sean

 --
 Sean Dague
 http://dague.net

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


This change has landed in stable/icehouse but...

stable/juno is currently blocked by oslo.messaging Master which
introduces oslo.middleware, which is not part of stable/juno.  I have
added pinning to stable/juno global-requirements via
https://review.openstack.org/#/c/134727/

This should unblock stable/juno, allowing this to progress.

Please be reviewing :)

--
Kind Regards,
Dave Walker

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] testtools 1.2.0 release breaks the world

[Dave Walker]

On 15 November 2014 19:51, Robert Collins robe...@robertcollins.net wrote:
 It probably needs to be backed out of stable/icehouse. The issue is
 that we were installing unittest2 via distro packages *and* testtools
 new dependency on unittest2 did not express a minimum version.

 We're just about to issue 1.2.1 which will have such a minimum version.

 And for the record, this was released on saturday, not friday :).

 -Rob

We've discussed this on IRC, but I fail to see how this is an
appropriate fix for stable/* . This would mean introducing new minima
to stable branches, which for stable branches is totally
inappropriate.

This causes the effect of both distros and deployers requiring a
higher version which they have not planned for.  If we pin
oslo.messaging (which is what we started agreeing in Paris), this
problems goes away.

--
Kind Regards,
Dave Walker

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] testtools 1.2.0 release breaks the world

[Dave Walker]

On 15 November 2014 20:23, Robert Collins robe...@robertcollins.net wrote:
 On 16 November 2014 09:03, Dave Walker em...@daviey.com wrote:
 On 15 November 2014 19:51, Robert Collins robe...@robertcollins.net wrote:
 It probably needs to be backed out of stable/icehouse. The issue is
 that we were installing unittest2 via distro packages *and* testtools
 new dependency on unittest2 did not express a minimum version.

 We're just about to issue 1.2.1 which will have such a minimum version.

 And for the record, this was released on saturday, not friday :).

 -Rob

 We've discussed this on IRC, but I fail to see how this is an
 appropriate fix for stable/* . This would mean introducing new minima
 to stable branches, which for stable branches is totally
 inappropriate.

 This causes the effect of both distros and deployers requiring a
 higher version which they have not planned for.  If we pin
 oslo.messaging (which is what we started agreeing in Paris), this
 problems goes away.

 Huh? I think you're working a different bug, no? The testtools thing
 is what I thought this thread was about :) The
 oslo.messaging/middleware thing is separate but both are happening at
 the same time on juno/stable. I was only referring to backing out a
 pin of testtools - which is needed because with single-version-tempest
 we can't pin any of tempests dependencies in just stable, we have to
 have the pins be consistent across all versions tested by tempest
 (with the current install strategy).

 -Rob

You are right, I accidently folded two issues into 1.  However, I do
not understand how we can resolve this issue the way you have outlined
without introducing a new minimum version on unittest2, which was not
previously a requirement on stable/*.

This surely has the same effect that I outlined?

Thanks

--
Kind Regards,
Dave Wa;ler

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] testtools 1.2.0 release breaks the world

[Dave Walker]

On 15 November 2014 21:22, Jeremy Stanley fu...@yuggoth.org wrote:
 On 2014-11-15 20:38:02 + (+), Dave Walker wrote:
 You are right, I accidently folded two issues into 1.  However, I do
 not understand how we can resolve this issue the way you have outlined
 without introducing a new minimum version on unittest2, which was not
 previously a requirement on stable/*.

 This surely has the same effect that I outlined?

 You only need a new unittest2 if you're using a new testtools. The
 argument that we're introducing a newer requirement there is
 somewhat circular. If you're installing with distro packages of
 testtools and unittest2 then presumably your distro has pre-selected
 versions of them which are known to interoperate?
 --
 Jeremy Stanley

Ah, Good Point.  I (wrongly?) assumed we were looking to put a minimum
version of unittest2 in requirements.  Which would cause this
undesired behaviour.  However, that doesn't need to be the case.  I
assume with this approach we WILL put an upperbound on unittest2 in
stable/* requirements?

If so - my point is mute. Pah.

--
Kind Regards,
Dave Walker

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [stable] New config options, no default change

[Dave Walker]

Hi,

Looking at a stable/juno cinder proposed change[0], I came across one
that introduces a new config option.

The default is a noop change for the behaviour, so no bad surprises on upgrade.

These sort of changes feel like they are outside the 'no config
changes' rule, but we have not really discussed this.

What do others think?

Thanks

[0] https://review.openstack.org/#/c/131987/

--
Kind Regards,
Dave Walker

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Add Modified Compute Driver to Nova

[Dave Walker]

On 10 Nov 2014 08:44, Mohammad Hosein Zarei hosein.za...@gmail.com
wrote:

 ​​
 Hi everyone
 I have a modified compute driver (my_driver.py and my_driver.pyc) and I
copied files to /opt/stack/nova/nova/virt/libvirt/ directory.
 Also, nova.conf modified to use this driver
(compute_driver=libvirt.my_driver.MyDriver).
 I can't start nova-compute with this modified driver and nova.conf. When
nova-compute start , it shows this error:
SNIP

Hi Mohammad,

Without providing a diff or at least further details on your changes you
are really tying our hands.

Not sure there is much we can do without this information.

Thanks

--
Kind Regards,
Dave Walker
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Keystone] external AuthN Identity Backend

[Dave Walker]

Hi,

Currently we have two ways of doing Identity Auth backends, these are
sql and ldap.

The SQL backend is the default and is for situations where Keyston is
the canonical Identity provider with username / password being
directly compared to the Keystone database.

LDAP is the current option if Keystone isn't the canonical Identity
provider and passes the username and password to an LDAP server for
comparison and retrieves the groups.

For a few releases we have supported External auth (or Kerberos),
where we authenticate the user at the edge and trust the REMOTE_USER
is valid.  In these situations Keystone doesn't require the Username
or Password to be valid.

Particularly in Kerberos situations, no password is used to
successfully authenticate at the edge.  This works well, but LDAP
cannot be used as no password is passed through.  The other option is
SQL, but that then requires a user to be created in Keystone first.

We do not seem to cover the situation where Identity is provided by an
external mechanism.  The only system currently available is Federation
via SAML, which isn't always the best fit.

Therefore, I'd like to suggest the introduction of a third backend.
This would be the external identity provider.  This would seem to be
pretty simple, as the current checks would simply return success (as
we trust auth at the edge), and not store user_id in the database, but
generate it at runtime.

The issue I have, is that this doesn't cover Group membership.

So, am I a:
 - Barking totally up the wrong tree
 - Add support to the current LDAP plugin to support external auth
(but still use LDAP for groups)
 - Write a standalone external plugin, but then do what for Groups?  I
would be reasonably happy to just have 1:1 mapping of users to groups.

Does this make sense?

Thanks

--
Kind Regards,
Daviey Walker

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] external AuthN Identity Backend

[Dave Walker]

Hi Steve,

Thanks for your response.  I am talking generally about the external
auth support.  One use case is Kerberos, but for the sake of argument
this could quite easily be Apache Basic auth.  The point is, we have
current support for entrusting AuthN outside of Keystone.

What I was trying to outline is that it seems that the current design
of external auth is that keystone is not in the auth pipeline as we
trust auth at the edge.  However, we then do additional auth within
keystone.

With external auth and SQL, we drop the user provided username and
password on the floor and use what was provided in REMOTE_USER (set by
the webserver).

Therefore the check as it currently stands in SQL is basically 'is
this username in the database'.  The LDAP plugin does Authentication
via username and password, which is clearly not sufficient for
external auth.  The LDAP plugin could be made to check in a similar
manner to SQL 'is this a valid user' - but this would seem to be a
duplicate check, as we already did this at the edge.

If the webserver granted access to keystone, the user has already been
checked to see if they are a valid user.  However, your response seems
to suggest that current external auth should be formally deprecated?

--
Kind Regards,
Daviey Walker

On 16 October 2014 19:28, Steve Martinelli steve...@ca.ibm.com wrote:
 I think it depends on what you mean by 'external identity provider' - I
 assume it's capable of speaking some sort of federation protocol. So I'd
 rather explore adding more support for additional federation
 protocols/standards, rather than making our own third backend.

 Steve

 Dave Walker em...@daviey.com wrote on 10/16/2014 02:15:07 PM:

 From: Dave Walker em...@daviey.com
 To: OpenStack Development Mailing List
 openstack-dev@lists.openstack.org,
 Date: 10/16/2014 02:20 PM
 Subject: [openstack-dev] [Keystone] external AuthN Identity Backend

 Hi,

 Currently we have two ways of doing Identity Auth backends, these are
 sql and ldap.

 The SQL backend is the default and is for situations where Keyston is
 the canonical Identity provider with username / password being
 directly compared to the Keystone database.

 LDAP is the current option if Keystone isn't the canonical Identity
 provider and passes the username and password to an LDAP server for
 comparison and retrieves the groups.

 For a few releases we have supported External auth (or Kerberos),
 where we authenticate the user at the edge and trust the REMOTE_USER
 is valid.  In these situations Keystone doesn't require the Username
 or Password to be valid.

 Particularly in Kerberos situations, no password is used to
 successfully authenticate at the edge.  This works well, but LDAP
 cannot be used as no password is passed through.  The other option is
 SQL, but that then requires a user to be created in Keystone first.

 We do not seem to cover the situation where Identity is provided by an
 external mechanism.  The only system currently available is Federation
 via SAML, which isn't always the best fit.

 Therefore, I'd like to suggest the introduction of a third backend.
 This would be the external identity provider.  This would seem to be
 pretty simple, as the current checks would simply return success (as
 we trust auth at the edge), and not store user_id in the database, but
 generate it at runtime.

 The issue I have, is that this doesn't cover Group membership.

 So, am I a:
  - Barking totally up the wrong tree
  - Add support to the current LDAP plugin to support external auth
 (but still use LDAP for groups)
  - Write a standalone external plugin, but then do what for Groups?  I
 would be reasonably happy to just have 1:1 mapping of users to groups.

 Does this make sense?

 Thanks

 --
 Kind Regards,
 Daviey Walker

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] external AuthN Identity Backend

[Dave Walker]

On 16 October 2014 20:07, David Stanek dsta...@dstanek.com wrote:
SNIP
 I may be missing something, but can you use the external auth method with
 the LDAP backend?


No, as the purpose of the LDAP backend is to validate user/pass
combination are valid.  With the external auth plugin, these are not
provided to keystone (and may not even exist).  If they did exist, we
would be doing auth at the edge and at the backend - which seems
needlessly expensive.

--
Kind Regards,
Daviey Walker

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] external AuthN Identity Backend

[Dave Walker]

Hi,

I think I considered the Federated plugin as a mismatch as it dealt
with 'remote' auth rather than 'external' auth.  I thought it was for
purely handling SSO / SAML2, and not being subordinate to auth with
the webserver.

I'll dig into the federation plugin more, thanks.

--
Kind Regards,
Dave Walker

On 16 October 2014 19:58, David Chadwick d.w.chadw...@kent.ac.uk wrote:
 Dave

 when federation is used, the user's group is stored in a mapping rule.
 So we do have a mechanism for storing group memberships without using
 LDAP or creating an entry for the user in the SQL backend. (The only
 time this is kinda not true is if we have a specific rule for each
 federated user, so that then each mapping rule is equivalent to an entry
 for each user). But usually we might expect many users to use the same
 mapping rule.

 Mapping rules should be usable for Kerberos logins. I dont know if the
 current code does have this ability or not, but if it doesn't, then it
 should be re-engineered to. (it was always in my design that all remote
 logins should have a mapping capability)

 regards

 David

 On 16/10/2014 19:15, Dave Walker wrote:
 Hi,

 Currently we have two ways of doing Identity Auth backends, these are
 sql and ldap.

 The SQL backend is the default and is for situations where Keyston is
 the canonical Identity provider with username / password being
 directly compared to the Keystone database.

 LDAP is the current option if Keystone isn't the canonical Identity
 provider and passes the username and password to an LDAP server for
 comparison and retrieves the groups.

 For a few releases we have supported External auth (or Kerberos),
 where we authenticate the user at the edge and trust the REMOTE_USER
 is valid.  In these situations Keystone doesn't require the Username
 or Password to be valid.

 Particularly in Kerberos situations, no password is used to
 successfully authenticate at the edge.  This works well, but LDAP
 cannot be used as no password is passed through.  The other option is
 SQL, but that then requires a user to be created in Keystone first.

 We do not seem to cover the situation where Identity is provided by an
 external mechanism.  The only system currently available is Federation
 via SAML, which isn't always the best fit.

 Therefore, I'd like to suggest the introduction of a third backend.
 This would be the external identity provider.  This would seem to be
 pretty simple, as the current checks would simply return success (as
 we trust auth at the edge), and not store user_id in the database, but
 generate it at runtime.

 The issue I have, is that this doesn't cover Group membership.

 So, am I a:
  - Barking totally up the wrong tree
  - Add support to the current LDAP plugin to support external auth
 (but still use LDAP for groups)
  - Write a standalone external plugin, but then do what for Groups?  I
 would be reasonably happy to just have 1:1 mapping of users to groups.

 Does this make sense?

 Thanks

 --
 Kind Regards,
 Daviey Walker

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] icehouse failure rates are somewhat catastrophic - who is actually maintaining it?

[Dave Walker]

On 2 Oct 2014 08:19, Alan Pevec ape...@gmail.com wrote:

  The original idea was that these stable branches would be maintained by
the
  distros, and that is clearly not happening if you look at the code
review

 Stable branches are maintained by the _upstream_ stable-maint team[1]
 where most members might be from (two) distros but please note that
 all PTLs are also included and there are members who are not from a
 distro.
 But you're right, if this stays mostly one distro effort, we'll pull
 out and do it on our own.
 /me looks at other non-named distros

  latency there. We need to sort that out before we even consider
supporting a
  release for more than the one year we currently do.

 Please consider that stable branches are also needed for the security
 fixes and we, as a responsible upstream project, need to provide that
 with or without distros. Stable branch was a master branch just few
 months ago and it inherited all the bugs present there, so everybody
 fixing a gate bug on master should consider backporting to stable at
 the same time. It can't be stable-maint-only responsiblity e.g.
 stable-maint doesn't have +2 in devstack stable/* or in tempest (now
 brancheless, so master) branches.

 Cheers,
 Alan

 [1] https://review.openstack.org/#/admin/groups/120,members


Hey,

When I initially proposed the concept of stable branches, it was indeed
targeted as a collaborative distro effort.

It became clear in the summit session that there was not just shared
interest from distros, but vendors and large consumers.

It was /not/ something that I envisaged would become a project
responsibility, just an area for the various types of consumer to
collaborate, rather than duplicating effort in downstreams.. Most likely
missing pretty important stability patches.

I didn't want dedicated point releases, just an always stable area where
consumers could pull/rebase from. This idea pretty much changed, and
vendors wanted a stamped-point release to make the situation clearer to
their users.

I think everyone would agree that the project and scope has grown pretty
significantly since the early days, and I agree that there does need to be
project-wide share of the burden of keeping the stable branches maintained,
with stable-maint becoming the driver. It can only scale if there is
sustained interest from each project.

I do not think it *can* now work with a small team of generalists, without
support from SME of projects.

I am pretty nervous of the point you make about Red Hat taking their ball
and going home if more distros don't commit to more effort. This is pretty
simply not the way to encourage more participation.

Sadly, the git pull stats cannot be public.. But I am pretty sure that a
reasonably large consumer-base slurp up the branches directly. If this is
true, then it is clear that the project has a responsibility to users..
Therefore, the quick fire point of talking about stable branch ongoing
feasibility is a bit rash.  The general project clearly isn't ready for
rolling release, so we need to talk about how we can make this work.

I have been absent from the stable-maint effort for the last year, but have
been tracking the stable mailing list.

This feels like the first credible 'we are struggling' that has been raised
- I actually believed it was reasonably healthy. It does seem that this
issue has been brewing for a while.

Therefore, I think we need to do a better effort of tracking weak areas in
the process. We do not have a decent TODO list.

Tracking what needs to be done, allows better granular sharing of the
burden.

This is not a problem of looking at gerrit stable/* open reviews but bugs
in the process.

Is the issue mostly about changing dependencies upper versions?

Should we consider whitelisting updated dependencies in requirements.txt,
rather than blacklisting/racing to backport a fix?

Are enough patchsets proposed from current Master?

Are project core's routinely asking themselves if a patchset should be
backported?

Are we tracking open-bugs on Master well enough as also affecting stable
releases?

I do not think we are struggling primarily with technical issues, but
procedural issues.

I hope we are all agreed we /need/ something. Let's talk about 'what' and
'how', rather than 'if'.

[I will look to be more involved with stable this cycle.]

--
Kind Regards,
Dave Walker
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Nova] Libvirt not migrating cdrom

[Dave Walker]

Hi,

Kicking open an old thread[0] about Libvirt not migrating cdrom
devices (config-drive) [0] with the attached LP bug[1].

It seems that the direction was to consider switching to vfat, as
libvirt supports this.  It isn't clear to me if the cdrom limitation
is specific to libvirt, nor if vfat could be made to work in windows
(seemed to imply there was a limitation).

I wanted to check if the reasoning for libvirt not allowing cdrom
migration had been considered?  Is it that libvirt blocks it, as it
'could' be a physical cdrom - rather than an iso?

It feels to me that pushing the fix down the stack into libvirt seems
like the correct solution?

[0] http://lists.openstack.org/pipermail/openstack-dev/2014-February/027394.html
[1] https://bugs.launchpad.net/nova/+bug/1246201

--
Kind Regards,
Dave Walker

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] HTTP Get and HEAD requests mismatch on resulting HTTP status (proposed REST API Response Status Changes)

[Dave Walker]

Hi,

This is very similar to an issue I encountered with Glance.  For some
unknown reason, we were adding a Location header for 200 responses.

When served behind apache+mod_fcgid, the module saw the Location
header and has a hard coded conversion to 302 Redirect.  This caused
glanceclient to follow the redirect loop continually.  As you can
imagine, the stealthy changing was a real oddity to debug.

More details are here:
https://bugs.launchpad.net/glance/+bug/1299095

Standardisation with standards helps avoid non-standard behaviour. :)

--
Kind Regards,
Dave Walker


On 2 July 2014 03:48, Robert Collins robe...@robertcollins.net wrote:
 Wearing my HTTP fanatic hat - I think this is actually an important
 change to do. Skew like this can cause all sorts of odd behaviours in
 client libraries.

 -Rob

 On 2 July 2014 13:13, Morgan Fainberg morgan.fainb...@gmail.com wrote:
 In the endeavor to move from the default deployment of Keystone being 
 eventlet (in devstack) to Apache + mod_wsgi, I noticed that there was an odd 
 mis-match on a single set of tempest tests relating to trusts. Under 
 eventlet a HTTP 204 No Content was being returned, but under mod_wsgi an 
 HTTP 200 OK was being returned. After some investigation it turned out that 
 in some cases mod_wsgi will rewrite HEAD requests to GET requests under the 
 hood; this is to ensure that the response from Apache is properly built when 
 dealing with filtering. A number of wsgi applications just return nothing on 
 a HEAD request, which is incorrect, so mod_wsgi tries to compensate.

 The HTTP spec states: The HEAD method is identical to GET except that the 
 server must not return any Entity-Body in the response. The metainformation 
 contained in the HTTP headers in response to a HEAD request should be 
 identical to the information sent in response to a GET request. This method 
 can be used for obtaining metainformation about the resource identified by 
 the Request-URI without transferring the Entity-Body itself. This method is 
 often used for testing hypertext links for validity, accessibility, and 
 recent modification.”

 Keystone has 3 Routes where HEAD will result in a 204 and GET will result in 
 a 200.

 * /v3/auth/tokens
 * /v2.0/tokens/{token_id}
 * /OS-TRUST/trusts/{trust_id}/roles/{role_id} --- This is the only one 
 tested by Tempest.

 The easiest solution is to correct the case where we are out of line with 
 the HTTP spec and ensure these cases return the same status code for GET and 
 HEAD methods. This however changes the response status of a public REST API. 
 Before we do this, I wanted to ensure the community, developers, and TC did 
 not have an issue with this correction. We are not changing the class of 
 status (e.g. 2xx to 4xx or vice-versa). This would simply be returning the 
 same response between GET and HEAD requests. The fix for this would be to 
 modify the 3 tempest tests in question to expect HTTP 200 instead of 204.

 There are a couple of other cases where Keystone registers a HEAD route but 
 no GET route (these would be corrected at the same time, to ensure 
 compatibility). The final correction is to enforce that Keystone would not 
 send any data on HEAD requests (it is possible to do so, but we have not had 
 it happen).

 You can see a proof-of-concept review that shows the tempest failures here: 
 https://review.openstack.org/#/c/104026

 If this change (even though it is in violation of 
 https://wiki.openstack.org/wiki/APIChangeGuidelines#Generally_Not_Acceptable 
 is acceptable, it will unblock the last of a very few things to have 
 Keystone default deploy via devstack under Apache (and gate upon it). Please 
 let me know if anyone has significant issues with this change / concerns as 
 I would like to finish up this road to mod_wsgi based Keystone as early in 
 the Juno cycle as possible.

 Cheers,
 Morgan Fainberg


 —
 Morgan Fainberg



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 --
 Robert Collins rbtcoll...@hp.com
 Distinguished Technologist
 HP Converged Cloud

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev