date:20160721


On 7/7/2016 8:05 AM, Eric Harney wrote:

On 07/07/2016 02:41 AM, Chen CH Ji wrote:

Hi,
   I am backporting https://review.openstack.org/#/c/333749/

   to stable/liberty and failed in gating job, then I submitted another
doc change https://review.openstack.org/#/c/338699/  to verify and seems failed
with same reason and I have no idea what's wrong in test... can someone help to
take a look or give some hint?
error is :


ft17.1: 
cinder.tests.unit.api.contrib.test_quotas.QuotaSetsControllerTest.test_delete_StringException:
 Empty attachments:
   pythonlogging:''
   stderr
   stdout

Traceback (most recent call last):
   File "cinder/tests/unit/api/contrib/test_quotas.py", line 100, in setUp
 self.fixture = self.useFixture(config_fixture.Config(auth_token.CONF))
AttributeError: 'module' object has no attribute 'CONF'





Yes, it looks like it.

I filed bug https://bugs.launchpad.net/cinder/+bug/1599855 for this.

It looks like the way we are setting keystonemiddleware options breaks
with newer keystonemiddleware releases.

keystonemiddleware 2.6.0 works here, 4.6.0 does not.

Thanks for the report.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



Cinder's tox.ini on stable/liberty isn't using upper-constraints, it 
should be and will fix the problem.


So cherry-pick https://review.openstack.org/#/c/312438/ and fix the 
branch used in the tox.ini and you should be good.


--

Thanks,

Matt Riedemann


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] Floating IP pool public not found

2016-07-21 Thread ????????

hi everyone,

when create node group template with sahara 4.0.0 in M version,an error occur:


$  openstack dataprocessing node group template create --json 
my_master_template_create_default.json
Floating IP pool public not found
Error ID: 62672c58-8fdd-40d9-8e6b-8aef0f6ab554



less my_master_template_create_default.json 
{
"plugin_name": "vanilla",
"hadoop_version": "2.7.1",
"node_processes": [
"namenode",
"resourcemanager"
],
"name": "vanilla-default-master",
"floating_ip_pool": "public",
"flavor_id": "6",
"auto_security_group": false
}



in sahara.log:
2016-07-21 21:40:05.216 1521 DEBUG keystoneclient.session 
[req-ee9a5b0d-b19a-418f-bdd3-018fc4634eb0 7fff70fbbf83441a9b3c4d91a5613825 
6cb156a82d0f486a9f50132be9438eb6 - - -] REQ: curl -g -i --insecure -X GET 
http://10.0.0.132:8774/v2/6cb156a82d0f486a9f50132be9438eb6/os-networks -H 
"User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: 
{SHA1}5971da83434fe662c7726b695a4007128c1dc383" _http_log_request 
/usr/lib/python2.7/site-packages/keystoneclient/session.py:206
2016-07-21 21:40:05.552 1521 DEBUG keystoneclient.session 
[req-ee9a5b0d-b19a-418f-bdd3-018fc4634eb0 7fff70fbbf83441a9b3c4d91a5613825 
6cb156a82d0f486a9f50132be9438eb6 - - -] RESP: [200] Date: Fri, 22 Jul 2016 
01:40:05 GMT Connection: keep-alive Content-Type: application/json 
Content-Length: 1297 X-Compute-Request-Id: 
req-3279a048-145e-4f8e-bb7b-f77d22037f2a 
RESP BODY: {"networks": [{"bridge": null, "vpn_public_port": null, 
"dhcp_start": null, "bridge_interface": null, "share_address": null, 
"updated_at": null, "id": "9c597886-d439-4fe2-b76b-5338d85aaf37", "cidr_v6": 
null, "deleted_at": null, "gateway": null, "rxtx_base": null, "label": 
"public", "priority": null, "project_id": null, "vpn_private_address": null, 
"deleted": null, "vlan": null, "broadcast": null, "netmask": null, "injected": 
null, "cidr": null, "vpn_public_address": null, "multi_host": null, 
"enable_dhcp": null, "dns2": null, "created_at": null, "host": null, "mtu": 
null, "gateway_v6": null, "netmask_v6": null, "dhcp_server": null, "dns1": 
null}, {"bridge": null, "vpn_public_port": null, "dhcp_start": null, 
"bridge_interface": null, "share_address": null, "updated_at": null, "id": 
"3aaa392b-af4d-4f70-9e2e-1b71a965ff7d", "cidr_v6": null, "deleted_at": null, 
"gateway": null, "rxtx_base": null, "label": "private", "priority": null, 
"project_id": null, "vpn_private_address": null, "deleted": null, "vlan": null, 
"broadcast": null, "netmask": null, "injected": null, "cidr": null, 
"vpn_public_address": null, "multi_host": null, "enable_dhcp": null, "dns2": 
null, "created_at": null, "host": null, "mtu": null, "gateway_v6": null, 
"netmask_v6": null, "dhcp_server": null, "dns1": null}]}
 _http_log_response 
/usr/lib/python2.7/site-packages/keystoneclient/session.py:231
2016-07-21 21:40:05.617 1521 DEBUG sahara.utils.openstack.base 
[req-ee9a5b0d-b19a-418f-bdd3-018fc4634eb0 7fff70fbbf83441a9b3c4d91a5613825 
6cb156a82d0f486a9f50132be9438eb6 - - -] Permanent error occurred during "find" 
execution: No Network matching {'id': u'public'}. (HTTP 404). 
execute_with_retries 
/usr/lib/python2.7/site-packages/sahara/utils/openstack/base.py:99
2016-07-21 21:40:05.670 1521 ERROR sahara.utils.api 
[req-ee9a5b0d-b19a-418f-bdd3-018fc4634eb0 7fff70fbbf83441a9b3c4d91a5613825 
6cb156a82d0f486a9f50132be9438eb6 - - -] Validation Error occurred: 
error_code=400, error_message=Floating IP pool public not found
Error ID: e369a75d-bf03-4de6-8192-23959967f45a, error_name=NOT_FOUND





with:
$  neutron net-list
+--+-+--+
| id   | name| subnets  
|
+--+-+--+
| 3aaa392b-af4d-4f70-9e2e-1b71a965ff7d | private | 
7cd8341e-9994-4a1d-a46f-22aedfde7bd8 10.0.0.0/24 |
| 9c597886-d439-4fe2-b76b-5338d85aaf37 | public  | 
0d7a6c95-3f6a-42ad-9cf8-c96e1d9c3c82 172.24.4.224/28 |
+--+-+--+



$  neutron subnet-list
+--++-+--+
| id   | name   | cidr| 
allocation_pools |
+--++-+--+
| 7cd8341e-9994-4a1d-a46f-22aedfde7bd8 | private_subnet | 10.0.0.0/24 | 
{"start": "10.0.0.2", "end": "10.0.0.254"}   |
| 0d7a6c95-3f6a-42ad-9cf8-c96e1d9c3c82 | public_subnet  | 172.24.4.224/28 | 
{"start": "172.24.4.226", "end": "172.24.4.238"} |

[openstack-dev] Running CI jobs on Ubuntu Xenial by default

2016-07-21 Thread Clark Boylan

Hello,

Recently we added python3.5 jobs that run on Ubuntu Xenial and switched
the PyPy jobs to run on Ubuntu Xenial. The next step in this process is
to switch the "python-jobs" and their database siblings to run on Ubuntu
Xenial by default as well. We will be doing this Monday July 25, 2016.
This should get the ball rolling on our switch for the remaining jobs as
well. Expect devstack/tempest jobs to also get this treatment next week.
This switch will only be for >= Newton (master today), Liberty and
Mitaka will continue to run on Ubuntu Trusty by default.

In preparation for this switch I have personally run the pep8, docs,
python27, and releasenotes tox envs for all projects that use the Zuul
python-jobs and python-db-jobs templates. This shook out a couple issues
that I have corrected and others I filed bugs for. While this isn't a
complete test for all projects using one of these jobs it does give a
good cross section so I am pretty confident that any issues will be
minor.

Known Issues:
* Mysql is updated to 5.7 and there are some user management and client
changes. SQLAlchemy/oslodb seem to handle this just fine though so most
projects likely won't notice.
* SSLv3 is gone gone gone. Python does not include SSLv3 on Xenial
because it is removed from the underlying SSL lib(s). If your project
has explicit support for SSLv3 you will need to address this. Keep in
mind that SSLv3 is not considered secure any more.

Why you should be excited for this:
* New versions of all the things means new features (but also likely new
bugs
* "Secure" version of python2.7. Those annoying warnings from requests
are gone.
* Modern(ish) 4.4 Linux Kernel
* Libvirt 1.3.1
* OVS 2.5
* Regardless of your opinion on Systemd this means all testing against
master will be on distros that use Systemd greatly simplifying process
management

Please let the infra team know if you have any questions,

Clark

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] Making TripleO CI easier to consume outside of TripleO CI

2016-07-21 Thread Wesley Hayutin

On Thu, Jul 21, 2016 at 5:47 PM, James Slagle 
wrote:

> On Tue, Jul 19, 2016 at 5:15 PM, Wesley Hayutin 
> wrote:
> >
> >
> > On Tue, Jul 19, 2016 at 2:44 PM, James Slagle 
> > wrote:
> >> Part of the goal of tripleo.sh was to mirror the commands in the
> >> documentation...that the same commands in the docs are in tripleo.sh.
> >> I know that has somewhat failed, but it was the goal anyway.
> >>
> >> Do you think integrating ansible into tripleo-ci changes that at all?
> >> tripleo-ci would be using ansible in some places, which in turn runs
> >> the commands (or their equivalent) that we actually document. Is the
> >> documentation still showing the same commands it does now, or is it
> >> showing running ansible as tripleo-ci would be doing?
> >
> >
> > Harry Rybacki and I are working on this now.  I think we have something
> that
> > is reasonable for when shell can be used and when ansible modules are
> > required.  I think he can make all this work public and everyone in
> TripleO
> > can keep tabs on the progress.
>
> Yes, I saw his email shortly after sending my reply. There is a lot to
> digest there, but it sounds promising. Perhaps we could start with
> something small and iteratively consume the generated docs as well. We
> could replace the manual docs over time by including sphinx docs at
> the right places that were automatically generated.
>

Agree,
He has a small example project where I think you could track the work and
progress.
https://github.com/HarryRybacki/tripleo-documentor

We're hoping to have the undercloud and overcloud steps templated and built
in the very near future.


> >
> >>
> >>
> >> I think I'm mostly in agreement with your #2 proposal, perhaps with
> >> the exception of having to rely on external roles. I don't think I
> >> would want to see tripleo-ci rely on ansible roles from a
> >> redhat-openstack organization on github.
> >>
> >> I know that we already have a lot of external dependencies in TripleO,
> >> and that not everything has to come from git.openstack.org. However, I
> >> think that we'd want to make the "source" for tripleo-ci be owned by
> >> the TripleO project and hosted by OpenStack infrastructure as much as
> >> possible. tripleo-quickstart already is, so I think it would be fine
> >> to start proposing some changes to tripleo-ci that use
> >> tripleo-quickstart to eliminate some duplication if everyone agrees
> >> with that. Maybe the repo-setup would be a good first iterative step.
> >>
> >> As for the external roles, I'm less of a fan of relying on these
> >> directly if they're not part of TripleO. I think the project should
> >> have ownership over how it's defined in CI to deploy/update/upgrade
> >> overclouds, etc.
> >
> >
> > +1 I think this can be handled in a couple ways depending on how many
> > additional git repos are acceptable to TripleO upstream.
> >
> > So maybe if I provide an example this will make more sense.  I think bare
> > metal will work as an example.
> >
> > There is a need for the code that drives CI for virt to be able to drive
> CI
> > for bare metal.  Certainly the bare metal use case will not be used
> nearly
> > as much as the virt workflow and I suspect we don't want code conflicts,
> > merge issues coming from the bare metal use case that may interrupt or
> block
> > the mainline virt use case.  I think TripleO still cares what the bare
> metal
> > code looks like, how it's developed, and if we can use it w/ 3rd party CI
> > and extra checks.  It's important to maintain bare metal roles in TripleO
> > but it's easier if they are in another git repository.   It also
> > demonstrates the composability of the CI.
> >
> > Another use case would be anything that may be downstream specific.  I
> can't
> > think of a great example atm, but there are use cases that CI should be
> able
> > to drive that will probably never be part of the mainstream tripleo ci
> jobs.
> >
> > I believe we can solve this by having just two git repos in the long
> run.  I
> > think one git repo would be for any code path that is used directly in a
> job
> > in tripleo-ci itself.  The second repo would contain multiple ansible
> roles,
> > call it tripleo-ci-extras.  The second repo would contain any extra roles
> > that need to be plugged in for a use case that is not in a tripleo-ci job
> > itself.  This would allow TripleO to manage and review the code w/o
> > impacting the day to day business of tripleo-ci.
> >
> > Something like:
> >
> > github.com/openstack/tripleo-ci-extras
> >
> > /ansible-role-upgrades
> > /ansible-role-image-build
> > /ansible-role-baremetal-undercloud
> > /ansible-role-baremetal-undercloud-post
> > /ansible-role-baremetal-overcloud
> >   /defaults
> >   /tasks/
> >   /templates
> >   setup.cfg
> >
> >>
> >> Are the roles generic enough that we could propose these as part of
> >> TripleO directly as new repos, and is there interest in doing that?
> >

Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)

2016-07-21 Thread Tripp, Travis S

On 22/07/16 10:04, Zane Bitter wrote:
> If we're not to end up with 20 different answers to the those 
> questions, we'll need some cross-project co-ordination and part of 
> that will involve depending on various projects for functionality 
> instead of implementing multiple different one-off solutions. Pick an 
> asynchronous message transport (Zaqar). Pick an event source 
> (Ceilometer? Searchlight?). Maybe pick an event sink (just Mistral? or 
> lots of sinks?).
>
> So it's architecture, but it is in a sense "user-space" architecture, 
> figuring out how services fit together into a cohesive whole, as 
> opposed to the questions you're talking about which are more 
> engineering-focused. I'd be very interested to know if you consider 
> this stuff in scope for your architecture group or if you think it 
> should have its own separate working group.
>

Well said, Zane.

I 100% have felt and continue to feel the pain that Kevin originally mentioned 
with simply being blocked on progress because the projects are often silo’d and 
there is no high level architecture group essentially saying it is okay for 
project X to have a dependency on project Y.

I know at least on Horizon, that summit after summit, cross cutting features 
that could really enhance the end user experience are rejected out of fear of 
adding any additional dependency (even if optional). I probably can’t even 
begin to count the number of times people have asked to add additional dynamic 
user preference customizations but been blocked because Horizon doesn’t want to 
add any kind of a dependency on any kind of a persistence mechanism.

Another problem I see is that project A has a high priority need to land a 
relatively simple patch in project B, but due to the extremely limited amount 
of time to actually get any reviews for non-priority features in project B, 
that only a few weeks into the release cycle you get blocked with -2 and a 
“better luck next time” message. Which effectively means you are barely a month 
or two into the current release and now are faced with the reality that it will 
be *at least* 9 months before you can hope to see your patch be released in the 
next “alphabet letter of your choice” release. In the meantime, the people 
paying the salaries of the developers working on OpenStack decide that progress 
is too slow and pull their people off all together. But don’t worry, the 
community elitists will just declare these to be drive by contributions and are 
happy to essentially pat themselves on the back for preventing those people 
from contributing code.

I am a core reviewer on a couple projects and I can honestly say that I believe 
the core reviewer teams are also a huge part of the bottleneck. This certainly 
isn’t due to cores being lazy, quite the contrary, but that perhaps it is too 
much to expect of core reviewers to essentially perform the product management 
role, the project management role, the architecture role, the evangelist role, 
the developer roles, and finally the quality assurance role (you know, the 
actual code review thing).

One of the most beautiful things about OpenStack is that the developers are 
empowered in a way that they simply are not within the confines of large 
proprietary enterprise software shops.  However, the inability to make progress 
from release to release whenever it comes to cross project integration and 
dependencies is an Achilles heel that I fear may be the downfall of OpenStack.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron][networking-ovn][networking-odl] Syncing neutron DB and OVN DB

2016-07-21 Thread Amitabha Biswas

Hi Numan,

Thanks for the proposal. We have also been thinking about this use-case.

If I’m reading this accurately (and I may not be), it seems that the proposal 
is to not have any OVN NB (CUD) operations (R operations outside the scope) 
done by the api_worker threads but rather by a new journal thread.

If this is indeed the case, I’d like to consider the scenario when there any N 
neutron nodes, each node with M worker threads. The journal thread at the each 
node contain list of pending operations. Could there be (sequence) dependency 
in the pending operations amongst each the journal threads in the nodes that 
prevents them from getting applied (for e.g. Logical_Router_Port and 
Logical_Switch_Port inter-dependency), because we are returning success on 
neutron operations that have still not been committed to the NB DB.

Couple of clarifications and thoughts below.

Thanks
Amitabha 

> On Jul 13, 2016, at 1:20 AM, Numan Siddique  wrote:
> 
> Adding the proper tags in subject
> 
> On Wed, Jul 13, 2016 at 1:22 PM, Numan Siddique  > wrote:
> Hi Neutrinos,
> 
> Presently, In the OVN ML2 driver we have 2 ways to sync neutron DB and OVN DB
>  - At neutron-server startup, OVN ML2 driver syncs the neutron DB and OVN DB 
> if sync mode is set to repair.
>  - Admin can run the "neutron-ovn-db-sync-util" to sync the DBs.
> 
> Recently, in the v2 of networking-odl ML2 driver (Please see (1) below which 
> has more details). (ODL folks please correct me if I am wrong here)
> 
>   - a journal thread is created which does the CRUD operations of neutron 
> resources asynchronously (i.e it sends the REST APIs to the ODL controller).

Would this be the equivalent of making OVSDB transactions to the OVN NB DB?

>   - a maintenance thread is created which does some cleanup periodically and 
> at startup does full sync if it detects ODL controller cold reboot.
> 
> 
> Few question I have
>  - can OVN ML2 driver take same or similar approach. Are there any advantages 
> in taking this approach ? One advantage is neutron resources can be 
> created/updated/deleted even if the OVN ML2 driver has lost connection to the 
> ovsdb-server. The journal thread would eventually sync these resources in the 
> OVN DB. I would like to know the communities thoughts on this.

If we can make it work, it would indeed be a huge plus for system wide upgrades 
and some corner cases in the code (ACL specifically), where the post_commit 
relies on all transactions to be successful and doesn’t revert the neutron db 
if something fails.

> 
>  - Are there are other ML2 drivers which might have to handle the DB sync's 
> (cases where the other controllers also maintain their own DBs) and how they 
> are handling it ?
> 
>  - Can a common approach be taken to sync the neutron DB and controller DBs ?
> 
> 
> ---
> 
> (1)
> Sync threads created by networking-odl ML2 driver
> --
> ODL ML2 driver creates 2 threads (threading.Thread module) at init
>  - Journal thread
>  - Maintenance thread
> 
> Journal thread
> 
> The journal module creates a new journal table by name “opendaylightjournal”  
> - 
> https://github.com/openstack/networking-odl/blob/master/networking_odl/db/models.py#L23
>  
> 
> 
> Journal thread will be in loop waiting for the sync event from the ODL ML2 
> driver.
> 
>  - ODL ML2 driver resource (network, subnet, port) precommit functions when 
> called by the ML2 plugin adds an entry in the “opendaylightjournal” table 
> with the resource data and sets the journal operation state for this entry to 
> “PENDING”.
>  - The corresponding resource postcommit function of the ODL ML2 plugin when 
> called, sets the sync event flag.
>  - A timer is also created which sets the sync event flag when it expires 
> (the default value is 10 seconds).
>  - Journal thread wakes up, looks into the “opendaylightjournal” table with 
> the entries with state “pending” and runs the CRUD operation on those 
> resources in the ODL DB. Once done, it sets the state to “completed”.
> 
> Maintenance thread
> --
> Maintenance thread does 3 operations
>  - JournalCleanup - Delete completed rows from journal table 
> “opendaylightjournal”.
>  - CleanupProcessing - Mark orphaned processing rows to pending.
>  - Full sync - Re-sync when detecting an ODL "cold reboot”.
> 
> 
> 
> Thanks
> Numan
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Proposing Jakub Libosvar for testing core

2016-07-21 Thread Doug Wiegley

+1

> On Jul 21, 2016, at 5:13 PM, Kevin Benton  wrote:
> 
> +1
> 
> On Thu, Jul 21, 2016 at 2:41 PM, Carl Baldwin  > wrote:
> +1 from me
> 
> On Thu, Jul 21, 2016 at 1:35 PM, Assaf Muller  > wrote:
> As Neutron's so called testing lieutenant I would like to propose
> Jakub Libosvar to be a core in the testing area.
> 
> Jakub has demonstrated his inherent interest in the testing area over
> the last few years, his reviews are consistently insightful and his
> numbers [1] are in line with others and I know will improve if given
> the responsibilities of a core reviewer. Jakub is deeply involved with
> the project's testing infrastructures and CI systems.
> 
> As a reminder the expectation from cores is found here [2], and
> specifically for cores interesting in helping out shaping Neutron's
> testing story:
> 
> * Guide community members to craft a testing strategy for features [3]
> * Ensure Neutron's testing infrastructures are sufficiently
> sophisticated to achieve the above.
> * Provide leadership when determining testing Do's & Don'ts [4]. What
> makes for an effective test?
> * Ensure the gate stays consistently green
> 
> And more tactically we're looking at finishing the Tempest/Neutron
> tests dedup [5] and to provide visual graphing for historical control
> and data plane performance results similar to [6].
> 
> [1] http://stackalytics.com/report/contribution/neutron/90 
> 
> [2] http://docs.openstack.org/developer/neutron/policies/neutron-teams.html 
> 
> [3] 
> http://docs.openstack.org/developer/neutron/devref/development.environment.html#testing-neutron
>  
> 
> [4] https://assafmuller.com/2015/05/17/testing-lightning-talk/ 
> 
> [5] https://etherpad.openstack.org/p/neutron-tempest-defork 
> 
> [6] https://www.youtube.com/watch?v=a0qlsH1hoKs=youtu.be=24m22s 
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe 
> 
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev 
> 
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe 
> 
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev 
> 
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)

2016-07-21 Thread Fei Long Wang


Thank you for making it more clear, Zane. I totally agree with you at here.

IMHO, as a cloud computing platform, it should be an ecosystem just like 
the others. That said, not only looking from bottom to top, but also 
looking from top to bottom. It should looks like an unified platform. In 
other words, we need to understand what are the app developer need. I'm 
sure except a good VM creation, they also need something else. To avoid 
it turns to be another chicken/egg problem, we could/should to eat our 
dog food, instead of just "waiting" a service get matured by itself, we 
could provide some support/help. Therefore, I think plugins for all 
could be a good start.


My 0.02

On 22/07/16 10:04, Zane Bitter wrote:

On 20/07/16 18:41, Clint Byrum wrote:

Excerpts from Fox, Kevin M's message of 2016-07-20 20:12:48 +:
And maybe this raises an interesting defininition mismatch in the 
conversation.


There is archetectural stuff like, do we support 7 different web 
frameworks, or do we standardize on flask... python vs go.




Yeah meh, that's developer centric implementation details and I think
not very interesting. To me the architectural questions are deeper. "How
do we do locking?", "How should we enable inter-process and inter-host
communication?", "How do we handle distributed transactions?" and "What
concurrency model should we use?".

Theres also the architectural stuff at the, what interactive surface 
do you expose to users/operators. How consistant is it. Does it have 
all the features, no matter where they are implemented to do work.


I believe this is the goal of the API-WG. But again, they're not there
to compel, they're there to advise, assist, and work. Ultimately, if an
API is created and is poor, like Linus, the community can definitely say
"No" and refuse to use that API.


No, the API-WG is pretty much just about coming up with standards for 
how individual REST APIs look. Kevin (IIUC) is talking about something 
at least as different from that as 'which web framework?' is from your 
list of architecture questions. It's questions like: How can 
applications running in the cloud authenticate themselves to the 
cloud? How can the user limit their authorisation to a minimal 
surface? How can the cloud notify applications of events? How can the 
user configure the cloud to respond to events without having to write 
their own service to process them? How should guest agents communicate 
with the cloud?


If we're not to end up with 20 different answers to the those 
questions, we'll need some cross-project co-ordination and part of 
that will involve depending on various projects for functionality 
instead of implementing multiple different one-off solutions. Pick an 
asynchronous message transport (Zaqar). Pick an event source 
(Ceilometer? Searchlight?). Maybe pick an event sink (just Mistral? or 
lots of sinks?).


So it's architecture, but it is in a sense "user-space" architecture, 
figuring out how services fit together into a cohesive whole, as 
opposed to the questions you're talking about which are more 
engineering-focused. I'd be very interested to know if you consider 
this stuff in scope for your architecture group or if you think it 
should have its own separate working group.


cheers,
Zane.

__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


--
Cheers & Best regards,
Fei Long Wang (王飞龙)
--
Senior Cloud Software Engineer
Tel: +64-48032246
Email: flw...@catalyst.net.nz
Catalyst IT Limited
Level 6, Catalyst House, 150 Willis Street, Wellington
--


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder] Volume Drivers unit tests

2016-07-21 Thread Knight, Clinton

Nate, you have to press Ctrl-C to see the in-progress test, that’s why you 
don’t see it in the logs.  The bug report shows this and points to the patch 
where it appeared to begin.  https://bugs.launchpad.net/cinder/+bug/1578986

Clinton

From: "Potter, Nathaniel" 
Reply-To: "OpenStack Development Mailing List (not for usage questions)" 

Date: Thursday, July 21, 2016 at 7:17 PM
To: "OpenStack Development Mailing List (not for usage questions)" 

Subject: Re: [openstack-dev] [cinder] Volume Drivers unit tests

Hi all,

I’m not totally sure that this is the same issue, but lately I’ve seen the gate 
tests fail while hanging at this point [1], but they say ‘ok’ rather than 
‘inprogress’. Has anyone else come across this? It only happens sometimes, and 
a recheck can get past it. The full log is here [2].

[1] http://paste.openstack.org/show/539314/
[2] 
http://logs.openstack.org/90/341090/6/check/gate-cinder-python34-db/ea65de5/console.html

Thanks,
Nate

From: yang, xing [mailto:xing.y...@emc.com]
Sent: Thursday, July 21, 2016 3:17 PM
To: OpenStack Development Mailing List (not for usage questions) 

Subject: Re: [openstack-dev] [cinder] Volume Drivers unit tests

Hi Ivan,

Do you have any logs for the VMAX driver?  We'll take a look.

Thanks,
Xing

From: Ivan Kolodyazhny [e...@e0ne.info]
Sent: Thursday, July 21, 2016 4:44 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [cinder] Volume Drivers unit tests
Thank you Xing,

The issue is related both to VNX and VMAX EMC drivers

Regards,
Ivan Kolodyazhny,
http://blog.e0ne.info/

On Thu, Jul 21, 2016 at 11:00 PM, yang, xing 
> wrote:
Hi Ivan,

Thanks for sending this out.  Regarding the issue in the EMC VNX driver unit 
tests, it is tracked by this bug 
https://bugs.launchpad.net/cinder/+bug/1578986.  The driver was recently 
refactored so this is probably a new issue introduced by the refactor.  We are 
investigating this issue.

Thanks,
Xing

From: Ivan Kolodyazhny [e...@e0ne.info]
Sent: Thursday, July 21, 2016 1:02 PM
To: OpenStack Development Mailing List
Subject: [openstack-dev] [cinder] Volume Drivers unit tests
Hi team,

First of all, I would like to apologize, if my mail is be too emotional. I 
spent too much of time to fix it and failed.
TL;DR;

What I want to say is: "Let's spend some time to make our tests better and fix 
all issues". Patch [1] is still unstable. Unit tests can pass or fail in a in a 
random order. Also, I've disabled some tests to pass CI.

Long version:

While I was working on patch "Move drivers unit tests to unit.volume.drivers 
directory" [1] I've found a lot of issues with our unit tests :(. Not all of 
them are already fixed, so that patch is still in progress

What did I found and what should we have to fix:

1) Execution time [2]. I don't want to argue what it unit tests, but 2-4 
seconds per tests should be non-acceptable, IMO.

2) Execution order. Seriously, do you know that our tests will fail or hang if 
execution order will change? Even if one test for diver A failed, some tests 
for driver B will fail too.

3) Lack of mock. It's a root cause for #2. We didn't mock sleeps and event 
loops right. We don't mock RPC call well too [3]. We don't have 
'cinder.openstack.common.rpc.impl_fake' module in Cinder tree.

In some drivers, we use oslo_service.loopingcall.FixedIntervalLoopingCall [4]. 
We've go ZeroIntervalLoopingCall [5] class in Cinder. Do we use it everywhere 
or mock FixedIntervalLoopingCall right? I don't think so, I've hacked 
oslo_service in my env to rise an exception if interval > 0. 297 tests failed. 
It means, our tests use sleep. We have to get rid of this. TBH, not only volume 
drivers unit tests failed. E.g. some API unit tests failed too.

4) Due to #3, sometimes unit tests hangs even on master branch with a minor 
changes.If I stop execution of such tests, usually I see something like [6]. In 
most of cases I see that following drivers' tests hangs: EMC, Huawei, Dell and 
RBD.

It's hard to debug such failures because the lack of tooling for eventlet 
debugging. Eventlet backdoors and gdb-python helps a bit. Maybe somebody know 
better solution for it.

[1] https://review.openstack.org/#/c/320148/
[2] http://paste.openstack.org/show/539081/
[3] https://github.com/openstack/cinder/search?utf8=%E2%9C%93=impl_fake
[4] use 
https://github.com/openstack/oslo.service/blob/master/oslo_service/loopingcall.py#L162
[5] 
https://github.com/openstack/cinder/blob/cfbb5bde4d9b37c39f6813fe685f987f8a990483/cinder/tests/unit/utils.py#L289
[6] http://paste.openstack.org/show/539090/

Regards,
Ivan Kolodyazhny,
http://blog.e0ne.info/

Re: [openstack-dev] [cinder] Volume Drivers unit tests

2016-07-21 Thread Potter, Nathaniel

Hi all,

I'm not totally sure that this is the same issue, but lately I've seen the gate 
tests fail while hanging at this point [1], but they say 'ok' rather than 
'inprogress'. Has anyone else come across this? It only happens sometimes, and 
a recheck can get past it. The full log is here [2].

[1] http://paste.openstack.org/show/539314/
[2] 
http://logs.openstack.org/90/341090/6/check/gate-cinder-python34-db/ea65de5/console.html

Thanks,
Nate


From: yang, xing [mailto:xing.y...@emc.com]
Sent: Thursday, July 21, 2016 3:17 PM
To: OpenStack Development Mailing List (not for usage questions) 

Subject: Re: [openstack-dev] [cinder] Volume Drivers unit tests

Hi Ivan,

Do you have any logs for the VMAX driver?  We'll take a look.

Thanks,
Xing



From: Ivan Kolodyazhny [e...@e0ne.info]
Sent: Thursday, July 21, 2016 4:44 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [cinder] Volume Drivers unit tests
Thank you Xing,

The issue is related both to VNX and VMAX EMC drivers

Regards,
Ivan Kolodyazhny,
http://blog.e0ne.info/

On Thu, Jul 21, 2016 at 11:00 PM, yang, xing 
> wrote:
Hi Ivan,

Thanks for sending this out.  Regarding the issue in the EMC VNX driver unit 
tests, it is tracked by this bug 
https://bugs.launchpad.net/cinder/+bug/1578986.  The driver was recently 
refactored so this is probably a new issue introduced by the refactor.  We are 
investigating this issue.

Thanks,
Xing



From: Ivan Kolodyazhny [e...@e0ne.info]
Sent: Thursday, July 21, 2016 1:02 PM
To: OpenStack Development Mailing List
Subject: [openstack-dev] [cinder] Volume Drivers unit tests
Hi team,

First of all, I would like to apologize, if my mail is be too emotional. I 
spent too much of time to fix it and failed.
TL;DR;

What I want to say is: "Let's spend some time to make our tests better and fix 
all issues". Patch [1] is still unstable. Unit tests can pass or fail in a in a 
random order. Also, I've disabled some tests to pass CI.


Long version:

While I was working on patch "Move drivers unit tests to unit.volume.drivers 
directory" [1] I've found a lot of issues with our unit tests :(. Not all of 
them are already fixed, so that patch is still in progress

What did I found and what should we have to fix:

1) Execution time [2]. I don't want to argue what it unit tests, but 2-4 
seconds per tests should be non-acceptable, IMO.

2) Execution order. Seriously, do you know that our tests will fail or hang if 
execution order will change? Even if one test for diver A failed, some tests 
for driver B will fail too.

3) Lack of mock. It's a root cause for #2. We didn't mock sleeps and event 
loops right. We don't mock RPC call well too [3]. We don't have 
'cinder.openstack.common.rpc.impl_fake' module in Cinder tree.

In some drivers, we use oslo_service.loopingcall.FixedIntervalLoopingCall [4]. 
We've go ZeroIntervalLoopingCall [5] class in Cinder. Do we use it everywhere 
or mock FixedIntervalLoopingCall right? I don't think so, I've hacked 
oslo_service in my env to rise an exception if interval > 0. 297 tests failed. 
It means, our tests use sleep. We have to get rid of this. TBH, not only volume 
drivers unit tests failed. E.g. some API unit tests failed too.


4) Due to #3, sometimes unit tests hangs even on master branch with a minor 
changes.If I stop execution of such tests, usually I see something like [6]. In 
most of cases I see that following drivers' tests hangs: EMC, Huawei, Dell and 
RBD.

It's hard to debug such failures because the lack of tooling for eventlet 
debugging. Eventlet backdoors and gdb-python helps a bit. Maybe somebody know 
better solution for it.

[1] https://review.openstack.org/#/c/320148/
[2] http://paste.openstack.org/show/539081/
[3] https://github.com/openstack/cinder/search?utf8=%E2%9C%93=impl_fake
[4] use 
https://github.com/openstack/oslo.service/blob/master/oslo_service/loopingcall.py#L162
[5] 
https://github.com/openstack/cinder/blob/cfbb5bde4d9b37c39f6813fe685f987f8a990483/cinder/tests/unit/utils.py#L289
[6] http://paste.openstack.org/show/539090/


Regards,
Ivan Kolodyazhny,
http://blog.e0ne.info/

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Proposing Jakub Libosvar for testing core

2016-07-21 Thread Kevin Benton

+1

On Thu, Jul 21, 2016 at 2:41 PM, Carl Baldwin  wrote:

> +1 from me
>
> On Thu, Jul 21, 2016 at 1:35 PM, Assaf Muller  wrote:
>
>> As Neutron's so called testing lieutenant I would like to propose
>> Jakub Libosvar to be a core in the testing area.
>>
>> Jakub has demonstrated his inherent interest in the testing area over
>> the last few years, his reviews are consistently insightful and his
>> numbers [1] are in line with others and I know will improve if given
>> the responsibilities of a core reviewer. Jakub is deeply involved with
>> the project's testing infrastructures and CI systems.
>>
>> As a reminder the expectation from cores is found here [2], and
>> specifically for cores interesting in helping out shaping Neutron's
>> testing story:
>>
>> * Guide community members to craft a testing strategy for features [3]
>> * Ensure Neutron's testing infrastructures are sufficiently
>> sophisticated to achieve the above.
>> * Provide leadership when determining testing Do's & Don'ts [4]. What
>> makes for an effective test?
>> * Ensure the gate stays consistently green
>>
>> And more tactically we're looking at finishing the Tempest/Neutron
>> tests dedup [5] and to provide visual graphing for historical control
>> and data plane performance results similar to [6].
>>
>> [1] http://stackalytics.com/report/contribution/neutron/90
>> [2]
>> http://docs.openstack.org/developer/neutron/policies/neutron-teams.html
>> [3]
>> http://docs.openstack.org/developer/neutron/devref/development.environment.html#testing-neutron
>> [4] https://assafmuller.com/2015/05/17/testing-lightning-talk/
>> [5] https://etherpad.openstack.org/p/neutron-tempest-defork
>> [6] https://www.youtube.com/watch?v=a0qlsH1hoKs=youtu.be=24m22s
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)

2016-07-21 Thread Fox, Kevin M

Zane, Thanks. You have managed to articulate my concern where I've failed so 
far. +1 :)

Kevin

From: Zane Bitter [zbit...@redhat.com]
Sent: Thursday, July 21, 2016 3:04 PM
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)

On 20/07/16 18:41, Clint Byrum wrote:
> Excerpts from Fox, Kevin M's message of 2016-07-20 20:12:48 +:
>> And maybe this raises an interesting defininition mismatch in the 
>> conversation.
>>
>> There is archetectural stuff like, do we support 7 different web frameworks, 
>> or do we standardize on flask... python vs go.
>>
>
> Yeah meh, that's developer centric implementation details and I think
> not very interesting. To me the architectural questions are deeper. "How
> do we do locking?", "How should we enable inter-process and inter-host
> communication?", "How do we handle distributed transactions?" and "What
> concurrency model should we use?".
>
>> Theres also the architectural stuff at the, what interactive surface do you 
>> expose to users/operators. How consistant is it. Does it have all the 
>> features, no matter where they are implemented to do work.
>
> I believe this is the goal of the API-WG. But again, they're not there
> to compel, they're there to advise, assist, and work. Ultimately, if an
> API is created and is poor, like Linus, the community can definitely say
> "No" and refuse to use that API.

No, the API-WG is pretty much just about coming up with standards for
how individual REST APIs look. Kevin (IIUC) is talking about something
at least as different from that as 'which web framework?' is from your
list of architecture questions. It's questions like: How can
applications running in the cloud authenticate themselves to the cloud?
How can the user limit their authorisation to a minimal surface? How can
the cloud notify applications of events? How can the user configure the
cloud to respond to events without having to write their own service to
process them? How should guest agents communicate with the cloud?

If we're not to end up with 20 different answers to the those questions,
we'll need some cross-project co-ordination and part of that will
involve depending on various projects for functionality instead of
implementing multiple different one-off solutions. Pick an asynchronous
message transport (Zaqar). Pick an event source (Ceilometer?
Searchlight?). Maybe pick an event sink (just Mistral? or lots of sinks?).

So it's architecture, but it is in a sense "user-space" architecture,
figuring out how services fit together into a cohesive whole, as opposed
to the questions you're talking about which are more
engineering-focused. I'd be very interested to know if you consider this
stuff in scope for your architecture group or if you think it should
have its own separate working group.

cheers,
Zane.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder] Volume Drivers unit tests

2016-07-21 Thread yang, xing

Hi Ivan,

Do you have any logs for the VMAX driver?  We'll take a look.

Thanks,
Xing



From: Ivan Kolodyazhny [e...@e0ne.info]
Sent: Thursday, July 21, 2016 4:44 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [cinder] Volume Drivers unit tests

Thank you Xing,

The issue is related both to VNX and VMAX EMC drivers

Regards,
Ivan Kolodyazhny,
http://blog.e0ne.info/

On Thu, Jul 21, 2016 at 11:00 PM, yang, xing 
> wrote:
Hi Ivan,

Thanks for sending this out.  Regarding the issue in the EMC VNX driver unit 
tests, it is tracked by this bug 
https://bugs.launchpad.net/cinder/+bug/1578986.  The driver was recently 
refactored so this is probably a new issue introduced by the refactor.  We are 
investigating this issue.

Thanks,
Xing



From: Ivan Kolodyazhny [e...@e0ne.info]
Sent: Thursday, July 21, 2016 1:02 PM
To: OpenStack Development Mailing List
Subject: [openstack-dev] [cinder] Volume Drivers unit tests

Hi team,

First of all, I would like to apologize, if my mail is be too emotional. I 
spent too much of time to fix it and failed.

TL;DR;

What I want to say is: "Let's spend some time to make our tests better and fix 
all issues". Patch [1] is still unstable. Unit tests can pass or fail in a in a 
random order. Also, I've disabled some tests to pass CI.


Long version:

While I was working on patch "Move drivers unit tests to unit.volume.drivers 
directory" [1] I've found a lot of issues with our unit tests :(. Not all of 
them are already fixed, so that patch is still in progress

What did I found and what should we have to fix:

1) Execution time [2]. I don't want to argue what it unit tests, but 2-4 
seconds per tests should be non-acceptable, IMO.

2) Execution order. Seriously, do you know that our tests will fail or hang if 
execution order will change? Even if one test for diver A failed, some tests 
for driver B will fail too.

3) Lack of mock. It's a root cause for #2. We didn't mock sleeps and event 
loops right. We don't mock RPC call well too [3]. We don't have 
'cinder.openstack.common.rpc.impl_fake' module in Cinder tree.

In some drivers, we use oslo_service.loopingcall.FixedIntervalLoopingCall [4]. 
We've go ZeroIntervalLoopingCall [5] class in Cinder. Do we use it everywhere 
or mock FixedIntervalLoopingCall right? I don't think so, I've hacked 
oslo_service in my env to rise an exception if interval > 0. 297 tests failed. 
It means, our tests use sleep. We have to get rid of this. TBH, not only volume 
drivers unit tests failed. E.g. some API unit tests failed too.


4) Due to #3, sometimes unit tests hangs even on master branch with a minor 
changes.If I stop execution of such tests, usually I see something like [6]. In 
most of cases I see that following drivers' tests hangs: EMC, Huawei, Dell and 
RBD.

It's hard to debug such failures because the lack of tooling for eventlet 
debugging. Eventlet backdoors and gdb-python helps a bit. Maybe somebody know 
better solution for it.

[1] https://review.openstack.org/#/c/320148/
[2] http://paste.openstack.org/show/539081/
[3] https://github.com/openstack/cinder/search?utf8=%E2%9C%93=impl_fake
[4] use 
https://github.com/openstack/oslo.service/blob/master/oslo_service/loopingcall.py#L162
[5] 
https://github.com/openstack/cinder/blob/cfbb5bde4d9b37c39f6813fe685f987f8a990483/cinder/tests/unit/utils.py#L289
[6] http://paste.openstack.org/show/539090/


Regards,
Ivan Kolodyazhny,
http://blog.e0ne.info/

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)

2016-07-21 Thread Zane Bitter


On 20/07/16 18:41, Clint Byrum wrote:

Excerpts from Fox, Kevin M's message of 2016-07-20 20:12:48 +:

And maybe this raises an interesting defininition mismatch in the conversation.

There is archetectural stuff like, do we support 7 different web frameworks, or 
do we standardize on flask... python vs go.



Yeah meh, that's developer centric implementation details and I think
not very interesting. To me the architectural questions are deeper. "How
do we do locking?", "How should we enable inter-process and inter-host
communication?", "How do we handle distributed transactions?" and "What
concurrency model should we use?".


Theres also the architectural stuff at the, what interactive surface do you 
expose to users/operators. How consistant is it. Does it have all the features, 
no matter where they are implemented to do work.


I believe this is the goal of the API-WG. But again, they're not there
to compel, they're there to advise, assist, and work. Ultimately, if an
API is created and is poor, like Linus, the community can definitely say
"No" and refuse to use that API.


No, the API-WG is pretty much just about coming up with standards for 
how individual REST APIs look. Kevin (IIUC) is talking about something 
at least as different from that as 'which web framework?' is from your 
list of architecture questions. It's questions like: How can 
applications running in the cloud authenticate themselves to the cloud? 
How can the user limit their authorisation to a minimal surface? How can 
the cloud notify applications of events? How can the user configure the 
cloud to respond to events without having to write their own service to 
process them? How should guest agents communicate with the cloud?


If we're not to end up with 20 different answers to the those questions, 
we'll need some cross-project co-ordination and part of that will 
involve depending on various projects for functionality instead of 
implementing multiple different one-off solutions. Pick an asynchronous 
message transport (Zaqar). Pick an event source (Ceilometer? 
Searchlight?). Maybe pick an event sink (just Mistral? or lots of sinks?).


So it's architecture, but it is in a sense "user-space" architecture, 
figuring out how services fit together into a cohesive whole, as opposed 
to the questions you're talking about which are more 
engineering-focused. I'd be very interested to know if you consider this 
stuff in scope for your architecture group or if you think it should 
have its own separate working group.


cheers,
Zane.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [tripleo] tripleo-test-cloud-rh2 local mirror server

2016-07-21 Thread Paul Belanger

Greetings,

I write today to see how I can remove this server from tripleo-test-cloud-rh2. I
have an open patch[1] currently to migrate tripleo-ci to use our AFS mirrors for
centos and epel.  However, I'm still struggling to see what else you are using
the local mirror for.

>From what I see, there appears to be some puppet modules in the mirror?

The reason I am doing this work, is to help bring tripleo inline with
openstack-infra tooling.  There shouldn't be the need for a project to maintain
its own infrastructure outside of openstack-infra.  If so, I see that as some
sort of a failure between the project and openstack-infra.   And with that in
mind, I am here to help fix that.

For the most part, I think we have everything currently in place to migrate away
from your locally mirror. I just need some help figuring what else is left and
then delete it.

[1] https://review.openstack.org/#/c/326143/

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [TripleO][DIB] Proposal to move DIB to its own project team

2016-07-21 Thread Gregory Haynes

Hello everyone,

The subject sort of says it all - I'd like to propose making
diskimage-builder its own project team.

When we started diskimage-builder and many of the other TripleO
components we designed them with the goal in mind of creating tools that
are useful outside of the TripleO context (in addition to fulfilling our
immediate needs).  To that effect diskimage-builder has become more of a
cross-project tool designed and used by several of the OpenStack
projects and as a result it no longer seems to make sense for
diskimage-builder to be part of the TripleO project team. Our two core
groups have diverged to a large extent over the last several cycles
which has removed much of the value of being part of that project team
while creating some awkward communication issues. To be clear - I
believe this is purely a result of the TripleO project team succeeding
in its goal to improve OpenStack by use of the virtuous cycle and this
is an ideal result of that goal.

Is this is something the DIB and TripleO folks agree/disagree with? If
we all agree then I think this should be a fairly straightforward
process, otherwise I welcome some discussion on the topic :).

Cheers,
Greg

-- 
  Gregory Haynes
  g...@greghaynes.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] Making TripleO CI easier to consume outside of TripleO CI

2016-07-21 Thread James Slagle

On Tue, Jul 19, 2016 at 5:15 PM, Wesley Hayutin  wrote:
>
>
> On Tue, Jul 19, 2016 at 2:44 PM, James Slagle 
> wrote:
>> Part of the goal of tripleo.sh was to mirror the commands in the
>> documentation...that the same commands in the docs are in tripleo.sh.
>> I know that has somewhat failed, but it was the goal anyway.
>>
>> Do you think integrating ansible into tripleo-ci changes that at all?
>> tripleo-ci would be using ansible in some places, which in turn runs
>> the commands (or their equivalent) that we actually document. Is the
>> documentation still showing the same commands it does now, or is it
>> showing running ansible as tripleo-ci would be doing?
>
>
> Harry Rybacki and I are working on this now.  I think we have something that
> is reasonable for when shell can be used and when ansible modules are
> required.  I think he can make all this work public and everyone in TripleO
> can keep tabs on the progress.

Yes, I saw his email shortly after sending my reply. There is a lot to
digest there, but it sounds promising. Perhaps we could start with
something small and iteratively consume the generated docs as well. We
could replace the manual docs over time by including sphinx docs at
the right places that were automatically generated.

>
>>
>>
>> I think I'm mostly in agreement with your #2 proposal, perhaps with
>> the exception of having to rely on external roles. I don't think I
>> would want to see tripleo-ci rely on ansible roles from a
>> redhat-openstack organization on github.
>>
>> I know that we already have a lot of external dependencies in TripleO,
>> and that not everything has to come from git.openstack.org. However, I
>> think that we'd want to make the "source" for tripleo-ci be owned by
>> the TripleO project and hosted by OpenStack infrastructure as much as
>> possible. tripleo-quickstart already is, so I think it would be fine
>> to start proposing some changes to tripleo-ci that use
>> tripleo-quickstart to eliminate some duplication if everyone agrees
>> with that. Maybe the repo-setup would be a good first iterative step.
>>
>> As for the external roles, I'm less of a fan of relying on these
>> directly if they're not part of TripleO. I think the project should
>> have ownership over how it's defined in CI to deploy/update/upgrade
>> overclouds, etc.
>
>
> +1 I think this can be handled in a couple ways depending on how many
> additional git repos are acceptable to TripleO upstream.
>
> So maybe if I provide an example this will make more sense.  I think bare
> metal will work as an example.
>
> There is a need for the code that drives CI for virt to be able to drive CI
> for bare metal.  Certainly the bare metal use case will not be used nearly
> as much as the virt workflow and I suspect we don't want code conflicts,
> merge issues coming from the bare metal use case that may interrupt or block
> the mainline virt use case.  I think TripleO still cares what the bare metal
> code looks like, how it's developed, and if we can use it w/ 3rd party CI
> and extra checks.  It's important to maintain bare metal roles in TripleO
> but it's easier if they are in another git repository.   It also
> demonstrates the composability of the CI.
>
> Another use case would be anything that may be downstream specific.  I can't
> think of a great example atm, but there are use cases that CI should be able
> to drive that will probably never be part of the mainstream tripleo ci jobs.
>
> I believe we can solve this by having just two git repos in the long run.  I
> think one git repo would be for any code path that is used directly in a job
> in tripleo-ci itself.  The second repo would contain multiple ansible roles,
> call it tripleo-ci-extras.  The second repo would contain any extra roles
> that need to be plugged in for a use case that is not in a tripleo-ci job
> itself.  This would allow TripleO to manage and review the code w/o
> impacting the day to day business of tripleo-ci.
>
> Something like:
>
> github.com/openstack/tripleo-ci-extras
>
> /ansible-role-upgrades
> /ansible-role-image-build
> /ansible-role-baremetal-undercloud
> /ansible-role-baremetal-undercloud-post
> /ansible-role-baremetal-overcloud
>   /defaults
>   /tasks/
>   /templates
>   setup.cfg
>
>>
>> Are the roles generic enough that we could propose these as part of
>> TripleO directly as new repos, and is there interest in doing that?
>
>
> That is also a possibility.  I think we'd have to take a hard look at some
> of these roles and determine whether or not it's worth having it's own git
> repo.
> Most are fairly generic imho.  For instance the image-build role can build
> images for tripleo, rdo, or rhos, the bare metal roles don't have any
> environment specific code they just expect the settings provided to the job
> to describe the environment in a particular way.   The upgrade role can be
> used for any tripleo version upstream or downstream.
>
> IMHO I think

Re: [openstack-dev] [Neutron] Proposing Jakub Libosvar for testing core

+1 from me

On Thu, Jul 21, 2016 at 1:35 PM, Assaf Muller  wrote:

> As Neutron's so called testing lieutenant I would like to propose
> Jakub Libosvar to be a core in the testing area.
>
> Jakub has demonstrated his inherent interest in the testing area over
> the last few years, his reviews are consistently insightful and his
> numbers [1] are in line with others and I know will improve if given
> the responsibilities of a core reviewer. Jakub is deeply involved with
> the project's testing infrastructures and CI systems.
>
> As a reminder the expectation from cores is found here [2], and
> specifically for cores interesting in helping out shaping Neutron's
> testing story:
>
> * Guide community members to craft a testing strategy for features [3]
> * Ensure Neutron's testing infrastructures are sufficiently
> sophisticated to achieve the above.
> * Provide leadership when determining testing Do's & Don'ts [4]. What
> makes for an effective test?
> * Ensure the gate stays consistently green
>
> And more tactically we're looking at finishing the Tempest/Neutron
> tests dedup [5] and to provide visual graphing for historical control
> and data plane performance results similar to [6].
>
> [1] http://stackalytics.com/report/contribution/neutron/90
> [2]
> http://docs.openstack.org/developer/neutron/policies/neutron-teams.html
> [3]
> http://docs.openstack.org/developer/neutron/devref/development.environment.html#testing-neutron
> [4] https://assafmuller.com/2015/05/17/testing-lightning-talk/
> [5] https://etherpad.openstack.org/p/neutron-tempest-defork
> [6] https://www.youtube.com/watch?v=a0qlsH1hoKs=youtu.be=24m22s
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Proposing Jakub Libosvar for testing core

2016-07-21 Thread Ihar Hrachyshka

Enormous +1 from me. Jakub reviews are always insightful, and he is one of  
those rare people who actually review tests, not just production code. He  
also proved his dedication the previous cycle when pushed the flow based  
firewall in the tree that is hopefully the future of ovs based gate.


Cheers,
Ihar

Henry Gessau  wrote:


Big +1 from me.

Assaf Muller  wrote:

As Neutron's so called testing lieutenant I would like to propose
Jakub Libosvar to be a core in the testing area.

Jakub has demonstrated his inherent interest in the testing area over
the last few years, his reviews are consistently insightful and his
numbers [1] are in line with others and I know will improve if given
the responsibilities of a core reviewer. Jakub is deeply involved with
the project's testing infrastructures and CI systems.

As a reminder the expectation from cores is found here [2], and
specifically for cores interesting in helping out shaping Neutron's
testing story:

* Guide community members to craft a testing strategy for features [3]
* Ensure Neutron's testing infrastructures are sufficiently
sophisticated to achieve the above.
* Provide leadership when determining testing Do's & Don'ts [4]. What
makes for an effective test?
* Ensure the gate stays consistently green

And more tactically we're looking at finishing the Tempest/Neutron
tests dedup [5] and to provide visual graphing for historical control
and data plane performance results similar to [6].

[1] http://stackalytics.com/report/contribution/neutron/90
[2]  
http://docs.openstack.org/developer/neutron/policies/neutron-teams.html
[3]  
http://docs.openstack.org/developer/neutron/devref/development.environment.html#testing-neutron

[4] https://assafmuller.com/2015/05/17/testing-lightning-talk/
[5] https://etherpad.openstack.org/p/neutron-tempest-defork
[6] https://www.youtube.com/watch?v=a0qlsH1hoKs=youtu.be=24m22s




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][stable] status update and call for action

I appreciate how you're trying to steer this big ship in a new direction to
improve support for our releases. I know it must be frustrating when it
doesn't turn as quickly as it should.

On Thu, Jun 30, 2016 at 8:56 AM, Ihar Hrachyshka 
wrote:
>
> For the start, I produced a bunch of topic specific LP dashboards,
> specifically:
>
> - ipv6: https://goo.gl/dyu1d1
> - dns: https://goo.gl/9H2BlK
> - l3-ipam-dhcp: https://goo.gl/v4XWE4
> - l3-dvr-backlog: https://goo.gl/sx0KL5
> - l3-ha: https://goo.gl/QIIRa1

Ihar, I've add the above five to the L3 team agenda [1] under our bug
topic. We will have a discussion about this in our meeting. Our team has
been putting progressively more emphasis on bugs and this is another step.
I don't know how this will turn out yet but I think it is worth bringing up
to the team.

- api: https://goo.gl/d66XtB
> - db: https://goo.gl/8NNtym
> - loadimpact: https://goo.gl/xQuKRc
> - ovs: https://goo.gl/Zr70co
> - linuxbridge: https://goo.gl/CrcCzU
> - sg-fw: https://goo.gl/K9lkdA
> - qos: https://goo.gl/9kRCJv
>
> (There are more tags to consider, but let’s start with those.)
>
> Is there will to help with the process?
>

I'm eager to help but I tend to run a little bit oversubscribed so it can
be difficult to change my behavior.  :)

Carl

[1] https://etherpad.openstack.org/p/neutron-l3-subteam
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Proposing Jakub Libosvar for testing core

2016-07-21 Thread Henry Gessau

Big +1 from me.

Assaf Muller  wrote:
> As Neutron's so called testing lieutenant I would like to propose
> Jakub Libosvar to be a core in the testing area.
> 
> Jakub has demonstrated his inherent interest in the testing area over
> the last few years, his reviews are consistently insightful and his
> numbers [1] are in line with others and I know will improve if given
> the responsibilities of a core reviewer. Jakub is deeply involved with
> the project's testing infrastructures and CI systems.
> 
> As a reminder the expectation from cores is found here [2], and
> specifically for cores interesting in helping out shaping Neutron's
> testing story:
> 
> * Guide community members to craft a testing strategy for features [3]
> * Ensure Neutron's testing infrastructures are sufficiently
> sophisticated to achieve the above.
> * Provide leadership when determining testing Do's & Don'ts [4]. What
> makes for an effective test?
> * Ensure the gate stays consistently green
> 
> And more tactically we're looking at finishing the Tempest/Neutron
> tests dedup [5] and to provide visual graphing for historical control
> and data plane performance results similar to [6].
> 
> [1] http://stackalytics.com/report/contribution/neutron/90
> [2] http://docs.openstack.org/developer/neutron/policies/neutron-teams.html
> [3] 
> http://docs.openstack.org/developer/neutron/devref/development.environment.html#testing-neutron
> [4] https://assafmuller.com/2015/05/17/testing-lightning-talk/
> [5] https://etherpad.openstack.org/p/neutron-tempest-defork
> [6] https://www.youtube.com/watch?v=a0qlsH1hoKs=youtu.be=24m22s



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [telemetry] Mascot

2016-07-21 Thread gordon chung

meerkat is a good option too. i think we have something to vote on :)

On 21/07/2016 2:35 PM, Ildikó Váncsa wrote:
> I had the Meerkat [1] in mind as Telemetry has a "family" of services and 
> meerkat lives in bigger groups too and a few of them stand sentry and listen 
> to any event or danger, etc. and "send alarms" to the others.
>
> From the previous options I like Fennec too.
>
> /Ildiko
>
> [1] https://en.wikipedia.org/wiki/Meerkat
>
>> -Original Message-
>> From: Chris Dent [mailto:cdent...@anticdent.org]
>> Sent: July 21, 2016 19:47
>> To: OpenStack Development Mailing List (not for usage questions)
>> Subject: Re: [openstack-dev] [telemetry] Mascot
>>
>> On Thu, 21 Jul 2016, gordon chung wrote:
>>> On 21/07/2016 11:36 AM, Doug Hellmann wrote:
 Something with big ears that are good for listening. A fennec [1]? A
 serval [2]?

 Doug

 [1] https://en.wikipedia.org/wiki/Fennec_fox
 [2] https://en.wikipedia.org/wiki/Serval
>>>
>>> i really like these suggestions, with a preference for the fennec. the
>>> big ears idea was great.
>>
>> +1 on the fennec
>>
>> --
>> Chris Dent   ┬─┬ノ( º _ ºノ) http://anticdent.org/
>> freenode: cdent tw: @anticdent
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder] Volume Drivers unit tests

2016-07-21 Thread Ivan Kolodyazhny

Thank you Xing,

The issue is related both to VNX and VMAX EMC drivers

Regards,
Ivan Kolodyazhny,
http://blog.e0ne.info/

On Thu, Jul 21, 2016 at 11:00 PM, yang, xing  wrote:

> Hi Ivan,
>
> Thanks for sending this out.  Regarding the issue in the EMC VNX driver
> unit tests, it is tracked by this bug
> https://bugs.launchpad.net/cinder/+bug/1578986.  The driver was recently
> refactored so this is probably a new issue introduced by the refactor.  We are
> investigating this issue.
>
> Thanks,
> Xing
>
>
> --
> *From:* Ivan Kolodyazhny [e...@e0ne.info]
> *Sent:* Thursday, July 21, 2016 1:02 PM
> *To:* OpenStack Development Mailing List
> *Subject:* [openstack-dev] [cinder] Volume Drivers unit tests
>
> Hi team,
>
> First of all, I would like to apologize, if my mail is be too emotional.
> I spent too much of time to fix it and failed.
> TL;DR;
>
> What I want to say is: "Let's spend some time to make our tests better and
> fix all issues". Patch [1] is still unstable. Unit tests can pass or fail
> in a in a random order. Also, I've disabled some tests to pass CI.
>
>
> Long version:
>
> While I was working on patch "Move drivers unit tests to
> unit.volume.drivers directory" [1] I've found a lot of issues with our unit
> tests :(. Not all of them are already fixed, so that patch is still in
> progress
>
> What did I found and what should we have to fix:
>
> 1) Execution time [2]. I don't want to argue what it unit tests, but 2-4
> seconds per tests should be non-acceptable, IMO.
>
> 2) Execution order. Seriously, do you know that our tests will fail or
> hang if execution order will change? Even if one test for diver A failed,
> some tests for driver B will fail too.
>
> 3) Lack of mock. It's a root cause for #2. We didn't mock sleeps and event
> loops right. We don't mock RPC call well too [3]. We don't
> have 'cinder.openstack.common.rpc.impl_fake' module in Cinder tree.
>
> In some drivers, we use oslo_service.loopingcall.FixedIntervalLoopingCall
> [4]. We've go ZeroIntervalLoopingCall [5] class in Cinder. Do we use it
> everywhere or mock FixedIntervalLoopingCall right? I don't think so, I've
> hacked oslo_service in my env to rise an exception if interval > 0. 297
> tests failed. It means, our tests use sleep. We have to get rid of this.
> TBH, not only volume drivers unit tests failed. E.g. some API unit tests
> failed too.
>
>
> 4) Due to #3, sometimes unit tests hangs even on master branch with a
> minor changes.If I stop execution of such tests, usually I see something
> like [6]. In most of cases I see that following drivers' tests hangs: EMC,
> Huawei, Dell and RBD.
>
> It's hard to debug such failures because the lack of tooling for eventlet
> debugging. Eventlet backdoors and gdb-python helps a bit. Maybe somebody
> know better solution for it.
>
> [1] https://review.openstack.org/#/c/320148/
> [2] http://paste.openstack.org/show/539081/
> [3] https://github.com/openstack/cinder/search?utf8=%E2%9C%93=impl_fake
> [4] use
> https://github.com/openstack/oslo.service/blob/master/oslo_service/loopingcall.py#L162
> [5]
> https://github.com/openstack/cinder/blob/cfbb5bde4d9b37c39f6813fe685f987f8a990483/cinder/tests/unit/utils.py#L289
> [6] http://paste.openstack.org/show/539090/
>
>
> Regards,
> Ivan Kolodyazhny,
> http://blog.e0ne.info/
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Neutron][oslo.db] Inspecting sqlite db during unit tests

Hi,

In Neutron, we run unit tests with an in-memory sqlite instance. It is
impossible, as far as I know, to inspect this database using the sqlite3
command line while the unit tests are running. So, we have to resort to
python / sqlalchemy to do it. This is inconvenient.

Months ago, I was able to get the unit tests to write the sqlite db to a
file so that I could inspect it while I was sitting at a breakpoint in the
code. That was very nice. Yesterday, I tried to repeat that while traveling
and was unable to figure it out. I had to time box my effort to move on to
other things.

As far as I remember, the mechanism that I used was to adjust the
neutron.conf for the tests [1]. I'm not totally sure about this because I
didn't take sufficient notes, I think because it was pretty easy to figure
it out at the time. This mechanism doesn't seem to have any effect these
days. I changed it to 'sqlite:tmp/unit-test.db' and never saw a file
created there.

I did a little bit of digging and I tried one more thing. That was to
set OS_TEST_DBAPI_ADMIN_CONNECTION='sqlite:tmp/unit-test.db' in the
environment before running tests. I was encouraged because this caused a
file to be created at that location but the file remained empty for the
duration of the run.

Does anyone know off the top of their head how to get unit tests in Neutron
to use a file based sqlite db?

Carl

[1]
https://github.com/openstack/neutron/blob/97c491294cf9eca0921336719d62d74ec4e1fa96/neutron/tests/etc/neutron.conf#L26
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Neutron] Proposing Jakub Libosvar for testing core

2016-07-21 Thread Assaf Muller

As Neutron's so called testing lieutenant I would like to propose
Jakub Libosvar to be a core in the testing area.

Jakub has demonstrated his inherent interest in the testing area over
the last few years, his reviews are consistently insightful and his
numbers [1] are in line with others and I know will improve if given
the responsibilities of a core reviewer. Jakub is deeply involved with
the project's testing infrastructures and CI systems.

As a reminder the expectation from cores is found here [2], and
specifically for cores interesting in helping out shaping Neutron's
testing story:

* Guide community members to craft a testing strategy for features [3]
* Ensure Neutron's testing infrastructures are sufficiently
sophisticated to achieve the above.
* Provide leadership when determining testing Do's & Don'ts [4]. What
makes for an effective test?
* Ensure the gate stays consistently green

And more tactically we're looking at finishing the Tempest/Neutron
tests dedup [5] and to provide visual graphing for historical control
and data plane performance results similar to [6].

[1] http://stackalytics.com/report/contribution/neutron/90
[2] http://docs.openstack.org/developer/neutron/policies/neutron-teams.html
[3] 
http://docs.openstack.org/developer/neutron/devref/development.environment.html#testing-neutron
[4] https://assafmuller.com/2015/05/17/testing-lightning-talk/
[5] https://etherpad.openstack.org/p/neutron-tempest-defork
[6] https://www.youtube.com/watch?v=a0qlsH1hoKs=youtu.be=24m22s

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] Plugin stability, failing tests etc.

2016-07-21 Thread Rob Cresswell

The aim wasn't to be focused on what's easiest for Horizon at all; it was a 
suggestion to let plugins keep moving during brief periods of instability when 
people make mistakes. We're aware of the plugins need for stability and correct 
deprecation, but we won't catch everything.

That said, there's been significant enough push back against the suggestion 
that I'll rethink it. I'll make sure we discuss the horizon / 
openstack_dashboard split again at the summit too.

Thanks everyone
Rob

On 21 July 2016 at 17:36, David Lyle 
> wrote:
I think basing the plugins against master is the best solution since
the plugin in development will want to work with the matching horizon
release. Finding issues sooner and hopefully one at a time is a lot
easier to address than a bundle of them at the end of a release cycle.
Hopefully, Horizon doesn't break plugins on a regular basis. If we
are, then we are not being focused enough around supporting plugins by
maintaining compatibility and focused too much around what's easiest
for Horizon in the development process. Horizon is no longer at the
top of the stack, we should develop with that awareness. Of course,
breaking changes will happen, but they should be limited and isolated.

David

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder] Volume Drivers unit tests

2016-07-21 Thread yang, xing

Hi Ivan,

Thanks for sending this out.  Regarding the issue in the EMC VNX driver unit 
tests, it is tracked by this bug 
https://bugs.launchpad.net/cinder/+bug/1578986.  The driver was recently 
refactored so this is probably a new issue introduced by the refactor.  We are 
investigating this issue.

Thanks,
Xing



From: Ivan Kolodyazhny [e...@e0ne.info]
Sent: Thursday, July 21, 2016 1:02 PM
To: OpenStack Development Mailing List
Subject: [openstack-dev] [cinder] Volume Drivers unit tests

Hi team,

First of all, I would like to apologize, if my mail is be too emotional. I 
spent too much of time to fix it and failed.

TL;DR;

What I want to say is: "Let's spend some time to make our tests better and fix 
all issues". Patch [1] is still unstable. Unit tests can pass or fail in a in a 
random order. Also, I've disabled some tests to pass CI.


Long version:

While I was working on patch "Move drivers unit tests to unit.volume.drivers 
directory" [1] I've found a lot of issues with our unit tests :(. Not all of 
them are already fixed, so that patch is still in progress

What did I found and what should we have to fix:

1) Execution time [2]. I don't want to argue what it unit tests, but 2-4 
seconds per tests should be non-acceptable, IMO.

2) Execution order. Seriously, do you know that our tests will fail or hang if 
execution order will change? Even if one test for diver A failed, some tests 
for driver B will fail too.

3) Lack of mock. It's a root cause for #2. We didn't mock sleeps and event 
loops right. We don't mock RPC call well too [3]. We don't have 
'cinder.openstack.common.rpc.impl_fake' module in Cinder tree.

In some drivers, we use oslo_service.loopingcall.FixedIntervalLoopingCall [4]. 
We've go ZeroIntervalLoopingCall [5] class in Cinder. Do we use it everywhere 
or mock FixedIntervalLoopingCall right? I don't think so, I've hacked 
oslo_service in my env to rise an exception if interval > 0. 297 tests failed. 
It means, our tests use sleep. We have to get rid of this. TBH, not only volume 
drivers unit tests failed. E.g. some API unit tests failed too.


4) Due to #3, sometimes unit tests hangs even on master branch with a minor 
changes.If I stop execution of such tests, usually I see something like [6]. In 
most of cases I see that following drivers' tests hangs: EMC, Huawei, Dell and 
RBD.

It's hard to debug such failures because the lack of tooling for eventlet 
debugging. Eventlet backdoors and gdb-python helps a bit. Maybe somebody know 
better solution for it.

[1] https://review.openstack.org/#/c/320148/
[2] http://paste.openstack.org/show/539081/
[3] https://github.com/openstack/cinder/search?utf8=%E2%9C%93=impl_fake
[4] use 
https://github.com/openstack/oslo.service/blob/master/oslo_service/loopingcall.py#L162
[5] 
https://github.com/openstack/cinder/blob/cfbb5bde4d9b37c39f6813fe685f987f8a990483/cinder/tests/unit/utils.py#L289
[6] http://paste.openstack.org/show/539090/


Regards,
Ivan Kolodyazhny,
http://blog.e0ne.info/
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Reminder to move newton specs to ocata directory by 8/4


On 7/21/2016 10:31 AM, Matt Riedemann wrote:

There are still quite a few specs up for review in the newton directory
in the nova-specs repo. Those should be moved to the ocata directory if
you plan on pursuing those for the Ocata release (keeping in mind we're
not actively reviewing specs for Ocata - but that doesn't mean you can't
push up a spec to get early feedback from interested parties).

For those that don't move by August 4th are subject to being abandoned.



This is for *non-priority* specs only. After 8/4 anything left over 
moves, priority or not.


--

Thanks,

Matt Riedemann


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [oslo.db] [CC neutron] CIDR overlap functionality and constraints

On Tue, Jul 19, 2016 at 7:40 AM, Mike Bayer  wrote:

> Oslo.db devs :
>
> We've developed a system by which CIDR math, such as that of detecting
> region overlaps, can be performed on a MySQL database within queries [1]
> [2].   This feature makes use of a custom stored function I helped to
> produce which provides functionality similar to that which Postgresql
> provides built in [3].   SQLite also supports a simple way to add CIDR math
> functions as well which I've demonstrated at [4].
>
> Note that I use the term "function" and not "procedure" to stress that
> this is not a "stored procedure" in the traditional sense of performing
> complex business logic and persistence operations - this CIDR function
> performs a calculation that is not at all specific to Openstack, and is
> provided already by other databases as a built-in, and nothing else.
>
> The rationale for network-math logic being performed in the relational
> database is so that SQL like SELECT, UPDATE, and INSERT can make use of
> CIDR overlaps and other network math, such as to locate records that
> correspond to network ranges in some way and of course to provide guards
> and constraints, like that of concurrent UPDATE statements against
> conflicting ranges as well as being able to produce INSERT constraints for
> similar reasons.   Both MySQL and Postgresql have support for network
> number functions, Postgresql just has a lot more.
>
> The INSERT constraint problem is also addressed by our patch and makes use
> of an INSERT trigger on MySQL [5], but on Postgresql we use a GIST index
> which has been shown to be more reliable under concurrent use than a
> trigger on this backend [6].
>
> Not surprisingly, there's a lot of verbosity to both the production of the
> MySQL CIDR overlap function and the corresponding trigger and constraint,
> as well as the fact that to support the addition of these functions /
> constraints at both the Alembic migration level as well as that of the
> model level (because we would like metadata.create_all() to work), they are
> currently stated twice within this patch within their full verbosity.
> This is sub-optimal, and while the patch here makes use of an Alembic
> recipe [7] to aid in the maintenance of special DDL constructs, it's adding
> lots of burden to the Neutron codebase that could be better stated
> elsewhere.
>
> The general verbosity and unfamiliarity of these well known SQL features
> is understandably being met with trepidation.  I've identified that this
> trepidation is likely rooted in the fact that unlike the many other
> elaborate SQL features we use like ALTER TABLE, savepoints, subqueries,
> SELECT FOR UPDATE, isolation levels, etc. etc., there is no warm and fuzzy
> abstraction layer here that is both greatly reducing the amount of explicit
> code needed to produce and upgrade the feature, as well as indicating that
> "someone else" will fix this system when it has problems.
>
> Rather than hobbling the entire Openstack ecosystem to using a small
> subset of what our relational databases are capable of, I'd like to propose
> that preferably somewhere in oslo.db, or elsewhere, we begin providing the
> foundation for the use of SQL features that are rooted in mechanisms such
> as triggers and small use of stored functions, and more specifically begin
> to produce network-math SQL features as the public API, starting with this
> one.
>

Mike,

This is pretty cool, I'll admit. I enjoyed looking through and learning
about some modern capabilities in Postgres. The thing is, I can only think
of one area in Neutron's API which would benefit from this. That is subnet
pools. Specifically, these operations could benefit:

- Create a subnet from a subnet pool.
- It would be helpful for the database to check overlap with other
subnets already allocated from the same pool. Now, we have to do a select
to check for overlap and then an insert later. Obviously, we've had to work
out a way to avoid races during the time between select and update.

- Adding a subnet pool to an address scope or updating a subnet pool
already under an address scope. These operations require that all of the
various subnet pools not have any mutually overlapping IP space.

These operations have been working pretty well. As I recall, it was tricky
to get them right to avoid races, but I think they're working correctly
now. There is one more operation that we are looking to enable in Newton
that could also benefit.

- Adopting a subnet in to an exist subnet pool.

None of these operations are expected to be very contentious and
performance hasn't really been a concern yet. If it were a big concern, I'd
be very interested in the GiST index solution because, as I understand it,
detecting overlap without that capability requires a linear search through
the existing records. But, GiST index capability isn't ubiquitous which
makes it difficult to get excited about for practical purposes. I do have
an

Re: [openstack-dev] [telemetry] Mascot

2016-07-21 Thread Ildikó Váncsa

I had the Meerkat [1] in mind as Telemetry has a "family" of services and 
meerkat lives in bigger groups too and a few of them stand sentry and listen to 
any event or danger, etc. and "send alarms" to the others.

From the previous options I like Fennec too.

/Ildiko

[1] https://en.wikipedia.org/wiki/Meerkat 

> -Original Message-
> From: Chris Dent [mailto:cdent...@anticdent.org]
> Sent: July 21, 2016 19:47
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: Re: [openstack-dev] [telemetry] Mascot
> 
> On Thu, 21 Jul 2016, gordon chung wrote:
> > On 21/07/2016 11:36 AM, Doug Hellmann wrote:
> >> Something with big ears that are good for listening. A fennec [1]? A
> >> serval [2]?
> >>
> >> Doug
> >>
> >> [1] https://en.wikipedia.org/wiki/Fennec_fox
> >> [2] https://en.wikipedia.org/wiki/Serval
> >
> > i really like these suggestions, with a preference for the fennec. the
> > big ears idea was great.
> 
> +1 on the fennec
> 
> --
> Chris Dent   ┬─┬ノ( º _ ºノ) http://anticdent.org/
> freenode: cdent tw: @anticdent
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [app-catalog] App Catalog mascot

2016-07-21 Thread Jeremy Stanley

On 2016-07-21 10:48:28 -0700 (-0700), Christopher Aedo wrote:
> It seems our etherpad disappeared into the ether, so let's just finish
> the voting here on the mailing list.  I'll list what I remember seeing
> on the pad, and if you're interested in voting on the app-catalog
> mascot please respond to this thread with your favorite creature.

I would like to figure out what's happening to occasionally jack up
pads (not sure if it's something that gets added in the content that
breaks loading or what, I haven't had time to dig into it myself).
Worth noting though, you can find your old content in the pad
history, like:

https://etherpad.openstack.org/p/app-catalog-mascot/timeslider

-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Openstack] OpenStack Newton B2 for Ubuntu 16.04 LTS and Ubuntu 16.10

2016-07-21 Thread Corey Bryant

Hi All,

The Ubuntu OpenStack team is pleased to announce the general availability
of OpenStack Newton B2 milestone in Ubuntu 16.10 and for Ubuntu 16.04 LTS
via the Ubuntu Cloud Archive.

Ubuntu 16.04 LTS


You can enable the Ubuntu Cloud Archive pocket for OpenStack Newton on
Ubuntu 16.04 installations by running the following commands:

sudo add-apt-repository cloud-archive:newton
sudo apt update

The Ubuntu Cloud Archive for Newton includes updates for Barbican,
Ceilometer, Cinder, Congress, Designate, Glance, Heat, Horizon, Ironic
(6.0.0), Keystone, Manila, Murano, Murano-Agent, Neutron, Neutron-FWaaS,
Neutron-LBaaS, Neutron-VPNaaS, Nova, Sahara, Senlin, Trove, Swift (2.9.0),
and Zaqar.

You can see the full list of packages and versions at [0].

Ubuntu 16.10


No extra steps required; just start installing OpenStack!

Branch Package Builds
-

If you want to try out the latest master branch updates, or updates to
stable branches, we are maintaining continuously integrated packages in the
following PPA’s:

   sudo add-apt-repository ppa:openstack-ubuntu-testing/liberty
   sudo add-apt-repository ppa:openstack-ubuntu-testing/mitaka
   sudo add-apt-repository ppa:openstack-ubuntu-testing/newton

bear in mind these are built per-commitish (30 min checks for new commits
at the moment) so ymmv from time-to-time.

Reporting bugs
--

Any issues please report bugs using the 'ubuntu-bug' tool:

  sudo ubuntu-bug nova-conductor

this will ensure that bugs get logged in the right place in Launchpad.

Thanks and have fun!

Cheers,

Corey
(on behalf of the Ubuntu OpenStack team)

[0]
http://reqorts.qa.ubuntu.com/reports/ubuntu-server/cloud-archive/newton_versions.html


-- 
Regards,
Corey
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [app-catalog] App Catalog mascot

2016-07-21 Thread Fox, Kevin M

I like the cat idea. The app cat has a very nice ring to it. :)

From: Christopher Aedo [d...@aedo.net]
Sent: Thursday, July 21, 2016 10:48 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [app-catalog] App Catalog mascot

On Thu, Jul 14, 2016 at 11:20 AM, Christopher Aedo  wrote:
> Today we decided to set up an etherpad for picking a mascot for the
> Community App Catalog.  If you'd like to propose an idea or vote on
> one of the ones already up there, please check out the etherpad -
> thanks!
>
> https://etherpad.openstack.org/p/app-catalog-mascot

It seems our etherpad disappeared into the ether, so let's just finish
the voting here on the mailing list.  I'll list what I remember seeing
on the pad, and if you're interested in voting on the app-catalog
mascot please respond to this thread with your favorite creature.

Quokka - cute but pretty obscure

Kangaroo - could have a pouch full of apps

Cat - for "cat"alog

I personally vote for Kangaroo :)

-Christopher

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [app-catalog] App Catalog mascot

2016-07-21 Thread Christopher Aedo

On Thu, Jul 14, 2016 at 11:20 AM, Christopher Aedo  wrote:
> Today we decided to set up an etherpad for picking a mascot for the
> Community App Catalog.  If you'd like to propose an idea or vote on
> one of the ones already up there, please check out the etherpad -
> thanks!
>
> https://etherpad.openstack.org/p/app-catalog-mascot

It seems our etherpad disappeared into the ether, so let's just finish
the voting here on the mailing list.  I'll list what I remember seeing
on the pad, and if you're interested in voting on the app-catalog
mascot please respond to this thread with your favorite creature.

Quokka - cute but pretty obscure

Kangaroo - could have a pouch full of apps

Cat - for "cat"alog

I personally vote for Kangaroo :)

-Christopher

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] moving tripleo-ci/test-environments into THT

2016-07-21 Thread Steven Hardy

On Thu, Jul 21, 2016 at 01:30:37PM -0400, James Slagle wrote:
> On Thu, Jul 21, 2016 at 1:16 PM, Ben Nemec  wrote:
> > On 07/21/2016 11:31 AM, Steven Hardy wrote:
> >> On Wed, Jul 20, 2016 at 10:32:23PM -0400, Emilien Macchi wrote:
> >>> Hi,
> >>>
> >>> We're currently using tripleo-ci to store our test-environments files
> >>> (ie: multinode.yaml, etc).
> >>> To make it compatible with our different versions of TripleO, we have 2 
> >>> options:
> >>>
> >>> * Duplicate templates and use bash conditionals in tripleo-ci scripts
> >>> to select which one we want at each release.
> >>> * Move them to THT (THT is branched & released).
> >>>
> >>> I would vote for option #2 for 2 reasons:
> >>> * we don't have to do complex conditionals in tripleo-ci
> >>> * we can easily consume it outside tripleo-ci (oooq one day?)
> >>> * we can easily make them evolve, when new composable services are
> >>> created for example.
> >>
> >> +1 I agree it's probably best to move these to t-h-t, although we should
> >> clearly identify those environments which are specific to CI setups (such
> >> as where we override the number of workers to minimise resource usage).
> >
> > Aren't they all specific to CI setups?  If not, they didn't belong in
> > tripleo-ci in the first place.
> >>
> >> I think maintaining them in tripleo-ci will prove inconvenient in the long
> >> term, e.g https://review.openstack.org/#/c/338551/ is failing now because
> >> we now have coupling between the ControllerServices parameter default
> >> and the multinode HA job.
> >
> > That's concerning.  I suppose it's a temporary problem while we finish
> > converting all of the existing services to composable format?  After
> > that service additions shouldn't be coupled as tightly.  They wouldn't
> > be tested by the multinode job until they're added to the list, but they
> > won't break the job either.
> >
> > In any case, the ControllerServices bit of that template seems like a
> > candidate for including in t-h-t as environments/allinone.yaml or
> > something.  I would say that's generally useful to everyone so it's
> > sensible to include in t-h-t.
> >
> > I'm less in favor of moving _everything_ out of tripleo-ci though.
> > tripleo-ci is essentially a user of t-h-t, which means it's the first
> > place we're going to find interface breakages.  If we do something that
> > breaks the existing "user" templates in tripleo-ci then we're causing
> > that same pain to our users and I think it's a good thing if it's also
> > painful for us.
> >
> > So call this a +1/-1.  I'm good with moving the generic bits of the
> > multinode job out, but I think the rest should stay in tripleo-ci.
> 
> I'm fine to move them.
> 
> But I agree with you that we should be careful about how that might
> mask issues in our user experience. ControllerServices is like
> ServiceNetMap in this regard. They are both really big large value
> lists. As a user, if you choose to modify that, it's now up to you to
> keep your modifications in sync with any changes done to the default
> value. If we end up adding a new service that ends up being required,
> you're not going to have that in your list if you've overridden it (as
> the swift patch shows).

This is true, but in this case I think we could convert the environment
defining ControllerServices for the multinode job to a more generic
sample environment, e.g environments/all_in_one.yaml or something?

> Perhaps we could add some additional interfaces such as
> ControllerServicesAdditions/ControllerServicesRemovals that operated
> on the default value of ControllerServices somehow.

Yes, I think both of these will be possible, the additions requirement will
be enabled by this heat spec, which Rabi is currently implementing:

https://review.openstack.org/#/c/330414/

When that lands, we'll be able to do something like this to append to e.g
ControllerServices in the various environment files:

https://etherpad.openstack.org/p/tripleo-heat-merge-strategies

For the ControllerServicesRemovals it's a bit less clear - we definitely
could add a blacklist parameter like this (which again could be appended
via the environment merging described above), and then use yaql to
calculate the subset from ControllerServices, but I'm thinking if there's a
cleaner way (perhaps an extra step beyond the "add" merge strategy, or
another heat function that can calculate the subset, "list_subtract"?).

Thanks,

Steve

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [telemetry] Mascot

2016-07-21 Thread Chris Dent


On Thu, 21 Jul 2016, gordon chung wrote:

On 21/07/2016 11:36 AM, Doug Hellmann wrote:

Something with big ears that are good for listening. A fennec [1]? A
serval [2]?

Doug

[1] https://en.wikipedia.org/wiki/Fennec_fox
[2] https://en.wikipedia.org/wiki/Serval


i really like these suggestions, with a preference for the fennec. the
big ears idea was great.


+1 on the fennec

--
Chris Dent   ┬─┬ノ( º _ ºノ) http://anticdent.org/
freenode: cdent tw: @anticdent__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [telemetry] Mascot

2016-07-21 Thread gordon chung



On 21/07/2016 11:36 AM, Doug Hellmann wrote:
> Something with big ears that are good for listening. A fennec [1]? A
> serval [2]?
>
> Doug
>
> [1] https://en.wikipedia.org/wiki/Fennec_fox
> [2] https://en.wikipedia.org/wiki/Serval

i really like these suggestions, with a preference for the fennec. the 
big ears idea was great.

i don't mind Galagos[1] either but i think i would still vote for the fennec

[1] https://en.wikipedia.org/wiki/Galago

cheers,

-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [horizon] Midcycle Summary

2016-07-21 Thread Rob Cresswell

Hi everyone,

We had the Horizon mid cycle meetup last week, and I wanted to highlight some 
of the discussion and decisions made.

Agenda: https://etherpad.openstack.org/p/horizon-newton-midcycle
Notes: https://etherpad.openstack.org/p/horizon-newton-midcycle-notes

- Deprecation of table inline edit: This was discussed at the summit as a 
potential item for cleanup. Inline edit in tables has only been implemented in 
a few places, but breaks often in both functionality and styling. The 
maintenance overhead was seen as bigger than the small advantages of using it, 
so we're in the process of removing the implementations in the Dashboard code 
and deprecating the framework parts.
https://review.openstack.org/#/c/343861/

- OSProfiler integration: OSProfiler (https://github.com/openstack/osprofiler) 
is a small library that traces requests as they pass through the stack. We're 
working on exposing this as information in a developer dashboard panel so that 
devs can optimise their API calls and discover bottlenecks.

- Angular tables with Django actions: One of the highlighted issues with 
angular adoption is the "all-or-nothing" approach as a panel has really needed 
to be fully angular to feel the benefits in responsiveness. We've investigating 
launching the existing Python actions from Angular tables, so that we can move 
to client side search and pagination faster. The python tables will follow the 
usual deprecation process (minimum of two cycles).

- Glance V2 support: This is a critical requirement for this cycle. We've come 
up against some issues with features differing from V1, and have generally 
resolved to avoid working around missing API calls to reduce any hacks within 
Horizon and use the API as intended.

- Schema form: Angular workflows in their current state represent a nice 
improvement in responsive validation and quicker load times. However, the 
implementation is a quite verbose, has a couple of workarounds we'd like to 
remove, and generally requires quite in depth knowledge of AngularJS to use 
well. This is not ideal, as we'd like to build a system existing Horizon devs 
and plugins can extend quickly without learning a new framework. We're in the 
process of adopting a library that allows us to define complex forms and 
workflows in a JSON structure, which should make it much easier to take 
advantage of the new features. https://review.openstack.org/#/c/332745/

Overall, the mid cycle was a great success, and I'd like to thank everyone for 
attending! If anyone has any questions, please feel free to email or ping me on 
IRC (robcresswell)

Rob
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova] Reminder to move newton specs to ocata directory by 8/4

There are still quite a few specs up for review in the newton directory 
in the nova-specs repo. Those should be moved to the ocata directory if 
you plan on pursuing those for the Ocata release (keeping in mind we're 
not actively reviewing specs for Ocata - but that doesn't mean you can't 
push up a spec to get early feedback from interested parties).


For those that don't move by August 4th are subject to being abandoned.

--

Thanks,

Matt Riedemann


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] moving tripleo-ci/test-environments into THT

2016-07-21 Thread James Slagle

On Thu, Jul 21, 2016 at 1:16 PM, Ben Nemec  wrote:
> On 07/21/2016 11:31 AM, Steven Hardy wrote:
>> On Wed, Jul 20, 2016 at 10:32:23PM -0400, Emilien Macchi wrote:
>>> Hi,
>>>
>>> We're currently using tripleo-ci to store our test-environments files
>>> (ie: multinode.yaml, etc).
>>> To make it compatible with our different versions of TripleO, we have 2 
>>> options:
>>>
>>> * Duplicate templates and use bash conditionals in tripleo-ci scripts
>>> to select which one we want at each release.
>>> * Move them to THT (THT is branched & released).
>>>
>>> I would vote for option #2 for 2 reasons:
>>> * we don't have to do complex conditionals in tripleo-ci
>>> * we can easily consume it outside tripleo-ci (oooq one day?)
>>> * we can easily make them evolve, when new composable services are
>>> created for example.
>>
>> +1 I agree it's probably best to move these to t-h-t, although we should
>> clearly identify those environments which are specific to CI setups (such
>> as where we override the number of workers to minimise resource usage).
>
> Aren't they all specific to CI setups?  If not, they didn't belong in
> tripleo-ci in the first place.
>>
>> I think maintaining them in tripleo-ci will prove inconvenient in the long
>> term, e.g https://review.openstack.org/#/c/338551/ is failing now because
>> we now have coupling between the ControllerServices parameter default
>> and the multinode HA job.
>
> That's concerning.  I suppose it's a temporary problem while we finish
> converting all of the existing services to composable format?  After
> that service additions shouldn't be coupled as tightly.  They wouldn't
> be tested by the multinode job until they're added to the list, but they
> won't break the job either.
>
> In any case, the ControllerServices bit of that template seems like a
> candidate for including in t-h-t as environments/allinone.yaml or
> something.  I would say that's generally useful to everyone so it's
> sensible to include in t-h-t.
>
> I'm less in favor of moving _everything_ out of tripleo-ci though.
> tripleo-ci is essentially a user of t-h-t, which means it's the first
> place we're going to find interface breakages.  If we do something that
> breaks the existing "user" templates in tripleo-ci then we're causing
> that same pain to our users and I think it's a good thing if it's also
> painful for us.
>
> So call this a +1/-1.  I'm good with moving the generic bits of the
> multinode job out, but I think the rest should stay in tripleo-ci.

I'm fine to move them.

But I agree with you that we should be careful about how that might
mask issues in our user experience. ControllerServices is like
ServiceNetMap in this regard. They are both really big large value
lists. As a user, if you choose to modify that, it's now up to you to
keep your modifications in sync with any changes done to the default
value. If we end up adding a new service that ends up being required,
you're not going to have that in your list if you've overridden it (as
the swift patch shows).

Perhaps we could add some additional interfaces such as
ControllerServicesAdditions/ControllerServicesRemovals that operated
on the default value of ControllerServices somehow.

-- 
-- James Slagle
--

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] moving tripleo-ci/test-environments into THT

2016-07-21 Thread Emilien Macchi

On Thu, Jul 21, 2016 at 1:16 PM, Ben Nemec  wrote:
> On 07/21/2016 11:31 AM, Steven Hardy wrote:
>> On Wed, Jul 20, 2016 at 10:32:23PM -0400, Emilien Macchi wrote:
>>> Hi,
>>>
>>> We're currently using tripleo-ci to store our test-environments files
>>> (ie: multinode.yaml, etc).
>>> To make it compatible with our different versions of TripleO, we have 2 
>>> options:
>>>
>>> * Duplicate templates and use bash conditionals in tripleo-ci scripts
>>> to select which one we want at each release.
>>> * Move them to THT (THT is branched & released).
>>>
>>> I would vote for option #2 for 2 reasons:
>>> * we don't have to do complex conditionals in tripleo-ci
>>> * we can easily consume it outside tripleo-ci (oooq one day?)
>>> * we can easily make them evolve, when new composable services are
>>> created for example.
>>
>> +1 I agree it's probably best to move these to t-h-t, although we should
>> clearly identify those environments which are specific to CI setups (such
>> as where we override the number of workers to minimise resource usage).
>
> Aren't they all specific to CI setups?  If not, they didn't belong in
> tripleo-ci in the first place.
>>
>> I think maintaining them in tripleo-ci will prove inconvenient in the long
>> term, e.g https://review.openstack.org/#/c/338551/ is failing now because
>> we now have coupling between the ControllerServices parameter default
>> and the multinode HA job.
>
> That's concerning.  I suppose it's a temporary problem while we finish
> converting all of the existing services to composable format?  After
> that service additions shouldn't be coupled as tightly.  They wouldn't
> be tested by the multinode job until they're added to the list, but they
> won't break the job either.
>
> In any case, the ControllerServices bit of that template seems like a
> candidate for including in t-h-t as environments/allinone.yaml or
> something.  I would say that's generally useful to everyone so it's
> sensible to include in t-h-t.
>
> I'm less in favor of moving _everything_ out of tripleo-ci though.
> tripleo-ci is essentially a user of t-h-t, which means it's the first
> place we're going to find interface breakages.  If we do something that
> breaks the existing "user" templates in tripleo-ci then we're causing
> that same pain to our users and I think it's a good thing if it's also
> painful for us.
>
> So call this a +1/-1.  I'm good with moving the generic bits of the
> multinode job out, but I think the rest should stay in tripleo-ci.
>

Your feedback is really good, I'll do baby steps and add you as
reviewer when I'll start the work, so we can iterate on the things we
want to move and keep in place things we don't.
-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] place of subport details in the API

2016-07-21 Thread Armando M.

On 21 July 2016 at 04:56, Bence Romsics  wrote:

> Hi,
>
> Looking at all the trunk port patches nicely coming along I was
> wondering where do we want to see all the subport details in the API?
>
> In the spec a trunk object does not have a 'sub_ports' attribute:
>
>
> http://specs.openstack.org/openstack/neutron-specs/specs/newton/vlan-aware-vms.html#rest-api-impact


This was added so that you can create trunk with sub_ports at once and
avoid two API calls.


>
>
> Instead subports can be added/removed/queried via extra resource actions
> like:
>

This is not either/or, you can do both: create a trunk just with a parent
port and add sub_ports at a later date, or do both at once.


>
> PUT /v2.0/trunks/$trunk_id/add_subports
> PUT /v2.0/trunks/$trunk_id/remove_subports
> GET /v2.0/trunks/$trunk_id/get_subports
>
> IIRC the reasoning was to give control to the API user if/when he
> wants to see the subport details, because that may be huge (for up to
> 4k subports).
>

Payloads at scale can be big, have you ever tried to list a network with a
gazillion subnets?


>
> The current patches mostly go the other way, having a sub_ports
> attribute on the trunk. Consistent with that
> /v2.0/trunks/$trunk_id/get_subports is not implemented.
>
>
I don't think so, I think the rationale is to make the API less chatty than
it needs to be. The spec was particularly lax about this, and it did not
prescribe any behavior so I would not state that we went the other way.


> So which way do we want this?


I think the way it is now is what we want.


> If we stick to the currently implemented approach I guess we can drop
> the get_subports action, right?
>

The get_subports may seem redundant, but I'd avoid an overly aggressive
optimization at this point. The CLI can nicely show the ports in tabular
form when invoked.


>
> Cheers,
> Bence Romsics
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [OSSN 0068] Repeated token revocation requests, can lead to service degradation or disruption

2016-07-21 Thread Luke Hinds

Repeated token revocation requests, can lead to service degradation or
disruption
---

### Summary ###
There is currently no limit to the frequency of keystone token
revocations that can be made by a single user, in any given time frame.

If a user repeatedly makes token requests, and then immediately revokes
the token, a performance degradation can occur and possible DoS (Denial
of Service) attacks could be directed towards keystone.

### Affected Services / Software ###
All services using keystone.

Mitaka, Liberty, Kilo, Nova, Juno, Havana, Icehouse, Grizzly, Folsom, Essex.

### Discussion ###
Token revocation can be self-served, with no restrictions enforced on
the number of token revocations made by any user (including service users).

If token revocations are made in quick succession, response times starts
to lengthen, due to the increasing entries made in the revocation_event
table.

With no form of rate limiting in place, a single user can cause the
OpenStack auth service to become poor in response time, resulting in a
DoS style attack.

A cleanup of revocation events does occur, based on token expiration
plus expiration_buffer (which is 30 minutes by default). However, with
the default token TTL of 3600 seconds, a user can potentially fill up
approximately several thousand events during that time.

### Recommended Actions ###
For current stable OpenStack releases (Mitaka and previous), operators
are recommended to deploy external rate-limiting proxies or web
application firewalls, to provide a front layer of protection to keystone.

The following solutions may be considered, however it is key that the
operator carefully plans and considers the individual performance needs
of users and services within their OpenStack cloud, when configuring any
rate limiting functionality.

 Repose 

# Rate Limiting Filter #
Repose provides a rate limiting filter, that can limit per IP address
and to a specific HTTP method (DELETE in relation to this OSSN).

The following config may be considered for a single node. For more
complex deployments, clusters can be constructed , utilizing a
distributed data-store.

# system-model.cfg.xml #
~~~


http://docs.openrepose.org/repose/system-model/v2.0;>











http://idenity-server.acme.com; root-path="/" port="35357"
default="true"/>





~~~

# ip-user.cfg.xml #
~~~

http://docs.openrepose.org/repose/ip-user/v1.0;>



192.168.0.0/24


0.0.0.0/0
0::0/0


~~~

* Note: Using the ip-user filter, will mean each IP address sending
requests to repose, will have its own rate-limit bucket. Therefore any
IP exceeding the limit, will be blocked - but only that IP. If you are
sending NAT'ed connections to repose, then you should consider, they
will also be seen as a single IP, and grouped accordingly.

# rate-limiting.cfg.xml #
~~~

http://docs.openrepose.org/repose/rate-limiting/v1.0;>











~~~

* Key points to note with the above. The rate limit is limited to DELETE
requests (which is the http method used to revoke a token), and to the
URI /auth/token. Any IP which exceeds 10 revoke requests per minute,
will be blocked for 1 minute.

Further details can be found on the openrepose wiki:

https://repose.atlassian.net/wiki/display/REPOSE/Rate+Limiting+filter

### Other possible solutions ###

 NGINX 
NGINX provides the limit_req_module, which can be used to provide a
global rate
limit. Using a map, it can be limited to just the DELETE method.

~~~
http {
  map $request_method $keystone {
  default "";
  DELETE$binary_remote_addr;
  }
  limit_req_zone $keystone zone=keystone:10m rate=10r/m;

  server {
 ...
 location /auth/token {
 limit_req zone=keystone;
 ...
   }
}
~~~

Further details can be found on the nginx site:

http://nginx.org/en/docs/http/ngx_http_limit_req_module.html

 HAProxy 

HAProxy can provide inherent rate-limiting, using stick-tables, with a
General Purpose Counter (gpc)

~~~
# Monitors the number of request sent by an IP over a period of 10 seconds
stick-table type ip size 1m expire 10s store gpc0,http_req_rate(10s)
tcp-request connection track-sc1 src
tcp-request connection reject if { src_get_gpc0 gt 0 }
~~~

Further details can be found on the haproxy website:

http://blog.haproxy.com/2012/02/27/use-a-load-balancer-as-a-first-row-of-defense-against-ddos)

 Apache 
A number of solutions can be explored here.

# mod_ratelimit #
http://httpd.apache.org/docs/2.4/mod/mod_ratelimit.html

# mod_qos #
http://opensource.adnovum.ch/mod_qos/dos.html

# mod_evasive #

Re: [openstack-dev] [tripleo] moving tripleo-ci/test-environments into THT

2016-07-21 Thread Ben Nemec

On 07/21/2016 11:31 AM, Steven Hardy wrote:
> On Wed, Jul 20, 2016 at 10:32:23PM -0400, Emilien Macchi wrote:
>> Hi,
>>
>> We're currently using tripleo-ci to store our test-environments files
>> (ie: multinode.yaml, etc).
>> To make it compatible with our different versions of TripleO, we have 2 
>> options:
>>
>> * Duplicate templates and use bash conditionals in tripleo-ci scripts
>> to select which one we want at each release.
>> * Move them to THT (THT is branched & released).
>>
>> I would vote for option #2 for 2 reasons:
>> * we don't have to do complex conditionals in tripleo-ci
>> * we can easily consume it outside tripleo-ci (oooq one day?)
>> * we can easily make them evolve, when new composable services are
>> created for example.
> 
> +1 I agree it's probably best to move these to t-h-t, although we should
> clearly identify those environments which are specific to CI setups (such
> as where we override the number of workers to minimise resource usage).

Aren't they all specific to CI setups?  If not, they didn't belong in
tripleo-ci in the first place.
> 
> I think maintaining them in tripleo-ci will prove inconvenient in the long
> term, e.g https://review.openstack.org/#/c/338551/ is failing now because
> we now have coupling between the ControllerServices parameter default
> and the multinode HA job.

That's concerning.  I suppose it's a temporary problem while we finish
converting all of the existing services to composable format?  After
that service additions shouldn't be coupled as tightly.  They wouldn't
be tested by the multinode job until they're added to the list, but they
won't break the job either.

In any case, the ControllerServices bit of that template seems like a
candidate for including in t-h-t as environments/allinone.yaml or
something.  I would say that's generally useful to everyone so it's
sensible to include in t-h-t.

I'm less in favor of moving _everything_ out of tripleo-ci though.
tripleo-ci is essentially a user of t-h-t, which means it's the first
place we're going to find interface breakages.  If we do something that
breaks the existing "user" templates in tripleo-ci then we're causing
that same pain to our users and I think it's a good thing if it's also
painful for us.

So call this a +1/-1.  I'm good with moving the generic bits of the
multinode job out, but I think the rest should stay in tripleo-ci.

-Ben

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [tooz] DLM benchmark results

2016-07-21 Thread Joshua Harlow


Hi John,

Thanks for gathering this info,

Do you have the versions of the backend that were used here 
(particularly relevant for etcd which has a new release pretty frequently).


It'd be useful to capture that info also :)

John Schwarz wrote:

Hi everyone,

Following [1], a few of us sat down during the last day of the Austin
Summit and discussed the possibility of adding formal support for
Tooz, specifically for the locking mechanism it provides. The
conclusion we reached was that benchmarks should be done to show if
and how Tooz affects the normal operation of Neutron (i.e. if locking
a resource using Zookeeper takes 3 seconds, it's not worthwhile at
all).

We've finally finished the benchmarks and they are available at [2].
They test a specific case: when creating an HA router a lock-free
algorithm is used to assign a vrid to a router (this is later used for
keepalived), and the benchmark specifically checks the effects of
locking that function with either Zookeeper or Etcd, using the no-Tooz
case as a baseline. The locking was checked in 2 different ways - one
which presents no contention (acquire() always succeeds immediately)
and one which presents contentions (acquire() may block until a
similar process for the invoking tenant is complete).

The benchmarks show that while using Tooz does raise the cost of an
operation, the effects are not as bad as we initially feared. In the
simple, single simultaneous request, using Zookeeper raised the
average time it took to create a router by 1.5% (from 11.811 to 11.988
seconds). On the more-realistic case of 6 simultaneous requests,
Zookeeper raised the cost by 3.74% (from 16.533 to 17.152 seconds).

It is important to note that the setup itself was overloaded - it was
built on a single baremetal hosting 5 VMs (4 of which were
controllers) and thus we were unable to go further - for example, 10
concurrent requests overloaded the server and caused some race
conditions to appear in the L3 scheduler (bugs will be opened soon),
so for this reason we haven't tested heavier samples and limited
ourselves to 6 simultaneous requests.

Also important to note that some kind of race condition was noticed in
tooz's etcd driver. We've discussed this with the tooz devs and
provided a patch that is supposed to fix them [3].
Lastly, races in the L3 HA Scheduler were found and we are yet to dig
into them and find out their cause - bugs will be opened for these as
well.

I've opened the summary [2] for comments so you're welcome to open a
discussion about the results both in the ML and on the doc itself.

(CC to all those who attended the Austin Summit meeting and other
interested parties)
Happy locking,

[1]: http://lists.openstack.org/pipermail/openstack-dev/2016-April/093199.html
[2]: 
https://docs.google.com/document/d/1jdI8gkQKBE0G9koR0nLiW02d5rwyWv_-gAp7yavt4w8
[3]: https://review.openstack.org/#/c/342096/

--
John Schwarz,
Senior Software Engineer,
Red Hat.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [cinder] Volume Drivers unit tests

2016-07-21 Thread Ivan Kolodyazhny

Hi team,

First of all, I would like to apologize, if my mail is be too emotional. I
spent too much of time to fix it and failed.
TL;DR;

What I want to say is: "Let's spend some time to make our tests better and
fix all issues". Patch [1] is still unstable. Unit tests can pass or fail
in a in a random order. Also, I've disabled some tests to pass CI.

Long version:

While I was working on patch "Move drivers unit tests to
unit.volume.drivers directory" [1] I've found a lot of issues with our unit
tests :(. Not all of them are already fixed, so that patch is still in
progress

What did I found and what should we have to fix:

1) Execution time [2]. I don't want to argue what it unit tests, but 2-4
seconds per tests should be non-acceptable, IMO.

2) Execution order. Seriously, do you know that our tests will fail or hang
if execution order will change? Even if one test for diver A failed, some
tests for driver B will fail too.

3) Lack of mock. It's a root cause for #2. We didn't mock sleeps and event
loops right. We don't mock RPC call well too [3]. We don't
have 'cinder.openstack.common.rpc.impl_fake' module in Cinder tree.

In some drivers, we use oslo_service.loopingcall.FixedIntervalLoopingCall
[4]. We've go ZeroIntervalLoopingCall [5] class in Cinder. Do we use it
everywhere or mock FixedIntervalLoopingCall right? I don't think so, I've
hacked oslo_service in my env to rise an exception if interval > 0. 297
tests failed. It means, our tests use sleep. We have to get rid of this.
TBH, not only volume drivers unit tests failed. E.g. some API unit tests
failed too.

4) Due to #3, sometimes unit tests hangs even on master branch with a minor
changes.If I stop execution of such tests, usually I see something like
[6]. In most of cases I see that following drivers' tests hangs: EMC,
Huawei, Dell and RBD.

It's hard to debug such failures because the lack of tooling for eventlet
debugging. Eventlet backdoors and gdb-python helps a bit. Maybe somebody
know better solution for it.

[1] https://review.openstack.org/#/c/320148/
[2] http://paste.openstack.org/show/539081/
[3] https://github.com/openstack/cinder/search?utf8=%E2%9C%93=impl_fake
[4] use
https://github.com/openstack/oslo.service/blob/master/oslo_service/loopingcall.py#L162
[5]
https://github.com/openstack/cinder/blob/cfbb5bde4d9b37c39f6813fe685f987f8a990483/cinder/tests/unit/utils.py#L289
[6] http://paste.openstack.org/show/539090/

Regards,
Ivan Kolodyazhny,
http://blog.e0ne.info/
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] moving tripleo-ci/test-environments into THT

2016-07-21 Thread Emilien Macchi

On Thu, Jul 21, 2016 at 12:31 PM, Steven Hardy  wrote:
> On Wed, Jul 20, 2016 at 10:32:23PM -0400, Emilien Macchi wrote:
>> Hi,
>>
>> We're currently using tripleo-ci to store our test-environments files
>> (ie: multinode.yaml, etc).
>> To make it compatible with our different versions of TripleO, we have 2 
>> options:
>>
>> * Duplicate templates and use bash conditionals in tripleo-ci scripts
>> to select which one we want at each release.
>> * Move them to THT (THT is branched & released).
>>
>> I would vote for option #2 for 2 reasons:
>> * we don't have to do complex conditionals in tripleo-ci
>> * we can easily consume it outside tripleo-ci (oooq one day?)
>> * we can easily make them evolve, when new composable services are
>> created for example.
>
> +1 I agree it's probably best to move these to t-h-t, although we should
> clearly identify those environments which are specific to CI setups (such
> as where we override the number of workers to minimise resource usage).
>
> I think maintaining them in tripleo-ci will prove inconvenient in the long
> term, e.g https://review.openstack.org/#/c/338551/ is failing now because
> we now have coupling between the ControllerServices parameter default
> and the multinode HA job.
>

ack - I'll work on it and make it pass CI before end of Newton.
-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [ Cinder] HA Active-Active meeting Tuesdays at 1600 UTC in openstack-cinder

2016-07-21 Thread D'Angelo, Scott

We've decided to meet each week to discuss status, patches, and testing of 
Cinder-volume Active-Active HA:

Tuesdays 1600 UTC in openstack-cinder (same time as weekly meeting, one day 
earlier)


https://etherpad.openstack.org/p/cinder-active-active-HA

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] Plugin stability, failing tests etc.

2016-07-21 Thread David Lyle

I think basing the plugins against master is the best solution since
the plugin in development will want to work with the matching horizon
release. Finding issues sooner and hopefully one at a time is a lot
easier to address than a bundle of them at the end of a release cycle.
Hopefully, Horizon doesn't break plugins on a regular basis. If we
are, then we are not being focused enough around supporting plugins by
maintaining compatibility and focused too much around what's easiest
for Horizon in the development process. Horizon is no longer at the
top of the stack, we should develop with that awareness. Of course,
breaking changes will happen, but they should be limited and isolated.

David

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] moving tripleo-ci/test-environments into THT

2016-07-21 Thread Steven Hardy

On Wed, Jul 20, 2016 at 10:32:23PM -0400, Emilien Macchi wrote:
> Hi,
> 
> We're currently using tripleo-ci to store our test-environments files
> (ie: multinode.yaml, etc).
> To make it compatible with our different versions of TripleO, we have 2 
> options:
> 
> * Duplicate templates and use bash conditionals in tripleo-ci scripts
> to select which one we want at each release.
> * Move them to THT (THT is branched & released).
> 
> I would vote for option #2 for 2 reasons:
> * we don't have to do complex conditionals in tripleo-ci
> * we can easily consume it outside tripleo-ci (oooq one day?)
> * we can easily make them evolve, when new composable services are
> created for example.

+1 I agree it's probably best to move these to t-h-t, although we should
clearly identify those environments which are specific to CI setups (such
as where we override the number of workers to minimise resource usage).

I think maintaining them in tripleo-ci will prove inconvenient in the long
term, e.g https://review.openstack.org/#/c/338551/ is failing now because
we now have coupling between the ControllerServices parameter default
and the multinode HA job.

Thanks,

Steve

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Congress] Congress horizon plugin - congressclient/congress API auth issue - help

2016-07-21 Thread Tim Hinrichs

So clearly an authentication problem then.

Anusha, do you have any ideas?  (Aimee, I think Anusha has worked with
Keystone authentication most recently, so she's your best bet.)

Tim

On Thu, Jul 21, 2016 at 8:59 AM Aimee Ukasick 
wrote:

> The  Policy/Data Sources web page throws the same errors. I am
> planning to recheck direct API calls using v3 auth today or tomorrow.
>
> aimee
>
> On Thu, Jul 21, 2016 at 10:49 AM, Tim Hinrichs  wrote:
> > Hi Aimee,
> >
> > Do the other APIs work?  That is, is it a general problem
> authenticating, or
> > is the problem limited to list_policies?
> >
> > Tim
> >
> > On Wed, Jul 20, 2016 at 3:54 PM Aimee Ukasick <
> aimeeu.opensou...@gmail.com>
> > wrote:
> >>
> >> Hi all,
> >>
> >> I've been working on Policy UI (Horizon): Unable to get policies
> >> list (devstack) (https://bugs.launchpad.net/congress/+bug/1602837)
> >> for the past 3 days. Anusha is correct - it's an authentication
> >> problem, but I have not been able to fix it.
> >>
> >> I grabbed the relevant code in congress.py from Anusha's horizon
> >> plugin model patchset (https://review.openstack.org/#/c/305063/3) and
> >> added try/catch blocks, logging statements (with error because I
> >> haven't figured out how to set the horizon log level).
> >>
> >>
> >> I am testing the code on devstack, which I cloned on 19 July 2016.
> >>
> >> With both v2 and v3 auth, congressclient.v1.client is created.
> >> The failure happens trying to call
> >> congressclient.v1.client.Client.list_policies().
> >> When using v2 auth, the error message is "Unable to get policies list:
> >> The resource could not be found"
> >> When using v3 auth, the error message is "Cannot authorize API client"
> >>
> >> I am assuming that congressclient.v1.client.Client is
> >>
> >>
> https://github.com/openstack/python-congressclient/blob/master/congressclient/v1/client.py
> >> and that client.list_policy() calls list_policy()in the
> >> python-congressclient
> >> which in turn calls the Congress API. Is this correct?
> >>
> >> Any ideas why with v3 auth, the python-congressclient cannot authorize
> the
> >> call to the API?
> >>
> >> I looked at other horizon plugin models (ceilometer, neutron, nova,
> >> cerberus, cloudkitty, trove, designate, manila) to see how they created
> >> the client. While the code to create a client is not identical,
> >> it is vastly different from the code to create a client
> >> in contrib/horizon/congress.py.
> >>
> >> Thanks in advance for any pointers.
> >>
> >> aimee
> >>
> >> Aimee Ukasick (aimeeu)
> >>
> >> v2 log:
> >> 2016-07-20 22:13:56.501455
> >> 2016-07-20 22:14:30.238233 * view.get_data calling policies =
> >> congress.policies_list(self.request) *
> >> 2016-07-20 22:14:30.238318 * self.request.path=
> >> /dashboard/admin/policies/
> >> 2016-07-20 22:14:30.238352 * congress.policies_list(request)
> >> BEGIN*
> >> 2016-07-20 22:14:30.238376 * calling client =
> >> congressclient(request)*
> >> 2016-07-20 22:14:30.238399 * congress.congressclient BEGIN*
> >> 2016-07-20 22:14:30.238454 * auth_url=
> http://192.168.56.103/identity
> >> 2016-07-20 22:14:30.238479 * calling get_keystone_session *
> >> 2016-07-20 22:14:30.238505 * congress.get_keystone_session BEGIN
> >> auth_url *http://192.168.56.103/identity
> >> 2016-07-20 22:14:30.238554 * path= /identity
> >> 2016-07-20 22:14:30.238578 * using V2 plugin to authenticate*
> >> 2016-07-20 22:14:30.238630 * v2 auth.get_auth_state=
> >> 2016-07-20 22:14:30.238656 None
> >> 2016-07-20 22:14:30.238677 * finished using V2 plugin to
> >> authenticate*
> >> 2016-07-20 22:14:30.238698 * creating session with auth *
> >> 2016-07-20 22:14:30.244407 * congress.get_keystone_session END*
> >> 2016-07-20 22:14:30.244462 * regtion_name= RegionOne
> >> 2016-07-20 22:14:30.244491 * calling
> congress_client.Client(**kwargs)
> >> 2016-07-20 22:14:30.247830 * congress.congressclient END*
> >> 2016-07-20 22:14:30.247902 * calling policies_list =
> >> client.list_policy()*
> >> 2016-07-20 22:14:30.248012 DEBUG:keystoneauth.identity.v2:Making
> >> authentication request to http://192.168.56.103/identity/tokens
> >> 2016-07-20 22:14:30.255023 DEBUG:keystoneauth.session:Request returned
> >> failure status: 404
> >> 2016-07-20 22:14:30.257546 Unable to get policies list: The resource
> >> could not be found.
> >>
> >>
> >> v3 log:
> >> 2016-07-20 22:09:22.912969
> >> 2016-07-20 22:09:31.907119 * view.get_data calling policies =
> >> congress.policies_list(self.request) *
> >> 2016-07-20 22:09:31.907973 * self.request.path=
> >> /dashboard/admin/policies/
> >> 2016-07-20 22:09:31.908122 * congress.policies_list(request)
> >> BEGIN*
> >> 2016-07-20 22:09:31.908250 * calling client =
> >> congressclient(request)*
> >> 2016-07-20 22:09:31.908386 * congress.congressclient BEGIN*
>

Re: [openstack-dev] [openstack-ansible] Project Mascot

2016-07-21 Thread Major Hayden

On 07/17/2016 10:48 PM, Carter, Kevin wrote:
> A little out of band from the meeting but maybe an "Osa Eucharitid" [
> http://www.petsfoto.com/insect-world/#foobox-1/0/Insect-Life1.jpg ]?
> 
> However the Cape Buffalo would be good too.

Join in the discussion in the Etherpad:

  https://etherpad.openstack.org/p/osa-mascot

--
Major Hayden

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] Logo

2016-07-21 Thread Brad Pokorny

+1 I like it!

From: Akira Yoshiyama 
>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" 
>
Date: Thursday, July 21, 2016 at 9:02 AM
To: "OpenStack Development Mailing List (not for usage questions)" 
>
Subject: Re: [openstack-dev] [horizon] Logo

More photos for Shiba Inu:

https://www.google.co.jp/search?q=%E6%9F%B4%E7%8A%AC=ivsn=lnms=isch=X

Akira

2016年7月17日日曜日、Diana 
Whitten>さんは書きました:
Dunno if there have been any suggestions, but I'd like to suggest a Shiba Inu 
for the Horizon logo mascot.

If you unfamiliar with a Shiba Inu, take a look here: 
http://vignette1.wikia.nocookie.net/sanicsource/images/9/97/Doge.jpg/revision/latest?cb=20160112233015

Our Shiba should definitely look shocked too.

- Diana


--
吉山あきら >
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [DIB] [TripleO] Should we have a DIB meeting?

2016-07-21 Thread Simon Leinen

Stephane Miller writes:
> I'm proposing that we have a regular, IRC-based meeting for the
> project. This could be done on its own, or as part of the TripleO
> meeting. I don't think we necessarily need to do this every week, but
> a fortnightly chance to get together to chat about big changes,
> design, etc would be great.

> DIB and TripleO DIB community, what are your thoughts?

+1

As a DIB (but otherwise not TripleO) user, I would welcome this.
-- 
Simon.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] Logo

2016-07-21 Thread Akira Yoshiyama

More photos for Shiba Inu:

https://www.google.co.jp/search?q=%E6%9F%B4%E7%8A%AC=ivsn=lnms=isch=X


Akira

2016年7月17日日曜日、Diana Whittenさんは書きました:

> Dunno if there have been any suggestions, but I'd like to suggest a Shiba
> Inu for the Horizon logo mascot.
>
> If you unfamiliar with a Shiba Inu, take a look here:
> http://vignette1.wikia.nocookie.net/sanicsource/images/9/97/Doge.jpg/revision/latest?cb=20160112233015
>
> Our Shiba should definitely look shocked too.
>
> - Diana
>


-- 
吉山あきら 
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)

Excerpts from Fox, Kevin M's message of 2016-07-19 21:59:29 +:
> Yeah. I'm not saying any project has done it out of malice. Everyone's just 
> doing whats best for their project. But it does not seem like there is an 
> overarching entity watching over the whole or (pushing, encouraging, 
> enticing, whatever words are appropriate here) projects to work on the 
> commons anymore. It use to be that incubating projects were pushed to help 
> the other projects integrate with them by the incubating project being 
> strongly encouraged to write the integrations themselves as part of the 
> incubation process. Now it seems like each project just spawns and then hopes 
> someone else will do the legwork. Using the carrot of incubation to further 
> the commons is not an ideal solution, but it was at least something.
> 
> Linux has an overarching entity, Linus for that task. He's there to make sure 
> that someone is really paying attention to integration of the whole thing 
> into a cohesive, usable whole. Sometimes he pushes back and makes sure 
> commons work happens as part of features getting in to ensure commons work 
> still gets done. I'm not advocating a benevolent dictator for OpenStack 
> though.
> 
> But what I thought what the TC's job was, was benevolent dictators,
which each subproject (or subsystem in linux terms) are required to
give up final say to, so that sometimes the projects have to sacrifice
a bit so that the whole can flourish and those benevolent dictators are
elected for a time, by the OpenStack community. (Actually, I think 
that kind of makes it a Democratic Republic... but I digress) Maybe I
misunderstood what the TC's about. But I think we still do need some
folks elected by the community to be involved in making sure OpenStack
as a whole has a cohesive technical architecture that actually
addresses user problems and that have some power to to stop the "this
feature belongs in this project", "no, it belongs in that project",
"no, lets spawn 3 new projects to cover that case" kinds of things,
make the difficult decision, and ask a project to help the community
out by going along with "a solution" and we all can move on. Critical
features have been stuck in this space for years and OpenStacks
competitors have had solutions for years. 
> 
> Interest in OpenStack as a whole has leveled off in about:
> 
> 
> From: Zane Bitter [zbit...@redhat.com]
> Sent: Tuesday, July 19, 2016 1:08 PM
> To: openstack-dev@lists.openstack.org
> Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)
> 
> On 14/07/16 16:30, Fox, Kevin M wrote:
> > I'm going to go ahead and ask the difficult question now as the answer is 
> > relevant to the attached proposal...
> >
> > Should we reconsider whether the big tent is the right approach going 
> > forward?
> >
> > There have been some major downsides I think to the Big Tent approach, such 
> > as:
> >  * Projects not working together because of fear of adding extra 
> > dependencies Ops don't already have
> >  * Reimplementing features, badly, over and over again in different 
> > projects instead of standardizing something.
> >  * More projects being created due to politics and not technical reasons.
> >  * Less cross project communication
> >  * Greater Operator pain at trying to assemble a more loose confederation 
> > of projects into something useful.
> >  * General pushing off more and more work to Operators/Users to deal with.
> >  * Worse User experience as cross project issues aren't being addressed.
> >  * Previously incubated projects such as Nova, Swift, etc getting a 
> > disproportionate say in things as they have a greater current user base, 
> > and its increasingly hard now for new projects to gain any traction.
> >  * Much less community pressure on projects to work together to elevate 
> > everyone. Architectural decisions are now made at individual project level 
> > with little done at the OpenStack level.
> >  * etc...
> 
> The thing is, all of these problems pre-date the big tent. Arguably they
> were even worse before because so many projects were not only not
> blessed by the TC as hard dependencies, but officially weren't even part
> of OpenStack. Say what you want about the big tent, but at least we
> haven't been treated to another Zaqar graduation review farce.
> 
> I think your complaint here is that the big tent hasn't done enough to
> solve these problems. That's a fair complaint.
> 
> I think what you're suggesting is missing is some sort of TC imprimatur
> to say that it's acceptable to take a hard dependency on a particular
> project, which is effectively what graduation from incubation meant
> previously (e.g. distributors were effectively, though not actually,
> obliged to package it at this point if they weren't already). Of course
> the only thing that can really _force_ people to adopt a project is
> DefCore, and that comes with a major chicken-and-egg

Re: [openstack-dev] [Congress] Congress horizon plugin - congressclient/congress API auth issue - help

2016-07-21 Thread Aimee Ukasick

The  Policy/Data Sources web page throws the same errors. I am
planning to recheck direct API calls using v3 auth today or tomorrow.

aimee

On Thu, Jul 21, 2016 at 10:49 AM, Tim Hinrichs  wrote:
> Hi Aimee,
>
> Do the other APIs work?  That is, is it a general problem authenticating, or
> is the problem limited to list_policies?
>
> Tim
>
> On Wed, Jul 20, 2016 at 3:54 PM Aimee Ukasick 
> wrote:
>>
>> Hi all,
>>
>> I've been working on Policy UI (Horizon): Unable to get policies
>> list (devstack) (https://bugs.launchpad.net/congress/+bug/1602837)
>> for the past 3 days. Anusha is correct - it's an authentication
>> problem, but I have not been able to fix it.
>>
>> I grabbed the relevant code in congress.py from Anusha's horizon
>> plugin model patchset (https://review.openstack.org/#/c/305063/3) and
>> added try/catch blocks, logging statements (with error because I
>> haven't figured out how to set the horizon log level).
>>
>>
>> I am testing the code on devstack, which I cloned on 19 July 2016.
>>
>> With both v2 and v3 auth, congressclient.v1.client is created.
>> The failure happens trying to call
>> congressclient.v1.client.Client.list_policies().
>> When using v2 auth, the error message is "Unable to get policies list:
>> The resource could not be found"
>> When using v3 auth, the error message is "Cannot authorize API client"
>>
>> I am assuming that congressclient.v1.client.Client is
>>
>> https://github.com/openstack/python-congressclient/blob/master/congressclient/v1/client.py
>> and that client.list_policy() calls list_policy()in the
>> python-congressclient
>> which in turn calls the Congress API. Is this correct?
>>
>> Any ideas why with v3 auth, the python-congressclient cannot authorize the
>> call to the API?
>>
>> I looked at other horizon plugin models (ceilometer, neutron, nova,
>> cerberus, cloudkitty, trove, designate, manila) to see how they created
>> the client. While the code to create a client is not identical,
>> it is vastly different from the code to create a client
>> in contrib/horizon/congress.py.
>>
>> Thanks in advance for any pointers.
>>
>> aimee
>>
>> Aimee Ukasick (aimeeu)
>>
>> v2 log:
>> 2016-07-20 22:13:56.501455
>> 2016-07-20 22:14:30.238233 * view.get_data calling policies =
>> congress.policies_list(self.request) *
>> 2016-07-20 22:14:30.238318 * self.request.path=
>> /dashboard/admin/policies/
>> 2016-07-20 22:14:30.238352 * congress.policies_list(request)
>> BEGIN*
>> 2016-07-20 22:14:30.238376 * calling client =
>> congressclient(request)*
>> 2016-07-20 22:14:30.238399 * congress.congressclient BEGIN*
>> 2016-07-20 22:14:30.238454 * auth_url= http://192.168.56.103/identity
>> 2016-07-20 22:14:30.238479 * calling get_keystone_session *
>> 2016-07-20 22:14:30.238505 * congress.get_keystone_session BEGIN
>> auth_url *http://192.168.56.103/identity
>> 2016-07-20 22:14:30.238554 * path= /identity
>> 2016-07-20 22:14:30.238578 * using V2 plugin to authenticate*
>> 2016-07-20 22:14:30.238630 * v2 auth.get_auth_state=
>> 2016-07-20 22:14:30.238656 None
>> 2016-07-20 22:14:30.238677 * finished using V2 plugin to
>> authenticate*
>> 2016-07-20 22:14:30.238698 * creating session with auth *
>> 2016-07-20 22:14:30.244407 * congress.get_keystone_session END*
>> 2016-07-20 22:14:30.244462 * regtion_name= RegionOne
>> 2016-07-20 22:14:30.244491 * calling congress_client.Client(**kwargs)
>> 2016-07-20 22:14:30.247830 * congress.congressclient END*
>> 2016-07-20 22:14:30.247902 * calling policies_list =
>> client.list_policy()*
>> 2016-07-20 22:14:30.248012 DEBUG:keystoneauth.identity.v2:Making
>> authentication request to http://192.168.56.103/identity/tokens
>> 2016-07-20 22:14:30.255023 DEBUG:keystoneauth.session:Request returned
>> failure status: 404
>> 2016-07-20 22:14:30.257546 Unable to get policies list: The resource
>> could not be found.
>>
>>
>> v3 log:
>> 2016-07-20 22:09:22.912969
>> 2016-07-20 22:09:31.907119 * view.get_data calling policies =
>> congress.policies_list(self.request) *
>> 2016-07-20 22:09:31.907973 * self.request.path=
>> /dashboard/admin/policies/
>> 2016-07-20 22:09:31.908122 * congress.policies_list(request)
>> BEGIN*
>> 2016-07-20 22:09:31.908250 * calling client =
>> congressclient(request)*
>> 2016-07-20 22:09:31.908386 * congress.congressclient BEGIN*
>> 2016-07-20 22:09:31.909034 * auth_url= http://192.168.56.103/identity
>> 2016-07-20 22:09:31.909217 * calling get_keystone_session *
>> 2016-07-20 22:09:31.909356 * congress.get_keystone_session BEGIN
>> auth_url *http://192.168.56.103/identity
>> 2016-07-20 22:09:31.909527 * path= /identity
>> 2016-07-20 22:09:31.909795 * using V3 plugin to authenticate*
>> 2016-07-20 22:09:31.910042 auth_url=http://192.168.56.103/identity
>> 2016-07-20 22:09:31.910175

Re: [openstack-dev] [Congress] Congress horizon plugin - congressclient/congress API auth issue - help

2016-07-21 Thread Tim Hinrichs

Hi Aimee,

Do the other APIs work?  That is, is it a general problem authenticating,
or is the problem limited to list_policies?

Tim

On Wed, Jul 20, 2016 at 3:54 PM Aimee Ukasick 
wrote:

> Hi all,
>
> I've been working on Policy UI (Horizon): Unable to get policies
> list (devstack) (https://bugs.launchpad.net/congress/+bug/1602837)
> for the past 3 days. Anusha is correct - it's an authentication
> problem, but I have not been able to fix it.
>
> I grabbed the relevant code in congress.py from Anusha's horizon
> plugin model patchset (https://review.openstack.org/#/c/305063/3) and
> added try/catch blocks, logging statements (with error because I
> haven't figured out how to set the horizon log level).
>
>
> I am testing the code on devstack, which I cloned on 19 July 2016.
>
> With both v2 and v3 auth, congressclient.v1.client is created.
> The failure happens trying to call
> congressclient.v1.client.Client.list_policies().
> When using v2 auth, the error message is "Unable to get policies list:
> The resource could not be found"
> When using v3 auth, the error message is "Cannot authorize API client"
>
> I am assuming that congressclient.v1.client.Client is
>
> https://github.com/openstack/python-congressclient/blob/master/congressclient/v1/client.py
> and that client.list_policy() calls list_policy()in the
> python-congressclient
> which in turn calls the Congress API. Is this correct?
>
> Any ideas why with v3 auth, the python-congressclient cannot authorize the
> call to the API?
>
> I looked at other horizon plugin models (ceilometer, neutron, nova,
> cerberus, cloudkitty, trove, designate, manila) to see how they created
> the client. While the code to create a client is not identical,
> it is vastly different from the code to create a client
> in contrib/horizon/congress.py.
>
> Thanks in advance for any pointers.
>
> aimee
>
> Aimee Ukasick (aimeeu)
>
> v2 log:
> 2016-07-20 22:13:56.501455
> 2016-07-20 22:14:30.238233 * view.get_data calling policies =
> congress.policies_list(self.request) *
> 2016-07-20 22:14:30.238318 * self.request.path=
> /dashboard/admin/policies/
> 2016-07-20 22:14:30.238352 * congress.policies_list(request) BEGIN*
> 2016-07-20 22:14:30.238376 * calling client =
> congressclient(request)*
> 2016-07-20 22:14:30.238399 * congress.congressclient BEGIN*
> 2016-07-20 22:14:30.238454 * auth_url= http://192.168.56.103/identity
> 2016-07-20  22:14:30.238479
> * calling get_keystone_session *
> 2016-07-20 22:14:30.238505 * congress.get_keystone_session BEGIN
> auth_url *http://192.168.56.103/identity
> 2016-07-20 22:14:30.238554 * path= /identity
> 2016-07-20 22:14:30.238578 * using V2 plugin to authenticate*
> 2016-07-20 22:14:30.238630 * v2 auth.get_auth_state=
> 2016-07-20 22:14:30.238656 None
> 2016-07-20 22:14:30.238677 * finished using V2 plugin to
> authenticate*
> 2016-07-20 22:14:30.238698 * creating session with auth *
> 2016-07-20 22:14:30.244407 * congress.get_keystone_session END*
> 2016-07-20 22:14:30.244462 * regtion_name= RegionOne
> 2016-07-20 22:14:30.244491 * calling congress_client.Client(**kwargs)
> 2016-07-20 22:14:30.247830 * congress.congressclient END*
> 2016-07-20 22:14:30.247902 * calling policies_list =
> client.list_policy()*
> 2016-07-20 22:14:30.248012 DEBUG:keystoneauth.identity.v2:Making
> authentication request to http://192.168.56.103/identity/tokens
> 2016-07-20 22:14:30.255023 DEBUG:keystoneauth.session:Request returned
> failure status: 404
> 2016-07-20 22:14:30.257546 Unable to get policies list: The resource
> could not be found.
>
>
> v3 log:
> 2016-07-20 22:09:22.912969
> 2016-07-20 22:09:31.907119 * view.get_data calling policies =
> congress.policies_list(self.request) *
> 2016-07-20 22:09:31.907973 * self.request.path=
> /dashboard/admin/policies/
> 2016-07-20 22:09:31.908122 * congress.policies_list(request) BEGIN*
> 2016-07-20 22:09:31.908250 * calling client =
> congressclient(request)*
> 2016-07-20 22:09:31.908386 * congress.congressclient BEGIN*
> 2016-07-20 22:09:31.909034 * auth_url= http://192.168.56.103/identity
> 2016-07-20  22:09:31.909217
> * calling get_keystone_session *
> 2016-07-20 22:09:31.909356 * congress.get_keystone_session BEGIN
> auth_url *http://192.168.56.103/identity
> 2016-07-20 22:09:31.909527 * path= /identity
> 2016-07-20 22:09:31.909795 * using V3 plugin to authenticate*
> 2016-07-20 22:09:31.910042 auth_url=http://192.168.56.103/identity
> 2016-07-20  22:09:31.910175
> token=d46339f2d0b5455db54909d6ed95a9cc
> 2016-07-20 22:09:31.910301 project_name=alt_demo
> 2016-07-20 22:09:31.910426 domain_name=Default
> 2016-07-20 22:09:31.910676 project_domain_name=default
>

Re: [openstack-dev] [tc][all] Plugins for all

Excerpts from Hayes, Graham's message of 2016-07-19 16:59:20 +:
> On 19/07/2016 16:39, Doug Hellmann wrote:
> > Excerpts from Hayes, Graham's message of 2016-07-18 17:13:09 +:
> >> On 18/07/2016 17:57, Thierry Carrez wrote:
> >>> Hayes, Graham wrote:
>  [...]
>  The point is that we were supposed to be a level field as a community
>  but if we have examples like this, there is not a level playing field.
> >>>
> >>> While I generally agree on your goals here (avoid special-casing some
> >>> projects in generic support projects like Tempest), I want to clarify
> >>> what we meant by "level playing field" in a recent resolution.
> >>
> >>
> >> Yes - it has been pointed out the title is probably overloading a term
> >> just used for a different purpose - I am more than willing to change it.
> >>
> >> I wasn't sure where I got the name, and I realised that was probably in
> >> my head from that resolution.
> >>
> >>> This was meant as a level playing field for contributors within a
> >>> project, not a level playing field between projects. The idea is that
> >>> any contributor joining any OpenStack project should not be technically
> >>> limited compared to other contributors on the same project. So, no
> >>> "secret sauce" that only a subset of developers on a project have access 
> >>> to.
> >>
> >> There is a correlation here - "special sauce" (not secret obviously)
> >> that only a subset of projects have access to.
> >>
> >>> I think I understand where you're gong when you say that all projects
> >>> should have equal chances, but keep in mind that (1) projects should not
> >>> really "compete" against each other (but rather all projects should
> >>> contribute to the success of OpenStack as a whole) and (2) some
> >>> OpenStack projects will always be more equal than others (for example we
> >>> require that every project integrates with Keystone, and I don't see
> >>> that changing).
> >>
> >> Yes, I agree we should not be competing. But was should not be asking
> >> the smaller projects to re-implement functionality, just because they
> >> did not get integrated in time.
> >>
> >> We require all projects to integrate with keystone for auth, as we
> >> require all projects to integrate with neutron for network operations
> >> and designate for DNS, I just see it as a requirement for using the
> >> other components of OpenStack for their defined purpose.
> >>
> >
> > It would be useful to have some specific information from the QA/Tempest
> > team (and any others with a similar situation) about whether the current
> > situation about how differences between in-tree tests and plugin tests
> > are allowed to use various APIs. For example, are there APIs only
> > available to in-tree tests that are going to stay that way? Or is this
> > just a matter of not having had time to "harden" or "certify" or
> > otherwise prepare those APIs for plugins to use them?
> 
> "Staying that way" is certainly the impression given to users from
> other projects.

OK, but is that an "impression" or is it a stated "policy"?

> In any case tempest is just an example. From my viewpoint, we need to
> make this a community default, to avoid even the short (which really
> ends up a long) term discrepancy between projects.

Before we start making lots of specific rules about how teams
coordinate, I would like to understand the problem those rules are meant
to solve, so thank you for providing that example. I still haven't heard
from the QA team, though. Ken'ichi?

> If the standard in the community is equal access, this means when the
> next testing tool, CLI, SDK, $cross_project_tool comes along, it is
> available to all projects equally.
> 
> If everyone uses the interfaces, they get better for all users of them,
> "big tent projects" and "tc-approved-release" alike. Having two
> way of doing the same thing means that there will always be
> discrepancies between people who are in the club, and those who are not.

I think I understand your motivation. It's not clear yet whether
there needs to be a new policy to change the existing intent,
or if a discussion just hasn't happened, or if someone simply needs
to edit some code.

Are there other examples we can talk about in the mean time?

Doug

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][all] Plugins for all

Excerpts from Doug Wiegley's message of 2016-07-19 13:58:59 -0600:
> 
> > On Jul 19, 2016, at 9:36 AM, Doug Hellmann  wrote:
> > 
> > Excerpts from Hayes, Graham's message of 2016-07-18 17:13:09 +:
> >> On 18/07/2016 17:57, Thierry Carrez wrote:
> >>> Hayes, Graham wrote:
>  [...]
>  The point is that we were supposed to be a level field as a community
>  but if we have examples like this, there is not a level playing field.
> >>> 
> >>> While I generally agree on your goals here (avoid special-casing some
> >>> projects in generic support projects like Tempest), I want to clarify
> >>> what we meant by "level playing field" in a recent resolution.
> >> 
> >> 
> >> Yes - it has been pointed out the title is probably overloading a term
> >> just used for a different purpose - I am more than willing to change it.
> >> 
> >> I wasn't sure where I got the name, and I realised that was probably in
> >> my head from that resolution.
> >> 
> >>> This was meant as a level playing field for contributors within a
> >>> project, not a level playing field between projects. The idea is that
> >>> any contributor joining any OpenStack project should not be technically
> >>> limited compared to other contributors on the same project. So, no
> >>> "secret sauce" that only a subset of developers on a project have access 
> >>> to.
> >> 
> >> There is a correlation here - "special sauce" (not secret obviously)
> >> that only a subset of projects have access to.
> >> 
> >>> I think I understand where you're gong when you say that all projects
> >>> should have equal chances, but keep in mind that (1) projects should not
> >>> really "compete" against each other (but rather all projects should
> >>> contribute to the success of OpenStack as a whole) and (2) some
> >>> OpenStack projects will always be more equal than others (for example we
> >>> require that every project integrates with Keystone, and I don't see
> >>> that changing).
> >> 
> >> Yes, I agree we should not be competing. But was should not be asking
> >> the smaller projects to re-implement functionality, just because they
> >> did not get integrated in time.
> >> 
> >> We require all projects to integrate with keystone for auth, as we
> >> require all projects to integrate with neutron for network operations
> >> and designate for DNS, I just see it as a requirement for using the
> >> other components of OpenStack for their defined purpose.
> >> 
> > 
> > It would be useful to have some specific information from the QA/Tempest
> > team (and any others with a similar situation) about whether the current
> > situation about how differences between in-tree tests and plugin tests
> > are allowed to use various APIs. For example, are there APIs only
> > available to in-tree tests that are going to stay that way? Or is this
> > just a matter of not having had time to "harden" or "certify" or
> > otherwise prepare those APIs for plugins to use them?
> 
> Doug, one example that I’m aware of — I’ve suggested moving neutron entirely 
> out of devstack and into the neutron repo, via the devstack plugin interface. 
> This was rejected, and the counter-argument given was that the folks that 
> maintain the integrated gate, which happen to be many of the same folks 
> maintaining devstack and the like, want to retain control of the items that 
> can break the integrated gate, and that it gets too hard to track down 
> individual project folks when the world is burning.
> 
> I don’t personally agree with the decision, but I can see some merit in that 
> argument.

Yes, this is one of the outcomes of being in a situation where other
projects require something as a dependency -- each side loses a little
control, and some centralization happens to ensure that we can maintain
the flow of work for OpenStack as a whole.

Doug

> 
> Thanks,
> doug
> 
> > 
> > Doug
> > 
> > __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [telemetry] Mascot

Excerpts from Julien Danjou's message of 2016-07-21 10:18:47 +0200:
> Hi team,
> 
> So we as you may have noticed, we have the possibility to have a proper
> design for our team mascot/logo, thanks to the foundation:
> 
>   http://www.openstack.org/project-mascots
> 
> After asking for precision, it appears that this would be a mascot for
> the Telemetry project/team. It means only a mascot for all of our main
> deliverable services (Ceilometer, Panko, Gnocchi & Aodh) and not one for
> each… which IMHO makes the thing harder.¹ But let's try!
> 
> So, ideas about what we could use as a mascot?
> 

Something with big ears that are good for listening. A fennec [1]? A
serval [2]?

Doug

[1] https://en.wikipedia.org/wiki/Fennec_fox
[2] https://en.wikipedia.org/wiki/Serval

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] repo split

2016-07-21 Thread Steven Dake (stdake)

I am voting -1 for now, but would likely change my vote after we branch
Newton.  I'm not a super big fan of votes way ahead of major events (such
as branching) because a bunch of things could change between now and then
and the vote would be binding.

Still community called the vote - so vote stands :)

Regards
-steve


On 7/20/16, 1:48 PM, "Ryan Hallisey"  wrote:

>Hello.
>
>The repo split discussion that started at summit was brought up again at
>the midcycle.
>The discussion was focused around splitting the Docker containers and
>Ansible code into
>two separate repos [1].
>
>One of the main opponents to the split is backports.  Backports will need
>to be done
>by hand for a few releases.  So far, there hasn't been a ton of
>backports, but that could
>always change.
>
>As for splitting, it provides a much clearer view of what pieces of the
>project are where.
>Kolla-ansible with its own repo will sit along side kolla-kubernetes as
>consumers of the
>kolla repo.
>
>The target for the split will be for day 1 of Occata. The core team will
>vote on
>the change of splitting kolla into kolla-ansible and kolla.
>
>Cores please respond with a +1/-1 to approve or disapprove the repo
>split. Any community
>member feel free to weigh in with your opinion.
>
>+1
>-Ryan
>
>[1] - https://etherpad.openstack.org/p/kolla-N-midcycle-repo-split
>
>__
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [ironic] weekly subteam status report

2016-07-21 Thread Loo, Ruby

Hi,

We are wriggly to present this week's subteam report for Ironic. As usual, this 
is pulled directly from the Ironic whiteboard[0] and formatted.

Bugs (dtantsur)
===
- Stats (diff with 11 July 2016)
- Ironic: 201 bugs (+4) + 201 wishlist items (+4). 13 new (+3), 145 in progress 
(+3), 0 critical, 29 high and 21 incomplete (-4)
- Inspector: 9 bugs + 21 wishlist items. 0 new, 8 in progress (+1), 0 critical, 
2 high and 2 incomplete
- Nova bugs with Ironic tag: 9 (-2). 0 new, 0 critical, 0 high

Network isolation (Neutron/Ironic work) (jroll, TheJulia, devananda)

* trello: 
https://trello.com/c/HWVHhxOj/1-multi-tenant-networking-network-isolation
- much code merged!
- still to merge: PortGroup support, Inspector support for LLDP, two Nova 
changes, documentation

Gate improvements (jlvillal, lucasagomes, dtantsur)
===
- network multitenancy test passing, will be moved from experimental to 
non-voting check queue this week

Agent top-level API promotion (dtantsur)

* trello: 
https://trello.com/c/37YuKIB8/28-promote-agent-vendor-passthru-to-core-api
- I'm tired of rebasing this series :/ ping me when you're ready to review and 
I'll rebase it again

Driver composition (dtantsur)
=
* trello: https://trello.com/c/fTya14y6/14-driver-composition
- blocked by
- agent API promotion
- ongoing discussions about interfaces defaults

Notifications (mariojv)
===
* trello: https://trello.com/c/MD8HNcwJ/17-notifications
- Status 7/18/2016: Still needs reviews. Rebased again.
- https://review.openstack.org/#/c/298461/
- https://review.openstack.org/#/c/321865/

Keystone policy support (JayF, devananda)
=* trello: 
https://trello.com/c/P5q3Es2z/15-keystone-policy-support
- Spec merged, code needs to be updated still.

Serial console (yossy, hshiina, yuikotakadamori)

* trello: https://trello.com/c/nm3I8djr/20-serial-console
- console_utils: merged
- IPMITool driver interface: needs review

Inspector (dtansur)
===
* trello: https://trello.com/c/PwH1pexJ/23-rescue-mode
- The grenade job is still blocked by this one-liner in grenade itself: 
https://review.openstack.org/337372 (too bad 8x +1 != 4x +2 ;)

.

Until next week,
--ruby

[0] https://etherpad.openstack.org/p/IronicWhiteBoard

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [fuel] Fuel meeting Canceled

2016-07-21 Thread Andrew Woodward

Again the agenda [1] is empty again so I'm calling it on the meeting this
week.

[1] https://etherpad.openstack.org/p/fuel-weekly-meeting-agenda
-- 

--

Andrew Woodward

Mirantis

Fuel Community Ambassador

Ceph Community
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Remove duplicate code using Data Driven Tests (DDT)