Re: [openstack-dev] FPGA as a dynamic nested resources

[Roman Dobosz]

On Tue, 19 Jul 2016 12:40:50 -0700
Ed Leafe  wrote:

> > It can identified 3 levels of FPGA resources, which can be nested one
> > on the others:
> >
> > 1. Whole FPGA. If used discrete FPGA, than even today it might be pass
> >   through to the VM.
> Can you explain why this would ever be useful? IOW, what can a VM do
> with an entire FPGA?

Private cloud, development purposes. It could be treated as FPGA-aaS.
Same goes for the unoccupied slots. Of course, because reprogramming
affects real, not virtualized hardware, there are security concerns
for allowing users to do that in public clouds for example.

The other reason, which is much more significant, there could be IPs
so big, that will take most of FPGA - so reconfiguration is needed
(although reconfiguration is out of scope for Nova) and we don't want
to wipe out other accelerator which might be currently in use. That's
why slots here also are treated as dynamic resources.

> > 3. Accelerator in region/FPGA. If there is an accelerator programmed
> >   in the slot, it is possible, that such accelerator provides us with
> >   Virtual Functions (similar to the SR-IOV), than every available VF
> >   can be treated as a resource.
>
> This is my understanding of what would be consumable: the slot / VF,
> which the VM could take advantage of.

Yes. That's the obvious scenario.

> > 4. It might be also necessary to track every VF individually, although
> >   I didn't assumed it will be needed, nevertheless with nested
> >   resources it should be easy to handle it.
>
> I’m still not seeing the need for nesting. If you have a single FPGA
> with 8 slots, when you program the slots with accelerators, you now
> have 8 consumable resources. The fact that they came from a
> particular FPGA unit doesn’t seem relevant from a scheduling
> perspective.

Unless you have one FPGA with 8 slots, which can become FPGA with 4
slots. From scheduling perspective you have to know, which FPGA
resources can be reconfigured, and which not, isn't it? Also, AFAIRC
to provide VM with VF, there is a need for providing libvirt with
address of such VF, right? That's why I've putted this last point.

The whole idea of getting FPGA as resource is its ability to swap
resources on demand. So it can be thought of as several available
hardware (means - accelerators, consumable by VMs) which most of the
time are not programmed in certain moment.

So, let's assume, that we have two hosts: HostA and HostB with FPGA
capable to provide 2 accelerators which exclusively use entire chip
(lets call them AX1 and AX2), and one other, which can use one of the
2 possible slots (AY). So, the situation is we have 3 possible
accelerators to use, and in worst case scenario only two available
places where they can be places.

Initially, there is no accelerator in use, cloud administrator define
all the IPs he have available (somehow - this part isn't defined yet -
but lets assume it is in place)

Now, user requests VM with certain flavor/image with AX1 and scheduler
knows, that it will fit into HostA and HostB, so HostA is chosen, FPGA
magically™ is prepared to hold AX1 accelerator and VM was started. Now
we have resource tree HostA FPGA->slot->AX1 and HostB FPGA.

Next, user requests another VM with AY accelerator, scheduler now
should know, that the only available option is HostB, so again magic
is happening, and there is a resource tree:

HostA FPGA
 +- slot1
 +- AX1

HostB FPGA
 +- slot1
 +- AY
 +- slot2


Now, what should happen if user remove VM with AY accelerator and
request another VM with AX2?

-- 
Cheers,
Roman Dobosz

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Nominate Alexey Stepanov for fuel-qa and fuel-devops core

[Vladimir Khlyunev]

+1

On Mon, Jul 18, 2016 at 3:14 PM, Dmitry Tyzhnenko 
wrote:

> +1
>
> On Fri, Jul 15, 2016 at 4:41 PM, Artem Panchenko 
> wrote:
>
>> +1
>>
>>
>> On 15.07.16 16:25, Tatyana Leontovich wrote:
>>
>> +1
>>
>> On Fri, Jul 15, 2016 at 4:08 PM, Anastasia Urlapova <
>> aurlap...@mirantis.com> wrote:
>>
>>> +1
>>>
>>> On Fri, Jul 15, 2016 at 4:02 PM, Andrey Sledzinskiy <
>>> asledzins...@mirantis.com> wrote:
>>>
 Hi,
 I'd like to nominate Alexey Stepanov for fuel-qa [0] and fuel-devops
 [1] core.

 Alexey is doing great job improving fuel-qa and fuel-devops projects.
 He's become an expert in code base in very short terms so I think he
 deserves to be a part of fuel-qa/fuel-devops core team.

 Please, vote for Alexey!

 [0]
 http://stackalytics.com/?release=all&module=fuel-qa&user_id=astepanov-m&metric=marks
 [1]
 http://stackalytics.com/?release=all&module=fuel-devops&user_id=astepanov-m&metric=marks

 --
 Thanks,
 Andrey Sledzinskiy
 QA Engineer,
 Mirantis, Kharkiv


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: 
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>> --
>> Artem Panchenko
>> QA Engineer
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>
> --
> WBR,
> Dmitry T.
> Fuel QA Engineer
> http://www.mirantis.com
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Best regards,
Vladimir Khlyunev
QA engineer,
Mirantis Inc.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [tricircle] A new contributor

[Shinobu Kinjo]

Hi Team,

Please welcome a new contributor (CCed) for the Tricircle project.

Cheers,
Shinobu

-- 
Email:
shin...@linux.com
shin...@redhat.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][qos] Egress minimum bandwidth assurance

[Alonso Hernandez, Rodolfo]

Hello Ihar:

Please, take a look to https://review.openstack.org/#/c/318531/.

>> does it mean you want to set some queues on ports that have no qos policy 
>> attached?
That’s exactly what I'm doing, because the unique way to shape traffic in OvS 
is at the end of the datapath. In other words, shaping is only possible for 
egress (OvS point of view) traffic. Because what I need is to limit VM-egress 
(OvS-ingress) traffic, what I'm doing is defined in 
https://review.openstack.org/#/c/318531/9/doc/source/devref/quality_of_service.rst@342.

Regards.


-Original Message-
From: Ihar Hrachyshka [mailto:ihrac...@redhat.com] 
Sent: Monday, July 18, 2016 11:53 AM
To: OpenStack Development Mailing List (not for usage questions) 

Subject: Re: [openstack-dev] [neutron][qos] Egress minimum bandwidth assurance

Rodolfo  wrote:

> Hello:
>
> I’m working in the egress minimum bandwidth assurance qos policy rule.
>
> I made a POC using the following architecture:
> -  Every time a new rule is created and assigned to a port:
> o   A new queue is created in OVS.
> o   Create/update the qos policy (OVS database) in all other ports in  
> br-int (assigned to the same vlan), br-tun and br-phy, updating the 
> queue value.
> o   Set the queue id to the incoming traffic to the OVS.
> -  Every time rule is updated:
> o   The values are updated in the ovs database.
> -  Every time a rule is unassigned to a port:
> o   The queue is removed from the qos policies. (OVS database)
> o   The OVS rule to set the queue id is removed.
>
> The problem is, based on the neutron extensions API, I can’t make any 
> updates in other ports affected by this rule. That means, for example: 
> if I create a new port, this port must have also a qos assigned (OVS
> database) using the existing queue. But because the Neutron qos rule 
> doesn’t apply to this port, no qos agent function is called and this 
> port is not correctly configured.

I am sorry if the question is trivial, but just for my understanding, does it 
mean you want to set some queues on ports that have no qos policy attached? Why 
is it the case?

Because if the policy IS attached to a port, then qos l2 agent driver will be 
triggered for all supported rules, including your new rule, on every relevant 
policy/rule change.

I am sure I miss something, so please enlighten me.

Ihar

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] FPGA as a dynamic nested resources

[Roman Dobosz]

On Tue, 19 Jul 2016 15:51:26 -0700
Ed Leafe  wrote:

> >> Why would a VM program the slot? Wouldn’t it usually be at the
> >> host level?
> >
> > Are there no cases where a VM might want to download a proprietary
> > program into an FPGA?
>
> That doesn’t sound right to me, but maybe I’m just not that familiar
> with FPGA specifics. In general, VMs don’t control their hosts. It
> would also bring up some complications, such as what should happen
> when you delete that VM: does the FPGA have to be reset to its
> original state?

Technically, it is not necessary to "erase" the FPGA. It might be
untouched, and in resource tracker it would be figured as free, so
than it can be programmed another accelerator, or passed to another VM
if needed. It may be also zeroed (programmed with empty IP) by
external entity which might be preferred option.

> >> I’m still not seeing the need for nesting. If you have a single
> >> FPGA with 8 slots, when you program the slots with accelerators,
> >> you now have 8 consumable resources. The fact that they came from
> >> a particular FPGA unit doesn’t seem relevant from a scheduling
> >> perspective.
> >
> > If you want to be able to provide an FPGA as either a whole
> > un-programmed FPGA or as pre-programmed resources, you'd
> > presumably need to know which whole FPGAs are available and which
> > have been fractionally allocated, no?
>
> An unprogrammed FPGA is a particular resource class. When you
> program it, you are removing one of that class and creating one or
> more of a new resource class (e.g., an encryption accelerator
> program). There isn’t a need to nest anything.

Although you have to track *where* you can schedule potential
accelerator, isn't it? Certain type of IP will need proper slot, so it
also have to be tracked. Nesting isn't necessary, but might be helpful
to manage the state of your resources.

> > I agree that if you are only going to have the host program the
> > FPGA and then make the resources available then the scheduler
> > doesn't need to know about whole FPGAs.
>
> That was where we left the discussion in Austin, so that was my
> assumption.

… as the first step, isn't it? No one is pushing to have this in
Newton. Even Ocata time frame seems like unrealistic.

-- 
Cheers,
Roman Dobosz

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] how to attach a physical disk to a qemu instance?

[wk]

hi:


   I want attach a physical disk to a qemu instance in devstack environment,But 
these methods have some problem.


   Method 1:
step1
reference  
http://ronaldevers.nl/2012/10/14/adding-a-physical-disk-kvm-libvirt.html
  virsh edit instance-0001


 you add the disk to the domain??s xml config file by hand. So open up 
/etc/libvirt/qemu/.xml in your favourite editor and add a  
section to the  section:



   
   
  

This will make the host??s /dev/md/storage available in the guest as 
/dev/vdb

step2
guest os can not find the new physical disk. Then reboot guest os, the xml 
is reverted to before,and guest os can not find the new phyisical disk.


Method 2:
step 1
[root@opentest ~]# virsh attach-disk instance-0001 /dev/md/storage vdb 
--cache none --config --type disk
Disk attached successfully

step 2
guest os can not find the new physical disk. Then reboot guest os, the xml 
is reverted to before,and guest os can not find the new phyisical disk.__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)

[Thierry Carrez]

Clint Byrum wrote:

[...]
But what I thought what the TC's job was, was benevolent dictators, which each subproject (or subsystem in linux terms) 
are required to give up final say to, so that sometimes the projects have to sacrifice a bit so that the whole can 
flourish and those benevolent dictators are elected for a time, by the OpenStack community. (Actually, I think  that 
kind of makes it a Democratic Republic... but I digress) Maybe I misunderstood what the TC's about. But I think we 
still do need some folks elected by the community to be involved in making sure OpenStack as a whole has a cohesive 
technical architecture that actually addresses user problems and that have some power to to stop the "this feature 
belongs in this project", "no, it belongs in that project", "no, lets spawn 3 new projects to cover 
that case" kinds of things, make the difficult decision, and ask a project to help the community out by going 
along with "a solution" and we all can move on. Critical features have been stu

ck

  in this space for years and OpenStacks competitors have had solutions for 
years.


You're right, this is the TC's job. However, the TC does it more by
exception, rather than by default. So while Linus and the subsystem
leaders in the kernel look after changes in general, the TC is there to
catch things that bubble out of the processes in place. So, I think the
TC needs contributors to bring _specific_ things that need to be handled,
and they will. They're just not going to be able to stand at the gate
and review every spec... this process only scales to the velocity and
breadth that OpenStack has if we get contributors involved.


It's always a balancing act -- we want the TC to keep an eye on the big 
picture, provide guidance, encourage convergence, and stop the bucket 
when needed. At the same time, developers don't want a set of people 
that do not have specific experience in a their project to constantly 
interfere with project development with random mandates.


However, I'd agree that the pendulum currently is still in the "not 
enough" territory rather than in the "too much" territory. We have a 
number of initiatives brewing though (clarifying principles, defining 
release goals, release stewards, architecture WG, PTG event...) which I 
think will improve things in the right direction, let's see how those 
pan out. Please be patient while we are rolling those out.


So in summary: yes there still are issues, but it is not a simple 
problem, and we are working on it.


--
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tricircle] A new contributor

[joehuang]

Hi, team,



Welcome Xiulin to join the team.



Best Regards

Chaoyi Huang ( joehuang )



From: Shinobu Kinjo [shinobu...@gmail.com]
Sent: 20 July 2016 15:49
To: OpenStack Development Mailing List (not for usage questions)
Cc: yinxiulin
Subject: [openstack-dev] [tricircle] A new contributor

Hi Team,

Please welcome a new contributor (CCed) for the Tricircle project.

Cheers,
Shinobu

--
Email:
shin...@linux.com
shin...@redhat.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [storlets] IRC meeting times

[kajinamit]

Hi Eran,

08:00 UTC is good for me.
I'm ok with Tuesday.

Thank you,
Takashi

-Original Message-
From: e...@itsonlyme.name [mailto:e...@itsonlyme.name] 
Sent: Wednesday, July 20, 2016 2:28 AM
To: OpenStack Development Mailing List (not for usage questions) 

Subject: Re: [openstack-dev] [storlets] IRC meeting times

Sure.
How about an hour later?
That is:
08:00 UTC
17:00 JST
11:00 IST

Also, are you ok with Tuesdays?

Quoting kajina...@nttdata.co.jp:

> Hi Eran,
>
>
> Thanks for your suggestion.
>
> Is it possible to make the meeting time one hour earlier (08:00 UTC) 
> or one hour later (09:00 UTC)?
>
> Thank you,
> Takashi
>
> -Original Message-
> From: e...@itsonlyme.name [mailto:e...@itsonlyme.name]
> Sent: Wednesday, July 13, 2016 10:35 PM
> To: openstack-dev@lists.openstack.org
> Subject: [openstack-dev] [storlets] IRC meeting times
>
> Hi,
> I would like to change the IRC meeting times to Tuesdays at 07:00AM 
> UTC
> That's:
> 16:00 JST
> 10:00 IST
>
> Any objections?
> Thanks!
> Eran
>
>
>
> __
>  OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: 
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> __
>  OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: 
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] FPGA as a dynamic nested resources

[Daniel P. Berrange]

On Tue, Jul 19, 2016 at 08:03:28PM +0200, Roman Dobosz wrote:
> Hi all,
> 
> Some time ago Jay Pipes published etherpad[1] with ideas around
> modelling nested resources, taking NUMA as an example. I was also
> encouraged ;) to start this thread, on last Nova scheduler meeting.
> 
> I was read mentioned etherpad and what hits me was that described
> scenario with NUMA cells resembles the way how FPGA can be managed. In
> some extent.
> 
> NUMA cell can be treated as a vessel for memory cells, and it is
> expressed as number of MB. So it is possible to extract the
> information from existing data and add another level of aggregation
> using only clever prepared SQL query.
> 
> I think, that problem might be broader, than using existing, tweaked a
> bit model. If we take a look into resources, which FPGA may expose,
> than it can be couple of levels, and each of them can be treated as
> resource.
> 
> It can identified 3 levels of FPGA resources, which can be nested one
> on the others:
> 
> 1. Whole FPGA. If used discrete FPGA, than even today it might be pass
>through to the VM.
> 
> 2. Region in FPGA. Some of the FPGA models can be divided into regions
>or slots. Also, for some model it is possible to (re)program such
>region individually - in this case there is a possibility to pass
>entire slot to the VM, so that it might be possible to reprogram
>such slot, and utilize the algorithm within the VM.
> 
> 3. Accelerator in region/FPGA. If there is an accelerator programmed
>in the slot, it is possible, that such accelerator provides us with
>Virtual Functions (similar to the SR-IOV), than every available VF
>can be treated as a resource.
> 
> 4. It might be also necessary to track every VF individually, although
>I didn't assumed it will be needed, nevertheless with nested
>resources it should be easy to handle it.
> 
> Correlation between such resources are a bit different from NUMA -
> while in NUMA case there is a possibility to either schedule a VM with
> some memory specified, or request memory within NUMA cell, in FPGA if
> there is slot taken, or accelerator already programmed and used, there
> is no way to offer FPGA as a whole to the tenant, until all
> accelerators and slots are free.
> 
> I've followed Jay idea about nested resources and having in mind
> blueprint[2] regarding dynamic resources I've prepared how it fit in.

[snip lots of complicated modelling]

> Thoughts?

I'd suggest you'll increase your chances of success with nova design
approval if you focus on implementing a really simple usage scheme for
FPGA as the first step in Nova. All the threads I've see go well off
into the weeds about trying to solve everybody's niche/edge cases
perfectly and as a result get very complicated.

For both NUMA and PCI dev assignment we got initial success by cutting
back scope and focusing on the doing the minimum possible to satisfy
the 90% common use cases, and ignoring the less common 10% initially.
Yes this is not optimal, but it is good enough to keep most people
happy without introducing massive complexity into the designs & impl.

For FPGA, I'd like to see an initial proposal that assumed the FPGA
is pre-programmed & pre-divided into a fixed number of slots and simply
deal with this. This is similar to how we dealt with PCI SR-IOV initially
where we assumed the dev is in VF-mode only. Only later did we start to
add cleverness around switching VF vs PF mode. For FPGA I think any kind
of dynamic re-allocation/re-configuration is better done as a stage 2

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [horizon] Plugin stability, failing tests etc.

[Rob Cresswell]

Hey all,

So we've had a few issues with plugin stability recently, and its apparent that 
many plugins are building off Horizon master as a dependency. I would really 
advise against this. A more manageable development process may be to:

- Base stable plugins against a stable release of Horizon
- Base WIP versions/new plugins off the last Horizon milestone, b2 in this 
case, and then bump the version and capture issues within the same patch. This 
should prevent random breakages, and should allow you to just look at the 
release notes for that milestone.

On the Horizon side, we've moved our FF a week earlier, so I think that week 
combined with the usual RC period should be enough time to fix bugs. We'll aim 
to make sure our release notes are complete with regards to breaking issues for 
plugins, and deprecate appropriately.

Rob
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] Plugin stability, failing tests etc.

[Vitaly Gridnev]

Hi,

testing against stable release of horizon doesn't have any sense of me.
when you have tests running against master and them are almost always
passing it's much easier to found commit that caused a failure and fix the
problem that it's introduced. anyway, it's great news that FF is moved a
week earlier for horizon, thanks!

On Wed, Jul 20, 2016 at 12:12 PM, Rob Cresswell <
robert.cressw...@outlook.com> wrote:

> Hey all,
>
> So we've had a few issues with plugin stability recently, and its apparent
> that many plugins are building off Horizon master as a dependency. I would
> really advise against this. A more manageable development process may be to:
>
> - Base stable plugins against a stable release of Horizon
> - Base WIP versions/new plugins off the last Horizon milestone, b2 in this
> case, and then bump the version and capture issues within the same patch.
> This should prevent random breakages, and should allow you to just look at
> the release notes for that milestone.
>
> On the Horizon side, we've moved our FF a week earlier, so I think that
> week combined with the usual RC period should be enough time to fix bugs.
> We'll aim to make sure our release notes are complete with regards to
> breaking issues for plugins, and deprecate appropriately.
>
> Rob
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Best Regards,
Vitaly Gridnev,
Project Technical Lead of OpenStack DataProcessing Program (Sahara)
Mirantis, Inc
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [oslo.db] [CC neutron] CIDR overlap functionality and constraints

[Julien Danjou]

On Tue, Jul 19 2016, Mike Bayer wrote:

Hi Mike,

> We've developed a system by which CIDR math, such as that of detecting region
> overlaps, can be performed on a MySQL database within queries [1] [2].   This
> feature makes use of a custom stored function I helped to produce which
> provides functionality similar to that which Postgresql provides built in [3].
> SQLite also supports a simple way to add CIDR math functions as well which 
> I've
> demonstrated at [4].

This is pretty cool.
(though it emphasizes all the sadness and energy spent to level down
things to MySQL and SQLite, sigh)

> The general verbosity and unfamiliarity of these well known SQL features is
> understandably being met with trepidation.  I've identified that this
> trepidation is likely rooted in the fact that unlike the many other elaborate
> SQL features we use like ALTER TABLE, savepoints, subqueries, SELECT FOR
> UPDATE, isolation levels, etc. etc., there is no warm and fuzzy abstraction
> layer here that is both greatly reducing the amount of explicit code needed to
> produce and upgrade the feature, as well as indicating that "someone else" 
> will
> fix this system when it has problems.
>
> Rather than hobbling the entire Openstack ecosystem to using a small subset of
> what our relational databases are capable of, I'd like to propose that
> preferably somewhere in oslo.db, or elsewhere, we begin providing the
> foundation for the use of SQL features that are rooted in mechanisms such as
> triggers and small use of stored functions, and more specifically begin to
> produce network-math SQL features as the public API, starting with this one.

It none of this fits in SQLAlchemy nor alembic, I'd be glad to welcome
it in oslo.db – or better, something more generic, more re-usable and
less OpenStack-centric, if that's an option.

Cheers,
-- 
Julien Danjou
/* Free Software hacker
   https://julien.danjou.info */


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)

[Julien Danjou]

On Tue, Jul 19 2016, Clint Byrum wrote:

> Perhaps if we form and start working together as a group, we can disect
> why nothing happened, build consensus on the most important thing to do
> next, and actually fix some architectural problems. The social structure
> that teams have is a huge part of the deadlock we find ourselves in
> with certain controversial changes. The idea is to unroll the dependency
> loop and start _somewhere_ rather than where a lot of these efforts die:
> starting _everywhere_.

I agree with your analysis, but I fail to see how e.g. a group of people
X stating that Y should work this way in Cinder is going to achieve any
change if nobody from Cinder is in X from the beginning.

This is basically what seems to happen in many [working] groups as far
as I can see.

-- 
Julien Danjou
;; Free Software hacker
;; https://julien.danjou.info


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [new][documentation] openstackdocstheme 1.4.0 release

[no-reply]

We are pleased to announce the release of:

openstackdocstheme 1.4.0: OpenStack Docs Theme

With source available at:

http://git.openstack.org/cgit/openstack/openstackdocstheme

Please report issues through launchpad:

http://bugs.launchpad.net/openstack-manuals

For more details, please see below.

1.4.0
^

In preparation for releasing updated API reference documentation using
this theme, we have a collection of new features and fixes.


New Features


* The ability to customise the bug title for the 'Report a Bug' link
  is now available. To customise the bug title used add the
  "bug_title" key with a value to "html_context" in the Sphinx
  configuration.

  For example:

 html_context = {"bug_title": 'Documentation bug', ...}

* Ensure Javascript and CSS files are pulled in programmatically to
  enable custom Javascript and CSS files.

* CSS adjustments to "inline" markup and contents indentation.

* Enable custom bug title link.

* Adds sidebar_mode for table of contents as an option for
  html_theme_options in conf.py.

* The sidebar Table of Contents can now be set to the full "toc"
  directive, or remain as the "toctree" directive.

  This can be set by setting ""sidebar_mode"" to ""toc"" in the
  "html_theme_options" option in "conf.py".

  For example:

 html_theme_options = {
  "sidebar_mode": "toc",
   }


Bug Fixes
*

* Use HTTPS for external dependencies.

* Replace deprecated library function os.popen() with subprocess.
  (1529836)

* Update contribute link in footer. (1421814)

* Hide duplicate titles and empty tocs in generated content.

Changes in openstackdocstheme 1.3.0..1.4.0
--

84152e2 Update Release Notes
16c47a0 Adds release notes items for next release
e822907 Make ``something`` more highlighted
13bd976 Allow the bug title to be customisable
ebf3a77 Broken Link
9561d12 Updated from global requirements
710f1d8 Increase indent level of doc top contents
71afab3 Updated from global requirements
b86eddc Actually include custom JS files
75e8fc6 Release note for new sidebar feature
4f70614 Use HTTPS for external deps
7388b01 Set side bar content to be configurable
c8ccaa7 Create empty folder for custom font files
bed0526 Fix text font-family of admonitions.
0593452 Allow for inclusion of custom JS files
a217b4c Updated from global requirements
6c7f8e4 Change summit video URL
9bb0f5e Updated from global requirements
ed6dee2 Update the Administrator Guide link
cd2b088 Clarify uses for this theme
50df6ac Allow cssfiles added by sphinx extensions
c2b3775 Add appropriate order list styles


Diffstat (except docs and test files)
-

README.rst |   9 +-
RELEASENOTES.rst   | 121 
...configure_access_and_security_for_instances.rst |  35 +++---
openstackdocstheme/theme/openstackdocs/css.html|   4 +-
openstackdocstheme/theme/openstackdocs/header.html |   2 +-
openstackdocstheme/theme/openstackdocs/js.html |   5 +
openstackdocstheme/theme/openstackdocs/layout.html |   4 +
.../theme/openstackdocs/localtoc.html  |   2 +
.../theme/openstackdocs/script_footer.html |   6 +-
.../theme/openstackdocs/sidebartoc.html|   4 +
.../theme/openstackdocs/static/css/combined.css|  49 +---
.../theme/openstackdocs/static/fonts/.gitkeep  |   0
.../theme/openstackdocs/static/js/docs.js  |  16 +--
openstackdocstheme/theme/openstackdocs/theme.conf  |   1 +
releasenotes/notes/bug-title-fdbefea0408e2cbf.yaml |  13 +++
.../notes/custom-bug-link-ec64bdf9ce357d16.yaml|  18 +++
.../notes/side-bar-config-d7e66388e252cadf.yaml|  16 +++
releasenotes/source/current.rst|   5 +
releasenotes/source/historic.rst   | 125 +
releasenotes/source/index.rst  |   3 +-
releasenotes/source/unreleased.rst |   5 -
requirements.txt   |   2 +-
test-requirements.txt  |   4 +-
25 files changed, 290 insertions(+), 201 deletions(-)


Requirements updates


diff --git a/requirements.txt b/requirements.txt
index 11bed1e..7dfa598 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -6 +6 @@ pbr>=1.6 # Apache-2.0
-requests!=2.9.0,>=2.8.1 # Apache-2.0
+requests>=2.10.0 # Apache-2.0
diff --git a/test-requirements.txt b/test-requirements.txt
index 51cd316..1abb102 100644
--- a/test-requirements.txt
+++ b/test-requirements.txt
@@ -8 +8 @@ hacking<0.11,>=0.10.0
-sphinx!=1.2.0,!=1.3b1,<1.3,>=1.1.2 # BSD
+sphinx!=1.3b1,<1.3,>=1.2.1 # BSD
@@ -10 +10 @@ sphinx!=1.2.0,!=1.3b1,<1.3,>=1.1.2 # BSD
-reno>=0.1.1 # Apache2
+reno>=1.8.0 # Apache2



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@l

[openstack-dev] [TripleO] Delorean fail blocks CI for stable branches

[Sagi Shnaidman]

HI,

we have a problem with delorean build of stable branches in TripleO CI[1],
and it seems like a rpm specs problem. It can be reproducible easily [2]
Please, help with solution to this problem, all info is in the bug[1]

Alan,
if you think we use dlrn wrong, please point me out which line is incorrect
in reproducing.

[1] https://bugs.launchpad.net/tripleo/+bug/1604039
[2] Reproducing:

sudo yum install -y createrepo git mock rpm-build yum-plugin-priorities
yum-utils gcc python-virtualenv libffi-devel openssl-devel
sudo usermod -G mock -a $(id -nu)

cd /tmp/
sudo rm -rf /tmp/test
mkdir /tmp/test && cd /tmp/test

git clone https://git.openstack.org/openstack/tripleo-heat-templates
cd tripleo-heat-templates/
git checkout -b stable/mitaka origin/stable/mitaka

cd ..
git clone https://github.com/openstack-packages/delorean.git
cd delorean
mkdir -p data
sed -i -e 's%--postinstall%%' scripts/build_rpm.sh

virtualenv venv
./venv/bin/pip install -U setuptools
./venv/bin/pip install pytz
./venv/bin/pip install .

sed -i -e "s%baseurl=.*%baseurl=https://trunk.rdoproject.org/centos7-mitaka%";
projects.ini
sed -i -e "s%distro=.*%distro=rpm-mitaka%" projects.ini
sed -i -e "s%source=.*%source=stable/mitaka%" projects.ini

cp -r ../tripleo-heat-templates data/openstack-tripleo-heat-templates

cd data/openstack-tripleo-heat-templates/
GITHASH=$(git rev-parse HEAD)
for BRANCH in master origin/master stable/liberty origin/stable/liberty
stable/mitaka origin/stable/mitaka; do
git checkout -b $BRANCH || git checkout $BRANCH
git reset --hard $GITHASH
done
cd /tmp/test/delorean

./venv/bin/dlrn --config-file projects.ini --head-only --package-name
openstack-tripleo-heat-templates --local


The projects.ini:

[DEFAULT]
datadir=./data
scriptsdir=./scripts
baseurl=https://trunk.rdoproject.org/centos7-mitaka
distro=rpm-mitaka
source=stable/mitaka
target=centos
smtpserver=
reponame=delorean
templatedir=./dlrn/templates
maxretries=3
pkginfo_driver=dlrn.drivers.rdoinfo.RdoInfoDriver
tags=
#tags=mitaka
rsyncdest=
rsyncport=22

[gitrepo_driver]


-- 
Best regards
Sagi Shnaidman
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] Delorean fail blocks CI for stable branches

[Alan Pevec]

> git clone https://git.openstack.org/openstack/tripleo-heat-templates
> cd tripleo-heat-templates/
> git checkout -b stable/mitaka origin/stable/mitaka

^ this is manually switching to the stable source branch

> sed -i -e "s%distro=.*%distro=rpm-mitaka%" projects.ini
> sed -i -e "s%source=.*%source=stable/mitaka%" projects.ini

^ this configures dlrn to the correct combination of distro and source
branches, but ...

> ./venv/bin/dlrn --config-file projects.ini --head-only --package-name
> openstack-tripleo-heat-templates --local

^ ... --local here keeps local checkout untouched, so you end up with
default rpm-master in distro git checkout.
If you remove --local it will reset local checkouts to the branches
specified in projects.ini

Alan

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Magnum] Does Docker-Swarm not support inter-node/vm inter-container networking ?

[Waines, Greg]

Thanks Ton,

When is the Docker libnetwork functionality forecasted to be available ?

Greg.

From: Ton Ngo 
Reply-To: "openstack-dev@lists.openstack.org" 

Date: Tuesday, July 19, 2016 at 6:58 PM
To: "openstack-dev@lists.openstack.org" 
Subject: Re: [openstack-dev] [Magnum] Does Docker-Swarm not support 
inter-node/vm inter-container networking ?


Hi Greg,
This is a capability that we are working on at this time: enabling Docker 
libnetwork using the Kuryr driver.
This will let you set up networks where the containers are connected to and 
they will be allocated unique routable IP.
In the current default networking mode, you can still let container provides 
service to each other through Docker
port mapping. The end point for the container service would be the IP address 
of the VM where the container is hosted
together with the port mapped. You just cannot ping from one container to 
another across VM's in this mode.
Ton Ngo,


[nactive hide details for "Waines, Greg" ---07/19/2016 11:12:50 AM---I 
hav]"Waines, Greg" ---07/19/2016 11:12:50 AM---I have successfully setup 
devstack with Magnum, following this link https://github.com/openstack/mag

From: "Waines, Greg" 
To: "openstack-dev@lists.openstack.org" 
Date: 07/19/2016 11:12 AM
Subject: [openstack-dev] [Magnum] Does Docker-Swarm not support inter-node/vm 
inter-container networking ?





I have successfully setup devstack with Magnum,
following this link 
https://github.com/openstack/magnum/blob/master/doc/source/dev/quickstart.rst#building-and-using-a-swarm-bay

I created a Docker-Swarm bay and successfully launched some simple containers.

I noticed that Docker-Swarm’s Container Networking Solution seems to simply be 
an SNAT on its swarm-node VM.
And noticed that Docker-Swarm assigns the same Container subnet to each 
swarm-node VM … and re-uses Container IP Addresses from this subnet across 
swarm-node VMs.

Given this … it does not appear that Docker-Swarm can support networking 
between two containers on different swarm-node VMs.
Is this True ?
OR
Is there a configuration option, thru Magnum or Docker-Swarm to enable this 
inter-node inter-container Networking ?

Greg.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] Plugin stability, failing tests etc.

[Hayes, Graham]

On 20/07/2016 10:16, Rob Cresswell wrote:
> Hey all,
>
> So we've had a few issues with plugin stability recently, and its
> apparent that many plugins are building off Horizon master as a
> dependency. I would really advise against this. A more manageable
> development process may be to:
>
> - Base stable plugins against a stable release of Horizon
> - Base WIP versions/new plugins off the last Horizon milestone, b2 in
> this case, and then bump the version and capture issues within the same
> patch. This should prevent random breakages, and should allow you to
> just look at the release notes for that milestone.

So this would mean changing tox.ini or requirements files after each
horizon release?

This dovetails nicely with the other thread about how we should be doing
cross project interactions.

What would be best, would be having horizon released as a separate
library on pypi like the clients and oslo libs.

Then openstack-dashboard, and all the plugins could rely on the same
base library, without hacks like:

   deps =
 -r{toxinidir}/requirements.txt
 -r{toxinidir}/test-requirements.txt
 http://tarballs.openstack.org/horizon/horizon-master.tar.gz

in tox.ini or

   # Testing Requirements
   http://tarballs.openstack.org/horizon/horizon-master.tar.gz#egg=horizon

in (test-)requirements.txt

Is that on roadmap?

> On the Horizon side, we've moved our FF a week earlier, so I think that
> week combined with the usual RC period should be enough time to fix
> bugs. We'll aim to make sure our release notes are complete with regards
> to breaking issues for plugins, and deprecate appropriately.
>
> Rob


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Magnum] Does Docker-Swarm not support inter-node/vm inter-container networking ?

[Ton Ngo]

Hi Greg,
We should have patches in the next few weeks and the target is to have
this functionality in the Newton cycle.
Ton Ngo,



From:   "Waines, Greg" 
To: "OpenStack Development Mailing List (not for usage questions)"

Date:   07/20/2016 04:51 AM
Subject:Re: [openstack-dev] [Magnum] Does Docker-Swarm not support
inter-node/vm inter-container networking ?



Thanks Ton,

When is the Docker libnetwork functionality forecasted to be available ?

Greg.

From: Ton Ngo 
Reply-To: "openstack-dev@lists.openstack.org"

Date: Tuesday, July 19, 2016 at 6:58 PM
To: "openstack-dev@lists.openstack.org" 
Subject: Re: [openstack-dev] [Magnum] Does Docker-Swarm not support
inter-node/vm inter-container networking ?



Hi Greg,
This is a capability that we are working on at this time: enabling Docker
libnetwork using the Kuryr driver.
This will let you set up networks where the containers are connected to and
they will be allocated unique routable IP.
In the current default networking mode, you can still let container
provides service to each other through Docker
port mapping. The end point for the container service would be the IP
address of the VM where the container is hosted
together with the port mapped. You just cannot ping from one container to
another across VM's in this mode.
Ton Ngo,


nactive hide details for "Waines, Greg" ---07/19/2016 11:12:50 AM---I hav
"Waines, Greg" ---07/19/2016 11:12:50 AM---I have successfully setup
devstack with Magnum, following this link https://github.com/openstack/mag

From: "Waines, Greg" 
To: "openstack-dev@lists.openstack.org" 
Date: 07/19/2016 11:12 AM
Subject: [openstack-dev] [Magnum] Does Docker-Swarm not support
inter-node/vm inter-container networking ?




I have successfully setup devstack with Magnum,
following this link
https://github.com/openstack/magnum/blob/master/doc/source/dev/quickstart.rst#building-and-using-a-swarm-bay


I created a Docker-Swarm bay and successfully launched some simple
containers.

I noticed that Docker-Swarm’s Container Networking Solution seems to simply
be an SNAT on its swarm-node VM.
And noticed that Docker-Swarm assigns the same Container subnet to each
swarm-node VM … and re-uses Container IP Addresses from this subnet across
swarm-node VMs.

Given this … it does not appear that Docker-Swarm can support networking
between two containers on different swarm-node VMs.
Is this True ?
OR
Is there a configuration option, thru Magnum or Docker-Swarm to enable this
inter-node inter-container Networking ?

Greg.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [monasca] Monasca Mid-Cycle Minutes

[Fabio Giannetti (fgiannet)]

Notes for Monasca July Mid Cycle 2016

Minutes July 19 from 10:30am PDT to

Dimensions Names/Values resources

This blueprint needs to be updated. The PATCH part needs to be updated. The new 
URL is metrics/dimensions/names/values
The use case is driven by Grafana and is part of the templating. In order to 
get the list of unique dimension names you have to do a query for all the 
metrics.
The blueprint is implemented already in Java and Python and the implementation 
has been gone through a good testing. It has been implemented for Vertica and 
InfluxDB.
Brad to update the blueprint to relect the latest design changes.

Open Discussion
Is the vision for this project to be only for Openstack or it is possible to 
extend it to other projects?
The only dependency of the project is on Keystone and once that is removed the 
project can be used.
It is possible in the Python version to remove Keystone from the Pipeline and 
manually provide the header data so then the API will find the context.
Making Monasca more separate from Openstack. This will require a pluggable 
authentication mechanism.
Grafana already supports various Multitenancy capabilties but Kibana is not 
that flexible. It seems both support LDAP.

Retrospective

What we have done good
Cool New feature added. Logging API, deterministic alarms, periodic 
notificaitons
Good progress in adding new features
Multiple metrics was a good perfomance improvement. From 60s to 1s.
Well integrated in the Openstack process.
Good/Flexible architecture
More visibility in the community. Limited contributions from the "other" teams.

What we could have done better
Installation is still complex and cumbersome, not well documented. Need of a 
Monasca administration guide. Better docs in general would help.
Create an official tutorial.
Replacing Java API in Persister. It is difficult for new contributor to come to 
the project. Single API change can take 2 weeks. Java+Python and 3 databases.
Focus on polishing what is already available instead of expanding the scope.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Glare][Heat][TripleO] Heat artifact type

[Mikhail Fedosin]

On Wed, Jul 20, 2016 at 5:12 AM, Qiming Teng 
wrote:

> On Tue, Jul 19, 2016 at 06:44:06PM +0300, Oleksii Chuprykov wrote:
> > Hello!
> >
> > Today it was announced that Glare is ready for public review
> > http://lists.openstack.org/pipermail/openstack-dev/2016-July/099553.html
> So
> > we are ready to start working on integration Heat with Glare and
> > implementing a POC. After discussions with Glare team we see two design
> > options:
> >
> > 1) Create one artifact type that will contain template, nested templates
> > and environments.
> > Pros: It is easy to maintain integrity. Since artifact is immutable, we
> can
> > guarantee the consistency and prevent from accidentally removing of
> > dependent environment.
> > Cons: If we need to add new environments to use them with template, we
> need
> > to create new artifact.
> >
> > 2) Create 2 artifact types: environment and template.
> > Pros: It is easy to add new environments. You just need to create new
> > dependency from template artifact to environment one.
> > Cons: Some environment can be (mistakenly) removed, and template that
> have
> > dependencies on it will be in inconsistent state.
>
> Option 2 looks more flexible to me. I'm not sure we are encouraging
> users to introduce or rely on a hard dependency from a template to an
> environment file. With that, it is still good to know whether glare
> supports the concept of 'reference' where a referenced artifact cannot
> be deleted.
>

Hey!

Indeed, option 2 is more flexible, but in this case users have to manually
control dependencies, which is may be hard sometimes. Also, initially Glare
won't support 'hard' dependencies, this feature will be added in next
version, because it requires additional discussions. For this reason I
recommend option 1 and let Glare control template consistency for you, it
won't allow users to break anything.

Best,
Mike


>
>  - Qiming
>

> > So we want to hear your opinions and suggestions on the matter. Thanks in
> > advance!
> >
> > Best regards,
> > Oleksii Chuprykov
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [requirements] Project Mascot

[Swapnil Kulkarni (coolsvap)]

Hello,

We had a discussion about project mascot for requirements team today
at the team meeting. Some of the options mentioned are added to [1].
If you have any option, please add to the list.
Also do not forget to provide preference. We will finalize the mascot
on Monday July 25.

[1] https://etherpad.openstack.org/p/requirements-team-mascot

--
Best Regards,
Swapnil

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [devstack] How to enable SSL in devStack?

[Rob Crittenden]

Andrey Pavlov wrote:

Hi,

When I ran devstack with SSL I found a bug and tried to fix it -
https://review.openstack.org/#/c/242812/
But no one agree with me.
Try to apply this patch - it may help.
Also there is a chance that new bugs present in devstack that
prevented to install it with SSL.


Seeing how some other things in your local.conf might help but when I 
tried to reproduce it I got the same error and it failed because Apache 
didn't have an SSL listener on 443.


I'm not sure I'd recommend direct SSL in any case. I'd recommend the 
tls-proxy service instead. Note that I'm pretty sure it has the same 
problem: it hasn't been updated to handle port 443 for Keystone.


I'm working on switching from stud to mod_proxy if you want to take a 
look and this problem is fixed there, https://review.openstack.org/301172


I'll see about adding a SSL listener to Keystone for the USE_SSL case in 
the next few days.


And yeah, it's a moving target. I have an experimental gate test for 
tlsproxy but it has to be requested explicitly. My plan is to enable it 
as non-voting once the mod_proxy changes land so it will at least be 
more obvious when things break (or maybe we can making it voting).


rob

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] Plugin stability, failing tests etc.

[Rob Cresswell]

Yes, it would mean changing your requirements after a release. So, for example 
you might run two gate tests:

- A voting Horizon-stable/milestone test, (or both)
- A non-voting Horizon-master test

That gives you a lot of control over making your tests passing (multiple 
patches to make the Horizon-master test pass, or all in one patch set alongside 
the horizon-milestone test bump), rather than random failures each week. You'd 
still be able to track the failures as they come in, but they won't break your 
gate each time.

I don't think where horizon (the library) lives would change how you version 
against it. We don't currently have any plans to separate the two; while we 
realise its a desirable change, weighing the work and risk involved in the 
split architecture vs the end result, we've chosen to work on other higher 
priority items for now (performance, filtering improvements, angular work etc.)

Rob



On 20 July 2016 at 13:38, Hayes, Graham 
mailto:graham.ha...@hpe.com>> wrote:
On 20/07/2016 10:16, Rob Cresswell wrote:
> Hey all,
>
> So we've had a few issues with plugin stability recently, and its
> apparent that many plugins are building off Horizon master as a
> dependency. I would really advise against this. A more manageable
> development process may be to:
>
> - Base stable plugins against a stable release of Horizon
> - Base WIP versions/new plugins off the last Horizon milestone, b2 in
> this case, and then bump the version and capture issues within the same
> patch. This should prevent random breakages, and should allow you to
> just look at the release notes for that milestone.

So this would mean changing tox.ini or requirements files after each
horizon release?

This dovetails nicely with the other thread about how we should be doing
cross project interactions.

What would be best, would be having horizon released as a separate
library on pypi like the clients and oslo libs.

Then openstack-dashboard, and all the plugins could rely on the same
base library, without hacks like:

   deps =
 -r{toxinidir}/requirements.txt
 -r{toxinidir}/test-requirements.txt
 http://tarballs.openstack.org/horizon/horizon-master.tar.gz

in tox.ini or

   # Testing Requirements
   http://tarballs.openstack.org/horizon/horizon-master.tar.gz#egg=horizon

in (test-)requirements.txt

Is that on roadmap?

> On the Horizon side, we've moved our FF a week earlier, so I think that
> week combined with the usual RC period should be enough time to fix
> bugs. We'll aim to make sure our release notes are complete with regards
> to breaking issues for plugins, and deprecate appropriately.
>
> Rob


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Monasca] Virtual Mid Cycle Coordinates - July 20

[Fabio Giannetti (fgiannet)]

>Monasca Mid Cycle Day 2
>July 20 2016
>7am to noon PDT
>
>Webex
>
>
>Join WebEx meeting:
>https://cisco.webex.com/ciscosales/j.php?MTID=m84f9f81d7c1c171be6365716522
>d
>e15e
>
>Meeting number: 205 141 519
>Meeting password: 8VyzUiyr
>
>Join by phone  
>+1-408-525-6800 Call-in toll number (US/Canada)
>+1-866-432-9903 Call-in toll-free number (US/Canada)
>Access code: 205 141 519
>Numeric meeting password: 01558880


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [heat][yaql] Heat map replacement options

[Zane Bitter]

On 19/07/16 11:04, Steven Hardy wrote:

On Fri, Jul 15, 2016 at 10:25:39AM +0100, Steven Hardy wrote:

Hi all,

I'm trying to figure out the cleanest way to do a replacement of values in
a mapping (json parameter) in a heat template, e.g:

  ServiceNetMap:
type: json
default:
  IronicApiNetwork: internal_api
  CephPublicNetwork: storage

  NetIpMap:
type: json
default:
  storage: 192.0.2.2
  internal_api: 192.0.2.5

How do I get
  OutputMap:
IronicApiNetwork: 192.0.2.5
CephPublicNetwork: 192.0.2.2

It seems like something yaql should be able to do, but I've so far failed
to figure out the syntax.

The other (possibly simpler) possibility is to implement a new hot
function, e.g something like:

  map_replace:
template: {get_param: ServiceNetMap}
value_replacements: {get_param: NetIpMap}


As a follow-up to this, I have posted this spec and implementation for
map_replace (syntax differs slightly from the keys above):

https://review.openstack.org/#/c/343696/

https://review.openstack.org/#/c/343731/

My aim is a simpler interface to the yaql example previously discussed, and
a better error path when things like key collisions occur. Reviews welcome! :)


This all merged in super-quick time before I even had the chance to 
look, but I was wondering... is there really any difference between this 
and str_replace?


Right now str_replace requires the template to be a string, but I can't 
see any reason for that to be the case. Why not just relax that 
requirement so the user can pass map or a list and any strings in the 
values/items will get (recursively) replaced by values from the params?


That'd be simpler than adding a separate function, and more flexible too 
since you can replace parts of values not just the whole thing (and it 
would also handle lists and nested data). The map_replace is very tied 
to this one particular use case, where the input is a map and the things 
you want to replace are top-level values in their entirety.


Just a thought.

cheers,
Zane.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Glare][Heat][TripleO] Heat artifact type

[Randall Burt]

FWIW, option 2 is almost required unless we plan to be able to bundle multiple 
environments with a single template. While having a single environment for a 
single template can be useful, the even *more* useful scenario (and the primary 
one driving the development of environments initially) is when you have options 
as to how a template behaves (use Trove for the backend or pop vms and software 
config to install a database). IMO, you'd want to de-couple environments from 
the templates given that multiple environment could work for the same template.
 
On Jul 20, 2016, at 8:58 AM, "Mikhail Fedosin" 
 wrote:

> 
> 
> On Wed, Jul 20, 2016 at 5:12 AM, Qiming Teng  
> wrote:
> On Tue, Jul 19, 2016 at 06:44:06PM +0300, Oleksii Chuprykov wrote:
> > Hello!
> >
> > Today it was announced that Glare is ready for public review
> > http://lists.openstack.org/pipermail/openstack-dev/2016-July/099553.html So
> > we are ready to start working on integration Heat with Glare and
> > implementing a POC. After discussions with Glare team we see two design
> > options:
> >
> > 1) Create one artifact type that will contain template, nested templates
> > and environments.
> > Pros: It is easy to maintain integrity. Since artifact is immutable, we can
> > guarantee the consistency and prevent from accidentally removing of
> > dependent environment.
> > Cons: If we need to add new environments to use them with template, we need
> > to create new artifact.
> >
> > 2) Create 2 artifact types: environment and template.
> > Pros: It is easy to add new environments. You just need to create new
> > dependency from template artifact to environment one.
> > Cons: Some environment can be (mistakenly) removed, and template that have
> > dependencies on it will be in inconsistent state.
> 
> Option 2 looks more flexible to me. I'm not sure we are encouraging
> users to introduce or rely on a hard dependency from a template to an
> environment file. With that, it is still good to know whether glare
> supports the concept of 'reference' where a referenced artifact cannot
> be deleted.
> 
> Hey! 
> 
> Indeed, option 2 is more flexible, but in this case users have to manually 
> control dependencies, which is may be hard sometimes. Also, initially Glare 
> won't support 'hard' dependencies, this feature will be added in next 
> version, because it requires additional discussions. For this reason I 
> recommend option 1 and let Glare control template consistency for you, it 
> won't allow users to break anything. 
> 
> Best,
> Mike
>  
> 
>  - Qiming
> 
> > So we want to hear your opinions and suggestions on the matter. Thanks in
> > advance!
> >
> > Best regards,
> > Oleksii Chuprykov
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)

[James Bottomley]

On Wed, 2016-07-20 at 11:58 +0200, Julien Danjou wrote:
> On Tue, Jul 19 2016, Clint Byrum wrote:
> 
> > Perhaps if we form and start working together as a group, we can 
> > disect why nothing happened, build consensus on the most important 
> > thing to do next, and actually fix some architectural problems.

Architecture is often blamed for lack of interlock but most people
forget that the need for interlock often isn't appreciated until after
the components are built.  This is why a lot of people embrace agile
methodology (an excuse for not seeing the problem a priori).

Conversely, architects who try to foresee all interlocks often end up
with a combinatorial explosion that makes it a huge burden simply to
begin the project (and then often get sidelined as 'powerpoint
engineers').

The trick is to do something in the middle: try to foresee and build
the most common interlocks, but sidestep the combinatorial explosion by
building something simple enough to adapt to any interlock requirement
that arises after completion.

> >  The social structure that teams have is a huge part of the
> > deadlock we find ourselves in with certain controversial changes.
> > The idea is to unroll the dependency loop and start _somewhere_
> > rather than where a lot of these efforts die: starting
> > _everywhere_.
> 
> I agree with your analysis, but I fail to see how e.g. a group of 
> people X stating that Y should work this way in Cinder is going to 
> achieve any change if nobody from Cinder is in X from the beginning.
> 
> This is basically what seems to happen in many [working] groups as 
> far as I can see.

So this is where the Open Source method takes over.  Change is produced
by those people who most care about it because they're invested.  To
take your Cinder example, you're unlikely to find them within Cinder
because any project has inertia that resists change.  It takes the
energy of the outside group X to force the change to Y, but this means
that X often gets to propose, develop and even code Y.  Essentially
they become drive by coders for Cinder.  This is where Open Source
differs from Enterprise because you have the code and the access, you
can do this.  However, you have to remember the inertia problem and
build what you're trying to do as incrementally as possible: the larger
the change, the bigger the resistance to it.  It's also a good test of
the value of the change: if group X can't really be bothered to code it
(and Cinder doesn't want it) then perhaps there's not really enough
value in it anyway and it shouldn't happen.

This latter observation is also an improvement over the enterprise
methods because enterprise architects do often mandate interlocks that
later turn out to be unnecessary (or at least of a lot less value than
was initially thought).

I suppose the executive summary of the above is that I don't think
you're describing a bug, I think you're describing a feature.

James


signature.asc
Description: This is a digitally signed message part
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [searchlight] Thursday July 21 meeting cancelled

[Tripp, Travis S]

Due to our midcycle hangout today [0], we are cancelling the searchlight IRC 
meeting tomorrow (Thursday July 21).

[0] https://etherpad.openstack.org/p/searchlight-newton-hangout
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] Delorean fail blocks CI for stable branches

[Sagi Shnaidman]

On Wed, Jul 20, 2016 at 2:29 PM, Alan Pevec  wrote:

> > git clone https://git.openstack.org/openstack/tripleo-heat-templates
> > cd tripleo-heat-templates/
> > git checkout -b stable/mitaka origin/stable/mitaka
>
> ^ this is manually switching to the stable source branch
>
> > sed -i -e "s%distro=.*%distro=rpm-mitaka%" projects.ini
> > sed -i -e "s%source=.*%source=stable/mitaka%" projects.ini
>
> ^ this configures dlrn to the correct combination of distro and source
> branches, but ...
>
> > ./venv/bin/dlrn --config-file projects.ini --head-only --package-name
> > openstack-tripleo-heat-templates --local
>
> ^ ... --local here keeps local checkout untouched, so you end up with
> default rpm-master in distro git checkout.
> If you remove --local it will reset local checkouts to the branches
> specified in projects.ini
>
> Alan,
I don't want to reset local checkouts and reset branches - I need to build
with these checkout, it's all CI is for.


> Alan
>



-- 
Best regards
Sagi Shnaidman
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [new][glance] glance_store 0.14.0 release (newton)

[no-reply]

We are psyched to announce the release of:

glance_store 0.14.0: OpenStack Image Service Store Library

This release is part of the newton release series.

With source available at:

http://git.openstack.org/cgit/openstack/glance_store

With package available at:

https://pypi.python.org/pypi/glance_store

Please report issues through launchpad:

http://bugs.launchpad.net/glance-store

For more details, please see below.

Changes in glance_store 0.13.0..0.14.0
--

79532ea Add bandit to pep8 and bandit testenv
fd84300 Remove unused variable in vmware store
84e6354 Imported Translations from Zanata
6e7c722 Updated from global requirements
4b6818d Check that size is a number
19d8df3 Replace dict.iterkeys with six.iterkeys to make PY3 compatible
3e20ff9 cinder: Fix get_size return value
5c07499 The function add calculation size_gb need improve
4b10efd Updated from global requirements
a3b298e Updated from global requirements
fd1e846 Fix argument order for assertEqual to (expected, observed)
f0087f3 Updated from global requirements
a678c26 Updated from global requirements
5829046 Remove -c from tox.ini
ea4483c tox respects upper-constraints.txt
9a58812 Updated from global requirements
922233f Updated from global requirements
55af8b5 Updated from global requirements
75cd233 Updated from global requirements
0a6124f Fix minor misspellings affecting Config Reference Guide
6f4bf26 Remove verbose option from glance_store tests
2e93319 Updated from global requirements
8eda73b Updated from global requirements
0a606a5 Improve help text of swift driver opts
40dded6 Updated from global requirements
1e87dfd Add functional tests for swift
25e5d19 Imported Translations from Zanata
7b439d6 Updated from global requirements
32d964f Updated from global requirements
4412580 Fix releasenotes to pass reno gates
3758022 Updated from global requirements
7b94d3c tox: use os-testr instead of testr
9addf29 Fix swiftclient mocks
ba8e51f Deprecate swift driver options properly
da74173 Fix typos in config files
1ae475c Setup defaults for swift driver authentication
1ce1f40 Fix doc generation warnings and errors
200249a trivial:fixing one W503 pep8 error
5c6c6e6 Module docs are not generated
3b53b0b Fix cinder store to support Cinder RemoteFS backends
3b637b4 Missing params in store_add_to_backend docstring
851508c Mock swiftclient's functions in tests
1d2474b Update reno for stable/mitaka
a9d6cce Add base for functional tests


Diffstat (except docs and test files)
-

.gitignore |   3 +
.testr.conf|   2 +-
functional_testing.conf.sample |   9 +
glance_store/_drivers/cinder.py|  12 +-
glance_store/_drivers/filesystem.py|   4 +-
glance_store/_drivers/http.py  |   2 +-
glance_store/_drivers/sheepdog.py  |   7 +-
glance_store/_drivers/swift/store.py   | 196 +++---
glance_store/_drivers/swift/utils.py   |  36 ++-
glance_store/_drivers/vmware_datastore.py  |   7 +-
glance_store/backend.py|   2 +
.../en_GB/LC_MESSAGES/glance_store-log-warning.po  |  19 ++
.../locale/en_GB/LC_MESSAGES/glance_store.po   | 246 +
.../es/LC_MESSAGES/glance_store-log-warning.po |   8 +-
.../fr/LC_MESSAGES/glance_store-log-warning.po |   8 +-
glance_store/locale/glance_store-log-warning.pot   |  25 --
glance_store/locale/glance_store.pot   | 290 -
...event-unauthorized-errors-ebb9cf2236595cd0.yaml |  12 +-
releasenotes/source/index.rst  |   1 +
.../locale/en_GB/LC_MESSAGES/releasenotes.po   | 113 
releasenotes/source/mitaka.rst |   6 +
requirements.txt   |  12 +-
setup.cfg  |   8 +-
test-requirements.txt  |  12 +-
tools/colorizer.py |   3 +-
tools/tox_install.sh   |  55 
tox.ini|  34 ++-
42 files changed, 1167 insertions(+), 557 deletions(-)


Requirements updates


diff --git a/requirements.txt b/requirements.txt
index f102881..a83ce1a 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -4 +4 @@
-oslo.config>=3.7.0 # Apache-2.0
+oslo.config>=3.10.0 # Apache-2.0
@@ -7,3 +7,3 @@ oslo.serialization>=1.10.0 # Apache-2.0
-oslo.utils>=3.5.0 # Apache-2.0
-oslo.concurrency>=3.5.0 # Apache-2.0
-stevedore>=1.5.0 # Apache-2.0
+oslo.utils>=3.14.0 # Apache-2.0
+oslo.concurrency>=3.8.0 # Apache-2.0
+stevedore>=1.10.0 # Apache-2.0
@@ -17,2 +17,2 @@ jsonschema!=2.5.0,<3.0.0,>=2.0.0 # MIT
-python-keystoneclient!=1.8.0,!=2.1.0,>=1.6.0 # Apache-2.0
-requests!=2.9.0,>=2.8.1 # Apache-2.0
+python-keystoneclient!=1.8.0,!=2.1.0,>=1.7.0 # 

Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)

[Fox, Kevin M]

+1 to the finding of a middle ground.

The problem I've seen with your suggested OpenSource solution is the current 
social monetary system of OpenStack makes it extremely difficult.

Each project currently prints its own currency. Reviews. It takes quite a few 
Reviews (time/effort) on a project to gain enough currency that you get payed 
attention to. And, one project doesn't honor another projects currency.

When these sorts of major cross project things need to be done though, you need 
to have folks with capital in many different projects and its very difficult to 
amass that much.

There is no OpenStack level currency (other then being elected as a TC member) 
that gets one project to stop and take the time to carefully consider what 
someone is saying when bringing up cross project issues.

Without some shared currency, then the problem becomes, the person invested in 
getting a solution, can propose and prototype and implement the feature all 
they want (often several times), but it never actually is accepted into the 
projects because a project:
 a) doesn't take the time to really understand the problem, feeling its trivial 
and just not accepting any reviews
 b) doesn't take the time to really understand the problem, so minimize it in 
their mind to a 'you should just use existing thing X...' or a heavy handily 
propose alternate solutions that really aren't solutions.
 c) they decide its better handled by another project and stall/block reviews, 
trying to push the implementation to go elsewhere. When multiple projects 
decide this, the ball just keeps getting bounced around without any solution 
for years.

The status quo is not working here. The current governance structure doesn't 
support progress.

Thanks,
Kevin

From: James Bottomley [james.bottom...@hansenpartnership.com]
Sent: Wednesday, July 20, 2016 8:31 AM
To: OpenStack Development Mailing List (not for usage questions); Clint Byrum
Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)

On Wed, 2016-07-20 at 11:58 +0200, Julien Danjou wrote:
> On Tue, Jul 19 2016, Clint Byrum wrote:
>
> > Perhaps if we form and start working together as a group, we can
> > disect why nothing happened, build consensus on the most important
> > thing to do next, and actually fix some architectural problems.

Architecture is often blamed for lack of interlock but most people
forget that the need for interlock often isn't appreciated until after
the components are built.  This is why a lot of people embrace agile
methodology (an excuse for not seeing the problem a priori).

Conversely, architects who try to foresee all interlocks often end up
with a combinatorial explosion that makes it a huge burden simply to
begin the project (and then often get sidelined as 'powerpoint
engineers').

The trick is to do something in the middle: try to foresee and build
the most common interlocks, but sidestep the combinatorial explosion by
building something simple enough to adapt to any interlock requirement
that arises after completion.

> >  The social structure that teams have is a huge part of the
> > deadlock we find ourselves in with certain controversial changes.
> > The idea is to unroll the dependency loop and start _somewhere_
> > rather than where a lot of these efforts die: starting
> > _everywhere_.
>
> I agree with your analysis, but I fail to see how e.g. a group of
> people X stating that Y should work this way in Cinder is going to
> achieve any change if nobody from Cinder is in X from the beginning.
>
> This is basically what seems to happen in many [working] groups as
> far as I can see.

So this is where the Open Source method takes over.  Change is produced
by those people who most care about it because they're invested.  To
take your Cinder example, you're unlikely to find them within Cinder
because any project has inertia that resists change.  It takes the
energy of the outside group X to force the change to Y, but this means
that X often gets to propose, develop and even code Y.  Essentially
they become drive by coders for Cinder.  This is where Open Source
differs from Enterprise because you have the code and the access, you
can do this.  However, you have to remember the inertia problem and
build what you're trying to do as incrementally as possible: the larger
the change, the bigger the resistance to it.  It's also a good test of
the value of the change: if group X can't really be bothered to code it
(and Cinder doesn't want it) then perhaps there's not really enough
value in it anyway and it shouldn't happen.

This latter observation is also an improvement over the enterprise
methods because enterprise architects do often mandate interlocks that
later turn out to be unnecessary (or at least of a lot less value than
was initially thought).

I suppose the executive summary of the above is that I don't think
you're describing a bug, I think you're describing a feature.

James

Re: [openstack-dev] FPGA as a dynamic nested resources

[Ed Leafe]

On Jul 20, 2016, at 2:07 AM, Daniel P. Berrange  wrote:

> For FPGA, I'd like to see an initial proposal that assumed the FPGA
> is pre-programmed & pre-divided into a fixed number of slots and simply
> deal with this. This is similar to how we dealt with PCI SR-IOV initially
> where we assumed the dev is in VF-mode only. Only later did we start to
> add cleverness around switching VF vs PF mode. For FPGA I think any kind
> of dynamic re-allocation/re-configuration is better done as a stage 2

+1 to this approach. I’m not convinced yet that Nova should be in the business 
of FPGA management, but once we get the basic functionality supporting FPGA 
working well, seeing what would be needed to add it would be much easier, and 
we could make a clearer determination as to whether this is feasible or not.


-- Ed Leafe






__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Glare][Heat][TripleO] Heat artifact type

[Fox, Kevin M]

I have a preference towards option 2 as well. I usually use templates with all 
the logic in it, and an environment file with just the specific parameters 
defined for launching an instance of the template so I can repeatedly 
deploy/delete/redeploy it.

I've got a good template set I think that would be awesome to see in a glare 
artefact.

Could this template set be wrapped up:
https://github.com/EMSL-MSC/heat-templates/tree/master/cfn/lib

And the main entrypoint template is:
https://github.com/EMSL-MSC/heat-templates/blob/master/cfn/lib/SimpleServer.yaml

Documentation on how to use it is here:
https://github.com/EMSL-MSC/heat-templates/blob/master/cfn/lib/SimpleServer.txt

With it implemented with Option 2, the user can just copy the two example 
environments at the bottom of the docs there, tweak it slightly and launch some 
fairly advanced servers.

Thanks,
Kevin

From: Mikhail Fedosin [mfedo...@mirantis.com]
Sent: Wednesday, July 20, 2016 6:58 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Glare][Heat][TripleO] Heat artifact type



On Wed, Jul 20, 2016 at 5:12 AM, Qiming Teng 
mailto:teng...@linux.vnet.ibm.com>> wrote:
On Tue, Jul 19, 2016 at 06:44:06PM +0300, Oleksii Chuprykov wrote:
> Hello!
>
> Today it was announced that Glare is ready for public review
> http://lists.openstack.org/pipermail/openstack-dev/2016-July/099553.html So
> we are ready to start working on integration Heat with Glare and
> implementing a POC. After discussions with Glare team we see two design
> options:
>
> 1) Create one artifact type that will contain template, nested templates
> and environments.
> Pros: It is easy to maintain integrity. Since artifact is immutable, we can
> guarantee the consistency and prevent from accidentally removing of
> dependent environment.
> Cons: If we need to add new environments to use them with template, we need
> to create new artifact.
>
> 2) Create 2 artifact types: environment and template.
> Pros: It is easy to add new environments. You just need to create new
> dependency from template artifact to environment one.
> Cons: Some environment can be (mistakenly) removed, and template that have
> dependencies on it will be in inconsistent state.

Option 2 looks more flexible to me. I'm not sure we are encouraging
users to introduce or rely on a hard dependency from a template to an
environment file. With that, it is still good to know whether glare
supports the concept of 'reference' where a referenced artifact cannot
be deleted.

Hey!

Indeed, option 2 is more flexible, but in this case users have to manually 
control dependencies, which is may be hard sometimes. Also, initially Glare 
won't support 'hard' dependencies, this feature will be added in next version, 
because it requires additional discussions. For this reason I recommend option 
1 and let Glare control template consistency for you, it won't allow users to 
break anything.

Best,
Mike


 - Qiming

> So we want to hear your opinions and suggestions on the matter. Thanks in
> advance!
>
> Best regards,
> Oleksii Chuprykov


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] Plugin stability, failing tests etc.

[Kirill Zaitsev]

Initially I don’t think I like the idea of making master-horizon job non-voting 
for murano-dashboard.

Here are some reasons: 
1) We would still need to fix murano-dashboard to work with master 
horizon (since we would need to be released together)
2) The breakage would be less visible
3) I can imagine a situation when a change would pass on master but 
would not pass on a previous stable point release (even worse this release can 
be n3). Say we’re trying to use some function/feature, that has just landed.

Those are my initial ideas about this, have give it a bit more time, to think 
more carefully.

BTW, I can fetch the numbers on how many times murano-dashboard gate was broken 
by changes in horizon, during M and N cycles, if you’re interested. Also for 
murano-dashboard we run our integration selenium tests as a 3d-party CI, so 
technically we’re not blocked by the job not working, in case we need to land 
some security patch. =)

-- 
Kirill Zaitsev
Murano Project Tech Lead
Software Engineer at
Mirantis, Inc

On 20 juillet 2016 at 17:10:46, Rob Cresswell (robert.cressw...@outlook.com) 
wrote:

Yes, it would mean changing your requirements after a release. So, for example 
you might run two gate tests:

- A voting Horizon-stable/milestone test, (or both)
- A non-voting Horizon-master test

That gives you a lot of control over making your tests passing (multiple 
patches to make the Horizon-master test pass, or all in one patch set alongside 
the horizon-milestone test bump), rather than random failures each week. You'd 
still be able to track the failures as they come in, but they won't break your 
gate each time.

I don't think where horizon (the library) lives would change how you version 
against it. We don't currently have any plans to separate the two; while we 
realise its a desirable change, weighing the work and risk involved in the 
split architecture vs the end result, we've chosen to work on other higher 
priority items for now (performance, filtering improvements, angular work etc.)

Rob



On 20 July 2016 at 13:38, Hayes, Graham  wrote:
On 20/07/2016 10:16, Rob Cresswell wrote:
> Hey all,
>
> So we've had a few issues with plugin stability recently, and its
> apparent that many plugins are building off Horizon master as a
> dependency. I would really advise against this. A more manageable
> development process may be to:
>
> - Base stable plugins against a stable release of Horizon
> - Base WIP versions/new plugins off the last Horizon milestone, b2 in
> this case, and then bump the version and capture issues within the same
> patch. This should prevent random breakages, and should allow you to
> just look at the release notes for that milestone.

So this would mean changing tox.ini or requirements files after each
horizon release?

This dovetails nicely with the other thread about how we should be doing
cross project interactions.

What would be best, would be having horizon released as a separate
library on pypi like the clients and oslo libs.

Then openstack-dashboard, and all the plugins could rely on the same
base library, without hacks like:

   deps =
     -r{toxinidir}/requirements.txt
     -r{toxinidir}/test-requirements.txt
     http://tarballs.openstack.org/horizon/horizon-master.tar.gz

in tox.ini or

   # Testing Requirements
   http://tarballs.openstack.org/horizon/horizon-master.tar.gz#egg=horizon

in (test-)requirements.txt

Is that on roadmap?

> On the Horizon side, we've moved our FF a week earlier, so I think that
> week combined with the usual RC period should be enough time to fix
> bugs. We'll aim to make sure our release notes are complete with regards
> to breaking issues for plugins, and deprecate appropriately.
>
> Rob


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__  
OpenStack Development Mailing List (not for usage questions)  
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe  
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev  


signature.asc
Description: Message signed with OpenPGP using AMPGpg
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] Delorean fail blocks CI for stable branches

[Alan Pevec]

>> ^ ... --local here keeps local checkout untouched, so you end up with
>> default rpm-master in distro git checkout.
>> If you remove --local it will reset local checkouts to the branches
>> specified in projects.ini
>>
> Alan,
> I don't want to reset local checkouts and reset branches - I need to build
> with these checkout, it's all CI is for.

DLRN finds matching packaging branch only when you let it reset git checkouts.
For tripleo-ci use-case we would need to add a new option to keep
source repo as-is and reset packaging checkout only, in the meantime
as a quickfix in tripleo.sh you could patch dlrn and set local=True in
[2]

Alan


[1] 
https://github.com/openstack-packages/DLRN/blob/master/dlrn/shell.py#L522-L536

[2] 
https://github.com/openstack-packages/DLRN/blob/master/dlrn/drivers/rdoinfo.py#L83-L84

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] Delorean fail blocks CI for stable branches

[Alan Pevec]

> as a quickfix in tripleo.sh you could patch dlrn and set local=True in

correction, patch local=False there while running dlrn command with
--local to keep source checkout as-is

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] OpenStack Kolla - External Ceph

[Jeffrey Zhang]

On Wed, Jul 6, 2016 at 1:47 AM, Steven Dake (stdake)  wrote:
>
> At this point, I feel we are going a direction in which we try to wrap
> everything anybody could possibly want to configure with Kolla by making
> extensive use of global.yml. We would have to introduce flags indicating a
> couple of different scenarios:
>
> 1. Deploy Ceph (already there: enable_ceph="yes")
> 2. Use Ceph with Glance (enable_ceph_glance="yes")
> 3. Use Ceph with Cinder (enable_ceph_cinder="yes")
> 4. Use Ceph with Nova (enable_ceph_nova="yes")
>
>
> I disagree.  If ceph is enabled, then ceph should be used, if ceph is not
> enabled, then ceph shouldn't be used.  That implies all of OpenStack either
> uses Ceph or not.  So we really just need enable_ceph.


why we need separate configuration for ceph? I think if ceph is
enable, all the service should use ceph, too.

-- 
Regards,
Jeffrey Zhang
Blog: http://xcodest.me

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] Plugin stability, failing tests etc.

[Hayes, Graham]

On 20/07/2016 15:13, Rob Cresswell wrote:
> Yes, it would mean changing your requirements after a release. So, for
> example you might run two gate tests:
>
> - A voting Horizon-stable/milestone test, (or both)
> - A non-voting Horizon-master test
>
> That gives you a lot of control over making your tests passing (multiple
> patches to make the Horizon-master test pass, or all in one patch set
> alongside the horizon-milestone test bump), rather than random failures
> each week. You'd still be able to track the failures as they come in,
> but they won't break your gate each time.

I don't want control, I want to consume a library in a standard way -
the way we consume libraries from the rest of openstack.

> I don't think where horizon (the library) lives would change how you
> version against it. We don't currently have any plans to separate the
> two; while we realise its a desirable change, weighing the work and risk
> involved in the split architecture vs the end result, we've chosen to
> work on other higher priority items for now (performance, filtering
> improvements, angular work etc.)

Well, if would stop us having a reference to git branch in our
requirements, and allow to use the standard global requirements
process to manage the dependency.

It also means that as a plugin I know that the released version of
my plugin has been tested in my gate with the exact version of the
code that the horizon team release.

We can still do multiple patches to fix any changes in the horizon
library, and in the tip of the chain update the version in requirements.

The risk has just been moved to the plugins, which is not ideal.

It also makes downstream consumption *a lot* easier.

>
>
> On 20 July 2016 at 13:38, Hayes, Graham  > wrote:
>
> On 20/07/2016 10:16, Rob Cresswell wrote:
> > Hey all,
> >
> > So we've had a few issues with plugin stability recently, and its
> > apparent that many plugins are building off Horizon master as a
> > dependency. I would really advise against this. A more manageable
> > development process may be to:
> >
> > - Base stable plugins against a stable release of Horizon
> > - Base WIP versions/new plugins off the last Horizon milestone, b2 in
> > this case, and then bump the version and capture issues within the same
> > patch. This should prevent random breakages, and should allow you to
> > just look at the release notes for that milestone.
>
> So this would mean changing tox.ini or requirements files after each
> horizon release?
>
> This dovetails nicely with the other thread about how we should be doing
> cross project interactions.
>
> What would be best, would be having horizon released as a separate
> library on pypi like the clients and oslo libs.
>
> Then openstack-dashboard, and all the plugins could rely on the same
> base library, without hacks like:
>
>deps =
>  -r{toxinidir}/requirements.txt
>  -r{toxinidir}/test-requirements.txt
>  http://tarballs.openstack.org/horizon/horizon-master.tar.gz
>
> in tox.ini or
>
># Testing Requirements
>
>  http://tarballs.openstack.org/horizon/horizon-master.tar.gz#egg=horizon
>
> in (test-)requirements.txt
>
> Is that on roadmap?
>
> > On the Horizon side, we've moved our FF a week earlier, so I think that
> > week combined with the usual RC period should be enough time to fix
> > bugs. We'll aim to make sure our release notes are complete with regards
> > to breaking issues for plugins, and deprecate appropriately.
> >
> > Rob
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> 
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)

[James Bottomley]

On Wed, 2016-07-20 at 16:08 +, Fox, Kevin M wrote:
> +1 to the finding of a middle ground.

Thanks ... I have actually been an enterprise architect (I just keep
very quiet about it when talking Open Source).

> The problem I've seen with your suggested OpenSource solution is the
> current social monetary system of OpenStack makes it extremely
> difficult.
> 
> Each project currently prints its own currency. Reviews. It takes
> quite a few Reviews (time/effort) on a project to gain enough
> currency that you get payed attention to. And, one project doesn't
> honor another projects currency.

OK, I accept your analogy, even though I would view currency as the
will to create and push patches.

The problem you describe: getting the recipients to listen and accept
your patches, is also a common one.  The first essential is simple
minimal patches because they're hard to reject.

Once you've overcome the reject barrier, there's the indifference one
(no-one says no, but no-one says yes).

Overcoming indifference involves partly knowing who to bother and when
(In openstack, it's quite easy since you know who the core reviewers
are) and also building a consensus for the patch; usually this involves
finding other people who need the feature and getting them to pipe up
(if you can't find other projects, then you can get potential users to
do this) even better if they help you write the patches.  Ideally, you
build your consensus before you actually push the patch set.  Sometimes
building consensus involves looking beyond your particular need to a
bigger one that would satisfy you but also pulls other people in.

> When these sorts of major cross project things need to be done
> though, you need to have folks with capital in many different
> projects and its very difficult to amass that much.
> 
> There is no OpenStack level currency (other then being elected as a
> TC member) that gets one project to stop and take the time to
> carefully consider what someone is saying when bringing up cross
> project issues.
> 
> Without some shared currency, then the problem becomes, the person
> invested in getting a solution, can propose and prototype and
> implement the feature all they want (often several times), but it
> never actually is accepted into the projects because a project:
>  a) doesn't take the time to really understand the problem, feeling
> its trivial and just not accepting any reviews
>  b) doesn't take the time to really understand the problem, so
> minimize it in their mind to a 'you should just use existing thing
> X...' or a heavy handily propose alternate solutions that really
> aren't solutions.
>  c) they decide its better handled by another project and stall/block
> reviews, trying to push the implementation to go elsewhere. When
> multiple projects decide this, the ball just keeps getting bounced
> around without any solution for years.
> 
> The status quo is not working here. The current governance structure
> doesn't support progress.

What you'll find you've described above is a process that doesn't
support drive by coders at all and, by extension therefore, doesn't
welcome newcomers, which is a big problem, but one I thought OpenStack
was tackling?

The bounce problem is annoying but not insuperable.  It mostly occurs
where there's overlap.  Often the best method for coping is to field
the bounce: produce the patch for the other project.  If it's smaller
and neater, perhaps the bounce was correct.  If it's bigger and uglier,
get the other project to say so and you now have a solid reason to go
back and say "we tried what you suggested and here's why it didn't
work".  Morally, you're now on higher ground because incorrect
technical advice is a personal failure for whoever bounced you (make
sure to capitalise on it if they try another bounce).

James


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] OpenStack Kolla - External Ceph

[Fox, Kevin M]

We use ceph with cinder and glance. I don't see a reason not to.

We do not set nova to use it for anything but cinder volumes though.

The reason being, if you set it up that way, your users have no way of opting 
out of the potential performance hit if using no local storage for non pets.

If you leave it nova local, you can always still get ceph remote storage for 
the whole vm by requesting the root disk be volume backed.

We also already have a ceph deployment managed without kolla, and that's 
unlikely to change.

So, for our site, our settings would probably be:
1. Deploy Ceph (enable_ceph="no")
2. Use Ceph with Glance (enable_ceph_glance="yes")
3. Use Ceph with Cinder (enable_ceph_cinder="yes")
4. Use Ceph with Nova (enable_ceph_nova="no")

Thanks,
Kevin

From: Jeffrey Zhang [zhang.lei@gmail.com]
Sent: Wednesday, July 20, 2016 9:51 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] OpenStack Kolla - External Ceph

On Wed, Jul 6, 2016 at 1:47 AM, Steven Dake (stdake)  wrote:
>
> At this point, I feel we are going a direction in which we try to wrap
> everything anybody could possibly want to configure with Kolla by making
> extensive use of global.yml. We would have to introduce flags indicating a
> couple of different scenarios:
>
> 1. Deploy Ceph (already there: enable_ceph="yes")
> 2. Use Ceph with Glance (enable_ceph_glance="yes")
> 3. Use Ceph with Cinder (enable_ceph_cinder="yes")
> 4. Use Ceph with Nova (enable_ceph_nova="yes")
>
>
> I disagree.  If ceph is enabled, then ceph should be used, if ceph is not
> enabled, then ceph shouldn't be used.  That implies all of OpenStack either
> uses Ceph or not.  So we really just need enable_ceph.


why we need separate configuration for ceph? I think if ceph is
enable, all the service should use ceph, too.

--
Regards,
Jeffrey Zhang
Blog: http://xcodest.me

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] Plugin stability, failing tests etc.

[Rob Cresswell]

Kirill: The failures are just as visible, since the cores still control merging 
anyway. The only difference is that if it takes a few days to fix something in 
upstream Horizon, you needn't block your own content in the meantime. Later in 
the release cycle (around N-3, for example) cores could just not merge failing 
tests. Regardless, this is just a recommendation, and if you're comfortable 
adjusting to issues as they come through, then thats fine to base off master.

Graham:  The "risk" I refer to is in that in any project architecture rewrite 
you can make mistakes and break functionality. So that risk only arises from a 
rewrite.

"It also means that as a plugin I know that the released version of my plugin 
has been tested in my gate with the exact version of the code that the horizon 
team release." - I don't understand this part. If we release a horizon lib, 
we'd still being doing milestone releases to PyPI. So again, whether you 
consume that as a tarball or a PyPI package makes little difference to the day 
to day testing of your plugin. Its the same code.

Ideally - yes, we'd probably split horizon as a separate library, but thats 
something to discuss at the summit and judge demand, because most discussions 
thus far have concluded that its a lower priority issue than others.

Rob



On 20 July 2016 at 17:36, Hayes, Graham 
mailto:graham.ha...@hpe.com>> wrote:
On 20/07/2016 15:13, Rob Cresswell wrote:
> Yes, it would mean changing your requirements after a release. So, for
> example you might run two gate tests:
>
> - A voting Horizon-stable/milestone test, (or both)
> - A non-voting Horizon-master test
>
> That gives you a lot of control over making your tests passing (multiple
> patches to make the Horizon-master test pass, or all in one patch set
> alongside the horizon-milestone test bump), rather than random failures
> each week. You'd still be able to track the failures as they come in,
> but they won't break your gate each time.

I don't want control, I want to consume a library in a standard way -
the way we consume libraries from the rest of openstack.

> I don't think where horizon (the library) lives would change how you
> version against it. We don't currently have any plans to separate the
> two; while we realise its a desirable change, weighing the work and risk
> involved in the split architecture vs the end result, we've chosen to
> work on other higher priority items for now (performance, filtering
> improvements, angular work etc.)

Well, if would stop us having a reference to git branch in our
requirements, and allow to use the standard global requirements
process to manage the dependency.

It also means that as a plugin I know that the released version of
my plugin has been tested in my gate with the exact version of the
code that the horizon team release.

We can still do multiple patches to fix any changes in the horizon
library, and in the tip of the chain update the version in requirements.

The risk has just been moved to the plugins, which is not ideal.

It also makes downstream consumption *a lot* easier.

>
>
> On 20 July 2016 at 13:38, Hayes, Graham 
> mailto:graham.ha...@hpe.com>
> >> wrote:
>
> On 20/07/2016 10:16, Rob Cresswell wrote:
> > Hey all,
> >
> > So we've had a few issues with plugin stability recently, and its
> > apparent that many plugins are building off Horizon master as a
> > dependency. I would really advise against this. A more manageable
> > development process may be to:
> >
> > - Base stable plugins against a stable release of Horizon
> > - Base WIP versions/new plugins off the last Horizon milestone, b2 in
> > this case, and then bump the version and capture issues within the same
> > patch. This should prevent random breakages, and should allow you to
> > just look at the release notes for that milestone.
>
> So this would mean changing tox.ini or requirements files after each
> horizon release?
>
> This dovetails nicely with the other thread about how we should be doing
> cross project interactions.
>
> What would be best, would be having horizon released as a separate
> library on pypi like the clients and oslo libs.
>
> Then openstack-dashboard, and all the plugins could rely on the same
> base library, without hacks like:
>
>deps =
>  -r{toxinidir}/requirements.txt
>  -r{toxinidir}/test-requirements.txt
>  http://tarballs.openstack.org/horizon/horizon-master.tar.gz
>
> in tox.ini or
>
># Testing Requirements
>
>  http://tarballs.openstack.org/horizon/horizon-master.tar.gz#egg=horizon
>
> in (test-)requirements.txt
>
> Is that on roadmap?
>
> > On the Horizon side, we've moved our FF a week earlier, so I think that
> > week combined with the usual RC period should be enough time to fix
> > bugs. We'll aim to make sure our release 

Re: [openstack-dev] [devstack] How to enable SSL in devStack?

[Rob Crittenden]

Rob Crittenden wrote:

Andrey Pavlov wrote:

Hi,

When I ran devstack with SSL I found a bug and tried to fix it -
https://review.openstack.org/#/c/242812/
But no one agree with me.
Try to apply this patch - it may help.
Also there is a chance that new bugs present in devstack that
prevented to install it with SSL.


Seeing how some other things in your local.conf might help but when I
tried to reproduce it I got the same error and it failed because Apache
didn't have an SSL listener on 443.

I'm not sure I'd recommend direct SSL in any case. I'd recommend the
tls-proxy service instead. Note that I'm pretty sure it has the same
problem: it hasn't been updated to handle port 443 for Keystone.

I'm working on switching from stud to mod_proxy if you want to take a
look and this problem is fixed there, https://review.openstack.org/301172

I'll see about adding a SSL listener to Keystone for the USE_SSL case in
the next few days.

And yeah, it's a moving target. I have an experimental gate test for
tlsproxy but it has to be requested explicitly. My plan is to enable it
as non-voting once the mod_proxy changes land so it will at least be
more obvious when things break (or maybe we can making it voting).


Fixing Keystone is easy. An Apache VirtualHost for 443 needs to be added.

But I found another, deeper problem: cinder won't listen on SSL. When 
they switched to using oslo_service for WSGI they completely removed the 
ability to use SSL. See bug https://bugs.launchpad.net/cinder/+bug/1590901


rob

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)

[Clint Byrum]

Excerpts from James Bottomley's message of 2016-07-20 08:31:34 -0700:
> So this is where the Open Source method takes over.  Change is produced
> by those people who most care about it because they're invested.  To
> take your Cinder example, you're unlikely to find them within Cinder
> because any project has inertia that resists change.  It takes the
> energy of the outside group X to force the change to Y, but this means
> that X often gets to propose, develop and even code Y.  Essentially
> they become drive by coders for Cinder.  This is where Open Source
> differs from Enterprise because you have the code and the access, you
> can do this.  However, you have to remember the inertia problem and
> build what you're trying to do as incrementally as possible: the larger
> the change, the bigger the resistance to it.  It's also a good test of
> the value of the change: if group X can't really be bothered to code it
> (and Cinder doesn't want it) then perhaps there's not really enough
> value in it anyway and it shouldn't happen.
> 

Thanks so much for stating this James. I couldn't agree more. A group
that can actually _accomplish_ change, and not just suggest it, is
exactly what we're working to start with the architecture WG. There are
plenty of people with the will to change, and I feel strongly that if
those people are given a structure and support, then they'll find the
time and space to complete these objectives.

I just want to make one nuance point about Cinder changes: the recent
DLM work, done outside any architecture working group, did actually come
from both inside and outside Cinder. The Cinder team realized something
was happening that would perhaps affect everyone, and raised it to the
cross-project level, which helped external individuals get involved. So
these initiatives can come from either direction. The key is that enough
momentum builds up to counter the natural inertia that you mentioned in
your message.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] Delorean fail blocks CI for stable branches

[Sagi Shnaidman]

How then it worked before? Can you show me the patch that broke this
functionality in delorean? It should be about 15 Jul when jobs started to
fail.
How then master branch works? It also runs on patched repo and succeeds.

I don't think we can use this workaround, each time this source file will
change - all our jobs will fail again? It's not even a workaround.
Please let's stop discussing and let's solve it finally, it blocks our CI
for stable patches.
I'd expect for a little bit more involvement in this issue and suggest you
or anybody who understand well delorean code and specs will try to solve it
when I provide him a whole TripleO CI dev environment with walking through
every CI step and any other info I can provide. Let's just sit and solve
it, otherwise we'll never get it working.

Thanks


On Wed, Jul 20, 2016 at 7:50 PM, Alan Pevec  wrote:

> > as a quickfix in tripleo.sh you could patch dlrn and set local=True in
>
> correction, patch local=False there while running dlrn command with
> --local to keep source checkout as-is
>



-- 
Best regards
Sagi Shnaidman
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] [RFC] ResourceProviderTags - Manage Capabilities with ResourceProvider

[Jay Pipes]

On 07/18/2016 01:45 PM, Matt Riedemann wrote:

On 7/15/2016 8:06 PM, Alex Xu wrote:


Actually I still think aggregates isn't good for Manage Caps, just as I
said in previous reply about Aggregates. One of reason is just same with
#2 you said :) And It's totally not managable. User is even no way to
query a specific host in which host-aggregate. And there isn't a
interface to query what metadata was related to the host by
host-aggregate. I prefer just keep the Aggregate as tool to group the
hosts. But yes, user still can use host-aggregate to manage host with
old way, let's user decide what is more convenient.


+1 to Alex's point. I just read through this thread and had the same
thought. If the point is to reduce complexity in the system and surface
capabilities to the end user, let's do that with resource provider tags,
not a mix of host aggregate metadata and resource provider tags so that
an operator has to set both, but also know in what situations he/she has
to set it and where.


Yeah, having the resource provider be tagged with capabilities versus 
having to manage aggregate tags may make some of the qualitative 
matching queries easier to grok. The query performance won't necessarily 
be any better, but they will likely be easier to read...



I'm hoping Jay or someone channeling Jay can hold my hand and walk me
safely through the evil forest that is image properties / flavor extra
specs / scheduler hints / host aggregates / resource providers / and the
plethora of scheduler filters that use them to build a concrete
picture/story tying this all together. I'm thinking like use cases, what
does the operator need to do


Are you asking how things are *currently* done in Nova? If so, I'll need 
to find some alcohol.


If you are asking about how we'd *like* all of the qualitative things to 
be requested and queried in the new placement API, then less alcohol is 
required.


The schema I'm thinking about on the placement engine side looks like this:

CREATE TABLE tags (
  id INT NOT NULL,
  name VARCHAR(200) NOT NULL,
  PRIMARY KEY (id),
  UNIQUE INDEX (name)
);

CREATE TABLE resource_provider_tags (
  resource_provider_id INT NOT NULL
  tag_id INT NOT NULL,
  PRIMARY KEY (resource_provider_id, tag_id),
  INDEX (tag_id)
);

On the Nova side, we need a mechanism of associating a set of 
capabilities that may either be required or preferred. The thing that we 
currently use for associating requested things in Nova is the flavor, so 
we'd need to define a mapping in Nova for the tags a flavor would 
require or prefer.


CREATE TABLE flavor_tags (
  flavor_id INT NOT NULL,
  tag_name VARCHAR(200) NOT NULL,
  is_required INT NOT NULL
);

We would need to have a call in the placement REST API to find the 
resource providers that matched a particular set of required or 
preferred capability tags. Such a call might look like the following:


GET /resource_providers
{
  "resources": {
"VCPU": 2,
"MEMORY_MB": 2048,
"DISK_GB": 100
  },
  "requires": [
"storage:ssd",
"compute:hw:x86:avx2",
  ],
  "prefers": [
"compute:virt:accelerated_whizzybang"
  ]
}

Disregard the quantitative side of the above request right now. We could 
answer the qualitative side of the equation with the following SQL query 
in the placement engine:


SELECT rp.uuid
FROM resource_providers AS rp, tags AS t1, tags AS t2, tags AS t3
INNER JOIN resource_provider_tags AS rpt1
ON rp.id = rpt1.resource_provider_id
AND rpt1.tag_id = t1.id
INNER JOIN resource_provider_tags AS rpt2
AND rpt1.resource_provider_id = rpt2.resource_provider_id
AND rpt2.tag_id = t2.id
LEFT JOIN resource_provider_tags AS rpt3
ON rp.id = rpt3.resource_provider_id
AND rpt3.tag_id = t3.id
GROUP BY rp.uuid
ORDER BY COUNT(COALESCE(rpt3.resource_provider_id, 0)) DESC
WHERE t1.name = 'storage:ssd'
AND t2.name = 'compute:hw:x86:avx2'
AND t3.name IN ('compute:virt:accelerated_whizzybang')

The above returns all resource providers having the 'storage:ssd' and 
'compute:hw:x86:avx2' tags and returns resource providers *first* that 
have the 'compute:virt:accelerated_whizzybang' tag.



, what does the end user of the cloud need
to do, etc. I think if we're going to do resource providers tags for
capabilities we also need to think about what we're replacing. Maybe
that's just host aggregate metadata, but what's the deprecation plan for
that?


Good question, as usual. My expectation would be that in Ocata, when we 
start adding the qualitative aspects to the placement REST API, we would 
introduce documentation that operators could use to translate common use 
cases that they were using flavor extra_specs and aggregate metadata for 
in the pre-placement world to the resource provider tags setup that 
would replace that functonality in the placement API world. Unlike most 
of the quantitative side of things, there really isn't a good way to 
"autoheal" or "autosetup" these things.



There is a ton to talk about here, so I'll leave that for the midcycle.
But le

Re: [openstack-dev] [Nova] [RFC] ResourceProviderTags - Manage Capabilities with ResourceProvider

[Jay Pipes]

On 07/13/2016 01:37 PM, Ed Leafe wrote:

On Jul 11, 2016, at 6:08 AM, Alex Xu  wrote:


For example, the capabilities can be defined as:

COMPUTE_HW_CAP_CPU_AVX
COMPUTE_HW_CAP_CPU_SSE

COMPUTE_HV_CAP_LIVE_MIGRATION
COMPUTE_HV_CAP_LIVE_SNAPSHOT


( The COMPUTE means this is coming from Nova. HW means this is hardware related 
Capabilities. HV means this is
 capabilities of Hypervisor. But the catalog of Capabilities can be discussed 
separated. This propose focus on the
 ResourceTags. We also have another idea about not using 'PREFIX' to manage the 
Tags. We can add attributes to the
 Tags. Then we have more control on the Tags. This will describe separately in 
the bottom. )


I was ready to start ranting about using horribly mangled names to represent 
data, and then saw your comment about attributes for tags. Yes, a thousand 
times yes to attributes! There can be several standards, such as ‘compute’ or 
‘networking’ that we use for some basic cross-cloud compatibility, but making 
them flexible is a must for adoption.


I disagree :) Adoption -- at least interoperable cloud adoption -- of 
this functionality will likely be hindered by super-flexible description 
of capabilities. I think having a set of "standard" capabilities that 
can be counted on to be cross-OpenStack-cloud compatible and a set of 
"dynamic" capabilities that are custom to a deployment would be a good 
thing to do.


Best,
-jay


I can update the qualitative request spec to add ResourceProviderTags as a 
possible implementation.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)

[Fox, Kevin M]

I wish it was so simple. Its not.

There is a good coding practice:
"The code is done, not when there is nothing more to add, but nothing more to 
remove"

Some of the more mature projects are in this mode of thinking now. (which is 
mostly good, really). They don't want to add features unless they see it as a 
benefit to their users. This is the problem, there is a disconnect between the 
view of Project X's users, and greater view of OpenStack Users.

Even accepting the smallest of patches to work towards the goal is unacceptable 
to projects if they believe it is not a helpful feature to their perceived user 
base, or helpful enough to them to justify adding more code to their project. 
Or the feeling that "just push it to a new project or a different one is 
better". This fractured view of OpenStack users at the project level is 
preventing progress on OpenStack as a whole.

Only an overarching Architectural group with some power to define what a user 
is, or the TC can address those sorts of issues.

Thanks,
Kevin

From: James Bottomley [james.bottom...@hansenpartnership.com]
Sent: Wednesday, July 20, 2016 9:57 AM
To: OpenStack Development Mailing List (not for usage questions); Clint Byrum
Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)

On Wed, 2016-07-20 at 16:08 +, Fox, Kevin M wrote:
> +1 to the finding of a middle ground.

Thanks ... I have actually been an enterprise architect (I just keep
very quiet about it when talking Open Source).

> The problem I've seen with your suggested OpenSource solution is the
> current social monetary system of OpenStack makes it extremely
> difficult.
>
> Each project currently prints its own currency. Reviews. It takes
> quite a few Reviews (time/effort) on a project to gain enough
> currency that you get payed attention to. And, one project doesn't
> honor another projects currency.

OK, I accept your analogy, even though I would view currency as the
will to create and push patches.

The problem you describe: getting the recipients to listen and accept
your patches, is also a common one.  The first essential is simple
minimal patches because they're hard to reject.

Once you've overcome the reject barrier, there's the indifference one
(no-one says no, but no-one says yes).

Overcoming indifference involves partly knowing who to bother and when
(In openstack, it's quite easy since you know who the core reviewers
are) and also building a consensus for the patch; usually this involves
finding other people who need the feature and getting them to pipe up
(if you can't find other projects, then you can get potential users to
do this) even better if they help you write the patches.  Ideally, you
build your consensus before you actually push the patch set.  Sometimes
building consensus involves looking beyond your particular need to a
bigger one that would satisfy you but also pulls other people in.

> When these sorts of major cross project things need to be done
> though, you need to have folks with capital in many different
> projects and its very difficult to amass that much.
>
> There is no OpenStack level currency (other then being elected as a
> TC member) that gets one project to stop and take the time to
> carefully consider what someone is saying when bringing up cross
> project issues.
>
> Without some shared currency, then the problem becomes, the person
> invested in getting a solution, can propose and prototype and
> implement the feature all they want (often several times), but it
> never actually is accepted into the projects because a project:
>  a) doesn't take the time to really understand the problem, feeling
> its trivial and just not accepting any reviews
>  b) doesn't take the time to really understand the problem, so
> minimize it in their mind to a 'you should just use existing thing
> X...' or a heavy handily propose alternate solutions that really
> aren't solutions.
>  c) they decide its better handled by another project and stall/block
> reviews, trying to push the implementation to go elsewhere. When
> multiple projects decide this, the ball just keeps getting bounced
> around without any solution for years.
>
> The status quo is not working here. The current governance structure
> doesn't support progress.

What you'll find you've described above is a process that doesn't
support drive by coders at all and, by extension therefore, doesn't
welcome newcomers, which is a big problem, but one I thought OpenStack
was tackling?

The bounce problem is annoying but not insuperable.  It mostly occurs
where there's overlap.  Often the best method for coping is to field
the bounce: produce the patch for the other project.  If it's smaller
and neater, perhaps the bounce was correct.  If it's bigger and uglier,
get the other project to say so and you now have a solid reason to go
back and say "we tried what you suggested and here's why it didn't
work".  Morally, you're now on h

Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)

[Duncan Thomas]

On 20 July 2016 at 19:57, James Bottomley <
james.bottom...@hansenpartnership.com> wrote:

>
> OK, I accept your analogy, even though I would view currency as the
> will to create and push patches.
>
> The problem you describe: getting the recipients to listen and accept
> your patches, is also a common one.  The first essential is simple
> minimal patches because they're hard to reject.
>
> Once you've overcome the reject barrier, there's the indifference one
> (no-one says no, but no-one says yes).
>
> [snip]

The trouble with drive-by architecture patches (or large feature patches of
any kind) is that it is often better *not* to merge them if you don't think
the contributor is  going to stick around for a while. This changes are
usually intrusive, and have repercussions that take time to discover. It's
often difficult to keep a change clean when the original author isn't
around to review the follow-on work.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [charms] Project mascot

[Billy Olsen]

I like the idea of the Kraken...

though I think I like the giant squid over an octopus, but either one
is in the same vein :-)

On Mon, Jul 18, 2016 at 1:27 AM, James Page  wrote:
> Hi All
>
> As an approved project, we need to provide some ideas for a project mascot
> for the Charms project (see [0]).
>
> Some suggestions as discussed on IRC:
>
> 1) cobra ('[snake] charming openstack') - which aligns with the Juju logo a
> little.
> 2) kraken ('many armed animal managing openstack') - but I think that falls
> into mythical creatures so its probably excluded so maybe octopus instead?
>
> It would be nice to have one or two more ideas - any suggestions?
>
> Cheers
>
> James
>
> [0] http://www.openstack.org/project-mascots
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] [RFC] ResourceProviderTags - Manage Capabilities with ResourceProvider

[Mooney, Sean K]

> -Original Message-
> From: Jay Pipes [mailto:jaypi...@gmail.com]
> Sent: Wednesday, July 20, 2016 7:16 PM
> To: openstack-dev@lists.openstack.org
> Subject: Re: [openstack-dev] [Nova] [RFC] ResourceProviderTags - Manage
> Capabilities with ResourceProvider
> 
> On 07/13/2016 01:37 PM, Ed Leafe wrote:
> > On Jul 11, 2016, at 6:08 AM, Alex Xu  wrote:
> >
> >> For example, the capabilities can be defined as:
> >>
> >> COMPUTE_HW_CAP_CPU_AVX
> >> COMPUTE_HW_CAP_CPU_SSE
> >> 
> >> COMPUTE_HV_CAP_LIVE_MIGRATION
> >> COMPUTE_HV_CAP_LIVE_SNAPSHOT
> >> 
> >>
> >> ( The COMPUTE means this is coming from Nova. HW means this is
> >> hardware related Capabilities. HV means this is  capabilities of
> >> Hypervisor. But the catalog of Capabilities can be discussed
> >> separated. This propose focus on the  ResourceTags. We also have
> >> another idea about not using 'PREFIX' to manage the Tags. We can add
> >> attributes to the  Tags. Then we have more control on the Tags. This
> >> will describe separately in the bottom. )
> >
> > I was ready to start ranting about using horribly mangled names to
> represent data, and then saw your comment about attributes for tags.
> Yes, a thousand times yes to attributes! There can be several
> standards, such as ‘compute’ or ‘networking’ that we use for some basic
> cross-cloud compatibility, but making them flexible is a must for
> adoption.
> 
> I disagree :) Adoption -- at least interoperable cloud adoption -- of
> this functionality will likely be hindered by super-flexible
> description of capabilities. I think having a set of "standard"
> capabilities that can be counted on to be cross-OpenStack-cloud
> compatible and a set of "dynamic" capabilities that are custom to a
> deployment would be a good thing to do.

[Mooney, Sean K] 
I know there is a bad memories when I metion CIM 
(http://www.dmtf.org/standards/cim)
for many on the nova team but if we are to use standard names we should probably
actually assess are there existing standads that we could adopt instead of 
defining
our own standard names in nova for the resources. 
For example 
http://schemas.dmtf.org/wbem/cim-html/2/CIM_ProcessorAllocationSettingData.html
Define the name for different instcution set extentions for example avx is 
DMTF:x86:AVX.
Some work has been done in glance to allow importing cim metadata from ovf 
files also
https://specs.openstack.org/openstack/glance-specs/specs/mitaka/implemented/cim-namespace-metadata-definitions.html

while I don’t think using the full cim information model is useful in this case 
using the name would be
from an inter-operability point of view as we not only would have standard 
names in openstack but those names
would conform to an existing standard.

We could still allow custom attribute but is see value in standardizing what 
can be standardized.


> 
> Best,
> -jay
> 
> > I can update the qualitative request spec to add ResourceProviderTags
> as a possible implementation.
> 
> ___
> ___
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-
> requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [trove] no weekly meeting next week

[Amrith Kumar]

We won't have our weekly meeting next week as we are going to be meeting anyway 
for our virtual mid-cycle.

Thanks,

-amrith

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Mascot/logo for your project

[Amrith Kumar]

Heidi,

The Trove team came up with several logo options. The result of a vote was 
decisive:


1.  Stingray [1]

2.  A Clam with a Pearl in it. [2]

I have heard of no one else wanting to use the Stingray so I would like to 
request that the foundation consider that our preference.

Among the options suggested was “Salmonella”. One person (nameless) explained 
it to me as “A small female fish”. The community will be happy to know that in 
the voting, Salmonella received zero votes. If any other team would like to 
take the idea of salmonella after reading this email, I’d request that they 
please credit the Trove team for this suggestion ☺ We promise that we won’t use 
that logo and there will be no conflict. Here’s a picture of “salmonella” [3].

Other options which did not garner enough support included a whale, a koala 
bear, and a sea dragon.

Thanks,

-amrith

[1] http://cdn.xl.thumbs.canstockphoto.com/canstock26887885.jpg
[2] 
http://image.shutterstock.com/z/stock-photo-an-open-sea-shell-with-a-pearl-inside-52231675.jpg
[3] https://c2.staticflickr.com/8/7032/6400406229_f99d5e268e_b.jpg




From: Heidi Joy Tretheway [mailto:heidi...@openstack.org]
Sent: Monday, July 11, 2016 11:00 AM
To: openstack-dev@lists.openstack.org
Subject: [openstack-dev] Mascot/logo for your project

The Foundation would like to help promote OpenStack projects in the big tent 
with branding and marketing services. The idea is to create a family of logos 
for OpenStack projects that are unique, yet immediately identifiable as part of 
OpenStack. We’ll be using these logos to promote your project on the OpenStack 
website, at the Summit and in marketing materials.


We’re asking project teams to choose a mascot to represent as their logo. Your 
team can select your own mascot, and then we’ll have an illustrator create the 
logo for you (and we also plan to print some special collateral for your team 
in Barcelona).


If your team already has a logo based on a mascot from nature, you’ll have 
first priority to keep that mascot, and the illustrator will restyle it to be 
consistent with the other projects. If you have a logo that doesn’t have a 
mascot from nature, we encourage your team to choose a mascot.


Here’s an FAQ and examples of what the logos can look like: 
http://www.openstack.org/project-mascots
We’ve also recorded a quick video with an overview of the project: 
https://youtu.be/LOdsuNr2T-o


You can get in touch with your PTL to participate in the logo choice 
discussion. If you have more questions, I’m happy to help. :-)


Cheers,
Heidi Joy

__
Heidi Joy Tretheway
Senior Marketing Manager, OpenStack Foundation
503 816 9769  |  skype: heidi.tretheway




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)

[Joshua Harlow]

Duncan Thomas wrote:



On 20 July 2016 at 19:57, James Bottomley
mailto:james.bottom...@hansenpartnership.com>> wrote:


OK, I accept your analogy, even though I would view currency as the
will to create and push patches.

The problem you describe: getting the recipients to listen and accept
your patches, is also a common one.  The first essential is simple
minimal patches because they're hard to reject.

Once you've overcome the reject barrier, there's the indifference one
(no-one says no, but no-one says yes).

[snip]

The trouble with drive-by architecture patches (or large feature patches
of any kind) is that it is often better *not* to merge them if you don't
think the contributor is  going to stick around for a while. This
changes are usually intrusive, and have repercussions that take time to
discover. It's often difficult to keep a change clean when the original
author isn't around to review the follow-on work.


Agreed, and knowing where yahoo and HP(e) are at right now (with regards 
to openstack and ...) these kind of things are a little more prevalent 
(with regards to quota, tasks...) now-a-days (for better or worse); not 
how I want it to be but it's reality.


Which I guess is why I'd be nice to have cross-project architecture 
'standardization' (? for lack of better word) with a more long term 
strategic vision (vs localized tactical visions). Such things are 
obviously not hard to get going (and are equally hard to sustain).




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)

[James Bottomley]

On Wed, 2016-07-20 at 21:24 +0300, Duncan Thomas wrote:
> On 20 July 2016 at 19:57, James Bottomley <
> james.bottom...@hansenpartnership.com> wrote:
> 
> > 
> > OK, I accept your analogy, even though I would view currency as the
> > will to create and push patches.
> > 
> > The problem you describe: getting the recipients to listen and
> > accept
> > your patches, is also a common one.  The first essential is simple
> > minimal patches because they're hard to reject.
> > 
> > Once you've overcome the reject barrier, there's the indifference
> > one
> > (no-one says no, but no-one says yes).
> > 
> > [snip]
> 
> The trouble with drive-by architecture patches (or large feature 
> patches of any kind)

OK, there's an assumption here: that the patch is large.

>  is that it is often better *not* to merge them if you don't think
> the contributor is  going to stick around for a while. This changes
> are usually intrusive, and have repercussions that take time to
> discover. It's often difficult to keep a change clean when the
> original author isn't around to review the follow-on work.

I understand (and do agree with) the Maintenance problem.  However, if
you're trying to change architecture, and you do it by small patches,
it looks easily reversible and is a lot easier to gain acceptance for. 
 The key is to think small not big (the latter being the temptation for
architecture) even if the end result will eventually be big.

Let me give you an example from ancient history.  The biggest
architectural change I pushed into Linux was switching from bus
specific to generic device APIs.  I did it at the time because I had a
SCSI device in the PA-RISC architecture that could sit on a variety of
internal busses and I didn't want a huge driver with five sets of
partially duplicated code for five different busses.

Here's the first patch (both dated 21 Dec 2002)

generic device DMA API

27 files changed, 750 insertions(+), 155 deletions(-)

And the second

allow pci primary busses to have parents in the device model

2 files changed, 14 insertions(+), 5 deletions(-)

The first is large because I'm actually introducing yet another
parallel DMA API (50% of it is documentation).  It looks OK because all
our other DMA APIs were bus specific.

The second is the key change because it allows cascades of bus
hierarchies of different types.

These two patches are easy to gain acceptance for because they're just
introducing new APIs and not altering existing stuff.  They're small
and easily revertible if I happened to disappear (or they proved not to
work out in practice).  They were, however, enough to allow me to
convert the PA-RISC architecture to the new model and write the driver
I wanted for the 53c700 chip.

The end point we're at today, where practically every device API in
Linux uses generic devices, is no longer revertible.  If it were done
as one patch, it would touch about 80% of all the files in Linux and be
larger than the kernel itself.  However, I didn't do that: I pushed the
minimum viable patch set that would support what I wanted to do (plus
the use cases for a couple of other interested parties I picked up
along the way).

James


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)

[James Bottomley]

On Wed, 2016-07-20 at 18:18 +, Fox, Kevin M wrote:
> I wish it was so simple. Its not.
> 
> There is a good coding practice:
> "The code is done, not when there is nothing more to add, but nothing
> more to remove"
> 
> Some of the more mature projects are in this mode of thinking now.
> (which is mostly good, really). They don't want to add features
> unless they see it as a benefit to their users. This is the problem,
> there is a disconnect between the view of Project X's users, and
> greater view of OpenStack Users.
> 
> Even accepting the smallest of patches to work towards the goal is
> unacceptable to projects if they believe it is not a helpful feature
> to their perceived user base, or helpful enough to them to justify
> adding more code to their project. Or the feeling that "just push it
> to a new project or a different one is better". This fractured view
> of OpenStack users at the project level is preventing progress on
> OpenStack as a whole.
> 
> Only an overarching Architectural group with some power to define
> what a user is, or the TC can address those sorts of issues.

I'll concede this requirement if you can point out to me who this group
is for the Linux Kernel.  If you're tempted to say "Linus", it's most
certainly not: while he does care about some architectural decisions,
he emphatically avoids most of them, which leaves the subsystem
maintainers (some equivalence to openstack projects/PTLs) doing this on
a case by case basis.

James

> Thanks,
> Kevin
> 
> From: James Bottomley [james.bottom...@hansenpartnership.com]
> Sent: Wednesday, July 20, 2016 9:57 AM
> To: OpenStack Development Mailing List (not for usage questions);
> Clint Byrum
> Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins
> for all)
> 
> On Wed, 2016-07-20 at 16:08 +, Fox, Kevin M wrote:
> > +1 to the finding of a middle ground.
> 
> Thanks ... I have actually been an enterprise architect (I just keep
> very quiet about it when talking Open Source).
> 
> > The problem I've seen with your suggested OpenSource solution is
> > the
> > current social monetary system of OpenStack makes it extremely
> > difficult.
> > 
> > Each project currently prints its own currency. Reviews. It takes
> > quite a few Reviews (time/effort) on a project to gain enough
> > currency that you get payed attention to. And, one project doesn't
> > honor another projects currency.
> 
> OK, I accept your analogy, even though I would view currency as the
> will to create and push patches.
> 
> The problem you describe: getting the recipients to listen and accept
> your patches, is also a common one.  The first essential is simple
> minimal patches because they're hard to reject.
> 
> Once you've overcome the reject barrier, there's the indifference one
> (no-one says no, but no-one says yes).
> 
> Overcoming indifference involves partly knowing who to bother and
> when
> (In openstack, it's quite easy since you know who the core reviewers
> are) and also building a consensus for the patch; usually this
> involves
> finding other people who need the feature and getting them to pipe up
> (if you can't find other projects, then you can get potential users
> to
> do this) even better if they help you write the patches.  Ideally,
> you
> build your consensus before you actually push the patch set. 
>  Sometimes
> building consensus involves looking beyond your particular need to a
> bigger one that would satisfy you but also pulls other people in.
> 
> > When these sorts of major cross project things need to be done
> > though, you need to have folks with capital in many different
> > projects and its very difficult to amass that much.
> > 
> > There is no OpenStack level currency (other then being elected as a
> > TC member) that gets one project to stop and take the time to
> > carefully consider what someone is saying when bringing up cross
> > project issues.
> > 
> > Without some shared currency, then the problem becomes, the person
> > invested in getting a solution, can propose and prototype and
> > implement the feature all they want (often several times), but it
> > never actually is accepted into the projects because a project:
> >  a) doesn't take the time to really understand the problem, feeling
> > its trivial and just not accepting any reviews
> >  b) doesn't take the time to really understand the problem, so
> > minimize it in their mind to a 'you should just use existing thing
> > X...' or a heavy handily propose alternate solutions that really
> > aren't solutions.
> >  c) they decide its better handled by another project and
> > stall/block
> > reviews, trying to push the implementation to go elsewhere. When
> > multiple projects decide this, the ball just keeps getting bounced
> > around without any solution for years.
> > 
> > The status quo is not working here. The current governance
> > structure
> > doesn't support progress.
> 
> What you'll find you've descri

Re: [openstack-dev] Mascot/logo for your project

[Antoine Cabot]

Hi Heidi,

The Watcher team cames up with several mascot options, here is the
list ordered by preference :
1. jelly fish
2. eagle
3. bee
4. hammer head shark

Thanks,

Antoine

On Wed, Jul 20, 2016 at 9:01 PM, Amrith Kumar  wrote:
> Heidi,
>
>
>
> The Trove team came up with several logo options. The result of a vote was
> decisive:
>
>
>
> 1.  Stingray [1]
>
> 2.  A Clam with a Pearl in it. [2]
>
>
>
> I have heard of no one else wanting to use the Stingray so I would like to
> request that the foundation consider that our preference.
>
>
>
> Among the options suggested was “Salmonella”. One person (nameless)
> explained it to me as “A small female fish”. The community will be happy to
> know that in the voting, Salmonella received zero votes. If any other team
> would like to take the idea of salmonella after reading this email, I’d
> request that they please credit the Trove team for this suggestion J We
> promise that we won’t use that logo and there will be no conflict. Here’s a
> picture of “salmonella” [3].
>
>
>
> Other options which did not garner enough support included a whale, a koala
> bear, and a sea dragon.
>
>
>
> Thanks,
>
>
>
> -amrith
>
>
>
> [1] http://cdn.xl.thumbs.canstockphoto.com/canstock26887885.jpg
>
> [2]
> http://image.shutterstock.com/z/stock-photo-an-open-sea-shell-with-a-pearl-inside-52231675.jpg
>
> [3] https://c2.staticflickr.com/8/7032/6400406229_f99d5e268e_b.jpg
>
>
>
>
>
>
>
>
>
> From: Heidi Joy Tretheway [mailto:heidi...@openstack.org]
> Sent: Monday, July 11, 2016 11:00 AM
> To: openstack-dev@lists.openstack.org
> Subject: [openstack-dev] Mascot/logo for your project
>
>
>
> The Foundation would like to help promote OpenStack projects in the big tent
> with branding and marketing services. The idea is to create a family of
> logos for OpenStack projects that are unique, yet immediately identifiable
> as part of OpenStack. We’ll be using these logos to promote your project on
> the OpenStack website, at the Summit and in marketing materials.
>
>
>
> We’re asking project teams to choose a mascot to represent as their logo.
> Your team can select your own mascot, and then we’ll have an illustrator
> create the logo for you (and we also plan to print some special collateral
> for your team in Barcelona).
>
>
>
> If your team already has a logo based on a mascot from nature, you’ll have
> first priority to keep that mascot, and the illustrator will restyle it to
> be consistent with the other projects. If you have a logo that doesn’t have
> a mascot from nature, we encourage your team to choose a mascot.
>
>
>
> Here’s an FAQ and examples of what the logos can look like:
> http://www.openstack.org/project-mascots
>
> We’ve also recorded a quick video with an overview of the project:
> https://youtu.be/LOdsuNr2T-o
>
>
>
> You can get in touch with your PTL to participate in the logo choice
> discussion. If you have more questions, I’m happy to help. :-)
>
>
>
> Cheers,
>
> Heidi Joy
>
>
>
> __
>
> Heidi Joy Tretheway
>
> Senior Marketing Manager, OpenStack Foundation
>
> 503 816 9769  |  skype: heidi.tretheway
>
>
>
>
>
>
>
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [tacker] No weekly meeting next week - July 26th, 2016

[Sridhar Ramaswamy]

As we are hosting our newton midcycle meetup next week [1], I'm
cancelling our regular weekly irc meeting. Please continue to reach
out to the team in #tacker IRC to flag any issues / reviews.

- Sridhar

[1] https://etherpad.openstack.org/p/tacker-newton-midcycle

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)

[Fox, Kevin M]

I would argue Linus, yes. As he's constantly stepping up when a subsistem tries 
and breaks something for a user or creates a user facing mess and says, no, 
subsystem, no. breaking userspace is unacceptable, or, we're not adding support 
for an api we have to support forever thats very poorly designed. Yes, he 
defers a lot to subsystem maintainers, as they have generally gotten the 
message of paying close attention to that kind of thing over time, and he 
hasn't had to speak up so much anymore. The rest really is best left up to the 
subsystems. But someone has to keep an eye on the big picture. The users of the 
whole thing. Users care about the linux kernel as a whole, and less so about 
any given subsystem.

Thanks,
Kevin

From: James Bottomley [james.bottom...@hansenpartnership.com]
Sent: Wednesday, July 20, 2016 12:42 PM
To: OpenStack Development Mailing List (not for usage questions); Clint Byrum
Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)

On Wed, 2016-07-20 at 18:18 +, Fox, Kevin M wrote:
> I wish it was so simple. Its not.
>
> There is a good coding practice:
> "The code is done, not when there is nothing more to add, but nothing
> more to remove"
>
> Some of the more mature projects are in this mode of thinking now.
> (which is mostly good, really). They don't want to add features
> unless they see it as a benefit to their users. This is the problem,
> there is a disconnect between the view of Project X's users, and
> greater view of OpenStack Users.
>
> Even accepting the smallest of patches to work towards the goal is
> unacceptable to projects if they believe it is not a helpful feature
> to their perceived user base, or helpful enough to them to justify
> adding more code to their project. Or the feeling that "just push it
> to a new project or a different one is better". This fractured view
> of OpenStack users at the project level is preventing progress on
> OpenStack as a whole.
>
> Only an overarching Architectural group with some power to define
> what a user is, or the TC can address those sorts of issues.

I'll concede this requirement if you can point out to me who this group
is for the Linux Kernel.  If you're tempted to say "Linus", it's most
certainly not: while he does care about some architectural decisions,
he emphatically avoids most of them, which leaves the subsystem
maintainers (some equivalence to openstack projects/PTLs) doing this on
a case by case basis.

James

> Thanks,
> Kevin
> 
> From: James Bottomley [james.bottom...@hansenpartnership.com]
> Sent: Wednesday, July 20, 2016 9:57 AM
> To: OpenStack Development Mailing List (not for usage questions);
> Clint Byrum
> Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins
> for all)
>
> On Wed, 2016-07-20 at 16:08 +, Fox, Kevin M wrote:
> > +1 to the finding of a middle ground.
>
> Thanks ... I have actually been an enterprise architect (I just keep
> very quiet about it when talking Open Source).
>
> > The problem I've seen with your suggested OpenSource solution is
> > the
> > current social monetary system of OpenStack makes it extremely
> > difficult.
> >
> > Each project currently prints its own currency. Reviews. It takes
> > quite a few Reviews (time/effort) on a project to gain enough
> > currency that you get payed attention to. And, one project doesn't
> > honor another projects currency.
>
> OK, I accept your analogy, even though I would view currency as the
> will to create and push patches.
>
> The problem you describe: getting the recipients to listen and accept
> your patches, is also a common one.  The first essential is simple
> minimal patches because they're hard to reject.
>
> Once you've overcome the reject barrier, there's the indifference one
> (no-one says no, but no-one says yes).
>
> Overcoming indifference involves partly knowing who to bother and
> when
> (In openstack, it's quite easy since you know who the core reviewers
> are) and also building a consensus for the patch; usually this
> involves
> finding other people who need the feature and getting them to pipe up
> (if you can't find other projects, then you can get potential users
> to
> do this) even better if they help you write the patches.  Ideally,
> you
> build your consensus before you actually push the patch set.
>  Sometimes
> building consensus involves looking beyond your particular need to a
> bigger one that would satisfy you but also pulls other people in.
>
> > When these sorts of major cross project things need to be done
> > though, you need to have folks with capital in many different
> > projects and its very difficult to amass that much.
> >
> > There is no OpenStack level currency (other then being elected as a
> > TC member) that gets one project to stop and take the time to
> > carefully consider what someone is saying when bringing up cross
> > project issues.
> >
> > Without s

Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)

[Fox, Kevin M]

And maybe this raises an interesting defininition mismatch in the conversation.

There is archetectural stuff like, do we support 7 different web frameworks, or 
do we standardize on flask... python vs go.

Theres also the architectural stuff at the, what interactive surface do you 
expose to users/operators. How consistant is it. Does it have all the features, 
no matter where they are implemented to do work.

They are somewhat related at the op level when debugging is involved.

But Im much more concerned with the latter archetecture getting attention then 
the former.

Thanks,
Kevin


From: James Bottomley
Sent: Wednesday, July 20, 2016 12:42:27 PM
To: OpenStack Development Mailing List (not for usage questions); Clint Byrum
Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)

On Wed, 2016-07-20 at 18:18 +, Fox, Kevin M wrote:
> I wish it was so simple. Its not.
>
> There is a good coding practice:
> "The code is done, not when there is nothing more to add, but nothing
> more to remove"
>
> Some of the more mature projects are in this mode of thinking now.
> (which is mostly good, really). They don't want to add features
> unless they see it as a benefit to their users. This is the problem,
> there is a disconnect between the view of Project X's users, and
> greater view of OpenStack Users.
>
> Even accepting the smallest of patches to work towards the goal is
> unacceptable to projects if they believe it is not a helpful feature
> to their perceived user base, or helpful enough to them to justify
> adding more code to their project. Or the feeling that "just push it
> to a new project or a different one is better". This fractured view
> of OpenStack users at the project level is preventing progress on
> OpenStack as a whole.
>
> Only an overarching Architectural group with some power to define
> what a user is, or the TC can address those sorts of issues.

I'll concede this requirement if you can point out to me who this group
is for the Linux Kernel.  If you're tempted to say "Linus", it's most
certainly not: while he does care about some architectural decisions,
he emphatically avoids most of them, which leaves the subsystem
maintainers (some equivalence to openstack projects/PTLs) doing this on
a case by case basis.

James

> Thanks,
> Kevin
> 
> From: James Bottomley [james.bottom...@hansenpartnership.com]
> Sent: Wednesday, July 20, 2016 9:57 AM
> To: OpenStack Development Mailing List (not for usage questions);
> Clint Byrum
> Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins
> for all)
>
> On Wed, 2016-07-20 at 16:08 +, Fox, Kevin M wrote:
> > +1 to the finding of a middle ground.
>
> Thanks ... I have actually been an enterprise architect (I just keep
> very quiet about it when talking Open Source).
>
> > The problem I've seen with your suggested OpenSource solution is
> > the
> > current social monetary system of OpenStack makes it extremely
> > difficult.
> >
> > Each project currently prints its own currency. Reviews. It takes
> > quite a few Reviews (time/effort) on a project to gain enough
> > currency that you get payed attention to. And, one project doesn't
> > honor another projects currency.
>
> OK, I accept your analogy, even though I would view currency as the
> will to create and push patches.
>
> The problem you describe: getting the recipients to listen and accept
> your patches, is also a common one.  The first essential is simple
> minimal patches because they're hard to reject.
>
> Once you've overcome the reject barrier, there's the indifference one
> (no-one says no, but no-one says yes).
>
> Overcoming indifference involves partly knowing who to bother and
> when
> (In openstack, it's quite easy since you know who the core reviewers
> are) and also building a consensus for the patch; usually this
> involves
> finding other people who need the feature and getting them to pipe up
> (if you can't find other projects, then you can get potential users
> to
> do this) even better if they help you write the patches.  Ideally,
> you
> build your consensus before you actually push the patch set.
>  Sometimes
> building consensus involves looking beyond your particular need to a
> bigger one that would satisfy you but also pulls other people in.
>
> > When these sorts of major cross project things need to be done
> > though, you need to have folks with capital in many different
> > projects and its very difficult to amass that much.
> >
> > There is no OpenStack level currency (other then being elected as a
> > TC member) that gets one project to stop and take the time to
> > carefully consider what someone is saying when bringing up cross
> > project issues.
> >
> > Without some shared currency, then the problem becomes, the person
> > invested in getting a solution, can propose and prototype and
> > implement the feature all they want (often several times), bu

Re: [openstack-dev] [nova][glance] snapshots are broken by default with newton and glance v2

[Matt Riedemann]

On 7/18/2016 4:41 AM, Erno Kuvaja wrote:

On Mon, Jul 18, 2016 at 5:19 AM, Nikhil Komawar  wrote:

Thanks Matt. I've scheduled for a release of the client this week.

On 7/16/16 4:09 AM, Matt Riedemann wrote:

This is more of a heads up than anything.

Our internal CI is running Tempest with images that don't have
kernel_id or ramdisk_id properties set.

We're running from master so nova defaults to use_glance_v1=False.

Because of this:

https://github.com/openstack/nova/blob/47358449d359a287d21426b4e1f18479a4d1fd36/nova/compute/api.py#L867-L868


and this:

https://github.com/openstack/nova/blob/47358449d359a287d21426b4e1f18479a4d1fd36/nova/image/glance.py#L835


The snapshot image properties get kernel_id and ramdisk_id set to None
since that's what the glance v2 schema requires.

However, python-glanceclient has it's own outdated copy of the schema
which doesn't allow null values for those properties, see bug:

https://bugs.launchpad.net/python-glanceclient/+bug/1596602

We don't hit this in the community CI because the image that Tempest
uses from devstack has the kernel_id and ramdisk_id properties set:

http://logs.openstack.org/52/335152/1/check/gate-tempest-dsvm-neutron-src-python-glanceclient/d393db9/logs/devstacklog.txt.gz#_2016-06-28_18_40_12_429


But for anyone else upgrading to Newton that has images without those
properties set and doesn't have use_glance_v1=True in nova.conf is
going to be broken.

Since we really want to get people off glance v1 and move to
deprecation in Ocata, we need to get this merged and released:

https://review.openstack.org/#/c/335152/

And bump the minimum required python-glanceclient in
global-requirements for Newton.

I'm not really sure why python-glanceclient even has it's own copy of
the image schema, that seems redundant and error prone given the
glance API already validates that, but it's kind of beside the point
right now.



--

Thanks,
Nikhil


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Hi Matt,

Something has broken on the way if that shipped schema actually affect
your operation. That should be used _only_ when the client is called
without connection to the Glance API (for example in a case you have
no env set and you call `glance help`), this is btw the reason why we
have that schema shipped with the client.

As soon as you execute a call that actually interacts with Glance API
we should be pulling the schema from there. So if this is not the
case, we've broken somewhere on the way. I'll try to find time to have
a look.

- Erno

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



Yeah I guess the schema fix in glanceclient didn't resolve the issue, 
but I haven't had the time to dig into the bug since we're doing the 
midcycle this week.


--

Thanks,

Matt Riedemann


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [devstack] libvirt/qemu source install plugin.

[Mooney, Sean K]

Hi
I recently had the need to test a feature (vhost-user reconnect) that was 
commit to the
qemu source tree a few weeks ago. As there has been no release since then I 
needed
to build from source so to that end I wrote a small devstack plugin to do just 
that.

I was thinking of opening a review to create a new repo to host the plugin under
The openstack namespace (openstack/devstack-plugin-libvirt-qemu) but before
I do I wanted to ask if others are interested In a devstack plugin that just 
compiles
and installs qemu and Libvirt?

Regards
Sean.





__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [kolla] repo split

[Ryan Hallisey]

Hello.

The repo split discussion that started at summit was brought up again at the 
midcycle.
The discussion was focused around splitting the Docker containers and Ansible 
code into
two separate repos [1].

One of the main opponents to the split is backports.  Backports will need to be 
done
by hand for a few releases.  So far, there hasn't been a ton of backports, but 
that could
always change.

As for splitting, it provides a much clearer view of what pieces of the project 
are where.
Kolla-ansible with its own repo will sit along side kolla-kubernetes as 
consumers of the
kolla repo.

The target for the split will be for day 1 of Occata. The core team will vote on
the change of splitting kolla into kolla-ansible and kolla.

Cores please respond with a +1/-1 to approve or disapprove the repo split. Any 
community
member feel free to weigh in with your opinion.

+1
-Ryan

[1] - https://etherpad.openstack.org/p/kolla-N-midcycle-repo-split

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Mascot/logo for your project

[Heidi Joy Tretheway]

Hi Julien, 

The logos are offered on a per-project basis, based on the Yaml file that 
listed 57 projects approved by the TC. Thanks for asking me to clarify!

Cheers,
Heidi Joy

__
Heidi Joy Tretheway
Senior Marketing Manager, OpenStack Foundation
503 816 9769  |  skype: heidi.tretheway





> On Jul 15, 2016, at 8:15 AM, Julien Danjou  wrote:
> 
> On Mon, Jul 11 2016, Heidi Joy Tretheway wrote:
> 
>> The Foundation would like to help promote OpenStack projects in the big tent
>> with branding and marketing services. The idea is to create a family of logos
>> for OpenStack projects that are unique, yet immediately identifiable as part 
>> of
>> OpenStack. We’ll be using these logos to promote your project on the 
>> OpenStack
>> website, at the Summit and in marketing materials.
> 
> Something is very unclear (but I'm getting used to it lol): are theses
> logo created for projects or teams?
> 
> We have 4 different projects (Aodh, Gnocchi, Ceilometer & Panko) in the
> Telemetry team. Should we have one or 4 mascots?
> 
> -- 
> Julien Danjou
> // Free Software hacker
> // https://julien.danjou.info

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [fuel-octane] Nominate Sergey Abramov to fuel-octane core

[Ilya Kharin]

Hello,

I would like to nominate Sergey Abramov to fuel-octane core due to his
significant contribution to the project [1] and [2].

Best regards,
Ilya Kharin.

[1] http://stackalytics.com/report/contribution/fuel-octane/90
[2]
http://stackalytics.com/?release=all&module=fuel-octane&metric=marks&user_id=sabramov
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] Freezer - tmp_file issue

[Straub, Albert]

Hi!
 
I’m rather new at this so let me know if this is the wrong place to ask for a 
change.  I was trying to run the Freezer code without the trickle executable 
but with a configuration file.  It wouldn’t run.  So, I took a look and it 
appears that if you do not have a trickle executable but you do have a config 
file that it will add a tmp_file key to the backup_args dictionary.  However, 
if you don’t have a trickle executable, it will try to pop out tmp_file which 
doesn’t exist and thus an exception is thrown and the program exits.  Would it 
be possible to have someone move the if backup_args.config: \ 
backup_args.__dict__[‘tmp_file’] = conf_file.name above and on the same indent 
as the if trickle_executable and then move the part in the else statement if 
backup_args.config to the same indent level as well?  That should prevent it 
from having a happy heart attack and exiting.
 
Thanks,
 
Al
 
 
if trickle_executable:
LOG.info("Info: Starting trickle ...")
trickle_command = '{0} -d {1} -u {2} '.\
format(trickle_executable,
   getattr(backup_args, 'download_limit') or -1,
   getattr(backup_args, 'upload_limit') or -1)
backup_args.__dict__['trickle_command'] = trickle_command
if backup_args.config:
backup_args.__dict__['tmp_file'] = conf_file.name
 
# maintain env variable not to get into infinite loop
if "tricklecount" in os.environ:
tricklecount = int(os.environ.get("tricklecount", 1))
tricklecount += 1
os.environ["tricklecount"] = str(tricklecount)
 
else:
os.environ["tricklecount"] = str(1)
else:
LOG.warn("Trickle not found. Switching to normal mode without "
 "limiting bandwidth")
if backup_args.config:
# remove index tmp_file from backup arguments dict
backup_args.__dict__.pop('tmp_file')
utils.delete_file(conf_file.name)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] App Catalog IRC meeting Thursday July 21st

[Christopher Aedo]

Join us Thursday for our weekly meeting, scheduled for July 21st at
17:00UTC in #openstack-meeting-3

The agenda can be found here, and please add to if you want to discuss
something with the Community App Catalog team:
https://wiki.openstack.org/wiki/Meetings/app-catalog

Tomorrow we will be talking more about our plan to implement GLARE as
a back-end for the Community App Catalog, and what we'll need to merge
in the next few weeks to make this a reality.  We will hopefully agree
on a mascot for the Community App Catalog as well.

Hope to see you there tomorrow!

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Mascot/logo for your project

[Zane Bitter]

On 19/07/16 12:26, gordon chung wrote:



On 19/07/16 03:50 AM, Julien Danjou wrote:

Heidi, could you confirm that it is a mascot per team or per project?

Honestly, I don't think it'll suit us to have only one mascot/logo. It's
gonna be hard to use a logo marked "Telemetry" as a branding for our 4
different projects that do different things – even if it's in the same
area.

hybrid animals. you can breed anything nowadays and build your own
Franken-mascot. :P


https://en.wikipedia.org/wiki/Wolpertinger


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Mascot/logo for your project

[Fox, Kevin M]

https://en.wikipedia.org/wiki/Platypus#/media/File:Wild_Platypus_4.jpg :)

Thanks,
Kevin

From: Zane Bitter [zbit...@redhat.com]
Sent: Wednesday, July 20, 2016 2:32 PM
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] Mascot/logo for your project

On 19/07/16 12:26, gordon chung wrote:
>
>
> On 19/07/16 03:50 AM, Julien Danjou wrote:
>> Heidi, could you confirm that it is a mascot per team or per project?
>>
>> Honestly, I don't think it'll suit us to have only one mascot/logo. It's
>> gonna be hard to use a logo marked "Telemetry" as a branding for our 4
>> different projects that do different things – even if it's in the same
>> area.
> hybrid animals. you can breed anything nowadays and build your own
> Franken-mascot. :P

https://en.wikipedia.org/wiki/Wolpertinger


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)

[James Bottomley]

On Wed, 2016-07-20 at 20:01 +, Fox, Kevin M wrote:
> I would argue Linus, yes. As he's constantly stepping up when a
> subsistem tries and breaks something for a user or creates a user
> facing mess and says, no, subsystem, no. breaking userspace is
> unacceptable, or, we're not adding support for an api we have to
> support forever thats very poorly designed.

Those are all vetos.  He doesn't compel one subsystem to accept the
patches of another, for instance.

>  Yes, he defers a lot to subsystem maintainers, as they have
> generally gotten the message of paying close attention to that kind
> of thing over time, and he hasn't had to speak up so much anymore.
> The rest really is best left up to the subsystems. But someone has to
> keep an eye on the big picture. The users of the whole thing. Users
> care about the linux kernel as a whole, and less so about any given
> subsystem.

He says "don't build this" (veto) he doesn't say "do build that"
(compulsion).  The problem I've heard a lot of people describing on
this thread is the latter: difficulty of getting one group to pay
attention to the needs of another.  Your "overarching Architectural
group with some power to define what a user is" is something like this.

The only power in Linux is the power to say "no".  The only way an
individual or a group builds acceptance for their own patches is on
their own.  Architectural decisions in this model are driven locally
not globally.

James


> Thanks,
> Kevin
> 
> From: James Bottomley [james.bottom...@hansenpartnership.com]
> Sent: Wednesday, July 20, 2016 12:42 PM
> To: OpenStack Development Mailing List (not for usage questions);
> Clint Byrum
> Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins
> for all)
> 
> On Wed, 2016-07-20 at 18:18 +, Fox, Kevin M wrote:
> > I wish it was so simple. Its not.
> > 
> > There is a good coding practice:
> > "The code is done, not when there is nothing more to add, but
> > nothing
> > more to remove"
> > 
> > Some of the more mature projects are in this mode of thinking now.
> > (which is mostly good, really). They don't want to add features
> > unless they see it as a benefit to their users. This is the
> > problem,
> > there is a disconnect between the view of Project X's users, and
> > greater view of OpenStack Users.
> > 
> > Even accepting the smallest of patches to work towards the goal is
> > unacceptable to projects if they believe it is not a helpful
> > feature
> > to their perceived user base, or helpful enough to them to justify
> > adding more code to their project. Or the feeling that "just push
> > it
> > to a new project or a different one is better". This fractured view
> > of OpenStack users at the project level is preventing progress on
> > OpenStack as a whole.
> > 
> > Only an overarching Architectural group with some power to define
> > what a user is, or the TC can address those sorts of issues.
> 
> I'll concede this requirement if you can point out to me who this
> group
> is for the Linux Kernel.  If you're tempted to say "Linus", it's most
> certainly not: while he does care about some architectural decisions,
> he emphatically avoids most of them, which leaves the subsystem
> maintainers (some equivalence to openstack projects/PTLs) doing this
> on
> a case by case basis.
> 
> James
> 
> > Thanks,
> > Kevin
> > 
> > From: James Bottomley [james.bottom...@hansenpartnership.com]
> > Sent: Wednesday, July 20, 2016 9:57 AM
> > To: OpenStack Development Mailing List (not for usage questions);
> > Clint Byrum
> > Subject: Re: [openstack-dev] [tc][all] Big tent? (Related to
> > Plugins
> > for all)
> > 
> > On Wed, 2016-07-20 at 16:08 +, Fox, Kevin M wrote:
> > > +1 to the finding of a middle ground.
> > 
> > Thanks ... I have actually been an enterprise architect (I just
> > keep
> > very quiet about it when talking Open Source).
> > 
> > > The problem I've seen with your suggested OpenSource solution is
> > > the
> > > current social monetary system of OpenStack makes it extremely
> > > difficult.
> > > 
> > > Each project currently prints its own currency. Reviews. It takes
> > > quite a few Reviews (time/effort) on a project to gain enough
> > > currency that you get payed attention to. And, one project
> > > doesn't
> > > honor another projects currency.
> > 
> > OK, I accept your analogy, even though I would view currency as the
> > will to create and push patches.
> > 
> > The problem you describe: getting the recipients to listen and
> > accept
> > your patches, is also a common one.  The first essential is simple
> > minimal patches because they're hard to reject.
> > 
> > Once you've overcome the reject barrier, there's the indifference
> > one
> > (no-one says no, but no-one says yes).
> > 
> > Overcoming indifference involves partly knowing who to bother and
> > when
> > (In openstack, it's quite easy since you know who the core
> 

Re: [openstack-dev] [Horizon] Angular action services and initScope

[Thai Q Tran]

You all sort of touched on what I am about to reiterate:
 
1. The scope was intended to be used as a way to propagate events up and down the chain (from table controller to step controller) since there was no real way to share information between controllers (remember that a modal dialog launches an entirely different fragment of HTML with no relationship to the current controller). Furthermore, we needed to use initScope to explicitly set the scope so that $modal does not use the $rootScope (which it does by default if scope is not set). As a side bonus, we were also able to use events to share data between the steps. The idea behind was to make steps independent and reusable in multiple workflows. The reality is that the event propagation required both an $emit and a $broadcast to actually share data between steps, this makes it much less desirable.
 
The root of the problem is, how does my table know to update once the form is submitted? Since then, a lot more work has been done in this area and I have not followed all of them (Matt and Tyr, feel free to chime in). I understand that Tyr made it so that events were no longer required, instead we can depend on promises via the result-handler function. This function gets triggered once the action is completed and would allow the table controller to do something after the action goes through. With the introduction of the generic table and registry, I am no longer certain how the data is updated in tables. If tables are notified of changes without the use of events, then the first step is to make this the standard.
 
2. Going with Richards idea of a shared workflow.model idea.
 
"So, here's my rough thought: workflow.model is an object with properties named for each of the workflow steps - using the step formName as the name (hell, schema form could probably make this a doddle). The workflow model is passed to the controller for each step, which uses its own named model to store the data captured by the step - and as a side effect it can poke at (and watch) the data captured by other steps, which is often useful. Workflow $modal resolution supplies the workflow model for the consumer of the workflow to then to something with all that data."
 
Event if we get rid of the events model completely, how are we planning on watching objects without scope? The main reason initScope existed is to act as a glue to allow sharing of data between the various controllers. This is the direct result of us using the $modal widget. While I agree that this introduces a sticky situation concerning the purity of services, I am unsure how we can resolve this.
 
I believe that the workflow should just be a list of steps. The model should instead reside in the create-volume service where it truly belongs. Each step can store a reference to the model and we are still able to poke/watch the data as Richard suggested.
 
To me there are two questions we should answer before we jump and do another rewrite.
1. How are we going to glue the various controllers together if we don't use scope.
2. How is data between the steps going to be shared.
 
I agree the solution we have isnt perfect, but at some point, we have to ask, is it good enough? And if it is, is it worth the cost of a rewrite when it can potentially break plugins and external code? To be clear, I am not against it, just adding my 2c.
 
- Original message -From: Richard Jones To: "OpenStack Development Mailing List (not for usage questions)" Cc:Subject: Re: [openstack-dev] [Horizon] Angular action services and initScopeDate: Mon, Jul 18, 2016 3:05 PM 
On 18 July 2016 at 11:49, Rob Cresswell  wrote:

Maybe I'm missing the point, but the schema stuff just defines a model and passes it to the schema directive via ctrl; doesn't that remove any issues with workflows? ui-bootstraps modal and modalinstance already provides a way to pass that data back to the initial location (table etc) again
 
In a nutshell, yes this is the goal I'm aiming for, though some of the details might vary.
 
 
      Richard 
__OpenStack Development Mailing List (not for usage questions)Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] repo split

[Michał Jastrzębski]

+1 I was reluctant to repo split by Newton timeframe as it was crucial
time to our project, people will start using it, and I didn't want to
cause needless confusion. Now time is right imho.

On 20 July 2016 at 15:48, Ryan Hallisey  wrote:
> Hello.
>
> The repo split discussion that started at summit was brought up again at the 
> midcycle.
> The discussion was focused around splitting the Docker containers and Ansible 
> code into
> two separate repos [1].
>
> One of the main opponents to the split is backports.  Backports will need to 
> be done
> by hand for a few releases.  So far, there hasn't been a ton of backports, 
> but that could
> always change.
>
> As for splitting, it provides a much clearer view of what pieces of the 
> project are where.
> Kolla-ansible with its own repo will sit along side kolla-kubernetes as 
> consumers of the
> kolla repo.
>
> The target for the split will be for day 1 of Occata. The core team will vote 
> on
> the change of splitting kolla into kolla-ansible and kolla.
>
> Cores please respond with a +1/-1 to approve or disapprove the repo split. 
> Any community
> member feel free to weigh in with your opinion.
>
> +1
> -Ryan
>
> [1] - https://etherpad.openstack.org/p/kolla-N-midcycle-repo-split
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][all] Big tent? (Related to Plugins for all)

[Clint Byrum]

Excerpts from Fox, Kevin M's message of 2016-07-20 20:12:48 +:
> And maybe this raises an interesting defininition mismatch in the 
> conversation.
> 
> There is archetectural stuff like, do we support 7 different web frameworks, 
> or do we standardize on flask... python vs go.
> 

Yeah meh, that's developer centric implementation details and I think
not very interesting. To me the architectural questions are deeper. "How
do we do locking?", "How should we enable inter-process and inter-host
communication?", "How do we handle distributed transactions?" and "What
concurrency model should we use?".

> Theres also the architectural stuff at the, what interactive surface do you 
> expose to users/operators. How consistant is it. Does it have all the 
> features, no matter where they are implemented to do work.

I believe this is the goal of the API-WG. But again, they're not there
to compel, they're there to advise, assist, and work. Ultimately, if an
API is created and is poor, like Linus, the community can definitely say
"No" and refuse to use that API.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Congress] Congress horizon plugin - congressclient/congress API auth issue - help

[Aimee Ukasick]

Hi all,

I've been working on Policy UI (Horizon): Unable to get policies
list (devstack) (https://bugs.launchpad.net/congress/+bug/1602837)
for the past 3 days. Anusha is correct - it's an authentication
problem, but I have not been able to fix it.

I grabbed the relevant code in congress.py from Anusha's horizon
plugin model patchset (https://review.openstack.org/#/c/305063/3) and
added try/catch blocks, logging statements (with error because I
haven't figured out how to set the horizon log level).


I am testing the code on devstack, which I cloned on 19 July 2016.

With both v2 and v3 auth, congressclient.v1.client is created.
The failure happens trying to call
congressclient.v1.client.Client.list_policies().
When using v2 auth, the error message is "Unable to get policies list:
The resource could not be found"
When using v3 auth, the error message is "Cannot authorize API client"

I am assuming that congressclient.v1.client.Client is
https://github.com/openstack/python-congressclient/blob/master/congressclient/v1/client.py
and that client.list_policy() calls list_policy()in the python-congressclient
which in turn calls the Congress API. Is this correct?

Any ideas why with v3 auth, the python-congressclient cannot authorize the
call to the API?

I looked at other horizon plugin models (ceilometer, neutron, nova,
cerberus, cloudkitty, trove, designate, manila) to see how they created
the client. While the code to create a client is not identical,
it is vastly different from the code to create a client
in contrib/horizon/congress.py.

Thanks in advance for any pointers.

aimee

Aimee Ukasick (aimeeu)

v2 log:
2016-07-20 22:13:56.501455
2016-07-20 22:14:30.238233 * view.get_data calling policies =
congress.policies_list(self.request) *
2016-07-20 22:14:30.238318 * self.request.path= /dashboard/admin/policies/
2016-07-20 22:14:30.238352 * congress.policies_list(request) BEGIN*
2016-07-20 22:14:30.238376 * calling client = congressclient(request)*
2016-07-20 22:14:30.238399 * congress.congressclient BEGIN*
2016-07-20 22:14:30.238454 * auth_url= http://192.168.56.103/identity
2016-07-20 22:14:30.238479 * calling get_keystone_session *
2016-07-20 22:14:30.238505 * congress.get_keystone_session BEGIN
auth_url *http://192.168.56.103/identity
2016-07-20 22:14:30.238554 * path= /identity
2016-07-20 22:14:30.238578 * using V2 plugin to authenticate*
2016-07-20 22:14:30.238630 * v2 auth.get_auth_state=
2016-07-20 22:14:30.238656 None
2016-07-20 22:14:30.238677 * finished using V2 plugin to authenticate*
2016-07-20 22:14:30.238698 * creating session with auth *
2016-07-20 22:14:30.244407 * congress.get_keystone_session END*
2016-07-20 22:14:30.244462 * regtion_name= RegionOne
2016-07-20 22:14:30.244491 * calling congress_client.Client(**kwargs)
2016-07-20 22:14:30.247830 * congress.congressclient END*
2016-07-20 22:14:30.247902 * calling policies_list =
client.list_policy()*
2016-07-20 22:14:30.248012 DEBUG:keystoneauth.identity.v2:Making
authentication request to http://192.168.56.103/identity/tokens
2016-07-20 22:14:30.255023 DEBUG:keystoneauth.session:Request returned
failure status: 404
2016-07-20 22:14:30.257546 Unable to get policies list: The resource
could not be found.


v3 log:
2016-07-20 22:09:22.912969
2016-07-20 22:09:31.907119 * view.get_data calling policies =
congress.policies_list(self.request) *
2016-07-20 22:09:31.907973 * self.request.path= /dashboard/admin/policies/
2016-07-20 22:09:31.908122 * congress.policies_list(request) BEGIN*
2016-07-20 22:09:31.908250 * calling client = congressclient(request)*
2016-07-20 22:09:31.908386 * congress.congressclient BEGIN*
2016-07-20 22:09:31.909034 * auth_url= http://192.168.56.103/identity
2016-07-20 22:09:31.909217 * calling get_keystone_session *
2016-07-20 22:09:31.909356 * congress.get_keystone_session BEGIN
auth_url *http://192.168.56.103/identity
2016-07-20 22:09:31.909527 * path= /identity
2016-07-20 22:09:31.909795 * using V3 plugin to authenticate*
2016-07-20 22:09:31.910042 auth_url=http://192.168.56.103/identity
2016-07-20 22:09:31.910175 token=d46339f2d0b5455db54909d6ed95a9cc
2016-07-20 22:09:31.910301 project_name=alt_demo
2016-07-20 22:09:31.910426 domain_name=Default
2016-07-20 22:09:31.910676 project_domain_name=default
2016-07-20 22:09:31.910866 * v3 auth.get_auth_state=
2016-07-20 22:09:31.910992 None
2016-07-20 22:09:31.914053 * finished using V3 plugin to authenticate*
2016-07-20 22:09:31.914100 * creating session with auth *
2016-07-20 22:09:31.922260 * congress.get_keystone_session END*
2016-07-20 22:09:31.922542 * regtion_name= RegionOne
2016-07-20 22:09:31.922676 * calling congress_client.Client(**kwargs)
2016-07-20 22:09:31.922822 * congress.congressclient END*
2016-07-20 22:09:31.922949 * calling 

Re: [openstack-dev] [cinder] [nova] os-brick privsep failures and an upgrade strategy?

[Sean Dague]

On 07/12/2016 06:25 AM, Matt Riedemann wrote:


We probably aren't doing anything while Sean Dague is on vacation. He's
back next week and we have the nova/cinder meetups, so I'm planning on
talking about the grenade issue in person and hopefully we'll have a
plan by the end of next week to move forward.


After some discussions at the Nova midcycle we threw together an 
approach where we just always allow privsep-helper from oslo.rootwrap.


https://review.openstack.org/344450

We did a sniff test of this, and it worked to roll over the upgrade 
boundary, without an etc change, and work with osbrick 1.4.0 (currently 
blacklisted because of the upgrade issue). While I realize it wasn't the 
favorite approach by many it works. It's 3 lines of functional change. 
If we land this, release, and bump the minimum, we've got the upgrade 
issue solved in this cycle.


Please take a look and see if we can agree to this path forward.

-Sean

--
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Horizon] On testing...

[Richard Jones]

Hi folks,

Sorry I didn't make the meeting this morning :-(

We touched on testing at the mid-cycle and again it came up in the meeting
this morning. We identified two issues:

1. Our integration test suite cannot grow to too many more tests because
they take too long to run (and get auto-killed). We could increase the
amount of time allowed for them, but we agreed that we shouldn't do this.
2. We don't have enough higher-level (integration leve) test coverage for
our newer angular interfaces.

These two are obviously at odds :-)

What we agreed on, I think, is that we're going to:

1. Reduce the granularity of the integration tests - use them to broadly
test whether an interface works at all when presented to a browser,
2. Replace many single interface tests with a fewer tests that poke at many
interfaces (thus removing the significant per-test overhead) - even
potentially having a single test that attempts to access every panel (as a
guard against gross Javascript incompatibilities or configuration issues
breaking panels),
3. Move the integration tests to tempest, and
4. Write some "django unit test suite" functional tests for our angular
code that tests more than just the single units we currently test (ie. mock
at a more remote level, checking that broader interactions work).

Does this sound about right?


 Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] repo split

[Britt Houser (bhouser)]

-0 (My vote doesn't count).  We had endless problems keeping containers and 
orchestration in sync when we had two repos.  I am really impressed that Mitaka 
ansible can orchestrate Liberty containers.  That really speaks volumes.  And  
I understand there is a stable ABI now, but surely at some point in the future 
the ABI will need some unforeseen change?  Will we have ABI v1 and v2?  Seems 
like that will bring headaches, at least based on my past experience.


Thx,
britt

On 7/20/16, 6:32 PM, "Michał Jastrzębski"  wrote:

>+1 I was reluctant to repo split by Newton timeframe as it was crucial
>time to our project, people will start using it, and I didn't want to
>cause needless confusion. Now time is right imho.
>
>On 20 July 2016 at 15:48, Ryan Hallisey  wrote:
>> Hello.
>>
>> The repo split discussion that started at summit was brought up again at the 
>> midcycle.
>> The discussion was focused around splitting the Docker containers and 
>> Ansible code into
>> two separate repos [1].
>>
>> One of the main opponents to the split is backports.  Backports will need to 
>> be done
>> by hand for a few releases.  So far, there hasn't been a ton of backports, 
>> but that could
>> always change.
>>
>> As for splitting, it provides a much clearer view of what pieces of the 
>> project are where.
>> Kolla-ansible with its own repo will sit along side kolla-kubernetes as 
>> consumers of the
>> kolla repo.
>>
>> The target for the split will be for day 1 of Occata. The core team will 
>> vote on
>> the change of splitting kolla into kolla-ansible and kolla.
>>
>> Cores please respond with a +1/-1 to approve or disapprove the repo split. 
>> Any community
>> member feel free to weigh in with your opinion.
>>
>> +1
>> -Ryan
>>
>> [1] - https://etherpad.openstack.org/p/kolla-N-midcycle-repo-split
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>__
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][all] Plugins for all

[Sean Dague]

On 07/18/2016 06:49 AM, Dmitry Tantsur wrote:

On 07/17/2016 11:04 PM, Jay Pipes wrote:

On 07/14/2016 12:21 PM, Hayes, Graham wrote:

A lot of the effects are hard to see, and are not insurmountable, but
do cause projects to re-invent the wheel.

For example, quotas - there is no way for a project that is not nova,
neutron, cinder to hook into the standard CLI, or UI for setting
quotas.


There *is* no standard CLI or UI for setting quotas.


They can be done as either extra commands
(openstack dns quota set --foo bar) or as custom panels, but not
the way other quotas get set.


This has nothing to do with the big tent and everything to do with the
fact that the community at large has conflated quotas -- i.e. the limit
of a particular class of resource that a user or tenant can consume --
with the usage of a particular class of resource. The two things are not
the same nor do they need to be handled by the same service, IMHO.

I've proposed before that quotas -- i.e. the *limits* for different
resource classes that a consumer of those resources has -- be handled by
the Keystone API. This is the endpoint that I personally think makes the
most sense to house this information.

Usage information is necessarily the domain of the individual service
projects who must control allocation/consumption of resources under
their control. It would be *helpful* to have a set of best-practice
guidelines for projects to follow in safely accounting for consumption
of resources, but "the big tent" has nothing to do with our community
failing to do this. We've failed to do this from the beginning of
OpenStack, well before the big tent was just a spark in the minds of the
TC.


Tempest plugins are another example. Approximately 30 of the 36
current plugins are using resources that are not supposed to be
used, and are an unstable interface.


What "resources" are you referring to above? What is the "unstable
interface" you are referring to? Tempest should only ever be validating
public REST APIs.


Projects in tree in tempest
are at a much better position, as any change to the internal
API will have to be fixed before the gate merges, but other
out of tree plugins are in a place where they can be broken at any
point.


An example here would be super-useful, since as mentioned above, Tempest
should only be validating public HTTP/REST APIs.


Not entirely related example, but still in support of the original point
(IMO): grenade currently does not catch smoke tests coming from tempest
plugins when running after upgrade. It's just one missing call [1], and
it probably would not go unnoticed if Nova tests did not run ;)

[1] https://review.openstack.org/337372


The patch is merged.

Also... missing tests have gone missing for long times on all kinds of 
projects, including nova / neutron. Missing tests require people keeping 
an eye on things, because they don't fail when they disappear.


-Sean

--
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][glance] snapshots are broken by default with newton and glance v2

[Nikhil Komawar]

Thanks Matt. The bug looked very descriptive so, I commented my thoughts
there https://bugs.launchpad.net/python-glanceclient/+bug/1596602

On 7/20/16 4:18 PM, Matt Riedemann wrote:
> On 7/18/2016 4:41 AM, Erno Kuvaja wrote:
>> On Mon, Jul 18, 2016 at 5:19 AM, Nikhil Komawar
>>  wrote:
>>> Thanks Matt. I've scheduled for a release of the client this week.
>>>
>>> On 7/16/16 4:09 AM, Matt Riedemann wrote:
 This is more of a heads up than anything.

 Our internal CI is running Tempest with images that don't have
 kernel_id or ramdisk_id properties set.

 We're running from master so nova defaults to use_glance_v1=False.

 Because of this:

 https://github.com/openstack/nova/blob/47358449d359a287d21426b4e1f18479a4d1fd36/nova/compute/api.py#L867-L868



 and this:

 https://github.com/openstack/nova/blob/47358449d359a287d21426b4e1f18479a4d1fd36/nova/image/glance.py#L835



 The snapshot image properties get kernel_id and ramdisk_id set to None
 since that's what the glance v2 schema requires.

 However, python-glanceclient has it's own outdated copy of the schema
 which doesn't allow null values for those properties, see bug:

 https://bugs.launchpad.net/python-glanceclient/+bug/1596602

 We don't hit this in the community CI because the image that Tempest
 uses from devstack has the kernel_id and ramdisk_id properties set:

 http://logs.openstack.org/52/335152/1/check/gate-tempest-dsvm-neutron-src-python-glanceclient/d393db9/logs/devstacklog.txt.gz#_2016-06-28_18_40_12_429



 But for anyone else upgrading to Newton that has images without those
 properties set and doesn't have use_glance_v1=True in nova.conf is
 going to be broken.

 Since we really want to get people off glance v1 and move to
 deprecation in Ocata, we need to get this merged and released:

 https://review.openstack.org/#/c/335152/

 And bump the minimum required python-glanceclient in
 global-requirements for Newton.

 I'm not really sure why python-glanceclient even has it's own copy of
 the image schema, that seems redundant and error prone given the
 glance API already validates that, but it's kind of beside the point
 right now.

>>>
>>> -- 
>>>
>>> Thanks,
>>> Nikhil
>>>
>>>
>>> __
>>>
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> Hi Matt,
>>
>> Something has broken on the way if that shipped schema actually affect
>> your operation. That should be used _only_ when the client is called
>> without connection to the Glance API (for example in a case you have
>> no env set and you call `glance help`), this is btw the reason why we
>> have that schema shipped with the client.
>>
>> As soon as you execute a call that actually interacts with Glance API
>> we should be pulling the schema from there. So if this is not the
>> case, we've broken somewhere on the way. I'll try to find time to have
>> a look.
>>
>> - Erno
>>
>> __
>>
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
> Yeah I guess the schema fix in glanceclient didn't resolve the issue,
> but I haven't had the time to dig into the bug since we're doing the
> midcycle this week.
>

-- 

Thanks,
Nikhil


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] Delorean fail blocks CI for stable branches

[Alan Pevec]

On Wed, Jul 20, 2016 at 7:49 PM, Sagi Shnaidman  wrote:
> How then it worked before? Can you show me the patch that broke this
> functionality in delorean? It should be about 15 Jul when jobs started to
> fail.

commented in lp

> How then master branch works? It also runs on patched repo and succeeds.

I explained that but looks like we're talking past each other.

> I don't think we can use this workaround, each time this source file will
> change - all our jobs will fail again? It's not even a workaround.
> Please let's stop discussing and let's solve it finally, it blocks our CI
> for stable patches.

Sure, I've assigned https://bugs.launchpad.net/tripleo/+bug/1604039 to
myself and proposed a patch.

Alan

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] On testing...

[Timur Sufiev]

I'm not sure if (3) was the thing we agreed upon (and if it will help us to
detect issues in Horizon) but the rest of options definitely make sense to
me.

Also there is

5. Parallelize integration tests.

On Wed, Jul 20, 2016 at 4:28 PM Richard Jones 
wrote:

> Hi folks,
>
> Sorry I didn't make the meeting this morning :-(
>
> We touched on testing at the mid-cycle and again it came up in the meeting
> this morning. We identified two issues:
>
> 1. Our integration test suite cannot grow to too many more tests because
> they take too long to run (and get auto-killed). We could increase the
> amount of time allowed for them, but we agreed that we shouldn't do this.
> 2. We don't have enough higher-level (integration leve) test coverage for
> our newer angular interfaces.
>
> These two are obviously at odds :-)
>
> What we agreed on, I think, is that we're going to:
>
> 1. Reduce the granularity of the integration tests - use them to broadly
> test whether an interface works at all when presented to a browser,
> 2. Replace many single interface tests with a fewer tests that poke at
> many interfaces (thus removing the significant per-test overhead) - even
> potentially having a single test that attempts to access every panel (as a
> guard against gross Javascript incompatibilities or configuration issues
> breaking panels),
> 3. Move the integration tests to tempest, and
> 4. Write some "django unit test suite" functional tests for our angular
> code that tests more than just the single units we currently test (ie. mock
> at a more remote level, checking that broader interactions work).
>
> Does this sound about right?
>
>
>  Richard
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] FPGA as a dynamic nested resources

[Fei K Chen]

Roman Dobosz  wrote on 2016/07/20 02:03:28:

> From: Roman Dobosz 
> To: openstack-dev 
> Date: 2016/07/20 02:07
> Subject: [openstack-dev] FPGA as a dynamic nested resources
>
> Hi all,
>
> Some time ago Jay Pipes published etherpad[1] with ideas around
> modelling nested resources, taking NUMA as an example. I was also
> encouraged ;) to start this thread, on last Nova scheduler meeting.
>
> I was read mentioned etherpad and what hits me was that described
> scenario with NUMA cells resembles the way how FPGA can be managed. In
> some extent.
>
> NUMA cell can be treated as a vessel for memory cells, and it is
> expressed as number of MB. So it is possible to extract the
> information from existing data and add another level of aggregation
> using only clever prepared SQL query.
>
> I think, that problem might be broader, than using existing, tweaked a
> bit model. If we take a look into resources, which FPGA may expose,
> than it can be couple of levels, and each of them can be treated as
> resource.
>
> It can identified 3 levels of FPGA resources, which can be nested one
> on the others:
>
> 1. Whole FPGA. If used discrete FPGA, than even today it might be pass
>through to the VM.
>
> 2. Region in FPGA. Some of the FPGA models can be divided into regions
>or slots. Also, for some model it is possible to (re)program such
>region individually - in this case there is a possibility to pass
>entire slot to the VM, so that it might be possible to reprogram
>such slot, and utilize the algorithm within the VM.
>
> 3. Accelerator in region/FPGA. If there is an accelerator programmed
>in the slot, it is possible, that such accelerator provides us with
>Virtual Functions (similar to the SR-IOV), than every available VF
>can be treated as a resource.
>
> 4. It might be also necessary to track every VF individually, although
>I didn't assumed it will be needed, nevertheless with nested
>resources it should be easy to handle it.
You need. For example you have 4 region and 8 VF. Some region is configured
with an accelerator so it can be shared to multi-VM (each consume a VF).
But
some other region is configured with private exclusive accelerator so it
can
only be bind to one VF. That's why we need to track both region and VF.

>
> Correlation between such resources are a bit different from NUMA -
> while in NUMA case there is a possibility to either schedule a VM with
> some memory specified, or request memory within NUMA cell, in FPGA if
> there is slot taken, or accelerator already programmed and used, there
> is no way to offer FPGA as a whole to the tenant, until all
> accelerators and slots are free.
>
> I've followed Jay idea about nested resources and having in mind
> blueprint[2] regarding dynamic resources I've prepared how it fit in.
>
> Tables are unchanged - it is a copy-paste from the etherpad[1]:
>
>
> CREATE TABLE resource_providers (
> id INT NOT NULL AUTOINCREMENT PRIMARY KEY,
> uuid CHAR(36) NOT NULL,
> name VARCHAR(100) NULL,
> root_provider_id INT NULL,
> parent_provider_id INT NULL
> );
>
> CREATE TABLE inventories (
> id INT NOT NULL AUTOINCREMENT PRIMARY KEY,
> resource_provider_id INT NOT NULL,
> resource_class_id INT NOT NULL,
> total INT NOT NULL,
> reserved INT NOT NULL,
> min_unit INT NOT NULL,
> max_unit INT NOT NULL,
> step_size INT NOT NULL,
> allocation_ratio INT NOT NULL
> );
>
> CREATE TABLE allocations (
> id INT NOT NULL AUTOINCREMENT PRIMARY KEY,
> resource_provider_id INT NOT NULL,
> consumer_uuid CHAR(36) NOT NULL,
> resource_class_id INT NOT NULL,
> used INT NOT NULL
> );
>
>
> Than lets fill the tables with data of following structure:
>
> -- FPGA-1
> --   +- FPGA-1 slot1 (taken), resource_provider_id:
> --   +- FPGA-1 slot2
> --   +- FPGA-1 slot2 acceleratorX
> --   +- FPGA-1 slot2 acceleratorX VF1 (taken)
> --   +- FPGA-1 slot2 acceleratorX VF2 (taken)
> --   +- FPGA-1 slot2 acceleratorX VF3 (taken)
> --   +- FPGA-1 slot2 acceleratorX VF4 (taken)
> --   +- FPGA-1 slot2 acceleratorX VF5
> --   +- ..
> --   +- FPGA-1 slot2 acceleratorX VF32
> --   +- FPGA-1 slot3
> -- FPGA-2
> --   +- FPGA-2 slot1
>
> where FPGA-1 and FPGA-2 are hosts with FPGA on board. There is also
> assumed, that new dynamic resources are created: id 1666 means 'FPGA'
> (although it might be simply standard class, which will be hardcoded
> ENUM), 1667 means 'FPGA slot' and 1668 'FPGA accelerator'.
>
>
> INSERT INTO resource_providers VALUES
> (1, '', 'FPGA-1', 1, NULL),
> (2, '', 'FPGA-1 slot 1', 1, 1),
> (3, '', 'FPGA-1 slot 2', 1, 1),
> (4, '', 'FPGA-1 slot 3', 1, 1),
> (5, '', 'FPGA-1 slot 2 acceleratorX', 1, 3),
> (6, '', 'FPGA-2', 6, NULL),
> (7, '', 'FPGA-2 slot', 6, 6);
>
>
> INSERT INTO inventories VALUES
> (1, 1, 1666, 1, 0, 1, 1, 1, 1.0),
> (2, 2, 1667, 1, 0, 1, 1, 1, 1.0),
> (3, 3, 1667, 1, 0, 1, 1, 1, 1.0),
> (4, 4, 16

Re: [openstack-dev] FPGA as a dynamic nested resources

[Fei K Chen]

Roman Dobosz  wrote on 2016/07/20 15:25:28:

> From: Roman Dobosz 
> To: "OpenStack Development Mailing List (not for usage questions)"
> 
> Cc: Ed Leafe 
> Date: 2016/07/20 15:30
> Subject: Re: [openstack-dev] FPGA as a dynamic nested resources
>
>
> > > 4. It might be also necessary to track every VF individually,
although
> > >   I didn't assumed it will be needed, nevertheless with nested
> > >   resources it should be easy to handle it.
> >
> > I’m still not seeing the need for nesting. If you have a single FPGA
> > with 8 slots, when you program the slots with accelerators, you now
> > have 8 consumable resources. The fact that they came from a
> > particular FPGA unit doesn’t seem relevant from a scheduling
> > perspective.
>
> Unless you have one FPGA with 8 slots, which can become FPGA with 4
> slots. From scheduling perspective you have to know, which FPGA
> resources can be reconfigured, and which not, isn't it? Also, AFAIRC
> to provide VM with VF, there is a need for providing libvirt with
> address of such VF, right? That's why I've putted this last point.
>
> The whole idea of getting FPGA as resource is its ability to swap
> resources on demand. So it can be thought of as several available
> hardware (means - accelerators, consumable by VMs) which most of the
> time are not programmed in certain moment.
>
Let's have more thought about the resource swapping. The number of run-time
accelerators is not limited by the number of region/slot. Inside FPGA,
there
can be some self-scheduling logic to schedule accelerators on regions by
using
the fast partial reconfiguration. It is not new, there are lots of such
design in FPGA academic.


-- Fei Chen
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] On testing...

[Richard Jones]

On 21 July 2016 at 10:13, Timur Sufiev  wrote:

> I'm not sure if (3) was the thing we agreed upon (and if it will help us
> to detect issues in Horizon) but the rest of options definitely make sense
> to me.
>

I think that's reasonable - moving the tests to tempest will make it more
difficult to run them locally.



> Also there is
>
> 5. Parallelize integration tests.
>

Ah, yep, good catch. I'm not sure whether this will negatively impact our
stability on some of the test nodes though :/


 Richard



> On Wed, Jul 20, 2016 at 4:28 PM Richard Jones 
> wrote:
>
>> Hi folks,
>>
>> Sorry I didn't make the meeting this morning :-(
>>
>> We touched on testing at the mid-cycle and again it came up in the
>> meeting this morning. We identified two issues:
>>
>> 1. Our integration test suite cannot grow to too many more tests because
>> they take too long to run (and get auto-killed). We could increase the
>> amount of time allowed for them, but we agreed that we shouldn't do this.
>> 2. We don't have enough higher-level (integration leve) test coverage for
>> our newer angular interfaces.
>>
>> These two are obviously at odds :-)
>>
>> What we agreed on, I think, is that we're going to:
>>
>> 1. Reduce the granularity of the integration tests - use them to broadly
>> test whether an interface works at all when presented to a browser,
>> 2. Replace many single interface tests with a fewer tests that poke at
>> many interfaces (thus removing the significant per-test overhead) - even
>> potentially having a single test that attempts to access every panel (as a
>> guard against gross Javascript incompatibilities or configuration issues
>> breaking panels),
>> 3. Move the integration tests to tempest, and
>> 4. Write some "django unit test suite" functional tests for our angular
>> code that tests more than just the single units we currently test (ie. mock
>> at a more remote level, checking that broader interactions work).
>>
>> Does this sound about right?
>>
>>
>>  Richard
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [tripleo] moving tripleo-ci/test-environments into THT

[Emilien Macchi]

Hi,

We're currently using tripleo-ci to store our test-environments files
(ie: multinode.yaml, etc).
To make it compatible with our different versions of TripleO, we have 2 options:

* Duplicate templates and use bash conditionals in tripleo-ci scripts
to select which one we want at each release.
* Move them to THT (THT is branched & released).

I would vote for option #2 for 2 reasons:
* we don't have to do complex conditionals in tripleo-ci
* we can easily consume it outside tripleo-ci (oooq one day?)
* we can easily make them evolve, when new composable services are
created for example.

Please let me know if I missed something or if I'm wrong, but I would
like to make the choice before the end of Newton, so we can scale our
way to manage {dev,ci}-templates.

Thanks,
-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] [RFC] ResourceProviderTags - Manage Capabilities with ResourceProvider

[Alex Xu]

2016-07-20 11:08 GMT-07:00 Jay Pipes :

> On 07/18/2016 01:45 PM, Matt Riedemann wrote:
>
>> On 7/15/2016 8:06 PM, Alex Xu wrote:
>>
>>>
>>> Actually I still think aggregates isn't good for Manage Caps, just as I
>>> said in previous reply about Aggregates. One of reason is just same with
>>> #2 you said :) And It's totally not managable. User is even no way to
>>> query a specific host in which host-aggregate. And there isn't a
>>> interface to query what metadata was related to the host by
>>> host-aggregate. I prefer just keep the Aggregate as tool to group the
>>> hosts. But yes, user still can use host-aggregate to manage host with
>>> old way, let's user decide what is more convenient.
>>>
>>
>> +1 to Alex's point. I just read through this thread and had the same
>> thought. If the point is to reduce complexity in the system and surface
>> capabilities to the end user, let's do that with resource provider tags,
>> not a mix of host aggregate metadata and resource provider tags so that
>> an operator has to set both, but also know in what situations he/she has
>> to set it and where.
>>
>
> Yeah, having the resource provider be tagged with capabilities versus
> having to manage aggregate tags may make some of the qualitative matching
> queries easier to grok. The query performance won't necessarily be any
> better, but they will likely be easier to read...
>
> I'm hoping Jay or someone channeling Jay can hold my hand and walk me
>> safely through the evil forest that is image properties / flavor extra
>> specs / scheduler hints / host aggregates / resource providers / and the
>> plethora of scheduler filters that use them to build a concrete
>> picture/story tying this all together. I'm thinking like use cases, what
>> does the operator need to do
>>
>
> Are you asking how things are *currently* done in Nova? If so, I'll need
> to find some alcohol.
>
> If you are asking about how we'd *like* all of the qualitative things to
> be requested and queried in the new placement API, then less alcohol is
> required.
>
> The schema I'm thinking about on the placement engine side looks like this:
>
> CREATE TABLE tags (
>   id INT NOT NULL,
>   name VARCHAR(200) NOT NULL,
>   PRIMARY KEY (id),
>   UNIQUE INDEX (name)
> );
>
> CREATE TABLE resource_provider_tags (
>   resource_provider_id INT NOT NULL
>   tag_id INT NOT NULL,
>   PRIMARY KEY (resource_provider_id, tag_id),
>   INDEX (tag_id)
> );
>
> On the Nova side, we need a mechanism of associating a set of capabilities
> that may either be required or preferred. The thing that we currently use
> for associating requested things in Nova is the flavor, so we'd need to
> define a mapping in Nova for the tags a flavor would require or prefer.
>
> CREATE TABLE flavor_tags (
>   flavor_id INT NOT NULL,
>   tag_name VARCHAR(200) NOT NULL,
>   is_required INT NOT NULL
> );
>
> We would need to have a call in the placement REST API to find the
> resource providers that matched a particular set of required or preferred
> capability tags. Such a call might look like the following:
>
> GET /resource_providers
> {
>   "resources": {
> "VCPU": 2,
> "MEMORY_MB": 2048,
> "DISK_GB": 100
>   },
>   "requires": [
> "storage:ssd",
> "compute:hw:x86:avx2",
>   ],
>   "prefers": [
> "compute:virt:accelerated_whizzybang"
>   ]
> }
>

so GET with a request body?


>
> Disregard the quantitative side of the above request right now. We could
> answer the qualitative side of the equation with the following SQL query in
> the placement engine:
>
> SELECT rp.uuid
> FROM resource_providers AS rp, tags AS t1, tags AS t2, tags AS t3
> INNER JOIN resource_provider_tags AS rpt1
> ON rp.id = rpt1.resource_provider_id
> AND rpt1.tag_id = t1.id
> INNER JOIN resource_provider_tags AS rpt2
> AND rpt1.resource_provider_id = rpt2.resource_provider_id
> AND rpt2.tag_id = t2.id
> LEFT JOIN resource_provider_tags AS rpt3
> ON rp.id = rpt3.resource_provider_id
> AND rpt3.tag_id = t3.id
> GROUP BY rp.uuid
> ORDER BY COUNT(COALESCE(rpt3.resource_provider_id, 0)) DESC
> WHERE t1.name = 'storage:ssd'
> AND t2.name = 'compute:hw:x86:avx2'
> AND t3.name IN ('compute:virt:accelerated_whizzybang')
>
> The above returns all resource providers having the 'storage:ssd' and
> 'compute:hw:x86:avx2' tags and returns resource providers *first* that have
> the 'compute:virt:accelerated_whizzybang' tag.
>
> , what does the end user of the cloud need
>> to do, etc. I think if we're going to do resource providers tags for
>> capabilities we also need to think about what we're replacing. Maybe
>> that's just host aggregate metadata, but what's the deprecation plan for
>> that?
>>
>
> Good question, as usual. My expectation would be that in Ocata, when we
> start adding the qualitative aspects to the placement REST API, we would
> introduce documentation that operators could use to translate common use
> cases that they were using flavor extra_specs and aggregate metadata for in
> the pre-placemen

Re: [openstack-dev] [Nova] [RFC] ResourceProviderTags - Manage Capabilities with ResourceProvider

[Alex Xu]

2016-07-20 11:43 GMT-07:00 Mooney, Sean K :

>
>
> > -Original Message-
> > From: Jay Pipes [mailto:jaypi...@gmail.com]
> > Sent: Wednesday, July 20, 2016 7:16 PM
> > To: openstack-dev@lists.openstack.org
> > Subject: Re: [openstack-dev] [Nova] [RFC] ResourceProviderTags - Manage
> > Capabilities with ResourceProvider
> >
> > On 07/13/2016 01:37 PM, Ed Leafe wrote:
> > > On Jul 11, 2016, at 6:08 AM, Alex Xu  wrote:
> > >
> > >> For example, the capabilities can be defined as:
> > >>
> > >> COMPUTE_HW_CAP_CPU_AVX
> > >> COMPUTE_HW_CAP_CPU_SSE
> > >> 
> > >> COMPUTE_HV_CAP_LIVE_MIGRATION
> > >> COMPUTE_HV_CAP_LIVE_SNAPSHOT
> > >> 
> > >>
> > >> ( The COMPUTE means this is coming from Nova. HW means this is
> > >> hardware related Capabilities. HV means this is  capabilities of
> > >> Hypervisor. But the catalog of Capabilities can be discussed
> > >> separated. This propose focus on the  ResourceTags. We also have
> > >> another idea about not using 'PREFIX' to manage the Tags. We can add
> > >> attributes to the  Tags. Then we have more control on the Tags. This
> > >> will describe separately in the bottom. )
> > >
> > > I was ready to start ranting about using horribly mangled names to
> > represent data, and then saw your comment about attributes for tags.
> > Yes, a thousand times yes to attributes! There can be several
> > standards, such as ‘compute’ or ‘networking’ that we use for some basic
> > cross-cloud compatibility, but making them flexible is a must for
> > adoption.
> >
> > I disagree :) Adoption -- at least interoperable cloud adoption -- of
> > this functionality will likely be hindered by super-flexible
> > description of capabilities. I think having a set of "standard"
> > capabilities that can be counted on to be cross-OpenStack-cloud
> > compatible and a set of "dynamic" capabilities that are custom to a
> > deployment would be a good thing to do.
>
> [Mooney, Sean K]
> I know there is a bad memories when I metion CIM (
> http://www.dmtf.org/standards/cim)
> for many on the nova team but if we are to use standard names we should
> probably
> actually assess are there existing standads that we could adopt instead of
> defining
> our own standard names in nova for the resources.
> For example
> http://schemas.dmtf.org/wbem/cim-html/2/CIM_ProcessorAllocationSettingData.html
> Define the name for different instcution set extentions for example avx is
> DMTF:x86:AVX.
> Some work has been done in glance to allow importing cim metadata from ovf
> files also
>
> https://specs.openstack.org/openstack/glance-specs/specs/mitaka/implemented/cim-namespace-metadata-definitions.html
>
> while I don’t think using the full cim information model is useful in this
> case using the name would be
> from an inter-operability point of view as we not only would have standard
> names in openstack but those names
> would conform to an existing standard.
>

Thanks! This is good suggestion. For 'DMTF:x86:AVX', we probably can
reference the 'x86:AVX', then we probably needs add some prefix like
'COMPUTE_HW_CPU'


>
> We could still allow custom attribute but is see value in standardizing
> what can be standardized.
>
>
> >
> > Best,
> > -jay
> >
> > > I can update the qualitative request spec to add ResourceProviderTags
> > as a possible implementation.
> >
> > ___
> > ___
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: OpenStack-dev-
> > requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Angular action services and initScope

[Richard Jones]

Thanks for the response, Thai!

Before I get into addressing your specifics below, I've put up a proof of
concept patch that modifies images create-volume to remove the need for
scope:

  https://review.openstack.org/#/c/345145


On 21 July 2016 at 08:23, Thai Q Tran  wrote:

> You all sort of touched on what I am about to reiterate:
>
> 1. The scope was intended to be used as a way to propagate events up and
> down the chain (from table controller to step controller) since there was
> no real way to share information between controllers (remember that a modal
> dialog launches an entirely different fragment of HTML with no relationship
> to the current controller). Furthermore, we needed to use initScope to
> explicitly set the scope so that $modal does not use the $rootScope (which
> it does by default if scope is not set). As a side bonus, we were also able
> to use events to share data between the steps. The idea behind was to *make
> steps independent* *and reusable* in multiple workflows. The reality is
> that the event propagation required both an $emit and a $broadcast to
> actually share data between steps, this makes it much less desirable.
>

We can definitely still share data between steps, and my proposal makes it
easier, I think: every step has access to the data captured by every other
step (through $scope.stepModels) which they can $watch on or even modify.
There's still a scope encompassing the wizard modal, it's just not tied
back up to the table scope.

As I said previously, I don't believe we should overly complicate things in
the pursuit of making steps reusable, since outside of a potential re-use
of metadata step (still to be proven whether that's even feasible) I
believe YAGNI.



> *The root of the problem is, how does my table know to update once the
> form is submitted?*
>

The table knows through the new ActionResult mechanism (which is
automatically handled by hz-resource-table and manually handled by the
handler provided to hz-dynamic-table ... and even more manually handled if
you don't use either of those ).



>  2. Going with Richards idea of a shared workflow.model idea.
> *...*
> To me there are two questions we should answer before we jump and do
> another rewrite.
> 1. How are we going to glue the various controllers together if we don't
> use scope.
> 2. How is data between the steps going to be shared.
>

I believe I've addressed these questions in my POC patch.


 Richard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] FPGA as a dynamic nested resources

[Roman Dobosz]

On Wed, 20 Jul 2016 10:07:12 +0100
"Daniel P. Berrange"  wrote:

Hey Daniel, thanks for the feedback.

> > Thoughts?
> 
> I'd suggest you'll increase your chances of success with nova design
> approval if you focus on implementing a really simple usage scheme for
> FPGA as the first step in Nova.

This. Maybe I'm wrong, but for me the minimal use case for FPGA would
be ability to schedule VM which need certain accelerator from multiple
potential ones on available FPGA/fixed slot. How insane does it sound?

Providing fixed, prepared earlier by DC administrator accelerator
resource, doesn't bring much value, beyond what we already have in
Nova, since PCI/SR-IOV passthrough might be used for accelerators,
which expose their functionality via VF.

> All the threads I've see go well off into the weeds about trying to 
> solve everybody's niche/edge cases  perfectly and as a result get 
> very complicated.

The topic is complicated :)

> For both NUMA and PCI dev assignment we got initial success by cutting
> back scope and focusing on the doing the minimum possible to satisfy
> the 90% common use cases, and ignoring the less common 10% initially.
> Yes this is not optimal, but it is good enough to keep most people
> happy without introducing massive complexity into the designs & impl.
> 
> For FPGA, I'd like to see an initial proposal that assumed the FPGA
> is pre-programmed & pre-divided into a fixed number of slots and simply
> deal with this. This is similar to how we dealt with PCI SR-IOV initially
> where we assumed the dev is in VF-mode only. Only later did we start to
> add cleverness around switching VF vs PF mode. For FPGA I think any kind
> of dynamic re-allocation/re-configuration is better done as a stage 2

Okay. That sounds reasonable.

-- 
Cheers,
Roman Dobosz

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [networking-ovs-dpdk] networking_ovs_dpdk.agent.ovs_dpdk_firewall.OVSFirewallDriver

[Gangur, Hrushikesh]

Is this firewall drive 
'networking_ovs_dpdk.agent.ovs_dpdk_firewall.OVSFirewallDriver' OpenStack 
upstream supported? Any drawback of using this with our OVS-DPDK limitations?

It looks it is part of devstack and works w/ Mitaka
https://github.com/openstack/networking-ovs-dpdk/blob/stable/mitaka/doc/source/getstarted/devstack/ubuntu.rst

Regards~hrushi
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] Delorean fail blocks CI for stable branches

[Sagi Shnaidman]

On Thu, Jul 21, 2016 at 3:11 AM, Alan Pevec  wrote:

> On Wed, Jul 20, 2016 at 7:49 PM, Sagi Shnaidman 
> wrote:
> > How then it worked before? Can you show me the patch that broke this
> > functionality in delorean? It should be about 15 Jul when jobs started to
> > fail.
>
> commented in lp
>
> > How then master branch works? It also runs on patched repo and succeeds.
>
> I explained that but looks like we're talking past each other.
>
> > I don't think we can use this workaround, each time this source file will
> > change - all our jobs will fail again? It's not even a workaround.
> > Please let's stop discussing and let's solve it finally, it blocks our CI
> > for stable patches.
>
> Sure, I've assigned https://bugs.launchpad.net/tripleo/+bug/1604039 to
> myself and proposed a patch.
>
>
It's a workaround for short time range, but NOT a solution, if you change
something in this one file, it'll be broken again. But it does NOT solve
the main issue - after recent changes in dlrn and specs we can't build repo
with delorean on stable branches.
I think it should be solved on DLRN side and should be provided a
appropriate interface to use it for CI purposes.
I opened an issue there:
https://github.com/openstack-packages/DLRN/issues/22
But you closed it, so I suppose we will not get any solution and help for
it from your side?

Should we move to other packaging tool?


> Alan
>



-- 
Best regards
Sagi Shnaidman
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] FPGA as a dynamic nested resources

[Roman Dobosz]

On Thu, 21 Jul 2016 08:44:21 +0800
Fei K Chen  wrote:

> > 4. It might be also necessary to track every VF individually, although
> >I didn't assumed it will be needed, nevertheless with nested
> >resources it should be easy to handle it.
> You need. For example you have 4 region and 8 VF. Some region is 
> configured with an accelerator so it can be shared to multi-VM (each 
> consume a VF). But some other region is configured with private 
> exclusive accelerator so it can only be bind to one VF. That's why 
> we need to track both region and VF.

Well, it depends. If there is no difference between the VF (all 
provides the same functionality) and we don't really care about the 
placement (external entity would take care of this) than we don't need 
this level. All the information will be hold by resource inventory and 
allocation.

OTOH if we need to store the information which VF is passed to which 
VM, than probably we need this level, or store VF addresses in 
inventory/allocation in some new filed.

-- 
Cheers,
Roman Dobosz

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] FPGA as a dynamic nested resources

[Roman Dobosz]

On Thu, 21 Jul 2016 08:56:07 +0800
Fei K Chen  wrote:

> > Unless you have one FPGA with 8 slots, which can become FPGA with 4
> > slots. From scheduling perspective you have to know, which FPGA
> > resources can be reconfigured, and which not, isn't it? Also, AFAIRC
> > to provide VM with VF, there is a need for providing libvirt with
> > address of such VF, right? That's why I've putted this last point.
> >
> > The whole idea of getting FPGA as resource is its ability to swap
> > resources on demand. So it can be thought of as several available
> > hardware (means - accelerators, consumable by VMs) which most of the
> > time are not programmed in certain moment.
> >
> Let's have more thought about the resource swapping. The number of 
> run-time accelerators is not limited by the number of region/slot. 
> Inside FPGA, there can be some self-scheduling logic to schedule 
> accelerators on regions by using the fast partial reconfiguration. 
> It is not new, there are lots of such design in FPGA academic.

Right, but not all devices have such functionality. And we are trying 
to make this solution common for most FPGA, right?

-- 
Cheers,
Roman Dobosz

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev