Re: [Openstack-operators] Fwd: Nova hypervisor uuid

[Ignazio Cassano]

Hello Mattm
Yes I mean sometimes I have same host/node names with different uuid in
compute_nodes table in nova database
I must delete nodes with uuid those not match with nova-hypervisor list
command.
At this time I have the following:
MariaDB [nova]> select hypervisor_hostname,uuid,deleted from compute_nodes;
+-+--+-+
| hypervisor_hostname | uuid | deleted |
+-+--+-+
| tst2-kvm02  | 802b21c2-11fb-4426-86b9-bf25c8a5ae1d |   0 |
| tst2-kvm01  | ce27803b-06cd-44a7-b927-1fa42c813b0f |   0 |
+-+--+-+
2 rows in set (0,00 sec)


But sometimes old uuid are inserted in the table .
I deleted again them.
I restarted kvm nodes and now the table is ok.
I also restarded each controller and the tables is ok.
I do not know because 3 days ago I had same compute nodes names with
different uuids.

Thanks and Regards
Ignazio

Il giorno mer 28 nov 2018 alle ore 17:54 Matt Riedemann 
ha scritto:

> On 11/28/2018 4:19 AM, Ignazio Cassano wrote:
> > Hi Matt, sorry but I lost your answer and Gianpiero forwarded it to me.
> > I am sure kvm nodes names are note changed.
> > Tables where uuid are duplicated are:
> > dataresource_providers in nova_api db
> > compute_nodes in nova db
> > Regards
> > Ignazio
>
> It would be easier if you simply dumped the result of a select query on
> the compute_nodes table where the duplicate nodes exist (you said
> duplicate UUIDs but I think you mean duplicate host/node names with
> different UUIDs, correct?).
>
> There is a unique constraint on host/hypervisor_hostname
> (nodename)/deleted:
>
>  schema.UniqueConstraint(
>  'host', 'hypervisor_hostname', 'deleted',
>  name="uniq_compute_nodes0host0hypervisor_hostname0deleted"),
>
> So I'm wondering if the deleted field is not 0 on one of those because
> if one is marked as deleted, then the compute service will create a new
> compute_nodes table record on startup (and associated resource provider).
>
> --
>
> Thanks,
>
> Matt
>
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack] TAPaaS on Queens?

[reedip banerjee]

Hi M.Ranganathan,
Tap-as-a-Service has a release for Stable/Queens and Stable/Rocky. However,
its not yet an official project in Openstack, so it might not be listed
there.

On Thu, Nov 29, 2018 at 7:21 AM M. Ranganathan  wrote:

> Hello all,
>
> I want to experiment with SNORT as a service on OpenStack. I looked at the
> TAPaaS project. However, I am not sure it runs on OpenStack Queens. I don't
> see it in the queens release
> https://releases.openstack.org/queens/index.html so I am wondering if
> this project is still alive (?)
>
>
> Thanks,
>
> Ranga
>
> --
> M. Ranganathan
>
> ___
> Mailing list:
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to : openstack@lists.openstack.org
> Unsubscribe :
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack



-- 
Thanks and Regards,
Reedip Banerjee
IRC: reedip
___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

[Openstack] How to create a tap port using openstack API?

[M. Ranganathan]

Hello,

I want to write my own VNF. For this I need to :

1. Create a network namespace.
2. Create a ovs internal port on a bridge with an appropriate tag.
3. Send the port to the network namespace.
4. Run a service in the network namespace that could (for example) read
packets.

Are there "openstack networking" commands to do steps 1-3 above or should
this be done manually?

( What I want to do, essentially is to create a tap port and read packets
from this tap port. Incidentally, the TAPaaS project was doing something
like this but it seems to be unsupported.)

Thanks,

Ranga


-- 
M. Ranganathan
___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

[Openstack] TAPaaS on Queens?

[M. Ranganathan]

Hello all,

I want to experiment with SNORT as a service on OpenStack. I looked at the
TAPaaS project. However, I am not sure it runs on OpenStack Queens. I don't
see it in the queens release
https://releases.openstack.org/queens/index.html so I am wondering if this
project is still alive (?)


Thanks,

Ranga

-- 
M. Ranganathan
___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

[Openstack] OpenStack federation and WAYF process with multiple IdPs

[Rafael Weingärtner]

Hello Openstackers,

I am testing the integration of OpenStack (acting as a service provider)
using Keycloak (as an identity provider) with OpenId Connect protocol. So
far everything is working, but when I enable more than one IdP, I get an
odd behavior. The “where are you from (WAYF)” process is happening twice,
one in Horizon (where the user selects the authentication provider A.K.A
IdP), and another one in Keystone via the Apache HTTPD OIDC module. I
assume this is happening because the actual application being authenticated
via OIDC is Keystone, and just afterwards, the other systems will
authenticate themselves via Keystone.

Has anybody else experienced/”dealt with” this situation? Is this by design?
Am I missing a parameter/configuration or something else?

The version of OpenStack that I am using is Rocky.

--
Rafael Weingärtner
___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [openstack-dev] [TripleO][Edge] Reduce base layer of containers for security and size of images (maintenance) sakes

[Dan Prince]

On Wed, 2018-11-28 at 13:28 -0500, James Slagle wrote:
> On Wed, Nov 28, 2018 at 12:31 PM Bogdan Dobrelya  > wrote:
> > Long story short, we cannot shoot both rabbits with a single shot,
> > not
> > with puppet :) May be we could with ansible replacing puppet
> > fully...
> > So splitting config and runtime images is the only choice yet to
> > address
> > the raised security concerns. And let's forget about edge cases for
> > now.
> > Tossing around a pair of extra bytes over 40,000 WAN-distributed
> > computes ain't gonna be our the biggest problem for sure.
> 
> I think it's this last point that is the crux of this discussion. We
> can agree to disagree about the merits of this proposal and whether
> it's a pre-optimzation or micro-optimization, which I admit are
> somewhat subjective terms. Ultimately, it seems to be about the "why"
> do we need to do this as to the reason why the conversation seems to
> be going in circles a bit.
> 
> I'm all for reducing container image size, but the reality is that
> this proposal doesn't necessarily help us with the Edge use cases we
> are talking about trying to solve.
> 
> Why would we even run the exact same puppet binary + manifest
> individually 40,000 times so that we can produce the exact same set
> of
> configuration files that differ only by things such as IP address,
> hostnames, and passwords? Maybe we should instead be thinking about
> how we can do that *1* time centrally, and produce a configuration
> that can be reused across 40,000 nodes with little effort. The
> opportunity for a significant impact in terms of how we can scale
> TripleO is much larger if we consider approaching these problems with
> a wider net of what we could do. There's opportunity for a lot of
> better reuse in TripleO, configuration is just one area. The plan and
> Heat stack (within the ResourceGroup) are some other areas.

We run Puppet for configuration because that is what we did on
baremetal and we didn't break backwards compatability for our
configuration options for upgrades. Our Puppet model relies on being
executed on each local host in order to splice in the correct IP
address and hostname. It executes in a distributed fashion, and works
fairly well considering the history of the project. It is robust,
guarantees no duplicate configs are being set, and is backwards
compatible with all the options TripleO supported on baremetal. Puppet
is arguably better for configuration than Ansible (which is what I hear
people most often suggest we replace it with). It suits our needs fine,
but it is perhaps a bit overkill considering we are only generating
config files.

I think the answer here is moving to something like Etcd. Perhaps
skipping over Ansible entirely as a config management tool (it is
arguably less capable than Puppet in this category anyway). Or we could
use Ansible for "legacy" services only, switch to Etcd for a majority
of the OpenStack services, and drop Puppet entirely (my favorite
option). Consolidating our technology stack would be wise.

We've already put some work and analysis into the Etcd effort. Just
need to push on it some more. Looking at the previous Kubernetes
prototypes for TripleO would be the place to start.

Config management migration is going to be tedious. Its technical debt
that needs to be handled at some point anyway. I think it is a general
TripleO improvement that could benefit all clouds, not just Edge.

Dan

> 
> At the same time, if some folks want to work on smaller optimizations
> (such as container image size), with an approach that can be agreed
> upon, then they should do so. We just ought to be careful about how
> we
> justify those changes so that we can carefully weigh the effort vs
> the
> payoff. In this specific case, I don't personally see this proposal
> helping us with Edge use cases in a meaningful way given the scope of
> the changes. That's not to say there aren't other use cases that
> could
> justify it though (such as the security points brought up earlier).
> 


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO][Edge] Reduce base layer of containers for security and size of images (maintenance) sakes

[James Slagle]

On Wed, Nov 28, 2018 at 12:31 PM Bogdan Dobrelya  wrote:
> Long story short, we cannot shoot both rabbits with a single shot, not
> with puppet :) May be we could with ansible replacing puppet fully...
> So splitting config and runtime images is the only choice yet to address
> the raised security concerns. And let's forget about edge cases for now.
> Tossing around a pair of extra bytes over 40,000 WAN-distributed
> computes ain't gonna be our the biggest problem for sure.

I think it's this last point that is the crux of this discussion. We
can agree to disagree about the merits of this proposal and whether
it's a pre-optimzation or micro-optimization, which I admit are
somewhat subjective terms. Ultimately, it seems to be about the "why"
do we need to do this as to the reason why the conversation seems to
be going in circles a bit.

I'm all for reducing container image size, but the reality is that
this proposal doesn't necessarily help us with the Edge use cases we
are talking about trying to solve.

Why would we even run the exact same puppet binary + manifest
individually 40,000 times so that we can produce the exact same set of
configuration files that differ only by things such as IP address,
hostnames, and passwords? Maybe we should instead be thinking about
how we can do that *1* time centrally, and produce a configuration
that can be reused across 40,000 nodes with little effort. The
opportunity for a significant impact in terms of how we can scale
TripleO is much larger if we consider approaching these problems with
a wider net of what we could do. There's opportunity for a lot of
better reuse in TripleO, configuration is just one area. The plan and
Heat stack (within the ResourceGroup) are some other areas.

At the same time, if some folks want to work on smaller optimizations
(such as container image size), with an approach that can be agreed
upon, then they should do so. We just ought to be careful about how we
justify those changes so that we can carefully weigh the effort vs the
payoff. In this specific case, I don't personally see this proposal
helping us with Edge use cases in a meaningful way given the scope of
the changes. That's not to say there aren't other use cases that could
justify it though (such as the security points brought up earlier).

-- 
-- James Slagle
--

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO][Edge] Reduce base layer of containers for security and size of images (maintenance) sakes

[Bogdan Dobrelya]

On 11/28/18 6:02 PM, Jiří Stránský wrote:





Reiterating again on previous points:

-I'd be fine removing systemd. But lets do it properly and not via 'rpm
-ev --nodeps'.
-Puppet and Ruby *are* required for configuration. We can certainly put
them in a separate container outside of the runtime service containers
but doing so would actually cost you much more space/bandwidth for each
service container. As both of these have to get downloaded to each node
anyway in order to generate config files with our current mechanisms
I'm not sure this buys you anything.


+1. I was actually under the impression that we concluded yesterday on 
IRC that this is the only thing that makes sense to seriously consider. 
But even then it's not a win-win -- we'd gain some security by leaner 
production images, but pay for it with space+bandwidth by duplicating 
image content (IOW we can help achieve one of the goals we had in mind 
by worsening the situation w/r/t the other goal we had in mind.)


Personally i'm not sold yet but it's something that i'd consider if we 
got measurements of how much more space/bandwidth usage this would 
consume, and if we got some further details/examples about how serious 
are the security concerns if we leave config mgmt tools in runtime images.


IIRC the other options (that were brought forward so far) were already 
dismissed in yesterday's IRC discussion and on the reviews. Bin/lib bind 
mounting being too hacky and fragile, and nsenter not really solving the 
problem (because it allows us to switch to having different bins/libs 
available, but it does not allow merging the availability of bins/libs 
from two containers into a single context).




We are going in circles here I think


+1. I think too much of the discussion focuses on "why it's bad to have 
config tools in runtime images", but IMO we all sorta agree that it 
would be better not to have them there, if it came at no cost.


I think to move forward, it would be interesting to know: if we do this 
(i'll borrow Dan's drawing):


|base container| --> |service container| --> |service container w/
Puppet installed|

How much more space and bandwidth would this consume per node (e.g. 
separately per controller, per compute). This could help with decision 
making.


As I've already evaluated in the related bug, that is:

puppet-* modules and manifests ~ 16MB
puppet with dependencies ~61MB
dependencies of the seemingly largest a dependency, systemd ~190MB

that would be an extra layer size for each of the container images to be 
downloaded/fetched into registries.


Given that we should decouple systemd from all/some of the dependencies 
(an example topic for RDO [0]), that could save a 190MB. But it seems we 
cannot break the love of puppet and systemd as it heavily relies on the 
latter and changing packaging like that would higly likely affect 
baremetal deployments with puppet and systemd co-operating.


Long story short, we cannot shoot both rabbits with a single shot, not 
with puppet :) May be we could with ansible replacing puppet fully...
So splitting config and runtime images is the only choice yet to address 
the raised security concerns. And let's forget about edge cases for now.
Tossing around a pair of extra bytes over 40,000 WAN-distributed 
computes ain't gonna be our the biggest problem for sure.


[0] https://review.rdoproject.org/r/#/q/topic:base-container-reduction





Dan



Thanks

Jirka

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



--
Best regards,
Bogdan Dobrelya,
Irc #bogdando

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO][Edge] Reduce base layer of containers for security and size of images (maintenance) sakes

[Jiří Stránský]

Reiterating again on previous points:

-I'd be fine removing systemd. But lets do it properly and not via 'rpm
-ev --nodeps'.
-Puppet and Ruby *are* required for configuration. We can certainly put
them in a separate container outside of the runtime service containers
but doing so would actually cost you much more space/bandwidth for each
service container. As both of these have to get downloaded to each node
anyway in order to generate config files with our current mechanisms
I'm not sure this buys you anything.


+1. I was actually under the impression that we concluded yesterday on 
IRC that this is the only thing that makes sense to seriously consider. 
But even then it's not a win-win -- we'd gain some security by leaner 
production images, but pay for it with space+bandwidth by duplicating 
image content (IOW we can help achieve one of the goals we had in mind 
by worsening the situation w/r/t the other goal we had in mind.)


Personally i'm not sold yet but it's something that i'd consider if we 
got measurements of how much more space/bandwidth usage this would 
consume, and if we got some further details/examples about how serious 
are the security concerns if we leave config mgmt tools in runtime images.


IIRC the other options (that were brought forward so far) were already 
dismissed in yesterday's IRC discussion and on the reviews. Bin/lib bind 
mounting being too hacky and fragile, and nsenter not really solving the 
problem (because it allows us to switch to having different bins/libs 
available, but it does not allow merging the availability of bins/libs 
from two containers into a single context).




We are going in circles here I think


+1. I think too much of the discussion focuses on "why it's bad to have 
config tools in runtime images", but IMO we all sorta agree that it 
would be better not to have them there, if it came at no cost.


I think to move forward, it would be interesting to know: if we do this 
(i'll borrow Dan's drawing):


|base container| --> |service container| --> |service container w/
Puppet installed|

How much more space and bandwidth would this consume per node (e.g. 
separately per controller, per compute). This could help with decision 
making.




Dan



Thanks

Jirka

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack-operators] Fwd: Nova hypervisor uuid

[Matt Riedemann]

On 11/28/2018 4:19 AM, Ignazio Cassano wrote:

Hi Matt, sorry but I lost your answer and Gianpiero forwarded it to me.
I am sure kvm nodes names are note changed.
Tables where uuid are duplicated are:
dataresource_providers in nova_api db
compute_nodes in nova db
Regards
Ignazio


It would be easier if you simply dumped the result of a select query on 
the compute_nodes table where the duplicate nodes exist (you said 
duplicate UUIDs but I think you mean duplicate host/node names with 
different UUIDs, correct?).


There is a unique constraint on host/hypervisor_hostname (nodename)/deleted:

schema.UniqueConstraint(
'host', 'hypervisor_hostname', 'deleted',
name="uniq_compute_nodes0host0hypervisor_hostname0deleted"),

So I'm wondering if the deleted field is not 0 on one of those because 
if one is marked as deleted, then the compute service will create a new 
compute_nodes table record on startup (and associated resource provider).


--

Thanks,

Matt

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [openstack-dev] [tripleo] Workflows Squad changes

[Ryan Brady]

On Wed, Nov 28, 2018 at 5:13 AM Jiri Tomasek  wrote:

> Hi all,
>
> Recently, the workflows squad has been reorganized and people from the
> squad are joining different squads. I would like to discuss how we are
> going to adjust to this situation to make sure that tripleo-common
> development is not going to be blocked in terms of feature work and reviews.
>
> With this change, most of the tripleo-common maintenance work goes
> naturally to UI & Validations squad as CLI and GUI are the consumers of the
> API provided by tripleo-common. Adriano Petrich from workflows squad has
> joined UI squad to take on this work.
>
> As a possible solution, I would like to propose Adriano as a core reviewer
> to tripleo-common and adding tripleo-ui cores right to +2 tripleo-common
> patches.
>

> It would be great to hear opinions especially former members of Workflows
> squad and regular contributors to tripleo-common on these changes and in
> general on how to establish regular reviews and maintenance to ensure that
> tripleo-common codebase is moved towards converging the CLI and GUI
> deployment workflow.
>

Well, I'm not really going that far and plan to continue working in
tripleo-common for the time being.  If that isn't sustainable during the
next cycle, I'll make sure to shout.


> Thanks
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 

RYAN BRADY

SENIOR SOFTWARE ENGINEER

Red Hat Inc 

rbr...@redhat.comT: (919)-890-8925 IM: rbrady

@redhatway    @redhatinc
   @redhatsnaps

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO][Edge] Reduce base layer of containers for security and size of images (maintenance) sakes

[Fox, Kevin M]

Ok, so you have the workflow in place, but it sounds like the containers are 
not laid out to best use that workflow. Puppet is in the base layer. That means 
whenever puppet gets updated, all the other containers must be too. And other 
such update coupling issues.

I'm with you, that binaries should not be copied from one container to another 
though.

Thanks,
Kevin

From: Dan Prince [dpri...@redhat.com]
Sent: Wednesday, November 28, 2018 5:31 AM
To: Former OpenStack Development Mailing List, use openstack-discuss now; 
openstack-disc...@lists.openstack.org
Subject: Re: [openstack-dev] [TripleO][Edge] Reduce base layer of containers 
for security and size of images (maintenance) sakes

On Wed, 2018-11-28 at 00:31 +, Fox, Kevin M wrote:
> The pod concept allows you to have one tool per container do one
> thing and do it well.
>
> You can have a container for generating config, and another container
> for consuming it.
>
> In a Kubernetes pod, if you still wanted to do puppet,
> you could have a pod that:
> 1. had an init container that ran puppet and dumped the resulting
> config to an emptyDir volume.
> 2. had your main container pull its config from the emptyDir volume.

We have basically implemented the same workflow in TripleO today. First
we execute Puppet in an "init container" (really just an ephemeral
container that generates the config files and then goes away). Then we
bind mount those configs into the service container.

One improvement we could make (which we aren't doing yet) is to use a
data container/volume to store the config files instead of using the
host. Sharing *data* within a 'pod' (set of containers, etc.) is
certainly a valid use of container volumes.

None of this is what we are really talking about in this thread though.
Most of the suggestions and patches are about making our base
container(s) smaller in size. And the means by which the patches do
that is to share binaries/applications across containers with custom
mounts/volumes. I don't think it is a good idea at all as it violates
encapsulation of the containers in general, regardless of whether we
use pods or not.

Dan


>
> Then each container would have no dependency on each other.
>
> In full blown Kubernetes cluster you might have puppet generate a
> configmap though and ship it to your main container directly. Thats
> another matter though. I think the example pod example above is still
> usable without k8s?
>
> Thanks,
> Kevin
> 
> From: Dan Prince [dpri...@redhat.com]
> Sent: Tuesday, November 27, 2018 10:10 AM
> To: OpenStack Development Mailing List (not for usage questions);
> openstack-disc...@lists.openstack.org
> Subject: Re: [openstack-dev] [TripleO][Edge] Reduce base layer of
> containers for security and size of images (maintenance) sakes
>
> On Tue, 2018-11-27 at 16:24 +0100, Bogdan Dobrelya wrote:
> > Changing the topic to follow the subject.
> >
> > [tl;dr] it's time to rearchitect container images to stop
> > incluiding
> > config-time only (puppet et al) bits, which are not needed runtime
> > and
> > pose security issues, like CVEs, to maintain daily.
>
> I think your assertion that we need to rearchitect the config images
> to
> container the puppet bits is incorrect here.
>
> After reviewing the patches you linked to below it appears that you
> are
> proposing we use --volumes-from to bind mount application binaries
> from
> one container into another. I don't believe this is a good pattern
> for
> containers. On baremetal if we followed the same pattern it would be
> like using an /nfs share to obtain access to binaries across the
> network to optimize local storage. Now... some people do this (like
> maybe high performance computing would launch an MPI job like this)
> but
> I don't think we should consider it best practice for our containers
> in
> TripleO.
>
> Each container should container its own binaries and libraries as
> much
> as possible. And while I do think we should be using --volumes-from
> more often in TripleO it would be for sharing *data* between
> containers, not binaries.
>
>
> > Background:
> > 1) For the Distributed Compute Node edge case, there is potentially
> > tens
> > of thousands of a single-compute-node remote edge sites connected
> > over
> > WAN to a single control plane, which is having high latency, like a
> > 100ms or so, and limited bandwith. Reducing the base layer size
> > becomes
> > a decent goal there. See the security background below.
>
> The reason we put Puppet into the base layer was in fact to prevent
> it
> from being downloaded multiple times. If we were to re-architect the
> image layers such that the child layers all contained their own
> copies
> of Puppet for example there would actually be a net increase in
> bandwidth and disk usage. So I would argue we are already addressing
> the goal of optimizing network and disk space.
>
> Moving it out of the base layer so 

Re: [openstack-dev] [tripleo] Workflows Squad changes

[Emilien Macchi]

On Wed, Nov 28, 2018 at 5:13 AM Jiri Tomasek  wrote:
[...]

> As a possible solution, I would like to propose Adriano as a core reviewer
> to tripleo-common and adding tripleo-ui cores right to +2 tripleo-common
> patches.
>
[...]

Not a member of the squad but +2 to the idea

Thanks for proposing,
-- 
Emilien Macchi
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO][Edge] Reduce base layer of containers for security and size of images (maintenance) sakes

[Sergii Golovatiuk]

Hi,
On Tue, Nov 27, 2018 at 7:13 PM Dan Prince  wrote:

> On Tue, 2018-11-27 at 16:24 +0100, Bogdan Dobrelya wrote:
> > Changing the topic to follow the subject.
> >
> > [tl;dr] it's time to rearchitect container images to stop incluiding
> > config-time only (puppet et al) bits, which are not needed runtime
> > and
> > pose security issues, like CVEs, to maintain daily.
>
> I think your assertion that we need to rearchitect the config images to
> container the puppet bits is incorrect here.
>
> After reviewing the patches you linked to below it appears that you are
> proposing we use --volumes-from to bind mount application binaries from
> one container into another. I don't believe this is a good pattern for
> containers. On baremetal if we followed the same pattern it would be
> like using an /nfs share to obtain access to binaries across the
> network to optimize local storage. Now... some people do this (like
> maybe high performance computing would launch an MPI job like this) but
> I don't think we should consider it best practice for our containers in
> TripleO.
>
> Each container should container its own binaries and libraries as much
> as possible. And while I do think we should be using --volumes-from
> more often in TripleO it would be for sharing *data* between
> containers, not binaries.
>
>
> >
> > Background:
> > 1) For the Distributed Compute Node edge case, there is potentially
> > tens
> > of thousands of a single-compute-node remote edge sites connected
> > over
> > WAN to a single control plane, which is having high latency, like a
> > 100ms or so, and limited bandwith. Reducing the base layer size
> > becomes
> > a decent goal there. See the security background below.
>
> The reason we put Puppet into the base layer was in fact to prevent it
> from being downloaded multiple times. If we were to re-architect the
> image layers such that the child layers all contained their own copies
> of Puppet for example there would actually be a net increase in
> bandwidth and disk usage. So I would argue we are already addressing
> the goal of optimizing network and disk space.
>
> Moving it out of the base layer so that you can patch it more often
> without disrupting other services is a valid concern. But addressing
> this concern while also preserving our definiation of a container (see
> above, a container should contain all of its binaries) is going to cost
> you something, namely disk and network space because Puppet would need
> to be duplicated in each child container.
>
> As Puppet is used to configure a majority of the services in TripleO
> having it in the base container makes most sense. And yes, if there are
> security patches for Puppet/Ruby those might result in a bunch of
> containers getting pushed. But let Docker layers take care of this I
> think... Don't try to solve things by constructing your own custom
> mounts and volumes to work around the issue.
>
>
> > 2) For a generic security (Day 2, maintenance) case, when
> > puppet/ruby/systemd/name-it gets a CVE fixed, the base layer has to
> > be
> > updated and all layers on top - to be rebuild, and all of those
> > layers,
> > to be re-fetched for cloud hosts and all containers to be
> > restarted...
> > And all of that because of some fixes that have nothing to OpenStack.
> > By
> > the remote edge sites as well, remember of "tens of thousands", high
> > latency and limited bandwith?..
> > 3) TripleO CI updates (including puppet*) packages in containers, not
> > in
> > a common base layer of those. So each a CI job has to update puppet*
> > and
> > its dependencies - ruby/systemd as well. Reducing numbers of packages
> > to
> > update for each container makes sense for CI as well.
> >
> > Implementation related:
> >
> > WIP patches [0],[1] for early review, uses a config "pod" approach,
> > does
> > not require to maintain a two sets of config vs runtime images.
> > Future
> > work: a) cronie requires systemd, we'd want to fix that also off the
> > base layer. b) rework to podman pods for docker-puppet.py instead of
> > --volumes-from a side car container (can't be backported for Queens
> > then, which is still nice to have a support for the Edge DCN case,
> > at
> > least downstream only perhaps).
> >
> > Some questions raised on IRC:
> >
> > Q: is having a service be able to configure itself really need to
> > involve a separate pod?
> > A: Highly likely yes, removing not-runtime things is a good idea and
> > pods is an established PaaS paradigm already. That will require some
> > changes in the architecture though (see the topic with WIP patches).
>
> I'm a little confused on this one. Are you suggesting that we have 2
> containers for each service? One with Puppet and one without?
>
> That is certainly possible, but to pull it off would likely require you
> to have things built like this:
>
>  |base container| --> |service container| --> |service container w/
> Puppet installed|
>
> The end result would be Puppet being 

Re: [Openstack] [OpenStack][cinder]Does cinder allocate disk to the volumes dynamically?

[Sean McGinnis]

On Wed, Nov 28, 2018 at 04:10:21PM +0330, Soheil Pourbafrani wrote:
> When I installed OpenStack using PackStack, the size of LVM group was 20G
> but I could create 18 volume each of which 20G size (so in the PackStack
> the Cinder allocate volumes dynamically), but installing OpenStack from
> repository, the Cinder allocate disk to volume statically because I have
> volume group in size 140G but I only can create 7 volume of size 20G. So
> How Can I configure Cinder to allocate disk dynamically?
> 

Not sure why you would be seeing a difference, but the default for Cinder is to
thin provision volumes.

This is controller through the following option in cinder.conf:

# Use thin provisioning for SAN volumes? (boolean value)
#san_thin_provision = true

Specific to LVM there is also the type option:

# Type of LVM volumes to deploy; (default, thin, or auto). Auto defaults to
# thin if thin is supported. (string value)
# Possible values:
# default - Thick-provisioned LVM.
# thin - Thin-provisioned LVM.
# auto - Defaults to thin when supported.
#lvm_type = auto

With the "auto" setting, that should thin provision the volumes unless you had
already configured the volume group ahead of time to not support it. That's
probably the most likely thing I can think of.

Sean

___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [openstack-dev] [tripleo] Let's improve upstream docs

[Natal Ngétal]

On Wed, Nov 28, 2018 at 4:19 PM Marios Andreou  wrote:
> great you are very welcome !
Thanks.

> not really, I mean "anything goes" as long as it's an improvement ( and the 
> usual review process will determine if it is or not :)  ). Could be as small 
> as typos or broken links/images, through to reorganising sections or even 
> bigger contributions like  completely new sections if you can and want. Take 
> a look at the existing patches that are on the bug for ideas
I see. I have made a first patch and I'm going to find what I can do
and continue to make code review.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] Let's improve upstream docs

[Marios Andreou]

On Wed, Nov 28, 2018 at 4:33 PM Natal Ngétal  wrote:

> On Tue, Nov 27, 2018 at 4:50 PM Marios Andreou  wrote:
> > as just mentioned in the tripleo weekly irc meeting [1] some of us are
> trying to make small weekly improvements to the tripleo docs  [2]. We are
> using this bug [3] for tracking and this effort is a result of some
> feedback during the recent Berlin summit.
> It's a good idea. The documentation of a project it's very important.
>
> > The general idea is 1 per week (or more if you can and want) -
> improvement/removal of stale content/identifying missing sections, or
> anything else you might care to propose. Please join us if you can, just
> add "Related-Bug: #1804642"  to your commit message
> I'm going to try to help you on this ticket. I started to make code
>

great you are very welcome !


> review. I would to know if you have more details about that. I mean,
> do you have examples of part able improved or something like that.
>
>
not really, I mean "anything goes" as long as it's an improvement ( and the
usual review process will determine if it is or not :)  ). Could be as
small as typos or broken links/images, through to reorganising sections or
even bigger contributions like  completely new sections if you can and
want. Take a look at the existing patches that are on the bug for ideas

thanks



> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack] unexpected distribution of compute instances in queens

[Jay Pipes]

On 11/28/2018 02:50 AM, Zufar Dhiyaulhaq wrote:

Hi,

Thank you. I am able to fix this issue by adding this configuration into 
nova configuration file in controller node.


driver=filter_scheduler


That's the default:

https://docs.openstack.org/ocata/config-reference/compute/config-options.html

So that was definitely not the solution to your problem.

My guess is that Sean's suggestion to randomize the allocation 
candidates fixed your issue.


Best,
-jay

___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [openstack-dev] [tripleo] Let's improve upstream docs

[Natal Ngétal]

On Tue, Nov 27, 2018 at 4:50 PM Marios Andreou  wrote:
> as just mentioned in the tripleo weekly irc meeting [1] some of us are trying 
> to make small weekly improvements to the tripleo docs  [2]. We are using this 
> bug [3] for tracking and this effort is a result of some feedback during the 
> recent Berlin summit.
It's a good idea. The documentation of a project it's very important.

> The general idea is 1 per week (or more if you can and want) - 
> improvement/removal of stale content/identifying missing sections, or 
> anything else you might care to propose. Please join us if you can, just add 
> "Related-Bug: #1804642"  to your commit message
I'm going to try to help you on this ticket. I started to make code
review. I would to know if you have more details about that. I mean,
do you have examples of part able improved or something like that.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO][Edge] Reduce base layer of containers for security and size of images (maintenance) sakes

[Dan Prince]

On Wed, 2018-11-28 at 15:12 +0100, Bogdan Dobrelya wrote:
> On 11/28/18 2:58 PM, Dan Prince wrote:
> > On Wed, 2018-11-28 at 12:45 +0100, Bogdan Dobrelya wrote:
> > > To follow up and explain the patches for code review:
> > > 
> > > The "header" patch https://review.openstack.org/620310 ->
> > > (requires)
> > > https://review.rdoproject.org/r/#/c/17534/, and also
> > > https://review.openstack.org/620061 -> (which in turn requires)
> > > https://review.openstack.org/619744 -> (Kolla change, the 1st to
> > > go)
> > > https://review.openstack.org/619736
> > 
> > This email was cross-posted to multiple lists and I think we may
> > have
> > lost some of the context in the process as the subject was changed.
> > 
> > Most of the suggestions and patches are about making our base
> > container(s) smaller in size. And the means by which the patches do
> > that is to share binaries/applications across containers with
> > custom
> > mounts/volumes. I've -2'd most of them. What concerns me however is
> > that some of the TripleO cores seemed open to this idea yesterday
> > on
> > IRC. Perhaps I've misread things but what you appear to be doing
> > here
> > is quite drastic I think we need to consider any of this carefully
> > before proceeding with any of it.
> > 
> > 
> > > Please also read the commit messages, I tried to explain all
> > > "Whys"
> > > very
> > > carefully. Just to sum up it here as well:
> > > 
> > > The current self-containing (config and runtime bits)
> > > architecture
> > > of
> > > containers badly affects:
> > > 
> > > * the size of the base layer and all containers images as an
> > > additional 300MB (adds an extra 30% of size).
> > 
> > You are accomplishing this by removing Puppet from the base
> > container,
> > but you are also creating another container in the process. This
> > would
> > still be required on all nodes as Puppet is our config tool. So you
> > would still be downloading some of this data anyways. Understood
> > your
> > reasons for doing this are that it avoids rebuilding all containers
> > when there is a change to any of these packages in the base
> > container.
> > What you are missing however is how often is it the case that
> > Puppet is
> > updated that something else in the base container isn't?
> 
> For CI jobs updating all containers, its quite an often to have
> changes 
> in openstack/tripleo puppet modules to pull in. IIUC, that
> automatically 
> picks up any updates for all of its dependencies and for the 
> dependencies of dependencies, and all that multiplied by a hundred
> of 
> total containers to get it updated. That is a *pain* we're used to
> have 
> these day for quite often timing out CI jobs... Ofc, the main cause
> is 
> delayed promotions though.

Regarding CI I made a separate suggestion on that below in that
rebuilding the base layer more often could be a good solution here. I
don't think the puppet-tripleo package is that large however so we
could just live with it.

> 
> For real deployments, I have no data for the cadence of minor updates
> in 
> puppet and tripleo & openstack modules for it, let's ask operators
> (as 
> we're happened to be in the merged openstack-discuss list)? For its 
> dependencies though, like systemd and ruby, I'm pretty sure it's
> quite 
> often to have CVEs fixed there. So I expect what "in the fields" 
> security fixes delivering for those might bring some unwanted hassle
> for 
> long-term maintenance of LTS releases. As Tengu noted on IRC:
> "well, between systemd, puppet and ruby, there are many security 
> concernes, almost every month... and also, what's the point keeping
> them 
> in runtime containers when they are useless?"

Reiterating again on previous points:

-I'd be fine removing systemd. But lets do it properly and not via 'rpm
-ev --nodeps'.
-Puppet and Ruby *are* required for configuration. We can certainly put
them in a separate container outside of the runtime service containers
but doing so would actually cost you much more space/bandwidth for each
service container. As both of these have to get downloaded to each node
anyway in order to generate config files with our current mechanisms
I'm not sure this buys you anything.

We are going in circles here I think

Dan

> 
> > I would wager that it is more rare than you'd think. Perhaps
> > looking at
> > the history of an OpenStack distribution would be a valid way to
> > assess
> > this more critically. Without this data to backup the numbers I'm
> > afraid what you are doing here falls into "pre-optimization"
> > territory
> > for me and I don't think the means used in the patches warrent the
> > benefits you mention here.
> > 
> > 
> > > * Edge cases, where we have containers images to be distributed,
> > > at
> > > least once to hit local registries, over high-latency and
> > > limited
> > > bandwith, highly unreliable WAN connections.
> > > * numbers of packages to update in CI for all containers for all
> > > services 

Re: [openstack-dev] [TripleO][Edge] Reduce base layer of containers for security and size of images (maintenance) sakes

[Bogdan Dobrelya]

On 11/28/18 2:58 PM, Dan Prince wrote:

On Wed, 2018-11-28 at 12:45 +0100, Bogdan Dobrelya wrote:

To follow up and explain the patches for code review:

The "header" patch https://review.openstack.org/620310 -> (requires)
https://review.rdoproject.org/r/#/c/17534/, and also
https://review.openstack.org/620061 -> (which in turn requires)
https://review.openstack.org/619744 -> (Kolla change, the 1st to go)
https://review.openstack.org/619736


This email was cross-posted to multiple lists and I think we may have
lost some of the context in the process as the subject was changed.

Most of the suggestions and patches are about making our base
container(s) smaller in size. And the means by which the patches do
that is to share binaries/applications across containers with custom
mounts/volumes. I've -2'd most of them. What concerns me however is
that some of the TripleO cores seemed open to this idea yesterday on
IRC. Perhaps I've misread things but what you appear to be doing here
is quite drastic I think we need to consider any of this carefully
before proceeding with any of it.




Please also read the commit messages, I tried to explain all "Whys"
very
carefully. Just to sum up it here as well:

The current self-containing (config and runtime bits) architecture
of
containers badly affects:

* the size of the base layer and all containers images as an
additional 300MB (adds an extra 30% of size).


You are accomplishing this by removing Puppet from the base container,
but you are also creating another container in the process. This would
still be required on all nodes as Puppet is our config tool. So you
would still be downloading some of this data anyways. Understood your
reasons for doing this are that it avoids rebuilding all containers
when there is a change to any of these packages in the base container.
What you are missing however is how often is it the case that Puppet is
updated that something else in the base container isn't?


For CI jobs updating all containers, its quite an often to have changes 
in openstack/tripleo puppet modules to pull in. IIUC, that automatically 
picks up any updates for all of its dependencies and for the 
dependencies of dependencies, and all that multiplied by a hundred of 
total containers to get it updated. That is a *pain* we're used to have 
these day for quite often timing out CI jobs... Ofc, the main cause is 
delayed promotions though.


For real deployments, I have no data for the cadence of minor updates in 
puppet and tripleo & openstack modules for it, let's ask operators (as 
we're happened to be in the merged openstack-discuss list)? For its 
dependencies though, like systemd and ruby, I'm pretty sure it's quite 
often to have CVEs fixed there. So I expect what "in the fields" 
security fixes delivering for those might bring some unwanted hassle for 
long-term maintenance of LTS releases. As Tengu noted on IRC:
"well, between systemd, puppet and ruby, there are many security 
concernes, almost every month... and also, what's the point keeping them 
in runtime containers when they are useless?"




I would wager that it is more rare than you'd think. Perhaps looking at
the history of an OpenStack distribution would be a valid way to assess
this more critically. Without this data to backup the numbers I'm
afraid what you are doing here falls into "pre-optimization" territory
for me and I don't think the means used in the patches warrent the
benefits you mention here.



* Edge cases, where we have containers images to be distributed, at
least once to hit local registries, over high-latency and limited
bandwith, highly unreliable WAN connections.
* numbers of packages to update in CI for all containers for all
services (CI jobs do not rebuild containers so each container gets
updated for those 300MB of extra size).


It would seem to me there are other ways to solve the CI containers
update problems. Rebuilding the base layer more often would solve this
right? If we always build our service containers off of a base layer
that is recent there should be no updates to the system/puppet packages
there in our CI pipelines.


* security and the surface of attacks, by introducing systemd et al
as
additional subjects for CVE fixes to maintain for all containers.


We aren't actually using systemd within our containers. I think those
packages are getting pulled in by an RPM dependency elsewhere. So
rather than using 'rpm -ev --nodeps' to remove it we could create a
sub-package for containers in those cases and install it instead. In
short rather than hack this to remove them why not pursue a proper
packaging fix?

In general I am a fan of getting things out of the base container we
don't need... so yeah lets do this. But lets do it properly.


* services uptime, by additional restarts of services related to
security maintanence of irrelevant to openstack components sitting
as a dead weight in containers images for ever.


Like I said 

Re: [openstack-dev] [TripleO][Edge] Reduce base layer of containers for security and size of images (maintenance) sakes

[Dan Prince]

On Wed, 2018-11-28 at 12:45 +0100, Bogdan Dobrelya wrote:
> To follow up and explain the patches for code review:
> 
> The "header" patch https://review.openstack.org/620310 -> (requires) 
> https://review.rdoproject.org/r/#/c/17534/, and also 
> https://review.openstack.org/620061 -> (which in turn requires)
> https://review.openstack.org/619744 -> (Kolla change, the 1st to go)
> https://review.openstack.org/619736

This email was cross-posted to multiple lists and I think we may have
lost some of the context in the process as the subject was changed.

Most of the suggestions and patches are about making our base
container(s) smaller in size. And the means by which the patches do
that is to share binaries/applications across containers with custom
mounts/volumes. I've -2'd most of them. What concerns me however is
that some of the TripleO cores seemed open to this idea yesterday on
IRC. Perhaps I've misread things but what you appear to be doing here
is quite drastic I think we need to consider any of this carefully
before proceeding with any of it.


> 
> Please also read the commit messages, I tried to explain all "Whys"
> very 
> carefully. Just to sum up it here as well:
> 
> The current self-containing (config and runtime bits) architecture
> of 
> containers badly affects:
> 
> * the size of the base layer and all containers images as an
>additional 300MB (adds an extra 30% of size).

You are accomplishing this by removing Puppet from the base container,
but you are also creating another container in the process. This would
still be required on all nodes as Puppet is our config tool. So you
would still be downloading some of this data anyways. Understood your
reasons for doing this are that it avoids rebuilding all containers
when there is a change to any of these packages in the base container.
What you are missing however is how often is it the case that Puppet is
updated that something else in the base container isn't?

I would wager that it is more rare than you'd think. Perhaps looking at
the history of an OpenStack distribution would be a valid way to assess
this more critically. Without this data to backup the numbers I'm
afraid what you are doing here falls into "pre-optimization" territory
for me and I don't think the means used in the patches warrent the
benefits you mention here.


> * Edge cases, where we have containers images to be distributed, at
>least once to hit local registries, over high-latency and limited
>bandwith, highly unreliable WAN connections.
> * numbers of packages to update in CI for all containers for all
>services (CI jobs do not rebuild containers so each container gets
>updated for those 300MB of extra size).

It would seem to me there are other ways to solve the CI containers
update problems. Rebuilding the base layer more often would solve this
right? If we always build our service containers off of a base layer
that is recent there should be no updates to the system/puppet packages
there in our CI pipelines.

> * security and the surface of attacks, by introducing systemd et al
> as
>additional subjects for CVE fixes to maintain for all containers.

We aren't actually using systemd within our containers. I think those
packages are getting pulled in by an RPM dependency elsewhere. So
rather than using 'rpm -ev --nodeps' to remove it we could create a
sub-package for containers in those cases and install it instead. In
short rather than hack this to remove them why not pursue a proper
packaging fix?

In general I am a fan of getting things out of the base container we
don't need... so yeah lets do this. But lets do it properly.

> * services uptime, by additional restarts of services related to
>security maintanence of irrelevant to openstack components sitting
>as a dead weight in containers images for ever.

Like I said above how often is it that these packages actually change
where something else in the base container doesn't? Perhaps we should
get more data here before blindly implementing a solution we aren't
sure really helps out in the real world.

> 
> On 11/27/18 4:08 PM, Bogdan Dobrelya wrote:
> > Changing the topic to follow the subject.
> > 
> > [tl;dr] it's time to rearchitect container images to stop
> > incluiding 
> > config-time only (puppet et al) bits, which are not needed runtime
> > and 
> > pose security issues, like CVEs, to maintain daily.
> > 
> > Background: 1) For the Distributed Compute Node edge case, there
> > is 
> > potentially tens of thousands of a single-compute-node remote edge
> > sites 
> > connected over WAN to a single control plane, which is having high 
> > latency, like a 100ms or so, and limited bandwith.
> > 2) For a generic security case,
> > 3) TripleO CI updates all
> > 
> > Challenge:
> > 
> > > Here is a related bug [1] and implementation [1] for that. PTAL
> > > folks!
> > > 
> > > [0] https://bugs.launchpad.net/tripleo/+bug/1804822
> > > [1] 
> > > 

Re: [openstack-dev] [TripleO][Edge][Kolla] Reduce base layer of containers for security and size of images (maintenance) sakes

[Bogdan Dobrelya]

Added Kolla tag as we all together might want to do something to that 
systemd included in containers via *multiple* package dependencies, like 
[0]. Ideally, that might be properly packaging all/some (like those 
names listed in [1]) of the places having it as a dependency, to stop 
doing that as of now it's Containers Time?.. As a temporary security 
band-aiding I was thinking of removing systemd via footers [1] as an 
extra layer added on top, but not sure that buys something good long-term.


[0] https://pastebin.com/RSaRsYgZ
[1] 
https://review.openstack.org/#/c/620310/2/container-images/tripleo_kolla_template_overrides.j2@680


On 11/28/18 12:45 PM, Bogdan Dobrelya wrote:

To follow up and explain the patches for code review:

The "header" patch https://review.openstack.org/620310 -> (requires) 
https://review.rdoproject.org/r/#/c/17534/, and also 
https://review.openstack.org/620061 -> (which in turn requires)

https://review.openstack.org/619744 -> (Kolla change, the 1st to go)
https://review.openstack.org/619736

Please also read the commit messages, I tried to explain all "Whys" very 
carefully. Just to sum up it here as well:


The current self-containing (config and runtime bits) architecture of 
containers badly affects:


* the size of the base layer and all containers images as an
   additional 300MB (adds an extra 30% of size).
* Edge cases, where we have containers images to be distributed, at
   least once to hit local registries, over high-latency and limited
   bandwith, highly unreliable WAN connections.
* numbers of packages to update in CI for all containers for all
   services (CI jobs do not rebuild containers so each container gets
   updated for those 300MB of extra size).
* security and the surface of attacks, by introducing systemd et al as
   additional subjects for CVE fixes to maintain for all containers.
* services uptime, by additional restarts of services related to
   security maintanence of irrelevant to openstack components sitting
   as a dead weight in containers images for ever.

On 11/27/18 4:08 PM, Bogdan Dobrelya wrote:

Changing the topic to follow the subject.

[tl;dr] it's time to rearchitect container images to stop incluiding 
config-time only (puppet et al) bits, which are not needed runtime and 
pose security issues, like CVEs, to maintain daily.


Background: 1) For the Distributed Compute Node edge case, there is 
potentially tens of thousands of a single-compute-node remote edge 
sites connected over WAN to a single control plane, which is having 
high latency, like a 100ms or so, and limited bandwith.

2) For a generic security case,
3) TripleO CI updates all

Challenge:


Here is a related bug [1] and implementation [1] for that. PTAL folks!

[0] https://bugs.launchpad.net/tripleo/+bug/1804822
[1] https://review.openstack.org/#/q/topic:base-container-reduction


Let's also think of removing puppet-tripleo from the base container.
It really brings the world-in (and yum updates in CI!) each job and 
each container!
So if we did so, we should then either install puppet-tripleo and co 
on the host and bind-mount it for the docker-puppet deployment task 
steps (bad idea IMO), OR use the magical --volumes-from 
 option to mount volumes from some 
"puppet-config" sidecar container inside each of the containers 
being launched by docker-puppet tooling.


On Wed, Oct 31, 2018 at 11:16 AM Harald Jensås redhat.com> wrote:

We add this to all images:

https://github.com/openstack/tripleo-common/blob/d35af75b0d8c4683a677660646e535cf972c98ef/container-images/tripleo_kolla_template_overrides.j2#L35 



/bin/sh -c yum -y install iproute iscsi-initiator-utils lvm2 python
socat sudo which openstack-tripleo-common-container-base rsync cronie
crudini openstack-selinux ansible python-shade puppet-tripleo python2-
kubernetes && yum clean all && rm -rf /var/cache/yum 276 MB
Is the additional 276 MB reasonable here?
openstack-selinux <- This package run relabling, does that kind of
touching the filesystem impact the size due to docker layers?

Also: python2-kubernetes is a fairly large package (18007990) do we use
that in every image? I don't see any tripleo related repos importing
from that when searching on Hound? The original commit message[1]
adding it states it is for future convenience.

On my undercloud we have 101 images, if we are downloading every 18 MB
per image thats almost 1.8 GB for a package we don't use? (I hope it's
not like this? With docker layers, we only download that 276 MB
transaction once? Or?)


[1] https://review.openstack.org/527927




--
Best regards,
Bogdan Dobrelya,
Irc #bogdando









--
Best regards,
Bogdan Dobrelya,
Irc #bogdando

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO][Edge] Reduce base layer of containers for security and size of images (maintenance) sakes

[Dan Prince]

On Wed, 2018-11-28 at 00:31 +, Fox, Kevin M wrote:
> The pod concept allows you to have one tool per container do one
> thing and do it well.
> 
> You can have a container for generating config, and another container
> for consuming it.
> 
> In a Kubernetes pod, if you still wanted to do puppet,
> you could have a pod that:
> 1. had an init container that ran puppet and dumped the resulting
> config to an emptyDir volume.
> 2. had your main container pull its config from the emptyDir volume.

We have basically implemented the same workflow in TripleO today. First
we execute Puppet in an "init container" (really just an ephemeral
container that generates the config files and then goes away). Then we
bind mount those configs into the service container.

One improvement we could make (which we aren't doing yet) is to use a
data container/volume to store the config files instead of using the
host. Sharing *data* within a 'pod' (set of containers, etc.) is
certainly a valid use of container volumes.

None of this is what we are really talking about in this thread though.
Most of the suggestions and patches are about making our base
container(s) smaller in size. And the means by which the patches do
that is to share binaries/applications across containers with custom
mounts/volumes. I don't think it is a good idea at all as it violates
encapsulation of the containers in general, regardless of whether we
use pods or not.

Dan


> 
> Then each container would have no dependency on each other.
> 
> In full blown Kubernetes cluster you might have puppet generate a
> configmap though and ship it to your main container directly. Thats
> another matter though. I think the example pod example above is still
> usable without k8s?
> 
> Thanks,
> Kevin
> 
> From: Dan Prince [dpri...@redhat.com]
> Sent: Tuesday, November 27, 2018 10:10 AM
> To: OpenStack Development Mailing List (not for usage questions); 
> openstack-disc...@lists.openstack.org
> Subject: Re: [openstack-dev] [TripleO][Edge] Reduce base layer of
> containers for security and size of images (maintenance) sakes
> 
> On Tue, 2018-11-27 at 16:24 +0100, Bogdan Dobrelya wrote:
> > Changing the topic to follow the subject.
> > 
> > [tl;dr] it's time to rearchitect container images to stop
> > incluiding
> > config-time only (puppet et al) bits, which are not needed runtime
> > and
> > pose security issues, like CVEs, to maintain daily.
> 
> I think your assertion that we need to rearchitect the config images
> to
> container the puppet bits is incorrect here.
> 
> After reviewing the patches you linked to below it appears that you
> are
> proposing we use --volumes-from to bind mount application binaries
> from
> one container into another. I don't believe this is a good pattern
> for
> containers. On baremetal if we followed the same pattern it would be
> like using an /nfs share to obtain access to binaries across the
> network to optimize local storage. Now... some people do this (like
> maybe high performance computing would launch an MPI job like this)
> but
> I don't think we should consider it best practice for our containers
> in
> TripleO.
> 
> Each container should container its own binaries and libraries as
> much
> as possible. And while I do think we should be using --volumes-from
> more often in TripleO it would be for sharing *data* between
> containers, not binaries.
> 
> 
> > Background:
> > 1) For the Distributed Compute Node edge case, there is potentially
> > tens
> > of thousands of a single-compute-node remote edge sites connected
> > over
> > WAN to a single control plane, which is having high latency, like a
> > 100ms or so, and limited bandwith. Reducing the base layer size
> > becomes
> > a decent goal there. See the security background below.
> 
> The reason we put Puppet into the base layer was in fact to prevent
> it
> from being downloaded multiple times. If we were to re-architect the
> image layers such that the child layers all contained their own
> copies
> of Puppet for example there would actually be a net increase in
> bandwidth and disk usage. So I would argue we are already addressing
> the goal of optimizing network and disk space.
> 
> Moving it out of the base layer so that you can patch it more often
> without disrupting other services is a valid concern. But addressing
> this concern while also preserving our definiation of a container
> (see
> above, a container should contain all of its binaries) is going to
> cost
> you something, namely disk and network space because Puppet would
> need
> to be duplicated in each child container.
> 
> As Puppet is used to configure a majority of the services in TripleO
> having it in the base container makes most sense. And yes, if there
> are
> security patches for Puppet/Ruby those might result in a bunch of
> containers getting pushed. But let Docker layers take care of this I
> think... Don't try to solve things by constructing your 

Re: [Openstack] [OpenStack][cinder]Does cinder allocate disk to the volumes dynamically?

[Soheil Pourbafrani]

When I installed OpenStack using PackStack, the size of LVM group was 20G
but I could create 18 volume each of which 20G size (so in the PackStack
the Cinder allocate volumes dynamically), but installing OpenStack from
repository, the Cinder allocate disk to volume statically because I have
volume group in size 140G but I only can create 7 volume of size 20G. So
How Can I configure Cinder to allocate disk dynamically?

On Wed, Nov 28, 2018 at 3:56 PM Soheil Pourbafrani 
wrote:

> Hi,
>
> I was wondering if the Cinder allocates disk to volumes statically or
> dynamically?
>
___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

[Openstack] [Nova] Instance creation problem

[Minjun Hong]

Hi. I'm setting up a Openstack system on the servers of my laboratory.
While I try to create an instance, a problem has occurred!
Instance creation was failed and it seems that libvirt failed to attaching
the vif to the instance.
When I create a virtual machine by using virsh tool (libvirt) manually,
there was no problem.

I add the logs as follows:

1. controller node

> "/var/log/nova/nova-conductor.log"
> 2018-11-28 21:18:13.033 2657 ERROR nova.scheduler.utils
> [req-291fdb2d-fa94-461c-9f5f-68d340791c77 3367829d9c004653bdc9102443bd4736
> 47270e4fb58045dc88b6f0f736286ffc - default default] [instance:
> 9c2d08f3-0680-4709-a64d-ae1729a11304] Error from last host: node1 (node
> node1): [u'Traceback (most recent call last):\n', u'  File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1840, in
> _do_build_and_run_instance\nfilter_properties, request_spec)\n', u'
> File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2120,
> in _build_and_run_instance\ninstance_uuid=instance.uuid,
> reason=six.text_type(e))\n', u"RescheduledException: Build of instance
> 9c2d08f3-0680-4709-a64d-ae1729a11304 was re-scheduled: internal error:
> libxenlight failed to create new domain 'instance-0008'\n"]
> 2018-11-28 21:18:13.033 2657 WARNING nova.scheduler.utils
> [req-291fdb2d-fa94-461c-9f5f-68d340791c77 3367829d9c004653bdc9102443bd4736
> 47270e4fb58045dc88b6f0f736286ffc - default default] Failed to
> compute_task_build_instances: Exceeded maximum number of retries. Exceeded
> max scheduling attempts 3 for instance
> 9c2d08f3-0680-4709-a64d-ae1729a11304. Last exception: internal error:
> libxenlight failed to create new domain 'instance-0008':
> MaxRetriesExceeded: Exceeded maximum number of retries. Exceeded max
> scheduling attempts 3 for instance 9c2d08f3-0680-4709-a64d-ae1729a11304.
> Last exception: internal error: libxenlight failed to create new domain
> 'instance-0008'
> 2018-11-28 21:18:13.034 2657 WARNING nova.scheduler.utils
> [req-291fdb2d-fa94-461c-9f5f-68d340791c77 3367829d9c004653bdc9102443bd4736
> 47270e4fb58045dc88b6f0f736286ffc - default default] [instance:
> 9c2d08f3-0680-4709-a64d-ae1729a11304] Setting instance to ERROR state.:
> MaxRetriesExceeded: Exceeded maximum number of retries. Exceeded max
> scheduling attempts 3 for instance 9c2d08f3-0680-4709-a64d-ae1729a11304.
> Last exception: internal error: libxenlight failed to create new domain
> 'instance-0008'
> 2018-11-28 21:18:13.067 2657 WARNING oslo_config.cfg
> [req-291fdb2d-fa94-461c-9f5f-68d340791c77 3367829d9c004653bdc9102443bd4736
> 47270e4fb58045dc88b6f0f736286ffc - default default] Option "url" from group
> "neutron" is deprecated for removal (Endpoint lookup uses the service
> catalog via common keystoneauth1 Adapter configuration options. In the
> current release, "url" will override this behavior, but will be ignored
> and/or removed in a future release. To achieve the same result, use the
> endpoint_override option instead.).  Its value may be silently ignored in
> the future.



> "/var/log/neutron/neutron-linuxbridge-agent.log"
> 2018-11-28 17:41:45.593 2476 INFO
> neutron.plugins.ml2.drivers.linuxbridge.agent.linuxbridge_neutron_agent [-]
> Interface mappings: {'provider': 'enp1s0f1'}
> 2018-11-28 17:41:45.593 2476 INFO
> neutron.plugins.ml2.drivers.linuxbridge.agent.linuxbridge_neutron_agent [-]
> Bridge mappings: {}
> 2018-11-28 17:41:45.624 2476 INFO
> neutron.plugins.ml2.drivers.linuxbridge.agent.linuxbridge_neutron_agent [-]
> Agent initialized successfully, now running...
> 2018-11-28 17:41:45.901 2476 INFO
> neutron.plugins.ml2.drivers.agent._common_agent
> [req-c447894c-9013-4bec-82e4-8b501414184d - - - - -] RPC agent_id:
> lba0369fa2714a
> 2018-11-28 17:41:45.907 2476 INFO neutron.agent.agent_extensions_manager
> [req-c447894c-9013-4bec-82e4-8b501414184d - - - - -] Loaded agent
> extensions: []
> 2018-11-28 17:41:46.121 2476 INFO
> neutron.plugins.ml2.drivers.agent._common_agent
> [req-c447894c-9013-4bec-82e4-8b501414184d - - - - -] Linux bridge agent
> Agent RPC Daemon Started!
> 2018-11-28 17:41:46.122 2476 INFO
> neutron.plugins.ml2.drivers.agent._common_agent
> [req-c447894c-9013-4bec-82e4-8b501414184d - - - - -] Linux bridge agent
> Agent out of sync with plugin!
> 2018-11-28 17:41:46.512 2476 INFO
> neutron.plugins.ml2.drivers.linuxbridge.agent.arp_protect
> [req-c447894c-9013-4bec-82e4-8b501414184d - - - - -] Clearing orphaned ARP
> spoofing entries for devices []
> 2018-11-28 17:41:47.020 2476 INFO
> neutron.plugins.ml2.drivers.linuxbridge.agent.arp_protect
> [req-c447894c-9013-4bec-82e4-8b501414184d - - - - -] Clearing orphaned ARP
> spoofing entries for devices []
> 2018-11-28 17:42:45.981 2476 ERROR
> neutron.plugins.ml2.drivers.agent._common_agent [-] Failed reporting
> state!: MessagingTimeout: Timed out waiting for a reply to message ID
> 20ef587240864120b878559ab821adbf
> 2018-11-28 17:42:45.981 2476 ERROR
> neutron.plugins.ml2.drivers.agent._common_agent 

[Openstack] [OpenStack][cinder]Does cinder allocate disk to the volumes dynamically?

[Soheil Pourbafrani]

Hi,

I was wondering if the Cinder allocates disk to volumes statically or
dynamically?
___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [openstack-dev] [TripleO][Edge] Reduce base layer of containers for security and size of images (maintenance) sakes

[Bogdan Dobrelya]

To follow up and explain the patches for code review:

The "header" patch https://review.openstack.org/620310 -> (requires) 
https://review.rdoproject.org/r/#/c/17534/, and also 
https://review.openstack.org/620061 -> (which in turn requires)

https://review.openstack.org/619744 -> (Kolla change, the 1st to go)
https://review.openstack.org/619736

Please also read the commit messages, I tried to explain all "Whys" very 
carefully. Just to sum up it here as well:


The current self-containing (config and runtime bits) architecture of 
containers badly affects:


* the size of the base layer and all containers images as an
  additional 300MB (adds an extra 30% of size).
* Edge cases, where we have containers images to be distributed, at
  least once to hit local registries, over high-latency and limited
  bandwith, highly unreliable WAN connections.
* numbers of packages to update in CI for all containers for all
  services (CI jobs do not rebuild containers so each container gets
  updated for those 300MB of extra size).
* security and the surface of attacks, by introducing systemd et al as
  additional subjects for CVE fixes to maintain for all containers.
* services uptime, by additional restarts of services related to
  security maintanence of irrelevant to openstack components sitting
  as a dead weight in containers images for ever.

On 11/27/18 4:08 PM, Bogdan Dobrelya wrote:

Changing the topic to follow the subject.

[tl;dr] it's time to rearchitect container images to stop incluiding 
config-time only (puppet et al) bits, which are not needed runtime and 
pose security issues, like CVEs, to maintain daily.


Background: 1) For the Distributed Compute Node edge case, there is 
potentially tens of thousands of a single-compute-node remote edge sites 
connected over WAN to a single control plane, which is having high 
latency, like a 100ms or so, and limited bandwith.

2) For a generic security case,
3) TripleO CI updates all

Challenge:


Here is a related bug [1] and implementation [1] for that. PTAL folks!

[0] https://bugs.launchpad.net/tripleo/+bug/1804822
[1] https://review.openstack.org/#/q/topic:base-container-reduction


Let's also think of removing puppet-tripleo from the base container.
It really brings the world-in (and yum updates in CI!) each job and 
each container!
So if we did so, we should then either install puppet-tripleo and co 
on the host and bind-mount it for the docker-puppet deployment task 
steps (bad idea IMO), OR use the magical --volumes-from 
 option to mount volumes from some 
"puppet-config" sidecar container inside each of the containers being 
launched by docker-puppet tooling.


On Wed, Oct 31, 2018 at 11:16 AM Harald Jensås  
wrote:

We add this to all images:

https://github.com/openstack/tripleo-common/blob/d35af75b0d8c4683a677660646e535cf972c98ef/container-images/tripleo_kolla_template_overrides.j2#L35 



/bin/sh -c yum -y install iproute iscsi-initiator-utils lvm2 python
socat sudo which openstack-tripleo-common-container-base rsync cronie
crudini openstack-selinux ansible python-shade puppet-tripleo python2-
kubernetes && yum clean all && rm -rf /var/cache/yum 276 MB
Is the additional 276 MB reasonable here?
openstack-selinux <- This package run relabling, does that kind of
touching the filesystem impact the size due to docker layers?

Also: python2-kubernetes is a fairly large package (18007990) do we use
that in every image? I don't see any tripleo related repos importing
from that when searching on Hound? The original commit message[1]
adding it states it is for future convenience.

On my undercloud we have 101 images, if we are downloading every 18 MB
per image thats almost 1.8 GB for a package we don't use? (I hope it's
not like this? With docker layers, we only download that 276 MB
transaction once? Or?)


[1] https://review.openstack.org/527927




--
Best regards,
Bogdan Dobrelya,
Irc #bogdando






--
Best regards,
Bogdan Dobrelya,
Irc #bogdando

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack-operators] Fwd: Nova hypervisor uuid

[Ignazio Cassano]

Hi Matt, sorry but I lost your answer and Gianpiero forwarded it to me.
I am sure kvm nodes names are note changed.
Tables where uuid are duplicated are:
dataresource_providers in nova_api db
compute_nodes in nova db
Regards
Ignazio

Il 28/Nov/2018 11:09 AM, "Gianpiero Ardissono"  ha
scritto:

>
> -- Forwarded message -
> From: Matt Riedemann 
> Date: mar 27 nov 2018, 19:03
> Subject: Re: [Openstack-operators] Nova hypervisor uuid
> To: 
>
>
> On 11/27/2018 11:32 AM, Ignazio Cassano wrote:
> > Hi  All,
> > Please anyone know where hypervisor uuid is retrived?
> > Sometime updating kmv nodes with yum update it changes and in nova
> > database 2 uuids are assigned to the same node.
> > regards
> > Ignazio
> >
> >
> >
> >
> > ___
> > OpenStack-operators mailing list
> > OpenStack-operators@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
> >
>
> To be clear, do you mean the computes_nodes.uuid column value in the
> cell database? Which is also used for the GET /os-hypervisors response
> 'id' value if using microversion >= 2.53. If so, that is generated
> randomly* when the compute_nodes table record is created:
>
>
> https://github.com/openstack/nova/blob/8545ba2af7476e0884b5e7fb90965bef92d605bc/nova/compute/resource_tracker.py#L588
>
>
> https://github.com/openstack/nova/blob/8545ba2af7476e0884b5e7fb90965bef92d605bc/nova/objects/compute_node.py#L312
>
> When you hit this problem, are you sure the hostname on the compute host
> is not changing? Because when nova-compute starts up, it should look for
> the existing compute node record by host name and node name, which for
> the libvirt driver should be the same. That lookup code is here:
>
>
> https://github.com/openstack/nova/blob/8545ba2af7476e0884b5e7fb90965bef92d605bc/nova/compute/resource_tracker.py#L815
>
> So the only way nova-compute should create a new compute_nodes table
> record for the same host is if the host/node name changes during the
> upgrade. Is the deleted value in the database the same (0) for both of
> those records?
>
> * The exception to this is for the ironic driver which re-uses the
> ironic node uuid as of this change:
> https://review.openstack.org/#/c/571535/
>
> --
>
> Thanks,
>
> Matt
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

[openstack-dev] [tripleo] Workflows Squad changes

[Jiri Tomasek]

Hi all,

Recently, the workflows squad has been reorganized and people from the
squad are joining different squads. I would like to discuss how we are
going to adjust to this situation to make sure that tripleo-common
development is not going to be blocked in terms of feature work and reviews.

With this change, most of the tripleo-common maintenance work goes
naturally to UI & Validations squad as CLI and GUI are the consumers of the
API provided by tripleo-common. Adriano Petrich from workflows squad has
joined UI squad to take on this work.

As a possible solution, I would like to propose Adriano as a core reviewer
to tripleo-common and adding tripleo-ui cores right to +2 tripleo-common
patches.

It would be great to hear opinions especially former members of Workflows
squad and regular contributors to tripleo-common on these changes and in
general on how to establish regular reviews and maintenance to ensure that
tripleo-common codebase is moved towards converging the CLI and GUI
deployment workflow.

Thanks
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev