subject:"\[openstack\-dev\] \[tc\]\[appcat\] The future of the App Catalog"

Excerpts from Christopher Aedo's message of 2017-03-10 19:30:18 -0800:
> On Fri, Mar 10, 2017 at 6:20 PM, Clint Byrum  wrote:
> > Excerpts from Fox, Kevin M's message of 2017-03-10 23:45:06 +:
> >> So, this is the kind of thinking I'm talking about... OpenStack today is
> >> more then just IaaS in the tent. Trove (DBaaS), Sahara (Hadoop,Spark,etc
> >> aaS), Zaqar (Messaging aaS) and many more services. But they seem to be
> >> treated as second class citizens, as they are not "IaaS".
> >>
> >
> > It's not that they're second class citizens. It's that their community
> > is smaller by count of users, operators, and developers. This should not
> > come as a surprise, because the lowest common denominator in any user
> > base will always receive more attention.
> >
> >> > Why should it strive to be anything except an excellent building block
> >> for other technologies?
> >>
> >> You misinterpret my statement. I'm in full agreement with you. The
> >> above services should be excellent building blocks too, but are suffering
> >> from lack of support from the IaaS layer. They deserve the ability to
> >> be excellent too, but need support/vision from the greater community
> >> that hasn't been forthcoming.
> >>
> >
> > You say it like there's some over arching plan to suppress parts of the
> > community and there's a pack of disgruntled developers who just can't
> > seem to get OpenStack to work for Trove/Sahara/AppCatalog/etc.
> >
> > We all have different reasons for contributing in the way we do.  Clearly,
> > not as many people contribute to the Trove story as do the pure VM-on-nova
> > story.
> >
> >> I agree with you, we should embrace the container folks and not treat
> >> them as separate. I think thats critical if we want to allow things
> >> like Sahara or Trove to really fulfil their potential. This is the path
> >> towards being an OpenSource AWS competitor, not just for being able to
> >> request vm's in a cloudy way.
> >>
> >> I think that looks something like:
> >> OpenStack Advanced Service (trove, sahara, etc) -> Kubernetes ->
> >> Nova VM or Ironic Bare Metal.
> >>
> >
> > That's a great idea. However, AFAICT, Nova is _not_ standing in Trove,
> > Sahara, or anyone else's way from doing this. Seriously, try it. I'm sure
> > it will work.  And in so doing, you will undoubtedly run into friction
> > from the APIs. But unless you can describe that _now_, you have to go try
> > it and tell us what broke first. And then you can likely submit feature
> > work to nova/neutron/cinder to make it better. I don't see anything in
> > the current trajectory of OpenStack that makes this hard. Why not just do
> > it? The way you ask, it's like you have a team of developers just sitting
> > around shaving yaks waiting for an important OpenStack development task.
> >
> > The real question is why aren't Murano, Trove and Sahara in most current
> > deployments? My guess is that it's because most of our current users
> > don't feel they need it. Until they do, Trove and Sahara will not be
> > priorities. If you want them to be priorities _pay somebody to make them
> > a priority_.
> 
> This particular point really caught my attention.  You imply that
> these additional services are not widely deployed because _users_
> don't want them.  The fact is most users are completely unaware of
> them because these services require the operator of the cloud to
> support them.  In fact they often require the operator of the cloud to
> support them from the initial deployment, as these services (and
> *most* OpenStack services) are frighteningly difficult to add to an
> already deployed cloud without downtime and high risk of associated
> issues.
> 
> I think it's unfair to claim these services are unpopular because
> users aren't asking for them when it's likely users aren't even aware
> of them (do OVH, Vexxhost, Dreamhost, Raskspace or others provide a
> user-facing list of potential OpenStack services with a voting option?
>  Not that I've ever seen!)
> 
> I bring this up to point out how much more popular ALL of these
> services would be if the _users_ were able to enable them without
> requiring operator intervention and support.
> 
> Based on our current architecture, it's nearly impossible for a new
> project to be deployed on a cloud without cloud-level admin
> privileges.  Additionally almost none of the projects could even work
> this way (with Rally being a notable exception).  I guess I'm kicking
> this dead horse because for a long time I've argued we need to back
> away from the tightly coupled nature of all the projects, but
> (speaking of horses) it seems that horse is already out of the barn.
> (I really wish I could work in one more proverb dealing with horses
> but it's getting late on a Friday so I'll stop now.)
> 

I see your point, and believe it is valid.

However, we do have something like a voting system in the market
economy. The users may not say "I want Trove". But they may very well say
"I don

Re: [openstack-dev] [tc][appcat] The future of the App Catalog

2017-03-10 Thread Christopher Aedo

On Fri, Mar 10, 2017 at 6:20 PM, Clint Byrum  wrote:
> Excerpts from Fox, Kevin M's message of 2017-03-10 23:45:06 +:
>> So, this is the kind of thinking I'm talking about... OpenStack today is
>> more then just IaaS in the tent. Trove (DBaaS), Sahara (Hadoop,Spark,etc
>> aaS), Zaqar (Messaging aaS) and many more services. But they seem to be
>> treated as second class citizens, as they are not "IaaS".
>>
>
> It's not that they're second class citizens. It's that their community
> is smaller by count of users, operators, and developers. This should not
> come as a surprise, because the lowest common denominator in any user
> base will always receive more attention.
>
>> > Why should it strive to be anything except an excellent building block
>> for other technologies?
>>
>> You misinterpret my statement. I'm in full agreement with you. The
>> above services should be excellent building blocks too, but are suffering
>> from lack of support from the IaaS layer. They deserve the ability to
>> be excellent too, but need support/vision from the greater community
>> that hasn't been forthcoming.
>>
>
> You say it like there's some over arching plan to suppress parts of the
> community and there's a pack of disgruntled developers who just can't
> seem to get OpenStack to work for Trove/Sahara/AppCatalog/etc.
>
> We all have different reasons for contributing in the way we do.  Clearly,
> not as many people contribute to the Trove story as do the pure VM-on-nova
> story.
>
>> I agree with you, we should embrace the container folks and not treat
>> them as separate. I think thats critical if we want to allow things
>> like Sahara or Trove to really fulfil their potential. This is the path
>> towards being an OpenSource AWS competitor, not just for being able to
>> request vm's in a cloudy way.
>>
>> I think that looks something like:
>> OpenStack Advanced Service (trove, sahara, etc) -> Kubernetes ->
>> Nova VM or Ironic Bare Metal.
>>
>
> That's a great idea. However, AFAICT, Nova is _not_ standing in Trove,
> Sahara, or anyone else's way from doing this. Seriously, try it. I'm sure
> it will work.  And in so doing, you will undoubtedly run into friction
> from the APIs. But unless you can describe that _now_, you have to go try
> it and tell us what broke first. And then you can likely submit feature
> work to nova/neutron/cinder to make it better. I don't see anything in
> the current trajectory of OpenStack that makes this hard. Why not just do
> it? The way you ask, it's like you have a team of developers just sitting
> around shaving yaks waiting for an important OpenStack development task.
>
> The real question is why aren't Murano, Trove and Sahara in most current
> deployments? My guess is that it's because most of our current users
> don't feel they need it. Until they do, Trove and Sahara will not be
> priorities. If you want them to be priorities _pay somebody to make them
> a priority_.

This particular point really caught my attention.  You imply that
these additional services are not widely deployed because _users_
don't want them.  The fact is most users are completely unaware of
them because these services require the operator of the cloud to
support them.  In fact they often require the operator of the cloud to
support them from the initial deployment, as these services (and
*most* OpenStack services) are frighteningly difficult to add to an
already deployed cloud without downtime and high risk of associated
issues.

I think it's unfair to claim these services are unpopular because
users aren't asking for them when it's likely users aren't even aware
of them (do OVH, Vexxhost, Dreamhost, Raskspace or others provide a
user-facing list of potential OpenStack services with a voting option?
 Not that I've ever seen!)

I bring this up to point out how much more popular ALL of these
services would be if the _users_ were able to enable them without
requiring operator intervention and support.

Based on our current architecture, it's nearly impossible for a new
project to be deployed on a cloud without cloud-level admin
privileges.  Additionally almost none of the projects could even work
this way (with Rally being a notable exception).  I guess I'm kicking
this dead horse because for a long time I've argued we need to back
away from the tightly coupled nature of all the projects, but
(speaking of horses) it seems that horse is already out of the barn.
(I really wish I could work in one more proverb dealing with horses
but it's getting late on a Friday so I'll stop now.)

-Christopher

>> Not what we have today:
>> OpenStack Advanced Service -> Nova VM or Ironic Bare Metal
>>
>> due to the focus on the api's of VM's being only for IaaS and not for
>> actually running cloud software on.
>>
>
> The above is an unfounded and unsupported claim. What _exactly_ would
> you want Nova/Neutron/Cinder's API to do differently to support "cloud
> software" better? Why is everyone dodging that question? Name one

Re: [openstack-dev] [tc][appcat] The future of the App Catalog

Excerpts from Zane Bitter's message of 2017-03-10 19:09:42 -0500:
> On 10/03/17 12:27, Monty Taylor wrote:
> > On 03/10/2017 10:59 AM, Clint Byrum wrote:
> You may be familiar with the Kuryr project, which integrates Kubernetes 
> deployments made by Magnum with Neutron networking so that other Nova 
> servers can talk directly to the containers and other fun stuff. IMHO 
> it's exactly the kind of thing OpenStack should be doing to make users' 
> lives better, and give a compelling reason to install k8s on top of 
> OpenStack instead of on bare metal.
> 
> So here's a fun thing I learned at the PTG: according to the Magnum 
> folks, the main thing preventing them from fully adopting Kuryr is that 
> the k8s application servers, provisioned with Nova, need to make API 
> calls to Neutron to set up the ports as containers move around. And 
> there's no secure way to give Keystone authentication credentials to an 
> application server to do what it needs - and, especially, to do *only* 
> what it needs.
> 
> http://lists.openstack.org/pipermail/openstack-dev/2016-October/105304.html
> 
> Keystone devs did agree in back in Austin that when they rejigger the 
> default policy files it will done in such a way as to make the 
> authorisation component of this feasible (by requiring a specific 
> reader/writer role, not just membership of the project, to access APIs), 
> but that change hasn't happened yet AFAIK. I suspect that it isn't their 
> top priority. Kevin has been campaigning for *years* to get Nova to 
> provide a secure way to inject credentials into a server in the same way 
> that this is built in to EC2, GCE and (I assume but haven't checked) 
> Azure. And they turned him down flat every time saying that this was not 
> Nova's problem.
> 
> Sorry, but if OpenStack isn't a good, secure platform for running 
> Kubernetes on then that is a HAIR ON FIRE-level *existential* problem in 
> 2017.
> 

You're very right on this. And kuryr not working the way it should is _a
disaster_. I was hoping it had worked itself out by now.

I've argued against certain aspects of instance-users in the past (and
for others). I think it might be worth it to take another shot at
actually writing up a spec again.

In the mean time, we could always have shade inject instance-user
creds... 

> We can't place too much blame on individual projects though, because I 
> believe the main reason this doesn't Just Work already is that there has 
> been an unspoken consensus that we needed to listen to users like you 
> but not to users like Kevin, and the elected leaders of our community 
> have done nothing to either disavow or officially adopt that consensus. 
> We _urgently_ need to decide if that's what we actually want and make 
> sure it is prominently documented so that both users and developers know 
> what's what.
> 
> FWIW I'm certain you must have hit this same issue in infra - probably 
> you were able to use pre-signed Swift URLs when uploading log files to 
> avoid needing credentials on servers allocated by nodepool? That's 
> great, but not every API in OpenStack has a pre-signed URL facility, and 
> nor should they have to.
> 

Indeed, that's how infra nodes store logs in swift.

> (BTW I proposed a workaround for Magnum/Kuryr at the PTG by using a 
> pre-signed Zaqar URL with a subscription triggering a Mistral workflow, 
> and I've started working on a POC.)
> 

What triggers the boot to kick over the bucket full of golf balls
though?

> > Considering computers as some-how inherently 'better' or 'worse' than
> > some of the 'cloud-native' concepts is hog wash. Different users have
> > different needs. As Clint points out - kubernetes needs to run
> > _somewhere_. CloudFoundry needs to run _somewhere_. So those are at
> > least two other potential users who are not me and my collection of
> > things I want to run that want to run in computers.
> 
> I think we might be starting to talk about different ideas. The whole 
> VMs vs. containers fight _is_ hogwash. You're right to call it out as 
> such. We hear far too much about it, and it's totally unfair to the 
> folks who work on the VM side. But that isn't what this discussion is about.
> 
> Google has done everyone a minor disservice by appropriating the term 
> "cloud-native" and using it in a context such that it's effectively been 
> redefined to mean "containers instead of VMs". I've personally stopped 
> using the term because it's more likely to generate confusion than clarity.
> 
> What "cloud-native" used to mean to me was an application that knows 
> it's running in the cloud, and uses the cloud's APIs. As opposed to 
> applications that could just as easily be running in a VPS or on bare 
> metal, but happen to be running in a VM provisioned by Nova.
> 

+1 for "apps that know they're in the cloud", and further apps that know
how to talk to their cloud.

And also +1 for listening to folks who want a little more help in
interacting with their cloud from

Re: [openstack-dev] [tc][appcat] The future of the App Catalog

Excerpts from Fox, Kevin M's message of 2017-03-10 23:45:06 +:
> So, this is the kind of thinking I'm talking about... OpenStack today is
> more then just IaaS in the tent. Trove (DBaaS), Sahara (Hadoop,Spark,etc
> aaS), Zaqar (Messaging aaS) and many more services. But they seem to be
> treated as second class citizens, as they are not "IaaS".
> 

It's not that they're second class citizens. It's that their community
is smaller by count of users, operators, and developers. This should not
come as a surprise, because the lowest common denominator in any user
base will always receive more attention.

> > Why should it strive to be anything except an excellent building block
> for other technologies?
> 
> You misinterpret my statement. I'm in full agreement with you. The
> above services should be excellent building blocks too, but are suffering
> from lack of support from the IaaS layer. They deserve the ability to
> be excellent too, but need support/vision from the greater community
> that hasn't been forthcoming.
> 

You say it like there's some over arching plan to suppress parts of the
community and there's a pack of disgruntled developers who just can't
seem to get OpenStack to work for Trove/Sahara/AppCatalog/etc.

We all have different reasons for contributing in the way we do.  Clearly,
not as many people contribute to the Trove story as do the pure VM-on-nova
story.

> I agree with you, we should embrace the container folks and not treat
> them as separate. I think thats critical if we want to allow things
> like Sahara or Trove to really fulfil their potential. This is the path
> towards being an OpenSource AWS competitor, not just for being able to
> request vm's in a cloudy way.
> 
> I think that looks something like:
> OpenStack Advanced Service (trove, sahara, etc) -> Kubernetes ->
> Nova VM or Ironic Bare Metal.
> 

That's a great idea. However, AFAICT, Nova is _not_ standing in Trove,
Sahara, or anyone else's way from doing this. Seriously, try it. I'm sure
it will work.  And in so doing, you will undoubtedly run into friction
from the APIs. But unless you can describe that _now_, you have to go try
it and tell us what broke first. And then you can likely submit feature
work to nova/neutron/cinder to make it better. I don't see anything in
the current trajectory of OpenStack that makes this hard. Why not just do
it? The way you ask, it's like you have a team of developers just sitting
around shaving yaks waiting for an important OpenStack development task.

The real question is why aren't Murano, Trove and Sahara in most current
deployments? My guess is that it's because most of our current users
don't feel they need it. Until they do, Trove and Sahara will not be
priorities. If you want them to be priorities _pay somebody to make them
a priority_.

> Not what we have today:
> OpenStack Advanced Service -> Nova VM or Ironic Bare Metal
> 
> due to the focus on the api's of VM's being only for IaaS and not for
> actually running cloud software on.
> 

The above is an unfounded and unsupported claim. What _exactly_ would
you want Nova/Neutron/Cinder's API to do differently to support "cloud
software" better? Why is everyone dodging that question? Name one
specific change and how it would actually be consumed.

I personally think it's just the general frustration that comes at
this stage of maturity of a project. There's a contraction of hype,
lower investment in general, and everyone is suppressing their panic
and trying to reason about what we can do to make it better, but nobody
actually knows. Meanwhile, our users and operators need us to keep
making things better. Some are even _paying us_ to make things better.

> I'm sorry if that sounds a bit confusing. Its hard to
> explain. I can try and elaborate if you want. Zane's
> posting here does help explain it a little: >
> http://www.zerobanana.com/archive/2015/04/24#a-vision-for-openstack
> 

Honest, I don't understand the reality that Zane's vision is driving at in
that post. More Zaqar? Why not just do that? What's standing in the way?

> The other alternative is to clearly define OpenStack to be just IaaS,
> and kick all the non IaaS stuff out of the tent. (I do not prefer
> this). It will hurt in the short term but long tern could be better
> for those projects then the current status quo as new opportunities for
> switching base dependencies will present themselves and no longer will
> be hamstrung by those that feel their use case isn't important or think
> that the existing api's are "good enough" to handle the use case.
> 

Why would we kick out perfectly healthy pieces of software that want
to be OpenStack-native? Nobody is saying that IaaS clouds aren't well
complimented by native higher level projects. But in the app catalog
case, there's little consumption, and waning contribution, so it's worth
considering whether its continued maintenance and existence is a drain
on the overall community. Sounds like it is, and we'll proba

Re: [openstack-dev] [tc][appcat] The future of the App Catalog


On 10/03/17 14:30, Jay Pipes wrote:

On 03/10/2017 01:37 PM, Zane Bitter wrote:

On 09/03/17 23:57, Renat Akhmerov wrote:

And this whole discussion is taking me to the question: is there really
any officially accepted strategy for OpenStack for 1, 3, 5 years? Is
there any ultimate community goal we’re moving to regardless of
underlying technologies (containers, virtualization etc.)? I know we’re
now considering various community goals like transition to Python 3.5
etc. but these goals don’t tell anything about our future as an IT
ecosystem from user perspective. I may assume that I’m just not aware of
it. I’d be glad if it was true. I’m eager to know the answers for these
questions.


Me too! There was one effort started by John Garbutt in December, a
technical vision for OpenStack in the form of a TC resolution in the
governance repo:

https://review.openstack.org/#/c/401226/

I wasn't a fan of the draft content, but I believe his intention was to
seed it with a formalised version of our de-facto historical position
(tl;dr legacy apps from the 90s). As long as that's a starting point for
the discussion and not a conclusion then I think this was a valuable
contribution. I commented with a non-exhaustive list of things that I
would expect to see at least debated in a vision for a cloud computing
platform, which I will reproduce here since it's relevant to this thread:

* Infinite scaling - the ability in principle to scale from zero to an
arbitrarily large number of users without rewriting your application
(e.g. if your application can store one file in Swift then there's no
theoretical limit to how many it can store. c.f. Cinder where at some
point you'd have to start juggling multiple volumes.)
* Granularity of allocation - pay only for the resources you actually
use, rather than to allocate a chunk that you may or may not be using
(so Nova->containers->FaaS, Cinder->Swift, Trove->??? [RIP MagnetoDB],
&c.)
* Full control of infrastructure - notwithstanding the above, maintain
Nova/Cinder/Neutron/Trove/&c. so that legacy applications, highly
specialised applications, and higher-level services like PaaS can make
fully customised use of the virtual infrastructure.
* Hardware virtualisation - make anything that might typically be done
in hardware available in a multi-tenant software-defined environment:
servers, routers, load balancers, firewalls, video codecs, GPGPUs,
FPGAs...
* Built-in reliability - don't require even the smallest apps to have 3
VMs + a cluster manager to enforce any reliability guarantees; provide
those guarantees using multi-tenant services that efficiently share
resources between applications (see also: Infinite scaling, Granularity
of allocation).
* Application control - (securely) give applications control over their
own infrastructure, so that no part of the application needs to reside
outside of the cloud.
* Integration - cloud services that effectively form part of the user's
application can communicate amongst themselves, where appropriate,
without the need for client-side glue (see also: Built-in reliability).
* Interoperability - the same applications can be deployed on a variety
of private and public OpenStack clouds.


Those are all interesting technical concepts to think about and discuss.
However, what Kevin said originally in his response about the OpenStack
community needing to decide what exactly it *is* and what scope
OpenStack should pursue, is the real foundational question that needs to
be addressed. Until it is, none of the rest of these topics have much
contextual relevance and are just a distraction, IMHO.


Hey Jay,
I respect your opinion here, and I agree that's the question we need to 
answer. TBH I'm happy any way we can get to an answer.


I suspect however, that approaching it directly will not prove feasible 
due to a lack of shared terminology. When everyone has a different idea 
in their head (and AFAICT they do) of what "IaaS" means, discussions 
devolve quite quickly into meaninglessness.


It seems to me that having the conversation at a different level of 
abstraction, that of the technical principles we think the solution 
should embody, could potentially be more productive. I believe it's 
fairly easy to judge what is in scope (including for services we haven't 
imagined yet) by referring back to principles at this level of 
abstraction if we can agree on them first.


I'm open to other approaches though.

cheers,
Zane.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][appcat] The future of the App Catalog

On 10/03/17 17:43, Clint Byrum wrote:

Excerpts from Fox, Kevin M's message of 2017-03-10 20:47:55 +:

Just because the market hasn't immediately shifted from IaaS to containers doesn't mean it won't
happen eventually, and that google's wrong in their push for containers over IaaS. It took a long
time (still ongoing) to move from physical machines to vm's. The same parallel will happen with
containers I believe. We're at least a few years out from it becoming more common place. That
doesn't mean folks can assume it will always be the way it is, and the "market has
spoken". "The only constants are death and taxes." :)

Your right in the assertion k8s needs a place to run, but that doesn't
necessarily mean OpenStack, unless we help integrate it and make it a great
place to run it.

I'm glad the IaaS stuff in OpenStack suits your needs. Thats great. It hasn't
served a great number of users needs though, and has been at least misleading
folks into believing it will for a number of years. If we want it to just be an
IaaS, lets free up those users to move on elsewhere.

Does OpenStack intend just to be an IaaS in a few years or something else?

I'm not a fan of describing the more limited use case as "just IaaS".
It's all infrastructure; the question for me is whom it's providing a
service to: the application running on that infrastructure, or only the
(human) operator of that application?

Why should it strive to be anything except an excellent building block
for other technologies?

Wrong question IMO. The right question is, what would make it an
excellent building block?

We need to set the bar higher than "same as bare metal ('apps will work
just like any other places') but you also get to manage an OpenStack
deployment too".

See my reply to Monty for details on some ways that OpenStack isn't as
excellent a building block as it ought to be.

By happy coincidence, the same features that would make OpenStack a
better building block for "other technologies" are in many cases also
what would make it a better place to run other applications of the type
that Kevin's users want to write. Since the "other technologies" are
themselves applications from OpenStack's perspective.

cheers,
Zane.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][appcat] The future of the App Catalog


On 10/03/17 12:27, Monty Taylor wrote:

On 03/10/2017 10:59 AM, Clint Byrum wrote:

I'm curious what you (Josh) or Zane would change too.
Containers/apps/kubes/etc. have to run on computers with storage and
networks. OpenStack provides a pretty rich set of features for giving
users computers with storage on networks, and operators a way to manage
those. So I fail to see how that is svn to "cloud native"'s git. It
seems more foundational and complimentary.


I agree with this strongly.

I get frustrated really quickly when people tell me that, as a user, I
_should_ want something different than what I actually do want.

It turns out, I _do_ want computers - or at least things that behave
like computers - for some percentage of my workloads. I'm not the only
one out there who wants that.

There are other humans out there who do not want computers. They don't
like computers at all. They want generic execution contexts into which
they can put their apps.

It's just as silly to tell those people that they _should_ want
computers as it is to tell me that I _should_ want a generic execution
context.


I totally agree with you. There is room for all kinds of applications on 
OpenStack, and the great thing about an open community is that they can 
all have a voice. In theory.



One of the wonderful things about the situation we're in now is that if
you stack a k8s on top of an OpenStack then you empower the USER to
decide what their workload is and which types of features it is - rather
than forcing a "cloud native" vision dreamed up by "thought leaders"
down everyone's throats.


I'd go even further than that - many workloads are likely a mix of 
_both_, and we need to empower users to be able to use the right tools 
for the right parts of the job and _integrate_ them together. That's 
where OpenStack can add huge value to k8s and the like.


You may be familiar with the Kuryr project, which integrates Kubernetes 
deployments made by Magnum with Neutron networking so that other Nova 
servers can talk directly to the containers and other fun stuff. IMHO 
it's exactly the kind of thing OpenStack should be doing to make users' 
lives better, and give a compelling reason to install k8s on top of 
OpenStack instead of on bare metal.


So here's a fun thing I learned at the PTG: according to the Magnum 
folks, the main thing preventing them from fully adopting Kuryr is that 
the k8s application servers, provisioned with Nova, need to make API 
calls to Neutron to set up the ports as containers move around. And 
there's no secure way to give Keystone authentication credentials to an 
application server to do what it needs - and, especially, to do *only* 
what it needs.


http://lists.openstack.org/pipermail/openstack-dev/2016-October/105304.html

Keystone devs did agree in back in Austin that when they rejigger the 
default policy files it will done in such a way as to make the 
authorisation component of this feasible (by requiring a specific 
reader/writer role, not just membership of the project, to access APIs), 
but that change hasn't happened yet AFAIK. I suspect that it isn't their 
top priority. Kevin has been campaigning for *years* to get Nova to 
provide a secure way to inject credentials into a server in the same way 
that this is built in to EC2, GCE and (I assume but haven't checked) 
Azure. And they turned him down flat every time saying that this was not 
Nova's problem.


Sorry, but if OpenStack isn't a good, secure platform for running 
Kubernetes on then that is a HAIR ON FIRE-level *existential* problem in 
2017.


We can't place too much blame on individual projects though, because I 
believe the main reason this doesn't Just Work already is that there has 
been an unspoken consensus that we needed to listen to users like you 
but not to users like Kevin, and the elected leaders of our community 
have done nothing to either disavow or officially adopt that consensus. 
We _urgently_ need to decide if that's what we actually want and make 
sure it is prominently documented so that both users and developers know 
what's what.


FWIW I'm certain you must have hit this same issue in infra - probably 
you were able to use pre-signed Swift URLs when uploading log files to 
avoid needing credentials on servers allocated by nodepool? That's 
great, but not every API in OpenStack has a pre-signed URL facility, and 
nor should they have to.


(BTW I proposed a workaround for Magnum/Kuryr at the PTG by using a 
pre-signed Zaqar URL with a subscription triggering a Mistral workflow, 
and I've started working on a POC.)



It also turns out that the market agrees. Google App Engine was not
successful, until Google added IaaS. Azure was not successful until
Microsoft added IaaS. Amazon, who have a container service and a
serverless service is all built around the ecosystem that is centered on
... that's right ... an IaaS.


I think this is the point where you're supposed to insert the disclaimer 
that, in any mar

Re: [openstack-dev] [tc][appcat] The future of the App Catalog

So, this is the kind of thinking I'm talking about... OpenStack today is more 
then just IaaS in the tent. Trove (DBaaS), Sahara (Hadoop,Spark,etc aaS), Zaqar 
(Messaging aaS) and many more services. But they seem to be treated as second 
class citizens, as they are not "IaaS".

> Why should it strive to be anything except an excellent building block
for other technologies?

You misinterpret my statement. I'm in full agreement with you. The above 
services should be excellent building blocks too, but are suffering from lack 
of support from the IaaS layer. They deserve the ability to be excellent too, 
but need support/vision from the greater community that hasn't been forthcoming.

I agree with you, we should embrace the container folks and not treat them as 
separate. I think thats critical if we want to allow things like Sahara or 
Trove to really fulfil their potential. This is the path towards being an 
OpenSource AWS competitor, not just for being able to request vm's in a cloudy 
way.

I think that looks something like:
OpenStack Advanced Service (trove, sahara, etc) -> Kubernetes -> Nova VM or 
Ironic Bare Metal.

Not what we have today:
OpenStack Advanced Service -> Nova VM or Ironic Bare Metal

due to the focus on the api's of VM's being only for IaaS and not for actually 
running cloud software on.

I'm sorry if that sounds a bit confusing. Its hard to explain. I can try and 
elaborate if you want. Zane's posting here does help explain it a little:
http://www.zerobanana.com/archive/2015/04/24#a-vision-for-openstack

The other alternative is to clearly define OpenStack to be just IaaS, and kick 
all the non IaaS stuff out of the tent. (I do not prefer this). It will hurt in 
the short term but long tern could be better for those projects then the 
current status quo as new opportunities for switching base dependencies will 
present themselves and no longer will be hamstrung by those that feel their use 
case isn't important or think that the existing api's are "good enough" to 
handle the use case.

Thanks,
Kevin
 

From: Clint Byrum [cl...@fewbar.com]
Sent: Friday, March 10, 2017 2:43 PM
To: openstack-dev
Subject: Re: [openstack-dev] [tc][appcat] The future of the App Catalog

Excerpts from Fox, Kevin M's message of 2017-03-10 20:47:55 +:
> Just because the market hasn't immediately shifted from IaaS to containers 
> doesn't mean it won't happen eventually, and that google's wrong in their 
> push for containers over IaaS. It took a long time (still ongoing) to move 
> from physical machines to vm's. The same parallel will happen with containers 
> I believe. We're at least a few years out from it becoming more common place. 
> That doesn't mean folks can assume it will always be the way it is, and the 
> "market has spoken". "The only constants are death and taxes." :)
>
> Your right in the assertion k8s needs a place to run, but that doesn't 
> necessarily mean OpenStack, unless we help integrate it and make it a great 
> place to run it.
>
> I'm glad the IaaS stuff in OpenStack suits your needs. Thats great. It hasn't 
> served a great number of users needs though, and has been at least misleading 
> folks into believing it will for a number of years. If we want it to just be 
> an IaaS, lets free up those users to move on elsewhere.
>
> Does OpenStack intend just to be an IaaS in a few years or something else?
>

Why should it strive to be anything except an excellent building block
for other technologies?

Containers have a community and there's no reason we should think of that
as separate from us. We're friends, and we overlap when it makes sense.

All the container tech that I see out there still needs computers /
disks / networks to run on. OpenStack is a pretty decent way to chop your
computers up and give fully automated control of all aspects of them to
multiple tenants.  What else would you want it to do that isn't already
an aspect of Kubernetes, CloudFoundry, etc.?

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][appcat] The future of the App Catalog

Excerpts from Fox, Kevin M's message of 2017-03-10 20:47:55 +:
> Just because the market hasn't immediately shifted from IaaS to containers 
> doesn't mean it won't happen eventually, and that google's wrong in their 
> push for containers over IaaS. It took a long time (still ongoing) to move 
> from physical machines to vm's. The same parallel will happen with containers 
> I believe. We're at least a few years out from it becoming more common place. 
> That doesn't mean folks can assume it will always be the way it is, and the 
> "market has spoken". "The only constants are death and taxes." :)
> 
> Your right in the assertion k8s needs a place to run, but that doesn't 
> necessarily mean OpenStack, unless we help integrate it and make it a great 
> place to run it.
> 
> I'm glad the IaaS stuff in OpenStack suits your needs. Thats great. It hasn't 
> served a great number of users needs though, and has been at least misleading 
> folks into believing it will for a number of years. If we want it to just be 
> an IaaS, lets free up those users to move on elsewhere.
> 
> Does OpenStack intend just to be an IaaS in a few years or something else?
> 

Why should it strive to be anything except an excellent building block
for other technologies?

Containers have a community and there's no reason we should think of that
as separate from us. We're friends, and we overlap when it makes sense.

All the container tech that I see out there still needs computers /
disks / networks to run on. OpenStack is a pretty decent way to chop your
computers up and give fully automated control of all aspects of them to
multiple tenants.  What else would you want it to do that isn't already
an aspect of Kubernetes, CloudFoundry, etc.?

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][appcat] The future of the App Catalog

Thats the power of opensource. You don't HAVE to do it with investors and 
business plans. You can do it in a garage, if you have the right idea! :)

Thanks,
Kevin

From: Clint Byrum [cl...@fewbar.com]
Sent: Friday, March 10, 2017 11:03 AM
To: openstack-dev
Subject: Re: [openstack-dev] [tc][appcat] The future of the App Catalog

Excerpts from Joshua Harlow's message of 2017-03-10 10:09:24 -0800:
> Clint Byrum wrote:
> > Excerpts from Joshua Harlow's message of 2017-03-09 21:53:58 -0800:
> >> Renat Akhmerov wrote:
> >>>> On 10 Mar 2017, at 06:02, Zane Bitter >>>> <mailto:zbit...@redhat.com>>  wrote:
> >>>>
> >>>> On 08/03/17 11:23, David Moreau Simard wrote:
> >>>>> The App Catalog, to me, sounds sort of like a weird message that
> >>>>> OpenStack somehow requires applications to be
> >>>>> packaged/installed/deployed differently.
> >>>>> If anything, perhaps we should spend more effort on advertising that
> >>>>> OpenStack provides bare metal or virtual compute resources and that
> >>>>> apps will work just like any other places.
> >>>> Look, it's true that legacy apps from the 90s will run on any VM you
> >>>> can give them. But the rest of the world has spent the last 15 years
> >>>> moving on from that. Applications of the future, and increasingly the
> >>>> present, span multiple VMs/containers, make use of services provided
> >>>> by the cloud, and interact with their own infrastructure. And users
> >>>> absolutely will need ways of packaging and deploying them that work
> >>>> with the underlying infrastructure. Even those apps from the 90s
> >>>> should be taking advantage of things like e.g. Neutron security
> >>>> groups, configuration of which is and will always be out of scope for
> >>>> Docker Hub images.
> >>>>
> >>>> So no, we should NOT spend more effort on advertising that we aim to
> >>>> become to cloud what Subversion is to version control. We've done far
> >>>> too much of that already IMHO.
> >>> 100% agree with that.
> >>>
> >>> And this whole discussion is taking me to the question: is there really
> >>> any officially accepted strategy for OpenStack for 1, 3, 5 years?
> >> I can propose what I would like for a strategy (it's not more VMs and
> >> more neutron security groups...), though if it involves (more) design by
> >> committee, count me out.
> >>
> >> I honestly believe we have to do the equivalent of a technology leapfrog
> >> if we actually want to be relevant; but maybe I'm to eager...
> >>
> >
> > Open source isn't really famous for technology leapfrogging.
>
> Time to get famous.
>
> I hate accepting what the status quo is just because it's not been
> famous (or easy, or turned out, or ...) before.
>

Good luck. I can't see how you get an investor to enable you to do that
in this context without an absolute _mountain_ of relatively predictable
service-industry profit involved.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][appcat] The future of the App Catalog

Just because the market hasn't immediately shifted from IaaS to containers 
doesn't mean it won't happen eventually, and that google's wrong in their push 
for containers over IaaS. It took a long time (still ongoing) to move from 
physical machines to vm's. The same parallel will happen with containers I 
believe. We're at least a few years out from it becoming more common place. 
That doesn't mean folks can assume it will always be the way it is, and the 
"market has spoken". "The only constants are death and taxes." :)

Your right in the assertion k8s needs a place to run, but that doesn't 
necessarily mean OpenStack, unless we help integrate it and make it a great 
place to run it.

I'm glad the IaaS stuff in OpenStack suits your needs. Thats great. It hasn't 
served a great number of users needs though, and has been at least misleading 
folks into believing it will for a number of years. If we want it to just be an 
IaaS, lets free up those users to move on elsewhere.

Does OpenStack intend just to be an IaaS in a few years or something else?

Thanks,
Kevin

From: Monty Taylor [mord...@inaugust.com]
Sent: Friday, March 10, 2017 9:27 AM
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [tc][appcat] The future of the App Catalog

On 03/10/2017 10:59 AM, Clint Byrum wrote:
> Excerpts from Joshua Harlow's message of 2017-03-09 21:53:58 -0800:
>> Renat Akhmerov wrote:
>>>
>>>> On 10 Mar 2017, at 06:02, Zane Bitter >>> <mailto:zbit...@redhat.com>> wrote:
>>>>
>>>> On 08/03/17 11:23, David Moreau Simard wrote:
>>>>> The App Catalog, to me, sounds sort of like a weird message that
>>>>> OpenStack somehow requires applications to be
>>>>> packaged/installed/deployed differently.
>>>>> If anything, perhaps we should spend more effort on advertising that
>>>>> OpenStack provides bare metal or virtual compute resources and that
>>>>> apps will work just like any other places.
>>>>
>>>> Look, it's true that legacy apps from the 90s will run on any VM you
>>>> can give them. But the rest of the world has spent the last 15 years
>>>> moving on from that. Applications of the future, and increasingly the
>>>> present, span multiple VMs/containers, make use of services provided
>>>> by the cloud, and interact with their own infrastructure. And users
>>>> absolutely will need ways of packaging and deploying them that work
>>>> with the underlying infrastructure. Even those apps from the 90s
>>>> should be taking advantage of things like e.g. Neutron security
>>>> groups, configuration of which is and will always be out of scope for
>>>> Docker Hub images.
>>>>
>>>> So no, we should NOT spend more effort on advertising that we aim to
>>>> become to cloud what Subversion is to version control. We've done far
>>>> too much of that already IMHO.
>>>
>>> 100% agree with that.
>>>
>>> And this whole discussion is taking me to the question: is there really
>>> any officially accepted strategy for OpenStack for 1, 3, 5 years?
>>
>> I can propose what I would like for a strategy (it's not more VMs and
>> more neutron security groups...), though if it involves (more) design by
>> committee, count me out.
>>
>> I honestly believe we have to do the equivalent of a technology leapfrog
>> if we actually want to be relevant; but maybe I'm to eager...
>>
>
> Open source isn't really famous for technology leapfrogging.
>
> And before you say "but Docker.." remember that Solaris had zones 13
> years ago.
>
> What a community like ours is good at doing is gathering all the
> exciting industry leading bleeding edge chaos into a boring commodity
> platform. What Zane is saying (and I agree with) is let's make sure we see
> the whole cloud forest and not just focus on the VM trees in front of us.
>
> I'm curious what you (Josh) or Zane would change too.
> Containers/apps/kubes/etc. have to run on computers with storage and
> networks. OpenStack provides a pretty rich set of features for giving
> users computers with storage on networks, and operators a way to manage
> those. So I fail to see how that is svn to "cloud native"'s git. It
> seems more foundational and complimentary.

I agree with this strongly.

I get frustrated really quickly when people tell me that, as a user, I
_should_ want something different than what I actually do want.

It turns out, I _do_ want comput

Re: [openstack-dev] [tc][appcat] The future of the App Catalog

But there is plenty of software out there that opensource has not made a boring 
commodity platform. AWS has tons of services that are quite useful that have no 
real implementation today. I have users being pushed out of OpenStack onto AWS 
simply because it is so much cheaper to build applications on top of AWS as the 
common services on AWS mean they don't need to implement the bits themselves.

This is where OpenStack could still get a competitive advantage. But on the 
current trajectory, the plumbing those types of things need to use in OpenStack 
today are not agile enough.

Take Trove as an example. The goal is to provide database as a service. The 
user doesn't care if it is launched on a vm, or if it happens through 
containers. But the latter is much faster to launch now, easier to code for, 
and easier to administrate for operators. But we're stuck trying to add more 
and more layers of abstraction to add compatibility with all the ways of 
deploying it on vm's and containers, rather then pick a winner and just go with 
it.

Thanks,
Kevin

From: Clint Byrum [cl...@fewbar.com]
Sent: Friday, March 10, 2017 8:59 AM
To: openstack-dev
Subject: Re: [openstack-dev] [tc][appcat] The future of the App Catalog

Excerpts from Joshua Harlow's message of 2017-03-09 21:53:58 -0800:
> Renat Akhmerov wrote:
> >
> >> On 10 Mar 2017, at 06:02, Zane Bitter  >> <mailto:zbit...@redhat.com>> wrote:
> >>
> >> On 08/03/17 11:23, David Moreau Simard wrote:
> >>> The App Catalog, to me, sounds sort of like a weird message that
> >>> OpenStack somehow requires applications to be
> >>> packaged/installed/deployed differently.
> >>> If anything, perhaps we should spend more effort on advertising that
> >>> OpenStack provides bare metal or virtual compute resources and that
> >>> apps will work just like any other places.
> >>
> >> Look, it's true that legacy apps from the 90s will run on any VM you
> >> can give them. But the rest of the world has spent the last 15 years
> >> moving on from that. Applications of the future, and increasingly the
> >> present, span multiple VMs/containers, make use of services provided
> >> by the cloud, and interact with their own infrastructure. And users
> >> absolutely will need ways of packaging and deploying them that work
> >> with the underlying infrastructure. Even those apps from the 90s
> >> should be taking advantage of things like e.g. Neutron security
> >> groups, configuration of which is and will always be out of scope for
> >> Docker Hub images.
> >>
> >> So no, we should NOT spend more effort on advertising that we aim to
> >> become to cloud what Subversion is to version control. We've done far
> >> too much of that already IMHO.
> >
> > 100% agree with that.
> >
> > And this whole discussion is taking me to the question: is there really
> > any officially accepted strategy for OpenStack for 1, 3, 5 years?
>
> I can propose what I would like for a strategy (it's not more VMs and
> more neutron security groups...), though if it involves (more) design by
> committee, count me out.
>
> I honestly believe we have to do the equivalent of a technology leapfrog
> if we actually want to be relevant; but maybe I'm to eager...
>

Open source isn't really famous for technology leapfrogging.

And before you say "but Docker.." remember that Solaris had zones 13
years ago.

What a community like ours is good at doing is gathering all the
exciting industry leading bleeding edge chaos into a boring commodity
platform. What Zane is saying (and I agree with) is let's make sure we see
the whole cloud forest and not just focus on the VM trees in front of us.

I'm curious what you (Josh) or Zane would change too.
Containers/apps/kubes/etc. have to run on computers with storage and
networks. OpenStack provides a pretty rich set of features for giving
users computers with storage on networks, and operators a way to manage
those. So I fail to see how that is svn to "cloud native"'s git. It
seems more foundational and complimentary.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][appcat] The future of the App Catalog


On 10/03/17 11:59, Clint Byrum wrote:
 What a community like ours is good at doing is gathering all the

exciting industry leading bleeding edge chaos into a boring commodity
platform. What Zane is saying (and I agree with) is let's make sure we see
the whole cloud forest and not just focus on the VM trees in front of us.

I'm curious what you (Josh) or Zane would change too.


Here's something simple that I've been pushing for 2 years and we still 
haven't done yet:


http://www.zerobanana.com/archive/2015/04/24#a-vision-for-openstack

There's plenty more to say, but perhaps this is not the place. If Josh 
starts a new thread I'll be happy to contribute.



Containers/apps/kubes/etc. have to run on computers with storage and
networks. OpenStack provides a pretty rich set of features for giving
users computers with storage on networks, and operators a way to manage
those. So I fail to see how that is svn to "cloud native"'s git. It
seems more foundational and complimentary.


I should have clarified the svn remark. It was a discussion about 
marketing, not technology.


svn is solid, stable, ubiquitous in a certain class of old-school 
Enterprise shops, and completely irrelevant to the future of software 
development. That's the future we're in danger of sleepwalking into.


cheers,
Zane.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][appcat] The future of the App Catalog

2017-03-10 Thread Jay Pipes


On 03/10/2017 01:37 PM, Zane Bitter wrote:

On 09/03/17 23:57, Renat Akhmerov wrote:

And this whole discussion is taking me to the question: is there really
any officially accepted strategy for OpenStack for 1, 3, 5 years? Is
there any ultimate community goal we’re moving to regardless of
underlying technologies (containers, virtualization etc.)? I know we’re
now considering various community goals like transition to Python 3.5
etc. but these goals don’t tell anything about our future as an IT
ecosystem from user perspective. I may assume that I’m just not aware of
it. I’d be glad if it was true. I’m eager to know the answers for these
questions.


Me too! There was one effort started by John Garbutt in December, a
technical vision for OpenStack in the form of a TC resolution in the
governance repo:

https://review.openstack.org/#/c/401226/

I wasn't a fan of the draft content, but I believe his intention was to
seed it with a formalised version of our de-facto historical position
(tl;dr legacy apps from the 90s). As long as that's a starting point for
the discussion and not a conclusion then I think this was a valuable
contribution. I commented with a non-exhaustive list of things that I
would expect to see at least debated in a vision for a cloud computing
platform, which I will reproduce here since it's relevant to this thread:

* Infinite scaling - the ability in principle to scale from zero to an
arbitrarily large number of users without rewriting your application
(e.g. if your application can store one file in Swift then there's no
theoretical limit to how many it can store. c.f. Cinder where at some
point you'd have to start juggling multiple volumes.)
* Granularity of allocation - pay only for the resources you actually
use, rather than to allocate a chunk that you may or may not be using
(so Nova->containers->FaaS, Cinder->Swift, Trove->??? [RIP MagnetoDB], &c.)
* Full control of infrastructure - notwithstanding the above, maintain
Nova/Cinder/Neutron/Trove/&c. so that legacy applications, highly
specialised applications, and higher-level services like PaaS can make
fully customised use of the virtual infrastructure.
* Hardware virtualisation - make anything that might typically be done
in hardware available in a multi-tenant software-defined environment:
servers, routers, load balancers, firewalls, video codecs, GPGPUs, FPGAs...
* Built-in reliability - don't require even the smallest apps to have 3
VMs + a cluster manager to enforce any reliability guarantees; provide
those guarantees using multi-tenant services that efficiently share
resources between applications (see also: Infinite scaling, Granularity
of allocation).
* Application control - (securely) give applications control over their
own infrastructure, so that no part of the application needs to reside
outside of the cloud.
* Integration - cloud services that effectively form part of the user's
application can communicate amongst themselves, where appropriate,
without the need for client-side glue (see also: Built-in reliability).
* Interoperability - the same applications can be deployed on a variety
of private and public OpenStack clouds.


Those are all interesting technical concepts to think about and discuss. 
However, what Kevin said originally in his response about the OpenStack 
community needing to decide what exactly it *is* and what scope 
OpenStack should pursue, is the real foundational question that needs to 
be addressed. Until it is, none of the rest of these topics have much 
contextual relevance and are just a distraction, IMHO.


Best,
-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][appcat] The future of the App Catalog

Excerpts from Joshua Harlow's message of 2017-03-10 10:09:24 -0800:
> Clint Byrum wrote:
> > Excerpts from Joshua Harlow's message of 2017-03-09 21:53:58 -0800:
> >> Renat Akhmerov wrote:
>  On 10 Mar 2017, at 06:02, Zane Bitter  >  wrote:
> 
>  On 08/03/17 11:23, David Moreau Simard wrote:
> > The App Catalog, to me, sounds sort of like a weird message that
> > OpenStack somehow requires applications to be
> > packaged/installed/deployed differently.
> > If anything, perhaps we should spend more effort on advertising that
> > OpenStack provides bare metal or virtual compute resources and that
> > apps will work just like any other places.
>  Look, it's true that legacy apps from the 90s will run on any VM you
>  can give them. But the rest of the world has spent the last 15 years
>  moving on from that. Applications of the future, and increasingly the
>  present, span multiple VMs/containers, make use of services provided
>  by the cloud, and interact with their own infrastructure. And users
>  absolutely will need ways of packaging and deploying them that work
>  with the underlying infrastructure. Even those apps from the 90s
>  should be taking advantage of things like e.g. Neutron security
>  groups, configuration of which is and will always be out of scope for
>  Docker Hub images.
> 
>  So no, we should NOT spend more effort on advertising that we aim to
>  become to cloud what Subversion is to version control. We've done far
>  too much of that already IMHO.
> >>> 100% agree with that.
> >>>
> >>> And this whole discussion is taking me to the question: is there really
> >>> any officially accepted strategy for OpenStack for 1, 3, 5 years?
> >> I can propose what I would like for a strategy (it's not more VMs and
> >> more neutron security groups...), though if it involves (more) design by
> >> committee, count me out.
> >>
> >> I honestly believe we have to do the equivalent of a technology leapfrog
> >> if we actually want to be relevant; but maybe I'm to eager...
> >>
> >
> > Open source isn't really famous for technology leapfrogging.
> 
> Time to get famous.
> 
> I hate accepting what the status quo is just because it's not been 
> famous (or easy, or turned out, or ...) before.
> 

Good luck. I can't see how you get an investor to enable you to do that
in this context without an absolute _mountain_ of relatively predictable
service-industry profit involved.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][appcat] The future of the App Catalog


On 09/03/17 23:57, Renat Akhmerov wrote:

And this whole discussion is taking me to the question: is there really
any officially accepted strategy for OpenStack for 1, 3, 5 years? Is
there any ultimate community goal we’re moving to regardless of
underlying technologies (containers, virtualization etc.)? I know we’re
now considering various community goals like transition to Python 3.5
etc. but these goals don’t tell anything about our future as an IT
ecosystem from user perspective. I may assume that I’m just not aware of
it. I’d be glad if it was true. I’m eager to know the answers for these
questions.


Me too! There was one effort started by John Garbutt in December, a 
technical vision for OpenStack in the form of a TC resolution in the 
governance repo:


https://review.openstack.org/#/c/401226/

I wasn't a fan of the draft content, but I believe his intention was to 
seed it with a formalised version of our de-facto historical position 
(tl;dr legacy apps from the 90s). As long as that's a starting point for 
the discussion and not a conclusion then I think this was a valuable 
contribution. I commented with a non-exhaustive list of things that I 
would expect to see at least debated in a vision for a cloud computing 
platform, which I will reproduce here since it's relevant to this thread:


* Infinite scaling - the ability in principle to scale from zero to an 
arbitrarily large number of users without rewriting your application 
(e.g. if your application can store one file in Swift then there's no 
theoretical limit to how many it can store. c.f. Cinder where at some 
point you'd have to start juggling multiple volumes.)
* Granularity of allocation - pay only for the resources you actually 
use, rather than to allocate a chunk that you may or may not be using 
(so Nova->containers->FaaS, Cinder->Swift, Trove->??? [RIP MagnetoDB], &c.)
* Full control of infrastructure - notwithstanding the above, maintain 
Nova/Cinder/Neutron/Trove/&c. so that legacy applications, highly 
specialised applications, and higher-level services like PaaS can make 
fully customised use of the virtual infrastructure.
* Hardware virtualisation - make anything that might typically be done 
in hardware available in a multi-tenant software-defined environment: 
servers, routers, load balancers, firewalls, video codecs, GPGPUs, FPGAs...
* Built-in reliability - don't require even the smallest apps to have 3 
VMs + a cluster manager to enforce any reliability guarantees; provide 
those guarantees using multi-tenant services that efficiently share 
resources between applications (see also: Infinite scaling, Granularity 
of allocation).
* Application control - (securely) give applications control over their 
own infrastructure, so that no part of the application needs to reside 
outside of the cloud.
* Integration - cloud services that effectively form part of the user's 
application can communicate amongst themselves, where appropriate, 
without the need for client-side glue (see also: Built-in reliability).
* Interoperability - the same applications can be deployed on a variety 
of private and public OpenStack clouds.


Anyway, the review was abandoned and moved to an etherpad (sans 
comments... sigh): 
https://etherpad.openstack.org/p/AtlantaPTG-SWG-OpenStackVision


Then the whole discussion was shelved, pending a resolution of the 
preceding discussion about a vision for the TC - IOW the TC had to 
decide whether anybody ought to care about the Technical Committee's 
technical vision for OpenStack (my 2c: it's right there in the name) 
before it could decide what that vision was. Of course you can't not 
decide - not deciding is a quieter way of deciding to stick with the 
status quo. Though given the amount of pushback they got against the 
relatively simple idea of moving off the imminently-EOL Python 2 runtime 
in a co-ordinated rather than ad-hoc fashion, I guess you couldn't 
really blame them.


*That* discussion (the TC vision one) was scheduled to happen in the 
Stewardship Working Group meeting at the PTG in Atlanta: 
https://etherpad.openstack.org/p/AtlantaPTG-SWG-TCVision


Then it was cancelled because it became clear that most of the TC 
members didn't plan to show up due to scheduling conflicts.


Looking through the TC mailing list, it appears it was rescheduled to 
happen this week at a joint TC/Board meeting in Boston. This looks to be 
one of the outputs: 
https://etherpad.openstack.org/p/ocata-12questions-exercise-board 
Hopefully we'll hear more in the next few days, since this _just_ 
happened. But progress toward a technical vision can't come fast enough 
for me - we needed this 3 years ago, and we don't *have* another year to 
figure it out.


cheers,
Zane.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/

Re: [openstack-dev] [tc][appcat] The future of the App Catalog

2017-03-10 Thread Joshua Harlow


Clint Byrum wrote:

Excerpts from Joshua Harlow's message of 2017-03-09 21:53:58 -0800:

Renat Akhmerov wrote:

On 10 Mar 2017, at 06:02, Zane Bittermailto:zbit...@redhat.com>>  wrote:

On 08/03/17 11:23, David Moreau Simard wrote:

The App Catalog, to me, sounds sort of like a weird message that
OpenStack somehow requires applications to be
packaged/installed/deployed differently.
If anything, perhaps we should spend more effort on advertising that
OpenStack provides bare metal or virtual compute resources and that
apps will work just like any other places.

Look, it's true that legacy apps from the 90s will run on any VM you
can give them. But the rest of the world has spent the last 15 years
moving on from that. Applications of the future, and increasingly the
present, span multiple VMs/containers, make use of services provided
by the cloud, and interact with their own infrastructure. And users
absolutely will need ways of packaging and deploying them that work
with the underlying infrastructure. Even those apps from the 90s
should be taking advantage of things like e.g. Neutron security
groups, configuration of which is and will always be out of scope for
Docker Hub images.

So no, we should NOT spend more effort on advertising that we aim to
become to cloud what Subversion is to version control. We've done far
too much of that already IMHO.

100% agree with that.

And this whole discussion is taking me to the question: is there really
any officially accepted strategy for OpenStack for 1, 3, 5 years?

I can propose what I would like for a strategy (it's not more VMs and
more neutron security groups...), though if it involves (more) design by
committee, count me out.

I honestly believe we have to do the equivalent of a technology leapfrog
if we actually want to be relevant; but maybe I'm to eager...



Open source isn't really famous for technology leapfrogging.


Time to get famous.

I hate accepting what the status quo is just because it's not been 
famous (or easy, or turned out, or ...) before.




And before you say "but Docker.." remember that Solaris had zones 13
years ago.

What a community like ours is good at doing is gathering all the
exciting industry leading bleeding edge chaos into a boring commodity
platform. What Zane is saying (and I agree with) is let's make sure we see
the whole cloud forest and not just focus on the VM trees in front of us.

I'm curious what you (Josh) or Zane would change too.
Containers/apps/kubes/etc. have to run on computers with storage and
networks. OpenStack provides a pretty rich set of features for giving
users computers with storage on networks, and operators a way to manage
those. So I fail to see how that is svn to "cloud native"'s git. It
seems more foundational and complimentary.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][appcat] The future of the App Catalog

2017-03-10 Thread Monty Taylor

On 03/10/2017 10:59 AM, Clint Byrum wrote:
> Excerpts from Joshua Harlow's message of 2017-03-09 21:53:58 -0800:
>> Renat Akhmerov wrote:
>>>
 On 10 Mar 2017, at 06:02, Zane Bitter >>> > wrote:

 On 08/03/17 11:23, David Moreau Simard wrote:
> The App Catalog, to me, sounds sort of like a weird message that
> OpenStack somehow requires applications to be
> packaged/installed/deployed differently.
> If anything, perhaps we should spend more effort on advertising that
> OpenStack provides bare metal or virtual compute resources and that
> apps will work just like any other places.

 Look, it's true that legacy apps from the 90s will run on any VM you
 can give them. But the rest of the world has spent the last 15 years
 moving on from that. Applications of the future, and increasingly the
 present, span multiple VMs/containers, make use of services provided
 by the cloud, and interact with their own infrastructure. And users
 absolutely will need ways of packaging and deploying them that work
 with the underlying infrastructure. Even those apps from the 90s
 should be taking advantage of things like e.g. Neutron security
 groups, configuration of which is and will always be out of scope for
 Docker Hub images.

 So no, we should NOT spend more effort on advertising that we aim to
 become to cloud what Subversion is to version control. We've done far
 too much of that already IMHO.
>>>
>>> 100% agree with that.
>>>
>>> And this whole discussion is taking me to the question: is there really
>>> any officially accepted strategy for OpenStack for 1, 3, 5 years?
>>
>> I can propose what I would like for a strategy (it's not more VMs and 
>> more neutron security groups...), though if it involves (more) design by 
>> committee, count me out.
>>
>> I honestly believe we have to do the equivalent of a technology leapfrog 
>> if we actually want to be relevant; but maybe I'm to eager...
>>
> 
> Open source isn't really famous for technology leapfrogging.
> 
> And before you say "but Docker.." remember that Solaris had zones 13
> years ago.
> 
> What a community like ours is good at doing is gathering all the
> exciting industry leading bleeding edge chaos into a boring commodity
> platform. What Zane is saying (and I agree with) is let's make sure we see
> the whole cloud forest and not just focus on the VM trees in front of us.
> 
> I'm curious what you (Josh) or Zane would change too.
> Containers/apps/kubes/etc. have to run on computers with storage and
> networks. OpenStack provides a pretty rich set of features for giving
> users computers with storage on networks, and operators a way to manage
> those. So I fail to see how that is svn to "cloud native"'s git. It
> seems more foundational and complimentary.

I agree with this strongly.

I get frustrated really quickly when people tell me that, as a user, I
_should_ want something different than what I actually do want.

It turns out, I _do_ want computers - or at least things that behave
like computers - for some percentage of my workloads. I'm not the only
one out there who wants that.

There are other humans out there who do not want computers. They don't
like computers at all. They want generic execution contexts into which
they can put their apps.

It's just as silly to tell those people that they _should_ want
computers as it is to tell me that I _should_ want a generic execution
context.

One of the wonderful things about the situation we're in now is that if
you stack a k8s on top of an OpenStack then you empower the USER to
decide what their workload is and which types of features it is - rather
than forcing a "cloud native" vision dreamed up by "thought leaders"
down everyone's throats.

It also turns out that the market agrees. Google App Engine was not
successful, until Google added IaaS. Azure was not successful until
Microsoft added IaaS. Amazon, who have a container service and a
serverless service is all built around the ecosystem that is centered on
... that's right ... an IaaS.

So rather than us trying to chase a thing we're not (we're not a
container or thin-app orchestration tool) - being comfortable with our
actual identity (IaaS provider of computers) and working with other
people who do other things ends up providing _users_ with the a real win.

Considering computers as some-how inherently 'better' or 'worse' than
some of the 'cloud-native' concepts is hog wash. Different users have
different needs. As Clint points out - kubernetes needs to run
_somewhere_. CloudFoundry needs to run _somewhere_. So those are at
least two other potential users who are not me and my collection of
things I want to run that want to run in computers.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?

Re: [openstack-dev] [tc][appcat] The future of the App Catalog