Re: [openstack-dev] [all] Proposal: Architecture Working Group

[joehuang]

Hi, all,



Thanks to intiate the architecture working group, would be glad to join the 
group if there is still a place to stand.



According to the comment from Thierry in the Tricircle big-tent application 
https://review.openstack.org/#/c/338796/: "From an OpenStack community 
standpoint, we need more agreement and consensus on how we want to tackle the 
"massive scaling" issues. Tactical solutions like Nova cells only work for one 
project. I hope that the newly-founded Architecture WG can openly discuss an 
architecture for scaling OpenStack clouds beyond their current limits, a vision 
we can all get behind.





This is not a technical reflection on the quality of the Tricircle work. I 
think it's very interesting, I think the project should continue experimenting 
with the solution and I definitely want the Architecture WG to consider the 
Tricircle approach to scaling of using a top cell, API proxies and helpers. If 
anything, it forces us into having a discussion we should have had a long time 
ago, and for that I'm grateful that it was proposed."



So kindly please put the scaling OpenStack clouds in the agenda items, and also 
take these use cases into consideration: 
https://docs.google.com/presentation/d/1Zkoi4vMOGN713Vv_YO0GP6YLyjLpQ7fRbHlirpq6ZK4/edit?usp=sharing,
 i.e, let's discuss how to scale OpenStack clouds into one site or multiple 
sites.



Thank you all in advance.



Best Regards

Chaoyi Huang ( joehuang )







http://lists.openstack.org/pipermail/openstack-dev/2016-June/097668.html



[openstack-dev] [all] Proposal: Architecture Working Group

Clint Byrum clint at fewbar.com 

Fri Jun 17 21:52:43 UTC 2016

  *   Previous message: [openstack-dev] OpenStack Developer Mailing List Digest 
May 14 to June 
17
  *   Next message: [openstack-dev] [all] Proposal: Architecture Working 
Group
  *   Messages sorted by: [ date 
] 
[ thread 
]
 [ subject 
]
 [ author 
]

ar·chi·tec·ture
ˈärkəˌtek(t)SHər/
noun
noun: architecture

1.

the art or practice of designing and constructing buildings.

synonyms:building design, building style, planning, building, construction;

formalarchitectonics

"modern architecture"

the style in which a building is designed or constructed, especially with 
regard to a specific period, place, or culture.

plural noun: architectures

"Victorian architecture"

2.

the complex or carefully designed structure of something.

"the chemical architecture of the human brain"

the conceptual structure and logical organization of a computer or 
computer-based system.

"a client/server architecture"

synonyms:structure, construction, organization, layout, design, build, 
anatomy, makeup;

informalsetup

"the architecture of a computer system"


Introduction
=

OpenStack is a big system. We have debated what it actually is [1],
and there are even t-shirts to poke fun at the fact that we don't have
good answers.

But this isn't what any of us wants. We'd like to be able to point
at something and proudly tell people "This is what we designed and
implemented."

And for each individual project, that is a possibility. Neutron can
tell you they designed how their agents and drivers work. Nova can
tell you that they designed the way conductors handle communication
with API nodes and compute nodes. But when we start talking about how
they interact with each other, it's clearly just a coincidental mash of
de-facto standards and specs that don't help anyone make decisions when
refactoring or adding on to the system.

Oslo and cross-project initiatives have brought some peace and order
to the implementation and engineering processes, but not to the design
process. New ideas still start largely in the project where they are
needed most, and often conflict with similar decisions and ideas in other
projects [dlm, taskflow, tooz, service discovery, state machines, glance
tasks, messaging patterns, database patterns, etc. etc.]. Often times this
creates a log jam where none of the projects adopt a solution that would
align with others. Most of the time when things finally come to a head
these things get done in a piecemeal fashion, where it's half done here,
1/3 over there, 1/4 there, and 3/4 over there..., which to the outside
looks like  chaos, because that's precisely what 

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Arkady_Kanevsky]

+1

-Original Message-
From: Nikhil Komawar [mailto:nik.koma...@gmail.com]
Sent: Monday, June 20, 2016 10:37 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [all] Proposal: Architecture Working Group

+1 , great idea.

if we can add a mission/objective based on the nice definitions you added, will 
help a long way in cross-project architecture evolution.
moreover, I'd like this to be a integration point for openstack projects (and 
not a silo) so that we can build the shared understanding we really need to 
build.

On 6/17/16 5:52 PM, Clint Byrum wrote:
> ar·chi·tec·ture
> ˈärkəˌtek(t)SHər/
> noun
> noun: architecture
>
> 1.
>
> the art or practice of designing and constructing buildings.
>
> synonyms:building design, building style, planning, building,
> construction;
>
> formalarchitectonics
>
> "modern architecture"
>
> the style in which a building is designed or constructed, especially with 
> regard to a specific period, place, or culture.
>
> plural noun: architectures
>
> "Victorian architecture"
>
> 2.
>
> the complex or carefully designed structure of something.
>
> "the chemical architecture of the human brain"
>
> the conceptual structure and logical organization of a computer or 
> computer-based system.
>
> "a client/server architecture"
>
> synonyms:structure, construction, organization, layout, design,
> build, anatomy, makeup;
>
> informalsetup
>
> "the architecture of a computer system"
>
>
> Introduction
> =
>
> OpenStack is a big system. We have debated what it actually is [1],
> and there are even t-shirts to poke fun at the fact that we don't have
> good answers.
>
> But this isn't what any of us wants. We'd like to be able to point at
> something and proudly tell people "This is what we designed and
> implemented."
>
> And for each individual project, that is a possibility. Neutron can
> tell you they designed how their agents and drivers work. Nova can
> tell you that they designed the way conductors handle communication
> with API nodes and compute nodes. But when we start talking about how
> they interact with each other, it's clearly just a coincidental mash
> of de-facto standards and specs that don't help anyone make decisions
> when refactoring or adding on to the system.
>
> Oslo and cross-project initiatives have brought some peace and order
> to the implementation and engineering processes, but not to the design
> process. New ideas still start largely in the project where they are
> needed most, and often conflict with similar decisions and ideas in
> other projects [dlm, taskflow, tooz, service discovery, state
> machines, glance tasks, messaging patterns, database patterns, etc.
> etc.]. Often times this creates a log jam where none of the projects
> adopt a solution that would align with others. Most of the time when
> things finally come to a head these things get done in a piecemeal
> fashion, where it's half done here,
> 1/3 over there, 1/4 there, and 3/4 over there..., which to the outside
> looks like chaos, because that's precisely what it is.
>
> And this isn't always a technical design problem. OpenStack, for
> instance, isn't really a micro service architecture. Of course, it
> might look like that in diagrams [2], but we all know it really isn't.
> The compute node is home to agents for every single concern, and the
> API interactions between the services is too tightly woven to consider
> many of them functional without the same lockstep version of other
> services together. A game to play is ask yourself what would happen if
> a service was isolated on its own island, how functional would its API
> be, if at all. Is this something that we want? No. But there doesn't
> seem to be a place where we can go to actually design, discuss,
> debate, and ratify changes that would help us get to the point of
> gathering the necessary will and capability to enact these efforts.
>
> Maybe nova-compute should be isolated from nova, with an API that
> nova, cinder and neutron talk to. Maybe we should make the scheduler
> cross-project aware and capable of scheduling more than just nova
> instances. Maybe we should have experimental groups that can look at
> how some of this functionality could perhaps be delegated to
> non-openstack projects. We hear that Mesos, for example to help with
> the scheduling aspects, but how do we discuss these outside hijacking
> threads on the mailing list? These are things that we all discuss in
> the hallways and bars and parties at the summit, but because they
> cross projects at the design level, and are inherently a lot of 

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Clint Byrum]

Excerpts from Mike Perez's message of 2016-06-30 14:10:30 -0700:
> On 09:02 Jun 30, Clint Byrum wrote:
> > Excerpts from Mike Perez's message of 2016-06-30 07:50:42 -0700:
> > > On 11:31 Jun 20, Clint Byrum wrote:
> > > > Excerpts from Joshua Harlow's message of 2016-06-17 15:33:25 -0700:
> > > > > Thanks for getting this started Clint,
> > > > > 
> > > > > I'm happy and excited to be involved in helping try to guide the 
> > > > > whole 
> > > > > ecosystem together (it's also why I like being in oslo) to a 
> > > > > architecture that is more cohesive (and is more of something that we 
> > > > > can 
> > > > > say to our current or future children that we were all involved and 
> > > > > proud to be involved in creating/maturing...).
> > > > > 
> > > > > At a start, for said first meeting, any kind of agenda come to mind, 
> > > > > or 
> > > > > will it be more a informal gathering to start (either is fine with 
> > > > > me)?
> > > > > 
> > > > 
> > > > I've been hesitant to fill this in too much as I'm still forming the
> > > > idea, but here are the items I think are most compelling to begin with:
> > > > 
> > > > * DLM's across OpenStack -- This is already under way[1], but it seems 
> > > > to
> > > >   have fizzled out. IMO that is because there's no working group who
> > > >   owns it. We need to actually write some plans.
> > > 
> > > Not meaning to nitpick, but I don't think this is a compelling reason for 
> > > the
> > > architecture working group. We need a group that wants to spend time on
> > > reviewing the drivers being proposed. This is like saying we need the
> > > architecture working group because no working group is actively reshaping 
> > > quotas
> > > cross-project. 
> > > 
> > 
> > That sounds like a reasoned deep argument, not a nitpick, so thank you
> > for making it.
> > 
> > However, I don't think lack of drivers is standing in the way of a DLM
> > effort. It is a lack of coordination. There was a race to the finish line
> > to make Consul and etcd drivers, but then, like the fish in finding Nemo,
> > the drivers are in bags floating in the bay.. now what?
> 
> Some drivers are still in review, or likely abandoned efforts so it's not
> really a bay of options as you're describing it.
> 

Heh, that kind of sounds like the same thing.. not a bay of options,
just options stuck between the fish tank and the bay.

> Cinder has continued forward with being the guinea pig as planned with Tooz.
> [1] I don't think this a great example for your argument because
> 
> 1) Not all projects need this.
> 
> 2) This was discussed in Tokyo and just done in Mitaka for Cinder. Why not 
> give
>projects time to evaluate when they're ready?
> 
> > Nobody owns this effort. Everybody gets busy. Nothing gets done. We
> > continue to bring it up in the hallway and wish we had time.
>
> I don't ever foresee a release where we say "All projects support DLM". In 
> fact
> I see things going as planned because:
> 
> 1) We have a project that carried it forward as planned.
> 2) We're purposely not repeat the MQ mess. Only DLM drivers with support from
>members of the community are surfacing up.
> 
> I would ask you instead, how exactly are you measuring success here?
>

That's a great question. I think the community did what I'd like to
see the working group do as it's first order of business: Mapped the
territory, and provided a plan to improve it. So to your point, there's no
need for an architecture working group if this always happens as planned
in all instances. I'd personally like to see it happen this way all the
time, which is the primary reason I'm motivated to coordinate this
group.

As a second order of business, I think this group would have a hard time
keeping momentum if all it did were write architectural plans. Each of
the designs it helps create need to be backed up with actual work. Who
cares if you drew a picture of a bridge: show me the bridge. :)

> > This is just a place to have a meeting and some people who get together
> > and say "hey is that done yet? Do you need help? is that still a
> > priority?". Could we do this as part of Oslo? Yes! But, I want this to
> > be about going one step higher, and actually taking the implementations
> > into the respective projects.
> 
> How about calling a cross-project meeting? [2] I have already spent the time
> organizing people who are interested from each appropriate project team that
> are eager to help [3]. Again you can call your posse whatever, but please work
> with the people already around to assist.
> 

That's exactly what I want to help do. So perhaps we do need to more
formally attach that second order of business to the existing cross
project processes. I'll noodle on that and see if I can more clearly
draw that line. Thanks for bringing up the overlap.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Mike Perez]

On 09:02 Jun 30, Clint Byrum wrote:
> Excerpts from Mike Perez's message of 2016-06-30 07:50:42 -0700:
> > On 11:31 Jun 20, Clint Byrum wrote:
> > > Excerpts from Joshua Harlow's message of 2016-06-17 15:33:25 -0700:
> > > > Thanks for getting this started Clint,
> > > > 
> > > > I'm happy and excited to be involved in helping try to guide the whole 
> > > > ecosystem together (it's also why I like being in oslo) to a 
> > > > architecture that is more cohesive (and is more of something that we 
> > > > can 
> > > > say to our current or future children that we were all involved and 
> > > > proud to be involved in creating/maturing...).
> > > > 
> > > > At a start, for said first meeting, any kind of agenda come to mind, or 
> > > > will it be more a informal gathering to start (either is fine with me)?
> > > > 
> > > 
> > > I've been hesitant to fill this in too much as I'm still forming the
> > > idea, but here are the items I think are most compelling to begin with:
> > > 
> > > * DLM's across OpenStack -- This is already under way[1], but it seems to
> > >   have fizzled out. IMO that is because there's no working group who
> > >   owns it. We need to actually write some plans.
> > 
> > Not meaning to nitpick, but I don't think this is a compelling reason for 
> > the
> > architecture working group. We need a group that wants to spend time on
> > reviewing the drivers being proposed. This is like saying we need the
> > architecture working group because no working group is actively reshaping 
> > quotas
> > cross-project. 
> > 
> 
> That sounds like a reasoned deep argument, not a nitpick, so thank you
> for making it.
> 
> However, I don't think lack of drivers is standing in the way of a DLM
> effort. It is a lack of coordination. There was a race to the finish line
> to make Consul and etcd drivers, but then, like the fish in finding Nemo,
> the drivers are in bags floating in the bay.. now what?

Some drivers are still in review, or likely abandoned efforts so it's not
really a bay of options as you're describing it.

Cinder has continued forward with being the guinea pig as planned with Tooz.
[1] I don't think this a great example for your argument because

1) Not all projects need this.

2) This was discussed in Tokyo and just done in Mitaka for Cinder. Why not give
   projects time to evaluate when they're ready?

> Nobody owns this effort. Everybody gets busy. Nothing gets done. We
> continue to bring it up in the hallway and wish we had time.
   
I don't ever foresee a release where we say "All projects support DLM". In fact
I see things going as planned because:

1) We have a project that carried it forward as planned.
2) We're purposely not repeat the MQ mess. Only DLM drivers with support from
   members of the community are surfacing up.

I would ask you instead, how exactly are you measuring success here?

> This is just a place to have a meeting and some people who get together
> and say "hey is that done yet? Do you need help? is that still a
> priority?". Could we do this as part of Oslo? Yes! But, I want this to
> be about going one step higher, and actually taking the implementations
> into the respective projects.

How about calling a cross-project meeting? [2] I have already spent the time
organizing people who are interested from each appropriate project team that
are eager to help [3]. Again you can call your posse whatever, but please work
with the people already around to assist.

[1] - https://review.openstack.org/#/c/185646/
[2] - http://docs.openstack.org/project-team-guide/cross-project.html#general
[3] - 
https://wiki.openstack.org/wiki/CrossProjectLiaisons#Cross-Project_Spec_Liaisons

-- 
Mike Perez

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Adam Lawson]

Okay I'll bite. I'm a working owner; Cloud/OpenStack/SDN architect slash
OpenStack SI business owner working with companies trying to extract value
from technology they don't understand. Or in ways they aren't familiar
with. Or with code they don't have time to build/maintain themselves.

This working group seems like we'll get to look at things from the
perspective of "what is openstack and how can we make it better for those
who want to use it" among other things. Sad reality is SI's and product
vendors make more money if OpenStack remains complicated so we'll be
working against working against a powerful money machine that funds this
project. I want OpenStack to address real non-theoretical and
non-marketing-BS cloud problems that are based in today's reality and in
advance of tomorrow's challenges. I hope we'll get that chance.

Today, it seems to me that this WG would focus un-crunching code, design
and evangelize opportunities for potential improvements for consideration
by the greater OpenStack community and the TC. No successful architecture
group I've ever participated wondered how do we can compel others to accept
our recommendations. Leave that to the business/OpenStack governance.

Ultimately, I totally agree with Clint in that if we avoid too much focus
on design enforcement, that's our first win. And in my mind, our designs
will not be absorbed nor accepted de facto anyway. I think the value will
however be recognized over time though and I'm totally down with that.

I'd like to participate with this Clint if there's room for one more. ; )

//adam


*Adam Lawson*

AQORN, Inc.
427 North Tatnall Street
Ste. 58461
Wilmington, Delaware 19801-2230
Toll-free: (844) 4-AQORN-NOW ext. 101
International: +1 302-387-4660
Direct: +1 916-246-2072

On Thu, Jun 30, 2016 at 11:16 AM, Joshua Harlow 
wrote:

> Mike Perez wrote:
>
>> On 11:31 Jun 20, Clint Byrum wrote:
>>
>>> Excerpts from Joshua Harlow's message of 2016-06-17 15:33:25 -0700:
>>>
 Thanks for getting this started Clint,

 I'm happy and excited to be involved in helping try to guide the whole
 ecosystem together (it's also why I like being in oslo) to a
 architecture that is more cohesive (and is more of something that we can
 say to our current or future children that we were all involved and
 proud to be involved in creating/maturing...).

 At a start, for said first meeting, any kind of agenda come to mind, or
 will it be more a informal gathering to start (either is fine with me)?

 I've been hesitant to fill this in too much as I'm still forming the
>>> idea, but here are the items I think are most compelling to begin with:
>>>
>>> * DLM's across OpenStack -- This is already under way[1], but it seems to
>>>have fizzled out. IMO that is because there's no working group who
>>>owns it. We need to actually write some plans.
>>>
>>
>> Not meaning to nitpick, but I don't think this is a compelling reason for
>> the
>> architecture working group. We need a group that wants to spend time on
>> reviewing the drivers being proposed. This is like saying we need the
>> architecture working group because no working group is actively reshaping
>> quotas
>> cross-project.
>>
>> With that said, I can see the architecture working group providing
>> information
>> on to a group actually reviewing/writing drivers for DLM and saying "Doing
>> mutexes with the mysql driver is crazy, I brought it in a environment and
>> have
>> such information to support that it is not reliable". THAT is useful and I
>> don't feel like people do enough of.
>>
>> My point is call your working group whatever you want (The Purple
>> Parrots), and
>> just go spearhead DLM, but don't make it about one of the most compelling
>> reasons for the existence of this group.
>>
>
> Sadly I feel if such a group formed it wouldn't be addressing the larger
> issue that this type of group is trying to address; the purple parrots
> would be a tactical team that could go do what u said, but that doesn't
> address the larger strategic goal of trying to improve the full situation
> (technical and architectural inconsistencies and 'fizzling out' solutions)
> that IMHO needs to be worked through.
>
> So yes, the tactical group needs to exist, and overall it likely will, but
> there also needs to be a strategic group that is being proactive about the
> issues and not just tactically reacting to things (which isn't imho
> healthy).
>
> -Josh
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Joshua Harlow]

Mike Perez wrote:

On 11:31 Jun 20, Clint Byrum wrote:

Excerpts from Joshua Harlow's message of 2016-06-17 15:33:25 -0700:

Thanks for getting this started Clint,

I'm happy and excited to be involved in helping try to guide the whole
ecosystem together (it's also why I like being in oslo) to a
architecture that is more cohesive (and is more of something that we can
say to our current or future children that we were all involved and
proud to be involved in creating/maturing...).

At a start, for said first meeting, any kind of agenda come to mind, or
will it be more a informal gathering to start (either is fine with me)?


I've been hesitant to fill this in too much as I'm still forming the
idea, but here are the items I think are most compelling to begin with:

* DLM's across OpenStack -- This is already under way[1], but it seems to
   have fizzled out. IMO that is because there's no working group who
   owns it. We need to actually write some plans.


Not meaning to nitpick, but I don't think this is a compelling reason for the
architecture working group. We need a group that wants to spend time on
reviewing the drivers being proposed. This is like saying we need the
architecture working group because no working group is actively reshaping quotas
cross-project.

With that said, I can see the architecture working group providing information
on to a group actually reviewing/writing drivers for DLM and saying "Doing
mutexes with the mysql driver is crazy, I brought it in a environment and have
such information to support that it is not reliable". THAT is useful and I
don't feel like people do enough of.

My point is call your working group whatever you want (The Purple Parrots), and
just go spearhead DLM, but don't make it about one of the most compelling
reasons for the existence of this group.


Sadly I feel if such a group formed it wouldn't be addressing the larger 
issue that this type of group is trying to address; the purple parrots 
would be a tactical team that could go do what u said, but that doesn't 
address the larger strategic goal of trying to improve the full 
situation (technical and architectural inconsistencies and 'fizzling 
out' solutions) that IMHO needs to be worked through.


So yes, the tactical group needs to exist, and overall it likely will, 
but there also needs to be a strategic group that is being proactive 
about the issues and not just tactically reacting to things (which isn't 
imho healthy).


-Josh

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Clint Byrum]

Excerpts from Mike Perez's message of 2016-06-30 07:50:42 -0700:
> On 11:31 Jun 20, Clint Byrum wrote:
> > Excerpts from Joshua Harlow's message of 2016-06-17 15:33:25 -0700:
> > > Thanks for getting this started Clint,
> > > 
> > > I'm happy and excited to be involved in helping try to guide the whole 
> > > ecosystem together (it's also why I like being in oslo) to a 
> > > architecture that is more cohesive (and is more of something that we can 
> > > say to our current or future children that we were all involved and 
> > > proud to be involved in creating/maturing...).
> > > 
> > > At a start, for said first meeting, any kind of agenda come to mind, or 
> > > will it be more a informal gathering to start (either is fine with me)?
> > > 
> > 
> > I've been hesitant to fill this in too much as I'm still forming the
> > idea, but here are the items I think are most compelling to begin with:
> > 
> > * DLM's across OpenStack -- This is already under way[1], but it seems to
> >   have fizzled out. IMO that is because there's no working group who
> >   owns it. We need to actually write some plans.
> 
> Not meaning to nitpick, but I don't think this is a compelling reason for the
> architecture working group. We need a group that wants to spend time on
> reviewing the drivers being proposed. This is like saying we need the
> architecture working group because no working group is actively reshaping 
> quotas
> cross-project. 
> 

That sounds like a reasoned deep argument, not a nitpick, so thank you
for making it.

However, I don't think lack of drivers is standing in the way of a DLM
effort. It is a lack of coordination. There was a race to the finish line
to make Consul and etcd drivers, but then, like the fish in finding Nemo,
the drivers are in bags floating in the bay.. now what?

Nobody owns this effort. Everybody gets busy. Nothing gets done. We
continue to bring it up in the hallway and wish we had time.

This is just a place to have a meeting and some people who get together
and say "hey is that done yet? Do you need help? is that still a
priority?". Could we do this as part of Oslo? Yes! But, I want this to
be about going one step higher, and actually taking the implementations
into the respective projects.

> With that said, I can see the architecture working group providing information
> on to a group actually reviewing/writing drivers for DLM and saying "Doing
> mutexes with the mysql driver is crazy, I brought it in a environment and have
> such information to support that it is not reliable". THAT is useful and I
> don't feel like people do enough of.
> 

Ugh, no, I don't want it to be a group of information providers. I'm
not talking about an Architecture Review Board.

It's a group for doers. People who design together, and build with
others. The DLM spec process was actually one of the reasons I wanted
to create this group. We did such a great job on the design side, but
we didn't really stick together on it and push it all the way through.
This group is my idea of how we stick together and complete work like
that.

> My point is call your working group whatever you want (The Purple Parrots), 
> and
> just go spearhead DLM, but don't make it about one of the most compelling
> reasons for the existence of this group.
> 

The Purple Parrots is already taken -- it's my new band. We're playing
at the house of blues on August 3rd.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Mike Perez]

On 11:31 Jun 20, Clint Byrum wrote:
> Excerpts from Joshua Harlow's message of 2016-06-17 15:33:25 -0700:
> > Thanks for getting this started Clint,
> > 
> > I'm happy and excited to be involved in helping try to guide the whole 
> > ecosystem together (it's also why I like being in oslo) to a 
> > architecture that is more cohesive (and is more of something that we can 
> > say to our current or future children that we were all involved and 
> > proud to be involved in creating/maturing...).
> > 
> > At a start, for said first meeting, any kind of agenda come to mind, or 
> > will it be more a informal gathering to start (either is fine with me)?
> > 
> 
> I've been hesitant to fill this in too much as I'm still forming the
> idea, but here are the items I think are most compelling to begin with:
> 
> * DLM's across OpenStack -- This is already under way[1], but it seems to
>   have fizzled out. IMO that is because there's no working group who
>   owns it. We need to actually write some plans.

Not meaning to nitpick, but I don't think this is a compelling reason for the
architecture working group. We need a group that wants to spend time on
reviewing the drivers being proposed. This is like saying we need the
architecture working group because no working group is actively reshaping quotas
cross-project. 

With that said, I can see the architecture working group providing information
on to a group actually reviewing/writing drivers for DLM and saying "Doing
mutexes with the mysql driver is crazy, I brought it in a environment and have
such information to support that it is not reliable". THAT is useful and I
don't feel like people do enough of.

My point is call your working group whatever you want (The Purple Parrots), and
just go spearhead DLM, but don't make it about one of the most compelling
reasons for the existence of this group.

-- 
Mike Perez

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Clint Byrum]

Thanks everyone for participating and remaining positive and focused
on improving OpenStack. I've posted a review, and I'd like to encourage
everyone to move any future discussion of the Architecture Working group
to that review.

https://review.openstack.org/335141

Excerpts from Clint Byrum's message of 2016-06-17 14:52:43 -0700:
> ar·chi·tec·ture
> ˈärkəˌtek(t)SHər/
> noun
> noun: architecture
> 
> 1. 
> 
> the art or practice of designing and constructing buildings.
> 
> synonyms:building design, building style, planning, building, 
> construction; 
> 
> formalarchitectonics 
> 
> "modern architecture"
> 
> the style in which a building is designed or constructed, especially with 
> regard to a specific period, place, or culture.
> 
> plural noun: architectures
> 
> "Victorian architecture"
> 
> 2. 
> 
> the complex or carefully designed structure of something.
> 
> "the chemical architecture of the human brain"
> 
> the conceptual structure and logical organization of a computer or 
> computer-based system.
> 
> "a client/server architecture"
> 
> synonyms:structure, construction, organization, layout, design, build, 
> anatomy, makeup; 
> 
> informalsetup 
> 
> "the architecture of a computer system"
> 
> 
> Introduction
> =
> 
> OpenStack is a big system. We have debated what it actually is [1],
> and there are even t-shirts to poke fun at the fact that we don't have
> good answers.
> 
> But this isn't what any of us wants. We'd like to be able to point
> at something and proudly tell people "This is what we designed and
> implemented."
> 
> And for each individual project, that is a possibility. Neutron can
> tell you they designed how their agents and drivers work. Nova can
> tell you that they designed the way conductors handle communication
> with API nodes and compute nodes. But when we start talking about how
> they interact with each other, it's clearly just a coincidental mash of
> de-facto standards and specs that don't help anyone make decisions when
> refactoring or adding on to the system.
> 
> Oslo and cross-project initiatives have brought some peace and order
> to the implementation and engineering processes, but not to the design
> process. New ideas still start largely in the project where they are
> needed most, and often conflict with similar decisions and ideas in other
> projects [dlm, taskflow, tooz, service discovery, state machines, glance
> tasks, messaging patterns, database patterns, etc. etc.]. Often times this
> creates a log jam where none of the projects adopt a solution that would
> align with others. Most of the time when things finally come to a head
> these things get done in a piecemeal fashion, where it's half done here,
> 1/3 over there, 1/4 there, and 3/4 over there..., which to the outside
> looks like  chaos, because that's precisely what it is.
> 
> And this isn't always a technical design problem. OpenStack, for instance,
> isn't really a micro service architecture. Of course, it might look like
> that in diagrams [2], but we all know it really isn't. The compute node is
> home to agents for every single concern, and the API interactions between
> the services is too tightly woven to consider many of them functional
> without the same lockstep version of other services together. A game to
> play is ask yourself what would happen if a service was isolated on its
> own island, how functional would its API be, if at all. Is this something
> that we want? No. But there doesn't seem to be a place where we can go
> to actually design, discuss, debate, and ratify changes that would help
> us get to the point of gathering the necessary will and capability to
> enact these efforts.
> 
> Maybe nova-compute should be isolated from nova, with an API that
> nova, cinder and neutron talk to. Maybe we should make the scheduler
> cross-project aware and capable of scheduling more than just nova
> instances. Maybe we should have experimental groups that can look at how
> some of this functionality could perhaps be delegated to non-openstack
> projects. We hear that Mesos, for example to help with the scheduling
> aspects, but how do we discuss these outside hijacking threads on the
> mailing list? These are things that we all discuss in the hallways
> and bars and parties at the summit, but because they cross projects at
> the design level, and are inherently a lot of social and technical and
> exploratory work, Many of us fear we never get to a place of turning
> our dreams into reality.
> 
> So, with that, I'd like to propose the creation of an Architecture Working
> Group. This group's charge would not be design by committee, but a place
> for architects to share their designs and gain support across projects
> to move forward with and ratify architectural decisions. That includes
> coordinating exploratory work that may turn into being the base of further
> architectural decisions for OpenStack. I 

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[darren chan]

It appears this group could assist with the Architecture Design Guide, which is 
undergoing a content restructure. We are looking for more technical 
contributors to provide guidance and contribute information to the guide. Shaun 
O'Meara has written a spec: 
https://specs.openstack.org/openstack/docs-specs/specs/newton/arch-guide-restructure.html.
        
If you are interested in helping out, contact Shilla or me, or feel free to 
join our biweekly Ops Guide/Arch Guide Specialty team meeting. For more 
details, see https://wiki.openstack.org/wiki/Documentation/OpsGuide.

Thanks!


On Saturday, 25 June 2016, 2:50, Clint Byrum  wrote:
 

 Excerpts from Zhipeng Huang's message of 2016-06-24 18:15:30 +0200:
> Hi Clint and Amrith,
> 
> Are you guys already working on the proposal ? Is there any public access
> to see the first draft ?
> 

I've started writing something up, and I hope to submit it for review
next week.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


  __
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Clint Byrum]

Excerpts from Zhipeng Huang's message of 2016-06-24 18:15:30 +0200:
> Hi Clint and Amrith,
> 
> Are you guys already working on the proposal ? Is there any public access
> to see the first draft ?
> 

I've started writing something up, and I hope to submit it for review
next week.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Zhipeng Huang]

Hi Clint and Amrith,

Are you guys already working on the proposal ? Is there any public access
to see the first draft ?

On Wed, Jun 22, 2016 at 11:23 PM, Mike Perez  wrote:

> On 10:27 Jun 20, Clint Byrum wrote:
> > Excerpts from Doug Wiegley's message of 2016-06-20 10:40:56 -0600:
> > > So, it sounds like you’ve just described the job of the TC. And they
> have so far refused to define OpenStack, leading to a series of derivative
> decisions that seem … inconsistent over time.
> > >
> > > How is this body going to be different?
> > >
> > > How will it have any teeth, and not just end up with the standard
> entrenched projects ignoring it?
> > >
>
> 
>
> > Engineers have no effective place to turn to right now when there is
> > a lack of design. The TC could of course do it, but what I want to do
> > is have a more open and free-flowing group that are laser focused on
> > providing support for the design of the system. I want to work out with
> > the community at large how we add weight to the designs we choose, and
> > one good option is for the Architecture Working Group to make proposals
> > to the openstack-specs repo, which the TC would ultimately approve.
> > That's not a new process, we already have it:
> >
> > http://docs.openstack.org/project-team-guide/cross-project.html
>
> Thanks for reminding me to update that. These are actually approved by the
> cross-project spec team now.
>
>
> http://governance.openstack.org/resolutions/20160414-grant-cross-project-spec-team-voting.html
>
> --
> Mike Perez
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Zhipeng (Howard) Huang

Standard Engineer
IT Standard & Patent/IT Prooduct Line
Huawei Technologies Co,. Ltd
Email: huangzhip...@huawei.com
Office: Huawei Industrial Base, Longgang, Shenzhen

(Previous)
Research Assistant
Mobile Ad-Hoc Network Lab, Calit2
University of California, Irvine
Email: zhipe...@uci.edu
Office: Calit2 Building Room 2402

OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Mike Perez]

On 10:27 Jun 20, Clint Byrum wrote:
> Excerpts from Doug Wiegley's message of 2016-06-20 10:40:56 -0600:
> > So, it sounds like you’ve just described the job of the TC. And they have 
> > so far refused to define OpenStack, leading to a series of derivative 
> > decisions that seem … inconsistent over time.
> > 
> > How is this body going to be different?
> > 
> > How will it have any teeth, and not just end up with the standard 
> > entrenched projects ignoring it?
> > 


 
> Engineers have no effective place to turn to right now when there is
> a lack of design. The TC could of course do it, but what I want to do
> is have a more open and free-flowing group that are laser focused on
> providing support for the design of the system. I want to work out with
> the community at large how we add weight to the designs we choose, and
> one good option is for the Architecture Working Group to make proposals
> to the openstack-specs repo, which the TC would ultimately approve.
> That's not a new process, we already have it:
> 
> http://docs.openstack.org/project-team-guide/cross-project.html

Thanks for reminding me to update that. These are actually approved by the
cross-project spec team now.

http://governance.openstack.org/resolutions/20160414-grant-cross-project-spec-team-voting.html

-- 
Mike Perez

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Amrith Kumar]

> -Original Message-
> From: Clint Byrum [mailto:cl...@fewbar.com]
> Sent: Wednesday, June 22, 2016 2:25 PM
> To: openstack-dev <openstack-dev@lists.openstack.org>
> Subject: Re: [openstack-dev] [all] Proposal: Architecture Working Group
> 
> Excerpts from Amrith Kumar's message of 2016-06-22 13:15:03 +:
> > Clint,
> >
> > In your original email, you proposed "So, with that, I'd like to propose
> the creation of an Architecture Working Group. This group's charge would
> not be design by committee, but a place for architects to share their
> designs and gain support across projects to move forward with and ratify
> architectural decisions."
> >
> > I like parts of this, and parts of this trouble me. But, I volunteered
> to be part of this activity because I have a real problem that this group
> could help me solve and I bet there are others who have this problem as
> well.
> >
> > As you say, there are often problems, questions, and challenges of an
> architectural nature, that have a scope larger than a single project. I
> would like there to be a forum whose primary focus is to provide an avenue
> where these can be discussed. I would like it to be a place less noisy
> than "take it to the ML" and be a place where one could pose these
> questions and potentially discuss solutions that other projects have
> adopted.
> >
> > The part I'm uncomfortable is the implied decision making authority of
> "ratifying architectural decisions". To ratify implies the ability to make
> official, the ability to "approve and sanction formally" and I ask whence
> came this power and authority?
> >
> > Who currently has this power and authority, and is that individual or
> group delegating it to this working group?
> >
> 
> When I say ratify there, what I mean is that this group would have regular
> members who work on the group. To ratify something, a majority of them
> would at least agree that this was something worth the group's time, and
> that the group should publish their architecture decisions publicly. The
> membership, I think, should be voluntary, and the only requirement be
> that members regularly participate in discussions and voting.
> 
> Formality is a useful tool here, which is the reason I chose the word
> 'ratify'. It asks that those who want to propose new ideas do so in an
> efficient manner that doesn't make noise on the mailing list and force
> everyone to come up with an opinion on the spot or forever lose the
> idea. We get a log of proposals, objections, and reasoning, to go along
> with our log of successes and failures in taking the proposals to reality.
> 
> But the only power this group wields is over itself. Of course,
> collaboration with the project teams is _critical_ for the success
> of these proposals. And if we succeed in improving some projects, but
> others resist, then it's up to those projects that have been improved
> to support us pushing forward or not.
> 
> This isn't all that different than the way Oslo specs and OpenStack
> specs work now. It's just that we'll have a group that organizes the
> efforts and keeps pressure on them.
> 

[amrith] May I then request that you use a word other than 'ratify'. You will 
understand my root of my concern in the link below.

http://www.merriam-webster.com/dictionary/ratify

What you are describing is more like "bless" or "endorse" and not "ratify" :)

> > While this ML thread is very interesting, it is also beginning to
> fragment and I would like to propose a spec in Gerrit with a draft charter
> for this working group and a review there.
> >
> 
> You're spot on Amrith. I've been noodling on a mission statement and
> was going to bring it up at next week's TC meeting, but we don't have to
> wait for that. Let's draft a charter now. Any suggestions on where that
> should be submitted? openstack-specs I suppose? Governance?
> 

[amrith] I will help with that. I have no idea where it should be proposed but 
I do know that there's no TC meeting next week. I will be sure to bend several 
of the TC members ears about this in Ann Arbor though :)

> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Clint Byrum]

Excerpts from Amrith Kumar's message of 2016-06-22 13:15:03 +:
> Clint,
> 
> In your original email, you proposed "So, with that, I'd like to propose the 
> creation of an Architecture Working Group. This group's charge would not be 
> design by committee, but a place for architects to share their designs and 
> gain support across projects to move forward with and ratify architectural 
> decisions."
> 
> I like parts of this, and parts of this trouble me. But, I volunteered to be 
> part of this activity because I have a real problem that this group could 
> help me solve and I bet there are others who have this problem as well.
> 
> As you say, there are often problems, questions, and challenges of an 
> architectural nature, that have a scope larger than a single project. I would 
> like there to be a forum whose primary focus is to provide an avenue where 
> these can be discussed. I would like it to be a place less noisy than "take 
> it to the ML" and be a place where one could pose these questions and 
> potentially discuss solutions that other projects have adopted.
> 
> The part I'm uncomfortable is the implied decision making authority of 
> "ratifying architectural decisions". To ratify implies the ability to make 
> official, the ability to "approve and sanction formally" and I ask whence 
> came this power and authority?
> 
> Who currently has this power and authority, and is that individual or group 
> delegating it to this working group?
> 

When I say ratify there, what I mean is that this group would have regular
members who work on the group. To ratify something, a majority of them
would at least agree that this was something worth the group's time, and
that the group should publish their architecture decisions publicly. The
membership, I think, should be voluntary, and the only requirement be
that members regularly participate in discussions and voting.

Formality is a useful tool here, which is the reason I chose the word
'ratify'. It asks that those who want to propose new ideas do so in an
efficient manner that doesn't make noise on the mailing list and force
everyone to come up with an opinion on the spot or forever lose the
idea. We get a log of proposals, objections, and reasoning, to go along
with our log of successes and failures in taking the proposals to reality.

But the only power this group wields is over itself. Of course,
collaboration with the project teams is _critical_ for the success
of these proposals. And if we succeed in improving some projects, but
others resist, then it's up to those projects that have been improved
to support us pushing forward or not.

This isn't all that different than the way Oslo specs and OpenStack
specs work now. It's just that we'll have a group that organizes the
efforts and keeps pressure on them.

> While this ML thread is very interesting, it is also beginning to fragment 
> and I would like to propose a spec in Gerrit with a draft charter for this 
> working group and a review there.
> 

You're spot on Amrith. I've been noodling on a mission statement and
was going to bring it up at next week's TC meeting, but we don't have to
wait for that. Let's draft a charter now. Any suggestions on where that
should be submitted? openstack-specs I suppose? Governance?

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Flavio Percoco]

On 20/06/16 11:31 -0700, Clint Byrum wrote:

Excerpts from Joshua Harlow's message of 2016-06-17 15:33:25 -0700:

Thanks for getting this started Clint,

I'm happy and excited to be involved in helping try to guide the whole
ecosystem together (it's also why I like being in oslo) to a
architecture that is more cohesive (and is more of something that we can
say to our current or future children that we were all involved and
proud to be involved in creating/maturing...).

At a start, for said first meeting, any kind of agenda come to mind, or
will it be more a informal gathering to start (either is fine with me)?



I've been hesitant to fill this in too much as I'm still forming the
idea, but here are the items I think are most compelling to begin with:

* DLM's across OpenStack -- This is already under way[1], but it seems to
 have fizzled out. IMO that is because there's no working group who
 owns it. We need to actually write some plans.

* Messaging patterns -- There was a recent discussion about nuances in
 oslo.messaging's implementation that vary driver to driver. I'd like to
 make sure we have all of the ways messaging is used written down and
 make sure groups have guidance on how each one should (or shouldn't)
 be used, including documentation of anti-patterns, and a plan for
 simplifying communication between processes in OpenStack where
 possible.

* True Microservices -- OpenStack's services are heavily
 interdependent. It makes it hard for an engineer to make an impact
 without becoming an expert on all of them, and it also leads to a heavy
 burden on operators and QA as they end up having to debug the system
 as a whole. We should write down how the system is intertwined now, and
 develop plans for unwinding that and allowing each service to stand on
 its own, including having stronger testing in isolation. (This includes
 exploring the thought that nova-compute could become its own project
 that all of the other things communicate with over well defined APIs).

These could keep a small group of architects and engineers busy for a
year or more. I'm sure others have things they'd like to design and
improve in OpenStack at this level.

Once we have broad agreement on such a group, and perhaps some guidance
from the TC, we can propose an agenda and prioritize efforts as part of
the first few meetings.


As you mentioned, these are topics that will keep a group of folks busy for a
year or more. Would it make sense to just pick one for now as a starting task
for this group and see how things evolve?

Flavio


[1] 
http://specs.openstack.org/openstack/openstack-specs/specs/chronicles-of-a-dlm.html

-Josh

Clint Byrum wrote:
> ar·chi·tec·ture
> ˈärkəˌtek(t)SHər/
> noun
> noun: architecture
>
>  1.
>
>  the art or practice of designing and constructing buildings.
>
>  synonyms:building design, building style, planning, building, 
construction;
>
>  formalarchitectonics
>
>  "modern architecture"
>
>  the style in which a building is designed or constructed, especially 
with regard to a specific period, place, or culture.
>
>  plural noun: architectures
>
>  "Victorian architecture"
>
>  2.
>
>  the complex or carefully designed structure of something.
>
>  "the chemical architecture of the human brain"
>
>  the conceptual structure and logical organization of a computer or 
computer-based system.
>
>  "a client/server architecture"
>
>  synonyms:structure, construction, organization, layout, design, build, 
anatomy, makeup;
>
>  informalsetup
>
>  "the architecture of a computer system"
>
>
> Introduction
> =
>
> OpenStack is a big system. We have debated what it actually is [1],
> and there are even t-shirts to poke fun at the fact that we don't have
> good answers.
>
> But this isn't what any of us wants. We'd like to be able to point
> at something and proudly tell people "This is what we designed and
> implemented."
>
> And for each individual project, that is a possibility. Neutron can
> tell you they designed how their agents and drivers work. Nova can
> tell you that they designed the way conductors handle communication
> with API nodes and compute nodes. But when we start talking about how
> they interact with each other, it's clearly just a coincidental mash of
> de-facto standards and specs that don't help anyone make decisions when
> refactoring or adding on to the system.
>
> Oslo and cross-project initiatives have brought some peace and order
> to the implementation and engineering processes, but not to the design
> process. New ideas still start largely in the project where they are
> needed most, and often conflict with similar decisions and ideas in other
> projects [dlm, taskflow, tooz, service discovery, state machines, glance
> tasks, messaging patterns, database patterns, etc. etc.]. Often times this
> creates a log jam where none of the projects adopt a solution that would
> align with others. Most of the 

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Flavio Percoco]

On 21/06/16 12:12 -0600, Doug Wiegley wrote:



On Jun 21, 2016, at 11:29 AM, Jay Pipes  wrote:

[snip]

Perhaps you weren't around for the endless (and pointless) discussions around what is "the core of 
OpenStack"? Or you weren't around for the endless (and conflicting, contradictory, and religious) 
battles that were waged during the old "incubation" and "graduation" procedures?

Ask Clint, Flavio and Devananda how fruitful the Marconi/Zaqar "architectural 
review" by the TC was.


That would be interesting to hear, and how this group would avoid and/or help 
such issues.


This is actually a good example that not only involved the TC but also folks
from outside the TC. A clear sign of collaboration between people interested in
the topic, folks involved in the governance model and folks actually working in
the project. Most of the discussions are googlable but here's one that came out
of the good all graduation discussion[0].

What we saw in those discussions was effectively a WG in full action working
with the TC. I wish we'd have had a better structure for such group back then
but we had a bigger problem to solve.

[0] 
http://lists.openstack.org/pipermail/openstack-dev/2014-September/044845.html

[long snip]

Flavio

--
@flaper87
Flavio Percoco


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Amrith Kumar]

Clint,

In your original email, you proposed "So, with that, I'd like to propose the 
creation of an Architecture Working Group. This group's charge would not be 
design by committee, but a place for architects to share their designs and gain 
support across projects to move forward with and ratify architectural 
decisions."

I like parts of this, and parts of this trouble me. But, I volunteered to be 
part of this activity because I have a real problem that this group could help 
me solve and I bet there are others who have this problem as well.

As you say, there are often problems, questions, and challenges of an 
architectural nature, that have a scope larger than a single project. I would 
like there to be a forum whose primary focus is to provide an avenue where 
these can be discussed. I would like it to be a place less noisy than "take it 
to the ML" and be a place where one could pose these questions and potentially 
discuss solutions that other projects have adopted.

The part I'm uncomfortable is the implied decision making authority of 
"ratifying architectural decisions". To ratify implies the ability to make 
official, the ability to "approve and sanction formally" and I ask whence came 
this power and authority?

Who currently has this power and authority, and is that individual or group 
delegating it to this working group?

While this ML thread is very interesting, it is also beginning to fragment and 
I would like to propose a spec in Gerrit with a draft charter for this working 
group and a review there.

-amrith


> -Original Message-
> From: Clint Byrum [mailto:cl...@fewbar.com]
> Sent: Tuesday, June 21, 2016 2:34 PM
> To: openstack-dev <openstack-dev@lists.openstack.org>
> Subject: Re: [openstack-dev] [all] Proposal: Architecture Working Group
> 
> Excerpts from Jay Pipes's message of 2016-06-21 12:47:46 -0400:
> > On 06/21/2016 04:25 AM, Chris Dent wrote:
> > > However, I worry deeply that it could become astronauts with finger
> > > paints.
> >
> > Yes. This.
> >
> > I will happily take software design suggestions from people that
> > demonstrate with code and benchmarks that their suggestion actually
> > works outside of the theoretical/academic/MartinFowler landscape and
> > actually solves a real problem better than existing code.
> >
> 
> So, I want to be careful not to descend too far into reductionism.
> Nobody is suggesting that an architecture WG goes off into the corner
> and applies theory to problems without practical application.
> 
> However, some things aren't measured in how fast, or even how reliable
> a piece of code is. Some things are measured in how understandable the
> actual code and deployment is.
> 
> If we architect a DLM solution, refactor out all of the one-off DLM-esque
> things, and performance and measured reliability stay flat, did we fail?
> What about when the locks break, and an operator has _one_ way to fix
> locks, instead of 5? How do we measure that?
> 
> So I think what I want to focus on is: We have to have some real basis
> on which to agree that an architecture that is selected is worth the
> effort to refactor things around it. But I can't promise that every
> problem has an objectively measurable solution. Part of the point of
> having a working group is to develop a discipline for evaluating hard
> problems like that.
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Flavio Percoco]

On 22/06/16 10:52 +0200, Thierry Carrez wrote:

Doug Wiegley wrote:

So I'd argue that you need both. You need the TC whenever a hard call has to be 
made, but in order to minimize the number of those hard calls (and favor 
consensus building) you also need working groups to build a bottom-up 
reasonable way forward.


This reads very strange to me, as I’d expect a group of technical leaders to 
both make hard calls *and* to be able to build consensus on overall direction 
and vision. They’re two sides of the same coin. What is it about our process 
that means the TC can’t build consensus on direction, but can only impose its 
will? I expect you didn’t mean it to sound that way, though. Is the workload 
too high on the bookkeeping to prevent the vision building?


Some TC members definitely have the time (and will, and experience) to 
work on architecture definition. A number of TC members (including 
Jay, Robert and myself) actually called for an "architecture 
workgroup" within the TC (the same way we have a communications 
workgroup, or a project team guide documentation workgroup) for some 
time now, but it never fully formed. One of the reasons is that we are 
all very busy and were all waiting for someone else to start it up. At 
the same time, I see no reason to limit that workgroup to TC members. 
There are other people interested. The two groups don't have to be 
mutually exclusive.


So I'm very happy that Clint picked up the ball where we left it. I 
expect TC members to participate in the group.


This ...


[...]
Don’t get me wrong, I welcome this initiative. I find it mildly disconcerting 
that the folks that I thought we were electing to fill this role will instead 
be filled by others, but the vacuum does need to be filled, and I thank Clint 
for stepping up.


And I don't see why architecture should be the reserved domain of 
precisely 13 people. We are electing the TC to make final calls where 
needed. Not to be the only people that are allowed to work on 
architecture. For that, an open workgroup sounds a thousand times 
better.


... and this!

Not only I believe architecture shouldn't be a reserved domain of the TC but I
also believe that *ANYONE* willing to step up and *contribute* to the TC tasks
should totally feel free to do so. Contributing is what all this is about. It's
not a matter of whether the TC is doing "well" its job (or at all) but a matter
of enabling others to contribute to fixing the pressing issues we have in the
community.

Part of the analysis and work made by the TC was to identify several of the pain
points the community has and to find the best way to work those issues out
without burning the members of the TC out and by making sure others feel enabled
to contribute and be awesome.

I don't see why having a dedicated group of people to solve some of the
identified issues should be seen as an overstep on the TC duties or, even worse,
as the TC weren't doing its job. My point of view is that this group of humans
is contributing and I'm happy to see that happening.

Flavio

--
@flaper87
Flavio Percoco


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Thierry Carrez]

Doug Wiegley wrote:

So I'd argue that you need both. You need the TC whenever a hard call has to be 
made, but in order to minimize the number of those hard calls (and favor 
consensus building) you also need working groups to build a bottom-up 
reasonable way forward.


This reads very strange to me, as I’d expect a group of technical leaders to 
both make hard calls *and* to be able to build consensus on overall direction 
and vision. They’re two sides of the same coin. What is it about our process 
that means the TC can’t build consensus on direction, but can only impose its 
will? I expect you didn’t mean it to sound that way, though. Is the workload 
too high on the bookkeeping to prevent the vision building?


Some TC members definitely have the time (and will, and experience) to 
work on architecture definition. A number of TC members (including Jay, 
Robert and myself) actually called for an "architecture workgroup" 
within the TC (the same way we have a communications workgroup, or a 
project team guide documentation workgroup) for some time now, but it 
never fully formed. One of the reasons is that we are all very busy and 
were all waiting for someone else to start it up. At the same time, I 
see no reason to limit that workgroup to TC members. There are other 
people interested. The two groups don't have to be mutually exclusive.


So I'm very happy that Clint picked up the ball where we left it. I 
expect TC members to participate in the group.



[...]
Don’t get me wrong, I welcome this initiative. I find it mildly disconcerting 
that the folks that I thought we were electing to fill this role will instead 
be filled by others, but the vacuum does need to be filled, and I thank Clint 
for stepping up.


And I don't see why architecture should be the reserved domain of 
precisely 13 people. We are electing the TC to make final calls where 
needed. Not to be the only people that are allowed to work on 
architecture. For that, an open workgroup sounds a thousand times better.


--
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Barrett, Carol L]

> On Jun 21, 2016, at 2:56 AM, Thierry Carrez  wrote:
> 
> Chris Dent wrote:
>> On Mon, 20 Jun 2016, Doug Wiegley wrote:
>>> On Jun 21, 2016, at 2:19 PM, Carol Barrett  
>>> wrote:
>>> So, it sounds like you've just described the job of the TC. And they 
>>> have so far refused to define OpenStack, leading to a series of 
>>> derivative decisions that seem ... inconsistent over time.
>> 
>> Thanks for writing down what I was thinking. I agree that OpenStack 
>> needs some architectural vision, direction, leadership, call it what 
>> you will. Every time I've voted for the _Technical_ Committee that 
>> leadership is what I've wanted my vote to be creating.
> 
> The TC is a representative body which is elected to make top-down decisions 
> on OpenStack. However, as much as our community loves the idea of "technical 
> leadership" and "vision", they hate the top-down decisions that come with it 
> (especially when that top-down decision doesn't go their way). They prefer 
> bottom-up consensus.
> 
> So I'd argue that you need both. You need the TC whenever a hard call has to 
> be made, but in order to minimize the number of those hard calls (and favor 
> consensus building) you also need working groups to build a bottom-up 
> reasonable way forward.

>>This reads very strange to me, as I'd expect a group of technical leaders to 
>>both make hard calls *and* to be able to build consensus on overall direction 
>>and vision. They're >>two sides of the same coin. What is it about our 
>>process that means the TC can't build consensus on direction, but can only 
>>impose its will? I expect you didn't mean it to >>sound that way, though. Is 
>>the workload too high on the bookkeeping to prevent the vision building? Are 
>>we too afraid of the implications of defining 'what is openstack?', >>and 
>>what it might mean to existing projects and the community? I'd think that in 
>>the long-run, it'd prevent seemingly unrelated topics from seeming to go 
>>sideways so >>often, and prevent a lot of these "hard calls".

+1. Making decisions is an element of being a leader. As a community, I believe 
we need this role filled.

>>But, I'm also on the fringe that is very ready to call the "big tent" a 
>>failed experiment in attempting to avoid hard calls, too.

> 
>> It may be that an architecture working group can provide some 
>> guidance that people will find useful. Against the odds I think those 
>> of us in the API-WG have actually managed to have a positive 
>> influence. We've not shaken things down to the foundations from which 
>> a great a glorious future may be born -- a lot of compromises have 
>> been made and not everybody wants to play along -- but things are 
>> going in the right direction, for some people, in some projects.
>> Maybe a similar thing can happen with architecture.
> 
> That is my hope. I see the API WG and the Architecture WG as groups of 
> experts in specific domains preparing recommendations and long-term plans. 
> They don't have authority to force them onto projects. Ideally projects adopt 
> them because they see them as the right way to do things.
> 
> And for the very few things that the TC deems necessary for OpenStack and 
> where bottom-up didn't get it in a specific project (if all else fails), the 
> TC can make a top-down request to a project to do things a certain way. The 
> project can them either comply or reject the TC oversight and become an 
> unofficial project.

>>Don't get me wrong, I welcome this initiative. I find it mildly disconcerting 
>>that the folks that I thought we were electing to fill this role will instead 
>>be filled by others, but the vacuum does need to be filled, and I thank Clint 
>>for stepping up.
+1. I appreciate Clint making this proposal. I think a cohesive, consistent 
architecture across OpenStack is crucial to our long term efficiency and 
sustaining a high rate of innovation.

Thanks,
Carol


> 
> --
> Thierry Carrez (ttx)
> 
> __
>  OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: 
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Clint Byrum]

Excerpts from Jay Pipes's message of 2016-06-21 12:47:46 -0400:
> On 06/21/2016 04:25 AM, Chris Dent wrote:
> > However, I worry deeply that it could become astronauts with finger
> > paints.
> 
> Yes. This.
> 
> I will happily take software design suggestions from people that 
> demonstrate with code and benchmarks that their suggestion actually 
> works outside of the theoretical/academic/MartinFowler landscape and 
> actually solves a real problem better than existing code.
> 

So, I want to be careful not to descend too far into reductionism.
Nobody is suggesting that an architecture WG goes off into the corner
and applies theory to problems without practical application.

However, some things aren't measured in how fast, or even how reliable
a piece of code is. Some things are measured in how understandable the
actual code and deployment is.

If we architect a DLM solution, refactor out all of the one-off DLM-esque
things, and performance and measured reliability stay flat, did we fail?
What about when the locks break, and an operator has _one_ way to fix
locks, instead of 5? How do we measure that?

So I think what I want to focus on is: We have to have some real basis
on which to agree that an architecture that is selected is worth the
effort to refactor things around it. But I can't promise that every
problem has an objectively measurable solution. Part of the point of
having a working group is to develop a discipline for evaluating hard
problems like that.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Clint Byrum]

Excerpts from Jay Pipes's message of 2016-06-21 13:29:32 -0400:
> On 06/21/2016 12:53 PM, Doug Wiegley wrote:
> > Don’t get me wrong, I welcome this initiative. I find it mildly
> > disconcerting that the folks that I thought we were electing to fill
> > this role will instead be filled by others, but the vacuum does need
> > to be filled, and I thank Clint for stepping up.
> 
> I don't particularly think it's a vacuum that can be filled by a small 
> group of "architects". My experience is that unless said architects are 
> actually involved in building the software at hand and have a good 
> understanding of why certain design choices were originally made in the 
> various projects, the end deliverable of such a group tends to be the 
> software equivalent of silly putty -- fun to play with but in the end, 
> relatively free of practical value.
> 

I so agree with this. I don't want this group to turn into an ivory
tower shaped fountain of suggestions. That is my nightmare actually!

What I want it to be is a group that facilitates the people who build
the components of the system coming together to collaborate on large
refactoring efforts and improving our ability to design the system
together, rather than in silos.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Doug Wiegley]

> On Jun 21, 2016, at 11:29 AM, Jay Pipes  wrote:
> 
> On 06/21/2016 12:53 PM, Doug Wiegley wrote:
>>> On Jun 21, 2016, at 2:56 AM, Thierry Carrez 
>>> wrote:
>>> Chris Dent wrote:
 On Mon, 20 Jun 2016, Doug Wiegley wrote:
> So, it sounds like you’ve just described the job of the TC. And
> they have so far refused to define OpenStack, leading to a
> series of derivative decisions that seem … inconsistent over
> time.
 
 Thanks for writing down what I was thinking. I agree that
 OpenStack needs some architectural vision, direction, leadership,
 call it what you will. Every time I've voted for the _Technical_
 Committee that leadership is what I've wanted my vote to be
 creating.
>>> 
>>> The TC is a representative body which is elected to make top-down
>>> decisions on OpenStack. However, as much as our community loves the
>>> idea of "technical leadership" and "vision", they hate the top-down
>>> decisions that come with it (especially when that top-down decision
>>> doesn't go their way). They prefer bottom-up consensus.
>>> 
>>> So I'd argue that you need both. You need the TC whenever a hard
>>> call has to be made, but in order to minimize the number of those
>>> hard calls (and favor consensus building) you also need working
>>> groups to build a bottom-up reasonable way forward.
>> 
>> This reads very strange to me, as I’d expect a group of technical
>> leaders to both make hard calls *and* to be able to build consensus
>> on overall direction and vision. They’re two sides of the same coin.
>> What is it about our process that means the TC can’t build consensus
>> on direction, but can only impose its will? I expect you didn’t mean
>> it to sound that way, though. Is the workload too high on the
>> bookkeeping to prevent the vision building? Are we too afraid of the
>> implications of defining ‘what is openstack?’, and what it might mean
>> to existing projects and the community? I’d think that in the
>> long-run, it’d prevent seemingly unrelated topics from seeming to go
>> sideways so often, and prevent a lot of these “hard calls”.
> 
> Perhaps you weren't around for the endless (and pointless) discussions around 
> what is "the core of OpenStack"? Or you weren't around for the endless (and 
> conflicting, contradictory, and religious) battles that were waged during the 
> old "incubation" and "graduation" procedures?
> 
> Ask Clint, Flavio and Devananda how fruitful the Marconi/Zaqar "architectural 
> review" by the TC was.

That would be interesting to hear, and how this group would avoid and/or help 
such issues.

> 
>> But, I’m also on the fringe that is very ready to call the “big tent”
>> a failed experiment in attempting to avoid hard calls, too.
> 
> That's because you think the big tent initiative is something that it is not.
> 
> And we've had this conversation before, but you insist on equating the big 
> tent initiative with the TC broadening what its definition of OpenStack was. 
> And that's not what the big tent initiative was, as I've said repeatedly.
> 
> The big tent initiative was specifically to change from a subjective set of 
> requirements that a project must meet in order to be "part of OpenStack" into 
> an objective set of requirements.

Wait, wait, wait. The TC has a definition of OpenStack?  Oooh, what is it?

And if we can’t answer that, how can there be objective criteria to meet an 
undefined quantity?  Oh wait, that’s what the tent is today. Except Go, of 
course.

This is tangent from the main thread topic, so I’ll let it go.

Thanks,
doug

> 
> We removed the subjective, bike-shedding, and cringe-inducing "graduation 
> review" from the application process and instead focused on documenting a 
> clear set of objective requirements that projects could obligate themselves 
> to meeting if they wanted to "be part of OpenStack".
> 
 It may be that an architecture working group can provide some
 guidance that people will find useful. Against the odds I think
 those of us in the API-WG have actually managed to have a
 positive influence. We've not shaken things down to the
 foundations from which a great a glorious future may be born -- a
 lot of compromises have been made and not everybody wants to play
 along -- but things are going in the right direction, for some
 people, in some projects. Maybe a similar thing can happen with
 architecture.
>>> 
>>> That is my hope. I see the API WG and the Architecture WG as groups
>>> of experts in specific domains preparing recommendations and
>>> long-term plans. They don't have authority to force them onto
>>> projects. Ideally projects adopt them because they see them as the
>>> right way to do things.
>>> 
>>> And for the very few things that the TC deems necessary for
>>> OpenStack and where bottom-up didn't get it in a specific project
>>> (if all else fails), the TC can make a top-down request to a

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Clint Byrum]

Excerpts from Ian Cordasco's message of 2016-06-21 11:12:40 -0500:
>  
> 
> -Original Message-
> From: Michael Krotscheck <krotsch...@gmail.com>
> Reply: OpenStack Development Mailing List (not for usage questions) 
> <openstack-dev@lists.openstack.org>
> Date: June 21, 2016 at 10:18:25
> To: OpenStack Development Mailing List (not for usage questions) 
> <openstack-dev@lists.openstack.org>
> Subject:  Re: [openstack-dev] [all] Proposal: Architecture Working Group
> 
> > On Mon, Jun 20, 2016 at 10:59 AM Clint Byrum wrote:
> >  
> > >
> > > As you should be, and we all must be. It's not going to happen if we
> > > just dream it. That's kind of the point. Let's write down a design _for
> > > the group that writes down designs_.
> > >
> >  
> > If I had any confidence that this group would have a significant impact on
> > making OpenStack more consistent, easy to work on, and easier to build apps
> > against, I'd be happy to help out.
> >  
> > The only thing that would give me that confidence is if the WG charter had
> > a TC-enforced section of: "If you do not implement this *thing* within two
> > cycles, your project will get kicked out of OpenStack".
> 
> I don't think that will or *should* ever happen. The documents produced by 
> this WG, to me, would be the set of best practices that should be aimed for, 
> not mandatory. Forcing someone to refactor some complex piece of architecture 
> in their project in < 1 year so the project can remain part of OpenStack 
> seems like someone begging for horrible bugs and regressions in critical 
> behaviour.
> 
> This is worse than saying "All projects should stop disabling hacking rules 
> in two cycles or they'll stop receiving OpenStack Infra resources for 
> testing." or "All projects need to write new versions of their API just to 
> conform with the API WG."
> 

Just to be clear, I'm thinking more "This is how a thing should work",
not "best practices". The difference being that we'd write down something
like " message busses should look like X, Y, or Z based on factors a, b,
and/or c", and then we'd actually go fix or document the divergent cases
in OpenStack.

A best practices group would not actually fix anything IMO. It has to be
a group of action. Of course, we'll ask for help from any project teams
that are involved, and coordinate on things like release schedules so
we don't complicate short term plans. But I don't want this to just be a
standards body. I want this to be the group that gets the ball rolling,
and then helps it keep rolling.

There's no such thing as a perfect system, so we need to work toward
a process that produces _good_ results. Enforcement without merit will
just drive a wedge into that process.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Jay Pipes]

On 06/21/2016 12:53 PM, Doug Wiegley wrote:

On Jun 21, 2016, at 2:56 AM, Thierry Carrez 
wrote:
Chris Dent wrote:

On Mon, 20 Jun 2016, Doug Wiegley wrote:

So, it sounds like you’ve just described the job of the TC. And
they have so far refused to define OpenStack, leading to a
series of derivative decisions that seem … inconsistent over
time.


Thanks for writing down what I was thinking. I agree that
OpenStack needs some architectural vision, direction, leadership,
call it what you will. Every time I've voted for the _Technical_
Committee that leadership is what I've wanted my vote to be
creating.


The TC is a representative body which is elected to make top-down
decisions on OpenStack. However, as much as our community loves the
idea of "technical leadership" and "vision", they hate the top-down
decisions that come with it (especially when that top-down decision
doesn't go their way). They prefer bottom-up consensus.

So I'd argue that you need both. You need the TC whenever a hard
call has to be made, but in order to minimize the number of those
hard calls (and favor consensus building) you also need working
groups to build a bottom-up reasonable way forward.


This reads very strange to me, as I’d expect a group of technical
leaders to both make hard calls *and* to be able to build consensus
on overall direction and vision. They’re two sides of the same coin.
What is it about our process that means the TC can’t build consensus
on direction, but can only impose its will? I expect you didn’t mean
it to sound that way, though. Is the workload too high on the
bookkeeping to prevent the vision building? Are we too afraid of the
implications of defining ‘what is openstack?’, and what it might mean
to existing projects and the community? I’d think that in the
long-run, it’d prevent seemingly unrelated topics from seeming to go
sideways so often, and prevent a lot of these “hard calls”.


Perhaps you weren't around for the endless (and pointless) discussions 
around what is "the core of OpenStack"? Or you weren't around for the 
endless (and conflicting, contradictory, and religious) battles that 
were waged during the old "incubation" and "graduation" procedures?


Ask Clint, Flavio and Devananda how fruitful the Marconi/Zaqar 
"architectural review" by the TC was.



But, I’m also on the fringe that is very ready to call the “big tent”
a failed experiment in attempting to avoid hard calls, too.


That's because you think the big tent initiative is something that it is 
not.


And we've had this conversation before, but you insist on equating the 
big tent initiative with the TC broadening what its definition of 
OpenStack was. And that's not what the big tent initiative was, as I've 
said repeatedly.


The big tent initiative was specifically to change from a subjective set 
of requirements that a project must meet in order to be "part of 
OpenStack" into an objective set of requirements.


We removed the subjective, bike-shedding, and cringe-inducing 
"graduation review" from the application process and instead focused on 
documenting a clear set of objective requirements that projects could 
obligate themselves to meeting if they wanted to "be part of OpenStack".



It may be that an architecture working group can provide some
guidance that people will find useful. Against the odds I think
those of us in the API-WG have actually managed to have a
positive influence. We've not shaken things down to the
foundations from which a great a glorious future may be born -- a
lot of compromises have been made and not everybody wants to play
along -- but things are going in the right direction, for some
people, in some projects. Maybe a similar thing can happen with
architecture.


That is my hope. I see the API WG and the Architecture WG as groups
of experts in specific domains preparing recommendations and
long-term plans. They don't have authority to force them onto
projects. Ideally projects adopt them because they see them as the
right way to do things.

And for the very few things that the TC deems necessary for
OpenStack and where bottom-up didn't get it in a specific project
(if all else fails), the TC can make a top-down request to a
project to do things a certain way. The project can them either
comply or reject the TC oversight and become an unofficial
project.


Don’t get me wrong, I welcome this initiative. I find it mildly
disconcerting that the folks that I thought we were electing to fill
this role will instead be filled by others, but the vacuum does need
to be filled, and I thank Clint for stepping up.


I don't particularly think it's a vacuum that can be filled by a small 
group of "architects". My experience is that unless said architects are 
actually involved in building the software at hand and have a good 
understanding of why certain design choices were originally made in the 
various projects, the end deliverable of such a group tends to be the 
software 

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Doug Wiegley]

> On Jun 21, 2016, at 2:56 AM, Thierry Carrez  wrote:
> 
> Chris Dent wrote:
>> On Mon, 20 Jun 2016, Doug Wiegley wrote:
>>> So, it sounds like you’ve just described the job of the TC. And they
>>> have so far refused to define OpenStack, leading to a series of
>>> derivative decisions that seem … inconsistent over time.
>> 
>> Thanks for writing down what I was thinking. I agree that OpenStack
>> needs some architectural vision, direction, leadership, call it what
>> you will. Every time I've voted for the _Technical_ Committee that
>> leadership is what I've wanted my vote to be creating.
> 
> The TC is a representative body which is elected to make top-down decisions 
> on OpenStack. However, as much as our community loves the idea of "technical 
> leadership" and "vision", they hate the top-down decisions that come with it 
> (especially when that top-down decision doesn't go their way). They prefer 
> bottom-up consensus.
> 
> So I'd argue that you need both. You need the TC whenever a hard call has to 
> be made, but in order to minimize the number of those hard calls (and favor 
> consensus building) you also need working groups to build a bottom-up 
> reasonable way forward.

This reads very strange to me, as I’d expect a group of technical leaders to 
both make hard calls *and* to be able to build consensus on overall direction 
and vision. They’re two sides of the same coin. What is it about our process 
that means the TC can’t build consensus on direction, but can only impose its 
will? I expect you didn’t mean it to sound that way, though. Is the workload 
too high on the bookkeeping to prevent the vision building? Are we too afraid 
of the implications of defining ‘what is openstack?’, and what it might mean to 
existing projects and the community? I’d think that in the long-run, it’d 
prevent seemingly unrelated topics from seeming to go sideways so often, and 
prevent a lot of these “hard calls”.

But, I’m also on the fringe that is very ready to call the “big tent” a failed 
experiment in attempting to avoid hard calls, too.

> 
>> It may be that an architecture working group can provide some
>> guidance that people will find useful. Against the odds I think
>> those of us in the API-WG have actually managed to have a positive
>> influence. We've not shaken things down to the foundations from
>> which a great a glorious future may be born -- a lot of compromises
>> have been made and not everybody wants to play along -- but things
>> are going in the right direction, for some people, in some projects.
>> Maybe a similar thing can happen with architecture.
> 
> That is my hope. I see the API WG and the Architecture WG as groups of 
> experts in specific domains preparing recommendations and long-term plans. 
> They don't have authority to force them onto projects. Ideally projects adopt 
> them because they see them as the right way to do things.
> 
> And for the very few things that the TC deems necessary for OpenStack and 
> where bottom-up didn't get it in a specific project (if all else fails), the 
> TC can make a top-down request to a project to do things a certain way. The 
> project can them either comply or reject the TC oversight and become an 
> unofficial project.

Don’t get me wrong, I welcome this initiative. I find it mildly disconcerting 
that the folks that I thought we were electing to fill this role will instead 
be filled by others, but the vacuum does need to be filled, and I thank Clint 
for stepping up.

Thanks,
doug


> 
> -- 
> Thierry Carrez (ttx)
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Jay Pipes]

On 06/21/2016 04:25 AM, Chris Dent wrote:

However, I worry deeply that it could become astronauts with finger
paints.


Yes. This.

I will happily take software design suggestions from people that 
demonstrate with code and benchmarks that their suggestion actually 
works outside of the theoretical/academic/MartinFowler landscape and 
actually solves a real problem better than existing code.


Best,
-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Ian Cordasco]

 

-Original Message-
From: Michael Krotscheck <krotsch...@gmail.com>
Reply: OpenStack Development Mailing List (not for usage questions) 
<openstack-dev@lists.openstack.org>
Date: June 21, 2016 at 10:18:25
To: OpenStack Development Mailing List (not for usage questions) 
<openstack-dev@lists.openstack.org>
Subject:  Re: [openstack-dev] [all] Proposal: Architecture Working Group

> On Mon, Jun 20, 2016 at 10:59 AM Clint Byrum wrote:
>  
> >
> > As you should be, and we all must be. It's not going to happen if we
> > just dream it. That's kind of the point. Let's write down a design _for
> > the group that writes down designs_.
> >
>  
> If I had any confidence that this group would have a significant impact on
> making OpenStack more consistent, easy to work on, and easier to build apps
> against, I'd be happy to help out.
>  
> The only thing that would give me that confidence is if the WG charter had
> a TC-enforced section of: "If you do not implement this *thing* within two
> cycles, your project will get kicked out of OpenStack".

I don't think that will or *should* ever happen. The documents produced by this 
WG, to me, would be the set of best practices that should be aimed for, not 
mandatory. Forcing someone to refactor some complex piece of architecture in 
their project in < 1 year so the project can remain part of OpenStack seems 
like someone begging for horrible bugs and regressions in critical behaviour.

This is worse than saying "All projects should stop disabling hacking rules in 
two cycles or they'll stop receiving OpenStack Infra resources for testing." or 
"All projects need to write new versions of their API just to conform with the 
API WG."

--  
Ian Cordasco


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Michael Krotscheck]

On Mon, Jun 20, 2016 at 10:59 AM Clint Byrum  wrote:

>
> As you should be, and we all must be. It's not going to happen if we
> just dream it. That's kind of the point. Let's write down a design _for
> the group that writes down designs_.
>

If I had any confidence that this group would have a significant impact on
making OpenStack more consistent, easy to work on, and easier to build apps
against, I'd be happy to help out.

The only thing that would give me that confidence is if the WG charter had
a TC-enforced section of: "If you do not implement this *thing* within two
cycles, your project will get kicked out of OpenStack".

Michael
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Clint Byrum]

Excerpts from Chris Dent's message of 2016-06-21 09:25:44 +0100:
> On Mon, 20 Jun 2016, Doug Wiegley wrote:
> 
> > So, it sounds like you’ve just described the job of the TC. And they
> > have so far refused to define OpenStack, leading to a series of
> > derivative decisions that seem … inconsistent over time.
> 
> Thanks for writing down what I was thinking. I agree that OpenStack
> needs some architectural vision, direction, leadership, call it what
> you will. Every time I've voted for the _Technical_ Committee that
> leadership is what I've wanted my vote to be creating.
> 

I think that's still part of it. The difference is that the TC is using
their expertise to resolve conflict and ratify decisions, while a working
group is creating a source of data and taking actions that hopefully
prevent the conflict from ever arising.

> It may be that an architecture working group can provide some
> guidance that people will find useful. Against the odds I think
> those of us in the API-WG have actually managed to have a positive
> influence. We've not shaken things down to the foundations from
> which a great a glorious future may be born -- a lot of compromises
> have been made and not everybody wants to play along -- but things
> are going in the right direction, for some people, in some projects.
> Maybe a similar thing can happen with architecture.
> 

I definitely saw what was happening with the API-WG and wanted to have a
similar effect on the design process.

> However, I worry deeply that it could become astronauts with finger
> paints. In the API working group at least we have the HTTP RFCs as
> foundational sources of authority to guide us. In something so
> fraught with opinion what are the sources of authority?
> 
> I was pointed at this a while ago
> 
>  https://wiki.openstack.org/wiki/BasicDesignTenets
> 
> It's full of lots of great rules that are frequently broken.
> 


This is a great point Chris. I definitely think we need to have some
authority to fall back on. The link above is a great start. I'd like to
invite others to think on that and share their sources of authority for
distributed system design.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Zane Bitter]

On 20/06/16 19:27, Clint Byrum wrote:

Excerpts from Doug Wiegley's message of 2016-06-20 10:40:56 -0600:

So, it sounds like you’ve just described the job of the TC.


It may sound like that, but the TC have repeatedly (and perhaps wisely) 
disclaimed that as part of their job. So any attempt to fill in this 
glaring gap in OpenStack (i.e. to attempt to design at a higher level 
than an individual project) should be welcomed with open arms IMHO. 
Nobody benefits when every project team is on their own to make 
architectural decisions in a vacuum, which is too often the case now.



And they have so far refused to define OpenStack, leading to a series of 
derivative decisions that seem … inconsistent over time.

How is this body going to be different?

How will it have any teeth, and not just end up with the standard entrenched 
projects ignoring it?



Hi Doug, thanks for your candid reply. I understand that there is a
concern and I want to address it. However, I feel like answering this
directly will start this working group out on the wrong foot.

We shouldn't need teeth. This isn't an adversarial working group that
will be challenging engineers any more than an architect of a building
challenges the builders while they work. An architect that ignores their
engineers is not going to complete many projects. Of course, engineers
may disagree on the cost of an architectural decision. But to disagree,
first we need to actually _make_ a decision on a design.

The goal of this group would be to provide a detailed architecture and
plans for the way the system should work and fit together as a whole. Like
any complex system, during implementation, things that architects weren't
aware of come to light. Something that seems simple turns out to be
complex. Something that seemed absolutely necessary can be factored out.

Nobody is suggesting designing OpenStack from the ground up, just that
where there isn't an agreed upon design, let's write down how the system
works now, and then make a design and a plan to actually improve it.

Engineers have no effective place to turn to right now when there is
a lack of design. The TC could of course do it, but what I want to do
is have a more open and free-flowing group that are laser focused on
providing support for the design of the system. I want to work out with
the community at large how we add weight to the designs we choose, and
one good option is for the Architecture Working Group to make proposals
to the openstack-specs repo, which the TC would ultimately approve.
That's not a new process, we already have it:

http://docs.openstack.org/project-team-guide/cross-project.html

I'm just suggesting a group that actually _focuses_ on the design
aspects of that process.

Without this, we are designing in real time and taking the shortest path
to achieve short term goals. This has positive and negative effects. I
think we've reached a point in OpenStack's evolution where the positive
effects of that are mostly realized, and now we should work to pay
down some of the negative effects by adopting some designs and beginning
refactoring of the system. It's not a fast process, so the longer we wait,
the longer we pay interest on that debt.


I can't +1 this enough. Thanks Clint for kicking off this initiative.

cheers,
Zane.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Bogdan Dobrelya]

On 06/20/2016 08:07 PM, Clint Byrum wrote:
> Excerpts from Jesse Cook's message of 2016-06-20 16:58:48 +:
>> +1
>>
>> The points about the PWG and TC are worth some consideration.
>>
>> From my perspective, I think it would make sense for the PWG to define the
>> expected behaviors of the system, which would be an input to the
>> architecture group. The architecture group would define both prescriptive
>> (where we'd like to be) and descriptive (where we actually are...roughly)
>> architectures. This would provide both the vision for the future state and
>> understanding of current state that is necessary for us to all swim in the
>> same general direction instead of constantly running into each other. I
>> don't see the architecture as something you push down, but rather something
>> that helps each contributor ask, "Does that get us closer to where we are
>> trying to go?" I absolutely think this is something that would provide a
>> huge benefit to the organization.
>>
> 
> That sounds about right Jesse. Thanks for bringing up the PWG. I
> definitely don't think an Architecture WG would want to answer "what
> is OpenStack" alone. More like "What should the OpenStack we described
> actually look like?".

IMHO, the group shall get a global state view first, which is to collect
and process (and perhaps do a *lot* of reverse engineering and  asking
developers AND users many questions) implicit knowledge hidden in
OpenStack components and libraries.

Then answer the question "What *does* the OpenStack actually look
like?". The answer may have a form of established contracts and
behaviour models - expected vs actual, and corner cases.
Yes, SLA, timeouts, failure modes, and patterns. Not generic things the
projects already have being auto-generated in dev docs, but very
specific reverse engineered and collected as a feedback or testing
discovered, *whitespaces*.

Examples: which DB patterns each of the project/common library do
leverage in code? Sounds simple, but will require a huge amount of
reverse engineering. How much of those are NOT ready to be used with A/A
read/write transactions? And for messaging patterns and failure modes
and corner cases handling - like lost or duplicated or racing data.
Retrying, timeouts - exposed in API and implcit/hardcoded. Failure
detection for stateful components like conductors/schedulers? What are
locking expectations (strong or relaxed) for the projects that do
leverage DLM?

So, those and many more firstly to be brought to the light then set in
stone. Only after that the group may proceed with the question "What
should the OpenStack we described actually look like?" in order to a)
fit unexpected or unclear behaviour into the contracts and document them
as well, b) improve.

> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


-- 
Best regards,
Bogdan Dobrelya,
Irc #bogdando

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Thierry Carrez]

Chris Dent wrote:

On Mon, 20 Jun 2016, Doug Wiegley wrote:

So, it sounds like you’ve just described the job of the TC. And they
have so far refused to define OpenStack, leading to a series of
derivative decisions that seem … inconsistent over time.


Thanks for writing down what I was thinking. I agree that OpenStack
needs some architectural vision, direction, leadership, call it what
you will. Every time I've voted for the _Technical_ Committee that
leadership is what I've wanted my vote to be creating.


The TC is a representative body which is elected to make top-down 
decisions on OpenStack. However, as much as our community loves the idea 
of "technical leadership" and "vision", they hate the top-down decisions 
that come with it (especially when that top-down decision doesn't go 
their way). They prefer bottom-up consensus.


So I'd argue that you need both. You need the TC whenever a hard call 
has to be made, but in order to minimize the number of those hard calls 
(and favor consensus building) you also need working groups to build a 
bottom-up reasonable way forward.



It may be that an architecture working group can provide some
guidance that people will find useful. Against the odds I think
those of us in the API-WG have actually managed to have a positive
influence. We've not shaken things down to the foundations from
which a great a glorious future may be born -- a lot of compromises
have been made and not everybody wants to play along -- but things
are going in the right direction, for some people, in some projects.
Maybe a similar thing can happen with architecture.


That is my hope. I see the API WG and the Architecture WG as groups of 
experts in specific domains preparing recommendations and long-term 
plans. They don't have authority to force them onto projects. Ideally 
projects adopt them because they see them as the right way to do things.


And for the very few things that the TC deems necessary for OpenStack 
and where bottom-up didn't get it in a specific project (if all else 
fails), the TC can make a top-down request to a project to do things a 
certain way. The project can them either comply or reject the TC 
oversight and become an unofficial project.


--
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Chris Dent]

On Mon, 20 Jun 2016, Doug Wiegley wrote:


So, it sounds like you’ve just described the job of the TC. And they
have so far refused to define OpenStack, leading to a series of
derivative decisions that seem … inconsistent over time.


Thanks for writing down what I was thinking. I agree that OpenStack
needs some architectural vision, direction, leadership, call it what
you will. Every time I've voted for the _Technical_ Committee that
leadership is what I've wanted my vote to be creating.

It may be that an architecture working group can provide some
guidance that people will find useful. Against the odds I think
those of us in the API-WG have actually managed to have a positive
influence. We've not shaken things down to the foundations from
which a great a glorious future may be born -- a lot of compromises
have been made and not everybody wants to play along -- but things
are going in the right direction, for some people, in some projects.
Maybe a similar thing can happen with architecture.

However, I worry deeply that it could become astronauts with finger
paints. In the API working group at least we have the HTTP RFCs as
foundational sources of authority to guide us. In something so
fraught with opinion what are the sources of authority?

I was pointed at this a while ago

https://wiki.openstack.org/wiki/BasicDesignTenets

It's full of lots of great rules that are frequently broken.

--
Chris Dent   (╯°□°)╯︵┻━┻http://anticdent.org/
freenode: cdent tw: @anticdent__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Fox, Kevin M]

Some other topics:

* Rolling upgrades - For example, I haven't figured out a way to safely drain 
an rpc based service rather then just shooting it and hoping things go well... 
Is it safe? This safety code should be built into them all consistently.

 * How to get events to users in a usable way. OpenStack Service X event -> 
Some system that duplicates the event stream based on requests from users -> 
Multiple Zaqar Queues? -> User? Can this co-exist with Horizon using it for 
updating its UI without polling?

 * The state of guest agents is abysmal. Each service uses a different way to 
talk between controller and vm, and each has its own problems and operators 
need to figure out workarounds for all. Its really painful. We need something 
like a unified guest agent that gets the logic right, and is plugable.

Thanks,
Kevin

From: Clint Byrum [cl...@fewbar.com]
Sent: Monday, June 20, 2016 11:31 AM
To: openstack-dev
Subject: Re: [openstack-dev] [all] Proposal: Architecture Working Group

Excerpts from Joshua Harlow's message of 2016-06-17 15:33:25 -0700:
> Thanks for getting this started Clint,
>
> I'm happy and excited to be involved in helping try to guide the whole
> ecosystem together (it's also why I like being in oslo) to a
> architecture that is more cohesive (and is more of something that we can
> say to our current or future children that we were all involved and
> proud to be involved in creating/maturing...).
>
> At a start, for said first meeting, any kind of agenda come to mind, or
> will it be more a informal gathering to start (either is fine with me)?
>

I've been hesitant to fill this in too much as I'm still forming the
idea, but here are the items I think are most compelling to begin with:

* DLM's across OpenStack -- This is already under way[1], but it seems to
  have fizzled out. IMO that is because there's no working group who
  owns it. We need to actually write some plans.

* Messaging patterns -- There was a recent discussion about nuances in
  oslo.messaging's implementation that vary driver to driver. I'd like to
  make sure we have all of the ways messaging is used written down and
  make sure groups have guidance on how each one should (or shouldn't)
  be used, including documentation of anti-patterns, and a plan for
  simplifying communication between processes in OpenStack where
  possible.

* True Microservices -- OpenStack's services are heavily
  interdependent. It makes it hard for an engineer to make an impact
  without becoming an expert on all of them, and it also leads to a heavy
  burden on operators and QA as they end up having to debug the system
  as a whole. We should write down how the system is intertwined now, and
  develop plans for unwinding that and allowing each service to stand on
  its own, including having stronger testing in isolation. (This includes
  exploring the thought that nova-compute could become its own project
  that all of the other things communicate with over well defined APIs).

These could keep a small group of architects and engineers busy for a
year or more. I'm sure others have things they'd like to design and
improve in OpenStack at this level.

Once we have broad agreement on such a group, and perhaps some guidance
from the TC, we can propose an agenda and prioritize efforts as part of
the first few meetings.

[1] 
http://specs.openstack.org/openstack/openstack-specs/specs/chronicles-of-a-dlm.html
> -Josh
>
> Clint Byrum wrote:
> > ar·chi·tec·ture
> > ˈärkəˌtek(t)SHər/
> > noun
> > noun: architecture
> >
> >  1.
> >
> >  the art or practice of designing and constructing buildings.
> >
> >  synonyms:building design, building style, planning, building, 
> > construction;
> >
> >  formalarchitectonics
> >
> >  "modern architecture"
> >
> >  the style in which a building is designed or constructed, especially 
> > with regard to a specific period, place, or culture.
> >
> >  plural noun: architectures
> >
> >  "Victorian architecture"
> >
> >  2.
> >
> >  the complex or carefully designed structure of something.
> >
> >  "the chemical architecture of the human brain"
> >
> >  the conceptual structure and logical organization of a computer or 
> > computer-based system.
> >
> >  "a client/server architecture"
> >
> >  synonyms:structure, construction, organization, layout, design, build, 
> > anatomy, makeup;
> >
> >  informalsetup
> >
> >  "the architecture of a computer system"
> >
> >
> > Introduction
> > =
> >
> > OpenStack is a big system. We ha

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Fox, Kevin M]

+1

From: Clint Byrum [cl...@fewbar.com]
Sent: Monday, June 20, 2016 10:27 AM
To: openstack-dev
Subject: Re: [openstack-dev] [all] Proposal: Architecture Working Group

Excerpts from Doug Wiegley's message of 2016-06-20 10:40:56 -0600:
> So, it sounds like you’ve just described the job of the TC. And they have so 
> far refused to define OpenStack, leading to a series of derivative decisions 
> that seem … inconsistent over time.
>
> How is this body going to be different?
>
> How will it have any teeth, and not just end up with the standard entrenched 
> projects ignoring it?
>

Hi Doug, thanks for your candid reply. I understand that there is a
concern and I want to address it. However, I feel like answering this
directly will start this working group out on the wrong foot.

We shouldn't need teeth. This isn't an adversarial working group that
will be challenging engineers any more than an architect of a building
challenges the builders while they work. An architect that ignores their
engineers is not going to complete many projects. Of course, engineers
may disagree on the cost of an architectural decision. But to disagree,
first we need to actually _make_ a decision on a design.

The goal of this group would be to provide a detailed architecture and
plans for the way the system should work and fit together as a whole. Like
any complex system, during implementation, things that architects weren't
aware of come to light. Something that seems simple turns out to be
complex. Something that seemed absolutely necessary can be factored out.

Nobody is suggesting designing OpenStack from the ground up, just that
where there isn't an agreed upon design, let's write down how the system
works now, and then make a design and a plan to actually improve it.

Engineers have no effective place to turn to right now when there is
a lack of design. The TC could of course do it, but what I want to do
is have a more open and free-flowing group that are laser focused on
providing support for the design of the system. I want to work out with
the community at large how we add weight to the designs we choose, and
one good option is for the Architecture Working Group to make proposals
to the openstack-specs repo, which the TC would ultimately approve.
That's not a new process, we already have it:

http://docs.openstack.org/project-team-guide/cross-project.html

I'm just suggesting a group that actually _focuses_ on the design
aspects of that process.

Without this, we are designing in real time and taking the shortest path
to achieve short term goals. This has positive and negative effects. I
think we've reached a point in OpenStack's evolution where the positive
effects of that are mostly realized, and now we should work to pay
down some of the negative effects by adopting some designs and beginning
refactoring of the system. It's not a fast process, so the longer we wait,
the longer we pay interest on that debt.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Clint Byrum]

Excerpts from Joshua Harlow's message of 2016-06-17 15:33:25 -0700:
> Thanks for getting this started Clint,
> 
> I'm happy and excited to be involved in helping try to guide the whole 
> ecosystem together (it's also why I like being in oslo) to a 
> architecture that is more cohesive (and is more of something that we can 
> say to our current or future children that we were all involved and 
> proud to be involved in creating/maturing...).
> 
> At a start, for said first meeting, any kind of agenda come to mind, or 
> will it be more a informal gathering to start (either is fine with me)?
> 

I've been hesitant to fill this in too much as I'm still forming the
idea, but here are the items I think are most compelling to begin with:

* DLM's across OpenStack -- This is already under way[1], but it seems to
  have fizzled out. IMO that is because there's no working group who
  owns it. We need to actually write some plans.

* Messaging patterns -- There was a recent discussion about nuances in
  oslo.messaging's implementation that vary driver to driver. I'd like to
  make sure we have all of the ways messaging is used written down and
  make sure groups have guidance on how each one should (or shouldn't)
  be used, including documentation of anti-patterns, and a plan for
  simplifying communication between processes in OpenStack where
  possible.

* True Microservices -- OpenStack's services are heavily
  interdependent. It makes it hard for an engineer to make an impact
  without becoming an expert on all of them, and it also leads to a heavy
  burden on operators and QA as they end up having to debug the system
  as a whole. We should write down how the system is intertwined now, and
  develop plans for unwinding that and allowing each service to stand on
  its own, including having stronger testing in isolation. (This includes
  exploring the thought that nova-compute could become its own project
  that all of the other things communicate with over well defined APIs).

These could keep a small group of architects and engineers busy for a
year or more. I'm sure others have things they'd like to design and
improve in OpenStack at this level.

Once we have broad agreement on such a group, and perhaps some guidance
from the TC, we can propose an agenda and prioritize efforts as part of
the first few meetings.

[1] 
http://specs.openstack.org/openstack/openstack-specs/specs/chronicles-of-a-dlm.html
> -Josh
> 
> Clint Byrum wrote:
> > ar·chi·tec·ture
> > ˈärkəˌtek(t)SHər/
> > noun
> > noun: architecture
> >
> >  1.
> >
> >  the art or practice of designing and constructing buildings.
> >
> >  synonyms:building design, building style, planning, building, 
> > construction;
> >
> >  formalarchitectonics
> >
> >  "modern architecture"
> >
> >  the style in which a building is designed or constructed, especially 
> > with regard to a specific period, place, or culture.
> >
> >  plural noun: architectures
> >
> >  "Victorian architecture"
> >
> >  2.
> >
> >  the complex or carefully designed structure of something.
> >
> >  "the chemical architecture of the human brain"
> >
> >  the conceptual structure and logical organization of a computer or 
> > computer-based system.
> >
> >  "a client/server architecture"
> >
> >  synonyms:structure, construction, organization, layout, design, build, 
> > anatomy, makeup;
> >
> >  informalsetup
> >
> >  "the architecture of a computer system"
> >
> >
> > Introduction
> > =
> >
> > OpenStack is a big system. We have debated what it actually is [1],
> > and there are even t-shirts to poke fun at the fact that we don't have
> > good answers.
> >
> > But this isn't what any of us wants. We'd like to be able to point
> > at something and proudly tell people "This is what we designed and
> > implemented."
> >
> > And for each individual project, that is a possibility. Neutron can
> > tell you they designed how their agents and drivers work. Nova can
> > tell you that they designed the way conductors handle communication
> > with API nodes and compute nodes. But when we start talking about how
> > they interact with each other, it's clearly just a coincidental mash of
> > de-facto standards and specs that don't help anyone make decisions when
> > refactoring or adding on to the system.
> >
> > Oslo and cross-project initiatives have brought some peace and order
> > to the implementation and engineering processes, but not to the design
> > process. New ideas still start largely in the project where they are
> > needed most, and often conflict with similar decisions and ideas in other
> > projects [dlm, taskflow, tooz, service discovery, state machines, glance
> > tasks, messaging patterns, database patterns, etc. etc.]. Often times this
> > creates a log jam where none of the projects adopt a solution that would
> > align with others. Most of the time when things finally come to a head
> > these things get 

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Clint Byrum]

Excerpts from Jesse Cook's message of 2016-06-20 16:58:48 +:
> +1
> 
> The points about the PWG and TC are worth some consideration.
> 
> From my perspective, I think it would make sense for the PWG to define the
> expected behaviors of the system, which would be an input to the
> architecture group. The architecture group would define both prescriptive
> (where we'd like to be) and descriptive (where we actually are...roughly)
> architectures. This would provide both the vision for the future state and
> understanding of current state that is necessary for us to all swim in the
> same general direction instead of constantly running into each other. I
> don't see the architecture as something you push down, but rather something
> that helps each contributor ask, "Does that get us closer to where we are
> trying to go?" I absolutely think this is something that would provide a
> huge benefit to the organization.
> 

That sounds about right Jesse. Thanks for bringing up the PWG. I
definitely don't think an Architecture WG would want to answer "what
is OpenStack" alone. More like "What should the OpenStack we described
actually look like?".

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Clint Byrum]

Excerpts from Michael Krotscheck's message of 2016-06-20 15:26:20 +:
> I like the idea in principle, but am bullish on the implementation.
> 

As you should be, and we all must be. It's not going to happen if we
just dream it. That's kind of the point. Let's write down a design _for
the group that writes down designs_.

> For example: we have the API-WG which fulfills part of an Architectural
> mission, as well as the Cross-Project WG which fulfills a different part.
> Yet there's no incentive, carrot or stick, that drives adoption of the
> approved specs, other than "Well if you want to do the work, have fun". In
> several cases, I've run into the excuses of "That's Hard/That breaks
> backwards compatibility/That's not profitable/I'm not paid to do that".
> 

If we write a bunch of specs which are too much of a burden to implement,
we're doing it wrong. The idea isn't to rip everything out.  It's to
acknowledge where there is a complete lack of design, write one that
is practical, and organize work to get it done.

> What's going to prevent [Insert Project Here] from coming along and saying
> "Oh, well, we don't like those decisions, so are going to ignore them."
> What provides the incentive for a project to adopt these recommendations?
> 

Absolutely nothing will prevent that, and I don't think putting any kind
of hard requirements for following this group's designs would be
productive until it has proven that it can actually improve OpenStack.
Perhaps if we do succeed in designing parts of the system, and some
teams find that useful, we can look at adding some of the parts of the
design to our more stringent requirements (like "use python or C"). But
that's not something I'd seek from day 1. That's just a recipe for
revolt.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Clint Byrum]

Excerpts from Doug Wiegley's message of 2016-06-20 10:40:56 -0600:
> So, it sounds like you’ve just described the job of the TC. And they have so 
> far refused to define OpenStack, leading to a series of derivative decisions 
> that seem … inconsistent over time.
> 
> How is this body going to be different?
> 
> How will it have any teeth, and not just end up with the standard entrenched 
> projects ignoring it?
> 

Hi Doug, thanks for your candid reply. I understand that there is a
concern and I want to address it. However, I feel like answering this
directly will start this working group out on the wrong foot.

We shouldn't need teeth. This isn't an adversarial working group that
will be challenging engineers any more than an architect of a building
challenges the builders while they work. An architect that ignores their
engineers is not going to complete many projects. Of course, engineers
may disagree on the cost of an architectural decision. But to disagree,
first we need to actually _make_ a decision on a design.

The goal of this group would be to provide a detailed architecture and
plans for the way the system should work and fit together as a whole. Like
any complex system, during implementation, things that architects weren't
aware of come to light. Something that seems simple turns out to be
complex. Something that seemed absolutely necessary can be factored out.

Nobody is suggesting designing OpenStack from the ground up, just that
where there isn't an agreed upon design, let's write down how the system
works now, and then make a design and a plan to actually improve it.

Engineers have no effective place to turn to right now when there is
a lack of design. The TC could of course do it, but what I want to do
is have a more open and free-flowing group that are laser focused on
providing support for the design of the system. I want to work out with
the community at large how we add weight to the designs we choose, and
one good option is for the Architecture Working Group to make proposals
to the openstack-specs repo, which the TC would ultimately approve.
That's not a new process, we already have it:

http://docs.openstack.org/project-team-guide/cross-project.html

I'm just suggesting a group that actually _focuses_ on the design
aspects of that process.

Without this, we are designing in real time and taking the shortest path
to achieve short term goals. This has positive and negative effects. I
think we've reached a point in OpenStack's evolution where the positive
effects of that are mostly realized, and now we should work to pay
down some of the negative effects by adopting some designs and beginning
refactoring of the system. It's not a fast process, so the longer we wait,
the longer we pay interest on that debt.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Jesse Cook]

+1

The points about the PWG and TC are worth some consideration.

>From my perspective, I think it would make sense for the PWG to define the
expected behaviors of the system, which would be an input to the
architecture group. The architecture group would define both prescriptive
(where we'd like to be) and descriptive (where we actually are...roughly)
architectures. This would provide both the vision for the future state and
understanding of current state that is necessary for us to all swim in the
same general direction instead of constantly running into each other. I
don't see the architecture as something you push down, but rather something
that helps each contributor ask, "Does that get us closer to where we are
trying to go?" I absolutely think this is something that would provide a
huge benefit to the organization.

On Mon, Jun 20, 2016 at 4:40 PM, Doug Wiegley 
wrote:

> So, it sounds like you’ve just described the job of the TC. And they have
> so far refused to define OpenStack, leading to a series of derivative
> decisions that seem … inconsistent over time.
>
> How is this body going to be different?
>
> How will it have any teeth, and not just end up with the standard
> entrenched projects ignoring it?
>
> Thanks,
> doug
>
>
> > On Jun 17, 2016, at 3:52 PM, Clint Byrum  wrote:
> >
> > ar·chi·tec·ture
> > ˈärkəˌtek(t)SHər/
> > noun
> > noun: architecture
> >
> >1.
> >
> >the art or practice of designing and constructing buildings.
> >
> >synonyms:building design, building style, planning, building,
> construction;
> >
> >formalarchitectonics
> >
> >"modern architecture"
> >
> >the style in which a building is designed or constructed, especially
> with regard to a specific period, place, or culture.
> >
> >plural noun: architectures
> >
> >"Victorian architecture"
> >
> >2.
> >
> >the complex or carefully designed structure of something.
> >
> >"the chemical architecture of the human brain"
> >
> >the conceptual structure and logical organization of a computer or
> computer-based system.
> >
> >"a client/server architecture"
> >
> >synonyms:structure, construction, organization, layout, design,
> build, anatomy, makeup;
> >
> >informalsetup
> >
> >"the architecture of a computer system"
> >
> >
> > Introduction
> > =
> >
> > OpenStack is a big system. We have debated what it actually is [1],
> > and there are even t-shirts to poke fun at the fact that we don't have
> > good answers.
> >
> > But this isn't what any of us wants. We'd like to be able to point
> > at something and proudly tell people "This is what we designed and
> > implemented."
> >
> > And for each individual project, that is a possibility. Neutron can
> > tell you they designed how their agents and drivers work. Nova can
> > tell you that they designed the way conductors handle communication
> > with API nodes and compute nodes. But when we start talking about how
> > they interact with each other, it's clearly just a coincidental mash of
> > de-facto standards and specs that don't help anyone make decisions when
> > refactoring or adding on to the system.
> >
> > Oslo and cross-project initiatives have brought some peace and order
> > to the implementation and engineering processes, but not to the design
> > process. New ideas still start largely in the project where they are
> > needed most, and often conflict with similar decisions and ideas in other
> > projects [dlm, taskflow, tooz, service discovery, state machines, glance
> > tasks, messaging patterns, database patterns, etc. etc.]. Often times
> this
> > creates a log jam where none of the projects adopt a solution that would
> > align with others. Most of the time when things finally come to a head
> > these things get done in a piecemeal fashion, where it's half done here,
> > 1/3 over there, 1/4 there, and 3/4 over there..., which to the outside
> > looks like  chaos, because that's precisely what it is.
> >
> > And this isn't always a technical design problem. OpenStack, for
> instance,
> > isn't really a micro service architecture. Of course, it might look like
> > that in diagrams [2], but we all know it really isn't. The compute node
> is
> > home to agents for every single concern, and the API interactions between
> > the services is too tightly woven to consider many of them functional
> > without the same lockstep version of other services together. A game to
> > play is ask yourself what would happen if a service was isolated on its
> > own island, how functional would its API be, if at all. Is this something
> > that we want? No. But there doesn't seem to be a place where we can go
> > to actually design, discuss, debate, and ratify changes that would help
> > us get to the point of gathering the necessary will and capability to
> > enact these efforts.
> >
> > Maybe nova-compute should be isolated from nova, with an API that
> > nova, 

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Doug Wiegley]

So, it sounds like you’ve just described the job of the TC. And they have so 
far refused to define OpenStack, leading to a series of derivative decisions 
that seem … inconsistent over time.

How is this body going to be different?

How will it have any teeth, and not just end up with the standard entrenched 
projects ignoring it?

Thanks,
doug


> On Jun 17, 2016, at 3:52 PM, Clint Byrum  wrote:
> 
> ar·chi·tec·ture
> ˈärkəˌtek(t)SHər/
> noun
> noun: architecture
> 
>1. 
> 
>the art or practice of designing and constructing buildings.
> 
>synonyms:building design, building style, planning, building, 
> construction; 
> 
>formalarchitectonics 
> 
>"modern architecture"
> 
>the style in which a building is designed or constructed, especially with 
> regard to a specific period, place, or culture.
> 
>plural noun: architectures
> 
>"Victorian architecture"
> 
>2. 
> 
>the complex or carefully designed structure of something.
> 
>"the chemical architecture of the human brain"
> 
>the conceptual structure and logical organization of a computer or 
> computer-based system.
> 
>"a client/server architecture"
> 
>synonyms:structure, construction, organization, layout, design, build, 
> anatomy, makeup; 
> 
>informalsetup 
> 
>"the architecture of a computer system"
> 
> 
> Introduction
> =
> 
> OpenStack is a big system. We have debated what it actually is [1],
> and there are even t-shirts to poke fun at the fact that we don't have
> good answers.
> 
> But this isn't what any of us wants. We'd like to be able to point
> at something and proudly tell people "This is what we designed and
> implemented."
> 
> And for each individual project, that is a possibility. Neutron can
> tell you they designed how their agents and drivers work. Nova can
> tell you that they designed the way conductors handle communication
> with API nodes and compute nodes. But when we start talking about how
> they interact with each other, it's clearly just a coincidental mash of
> de-facto standards and specs that don't help anyone make decisions when
> refactoring or adding on to the system.
> 
> Oslo and cross-project initiatives have brought some peace and order
> to the implementation and engineering processes, but not to the design
> process. New ideas still start largely in the project where they are
> needed most, and often conflict with similar decisions and ideas in other
> projects [dlm, taskflow, tooz, service discovery, state machines, glance
> tasks, messaging patterns, database patterns, etc. etc.]. Often times this
> creates a log jam where none of the projects adopt a solution that would
> align with others. Most of the time when things finally come to a head
> these things get done in a piecemeal fashion, where it's half done here,
> 1/3 over there, 1/4 there, and 3/4 over there..., which to the outside
> looks like  chaos, because that's precisely what it is.
> 
> And this isn't always a technical design problem. OpenStack, for instance,
> isn't really a micro service architecture. Of course, it might look like
> that in diagrams [2], but we all know it really isn't. The compute node is
> home to agents for every single concern, and the API interactions between
> the services is too tightly woven to consider many of them functional
> without the same lockstep version of other services together. A game to
> play is ask yourself what would happen if a service was isolated on its
> own island, how functional would its API be, if at all. Is this something
> that we want? No. But there doesn't seem to be a place where we can go
> to actually design, discuss, debate, and ratify changes that would help
> us get to the point of gathering the necessary will and capability to
> enact these efforts.
> 
> Maybe nova-compute should be isolated from nova, with an API that
> nova, cinder and neutron talk to. Maybe we should make the scheduler
> cross-project aware and capable of scheduling more than just nova
> instances. Maybe we should have experimental groups that can look at how
> some of this functionality could perhaps be delegated to non-openstack
> projects. We hear that Mesos, for example to help with the scheduling
> aspects, but how do we discuss these outside hijacking threads on the
> mailing list? These are things that we all discuss in the hallways
> and bars and parties at the summit, but because they cross projects at
> the design level, and are inherently a lot of social and technical and
> exploratory work, Many of us fear we never get to a place of turning
> our dreams into reality.
> 
> So, with that, I'd like to propose the creation of an Architecture Working
> Group. This group's charge would not be design by committee, but a place
> for architects to share their designs and gain support across projects
> to move forward with and ratify architectural decisions. That includes
> coordinating exploratory work that may turn 

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Nikhil Komawar]

+1 , great idea.

if we can add a mission/objective based on the nice definitions you
added, will help a long way in cross-project architecture evolution.
moreover, I'd like this to be a integration point for openstack projects
(and not a silo) so that we can build the shared understanding we really
need to build.

On 6/17/16 5:52 PM, Clint Byrum wrote:
> ar·chi·tec·ture
> ˈärkəˌtek(t)SHər/
> noun
> noun: architecture
>
> 1. 
>
> the art or practice of designing and constructing buildings.
>
> synonyms:building design, building style, planning, building, 
> construction; 
>
> formalarchitectonics 
>
> "modern architecture"
>
> the style in which a building is designed or constructed, especially with 
> regard to a specific period, place, or culture.
>
> plural noun: architectures
>
> "Victorian architecture"
>
> 2. 
>
> the complex or carefully designed structure of something.
>
> "the chemical architecture of the human brain"
>
> the conceptual structure and logical organization of a computer or 
> computer-based system.
>
> "a client/server architecture"
>
> synonyms:structure, construction, organization, layout, design, build, 
> anatomy, makeup; 
>
> informalsetup 
>
> "the architecture of a computer system"
>
>
> Introduction
> =
>
> OpenStack is a big system. We have debated what it actually is [1],
> and there are even t-shirts to poke fun at the fact that we don't have
> good answers.
>
> But this isn't what any of us wants. We'd like to be able to point
> at something and proudly tell people "This is what we designed and
> implemented."
>
> And for each individual project, that is a possibility. Neutron can
> tell you they designed how their agents and drivers work. Nova can
> tell you that they designed the way conductors handle communication
> with API nodes and compute nodes. But when we start talking about how
> they interact with each other, it's clearly just a coincidental mash of
> de-facto standards and specs that don't help anyone make decisions when
> refactoring or adding on to the system.
>
> Oslo and cross-project initiatives have brought some peace and order
> to the implementation and engineering processes, but not to the design
> process. New ideas still start largely in the project where they are
> needed most, and often conflict with similar decisions and ideas in other
> projects [dlm, taskflow, tooz, service discovery, state machines, glance
> tasks, messaging patterns, database patterns, etc. etc.]. Often times this
> creates a log jam where none of the projects adopt a solution that would
> align with others. Most of the time when things finally come to a head
> these things get done in a piecemeal fashion, where it's half done here,
> 1/3 over there, 1/4 there, and 3/4 over there..., which to the outside
> looks like  chaos, because that's precisely what it is.
>
> And this isn't always a technical design problem. OpenStack, for instance,
> isn't really a micro service architecture. Of course, it might look like
> that in diagrams [2], but we all know it really isn't. The compute node is
> home to agents for every single concern, and the API interactions between
> the services is too tightly woven to consider many of them functional
> without the same lockstep version of other services together. A game to
> play is ask yourself what would happen if a service was isolated on its
> own island, how functional would its API be, if at all. Is this something
> that we want? No. But there doesn't seem to be a place where we can go
> to actually design, discuss, debate, and ratify changes that would help
> us get to the point of gathering the necessary will and capability to
> enact these efforts.
>
> Maybe nova-compute should be isolated from nova, with an API that
> nova, cinder and neutron talk to. Maybe we should make the scheduler
> cross-project aware and capable of scheduling more than just nova
> instances. Maybe we should have experimental groups that can look at how
> some of this functionality could perhaps be delegated to non-openstack
> projects. We hear that Mesos, for example to help with the scheduling
> aspects, but how do we discuss these outside hijacking threads on the
> mailing list? These are things that we all discuss in the hallways
> and bars and parties at the summit, but because they cross projects at
> the design level, and are inherently a lot of social and technical and
> exploratory work, Many of us fear we never get to a place of turning
> our dreams into reality.
>
> So, with that, I'd like to propose the creation of an Architecture Working
> Group. This group's charge would not be design by committee, but a place
> for architects to share their designs and gain support across projects
> to move forward with and ratify architectural decisions. That includes
> coordinating exploratory work that may turn into being the base of further
> architectural decisions for OpenStack. I 

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Michael Krotscheck]

I like the idea in principle, but am bullish on the implementation.

For example: we have the API-WG which fulfills part of an Architectural
mission, as well as the Cross-Project WG which fulfills a different part.
Yet there's no incentive, carrot or stick, that drives adoption of the
approved specs, other than "Well if you want to do the work, have fun". In
several cases, I've run into the excuses of "That's Hard/That breaks
backwards compatibility/That's not profitable/I'm not paid to do that".

What's going to prevent [Insert Project Here] from coming along and saying
"Oh, well, we don't like those decisions, so are going to ignore them."
What provides the incentive for a project to adopt these recommendations?

Michael

On Mon, Jun 20, 2016 at 2:25 AM Denis Makogon  wrote:

> Hello Clint.
>
> I'd like to take part as well, so count me in.
>
> Kind regards,
> Denys Makogon
>
>
> 2016-06-20 10:23 GMT+03:00 Ghe Rivero :
>
>> Hi all!
>> I really like the idea of the group, so count me in!
>>
>> Ghe Rivero
>>
>> Quoting Clint Byrum (2016-06-17 23:52:43)
>> > ar·chi·tec·ture
>> > ˈärkəˌtek(t)SHər/
>> > noun
>> > noun: architecture
>> >
>> > 1.
>> >
>> > the art or practice of designing and constructing buildings.
>> >
>> > synonyms:building design, building style, planning, building,
>> construction;
>> >
>> > formalarchitectonics
>> >
>> > "modern architecture"
>> >
>> > the style in which a building is designed or constructed,
>> especially with regard to a specific period, place, or culture.
>> >
>> > plural noun: architectures
>> >
>> > "Victorian architecture"
>> >
>> > 2.
>> >
>> > the complex or carefully designed structure of something.
>> >
>> > "the chemical architecture of the human brain"
>> >
>> > the conceptual structure and logical organization of a computer or
>> computer-based system.
>> >
>> > "a client/server architecture"
>> >
>> > synonyms:structure, construction, organization, layout, design,
>> build, anatomy, makeup;
>> >
>> > informalsetup
>> >
>> > "the architecture of a computer system"
>> >
>> >
>> > Introduction
>> > =
>> >
>> > OpenStack is a big system. We have debated what it actually is [1],
>> > and there are even t-shirts to poke fun at the fact that we don't have
>> > good answers.
>> >
>> > But this isn't what any of us wants. We'd like to be able to point
>> > at something and proudly tell people "This is what we designed and
>> > implemented."
>> >
>> > And for each individual project, that is a possibility. Neutron can
>> > tell you they designed how their agents and drivers work. Nova can
>> > tell you that they designed the way conductors handle communication
>> > with API nodes and compute nodes. But when we start talking about how
>> > they interact with each other, it's clearly just a coincidental mash of
>> > de-facto standards and specs that don't help anyone make decisions when
>> > refactoring or adding on to the system.
>> >
>> > Oslo and cross-project initiatives have brought some peace and order
>> > to the implementation and engineering processes, but not to the design
>> > process. New ideas still start largely in the project where they are
>> > needed most, and often conflict with similar decisions and ideas in
>> other
>> > projects [dlm, taskflow, tooz, service discovery, state machines, glance
>> > tasks, messaging patterns, database patterns, etc. etc.]. Often times
>> this
>> > creates a log jam where none of the projects adopt a solution that would
>> > align with others. Most of the time when things finally come to a head
>> > these things get done in a piecemeal fashion, where it's half done here,
>> > 1/3 over there, 1/4 there, and 3/4 over there..., which to the outside
>> > looks like  chaos, because that's precisely what it is.
>> >
>> > And this isn't always a technical design problem. OpenStack, for
>> instance,
>> > isn't really a micro service architecture. Of course, it might look like
>> > that in diagrams [2], but we all know it really isn't. The compute node
>> is
>> > home to agents for every single concern, and the API interactions
>> between
>> > the services is too tightly woven to consider many of them functional
>> > without the same lockstep version of other services together. A game to
>> > play is ask yourself what would happen if a service was isolated on its
>> > own island, how functional would its API be, if at all. Is this
>> something
>> > that we want? No. But there doesn't seem to be a place where we can go
>> > to actually design, discuss, debate, and ratify changes that would help
>> > us get to the point of gathering the necessary will and capability to
>> > enact these efforts.
>> >
>> > Maybe nova-compute should be isolated from nova, with an API that
>> > nova, cinder and neutron talk to. Maybe we should make the scheduler
>> > cross-project aware and capable of scheduling more than just nova

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Denis Makogon]

Hello Clint.

I'd like to take part as well, so count me in.

Kind regards,
Denys Makogon


2016-06-20 10:23 GMT+03:00 Ghe Rivero :

> Hi all!
> I really like the idea of the group, so count me in!
>
> Ghe Rivero
>
> Quoting Clint Byrum (2016-06-17 23:52:43)
> > ar·chi·tec·ture
> > ˈärkəˌtek(t)SHər/
> > noun
> > noun: architecture
> >
> > 1.
> >
> > the art or practice of designing and constructing buildings.
> >
> > synonyms:building design, building style, planning, building,
> construction;
> >
> > formalarchitectonics
> >
> > "modern architecture"
> >
> > the style in which a building is designed or constructed, especially
> with regard to a specific period, place, or culture.
> >
> > plural noun: architectures
> >
> > "Victorian architecture"
> >
> > 2.
> >
> > the complex or carefully designed structure of something.
> >
> > "the chemical architecture of the human brain"
> >
> > the conceptual structure and logical organization of a computer or
> computer-based system.
> >
> > "a client/server architecture"
> >
> > synonyms:structure, construction, organization, layout, design,
> build, anatomy, makeup;
> >
> > informalsetup
> >
> > "the architecture of a computer system"
> >
> >
> > Introduction
> > =
> >
> > OpenStack is a big system. We have debated what it actually is [1],
> > and there are even t-shirts to poke fun at the fact that we don't have
> > good answers.
> >
> > But this isn't what any of us wants. We'd like to be able to point
> > at something and proudly tell people "This is what we designed and
> > implemented."
> >
> > And for each individual project, that is a possibility. Neutron can
> > tell you they designed how their agents and drivers work. Nova can
> > tell you that they designed the way conductors handle communication
> > with API nodes and compute nodes. But when we start talking about how
> > they interact with each other, it's clearly just a coincidental mash of
> > de-facto standards and specs that don't help anyone make decisions when
> > refactoring or adding on to the system.
> >
> > Oslo and cross-project initiatives have brought some peace and order
> > to the implementation and engineering processes, but not to the design
> > process. New ideas still start largely in the project where they are
> > needed most, and often conflict with similar decisions and ideas in other
> > projects [dlm, taskflow, tooz, service discovery, state machines, glance
> > tasks, messaging patterns, database patterns, etc. etc.]. Often times
> this
> > creates a log jam where none of the projects adopt a solution that would
> > align with others. Most of the time when things finally come to a head
> > these things get done in a piecemeal fashion, where it's half done here,
> > 1/3 over there, 1/4 there, and 3/4 over there..., which to the outside
> > looks like  chaos, because that's precisely what it is.
> >
> > And this isn't always a technical design problem. OpenStack, for
> instance,
> > isn't really a micro service architecture. Of course, it might look like
> > that in diagrams [2], but we all know it really isn't. The compute node
> is
> > home to agents for every single concern, and the API interactions between
> > the services is too tightly woven to consider many of them functional
> > without the same lockstep version of other services together. A game to
> > play is ask yourself what would happen if a service was isolated on its
> > own island, how functional would its API be, if at all. Is this something
> > that we want? No. But there doesn't seem to be a place where we can go
> > to actually design, discuss, debate, and ratify changes that would help
> > us get to the point of gathering the necessary will and capability to
> > enact these efforts.
> >
> > Maybe nova-compute should be isolated from nova, with an API that
> > nova, cinder and neutron talk to. Maybe we should make the scheduler
> > cross-project aware and capable of scheduling more than just nova
> > instances. Maybe we should have experimental groups that can look at how
> > some of this functionality could perhaps be delegated to non-openstack
> > projects. We hear that Mesos, for example to help with the scheduling
> > aspects, but how do we discuss these outside hijacking threads on the
> > mailing list? These are things that we all discuss in the hallways
> > and bars and parties at the summit, but because they cross projects at
> > the design level, and are inherently a lot of social and technical and
> > exploratory work, Many of us fear we never get to a place of turning
> > our dreams into reality.
> >
> > So, with that, I'd like to propose the creation of an Architecture
> Working
> > Group. This group's charge would not be design by committee, but a place
> > for architects to share their designs and gain support across projects
> > to move forward with and ratify architectural decisions. That 

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Ghe Rivero]

Hi all!
I really like the idea of the group, so count me in!

Ghe Rivero

Quoting Clint Byrum (2016-06-17 23:52:43)
> ar·chi·tec·ture
> ˈärkəˌtek(t)SHər/
> noun
> noun: architecture
> 
> 1. 
> 
> the art or practice of designing and constructing buildings.
> 
> synonyms:building design, building style, planning, building, 
> construction; 
> 
> formalarchitectonics 
> 
> "modern architecture"
> 
> the style in which a building is designed or constructed, especially with 
> regard to a specific period, place, or culture.
> 
> plural noun: architectures
> 
> "Victorian architecture"
> 
> 2. 
> 
> the complex or carefully designed structure of something.
> 
> "the chemical architecture of the human brain"
> 
> the conceptual structure and logical organization of a computer or 
> computer-based system.
> 
> "a client/server architecture"
> 
> synonyms:structure, construction, organization, layout, design, build, 
> anatomy, makeup; 
> 
> informalsetup 
> 
> "the architecture of a computer system"
> 
> 
> Introduction
> =
> 
> OpenStack is a big system. We have debated what it actually is [1],
> and there are even t-shirts to poke fun at the fact that we don't have
> good answers.
> 
> But this isn't what any of us wants. We'd like to be able to point
> at something and proudly tell people "This is what we designed and
> implemented."
> 
> And for each individual project, that is a possibility. Neutron can
> tell you they designed how their agents and drivers work. Nova can
> tell you that they designed the way conductors handle communication
> with API nodes and compute nodes. But when we start talking about how
> they interact with each other, it's clearly just a coincidental mash of
> de-facto standards and specs that don't help anyone make decisions when
> refactoring or adding on to the system.
> 
> Oslo and cross-project initiatives have brought some peace and order
> to the implementation and engineering processes, but not to the design
> process. New ideas still start largely in the project where they are
> needed most, and often conflict with similar decisions and ideas in other
> projects [dlm, taskflow, tooz, service discovery, state machines, glance
> tasks, messaging patterns, database patterns, etc. etc.]. Often times this
> creates a log jam where none of the projects adopt a solution that would
> align with others. Most of the time when things finally come to a head
> these things get done in a piecemeal fashion, where it's half done here,
> 1/3 over there, 1/4 there, and 3/4 over there..., which to the outside
> looks like  chaos, because that's precisely what it is.
> 
> And this isn't always a technical design problem. OpenStack, for instance,
> isn't really a micro service architecture. Of course, it might look like
> that in diagrams [2], but we all know it really isn't. The compute node is
> home to agents for every single concern, and the API interactions between
> the services is too tightly woven to consider many of them functional
> without the same lockstep version of other services together. A game to
> play is ask yourself what would happen if a service was isolated on its
> own island, how functional would its API be, if at all. Is this something
> that we want? No. But there doesn't seem to be a place where we can go
> to actually design, discuss, debate, and ratify changes that would help
> us get to the point of gathering the necessary will and capability to
> enact these efforts.
> 
> Maybe nova-compute should be isolated from nova, with an API that
> nova, cinder and neutron talk to. Maybe we should make the scheduler
> cross-project aware and capable of scheduling more than just nova
> instances. Maybe we should have experimental groups that can look at how
> some of this functionality could perhaps be delegated to non-openstack
> projects. We hear that Mesos, for example to help with the scheduling
> aspects, but how do we discuss these outside hijacking threads on the
> mailing list? These are things that we all discuss in the hallways
> and bars and parties at the summit, but because they cross projects at
> the design level, and are inherently a lot of social and technical and
> exploratory work, Many of us fear we never get to a place of turning
> our dreams into reality.
> 
> So, with that, I'd like to propose the creation of an Architecture Working
> Group. This group's charge would not be design by committee, but a place
> for architects to share their designs and gain support across projects
> to move forward with and ratify architectural decisions. That includes
> coordinating exploratory work that may turn into being the base of further
> architectural decisions for OpenStack. I would expect that the people
> in this group would largely be senior at the companies involved and,
> if done correctly, they can help prioritize this work by advocating for
> people/fellow engineers to actually 

Re: [openstack-dev] [all] Proposal: Architecture Working Group

[Joshua Harlow]

Thanks for getting this started Clint,

I'm happy and excited to be involved in helping try to guide the whole 
ecosystem together (it's also why I like being in oslo) to a 
architecture that is more cohesive (and is more of something that we can 
say to our current or future children that we were all involved and 
proud to be involved in creating/maturing...).


At a start, for said first meeting, any kind of agenda come to mind, or 
will it be more a informal gathering to start (either is fine with me)?


-Josh

Clint Byrum wrote:

ar·chi·tec·ture
ˈärkəˌtek(t)SHər/
noun
noun: architecture

 1.

 the art or practice of designing and constructing buildings.

 synonyms:building design, building style, planning, building, construction;

 formalarchitectonics

 "modern architecture"

 the style in which a building is designed or constructed, especially with 
regard to a specific period, place, or culture.

 plural noun: architectures

 "Victorian architecture"

 2.

 the complex or carefully designed structure of something.

 "the chemical architecture of the human brain"

 the conceptual structure and logical organization of a computer or 
computer-based system.

 "a client/server architecture"

 synonyms:structure, construction, organization, layout, design, build, 
anatomy, makeup;

 informalsetup

 "the architecture of a computer system"


Introduction
=

OpenStack is a big system. We have debated what it actually is [1],
and there are even t-shirts to poke fun at the fact that we don't have
good answers.

But this isn't what any of us wants. We'd like to be able to point
at something and proudly tell people "This is what we designed and
implemented."

And for each individual project, that is a possibility. Neutron can
tell you they designed how their agents and drivers work. Nova can
tell you that they designed the way conductors handle communication
with API nodes and compute nodes. But when we start talking about how
they interact with each other, it's clearly just a coincidental mash of
de-facto standards and specs that don't help anyone make decisions when
refactoring or adding on to the system.

Oslo and cross-project initiatives have brought some peace and order
to the implementation and engineering processes, but not to the design
process. New ideas still start largely in the project where they are
needed most, and often conflict with similar decisions and ideas in other
projects [dlm, taskflow, tooz, service discovery, state machines, glance
tasks, messaging patterns, database patterns, etc. etc.]. Often times this
creates a log jam where none of the projects adopt a solution that would
align with others. Most of the time when things finally come to a head
these things get done in a piecemeal fashion, where it's half done here,
1/3 over there, 1/4 there, and 3/4 over there..., which to the outside
looks like  chaos, because that's precisely what it is.

And this isn't always a technical design problem. OpenStack, for instance,
isn't really a micro service architecture. Of course, it might look like
that in diagrams [2], but we all know it really isn't. The compute node is
home to agents for every single concern, and the API interactions between
the services is too tightly woven to consider many of them functional
without the same lockstep version of other services together. A game to
play is ask yourself what would happen if a service was isolated on its
own island, how functional would its API be, if at all. Is this something
that we want? No. But there doesn't seem to be a place where we can go
to actually design, discuss, debate, and ratify changes that would help
us get to the point of gathering the necessary will and capability to
enact these efforts.

Maybe nova-compute should be isolated from nova, with an API that
nova, cinder and neutron talk to. Maybe we should make the scheduler
cross-project aware and capable of scheduling more than just nova
instances. Maybe we should have experimental groups that can look at how
some of this functionality could perhaps be delegated to non-openstack
projects. We hear that Mesos, for example to help with the scheduling
aspects, but how do we discuss these outside hijacking threads on the
mailing list? These are things that we all discuss in the hallways
and bars and parties at the summit, but because they cross projects at
the design level, and are inherently a lot of social and technical and
exploratory work, Many of us fear we never get to a place of turning
our dreams into reality.

So, with that, I'd like to propose the creation of an Architecture Working
Group. This group's charge would not be design by committee, but a place
for architects to share their designs and gain support across projects
to move forward with and ratify architectural decisions. That includes
coordinating exploratory work that may turn into being the base of further
architectural decisions for