Re: [openstack-dev] [TripleO] Austin summit - session recap/summary

[Derek Higgins]

On 19 May 2016 5:38 pm, "Paul Belanger"  wrote:
>
> On Thu, May 19, 2016 at 03:50:15PM +0100, Derek Higgins wrote:
> > On 18 May 2016 at 13:34, Paul Belanger  wrote:
> > > On Wed, May 18, 2016 at 12:22:55PM +0100, Derek Higgins wrote:
> > >> On 6 May 2016 at 14:18, Paul Belanger  wrote:
> > >> > On Tue, May 03, 2016 at 05:34:55PM +0100, Steven Hardy wrote:
> > >> >> Hi all,
> > >> >>
> > >> >> Some folks have requested a summary of our summit sessions, as
has been
> > >> >> provided for some other projects.
> > >> >>
> > >> >> I'll probably go into more detail on some of these topics either
via
> > >> >> subsequent more focussed threads an/or some blog posts but what
follows is
> > >> >> an overview of our summit sessions[1] with notable actions or
decisions
> > >> >> highlighted.  I'm including some of my own thoughts and
conclusions, folks
> > >> >> are welcome/encouraged to follow up with their own clarifications
or
> > >> >> different perspectives :)
> > >> >>
> > >> >> TripleO had a total of 5 sessions in Austin I'll cover them
one-by-one:
> > >> >>
> > >> >> -
> > >> >> Upgrades - current status and roadmap
> > >> >> -
> > >> >>
> > >> >> In this session we discussed the current state of upgrades -
initial
> > >> >> support for full major version upgrades has been implemented, but
the
> > >> >> implementation is monolithic, highly coupled to pacemaker, and
inflexible
> > >> >> with regard to third-party extraconfig changes.
> > >> >>
> > >> >> The main outcomes were that we will add support for more granular
> > >> >> definition of the upgrade lifecycle to the new composable
services format,
> > >> >> and that we will explore moving towards the proposed lightweight
HA
> > >> >> architecture to reduce the need for so much pacemaker specific
logic.
> > >> >>
> > >> >> We also agreed that investigating use of mistral to drive upgrade
workflows
> > >> >> was a good idea - currently we have a mixture of scripts combined
with Heat
> > >> >> to drive the upgrade process, and some refactoring into discrete
mistral
> > >> >> workflows may provide a more maintainable solution.  Potential
for using
> > >> >> the existing SoftwareDeployment approach directly via mistral
(outside of
> > >> >> the heat templates) was also discussed as something to be further
> > >> >> investigated and prototyped.
> > >> >>
> > >> >> We also touched on the CI implications of upgrades - we've got an
upgrades
> > >> >> job now, but we need to ensure coverage of full
release-to-release upgrades
> > >> >> (not just commit to commit).
> > >> >>
> > >> >> ---
> > >> >> Containerization status/roadmap
> > >> >> ---
> > >> >>
> > >> >> In this session we discussed the current status of containers in
TripleO
> > >> >> (which is to say, the container based compute node which deploys
containers
> > >> >> via Heat onto an an Atomic host node that is also deployed via
Heat), and
> > >> >> what strategy is most appropriate to achieve a fully
containerized TripleO
> > >> >> deployment.
> > >> >>
> > >> >> Several folks from Kolla participated in the session, and there
was
> > >> >> significant focus on where work may happen such that further
collaboration
> > >> >> between communities is possible.  To some extent this discussion
on where
> > >> >> (as opposed to how) proved a distraction and prevented much
discussion on
> > >> >> supportable architectural implementation for TripleO, thus what
follows is
> > >> >> mostly my perspective on the issues that exist:
> > >> >>
> > >> >> Significant uncertainty exists wrt integration between Kolla and
TripleO -
> > >> >> there's largely consensus that we want to consume the container
images
> > >> >> defined by the Kolla community, but much less agreement that we
can
> > >> >> feasably switch to the ansible-orchestrated deployment/config flow
> > >> >> supported by Kolla without breaking many of our primary operator
interfaces
> > >> >> in a fundamentally unacceptable way, for example:
> > >> >>
> > >> >> - The Mistral based API is being implemented on the expectation
that the
> > >> >>   primary interface to TripleO deployments is a parameters schema
exposed
> > >> >>   by a series of Heat templates - this is no longer true in a
"split stack"
> > >> >>   model where we have to hand off to an alternate service
orchestration tool.
> > >> >>
> > >> >> - The tripleo-ui (based on the Mistral based API) consumes heat
parameter
> > >> >>   schema to build it's UI, and Ansible doesn't support the
necessary
> > >> >>   parameter schema definition (such as types and descriptions) to
enable
> > >> >>   this pattern to be replicated.  Ansible also doesn't provide a
HTTP API,
> > >> >>   so we'd still have to maintain and API surface for the (non
python) UI to
> > >> >>   consume.
> > >> >>
> > >> >> We also discussed 

Re: [openstack-dev] [TripleO] Austin summit - session recap/summary

[Paul Belanger]

On Thu, May 19, 2016 at 03:50:15PM +0100, Derek Higgins wrote:
> On 18 May 2016 at 13:34, Paul Belanger  wrote:
> > On Wed, May 18, 2016 at 12:22:55PM +0100, Derek Higgins wrote:
> >> On 6 May 2016 at 14:18, Paul Belanger  wrote:
> >> > On Tue, May 03, 2016 at 05:34:55PM +0100, Steven Hardy wrote:
> >> >> Hi all,
> >> >>
> >> >> Some folks have requested a summary of our summit sessions, as has been
> >> >> provided for some other projects.
> >> >>
> >> >> I'll probably go into more detail on some of these topics either via
> >> >> subsequent more focussed threads an/or some blog posts but what follows 
> >> >> is
> >> >> an overview of our summit sessions[1] with notable actions or decisions
> >> >> highlighted.  I'm including some of my own thoughts and conclusions, 
> >> >> folks
> >> >> are welcome/encouraged to follow up with their own clarifications or
> >> >> different perspectives :)
> >> >>
> >> >> TripleO had a total of 5 sessions in Austin I'll cover them one-by-one:
> >> >>
> >> >> -
> >> >> Upgrades - current status and roadmap
> >> >> -
> >> >>
> >> >> In this session we discussed the current state of upgrades - initial
> >> >> support for full major version upgrades has been implemented, but the
> >> >> implementation is monolithic, highly coupled to pacemaker, and 
> >> >> inflexible
> >> >> with regard to third-party extraconfig changes.
> >> >>
> >> >> The main outcomes were that we will add support for more granular
> >> >> definition of the upgrade lifecycle to the new composable services 
> >> >> format,
> >> >> and that we will explore moving towards the proposed lightweight HA
> >> >> architecture to reduce the need for so much pacemaker specific logic.
> >> >>
> >> >> We also agreed that investigating use of mistral to drive upgrade 
> >> >> workflows
> >> >> was a good idea - currently we have a mixture of scripts combined with 
> >> >> Heat
> >> >> to drive the upgrade process, and some refactoring into discrete mistral
> >> >> workflows may provide a more maintainable solution.  Potential for using
> >> >> the existing SoftwareDeployment approach directly via mistral (outside 
> >> >> of
> >> >> the heat templates) was also discussed as something to be further
> >> >> investigated and prototyped.
> >> >>
> >> >> We also touched on the CI implications of upgrades - we've got an 
> >> >> upgrades
> >> >> job now, but we need to ensure coverage of full release-to-release 
> >> >> upgrades
> >> >> (not just commit to commit).
> >> >>
> >> >> ---
> >> >> Containerization status/roadmap
> >> >> ---
> >> >>
> >> >> In this session we discussed the current status of containers in TripleO
> >> >> (which is to say, the container based compute node which deploys 
> >> >> containers
> >> >> via Heat onto an an Atomic host node that is also deployed via Heat), 
> >> >> and
> >> >> what strategy is most appropriate to achieve a fully containerized 
> >> >> TripleO
> >> >> deployment.
> >> >>
> >> >> Several folks from Kolla participated in the session, and there was
> >> >> significant focus on where work may happen such that further 
> >> >> collaboration
> >> >> between communities is possible.  To some extent this discussion on 
> >> >> where
> >> >> (as opposed to how) proved a distraction and prevented much discussion 
> >> >> on
> >> >> supportable architectural implementation for TripleO, thus what follows 
> >> >> is
> >> >> mostly my perspective on the issues that exist:
> >> >>
> >> >> Significant uncertainty exists wrt integration between Kolla and 
> >> >> TripleO -
> >> >> there's largely consensus that we want to consume the container images
> >> >> defined by the Kolla community, but much less agreement that we can
> >> >> feasably switch to the ansible-orchestrated deployment/config flow
> >> >> supported by Kolla without breaking many of our primary operator 
> >> >> interfaces
> >> >> in a fundamentally unacceptable way, for example:
> >> >>
> >> >> - The Mistral based API is being implemented on the expectation that the
> >> >>   primary interface to TripleO deployments is a parameters schema 
> >> >> exposed
> >> >>   by a series of Heat templates - this is no longer true in a "split 
> >> >> stack"
> >> >>   model where we have to hand off to an alternate service orchestration 
> >> >> tool.
> >> >>
> >> >> - The tripleo-ui (based on the Mistral based API) consumes heat 
> >> >> parameter
> >> >>   schema to build it's UI, and Ansible doesn't support the necessary
> >> >>   parameter schema definition (such as types and descriptions) to enable
> >> >>   this pattern to be replicated.  Ansible also doesn't provide a HTTP 
> >> >> API,
> >> >>   so we'd still have to maintain and API surface for the (non python) 
> >> >> UI to
> >> >>   consume.
> >> >>
> >> >> We also discussed ideas around integration 

Re: [openstack-dev] [TripleO] Austin summit - session recap/summary

[Paul Belanger]

On Wed, May 18, 2016 at 08:34:40AM -0400, Paul Belanger wrote:
> On Wed, May 18, 2016 at 12:22:55PM +0100, Derek Higgins wrote:
> > On 6 May 2016 at 14:18, Paul Belanger  wrote:
> > > On Tue, May 03, 2016 at 05:34:55PM +0100, Steven Hardy wrote:
> > >> Hi all,
> > >>
> > >> Some folks have requested a summary of our summit sessions, as has been
> > >> provided for some other projects.
> > >>
> > >> I'll probably go into more detail on some of these topics either via
> > >> subsequent more focussed threads an/or some blog posts but what follows 
> > >> is
> > >> an overview of our summit sessions[1] with notable actions or decisions
> > >> highlighted.  I'm including some of my own thoughts and conclusions, 
> > >> folks
> > >> are welcome/encouraged to follow up with their own clarifications or
> > >> different perspectives :)
> > >>
> > >> TripleO had a total of 5 sessions in Austin I'll cover them one-by-one:
> > >>
> > >> -
> > >> Upgrades - current status and roadmap
> > >> -
> > >>
> > >> In this session we discussed the current state of upgrades - initial
> > >> support for full major version upgrades has been implemented, but the
> > >> implementation is monolithic, highly coupled to pacemaker, and inflexible
> > >> with regard to third-party extraconfig changes.
> > >>
> > >> The main outcomes were that we will add support for more granular
> > >> definition of the upgrade lifecycle to the new composable services 
> > >> format,
> > >> and that we will explore moving towards the proposed lightweight HA
> > >> architecture to reduce the need for so much pacemaker specific logic.
> > >>
> > >> We also agreed that investigating use of mistral to drive upgrade 
> > >> workflows
> > >> was a good idea - currently we have a mixture of scripts combined with 
> > >> Heat
> > >> to drive the upgrade process, and some refactoring into discrete mistral
> > >> workflows may provide a more maintainable solution.  Potential for using
> > >> the existing SoftwareDeployment approach directly via mistral (outside of
> > >> the heat templates) was also discussed as something to be further
> > >> investigated and prototyped.
> > >>
> > >> We also touched on the CI implications of upgrades - we've got an 
> > >> upgrades
> > >> job now, but we need to ensure coverage of full release-to-release 
> > >> upgrades
> > >> (not just commit to commit).
> > >>
> > >> ---
> > >> Containerization status/roadmap
> > >> ---
> > >>
> > >> In this session we discussed the current status of containers in TripleO
> > >> (which is to say, the container based compute node which deploys 
> > >> containers
> > >> via Heat onto an an Atomic host node that is also deployed via Heat), and
> > >> what strategy is most appropriate to achieve a fully containerized 
> > >> TripleO
> > >> deployment.
> > >>
> > >> Several folks from Kolla participated in the session, and there was
> > >> significant focus on where work may happen such that further 
> > >> collaboration
> > >> between communities is possible.  To some extent this discussion on where
> > >> (as opposed to how) proved a distraction and prevented much discussion on
> > >> supportable architectural implementation for TripleO, thus what follows 
> > >> is
> > >> mostly my perspective on the issues that exist:
> > >>
> > >> Significant uncertainty exists wrt integration between Kolla and TripleO 
> > >> -
> > >> there's largely consensus that we want to consume the container images
> > >> defined by the Kolla community, but much less agreement that we can
> > >> feasably switch to the ansible-orchestrated deployment/config flow
> > >> supported by Kolla without breaking many of our primary operator 
> > >> interfaces
> > >> in a fundamentally unacceptable way, for example:
> > >>
> > >> - The Mistral based API is being implemented on the expectation that the
> > >>   primary interface to TripleO deployments is a parameters schema exposed
> > >>   by a series of Heat templates - this is no longer true in a "split 
> > >> stack"
> > >>   model where we have to hand off to an alternate service orchestration 
> > >> tool.
> > >>
> > >> - The tripleo-ui (based on the Mistral based API) consumes heat parameter
> > >>   schema to build it's UI, and Ansible doesn't support the necessary
> > >>   parameter schema definition (such as types and descriptions) to enable
> > >>   this pattern to be replicated.  Ansible also doesn't provide a HTTP 
> > >> API,
> > >>   so we'd still have to maintain and API surface for the (non python) UI 
> > >> to
> > >>   consume.
> > >>
> > >> We also discussed ideas around integration with kubernetes (a hot topic 
> > >> on
> > >> the Kolla track this summit), but again this proved inconclusive beyond
> > >> that yes someone should try developing a PoC to stimulate further
> > >> discussion.  Again, significant 

Re: [openstack-dev] [TripleO] Austin summit - session recap/summary

[Derek Higgins]

On 18 May 2016 at 13:34, Paul Belanger  wrote:
> On Wed, May 18, 2016 at 12:22:55PM +0100, Derek Higgins wrote:
>> On 6 May 2016 at 14:18, Paul Belanger  wrote:
>> > On Tue, May 03, 2016 at 05:34:55PM +0100, Steven Hardy wrote:
>> >> Hi all,
>> >>
>> >> Some folks have requested a summary of our summit sessions, as has been
>> >> provided for some other projects.
>> >>
>> >> I'll probably go into more detail on some of these topics either via
>> >> subsequent more focussed threads an/or some blog posts but what follows is
>> >> an overview of our summit sessions[1] with notable actions or decisions
>> >> highlighted.  I'm including some of my own thoughts and conclusions, folks
>> >> are welcome/encouraged to follow up with their own clarifications or
>> >> different perspectives :)
>> >>
>> >> TripleO had a total of 5 sessions in Austin I'll cover them one-by-one:
>> >>
>> >> -
>> >> Upgrades - current status and roadmap
>> >> -
>> >>
>> >> In this session we discussed the current state of upgrades - initial
>> >> support for full major version upgrades has been implemented, but the
>> >> implementation is monolithic, highly coupled to pacemaker, and inflexible
>> >> with regard to third-party extraconfig changes.
>> >>
>> >> The main outcomes were that we will add support for more granular
>> >> definition of the upgrade lifecycle to the new composable services format,
>> >> and that we will explore moving towards the proposed lightweight HA
>> >> architecture to reduce the need for so much pacemaker specific logic.
>> >>
>> >> We also agreed that investigating use of mistral to drive upgrade 
>> >> workflows
>> >> was a good idea - currently we have a mixture of scripts combined with 
>> >> Heat
>> >> to drive the upgrade process, and some refactoring into discrete mistral
>> >> workflows may provide a more maintainable solution.  Potential for using
>> >> the existing SoftwareDeployment approach directly via mistral (outside of
>> >> the heat templates) was also discussed as something to be further
>> >> investigated and prototyped.
>> >>
>> >> We also touched on the CI implications of upgrades - we've got an upgrades
>> >> job now, but we need to ensure coverage of full release-to-release 
>> >> upgrades
>> >> (not just commit to commit).
>> >>
>> >> ---
>> >> Containerization status/roadmap
>> >> ---
>> >>
>> >> In this session we discussed the current status of containers in TripleO
>> >> (which is to say, the container based compute node which deploys 
>> >> containers
>> >> via Heat onto an an Atomic host node that is also deployed via Heat), and
>> >> what strategy is most appropriate to achieve a fully containerized TripleO
>> >> deployment.
>> >>
>> >> Several folks from Kolla participated in the session, and there was
>> >> significant focus on where work may happen such that further collaboration
>> >> between communities is possible.  To some extent this discussion on where
>> >> (as opposed to how) proved a distraction and prevented much discussion on
>> >> supportable architectural implementation for TripleO, thus what follows is
>> >> mostly my perspective on the issues that exist:
>> >>
>> >> Significant uncertainty exists wrt integration between Kolla and TripleO -
>> >> there's largely consensus that we want to consume the container images
>> >> defined by the Kolla community, but much less agreement that we can
>> >> feasably switch to the ansible-orchestrated deployment/config flow
>> >> supported by Kolla without breaking many of our primary operator 
>> >> interfaces
>> >> in a fundamentally unacceptable way, for example:
>> >>
>> >> - The Mistral based API is being implemented on the expectation that the
>> >>   primary interface to TripleO deployments is a parameters schema exposed
>> >>   by a series of Heat templates - this is no longer true in a "split 
>> >> stack"
>> >>   model where we have to hand off to an alternate service orchestration 
>> >> tool.
>> >>
>> >> - The tripleo-ui (based on the Mistral based API) consumes heat parameter
>> >>   schema to build it's UI, and Ansible doesn't support the necessary
>> >>   parameter schema definition (such as types and descriptions) to enable
>> >>   this pattern to be replicated.  Ansible also doesn't provide a HTTP API,
>> >>   so we'd still have to maintain and API surface for the (non python) UI 
>> >> to
>> >>   consume.
>> >>
>> >> We also discussed ideas around integration with kubernetes (a hot topic on
>> >> the Kolla track this summit), but again this proved inconclusive beyond
>> >> that yes someone should try developing a PoC to stimulate further
>> >> discussion.  Again, significant challenges exist:
>> >>
>> >> - We still need to maintain the Heat parameter interfaces for the API/UI,
>> >>   and there is also a strong preference to maintain 

Re: [openstack-dev] [TripleO] Austin summit - session recap/summary

[Paul Belanger]

On Wed, May 18, 2016 at 12:22:55PM +0100, Derek Higgins wrote:
> On 6 May 2016 at 14:18, Paul Belanger  wrote:
> > On Tue, May 03, 2016 at 05:34:55PM +0100, Steven Hardy wrote:
> >> Hi all,
> >>
> >> Some folks have requested a summary of our summit sessions, as has been
> >> provided for some other projects.
> >>
> >> I'll probably go into more detail on some of these topics either via
> >> subsequent more focussed threads an/or some blog posts but what follows is
> >> an overview of our summit sessions[1] with notable actions or decisions
> >> highlighted.  I'm including some of my own thoughts and conclusions, folks
> >> are welcome/encouraged to follow up with their own clarifications or
> >> different perspectives :)
> >>
> >> TripleO had a total of 5 sessions in Austin I'll cover them one-by-one:
> >>
> >> -
> >> Upgrades - current status and roadmap
> >> -
> >>
> >> In this session we discussed the current state of upgrades - initial
> >> support for full major version upgrades has been implemented, but the
> >> implementation is monolithic, highly coupled to pacemaker, and inflexible
> >> with regard to third-party extraconfig changes.
> >>
> >> The main outcomes were that we will add support for more granular
> >> definition of the upgrade lifecycle to the new composable services format,
> >> and that we will explore moving towards the proposed lightweight HA
> >> architecture to reduce the need for so much pacemaker specific logic.
> >>
> >> We also agreed that investigating use of mistral to drive upgrade workflows
> >> was a good idea - currently we have a mixture of scripts combined with Heat
> >> to drive the upgrade process, and some refactoring into discrete mistral
> >> workflows may provide a more maintainable solution.  Potential for using
> >> the existing SoftwareDeployment approach directly via mistral (outside of
> >> the heat templates) was also discussed as something to be further
> >> investigated and prototyped.
> >>
> >> We also touched on the CI implications of upgrades - we've got an upgrades
> >> job now, but we need to ensure coverage of full release-to-release upgrades
> >> (not just commit to commit).
> >>
> >> ---
> >> Containerization status/roadmap
> >> ---
> >>
> >> In this session we discussed the current status of containers in TripleO
> >> (which is to say, the container based compute node which deploys containers
> >> via Heat onto an an Atomic host node that is also deployed via Heat), and
> >> what strategy is most appropriate to achieve a fully containerized TripleO
> >> deployment.
> >>
> >> Several folks from Kolla participated in the session, and there was
> >> significant focus on where work may happen such that further collaboration
> >> between communities is possible.  To some extent this discussion on where
> >> (as opposed to how) proved a distraction and prevented much discussion on
> >> supportable architectural implementation for TripleO, thus what follows is
> >> mostly my perspective on the issues that exist:
> >>
> >> Significant uncertainty exists wrt integration between Kolla and TripleO -
> >> there's largely consensus that we want to consume the container images
> >> defined by the Kolla community, but much less agreement that we can
> >> feasably switch to the ansible-orchestrated deployment/config flow
> >> supported by Kolla without breaking many of our primary operator interfaces
> >> in a fundamentally unacceptable way, for example:
> >>
> >> - The Mistral based API is being implemented on the expectation that the
> >>   primary interface to TripleO deployments is a parameters schema exposed
> >>   by a series of Heat templates - this is no longer true in a "split stack"
> >>   model where we have to hand off to an alternate service orchestration 
> >> tool.
> >>
> >> - The tripleo-ui (based on the Mistral based API) consumes heat parameter
> >>   schema to build it's UI, and Ansible doesn't support the necessary
> >>   parameter schema definition (such as types and descriptions) to enable
> >>   this pattern to be replicated.  Ansible also doesn't provide a HTTP API,
> >>   so we'd still have to maintain and API surface for the (non python) UI to
> >>   consume.
> >>
> >> We also discussed ideas around integration with kubernetes (a hot topic on
> >> the Kolla track this summit), but again this proved inconclusive beyond
> >> that yes someone should try developing a PoC to stimulate further
> >> discussion.  Again, significant challenges exist:
> >>
> >> - We still need to maintain the Heat parameter interfaces for the API/UI,
> >>   and there is also a strong preference to maintain puppet as a tool for
> >>   generating service configuration (so that existing operator integrations
> >>   via puppet continue to function) - this is a barrier to directly
> >>   consuming the 

Re: [openstack-dev] [TripleO] Austin summit - session recap/summary

[Derek Higgins]

On 6 May 2016 at 14:18, Paul Belanger  wrote:
> On Tue, May 03, 2016 at 05:34:55PM +0100, Steven Hardy wrote:
>> Hi all,
>>
>> Some folks have requested a summary of our summit sessions, as has been
>> provided for some other projects.
>>
>> I'll probably go into more detail on some of these topics either via
>> subsequent more focussed threads an/or some blog posts but what follows is
>> an overview of our summit sessions[1] with notable actions or decisions
>> highlighted.  I'm including some of my own thoughts and conclusions, folks
>> are welcome/encouraged to follow up with their own clarifications or
>> different perspectives :)
>>
>> TripleO had a total of 5 sessions in Austin I'll cover them one-by-one:
>>
>> -
>> Upgrades - current status and roadmap
>> -
>>
>> In this session we discussed the current state of upgrades - initial
>> support for full major version upgrades has been implemented, but the
>> implementation is monolithic, highly coupled to pacemaker, and inflexible
>> with regard to third-party extraconfig changes.
>>
>> The main outcomes were that we will add support for more granular
>> definition of the upgrade lifecycle to the new composable services format,
>> and that we will explore moving towards the proposed lightweight HA
>> architecture to reduce the need for so much pacemaker specific logic.
>>
>> We also agreed that investigating use of mistral to drive upgrade workflows
>> was a good idea - currently we have a mixture of scripts combined with Heat
>> to drive the upgrade process, and some refactoring into discrete mistral
>> workflows may provide a more maintainable solution.  Potential for using
>> the existing SoftwareDeployment approach directly via mistral (outside of
>> the heat templates) was also discussed as something to be further
>> investigated and prototyped.
>>
>> We also touched on the CI implications of upgrades - we've got an upgrades
>> job now, but we need to ensure coverage of full release-to-release upgrades
>> (not just commit to commit).
>>
>> ---
>> Containerization status/roadmap
>> ---
>>
>> In this session we discussed the current status of containers in TripleO
>> (which is to say, the container based compute node which deploys containers
>> via Heat onto an an Atomic host node that is also deployed via Heat), and
>> what strategy is most appropriate to achieve a fully containerized TripleO
>> deployment.
>>
>> Several folks from Kolla participated in the session, and there was
>> significant focus on where work may happen such that further collaboration
>> between communities is possible.  To some extent this discussion on where
>> (as opposed to how) proved a distraction and prevented much discussion on
>> supportable architectural implementation for TripleO, thus what follows is
>> mostly my perspective on the issues that exist:
>>
>> Significant uncertainty exists wrt integration between Kolla and TripleO -
>> there's largely consensus that we want to consume the container images
>> defined by the Kolla community, but much less agreement that we can
>> feasably switch to the ansible-orchestrated deployment/config flow
>> supported by Kolla without breaking many of our primary operator interfaces
>> in a fundamentally unacceptable way, for example:
>>
>> - The Mistral based API is being implemented on the expectation that the
>>   primary interface to TripleO deployments is a parameters schema exposed
>>   by a series of Heat templates - this is no longer true in a "split stack"
>>   model where we have to hand off to an alternate service orchestration tool.
>>
>> - The tripleo-ui (based on the Mistral based API) consumes heat parameter
>>   schema to build it's UI, and Ansible doesn't support the necessary
>>   parameter schema definition (such as types and descriptions) to enable
>>   this pattern to be replicated.  Ansible also doesn't provide a HTTP API,
>>   so we'd still have to maintain and API surface for the (non python) UI to
>>   consume.
>>
>> We also discussed ideas around integration with kubernetes (a hot topic on
>> the Kolla track this summit), but again this proved inconclusive beyond
>> that yes someone should try developing a PoC to stimulate further
>> discussion.  Again, significant challenges exist:
>>
>> - We still need to maintain the Heat parameter interfaces for the API/UI,
>>   and there is also a strong preference to maintain puppet as a tool for
>>   generating service configuration (so that existing operator integrations
>>   via puppet continue to function) - this is a barrier to directly
>>   consuming the kolla-kubernetes effort directly.
>>
>> - A COE layer like kubernetes is a poor fit for deployments where operators
>>   require strict control of service placement (e.g exactly which nodes a 
>> service
>>   runs on, IP address assignments to specific nodes 

Re: [openstack-dev] [TripleO] Austin summit - session recap/summary

[Paul Belanger]

On Mon, May 09, 2016 at 04:50:19PM -0400, Paul Belanger wrote:
> On Fri, May 06, 2016 at 09:18:03AM -0400, Paul Belanger wrote:
> > On Tue, May 03, 2016 at 05:34:55PM +0100, Steven Hardy wrote:
> > > Hi all,
> > > 
> > > Some folks have requested a summary of our summit sessions, as has been
> > > provided for some other projects.
> > > 
> > > I'll probably go into more detail on some of these topics either via
> > > subsequent more focussed threads an/or some blog posts but what follows is
> > > an overview of our summit sessions[1] with notable actions or decisions
> > > highlighted.  I'm including some of my own thoughts and conclusions, folks
> > > are welcome/encouraged to follow up with their own clarifications or
> > > different perspectives :)
> > > 
> > > TripleO had a total of 5 sessions in Austin I'll cover them one-by-one:
> > > 
> > > -
> > > Upgrades - current status and roadmap
> > > -
> > > 
> > > In this session we discussed the current state of upgrades - initial
> > > support for full major version upgrades has been implemented, but the
> > > implementation is monolithic, highly coupled to pacemaker, and inflexible
> > > with regard to third-party extraconfig changes.
> > > 
> > > The main outcomes were that we will add support for more granular
> > > definition of the upgrade lifecycle to the new composable services format,
> > > and that we will explore moving towards the proposed lightweight HA
> > > architecture to reduce the need for so much pacemaker specific logic.
> > > 
> > > We also agreed that investigating use of mistral to drive upgrade 
> > > workflows
> > > was a good idea - currently we have a mixture of scripts combined with 
> > > Heat
> > > to drive the upgrade process, and some refactoring into discrete mistral
> > > workflows may provide a more maintainable solution.  Potential for using
> > > the existing SoftwareDeployment approach directly via mistral (outside of
> > > the heat templates) was also discussed as something to be further
> > > investigated and prototyped.
> > > 
> > > We also touched on the CI implications of upgrades - we've got an upgrades
> > > job now, but we need to ensure coverage of full release-to-release 
> > > upgrades
> > > (not just commit to commit).
> > > 
> > > ---
> > > Containerization status/roadmap
> > > ---
> > > 
> > > In this session we discussed the current status of containers in TripleO
> > > (which is to say, the container based compute node which deploys 
> > > containers
> > > via Heat onto an an Atomic host node that is also deployed via Heat), and
> > > what strategy is most appropriate to achieve a fully containerized TripleO
> > > deployment.
> > > 
> > > Several folks from Kolla participated in the session, and there was
> > > significant focus on where work may happen such that further collaboration
> > > between communities is possible.  To some extent this discussion on where
> > > (as opposed to how) proved a distraction and prevented much discussion on
> > > supportable architectural implementation for TripleO, thus what follows is
> > > mostly my perspective on the issues that exist:
> > > 
> > > Significant uncertainty exists wrt integration between Kolla and TripleO -
> > > there's largely consensus that we want to consume the container images
> > > defined by the Kolla community, but much less agreement that we can
> > > feasably switch to the ansible-orchestrated deployment/config flow
> > > supported by Kolla without breaking many of our primary operator 
> > > interfaces
> > > in a fundamentally unacceptable way, for example:
> > > 
> > > - The Mistral based API is being implemented on the expectation that the
> > >   primary interface to TripleO deployments is a parameters schema exposed
> > >   by a series of Heat templates - this is no longer true in a "split 
> > > stack"
> > >   model where we have to hand off to an alternate service orchestration 
> > > tool.
> > > 
> > > - The tripleo-ui (based on the Mistral based API) consumes heat parameter
> > >   schema to build it's UI, and Ansible doesn't support the necessary
> > >   parameter schema definition (such as types and descriptions) to enable
> > >   this pattern to be replicated.  Ansible also doesn't provide a HTTP API,
> > >   so we'd still have to maintain and API surface for the (non python) UI 
> > > to
> > >   consume.
> > > 
> > > We also discussed ideas around integration with kubernetes (a hot topic on
> > > the Kolla track this summit), but again this proved inconclusive beyond
> > > that yes someone should try developing a PoC to stimulate further
> > > discussion.  Again, significant challenges exist:
> > > 
> > > - We still need to maintain the Heat parameter interfaces for the API/UI,
> > >   and there is also a strong preference to maintain puppet as a tool for
> > >   generating service configuration (so 

Re: [openstack-dev] [TripleO] Austin summit - session recap/summary

[Paul Belanger]

On Fri, May 06, 2016 at 09:18:03AM -0400, Paul Belanger wrote:
> On Tue, May 03, 2016 at 05:34:55PM +0100, Steven Hardy wrote:
> > Hi all,
> > 
> > Some folks have requested a summary of our summit sessions, as has been
> > provided for some other projects.
> > 
> > I'll probably go into more detail on some of these topics either via
> > subsequent more focussed threads an/or some blog posts but what follows is
> > an overview of our summit sessions[1] with notable actions or decisions
> > highlighted.  I'm including some of my own thoughts and conclusions, folks
> > are welcome/encouraged to follow up with their own clarifications or
> > different perspectives :)
> > 
> > TripleO had a total of 5 sessions in Austin I'll cover them one-by-one:
> > 
> > -
> > Upgrades - current status and roadmap
> > -
> > 
> > In this session we discussed the current state of upgrades - initial
> > support for full major version upgrades has been implemented, but the
> > implementation is monolithic, highly coupled to pacemaker, and inflexible
> > with regard to third-party extraconfig changes.
> > 
> > The main outcomes were that we will add support for more granular
> > definition of the upgrade lifecycle to the new composable services format,
> > and that we will explore moving towards the proposed lightweight HA
> > architecture to reduce the need for so much pacemaker specific logic.
> > 
> > We also agreed that investigating use of mistral to drive upgrade workflows
> > was a good idea - currently we have a mixture of scripts combined with Heat
> > to drive the upgrade process, and some refactoring into discrete mistral
> > workflows may provide a more maintainable solution.  Potential for using
> > the existing SoftwareDeployment approach directly via mistral (outside of
> > the heat templates) was also discussed as something to be further
> > investigated and prototyped.
> > 
> > We also touched on the CI implications of upgrades - we've got an upgrades
> > job now, but we need to ensure coverage of full release-to-release upgrades
> > (not just commit to commit).
> > 
> > ---
> > Containerization status/roadmap
> > ---
> > 
> > In this session we discussed the current status of containers in TripleO
> > (which is to say, the container based compute node which deploys containers
> > via Heat onto an an Atomic host node that is also deployed via Heat), and
> > what strategy is most appropriate to achieve a fully containerized TripleO
> > deployment.
> > 
> > Several folks from Kolla participated in the session, and there was
> > significant focus on where work may happen such that further collaboration
> > between communities is possible.  To some extent this discussion on where
> > (as opposed to how) proved a distraction and prevented much discussion on
> > supportable architectural implementation for TripleO, thus what follows is
> > mostly my perspective on the issues that exist:
> > 
> > Significant uncertainty exists wrt integration between Kolla and TripleO -
> > there's largely consensus that we want to consume the container images
> > defined by the Kolla community, but much less agreement that we can
> > feasably switch to the ansible-orchestrated deployment/config flow
> > supported by Kolla without breaking many of our primary operator interfaces
> > in a fundamentally unacceptable way, for example:
> > 
> > - The Mistral based API is being implemented on the expectation that the
> >   primary interface to TripleO deployments is a parameters schema exposed
> >   by a series of Heat templates - this is no longer true in a "split stack"
> >   model where we have to hand off to an alternate service orchestration 
> > tool.
> > 
> > - The tripleo-ui (based on the Mistral based API) consumes heat parameter
> >   schema to build it's UI, and Ansible doesn't support the necessary
> >   parameter schema definition (such as types and descriptions) to enable
> >   this pattern to be replicated.  Ansible also doesn't provide a HTTP API,
> >   so we'd still have to maintain and API surface for the (non python) UI to
> >   consume.
> > 
> > We also discussed ideas around integration with kubernetes (a hot topic on
> > the Kolla track this summit), but again this proved inconclusive beyond
> > that yes someone should try developing a PoC to stimulate further
> > discussion.  Again, significant challenges exist:
> > 
> > - We still need to maintain the Heat parameter interfaces for the API/UI,
> >   and there is also a strong preference to maintain puppet as a tool for
> >   generating service configuration (so that existing operator integrations
> >   via puppet continue to function) - this is a barrier to directly
> >   consuming the kolla-kubernetes effort directly.
> > 
> > - A COE layer like kubernetes is a poor fit for deployments where operators
> >   require strict control of service 

Re: [openstack-dev] [TripleO] Austin summit - session recap/summary

[Paul Belanger]

On Tue, May 03, 2016 at 05:34:55PM +0100, Steven Hardy wrote:
> Hi all,
> 
> Some folks have requested a summary of our summit sessions, as has been
> provided for some other projects.
> 
> I'll probably go into more detail on some of these topics either via
> subsequent more focussed threads an/or some blog posts but what follows is
> an overview of our summit sessions[1] with notable actions or decisions
> highlighted.  I'm including some of my own thoughts and conclusions, folks
> are welcome/encouraged to follow up with their own clarifications or
> different perspectives :)
> 
> TripleO had a total of 5 sessions in Austin I'll cover them one-by-one:
> 
> -
> Upgrades - current status and roadmap
> -
> 
> In this session we discussed the current state of upgrades - initial
> support for full major version upgrades has been implemented, but the
> implementation is monolithic, highly coupled to pacemaker, and inflexible
> with regard to third-party extraconfig changes.
> 
> The main outcomes were that we will add support for more granular
> definition of the upgrade lifecycle to the new composable services format,
> and that we will explore moving towards the proposed lightweight HA
> architecture to reduce the need for so much pacemaker specific logic.
> 
> We also agreed that investigating use of mistral to drive upgrade workflows
> was a good idea - currently we have a mixture of scripts combined with Heat
> to drive the upgrade process, and some refactoring into discrete mistral
> workflows may provide a more maintainable solution.  Potential for using
> the existing SoftwareDeployment approach directly via mistral (outside of
> the heat templates) was also discussed as something to be further
> investigated and prototyped.
> 
> We also touched on the CI implications of upgrades - we've got an upgrades
> job now, but we need to ensure coverage of full release-to-release upgrades
> (not just commit to commit).
> 
> ---
> Containerization status/roadmap
> ---
> 
> In this session we discussed the current status of containers in TripleO
> (which is to say, the container based compute node which deploys containers
> via Heat onto an an Atomic host node that is also deployed via Heat), and
> what strategy is most appropriate to achieve a fully containerized TripleO
> deployment.
> 
> Several folks from Kolla participated in the session, and there was
> significant focus on where work may happen such that further collaboration
> between communities is possible.  To some extent this discussion on where
> (as opposed to how) proved a distraction and prevented much discussion on
> supportable architectural implementation for TripleO, thus what follows is
> mostly my perspective on the issues that exist:
> 
> Significant uncertainty exists wrt integration between Kolla and TripleO -
> there's largely consensus that we want to consume the container images
> defined by the Kolla community, but much less agreement that we can
> feasably switch to the ansible-orchestrated deployment/config flow
> supported by Kolla without breaking many of our primary operator interfaces
> in a fundamentally unacceptable way, for example:
> 
> - The Mistral based API is being implemented on the expectation that the
>   primary interface to TripleO deployments is a parameters schema exposed
>   by a series of Heat templates - this is no longer true in a "split stack"
>   model where we have to hand off to an alternate service orchestration tool.
> 
> - The tripleo-ui (based on the Mistral based API) consumes heat parameter
>   schema to build it's UI, and Ansible doesn't support the necessary
>   parameter schema definition (such as types and descriptions) to enable
>   this pattern to be replicated.  Ansible also doesn't provide a HTTP API,
>   so we'd still have to maintain and API surface for the (non python) UI to
>   consume.
> 
> We also discussed ideas around integration with kubernetes (a hot topic on
> the Kolla track this summit), but again this proved inconclusive beyond
> that yes someone should try developing a PoC to stimulate further
> discussion.  Again, significant challenges exist:
> 
> - We still need to maintain the Heat parameter interfaces for the API/UI,
>   and there is also a strong preference to maintain puppet as a tool for
>   generating service configuration (so that existing operator integrations
>   via puppet continue to function) - this is a barrier to directly
>   consuming the kolla-kubernetes effort directly.
> 
> - A COE layer like kubernetes is a poor fit for deployments where operators
>   require strict control of service placement (e.g exactly which nodes a 
> service
>   runs on, IP address assignments to specific nodes etc) - this is already
>   a strong requirement for TripleO users and we need to figure out if/how
>   it's possible to control container 

Re: [openstack-dev] [TripleO] Austin summit - session recap/summary

[Sven Anderson]

Thanks a ton, Steve! I have to admit, although I was at the summit in
person, I can get out of your write-up a lot more than from the sessions
itself. (Probably because of a mixture of not being a native speaker and
being new to tripleO.)

Cheers,

Sven
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] Austin summit - session recap/summary

[Jason Rist]

On 05/03/2016 10:34 AM, Steven Hardy wrote:
> Hi all,
>
> Some folks have requested a summary of our summit sessions, as has been
> provided for some other projects.
>
> I'll probably go into more detail on some of these topics either via
> subsequent more focussed threads an/or some blog posts but what follows is
> an overview of our summit sessions[1] with notable actions or decisions
> highlighted.  I'm including some of my own thoughts and conclusions, folks
> are welcome/encouraged to follow up with their own clarifications or
> different perspectives :)
>
> TripleO had a total of 5 sessions in Austin I'll cover them one-by-one:
>
> -
> Upgrades - current status and roadmap
> -
>
> In this session we discussed the current state of upgrades - initial
> support for full major version upgrades has been implemented, but the
> implementation is monolithic, highly coupled to pacemaker, and inflexible
> with regard to third-party extraconfig changes.
>
> The main outcomes were that we will add support for more granular
> definition of the upgrade lifecycle to the new composable services format,
> and that we will explore moving towards the proposed lightweight HA
> architecture to reduce the need for so much pacemaker specific logic.
>
> We also agreed that investigating use of mistral to drive upgrade workflows
> was a good idea - currently we have a mixture of scripts combined with Heat
> to drive the upgrade process, and some refactoring into discrete mistral
> workflows may provide a more maintainable solution.  Potential for using
> the existing SoftwareDeployment approach directly via mistral (outside of
> the heat templates) was also discussed as something to be further
> investigated and prototyped.
>
> We also touched on the CI implications of upgrades - we've got an upgrades
> job now, but we need to ensure coverage of full release-to-release upgrades
> (not just commit to commit).
>
> ---
> Containerization status/roadmap
> ---
>
> In this session we discussed the current status of containers in TripleO
> (which is to say, the container based compute node which deploys containers
> via Heat onto an an Atomic host node that is also deployed via Heat), and
> what strategy is most appropriate to achieve a fully containerized TripleO
> deployment.
>
> Several folks from Kolla participated in the session, and there was
> significant focus on where work may happen such that further collaboration
> between communities is possible.  To some extent this discussion on where
> (as opposed to how) proved a distraction and prevented much discussion on
> supportable architectural implementation for TripleO, thus what follows is
> mostly my perspective on the issues that exist:
>
> Significant uncertainty exists wrt integration between Kolla and TripleO -
> there's largely consensus that we want to consume the container images
> defined by the Kolla community, but much less agreement that we can
> feasably switch to the ansible-orchestrated deployment/config flow
> supported by Kolla without breaking many of our primary operator interfaces
> in a fundamentally unacceptable way, for example:
>
> - The Mistral based API is being implemented on the expectation that the
>   primary interface to TripleO deployments is a parameters schema exposed
>   by a series of Heat templates - this is no longer true in a "split stack"
>   model where we have to hand off to an alternate service orchestration tool.
>
> - The tripleo-ui (based on the Mistral based API) consumes heat parameter
>   schema to build it's UI, and Ansible doesn't support the necessary
>   parameter schema definition (such as types and descriptions) to enable
>   this pattern to be replicated.  Ansible also doesn't provide a HTTP API,
>   so we'd still have to maintain and API surface for the (non python) UI to
>   consume.
>
> We also discussed ideas around integration with kubernetes (a hot topic on
> the Kolla track this summit), but again this proved inconclusive beyond
> that yes someone should try developing a PoC to stimulate further
> discussion.  Again, significant challenges exist:
>
> - We still need to maintain the Heat parameter interfaces for the API/UI,
>   and there is also a strong preference to maintain puppet as a tool for
>   generating service configuration (so that existing operator integrations
>   via puppet continue to function) - this is a barrier to directly
>   consuming the kolla-kubernetes effort directly.
>
> - A COE layer like kubernetes is a poor fit for deployments where operators
>   require strict control of service placement (e.g exactly which nodes a 
> service
>   runs on, IP address assignments to specific nodes etc) - this is already
>   a strong requirement for TripleO users and we need to figure out if/how
>   it's possible to control container placement per node/namespace.
>
> - There 

[openstack-dev] [TripleO] Austin summit - session recap/summary

[Steven Hardy]

Hi all,

Some folks have requested a summary of our summit sessions, as has been
provided for some other projects.

I'll probably go into more detail on some of these topics either via
subsequent more focussed threads an/or some blog posts but what follows is
an overview of our summit sessions[1] with notable actions or decisions
highlighted.  I'm including some of my own thoughts and conclusions, folks
are welcome/encouraged to follow up with their own clarifications or
different perspectives :)

TripleO had a total of 5 sessions in Austin I'll cover them one-by-one:

-
Upgrades - current status and roadmap
-

In this session we discussed the current state of upgrades - initial
support for full major version upgrades has been implemented, but the
implementation is monolithic, highly coupled to pacemaker, and inflexible
with regard to third-party extraconfig changes.

The main outcomes were that we will add support for more granular
definition of the upgrade lifecycle to the new composable services format,
and that we will explore moving towards the proposed lightweight HA
architecture to reduce the need for so much pacemaker specific logic.

We also agreed that investigating use of mistral to drive upgrade workflows
was a good idea - currently we have a mixture of scripts combined with Heat
to drive the upgrade process, and some refactoring into discrete mistral
workflows may provide a more maintainable solution.  Potential for using
the existing SoftwareDeployment approach directly via mistral (outside of
the heat templates) was also discussed as something to be further
investigated and prototyped.

We also touched on the CI implications of upgrades - we've got an upgrades
job now, but we need to ensure coverage of full release-to-release upgrades
(not just commit to commit).

---
Containerization status/roadmap
---

In this session we discussed the current status of containers in TripleO
(which is to say, the container based compute node which deploys containers
via Heat onto an an Atomic host node that is also deployed via Heat), and
what strategy is most appropriate to achieve a fully containerized TripleO
deployment.

Several folks from Kolla participated in the session, and there was
significant focus on where work may happen such that further collaboration
between communities is possible.  To some extent this discussion on where
(as opposed to how) proved a distraction and prevented much discussion on
supportable architectural implementation for TripleO, thus what follows is
mostly my perspective on the issues that exist:

Significant uncertainty exists wrt integration between Kolla and TripleO -
there's largely consensus that we want to consume the container images
defined by the Kolla community, but much less agreement that we can
feasably switch to the ansible-orchestrated deployment/config flow
supported by Kolla without breaking many of our primary operator interfaces
in a fundamentally unacceptable way, for example:

- The Mistral based API is being implemented on the expectation that the
  primary interface to TripleO deployments is a parameters schema exposed
  by a series of Heat templates - this is no longer true in a "split stack"
  model where we have to hand off to an alternate service orchestration tool.

- The tripleo-ui (based on the Mistral based API) consumes heat parameter
  schema to build it's UI, and Ansible doesn't support the necessary
  parameter schema definition (such as types and descriptions) to enable
  this pattern to be replicated.  Ansible also doesn't provide a HTTP API,
  so we'd still have to maintain and API surface for the (non python) UI to
  consume.

We also discussed ideas around integration with kubernetes (a hot topic on
the Kolla track this summit), but again this proved inconclusive beyond
that yes someone should try developing a PoC to stimulate further
discussion.  Again, significant challenges exist:

- We still need to maintain the Heat parameter interfaces for the API/UI,
  and there is also a strong preference to maintain puppet as a tool for
  generating service configuration (so that existing operator integrations
  via puppet continue to function) - this is a barrier to directly
  consuming the kolla-kubernetes effort directly.

- A COE layer like kubernetes is a poor fit for deployments where operators
  require strict control of service placement (e.g exactly which nodes a service
  runs on, IP address assignments to specific nodes etc) - this is already
  a strong requirement for TripleO users and we need to figure out if/how
  it's possible to control container placement per node/namespace.

- There are several uncertainties regarding the HA architecture, such as
  how do we achieve fencing for nodes (which is currently provided via
  pacemaker), in particular the HA model for real production deployments
  via