Re: [openstack-dev] [Fuel] Install fuel-libraryX.Y as a package on slave nodes

[Dmitry Borodaenko]

+1 to y'all :)

We already have a blueprint to enable building Fuel packages with
Perestroika:
https://blueprints.launchpad.net/fuel/+spec/build-fuel-packages-using-perestroika

Between that and packaging Perestroika itself as a self-sufficient tool
that a developer can easily set up and run locally (which we also need a
blueprint for), I think we'd have enough tools to distribute
fuel-library to target node as packages both in production, and, without
too much additional effort, as part of development workflow.

-DmitryB

On Fri, Sep 11, 2015 at 10:59:40PM +0300, Andrey Danin wrote:
> I support this proposal but I just wanted to mention that we'll lose an
> easy way to develop manifests. I agree that manifests in this case have no
> difference with Neutron code, for instance. But anyway I +1 this,
> especially with Vova Kuklin's additions.
> 
> On Thu, Sep 10, 2015 at 12:25 PM, Vladimir Kuklin 
> wrote:
> 
> > Folks
> >
> > I have a strong +1 for the proposal to decouple master node and slave
> > nodes.
> > Here are the stregnths of this approach
> > 1) We can always decide which particular node runs which particular set of
> > manifests. This will allow us to do be able to apply/roll back changes
> > node-by-node. This is very important from operations perspective.
> > 2) We can decouple master and slave nodes manifests and not drag new
> > library version onto the master node when it is not needed. This allows us
> > to decrease probability of regressions
> > 3) This makes life easier for the user - you just run 'apt-get/yum
> > install' instead of some difficult to digest `mco` command.
> >
> > The only weakness that I see here is on mentioned by Andrey. I think we
> > can fix it by providing developers with clean and simple way of building
> > library package on the fly. This will make developers life easier enough to
> > work with proposed approach.
> >
> > Also, we need to provide ways for better UX, like provide one button/api
> > call for:
> >
> > 1) update all manifests on particular nodes (e.g. all or only a part of
> > nodes of the cluster) to particular version
> > 2)  revert all manifests back to the version which is provided by the
> > particular GA release
> > 3) 
> >
> > So far I would mark need for simple package-building system for developer
> > as a dependency for the proposed change, but I do not see any other way
> > than proceeding with it.
> >
> >
> >
> > On Thu, Sep 10, 2015 at 11:50 AM, Sergii Golovatiuk <
> > sgolovat...@mirantis.com> wrote:
> >
> >> Oleg,
> >>
> >> Alex gave a perfect example regarding support folks when they need to fix
> >> something really quick. It's client's choice what to patch or not. You may
> >> like it or not, but it's client's choice.
> >>
> >> On 10 Sep 2015, at 09:33, Oleg Gelbukh  wrote:
> >>
> >> Alex,
> >>
> >> I absolutely understand the point you are making about need for
> >> deployment engineers to modify things 'on the fly' in customer environment.
> >> It's makes things really flexible and lowers the entry barrier for sure.
> >>
> >> However, I would like to note that in my opinion this kind on 'monkey
> >> patching' is actually a bad practice for any environments other than dev
> >> ones. It immediately leads to emergence of unsupportable frankenclouds. I
> >> would greet any modification to the workflow that will discourage people
> >> from doing that.
> >>
> >> --
> >> Best regards,
> >> Oleg Gelbukh
> >>
> >> On Wed, Sep 9, 2015 at 5:56 PM, Alex Schultz 
> >> wrote:
> >>
> >>> Hey Vladimir,
> >>>
> >>>
> >>>
>  Regarding plugins: plugins are welcome to install specific additional
>  DEB/RPM repos on the master node, or just configure cluster to use
>  additional onl?ne repos, where all necessary packages (including plugin
>  specific puppet manifests) are to be available. Current granular 
>  deployment
>  approach makes it easy to append specific pre-deployment tasks
>  (master/slave does not matter). Correct me if I am wrong.
> 
> 
> >>> Don't get me wrong, I think it would be good to move to a fuel-library
> >>> distributed via package only.  I'm bringing these points up to indicate
> >>> that there is many other things that live in the fuel library puppet path
> >>> than just the fuel-library package.  The plugin example is just one place
> >>> that we will need to invest in further design and work to move to the
> >>> package only distribution.  What I don't want is some partially executed
> >>> work that only works for one type of deployment and creates headaches for
> >>> the people actually having to use fuel.  The deployment engineers and
> >>> customers who actually perform these actions should be asked about
> >>> packaging and their comfort level with this type of requirements.  I don't
> >>> have a complete understanding of the all the things supported today by the
> >>> fuel plugin system so it would be nice to get 

Re: [openstack-dev] [Fuel] Install fuel-libraryX.Y as a package on slave nodes

[Andrey Danin]

I support this proposal but I just wanted to mention that we'll lose an
easy way to develop manifests. I agree that manifests in this case have no
difference with Neutron code, for instance. But anyway I +1 this,
especially with Vova Kuklin's additions.

On Thu, Sep 10, 2015 at 12:25 PM, Vladimir Kuklin 
wrote:

> Folks
>
> I have a strong +1 for the proposal to decouple master node and slave
> nodes.
> Here are the stregnths of this approach
> 1) We can always decide which particular node runs which particular set of
> manifests. This will allow us to do be able to apply/roll back changes
> node-by-node. This is very important from operations perspective.
> 2) We can decouple master and slave nodes manifests and not drag new
> library version onto the master node when it is not needed. This allows us
> to decrease probability of regressions
> 3) This makes life easier for the user - you just run 'apt-get/yum
> install' instead of some difficult to digest `mco` command.
>
> The only weakness that I see here is on mentioned by Andrey. I think we
> can fix it by providing developers with clean and simple way of building
> library package on the fly. This will make developers life easier enough to
> work with proposed approach.
>
> Also, we need to provide ways for better UX, like provide one button/api
> call for:
>
> 1) update all manifests on particular nodes (e.g. all or only a part of
> nodes of the cluster) to particular version
> 2)  revert all manifests back to the version which is provided by the
> particular GA release
> 3) 
>
> So far I would mark need for simple package-building system for developer
> as a dependency for the proposed change, but I do not see any other way
> than proceeding with it.
>
>
>
> On Thu, Sep 10, 2015 at 11:50 AM, Sergii Golovatiuk <
> sgolovat...@mirantis.com> wrote:
>
>> Oleg,
>>
>> Alex gave a perfect example regarding support folks when they need to fix
>> something really quick. It's client's choice what to patch or not. You may
>> like it or not, but it's client's choice.
>>
>> On 10 Sep 2015, at 09:33, Oleg Gelbukh  wrote:
>>
>> Alex,
>>
>> I absolutely understand the point you are making about need for
>> deployment engineers to modify things 'on the fly' in customer environment.
>> It's makes things really flexible and lowers the entry barrier for sure.
>>
>> However, I would like to note that in my opinion this kind on 'monkey
>> patching' is actually a bad practice for any environments other than dev
>> ones. It immediately leads to emergence of unsupportable frankenclouds. I
>> would greet any modification to the workflow that will discourage people
>> from doing that.
>>
>> --
>> Best regards,
>> Oleg Gelbukh
>>
>> On Wed, Sep 9, 2015 at 5:56 PM, Alex Schultz 
>> wrote:
>>
>>> Hey Vladimir,
>>>
>>>
>>>
 Regarding plugins: plugins are welcome to install specific additional
 DEB/RPM repos on the master node, or just configure cluster to use
 additional onl?ne repos, where all necessary packages (including plugin
 specific puppet manifests) are to be available. Current granular deployment
 approach makes it easy to append specific pre-deployment tasks
 (master/slave does not matter). Correct me if I am wrong.


>>> Don't get me wrong, I think it would be good to move to a fuel-library
>>> distributed via package only.  I'm bringing these points up to indicate
>>> that there is many other things that live in the fuel library puppet path
>>> than just the fuel-library package.  The plugin example is just one place
>>> that we will need to invest in further design and work to move to the
>>> package only distribution.  What I don't want is some partially executed
>>> work that only works for one type of deployment and creates headaches for
>>> the people actually having to use fuel.  The deployment engineers and
>>> customers who actually perform these actions should be asked about
>>> packaging and their comfort level with this type of requirements.  I don't
>>> have a complete understanding of the all the things supported today by the
>>> fuel plugin system so it would be nice to get someone who is more familiar
>>> to weigh in on this idea. Currently plugins are only rpms (no debs) and I
>>> don't think we are building fuel-library debs at this time either.  So
>>> without some work on both sides, we cannot move to just packages.
>>>
>>>
 Regarding flexibility: having several versioned directories with puppet
 modules on the master node, having several fuel-libraryX.Y packages
 installed on the master node makes things "exquisitely convoluted" rather
 than flexible. Like I said, it is flexible enough to use mcollective, plain
 rsync, etc. if you really need to do things manually. But we have
 convenient service (Perestroika) which builds packages in minutes if you
 need. Moreover, In the nearest future (by 8.0) Perestroika will be
 

Re: [openstack-dev] [Fuel] Install fuel-libraryX.Y as a package on slave nodes

[Mike Scherbakov]

+1 to Alex & Andrey. Let's just be very careful, and consider all the use
cases before making a change.
If we can have answers to all the use cases, then we are good to go.

Important thing which we need to fix now - is to enable easy UX for making
changes to environments after deployments. Like standard configuration
management allows you to do. Namely, being able to:
a) modify params on settings tab
b) modify templates / puppet manifests
and apply changes easily to nodes.

Now, we can do b) easy and just click Deploy button or run two-three
commands [1]. a) requires changes in Nailgun code to allow post-deployment
modification of settings (we currently lock settings tab after deployment).

If we switch to package installation and this workflow (change to
manifests + 2-3 commands to rsync/run puppet on nodes) will become a
nightmare - then we'll need to figure out something else. It has to be easy
to do development and use Fuel as configuration management tool.

[1] https://bugs.launchpad.net/fuel/+bug/1385615

On Wed, Sep 9, 2015 at 8:01 AM Alex Schultz  wrote:

> Hey Vladimir,
>
>
>
>> Regarding plugins: plugins are welcome to install specific additional
>> DEB/RPM repos on the master node, or just configure cluster to use
>> additional onl?ne repos, where all necessary packages (including plugin
>> specific puppet manifests) are to be available. Current granular deployment
>> approach makes it easy to append specific pre-deployment tasks
>> (master/slave does not matter). Correct me if I am wrong.
>>
>>
> Don't get me wrong, I think it would be good to move to a fuel-library
> distributed via package only.  I'm bringing these points up to indicate
> that there is many other things that live in the fuel library puppet path
> than just the fuel-library package.  The plugin example is just one place
> that we will need to invest in further design and work to move to the
> package only distribution.  What I don't want is some partially executed
> work that only works for one type of deployment and creates headaches for
> the people actually having to use fuel.  The deployment engineers and
> customers who actually perform these actions should be asked about
> packaging and their comfort level with this type of requirements.  I don't
> have a complete understanding of the all the things supported today by the
> fuel plugin system so it would be nice to get someone who is more familiar
> to weigh in on this idea. Currently plugins are only rpms (no debs) and I
> don't think we are building fuel-library debs at this time either.  So
> without some work on both sides, we cannot move to just packages.
>
>
>> Regarding flexibility: having several versioned directories with puppet
>> modules on the master node, having several fuel-libraryX.Y packages
>> installed on the master node makes things "exquisitely convoluted" rather
>> than flexible. Like I said, it is flexible enough to use mcollective, plain
>> rsync, etc. if you really need to do things manually. But we have
>> convenient service (Perestroika) which builds packages in minutes if you
>> need. Moreover, In the nearest future (by 8.0) Perestroika will be
>> available as an application independent from CI. So, what is wrong with
>> building fuel-library package? What if you want to troubleshoot nova (we
>> install it using packages)? Should we also use rsync for everything else
>> like nova, mysql, etc.?
>>
>>
> Yes, we do have a service like Perestroika to build packages for us.  That
> doesn't mean everyone else does or has access to do that today.  Setting up
> a build system is a major undertaking and making that a hard requirement to
> interact with our product may be a bit much for some customers.  In
> speaking with some support folks, there are times when files have to be
> munged to get around issues because there is no package or things are on
> fire so they can't wait for a package to become available for a fix.  We
> need to be careful not to impose limits without proper justification and
> due diligence.  We already build the fuel-library package, so there's no
> reason you couldn't try switching the rsync to install the package if it's
> available on a mirror.  I just think you're going to run into the issues I
> mentioned which need to be solved before we could just mark it done.
>
> -Alex
>
>
>
>> Vladimir Kozhukalov
>>
>> On Wed, Sep 9, 2015 at 4:39 PM, Alex Schultz 
>> wrote:
>>
>>> I agree that we shouldn't need to sync as we should be able to just
>>> update the fuel-library package. That being said, I think there might be a
>>> few issues with this method. The first issue is with plugins and how to
>>> properly handle the distribution of the plugins as they may also include
>>> puppet code that needs to be installed on the other nodes for a deployment.
>>> Currently I do not believe we install the plugin packages anywhere except
>>> the master and when they do get installed there may be 

Re: [openstack-dev] [Fuel] Install fuel-libraryX.Y as a package on slave nodes

[Oleg Gelbukh]

Alex,

I absolutely understand the point you are making about need for deployment
engineers to modify things 'on the fly' in customer environment. It's makes
things really flexible and lowers the entry barrier for sure.

However, I would like to note that in my opinion this kind on 'monkey
patching' is actually a bad practice for any environments other than dev
ones. It immediately leads to emergence of unsupportable frankenclouds. I
would greet any modification to the workflow that will discourage people
from doing that.

--
Best regards,
Oleg Gelbukh

On Wed, Sep 9, 2015 at 5:56 PM, Alex Schultz  wrote:

> Hey Vladimir,
>
>
>
>> Regarding plugins: plugins are welcome to install specific additional
>> DEB/RPM repos on the master node, or just configure cluster to use
>> additional onl?ne repos, where all necessary packages (including plugin
>> specific puppet manifests) are to be available. Current granular deployment
>> approach makes it easy to append specific pre-deployment tasks
>> (master/slave does not matter). Correct me if I am wrong.
>>
>>
> Don't get me wrong, I think it would be good to move to a fuel-library
> distributed via package only.  I'm bringing these points up to indicate
> that there is many other things that live in the fuel library puppet path
> than just the fuel-library package.  The plugin example is just one place
> that we will need to invest in further design and work to move to the
> package only distribution.  What I don't want is some partially executed
> work that only works for one type of deployment and creates headaches for
> the people actually having to use fuel.  The deployment engineers and
> customers who actually perform these actions should be asked about
> packaging and their comfort level with this type of requirements.  I don't
> have a complete understanding of the all the things supported today by the
> fuel plugin system so it would be nice to get someone who is more familiar
> to weigh in on this idea. Currently plugins are only rpms (no debs) and I
> don't think we are building fuel-library debs at this time either.  So
> without some work on both sides, we cannot move to just packages.
>
>
>> Regarding flexibility: having several versioned directories with puppet
>> modules on the master node, having several fuel-libraryX.Y packages
>> installed on the master node makes things "exquisitely convoluted" rather
>> than flexible. Like I said, it is flexible enough to use mcollective, plain
>> rsync, etc. if you really need to do things manually. But we have
>> convenient service (Perestroika) which builds packages in minutes if you
>> need. Moreover, In the nearest future (by 8.0) Perestroika will be
>> available as an application independent from CI. So, what is wrong with
>> building fuel-library package? What if you want to troubleshoot nova (we
>> install it using packages)? Should we also use rsync for everything else
>> like nova, mysql, etc.?
>>
>>
> Yes, we do have a service like Perestroika to build packages for us.  That
> doesn't mean everyone else does or has access to do that today.  Setting up
> a build system is a major undertaking and making that a hard requirement to
> interact with our product may be a bit much for some customers.  In
> speaking with some support folks, there are times when files have to be
> munged to get around issues because there is no package or things are on
> fire so they can't wait for a package to become available for a fix.  We
> need to be careful not to impose limits without proper justification and
> due diligence.  We already build the fuel-library package, so there's no
> reason you couldn't try switching the rsync to install the package if it's
> available on a mirror.  I just think you're going to run into the issues I
> mentioned which need to be solved before we could just mark it done.
>
> -Alex
>
>
>
>> Vladimir Kozhukalov
>>
>> On Wed, Sep 9, 2015 at 4:39 PM, Alex Schultz 
>> wrote:
>>
>>> I agree that we shouldn't need to sync as we should be able to just
>>> update the fuel-library package. That being said, I think there might be a
>>> few issues with this method. The first issue is with plugins and how to
>>> properly handle the distribution of the plugins as they may also include
>>> puppet code that needs to be installed on the other nodes for a deployment.
>>> Currently I do not believe we install the plugin packages anywhere except
>>> the master and when they do get installed there may be some post-install
>>> actions that are only valid for the master.  Another issue is being
>>> flexible enough to allow for deployment engineers to make custom changes
>>> for a given environment.  Unless we can provide an improved process to
>>> allow for people to provide in place modifications for an environment, we
>>> can't do away with the rsync.
>>>
>>> If we want to go completely down the package route (and we probably
>>> should), we need to make sure 

Re: [openstack-dev] [Fuel] Install fuel-libraryX.Y as a package on slave nodes

[Sergii Golovatiuk]

Oleg,

Alex gave a perfect example regarding support folks when they need to fix
something really quick. It's client's choice what to patch or not. You may
like it or not, but it's client's choice.

On 10 Sep 2015, at 09:33, Oleg Gelbukh  wrote:

Alex,

I absolutely understand the point you are making about need for deployment
engineers to modify things 'on the fly' in customer environment. It's makes
things really flexible and lowers the entry barrier for sure.

However, I would like to note that in my opinion this kind on 'monkey
patching' is actually a bad practice for any environments other than dev
ones. It immediately leads to emergence of unsupportable frankenclouds. I
would greet any modification to the workflow that will discourage people
from doing that.

--
Best regards,
Oleg Gelbukh

On Wed, Sep 9, 2015 at 5:56 PM, Alex Schultz  wrote:

> Hey Vladimir,
>
>
>
>> Regarding plugins: plugins are welcome to install specific additional
>> DEB/RPM repos on the master node, or just configure cluster to use
>> additional onl?ne repos, where all necessary packages (including plugin
>> specific puppet manifests) are to be available. Current granular deployment
>> approach makes it easy to append specific pre-deployment tasks
>> (master/slave does not matter). Correct me if I am wrong.
>>
>>
> Don't get me wrong, I think it would be good to move to a fuel-library
> distributed via package only.  I'm bringing these points up to indicate
> that there is many other things that live in the fuel library puppet path
> than just the fuel-library package.  The plugin example is just one place
> that we will need to invest in further design and work to move to the
> package only distribution.  What I don't want is some partially executed
> work that only works for one type of deployment and creates headaches for
> the people actually having to use fuel.  The deployment engineers and
> customers who actually perform these actions should be asked about
> packaging and their comfort level with this type of requirements.  I don't
> have a complete understanding of the all the things supported today by the
> fuel plugin system so it would be nice to get someone who is more familiar
> to weigh in on this idea. Currently plugins are only rpms (no debs) and I
> don't think we are building fuel-library debs at this time either.  So
> without some work on both sides, we cannot move to just packages.
>
>
>> Regarding flexibility: having several versioned directories with puppet
>> modules on the master node, having several fuel-libraryX.Y packages
>> installed on the master node makes things "exquisitely convoluted" rather
>> than flexible. Like I said, it is flexible enough to use mcollective, plain
>> rsync, etc. if you really need to do things manually. But we have
>> convenient service (Perestroika) which builds packages in minutes if you
>> need. Moreover, In the nearest future (by 8.0) Perestroika will be
>> available as an application independent from CI. So, what is wrong with
>> building fuel-library package? What if you want to troubleshoot nova (we
>> install it using packages)? Should we also use rsync for everything else
>> like nova, mysql, etc.?
>>
>>
> Yes, we do have a service like Perestroika to build packages for us.  That
> doesn't mean everyone else does or has access to do that today.  Setting up
> a build system is a major undertaking and making that a hard requirement to
> interact with our product may be a bit much for some customers.  In
> speaking with some support folks, there are times when files have to be
> munged to get around issues because there is no package or things are on
> fire so they can't wait for a package to become available for a fix.  We
> need to be careful not to impose limits without proper justification and
> due diligence.  We already build the fuel-library package, so there's no
> reason you couldn't try switching the rsync to install the package if it's
> available on a mirror.  I just think you're going to run into the issues I
> mentioned which need to be solved before we could just mark it done.
>
> -Alex
>
>
>
>> Vladimir Kozhukalov
>>
>> On Wed, Sep 9, 2015 at 4:39 PM, Alex Schultz 
>> wrote:
>>
>>> I agree that we shouldn't need to sync as we should be able to just
>>> update the fuel-library package. That being said, I think there might be a
>>> few issues with this method. The first issue is with plugins and how to
>>> properly handle the distribution of the plugins as they may also include
>>> puppet code that needs to be installed on the other nodes for a deployment.
>>> Currently I do not believe we install the plugin packages anywhere except
>>> the master and when they do get installed there may be some post-install
>>> actions that are only valid for the master.  Another issue is being
>>> flexible enough to allow for deployment engineers to make custom changes
>>> for a given environment.  

Re: [openstack-dev] [Fuel] Install fuel-libraryX.Y as a package on slave nodes

[Vladimir Kuklin]

Folks

I have a strong +1 for the proposal to decouple master node and slave nodes.
Here are the stregnths of this approach
1) We can always decide which particular node runs which particular set of
manifests. This will allow us to do be able to apply/roll back changes
node-by-node. This is very important from operations perspective.
2) We can decouple master and slave nodes manifests and not drag new
library version onto the master node when it is not needed. This allows us
to decrease probability of regressions
3) This makes life easier for the user - you just run 'apt-get/yum install'
instead of some difficult to digest `mco` command.

The only weakness that I see here is on mentioned by Andrey. I think we can
fix it by providing developers with clean and simple way of building
library package on the fly. This will make developers life easier enough to
work with proposed approach.

Also, we need to provide ways for better UX, like provide one button/api
call for:

1) update all manifests on particular nodes (e.g. all or only a part of
nodes of the cluster) to particular version
2)  revert all manifests back to the version which is provided by the
particular GA release
3) 

So far I would mark need for simple package-building system for developer
as a dependency for the proposed change, but I do not see any other way
than proceeding with it.



On Thu, Sep 10, 2015 at 11:50 AM, Sergii Golovatiuk <
sgolovat...@mirantis.com> wrote:

> Oleg,
>
> Alex gave a perfect example regarding support folks when they need to fix
> something really quick. It's client's choice what to patch or not. You may
> like it or not, but it's client's choice.
>
> On 10 Sep 2015, at 09:33, Oleg Gelbukh  wrote:
>
> Alex,
>
> I absolutely understand the point you are making about need for deployment
> engineers to modify things 'on the fly' in customer environment. It's makes
> things really flexible and lowers the entry barrier for sure.
>
> However, I would like to note that in my opinion this kind on 'monkey
> patching' is actually a bad practice for any environments other than dev
> ones. It immediately leads to emergence of unsupportable frankenclouds. I
> would greet any modification to the workflow that will discourage people
> from doing that.
>
> --
> Best regards,
> Oleg Gelbukh
>
> On Wed, Sep 9, 2015 at 5:56 PM, Alex Schultz 
> wrote:
>
>> Hey Vladimir,
>>
>>
>>
>>> Regarding plugins: plugins are welcome to install specific additional
>>> DEB/RPM repos on the master node, or just configure cluster to use
>>> additional onl?ne repos, where all necessary packages (including plugin
>>> specific puppet manifests) are to be available. Current granular deployment
>>> approach makes it easy to append specific pre-deployment tasks
>>> (master/slave does not matter). Correct me if I am wrong.
>>>
>>>
>> Don't get me wrong, I think it would be good to move to a fuel-library
>> distributed via package only.  I'm bringing these points up to indicate
>> that there is many other things that live in the fuel library puppet path
>> than just the fuel-library package.  The plugin example is just one place
>> that we will need to invest in further design and work to move to the
>> package only distribution.  What I don't want is some partially executed
>> work that only works for one type of deployment and creates headaches for
>> the people actually having to use fuel.  The deployment engineers and
>> customers who actually perform these actions should be asked about
>> packaging and their comfort level with this type of requirements.  I don't
>> have a complete understanding of the all the things supported today by the
>> fuel plugin system so it would be nice to get someone who is more familiar
>> to weigh in on this idea. Currently plugins are only rpms (no debs) and I
>> don't think we are building fuel-library debs at this time either.  So
>> without some work on both sides, we cannot move to just packages.
>>
>>
>>> Regarding flexibility: having several versioned directories with puppet
>>> modules on the master node, having several fuel-libraryX.Y packages
>>> installed on the master node makes things "exquisitely convoluted" rather
>>> than flexible. Like I said, it is flexible enough to use mcollective, plain
>>> rsync, etc. if you really need to do things manually. But we have
>>> convenient service (Perestroika) which builds packages in minutes if you
>>> need. Moreover, In the nearest future (by 8.0) Perestroika will be
>>> available as an application independent from CI. So, what is wrong with
>>> building fuel-library package? What if you want to troubleshoot nova (we
>>> install it using packages)? Should we also use rsync for everything else
>>> like nova, mysql, etc.?
>>>
>>>
>> Yes, we do have a service like Perestroika to build packages for us.
>> That doesn't mean everyone else does or has access to do that today.
>> Setting up a build system is a major undertaking and 

Re: [openstack-dev] [Fuel] Install fuel-libraryX.Y as a package on slave nodes

[Andrey Danin]

I disagree from the development point of view. Now I just change manifests
on Fuel node and redeploy cluster to apply that changes. With your proposal
I'll need to build a new package and add it to a repo every time I change
something.

On Tue, Sep 8, 2015 at 11:41 PM, Vladimir Kozhukalov <
vkozhuka...@mirantis.com> wrote:

> Dear colleagues,
>
> Currently, we install fuel-libraryX.Y package(s) on the master node and
> then right before starting actual deployment we rsync [1] puppet modules
> (one of installed versions) from the master node to slave nodes. Such a
> flow makes things much more complicated than they could be if we installed
> puppet modules on slave nodes as rpm/deb packages. Deployment itself is
> parameterized by repo urls (upstream + mos) and this pre-deployment task
> could be nothing more than just installing fuel-library package from mos
> repo defined for a cluster. We would not have several versions of
> fuel-library on the master node, we would not need that complicated upgrade
> stuff like we currently have for puppet modules.
>
> Please give your opinions on this.
>
>
> [1]
> https://github.com/stackforge/fuel-web/blob/master/nailgun/nailgun/orchestrator/tasks_serializer.py#L205-L218
>
> Vladimir Kozhukalov
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Andrey Danin
ada...@mirantis.com
skype: gcon.monolake
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Install fuel-libraryX.Y as a package on slave nodes

[Dmitry Pyzhov]

Vladimir,

thanks for bringing this up. It greatly correlates with the idea of
modularity. Everything related to an openstack release should be put in one
place and should be managed as a solid bundle on the master node. Package
repository is the first solution that comes to the mind and it looks pretty
good. Puppet modules, openstack.yaml and maybe even serialisers should be
stored in packages in the openstack release repository. And eventually
every other piece of our software should get rid of release-specific logic.

On Tue, Sep 8, 2015 at 11:41 PM, Vladimir Kozhukalov <
vkozhuka...@mirantis.com> wrote:

> Dear colleagues,
>
> Currently, we install fuel-libraryX.Y package(s) on the master node and
> then right before starting actual deployment we rsync [1] puppet modules
> (one of installed versions) from the master node to slave nodes. Such a
> flow makes things much more complicated than they could be if we installed
> puppet modules on slave nodes as rpm/deb packages. Deployment itself is
> parameterized by repo urls (upstream + mos) and this pre-deployment task
> could be nothing more than just installing fuel-library package from mos
> repo defined for a cluster. We would not have several versions of
> fuel-library on the master node, we would not need that complicated upgrade
> stuff like we currently have for puppet modules.
>
> Please give your opinions on this.
>
>
> [1]
> https://github.com/stackforge/fuel-web/blob/master/nailgun/nailgun/orchestrator/tasks_serializer.py#L205-L218
>
> Vladimir Kozhukalov
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Install fuel-libraryX.Y as a package on slave nodes

[Alex Schultz]

I agree that we shouldn't need to sync as we should be able to just update
the fuel-library package. That being said, I think there might be a few
issues with this method. The first issue is with plugins and how to
properly handle the distribution of the plugins as they may also include
puppet code that needs to be installed on the other nodes for a deployment.
Currently I do not believe we install the plugin packages anywhere except
the master and when they do get installed there may be some post-install
actions that are only valid for the master.  Another issue is being
flexible enough to allow for deployment engineers to make custom changes
for a given environment.  Unless we can provide an improved process to
allow for people to provide in place modifications for an environment, we
can't do away with the rsync.

If we want to go completely down the package route (and we probably
should), we need to make sure that all of the other pieces that currently
go together to make a complete fuel deployment can be updated in the same
way.

-Alex

On Wed, Sep 9, 2015 at 8:15 AM, Andrey Danin  wrote:

> I don't think juggling with repos and pull requests is easier than direct
> editing of files on Fuel node. Do we have Perestorika installed on Fuel
> node in 7.0?
>
> On Wed, Sep 9, 2015 at 3:47 PM, Vladimir Kozhukalov <
> vkozhuka...@mirantis.com> wrote:
>
>> Andrey,
>>
>> This change is going to make things even easier. Currently you don't need
>> to build fuel-library package manually, Perestroika is going to do it for
>> you. It builds necessary packages during minutes for every review request
>> and packaging ci even tests it for you. You just need to make necessary
>> changes not on master node but on your MACBOOK using your favorite editor.
>> Then you need to commit this change and send this patch on review. If you
>> want to test this patch manually, you just need to append this CR repo
>> (example is here [1]) to the list of repos you define for your cluster and
>> start deployment. Anyway, you still have rsync, mcollective and other old
>> plain tools to run deployment manually.
>>
>> [1] http://perestroika-repo-tst.infra.mirantis.net/review/CR-221719/
>>
>>
>>
>> Vladimir Kozhukalov
>>
>> On Wed, Sep 9, 2015 at 2:48 PM, Dmitry Pyzhov 
>> wrote:
>>
>>> Vladimir,
>>>
>>> thanks for bringing this up. It greatly correlates with the idea of
>>> modularity. Everything related to an openstack release should be put in one
>>> place and should be managed as a solid bundle on the master node. Package
>>> repository is the first solution that comes to the mind and it looks pretty
>>> good. Puppet modules, openstack.yaml and maybe even serialisers should be
>>> stored in packages in the openstack release repository. And eventually
>>> every other piece of our software should get rid of release-specific logic.
>>>
>>> On Tue, Sep 8, 2015 at 11:41 PM, Vladimir Kozhukalov <
>>> vkozhuka...@mirantis.com> wrote:
>>>
 Dear colleagues,

 Currently, we install fuel-libraryX.Y package(s) on the master node and
 then right before starting actual deployment we rsync [1] puppet modules
 (one of installed versions) from the master node to slave nodes. Such a
 flow makes things much more complicated than they could be if we installed
 puppet modules on slave nodes as rpm/deb packages. Deployment itself is
 parameterized by repo urls (upstream + mos) and this pre-deployment task
 could be nothing more than just installing fuel-library package from mos
 repo defined for a cluster. We would not have several versions of
 fuel-library on the master node, we would not need that complicated upgrade
 stuff like we currently have for puppet modules.

 Please give your opinions on this.


 [1]
 https://github.com/stackforge/fuel-web/blob/master/nailgun/nailgun/orchestrator/tasks_serializer.py#L205-L218

 Vladimir Kozhukalov


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>
> --
> Andrey Danin
> ada...@mirantis.com
> skype: gcon.monolake
>
> 

Re: [openstack-dev] [Fuel] Install fuel-libraryX.Y as a package on slave nodes

[Vladimir Kozhukalov]

Andrey,

This change is going to make things even easier. Currently you don't need
to build fuel-library package manually, Perestroika is going to do it for
you. It builds necessary packages during minutes for every review request
and packaging ci even tests it for you. You just need to make necessary
changes not on master node but on your MACBOOK using your favorite editor.
Then you need to commit this change and send this patch on review. If you
want to test this patch manually, you just need to append this CR repo
(example is here [1]) to the list of repos you define for your cluster and
start deployment. Anyway, you still have rsync, mcollective and other old
plain tools to run deployment manually.

[1] http://perestroika-repo-tst.infra.mirantis.net/review/CR-221719/



Vladimir Kozhukalov

On Wed, Sep 9, 2015 at 2:48 PM, Dmitry Pyzhov  wrote:

> Vladimir,
>
> thanks for bringing this up. It greatly correlates with the idea of
> modularity. Everything related to an openstack release should be put in one
> place and should be managed as a solid bundle on the master node. Package
> repository is the first solution that comes to the mind and it looks pretty
> good. Puppet modules, openstack.yaml and maybe even serialisers should be
> stored in packages in the openstack release repository. And eventually
> every other piece of our software should get rid of release-specific logic.
>
> On Tue, Sep 8, 2015 at 11:41 PM, Vladimir Kozhukalov <
> vkozhuka...@mirantis.com> wrote:
>
>> Dear colleagues,
>>
>> Currently, we install fuel-libraryX.Y package(s) on the master node and
>> then right before starting actual deployment we rsync [1] puppet modules
>> (one of installed versions) from the master node to slave nodes. Such a
>> flow makes things much more complicated than they could be if we installed
>> puppet modules on slave nodes as rpm/deb packages. Deployment itself is
>> parameterized by repo urls (upstream + mos) and this pre-deployment task
>> could be nothing more than just installing fuel-library package from mos
>> repo defined for a cluster. We would not have several versions of
>> fuel-library on the master node, we would not need that complicated upgrade
>> stuff like we currently have for puppet modules.
>>
>> Please give your opinions on this.
>>
>>
>> [1]
>> https://github.com/stackforge/fuel-web/blob/master/nailgun/nailgun/orchestrator/tasks_serializer.py#L205-L218
>>
>> Vladimir Kozhukalov
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Install fuel-libraryX.Y as a package on slave nodes

[Andrey Danin]

I don't think juggling with repos and pull requests is easier than direct
editing of files on Fuel node. Do we have Perestorika installed on Fuel
node in 7.0?

On Wed, Sep 9, 2015 at 3:47 PM, Vladimir Kozhukalov <
vkozhuka...@mirantis.com> wrote:

> Andrey,
>
> This change is going to make things even easier. Currently you don't need
> to build fuel-library package manually, Perestroika is going to do it for
> you. It builds necessary packages during minutes for every review request
> and packaging ci even tests it for you. You just need to make necessary
> changes not on master node but on your MACBOOK using your favorite editor.
> Then you need to commit this change and send this patch on review. If you
> want to test this patch manually, you just need to append this CR repo
> (example is here [1]) to the list of repos you define for your cluster and
> start deployment. Anyway, you still have rsync, mcollective and other old
> plain tools to run deployment manually.
>
> [1] http://perestroika-repo-tst.infra.mirantis.net/review/CR-221719/
>
>
>
> Vladimir Kozhukalov
>
> On Wed, Sep 9, 2015 at 2:48 PM, Dmitry Pyzhov 
> wrote:
>
>> Vladimir,
>>
>> thanks for bringing this up. It greatly correlates with the idea of
>> modularity. Everything related to an openstack release should be put in one
>> place and should be managed as a solid bundle on the master node. Package
>> repository is the first solution that comes to the mind and it looks pretty
>> good. Puppet modules, openstack.yaml and maybe even serialisers should be
>> stored in packages in the openstack release repository. And eventually
>> every other piece of our software should get rid of release-specific logic.
>>
>> On Tue, Sep 8, 2015 at 11:41 PM, Vladimir Kozhukalov <
>> vkozhuka...@mirantis.com> wrote:
>>
>>> Dear colleagues,
>>>
>>> Currently, we install fuel-libraryX.Y package(s) on the master node and
>>> then right before starting actual deployment we rsync [1] puppet modules
>>> (one of installed versions) from the master node to slave nodes. Such a
>>> flow makes things much more complicated than they could be if we installed
>>> puppet modules on slave nodes as rpm/deb packages. Deployment itself is
>>> parameterized by repo urls (upstream + mos) and this pre-deployment task
>>> could be nothing more than just installing fuel-library package from mos
>>> repo defined for a cluster. We would not have several versions of
>>> fuel-library on the master node, we would not need that complicated upgrade
>>> stuff like we currently have for puppet modules.
>>>
>>> Please give your opinions on this.
>>>
>>>
>>> [1]
>>> https://github.com/stackforge/fuel-web/blob/master/nailgun/nailgun/orchestrator/tasks_serializer.py#L205-L218
>>>
>>> Vladimir Kozhukalov
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Andrey Danin
ada...@mirantis.com
skype: gcon.monolake
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Install fuel-libraryX.Y as a package on slave nodes

[Vladimir Kozhukalov]

No, Perestroika is not available on the Fuel master node and it is not
going to be available in the future. But Perestroika is going to be
re-worked so as to make it is possible to used separately from CI. It is
gonna be a python application to make package building as easy for a
developer/user as possible. Anyway I think this argument that it is easier
to develop is not that kind of argument which can prevail when discussing
production ready delivery approach.

Vladimir Kozhukalov

On Wed, Sep 9, 2015 at 4:15 PM, Andrey Danin  wrote:

> I don't think juggling with repos and pull requests is easier than direct
> editing of files on Fuel node. Do we have Perestorika installed on Fuel
> node in 7.0?
>
> On Wed, Sep 9, 2015 at 3:47 PM, Vladimir Kozhukalov <
> vkozhuka...@mirantis.com> wrote:
>
>> Andrey,
>>
>> This change is going to make things even easier. Currently you don't need
>> to build fuel-library package manually, Perestroika is going to do it for
>> you. It builds necessary packages during minutes for every review request
>> and packaging ci even tests it for you. You just need to make necessary
>> changes not on master node but on your MACBOOK using your favorite editor.
>> Then you need to commit this change and send this patch on review. If you
>> want to test this patch manually, you just need to append this CR repo
>> (example is here [1]) to the list of repos you define for your cluster and
>> start deployment. Anyway, you still have rsync, mcollective and other old
>> plain tools to run deployment manually.
>>
>> [1] http://perestroika-repo-tst.infra.mirantis.net/review/CR-221719/
>>
>>
>>
>> Vladimir Kozhukalov
>>
>> On Wed, Sep 9, 2015 at 2:48 PM, Dmitry Pyzhov 
>> wrote:
>>
>>> Vladimir,
>>>
>>> thanks for bringing this up. It greatly correlates with the idea of
>>> modularity. Everything related to an openstack release should be put in one
>>> place and should be managed as a solid bundle on the master node. Package
>>> repository is the first solution that comes to the mind and it looks pretty
>>> good. Puppet modules, openstack.yaml and maybe even serialisers should be
>>> stored in packages in the openstack release repository. And eventually
>>> every other piece of our software should get rid of release-specific logic.
>>>
>>> On Tue, Sep 8, 2015 at 11:41 PM, Vladimir Kozhukalov <
>>> vkozhuka...@mirantis.com> wrote:
>>>
 Dear colleagues,

 Currently, we install fuel-libraryX.Y package(s) on the master node and
 then right before starting actual deployment we rsync [1] puppet modules
 (one of installed versions) from the master node to slave nodes. Such a
 flow makes things much more complicated than they could be if we installed
 puppet modules on slave nodes as rpm/deb packages. Deployment itself is
 parameterized by repo urls (upstream + mos) and this pre-deployment task
 could be nothing more than just installing fuel-library package from mos
 repo defined for a cluster. We would not have several versions of
 fuel-library on the master node, we would not need that complicated upgrade
 stuff like we currently have for puppet modules.

 Please give your opinions on this.


 [1]
 https://github.com/stackforge/fuel-web/blob/master/nailgun/nailgun/orchestrator/tasks_serializer.py#L205-L218

 Vladimir Kozhukalov


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>
> --
> Andrey Danin
> ada...@mirantis.com
> skype: gcon.monolake
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Install fuel-libraryX.Y as a package on slave nodes

[Vladimir Kozhukalov]

Alex,

Regarding plugins: plugins are welcome to install specific additional
DEB/RPM repos on the master node, or just configure cluster to use
additional onl?ne repos, where all necessary packages (including plugin
specific puppet manifests) are to be available. Current granular deployment
approach makes it easy to append specific pre-deployment tasks
(master/slave does not matter). Correct me if I am wrong.

Regarding flexibility: having several versioned directories with puppet
modules on the master node, having several fuel-libraryX.Y packages
installed on the master node makes things "exquisitely convoluted" rather
than flexible. Like I said, it is flexible enough to use mcollective, plain
rsync, etc. if you really need to do things manually. But we have
convenient service (Perestroika) which builds packages in minutes if you
need. Moreover, In the nearest future (by 8.0) Perestroika will be
available as an application independent from CI. So, what is wrong with
building fuel-library package? What if you want to troubleshoot nova (we
install it using packages)? Should we also use rsync for everything else
like nova, mysql, etc.?

Vladimir Kozhukalov

On Wed, Sep 9, 2015 at 4:39 PM, Alex Schultz  wrote:

> I agree that we shouldn't need to sync as we should be able to just update
> the fuel-library package. That being said, I think there might be a few
> issues with this method. The first issue is with plugins and how to
> properly handle the distribution of the plugins as they may also include
> puppet code that needs to be installed on the other nodes for a deployment.
> Currently I do not believe we install the plugin packages anywhere except
> the master and when they do get installed there may be some post-install
> actions that are only valid for the master.  Another issue is being
> flexible enough to allow for deployment engineers to make custom changes
> for a given environment.  Unless we can provide an improved process to
> allow for people to provide in place modifications for an environment, we
> can't do away with the rsync.
>
> If we want to go completely down the package route (and we probably
> should), we need to make sure that all of the other pieces that currently
> go together to make a complete fuel deployment can be updated in the same
> way.
>
> -Alex
>
> On Wed, Sep 9, 2015 at 8:15 AM, Andrey Danin  wrote:
>
>> I don't think juggling with repos and pull requests is easier than direct
>> editing of files on Fuel node. Do we have Perestorika installed on Fuel
>> node in 7.0?
>>
>> On Wed, Sep 9, 2015 at 3:47 PM, Vladimir Kozhukalov <
>> vkozhuka...@mirantis.com> wrote:
>>
>>> Andrey,
>>>
>>> This change is going to make things even easier. Currently you don't
>>> need to build fuel-library package manually, Perestroika is going to do it
>>> for you. It builds necessary packages during minutes for every review
>>> request and packaging ci even tests it for you. You just need to make
>>> necessary changes not on master node but on your MACBOOK using your
>>> favorite editor. Then you need to commit this change and send this patch on
>>> review. If you want to test this patch manually, you just need to append
>>> this CR repo (example is here [1]) to the list of repos you define for your
>>> cluster and start deployment. Anyway, you still have rsync, mcollective and
>>> other old plain tools to run deployment manually.
>>>
>>> [1] http://perestroika-repo-tst.infra.mirantis.net/review/CR-221719/
>>>
>>>
>>>
>>> Vladimir Kozhukalov
>>>
>>> On Wed, Sep 9, 2015 at 2:48 PM, Dmitry Pyzhov 
>>> wrote:
>>>
 Vladimir,

 thanks for bringing this up. It greatly correlates with the idea of
 modularity. Everything related to an openstack release should be put in one
 place and should be managed as a solid bundle on the master node. Package
 repository is the first solution that comes to the mind and it looks pretty
 good. Puppet modules, openstack.yaml and maybe even serialisers should be
 stored in packages in the openstack release repository. And eventually
 every other piece of our software should get rid of release-specific logic.

 On Tue, Sep 8, 2015 at 11:41 PM, Vladimir Kozhukalov <
 vkozhuka...@mirantis.com> wrote:

> Dear colleagues,
>
> Currently, we install fuel-libraryX.Y package(s) on the master node
> and then right before starting actual deployment we rsync [1] puppet
> modules (one of installed versions) from the master node to slave nodes.
> Such a flow makes things much more complicated than they could be if we
> installed puppet modules on slave nodes as rpm/deb packages. Deployment
> itself is parameterized by repo urls (upstream + mos) and this
> pre-deployment task could be nothing more than just installing 
> fuel-library
> package from mos repo defined for a cluster. We would not have several
> versions of 

Re: [openstack-dev] [Fuel] Install fuel-libraryX.Y as a package on slave nodes

[Dmitry Pyzhov]

Andrey, you have highlighted important case. I hope you agree that this
case is not a blocker for the proposal. From the developer's point of view
packages are awful and we should use raw git repos on every node. It could
make developer's life way easier. But from architecture perspective it
would be a disaster.

Rsync is just another legacy part of our architecture. We had puppet master
before. We have rsync now. Let's see what we should use in future and how
we can make it convenient for developers.

On Wed, Sep 9, 2015 at 2:47 PM, Andrey Danin  wrote:

> I disagree from the development point of view. Now I just change manifests
> on Fuel node and redeploy cluster to apply that changes. With your proposal
> I'll need to build a new package and add it to a repo every time I change
> something.
>
> On Tue, Sep 8, 2015 at 11:41 PM, Vladimir Kozhukalov <
> vkozhuka...@mirantis.com> wrote:
>
>> Dear colleagues,
>>
>> Currently, we install fuel-libraryX.Y package(s) on the master node and
>> then right before starting actual deployment we rsync [1] puppet modules
>> (one of installed versions) from the master node to slave nodes. Such a
>> flow makes things much more complicated than they could be if we installed
>> puppet modules on slave nodes as rpm/deb packages. Deployment itself is
>> parameterized by repo urls (upstream + mos) and this pre-deployment task
>> could be nothing more than just installing fuel-library package from mos
>> repo defined for a cluster. We would not have several versions of
>> fuel-library on the master node, we would not need that complicated upgrade
>> stuff like we currently have for puppet modules.
>>
>> Please give your opinions on this.
>>
>>
>> [1]
>> https://github.com/stackforge/fuel-web/blob/master/nailgun/nailgun/orchestrator/tasks_serializer.py#L205-L218
>>
>> Vladimir Kozhukalov
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>
> --
> Andrey Danin
> ada...@mirantis.com
> skype: gcon.monolake
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Install fuel-libraryX.Y as a package on slave nodes

[Alex Schultz]

Hey Vladimir,



> Regarding plugins: plugins are welcome to install specific additional
> DEB/RPM repos on the master node, or just configure cluster to use
> additional onl?ne repos, where all necessary packages (including plugin
> specific puppet manifests) are to be available. Current granular deployment
> approach makes it easy to append specific pre-deployment tasks
> (master/slave does not matter). Correct me if I am wrong.
>
>
Don't get me wrong, I think it would be good to move to a fuel-library
distributed via package only.  I'm bringing these points up to indicate
that there is many other things that live in the fuel library puppet path
than just the fuel-library package.  The plugin example is just one place
that we will need to invest in further design and work to move to the
package only distribution.  What I don't want is some partially executed
work that only works for one type of deployment and creates headaches for
the people actually having to use fuel.  The deployment engineers and
customers who actually perform these actions should be asked about
packaging and their comfort level with this type of requirements.  I don't
have a complete understanding of the all the things supported today by the
fuel plugin system so it would be nice to get someone who is more familiar
to weigh in on this idea. Currently plugins are only rpms (no debs) and I
don't think we are building fuel-library debs at this time either.  So
without some work on both sides, we cannot move to just packages.


> Regarding flexibility: having several versioned directories with puppet
> modules on the master node, having several fuel-libraryX.Y packages
> installed on the master node makes things "exquisitely convoluted" rather
> than flexible. Like I said, it is flexible enough to use mcollective, plain
> rsync, etc. if you really need to do things manually. But we have
> convenient service (Perestroika) which builds packages in minutes if you
> need. Moreover, In the nearest future (by 8.0) Perestroika will be
> available as an application independent from CI. So, what is wrong with
> building fuel-library package? What if you want to troubleshoot nova (we
> install it using packages)? Should we also use rsync for everything else
> like nova, mysql, etc.?
>
>
Yes, we do have a service like Perestroika to build packages for us.  That
doesn't mean everyone else does or has access to do that today.  Setting up
a build system is a major undertaking and making that a hard requirement to
interact with our product may be a bit much for some customers.  In
speaking with some support folks, there are times when files have to be
munged to get around issues because there is no package or things are on
fire so they can't wait for a package to become available for a fix.  We
need to be careful not to impose limits without proper justification and
due diligence.  We already build the fuel-library package, so there's no
reason you couldn't try switching the rsync to install the package if it's
available on a mirror.  I just think you're going to run into the issues I
mentioned which need to be solved before we could just mark it done.

-Alex



> Vladimir Kozhukalov
>
> On Wed, Sep 9, 2015 at 4:39 PM, Alex Schultz 
> wrote:
>
>> I agree that we shouldn't need to sync as we should be able to just
>> update the fuel-library package. That being said, I think there might be a
>> few issues with this method. The first issue is with plugins and how to
>> properly handle the distribution of the plugins as they may also include
>> puppet code that needs to be installed on the other nodes for a deployment.
>> Currently I do not believe we install the plugin packages anywhere except
>> the master and when they do get installed there may be some post-install
>> actions that are only valid for the master.  Another issue is being
>> flexible enough to allow for deployment engineers to make custom changes
>> for a given environment.  Unless we can provide an improved process to
>> allow for people to provide in place modifications for an environment, we
>> can't do away with the rsync.
>>
>> If we want to go completely down the package route (and we probably
>> should), we need to make sure that all of the other pieces that currently
>> go together to make a complete fuel deployment can be updated in the same
>> way.
>>
>> -Alex
>>
>>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Fuel] Install fuel-libraryX.Y as a package on slave nodes

[Vladimir Kozhukalov]

Dear colleagues,

Currently, we install fuel-libraryX.Y package(s) on the master node and
then right before starting actual deployment we rsync [1] puppet modules
(one of installed versions) from the master node to slave nodes. Such a
flow makes things much more complicated than they could be if we installed
puppet modules on slave nodes as rpm/deb packages. Deployment itself is
parameterized by repo urls (upstream + mos) and this pre-deployment task
could be nothing more than just installing fuel-library package from mos
repo defined for a cluster. We would not have several versions of
fuel-library on the master node, we would not need that complicated upgrade
stuff like we currently have for puppet modules.

Please give your opinions on this.


[1]
https://github.com/stackforge/fuel-web/blob/master/nailgun/nailgun/orchestrator/tasks_serializer.py#L205-L218

Vladimir Kozhukalov
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev