Re: [openstack-dev] [DevStack] A grenade for DevStack?

2014-12-15 Thread Dean Troyer
On Mon, Dec 15, 2014 at 2:03 PM, Sean Dague  wrote:
>
> On 12/15/2014 02:33 PM, Collins, Sean wrote:
> > It'd be great to somehow make a long lived dsvm node and job where
> > DevStack is continually deployed to it and restacked, to check for these
> > kinds of errors?
>

I want to be careful here to not let an expectation develop that DevStack
should be used in any long-running deployment.  Refreshing a VM or a new OS
install on bare metal should be expected, often. IMO the only bits that you
should expect to refresh quickly are the git repos.


> One of the things we don't test on the devstack side at all is that
> clean.sh takes us back down to baseline, which I think was the real
> issue here - https://review.openstack.org/#/c/141891/
>
> I would not be opposed to adding cleanup testing at the end of any
> devstack run that ensures everything is shut down correctly and cleaned
> up to a base level.
>

clean.sh makes an attempt to remove enough of OpenStack and its
dependencies to be able to change the configuration, such as switching
databases or adding/removing services, and re-run stack.sh.  I'm not a fan
of tracking installed OS package dependencies and Python packages so it can
all be undone. But the above is a bug...

dt


-- 

Dean Troyer
dtro...@gmail.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [DevStack] A grenade for DevStack?

2014-12-15 Thread Sean Dague
On 12/15/2014 02:33 PM, Collins, Sean wrote:
> Hi,
> 
> I've been bitten by a couple bugs lately on DevStack installs that have
> been long lived. One is a built by Vagrant, and the other is bare metal
> hardware.
> 
> https://bugs.launchpad.net/devstack/+bug/1395776
> 
> In this instance, a commit was made to rollback the introduction of
> MariaDB in ubuntu, but it does not appear that there was anything done
> to revert the change in existing deployments, so I had to go and fix by
> hand on the bare metal lab because I didn't want to waste a lot of time
> rebuilding the whole lab from scratch just to fix.
> 
> I also got bit by this on my Vagrant node, but I nuked and paved to fix.
> 
> https://bugs.launchpad.net/devstack/+bug/1402762
> 
> It'd be great to somehow make a long lived dsvm node and job where
> DevStack is continually deployed to it and restacked, to check for these
> kinds of errors?

One of the things we don't test on the devstack side at all is that
clean.sh takes us back down to baseline, which I think was the real
issue here - https://review.openstack.org/#/c/141891/

I would not be opposed to adding cleanup testing at the end of any
devstack run that ensures everything is shut down correctly and cleaned
up to a base level.

-Sean

-- 
Sean Dague
http://dague.net

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [DevStack] A grenade for DevStack?

2014-12-15 Thread Collins, Sean
Hi,

I've been bitten by a couple bugs lately on DevStack installs that have
been long lived. One is a built by Vagrant, and the other is bare metal
hardware.

https://bugs.launchpad.net/devstack/+bug/1395776

In this instance, a commit was made to rollback the introduction of
MariaDB in ubuntu, but it does not appear that there was anything done
to revert the change in existing deployments, so I had to go and fix by
hand on the bare metal lab because I didn't want to waste a lot of time
rebuilding the whole lab from scratch just to fix.

I also got bit by this on my Vagrant node, but I nuked and paved to fix.

https://bugs.launchpad.net/devstack/+bug/1402762

It'd be great to somehow make a long lived dsvm node and job where
DevStack is continually deployed to it and restacked, to check for these
kinds of errors?

-- 
Sean M. Collins
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev