Re: [Cloud-init-dev] [Merge] ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master
Hi. I've marked this 'merged' as I think it is now in trunk under 76d58265e34851b78e952a7f275340863c90a9f5. If you disagree, please feel free to re-open. -- https://code.launchpad.net/~wesley-wiedenmeier/cloud-init/+git/cloud-init/+merge/314496 Your team cloud-init commiters is requested to review the proposed merge of ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
[Cloud-init-dev] [Merge] ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master
The proposal to merge ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master has been updated. Status: Work in progress => Merged For more details, see: https://code.launchpad.net/~wesley-wiedenmeier/cloud-init/+git/cloud-init/+merge/314496 -- Your team cloud-init commiters is requested to review the proposed merge of ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
Re: [Cloud-init-dev] [Merge] ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master
I'm going to mark this 'work in progress'. Josh has integrated this branch and others into his merge proposal at https://code.launchpad.net/~powersj/cloud-init/+git/cloud-init/+merge/324136 . We do plan to pull that all soon. Thanks for your work, Wesley! -- https://code.launchpad.net/~wesley-wiedenmeier/cloud-init/+git/cloud-init/+merge/314496 Your team cloud-init commiters is requested to review the proposed merge of ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
[Cloud-init-dev] [Merge] ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master
The proposal to merge ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master has been updated. Status: Needs review => Work in progress For more details, see: https://code.launchpad.net/~wesley-wiedenmeier/cloud-init/+git/cloud-init/+merge/314496 -- Your team cloud-init commiters is requested to review the proposed merge of ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
Re: [Cloud-init-dev] [Merge] ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master
Review: Approve continuous-integration PASSED: Continuous integration, rev:51fa870b721f0c43f570342c7799d783c1d65a5e https://jenkins.ubuntu.com/server/job/cloud-init-ci/137/ Executed test runs: SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-amd64/137 SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-arm64/137 SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-ppc64el/137 SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-s390x/137 SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=vm-i386/137 Click here to trigger a rebuild: https://jenkins.ubuntu.com/server/job/cloud-init-ci/137/rebuild -- https://code.launchpad.net/~wesley-wiedenmeier/cloud-init/+git/cloud-init/+merge/314496 Your team cloud init development team is requested to review the proposed merge of ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
Re: [Cloud-init-dev] [Merge] ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master
Review: Approve continuous-integration PASSED: Continuous integration, rev:e63726f671ac88aabf83992e05a187a05d8c7431 https://jenkins.ubuntu.com/server/job/cloud-init-ci/133/ Executed test runs: SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-amd64/133 SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-arm64/133 SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-ppc64el/133 SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-s390x/133 SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=vm-i386/133 Click here to trigger a rebuild: https://jenkins.ubuntu.com/server/job/cloud-init-ci/133/rebuild -- https://code.launchpad.net/~wesley-wiedenmeier/cloud-init/+git/cloud-init/+merge/314496 Your team cloud init development team is requested to review the proposed merge of ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
[Cloud-init-dev] [Merge] ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master
Wesley Wiedenmeier has proposed merging
~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into
cloud-init:master.
Requested reviews:
Server Team CI bot (server-team-bot): continuous-integration
Joshua Powers (powersj)
cloud init development team (cloud-init-dev)
For more details, see:
https://code.launchpad.net/~wesley-wiedenmeier/cloud-init/+git/cloud-init/+merge/314496
Integration Testing: provide commands to run tests from current tree
- Add 'bddeb' command to build a deb from the current working tree
cleanly in a container, so deps don't have to be installed on host
- Add 'tree_collect' and 'tree_run' commands, to build deb from
current working tree and call collect or run with deb
- Fix bug in instance.pull_file()
--
Your team cloud init development team is requested to review the proposed merge
of ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into
cloud-init:master.
diff --git a/tests/cloud_tests/__init__.py b/tests/cloud_tests/__init__.py
index 099c357..7959bd9 100644
--- a/tests/cloud_tests/__init__.py
+++ b/tests/cloud_tests/__init__.py
@@ -6,6 +6,7 @@ import os
BASE_DIR = os.path.dirname(os.path.abspath(__file__))
TESTCASES_DIR = os.path.join(BASE_DIR, 'testcases')
TEST_CONF_DIR = os.path.join(BASE_DIR, 'configs')
+TREE_BASE = os.sep.join(BASE_DIR.split(os.sep)[:-2])
def _initialize_logging():
diff --git a/tests/cloud_tests/__main__.py b/tests/cloud_tests/__main__.py
index ef7d187..74f29f6 100644
--- a/tests/cloud_tests/__main__.py
+++ b/tests/cloud_tests/__main__.py
@@ -2,11 +2,9 @@
import argparse
import logging
-import shutil
import sys
-import tempfile
-from tests.cloud_tests import (args, collect, manage, verify)
+from tests.cloud_tests import args, bddeb, collect, manage, run_funcs, verify
from tests.cloud_tests import LOG
@@ -22,28 +20,6 @@ def configure_log(args):
LOG.setLevel(level)
-def run(args):
-"""
-run full test suite
-"""
-failed = 0
-args.data_dir = tempfile.mkdtemp(prefix='cloud_test_data_')
-LOG.debug('using tmpdir %s', args.data_dir)
-try:
-failed += collect.collect(args)
-failed += verify.verify(args)
-except Exception:
-failed += 1
-raise
-finally:
-# TODO: make this configurable via environ or cmdline
-if failed:
-LOG.warn('some tests failed, leaving data in %s', args.data_dir)
-else:
-shutil.rmtree(args.data_dir)
-return failed
-
-
def main():
"""
entry point for cloud test suite
@@ -80,9 +56,12 @@ def main():
# run handler
LOG.debug('running with args: %s\n', parsed)
return {
+'bddeb': bddeb.bddeb,
'collect': collect.collect,
'create': manage.create,
-'run': run,
+'run': run_funcs.run,
+'tree_collect': run_funcs.tree_collect,
+'tree_run': run_funcs.tree_run,
'verify': verify.verify,
}[parsed.subcmd](parsed)
diff --git a/tests/cloud_tests/args.py b/tests/cloud_tests/args.py
index b68cc98..d051737 100644
--- a/tests/cloud_tests/args.py
+++ b/tests/cloud_tests/args.py
@@ -3,9 +3,24 @@
import os
from tests.cloud_tests import config, util
-from tests.cloud_tests import LOG
+from tests.cloud_tests import LOG, TREE_BASE
ARG_SETS = {
+'BDDEB': (
+(('--bddeb-args',),
+ {'help': 'args to pass through to bddeb',
+ 'action': 'store', 'default': None, 'required': False}),
+(('--build-os',),
+ {'help': 'OS to use as build system (default is xenial)',
+ 'action': 'store', 'choices': config.list_enabled_distros(),
+ 'default': 'xenial', 'required': False}),
+(('--build-platform',),
+ {'help': 'platform to use for build system (default is lxd)',
+ 'action': 'store', 'choices': config.list_enabled_platforms(),
+ 'default': 'lxd', 'required': False}),
+(('--cloud-init',),
+ {'help': 'path to base of cloud-init tree', 'metavar': 'DIR',
+ 'action': 'store', 'required': False, 'default': TREE_BASE}),),
'COLLECT': (
(('-p', '--platform'),
{'help': 'platform(s) to run tests on', 'metavar': 'PLATFORM',
@@ -42,6 +57,10 @@ ARG_SETS = {
(('-d', '--data-dir'),
{'help': 'directory to store test data in',
'action': 'store', 'metavar': 'DIR', 'required': True}),),
+'OUTPUT_DEB': (
+(('--deb',),
+ {'help': 'path to write output deb to', 'metavar': 'FILE',
+ 'action': 'store', 'required': True}),),
'RESULT': (
(('-r', '--result'),
{'help': 'file to write results to',
@@ -66,10 +85,16 @@ ARG_SETS = {
}
SUBCMDS = {
+'bddeb': ('build cloud-init deb from tree',
+ ('BDDEB', 'OUTPUT_DEB', 'INTERFACE')),
'collect': ('collect test data',
('COLLECT', 'INTERFACE', 'OUTPUT', 'RESULT', 'SETUP')),
'create': ('create new test cas
Re: [Cloud-init-dev] [Merge] ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master
Review: Approve continuous-integration PASSED: Continuous integration, rev:ac847078d33be75a7b74390e13f9b3144f75f604 https://jenkins.ubuntu.com/server/job/cloud-init-ci/119/ Executed test runs: SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-amd64/119 SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-arm64/119 SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-ppc64el/119 SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-s390x/119 SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=vm-i386/119 Click here to trigger a rebuild: https://jenkins.ubuntu.com/server/job/cloud-init-ci/119/rebuild -- https://code.launchpad.net/~wesley-wiedenmeier/cloud-init/+git/cloud-init/+merge/314496 Your team cloud init development team is requested to review the proposed merge of ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
Re: [Cloud-init-dev] [Merge] ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master
This branch should be ready to merge into trunk now. There were some additional improvements to the build system that had been discussed, but it may be best to work on those later. If the build system is improved in the future, then this code can easily be updated as well. The tree_run command that this branch provides is useful enough on its own that this is worth merging in on its own, although this is a lower priority than either the main integration-testing update or the distro feature flags branch that is being worked on right now. -- https://code.launchpad.net/~wesley-wiedenmeier/cloud-init/+git/cloud-init/+merge/314496 Your team cloud init development team is requested to review the proposed merge of ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
Re: [Cloud-init-dev] [Merge] ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master
This branch is not yet ready to merge. The above comment was meant for the main merge proposal at https://code.launchpad.net/~wesley-wiedenmeier/cloud-init/+git/cloud-init/+merge/308218 The build system and the bddeb functionality for cloud-tests can be improved further. -- https://code.launchpad.net/~wesley-wiedenmeier/cloud-init/+git/cloud-init/+merge/314496 Your team cloud init development team is requested to review the proposed merge of ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
[Cloud-init-dev] [Merge] ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master
The proposal to merge ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master has been updated. Status: Needs review => Work in progress For more details, see: https://code.launchpad.net/~wesley-wiedenmeier/cloud-init/+git/cloud-init/+merge/314496 -- Your team cloud init development team is requested to review the proposed merge of ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
[Cloud-init-dev] [Merge] ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master
Wesley Wiedenmeier has proposed merging
~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into
cloud-init:master.
Requested reviews:
Joshua Powers (powersj)
cloud init development team (cloud-init-dev)
For more details, see:
https://code.launchpad.net/~wesley-wiedenmeier/cloud-init/+git/cloud-init/+merge/314496
Enable running cloud_tests on current working tree, add citest tox envs
- Add 'bddeb' command to build a deb from the current working tree
cleanly in a container, so deps don't have to be installed on host
- Add 'tree_collect' and 'tree_run' commands, to build deb from
current working tree and call collect or run with deb
- Add 'citest' tox env that uses correct pylxd to call cloud_tests
- Add 'citest_run' tox env that runs cloud_tests on current tree
--
Your team cloud init development team is requested to review the proposed merge
of ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into
cloud-init:master.
diff --git a/tests/cloud_tests/__init__.py b/tests/cloud_tests/__init__.py
index 099c357..7959bd9 100644
--- a/tests/cloud_tests/__init__.py
+++ b/tests/cloud_tests/__init__.py
@@ -6,6 +6,7 @@ import os
BASE_DIR = os.path.dirname(os.path.abspath(__file__))
TESTCASES_DIR = os.path.join(BASE_DIR, 'testcases')
TEST_CONF_DIR = os.path.join(BASE_DIR, 'configs')
+TREE_BASE = os.sep.join(BASE_DIR.split(os.sep)[:-2])
def _initialize_logging():
diff --git a/tests/cloud_tests/__main__.py b/tests/cloud_tests/__main__.py
index ef7d187..74f29f6 100644
--- a/tests/cloud_tests/__main__.py
+++ b/tests/cloud_tests/__main__.py
@@ -2,11 +2,9 @@
import argparse
import logging
-import shutil
import sys
-import tempfile
-from tests.cloud_tests import (args, collect, manage, verify)
+from tests.cloud_tests import args, bddeb, collect, manage, run_funcs, verify
from tests.cloud_tests import LOG
@@ -22,28 +20,6 @@ def configure_log(args):
LOG.setLevel(level)
-def run(args):
-"""
-run full test suite
-"""
-failed = 0
-args.data_dir = tempfile.mkdtemp(prefix='cloud_test_data_')
-LOG.debug('using tmpdir %s', args.data_dir)
-try:
-failed += collect.collect(args)
-failed += verify.verify(args)
-except Exception:
-failed += 1
-raise
-finally:
-# TODO: make this configurable via environ or cmdline
-if failed:
-LOG.warn('some tests failed, leaving data in %s', args.data_dir)
-else:
-shutil.rmtree(args.data_dir)
-return failed
-
-
def main():
"""
entry point for cloud test suite
@@ -80,9 +56,12 @@ def main():
# run handler
LOG.debug('running with args: %s\n', parsed)
return {
+'bddeb': bddeb.bddeb,
'collect': collect.collect,
'create': manage.create,
-'run': run,
+'run': run_funcs.run,
+'tree_collect': run_funcs.tree_collect,
+'tree_run': run_funcs.tree_run,
'verify': verify.verify,
}[parsed.subcmd](parsed)
diff --git a/tests/cloud_tests/args.py b/tests/cloud_tests/args.py
index b68cc98..3c334ed 100644
--- a/tests/cloud_tests/args.py
+++ b/tests/cloud_tests/args.py
@@ -3,9 +3,24 @@
import os
from tests.cloud_tests import config, util
-from tests.cloud_tests import LOG
+from tests.cloud_tests import LOG, TREE_BASE
ARG_SETS = {
+'BDDEB': (
+(('--bddeb-args',),
+ {'help': 'args to pass through to bddeb',
+ 'action': 'store', 'default': None, 'required': False}),
+(('--build-os',),
+ {'help': 'OS to use as build system (default is xenial)',
+ 'action': 'store', 'choices': config.list_enabled_distros(),
+ 'default': 'xenial', 'required': False}),
+(('--build-platform',),
+ {'help': 'platform to use for build system (default is lxd)',
+ 'action': 'store', 'choices': config.list_enabled_platforms(),
+ 'default': 'lxd', 'required': False}),
+(('--cloud-init',),
+ {'help': 'path to base of cloud-init tree', 'metavar': 'DIR',
+ 'action': 'store', 'required': False, 'default': TREE_BASE}),),
'COLLECT': (
(('-p', '--platform'),
{'help': 'platform(s) to run tests on', 'metavar': 'PLATFORM',
@@ -42,6 +57,10 @@ ARG_SETS = {
(('-d', '--data-dir'),
{'help': 'directory to store test data in',
'action': 'store', 'metavar': 'DIR', 'required': True}),),
+'OUTPUT_DEB': (
+(('--deb',),
+ {'help': 'path to write output deb to', 'metavar': 'FILE',
+ 'action': 'store', 'required': True}),),
'RESULT': (
(('-r', '--result'),
{'help': 'file to write results to',
@@ -66,10 +85,16 @@ ARG_SETS = {
}
SUBCMDS = {
+'bddeb': ('build cloud-init deb from tree',
+ ('BDDEB', 'OUTPUT_DEB', 'INTERFACE')),
'collect': ('collect test data',
('COLLECT', 'INTERFACE', 'OUTPUT', 'RESULT', 'SETUP')
Re: [Cloud-init-dev] [Merge] ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master
This branch should mostly be ready to merge in now. There is additonal work to be done on getting the cc_apt_configure issue resolved in debian jessie, and the hostnamectl issue resolved on centos70, but that can wait, as these are most likely not issues caused by the test suite. Support for package management on centos still needs to be switched to dnf, but since centos is blocked on the hostnamectl issue, that can wait as well. The cleanup to system_ready_scripts has been done, and the ubuntu release config is back to using the ubuntu daily images. Ubuntu releases xenial -> zesty have been run with a couple of test configs successfully, both installing from ppa:cloud-init-dev/daily and from --deb with a deb of 0.7.9, and trusty has been run successfully with the version of cloud-init it ships with on modules/final_message. Additional testing of this merge may be good, but the tests currently being run by jenkins should work. -- https://code.launchpad.net/~wesley-wiedenmeier/cloud-init/+git/cloud-init/+merge/314496 Your team cloud init development team is requested to review the proposed merge of ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
[Cloud-init-dev] [Merge] ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master
The proposal to merge ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master has been updated. Status: Needs review => Work in progress For more details, see: https://code.launchpad.net/~wesley-wiedenmeier/cloud-init/+git/cloud-init/+merge/314496 -- Your team cloud init development team is requested to review the proposed merge of ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
Re: [Cloud-init-dev] [Merge] ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master
Since there is additional work that can be done to improve the build system, this branch and this merge proposal will be used for development. Below is some of the discussion on bddeb for reference. > >> * creating the tarball.. > >> I think what we want to do here is: > >> a.) make-tarball on "this side" > >> b.) modify bddeb to be able to just create a debian/ dir > >> either > >> i.) have it create the orig tarball and a debian.tar.gz > >> into a provided (empty) output dir. > >> ii.) have it just create the debian.tar.gz and we create > >> the tarball example of doing that > >> http://paste.ubuntu.com/23870964/ > >> c.) move the tarball and debian dir over, extract tarball, > >> extract debian/ > >> d.) mk-build-deps --install debian/control > >> e.) dpkg-buildpackage > >> > >> doing this should mean we do not have to 'install additional > >> build deps' but rather just have mk-build-deps do the right > >> thing. > > >I had been trying to avoid modifying anything outside cloud_tests, > >but adding the --debian-tar-only option in bddeb definitely does > >simplify this. This may be a useful feature in general for debugging > >cloud-init builds that do not work or doing custom deb builds, as > >going through the template system can be inconvenient. > > >This also avoids having to go through a source repo in case the > >current version of cloud-init in tree has different deps from the > >one in the source repo. > > > Using dpkg-buildpackage instead of debuild may be quicker too. > > Yeah, it makes the build much simpler in the container. I'm tempted > to further improve bddeb to list what packages need installing even, > then we could drop the mk-build-deps and equivs ... but that is not > necessary now. The complexity in that comes really from the obnoxious > 'python3-pyflakes | pyflakes (<< 1.1.0-2)' > > in package/debian/control.in Yeah, I kinda like the idea of bddeb to handle build deps entirely. That may make things simpler for users who want to build as well since there have been lots of questions on irc about how to build packages. We should probably add more documentation on cloud-inits build system at some point as well. > >> * when building with dpkg-buildpackage or debuild, good to > >> allow setting DEB_BUILD_OPTIONS=nocheck (which will not run pep8 and > >> nosetests) > > >Nice, I didn't know that was an option. Lets us avoid the additional > >build deps step easily, since that was mostly to avoid failing because > >testing stuff wasn't there. > > There is info at https://bugs.launchpad.net/cloud-init/+bug/1652329 > on what the plan here is... ultimately, package build wont run pep8, but > would still run nosetests and useful to be able to not run them. I do like the idea of dropping pep8 in the build, since the actual source used for the builds whill have already been checked. > >> * i think the above path means we do not depend on deb-src lines. > > >The switch back to ubuntu daily will handle that, but this also > >avoids going through source repo at all which can be nice if a local > >change adds a new build dep. > > And simply use of mk-build-deps would too.. > I know i'm kind of over-doing this, but i really like a "build a deb" > thing that just can be easily used, and very ideally could even > re-use an existing container, so i dont have to go through the apt-get update > stuff there... so ultimately, i'd like something that just checks deps > and builds and installs packages (and apt-get update) if needed. I definitely like 'build a deb' as well, just to simplify the current build system. I think it makes sense to pull the bddeb part of this merge into a separate branch and continue to develop it further as well. It may be good once kvm platform is supported to add an option to build there as well for users building not on a system with lxd. -- https://code.launchpad.net/~wesley-wiedenmeier/cloud-init/+git/cloud-init/+merge/314496 Your team cloud init development team is requested to review the proposed merge of ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
Re: [Cloud-init-dev] [Merge] ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master
This merge proposal is no longer required, as everything here is included in the main one at: https://code.launchpad.net/~wesley-wiedenmeier/cloud-init/+git/cloud-init/+merge/308218 -- https://code.launchpad.net/~wesley-wiedenmeier/cloud-init/+git/cloud-init/+merge/314496 Your team cloud init development team is requested to review the proposed merge of ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
Re: [Cloud-init-dev] [Merge] ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master
Review: Approve LGTM Thanks for the changes! Ran the following successfully with no changes to timeouts: $ tox $ tox -e citest_tree_run $ tox -e citest -- tree_run -n zesty -t tests/cloud_tests/configs/modules/write_files.yaml $ python3 -m tests.cloud_tests tree_run -v -n zesty -t tests/cloud_tests/configs/modules/timezone.yaml -- https://code.launchpad.net/~wesley-wiedenmeier/cloud-init/+git/cloud-init/+merge/314496 Your team cloud init development team is requested to review the proposed merge of ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
Re: [Cloud-init-dev] [Merge] ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master
Timeouts will no longer be an issue for bddeb as dep installation is now handled after boot. I think this should be ready to merge, I've tested again with tree_collect, tree_run, and just with a normal 'run --ppa' -- https://code.launchpad.net/~wesley-wiedenmeier/cloud-init/+git/cloud-init/+merge/314496 Your team cloud init development team is requested to review the proposed merge of ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
Re: [Cloud-init-dev] [Merge] ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master
Thanks for reviewing > 1) Had to increase the timeout as 120 seconds was not enough. I bumped to 600. I'll go ahead and increase the timout a bit, or possibly add the option for the call to snapshot.launch() in bddeb to specify a longer timeout that normal, since the bddeb instance should be expected to take longer to boot than other instances. It would also work to not use cloud-init to install devscripts, and just apt-get it during the build process, that may be cleaner, and would allow finer grained logging messages. > 2) the tox target "cittest_run" actually does a "tree_run" In order to not be > confused between run versus tree_run I think this should really be > "cittest_tree_run" Good point, citest_run = tree_run may be confusing, I'll get the name switched over. > lxc list was clean afterwards as expected. I believe I only missed a ppa test. I think that is plenty of testing as the changes shouldn't have affected anything other than bddeb and adding the new commands. I'll run through with --ppa once to be sure though. > The only warning message I got other than those for pylxd's 2.2 deprecation > was: > /home/powersj/Work/repos/cloud-init-wesley/tests/cloud_tests/collect.py:48: > DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead I always forget whether its LOG.warn or LOG.warning that is depricated, got it wrong here, good catch. I'll fix here and grep through to check if its wrong anywhere else. I should have all of that done at some point tonight, should be ready for merge tomorrow -- https://code.launchpad.net/~wesley-wiedenmeier/cloud-init/+git/cloud-init/+merge/314496 Your team cloud init development team is requested to review the proposed merge of ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
Re: [Cloud-init-dev] [Merge] ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master
Review: Approve LGTM with two comments: 1) Had to increase the timeout as 120 seconds was not enough. I bumped to 600. 2) the tox target "cittest_run" actually does a "tree_run" In order to not be confused between run versus tree_run I think this should really be "cittest_tree_run" Tested by running the following on my local system with pylxd 2.1.3: # lint $ tox # runs with options $ python3 -m tests.cloud_tests run -v -n trusty $ python3 -m tests.cloud_tests run -v -n xenial $ python3 -m tests.cloud_tests run -v -n yakkety --repo 'deb http://archive.ubuntu.com/ubuntu/ yakkety main' $ python3 -m tests.cloud_tests run -v -n xenial --deb cloud-init_0.7.9-0ubuntu1_all.deb # new tox based runs + tree_run $ tox -e citest_run $ tox -e citest -- tree_run -n zesty -t tests/cloud_tests/configs/modules/write_files.yaml $ python3 -m tests.cloud_tests tree_run -v -n zesty -t tests/cloud_tests/configs/modules/timezone.yaml lxc list was clean afterwards as expected. I believe I only missed a ppa test. The only warning message I got other than those for pylxd's 2.2 deprecation was: /home/powersj/Work/repos/cloud-init-wesley/tests/cloud_tests/collect.py:48: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead -- https://code.launchpad.net/~wesley-wiedenmeier/cloud-init/+git/cloud-init/+merge/314496 Your team cloud init development team is requested to review the proposed merge of ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
[Cloud-init-dev] [Merge] ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into cloud-init:master
Wesley Wiedenmeier has proposed merging
~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into
cloud-init:master.
Requested reviews:
cloud init development team (cloud-init-dev)
For more details, see:
https://code.launchpad.net/~wesley-wiedenmeier/cloud-init/+git/cloud-init/+merge/314496
Enable running cloud_tests on current working tree, add citest tox envs
- Add 'bddeb' command to build a deb from the current working tree
cleanly in a container, so deps don't have to be installed on host
- Add 'tree_collect' and 'tree_run' commands, to build deb from
current working tree and call collect or run with deb
- Add 'citest' tox env that uses correct pylxd to call cloud_tests
- Add 'citest_run' tox env that runs cloud_tests on current tree
--
Your team cloud init development team is requested to review the proposed merge
of ~wesley-wiedenmeier/cloud-init:integration-testing-invocation-cleanup into
cloud-init:master.
diff --git a/tests/cloud_tests/__init__.py b/tests/cloud_tests/__init__.py
index 099c357..7959bd9 100644
--- a/tests/cloud_tests/__init__.py
+++ b/tests/cloud_tests/__init__.py
@@ -6,6 +6,7 @@ import os
BASE_DIR = os.path.dirname(os.path.abspath(__file__))
TESTCASES_DIR = os.path.join(BASE_DIR, 'testcases')
TEST_CONF_DIR = os.path.join(BASE_DIR, 'configs')
+TREE_BASE = os.sep.join(BASE_DIR.split(os.sep)[:-2])
def _initialize_logging():
diff --git a/tests/cloud_tests/__main__.py b/tests/cloud_tests/__main__.py
index ef7d187..74f29f6 100644
--- a/tests/cloud_tests/__main__.py
+++ b/tests/cloud_tests/__main__.py
@@ -2,11 +2,9 @@
import argparse
import logging
-import shutil
import sys
-import tempfile
-from tests.cloud_tests import (args, collect, manage, verify)
+from tests.cloud_tests import args, bddeb, collect, manage, run_funcs, verify
from tests.cloud_tests import LOG
@@ -22,28 +20,6 @@ def configure_log(args):
LOG.setLevel(level)
-def run(args):
-"""
-run full test suite
-"""
-failed = 0
-args.data_dir = tempfile.mkdtemp(prefix='cloud_test_data_')
-LOG.debug('using tmpdir %s', args.data_dir)
-try:
-failed += collect.collect(args)
-failed += verify.verify(args)
-except Exception:
-failed += 1
-raise
-finally:
-# TODO: make this configurable via environ or cmdline
-if failed:
-LOG.warn('some tests failed, leaving data in %s', args.data_dir)
-else:
-shutil.rmtree(args.data_dir)
-return failed
-
-
def main():
"""
entry point for cloud test suite
@@ -80,9 +56,12 @@ def main():
# run handler
LOG.debug('running with args: %s\n', parsed)
return {
+'bddeb': bddeb.bddeb,
'collect': collect.collect,
'create': manage.create,
-'run': run,
+'run': run_funcs.run,
+'tree_collect': run_funcs.tree_collect,
+'tree_run': run_funcs.tree_run,
'verify': verify.verify,
}[parsed.subcmd](parsed)
diff --git a/tests/cloud_tests/args.py b/tests/cloud_tests/args.py
index b68cc98..d051737 100644
--- a/tests/cloud_tests/args.py
+++ b/tests/cloud_tests/args.py
@@ -3,9 +3,24 @@
import os
from tests.cloud_tests import config, util
-from tests.cloud_tests import LOG
+from tests.cloud_tests import LOG, TREE_BASE
ARG_SETS = {
+'BDDEB': (
+(('--bddeb-args',),
+ {'help': 'args to pass through to bddeb',
+ 'action': 'store', 'default': None, 'required': False}),
+(('--build-os',),
+ {'help': 'OS to use as build system (default is xenial)',
+ 'action': 'store', 'choices': config.list_enabled_distros(),
+ 'default': 'xenial', 'required': False}),
+(('--build-platform',),
+ {'help': 'platform to use for build system (default is lxd)',
+ 'action': 'store', 'choices': config.list_enabled_platforms(),
+ 'default': 'lxd', 'required': False}),
+(('--cloud-init',),
+ {'help': 'path to base of cloud-init tree', 'metavar': 'DIR',
+ 'action': 'store', 'required': False, 'default': TREE_BASE}),),
'COLLECT': (
(('-p', '--platform'),
{'help': 'platform(s) to run tests on', 'metavar': 'PLATFORM',
@@ -42,6 +57,10 @@ ARG_SETS = {
(('-d', '--data-dir'),
{'help': 'directory to store test data in',
'action': 'store', 'metavar': 'DIR', 'required': True}),),
+'OUTPUT_DEB': (
+(('--deb',),
+ {'help': 'path to write output deb to', 'metavar': 'FILE',
+ 'action': 'store', 'required': True}),),
'RESULT': (
(('-r', '--result'),
{'help': 'file to write results to',
@@ -66,10 +85,16 @@ ARG_SETS = {
}
SUBCMDS = {
+'bddeb': ('build cloud-init deb from tree',
+ ('BDDEB', 'OUTPUT_DEB', 'INTERFACE')),
'collect': ('collect test data',
('COLLECT', 'INTERFACE', 'OUTPUT', 'RESULT', 'SETUP')),
'create': ('create
