Hello community, here is the log from the commit of package sesdev for openSUSE:Factory checked in at 2020-03-06 21:27:42 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/sesdev (Old) and /work/SRC/openSUSE:Factory/.sesdev.new.26092 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "sesdev" Fri Mar 6 21:27:42 2020 rev:4 rq:781919 version:1.1.6+1583428626.ga138314 Changes: -------- --- /work/SRC/openSUSE:Factory/sesdev/sesdev.changes 2020-02-26 15:03:52.413050773 +0100 +++ /work/SRC/openSUSE:Factory/.sesdev.new.26092/sesdev.changes 2020-03-06 21:27:45.785567300 +0100 @@ -1,0 +2,15 @@ +Thu Mar 5 17:17:12 UTC 2020 - Nathan Cutler <[email protected]> + +- Update to 1.1.6+1583428626.ga138314: + + upstream 1.1.6 release (2020-03-05) + * sesdev: add --non-interactive option to "sesdev create" (PR #125) + * Implement "sesdev qa-test" command (PR #129) + * qa: assert "ceph versions" matches "ceph --version" (PR #131) + * seslib: correct downstream container for "sesdev create {ses7,octopus}" (PR #130) + * provision.sh: remove Python 2 so it doesn't pollute the environment (PR #133) + * Rename --ceph-container-image to --image-path (PR #115) + * provision: install "command-not-found", "supportutils", etc. in test environments (PR #123) + * Return with a non-zero exit code in a failure case (PR #127) + * seslib: rename --deploy-bootstrap to --cephadm-bootstrap (PR #137) + +------------------------------------------------------------------- Old: ---- sesdev-1.1.5+1582717868.g68df753.tar.gz New: ---- sesdev-1.1.6+1583428626.ga138314.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ sesdev.spec ++++++ --- /var/tmp/diff_new_pack.CXNlfg/_old 2020-03-06 21:27:46.373567662 +0100 +++ /var/tmp/diff_new_pack.CXNlfg/_new 2020-03-06 21:27:46.377567664 +0100 @@ -20,7 +20,7 @@ %endif Name: sesdev -Version: 1.1.5+1582717868.g68df753 +Version: 1.1.6+1583428626.ga138314 Release: 1%{?dist} Summary: CLI tool to deploy and manage SES clusters License: MIT ++++++ sesdev-1.1.5+1582717868.g68df753.tar.gz -> sesdev-1.1.6+1583428626.ga138314.tar.gz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/sesdev-1.1.5+1582717868.g68df753/CHANGELOG.md new/sesdev-1.1.6+1583428626.ga138314/CHANGELOG.md --- old/sesdev-1.1.5+1582717868.g68df753/CHANGELOG.md 2020-02-26 12:51:07.824981728 +0100 +++ new/sesdev-1.1.6+1583428626.ga138314/CHANGELOG.md 2020-03-05 18:17:06.306574297 +0100 @@ -7,6 +7,23 @@ ## [Unreleased] +## [1.1.6] - 2020-03-05 + +### Added +sesdev: add --non-interactive option to "sesdev create" (PR #125) +Implement "sesdev qa-test" command (PR #129) +qa: assert "ceph versions" matches "ceph --version" (PR #131) + +### Fixed +seslib: correct downstream container for "sesdev create {ses7,octopus}" (PR #130) +provision.sh: remove Python 2 so it doesn't pollute the environment (PR #133) + +### Changed +Rename --ceph-container-image to --image-path (PR #115) +provision: install "command-not-found", "supportutils", etc. in test environments (PR #123) +Return with a non-zero exit code in a failure case (PR #127) +seslib: rename --deploy-bootstrap to --cephadm-bootstrap (PR #137) + ## [1.1.5] - 2020-02-26 ### Fixed @@ -196,7 +213,8 @@ - Minimal README with a few usage instructions. - The CHANGELOG file. -[unreleased]: https://github.com/SUSE/sesdev/compare/v1.1.5...HEAD +[unreleased]: https://github.com/SUSE/sesdev/compare/v1.1.6...HEAD +[1.1.6]: https://github.com/SUSE/sesdev/releases/tag/v1.1.6 [1.1.5]: https://github.com/SUSE/sesdev/releases/tag/v1.1.5 [1.1.4]: https://github.com/SUSE/sesdev/releases/tag/v1.1.4 [1.1.3]: https://github.com/SUSE/sesdev/releases/tag/v1.1.3 diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/sesdev-1.1.5+1582717868.g68df753/README.md new/sesdev-1.1.6+1583428626.ga138314/README.md --- old/sesdev-1.1.5+1582717868.g68df753/README.md 2020-02-26 12:51:07.828981726 +0100 +++ new/sesdev-1.1.6+1583428626.ga138314/README.md 2020-03-05 18:17:06.306574297 +0100 @@ -1,4 +1,4 @@ -# sesdev - deploy and manage SES/Ceph clusters [](https://travis-ci.org/SUSE/sesdev) +# sesdev - deploy and manage SES/Ceph clusters<br/> [](https://travis-ci.org/SUSE/sesdev) `sesdev` is a CLI tool to deploy Ceph clusters (both the upstream and SUSE downstream versions). @@ -37,6 +37,10 @@ * [Symptom](#symptom) * [Analysis](#analysis) * [Resolution](#resolution) + * [Storage pool not found: no storage pool with matching name 'default'](#storage-pool-not-found-no-storage-pool-with-matching-name-default) + * [Symptom](#symptom-1) + * [Analysis](#analysis-1) + * [Resolution](#resolution-1) ## Installation @@ -84,7 +88,8 @@ $ sudo zypper -n install vagrant vagrant-libvirt ``` -Where `<repo>` can be `openSUSE_Leap_15.1` or `openSUSE_Tumbleweed`. +Where `<repo>` can be any of the openSUSE build targets currently enabled for +the [Virtualization:vagrant/vagrant package in the openSUSE Build Service](https://build.opensuse.org/package/show/Virtualization:vagrant/vagrant). #### Install sesdev from package @@ -99,7 +104,8 @@ $ sudo zypper install sesdev ``` -Where `<repo>` can be `openSUSE_Leap_15.1`, `openSUSE_Leap_15.2` or `openSUSE_Tumbleweed`. +Where `<repo>` can be any of the openSUSE build targets currently enabled for +the [sesdev package in the openSUSE Build Service](https://build.opensuse.org/package/show/filesystems:ceph/sesdev). At this point, sesdev should be installed and ready to use: refer to the "Usage" chapter, below, for further information. @@ -123,7 +129,8 @@ dnf install sesdev ``` -Where `<distro>` can be either `Fedora_29` or `Fedora_30`. +Where `<distro>` can be any of the Fedora build targets currently enabled for +the [sesdev package in the openSUSE Build Service](https://build.opensuse.org/package/show/filesystems:ceph/sesdev). At this point, sesdev should be installed and ready to use: refer to the "Usage" chapter, below, for further information. @@ -352,3 +359,64 @@ $ # For each of the volumes associated with one of the deleted machines, do: $ sudo virsh vol-delete --pool default <THE_VOLUME> ``` + +### Storage pool not found: no storage pool with matching name 'default' + +#### Symptom + +You run `ses create` but it does nothing and gives you a traceback ending with +an error: + +``` +libvirt.libvirtError: Storage pool not found: no storage pool with matching name 'default' +``` + +#### Analysis + +For whatever reason, your libvirt deployment does not have a default pool +defined. You can verify this by doing: + +``` +$ sudo virsh pool-list +``` + +In a working deployment, it says: + +``` + Name State Autostart +------------------------------- + default active no +``` + +but in this case the "default" storage pool is missing. (One user hit this when +deploying sesdev on SLE-15-SP1.) + +#### Resolution + +The "libvirt-daemon" RPM owns a directory `/var/lib/libvirt/images` which is +intended to be associated with the default storage pool: + +``` +$ sudo rpm -qf /var/lib/libvirt/images +libvirt-daemon-5.1.0-lp151.7.6.1.x86_64 +``` + +Assuming this directory exists and is empty, you can simply create a storage +pool called "default" that points to this directory, and the issue will be +resolved: + +``` +$ sudo virsh pool-define /dev/stdin <<EOF +<pool type='dir'> + <name>default</name> + <target> + <path>/var/lib/libvirt/images</path> + </target> +</pool> +EOF +$ sudo virsh pool-start default +$ sudo virsh pool-autostart default +``` + +Credits to Federico Simoncelli for the resolution, which I took from +[his post here](https://github.com/simon3z/virt-deploy/issues/8#issuecomment-73111541) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/sesdev-1.1.5+1582717868.g68df753/qa/common/common.sh new/sesdev-1.1.6+1583428626.ga138314/qa/common/common.sh --- old/sesdev-1.1.5+1582717868.g68df753/qa/common/common.sh 2020-02-26 12:51:07.828981726 +0100 +++ new/sesdev-1.1.6+1583428626.ga138314/qa/common/common.sh 2020-03-05 18:17:06.306574297 +0100 @@ -119,11 +119,11 @@ echo } -function ceph_version_test { +function ceph_rpm_version_test { # test that ceph RPM version matches "ceph --version" # for a loose definition of "matches" echo - echo "WWWW: ceph_version_test" + echo "WWWW: ceph_rpm_version_test" set -x rpm -q ceph-common set +x @@ -141,7 +141,25 @@ set -x test "$RPM_CEPH_VERSION" = "$CEPH_CEPH_VERSION" set +x - echo "ceph_version_test: OK" + echo "ceph_rpm_version_test: OK" + echo +} + +function ceph_daemon_versions_test { + local version_host="" + local version_daemon="" + echo + echo "WWWW: ceph_daemon_versions_test" + set -x + ceph --version + ceph versions + set +x + version_host="$(_extract_ceph_version "$(ceph --version)")" + version_daemon="$(_extract_ceph_version "$(ceph versions | jq -r '.overall | keys[]')")" + set -x + test "$version_host" = "$version_daemon" + set +x + echo "ceph_daemon_versions_test: OK" echo } diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/sesdev-1.1.5+1582717868.g68df753/qa/common/helper.sh new/sesdev-1.1.6+1583428626.ga138314/qa/common/helper.sh --- old/sesdev-1.1.5+1582717868.g68df753/qa/common/helper.sh 2020-02-26 12:51:07.828981726 +0100 +++ new/sesdev-1.1.6+1583428626.ga138314/qa/common/helper.sh 2020-03-05 18:17:06.306574297 +0100 @@ -39,3 +39,12 @@ done } +function _extract_ceph_version { + # given a command that outputs a string like this: + # + # ceph version 15.1.0-1521-gcdf35413a0 (cdf35413a036bd1aa59a8c718bb177839c45cab1) octopus (rc) + # + # return just the part before the first parentheses + local full_version_string="$1" + expr match "$full_version_string" '\(ceph version [^[:space:]]\+\)' +} diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/sesdev-1.1.5+1582717868.g68df753/qa/health-ok.sh new/sesdev-1.1.6+1583428626.ga138314/qa/health-ok.sh --- old/sesdev-1.1.5+1582717868.g68df753/qa/health-ok.sh 2020-02-26 12:51:07.828981726 +0100 +++ new/sesdev-1.1.6+1583428626.ga138314/qa/health-ok.sh 2020-03-05 18:17:06.306574297 +0100 @@ -30,26 +30,28 @@ echo echo "Usage:" echo " $SCRIPTNAME [-h,--help] [--igw=X] [--mds=X] [--mgr=X]" - echo " [--mon=X] [--nfs-ganesha=X] [--rgw=X]" + echo " [--mon=X] [--nfs-ganesha=X] [--promiscuous-versions] [--rgw=X]" + echo " [--total-nodes=X]" echo echo "Options:" - echo " --help Display this usage message" - echo " --igw-nodes expected number of nodes with iSCSI Gateway" - echo " --mds-nodes expected number of nodes with MDS" - echo " --mgr-nodes expected number of nodes with MGR" - echo " --mon-nodes expected number of nodes with MON" - echo " --nfs-ganesha-nodes expected number of nodes with NFS-Ganesha" - echo " --osd-nodes expected number of nodes with OSD" - echo " --osds expected total number of OSDs in cluster" - echo " --rgw-nodes expected number of nodes with RGW" - echo + echo " --help Display this usage message" + echo " --igw-nodes expected number of nodes with iSCSI Gateway" + echo " --mds-nodes expected number of nodes with MDS" + echo " --mgr-nodes expected number of nodes with MGR" + echo " --mon-nodes expected number of nodes with MON" + echo " --nfs-ganesha-nodes expected number of nodes with NFS-Ganesha" + echo " --osd-nodes expected number of nodes with OSD" + echo " --osds expected total number of OSDs in cluster" + echo " --promiscuous-versions Do not insist that daemon versions match \"ceph --version\"" + echo " --rgw-nodes expected number of nodes with RGW" + echo " --total-nodes expected total number of nodes in cluster" exit 1 } assert_enhanced_getopt TEMP=$(getopt -o h \ ---long "help,igw-nodes:,mds-nodes:,mgr-nodes:,mon-nodes:,nfs-ganesha-nodes:,osd-nodes:,osds:,rgw-nodes:,total-nodes:" \ +--long "help,igw-nodes:,mds-nodes:,mgr-nodes:,mon-nodes:,nfs-ganesha-nodes:,osd-nodes:,osds:,promiscuous-versions,rgw-nodes:,total-nodes:" \ -n 'health-ok.sh' -- "$@") if [ $? != 0 ] ; then echo "Terminating..." >&2 ; exit 1 ; fi @@ -63,6 +65,7 @@ NFS_GANESHA_NODES="" OSD_NODES="" OSDS="" +PROMISCUOUS_VERSIONS="" RGW_NODES="" TOTAL_NODES="" while true ; do @@ -74,6 +77,7 @@ --nfs-ganesha-nodes) shift ; NFS_GANESHA_NODES="$1" ; shift ;; --osd-nodes) shift ; OSD_NODES="$1" ; shift ;; --osds) shift ; OSDS="$1" ; shift ;; + --promiscuous-versions) PROMISCUOUS_VERSIONS="$1"; shift ;; --rgw-nodes) shift ; RGW_NODES="$1" ; shift ;; --total-nodes) shift ; TOTAL_NODES="$1" ; shift ;; -h|--help) usage ;; # does not return @@ -108,7 +112,8 @@ # run tests support_cop_out_test -ceph_version_test +ceph_rpm_version_test ceph_cluster_running_test +test -z "$PROMISCUOUS_VERSIONS" && ceph_daemon_versions_test ceph_health_test number_of_nodes_actual_vs_expected_test diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/sesdev-1.1.5+1582717868.g68df753/sesdev/__init__.py new/sesdev-1.1.6+1583428626.ga138314/sesdev/__init__.py --- old/sesdev-1.1.5+1582717868.g68df753/sesdev/__init__.py 2020-02-26 12:51:07.828981726 +0100 +++ new/sesdev-1.1.6+1583428626.ga138314/sesdev/__init__.py 2020-03-05 18:17:06.306574297 +0100 @@ -18,6 +18,8 @@ except SesDevException as ex: logger.exception(ex) click.echo(str(ex)) + return 1 + return 0 def _decorator_composer(decorator_list, func): @@ -65,10 +67,10 @@ help='ceph-salt Git repo URL'), click.option('--ceph-salt-branch', type=str, default=None, help='ceph-salt Git branch'), - click.option('--ceph-container-image', type=str, default=None, - help='container image path for Ceph daemons'), - click.option('--deploy-bootstrap/--no-deploy-bootstrap', default=True, - help='Run ceph-daemon bootstrap during deployment. ' + click.option('--image-path', type=str, default=None, + help='registry path from which to download Ceph container image'), + click.option('--cephadm-bootstrap/--no-cephadm-bootstrap', default=True, + help='Run cephadm bootstrap during deployment. ' '(If false all other --deploy-* options will be disabled)'), click.option('--deploy-mons/--no-deploy-mons', default=True, help='Deploy Ceph Mons'), click.option('--deploy-mgrs/--no-deploy-mgrs', default=True, help='Deploy Ceph Mgrs'), @@ -105,7 +107,7 @@ help='Custom zypper repo URL. The repo will be added to each node.'), click.option('--repo-priority/--no-repo-priority', default=True, help="Automatically set priority on custom zypper repos"), - click.option('--qa-test/--no-qa-test', default=False, + click.option('--qa-test/--no-qa-test', 'qa_test_opt', default=False, help="Automatically run integration tests on the deployed cluster"), click.option('--scc-user', type=str, default=None, help='SCC organization username'), @@ -113,6 +115,8 @@ help='SCC organization password'), click.option('--domain', type=str, default='{}.com', help='Domain name to use'), + click.option('-n', '--non-interactive', is_flag=True, default=False, + help='Do not ask the user if they really want to'), ] return _decorator_composer(click_options, func) @@ -376,11 +380,12 @@ ram, disk_size, repo_priority, - qa_test, + qa_test_opt, vagrant_box, scc_user, scc_pass, domain, + non_interactive, deepsea_cli=None, stop_before_deepsea_stage=None, deepsea_repo=None, @@ -389,8 +394,8 @@ ceph_salt_branch=None, stop_before_ceph_salt_config=False, stop_before_ceph_salt_deploy=False, - ceph_container_image=None, - deploy_bootstrap=True, + image_path=None, + cephadm_bootstrap=True, deploy_mons=True, deploy_mgrs=True, deploy_osds=True, @@ -473,8 +478,8 @@ if repo_priority is not None: settings_dict['repo_priority'] = repo_priority - if qa_test is not None: - settings_dict['qa_test'] = qa_test + if qa_test_opt is not None: + settings_dict['qa_test_opt'] = qa_test_opt if vagrant_box: settings_dict['vagrant_box'] = vagrant_box @@ -488,6 +493,9 @@ if domain: settings_dict['domain'] = domain + if non_interactive: + settings_dict['non_interactive'] = non_interactive + if ceph_salt_repo: settings_dict['ceph_salt_git_repo'] = ceph_salt_repo @@ -500,11 +508,11 @@ if stop_before_ceph_salt_deploy: settings_dict['stop_before_ceph_salt_deploy'] = stop_before_ceph_salt_deploy - if ceph_container_image: - settings_dict['ceph_container_image'] = ceph_container_image + if image_path: + settings_dict['image_path'] = image_path - if not deploy_bootstrap: - settings_dict['ceph_salt_deploy_bootstrap'] = False + if not cephadm_bootstrap: + settings_dict['ceph_salt_cephadm_bootstrap'] = False settings_dict['ceph_salt_deploy_mons'] = False settings_dict['ceph_salt_deploy_mgrs'] = False settings_dict['ceph_salt_deploy_osds'] = False @@ -527,11 +535,19 @@ def _create_command(deployment_id, deploy, settings_dict): settings = seslib.Settings(**settings_dict) dep = seslib.Deployment.create(deployment_id, settings) + really_want_to = None click.echo("=== Creating deployment with the following configuration ===") click.echo(dep.status()) if deploy: + if getattr(settings, 'non_interactive', False): + really_want_to = True + else: + really_want_to = click.confirm( + 'Do you want to continue with the deployment?', + default=True, + ) try: - if click.confirm('Do you want to continue with the deployment?', default=True): + if really_want_to: dep.start(_print_log) click.echo("=== Deployment Finished ===") click.echo() @@ -738,6 +754,16 @@ @cli.command() @click.argument('deployment_id') +def qa_test(deployment_id): + """ + Runs QA test on an already-deployed cluster. + """ + dep = seslib.Deployment.load(deployment_id) + dep.qa_test(_print_log) + + [email protected]() [email protected]('deployment_id') @click.argument('node', required=False) def stop(deployment_id, node=None): """ @@ -817,4 +843,4 @@ if __name__ == '__main__': - sesdev_main() + sys.exit(sesdev_main()) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/sesdev-1.1.5+1582717868.g68df753/sesdev.spec new/sesdev-1.1.6+1583428626.ga138314/sesdev.spec --- old/sesdev-1.1.5+1582717868.g68df753/sesdev.spec 2020-02-26 12:51:08.152981539 +0100 +++ new/sesdev-1.1.6+1583428626.ga138314/sesdev.spec 2020-03-05 18:17:06.602574216 +0100 @@ -20,7 +20,7 @@ %endif Name: sesdev -Version: 1.1.5+1582717868.g68df753 +Version: 1.1.6+1583428626.ga138314 Release: 1%{?dist} Summary: CLI tool to deploy and manage SES clusters License: MIT diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/sesdev-1.1.5+1582717868.g68df753/seslib/__init__.py new/sesdev-1.1.6+1583428626.ga138314/seslib/__init__.py --- old/sesdev-1.1.5+1582717868.g68df753/seslib/__init__.py 2020-02-26 12:51:07.828981726 +0100 +++ new/sesdev-1.1.6+1583428626.ga138314/seslib/__init__.py 2020-03-05 18:17:06.306574297 +0100 @@ -281,6 +281,11 @@ 'help': 'The domain name for nodes', 'default': '{}.com' }, + 'non_interactive': { + 'type': bool, + 'help': 'Whether the user wants to be asked', + 'default': False + }, 'deployment_tool': { 'type': str, 'help': 'Deployment tool to deploy the Ceph cluster. Currently only deepsea is supported', @@ -316,7 +321,7 @@ 'help': 'Automatically set priority on custom zypper repos', 'default': True }, - 'qa_test': { + 'qa_test_opt': { 'type': bool, 'help': 'Automatically run integration tests on the deployed cluster', 'default': False @@ -351,34 +356,34 @@ 'help': 'Stops deployment before ceph-salt deploy', 'default': False }, - 'ceph_container_image': { + 'image_path': { 'type': str, 'help': 'Container image path for Ceph daemons', 'default': None }, - 'ceph_salt_deploy_bootstrap': { + 'ceph_salt_cephadm_bootstrap': { 'type': bool, - 'help': 'Enable deployment bootstrap (aka ceph-daemon bootstrap) in ceph-salt', + 'help': 'Tell ceph-salt to run "cephadm bootstrap" during deployment', 'default': True }, 'ceph_salt_deploy_mons': { 'type': bool, - 'help': 'Enable deployment of Ceph Mons in ceph-salt', + 'help': 'Tell ceph-salt to deploy Ceph MONs', 'default': True }, 'ceph_salt_deploy_mgrs': { 'type': bool, - 'help': 'Enable deployment of Ceph Mgrs in ceph-salt', + 'help': 'Tell ceph-salt to deploy Ceph MGRs', 'default': True }, 'ceph_salt_deploy_osds': { 'type': bool, - 'help': 'Enable deployment of Ceph OSDs in ceph-salt', + 'help': 'Tell ceph-salt to deploy Ceph OSDs', 'default': True }, 'ceph_salt_deploy': { 'type': bool, - 'help': 'Use `ceph-salt deploy` command to run ceph-salt formula', + 'help': 'Use "ceph-salt deploy" instead of applying the ceph-salt highstate directly', 'default': True }, 'caasp_deploy_ses': { @@ -639,13 +644,13 @@ if self.settings.deployment_tool is None and self.settings.version != 'caasp4': self.settings.deployment_tool = VERSION_PREFERRED_DEPLOYMENT_TOOL[self.settings.version] - if self.settings.ceph_container_image is None: + if self.settings.image_path is None: if self.settings.version == 'ses7': - self.settings.ceph_container_image = \ - 'registry.suse.de/devel/storage/7.0/cr/images/ses/7/ceph/ceph' + self.settings.image_path = \ + 'registry.suse.de/devel/storage/7.0/containers/ses/7/ceph/ceph' else: - self.settings.ceph_container_image = \ - 'docker.io/ceph/daemon-base:latest-master-devel' + self.settings.image_path = \ + 'registry.opensuse.org/filesystems/ceph/master/upstream/images/ceph/ceph' if not self.settings.libvirt_networks: self._generate_static_networks() @@ -853,7 +858,7 @@ raise VersionOSNotSupported(self.settings.version, self.settings.os) try: - if self.settings.qa_test: + if self.settings.qa_test_opt: version_repos += VERSION_QA_REPO_MAPPING[self.settings.version][self.settings.os] except KeyError: raise VersionQANotSupported(self.settings.version, self.settings.os) @@ -888,7 +893,7 @@ 'version_repos': version_repos, 'os_base_repos': os_base_repos, 'repo_priority': self.settings.repo_priority, - 'qa_test': self.settings.qa_test, + 'qa_test': self.settings.qa_test_opt, 'ganesha_nodes': self.node_counts["ganesha"], 'igw_nodes': self.node_counts["igw"], 'mds_nodes': self.node_counts["mds"], @@ -903,8 +908,8 @@ 'ceph_salt_git_branch': self.settings.ceph_salt_git_branch, 'stop_before_ceph_salt_config': self.settings.stop_before_ceph_salt_config, 'stop_before_ceph_salt_deploy': self.settings.stop_before_ceph_salt_deploy, - 'ceph_container_image': self.settings.ceph_container_image, - 'ceph_salt_deploy_bootstrap': self.settings.ceph_salt_deploy_bootstrap, + 'image_path': self.settings.image_path, + 'ceph_salt_cephadm_bootstrap': self.settings.ceph_salt_cephadm_bootstrap, 'ceph_salt_deploy_mons': self.settings.ceph_salt_deploy_mons, 'ceph_salt_deploy_mgrs': self.settings.ceph_salt_deploy_mgrs, 'ceph_salt_deploy_osds': self.settings.ceph_salt_deploy_osds, @@ -1100,11 +1105,10 @@ disk.size) dev_letter += 1 result += " - repo_priority: {}\n".format(self.settings.repo_priority) - result += " - qa_test: {}\n".format(self.settings.qa_test) + result += " - qa_test: {}\n".format(self.settings.qa_test_opt) if self.settings.version in ['octopus', 'ses7'] \ and self.settings.deployment_tool == 'orchestrator': - result += " - container_images:\n" - result += " - ceph: {}\n".format(self.settings.ceph_container_image) + result += " - image_path: {}\n".format(self.settings.image_path) if v.repos: result += " - custom_repos:\n" for repo in v.repos: @@ -1199,6 +1203,13 @@ def scp(self, recursive, source, destination): tools.run_interactive(self._scp_cmd(recursive, source, destination)) + def qa_test(self, log_handler): + tools.run_async( + ["vagrant", "provision", "--provision-with", "qa-test"], + log_handler, + self.dep_dir + ) + def _find_service_node(self, service): if service == 'grafana' and self.settings.version == 'ses5': return 'admin' diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/sesdev-1.1.5+1582717868.g68df753/seslib/templates/Vagrantfile.j2 new/sesdev-1.1.6+1583428626.ga138314/seslib/templates/Vagrantfile.j2 --- old/sesdev-1.1.5+1582717868.g68df753/seslib/templates/Vagrantfile.j2 2020-02-26 12:51:07.828981726 +0100 +++ new/sesdev-1.1.6+1583428626.ga138314/seslib/templates/Vagrantfile.j2 2020-03-05 18:17:06.306574297 +0100 @@ -20,10 +20,8 @@ {% if node == admin %} node.vm.provision "file", source: "bin/", destination: "/home/vagrant/" -{% if qa_test is defined and qa_test is sameas true %} node.vm.provision "file", source: "{{ sesdev_path_to_qa }}", destination: "/home/vagrant/sesdev-qa" {% endif %} -{% endif %} node.vm.synced_folder ".", "/vagrant", disabled: true @@ -31,4 +29,8 @@ end {% endfor %} + + config.vm.provision "qa-test", type: "shell", run: "never" do |s| + s.inline = "/home/vagrant/qa-test.sh" + end end diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/sesdev-1.1.5+1582717868.g68df753/seslib/templates/ceph-salt/ceph_salt_deployment.sh.j2 new/sesdev-1.1.6+1583428626.ga138314/seslib/templates/ceph-salt/ceph_salt_deployment.sh.j2 --- old/sesdev-1.1.5+1582717868.g68df753/seslib/templates/ceph-salt/ceph_salt_deployment.sh.j2 2020-02-26 12:51:07.832981723 +0100 +++ new/sesdev-1.1.6+1583428626.ga138314/seslib/templates/ceph-salt/ceph_salt_deployment.sh.j2 2020-03-05 18:17:06.306574297 +0100 @@ -62,12 +62,12 @@ ceph-salt config /System_Update/Packages disable ceph-salt config /System_Update/Reboot disable ceph-salt config /SSH/ generate -{% if ceph_container_image %} -ceph-salt config /Containers/Images/ceph set {{ ceph_container_image }} +{% if image_path %} +ceph-salt config /Containers/Images/ceph set {{ image_path }} {% endif %} ceph-salt config /Time_Server/Server_Hostname set {{ admin.fqdn }} ceph-salt config /Time_Server/External_Servers add 0.pt.pool.ntp.org -{% if not ceph_salt_deploy_bootstrap %} +{% if not ceph_salt_cephadm_bootstrap %} ceph-salt config /Deployment/Bootstrap disable {% endif %} {% if ceph_salt_deploy_mons %} @@ -107,7 +107,29 @@ salt -G 'ceph-salt:member' state.apply ceph-salt {% endif %} -{% if qa_test is defined and qa_test is sameas true %} -/home/vagrant/sesdev-qa/health-ok.sh --total-nodes={{ nodes|length }} --nfs-ganesha-nodes={{ ganesha_nodes }} --igw-nodes={{ igw_nodes }} --mds-nodes={{ mds_nodes }} --mgr-nodes={{ mgr_nodes }} --mon-nodes={{ mon_nodes }} --osd-nodes={{ storage_nodes }} --osds={{ total_osds }} --rgw-nodes={{ rgw_nodes }} +{% set qa_test_script = "/home/vagrant/qa-test.sh" %} +{%- set qa_test_cmd = "/home/vagrant/sesdev-qa/health-ok.sh" + ~ " --total-nodes=" ~ nodes|length + ~ " --nfs-ganesha-nodes=" ~ ganesha_nodes + ~ " --igw-nodes=" ~ igw_nodes + ~ " --mds-nodes=" ~ mds_nodes + ~ " --mgr-nodes=" ~ mgr_nodes + ~ " --mon-nodes=" ~ mon_nodes + ~ " --osd-nodes=" ~ storage_nodes + ~ " --osds=" ~ total_osds + ~ " --rgw-nodes=" ~ rgw_nodes +-%} + +cat > {{ qa_test_script }} << EOF +#!/bin/bash +set -x +if [[ $(hostname) == admin* ]] ; then + {{ qa_test_cmd }} +fi +EOF +chmod 755 {{ qa_test_script }} + +{% if qa_test is sameas true %} +{{ qa_test_cmd }} {% endif %} diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/sesdev-1.1.5+1582717868.g68df753/seslib/templates/provision.sh.j2 new/sesdev-1.1.6+1583428626.ga138314/seslib/templates/provision.sh.j2 --- old/sesdev-1.1.5+1582717868.g68df753/seslib/templates/provision.sh.j2 2020-02-26 12:51:07.832981723 +0100 +++ new/sesdev-1.1.6+1583428626.ga138314/seslib/templates/provision.sh.j2 2020-03-05 18:17:06.310574297 +0100 @@ -13,6 +13,11 @@ # remove "which" RPM because testing environments typically don't have it installed zypper -n rm which || true +# remove Python 2 so it doesn't pollute the environment +{% if os == 'leap-15.1' or os == 'leap-15.2' or os == 'sles-15-sp1' or os == 'sles-15-sp2' %} +zypper -n rm python-base || true +{% endif %} + {% if _node.public_address %} echo "{{ _node.public_address }} {{ _node.fqdn }} {{ _node.name }}" >> /etc/hosts {% endif %} @@ -33,20 +38,36 @@ zypper ar -f http://download.suse.de/ibs/SUSE:/CA/SLE_15_SP2/SUSE:CA.repo {% endif %} -{% if version == 'ses7' or version == 'ses6' or version == 'caasp4' %} +{% if version == 'ses6' or version == 'caasp4' %} zypper ar -f http://download.suse.de/ibs/SUSE:/CA/SLE_15_SP1/SUSE:CA.repo {% endif %} zypper --gpg-auto-import-keys ref +{% set basic_pkgs_to_install = [ + 'vim', + 'git-core', + 'iputils', + 'jq', + 'make', + 'iptables', + 'patch', + 'man', + 'command-not-found', + ] %} {% if version == 'ses5' %} -zypper -n install vim git-core iputils jq make iptables patch man +zypper -n install {{ basic_pkgs_to_install | join(' ') }} {% else %} -zypper -n install vim git iputils hostname jq make iptables patch man +zypper -n install {{ basic_pkgs_to_install | join(' ') }} hostname {% endif %} -{% if version == 'ses7' or version == 'caasp4' %} -zypper -n install ca-certificates-suse +{% if os.startswith("sle") %} +{% set sle_pkgs_to_install = [ + 'ca-certificates-suse', + 'supportutils', + 'supportutils-plugin-ses', + ] %} +zypper -n install {{ sle_pkgs_to_install | join(' ') }} {% endif %} {% for repo in node.repos %}
