Hi, There are three reported failures on the same test: 002_bootstrap.add_master_storage_domain
This seems to be related to packages issues. Please note that all 3 suits failed. Basic suit failed because of ovirt-host-4.2.0-0.0 and i think all related issues came from there. upgrade suits fail for different packaging issues (will paste issues at the end of the mail but please look at lago logs for more info) ** *Link to suspected patches: This is the patch that was reported* * * **https://gerrit.ovirt.org/#/c/83403/** * * *Link to Job:* * * **http://jenkins.ovirt.org/job/ovirt-4.1_change-queue-tester/1232** * * *Link to all logs:* * * **http://jenkins.ovirt.org/job/ovirt-4.1_change-queue-tester/1232/artifact/exported-artifacts/upgrade-from-prevrelease-suit-4.1-el7/lago_logs/lago.log** **http://jenkins.ovirt.org/job/ovirt-4.1_change-queue-tester/1232/artifact/exported-artifacts/basic-suit-4.1-el7/lago_logs/lago.log** **http://jenkins.ovirt.org/job/ovirt-4.1_change-queue-tester/1232/artifact/exported-artifacts/upgrade-from-release-suit-4.1-el7/lago_logs/lago.log ** **http://jenkins.ovirt.org/job/ovirt-4.1_change-queue-tester/1232/artifact/** * * *(Relevant) error snippet from the log: * * <error> basic suit: * * * ---> Package rubygem-json.x86_64 0:1.7.7-30.el7 will be installed --> Finished Dependency Resolution You could try using --skip-broken to work around the problem You could try running: rpm -Va --nofiles --nodigest 2017-11-01 12:59:07,818::ssh.py::ssh::96::lago.ssh::DEBUG::Command 71738e12 on lago-basic-suite-4-1-host-0 errors: Error: Package: ovirt-host-4.2.0-0.0.master.20171019135233.git1921fc6.el7.centos.noarch (alocalsync) Requires: vdsm-client >= 4.20.0 Available: vdsm-client-4.19.35-2.gitc1d5a55.el7.centos.noarch (alocalsync) vdsm-client = 4.19.35-2.gitc1d5a55.el7.centos Error: Package: ovirt-host-4.2.0-0.0.master.20171019135233.git1921fc6.el7.centos.noarch (alocalsync) Requires: vdsm >= 4.20.0 Available: vdsm-4.19.35-2.gitc1d5a55.el7.centos.x86_64 (alocalsync) vdsm = 4.19.35-2.gitc1d5a55.el7.centos upgrade-from-prevrelease-suit-4.1-el7upgrade-from-prevrelease-suit-4.1-el7 2017-11-01 12:54:36,678::log_utils.py::__enter__::600::lago.ssh::DEBUG::start task:4a7a2559-9f73-4c95-a592-a6c6144843eb:Get ssh client for lago-upgrade-from-prevrelease-suite-4-1-engine: 2017-11-01 12:54:36,678::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still got 1 tries for lago-upgrade-from-prevrelease-suite-4-1-engine 2017-11-01 12:54:39,685::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket error connecting to lago-upgrade-from-prevrelease-suite-4-1-engine: [Errno None] Unable to connect to port 22 on 192.168.203.2 2017-11-01 12:54:40,686::log_utils.py::__exit__::611::lago.ssh::DEBUG::end task:4a7a2559-9f73-4c95-a592-a6c6144843eb:Get ssh client for lago-upgrade-from-prevrelease-suite-4-1-engine: 2017-11-01 12:54:40,686::ssh.py::wait_for_ssh::129::lago.ssh::DEBUG::Got exception while sshing to lago-upgrade-from-prevrelease-suite-4-1-engine: Timed out (in 4 s) trying to ssh to lago-upgrade-from-prevrelease-suite-4-1-engine 2017-11-01 12:54:41,688::log_utils.py::__enter__::600::lago.ssh::DEBUG::start task:f0d5d670-e595-4b3c-af10-5ce91d70c431:Get ssh client for lago-upgrade-from-prevrelease-suite-4-1-engine: 2017-11-01 12:54:41,688::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still got 1 tries for lago-upgrade-from-prevrelease-suite-4-1-engine 2017-11-01 12:54:41,689::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket error connecting to lago-upgrade-from-prevrelease-suite-4-1-engine: [Errno None] Unable to connect to port 22 on 192.168.203.2 2017-11-01 12:54:42,690::log_utils.py::__exit__::611::lago.ssh::DEBUG::end task:f0d5d670-e595-4b3c-af10-5ce91d70c431:Get ssh client for lago-upgrade-from-prevrelease-suite-4-1-engine: 2017-11-01 12:54:42,690::ssh.py::wait_for_ssh::129::lago.ssh::DEBUG::Got exception while sshing to lago-upgrade-from-prevrelease-suite-4-1-engine: Timed out (in 1 s) trying to ssh to lago-upgrade-from-prevrelease-suite-4-1-engine 2017-11-01 12:54:43,692::log_utils.py::__enter__::600::lago.ssh::DEBUG::start task:d151f339-0f76-4f42-869f-c08a533a3e54:Get ssh client for lago-upgrade-from-prevrelease-suite-4-1-engine: 2017-11-01 12:54:43,692::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still got 1 tries for lago-upgrade-from-prevrelease-suite-4-1-engine 2017-11-01 12:54:43,692::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket error connecting to lago-upgrade-from-prevrelease-suite-4-1-engine: [Errno None] Unable to connect to port 22 on 192.168.203.2 2017-11-01 12:54:44,694::log_utils.py::__exit__::611::lago.ssh::DEBUG::end task:d151f339-0f76-4f42-869f-c08a533a3e54:Get ssh client for lago-upgrade-from-prevrelease-suite-4-1-engine: 2017-11-01 12:54:44,694::ssh.py::wait_for_ssh::129::lago.ssh::DEBUG::Got exception while sshing to lago-upgrade-from-prevrelease-suite-4-1-engine: Timed out (in 1 s) trying to ssh to lago-upgrade-from-prevrelease-suite-4-1-engine 2017-11-01 12:54:45,695::log_utils.py::__enter__::600::lago.ssh::DEBUG::start task:8646939f-b59a-42b1-8ebc-1dec54e5d138:Get ssh client for lago-upgrade-from-prevrelease-suite-4-1-engine: 2017-11-01 12:54:45,695::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still got 1 tries for lago-upgrade-from-prevrelease-suite-4-1-engine Installing : ovirt-vmconsole-1.0.4-1.el7.centos.noarch 154/322 Failed to resolve booleanif statement at /etc/selinux/targeted/tmp/modules/400/ovirt_vmconsole/cil:588 semodule: Failed! Installing : ovirt-vmconsole-proxy-1.0.4-1.el7.centos.noarch 155/322 libsemanage.semanage_read_policydb: Could not open kernel policy /etc/selinux/targeted/active/policy.kern for reading. (No such file or directory). OSError: No such file or directory 134/141): vdsm-xmlrpc-4.19.35-2.gitc1d5a55.el7.centos.noa | 25 kB 00:00 (135/141): vdsm-yajsonrpc-4.19.35-2.gitc1d5a55.el7.centos. | 28 kB 00:00 ovirt-engine-yarn-0.19.1-2.el7.centos.x86_64: [Errno 256] No more mirrors to try. ovirt-scheduler-proxy-0.1.7-1.el7.centos.noarch: [Errno 256] No more mirrors to try. ovirt-provider-ovn-1.0-8.el7.centos.noarch: [Errno 256] No more mirrors to try. ovirt-provider-ovn-driver-1.0-8.el7.centos.noarch: [Errno 256] No more mirrors to try. ovirt-engine-setup-plugin-live-4.1.0-1.el7.centos.noarch: [Errno 256] No more mirrors to try. unboundid-ldapsdk-3.2.0-1.el7.noarch: [Errno 256] No more mirrors to try. 2017-11-01 12:59:18,385::log_utils.py::__exit__::611::lago.utils::DEBUG::end task:da936503-c329-49b1-acfc-7658379be2fc:Run command: "reposync" "--config" "/home/jenkins/workspace/ovirt-4.1_change-queue-tester/ovirt-system-tests/upgrade-from-prevrelease-suite-4.1/reposync-config.repo" "--download_path" "/var/lib/lago" "--newest-only" "--delete" "--cachedir" "/tmp/reposync_i77mEt/cache" "--repoid" "ovirt-4.1-el7": 2017-11-01 12:59:18,385::reposetup.py::sync_rpm_repository::200::ovirtlago.reposetup::INFO:: - repo: ovirt-4.1-el7: failed, re-running. 2017-11-01 12:59:18,409::reposetup.py::_fix_reposync_issues::146::ovirtlago.reposetup::DEBUG::detected package errors in reposync output in repo_path:/var/lib/lago/ovirt-4.1-el7: ovirt-engine-setup-plugin-live,ovirt-provider-ovn-1.0-8.el7.c,unboundid-ldapsdk-3.2.0-1.el7.,ovirt-engine-setup-plugin-live-4.1.0-1.el7.centos.noarch,ovirt-provider-ovn-driver-1.0-8.el7.centos.noarch,ovirt-engine-yarn-0.19.1-2.el7,ovirt-provider-ovn-1.0-8.el7.centos.noarch,ovirt-scheduler-proxy-0.1.7-1.el7.centos.noarch,ovirt-scheduler-proxy-0.1.7-1.,unboundid-ldapsdk-3.2.0-1.el7.noarch,ovirt-provider-ovn-driver-1.0-,ovirt-engine-yarn-0.19.1-2.el7.centos.x86_64 2017-11-01 12:59:18,411::reposetup.py::_fix_reposync_issues::157::ovirtlago.reposetup::INFO:: - removing conflicting RPM: /var/lib/lago/ovirt-4.1-el7/noarch/ovirt-engine-setup-plugin-live-4.1.0-1.el7.centos.noarch.rpm 2017-11-01 12:59:18,412::reposetup.py::_fix_reposync_issues::157::ovirtlago.reposetup::INFO:: - removing conflicting RPM: /var/lib/lago/ovirt-4.1-el7/noarch/ovirt-provider-ovn-1.0-8.el7.centos.noarch.rpm 2017-11-01 12:59:18,412::reposetup.py::_fix_reposync_issues::157::ovirtlago.reposetup::INFO:: - removing conflicting RPM: /var/lib/lago/ovirt-4.1-el7/noarch/ovirt-provider-ovn-driver-1.0-8.el7.centos.noarch.rpm 2017-11-01 12:59:18,412::reposetup.py::_fix_reposync_issues::157::ovirtlago.reposetup::INFO:: - removing conflicting RPM: /var/lib/lago/ovirt-4.1-el7/noarch/unboundid-ldapsdk-3.2.0-1.el7.noarch.rpm 2017-11-01 12:59:18,413::reposetup.py::_fix_reposync_issues::157::ovirtlago.reposetup::INFO:: - removing conflicting RPM: /var/lib/lago/ovirt-4.1-el7/noarch/ovirt-scheduler-proxy-0.1.7-1.el7.centos.noarch.rpm 2017-11-01 12:59:18,415::reposetup.py::_fix_reposync_issues::157::ovirtlago.reposetup::INFO:: - removing conflicting RPM: /var/lib/lago/ovirt-4.1-el7/x86_64/ovirt-engine-yarn-0.19.1-2.el7.centos.x86_64.rpm 2017-11-01 12:59:18,416::reposetup.py::_fix_reposync_issues::167::ovirtlago.reposetup::DEBUG::removed 6 conflicting packages, see https://bugzilla.redhat.com//show_bug.cgi?id=1399235 for more details. 2017-11-01 12:59:18,416::log_utils.py::__enter__::600::lago.utils::DEBUG::start task:6e42e37e-deca-4754-9009-17b063f931a2:Run command: "reposync" "--config" "/home/jenkins/workspace/ovirt-4.1_change-queue-tester/ovirt-system-tests/upgrade-from-prevrelease-suite-4.1/reposync-config.repo" "--download_path" "/var/lib/lago" "--newest-only" "--delete" "--cachedir" "/tmp/reposync_i77mEt/cache" "--repoid" "ovirt-4.1-el7": 2017-11-01 12:59:20,628::utils.py::_run_command::189::lago.utils::DEBUG::6e42e37e-deca-4754-9009-17b063f931a2: command exit with return code: 0 2017-11-01 12:59:20,628::utils.py::_run_command::192::lago.utils::DEBUG::6e42e37e-deca-4754-9009-17b063f931a2: command stdout: Package ovirt-engine-setup-plugin-live-4.1.0-1.el7.centos.noarch.rpm is not signed 2017-11-01 13:02:29,826::ssh.py::ssh::96::lago.ssh::DEBUG::Command eaba52c4 on lago-upgrade-from-prevrelease-suite-4-1-engine errors: cat: /root/multipath.txt: No such file or directory upgrade-from-release-suit-4.1-el7 virt-engine-yarn-0.19.1-2.el7.centos.x86_64: [Errno 256] No more mirrors to try. ovirt-scheduler-proxy-0.1.7-1.el7.centos.noarch: [Errno 256] No more mirrors to try. ovirt-provider-ovn-1.0-8.el7.centos.noarch: [Errno 256] No more mirrors to try. ovirt-provider-ovn-driver-1.0-8.el7.centos.noarch: [Errno 256] No more mirrors to try. ovirt-engine-setup-plugin-live-4.1.0-1.el7.centos.noarch: [Errno 256] No more mirrors to try. unboundid-ldapsdk-3.2.0-1.el7.noarch: [Errno 256] No more mirrors to try. 2017-11-01 12:53:12,198::utils.py::_run_command::194::lago.utils::DEBUG::07b8a1a2-7149-49f1-b837-cba126d81521: command stderr: BDB2053 Freeing read locks for locker 0x8: 49578/140137252079680 BDB2053 Freeing read locks for locker 0xa: 49578/140137252079680 BDB2053 Freeing read locks for locker 0xb: 49578/140137252079680 BDB2053 Freeing read locks for locker 0xc: 49578/140137252079680 2017-11-01 12:53:12,198::log_utils.py::__exit__::611::lago.utils::DEBUG::end task:07b8a1a2-7149-49f1-b837-cba126d81521:Run command: "reposync" "--config" "/home/jenkins/workspace/ovirt-4.1_change-queue-tester/ovirt-system-tests/upgrade-from-release-suite-4.1/pre-reposync-config.repo" "--download_path" "/var/lib/lago" "--newest-only" "--delete" "--cachedir" "/tmp/reposync_v99dud/cache" "--repoid" "ovirt-4.1-el7": 2017-11-01 12:53:12,199::reposetup.py::sync_rpm_repository::200::ovirtlago.reposetup::INFO:: - repo: ovirt-4.1-el7: failed, re-running. 2017-11-01 12:53:12,225::reposetup.py::_fix_reposync_issues::146::ovirtlago.reposetup::DEBUG::detected package errors in reposync output in repo_path:/var/lib/lago/ovirt-4.1-el7: ovirt-engine-setup-plugin-live,ovirt-provider-ovn-1.0-8.el7.c,unboundid-ldapsdk-3.2.0-1.el7.,ovirt-engine-setup-plugin-live-4.1.0-1.el7.centos.noarch,ovirt-provider-ovn-driver-1.0-8.el7.centos.noarch,ovirt-engine-yarn-0.19.1-2.el7,ovirt-provider-ovn-1.0-8.el7.centos.noarch,ovirt-scheduler-proxy-0.1.7-1.el7.centos.noarch,ovirt-scheduler-proxy-0.1.7-1.,unboundid-ldapsdk-3.2.0-1.el7.noarch,ovirt-provider-ovn-driver-1.0-,ovirt-engine-yarn-0.19.1-2.el7.centos.x86_64 2017-11-01 12:53:12,227::reposetup.py::_fix_reposync_issues::157::ovirtlago.reposetup::INFO:: - removing conflicting RPM: /var/lib/lago/ovirt-4.1-el7/noarch/ovirt-scheduler-proxy-0.1.7-1.el7.centos.noarch.rpm 2017-11-01 12:53:12,227::reposetup.py::_fix_reposync_issues::157::ovirtlago.reposetup::INFO:: - removing conflicting RPM: /var/lib/lago/ovirt-4.1-el7/noarch/ovirt-provider-ovn-driver-1.0-8.el7.centos.noarch.rpm 2017-11-01 12:53:12,228::reposetup.py::_fix_reposync_issues::157::ovirtlago.reposetup::INFO:: - removing conflicting RPM: /var/lib/lago/ovirt-4.1-el7/noarch/ovirt-engine-setup-plugin-live-4.1.0-1.el7.centos.noarch.rpm 2017-11-01 12:53:12,228::reposetup.py::_fix_reposync_issues::157::ovirtlago.reposetup::INFO:: - removing conflicting RPM: /var/lib/lago/ovirt-4.1-el7/noarch/ovirt-provider-ovn-1.0-8.el7.centos.noarch.rpm 2017-11-01 12:53:12,229::reposetup.py::_fix_reposync_issues::157::ovirtlago.reposetup::INFO:: - removing conflicting RPM: /var/lib/lago/ovirt-4.1-el7/noarch/unboundid-ldapsdk-3.2.0-1.el7.noarch.rpm 2017-11-01 12:53:12,231::reposetup.py::_fix_reposync_issues::157::ovirtlago.reposetup::INFO:: - removing conflicting RPM: /var/lib/lago/ovirt-4.1-el7/x86_64/ovirt-engine-yarn-0.19.1-2.el7.centos.x86_64.rpm 2017-11-01 12:53:12,232::reposetup.py::_fix_reposync_issues::167::ovirtlago.reposetup::DEBUG::removed 6 conflicting packages, see https://bugzilla.redhat.com//show_bug.cgi?id=1399235 for more details. 2017-11-01 12:53:12,232::log_utils.py::__enter__::600::lago.utils::DEBUG::start task:4c36637d-e4d7-488d-a0e5-590e4b1e28e6:Run command: "reposync" "--config" "/home/jenkins/workspace/ovirt-4.1_change-queue-tester/ovirt-system-tests/upgrade-from-release-suite-4.1/pre-reposync-config.repo" "--download_path" "/var/lib/lago" "--newest-only" "--delete" "--cachedir" "/tmp/reposync_v99dud/cache" "--repoid" "ovirt-4.1-el7": 2017-11-01 12:53:14,457::utils.py::_run_command::189::lago.utils::DEBUG::4c36637d-e4d7-488d-a0e5-590e4b1e28e6: command exit with return code: 0 2017-11-01 12:53:14,457::utils.py::_run_command::192::lago.utils::DEBUG::4c36637d-e4d7-488d-a0e5-590e4b1e28e6: command stdout: Package ovirt-engine-setup-plugin-live-4.1.0-1.el7.centos.noarch.rpm is not signed 2017-11-01 12:54:52,287::log_utils.py::__enter__::600::lago.prefix::INFO::[0m[0m 2017-11-01 12:54:52,288::log_utils.py::__enter__::600::lago.ssh::DEBUG::start task:1cfeda92-2104-4c35-8868-37f1b3e175ed:Get ssh client for lago-upgrade-from-release-suite-4-1-engine: 2017-11-01 12:54:52,288::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still got 1 tries for lago-upgrade-from-release-suite-4-1-engine 2017-11-01 12:54:55,294::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket error connecting to lago-upgrade-from-release-suite-4-1-engine: [Errno None] Unable to connect to port 22 on 192.168.200.2 2017-11-01 12:54:56,295::log_utils.py::__exit__::611::lago.ssh::DEBUG::end task:1cfeda92-2104-4c35-8868-37f1b3e175ed:Get ssh client for lago-upgrade-from-release-suite-4-1-engine: 2017-11-01 12:54:56,296::ssh.py::wait_for_ssh::129::lago.ssh::DEBUG::Got exception while sshing to lago-upgrade-from-release-suite-4-1-engine: Timed out (in 4 s) trying to ssh to lago-upgrade-from-release-suite-4-1-engine 2017-11-01 12:54:57,297::log_utils.py::__enter__::600::lago.ssh::DEBUG::start task:09223161-d627-4306-abb0-218678b6eb1f:Get ssh client for lago-upgrade-from-release-suite-4-1-engine: 2017-11-01 12:54:57,297::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still got 1 tries for lago-upgrade-from-release-suite-4-1-engine 2017-11-01 12:54:57,298::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket error connecting to lago-upgrade-from-release-suite-4-1-engine: [Errno None] Unable to connect to port 22 on 192.168.200.2 2017-11-01 12:54:58,299::log_utils.py::__exit__::611::lago.ssh::DEBUG::end task:09223161-d627-4306-abb0-218678b6eb1f:Get ssh client for lago-upgrade-from-release-suite-4-1-engine: 2017-11-01 12:54:58,299::ssh.py::wait_for_ssh::129::lago.ssh::DEBUG::Got exception while sshing to lago-upgrade-from-release-suite-4-1-engine: Timed out (in 1 s) trying to ssh to lago-upgrade-from-release-suite-4-1-engine 2017-11-01 12:54:59,300::log_utils.py::__enter__::600::lago.ssh::DEBUG::start task:0981c19b-77bf-43fd-8e2f-f5519f74dbf0:Get ssh client for lago-upgrade-from-release-suite-4-1-engine: 2017-11-01 12:54:59,301::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still got 1 tries for lago-upgrade-from-release-suite-4-1-engine 2017-11-01 12:54:59,301::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket error connecting to lago-upgrade-from-release-suite-4-1-engine: [Errno None] Unable to connect to port 22 on 192.168.200.2 2017-11-01 12:55:00,302::log_utils.py::__exit__::611::lago.ssh::DEBUG::end task:0981c19b-77bf-43fd-8e2f-f5519f74dbf0:Get ssh client for lago-upgrade-from-release-suite-4-1-engine: 2017-11-01 12:55:00,303::ssh.py::wait_for_ssh::129::lago.ssh::DEBUG::Got exception while sshing to lago-upgrade-from-release-suite-4-1-engine: Timed out (in 1 s) trying to ssh to lago-upgrade-from-release-suite-4-1-engine 2017-11-01 12:55:01,304::log_utils.py::__enter__::600::lago.ssh::DEBUG::start task:80c0243d-c589-496d-a4c0-75e62e37e17e:Get ssh client for lago-upgrade-from-release-suite-4-1-engine: 2017-11-01 12:55:01,304::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still got 1 tries for lago-upgrade-from-release-suite-4-1-engine 2017-11-01 12:55:01,471::log_utils.py::__exit__::611::lago.ssh::DEBUG::end task:80c0243d-c589-496d-a4c0-75e62e37e17e:Get ssh client for lago-upgrade-from-release-suite-4-1-engine: 2017-11-01 12:55:01,588::ssh.py::ssh::58::lago.ssh::DEBUG::Running dfa2bbf2 on lago-upgrade-from-release-suite-4-1-engine: true 2017-11-01 12:55:01,611::ssh.py::ssh::81::lago.ssh::DEBUG::Command dfa2bbf2 on lago-upgrade-from-release-suite-4-1-engine returned with 0 2017-11-01 12:55:01,611::ssh.py::wait_for_ssh::153::lago.ssh::DEBUG::Wait succeeded for ssh to lago-upgrade-from-release-suite-4-1-engine 2017-11-01 12:55:01,611::log_utils.py::__exit__::611::lago.prefix::INFO::[32mSuccess[0m (in 0:00:09) 2017-11-01 12:55:01,612::log_utils.py::__enter__::600::lago.prefix::INFO::[0m[0m 2017-11-01 12:55: * </error> * ** ** * *
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
