The revert was merged. On Thu, Jan 25, 2018 at 12:32 PM, Daniel Belenky <[email protected]> wrote:
> Have you tried running OST with rpms from the suspected patch to reproduce? > I meant reproducing it manually, not though OST. > On Thu, Jan 25, 2018 at 12:24 PM Edward Haas <[email protected]> wrote: > >> We have two options, a revert or a fix: >> Revert: https://gerrit.ovirt.org/#/c/86789/ >> Fix: https://gerrit.ovirt.org/#/c/86785/ >> >> We are not sure about the fix because we cannot reproduce the problem >> manually. >> >> >> On Thu, Jan 25, 2018 at 10:45 AM, Eyal Edri <[email protected]> wrote: >> >>> Once you have RPMs, you can run the upgrade suite from the manual job. >>> >>> On Thu, Jan 25, 2018 at 10:43 AM, Edward Haas <[email protected]> wrote: >>> >>>> Can we test if this one fixes this problem? >>>> https://gerrit.ovirt.org/#/c/86781 >>>> >>>> On Thu, Jan 25, 2018 at 10:00 AM, Eyal Edri <[email protected]> wrote: >>>> >>>>> Indeed, the patch looks relevant, >>>>> Dan, can we revert it or send a fix ASAP to avoid building up a large >>>>> queue? >>>>> >>>>> On Thu, Jan 25, 2018 at 9:29 AM, Daniel Belenky <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> We failed to setup host in OST upgrade from 4.1 to master suite. >>>>>> Please note that the upgrade suite installs 4.1 engine, then upgrades >>>>>> it to master and then tries to set up a host. >>>>>> >>>>>> *Links:* >>>>>> >>>>>> 1. Link to failed job >>>>>> >>>>>> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5093/artifact/exported-artifacts/upgrade-from-release-suit-master-el7/test_logs/upgrade-from-release-suite-master/post-002_bootstrap.py/> >>>>>> 2. Suspected patch: Gerrit 86474/33 >>>>>> <https://gerrit.ovirt.org/#/c/86474/33> >>>>>> >>>>>> *Error snippet from engine.log (engine):* >>>>>> >>>>>> 2018-01-24 15:13:20,257-05 ERROR >>>>>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] >>>>>> (VdsDeploy) [34609a2f] EVENT_ID: VDS_INSTALL_IN_PROGRESS_ERROR(511), An >>>>>> error has occurred during installation of Host >>>>>> lago-upgrade-from-release-suite-master-host0: Failed to execute stage >>>>>> 'Closing up': Failed to start service 'vdsmd'. >>>>>> 2018-01-24 15:13:20,301-05 INFO >>>>>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] >>>>>> (VdsDeploy) [34609a2f] EVENT_ID: VDS_INSTALL_IN_PROGRESS(509), >>>>>> Installing Host lago-upgrade-from-release-suite-master-host0. Stage: >>>>>> Clean up. >>>>>> 2018-01-24 15:13:20,304-05 INFO >>>>>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] >>>>>> (VdsDeploy) [34609a2f] EVENT_ID: VDS_INSTALL_IN_PROGRESS(509), >>>>>> Installing Host lago-upgrade-from-release-suite-master-host0. Stage: >>>>>> Pre-termination. >>>>>> 2018-01-24 15:13:20,332-05 INFO >>>>>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] >>>>>> (VdsDeploy) [34609a2f] EVENT_ID: VDS_INSTALL_IN_PROGRESS(509), >>>>>> Installing Host lago-upgrade-from-release-suite-master-host0. Retrieving >>>>>> installation logs to: >>>>>> '/var/log/ovirt-engine/host-deploy/ovirt-host-deploy-20180124151320-lago-upgrade-from-release-suite-master-host0-34609a2f.log'. >>>>>> 2018-01-24 15:13:29,227-05 INFO >>>>>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] >>>>>> (VdsDeploy) [34609a2f] EVENT_ID: VDS_INSTALL_IN_PROGRESS(509), >>>>>> Installing Host lago-upgrade-from-release-suite-master-host0. Stage: >>>>>> Termination. >>>>>> 2018-01-24 15:13:29,321-05 ERROR >>>>>> [org.ovirt.engine.core.uutils.ssh.SSHDialog] >>>>>> (EE-ManagedThreadFactory-engine-Thread-1) [34609a2f] SSH error running >>>>>> command root@lago-upgrade-from-release-suite-master-host0:'umask 0077; >>>>>> MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -d -t ovirt-XXXXXXXXXX)"; trap >>>>>> "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > >>>>>> /dev/null 2>&1" 0; tar --warning=no-timestamp -C "${MYTMP}" -x && >>>>>> "${MYTMP}"/ovirt-host-deploy DIALOG/dialect=str:machine >>>>>> DIALOG/customization=bool:True': IOException: Command returned failure >>>>>> code 1 during SSH session >>>>>> 'root@lago-upgrade-from-release-suite-master-host0' >>>>>> 2018-01-24 15:13:29,322-05 ERROR >>>>>> [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] >>>>>> (EE-ManagedThreadFactory-engine-Thread-1) [34609a2f] Error during host >>>>>> lago-upgrade-from-release-suite-master-host0 install >>>>>> 2018-01-24 15:13:29,324-05 ERROR >>>>>> [org.ovirt.engine.core.bll.hostdeploy.InstallVdsInternalCommand] >>>>>> (EE-ManagedThreadFactory-engine-Thread-1) [34609a2f] Host installation >>>>>> failed for host '4d681c3b-e8db-4a71-b5e3-0db096e3ae9c', >>>>>> 'lago-upgrade-from-release-suite-master-host0': Command returned failure >>>>>> code 1 during SSH session >>>>>> 'root@lago-upgrade-from-release-suite-master-host0' >>>>>> 2018-01-24 15:13:29,330-05 INFO >>>>>> [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] >>>>>> (EE-ManagedThreadFactory-engine-Thread-1) [34609a2f] START, >>>>>> SetVdsStatusVDSCommand(HostName = >>>>>> lago-upgrade-from-release-suite-master-host0, >>>>>> SetVdsStatusVDSCommandParameters:{hostId='4d681c3b-e8db-4a71-b5e3-0db096e3ae9c', >>>>>> status='InstallFailed', nonOperationalReason='NONE', >>>>>> stopSpmFailureLogged='false', maintenanceReason='null'}), log id: >>>>>> 5e6c4a3e >>>>>> 2018-01-24 15:13:29,339-05 INFO >>>>>> [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] >>>>>> (EE-ManagedThreadFactory-engine-Thread-1) [34609a2f] FINISH, >>>>>> SetVdsStatusVDSCommand, log id: 5e6c4a3e >>>>>> 2018-01-24 15:13:29,346-05 ERROR >>>>>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] >>>>>> (EE-ManagedThreadFactory-engine-Thread-1) [34609a2f] EVENT_ID: >>>>>> VDS_INSTALL_FAILED(505), Host >>>>>> lago-upgrade-from-release-suite-master-host0 installation failed. >>>>>> Command returned failure code 1 during SSH session >>>>>> 'root@lago-upgrade-from-release-suite-master-host0'. >>>>>> >>>>>> *Error snippet from /var/log/messages (host0):* >>>>>> >>>>>> 15:13:19 host0 NetworkManager[580]: <info> (bondscan-Ncw7DP): new Bond >>>>>> device (carrier: OFF, driver: 'bonding', ifindex: 4) >>>>>> 15:13:19 host0 vdsm-tool: Traceback (most recent call last): >>>>>> 15:13:19 host0 vdsm-tool: File "/usr/bin/vdsm-tool", line 219, in main >>>>>> 15:13:19 host0 vdsm-tool: return tool_command[cmd]["command"](*args) >>>>>> 15:13:19 host0 vdsm-tool: File >>>>>> "/usr/lib/python2.7/site-packages/vdsm/tool/network.py", line 97, in >>>>>> dump_bonding_options >>>>>> 15:13:19 host0 vdsm-tool: sysfs_options_mapper.dump_bonding_options() >>>>>> 15:13:19 host0 vdsm-tool: File >>>>>> "/usr/lib/python2.7/site-packages/vdsm/network/link/bond/sysfs_options_mapper.py", >>>>>> line 48, in dump_bonding_options >>>>>> 15:13:19 host0 vdsm-tool: jdump(_get_default_bonding_options(), f) >>>>>> 15:13:19 host0 vdsm-tool: File >>>>>> "/usr/lib/python2.7/site-packages/vdsm/network/link/bond/sysfs_options_mapper.py", >>>>>> line 60, in _get_default_bonding_options >>>>>> 15:13:19 host0 vdsm-tool: with _bond_device(bond_name): >>>>>> 15:13:19 host0 vdsm-tool: File "/usr/lib64/python2.7/contextlib.py", >>>>>> line 17, in __enter__ >>>>>> 15:13:19 host0 vdsm-tool: return self.gen.next() >>>>>> 15:13:19 host0 vdsm-tool: File >>>>>> "/usr/lib/python2.7/site-packages/vdsm/network/link/bond/sysfs_options_mapper.py", >>>>>> line 102, in _bond_device >>>>>> 15:13:19 host0 vdsm-tool: _unmanage_nm_device(bond_name) >>>>>> 15:13:19 host0 vdsm-tool: File >>>>>> "/usr/lib/python2.7/site-packages/vdsm/network/link/bond/sysfs_options_mapper.py", >>>>>> line 116, in _unmanage_nm_device >>>>>> 15:13:19 host0 vdsm-tool: dev.managed = False >>>>>> 15:13:19 host0 vdsm-tool: File >>>>>> "/usr/lib/python2.7/site-packages/vdsm/network/nm/networkmanager.py", >>>>>> line 90, in managed >>>>>> 15:13:19 host0 vdsm-tool: self._device.managed = value >>>>>> 15:13:19 host0 vdsm-tool: File >>>>>> "/usr/lib/python2.7/site-packages/vdsm/network/nm/nmdbus/device.py", >>>>>> line 81, in managed >>>>>> 15:13:19 host0 vdsm-tool: return self._set_property('Managed', value) >>>>>> 15:13:19 host0 vdsm-tool: File >>>>>> "/usr/lib/python2.7/site-packages/vdsm/network/nm/nmdbus/device.py", >>>>>> line 88, in _set_property >>>>>> 15:13:19 host0 vdsm-tool: self.IF_NAME, property_name, property_value) >>>>>> 15:13:19 host0 vdsm-tool: File >>>>>> "/usr/lib64/python2.7/site-packages/dbus/proxies.py", line 70, in >>>>>> __call__ >>>>>> 15:13:19 host0 vdsm-tool: return self._proxy_method(*args, **keywords) >>>>>> 15:13:19 host0 vdsm-tool: File >>>>>> "/usr/lib64/python2.7/site-packages/dbus/proxies.py", line 145, in >>>>>> __call__ >>>>>> 15:13:19 host0 vdsm-tool: **keywords) >>>>>> 15:13:19 host0 vdsm-tool: File >>>>>> "/usr/lib64/python2.7/site-packages/dbus/connection.py", line 651, in >>>>>> call_blocking >>>>>> 15:13:19 host0 vdsm-tool: message, timeout) >>>>>> 15:13:19 host0 vdsm-tool: DBusException: >>>>>> org.freedesktop.DBus.Error.AccessDenied: Property "Managed" of interface >>>>>> "org.freedesktop.NetworkManager.Device" is not settable >>>>>> >>>>>> >>>>>> Thanks, >>>>>> -- >>>>>> >>>>>> DANIEL BELENKY >>>>>> >>>>>> RHV DEVOPS >>>>>> >>>>>> _______________________________________________ >>>>>> Devel mailing list >>>>>> [email protected] >>>>>> http://lists.ovirt.org/mailman/listinfo/devel >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Eyal edri >>>>> >>>>> >>>>> MANAGER >>>>> >>>>> RHV DevOps >>>>> >>>>> EMEA VIRTUALIZATION R&D >>>>> >>>>> >>>>> Red Hat EMEA <https://www.redhat.com/> >>>>> <https://red.ht/sig> TRIED. TESTED. TRUSTED. >>>>> <https://redhat.com/trusted> >>>>> phone: +972-9-7692018 <+972%209-769-2018> >>>>> irc: eedri (on #tlv #rhev-dev #rhev-integ) >>>>> >>>> >>>> >>> >>> >>> -- >>> >>> Eyal edri >>> >>> >>> MANAGER >>> >>> RHV DevOps >>> >>> EMEA VIRTUALIZATION R&D >>> >>> >>> Red Hat EMEA <https://www.redhat.com/> >>> <https://red.ht/sig> TRIED. TESTED. TRUSTED. >>> <https://redhat.com/trusted> >>> phone: +972-9-7692018 <+972%209-769-2018> >>> irc: eedri (on #tlv #rhev-dev #rhev-integ) >>> >> >> -- > Daniel Belenky > RHV DevOps >
_______________________________________________ Devel mailing list [email protected] http://lists.ovirt.org/mailman/listinfo/devel
