On Fri, Sep 14, 2018 at 5:15 PM, Dafna Ron <[email protected]> wrote: > didi and Sandro are both not working today. >
So we will need to wait until Sunday/Monday, because I don't think anyone else from the team really understand otopi internals. > > if they are not back by Friday, can we try to find the patch that caused > this and revert? > > CQ reported this as the cause: https://gerrit.ovirt.org/#/c/94081/3 > This is definitely not related, it has nothing to do with adding host > > But it did not seem related to this issue, however, it was merged 11 days > ago and we could have missed this due to several regressions during these > 10 days. > > > On Fri, Sep 14, 2018 at 4:06 PM, Martin Perina <[email protected]> wrote: > >> >> >> On Fri, Sep 14, 2018 at 4:51 PM, Ravi Shankar Nori <[email protected]> >> wrote: >> >>> I see the same errors on my dev env. From the logs attached by Andrej >>> the response received by otopi has a bunch of null chars before the actual >>> response CONFIRM DEPLOY_PROCEED=yes >>> >>> >>> >>> 2018-09-14 15:49:23,018+0200 DEBUG otopi.plugins.otopi.dialog.machine >>> dialog.__logString:204 DIALOG:SEND ### Response is CONFIRM >>> DEPLOY_PROCEED=yes|no or ABORT DEPLOY_PROCEED >>> >>> ^@^@^@^@^@^@^@^@^@CONFIRM DEPLOY_PROCEED=yes >>> >> >> Didi/Sandro, could you please take a look? Below error seems like some >> issue in otopi, where an error is raised when handling binary input: >> >> >> 2018-09-14 15:49:23,032+0200 DEBUG otopi.context >> context._executeMethod:143 method exception >> >> >> Traceback (most recent call last): >> >> >> >> File "/usr/lib/python2.7/site-packages/otopi/context.py", line 133, in >> _executeMethod >> >> method['method']() >> >> >> >> File "/tmp/ovirt-O6CfS4aUHI/otopi-plugins/ovirt-host-deploy/core/misc.py", >> line 87, in _confirm >> >> prompt=True, >> >> >> >> File "/tmp/ovirt-O6CfS4aUHI/otopi-plugins/otopi/dialog/machine.py", >> line 478, in confirm >> >> code=opcode, >> >> >> >>> On Fri, Sep 14, 2018 at 10:44 AM, Dafna Ron <[email protected]> wrote: >>> >>>> if you run it with mock you would remove any environmental conditions >>>> that can effect the outcome so I recommend using mock >>>> >>>> >>>> On Fri, Sep 14, 2018 at 3:32 PM, Martin Perina <[email protected]> >>>> wrote: >>>> >>>>> >>>>> >>>>> On Fri, Sep 14, 2018 at 3:49 PM, Dafna Ron <[email protected]> wrote: >>>>> >>>>>> did you use mock to reproduce? >>>>>> >>>>> >>>>> No, just run_suite under myself >>>>> >>>>>> >>>>>> On Fri, Sep 14, 2018 at 2:39 PM, Martin Perina <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> the problem is that we haven't fetched the temporary host-deploy log >>>>>>> from /tmp directory, so we don't know which string that host-deploy >>>>>>> process >>>>>>> sent to engine is causing that issue. I tried to reproduce on my local >>>>>>> machine, but I was unable to reproduce it, 002_bootstrap phase finished >>>>>>> successfully (other phases are still running). >>>>>>> >>>>>>> So if anyone is able to reproduce, please try to fetch host-deploy >>>>>>> log from /tmp directory after the error is raised and share it. >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> Martin >>>>>>> >>>>>>> >>>>>>> On Fri, Sep 14, 2018 at 1:52 PM, Dafna Ron <[email protected]> wrote: >>>>>>> >>>>>>>> Full logs can be found here: >>>>>>>> >>>>>>>> https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovi >>>>>>>> rt-master_change-queue-tester/10307/artifact/upgrade-from-re >>>>>>>> lease-suite.el7.x86_64/test_logs/upgrade-from-release-suite- >>>>>>>> master/post-002_bootstrap.py/ >>>>>>>> >>>>>>>> On Fri, Sep 14, 2018 at 12:48 PM, Dafna Ron <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> The previous regression was resolved and we now have a new >>>>>>>>> regression. >>>>>>>>> >>>>>>>>> I don't think that the reported change is related so can someone >>>>>>>>> from ovirt-engine take a look? >>>>>>>>> >>>>>>>>> The failure is add host on the upgrade suite. >>>>>>>>> >>>>>>>>> Please note that we have not had an engine-ovirt build for over 10 >>>>>>>>> days due to several consecutive regressions and I would ask you to >>>>>>>>> stop >>>>>>>>> merging until we can stabilize the project and have a new package of >>>>>>>>> engine. >>>>>>>>> >>>>>>>>> error: >>>>>>>>> >>>>>>>>> 2018-09-14 05:51:07,670-04 INFO >>>>>>>>> [org.ovirt.engine.core.uutils.ssh.SSHDialog] >>>>>>>>> (EE-ManagedThreadFactory-engine-Thread-1) [5c91fcbd] SSH execute >>>>>>>>> 'root@lago-upgrade-from-release-suite-master-host-0' 'umask 0077; >>>>>>>>> MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -d -t >>>>>>>>> ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null >>>>>>>>> 2>&1; rm >>>>>>>>> -fr \"${MYTMP}\" > /dev/null 2>&1" 0; tar -b1 --warning=no-timestamp >>>>>>>>> -C >>>>>>>>> "${MYTMP}" -x && "${MYTMP}"/ovirt-host-deploy >>>>>>>>> DIALOG/dialect=str:machine >>>>>>>>> DIALOG/customization=bool:True' >>>>>>>>> 2018-09-14 05:51:08,550-04 INFO [org.ovirt.engine.core.dal.dbb >>>>>>>>> roker.auditloghandling.AuditLogDirector] (VdsDeploy) [5c91fcbd] >>>>>>>>> EVENT_ID: VDS_INSTALL_IN_PROGRESS(509), Installing Host >>>>>>>>> lago-upgrade-from-release-suite-master-host-0. Stage: >>>>>>>>> Initializing. >>>>>>>>> 2018-09-14 05:51:08,565-04 INFO >>>>>>>>> [org.ovirt.engine.core.utils.transaction.TransactionSupport] >>>>>>>>> (VdsDeploy) [5c91fcbd] transaction rolled back >>>>>>>>> 2018-09-14 05:51:08,574-04 ERROR >>>>>>>>> [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] >>>>>>>>> (VdsDeploy) [5c91fcbd] Error during deploy dialog >>>>>>>>> 2018-09-14 05:51:08,578-04 ERROR >>>>>>>>> [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] >>>>>>>>> (EE-ManagedThreadFactory-engine-Thread-1) [5c91fcbd] Error during >>>>>>>>> host lago-upgrade-from-release-suite-master-host-0 install >>>>>>>>> 2018-09-14 05:51:08,586-04 ERROR [org.ovirt.engine.core.dal.dbb >>>>>>>>> roker.auditloghandling.AuditLogDirector] >>>>>>>>> (EE-ManagedThreadFactory-engine-Thread-1) [5c91fcbd] EVENT_ID: >>>>>>>>> VDS_INSTALL_IN_PROGRESS_ERROR(511), An error has occurred during >>>>>>>>> installation of Host lago-upgrade-from-release-suite-master-host-0: >>>>>>>>> CallableStatementCallback; SQL [{call insertauditlog(?, ?, ?, ?, ?, >>>>>>>>> ?, ?, >>>>>>>>> ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, >>>>>>>>> ?, ?, >>>>>>>>> ?, ?)}ERROR: invalid byte sequence for encoding "UTF8": 0x00; nested >>>>>>>>> exception is org.postgresql.util.PSQLException: ERROR: invalid >>>>>>>>> byte sequence for encoding "UTF8": 0x00. >>>>>>>>> 2018-09-14 05:51:08,586-04 ERROR >>>>>>>>> [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] >>>>>>>>> (EE-ManagedThreadFactory-engine-Thread-1) [5c91fcbd] Error during >>>>>>>>> host lago-upgrade-from-release-suite-master-host-0 install, >>>>>>>>> preferring first exception: CallableStatementCallback; SQL [{call >>>>>>>>> insertauditlog(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, >>>>>>>>> ?, ?, >>>>>>>>> ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)}ERROR: invalid byte >>>>>>>>> sequence for >>>>>>>>> encoding "UTF8": 0x00; nested exception is >>>>>>>>> org.postgresql.util.PSQLException: >>>>>>>>> ERROR: invalid byte sequence for encoding "UTF8": 0x00 >>>>>>>>> 2018-09-14 05:51:08,586-04 ERROR [org.ovirt.engine.core.bll.hos >>>>>>>>> tdeploy.InstallVdsInternalCommand] >>>>>>>>> (EE-ManagedThreadFactory-engine-Thread-1) >>>>>>>>> [5c91fcbd] Host installation failed for host >>>>>>>>> 'e475e93a-63b3-4573-b242-162c2ed864f0', >>>>>>>>> 'lago-upgrade-from-release-suite-master-host-0': >>>>>>>>> CallableStatementCallback; SQL [{call insertauditlog(?, ?, ?, ?, ?, >>>>>>>>> ?, ?, >>>>>>>>> ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, >>>>>>>>> ?, ?, >>>>>>>>> ?, ?)}ERROR: invalid byte sequence for encoding "UTF8": 0x00; nested >>>>>>>>> exception is org.postgresql.util.PSQLException: ERROR: invalid >>>>>>>>> byte sequence for encoding "UTF8": 0x00 >>>>>>>>> 2018-09-14 05:51:08,615-04 INFO >>>>>>>>> [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] >>>>>>>>> (EE-ManagedThreadFactory-engine-Thread-1) [5c91fcbd] START, >>>>>>>>> SetVdsStatusVDSCommand(HostName = >>>>>>>>> lago-upgrade-from-release-suite-master-host-0, >>>>>>>>> SetVdsStatusVDSCommandParameters:{hostId='e475e93a-63b3-4573-b242-162c2ed864f0', >>>>>>>>> status='InstallFailed', nonOperationalReason='NONE', >>>>>>>>> stopSpmFailureLogged='false', maintenanceReason='null'}), log id: >>>>>>>>> 146cdc08 >>>>>>>>> 2018-09-14 05:51:08,626-04 INFO >>>>>>>>> [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] >>>>>>>>> (EE-ManagedThreadFactory-engine-Thread-1) [5c91fcbd] FINISH, >>>>>>>>> SetVdsStatusVDSCommand, return: , log id: 146cdc08 >>>>>>>>> 2018-09-14 05:51:08,639-04 ERROR [org.ovirt.engine.core.dal.dbb >>>>>>>>> roker.auditloghandling.AuditLogDirector] >>>>>>>>> (EE-ManagedThreadFactory-engine-Thread-1) [5c91fcbd] EVENT_ID: >>>>>>>>> VDS_INSTALL_FAILED(505), Host >>>>>>>>> lago-upgrade-from-release-suite-master-host-0 >>>>>>>>> installation failed. CallableStatementCallback; SQL [{call >>>>>>>>> insertauditlog(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, >>>>>>>>> ?, ?, >>>>>>>>> ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)}ERROR: invalid byte >>>>>>>>> sequence for >>>>>>>>> encoding "UTF8": 0x00; nested exception is >>>>>>>>> org.postgresql.util.PSQLException: >>>>>>>>> ERROR: invalid byte sequence for encoding "UTF8": 0x00. >>>>>>>>> 2018-09-14 05:51:08,652-04 INFO [org.ovirt.engine.core.bll.hos >>>>>>>>> tdeploy.InstallVdsInternalCommand] >>>>>>>>> (EE-ManagedThreadFactory-engine-Thread-1) >>>>>>>>> [5c91fcbd] Lock freed to object 'EngineLock:{exclusiveLocks='[ >>>>>>>>> e475e93a-63b3-4573-b242-162c2ed864f0=VDS]', sharedLocks=''}' >>>>>>>>> 2018-09-14 05:51:37,996-04 INFO >>>>>>>>> [org.ovirt.engine.core.bll.quota.QuotaManager] >>>>>>>>> (EE-ManagedThreadFactory-engineScheduled-Thread-44) [] Quota >>>>>>>>> Cache updated. (19 msec) >>>>>>>>> (END) >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Dafna >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Martin Perina >>>>>>> Associate Manager, Software Engineering >>>>>>> Red Hat Czech s.r.o. >>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Martin Perina >>>>> Associate Manager, Software Engineering >>>>> Red Hat Czech s.r.o. >>>>> >>>> >>>> >>> >> >> >> -- >> Martin Perina >> Associate Manager, Software Engineering >> Red Hat Czech s.r.o. >> > > -- Martin Perina Associate Manager, Software Engineering Red Hat Czech s.r.o.
_______________________________________________ Infra mailing list -- [email protected] To unsubscribe send an email to [email protected] Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/[email protected]/message/APX6YMIOHP6OQYFW2Z4GBDO5JRWJXJ37/
