On Fri, Sep 14, 2018 at 6:06 PM Martin Perina <[email protected]> wrote: > > > > On Fri, Sep 14, 2018 at 4:51 PM, Ravi Shankar Nori <[email protected]> wrote: >> >> I see the same errors on my dev env. From the logs attached by Andrej the >> response received by otopi has a bunch of null chars before the actual >> response CONFIRM DEPLOY_PROCEED=yes >> >> >> >> 2018-09-14 15:49:23,018+0200 DEBUG otopi.plugins.otopi.dialog.machine >> dialog.__logString:204 DIALOG:SEND ### Response is CONFIRM >> DEPLOY_PROCEED=yes|no or ABORT DEPLOY_PROCEED >> >> ^@^@^@^@^@^@^@^@^@CONFIRM DEPLOY_PROCEED=yes > > > Didi/Sandro, could you please take a look? Below error seems like some issue > in otopi, where an error is raised when handling binary input:
Not sure the issue is "binary input" in general, but simply illegal input. The prompt expects, as it says, one of these 3 replies: CONFIRM DEPLOY_PROCEED=yes CONFIRM DEPLOY_PROCEED=no ABORT DEPLOY_PROCEED Instead, judging from the file supplied by Andrej, it gets from the engine: <7169 null bytes>CONFIRM DEPLOY_PROCEED=yes So either the engine now sends, for some reason, 7169 null bytes, in this response, or there is some low-level change causing this to be eventually supplied to otopi - a change in apache-sshd, openssh, some library, the kernel, no idea. Well, thinking a bit, I have a wild guess: Perhaps it's related to the patch introduced recently to change the tar blocking? > > > 2018-09-14 15:49:23,032+0200 DEBUG otopi.context context._executeMethod:143 > method exception > Traceback (most recent call last): > File "/usr/lib/python2.7/site-packages/otopi/context.py", line 133, in > _executeMethod > method['method']() > File "/tmp/ovirt-O6CfS4aUHI/otopi-plugins/ovirt-host-deploy/core/misc.py", > line 87, in _confirm > prompt=True, > File "/tmp/ovirt-O6CfS4aUHI/otopi-plugins/otopi/dialog/machine.py", line > 478, in confirm > code=opcode, > > >> >> On Fri, Sep 14, 2018 at 10:44 AM, Dafna Ron <[email protected]> wrote: >>> >>> if you run it with mock you would remove any environmental conditions that >>> can effect the outcome so I recommend using mock >>> >>> >>> On Fri, Sep 14, 2018 at 3:32 PM, Martin Perina <[email protected]> wrote: >>>> >>>> >>>> >>>> On Fri, Sep 14, 2018 at 3:49 PM, Dafna Ron <[email protected]> wrote: >>>>> >>>>> did you use mock to reproduce? >>>> >>>> >>>> No, just run_suite under myself >>>>> >>>>> >>>>> On Fri, Sep 14, 2018 at 2:39 PM, Martin Perina <[email protected]> wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> the problem is that we haven't fetched the temporary host-deploy log >>>>>> from /tmp directory, so we don't know which string that host-deploy >>>>>> process sent to engine is causing that issue. I tried to reproduce on my >>>>>> local machine, but I was unable to reproduce it, 002_bootstrap phase >>>>>> finished successfully (other phases are still running). >>>>>> >>>>>> So if anyone is able to reproduce, please try to fetch host-deploy log >>>>>> from /tmp directory after the error is raised and share it. >>>>>> >>>>>> Thanks >>>>>> >>>>>> Martin >>>>>> >>>>>> >>>>>> On Fri, Sep 14, 2018 at 1:52 PM, Dafna Ron <[email protected]> wrote: >>>>>>> >>>>>>> Full logs can be found here: >>>>>>> >>>>>>> https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change-queue-tester/10307/artifact/upgrade-from-release-suite.el7.x86_64/test_logs/upgrade-from-release-suite-master/post-002_bootstrap.py/ >>>>>>> >>>>>>> On Fri, Sep 14, 2018 at 12:48 PM, Dafna Ron <[email protected]> wrote: >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> The previous regression was resolved and we now have a new regression. >>>>>>>> >>>>>>>> I don't think that the reported change is related so can someone from >>>>>>>> ovirt-engine take a look? >>>>>>>> >>>>>>>> The failure is add host on the upgrade suite. >>>>>>>> >>>>>>>> Please note that we have not had an engine-ovirt build for over 10 >>>>>>>> days due to several consecutive regressions and I would ask you to >>>>>>>> stop merging until we can stabilize the project and have a new package >>>>>>>> of engine. >>>>>>>> >>>>>>>> error: >>>>>>>> >>>>>>>> 2018-09-14 05:51:07,670-04 INFO >>>>>>>> [org.ovirt.engine.core.uutils.ssh.SSHDialog] >>>>>>>> (EE-ManagedThreadFactory-engine-Thread-1) [5c91fcbd] SSH execute >>>>>>>> 'root@lago-upgrade-from-release-suite-master-host-0' 'umask 0077; >>>>>>>> MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -d -t ovirt-XXXXXXXXXX)"; >>>>>>>> trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr >>>>>>>> \"${MYTMP}\" > /dev/null 2>&1" 0; tar -b1 --warning=no-timestamp -C >>>>>>>> "${MYTMP}" -x && "${MYTMP}"/ovirt-host-deploy >>>>>>>> DIALOG/dialect=str:machine DIALOG/customization=bool:True' >>>>>>>> 2018-09-14 05:51:08,550-04 INFO >>>>>>>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] >>>>>>>> (VdsDeploy) [5c91fcbd] EVENT_ID: VDS_INSTALL_IN_PROGRESS(509), >>>>>>>> Installing Host lago-upgrade-from-release-suite-master-host-0. Stage: >>>>>>>> Initializing. >>>>>>>> 2018-09-14 05:51:08,565-04 INFO >>>>>>>> [org.ovirt.engine.core.utils.transaction.TransactionSupport] >>>>>>>> (VdsDeploy) [5c91fcbd] transaction rolled back >>>>>>>> 2018-09-14 05:51:08,574-04 ERROR >>>>>>>> [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) >>>>>>>> [5c91fcbd] Error during deploy dialog >>>>>>>> 2018-09-14 05:51:08,578-04 ERROR >>>>>>>> [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] >>>>>>>> (EE-ManagedThreadFactory-engine-Thread-1) [5c91fcbd] Error during host >>>>>>>> lago-upgrade-from-release-suite-master-host-0 install >>>>>>>> 2018-09-14 05:51:08,586-04 ERROR >>>>>>>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] >>>>>>>> (EE-ManagedThreadFactory-engine-Thread-1) [5c91fcbd] EVENT_ID: >>>>>>>> VDS_INSTALL_IN_PROGRESS_ERROR(511), An error has occurred during >>>>>>>> installation of Host lago-upgrade-from-release-suite-master-host-0: >>>>>>>> CallableStatementCallback; SQL [{call insertauditlog(?, ?, ?, ?, ?, ?, >>>>>>>> ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, >>>>>>>> ?, ?, ?, ?, ?)}ERROR: invalid byte sequence for encoding "UTF8": 0x00; >>>>>>>> nested exception is org.postgresql.util.PSQLException: ERROR: invalid >>>>>>>> byte sequence for encoding "UTF8": 0x00. >>>>>>>> 2018-09-14 05:51:08,586-04 ERROR >>>>>>>> [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] >>>>>>>> (EE-ManagedThreadFactory-engine-Thread-1) [5c91fcbd] Error during host >>>>>>>> lago-upgrade-from-release-suite-master-host-0 install, preferring >>>>>>>> first exception: CallableStatementCallback; SQL [{call >>>>>>>> insertauditlog(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, >>>>>>>> ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)}ERROR: invalid byte >>>>>>>> sequence for encoding "UTF8": 0x00; nested exception is >>>>>>>> org.postgresql.util.PSQLException: ERROR: invalid byte sequence for >>>>>>>> encoding "UTF8": 0x00 >>>>>>>> 2018-09-14 05:51:08,586-04 ERROR >>>>>>>> [org.ovirt.engine.core.bll.hostdeploy.InstallVdsInternalCommand] >>>>>>>> (EE-ManagedThreadFactory-engine-Thread-1) [5c91fcbd] Host installation >>>>>>>> failed for host 'e475e93a-63b3-4573-b242-162c2ed864f0', >>>>>>>> 'lago-upgrade-from-release-suite-master-host-0': >>>>>>>> CallableStatementCallback; SQL [{call insertauditlog(?, ?, ?, ?, ?, ?, >>>>>>>> ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, >>>>>>>> ?, ?, ?, ?, ?)}ERROR: invalid byte sequence for encoding "UTF8": 0x00; >>>>>>>> nested exception is org.postgresql.util.PSQLException: ERROR: invalid >>>>>>>> byte sequence for encoding "UTF8": 0x00 >>>>>>>> 2018-09-14 05:51:08,615-04 INFO >>>>>>>> [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] >>>>>>>> (EE-ManagedThreadFactory-engine-Thread-1) [5c91fcbd] START, >>>>>>>> SetVdsStatusVDSCommand(HostName = >>>>>>>> lago-upgrade-from-release-suite-master-host-0, >>>>>>>> SetVdsStatusVDSCommandParameters:{hostId='e475e93a-63b3-4573-b242-162c2ed864f0', >>>>>>>> status='InstallFailed', nonOperationalReason='NONE', >>>>>>>> stopSpmFailureLogged='false', maintenanceReason='null'}), log id: >>>>>>>> 146cdc08 >>>>>>>> 2018-09-14 05:51:08,626-04 INFO >>>>>>>> [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] >>>>>>>> (EE-ManagedThreadFactory-engine-Thread-1) [5c91fcbd] FINISH, >>>>>>>> SetVdsStatusVDSCommand, return: , log id: 146cdc08 >>>>>>>> 2018-09-14 05:51:08,639-04 ERROR >>>>>>>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] >>>>>>>> (EE-ManagedThreadFactory-engine-Thread-1) [5c91fcbd] EVENT_ID: >>>>>>>> VDS_INSTALL_FAILED(505), Host >>>>>>>> lago-upgrade-from-release-suite-master-host-0 installation failed. >>>>>>>> CallableStatementCallback; SQL [{call insertauditlog(?, ?, ?, ?, ?, ?, >>>>>>>> ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, >>>>>>>> ?, ?, ?, ?, ?)}ERROR: invalid byte sequence for encoding "UTF8": 0x00; >>>>>>>> nested exception is org.postgresql.util.PSQLException: ERROR: invalid >>>>>>>> byte sequence for encoding "UTF8": 0x00. >>>>>>>> 2018-09-14 05:51:08,652-04 INFO >>>>>>>> [org.ovirt.engine.core.bll.hostdeploy.InstallVdsInternalCommand] >>>>>>>> (EE-ManagedThreadFactory-engine-Thread-1) [5c91fcbd] Lock freed to >>>>>>>> object >>>>>>>> 'EngineLock:{exclusiveLocks='[e475e93a-63b3-4573-b242-162c2ed864f0=VDS]', >>>>>>>> sharedLocks=''}' >>>>>>>> 2018-09-14 05:51:37,996-04 INFO >>>>>>>> [org.ovirt.engine.core.bll.quota.QuotaManager] >>>>>>>> (EE-ManagedThreadFactory-engineScheduled-Thread-44) [] Quota Cache >>>>>>>> updated. (19 msec) >>>>>>>> (END) >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Dafna >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Martin Perina >>>>>> Associate Manager, Software Engineering >>>>>> Red Hat Czech s.r.o. >>>>> >>>>> >>>> >>>> >>>> >>>> -- >>>> Martin Perina >>>> Associate Manager, Software Engineering >>>> Red Hat Czech s.r.o. >>> >>> >> > > > > -- > Martin Perina > Associate Manager, Software Engineering > Red Hat Czech s.r.o. -- Didi _______________________________________________ Infra mailing list -- [email protected] To unsubscribe send an email to [email protected] Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/[email protected]/message/5GXUOMRABS7YMMQFME4VY4OPRTS5I4JJ/
