Posting a public pastebin url [1]. Apologies for using the private one before.
[1] https://pastebin.com/wrw5ME7j A. On Wed, Apr 1, 2020 at 12:31 PM Artur Socha <[email protected]> wrote: > > Adding request content: > http://pastebin.test.redhat.com/850652 > > A. > > On Wed, Apr 1, 2020 at 12:28 PM Artur Socha <[email protected]> wrote: >> >> I have debug the flow until the moment the request is being seng via http >> client to ansible runner service and until that point it was correct. The >> json did contain correctly formatted ovirt_ca_cert. >> Artur >> >> On Wed, Apr 1, 2020 at 12:26 PM Marcin Sobczyk <[email protected]> wrote: >>> >>> >>> >>> On 4/1/20 11:54 AM, Martin Perina wrote: >>> >>> >>> >>> On Wed, Apr 1, 2020 at 11:15 AM Marcin Sobczyk <[email protected]> wrote: >>>> >>>> >>>> >>>> On 4/1/20 11:06 AM, Marcin Sobczyk wrote: >>>> > >>>> > >>>> > On 4/1/20 9:51 AM, Marcin Sobczyk wrote: >>>> >> Hi, >>>> >> >>>> >> On 4/1/20 8:44 AM, Yedidyah Bar David wrote: >>>> >>> On Wed, Apr 1, 2020 at 6:21 AM <[email protected]> wrote: >>>> >>>> Project: >>>> >>>> https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/ >>>> >>>> >>>> >>>> Build: >>>> >>>> https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/1548/ >>>> >>> Previous build 1547 passed!, after many months of failing, thanks to >>>> >>> Evgeny's work >>>> >>> in recent weeks. Above one failed. >>>> >>> I think the root cause is that the engine tried to connect to vdsm >>>> >>> right after >>>> >>> successfully finishing ansible host-deploy, but failed. vdsm.log has: >>>> >>> >>>> >>> https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/1548/artifact/exported-artifacts/test_logs/he-basic-suite-master/post-he_deploy/lago-he-basic-suite-master-host-0/_var_log/vdsm/vdsm.log >>>> >>> >>>> >>> >>>> >>> 2020-03-31 22:58:49,773-0400 ERROR (Reactor thread) [vds.dispatcher] >>>> >>> uncaptured python exception, closing channel >>>> >>> <yajsonrpc.betterAsyncore.Dispatcher connected >>>> >>> ('::ffff:192.168.222.76', 46754, 0, 0) at 0x7f416c150a90> (<class >>>> >>> 'ssl.SSLError'>:[X509] no certificate or crl found (_ssl.c:3771) >>>> >>> [/usr/lib64/python3.6/asyncore.py|readwrite|110] >>>> >>> [/usr/lib64/python3.6/asyncore.py|handle_write_event|442] >>>> >>> [/usr/lib/python3.6/site-packages/yajsonrpc/betterAsyncore.py|handle_write|74] >>>> >>> >>>> >>> [/usr/lib/python3.6/site-packages/yajsonrpc/betterAsyncore.py|_delegate_call|168] >>>> >>> >>>> >>> [/usr/lib/python3.6/site-packages/vdsm/sslutils.py|handle_write|190] >>>> >>> [/usr/lib/python3.6/site-packages/vdsm/sslutils.py|_handle_io|194] >>>> >>> [/usr/lib/python3.6/site-packages/vdsm/sslutils.py|_set_up_socket|154]) >>>> >>> (betterAsyncore:179) >>>> >>> >>>> >>> Not sure what might have caused this. Can anyone have a look? Thanks. >>>> >> Probably caused by https://gerrit.ovirt.org/108016 >>>> >> Looking into this. >>>> >> >>>> > Turns out that the patch is not the cause of the error per se - it simply >>>> > uncovered a different problem - the CA on the hosts is broken: >>>> > >>>> > [root@lago-basic-suite-master-host-0 certs]# openssl x509 -in >>>> > /etc/pki/vdsm/certs/cacert.pem -text >>>> > unable to load certificate >>>> > 139987452258112:error:0909006C:PEM routines:get_name:no start >>>> > line:crypto/pem/pem_lib.c:745:Expecting: TRUSTED CERTIFICATE >>>> It looks like they have spaces instead of newlines. >>>> When I manually replaced the spaces to newlines, openssl is able to read >>>> them. >>> >>> >>> Martin/Dana, couldn't this be caused by any recent changes in >>> ansible-runner integrations? >>> >>> This looks like a suspect to me: >>> >>> https://gerrit.ovirt.org/#/c/107683/5/backend/manager/modules/bll/src/main/java/org/ovirt/engine/core/common/utils/ansible/AnsibleRunnerHTTPClient.java >>> >>>> >>>> > >>>> >>> >>>> >>>> Build Number: 1548 >>>> >>>> Build Status: Failure >>>> >>>> Triggered By: Started by timer >>>> >>>> >>>> >>>> ------------------------------------- >>>> >>>> Changes Since Last Success: >>>> >>>> ------------------------------------- >>>> >>>> Changes for Build #1548 >>>> >>>> [Galit Rosenthal] Fix the repo for suites that weren't moved to no >>>> >>>> reposync >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> ----------------- >>>> >>>> Failed Tests: >>>> >>>> ----------------- >>>> >>>> No tests ran. >>>> >>> >>>> >>> >>>> >> >>>> > >>>> >>> >>> >>> -- >>> Martin Perina >>> Manager, Software Engineering >>> Red Hat Czech s.r.o. >>> >>> >> >> >> -- >> >> Artur Socha >> >> Senior Software Engineer, RHV >> >> Red Hat > > > > -- > > Artur Socha > > Senior Software Engineer, RHV > > Red Hat -- Artur Socha Senior Software Engineer, RHV Red Hat _______________________________________________ Devel mailing list -- [email protected] To unsubscribe send an email to [email protected] Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/[email protected]/message/SCSOS5EPQUUHM7DMIM5RPGKFEBFASWBF/
