On Fri, Apr 26, 2019 at 12:34 AM Edward Berger <[email protected]> wrote: > > Today, I updated engine to the latest Version 4.3.3.6-1.el7. > Trying to upload an install iso to the hosted storage domain gives same disk > paused by system issue. > > Apr 25, 2019, 5:26:18 PMTransfer was stopped by system. Reason: failed to add > image ticket to ovirt-imageio-proxy. > > > on the node which was running ovirt-image > > /var/log/ovirt-imageio-daemon contains: > 2019-04-25 17:08:51,757 INFO (Thread-2) [http] OPEN client=local > 2019-04-25 17:08:51,758 INFO (Thread-2) [tickets] [local] ADD > ticket={u'uuid': u'6971e886-5339-4df1-88df-445e51fba1c8', u'ops': [u'write'], > u'url': u'fi > le:///rhev/data-center/mnt/glusterSD/10.200.0.131:_engine/0ec355b1-e665-4fdf-87c4-87e1d636c76d/images/4121f1fb-c508-47bc-9acb-a10da9461358/04658323-b267-4f > 66-96f1-6c5848c3d2be', u'sparse': True, u'timeout': 300, u'transfer_id': > u'e85ce170-03e6-480d-bbc5-8551c266cc97', u'size': 1045426176} > 2019-04-25 17:08:51,759 INFO (Thread-2) [http] CLOSE client=local > [connection=0.001420/1, dispatch=0.000634/1] > > going to that directory above shows > # ls -l > total 1025 > -rw-rw----. 1 vdsm kvm 1045426176 Apr 25 17:08 > 04658323-b267-4f66-96f1-6c5848c3d2be > -rw-rw----. 1 vdsm kvm 1048576 Apr 25 17:08 > 04658323-b267-4f66-96f1-6c5848c3d2be.lease > -rw-r--r--. 1 vdsm kvm 345 Apr 25 17:08 > 04658323-b267-4f66-96f1-6c5848c3d2be.meta > > image is showing same size as the iso file. > > On engine > # systemctl status -l ovirt-imageio-proxy.service > ● ovirt-imageio-proxy.service - oVirt ImageIO Proxy > Loaded: loaded (/usr/lib/systemd/system/ovirt-imageio-proxy.service; > enabled; vendor preset: disabled) > Active: active (running) since Thu 2019-04-25 16:36:24 EDT; 51min ago > Main PID: 56989 (ovirt-imageio-p) > CGroup: /system.slice/ovirt-imageio-proxy.service > └─56989 /usr/bin/python2 /usr/bin/ovirt-imageio-proxy > > Apr 25 16:36:24 dispatch.yard.psc.edu systemd[1]: Starting oVirt ImageIO > Proxy... > Apr 25 16:36:24 dispatch.yard.psc.edu systemd[1]: Started oVirt ImageIO Proxy. > Apr 25 17:08:30 dispatch.yard.psc.edu ovirt-imageio-proxy[56989]: > 128.182.168.116 - - [25/Apr/2019 17:08:30] "GET /info/ HTTP/1.1" 200 20 > Apr 25 17:25:46 dispatch.yard.psc.edu ovirt-imageio-proxy[56989]: > 128.182.168.116 - - [25/Apr/2019 17:25:46] "GET /info/ HTTP/1.1" 200 20 > > # cat image-proxy.log > (MainThread) INFO 2019-04-25 16:25:55,755 image_proxy:43:root:(main) Server > shut down, exiting > (MainThread) INFO 2019-04-25 16:36:24,874 server:45:server:(start) Starting > (pid=56989, version=1.5.1) > (MainThread) INFO 2019-04-25 16:36:24,879 image_proxy:34:root:(main) Server > started, successfully notified systemd > > > On Tue, Apr 23, 2019 at 4:51 PM Edward Berger <[email protected]> wrote: >> >> Previously I had issues with the upgrades to 4.3.3 failing because of >> "stale" image transfer data, so I removed it from the database using the >> info given here on the mailing list and was able to complete the oVirt node >> and engine upgrades. >> >> Now I have a new problem. I can't upload a disk image anymore, which used >> to work. >> "test connection" returns success. >> >> In the dashboard Storage/Disks view, it starts and then stops with "paused >> by system"
Hi! Are you trying to use a 3rd-party CA for https access [4][5]? Please check this thread: [1] https://lists.ovirt.org/archives/list/[email protected]/thread/NZBGRCKW6WA4WISTCRDJIFUCMNIKJ2CG/#NZBGRCKW6WA4WISTCRDJIFUCMNIKJ2CG Part of this was already fixed: [2] https://bugzilla.redhat.com/show_bug.cgi?id=1687301 And another part is still pending, although you should easily "fix" manually (editing imageio-proxy.conf): [3] https://bugzilla.redhat.com/show_bug.cgi?id=1637809 If so: 1. Please try again following the entire procedure for using 3rd-party ca. 2. Please also edit /etc/ovirt-imageio-proxy/ovirt-imageio-proxy.conf as explained in above bug [3], and then restart imageio-proxy and the engine. If it still fails, please check/share all relevant differences between a vanilla setup and yours, including all the changes you did as part of following the 3rd-party-ca procedure, custom/different config files, changes you did to the system-wide truststore, etc. [4] https://www.ovirt.org/documentation/admin-guide/appe-oVirt_and_SSL.html [5] https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.2/html/administration_guide/appe-red_hat_enterprise_virtualization_and_ssl >> >> I tried pointing it at another node, same problem. I tried restarting >> services ovirt-imageio-proxy on engine and ovirt-imageio-daemon on the node, >> same failure. >> >> in the dashboard events when trying to upload disk I get this error >> Transfer was stopped by system. Reason: failed to add image ticket to >> ovirt-imageio-proxy. >> >> or when trying to resume transfer "paused by system" giving it the local >> path again. >> Transfer was stopped by system. Reason: failure in transfer image >> ticket renewal. >> >> I'm not sure what logs I should be looking deeper into.. The only relevant ones are the engine's logs and imageio-proxy's, and sadly they didn't help me enough to debug this when I did. >> >> Here's what the database says >> engine=# select * from image_transfers; >> -[ RECORD 1 ]-------------+------------------------------------- >> command_id | 59dce2b1-f8ba-44dd-9df9-c6e39773a3f9 >> command_type | 1024 >> phase | 4 >> last_updated | 2019-04-23 12:22:25.098-04 >> message | Uploading from byte 0 >> vds_id | >> disk_id | 6693d5ac-d3eb-43d7-9abe-97c5197efc23 >> imaged_ticket_id | >> proxy_uri | >> signed_ticket | >> bytes_sent | 0 >> bytes_total | 1996488704 >> type | 2 >> active | f >> daemon_uri | >> client_inactivity_timeout | 60 >> -[ RECORD 2 ]-------------+------------------------------------- >> command_id | 16898219-be5b-4826-8f20-3355fa47272a >> command_type | 1024 >> phase | 4 >> last_updated | 2019-04-23 16:15:34.591-04 >> message | Uploading from byte 0 >> vds_id | >> disk_id | 3b8e3053-bfe4-49d3-abd2-5452b1674400 >> imaged_ticket_id | >> proxy_uri | >> signed_ticket | >> bytes_sent | 0 >> bytes_total | 1998585856 >> type | 2 >> active | f >> daemon_uri | >> client_inactivity_timeout | 60 >> >> I really need the image uploading to work. Any suggestions on what to do >> next? I am ignoring for now your debug attempts looking at logs on the hosts etc., we can go back to that if above is not enough. Good luck and best regards, -- Didi _______________________________________________ Users mailing list -- [email protected] To unsubscribe send an email to [email protected] Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/[email protected]/message/HQKNN42VM35RAKEOZJRYAQBRR7CNWZWH/
