Public bug reported: Hi Team,
I would like to understand why and how does glance-api pod generate HTTP 409 errors, i did look into the available documentation and code but had no conclusion. I referred the below : https://github.com/openstack/glance/blob/stable/train/doc/source/user/glanceapi.rst https://github.com/openstack/glance/blob/54329c6a21b0d3f845b09e79f710fc795976a175/releasenotes/source/locale/ja/LC_MESSAGES/releasenotes.po https://bugs.launchpad.net/glance/+bug/1229823 https://docs.openstack.org/glance/pike/configuration/configuring.html#configuring-the-swift-storage-backend I wonder the HTTP 409 conflicts occur due to the below flags, are the api calls parallel or sequential for deletion ? i.e. a HTTP 409 means a Conflict response status code, maybe the deletion calls dont wait until the deletion is completed ? : swift_store_large_object_size = 5120 swift_store_large_object_chunk_size = 512 (default is 200Mb (i dont see HTTP 409), would it generate HTTP 409 if its increased to 500Mb) glance-api logs : 2020-07-18 01:55:46,627.627 52 ERROR glance.common.wsgi [req-236a9c8c- 396e-42a1-8987-f847923c7e13 f1083795e1da57ca00ff8c967ad0c3d80751fe341a1e64046869e0ae0770cc1d 7e49c7a15b4a4f149cae86a0c1366afa - ec213443e8834473b579f7bea9e8c194 ec213443e8834473b579f7bea9e8c194] Caught error: Container DELETE failed: https://xxx:443/v1/AUTH_7e49c7a15b4a4f149cae86a0c1366afa/glance_1352ddc3 -12ba-4afe-9c89-304cefd90ef5 409 Conflict [first 60 chars of response] b'<html><h1>Conflict</h1><p>There was a conflict when trying t': swiftclient.exceptions.ClientException: Container DELETE failed: https://xxx:443/v1/AUTH_7e49c7a15b4a4f149cae86a0c1366afa/glance_1352ddc3 -12ba-4afe-9c89-304cefd90ef5 409 Conflict [first 60 chars of response] b'<html><h1>Conflict</h1><p>There was a conflict when trying t' 2020-07-18 01:55:46,707.707 52 INFO eventlet.wsgi.server [req-236a9c8c- 396e-42a1-8987-f847923c7e13 f1083795e1da57ca00ff8c967ad0c3d80751fe341a1e64046869e0ae0770cc1d 7e49c7a15b4a4f149cae86a0c1366afa - ec213443e8834473b579f7bea9e8c194 ec213443e8834473b579f7bea9e8c194] 10.46.14.92,100.85.0.29 - - [18/Jul/2020 01:55:46] "DELETE /v2/images/1352ddc3-12ba-4afe- 9c89-304cefd90ef5 HTTP/1.1" 500 449 11.140969 Openstack Glance Version : Train Glance-api.conf : https://github.com/sapcc/helm-charts/blob/master/openstack/glance/templates/etc/_glance-api.conf.tpl Swift-api.conf Swift conf file : https://github.com/sapcc/helm-charts/blob/master/openstack/swift/templates/etc/_proxy-server.conf.tpl ** Affects: glance Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to Glance. https://bugs.launchpad.net/bugs/1890580 Title: HTTP 409 error from glance-api with swift backend Status in Glance: New Bug description: Hi Team, I would like to understand why and how does glance-api pod generate HTTP 409 errors, i did look into the available documentation and code but had no conclusion. I referred the below : https://github.com/openstack/glance/blob/stable/train/doc/source/user/glanceapi.rst https://github.com/openstack/glance/blob/54329c6a21b0d3f845b09e79f710fc795976a175/releasenotes/source/locale/ja/LC_MESSAGES/releasenotes.po https://bugs.launchpad.net/glance/+bug/1229823 https://docs.openstack.org/glance/pike/configuration/configuring.html#configuring-the-swift-storage-backend I wonder the HTTP 409 conflicts occur due to the below flags, are the api calls parallel or sequential for deletion ? i.e. a HTTP 409 means a Conflict response status code, maybe the deletion calls dont wait until the deletion is completed ? : swift_store_large_object_size = 5120 swift_store_large_object_chunk_size = 512 (default is 200Mb (i dont see HTTP 409), would it generate HTTP 409 if its increased to 500Mb) glance-api logs : 2020-07-18 01:55:46,627.627 52 ERROR glance.common.wsgi [req-236a9c8c- 396e-42a1-8987-f847923c7e13 f1083795e1da57ca00ff8c967ad0c3d80751fe341a1e64046869e0ae0770cc1d 7e49c7a15b4a4f149cae86a0c1366afa - ec213443e8834473b579f7bea9e8c194 ec213443e8834473b579f7bea9e8c194] Caught error: Container DELETE failed: https://xxx:443/v1/AUTH_7e49c7a15b4a4f149cae86a0c1366afa/glance_1352ddc3 -12ba-4afe-9c89-304cefd90ef5 409 Conflict [first 60 chars of response] b'<html><h1>Conflict</h1><p>There was a conflict when trying t': swiftclient.exceptions.ClientException: Container DELETE failed: https://xxx:443/v1/AUTH_7e49c7a15b4a4f149cae86a0c1366afa/glance_1352ddc3 -12ba-4afe-9c89-304cefd90ef5 409 Conflict [first 60 chars of response] b'<html><h1>Conflict</h1><p>There was a conflict when trying t' 2020-07-18 01:55:46,707.707 52 INFO eventlet.wsgi.server [req- 236a9c8c-396e-42a1-8987-f847923c7e13 f1083795e1da57ca00ff8c967ad0c3d80751fe341a1e64046869e0ae0770cc1d 7e49c7a15b4a4f149cae86a0c1366afa - ec213443e8834473b579f7bea9e8c194 ec213443e8834473b579f7bea9e8c194] 10.46.14.92,100.85.0.29 - - [18/Jul/2020 01:55:46] "DELETE /v2/images/1352ddc3-12ba-4afe- 9c89-304cefd90ef5 HTTP/1.1" 500 449 11.140969 Openstack Glance Version : Train Glance-api.conf : https://github.com/sapcc/helm-charts/blob/master/openstack/glance/templates/etc/_glance-api.conf.tpl Swift-api.conf Swift conf file : https://github.com/sapcc/helm-charts/blob/master/openstack/swift/templates/etc/_proxy-server.conf.tpl To manage notifications about this bug go to: https://bugs.launchpad.net/glance/+bug/1890580/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
