Public bug reported: OpenStack datasource does not deal with 404 Not Found correctly.If we boot a vm without userdata in OpenStack environment, CloudInit retry to get usedata when 404 Not Found returned. The GuestOS start up times will increase.I think it's not needed and a waste of time.
Cloud-Init LOG: 2018-03-19 11:09:16,391 - url_helper.py[DEBUG]: [0/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration 2018-03-19 11:09:16,414 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again 2018-03-19 11:09:17,415 - url_helper.py[DEBUG]: [1/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration 2018-03-19 11:09:17,445 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again 2018-03-19 11:09:18,446 - url_helper.py[DEBUG]: [2/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration 2018-03-19 11:09:18,468 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again 2018-03-19 11:09:19,470 - url_helper.py[DEBUG]: [3/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration 2018-03-19 11:09:19,535 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again 2018-03-19 11:09:20,536 - url_helper.py[DEBUG]: [4/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration 2018-03-19 11:09:20,556 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again 2018-03-19 11:09:21,557 - url_helper.py[DEBUG]: [5/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration 2018-03-19 11:09:21,584 - openstack.py[DEBUG]: Failed reading optional path http://169.254.169.254:80/openstack/2015-10-15/user_data due to: 404 Client Error: Not Found Go through the code, I found the following logic which i think it's not correct. cloudinit/sources/helpers/openstack.py def should_retry_cb(_request_args, cause): try: code = int(cause.code) if code >= 400: return False except (TypeError, ValueError): # Older versions of requests didn't have a code. pass return True response = url_helper.readurl(path, retries=self.retries, ssl_details=self.ssl_details, timeout=self.timeout, exception_cb=should_retry_cb) 1.When 404 returned, do not need to retry. 2.exception_cb=should_retry_cb need to be changed to exception_cb=should_skip_retry_cb. May be a Bug here:) ** Affects: cloud-init Importance: Undecided Status: New ** Description changed: OpenStack datasource does not deal with 404 Not Found correctly.If we boot a vm without userdata in OpenStack environment, CloudInit retry to get usedata when 404 Not Found returned. The GuestOS start up times will increase.I think it's not needed and a waste of time. - Cloud-Init LOG: 2018-03-19 11:09:16,391 - url_helper.py[DEBUG]: [0/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration 2018-03-19 11:09:16,414 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again 2018-03-19 11:09:17,415 - url_helper.py[DEBUG]: [1/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration 2018-03-19 11:09:17,445 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again 2018-03-19 11:09:18,446 - url_helper.py[DEBUG]: [2/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration 2018-03-19 11:09:18,468 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again 2018-03-19 11:09:19,470 - url_helper.py[DEBUG]: [3/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration 2018-03-19 11:09:19,535 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again 2018-03-19 11:09:20,536 - url_helper.py[DEBUG]: [4/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration 2018-03-19 11:09:20,556 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again 2018-03-19 11:09:21,557 - url_helper.py[DEBUG]: [5/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration 2018-03-19 11:09:21,584 - openstack.py[DEBUG]: Failed reading optional path http://169.254.169.254:80/openstack/2015-10-15/user_data due to: 404 Client Error: Not Found - Go through the code, I found the following logic which i think it's not correct. cloudinit/sources/helpers/openstack.py - def should_retry_cb(_request_args, cause): - try: - code = int(cause.code) - if code >= 400: - return False - except (TypeError, ValueError): - # Older versions of requests didn't have a code. - pass - return True + def should_retry_cb(_request_args, cause): + try: + code = int(cause.code) + if code >= 400: + return False + except (TypeError, ValueError): + # Older versions of requests didn't have a code. + pass + return True - response = url_helper.readurl(path, - retries=self.retries, - ssl_details=self.ssl_details, - timeout=self.timeout, - exception_cb=should_retry_cb) + response = url_helper.readurl(path, + retries=self.retries, + ssl_details=self.ssl_details, + timeout=self.timeout, + exception_cb=should_retry_cb) + 1.When 404 returned, do not need to retry. 2.exception_cb=should_retry_cb need to be changed to exception_cb=should_skip_retry_cb. May be a Bug here:) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to cloud-init. https://bugs.launchpad.net/bugs/1757054 Title: OpenStack datasource does not deal with 404 Not Found correctly Status in cloud-init: New Bug description: OpenStack datasource does not deal with 404 Not Found correctly.If we boot a vm without userdata in OpenStack environment, CloudInit retry to get usedata when 404 Not Found returned. The GuestOS start up times will increase.I think it's not needed and a waste of time. Cloud-Init LOG: 2018-03-19 11:09:16,391 - url_helper.py[DEBUG]: [0/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration 2018-03-19 11:09:16,414 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again 2018-03-19 11:09:17,415 - url_helper.py[DEBUG]: [1/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration 2018-03-19 11:09:17,445 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again 2018-03-19 11:09:18,446 - url_helper.py[DEBUG]: [2/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration 2018-03-19 11:09:18,468 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again 2018-03-19 11:09:19,470 - url_helper.py[DEBUG]: [3/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration 2018-03-19 11:09:19,535 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again 2018-03-19 11:09:20,536 - url_helper.py[DEBUG]: [4/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration 2018-03-19 11:09:20,556 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again 2018-03-19 11:09:21,557 - url_helper.py[DEBUG]: [5/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration 2018-03-19 11:09:21,584 - openstack.py[DEBUG]: Failed reading optional path http://169.254.169.254:80/openstack/2015-10-15/user_data due to: 404 Client Error: Not Found Go through the code, I found the following logic which i think it's not correct. cloudinit/sources/helpers/openstack.py def should_retry_cb(_request_args, cause): try: code = int(cause.code) if code >= 400: return False except (TypeError, ValueError): # Older versions of requests didn't have a code. pass return True response = url_helper.readurl(path, retries=self.retries, ssl_details=self.ssl_details, timeout=self.timeout, exception_cb=should_retry_cb) 1.When 404 returned, do not need to retry. 2.exception_cb=should_retry_cb need to be changed to exception_cb=should_skip_retry_cb. May be a Bug here:) To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-init/+bug/1757054/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
