Public bug reported: As trying to integrate the Keystone with Redis as the caching layer (the procedures are working fine with the Memcached), the following error is being shown: The information: OS = Ubuntu 20.04 Openstack repo: Ubuntu cloud (Wallaby edition) keystone version 19.0.0
When i issue the "openstack endpoint list --service identity" command, and the output shows the "Internal Server Error (HTTP 500)". ==================================== The following error is the error shown in the /var/log/apache2/keystone.log: here is the link for https://paste.ubuntu.com/p/CmrgZ2JrkX/ the following is the content of the link: 2021-05-01 16:37:17.350115 mod_wsgi (pid=241861): Exception occurred processing WSGI script '/usr/bin/keystone-wsgi-public'. 2021-05-01 16:37:17.358549 Traceback (most recent call last): 2021-05-01 16:37:17.359016 File "/usr/lib/python3/dist-packages/flask/app.py", line 2463, in __call__ 2021-05-01 16:37:17.359044 return self.wsgi_app(environ, start_response) 2021-05-01 16:37:17.359072 File "/usr/lib/python3/dist-packages/werkzeug/middleware/proxy_fix.py", line 232, in __call__ 2021-05-01 16:37:17.359083 return self.app(environ, start_response) 2021-05-01 16:37:17.359104 File "/usr/lib/python3/dist-packages/webob/dec.py", line 129, in __call__ 2021-05-01 16:37:17.359114 resp = self.call_func(req, *args, **kw) 2021-05-01 16:37:17.359134 File "/usr/lib/python3/dist-packages/webob/dec.py", line 193, in call_func 2021-05-01 16:37:17.359144 return self.func(req, *args, **kwargs) 2021-05-01 16:37:17.359165 File "/usr/lib/python3/dist-packages/oslo_middleware/base.py", line 124, in __call__ 2021-05-01 16:37:17.359174 response = req.get_response(self.application) 2021-05-01 16:37:17.359195 File "/usr/lib/python3/dist-packages/webob/request.py", line 1313, in send 2021-05-01 16:37:17.359204 status, headers, app_iter = self.call_application( 2021-05-01 16:37:17.359225 File "/usr/lib/python3/dist-packages/webob/request.py", line 1278, in call_application 2021-05-01 16:37:17.359239 app_iter = application(self.environ, start_response) 2021-05-01 16:37:17.359260 File "/usr/lib/python3/dist-packages/webob/dec.py", line 143, in __call__ 2021-05-01 16:37:17.359271 return resp(environ, start_response) 2021-05-01 16:37:17.359293 File "/usr/lib/python3/dist-packages/webob/dec.py", line 129, in __call__ 2021-05-01 16:37:17.359304 resp = self.call_func(req, *args, **kw) 2021-05-01 16:37:17.359325 File "/usr/lib/python3/dist-packages/webob/dec.py", line 193, in call_func 2021-05-01 16:37:17.359339 return self.func(req, *args, **kwargs) 2021-05-01 16:37:17.359363 File "/usr/lib/python3/dist-packages/oslo_middleware/base.py", line 124, in __call__ 2021-05-01 16:37:17.359373 response = req.get_response(self.application) 2021-05-01 16:37:17.359395 File "/usr/lib/python3/dist-packages/webob/request.py", line 1313, in send 2021-05-01 16:37:17.359406 status, headers, app_iter = self.call_application( 2021-05-01 16:37:17.359429 File "/usr/lib/python3/dist-packages/webob/request.py", line 1278, in call_application 2021-05-01 16:37:17.359439 app_iter = application(self.environ, start_response) 2021-05-01 16:37:17.359460 File "/usr/lib/python3/dist-packages/webob/dec.py", line 129, in __call__ 2021-05-01 16:37:17.359470 resp = self.call_func(req, *args, **kw) 2021-05-01 16:37:17.359492 File "/usr/lib/python3/dist-packages/webob/dec.py", line 193, in call_func 2021-05-01 16:37:17.359502 return self.func(req, *args, **kwargs) 2021-05-01 16:37:17.359523 File "/usr/lib/python3/dist-packages/osprofiler/web.py", line 112, in __call__ 2021-05-01 16:37:17.359534 return request.get_response(self.application) 2021-05-01 16:37:17.359579 File "/usr/lib/python3/dist-packages/webob/request.py", line 1313, in send 2021-05-01 16:37:17.359589 status, headers, app_iter = self.call_application( 2021-05-01 16:37:17.359611 File "/usr/lib/python3/dist-packages/webob/request.py", line 1278, in call_application 2021-05-01 16:37:17.359625 app_iter = application(self.environ, start_response) 2021-05-01 16:37:17.359648 File "/usr/lib/python3/dist-packages/webob/dec.py", line 129, in __call__ 2021-05-01 16:37:17.359658 resp = self.call_func(req, *args, **kw) 2021-05-01 16:37:17.359679 File "/usr/lib/python3/dist-packages/webob/dec.py", line 193, in call_func 2021-05-01 16:37:17.359688 return self.func(req, *args, **kwargs) 2021-05-01 16:37:17.359709 File "/usr/lib/python3/dist-packages/oslo_middleware/request_id.py", line 58, in __call__ 2021-05-01 16:37:17.359722 response = req.get_response(self.application) 2021-05-01 16:37:17.359745 File "/usr/lib/python3/dist-packages/webob/request.py", line 1313, in send 2021-05-01 16:37:17.359754 status, headers, app_iter = self.call_application( 2021-05-01 16:37:17.359776 File "/usr/lib/python3/dist-packages/webob/request.py", line 1278, in call_application 2021-05-01 16:37:17.359785 app_iter = application(self.environ, start_response) 2021-05-01 16:37:17.359806 File "/usr/lib/python3/dist-packages/keystone/server/flask/request_processing/middleware/url_normalize.py", line 38, in __call__ 2021-05-01 16:37:17.359817 return self.app(environ, start_response) 2021-05-01 16:37:17.359839 File "/usr/lib/python3/dist-packages/webob/dec.py", line 129, in __call__ 2021-05-01 16:37:17.359849 resp = self.call_func(req, *args, **kw) 2021-05-01 16:37:17.359871 File "/usr/lib/python3/dist-packages/webob/dec.py", line 193, in call_func 2021-05-01 16:37:17.359880 return self.func(req, *args, **kwargs) 2021-05-01 16:37:17.359902 File "/usr/lib/python3/dist-packages/keystonemiddleware/auth_token/__init__.py", line 341, in __call__ 2021-05-01 16:37:17.359912 response = req.get_response(self._app) 2021-05-01 16:37:17.359933 File "/usr/lib/python3/dist-packages/webob/request.py", line 1313, in send 2021-05-01 16:37:17.359942 status, headers, app_iter = self.call_application( 2021-05-01 16:37:17.359963 File "/usr/lib/python3/dist-packages/webob/request.py", line 1278, in call_application 2021-05-01 16:37:17.359973 app_iter = application(self.environ, start_response) 2021-05-01 16:37:17.359994 File "/usr/lib/python3/dist-packages/werkzeug/middleware/dispatcher.py", line 66, in __call__ 2021-05-01 16:37:17.360004 return app(environ, start_response) 2021-05-01 16:37:17.360026 File "/usr/lib/python3/dist-packages/flask/app.py", line 2449, in wsgi_app 2021-05-01 16:37:17.360036 response = self.handle_exception(e) 2021-05-01 16:37:17.360057 File "/usr/lib/python3/dist-packages/flask_restful/__init__.py", line 272, in error_router 2021-05-01 16:37:17.360066 return original_handler(e) 2021-05-01 16:37:17.360087 File "/usr/lib/python3/dist-packages/flask_restful/__init__.py", line 272, in error_router 2021-05-01 16:37:17.360097 return original_handler(e) 2021-05-01 16:37:17.360118 File "/usr/lib/python3/dist-packages/flask_restful/__init__.py", line 272, in error_router 2021-05-01 16:37:17.360127 return original_handler(e) 2021-05-01 16:37:17.360148 [Previous line repeated 27 more times] 2021-05-01 16:37:17.360171 File "/usr/lib/python3/dist-packages/flask/app.py", line 1866, in handle_exception 2021-05-01 16:37:17.360182 reraise(exc_type, exc_value, tb) 2021-05-01 16:37:17.360203 File "/usr/lib/python3/dist-packages/flask/_compat.py", line 38, in reraise 2021-05-01 16:37:17.360213 raise value.with_traceback(tb) 2021-05-01 16:37:17.360233 File "/usr/lib/python3/dist-packages/flask/app.py", line 2446, in wsgi_app 2021-05-01 16:37:17.360243 response = self.full_dispatch_request() 2021-05-01 16:37:17.360264 File "/usr/lib/python3/dist-packages/flask/app.py", line 1951, in full_dispatch_request 2021-05-01 16:37:17.360284 rv = self.handle_user_exception(e) 2021-05-01 16:37:17.360308 File "/usr/lib/python3/dist-packages/flask_restful/__init__.py", line 272, in error_router 2021-05-01 16:37:17.360317 return original_handler(e) 2021-05-01 16:37:17.360338 File "/usr/lib/python3/dist-packages/flask_restful/__init__.py", line 272, in error_router 2021-05-01 16:37:17.360352 return original_handler(e) 2021-05-01 16:37:17.360375 File "/usr/lib/python3/dist-packages/flask_restful/__init__.py", line 272, in error_router 2021-05-01 16:37:17.360385 return original_handler(e) 2021-05-01 16:37:17.360406 [Previous line repeated 27 more times] 2021-05-01 16:37:17.360429 File "/usr/lib/python3/dist-packages/flask/app.py", line 1820, in handle_user_exception 2021-05-01 16:37:17.360440 reraise(exc_type, exc_value, tb) 2021-05-01 16:37:17.360461 File "/usr/lib/python3/dist-packages/flask/_compat.py", line 38, in reraise 2021-05-01 16:37:17.360471 raise value.with_traceback(tb) 2021-05-01 16:37:17.360493 File "/usr/lib/python3/dist-packages/flask/app.py", line 1949, in full_dispatch_request 2021-05-01 16:37:17.360502 rv = self.dispatch_request() 2021-05-01 16:37:17.360523 File "/usr/lib/python3/dist-packages/flask/app.py", line 1935, in dispatch_request 2021-05-01 16:37:17.360533 return self.view_functions[rule.endpoint](**req.view_args) 2021-05-01 16:37:17.360554 File "/usr/lib/python3/dist-packages/flask_restful/__init__.py", line 468, in wrapper 2021-05-01 16:37:17.360565 resp = resource(*args, **kwargs) 2021-05-01 16:37:17.360586 File "/usr/lib/python3/dist-packages/flask/views.py", line 89, in view 2021-05-01 16:37:17.360596 return self.dispatch_request(*args, **kwargs) 2021-05-01 16:37:17.360617 File "/usr/lib/python3/dist-packages/flask_restful/__init__.py", line 583, in dispatch_request 2021-05-01 16:37:17.360627 resp = meth(*args, **kwargs) 2021-05-01 16:37:17.360648 File "/usr/lib/python3/dist-packages/keystone/server/flask/common.py", line 1064, in wrapper 2021-05-01 16:37:17.360659 return f(*args, **kwargs) 2021-05-01 16:37:17.360680 File "/usr/lib/python3/dist-packages/keystone/api/auth.py", line 315, in post 2021-05-01 16:37:17.360690 token = authentication.authenticate_for_token(auth_data) 2021-05-01 16:37:17.360711 File "/usr/lib/python3/dist-packages/keystone/api/_shared/authentication.py", line 182, in authenticate_for_token 2021-05-01 16:37:17.360721 auth_info = core.AuthInfo.create(auth=auth) 2021-05-01 16:37:17.360743 File "/usr/lib/python3/dist-packages/keystone/auth/core.py", line 126, in create 2021-05-01 16:37:17.360752 auth_info._validate_and_normalize_auth_data(scope_only) 2021-05-01 16:37:17.360773 File "/usr/lib/python3/dist-packages/keystone/auth/core.py", line 340, in _validate_and_normalize_auth_data 2021-05-01 16:37:17.360787 self._validate_and_normalize_scope_data() 2021-05-01 16:37:17.360810 File "/usr/lib/python3/dist-packages/keystone/auth/core.py", line 290, in _validate_and_normalize_scope_data 2021-05-01 16:37:17.360820 project_ref = self._lookup_project(self.auth['scope']['project']) 2021-05-01 16:37:17.360841 File "/usr/lib/python3/dist-packages/keystone/auth/core.py", line 196, in _lookup_project 2021-05-01 16:37:17.360851 domain_ref = self._lookup_domain(project_info['domain']) 2021-05-01 16:37:17.360873 File "/usr/lib/python3/dist-packages/keystone/auth/core.py", line 171, in _lookup_domain 2021-05-01 16:37:17.360882 domain_ref = PROVIDERS.resource_api.get_domain_by_name( 2021-05-01 16:37:17.360903 File "/usr/lib/python3/dist-packages/keystone/common/manager.py", line 115, in wrapped 2021-05-01 16:37:17.360914 __ret_val = __f(*args, **kwargs) 2021-05-01 16:37:17.360935 File "<decorator-gen-28>", line 2, in get_domain_by_name 2021-05-01 16:37:17.360958 File "/usr/lib/python3/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func 2021-05-01 16:37:17.360969 return self.get_or_create( 2021-05-01 16:37:17.360990 File "/usr/lib/python3/dist-packages/dogpile/cache/region.py", line 974, in get_or_create 2021-05-01 16:37:17.361055 key = self.key_mangler(key) 2021-05-01 16:37:17.361083 File "/usr/lib/python3/dist-packages/keystone/common/cache/core.py", line 86, in key_mangler 2021-05-01 16:37:17.361095 key = '%s:%s' % (key, invalidation_manager.region_id) 2021-05-01 16:37:17.361117 File "/usr/lib/python3/dist-packages/keystone/common/cache/core.py", line 43, in region_id 2021-05-01 16:37:17.361126 return self._invalidation_region.get_or_create( 2021-05-01 16:37:17.361148 File "/usr/lib/python3/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create 2021-05-01 16:37:17.361158 with Lock( 2021-05-01 16:37:17.361179 File "/usr/lib/python3/dist-packages/dogpile/lock.py", line 185, in __enter__ 2021-05-01 16:37:17.361189 return self._enter() 2021-05-01 16:37:17.361209 File "/usr/lib/python3/dist-packages/dogpile/lock.py", line 87, in _enter 2021-05-01 16:37:17.361223 value = value_fn() 2021-05-01 16:37:17.361246 File "/usr/lib/python3/dist-packages/dogpile/cache/region.py", line 977, in get_value 2021-05-01 16:37:17.361256 value = self._get_from_backend(key) 2021-05-01 16:37:17.361277 File "/usr/lib/python3/dist-packages/dogpile/cache/region.py", line 1265, in _get_from_backend 2021-05-01 16:37:17.361286 self.backend.get_serialized(key) 2021-05-01 16:37:17.361307 File "/usr/lib/python3/dist-packages/dogpile/cache/api.py", line 288, in get_serialized 2021-05-01 16:37:17.361317 return cast(SerializedReturnType, self.get(key)) 2021-05-01 16:37:17.361338 File "/usr/lib/python3/dist-packages/keystone/common/cache/_context_cache.py", line 74, in get 2021-05-01 16:37:17.361348 value = self.proxied.get(key) 2021-05-01 16:37:17.361369 File "/usr/lib/python3/dist-packages/dogpile/cache/api.py", line 245, in get 2021-05-01 16:37:17.361378 raise NotImplementedError() 2021-05-01 16:37:17.361460 NotImplementedError follwoing is the keystone.conf configuration for both Apache: ======================================================== cat /etc/apache2/sites-enabled/keystone.conf Listen 5000 <VirtualHost *:5000> WSGIScriptAlias / /usr/bin/keystone-wsgi-public WSGIDaemonProcess keystone-public processes=5 threads=1 user=keystone group=keystone display-name=%{GROUP} WSGIProcessGroup keystone-public WSGIApplicationGroup %{GLOBAL} WSGIPassAuthorization On LimitRequestBody 114688 <IfVersion >= 2.4> ErrorLogFormat "%{cu}t %M" </IfVersion> ErrorLog /var/log/apache2/keystone.log CustomLog /var/log/apache2/keystone_access.log combined <Directory /usr/bin> <IfVersion >= 2.4> Require all granted </IfVersion> <IfVersion < 2.4> Order allow,deny Allow from all </IfVersion> </Directory> </VirtualHost> Alias /identity /usr/bin/keystone-wsgi-public <Location /identity> SetHandler wsgi-script Options +ExecCGI WSGIProcessGroup keystone-public WSGIApplicationGroup %{GLOBAL} WSGIPassAuthorization On </Location> ==================================== Content of /etc/keystone/keystone.conf [DEFAULT] log_dir = /var/log/keystone default_log_levels = oslo.cache=DEBUG,dogpile.core.dogpile=DEBUG [application_credential] [assignment] [auth] [cache] debug_cache_backend = True backend = dogpile.cache.redis backend_argument=url:redis://10.10.13.210/6379/0 enabled = true [catalog] [cors] [credential] [database] connection = mysql+pymysql://keystone:[email protected]/keystone [domain_config] [endpoint_filter] [endpoint_policy] [eventlet_server] [extra_headers] Distribution = Ubuntu [federation] [fernet_receipts] [fernet_tokens] [healthcheck] [identity] [identity_mapping] [jwt_tokens] [ldap] [memcache] [oauth1] [oslo_messaging_amqp] [oslo_messaging_kafka] [oslo_messaging_notifications] [oslo_messaging_rabbit] [oslo_middleware] [oslo_policy] [policy] [profiler] [receipt] [resource] [revoke] [role] [saml] [security_compliance] [shadow_users] [token] provider = fernet caching = true driver = keystone.token.persistence.backends.kvs.Token [tokenless_auth] [totp] [trust] [unified_limit] [wsgi] ** Affects: keystone Importance: Undecided Status: New ** Tags: redis -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Identity (keystone). https://bugs.launchpad.net/bugs/1926836 Title: Keystone Redis Caching Status in OpenStack Identity (keystone): New Bug description: As trying to integrate the Keystone with Redis as the caching layer (the procedures are working fine with the Memcached), the following error is being shown: The information: OS = Ubuntu 20.04 Openstack repo: Ubuntu cloud (Wallaby edition) keystone version 19.0.0 When i issue the "openstack endpoint list --service identity" command, and the output shows the "Internal Server Error (HTTP 500)". ==================================== The following error is the error shown in the /var/log/apache2/keystone.log: here is the link for https://paste.ubuntu.com/p/CmrgZ2JrkX/ the following is the content of the link: 2021-05-01 16:37:17.350115 mod_wsgi (pid=241861): Exception occurred processing WSGI script '/usr/bin/keystone-wsgi-public'. 2021-05-01 16:37:17.358549 Traceback (most recent call last): 2021-05-01 16:37:17.359016 File "/usr/lib/python3/dist-packages/flask/app.py", line 2463, in __call__ 2021-05-01 16:37:17.359044 return self.wsgi_app(environ, start_response) 2021-05-01 16:37:17.359072 File "/usr/lib/python3/dist-packages/werkzeug/middleware/proxy_fix.py", line 232, in __call__ 2021-05-01 16:37:17.359083 return self.app(environ, start_response) 2021-05-01 16:37:17.359104 File "/usr/lib/python3/dist-packages/webob/dec.py", line 129, in __call__ 2021-05-01 16:37:17.359114 resp = self.call_func(req, *args, **kw) 2021-05-01 16:37:17.359134 File "/usr/lib/python3/dist-packages/webob/dec.py", line 193, in call_func 2021-05-01 16:37:17.359144 return self.func(req, *args, **kwargs) 2021-05-01 16:37:17.359165 File "/usr/lib/python3/dist-packages/oslo_middleware/base.py", line 124, in __call__ 2021-05-01 16:37:17.359174 response = req.get_response(self.application) 2021-05-01 16:37:17.359195 File "/usr/lib/python3/dist-packages/webob/request.py", line 1313, in send 2021-05-01 16:37:17.359204 status, headers, app_iter = self.call_application( 2021-05-01 16:37:17.359225 File "/usr/lib/python3/dist-packages/webob/request.py", line 1278, in call_application 2021-05-01 16:37:17.359239 app_iter = application(self.environ, start_response) 2021-05-01 16:37:17.359260 File "/usr/lib/python3/dist-packages/webob/dec.py", line 143, in __call__ 2021-05-01 16:37:17.359271 return resp(environ, start_response) 2021-05-01 16:37:17.359293 File "/usr/lib/python3/dist-packages/webob/dec.py", line 129, in __call__ 2021-05-01 16:37:17.359304 resp = self.call_func(req, *args, **kw) 2021-05-01 16:37:17.359325 File "/usr/lib/python3/dist-packages/webob/dec.py", line 193, in call_func 2021-05-01 16:37:17.359339 return self.func(req, *args, **kwargs) 2021-05-01 16:37:17.359363 File "/usr/lib/python3/dist-packages/oslo_middleware/base.py", line 124, in __call__ 2021-05-01 16:37:17.359373 response = req.get_response(self.application) 2021-05-01 16:37:17.359395 File "/usr/lib/python3/dist-packages/webob/request.py", line 1313, in send 2021-05-01 16:37:17.359406 status, headers, app_iter = self.call_application( 2021-05-01 16:37:17.359429 File "/usr/lib/python3/dist-packages/webob/request.py", line 1278, in call_application 2021-05-01 16:37:17.359439 app_iter = application(self.environ, start_response) 2021-05-01 16:37:17.359460 File "/usr/lib/python3/dist-packages/webob/dec.py", line 129, in __call__ 2021-05-01 16:37:17.359470 resp = self.call_func(req, *args, **kw) 2021-05-01 16:37:17.359492 File "/usr/lib/python3/dist-packages/webob/dec.py", line 193, in call_func 2021-05-01 16:37:17.359502 return self.func(req, *args, **kwargs) 2021-05-01 16:37:17.359523 File "/usr/lib/python3/dist-packages/osprofiler/web.py", line 112, in __call__ 2021-05-01 16:37:17.359534 return request.get_response(self.application) 2021-05-01 16:37:17.359579 File "/usr/lib/python3/dist-packages/webob/request.py", line 1313, in send 2021-05-01 16:37:17.359589 status, headers, app_iter = self.call_application( 2021-05-01 16:37:17.359611 File "/usr/lib/python3/dist-packages/webob/request.py", line 1278, in call_application 2021-05-01 16:37:17.359625 app_iter = application(self.environ, start_response) 2021-05-01 16:37:17.359648 File "/usr/lib/python3/dist-packages/webob/dec.py", line 129, in __call__ 2021-05-01 16:37:17.359658 resp = self.call_func(req, *args, **kw) 2021-05-01 16:37:17.359679 File "/usr/lib/python3/dist-packages/webob/dec.py", line 193, in call_func 2021-05-01 16:37:17.359688 return self.func(req, *args, **kwargs) 2021-05-01 16:37:17.359709 File "/usr/lib/python3/dist-packages/oslo_middleware/request_id.py", line 58, in __call__ 2021-05-01 16:37:17.359722 response = req.get_response(self.application) 2021-05-01 16:37:17.359745 File "/usr/lib/python3/dist-packages/webob/request.py", line 1313, in send 2021-05-01 16:37:17.359754 status, headers, app_iter = self.call_application( 2021-05-01 16:37:17.359776 File "/usr/lib/python3/dist-packages/webob/request.py", line 1278, in call_application 2021-05-01 16:37:17.359785 app_iter = application(self.environ, start_response) 2021-05-01 16:37:17.359806 File "/usr/lib/python3/dist-packages/keystone/server/flask/request_processing/middleware/url_normalize.py", line 38, in __call__ 2021-05-01 16:37:17.359817 return self.app(environ, start_response) 2021-05-01 16:37:17.359839 File "/usr/lib/python3/dist-packages/webob/dec.py", line 129, in __call__ 2021-05-01 16:37:17.359849 resp = self.call_func(req, *args, **kw) 2021-05-01 16:37:17.359871 File "/usr/lib/python3/dist-packages/webob/dec.py", line 193, in call_func 2021-05-01 16:37:17.359880 return self.func(req, *args, **kwargs) 2021-05-01 16:37:17.359902 File "/usr/lib/python3/dist-packages/keystonemiddleware/auth_token/__init__.py", line 341, in __call__ 2021-05-01 16:37:17.359912 response = req.get_response(self._app) 2021-05-01 16:37:17.359933 File "/usr/lib/python3/dist-packages/webob/request.py", line 1313, in send 2021-05-01 16:37:17.359942 status, headers, app_iter = self.call_application( 2021-05-01 16:37:17.359963 File "/usr/lib/python3/dist-packages/webob/request.py", line 1278, in call_application 2021-05-01 16:37:17.359973 app_iter = application(self.environ, start_response) 2021-05-01 16:37:17.359994 File "/usr/lib/python3/dist-packages/werkzeug/middleware/dispatcher.py", line 66, in __call__ 2021-05-01 16:37:17.360004 return app(environ, start_response) 2021-05-01 16:37:17.360026 File "/usr/lib/python3/dist-packages/flask/app.py", line 2449, in wsgi_app 2021-05-01 16:37:17.360036 response = self.handle_exception(e) 2021-05-01 16:37:17.360057 File "/usr/lib/python3/dist-packages/flask_restful/__init__.py", line 272, in error_router 2021-05-01 16:37:17.360066 return original_handler(e) 2021-05-01 16:37:17.360087 File "/usr/lib/python3/dist-packages/flask_restful/__init__.py", line 272, in error_router 2021-05-01 16:37:17.360097 return original_handler(e) 2021-05-01 16:37:17.360118 File "/usr/lib/python3/dist-packages/flask_restful/__init__.py", line 272, in error_router 2021-05-01 16:37:17.360127 return original_handler(e) 2021-05-01 16:37:17.360148 [Previous line repeated 27 more times] 2021-05-01 16:37:17.360171 File "/usr/lib/python3/dist-packages/flask/app.py", line 1866, in handle_exception 2021-05-01 16:37:17.360182 reraise(exc_type, exc_value, tb) 2021-05-01 16:37:17.360203 File "/usr/lib/python3/dist-packages/flask/_compat.py", line 38, in reraise 2021-05-01 16:37:17.360213 raise value.with_traceback(tb) 2021-05-01 16:37:17.360233 File "/usr/lib/python3/dist-packages/flask/app.py", line 2446, in wsgi_app 2021-05-01 16:37:17.360243 response = self.full_dispatch_request() 2021-05-01 16:37:17.360264 File "/usr/lib/python3/dist-packages/flask/app.py", line 1951, in full_dispatch_request 2021-05-01 16:37:17.360284 rv = self.handle_user_exception(e) 2021-05-01 16:37:17.360308 File "/usr/lib/python3/dist-packages/flask_restful/__init__.py", line 272, in error_router 2021-05-01 16:37:17.360317 return original_handler(e) 2021-05-01 16:37:17.360338 File "/usr/lib/python3/dist-packages/flask_restful/__init__.py", line 272, in error_router 2021-05-01 16:37:17.360352 return original_handler(e) 2021-05-01 16:37:17.360375 File "/usr/lib/python3/dist-packages/flask_restful/__init__.py", line 272, in error_router 2021-05-01 16:37:17.360385 return original_handler(e) 2021-05-01 16:37:17.360406 [Previous line repeated 27 more times] 2021-05-01 16:37:17.360429 File "/usr/lib/python3/dist-packages/flask/app.py", line 1820, in handle_user_exception 2021-05-01 16:37:17.360440 reraise(exc_type, exc_value, tb) 2021-05-01 16:37:17.360461 File "/usr/lib/python3/dist-packages/flask/_compat.py", line 38, in reraise 2021-05-01 16:37:17.360471 raise value.with_traceback(tb) 2021-05-01 16:37:17.360493 File "/usr/lib/python3/dist-packages/flask/app.py", line 1949, in full_dispatch_request 2021-05-01 16:37:17.360502 rv = self.dispatch_request() 2021-05-01 16:37:17.360523 File "/usr/lib/python3/dist-packages/flask/app.py", line 1935, in dispatch_request 2021-05-01 16:37:17.360533 return self.view_functions[rule.endpoint](**req.view_args) 2021-05-01 16:37:17.360554 File "/usr/lib/python3/dist-packages/flask_restful/__init__.py", line 468, in wrapper 2021-05-01 16:37:17.360565 resp = resource(*args, **kwargs) 2021-05-01 16:37:17.360586 File "/usr/lib/python3/dist-packages/flask/views.py", line 89, in view 2021-05-01 16:37:17.360596 return self.dispatch_request(*args, **kwargs) 2021-05-01 16:37:17.360617 File "/usr/lib/python3/dist-packages/flask_restful/__init__.py", line 583, in dispatch_request 2021-05-01 16:37:17.360627 resp = meth(*args, **kwargs) 2021-05-01 16:37:17.360648 File "/usr/lib/python3/dist-packages/keystone/server/flask/common.py", line 1064, in wrapper 2021-05-01 16:37:17.360659 return f(*args, **kwargs) 2021-05-01 16:37:17.360680 File "/usr/lib/python3/dist-packages/keystone/api/auth.py", line 315, in post 2021-05-01 16:37:17.360690 token = authentication.authenticate_for_token(auth_data) 2021-05-01 16:37:17.360711 File "/usr/lib/python3/dist-packages/keystone/api/_shared/authentication.py", line 182, in authenticate_for_token 2021-05-01 16:37:17.360721 auth_info = core.AuthInfo.create(auth=auth) 2021-05-01 16:37:17.360743 File "/usr/lib/python3/dist-packages/keystone/auth/core.py", line 126, in create 2021-05-01 16:37:17.360752 auth_info._validate_and_normalize_auth_data(scope_only) 2021-05-01 16:37:17.360773 File "/usr/lib/python3/dist-packages/keystone/auth/core.py", line 340, in _validate_and_normalize_auth_data 2021-05-01 16:37:17.360787 self._validate_and_normalize_scope_data() 2021-05-01 16:37:17.360810 File "/usr/lib/python3/dist-packages/keystone/auth/core.py", line 290, in _validate_and_normalize_scope_data 2021-05-01 16:37:17.360820 project_ref = self._lookup_project(self.auth['scope']['project']) 2021-05-01 16:37:17.360841 File "/usr/lib/python3/dist-packages/keystone/auth/core.py", line 196, in _lookup_project 2021-05-01 16:37:17.360851 domain_ref = self._lookup_domain(project_info['domain']) 2021-05-01 16:37:17.360873 File "/usr/lib/python3/dist-packages/keystone/auth/core.py", line 171, in _lookup_domain 2021-05-01 16:37:17.360882 domain_ref = PROVIDERS.resource_api.get_domain_by_name( 2021-05-01 16:37:17.360903 File "/usr/lib/python3/dist-packages/keystone/common/manager.py", line 115, in wrapped 2021-05-01 16:37:17.360914 __ret_val = __f(*args, **kwargs) 2021-05-01 16:37:17.360935 File "<decorator-gen-28>", line 2, in get_domain_by_name 2021-05-01 16:37:17.360958 File "/usr/lib/python3/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func 2021-05-01 16:37:17.360969 return self.get_or_create( 2021-05-01 16:37:17.360990 File "/usr/lib/python3/dist-packages/dogpile/cache/region.py", line 974, in get_or_create 2021-05-01 16:37:17.361055 key = self.key_mangler(key) 2021-05-01 16:37:17.361083 File "/usr/lib/python3/dist-packages/keystone/common/cache/core.py", line 86, in key_mangler 2021-05-01 16:37:17.361095 key = '%s:%s' % (key, invalidation_manager.region_id) 2021-05-01 16:37:17.361117 File "/usr/lib/python3/dist-packages/keystone/common/cache/core.py", line 43, in region_id 2021-05-01 16:37:17.361126 return self._invalidation_region.get_or_create( 2021-05-01 16:37:17.361148 File "/usr/lib/python3/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create 2021-05-01 16:37:17.361158 with Lock( 2021-05-01 16:37:17.361179 File "/usr/lib/python3/dist-packages/dogpile/lock.py", line 185, in __enter__ 2021-05-01 16:37:17.361189 return self._enter() 2021-05-01 16:37:17.361209 File "/usr/lib/python3/dist-packages/dogpile/lock.py", line 87, in _enter 2021-05-01 16:37:17.361223 value = value_fn() 2021-05-01 16:37:17.361246 File "/usr/lib/python3/dist-packages/dogpile/cache/region.py", line 977, in get_value 2021-05-01 16:37:17.361256 value = self._get_from_backend(key) 2021-05-01 16:37:17.361277 File "/usr/lib/python3/dist-packages/dogpile/cache/region.py", line 1265, in _get_from_backend 2021-05-01 16:37:17.361286 self.backend.get_serialized(key) 2021-05-01 16:37:17.361307 File "/usr/lib/python3/dist-packages/dogpile/cache/api.py", line 288, in get_serialized 2021-05-01 16:37:17.361317 return cast(SerializedReturnType, self.get(key)) 2021-05-01 16:37:17.361338 File "/usr/lib/python3/dist-packages/keystone/common/cache/_context_cache.py", line 74, in get 2021-05-01 16:37:17.361348 value = self.proxied.get(key) 2021-05-01 16:37:17.361369 File "/usr/lib/python3/dist-packages/dogpile/cache/api.py", line 245, in get 2021-05-01 16:37:17.361378 raise NotImplementedError() 2021-05-01 16:37:17.361460 NotImplementedError follwoing is the keystone.conf configuration for both Apache: ======================================================== cat /etc/apache2/sites-enabled/keystone.conf Listen 5000 <VirtualHost *:5000> WSGIScriptAlias / /usr/bin/keystone-wsgi-public WSGIDaemonProcess keystone-public processes=5 threads=1 user=keystone group=keystone display-name=%{GROUP} WSGIProcessGroup keystone-public WSGIApplicationGroup %{GLOBAL} WSGIPassAuthorization On LimitRequestBody 114688 <IfVersion >= 2.4> ErrorLogFormat "%{cu}t %M" </IfVersion> ErrorLog /var/log/apache2/keystone.log CustomLog /var/log/apache2/keystone_access.log combined <Directory /usr/bin> <IfVersion >= 2.4> Require all granted </IfVersion> <IfVersion < 2.4> Order allow,deny Allow from all </IfVersion> </Directory> </VirtualHost> Alias /identity /usr/bin/keystone-wsgi-public <Location /identity> SetHandler wsgi-script Options +ExecCGI WSGIProcessGroup keystone-public WSGIApplicationGroup %{GLOBAL} WSGIPassAuthorization On </Location> ==================================== Content of /etc/keystone/keystone.conf [DEFAULT] log_dir = /var/log/keystone default_log_levels = oslo.cache=DEBUG,dogpile.core.dogpile=DEBUG [application_credential] [assignment] [auth] [cache] debug_cache_backend = True backend = dogpile.cache.redis backend_argument=url:redis://10.10.13.210/6379/0 enabled = true [catalog] [cors] [credential] [database] connection = mysql+pymysql://keystone:[email protected]/keystone [domain_config] [endpoint_filter] [endpoint_policy] [eventlet_server] [extra_headers] Distribution = Ubuntu [federation] [fernet_receipts] [fernet_tokens] [healthcheck] [identity] [identity_mapping] [jwt_tokens] [ldap] [memcache] [oauth1] [oslo_messaging_amqp] [oslo_messaging_kafka] [oslo_messaging_notifications] [oslo_messaging_rabbit] [oslo_middleware] [oslo_policy] [policy] [profiler] [receipt] [resource] [revoke] [role] [saml] [security_compliance] [shadow_users] [token] provider = fernet caching = true driver = keystone.token.persistence.backends.kvs.Token [tokenless_auth] [totp] [trust] [unified_limit] [wsgi] To manage notifications about this bug go to: https://bugs.launchpad.net/keystone/+bug/1926836/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
