[Openstack-operators] [nova] Deprecation of ComputeFilter
Hi, First, sorry for cross-posting on both dev and operator MLs but I also would like to get operators feedback. So, I was reviewing the scheduler ComputeFilter and I was wondering why the logic should be in a filter. We indeed already have a check on the service information each time that a request is coming in, which is done by HostManager.get_all_host_states() - basically called by FilterScheduler._schedule() Instead, I think it is error-prone to leave that logic in a filter because it can easily be accidentally removed from the list of filters. Besides, the semantics of the filter is not well known and operators could not understand that it is filtering on a Service RPC status, not the real compute node behind it. In order to keep a possibility for operators to explicitely ask the FilterScheduler to also filter on disabled hosts, I propose to add a config option which would be self-explicit. So, I made a quick POC for showing how we could move the logic to HostManager [1]. Feel free to review it and share your thoughts both on the change and here, because I want to make sure that we get a consensus on the removal before really submitting anything. [1] https://review.openstack.org/#/c/162180/ -Sylvain ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
Re: [Openstack-operators] Adding network node (Neutron agents) and test before deploying customer resource
Hi,
It is a good idea that we have such option.
Regarding the option name, I prefer enable_new_agents over start_new_agents
because new agent itself starts and is working even if it is disabled.
Akihiro
2015-03-06 17:35 GMT+09:00 Toshikazu Ichikawa
ichikawa.toshik...@lab.ntt.co.jp:
Hi,
Here is my additional thought.
Nova provides /v2/{tenant_id}/os-services API[3] which is to block
scheduling for a service. Cinder also has /v1/{tenant_id}/os-services
API[4]. When a service (nova-compute or cinder-volume) is disabled, the
service is not selected to accomodate a new VM or volume by scheduler. This
provides same concept as admin_state_up=False of Neutron. (Nova and Cinder
calls a process service, where Neutron calls it agent.)
The enable_new_services of Nova or Cinder is a config option included in
nova.conf or cinder.conf. The user can't change it through API. (I don't
think the user have to change it through API.) The default value is true
which means it enables services by default. I believe it's good choice as
only production environment requires to change it false with additional
setup cost of each service.
Therefore, I believe adding a config option such as start_new_agents
(default: true) to neutron's configuration provides consistent experience
to operators to maintain nodes. The true value of start_new_agents makes
the agent status of admin_state_up true at the addition of agent (this
is current behavior). The false value makes it false for production
environment(new).
[3]
http://developer.openstack.org/api-ref-compute-v2-ext.html#ext-os-services
[4] you can call this one by CLI: cinder service-enable/service-disable
Thanks,
Kazu
From: Toshikazu Ichikawa [mailto:ichikawa.toshik...@lab.ntt.co.jp]
Sent: Thursday, March 05, 2015 10:05 AM
To: openstack-operators@lists.openstack.org
Subject: [Openstack-operators] Adding network node (Neutron agents) and test
before deploying customer resource
Hi,
I'm looking for the way to test a newly added network node by deploying test
resource before any customer resource on the node is deployed. I've learned
in this ML that Nova and Cinder has the setting of enable_new_services in
each conf to disable the initial service status to archive this.
My question is Is there any function/configuration to do same thing for
Neutron?
I know there is on-going bug fix to implement the function to block
scheduling for Neutron agent[1].
As mentioned here [2], this fix may enable only administrators deploy
routers/dhcp-servers for test rather than having customer's one.
However, the initial admin_state_up status of agent still remains true
right after the agent or node is added.
That means it still happens the customer routers/dhcp-servers are deployed
the node before changing the status by manual.
To resolve this, I believe a feature similar to enable_new_services of
Nova/Cinder should be implemented to Neutron to change initial
admin_state_up value.
Do you know any existing function, blueprint or other approach to achieve
same goal?
Or, Is this the feature what you agree to want and should be proposed as a
new blueprint?
I'd like to have neutron operators comments and suggestions.
[1] https://bugs.launchpad.net/neutron/+bug/1408488
[2]
http://lists.openstack.org/pipermail/openstack-dev/2015-January/054007.html
Thanks,
Kazu
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
--
Akihiro Motoki amot...@gmail.com
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
Re: [Openstack-operators] [nova] Deprecation of ComputeFilter
On 03/06/2015 10:43 AM, Jesse Keating wrote: On 3/6/15 10:27 AM, Jay Pipes wrote: As for adding another CONF option, I'm -1 on that. I see no valid reason to schedule workloads to disabled hosts. There may be a better way to skin this cat, but one scenario is we have a host that has alerted, we want to evacuate it and prevent any future builds from going there so we disable it. But after repairing, we want to introduce OUR load to it first to test it before putting CUSTOMER load on it, so we'd like to be able to schedule something to the disabled host. Have you ever done this in practice? One way of doing this would be to enable the host after adding it to a host aggregate that only has your administrative tenant allowed. Then launch an instance specifying some host aggregate extra_spec tag and the launch request will go to that host... Best, -jay ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
Re: [Openstack-operators] [nova] Deprecation of ComputeFilter
On 03/06/2015 07:19 AM, Sylvain Bauza wrote: Hi, First, sorry for cross-posting on both dev and operator MLs but I also would like to get operators feedback. So, I was reviewing the scheduler ComputeFilter and I was wondering why the logic should be in a filter. We indeed already have a check on the service information each time that a request is coming in, which is done by HostManager.get_all_host_states() - basically called by FilterScheduler._schedule() Instead, I think it is error-prone to leave that logic in a filter because it can easily be accidentally removed from the list of filters. Besides, the semantics of the filter is not well known and operators could not understand that it is filtering on a Service RPC status, not the real compute node behind it. In order to keep a possibility for operators to explicitely ask the FilterScheduler to also filter on disabled hosts, I propose to add a config option which would be self-explicit. So, I made a quick POC for showing how we could move the logic to HostManager [1]. Feel free to review it and share your thoughts both on the change and here, because I want to make sure that we get a consensus on the removal before really submitting anything. [1] https://review.openstack.org/#/c/162180/ As mentioned on the bug, I don't think there's a real reason to even deprecate the filter. Just remove it, IMHO. It doesn't do anything at all. As for adding another CONF option, I'm -1 on that. I see no valid reason to schedule workloads to disabled hosts. Best, -jay ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
Re: [Openstack-operators] [nova] Deprecation of ComputeFilter
On 3/6/15 10:27 AM, Jay Pipes wrote: As for adding another CONF option, I'm -1 on that. I see no valid reason to schedule workloads to disabled hosts. There may be a better way to skin this cat, but one scenario is we have a host that has alerted, we want to evacuate it and prevent any future builds from going there so we disable it. But after repairing, we want to introduce OUR load to it first to test it before putting CUSTOMER load on it, so we'd like to be able to schedule something to the disabled host. -- -jlk ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
Re: [Openstack-operators] [nova] Deprecation of ComputeFilter
On 03/06/2015 10:54 AM, Jesse Keating wrote: On 3/6/15 10:48 AM, Jay Pipes wrote: Have you ever done this in practice? One way of doing this would be to enable the host after adding it to a host aggregate that only has your administrative tenant allowed. Then launch an instance specifying some host aggregate extra_spec tag and the launch request will go to that host... At Rackspace scheduling builds against disabled hosts has been done, or I am misremembering. Cool, good to know. Just trying to get my head around the use cases. As I did say, there are probably other ways around it. A host group AZ might just work. Yeah, I think a solution that doesn't rely on a CONF option would be my preference. Allowing administrative override of scheduling decisions entirely is OK, I guess. But I'd almost prefer an ability that simply sidesteps the scheduler altogether and allows the admin to directly launch an instance on a compute node directly without even needing to go through the RESTful tenant API at all. Anyway, something to ponder... Best, -jay ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
Re: [Openstack-operators] Help with glance issue after upgrade from Icehouse to Juno
What about the other glance logfiles? It looks like it may be calling out to a different server and thats failing... Thanks, Kevin From: Nathan Stratton [nat...@robotics.net] Sent: Friday, March 06, 2015 11:42 AM To: openstack-oper. Subject: [Openstack-operators] Help with glance issue after upgrade from Icehouse to Juno Everything is working but glance, I can't even glance image-list. My logs don't look like they are saying anything useful. 2015-03-06 14:39:19.625 3973 INFO glance.registry.client.v1.client [1e5b5ed8-430a-4aec-928b-ebe14573ce42 a7602f2a62f046cda415c2ef3ff0a91c b2b9a5f24d4b48d687efa67efde1dd6d - - -] Registry client request GET /images/detail raised ServerError 2015-03-06 14:39:19.626 3973 INFO glance.wsgi.server [1e5b5ed8-430a-4aec-928b-ebe14573ce42 a7602f2a62f046cda415c2ef3ff0a91c b2b9a5f24d4b48d687efa67efde1dd6d - - -] Traceback (most recent call last): File /usr/lib/python2.7/site-packages/eventlet/wsgi.py, line 433, in handle_one_response result = self.application(self.environ, start_response) File /usr/lib/python2.7/site-packages/webob/dec.py, line 130, in __call__ resp = self.call_func(req, *args, **self.kwargs) File /usr/lib/python2.7/site-packages/webob/dec.py, line 195, in call_func return self.func(req, *args, **kwargs) File /usr/lib/python2.7/site-packages/glance/common/wsgi.py, line 394, in __call__ response = req.get_response(self.application) File /usr/lib/python2.7/site-packages/webob/request.py, line 1296, in send application, catch_exc_info=False) File /usr/lib/python2.7/site-packages/webob/request.py, line 1260, in call_application app_iter = application(self.environ, start_response) File /usr/lib/python2.7/site-packages/webob/dec.py, line 130, in __call__ resp = self.call_func(req, *args, **self.kwargs) File /usr/lib/python2.7/site-packages/webob/dec.py, line 195, in call_func return self.func(req, *args, **kwargs) File /usr/lib/python2.7/site-packages/osprofiler/web.py, line 99, in __call__ return request.get_response(self.application) File /usr/lib/python2.7/site-packages/webob/request.py, line 1296, in send application, catch_exc_info=False) File /usr/lib/python2.7/site-packages/webob/request.py, line 1260, in call_application app_iter = application(self.environ, start_response) File /usr/lib/python2.7/site-packages/keystonemiddleware/auth_token.py, line 748, in __call__ return self._call_app(env, start_response) File /usr/lib/python2.7/site-packages/keystonemiddleware/auth_token.py, line 684, in _call_app return self._app(env, _fake_start_response) File /usr/lib/python2.7/site-packages/webob/dec.py, line 130, in __call__ resp = self.call_func(req, *args, **self.kwargs) File /usr/lib/python2.7/site-packages/webob/dec.py, line 195, in call_func return self.func(req, *args, **kwargs) File /usr/lib/python2.7/site-packages/glance/common/wsgi.py, line 394, in __call__ response = req.get_response(self.application) File /usr/lib/python2.7/site-packages/webob/request.py, line 1296, in send application, catch_exc_info=False) File /usr/lib/python2.7/site-packages/webob/request.py, line 1260, in call_application app_iter = application(self.environ, start_response) File /usr/lib/python2.7/site-packages/paste/urlmap.py, line 203, in __call__ return app(environ, start_response) File /usr/lib/python2.7/site-packages/webob/dec.py, line 144, in __call__ return resp(environ, start_response) File /usr/lib/python2.7/site-packages/routes/middleware.py, line 131, in __call__ response = self.app(environ, start_response) File /usr/lib/python2.7/site-packages/webob/dec.py, line 144, in __call__ return resp(environ, start_response) File /usr/lib/python2.7/site-packages/webob/dec.py, line 130, in __call__ resp = self.call_func(req, *args, **self.kwargs) File /usr/lib/python2.7/site-packages/webob/dec.py, line 195, in call_func return self.func(req, *args, **kwargs) File /usr/lib/python2.7/site-packages/glance/common/wsgi.py, line 683, in __call__ request, **action_args) File /usr/lib/python2.7/site-packages/glance/common/wsgi.py, line 707, in dispatch return method(*args, **kwargs) File /usr/lib/python2.7/site-packages/glance/api/v1/images.py, line 347, in detail images = registry.get_images_detail(req.context, **params) File /usr/lib/python2.7/site-packages/glance/registry/client/v1/api.py, line 150, in get_images_detail return c.get_images_detailed(**kwargs) File /usr/lib/python2.7/site-packages/glance/registry/client/v1/client.py, line 144, in get_images_detailed res = self.do_request(GET, /images/detail, params=params) File /usr/lib/python2.7/site-packages/glance/registry/client/v1/client.py, line 130, in do_request 'exc_name': exc_name}) File /usr/lib/python2.7/site-packages/glance/openstack/common/excutils.py, line 82, in __exit__
[Openstack-operators] Help with glance issue after upgrade from Icehouse to Juno
Everything is working but glance, I can't even glance image-list. My logs don't look like they are saying anything useful. 2015-03-06 14:39:19.625 3973 INFO glance.registry.client.v1.client [1e5b5ed8-430a-4aec-928b-ebe14573ce42 a7602f2a62f046cda415c2ef3ff0a91c b2b9a5f24d4b48d687efa67efde1dd6d - - -] Registry client request GET /images/detail raised ServerError 2015-03-06 14:39:19.626 3973 INFO glance.wsgi.server [1e5b5ed8-430a-4aec-928b-ebe14573ce42 a7602f2a62f046cda415c2ef3ff0a91c b2b9a5f24d4b48d687efa67efde1dd6d - - -] Traceback (most recent call last): File /usr/lib/python2.7/site-packages/eventlet/wsgi.py, line 433, in handle_one_response result = self.application(self.environ, start_response) File /usr/lib/python2.7/site-packages/webob/dec.py, line 130, in __call__ resp = self.call_func(req, *args, **self.kwargs) File /usr/lib/python2.7/site-packages/webob/dec.py, line 195, in call_func return self.func(req, *args, **kwargs) File /usr/lib/python2.7/site-packages/glance/common/wsgi.py, line 394, in __call__ response = req.get_response(self.application) File /usr/lib/python2.7/site-packages/webob/request.py, line 1296, in send application, catch_exc_info=False) File /usr/lib/python2.7/site-packages/webob/request.py, line 1260, in call_application app_iter = application(self.environ, start_response) File /usr/lib/python2.7/site-packages/webob/dec.py, line 130, in __call__ resp = self.call_func(req, *args, **self.kwargs) File /usr/lib/python2.7/site-packages/webob/dec.py, line 195, in call_func return self.func(req, *args, **kwargs) File /usr/lib/python2.7/site-packages/osprofiler/web.py, line 99, in __call__ return request.get_response(self.application) File /usr/lib/python2.7/site-packages/webob/request.py, line 1296, in send application, catch_exc_info=False) File /usr/lib/python2.7/site-packages/webob/request.py, line 1260, in call_application app_iter = application(self.environ, start_response) File /usr/lib/python2.7/site-packages/keystonemiddleware/auth_token.py, line 748, in __call__ return self._call_app(env, start_response) File /usr/lib/python2.7/site-packages/keystonemiddleware/auth_token.py, line 684, in _call_app return self._app(env, _fake_start_response) File /usr/lib/python2.7/site-packages/webob/dec.py, line 130, in __call__ resp = self.call_func(req, *args, **self.kwargs) File /usr/lib/python2.7/site-packages/webob/dec.py, line 195, in call_func return self.func(req, *args, **kwargs) File /usr/lib/python2.7/site-packages/glance/common/wsgi.py, line 394, in __call__ response = req.get_response(self.application) File /usr/lib/python2.7/site-packages/webob/request.py, line 1296, in send application, catch_exc_info=False) File /usr/lib/python2.7/site-packages/webob/request.py, line 1260, in call_application app_iter = application(self.environ, start_response) File /usr/lib/python2.7/site-packages/paste/urlmap.py, line 203, in __call__ return app(environ, start_response) File /usr/lib/python2.7/site-packages/webob/dec.py, line 144, in __call__ return resp(environ, start_response) File /usr/lib/python2.7/site-packages/routes/middleware.py, line 131, in __call__ response = self.app(environ, start_response) File /usr/lib/python2.7/site-packages/webob/dec.py, line 144, in __call__ return resp(environ, start_response) File /usr/lib/python2.7/site-packages/webob/dec.py, line 130, in __call__ resp = self.call_func(req, *args, **self.kwargs) File /usr/lib/python2.7/site-packages/webob/dec.py, line 195, in call_func return self.func(req, *args, **kwargs) File /usr/lib/python2.7/site-packages/glance/common/wsgi.py, line 683, in __call__ request, **action_args) File /usr/lib/python2.7/site-packages/glance/common/wsgi.py, line 707, in dispatch return method(*args, **kwargs) File /usr/lib/python2.7/site-packages/glance/api/v1/images.py, line 347, in detail images = registry.get_images_detail(req.context, **params) File /usr/lib/python2.7/site-packages/glance/registry/client/v1/api.py, line 150, in get_images_detail return c.get_images_detailed(**kwargs) File /usr/lib/python2.7/site-packages/glance/registry/client/v1/client.py, line 144, in get_images_detailed res = self.do_request(GET, /images/detail, params=params) File /usr/lib/python2.7/site-packages/glance/registry/client/v1/client.py, line 130, in do_request 'exc_name': exc_name}) File /usr/lib/python2.7/site-packages/glance/openstack/common/excutils.py, line 82, in __exit__ six.reraise(self.type_, self.value, self.tb) File /usr/lib/python2.7/site-packages/glance/registry/client/v1/client.py, line 115, in do_request **kwargs) File /usr/lib/python2.7/site-packages/glance/common/client.py, line 68, in wrapped return func(self, *args, **kwargs) File /usr/lib/python2.7/site-packages/glance/common/client.py, line 373, in do_request
Re: [Openstack-operators] Help with glance issue after upgrade from Icehouse to Juno
Found it, I updated glance-api.conf with identity_uri =
http://10.71.0.218:35357, but did not also update glance-registry.conf.
nathan stratton | vp technology | broadsoft, inc | +1-240-404-6580 |
www.broadsoft.com
On Fri, Mar 6, 2015 at 3:37 PM, Nathan Stratton nat...@robotics.net wrote:
Sorry about that, never thought to look in registry.log, it points to the
issue. However my config has admin_password={password} what could change in
the upgrade that would now give me access denied?
2015-03-06 14:39:19.623 2688 ERROR glance.registry.api.v1.images
[73bbe61a-d10a-4526-a6a1-1d8a50a56d0e - - - - -] Unable to get images
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images Traceback
(most recent call last):
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/glance/registry/api/v1/images.py, line
122, in _get_images
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
**params)
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/glance/db/sqlalchemy/api.py, line 564,
in image_get_all
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
visibility)
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/glance/db/sqlalchemy/api.py, line 484,
in _select_images_query
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
session = get_session()
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/glance/db/sqlalchemy/api.py, line 97, in
get_session
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
facade = _create_facade_lazily()
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/glance/db/sqlalchemy/api.py, line 82, in
_create_facade_lazily
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
_FACADE = session.EngineFacade.from_config(CONF)
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/oslo/db/sqlalchemy/session.py, line 816,
in from_config
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
retry_interval=conf.database.retry_interval)
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/oslo/db/sqlalchemy/session.py, line 732,
in __init__
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
**engine_kwargs)
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/oslo/db/sqlalchemy/session.py, line 409,
in create_engine
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
_test_connection(engine, max_retries, retry_interval)
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/oslo/db/sqlalchemy/session.py, line 549,
in _test_connection
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
return exc_filters.handle_connect_error(engine)
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/oslo/db/sqlalchemy/exc_filters.py, line
351, in handle_connect_error
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
handler(ctx)
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/oslo/db/sqlalchemy/exc_filters.py, line
323, in handler
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
context.is_disconnect)
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/oslo/db/sqlalchemy/exc_filters.py, line
254, in _raise_operational_errors_directly_filter
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images raise
operational_error
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
OperationalError: (OperationalError) (1045, Access denied for user
'glance'@'localhost' (using password: YES)) None None
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
2015-03-06 14:39:19.624 2688 INFO glance.wsgi.server
[73bbe61a-d10a-4526-a6a1-1d8a50a56d0e - - - - -] Traceback (most recent
call last):
File /usr/lib/python2.7/site-packages/eventlet/wsgi.py, line 433, in
handle_one_response
result = self.application(self.environ, start_response)
File /usr/lib/python2.7/site-packages/webob/dec.py, line 130, in
__call__
resp = self.call_func(req, *args, **self.kwargs)
nathan stratton | vp technology | broadsoft, inc | +1-240-404-6580 |
www.broadsoft.com
On Fri, Mar 6, 2015 at 3:08 PM, Fox, Kevin M kevin@pnnl.gov wrote:
What about the other glance logfiles? It looks like it may be calling
out to a different server and thats failing...
Thanks,
Kevin
--
*From:* Nathan Stratton [nat...@robotics.net]
Re: [Openstack-operators] Help with glance issue after upgrade from Icehouse to Juno
Sorry about that, never thought to look in registry.log, it points to the
issue. However my config has admin_password={password} what could change in
the upgrade that would now give me access denied?
2015-03-06 14:39:19.623 2688 ERROR glance.registry.api.v1.images
[73bbe61a-d10a-4526-a6a1-1d8a50a56d0e - - - - -] Unable to get images
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images Traceback
(most recent call last):
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/glance/registry/api/v1/images.py, line
122, in _get_images
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
**params)
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/glance/db/sqlalchemy/api.py, line 564,
in image_get_all
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
visibility)
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/glance/db/sqlalchemy/api.py, line 484,
in _select_images_query
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
session = get_session()
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/glance/db/sqlalchemy/api.py, line 97, in
get_session
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images facade
= _create_facade_lazily()
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/glance/db/sqlalchemy/api.py, line 82, in
_create_facade_lazily
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
_FACADE = session.EngineFacade.from_config(CONF)
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/oslo/db/sqlalchemy/session.py, line 816,
in from_config
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
retry_interval=conf.database.retry_interval)
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/oslo/db/sqlalchemy/session.py, line 732,
in __init__
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
**engine_kwargs)
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/oslo/db/sqlalchemy/session.py, line 409,
in create_engine
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
_test_connection(engine, max_retries, retry_interval)
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/oslo/db/sqlalchemy/session.py, line 549,
in _test_connection
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images return
exc_filters.handle_connect_error(engine)
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/oslo/db/sqlalchemy/exc_filters.py, line
351, in handle_connect_error
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
handler(ctx)
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/oslo/db/sqlalchemy/exc_filters.py, line
323, in handler
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
context.is_disconnect)
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/oslo/db/sqlalchemy/exc_filters.py, line
254, in _raise_operational_errors_directly_filter
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images raise
operational_error
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
OperationalError: (OperationalError) (1045, Access denied for user
'glance'@'localhost' (using password: YES)) None None
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
2015-03-06 14:39:19.624 2688 INFO glance.wsgi.server
[73bbe61a-d10a-4526-a6a1-1d8a50a56d0e - - - - -] Traceback (most recent
call last):
File /usr/lib/python2.7/site-packages/eventlet/wsgi.py, line 433, in
handle_one_response
result = self.application(self.environ, start_response)
File /usr/lib/python2.7/site-packages/webob/dec.py, line 130, in
__call__
resp = self.call_func(req, *args, **self.kwargs)
nathan stratton | vp technology | broadsoft, inc | +1-240-404-6580 |
www.broadsoft.com
On Fri, Mar 6, 2015 at 3:08 PM, Fox, Kevin M kevin@pnnl.gov wrote:
What about the other glance logfiles? It looks like it may be calling
out to a different server and thats failing...
Thanks,
Kevin
--
*From:* Nathan Stratton [nat...@robotics.net]
*Sent:* Friday, March 06, 2015 11:42 AM
*To:* openstack-oper.
*Subject:* [Openstack-operators] Help with glance issue after upgrade
from Icehouse to Juno
Everything is working but glance, I can't even glance image-list. My
logs don't look like they are saying anything useful.
2015-03-06 14:39:19.625 3973 INFO glance.registry.client.v1.client
Re: [Openstack-operators] Help with glance issue after upgrade from Icehouse to Juno
That looks like a database connection error, not a keystone error. Double
check your DB connection string / credentials and see if you can connect
with the mysql client.
On Fri, Mar 6, 2015 at 3:37 PM, Nathan Stratton nat...@robotics.net wrote:
Sorry about that, never thought to look in registry.log, it points to the
issue. However my config has admin_password={password} what could change in
the upgrade that would now give me access denied?
2015-03-06 14:39:19.623 2688 ERROR glance.registry.api.v1.images
[73bbe61a-d10a-4526-a6a1-1d8a50a56d0e - - - - -] Unable to get images
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images Traceback
(most recent call last):
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/glance/registry/api/v1/images.py, line
122, in _get_images
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
**params)
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/glance/db/sqlalchemy/api.py, line 564,
in image_get_all
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
visibility)
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/glance/db/sqlalchemy/api.py, line 484,
in _select_images_query
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
session = get_session()
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/glance/db/sqlalchemy/api.py, line 97, in
get_session
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
facade = _create_facade_lazily()
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/glance/db/sqlalchemy/api.py, line 82, in
_create_facade_lazily
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
_FACADE = session.EngineFacade.from_config(CONF)
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/oslo/db/sqlalchemy/session.py, line 816,
in from_config
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
retry_interval=conf.database.retry_interval)
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/oslo/db/sqlalchemy/session.py, line 732,
in __init__
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
**engine_kwargs)
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/oslo/db/sqlalchemy/session.py, line 409,
in create_engine
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
_test_connection(engine, max_retries, retry_interval)
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/oslo/db/sqlalchemy/session.py, line 549,
in _test_connection
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
return exc_filters.handle_connect_error(engine)
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/oslo/db/sqlalchemy/exc_filters.py, line
351, in handle_connect_error
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
handler(ctx)
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/oslo/db/sqlalchemy/exc_filters.py, line
323, in handler
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
context.is_disconnect)
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images File
/usr/lib/python2.7/site-packages/oslo/db/sqlalchemy/exc_filters.py, line
254, in _raise_operational_errors_directly_filter
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images raise
operational_error
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
OperationalError: (OperationalError) (1045, Access denied for user
'glance'@'localhost' (using password: YES)) None None
2015-03-06 14:39:19.623 2688 TRACE glance.registry.api.v1.images
2015-03-06 14:39:19.624 2688 INFO glance.wsgi.server
[73bbe61a-d10a-4526-a6a1-1d8a50a56d0e - - - - -] Traceback (most recent
call last):
File /usr/lib/python2.7/site-packages/eventlet/wsgi.py, line 433, in
handle_one_response
result = self.application(self.environ, start_response)
File /usr/lib/python2.7/site-packages/webob/dec.py, line 130, in
__call__
resp = self.call_func(req, *args, **self.kwargs)
nathan stratton | vp technology | broadsoft, inc | +1-240-404-6580 |
www.broadsoft.com
On Fri, Mar 6, 2015 at 3:08 PM, Fox, Kevin M kevin@pnnl.gov wrote:
What about the other glance logfiles? It looks like it may be calling
out to a different server and thats failing...
Thanks,
Kevin
--
*From:* Nathan Stratton [nat...@robotics.net]
*Sent:* Friday, March 06, 2015 11:42 AM
*To:*
Re: [Openstack-operators] [nova] Deprecation of ComputeFilter
(Adding back the -dev ML as it was removed) Le 06/03/2015 20:25, Jay Pipes a écrit : On 03/06/2015 10:54 AM, Jesse Keating wrote: On 3/6/15 10:48 AM, Jay Pipes wrote: Have you ever done this in practice? One way of doing this would be to enable the host after adding it to a host aggregate that only has your administrative tenant allowed. Then launch an instance specifying some host aggregate extra_spec tag and the launch request will go to that host... At Rackspace scheduling builds against disabled hosts has been done, or I am misremembering. Cool, good to know. Just trying to get my head around the use cases. As I did say, there are probably other ways around it. A host group AZ might just work. Yeah, I think a solution that doesn't rely on a CONF option would be my preference. Allowing administrative override of scheduling decisions entirely is OK, I guess. But I'd almost prefer an ability that simply sidesteps the scheduler altogether and allows the admin to directly launch an instance on a compute node directly without even needing to go through the RESTful tenant API at all. Just to be clear, when evacuating or live-migrating VMs by specifying a destination host, it totally overrides the scheduler and doesn't call it, but rather call the Compute Manager directly. In that case, there is no need to keep the ComputeFilter, because it won't never be called anyway. That said, I know there is a pretty old hack by using AZs for specifying a destination host in a boot command and I wonder if it does the same behaviour. If not, it should do exactly like the above, and just ask the compute node directly without querying the Scheduler. The actual only thing when the Scheduler would need to look at non-active nodes would be when waiting to use aggregate extra fields and matching flavor for sending VM requests to a specific set of hosts within an aggregate, where if by removing the inactive nodes, it would just call the active ones. Maybe it's not worth good for leaving the CONF option as Jay mentioned, so I have to admit I'm in favor of removing the possibility to do this. Thoughts ? -Sylvain Anyway, something to ponder... Best, -jay ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
[Openstack-operators] Operators Mid-cycle: Telco Working Group [NFV][Telco]
Operators Team Members, I am also reaching out to you because next week with the assistance of a few others I will be moderating a session named Telco. Those of you who have been attending the weekly IRC meetings will know that we have been drafting an agenda here: https://etherpad.openstack.org/p/PHL-ops-telco As we intend to, as part of the session, walk through the new process for documenting, submitting, reviewing, and updating use cases in gerrit it would streamline things significantly for those who want to try it out as we go if you first ensure that you: * Have a Launchpad account: https://launchpad.net/+login * Have joined the foundation: https://www.openstack.org/join/ * Have confirmed you can sign-on to Gerrit (uses the Launchpad login system): https://review.openstack.org * Those of you who wish to submit new use cases will also need to sign the CLA, this is not required to simply review other use cases though: https://review.openstack.org/#/settings/agreements For more detail on any of these steps please head to http://docs.openstack.org/infra/manual/developers.html#account-setup or jump into #openstack-nfv and we can try and help you out. Thanks in advance, Steve ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
Re: [Openstack-operators] Adding network node (Neutron agents) and test before deploying customer resource
Hi,
Here is my additional thought.
Nova provides /v2/{tenant_id}/os-services API[3] which is to block
scheduling for a service. Cinder also has /v1/{tenant_id}/os-services
API[4]. When a service (nova-compute or cinder-volume) is disabled, the
service is not selected to accomodate a new VM or volume by scheduler. This
provides same concept as admin_state_up=False of Neutron. (Nova and Cinder
calls a process service, where Neutron calls it agent.)
The enable_new_services of Nova or Cinder is a config option included in
nova.conf or cinder.conf. The user can't change it through API. (I don't
think the user have to change it through API.) The default value is true
which means it enables services by default. I believe it's good choice as
only production environment requires to change it false with additional
setup cost of each service.
Therefore, I believe adding a config option such as start_new_agents
(default: true) to neutron's configuration provides consistent experience
to operators to maintain nodes. The true value of start_new_agents makes
the agent status of admin_state_up true at the addition of agent (this
is current behavior). The false value makes it false for production
environment(new).
[3]
http://developer.openstack.org/api-ref-compute-v2-ext.html#ext-os-services
[4] you can call this one by CLI: cinder service-enable/service-disable
Thanks,
Kazu
From: Toshikazu Ichikawa [mailto:ichikawa.toshik...@lab.ntt.co.jp]
Sent: Thursday, March 05, 2015 10:05 AM
To: openstack-operators@lists.openstack.org
Subject: [Openstack-operators] Adding network node (Neutron agents) and test
before deploying customer resource
Hi,
I'm looking for the way to test a newly added network node by deploying test
resource before any customer resource on the node is deployed. I've learned
in this ML that Nova and Cinder has the setting of enable_new_services in
each conf to disable the initial service status to archive this.
My question is Is there any function/configuration to do same thing for
Neutron?
I know there is on-going bug fix to implement the function to block
scheduling for Neutron agent[1].
As mentioned here [2], this fix may enable only administrators deploy
routers/dhcp-servers for test rather than having customer's one.
However, the initial admin_state_up status of agent still remains true
right after the agent or node is added.
That means it still happens the customer routers/dhcp-servers are deployed
the node before changing the status by manual.
To resolve this, I believe a feature similar to enable_new_services of
Nova/Cinder should be implemented to Neutron to change initial
admin_state_up value.
Do you know any existing function, blueprint or other approach to achieve
same goal?
Or, Is this the feature what you agree to want and should be proposed as a
new blueprint?
I'd like to have neutron operators comments and suggestions.
[1] https://bugs.launchpad.net/neutron/+bug/1408488
https://bugs.launchpad.net/neutron/+bug/1408488
[2]
http://lists.openstack.org/pipermail/openstack-dev/2015-January/054007.html
http://lists.openstack.org/pipermail/openstack-dev/2015-January/054007.html
Thanks,
Kazu
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
Re: [Openstack-operators] Adding network node (Neutron agents) and test before deploying customer resource
On 03/06/2015 04:35 PM, Toshikazu Ichikawa wrote: Therefore, I believe adding a config option such as start_new_agents (default: true) to neutron's configuration provides consistent experience to operators to maintain nodes. The true value of start_new_agents makes the agent status of admin_state_up true at the addition of agent (this is current behavior). The false value makes it false for production environment(new). +1 -- 1AE0 322E B8F7 4717 BDEA BF1D 44BB 1BA7 9F6C 6333 ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
