Hi,
When testing high load scenarios, e.g. issuing 100 volume attachments, we are 
running into timeout problems between nova, cinder and the centralized storage 
backend.
Has anybody experienced similar problems?
/Tobi


https://blueprints.launchpad.net/nova/+spec/volume-status-polling
https://blueprints.launchpad.net/cinder/+spec/volume-status-polling


From: Tobias Engelbert [mailto:tobias.engelb...@ericsson.com]
Sent: Monday, November 03, 2014 11:33 PM
To: openstack-dev@lists.openstack.org
Subject: [openstack-dev] [cinder][nova] Timeout problems

Hi,
Parallel volume operations lead to inconsistencies between the OpenStack 
database and the deployed view on a centralized storage backend.

When performing multiple volume operation on a centralized storage backend it 
can come to timeouts on the OpenStack side. These timeouts can be the RPC 
timeout or e.g. in high availability scenarios, the HA proxy timeout.
When nova wants to attach a volume, it triggers the status change from 
available to attaching and sends initialize_connection via cinderclient to 
cinder API via the REST API. Cinder API performs a synchronous CALL to cinder 
volume, then via the driver the centralized storage backend is contacted. When 
now a timeout occurs, nova triggers the database to change the volume status 
from attaching to available. Meanwhile the centralized storage backend 
performs, what was originally requested. Here we can have a mismatch between 
database and the real view of the centralized storage backend.

I would be curious if this behavior is also seen by others and would like to 
discuss possible solution.
/Tobi

See also
https://blueprints.launchpad.net/cinder/+spec/volume-status-polling

11<javascript:;>


https://review.openstack.org/#/c/132225/
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Reply via email to