Hi there,

we got a nasty situation yesterday in our OVirt 3.5.6 environment. 
We ran a LSM that failed during the cleanup operation. To be precise 
when the process deleted an image on the source NFS storage. 

Engine log gives:

2016-01-11 20:49:45,120 INFO  
[org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] 
(org.ovirt.thread.pool-8-thread-14) [77277f0] START, 
DeleteImageGroupVDSCommand( storagePoolId = 
94ed7a19-fade-4bd6-83f2-2cbb2f730b95, ignoreFailoverLimit = false, 
storageDomainId = 272ec473-6041-42ee-bd1a-732789dd18d4, imageGroupId = 
aed132ef-703a-44d0-b875-db8c0d2c1a92, postZeros = false, forceDelete = false), 
log id: b52d59c
...
2016-01-11 20:50:45,206 ERROR 
[org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] 
(org.ovirt.thread.pool-8-thread-14) [77277f0] Failed in DeleteImageGroupVDS 
method

VDSM (SPM) log gives:

Thread-97::DEBUG::2016-01-11 
20:49:45,737::fileSD::384::Storage.StorageDomain::(deleteImage) Removing file: 
/rhev/data-center/mnt/1.2.3.4:_var_nas2_OVirtIB/272ec473-6041-42ee-bd1a-732789dd18d4/images/_remojzBd1r/0d623afb-291e-4f4c-acba-caecb125c4ed
...
Thread-97::ERROR::2016-01-11 
20:50:45,737::task::866::Storage.TaskManager.Task::(_setError) 
Task=`cd477878-47b4-44b1-85a3-b5da19543a5e`::Unexpected error
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/task.py", line 873, in _run
    return fn(*args, **kargs)
  File "/usr/share/vdsm/logUtils.py", line 45, in wrapper
    res = f(*args, **kwargs)
  File "/usr/share/vdsm/storage/hsm.py", line 1549, in deleteImage
    pool.deleteImage(dom, imgUUID, volsByImg)
  File "/usr/share/vdsm/storage/securable.py", line 77, in wrapper
    return method(self, *args, **kwargs)
  File "/usr/share/vdsm/storage/sp.py", line 1884, in deleteImage
    domain.deleteImage(domain.sdUUID, imgUUID, volsByImg)
  File "/usr/share/vdsm/storage/fileSD.py", line 385, in deleteImage
    self.oop.os.remove(volPath)
  File "/usr/share/vdsm/storage/outOfProcess.py", line 245, in remove
    self._iop.unlink(path)
  File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 455, in 
unlink
    return self._sendCommand("unlink", {"path": path}, self.timeout)
  File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 385, in 
_sendCommand
    raise Timeout(os.strerror(errno.ETIMEDOUT))
Timeout: Connection timed out

Reading the docs I got the idea that vdsm default 60 second timeout
for IO operations might be changed within /etc/vdsm/vdsm.conf

[irs]
process_pool_timeout = 180

Can anyone confirm that this will solve the problem?

Markus





****************************************************************************
Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte
Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail
irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und
vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte
Weitergabe dieser Mail ist nicht gestattet.

Über das Internet versandte E-Mails können unter fremden Namen erstellt oder
manipuliert werden. Deshalb ist diese als E-Mail verschickte Nachricht keine
rechtsverbindliche Willenserklärung.

Collogia
Unternehmensberatung AG
Ubierring 11
D-50678 Köln

Vorstand:
Kadir Akin
Dr. Michael Höhnerbach

Vorsitzender des Aufsichtsrates:
Hans Kristian Langva

Registergericht: Amtsgericht Köln
Registernummer: HRB 52 497

This e-mail may contain confidential and/or privileged information. If you
are not the intended recipient (or have received this e-mail in error)
please notify the sender immediately and destroy this e-mail. Any
unauthorized copying, disclosure or distribution of the material in this
e-mail is strictly forbidden.

e-mails sent over the internet may have been written under a wrong name or
been manipulated. That is why this message sent as an e-mail is not a
legally binding declaration of intention.

Collogia
Unternehmensberatung AG
Ubierring 11
D-50678 Köln

executive board:
Kadir Akin
Dr. Michael Höhnerbach

President of the supervisory board:
Hans Kristian Langva

Registry office: district court Cologne
Register number: HRB 52 497

****************************************************************************
_______________________________________________
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Reply via email to