Re: [ovirt-users] Hosted Engine/NFS Troubles
On Tue, Jul 18, 2017 at 7:09 AM, Pavel Gashev wrote:
> Phillip,
>
>
>
> The relevant lines from the vdsm logs are the following:
>
>
>
> jsonrpc.Executor/6::INFO::2017-07-17
> 14:24:41,005::logUtils::49::dispatcher::(wrapper)
> Run and protect: connectStorageServer(domType=1,
> spUUID=u'----', conList=[{u'protocol
>
> _version': 3, u'connection': u'192.168.1.21:/srv/ovirt', u'user': u'kvm',
> u'id': u'dbeb8ab4-849f-4728-8ee9-f891bb84ce2f'}], options=None)
>
> jsonrpc.Executor/6::DEBUG::2017-07-17
> 14:24:41,006::fileUtils::209::Storage.fileUtils::(createdir)
> Creating directory: /rhev/data-center/mnt/192.168.1.21:_srv_ovirt mode:
> None
>
> jsonrpc.Executor/6::DEBUG::2017-07-17
> 14:24:41,007::fileUtils::218::Storage.fileUtils::(createdir)
> Using existing directory: /rhev/data-center/mnt/192.168.1.21:_srv_ovirt
>
> jsonrpc.Executor/6::INFO::2017-07-17
> 14:24:41,007::mount::226::storage.Mount::(mount)
> mounting 192.168.1.21:/srv/ovirt at /rhev/data-center/mnt/192.168.
> 1.21:_srv_ovirt
>
> jsonrpc.Executor/6::ERROR::2017-07-17 14:26:46,098::hsm::2403::
> Storage.HSM::(connectStorageServer) Could not connect to storageServer
>
> Traceback (most recent call last):
>
> File "/usr/share/vdsm/storage/hsm.py", line 2400, in
> connectStorageServer
>
> conObj.connect()
>
> File "/usr/share/vdsm/storage/storageServer.py", line 456, in connect
>
> return self._mountCon.connect()
>
> File "/usr/share/vdsm/storage/storageServer.py", line 238, in connect
>
> six.reraise(t, v, tb)
>
> File "/usr/share/vdsm/storage/storageServer.py", line 230, in connect
>
> self._mount.mount(self.options, self._vfsType, cgroup=self.CGROUP)
>
> File "/usr/lib/python2.7/site-packages/vdsm/storage/mount.py", line
> 229, in mount
>
> timeout=timeout, cgroup=cgroup)
>
> File "/usr/lib/python2.7/site-packages/vdsm/supervdsm.py", line 53, in
> __call__
>
> return callMethod()
>
> File "/usr/lib/python2.7/site-packages/vdsm/supervdsm.py", line 51, in
>
>
> **kwargs)
>
> File "", line 2, in mount
>
> File "/usr/lib64/python2.7/multiprocessing/managers.py", line 773, in
> _callmethod
>
> raise convert_to_error(kind, result)
>
> MountError: (32, ';mount.nfs: *Connection timed out*\n')
>
I saw this as well, but I don't understand why I don't have the same
problem when mounting the domain manually and writing to it as the vdsm
user. Thanks for pointing it out, though. I should have included that in
the original message.
>
>
>
>
> *From: * on behalf of Phillip Bailey <
> phbai...@redhat.com>
> *Date: *Tuesday, 18 July 2017 at 13:48
> *To: *Luca 'remix_tj' Lorenzetto
> *Cc: *users
> *Subject: *Re: [ovirt-users] Hosted Engine/NFS Troubles
>
>
>
> On Mon, Jul 17, 2017 at 3:34 PM, Luca 'remix_tj' Lorenzetto <
> lorenzetto.l...@gmail.com> wrote:
>
> On Mon, Jul 17, 2017 at 9:05 PM, Phillip Bailey
> wrote:
> > Hi,
> >
> > I'm having trouble with my hosted engine setup (v4.0) and could use some
> > help. The problem I'm having is that whenever I try to add additional
> hosts
> > to the setup via webadmin, the operation fails due to storage-related
> > issues.
> >
> > webadmin shows the following error messages:
> >
> > "Host cannot access the Storage Domain(s) hosted_storage
> > attached to the Data Center Default. Setting Host state to
> Non-Operational.
> > Failed to connect Host ovirt-node-1 to Storage Pool Default"
> >
>
> Hi Phillip,
>
> your hosted engine storage is on nfs, right? Did you test if you can
> mount manually on each host?
>
> Hi Luca,
>
>
>
> Yes, both storage domains are on NFS (v3) and I am able to successfully
> mount them manually on the hosts.
>
>
>
> Luca
>
>
>
> --
> "E' assurdo impiegare gli uomini di intelligenza eccellente per fare
> calcoli che potrebbero essere affidati a chiunque se si usassero delle
> macchine"
> Gottfried Wilhelm von Leibnitz, Filosofo e Matematico (1646-1716)
>
> "Internet è la più grande biblioteca del mondo.
> Ma il problema è che i libri sono tutti sparsi sul pavimento"
> John Allen Paulos, Matematico (1945-vivente)
>
> Luca 'remix_tj' Lorenzetto, http://www.remixtj.net , <
> lorenzetto.l...@gmail.com>
>
>
>
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Hosted Engine/NFS Troubles
On Tue, Jul 18, 2017 at 7:04 AM, Staniforth, Paul < p.stanifo...@leedsbeckett.ac.uk> wrote: > I used the troubleshooting guide at > > > documentation/how-to/troubleshooting/troubleshooting-nfs-storage-issues/ > > > and exported using > > > (rw,sync,no_subtree_check,all_squash,anonuid=36,anongid=36) > > > This was for the DATA_DOMAIN not hosted storage but it may help. > > > My exports file has the same line, so I don't believe that's the problem. I went through the recommended steps on the troubleshooting guide and everything looks to be fine, but the problem still exists. The troubleshooting guide never came up in my previous googling of the problem though, so thanks for pointing me to that. =) > > Regards, > > > Paul S. > -- > *From:* users-boun...@ovirt.org on behalf of > Phillip Bailey > *Sent:* 17 July 2017 20:05 > *To:* users > *Subject:* [ovirt-users] Hosted Engine/NFS Troubles > > Hi, > > I'm having trouble with my hosted engine setup (v4.0) and could use some > help. The problem I'm having is that whenever I try to add additional > hosts to the setup via webadmin, the operation fails due to storage-related > issues. > > webadmin shows the following error messages: > > "Host cannot access the Storage Domain(s) hosted_storage > attached to the Data Center Default. Setting Host state to Non-Operational. > Failed to connect Host ovirt-node-1 to Storage Pool Default" > > > The VDSM log from the host shows the following error message: > > "Thread-18::ERROR::2017-07-17 13:01:11,483::sdc::146::Storag > e.StorageDomainCache::(_findDomain) domain > ca044720-e5cf-40a8-8b21-57a17026db7c > not found > Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 144, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 174, in _findUnfetchedDomain > raise se.StorageDomainDoesNotExist(sdUUID) > StorageDomainDoesNotExist: Storage domain does not exist: > (u'ca044720-e5cf-40a8-8b21-57a17026db7c',)" > > > The engine log shows the following error messages: > > "2017-07-17 18:32:11,409 ERROR > [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] > (org.ovirt.thread.pool-6-thread-34) [] Domain > 'ca044720-e5cf-40a8-8b21-57a17026db7c:hosted_storage' was reported with > error code '358' > 2017-07-17 18:32:11,410 ERROR [org.ovirt.engine.core.bll.InitVdsOnUpCommand] > (org.ovirt.thread.pool-6-thread-34) [] Storage Domain 'hosted_storage' of > pool 'Default' is in problem in host 'ovirt-node-1' > 2017-07-17 18:32:11,487 ERROR [org.ovirt.engine.core.dal.dbb > roker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-34) > [] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: > Host ovirt-node-1 reports about one of the Active Storage Domains as > Problematic." > > > I have ownership set to vdsm/kvm and full rwx rights enabled on both > directories. I have successfully mounted both the master domain and the > hosted_storage manually on one of the hosts I'm trying to add. I have > attached the engine log and the VDSM log for that host. > > Could someone please help me figure out what's causing this? > > -Phillip Bailey > To view the terms under which this email is distributed, please go to:- > http://disclaimer.leedsbeckett.ac.uk/disclaimer/disclaimer.html > > ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Hosted Engine/NFS Troubles
I used the troubleshooting guide at documentation/how-to/troubleshooting/troubleshooting-nfs-storage-issues/ and exported using (rw,sync,no_subtree_check,all_squash,anonuid=36,anongid=36) This was for the DATA_DOMAIN not hosted storage but it may help. Regards, Paul S. From: users-boun...@ovirt.org on behalf of Phillip Bailey Sent: 17 July 2017 20:05 To: users Subject: [ovirt-users] Hosted Engine/NFS Troubles Hi, I'm having trouble with my hosted engine setup (v4.0) and could use some help. The problem I'm having is that whenever I try to add additional hosts to the setup via webadmin, the operation fails due to storage-related issues. webadmin shows the following error messages: "Host cannot access the Storage Domain(s) hosted_storage attached to the Data Center Default. Setting Host state to Non-Operational. Failed to connect Host ovirt-node-1 to Storage Pool Default" The VDSM log from the host shows the following error message: "Thread-18::ERROR::2017-07-17 13:01:11,483::sdc::146::Storage.StorageDomainCache::(_findDomain) domain ca044720-e5cf-40a8-8b21-57a17026db7c not found Traceback (most recent call last): File "/usr/share/vdsm/storage/sdc.py", line 144, in _findDomain dom = findMethod(sdUUID) File "/usr/share/vdsm/storage/sdc.py", line 174, in _findUnfetchedDomain raise se.StorageDomainDoesNotExist(sdUUID) StorageDomainDoesNotExist: Storage domain does not exist: (u'ca044720-e5cf-40a8-8b21-57a17026db7c',)" The engine log shows the following error messages: "2017-07-17 18:32:11,409 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-34) [] Domain 'ca044720-e5cf-40a8-8b21-57a17026db7c:hosted_storage' was reported with error code '358' 2017-07-17 18:32:11,410 ERROR [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (org.ovirt.thread.pool-6-thread-34) [] Storage Domain 'hosted_storage' of pool 'Default' is in problem in host 'ovirt-node-1' 2017-07-17 18:32:11,487 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-34) [] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host ovirt-node-1 reports about one of the Active Storage Domains as Problematic." I have ownership set to vdsm/kvm and full rwx rights enabled on both directories. I have successfully mounted both the master domain and the hosted_storage manually on one of the hosts I'm trying to add. I have attached the engine log and the VDSM log for that host. Could someone please help me figure out what's causing this? -Phillip Bailey To view the terms under which this email is distributed, please go to:- http://disclaimer.leedsbeckett.ac.uk/disclaimer/disclaimer.html ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Hosted Engine/NFS Troubles
Phillip,
The relevant lines from the vdsm logs are the following:
jsonrpc.Executor/6::INFO::2017-07-17
14:24:41,005::logUtils::49::dispatcher::(wrapper) Run and protect:
connectStorageServer(domType=1, spUUID=u'----',
conList=[{u'protocol
_version': 3, u'connection': u'192.168.1.21:/srv/ovirt', u'user': u'kvm',
u'id': u'dbeb8ab4-849f-4728-8ee9-f891bb84ce2f'}], options=None)
jsonrpc.Executor/6::DEBUG::2017-07-17
14:24:41,006::fileUtils::209::Storage.fileUtils::(createdir) Creating
directory: /rhev/data-center/mnt/192.168.1.21:_srv_ovirt mode: None
jsonrpc.Executor/6::DEBUG::2017-07-17
14:24:41,007::fileUtils::218::Storage.fileUtils::(createdir) Using existing
directory: /rhev/data-center/mnt/192.168.1.21:_srv_ovirt
jsonrpc.Executor/6::INFO::2017-07-17
14:24:41,007::mount::226::storage.Mount::(mount) mounting
192.168.1.21:/srv/ovirt at /rhev/data-center/mnt/192.168.1.21:_srv_ovirt
jsonrpc.Executor/6::ERROR::2017-07-17
14:26:46,098::hsm::2403::Storage.HSM::(connectStorageServer) Could not connect
to storageServer
Traceback (most recent call last):
File "/usr/share/vdsm/storage/hsm.py", line 2400, in connectStorageServer
conObj.connect()
File "/usr/share/vdsm/storage/storageServer.py", line 456, in connect
return self._mountCon.connect()
File "/usr/share/vdsm/storage/storageServer.py", line 238, in connect
six.reraise(t, v, tb)
File "/usr/share/vdsm/storage/storageServer.py", line 230, in connect
self._mount.mount(self.options, self._vfsType, cgroup=self.CGROUP)
File "/usr/lib/python2.7/site-packages/vdsm/storage/mount.py", line 229, in
mount
timeout=timeout, cgroup=cgroup)
File "/usr/lib/python2.7/site-packages/vdsm/supervdsm.py", line 53, in
__call__
return callMethod()
File "/usr/lib/python2.7/site-packages/vdsm/supervdsm.py", line 51, in
**kwargs)
File "", line 2, in mount
File "/usr/lib64/python2.7/multiprocessing/managers.py", line 773, in
_callmethod
raise convert_to_error(kind, result)
MountError: (32, ';mount.nfs: Connection timed out\n')
From: on behalf of Phillip Bailey
Date: Tuesday, 18 July 2017 at 13:48
To: Luca 'remix_tj' Lorenzetto
Cc: users
Subject: Re: [ovirt-users] Hosted Engine/NFS Troubles
On Mon, Jul 17, 2017 at 3:34 PM, Luca 'remix_tj' Lorenzetto
mailto:lorenzetto.l...@gmail.com>> wrote:
On Mon, Jul 17, 2017 at 9:05 PM, Phillip Bailey
mailto:phbai...@redhat.com>> wrote:
> Hi,
>
> I'm having trouble with my hosted engine setup (v4.0) and could use some
> help. The problem I'm having is that whenever I try to add additional hosts
> to the setup via webadmin, the operation fails due to storage-related
> issues.
>
> webadmin shows the following error messages:
>
> "Host cannot access the Storage Domain(s) hosted_storage
> attached to the Data Center Default. Setting Host state to Non-Operational.
> Failed to connect Host ovirt-node-1 to Storage Pool Default"
>
Hi Phillip,
your hosted engine storage is on nfs, right? Did you test if you can
mount manually on each host?
Hi Luca,
Yes, both storage domains are on NFS (v3) and I am able to successfully mount
them manually on the hosts.
Luca
--
"E' assurdo impiegare gli uomini di intelligenza eccellente per fare
calcoli che potrebbero essere affidati a chiunque se si usassero delle
macchine"
Gottfried Wilhelm von Leibnitz, Filosofo e Matematico (1646-1716)
"Internet è la più grande biblioteca del mondo.
Ma il problema è che i libri sono tutti sparsi sul pavimento"
John Allen Paulos, Matematico (1945-vivente)
Luca 'remix_tj' Lorenzetto, http://www.remixtj.net ,
mailto:lorenzetto.l...@gmail.com>>
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Hosted Engine/NFS Troubles
On Mon, Jul 17, 2017 at 3:34 PM, Luca 'remix_tj' Lorenzetto < lorenzetto.l...@gmail.com> wrote: > On Mon, Jul 17, 2017 at 9:05 PM, Phillip Bailey > wrote: > > Hi, > > > > I'm having trouble with my hosted engine setup (v4.0) and could use some > > help. The problem I'm having is that whenever I try to add additional > hosts > > to the setup via webadmin, the operation fails due to storage-related > > issues. > > > > webadmin shows the following error messages: > > > > "Host cannot access the Storage Domain(s) hosted_storage > > attached to the Data Center Default. Setting Host state to > Non-Operational. > > Failed to connect Host ovirt-node-1 to Storage Pool Default" > > > > Hi Phillip, > > your hosted engine storage is on nfs, right? Did you test if you can > mount manually on each host? > > Hi Luca, Yes, both storage domains are on NFS (v3) and I am able to successfully mount them manually on the hosts. Luca > > > > -- > "E' assurdo impiegare gli uomini di intelligenza eccellente per fare > calcoli che potrebbero essere affidati a chiunque se si usassero delle > macchine" > Gottfried Wilhelm von Leibnitz, Filosofo e Matematico (1646-1716) > > "Internet è la più grande biblioteca del mondo. > Ma il problema è che i libri sono tutti sparsi sul pavimento" > John Allen Paulos, Matematico (1945-vivente) > > Luca 'remix_tj' Lorenzetto, http://www.remixtj.net , < > lorenzetto.l...@gmail.com> > ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Hosted Engine/NFS Troubles
On Mon, Jul 17, 2017 at 9:05 PM, Phillip Bailey wrote: > Hi, > > I'm having trouble with my hosted engine setup (v4.0) and could use some > help. The problem I'm having is that whenever I try to add additional hosts > to the setup via webadmin, the operation fails due to storage-related > issues. > > webadmin shows the following error messages: > > "Host cannot access the Storage Domain(s) hosted_storage > attached to the Data Center Default. Setting Host state to Non-Operational. > Failed to connect Host ovirt-node-1 to Storage Pool Default" > Hi Phillip, your hosted engine storage is on nfs, right? Did you test if you can mount manually on each host? Luca -- "E' assurdo impiegare gli uomini di intelligenza eccellente per fare calcoli che potrebbero essere affidati a chiunque se si usassero delle macchine" Gottfried Wilhelm von Leibnitz, Filosofo e Matematico (1646-1716) "Internet è la più grande biblioteca del mondo. Ma il problema è che i libri sono tutti sparsi sul pavimento" John Allen Paulos, Matematico (1945-vivente) Luca 'remix_tj' Lorenzetto, http://www.remixtj.net , ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
