[ovirt-users] RPC call StoragePool.connect failed: Cannot find master domain
Hi all, I'm testing the new oVirt version 4.2.6, with the ovirt-engine installed as a service in a physical host. I tried to recover an engine-backup from a previous ovirt deployment, where the ovirt-engine was self-hosted inside a VM , and it seems that all work properly, because the cluster, hosts, and network configuration was correctly redeployed. The problem was when in the new cluster, I put all the nodes on Maintenance to upgrade it, and when they go back to the UP state, the datacenter was on a Non Responsive Status, because the Master Storage Domain is Inactive, and if I try to reactivate it, I obtain this error message: Failed Activating Storage Domain gpfs_kvm on Data Center BSC-CNS I don't have any hosts in the cluster as a SPM. Due to this situacion, I've some questions: It's possible to force one host to be an SPM? I failed when I put all the nodes on maintenance to upgrade, beacause it's better to maintain one host active as an SPM to protect the cluster, and the storage? Maybe the restored config from a diferent architecture (hosted-engine vs engine installed on host), can affect the Storage? (Previously the Master Storage Domain was on NFS, but I delete it and changed the role of Master to one Posix domain added) At log level on one compute, I can see this events: 2018-09-19 11:40:11,878+0200 INFO (jsonrpc/5) [vdsm.api] FINISH connectStoragePool error=Cannot find master domain: u'spUUID=32168096-b763-11e8-a7aa-000af7b8b6ba, msdUUID=cb99c414-0d93-41b9-9396-d8a607652b49' from=:::10.2.1.101,58538, task_id=eedeba4d-99bc-4e18-8f24-f501b73e0da6 (api:50) 2018-09-19 11:40:11,879+0200 ERROR (jsonrpc/5) [storage.TaskManager.Task] (Task='eedeba4d-99bc-4e18-8f24-f501b73e0da6') Unexpected error (task:875) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run return fn(*args, **kargs) File "", line 2, in connectStoragePool File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in method ret = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 1035, in connectStoragePool spUUID, hostID, msdUUID, masterVersion, domainsMap) File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 1097, in _connectStoragePool res = pool.connect(hostID, msdUUID, masterVersion) File "/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 700, in connect self.__rebuild(msdUUID=msdUUID, masterVersion=masterVersion) File "/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 1274, in __rebuild self.setMasterDomain(msdUUID, masterVersion) File "/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 1495, in setMasterDomain raise se.StoragePoolMasterNotFound(self.spUUID, msdUUID) StoragePoolMasterNotFound: Cannot find master domain: u'spUUID=32168096-b763-11e8-a7aa-000af7b8b6ba, msdUUID=cb99c414-0d93-41b9-9396-d8a607652b49' 2018-09-19 11:40:11,879+0200 INFO (jsonrpc/5) [storage.TaskManager.Task] (Task='eedeba4d-99bc-4e18-8f24-f501b73e0da6') aborting: Task is aborted: "Cannot find master domain: u'spUUID=32168096-b763-11e8-a7aa-000af7b8b6ba, msdUUID=cb99c414-0d93-41b9-9396-d8a607652b49'" - code 304 (task:1181) 2018-09-19 11:40:11,879+0200 ERROR (jsonrpc/5) [storage.Dispatcher] FINISH connectStoragePool error=Cannot find master domain: u'spUUID=32168096-b763-11e8-a7aa-000af7b8b6ba, msdUUID=cb99c414-0d93-41b9-9396-d8a607652b49' (dispatcher:82) 2018-09-19 11:40:11,880+0200 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call StoragePool.connect failed (error 304) in 0.31 seconds (__init__:573) 2018-09-19 11:40:11,907+0200 INFO (jsonrpc/1) [vdsm.api] START getSpmStatus(spUUID=u'32168096-b763-11e8-a7aa-000af7b8b6ba', options=None) from=:::10.2.1.101,58538, task_id=4133fa8c-88f8-44c5-a8b8-18ac6c67771e (api:46) 2018-09-19 11:40:11,907+0200 INFO (jsonrpc/1) [vdsm.api] FINISH getSpmStatus error=Unknown pool id, pool not connected: (u'32168096-b763-11e8-a7aa-000af7b8b6ba',) from=:::10.2.1.101,58538, task_id=4133fa8c-88f8-44c5-a8b8-18ac6c67771e (api:50) 2018-09-19 11:40:11,908+0200 ERROR (jsonrpc/1) [storage.TaskManager.Task] (Task='4133fa8c-88f8-44c5-a8b8-18ac6c67771e') Unexpected error (task:875) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run return fn(*args, **kargs) File "", line 2, in getSpmStatus File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in method ret = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 634, in getSpmStatus pool = self.getPool(spUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 350, in getPool raise se.StoragePoolUnknown(spUUID) StoragePoolUnknown: Unknown pool id, pool not connected: (u'32168096-b763-11e8-a7aa-000af7b8b6ba',) 2018-09-19 11:40:11,908+0200 INFO (jsonrpc/1) [storage.TaskManager.Task]
[ovirt-users] Re: OVIRT 4.2.6: Deploying Errors + Official Repository Problems
Hi all, > > If this happens randomly after the installation you may be suffering some > dns or other network related issue. > This is the key, I had an issue on my DNS configuration that affects the installation procedure when I'm trying to pull new packages from the repos. I consider that this was solved and the thread can be closed. Thanks for your help. Regards! ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/EQAN5NA4KK2IEMZVKY7UM3JKXF6PNFDS/
[ovirt-users] OVIRT 4.2.6: Deploying Errors + Official Repository Problems
Hi all,
We are trying to install the new release of OVIRT 4.2.6 on freshly installed
machines based on RHEL 7.5.
First of all, we encountered a strange error when install the ovirt-release42
package, it adds the necessary public repositories on /etc/yum.repos.d/, but
after do that, sometimes when we try to update or install the other required
packages we saw:
Cannot find a valid baseurl for repo: ovirt-4.2/$releasever
Why cannot configure this path variable automatically or what can influence to
obtain this problem?
In the other hand, when the repos sometimes are well configured and we can
install the packages correctly, we found other problems related to the deploy
of the ovirt-engine (OVA appliance based).
Trying through cockpit, we found this error:
[ ERROR ] fatal: [localhost]: FAILED! => {"changed": true, "cmd":
["virt-install", "-n", "HostedEngineLocal", "--os-variant", "rhel7",
"--virt-type", "kvm", "--memory", "16348", "--vcpus", "4", "--network",
"network=default,mac=00:16:3e:05:c3:66,model=virtio", "--disk",
"/var/tmp/localvmj4Y_f1/images/65f7f081-4d9e-43ae-926f-25807f075f1d/a0a00e73-d3ea-4b9b-bd26-06fe189931f2",
"--import", "--disk", "path=/var/tmp/localvmj4Y_f1/seed.iso,device=cdrom",
"--noautoconsole", "--rng", "/dev/random", "--graphics", "vnc", "--video",
"vga", "--sound", "none", "--controller", "usb,model=none", "--memballoon",
"none", "--boot", "hd,menu=off", "--clock", "kvmclock_present=yes"], "delta":
"0:00:02.122539", "end": "2018-09-06 00:44:18.089510", "msg": "non-zero return
code", "rc": 1, "start": "2018-09-06 00:44:15.966971", "stderr":
"/usr/lib/python2.7/site-packages/requests/__init__.py:91:
RequestsDependencyWarning: urllib3 (1.21.1) or chardet (2.2.1) doesn't match a
supported version!\n RequestsDepen
dencyWarning)\nERROR internal error: qemu unexpectedly closed the monitor:
qemu: could not load PC BIOS 'bios-256k.bin'\nDomain installation does not
appear to have been successful.\nIf it was, you can restart your domain by
running:\n virsh --connect qemu:///system start HostedEngineLocal\notherwise,
please restart your installation.", "stderr_lines":
["/usr/lib/python2.7/site-packages/requests/__init__.py:91:
RequestsDependencyWarning: urllib3 (1.21.1) or chardet (2.2.1) doesn't match a
supported version!", " RequestsDependencyWarning)", "ERROR internal error: qemu
unexpectedly closed the monitor: qemu: could not load PC BIOS 'bios-256k.bin'",
"Domain installation does not appear to have been successful.", "If it was, you
can restart your domain by running:", " virsh --connect qemu:///system start
HostedEngineLocal", "otherwise, please restart your installation."], "stdout":
"\nStarting install...", "stdout_lines": ["", "Starting install..."]}
Seem that it's related to Python packages versions (urllib3 and chardet), but
we wasn't sure about that. How could workaround this problem? Trying to install
this libraries manually?
We tried to found the same error over the Lists and Community, but we don't
found some info about that.
Thank for your help in advance.
Regards!
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/B6UWBUHBOERLWX42GTY4OEMJGDDVNMMV/
[ovirt-users] Re: OVIRT 4.2: MountError on Posix Compliant FS
Hi Sandro, I finally mounted the GPFS Filesystem as a Data Domain using this parameters on the form for New Domain : Domain Function: Data Name: name_of_the_domain Storage Type: Posix Compliant FS Path: gpfs_kvm (the name of the "device"/"filesystem" created, not an absoluthe path) VFS Type: gpfs Mount Options: rw,relatime,dev=gpfs_kvm (the dev parameter is the key to mount the filesystem). Due to a changes implemented on the Spectrum Scale / GPFS last versions, removing the block device on /dev, you must to specify this last parameter to mount correctly the filesystem. Another point is to check the correct privileges and ownership of the directory, to allow Ovirt to mount it correctly, is recommended to mount firts manually the filesystem with mmmount, and change the owner and privileges over the mountpoint. Thanks for your help. I consider that this thread is closed. ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/BJXDHLTLQZNGEUPLJTVH3EPFX2JTFL4K/
[ovirt-users] Re: OVIRT 4.2: MountError on Posix Compliant FS
> On Fri, Jul 6, 2018 at 2:48 PM Sandro Bonazzola wrote:
Hi Nir, thanks for your help.
> You can find vdsm mount command in supervdsmd.log.
I checked the file at: /var/log/vdsm/supervdsm.log, but without any relevant
information about mount or gpfs, including other words like: mnt, posix, domain
or file. If you consider, we can add it too.
-rw-r--r-- 1 root root0 Jul 4 11:58 supervdsm.log
> Did you look in /var/log/messages or demsg as the error message suggests?
As I commented before, i can't find any relevant messages on the dmesg output
when I tried to add a new storage domain. The only reports that I can saw, was
on the vdsm.log
2018-07-06 09:00:54,861+0200 INFO (jsonrpc/3) [vdsm.api] START
connectStorageServer(domType=6, spUUID=u'----',
conList=[{u'mnt_options': u'loop,bind', u'id':
u'----', u'connection': u'/gpfs/kvm', u'iqn':
u'', u'user': u'', u'tpgt': u'1', u'vfs_type': u'gpfs', u'password':
'', u'port': u''}], options=None) from=:::10.2.1.254,34696,
flow_id=6bd725cf-a982-4d2a-a3f5-edbcc5f39d64,
task_id=570791fc-46f7-4900-a1bc-834670988259 (api:46)
2018-07-06 09:00:54,864+0200 INFO (jsonrpc/3)
[storage.StorageServer.MountConnection] Creating directory
u'/rhev/data-center/mnt/_gpfs_kvm' (storageServer:167)
2018-07-06 09:00:54,864+0200 INFO (jsonrpc/3) [storage.fileUtils] Creating
directory: /rhev/data-center/mnt/_gpfs_kvm mode: None (fileUtils:197)
2018-07-06 09:00:54,864+0200 INFO (jsonrpc/3) [storage.Mount] mounting
/gpfs/kvm at /rhev/data-center/mnt/_gpfs_kvm (mount:204)
OSError: [Errno 2] Mount of `/gpfs/kvm` at `/rhev/data-center/mnt/_gpfs_kvm`
does not exist
>
> We support POSIX compliant file systems. If GPFS is POSIX compliant, it
> should
> work, but I don't think we test it.
> Elad, do we test GPFS?
>
> You should start with mounting the file system manually. When you have a
> working
> mount command, you may need to add some mount options to the storage domain
> additional mount options.
Apparently, on some documentation I read that GPFS is fully supported with
Ovirt and RHEV environments, so I consider that GPFS is a POSIX compliant
filesystem. In addition, when I configured the fs, I defined the type POSIX for
two parameters during the creation:
-D posixFile locking semantics in effect
-k posixACL semantics in effect
On the ovirt Add Storage Domain Form, I put on Custom Connection Parameters,
the flags that I encountered on the mount entry for the two GPFS file systems:
rw,relatime
I'm looking for a workaround to try to add this fs on the environment. If you
consider that I can collaborate checking other confs or files, I'm ready to do
that.
Kind regards and thanks in advance!
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/KCJHSPRP42KUKVZPHAJYBHJM42BWLDNE/
[ovirt-users] Re: OVIRT 4.2: MountError on Posix Compliant FS
> 2018-07-06 11:26 GMT+02:00 Hi, welcome to oVirt community! Hi Sandro, thanks in advance for your help, > Adding Nir and Tal for all your questions. > In the meanwhile, can you please provide a sos report from the host? Did > dmesg provide any useful information? I search on the dmesg output without results, I can't find relevant messages at the moment when I try to add the new storage domain or later. As you say, I attached a sosreport on this webpage (due to limits on the mailserver), maybe it's useful to find the cause of the problem. If you need more logs, confs, etc, I can add it without problems because we are on a testing environment. SOSREPORT - https://jirafeau.net/f.php?h=3MT_XlnI=1 Kind regards! ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/QQ2IYIQFDMEEUNQNMCQ2FFUQDZUSDJ5I/
[ovirt-users] OVIRT 4.2: MountError on Posix Compliant FS
Hi all,
I've deployed the ovirt Version 4.2.4.5-1.el7 on a small cluster and I'm trying
to use as datastorage domain a configured Spectrum Scale (GPFS) distributed
filesystem for test it.
I've completed the configuration of the storage, and the filesystem defined are
correctly mounted on the client hosts as we can see:
gpfs_kvmgpfs 233T 288M 233T 1% /gpfs/kvm
gpfs_fast gpfs 8.8T 5.2G 8.8T 1% /gpfs/fast
The content output of the mount comand is:
gpfs_kvm on /gpfs/kvm type gpfs (rw,relatime)
gpfs_fast on /gpfs/fast type gpfs (rw,relatime)
I can write and read to the mounted filesystem correctly, but when I try to add
the local mounted filesystem I encountered some errors related to the mounting
process of the storage.
The parameters passed to add new storage domain are:
Data Center: Default (V4)
Name: gpfs_kvm
Description: VM data on GPFS
Domain Function: Data
Storage Type: Posix Compliant FS
Host to Use: kvm1c01
Path: /gpfs/kvm
VFS Type: gpfs
Mount Options: rw, relatime
The ovirt error that I obtained is:
Error while executing action Add Storage Connection: Problem while trying
to mount target
On the /var/log/vdsm/vdsm.log I can see:
2018-07-05 09:07:43,400+0200 INFO (jsonrpc/0) [vdsm.api] START
connectStorageServer(domType=6, spUUID=u'----',
conList=[{u'mnt_options': u'rw,relatime', u'id':
u'----', u'connection': u'/gpfs/kvm', u'iqn':
u'', u'user': u'', u'tpgt': u'1', u'vfs_type': u'gpfs', u'password':
'', u'port': u''}], options=None) from=:::10.2.1.254,43908,
flow_id=498668a0-a240-469f-ac33-f8c7bdeb481f,
task_id=857460ed-5f0e-4bc6-ba54-e8f2c72e9ac2 (api:46)
2018-07-05 09:07:43,403+0200 INFO (jsonrpc/0)
[storage.StorageServer.MountConnection] Creating directory
u'/rhev/data-center/mnt/_gpfs_kvm' (storageServer:167)
2018-07-05 09:07:43,404+0200 INFO (jsonrpc/0) [storage.fileUtils] Creating
directory: /rhev/data-center/mnt/_gpfs_kvm mode: None (fileUtils:197)
2018-07-05 09:07:43,404+0200 INFO (jsonrpc/0) [storage.Mount] mounting
/gpfs/kvm at /rhev/data-center/mnt/_gpfs_kvm (mount:204)
MountError: (32, ';mount: wrong fs type, bad option, bad superblock on
/gpfs/kvm,\n missing codepage or helper program, or other error\n\n
In some cases useful info is found in syslog - try\n dmesg | tail or
so.\n')
I know, that from previous versions on the GPFS implementation, they removed
the device on /dev, due to incompatibilities with systemd. I don't know if this
change affect the ovirt mounting process.
Can you help me to add this filesystem to the ovirt environment?
The parameters that I used previously are ok? Or i need to do some modification?
Its possible that the process fails because i dont have a device related to the
gpfs filesystems on /dev?
Can we apply some kind of workaround to mount manually the filesystem to the
ovirt environment? Ex. create the dir /rhev/data-center/mnt/_gpfs_kvm manually
and then mount the /gpfs/kvm over this?
It's posible to modify the code to bypass some comprobations or something?
Reading the available documentation over Internet I find that ovirt was
compatible with this fs (gpfs) implementation, because it's POSIX Compliant,
this is a main reason for test it in our cluster.
It remains compatible on the actual versions? Or maybe there are changes that
brokes this integration?
Many thanks for all in advance!
Kind regards!
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/LLXLSI4ZQFU32PV7AXLYD5LJBS23HBKO/
