Re: [ovirt-users] can not use iscsi storage type onovirtandGlusterfshyper-convergedenvironment
Hi Nir : before change code ,supervdsm report error log like those : === --> MainProcess|jsonrpc.Executor/2::DEBUG::2016-12-02 17:12:16,372::supervdsmServer::92::SuperVdsm.ServerCallback::(wrapper) call getPathsStatus with () {} MainProcess|jsonrpc.Executor/2::DEBUG::2016-12-02 17:12:16,373::devicemapper::154::Storage.Misc.excCmd::(_getPathsStatus) /usr/bin/taskset --cpu-list 0-7 /usr/sbin/dmsetup status (cwd None) MainProcess|jsonrpc.Executor/2::DEBUG::2016-12-02 17:12:16,377::devicemapper::154::Storage.Misc.excCmd::(_getPathsStatus) SUCCESS: = ''; = 0 MainProcess|jsonrpc.Executor/2::ERROR::2016-12-02 17:12:16,378::supervdsmServer::96::SuperVdsm.ServerCallback::(wrapper) Error in getPathsStatus === < my environment have other dm device , dmsetup status will show like this as follow, so will occure above error ! > flash_sdd: 0 976771072 flashcache stats: reads(5723182), writes(47879823) read hits(2938125), read hit percent(51) write hits(12617126) write hit percent(26) dirty write hits(4592227) dirty write hit percent(9) replacement(701168), write replacement(3189334) write invalidates(0), read invalidates(1) <-- use your patch to test , add or import iscsi type storage domain have no problem , log like follow : == -> MainProcess|jsonrpc.Executor/1::DEBUG::2016-12-06 08:58:07,439::supervdsmServer::92::SuperVdsm.ServerCallback::(wrapper) call getPathsStatus with () {} MainProcess|jsonrpc.Executor/1::DEBUG::2016-12-06 08:58:07,439::devicemapper::155::Storage.Misc.excCmd::(_getPathsStatus) /usr/bin/taskset --cpu-list 0-7 /usr/sbin/dmsetup status --target multipath (cwd None) MainProcess|jsonrpc.Executor/1::DEBUG::2016-12-06 08:58:07,443::devicemapper::155::Storage.Misc.excCmd::(_getPathsStatus) SUCCESS: = ''; = 0 = Regards, humaorong -- Original -- From: "Nir Soffer"; Date: Tue, Dec 6, 2016 01:41 AM To: "胡茂荣"; Cc: "users"; "Jeff Nelson"; "胡晓宇"; Subject: Re: [ovirt-users] can not use iscsi storage type onovirtandGlusterfshyper-convergedenvironment Hi 胡茂荣, Can you test this patch? https://gerrit.ovirt.org/67844 I also need more information on your setup, to add more details to the commit message. Thanks for reporting this, Nir On Mon, Dec 5, 2016 at 10:28 AM, 胡茂荣 wrote: Thanks for Yaniv Kaul , change code need build vdsm source code , and if only change /usr/share/vdsm/storage/devicemapper.py will not really take effect . could this problem as a bug , and correct it to ovirt vdsm source code ? -- Original -- From: "Yaniv Kaul"; Date: Sun, Dec 4, 2016 07:07 PM To: "胡茂荣"; Cc: "胡晓宇"; "users"; "Sahina Bose"; "Jeff Nelson"; Subject: Re: [ovirt-users] can not use iscsi storage type onovirtandGlusterfshyper-converged environment On Dec 2, 2016 11:53 AM, "胡茂荣" wrote: I find supervdsm used " /usr/sbin/dmsetup status" : MainProcess|jsonrpc.Executor/2::DEBUG::2016-12-02 17:12:16,372::supervdsmServer::92::SuperVdsm.ServerCallback::(wrapper) call getPathsStatus with () {} MainProcess|jsonrpc.Executor/2::DEBUG::2016-12-02 17:12:16,373::devicemapper::154::Storage.Misc.excCmd::(_getPathsStatus) /usr/bin/taskset --cpu-list 0-7 /usr/sbin/dmsetup status (cwd None) MainProcess|jsonrpc.Executor/2::DEBUG::2016-12-02 17:12:16,377::devicemapper::154::Storage.Misc.excCmd::(_getPathsStatus) SUCCESS: = ''; = 0 MainProcess|jsonrpc.Executor/2::ERROR::2016-12-02 17:12:16,378::supervdsmServer::96::SuperVdsm.ServerCallback::(wrapper) Error in getPathsStatus problem : how can I change Storage.Misc.excCmd _getPathsStatus " /usr/bin/taskset --cpu-list 0-7 /usr/sbin/dmsetup status" to : /usr/bin/taskset --cpu-list 0-7 /usr/sbin/dmsetup status --target multipah I think add iscsi type storage , if supervdsm scan mulitpah will solve my problem .(my environment have other dm devices, use "dmsetup status" will show them, and vdsm get dm path status will occur error ) so I changed some as follow : (1) I define EXT_DMSETUP_STATUS in /usr/lib/python2.7/site-packages/vdsm/constants.py : /usr/lib/python2.7/site-packages/vdsm/constants.py:EXT_DMSETUP = '/usr/sbin/dmsetup' /usr/lib/python2.7/site-packages/vdsm/constants.py:EXT_
Re: [ovirt-users] can not use iscsi storage type onovirtandGlusterfshyper-converged environment
Thanks for Yaniv Kaul , change code need build vdsm source code , and if only change /usr/share/vdsm/storage/devicemapper.py will not really take effect . could this problem as a bug , and correct it to ovirt vdsm source code ? -- Original -- From: "Yaniv Kaul"; Date: Sun, Dec 4, 2016 07:07 PM To: "胡茂荣"; Cc: "胡晓宇"; "users"; "Sahina Bose"; "Jeff Nelson"; Subject: Re: [ovirt-users] can not use iscsi storage type onovirtandGlusterfshyper-converged environment On Dec 2, 2016 11:53 AM, "胡茂荣" wrote: I find supervdsm used " /usr/sbin/dmsetup status" : MainProcess|jsonrpc.Executor/2::DEBUG::2016-12-02 17:12:16,372::supervdsmServer::92::SuperVdsm.ServerCallback::(wrapper) call getPathsStatus with () {} MainProcess|jsonrpc.Executor/2::DEBUG::2016-12-02 17:12:16,373::devicemapper::154::Storage.Misc.excCmd::(_getPathsStatus) /usr/bin/taskset --cpu-list 0-7 /usr/sbin/dmsetup status (cwd None) MainProcess|jsonrpc.Executor/2::DEBUG::2016-12-02 17:12:16,377::devicemapper::154::Storage.Misc.excCmd::(_getPathsStatus) SUCCESS: = ''; = 0 MainProcess|jsonrpc.Executor/2::ERROR::2016-12-02 17:12:16,378::supervdsmServer::96::SuperVdsm.ServerCallback::(wrapper) Error in getPathsStatus problem : how can I change Storage.Misc.excCmd _getPathsStatus " /usr/bin/taskset --cpu-list 0-7 /usr/sbin/dmsetup status" to : /usr/bin/taskset --cpu-list 0-7 /usr/sbin/dmsetup status --target multipah I think add iscsi type storage , if supervdsm scan mulitpah will solve my problem .(my environment have other dm devices, use "dmsetup status" will show them, and vdsm get dm path status will occur error ) so I changed some as follow : (1) I define EXT_DMSETUP_STATUS in /usr/lib/python2.7/site-packages/vdsm/constants.py : /usr/lib/python2.7/site-packages/vdsm/constants.py:EXT_DMSETUP = '/usr/sbin/dmsetup' /usr/lib/python2.7/site-packages/vdsm/constants.py:EXT_DMSETUP_STATUS = "/usr/sbin/dmsetup status --target multipath" (2) /usr/share/vdsm/storage/devicemapper.py add : from vdsm.constants import EXT_DMSETUP_STATUS and changed getPathsStatus cmd to " EXT_DMSETUP_STATUS" : def _getPathsStatus(): cmd = [EXT_DMSETUP_STATUS]# before : cmd=[EXT_DMSETUP,"status"] Why not change this to: cmd = [EXT_DMSETUP, "status", "--target", "multipath"] Y. rc, out, err = misc.execCmd(cmd) === but log in supervdsm log also not change . Please help me ,how to change code to let supervdsm exec "/usr/sbin/dmsetup status --target multipath" in function getPathsStatus() 。 -- Original -- From: "胡茂荣"; Date: Fri, Nov 25, 2016 05:44 PM To: "Sahina Bose"; Cc: "Maor Lipchuk"; "Jeff Nelson"; "users"; Subject: Re: [ovirt-users] can not use iscsi storage type on ovirtandGlusterfshyper-converged environment ===--- ###vdsm or supervdsm log report : MainProcess|jsonrpc.Executor/7::ERROR::2016-11-01 11:07:00,178::supervdsmServer::96::SuperVdsm.ServerCallback::(wrapper) Error in getPathsStatus MainProcess|jsonrpc.Executor/4::ERROR::2016-11-01 11:07:20,964::supervdsmServer::96::SuperVdsm.ServerCallback::(wrapper) Error in getPathsStatus some code info > [root@horeba storage]# pwd /usr/share/vdsm/storage [root@horeba storage]# grep "getPathsStatus" -R ./ ./devicemapper.py:def _getPathsStatus(): ./devicemapper.py:def getPathsStatus(): ./devicemapper.py:return getProxy().getPathsStatus() ./multipath.py:pathStatuses = devicemapper.getPathsStatus() def _getPathsStatus(): cmd = [EXT_DMSETUP, "status"] rc, out, err = misc.execCmd(cmd) if rc != 0: raise Exception("Could not get device statuses") res = {} for statusLine in out: try: devName, statusLine = statusLine.split(":", 1) except ValueError: if len(out) == 1: # return an empty dict when status output is: No devices found return res else: raise for m in PATH_STATUS_RE.finditer(statusLine): devNum, status = m.groups() physdevName = findDev(*[int(i) for i in devNum.split(":")]) res[physdevName] = {"A": "active", "F": "failed"}[status] return res def getPathsStatus(): return getProxy().getPathsStatus() ==
Re: [ovirt-users] can not use iscsi storage type on ovirtandGlusterfshyper-converged environment
I find supervdsm used " /usr/sbin/dmsetup status" : MainProcess|jsonrpc.Executor/2::DEBUG::2016-12-02 17:12:16,372::supervdsmServer::92::SuperVdsm.ServerCallback::(wrapper) call getPathsStatus with () {} MainProcess|jsonrpc.Executor/2::DEBUG::2016-12-02 17:12:16,373::devicemapper::154::Storage.Misc.excCmd::(_getPathsStatus) /usr/bin/taskset --cpu-list 0-7 /usr/sbin/dmsetup status (cwd None) MainProcess|jsonrpc.Executor/2::DEBUG::2016-12-02 17:12:16,377::devicemapper::154::Storage.Misc.excCmd::(_getPathsStatus) SUCCESS: = ''; = 0 MainProcess|jsonrpc.Executor/2::ERROR::2016-12-02 17:12:16,378::supervdsmServer::96::SuperVdsm.ServerCallback::(wrapper) Error in getPathsStatus problem : how can I change Storage.Misc.excCmd _getPathsStatus " /usr/bin/taskset --cpu-list 0-7 /usr/sbin/dmsetup status" to : /usr/bin/taskset --cpu-list 0-7 /usr/sbin/dmsetup status --target multipah I think add iscsi type storage , if supervdsm scan mulitpah will solve my problem .(my environment have other dm devices, use "dmsetup status" will show them, and vdsm get dm path status will occur error ) so I changed some as follow : (1) I define EXT_DMSETUP_STATUS in /usr/lib/python2.7/site-packages/vdsm/constants.py : /usr/lib/python2.7/site-packages/vdsm/constants.py:EXT_DMSETUP = '/usr/sbin/dmsetup' /usr/lib/python2.7/site-packages/vdsm/constants.py:EXT_DMSETUP_STATUS = "/usr/sbin/dmsetup status --target multipath" (2) /usr/share/vdsm/storage/devicemapper.py add : from vdsm.constants import EXT_DMSETUP_STATUS and changed getPathsStatus cmd to " EXT_DMSETUP_STATUS" : def _getPathsStatus(): cmd = [EXT_DMSETUP_STATUS]# before : cmd=[EXT_DMSETUP,"status"] rc, out, err = misc.execCmd(cmd) === but log in supervdsm log also not change . Please help me ,how to change code to let supervdsm exec "/usr/sbin/dmsetup status --target multipath" in function getPathsStatus() 。 -- Original -- From: "胡茂荣"; Date: Fri, Nov 25, 2016 05:44 PM To: "Sahina Bose"; Cc: "Maor Lipchuk"; "Jeff Nelson"; "users"; Subject: Re: [ovirt-users] can not use iscsi storage type on ovirtandGlusterfshyper-converged environment ===--- ###vdsm or supervdsm log report : MainProcess|jsonrpc.Executor/7::ERROR::2016-11-01 11:07:00,178::supervdsmServer::96::SuperVdsm.ServerCallback::(wrapper) Error in getPathsStatus MainProcess|jsonrpc.Executor/4::ERROR::2016-11-01 11:07:20,964::supervdsmServer::96::SuperVdsm.ServerCallback::(wrapper) Error in getPathsStatus some code info > [root@horeba storage]# pwd /usr/share/vdsm/storage [root@horeba storage]# grep "getPathsStatus" -R ./ ./devicemapper.py:def _getPathsStatus(): ./devicemapper.py:def getPathsStatus(): ./devicemapper.py:return getProxy().getPathsStatus() ./multipath.py:pathStatuses = devicemapper.getPathsStatus() def _getPathsStatus(): cmd = [EXT_DMSETUP, "status"] rc, out, err = misc.execCmd(cmd) if rc != 0: raise Exception("Could not get device statuses") res = {} for statusLine in out: try: devName, statusLine = statusLine.split(":", 1) except ValueError: if len(out) == 1: # return an empty dict when status output is: No devices found return res else: raise for m in PATH_STATUS_RE.finditer(statusLine): devNum, status = m.groups() physdevName = findDev(*[int(i) for i in devNum.split(":")]) res[physdevName] = {"A": "active", "F": "failed"}[status] return res def getPathsStatus(): return getProxy().getPathsStatus() = and flashcache dm device will error when use getPathsStatus() function . could change code not check flashcache dm device ? dmsetup info ---> [root@horebc ~]# dmsetup status flash_sdb: 0 976771072 flashcache stats: reads(1388761), writes(15548965) read hits(1235671), read hit percent(88) write hits(6539144) write hit percent(42) dirty write hits(21372) dirty write hit percent(0) replacement(147711), write replacement(524881) write invalidates(0), read invalidates(1) pending enqueues(810), pending inval(810) metadata dirties(15196370), metadata cleans(15196322) metadata batch(30087377) metadata ssd writes(305315)
Re: [ovirt-users] can not use iscsi storage type on ovirtandGlusterfshyper-converged environment
FaWvxa9wbHy7WrfiXNsP4F2J3gg0 flash_sdd 253 1 L--w11 0 and flashcache dm have no UUID ,I think it can export it before checking . humaorong 2016-11-25 -- Original -- From: "胡茂荣"; Date: Fri, Nov 25, 2016 01:18 PM To: "Sahina Bose"; Cc: "Maor Lipchuk"; "Jeff Nelson"; "users"; Subject: Re: [ovirt-users] can not use iscsi storage type on ovirtandGlusterfshyper-converged environment I find this problem more info : I use flashcache on ovirt hosts , have dm device in /dev/mapper/ : [root@horeba init.d]# dmsetup info -C (and I set them in multipath blacklist ) Name Maj Min Stat Open Targ Event UUID flash_sdb253 0 L--w01 0 flash_sdf253 2 L--w01 0 flash_sdd253 1 L--w01 0 [root@horeba init.d]# multipath -l [root@horeba init.d]# [root@horeba init.d]# ll /dev/mapper/ total 0 crw--- 1 root root 10, 236 Nov 25 10:09 control lrwxrwxrwx 1 root root 7 Nov 25 12:51 flash_sdb -> ../dm-0 lrwxrwxrwx 1 root root 7 Nov 25 12:51 flash_sdd -> ../dm-1 lrwxrwxrwx 1 root root 7 Nov 25 12:51 flash_sdf -> ../dm-2 on this condition , ovirt UI add iscsi type storage will fail . If I delete the flashcache device /dev/mapper/flash_* , ovirt UI add iscsi type storage have no problem . I need flashcache for using ssd cache on my environment , how can I use iscsi type storage on this environment , please help me , Thanks! -- Original -- From: "Sahina Bose"; Date: Thu, Nov 3, 2016 05:54 PM To: "胡茂荣"; Cc: "Maor Lipchuk"; "Jeff Nelson"; "users"; Subject: Re: [ovirt-users] can not use iscsi storage type on ovirtandGlusterfshyper-converged environment A wild guess, not sure if it is related - can you check if multipathd service is enabled. If you set up your oVirt-Gluster hyperconverged environment via gdeploy, multipathd service is disabled and the /etc/multipath.conf is edited to blacklist all devices - this was to fix https://bugzilla.redhat.com/show_bug.cgi?id=1303940 Since you mention you're unable to add iscsi storage only in this environment, thought it's worth checking. On Thu, Nov 3, 2016 at 6:40 AM, 胡茂荣 wrote: my environment rpm are : [root@horeba ~]# rpm -q vdsm vdsm-4.18.13-1.el7.centos.x86_64 [root@horeba ~]# rpm -aq | grep ovirt ovirt-vmconsole-host-1.0.4-1.el7.centos.noarch ovirt-imageio-common-0.4.0-1.el7.noarch ovirt-hosted-engine-setup-2.0.2.2-1.el7.centos.noarch ovirt-imageio-daemon-0.4.0-1.el7.noarch ovirt-engine-appliance-4.0-20160928.1.el7.centos.noarch ovirt-vmconsole-1.0.4-1.el7.centos.noarch ovirt-host-deploy-1.5.2-1.el7.centos.noarch ovirt-hosted-engine-ha-2.0.4-1.el7.centos.noarch ovirt-release40-4.0.4-1.noarch ovirt-setup-lib-1.0.2-1.el7.centos.noarch ovirt-engine-sdk-python-3.6.9.1-1.el7.centos.noarch and I test if not on ' ovirt and Glusterfs hyper-converged environment ' , ovirt WEB UI add iscsi storage work ok . -- Original -- From: "Maor Lipchuk"; Date: Wed, Nov 2, 2016 07:37 PM To: "胡茂荣"; Cc: "users"; "Jeff Nelson"; "Nir Soffer"; Subject: Re: [ovirt-users] can not use iscsi storage type on ovirtandGlusterfs hyper-converged environment Thanks for the logs, What kind of VDSM version are you using? "rpm -q vdsm" There seems to be a similar issue which was reported recently in the VDSM area (see https://bugzilla.redhat.com/show_bug.cgi?id=1197292) It should be fixed in later versions of VDSM vdsm-4.16.12-2.el7ev.x86_64 Adding also Nir and Jeff to the thread, if they have any insights Regards, Maor On Wed, Nov 2, 2016 at 4:11 AM, 胡茂荣 wrote: Hi Maor: vdsm/supervdsm/engine log on attachment . I mkfs.xfs the lun block device and mount to /mnt , dd write ,dmesg not report error ,dd result is ok : /dev/sdi 50G 33M 50G 1% /mnt [root@horebc mnt]# for i in `seq 3`; do dd if=/dev/zero of=./file bs=1G count=1 oflag=direct ; done 1+0 records in 1+0 records out 1073741824 bytes (1.1 GB) copied, 13.3232 s, 80.6 MB/s 1+0 records in 1+0 records out 1073741824 bytes (1.1 GB) copied, 9.89988 s, 108 MB/s 1+0 records in 1+0 records out 1073741824 bytes (1.1 GB) copied, 14.0143 s, 76.6 MB/s my envirnment have three network segments (hosts have 3 network segments ) : engine and glusterfs mount : 192.168.11.X/24 glusterfs brick : 192.168.10.x/24 iscsi : 192.168.
Re: [ovirt-users] can not use iscsi storage type on ovirtandGlusterfshyper-converged environment
I find this problem more info : I use flashcache on ovirt hosts , have dm device in /dev/mapper/ : [root@horeba init.d]# dmsetup info -C (and I set them in multipath blacklist ) Name Maj Min Stat Open Targ Event UUID flash_sdb253 0 L--w01 0 flash_sdf253 2 L--w01 0 flash_sdd253 1 L--w01 0 [root@horeba init.d]# multipath -l [root@horeba init.d]# [root@horeba init.d]# ll /dev/mapper/ total 0 crw--- 1 root root 10, 236 Nov 25 10:09 control lrwxrwxrwx 1 root root 7 Nov 25 12:51 flash_sdb -> ../dm-0 lrwxrwxrwx 1 root root 7 Nov 25 12:51 flash_sdd -> ../dm-1 lrwxrwxrwx 1 root root 7 Nov 25 12:51 flash_sdf -> ../dm-2 on this condition , ovirt UI add iscsi type storage will fail . If I delete the flashcache device /dev/mapper/flash_* , ovirt UI add iscsi type storage have no problem . I need flashcache for using ssd cache on my environment , how can I use iscsi type storage on this environment , please help me , Thanks! -- Original -- From: "Sahina Bose"; Date: Thu, Nov 3, 2016 05:54 PM To: "胡茂荣"; Cc: "Maor Lipchuk"; "Jeff Nelson"; "users"; Subject: Re: [ovirt-users] can not use iscsi storage type on ovirtandGlusterfshyper-converged environment A wild guess, not sure if it is related - can you check if multipathd service is enabled. If you set up your oVirt-Gluster hyperconverged environment via gdeploy, multipathd service is disabled and the /etc/multipath.conf is edited to blacklist all devices - this was to fix https://bugzilla.redhat.com/show_bug.cgi?id=1303940 Since you mention you're unable to add iscsi storage only in this environment, thought it's worth checking. On Thu, Nov 3, 2016 at 6:40 AM, 胡茂荣 wrote: my environment rpm are : [root@horeba ~]# rpm -q vdsm vdsm-4.18.13-1.el7.centos.x86_64 [root@horeba ~]# rpm -aq | grep ovirt ovirt-vmconsole-host-1.0.4-1.el7.centos.noarch ovirt-imageio-common-0.4.0-1.el7.noarch ovirt-hosted-engine-setup-2.0.2.2-1.el7.centos.noarch ovirt-imageio-daemon-0.4.0-1.el7.noarch ovirt-engine-appliance-4.0-20160928.1.el7.centos.noarch ovirt-vmconsole-1.0.4-1.el7.centos.noarch ovirt-host-deploy-1.5.2-1.el7.centos.noarch ovirt-hosted-engine-ha-2.0.4-1.el7.centos.noarch ovirt-release40-4.0.4-1.noarch ovirt-setup-lib-1.0.2-1.el7.centos.noarch ovirt-engine-sdk-python-3.6.9.1-1.el7.centos.noarch and I test if not on ' ovirt and Glusterfs hyper-converged environment ' , ovirt WEB UI add iscsi storage work ok . -- Original ------ From: "Maor Lipchuk"; Date: Wed, Nov 2, 2016 07:37 PM To: "胡茂荣"; Cc: "users"; "Jeff Nelson"; "Nir Soffer"; Subject: Re: [ovirt-users] can not use iscsi storage type on ovirtandGlusterfs hyper-converged environment Thanks for the logs, What kind of VDSM version are you using? "rpm -q vdsm" There seems to be a similar issue which was reported recently in the VDSM area (see https://bugzilla.redhat.com/show_bug.cgi?id=1197292) It should be fixed in later versions of VDSM vdsm-4.16.12-2.el7ev.x86_64 Adding also Nir and Jeff to the thread, if they have any insights Regards, Maor On Wed, Nov 2, 2016 at 4:11 AM, 胡茂荣 wrote: Hi Maor: vdsm/supervdsm/engine log on attachment . I mkfs.xfs the lun block device and mount to /mnt , dd write ,dmesg not report error ,dd result is ok : /dev/sdi 50G 33M 50G 1% /mnt [root@horebc mnt]# for i in `seq 3`; do dd if=/dev/zero of=./file bs=1G count=1 oflag=direct ; done 1+0 records in 1+0 records out 1073741824 bytes (1.1 GB) copied, 13.3232 s, 80.6 MB/s 1+0 records in 1+0 records out 1073741824 bytes (1.1 GB) copied, 9.89988 s, 108 MB/s 1+0 records in 1+0 records out 1073741824 bytes (1.1 GB) copied, 14.0143 s, 76.6 MB/s my envirnment have three network segments (hosts have 3 network segments ) : engine and glusterfs mount : 192.168.11.X/24 glusterfs brick : 192.168.10.x/24 iscsi : 192.168.1.0/24 and I add 192.168.1.0/24 to engine vm , ovirt web UI report the same error . humaorong 2016-11-2 -- Original -- From: "Maor Lipchuk"; Date: Tue, Nov 1, 2016 08:14 PM To: "胡茂荣"; Cc: "users"; Subject: Re: [ovirt-users] can not use iscsi storage type on ovirt andGlusterfs hyper-converged environment Hi 胡茂荣 Can u please also add the VDSM and engine logs. If you try discover and connect to those luns directly from your Host does it work? Regards, Maor On Tue, Nov 1, 2016 at 6:12 AM, 胡茂荣 wrote: on ovirt a
Re: [ovirt-users] can not use iscsi storage type on ovirtandGlusterfshyper-converged environment
Hi Sahina Bose: my envirnment ,the /etc/multipath.conf is edited to blacklist all devices before . Now I change blaklist to exist physical device ,and multipahd is enabled . logout the lun block device by manual , then add iscsi type storage by ovirt UI , the error alse exist , add iscsi storage fail . and https://bugzilla.redhat.com/show_bug.cgi?id=1303940 bug last report time on 2016-09 -15 .my vdsm install on sep 8: -rw-r--r--. 1 root root 703032 Sep 8 00:27 vdsm-4.18.13-1.el7.centos.x86_64.rpm -rw-r--r--. 1 root root 54076 Sep 8 00:25 vdsm-api-4.18.13-1.el7.centos.noarch.rpm -rw-r--r--. 1 root root 68524 Sep 8 00:26 vdsm-cli-4.18.13-1.el7.centos.noarch.rpm -rw-r--r--. 1 root root 54580 Sep 8 00:26 vdsm-gluster-4.18.13-1.el7.centos.noarch.rpm ovirt version : 0.4.0 I will update vdsm and ovirt pakages to test . Thanks very much ! -- Original -- From: "Sahina Bose"; Date: Thu, Nov 3, 2016 05:54 PM To: "胡茂荣"; Cc: "Maor Lipchuk"; "Jeff Nelson"; "users"; Subject: Re: [ovirt-users] can not use iscsi storage type on ovirtandGlusterfshyper-converged environment A wild guess, not sure if it is related - can you check if multipathd service is enabled. If you set up your oVirt-Gluster hyperconverged environment via gdeploy, multipathd service is disabled and the /etc/multipath.conf is edited to blacklist all devices - this was to fix https://bugzilla.redhat.com/show_bug.cgi?id=1303940 Since you mention you're unable to add iscsi storage only in this environment, thought it's worth checking. On Thu, Nov 3, 2016 at 6:40 AM, 胡茂荣 wrote: my environment rpm are : [root@horeba ~]# rpm -q vdsm vdsm-4.18.13-1.el7.centos.x86_64 [root@horeba ~]# rpm -aq | grep ovirt ovirt-vmconsole-host-1.0.4-1.el7.centos.noarch ovirt-imageio-common-0.4.0-1.el7.noarch ovirt-hosted-engine-setup-2.0.2.2-1.el7.centos.noarch ovirt-imageio-daemon-0.4.0-1.el7.noarch ovirt-engine-appliance-4.0-20160928.1.el7.centos.noarch ovirt-vmconsole-1.0.4-1.el7.centos.noarch ovirt-host-deploy-1.5.2-1.el7.centos.noarch ovirt-hosted-engine-ha-2.0.4-1.el7.centos.noarch ovirt-release40-4.0.4-1.noarch ovirt-setup-lib-1.0.2-1.el7.centos.noarch ovirt-engine-sdk-python-3.6.9.1-1.el7.centos.noarch and I test if not on ' ovirt and Glusterfs hyper-converged environment ' , ovirt WEB UI add iscsi storage work ok . -- Original ------ From: "Maor Lipchuk"; Date: Wed, Nov 2, 2016 07:37 PM To: "胡茂荣"; Cc: "users"; "Jeff Nelson"; "Nir Soffer"; Subject: Re: [ovirt-users] can not use iscsi storage type on ovirtandGlusterfs hyper-converged environment Thanks for the logs, What kind of VDSM version are you using? "rpm -q vdsm" There seems to be a similar issue which was reported recently in the VDSM area (see https://bugzilla.redhat.com/show_bug.cgi?id=1197292) It should be fixed in later versions of VDSM vdsm-4.16.12-2.el7ev.x86_64 Adding also Nir and Jeff to the thread, if they have any insights Regards, Maor On Wed, Nov 2, 2016 at 4:11 AM, 胡茂荣 wrote: Hi Maor: vdsm/supervdsm/engine log on attachment . I mkfs.xfs the lun block device and mount to /mnt , dd write ,dmesg not report error ,dd result is ok : /dev/sdi 50G 33M 50G 1% /mnt [root@horebc mnt]# for i in `seq 3`; do dd if=/dev/zero of=./file bs=1G count=1 oflag=direct ; done 1+0 records in 1+0 records out 1073741824 bytes (1.1 GB) copied, 13.3232 s, 80.6 MB/s 1+0 records in 1+0 records out 1073741824 bytes (1.1 GB) copied, 9.89988 s, 108 MB/s 1+0 records in 1+0 records out 1073741824 bytes (1.1 GB) copied, 14.0143 s, 76.6 MB/s my envirnment have three network segments (hosts have 3 network segments ) : engine and glusterfs mount : 192.168.11.X/24 glusterfs brick : 192.168.10.x/24 iscsi : 192.168.1.0/24 and I add 192.168.1.0/24 to engine vm , ovirt web UI report the same error . humaorong 2016-11-2 -- Original -- From: "Maor Lipchuk"; Date: Tue, Nov 1, 2016 08:14 PM To: "胡茂荣"; Cc: "users"; Subject: Re: [ovirt-users] can not use iscsi storage type on ovirt andGlusterfs hyper-converged environment Hi 胡茂荣 Can u please also add the VDSM and engine logs. If you try discover and connect to those luns directly from your Host does it work? Regards, Maor On Tue, Nov 1, 2016 at 6:12 AM, 胡茂荣 wrote: on ovirt and Glusterfs hyper-converged environment , can not use iscsi storage type , UI report error: "Could not retrieve LUNs, please check your storage." , vdsm report :"VDSM hosted_engine_3 command failed: Error block device action: ()" . but this block device
Re: [ovirt-users] can not use iscsi storage type on ovirtandGlusterfs hyper-converged environment
my environment rpm are : [root@horeba ~]# rpm -q vdsm vdsm-4.18.13-1.el7.centos.x86_64 [root@horeba ~]# rpm -aq | grep ovirt ovirt-vmconsole-host-1.0.4-1.el7.centos.noarch ovirt-imageio-common-0.4.0-1.el7.noarch ovirt-hosted-engine-setup-2.0.2.2-1.el7.centos.noarch ovirt-imageio-daemon-0.4.0-1.el7.noarch ovirt-engine-appliance-4.0-20160928.1.el7.centos.noarch ovirt-vmconsole-1.0.4-1.el7.centos.noarch ovirt-host-deploy-1.5.2-1.el7.centos.noarch ovirt-hosted-engine-ha-2.0.4-1.el7.centos.noarch ovirt-release40-4.0.4-1.noarch ovirt-setup-lib-1.0.2-1.el7.centos.noarch ovirt-engine-sdk-python-3.6.9.1-1.el7.centos.noarch and I test if not on ' ovirt and Glusterfs hyper-converged environment ' , ovirt WEB UI add iscsi storage work ok . -- Original -- From: "Maor Lipchuk"; Date: Wed, Nov 2, 2016 07:37 PM To: "胡茂荣"; Cc: "users"; "Jeff Nelson"; "Nir Soffer"; Subject: Re: [ovirt-users] can not use iscsi storage type on ovirtandGlusterfs hyper-converged environment Thanks for the logs, What kind of VDSM version are you using? "rpm -q vdsm" There seems to be a similar issue which was reported recently in the VDSM area (see https://bugzilla.redhat.com/show_bug.cgi?id=1197292) It should be fixed in later versions of VDSM vdsm-4.16.12-2.el7ev.x86_64 Adding also Nir and Jeff to the thread, if they have any insights Regards, Maor On Wed, Nov 2, 2016 at 4:11 AM, 胡茂荣 wrote: Hi Maor: vdsm/supervdsm/engine log on attachment . I mkfs.xfs the lun block device and mount to /mnt , dd write ,dmesg not report error ,dd result is ok : /dev/sdi 50G 33M 50G 1% /mnt [root@horebc mnt]# for i in `seq 3`; do dd if=/dev/zero of=./file bs=1G count=1 oflag=direct ; done 1+0 records in 1+0 records out 1073741824 bytes (1.1 GB) copied, 13.3232 s, 80.6 MB/s 1+0 records in 1+0 records out 1073741824 bytes (1.1 GB) copied, 9.89988 s, 108 MB/s 1+0 records in 1+0 records out 1073741824 bytes (1.1 GB) copied, 14.0143 s, 76.6 MB/s my envirnment have three network segments (hosts have 3 network segments ) : engine and glusterfs mount : 192.168.11.X/24 glusterfs brick : 192.168.10.x/24 iscsi : 192.168.1.0/24 and I add 192.168.1.0/24 to engine vm , ovirt web UI report the same error . humaorong 2016-11-2 -- Original ------ From: "Maor Lipchuk"; Date: Tue, Nov 1, 2016 08:14 PM To: "胡茂荣"; Cc: "users"; Subject: Re: [ovirt-users] can not use iscsi storage type on ovirt andGlusterfs hyper-converged environment Hi 胡茂荣 Can u please also add the VDSM and engine logs. If you try discover and connect to those luns directly from your Host does it work? Regards, Maor On Tue, Nov 1, 2016 at 6:12 AM, 胡茂荣 wrote: on ovirt and Glusterfs hyper-converged environment , can not use iscsi storage type , UI report error: "Could not retrieve LUNs, please check your storage." , vdsm report :"VDSM hosted_engine_3 command failed: Error block device action: ()" . but this block device alse login on centos 7 host : = ## lsscsi [7:0:0:0]diskSCST_BIO DEVFOR_OVIRT_rbd 221 /dev/sdi ## dmesg : [684521.131186] sd 7:0:0:0: [sdi] Attached SCSI disk ===--- ###vdsm or supervdsm log report : MainProcess|jsonrpc.Executor/7::ERROR::2016-11-01 11:07:00,178::supervdsmServer::96::SuperVdsm.ServerCallback::(wrapper) Error in getPathsStatus MainProcess|jsonrpc.Executor/4::ERROR::2016-11-01 11:07:20,964::supervdsmServer::96::SuperVdsm.ServerCallback::(wrapper) Error in getPathsStatus jsonrpc.Executor/4::DEBUG::2016-11-01 11:07:04,251::iscsi::434::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds jsonrpc.Executor/5::INFO::2016-11-01 11:07:19,413::iscsi::567::Storage.ISCSI::(setRpFilterIfNeeded) iSCSI iface.net_ifacename not provided. Skipping. 11:09:15,753::iscsiadm::119::Storage.Misc.excCmd::(_runCmd) /usr/bin/taskset --cpu-list 0-7 /usr/bin/sudo -n /usr/sbin/iscsiadm -m session -R (cwd None) == the other info please the attachment "bug-info.doc". this prolem on ovirt3.6 and 4.X ovirt and Glusterfs hyper-converged environment . how can I use iscsi storage type on ovirt and Glusterfs hyper-converged environment .Please help me ! humaorong 2016-11-1 ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
[ovirt-users] can not use iscsi storage type on ovirt and Glusterfs hyper-converged environment
on ovirt and Glusterfs hyper-converged environment , can not use iscsi storage type , UI report error: "Could not retrieve LUNs, please check your storage." , vdsm report :"VDSM hosted_engine_3 command failed: Error block device action: ()" . but this block device alse login on centos 7 host : = ## lsscsi [7:0:0:0]diskSCST_BIO DEVFOR_OVIRT_rbd 221 /dev/sdi ## dmesg : [684521.131186] sd 7:0:0:0: [sdi] Attached SCSI disk ===--- ###vdsm or supervdsm log report : MainProcess|jsonrpc.Executor/7::ERROR::2016-11-01 11:07:00,178::supervdsmServer::96::SuperVdsm.ServerCallback::(wrapper) Error in getPathsStatus MainProcess|jsonrpc.Executor/4::ERROR::2016-11-01 11:07:20,964::supervdsmServer::96::SuperVdsm.ServerCallback::(wrapper) Error in getPathsStatus jsonrpc.Executor/4::DEBUG::2016-11-01 11:07:04,251::iscsi::434::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds jsonrpc.Executor/5::INFO::2016-11-01 11:07:19,413::iscsi::567::Storage.ISCSI::(setRpFilterIfNeeded) iSCSI iface.net_ifacename not provided. Skipping. 11:09:15,753::iscsiadm::119::Storage.Misc.excCmd::(_runCmd) /usr/bin/taskset --cpu-list 0-7 /usr/bin/sudo -n /usr/sbin/iscsiadm -m session -R (cwd None) == the other info please the attachment "bug-info.doc". this prolem on ovirt3.6 and 4.X ovirt and Glusterfs hyper-converged environment . how can I use iscsi storage type on ovirt and Glusterfs hyper-converged environment .Please help me ! humaorong 2016-11-1 bug-info.doc Description: Binary data ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
[ovirt-users] glusterfs used swap , then reboot host , Ovirt Engine vm can not up on my ovirt and Glusterfs hyper-converged environment
Hi : my problem: on ovirt and Glusterfs hyper-converged environment , I create 30 vms,and start vms ,find vm running slowlly , then use ‘free ’ command on host ,find that ,three hosts used swap , I shutdown all vm but not engine vm , after three hours ,the swap memory can not be recovered . I reboot three hosts , after hosts up ,I find that , engine can not up , and set global maintenance ,try start engine manually , engine still can not start . three host Engine status : {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "down"}, like: Status up-to-date : True Hostname : horeba Host ID: 1 Engine status : {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "down"} Score : 0 stopped: False Local maintenance : False crc32 : bf11e363 Host timestamp : 1072 please help me , how can I start engine ? or how can I recover my environmont ? engine can not up atfter reboot hosts.doc Description: Binary data ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
[ovirt-users] can I use ovirt webmin UI create glusterfs volume and not use ovirtmgmt ip
My problem is : my environment : [root@node44 bin]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.8.93 node93.gluster.com 192.168.8.44 node44.gluster.com 192.168.8.45 node45.gluster.com 192.168.1.93 node93.host.com 192.168.1.44 node44.host.com 192.168.1.45 node45.host.com 192.168.1.9 node9.engine.com and glusterfs peer : Hostname: node45.gluster.com Hostname: node93.gluster.com Hostname: node44.gluster.com ovirtmgmt port : Hostname: node45.host.com Hostname: node93.host.com Hostname: node44.host.com Then I want create glusterfs volume by ovirt webmin UI , but webmin UI show only ovirtmgmt port : how can I use glusterfs peer port create glusterfs volume by ovirt webmin UI ?___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users