Re: [ovirt-users] Setting up GeoReplication
On Mon, May 15, 2017 at 7:40 PM, Jim Kusznirwrote: > I tried to create a gluster volume on the georep node by running: > > gluster volume create engine-rep replica 1 georep.nwfiber.com:/mnt/gluste > r/engine-rep > > I got back an error saying replica must be > 1. So I tried to create it > again: > "replica 1" is not required on command when you're not replicating to another server. So, gluster volume create engine-rep georep.nwfiber.com:/mnt/gluster/engine-rep > gluster volume create engine-rep replica 2 > georep.nwfiber.com:/mnt/gluster/engine-rep > server2.nwfiber.com:/mnt/gluster/engine-rep > > where server2 did not exist. That failed too, but I don't recall the > error message. > > gluster is installed, but when I try and start it with the init script, it > fails to start with a complaint about reading the block file; my googling > indicated that's the error you get until you've created a gluster volume, > and that was the first clue to me that maybe I needed to create one first. > > So, how do I create a replica 1 volume? > > > Thinking way ahead, I have a related replica question: Currently my ovirt > nodes are also my gluster nodes (replica 2 arbitrar 1). Eventually I'll > want to pull my gluster off onto dedicated hardware I suspect. If I do so, > do I need 3 servers, or is a replica 2 sufficient? I guess I could have an > ovirt node continue to be an arbitrar... I would eventually like to > distribute my ovirt cluster accross multiple locations with the option for > remote failover (say location A looses all its network and/or power; have > important VMs started at location B in addition to location B's normal > VMs). I assume at this point the recommended arch would be: > > 2 Gluster servers at each location > Each location has a gluster volume for that location, and is georep for > the other location (so all my data will physically exist on 4 gluster > servers). I probably won't have more than 2 or 3 ovirt hosts at each > location, so I don't expect this to be a "heavy use" system. > To move to a dedicated gluster setup, 2 gluster servers + 1 oVirt node holding the arbiter brick should work. Distributed or stretch cluster does not work off the bat yet. If a volume is geo-replicated to another location, the destination volume cannot be a storage domain in a cluster, as geo-replication needs the destination volume data to be in sync with master. Another option is for the master volume/cluster is spread across geographies - but this requires the network latency to be very low in order for this to work as all operations are synchronous in gluster. We're working on features in both gluster and Ovirt to support the stretched cluster requirements. > Am I on track? I'd be interested to learn what others suggest for this > deployment model. > > On Sun, May 14, 2017 at 11:09 PM, Sahina Bose wrote: > >> Adding Aravinda >> >> On Sat, May 13, 2017 at 11:21 PM, Jim Kusznir >> wrote: >> >>> Hi All: >>> >>> I've been trying to set up georeplication for a while now, but can't >>> seem to make it work. I've found documentation on the web (mostly >>> https://gluster.readthedocs.io/en/refactor/Administr >>> ator%20Guide/Geo%20Replication/), and I found http://blog.gluster.org/ >>> 2015/09/introducing-georepsetup-gluster-geo-replication-setup-tool/ >>> >>> Unfortunately, it seems that some critical steps are missing from both, >>> and I can't figure out for sure what they are. >>> >>> My environment: >>> >>> Production: replica 2 + arbitrator running on my 3-node oVirt cluster, 3 >>> volumes (engine, data, iso). >>> >>> New geo-replication: Raspberry Pi3 with USB hard drive shoved in some >>> other data closet off-site. >>> >>> I've installed rasbian-lite, and after much fighting, got >>> glusterfs-*-3.8.11 installed. I've created my mountpoint (USB hard drive, >>> much larger than my gluster volumes), and then ran the command. I get this >>> far: >>> >>> [OK] georep.nwfiber.com is Reachable(Port 22) >>> [OK] SSH Connection established r...@georep.nwfiber.com >>> [OK] Master Volume and Slave Volume are compatible (Version: 3.8.11) >>> [NOT OK] Unable to Mount Gluster Volume georep.nwfiber.com:engine-rep >>> >>> Trying it with the steps in the gluster docs also has the same problem. >>> No long files are generated on the slave. Log files on the master include: >>> >>> [root@ovirt1 geo-replication]# more georepsetup.mount.log >>> [2017-05-13 17:26:27.318599] I [MSGID: 100030] [glusterfsd.c:2454:main] >>> 0-glusterfs: Started running glusterfs version 3.8.11 (args: >>> glusterfs --xlator-option="*dht.lookup-unhashed=off" --volfile-server >>> localhost --volfile-id engine -l /var/log/glusterfs/geo-repli >>> cation/georepsetup.mount.log --client-pid=-1 /tmp/georepsetup_wZtfkN) >>> [2017-05-13 17:26:27.341170] I [MSGID: 101190] >>> [event-epoll.c:628:event_dispatch_epoll_worker] 0-epoll: Started thread >>> with index
Re: [ovirt-users] Setting up GeoReplication
I tried to create a gluster volume on the georep node by running: gluster volume create engine-rep replica 1 georep.nwfiber.com: /mnt/gluster/engine-rep I got back an error saying replica must be > 1. So I tried to create it again: gluster volume create engine-rep replica 2 georep.nwfiber.com:/mnt/gluster/engine-rep server2.nwfiber.com:/mnt/gluster/engine-rep where server2 did not exist. That failed too, but I don't recall the error message. gluster is installed, but when I try and start it with the init script, it fails to start with a complaint about reading the block file; my googling indicated that's the error you get until you've created a gluster volume, and that was the first clue to me that maybe I needed to create one first. So, how do I create a replica 1 volume? Thinking way ahead, I have a related replica question: Currently my ovirt nodes are also my gluster nodes (replica 2 arbitrar 1). Eventually I'll want to pull my gluster off onto dedicated hardware I suspect. If I do so, do I need 3 servers, or is a replica 2 sufficient? I guess I could have an ovirt node continue to be an arbitrar... I would eventually like to distribute my ovirt cluster accross multiple locations with the option for remote failover (say location A looses all its network and/or power; have important VMs started at location B in addition to location B's normal VMs). I assume at this point the recommended arch would be: 2 Gluster servers at each location Each location has a gluster volume for that location, and is georep for the other location (so all my data will physically exist on 4 gluster servers). I probably won't have more than 2 or 3 ovirt hosts at each location, so I don't expect this to be a "heavy use" system. Am I on track? I'd be interested to learn what others suggest for this deployment model. On Sun, May 14, 2017 at 11:09 PM, Sahina Bosewrote: > Adding Aravinda > > On Sat, May 13, 2017 at 11:21 PM, Jim Kusznir wrote: > >> Hi All: >> >> I've been trying to set up georeplication for a while now, but can't seem >> to make it work. I've found documentation on the web (mostly >> https://gluster.readthedocs.io/en/refactor/Administr >> ator%20Guide/Geo%20Replication/), and I found http://blog.gluster.org/ >> 2015/09/introducing-georepsetup-gluster-geo-replication-setup-tool/ >> >> Unfortunately, it seems that some critical steps are missing from both, >> and I can't figure out for sure what they are. >> >> My environment: >> >> Production: replica 2 + arbitrator running on my 3-node oVirt cluster, 3 >> volumes (engine, data, iso). >> >> New geo-replication: Raspberry Pi3 with USB hard drive shoved in some >> other data closet off-site. >> >> I've installed rasbian-lite, and after much fighting, got >> glusterfs-*-3.8.11 installed. I've created my mountpoint (USB hard drive, >> much larger than my gluster volumes), and then ran the command. I get this >> far: >> >> [OK] georep.nwfiber.com is Reachable(Port 22) >> [OK] SSH Connection established r...@georep.nwfiber.com >> [OK] Master Volume and Slave Volume are compatible (Version: 3.8.11) >> [NOT OK] Unable to Mount Gluster Volume georep.nwfiber.com:engine-rep >> >> Trying it with the steps in the gluster docs also has the same problem. >> No long files are generated on the slave. Log files on the master include: >> >> [root@ovirt1 geo-replication]# more georepsetup.mount.log >> [2017-05-13 17:26:27.318599] I [MSGID: 100030] [glusterfsd.c:2454:main] >> 0-glusterfs: Started running glusterfs version 3.8.11 (args: >> glusterfs --xlator-option="*dht.lookup-unhashed=off" --volfile-server >> localhost --volfile-id engine -l /var/log/glusterfs/geo-repli >> cation/georepsetup.mount.log --client-pid=-1 /tmp/georepsetup_wZtfkN) >> [2017-05-13 17:26:27.341170] I [MSGID: 101190] >> [event-epoll.c:628:event_dispatch_epoll_worker] 0-epoll: Started thread >> with index 1 >> [2017-05-13 17:26:27.341260] E [socket.c:2309:socket_connect_finish] >> 0-glusterfs: connection to ::1:24007 failed (Connection refused >> ) >> [2017-05-13 17:26:27.341846] E [glusterfsd-mgmt.c:1908:mgmt_rpc_notify] >> 0-glusterfsd-mgmt: failed to connect with remote-host: local >> host (Transport endpoint is not connected) >> [2017-05-13 17:26:31.335849] I [MSGID: 101190] >> [event-epoll.c:628:event_dispatch_epoll_worker] 0-epoll: Started thread >> with index 2 >> [2017-05-13 17:26:31.337545] I [MSGID: 114020] [client.c:2356:notify] >> 0-engine-client-0: parent translators are ready, attempting co >> nnect on transport >> [2017-05-13 17:26:31.344485] I [MSGID: 114020] [client.c:2356:notify] >> 0-engine-client-1: parent translators are ready, attempting co >> nnect on transport >> [2017-05-13 17:26:31.345146] I [rpc-clnt.c:1965:rpc_clnt_reconfig] >> 0-engine-client-0: changing port to 49157 (from 0) >> [2017-05-13 17:26:31.350868] I [MSGID: 114020] [client.c:2356:notify] >> 0-engine-client-2: parent translators are ready,
Re: [ovirt-users] Setting up GeoReplication
Adding Aravinda On Sat, May 13, 2017 at 11:21 PM, Jim Kusznirwrote: > Hi All: > > I've been trying to set up georeplication for a while now, but can't seem > to make it work. I've found documentation on the web (mostly > https://gluster.readthedocs.io/en/refactor/Administrator%20Guide/Geo% > 20Replication/), and I found http://blog.gluster.org/2015/09/introducing- > georepsetup-gluster-geo-replication-setup-tool/ > > Unfortunately, it seems that some critical steps are missing from both, > and I can't figure out for sure what they are. > > My environment: > > Production: replica 2 + arbitrator running on my 3-node oVirt cluster, 3 > volumes (engine, data, iso). > > New geo-replication: Raspberry Pi3 with USB hard drive shoved in some > other data closet off-site. > > I've installed rasbian-lite, and after much fighting, got > glusterfs-*-3.8.11 installed. I've created my mountpoint (USB hard drive, > much larger than my gluster volumes), and then ran the command. I get this > far: > > [OK] georep.nwfiber.com is Reachable(Port 22) > [OK] SSH Connection established r...@georep.nwfiber.com > [OK] Master Volume and Slave Volume are compatible (Version: 3.8.11) > [NOT OK] Unable to Mount Gluster Volume georep.nwfiber.com:engine-rep > > Trying it with the steps in the gluster docs also has the same problem. > No long files are generated on the slave. Log files on the master include: > > [root@ovirt1 geo-replication]# more georepsetup.mount.log > [2017-05-13 17:26:27.318599] I [MSGID: 100030] [glusterfsd.c:2454:main] > 0-glusterfs: Started running glusterfs version 3.8.11 (args: > glusterfs --xlator-option="*dht.lookup-unhashed=off" --volfile-server > localhost --volfile-id engine -l /var/log/glusterfs/geo-repli > cation/georepsetup.mount.log --client-pid=-1 /tmp/georepsetup_wZtfkN) > [2017-05-13 17:26:27.341170] I [MSGID: 101190] > [event-epoll.c:628:event_dispatch_epoll_worker] > 0-epoll: Started thread with index 1 > [2017-05-13 17:26:27.341260] E [socket.c:2309:socket_connect_finish] > 0-glusterfs: connection to ::1:24007 failed (Connection refused > ) > [2017-05-13 17:26:27.341846] E [glusterfsd-mgmt.c:1908:mgmt_rpc_notify] > 0-glusterfsd-mgmt: failed to connect with remote-host: local > host (Transport endpoint is not connected) > [2017-05-13 17:26:31.335849] I [MSGID: 101190] > [event-epoll.c:628:event_dispatch_epoll_worker] > 0-epoll: Started thread with index 2 > [2017-05-13 17:26:31.337545] I [MSGID: 114020] [client.c:2356:notify] > 0-engine-client-0: parent translators are ready, attempting co > nnect on transport > [2017-05-13 17:26:31.344485] I [MSGID: 114020] [client.c:2356:notify] > 0-engine-client-1: parent translators are ready, attempting co > nnect on transport > [2017-05-13 17:26:31.345146] I [rpc-clnt.c:1965:rpc_clnt_reconfig] > 0-engine-client-0: changing port to 49157 (from 0) > [2017-05-13 17:26:31.350868] I [MSGID: 114020] [client.c:2356:notify] > 0-engine-client-2: parent translators are ready, attempting co > nnect on transport > [2017-05-13 17:26:31.355946] I [MSGID: 114057] [client-handshake.c:1440: > select_server_supported_programs] 0-engine-client-0: Using P > rogram GlusterFS 3.3, Num (1298437), Version (330) > [2017-05-13 17:26:31.356280] I [rpc-clnt.c:1965:rpc_clnt_reconfig] > 0-engine-client-1: changing port to 49157 (from 0) > Final graph: > +--- > ---+ > 1: volume engine-client-0 > 2: type protocol/client > 3: option clnt-lk-version 1 > 4: option volfile-checksum 0 > 5: option volfile-key engine > 6: option client-version 3.8.11 > 7: option process-uuid ovirt1.nwfiber.com-25660-2017/ > 05/13-17:26:27:311929-engine-client-0-0-0 > 8: option fops-version 1298437 > 9: option ping-timeout 30 > 10: option remote-host ovirt1.nwfiber.com > 11: option remote-subvolume /gluster/brick1/engine > 12: option transport-type socket > 13: option username 028984cf-0399-42e6-b04b-bb9b1685c536 > 14: option password eae737cc-9659-405f-865e-9a7ef97a3307 > 15: option filter-O_DIRECT off > 16: option send-gids true > 17: end-volume > 18: > 19: volume engine-client-1 > 20: type protocol/client > 21: option ping-timeout 30 > 22: option remote-host ovirt2.nwfiber.com > 23: option remote-subvolume /gluster/brick1/engine > 24: option transport-type socket > 25: option username 028984cf-0399-42e6-b04b-bb9b1685c536 > 26: option password eae737cc-9659-405f-865e-9a7ef97a3307 > 27: option filter-O_DIRECT off > 28: option send-gids true > 29: end-volume > 30: > 31: volume engine-client-2 > 32: type protocol/client > 33: option ping-timeout 30 > 34: option remote-host ovirt3.nwfiber.com > 35: option remote-subvolume /gluster/brick1/engine > 36: option transport-type socket > 37: option username 028984cf-0399-42e6-b04b-bb9b1685c536 >
[ovirt-users] Setting up GeoReplication
Hi All: I've been trying to set up georeplication for a while now, but can't seem to make it work. I've found documentation on the web (mostly https://gluster.readthedocs.io/en/refactor/Administrator%20Guide/Geo%20Replication/), and I found http://blog.gluster.org/2015/09/introducing-georepsetup-gluster-geo-replication-setup-tool/ Unfortunately, it seems that some critical steps are missing from both, and I can't figure out for sure what they are. My environment: Production: replica 2 + arbitrator running on my 3-node oVirt cluster, 3 volumes (engine, data, iso). New geo-replication: Raspberry Pi3 with USB hard drive shoved in some other data closet off-site. I've installed rasbian-lite, and after much fighting, got glusterfs-*-3.8.11 installed. I've created my mountpoint (USB hard drive, much larger than my gluster volumes), and then ran the command. I get this far: [OK] georep.nwfiber.com is Reachable(Port 22) [OK] SSH Connection established r...@georep.nwfiber.com [OK] Master Volume and Slave Volume are compatible (Version: 3.8.11) [NOT OK] Unable to Mount Gluster Volume georep.nwfiber.com:engine-rep Trying it with the steps in the gluster docs also has the same problem. No long files are generated on the slave. Log files on the master include: [root@ovirt1 geo-replication]# more georepsetup.mount.log [2017-05-13 17:26:27.318599] I [MSGID: 100030] [glusterfsd.c:2454:main] 0-glusterfs: Started running glusterfs version 3.8.11 (args: glusterfs --xlator-option="*dht.lookup-unhashed=off" --volfile-server localhost --volfile-id engine -l /var/log/glusterfs/geo-repli cation/georepsetup.mount.log --client-pid=-1 /tmp/georepsetup_wZtfkN) [2017-05-13 17:26:27.341170] I [MSGID: 101190] [event-epoll.c:628:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1 [2017-05-13 17:26:27.341260] E [socket.c:2309:socket_connect_finish] 0-glusterfs: connection to ::1:24007 failed (Connection refused ) [2017-05-13 17:26:27.341846] E [glusterfsd-mgmt.c:1908:mgmt_rpc_notify] 0-glusterfsd-mgmt: failed to connect with remote-host: local host (Transport endpoint is not connected) [2017-05-13 17:26:31.335849] I [MSGID: 101190] [event-epoll.c:628:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2 [2017-05-13 17:26:31.337545] I [MSGID: 114020] [client.c:2356:notify] 0-engine-client-0: parent translators are ready, attempting co nnect on transport [2017-05-13 17:26:31.344485] I [MSGID: 114020] [client.c:2356:notify] 0-engine-client-1: parent translators are ready, attempting co nnect on transport [2017-05-13 17:26:31.345146] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 0-engine-client-0: changing port to 49157 (from 0) [2017-05-13 17:26:31.350868] I [MSGID: 114020] [client.c:2356:notify] 0-engine-client-2: parent translators are ready, attempting co nnect on transport [2017-05-13 17:26:31.355946] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 0-engine-client-0: Using P rogram GlusterFS 3.3, Num (1298437), Version (330) [2017-05-13 17:26:31.356280] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 0-engine-client-1: changing port to 49157 (from 0) Final graph: +--+ 1: volume engine-client-0 2: type protocol/client 3: option clnt-lk-version 1 4: option volfile-checksum 0 5: option volfile-key engine 6: option client-version 3.8.11 7: option process-uuid ovirt1.nwfiber.com-25660-2017/05/13-17:26:27:311929-engine-client-0-0-0 8: option fops-version 1298437 9: option ping-timeout 30 10: option remote-host ovirt1.nwfiber.com 11: option remote-subvolume /gluster/brick1/engine 12: option transport-type socket 13: option username 028984cf-0399-42e6-b04b-bb9b1685c536 14: option password eae737cc-9659-405f-865e-9a7ef97a3307 15: option filter-O_DIRECT off 16: option send-gids true 17: end-volume 18: 19: volume engine-client-1 20: type protocol/client 21: option ping-timeout 30 22: option remote-host ovirt2.nwfiber.com 23: option remote-subvolume /gluster/brick1/engine 24: option transport-type socket 25: option username 028984cf-0399-42e6-b04b-bb9b1685c536 26: option password eae737cc-9659-405f-865e-9a7ef97a3307 27: option filter-O_DIRECT off 28: option send-gids true 29: end-volume 30: 31: volume engine-client-2 32: type protocol/client 33: option ping-timeout 30 34: option remote-host ovirt3.nwfiber.com 35: option remote-subvolume /gluster/brick1/engine 36: option transport-type socket 37: option username 028984cf-0399-42e6-b04b-bb9b1685c536 38: option password eae737cc-9659-405f-865e-9a7ef97a3307 39: option filter-O_DIRECT off 40: option send-gids true 41: end-volume 42: 43: volume engine-replicate-0 44: type cluster/replicate 45: option arbiter-count 1 46: option data-self-heal-algorithm full 47: option