Re: [Gluster-users] geo-replication {error=12} on one primary node

Hi, I,m still testing and I found that I can force the error by changing the shell from the unprivileged user, on the secondary node, from bash to sh. In the first try I used "useradd -G geogruppe -m geobenutzer" so my user gets /bin/sh (the dash) as default shell. Then the error occurs.

Re: [Gluster-users] geo-replication {error=12} on one primary node

Hi Anant, shame on me ^.^. I forgot to install rsync on that host. Switching to log-level DEBUG helped me to find the problem. Without log-level DEBUG the host is not showing the missing rsync. Maybe that could be changed. So thank you for the hint. Stefan Am 13.02.24 um 20:32 schrieb

Re: [Gluster-users] geo-replication {error=12} on one primary node

Hi @Stefan Kania, Please try to enable the geo-replication debug logs using the following command on the primary server, and recheck or resend the logs. gluster volume geo-replication privol01 geobenutzer@s01.gluster::secvol01 config log-level DEBUG​ Thanks,

Re: [Gluster-users] Geo-replication status is getting Faulty after few    seconds

hanks, Anant From: Strahil Nikolov Sent: 31 January 2024 4:18 PM To: Anant Saraswat ; Aravinda Cc: gluster-users@gluster.org Subject: Re: [Gluster-users] Geo-replication status is getting Faulty after few seconds EXTERNAL: Do not click links or open attachments if you do not recognize

Re: [Gluster-users] Geo-replication status is getting Faulty after few    seconds

Hi Anant, What version of Gluster are you using ? Best Regards,Strahil Nikolov Community Meeting Calendar: Schedule - Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC Bridge: https://meet.google.com/cpu-eiue-hvk Gluster-users mailing list Gluster-users@gluster.org

Re: [Gluster-users] Geo-replication status is getting Faulty after few seconds

Saraswat Sent: 30 January 2024 2:14 PM To: gluster-users@gluster.org ; Strahil Nikolov Subject: Re: [Gluster-users] Geo-replication status is getting Faulty after few seconds EXTERNAL: Do not click links or open attachments if you do not recognize the sender. Hello Everyone, I am looking

Re: [Gluster-users] Geo-replication status is getting Faulty after few seconds

__ From: Gluster-users on behalf of Anant Saraswat Sent: 29 January 2024 12:20 AM To: gluster-users@gluster.org ; Strahil Nikolov Subject: Re: [Gluster-users] Geo-replication status is getting Faulty after few seconds EXTERNAL: Do not click links or open attachments if y

Re: [Gluster-users] Geo-replication status is getting Faulty after few seconds

/secret.pem root@drtier1data Last login: Mon Jan 29 00:14:41 2024 from [root@drtier1data ~]# Thanks, Anant From: Strahil Nikolov Sent: 28 January 2024 10:07 PM To: Anant Saraswat ; gluster-users@gluster.org Subject: Re: [Gluster-users] Geo-replication status is getting

Re: [Gluster-users] Geo-replication status is getting Faulty after few seconds

: 28 January 2024 1:33 AM To: Strahil Nikolov ; gluster-users@gluster.org Subject: Re: [Gluster-users] Geo-replication status is getting Faulty after few seconds Hi @Strahil Nikolov, I have checked the ssh connection from all the master servers and I can sshdrtier1data from master1 and master2

Re: [Gluster-users] Geo-replication status is getting Faulty after few seconds

January 2024 1:33 AM To: Strahil Nikolov ; gluster-users@gluster.org Subject: Re: [Gluster-users] Geo-replication status is getting Faulty after few seconds Hi @Strahil Nikolov<mailto:hunter86...@yahoo.com>, I have checked the ssh connection from all the master servers and I can ssh drtie

Re: [Gluster-users] Geo-replication status is getting Faulty after few seconds

uto -S /tmp/gsyncd-aux-ssh-0exuoeg7/75785990b3233f5dbbab9f43cc3ed895.sock drtier1data:/proc/553418/cwd}, {error=3}] Many thanks, Anant From: Strahil Nikolov Sent: 27 January 2024 5:25 AM To: gluster-users@gluster.org ; Anant Saraswat Subject: Re: [Gluster-users] Geo-

Re: [Gluster-users] Geo-replication status is getting Faulty after few seconds

Don't forget to test with the georep key. I think it was /var/lib/glusterd/geo-replication/secret.pem Best Regards, Strahil Nikolov В събота, 27 януари 2024 г. в 07:24:07 ч. Гринуич+2, Strahil Nikolov написа: Hi Anant, i would first start checking if you can do ssh from all masters to

Re: [Gluster-users] Geo-replication status is getting Faulty after few seconds

Hi Anant, i would first start checking if you can do ssh from all masters to the slave node.If you haven't setup a dedicated user for the session, then gluster is using root. Best Regards, Strahil Nikolov В петък, 26 януари 2024 г. в 18:07:59 ч. Гринуич+2, Anant Saraswat написа:

Re: [Gluster-users] Geo-replication status is getting Faulty after few seconds

Hi All, I have run the following commands on master3, and that has added master3 to geo-replication. gluster system:: execute gsec_create gluster volume geo-replication tier1data drtier1data::drtier1data create push-pem force gluster volume geo-replication tier1data drtier1data::drtier1data

Re: [Gluster-users] Geo replication procedure for DR

To be honest, I have never reached that point but I think that if the original volume is too outdated it makes sense to setup a new volume on primary site and run a replication from the DR to primary site and then schedule a cut-over (Read-only DR volume, remove replication, point all clients

Re: [Gluster-users] Geo replication procedure for DR

Dear Strahil, Thank you for the detailed command. So once you want to switch all traffic to the DR site in case of disaster one should first disable the read-only setting on the secondary volume on the slave site. What happens after when the master site is back online? What's the procedure

Re: [Gluster-users] Geo replication procedure for DR

It's just a setting on the target volume: gluster volume set read-only OFF Best Regards,Strahil Nikolov  On Mon, Jun 5, 2023 at 22:30, mabi wrote: Hello, I was reading the geo replication documentation here: https://docs.gluster.org/en/main/Administrator-Guide/Geo-Replication/ and I

Re: [Gluster-users] Geo-Replication Stuck In "Hybrid Crawl"

Hi, Yes, I have checked both /var/log/gluster/geo-replication/ (on primary nodes) and /var/log/gluster/geo-replication-slaves/ (on slave node) but not finding any relevant information despite the fact that I've set all log levels to DEBUG. Looked at gsyncd.logs and bricks logs. At this point, I'm

Re: [Gluster-users] Geo-replication and changelogs cleaning

Hi, I will be pleased to make a test with your fix. Cordialement, Regards, Lionel BEARD CLS - IT & Operations De : Aravinda VK Envoyé : mercredi 1 septembre 2021 15:58 À : Beard Lionel Cc : gluster-users@gluster.org; sacha...@redhat.com Objet : Re: [Gluster-users] Geo-replica

Re: [Gluster-users] Geo-replication and changelogs cleaning

Hi, I think the “archive_gluster_changelogs” repo not updated after the backend changelogs are restructured(This Patch: https://github.com/gluster/glusterfs/commit/ec3df84dcfd7ccda0a18fa75e3b425c090209adf#diff-64c754d7b6ec77154042671072debc69456f43b3abc34354d5b818937635600f

Re: [Gluster-users] Geo-replication and changelogs cleaning

+aravi...@kadalu.io On Tue, Aug 31, 2021 at 6:26 PM Beard Lionel wrote: > Hi everyone, > > > > I am currently using GlusterFS 8.4 on Ubuntu Bionic for an application > hosted in Azure, with geo-replication configured for most of the volumes. > > But the way changelogs files are managed is a

Re: [Gluster-users] Geo-replication adding new master node

Hi Aravinda, We ran a "gluster system:: execute gsec_create" and "georep create push-pem" with force option as suggested, and then a "gluster volume geo-replication ... status" reported the two new master nodes as being in "Created" status. We did a geo-replication "stop" and then "start" and are

Re: [Gluster-users] Geo-replication adding new master node

Hi Aravinda, Thank you very much - we will give that a try. On Mon, 31 May 2021 at 20:29, Aravinda VK wrote: > Hi David, > > On 31-May-2021, at 10:37 AM, David Cunningham > wrote: > > Hello, > > We have a GlusterFS configuration with mirrored nodes on the master side > geo-replicating to

Re: [Gluster-users] Geo-replication adding new master node

Hi David, > On 31-May-2021, at 10:37 AM, David Cunningham > wrote: > > Hello, > > We have a GlusterFS configuration with mirrored nodes on the master side > geo-replicating to mirrored nodes on the secondary side. > > When geo-replication is initially created it seems to automatically add

Re: [Gluster-users] Geo-Replication - UnicodeEncodeError: 'utf-8' codec can't encode character '\udcfc' in position 78: surrogates not allowed

Hi Andreas, recently i have been faced with the same fault. I'm pretty sure you are speaking german, that's why a translation should not be necessary. I found the reason by tracing a certain process which points to the gsyncd.log and looking backward from the error until i found some

Re: [Gluster-users] Geo-Replication - UnicodeEncodeError: 'utf-8' codec can't encode character '\udcfc' in position 78: surrogates not allowed

Hi Dietmar, thank you for your reply. I've also started to trace this down and you are correct, the directory does contain filenames with 'special' characters (umlauts), but renaming them as a workaround unfortunately is not an option. So the question really is why does it fail on those

Re: [Gluster-users] Geo-replication status Faulty

If you can afford the extra space , set the logs to TRACE and after reasonable timeframe lower them back. Despite RH's gluster versioning is different - this thread should help:

Re: [Gluster-users] Geo-replication status Faulty

It could be a "simple" bug - software has bugs and regressions. I would recommend you to ping the debian mailing list - at least it won't hurt. Best Regards, Strahil Nikolov В вторник, 27 октомври 2020 г., 20:10:39 Гринуич+2, Gilberto Nunes написа: [SOLVED] Well... It seems to me

Re: [Gluster-users] Geo-replication status Faulty

Not so fast with my solution! After shutting the other node in the head, get FAULTY stat again... The only failure I saw in this thing regarding xattr value... [2020-10-27 19:20:07.718897] E [syncdutils(worker /DATA/vms):110:gf_mount_ready] : failed to get the xattr value Don't know if I am

Re: [Gluster-users] Geo-replication status Faulty

[SOLVED] Well... It seems to me that pure Debian Linux 10 has some problem with XFS, which is the FS that I used. It's not accept attr2 mount options. Interestingly enough, I have now used Proxmox 6.x, which is Debian based, I am now able to use the attr2 mount point option. Then the Faulty

Re: [Gluster-users] Geo-replication status Faulty

>> IIUC you're begging for split-brain ... Not at all! I have used this configuration and there isn't any split brain at all! But if I do not use it, then I get a split brain. Regarding count 2 I will see it! Thanks --- Gilberto Nunes Ferreira Em ter., 27 de out. de 2020 às 09:37, Diego

Re: [Gluster-users] Geo-replication status Faulty

Il 27/10/20 13:15, Gilberto Nunes ha scritto: > I have applied this parameters to the 2-node gluster: > gluster vol set VMS cluster.heal-timeout 10 > gluster volume heal VMS enable > gluster vol set VMS cluster.quorum-reads false > gluster vol set VMS cluster.quorum-count 1 Urgh! IIUC you're

Re: [Gluster-users] Geo-replication status Faulty

Hi Aravinda Let me thank you for that nice tools... It helps me a lot. And yes! Indeed I think this is the case, but why does gluster03 (which is the backup server) not continue since gluster02 are still online?? That puzzles me... --- Gilberto Nunes Ferreira Em ter., 27 de out. de 2020 às

Re: [Gluster-users] Geo-replication status Faulty

Dear Felix I have applied this parameters to the 2-node gluster: gluster vol set VMS cluster.heal-timeout 10 gluster volume heal VMS enable gluster vol set VMS cluster.quorum-reads false gluster vol set VMS cluster.quorum-count 1 gluster vol set VMS network.ping-timeout 2 gluster volume set VMS

Re: [Gluster-users] Geo-replication status Faulty

Hi Gilberto, Happy to see georepsetup tool is useful for you. The repo I moved to https://github.com/aravindavk/gluster-georep-tools (renamed as “gluster-georep-setup”). I think the georep command failure is due to respective node’s(peer)

Re: [Gluster-users] Geo-replication status Faulty

Dear Gilberto, If I am right, you ran into server-quorum if you startet a 2-node replica and shutdown one host. From my perspective, its fine. Please correct me if I am wrong here. Regards, Felix On 27/10/2020 01:46, Gilberto Nunes wrote: Well I do not reboot the host. I shut down the

Re: [Gluster-users] Geo-replication status Faulty

Usually there is always only 1 "master" , but when you power off one of the 2 nodes - the geo rep should handle that and the second node should take the job. How long did you wait after gluster1 has been rebooted ? Best Regards, Strahil Nikolov В понеделник, 26 октомври 2020 г., 22:46:21

Re: [Gluster-users] Geo-replication status Faulty

Well I do not reboot the host. I shut down the host. Then after 15 min give up. Don't know why that happened. I will try it latter --- Gilberto Nunes Ferreira Em seg., 26 de out. de 2020 às 21:31, Strahil Nikolov escreveu: > Usually there is always only 1 "master" , but when you power

Re: [Gluster-users] Geo-replication status Faulty

I was able to solve the issue restarting all servers. Now I have another issue! I just powered off the gluster01 server and then the geo-replication entered in faulty status. I tried to stop and start the gluster geo-replication like that: gluster volume geo-replication DATA

Re: [Gluster-users] Geo-replication log file not closed

Hello all, Apparently we don't want to "kill -HUP" the two processes that have rotated log file still open: root 4495 1 0 Aug10 ?00:00:59 /usr/bin/python2 /usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/gsyncd.py --path=/nodirectwritedata/gluster/gvol0 --monitor -c

Re: [Gluster-users] Geo-replication causes OOM

Thanks Strahil, Would the geo rep process be the gsyncd.py proceses? It seems like it's the glusterfsd and auxiliary mounts that are holding all the memory right now... Could this be related to the open-behind bug mentioned here: https://github.com/gluster/glusterfs/issues/1444  and here:

Re: [Gluster-users] Geo-replication causes OOM

Hey Matthew, Can you check with valgrind the memory leak ? It will be something like: Find the geo rep process via ps and note all parameters it was started with . Next stop geo rep. Then start it with valgrind : valgrind --log-file="filename" --tool=memcheck --leak-check=full It might

Re: [Gluster-users] Geo-replication completely broken

Hi Felix, It seems I missed your reply with the change log that Shwetha requested. Best Regards, Strahil Nikolov На 3 юли 2020 г. 11:16:30 GMT+03:00, "Felix Kölzow" написа: >Dear Users, >the geo-replication is still broken. This is not really a comfortable >situation. >Does any user has had

Re: [Gluster-users] Geo-replication completely broken

Dear Users, the geo-replication is still broken. This is not really a comfortable situation. Does any user has had the same experience and is able to share a possible workaround? We are actually running gluster v6.0 Regards, Felix On 25/06/2020 10:04, Shwetha Acharya wrote: Hi Rob and Felix,

Re: [Gluster-users] Geo-replication completely broken

Hi Rob and Felix, Please share the *-changes.log files and brick logs, which will help in analysis of the issue. Regards, Shwetha On Thu, Jun 25, 2020 at 1:26 PM Felix Kölzow wrote: > Hey Rob, > > > same issue for our third volume. Have a look at the logs just from right > now (below). > >

Re: [Gluster-users] Geo-replication completely broken

Hey Rob, same issue for our third volume. Have a look at the logs just from right now (below). Question: You removed the htime files and the old changelogs. Just rm the files or is there something to pay more attention before removing the changelog files and the htime file. Regards, Felix

Re: [Gluster-users] Geo-replication: Entry not present on master. Fixing gfid mismatch in slave

Hi Strahil, Thank you for that, and the point you can't 'cd' to the .gfid directory. I think the customer is going to live with the higher CPU usage as it's still well within acceptable limits, and other things demand our time. Thanks again for your input! On Fri, 12 Jun 2020 at 16:06,

Re: [Gluster-users] Geo-replication: Entry not present on master. Fixing gfid mismatch in slave

Hello David, The .gfid directory is there but you cannot traverse (cd) in it - you need to specify just like in the example.I had some cases where the 'transprt endpoint is not connected' was received, but usually this is due to a gfid missing. About the meetings, one of the topics is

Re: [Gluster-users] Geo-replication: Entry not present on master. Fixing gfid mismatch in slave

Hi Strahil, Is there a trick to getting the .gfid directory to appear besides adding "-o aux-gfid-mount" to the mount? I mounted it using "mount -t glusterfs -o aux-gfid-mount cafs30:/gvol0 /mnt/glusterfs" and there's no .gfid directory under /mnt/glusterfs. I haven't tried joining a gluster

Re: [Gluster-users] Geo-replication: Entry not present on master. Fixing gfid mismatch in slave

You can try the path of a file based on gfid (method 2) via: https://docs.gluster.org/en/latest/Troubleshooting/gfid-to-path/ The gfids from the strace should be there, but if the file was renamed/deleted - it is normall to be missing. Have you joined the last gluster meeting to discuss the

Re: [Gluster-users] Geo-replication: Entry not present on master. Fixing gfid mismatch in slave

Hi Strahil, Thanks for that. I did search for a file with the gfid in the name, on both the master nodes and geo-replication slave, but none of them had such a file. I guess maybe by the time I looked the file had been deleted? Either that or something is more seriously wrong with invalid gfids.

Re: [Gluster-users] Geo-replication: Entry not present on master. Fixing gfid mismatch in slave

Hey David, Sadly I just have a feeling that on any brick there is a gfid mismatch, but I could be wrong. As you have the gfid list, please check on all bricks (both master and slave) that the file exists (not the one in .gluster , but the real one) and it has the same gfid. You can

Re: [Gluster-users] Geo-replication: Entry not present on master. Fixing gfid mismatch in slave

Hi Strahil, Thank you for that. Do you know if these "Stale file handle" errors on the geo-replication slave could be related? [2020-06-10 01:02:32.268989] E [MSGID: 109040] [dht-helper.c:1332:dht_migration_complete_check_task] 0-gvol0-dht: /.gfid/d4265a0c-d881-48d8-8ca1-0920ab5ae9ba: failed to

Re: [Gluster-users] Geo-replication: Entry not present on master. Fixing gfid mismatch in slave

Hey David, Can you check the cpu usage in the sar on the rest of the cluster (going backwards from the day you found the high cpu usage), so we can know if this behaviour was obseerved on other nodes. Maybe that behaviour was "normal" for the push node (which could be another one) . As

Re: [Gluster-users] Geo-replication: Entry not present on master. Fixing gfid mismatch in slave

Hi Sankarshan, Thanks for that. So what should we look for to figure out what this process is doing? In /var/log/glusterfs/geo-replication/gvol0_nvfs10_gvol0/gsyncd.log we see something like the following logged regularly: [[2020-06-09 02:01:19.670595] D [master(worker

Re: [Gluster-users] Geo-replication: Entry not present on master. Fixing gfid mismatch in slave

Reading through the thread it occurs to me that it would be a stronger approach to understand the workload (a general description of the application) and in terms of the releases of GlusterFS running, assess if there are new issues to be addressed or if existing sets of patches tend to work.

Re: [Gluster-users] Geo-replication: Entry not present on master. Fixing gfid mismatch in slave

Hi Strahil, The CPU is still quite high, with "top" regularly showing 100% CPU usage by that process. However it's not clear whether this is really a problem, or if it's just normal geo-replication activity. While CPU usage was not previously as high on this server, it's not clear whether

Re: [Gluster-users] Geo-replication: Entry not present on master. Fixing gfid mismatch in slave

Hey David, can you check the old logs for gfid mismatch and get a list of files that were causing the high cpu . Maybe they are related somehow (maybe created by the same software , same client version or something else) which could help about that. Also take a look in

Re: [Gluster-users] Geo-replication: Entry not present on master. Fixing gfid mismatch in slave

Hi Sunny and Strahil, Thanks again for your responses. We don't have a lot of renaming activity - maybe some, but not a lot. We do have files which are open for writing for quite a while - they're call recordings being written as the call happens. We've installed GlusterFS using the Ubuntu

Re: [Gluster-users] Geo-replication: Entry not present on master. Fixing gfid mismatch in slave

Hi David, in which log do you see the entries ? I think I got an explanation why you see the process only on one of the master nodes - geo-rep session is established from only 1 master node /I hope someone corrects me if I'm wrong/ to one slave node. Thus it will be natural to see the high

Re: [Gluster-users] Geo-replication: Entry not present on master. Fixing gfid mismatch in slave

Hi David, You haven't answered my previous question regarding the type of your workload. --- You can use the below command to enable debug log. `gluster vol geo-rep :: config log-level DEBUG` and after capturing log again switch back to info mode: `gluster vol geo-rep :: config log-level

Re: [Gluster-users] Geo-replication: Entry not present on master. Fixing gfid mismatch in slave

Hi Strahil and Sunny, Thank you for the replies. I checked the gfid on the master and slaves and they are the same. After moving the file away and back again it doesn't seem to be having the issue with that file any more. We are still getting higher CPU usage on one of the master nodes than the

Re: [Gluster-users] Geo-replication: Entry not present on master. Fixing gfid mismatch in slave

Hi David, Looks like you are running a workload that involves lots of rename and geo-rep is trying to handle those. you can try below patches which will give you performance benefits. [1]. https://review.gluster.org/#/c/glusterfs/+/23570/ [2]. https://review.gluster.org/#/c/glusterfs/+/23459/

Re: [Gluster-users] Geo-replication: Entry not present on master. Fixing gfid mismatch in slave

Hey David, for me a gfid mismatch means that the file was replaced/recreated - just like vim in linux does (and it is expected for config file). Have you checked the gfid of the file on both source and destination, do they really match or they are different ? What happens when you

Re: [Gluster-users] Geo-Replication File not Found on /.glusterfs/XX/XX/XXXXXXXXXXXX

Hi, I have verified that the system is read-only, it does not let me delete or create files inside the slave volume. I send you logs of what I have before stopping the Geo-replication. Archivos.log -- [2020-03-18 20:47:57.950339] I [MSGID: 100030] [glusterfsd.c:2867:main] 0-/

Re: [Gluster-users] Geo-Replication File not Found on /.glusterfs/XX/XX/XXXXXXXXXXXX

Hi Senén, By any chance you perform any operation on slave volume; like deleting data directly from slave volume. Also If possible please share geo-rep slave logs. /sunny On Wed, Mar 25, 2020 at 9:15 AM Senén Vidal Blanco wrote: > > Hi, > I have a problem with the Geo-Replication system. >

Re: [Gluster-users] geo-replication sync issue

On March 18, 2020 1:41:15 PM GMT+02:00, "Etem Bayoğlu" wrote: >Yes I had tried.. my observation in my issue is : glusterfs crawler did >not >exit from a specific directory that had been synced already. Like a >infinite loop. It was crawling that directory endlessly. I tried so >many >things an

Re: [Gluster-users] geo-replication sync issue

Yes I had tried.. my observation in my issue is : glusterfs crawler did not exit from a specific directory that had been synced already. Like a infinite loop. It was crawling that directory endlessly. I tried so many things an time goes on. So I gave up and switched to nfs + rsync for now. This

Re: [Gluster-users] geo-replication sync issue

Could you try disabling syncing xattrs and check ? gluster vol geo-rep :: config sync-xattrs false On Fri, Mar 13, 2020 at 1:42 AM Strahil Nikolov wrote: > On March 12, 2020 9:41:45 AM GMT+02:00, "Etem Bayoğlu" < > etembayo...@gmail.com> wrote: > >Hello again, > > > >These are gsyncd.log from

Re: [Gluster-users] geo-replication sync issue

On March 12, 2020 9:41:45 AM GMT+02:00, "Etem Bayoğlu" wrote: >Hello again, > >These are gsyncd.log from master on DEBUG level. It tells entering >directory, synced files , and gfid information > >[2020-03-12 07:18:16.702286] D [master(worker >/srv/media-storage):324:regjob] _GMaster: synced

Re: [Gluster-users] geo-replication sync issue

Hello again, These are gsyncd.log from master on DEBUG level. It tells entering directory, synced files , and gfid information [2020-03-12 07:18:16.702286] D [master(worker /srv/media-storage):324:regjob] _GMaster: synced file=.gfid/358fe62c-c7e8-449a-90dd-1cc1a3b7a346 [2020-03-12

Re: [Gluster-users] geo-replication sync issue

Hi, here my slave node logs at the time sync stopped: [2020-03-08 03:33:01.489559] I [glusterfsd-mgmt.c:2282:mgmt_getspec_cbk] 0-glusterfs: No change in volfile,continuing [2020-03-08 03:33:01.489298] I [MSGID: 100011] [glusterfsd.c:1679:reincarnate] 0-glusterfsd: Fetching the volume file from

Re: [Gluster-users] geo-replication sync issue

On March 11, 2020 10:17:05 PM GMT+02:00, "Etem Bayoğlu" wrote: >Hi Strahil, > >Thank you for your response. when I tail logs on both master and slave >I >get this: > >on slave, from >/var/log/glusterfs/geo-replication-slaves//mnt-XXX.log >file: > >[2020-03-11 19:53:32.721509] E

Re: [Gluster-users] geo-replication sync issue

Hi Strahil, Thank you for your response. when I tail logs on both master and slave I get this: on slave, from /var/log/glusterfs/geo-replication-slaves//mnt-XXX.log file: [2020-03-11 19:53:32.721509] E [fuse-bridge.c:227:check_and_dump_fuse_W] (-->

Re: [Gluster-users] geo-replication sync issue

On March 11, 2020 10:09:27 AM GMT+02:00, "Etem Bayoğlu" wrote: >Hello community, > >I've set up a glusterfs geo-replication node for disaster recovery. I >manage about 10TB media data on a gluster volume and I want to sync all >data to remote location over WAN. So, I created a slave node volume

Re: [Gluster-users] Geo-replication

Hi Aravinda and Strahil, The cluster is new so it wasn't a big deal to re-do using public addresses. That's done and geo-replication is working. Thank you for your help! On Wed, 4 Mar 2020 at 17:17, Aravinda VK wrote: > Hi David, > > I like the Strahil’s idea of adding remote IPs in

Re: [Gluster-users] Geo-replication

Hi David, I like the Strahil’s idea of adding remote IPs in /etc/hosts with same name as used in B cluster. Since Geo-replication uses ssh for syncing it should work. Only issue I can think about is if the hostname of cluster B conflicts with hostnames of Cluster A. — regards Aravinda

Re: [Gluster-users] Geo-replication

Hi Strahil, The B cluster are communicating with each other via a LAN, and it seems the A cluster has got B's LAN addresses (which aren't accessible from the internet including the A cluster) through the geo-replication process. That being the case, I think we'll have to re-do the B cluster to

Re: [Gluster-users] Geo-replication

On March 3, 2020 4:13:38 AM GMT+02:00, David Cunningham wrote: >Hello, > >Thanks for that. When we re-tried with push-pem from cafs10 (on the >A/master cluster) it failed with "Unable to mount and fetch slave >volume >details." and in the logs we see: > >[2020-03-03 02:07:42.614911] E

Re: [Gluster-users] Geo-replication

Hello, Thanks for that. When we re-tried with push-pem from cafs10 (on the A/master cluster) it failed with "Unable to mount and fetch slave volume details." and in the logs we see: [2020-03-03 02:07:42.614911] E [name.c:258:af_inet_client_get_remote_sockaddr] 0-gvol0-client-0: DNS resolution

Re: [Gluster-users] Geo-replication

Looks like setup issue to me. Copying SSH keys manually is not required. Command prefix is required while adding to authorized_keys file in each remote nodes. That will not be available if ssh keys are added manually. Geo-rep specifies /nonexisting/gsyncd in the command to make sure it

Re: [Gluster-users] Geo-replication

On March 2, 2020 2:33:10 AM GMT+02:00, David Cunningham wrote: >Hello, > >We've set up geo-replication but it isn't actually syncing. Scenario is >that we have two GFS clusters. Cluster A has nodes cafs10, cafs20, and >cafs30, replicating with each other over a LAN. Cluster B has nodes >nvfs10,

Re: [Gluster-users] Geo-replication

Hello, We've set up geo-replication but it isn't actually syncing. Scenario is that we have two GFS clusters. Cluster A has nodes cafs10, cafs20, and cafs30, replicating with each other over a LAN. Cluster B has nodes nvfs10, nvfs20, and nvfs30 also replicating with each other over a LAN. We are

Re: [Gluster-users] Geo-replication

Hi Aravinda and Sunny, Thank you for the replies. We have 3 replicating nodes on the master side, and want to geo-replicate their data to the remote slave side. As I understand it if the master node which had the geo-replication create command run goes down then another node will take over

Re: [Gluster-users] Geo-replication

Hi David, > On 25-Feb-2020, at 3:45 AM, David Cunningham > wrote: > > Hello, > > I've a couple of questions on geo-replication that hopefully someone can help > with: > > 1. If there are multiple nodes in a cluster on the master side (pushing > updates to the geo-replication slave), which

Re: [Gluster-users] Geo-replication /var/lib space question

All '.processed' directories (under working_dir and working_dir/.history) contain processed changelogs and is no longer required by geo-replication apart from debugging purposes. That directory can be cleaned up if it's consuming too much space. On Wed, Feb 12, 2020 at 11:36 PM Sunny Kumar

Re: [Gluster-users] Geo-replication /var/lib space question

Hi Alexander, Yes that is geo-replication working directory and you can run the below command to get the location. gluster vol geo-rep :: config working_dir This directory contains parsed changelogs from backend brick which are ready to be processed. After a batch is processed it will be

Re: [Gluster-users] Geo-Replication Issue while upgrading

Thanks Deepu. I will investigate this can you just summarize the steps which can be helpful in reproducing this issue. /sunny On Fri, Nov 29, 2019 at 7:29 AM deepu srinivasan wrote: > > Hi Sunny > The issue seems to be a bug. > The issue got fixed when I restarted the glusterd daemon in the

Re: [Gluster-users] Geo-Replication Issue while upgrading

Hi Deepu, Can you try this: ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 sas@192.168.185.118 "sudo gluster volume status" /sunny On Thu, Nov 28, 2019 at 12:14 PM deepu srinivasan wrote: >> >> MASTER NODEMASTER VOL

Re: [Gluster-users] Geo-Replication Issue while upgrading

Hi Deepu, Looks like this is error generated due to ssh restrictions: Can you please check and confirm ssh is properly configured? 2019-11-28 11:59:12.934436] E [syncdutils(worker /home/sas/gluster/data/code-misc):809:logerr] Popen: ssh>

Re: [Gluster-users] Geo-Replication what is transferred

As far as I know , when sharding is enabled - each shard will be synced separately, while a whole file will be transferred when sharding is not enabled. Is striping still supported ? I think sharding should be used. Best Regards, Strahil Nikolov On Sep 3, 2019 23:47, Petric Frank wrote: > >

Re: [Gluster-users] Geo-Replication what is transferred

Hi Petric, On Wed, Sep 4, 2019 at 2:23 AM Petric Frank wrote: > Hello, > > given a geo-replicated file of 20 GBytes in size. > > If one byte in this file is changed, what will be transferred ? > - the changed byte > - the block/sector the containing the changed byte > - the complete file >

Re: [Gluster-users] geo-replication won't start

Hi Lucian, Slave must be a gluster volume. Data from master volume gets replicated into the slave volume after creation of the geo-rep session. You can try creating the session again using the steps mentioned in this link https://docs.gluster.org/en/latest/Administrator%20Guide/Geo

Re: [Gluster-users] Geo Replication Failure: libgfchangelog.so: cannot open shared object file

n > > > > I will try a clean install on a new image and document it more precisely. > > > > Thanks, > > > > Cédric > > > > > > De : Andy Coates > Envoyé : mercredi 28 août 2019 05:18 > À : ROUVRAIS Cedric ResgBscRscDef > Cc : gluster-us

Re: [Gluster-users] Geo Replication Failure: libgfchangelog.so: cannot open shared object file

: Andy Coates Envoyé : mercredi 28 août 2019 05:18 À : ROUVRAIS Cedric ResgBscRscDef Cc : gluster-users@gluster.org Objet : Re: [Gluster-users] Geo Replication Failure: libgfchangelog.so: cannot open shared object file We saw this with 4.1.x RPM on OEL (can't recall which specific version

Re: [Gluster-users] Geo Replication Failure: libgfchangelog.so: cannot open shared object file

We saw this with 4.1.x RPM on OEL (can't recall which specific version and haven't checked if its fixed in later, at least up to 4.1.6), but the issue seemed to be it just wasn't symlinked for some reason, so we symlinked libgfchangelog.so to /lib64/libgfchangelog.so.0 Not sure if the python code

Re: [Gluster-users] Geo Replication Stop even after migratingto 5.6

Hi Guys Yes, I will try the root geo-rep setup and update you back. Meanwhile is there any procedure for the below-quoted info in the docs? > Synchronization is not complete > > *Description*: GlusterFS geo-replication did not synchronize the data > completely but the geo-replication status

Re: [Gluster-users] Geo Replication Stop even after migratingto 5.6

Hi Any updates on this On Thu, Jun 13, 2019 at 6:59 PM deepu srinivasan wrote: > > > -- Forwarded message - > From: deepu srinivasan > Date: Thu, Jun 13, 2019 at 5:43 PM > Subject: Geo Replication Stop even after migratingto 5.6 > To: , Kotresh Hiremath Ravishankar < >

Re: [Gluster-users] Geo Replication stops replicating

Hi Sunny Sorry, that was a typo. I used the following command. > gluster-mountbroker add code-misc sas > On Thu, Jun 6, 2019 at 6:23 PM Sunny Kumar wrote: > You should not have used this one: > > > > gluster-mountbroker remove --volume code-misc --user sas > > -- This one is to remove

Re: [Gluster-users] Geo Replication stops replicating

Hi Sunny Please find the logs attached > The message "E [MSGID: 106061] > [glusterd-mountbroker.c:555:glusterd_do_mount] 0-management: 'option > mountbroker-root' missing in glusterd vol file" repeated 13 times between > [2019-06-06 11:51:43.986788] and [2019-06-06 11:52:32.764546] > > The

  1   2   3   4   5   >