Above error can be tracked here:

https://bugzilla.redhat.com/show_bug.cgi?id=1709248

and patch link:
https://review.gluster.org/#/c/glusterfs/+/22716/

You can apply patch and test it however its waiting on regression to
pass and merge.

-Sunny


On Thu, Jun 6, 2019 at 4:00 PM deepu srinivasan <[email protected]> wrote:
>
> Hi
> I have followed the following steps to create the geo-replication but the 
> status seems to be in a faulty state.
>
> Steps :
>
> Installed cluster version 5.6 in totally six nodes.
>>
>> glusterfs 5.6
>>
>> Repository revision: git://git.gluster.org/glusterfs.git
>>
>> Copyright (c) 2006-2016 Red Hat, Inc. <https://www.gluster.org/>
>>
>> GlusterFS comes with ABSOLUTELY NO WARRANTY.
>>
>> It is licensed to you under your choice of the GNU Lesser
>>
>> General Public License, version 3 or any later version (LGPLv3
>>
>> or later), or the GNU General Public License, version 2 (GPLv2),
>>
>> in all cases as published by the Free Software Foundation
>
>
> peer_probed the first three nodes and second three nodes.
>
>
>
> Added new volume in both the clusters
>
>
>
> execute gluster-mountbroker commands and restarted glusterd.
>>
>> gluster-mountbroker setup /var/mountbroker-root sas
>>
>> gluster-mountbroker remove --volume code-misc --user sas
>
>
> configured a passwordless sssh from master to slave
>>
>> ssh-keygen; ssh-copy-id [email protected]
>
> created a common pem pub file
>>
>> gluster system:: execute gsec_create
>
> created geo-replication session.
>>
>> gluster volume geo-replication code-misc [email protected]::code-misc 
>> create push-pem
>
>  executed the following command in slave
>>
>> /usr/libexec/glusterfs/set_geo_rep_pem_keys.sh sas code-misc code-misc
>
> started the gluster geo-replication.
>>
>> gluster volume geo-replication code-misc [email protected]::code-misc start
>
>
> Now the geo-replication works fine.
> Tested with 2000 files All seems to sync finely.
>
> Now I updated all the node to version 6.2 by using rpms which were built by 
> the source code in a docker container in my personal machine.
>
>
>> gluster --version
>>
>> glusterfs 6.2
>>
>> Repository revision: git://git.gluster.org/glusterfs.git
>>
>> Copyright (c) 2006-2016 Red Hat, Inc. <https://www.gluster.org/>
>>
>> GlusterFS comes with ABSOLUTELY NO WARRANTY.
>>
>> It is licensed to you under your choice of the GNU Lesser
>>
>> General Public License, version 3 or any later version (LGPLv3
>>
>> or later), or the GNU General Public License, version 2 (GPLv2),
>>
>> in all cases as published by the Free Software Foundation.
>
>
> I have stopped the glusterd daemons in all the node along with the volume and 
> geo-replication.
> Now I started the daemons, volume and geo-replication session the status 
> seems to be faulty.
> Also noted that the result of "gluster-mountbroker status" command always end 
> in python exception like this
>>
>> Traceback (most recent call last):
>>
>>   File "/usr/sbin/gluster-mountbroker", line 396, in <module>
>>
>>     runcli()
>>
>>   File "/usr/lib/python2.7/site-packages/gluster/cliutils/cliutils.py", line 
>> 225, in runcli
>>
>>     cls.run(args)
>>
>>   File "/usr/sbin/gluster-mountbroker", line 275, in run
>>
>>     out = execute_in_peers("node-status")
>>
>>   File "/usr/lib/python2.7/site-packages/gluster/cliutils/cliutils.py", line 
>> 127, in execute_in_peers
>>
>>     raise GlusterCmdException((rc, out, err, " ".join(cmd)))
>>
>> gluster.cliutils.cliutils.GlusterCmdException: (1, '', 'Unable to end. Error 
>> : Success\n', 'gluster system:: execute mountbroker.py node-status')
>
>
> Is it I or everyone gets an error for gluster-mountbroker command for gluster 
> version greater than 6.0?. Please help.
>
> Thank you
> Deepak
>
>
> On Thu, Jun 6, 2019 at 10:35 AM Sunny Kumar <[email protected]> wrote:
>>
>> Hi,
>>
>> Updated link for documentation :
>>
>> --  
>> https://docs.gluster.org/en/latest/Administrator%20Guide/Geo%20Replication/
>>
>> You can use this tool as well:
>> http://aravindavk.in/blog/gluster-georep-tools/
>>
>> -Sunny
>>
>> On Thu, Jun 6, 2019 at 10:29 AM Kotresh Hiremath Ravishankar
>> <[email protected]> wrote:
>> >
>> > Hi,
>> >
>> > I think the steps to setup non-root geo-rep is not followed properly. The 
>> > following entry is missing in glusterd vol file which is required.
>> >
>> > The message "E [MSGID: 106061] 
>> > [glusterd-mountbroker.c:555:glusterd_do_mount] 0-management: 'option 
>> > mountbroker-root' missing in glusterd vol file" repeated 33 times between 
>> > [2019-06-05 08:50:46.361384] and [2019-06-05 08:52:34.019757]
>> >
>> > Could you please the steps from below?
>> >
>> > https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3.4/html-single/administration_guide/index#Setting_Up_the_Environment_for_a_Secure_Geo-replication_Slave
>> >
>> > And let us know if you still face the issue.
>> >
>> >
>> >
>> >
>> > On Thu, Jun 6, 2019 at 10:24 AM deepu srinivasan <[email protected]> 
>> > wrote:
>> >>
>> >> Hi Kotresh, Sunny
>> >> I Have mailed the logs I found in one of the slave machines. Is there 
>> >> anything to do with permission? Please help.
>> >>
>> >> On Wed, Jun 5, 2019 at 2:28 PM deepu srinivasan <[email protected]> 
>> >> wrote:
>> >>>
>> >>> Hi Kotresh, Sunny
>> >>> Found this log in the slave machine.
>> >>>>
>> >>>> [2019-06-05 08:49:10.632583] I [MSGID: 106488] 
>> >>>> [glusterd-handler.c:1559:__glusterd_handle_cli_get_volume] 
>> >>>> 0-management: Received get vol req
>> >>>>
>> >>>> The message "I [MSGID: 106488] 
>> >>>> [glusterd-handler.c:1559:__glusterd_handle_cli_get_volume] 
>> >>>> 0-management: Received get vol req" repeated 2 times between 
>> >>>> [2019-06-05 08:49:10.632583] and [2019-06-05 08:49:10.670863]
>> >>>>
>> >>>> The message "I [MSGID: 106496] 
>> >>>> [glusterd-handler.c:3187:__glusterd_handle_mount] 0-glusterd: Received 
>> >>>> mount req" repeated 34 times between [2019-06-05 08:48:41.005398] and 
>> >>>> [2019-06-05 08:50:37.254063]
>> >>>>
>> >>>> The message "E [MSGID: 106061] 
>> >>>> [glusterd-mountbroker.c:555:glusterd_do_mount] 0-management: 'option 
>> >>>> mountbroker-root' missing in glusterd vol file" repeated 34 times 
>> >>>> between [2019-06-05 08:48:41.005434] and [2019-06-05 08:50:37.254079]
>> >>>>
>> >>>> The message "W [MSGID: 106176] 
>> >>>> [glusterd-mountbroker.c:719:glusterd_do_mount] 0-management: 
>> >>>> unsuccessful mount request [No such file or directory]" repeated 34 
>> >>>> times between [2019-06-05 08:48:41.005444] and [2019-06-05 
>> >>>> 08:50:37.254080]
>> >>>>
>> >>>> [2019-06-05 08:50:46.361347] I [MSGID: 106496] 
>> >>>> [glusterd-handler.c:3187:__glusterd_handle_mount] 0-glusterd: Received 
>> >>>> mount req
>> >>>>
>> >>>> [2019-06-05 08:50:46.361384] E [MSGID: 106061] 
>> >>>> [glusterd-mountbroker.c:555:glusterd_do_mount] 0-management: 'option 
>> >>>> mountbroker-root' missing in glusterd vol file
>> >>>>
>> >>>> [2019-06-05 08:50:46.361419] W [MSGID: 106176] 
>> >>>> [glusterd-mountbroker.c:719:glusterd_do_mount] 0-management: 
>> >>>> unsuccessful mount request [No such file or directory]
>> >>>>
>> >>>> The message "I [MSGID: 106496] 
>> >>>> [glusterd-handler.c:3187:__glusterd_handle_mount] 0-glusterd: Received 
>> >>>> mount req" repeated 33 times between [2019-06-05 08:50:46.361347] and 
>> >>>> [2019-06-05 08:52:34.019741]
>> >>>>
>> >>>> The message "E [MSGID: 106061] 
>> >>>> [glusterd-mountbroker.c:555:glusterd_do_mount] 0-management: 'option 
>> >>>> mountbroker-root' missing in glusterd vol file" repeated 33 times 
>> >>>> between [2019-06-05 08:50:46.361384] and [2019-06-05 08:52:34.019757]
>> >>>>
>> >>>> The message "W [MSGID: 106176] 
>> >>>> [glusterd-mountbroker.c:719:glusterd_do_mount] 0-management: 
>> >>>> unsuccessful mount request [No such file or directory]" repeated 33 
>> >>>> times between [2019-06-05 08:50:46.361419] and [2019-06-05 
>> >>>> 08:52:34.019758]
>> >>>>
>> >>>> [2019-06-05 08:52:44.426839] I [MSGID: 106496] 
>> >>>> [glusterd-handler.c:3187:__glusterd_handle_mount] 0-glusterd: Received 
>> >>>> mount req
>> >>>>
>> >>>> [2019-06-05 08:52:44.426886] E [MSGID: 106061] 
>> >>>> [glusterd-mountbroker.c:555:glusterd_do_mount] 0-management: 'option 
>> >>>> mountbroker-root' missing in glusterd vol file
>> >>>>
>> >>>> [2019-06-05 08:52:44.426896] W [MSGID: 106176] 
>> >>>> [glusterd-mountbroker.c:719:glusterd_do_mount] 0-management: 
>> >>>> unsuccessful mount request [No such file or directory]
>> >>>
>> >>>
>> >>> On Wed, Jun 5, 2019 at 1:06 AM deepu srinivasan <[email protected]> 
>> >>> wrote:
>> >>>>
>> >>>> Thankyou Kotresh
>> >>>>
>> >>>> On Tue, Jun 4, 2019, 11:20 PM Kotresh Hiremath Ravishankar 
>> >>>> <[email protected]> wrote:
>> >>>>>
>> >>>>> Ccing Sunny, who was investing similar issue.
>> >>>>>
>> >>>>> On Tue, Jun 4, 2019 at 5:46 PM deepu srinivasan <[email protected]> 
>> >>>>> wrote:
>> >>>>>>
>> >>>>>> Have already added the path in bashrc . Still in faulty state
>> >>>>>>
>> >>>>>> On Tue, Jun 4, 2019, 5:27 PM Kotresh Hiremath Ravishankar 
>> >>>>>> <[email protected]> wrote:
>> >>>>>>>
>> >>>>>>> could you please try adding /usr/sbin to $PATH for user 'sas'? If 
>> >>>>>>> it's bash, add 'export PATH=/usr/sbin:$PATH' in
>> >>>>>>> /home/sas/.bashrc
>> >>>>>>>
>> >>>>>>> On Tue, Jun 4, 2019 at 5:24 PM deepu srinivasan <[email protected]> 
>> >>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>> Hi Kortesh
>> >>>>>>>> Please find the logs of the above error
>> >>>>>>>> Master log snippet
>> >>>>>>>>>
>> >>>>>>>>> [2019-06-04 11:52:09.254731] I [resource(worker 
>> >>>>>>>>> /home/sas/gluster/data/code-misc):1379:connect_remote] SSH: 
>> >>>>>>>>> Initializing SSH connection between master and slave...
>> >>>>>>>>>  [2019-06-04 11:52:09.308923] D [repce(worker 
>> >>>>>>>>> /home/sas/gluster/data/code-misc):196:push] RepceClient: call 
>> >>>>>>>>> 89724:139652759443264:1559649129.31 __repce_version__() ...
>> >>>>>>>>>  [2019-06-04 11:52:09.602792] E [syncdutils(worker 
>> >>>>>>>>> /home/sas/gluster/data/code-misc):311:log_raise_exception] <top>: 
>> >>>>>>>>> connection to peer is broken
>> >>>>>>>>>  [2019-06-04 11:52:09.603312] E [syncdutils(worker 
>> >>>>>>>>> /home/sas/gluster/data/code-misc):805:errlog] Popen: command 
>> >>>>>>>>> returned error   cmd=ssh -oPasswordAuthentication=no 
>> >>>>>>>>> -oStrictHostKeyChecking=no -i /var/lib/ 
>> >>>>>>>>> glusterd/geo-replication/secret.pem -p 22 -oControlMaster=auto -S 
>> >>>>>>>>> /tmp/gsyncd-aux-ssh-4aL2tc/d893f66e0addc32f7d0080bb503f5185.sock 
>> >>>>>>>>> [email protected] /usr/libexec/glusterfs/gsyncd slave code-misc 
>> >>>>>>>>> sas@   192.168.185.107::code-misc --master-node 192.168.185.106 
>> >>>>>>>>> --master-node-id 851b64d0-d885-4ae9-9b38-ab5b15db0fec 
>> >>>>>>>>> --master-brick /home/sas/gluster/data/code-misc --local-node 
>> >>>>>>>>> 192.168.185.122 --local-node-   id 
>> >>>>>>>>> bcaa7af6-c3a1-4411-8e99-4ebecb32eb6a --slave-timeout 120 
>> >>>>>>>>> --slave-log-level DEBUG --slave-gluster-log-level INFO 
>> >>>>>>>>> --slave-gluster-command-dir /usr/sbin   error=1
>> >>>>>>>>>  [2019-06-04 11:52:09.614996] I [repce(agent 
>> >>>>>>>>> /home/sas/gluster/data/code-misc):97:service_loop] RepceServer: 
>> >>>>>>>>> terminating on reaching EOF.
>> >>>>>>>>>  [2019-06-04 11:52:09.615545] D [monitor(monitor):271:monitor] 
>> >>>>>>>>> Monitor: worker(/home/sas/gluster/data/code-misc) connected
>> >>>>>>>>>  [2019-06-04 11:52:09.616528] I [monitor(monitor):278:monitor] 
>> >>>>>>>>> Monitor: worker died in startup phase 
>> >>>>>>>>> brick=/home/sas/gluster/data/code-misc
>> >>>>>>>>>  [2019-06-04 11:52:09.619391] I 
>> >>>>>>>>> [gsyncdstatus(monitor):248:set_worker_status] GeorepStatus: Worker 
>> >>>>>>>>> Status Change status=Faulty
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> Slave log snippet
>> >>>>>>>>>
>> >>>>>>>>> [2019-06-04 11:50:09.782668] E [syncdutils(slave 
>> >>>>>>>>> 192.168.185.106/home/sas/gluster/data/code-misc):809:logerr] 
>> >>>>>>>>> Popen: /usr/sbin/gluster> 2 : failed with this errno (No such file 
>> >>>>>>>>> or directory)
>> >>>>>>>>> [2019-06-04 11:50:11.188167] W [gsyncd(slave 
>> >>>>>>>>> 192.168.185.125/home/sas/gluster/data/code-misc):305:main] <top>: 
>> >>>>>>>>> Session config file not exists, using the default config 
>> >>>>>>>>> path=/var/lib/glusterd/geo-replication/code-misc_192.168.185.107_code-misc/gsyncd.conf
>> >>>>>>>>> [2019-06-04 11:50:11.201070] I [resource(slave 
>> >>>>>>>>> 192.168.185.125/home/sas/gluster/data/code-misc):1098:connect] 
>> >>>>>>>>> GLUSTER: Mounting gluster volume locally...
>> >>>>>>>>> [2019-06-04 11:50:11.271231] E [resource(slave 
>> >>>>>>>>> 192.168.185.125/home/sas/gluster/data/code-misc):1006:handle_mounter]
>> >>>>>>>>>  MountbrokerMounter: glusterd answered mnt=
>> >>>>>>>>> [2019-06-04 11:50:11.271998] E [syncdutils(slave 
>> >>>>>>>>> 192.168.185.125/home/sas/gluster/data/code-misc):805:errlog] 
>> >>>>>>>>> Popen: command returned error cmd=/usr/sbin/gluster 
>> >>>>>>>>> --remote-host=localhost system:: mount sas user-map-root=sas 
>> >>>>>>>>> aux-gfid-mount acl log-level=INFO 
>> >>>>>>>>> log-file=/var/log/glusterfs/geo-replication-slaves/code-misc_192.168.185.107_code-misc/mnt-192.168.185.125-home-sas-gluster-data-code-misc.log
>> >>>>>>>>>  volfile-server=localhost volfile-id=code-misc client-pid=-1 
>> >>>>>>>>> error=1
>> >>>>>>>>> [2019-06-04 11:50:11.272113] E [syncdutils(slave 
>> >>>>>>>>> 192.168.185.125/home/sas/gluster/data/code-misc):809:logerr] 
>> >>>>>>>>> Popen: /usr/sbin/gluster> 2 : failed with this errno (No such file 
>> >>>>>>>>> or directory)
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> On Tue, Jun 4, 2019 at 5:10 PM deepu srinivasan 
>> >>>>>>>> <[email protected]> wrote:
>> >>>>>>>>>
>> >>>>>>>>> Hi
>> >>>>>>>>> As discussed I have upgraded gluster from 4.1 to 6.2 version. But 
>> >>>>>>>>> the Geo replication failed to start.
>> >>>>>>>>> Stays in faulty state
>> >>>>>>>>>
>> >>>>>>>>> On Fri, May 31, 2019, 5:32 PM deepu srinivasan 
>> >>>>>>>>> <[email protected]> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>> Checked the data. It remains in 2708. No progress.
>> >>>>>>>>>>
>> >>>>>>>>>> On Fri, May 31, 2019 at 4:36 PM Kotresh Hiremath Ravishankar 
>> >>>>>>>>>> <[email protected]> wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>> That means it could be working and the defunct process might be 
>> >>>>>>>>>>> some old zombie one. Could you check, that data progress ?
>> >>>>>>>>>>>
>> >>>>>>>>>>> On Fri, May 31, 2019 at 4:29 PM deepu srinivasan 
>> >>>>>>>>>>> <[email protected]> wrote:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Hi
>> >>>>>>>>>>>> When i change the rsync option the rsync process doesnt seem to 
>> >>>>>>>>>>>> start . Only a defunt process is listed in ps aux. Only when i 
>> >>>>>>>>>>>> set rsync option to " " and restart all the process the rsync 
>> >>>>>>>>>>>> process is listed in ps aux.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> On Fri, May 31, 2019 at 4:23 PM Kotresh Hiremath Ravishankar 
>> >>>>>>>>>>>> <[email protected]> wrote:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Yes, rsync config option should have fixed this issue.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Could you share the output of the following?
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> 1. gluster volume geo-replication <MASTERVOL> 
>> >>>>>>>>>>>>> <SLAVEHOST>::<SLAVEVOL> config rsync-options
>> >>>>>>>>>>>>> 2. ps -ef | grep rsync
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> On Fri, May 31, 2019 at 4:11 PM deepu srinivasan 
>> >>>>>>>>>>>>> <[email protected]> wrote:
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Done.
>> >>>>>>>>>>>>>> We got the following result .
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> 1559298781.338234 write(2, "rsync: link_stat 
>> >>>>>>>>>>>>>>> \"/tmp/gsyncd-aux-mount-EEJ_sY/.gfid/3fa6aed8-802e-4efe-9903-8bc171176d88\"
>> >>>>>>>>>>>>>>>  failed: No such file or directory (2)", 128
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> seems like a file is missing ?
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> On Fri, May 31, 2019 at 3:25 PM Kotresh Hiremath Ravishankar 
>> >>>>>>>>>>>>>> <[email protected]> wrote:
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Hi,
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Could you take the strace with with more string size? The 
>> >>>>>>>>>>>>>>> argument strings are truncated.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> strace -s 500 -ttt -T -p <rsync pid>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> On Fri, May 31, 2019 at 3:17 PM deepu srinivasan 
>> >>>>>>>>>>>>>>> <[email protected]> wrote:
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Hi Kotresh
>> >>>>>>>>>>>>>>>> The above-mentioned work around did not work properly.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> On Fri, May 31, 2019 at 3:16 PM deepu srinivasan 
>> >>>>>>>>>>>>>>>> <[email protected]> wrote:
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> Hi Kotresh
>> >>>>>>>>>>>>>>>>> We have tried the above-mentioned rsync option and we are 
>> >>>>>>>>>>>>>>>>> planning to have the version upgrade to 6.0.
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> On Fri, May 31, 2019 at 11:04 AM Kotresh Hiremath 
>> >>>>>>>>>>>>>>>>> Ravishankar <[email protected]> wrote:
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Hi,
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> This looks like the hang because stderr buffer filled up 
>> >>>>>>>>>>>>>>>>>> with errors messages and no one reading it.
>> >>>>>>>>>>>>>>>>>> I think this issue is fixed in latest releases. As a 
>> >>>>>>>>>>>>>>>>>> workaround, you can do following and check if it works.
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Prerequisite:
>> >>>>>>>>>>>>>>>>>>  rsync version should be > 3.1.0
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Workaround:
>> >>>>>>>>>>>>>>>>>> gluster volume geo-replication <MASTERVOL> 
>> >>>>>>>>>>>>>>>>>> <SLAVEHOST>::<SLAVEVOL> config rsync-options 
>> >>>>>>>>>>>>>>>>>> "--ignore-missing-args"
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>>>> Kotresh HR
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> On Thu, May 30, 2019 at 5:39 PM deepu srinivasan 
>> >>>>>>>>>>>>>>>>>> <[email protected]> wrote:
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Hi
>> >>>>>>>>>>>>>>>>>>> We were evaluating Gluster geo Replication between two 
>> >>>>>>>>>>>>>>>>>>> DCs one is in US west and one is in US east. We took 
>> >>>>>>>>>>>>>>>>>>> multiple trials for different file size.
>> >>>>>>>>>>>>>>>>>>> The Geo Replication tends to stop replicating but while 
>> >>>>>>>>>>>>>>>>>>> checking the status it appears to be in Active state. 
>> >>>>>>>>>>>>>>>>>>> But the slave volume did not increase in size.
>> >>>>>>>>>>>>>>>>>>> So we have restarted the geo-replication session and 
>> >>>>>>>>>>>>>>>>>>> checked the status. The status was in an active state 
>> >>>>>>>>>>>>>>>>>>> and it was in History Crawl for a long time. We have 
>> >>>>>>>>>>>>>>>>>>> enabled the DEBUG mode in logging and checked for any 
>> >>>>>>>>>>>>>>>>>>> error.
>> >>>>>>>>>>>>>>>>>>> There was around 2000 file appeared for syncing 
>> >>>>>>>>>>>>>>>>>>> candidate. The Rsync process starts but the rsync did 
>> >>>>>>>>>>>>>>>>>>> not happen in the slave volume. Every time the rsync 
>> >>>>>>>>>>>>>>>>>>> process appears in the "ps auxxx" list but the 
>> >>>>>>>>>>>>>>>>>>> replication did not happen in the slave end. What would 
>> >>>>>>>>>>>>>>>>>>> be the cause of this problem? Is there anyway to debug 
>> >>>>>>>>>>>>>>>>>>> it?
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> We have also checked the strace of the rync program.
>> >>>>>>>>>>>>>>>>>>> it displays something like this
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> "write(2, "rsync: link_stat \"/tmp/gsyncd-au"..., 128"
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> We are using the below specs
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Gluster version - 4.1.7
>> >>>>>>>>>>>>>>>>>>> Sync mode - rsync
>> >>>>>>>>>>>>>>>>>>> Volume - 1x3 in each end (master and slave)
>> >>>>>>>>>>>>>>>>>>> Intranet Bandwidth - 10 Gig
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>>>>> Thanks and Regards,
>> >>>>>>>>>>>>>>>>>> Kotresh H R
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>>> Thanks and Regards,
>> >>>>>>>>>>>>>>> Kotresh H R
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> --
>> >>>>>>>>>>>>> Thanks and Regards,
>> >>>>>>>>>>>>> Kotresh H R
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> --
>> >>>>>>>>>>> Thanks and Regards,
>> >>>>>>>>>>> Kotresh H R
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> --
>> >>>>>>> Thanks and Regards,
>> >>>>>>> Kotresh H R
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> Thanks and Regards,
>> >>>>> Kotresh H R
>> >
>> >
>> >
>> > --
>> > Thanks and Regards,
>> > Kotresh H R
_______________________________________________
Gluster-users mailing list
[email protected]
https://lists.gluster.org/mailman/listinfo/gluster-users

Reply via email to