Hi Naranderan,
On Sat, May 30, 2020 at 3:52 PM Strahil Nikolov <hunter86...@yahoo.com> wrote: > > Hello Naranderan, > > what OS are you using ? Do you have SELINUX in enforcing mode (verify via > 'sestatus') ? > > Best Regards, > Strahil Nikolov > > > > > > > В събота, 30 май 2020 г., 13:33:05 ч. Гринуич+3, Naranderan Ramakrishnan > <rnara...@gmail.com> написа: > > > > > > Dear Developers/Users, > > A geo-rep session of a sub-volume is in 'faulty' status. Please find the > setup and log details below. > > Setup Details: > > Gluster version - 7.0 > > Volume configuration - 2x3 (DxR) > > gysncd permission(master) - root > > gysncd permission(slave) - sas (non-root) > > glusterd, glusterfsd permissions(master) - root > > glusterd, glusterfsd permissions(slave) - root > > Log details: > In the master gyncd log, this traceback is printed repeatedly. > > [2020-05-22 12:09:43.838727] I [master(worker > > /home/sas/gluster/data/code-ide):1991:syncjob] Syncer: Sync Time Taken > > duration=0.4240 num_files=1 job=1 return_code=0 > > [2020-05-22 12:09:43.944392] E [repce(worker > > /home/sas/gluster/data/code-ide):214:__call__] RepceClient: call failed > > call=261471:140535761106752:1590149383.8 method=entry_ops error=OSError > > [2020-05-22 12:09:43.944746] E [syncdutils(worker > > /home/sas/gluster/data/code-ide):338:log_raise_exception] <top>: FAIL: > > Traceback (most recent call last): > > File "/usr/libexec/glusterfs/python/syncdaemon/gsyncd.py", line 325, in > > main > > func(args) > > File "/usr/libexec/glusterfs/python/syncdaemon/subcmds.py", line 86, in > > subcmd_worker > > local.service_loop(remote) > > File "/usr/libexec/glusterfs/python/syncdaemon/resource.py", line 1305, > > in service_loop > > g3.crawlwrap(oneshot=True) > > File "/usr/libexec/glusterfs/python/syncdaemon/master.py", line 602, in > > crawlwrap > > self.crawl() > > File "/usr/libexec/glusterfs/python/syncdaemon/master.py", line 1592, in > > crawl > > self.changelogs_batch_process(changes) > > File "/usr/libexec/glusterfs/python/syncdaemon/master.py", line 1492, in > > changelogs_batch_process > > self.process(batch) > > File "/usr/libexec/glusterfs/python/syncdaemon/master.py", line 1327, in > > process > > self.process_change(change, done, retry) > > File "/usr/libexec/glusterfs/python/syncdaemon/master.py", line 1221, in > > process_change > > failures = self.slave.server.entry_ops(entries) > > File "/usr/libexec/glusterfs/python/syncdaemon/repce.py", line 233, in > > __call__ > > return self.ins(self.meth, *a) > > File "/usr/libexec/glusterfs/python/syncdaemon/repce.py", line 215, in > > __call__ > > raise res > > OSError: [Errno 13] Permission denied: > > '/home/sas/gluster/data/code-ide/.glusterfs/c2/bf/c2bff066-b10e-468a-a67e-b8b501a8951e' > > [2020-05-22 12:09:43.968710] I [repce(agent > > /home/sas/gluster/data/code-ide):97:service_loop] RepceServer: terminating > > on reaching EOF. > > [2020-05-22 12:09:44.912470] I [monitor(monitor):280:monitor] Monitor: > > worker died in startup phase brick=/home/sas/gluster/data/code-ide > > [2020-05-22 12:09:44.913692] I > > [gsyncdstatus(monitor):248:set_worker_status] GeorepStatus: Worker Status > > Change status=Faulty > > In salve end, these are printed repeatedly. > > > > [2020-05-22 11:23:26.65115] W [gsyncd(slave > > 10.47.8.153/home/sas/gluster/data/code-ide):307:main] <top>: Session config > > file not exists, using the default config > > path=/var/lib/glusterd/geo-replication/code-ide_10.37.11.252_code-ide/gsyncd.conf > > [2020-05-22 11:23:26.77414] I [resource(slave > > 10.47.8.153/home/sas/gluster/data/code-ide):1105:connect] GLUSTER: Mounting > > gluster volume locally... > > [2020-05-22 11:23:27.297466] I [resource(slave > > 10.47.8.153/home/sas/gluster/data/code-ide):1128:connect] GLUSTER: Mounted > > gluster volume duration=1.2199 > > [2020-05-22 11:23:27.298125] I [resource(slave > > 10.47.8.153/home/sas/gluster/data/code-ide):1155:service_loop] GLUSTER: > > slave listening > > [2020-05-22 11:23:32.654939] E [repce(slave > > 10.47.8.153/home/sas/gluster/data/code-ide):122:worker] <top>: call failed: > > Traceback (most recent call last): > > File "/usr/libexec/glusterfs/python/syncdaemon/repce.py", line 118, in > > worker > > res = getattr(self.obj, rmeth)(*in_data[2:]) > > File "/usr/libexec/glusterfs/python/syncdaemon/resource.py", line 706, in > > entry_ops > > collect_failure(e, cmd_ret, uid, gid) > > File "/usr/libexec/glusterfs/python/syncdaemon/resource.py", line 444, in > > collect_failure > > disk_gfid) > > File "/usr/libexec/glusterfs/python/syncdaemon/syncdutils.py", line 687, > > in get_slv_dir_path > > [ENOENT], [ESTALE]) > > File "/usr/libexec/glusterfs/python/syncdaemon/syncdutils.py", line 546, > > in errno_wrap > > return call(*arg) > > OSError: [Errno 13] Permission denied: > > '/home/sas/gluster/data/code-ide/.glusterfs/c2/bf/c2bff066-b10e-468a-a67e-b8b501a8951e' > > [2020-05-22 11:23:32.741317] I [repce(slave > > 10.47.8.153/home/sas/gluster/data/code-ide):97:service_loop] RepceServer: > > terminating on reaching EOF. > > > Additional info: > Parallel to this > GFID(/home/sas/gluster/data/code-ide/.glusterfs/c2/bf/c2bff066-b10e-468a-a67e-b8b501a8951e) > mentioned in master gyscnd log, there are some files with ---------T > permission & trusted.glusterfs.dht.linkto extended attribute in the master > subvolume for which geo-rep session is in faulty status. Not sure whether > this is related to this geo-rep issue or not. > These are linkto files which can be the result of rebalance/rename. It should not be creating any problem. > I have attached a few screenshots and log stmts for further info. Please let > us know how we should solve this. This issue has been fixed in master. You can find the patch link here[1]. It was backported with release-7 and should be available with the latest release. [1]. https://review.gluster.org/#/c/glusterfs/+/23584/. > Thanks in advance. > > Regards, > Naranderan R > > ________ > > > > Community Meeting Calendar: > > Schedule - > Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC > Bridge: https://bluejeans.com/441850968 > > Gluster-users mailing list > Gluster-users@gluster.org > https://lists.gluster.org/mailman/listinfo/gluster-users > ________ > > > > Community Meeting Calendar: > > Schedule - > Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC > Bridge: https://bluejeans.com/441850968 > > Gluster-users mailing list > Gluster-users@gluster.org > https://lists.gluster.org/mailman/listinfo/gluster-users ________ Community Meeting Calendar: Schedule - Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC Bridge: https://bluejeans.com/441850968 Gluster-users mailing list Gluster-users@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-users
