It is clear that rsync is failing. Are the rsync versions on all masters and slave nodes same? I have seen that has caused problems sometimes.
-Kotresh HR On Wed, Jan 24, 2018 at 10:29 PM, Dietmar Putz <dietmar.p...@3qsdn.com> wrote: > Hi all, > i have made some tests on the latest Ubuntu 16.04.3 server image. Upgrades > were disabled... > the configuration was always the same...a distributed replicated volume on > 4 VM's with geo-replication to a dist. repl .volume on 4 VM's. > i started with 3.7.20, upgrade to 3.8.15, to 3.10.9 to 3.12.5. After each > upgrade i have tested the geo-replication which worked well anytime. > then i have made an update / upgrade on the first master node. directly > after upgrade the below shown error appeared on that node. > after upgrade on the second master node the error appeared there also... > geo replication is faulty. > > this error affects gfs 3.7.20, 3.8.15, 3.10.9 and 3.12.5 on Ubuntu 16.04.3 > in one test i have updated rsync from 3.1.1 to 3.1.2 but with no effect. > > does anyone else experienced this behavior...any idea ? > > best regards > Dietmar > > > gfs 3.12.5 geo-rep log on master : > > [2018-01-24 15:50:35.347959] I [master(/brick1/mvol1):1385:crawl] > _GMaster: slave's time stime=(1516808792, 0) > [2018-01-24 15:50:35.604094] I [master(/brick1/mvol1):1863:syncjob] > Syncer: Sync Time Taken duration=0.0294 num_files=1 job=2 > return_code=3 > [2018-01-24 15:50:35.605490] E [resource(/brick1/mvol1):210:errlog] > Popen: command returned error cmd=rsync -aR0 --inplace --files-from=- > --super --stats --numeric-ids --no-implied-dirs --existing --xattrs --acls > --ignore-missing-args . -e ssh -oPasswordAuthentication=no > -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem > -p 22 -oControlMaster=auto -S /tmp/gsyncd-aux-ssh-MZwEp2/ > cbad1c5f88978ecd713bdb1478fbabbe.sock --compress root@gl-node5-int > :/proc/2013/cwd error=3 > [2018-01-24 15:50:35.628978] I [syncdutils(/brick1/mvol1):271:finalize] > <top>: exiting. > > > > after this upgrade one server fails : > Start-Date: 2018-01-18 04:33:52 > Commandline: /usr/bin/unattended-upgrade > Upgrade: > libdns-export162:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8, > 1:9.10.3.dfsg.P4-8ubuntu1.10), > libisccfg140:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8, > 1:9.10.3.dfsg.P4-8ubuntu1.10), > bind9-host:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8, > 1:9.10.3.dfsg.P4-8ubuntu1.10), > dnsutils:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8, > 1:9.10.3.dfsg.P4-8ubuntu1.10), > libc6:amd64 (2.23-0ubuntu9, 2.23-0ubuntu10), > libisc160:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8, > 1:9.10.3.dfsg.P4-8ubuntu1.10), > locales:amd64 (2.23-0ubuntu9, 2.23-0ubuntu10), > libisc-export160:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8, > 1:9.10.3.dfsg.P4-8ubuntu1.10), > libc-bin:amd64 (2.23-0ubuntu9, 2.23-0ubuntu10), > liblwres141:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8, > 1:9.10.3.dfsg.P4-8ubuntu1.10), > libdns162:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8, > 1:9.10.3.dfsg.P4-8ubuntu1.10), > multiarch-support:amd64 (2.23-0ubuntu9, 2.23-0ubuntu10), > libisccc140:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8, > 1:9.10.3.dfsg.P4-8ubuntu1.10), > libbind9-140:amd64 (1:9.10.3.dfsg.P4-8ubuntu1.8, > 1:9.10.3.dfsg.P4-8ubuntu1.10) > End-Date: 2018-01-18 04:34:32 > > > > strace rsync : > > 30743 23:34:47 newfstatat(3, "6737", {st_mode=S_IFDIR|0755, st_size=4096, > ...}, AT_SYMLINK_NOFOLLOW) = 0 > 30743 23:34:47 newfstatat(3, "6741", {st_mode=S_IFDIR|0755, st_size=4096, > ...}, AT_SYMLINK_NOFOLLOW) = 0 > 30743 23:34:47 getdents(3, /* 0 entries */, 131072) = 0 > 30743 23:34:47 munmap(0x7fa4feae7000, 135168) = 0 > 30743 23:34:47 close(3) = 0 > 30743 23:34:47 write(2, "rsync: getcwd(): No such file or directory (2)", > 46) = 46 > 30743 23:34:47 write(2, "\n", 1) = 1 > 30743 23:34:47 rt_sigaction(SIGUSR1, {SIG_IGN, [], SA_RESTORER, > 0x7fa4fdf404b0}, NULL, 8) = 0 > 30743 23:34:47 rt_sigaction(SIGUSR2, {SIG_IGN, [], SA_RESTORER, > 0x7fa4fdf404b0}, NULL, 8) = 0 > 30743 23:34:47 write(2, "rsync error: errors selecting input/output files, > dirs (code 3) at util.c(1056) [Receiver=3.1.1]", 96) = 96 > 30743 23:34:47 write(2, "\n", 1) = 1 > 30743 23:34:47 exit_group(3) = ? > 30743 23:34:47 +++ exited with 3 +++ > > > > > Am 19.01.2018 um 17:27 schrieb Joe Julian: > > ubuntu 16.04 > > > -- > Dietmar Putz > 3Q GmbH > Kurfürstendamm 102 > D-10711 Berlin > > Mobile: +49 171 / 90 160 39 > Mail: dietmar.p...@3qsdn.com > > > _______________________________________________ > Gluster-users mailing list > Gluster-users@gluster.org > http://lists.gluster.org/mailman/listinfo/gluster-users > -- Thanks and Regards, Kotresh H R
_______________________________________________ Gluster-users mailing list Gluster-users@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-users