Re: [PLUG] Rsync again
I would add checking of the return code (exit code) for the 'cd' commands. As it is now those could fail but something latter would have to detect the failure. On Sat, Oct 6, 2018, 5:28 PM John Jason Jordan wrote: > On Sat, 06 Oct 2018 13:08:21 -0700 > Tomas K dijo: > > >Simple if statement in your backup script should take care of it. > >You can do it number of ways - here is one - checking that there is any > >content in the directory before starting rsync (in bash): > > > >#!/bin/bash > >backupTargetDir=/media/jjj/Synology > >cd $backupTargetDir > ># wait 30 seconds for the directory to mount > >sleep 30 > >filesInDir=$(ls | wc -l) > >if (( $filesInDir > 0 )); then > > echo "INFO: Backing up files to $backupTargetDir" > > rsync > >else > > echo "WARNING: No files in $backupTargetDir" > >fi > > ¡Me gusta! > > Here is how I finalized it: > > #!/bin/bash > cd .. > cd /media/jjj/Movies > backupTargetDir=/media/jjj/Synology > cd $backupTargetDir > # wait 30 seconds for the directory to mount > sleep 30 > filesInDir=$(ls | wc -l) > if (( $filesInDir > 0 )); then > echo "INFO: Backing up files to $backupTargetDir" > rsync -rptog --progress --stats --delete > --exclude-from=/media/jjj/Movies/rsync_exclusions /media/jjj/Movies/ > /media/jjj/Synology > else echo "WARNING: No files in $backupTargetDir" > fi > > I could probably add another if statement to be sure > that /media/JJJ/Movies is mounted, but that's my work directory, so > it's highly unlikely that it would not be mounted. > > I tested it and it works fine! Thanks for the help! > ___ > PLUG mailing list > PLUG@pdxlinux.org > http://lists.pdxlinux.org/mailman/listinfo/plug > ___ PLUG mailing list PLUG@pdxlinux.org http://lists.pdxlinux.org/mailman/listinfo/plug
Re: [PLUG] Rsync again
On Sat, 06 Oct 2018 13:08:21 -0700 Tomas K dijo: >Simple if statement in your backup script should take care of it. >You can do it number of ways - here is one - checking that there is any >content in the directory before starting rsync (in bash): > >#!/bin/bash >backupTargetDir=/media/jjj/Synology >cd $backupTargetDir ># wait 30 seconds for the directory to mount >sleep 30 >filesInDir=$(ls | wc -l) >if (( $filesInDir > 0 )); then > echo "INFO: Backing up files to $backupTargetDir" > rsync >else > echo "WARNING: No files in $backupTargetDir" >fi ¡Me gusta! Here is how I finalized it: #!/bin/bash cd .. cd /media/jjj/Movies backupTargetDir=/media/jjj/Synology cd $backupTargetDir # wait 30 seconds for the directory to mount sleep 30 filesInDir=$(ls | wc -l) if (( $filesInDir > 0 )); then echo "INFO: Backing up files to $backupTargetDir" rsync -rptog --progress --stats --delete --exclude-from=/media/jjj/Movies/rsync_exclusions /media/jjj/Movies/ /media/jjj/Synology else echo "WARNING: No files in $backupTargetDir" fi I could probably add another if statement to be sure that /media/JJJ/Movies is mounted, but that's my work directory, so it's highly unlikely that it would not be mounted. I tested it and it works fine! Thanks for the help! ___ PLUG mailing list PLUG@pdxlinux.org http://lists.pdxlinux.org/mailman/listinfo/plug
Re: [PLUG] Rsync again
On 10/06/2018 02:50 PM, John Jason Jordan wrote: On Sat, 6 Oct 2018 12:23:04 -0700 Erik Lane dijo: That would be nice to be able to stop it in that way. I'm not a Linux guru, so I'd also like to hear if there's some way to do that. However, I just wanted to say that I always use -P as one of the options, and this will resume partial downloads. So if you have to stop it and then resume later, it will just pick up where it left off. Not quite what you were asking for, but another way to solve the same problem. (At least as far as I understood your question.) Well, the problem resolved itself, at least in this case. The reason I wanted to stop it was because I was watching and it was backing up files that I was pretty sure had already been backed up, and the only reason it might be doing that was if the backup device was not mounted. As it turns out my guess was right, 15 minutes later rsync stopped all by itself when it ran out of space on the mount point drive. And this is not the first time I have copied things to a drive that I thought was mounted, but was not. It's a perennial problem, and it's high time I figured out a way to keep it from happening. If you are using/allowing the system to automagically mount your device you should test if it is there before starting the backup. I used this method years ago to back up my club accounting data. OOPS ... wrong! I was pushing the files to my ownCloud server so I used ping to test for it's existence. Here is the script (obfuscated to protect the guilty.) You could figure out your own test. #!/bin/bash /bin/ping -c 1 cloud.host.net 2> /dev/null | grep " 0% packet loss" > /dev/null if [[ $? == 0 ]]; then echo cloud found rsync -n -avz --update /home/user/Data/FILE /user/ownCloud/CLUB/DATA else echo cloud not found fi \\||/ Rod -- ___ PLUG mailing list PLUG@pdxlinux.org http://lists.pdxlinux.org/mailman/listinfo/plug ___ PLUG mailing list PLUG@pdxlinux.org http://lists.pdxlinux.org/mailman/listinfo/plug
Re: [PLUG] Rsync again
On Sat, 6 Oct 2018 12:23:04 -0700 Erik Lane dijo: >That would be nice to be able to stop it in that way. I'm not a Linux >guru, so I'd also like to hear if there's some way to do that. > >However, I just wanted to say that I always use -P as one of the >options, and this will resume partial downloads. So if you have to >stop it and then resume later, it will just pick up where it left off. > >Not quite what you were asking for, but another way to solve the same >problem. (At least as far as I understood your question.) Well, the problem resolved itself, at least in this case. The reason I wanted to stop it was because I was watching and it was backing up files that I was pretty sure had already been backed up, and the only reason it might be doing that was if the backup device was not mounted. As it turns out my guess was right, 15 minutes later rsync stopped all by itself when it ran out of space on the mount point drive. And this is not the first time I have copied things to a drive that I thought was mounted, but was not. It's a perennial problem, and it's high time I figured out a way to keep it from happening. ___ PLUG mailing list PLUG@pdxlinux.org http://lists.pdxlinux.org/mailman/listinfo/plug
Re: [PLUG] Rsync again
The answer is - Do not start rsync until your NAS is mounted and working. Simple if statement in your backup script should take care of it. You can do it number of ways - here is one - checking that there is any content in the directory before starting rsync (in bash): #!/bin/bash backupTargetDir=/media/jjj/Synology cd $backupTargetDir # wait 30 seconds for the directory to mount sleep 30 filesInDir=$(ls | wc -l) if (( $filesInDir > 0 )); then echo "INFO: Backing up files to $backupTargetDir" rsync else echo "WARNING: No files in $backupTargetDir" fi Another way would be to use mount to verify that your NAS is mounted Tomas On Sat, 2018-10-06 at 11:37 -0700, John Jason Jordan wrote: > On Sat, 6 Oct 2018 10:17:05 -0700 > John Jason Jordan dijo: > > I need some advice with respect to the following command, which makes > my Synology NAS device a mirror of my external USB storage enclosure: > > rsync -rptog --progress --stats --delete > --exclude-from=/media/jjj/Movies/rsync_exclusions > /media/jjj/Movies/ /media/jjj/Synology > > The first problem is that sometimes I get an error message that it > can't find /media/jjj/Movies. I found that a simple 'cd ..' followed > by > 'cd /media/jjj/Movies' solves the problem. I plan to convert the > rsync > command above to a simple script, adding the cd commands in front of > the rsync command. > > But there is a second, even worse problem: It sometimes can't find > the > Synology NAS device. And if it can't find the device it dumps > everything into the local hard drive where /media/jjj/Synology is > (supposed to be) mounted. The hard drive has a couple hundred GB of > free > space, but since the external USB device has over 7TB of data on it > the > local disk is quickly filled up, and then the rsync command fails > with > 'the device is full.' > > The Synology is mounted with this line in fstab: > > 192.168.1.115:/volume1/Synology /media/jjj/Synology nfs > auto,user 0 0 > > I need to add another command before the rsync command that checks to > be sure the Synology is mounted where it's supposed to be and mount > it > there if it is not already mounted. > > And finally, I need to add something after the rsync command that > checks to be sure the command executed as it is supposed to and give > me > a popup that it did, or that it did not. I have gxmessage installed > and > I could probably figure out how to make it pop up the messages, but I > don't know how to check that the rsync command executed properly. > Could > I just check the two filesystems to see if they are equal in all > respects? > > Suggestions? > ___ > PLUG mailing list > PLUG@pdxlinux.org > http://lists.pdxlinux.org/mailman/listinfo/plug ___ PLUG mailing list PLUG@pdxlinux.org http://lists.pdxlinux.org/mailman/listinfo/plug
Re: [PLUG] Rsync again
That would be nice to be able to stop it in that way. I'm not a Linux guru, so I'd also like to hear if there's some way to do that. However, I just wanted to say that I always use -P as one of the options, and this will resume partial downloads. So if you have to stop it and then resume later, it will just pick up where it left off. Not quite what you were asking for, but another way to solve the same problem. (At least as far as I understood your question.) Thanks, Erik On Sat, Oct 6, 2018 at 10:17 AM John Jason Jordan wrote: > Is there a way to stop an rsync command so that it gets to the end of > the file it is writing before it stops, less you end up with a > partially transferred file? > ___ > PLUG mailing list > PLUG@pdxlinux.org > http://lists.pdxlinux.org/mailman/listinfo/plug > ___ PLUG mailing list PLUG@pdxlinux.org http://lists.pdxlinux.org/mailman/listinfo/plug
Re: [PLUG] Rsync again
On Sat, 6 Oct 2018 10:17:05 -0700 John Jason Jordan dijo: I need some advice with respect to the following command, which makes my Synology NAS device a mirror of my external USB storage enclosure: rsync -rptog --progress --stats --delete --exclude-from=/media/jjj/Movies/rsync_exclusions /media/jjj/Movies/ /media/jjj/Synology The first problem is that sometimes I get an error message that it can't find /media/jjj/Movies. I found that a simple 'cd ..' followed by 'cd /media/jjj/Movies' solves the problem. I plan to convert the rsync command above to a simple script, adding the cd commands in front of the rsync command. But there is a second, even worse problem: It sometimes can't find the Synology NAS device. And if it can't find the device it dumps everything into the local hard drive where /media/jjj/Synology is (supposed to be) mounted. The hard drive has a couple hundred GB of free space, but since the external USB device has over 7TB of data on it the local disk is quickly filled up, and then the rsync command fails with 'the device is full.' The Synology is mounted with this line in fstab: 192.168.1.115:/volume1/Synology /media/jjj/Synology nfs auto,user 0 0 I need to add another command before the rsync command that checks to be sure the Synology is mounted where it's supposed to be and mount it there if it is not already mounted. And finally, I need to add something after the rsync command that checks to be sure the command executed as it is supposed to and give me a popup that it did, or that it did not. I have gxmessage installed and I could probably figure out how to make it pop up the messages, but I don't know how to check that the rsync command executed properly. Could I just check the two filesystems to see if they are equal in all respects? Suggestions? ___ PLUG mailing list PLUG@pdxlinux.org http://lists.pdxlinux.org/mailman/listinfo/plug
[PLUG] Rsync again
Is there a way to stop an rsync command so that it gets to the end of the file it is writing before it stops, less you end up with a partially transferred file? ___ PLUG mailing list PLUG@pdxlinux.org http://lists.pdxlinux.org/mailman/listinfo/plug
