Re: Troubles with rsync
On Mon, 2018-06-11 at 23:43 -0400, Robert McBroom via users wrote: > On 06/11/2018 06:57 AM, Patrick O'Callaghan wrote: > > On Mon, 2018-06-11 at 01:22 -0400, Robert McBroom via users wrote: > > > On 06/10/2018 09:53 PM, Samuel Sieb wrote: > > > > On 06/08/2018 08:48 PM, Robert McBroom via users wrote: > > > > > On 06/08/2018 10:57 PM, Robert McBroom wrote: > > > > > > System is mounted with CIFS with write permission. NAS only uses > > > > > > version 1.0. > > > > > > > > > > > > Sample effects > > > > > > rsync . -av /mnt/external/f27/Packages > > > > > > > > > > > > rsync: mkstemp > > > > > > "/mnt/external/f27/Packages/.ffmpeg-libs-3.3.6-1.fc27.x86_64.rpm.eZUrRd" > > > > > > failed: Permission denied (13) > > > > > > > > It's likely that you can't create files that start with . on CIFS or > > > > NTFS. Try using the "--inplace" option with rsync. > > > > > > No problem with . filenames in general just in this instance the > > > transmission of the files is inhibited. Apparently something is > > > happening with the temporary file creation. > > > Using "--inplace" does the copy as desired. There is the message about > > > the permissions which are apparently inherited from the directory. > > > > Slightly OT: if you have admin access to the NAS you might want to > > consider using rsyncd (rsync daemon) on the server side. It's much more > > efficient because it doesn't require rsync on your client machine to > > read entire files over the network to check for changes. Rsync does > > this in order to calculate deltas, but the effect is counterproductive > > when it's doing it over NFS. Using rsyncd I've measured a 10x > > improvement on my system (with rsnapshot, which is built on top of > > rsync). > > Everything is hidden under the vendor's web based interface on ROM. It may still be possible to get an admin Shell. These things are often not widely documented. My own (rather old) Iomega NAS does support it, even though the docs all refer you to the web interface. Google is your best bet here. poc ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org/message/J4UBGQ6UUD4Y3SIOB7PGHKPNAOXEKMFM/
Re: Troubles with rsync
On 06/11/2018 08:43 PM, Robert McBroom via users wrote: Everything is hidden under the vendor's web based interface on ROM. That's usually the case, but sometimes there are third party addons or firmware replacements available. ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org/message/CJQMIX5H2MOVPUSDY7N4MHQYL7DNFUWV/
Re: Troubles with rsync
On 06/11/2018 06:57 AM, Patrick O'Callaghan wrote: On Mon, 2018-06-11 at 01:22 -0400, Robert McBroom via users wrote: On 06/10/2018 09:53 PM, Samuel Sieb wrote: On 06/08/2018 08:48 PM, Robert McBroom via users wrote: On 06/08/2018 10:57 PM, Robert McBroom wrote: System is mounted with CIFS with write permission. NAS only uses version 1.0. Sample effects rsync . -av /mnt/external/f27/Packages rsync: mkstemp "/mnt/external/f27/Packages/.ffmpeg-libs-3.3.6-1.fc27.x86_64.rpm.eZUrRd" failed: Permission denied (13) It's likely that you can't create files that start with . on CIFS or NTFS. Try using the "--inplace" option with rsync. No problem with . filenames in general just in this instance the transmission of the files is inhibited. Apparently something is happening with the temporary file creation. Using "--inplace" does the copy as desired. There is the message about the permissions which are apparently inherited from the directory. Slightly OT: if you have admin access to the NAS you might want to consider using rsyncd (rsync daemon) on the server side. It's much more efficient because it doesn't require rsync on your client machine to read entire files over the network to check for changes. Rsync does this in order to calculate deltas, but the effect is counterproductive when it's doing it over NFS. Using rsyncd I've measured a 10x improvement on my system (with rsnapshot, which is built on top of rsync). Everything is hidden under the vendor's web based interface on ROM. ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org/message/GUCMRVZN7MTKZTJA5D6GV6TQ3SCC6STF/
Re: Troubles with rsync
On Mon, 2018-06-11 at 10:41 -0700, Samuel Sieb wrote: > On 06/11/2018 03:57 AM, Patrick O'Callaghan wrote: > > Slightly OT: if you have admin access to the NAS you might want to > > consider using rsyncd (rsync daemon) on the server side. It's much more > > efficient because it doesn't require rsync on your client machine to > > read entire files over the network to check for changes. Rsync does > > this in order to calculate deltas, but the effect is counterproductive > > when it's doing it over NFS. Using rsyncd I've measured a 10x > > improvement on my system (with rsnapshot, which is built on top of > > rsync). > > Or similarly, enable ssh on the NAS and do rsync over that. Yes, much the same. poc ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org/message/MLPZOQMBER7ZJCMW5UA4QZPQJBMDRE4Y/
Re: Troubles with rsync
On 06/11/2018 03:57 AM, Patrick O'Callaghan wrote: Slightly OT: if you have admin access to the NAS you might want to consider using rsyncd (rsync daemon) on the server side. It's much more efficient because it doesn't require rsync on your client machine to read entire files over the network to check for changes. Rsync does this in order to calculate deltas, but the effect is counterproductive when it's doing it over NFS. Using rsyncd I've measured a 10x improvement on my system (with rsnapshot, which is built on top of rsync). Or similarly, enable ssh on the NAS and do rsync over that. ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org/message/NPBYCMJRIQCP7UBKIFYUD4DJVTPL7ZHE/
Re: Troubles with rsync
On Mon, 2018-06-11 at 01:22 -0400, Robert McBroom via users wrote: > On 06/10/2018 09:53 PM, Samuel Sieb wrote: > > On 06/08/2018 08:48 PM, Robert McBroom via users wrote: > > > On 06/08/2018 10:57 PM, Robert McBroom wrote: > > > > System is mounted with CIFS with write permission. NAS only uses > > > > version 1.0. > > > > > > > > Sample effects > > > > rsync . -av /mnt/external/f27/Packages > > > > > > > > rsync: mkstemp > > > > "/mnt/external/f27/Packages/.ffmpeg-libs-3.3.6-1.fc27.x86_64.rpm.eZUrRd" > > > > > > > > failed: Permission denied (13) > > > > It's likely that you can't create files that start with . on CIFS or > > NTFS. Try using the "--inplace" option with rsync. > > No problem with . filenames in general just in this instance the > transmission of the files is inhibited. Apparently something is > happening with the temporary file creation. > Using "--inplace" does the copy as desired. There is the message about > the permissions which are apparently inherited from the directory. Slightly OT: if you have admin access to the NAS you might want to consider using rsyncd (rsync daemon) on the server side. It's much more efficient because it doesn't require rsync on your client machine to read entire files over the network to check for changes. Rsync does this in order to calculate deltas, but the effect is counterproductive when it's doing it over NFS. Using rsyncd I've measured a 10x improvement on my system (with rsnapshot, which is built on top of rsync). poc ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org/message/LICTA55XD4RY5NAMB7DQN42BMK7XTCJD/
Re: Troubles with rsync
On 06/10/2018 09:53 PM, Samuel Sieb wrote: On 06/08/2018 08:48 PM, Robert McBroom via users wrote: On 06/08/2018 10:57 PM, Robert McBroom wrote: System is mounted with CIFS with write permission. NAS only uses version 1.0. Sample effects rsync . -av /mnt/external/f27/Packages rsync: mkstemp "/mnt/external/f27/Packages/.ffmpeg-libs-3.3.6-1.fc27.x86_64.rpm.eZUrRd" failed: Permission denied (13) It's likely that you can't create files that start with . on CIFS or NTFS. Try using the "--inplace" option with rsync. No problem with . filenames in general just in this instance the transmission of the files is inhibited. Apparently something is happening with the temporary file creation. Using "--inplace" does the copy as desired. There is the message about the permissions which are apparently inherited from the directory. rsync: failed to set permissions on "/mnt/external/test/autocorr-en-5.4.7.2-2.fc27.noarch.rpm": Permission denied (13) rsync: failed to set permissions on "/mnt/external/test/cryptsetup-1.7.5-4.fc27.x86_64.rpm": Permission denied (13) rsync: failed to set permissions on "/mnt/external/test/cryptsetup-libs-1.7.5-4.fc27.x86_64.rpm": Permission denied (13) Using cpio command, files are transfered with a message about the mode change failure. find . -print |cpio -p -d -m /mnt/external/f27/Packages cpio: /mnt/external/f27/Packages/./ffmpeg-libs-3.3.6-1.fc27.x86_64.rpm: Cannot change mode to rw-r--r--: Permission denied NTFS permissions are also different and don't match the Unix permission settings. You will get the same errors from rsync as well. The chmod command has no effect on the permission settings of the ntfs files. There are probably some combinations that would work like removing write from the user or all, but in most cases the permission set isn't possible to represent on NTFS. ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org/message/3LVBRFF7F7JIRFZ2BKAMDETXDYBBECPQ/ ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org/message/LFBUK7G3ALJRKQGS6M4A444QMKPV3E3D/
Re: Troubles with rsync
On 06/08/2018 08:48 PM, Robert McBroom via users wrote: On 06/08/2018 10:57 PM, Robert McBroom wrote: System is mounted with CIFS with write permission. NAS only uses version 1.0. Sample effects rsync . -av /mnt/external/f27/Packages rsync: mkstemp "/mnt/external/f27/Packages/.ffmpeg-libs-3.3.6-1.fc27.x86_64.rpm.eZUrRd" failed: Permission denied (13) It's likely that you can't create files that start with . on CIFS or NTFS. Try using the "--inplace" option with rsync. Using cpio command, files are transfered with a message about the mode change failure. find . -print |cpio -p -d -m /mnt/external/f27/Packages cpio: /mnt/external/f27/Packages/./ffmpeg-libs-3.3.6-1.fc27.x86_64.rpm: Cannot change mode to rw-r--r--: Permission denied NTFS permissions are also different and don't match the Unix permission settings. You will get the same errors from rsync as well. The chmod command has no effect on the permission settings of the ntfs files. There are probably some combinations that would work like removing write from the user or all, but in most cases the permission set isn't possible to represent on NTFS. ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org/message/3LVBRFF7F7JIRFZ2BKAMDETXDYBBECPQ/
Re: Troubles with rsync
On 06/08/2018 10:57 PM, Robert McBroom wrote: On 06/08/2018 02:02 PM, Rick Stevens wrote: On 06/08/2018 10:52 AM, Robert McBroom via users wrote: I keep a repo of the packages for Fedora on a NAS system on a linksys router. The file system is ntfs. rsync has stopped being able to synchronize the package files from the updates in cache/dnf. Piping the updates through cpio works but complains about not being able to set the permissions on the transferred files to the same as the source. ntfs has permissions of 775 and those for the package files in updates are 644. Is there a way to get rsync ntfs aware? rsync doesn't care what the source or target filesystem are. If rsync is running as a non-root user and that user isn't the owner or group of the target filesystem's directories, then yeah, it's not going to be able to change the permissions (the rsync user has to have write permission on the target). System is mounted with CIFS with write permission. NAS only uses version 1.0. Sample effects rsync . -av /mnt/external/f27/Packages sending incremental file list rsync: failed to set permissions on "/mnt/external/f27/Packages/.": Permission denied (13) ./ ffmpeg-libs-3.3.6-1.fc27.x86_64.rpm gstreamer1-plugins-ugly-1.12.4-3.fc27.x86_64.rpm k3b-extras-freeworld-17.08.3-1.fc27.x86_64.rpm vlc-3.0.0-0.48.git20180109.rc5.fc27.x86_64.rpm vlc-core-3.0.0-0.48.git20180109.rc5.fc27.x86_64.rpm rsync: mkstemp "/mnt/external/f27/Packages/.ffmpeg-libs-3.3.6-1.fc27.x86_64.rpm.eZUrRd" failed: Permission denied (13) rsync: mkstemp "/mnt/external/f27/Packages/.gstreamer1-plugins-ugly-1.12.4-3.fc27.x86_64.rpm.kK2GDX" failed: Permission denied (13) rsync: mkstemp "/mnt/external/f27/Packages/.k3b-extras-freeworld-17.08.3-1.fc27.x86_64.rpm.HlwOHH" failed: Permission denied (13) rsync: mkstemp "/mnt/external/f27/Packages/.vlc-3.0.0-0.48.git20180109.rc5.fc27.x86_64.rpm.XtxTXr" failed: Permission denied (13) rsync: mkstemp "/mnt/external/f27/Packages/.vlc-core-3.0.0-0.48.git20180109.rc5.fc27.x86_64.rpm.zXJevc" failed: Permission denied (13) sent 19,244,411 bytes received 875 bytes 38,490,572.00 bytes/sec total size is 19,239,172 speedup is 1.00 rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1189) [sender=3.1.3] Message says the file bytes were sent but nothing is written to the NAS. Using cpio command, files are transfered with a message about the mode change failure. find . -print |cpio -p -d -m /mnt/external/f27/Packages cpio: /mnt/external/f27/Packages/./ffmpeg-libs-3.3.6-1.fc27.x86_64.rpm: Cannot change mode to rw-r--r--: Permission denied cpio: /mnt/external/f27/Packages/./gstreamer1-plugins-ugly-1.12.4-3.fc27.x86_64.rpm: Cannot change mode to rw-r--r--: Permission denied cpio: /mnt/external/f27/Packages/./vlc-core-3.0.0-0.48.git20180109.rc5.fc27.x86_64.rpm: Cannot change mode to rw-r--r--: Permission denied cpio: /mnt/external/f27/Packages/./vlc-3.0.0-0.48.git20180109.rc5.fc27.x86_64.rpm: Cannot change mode to rw-r--r--: Permission denied cpio: /mnt/external/f27/Packages/./k3b-extras-freeworld-17.08.3-1.fc27.x86_64.rpm: Cannot change mode to rw-r--r--: Permission denied 37577 blocks Directory entries on the source are of the form -rw-r--r--. 1 root root 212504 Jun 5 15:37 autocorr-en-5.4.7.2-2.fc27.noarch.rpm -rw-r--r--. 1 root root 138408 Jun 8 09:03 cryptsetup-1.7.5-4.fc27.x86_64.rpm -rw-r--r--. 1 root root 233448 Jun 8 09:03 cryptsetup-libs-1.7.5-4.fc27.x86_64.rpm all entries on the NAS have the mode set as -rwxrwxr-x. 1 root root 212504 Jun 5 15:37 autocorr-en-5.4.7.2-2.fc27.noarch.rpm -rwxrwxr-x. 1 root root 138408 Jun 8 09:03 cryptsetup-1.7.5-4.fc27.x86_64.rpm -rwxrwxr-x. 1 root root 233448 Jun 8 09:03 cryptsetup-libs-1.7.5-4.fc27.x86_64.rpm mounting a ntfs files system has entries with the mode set of the form -rwxrwxrwx. 1 root root 150 Dec 1 2012 case8.bat -rwxrwxrwx. 1 root root 259 Oct 31 2012 chain1 -rwxrwxrwx. 1 root root 65379 Feb 18 2007 chnllineup.pdf The chmod command has no effect on the permission settings of the ntfs files. - ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org/message/7EV4IMR2UV3VFROY3IE2IWYQUFRPTTST/
Re: Troubles with rsync
On 06/08/2018 10:52 AM, Robert McBroom via users wrote: > I keep a repo of the packages for Fedora on a NAS system on a linksys > router. The file system is ntfs. rsync has stopped being able to > synchronize the package files from the updates in cache/dnf. Piping the > updates through cpio works but complains about not being able to set the > permissions on the transferred files to the same as the source. ntfs > has permissions of 775 and those for the package files in updates are > 644. Is there a way to get rsync ntfs aware? rsync doesn't care what the source or target filesystem are. If rsync is running as a non-root user and that user isn't the owner or group of the target filesystem's directories, then yeah, it's not going to be able to change the permissions (the rsync user has to have write permission on the target). It'd help if you'd post the rsync error messages you're getting. -- - Rick Stevens, Systems Engineer, AllDigitalri...@alldigital.com - - AIM/Skype: therps2ICQ: 226437340 Yahoo: origrps2 - -- - A squeegee, by any other name, wouldn't sound as funny. - -- ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org/message/C2MYRUJX7MOUFQ5O3K4NOFF6HZ3WNQCV/
Re: Troubles with rsync
On 06/08/2018 10:52 AM, Robert McBroom via users wrote: I keep a repo of the packages for Fedora on a NAS system on a linksys router. The file system is ntfs. rsync has stopped being able to synchronize the package files from the updates in cache/dnf. Piping the updates through cpio works but complains about not being able to set the permissions on the transferred files to the same as the source. ntfs has permissions of 775 and those for the package files in updates are 644. Is there a way to get rsync ntfs aware? What is the error from rsync? Also, if it's a NAS, then the file system your computer sees is not NTFS, but either CIFS (SMB) or NFS. What does the output from mount for the mount point show? ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org/message/BEBUN7PHBFUV6YSYX4RZYLYON5HJY6A5/
