Re: OT: rsync on Mac OSX
On Jul 12, 2013, at 2:57 PM, kpn...@pobox.com wrote: I thought MacOS X's rsync did handle resource forks if you gave it the proper option. The resource fork is reported by rsync in the usual convention of having ._ prefixed to the filename. My understanding was that the files named ._foo were plain files that included the metadata that makes up the resource fork. The ._ file is not really the resource fork, but a workaround for filesystems that do not support resource forks. As such, they would be copied by rsync just fine. Now as to the Mac OS X rsync understanding resource forks, that I cannot speak to, but it should be easy to test. Copy a directory from an HFS+ volume to a non-Mac OS X volume (NFS for example) using rsync and see if it creates the ._ files to go with the data. -- Paul Kraus Deputy Technical Director, LoneStarCon 3 Sound Coordinator, Schenectady Light Opera Company ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: OT: rsync on Mac OSX
On Fri, Jul 12, 2013 at 11:25 AM, Chris Maness ch...@chrismaness.comwrote: Thank you for the detailed description of what resource forks are. One more clue in this mystery is that appending .mov extension to it fixes the problem. That makes some sense, since without the resource fork some MacOS software would have trouble identifying the type of the file. The extension provides that information in another way. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: OT: rsync on Mac OSX
On Tue, Jul 2, 2013 at 3:11 PM, Chris Maness ch...@chrismaness.com wrote: Since you are going to wait anyway, why don't you try peeking at some of the file checksums while this is running? MacOS X comes with a shasum utility which implements SHA-256 checksums, so you should be able to look at a few random samples of these files, e.g. by running on the source disk: shasum -a 256 source_directory/file/path/to/some/file.ext shasum -a 256 copied_directory/file/path/to/some/file.ext If these are the same, then the applications look elsewhere, e.g. in the 'hidden' .DS_Store stuff some MacOS directories contain. But if the checksums are different, well, then there's your problem. Checksums are the same. All other files still work however the HUGE rendered Final Cut Pro output, so I guess it is something in .DS_Store. Last time I just gave up and recopied everything by a simple cut and paste and that solved the problem. I made a small change on the project today, and I don't want to have to copy the WHOLE thing again just for a small delta. I already synced the directories, but the new rendered files are still un-openable in any application even though the checksums match. Really weird. However, the project will still open and work on FCP. Just the 12Gb rendered movie files will not play on anything even FCP. If I delete .DS_Store will the system regenerate it with the appropriate file associations? I know this is a little off topic, but Mac OSX is based on BSD. You guys are also the smartest around :D Thanks, Chris Maness ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: OT: rsync on Mac OSX
On 12 July 2013, at 10:49, Chris Maness ch...@chrismaness.com wrote: On Tue, Jul 2, 2013 at 3:11 PM, Chris Maness ch...@chrismaness.com wrote: Since you are going to wait anyway, why don't you try peeking at some of the file checksums while this is running? MacOS X comes with a shasum utility which implements SHA-256 checksums, so you should be able to look at a few random samples of these files, e.g. by running on the source disk: shasum -a 256 source_directory/file/path/to/some/file.ext shasum -a 256 copied_directory/file/path/to/some/file.ext If these are the same, then the applications look elsewhere, e.g. in the 'hidden' .DS_Store stuff some MacOS directories contain. But if the checksums are different, well, then there's your problem. Checksums are the same. All other files still work however the HUGE rendered Final Cut Pro output, so I guess it is something in .DS_Store. Last time I just gave up and recopied everything by a simple cut and paste and that solved the problem. I made a small change on the project today, and I don't want to have to copy the WHOLE thing again just for a small delta. I already synced the directories, but the new rendered files are still un-openable in any application even though the checksums match. Really weird. However, the project will still open and work on FCP. Just the 12Gb rendered movie files will not play on anything even FCP. If I delete .DS_Store will the system regenerate it with the appropriate file associations? I know this is a little off topic, but Mac OSX is based on BSD. You guys are also the smartest around :D Rsync on the Mac only opens and copies the data forks. It does not copy the resource forks. There are still a few applications that use resource forks. Likewise the checksum apps work on the data forks only. There is a utility that is a modified rsync that does handle resource forks. I no longer remember what its name is. Its been a number of years since I last used it. I normally rsync from FreeBSD systems to Mac systems. I use Minis as off-site backups. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: OT: rsync on Mac OSX
On Fri, Jul 12, 2013 at 11:12 AM, Paul Kraus p...@kraus-haus.org wrote: Dropping the list … On Jul 12, 2013, at 1:49 PM, Chris Maness ch...@chrismaness.com wrote: Checksums are the same. All other files still work however the HUGE rendered Final Cut Pro output, so I guess it is something in .DS_Store. Last time I just gave up and recopied everything by a simple cut and paste and that solved the problem. I made a small change on the project today, and I don't want to have to copy the WHOLE thing again just for a small delta. I already synced the directories, but the new rendered files are still un-openable in any application even though the checksums match. Really weird. However, the project will still open and work on FCP. Just the 12Gb rendered movie files will not play on anything even FCP. If I delete .DS_Store will the system regenerate it with the appropriate file associations? The .DS_Store files are created by the Finder when you view a directory. Are both source and destination on Mac HFS+ volumes ? If so, then you are probably missing the resource forks. Back in the very old days of Mac OS (way before 10.x), Mac OS files had two parts, the data part that contained the, well, data, and the resource fork that contained the meta-data that Mac OS used to associate a file with an application. HFS+ volumes on Mac OS X still include the resource forks, but foreign filesystems (NFS, UFS, FAT, etc.) do not. The work around that Apple came up with is to create .DS_Store and ._foo files to store this metadata on non HFS+ volumes. You could try using ditto instead of rsync. ditto is a BSD derived copy utility similar to rysnc, but I know that the Mac OS X version understands resource forks and copies them as necessary. ditto may not be able to just copy changed blocks within a file, so you may still have to recopy the entire file. But…. I am also a little puzzled because applications on Mac OS X do not NEED the resource fork to open a file, just to know which application to use (and what options to hand it) to open a given file. A complete video file, even without resource forks, should be able to be opened if you explicitly telly he application to File - Open …. With the checksums matching it is even odder. I expect that the large sizes (over 4 GB) are a contributing factor. Good luck and let me know what you find. -- Paul Kraus Deputy Technical Director, LoneStarCon 3 Sound Coordinator, Schenectady Light Opera Company Thank you for the detailed description of what resource forks are. One more clue in this mystery is that appending .mov extension to it fixes the problem. I have never ran into this before, and I have even used rsync to back up movie projects before. It is not a big deal, but I always try to take the time to understand why things behave the way they do. I also suspect it has something to do with file size since all of the smaller files do not have this issue. Thanks, Chris Maness ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
OT: rsync on Mac OSX
I have been using rsync with Mac OSX with no issues until today. I generally use it instead of the copy command because if the copy fails on large files, I can pick up where I left off. I have backed up entire Final Cut Pro projects this way with no issues. However, I recently synced a drive to a folder in another drive, and the OS does not recognize the final rendered files as quicktime files. The files work fine in the parent drive. I have no idea what might be going on. I used the flags: rsync -vaur like I always do. Any suggestions? Thanks, Chris Maness ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: OT: rsync on Mac OSX
On Tue, 2 Jul 2013 13:35:00 -0700, Chris Maness ch...@chrismaness.com wrote: I have been using rsync with Mac OSX with no issues until today. I generally use it instead of the copy command because if the copy fails on large files, I can pick up where I left off. I have backed up entire Final Cut Pro projects this way with no issues. However, I recently synced a drive to a folder in another drive, and the OS does not recognize the final rendered files as quicktime files. The files work fine in the parent drive. I have no idea what might be going on. I used the flags: rsync -vaur like I always do. Any suggestions? This is a FreeBSD list, so any issues rsync may have with MacOS X are not very relevant to what FreeBSD is doing or would do. Having said that though, can you try without the -u option? Maybe modification times are newer on the target drive and rsync skips everything. You should probably also enable --stats and have a look at the final report of rsync, to see if it actually sync'ed any files, or skipped all of them because of mtime checks. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: OT: rsync on Mac OSX
On Tue, Jul 2, 2013 at 2:30 PM, Giorgos Keramidas keram...@ceid.upatras.grwrote: On Tue, 2 Jul 2013 13:35:00 -0700, Chris Maness ch...@chrismaness.com wrote: I have been using rsync with Mac OSX with no issues until today. I generally use it instead of the copy command because if the copy fails on large files, I can pick up where I left off. I have backed up entire Final Cut Pro projects this way with no issues. However, I recently synced a drive to a folder in another drive, and the OS does not recognize the final rendered files as quicktime files. The files work fine in the parent drive. I have no idea what might be going on. I used the flags: rsync -vaur like I always do. Any suggestions? This is a FreeBSD list, so any issues rsync may have with MacOS X are not very relevant to what FreeBSD is doing or would do. Having said that though, can you try without the -u option? Maybe modification times are newer on the target drive and rsync skips everything. You should probably also enable --stats and have a look at the final report of rsync, to see if it actually sync'ed any files, or skipped all of them because of mtime checks. Yep, the files copied, and I used touch to force them to recopy. However, the files that were copied are not recognizable by their native aps. Just big junk files. I have no clue what happened. I am just copying everything by a simple cut and paste this time. However, this directory is HUGE and I won't know until about 18 hours from now. Thanks, Chris Maness ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: OT: rsync on Mac OSX
On Tue, 2 Jul 2013 14:48:03 -0700, Chris Maness ch...@chrismaness.com wrote: On Tue, Jul 2, 2013 at 2:30 PM, Giorgos Keramidas keram...@ceid.upatras.grwrote: On Tue, 2 Jul 2013 13:35:00 -0700, Chris Maness ch...@chrismaness.com wrote: I have been using rsync with Mac OSX with no issues until today. I generally use it instead of the copy command because if the copy fails on large files, I can pick up where I left off. I have backed up entire Final Cut Pro projects this way with no issues. However, I recently synced a drive to a folder in another drive, and the OS does not recognize the final rendered files as quicktime files. The files work fine in the parent drive. I have no idea what might be going on. I used the flags: rsync -vaur like I always do. Any suggestions? This is a FreeBSD list, so any issues rsync may have with MacOS X are not very relevant to what FreeBSD is doing or would do. Having said that though, can you try without the -u option? Maybe modification times are newer on the target drive and rsync skips everything. You should probably also enable --stats and have a look at the final report of rsync, to see if it actually sync'ed any files, or skipped all of them because of mtime checks. Yep, the files copied, and I used touch to force them to recopy. However, the files that were copied are not recognizable by their native aps. Just big junk files. I have no clue what happened. I am just copying everything by a simple cut and paste this time. However, this directory is HUGE and I won't know until about 18 hours from now. Since you are going to wait anyway, why don't you try peeking at some of the file checksums while this is running? MacOS X comes with a shasum utility which implements SHA-256 checksums, so you should be able to look at a few random samples of these files, e.g. by running on the source disk: shasum -a 256 source_directory/file/path/to/some/file.ext shasum -a 256 copied_directory/file/path/to/some/file.ext If these are the same, then the applications look elsewhere, e.g. in the 'hidden' .DS_Store stuff some MacOS directories contain. But if the checksums are different, well, then there's your problem. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: OT: rsync on Mac OSX
Since you are going to wait anyway, why don't you try peeking at some of the file checksums while this is running? MacOS X comes with a shasum utility which implements SHA-256 checksums, so you should be able to look at a few random samples of these files, e.g. by running on the source disk: shasum -a 256 source_directory/file/path/to/some/file.ext shasum -a 256 copied_directory/file/path/to/some/file.ext If these are the same, then the applications look elsewhere, e.g. in the 'hidden' .DS_Store stuff some MacOS directories contain. But if the checksums are different, well, then there's your problem. That could be the issue. I did see some permission warnings with .DS_Store. However, to make space I had to dump all of it, so no files to do a post mortem on. I will have to check that out if things go awry this time. Thanks, Chris Maness ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
