Re: PATCH: preserve osx creation-date (was: Using pre2 for backing up a mac)
On Oct 15, 2007, at 1:55 AM, Wayne Davison wrote:
On Sun, Oct 14, 2007 at 08:14:57PM +0200, Wesley W. Terpstra wrote:
I've attached a patch which does this. Currently resource forks
and finder
info get placed into an extended attribute transparently by osx
(com.apple.{ResourceFork/FinderInfo}). This patch makes another
extended
attribute called com.apple.CreationDate.
Thanks! I've twiddled your patch a bit and checked it in as a file in
the patches dir (for now, at least): osx-create-time.diff
Found another bug: Some filesystems don't have creation dates
(*shock*). If you tried to read it on these filesystems, it will
fail, so one needs to check if the attribute exists first.
Revised patch attached.
check-first.patch
Description: Binary data
--
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: PATCH: preserve osx creation-date (was: Using pre2 for backing up a mac)
On Oct 15, 2007, at 1:55 AM, Wayne Davison wrote:
On Sun, Oct 14, 2007 at 08:14:57PM +0200, Wesley W. Terpstra wrote:
I've attached a patch which does this. Currently resource forks
and finder
info get placed into an extended attribute transparently by osx
(com.apple.{ResourceFork/FinderInfo}). This patch makes another
extended
attribute called com.apple.CreationDate.
Thanks! I've twiddled your patch a bit and checked it in as a file in
the patches dir (for now, at least): osx-create-time.diff
I changed the storing of the data to store the time value as 12
bytes of
binary data -- 8 for the time_t value (for future expansion) and 4 for
the nsec value (which always be 0 at present).
I fail to see the advantage of a binary version. You add concerns
about byte-order and make it unreadable to a sysadmin. (I also don't
like how the ACL stuff is binary) There was no problem with 8 byte
time_t vs. 4 byte values with the other format either as it would've
been good until year . If stat is textual, why not this? Saving a
few bytes is not a good reason ...
The name of the attribute was changed to com.apple.crtime96 (for
the moment) . Since it is not an official com.apple.* value, I
didn't want to use a name that Apple might choose in the future.
The name should probably go in the rsync.* namespace, but I'd need
to move the code for reading and writing the value out of lib/
sysxattrs.c into xattrs.c to do that (I believe).
Yeah, I suppose it should go in the rsync namespace too.
I also enabled the XATTR_NOFOLLOW option for {get,set}attrlist().
This option doesn't exist for these system calls. There is, however,
FSOPT_NOFOLLOW.
--
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: PATCH: preserve osx creation-date (was: Using pre2 for backing up a mac)
On Oct 15, 2007, at 12:13 PM, Wesley W. Terpstra wrote:
On Oct 15, 2007, at 1:55 AM, Wayne Davison wrote:
The name of the attribute was changed to com.apple.crtime96 (for
the moment) . Since it is not an official com.apple.* value, I
didn't want to use a name that Apple might choose in the future.
The name should probably go in the rsync.* namespace, but I'd need
to move the code for reading and writing the value out of lib/
sysxattrs.c into xattrs.c to do that (I believe).
Yeah, I suppose it should go in the rsync namespace too.
I notice that rsync is starting to converge towards a particular use
of xattrs:
Copy meta-data that has an analogous concept on the remote host.
If -X is specified, copy anything that has no analogue into an
extended attribute.
In looking over the bsd flags patch, I notice that it is relatively
complicated and still doesn't move the data into an xattr on a system
which has no support.
I wonder if the right command-line options might be something like this:
-X ... enables copying of user extended attributes
--flags ... enables copying of bsd flags
--osx (or similar) ... enables copying of osx related meta-data
(ResourceFork/FinderInfo/CreationDate)
-A ... enables copying of ACL information
-M ... maps any unsupported meta-data into an extended attribute on
the remote side (in the rsync namespace)
-Q ... silently drop unsupported meta-data
I'm not sure if --flags and --osx should be implied by -p, but I
think so.
Then, use a pseudo-design pattern where every type of meta-data rsync
can get/set has do_{get,set}_{acls,flags,owner,group,...}. A platform
also provides a sys_supports_{acls,flags,...}(filename). This returns
true/false if the meta-data is supported for that file (filesystems
differ, as do platforms). Then, if the method could ever return true,
the platform must provide matching sys_{set,get,could_set}_
(filename) methods. could_set reports if there are sufficient
permissions for the operation to succeed.
A design pattern like this could be used to easily extend rsync
whenever new types of meta-data appear. For each piece of meta-data,
the do_{get,set} methods look like this:
do_set_X(file, ...):
if (sys_supports_X(file)) {
if (fake_super and NOT sys_could_set_X(file)) {
return set_xattr(file, X, ...)
} else {
return sys_set_X(file, ...)
}
} else {
if (-M) {
return set_xattr(file, X, ...)
} else if (! -Q) {
warning: meta-data X is unsupported on platform Y for file, use -
Q to silence this warning, or -M to map it into an xattr.
return
}
}
do_get_X:
if (sys_supports_X(file)) {
if (fake_super and NOT sys_could_set_X(file)) {
return get_xattr(file, X)
} else {
return sys_get_X(file)
}
} else {
if (-M) {
return get_xattr(file, X)
}
if (! -Q) {
if (exist_xattr(file, X)) warning: discarding meta-data X that
was stored in an xattr
}
return nothing_to_sync
}
Real implementations would be complicated by meta-data that is
partially convertible. For example, file permissions and ACLs. But
logically, from a user point of view, one wants the above behaviour.
What do you think?
--
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: PATCH: preserve osx creation-date (was: Using pre2 for backing up a mac)
This is a general topic I'm interested in, as I rely on rsync to back
up my mac osx files
to a remote file server. Unfortunately, the file server is a sun
workstation, and I don't know
if it supports xattr's, or large xattr's that arise from resource
forks, and I'm not sure if the sys admins
back up resource forks when they backup the file server to tape.
I hacked together a bit of code to deal with this ( http://
www.shoup.net/xattr ).
The idea is to generate shadow files that store all the extra meta
data on the file server.
To avoid hacking rsync internals, these shadow files are stored in a
shadow directory with the
same structure as the original.
What would be nice is if rsync supported this directly, with the
shadow files (with names appropriately tweaked)
stored in the same directory structure. This is essentially Apple's
solution for dealing with foreign file systems
that are mounted locally (e.g., NFS shares or local UFS volumes), but
this does not work at all
between mac osx and a remote file server accessed via ssh.
I've seen some rsync patches on the web that do just this, but it
would be much better if it were in the main rsync source.
-- Victor Shoup
On Oct 15, 2007, at 6:13 AM, Wesley W. Terpstra wrote:
On Oct 15, 2007, at 1:55 AM, Wayne Davison wrote:
On Sun, Oct 14, 2007 at 08:14:57PM +0200, Wesley W. Terpstra wrote:
I've attached a patch which does this. Currently resource forks
and finder
info get placed into an extended attribute transparently by osx
(com.apple.{ResourceFork/FinderInfo}). This patch makes another
extended
attribute called com.apple.CreationDate.
Thanks! I've twiddled your patch a bit and checked it in as a
file in
the patches dir (for now, at least): osx-create-time.diff
I changed the storing of the data to store the time value as 12
bytes of
binary data -- 8 for the time_t value (for future expansion) and 4
for
the nsec value (which always be 0 at present).
I fail to see the advantage of a binary version. You add concerns
about byte-order and make it unreadable to a sysadmin. (I also
don't like how the ACL stuff is binary) There was no problem with 8
byte time_t vs. 4 byte values with the other format either as it
would've been good until year . If stat is textual, why not
this? Saving a few bytes is not a good reason ...
The name of the attribute was changed to com.apple.crtime96 (for
the moment) . Since it is not an official com.apple.* value, I
didn't want to use a name that Apple might choose in the future.
The name should probably go in the rsync.* namespace, but I'd need
to move the code for reading and writing the value out of lib/
sysxattrs.c into xattrs.c to do that (I believe).
Yeah, I suppose it should go in the rsync namespace too.
I also enabled the XATTR_NOFOLLOW option for {get,set}attrlist().
This option doesn't exist for these system calls. There is,
however, FSOPT_NOFOLLOW.
--
To unsubscribe or change options: https://lists.samba.org/mailman/
listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-
questions.html
--
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: PATCH: preserve osx creation-date (was: Using pre2 for backing up a mac)
On Oct 15, 2007, at 5:07 PM, Victor Shoup wrote: What would be nice is if rsync supported this directly, with the shadow files (with names appropriately tweaked) stored in the same directory structure. This is essentially Apple's solution for dealing with foreign file systems that are mounted locally (e.g., NFS shares or local UFS volumes), but this does not work at all between mac osx and a remote file server accessed via ssh. The problem with doing this is that you've separated the meta-data from the file. I don't think rsync should do this itself. It introduces problems when other tools interact with your backup. It's very easy to lose the meta-data. Unfortunately, ext3 seems to have a very low limit on extended attributes (resource forks tend to be too big). XFS works great, but on a couple of my machines (one is an Infrant ReadyNAS) I can't avoid using ext3. For this scenario I've written a FUSE filesystem that simply redirects extended attributes into an sqlite3 database (which it keys by inode, so nothing breaks). This accomplishes what you wanted with your shadow files, but on the filesystem level it doesn't separate the meta-data from the file. It also means rsync can stick to doing just its own job. It seems Solaris has FUSE, so maybe this would work for you too... http://opensolaris.org/os/project/fuse/ -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
PATCH: preserve osx creation-date (was: Using pre2 for backing up a mac)
On Oct 13, 2007, at 4:07 PM, Wesley W. Terpstra wrote:
rsync 3pre2 now handles all but three items:
1. bsd flags
2. locked flag (supposedly finder meta-data)
3. creation date
The creation date is AFAIK a mac os specific piece of meta-data and
can be accessed via the set/getattrlist + ATTR_CMN_CRTIME system
call on a mac. Again, this meta-data might be reasonably converted
into an extended attribute.
I've attached a patch which does this. Currently resource forks and
finder info get placed into an extended attribute transparently by
osx (com.apple.{ResourceFork/FinderInfo}). This patch makes another
extended attribute called com.apple.CreationDate. The only issue
I've found so far is that it gives warning when copying a symlink to
linux as one can't set extended attributes there. It even works when
copying from hfs to fat32 on the same mac. I don't understand where
osx saves the creation date in a fat filesystem, but somehow it works!?
I admit it's a bit hack-ish to put this in the extended attribute get/
set methods, but this mimics the behaviour of the other osx-specific
meta-data. Also, it seems reasonable to me that it should be copied
when '-X' is specified. There is no fgetattrlist method, but it seems
that sys_fgetxattr is actually never used b/c x_fstat is not used.
creation-date.patch
Description: Binary data
--
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: PATCH: preserve osx creation-date (was: Using pre2 for backing up a mac)
On Sun, Oct 14, 2007 at 08:14:57PM +0200, Wesley W. Terpstra wrote:
I've attached a patch which does this. Currently resource forks and finder
info get placed into an extended attribute transparently by osx
(com.apple.{ResourceFork/FinderInfo}). This patch makes another extended
attribute called com.apple.CreationDate.
Thanks! I've twiddled your patch a bit and checked it in as a file in
the patches dir (for now, at least): osx-create-time.diff
I changed the storing of the data to store the time value as 12 bytes of
binary data -- 8 for the time_t value (for future expansion) and 4 for
the nsec value (which always be 0 at present). The name of the
attribute was changed to com.apple.crtime96 (for the moment) . Since it
is not an official com.apple.* value, I didn't want to use a name that
Apple might choose in the future. The name should probably go in the
rsync.* namespace, but I'd need to move the code for reading and writing
the value out of lib/sysxattrs.c into xattrs.c to do that (I believe).
I also enabled the XATTR_NOFOLLOW option for {get,set}attrlist().
..wayne..
--
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
