Re: [9fans] p9p vac/vacfs compatibility: uid/gid, ctime, 9P2000.L, 9pserve
OK, I'm stumped. Maybe I'm making this more complicated than it has to be. Posing my question as simply as possible: If plan9port's vac is used to archive a Linux file system to venti, is there any way to access that file system in a way that exposes symbolic links, device nodes, sockets, and FIFOs in the archive? The Vac file system does already handle some foreign file system properties, but not everything. If you don't want to change the Vac file system format, you'd probably better use vbackup(8) to store an Unix file system directly on Venti. (1) When using Linux's v9fs to mount a file system served by plan9port's vacfs, the symlinks and device nodes all stat as regular files. The file size of symlinks reflects the length of the link target, and the device files have a size of 3 bytes (for type, major, and minor). This happens whether or not I include version=9p2000.u or 9p2000.L as mount options. That's fine, because for both DMSYMLINK and DMDEVICE, the required information (respectively target and type/major/minor) is written in the data block. Using 9p2000.u or 9p2000.l is not likely to change anything, since vacfs and 9pserve only serve 9P2000 anyway. When I use 9pfuse to mount the file system, I get a little further: lrw-rw 1 root root6 Mar 10 2005 video - -rw-rw 1 root root3 Mar 10 2005 video10 The symbolic links show up as links, but with the null string () as target. The Linux kernel interperets null symlinks to mean ./, causing all the links to refer to the directory in which they reside. :( The device files still present as ordinary files with a size of 3 bytes. Running readlink on the symbolic links always returns the null string. I'm still not sure where the problem is. V9fs only seems to display file and directory modes. 9pfuse seems to display symlink too. But neither of two seems to correctly interpret the special files from their mode and data. The vac code looks correct. I just found a bug in the handling of DMSYMLINK and DMDEVICE, but it's not directly related to your problem. (2) plan9port's unvac explicitly skips extraction of files which are not regular files or directories. So, unvac-ing the archive isn't possible, either. :( Unvac is a special case because it extracts files from a file system to another file system. But some properties may not be settable on the target file system. Unvac is rather safe concerning the files it extracts and the properties it sets. Fell free to change its behavior. (4) The 9p utility provided by plan9port gets me access to a little more metadata. Using the 9p ls command: -Lrwxrwxrwx M 0 root root 5 Jan 2 03:16 modem D-rw-rw M 0 root disk 3 Mar 10 2005 hda You can see the symlink and device bits set in the mode. (The D represents a device node; d indicates a directory.) The mode itself is just a 32-bit integer and is perfectly handled by both Vac and 9P. Using the 9p stat command: 'modem' 'root' 'root' '' q (0003 0 ) m 020777 at 1325474216 mt 1325474215 l 5 t 0 d 0 'tty3' 'root' 'tty' '' q (1528 0 ) m 04660 at 1110476900 mt 1110476900 l 3 t 0 d 0 The targets of symlinks are still showing up as the null string. (I'm assuming that's what the '' represents.) No, this is the mid, which is not handled on Unix file systems. The 9P stat format is described in stat(9p). The symlink target is in the data blocks. (5) Installing Inferno just to access vac'd Linux file systems would really be tantamount to throwing up my hands and giving up on plan9port. I'd like use plan9port, if possible, rather than have to layer a whole new OS on top of Linux+p9p. I can't see how Inferno could help you. The Inferno vacfs doesn't even handle any Unix file system property. -- David du Colombier
Re: [9fans] p9p vac/vacfs compatibility: uid/gid, ctime, 9P2000.L, 9pserve
When I use 9pfuse to mount the file system, I get a little further: lrw-rw 1 root root6 Mar 10 2005 video - -rw-rw 1 root root3 Mar 10 2005 video10 The symbolic links show up as links, but with the null string () as target. The Linux kernel interperets null symlinks to mean ./, causing all the links to refer to the directory in which they reside. :( The device files still present as ordinary files with a size of 3 bytes. Running readlink on the symbolic links always returns the null string. OK, I found the problem. At least concerning 9pfuse and symlink. 9pfuse read the symlink target from Dir.ext (libc.h), which is a 9P2000.u extension. But 9P2000.u is not supported anymore. It should read the symlink target from the data instead. I've something that work, but it's ugly. As supposed, other Unix special files don't seem to be currently supported by 9pfuse at all. Honestly, I've been using Vac for years and I never encountered the need to store Unix special files. If you want to back up a full Unix file system to Venti, you should simply use vbackup(8). -- David du Colombier
Re: [9fans] p9p vac/vacfs compatibility: uid/gid, ctime, 9P2000.L, 9pserve
David du Colombier 0in...@gmail.com writes: OK, I found the problem. At least concerning 9pfuse and symlink. OK, great. So I'm not going crazy, after all. ;) I've something that work, but it's ugly. Would you care to share? As supposed, other Unix special files don't seem to be currently supported by 9pfuse at all. I think that would require one or two new command line options to 9pfuse, to pass -o dev,suid to fusermount. (fusermount defaults to -o nodev,nosuid.) Come to think of it, 9pfuse should probably offer a general way to pass mount options to fusermount, as a matter of indirection. If you want to back up a full Unix file system to Venti, you should simply use vbackup(8). That could work, if I also wanted to archive everything under /tmp (and all the other temporary files peppered about Unix file systems). -- +---+ |Smiley smi...@icebubble.orgPGP key ID:BC549F8B | |Fingerprint: 9329 DB4A 30F5 6EDA D2BA 3489 DAB7 555A BC54 9F8B| +---+
Re: [9fans] p9p vac/vacfs compatibility: uid/gid, ctime, 9P2000.L, 9pserve
Would you care to share?
Sure. But beware it's just a quick hack.
diff -r 92a579c6549c src/cmd/9pfuse/main.c
--- a/src/cmd/9pfuse/main.c Mon Dec 12 16:44:50 2011 -0500
+++ b/src/cmd/9pfuse/main.c Mon Jan 02 19:00:00 2012 +0100
@@ -837,6 +837,8 @@
void
fusereadlink(FuseMsg *m)
{
+ int n;
+ uchar *buf;
Dir *d;
CFid *fid;
@@ -852,9 +854,19 @@
replyfuseerrno(m, EINVAL);
return;
}
- replyfuse(m, d-ext, strlen(d-ext));
+ n = d-length;
free(d);
- return;
+ if(n fusemaxwrite)
+ n = fusemaxwrite;
+ buf = emalloc(n);
+ n = fsread(fid, buf, n);
+ if(n 0){
+ free(buf);
+ replyfuseerrstr(m);
+ return;
+ }
+ replyfuse(m, buf, n);
+ free(buf);
}
/*
diff -r 92a579c6549c src/cmd/vac/vacfs.c
--- a/src/cmd/vac/vacfs.c Mon Dec 12 16:44:50 2011 -0500
+++ b/src/cmd/vac/vacfs.c Mon Jan 02 19:00:00 2012 +0100
@@ -548,8 +548,6 @@
n = vacdirread(f, buf, off, cnt);
else if(vacfilegetmode(f-file)ModeDevice)
return vtstrdup(device);
- else if(vacfilegetmode(f-file)ModeLink)
- return vtstrdup(symbolic link);
else if(vacfilegetmode(f-file)ModeNamedPipe)
return vtstrdup(named pipe);
else
Then symlinks are working. Now you need to fix the uid/gid
problem and handle devices, named pipes and sockets in 9pserve.
Also, you should apply this patch:
http://codereview.appspot.com/download/issue5500097_1.diff
It fixes two problems:
- fix segfault when the first file is DMSYMLINK or DMDEVICE,
- set the buffer to zero before calling readlink.
Symlinks will not work without this patch.
--
David du Colombier
Re: [9fans] p9p vac/vacfs compatibility: uid/gid, ctime, 9P2000.L, 9pserve
OK, I'm stumped. Maybe I'm making this more complicated than it has to be. Posing my question as simply as possible: If plan9port's vac is used to archive a Linux file system to venti, is there any way to access that file system in a way that exposes symbolic links, device nodes, sockets, and FIFOs in the archive? There are a number of possibilities I can think of (listed in approximate order of increasing obscurity): (1) plan9port's vacfs (2) plan9port's unvac (3) Netfiles (under acme) (4) plan9port's 9p command (5) Inferno I have a small test Linux filesystem with a mixture of regular files, directories, and character and block devices which I've archived (successfully!) using plan9port's vac. I can see the special files in the venti data. They're in there! But... (1) When using Linux's v9fs to mount a file system served by plan9port's vacfs, the symlinks and device nodes all stat as regular files. The file size of symlinks reflects the length of the link target, and the device files have a size of 3 bytes (for type, major, and minor). This happens whether or not I include version=9p2000.u or 9p2000.L as mount options. When I use 9pfuse to mount the file system, I get a little further: lrw-rw 1 root root6 Mar 10 2005 video - -rw-rw 1 root root3 Mar 10 2005 video10 The symbolic links show up as links, but with the null string () as target. The Linux kernel interperets null symlinks to mean ./, causing all the links to refer to the directory in which they reside. :( The device files still present as ordinary files with a size of 3 bytes. Running readlink on the symbolic links always returns the null string. (2) plan9port's unvac explicitly skips extraction of files which are not regular files or directories. So, unvac-ing the archive isn't possible, either. :( (3) According to the Plan 9 wiki, Netfiles will only work with versions of OpenSSH older than the version I have. (4) The 9p utility provided by plan9port gets me access to a little more metadata. Using the 9p ls command: -Lrwxrwxrwx M 0 root root 5 Jan 2 03:16 modem D-rw-rw M 0 root disk 3 Mar 10 2005 hda You can see the symlink and device bits set in the mode. (The D represents a device node; d indicates a directory.) Using the 9p stat command: 'modem' 'root' 'root' '' q (0003 0 ) m 020777 at 1325474216 mt 1325474215 l 5 t 0 d 0 'tty3' 'root' 'tty' '' q (1528 0 ) m 04660 at 1110476900 mt 1110476900 l 3 t 0 d 0 The targets of symlinks are still showing up as the null string. (I'm assuming that's what the '' represents.) (5) Installing Inferno just to access vac'd Linux file systems would really be tantamount to throwing up my hands and giving up on plan9port. I'd like use plan9port, if possible, rather than have to layer a whole new OS on top of Linux+p9p. So, I'm fresh out of ideas. Is there any way, at all, to present the full contents of a Linux filesystem that's been archived using the plan9port vac command? Thanks! -- +---+ |Smiley smi...@icebubble.orgPGP key ID:BC549F8B | |Fingerprint: 9329 DB4A 30F5 6EDA D2BA 3489 DAB7 555A BC54 9F8B| +---+
Re: [9fans] p9p vac/vacfs compatibility: uid/gid, ctime, 9P2000.L, 9pserve
On Wed, Dec 28, 2011 at 4:48 PM, David du Colombier 0in...@gmail.com wrote: Forget about 9p2000.u, it's deprecated. Who decided that .u is deprecated? I definitely didn't and I am using it for all my file servers that are supposed to work on Unix. Thanks, Lucho
Re: [9fans] p9p vac/vacfs compatibility: uid/gid, ctime, 9P2000.L, 9pserve
Who decided that .u is deprecated? I definitely didn't and I am using it for all my file servers that are supposed to work on Unix. Sorry, I mean the plan9port support for 9p2000.u is deprecated. It was removed in december 2010. -- David du Colombier
Re: [9fans] p9p vac/vacfs compatibility: uid/gid, ctime, 9P2000.L, 9pserve
IIRC, the RFC for it didn't indicate deprecated status. (Perhaps IETF could use an update.) Assuming it has been deprecated, with what has 9p2000.u been replaced? As far I know, these RFCs were early draft and were never completed nor submitted. I think the 9P2000.u authors moved to a new approach, which was called 9P2010 and is now called 9P2000.l. I don't know anything about it however. Anyway, plan9port applications only speak 9P2000. -- David du Colombier
Re: [9fans] p9p vac/vacfs compatibility: uid/gid, ctime, 9P2000.L, 9pserve
David du Colombier 0in...@gmail.com writes: The problem you are encountering is that v9fs doesn't translate uid/gid strings back to their numeric equivalent in /etc/passwd. Since Unix doesn't know how to handle these strings, it falls back to -1. I had suspected that. However, the uid/gid value being displayed is not -1. It's -2. Maybe that's what threw me. Perhaps, somewhere, there's: #define EDRIVECRAZY -2 Forget about 9p2000.u, it's deprecated. IIRC, the RFC for it didn't indicate deprecated status. (Perhaps IETF could use an update.) Assuming it has been deprecated, with what has 9p2000.u been replaced? -- +---+ |Smiley smi...@icebubble.orgPGP key ID:BC549F8B | |Fingerprint: 9329 DB4A 30F5 6EDA D2BA 3489 DAB7 555A BC54 9F8B| +---+
[9fans] p9p vac/vacfs compatibility: uid/gid, ctime, 9P2000.L, 9pserve
Hello,
I'm trying to use vac/vacfs on p9p, using the Linux v9fs driver (-t 9p)
with unix domain sockets as transport (-o trans=unix). And I'm
confused. :(
When I mount the file system, all the files are owned by uid = gid = (-2
mod 2^32), unless I tell the Linux kernel to mount the fs as a specific
user/group. In the latter case, all files present with the specified
user/group (as expected). I can't find any way to get at the actual
uid/gids owning the files/directories vac'd. I have tried explicitly
specifying -o version=9p2000.u, but it exhibits the same symptoms: all
files and directories appear to be owned by uid = gid = (-2 mod 2^32).
All uids/gids in the vac'd tree existed in /etc/{passwd,group} at the
time of vac'ing, as well as at mount time.
So, I took a little looksie in /usr/lib/plan9/src/cmd/vac/...
/usr/lib/plan9/src/cmd/vac/vac.c:48
#ifdef PLAN9PORT
/*
* We're between a rock and a hard place here.
* The pw library (getpwnam, etc.) reads the
* password and group files into an on-stack buffer,
* so if you have some huge groups, you overflow
* the stack. Because of this, the thread library turns
* it off by default, so that dirstat returns 14571 instead of rsc.
* But for vac we want names. So cautiously turn the pwlibrary
* back on (see threadmain) and make the main thread stack huge.
*/
extern int _p9usepwlibrary;
int mainstacksize = 4*1024*1024;
#endif
/usr/lib/plan9/src/cmd/vac/vac.c:399
vd-uid = dir-uid;
vd-gid = dir-gid;
vd-mid = dir-muid;
So, the source *appears* to stow uid/gid to venti as character strings.
BTW,
/usr/lib/plan9/src/cmd/vac/vac.c:406
vd-ctime = dir-mtime; /* ctime: not available on plan 9 */
Does this mean that vac doesn't store ctimes on p9p? Shouldn't this be
wrapped in something like:
#ifdef PLAN9PORT
vd-ctime = dir-ctime; /* ctime: available on p9p */
#else
vd-ctime = dir-mtime; /* ctime: not available on plan 9 */
#endif
Then, we have:
/usr/lib/plan9/src/cmd/vac/vacfs.c:696
dir.atime = vd-atime;
dir.mtime = vd-mtime;
dir.length = vd-size;
dir.name = vd-elem;
dir.uid = vd-uid;
dir.gid = vd-gid;
dir.muid = vd-mid;
#ifdef PLAN9PORT
dir.ext = ext;
dir.uidnum = atoi(vd-uid);
dir.gidnum = atoi(vd-gid);
#endif
Those dir.uidnum and dir.gidnum don't appear to be referenced anywhere
else in the code. But it makes me wonder... are the character string
uid/gids vac'd the textual names from /etc/{passwd,group}, or just ASCII
versions of the numeric ids, a la format %d?
And just what is 9P2000.L, anyway? I can't seem to find any
documentation on it.
Last, but not least, can anyone tell me why 9pserve is taking up so much
CPU time? From the docs, it looks like 9pserve should just be muxing 9p
connections and passes messages around. I had expected that disk
bandwith would be the bottleneck in my setup. But, as it turns out, the
bottleneck is actually this little fellow, 9pserve. I have no idea why
it's taking so much CPU to multiplex 9p, especially since there's only
one connection being multiplexed! Is there any way to bypass 9pserve
(since I only need one connection for now), i.e., doing 9p on sockets or
shared file descriptors?
Thanks!
--
+---+
|Smiley smi...@icebubble.orgPGP key ID:BC549F8B |
|Fingerprint: 9329 DB4A 30F5 6EDA D2BA 3489 DAB7 555A BC54 9F8B|
+---+
Re: [9fans] p9p vac/vacfs compatibility: uid/gid, ctime, 9P2000.L, 9pserve
When I mount the file system, all the files are owned by uid = gid =
(-2 mod 2^32), unless I tell the Linux kernel to mount the fs as a
specific user/group. In the latter case, all files present with the
specified user/group (as expected). I can't find any way to get at
the actual uid/gids owning the files/directories vac'd. I have tried
explicitly specifying -o version=9p2000.u, but it exhibits the same
symptoms: all files and directories appear to be owned by uid = gid =
(-2 mod 2^32). All uids/gids in the vac'd tree existed
in /etc/{passwd,group} at the time of vac'ing, as well as at mount
time.
The problem you are encountering is that v9fs doesn't translate
uid/gid strings back to their numeric equivalent in /etc/passwd.
Since Unix doesn't know how to handle these strings, it falls
back to -1.
Forget about 9p2000.u, it's deprecated.
So, the source *appears* to stow uid/gid to venti as character
strings.
This is the case. You can show the actual uid/gid from the
Plan 9 vacfs and even on p9p, using vacfs -d.
Does this mean that vac doesn't store ctimes on p9p?
Shouldn't this be wrapped in something like:
Yes, vac doesn't store ctime. Maybe we could store it in vac,
as you suggest, but 9P doesn't handle it anyway.
Those dir.uidnum and dir.gidnum don't appear to be referenced anywhere
else in the code.
I never paid much attention to it. It looks unused.
But it makes me wonder... are the character string
uid/gids vac'd the textual names from /etc/{passwd,group}, or just
ASCII versions of the numeric ids, a la format %d?
They are the textual names.
The problem you are describing is not related to vac nor vacfs.
They both handle uid/gid properly, and, as you saw, p9p vac can
even translate numeric uid/gid to their string representation.
--
David du Colombier
