Re: [9fans] p9p venti sync?
% venti/venti 2010/1116 20:44:14 venti: conf...2010/1116 20:44:14 err 4: read /Users/dave/venti/disks/bloom offset 0x0 count 65536 buf 380 returned 65536: No such file or directory A read of 65536 returning 65536 should not be an error. If you want to dig in, this is $PLAN9/src/cmd/venti/srv/part.c:/^prwb. There have been problems with short reads before. Russ
Re: [9fans] That deadlock, again
lock loops are about Locks (spin locks), not QLocks. the relative ordering of any two calls to qlock and qunlock is irrelevant. russ
Re: [9fans] p9p venti sync?
On Wed, Nov 17, 2010 at 6:36 AM, Russ Cox r...@swtch.com wrote: % venti/venti 2010/1116 20:44:14 venti: conf...2010/1116 20:44:14 err 4: read /Users/dave/venti/disks/bloom offset 0x0 count 65536 buf 380 returned 65536: No such file or directory A read of 65536 returning 65536 should not be an error. If you want to dig in, this is $PLAN9/src/cmd/venti/srv/part.c:/^prwb. There have been problems with short reads before. Russ I will look at this later tonight. It only happens when using a bloom file apparently so far. Dave
[9fans] Anyone using p9p or Plan 9 venti as a more generic backup system?
I'm giving consideration to maintaining a venti-based setup for my house for all the digital media we have (since getting our Apple TV, we've had more stuff to stream around the house). I've just now started playing with things like vac/unvac, to backup and extract trees of my HFS+ file system and I wonder about a few things. What do people do if they ever lose their venti scores? Seems like this is handle-less data now, and fairly useless. I figure I could keep a vac archive of venti scores, then I'd only need to remember one, and name the files I store the scores in reasonably, but if that's lost... what are my options? Dave
[9fans] nfactotum update
i pushed a minor update to sources. i've been able to boot an auth server with nfactotum built into the kernel. i've been able to cpu and drawterm into this machine. but testing has been light (and at 4am) so ymmv. - erik
Re: [9fans] Anyone using p9p or Plan 9 venti as a more generic backup system?
On Wed, Nov 17, 2010 at 11:51 AM, David Leimbach leim...@gmail.com wrote: I'm giving consideration to maintaining a venti-based setup for my house for all the digital media we have (since getting our Apple TV, we've had more stuff to stream around the house). I've just now started playing with things like vac/unvac, to backup and extract trees of my HFS+ file system and I wonder about a few things. What do people do if they ever lose their venti scores? Seems like this is handle-less data now, and fairly useless. I figure I could keep a vac archive of venti scores, then I'd only need to remember one, and name the files I store the scores in reasonably, but if that's lost... what are my options? Dave There is a script floating around (dumpvacroots or somesuch) that lets you recover vac scores given access to the venti arenas. I'd be very careful with vac -m and -a on Unix; both have been at the root of considerable data-loss on a unix venti for me. I'd recommend vac-ing tarballs, rather than using vac's on unix trees directly. But your mileage may vary... -- vs
Re: [9fans] Anyone using p9p or Plan 9 venti as a more generic backup system?
On Wednesday 17 November 2010 18:14:35 Venkatesh Srinivas wrote: (...) I'd be very careful with vac -m and -a on Unix; both have been at the root of considerable data-loss on a unix venti for me. I'd recommend vac-ing tarballs, rather than using vac's on unix trees directly. But your mileage may vary... could you please elaborate a bit about that data loss? traversing symlinks breaks? some files not getting read by vac at all? (I'm interested in using p9p vac+venti in similar manner, but on Linux w/ GNU stuff) -- dexen deVries ``One can't proceed from the informal to the formal by formal means.''
Re: [9fans] Anyone using p9p or Plan 9 venti as a more generic backup system?
On Wed, Nov 17, 2010 at 9:14 AM, Venkatesh Srinivas m...@acm.jhu.edu wrote: On Wed, Nov 17, 2010 at 11:51 AM, David Leimbach leim...@gmail.com wrote: I'm giving consideration to maintaining a venti-based setup for my house for all the digital media we have (since getting our Apple TV, we've had more stuff to stream around the house). I've just now started playing with things like vac/unvac, to backup and extract trees of my HFS+ file system and I wonder about a few things. What do people do if they ever lose their venti scores? Seems like this is handle-less data now, and fairly useless. I figure I could keep a vac archive of venti scores, then I'd only need to remember one, and name the files I store the scores in reasonably, but if that's lost... what are my options? Dave There is a script floating around (dumpvacroots or somesuch) that lets you recover vac scores given access to the venti arenas. I'd be very careful with vac -m and -a on Unix; both have been at the root of considerable data-loss on a unix venti for me. I'd recommend vac-ing tarballs, rather than using vac's on unix trees directly. But your mileage may vary... This is mainly a form of secondary backup for me for now, but given what I learn about it over time, it could become a primary. Are there any open problem reports around this? I might be interested in tackling some of these, or at least trying to reproduce them. I still do some rsync based backups anyway. -- vs
Re: [9fans] Anyone using p9p or Plan 9 venti as a more generic backup system?
On Wed, Nov 17, 2010 at 12:23 PM, dexen deVries dexen.devr...@gmail.com wrote: On Wednesday 17 November 2010 18:14:35 Venkatesh Srinivas wrote: (...) I'd be very careful with vac -m and -a on Unix; both have been at the root of considerable data-loss on a unix venti for me. I'd recommend vac-ing tarballs, rather than using vac's on unix trees directly. But your mileage may vary... could you please elaborate a bit about that data loss? traversing symlinks breaks? some files not getting read by vac at all? I have a number of vac archives that were made by vac-ing my homedir that cannot be mounted (venti can't find some of the scores in the tree). However, they did pass scrubbing (scrub was a tool i wrote that is sorta like half of venti/copy -- it traverses a tree, checks hashes, but doesn't write them out to anything). I have a number of trees in which somehow the SHA1 of the empty string was inserted: 'da39a3ee...'. And I have a number of trees that were venti/copied from one venti to another but somehow damaged (scores missing) after the copy, without seeing errors. vac -a in particular was good at producing archives which either don't mount or are filled with missing scores or zero-scores vs sha1(empty string) problems. Unvac is not useful for anything involving directories, since it outputs dirs which cannot be modified. http://groups.google.com/group/plan9port-dev/browse_thread/thread/8b0e2bc4b45d12b0 is a reasonably common bug from vac, seen on deep-ish directory trees. Two threads i can't find on the google groups interface, from p9p-dev as well: 1) 8/12/09 'venti/vac/vacfs trouble' and 2) 8/17/09 'vac errors'. Both bugs still exist afaik. I've seen this: cache block 501: type 8 score lots of zeros1f6 iostate 1 addr 502 ref 0 nlock 1 cache block 502: type 8 score lots of zeros1f7 iostate 1 addr 503 ref 0 nlock 1 cache block 503: type 8 score lots of zeros1f8 iostate 1 addr 504 ref 0 nlock 1 (output starts at 0x000...000 ;; goes up to score 0x000...200). vtcachebumpblock: no free blocks in vtCachezsh: IOT instruction ./vac -s -v -h tcp!acm.jhu.edu!17034 /export/home happen on deep-ish directory trees as well; I never managaed to track it down, though. Most recently, I've run two venti servers; vac targets the first one, every night the score is venti/copied to the second server. After two weeks, the two servers (with identical initial configuration) had different size and block counts from the http://.../storage. -- vs
Re: [9fans] Anyone using p9p or Plan 9 venti as a more generic backup system?
On Wed, Nov 17, 2010 at 9:23 AM, dexen deVries dexen.devr...@gmail.comwrote: On Wednesday 17 November 2010 18:14:35 Venkatesh Srinivas wrote: (...) I'd be very careful with vac -m and -a on Unix; both have been at the root of considerable data-loss on a unix venti for me. I'd recommend vac-ing tarballs, rather than using vac's on unix trees directly. But your mileage may vary... could you please elaborate a bit about that data loss? traversing symlinks breaks? some files not getting read by vac at all? (I'm interested in using p9p vac+venti in similar manner, but on Linux w/ GNU stuff) -- dexen deVries I could imagine vac/unvac not dealing with resource forks or POSIX extended attributes and such properly, as well as potentially having difficulty with symlinks, but having dealt with stuff like that in xar, I don't think it's too difficult to address. I may need to read up on venti and see what sorts of data types it supports. Might be time to add some extensions? Dave ``One can't proceed from the informal to the formal by formal means.''
Re: [9fans] Plan9 development
On 11/14/2010 04:44 PM, Charles Forsyth wrote: the list of unimplemented items in /sys/src/cmd/cc/c99* is: snip i can think of something else that's not been noticed, but what other things have you found? Why is __func__ listed as “unwanted”? I’ve found it useful for some logging functions. --Joel
Re: [9fans] That deadlock, again
one could move:
up-qpc = getcallerpc(q);
from qlock() before the lock(q-use); so we can see from where that
qlock gets called that hangs the exportfs call, or add another magic
debug pointer (qpctry) to the proc stucture and print it in dumpaproc().
Cinap, I tried your debugging code and got an odd panic at boot time.
Consistently:
panic: kernel fault: no user process pc=0xf01e739e addr=0x09e8
Having a look with acid, this seems to be caused by an attempt at
setting the debug PC (your up-qpctry) at a time when up has no
value yet.
Strangely, later in the qlock() code up is checked and a panic
issued if zero. I'm missing something here: it is possible to execute
this code
/sys/src/9/port/qlock.c:29,37 (more or less)
lock(q-use);
rwstats.qlock++;
if(!q-locked) {
q-locked = 1;
unlock(q-use);
return;
}
which is immediately followed by
if(up == 0)
panic(qlock);
If up is nil, but it looks like a bit of a one-way operation.
Anyway, I have moved the assignment to qpctry to after up is
tested. Let's see what happens. I'll have to get back to you once
the system is back up.
++L
Re: [9fans] That deadlock, again
Anyway, I have moved the assignment to qpctry to after up is tested. Let's see what happens. I'll have to get back to you once the system is back up. The system is working now. I have to wait for a problem to arise, next. ++L
Re: [9fans] That deadlock, again
Strangely, later in the qlock() code up is checked and a panic issued if zero. I'm missing something here: it is possible to execute this code you must be in process context to qlock, because only processes can sleep. - erik
Re: [9fans] Plan9 development
Why is __func__ listed as “unwanted”? I’ve found it useful for some logging functions. i think the correct interpretation of unwanted in this context is either don't want or don't want to implement. one former member of the don't-want list was varadic macros, which are now supported by both the compiler and cpp. i used to think that __func__ would be useful, but i've never actually found a use for it. there almost always seems to be a better option. - erik
[9fans] apic id =8 problems
i tried out the changes to pc/^(apic.c mp.^(c h)) to explictly support processors with apic id = 8. unfortunately my testing machine hangs with the changes even though its processors have apic ids 0-7. chula; aux/cpuid -t thread 0001 0002 0100 core0004 0008 0201 mach0 apic mach1 apic 0001 mach2 apic 0002 mach3 apic 0003 mach4 apic 0004 mach5 apic 0005 mach6 apic 0006 mach7 apic 0007 i'm just reporting this because i likely won't have much time to investigate in the short term. - erik
