>> the other part of the argument — the "write hole"
>> depends on two things that i don't think are universal
>> a) zfs' demand for transactional storage
>
> Huh?!?
why else would the zfs guys be worried about a
"write hole" for zfs?
what would happen to a raid-z if a write returned
as successf
On Jan 26, 2009, at 9:37 AM, erik quanstrom wrote:
the other part of the argument — the "write hole"
depends on two things that i don't think are universal
a) zfs' demand for transactional storage
Huh?!?
b) a particular raid implentation.
fancy raid cards
I think you missed what I in RAID
On Jan 26, 2009, at 8:39 AM, erik quanstrom wrote:
This approach will work too. But it seems that asking fossil
to verify a checksum when the block is about to go to venti
is not that much of an overhead.
if checksumming is a good idea, shouldn't it be available outside
fossil?
It is availabl
> Now, suppose I have a fossil buffer that I constantly snap to venti.
> That will build quite a lengthy chain of VtRoots. Then my fossil
> buffer gets totally corrupted. I no longer know what was the
> score of the most recent snapshot. And I don't think I know of any
> way to find that out.
the
On Mon, 2009-01-26 at 08:22 -0800, Russ Cox wrote:
> > As for me, here's my wish list so far. It is all about fossil, since
> > it looks like venti is quite fine (at least for my purposes) the
> > way it is:
> > 1. Block consistency. Yes I know the argument here is that you
> > can always r
> Yes. See here for details:
>http://blogs.sun.com/bonwick/entry/raid_z
since these arguments rely heavily on the meme that
software raid == bad
i have a hard time signing on. i believe i'm repeating
myself by saying that afik, there is no such thing as pure
hardware raid; that is, th
> This approach will work too. But it seems that asking fossil
> to verify a checksum when the block is about to go to venti
> is not that much of an overhead.
if checksumming is a good idea, shouldn't it be available outside
fossil?
perhaps the argument is that it might be more efficient
to imp
> As for me, here's my wish list so far. It is all about fossil, since
> it looks like venti is quite fine (at least for my purposes) the
> way it is:
> 1. Block consistency. Yes I know the argument here is that you
> can always roll-back to the last known archival snapshot on venti.
>
On Mon, 2009-01-26 at 08:53 -0500, erik quanstrom wrote:
> > It depends on the vdev configuration. You can do simple mirroring
> > or you can do RAID-Z (which is more or less RAID-5 done properly).
>
> "raid5 done properly"? could you back up this claim?
Yes. See here for details:
http://blog
On Mon, 2009-01-26 at 08:42 -0500, erik quanstrom wrote:
> > That is my *entire* point. If fossil doesn't tell you that
> > the data in its buffer was/is corrupted -- you have no
> > reason to rollback.
>
> if you're that worried, you do not need to modify fossil.
> why don't you write a sdecc d
> It depends on the vdev configuration. You can do simple mirroring
> or you can do RAID-Z (which is more or less RAID-5 done properly).
"raid5 done properly"? could you back up this claim?
also, with services like ec2, it's no use doing raid since all
your data could be on the same drive, regar
> > it's important to keep in mind that fossil is just a write buffer.
> > it is not intended for the perminant storage of data.
>
> Sure. But it must store the data *intact* long enough
> for me to be able to do a snap. It has to be able to
> at least warn me about data corruption.
do you have
On Tue, 2009-01-20 at 21:02 -0500, erik quanstrom wrote:
> > In such a setup a corrupted block from a fossil
> > partition will go undetected and could end up
> > being stored in venti. At that point it will become
> > venti "problem".
>
> it's important to keep in mind that fossil is just a writ
On Fri, 2009-01-23 at 22:36 -0500, erik quanstrom wrote:
> > You never know when end-to-end data consistency will start to really
> > matter. Just the other day I attended the cloud conference where
> > some Amazon EC2 customers were swapping stories of Amazon's networking
> > "stack" malfunctioni
On Tue, 2009-01-20 at 16:52 -0700, andrey mirtchovski wrote:
> for my personal $0.02 i will say that this argument seems to revolve
> around trying to bend fossil and venti to match the functionality of
> zfs and the design decisions of the team that wrote it.
That is NOT the conversation I'm int
> You never know when end-to-end data consistency will start to really
> matter. Just the other day I attended the cloud conference where
> some Amazon EC2 customers were swapping stories of Amazon's networking
> "stack" malfunctioning and silently corrupting data that was written
> onto EBS. All
> After spending sometime reading the sources and grokking fossil
> I don't think it is a walking disaster. Far from it.
>
> There are a couple of places where things can be improved,
> to make *me* happier (YMMV), and I'll try to focus on these
> in replying to Andrei's email. Just to get some
On Wed, 2009-01-21 at 20:02 +0100, Uriel wrote:
> On Wed, Jan 21, 2009 at 2:43 AM, Roman V. Shaposhnik wrote:
> > Sure, but I can't really use venti without using
> > fossil (again: we are talking about a typical setup
> > here not something like vac/vacfs), can I?
> >
> > If I can NOT than fossi
On Wed, 2009-01-21 at 19:53 +, Steve Simon wrote:
> > Fossil has always been a weak link, and probably will always be until
> > somebody replaces it. There was some idea of replacing it with a
> > version of ken's fs that uses a venti backend...
> >
> > Venti's rock solid design is the only thi
> Fossil has always been a weak link, and probably will always be until
> somebody replaces it. There was some idea of replacing it with a
> version of ken's fs that uses a venti backend...
i looked into how that would go enough to see
that venti would work at cross purposes to the
fs. having a w
> Fossil has always been a weak link, and probably will always be until
> somebody replaces it. There was some idea of replacing it with a
> version of ken's fs that uses a venti backend...
>
> Venti's rock solid design is the only thing that makes fossil
> minimally tolerable despite its usual ten
On Wed, Jan 21, 2009 at 2:43 AM, Roman V. Shaposhnik wrote:
> I was specifically referring to a "normal operations"
> to conjure an image of a typical setup of fossil+venti.
>
> In such a setup a corrupted block from a fossil
> partition will go undetected and could end up
> being stored in venti.
On Wed Jan 21 01:40:13 EST 2009, st...@quintile.net wrote:
> > ... fossil does have the functionality to serve two
> > different file systems from two different disks, but i don't think
> > anyone has used that ...
>
> I do this, 'main' backed up by venti and 'other' which holds useful stuff
> th
> ... fossil does have the functionality to serve two
> different file systems from two different disks, but i don't think
> anyone has used that ...
I do this, 'main' backed up by venti and 'other' which holds useful stuff
that needn't be backed up, e.g. RFCs, cdrom images, datasheets etc. This
>> What's the use of copying arenas to CD/DVD? Is it purely
>> back up, since they have to stay on-line forever?
> people who back up to cd/dvd can answer that :)
My venti, which backs a fossil used by 70 student accounts, of
which five are currently "active", fills arenas *very* slowly.
I burn
> > well, there's your problem. you corrupted
> > the cache, not the venti store. (you have no
> > venti store in this example.)
>
> I was specifically referring to a "normal operations"
> to conjure an image of a typical setup of fossil+venti.
>
> In such a setup a corrupted block from a fossi
On Tue, 2009-01-20 at 18:36 -0500, erik quanstrom wrote:
> > > > Got it. However, I'm still not fully convinced there's a definite edge
> > > > one way or the other. Don't get me wrong: I'm not trying to defend
> > > > ZFS (I don't think it needs defending, anyway) but rather I'm trying
> > > > to
> Is it how it was from the get go, or did you use venti-based solutions
> before?
it's how i found it.
>> i have two zfs servers and about 10 pools of
>> different sizes with several hundred different zfs filesystems and
>> volumes of raw disk exported via iscsi.
>
> What kind of clients are on
> > > Got it. However, I'm still not fully convinced there's a definite edge
> > > one way or the other. Don't get me wrong: I'm not trying to defend
> > > ZFS (I don't think it needs defending, anyway) but rather I'm trying
> > > to test my mental model of how both work.
> >
> > if you end up rew
On Tue, 2009-01-20 at 09:19 -0500, erik quanstrom wrote:
> > > in the case of zfs, my claim is that since zfs can reuse blocks, two
> > > vdev backups, each with corruption or missing data in different places
> > > are pretty well useless.
> >
> >
> > Got it. However, I'm still not fully convince
> 4. If I have a venti server and a bunch of sha1 codes, can I somehow
> instantiate a single fossil serving all of them under /archive?
Not at present, there is no way to insert a vac score into a fossil hierarchy
other than at the root of the hierarchy (flfmt -v).
what you can do is cop
> 1. What's the use of copying arenas to CD/DVD? Is it purely back up,
> since they have to stay on-line forever?
backup.
> 2. Would fossil/venti notice silent data corruptions in blocks?
venti would. the score wouldn't match the block.
> 3. Do you think its a good idea to hav
> > in the case of zfs, my claim is that since zfs can reuse blocks, two
> > vdev backups, each with corruption or missing data in different places
> > are pretty well useless.
>
>
> Got it. However, I'm still not fully convinced there's a definite edge
> one way or the other. Don't get me wrong:
Hi Andrey!
Sorry, it took me a longer time to dig through the code than
I hoped to. So, if you're still game...
On Jan 6, 2009, at 6:22 AM, andrey mirtchovski wrote:
i'm using zfs right now for a project storing a few terabytes worth of
data and vm images.
Is it how it was from the get go, or
I think I'm now ready to pick up this old thread (if anybody's still
interested...)
On Jan 7, 2009, at 5:11 PM, erik quanstrom wrote:
Lets see. May be its my misinterpretation of what venti does. But so
far I understand that it boils down to: I give venti a block of any
length, it gives me a sco
> Lets see. May be its my misinterpretation of what venti does. But so
> far I understand that it boils down to: I give venti a block of any
> length, it gives me a score back. Now internally, venti might decide
just a clarification. this is done by the client. from venti(6):
Files and Di
On Tue, 2009-01-06 at 18:44 -0500, erik quanstrom wrote:
> >> a big difference between the decisions is in data integrety.
> >> it's much easier to break a fs that rewrites than it is a
> >> worm-based fs.
> >
> > True. But there's a grey area here: an FS that *never* rewrites
> > live blocks, bu
>> a big difference between the decisions is in data integrety.
>> it's much easier to break a fs that rewrites than it is a
>> worm-based fs.
>
> True. But there's a grey area here: an FS that *never* rewrites
> live blocks, but can reclaim dead ones. That's essentially
> what ZFS does.
unfortu
On Tue, 2009-01-06 at 11:19 -0500, erik quanstrom wrote:
> very interesting post.
indeed. I actually need some time to digest it ;-)
> > this is an example of the design decision difference between
> > fossil/venti and zfs: venti commits storage permanently and everything
> > becomes a snapshot,
very interesting post.
> this is an example of the design decision difference between
> fossil/venti and zfs: venti commits storage permanently and everything
> becomes a snapshot, while the designers of zfs decided to create a
> two-stage process introducing a read-only intermediary between the
>
i'm using zfs right now for a project storing a few terabytes worth of
data and vm images. i have two zfs servers and about 10 pools of
different sizes with several hundred different zfs filesystems and
volumes of raw disk exported via iscsi. clones play a vital part in
the whole set up (they numbe
> >> I'm still trying to figure out what kind of approximation of the
> >> above
> >> would be possible with fossil/venti.
> >
> > how about making a copy? venti will coalesce duplicate blocks.
>
> But wouldn't you still need to send these blocks over the wire (thus
> consuming bandwidth and ti
Cool! Looks like I found a "bi-lingual" person! ;-) Andrey,
would you mind if I ask you to translate some other things
between ZFS and venti/fossil for me?
On Jan 4, 2009, at 9:24 PM, andrey mirtchovski wrote:
Well, I guess I really got spoiled by ZFS's ability to do things like
$ zfs snapshot
On Jan 4, 2009, at 9:12 PM, erik quanstrom wrote:
Well, I guess I really got spoiled by ZFS's ability to do things like
$ zfs snapshot pool/projects/f...@yourtextgoeshere
and especially:
$ zfs clone pool/projects/f...@yourtextgoeshere pool/projects/
branch
I'm still trying to figure out
> fossil/venti through the lens of ZFS. I guess its not a coincedence
> that ZFS actually has a built-in support for the kind of history
> transfer you were implementing.
the transfer would have been trivial, had the filesystems been
compatable. what i did was reenact the actions that built the
o
> Well, I guess I really got spoiled by ZFS's ability to do things like
>$ zfs snapshot pool/projects/f...@yourtextgoeshere
at the console type "snap". if you're allowing snaps to be mounted on
the local fs then the equivalent would be "mkdir /YourTextGoesHere;
bind /n/dump/... / /YourTextGoes
> Well, I guess I really got spoiled by ZFS's ability to do things like
> $ zfs snapshot pool/projects/f...@yourtextgoeshere
> and especially:
> $ zfs clone pool/projects/f...@yourtextgoeshere pool/projects/branch
>
> I'm still trying to figure out what kind of approximation of the above
>
On Mon, 2008-12-29 at 19:57 -0500, erik quanstrom wrote:
> history is not a property of venti. venti is a sparse virtual drive
> with ~2^80 bits storage. blocks are addressed by sha1 hash of
> their content. fossil is the fileserver. the analogy would be a change
> in fossil format. my techniqu
On Sun, 2009-01-04 at 07:03 +0900, sqweek wrote:
> On Tue, Dec 30, 2008 at 8:54 AM, Roman Shaposhnik wrote:
> > Personally, though, I'd say that the usefulness of the
> > dump would be greatly improved
> > if one had an ability to do ad-hoc archival snapshots AND assigning tags,
> > not only dates
On Tue, Dec 30, 2008 at 8:54 AM, Roman Shaposhnik wrote:
> Personally, though, I'd say that the usefulness of the
> dump would be greatly improved
> if one had an ability to do ad-hoc archival snapshots AND assigning tags,
> not only dates to them.
Tags don't make that much sense in this context
http://code.google.com/hosting/createProject
On Tue, Dec 30, 2008 at 12:31 PM, Uriel wrote:
> On Tue, Dec 30, 2008 at 4:06 PM, C H Forsyth wrote:
>>>Knowing *who* made the change is often even more useful than the change
>>>comment.
>>
>> yes. i use ls -lm on our trees, but that might not work
On Tue, Dec 30, 2008 at 4:06 PM, C H Forsyth wrote:
>>Knowing *who* made the change is often even more useful than the change
>>comment.
>
> yes. i use ls -lm on our trees, but that might not work on less direct things
> like sources.
It would work if the development trees were public...
uriel
>Knowing *who* made the change is often even more useful than the change
>comment.
yes. i use ls -lm on our trees, but that might not work on less direct things
like sources.
Knowing *who* made the change is often even more useful than the change comment.
uriel
On Tue, Dec 30, 2008 at 2:48 AM, Charles Forsyth wrote:
>>> i've rarely found per-change histories to be any more useful than
>>> most other comments, i'm afraid.
>
>>And that meant that math texts and math te
>> i've rarely found per-change histories to be any more useful than
>> most other comments, i'm afraid.
>And that meant that math texts and math teaching was all about polished
>final results.
ah. my statement was ambiguous.
i meant per-change chatter in the history, not the changes in the h
> I don't deny that 9fs dump is quite useful and it seems to match the
> organization of Plan9 developer
> club pretty well. Personally, though, I'd say that the usefulness of
> the dump would be greatly improved
> if one had an ability to do ad-hoc archival snapshots AND assigning
> tags, no
> ...but your article answered that last question completely. Although,
> I wonder whether direct transfer of history between two venti
> servers would be possible.
if one were to transfer history between two fs with the same on-disk
format, a simple device copy would be sufficient. i was moving
On Dec 27, 2008, at 3:56 AM, erik quanstrom wrote:
I'm actually still trying to figure out how replica/* fits together
with
sources being a fossil server. These two, somehow, have to
click, but I haven't figured out the connection just yet. Any
pointers
to the good docs?
there's no connect
So is it time for a new file server then? :D
On Dec 26, 2008, at 5:27 AM, Charles Forsyth wrote:
while a descriptive history is good, it takes a lot of extra work
to generate.
i've rarely found per-change histories to be any more useful than
most other comments, i'm afraid.
I believe that it all depends on what is it that you look at
I'm baffled. Slap me, or kick me--your choice.
--On Saturday, December 27, 2008 11:36 AM +0100 tlaro...@polynum.com wrote:
On Sat, Dec 27, 2008 at 06:04:42AM +, Eris Discordia wrote:
"it all begins with Adam and Steve," as Brian Stuart suggests, ways have
been found of managing large teams
> I'm actually still trying to figure out how replica/* fits together with
> sources being a fossil server. These two, somehow, have to
> click, but I haven't figured out the connection just yet. Any pointers
> to the good docs?
there's no connection. replica would work without a fossil server.
f
On Sat, Dec 27, 2008 at 06:04:42AM +, Eris Discordia wrote:
> "it all begins with Adam and Steve," as Brian Stuart suggests, ways have
> been found of managing large teams of people with different specializations
> and those ways work. The Mgmt has a raison d'etre, despite what
> techno-peop
On Dec 25, 2008, at 8:57 PM, Anthony Sorace wrote:
erik offered some suggestions for hosting various bits of things
outside 9vx and connecting to that in order to get the dumps. those
options are valid, but you can just as well host the entire thing
within 9vx. it's not the default configuration,
On Dec 25, 2008, at 6:37 AM, erik quanstrom wrote:
despite the season, and typical attitudes, i don't think that
development practices are a spiritual or moral decision.
they are a practical one.
Absolutely! Agreed 100%. My original question was not
at all aimed at "saving" Plan9 development
building a pyramid, starting at the top is one of those things
that just doesn't scale.
For that, you have "bottom-up," right? But there's no "meet-in-the-middle"
for a pyramid, or for software. Unless, the big picture is small enough to
fit in one man's head and let him "context-switch" back
> building a pyramid, starting at the top is one of those things
> that just doesn't scale.
But if you figure out how, it's probably worth a Nobel.
BLS
> Know why Mel is no more in business? 'Cause one man can only do so much
> work. The Empire State took many men to build, so did Khufu's pyramid, and
> there was no whining about "many mechanisms that don't work well together."
> Now go call your managers "PHBs."
building a pyramid, starting a
The Story of Mel
[...]
I compared Mel's hand-optimized programs with the same code massaged by
the optimizing assembler program, and Mel's always ran faster. That was
because the "top-down" method of program design hadn't been invented
yet, and Mel wouldn't have used it anyway. He wrote the inne
> On Fri, Dec 26, 2008 at 01:20:17PM -0500, erik quanstrom wrote:
>> appropriately, this being a plan 9 list and all, i find code
>> written from the bottom up easier to read.
>
> Depending on the task (on the aim of the software), one happens to split
> from top to bottom, and to review and amend
On Fri, Dec 26, 2008 at 01:20:17PM -0500, erik quanstrom wrote:
> appropriately, this being a plan 9 list and all, i find code
> written from the bottom up easier to read.
>
Depending on the task (on the aim of the software), one happens to split
from top to bottom, and to review and amend from b
>> Back when I used CWEB on a regular basis (I don't find myself
>> writing as much substantive code from scratch of late), I
is it just me, or is hard to read someone else's cweb code?
if it's not just me...
i wonder if the same reason it's easy to write from the top
down doesn't make it hard to
On Fri, Dec 26, 2008 at 11:25:33AM -0600, blstu...@bellsouth.net wrote:
>
> Back when I used CWEB on a regular basis (I don't find myself
> writing as much substantive code from scratch of late), I
> experienced an interesting phenomenon. I could write
> pretty good code, almost as a stream of co
> I use CWEB (D. Knuth and Levy's) intensively and it is indeed
> invaluable.
> It doesn't magically improve code (my first attempts have just shown
> how poor my programming was: it's a magnifying glass, and one just saw
> with it bug's blinking eyes with bright smiles).
Back when I used CWEB o
On Fri, Dec 26, 2008 at 01:27:49PM +, Charles Forsyth wrote:
> perhaps literate programming will fix that if it ever takes off.
I use CWEB (D. Knuth and Levy's) intensively and it is indeed
invaluable.
It doesn't magically improve code (my first attempts have just shown
how poor my programming
>the advantage of dump and snap is that the scope is the whole system:
>including emails, discussion documents,
>the code, supporting tools -- everything in digital form. if software works
>differently today
>compared to yesterday, then
sorry, got cut off. then in most cases, i'd expect 9fs
>while a descriptive history is good, it takes a lot of extra work
>to generate.
i've rarely found per-change histories to be any more useful than most other
comments, i'm afraid.
you'd hope it would answer "what was he thinking?" but i found either it was
obvious or i still had to ask.
still, p
> using fossil for your root, instead of #Z, will obviously cost you the
> benefits of #Z - namely, the pass-through transparency. if your
> primary interest is for replica/*, though, you might consider the
> direction i've been headed: root from fossil, but import $home or /usr
> from #Z.
That's
depends what you mean by "extra". if that means "outside 9vx", then
yes; if it means "besides what 9vx uses by default", no.
yesterday(1) relies on having dump-style snapshots. 9vx, as shipped,
gets its root file system from #Z, which doesn't have snapshots.
erik offered some suggestions for host
> True, but I'd really like to NOT have any extra software running and
> still have and ability to do replica/* and yesterday under 9vx.
I'm only vaguely familiar with 9vx, so there I can't speak, but you
can certainly do replica/* as it is a user-level tool and as for
yesterday, you can apply it
On Dec 24, 2008, at 10:40 PM, erik quanstrom wrote:
Is there any preferred way to get changelogs / diffs these days?
yesterday -d ...
when i'm especially curious or anxious.
But yesterday won't work in a more lightweight environment (such as
9vx) will it?
exactly the same as plan 9 does.
a
> I surely hope the festive mood of the season will protect me from being
> ostracized for asking this, but is there any chance to map Plan9
> development practices to some of the established ways of source
> code management? I mostly long for things like being able to browse
> Plan9 history with a
On Dec 22, 2008, at 8:46 PM, Nathaniel W Filardo wrote:
Hi,
The contrib index mentions that daily changelogs for Plan 9 are in
sources/extra/changes, but those haven't been updated since early
2007.
Is there any preferred way to get changelogs / diffs these days?
Relatedly, is there a bette
>>> Is there any preferred way to get changelogs / diffs these days?
>>
>> yesterday -d ...
>> when i'm especially curious or anxious.
>
> But yesterday won't work in a more lightweight environment (such as
> 9vx) will it?
exactly the same as plan 9 does.
as long as the fs supports a dump fs, 9v
On Dec 22, 2008, at 8:41 AM, Charles Forsyth wrote:
Is there any preferred way to get changelogs / diffs these days?
yesterday -d ...
when i'm especially curious or anxious.
But yesterday won't work in a more lightweight environment (such as
9vx) will it?
it probably wouldn't hurt to have a
> Hi,
>
> The contrib index mentions that daily changelogs for Plan 9 are in
> sources/extra/changes, but those haven't been updated since early 2007.
> Is there any preferred way to get changelogs / diffs these days?
Relatedly, is there a better way to mirror the development history of Plan 9
tha
It is pretty much a question of it being a totally backwards way of
doing things, with one set of people doing the changes, and another
set of people guessing the meaning of the changes writing the
changelog.
(This is claimed to be due to the first set of people not having the
time to writing down
2008/12/22 Venkatesh Srinivas :
> Hi,
>
> The contrib index mentions that daily changelogs for Plan 9 are in
> sources/extra/changes, but those haven't been updated since early 2007.
> Is there any preferred way to get changelogs / diffs these days?
I used to maintain the changelogs, but ended up
>Is there any preferred way to get changelogs / diffs these days?
i use
9fs sources
diff /whatever /n/sources/plan9/whatever
and after a pull
yesterday -d ...
when i'm especially curious or anxious.
it probably wouldn't hurt to have a DMEXCL+DMAPPEND file (!)
maintained by the command that app
> Also, in sources/patch, there are patches neither in applied/ or sorry/.
> Are these patches in queue? Applied? Not applied?
in the queue and not applied.
- erik
Hi,
The contrib index mentions that daily changelogs for Plan 9 are in
sources/extra/changes, but those haven't been updated since early 2007.
Is there any preferred way to get changelogs / diffs these days?
Also, in sources/patch, there are patches neither in applied/ or sorry/.
Are these patch
91 matches
Mail list logo
