[N.B. I pressed Brian about this in conversation on IRC yesterday, arguing that it wouldn't hurt to make users explicitly enable immutable-dirs-in-backups before they could use them.]
On Thursday, 2009-11-19, at 22:01 , Brian Warner wrote: > * "tahoe backup" is incredibly faster with immutable directories ... > * the directories it creates are repairable ... > * The only directory that has a DIR2-CHK objects placed into it is > the top-level target of the "tahoe backup" command ... > * I don't believe that "tahoe backup" is the tool of choice for > sharing files ... > So yeah, if we were talking about changing "tahoe mkdir" to create > new caps, then I'd agree with a conservative approach and add a > config knob to slowly introduce the new feature over time. But for > "tahoe backup", I think the benefits outweigh the downsides. Okay, good arguments! Let's do it your way! Now about the more general principle for future reference: >> I would like people to get used to the idea that they can always >> upgrade to the new version of Tahoe-LAFS and continue sharing with >> their more conservative friends without having to even think >> about these issues. > > Ok, but then how is it possible to ever introduce new features? By an explicit action on the part of the user. This could be setting a configuration option or (better) just clicking on a different button in the WUI or invoking a different command in the CLI or adding a "--newfangled-awesomeness" option in the CLI. Also, with sufficient passage of time we -- the Tahoe-LAFS developers -- can observe that people have in practice upgraded to a sufficiently new version that we can make the new feature default on in the next release of Tahoe-LAFS after that. (See below about that.) > A good concrete example would be Elk Point.. N.B. Elk Point won't necessarly be the next cap format. There is a project to invent a New Cap format [1], and Elk Point is one candidate for that design. I really like Elk Point -- a lot! -- but we might end up implementing a different candidate (at least at first), or we might end up implementing a subset of Elk Point at first for reasons of simplicity -- ease of implementation, verification, testing, deployment. The next step on New Cap Design project is, I think, to document the alternatives in a comparable level of detail to the current Elk Point docs -- svg diagrams, What Could Go Wrong [2], etc. But yes, a good concrete example would be Elk Point or some other New Cap format. > This could mean that some snazzy new features might be implemented > and shipped but go effectively unused for a year or more, depending > upon the release schedule and our overall patience with old versions. Be careful about this assumption that seems to be implicit in this statement -- that new features go unused if they are not turned on by default. If the feature isn't desirable enough that the user learns about it and explicitly indicates the willingness to use it (at the cost of sharing files with her stick-in-the-mud friends), then perhaps that means it's just not that important. They can have it by default a few years later. :-) But I suspect that good new features *would* get used before they are turned on by default. For example, suppose New Caps come out in Tahoe-LAFS v1.7, are faster to create and use, offer better security, and generate smaller, prettier, and more usable caps. Suppose that the "create a new directory" button continues to produce old-caps just like it did in Tahoe-LAFS v1.6. But, suppose that there is a checkbox next to it that says "new cap format". Users can easily (without even reading the NEWS.txt) figure out how the button behaves differently with and without the checkbox, figure out that their old-version-using friends can't read the new caps, and then decide for themselves whether to keep using old-caps or to persuade their friends to upgrade. Let me restate my use case now that you've persuaded me that immutable dirs in backupdb is okay. It's not that new features have to be disabled by default, it's that: A user wants to know if she can upgrade to the new release of Tahoe- LAFS, briefly glance at the NEWS.txt file, and continue to interoperate with her old-version-using friends, without significantly altering her workflow. I still think this use case is important, because if we miss it by too much then our user base splits into two camps and we have to either support both or abandon the stick-in-the-muds. I think it is possible to really screw up here by overestimating the amount of work users are willing to undertake in order to upgrade. For example, the darcs project made this mistake and now they have two repository formats to support and so far nobody has figured out how to get the stick-in-the-muds (which includes us) to make the jump to the new format. For another example, the Python project made a move like this (the backwards-incompatible Python 3), and I'm still waiting to find out how many years, if ever, before stick-in-the-muds like us make the jump to Python 3. But, I think you're right that there are some cases, such as immutable-dirs in backupdb, where the current user base and their current workflows don't run afoul of the backwards-compatibility issue. By the way, I checked and allmydata.com customers who use the allmydata.com Windows client are still using Tahoe-LAFS v1.1.0, which came out in June of 2008, plus a few critical bugfixes to the WUI which were backported from newer releases of Tahoe-LAFS. (A few of the "power user" customers of allmydata.com use Tahoe-LAFS v1.4.1 or v1.5.0 with the allmydata.com grid, on Linux and on Mac OS X.) I'm fairly sure that none of those allmydata.com-Windows-client-using users wants to interoperate with modern Tahoe-LAFS users, so I'm not worried about that, but it is interesting to note. I wouldn't be surprised if the addition of immutable directories is the final straw that prompts allmydata.com to make the jump to a modern version of Tahoe-LAFS, bringing them up to Tahoe-LAFS v1.6. (It would improve client performance and reduce load on the allmydata.com storage grid considerably.) The other pool of potential stick-in-the-muds is Ubuntu users. The current version of Ubuntu -- 9.10 (Karmic Koala) -- comes with Tahoe- LAFS v1.5. I think we'll be able to get Tahoe-LAFS v1.6 into Ubuntu 10.04 if we pay attention to their schedule [3]. If that happens then the people who use Tahoe-LAFS as packaged by Ubuntu will probably upgrade to Tahoe-LAFS v1.6 because Ubuntu 10.04 is an LTS (Long-Term-Support) release. After that, a lot of them will not upgrade to a newer Ubuntu for many years. Whatever version of Tahoe- LAFS ships in Ubuntu 10.04 will remain in widespread use for at least five years. >> This is also why "forward-compatibility" features are so important >> to me. They can sometimes ease these transitions. > > Yes! I'm very glad you pushed for #708 (tolerate unknown nodes), > because having it implemented in v1.5 made DIR2-CHK much easier to > deploy in v1.6, and makes me feel more confident about things like > Elk Point. Yes! That worked out well! Now think about what features you would like to add to Tahoe-LAFS v2.0, and then think about what we can do in v1.6 or v1.7 to provide forward-compatibility with those features. :-) Regards, Zooko [1] http://allmydata.org/trac/tahoe/wiki/NewCapDesign [2] http://allmydata.org/trac/tahoe/wiki/NewCaps/WhatCouldGoWrong [3] https://wiki.ubuntu.com/LucidReleaseSchedule? action=show&redirect=LucidLynxSchedule > _______________________________________________ tahoe-dev mailing list [email protected] http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev
