Re: [zfs-discuss] RFE: creating multiple clones in one zfs(1) call and one txg
On 29 Mar 2009, at 23:19, Jeff Bonwick wrote: I agree with Chris -- I'd much rather do something like: zfs clone snap1 clone1 snap2 clone2 snap3 clone3 ... than introduce a pattern grammar. Supporting multiple snap/clone pairs on the command line allows you to do just about anything atomically. Can you elucidate how this will help me take a single snap and clone it 1000 times, quickly and with minimum fuss? -a ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] RFE: creating multiple clones in one zfs(1) call and one txg
Hi, The inability to create more than 1 clone at a time (ie: in separate TXGs) is something which has hampered me (and several projects on which I have worked) for some years, now. Specifically I am looking at various forms of diskless grid/cloud environments where you create a golden image, snapshot it, and then clone that snapshot perhaps 1000 times for 1000 machines... poking the image slightly each time, and letting DHCP pick up the administrative slack of systems management. To create 1000 clones in this fashion (for i in `range 1 1000` do ; zfs clone [...] ; done) may take well over 1 hour, because 1000 TXGs need to be set up and committed. I would like to create 1000 clones in rather less than a couple of minutes. I've kicked this idea around with Darren Moffat and he informs me that it is painful to achieve the obvious solution: zfs clone tank/s...@version tank/foo tank/bar tank/baz ... ...because to explicitly specify each (of multiple) clone names on the cmdline causes multiple calls to ioctl() and therefore multiple TXGs, and thus slowness. Thus we hit on this proposal, for your consideration: zfs multiclone tank/f...@1 tank/PATTERN BEGIN END [STRIDE] - implement limited but comprehensive snprintf semantics for PATTERN - support: %d, %3d, %03d, FOO%d, FOO%dBAR, FOO%#08xBAR - includes decimal, hex, octal - BEGIN, END, STRIDE all decimal positive integers - STRIDE optional, defaults to 1 - creation begins at BEGIN, increments by STRIDE, continues until END is exceeded Examples: - zfs multiclone tank/sh...@1 tank/fish%02d 0 7 2 - creates: /tank/fish00 /tank/fish02 /tank/fish04 /tank/fish06 - zfs multiclone tank/gold-image tank/diskless/node%d.root 1 100 - ...is pretty obvious. (.../node1.root/ etc) What do you think? - alec ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] RFE: creating multiple clones in one zfs(1) call and one txg
I would like to apologise to those reading via the forums, because I used BNF anglebrackets and even though I sent a plaintext message, it lost my text as HTML... zfs multiclone tank/f...@1 tank/PATTERN BEGIN END [STRIDE] -a ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] RFE: creating multiple clones in one zfs(1) call and one txg
So much for an easy-to-use CLI :-O How about feeding in a file containing names instead (qv fmthard -s)? Not terribly script-friendly; I suffered that sort of thing with zonecfg and zoneadm (create a controlfile and squirt it into another command) and deemed it a horrible hack. They are still too broken for me to face using in their raw state except on special occasions... Same reason I never use fgrep, it's just not the true Unix way. If it *was* the true Unix way, you would never have written something like: rm `cat files-to-delete.txt` ...or anything else in backticks in your life; nor would you ever have used xargs... :-) -a ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] RFE: creating multiple clones in one zfs(1) call and one txg
The reason for the pattern based filenames is because: a) that is probably what is wanted most of the time anyway b) it is easy to pass from userland to kernel - you pass the rules (after some userland sanity checking first) as is. Just to quote what I wrote back in 2006 (ahem) which would *also* fit the single ioctl() model: 2) zfs clone -C N snapshot filesystemXX - ie: following the inspiration of mktemp(3c), when invokes with -C (count) then N clones are created, numbered 0 thru (N-1), where the counter number for each one will replace XX in the filesystem pattern, zero-padded left if needed. For example: zfs clone -C 1000 pool/ xx...@x pool/fishXX ...yields clones named pool/fish00 through pool/fish000999. ...but I like the control of having a snprintf() pattern with START/ STOP/STEP, more. It brings out the BASIC programmer in me... - alec ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Recovering from ZFS-8000-CS: Destroy and re-create the pool from a backup source.
Hi All, Here's a synopsis of my morning: 23:43:17 suzi:~ $ fmdump TIME UUID SUNW-MSG-ID Sep 13 05:40:06.2394 5ad39b52-c2e7-6d53-b937-d43694ed2568 ZFS-8000-GH Sep 13 05:50:05.5643 14d37f29-d93d-ef9d-d699-d617dc65b44c ZFS-8000-GH Sep 13 05:50:05.8213 143ff762-7323-45b1-951d-9184ff52a0c5 ZFS-8000-GH Sep 13 05:50:06.1178 70b4b99e-d130-4458-8744-a7ca57a623d3 ZFS-8000-GH Sep 13 06:02:05.4802 58cc717b-33b9-cac7-89eb-e940d675df8d ZFS-8000-GH Sep 13 08:13:03.7450 21416ee5-ded3-4b3f-e445-c55926d9af27 ZFS-8000-GH Sep 13 09:00:54.8115 3add3265-d1c3-4996-f05a-c006e42f150f ZFS-8000-CS Sep 13 12:10:32.6606 75a93512-986c-4c76-b8dc-f76ef75c00a2 ZFS-8000-GH Sep 13 12:12:11.4769 f6080484-c9d4-eba8-cc3d-ad6e8648f39d ZFS-8000-GH Sep 13 17:36:43.8303 53ba97f3-8599-4524-f37a-be1eb5fe5cf4 ZFS-8000-GH Sep 13 17:38:29.7262 f19e6b27-f79e-6b37-94a6-87f88a88858c ZFS-8000-GH Sep 13 17:38:30.3261 4fc7fc52-eaa3-4d5f-cecf-d1033aff67f1 ZFS-8000-GH Sep 13 18:14:47.4240 bcd60917-023f-e2a7-a9db-d7b50fae7784 ZFS-8000-GH Note especially http://www.sun.com/msg/ZFS-8000-CS which says: Destroy and re-create the pool from a backup source Take the documented action to resolve the problem. What happened next is documented at http://www.crypticide.com/ dropsafe/article/2162 - alec ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] An Academic Sysadmin's Lament for ZFS ?
Mounts under /net are derived from the filesystems actually shared from the servers; the automount daemon uses the MOUNT protocol to determine this. If you're looking at a path not already seen, the information will be fresh, but that's where the good news ends. I know that, yes, but why can't we put such an abstraction elsewhere in the name space? One thing I have always disliked about /net mounts is that they're too magical; it should be possible to replicate them in some form in other mount maps. In short, you're proposing a solution to the zillions-of-nfs-exports issue, which instead of using wait for v4 to implement a server-side export consolidation thingy, would instead be a better, smarter / net-alike on v2/v3, but give it a sensible name and better namespace semantics? I could go for that... -a ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Is there _any_ suitable motherboard?
Does anyone on this list have experience with a recent board with 6 or more SATA ports that they know is supported? Well so far I have only populated 5 of the ports I have available, but my writeup with my 9-port SATS ASUS mobo is at: http://www.crypticide.com/dropsafe/article/2091 ...and I hope to run a few more tests this weekend, time permitting. But you know this, since you've already commented there. :-) - alec -- Alec Muffett http://www.google.com/search?q=alec+muffett ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Karma Re: [zfs-discuss] Re: Best use of 4 drives?
As I understand matters, from my notes to design the perfect home NAS server :-) 1) you want to give ZFS entire spindles if at all possible; that will mean it can enable and utilise the drive's hardware write cache properly, leading to a performance boost. You want to do this if you can. Alas it knocks out the split all disks into 7 493Gb partitions design concept. 2) I've considered pivot-root solutions based around a USB stick or drive; cute, but I want a single tower box and no dongles 3) This leads me to the following design points: - enormous tower case with 10+ bays - HE/high-efficency mobo with 8+ SATA capability - crank down the CPU, big fans, etc... quiet - 1x [small/cheap]Gb Drive @ 1+rpm for root / swap / alternate boot environments - 4x 750Gb SATA @ 7200rpm for full-spindle RAID-Z - populate the spare SATA ports when 1Tb disks hit the price point; make a separate RAIDZ and drop *that* into the existing pool. This - curiously - echoes the Unixes of my youth (and earlier!) where root was a small fast disk for swapping and access to key utilities which were used frequently (hence /bin and /lib) - whereas usr was a bigger, slower, cheaper disk, where the less frequently-used stuff was stored (/usr/bin, home directories, etc)... Funny how the karmic wheel turns; I was suffering from the above architecture until the early 1990s - arguably we still suffer from it today, watch Perl building some time - and now I am redesigning the same thing but at least now the whole OS squeezes into the small disk pretty easily. :-) As an aside there is nothing wrong with using ZFS - eg: a zvol - as a swap device; but just as you say, if we use real disks for root then they will be so big that there's probably no point in pushing swap off to ZFS. -a -- Alec Muffett http://www.google.com/search?q=alec-muffett ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Re: A quick ZFS question: RAID-Z Disk Replacement + Growth ?
Hi All, My mate Chris posed me the following; rather than flail about with engineering friends trying to get a definitive-de-jour answer, I thought instead to introduce him to the relevant opensolaris forum in the hope of broadening the latter's appeal. So: RAID-Z hot swap to larger disks and thereby upgrade? Doable, not, or unwise? I suspect the proper thing to do would be to build the six new large disks into a new RAID-Z vdev, add it as a mirror of the older, smaller-disk RAID-Z vdev, rezilver to zynchronize them, and then break the mirror. But I'd like a second opinion? - alec Hiya Alec, Mark Baily here at VPAC asked me a ZFS question which I can't answer (except by guessing) and I said I'd ask you in case you had a quick answer as you've played with it a bit more. Mark wrote: Can the pool's size be increased by replacing [6 old 320Gb disks] with a new set of 6 HD's say 640GB each, changing one disk at a time and waiting for data to be resilvered before replacing the next disk, then when all 6 disks are resilvered the size will automatically be doubled? My guess is that the answer would be yes, that should work, but that is just a guess! Is this something you've played with ? cheers! Chris -- Alec Muffett http://www.google.com/search?q=alec+muffett ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] New Feature Idea: ZFS Views ?
yeah, but doing a full tree walk under the covers in that first readdir will Suck. I'd rather the view creation be a bit slowerer in order that the first access to the view would be responsive. Pick your poison. O(N) creation or O(N) initial traversal. the low-hanging fruit is a view as a window onto a static snapshot. why not just create the view instantly and then filter all operations which scan a directory from that point onwards, try growing it incrementally upon demand? -a ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] New Feature Idea: ZFS Views ?
How is it different from creating a snapshot and running find on it? More convenient? Faster? Something else? Ever encountered an operating system that allows you to mount and manipulate a ZIP file as a filesystem? Why did anyone bother to do that, when they could just zip -l and pipe the output through more? - alec ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
