[zfs-discuss] Building an On-Site and Off-Size ZFS server, replication question
Good morning. I am in the process of planning a system which will have 2 ZFS servers, one on site, one off site. The on site server will be used by workstations and servers in house, and most of that will stay in house. There will, however, be data i want backed up somewhere else, which is where the offsite server comes in... This server will be sitting in a Data Center and will have some storage available to it (the whole server currently has 2 3Tb drives, though they are not dedicated to the ZFS box, they are on VMware ESXi). There is then some storage (currently 100Gb, but more can be requested) of SFTP enabled backup which i plan to use for some snapshots, but more on that later. Anyway, i want to confirm my plan and make sure i am not missing anything here... * build server in house with storage, pools, etc... * have a server in data center with enough storage for its reason, plus the extra for offsite backup * have one pool set as my offsite pool... anything in here should be backed up off site also... * possibly have another set as very offsite which will also be pushed to the SFTP server, but not sure... * give these pools out via SMB/NFS/iSCSI * every 6 or so hours take a snapshot of the 2 offsite pools. * do a ZFS send to the data center box * nightly, on the very offsite pool, do a ZFS send to the SFTP server * if anything goes wrong (my server dies, DC server dies, etc), Panic, download, pray... the usual... :) Anyway, I want to make sure i am doing this correctly... Is there anything on that list that sounds stupid or am i doing anything wrong? am i missing anything? Also, as a follow up question, but slightly unrelated, when it comes to the ZFS Send, i could use SSH to do the send, directly to the machine... Or i could upload the compressed, and possibly encrypted dump to the server... Which, for resume-ability and speed, would be suggested? And if i where to go with an upload option, any suggestions on what i should use? Thanks. -- Tiernan O'Toole blog.lotas-smartman.net www.geekphotographer.com www.tiernanotoole.ie ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Changing rpool device paths/drivers
2012-10-05 4:57, Jerry Kemp wrote: Its been awhile, but it seems like in the past, you would power the system down, boot from removable media, import your pool then destroy or archive the /etc/zfs/zpool.cache, and possibly your /etc/path_to_inst file, power down again and re-arrange your hardware, then come up one final time with a reconfigure boot. Or something like that. Well, as I wrote in the OP, now the procedure is simpler; in your words: * power the system down and re-arrange your hardware (BIOS settings in case of SATA/Legacy=IDE switch) * boot from removable media, * import your rpool * export rpool * reboot from rpool Your procedure CAN be useful i.e. if a secondary userdata pool fails to import and causes kernel panics and such, or relies on external hardware (NAS/SAN) which is no longer available; then by deleting the rpool.cache you can avoid its automatic import upon OS boot. This does not seem to be employed by rpool import routine. This has a few bad steps I want to avoid: 1) Use of extra media. I'd go for at most a self-sufficient failsafe boot image like that we had in SXCE/Sol10 (basically, it's just a bigger boot_archive that you can log into); preferably the OS should not need even that and switch to new rpool connection technology on the fly during boot. 2) Reliance on some remapped PCI path numbers (i.e. it is often not vendorid:devid, but a pci@number kind of address), which might be changeable between boots if your enumeration is not cut in stone for some reason. For example, I do worry whether the LiveUSB boots can make the HDD seem to be at a different path than the plain HDD boots - due to insertion/removal of a whole storage device tree and change of BIOS boot order. (This laptop has no CD/DVD, and I don't think buying and adding/removing a USB CD/DVD would be a substantial change to adding/removing a USB HDD as I do now.) Whatever device paths the live-media bootup sees, it writes into the rpool headers upon successful import/export, and those strings (only?) are probed by boot from rpool. That is, the newly booted kernel does see enough of the pool to find these headers, then follows them and perhaps finds no storage hardware at that address. Well then, search/import the pool from the device WHERE you found those headers? Duh? ///Jim ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Building an On-Site and Off-Size ZFS server, replication question
2012-10-05 11:17, Tiernan OToole wrote: Also, as a follow up question, but slightly unrelated, when it comes to the ZFS Send, i could use SSH to do the send, directly to the machine... Or i could upload the compressed, and possibly encrypted dump to the server... Which, for resume-ability and speed, would be suggested? And if i where to go with an upload option, any suggestions on what i should use? As for this, the answer depends on network bandwidth, reliability, and snapshot file size - ultimately, on the probability and retry cost of an error during transmission. Many posters on the list strongly object to using files as storage for snapshot streams, because in reliability this is (may be) worse than a single-disk pool and bitrot on it - a single-bit error in a snapshot file can render it and all newer snapshots invalid and un-importable. Still, given enough scratch space on the sending and receiving sides and a bad (slow, glitchy) network in-between, I did go with compressed files of zfs-send streams (perhaps making recursion myself and using smaller files of one snapshot each - YMMV). For compression on multiCPU senders I can strongly suggest pigz --fast $filename (I did have problems in pigz-1.7.1 compressing several files with one command, maybe that's fixed now). If you're tight on space/transfer size more than on CPU, you can try other parallel algos - pbzip2, p7zip, etc. Likewise, you can also pass the file into an encryptor of your choice. Then I can rsync these files to the receiving server, using rsync -c and/or md5sum, sha256sum, sha1sum or whatever tool(s) of your liking to validate that the files received match those sent - better safe than sorry. I'm usually using rsync -cavPHK for any work, which gives you retryable transfers in case network goes down, status bar, directory recursion and hardlink support among other things. NFS is also retryable if so configured (even if the receiver gets rebooted in the process), and if you, for example, already have VPN between two sites, you might find it faster than rsync which involves extra encryption - maybe redundant in VPN case. When the scratch area on the receiver has got and validated the compressed snapshot stream, I can gzcat it and pipe into zfs recv. This ultimately validates that the zfs-send stream arrived intact and is fully receivable, and only then I can delete the temporary files involved - or retry the send from different steps (it is possible that the initial file was corrupted in RAM, etc.) Note that such approach via files essentially disables zfs-send deduplication which may be available in protocol between two active zfs commands, but AFAIK this does not preclude you from receiving data into deduped datasets - local dedup happens upon block writes anyway, like compression, encryption and stuff like that. HTH, //Jim Klimov ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Building an On-Site and Off-Size ZFS server, replication question
Thanks for that Jim! Sounds like a plan there... One question about the storing ZFS dumps in a file... So, the idea of storing the data in a SFTP server which has an unknown underlying file system... Is that defiantly off limits, or can it be done? and should i be doing a full dump or just an incremental one? maybe incremental daily, and then a full dump weekly? Thanks! --Tiernan On Fri, Oct 5, 2012 at 9:36 AM, Jim Klimov jimkli...@cos.ru wrote: 2012-10-05 11:17, Tiernan OToole wrote: Also, as a follow up question, but slightly unrelated, when it comes to the ZFS Send, i could use SSH to do the send, directly to the machine... Or i could upload the compressed, and possibly encrypted dump to the server... Which, for resume-ability and speed, would be suggested? And if i where to go with an upload option, any suggestions on what i should use? As for this, the answer depends on network bandwidth, reliability, and snapshot file size - ultimately, on the probability and retry cost of an error during transmission. Many posters on the list strongly object to using files as storage for snapshot streams, because in reliability this is (may be) worse than a single-disk pool and bitrot on it - a single-bit error in a snapshot file can render it and all newer snapshots invalid and un-importable. Still, given enough scratch space on the sending and receiving sides and a bad (slow, glitchy) network in-between, I did go with compressed files of zfs-send streams (perhaps making recursion myself and using smaller files of one snapshot each - YMMV). For compression on multiCPU senders I can strongly suggest pigz --fast $filename (I did have problems in pigz-1.7.1 compressing several files with one command, maybe that's fixed now). If you're tight on space/transfer size more than on CPU, you can try other parallel algos - pbzip2, p7zip, etc. Likewise, you can also pass the file into an encryptor of your choice. Then I can rsync these files to the receiving server, using rsync -c and/or md5sum, sha256sum, sha1sum or whatever tool(s) of your liking to validate that the files received match those sent - better safe than sorry. I'm usually using rsync -cavPHK for any work, which gives you retryable transfers in case network goes down, status bar, directory recursion and hardlink support among other things. NFS is also retryable if so configured (even if the receiver gets rebooted in the process), and if you, for example, already have VPN between two sites, you might find it faster than rsync which involves extra encryption - maybe redundant in VPN case. When the scratch area on the receiver has got and validated the compressed snapshot stream, I can gzcat it and pipe into zfs recv. This ultimately validates that the zfs-send stream arrived intact and is fully receivable, and only then I can delete the temporary files involved - or retry the send from different steps (it is possible that the initial file was corrupted in RAM, etc.) Note that such approach via files essentially disables zfs-send deduplication which may be available in protocol between two active zfs commands, but AFAIK this does not preclude you from receiving data into deduped datasets - local dedup happens upon block writes anyway, like compression, encryption and stuff like that. HTH, //Jim Klimov __**_ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/**mailman/listinfo/zfs-discusshttp://mail.opensolaris.org/mailman/listinfo/zfs-discuss -- Tiernan O'Toole blog.lotas-smartman.net www.geekphotographer.com www.tiernanotoole.ie ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Building an On-Site and Off-Size ZFS server, replication question
2012-10-05 13:13, Tiernan OToole wrote: Thanks for that Jim! Sounds like a plan there... One question about the storing ZFS dumps in a file... So, the idea of storing the data in a SFTP server which has an unknown underlying file system... Is that defiantly off limits, or can it be done? Mileages do vary. Maybe you should pack the stream files into archives with error-correction codes (or at least verification CRCs) like ZIP, RAR, likely p7zip, maybe others; and also keep checksum files. At least this can help detect or even fix small nasty surprises. The general concern is that zfs send streams have no built-in redundancy, I'm not sure about error-checking - likely it is there. And it is widely assumed that this being a stream, a small error can redirect the flow widely differently from expectations and cause the whole dataset state to be invalid (likely this snapshot receiving will be aborted, and then you can't receive any newer ones over it). That said, some people do keep the streams on tape; the NDMP tools and protocol from Sun IIRC do the same for backups. So it's not off-limits, but precautions may be due (keep 2+ copies, do CRC/ECC and so on). and should i be doing a full dump or just an incremental one? maybe incremental daily, and then a full dump weekly? A full dump of a large filesystem can be unbearably large for storage and transfers. Still, the idea of storing occasional full snapshots and a full history of incrementals (so that you can try to recover starting from any of the full snapshots you have) sounds sane. This way you have a sort of second copy by virtue of a full snapshot incorporating some state of the dataset and if the most recent one is broken - you can try to recover with the one(s) before it and applying more incremental snapshots. Likewise, errors in very old snapshots become irrelevant when a newer full snapshot is intact. But sometimes two or more things can break - or be detected to break - at once ;) In particular, regular drills should be done (and provisioned for) to test that you can in fact recover from your backups, and that they do contain all the data you need. Older configs can become obsolete as live systems evolve... HTH, //Jim ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Building an On-Site and Off-Size ZFS server, replication question
Thanks again Jim. Very handy info. This is now my weekend project, so hopefully things go well! --Tiernan On Fri, Oct 5, 2012 at 10:40 AM, Jim Klimov jimkli...@cos.ru wrote: 2012-10-05 13:13, Tiernan OToole wrote: Thanks for that Jim! Sounds like a plan there... One question about the storing ZFS dumps in a file... So, the idea of storing the data in a SFTP server which has an unknown underlying file system... Is that defiantly off limits, or can it be done? Mileages do vary. Maybe you should pack the stream files into archives with error-correction codes (or at least verification CRCs) like ZIP, RAR, likely p7zip, maybe others; and also keep checksum files. At least this can help detect or even fix small nasty surprises. The general concern is that zfs send streams have no built-in redundancy, I'm not sure about error-checking - likely it is there. And it is widely assumed that this being a stream, a small error can redirect the flow widely differently from expectations and cause the whole dataset state to be invalid (likely this snapshot receiving will be aborted, and then you can't receive any newer ones over it). That said, some people do keep the streams on tape; the NDMP tools and protocol from Sun IIRC do the same for backups. So it's not off-limits, but precautions may be due (keep 2+ copies, do CRC/ECC and so on). and should i be doing a full dump or just an incremental one? maybe incremental daily, and then a full dump weekly? A full dump of a large filesystem can be unbearably large for storage and transfers. Still, the idea of storing occasional full snapshots and a full history of incrementals (so that you can try to recover starting from any of the full snapshots you have) sounds sane. This way you have a sort of second copy by virtue of a full snapshot incorporating some state of the dataset and if the most recent one is broken - you can try to recover with the one(s) before it and applying more incremental snapshots. Likewise, errors in very old snapshots become irrelevant when a newer full snapshot is intact. But sometimes two or more things can break - or be detected to break - at once ;) In particular, regular drills should be done (and provisioned for) to test that you can in fact recover from your backups, and that they do contain all the data you need. Older configs can become obsolete as live systems evolve... HTH, //Jim __**_ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/**mailman/listinfo/zfs-discusshttp://mail.opensolaris.org/mailman/listinfo/zfs-discuss -- Tiernan O'Toole blog.lotas-smartman.net www.geekphotographer.com www.tiernanotoole.ie ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Building an On-Site and Off-Size ZFS server, replication question
On 10/05/12 21:36, Jim Klimov wrote: 2012-10-05 11:17, Tiernan OToole wrote: Also, as a follow up question, but slightly unrelated, when it comes to the ZFS Send, i could use SSH to do the send, directly to the machine... Or i could upload the compressed, and possibly encrypted dump to the server... Which, for resume-ability and speed, would be suggested? And if i where to go with an upload option, any suggestions on what i should use? As for this, the answer depends on network bandwidth, reliability, and snapshot file size - ultimately, on the probability and retry cost of an error during transmission. Many posters on the list strongly object to using files as storage for snapshot streams, because in reliability this is (may be) worse than a single-disk pool and bitrot on it - a single-bit error in a snapshot file can render it and all newer snapshots invalid and un-importable. Still, given enough scratch space on the sending and receiving sides and a bad (slow, glitchy) network in-between, I did go with compressed files of zfs-send streams (perhaps making recursion myself and using smaller files of one snapshot each - YMMV). For compression on multiCPU senders I can strongly suggest pigz --fast $filename (I did have problems in pigz-1.7.1 compressing several files with one command, maybe that's fixed now). If you're tight on space/transfer size more than on CPU, you can try other parallel algos - pbzip2, p7zip, etc. Likewise, you can also pass the file into an encryptor of your choice. I do have to suffer a slow, glitchy WAN to a remote server and rather than send stream files, I broke the data on the remote server into a more fine grained set of filesystems than I would do normally. In this case, I made the directories under what would have been the leaf filesystems filesystems themselves. By spreading the data over more filesystems, the individual incremental sends are smaller, so there is less data to resend if the link burps during a transfer. -- Ian. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Building an On-Site and Off-Size ZFS server, replication question
Thanks Ian. That sounds like an option also. The plan was to break up the file systems anyway, since some i will want to be replicated remotely, and others not as much. --Tiernan On Fri, Oct 5, 2012 at 11:17 AM, Ian Collins i...@ianshome.com wrote: On 10/05/12 21:36, Jim Klimov wrote: 2012-10-05 11:17, Tiernan OToole wrote: Also, as a follow up question, but slightly unrelated, when it comes to the ZFS Send, i could use SSH to do the send, directly to the machine... Or i could upload the compressed, and possibly encrypted dump to the server... Which, for resume-ability and speed, would be suggested? And if i where to go with an upload option, any suggestions on what i should use? As for this, the answer depends on network bandwidth, reliability, and snapshot file size - ultimately, on the probability and retry cost of an error during transmission. Many posters on the list strongly object to using files as storage for snapshot streams, because in reliability this is (may be) worse than a single-disk pool and bitrot on it - a single-bit error in a snapshot file can render it and all newer snapshots invalid and un-importable. Still, given enough scratch space on the sending and receiving sides and a bad (slow, glitchy) network in-between, I did go with compressed files of zfs-send streams (perhaps making recursion myself and using smaller files of one snapshot each - YMMV). For compression on multiCPU senders I can strongly suggest pigz --fast $filename (I did have problems in pigz-1.7.1 compressing several files with one command, maybe that's fixed now). If you're tight on space/transfer size more than on CPU, you can try other parallel algos - pbzip2, p7zip, etc. Likewise, you can also pass the file into an encryptor of your choice. I do have to suffer a slow, glitchy WAN to a remote server and rather than send stream files, I broke the data on the remote server into a more fine grained set of filesystems than I would do normally. In this case, I made the directories under what would have been the leaf filesystems filesystems themselves. By spreading the data over more filesystems, the individual incremental sends are smaller, so there is less data to resend if the link burps during a transfer. -- Ian. __**_ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/**mailman/listinfo/zfs-discusshttp://mail.opensolaris.org/mailman/listinfo/zfs-discuss -- Tiernan O'Toole blog.lotas-smartman.net www.geekphotographer.com www.tiernanotoole.ie ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] vm server storage mirror
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Jim Klimov Well, it seems just like a peculiar effect of required vs. optional dependencies. The loop is in the default installation. Details: # svcprop filesystem/usr | grep scheduler svc:/system/filesystem/usr:default/:properties/scheduler_usr/entities fmri svc:/system/scheduler svc:/system/filesystem/usr:default/:properties/scheduler_usr/external boolean true svc:/system/filesystem/usr:default/:properties/scheduler_usr/grouping astring optional_all svc:/system/filesystem/usr:default/:properties/scheduler_usr/restart_on astring none svc:/system/filesystem/usr:default/:properties/scheduler_usr/type astring service # svcprop scheduler | grep minimal svc:/application/cups/scheduler:default/:properties/filesystem_minimal/ent ities fmri svc:/system/filesystem/minimal svc:/application/cups/scheduler:default/:properties/filesystem_minimal/gro uping astring require_all svc:/application/cups/scheduler:default/:properties/filesystem_minimal/res tart_on astring none svc:/application/cups/scheduler:default/:properties/filesystem_minimal/typ e astring service # svcprop filesystem/minimal | grep usr usr/entities fmri svc:/system/filesystem/usr usr/grouping astring require_all usr/restart_on astring none usr/type astring service I must be missing something - I don't see anything above that indicates any required vs optional dependencies. I'm not quite sure what I'm supposed to be seeing in the svcprop outputs you pasted... The answer to this question, will in all likelihood, shed some light on my situation, trying to understand why my iscsi mounted zpool import/export service is failing to go down or come up in the order I expected, when it's dependent on the iscsi initiator. Likewise - see what dependency type you introduced, and verify that you've svcadm refreshed the service after config changes. Thank you for the suggestion - I like the direction this is heading, but I don't know how to do that yet. (This email is the first I ever heard of it.) Rest assured, I'll be googling and reading more man pages in the meantime. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Making ZIL faster
From: Neil Perrin [mailto:neil.per...@oracle.com] In general - yes, but it really depends. Multiple synchronous writes of any size across multiple file systems will fan out across the log devices. That is because there is a separate independent log chain for each file system. Also large synchronous writes (eg 1MB) within a specific file system will be spread out. The ZIL code will try to allocate a block to hold all the records it needs to commit up to the largest block size - which currently for you should be 128KB. Anything larger will allocate a new block - on a different device if there are multiple devices. However, lots of small synchronous writes to the same file system might not use more than one 128K block and benefit from multiple slog devices. That is an awesome explanation. Thank you. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Building an On-Site and Off-Size ZFS server, replication question
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Tiernan OToole I am in the process of planning a system which will have 2 ZFS servers, one on site, one off site. The on site server will be used by workstations and servers in house, and most of that will stay in house. There will, however, be data i want backed up somewhere else, which is where the offsite server comes in... This server will be sitting in a Data Center and will have some storage available to it (the whole server currently has 2 3Tb drives, though they are not dedicated to the ZFS box, they are on VMware ESXi). There is then some storage (currently 100Gb, but more can be requested) of SFTP enabled backup which i plan to use for some snapshots, but more on that later. Anyway, i want to confirm my plan and make sure i am not missing anything here... * build server in house with storage, pools, etc... * have a server in data center with enough storage for its reason, plus the extra for offsite backup * have one pool set as my offsite pool... anything in here should be backed up off site also... * possibly have another set as very offsite which will also be pushed to the SFTP server, but not sure... * give these pools out via SMB/NFS/iSCSI * every 6 or so hours take a snapshot of the 2 offsite pools. * do a ZFS send to the data center box * nightly, on the very offsite pool, do a ZFS send to the SFTP server * if anything goes wrong (my server dies, DC server dies, etc), Panic, download, pray... the usual... :) Anyway, I want to make sure i am doing this correctly... Is there anything on that list that sounds stupid or am i doing anything wrong? am i missing anything? Also, as a follow up question, but slightly unrelated, when it comes to the ZFS Send, i could use SSH to do the send, directly to the machine... Or i could upload the compressed, and possibly encrypted dump to the server... Which, for resume-ability and speed, would be suggested? And if i where to go with an upload option, any suggestions on what i should use? It is recommended, whenever possible, you should pipe the zfs send directly into a zfs receive on the receiving system. For two solid reasons: If a single bit is corrupted, the whole stream checksum is wrong and therefore the whole stream is rejected. So if this occurs, you want to detect it (in the form of one incremental failed) and then correct it (in the form of the next incremental succeeding). Whereas, if you store your streams on storage, it will go undetected, and everything after that point will be broken. If you need to do a restore, from a stream stored on storage, then your only choice is to restore the whole stream. You cannot look inside and just get one file. But if you had been doing send | receive, then you obviously can look inside the receiving filesystem and extract some individual specifics. If the recipient system doesn't support zfs receive, you might consider exporting an iscsi device, and allowing the sender system deal with it directly. Or share a filesystem (such as NFS) with the sender system, and let the sender create a recipient filesystem inside a file container, so the sender can deal with it directly. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Building an On-Site and Off-Size ZFS server, replication question
On Fri, Oct 5, 2012 at 3:17 AM, Ian Collins i...@ianshome.com wrote: I do have to suffer a slow, glitchy WAN to a remote server and rather than send stream files, I broke the data *on the remote server* into a more fine grained set of filesystems than I would do normally. In this case, I made the directories under what would have been the leaf filesystems filesystems themselves. Meaning you also broke the data on the LOCAL server into the same set of more granular filesystems? Or is it now possible to zfs send a subdirectory of a filesystem? ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] vm server storage mirror
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Edward Ned Harvey I must be missing something - I don't see anything above that indicates any required vs optional dependencies. Ok, I see that now. (Thanks to the SMF FAQ). A dependency may have grouping optional_all, require_any, or require_all. Mine is require_all, and I figured out the problem. I had my automatic zpool import/export script dependent on the initiator ... But it wasn't the initiator going down first. It was the target going down first. So the solution is like this: sudo svccfg -s svc:/network/iscsi/initiator:default svc:/network/iscsi/initiator:default addpg iscsi-target dependency svc:/network/iscsi/initiator:default setprop iscsi-target/grouping = astring: require_all svc:/network/iscsi/initiator:default setprop iscsi-target/restart_on = astring: none svc:/network/iscsi/initiator:default setprop iscsi-target/type = astring: service svc:/network/iscsi/initiator:default setprop iscsi-target/entities = fmri: svc:/network/iscsi/target:default svc:/network/iscsi/initiator:default exit sudo svcadm refresh svc:/network/iscsi/initiator:default And additionally, create the SMF service dependent on initiator, which will import/export the iscsi pools automatically. http://nedharvey.com/blog/?p=105 ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Building an On-Site and Off-Size ZFS server, replication question
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Frank Cusack On Fri, Oct 5, 2012 at 3:17 AM, Ian Collins i...@ianshome.com wrote: I do have to suffer a slow, glitchy WAN to a remote server and rather than send stream files, I broke the data on the remote server into a more fine grained set of filesystems than I would do normally. In this case, I made the directories under what would have been the leaf filesystems filesystems themselves. Meaning you also broke the data on the LOCAL server into the same set of more granular filesystems? Or is it now possible to zfs send a subdirectory of a filesystem? zfs create instead of mkdir As Ian said - he didn't zfs send subdirs, he made filesystems where he otherwise would have used subdirs. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Building an On-Site and Off-Size ZFS server, replication question
On 10/06/12 07:57, Edward Ned Harvey (opensolarisisdeadlongliveopensolaris) wrote: From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Frank Cusack On Fri, Oct 5, 2012 at 3:17 AM, Ian Collinsi...@ianshome.com wrote: I do have to suffer a slow, glitchy WAN to a remote server and rather than send stream files, I broke the data on the remote server into a more fine grained set of filesystems than I would do normally. In this case, I made the directories under what would have been the leaf filesystems filesystems themselves. Meaning you also broke the data on the LOCAL server into the same set of more granular filesystems? Or is it now possible to zfs send a subdirectory of a filesystem? zfs create instead of mkdir As Ian said - he didn't zfs send subdirs, he made filesystems where he otherwise would have used subdirs. That right. I do have a lot of what would appear to be unnecessary filesystems, but after loosing the WAN 3 days into a large transfer, a change of tactic was required! -- Ian. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] How many disk in one pool
Hi all, I'm actually running ZFS under FreeBSD. I've a question about how many disks I «can» have in one pool. At this moment I'm running with one server (FreeBSD 9.0) with 4 MD1200 (Dell) meaning 48 disks. I've configure with 4 raidz2 in the pool (one on each MD1200) On what I understand I can add more more MD1200. But if I loose one MD1200 for any reason I lost the entire pool. In your experience what's the «limit» ? 100 disk ? How FreeBSD manage 100 disk ? /dev/da100 ? Regards. JAS -- Albert SHIH DIO bâtiment 15 Observatoire de Paris 5 Place Jules Janssen 92195 Meudon Cedex Téléphone : 01 45 07 76 26/06 86 69 95 71 xmpp: j...@obspm.fr Heure local/Local time: ven 5 oct 2012 22:52:22 CEST ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] How many disk in one pool
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Albert Shih I'm actually running ZFS under FreeBSD. I've a question about how many disks I can have in one pool. At this moment I'm running with one server (FreeBSD 9.0) with 4 MD1200 (Dell) meaning 48 disks. I've configure with 4 raidz2 in the pool (one on each MD1200) On what I understand I can add more more MD1200. But if I loose one MD1200 for any reason I lost the entire pool. In your experience what's the limit ? 100 disk ? How FreeBSD manage 100 disk ? /dev/da100 ? Correct about if you lose one storage tray you lose the pool. Ideally you would span your redundancy across trays as well as across disks - but in your situation, 12 disks in raidz2 - and 4 trays - it's just not realistic for you. You would have to significantly increase cost (not to mention rebuild pool) in order to keep the same available disk space and gain the redundancy. Go ahead and add more trays. I've never heard of any limit of number of disks you can have in ZFS. I'm sure there is a limit, but whatever it is, you're nowhere near it. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] How many disk in one pool
On Oct 5, 2012, at 1:57 PM, Albert Shih albert.s...@obspm.fr wrote: Hi all, I'm actually running ZFS under FreeBSD. I've a question about how many disks I «can» have in one pool. At this moment I'm running with one server (FreeBSD 9.0) with 4 MD1200 (Dell) meaning 48 disks. I've configure with 4 raidz2 in the pool (one on each MD1200) On what I understand I can add more more MD1200. But if I loose one MD1200 for any reason I lost the entire pool. In your experience what's the «limit» ? 100 disk ? I can't speak for current FreeBSD, but I've seen more than 400 disks (HDDs) in a single pool. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Building an On-Site and Off-Size ZFS server, replication question
On Fri, Oct 5, 2012 at 1:28 PM, Ian Collins i...@ianshome.com wrote: I do have a lot of what would appear to be unnecessary filesystems, but after loosing the WAN 3 days into a large transfer, a change of tactic was required! I've recently (last year or so) gone the other way, and have made an effort to combine filesystems. I'm now thinking of remote replication so maybe I'll break them up again. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
