Re: hast, zfs, unable to flush disk cache
On Fri, Mar 01, 2013 at 10:25:50PM +0200, Mikolaj Golub wrote: On Fri, Mar 01, 2013 at 11:39:23AM -0600, Chad M Stewart wrote: I've setup a 2 node cluster using HAST. I'd previously had this running 9.0 and then rebuilt the nodes using 9.1. Under 9.0 I used vfs.zfs.vdev.bio_flush_disable=1 but that setting does not appear to work in 9.1. The other difference, previous build had root disk on UFS, while this build has only ZFS based file systems. FreeBSD node1.san 9.1-RELEASE FreeBSD 9.1-RELEASE #0 r243826: Tue Dec 4 06:55:39 UTC 2012 r...@obrian.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC i386 Mar 1 17:07:25 node1 hastd[1446]: [disk5] (primary) Unable to flush disk cache on activemap update: Operation not supported by device. Mar 1 17:07:28 node1 hastd[1440]: [disk3] (primary) Unable to flush disk cache on activemap update: Operation not supported by device. Mar 1 17:07:28 node1 hastd[1437]: [disk2] (primary) Unable to flush disk cache on activemap update: Operation not supported by device. Mar 1 17:07:28 node1 hastd[1434]: [disk1] (primary) Unable to flush disk cache on activemap update: Operation not supported by device. Mar 1 17:07:28 node1 hastd[1446]: [disk5] (primary) Unable to flush disk cache on activemap update: Operation not supported by device. Mar 1 17:07:28 node1 hastd[1443]: [disk4] (primary) Unable to flush disk cache on activemap update: Operation not supported by device. I tried setting zfs set zfs:zfs_nocacheflush=1 pool but that diddn't appear to work either. I get a lot of lines in my log file because of this. I have also tried zfs set sync=disabled pool but HAST still outputs those lines in the log file. These flushes were generated by HAST itself when it tried to flush activemap updates, that is why disabling BIO_FLUSH for ZFS dis not help much. Setting metaflush off in hast.conf should help though. BTW, hastd tries to detect devices that do not support BIO_FLUSH, checking for the returned errno, and automatically disable flushes if the errno is EOPNOTSUPP (Operation not supported). Unfortunately, your device returned ENODEV (Operation not supported by device). What device do you have? Pawel, do you think it would be a good idea to automatically disable activemap flush for ENODEV case too? It would be better to find the driver that returns ENODEV and fix it, IMHO. -- Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://tupytaj.pl pgpUGDh098WMq.pgp Description: PGP signature
Re: HAST - detect failure and restore avoiding an outage?
On Sun, Feb 24, 2013 at 12:05:06PM +0200, Mikolaj Golub wrote: On Sat, Feb 23, 2013 at 09:51:03PM +0100, Pawel Jakub Dawidek wrote: I'm fine with the patchi except for missing breaks in switch added to hastd/primary.c. Oops. Fixed. Thanks! I'm also wondering... You count all those errors separately just to print them as one number. If we do that already let's print them separately, eg. local i/o errors: read(0), write(3), delete(5), flush(9) The idea was that hastd provided all available counters, and hastctl showed only aggregated counter just to save a screen space, but if one wanted to write its own utility to monitor hastd, which would talk directly to hastd via socket, she would be able to see all counters separately. But your idea with writing errors in one string looks better, as it allows to save a screen space and provide more detailed info. I would prefer a little different output though: role: secondary provname: test localpath: /dev/md102 extentsize: 2097152 (2.0MB) keepdirty: 0 remoteaddr: kopusha:7771 replication: memsync status: complete dirty: 0 (0B) statistics: reads: 13 writes: 521 deletes: 0 flushes: 0 activemap updates: 0 local i/o errors: read: 13, write: 425, delete: 0, flush: 0 but don't have a strong opinion and will be ok with yours if you don't like my version. My only comment would be to keep that in one line so it is easier to grep. And merging those two lines won't exceed 80 chars. BTW. Why not to count activemap update errors as write and flush errors? I need (internally) separate counters for activemap errors because they are updated by the different thread and I wouldn't want to introduce locking for error counter update operations. As hastctl was supposed to show an aggregated counter I didn't bother much how to make activemap update errors to count as write and flush errors. I improved this too in the updated patch: http://people.freebsd.org/~trociny/hast.stat_error.2.patch The patch looks good. -- Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://tupytaj.pl pgpWIEUPJOQes.pgp Description: PGP signature
Re: HAST - detect failure and restore avoiding an outage?
On Fri, Feb 22, 2013 at 12:00:43AM +0200, Mikolaj Golub wrote: It looks currently logs are only way to detect errors from hastd side. Here is a patch that adds local i/o error statistics, accessable avia hastctl: http://people.freebsd.org/~trociny/hast.stat_error.1.patch hastctl output: role: secondary provname: test localpath: /dev/md102 extentsize: 2097152 (2.0MB) keepdirty: 0 remoteaddr: kopusha:7771 replication: memsync status: complete dirty: 0 (0B) statistics: reads: 0 writes: 366 deletes: 0 flushes: 0 activemap updates: 0 local i/o errors: 269 Pawel, what do you think about this patch? I'm fine with the patchi except for missing breaks in switch added to hastd/primary.c. I'm also wondering... You count all those errors separately just to print them as one number. If we do that already let's print them separately, eg. local i/o errors: read(0), write(3), delete(5), flush(9) BTW. Why not to count activemap update errors as write and flush errors? -- Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://tupytaj.pl pgpX_5TbRWObv.pgp Description: PGP signature
Re: ZFS promote failure
On Wed, May 11, 2011 at 11:50:51AM -0400, Dave Cundiff wrote: Hello, I'm seeing a strange problem trying to use zfs promote. [root@san2]# zfs snapshot san/sr@snap [root@san2]# zfs clone san/sr@snap san/sr5 [root@san2]# zfs promote san/sr5 cannot promote 'san/sr5': dataset is busy Being a freshly created dataset I'm not sure how it would be busy. Are there any caveats to using zfs promote on zvols? The snapshots are of ext3 formatted zvols. I don't really need to promote them but wanted to in case I needed to destroy the source for some reason. Which FreeBSD version is this? I just tried it on 9-CURRENT with ZFSv28 and it works just fine, even if I've file systems mounted on top of those ZVOLs. -- Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com pgpP8lHdv8hko.pgp Description: PGP signature
Re: FreeBSD 8.1-PRERELEASE: property 'jailed' not supported on FreeBSD: permission denied
On Fri, May 21, 2010 at 03:55:59PM +0400, Eugene Mitrofanov wrote: Hi The command zfs set jailed=on tank/s1 is failed with the message property 'jailed' not supported on FreeBSD: permission denied. Output of zfs get jailed tank/s1 shows me that the property jailed is still exists: NAME PROPERTY VALUESOURCE tank/s1 jailedoff default How can I change its value? It was accidentally marked as OpenSolaris-specific. Should be fixed as of r208684 in HEAD and I plan to merge it to stable/8 in few days. Thanks for the report! -- Pawel Jakub Dawidek http://www.wheelsystems.com p...@freebsd.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! pgpWpZWCOlJQ4.pgp Description: PGP signature
Re: gmirror gm0 destroyed on shutdown; GPT corrupt
On Sat, Jun 27, 2009 at 06:20:49PM -0700, Marcel Moolenaar wrote: Using the last sector is not only flawed because it creates a race condition, it's flawed in the assumption that you can always make a geom part of a mirror by storing meta-data on the geom without causing corruption. This whole idea of using the last sector was so that a fully partitioned disk with data could be turned into a mirrored disk. A neat idea, but hardly the basis for a generic mirroring implementation when it silently corrupts a disk. This wasn't the idea:) People started putting gmirror on top of partitioned disk, because it was easier/simpler/faster than creating mirror, partitioning and copying the data. I for one never put mirror on already partitioned disk. Although it is sometimes safe to use the last sector. Gjournal already looks for UFS and if UFS is in place, it figures out if the last sector is in use - it isn't if partition size is not multiple of UFS block size. I think it's better to change gmirror to use the first sector on the provider. This never creates a race condition and as such, you don't need to invent a priority scheme, that has it's own set of flaws on top of it. The only downside is that it's not easy to make a fully partitioned and populated disk part of a mirror: one would need to move the data forward one sector to free the first sector. This we can actually do by inserting a GEOM that does it while I/O is still ongoing. The good thing is: we need a class that does exactly this for implementing the move verb in gpart. There were two reasons to use the last sector instead of first: 1. You want to be able to boot from gmirror. If all your data will be moved forward your boot sectors and kernel will be harder to find. 2. For recovery reasons you may want to turn off gmirror and still be able to access your data. Note that gmirror can handle the case where disk, slice and partition share the same last sector - it simply stores provider size in its metadata, so once it gets disk for tasting it detects its too big and ignores it, then slice will be given for tasting, but it also has larger size than expected and will be ignored as well. Finally partition will be tasted and gmirror configured. -- Pawel Jakub Dawidek http://www.wheel.pl p...@freebsd.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! pgpXtFT4O58hK.pgp Description: PGP signature
Re: gjournal fsck
On Thu, Aug 28, 2008 at 11:03:49AM -0400, Brian McCann wrote: On Thu, Aug 28, 2008 at 10:51 AM, Ivan Voras [EMAIL PROTECTED] wrote: Does gjournal complain about your drive, for example that it doesn't support BIOFLUSH? Actually yes...I meant to post them in my last message, but hit send too early...here's my output from boot (from dmesg.boot) GEOM_JOURNAL: Journal 478661671: da1 contains data. GEOM_JOURNAL: Journal 478661671: da1 contains journal. GEOM_JOURNAL: Journal da1 clean. (da1:mly0:4:1:0): SYNCHRONIZE CACHE. CDB: 35 0 0 0 0 0 0 0 0 0 (da1:mly0:4:1:0): CAM Status: SCSI Status Error (da1:mly0:4:1:0): SCSI Status: Check Condition (da1:mly0:4:1:0): ILLEGAL REQUEST asc:24,0 (da1:mly0:4:1:0): Invalid field in CDB (da1:mly0:4:1:0): Unretryable error GEOM_JOURNAL: BIO_FLUSH not supported by da1. GEOM_JOURNAL: Journal 3065355517: da2 contains data. GEOM_JOURNAL: Journal 3065355517: da2 contains journal. GEOM_JOURNAL: Journal da2 clean. (da2:mly1:4:0:0): SYNCHRONIZE CACHE. CDB: 35 0 0 0 0 0 0 0 0 0 (da2:mly1:4:0:0): CAM Status: SCSI Status Error (da2:mly1:4:0:0): SCSI Status: Check Condition (da2:mly1:4:0:0): ILLEGAL REQUEST asc:24,0 (da2:mly1:4:0:0): Invalid field in CDB (da2:mly1:4:0:0): Unretryable error GEOM_JOURNAL: BIO_FLUSH not supported by da2. Also, did fsck actually do something when it was started (did it find anything corrupted)? No...it didn't find anything wrong / anything to fix. (and one of the file systems was being written to at the time...so at least journaling appears to be working) Something else I noticed...Manolis' article says it should say journal consistent...whereas mine says Journal clean. I don't know what the differences mean, or what the BIO_FLUSH means...but I'm hoping you can tell me. :) Don't worry about those if you have memory-backed RAID card. BIO_FLUSH is there to tell the disk to flush its write caches, but if it is memory-backed, there is low risk on losing unwritted data. This is not the cause of your problem. BTW. journal ... clean means that there was clean shutdown and there was no need to replay journal, while journal ... consistent means that there was unclean shutdown and journal replay was successful. -- Pawel Jakub Dawidek http://www.wheel.pl [EMAIL PROTECTED] http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! pgp9J7tgcfgnR.pgp Description: PGP signature
Re: gjournal fsck
On Thu, Aug 28, 2008 at 05:44:32PM +0300, Manolis Kiagias wrote: Brian McCann wrote: Hi all. I'm having some problems with several servers I've built recently (7.0-RELEASE) that are using gjournal. I had two reboot a few days ago (un-related to FreeBSD problems I think)...but when they came back up, the file systems wouldn't mount since they were not clean. Now, I understand that UFS knows nothing about the fact that it's journaled, and the journaling knows nothing about UFS...but it's my understanding that by using gjournal, you should really never need to fsck a file system. However, the only way to get them to mount is by doing the fsck. Is there something else I should be doing instead of fsck? And since I know it will probably come up, I built the file systems using the instructions and notes at http://www.freebsd.org/cgi/man.cgi?query=gjournalapropos=0sektion=0manpath=FreeBSD+7.0-RELEASEformat=html. Any help would be greatly appreciated! Thanks! --Brian You may wish to have a look at this article: http://www.freebsd.org/doc/en_US.ISO8859-1/articles/gjournal-desktop In particular, you should make sure you use tunefs to enable Journaling and disable soft update on the journaled filesystems, i.e.: tunefs -J enable -n disable /dev/ad0s1f.journal Mount them using the async option: /dev/ad0s1f.journal /usrufs rw,async2 2 Note that the pass # still indicates the filesystem should be checked. While I was writing the article, I was trying several scenarios were I had the pass # set to 0, thinking that a gjournaled filesystem would not need fsck at all. I would then press the reset button. In most cases, the system would refuse to mount them. However with the pass # set, the fsck would finish almost immediately, since the actual consistency check takes place when the gjournal module is loaded (you will get a journal consistent after a bad reboot) and before fstab is even parsed. All fsck does in this case is simply confirm to the system it is a clean volume. In short, leaving the pass # to something that would cause an fsck is the safe way to go. The fsck will be almost instant anyway. Not exactly. Gjournal handles all inconsistencies, except one - orphaned files. An orphaned file is a file that some open, deleted, but haven't closed. If you have panic or power failure at this point, there will be unreferenced file. When fsck detects that it operates on gjournaled file system, it will do a very fast lookup of such orphaned files. It is optimized so that total number of such files is kept in superblock - if this counter is 0, we're done. If it is not zero, then fsck scans only cylinder groups headers, there is another counter in each header says how many orphaned files is in this CG. If CG's counter is 0, the whole CG can be skipped. Once we found X dirty CGs (X is stored in superblock) we can also stop earlier. All in all, there are very few such files (if any), so fsck is way, way faster with gjournal (eg. ~1 minute instead of ~8 hours on huge file system). -- Pawel Jakub Dawidek http://www.wheel.pl [EMAIL PROTECTED] http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! pgppLX1W7egLS.pgp Description: PGP signature
Re: gjournal fsck
On Thu, Aug 28, 2008 at 10:43:58AM -0400, Brian McCann wrote: On Thu, Aug 28, 2008 at 10:34 AM, Ivan Voras [EMAIL PROTECTED] wrote: Brian McCann wrote: Hi all. I'm having some problems with several servers I've built recently (7.0-RELEASE) that are using gjournal. I had two reboot a few days ago (un-related to FreeBSD problems I think)...but when they came back up, the file systems wouldn't mount since they were not clean. Now, I understand that UFS knows nothing about the fact that it's journaled, and the journaling knows nothing about UFS...but it's Actually UFS needs to know about gjournal for just this purpose. There's a special option to newfs that tells the file system to be aware of gjournal and it should request fscks. my understanding that by using gjournal, you should really never need to fsck a file system. However, the only way to get them to mount is by doing the fsck. Is there something else I should be doing instead of fsck? man 8 tunefs Yes...I apologize...I did that when I built the file system with newfs -J. Here's the tunefs -p output for one of the file systems: # tunefs -p /dev/da2.journal tunefs: ACLs: (-a) disabled tunefs: MAC multilabel: (-l) disabled tunefs: soft updates: (-n) disabled tunefs: gjournal: (-J) enabled tunefs: maximum blocks per file in a cylinder group: (-e) 2048 tunefs: average file size: (-f)16384 tunefs: average number of files in a directory: (-s) 64 tunefs: minimum percentage of free space: (-m) 8% tunefs: optimization preference: (-o) time tunefs: volume label: (-L) I'm concerned about this because the ones I've had problems with recently are 1.1TB arrays...I've got 2 8.6TB arrays that are journaled as well...and if I ever have to fsck them...that could take 1/2 the day... Any other thoughts? Everything looks good. Do you have 'fsck_y_enable' in your rc.conf by any chance? gjournal mode in fsck is only used for -p option, AFAIR. -- Pawel Jakub Dawidek http://www.wheel.pl [EMAIL PROTECTED] http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! pgpjQcSutFUWM.pgp Description: PGP signature
Re: Delaying mount of UFS filesystem in ZFS pool
On Fri, May 23, 2008 at 02:04:56PM -0700, Aaron Holmes wrote: I have a UFS filesystem inside a zpool: tank on /tank (zfs, local) /dev/zvol/tank/ufs on /mnt/ufs (ufs, local, acls) If I add that entry (/dev/zvol/tank/ufs) to /etc/fstab, it will try to mount as a critical filesystem on boot, however, because ZFS hasn't yet loaded, this fails and causes all sorts of fun for me. Currently I have that filesystem mounting via a cronjob that checks every minute if it's mounted.. definitely not ideal. I need this filesystem in /etc/fstab so I can setup quotas on it (if there is some other way to get quotas working, great, point me to a link or two). So what I'm thinking for a solution is to delay the mount of this filesystem until ZFS has loaded, but I'm not sure of a way to do this with the filesystem in /etc/fstab, and without extensive hacking to one or more rc scripts. Ideas? Adding 'late' flag in Options section to the fstab entry may help, although I don't think it will help with quotas: # rcorder /etc/rc.d/* [...] /etc/rc.d/mountcritlocal [...] /etc/rc.d/zfs [...] /etc/rc.d/mountcritremote [...] /etc/rc.d/quota [...] /etc/rc.d/mountlate [...] We might consider running rc.d/quota after rc.d/mountlate, not sure if it won't break something else. I added freebsd-rc@ to CC. -- Pawel Jakub Dawidek http://www.wheel.pl [EMAIL PROTECTED] http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! pgpDVThx2XiWV.pgp Description: PGP signature
Re: zfs list and non-root user
On Fri, Apr 11, 2008 at 01:41:28PM -0500, Mark Kane wrote: On Fri, Apr 11, 2008, at 15:13:16 +0200, Norman Maurer wrote: Hi all, is it normal that I can't do a 'zfs list' ( for example ) as non-root user ? $ zfs list internal error: failed to initialize ZFS library I think there is really a use case for use some zfs commands as non-root user.. Thx Norman Hi. One way to do this as a non-root user is to add the account to the operator group. This is what I do on my personal desktop machine and it has worked fine, but I understand that may not be best in all cases. You might also try changing the permissions on /dev/zfs. I don't do this method and I'm not sure if it's a proper way, but from trying it very briefly it seems to work correctly with the user not in the operator group. In Solaris anyone can open /dev/zfs and the kernel side of ZFS decides if the user has permission to perform some action or not. In FreeBSD we try to be more careful for now, but it will change soon, once we import delegated administration functionality. Although... The error above (failed to initialize ZFS library) most likely means that zfs.ko module wasn't loaded. zfs(8) tries to do that automatically, but of course it will only succeed if we are root. In this case zfs.ko has to be manually loaded by root and then members of operator group can use zfs(8) command. -- Pawel Jakub Dawidek http://www.wheel.pl [EMAIL PROTECTED] http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! pgpFjNMczD1Yo.pgp Description: PGP signature
Re: Booting a GELI encrypted hard disk
On Thu, Oct 25, 2007 at 03:53:34PM +0200, Oliver Fromme wrote: The pen-drive is not needed for your system to run and you can be easly take it with you, which is not always the case for your laptop. Are you saying that the USB pen-drive can be removed while the system is running (after it has booted)? I remember that it was impossible in the past to remove the root vnode (which in this case would be the /boot file system from the pen-drive). Did that change recently? Or is there a way to change the system's root vnode from the pen-drive to the root file system on the encrypted disk? If so, then how? The boot directory is different that root file system. /boot/ directory is only accessed by loader before root file system is mounted. The root file system can be mounted from encrypted disk, because loader loads the kernel (and eventually geom_eli.ko module) from the /boot/ directory. Most of the time /boot/ directory is on the root file system, but there is no need for that - you can boot from different /boot/ directory and have different /boot/ directory in your root file system. -- Pawel Jakub Dawidek http://www.wheel.pl [EMAIL PROTECTED] http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! pgpb9gKin9aG5.pgp Description: PGP signature
Re: Booting a GELI encrypted hard disk
On Thu, Oct 25, 2007 at 12:46:53AM +0800, Daniel Marsh wrote: Even if all data on a drive is encrypted, the partition table is not. Software based disk encryption works on partitions. That's not true. One can configure full disk encryption using GELI. To do it you need to have a small USB pen-drive or CD-ROM with /boot/ directory, but that's all you need. Then you actually boot from your unencrypted pen-drive, but mount all file systems from encrypted disk. The pen-drive is not needed for your system to run and you can be easly take it with you, which is not always the case for your laptop. How far into the boot sequence do you get before your system crashes without the key present? I would assume as far as reading the / partition to get the kernel etc... It would have read the partition table and the boot loader, known which partition was the active partition and tried booting it. Now, to identify what OS this disk has on it you can check the partition table and see what type has been set for each slice/partition. You will be able to see that there is a BSD style slice on the disk just by running `fdisk /dev/mystolendiskdevice` You now know it's a BSD OS, you could then make a guess as to what version of BSD by the type of machine it was taken from, based on what hardware is supported by each BSD. I believe their slices and layout are identical but the file systems differ. The person with your disk could then start trying to determine what kind of disk encryption is in place. That's all irrelevant. Security of GELI (or any sane cryptographic system) doesn't depend on secrecy of algorithms used. -- Pawel Jakub Dawidek http://www.wheel.pl [EMAIL PROTECTED] http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! pgpA1kmMdGF3e.pgp Description: PGP signature
Re: Booting a GELI encrypted hard disk
On Wed, Oct 10, 2007 at 07:53:49PM +0200, Roland Smith wrote: On Wed, Oct 10, 2007 at 09:04:34AM -0400, Steve Bertrand wrote: Hi all, I am voraciously attempting to get a FreeBSD system to boot from a GELI encrypted hard disk, but am having problems. You don't need to encrypt the whole harddisk. You can encrypt separate slices. There is no need to encrypt stuff like / or /usr; what is there that needs to be kept secret? Maybe not encryption, but integrity protection is very important for laptops. GELI supports integrity protection for a while now. If you don't protect integrity of your entire laptop disk, it is trivial to trojan userland utilities and/or kernel and steal your password. If someone needs your data, he can dump encrypted partition, trojan your system and once you connect to the internet and attach your encrypted partition, the trojan will send the password to the attacker. Many people often leave their laptops in hotels rooms, for example. -- Pawel Jakub Dawidek http://www.wheel.pl [EMAIL PROTECTED] http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! pgp48YVSMzslF.pgp Description: PGP signature
Re: GEOM ELI: MD5 hash mismatch for /dev/mirror/gm0s1f
On Sun, Sep 09, 2007 at 08:07:49PM +0200, Thomas Hobbes wrote: Hi, what is this kind of error's meaning? It means that metadata (in the last provider's sector) is in an inconsistent state. Someone/something changed it. -- Pawel Jakub Dawidek http://www.wheel.pl [EMAIL PROTECTED] http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! pgpxaReQnQvyC.pgp Description: PGP signature
Re: gmirror woes on 6.2-S, Aug 1
On Wed, Aug 15, 2007 at 02:57:19PM -0700, Kelsey Cummings wrote: FreeBSD meno 6.2-STABLE FreeBSD 6.2-STABLE #3: Wed Aug 1 08:21:29 PDT 2007 root@:/usr/obj/usr/src/sys/SMP i386 I just swapped a gmirror pair of disks into a new box and have run into a problem that I can't seem to figure out. Upon boot it reports the following and doesn't appear to see ad10, the other disk in the mirror set. GEOM_MIRROR: Device gm0 created (id=2427626556). GEOM_MIRROR: Device gm0: provider ad12 detected. GEOM_MIRROR: Force device gm0 start due to timeout. GEOM_MIRROR: Device gm0: provider ad12 activated. GEOM_MIRROR: Device gm0: provider mirror/gm0 launched. However, ad10 was seen at boot and is currently attached. meno# atacontrol list | grep ad Master: ad10 WDC WD3000JD-00KLB0/08.05J08 Serial ATA v1.0 Master: ad12 WDC WD3000JD-00KLB0/08.05J08 Serial ATA v1.0 So: meno# gmirror forget gm0 meno# gmirror insert gm0 ad10 meno# gmirror status NameStatus Components mirror/gm0 DEGRADED ad12 ad10 (0%) ... Which compelete and everything appears to work fine, until I reboot and repeat the cycle. Has anyone run into anything similar, if so, what is the fix? Could I add: kern.geom.mirror.debug=2 to your /boot/loader.conf and reboot? It should print provider it tastes, this will show us if ad10 is given for tasting at all. -- Pawel Jakub Dawidek http://www.wheel.pl [EMAIL PROTECTED] http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! pgpsqXgfNvWkl.pgp Description: PGP signature
Re: geom - help ...
On Thu, Sep 21, 2006 at 11:49:20AM +0200, Ivan Voras wrote: Marc G. Fournier wrote: Basically, I have 5x72G drives ... I'd *love* to do RAID5 with them, but that doesn't appear to be available right now Maybe you could try geom_raid3 instead? It's pretty nice, especially for databases. The only caveat is that you cannot boot off it if you use all the drives. Maybe raid3(3 drives for data)+mirror(2 disks for system) will be useful to you (since you can boot a mirrored drive). (note that geom_raid3 is not RAID3 as theoretically defined, but more like RAID4). This is RAID3, RAID4 is totally different. -- Pawel Jakub Dawidek http://www.wheel.pl [EMAIL PROTECTED] http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! pgphQi3Z4pHYM.pgp Description: PGP signature
