On 1/19/2005 10:57 PM Ted Mittelstaedt wrote:
Hi Drew,
Please read the following:
http://www.vinumvm.org/vinum/how-to-debug.html
I have.
And follow the instructions exactly. And I mean exactly.I thought I had, with the exception of the /var/log/vinum_history and /var/log/messages files due to the size. However I offered to provide them separately.
Yes, however I thought that vinum is still the way to go with 4-STABLE versions. Is this still true?Also keep the following in mind, Greg will try to help but note carefully the sentence on this webpage:
"Since I wrote it, FreeBSD has changed its I/O structure, breaking many
things in Vinum. At the time of writing, a new version, provisionally
called gvinum, is being written"
I can understand that about Greg. I'm sure he has so many emails to deal with each day that there's no point spending time on replies that don't offer any benefit. But what really blows me away is that my volumes work if they are deleted and re-created. This tells me my volume is not "screwed" because they work. I just don't understand why they don't survive reboots. These are the same volumes that I've had since version 4.1 and they've worked flawlessly -- well there was the time I tried to add a firewire drive to a concatenated volume but had the SCSI delay set too low in the kernel so the drive was not available when vinum started. However once I figured that out (with help from Hidetoshi Shimokawa, the driver author), everything worked fine again. However the upgrade from 4.9 to 4.10 royally screwed up the vinum volumes and they haven't been right since.I myself have had one serious crash on a vinum RAID volume as a result of a SCSI cable problem that blew away the volume. (2 drives were corrupted, instead of just one, making it impossible for the volume manager to repair by itself) I sent all the info to Greg but ultimately he wasn't able to offer any suggestions on recovering the array so I just wiped it and started over.
Note that Greg DID NOT recommend wiping the array. In fact he
didn't recommend anything. The lack of any recommendation
appears to be his way of telling you "your volume is screwed,
wipe it and start over" Like most UNIX commands, if Greg
has nothing to offer, he says nothing at all, he won't tell you he
has nothing to offer. So, the lack of a response to your
original post you can probably take as an answer, to be honest.
This did teach me a lesson that I kind of knew already butAgreed. I have backups for crash recovery. With the exception of /usr, I used vinum so that I could have large drives on which to store backups, mp3s, mpegs, online copies of software, etc.
didn't think too much about. That is, a software array is no substitute
for a hardware array. In other words, vinum is a great thing
if what your wanting to do is use a bunch of cheap disks and
cheap controller cards to either get a giant partition, or to
stripe them together and get faster access. But it's not so
good if the intent is to get some crash recovery.
I don't use and have never used vinum for /etc, /, /usr, /varYou have a valid point and I will heed your advice when I finally wipe and upgrade to 5-STABLE. In the beginning, it seemed to be a good idea to use vinum to stripe my /usr for speed. Since this is just a home/hobby system, I probably never really needed the speed increase but I wanted to experiment.
or any other system partitions. I only use it for partitions
that I want to mount AFTER the system is booted. If I were in
your shoes I'd nuke your system and start all over again and
rethink how I had it laid out. I would use a single disk for
the system then take the rest of the disks and put them together
under vinum. Then I'd mount that on /ftp and I'd softlink
whatever thing is gopping up space under /usr, for example
/usr/local/www, to a directory under /ftp
Vinum isn't going to give you any crash recoveryYou're right and my nose is still straight. :) See above as to "why" I striped /usr.
for /usr so there is really no point in making /usr a vinum
volume. Beyond that I really don't understand why you
are putting /usr as a vinum volume, espically as you yourself
said "Fortunately this volume is up and running or I would
really be in a mess" I mean, your basically saying your
hitting yourself in the face and you feel fortunate you
haven't broken your nose yet.
Anyway, one other thing I will bring up: How exactly did you update your system? Did you nuke and repave it? Or did you follow the instructions here EXACTLY:
http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/makeworld.html
As I have since version 4.1. My steps are:
make buildworld
make buildkernel
make installkernel
make installworld
mergemaster (accept replacement of all files I have not modified and merge my mods with new files)
reboot
I see 19.4.1 has two steps between "make installkernel" and "make installworld" that I did not follow. Do you suspect this is where my problem lies? I need to upgrade to the latest security level so I could try this.
If you didn't do one or the other of these things then nobody is going to help you.
Ted
Thanks your your time and input!
Drew
-----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Drew Tomlinson Sent: Tuesday, January 18, 2005 2:00 PM To: FreeBSD Questions Subject: One Last Plea For Vinum Assistance
I sent the message below a couple of times but did not receive any response. I assume that it's because either I have a really difficult problem or am asking something really stupid. :) But anyway, I want to install additional memory in this machine and am sure I will run across the same problems after shutting down. So if anyone has any suggestions on how I might solve this issue, I'd really appreciate the input.
Thanks,
Drew
--- Original Message --- Since an upgrade from 4.9 to 4.10, I've had problems with vinum. The basic problem is that upon reboot, two of my vinum drives show up as "referenced" and thus create the associated chaos. I've tried many things and fiddled around quite a bit so I can't say exactly what I've done. I can include all of the entries in the history file since Oct. 31 if that's a help but it would be a long list.
So prior to digging that deep, I will describe where I stand currently and where I want to finish. Currently, I have one vinum volume that I use for /usr. Fortunately this volume is up and running or I would really be in a mess. Here's the 'vinum list' output in this state:
blacklamb# vinum vinum -> list 2 drives: D disk1 State: up Device /dev/da0s1h Avail: 0/8383 MB (0%) D disk2 State: up Device /dev/da1s1h Avail: 0/8383 MB (0%)
1 volumes: V usr State: up Plexes: 1 Size: 16 GB
1 plexes: P usr.p0 S State: up Subdisks: 2 Size: 16 GB
2 subdisks: S usr.p0.s0 State: up PO: 0 B Size: 8383 MB S usr.p0.s1 State: up PO: 256 kB Size: 8383 MB
I want to add another volume and mount it on /ftp. After creating the volume, vinum sees it and it appears OK as indicated in this output:
vinum -> list 5 drives: D disk1 State: up Device /dev/da0s1h Avail: 0/8383 MB (0%) D disk2 State: up Device /dev/da1s1h Avail: 0/8383 MB (0%) D ftp1 State: up Device /dev/ad0s1h Avail: 0/76319 MB (0%) D ftp2 State: up Device /dev/ad1s1h Avail: 0/76319 MB (0%) D ftp3 State: up Device /dev/da3s1h Avail: 0/114473 MB (0%)
2 volumes: V usr State: up Plexes: 1 Size: 16 GB V ftp State: up Plexes: 1 Size: 260 GB
2 plexes: P usr.p0 S State: up Subdisks: 2 Size: 16 GB P ftp.p0 C State: up Subdisks: 3 Size: 260 GB
5 subdisks: S usr.p0.s0 State: up PO: 0 B Size: 8383 MB S usr.p0.s1 State: up PO: 256 kB Size: 8383 MB S ftp.p0.s0 State: up PO: 0 B Size: 74 GB S ftp.p0.s1 State: up PO: 74 GB Size: 74 GB S ftp.p0.s2 State: up PO: 149 GB Size: 111 GB
Next I 'quit' the vinum command line and fsck my newly created volume. It finishes successfully and I mount it:
blacklamb# df Filesystem 1K-blocks Used Avail Capacity Mounted on /dev/da0s1a 302350 102088 176074 37% / /dev/vinum/usr 16639674 5433762 9874739 35% /usr procfs 4 4 0 100% /proc /dev/vinum/ftp 265119539 119729707 132133856 48% /ftp
I recheck with the 'vinum list' command and the output is the same as above. Now I cross my fingers and reboot to see if the volume comes up properly on startup. However ftp3 (/dev/da3s1h) comes up 'referenced' and causes problems.
Mounting root from ufs:/dev/da0s1a dumpon: crash dumps to /dev/da1s1b (13, 131081) vinum: loaded vinum: reading configuration from /dev/ad1s1h vinum: ftp.p0.s2 is crashed vinum: ftp.p0 is corrupt vinum: updating configuration from /dev/ad0s1h vinum: updating configuration from /dev/da1s1h vinum: updating configuration from /dev/da0s1h vinum: /dev is mounted read-only, not rebuilding /dev/vinum Warning: defective objects
D ftp3 State: referenced Device Avail: 0/0 MB P ftp.p0 C State: corrupt Subdisks: 3 Size: 260 GB S ftp.p0.s2 State: crashed PO: 149 GB Size: 111 GB swapon: adding /dev/da1s1b as swap device Automatic boot in progress... /dev/da0s1a: FILESYSTEM CLEAN; SKIPPING CHECKS /dev/da0s1a: clean, 100130 free (938 frags, 12399 blocks, 0.6% fragmentation) /dev/vinum/ftp: CANNOT READ: BLK 546979872 /dev/vinum/ftp: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY. /dev/vinum/ftp: CANNOT WRITE: BLK 2800 /dev/vinum/ftp: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY. /dev/vinum/usr: FILESYSTEM CLEAN; SKIPPING CHECKS /dev/vinum/usr: clean, 11205903 free (152463 frags, 1381680 blocks, 0.9% fragmentation) THE FOLLOWING FILE SYSTEM HAD AN UNEXPECTED INCONSISTENCY: /dev/vinum/ftp (/ftp) File system preen failed, trying fsck -y . . . ** /dev/da0s1a ** Last Mounted on / ** Root file system ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 2053 files, 51045 used, 100130 free (938 frags, 12399 blocks, 0.6% fragmentation) ** /dev/vinum/usr ** Last Mounted on /usr ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 401072 files, 5433771 used, 11205903 free (152463 frags, 1381680 blocks, 0.9% fragmentation) ** /dev/vinum/ftp
CANNOT READ: BLK 546979872 UNEXPECTED SOFT UPDATE INCONSISTENCY
CONTINUE? yes
THE FOLLOWING DISK SECTORS COULD NOT BE READ: 546979872, 546979873, 546979874, 546979875, /dev/vinum/ftp: CANNOT FIGURE OUT FILE SYSTEM PARTITION
UNEXPECTED SOFT UPDATE INCONSISTENCY WARNING: R/W mount of /ftp denied. Filesystem is not clean - run fsck mount: /dev/vinum/ftp: Operation not permitted Mounting /etc/fstab filesystems failed, startup aborted Enter full pathname of shell or RETURN for /bin/sh:
I enter the shell and run 'vinum list':
vinum -> list 4 drives: D ftp1 State: up Device /dev/ad0s1h Avail: 0/76319 MB (0%) D ftp2 State: up Device /dev/ad1s1h Avail: 0/76319 MB (0%) D disk1 State: up Device /dev/da0s1h Avail: 0/8383 MB (0%) D disk2 State: up Device /dev/da1s1h Avail: 0/8383 MB (0%) D ftp3 State: referenced Device Avail: 0/0 MB
2 volumes: V usr State: up Plexes: 1 Size: 16 GB V ftp State: up Plexes: 1 Size: 260 GB
2 plexes: P usr.p0 S State: up Subdisks: 2 Size: 16 GB P ftp.p0 C State: corrupt Subdisks: 3 Size: 260 GB
5 subdisks: S usr.p0.s0 State: up PO: 0 B Size: 8383 MB S usr.p0.s1 State: up PO: 256 kB Size: 8383 MB S ftp.p0.s0 State: up PO: 0 B Size: 74 GB S ftp.p0.s1 State: up PO: 74 GB Size: 74 GB S ftp.p0.s2 State: crashed PO: 149 GB Size: 111 GB
Now I delete all references to the ftp volume by running these commands:
vinum -> rm -f ftp.p0.s2 vinum: removing ftp.p0.s2 Correcting length of ftp.p0: was 547043829, is 312602446 vinum: ftp.p0 is up vinum -> rm -f ftp.p0.s1 vinum: removing ftp.p0.s1 Correcting length of ftp.p0: was 312602446, is 156301223 vinum -> rm -f ftp.p0.s0 vinum: removing ftp.p0.s0 vinum -> rm -f ftp.p0 vinum: removing ftp.p0 vinum: ftp is down vinum -> rm -f ftp vinum: removing ftp vinum -> rm -f ftp3 vinum -> rm -f ftp2 vinum -> rm -f ftp1
Yet the 'reference to ftp3 remains.
vinum -> list 1 drives: D disk1 State: up Device /dev/da0s1h Avail: 0/8383 MB (0%) D disk2 State: up Device /dev/da1s1h Avail: 0/8383 MB (0%) D ftp3 State: referenced Device Avail: 0/0 MB
1 volumes: V usr State: up Plexes: 1 Size: 16 GB
1 plexes: P usr.p0 S State: up Subdisks: 2 Size: 16 GB
2 subdisks: S usr.p0.s0 State: up PO: 0 B Size: 8383 MB S usr.p0.s1 State: up PO: 256 kB Size: 8383 MB
Why does the list output show "1 drives:" when there are three listed and two working? Anyway, I've been here before and I quit vinum. Then I issue these commands:
# df Filesystem 512-blocks Used Avail Capacity Mounted on /dev/da0s1a 604700 204180 352144 37% / /dev/vinum/usr 33279348 10867544 19749458 35% /usr procfs 8 8 0 100% /proc # umount /usr # vinum stop vinum: unloaded vinum unloaded # vinum start vinum: loaded vinum: reading configuration from /dev/da1s1h vinum: updating configuration from /dev/da0s1h
Then I issue a 'vinum list' and all seems well.
# vinum vinum -> list 2 drives: D disk1 State: up Device /dev/da0s1h Avail: 0/8383 MB (0%) D disk2 State: up Device /dev/da1s1h Avail: 0/8383 MB (0%)
1 volumes: V usr State: up Plexes: 1 Size: 16 GB
1 plexes: P usr.p0 S State: up Subdisks: 2 Size: 16 GB
2 subdisks: S usr.p0.s0 State: up PO: 0 B Size: 8383 MB S usr.p0.s1 State: up PO: 256 kB Size: 8383 MB vinum ->
Since everything appears correct, this seems like a good place to do 'vinum saveconfig'. Next I edit /etc/fstab so only /ftp is not mounted and reboot. The system reboots fine and the 'vinum list' is as it is immediately above.
Running in full production mode, I create the ftp volume again:
vinum -> create -f vinum_ftp.conf Dec 24 16:37:45 blacklamb /kernel: vinum: drive ftp1 is up Dec 24 16:37:45 blacklamb /kernel: vinum: drive ftp2 is up Dec 24 16:37:45 blacklamb /kernel: vinum: drive ftp3 is up 5 drives: D disk1 State: up Device /dev/da0s1h Avail: 0/8383 MB (0%) D disk2 State: up Device /dev/da1s1h Avail: 0/8383 MB (0%) D ftp1 State: up Device /dev/ad0s1h Avail: 0/76319 MB (0%) D ftp2 State: up Device /dev/ad1s1h Avail: 0/76319 MB (0%) D ftp3 State: up Device /dev/da3s1h Avail: 0/114473 MB (0%)
2 volumes: V usr State: up Plexes: 1 Size: 16 GB V ftp State: up Plexes: 1 Size: 260 GB
2 plexes: P usr.p0 S State: up Subdisks: 2 Size: 16 GB P ftp.p0 C State: up Subdisks: 3 Size: 260 GB
5 subdisks: S usr.p0.s0 State: up PO: 0 B Size: 8383 MB S usr.p0.s1 State: up PO: 256 kB Size: 8383 MB S ftp.p0.s0 State: up PO: 0 B Size: 74 GB S ftp.p0.s1 State: up PO: 74 GB Size: 74 GB S ftp.p0.s2 State: up PO: 149 GB Size: 111 GB Dec 24 16:37:45 blacklamb /kernel: vinum: ftp.p0.s0 is up Dec 24 16:37:45 blacklamb /kernel: vinum: ftp.p0.s1 is up Dec 24 16:37:45 blacklamb /kernel: vinum: ftp.p0.s2 is up Dec 24 16:37:45 blacklamb /kernel: vinum: ftp.p0 is up Dec 24 16:37:45 blacklamb /kernel: vinum: ftp is up
After running fsck, I can mount and access the volume. The system continues to run fine until the next reboot where the same problems start all over again. I also have the same problem with another volume I wish to run called "backup". How can I fix my vinum volumes so they survive reboots?
Thanks for your help and Happy Holidays!
Drew
_______________________________________________ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
