Re: Frustration with SCSI system
On Wed, Sep 20, 2000 at 01:24:34PM -0700, Edward Elhauge wrote: OK, vinum is good. But my understanding is that you can't use vinum on your root partition. By Murphy's Law it always seems to be root that gets screwed up. And that also causes the biggest problems because then you have to yank the system apart and find another host disk for booting. What about "duplicating and updating" /, /var and /usr onto a 2nd disk ? This could be automatically done by a shellscript. You only need to fine tune /mirror/etc/fstab to boot from the correct disk after a such a "backup/mirror" run If you install the freebsd boot manager you can even easily test booting from the 2nd drive without having to make special settings in the forth boot loader. You could manage this using three disks I think Disk: da0 da1 da2 Fs: + root-fs | s1a s1a (backup)| | + | swap| swapswap| | + swap | var-fs | s1f s1f (backup)| | + | usr-fs | s1g s1g (backup)| | +---+ vinum RAID5 | /usr/local | +---+ vinum RAID5 | /home | +---+ I'm not sure, maybe its even possible to put /var onto a Vinum Software RAID-5 .. Andreas /// -- Andreas Klemm Powered by FreeBSD SMP Songs from our band 64Bitshttp://www.apsfilter.org/64bits.html My homepage http://people.FreeBSD.ORG/~andreas Please note: Apsfilter got a NEW HOMEhttp://www.apsfilter.org/ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Frustration with SCSI system
Warner Losh wrote: In message [EMAIL PROTECTED] Douglas Swarin writes: : Ideally, I would use one of the IDE flash-based drives on the market. One : brand is SanDisk, and they take a standard IDE connector and fit into a : 3.5" drive bay. You can get them very reasonably priced up to 128MB or : so, which is just fine for a boot partition. Since flash drives have no : moving parts, mechanical failure is not an issue, and since the root : partition is not written to much, the flash will not wear out for a : long time (flash cells wear out after about 100,000 writes; the flash : drives do load balancing and stuff to ensure that the (many) cells in : the drive are written to evenly). We use these devices heavily at Timing Solutions. Or rather we use a IDE - CF adapter and haven't had any devices wear out. And some of these devices have had rather heavy use. I think that it is closer to 1 million writes per cell, but I don't have my spec sheets handy. The newer devices do 1 million writes per cell. When I left Xylan earlier this year, some of our early (late '94 or early '95) flash devices were just beginning to fail. These were development machines that saw a lot of write cycles, and their home-grown flash filesystem does a pessimal job of rewriting the same cells over and over again. Are you sure that they do write balancing? The indications I have from the base chip technology is that they don't. I could have missed that in the data sheets. It has been a little while since I looked at them, so I might be misremembering. I can't seem to find the data sheets I looked at before. SanDisk does, in the controller chip. Good technology. -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC [EMAIL PROTECTED] http://softweyr.com/ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Frustration with SCSI system
Ideally, I would use one of the IDE flash-based drives on the market. One brand is SanDisk, and they take a standard IDE connector and fit into a 3.5" drive bay. You can get them very reasonably priced up to 128MB or so, which is just fine for a boot partition. Since flash drives have no moving parts, mechanical failure is not an issue, and since the root partition is not written to much, the flash will not wear out for a long time (flash cells wear out after about 100,000 writes; the flash drives do load balancing and stuff to ensure that the (many) cells in the drive are written to evenly). I would suggest just not writing them at all if you can avoid it. They seem to need (and are spec'd for) about 500ms of power after a write which isn't feasible with some power-off situations. I've had them occasionally behave worse and develop "bad" sectors under what i thought to be normal operating conditions (meaning, I thought I satisfied the above limitation). The bright side is that reclaiming "bad" sectors is just rewriting them, but if that's in the middle of something useful, like an inode for something in /lib, your superblock, the data portion of /etc/passwd, etc., you're in uncomfortable shape. I would definitely advocate (and actively use them for) completely readonly disks, either by running completely readonly straight from the SanDisk or by keeping a (compressed?) readonly filesystem on the disk, booting readonly into a bootstrap environment, and copying or expanding a read/write root disk into a memory-based disk device (MD, MFS), and calling that your / partition. it's admittedly a rather klunky solution for a general purpose install. It also makes putting /etc on non-volatile media difficult, but it is still workable with a bit of ingenuity, or maybe by using a small bootstrap /etc which is remounted with a stable vinum partition (I haven't actually tried this; implementation is left as an exercise for the reader). Another consideration is that they have slow write performance relative to typical IDE or (especially) memory-based media. - joel Doug On Thu, Sep 21, 2000 at 03:29:17PM -0700, Keith Kemp wrote: On the topic of Vinum, what do you guys do about the / partion since it appears that a vinum partion can not be the boot partion. I would hate to have the drive with my boot partion fail and be left with a non working server. Keith Kemp -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On Behalf Of Alfred Perlstein Sent: Wednesday, September 20, 2000 12:51 PM To: Edward Elhauge Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: Re: Frustration with SCSI system * Edward Elhauge [EMAIL PROTECTED] [000920 12:48] wrote: Hello Freebsders, I've been using FreeBSD over the last 6 years (since I switched from NetBSD) to run a small ISP out of my basement. I've had about six disk crashes in as many years and still don't know how to work reliably with them. "man vinum" software mirroring == good. :) -Alfred To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-isp" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Frustration with SCSI system
On Thu, Sep 21, 2000, Douglas Swarin wrote: Ideally, I would use one of the IDE flash-based drives on the market. One brand is SanDisk, and they take a standard IDE connector and fit into a 3.5" drive bay. You can get them very reasonably priced up to 128MB or so, which is just fine for a boot partition. Since flash drives have no moving parts, mechanical failure is not an issue, and since the root partition is not written to much, the flash will not wear out for a long time (flash cells wear out after about 100,000 writes; the flash drives do load balancing and stuff to ensure that the (many) cells in the drive are written to evenly). The neater magic will come out later when the only thing that needs to be machine-boot-readable is /boot, and your / can be vinum. Then all you need to have "hotswap" is your kernel+loader, which you could possibly get away even on a floppy disk. Adrian -- Adrian Chadd"The main reason Santa is so jolly is [EMAIL PROTECTED] because he knows where all the bad girls live." -- Random IRC quote To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
RE: Frustration with SCSI system
On the topic of Vinum, what do you guys do about the / partion since it appears that a vinum partion can not be the boot partion. I would hate to have the drive with my boot partion fail and be left with a non working server. Keith Kemp -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On Behalf Of Alfred Perlstein Sent: Wednesday, September 20, 2000 12:51 PM To: Edward Elhauge Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: Re: Frustration with SCSI system * Edward Elhauge [EMAIL PROTECTED] [000920 12:48] wrote: Hello Freebsders, I've been using FreeBSD over the last 6 years (since I switched from NetBSD) to run a small ISP out of my basement. I've had about six disk crashes in as many years and still don't know how to work reliably with them. "man vinum" software mirroring == good. :) -Alfred To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-isp" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Frustration with SCSI system
Ideally, I would use one of the IDE flash-based drives on the market. One brand is SanDisk, and they take a standard IDE connector and fit into a 3.5" drive bay. You can get them very reasonably priced up to 128MB or so, which is just fine for a boot partition. Since flash drives have no moving parts, mechanical failure is not an issue, and since the root partition is not written to much, the flash will not wear out for a long time (flash cells wear out after about 100,000 writes; the flash drives do load balancing and stuff to ensure that the (many) cells in the drive are written to evenly). Doug On Thu, Sep 21, 2000 at 03:29:17PM -0700, Keith Kemp wrote: On the topic of Vinum, what do you guys do about the / partion since it appears that a vinum partion can not be the boot partion. I would hate to have the drive with my boot partion fail and be left with a non working server. Keith Kemp -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On Behalf Of Alfred Perlstein Sent: Wednesday, September 20, 2000 12:51 PM To: Edward Elhauge Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: Re: Frustration with SCSI system * Edward Elhauge [EMAIL PROTECTED] [000920 12:48] wrote: Hello Freebsders, I've been using FreeBSD over the last 6 years (since I switched from NetBSD) to run a small ISP out of my basement. I've had about six disk crashes in as many years and still don't know how to work reliably with them. "man vinum" software mirroring == good. :) -Alfred To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-isp" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Frustration with SCSI system
In message [EMAIL PROTECTED] Douglas Swarin writes: : Ideally, I would use one of the IDE flash-based drives on the market. One : brand is SanDisk, and they take a standard IDE connector and fit into a : 3.5" drive bay. You can get them very reasonably priced up to 128MB or : so, which is just fine for a boot partition. Since flash drives have no : moving parts, mechanical failure is not an issue, and since the root : partition is not written to much, the flash will not wear out for a : long time (flash cells wear out after about 100,000 writes; the flash : drives do load balancing and stuff to ensure that the (many) cells in : the drive are written to evenly). We use these devices heavily at Timing Solutions. Or rather we use a IDE - CF adapter and haven't had any devices wear out. And some of these devices have had rather heavy use. I think that it is closer to 1 million writes per cell, but I don't have my spec sheets handy. Are you sure that they do write balancing? The indications I have from the base chip technology is that they don't. I could have missed that in the data sheets. It has been a little while since I looked at them, so I might be misremembering. I can't seem to find the data sheets I looked at before. In any event, this works well. I usually have / be read only. This can be practacle if you don't have any users that desire to change their passwords... Since I have serveral machines that have an extremely limited number of users on, this works well. One can also mount / rw if you need to do maintenance on it for whatever reason. Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Frustration with SCSI system
Whoops, sorry about the previous misfire...
In message [EMAIL PROTECTED] Edward Elhauge writes:
: to autorecover on bad sectors, but every system that I've had to recover
: seems to be in a state where the bad sectors aren't remapping. I've tried
I've often wanted to write a bad block remapper. While SCSI is
supposed to do this automatically, I've found that a scan on any
adaptec controller will remap these blocks (forces the remapping).
I've got a program which attempts to read all blocks and may be able
to rewrite bad blocks. It's nothing fancy. Only works if you've got
auto-reallocate turned on in the drive.
% cat diskscan.c
#includestdio.h
#includeunistd.h
#includeerrno.h
#includefcntl.h
int fixup(off, fd)
off_t off;
int fd;
{
char buffer[512];
int i, rval;
for (i = 0; i 65536; i += 512) {
if (lseek(fd, off + i, SEEK_SET) 0) {
doerror("lseek", off + i, 0);
}
if ((rval = read(fd, buffer, sizeof(buffer))) != sizeof(buffer)) {
if (errno) {
if (lseek(fd, off + i, SEEK_SET) 0) {
doerror("lseek", off + i, 0);
}
write(fd, buffer, sizeof(buffer));
}
}
}
}
int doerror(str, off, type)
char *str;
off_t off;
int type;
{
static int ign = 0;
char buffer[80];
fprintf(stderr, "\nError at %qx, ", (quad_t) off);
perror(str);
if (! ign) {
if (type) {
fprintf(stderr, "Attempt to correct? (y/n/a) ");
} else {
fprintf(stderr, "Press 'y' to continue: ");
}
while (! ign) {
fgets(buffer, sizeof(buffer), stdin);
if (*buffer == 'y') {
return(0);
}
if (*buffer == 'n') {
return(1);
}
if (*buffer == 'a') {
ign++;
return(0);
}
}
}
return(0);
}
int main(argc, argv)
int argc;
char *argv[];
{
int fd, rval;
char buffer[65536];
off_t off = 0;
int eof = 0;
int count = 0;
if (argc != 2) {
fprintf(stderr, "usage: diskscan dev\n");
exit(1);
}
if ((fd = open(argv[1], O_RDWR, 0644)) 0) {
perror(argv[1]);
}
while (! eof) {
if (! count) {
fprintf(stderr, "%qx, ", (quad_t) off);
}
count++;
count %= 8;
if (lseek(fd, off, SEEK_SET) 0) {
doerror("lseek", off, 0);
}
if ((rval = read(fd, buffer, sizeof(buffer))) != sizeof(buffer)) {
if (errno) {
if (! doerror("read", off, 1)) {
fixup(off, fd);
}
}
}
off += sizeof(buffer);
}
}
I don't even guarantee that it's correct, but I do use it with some
success... vinum takes an entire drive offline when it sees an error,
and I use this to scan for and fix errors before turning the drive back
on.
--
... Joe
---
Joe Greco - Systems Administrator [EMAIL PROTECTED]
Solaria Public Access UNIX - Milwaukee, WI 414/342-4847
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: Frustration with SCSI system
Path: news.sol.net!newsops.execpc.com!169.207.30.19.MISMATCH!spool0-nwblwi.newsops.execpc.com!newspump.sol.net!news.execpc.com!newshub.sol.net!sol.net!newspeer.sol.net!ns.sol.net!lists.sol.net!not-for-mail Newsgroups: sol.lists.freebsd.hackers,sol.lists.freebsd.isp Date: 20 Sep 2000 22:02:50 + Sender: [EMAIL PROTECTED] Approved: [EMAIL PROTECTED] Organization: sol.net Network Services - Milwaukee, WI References: [EMAIL PROTECTED] X-To: [EMAIL PROTECTED] X-Cc: [EMAIL PROTECTED], [EMAIL PROTECTED] From: [EMAIL PROTECTED] Subject: Re: Frustration with SCSI system Message-ID: [EMAIL PROTECTED] Xref: news.sol.net sol.lists.freebsd.hackers:21298 sol.lists.freebsd.isp:6352 In message [EMAIL PROTECTED] Edward Elhauge writes: : to autorecover on bad sectors, but every system that I've had to recover : seems to be in a state where the bad sectors aren't remapping. I've tried I've often wanted to write a bad block remapper. While SCSI is supposed to do this automatically, I've found that a scan on any adaptec controller will remap these blocks (forces the remapping). About 10% of the time that's all a drive with this problem needs to survive indefinitely. The other 90% of the time the disk is about to go tits up in a heap big time way and warrantee replacement is recommended. About 75% of the time a rescan + immmediate dump will save me. I've had 2 disks that seem to have lost their bad block mappings that the adaptec verify function has saved me from sending them back (they were out of warantee anyway). However, on the other 20ish disks I've tried this on have died within days of doing this. Even if we had bad144 support, the drive will need so many bad blocks remapped in a short period of time that it isn't worth while. Finally, I've found that climate controlled and dust free environments help a lot. RAID hardware/software is definitely the right way to deal when you go to the next level. Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message -- ... Joe --- Joe Greco - Systems Administrator [EMAIL PROTECTED] Solaria Public Access UNIX - Milwaukee, WI 414/342-4847 To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Frustration with SCSI system
On Thu, Sep 21, 2000 at 05:44:32PM -0600, Warner Losh wrote: In message [EMAIL PROTECTED] Douglas Swarin writes: : Ideally, I would use one of the IDE flash-based drives on the market. One : brand is SanDisk, and they take a standard IDE connector and fit into a : 3.5" drive bay. You can get them very reasonably priced up to 128MB or : so, which is just fine for a boot partition. Since flash drives have no : moving parts, mechanical failure is not an issue, and since the root : partition is not written to much, the flash will not wear out for a : long time (flash cells wear out after about 100,000 writes; the flash : drives do load balancing and stuff to ensure that the (many) cells in : the drive are written to evenly). We use these devices heavily at Timing Solutions. Or rather we use a IDE - CF adapter and haven't had any devices wear out. And some of these devices have had rather heavy use. I think that it is closer to 1 million writes per cell, but I don't have my spec sheets handy. Are you sure that they do write balancing? The indications I have from the base chip technology is that they don't. I could have missed that in the data sheets. It has been a little while since I looked at them, so I might be misremembering. I can't seem to find the data sheets I looked at before. In any event, this works well. I usually have / be read only. This can be practacle if you don't have any users that desire to change their passwords... Since I have serveral machines that have an extremely limited number of users on, this works well. One can also mount / rw if you need to do maintenance on it for whatever reason. I was being pessimistic as to the number of writes. I have heard figures in the range of a million writes per cell as well. The technology is always improving, of course. As to the write balancing, I've seen no hard technical data on it, but I've heard it quoted from many sources. So I may be completely wrong. A brief Google search gives http://www.embedded.com/98/9801spec.htm which mentions that ATA flash can indeed do such balancing about 1/3 of the way down the page (search for 'evenly' in the text). The technical term appears to be 'wear leveling'. But I digress. I'm glad to hear this method validated by someone with much more experience in the field. Doug To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Frustration with SCSI system
* Edward Elhauge [EMAIL PROTECTED] [000920 12:48] wrote: Hello Freebsders, I've been using FreeBSD over the last 6 years (since I switched from NetBSD) to run a small ISP out of my basement. I've had about six disk crashes in as many years and still don't know how to work reliably with them. "man vinum" software mirroring == good. :) -Alfred To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Frustration with SCSI system
On Wed, Sep 20, 2000 at 12:58:27PM -0700, Edward Elhauge wrote: I've been using FreeBSD over the last 6 years (since I switched from NetBSD) to run a small ISP out of my basement. I've had about six disk crashes in as many years and still don't know how to work reliably with them. I have installed UPS boxes on each machine and that seems to have lowered the incidence of failure, but failures still happen; yesterday during our heat wave in the San Francisco area (possible brownouts also) I had another. How hot gets your machine room/basement? OK, so you have to expect these things; but I never seem to find an easy way to recover these systems. The only thing that I've seen work has been to mount the disk on another system, back it up, reformat the drive, copy things back over and find out what was trashed. THERE MUST BE A BETTER WAY. Yes. It is called RAID. Either in hardware or in software, using vinum. It seems like SCSI systems can't use the bad144 program. They are supposed to autorecover on bad sectors, but every system that I've had to recover seems to be in a state where the bad sectors aren't remapping. I've tried buying top of the line hardware, and it does work faster, but no more reliably than the cheap stuff. Once you get in this state it is difficult to mount the partitions so as to recover what is there. There are a finite amount of replacement blocks on each SCSI disk. Once they are given out you are toast. I really need some good advice here. Do I need to buy RAID hardware for each and every server in my network? Is there some way to force the SCSI system to remap bad drives? The error I'm getting is: MEDIUM ERROR info:1010f asc:14,1 Is automatic READ/WRITE remapping enabled on those drives? The real disk gurus (Ken, Justin) will want to know which disk types you have. camcontrol devlist or the dmesg.boot will tell them -- Wilko Bulte [EMAIL PROTECTED] Arnhem, the Netherlands To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Frustration with SCSI system
I've been using FreeBSD over the last 6 years (since I switched from NetBSD) to run a small ISP out of my basement. I've had about six disk crashes in as many years and still don't know how to work reliably with them. "man vinum" software mirroring == good. What would be the incentives to using vinum instead of simply concatenating drives in a RAID-1 array (or whatever) using ccd(4)? To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Frustration with SCSI system
I have installed UPS boxes on each machine and that seems to have lowered the incidence of failure, but failures still happen; yesterday during our heat wave in the San Francisco area (possible brownouts also) I had another. Lots of fans in the cases... I had a fan go out in one of mine just a couple of days ago. It was about 75 or so, pulled the computer apart and almost burned myself on the drive! It was too hot to touch. Luckily it only ran that way for a day or two... but I'd still suspect it from no on and will be moving it down to a less critical system ASAP. I suspect you're seeing similar problems. Rick To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Frustration with SCSI system
OK, vinum is good. But my understanding is that you can't use vinum on your root partition. By Murphy's Law it always seems to be root that gets screwed up. And that also causes the biggest problems because then you have to yank the system apart and find another host disk for booting. Well spoken Alfred Perlstein [EMAIL PROTECTED] wrote: * Edward Elhauge [EMAIL PROTECTED] [000920 12:48] wrote: Hello Freebsders, I've been using FreeBSD over the last 6 years (since I switched from NetBSD) to run a small ISP out of my basement. I've had about six disk crashes in as many years and still don't know how to work reliably with them. "man vinum" software mirroring == good. :) -Alfred -- Edward Elhauge [EMAIL PROTECTED] -- Uncanny Inc., San Francisco "War is like love; it always finds a way." -- Bertold Brecht To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Frustration with SCSI system
On Wed, Sep 20, 2000 at 04:09:13PM -0400, Marc Tardif wrote: I've been using FreeBSD over the last 6 years (since I switched from NetBSD) to run a small ISP out of my basement. I've had about six disk crashes in as many years and still don't know how to work reliably with them. "man vinum" software mirroring == good. What would be the incentives to using vinum instead of simply concatenating drives in a RAID-1 array (or whatever) using ccd(4)? ?? Concatenating != mirroring aka RAID-1 -- Wilko Bulte [EMAIL PROTECTED] Arnhem, the Netherlands To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Frustration with SCSI system
On Wed, 20 Sep 2000, Marc Tardif wrote: : I've been using FreeBSD over the last 6 years (since I switched from : NetBSD) to run a small ISP out of my basement. : : I've had about six disk crashes in as many years and still don't know how : to work reliably with them. : : "man vinum" : : software mirroring == good. : :What would be the incentives to using vinum instead of simply :concatenating drives in a RAID-1 array (or whatever) using ccd(4)? Concatenating drives is a decidedly different thing than mirroring them is. Mirroring allows you to recover from failed disks without having to restore from tape. David To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Frustration with SCSI system
If you're willing to go through strange install contortions, you can boot off of an MFS (Or MD, depending on what version you use ) root filesystem (copies stored in separate partition, on both disks you are mirroring) and then have everything else mirrored. Then at least your running system doesn't rely on any unmirrored disks. You can put a minimal root filesystem in memory and then have the rest of the stuff moved out via either symlinks or via just moving things around. Ie if you want to set up a bunch of boxes, it's nice to have most of the rc bootup stuff really reside on the vinum'd user... At any rate, from what I understand, it's the boot loader that doesn't understand vinum disks. put a kernel and minimum stuff somewhere the bootloader can find it and then mirror the rest. If one of the disks goes out, things keep working (root in memory, usr mirrored) and if you have to reboot, just make sure the good disk is the preferred boot device in your scsi bios. On Wed, 20 Sep 2000, Edward Elhauge wrote: OK, vinum is good. But my understanding is that you can't use vinum on your root partition. By Murphy's Law it always seems to be root that gets screwed up. And that also causes the biggest problems because then you have to yank the system apart and find another host disk for booting. Well spoken Alfred Perlstein [EMAIL PROTECTED] wrote: * Edward Elhauge [EMAIL PROTECTED] [000920 12:48] wrote: Hello Freebsders, I've been using FreeBSD over the last 6 years (since I switched from NetBSD) to run a small ISP out of my basement. I've had about six disk crashes in as many years and still don't know how to work reliably with them. "man vinum" software mirroring == good. :) -Alfred -- Edward Elhauge [EMAIL PROTECTED] -- Uncanny Inc., San Francisco "War is like love; it always finds a way." -- Bertold Brecht To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message -- Fred Clift - [EMAIL PROTECTED] -- Remember: If brute force doesn't work, you're just not using enough. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Frustration with SCSI system
* Marc Tardif [EMAIL PROTECTED] [000920 13:06] wrote: I've been using FreeBSD over the last 6 years (since I switched from NetBSD) to run a small ISP out of my basement. I've had about six disk crashes in as many years and still don't know how to work reliably with them. "man vinum" software mirroring == good. What would be the incentives to using vinum instead of simply concatenating drives in a RAID-1 array (or whatever) using ccd(4)? vinum can rebuild mirrored disks on the fly if you go with hotswap. -Alfred To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Frustration with SCSI system
On Wed, Sep 20, 2000 at 01:24:34PM -0700, Edward Elhauge wrote: OK, vinum is good. But my understanding is that you can't use vinum on your root partition. By Murphy's Law it always seems to be root that gets screwed up. And that also causes the biggest problems because then you have to yank the system apart and find another host disk for booting. The root Filesytems doesn't really change. In general if you don't edit something in /etc or add accounts you can even be happy with a readonly /. I can't see any needs for mirrors beside that the host should keep running in case of a disk failure. A simple backup of the small /etc is sufficient. You don't have to find a new / disk if you already created one. A mirror would allocate it anyway. Nevertheless Greg showed me an df output with a mirrored / filesystem but as I usually have stone old drives (90M drives are more than enough ;) for / I never tried it myself because I don't own 2 identic of them. I don't know if it was plain vinum or some magic. -- B.Walter COSMO-Project http://www.cosmo-project.de [EMAIL PROTECTED] Usergroup [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Frustration with SCSI system
On Wed, 20 Sep 2000, Alfred Perlstein wrote: * Edward Elhauge [EMAIL PROTECTED] [000920 12:48] wrote: Hello Freebsders, I've been using FreeBSD over the last 6 years (since I switched from NetBSD) to run a small ISP out of my basement. I've had about six disk crashes in as many years and still don't know how to work reliably with them. "man vinum" software mirroring == good. The question should be, "How much you want to spend?" Depending on how you answer that question, you could choose either software or hardware RAID. I've always had better luck with hardware RAID cards compared to software RAID's. ALthough vinum sounds like a great package, I have little experience with it...only ccd, which is why I went with a hardware solution. If you got money, get a RAID controller (supported by FreeBSD). Then you don't have the root limitation that comes with vinum. If you don't have money, use vinum. Either way, use RAID. Best of luck. Nick Rogness - Drive defensively. Buy a tank. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Frustration with SCSI system
Keep your disk cool. If you're getting MEDIUM errors, you're disks are getting toasted. I'm also in SF, and I plain mostly have been shut down the last two days. Hello Freebsders, I've been using FreeBSD over the last 6 years (since I switched from NetBSD) to run a small ISP out of my basement. I've had about six disk crashes in as many years and still don't know how to work reliably with them. I have installed UPS boxes on each machine and that seems to have lowered the incidence of failure, but failures still happen; yesterday during our heat wave in the San Francisco area (possible brownouts also) I had another. OK, so you have to expect these things; but I never seem to find an easy way to recover these systems. The only thing that I've seen work has been to mount the disk on another system, back it up, reformat the drive, copy things back over and find out what was trashed. THERE MUST BE A BETTER WAY. It seems like SCSI systems can't use the bad144 program. They are supposed to autorecover on bad sectors, but every system that I've had to recover seems to be in a state where the bad sectors aren't remapping. I've tried buying top of the line hardware, and it does work faster, but no more reliably than the cheap stuff. Once you get in this state it is difficult to mount the partitions so as to recover what is there. I really need some good advice here. Do I need to buy RAID hardware for each and every server in my network? Is there some way to force the SCSI system to remap bad drives? The error I'm getting is: MEDIUM ERROR info:1010f asc:14,1 My configuration is: 1) Pentium-S 100 Mz 2) 128M 3) Adaptec 2940 Ultra/Ultra SCSI with Bios 1.25 4) Seagate ST3437 4GB 5) FreeBSD 3.4 Any advice on how to efficiently bring my server back up or how I can reengineer my system to avoid this in the future, will be greatly appreciated. -- Edward Elhauge [EMAIL PROTECTED] -- Uncanny Inc., San Francisco "War is like love; it always finds a way." -- Bertold Brecht To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
mirroring / (was Re: Frustration with SCSI system)
"Edward" == Edward Elhauge [EMAIL PROTECTED] writes: Edward OK, vinum is good. But my understanding is that you can't Edward use vinum on your root partition. By Murphy's Law it Edward always seems to be root that gets screwed up. And that Edward also causes the biggest problems because then you have to Edward yank the system apart and find another host disk for Edward booting. Keep a spare root partition on another disk. If both disks (real root and spare root) have the same geometry, and you allocated identically sized root filesystems, you can dd the live root partition to the spare from cron. If they aren't the same size you would need to keep the spare mounted and do a 'dump ... | restore ...' instead. Alternatively, forgo all the cron-based sync magic (which *is* a bit dodgy when you're copying a live filesystem) and just manually sync the spare root to the real root whenever you make significant changes (new kernel, password file updates, etc). --lyndon To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Frustration with SCSI system
On Wed, Sep 20, 2000 at 01:48:06PM -0700, Alfred Perlstein wrote: * Bernd Walter [EMAIL PROTECTED] [000920 13:43] wrote: On Wed, Sep 20, 2000 at 01:24:34PM -0700, Edward Elhauge wrote: OK, vinum is good. But my understanding is that you can't use vinum on your root partition. By Murphy's Law it always seems to be root that gets screwed up. And that also causes the biggest problems because then you have to yank the system apart and find another host disk for booting. The root Filesytems doesn't really change. In general if you don't edit something in /etc or add accounts you can even be happy with a readonly /. I can't see any needs for mirrors beside that the host should keep running in case of a disk failure. A simple backup of the small /etc is sufficient. There's a "Zen of backup" somewhere that explains why you want to backup system files. Basically: hotswap spare read-only / and /usr == near 0 downtime, reinstall the OS == 20-120 minutes of downtime. There is basicly no big difference in extracting the bin tgzs from the distribution compared to restoring it from a backup but the extra space won't hurt. Of course you need to have the binaries somewhere. Your numbers seem to be realistic to me but the restore case is missing here. Comparing a reinstall with a hotspare solution doesn't show a difference on wether to backup system files or not. And I was talking about / - That means for me no /var or /usr which can be mirrored (What doesn't mean that backups are not sensefull). -- B.Walter COSMO-Project http://www.cosmo-project.de [EMAIL PROTECTED] Usergroup [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Frustration with SCSI system
Edward Elhauge writes: I've been using FreeBSD over the last 6 years (since I switched from NetBSD) to run a small ISP out of my basement. I've had about six disk crashes in as many years and still don't know how to work reliably with them. I have installed UPS boxes on each machine and that seems to have lowered the incidence of failure, but failures still happen; yesterday during our heat wave in the San Francisco area (possible brownouts also) I had another. OK, so you have to expect these things; but I never seem to find an easy way to recover these systems. The only thing that I've seen work has been to mount the disk on another system, back it up, reformat the drive, copy things back over and find out what was trashed. THERE MUST BE A BETTER WAY. It seems like SCSI systems can't use the bad144 program. They are supposed to autorecover on bad sectors, but every system that I've had to recover seems to be in a state where the bad sectors aren't remapping. I've tried buying top of the line hardware, and it does work faster, but no more reliably than the cheap stuff. Once you get in this state it is difficult to mount the partitions so as to recover what is there. I really need some good advice here. Do I need to buy RAID hardware for each and every server in my network? Is there some way to force the SCSI system to remap bad drives? The error I'm getting is: MEDIUM ERROR info:1010f asc:14,1 My configuration is: 1) Pentium-S 100 Mz 2) 128M 3) Adaptec 2940 Ultra/Ultra SCSI with Bios 1.25 4) Seagate ST3437 4GB 5) FreeBSD 3.4 Any advice on how to efficiently bring my server back up or how I can reengineer my system to avoid this in the future, will be greatly appreciated. I work since 1991 with computer hardware and know exact that SCSI drives is about ten times less reliability than IDE. Yes, I understand that SCSI was more ... extremal may be. I am wery glad that now mostly no need in SCSI drives at all. Just use good IDE drives, may be second root and regular dumps to, for example DDS-4 strimer. It is cost effective. If time to repare is critical, use spare IDE disk and special working place with direct crossover ethernet to dump server in addition to common ethernet. This helps you to paralelise repare work and do fast restore for low additional cost. Do not forget reblock dumps when write to tape. If your system is old enough - use Promise UDMA/66 controller. About temperature you already read. Some modern IDE drives has low power consumption and fast enough. If you need in big space - use RAID with IDE disks and SCSI external interface. Remember that most PCI are only 130 MB/sec wide. AND REMOVE FreeBSD 3.X AT ALL!!! 4.X is far more stable. It is easy, not expansive and works good. Sorry my English is not so good as I want. -- @BABOLO http://links.ru/ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Frustration with SCSI system
In message [EMAIL PROTECTED] Edward Elhauge writes: : to autorecover on bad sectors, but every system that I've had to recover : seems to be in a state where the bad sectors aren't remapping. I've tried I've often wanted to write a bad block remapper. While SCSI is supposed to do this automatically, I've found that a scan on any adaptec controller will remap these blocks (forces the remapping). About 10% of the time that's all a drive with this problem needs to survive indefinitely. The other 90% of the time the disk is about to go tits up in a heap big time way and warrantee replacement is recommended. About 75% of the time a rescan + immmediate dump will save me. I've had 2 disks that seem to have lost their bad block mappings that the adaptec verify function has saved me from sending them back (they were out of warantee anyway). However, on the other 20ish disks I've tried this on have died within days of doing this. Even if we had bad144 support, the drive will need so many bad blocks remapped in a short period of time that it isn't worth while. Finally, I've found that climate controlled and dust free environments help a lot. RAID hardware/software is definitely the right way to deal when you go to the next level. Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Frustration with SCSI system
In message [EMAIL PROTECTED] Marc Tardif writes: : What would be the incentives to using vinum instead of simply : concatenating drives in a RAID-1 array (or whatever) using ccd(4)? RAID-5 now seems to be supported, which lets you take the loss of a single disk more easily. Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Frustration with SCSI system
On Thu, 21 Sep 2000, Aleksandr A.Babaylov wrote: :I work since 1991 with computer hardware and know exact :that SCSI drives is about ten times less reliability than :IDE. Yes, I understand that SCSI was more ... extremal may be. :I am wery glad that now mostly no need in SCSI drives at all. :Just use good IDE drives, may be second root and regular :dumps to, for example DDS-4 strimer. It is cost effective. This is totatlly contrary to my experience. Heck, I've got a fair number of SCSI disks that predate 1991, happily spinning away. SCSI just works, on everything I've ever used it.I've had a occaisonal problems with things like termination. High quality cables and enclosures solve this. I wouldn't let an IDE disk get within thinking distance of machine whose reliability I cared about. David To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Frustration with SCSI system
David Scheidt writes: On Thu, 21 Sep 2000, Aleksandr A.Babaylov wrote: :I work since 1991 with computer hardware and know exact :that SCSI drives is about ten times less reliability than :IDE. Yes, I understand that SCSI was more ... extremal may be. :I am wery glad that now mostly no need in SCSI drives at all. :Just use good IDE drives, may be second root and regular :dumps to, for example DDS-4 strimer. It is cost effective. This is totatlly contrary to my experience. Heck, I've got a fair number of SCSI disks that predate 1991, happily spinning away. SCSI just works, on everything I've ever used it.I've had a occaisonal problems with things like termination. High quality cables and enclosures solve this. I wouldn't let an IDE disk get within thinking distance of machine whose reliability I cared about. Cabling... most of troubles caused by cables for me - it is reason I do not believe external devices. Most of IDE breaks was long ago - last about 3 or 4 years ago. SCSI drives breaks are quite regular - 1 or 2 in at least 5 last years. this is for about 50 SCSI drives near me and about 3 times more IDE drives. This is my expierency - you have another. -- @BABOLO http://links.ru/ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Frustration with SCSI system
Cabling... most of troubles caused by cables for me - it is reason I do not believe external devices. Most of IDE breaks was long ago - last about 3 or 4 years ago. SCSI drives breaks are quite regular - 1 or 2 in at least 5 last years. this is for about 50 SCSI drives near me and about 3 times more IDE drives. This is my expierency - you have another. Which is most likely heat related... that's been my experience anyways... :) In a well cooled system, that needs reliability... SCSI all the way... Rick To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Frustration with SCSI system
"Aleksandr A.Babaylov" wrote: David Scheidt writes: On Thu, 21 Sep 2000, Aleksandr A.Babaylov wrote: :I work since 1991 with computer hardware and know exact :that SCSI drives is about ten times less reliability than :IDE. Yes, I understand that SCSI was more ... extremal may be. :I am wery glad that now mostly no need in SCSI drives at all. :Just use good IDE drives, may be second root and regular :dumps to, for example DDS-4 strimer. It is cost effective. This is totatlly contrary to my experience. Heck, I've got a fair number of SCSI disks that predate 1991, happily spinning away. SCSI just works, on everything I've ever used it.I've had a occaisonal problems with things like termination. High quality cables and enclosures solve this. I wouldn't let an IDE disk get within thinking distance of machine whose reliability I cared about. Cabling... most of troubles caused by cables for me - it is reason I do not believe external devices. Eh, that's something unusual. Unless you buy cheap bad cables or try to solder them by yourself. Most of IDE breaks was long ago - last about 3 or 4 years ago. SCSI drives breaks are quite regular - 1 or 2 in at least 5 last years. this is for about 50 SCSI drives near me and about 3 times more IDE drives. This is my expierency - you have another. Overheating. Newer SCSI disks can be found only in high-performance versions, so they tend to generate more heat and be more sensitive to cooling. Plus if your SCSI disks are used more intensely (and they probably are) this would affect their longevity. Plus different manufacturers have different reliability - if you use Seagate SCSI disks and someone else's IDE then you most certainly will see a lot more SCSI disk failures. -SB, Seagate Hater To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Frustration with SCSI system
I've used various Seagate SCSI drives exclusively in all of my boxes and only had one failure, which I was still able to recover all the data from before replacing it. The first box I built back in '97 had an UW Seagate in it that I bought used, and it was very heavily used for 2 years, and I still have that original configuration for various small projects. Right now I have 4 boxes with Barracuda drives (7200RPM) almost to the ceiling in a closet at one site that temps can get up to 80 degrees F, with a constant fan on them, all doing fine.. I build a couple boxes with Seagate Cheetah's in them, for a guy who has an ISP out of his house. He also has (literally) 12 cats and dogs whose hair ends up in the filters of his servers. They too are heavily used, and have had no problems. It's almost like it goes on luck. ;-) At 08:47 PM 9/20/2000 -0400, Sergey Babkin wrote: "Aleksandr A.Babaylov" wrote: David Scheidt writes: On Thu, 21 Sep 2000, Aleksandr A.Babaylov wrote: :I work since 1991 with computer hardware and know exact :that SCSI drives is about ten times less reliability than :IDE. Yes, I understand that SCSI was more ... extremal may be. :I am wery glad that now mostly no need in SCSI drives at all. :Just use good IDE drives, may be second root and regular :dumps to, for example DDS-4 strimer. It is cost effective. This is totatlly contrary to my experience. Heck, I've got a fair number of SCSI disks that predate 1991, happily spinning away. SCSI just works, on everything I've ever used it.I've had a occaisonal problems with things like termination. High quality cables and enclosures solve this. I wouldn't let an IDE disk get within thinking distance of machine whose reliability I cared about. Cabling... most of troubles caused by cables for me - it is reason I do not believe external devices. Eh, that's something unusual. Unless you buy cheap bad cables or try to solder them by yourself. Most of IDE breaks was long ago - last about 3 or 4 years ago. SCSI drives breaks are quite regular - 1 or 2 in at least 5 last years. this is for about 50 SCSI drives near me and about 3 times more IDE drives. This is my expierency - you have another. Overheating. Newer SCSI disks can be found only in high-performance versions, so they tend to generate more heat and be more sensitive to cooling. Plus if your SCSI disks are used more intensely (and they probably are) this would affect their longevity. Plus different manufacturers have different reliability - if you use Seagate SCSI disks and someone else's IDE then you most certainly will see a lot more SCSI disk failures. -SB, Seagate Hater To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-isp" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Frustration with SCSI system
On Wed, 20 Sep 2000, Sergey Babkin wrote: :Plus different manufacturers have different reliability - :if you use Seagate SCSI disks and someone else's IDE then you most :certainly will see a lot more SCSI disk failures. : :-SB, Seagate Hater : I've had almost a thousand Seagates in service for about a year without a single failure. We've replaced 5 or 6 controllers, and a bunch of cables (when the machines first went into service. The techs are murder at bending pins.). I'm quite impressed. : David To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Frustration with SCSI system
Sergey Babkin writes: "Aleksandr A.Babaylov" wrote: David Scheidt writes: . SCSI just works, on everything I've ever used it.I've had a occaisonal problems with things like termination. High quality cables and enclosures solve this. I wouldn't let an IDE disk get within thinking distance of machine whose reliability I cared about. Cabling... most of troubles caused by cables for me - it is reason I do not believe external devices. Eh, that's something unusual. Unless you buy cheap bad cables or try to solder them by yourself. No, good cable in bad place - after touch time to time some external cables change its state. -- @BABOLO http://links.ru/ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
