Re: File Server: fsck, memory requirements and large disk drives
Thanks for your replies. I learned a lot from them. I probably end up using multiple larger but manageable partitions until I learn enough about ZFS or another alternative. At least I know what issues I currently have. Maybe a manual fsck would be appropriate to see if checking the disks is possible in a reasonable time. Richard Toohey wrote: cpu0: Intel(R) Pentium(R) D CPU 3.00GHz (GenuineIntel 686-class) 3GHz ... real mem = 1071722496 (1022MB) Richard, we envy you. Or better said I should spend some money on decent hardware instead of using old stuff all the time. cpu0: Intel Pentium III (GenuineIntel 686-class, 128KB L2 cache) 698 MHz ... real mem = 535392256 (510MB) avail mem = 509263872 (485MB) ... em0 at pci1 dev 10 function 0 Intel PRO/1000MT (82540EM) rev 0x02: irq 5, ... wd0 at pciide0 channel 0 drive 0: ST3250824AS wd0: 16-sector PIO, LBA48, 238475MB, 488397168 sectors wd0(pciide0:0:0): using BIOS timings, Ultra-DMA mode 6 wd1 at pciide0 channel 1 drive 0: ST31500341AS wd1: 16-sector PIO, LBA48, 1430799MB, 2930277168 sectors wd1(pciide0:1:0): using BIOS timings, Ultra-DMA mode 6 wd2 at pciide1 channel 0 drive 0: ST3120023A wd2: 16-sector PIO, LBA, 114473MB, 234441648 sectors wd2(pciide1:0:0): using PIO mode 4, Ultra-DMA mode 4 wd3 at pciide1 channel 0 drive 1: IC35L060AVVA07-0 wd3: 16-sector PIO, LBA, 58644MB, 120103200 sectors wd3(pciide1:0:1): using PIO mode 4, Ultra-DMA mode 4 wd4 at pciide1 channel 1 drive 0: ST380021A wd4: 16-sector PIO, LBA, 76319MB, 156301488 sectors wd4(pciide1:1:0): using PIO mode 4, Ultra-DMA mode 4 ... spdmem0 at iic0 addr 0x50: 256MB SDRAM non-parity PC133CL2 spdmem1 at iic0 addr 0x51: 256MB SDRAM non-parity PC133CL2 I know that's probably a lot of drives for the standard power supply the desktop computer has. It's certainly a point of failure that my UPS won't take care of.
Re: File Server: fsck, memory requirements and large disk drives
On 2/03/2010, at 1:40 PM, Rob Sheldon wrote: On Sat, 27 Feb 2010 18:19:57 +0100, Claus Niesen cnie...@gmx.net wrote: I'm trying to figure out the best way to setup a home file server. I have a 700MHz Celeron with 512MB RAM (maxed out), a gigabit network adapter and 1.5TB hard drive along with a few smaller ones. Currently it is set up with OpenBSD and samba. The 1.5 TB hard drive is partitioned in three equal partition so I have a chance to pass the fsck if ever needed. This setup works well, except that I have to partition the drive into smaller partitions. I really would like not to be bound by the partition size restriction. But of course I would also like to be able to reboot the server and access the data after a power failure or such. And read-only mode isn't an option either. I've done exactly this -- a large hard drive (1TB in my case) on a puny computer with specs similar to yours. fsck was do-able but extremely painful. In my case, it was also doing software RAID, but still -- it took 26 hours to mount all filesystems after an interruption. Not good. 26 hours!? I have a Dell SC440 with a couple of 500 Gb drives in, and the fsck as part of an upgrade is around 45 minutes to 1 hour per drive. (Figures are completely from memory.) No RAID, or anything like that, just 2 x 500Gb SATA drives. fsck as part of version upgrade, happy drives. cpu0: Intel(R) Pentium(R) D CPU 3.00GHz (GenuineIntel 686-class) 3 GHz ... real mem = 1071722496 (1022MB) avail mem = 1027489792 (979MB) ... wd0 at pciide1 channel 0 drive 0: ST3500630NS wd0: 16-sector PIO, LBA48, 476940MB, 976773168 sectors wd1 at pciide1 channel 0 drive 1: ST3500630NS wd1: 16-sector PIO, LBA48, 476940MB, 976773168 sectors wd0(pciide1:0:0): using PIO mode 4, Ultra-DMA mode 5 wd1(pciide1:0:1): using PIO mode 4, Ultra-DMA mode 5 ichiic0 at pci0 dev 31 function 3 Intel 82801GB SMBus rev 0x01: apic 8 int 17 (irq 10) iic0 at ichiic0 spdmem0 at iic0 addr 0x50: 512MB DDR2 SDRAM ECC PC2-5300CL5 spdmem1 at iic0 addr 0x52: 512MB DDR2 SDRAM ECC PC2-5300CL5 ... An fsck of 1 TB will not necessarily take 26 hours, so definitely YMMV. I know you said your timing was on crappy hardware, so interested in what might be normal for bigger drives on more modern hardware. I've probably completely missed the point, so get the cluesticks out! The only way around this is to (dramatically) change your block size, and then you might end up wasting a significant amount of disk depending on the type of files you're storing on it. As fond as I am of OpenBSD, this just isn't something it's very good at at the moment. (Alternatively, you could install an uninterruptible power supply with a USB connection, set up monitoring in OpenBSD, and have the machine cleanly shut itself down if the battery runs out. But that's kinda just a stopgap solution.) - R. -- [__ Robert Sheldon [__ Founder, No Problem [__ Information technology support and services [__ Software and web design and development [__ (530) 575-0278 [__ You must be the change you wish to see in the world. -- Mahatma Gandhi
Re: File Server: fsck, memory requirements and large disk drives
On Sat, 27 Feb 2010 18:19:57 +0100, Claus Niesen cnie...@gmx.net wrote: I'm trying to figure out the best way to setup a home file server. I have a 700MHz Celeron with 512MB RAM (maxed out), a gigabit network adapter and 1.5TB hard drive along with a few smaller ones. Currently it is set up with OpenBSD and samba. The 1.5 TB hard drive is partitioned in three equal partition so I have a chance to pass the fsck if ever needed. This setup works well, except that I have to partition the drive into smaller partitions. I really would like not to be bound by the partition size restriction. But of course I would also like to be able to reboot the server and access the data after a power failure or such. And read-only mode isn't an option either. I've done exactly this -- a large hard drive (1TB in my case) on a puny computer with specs similar to yours. fsck was do-able but extremely painful. In my case, it was also doing software RAID, but still -- it took 26 hours to mount all filesystems after an interruption. Not good. The only way around this is to (dramatically) change your block size, and then you might end up wasting a significant amount of disk depending on the type of files you're storing on it. As fond as I am of OpenBSD, this just isn't something it's very good at at the moment. (Alternatively, you could install an uninterruptible power supply with a USB connection, set up monitoring in OpenBSD, and have the machine cleanly shut itself down if the battery runs out. But that's kinda just a stopgap solution.) - R. -- [__ Robert Sheldon [__ Founder, No Problem [__ Information technology support and services [__ Software and web design and development [__ (530) 575-0278 [__ You must be the change you wish to see in the world. -- Mahatma Gandhi
Re: File Server: fsck, memory requirements and large disk drives
On Sat, Feb 27, 2010 at 07:12:17PM -0800, Bohdan Tashchuk wrote: The 1.5 TB hard drive is partitioned in three equal partition so I have a chance to pass the fsck if ever needed. You may still have difficulty passing fsck. By default OpenBSD will attempt to fsck all three partitions in parallel. See this thread from last month where I mentioned a change to /etc/rc which will help. http://kerneltrap.org/mailarchive/openbsd-misc/2010/1/5/6318963 Not true. fsck will only do a parallel check the partitions are on a separate device. -Otto
: File Server: fsck, memory requirements and large disk drives
Not true. fsck will only do a parallel check the partitions are on a separate device. That makes sense. You would have a lot of disk thrashing if you tried to check two partitions in parallel on the same drive.
File Server: fsck, memory requirements and large disk drives
I'm trying to figure out the best way to setup a home file server. I have a 700MHz Celeron with 512MB RAM (maxed out), a gigabit network adapter and 1.5TB hard drive along with a few smaller ones. Currently it is set up with OpenBSD and samba. The 1.5 TB hard drive is partitioned in three equal partition so I have a chance to pass the fsck if ever needed. This setup works well, except that I have to partition the drive into smaller partitions. I really would like not to be bound by the partition size restriction. But of course I would also like to be able to reboot the server and access the data after a power failure or such. And read-only mode isn't an option either. Thanks to great documentation of OpenBSD (http://www.openbsd.org/faq/faq14.html#LargeDrive) I know that I'm out of luck with default file system (FFS) on OpenBSD. What I'm not sure about is if a different file system on OpenBSD could alleviate the memory issue. Also, would I encounter the same memory requirements for fsck with other operating systems such as the FreeNAS (FreeBSD)? -- Sicherer, schneller und einfacher. Die aktuellen Internet-Browser - jetzt kostenlos herunterladen! http://portal.gmx.net/de/go/chbrowser
Re: File Server: fsck, memory requirements and large disk drives
The 1.5 TB hard drive is partitioned in three equal partition so I have a chance to pass the fsck if ever needed. You may still have difficulty passing fsck. By default OpenBSD will attempt to fsck all three partitions in parallel. See this thread from last month where I mentioned a change to /etc/rc which will help. http://kerneltrap.org/mailarchive/openbsd-misc/2010/1/5/6318963
Re: File Server: fsck, memory requirements and large disk drives
On Sat, Feb 27, 2010 at 12:19 PM, Claus Niesen cnie...@gmx.net wrote: Thanks to great documentation of OpenBSD (http://www.openbsd.org/faq/faq14.html#LargeDrive) I know that I'm out of luck with default file system (FFS) on OpenBSD. What I'm not sure about is if a different file system on OpenBSD could alleviate the memory issue. Also, would I encounter the same memory requirements for fsck with other operating systems such as the FreeNAS (FreeBSD)? The memory required to check a filesystem is a function of the things (inodes, directories, ...) in the filesystem that need checking. There's no way around that, regardless of filesystem or operating system. You need a (journaled) filesystem that doesn't need checking. OpenBSD doesn't support any.
