Re: Web server Partitions - /tmp
On Sat, 20 Dec 2003, Karsten M. Self wrote: but ... if /tmp accidentally mounted under root fs instead of separate partition, than i consider the box as having gone bonkers and not working right There's not working right (in which case I agree), and there's not working. Point is: the system will be runnable. You can attempt to rectify the problem. You can mount an alternate partition (if available) to /tmp. you usually can't fix the problem after the system is up .. - best way is to back down to single use, and fix /tmp problems so it mounts properly during the next bootup if for example, you're running X11, or mysql ... you will not be able to mount another new partition onto /tmp ... - /tmp is already in use by other apps so you cant unmount it w/o killing x11 or mysql or other apps - any new partitions mounted to new /tmp will hide old /tmp from any prev apps that started and is using /tmp space and more likely that not many people have a spare unused 128MB or so sized partition lying around .. to format and chmod 1777 for fixing the incorrect /tmp yup.. the box is runnable .. just not working right .. c ya alvin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Web server Partitions - /tmp
on Fri, Dec 19, 2003 at 12:32:26PM -0800, Alvin Oga ([EMAIL PROTECTED]) wrote: On Fri, 19 Dec 2003, Karsten M. Self wrote: You left off full attributions: Alvin Oga wrote: if /tmp is a separate partition and it cannot mount it during bootup, nothing will work right if the app depends on /tmp, not just apache Wrong. The mount point directory will exist, and will serve as /tmp. The normally mounted filesystem won't be there. You'll just be using your root FS as /tmp. yes.. you're right ... but ... if /tmp accidentally mounted under root fs instead of separate partition, than i consider the box as having gone bonkers and not working right There's not working right (in which case I agree), and there's not working. Point is: the system will be runnable. You can attempt to rectify the problem. You can mount an alternate partition (if available) to /tmp. ... the box is not running as intended ( /tmp as a separate partition ) and the system will die miserably one day soon and lots of apps that uses /tmp will go (even more) bonkers - guess i should been clearer This is often the case. - bad, unpredictable things happen when / gets full Yes, this is why I strongly recommend a separate /tmp partition. Peace. -- Karsten M. Self [EMAIL PROTECTED]http://kmself.home.netcom.com/ What Part of Gestalt don't you understand? First they came for the Communists, and I didn't speak up, because I wasn't a Communist. Then they came for the Jews, and I didn't speak up, because I wasn't a Jew. Then they came for the Catholics, and I didn't speak up, because I was a Protestant. Then they came for me, and by that time there was no one left to speak up for me. -- Rev. Martin Niemoller, 1945 pgp0.pgp Description: PGP signature
Re: Web server Partitions
hi ya andrew raid can break due to: - (1) disk failures - the silly system takes forever ( dayz ) to resync itself - too many disks failures renders the entire raid useless or the system can be on a non-raided disk and raid5 for data only - have an 2nd system disk for backup and go live by simply changing its ip# and hostname there is no point to raiding /tmp ... - if the system dies ... all temp data in /tmp wont matter - swap is already semi-raided by the kernel and if it dies... swap data is generally useless anyway c ya alvin I was thinking about this idea, so /tmp is on raid. Now temp dies, and you reboot, and now apache won't start? I've decided to start making my raid syncs into smaller sizes, so they can resync back faster. I've found that some volumes just break sync, it's always one disk or partition consistently pukes out, Why is always the same disk/partition? I think I will just make a raid 0 partition for temp, as you mentioned, if the disk dies all the partitions are dead. Have you noticed any syn speed difference with differnt kernels? In related news I finally got debian to boot from a software raid partition as root. Start to finish..yippee. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Web server Partitions
on Tue, Dec 16, 2003 at 01:38:50PM +1000, Braxton Neate ([EMAIL PROTECTED]) wrote: I'm wondering what other people would recommend in the way of partitioning? http://kmself.home.netcom.com/Linux/FAQs/partition.html You mentioned thus their: Mount options typically restrict features of the partition, including whether it can support executables, SUID files, and device files. While content on static partitions can change, it typically doesn't over a normal use cycle, and is only modified during system upgrades. Debian provides options to allow remounting partitions as writable and/or read-only during the upgrade process, see system documentation for more information [Ed: I should be more specific, it's apt related stuff 2001/04/13]. Can you provide more information about this? This sounds like a useful security setup. --Luke -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Web server Partitions
See generally my guide previously posted. Use LVM. this guide: http://kmself.home.netcom.com/Linux/FAQs/partition.html Mentions that you should not use reiserfs as your boot partition? Can you extrapolate, I've used reiserfs as my boot partiton with no problems, and would be interested in what you have encountered: The only significant consideration here was that I'm running Reiserfs on most partitions. There are some issues reading a Reiserfs partition on boot from LILO, so I have one ext2fs partition, mounted read-only, as /boot. In completelly unrelated comments, it seems a lot of people have good documentation spread all over the net, wouldn't it be nice if more of this was consolidated on the debian site? --Luke CS Sysadmin, Montana State University-Bozeman -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Web server Partitions - me
Alvin Oga said: I've decided to start making my raid syncs into smaller sizes, so they can resync back faster. the size of the raid has NOTHING to do with resync faster in general the number of files and data that have to be sync between the degraded raid and the newly inserted disk does make a difference - faster oyu notice a dead/dying raid disk and replace it implies that there is less time wasted in degraded raid mode and minimizes data loss if another disk dies I did not know that. What I meant, was if it is a partition size the resync will occur faster then if it is a giant partition size. I think I'm just going to put spare backup disk in the system. I've found that some volumes just break sync, huh ?? curious .. what and how ?? I have a raid 5 partiton hde,hdf,hdg,hdh. And sometimes hde gets confused and I have to raidhotadd then raidhotadd. Happens every couple of weeks, always the same partition that decides to go on vacation from the raid set. sync is always required for nfs or raid or whatever ther apps that like to have sync specified was not referring to sync mount option. for clarification, i did NOT mention to use raid0 for temp ( /tmp ) - there is also zero point in making /tmp raid0 unless one is doing say some huge GB-sized app that requires lots of GB-sized temp data in /tmp No I thought about doing it. - in general, i do not recommend raid0 ( stripping ), unless you want: - makes a bunch of smaller disks look like one bigger disk - allows you to read data 2x faster if you mirror it ( raid0 across md0 and md1 ( where both mdo0 and md1 is a mirror of md0 ) Good information, to much effort to stripe and mirror, I'll just stay with raid1 or raid5. - differences in raid http://www.1U-Raid5.net/Differences imho ... a properly partitioned and installed linux, for my reasons/purposes is: / 256MB /tmp256MB /var512MB /usr4096MB swap512MB /optrest of disk ( aka /home ) this is what I have used; / 1G /boot 100M swap512MB /optrest of disk ( aka /home ) Should make seperate /tmp /var partitions on my webserver, thanks for reminding me. Good information on disk sizes, setting up new webserver now that I go raid working, decided to go with reiser, now I have some ideas for partitions. -- if you need more disk space for whatever, than move that directory to user area ( /home ) and keep the system clean so you can fix or restore the system whenver needed in a few minutes I use systemimager to rsync upgrades/downloads remotelly in a few minutes. I can do a remote os upgrade download,backup. Works great. other partiton schmes http://www.Linux-1U.net/Partitions Will take a look at it, too much info...my belly is full. Have you noticed any syn speed difference with differnt kernels? nope ... havent tested for it either Anecdotally I noticed that optimized an athlon or similar 2.4.22 kernels on my systems get 14meg, and stock kernels get slower on syncs. But have not scientifically tested it. It appears, in my completelly unsubstantiated opinionthat raid works much faster in later kernels. -- --Luke CS Sysadmin, Montana State University-Bozeman -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Web server Partitions - reiserfs
Alvin Oga said: reiserfs-3.6 w/ linux-2.4.23 seems to work fine ( normally ) now for / and lilo and everything else ( using it daily ) prior versions/combinations i tried failed miserably Have you notices any corruption using reiserfs, does it sync a lot faster on reboots compared to ext3. I've had a 350G ext3 raid5 parttions that take hours to resync if it is shut off accidentally. I was hoping reiser could give me better redundancy. -- --Luke CS Sysadmin, Montana State University-Bozeman -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Web server Partitions - me
On Fri, 19 Dec 2003, Lucas Albers wrote: hi ya andrew raid can break due to: - (1) disk failures - the silly system takes forever ( dayz ) to resync itself - too many disks failures renders the entire raid useless or the system can be on a non-raided disk and raid5 for data only - have an 2nd system disk for backup and go live by simply changing its ip# and hostname there is no point to raiding /tmp ... - if the system dies ... all temp data in /tmp wont matter - swap is already semi-raided by the kernel and if it dies... swap data is generally useless anyway c ya alvin I was thinking about this idea, so /tmp is on raid. Now temp dies, and you reboot, and now apache won't start? if /tmp is a separate partition and it cannot mount it during bootup, nothing will work right if the app depends on /tmp, not just apache /tmp will get into trouble if you improperly powered off I've decided to start making my raid syncs into smaller sizes, so they can resync back faster. the size of the raid has NOTHING to do with resync faster in general the number of files and data that have to be sync between the degraded raid and the newly inserted disk does make a difference - faster oyu notice a dead/dying raid disk and replace it implies that there is less time wasted in degraded raid mode and minimizes data loss if another disk dies I've found that some volumes just break sync, huh ?? curious .. what and how ?? sync is always required for nfs or raid or whatever ther apps that like to have sync specified nothing to do with volumes ?? it's always one disk or partition consistently pukes out, Why is always the same disk/partition? what disk/partition is puking?? how is it puking ??? I think I will just make a raid 0 partition for temp, as you mentioned, if the disk dies all the partitions are dead. for clarification, i did NOT mention to use raid0 for temp ( /tmp ) - there is also zero point in making /tmp raid0 unless one is doing say some huge GB-sized app that requires lots of GB-sized temp data in /tmp - in general, i do not recommend raid0 ( stripping ), unless you want: - makes a bunch of smaller disks look like one bigger disk - allows you to read data 2x faster if you mirror it ( raid0 across md0 and md1 ( where both mdo0 and md1 is a mirror of md0 ) - differences in raid http://www.1U-Raid5.net/Differences imho ... a properly partitioned and installed linux, for my reasons/purposes is: / 256MB /tmp256MB /var512MB /usr4096MB swap512MB /optrest of disk ( aka /home ) -- if you need more disk space for whatever, than move that directory to user area ( /home ) and keep the system clean so you can fix or restore the system whenver needed in a few minutes other partiton schmes http://www.Linux-1U.net/Partitions Have you noticed any syn speed difference with differnt kernels? nope ... havent tested for it either In related news I finally got debian to boot from a software raid partition as root. Start to finish..yippee. congrats c ya alvin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Web server Partitions - reiserfs
On Fri, 19 Dec 2003, Lucas Albers wrote: Mentions that you should not use reiserfs as your boot partition? Can you extrapolate, I've used reiserfs as my boot partiton with no problems, and would be interested in what you have encountered: The only significant consideration here was that I'm running Reiserfs on most partitions. There are some issues reading a Reiserfs partition on boot from LILO, so I have one ext2fs partition, mounted read-only, as /boot. reiserfs-3.6 w/ linux-2.4.23 seems to work fine ( normally ) now for / and lilo and everything else ( using it daily ) prior versions/combinations i tried failed miserably since you have /boot mounted ro, you will not be able to update w/ kernel upgrades c ya alvin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Web server Partitions - me
hi ya lucas On Fri, 19 Dec 2003, Lucas Albers wrote: What I meant, was if it is a partition size the resync will occur faster then if it is a giant partition size. makes no difference ... 100GB of data to sync is 100GB of data no matter how small .. but if you spread 100GB to 20GB each on 5 disks .. that will be 1/5 the time to resync the 100GB of data ( the whole point of raid5 ) but it will be say 2x slower to write the 100GB (20GB) of data to 5 disks I think I'm just going to put spare backup disk in the system. usually simpler to use 1 disk for spare.. as long as everythng fit and you dont have to worry about any config errors I've found that some volumes just break sync, ... I have a raid 5 partiton hde,hdf,hdg,hdh. And sometimes hde gets confused and I have to raidhotadd then raidhotadd. Happens every couple of weeks, always the same partition that decides to go on vacation from the raid set. sounds like hd3 is a dying disk - do you have SMART turned on it ?? - do you check the cables for it ??? make sure it is ata-100 cables instead of the cheap/fat ata-33 cables you should NOT have to do that ... raidhotremove/raidhotadd... - in general, i do not recommend raid0 ( stripping ), unless you want: - makes a bunch of smaller disks look like one bigger disk - allows you to read data 2x faster if you mirror it ( raid0 across md0 and md1 ( where both mdo0 and md1 is a mirror of md0 ) Good information, to much effort to stripe and mirror, I'll just stay with raid1 or raid5. yup... and lots of hidden problkems with mirror too ... - erase importantfile.txt and its almost instantly gone from the mirror ..ooops this is what I have used; / 1G /boot 100M swap512MB /optrest of disk ( aka /home ) Should make seperate /tmp /var partitions on my webserver, thanks for reminding me. /tmp is 100x ( 1000x ) more important than having /boot other partiton schmes http://www.Linux-1U.net/Partitions Will take a look at it, too much info...my belly is full. raiding on top of all that will overflow the full tummy :-0 Anecdotally I noticed that optimized an athlon or similar 2.4.22 kernels on my systems get 14meg, and stock kernels get slower on syncs. yes... you should see faster performance on tuned kernels ... and if you wanna tune it more .. http://www.Linux-1U.net/Tuning - there's some pretty ( partition raid ) pics at the bottom too c ya alvin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Web server Partitions - reiserfs
On Fri, 19 Dec 2003, Lucas Albers wrote: Alvin Oga said: reiserfs-3.6 w/ linux-2.4.23 seems to work fine ( normally ) now for / and lilo and everything else ( using it daily ) prior versions/combinations i tried failed miserably Have you notices any corruption using reiserfs, does it sync a lot faster on reboots compared to ext3. yup... prior to reiserfs-3.6 it always crashed for my test box ... ( amd k6-350 for slackware, redhat, so i didnt even try debian ) I've had a 350G ext3 raid5 parttions that take hours to resync if it is shut off accidentally. yes... dont bother with ext3 if you're lookign at large disks and dont want tospend the time twiddling your thumb while it does its self check I was hoping reiser could give me better redundancy. supposed to be better - i assume yoy dont mean redundancy but that you dont want to wait around for fsck checks of the fs after accidental or silly or testing power off xfs or jfs supposed to be even better for journaling ... c ya alvin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Web server Partitions
on Fri, Dec 19, 2003 at 01:54:28AM -0700, Lucas Albers ([EMAIL PROTECTED]) wrote: See generally my guide previously posted. Use LVM. this guide: http://kmself.home.netcom.com/Linux/FAQs/partition.html Mentions that you should not use reiserfs as your boot partition? Note the date on that article. It's about 2.5 years old, and things have changed some. I should note on the document itself that there's an oupdated version elsewhere. Can you extrapolate, I've used reiserfs as my boot partiton with no problems, and would be interested in what you have encountered: Frankly, I don't recall at present. At the time, IIRC, rieserfs needed a special command or option to leave files in fixed locations or directly addressable. This may have been fixed since. The caution almost certainly doesn't hold. However, reiserfs (as other journaled filesystems) *does* require storage space for the journal file. Which on a smallish root filesystem takes up a significant amount of space (32 MiB, IIRC, for reiserfs). Reiser uses a fixed size journal file, while ext3's is either more proportional to the partition size or is just smaller overall. The only significant consideration here was that I'm running Reiserfs on most partitions. There are some issues reading a Reiserfs partition on boot from LILO, so I have one ext2fs partition, mounted read-only, as /boot. In completelly unrelated comments, it seems a lot of people have good documentation spread all over the net, wouldn't it be nice if more of this was consolidated on the debian site? Well, some of it's worked into standard stuff. My partitioning notes are also included, in part, in the Linux Partitioning HOWTO. My chroot install notes were adapted for use in the Debian Installation Manual. And there's Google. Peace. -- Karsten M. Self [EMAIL PROTECTED]http://kmself.home.netcom.com/ What Part of Gestalt don't you understand? Support the EFF, they support you: http://www.eff.org/ pgp0.pgp Description: PGP signature
Re: Web server Partitions
on Fri, Dec 19, 2003 at 01:48:45AM -0700, Lucas Albers ([EMAIL PROTECTED]) wrote: on Tue, Dec 16, 2003 at 01:38:50PM +1000, Braxton Neate ([EMAIL PROTECTED]) wrote: I'm wondering what other people would recommend in the way of partitioning? http://kmself.home.netcom.com/Linux/FAQs/partition.html You mentioned thus their: Mount options typically restrict features of the partition, including whether it can support executables, SUID files, and device files. While content on static partitions can change, it typically doesn't over a normal use cycle, and is only modified during system upgrades. Debian provides options to allow remounting partitions as writable and/or read-only during the upgrade process, see system documentation for more information [Ed: I should be more specific, it's apt related stuff 2001/04/13]. Can you provide more information about this? See recent discussion on list, or http://twiki.iwethey.org/Main/NixPartitioning Peace. -- Karsten M. Self [EMAIL PROTECTED]http://kmself.home.netcom.com/ What Part of Gestalt don't you understand? LNX-BBC: Bootable GNU/Linux -- Don't leave /home without it. http://www.lnx-bbc.org/ pgp0.pgp Description: PGP signature
Re: Web server Partitions
Karsten M. Self said: However, reiserfs (as other journaled filesystems) *does* require storage space for the journal file. Which on a smallish root filesystem takes up a significant amount of space (32 MiB, IIRC, for reiserfs). Reiser uses a fixed size journal file, while ext3's is either more proportional to the partition size or is just smaller overall. Good information, though; thanks for the feedback, I had not considered why not to use reiserfs on small 100m boot partition. It makes sense to use ext3 on a small boot partition, because of the proportional journal size. Well, some of it's worked into standard stuff. My partitioning notes are also included, in part, in the Linux Partitioning HOWTO. My chroot install notes were adapted for use in the Debian Installation Manual. And there's Google. Adding value to the world, way to go!. (You and google.) -- --Luke CS Sysadmin, Montana State University-Bozeman -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Web server Partitions - me
whatup alvin, Alvin Oga said: I think I'm just going to put spare backup disk in the system. usually simpler to use 1 disk for spare.. as long as everythng fit and you dont have to worry about any config errors I've found that some volumes just break sync, I have a raid 5 partiton hde,hdf,hdg,hdh. And sometimes hde gets confused and I have to raidhotadd then raidhotadd. sounds like hd3 is a dying disk - do you have SMART turned on it ?? - do you check the cables for it ??? make sure it is ata-100 cables instead of the cheap/fat ata-33 cables you should NOT have to do that ... raidhotremove/raidhotadd... That disk has 3 seperate raid volumes on it, and only the one raid volume has problems, the other raid volumes never have problems: See /proc/mdstat: /dev/hde3 has hte problem Personalities : [raid1] [raid5] md2 : active raid5 hde3[0] hdf3[1] hdg3[2] hdh3[3] 358339200 blocks level 5, 64k chunk, algorithm 0 [4/4] [] md1 : active raid5 hde2[0] hdf2[1] hdg2[2] hdh2[3] 1536000 blocks level 5, 64k chunk, algorithm 0 [4/4] [] md0 : active raid1 hde1[0] hdf1[1] hdg1[2] hdh1[3] 102208 blocks [3/3] [UUU] I'm just say hte heck with tuning it, when people complain, I tune. Until then I just use the default. Only tuning because I dump a lot of data off this system, so want fast disk reads. Just remembered, turn on noatime that should speed it up. and if you wanna tune it more .. http://www.Linux-1U.net/Tuning - there's some pretty ( partition raid ) pics at the bottom too alvin -- --Luke CS Sysadmin, Montana State University-Bozeman -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Web server Partitions - reiserfs
Alvin Oga said: I was hoping reiser could give me better redundancy. supposed to be better - i assume yoy dont mean redundancy but that you dont want to wait around for fsck checks of the fs after accidental or silly or testing power off xfs or jfs supposed to be even better for journaling ... yeah, I'll just try one thing at a time. Don't trust xfs/jfs yet. I just want fast fsck time -- --Luke CS Sysadmin, Montana State University-Bozeman -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Web server Partitions - me
on Fri, Dec 19, 2003 at 01:07:47AM -0800, Alvin Oga ([EMAIL PROTECTED]) wrote: On Fri, 19 Dec 2003, Lucas Albers wrote: hi ya andrew raid can break due to: - (1) disk failures - the silly system takes forever ( dayz ) to resync itself - too many disks failures renders the entire raid useless or the system can be on a non-raided disk and raid5 for data only - have an 2nd system disk for backup and go live by simply changing its ip# and hostname there is no point to raiding /tmp ... - if the system dies ... all temp data in /tmp wont matter - swap is already semi-raided by the kernel and if it dies... swap data is generally useless anyway c ya alvin I was thinking about this idea, so /tmp is on raid. Now temp dies, and you reboot, and now apache won't start? if /tmp is a separate partition and it cannot mount it during bootup, nothing will work right if the app depends on /tmp, not just apache Wrong. The mount point directory will exist, and will serve as /tmp. The normally mounted filesystem won't be there. You'll just be using your root FS as /tmp. RAIDing /tmp is generally pretty silly in any regard -- you don't need data reliability, and generally take a performance hit doing this. *Striping* /tmp might be called for. You can park any arbitrary filesystem under /tmp to squeeze by in a pinch anyway. Peace. -- Karsten M. Self [EMAIL PROTECTED]http://kmself.home.netcom.com/ What Part of Gestalt don't you understand? Backgrounder on the Caldera/SCO vs. IBM and Linux dispute. http://sco.iwethey.org/ pgp0.pgp Description: PGP signature
Re: Web server Partitions - /tmp
On Fri, 19 Dec 2003, Karsten M. Self wrote: I was thinking about this idea, so /tmp is on raid. Now temp dies, and you reboot, and now apache won't start? if /tmp is a separate partition and it cannot mount it during bootup, nothing will work right if the app depends on /tmp, not just apache Wrong. The mount point directory will exist, and will serve as /tmp. The normally mounted filesystem won't be there. You'll just be using your root FS as /tmp. yes.. you're right ... but ... if /tmp accidentally mounted under root fs instead of separate partition, than i consider the box as having gone bonkers and not working right ... the box is not running as intended ( /tmp as a separate partition ) and the system will die miserably one day soon and lots of apps that uses /tmp will go (even more) bonkers - guess i should been clearer - bad, unpredictable things happen when / gets full ( when /tmp is used under root's fs ) c ya alvin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Web server Partitions - swap
On Wednesday 17 December 2003 00:55, Alvin Oga wrote: in the old days memory was say 4K total ... Yeah, I was just wondering because on my laptop, complete with KDE desktop and tons of programs running and all, 512MB RAM and something like 270MB of swap, my swap is 99% free. I'm not suggesting to get rid of all swap, I was just thinking that a gig of swap would be kinda wasteful for me. I can't really say anything about servers under heavy load, of course... -- --- Magnus von Koeller --- email:[EMAIL PROTECTED] address: International University Campus 9, App. 13 D-76646 Bruchsal / Germany phone:+49-7251-700-659 mobile: +49-179-4562940 web: http://www.vonkoeller.de pgp0.pgp Description: signature
Re: Web server Partitions - bigmem
On Tue, 16 Dec 2003, Nate Duehr wrote: and if one uses lots of memory space ... one should patch the kernel for bigmem support too hehehe ... sorry.. couldn't resist Uhhh... isn't that what I said? ;-) i donno, i always thought that turning on 4GB or 64GB was different than applying the bigmem patches .. - i always turn on 4GB mem support by default c ya alvin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Web server Partitions
on Tue, Dec 16, 2003 at 10:23:50PM +0100, Magnus von Koeller ([EMAIL PROTECTED]) wrote: Content-Description: signed data On Tuesday 16 December 2003 22:12, Karsten M. Self wrote: SWAP - 1.5GB Rule of thumb: ?1-2x RAM. I never understood that rule... In what way does it make sense that I need more swap because I have more RAM? Seriously, I'd really like to understand this. As several people have pointed out: - Executables themselves aren't swapped. They're mapped to disk (that is, the on-disk binary image is the same as what's in RAM, no need for another image in swap). What's swapped is... - Data segment. Any information that the program holds in memory. Your Squid cache. Browser data. Wordprocessing documents. Anything that's not the executable and/or libs themselves. - Swapping is slow. Or more specifically: *paging* is slow. Reading or writing to swap. However... - The state of being swapped *doesn't* impact performance, _so long as_ that state doesn't change. E.g.: you launch program foo, which loads, but doesn't do much. Later you launch bar and baz. Foo swaps out to make room for baz. There's a slight performance hit as foo is written to disk. But there's _no_ ongoing hit _so long as foo stays swapped_. You can monitor this with 'iostat'. Swap as a memory management strategy works well if: - You have one or more applications which use large amounts of data-segement memory, in aggregate. - These programs don't change active state often. That is: they go into swap and stay there. Say, a word processor you hack on a bit at a time, but leave idle for many minutes, or hours. Swap works poorly if you have little memory on your system, many programs, with data-intensive use, and they move in and out of swap frequently. This is known as thrashing, and is what most people allude to when they say swap is bad. There are a few additional twists: - Linux, unlike some OSs, *doesn't need* swap to function, though you almost always are better off with than without at least some. That is: you can run the system with no swap at all. Of course, this means that when you're out of memory, you're out of memory. Swap is a buffer, insurance. There's no need to balance swap and memory as for some other OSs. - (Possibly supersceded) Too much swap can be detrimental. Or at least this was the case with older kernels. Because memory was necessary to address memory, allocating too much swap actually cut into available real memory. - You can add swap in an emergency either by creating a swapfile, or by activating a previously unused swap partition. Think of swap as a place to put stuff you're not currently working on. If your computer is an office, and your CPU is a desk, then memory is the active desktop where current tasks sit, while the credenza across the room is swap, and the filing cabinet is memory. You can file up the credenza with current projects that you're not actively working on, if your desk gets cluttered. It's not the amount of stuff on the credenza, but the trips back and forth that slow you down. The advantage is you don't have to file everything and start all over next time you want to work on the project. Peace. -- Karsten M. Self [EMAIL PROTECTED]http://kmself.home.netcom.com/ What Part of Gestalt don't you understand? Reject EU Software Patents! http://swpat.ffii.org/ pgp0.pgp Description: PGP signature
Re: Web server Partitions
John Hasler [EMAIL PROTECTED] writes: Bijan Soleymani writes: Many programs are huge but only small parts of them need to be in memory the rest can be maintained in swap. Executables are not swapped out. They don't need to be because they are not altered and so can just be read back in from disk. Not the executable but the program's data. Sorry if I was vague and unclear. Bijan -- Bijan Soleymani [EMAIL PROTECTED] http://www.crasseux.com -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Web server Partitions - swap
on Wed, Dec 17, 2003 at 09:46:38AM +0100, Magnus von Koeller ([EMAIL PROTECTED]) wrote: Content-Description: signed data On Wednesday 17 December 2003 00:55, Alvin Oga wrote: in the old days ?memory was say 4K total ... Yeah, I was just wondering because on my laptop, complete with KDE desktop and tons of programs running and all, 512MB RAM and something like 270MB of swap, my swap is 99% free. I'm not suggesting to get rid of all swap, I was just thinking that a gig of swap would be kinda wasteful for me. I can't really say anything about servers under heavy load, of course... Depends on work habits. I've got 512 MiB RAM, 1 GiB swap configured, 372 MiB swap used. Heavy web browsing (Galeon has memory leaks), image editing, or sound editing can really cut into that. Ditto some database work. Peace. -- Karsten M. Self [EMAIL PROTECTED]http://kmself.home.netcom.com/ What Part of Gestalt don't you understand? What doesn't kill you makes you stranger. -- Karsten M. Self, misreading as usual, San Marcos Pass Rd., 1988 pgp0.pgp Description: PGP signature
RE: Web server Partitions
With that in mind, I would divy up your drive as follows (the following assumes that the server doesn't have any major mail server roles (/var/), that /usr/local/ will be free of anything major, that there's no NFS mounting, and that the server will run a database that will keep things somethere in /lib/, and that /home/ will stay mostly free of general user files. / - 10 GB /home/- 20 GB /lib/ - 2 GB /var/ - 1 GB /tmp/ - 5 GB swap - 2 GB On Tue, 2003-12-16 at 17:52, Braxton Neate wrote: I have separate directories for each website but they are in /var. Is there any particular reason why you have yours in /home? I tend to keep /home strictly for users home directory's. There isn't many users that will have shell access, but my home directory is usually quite large 3=). The plan is that all development of code for websites will be done by a user in there shell account uploaded to CVS and then when it is ready it will be deployed in /var/www or /var/www2 etc. I currently have 1GB of swap space which seams sufficient, 2GB seems a bit excessive. I was told that the rule of thumb is double the amount of physical RAM. My main concern is running out of space in a partition once everything is setup and running. So I want to be sure before I go ahead. It's a shame that there isn't a tool for Linux like Partition Magic. I have always been to freaked out to resize partitions on an existing installation of Linux. What do people think about the following: / - 7GB /usr - 10GB /home - 10GB /var - 10GB /tmp - 1.5GB SWAP - 1.5GB I suspect my requirements are slightly different to yours. About five sites are 'root' sites. That is, sites developed by company staff. The rest (and there's more than a few) are generic websites that we host for various people, varying from simple .html and jpeg types in ~/public_html/ to php/mysql wonders with proper domains. The only other things that appear in home directories are the mbox files for the IMAP servers (something I wish I'd actually thought about). Most people have shell access, but few use it. Given your development method, I see nothing wrong with the way you intend to do. Your partition seems fine, although it's probably not worth having such a large / partition if you are going to separate out /usr/. Without /var, /usr, /home, and /tmp, the entire / partition will probably weigh in at less than 40 MB, unless you have some truly large files in /etc/. Regards Edward -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
RE: Web server Partitions
hi ya braxton On Tue, 16 Dec 2003, Braxton Neate wrote: My main concern is running out of space in a partition once everything is setup and running. So I want to be sure before I go ahead. It's a .. there are many different partition schools of thought ... http://www.Linux-1U.net/Partitions mine is.. - the system must be easily backed up and restored just in case - i backup user stuff system is already backed up on millions of servers - the system must be fixable onsite ( w/o reinstall ) because you forgot to bring that sparedisk, or bring bbc, or bring knoppix - system data is to be kept separate from user stuff - security of the server is more important than convenience to windoze folks wanting to run frontpage ( or equiv ) - users should never run out of space ( other than filling up the disk ) What do people think about the following: / - 7GB / should only be 256MB or less - if the machine dies, only 256MB needs to be clean for you to be able to use the machine to fix itself if its fixable /usr - 10GB /usr/local is the issue, do you lave it in system area or do you symlink to user are /usr/local /home - 10GB should be the whole disk ... ( maximum amount available/left over ) /var - 10GB if you want to save weblogs, you can move it to /home since users want that info system use very little space in /var /tmp - 1.5GB there is zero need to have so much /tmp space ?? 128MB or 256MB is more than enough for 95% of all apps i seen SWAP - 1.5GB add real memory for each 256MB of swapp beng used swap size of 2x real memory was the old rule of thumb when memory was 10x the costs of disks ... 2GB of swap is 10x too slow for a 1GB of real memory way cheaper/faster/easier to add a 2nd 1GB stick of mem -- you'd be painfully slow when using swap c ya alvin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Web server Partitions
Braxton Neate [EMAIL PROTECTED] writes: I know this is a question that gets asked a lot, but googling around I can't seem to find a good answer. I'm re-installing a web/sql server which currently has one large root partition and a swap partition. This is obviously not the best setup. I'm wondering what other people would recommend in the way of partitioning? The server is a 2x 800mhz PIII with 512MB RAM and a 40GB hard drive. It depends on your exact needs. Assuming you have no normal interactive users, I'd probably set it up as /var/www -- big enough, maybe 10-15 GB /var/lib/postgres (or whever) -- big enough, maybe 10-15 GB swap -- 0.5-1 GB / -- Whatever's left On this sort of system, the main benefits you get from partitioning are fault isolation: if something gets confused on your root filesystem, and fsck can't recover it, you haven't lost your data. Alternatively, if you decide to reinstall the system, this partitioning scheme lets you reinstall software but keep data. If you have a substantial amount of built-from-source software, you might also consider a partition for /usr/local for similar reasons. Adding more partitions, in my experience, doesn't make the system more manageable; a common thing to happen is to install the system with a small /var partition and then later realize that using APT is painful because it wants lots of space in /var/cache/apt. Or you make /usr too small, or /tmp too small, and run into problems later on. -- David Maze [EMAIL PROTECTED] http://people.debian.org/~dmaze/ Theoretical politics is interesting. Politicking should be illegal. -- Abra Mitchell -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Web server Partitions
On Tue, 16 Dec 2003 17:12:08 +1300, Edward Murrell wrote: The first thing I'd look at doing is moving the default webpage to a [ Edward's advice on partitioning the web server} I also have a server with 6 SCSI drives and a hardware RAID controller. It will be a web server initially but eventually will be a light-load database and shell server also. Should all the paritions be included into one RAID volume or is there any reason to put some partitions (/tmp? /?) on a non-RAID'ed drive? -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Web server Partitions
Incoming from Alvin Oga: /var - 10GB if you want to save weblogs, you can move it to /home since users want that info system use very little space in /var ... exception being /var/cache/apt/archives. That can be a symlink to somewhere else though. -- Any technology distinguishable from magic is insufficiently advanced. (*) http://www.spots.ab.ca/~keeling - - -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Web server Partitions
hi ya andrew On Tue, 16 Dec 2003, Andrew Malcolmson wrote: On Tue, 16 Dec 2003 17:12:08 +1300, Edward Murrell wrote: The first thing I'd look at doing is moving the default webpage to a [ Edward's advice on partitioning the web server} I also have a server with 6 SCSI drives and a hardware RAID controller. It will be a web server initially but eventually will be a light-load database and shell server also. Should all the paritions be included into one RAID volume or is there any reason to put some partitions (/tmp? /?) on a non-RAID'ed drive? that depends on your ability to re-assemble a broken/dead raid system and still make it work after you fiddled with the broken raid the answer also depends on what is the purpose of the raid system - to protect against 1 disk failure or to increase disk read/write thruputs raid can break due to: - (1) disk failures - the silly system takes forever ( dayz ) to resync itself - too many disks failures renders the entire raid useless - data and system on same raid - something goes bad, you lose both system and data - whether you can boot off raid5 is a trick question of how you built the kernel and a customized initrd ( no different than booting off scsi disks, ( you cant read the scsi disks till you boot a ( scsi-capable kernel - fix initrd to solve the problem - users will do anything and everything to break the system in the name of convenience and better/faster/easier for them ways around it - system should be on raid-mirroring and data on raid5 system should be mirrored and than stripped ( better/easier ) or alternatively stripped and than mirrored or the system can be on a non-raided disk and raid5 for data only - have an 2nd system disk for backup and go live by simply changing its ip# and hostname ( even simpler ) - i prefer to keep system separate from user data (diff disks) there is no point to raiding /tmp ... - if the system dies ... all temp data in /tmp wont matter - swap is already semi-raided by the kernel and if it dies... swap data is generally useless anyway c ya alvin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Web server Partitions
On Tuesday 16 December 2003 05:35 am, Edward Murrell wrote: the IMAP servers (something I wish I'd actually thought about). Most people have shell access, but few use it. Perhaps you should disable their shells (set 'em to /bin/false or whatever you like) but leave their user accounts? Just a thought... -- Nate Duehr, [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Web server Partitions
on Tue, Dec 16, 2003 at 01:38:50PM +1000, Braxton Neate ([EMAIL PROTECTED]) wrote: Hi Everyone, I know this is a question that gets asked a lot, but googling around I can't seem to find a good answer. I'm re-installing a web/sql server which currently has one large root partition and a swap partition. This is obviously not the best setup. I'm wondering what other people would recommend in the way of partitioning? The server is a 2x 800mhz PIII with 512MB RAM and a 40GB hard drive. There will be 3 main users of this system. This box would probably get around 200 hits a day max, but also hosts and internal intranet as well as an external website so there is quite a bit of data in /var/www. http://kmself.home.netcom.com/Linux/FAQs/partition.html Peace. -- Karsten M. Self [EMAIL PROTECTED]http://kmself.home.netcom.com/ What Part of Gestalt don't you understand? In that case, I challenge you to a battle of wits. For the princess? To the death? I accept. - Princess Bride pgp0.pgp Description: PGP signature
Re: Web server Partitions
See generally my guide previously posted. on Tue, Dec 16, 2003 at 02:52:54PM +1000, Braxton Neate ([EMAIL PROTECTED]) wrote: I currently have 1GB of swap space which seams sufficient, 2GB seems a bit excessive. I was told that the rule of thumb is double the amount of physical RAM. My main concern is running out of space in a partition once everything is setup and running. Use LVM. What do people think about the following: / - 7GB /usr - 10GB About 3x overkill /home - 10GB Under served. /var - 10GB 10x overkill. /tmp - 1.5GB 100x overkill. 150-250 MiB /tmp is sufficient for virtually all purposes. SWAP - 1.5GB Rule of thumb: 1-2x RAM. Effectively: create a partition sized to your current RAM. Replicate this to the total system RAM. Mount 1-2 of these partitions. E.g.: - System has 1 GiB RAM and supports 4 GiB RAM: - Create four 1-GiB swap partitions. Mount one or two of these. You'll expand into the remaining two as you add system RAM. Peace. -- Karsten M. Self [EMAIL PROTECTED]http://kmself.home.netcom.com/ What Part of Gestalt don't you understand? TWikIWeThey: An experiment in collective intelligence. Stupidity. Whatever. Technical docs, discussion, reviews, opinion. http://twiki.iwethey.org/ pgp0.pgp Description: PGP signature
Re: Web server Partitions
On Tuesday 16 December 2003 22:12, Karsten M. Self wrote: SWAP - 1.5GB Rule of thumb: 1-2x RAM. I never understood that rule... In what way does it make sense that I need more swap because I have more RAM? Seriously, I'd really like to understand this. -- --- Magnus von Koeller --- email:[EMAIL PROTECTED] address: International University Campus 9, App. 13 D-76646 Bruchsal / Germany phone:+49-7251-700-659 mobile: +49-179-4562940 web: http://www.vonkoeller.de pgp0.pgp Description: signature
Re: Web server Partitions
On Tue, Dec 16, 2003 at 10:23:50PM +0100, Magnus von Koeller wrote: Content-Description: signed data On Tuesday 16 December 2003 22:12, Karsten M. Self wrote: SWAP - 1.5GB Rule of thumb: ?1-2x RAM. I never understood that rule... In what way does it make sense that I need more swap because I have more RAM? Seriously, I'd really like to understand this. Many programs are huge but only small parts of them need to be in memory the rest can be maintained in swap. So you can run more programs than physical ram alone can handle. But if you run too many programs then you'll end up swapping the necessary parts of programs into and out of memory each time the kernel switches processes. At which point performance goes to hell. So there's a point at which more swap (without adding more ram) will be pointless and you'd be better off having the kernel kill processes to free ram. On the flip side if you have more ram you can make effective use of more swap. For example assume you have a large database program that needs 1000 megs of memory but only 100 is used at a time. You can have 128 megs ram and 1.5 gigs swap and this is fine. But if you had two of these running at the same time with 128 megs of ram then the performance is going to be unacceptable no matter how much swap you have. This is because they each need 100 megs of ram (200 total), failing that they keep swapping that 100 megs in and out each time the OS switches from one to the other. Bijan -- Bijan Soleymani [EMAIL PROTECTED] http://www.crasseux.com signature.asc Description: Digital signature
RE: Web server Partitions
David, I tend to agree with you that Adding more partitions, doesn't make the system more manageable. As this is a web server not a shell server with 100's of users. All the important data is in /var. All the important data being websites, logs, MySQL data. The only other important data really is the system configuration which is in /etc but that is restorable from backups. Even home directory's aren't that important, but do get backed up. As unfortunately we are in a mostly windows environment and our file servers and mail servers are Windows2k and Exchange2k. *shivers* One partition I forgot to mention in my post was /boot, a lot of people recommend having the kernel on a separate partition. I'm guessing this wouldn't exceed 10MB but 50MB is probably a safe size. After pondering a few reply's I'm thinking of the following: /boot - 50MB / - 23GB (remembering that /usr /home etc. will be directory's underneath this) /var - 15GB (will contain logs, websites, and SQL data) SWAP1 - 1GB SWAP2 - 1GB Apparently the kernel can balance loads between 2 swap partitions like it can with multiple processors, so it would make sense to have 2 1GB partitions rather than a 1 2GB partition. However this machine rarely swaps. Another question which arises is should I partition the disk in any particular order i.e. does it mater if / is hda1 or hda5? Thanks for everyone's replies! -Braxton -Original Message- From: David Z Maze [mailto:[EMAIL PROTECTED] Sent: Wednesday, 17 December 2003 1:59 AM To: [EMAIL PROTECTED] Subject: Re: Web server Partitions Braxton Neate [EMAIL PROTECTED] writes: I know this is a question that gets asked a lot, but googling around I can't seem to find a good answer. I'm re-installing a web/sql server which currently has one large root partition and a swap partition. This is obviously not the best setup. I'm wondering what other people would recommend in the way of partitioning? The server is a 2x 800mhz PIII with 512MB RAM and a 40GB hard drive. It depends on your exact needs. Assuming you have no normal interactive users, I'd probably set it up as /var/www -- big enough, maybe 10-15 GB /var/lib/postgres (or whever) -- big enough, maybe 10-15 GB swap -- 0.5-1 GB / -- Whatever's left On this sort of system, the main benefits you get from partitioning are fault isolation: if something gets confused on your root filesystem, and fsck can't recover it, you haven't lost your data. Alternatively, if you decide to reinstall the system, this partitioning scheme lets you reinstall software but keep data. If you have a substantial amount of built-from-source software, you might also consider a partition for /usr/local for similar reasons. Adding more partitions, in my experience, doesn't make the system more manageable; a common thing to happen is to install the system with a small /var partition and then later realize that using APT is painful because it wants lots of space in /var/cache/apt. Or you make /usr too small, or /tmp too small, and run into problems later on. -- David Maze [EMAIL PROTECTED] http://people.debian.org/~dmaze/ Theoretical politics is interesting. Politicking should be illegal. -- Abra Mitchell -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
RE: Web server Partitions
On Wed, 17 Dec 2003, Braxton Neate wrote:
After pondering a few reply's I'm thinking of the following:
/boot - 50MB
geez .. that should give you room for about 20-25 kernels
/ - 23GB (remembering that /usr /home etc. will be directory's
underneath this)
hummm ... you're asking for trouble when thngs go bonkers
i'd modify that still... ( to my preferences )
/ - 128/256MB
/usr- 2MB - 4MB
/home - rest of disk thats left over
/tmp- chmod 1777 /tmp -- critical issue ??
( you want this t flag set )
/var - 15GB (will contain logs, websites, and SQL data)
sql and web stuff ( all user stuff ) belongs in /home/{sql, weblogs}
and symlinked from where ever in the defafult system it likes to keep
things
SWAP1 - 1GB
SWAP2 - 1GB
Apparently the kernel can balance loads between 2 swap partitions like
it can with multiple processors, so it would make sense to have 2 1GB
partitions rather than a 1 2GB partition. However this machine rarely
swaps.
hopefully the machine never uses swap ... if it does, add more memory
Another question which arises is should I partition the disk in any
particular order i.e. does it mater if / is hda1 or hda5?
yes ... it makes a big difference ...
will we do the work to figure out whether its 5%-10%-25% faster when
partitioned in the right order for the apps that's installed and
which ones runs most often ... we probably wont bother testing
the different partition schemes ( its somebody else's phd thesis )
c ya
alvin
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Web server Partitions - swap
On Tue, 16 Dec 2003, Magnus von Koeller wrote: On Tuesday 16 December 2003 22:12, Karsten M. Self wrote: SWAP - 1.5GB Rule of thumb: 1-2x RAM. I never understood that rule... In what way does it make sense that I need more swap because I have more RAM? Seriously, I'd really like to understand this. in the old days memory was say 4K total ... but the programs sizes w/ data totaled at around say 8K or 16K.. - some folks went around, when building the boxes with 100K or 1MB disks, to allow for 2x of memory for swap of the total memory that the machine was capable of holding ( ie .. your program grew bigger, it meant you get ( a bigger machine and more memory -- you learned to write nice ( and clean and compact code 30 years later, its easier/cheaper to just add a new stick of memory - having some swap prevents your system from doing a random self-reboot or hanging forever whenever it runs out of virtual memory - simple test ... turn off swapp swapoff /dev/hda[your-swap-partition] - do indefinite memory allocation ... till it dies ( it should die at around near your real memory size ) - its a small itty-bitty program ... but it chews up lots of simulated data that resides in memory - the above assumes you dont have a swap file nor swap partition - lots of embedded boxes does not have swap space ... since it knows what kind of apps it'd be running c ya alvin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
RE: Web server Partitions - simple partitions
hi ya braxton On Wed, 17 Dec 2003, Braxton Neate wrote: I tend to agree with you that Adding more partitions, doesn't make the system more manageable. yup.. it does make it more manageable per se but it does allow you the luxury of fixing the system if ti goes bonkers by having / be 128MB or 256MB instead of lots-n-lots of disk that has to be clean before it can boot itself One partition I forgot to mention in my post was /boot, a lot of people recommend having the kernel on a separate partition. I'm guessing this wouldn't exceed 10MB but 50MB is probably a safe size. the simplest install ... ( relatively sane way ) swap partition of some size rest of disk -- no other partitions -- -- why bother w/ partitions -- if one wants it simple -- and this 2 partition scheme usually isntall itself by default(?) so no typing/thinking is needed ?? --- and getting back to making things messy: /boot is NOT needed, unless the kernel is about the 1024 cylinder boundry - a left over artifact like swap == 2x real memory ( if one has 4GB of memory, in a large PC, does that mean ( we need 8GB of swap ?? ... nah.. not many apps need that ( much memory other that oracle and cae/cad simulations ( and bunch of special apps and if you did have /boot ... how many 1MB sized kernels does that /boot partition hold ??? 1GB /boot implies about 1,000 kernels +/- :-) ( assuming you dont put anything else there ) if one is wondering about /boot ... it's more important toponder the reasons for /tmp as a separate partition c ta alvin After pondering a few reply's I'm thinking of the following: /boot - 50MB / - 23GB (remembering that /usr /home etc. will be directory's underneath this) /var - 15GB (will contain logs, websites, and SQL data) SWAP1 - 1GB SWAP2 - 1GB Apparently the kernel can balance loads between 2 swap partitions like it can with multiple processors, so it would make sense to have 2 1GB partitions rather than a 1 2GB partition. However this machine rarely swaps. Another question which arises is should I partition the disk in any particular order i.e. does it mater if / is hda1 or hda5? Thanks for everyone's replies! -Braxton -Original Message- From: David Z Maze [mailto:[EMAIL PROTECTED] Sent: Wednesday, 17 December 2003 1:59 AM To: [EMAIL PROTECTED] Subject: Re: Web server Partitions Braxton Neate [EMAIL PROTECTED] writes: I know this is a question that gets asked a lot, but googling around I can't seem to find a good answer. I'm re-installing a web/sql server which currently has one large root partition and a swap partition. This is obviously not the best setup. I'm wondering what other people would recommend in the way of partitioning? The server is a 2x 800mhz PIII with 512MB RAM and a 40GB hard drive. It depends on your exact needs. Assuming you have no normal interactive users, I'd probably set it up as /var/www -- big enough, maybe 10-15 GB /var/lib/postgres (or whever) -- big enough, maybe 10-15 GB swap -- 0.5-1 GB / -- Whatever's left On this sort of system, the main benefits you get from partitioning are fault isolation: if something gets confused on your root filesystem, and fsck can't recover it, you haven't lost your data. Alternatively, if you decide to reinstall the system, this partitioning scheme lets you reinstall software but keep data. If you have a substantial amount of built-from-source software, you might also consider a partition for /usr/local for similar reasons. Adding more partitions, in my experience, doesn't make the system more manageable; a common thing to happen is to install the system with a small /var partition and then later realize that using APT is painful because it wants lots of space in /var/cache/apt. Or you make /usr too small, or /tmp too small, and run into problems later on. -- David Maze [EMAIL PROTECTED] http://people.debian.org/~dmaze/ Theoretical politics is interesting. Politicking should be illegal. -- Abra Mitchell -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Web server Partitions - swap
On Tue, Dec 16, 2003 at 03:55:27PM -0800, Alvin Oga wrote: 30 years later, its easier/cheaper to just add a new stick of memory - having some swap prevents your system from doing a random self-reboot or hanging forever whenever it runs out of virtual memory Don't know about Linux, but in Windows on a Desktop PC with 1.5 GB of Ram, and only 100 mb committed, you'll still see some swap activity. We used to bitch at the devs about that: I'm grossly aproximating the answer but it was something along the lines of being able to make sure the kernel would be able to handle disasters without panic. Lots of people on their workstations these days just put 2GB in the machine and run without swap. It's actually possible to have too much swap (with say 8GB or more of ram) -- it slows things down. Does any of this apply in Linux? -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Web server Partitions
Bijan Soleymani writes: Many programs are huge but only small parts of them need to be in memory the rest can be maintained in swap. Executables are not swapped out. They don't need to be because they are not altered and so can just be read back in from disk. -- John Hasler [EMAIL PROTECTED] Dancing Horse Hill Elmwood, Wisconsin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Web server Partitions
On Tuesday, Dec 16, 2003, at 14:23 America/Denver, Magnus von Koeller wrote: On Tuesday 16 December 2003 22:12, Karsten M. Self wrote: SWAP - 1.5GB Rule of thumb: 1-2x RAM. I never understood that rule... In what way does it make sense that I need more swap because I have more RAM? Seriously, I'd really like to understand this. I'm not sure if the linux kernel has the problems of some legacy Unix's, but I personally have watched Solaris and HP-UX boxes both die unhealthy deaths if there isn't at least an equal amount of swap to physical RAM set up on them. At the time, we didn't have time (or test hardware to test on later) to figure out why, we just had to add swap immediately for them to recover. They both needed to swap things out of RAM and couldn't. As soon as swap was added - poof, machine recovered. Both were heavily loaded Oracle servers. Both wanted to swap almost everything out. And yeah, both needed to be on bigger boxes with more physical RAM. The joys of outgrowing your servers... heh. Nate, [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Web server Partitions
On Tuesday, Dec 16, 2003, at 16:23 America/Denver, Braxton Neate wrote: SWAP1 - 1GB SWAP2 - 1GB Apparently the kernel can balance loads between 2 swap partitions like it can with multiple processors, so it would make sense to have 2 1GB partitions rather than a 1 2GB partition. However this machine rarely swaps. Having both swap partitions on the same drive and drive controller won't buy you much, but having two of them on completely separate drives and controllers (say a multi-controller SCSI system, rare, but hey maybe you have such a beast) allows for somewhat better performance as long as the kernel chooses accurately which disk is quieter when it goes looking for swap space. I've never looked at the swap code nor tested this, but it stands to reason that two swaps on the same drive and controller shouldn't buy you much of anything in the way of performance. Nate, [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Web server Partitions - simple partitions
On Tuesday, Dec 16, 2003, at 17:10 America/Denver, Alvin Oga wrote: ( if one has 4GB of memory, in a large PC, does that mean ( we need 8GB of swap ?? ... nah.. not many apps need that ( much memory other that oracle and cae/cad simulations ( and bunch of special apps If one has more than 2GB of physical memory, one needs to recompile the kernel with high memory support for optimal operation. Nate, [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Web server Partitions - bigmem
hi ya nate On Tue, 16 Dec 2003, Nate Duehr wrote: On Tuesday, Dec 16, 2003, at 17:10 America/Denver, Alvin Oga wrote: ( if one has 4GB of memory, in a large PC, does that mean ( we need 8GB of swap ?? ... nah.. not many apps need that ( much memory other that oracle and cae/cad simulations ( and bunch of special apps If one has more than 2GB of physical memory, one needs to recompile the kernel with high memory support for optimal operation. and if one uses lots of memory space ... one should patch the kernel for bigmem support too hehehe ... sorry.. couldn't resist c ya alvin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Web server Partitions - bigmem
On Tuesday, Dec 16, 2003, at 20:02 America/Denver, Alvin Oga wrote: and if one uses lots of memory space ... one should patch the kernel for bigmem support too hehehe ... sorry.. couldn't resist Uhhh... isn't that what I said? ;-) Man, must be tired tonight... Nate -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Web server Partitions
Hi Everyone, I know this is a question that gets asked a lot, but googling around I can't seem to find a good answer. I'm re-installing a web/sql server which currently has one large root partition and a swap partition. This is obviously not the best setup. I'm wondering what other people would recommend in the way of partitioning? The server is a 2x 800mhz PIII with 512MB RAM and a 40GB hard drive. There will be 3 main users of this system. This box would probably get around 200 hits a day max, but also hosts and internal intranet as well as an external website so there is quite a bit of data in /var/www. -Braxton Neate -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Web server Partitions
On Tue, 2003-12-16 at 16:38, Braxton Neate wrote: Hi Everyone, I know this is a question that gets asked a lot, but googling around I can't seem to find a good answer. I'm re-installing a web/sql server which currently has one large root partition and a swap partition. This is obviously not the best setup. I'm wondering what other people would recommend in the way of partitioning? The server is a 2x 800mhz PIII with 512MB RAM and a 40GB hard drive. There will be 3 main users of this system. This box would probably get around 200 hits a day max, but also hosts and internal intranet as well as an external website so there is quite a bit of data in /var/www. -Braxton Neate The first thing I'd look at doing is moving the default webpage to a directory in /home/. In the case of my companies webserver, there's /home/mcnz/ directory, with various company sites in directories off that. People who have admin rights to the website accounts are part of the webedit group. With that in mind, I would divy up your drive as follows (the following assumes that the server doesn't have any major mail server roles (/var/), that /usr/local/ will be free of anything major, that there's no NFS mounting, and that the server will run a database that will keep things somethere in /lib/, and that /home/ will stay mostly free of general user files. / - 10 GB /home/ - 20 GB /lib/ - 2 GB /var/ - 1 GB /tmp/ - 5 GB swap- 2 GB The reasoning behind large swap, /tmp/, /var/, and /lib/ file systems that are partition off, is due to Joe Random Webdeveloper doing some creative web developing on the server. At some point, someone is likely to try and do something funky like try and run licq through php, and chew through several gigs of storage of infinitely recursive log files. (Yes, I had something like this a few years ago, thankfully, it was on a test server.) I have a webserver with a similar setup, and the swap goes unused most of the time, but with anything where people have shell access, eventually someone does something stupid, and you end up being thankful for that extra memory space. The reason for /tmp/ comes from various things that write there, that don't always clean up properly, and things that put files there before doing something to them, and that happening multiple times. One of my users once had a stats package that chewed through 1500 MB of data in about two hours, and then would condense it down to less than 100 kb of text. It took me about two weeks to discover why users would complain of running out of space, but whenever I got the email, they would have more than enough. At the end of the day, partitioning the drives up seems to be a case of not so much what's going to work, but what's going to go wrong. Regards Edward -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
RE: Web server Partitions
I have separate directories for each website but they are in /var. Is there any particular reason why you have yours in /home? I tend to keep /home strictly for users home directory's. There isn't many users that will have shell access, but my home directory is usually quite large 3=). The plan is that all development of code for websites will be done by a user in there shell account uploaded to CVS and then when it is ready it will be deployed in /var/www or /var/www2 etc. I currently have 1GB of swap space which seams sufficient, 2GB seems a bit excessive. I was told that the rule of thumb is double the amount of physical RAM. My main concern is running out of space in a partition once everything is setup and running. So I want to be sure before I go ahead. It's a shame that there isn't a tool for Linux like Partition Magic. I have always been to freaked out to resize partitions on an existing installation of Linux. What do people think about the following: / - 7GB /usr - 10GB /home - 10GB /var - 10GB /tmp - 1.5GB SWAP - 1.5GB -Braxton -Original Message- From: Edward Murrell [mailto:[EMAIL PROTECTED] Sent: Tuesday, 16 December 2003 2:12 PM To: [EMAIL PROTECTED] Subject: Re: Web server Partitions On Tue, 2003-12-16 at 16:38, Braxton Neate wrote: Hi Everyone, I know this is a question that gets asked a lot, but googling around I can't seem to find a good answer. I'm re-installing a web/sql server which currently has one large root partition and a swap partition. This is obviously not the best setup. I'm wondering what other people would recommend in the way of partitioning? The server is a 2x 800mhz PIII with 512MB RAM and a 40GB hard drive. There will be 3 main users of this system. This box would probably get around 200 hits a day max, but also hosts and internal intranet as well as an external website so there is quite a bit of data in /var/www. -Braxton Neate The first thing I'd look at doing is moving the default webpage to a directory in /home/. In the case of my companies webserver, there's /home/mcnz/ directory, with various company sites in directories off that. People who have admin rights to the website accounts are part of the webedit group. With that in mind, I would divy up your drive as follows (the following assumes that the server doesn't have any major mail server roles (/var/), that /usr/local/ will be free of anything major, that there's no NFS mounting, and that the server will run a database that will keep things somethere in /lib/, and that /home/ will stay mostly free of general user files. / - 10 GB /home/ - 20 GB /lib/ - 2 GB /var/ - 1 GB /tmp/ - 5 GB swap- 2 GB The reasoning behind large swap, /tmp/, /var/, and /lib/ file systems that are partition off, is due to Joe Random Webdeveloper doing some creative web developing on the server. At some point, someone is likely to try and do something funky like try and run licq through php, and chew through several gigs of storage of infinitely recursive log files. (Yes, I had something like this a few years ago, thankfully, it was on a test server.) I have a webserver with a similar setup, and the swap goes unused most of the time, but with anything where people have shell access, eventually someone does something stupid, and you end up being thankful for that extra memory space. The reason for /tmp/ comes from various things that write there, that don't always clean up properly, and things that put files there before doing something to them, and that happening multiple times. One of my users once had a stats package that chewed through 1500 MB of data in about two hours, and then would condense it down to less than 100 kb of text. It took me about two weeks to discover why users would complain of running out of space, but whenever I got the email, they would have more than enough. At the end of the day, partitioning the drives up seems to be a case of not so much what's going to work, but what's going to go wrong. Regards Edward -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
