Re: [gentoo-user] Filesystem with lowest CPU load, acceptable emerge performance, and stable?
It all comes down to what do you want to prioritize here. If you want minimal downtimes in case that there's a power source failure of any kind, then you probably want ext4 which will give you the fastest fsck times. Or, you might want to check into sqashfs on a flash drive for your rootfs and use whatever else for writable parts (/tmp,/var/log/, etc.), and update only when strictly necessary (GLSAs can probably help you there). After all, as someone else said above, this machine just needs to do one thing, and do it well. If you plan to make stage4/5/6 or whatever the trend is nowadays to name it, you don't even need portage or a toolchain in that box, and having it will only be a security risk since some rootkits comes in the form of a kernel module that needs to be compiled for your specific kernel and architecture (eliminating the kernel sources and the compiler you sort that out from the very root). In any case, the cpu won't be a limiting factor or a bottleneck, whatever your definitive choice shall be. -- Jesús Guerrero Botella
Re: [gentoo-user] Filesystem with lowest CPU load, acceptable emerge performance, and stable?
So, can anyone recommend me a filesystem that fulfills my following needs: Scenario: vFirewall (virtual Firewall) that is going to be deployed at my IaaS Cloud Provider. Disk I/O Characteristic: Occasional writes during 'normal' usage, once-a-week eix-sync + emerge -avuD Priority: Stable (i.e., less chance of corruption), least CPU usage. My Google-Fu seems to indicate either XFS or JFS; what do you think? IMHO a firewall (physical or virtual) is something that fits strictly into the appliance category. It must do only one thing and do it well, with as little complexity and maintenance overhead as possible. Why in the world would anyone want to run gentoo (which among the rest needs portage and a whole compiler stack) -- or for that matter any other full-fledged linux distribution -- on something like that in production is beyond me... That said, XFS and JFS are targeted at completely different use cases and are way too complex for your scenario. Without appropriately-sized hardware I'm not even sure XFS fits in the stable category. Stick to ext3, keeping an eye on the inode count for /usr/portage as the default value on a small partition probably won't be enough. Fs-related CPU usage in a firewall (which has nearly zero disk activity when up and running) is mostly a non-issue unless you need some form of heavy logging or you're doing something wrong. Weekly updates, on the other hand are exposing you to the risk of random breakages and -- if you compile from source -- are going to cost you a serious amount of CPU. My advice would be to limit updates to those fixing known vulnerabilities, and even then compiling somewhere else and doing binary installs would be preferable. andrea
Re: [gentoo-user] Filesystem with lowest CPU load, acceptable emerge performance, and stable?
On Wed, Sep 07, 2011 at 12:26:15AM +0700, Pandu Poluan wrote So, can anyone recommend me a filesystem that fulfills my following needs: Scenario: vFirewall (virtual Firewall) that is going to be deployed at my IaaS Cloud Provider. Disk I/O Characteristic: Occasional writes during 'normal' usage, once-a-week eix-sync + emerge -avuD Priority: Stable (i.e., less chance of corruption), least CPU usage. My Google-Fu seems to indicate either XFS or JFS; what do you think? Try thinking outside the box. Do you really need more than extfs2? That should be the ultimate in low-overhead writing on the device. Another option is to send the log data out on UDP port 514 to be logged on another machine. A cute trick is to have /etc/conf.d/net as follows config_eth0= 192.168.123.2/24 broadcast 192.168.123.255 routes_eth0= default via 192.168.123.254 And then send the log data to the broadcast address 192.168.123.255 UDP port 514. Any computer with the same broadcast address can receive the log data. You can even have multiple computers sending out, and multiple computers receiving. One of the first things an attacker does after compromising a machine is to wipe the logs on that machine to cover his tracks. If the log data goes to multiple different machines, it will be much more difficult to wipe. Another strategy, on the paranoid side, is to have the router sending logs to a machine like 192.168.123.45, and also have a machine on a totally different IP address (e.g. 10.0.0.1) with its NIC set to promiscuous mode, listen for and save the log data. -- Walter Dnes waltd...@waltdnes.org
Re: [gentoo-user] Filesystem with lowest CPU load, acceptable emerge performance, and stable?
On Thu, Sep 8, 2011 at 05:15, kashani kashani-l...@badapple.net wrote: On 9/7/2011 5:25 AM, Pandu Poluan wrote: Well, for all my other servers, I standardized on ext4. Since a vFirewall have to perform lots of packet-juggling, I'd rather dedicate the CPU time to the kernel rather than the HD I/O. Of course, a vFirewall needs to be updated every now and then, but everytime an update is called for, it should not overly tax the CPU and degrade the netfilter framework. Rgds, You are making my point for me, but not realizing the end result of the logic. There isn't any filesystem change that is going to affect CPU usage by more than a few percentage points in the use case you've described. Rsync, portage, and gcc use a massive amount of CPU compared to the amount the filesystem changes will use other than brief points during the rsync. Additionally most benchmarks are testing filesystem throughput and comparing it to CPU. Because disk IO isn't under pressure in your scenario you're unlikely to see the pathological use of CPU that can highlight the differences between filesystems. Gosh, you're right! (And Jesús' reply also remind me). What was I thinking . That said, you have a few reasonable choices. 1. Move to a binary distro 2. Use buildpkg on a clone of this server and only install packages on your Firewall. 3. NFS mount /usr/portage when you need it and dist build on another server 4. Don't upgrade 5. Get a firewall server with more CPU so that it doesn't matter 6. Script a new firewall server install every x months and swap it into place and drop the original server. 7. Some combination of the above. I think I'll do (6). Attach a HD to another VM, install a similar system on that HD (chroot-ed, of course), update that regularly, make a stage5 (or 6 or whatevs) of the (ch)root, then do a 'tar xJf' on the firewall proper. So, a different scenario, then: Sometimes I need to log stuffs (via ULOG) or do a tcpdump. Will JFS give me additional benefit to ext4? Or should I just stick with ext4? Rgds, -- FdS Pandu E Poluan ~ IT Optimizer ~ • LOPSA Member #15248 • Blog : http://pepoluan.tumblr.com • Linked-In : http://id.linkedin.com/in/pepoluan
Re: [gentoo-user] Filesystem with lowest CPU load, acceptable emerge performance, and stable?
On 9/8/2011 12:52 AM, Pandu Poluan wrote: So, a different scenario, then: Sometimes I need to log stuffs (via ULOG) or do a tcpdump. Will JFS give me additional benefit to ext4? Or should I just stick with ext4? Simplest performance gain for appends is to drop atime/dirtime from the file or filesystem. It's a fairly common practice on database servers though the gains are relatively minor. I'm not sure how much it would affect logging, but it would be fairly easy to test. kashani
Re: [gentoo-user] Filesystem with lowest CPU load, acceptable emerge performance, and stable?
Am 06.09.2011 21:24, schrieb James Broadhead: On 6 September 2011 19:55, Permjacov Evgeniy permea...@gmail.com wrote: On 09/06/2011 09:26 PM, Pandu Poluan wrote: Disk I/O Characteristic: Occasional writes during 'normal' usage, once-a-week eix-sync + emerge -avuD Priority: Stable (i.e., less chance of corruption), least CPU usage. You would have to profile this, but I imagine that the best approach would be to compile in a RAM disk and copy. I think that you're probably trying to optimise the wrong part of this problem. As for ext3/ext4, the improvements to fsck alone make ext4 the FS of choice between the two. JB Pandu is building a firewall. Putting a ton of RAM in it just for the sake for system updates is plain overkill and -- depending on his IaaS provider -- pretty expensive. Regards, Florian Philipp signature.asc Description: OpenPGP digital signature
Re: [gentoo-user] Filesystem with lowest CPU load, acceptable emerge performance, and stable?
On Wed, Sep 7, 2011 at 19:06, Florian Philipp li...@binarywings.net wrote: Am 06.09.2011 21:24, schrieb James Broadhead: On 6 September 2011 19:55, Permjacov Evgeniy permea...@gmail.com wrote: On 09/06/2011 09:26 PM, Pandu Poluan wrote: Disk I/O Characteristic: Occasional writes during 'normal' usage, once-a-week eix-sync + emerge -avuD Priority: Stable (i.e., less chance of corruption), least CPU usage. You would have to profile this, but I imagine that the best approach would be to compile in a RAM disk and copy. I think that you're probably trying to optimise the wrong part of this problem. As for ext3/ext4, the improvements to fsck alone make ext4 the FS of choice between the two. JB Pandu is building a firewall. Putting a ton of RAM in it just for the sake for system updates is plain overkill and -- depending on his IaaS provider -- pretty expensive. Indeed. If I need more RAM, they will only sell a complete package of vCPU+RAM+Storage, which will then be merged with my current package. They do offer Storage-only add-on packages, though. But that's beside the point. Rgds, -- FdS Pandu E Poluan ~ IT Optimizer ~ • LOPSA Member #15248 • Blog : http://pepoluan.tumblr.com • Linked-In : http://id.linkedin.com/in/pepoluan
Re: [gentoo-user] Filesystem with lowest CPU load, acceptable emerge performance, and stable?
On Wed, Sep 7, 2011 at 01:15, kashani kashani-l...@badapple.net wrote: On 9/6/2011 10:26 AM, Pandu Poluan wrote: So, can anyone recommend me a filesystem that fulfills my following needs: Scenario: vFirewall (virtual Firewall) that is going to be deployed at my IaaS Cloud Provider. Disk I/O Characteristic: Occasional writes during 'normal' usage, once-a-week eix-sync + emerge -avuD Priority: Stable (i.e., less chance of corruption), least CPU usage. My Google-Fu seems to indicate either XFS or JFS; what do you think? I think it's a useless local optimization for no real world gain which only increases the complexity of your systems. Use the same filesystem you use on all your other servers. Well, for all my other servers, I standardized on ext4. Since a vFirewall have to perform lots of packet-juggling, I'd rather dedicate the CPU time to the kernel rather than the HD I/O. Of course, a vFirewall needs to be updated every now and then, but everytime an update is called for, it should not overly tax the CPU and degrade the netfilter framework. Rgds, -- FdS Pandu E Poluan ~ IT Optimizer ~ • LOPSA Member #15248 • Blog : http://pepoluan.tumblr.com • Linked-In : http://id.linkedin.com/in/pepoluan
Re: [gentoo-user] Filesystem with lowest CPU load, acceptable emerge performance, and stable?
On Wed, Sep 7, 2011 at 02:24, James Broadhead jamesbroadh...@gmail.com wrote: On 6 September 2011 19:55, Permjacov Evgeniy permea...@gmail.com wrote: On 09/06/2011 09:26 PM, Pandu Poluan wrote: Disk I/O Characteristic: Occasional writes during 'normal' usage, once-a-week eix-sync + emerge -avuD Priority: Stable (i.e., less chance of corruption), least CPU usage. You would have to profile this, but I imagine that the best approach would be to compile in a RAM disk and copy. I think that you're probably trying to optimise the wrong part of this problem. Hmmm... that gives me an idea... If I have some free time, I'll experiment with doing an 'emerge -e @world' on the various filesystems, and recording their total time *and* CPU load. Is the `sar` utility good enough to record CPU load? Rgds, -- FdS Pandu E Poluan ~ IT Optimizer ~ • LOPSA Member #15248 • Blog : http://pepoluan.tumblr.com • Linked-In : http://id.linkedin.com/in/pepoluan
Re: [gentoo-user] Filesystem with lowest CPU load, acceptable emerge performance, and stable?
JFS is very soft on cpu usage, and ext4 does a very good job overall. fsck times for ext4 makes it probably the best choice for a server, plus it has more eyes watching over it. In any case, I wanted to call your attention that this might not be the best choice anyway. If you truly want portage to have the minimum possible impact on the performance of your machine(s) you should probably be using other machine(s) to build binary packages, then use these package in the target system(s) (i.e. with emerge -K). -- Jesús Guerrero Botella
Re: [gentoo-user] Filesystem with lowest CPU load, acceptable emerge performance, and stable?
On 9/7/2011 5:25 AM, Pandu Poluan wrote: On Wed, Sep 7, 2011 at 01:15, kashanikashani-l...@badapple.net wrote: On 9/6/2011 10:26 AM, Pandu Poluan wrote: So, can anyone recommend me a filesystem that fulfills my following needs: Scenario: vFirewall (virtual Firewall) that is going to be deployed at my IaaS Cloud Provider. Disk I/O Characteristic: Occasional writes during 'normal' usage, once-a-week eix-sync + emerge -avuD Priority: Stable (i.e., less chance of corruption), least CPU usage. My Google-Fu seems to indicate either XFS or JFS; what do you think? I think it's a useless local optimization for no real world gain which only increases the complexity of your systems. Use the same filesystem you use on all your other servers. Well, for all my other servers, I standardized on ext4. Since a vFirewall have to perform lots of packet-juggling, I'd rather dedicate the CPU time to the kernel rather than the HD I/O. Of course, a vFirewall needs to be updated every now and then, but everytime an update is called for, it should not overly tax the CPU and degrade the netfilter framework. Rgds, You are making my point for me, but not realizing the end result of the logic. There isn't any filesystem change that is going to affect CPU usage by more than a few percentage points in the use case you've described. Rsync, portage, and gcc use a massive amount of CPU compared to the amount the filesystem changes will use other than brief points during the rsync. Additionally most benchmarks are testing filesystem throughput and comparing it to CPU. Because disk IO isn't under pressure in your scenario you're unlikely to see the pathological use of CPU that can highlight the differences between filesystems. That said, you have a few reasonable choices. 1. Move to a binary distro 2. Use buildpkg on a clone of this server and only install packages on your Firewall. 3. NFS mount /usr/portage when you need it and dist build on another server 4. Don't upgrade 5. Get a firewall server with more CPU so that it doesn't matter 6. Script a new firewall server install every x months and swap it into place and drop the original server. 7. Some combination of the above. kashani
[gentoo-user] Filesystem with lowest CPU load, acceptable emerge performance, and stable?
So, can anyone recommend me a filesystem that fulfills my following needs: Scenario: vFirewall (virtual Firewall) that is going to be deployed at my IaaS Cloud Provider. Disk I/O Characteristic: Occasional writes during 'normal' usage, once-a-week eix-sync + emerge -avuD Priority: Stable (i.e., less chance of corruption), least CPU usage. My Google-Fu seems to indicate either XFS or JFS; what do you think? Rgds, -- -- Pandu E Poluan - IT Optimizer My website: http://pandu.poluan.info/
Re: [gentoo-user] Filesystem with lowest CPU load, acceptable emerge performance, and stable?
On 9/6/2011 10:26 AM, Pandu Poluan wrote: So, can anyone recommend me a filesystem that fulfills my following needs: Scenario: vFirewall (virtual Firewall) that is going to be deployed at my IaaS Cloud Provider. Disk I/O Characteristic: Occasional writes during 'normal' usage, once-a-week eix-sync + emerge -avuD Priority: Stable (i.e., less chance of corruption), least CPU usage. My Google-Fu seems to indicate either XFS or JFS; what do you think? I think it's a useless local optimization for no real world gain which only increases the complexity of your systems. Use the same filesystem you use on all your other servers. kashani
Re: [gentoo-user] Filesystem with lowest CPU load, acceptable emerge performance, and stable?
On 09/06/2011 09:26 PM, Pandu Poluan wrote: So, can anyone recommend me a filesystem that fulfills my following needs: Scenario: vFirewall (virtual Firewall) that is going to be deployed at my IaaS Cloud Provider. Disk I/O Characteristic: Occasional writes during 'normal' usage, once-a-week eix-sync + emerge -avuD Priority: Stable (i.e., less chance of corruption), least CPU usage. My Google-Fu seems to indicate either XFS or JFS; what do you think? Rgds, The best fs for emerge is tmpfs on TMP_PORTDIR. I run box with tmpfs on both /var/tmp and /tmp and happy with it -) For fs CPU usage is nothing, IO usage is a real problem and weak point. Thus, you are free to choose any fs with full journaling. ext3 allows full journaling as option, as well as ext4 and ext4 is little faster if tuned properly. JFS/XFS journals metadata only. Remember that journaling makes writes (i.e. emerge) a bit slower.
Re: [gentoo-user] Filesystem with lowest CPU load, acceptable emerge performance, and stable?
On Tue, Sep 6, 2011 at 2:55 PM, Permjacov Evgeniy permea...@gmail.com wrote: On 09/06/2011 09:26 PM, Pandu Poluan wrote: So, can anyone recommend me a filesystem that fulfills my following needs: Scenario: vFirewall (virtual Firewall) that is going to be deployed at my IaaS Cloud Provider. Disk I/O Characteristic: Occasional writes during 'normal' usage, once-a-week eix-sync + emerge -avuD Priority: Stable (i.e., less chance of corruption), least CPU usage. My Google-Fu seems to indicate either XFS or JFS; what do you think? Rgds, The best fs for emerge is tmpfs on TMP_PORTDIR. I run box with tmpfs on both /var/tmp and /tmp and happy with it -) Watch out that some ebuilds can and will fail if you exceed the capacity of your tmpfs. Numerous factors will contribute to the space required by portage during an emerge. -- :wq
Re: [gentoo-user] Filesystem with lowest CPU load, acceptable emerge performance, and stable?
On 6 September 2011 19:55, Permjacov Evgeniy permea...@gmail.com wrote: On 09/06/2011 09:26 PM, Pandu Poluan wrote: Disk I/O Characteristic: Occasional writes during 'normal' usage, once-a-week eix-sync + emerge -avuD Priority: Stable (i.e., less chance of corruption), least CPU usage. You would have to profile this, but I imagine that the best approach would be to compile in a RAM disk and copy. I think that you're probably trying to optimise the wrong part of this problem. As for ext3/ext4, the improvements to fsck alone make ext4 the FS of choice between the two. JB
