Re: Best file system for a busy webserver

[Adam Vande More]

On Thu, Aug 16, 2012 at 10:45 AM, Paul Schmehl wrote:

> Does anyone have any opinions on which file system is best for a busy
> webserver (7 million hits/month)?  Is anyone one system noticeably better
> than any other?
>
> Just curious.  I'm getting ready to setup a new box running FreeBSD 9, and
> since I'm starting from scratch, I'm questioning all my previous
> assumptions.
>

Sounds like you have ample hardware, so I would probably consider ZFS.  You
get a lot of other options with it which simply aren't available or harder
to manage on a UFS system.  Things like data integrity, ZIL/ARC, live
low-cost snapshots, diff'ing the snapshot, transparent compression, etc all
come with ZFS.  Great tools for certain scenarios.  Properly setup, ZFS
RAID functionality will own any hardware raid solution ever presented
because ZFS doesn't rely on a battery for consistency, nor do they provide
most other features stated including integrity oriented ones.  ZFS is
intended to work with raw disk/JBOD. Good controllers are still important,
they simply don't have the knowledge to use them at peak efficiency.

I don't see much benefit to SSD's for this use case.  All the common files
should be in the fs cache which is at least an order of magnitude faster
than flash based memory, and finding enterprise SSD's(preferably SLC) which
obey FLUSH commands appropriately and have a capicitor appropriate to
production use is something more of a crapshoot than traditional SATA/SAS
drives.

All that being said, UFS is fine too.  I use it most often for light VM
installs and where resources are scarce.  However the 2 single biggest ZFS
feature I like are the data integrity and transparent compression are
wonderful which aren't available in UFS.  ZFS snapshots are much more
functional as well and go well w/ zfs send/receive.

-- 
Adam Vande More
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Best file system for a busy webserver

[Wojciech Puchar]

the OCZ Vertex IIIs (About $1/G these days) wired into a *hardware*
RAID controller setup to mirror them.  This gives you blazing speed

just like i would read some popular street PC newspaper.


___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Best file system for a busy webserver

[Wojciech Puchar]

OK I would say there's no pressing reason to consider ZFS for this


another ZFS fanatics. it is about performance.


direction for a filesystem, at 15GB if performance ever becomes a problem a
RAID1 of SSDs with UFS would make it fly probably into the hundreds of hits
per second range.
classic for ZFS and "modern things" fanatics. lots of talk about high end 
hardware nothing about a thread.


___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Best file system for a busy webserver

[Tim Daneliuk]

On 08/16/2012 01:16 PM, Paul Schmehl wrote:


Paul Schmehl  wrote:


Does anyone have any opinions on which file system is best for a busy
webserver (7 million hits/month)?  Is anyone one system noticeably
better  than any other?




With only 15G of data, I'd recommend a pair of 60G SSD drives like
the OCZ Vertex IIIs (About $1/G these days) wired into a *hardware*
RAID controller setup to mirror them.  This gives you blazing speed
and reliability.  If you want to add another drive, you can make it
RAID 5 which - with the right cabinet and mounting hardware - would
give you hotswap capability.

I know people are fond of software RAID but I personally do not
consider this a very high reliability technology unless you're
running true datacenter class hardware with redundant everything
(disk, NIC, fiber ...) and that's probably overkill in this case.
Good RAID controllers are available from a number of manufacturers.
I dunno if FreeBSD supports them, but Rocket has a good reputation
(though I've never used them) as do both Adaptec and LSI.

In any case, a controller plus 3 drives would probably only set you
back in the $500-ish area which seems like a reasonable price point.

Furthermore, depending on the amount of stuff that you're serving
that is static vs. dynamic, you may get benefit from increasing
memory (thereby increasing the likelihood of a cache hit) and increasing
the minimum/threshold values for the number of httpd processing running
all the time.
--

Tim Daneliuk tun...@tundraware.com
PGP Key: http://www.tundraware.com/PGP/

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Best file system for a busy webserver

[Paul Schmehl]

--On August 16, 2012 9:42:30 PM +0100 Steve O'Hara-Smith  
wrote:



I don't even know where to begin.  There's about 15G of data on the
server.


OK I would say there's no pressing reason to consider ZFS for this
purpose. You'd save a bit of time in crash recovery with no fsck going on,
and perhaps the checksum mechanism would give some peace of mind - but
really in 15GB silent corruption is a very slow process - now if it were
15TB ...



Thanks.


last pid: 40369;  load averages:  0.01,  0.03,  0.00
up 104+09:33:44 13:14:49
137 processes: 1 running, 136 sleeping
CPU:  0.7% user,  0.0% nice,  0.1% system,  0.0% interrupt, 99.2% idle
Mem: 229M Active, 6108M Inact, 1056M Wired, 15M Cache, 828M Buf, 514M
Free Swap: 16G Total, 28K Used, 16G Free


OTOH you have plenty of memory lying around doing nothing much
(6108M inactive) so you can easily support ZFS if you want to play with
it's features (the smooth integration of volume management and filesystem
is rather cool).



It's hard, nowadays, to buy a server that's too small for our needs.  Most 
of them are way overspec'd for what this server does.  Which is a nice 
luxury to have.

>

It sounds like you have backups or at least some means of restoring
the site in the event of disaster so that's all good.


Yes, daily, and the servers are always configured in RAID1.


If there was a
pressing need to be able to get back up fairly quickly and easily I'd be
suggesting ZFS in RAID1 with a hot swap bay in which a third disc goes,
attached as a third mirror, periodically split it off the mirror take
it off site, and replace it with the one that's been off site.

There's really nothing here that's pushing you in any particular
direction for a filesystem, at 15GB if performance ever becomes a problem
a RAID1 of SSDs with UFS would make it fly probably into the hundreds of
hits per second range.


Thanks for the input, Steve.  I appreciate it.

--
Paul Schmehl, Senior Infosec Analyst
As if it wasn't already obvious, my opinions
are my own and not those of my employer.
***
"It is as useless to argue with those who have
renounced the use of reason as to administer
medication to the dead." Thomas Jefferson
"There are some ideas so wrong that only a very
intelligent person could believe in them." George Orwell

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Best file system for a busy webserver

[Steve O'Hara-Smith]

On Thu, 16 Aug 2012 13:16:26 -0500
Paul Schmehl  wrote:

> --On August 16, 2012 6:02:57 PM +0100 Steve O'Hara-Smith
>  wrote:
> 
> > On Thu, 16 Aug 2012 10:45:25 -0500
> > Paul Schmehl  wrote:
> >
> >> Does anyone have any opinions on which file system is best for a busy
> >> webserver (7 million hits/month)?  Is anyone one system noticeably
> >> better  than any other?
> >
> > That's an average of about 3 hits per second. If it's static
> > pages then pretty much anything will handle it easily (but please don't
> > use FAT). If it's dynamic then the whole problem is more complex than a
> > simple page rate. If that load is bursty it may make a difference too.
> >
> 
> Thanks for the reply.  It's a combination.  There are many static pages, 
> but there is also a php-mysql forum that generates pages on the fly.  It 
> accounts for about half of the traffic.  I've always used ufs but am 
> wondering if switching to zfs would make sense.
> 
> This stats page might answer some of your questions: 
>  
> Basically traffic is steady but it's busiest in the evenings (US time
> zones)
> 
> > Other considerations may come into play - how big is this
> > filesystem (number of files, maximum number of entries in a directory,
> > volume of data) ? Are there many users needing to be protected from each
> > other ? What about archives ? snapshots ? growth ? churn ? uptime
> > requirements, disaster recovery time ?
> 
> I don't even know where to begin.  There's about 15G of data on the
> server.

OK I would say there's no pressing reason to consider ZFS for this
purpose. You'd save a bit of time in crash recovery with no fsck going on,
and perhaps the checksum mechanism would give some peace of mind - but
really in 15GB silent corruption is a very slow process - now if it were
15TB ...

> last pid: 40369;  load averages:  0.01,  0.03,  0.00 
> up 104+09:33:44 13:14:49
> 137 processes: 1 running, 136 sleeping
> CPU:  0.7% user,  0.0% nice,  0.1% system,  0.0% interrupt, 99.2% idle
> Mem: 229M Active, 6108M Inact, 1056M Wired, 15M Cache, 828M Buf, 514M Free
> Swap: 16G Total, 28K Used, 16G Free

OTOH you have plenty of memory lying around doing nothing much
(6108M inactive) so you can easily support ZFS if you want to play with
it's features (the smooth integration of volume management and filesystem
is rather cool).

> The system is not being stressed.
> 
> If by users, you means shell accounts, there's two, so that's not really
> an issue.

OK so no need for fancy quota schemes then.

> Uptime is not an issue.  The owners have repeatedly said if the site is 
> down for two days they don't care.  (The forum users don't feel that way 
> though!)  We've had one "disaster" (hard drive failure and raid failed 
> while I was on vacation), and it took about 36 hours to get back online, 
> but that was 10 years ago.  The site doesn't go down - it's running on 
> FreeBSD. :-)

It sounds like you have backups or at least some means of restoring
the site in the event of disaster so that's all good. If there was a
pressing need to be able to get back up fairly quickly and easily I'd be
suggesting ZFS in RAID1 with a hot swap bay in which a third disc goes,
attached as a third mirror, periodically split it off the mirror take
it off site, and replace it with the one that's been off site.

There's really nothing here that's pushing you in any particular
direction for a filesystem, at 15GB if performance ever becomes a problem a
RAID1 of SSDs with UFS would make it fly probably into the hundreds of hits
per second range.

-- 
Steve O'Hara-Smith  |   Directable Mirror Arrays
C:>WIN  | A better way to focus the sun
The computer obeys and wins.|licences available see
You lose and Bill collects. |http://www.sohara.org/
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Best file system for a busy webserver

[Robert Huff]

Paul Schmehl writes:

>  >That's an average of about 3 hits per second. If it's static pages
>  > then pretty much anything will handle it easily (but please don't use
>  > FAT). If it's dynamic then the whole problem is more complex than a
>  > simple page rate. If that load is bursty it may make a difference too.
>  >
>  
>  Thanks for the reply.  It's a combination.  There are many static
>  pages, but there is also a php-mysql forum that generates pages
>  on the fly.  It accounts for about half of the traffic.  I've
>  always used ufs but am wondering if switching to zfs would make
>  sense.

ZFS is known to use much more RAM than UFS.  While (from the
'top' below) you have enough ... is that RAM best used for ZFS, or
for something else?


Robert Huff

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Best file system for a busy webserver

[Paul Schmehl]

--On August 16, 2012 6:02:57 PM +0100 Steve O'Hara-Smith  
wrote:



On Thu, 16 Aug 2012 10:45:25 -0500
Paul Schmehl  wrote:


Does anyone have any opinions on which file system is best for a busy
webserver (7 million hits/month)?  Is anyone one system noticeably
better  than any other?


That's an average of about 3 hits per second. If it's static pages
then pretty much anything will handle it easily (but please don't use
FAT). If it's dynamic then the whole problem is more complex than a
simple page rate. If that load is bursty it may make a difference too.



Thanks for the reply.  It's a combination.  There are many static pages, 
but there is also a php-mysql forum that generates pages on the fly.  It 
accounts for about half of the traffic.  I've always used ufs but am 
wondering if switching to zfs would make sense.


This stats page might answer some of your questions: 

Re: Best file system for a busy webserver

[Robert Bonomi]

> Date: Thu, 16 Aug 2012 10:45:25 -0500
> From: Paul Schmehl 
> To: FreeBSD Questions List 
> Subject: Best file system for a busy webserver
>
> Does anyone have any opinions on which file system is best for a busy 
> webserver (7 million hits/month)?  Is anyone one system noticeably better 
> than any other?
>
> Just curious.  I'm getting ready to setup a new box running FreeBSD 9, and 
> since I'm starting from scratch, I'm questioning all my previous 
> assumptions.

"Insufficient data" for a meaningful answer.

A _LOT_ depends on the natue of the pages being served, 

Is the underlying data fairly 'static', or is it being frequently updated?

If 'updated', you need to take into consideration things like 'how often',
'how large', and 'how localized' (in terms of the filesystem structure),
are the updates.

If file access is almost exclusively reads, the filesystem choice doesn't
make much difference O/S 'caching', which occurs above the filesystem level,
will handle the 'most frequently accessed' stuff.

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Best file system for a busy webserver

[Steve O'Hara-Smith]

On Thu, 16 Aug 2012 10:45:25 -0500
Paul Schmehl  wrote:

> Does anyone have any opinions on which file system is best for a busy 
> webserver (7 million hits/month)?  Is anyone one system noticeably better 
> than any other?

That's an average of about 3 hits per second. If it's static pages
then pretty much anything will handle it easily (but please don't use FAT).
If it's dynamic then the whole problem is more complex than a simple page
rate. If that load is bursty it may make a difference too.

Other considerations may come into play - how big is this
filesystem (number of files, maximum number of entries in a directory,
volume of data) ? Are there many users needing to be protected from each
other ? What about archives ? snapshots ? growth ? churn ? uptime
requirements, disaster recovery time ?

-- 
Steve O'Hara-Smith  |   Directable Mirror Arrays
C:>WIN  | A better way to focus the sun
The computer obeys and wins.|licences available see
You lose and Bill collects. |http://www.sohara.org/
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Re: Best file system for a busy webserver

[Wojciech Puchar]

Does anyone have any opinions on which file system is best for a busy 
webserver (7 million hits/month)?  Is anyone one system noticeably better 
than any other?


Use stock UFS, just configure it properly. most importantly noatime.

Amount of cached data is more important than hit count. Unless your 
webpage is incredibly bad design or constantly load different set of large 
amount of small file - filesystem shouldn't be a limit.


Repetitive file fetches would go from cache.

Just curious.  I'm getting ready to setup a new box running FreeBSD 9, and 
since I'm starting from scratch, I'm questioning all my previous assumptions.


Small files will be cached, if you push data from large set of big files 
that will not fit cache, make sure transfers will be fine.


use 32kB block size, 4kB fragment size for UFS

add

options MAXPHYS=2097152

(or even twice of that) to your kernel config so there will be large 
transfers from disk.


This tuning will not make any harm to small files.


My recommendation is for serving files by WWW (or actually - by any 
means).


If you ask for SQL database subsystem then answer is completely different:

make sure all database fits memory cache, or is on SSD or it WILL BE SLOW 
no matter what you use.


Do everything you can to limit amount of sync writes.

if you use SSD and your database software allow dedicating raw partition - 
do it. If not - it is not crucial but useful, avoid double buffering of 
unix cache and database cache.

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"

Best file system for a busy webserver

[Paul Schmehl]

Does anyone have any opinions on which file system is best for a busy 
webserver (7 million hits/month)?  Is anyone one system noticeably better 
than any other?


Just curious.  I'm getting ready to setup a new box running FreeBSD 9, and 
since I'm starting from scratch, I'm questioning all my previous 
assumptions.


--
Paul Schmehl, Senior Infosec Analyst
As if it wasn't already obvious, my opinions
are my own and not those of my employer.
***
"It is as useless to argue with those who have
renounced the use of reason as to administer
medication to the dead." Thomas Jefferson
"There are some ideas so wrong that only a very
intelligent person could believe in them." George Orwell

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"