Re: [rt-users] What software is recommended for high-loaded RT3.8-latest?
Stuart Browne stuart.bro...@ausregistry.com.au writes: In short: XFS is fast and not reliable. I think you exaggerate a bit. We have hundreds of terabytes of storage on XFS. We see very few reliability problems. -- Leif Nixon -Systems expert National Supercomputer Centre- Linkoping University ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
[rt-users] What software is recommended for high-loaded RT3.8-latest?
Hi all. I'm going to migrate our rt installation to latest version. We'll install clean RT on new hardware and them migrate DB and custom modifications. Some points about our rt installation: - db size - more than 30G; - mostly 10 tickets; - 4000 transactions per day. Can you please advice software for serving such high-loaded system: - FreeBSD or Linux? - File system: Ext3/XFS/JFS/...? - apache 2.2 or nginx? - MySQL or Postgresql? Any advice will be appreciated. -- Agnislav Onufrijchuk PortaOne, Inc., RT Developer Tel: +1-866-SIP VOIP (+1 866 747 8647) ext. 7670 Meet us on April 14-15 at Booth 1202 Billing OSS World Conference Expo Rio All-Suite Hotel Casino, Las Vegas ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] What software is recommended for high-loaded RT3.8-latest?
Thu 02 Apr 2009 09:57:57 AM GMT Agnislav Onufrijchuk wrote: Hi all. I'm going to migrate our rt installation to latest version. We'll install clean RT on new hardware and them migrate DB and custom modifications. Some points about our rt installation: - db size - more than 30G; - mostly 10 tickets; - 4000 transactions per day. Can you please advice software for serving such high-loaded system: - FreeBSD or Linux? Slackware Linux. Perfect balance of security and stability and with a custom-generic kernel the RAM footprint is comparatively tiny and makes for a very responsive server. Whatever OS you choose, make sure you do a manual RT install, don't rely on someone's pre-packaged system. Also, I recommend making sure all of your perl modules are installed via CPAN not a packaging system to ensure no upstream modifications and a simple upgrade path. In Slackware both of the above are a given. - File system: Ext3/XFS/JFS/...? Debatable. I would probably say Ext3 myself, but then for the level of transactions you're talking about you are on the border where J/X/Reiser could prove themselves useful. Wouldn't hurt to do some benchmarking. For what it's worth, don't take recoverability into account in your decision, just make backups. Trying to perform file-system data recovery in that type of environment is a waste of time on any FS. - apache 2.2 or nginx? Apache. No Question. - MySQL or Postgresql? Debatable. I think for me it would depend on what is in use in the rest of your architecture. If you are a fully MySQL house, as we are here, then it makes sense to keep it all the same since you can share primary/failover servers and your people-processes are harmonious. If you don't really have a dependency on either then... well it's up to you. I'm used to MySQL and having it at the core of nearly all DB-dependant applications here has been useful, but many would argue that for a larger system like yours PG wouyld give you better performance. Again, a bit of benchmarking wouldn't go amiss. -- Kind Regards, __ Mike Peachey, IT Tel: +44 114 281 2655 Fax: +44 114 281 2951 Jennic Ltd, Furnival Street, Sheffield, S1 4QT, UK Comp Reg No: 3191371 - Registered In England http://www.jennic.com __ ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] What software is recommended for high-loaded RT3.8-latest?
Hi, I have included my comments below. It is important to consider your skills/strengths when making these choices. i.e. If you have experience with one database or OS, you should consider using them instead of trying to build expertise in a new environment. That being said... On Thu, Apr 02, 2009 at 11:57:57AM +0300, Agnislav Onufrijchuk wrote: Hi all. I'm going to migrate our rt installation to latest version. We'll install clean RT on new hardware and them migrate DB and custom modifications. Some points about our rt installation: - db size - more than 30G; - mostly 10 tickets; - 4000 transactions per day. Are these 4000 tickets per day or 4000 updates total? 10 tickets is not very many if you actually generate 4000 tickets per day. Do you shred old tickets to remove them from your DB? Can you please advice software for serving such high-loaded system: - FreeBSD or Linux? Either would be acceptable, given 4000 tickets per day = 500 per hour for an 8 hour day = less than 10 tickets per minute is not much of a CPU load for today's hardware if the I/O subsystem is up to the task.. - File system: Ext3/XFS/JFS/...? Use the supported/recommended one for your chosen OS. - apache 2.2 or nginx? Apache all the way. - MySQL or Postgresql? We use PostgreSQL here because the release quality does not vary as wildly and MySQL. Check the mailing list for problems caused by particular versions of MySQL. If you pick a tested version, it will work well. PostgreSQL also support full text index support that make searching ticket body content extremely fast. We also use the Slony replication software to keep a warm spare RT system ready to go, in case the primary system has a hardware problem. We really want to have redundancy in our ticket system because it should be up even if everything else is down. :) Hope this helps. Cheers, Ken Any advice will be appreciated. ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
[rt-users] What software is recommended for high-loaded RT3.8-latest?
Mike Peachey wrote: Slackware Linux. Perfect balance of security and stability and with a custom-generic kernel the RAM footprint is comparatively tiny and makes for a very responsive server. I have no enough experience with Slackware. AFAIK, it's simple as BSD. Is it true? Whatever OS you choose, make sure you do a manual RT install, don't rely on someone's pre-packaged system. Also, I recommend making sure all of your perl modules are installed via CPAN not a packaging system to ensure no upstream modifications and a simple upgrade path. Sure :) - File system: Ext3/XFS/JFS/...? Debatable. I would probably say Ext3 myself, but then for the level of transactions you're talking about you are on the border where J/X/Reiser could prove themselves useful. Wouldn't hurt to do some benchmarking. For what it's worth, don't take recoverability into account in your decision, just make backups. Trying to perform file-system data recovery in that type of environment is a waste of time on any FS. AFAIK, they're all provide good data safety. Now we're using MySQL InnoDB, I think XFS should be fast enough. But we may migrate to PostgreSQL. AFAIK it uses a number of files (I may be wrong) to serve its DB. So, there can be Reiser/JFS. - apache 2.2 or nginx? Apache. No Question. Why? nginx supports FastCGI too and it is recommended to use on dedicated projects. - MySQL or Postgresql? Debatable. I think for me it would depend on what is in use in the rest of your architecture. If you are a fully MySQL house, as we are here, then it makes sense to keep it all the same since you can share primary/failover servers and your people-processes are harmonious. If you don't really have a dependency on either then... well it's up to you. I'm used to MySQL and having it at the core of nearly all DB-dependant applications here has been useful, but many would argue that for a larger system like yours PG wouyld give you better performance. Again, a bit of benchmarking wouldn't go amiss. Thank you! -- Agnislav Onufrijchuk PortaOne, Inc., RT Developer Tel: +1-866-SIP VOIP (+1 866 747 8647) ext. 7670 Meet us on April 14-15 at Booth 1202 Billing OSS World Conference Expo Rio All-Suite Hotel Casino, Las Vegas ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] What software is recommended for high-loaded RT3.8-latest?
Thu 02 Apr 2009 14:51:06 GMT Agnislav Onufrijchuk wrote: Mike Peachey wrote: Slackware Linux. Perfect balance of security and stability and with a custom-generic kernel the RAM footprint is comparatively tiny and makes for a very responsive server. I have no enough experience with Slackware. AFAIK, it's simple as BSD. Is it true? It is the oldest and most unix-like and vanilla distributions of linux. Simplicity is at its heart along with security and stability. I use it on Servers, Desktops and Laptops alike. AFAIK, they're all provide good data safety. Now we're using MySQL InnoDB, I think XFS should be fast enough. But we may migrate to PostgreSQL. AFAIK it uses a number of files (I may be wrong) to serve its DB. So, there can be Reiser/JFS. Whatever you pick, build it into your kernel and you'll be fine :) - apache 2.2 or nginx? Apache. No Question. Why? nginx supports FastCGI too and it is recommended to use on dedicated projects. Let me put it this way.. when you run into trouble, you want to be on the same server that 99.9% of RT users are running. -- Kind Regards, __ Mike Peachey, IT Tel: +44 114 281 2655 Fax: +44 114 281 2951 Jennic Ltd, Furnival Street, Sheffield, S1 4QT, UK Comp Reg No: 3191371 - Registered In England http://www.jennic.com __ ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] What software is recommended for high-loaded RT3.8-latest?
Kenneth Marshall wrote: Are these 4000 tickets per day or 4000 updates total? 10 tickets is not very many if you actually generate 4000 tickets per day. Do you shred old tickets to remove them from your DB? 4000 Transactions, not tickets per day. - File system: Ext3/XFS/JFS/...? Use the supported/recommended one for your chosen OS. Even if I'll choose FreeBSD, I will not use UFS :) It's too slow. - apache 2.2 or nginx? Apache all the way. What advantages does it have? - MySQL or Postgresql? We use PostgreSQL here because the release quality does not vary as wildly and MySQL. Check the mailing list for problems caused by particular versions of MySQL. If you pick a tested version, it will work well. PostgreSQL also support full text index support that make searching ticket body content extremely fast. We also use the Slony replication software to keep a warm spare RT system ready to go, in case the primary system has a hardware problem. We really want to have redundancy in our ticket system because it should be up even if everything else is down. :) FTS - is one of the advantages of PostgreSQL we look for. Thank you for help! -- Agnislav Onufrijchuk PortaOne, Inc., RT Developer Tel: +1-866-SIP VOIP (+1 866 747 8647) ext. 7670 Meet us on April 14-15 at Booth 1202 Billing OSS World Conference Expo Rio All-Suite Hotel Casino, Las Vegas ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] What software is recommended for high-loaded RT3.8-latest?
Are these 4000 tickets per day or 4000 updates total? 10 tickets is not very many if you actually generate 4000 tickets per day. Do you shred old tickets to remove them from your DB? One more thing: we have 4000 transactions, but we have a number of long SELECT queries every day. No, we didn't shred any of tickets yet, because currently we use 3.4.4 version :( -- Agnislav Onufrijchuk PortaOne, Inc., RT Developer Tel: +1-866-SIP VOIP (+1 866 747 8647) ext. 7670 Meet us on April 14-15 at Booth 1202 Billing OSS World Conference Expo Rio All-Suite Hotel Casino, Las Vegas ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] What software is recommended for high-loaded RT3.8-latest?
On Thu, Apr 02, 2009 at 04:58:34PM +0300, Agnislav Onufrijchuk wrote: Kenneth Marshall wrote: Are these 4000 tickets per day or 4000 updates total? 10 tickets is not very many if you actually generate 4000 tickets per day. Do you shred old tickets to remove them from your DB? 4000 Transactions, not tickets per day. Good, that means every alternative will meet your needs performance-wise. - File system: Ext3/XFS/JFS/...? Use the supported/recommended one for your chosen OS. Even if I'll choose FreeBSD, I will not use UFS :) It's too slow. Makes sense. - apache 2.2 or nginx? Apache all the way. What advantages does it have? The biggest advantage is the userbase for when you have a problem. - MySQL or Postgresql? We use PostgreSQL here because the release quality does not vary as wildly and MySQL. Check the mailing list for problems caused by particular versions of MySQL. If you pick a tested version, it will work well. PostgreSQL also support full text index support that make searching ticket body content extremely fast. We also use the Slony replication software to keep a warm spare RT system ready to go, in case the primary system has a hardware problem. We really want to have redundancy in our ticket system because it should be up even if everything else is down. :) FTS - is one of the advantages of PostgreSQL we look for. I am partial to FTS and it definitely rocks on PostgreSQL. Cheers, Ken ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] What software is recommended for high-loaded RT3.8-latest?
- apache 2.2 or nginx? Apache. No Question. Why? nginx supports FastCGI too and it is recommended to use on dedicated projects. Let me put it this way.. when you run into trouble, you want to be on the same server that 99.9% of RT users are running. Agree :) Thanks for help! -- Agnislav Onufrijchuk PortaOne, Inc., RT Developer Tel: +1-866-SIP VOIP (+1 866 747 8647) ext. 7670 Meet us on April 14-15 at Booth 1202 Billing OSS World Conference Expo Rio All-Suite Hotel Casino, Las Vegas ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] What software is recommended for high-loaded RT3.8-latest?
On Thu, Apr 02, 2009 at 05:03:30PM +0300, Agnislav Onufrijchuk wrote: Are these 4000 tickets per day or 4000 updates total? 10 tickets is not very many if you actually generate 4000 tickets per day. Do you shred old tickets to remove them from your DB? One more thing: we have 4000 transactions, but we have a number of long SELECT queries every day. No, we didn't shred any of tickets yet, because currently we use 3.4.4 version :( Have you run EXPLAIN or the MySQL equivalent to see what is taking the time in the long queries? Maybe adjusting your indexes would help. Cheers, Ken ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] What software is recommended for high-loaded RT3.8-latest?
I have no enough experience with Slackware. AFAIK, it's simple as BSD. Is it true? I second the Slackware vote. I use it for everything. It's the ultimate distro for reliability-through-simplicity. Install nothing you don't need. If you don't need xxgdb on your production server, don't install it. (I don't even have X installed on my servers, but that's just me). Incidentally I frequently have servers go 200 days + without so much as a reboot, even high volume FTP servers like this one: r...@:~# uptime 13:13:47 up 246 days, 1:53, 2 users, load average: 0.23, 0.14, 0.05 AFAIK, they're all provide good data safety. Now we're using MySQL InnoDB, I think XFS should be fast enough. But we may migrate to PostgreSQL. AFAIK it uses a number of files (I may be wrong) to serve its DB. So, there can be Reiser/JFS. Filesystems are something I've spent a LOT of time on, so I know something about this. XFS has not-so-good safety. The fsck / repair tools don't work on very large filesystems because they need massive amount of memory -- more often than not, more than you have. If you need to fsck XFS, odds are you'll be formatting it instead. That said, it is deliciously fast and scalable when properly optioned. Use a RAID controller with battery backup and you should be fine; otherwise turn off write-back caching. Or, test your backups frequently for restorability. :) reiserfs is similiar to XFS with safety. A fsck almost never works because everything's a tree -- once the tree is scrambled, everything in the tree below that point is scrambled too. This is even a bigger risk if you don't make the filesystem with notail. You'll be formatting, not fsck'ing. It also doesn't scale well and its performance with large files is horrid. JFS on the other hand, has wonderful repair tools and decent scalability. Unfortunately, the performance of JFS degrades exponentially with the number of inodes used (files directories) as it searches everything rather linearly, and the inode structure is necessarily inefficient to make it easily repairable. Not recommended for a filesystem with gobs and gobs of small files. (Unfortunately, I have two 12TB RAID arrays formatted JFS with over 14 million small files on them, and if they weren't in production, I'd change it in a heart-beat). ext4 is no longer in development mode and is considered production quality (in kernel 2.6.28 and newer). I highly recommend using it over the other options. It is extent-based rather than block-mapped (if you format it as such), it has the reliability of ext3 and then some (as the journal is checksummed), and its even faster with lots of small files than reiser if you create the filesystem with the dir_index option (which creates a hash of directory entries that is even faster than reiser's b-tree). It even fsck's faster than ext3 because it skips unallocated space. In short: XFS is fast and not reliable. JFS is very reliable, but slow. Reiserfs is a pitiful joke (which can used successfully by the daring lucky). ext4 gives you everything you always asked for: the speed of XFS (ok, almost), the fast lookups of reiser, and the reliability of JFS :) -- -- Tom Lahti BIT Statement LLC (425)251-0833 x 117 http://www.bitstatement.net/ -- ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] What software is recommended for high-loaded RT3.8-latest?
Publicly faced: Web server : 22:05:01 up 566 days, 10:38, 0 users, load average: 0.04, 0.03, 0.02 Mail server: 09:04:10 up 573 days, 15:16, 0 users, load average: 0.16, 0.04, 0.01 Name server: 22:04:16 up 573 days, 21:34, 0 users, load average: 0.02, 0.05, 0.00 RHEL5. Simplicity isn't always better. Management of distribution counts for a lot. In any case, choose which ever platform you're comfortable with. Stuart -Original Message- From: rt-users-boun...@lists.bestpractical.com [mailto:rt-users-boun...@lists.bestpractical.com] On Behalf Of Tom Lahti Sent: Friday, 3 April 2009 07:17 To: Agnislav Onufrijchuk Cc: rt-users@lists.bestpractical.com Subject: Re: [rt-users] What software is recommended for high-loaded RT3.8-latest? I have no enough experience with Slackware. AFAIK, it's simple as BSD. Is it true? I second the Slackware vote. I use it for everything. It's the ultimate distro for reliability-through-simplicity. Install nothing you don't need. If you don't need xxgdb on your production server, don't install it. (I don't even have X installed on my servers, but that's just me). Incidentally I frequently have servers go 200 days + without so much as a reboot, even high volume FTP servers like this one: r...@:~# uptime 13:13:47 up 246 days, 1:53, 2 users, load average: 0.23, 0.14, 0.05 AFAIK, they're all provide good data safety. Now we're using MySQL InnoDB, I think XFS should be fast enough. But we may migrate to PostgreSQL. AFAIK it uses a number of files (I may be wrong) to serve its DB. So, there can be Reiser/JFS. Filesystems are something I've spent a LOT of time on, so I know something about this. XFS has not-so-good safety. The fsck / repair tools don't work on very large filesystems because they need massive amount of memory -- more often than not, more than you have. If you need to fsck XFS, odds are you'll be formatting it instead. That said, it is deliciously fast and scalable when properly optioned. Use a RAID controller with battery backup and you should be fine; otherwise turn off write-back caching. Or, test your backups frequently for restorability. :) reiserfs is similiar to XFS with safety. A fsck almost never works because everything's a tree -- once the tree is scrambled, everything in the tree below that point is scrambled too. This is even a bigger risk if you don't make the filesystem with notail. You'll be formatting, not fsck'ing. It also doesn't scale well and its performance with large files is horrid. JFS on the other hand, has wonderful repair tools and decent scalability. Unfortunately, the performance of JFS degrades exponentially with the number of inodes used (files directories) as it searches everything rather linearly, and the inode structure is necessarily inefficient to make it easily repairable. Not recommended for a filesystem with gobs and gobs of small files. (Unfortunately, I have two 12TB RAID arrays formatted JFS with over 14 million small files on them, and if they weren't in production, I'd change it in a heart-beat). ext4 is no longer in development mode and is considered production quality (in kernel 2.6.28 and newer). I highly recommend using it over the other options. It is extent-based rather than block-mapped (if you format it as such), it has the reliability of ext3 and then some (as the journal is checksummed), and its even faster with lots of small files than reiser if you create the filesystem with the dir_index option (which creates a hash of directory entries that is even faster than reiser's b-tree). It even fsck's faster than ext3 because it skips unallocated space. In short: XFS is fast and not reliable. JFS is very reliable, but slow. Reiserfs is a pitiful joke (which can used successfully by the daring lucky). ext4 gives you everything you always asked for: the speed of XFS (ok, almost), the fast lookups of reiser, and the reliability of JFS :) -- -- Tom Lahti BIT Statement LLC (425)251-0833 x 117 http://www.bitstatement.net/ -- ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
