Re: [PERFORM] High Performance/High Reliability File system on SuSE64
Christopher Weimann wrote: > On 01/23/2004-10:18AM, Joshua D. Drake wrote: > > > > XFS also has the interesting ability (although I have yet to test it) > > that will allow you > > to take a snapshot of the filesystem. Thus you can have filesystem level > > backups > > of the PGDATA directory that are consistent even though the database is > > running. > > You can do snapshots in FreeBSD 5.x with UFS2 as well but that ( > nor XFS snapshots ) will let you backup with the database server > running. Just because you will get the file exactly as it was at > a particular instant does not mean that the postmaster did not > still have some some data that was not flushed to disk yet. Uh, yea, it does. If the snapshot includes all of /data, including WAL/xlog, you can then back up the snapshot and restore it on another machine. It will restart just like a crash. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [PERFORM] High Performance/High Reliability File system on SuSE64
Christopher Weimann <[EMAIL PROTECTED]> writes: > You can do snapshots in FreeBSD 5.x with UFS2 as well but that ( > nor XFS snapshots ) will let you backup with the database server > running. Just because you will get the file exactly as it was at > a particular instant does not mean that the postmaster did not > still have some some data that was not flushed to disk yet. It *will* work, if you have an instantaneous filesystem snapshot covering the entire $PGDATA directory tree (both data files and WAL). Restarting the postmaster on the backup will result in a WAL replay sequence, and at the end the data files will be consistent. If this were not so, we'd not be crash-proof. The instantaneous snapshot is exactly equivalent to the on-disk state at the moment of a kernel crash or power failure, no? regards, tom lane ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PERFORM] High Performance/High Reliability File system on SuSE64
On 01/23/2004-10:18AM, Joshua D. Drake wrote: > > XFS also has the interesting ability (although I have yet to test it) > that will allow you > to take a snapshot of the filesystem. Thus you can have filesystem level > backups > of the PGDATA directory that are consistent even though the database is > running. You can do snapshots in FreeBSD 5.x with UFS2 as well but that ( nor XFS snapshots ) will let you backup with the database server running. Just because you will get the file exactly as it was at a particular instant does not mean that the postmaster did not still have some some data that was not flushed to disk yet. ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [PERFORM] High Performance/High Reliability File system on SuSE64
> Hello All > > Just wanted to gather opinions on what file system has the best balance between > performance and > reliability when used on a quad processor machine running SuSE64. Thanks > > DAve I was reading the article 'Behind the ALTIX 3000' in the Feb. 2003 Linux Journal, and it mentioned the following paper which compares Ext2, Ext3, ReiserFS, XFS, and JFS: Filesystem Performance and Scalability in Linux 2.4.17 Originally published in Proceedings of the FREENIX Track: 2002 USENIX Annual Technical Conference http://oss.sgi.com/projects/xfs/papers/filesystem-perf-tm.pdf Quoting the abtract: Although the best-performing filesystem varies depending on the benchmark and system used, some larger trends are evident in the data. On the smaller systems, the best-performing file system is often Ext2, Ext3 or ReiserFS. For the larger systems and higher loads, XFS can provide the best overall performance. George Essig ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PERFORM] High Performance/High Reliability File system on SuSE64
Christopher Browne <[EMAIL PROTECTED]> writes: > Ah, but there is a bit of a 'problem' nonetheless; XFS is not > 'officially supported' as part of the Linux kernel until version 2.6, > which is still pretty "bleeding edge." Until 2.6 solidifies a bit > more (aside: based on experiences with 2.6.0, "quite a lot more"), it > is a "patchy" add-on to the 'stable' 2.4 kernel series. Actually I think XFS is in 2.4.25. -- greg ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PERFORM] High Performance/High Reliability File system on SuSE64
[EMAIL PROTECTED] ("Joshua D. Drake") writes:
>>Well, I'd point to one major factor with RHAT; they employ Stephen
>>Tweedie, creator of ext3, and have been paying him to work on it for
>>some time now. If they _didn't_ promote use of ext3, they would be
>>very much vulnerable to the "won't eat their own dogfood" criticism.
>
> True but frankly, they shouldn't. EXT3 has some serious issues. In fact
> if you are running a stock RH kernel before 2.4.20 you can destroy your
> PostgreSQL database with it.
>
> Not to mention how slow it is ;)
I'm not defending ext3's merits; just the clear reason why RHAT uses
it :-).
>>>XFS has been around a LONG time, and on Linux for a couple of years
>>>now. Plus I believe it is the default FS for all of the really high
>>>end stuff SGI is doing with Linux.
>>>
>>>
>>
>>Ah, but there is a bit of a 'problem' nonetheless; XFS is not
>>'officially supported' as part of the Linux kernel until version 2.6,
>> which is still pretty "bleeding edge."
>>
> That is not true see:
>
> http://kerneltrap.org/node/view/1751
Well, I just downloaded 2.4.24 this week, and I don't see XFS included
in it. I see ReiserFS, ext3, and JFS, but not XFS.
>>Until 2.6 solidifies a bit more (aside: based on experiences with
>>2.6.0, "quite a lot more"), it is a "patchy" add-on to the 'stable'
>>2.4 kernel series.
>>
> Again see above :)
>>Do the patches work? As far as I have heard, quite well indeed. But
>>the fact of it not having been 'official' is a fair little bit of a
>>downside.
>
> What is official?
"Is it included in the kernel sources hosted at ftp.kernel.org?"
--
let name="cbbrowne" and tld="libertyrms.info" in name ^ "@" ^ tld;;
Re: [PERFORM] High Performance/High Reliability File system on SuSE64
Mark Kirkwood wrote a little unclearly: And yes, I would recommend xfs - noticeably faster than ext3, and no sign of any mysterious hangs under load. I was thinking about the reported mini-hangs that folks are seeing with jfs, except the all important keyword "jfs" didnt make it out of my head and into the 2nd bit of the message :-( Sorry for the confusion. So I meant "faster that ext3, and without strange mini hangs like jfs...". Joshua wrote: The hangs you are having are due to several issues... one of them is the way ext3 syncs. Never suffered this personally, it was an unexpected filesystem corruption with ext3 that "encouraged" me to try out xfs instead. best wishes Mark ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [PERFORM] High Performance/High Reliability File system on SuSE64
Paul Ganainm <[EMAIL PROTECTED]> writes: > [EMAIL PROTECTED] says... > > >> XFS.. hands down. > > > Off topic question here, but I'm a bit at a loss to understand exactly > what sgi are doing. > > > I thought that they were removing IRIX and going with Linux as the OS to > their high end graphical workstations, yet I see they still have IRIX on > their site. > > > What, exactly, is the story with this? Well, they do have a very large installed base of Irix systems, and a certain obligation to keep supporting them... As long as there are people whi still want MIPS/Irix they will probably keep selling at least a few models. -Doug ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [PERFORM] High Performance/High Reliability File system on SuSE64
[EMAIL PROTECTED] says... > XFS.. hands down. Off topic question here, but I'm a bit at a loss to understand exactly what sgi are doing. I thought that they were removing IRIX and going with Linux as the OS to their high end graphical workstations, yet I see they still have IRIX on their site. What, exactly, is the story with this? Paul... -- plinehan y_a_h_o_o and d_o_t com C++ Builder 5 SP1, Interbase 6.0.1.6 IBX 5.04 W2K Pro Please do not top-post. "XML avoids the fundamental question of what we should do, by focusing entirely on how we should do it." quote from http://www.metatorial.com ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [PERFORM] High Performance/High Reliability File system on SuSE64
Mark Kirkwood wrote: They seem pretty clean (have patched vanilla kernels + xfs for Mandrake 9.2/9.0). And yes, I would recommend xfs - noticeably faster than ext3, and no sign of any mysterious hangs under load. The hangs you are having are due to several issues... one of them is the way ext3 syncs. What kernel version are you running? Sincerely, Joshua Drake best wishes Mark Christopher Browne wrote: Do the patches work? As far as I have heard, quite well indeed. But the fact of it not having been 'official' is a fair little bit of a downside. ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED] -- Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - [EMAIL PROTECTED] - http://www.commandprompt.com PostgreSQL Replicator -- production quality replication for PostgreSQL ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PERFORM] High Performance/High Reliability File system on SuSE64
They seem pretty clean (have patched vanilla kernels + xfs for Mandrake 9.2/9.0). And yes, I would recommend xfs - noticeably faster than ext3, and no sign of any mysterious hangs under load. best wishes Mark Christopher Browne wrote: Do the patches work? As far as I have heard, quite well indeed. But the fact of it not having been 'official' is a fair little bit of a downside. ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PERFORM] High Performance/High Reliability File system on SuSE64
You can do snapshots in FreeBSD 5.x with UFS2 as well but that ( nor XFS snapshots ) will let you backup with the database server running. Just because you will get the file exactly as it was at a particular instant does not mean that the postmaster did not still have some some data that was not flushed to disk yet. Ahh... isn't that what fsync is for? -- Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - [EMAIL PROTECTED] - http://www.commandprompt.com Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PERFORM] High Performance/High Reliability File system on SuSE64
Well, I'd point to one major factor with RHAT; they employ Stephen Tweedie, creator of ext3, and have been paying him to work on it for some time now. If they _didn't_ promote use of ext3, they would be very much vulnerable to the "won't eat their own dogfood" criticism. True but frankly, they shouldn't. EXT3 has some serious issues. In fact if you are running a stock RH kernel before 2.4.20 you can destroy your PostgreSQL database with it. Not to mention how slow it is ;) XFS has been around a LONG time, and on Linux for a couple of years now. Plus I believe it is the default FS for all of the really high end stuff SGI is doing with Linux. Ah, but there is a bit of a 'problem' nonetheless; XFS is not 'officially supported' as part of the Linux kernel until version 2.6, which is still pretty "bleeding edge." That is not true see: http://kerneltrap.org/node/view/1751 Until 2.6 solidifies a bit more (aside: based on experiences with 2.6.0, "quite a lot more"), it is a "patchy" add-on to the 'stable' 2.4 kernel series. Again see above :) Do the patches work? As far as I have heard, quite well indeed. But the fact of it not having been 'official' is a fair little bit of a downside. What is official? Sincerely, Joshua D. Drake I would (and do) trust XFS currently over ANY other journalled option on Linux. I'm getting less and less inclined to trust ext3 or JFS, which "floats upwards" any other boats that are lingering around... -- Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - [EMAIL PROTECTED] - http://www.commandprompt.com Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [PERFORM] High Performance/High Reliability File system on SuSE64
[EMAIL PROTECTED] ("Joshua D. Drake") writes:
>>Yes, I guess I shoulda thought of that, eh? Thanks. The docs do
>>suggest that there are some significant differences between the two
>>versions of the filesystem, so I'm not sure how sanguine I'd be about
>>the degree of "testing" the filesystem has received on Linux. On the
>>
> Well SuSE ships with XFS and SuSE tends to be really good about
> testing. Better than RedHat IMHO. Just the fact that RedHat uses
> ext3 as the default is a black eye.
Well, I'd point to one major factor with RHAT; they employ Stephen
Tweedie, creator of ext3, and have been paying him to work on it for
some time now. If they _didn't_ promote use of ext3, they would be
very much vulnerable to the "won't eat their own dogfood" criticism.
> XFS has been around a LONG time, and on Linux for a couple of years
> now. Plus I believe it is the default FS for all of the really high
> end stuff SGI is doing with Linux.
Ah, but there is a bit of a 'problem' nonetheless; XFS is not
'officially supported' as part of the Linux kernel until version 2.6,
which is still pretty "bleeding edge." Until 2.6 solidifies a bit
more (aside: based on experiences with 2.6.0, "quite a lot more"), it
is a "patchy" add-on to the 'stable' 2.4 kernel series.
Do the patches work? As far as I have heard, quite well indeed. But
the fact of it not having been 'official' is a fair little bit of a
downside.
> I would (and do) trust XFS currently over ANY other journalled
> option on Linux.
I'm getting less and less inclined to trust ext3 or JFS, which "floats
upwards" any other boats that are lingering around...
--
let name="cbbrowne" and tld="libertyrms.info" in String.concat "@" [name;tld];;
Re: [PERFORM] High Performance/High Reliability File system on SuSE64
Yes, I guess I shoulda thought of that, eh? Thanks. The docs do suggest that there are some significant differences between the two versions of the filesystem, so I'm not sure how sanguine I'd be about the degree of "testing" the filesystem has received on Linux. On the Well SuSE ships with XFS and SuSE tends to be really good about testing. Better than RedHat IMHO. Just the fact that RedHat uses ext3 as the default is a black eye. XFS has been around a LONG time, and on Linux for a couple of years now. Plus I believe it is the default FS for all of the really high end stuff SGI is doing with Linux. I would (and do) trust XFS currently over ANY other journalled option on Linux. Sincerely, Joshua D. Drake other hand, I wouldn't be surprised if it were no worse than the other options. A -- Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - [EMAIL PROTECTED] - http://www.commandprompt.com Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PERFORM] High Performance/High Reliability File system on SuSE64
On Fri, Jan 23, 2004 at 11:05:41AM -0800, Joshua D. Drake wrote: > http://oss.sgi.com/projects/xfs/ Yes, I guess I shoulda thought of that, eh? Thanks. The docs do suggest that there are some significant differences between the two versions of the filesystem, so I'm not sure how sanguine I'd be about the degree of "testing" the filesystem has received on Linux. On the other hand, I wouldn't be surprised if it were no worse than the other options. A -- Andrew Sullivan | [EMAIL PROTECTED] The fact that technology doesn't work is no bar to success in the marketplace. --Philip Greenspun ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PERFORM] High Performance/High Reliability File system on SuSE64
There is nothing else on Linux that comes close to that. Plus XFS has been proven in a 64 bit environment (Irix). I had lots of happy experiences with XFS when administering IRIX boxes[1], but I don't know what differences the Linux port entailed. Do you have details on that? http://oss.sgi.com/projects/xfs/ We're certainly looking for an option over JFS at the moment. A [1] I will note, however, that it was practically the only happy experience I had with them. IRIX made the early Debian installer look positively user-friendly, and SGI's desire to make everything whiz-bang nifty by running practically every binary setuid root gave me fits. But XFS was nice. -- Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - [EMAIL PROTECTED] - http://www.commandprompt.com Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PERFORM] High Performance/High Reliability File system on SuSE64
On Fri, Jan 23, 2004 at 10:18:35AM -0800, Joshua D. Drake wrote: > > > Not I. We have had issues with JFS and data corruption on a powerout but > XFS has been rock solid in all of our tests. Sorry, it was Josh Berkus: http://archives.postgresql.org/pgsql-performance/2004-01/msg00086.php > There is nothing else on Linux that comes close to that. Plus XFS has been > proven in a 64 bit environment (Irix). I had lots of happy experiences with XFS when administering IRIX boxes[1], but I don't know what differences the Linux port entailed. Do you have details on that? We're certainly looking for an option over JFS at the moment. A [1] I will note, however, that it was practically the only happy experience I had with them. IRIX made the early Debian installer look positively user-friendly, and SGI's desire to make everything whiz-bang nifty by running practically every binary setuid root gave me fits. But XFS was nice. -- Andrew Sullivan | [EMAIL PROTECTED] In the future this spectacle of the middle classes shocking the avant- garde will probably become the textbook definition of Postmodernism. --Brad Holland ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [PERFORM] High Performance/High Reliability File system on SuSE64
XFS.. hands down. I thought it was you who recently said you thought there was some sort of possible caching problem there? Not I. We have had issues with JFS and data corruption on a powerout but XFS has been rock solid in all of our tests. XFS also has the interesting ability (although I have yet to test it) that will allow you to take a snapshot of the filesystem. Thus you can have filesystem level backups of the PGDATA directory that are consistent even though the database is running. There is nothing else on Linux that comes close to that. Plus XFS has been proven in a 64 bit environment (Irix). Sincerely, Joshua D. Drake A -- Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - [EMAIL PROTECTED] - http://www.commandprompt.com Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [PERFORM] High Performance/High Reliability File system on SuSE64
On Fri, Jan 23, 2004 at 08:51:03AM -0800, Joshua D. Drake wrote: > > > XFS.. hands down. I thought it was you who recently said you thought there was some sort of possible caching problem there? A -- Andrew Sullivan | [EMAIL PROTECTED] The plural of anecdote is not data. --Roger Brinner ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PERFORM] High Performance/High Reliability File system on SuSE64
Dave Thompson wrote: Hello All Just wanted to gather opinions on what file system has the best balance between performance and reliability when used on a quad processor machine running SuSE64. Thanks XFS.. hands down. DAve -- Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - [EMAIL PROTECTED] - http://www.commandprompt.com PostgreSQL Replicator -- production quality replication for PostgreSQL
