On 26 Sep 2002, Neil Conway wrote:
The fact that ext2 defaults to asynchronous mode and UFS (at least on
the BSDs) defaults to synchronous mode seems like a total non-issue to
me. Is there any more to the alleged difference in reliability?
It was sort of pointed out here, but perhaps not
Some of you may be interested in this seemingly exhaustive benchmark
between ext2/3, ReiserFS, JFS, and XFS.
http://www.osdl.org/presentations/lwe-jgfs.pdf
---(end of broadcast)---
TIP 6: Have you searched our list archives?
Hey, excellent. Thanks!
Based on that, it appears that XFS is a pretty good FS to use. For me,
the real surprise was how well reiserfs performed.
Greg
On Thu, 2002-10-03 at 18:09, Mike Benoit wrote:
Some of you may be interested in this seemingly exhaustive benchmark
between ext2/3,
Greg Copeland wrote:
-- Start of PGP signed section.
Hey, excellent. Thanks!
Based on that, it appears that XFS is a pretty good FS to use. For me,
the real surprise was how well reiserfs performed.
OK, hardware performance paper updated:
]
Subject: Re: [HACKERS] [GENERAL] Performance while loading data and
indexing
Neil Conway wrote:
Bruce Momjian [EMAIL PROTECTED] writes:
The paper does recommend ext3, but the differences between file systems
are very small.
Well, I only did a very rough benchmark (a few runs of pgbench
Tom Lane [EMAIL PROTECTED] writes:
We'd be happiest with a filesystem that journals its own metadata and
not the user data in the file(s). I dunno if there are any.
Most journalling file systems work this way. Data journalling is not
very widespread, AFAIK.
--
Florian Weimer
I'll preface this by saying that while I have a large database, it doesn't
require quite the performace you're talking about here.
On Thu, Sep 26, 2002 at 02:05:44PM +0530, Shridhar Daithankar wrote:
1) Database load time from flat file using copy is very high
2) Creating index takes huge
Hi Shridhar,
Shridhar Daithankar wrote:
snip
3) Any suggsestions for runtime as data load and query will be going in
parallel.
That sounds unusual. From reading this, it *sounds* like you'll be
running queries against an incomplete dataset, or maybe just running the
queries that affect the
On 26 Sep 2002 at 19:17, Justin Clift wrote:
Shridhar Daithankar wrote:
snip
3) Any suggsestions for runtime as data load and query will be going in
parallel.
That sounds unusual. From reading this, it *sounds* like you'll be
running queries against an incomplete dataset, or maybe just
On 26 Sep 2002 at 19:05, Martijn van Oosterhout wrote:
On Thu, Sep 26, 2002 at 02:05:44PM +0530, Shridhar Daithankar wrote:
1) Database load time from flat file using copy is very high
2) Creating index takes huge amount of time.
3) Any suggsestions for runtime as data load and query will
Shridhar Daithankar wrote:
snip
fsync=true (Sad but true. Left untouched.. Will that make a difference on
SCSI?)
Definitely. Have directly measured a ~ 2x tps throughput increase on
FreeBSD when leaving fsync off whilst performance measuring stuff
recently (PG 7.2.2). Like
Shridhar Daithankar wrote:
On 26 Sep 2002 at 19:05, Martijn van Oosterhout wrote:
snip
fsync IIRC only affects the WAL buffers now but it may be quite expensive,
especially considering it's running on every transaction commit. Oh, your
WAL files are on a seperate disk from the data?
I have seen various benchmarks where XFS seems to perform best when it
comes to huge amounts of data and many files (due to balanced internal
b+ trees).
also, XFS seems to be VERY mature and very stable.
ext2/3 don't seem to be that fast in most of the benchmarks.
i did some testing with
I tend to agree with this though I have nothing to back up it with. My
impression is that XFS does very well for large files. Accepting that
as fact?, my impression is that XFS historically does well for
database's. Again, I have nothing to back that up other than hear-say
and conjecture.
Shridhar Daithankar [EMAIL PROTECTED] writes:
RedHat7.2/PostgreSQL7.1.3
I'd suggest a newer release of Postgres ... 7.1.3 is pretty old ...
Create unique composite index on 2 char and a timestamp field: 25226 sec.
What do you mean by char exactly? If it's really char(N), how much
are you
On 26 Sep 2002 at 10:33, Tom Lane wrote:
Shridhar Daithankar [EMAIL PROTECTED] writes:
RedHat7.2/PostgreSQL7.1.3
I'd suggest a newer release of Postgres ... 7.1.3 is pretty old ...
I agree.. downloadind 7.2.2 right away..
Create unique composite index on 2 char and a timestamp field:
On 26 Sep 2002 at 10:42, Tom Lane wrote:
Justin Clift [EMAIL PROTECTED] writes:
If it's any help, when I was testing recently with WAL on a separate
drive, the WAL logs were doing more readwrites per second than the main
data drive.
... but way fewer seeks. For anything involving lots
On Thursday 26 September 2002 21:52, Shridhar Daithankar wrote:
I might have found the bottleneck, although by accident. Mysql was running
out of space while creating index. So my friend shut down mysql and tried
to move things by hand to create links. He noticed that even things like cp
Shridhar Daithankar wrote:
snip
My friend argues for ext2 to eliminate journalling overhead but I favour
reiserfs personally having used it in pgbench with 10M rows on paltry 20GB IDE
disk for 25 tps..
If it's any help, the setup I mentioned before with differnt disks for
the data and the WAL
On 27 Sep 2002 at 1:12, Justin Clift wrote:
Shridhar Daithankar wrote:
As a curiosity point, how predictable are the queries you're going to be
running on your database? They sound very simple and very predicatable.
Mostly predictable selects. Not a domain expert on telecom so not very
On Thu, 2002-09-26 at 09:52, Shridhar Daithankar wrote:
My friend argues for ext2 to eliminate journalling overhead but I favour
reiserfs personally having used it in pgbench with 10M rows on paltry 20GB IDE
disk for 25 tps..
We will be attempting raiserfs and/or XFS if required. I know
Shridhar Daithankar wrote:
I might have found the bottleneck, although by accident. Mysql was running out
of space while creating index. So my friend shut down mysql and tried to move
things by hand to create links. He noticed that even things like cp were
terribly slow and it hit us.. May
Greg Copeland [EMAIL PROTECTED] writes:
I'm not sure about reiserfs or ext3 but with XFS, you can create your
log on another disk. Also worth noting is that you can also configure
the size and number of log buffers. There are also some other
performance type enhancements you can fiddle
On Thu, 2002-09-26 at 11:41, Bruce Momjian wrote:
Shridhar Daithankar wrote:
I might have found the bottleneck, although by accident. Mysql was running out
of space while creating index. So my friend shut down mysql and tried to move
things by hand to create links. He noticed that even
On Thu, 2002-09-26 at 11:41, Bruce Momjian wrote:
Shridhar Daithankar wrote:
I might have found the bottleneck, although by accident. Mysql was running out
of space while creating index. So my friend shut down mysql and tried to move
things by hand to create links. He noticed that even
Greg Copeland wrote:
The paper does recommend ext3, but the differences between file systems
are very small. If you are seeing 'cp' as slow, I wonder if it may be
something more general, like poorly tuned hardware or something. You can
use 'dd' to throw some data around the file system
Neil Conway wrote:
Bruce Momjian [EMAIL PROTECTED] writes:
The paper does recommend ext3, but the differences between file systems
are very small.
Well, I only did a very rough benchmark (a few runs of pgbench), but
the results I found were drastically different: ext2 was significantly
Neil Conway wrote:
Bruce Momjian [EMAIL PROTECTED] writes:
The paper does recommend ext3, but the differences between file systems
are very small.
Well, I only did a very rough benchmark (a few runs of pgbench), but
the results I found were drastically different: ext2 was significantly
Bruce Momjian [EMAIL PROTECTED] writes:
Wow. That leaves no good Linux file system alternatives.
PostgreSQL just wants an ordinary file system that has reliable
recovery from a crash.
I'm not really familiar with the reasoning behind ext2's reputation as
recovering poorly from crashes; if we
Neil Conway wrote:
Bruce Momjian [EMAIL PROTECTED] writes:
Wow. That leaves no good Linux file system alternatives.
PostgreSQL just wants an ordinary file system that has reliable
recovery from a crash.
I'm not really familiar with the reasoning behind ext2's reputation as
recovering
On Thu, 2002-09-26 at 16:03, Neil Conway wrote:
Bruce Momjian [EMAIL PROTECTED] writes:
Wow. That leaves no good Linux file system alternatives.
PostgreSQL just wants an ordinary file system that has reliable
recovery from a crash.
I'm not really familiar with the reasoning behind
Greg Copeland [EMAIL PROTECTED] writes:
On Thu, 2002-09-26 at 16:03, Neil Conway wrote:
I'm not really familiar with the reasoning behind ext2's
reputation as recovering poorly from crashes; if we fsync a WAL
record to disk before we lose power, can't we recover reliably,
even with ext2?
Bruce Momjian [EMAIL PROTECTED] writes:
Can anyone clarify if data=writeback is safe for PostgreSQL.
Specifically, are the data files recovered properly or is this option
only for a filesystem containing WAL?
data=writeback means that no data is journaled, just metadata (which
is like XFS
Neil Conway [EMAIL PROTECTED] writes:
I'm not really familiar with the reasoning behind ext2's reputation as
recovering poorly from crashes; if we fsync a WAL record to disk
before we lose power, can't we recover reliably, even with ext2?
Up to a point. We do assume that the filesystem won't
Tom Lane [EMAIL PROTECTED] writes:
We'd be happiest with a filesystem that journals its own metadata and
not the user data in the file(s). I dunno if there are any.
ext3 with data=writeback? (See my previous message to Bruce).
-Doug
---(end of
Neil Conway wrote:
Greg Copeland [EMAIL PROTECTED] writes:
On Thu, 2002-09-26 at 16:03, Neil Conway wrote:
I'm not really familiar with the reasoning behind ext2's
reputation as recovering poorly from crashes; if we fsync a WAL
record to disk before we lose power, can't we recover
Doug McNaught wrote:
Tom Lane [EMAIL PROTECTED] writes:
We'd be happiest with a filesystem that journals its own metadata and
not the user data in the file(s). I dunno if there are any.
ext3 with data=writeback? (See my previous message to Bruce).
OK, so that makes ext3 crash safe
On Thu, 2002-09-26 at 17:39, Bruce Momjian wrote:
Neil Conway wrote:
Greg Copeland [EMAIL PROTECTED] writes:
On Thu, 2002-09-26 at 16:03, Neil Conway wrote:
I'm not really familiar with the reasoning behind ext2's
reputation as recovering poorly from crashes; if we fsync a WAL
Rod Taylor wrote:
Yes, before UFS had soft updates, the synchronous nature of UFS made it
slower than ext2, but now with soft updates, that performance difference
is gone so you have two files systems, ext2 and ufs, similar peformance,
but one is crash-safe and the other is not.
Note
On Thu, 2002-09-26 at 17:47, Bruce Momjian wrote:
Rod Taylor wrote:
Yes, before UFS had soft updates, the synchronous nature of UFS made it
slower than ext2, but now with soft updates, that performance difference
is gone so you have two files systems, ext2 and ufs, similar peformance,
Rod Taylor wrote:
On Thu, 2002-09-26 at 17:47, Bruce Momjian wrote:
Rod Taylor wrote:
Yes, before UFS had soft updates, the synchronous nature of UFS made it
slower than ext2, but now with soft updates, that performance difference
is gone so you have two files systems, ext2 and
Bruce Momjian [EMAIL PROTECTED] writes:
Doug McNaught wrote:
Tom Lane [EMAIL PROTECTED] writes:
We'd be happiest with a filesystem that journals its own metadata and
not the user data in the file(s). I dunno if there are any.
ext3 with data=writeback? (See my previous message
Doug McNaught [EMAIL PROTECTED] writes:
data=writeback means that no data is journaled, just metadata (which
is like XFS or Reiser). An fsync() call should still do what it
normally does, commit the writes to disk before returning.
data=journal journals all data and is the slowest and
Hello!
On Thu, 26 Sep 2002, Bruce Momjian wrote:
I'm not really familiar with the reasoning behind ext2's reputation as
recovering poorly from crashes; if we fsync a WAL record to disk
On relatively big volumes ext2 recovery can end up in formatting the fs
under certain cirrumstances.;-)
44 matches
Mail list logo
