On 26 Sep 2002, Neil Conway wrote:
> The fact that ext2 defaults to asynchronous mode and UFS (at least on
> the BSDs) defaults to synchronous mode seems like a total non-issue to
> me. Is there any more to the alleged difference in reliability?
It was sort of pointed out here, but perhaps not m
Greg Copeland wrote:
-- Start of PGP signed section.
> Hey, excellent. Thanks!
>
> Based on that, it appears that XFS is a pretty good FS to use. For me,
> the real surprise was how well reiserfs performed.
>
OK, hardware performance paper updated:
---
Hey, excellent. Thanks!
Based on that, it appears that XFS is a pretty good FS to use. For me,
the real surprise was how well reiserfs performed.
Greg
On Thu, 2002-10-03 at 18:09, Mike Benoit wrote:
> Some of you may be interested in this seemingly exhaustive benchmark
> between ext2/3, Reise
Some of you may be interested in this seemingly exhaustive benchmark
between ext2/3, ReiserFS, JFS, and XFS.
http://www.osdl.org/presentations/lwe-jgfs.pdf
---(end of broadcast)---
TIP 6: Have you searched our list archives?
http://archives.post
]
Subject: Re: [HACKERS] [GENERAL] Performance while loading data and
indexing
Neil Conway wrote:
> Bruce Momjian <[EMAIL PROTECTED]> writes:
> > The paper does recommend ext3, but the differences between file systems
> > are very small.
>
> Well, I only did a very rou
Tom Lane <[EMAIL PROTECTED]> writes:
> We'd be happiest with a filesystem that journals its own metadata and
> not the user data in the file(s). I dunno if there are any.
Most journalling file systems work this way. Data journalling is not
very widespread, AFAIK.
--
Florian Weimer
Hello!
On Thu, 26 Sep 2002, Bruce Momjian wrote:
> > I'm not really familiar with the reasoning behind ext2's reputation as
> > recovering poorly from crashes; if we fsync a WAL record to disk
On relatively big volumes ext2 recovery can end up in formatting the fs
under certain cirrumstances.;
Doug McNaught <[EMAIL PROTECTED]> writes:
> "data=writeback" means that no data is journaled, just metadata (which
> is like XFS or Reiser). An fsync() call should still do what it
> normally does, commit the writes to disk before returning.
> "data=journal" journals all data and is the slowest a
Bruce Momjian <[EMAIL PROTECTED]> writes:
> Doug McNaught wrote:
> > Tom Lane <[EMAIL PROTECTED]> writes:
> >
> > > We'd be happiest with a filesystem that journals its own metadata and
> > > not the user data in the file(s). I dunno if there are any.
> >
> > ext3 with data=writeback? (See my
Rod Taylor wrote:
> On Thu, 2002-09-26 at 17:47, Bruce Momjian wrote:
> > Rod Taylor wrote:
> > > > Yes, before UFS had soft updates, the synchronous nature of UFS made it
> > > > slower than ext2, but now with soft updates, that performance difference
> > > > is gone so you have two files systems
On Thu, 2002-09-26 at 17:47, Bruce Momjian wrote:
> Rod Taylor wrote:
> > > Yes, before UFS had soft updates, the synchronous nature of UFS made it
> > > slower than ext2, but now with soft updates, that performance difference
> > > is gone so you have two files systems, ext2 and ufs, similar pefo
Rod Taylor wrote:
> > Yes, before UFS had soft updates, the synchronous nature of UFS made it
> > slower than ext2, but now with soft updates, that performance difference
> > is gone so you have two files systems, ext2 and ufs, similar peformance,
> > but one is crash-safe and the other is not.
>
On Thu, 2002-09-26 at 17:39, Bruce Momjian wrote:
> Neil Conway wrote:
> > Greg Copeland <[EMAIL PROTECTED]> writes:
> > > On Thu, 2002-09-26 at 16:03, Neil Conway wrote:
> > > > I'm not really familiar with the reasoning behind ext2's
> > > > reputation as recovering poorly from crashes; if we fs
Doug McNaught wrote:
> Tom Lane <[EMAIL PROTECTED]> writes:
>
> > We'd be happiest with a filesystem that journals its own metadata and
> > not the user data in the file(s). I dunno if there are any.
>
> ext3 with data=writeback? (See my previous message to Bruce).
OK, so that makes ext3 cras
Neil Conway wrote:
> Greg Copeland <[EMAIL PROTECTED]> writes:
> > On Thu, 2002-09-26 at 16:03, Neil Conway wrote:
> > > I'm not really familiar with the reasoning behind ext2's
> > > reputation as recovering poorly from crashes; if we fsync a WAL
> > > record to disk before we lose power, can't w
Tom Lane <[EMAIL PROTECTED]> writes:
> We'd be happiest with a filesystem that journals its own metadata and
> not the user data in the file(s). I dunno if there are any.
ext3 with data=writeback? (See my previous message to Bruce).
-Doug
---(end of broadcast)
Neil Conway <[EMAIL PROTECTED]> writes:
> I'm not really familiar with the reasoning behind ext2's reputation as
> recovering poorly from crashes; if we fsync a WAL record to disk
> before we lose power, can't we recover reliably, even with ext2?
Up to a point. We do assume that the filesystem w
Bruce Momjian <[EMAIL PROTECTED]> writes:
> Can anyone clarify if "data=writeback" is safe for PostgreSQL.
> Specifically, are the data files recovered properly or is this option
> only for a filesystem containing WAL?
"data=writeback" means that no data is journaled, just metadata (which
is li
Greg Copeland <[EMAIL PROTECTED]> writes:
> On Thu, 2002-09-26 at 16:03, Neil Conway wrote:
> > I'm not really familiar with the reasoning behind ext2's
> > reputation as recovering poorly from crashes; if we fsync a WAL
> > record to disk before we lose power, can't we recover reliably,
> > even
On Thu, 2002-09-26 at 16:03, Neil Conway wrote:
> Bruce Momjian <[EMAIL PROTECTED]> writes:
> > Wow. That leaves no good Linux file system alternatives.
> > PostgreSQL just wants an ordinary file system that has reliable
> > recovery from a crash.
>
> I'm not really familiar with the reasoning b
Neil Conway wrote:
> Bruce Momjian <[EMAIL PROTECTED]> writes:
> > Wow. That leaves no good Linux file system alternatives.
> > PostgreSQL just wants an ordinary file system that has reliable
> > recovery from a crash.
>
> I'm not really familiar with the reasoning behind ext2's reputation as
>
I tend to agree with this though I have nothing to back up it with. My
impression is that XFS does very well for large files. Accepting that
as fact?, my impression is that XFS historically does well for
database's. Again, I have nothing to back that up other than hear-say
and conjecture.
Greg
Bruce Momjian <[EMAIL PROTECTED]> writes:
> Wow. That leaves no good Linux file system alternatives.
> PostgreSQL just wants an ordinary file system that has reliable
> recovery from a crash.
I'm not really familiar with the reasoning behind ext2's reputation as
recovering poorly from crashes; i
Neil Conway wrote:
> Bruce Momjian <[EMAIL PROTECTED]> writes:
> > The paper does recommend ext3, but the differences between file systems
> > are very small.
>
> Well, I only did a very rough benchmark (a few runs of pgbench), but
> the results I found were drastically different: ext2 was signif
I have seen various benchmarks where XFS seems to perform best when it
comes to huge amounts of data and many files (due to balanced internal
b+ trees).
also, XFS seems to be VERY mature and very stable.
ext2/3 don't seem to be that fast in most of the benchmarks.
i did some testing with reiser
Neil Conway wrote:
> Bruce Momjian <[EMAIL PROTECTED]> writes:
> > The paper does recommend ext3, but the differences between file systems
> > are very small.
>
> Well, I only did a very rough benchmark (a few runs of pgbench), but
> the results I found were drastically different: ext2 was signif
Bruce Momjian <[EMAIL PROTECTED]> writes:
> The paper does recommend ext3, but the differences between file systems
> are very small.
Well, I only did a very rough benchmark (a few runs of pgbench), but
the results I found were drastically different: ext2 was significantly
faster (~50%) than ext3
Greg Copeland wrote:
> > The paper does recommend ext3, but the differences between file systems
> > are very small. If you are seeing 'cp' as slow, I wonder if it may be
> > something more general, like poorly tuned hardware or something. You can
> > use 'dd' to throw some data around the file sy
On Thu, 2002-09-26 at 11:41, Bruce Momjian wrote:
> Shridhar Daithankar wrote:
> > I might have found the bottleneck, although by accident. Mysql was running out
> > of space while creating index. So my friend shut down mysql and tried to move
> > things by hand to create links. He noticed that
On Thu, 2002-09-26 at 11:41, Bruce Momjian wrote:
> Shridhar Daithankar wrote:
> > I might have found the bottleneck, although by accident. Mysql was running out
> > of space while creating index. So my friend shut down mysql and tried to move
> > things by hand to create links. He noticed that
Greg Copeland <[EMAIL PROTECTED]> writes:
> I'm not sure about reiserfs or ext3 but with XFS, you can create your
> log on another disk. Also worth noting is that you can also configure
> the size and number of log buffers. There are also some other
> performance type enhancements you can fiddl
Shridhar Daithankar wrote:
> I might have found the bottleneck, although by accident. Mysql was running out
> of space while creating index. So my friend shut down mysql and tried to move
> things by hand to create links. He noticed that even things like cp were
> terribly slow and it hit us..
On Thu, 2002-09-26 at 09:52, Shridhar Daithankar wrote:
> My friend argues for ext2 to eliminate journalling overhead but I favour
> reiserfs personally having used it in pgbench with 10M rows on paltry 20GB IDE
> disk for 25 tps..
>
> We will be attempting raiserfs and/or XFS if required. I kn
On 27 Sep 2002 at 1:12, Justin Clift wrote:
> Shridhar Daithankar wrote:
> As a curiosity point, how predictable are the queries you're going to be
> running on your database? They sound very simple and very predicatable.
Mostly predictable selects. Not a domain expert on telecom so not very su
Shridhar Daithankar wrote:
> My friend argues for ext2 to eliminate journalling overhead but I favour
> reiserfs personally having used it in pgbench with 10M rows on paltry 20GB IDE
> disk for 25 tps..
If it's any help, the setup I mentioned before with differnt disks for
the data and the WAL f
On Thursday 26 September 2002 21:52, Shridhar Daithankar wrote:
> I might have found the bottleneck, although by accident. Mysql was running
> out of space while creating index. So my friend shut down mysql and tried
> to move things by hand to create links. He noticed that even things like cp
>
On 26 Sep 2002 at 10:42, Tom Lane wrote:
> Justin Clift <[EMAIL PROTECTED]> writes:
> > If it's any help, when I was testing recently with WAL on a separate
> > drive, the WAL logs were doing more read&writes per second than the main
> > data drive.
>
> ... but way fewer seeks. For anything inv
On 26 Sep 2002 at 10:33, Tom Lane wrote:
> "Shridhar Daithankar" <[EMAIL PROTECTED]> writes:
> > RedHat7.2/PostgreSQL7.1.3
>
> I'd suggest a newer release of Postgres ... 7.1.3 is pretty old ...
I agree.. downloadind 7.2.2 right away..
> > Create unique composite index on 2 char and a timestam
Justin Clift <[EMAIL PROTECTED]> writes:
>> On 26 Sep 2002 at 19:05, Martijn van Oosterhout wrote:
>>> fsync IIRC only affects the WAL buffers now but it may be quite expensive,
>>> especially considering it's running on every transaction commit. Oh, your
>>> WAL files are on a seperate disk from
"Shridhar Daithankar" <[EMAIL PROTECTED]> writes:
> RedHat7.2/PostgreSQL7.1.3
I'd suggest a newer release of Postgres ... 7.1.3 is pretty old ...
> Create unique composite index on 2 char and a timestamp field: 25226 sec.
What do you mean by "char" exactly? If it's really char(N), how much
ar
Shridhar Daithankar wrote:
>
> On 26 Sep 2002 at 19:05, Martijn van Oosterhout wrote:
> > fsync IIRC only affects the WAL buffers now but it may be quite expensive,
> > especially considering it's running on every transaction commit. Oh, your
> > WAL files are on a seperate disk from the data?
>
Shridhar Daithankar wrote:
> > > fsync=true (Sad but true. Left untouched.. Will that make a difference on
> > > SCSI?)
> >
> > Definitely. Have directly measured a ~ 2x tps throughput increase on
> > FreeBSD when leaving fsync off whilst performance measuring stuff
> > recently (PG 7.2.2). Lik
On 26 Sep 2002 at 19:05, Martijn van Oosterhout wrote:
> On Thu, Sep 26, 2002 at 02:05:44PM +0530, Shridhar Daithankar wrote:
> > 1) Database load time from flat file using copy is very high
> > 2) Creating index takes huge amount of time.
> > 3) Any suggsestions for runtime as data load and quer
On 26 Sep 2002 at 19:17, Justin Clift wrote:
> Shridhar Daithankar wrote:
>
> > 3) Any suggsestions for runtime as data load and query will be going in
> > parallel.
>
> That sounds unusual. From reading this, it *sounds* like you'll be
> running queries against an incomplete dataset, or maybe
Hi Shridhar,
Shridhar Daithankar wrote:
> 3) Any suggsestions for runtime as data load and query will be going in
> parallel.
That sounds unusual. From reading this, it *sounds* like you'll be
running queries against an incomplete dataset, or maybe just running the
queries that affect the tabl
I'll preface this by saying that while I have a large database, it doesn't
require quite the performace you're talking about here.
On Thu, Sep 26, 2002 at 02:05:44PM +0530, Shridhar Daithankar wrote:
> 1) Database load time from flat file using copy is very high
> 2) Creating index takes huge amo
46 matches
Mail list logo
