On 08/08/2003 11:28 mixo wrote:
I have just installed redhat linux 9 which ships with Pg
7.3.2. Pg has to be setup so that data inserts (blobs) should
be able to handle at least 8M at a time. The machine has
two P III 933MHz CPU's, 1.128G RAM (512M*2 + 128M), and
a 36 Gig hd with 1 Gig swap and 3 e
On Mon, Aug 11, 2003 at 06:59:30PM -0400, Bruce Momjian wrote:
> Uh, the ext2 developers say it isn't 100% reliable --- at least that is
> that was told. I don't know any personally, but I mentioned it while I
> was visiting Red Hat, and they didn't refute it.
IMHO, if we're going to say "don't u
On Mon, 11 Aug 2003, Bruce Momjian wrote:
> scott.marlowe wrote:
> > On Fri, 8 Aug 2003, Andrew Sullivan wrote:
> >
> > > On Fri, Aug 08, 2003 at 09:40:20AM -0700, Jonathan Gardner wrote:
> > > >
> > > > Redhat puts ext3 on by default. Consider switching to a non-journaling FS
> > > > (ext2?) w
scott.marlowe wrote:
> On Fri, 8 Aug 2003, Andrew Sullivan wrote:
>
> > On Fri, Aug 08, 2003 at 09:40:20AM -0700, Jonathan Gardner wrote:
> > >
> > > Redhat puts ext3 on by default. Consider switching to a non-journaling FS
> > > (ext2?) with the partition that holds your data and WAL.
> >
> >
On Wed, 2003-08-13 at 01:47, Christopher Kings-Lynne wrote:
> > So, group the inserts in transactions with maybe 1000 commands each. It
> > will go much faster. It can then cache the rows and in the end just make
> > sure all 1000 have been written out on disk.
>
> More than that, he should be usi
On Tue, 12 Aug 2003, mixo wrote:
> that I am currently importing data into Pg which is about 2.9 Gigs.
> Unfortunately, to maintain data intergrity, data is inserted into a table
> one row at a time.'
So you don't put a number of inserts into one transaction?
If you don't do that then postgresql
Jeff <[EMAIL PROTECTED]> writes:
> On Tue, 12 Aug 2003, Christopher Browne wrote:
>> Are you _certain_ that's still true? Have you a metric that shows
>> Informix being 10x faster on a modern system? That would be quite
>> surprising...
> We were forced (for budget reason) to switch from raw dis
On Tue, 12 Aug 2003, Christopher Browne wrote:
> Are you _certain_ that's still true? Have you a metric that shows
> Informix being 10x faster on a modern system? That would be quite
> surprising...
>
We were forced (for budget reason) to switch from raw disk to cooked files
on our informix db.
Ron Johnson wrote:
> On Tue, 2003-08-12 at 13:39, Bruce Momjian wrote:
> > OK, I got some hard evidence. Here is a discussion on the Linux kernel
> > mailing list with postings from Allen Cox (ac Linux kernels) and Stephen
> > Tweedie (ext3 author).
> >
> > http://www.tux.org/hypermail/linux-
[EMAIL PROTECTED] (Jeff) writes:
> On Tue, 12 Aug 2003, Christopher Browne wrote:
>> Are you _certain_ that's still true? Have you a metric that shows
>> Informix being 10x faster on a modern system? That would be quite
>> surprising...
> We were forced (for budget reason) to switch from raw dis
> Andrew Sullivan's fairly regular response is that he tried (albeit
> not VASTLY extensively) to distinguish between disks when working
> with fibre arrays, and he couldn't measure an improvement in
> shifting WAL (the OBVIOUS thing to shift) to separate disks.
Real quick... the faster the drives
On Wed, 2003-08-13 at 10:46, Josh Berkus wrote:
> Jeff,
>
[snip]
> > The other advantage (which I hinted to above) with raw disks is being able
> > to optimize queries to take advantage of it. Informix is multithreaded
> > and it will spawn off multiple "readers" to do say, a seq scan (and merge
Josh Berkus wrote:
> Jeff,
>
> > Informix, etc. have spent a lot of time and money working on it.
> > They also have the advantage of having many paid fulltime
> > developers who are doing this for a job, not as a weekend hobby
> > (Compared to the what? 2-3 full time PG developers).
>
> I think
Thanks to everyone who responded. It's a pity that the discussion has gone
the ext2 vs ext3 route. The main reason I asked my original question is
that I am currently importing data into Pg which is about 2.9 Gigs.
Unfortunately, to maintain data intergrity, data is inserted into a table
one row at
Shridhar Daithankar wrote:
On 11 Aug 2003 at 23:42, Ron Johnson wrote:
On Mon, 2003-08-11 at 19:50, Christopher Kings-Lynne wrote:
Well, yeah. But given the Linux propensity for introducing major
features in "minor" releases (and thereby introducing all the
attendant bugs), I'd think twice abou
On Tue, Aug 12, 2003 at 02:39:19PM -0400, Bill Moran wrote:
> Meaning ... just tell it a raw partition to keep the data on and
> Postgre would create its own "filesystem" ... obviously, doing that
> would allow Postgre to bypass all the failings of all filesystems
> and rely entirely apon its own r
On Tue, 12 Aug 2003, Christopher Kings-Lynne wrote:
> > Well, yeah. But given the Linux propensity for introducing major
> > features in "minor" releases (and thereby introducing all the
> > attendant bugs), I'd think twice about using _any_ Linux feature
> > until it's been through a major versi
On Tue, Aug 12, 2003 at 12:52:46AM -0400, Bruce Momjian wrote:
I don't use Linux and was just repeating what I had heard from others,
> and read in postings. I don't have any first-hand experience with ext2
> (except for a laptop I borrowed that wouldn't boot after being shut
> off), but others o
On Tue, 2003-08-12 at 13:39, Bruce Momjian wrote:
> OK, I got some hard evidence. Here is a discussion on the Linux kernel
> mailing list with postings from Allen Cox (ac Linux kernels) and Stephen
> Tweedie (ext3 author).
>
> http://www.tux.org/hypermail/linux-kernel/1999week14/subject.htm
Uh, the ext2 developers say it isn't 100% reliable --- at least that is
that was told. I don't know any personally, but I mentioned it while I
was visiting Red Hat, and they didn't refute it.
Now, the failure window might be quite small, but I have seen it happen
myself, and have heard it from o
OK, I got some hard evidence. Here is a discussion on the Linux kernel
mailing list with postings from Allen Cox (ac Linux kernels) and Stephen
Tweedie (ext3 author).
http://www.tux.org/hypermail/linux-kernel/1999week14/subject.html#start
Search for "softupdates and ext2".
Here is the
e to make that solid. But the performance gains are significant.
Greg W.
-Original Message-
From: Bill Moran [mailto:[EMAIL PROTECTED]
Sent: Tue 8/12/2003 11:39 AM
To:
Cc: PgSQL Performance ML
Subject: Re: [PERFORM] Perfomance Tuning
Shridhar Daithankar wrote:
> On 11
On Tue, 12 Aug 2003, Neil Conway wrote:
> On Tue, Aug 12, 2003 at 12:52:46AM -0400, Bruce Momjian wrote:
> I don't use Linux and was just repeating what I had heard from others,
> > and read in postings. I don't have any first-hand experience with ext2
> > (except for a laptop I borrowed that wo
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On Friday 08 August 2003 03:28, mixo wrote:
> I have just installed redhat linux 9 which ships with Pg
> 7.3.2. Pg has to be setup so that data inserts (blobs) should
> be able to handle at least 8M at a time. The machine has
> two P III 933MHz CPU's,
> > > Well, yeah. But given the Linux propensity for introducing major
> > > features in "minor" releases (and thereby introducing all the
> > > attendant bugs), I'd think twice about using _any_ Linux feature
> > > until it's been through a major version (e.g. things introduced in
> > > 2.4.x won
On Fri, 8 Aug 2003, Andrew Sullivan wrote:
> On Fri, Aug 08, 2003 at 09:40:20AM -0700, Jonathan Gardner wrote:
> >
> > Redhat puts ext3 on by default. Consider switching to a non-journaling FS
> > (ext2?) with the partition that holds your data and WAL.
>
> I would give you exactly the opposite
On 11 Aug 2003, Ron Johnson wrote:
> On Mon, 2003-08-11 at 19:50, Christopher Kings-Lynne wrote:
> > > Well, yeah. But given the Linux propensity for introducing major
> > > features in "minor" releases (and thereby introducing all the
> > > attendant bugs), I'd think twice about using _any_ Linu
Bruce Momjian <[EMAIL PROTECTED]> writes:
> Rod Taylor wrote:
>> On Fri, 2003-08-08 at 14:53, Andrew Sullivan wrote:
>>> I would give you exactly the opposite advice: _never_ use a
>>> non-journalling fs for your data and WAL. I suppose if you can
>>> afford to lose some transactions, you can do w
> So, group the inserts in transactions with maybe 1000 commands each. It
> will go much faster. It can then cache the rows and in the end just make
> sure all 1000 have been written out on disk.
More than that, he should be using COPY - it's 10x faster than even grouped
inserts.
Chris
> Well, yeah. But given the Linux propensity for introducing major
> features in "minor" releases (and thereby introducing all the
> attendant bugs), I'd think twice about using _any_ Linux feature
> until it's been through a major version (e.g. things introduced in
> 2.4.x won't really be stable
I have just installed redhat linux 9 which ships with Pg
7.3.2. Pg has to be setup so that data inserts (blobs) should
be able to handle at least 8M at a time. The machine has
two P III 933MHz CPU's, 1.128G RAM (512M*2 + 128M), and
a 36 Gig hd with 1 Gig swap and 3 equal size ext3 partitions.
What
On Mon, Aug 11, 2003 at 08:47:07AM -0600, scott.marlowe wrote:
> This isn't saying to not use jounraling, but I would definitely test it
> under load first to make sure it's not gonna lose data or get corrupted.
Well, yeah. But given the Linux propensity for introducing major
features in "minor"
Martha Stewart called it a Good Thing [EMAIL PROTECTED] ("Gregory S. Williamson")wrote:
> FWIW, Informix can be run using a "cooked" (Unix) file for storing
> data or it uses "raw" disk space and bypasses the ordinary (high
> level) UNIX controllers and does its own reads/writes. About 10
> times f
On Wed, 2003-08-13 at 09:37, Tom Lane wrote:
> Jeff <[EMAIL PROTECTED]> writes:
> > On Tue, 12 Aug 2003, Christopher Browne wrote:
> >> Are you _certain_ that's still true? Have you a metric that shows
> >> Informix being 10x faster on a modern system? That would be quite
> >> surprising...
>
>
[EMAIL PROTECTED] (Josh Berkus) writes:
>> The other advantage (which I hinted to above) with raw disks is being able
>> to optimize queries to take advantage of it. Informix is multithreaded
>> and it will spawn off multiple "readers" to do say, a seq scan (and merge
>> the results at the end).
>
Jeff,
> Informix, etc. have spent a lot of time and money working on it.
> They also have the advantage of having many paid fulltime
> developers who are doing this for a job, not as a weekend hobby
> (Compared to the what? 2-3 full time PG developers).
I think 4-6 full-time, actually, plus about
On 11 Aug 2003 at 23:42, Ron Johnson wrote:
> On Mon, 2003-08-11 at 19:50, Christopher Kings-Lynne wrote:
> > > Well, yeah. But given the Linux propensity for introducing major
> > > features in "minor" releases (and thereby introducing all the
> > > attendant bugs), I'd think twice about using _
Rod Taylor wrote:
-- Start of PGP signed section.
> On Fri, 2003-08-08 at 14:53, Andrew Sullivan wrote:
> > On Fri, Aug 08, 2003 at 09:40:20AM -0700, Jonathan Gardner wrote:
> > >
> > > Redhat puts ext3 on by default. Consider switching to a non-journaling FS
> > > (ext2?) with the partition that
On Fri, 8 Aug 2003, mixo wrote:
> I have just installed redhat linux 9 which ships with Pg
> 7.3.2. Pg has to be setup so that data inserts (blobs) should
> be able to handle at least 8M at a time.
Nothing has to be done to tune postgresql to handle this, 8 Meg blobs are
no problem as far as I k
> be able to handle at least 8M at a time. The machine has
> two P III 933MHz CPU's, 1.128G RAM (512M*2 + 128M), and
> a 36 Gig hd with 1 Gig swap and 3 equal size ext3 partitions.
> What would be the recomended setup for good performance
> considering that the db will have about 15 users for
> 9 h
On Mon, 2003-08-11 at 19:50, Christopher Kings-Lynne wrote:
> > Well, yeah. But given the Linux propensity for introducing major
> > features in "minor" releases (and thereby introducing all the
> > attendant bugs), I'd think twice about using _any_ Linux feature
> > until it's been through a majo
Neil Conway wrote:
> On Mon, Aug 11, 2003 at 06:59:30PM -0400, Bruce Momjian wrote:
> > Uh, the ext2 developers say it isn't 100% reliable --- at least that is
> > that was told. I don't know any personally, but I mentioned it while I
> > was visiting Red Hat, and they didn't refute it.
>
> IMHO,
On Mon, 2003-08-11 at 15:16, Bruce Momjian wrote:
That _would_ work if ext2 was a reliable file system --- it is not.
Bruce-
I'd like to know your evidence for this. I'm not refuting it, but I'm a >7 year linux user (including several clusters, all of which have run ext2 or ext3) and keep a
On Fri, Aug 08, 2003 at 09:40:20AM -0700, Jonathan Gardner wrote:
>
> Redhat puts ext3 on by default. Consider switching to a non-journaling FS
> (ext2?) with the partition that holds your data and WAL.
I would give you exactly the opposite advice: _never_ use a
non-journalling fs for your data
On 8 Aug 2003 at 12:28, mixo wrote:
> I have just installed redhat linux 9 which ships with Pg
> 7.3.2. Pg has to be setup so that data inserts (blobs) should
> be able to handle at least 8M at a time. The machine has
> two P III 933MHz CPU's, 1.128G RAM (512M*2 + 128M), and
> a 36 Gig hd with 1 G
On Fri, Aug 08, 2003 at 03:34:44PM -0400, Bruce Momjian wrote:
>
> It is true that ext2 isn't good because the file system may not recover,
> but BSD UFS isn't a journalled file system, but does guarantee file
> system recovery after a crash --- it is especially good using soft
> updates.
Sorry.
> > Agreed.. WAL cannot recover something when WAL no longer exists due to a
> > filesystem corruption.
>
> It is true that ext2 isn't good because the file system may not recover,
> but BSD UFS isn't a journalled file system, but does guarantee file
> system recovery after a crash --- it is espec
On Fri, 2003-08-08 at 14:53, Andrew Sullivan wrote:
> On Fri, Aug 08, 2003 at 09:40:20AM -0700, Jonathan Gardner wrote:
> >
> > Redhat puts ext3 on by default. Consider switching to a non-journaling FS
> > (ext2?) with the partition that holds your data and WAL.
>
> I would give you exactly the
48 matches
Mail list logo
