On 7/19/05, Christopher Petrilli <[EMAIL PROTECTED]> wrote:
> It looks like the CVS HEAD is definately "better," but not by a huge
> amount. The only difference is I wasn't run autovacuum in the
> background (default settings), but I don't think this explains it.
> Here's a graph of the difference
On 7/19/05, Christopher Petrilli <[EMAIL PROTECTED]> wrote:
> On 7/19/05, Tom Lane <[EMAIL PROTECTED]> wrote:
> > Christopher Petrilli <[EMAIL PROTECTED]> writes:
> > > On 7/19/05, Tom Lane <[EMAIL PROTECTED]> wrote:
> > >> How *exactly* are you invoking psql?
> >
> > > It is a subprocess of a Pyth
On 7/19/05, Tom Lane <[EMAIL PROTECTED]> wrote:
> Christopher Petrilli <[EMAIL PROTECTED]> writes:
> > On 7/19/05, Tom Lane <[EMAIL PROTECTED]> wrote:
> >> How *exactly* are you invoking psql?
>
> > It is a subprocess of a Python process, driven using a pexpect
> > interchange. I send the COPY com
It is a subprocess of a Python process, driven using a pexpect
interchange. I send the COPY command, then wait for the '=#' to come
back.
did you try sending the COPY as a normal query through psycopg ?
---(end of broadcast)---
TIP 9: In
Christopher Petrilli <[EMAIL PROTECTED]> writes:
> On 7/19/05, Tom Lane <[EMAIL PROTECTED]> wrote:
>> How *exactly* are you invoking psql?
> It is a subprocess of a Python process, driven using a pexpect
> interchange. I send the COPY command, then wait for the '=#' to come
> back.
Some weird int
On 7/19/05, Tom Lane <[EMAIL PROTECTED]> wrote:
> Christopher Petrilli <[EMAIL PROTECTED]> writes:
> >> Are you sure the backend is reading directly from the file, and not
> >> through psql? (\copy, or COPY FROM STDIN, would go through psql.)
>
> > The exact command is:
> > COPY test (columnlist.
Christopher Petrilli <[EMAIL PROTECTED]> writes:
>> Are you sure the backend is reading directly from the file, and not
>> through psql? (\copy, or COPY FROM STDIN, would go through psql.)
> The exact command is:
> COPY test (columnlist...) FROM '/tmp/loadfile';
I tried to replicate this by putt
On 7/19/05, Tom Lane <[EMAIL PROTECTED]> wrote:
> Christopher Petrilli <[EMAIL PROTECTED]> writes:
> > As I'm doing this, I'm noticing something *VERY* disturbing to me:
> > postmaster backend: 20.3% CPU
> > psql frontend: 61.2% CPU
>
> > WTF? The only thing going through the front end is the COP
Christopher Petrilli <[EMAIL PROTECTED]> writes:
> As I'm doing this, I'm noticing something *VERY* disturbing to me:
> postmaster backend: 20.3% CPU
> psql frontend: 61.2% CPU
> WTF? The only thing going through the front end is the COPY command,
> and it's sent to the backend to read from a fil
As I'm doing this, I'm noticing something *VERY* disturbing to me:
postmaster backend: 20.3% CPU
psql frontend: 61.2% CPU
WTF? The only thing going through the front end is the COPY command,
and it's sent to the backend to read from a file?
Chris
--
| Christopher Petrilli
| [EMAIL PROTECTED]
Christopher Petrilli <[EMAIL PROTECTED]> writes:
> On 7/19/05, Tom Lane <[EMAIL PROTECTED]> wrote:
>> I'm suddenly wondering if the performance dropoff corresponds to the
>> point where the indexes have grown large enough to not fit in shared
>> buffers anymore. If I understand correctly, the 5000
On 7/19/05, PFC <[EMAIL PROTECTED]> wrote:
>
>
> > I think PFC's question was not directed towards modeling your
> > application, but about helping us understand what is going wrong
> > (so we can fix it).
>
> Exactly, I was wondering if this delay would allow things to get
> flushed,
On 7/19/05, Tom Lane <[EMAIL PROTECTED]> wrote:
> Christopher Petrilli <[EMAIL PROTECTED]> writes:
> > Not sure... my benchmark is designed to represent what the database
> > will do under "typical" circumstances, and unfortunately these are
> > typical for the application. However, I can see abo
I think PFC's question was not directed towards modeling your
application, but about helping us understand what is going wrong
(so we can fix it).
Exactly, I was wondering if this delay would allow things to get flushed,
for instance, which would give information about the problem (if giv
total. If the insertion pattern is sufficiently random that the entire
index ranges are "hot" then you might not have enough RAM.
Try doing the test dropping some of your indexes and see if it moves the
number of iterations after which it becomes slow.
---(end of
Christopher Petrilli <[EMAIL PROTECTED]> writes:
> On 7/19/05, PFC <[EMAIL PROTECTED]> wrote:
>> What happens if, say at iteration 6000 (a bit after the mess starts), you
>> pause it for a few minutes and resume. Will it restart with a plateau like
>> at the beginning of the test ? or not ?
> Not
On 7/19/05, PFC <[EMAIL PROTECTED]> wrote:
>
> What happens if, say at iteration 6000 (a bit after the mess starts),
> you
> pause it for a few minutes and resume. Will it restart with a plateau like
> at the beginning of the test ? or not ?
Not sure... my benchmark is designed to repres
What happens if, say at iteration 6000 (a bit after the mess starts), you
pause it for a few minutes and resume. Will it restart with a plateau like
at the beginning of the test ? or not ?
What if, during this pause, you disconnect and reconnect, or restart the
postmaster, or vacuum, or a
Christopher Petrilli <[EMAIL PROTECTED]> writes:
> Here's a graph of the differences and density of behavior:
> http://blog.amber.org/diagrams/pgsql_copy_803_cvs.png
> I can provide the raw data.
How about the complete test case? There's something awfully odd going
on there, and I'd like to fin
On 7/18/05, Tom Lane <[EMAIL PROTECTED]> wrote:
> > The table has 15 columns, 5 indexes (character, inet and timestamp).
> > No foreign keys. The only other thing running on the machine was the
> > application actually DOING the benchmarking, written in Python
> > (psycopg), but it was, according t
Christopher Petrilli <[EMAIL PROTECTED]> writes:
> On 7/18/05, Tom Lane <[EMAIL PROTECTED]> wrote:
>> I have no idea at all what's causing the sudden falloff in performance
>> after about 1 iterations. COPY per se ought to be about a
>> constant-time operation, since APPEND is (or should be) c
On 7/18/05, Tom Lane <[EMAIL PROTECTED]> wrote:
> Christopher Petrilli <[EMAIL PROTECTED]> writes:
> > On 7/18/05, Tom Lane <[EMAIL PROTECTED]> wrote:
> >> I have no idea at all what's causing the sudden falloff in performance
> >> after about 1 iterations. COPY per se ought to be about a
> >>
Christopher Petrilli <[EMAIL PROTECTED]> writes:
> http://blog.amber.org/diagrams/comparison_mysql_pgsql.png
> Notice the VERY steep drop off.
Hmm. Whatever that is, it's not checkpoint's fault. I would interpret
the regular upticks in the Postgres times (every several hundred
iterations) as be
On 7/18/05, Vivek Khera <[EMAIL PROTECTED]> wrote:
>
> On Jul 17, 2005, at 1:08 PM, Christopher Petrilli wrote:
>
> > Normally, checkpoint_segments can help absorb some of that, but my
> > experience is that if I crank the number up, it simply delays the
> > impact, and when it occurs, it takes a
On Jul 17, 2005, at 1:08 PM, Christopher Petrilli wrote:
Normally, checkpoint_segments can help absorb some of that, but my
experience is that if I crank the number up, it simply delays the
impact, and when it occurs, it takes a VERY long time (minutes) to
clear.
There comes a point where you
On 7/18/05, Tom Lane <[EMAIL PROTECTED]> wrote:
> Christopher Petrilli <[EMAIL PROTECTED]> writes:
> > http://blog.amber.org/diagrams/comparison_mysql_pgsql.png
>
> > Notice the VERY steep drop off.
>
> Hmm. Whatever that is, it's not checkpoint's fault. I would interpret
> the regular upticks
Christopher Petrilli <[EMAIL PROTECTED]> writes:
> I have a unique scenerio. My DB is under "continual load", meaning
> that I am constantly using COPY to insert new data into the DB. There
> is no "quiet period" for the database, at least not for hours on end.
> Normally, checkpoint_segments ca
27 matches
Mail list logo
