-- it
doesn't make too much difference whether it's a bug or not.
Performance problems sometimes end up as bugs and sometimes end up being
treated more like an enhancement; but most of the progress is made on
pgsql-performance regardless.
Regards,
Jeff Davis
--
Sent via pgsql-performance
of the update.
Regards,
Jeff Davis
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
which it levels off. I'm not
sure exactly when that point is, but after that, the downsides of
keeping a transaction open (like inability to remove the previous
version of an updated tuple) take over.
Regards,
Jeff Davis
--
Sent via pgsql-performance mailing list (pgsql-performance
(don't just use ROLLBACK TO, that still leaves the
savepoint there); having 1500 open subtransactions might cause
performance problems elsewhere.
Regards,
Jeff Davis
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http
.
Regards,
Jeff Davis
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
where the index doesn't fit in memory. Mark, can you please try your
experiments on the 9.2beta and tell us whether that helps you?
Regards,
Jeff Davis
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http
backup
first!):
http://www.postgresql.org/docs/9.1/static/app-pgresetxlog.html
And try setting the current transaction ID to just before the delete
ran. Then you may be able to use pg_dump or otherwise export the deleted
rows.
Regards,
Jeff Davis
--
Sent via pgsql-performance mailing list
away, and they might end up getting lost.
Regards,
Jeff Davis
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
)? Fundamentally,
these records aren't changing, you are just trying to interpret them in
the context of the current day. That should be done using a SELECT, not
an UPDATE.
Regards,
Jeff Davis
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes
versa depending on which it thought would be faster.
Although those two queries are semantically the same (I think), a lot of
very similar pairs of queries are not equivalent. For instance, if it
was a NOT IN you couldn't change that to a NOT EXISTS.
Regards,
Jeff Davis
--
Sent via pgsql
around a couple ms at minimum, it's probably some kind of
IO latency. You could see that by wrapping the statements in a big
transaction (BEGIN/END block) -- I bet the inserts go very quickly and
the final commit takes longer.
Regards,
Jeff Davis
--
Sent via pgsql-performance mailing
the records are lying in child tables.
Run EXPLAIN ANALYZE on each of those queries, and post the results.
See http://wiki.postgresql.org/wiki/SlowQueryQuestions for a guide on
how to give the necessary information for others to help.
Regards,
Jeff Davis
--
Sent via pgsql-performance
for specific purposes
will probably be rejected. I am in no way implying that you are
approaching it this way -- I am just trying to characterize an approach
that won't make progress.
Regards,
Jeff Davis
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make
. Is there a
chance that it's flooded with connection attempts?
Usually the work is done by backend processes, not the postmaster. The
postmaster just does some management like accepting connections and
starting new processes.
Regards,
Jeff Davis
--
Sent via pgsql-performance mailing list (pgsql
that is silent when the file
doesn't exist so that it doesn't pollute your logs. I'm not sure why the
documentation suggests cp.
Regards,
Jeff Davis
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org
.
Radix sort only works for some data types and some total orders.
However, it would be very nice to use radix sorting where it does work.
That would require some extensions to the type system, but it could be
done.
The GPU issue is orthogonal.
Regards,
Jeff Davis
--
Sent via pgsql
of a
drive in terms of the amount of data at risk, especially with a DBMS.
Depending on which 256K you lose, you might as well lose your entire
database.
That may be an exaggeration, but the point is that it's not as simple as
this drive is only risking 256K data loss per outage.
Regards,
Jeff
,
Jeff Davis
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
as scheduler
benchmarks). The server was modified to record a log message every N
page accesses.
Regards,
Jeff Davis
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
are removed.
Regards,
Jeff Davis
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
performance impact? If it doesn't need to be fsync'd, the kernel
probably shouldn't have written it to the disk yet anyway, right (I'm
assuming here that the OS buffer cache is much larger than the disk
write cache)?
Regards,
Jeff Davis
--
Sent via pgsql-performance mailing list (pgsql
a single index scan with two quals:
Index Cond: ((ftsbody_body_fts @@
to_tsquery('commonterm'::text)) AND (ftsbody_body_fts @@
to_tsquery('spellerror'::text)))
So it's entirely up to GIN how to execute that.
Regards,
Jeff Davis
--
Sent via pgsql-performance mailing list
should be more
expensive than the second.
Regards,
Jeff Davis
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
,
Jeff Davis
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
of the
existing data like dictionaries and stop words, plus this new common
words. Also, it would mean that almost every match requires RECHECK. It
would be interesting to know how common a word needs to be before it's
better to leave it out of the index.
Regards,
Jeff Davis
--
Sent via
automatically add the common words above some
frequency threshold to the list?
Regards,
Jeff Davis
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
a look into it.
Regards,
Jeff Davis
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
for
opportunities to set hint bits or freeze tuples.
Regards,
Jeff Davis
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
PostgreSQL
is choosing and what it estimates the costs to be. If it's the output of
EXPLAIN ANALYZE, it also runs the query and tells us what the costs
really are.
From that, we can see where the planner is going wrong, and what you
might do to change it.
Regards,
Jeff Davis
--
Sent via pgsql
), the
player condition must be very selective, but PostgreSQL doesn't care
because it already thinks that the date range is selective.
Regards,
Jeff Davis
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http
On Fri, 2008-09-19 at 11:25 -0700, Jeff Davis wrote:
What's the n_distinct for start_time?
Actually, I take that back. Apparently, PostgreSQL can't change x
BETWEEN y AND y into x=y, so PostgreSQL can't use n_distinct at all.
That's your problem. If it's one day only, change it to equality
out broken.
Regards,
Jeff Davis
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
on shared memory. So what's the point of such a bad
design?
Regards,
Jeff Davis
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
be accomplished by increasing the statistics for that column (and
therefore all of your distinct values would fit in the MCV list). Also
the statistics aren't guaranteed to be perfectly up-to-date, so an
estimate of zero might be risky.
Regards,
Jeff Davis
--
Sent via pgsql-performance mailing list
lock on the table so any other sessions will be locked
out of it for the duration of your test query.
It may cause catalog bloat though, right?
Regards,
Jeff Davis
---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project
parallelism to the restore process.
I like this approach more. I think that pg_restore is the right place to
do this, if we can make the options reasonably simple enough to use.
See:
http://archives.postgresql.org/pgsql-hackers/2008-02/msg00699.php
Regards,
Jeff Davis
On Wed, 2008-02-20 at 18:18 +, Matthew wrote:
On Wed, 20 Feb 2008, Jeff Davis wrote:
However, building indexes in parallel would allow better CPU
utilization.
We have a process here that dumps a large quantity of data into an empty
database, much like pg_restore, and then creates all
again, especially with localized text.
Regards,
Jeff Davis
---(end of broadcast)---
TIP 4: Have you searched our list archives?
http://archives.postgresql.org
start using the database while
the rest of the indexes are building (use CREATE INDEX CONCURRENTLY).
Regards,
Jeff Davis
---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index
only one core at a time.
Regards,
Jeff Davis
---(end of broadcast)---
TIP 6: explain analyze is your friend
asynchronously, and then the WAL writer tries
to make those transactions durable, CFQ might think that the WAL writer
is unfairly using a lot of I/O. This is just speculation though.
Regards,
Jeff Davis
---(end of broadcast)---
TIP 3: Have you
. On top of that, the database scale should be bigger
He was referring to the CFQ I/O scheduler. I don't think that will
affect pgbench itself, because it doesn't read/write to disk, right?
Regards,
Jeff Davis
---(end of broadcast)---
TIP 2
On Fri, 2007-12-14 at 11:29 -0800, Jeff Davis wrote:
bigtable has about 60M records, about 2M of which are dead at the time
of VACUUM. Shared_buffers are about 1GB, and the machine has 4GB of
memory.
Forgot to mention: version 8.2.4
Regards,
Jeff Davis
? Are many of these lseeks no-ops or something?
Regards,
Jeff Davis
---(end of broadcast)---
TIP 6: explain analyze is your friend
On Fri, 2007-12-14 at 19:04 -0500, Tom Lane wrote:
Jeff Davis [EMAIL PROTECTED] writes:
Where am I going wrong? Are many of these lseeks no-ops or something?
They're not supposed to be, but if you only tracked seeks and not
reads or writes, it's hard to be sure what's going on.
The seeks
, if you have a relatively small number of topic_ids, is
to break it into separate tables, one for each topic_id.
Regards,
Jeff Davis
---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe
, but we won't know
until you post EXPLAIN ANALYZE results.
Regards,
Jeff Davis
---(end of broadcast)---
TIP 6: explain analyze is your friend
I'm working with postgres 8.0.1,
For the query in question, what would be faster than a seqscan? It
doesn't read the whole table, it only reads until it satisfies the limit
clause.
Regards,
Jeff Davis
---(end of broadcast)---
TIP 9
, if
the I/O is asynchronous and something intelligent (OS or controller) can
schedule it.
Regards,
Jeff Davis
---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster
slony?? and why slony??
This question should be asked on the slony1-general list, you'll get
more responses there.
The benefit of using slony is that you can read from many servers rather
than just one.
Regards,
Jeff Davis
---(end of broadcast
origin even if the old origin
is completely gone, however you will lose the transactions that haven't
been replicated yet.
To have a new node join the cluster, you SUBSCRIBE SET, and you can MOVE
SET to it later if you want that to be the master.
Regards,
Jeff Davis
. The OS
does it's own caching, and so many of those reads might come from the OS
buffer cache, and not the disk itself.
Also, if you're concerned with the number since the last server restart,
make sure you have stats_reset_on_server_start set appropriately.
Regards,
Jeff Davis
and execute the query there?
Have you already considered using views with specific privileges to
separate your customers?
Regards,
Jeff Davis
---(end of broadcast)---
TIP 4: Have you searched our list archives?
http
that you can use fewer connections and then the
OS could still handle it?
Regards,
Jeff Davis
---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster
to take statistical samples of the data in a table without reading
the entire thing?
Regards,
Jeff Davis
---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL
will use a BitmapAnd to combine them?
Regards,
Jeff Davis
---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get
or
Cartesian product?
Regards,
Jeff Davis
---(end of broadcast)---
TIP 6: explain analyze is your friend
for
your data), but there would be no guarantee.
Regards,
Jeff Davis
---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can
,
Jeff Davis
---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster
very quickly. As someone already
mentioned, it also has ranking features.
Regards,
Jeff Davis
---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faq
have successfully made it to disk.
If the write cache holds those data pages, and then loses them, there's
no way for PostgreSQL to recover. So use a battery backed cache or turn
off the write cache.
Regards,
Jeff Davis
---(end of broadcast
On Tue, 2007-02-27 at 01:11 +0100, Peter Kovacs wrote:
On 2/26/07, Jeff Davis [EMAIL PROTECTED] wrote:
On Sun, 2007-02-25 at 23:11 +0100, Peter Kovacs wrote:
A related question:
Is it sufficient to disable write cache only on the disk where pg_xlog
is located? Or should write cache
disk I/O. Is there a good way to
measure how much extra I/O (and WAL volume) is caused by the
checkpoints? Also, it would be good to know how much total I/O is caused
by a checkpoint so that I know if bgwriter is doing it's job.
Regards,
Jeff Davis
---(end
brief
searching and nothing turned up. Do you have a link to the discussion or
the patch?
Regards,
Jeff Davis
---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your
the slow statements in the logs. Once you have
found the slow statements, do an EXPLAIN and an EXPLAIN ANALYZE on those
statements. That will tell you exactly what you need to know.
Regards,
Jeff Davis
---(end of broadcast)---
TIP 6: explain
, hash-based aggregation, and hash-based
processing of IN subqueries.
Regards,
Jeff Davis
---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster
be insignificant.
Regards,
Jeff Davis
---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faq
experiencing.
Am I missing something?
Regards,
Jeff Davis
---(end of broadcast)---
TIP 6: explain analyze is your friend
needs to find the max
between both of them. PostgreSQL isn't smart enough to recognize that it
can use two indexes, find the max in each one, and find the max of those
two values.
Regards,
Jeff Davis
---(end of broadcast)---
TIP 2: Don't
the result of a union by merging the results of
two index scans, I think the problem would be solved. Is there something
preventing this, or is it just something that needs to be added to the
planner?
Regards,
Jeff Davis
---(end of broadcast
On Wed, 2006-10-18 at 17:35 -0500, Jim C. Nasby wrote:
On Wed, Oct 18, 2006 at 03:32:15PM -0700, Jeff Davis wrote:
On Wed, 2006-10-18 at 17:10 -0500, Jim C. Nasby wrote:
Sorry, don't have the earlier part of this thread, but what about...
SELECT greatest(max(a), max(b
not suggesting that we do that, but it seems better then embedding
the hints in the queries themselves.
Regards,
Jeff Davis
---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail
against the
principles of a relational database, which separate the logical query
from the physical storage.
Regards,
Jeff Davis
---(end of broadcast)---
TIP 4: Have you searched our list archives?
http
On Thu, 2006-10-12 at 14:34 -0500, Jim C. Nasby wrote:
On Thu, Oct 12, 2006 at 09:42:55AM -0700, Jeff Davis wrote:
On Thu, 2006-10-12 at 10:14 -0500, Jim C. Nasby wrote:
The syntax these hints is something arbitrary. I'm borrowing Oracle's
idea of embedding hints in comments, but we can
On Thu, 2006-10-12 at 17:28 -0400, Tom Lane wrote:
[ trying once again to push this thread over to -hackers where it belongs ]
Arjen van der Meijden [EMAIL PROTECTED] writes:
On 12-10-2006 21:07 Jeff Davis wrote:
On Thu, 2006-10-12 at 19:15 +0200, Csaba Nagy wrote:
To formalize
doesn't represent your application's needs.
Regards,
Jeff Davis
Regards,
Jeff Davis
---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faq
to using
typical relational constructs. Consider whether sub-selects or
aggregates in conjunction with set-returning functions can achieve what
you want. PostgreSQL is smart enough to only read the big table once if
possible.
Regards,
Jeff Davis
---(end of broadcast
table and doesn't
interfere with the rest of the operations.
Regards,
Jeff Davis
---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings
. In a small
table, the threshold is the dominant term. But both are taken into
account.
Regards,
Jeff Davis
---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your
leaves 6GB for the other process, so you should be fine.
Regards,
Jeff Davis
---(end of broadcast)---
TIP 4: Have you searched our list archives?
http://archives.postgresql.org
plans will change.
There aren't multiple INSERT plans (however, there could be a subselect
or something, which would be planned separately). INSERT is INSERT. That
means effective_cache_size will have zero effect on INSERT.
Regards,
Jeff Davis
---(end of broadcast
be improved. It gets complicated quickly though,
and it's hard to generalize the effect that a performance setting will
have. They are all very interdependent.
Regards,
Jeff Davis
---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ
On Thu, 2006-09-14 at 21:04 -0400, Michael Stone wrote:
On Thu, Sep 14, 2006 at 05:52:02PM -0700, Jeff Davis wrote:
Any long-running system will have very little free memory. Free memory
is wasted memory, so the OS finds some use for it.
The important part of the output of free
, since that
is saying it's cheaper to get a random page than a sequential one.
What was your original random_page_cost, and what is the new value you
set it to?
Regards,
Jeff Davis
---(end of broadcast)---
TIP 1: if posting/reading through
to get a random page than a sequential one.
What was your original random_page_cost, and what is the new value you
set it to?
Regards,
Jeff Davis
I tried it at several levels. It was initially at 4 (the default). I
tried 3 and 2 with no changes. When I set it to 1
, and many
improvements have been made since then.
Regards,
Jeff Davis
---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do
, the requirements might be quite
reasonable. However, if or when you do need to search through that data,
expect it to take a long time.
Regards,
Jeff Davis
---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
on the matter, or is it highly application-
dependent? I am not doing any huge amount of 64-bit arithmetic.
I am using Woodcrest, not Opteron.
Regards,
Jeff Davis
---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ
highly recommend upgrading if at all possible. That's quite an old
version.
Hope this helps,
Jeff Davis
---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faq
On Wed, 2006-08-23 at 21:50 -0400, Michael Stone wrote:
On Wed, Aug 23, 2006 at 03:23:03PM -0700, Jeff Davis wrote:
Also, do ext2 or UFS without soft updates run the risk of losing or
corrupting my data?
I suggest you check the list archives; there's a lot of stuff about
filesystems
On Thu, 2006-08-24 at 09:21 -0400, Merlin Moncure wrote:
On 8/22/06, Jeff Davis [EMAIL PROTECTED] wrote:
On Tue, 2006-08-22 at 17:56 -0400, Bucky Jordan wrote:
Very interesting. I always hear that people avoid RAID 5 on database
servers, but I suppose it always depends. Is the parity
that
JFS was a good choice. However, I assume things have changed somewhat
since then. Does anyone have a pointer to some newer results?
Regards,
Jeff Davis
---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster
or are Opterons significantly better?
I may go for more storage as well (i.e. getting 300GB disks), but I am
still determining the potential need for storage. I can get more RAM at
a later date if necessary also.
Regards,
Jeff Davis
---(end of broadcast
this is caused by either:
(1) You're not vacuuming as a superuser, so it's not able to vacuum
everything.
(2) You have a long-running transaction that never completed for some
strange reason.
Hope this helps,
Jeff Davis
---(end of broadcast)---
TIP
opteron equivalent ;)
Based on your results, I think the Intels should be fine. Does each of
the cores have independent access to memory (therefore making memory
access more parallel)?
Thanks very much for the information!
Regards,
Jeff Davis
---(end of broadcast
-backed writeback cache? How much?
With PostgreSql 8.1.4
We don't have i/o bottle neck.
Well, chances are PostgreSQL is waiting for fsync, which means you do
have an I/O bottleneck (however, you're not using all of your I/O
bandwidth, most likely).
Regards,
Jeff Davis
suggestions are appreciated. Is there a common place to look
for hardware suggestions (like a postgresql hardware FAQ)?
Regards,
Jeff Davis
---(end of broadcast)---
TIP 4: Have you searched our list archives?
http
97 matches
Mail list logo