[PERFORM] multi-threaded pgloader needs your tests

2008-02-26 Thread Dimitri Fontaine
Hi, You may remember some thread about data loading performances and multi-threading support in pgloader: http://archives.postgresql.org/pgsql-performance/2008-02/msg00081.php The pgloader code to handle this is now ready to get tested, a more structured project could talk about a Release

Re: [PERFORM] response time when querying via JDBC and via psql differs

2008-02-26 Thread valgog
Do not use setString() method to pass the parameter to the PreparedStatement in JDBC. Construct an SQL query string as you write it here and query the database with this new SQL string. This will make the planner to recreate a plan every time for every new SQL string per session (that is not

Re: [PERFORM] LISTEN / NOTIFY performance in 8.3

2008-02-26 Thread Joel Stevenson
At 11:58 PM -0500 2/23/08, Tom Lane wrote: Attached is a perl script that sort of simulates what's going on. Thanks for the script. It's not showing any particular problems here, though. With log_min_duration_statement = 10, the only statements that (slightly) exceed 10ms are the select

Re: [PERFORM] PG planning randomly ?

2008-02-26 Thread Tom Lane
Laurent Raufaste [EMAIL PROTECTED] writes: I'm having some issues with this simple query: SELECT _comment.*, _article.title AS article_title, _article.reference AS article_reference FROM _comment INNER JOIN _article ON _article.id = _comment.parent_id WHERE

Re: [PERFORM] Q on views and performance

2008-02-26 Thread Kynn Jones
On Mon, Feb 25, 2008 at 11:56 AM, Matthew [EMAIL PROTECTED] wrote: On Mon, 25 Feb 2008, Kynn Jones wrote: This is just GREAT!!! It fits the problem to a tee. It makes the queries quick then? It is good that you ask. Clearly you know the story: a brilliant-sounding optimization that in

Re: [PERFORM] LISTEN / NOTIFY performance in 8.3

2008-02-26 Thread Tom Lane
Joel Stevenson [EMAIL PROTECTED] writes: Now that I see a little bit more of what goes on under the hood of this function I see that it's still basically table-driven and I'll adjust my expectations accordingly, Yeah, there's been discussion of replacing the implementation with some

Re: [PERFORM] PG planning randomly ?

2008-02-26 Thread Tom Lane
Laurent Raufaste [EMAIL PROTECTED] writes: 2008/2/26, Tom Lane [EMAIL PROTECTED]: If it's 8.2 or later then increasing the stats target for _comment.path to 100 or more would likely help. I'm using PG 8.2.4. We are using 100 as default_statistics_target by default and all our column are

Re: [PERFORM] PG planning randomly ?

2008-02-26 Thread Laurent Raufaste
2008/2/26, Tom Lane [EMAIL PROTECTED]: Laurent Raufaste [EMAIL PROTECTED] writes: 2008/2/26, Tom Lane [EMAIL PROTECTED]: If it's 8.2 or later then increasing the stats target for _comment.path to 100 or more would likely help. I'm using PG 8.2.4. We are using 100 as

Re: [PERFORM] PG planning randomly ?

2008-02-26 Thread Scott Marlowe
On Tue, Feb 26, 2008 at 11:12 AM, Laurent Raufaste [EMAIL PROTECTED] wrote: 2008/2/26, Tom Lane [EMAIL PROTECTED]: What PG version is this? If it's 8.2 or later then increasing the stats target for _comment.path to 100 or more would likely help. I'm using PG 8.2.4. We are

[PERFORM] disabling an index without deleting it?

2008-02-26 Thread Peter Koczan
This might be a weird question...is there any way to disable a particular index without dropping it? There are a few queries I run where I'd like to test out the effects of having (and not having) different indexes on particular query plans and performance. I'd really prefer not to have to drop

Re: [PERFORM] disabling an index without deleting it?

2008-02-26 Thread Scott Marlowe
On Tue, Feb 26, 2008 at 2:46 PM, Peter Koczan [EMAIL PROTECTED] wrote: This might be a weird question...is there any way to disable a particular index without dropping it? There are a few queries I run where I'd like to test out the effects of having (and not having) different indexes on

Re: [PERFORM] disabling an index without deleting it?

2008-02-26 Thread Joshua D. Drake
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Tue, 26 Feb 2008 14:57:51 -0600 Scott Marlowe [EMAIL PROTECTED] wrote: The brick to the head method would use set enable_indexscan = off; However, you can delete an index without actually deleting it like so: begin; drop index abc_dx;

Re: [PERFORM] PG planning randomly ?

2008-02-26 Thread Tom Lane
Laurent Raufaste [EMAIL PROTECTED] writes: 2008/2/26, Tom Lane [EMAIL PROTECTED]: ... I'm wondering if you have a definition of operator @ that doesn't specify the new selectivity estimator. Please try a pg_dump -s and see what it shows as the definition of @. Here's the first definition of

Re: [PERFORM] disabling an index without deleting it?

2008-02-26 Thread Joshua D. Drake
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Tue, 26 Feb 2008 17:22:40 -0500 Tom Lane [EMAIL PROTECTED] wrote: Joshua D. Drake [EMAIL PROTECTED] writes: Scott Marlowe [EMAIL PROTECTED] wrote: begin; drop index abc_dx; select rollback; and viola, your index is still

Re: [PERFORM] LISTEN / NOTIFY performance in 8.3

2008-02-26 Thread James Mansion
Tom Lane wrote: read-modify-write type of operation it uses an exclusive lock, so only one can clear its pg_listener entry at a time. The 'waiting' ones you are seeing are stacked up behind whichever one has the lock at the moment. They shouldn't be waiting for long. I certainly hadn't

Re: [PERFORM] LISTEN / NOTIFY performance in 8.3

2008-02-26 Thread Tom Lane
Joel Stevenson [EMAIL PROTECTED] writes: What's really baffling is that there are plenty of other OLTP queries going in multiple backends simultaneously that don't fall over my 300ms query log threshold, and yet NOTIFY and LISTEN consistently do. What's more it's looks like it's only

Re: [PERFORM] LISTEN / NOTIFY performance in 8.3

2008-02-26 Thread Tom Lane
James Mansion [EMAIL PROTECTED] writes: I certainly hadn't expected that to be the implementation technique - isn't it smply that we need a sngle flag per worker process and can set/test-and-clear with atomic operations and then a signal to wake them up? Hardly --- how's that going to pass

Re: [PERFORM] disabling an index without deleting it?

2008-02-26 Thread Markus Bertheau
2008/2/27, Tom Lane [EMAIL PROTECTED]: Joshua D. Drake [EMAIL PROTECTED] writes: Scott Marlowe [EMAIL PROTECTED] wrote: begin; drop index abc_dx; select rollback; and viola, your index is still there. note that there are likely some locking issues with this, so be

Re: [PERFORM] disabling an index without deleting it?

2008-02-26 Thread Scott Marlowe
On Tue, Feb 26, 2008 at 8:48 PM, Markus Bertheau [EMAIL PROTECTED] wrote: 2008/2/27, Tom Lane [EMAIL PROTECTED]: Joshua D. Drake [EMAIL PROTECTED] writes: Scott Marlowe [EMAIL PROTECTED] wrote: begin; drop index abc_dx; select rollback; and viola,

Re: [PERFORM] disabling an index without deleting it?

2008-02-26 Thread Tom Lane
Markus Bertheau [EMAIL PROTECTED] writes: 2008/2/27, Tom Lane [EMAIL PROTECTED]: No, what makes you think that? The index won't change at all in the above example. The major problem is, as Scott says, that DROP INDEX takes exclusive lock on the table so any other sessions will be locked out

Re: [PERFORM] disabling an index without deleting it?

2008-02-26 Thread Scott Marlowe
On Tue, Feb 26, 2008 at 10:48 PM, Tom Lane [EMAIL PROTECTED] wrote: Markus Bertheau [EMAIL PROTECTED] writes: 2008/2/27, Tom Lane [EMAIL PROTECTED]: No, what makes you think that? The index won't change at all in the above example. The major problem is, as Scott says, that DROP INDEX

Re: [PERFORM] disabling an index without deleting it?

2008-02-26 Thread Tom Lane
Markus Bertheau [EMAIL PROTECTED] writes: On the other hand, if the only reason to have that feature is to plan and execute queries pretending that one index doesn't exist, then DROP INDEX DEFERRED is not the most straightforward syntax. Yeah, I was just about to mention that 8.3 has a hook

Re: [PERFORM] disabling an index without deleting it?

2008-02-26 Thread Markus Bertheau
2008/2/27, Scott Marlowe [EMAIL PROTECTED]: On Tue, Feb 26, 2008 at 10:48 PM, Tom Lane [EMAIL PROTECTED] wrote: Markus Bertheau [EMAIL PROTECTED] writes: 2008/2/27, Tom Lane [EMAIL PROTECTED]: No, what makes you think that? The index won't change at all in the above example.

Re: [PERFORM] LISTEN / NOTIFY performance in 8.3

2008-02-26 Thread James Mansion
Tom Lane wrote: Hardly --- how's that going to pass a notify name? Also, a lot of people want some payload data in a notify, not just a condition name; any reimplementation that doesn't address that desire probably won't get accepted. Ah - forgot about the name. At least there need be just