Re: [PERFORM] performance libpq vs JDBC

2010-12-16 Thread Divakar Singh
Can you trying writing libpq program using COPY functions? I hope it will be better than prepared statements. Best Regards, Divakar From: Werner Scholtes To: "pgsql-performance@postgresql.org" Sent: Wed, December 15, 2010 8:21:55 PM Subject: [PERFORM] perfor

Re: [PERFORM] performance libpq vs JDBC

2010-12-16 Thread Werner Scholtes
Unfortunately I cannot use COPY funtion, since I need the performance of JDBC for update and delete statements in C++ libpq-program as well. I wonder how JDBC PreparedStatement.addBatch() and PreparedStatement.executeBatch() work. They need to have a more efficient protocol to send bulks of pa

Re: [PERFORM] performance libpq vs JDBC

2010-12-16 Thread Divakar Singh
If you have all records before issuing Insert, you can do it like: insert into xxx values (a,b,c), (d,e,f), ..; an example: http://kaiv.wordpress.com/2007/07/19/faster-insert-for-multiple-rows Best Regards, Divakar From: Werner Scholtes To: Divakar Sing

Re: [PERFORM] performance libpq vs JDBC

2010-12-16 Thread Werner Scholtes
What about update and delete? In case of an update I have all records to be updated and in case of an delete I have all primary key values of records to be deleted. Von: pgsql-performance-ow...@postgresql.org [mailto:pgsql-performance-ow...@postgresql.org] Im Auftrag von Divakar Singh Gesendet:

Re: [PERFORM] performance libpq vs JDBC

2010-12-16 Thread Divakar Singh
Update and delete are the operations which affect more than 1 row in general. The only thing is that the criteria has to be the same for all rows. If you have different criteria for different rows in case of update or delete, you will have to fire 2 queries. I mean, if you want to do 1. delete fr

Re: [PERFORM] performance libpq vs JDBC

2010-12-16 Thread Richard Huxton
On 16/12/10 09:21, Werner Scholtes wrote: I assume that the wire protocol of PostgreSQL allows to transmit multiple rows at once, but libpq doesn't have an interface to access it. Is that right? Sounds wrong to me. The libpq client is the default reference implementation of the protocol. If th

[PERFORM] How to get FK to use new index without restarting the database

2010-12-16 Thread Eric Comeau
Is there a way force the db to re-evaluate its execution plan for a FK without bouncing the DB? PostgreSQL 8.1.17 In our latest release our developers have implemented some new foreign keys but forgot to create indexes on these keys. The problem surfaced at one of our client installs wher

Re: [PERFORM] performance libpq vs JDBC

2010-12-16 Thread Werner Scholtes
Thanks a lot for your advice. I found the difference: My Java program sends one huge SQL string containing 1000 INSERT statements separated by ';' (without using prepared statements at all!), whereas my C++ program sends one INSERT statement with parameters to be prepared and after that 1000 tim

Re: [PERFORM] How to get FK to use new index without restarting the database

2010-12-16 Thread Jayadevan M
Hello, > Is there a way force the db to re-evaluate its execution plan for a FK > without bouncing the DB? > > PostgreSQL 8.1.17 > > In our latest release our developers have implemented some new foreign > keys but forgot to create indexes on these keys. > > The problem surfaced at one of ou

Re: [PERFORM] performance libpq vs JDBC

2010-12-16 Thread Richard Huxton
On 16/12/10 12:28, Werner Scholtes wrote: Thanks a lot for your advice. I found the difference: My Java program sends one huge SQL string containing 1000 INSERT statements separated by ';' (without using prepared statements at all!), whereas my C++ program sends one INSERT statement with paramete

Re: [PERFORM] How to get FK to use new index without restarting the database

2010-12-16 Thread Richard Huxton
On 16/12/10 12:12, Eric Comeau wrote: The problem surfaced at one of our client installs where a maintenance DELETE query was running for over 24 hrs. We have since then identified the missing indexes and have sent the client a script to create them, but in our testing we could not been able to

Re: [PERFORM] How to get FK to use new index without restarting the database

2010-12-16 Thread Eric Comeau
On 10-12-16 07:34 AM, Jayadevan M wrote: Hello, Is there a way force the db to re-evaluate its execution plan for a FK without bouncing the DB? PostgreSQL 8.1.17 In our latest release our developers have implemented some new foreign keys but forgot to create indexes on these keys. The prob

Re: [PERFORM] Help with bulk read performance

2010-12-16 Thread Krzysztof Nienartowicz
Hello Daniel, We have the same scenario for the native Java arrays, so we are storing bytea and doing conversion at the client side, but for the server side SQL, plJava comes very handy: No sure how you want to create stored procedures to convert internally but this is how we do this: One has

Re: [PERFORM] performance libpq vs JDBC

2010-12-16 Thread Merlin Moncure
On Thu, Dec 16, 2010 at 7:14 AM, Richard Huxton wrote: > On 16/12/10 09:21, Werner Scholtes wrote: >> >> I assume that the wire protocol of PostgreSQL allows to transmit >> multiple rows at once, but libpq doesn't have an interface to access it. >> Is that right? > > Sounds wrong to me. The libpq

Re: [PERFORM] Help with bulk read performance

2010-12-16 Thread Pierre C
If the data are stored as a byte array but retrieve into a ResultSet, the unpacking time goes up by an order of magnitude and the observed total throughput is 25 MB/s. If the data are stored in a Postgres float array and unpacked into a byte stream, the observed throughput is 20 MB/s. fl

Re: [PERFORM] How to get FK to use new index without restarting the database

2010-12-16 Thread Tom Lane
Eric Comeau writes: > Is there a way force the db to re-evaluate its execution plan for a FK > without bouncing the DB? > PostgreSQL 8.1.17 You don't need to bounce the whole DB, but you will need to start fresh sessions. We didn't add automatic invalidation of those plans until 8.3.

Re: [PERFORM] Index Bloat - how to tell?

2010-12-16 Thread Mladen Gogala
Dave Crooke wrote: There is a plugin called pgstattuple which can be quite informative however, it actually does a full scan of the table / index files, which may be a bit invasive depending on your environment and load. http://www.postgresql.org/docs/current/static/pgstattuple.html It's

Re: [PERFORM] How to get FK to use new index without restarting the database

2010-12-16 Thread Eric Comeau
On 10-12-16 11:27 AM, Tom Lane wrote: Eric Comeau writes: Is there a way force the db to re-evaluate its execution plan for a FK without bouncing the DB? PostgreSQL 8.1.17 You don't need to bounce the whole DB, but you will need to start fresh sessions. We didn't add automatic invalida