Re: [GENERAL] Cheapest way to poll for notifications?

2009-12-18 Thread Craig Ringer
On 11/12/2009 11:39 PM, Craig Ringer wrote: As for the JDBC driver - it turns out that the documentation is out-of-date and/or misleading. The JDBC driver *does* support reading notifications the backend has pushed to its receive buffer, and does *not* have to poll the backend or issue a stateme

Re: [GENERAL] PL/Perl Performance Problems

2009-12-18 Thread Alex -
Yes I do, but this is the pl/perl function called by a batch job i run. before the pl/perl function is called i insert 2x200k records into 2 tables (200k per table). First i thought that it might be a problem with the perl function, but then i noticed that it even started earlier with the simpl

Re: [GENERAL] pulling metadata creation strings from metadata tables?

2009-12-18 Thread Craig Ringer
On 19/12/2009 2:24 PM, Gauthier, Dave wrote: Hi: Are metadata creation strings, in sql form, stored anywhere in the metadata tables? Not AFAIK, no. How would you handle subsequent ALTER commands and the like? What if you ALTERed two tables in a way that could only succeed if done in a parti

Re: [GENERAL] PL/Perl Performance Problems

2009-12-18 Thread Scott Marlowe
According to your original post, you do selects in step 1 and 2... Or is this a different job and I've lost the thread (happens to me plenty :) ) 1. Selects about 20 Records from Table A ( - loops though the list and deletes in total about 50k records in Table B 2. For each record form Table A

Re: [GENERAL] PL/Perl Performance Problems

2009-12-18 Thread Alex -
On a 2nd thought... where does the cach come into play when i only do inserts and no selects. Alex > Date: Fri, 18 Dec 2009 23:45:07 -0700 > Subject: Re: [GENERAL] PL/Perl Performance Problems > From: scott.marl...@gmail.com > To: ainto...@hotmail.com > CC: t...@sss.pgh.pa.us; pgsql-general@post

Re: [GENERAL] PL/Perl Performance Problems

2009-12-18 Thread Alex -
I have the standard ones and no others. # run-parts01 * * * * root run-parts /etc/cron.hourly02 4 * * * root run-parts /etc/cron.daily22 4 * * 0 root run-parts /etc/cron.weekly42 4 1 * * root run-parts /etc/cron.monthly ls -l /etc/cron*-rw-r--r-- 1 root root0 Apr 6 2009 /etc/cron.deny-rw

Re: [GENERAL] PL/Perl Performance Problems

2009-12-18 Thread Scott Marlowe
On Fri, Dec 18, 2009 at 11:37 PM, Alex - wrote: > Hmm... > how can that be. This is happening every day, so its not a one off or > happens once in the morning then in the afternoon. There is also no other > task running on the system, its dedicated to postgres. > Could the Autovacuum cause problem

Re: [GENERAL] PL/Perl Performance Problems

2009-12-18 Thread Alex -
Hmm...how can that be. This is happening every day, so its not a one off or happens once in the morning then in the afternoon. There is also no other task running on the system, its dedicated to postgres. Could the Autovacuum cause problems? Starting to invoke Analyze at the beginning of the da

[GENERAL] pulling metadata creation strings from metadata tables?

2009-12-18 Thread Gauthier, Dave
Hi: Are metadata creation strings, in sql form, stored anywhere in the metadata tables? So if I... create table foo (x int); I would like to pull that same string from somewhere. Thanks in Advance

Re: [GENERAL] Extracting SQL from logs in a usable format

2009-12-18 Thread Greg Smith
Chris Ernst wrote: I started writing my own log parser to pull the statements from the postgres logs, but as I get in to the details, it's not quite as straight forward as I had thought. Keeping track of which client connections have prepared queries, merging the correct arguments and maintainin

Re: [GENERAL] Selecting from table into an array var

2009-12-18 Thread Postgres User
On Fri, Dec 18, 2009 at 9:53 PM, Merlin Moncure wrote: > On Fri, Dec 18, 2009 at 11:35 PM, Pavel Stehule > wrote: >> 2009/12/19 Postgres User : >>> Hi, >>> >>> I'm trying to write a very simple function statement to select a >>> single integer field from a table and save it into an int array. Fo

Re: [GENERAL] PL/Perl Performance Problems

2009-12-18 Thread Scott Marlowe
Note that you seem to have a lot more IO wait in the first run than in the second, which means that the task is hitting the disks more in the first run than in the second one. Once IO wait starts to climb, performance starts to dive, generally. -- Sent via pgsql-general mailing list (pgsql-gener

Re: [GENERAL] Selecting from table into an array var

2009-12-18 Thread Merlin Moncure
On Fri, Dec 18, 2009 at 11:35 PM, Pavel Stehule wrote: > 2009/12/19 Postgres User : >> Hi, >> >> I'm trying to write a very simple function statement to select a >> single integer field from a table and save it into an int array. For >> some reason I can't seem to find the correct syntax: >> >> CR

Re: [GENERAL] Selecting from table into an array var

2009-12-18 Thread Pavel Stehule
2009/12/19 Postgres User : > Hi, > > I'm trying to write a very simple function statement to select a > single integer field from a table and save it into an int array. For > some reason I can't seem to find the correct syntax: > > CREATE TABLE sample ( >    id  integer > ); > > and then within a f

[GENERAL] Selecting from table into an array var

2009-12-18 Thread Postgres User
Hi, I'm trying to write a very simple function statement to select a single integer field from a table and save it into an int array. For some reason I can't seem to find the correct syntax: CREATE TABLE sample ( id integer ); and then within a function: my_array int[]; my_array = SELECT A

Re: [GENERAL] PL/Perl Performance Problems

2009-12-18 Thread Alex -
I run AutovacuumI run a Vaccum Full Analyze every SatI re-index the tables every Sat > Date: Fri, 18 Dec 2009 21:20:23 -0300 > From: alvhe...@commandprompt.com > To: ainto...@hotmail.com > CC: t...@sss.pgh.pa.us; pgsql-general@postgresql.org > Subject: Re: [GENERAL] PL/Perl Performance Problems

Re: [GENERAL] PL/Perl Performance Problems

2009-12-18 Thread Alvaro Herrera
Alex - wrote: > > Tom, I am logging these stats now, but i am having a similar issue. both jobs > in the morning and after noon insert about 400k records (200k each) > In the morning that job takes 450secIn the afternoon only 150; No select, > simple parsing a file and insert the records > These

Re: [GENERAL] Extracting SQL from logs in a usable format

2009-12-18 Thread Chris Ernst
Rory Campbell-Lange wrote: > On 18/12/09, Chris Ernst (cer...@esoft.com) wrote: >> I have a project where I need to be able to capture every query from a >> production system into a file such that I can "replay" them on a staging >> system. Does such a thing already exist or should I start writing

Re: [GENERAL] PL/Perl Performance Problems

2009-12-18 Thread Alex -
Tom, I am logging these stats now, but i am having a similar issue. both jobs in the morning and after noon insert about 400k records (200k each) In the morning that job takes 450secIn the afternoon only 150; No select, simple parsing a file and insert the records These two tables don't grow as

Re: [GENERAL] Extracting SQL from logs in a usable format

2009-12-18 Thread Rory Campbell-Lange
On 18/12/09, Chris Ernst (cer...@esoft.com) wrote: > I have a project where I need to be able to capture every query from a > production system into a file such that I can "replay" them on a staging > system. Does such a thing already exist or should I start writing my > own log parser? I'm sure

[GENERAL] Extended Query, flush or sync ?

2009-12-18 Thread Raimon Fernandez
Hello, It's not clear for me if I have to issue a flush or sync after each process of an extended query. It's almost working for me only when I send a sync, but not when I send a flush. With the flush, the connection seems freezed, or at least, I don't get any data from postgre. - Send the p

[GENERAL] Extracting SQL from logs in a usable format

2009-12-18 Thread Chris Ernst
Hi all, I have a project where I need to be able to capture every query from a production system into a file such that I can "replay" them on a staging system. Does such a thing already exist or should I start writing my own log parser? Thank you for your help. Chris Ernst eSoft, Inc. -- Sent

[GENERAL] defining yuor own commands in PG ?

2009-12-18 Thread Gauthier, Dave
Can you define your own commands in PG. E.g., if users from other DBs use "describe foo" to get the metadata for foo, is there a way I can create a command "describe" to = "\d" ? Thanks in Advance

Re: [GENERAL] PL/Perl Performance Problems

2009-12-18 Thread Alex -
Thank, I will check that out. > To: ainto...@hotmail.com > CC: pgsql-general@postgresql.org > Subject: Re: [GENERAL] PL/Perl Performance Problems > Date: Fri, 18 Dec 2009 15:36:15 -0500 > From: t...@sss.pgh.pa.us > > Alex - writes: > > I actually looked at that too, but there is no swapping go

Re: [GENERAL] PL/Perl Performance Problems

2009-12-18 Thread Tom Lane
Alex - writes: > I actually looked at that too, but there is no swapping going on. If you were only watching for swapping, that wouldn't catch what I'm talking about. Replacing cached disk buffers with other disk data doesn't count as swapping in any system I've used. re

Re: [GENERAL] PL/Perl Performance Problems

2009-12-18 Thread Alex -
I actually looked at that too, but there is no swapping going on. The system also has 16GB memory allocated to postgres and during the processing there is not other process really active. > To: ainto...@hotmail.com > CC: pgsql-general@postgresql.org > Subject: Re: [GENERAL] PL/Perl Performance

Re: [GENERAL] PL/Perl Performance Problems

2009-12-18 Thread Tom Lane
Alex - writes: > Now here is what I noticed. > a) if I run it in the morning, processing starts very slow, but after a few > thousand records it will speed up until I actually get about 100 records > processed per millisecond. > b) it sometime takes about 5-10k records till i really get up to sp

[GENERAL] PL/Perl Performance Problems

2009-12-18 Thread Alex -
Hi, I am experiencing some strange behavior when executing a not too complicated pl/perl function. The Function is not too complicated. It does... 1. Selects about 20 Records from Table A ( - loops though the list and deletes in total about 50k records in Table B2. For each record form Table A

Re: [GENERAL] Justifying a PG over MySQL approach to a project

2009-12-18 Thread Lincoln Yeoh
At 11:28 AM 12/18/2009, Scott Marlowe wrote: On Thu, Dec 17, 2009 at 7:51 PM, David Boreham wrote: > Scott Marlowe wrote: >> >> I would recommend using a traffic shaping router (like the one built >> into the linux kernel and controlled by tc / iptables) to simulate a >> long distance connection

Re: [GENERAL] Justifying a PG over MySQL approach to a project

2009-12-18 Thread David Boreham
Lincoln Yeoh wrote: It seems you currently can only control outbound traffic from an interface, so you'd have to set stuff on both interfaces to "shape" upstream and downstream - this is not so convenient in some network topologies. This is more a property of the universe than the software ;)

[GENERAL] Triggers made with plpythonu performance issue

2009-12-18 Thread sabrina miller
Hi everybody, My requirements was: + Made a table charge to be partitioned by carrier and month + summarize by charges + summarize by users, + each summarization must be by month and several others columns. Here is the database CREATE TABLE charges.charge ( id serial NOT NULL, transaction

[GENERAL] column level encryption & select rules

2009-12-18 Thread Little, Douglas
Hi, I've implemented a scheme for column level encryption that uses table triggers (isrt/update) to encrypt the input data, and a view To perform the decrypt. It's working ok, but I'm having trouble altering my objects because of the dependents. To implement the scheme, I have to generate the

Re: [GENERAL] pg_dump and ON DELETE CASCADE problem

2009-12-18 Thread Tom Lane
Adrian Klaver writes: > On Thursday 17 December 2009 6:39:45 pm CG wrote: >> CREATE INDEX packet_search_trigram_packet_uuid_idx >> ON dpo.packet_search_trigram >> USING hash >> (packet_uuid); > You might want to take a look at upgrading to 8.4.2 per this from the release > notes: Actually, what

Re: [GENERAL] pg_dump and ON DELETE CASCADE problem

2009-12-18 Thread Adrian Klaver
On Thursday 17 December 2009 6:39:45 pm CG wrote: > --- On Thu, 12/17/09, Adrian Klaver wrote: > > Would it be possible to see the table schemas and indices > > ? > > Sure (you asked for it!!) : > > > CREATE TABLE dpo.packet_search_trigram > ( > id integer NOT NULL DEFAULT > nextval('packet_sea

Re: [GENERAL] How to remove non-UTF values from a table?

2009-12-18 Thread Bruce Momjian
Phoenix Kiula wrote: > On Fri, Dec 18, 2009 at 3:54 AM, Bruce Momjian wrote: > > Bruce Momjian wrote: > >> pg_migrator (not pg_upgrade) has been used by many people to migrate > >> from 8.3 to 8.4. ?I just helped someone yesterday with a migration. > >> pg_migrator threw an error because they had

Re: [GENERAL] How to remove non-UTF values from a table?

2009-12-18 Thread Phoenix Kiula
On Fri, Dec 18, 2009 at 3:54 AM, Bruce Momjian wrote: > Bruce Momjian wrote: >> pg_migrator (not pg_upgrade) has been used by many people to migrate >> from 8.3 to 8.4.  I just helped someone yesterday with a migration. >> pg_migrator threw an error because they had reindexed pg_largeobject, >> an

[GENERAL] Charset Win1250 on Windows and Ubuntu

2009-12-18 Thread Durumdara
Hi! I have a software that uses Postgresql. This program (and website) developed and working on Window (XP/2003), with native charset (win1250). Prior week we got a special request to install this software to a Linux server. Yesterday I installed Ubu9.10 on VirtualBox, and tried to moving the da

Re: [GENERAL] Extended Query using the Frontend/Backend Protocol 3.0

2009-12-18 Thread Raimon Fernandez
On 18/12/2009, at 2:26, John DeSoi wrote: > > On Dec 17, 2009, at 11:13 AM, Raimon Fernandez wrote: > >> I'm trying to integrate the extended query protocol with my libraries. >> >> I'm sending a simple SELECT to validate the method, but I'm getting an >> Invalid Message Format. > > I did no