Re: [GENERAL] simple functions, huge overhead, no cache

2010-07-12 Thread Craig Ringer
On 11/07/2010 5:46 AM, Pavel Stehule wrote: any using a non simple expression is very slow - so there can be some a warning when people use it. Sometimes people don't know (me too), when use expensive expression for example rowvar := (10,20) it isn't simple - I am not sure, if it is true

Re: [GENERAL] Queries about PostgreSQL PITR

2010-07-12 Thread Fujii Masao
On Fri, Jul 9, 2010 at 6:47 PM, Jayadevan M jayadevan.maym...@ibsplc.com wrote: So recovery happened to a point after I dropped the first table and before I dropped the second table. Why ? Because you didn't disable recovery_target_inclusive, I guess.

Re: [GENERAL] simple functions, huge overhead, no cache

2010-07-12 Thread Pavel Stehule
2010/7/12 Craig Ringer cr...@postnewspapers.com.au: On 11/07/2010 5:46 AM, Pavel Stehule wrote: any using a non simple expression is very slow - so there can be some a warning when people use it. Sometimes people don't know (me too), when use expensive expression for example rowvar :=

Re: [GENERAL] simple functions, huge overhead, no cache

2010-07-12 Thread Dave Page
On Mon, Jul 12, 2010 at 7:06 AM, Craig Ringer cr...@postnewspapers.com.au wrote: It seems like a profiler, which is designed to filter and organize the collected data, and which can be attached only to specific functions that you want to know about, might be a better job. As there's already a

Re: [GENERAL] Queries about PostgreSQL PITR

2010-07-12 Thread Jayadevan M
Hi, Because you didn't disable recovery_target_inclusive, I guess. http://www.postgresql.org/docs/8.4/static/continuous-archiving.html#RECOVERY-TARGET-INCLUSIVE Thanks. I was almost sure this will fix it. But the issue seems to be something else. Even if I give a time that is a few more minutes

Re: [GENERAL] PG_DUMP very slow because of STDOUT ??

2010-07-12 Thread Andras Fabian
Hi Tom (or others), are there some recommended settings/ways to use oprofile on a situation like this??? I got it working, have seen a first profile report, but then managed to completely freeze the server on a second try with different oprofile settings (next tests will go against the newly

Re: [GENERAL] Queries about PostgreSQL PITR

2010-07-12 Thread Fujii Masao
On Mon, Jul 12, 2010 at 5:29 PM, Jayadevan M jayadevan.maym...@ibsplc.com wrote: Hi, Because you didn't disable recovery_target_inclusive, I guess. http://www.postgresql.org/docs/8.4/static/continuous-archiving.html#RECOVERY-TARGET-INCLUSIVE Thanks. I was almost sure this will fix it. But the

[GENERAL] PostgreSQL PITR - more doubts

2010-07-12 Thread Jayadevan M
Hello all, One doubt about how PostgreSQL PITR works. Let us say I have all the archived WALs for the past week with archive_command = 'cp -i %p /home/postgres/archive/%f /dev/null' I took a base backup last night. If I try to recover the server today after copying the base backup from

[GENERAL] PostgreSQL 9.0 beta 3 release announcement

2010-07-12 Thread Thom Brown
Could someone clarify the info in this paragraph: Note that, due to a system catalog change, an initdb and database reload will be required for upgrading from 9.0Beta1. We encourage users to use this opportunity to test pg_upgrade for the upgrade from Beta2 or an earlier version of 9.0. Please

Re: [GENERAL] PG_DUMP very slow because of STDOUT ??

2010-07-12 Thread Andras Fabian
This STDOU issue gets even weirder. Now I have set up our two new servers (identical hw/sw) as I would have needed to do so anyways. After having PG running, I also set up the same test scenario as I have it on our problematic servers, and started the COPY-to-STDOUT experiment. And you know

[GENERAL] Configure Postgres From SQL

2010-07-12 Thread Tom Wilcox
Hi, Is it possible to configure postgres from SQL? I am interested in turning off fsync for a set of queries (that take ages to run) and then turn fsync back on again afterwards. Cheers, Tom -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your

Re: [GENERAL] Configure Postgres From SQL

2010-07-12 Thread Thom Brown
On 12 July 2010 14:29, Tom Wilcox hungry...@gmail.com wrote: Hi, Is it possible to configure postgres from SQL? I am interested in turning off fsync for a set of queries (that take ages to run) and then turn fsync back on again afterwards. Cheers, Tom You can only change that option in

Re: [GENERAL] Configure Postgres From SQL

2010-07-12 Thread tv
Hi, Is it possible to configure postgres from SQL? I am interested in turning off fsync for a set of queries (that take ages to run) and then turn fsync back on again afterwards. There are things that can be changed at runtime using SQL - in that case you may just type SET enable_seqscan =

Re: [GENERAL] Configure Postgres From SQL

2010-07-12 Thread Thom Brown
On 12 July 2010 14:50, Tom Wilcox hungry...@gmail.com wrote: Hi Thom, I am performing update statements that are applied to a single table that is about 96GB in size. These updates are grouped together in a single transaction. This transaction runs until the machine runs out of disk space.

Re: [GENERAL] Configure Postgres From SQL

2010-07-12 Thread Adrian Klaver
On Monday 12 July 2010 6:29:14 am Tom Wilcox wrote: Hi, Is it possible to configure postgres from SQL? Yes to a degree, see here: http://www.postgresql.org/docs/8.4/interactive/functions-admin.html I am interested in turning off fsync for a set of queries (that take ages to run) and then

Re: [GENERAL] simple functions, huge overhead, no cache

2010-07-12 Thread Josip Rodin
On Mon, Jul 12, 2010 at 02:06:43PM +0800, Craig Ringer wrote: Meh, personally I'll stick to the good old profiling methods is it fast enough, \timing, and explain analyze. I agree. Some hint could be included in 'explain analyze' output, maybe just to separate the timings for things that are

Re: [GENERAL] simple functions, huge overhead, no cache

2010-07-12 Thread Pavel Stehule
2010/7/12 Josip Rodin j...@entuzijast.net: On Mon, Jul 12, 2010 at 02:06:43PM +0800, Craig Ringer wrote: Meh, personally I'll stick to the good old profiling methods is it fast enough, \timing, and explain analyze. I agree. Some hint could be included in 'explain analyze' output, maybe just

Re: [GENERAL] Configure Postgres From SQL

2010-07-12 Thread Tom Wilcox
Hi Thom, Yeah They can be divided up, but my main issue is that I would like these functions wrapped up so that the client (who has little to no experience using PostgreSQL) can just run a SQL function that will execute all of these updates and prepare many tables and functions for a

Re: [GENERAL] Configure Postgres From SQL

2010-07-12 Thread Guillaume Lelarge
Le 12/07/2010 17:02, Tom Wilcox a écrit : Hi Thom, Yeah They can be divided up, but my main issue is that I would like these functions wrapped up so that the client (who has little to no experience using PostgreSQL) can just run a SQL function that will execute all of these updates and

Re: [GENERAL] Configure Postgres From SQL

2010-07-12 Thread Andres Freund
On Monday 12 July 2010 15:29:14 Tom Wilcox wrote: Hi, Is it possible to configure postgres from SQL? I am interested in turning off fsync for a set of queries (that take ages to run) and then turn fsync back on again afterwards. disabling fsync is nearly never a good idea. What you can

Re: [GENERAL] getting the last N tuples of a query

2010-07-12 Thread Merlin Moncure
On Thu, Jul 8, 2010 at 9:09 PM, Kenichiro Tanaka ketan...@ashisuto.co.jp wrote: Hello. I agree Ben. But,I try your question as an SQL puzzle. Doses this SQL meet what you want? select * from wantlast offset (select count(*)-10 from wantlast); that works, but for any non trivial query it's

Re: [GENERAL] ERROR: canceling statement due to statement timeout

2010-07-12 Thread Tim
On 6 July 2010 16:36, Vick Khera vi...@khera.org wrote: On Fri, Jul 2, 2010 at 12:22 PM, Tim tavs...@gmail.com wrote: I've had a website up for a couple of months and it's starting to get these db timeouts as traffic has increased to say 1k pageviews a day. Are you using any two-phase commit

Re: [GENERAL] simple functions, huge overhead, no cache

2010-07-12 Thread Josip Rodin
On Mon, Jul 12, 2010 at 04:38:48PM +0200, Pavel Stehule wrote: 2010/7/12 Josip Rodin j...@entuzijast.net: On Mon, Jul 12, 2010 at 02:06:43PM +0800, Craig Ringer wrote: Meh, personally I'll stick to the good old profiling methods is it fast enough, \timing, and explain analyze. I agree.

Re: [GENERAL] simple functions, huge overhead, no cache

2010-07-12 Thread Pavel Stehule
2010/7/12 Josip Rodin j...@entuzijast.net: On Mon, Jul 12, 2010 at 04:38:48PM +0200, Pavel Stehule wrote: 2010/7/12 Josip Rodin j...@entuzijast.net: On Mon, Jul 12, 2010 at 02:06:43PM +0800, Craig Ringer wrote: Meh, personally I'll stick to the good old profiling methods is it fast enough,

Re: [GENERAL] Configure Postgres From SQL

2010-07-12 Thread Brad Nicholson
On Mon, 2010-07-12 at 14:57 +0100, Thom Brown wrote: On 12 July 2010 14:50, Tom Wilcox hungry...@gmail.com wrote: Hi Thom, I am performing update statements that are applied to a single table that is about 96GB in size. These updates are grouped together in a single transaction. This

Re: [GENERAL] Configure Postgres From SQL

2010-07-12 Thread Scott Marlowe
On Mon, Jul 12, 2010 at 7:57 AM, Thom Brown thombr...@gmail.com wrote: On 12 July 2010 14:50, Tom Wilcox hungry...@gmail.com wrote: Hi Thom, I am performing update statements that are applied to a single table that is about 96GB in size. These updates are grouped together in a single

Re: [GENERAL] PG_DUMP very slow because of STDOUT ??

2010-07-12 Thread Greg Smith
Andras Fabian wrote: - all fast servers show the COPY process as being in the state Rs (runnable (on run queue)) - on the still slow server, this process is in 9 out of 10 samples in Ds (uninterruptible sleep (usually IO)) I've run into significant performance regressions in PostgreSQL

Re: [GENERAL] Configure Postgres From SQL

2010-07-12 Thread Greg Smith
Andres Freund wrote: What you can change (and that makes quite a bit of sense in some situations) is the synchronous_commit setting. Right. In almost every case where people think they want to disable fsync, what they really should be doing instead is turning off synchronous

Re: [GENERAL] PostgreSQL 9.0 beta 3 release announcement

2010-07-12 Thread Bruce Momjian
Thom Brown wrote: Could someone clarify the info in this paragraph: Note that, due to a system catalog change, an initdb and database reload will be required for upgrading from 9.0Beta1. We encourage users to use this opportunity to test pg_upgrade for the upgrade from Beta2 or an earlier

Re: [GENERAL] PostgreSQL 9.0 beta 3 release announcement

2010-07-12 Thread Thomas Kellerer
Bruce Momjian wrote on 12.07.2010 21:34: Thom Brown wrote: Could someone clarify the info in this paragraph: Note that, due to a system catalog change, an initdb and database reload will be required for upgrading from 9.0Beta1. We encourage users to use this opportunity to test pg_upgrade for

Re: [GENERAL] Configure Postgres From SQL

2010-07-12 Thread Tom Wilcox
I could perform the settings manually (set config, restart svr, execute script, come back 2 days later, reset config, restart svr, execute more script,...), but that sort of defeats the point. My aim to have the simplest, automatic setup possible. Preferably completely contained within

Re: [GENERAL] Configure Postgres From SQL

2010-07-12 Thread Scott Marlowe
Please don't top post. On Mon, Jul 12, 2010 at 2:20 PM, Tom Wilcox hungry...@gmail.com wrote: On 12/07/2010 19:26, Scott Marlowe wrote: On Mon, Jul 12, 2010 at 7:57 AM, Thom Brownthombr...@gmail.com  wrote: On 12 July 2010 14:50, Tom Wilcoxhungry...@gmail.com  wrote: Hi Thom, I am

[GENERAL] Testing 9.0beta3 and pg_upgrade

2010-07-12 Thread Thomas Kellerer
Hi, I'm trying pg_upgrade on my Windows installation and I have two suggestions for the manual regarding pg_upgrade: When specifying directories, pg_upgrade *requires* a forward slash as the path separator. This is (still) uncommon in the Windows world (although Windows does support it) and

[GENERAL] Redundant database objects.

2010-07-12 Thread Andrew Bartley
Hi all, Our project has been running for 10 years now. We have a large number of orphaned or redundant tables, views, and functions, due to many years of inadequate source management. We are running PostgreSQL 8.3.7 on i486-pc-linux-gnu, compiled by GCC gcc-4.3.real (Debian 4.3.2-1.1) 4.3.2

Re: [GENERAL] Redundant database objects.

2010-07-12 Thread Joe Conway
On 07/12/2010 02:40 PM, Andrew Bartley wrote: We have a large number of orphaned or redundant tables, views, and functions, due to many years of inadequate source management. We are running PostgreSQL 8.3.7 on i486-pc-linux-gnu, compiled by GCC gcc-4.3.real (Debian 4.3.2-1.1) 4.3.2 Is

Re: [GENERAL] Redundant database objects.

2010-07-12 Thread Andrew Bartley
Thanks Joe, Unfortunately these views only give me what appears to be a certain time frame. This does not help all that much. It will give a list of tables, indexes and sequences that have been used in the time frame, so that is at least a start. It would be good if there was a timestamp (last

Re: [GENERAL] Redundant database objects.

2010-07-12 Thread Greg Smith
Andrew Bartley wrote: Unfortunately these views only give me what appears to be a certain time frame. This does not help all that much. It will give a list of tables, indexes and sequences that have been used in the time frame, so that is at least a start. You can use pg_stat_reset() to

Re: [GENERAL] Redundant database objects.

2010-07-12 Thread Andrew Bartley
Thanks Greg, It seems that the underlying stats tables are reset on a periodic basis, can i stop this process? Is it a .conf setting? I have had a good look around, nothing sticks out. If I can stop it, then i could use pg_stat_reset() then monitor the stat views over an extended period without

Re: [GENERAL] PG_DUMP very slow because of STDOUT ??

2010-07-12 Thread Scott Marlowe
On Mon, Jul 12, 2010 at 7:03 AM, Andras Fabian fab...@atrada.net wrote: This STDOU issue gets even weirder. Now I have set up our two new servers (identical hw/sw) as I would have needed to do so anyways. After having PG running, I also set up the same test scenario as I have it on our

Re: [GENERAL] PostgreSQL 9.0 beta 3 release announcement

2010-07-12 Thread Bruce Momjian
Thomas Kellerer wrote: Bruce Momjian wrote on 12.07.2010 21:34: Thom Brown wrote: Could someone clarify the info in this paragraph: Note that, due to a system catalog change, an initdb and database reload will be required for upgrading from 9.0Beta1. We encourage users to use this

Re: [GENERAL] PG_DUMP very slow because of STDOUT ??

2010-07-12 Thread Craig Ringer
On 12/07/10 21:03, Andras Fabian wrote: This STDOU issue gets even weirder. Now I have set up our two new servers (identical hw/sw) as I would have needed to do so anyways. After having PG running, I also set up the same test scenario as I have it on our problematic servers, and started the

Re: [GENERAL] Testing 9.0beta3 and pg_upgrade

2010-07-12 Thread Craig Ringer
On 13/07/10 05:29, Thomas Kellerer wrote: I would suggest to either manually change the autocommit mode from within pg_upgrade or to add a note in the manual to disable/remove this setting from psqlrc.conf before running pg_upgrade. Personally I think the first option would be the better one.

Re: [GENERAL] No PL/PHP ? Any reason?

2010-07-12 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 Joshua D. Drake wrote: * No trusted/untrusted versions This is false. There are both. Ah, good news, glad I was misinformed. I'm curious, what mechanism does it use for trusted? * Not even in contrib or pgfoundry or github No. No reason

Re: [GENERAL] No PL/PHP ? Any reason?

2010-07-12 Thread Joshua D. Drake
On Tue, 2010-07-13 at 03:42 +, Greg Sabino Mullane wrote: -BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 Joshua D. Drake wrote: * No trusted/untrusted versions This is false. There are both. Ah, good news, glad I was misinformed. I'm curious, what mechanism does it use

Re: [GENERAL] \COPY ... CSV with hex escapes

2010-07-12 Thread Craig Ringer
On 08/07/10 17:42, Alban Hertroys wrote: On 8 Jul 2010, at 4:21, Craig Ringer wrote: Yes, that's ancient. It is handled quite happily by \copy in csv mode, except that when csv mode is active, \xnn escapes do not seem to be processed. So I can have *either* \xnn escape processing *or*

Re: [GENERAL] No PL/PHP ? Any reason?

2010-07-12 Thread Alvaro Herrera
Excerpts from Joshua D. Drake's message of mar jul 13 00:00:07 -0400 2010: On Tue, 2010-07-13 at 03:42 +, Greg Sabino Mullane wrote: -BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 Joshua D. Drake wrote: * No trusted/untrusted versions This is false. There are both.

[GENERAL] Why can't I see the definition of my relations

2010-07-12 Thread Andrew Falanga
Hi, I just finished defining a couple of tables with PgAdmin III and I'm seeing something peculiar. I'm not sure what the problem is. When I connect to the DB using psql and do \d table I get an error saying that there's not relations by that name. What? When I do, \d I see the tables listed.

Re: [GENERAL] Why can't I see the definition of my relations

2010-07-12 Thread A. Kretschmer
In response to Andrew Falanga : Hi, I just finished defining a couple of tables with PgAdmin III and I'm seeing something peculiar. I'm not sure what the problem is. When I connect to the DB using psql and do \d table I get an error saying that there's not relations by that name. What?

Re: [GENERAL] Redundant database objects.

2010-07-12 Thread Alexander Pyhalov
Hello. When we moved old projects from postgresql 7.x to 8.4, I just looked at modification time for files in base/dboid/toid . So, I could determine, that some databases were inactive (precisely,not updated) for about a year and move them to archive... Andrew Bartley wrote: Is there an