Re: [HACKERS] recent ALTER whatever .. SET SCHEMA refactoring

2013-01-08 Thread Kohei KaiGai
2013/1/7 Robert Haas robertmh...@gmail.com: On Mon, Jan 7, 2013 at 2:14 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Kohei KaiGai escribió: Function and collation are candidates of this special case handling; here are just two kinds of object. Another idea is to add a

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-08 Thread Takeshi Yamamuro
Hi, Why would that be a good tradeoff to make? Larger stored values require more I/O, which is likely to swamp any CPU savings in the compression step. Not to mention that a value once written may be read many times, so the extra I/O cost could be multiplied many times over later on. I

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-08 Thread Takeshi Yamamuro
So why don't we use LZ4? +1 Agree though, I think there're still patent issues there. regards, -- Takeshi Yamamuro NTT Cyber Communications Laboratory Group Software Innovation Center (Open Source Software Center) Tel: +81-3-5860-5057 Fax: +81-3-5463-5490

Re: [HACKERS] lazy_vacuum_heap()'s removal of HEAPTUPLE_DEAD tuples

2013-01-08 Thread Pavan Deolasee
On Tue, Jan 8, 2013 at 8:19 AM, Noah Misch n...@leadboat.com wrote: At that point in the investigation, I realized that the cost of being able to remove entire tuples in lazy_vacuum_heap() greatly exceeds the benefit. Again, the benefit is being able to remove tuples whose inserting

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-08 Thread Takeshi Yamamuro
Hi, (2013/01/07 22:36), Greg Stark wrote: On Mon, Jan 7, 2013 at 10:21 AM, John R Piercepie...@hogranch.com wrote: On 1/7/2013 2:05 AM, Andres Freund wrote: I think there should be enough bits available in the toast pointer to indicate the type of compression. I seem to remember somebody

Re: [HACKERS] PL/Python result object str handler

2013-01-08 Thread Magnus Hagander
On Tue, Jan 8, 2013 at 3:58 AM, Peter Eisentraut pete...@gmx.net wrote: For debugging PL/Python functions, I'm often tempted to write something like rv = plpy.execute(...) plpy.info(rv) which prints something unhelpful like PLyResult object at 0xb461d8d8 By implementing a str handler

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-08 Thread Hannu Krosing
On 01/08/2013 10:19 AM, Takeshi Yamamuro wrote: Hi, (2013/01/07 22:36), Greg Stark wrote: On Mon, Jan 7, 2013 at 10:21 AM, John R Piercepie...@hogranch.com wrote: On 1/7/2013 2:05 AM, Andres Freund wrote: I think there should be enough bits available in the toast pointer to indicate the

Re: [HACKERS] recent ALTER whatever .. SET SCHEMA refactoring

2013-01-08 Thread Robert Haas
On Tue, Jan 8, 2013 at 4:05 AM, Kohei KaiGai kai...@kaigai.gr.jp wrote: Does it make sense an idea to invoke AlterFunctionNamespace_oid() or AlterCollationNamespace_oid() from AlterObjectNamespace_internal() for checks of namespace conflicts? It can handle special cases with keeping modularity

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-08 Thread Robert Haas
On Tue, Jan 8, 2013 at 4:04 AM, Takeshi Yamamuro yamamuro.take...@lab.ntt.co.jp wrote: Apart from my patch, what I care is that the current one might be much slow against I/O. For example, when compressing and writing large values, compressing data (20-40MiB/s) might be a dragger against

Re: [HACKERS] Re: Proposal: Store timestamptz of database creation on pg_database

2013-01-08 Thread Robert Haas
On Fri, Jan 4, 2013 at 1:07 PM, Peter Eisentraut pete...@gmx.net wrote: On 1/3/13 3:26 PM, Robert Haas wrote: It's true, as we've often said here, that leveraging the OS facilities means that we get the benefit of improving OS facilities for free - but it also means that we never exceed what

Re: [HACKERS] recent ALTER whatever .. SET SCHEMA refactoring

2013-01-08 Thread Kohei KaiGai
2013/1/8 Robert Haas robertmh...@gmail.com: On Tue, Jan 8, 2013 at 4:05 AM, Kohei KaiGai kai...@kaigai.gr.jp wrote: Does it make sense an idea to invoke AlterFunctionNamespace_oid() or AlterCollationNamespace_oid() from AlterObjectNamespace_internal() for checks of namespace conflicts? It can

Re: [HACKERS] Extra XLOG in Checkpoint for StandbySnapshot

2013-01-08 Thread Amit Kapila
On Monday, January 07, 2013 7:15 PM Andres Freund wrote: On 2013-01-07 19:03:35 +0530, Amit Kapila wrote: On Monday, January 07, 2013 6:30 PM Simon Riggs wrote: On 7 January 2013 12:39, Amit Kapila amit.kap...@huawei.com wrote: So We can modify to change this in function

Re: [HACKERS] Extra XLOG in Checkpoint for StandbySnapshot

2013-01-08 Thread Andres Freund
On 2013-01-08 19:51:39 +0530, Amit Kapila wrote: On Monday, January 07, 2013 7:15 PM Andres Freund wrote: On 2013-01-07 19:03:35 +0530, Amit Kapila wrote: On Monday, January 07, 2013 6:30 PM Simon Riggs wrote: On 7 January 2013 12:39, Amit Kapila amit.kap...@huawei.com wrote:

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-08 Thread Claudio Freire
On Tue, Jan 8, 2013 at 10:20 AM, Robert Haas robertmh...@gmail.com wrote: On Tue, Jan 8, 2013 at 4:04 AM, Takeshi Yamamuro yamamuro.take...@lab.ntt.co.jp wrote: Apart from my patch, what I care is that the current one might be much slow against I/O. For example, when compressing and writing

Re: [HACKERS] proposal: Set effective_cache_size to greater of .conf value, shared_buffers

2013-01-08 Thread Claudio Freire
On Tue, Jan 8, 2013 at 11:39 AM, Merlin Moncure mmonc...@gmail.com wrote: Reference: http://postgresql.1045698.n5.nabble.com/Simple-join-doesn-t-use-index-td5738689.html This is a pretty common gotcha: user sets shared_buffers but misses the esoteric but important effective_cache_size. ISTM

Re: [HACKERS] json api WIP patch

2013-01-08 Thread Andrew Dunstan
On 01/08/2013 01:45 AM, james wrote: The processing functions have been extended to provide populate_record() and populate_recordset() functions.The latter in particular could be useful in decomposing a piece of json representing an array of flat objects (a fairly common pattern) into a set

Re: [HACKERS] Extra XLOG in Checkpoint for StandbySnapshot

2013-01-08 Thread Amit Kapila
On Tuesday, January 08, 2013 8:01 PM Andres Freund wrote: On 2013-01-08 19:51:39 +0530, Amit Kapila wrote: On Monday, January 07, 2013 7:15 PM Andres Freund wrote: On 2013-01-07 19:03:35 +0530, Amit Kapila wrote: On Monday, January 07, 2013 6:30 PM Simon Riggs wrote: On 7 January

Re: [HACKERS] Re: Proposal: Store timestamptz of database creation on pg_database

2013-01-08 Thread Robert Haas
On Sat, Jan 5, 2013 at 11:04 AM, Stephen Frost sfr...@snowman.net wrote: * Fabrízio de Royes Mello (fabriziome...@gmail.com) wrote: * also we discuss about create two new catalogs, one local and another shared (like pg_description and pg_shdescription) to track creation times of all database

Re: [HACKERS] Extra XLOG in Checkpoint for StandbySnapshot

2013-01-08 Thread Andres Freund
On 2013-01-08 20:33:28 +0530, Amit Kapila wrote: On Tuesday, January 08, 2013 8:01 PM Andres Freund wrote: On 2013-01-08 19:51:39 +0530, Amit Kapila wrote: On Monday, January 07, 2013 7:15 PM Andres Freund wrote: On 2013-01-07 19:03:35 +0530, Amit Kapila wrote: On Monday, January

[HACKERS] Weird Assert failure in GetLockStatusData()

2013-01-08 Thread Tom Lane
This is a bit disturbing: http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=bushpigdt=2013-01-07%2019%3A15%3A02 The key bit is [50eb2156.651e:6] LOG: execute isolationtester_waiting: SELECT 1 FROM pg_locks holder, pg_locks waiter WHERE NOT waiter.granted AND waiter.pid = $1 AND

Re: [HACKERS] PL/Python result object str handler

2013-01-08 Thread Daniele Varrazzo
On Tue, Jan 8, 2013 at 9:32 AM, Magnus Hagander mag...@hagander.net wrote: On Tue, Jan 8, 2013 at 3:58 AM, Peter Eisentraut pete...@gmx.net wrote: For debugging PL/Python functions, I'm often tempted to write something like rv = plpy.execute(...) plpy.info(rv) which prints something

[HACKERS] pg_upgrade regression test litters the source tree with log files

2013-01-08 Thread Tom Lane
In a tree in which I previously ran make check in contrib/pg_upgrade: $ make -s distclean $ git status # On branch master # Untracked files: # (use git add file... to include in what will be committed) # # contrib/pg_upgrade/pg_upgrade_dump_1.log #

Re: [HACKERS] Cascading replication: should we detect/prevent cycles?

2013-01-08 Thread Josh Berkus
On 1/5/13 1:21 PM, Peter Geoghegan wrote: On 21 December 2012 14:08, Robert Haas robertmh...@gmail.com wrote: I'm sure it's possible; I don't *think* it's terribly easy. I'm inclined to agree that this isn't a terribly pressing issue. Certainly, the need to introduce a bunch of new

[HACKERS] [PATCH] xlogreader-v4

2013-01-08 Thread Andres Freund
From: Andres Freund and...@2ndquadrant.com Subject: [PATCH] xlogreader-v4 In-Reply-To: Hi, this is the latest and obviously best version of xlogreader xlogdump with changes both from Heikki and me. Changes: * windows build support for pg_xlogdump * xlogdump moved to contrib * xlogdump option

[HACKERS] [PATCH 1/5] Centralize Assert* macros into c.h so its common between backend/frontend

2013-01-08 Thread Andres Freund
From: Andres Freund and...@anarazel.de c.h already had parts of the assert support (StaticAssert*) and its the shared file between postgres.h and postgres_fe.h. This makes it easier to build frontend programs which have to do the hack. --- src/include/c.h | 65

[HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Andres Freund
From: Andres Freund and...@anarazel.de relpathbackend() (via some of its wrappers) is used in *_desc routines which we want to be useable without a backend environment arround. Change signature to return a 'const char *' to make misuse easier to detect. That necessicates also changing the

[HACKERS] [PATCH 5/5] remove spurious space in running_xact's _desc function

2013-01-08 Thread Andres Freund
From: Andres Freund and...@anarazel.de --- src/backend/access/rmgrdesc/standbydesc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/backend/access/rmgrdesc/standbydesc.c b/src/backend/access/rmgrdesc/standbydesc.c index c38892b..5fb6f54 100644 ---

[HACKERS] [PATCH 4/5] Add pg_xlogdump contrib module

2013-01-08 Thread Andres Freund
From: Andres Freund and...@anarazel.de Authors: Andres Freund, Heikki Linnakangas --- contrib/Makefile | 1 + contrib/pg_xlogdump/Makefile | 37 +++ contrib/pg_xlogdump/compat.c | 58 contrib/pg_xlogdump/pg_xlogdump.c | 654

Re: [HACKERS] [PATCH] xlogreader-v4

2013-01-08 Thread Thom Brown
On 8 January 2013 19:09, Andres Freund and...@2ndquadrant.com wrote: From: Andres Freund and...@2ndquadrant.com Subject: [PATCH] xlogreader-v4 In-Reply-To: Hi, this is the latest and obviously best version of xlogreader xlogdump with changes both from Heikki and me. Aren't you

Re: [HACKERS] [PATCH] xlogreader-v4

2013-01-08 Thread Thom Brown
On 8 January 2013 19:15, Thom Brown t...@linux.com wrote: On 8 January 2013 19:09, Andres Freund and...@2ndquadrant.com wrote: From: Andres Freund and...@2ndquadrant.com Subject: [PATCH] xlogreader-v4 In-Reply-To: Hi, this is the latest and obviously best version of xlogreader xlogdump

Re: [HACKERS] [PATCH 1/5] Centralize Assert* macros into c.h so its common between backend/frontend

2013-01-08 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: From: Andres Freund and...@anarazel.de c.h already had parts of the assert support (StaticAssert*) and its the shared file between postgres.h and postgres_fe.h. This makes it easier to build frontend programs which have to do the hack. This patch

Re: [HACKERS] [PATCH] xlogreader-v4

2013-01-08 Thread Andres Freund
On 2013-01-08 20:09:42 +0100, Andres Freund wrote: From: Andres Freund and...@2ndquadrant.com Subject: [PATCH] xlogreader-v4 In-Reply-To: Hi, this is the latest and obviously best version of xlogreader xlogdump with changes both from Heikki and me. Changes: * windows build support

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: maxpg From: Andres Freund and...@anarazel.de relpathbackend() (via some of its wrappers) is used in *_desc routines which we want to be useable without a backend environment arround. I'm 100% unimpressed with making relpathbackend return a pointer

[HACKERS] Re: [PATCH 1/5] Centralize Assert* macros into c.h so its common between backend/frontend

2013-01-08 Thread Andres Freund
On 2013-01-08 14:25:06 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: From: Andres Freund and...@anarazel.de c.h already had parts of the assert support (StaticAssert*) and its the shared file between postgres.h and postgres_fe.h. This makes it easier to build

Re: [HACKERS] [PATCH 1/5] Centralize Assert* macros into c.h so its common between backend/frontend

2013-01-08 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2013-01-08 14:25:06 -0500, Tom Lane wrote: This patch seems unnecessary given that we already put a version of Assert() into postgres_fe.h. The problem is that some (including existing) pieces of code need to include postgres.h itself, those

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Andres Freund
On 2013-01-08 14:28:14 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: maxpg From: Andres Freund and...@anarazel.de relpathbackend() (via some of its wrappers) is used in *_desc routines which we want to be useable without a backend environment arround. I'm 100%

Re: [HACKERS] Cascading replication: should we detect/prevent cycles?

2013-01-08 Thread Simon Riggs
On 8 January 2013 18:46, Josh Berkus j...@agliodbs.com wrote: On 1/5/13 1:21 PM, Peter Geoghegan wrote: On 21 December 2012 14:08, Robert Haas robertmh...@gmail.com wrote: I'm sure it's possible; I don't *think* it's terribly easy. I'm inclined to agree that this isn't a terribly pressing

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2013-01-08 14:28:14 -0500, Tom Lane wrote: I'm 100% unimpressed with making relpathbackend return a pointer to a static buffer. Who's to say whether that won't create bugs due to overlapping usages? I say it ;). I've gone through all callers

Re: [HACKERS] Cascading replication: should we detect/prevent cycles?

2013-01-08 Thread David Fetter
On Tue, Jan 08, 2013 at 10:46:12AM -0800, Josh Berkus wrote: On 1/5/13 1:21 PM, Peter Geoghegan wrote: On 21 December 2012 14:08, Robert Haas robertmh...@gmail.com wrote: I'm sure it's possible; I don't *think* it's terribly easy. I'm inclined to agree that this isn't a terribly pressing

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Andres Freund
On 2013-01-08 14:53:29 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2013-01-08 14:28:14 -0500, Tom Lane wrote: I'm 100% unimpressed with making relpathbackend return a pointer to a static buffer. Who's to say whether that won't create bugs due to overlapping

Re: [HACKERS] json api WIP patch

2013-01-08 Thread james
I had been wondering how to do such an insertion efficiently in the context of SPI, but it seems that there is no SPI_copy equiv that would allow a query parse and plan to be avoided. Your query above would need to be planned too, although the plan will be trivial. Ah yes, I meant that I

Re: [HACKERS] json api WIP patch

2013-01-08 Thread Andrew Dunstan
On 01/08/2013 09:58 AM, Andrew Dunstan wrote: If you have such a datum, parsing it involves having it in memory and then taking a copy (I wonder if we could avoid that step - will take a look). Here is a Proof Of Concept patch against my development tip on what's involved in getting the

[HACKERS] Re: [PATCH 1/5] Centralize Assert* macros into c.h so its common between backend/frontend

2013-01-08 Thread Andres Freund
On 2013-01-08 14:35:12 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2013-01-08 14:25:06 -0500, Tom Lane wrote: This patch seems unnecessary given that we already put a version of Assert() into postgres_fe.h. The problem is that some (including existing)

Re: [HACKERS] json api WIP patch

2013-01-08 Thread Andrew Dunstan
On 01/08/2013 03:12 PM, Andrew Dunstan wrote: On 01/08/2013 09:58 AM, Andrew Dunstan wrote: If you have such a datum, parsing it involves having it in memory and then taking a copy (I wonder if we could avoid that step - will take a look). Here is a Proof Of Concept patch against my

Re: [HACKERS] json api WIP patch

2013-01-08 Thread Andrew Dunstan
On 01/08/2013 03:07 PM, james wrote: Yes - but I don't think I can use COPY from a stored proc context can I? If I could use binary COPY from a stored proc that has received a binary param and unpacked to the data, it would be handy. You can use COPY from a stored procedure, but only

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: Uhm, we don't have need palloc support and I don't think relpathbackend() is a good justification for adding it. I've said from the very beginning of this effort that it would be impossible to share any meaningful amount of code between frontend and

Re: [HACKERS] Cascading replication: should we detect/prevent cycles?

2013-01-08 Thread Simon Riggs
On 8 January 2013 19:53, David Fetter da...@fetter.org wrote: On Tue, Jan 08, 2013 at 10:46:12AM -0800, Josh Berkus wrote: On 1/5/13 1:21 PM, Peter Geoghegan wrote: On 21 December 2012 14:08, Robert Haas robertmh...@gmail.com wrote: I'm sure it's possible; I don't *think* it's terribly easy.

Re: [HACKERS] Re: [PATCH 1/5] Centralize Assert* macros into c.h so its common between backend/frontend

2013-01-08 Thread Alvaro Herrera
Andres Freund wrote: On 2013-01-08 14:35:12 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2013-01-08 14:25:06 -0500, Tom Lane wrote: This patch seems unnecessary given that we already put a version of Assert() into postgres_fe.h. The problem is that

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Alvaro Herrera
Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: Uhm, we don't have need palloc support and I don't think relpathbackend() is a good justification for adding it. I've said from the very beginning of this effort that it would be impossible to share any meaningful amount of

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Andres Freund
On 2013-01-08 15:27:23 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: Uhm, we don't have need palloc support and I don't think relpathbackend() is a good justification for adding it. I've said from the very beginning of this effort that it would be impossible to

Re: [HACKERS] Re: [PATCH 1/5] Centralize Assert* macros into c.h so its common between backend/frontend

2013-01-08 Thread Andres Freund
On 2013-01-08 17:36:19 -0300, Alvaro Herrera wrote: Andres Freund wrote: On 2013-01-08 14:35:12 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2013-01-08 14:25:06 -0500, Tom Lane wrote: This patch seems unnecessary given that we already put a version of

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: To what extent do you want palloc et al. emulation? Provide actual pools or just make redirect to malloc and provide the required symbols (at the very least CurrentMemoryContext)? I don't see any need for memory pools, at least not for frontend

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Heikki Linnakangas
On 08.01.2013 22:39, Andres Freund wrote: On 2013-01-08 15:27:23 -0500, Tom Lane wrote: Andres Freundand...@2ndquadrant.com writes: Uhm, we don't have need palloc support and I don't think relpathbackend() is a good justification for adding it. I've said from the very beginning of this

Re: [HACKERS] Re: [PATCH 1/5] Centralize Assert* macros into c.h so its common between backend/frontend

2013-01-08 Thread Tom Lane
Alvaro Herrera alvhe...@2ndquadrant.com writes: Andres Freund wrote: Sorry, misremembered the problem somewhat. The problem is that code that includes postgres.h atm ends up with ExceptionalCondition() et al. declared even if FRONTEND is defined. So if anything uses an assert you need to

Re: [HACKERS] PATCH: optimized DROP of multiple tables within a transaction

2013-01-08 Thread Tomas Vondra
On 8.1.2013 03:47, Shigeru Hanada wrote: * naming of DROP_RELATIONS_BSEARCH_LIMIT (or off-by-one bug?) IIUC bsearch is used when # of relations to be dropped is *more* than the value of DROP_RELATIONS_BSEARCH_LIMIT (10). IMO this behavior is not what the macro name implies; I thought that

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Andres Freund
On 2013-01-08 22:47:43 +0200, Heikki Linnakangas wrote: On 08.01.2013 22:39, Andres Freund wrote: On 2013-01-08 15:27:23 -0500, Tom Lane wrote: Andres Freundand...@2ndquadrant.com writes: Uhm, we don't have need palloc support and I don't think relpathbackend() is a good justification for

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Heikki Linnakangas
On 08.01.2013 23:00, Andres Freund wrote: Note that the xlogreader facility doesn't need any more emulation. I'm quite satisfied with that part of the patch now. However, the rmgr desc routines do, and I'm not very happy with those. Not sure what to do about it. As you said, we could add enough

Re: [HACKERS] pg_upgrade regression test litters the source tree with log files

2013-01-08 Thread Bruce Momjian
On Tue, Jan 8, 2013 at 01:08:44PM -0500, Tom Lane wrote: In a tree in which I previously ran make check in contrib/pg_upgrade: $ make -s distclean $ git status # On branch master # Untracked files: # (use git add file... to include in what will be committed) # #

Re: [HACKERS] pg_upgrade regression test litters the source tree with log files

2013-01-08 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: On Tue, Jan 8, 2013 at 01:08:44PM -0500, Tom Lane wrote: In a tree in which I previously ran make check in contrib/pg_upgrade: $ make -s distclean $ git status # On branch master # Untracked files: # (use git add file... to include in what will be

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Andres Freund
On 2013-01-08 15:45:07 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: To what extent do you want palloc et al. emulation? Provide actual pools or just make redirect to malloc and provide the required symbols (at the very least CurrentMemoryContext)? I don't see any

Re: [HACKERS] pg_upgrade regression test litters the source tree with log files

2013-01-08 Thread Peter Eisentraut
On 1/8/13 4:04 PM, Bruce Momjian wrote: On Tue, Jan 8, 2013 at 01:08:44PM -0500, Tom Lane wrote: In a tree in which I previously ran make check in contrib/pg_upgrade: $ make -s distclean $ git status # On branch master # Untracked files: # (use git add file... to include in what will be

Re: [HACKERS] pg_upgrade regression test litters the source tree with log files

2013-01-08 Thread Bruce Momjian
On Tue, Jan 8, 2013 at 04:08:42PM -0500, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: On Tue, Jan 8, 2013 at 01:08:44PM -0500, Tom Lane wrote: In a tree in which I previously ran make check in contrib/pg_upgrade: $ make -s distclean $ git status # On branch master #

Re: [HACKERS] pg_upgrade regression test litters the source tree with log files

2013-01-08 Thread Bruce Momjian
On Tue, Jan 8, 2013 at 04:11:41PM -0500, Peter Eisentraut wrote: On 1/8/13 4:04 PM, Bruce Momjian wrote: On Tue, Jan 8, 2013 at 01:08:44PM -0500, Tom Lane wrote: In a tree in which I previously ran make check in contrib/pg_upgrade: $ make -s distclean $ git status # On branch master

[HACKERS] Re: [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Andres Freund
On 2013-01-08 23:02:15 +0200, Heikki Linnakangas wrote: On 08.01.2013 23:00, Andres Freund wrote: Note that the xlogreader facility doesn't need any more emulation. I'm quite satisfied with that part of the patch now. However, the rmgr desc routines do, and I'm not very happy with those. Not

Re: [HACKERS] json api WIP patch

2013-01-08 Thread Peter Eisentraut
On 1/7/13 5:15 PM, Andrew Dunstan wrote: You (Merlin) have kindly volunteered to work on documentation, so before we go too far with that any bikeshedding on names, or on the functionality being provided, should now take place. Hmm, I was going to say, this patch contains no documentation, so

Re: [HACKERS] PL/Python result object str handler

2013-01-08 Thread Peter Eisentraut
On 1/8/13 4:32 AM, Magnus Hagander wrote: How does it work if there are many rows in there? Say the result contains 10,000 rows - will the string contain all of them? If so, might it be worthwhile to cap the number of rows shown and then follow with a ... or something? I don't think so. Any

Re: [HACKERS] PL/Python result object str handler

2013-01-08 Thread Magnus Hagander
On Tue, Jan 8, 2013 at 10:23 PM, Peter Eisentraut pete...@gmx.net wrote: On 1/8/13 4:32 AM, Magnus Hagander wrote: How does it work if there are many rows in there? Say the result contains 10,000 rows - will the string contain all of them? If so, might it be worthwhile to cap the number of

Re: [HACKERS] PL/Python result object str handler

2013-01-08 Thread Peter Eisentraut
On 1/8/13 11:55 AM, Daniele Varrazzo wrote: PLyResult status=5 nrows=2 rows=[{'foo': 1, 'bar': '11'}, {'foo': 2, 'bar': '22'}] This looks more a repr-style format to me (if you implement repr but not str, the latter will default to the former). The repr style was the only guideline I found.

Re: [HACKERS] PATCH: optimized DROP of multiple tables within a transaction

2013-01-08 Thread Alvaro Herrera
Tomas Vondra wrote: On 8.1.2013 03:47, Shigeru Hanada wrote: * +1 for Alvaro's suggestion about avoiding palloc traffic by starting with 8 elements or so. Done. Not yet. Initial size of srels array is still 1 element. I've checked the patch and I see there 'maxrels = 8' - or do

Re: [HACKERS] json api WIP patch

2013-01-08 Thread james
You can use COPY from a stored procedure, but only to and from files. I think that's in the chocolate fireguard realm though as far as efficiency for this sort of scenario goes, even if its handled by retaining an mmap'd file as workspace. If SPI provided a way to perform a copy to a

Re: [HACKERS] json api WIP patch

2013-01-08 Thread Merlin Moncure
On Tue, Jan 8, 2013 at 3:19 PM, Peter Eisentraut pete...@gmx.net wrote: On 1/7/13 5:15 PM, Andrew Dunstan wrote: You (Merlin) have kindly volunteered to work on documentation, so before we go too far with that any bikeshedding on names, or on the functionality being provided, should now take

Re: [HACKERS] PATCH: optimized DROP of multiple tables within a transaction

2013-01-08 Thread Tomas Vondra
On 8.1.2013 22:30, Alvaro Herrera wrote: Tomas Vondra wrote: On 8.1.2013 03:47, Shigeru Hanada wrote: * +1 for Alvaro's suggestion about avoiding palloc traffic by starting with 8 elements or so. Done. Not yet. Initial size of srels array is still 1 element. I've checked the patch and

Re: [HACKERS] Weird Assert failure in GetLockStatusData()

2013-01-08 Thread Tom Lane
I wrote: This is a bit disturbing: http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=bushpigdt=2013-01-07%2019%3A15%3A02 ... The assertion failure seems to indicate that the number of LockMethodProcLockHash entries found by hash_seq_search didn't match the number that had been counted

Re: [HACKERS] json api WIP patch

2013-01-08 Thread Andrew Dunstan
On 01/08/2013 04:32 PM, Merlin Moncure wrote: On Tue, Jan 8, 2013 at 3:19 PM, Peter Eisentrautpete...@gmx.net wrote: On 1/7/13 5:15 PM, Andrew Dunstan wrote: You (Merlin) have kindly volunteered to work on documentation, so before we go too far with that any bikeshedding on names, or on the

Re: [HACKERS] Cascading replication: should we detect/prevent cycles?

2013-01-08 Thread Daniel Farina
On Tue, Jan 8, 2013 at 11:51 AM, Simon Riggs si...@2ndquadrant.com wrote: On 8 January 2013 18:46, Josh Berkus j...@agliodbs.com wrote: On 1/5/13 1:21 PM, Peter Geoghegan wrote: On 21 December 2012 14:08, Robert Haas robertmh...@gmail.com wrote: I'm sure it's possible; I don't *think* it's

Re: [HACKERS] Re: Proposal: Store timestamptz of database creation on pg_database

2013-01-08 Thread Peter Eisentraut
On 1/5/13 11:04 AM, Stephen Frost wrote: Creating a separate catalog (or two) every time we want to track XYZ for all objects is rather overkill... Thinking about this a bit more, and noting that pg_description/shdescription more-or-less already exist as a framework for tracking 'something'

[HACKERS] Index build temp files

2013-01-08 Thread Stephen Frost
Greetings, We were surprised recently to note that the temp files that are created during a CREATE INDEX don't go into either a temp tablespace set for the user or into the tablespace which the CREATE INDEX specifies. Instead, they go only into base/pgsql_tmp/. This doesn't allow for

Re: [HACKERS] I s this a bug of spgist index in a heavy write condition?

2013-01-08 Thread Tom Lane
=?gb2312?B?wO66o8H6?= hailong...@qunar.com writes: I am very excited to say that I may have created a test case! I've been running variants of this example for most of the afternoon, and have not seen a failure :-(. So I'm afraid there is some aspect of your situation that you've not provided

Re: [HACKERS] Re: Proposal: Store timestamptz of database creation on pg_database

2013-01-08 Thread Pavel Stehule
2013/1/8 Peter Eisentraut pete...@gmx.net: On 1/5/13 11:04 AM, Stephen Frost wrote: Creating a separate catalog (or two) every time we want to track XYZ for all objects is rather overkill... Thinking about this a bit more, and noting that pg_description/shdescription more-or-less already

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-08 Thread Robert Haas
On Tue, Jan 8, 2013 at 9:51 AM, Claudio Freire klaussfre...@gmail.com wrote: On Tue, Jan 8, 2013 at 10:20 AM, Robert Haas robertmh...@gmail.com wrote: On Tue, Jan 8, 2013 at 4:04 AM, Takeshi Yamamuro yamamuro.take...@lab.ntt.co.jp wrote: Apart from my patch, what I care is that the current one

Re: [HACKERS] Index build temp files

2013-01-08 Thread Bruce Momjian
On Tue, Jan 8, 2013 at 05:09:47PM -0500, Stephen Frost wrote: Greetings, We were surprised recently to note that the temp files that are created during a CREATE INDEX don't go into either a temp tablespace set for the user or into the tablespace which the CREATE INDEX specifies.

Re: [HACKERS] Re: Proposal: Store timestamptz of database creation on pg_database

2013-01-08 Thread Stephen Frost
* Pavel Stehule (pavel.steh...@gmail.com) wrote: 2013/1/8 Peter Eisentraut pete...@gmx.net: On 1/5/13 11:04 AM, Stephen Frost wrote: Yeah, actually, the other day I was thinking we should get rid of all the system catalogs and use a big EAV-like schema instead. We're not getting any

Re: [HACKERS] Index build temp files

2013-01-08 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: On Tue, Jan 8, 2013 at 05:09:47PM -0500, Stephen Frost wrote: Greetings, We were surprised recently to note that the temp files that are created during a CREATE INDEX don't go into either a temp tablespace set for the user or into the

Re: [HACKERS] proposal: Set effective_cache_size to greater of .conf value, shared_buffers

2013-01-08 Thread Robert Haas
On Tue, Jan 8, 2013 at 9:53 AM, Claudio Freire klaussfre...@gmail.com wrote: On Tue, Jan 8, 2013 at 11:39 AM, Merlin Moncure mmonc...@gmail.com wrote: Reference: http://postgresql.1045698.n5.nabble.com/Simple-join-doesn-t-use-index-td5738689.html This is a pretty common gotcha: user sets

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Robert Haas
On Tue, Jan 8, 2013 at 3:00 PM, Andres Freund and...@2ndquadrant.com wrote: Uhm, we don't have need palloc support and I don't think relpathbackend() is a good justification for adding it. FWIW, I'm with Tom on this one. Any meaningful code sharing is going to need that, so we might as well

Re: [HACKERS] Weird Assert failure in GetLockStatusData()

2013-01-08 Thread Tom Lane
I wrote: After digging around a bit, I can find only one place where it looks like somebody might be messing with the LockMethodProcLockHash table while not holding the appropriate lock-partition LWLock(s): 1. VirtualXactLock finds target xact holds its VXID lock fast-path. 2.

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Alvaro Herrera
Robert Haas escribió: And functions that return static buffers are evil incarnate. I've spent way too much of my life dealing with the supreme idiocy that is fmtId(). +1 If someone ever finds a way to make that go away, I will buy them a beverage of their choice at the next conference

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Andres Freund
On 2013-01-08 17:28:33 -0500, Robert Haas wrote: On Tue, Jan 8, 2013 at 3:00 PM, Andres Freund and...@2ndquadrant.com wrote: Uhm, we don't have need palloc support and I don't think relpathbackend() is a good justification for adding it. FWIW, I'm with Tom on this one. Any meaningful code

Re: [HACKERS] proposal: Set effective_cache_size to greater of .conf value, shared_buffers

2013-01-08 Thread Bruce Momjian
On Tue, Jan 8, 2013 at 05:23:36PM -0500, Robert Haas wrote: Rather, I'd propose the default setting should be -1 or something default and automagic that works most of the time (but not all). +1. I've found that a value of three-quarters of system memory works pretty well most of the

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: And functions that return static buffers are evil incarnate. I've spent way too much of my life dealing with the supreme idiocy that is fmtId(). If someone ever finds a way to make that go away, I will buy them a beverage of their choice at the next

Re: [HACKERS] Index build temp files

2013-01-08 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: * Bruce Momjian (br...@momjian.us) wrote: Well, our docs for temp_tablespaces says: This variable specifies tablespaces in which to create temporary objects (temp tables and indexes on temp tables) when a commandCREATE/ command does not explicitly

Re: [HACKERS] Index build temp files

2013-01-08 Thread Simon Riggs
On 9 January 2013 00:04, Tom Lane t...@sss.pgh.pa.us wrote: I don't think it's a bug. What you seem to be proposing is that CREATE INDEX ought to ignore temp_tablespaces and instead always put its temp files in the tablespace where the finished index will reside. I don't think that's what he

Re: [HACKERS] proposal: Set effective_cache_size to greater of .conf value, shared_buffers

2013-01-08 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Jan 8, 2013 at 9:53 AM, Claudio Freire klaussfre...@gmail.com wrote: Rather, I'd propose the default setting should be -1 or something default and automagic that works most of the time (but not all). A cruder heuristic that might be useful is

Re: [HACKERS] Index build temp files

2013-01-08 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On 9 January 2013 00:04, Tom Lane t...@sss.pgh.pa.us wrote: I don't think it's a bug. What you seem to be proposing is that CREATE INDEX ought to ignore temp_tablespaces and instead always put its temp files in the tablespace where the finished index

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Robert Haas
On Tue, Jan 8, 2013 at 6:23 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: And functions that return static buffers are evil incarnate. I've spent way too much of my life dealing with the supreme idiocy that is fmtId(). If someone ever finds a way to make

Re: [HACKERS] proposal: Set effective_cache_size to greater of .conf value, shared_buffers

2013-01-08 Thread Robert Haas
On Tue, Jan 8, 2013 at 7:17 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Jan 8, 2013 at 9:53 AM, Claudio Freire klaussfre...@gmail.com wrote: Rather, I'd propose the default setting should be -1 or something default and automagic that works most of

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: I was thinking more about a sprintf()-type function that only understands a handful of escapes, but adds the additional and novel escapes %I (quote as identifier) and %L (quote as literal). I think that would allow a great deal of code simplification,

Re: [HACKERS] [PATCH 2/5] Make relpathbackend return a statically result instead of palloc()'ing it

2013-01-08 Thread Robert Haas
On Tue, Jan 8, 2013 at 7:57 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: I was thinking more about a sprintf()-type function that only understands a handful of escapes, but adds the additional and novel escapes %I (quote as identifier) and %L (quote as

Re: [HACKERS] proposal: Set effective_cache_size to greater of .conf value, shared_buffers

2013-01-08 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Jan 8, 2013 at 7:17 PM, Tom Lane t...@sss.pgh.pa.us wrote: ... And I don't especially like the idea of trying to make it depend directly on the box's physical RAM, for the same practical reasons Robert mentioned. For the record, I don't

Re: [HACKERS] pg_dump transaction's read-only mode

2013-01-08 Thread Gurjeet Singh
On Mon, Dec 31, 2012 at 1:38 AM, Pavan Deolasee pavan.deola...@gmail.comwrote: On Sun, Dec 30, 2012 at 12:38 AM, Stephen Frost sfr...@snowman.net wrote: * Pavan Deolasee (pavan.deola...@gmail.com) wrote: On Fri, Sep 7, 2012 at 6:06 PM, Kevin Grittner That makes sense to me. The reason I

  1   2   >