Re: [HACKERS] Transaction-scope advisory locks

2010-12-14 Thread Marko Tiikkaja
On 2010-12-14 4:23 AM +0200, Tom Lane wrote: Marko Tiikkajamarko.tiikk...@cs.helsinki.fi writes: On 2010-12-14 1:08 AM +0200, Szymon Guz wrote: In my opinion changing current behavior is not a good idea. I know some software that relies on current behavior and this would break it. Maybe add

Re: [HACKERS] pg_execute_from_file, patch v10

2010-12-14 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: Has anyone thought twice about the security implications of that? Not to mention that in most cases, the very last thing we want is to have to specify an exact full path? Well, the security is left same as before, superuser only. And Itagaki showed that

Re: [HACKERS] pg_execute_from_file, patch v10

2010-12-14 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: CREATE EXTENSION will be superuser to start with, no doubt, but I think we'll someday want to allow it to database owners, just as happened with CREATE LANGUAGE. Let's not build it on top of operations that inherently involve security problems, especially

Re: [HACKERS] pg_execute_from_file, patch v10

2010-12-14 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: Well, I think it is best when a patch has just one purpose. This seems to be sort of an odd hodge-podge of things. The purpose here is clean-up the existing pg_read_file() facility so that it's easy to build pg_execute_sql_file() on top of it.

Re: [HACKERS] Transaction-scope advisory locks

2010-12-14 Thread Dimitri Fontaine
Marko Tiikkaja marko.tiikk...@cs.helsinki.fi writes: I often find myself wanting advisory locks that are automatically released when the transaction ends, so here's a small patch trying to do just that. Excellent idea, I sure need that (been doing some pl stuff to track locks granted then

Re: [HACKERS] Transaction-scope advisory locks

2010-12-14 Thread Andres Freund
On Tuesday 14 December 2010 00:14:22 Marko Tiikkaja wrote: The lock space is the same though, but I don't feel strongly about it. I feel strongly that it needs the same locking space. I pretty frequently have the need for multiple clients trying to acquiring a lock in transaction scope (i.e.

Re: [HACKERS] hstores in pl/python

2010-12-14 Thread Peter Eisentraut
On mån, 2010-12-13 at 10:55 -0500, Tom Lane wrote: We don't normally invent specialized syntax for a specific datatype. Not even if it's in core. I think the idea would be to make associative arrays a kind of second-order object like arrays, instead of a data type. -- Sent via pgsql-hackers

Re: [HACKERS] Instrument checkpoint sync calls

2010-12-14 Thread Greg Smith
Robert Haas wrote: I took a look at this and it looks generally good, but I'm wondering why md.c is converting the results from an exact value to a floating point, only to have xlog.c turn around and convert back to an integer. I think it could just return milliseconds directly, or if you're

Re: [HACKERS] CommitFest wrap-up

2010-12-14 Thread Florian Pflug
On Dec13, 2010, at 18:37 , Robert Haas wrote: We're now just a day or two from the end of this CommitFest and there are still a LOT of open patches - to be specific, 23.Here's a brief synopsis of where we are with the others, all IMO of course. Thanks for putting this together! - serializable

Re: [HACKERS] hstores in pl/python

2010-12-14 Thread Peter Eisentraut
On mån, 2010-12-13 at 08:50 +0100, Jan Urbański wrote: It would be cool to be able to transparently use hstores as Python dictionaries and vice versa. It would be easy enough with hstore as a core type, but with hstore as an addon it's not that easy. I have been thinking about this class of

Re: [HACKERS] SQL/MED - core functionality

2010-12-14 Thread Peter Eisentraut
On mån, 2010-12-13 at 14:45 +0900, Shigeru HANADA wrote: Simple FDWs such as File FDW might not have concept of user on remote side. In such case, it would be enough to control access privilege per local user with GRANT/REVOKE SELECT statement. Right. But it depends on the implementation.

Re: [HACKERS] CommitFest wrap-up

2010-12-14 Thread Robert Haas
On Tue, Dec 14, 2010 at 7:51 AM, Florian Pflug f...@phlo.org wrote: - serializable lock consistency - I am fairly certain this needs rebasing.  I don't have time to deal with it right away.  That sucks, because I think this is a really important change. I can try to find some time to update

Re: [HACKERS] SQL/MED - core functionality

2010-12-14 Thread Shigeru HANADA
On Mon, 13 Dec 2010 21:51:40 -0500 Robert Haas robertmh...@gmail.com wrote: This actually doesn't apply cleanly. There's a hunk in pg_class.h that is failing. I might have missed recent changes about pg_class.relistemp. I've fixed in local repo. I think attgenoptions is a poor choice of name

Re: [HACKERS] Transaction-scope advisory locks

2010-12-14 Thread Merlin Moncure
On Tue, Dec 14, 2010 at 7:07 AM, Andres Freund and...@anarazel.de wrote: On Tuesday 14 December 2010 00:14:22 Marko Tiikkaja wrote: The lock space is the same though, but I don't feel strongly about it. I feel strongly that it needs the same locking space. I pretty frequently have the need for

Re: [HACKERS] Transaction-scope advisory locks

2010-12-14 Thread Marko Tiikkaja
On 2010-12-14 4:19 PM +0200, Merlin Moncure wrote: On Tue, Dec 14, 2010 at 7:07 AM, Andres Freundand...@anarazel.de wrote: On Tuesday 14 December 2010 00:14:22 Marko Tiikkaja wrote: The lock space is the same though, but I don't feel strongly about it. I feel strongly that it needs the same

Re: [HACKERS] Instrument checkpoint sync calls

2010-12-14 Thread Alvaro Herrera
I gave this patch a look and it seems pretty good to me, except that I'm uncomfortable with the idea of mdsync filling in the details for CheckpointStats fields directly. Would it work to pass a struct (say SmgrSyncStats) from CheckPointBuffers to smgrsync and from there to mdsync, have this

Re: [HACKERS] Transaction-scope advisory locks

2010-12-14 Thread Andres Freund
On Tuesday 14 December 2010 15:19:32 Merlin Moncure wrote: On Tue, Dec 14, 2010 at 7:07 AM, Andres Freund and...@anarazel.de wrote: On Tuesday 14 December 2010 00:14:22 Marko Tiikkaja wrote: The lock space is the same though, but I don't feel strongly about it. I feel strongly that it

Re: [HACKERS] Instrument checkpoint sync calls

2010-12-14 Thread Robert Haas
On Tue, Dec 14, 2010 at 9:29 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: I gave this patch a look and it seems pretty good to me, except Err, woops. I just committed this as-is. Sorry. that I'm uncomfortable with the idea of mdsync filling in the details for CheckpointStats fields

Re: [HACKERS] SQL/MED - core functionality

2010-12-14 Thread Robert Haas
On Tue, Dec 14, 2010 at 1:16 AM, Pavel Stehule pavel.steh...@gmail.com wrote: On the other hand, I don't really see any advantage to allowing rules on foreign tables - ever.  Unless there's some reason we really need that, my gut feeling would be to rip it out and forget about it. views,

Re: [HACKERS] SQL/MED - core functionality

2010-12-14 Thread Itagaki Takahiro
On Tue, Dec 14, 2010 at 23:38, Robert Haas robertmh...@gmail.com wrote: On Tue, Dec 14, 2010 at 1:16 AM, Pavel Stehule pavel.steh...@gmail.com wrote: On the other hand, I don't really see any advantage to allowing rules on foreign tables - ever.  Unless there's some reason we really need

Re: [HACKERS] SQL/MED - core functionality

2010-12-14 Thread Robert Haas
On Tue, Dec 14, 2010 at 9:06 AM, Shigeru HANADA han...@metrosystems.co.jp wrote: I'll simplify the patch and post patches 1-4 of below first. essential part 1) Basic syntax for FOREIGN TABLE and FDW HANDLER 2) FDW API and ForeignScan execution # These patches are split just to make review

Re: [HACKERS] SQL/MED - core functionality

2010-12-14 Thread Robert Haas
On Tue, Dec 14, 2010 at 9:42 AM, Itagaki Takahiro itagaki.takah...@gmail.com wrote: On Tue, Dec 14, 2010 at 23:38, Robert Haas robertmh...@gmail.com wrote: On Tue, Dec 14, 2010 at 1:16 AM, Pavel Stehule pavel.steh...@gmail.com wrote: On the other hand, I don't really see any advantage to

Re: [HACKERS] Transaction-scope advisory locks

2010-12-14 Thread Tom Lane
Merlin Moncure mmonc...@gmail.com writes: Not that I'm necessarily against the proposal, but what does this do that can't already be done by locking a table or a table's row? I agree with Andres' point about this: sometimes it'd be more convenient for an advisory lock to be released

Re: [HACKERS] Transaction-scope advisory locks

2010-12-14 Thread Andrew Dunstan
On 12/14/2010 09:51 AM, Tom Lane wrote: Merlin Moncuremmonc...@gmail.com writes: Not that I'm necessarily against the proposal, but what does this do that can't already be done by locking a table or a table's row? I agree with Andres' point about this: sometimes it'd be more convenient for

Re: [HACKERS] SQL/MED - core functionality

2010-12-14 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Dec 14, 2010 at 9:42 AM, Itagaki Takahiro itagaki.takah...@gmail.com wrote: We need RULEs or INSTEAD OF TRIGGERs to support updatable foreign tables. We do? Why can't the support for updating foreign tables be built-in rather than

Re: [HACKERS] Transaction-scope advisory locks

2010-12-14 Thread Merlin Moncure
On Tue, Dec 14, 2010 at 9:51 AM, Tom Lane t...@sss.pgh.pa.us wrote: Merlin Moncure mmonc...@gmail.com writes: Not that I'm necessarily against the proposal, but what does this do that can't already be done by locking a table or a table's row? I agree with Andres' point about this: sometimes

Re: [HACKERS] Tab completion for view triggers in psql

2010-12-14 Thread David Fetter
On Mon, Dec 13, 2010 at 10:48:54PM -0500, Robert Haas wrote: On Tue, Nov 30, 2010 at 9:15 AM, David Fetter da...@fetter.org wrote: Patch attached. If you think my changes are ok, please change the patch status to Ready for Committer. Done :) I have committed part of this patch. Great!

Re: [HACKERS] SQL/MED - core functionality

2010-12-14 Thread Itagaki Takahiro
On Tue, Dec 14, 2010 at 23:45, Robert Haas robertmh...@gmail.com wrote: We need RULEs or INSTEAD OF TRIGGERs to support updatable foreign tables. We do?  Why can't the support for updating foreign tables be built-in rather than trigger-based? Do we have any concrete idea for the built-in

Re: [HACKERS] SQL/MED - core functionality

2010-12-14 Thread David Fetter
On Wed, Dec 15, 2010 at 12:48:59AM +0900, Itagaki Takahiro wrote: On Tue, Dec 14, 2010 at 23:45, Robert Haas robertmh...@gmail.com wrote: We need RULEs or INSTEAD OF TRIGGERs to support updatable foreign tables. We do?  Why can't the support for updating foreign tables be built-in

Re: [HACKERS] SQL/MED - core functionality

2010-12-14 Thread Robert Haas
On Tue, Dec 14, 2010 at 10:48 AM, Itagaki Takahiro itagaki.takah...@gmail.com wrote: On Tue, Dec 14, 2010 at 23:45, Robert Haas robertmh...@gmail.com wrote: We need RULEs or INSTEAD OF TRIGGERs to support updatable foreign tables. We do?  Why can't the support for updating foreign tables be

Re: [HACKERS] Instrument checkpoint sync calls

2010-12-14 Thread Alvaro Herrera
Excerpts from Robert Haas's message of mar dic 14 11:34:55 -0300 2010: On Tue, Dec 14, 2010 at 9:29 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: I gave this patch a look and it seems pretty good to me, except Err, woops. I just committed this as-is. Sorry. I noticed :-) that

Re: [HACKERS] pg_execute_from_file, patch v10

2010-12-14 Thread Itagaki Takahiro
On Tue, Dec 14, 2010 at 18:01, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: In any case, I concur with what I gather Robert is thinking, which is that there is no good reason to be exposing any of this at the SQL level. That used to be done this way, you know, in versions between 0 and 6 of

Re: [HACKERS] hstores in pl/python

2010-12-14 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On mån, 2010-12-13 at 10:55 -0500, Tom Lane wrote: We don't normally invent specialized syntax for a specific datatype. Not even if it's in core. I think the idea would be to make associative arrays a kind of second-order object like arrays, instead

Re: [HACKERS] hstores in pl/python

2010-12-14 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On mån, 2010-12-13 at 08:50 +0100, Jan Urbański wrote: It would be cool to be able to transparently use hstores as Python dictionaries and vice versa. It would be easy enough with hstore as a core type, but with hstore as an addon it's not that easy.

Re: [HACKERS] hstores in pl/python

2010-12-14 Thread Dmitriy Igrishin
2010/12/14 Pavel Stehule pavel.steh...@gmail.com 2010/12/13 Dmitriy Igrishin dmit...@gmail.com: 2010/12/14 Pavel Stehule pavel.steh...@gmail.com 2010/12/13 Dmitriy Igrishin dmit...@gmail.com: 2010/12/13 Pavel Stehule pavel.steh...@gmail.com 2010/12/13 Dmitriy Igrishin

Re: [HACKERS] ALTER TABLE ... ADD FOREIGN KEY ... NOT ENFORCED

2010-12-14 Thread Chris Browne
t...@sss.pgh.pa.us (Tom Lane) writes: Robert Haas robertmh...@gmail.com writes: ... On the other hand, there's clearly also a use case for this behavior. If a bulk load of prevalidated data forces an expensive revalidation of constraints that are already known to hold, there's a real chance

Re: [HACKERS] Complier warnings on mingw gcc 4.5.0

2010-12-14 Thread Andrew Dunstan
On 12/13/2010 06:45 PM, I wrote: [ problem on Mingw with 'FATAL: parameter port cannot be changed without restarting the server' if client connection options are sent ] It appears not to be related to how the environment is set at all, but to how the backend is handling PGOPTIONS.

Re: [HACKERS] Re: Proposed Windows-specific change: Enable crash dumps (like core files)

2010-12-14 Thread Tom Lane
Craig Ringer cr...@postnewspapers.com.au writes: I've attached an updated patch that fixes a failure when compiling on gcc/linux. The no-op inline installCrashDumpHandler() for unsupported platforms was not declared static, so it was not being optimized out of objects it wasn't used in and

Re: [HACKERS] hstores in pl/python

2010-12-14 Thread David E. Wheeler
On Dec 13, 2010, at 11:37 PM, Jan Urbański wrote: A function with a hstore parameter called x would get a Python dictionary as its input. A function said to be returning a hstore could return a dictionary and if it would have only string keys/values, it would be changed into a hstore (and

Re: [HACKERS] hstores in pl/python

2010-12-14 Thread Robert Haas
On Tue, Dec 14, 2010 at 11:51 AM, Tom Lane t...@sss.pgh.pa.us wrote: Peter Eisentraut pete...@gmx.net writes: On mån, 2010-12-13 at 10:55 -0500, Tom Lane wrote: We don't normally invent specialized syntax for a specific datatype. Not even if it's in core. I think the idea would be to make

Re: [HACKERS] Transaction-scope advisory locks

2010-12-14 Thread Tom Lane
Marko Tiikkaja marko.tiikk...@cs.helsinki.fi writes: On 2010-12-14 4:23 AM +0200, Tom Lane wrote: Uh, I don't think so. It sure looks like you have changed the user lockmethod to be transactional, ie, auto-release on commit/abort. I was under the impression that passing sessionLock=true to

Re: [HACKERS] BufFreelistLock

2010-12-14 Thread Jeff Janes
On Sun, Dec 12, 2010 at 6:48 PM, Jim Nasby j...@nasby.net wrote: On Dec 10, 2010, at 10:49 AM, Tom Lane wrote: Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Jeff Janes's message of vie dic 10 12:24:34 -0300 2010: As far as I can tell, bgwriter never adds things to the

Re: [HACKERS] Complier warnings on mingw gcc 4.5.0

2010-12-14 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 12/13/2010 06:45 PM, I wrote: [ problem on Mingw with 'FATAL: parameter port cannot be changed without restarting the server' if client connection options are sent ] Regina Obe has pointed out to me, and I have confirmed, that this does not

Re: [HACKERS] hstores in pl/python

2010-12-14 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Dec 14, 2010 at 11:51 AM, Tom Lane t...@sss.pgh.pa.us wrote: If the element values all have to be the same type, you've basically got hstore. Not exactly, because in hstore all the element values have to be, specifically, text. Having

Re: [HACKERS] hstores in pl/python

2010-12-14 Thread Andrew Dunstan
On 12/14/2010 12:06 PM, Robert Haas wrote: I haven't actually figured out what the benefit would be, other than buzzword compliance and a chance to invent some random nonstandard syntax. If the element values all have to be the same type, you've basically got hstore. Not exactly, because in

Re: [HACKERS] Complier warnings on mingw gcc 4.5.0

2010-12-14 Thread Andrew Dunstan
On 12/14/2010 12:10 PM, Tom Lane wrote: Andrew Dunstanand...@dunslane.net writes: On 12/13/2010 06:45 PM, I wrote: [ problem on Mingw with 'FATAL: parameter port cannot be changed without restarting the server' if client connection options are sent ] Regina Obe has pointed out to me, and I

Re: [HACKERS] hstores in pl/python

2010-12-14 Thread Robert Haas
2010/12/14 Andrew Dunstan and...@dunslane.net: On 12/14/2010 12:06 PM, Robert Haas wrote: I haven't actually figured out what the benefit would be, other than buzzword compliance and a chance to invent some random nonstandard syntax.  If the element values all have to be the same type, you've

Re: [HACKERS] hstores in pl/python

2010-12-14 Thread Pavel Stehule
2010/12/14 Tom Lane t...@sss.pgh.pa.us: Robert Haas robertmh...@gmail.com writes: On Tue, Dec 14, 2010 at 11:51 AM, Tom Lane t...@sss.pgh.pa.us wrote: If the element values all have to be the same type, you've basically got hstore. Not exactly, because in hstore all the element values have

Re: [HACKERS] hstores in pl/python

2010-12-14 Thread Robert Haas
2010/12/14 Tom Lane t...@sss.pgh.pa.us: Robert Haas robertmh...@gmail.com writes: On Tue, Dec 14, 2010 at 11:51 AM, Tom Lane t...@sss.pgh.pa.us wrote: If the element values all have to be the same type, you've basically got hstore. Not exactly, because in hstore all the element values have

Re: [HACKERS] hstores in pl/python

2010-12-14 Thread Oleg Bartunov
On Tue, 14 Dec 2010, Robert Haas wrote: On Tue, Dec 14, 2010 at 11:51 AM, Tom Lane t...@sss.pgh.pa.us wrote: Peter Eisentraut pete...@gmx.net writes: On m?n, 2010-12-13 at 10:55 -0500, Tom Lane wrote: We don't normally invent specialized syntax for a specific datatype. Not even if it's in

Re: [HACKERS] hstores in pl/python

2010-12-14 Thread Oleg Bartunov
On Tue, 14 Dec 2010, Andrew Dunstan wrote: On 12/14/2010 12:06 PM, Robert Haas wrote: I haven't actually figured out what the benefit would be, other than buzzword compliance and a chance to invent some random nonstandard syntax. If the element values all have to be the same type, you've

Re: [HACKERS] hstores in pl/python

2010-12-14 Thread Robert Haas
On Tue, Dec 14, 2010 at 12:25 PM, Oleg Bartunov o...@sai.msu.su wrote: Just wondering about JSON, are there anyone who signed already to work on JSON or it's just a theoretical discussions ? If so, I agree, having JSON properly implemented and simple wrapper for hstore just for compatibility,

Re: [HACKERS] Complier warnings on mingw gcc 4.5.0

2010-12-14 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 12/13/2010 04:34 PM, Tom Lane wrote: Oh really ... are we using src/port/unsetenv.c on that platform? I wonder if that little hack is incompatible with latest mingw libraries ... It is using pgwin32_putenv() and pgwin32_unsetenv(). It appears not

Re: [HACKERS] hstores in pl/python

2010-12-14 Thread Oleg Bartunov
On Tue, 14 Dec 2010, Robert Haas wrote: On Tue, Dec 14, 2010 at 12:25 PM, Oleg Bartunov o...@sai.msu.su wrote: Just wondering about JSON, are there anyone who signed already to work on JSON or it's just a theoretical discussions ? If so, I agree, having JSON properly implemented and simple

Re: [HACKERS] hstores in pl/python

2010-12-14 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: 2010/12/14 Andrew Dunstan and...@dunslane.net: On 12/14/2010 12:06 PM, Robert Haas wrote: Not exactly, because in hstore all the element values have to be, specifically, text.  Having hstores of other kinds of objects would, presumably, be useful. I

Re: [HACKERS] hstores in pl/python

2010-12-14 Thread Tom Lane
Oleg Bartunov o...@sai.msu.su writes: On Tue, 14 Dec 2010, Robert Haas wrote: On Tue, Dec 14, 2010 at 12:25 PM, Oleg Bartunov o...@sai.msu.su wrote: Just wondering about JSON, are there anyone who signed already to work on JSON or it's just a theoretical discussions ? Three different people

Re: [HACKERS] pg_execute_from_file, patch v10

2010-12-14 Thread Robert Haas
On Tue, Dec 14, 2010 at 11:48 AM, Itagaki Takahiro itagaki.takah...@gmail.com wrote: I'm confused which part of the patch is the point of the discussion.  1. Relax pg_read_file() to be able to read any files.  2. pg_read_binary_file()  3. pg_execute_sql_string/file() As I pointed out, 1 is

[HACKERS] ALTER TABLE ... REPLACE WITH

2010-12-14 Thread Simon Riggs
There are various applications where we want to completely replace the contents of a table with new/re-calculated data. It seems fairly obvious to be able to do this like... 1. Prepare new data into new_table and build indexes 2. Swap old for new BEGIN; DROP TABLE old_table; ALTER TABLE

Re: [HACKERS] pg_execute_from_file, patch v10

2010-12-14 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: So there are really four changes in here, right? 1. Relax pg_read_file() to be able to read any files. 2. pg_read_binary_file() 3. pg_execute_sql_string()/file() 4. ability to read a file in a given encoding (rather than the client encoding) I

Re: [HACKERS] pg_execute_from_file, patch v10

2010-12-14 Thread Robert Haas
On Tue, Dec 14, 2010 at 1:38 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: So there are really four changes in here, right? 1. Relax pg_read_file() to be able to read any files. 2. pg_read_binary_file() 3. pg_execute_sql_string()/file() 4. ability to read

Re: [HACKERS] Instrument checkpoint sync calls

2010-12-14 Thread Robert Haas
On Tue, Dec 14, 2010 at 11:47 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: that I'm uncomfortable with the idea of mdsync filling in the details for CheckpointStats fields directly.  Would it work to pass a struct (say SmgrSyncStats) from CheckPointBuffers to smgrsync and from

Re: [HACKERS] ALTER TABLE ... REPLACE WITH

2010-12-14 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: There are various applications where we want to completely replace the contents of a table with new/re-calculated data. It seems fairly obvious to be able to do this like... 1. Prepare new data into new_table and build indexes 2. Swap old for new

Re: [HACKERS] ALTER TABLE ... REPLACE WITH

2010-12-14 Thread Simon Riggs
On Tue, 2010-12-14 at 13:54 -0500, Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: There are various applications where we want to completely replace the contents of a table with new/re-calculated data. It seems fairly obvious to be able to do this like... 1. Prepare new data

Re: [HACKERS] ALTER TABLE ... REPLACE WITH

2010-12-14 Thread Robert Haas
On Tue, Dec 14, 2010 at 1:54 PM, Tom Lane t...@sss.pgh.pa.us wrote: BEGIN; TRUNCATE TABLE; ... load new data ... COMMIT; Because then you have to take an AccessExclusiveLock on the target table, of course. If we had some kind of TRUNCATE CONCURRENTLY, I think that'd address a large portion

Re: [HACKERS] hstores in pl/python

2010-12-14 Thread David E. Wheeler
On Dec 14, 2010, at 9:31 AM, Robert Haas wrote: Three different people developed patches, and I think we don't really have unanimity on which way to go with it. I've kind of been thinking we should wait for a broader consensus on which way to go with it... There needs to be a discussion for

[HACKERS] Triggered assertion !(tp.t_data-t_infomask HEAP_XMAX_INVALID) in heap_delete() on HEAD [PATCH]

2010-12-14 Thread Florian Pflug
Hi In the process of re-verifying my serializable lock consistency patch, I ran the fk_concurrency testsuite against *unpatched* HEAD for comparison. My build of HEAD had asserts enabled, and I promptly triggered Assert(!(tp.t_data-t_infomask HEAP_XMAX_INVALID)) in heap_delete(). The seems

Re: [HACKERS] ALTER TABLE ... REPLACE WITH

2010-12-14 Thread Josh Berkus
On 12/14/10 11:07 AM, Robert Haas wrote: Because then you have to take an AccessExclusiveLock on the target table, of course. Well, you have to do that for DROP TABLE as well, and I don't see any way around doing it for REPLACE WITH. As for the utility of this command: there is no question

Re: [HACKERS] ALTER TABLE ... REPLACE WITH

2010-12-14 Thread Heikki Linnakangas
On 14.12.2010 20:27, Simon Riggs wrote: There are various applications where we want to completely replace the contents of a table with new/re-calculated data. It seems fairly obvious to be able to do this like... 1. Prepare new data into new_table and build indexes 2. Swap old for new BEGIN;

Re: [HACKERS] ALTER TABLE ... REPLACE WITH

2010-12-14 Thread Robert Haas
On Tue, Dec 14, 2010 at 2:34 PM, Josh Berkus j...@agliodbs.com wrote: On 12/14/10 11:07 AM, Robert Haas wrote: Because then you have to take an AccessExclusiveLock on the target table, of course. Well, you have to do that for DROP TABLE as well, and I don't see any way around doing it for

Re: [HACKERS] pg_execute_from_file, patch v10

2010-12-14 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: Robert Haas robertmh...@gmail.com writes: So there are really four changes in here, right? 1. Relax pg_read_file() to be able to read any files. 2. pg_read_binary_file() 3. pg_execute_sql_string()/file() 4. ability to read a file in a given encoding

Re: [HACKERS] ALTER TABLE ... REPLACE WITH

2010-12-14 Thread Simon Riggs
On Tue, 2010-12-14 at 21:35 +0200, Heikki Linnakangas wrote: On 14.12.2010 20:27, Simon Riggs wrote: There are various applications where we want to completely replace the contents of a table with new/re-calculated data. It seems fairly obvious to be able to do this like... 1. Prepare

Re: [HACKERS] ALTER TABLE ... REPLACE WITH

2010-12-14 Thread Simon Riggs
On Tue, 2010-12-14 at 11:34 -0800, Josh Berkus wrote: In order for REPLACE WITH to be really useful, though, we need a command cloning at table design with *all* constraints, FKs, keys, and indexes. Currently, I still don't think we have that ... do we? Being able to vary the indexes when

Re: [HACKERS] ALTER TABLE ... REPLACE WITH

2010-12-14 Thread Simon Riggs
On Tue, 2010-12-14 at 11:34 -0800, Josh Berkus wrote: As for the utility of this command: there is no question that I would use it. I'm not sure I like the syntax (I'd prefer REPLACE TABLE WITH _), but that's painting the bike shed. REPLACE TABLE ying WITH yang is probably easier to

Re: [HACKERS] hstores in pl/python

2010-12-14 Thread Jan Urbański
On 14/12/10 18:05, David E. Wheeler wrote: On Dec 13, 2010, at 11:37 PM, Jan Urbański wrote: A function said to be returning a hstore could return a dictionary and if it would have only string keys/values, it would be changed into a hstore (and if not, throw an ERROR). It doesn't turn a

Re: [HACKERS] hstores in pl/python

2010-12-14 Thread David E. Wheeler
On Dec 14, 2010, at 11:52 AM, Jan Urbański wrote: If the function is declared to return a hstore, it transforms the dictionary to a hstore. Oh, right. Duh. Can you overload the stringification of a dictionary to return the hstore string representation? Mmm, interesting thought. I don't

Re: [HACKERS] Triggered assertion !(tp.t_data-t_infomask HEAP_XMAX_INVALID) in heap_delete() on HEAD [PATCH]

2010-12-14 Thread Tom Lane
Florian Pflug f...@phlo.org writes: In the process of re-verifying my serializable lock consistency patch, I ran the fk_concurrency testsuite against *unpatched* HEAD for comparison. My build of HEAD had asserts enabled, and I promptly triggered Assert(!(tp.t_data-t_infomask

Re: [HACKERS] Triggered assertion !(tp.t_data-t_infomask HEAP_XMAX_INVALID) in heap_delete() on HEAD [PATCH]

2010-12-14 Thread Florian Pflug
On Dec14, 2010, at 21:18 , Tom Lane wrote: Florian Pflug f...@phlo.org writes: In the process of re-verifying my serializable lock consistency patch, I ran the fk_concurrency testsuite against *unpatched* HEAD for comparison. My build of HEAD had asserts enabled, and I promptly triggered

Re: [HACKERS] Triggered assertion !(tp.t_data-t_infomask HEAP_XMAX_INVALID) in heap_delete() on HEAD [PATCH]

2010-12-14 Thread Tom Lane
Florian Pflug f...@phlo.org writes: Could you explain what seems to be wrong with my patch? I'm unconvinced that this is the proper response to whatever the problem is; and if it is the right response, it seems to still need a good bit more work. You didn't even update the functions' header

Re: [HACKERS] Transaction-scope advisory locks

2010-12-14 Thread Marko Tiikkaja
On 2010-12-14 7:05 PM +0200, Tom Lane wrote: Marko Tiikkajamarko.tiikk...@cs.helsinki.fi writes: On 2010-12-14 4:23 AM +0200, Tom Lane wrote: Uh, I don't think so. It sure looks like you have changed the user lockmethod to be transactional, ie, auto-release on commit/abort. I was under

[HACKERS] unlogged tables vs. GIST

2010-12-14 Thread Robert Haas
On Sat, Nov 13, 2010 at 9:09 PM, Robert Haas robertmh...@gmail.com wrote: The fact that it's easy doesn't make it workable.  I would point out for starters that AMs might (do) put WAL locations and/or XIDs into indexes. Occasionally copying very old LSNs or XIDs back into active files seems

Re: [HACKERS] Complier warnings on mingw gcc 4.5.0

2010-12-14 Thread Andrew Dunstan
On 12/14/2010 12:42 PM, Tom Lane wrote: This seems quite odd now that I look at it. The packet contents imply that libpq saw PGOPTIONS=-c log_min_messages=warning and no other environment variables that would cause it to append stuff to the connection request. Which is not at all how

Re: [HACKERS] unlogged tables vs. GIST

2010-12-14 Thread Heikki Linnakangas
On 14.12.2010 23:06, Robert Haas wrote: On Sat, Nov 13, 2010 at 9:09 PM, Robert Haasrobertmh...@gmail.com wrote: The fact that it's easy doesn't make it workable. I would point out for starters that AMs might (do) put WAL locations and/or XIDs into indexes. Occasionally copying very old LSNs

Re: [HACKERS] unlogged tables vs. GIST

2010-12-14 Thread Robert Haas
On Tue, Dec 14, 2010 at 4:24 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Hmm, the first idea that comes to mind is to use a counter like the GetXLogRecPtrForTemp() counter I used for temp tables, but global, in shared memory. However, that's a bit problematic because if we

Re: [HACKERS] Triggered assertion !(tp.t_data-t_infomask HEAP_XMAX_INVALID) in heap_delete() on HEAD [PATCH]

2010-12-14 Thread Florian Pflug
On Dec14, 2010, at 21:52 , Tom Lane wrote: Florian Pflug f...@phlo.org writes: Could you explain what seems to be wrong with my patch? I'm unconvinced that this is the proper response to whatever the problem is; Well, you didn't comment on the part of my previous e-mail that *did* explain

Re: [HACKERS] BufFreelistLock

2010-12-14 Thread Jim Nasby
On Dec 14, 2010, at 11:08 AM, Jeff Janes wrote: On Sun, Dec 12, 2010 at 6:48 PM, Jim Nasby j...@nasby.net wrote: BTW, when we moved from 96G to 192G servers I tried increasing shared buffers from 8G to 28G and performance went down enough to be noticeable (we don't have any good

Re: [HACKERS] unlogged tables vs. GIST

2010-12-14 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Dec 14, 2010 at 4:24 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Hmm, the first idea that comes to mind is to use a counter like the GetXLogRecPtrForTemp() counter I used for temp tables, but global, in shared memory.

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-14 Thread Robert Haas
On Tue, Dec 7, 2010 at 3:23 AM, Koichi Suzuki koichi@gmail.com wrote: This is what Postgres-XC is doing between a coordinator and a datanode.    Coordinator may correspond to poolers/loadbalancers. Does anyone think it makes sense to extract XC implementation of snapshot shipping to

Re: [HACKERS] unlogged tables vs. GIST

2010-12-14 Thread Robert Haas
On Tue, Dec 14, 2010 at 4:55 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Dec 14, 2010 at 4:24 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Hmm, the first idea that comes to mind is to use a counter like the

[HACKERS] Segfault related to pg_authid when running initdb from git master

2010-12-14 Thread Peter Geoghegan
Here's the output I see when $SUBJECT occurs, on a pg freshly built from git master with --enable-debug and --enable-cassert: [postg...@peter bin]$ uname -a Linux peter.laptop 2.6.35.9-64.fc14.x86_64 #1 SMP Fri Dec 3 12:19:41 UTC 2010 x86_64 x86_64 x86_64 GNU/Linux [postg...@peter bin]$ ./initdb

Re: [HACKERS] hstores in pl/python

2010-12-14 Thread Jan Urbański
On 14/12/10 17:52, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: On mån, 2010-12-13 at 08:50 +0100, Jan Urbański wrote: It would be cool to be able to transparently use hstores as Python dictionaries and vice versa. It would be easy enough with hstore as a core type, but with

Re: [HACKERS] pg_execute_from_file, patch v10

2010-12-14 Thread Itagaki Takahiro
On Wed, Dec 15, 2010 at 04:39, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Well, in fact, the extension's code is using either execute_sql_file() or read_text_file_with_endoding() then @extschema@ replacement then execute_sql_string(), all those functions called directly thanks to #include

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-14 Thread Koichi Suzuki
Robert; Thank you very much for your advice. Indeed, I'm considering to change the license to PostgreSQL's one. It may take a bit more though... -- Koichi Suzuki 2010/12/15 Robert Haas robertmh...@gmail.com: On Tue, Dec 7, 2010 at 3:23 AM, Koichi Suzuki koichi@gmail.com wrote:

Re: [HACKERS] ALTER TABLE ... REPLACE WITH

2010-12-14 Thread Simon Riggs
On Tue, 2010-12-14 at 16:19 -0800, Josh Berkus wrote: Without some means of doing a clone of the table in a single command, you've eliminated half the scripting work, but not helped at all with the other half. I'm not trying to eliminate scripting work, I'm trying to minimise the lock window

[HACKERS] Getting ERROR: no unpinned buffers available on HEAD, should I investigate?

2010-12-14 Thread Florian Pflug
Hi When I run my FK concurrency test suite on an unpatched HEAD, I'm getting quite a lot of no unpinned buffers available errors. Increasing shared_buffers from 32MB (the default) to 64MB makes the errors go away, as does setting fsync=off. I'm not sure how many buffers a particular backend

Re: [HACKERS] hstores in pl/python

2010-12-14 Thread James William Pye
On Dec 13, 2010, at 6:16 PM, Tom Lane wrote: how do you identify which type OID is really hstore? How about an identification field on pg_type? CREATE TYPE hstore ..., IDENTIFIER 'org.postgresql.hstore'; -- Where the identifier is an arbitrary string. Type information can be looked up by the

Re: [HACKERS] [PATCH] V3: Idle in transaction cancellation

2010-12-14 Thread Robert Haas
On Sat, Oct 30, 2010 at 4:49 AM, Andres Freund and...@anarazel.de wrote: Here is a proposed patch which enables cancellation of $subject. Disclaimer: This isn't my area of expertise, so take the below with a grain or seven of salt. It sort of looks to me like the LOG_NO_CLIENT error flag and

Re: [HACKERS] ALTER TABLE ... REPLACE WITH

2010-12-14 Thread Josh Berkus
On 12/14/10 11:43 AM, Simon Riggs wrote: On Tue, 2010-12-14 at 11:34 -0800, Josh Berkus wrote: In order for REPLACE WITH to be really useful, though, we need a command cloning at table design with *all* constraints, FKs, keys, and indexes. Currently, I still don't think we have that ... do

Re: [HACKERS] Triggered assertion !(tp.t_data-t_infomask HEAP_XMAX_INVALID) in heap_delete() on HEAD [PATCH]

2010-12-14 Thread Florian Pflug
On Dec14, 2010, at 22:34 , Florian Pflug wrote: For non-assert-enabled builds, the only effect of the patch is thus to consistently return InvalidTransactionId if the crosscheck snapshot turns HeapTupleMayBeUpdated into HeapTupleUpdated. Which certainly seems to be an improvement over

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-14 Thread Robert Haas
On Tue, Dec 14, 2010 at 7:06 PM, Koichi Suzuki koichi@gmail.com wrote: Thank you very much for your advice.   Indeed, I'm considering to change the license to PostgreSQL's one.   It may take a bit more though... You wouldn't necessarily need to relicense all of Postgres-XC (although that

Re: [HACKERS] Segfault related to pg_authid when running initdb from git master

2010-12-14 Thread Robert Haas
On Tue, Dec 14, 2010 at 5:21 PM, Peter Geoghegan peter.geoghega...@gmail.com wrote: Here's the output I see when $SUBJECT occurs, on a pg freshly built from git master with --enable-debug and --enable-cassert: I am suspicious of the fact that you are invoking initdb as ./initdb. Is it possible

  1   2   >