Re: [HACKERS] [PATCH] postgres_fdw extension support

2015-07-15 Thread Michael Paquier
On Thu, Jul 16, 2015 at 12:43 PM, Tom Lane t...@sss.pgh.pa.us wrote: Michael Paquier michael.paqu...@gmail.com writes: On Thu, Jul 16, 2015 at 3:43 AM, Paul Ramsey pram...@cleverelephant.ca wrote: Attached is a patch that implements the extension support discussed at PgCon this year during

Re: [HACKERS] [DESIGN] Incremental checksums

2015-07-15 Thread Amit Kapila
On Wed, Jul 15, 2015 at 9:13 PM, David Christensen da...@endpoint.com wrote: On Jul 15, 2015, at 3:18 AM, Amit Kapila amit.kapil...@gmail.com wrote: - pg_disable_checksums(void) = turn checksums off for a cluster. Sets the state to disabled, which means bg_worker will not do anything.

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-07-15 Thread dinesh kumar
Hi On Wed, Jul 15, 2015 at 9:27 PM, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Jul 16, 2015 at 5:18 AM, Rahila Syed rahilasye...@gmail.com wrote: Hello, Please find attached updated patch with an interface to calculate command progress in pgstat.c. Thanks for updating the

Re: [HACKERS] [PATCH] postgres_fdw extension support

2015-07-15 Thread Tom Lane
Michael Paquier michael.paqu...@gmail.com writes: On Thu, Jul 16, 2015 at 3:43 AM, Paul Ramsey pram...@cleverelephant.ca wrote: Attached is a patch that implements the extension support discussed at PgCon this year during the FDW unconference sesssion. ... Thinking a bit wider, why is this

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-07-15 Thread Fujii Masao
On Thu, Jul 16, 2015 at 5:18 AM, Rahila Syed rahilasye...@gmail.com wrote: Hello, Please find attached updated patch with an interface to calculate command progress in pgstat.c. Thanks for updating the patch! I got the following compiler warning. guc.c:2316: warning: initialization makes

Re: [HACKERS] optimizing vacuum truncation scans

2015-07-15 Thread Haribabu Kommi
On Wed, Jul 15, 2015 at 11:19 PM, Amit Kapila amit.kapil...@gmail.com wrote: On Mon, Jun 29, 2015 at 11:24 AM, Jeff Janes jeff.ja...@gmail.com wrote: Attached is a patch that implements the vm scan for truncation. It introduces a variable to hold the last blkno which was skipped during the

Re: [HACKERS] Re: [COMMITTERS] pgsql: Map basebackup tablespaces using a tablespace_map file

2015-07-15 Thread Amit Kapila
On Fri, Jul 3, 2015 at 8:45 AM, Amit Kapila amit.kapil...@gmail.com wrote: On Thu, Jul 2, 2015 at 7:44 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Can we reduce that to a single one? Maybe the first one could be errdetail or something. I think it is better other way (basically

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-07-15 Thread Sameer Thakur-2
Hello, Your current design completely misses the time taken to scan indexes, which is significant. I tried to address this issue in the attached patch. The patch calculates index scan progress by measuring against scanned pages in LVRelStats. It checks for a change current page being scanned and

Re: [HACKERS] Parallel Seq Scan

2015-07-15 Thread Amit Kapila
On Wed, Jul 15, 2015 at 2:14 PM, Antonin Houska a...@cybertec.at wrote: Amit Kapila amit.kapil...@gmail.com wrote: Attached, find the rebased version of patch. [I haven't read this thread so far, sorry for possibly redundant comment.] I noticed that false is passed for required_outer

[HACKERS] option for psql - short list non template database

2015-07-15 Thread Pavel Stehule
Hi terrible often I use pattern psql -c select datname from pg_database where not datistemplate and datallowconn postgres What about introduction new long option that does it? psql -At -list --names --without-templates Regards Pavel

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-07-15 Thread Michael Paquier
On Thu, Jul 16, 2015 at 1:30 PM, Sameer Thakur-2 wrote: Thoughts? regards Sameer IndexScanProgress.patch http://postgresql.nabble.com/file/n5858109/IndexScanProgress.patch I am not really willing to show up as the picky guy here, but could it be possible to receive those patches as attached

Re: [HACKERS] [PATCH] postgres_fdw extension support

2015-07-15 Thread Amit Langote
On 2015-07-16 PM 12:43, Tom Lane wrote: The basic issue here is how can a user control which functions/operators can be sent for remote execution?. While it's certainly true that sometimes you might want function-by-function control of that, Paul's point was that extension-level granularity

Re: [HACKERS] [PATCH] postgres_fdw extension support

2015-07-15 Thread Michael Paquier
On Thu, Jul 16, 2015 at 3:43 AM, Paul Ramsey pram...@cleverelephant.ca wrote: Attached is a patch that implements the extension support discussed at PgCon this year during the FDW unconference sesssion. Highlights: * Pass extension operators and functions to the foreign server * Only send

Re: [HACKERS] assessing parallel-safety

2015-07-15 Thread Amit Kapila
On Thu, Jul 16, 2015 at 2:02 AM, Robert Haas robertmh...@gmail.com wrote: exec_stmt_execsql is called by exec_stmt_open and exec_stmt_forc. Those are cursor operations and thus - I think - parallelism can't be used there. Right, but it also gets called from exec_stmt where a parallel-safe

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-07-15 Thread Thakur, Sameer
Hello, I am not really willing to show up as the picky guy here, but could it be possible to receive those patches as attached to emails instead of having them referenced by URL? I imagine that you are directly using the nabble interface. Just configured a new mail client for nabble, did not

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-07-15 Thread Amit Langote
On 2015-07-16 AM 05:18, Rahila Syed wrote: GUC parameter 'pgstat_track_progress' is currently PGC_SUSET in line with 'track_activities' GUC parameter. Naming the GUC pgstat* seems a little inconsistent. It could be called, say,

Re: [HACKERS] security labels on databases are bad for dump restore

2015-07-15 Thread Andres Freund
On 2015-07-14 13:09:26 -0400, Adam Brightwell wrote: All, I won't have time to do anything about this anytime soon, but I think we should fix that at some point. Shall I put this on the todo? Or do we want to create an 'open items' page that's not major version specific? I think

Re: [HACKERS] [DESIGN] Incremental checksums

2015-07-15 Thread Andres Freund
On 2015-07-15 12:48:40 +0530, Amit Kapila wrote: If during scan of a relation, after doing checksum for half of the blocks in relation, system crashes, then in the above scheme a restart would need to again read all the blocks even though some of the blocks are already checksummed in previous

Re: [HACKERS] security labels on databases are bad for dump restore

2015-07-15 Thread Alvaro Herrera
Andres Freund wrote: One thing worth mentioning is that arguably the problem is caused by the fact that we're here emitting database level information in pg_dump, normally only done for dumpall. ... the reason for which is probably the lack of CURRENT_DATABASE as a database specifier. It

Re: [HACKERS] security labels on databases are bad for dump restore

2015-07-15 Thread Andres Freund
On 2015-07-15 12:04:40 +0300, Alvaro Herrera wrote: Andres Freund wrote: One thing worth mentioning is that arguably the problem is caused by the fact that we're here emitting database level information in pg_dump, normally only done for dumpall. ... the reason for which is probably the

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-07-15 Thread Michael Paquier
On Wed, Jul 15, 2015 at 3:53 PM, Simon Riggs si...@2ndquadrant.com wrote: pg_replslot has persistent state. We are discussing permanent configuration data for which I don't see the need to create an additional parallel infrastructure just to store a string given stated objection that the string

Re: [HACKERS] Could be improved point of UPSERT

2015-07-15 Thread Yourfriend
For the most cases I mentioned, we don't request a strict gapless sequence for the Invoice ID, the essential requirement is unique. We just hope that there is no obviously gap in most situations. From the test of UPSERT, there are quite a few chances to generate a big gap when UPSERT multi

Re: [HACKERS] Parallel Seq Scan

2015-07-15 Thread Antonin Houska
Amit Kapila amit.kapil...@gmail.com wrote: Attached, find the rebased version of patch. [I haven't read this thread so far, sorry for possibly redundant comment.] I noticed that false is passed for required_outer agrument of create_partialseqscan_path(), while NULL seems to be cleaner in terms

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-07-15 Thread Simon Riggs
On 7 July 2015 at 07:03, Michael Paquier michael.paqu...@gmail.com wrote: On Tue, Jul 7, 2015 at 2:19 PM, Josh Berkus j...@agliodbs.com wrote: On 07/06/2015 09:56 PM, Michael Paquier wrote: On Tue, Jul 7, 2015 at 12:51 PM, Josh Berkus j...@agliodbs.com wrote: On 07/06/2015 06:40 PM,

Re: [HACKERS] Could be improved point of UPSERT

2015-07-15 Thread Yourfriend
In my example, I just give each record a different ID to access it efficiently. In our business cases, some times, we also use some prefix letter like 'SO', PO' combining with the current year, month and then a sequence to make a invoice ID, for example, SO201507_1001, PO201503_1280, etc. As

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-07-15 Thread Simon Riggs
On 29 June 2015 at 18:40, Josh Berkus j...@agliodbs.com wrote: I'm in favor of a more robust and sophisticated synch rep. But not if nobody not on this mailing list can configure it, and not if even we don't know what it will do in an actual failure situation. That's the key point. Editing

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-07-15 Thread Amit Kapila
On Fri, Jun 26, 2015 at 11:16 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Thu, Jun 25, 2015 at 8:32 PM, Simon Riggs wrote: Let's start with a complex, fully described use case then work out how to specify what we want. Well, one of the most simple cases where quorum commit and

Re: ctidscan as an example of custom-scan (Re: [HACKERS] [v9.5] Custom Plan API)

2015-07-15 Thread Kouhei Kaigai
-Original Message- From: Michael Paquier [mailto:michael.paqu...@gmail.com] Sent: Wednesday, July 15, 2015 3:29 PM To: Kaigai Kouhei(海外 浩平) Cc: Tom Lane; Robert Haas; Alvaro Herrera; hlinnaka; Jim Nasby; Kohei KaiGai; PgHacker; Simon Riggs Subject: Re: ctidscan as an example of

Re: [HACKERS] Memory Accounting v11

2015-07-15 Thread Jeff Davis
On Tue, 2015-07-14 at 16:19 -0400, Robert Haas wrote: tuplesort.c does its own accounting, and TBH that seems like the right thing to do here, too. The difficulty is, I think, that some transition functions use an internal data type for the transition state, which might not be a single

Re: ctidscan as an example of custom-scan (Re: [HACKERS] [v9.5] Custom Plan API)

2015-07-15 Thread Michael Paquier
On Wed, Jul 15, 2015 at 10:12 AM, Kouhei Kaigai kai...@ak.jp.nec.com wrote: We never guarantee the interface compatibility between major version up. If we add/modify interface on v9.6, it is duty for developer of extensions to follow the new version, even not specific to custom-scan provider.

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-07-15 Thread Simon Riggs
On 2 July 2015 at 19:50, Josh Berkus j...@agliodbs.com wrote: So there's two parts to this: 1. I need to ensure that data is replicated to X places. 2. I need to *know* which places data was synchronously replicated to when the master goes down. My entire point is that (1) alone is

Re: [HACKERS] [DESIGN] Incremental checksums

2015-07-15 Thread Amit Kapila
On Tue, Jul 14, 2015 at 1:56 AM, David Christensen da...@endpoint.com wrote: For any relation that it finds in the database which is not checksummed, it starts an actual worker to handle the checksum process for this table. Since the state of the cluster is already either enforcing or

Re: [HACKERS] Could be improved point of UPSERT

2015-07-15 Thread Peter Geoghegan
On Wed, Jul 15, 2015 at 12:01 AM, Yourfriend doudou...@gmail.com wrote: for example, SO201507_1001, PO201503_1280, etc. As these IDs would be the most important attribute to the business, so, we hope there is no gap for the IDs. That's a requirement I've heard a number of times before. If

Re: [HACKERS] Memory Accounting v11

2015-07-15 Thread Atri Sharma
On Wed, Jul 15, 2015 at 12:57 PM, Jeff Davis pg...@j-davis.com wrote: On Tue, 2015-07-14 at 16:19 -0400, Robert Haas wrote: tuplesort.c does its own accounting, and TBH that seems like the right thing to do here, too. The difficulty is, I think, that some transition functions use an

Re: [HACKERS] [PATCH] Generalized JSON output functions

2015-07-15 Thread Andrew Dunstan
On 07/15/2015 10:52 AM, Robert Haas wrote: On Mon, Jul 13, 2015 at 10:46 AM, Ryan Pedela rped...@datalanche.com wrote: As far as large numbers in JSON, I think Postgres is doing the right thing and should not be changed. It is Javascript that is stupid here, and I don't think it is wise to add

Re: [HACKERS] [PATCH] Generalized JSON output functions

2015-07-15 Thread Ryan Pedela
On Wed, Jul 15, 2015 at 11:10 AM, Ryan Pedela rped...@datalanche.com wrote: On Wed, Jul 15, 2015 at 8:52 AM, Robert Haas robertmh...@gmail.com wrote: FWIW, I don't agree. If it's not easy to read the JSON that PostgreSQL generates using JavaScript, then a lot of people are just going to

[HACKERS] Retrieve the snapshot's LSN

2015-07-15 Thread Florent Guiliani
Hello everyone, I would need to start a read repeatable transaction and retrieve the corresponding LSN. I'm looking for pointers or Ideas on how to achieve this. Andres F. suggested me to extend pg_export_snapshot() [1] and call GetLatestSnapshot() [2] while reliably retrieving the current LSN.

Re: [HACKERS] [PATCH] Generalized JSON output functions

2015-07-15 Thread Ryan Pedela
On Wed, Jul 15, 2015 at 8:52 AM, Robert Haas robertmh...@gmail.com wrote: FWIW, I don't agree. If it's not easy to read the JSON that PostgreSQL generates using JavaScript, then a lot of people are just going to give up on doing it, and IMO that would be sad. Telling people that they have

[HACKERS] [PATCH] postgres_fdw extension support

2015-07-15 Thread Paul Ramsey
Hi all, Attached is a patch that implements the extension support discussed at PgCon this year during the FDW unconference sesssion. Highlights: * Pass extension operators and functions to the foreign server * Only send ops/funcs if the foreign server is declared to support the relevant

Re: [HACKERS] Memory Accounting v11

2015-07-15 Thread Jeff Davis
On Wed, 2015-07-15 at 12:59 +0530, Atri Sharma wrote: I think a heuristic would be more suited here and ignoring memory consumption for internal types means that we are not making the memory accounting useful for a set of usecases. OK, I will drop this patch and proceed with the HashAgg

Re: [HACKERS] Implementation of global temporary tables?

2015-07-15 Thread Andrew Dunstan
On 07/15/2015 07:58 AM, Simon Riggs wrote: For me the design summary is this * CREATE GLOBAL TEMP TABLE creates catalog entries like a normal table but with different relkind * When we see a request to INSERT, DEL, UPD, SEL from the temp table, if it does not exist we create it as a TEMP

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-07-15 Thread Alvaro Herrera
Simon Riggs wrote: JSON seems the most sensible format for the string. Inventing a new one doesn't make sense. Most important for me is the ability to programmatically manipulate/edit the config string, which would be harder with a new custom format. Do we need to keep the value consistent

Re: [HACKERS] Implementation of global temporary tables?

2015-07-15 Thread Simon Riggs
On 14 July 2015 at 23:20, Jim Nasby jim.na...@bluetreble.com wrote: On 7/9/15 12:45 AM, Pavel Stehule wrote: 2015-07-09 7:32 GMT+02:00 Zhaomo Yang zhy...@cs.ucsd.edu mailto:zhy...@cs.ucsd.edu: I am not sure, if it is not useless work. I don't understand why an implementation

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-07-15 Thread Simon Riggs
On 15 July 2015 at 10:03, Michael Paquier michael.paqu...@gmail.com wrote: OK, so this is leading us to the following points: - Use a JSON object to define the quorum/priority groups for the sync state. - Store it as a GUC, and use the check hook to validate its format, which is what we

Re: [HACKERS] Implementation of global temporary tables?

2015-07-15 Thread Pavel Stehule
2015-07-15 13:58 GMT+02:00 Simon Riggs si...@2ndquadrant.com: On 14 July 2015 at 23:20, Jim Nasby jim.na...@bluetreble.com wrote: On 7/9/15 12:45 AM, Pavel Stehule wrote: 2015-07-09 7:32 GMT+02:00 Zhaomo Yang zhy...@cs.ucsd.edu mailto:zhy...@cs.ucsd.edu: I am not sure, if it is not

Re: [HACKERS] Memory Accounting v11

2015-07-15 Thread Robert Haas
On Tue, Jul 14, 2015 at 9:14 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: Firstly, do we really have good benchmarks and measurements? I really doubt that. We do have some numbers for REINDEX, where we observed 0.5-1% regression on noisy results from a Power machine (and we've been

Re: [HACKERS] Memory Accounting v11

2015-07-15 Thread Robert Haas
On Wed, Jul 15, 2015 at 3:27 AM, Jeff Davis pg...@j-davis.com wrote: On Tue, 2015-07-14 at 16:19 -0400, Robert Haas wrote: tuplesort.c does its own accounting, and TBH that seems like the right thing to do here, too. The difficulty is, I think, that some transition functions use an internal

Re: [HACKERS] patch : Allow toast tables to be moved to a different tablespace

2015-07-15 Thread Robert Haas
On Tue, Jul 14, 2015 at 5:57 PM, Jim Nasby jim.na...@bluetreble.com wrote: On 7/7/15 7:07 AM, Andres Freund wrote: On 2015-07-03 18:03:58 -0400, Tom Lane wrote: I have just looked through this thread, and TBH I think we should reject this patch altogether --- not RWF, but no we don't want

Re: [HACKERS] [PATCH] Function to get size of asynchronous notification queue

2015-07-15 Thread Gurjeet Singh
On Thu, Jun 25, 2015 at 8:43 PM, Brendan Jurd dire...@gmail.com wrote: On Fri, 26 Jun 2015 at 06:03 Gurjeet Singh gurj...@singh.im wrote: s/proportion/fraction/ I think of these as synonymous -- do you have any particular reason to prefer fraction? I don't feel strongly about it either

Re: [HACKERS] Grouping Sets: Fix unrecognized node type bug

2015-07-15 Thread Andrew Gierth
Jeevan == Jeevan Chalke jeevan.cha...@enterprisedb.com writes: Jeevan Hi, Jeevan It looks like we do support nested GROUPING SETS, I mean Sets Jeevan withing Sets, not other types. However this nesting is broken. Good catch, but I'm not yet sure your fix is correct; I'll need to look into

[HACKERS] [PATCH] Comment fix for miscinit.c

2015-07-15 Thread David Christensen
Quick comment fix for edit issue. 0001-Fix-comment.patch Description: Binary data -- David Christensen PostgreSQL Team Manager End Point Corporation da...@endpoint.com 785-727-1171 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-07-15 Thread Rahila Syed
Hello, Please find attached updated patch with an interface to calculate command progress in pgstat.c. This interface currently implements VACUUM progress tracking . A column named percent_complete has been added in pg_stat_activity to report progress. VACUUM calls the progress calculation

Re: [HACKERS] Memory Accounting v11

2015-07-15 Thread Tomas Vondra
Hi, On 07/15/2015 09:21 PM, Robert Haas wrote: On Tue, Jul 14, 2015 at 9:14 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: Firstly, do we really have good benchmarks and measurements? I really doubt that. We do have some numbers for REINDEX, where we observed 0.5-1% regression on noisy

Re: [HACKERS] Memory Accounting v11

2015-07-15 Thread Tomas Vondra
Hi, On 07/15/2015 07:07 PM, Jeff Davis wrote: On Wed, 2015-07-15 at 12:59 +0530, Atri Sharma wrote: I think a heuristic would be more suited here and ignoring memory consumption for internal types means that we are not making the memory accounting useful for a set of usecases. OK, I will

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-07-15 Thread Simon Riggs
On 15 July 2015 at 12:25, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Simon Riggs wrote: JSON seems the most sensible format for the string. Inventing a new one doesn't make sense. Most important for me is the ability to programmatically manipulate/edit the config string, which would

Re: [HACKERS] Making ParameterStatus available for more parameter types?

2015-07-15 Thread Robert Haas
On Sun, Jul 12, 2015 at 5:30 AM, Shay Rojansky r...@roji.org wrote: The ParameterStatus message is currently sent for a hard-wired set of parameters (http://www.postgresql.org/docs/current/static/protocol-flow.html#PROTOCOL-ASYNC). Just wanted to let you know that making this more flexible

Re: [HACKERS] Implementation of global temporary tables?

2015-07-15 Thread Simon Riggs
On 15 July 2015 at 13:26, Andrew Dunstan and...@dunslane.net wrote: On 07/15/2015 07:58 AM, Simon Riggs wrote: For me the design summary is this * CREATE GLOBAL TEMP TABLE creates catalog entries like a normal table but with different relkind * When we see a request to INSERT, DEL, UPD,

Re: [HACKERS] Implementation of global temporary tables?

2015-07-15 Thread Pavel Stehule
2015-07-15 15:53 GMT+02:00 Zhaomo Yang zhy...@cs.ucsd.edu: there is other question - what is effect of ALTER TABLE of global temp table on instances of this table in active sessions? As I said, we need to first agree on the behaviors of the existing commands. I can think of two options

Re: [HACKERS] Implementation of global temporary tables?

2015-07-15 Thread Simon Riggs
On 15 July 2015 at 16:44, Andres Freund and...@anarazel.de wrote: On 2015-07-15 16:36:12 +0100, Simon Riggs wrote: On 15 July 2015 at 16:28, Andres Freund and...@anarazel.de wrote: I think that's generally a fair point. But here we're discussing to add a fair amount of wrinkles with the

Re: [HACKERS] Implementation of global temporary tables?

2015-07-15 Thread Pavel Stehule
2015-07-15 15:21 GMT+02:00 Zhaomo Yang zhy...@cs.ucsd.edu: Sounds fine in general. I'm a bit curious to know what are the locking implications of vivifying the table on access. The locking implications depend on how we interpret the existing commands in the context of global temp tables

[HACKERS] Grouping Sets: Fix unrecognized node type bug

2015-07-15 Thread Jeevan Chalke
Hi, It looks like we do support nested GROUPING SETS, I mean Sets withing Sets, not other types. However this nesting is broken. Here is the simple example where I would expect three rows in the result. But unfortunately it is giving unrecognized node type error. Which is something weird and

Re: [HACKERS] optimizing vacuum truncation scans

2015-07-15 Thread Amit Kapila
On Mon, Jun 29, 2015 at 11:24 AM, Jeff Janes jeff.ja...@gmail.com wrote: Attached is a patch that implements the vm scan for truncation. It introduces a variable to hold the last blkno which was skipped during the forward portion. Any blocks after both this blkno and after the last inspected

Re: [HACKERS] Implementation of global temporary tables?

2015-07-15 Thread Zhaomo Yang
there is other question - what is effect of ALTER TABLE of global temp table on instances of this table in active sessions? As I said, we need to first agree on the behaviors of the existing commands. I can think of two options now for ALTER TABLE: 1) only allow ALTER TABLE when there is no

Re: [HACKERS] Implementation of global temporary tables?

2015-07-15 Thread Simon Riggs
On 15 July 2015 at 15:57, Andres Freund and...@anarazel.de wrote: On 2015-07-15 16:52:49 +0200, Andres Freund wrote: Why do we need to create that copy? We can just use the relfilenode in all backends by having the backendid in the filename? Yes, there's a some amount of additional code

Re: [HACKERS] Implementation of global temporary tables?

2015-07-15 Thread Zhaomo Yang
Sounds fine in general. I'm a bit curious to know what are the locking implications of vivifying the table on access. The locking implications depend on how we interpret the existing commands in the context of global temp tables and I think we should discuss and agree on the behaviors of the

Re: [HACKERS] Implementation of global temporary tables?

2015-07-15 Thread Andres Freund
On 2015-07-15 16:24:52 +0100, Simon Riggs wrote: It may be possible to do this, though I'm sure there's a wrinkle somewhere. But there doesn't seem to be a need to overload the main feature request with additional requirements. Doing that is just scope creep that prevents us getting features

Re: [HACKERS] Implementation of global temporary tables?

2015-07-15 Thread Simon Riggs
On 15 July 2015 at 16:28, Andres Freund and...@anarazel.de wrote: On 2015-07-15 16:24:52 +0100, Simon Riggs wrote: It may be possible to do this, though I'm sure there's a wrinkle somewhere. But there doesn't seem to be a need to overload the main feature request with additional

Re: [HACKERS] Implementation of global temporary tables?

2015-07-15 Thread Tom Lane
Andres Freund and...@anarazel.de writes: On 2015-07-15 16:24:52 +0100, Simon Riggs wrote: It may be possible to do this, though I'm sure there's a wrinkle somewhere. But there doesn't seem to be a need to overload the main feature request with additional requirements. Doing that is just scope

Re: [HACKERS] Implementation of global temporary tables?

2015-07-15 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On 15 July 2015 at 16:28, Andres Freund and...@anarazel.de wrote: I think that's generally a fair point. But here we're discussing to add a fair amount of wrinkles with the copy approach. The fact alone that the oid is different will have some ugly

Re: [HACKERS] [DESIGN] Incremental checksums

2015-07-15 Thread David Christensen
On Jul 15, 2015, at 3:18 AM, Amit Kapila amit.kapil...@gmail.com wrote: - pg_disable_checksums(void) = turn checksums off for a cluster. Sets the state to disabled, which means bg_worker will not do anything. - pg_request_checksum_cycle(void) = if checksums are enabled,

Re: [HACKERS] LWLock deadlock and gdb advice

2015-07-15 Thread Heikki Linnakangas
On 06/30/2015 11:24 PM, Andres Freund wrote: On 2015-06-30 22:19:02 +0300, Heikki Linnakangas wrote: Hm. Right. A recheck of the value after the queuing should be sufficient to fix? That's how we deal with the exact same scenarios for the normal lock state, so that shouldn't be very hard to

Re: [HACKERS] Implementation of global temporary tables?

2015-07-15 Thread Andres Freund
On 2015-07-15 16:36:12 +0100, Simon Riggs wrote: On 15 July 2015 at 16:28, Andres Freund and...@anarazel.de wrote: I think that's generally a fair point. But here we're discussing to add a fair amount of wrinkles with the copy approach. The fact alone that the oid is different will have

Re: [HACKERS] [PATCH] Generalized JSON output functions

2015-07-15 Thread Robert Haas
On Mon, Jul 13, 2015 at 10:46 AM, Ryan Pedela rped...@datalanche.com wrote: As far as large numbers in JSON, I think Postgres is doing the right thing and should not be changed. It is Javascript that is stupid here, and I don't think it is wise to add something to core just because one client

Re: [HACKERS] Implementation of global temporary tables?

2015-07-15 Thread Andres Freund
On 2015-07-15 12:58:51 +0100, Simon Riggs wrote: On 14 July 2015 at 23:20, Jim Nasby jim.na...@bluetreble.com wrote: Pavel's original description of how to do this seem valid, and from the link Tom agreed in 2009. For me the design summary is this * CREATE GLOBAL TEMP TABLE creates

Re: [HACKERS] Implementation of global temporary tables?

2015-07-15 Thread Andres Freund
On 2015-07-15 16:52:49 +0200, Andres Freund wrote: Why do we need to create that copy? We can just use the relfilenode in all backends by having the backendid in the filename? Yes, there's a some amount of additional code needed, but it's not that much? I actually think it might end up being

Re: [HACKERS] First Aggregate Funtion?

2015-07-15 Thread Robert Haas
On Tue, Jul 14, 2015 at 9:23 AM, sudalai sudala...@gmail.com wrote: The above implementation of first aggregate returns the first non-NULL item value. To get *first row item value* for a column use the below implementation. -- create a function that push at most two element on given array

Re: ctidscan as an example of custom-scan (Re: [HACKERS] [v9.5] Custom Plan API)

2015-07-15 Thread Robert Haas
On Tue, Jul 14, 2015 at 6:04 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Both you and Andres have articulated the concern that CustomScan isn't actually useful, but I still don't really understand why not. I'm curious, for example, whether CustomScan would

Re: ctidscan as an example of custom-scan (Re: [HACKERS] [v9.5] Custom Plan API)

2015-07-15 Thread Robert Haas
On Wed, Jul 15, 2015 at 2:28 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Wed, Jul 15, 2015 at 10:12 AM, Kouhei Kaigai kai...@ak.jp.nec.com wrote: We never guarantee the interface compatibility between major version up. If we add/modify interface on v9.6, it is duty for developer of