Re: [HACKERS] CREATE TABLE IF NOT EXISTS AS

2013-11-20 Thread Craig Ringer
On 11/20/2013 03:41 PM, Pavel Stehule wrote: It'd be great if there was a sane way to implement CREATE OR REPLACE TABLE - since that's what people really want a lot of the time. Ensure that at the end of this command the table looks like this. There's just no sane way to do

[HACKERS] Easily reading debug_print_plan

2013-11-20 Thread Craig Ringer
Hi all I'm spending a lot of time staring at parse and plan trees at the moment, and I'm finding reading them rather cumbersome. For those of you who do this a lot, do you use any sort of tooling to help you out? Just being able to collapse and expand subtrees would be a lifesaver. If it's a

Re: [HACKERS] Easily reading debug_print_plan

2013-11-20 Thread Antonin Houska
On 11/20/2013 09:12 AM, Craig Ringer wrote: Hi all I'm spending a lot of time staring at parse and plan trees at the moment, and I'm finding reading them rather cumbersome. For those of you who do this a lot, do you use any sort of tooling to help you out? vim editor. The '%' shortcut

Re: [HACKERS] Easily reading debug_print_plan

2013-11-20 Thread Craig Ringer
On 11/20/2013 04:12 PM, Craig Ringer wrote: Hi all I'm spending a lot of time staring at parse and plan trees at the moment, and I'm finding reading them rather cumbersome. For those of you who do this a lot, do you use any sort of tooling to help you out? Just being able to collapse and

Re: [HACKERS] Easily reading debug_print_plan

2013-11-20 Thread Craig Ringer
On 11/20/2013 04:22 PM, Antonin Houska wrote: vim editor. The '%' shortcut can be used to jump between opening and closing brackets and thus skip smaller or bigger parts of the output. IMO, this output is primarily for hackers (as opposed to application developers or users) and hacker should

Re: [HACKERS] CLUSTER FREEZE

2013-11-20 Thread David Rowley
On Tue, Nov 19, 2013 at 11:35 PM, David Rowley dgrowle...@gmail.com wrote: I think that the patch should include some sort of notes in the documents to say that cluster performs freezing of tuples. I've attached a patch which adds something there, but I'm not 100% sure it is the right thing.

[HACKERS] VACUUM for TOASTed objects

2013-11-20 Thread Soroosh Sardari
Hi The vacuum procedure do rewrite for a table but, what happened if the table has some TOASTed columns? Please, help me to find a module or function in source code which is responsible for vaccuming the TOAST relation. Regards, Soroosh Sardari

Re: [HACKERS] Autoconf 2.69 update

2013-11-20 Thread Oskari Saarenmaa
15.11.2013 05:00, Peter Eisentraut kirjoitti: I'm proposing that we upgrade our Autoconf to 2.69, which is the latest right now (release date 2012-04-24). There are no changes in the source needed, just tweak the version number in configure.in (see below) and run autoreconf. I've compared the

Re: [HACKERS] -d option for pg_isready is broken

2013-11-20 Thread Fujii Masao
On Wed, Nov 20, 2013 at 3:12 AM, Robert Haas robertmh...@gmail.com wrote: On Tue, Nov 19, 2013 at 1:10 PM, Fujii Masao masao.fu...@gmail.com wrote: On Tue, Nov 19, 2013 at 11:51 PM, Robert Haas robertmh...@gmail.com wrote: On Fri, Nov 15, 2013 at 9:01 PM, Fabrízio de Royes Mello

Re: [HACKERS] COPY table FROM STDIN doesn't show count tag

2013-11-20 Thread Amit Khandekar
On 19 November 2013 16:05, Rajeev rastogi rajeev.rast...@huawei.com wrote: On 19 November 2013, Amit Khandekar wrote: On 18 November 2013 18:00, Rajeev rastogi rajeev.rast...@huawei.com wrote: On 18 November 2013, Amit Khandekar wrote: Please find the patch for the same and let me know

Re: [HACKERS] VACUUM for TOASTed objects

2013-11-20 Thread Michael Paquier
On Wed, Nov 20, 2013 at 5:46 PM, Soroosh Sardari soroosh.sard...@gmail.com wrote: Hi The vacuum procedure do rewrite for a table but, what happened if the table has some TOASTed columns? Please, help me to find a module or function in source code which is responsible for vaccuming the

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-20 Thread Heikki Linnakangas
On 19.11.2013 16:22, Andres Freund wrote: On 2013-11-19 15:20:01 +0100, Andres Freund wrote: Imo something the attached patch should be done. The description I came up with is: Fix Hot-Standby initialization of clog and subtrans. Looks ok for a back-patchable fix. It's a bit bizarre

Re: [HACKERS] New option for pg_basebackup, to specify a different directory for pg_xlog

2013-11-20 Thread Haribabu kommi
On 19 November 2013 19:12 Fujii Masao wrote: On Tue, Nov 19, 2013 at 9:14 PM, Haribabu kommi haribabu.ko...@huawei.com wrote: On 18 November 2013 23:30 Fujii Masao wrote: On Tue, Nov 19, 2013 at 12:01 AM, Haribabu kommi haribabu.ko...@huawei.com wrote: Thanks for newer version of the

Re: [HACKERS] Review: pre-commit triggers

2013-11-20 Thread Ian Lawrence Barwick
2013/11/20 Tom Lane t...@sss.pgh.pa.us: Robert Haas robertmh...@gmail.com writes: On Mon, Nov 18, 2013 at 9:39 AM, Ian Lawrence Barwick barw...@gmail.com wrote: I'd expect this to lead to a failed transaction block, or at least some sort of notice that the transaction itself has been rolled

Re: [HACKERS] Get more from indices.

2013-11-20 Thread Etsuro Fujita
Kyotaro HORIGUCHI wrote: Hello, I've totally refactored the series of patches and cut out the appropriate portion as 'unique (and non-nullable) index stuff'. As the discussion before, it got rid of path distinctness. This patch works only on index 'full-orederedness', i.e., unique index on

Re: [HACKERS] Suggestion: Issue warning when calling SET TRANSACTION outside transaction block

2013-11-20 Thread Bruce Momjian
On Tue, Nov 19, 2013 at 10:21:47PM -0500, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: Does anyone know if this C comment justifies why ABORT is a NOTICE and not WARNING? /* * The user issued ABORT when not inside a transaction. Issue a

Re: [HACKERS] information schema parameter_default implementation

2013-11-20 Thread Rodolfo Campero
Peter, This patch no longer applies, because CATALOG_VERSION_NO in src/include/catalog/catversion.h has changed. I touched the patch and got it to apply without other problems (I haven't built yet). Regards, 2013/11/14 Peter Eisentraut pete...@gmx.net Updated patch attached. On Sat,

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-20 Thread Andres Freund
On 2013-11-20 12:48:50 +0200, Heikki Linnakangas wrote: On 19.11.2013 16:22, Andres Freund wrote: On 2013-11-19 15:20:01 +0100, Andres Freund wrote: Imo something the attached patch should be done. The description I came up with is: Fix Hot-Standby initialization of clog and subtrans.

Re: [HACKERS] information schema parameter_default implementation

2013-11-20 Thread Rodolfo Campero
2013/11/20 Rodolfo Campero rodolfo.camp...@anachronics.com Peter, This patch no longer applies, because CATALOG_VERSION_NO in src/include/catalog/catversion.h has changed. I touched the patch and got it to apply without other problems (I haven't built yet). Make fails: [...] make -C

Re: [HACKERS] COPY table FROM STDIN doesn't show count tag

2013-11-20 Thread Amit Khandekar
On 20 November 2013 17:40, Rajeev rastogi rajeev.rast...@huawei.com wrote: You mean to say that I should change the patch to keep only COPY FROM related changes and remove changes related to COPY TO. If yes, then I shall change the patch accordingly and also mention same in documentation

Re: [HACKERS] COPY table FROM STDIN doesn't show count tag

2013-11-20 Thread Robert Haas
On Wed, Nov 20, 2013 at 4:56 AM, Amit Khandekar amit.khande...@enterprisedb.com wrote: So I think it is best to solve this as a different issue, and we should , for this commitfest, fix only COPY FROM. Once the \COPY existing issue is solved, only then we can start printing the \COPY TO status

Re: [HACKERS] additional json functionality

2013-11-20 Thread Robert Haas
On Tue, Nov 19, 2013 at 1:43 PM, David Johnston pol...@yahoo.com wrote: IMO A reasonable default cast function should error if the json contents require anything more than a straight parse to be stored into jsonb. If the user still needs to make the conversion we should have a standard and

Re: [HACKERS] Easily reading debug_print_plan

2013-11-20 Thread Dimitri Fontaine
Craig Ringer cr...@2ndquadrant.com writes: That's what I'm currently doing, I just wanted something that makes it quicker and easier. Jumping around the tree is good, but easy collapse/expand would be much better. As I'm using psql under an Emacs M-x shell buffer, I wanted to experiment with

Re: [HACKERS] Turning recovery.conf into GUCs

2013-11-20 Thread Jaime Casanova
On Tue, Nov 19, 2013 at 3:32 PM, Andres Freund and...@2ndquadrant.com wrote: On 2013-11-19 22:09:48 +0900, Michael Paquier wrote: On Tue, Nov 19, 2013 at 2:27 AM, Andres Freund and...@2ndquadrant.com wrote: * I am not sure I like recovery.trigger as a name. It seems to close to what

Re: [HACKERS] Turning recovery.conf into GUCs

2013-11-20 Thread Andres Freund
On 2013-11-19 22:24:19 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2013-11-19 22:09:48 +0900, Michael Paquier wrote: On Tue, Nov 19, 2013 at 2:27 AM, Andres Freund and...@2ndquadrant.com wrote: * Why did you change some of the recovery gucs to lowercase names,

Re: [HACKERS] additional json functionality

2013-11-20 Thread Andrew Dunstan
On 11/20/2013 07:52 AM, Robert Haas wrote: On Tue, Nov 19, 2013 at 1:43 PM, David Johnston pol...@yahoo.com wrote: IMO A reasonable default cast function should error if the json contents require anything more than a straight parse to be stored into jsonb. If the user still needs to make the

Re: [HACKERS] Turning recovery.conf into GUCs

2013-11-20 Thread Andres Freund
On 2013-11-20 08:10:44 -0500, Jaime Casanova wrote: On Tue, Nov 19, 2013 at 3:32 PM, Andres Freund and...@2ndquadrant.com wrote: On 2013-11-19 22:09:48 +0900, Michael Paquier wrote: On Tue, Nov 19, 2013 at 2:27 AM, Andres Freund and...@2ndquadrant.com wrote: * I am not sure I like

Re: [HACKERS] Handling GIN incomplete splits

2013-11-20 Thread Michael Paquier
Here are some comments about the 4th patch. 1) Compiles without warnings, passes make check. 2) s/ginFinshSplit/ginFinishSplit 3) Server crashes when trying to create a gin index index creation (see example of previous email with pg_trgm). Here is the backtrace of the crash: * thread #1: tid =

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-20 Thread Kevin Grittner
Andres Freund and...@2ndquadrant.com wrote: On 2013-11-19 09:33:34 -0500, Andrew Dunstan wrote: On 11/19/2013 09:20 AM, Andres Freund wrote: Imo this warrants and expedited point release :( +1 I presume anyone who is vulnerable to it would need to recreate their secondary servers to get

[HACKERS] Dynamic Shared Memory stuff

2013-11-20 Thread Heikki Linnakangas
I'm trying to catch up on all of this dynamic shared memory stuff. A bunch of random questions and complaints: What kind of usage are we trying to cater with the dynamic shared memory? How many allocations? What size will they have have typically, minimum and maximum? I looked at the message

Re: [HACKERS] information schema parameter_default implementation

2013-11-20 Thread Peter Eisentraut
Updated patch From f82bc0c522b7c238b1dd8e5bb3495babd5b6192a Mon Sep 17 00:00:00 2001 From: Peter Eisentraut pete...@gmx.net Date: Thu, 14 Nov 2013 21:43:15 -0500 Subject: [PATCH v2] Implement information_schema.parameters.parameter_default column Reviewed-by: Ali Dar ali.munir@gmail.com

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-20 Thread Andres Freund
On 2013-11-20 05:30:39 -0800, Kevin Grittner wrote: Yes. There's less expensive ways to do it, but those seem to complicated to suggest. Wouldn't a database VACUUM FREEZE fix it, with WAL-logged writing of everything that doesn't yet have hint bits set? Besides also being pretty expensive

Re: [HACKERS] [PATCH] configure: allow adding a custom string to PG_VERSION

2013-11-20 Thread Michael Paquier
On Wed, Nov 20, 2013 at 8:08 AM, Oskari Saarenmaa o...@ohmu.fi wrote: On Mon, Nov 18, 2013 at 08:48:13PM -0500, Peter Eisentraut wrote: On Tue, 2013-11-05 at 18:29 +0200, Oskari Saarenmaa wrote: This can be used to tag custom built packages with an extra version string such as the git

ECPG fixes, was: Re: [HACKERS] ECPG FETCH readahead

2013-11-20 Thread Boszormenyi Zoltan
2013-11-12 07:15 keltezéssel, Boszormenyi Zoltan írta: 2013-11-12 07:01 keltezéssel, Noah Misch írta: On Mon, Nov 11, 2013 at 10:17:54AM +0100, Boszormenyi Zoltan wrote: The old contents of my GIT repository was removed so you need to clone it fresh.

[HACKERS] Modify the DECLARE CURSOR command tag depending on the scrollable flag

2013-11-20 Thread Boszormenyi Zoltan
2013-11-20 14:41 keltezéssel, Boszormenyi Zoltan írta: 2013-11-12 07:15 keltezéssel, Boszormenyi Zoltan írta: 2013-11-12 07:01 keltezéssel, Noah Misch írta: On Mon, Nov 11, 2013 at 10:17:54AM +0100, Boszormenyi Zoltan wrote: The old contents of my GIT repository was removed so you need to

[HACKERS] ECPG infrastructure changes, part 2, was: Re: ECPG fixes

2013-11-20 Thread Boszormenyi Zoltan
2013-11-20 14:41 keltezéssel, Boszormenyi Zoltan írta: 2013-11-12 07:15 keltezéssel, Boszormenyi Zoltan írta: 2013-11-12 07:01 keltezéssel, Noah Misch írta: On Mon, Nov 11, 2013 at 10:17:54AM +0100, Boszormenyi Zoltan wrote: The old contents of my GIT repository was removed so you need to

[HACKERS] ECPG infrastructure changes, part 3, was: Re: ECPG fixes

2013-11-20 Thread Boszormenyi Zoltan
2013-11-20 14:41 keltezéssel, Boszormenyi Zoltan írta: 2013-11-12 07:15 keltezéssel, Boszormenyi Zoltan írta: 2013-11-12 07:01 keltezéssel, Noah Misch írta: On Mon, Nov 11, 2013 at 10:17:54AM +0100, Boszormenyi Zoltan wrote: The old contents of my GIT repository was removed so you need to

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-20 Thread Kevin Grittner
Andres Freund and...@2ndquadrant.com wrote: On 2013-11-20 05:30:39 -0800, Kevin Grittner wrote: Wouldn't a database VACUUM FREEZE fix it, with WAL-logged writing of everything that doesn't yet have hint bits set? Besides also being pretty expensive it still wouldn't correct the clog - and

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-20 Thread Andres Freund
On 2013-11-20 05:59:58 -0800, Kevin Grittner wrote: Andres Freund and...@2ndquadrant.com wrote: On 2013-11-20 05:30:39 -0800, Kevin Grittner wrote: Wouldn't a database VACUUM FREEZE fix it, with WAL-logged writing of everything that doesn't yet have hint bits set? Besides also being

[HACKERS] ECPG infrastructure changes, part 4, was: Re: ECPG fixes

2013-11-20 Thread Boszormenyi Zoltan
2013-11-20 14:41 keltezéssel, Boszormenyi Zoltan írta: 2013-11-12 07:15 keltezéssel, Boszormenyi Zoltan írta: 2013-11-12 07:01 keltezéssel, Noah Misch írta: On Mon, Nov 11, 2013 at 10:17:54AM +0100, Boszormenyi Zoltan wrote: The old contents of my GIT repository was removed so you need to

Re: [HACKERS] nested hstore patch

2013-11-20 Thread Peter Eisentraut
On 11/12/13, 1:35 PM, Teodor Sigaev wrote: Hi! Attatched patch adds nesting feature, types (string, boll and numeric values), arrays and scalar to hstore type. Documentation doesn't build: openjade:hstore.sgml:206:16:E: document type does not allow element VARLISTENTRY here; assuming

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-20 Thread Kevin Grittner
Andres Freund and...@2ndquadrant.com wrote: On 2013-11-20 05:59:58 -0800, Kevin Grittner wrote: I don't understand where that would make sense; especially since I thought that a database FREEZE followed by a checkpoint releases old clog space anyway. It only releases them up to the (cluster

Re: [HACKERS] Easily reading debug_print_plan

2013-11-20 Thread Tom Lane
Craig Ringer cr...@2ndquadrant.com writes: I'm spending a lot of time staring at parse and plan trees at the moment, and I'm finding reading them rather cumbersome. Is there a particular reason you're doing that rather than looking at EXPLAIN output? Only the latter is meant to be at all

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-20 Thread Andres Freund
On 2013-11-20 06:21:13 -0800, Kevin Grittner wrote: Andres Freund and...@2ndquadrant.com wrote: On 2013-11-20 05:59:58 -0800, Kevin Grittner wrote: I don't understand where that would make sense; especially since I thought that a database FREEZE followed by a checkpoint releases old

Re: [HACKERS] Autoconf 2.69 update

2013-11-20 Thread Tom Lane
Oskari Saarenmaa o...@ohmu.fi writes: ISTM autoconf has been better with backwards compatibility lately. Maybe the fatal error could be changed to a warning and/or the check for version == 2.63 be replaced with a check for version = 2.63? Seems a bit risky to me. Now, Red Hat diked that

Re: [HACKERS] Autoconf 2.69 update

2013-11-20 Thread Andres Freund
On 2013-11-20 09:53:53 -0500, Tom Lane wrote: As a rule, you're not supposed to bother including the configure output script in a submitted diff anyway. Certainly any committer worth his commit bit is going to ignore it and redo autoconf for himself. The committer maybe, but it's a PITA for

Re: [HACKERS] Suggestion: Issue warning when calling SET TRANSACTION outside transaction block

2013-11-20 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: On Tue, Nov 19, 2013 at 10:21:47PM -0500, Tom Lane wrote: My personal standpoint is that I don't care much whether these messages are NOTICE or WARNING. What I'm not happy about is promoting cases that have been non-error conditions for years into

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-20 Thread Kevin Grittner
Andres Freund and...@2ndquadrant.com wrote: On 2013-11-20 06:21:13 -0800, Kevin Grittner wrote: So as long as there are no open transactions or prepared transactions on the master which started before the release with the fix is applied, VACUUM FREEZE would be guaranteed to work? Since

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Peter Eisentraut
On 11/14/13, 1:41 AM, Amit Kapila wrote: Security Concern - If a user can specify libpq connection options, he can now execute any file he wants by passing it as standalone_backend. Method to resolve Security concern

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-20 Thread Andres Freund
On 2013-11-20 07:06:04 -0800, Kevin Grittner wrote: That's not a bad point. So the way to fix it would be: 1) Restart the standby to the new minor release, wait for catchup 2) Restart the primary (fast or smart) to the new minor release 3) Acquire enough new xids to make sure we cross a

Re: [HACKERS] Suggestion: Issue warning when calling SET TRANSACTION outside transaction block

2013-11-20 Thread Bruce Momjian
On Wed, Nov 20, 2013 at 10:04:22AM -0500, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: On Tue, Nov 19, 2013 at 10:21:47PM -0500, Tom Lane wrote: My personal standpoint is that I don't care much whether these messages are NOTICE or WARNING. What I'm not happy about is promoting

Re: [HACKERS] Shave a few instructions from child-process startup sequence

2013-11-20 Thread Peter Eisentraut
On 11/5/13, 2:47 AM, Gurjeet Singh wrote: On Mon, Nov 4, 2013 at 12:20 AM, Tom Lane t...@sss.pgh.pa.us mailto:t...@sss.pgh.pa.us wrote: But we're not buying much. A few instructions during postmaster shutdown is entirely negligible. The patch is for ClosePostmasterPorts(),

Re: [HACKERS] Autoconf 2.69 update

2013-11-20 Thread Magnus Hagander
On Wed, Nov 20, 2013 at 3:58 PM, Andres Freund and...@2ndquadrant.comwrote: On 2013-11-20 09:53:53 -0500, Tom Lane wrote: As a rule, you're not supposed to bother including the configure output script in a submitted diff anyway. Certainly any committer worth his commit bit is going to

Re: [HACKERS] Autoconf 2.69 update

2013-11-20 Thread Andrew Dunstan
On 11/20/2013 10:28 AM, Magnus Hagander wrote: On Wed, Nov 20, 2013 at 3:58 PM, Andres Freund and...@2ndquadrant.com mailto:and...@2ndquadrant.com wrote: On 2013-11-20 09:53:53 -0500, Tom Lane wrote: As a rule, you're not supposed to bother including the configure output

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: I would consider sidestepping this entire issue by having the stand-alone backend create a Unix-domain socket and have a client connect to that in the normal way. Hmm. But that requires the stand-alone backend to take on at least some properties of a

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Andres Freund
On 2013-11-20 10:48:20 -0500, Tom Lane wrote: constraining what can be executed as a standalone backend. Would it work to insist that psql/pg_dump launch the program named postgres from the same bin directory they're in, rather than accepting a path from the connection string? But why do we

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2013-11-20 10:48:20 -0500, Tom Lane wrote: constraining what can be executed as a standalone backend. Would it work to insist that psql/pg_dump launch the program named postgres from the same bin directory they're in, rather than accepting a

Re: [HACKERS] Handling GIN incomplete splits

2013-11-20 Thread Heikki Linnakangas
On 19.11.2013 14:48, Michael Paquier wrote: Here is a review of the first three patches: 1) Further gin refactoring: make check passes (core tests and contrib tests). Code compiles without warnings. Committed. Then... About the patch... Even if I got little experience with code of gin,

Re: [HACKERS] Extra functionality to createuser

2013-11-20 Thread Christopher Browne
On Tue, Nov 19, 2013 at 11:54 PM, Amit Kapila amit.kapil...@gmail.com wrote: On further tests, I found inconsistency in behavior when some special characters are used in role names. 1. Test for role name containing quotes a. In psql, create a role containing quotes in role name.

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Andres Freund
On 2013-11-20 11:08:33 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2013-11-20 10:48:20 -0500, Tom Lane wrote: constraining what can be executed as a standalone backend. Would it work to insist that psql/pg_dump launch the program named postgres from the same

Re: [HACKERS] additional json functionality

2013-11-20 Thread Hannu Krosing
On 11/18/2013 06:49 PM, Josh Berkus wrote: On 11/18/2013 06:13 AM, Peter Eisentraut wrote: On 11/15/13, 6:15 PM, Josh Berkus wrote: Thing is, I'm not particularly concerned about *Merlin's* specific use case, which there are ways around. What I am concerned about is that we may have users who

Re: [HACKERS] Extra functionality to createuser

2013-11-20 Thread Christopher Browne
Wait, that doesn't work if more than one role is added, as they get merged together by the quoting. A somewhat ugly amount of quoting can be done at the shell level to induce double quotes. $ createuser -g \test_rol'e_3\ usequoted3 I note that similar (with not quite identical behaviour) issues

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-20 Thread Heikki Linnakangas
On 20.11.2013 17:06, Kevin Grittner wrote: Andres Freund and...@2ndquadrant.com wrote: On 2013-11-20 06:21:13 -0800, Kevin Grittner wrote: So as long as there are no open transactions or prepared transactions on the master which started before the release with the fix is applied,

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: I think we'd be better off trying to fix the security issue by constraining what can be executed as a standalone backend. Would it work to insist that psql/pg_dump launch the program named postgres from the same bin directory they're in, rather than

Re: [HACKERS] review: autovacuum_work_mem

2013-11-20 Thread Nigel Heron
On Mon, Nov 18, 2013 at 11:36 PM, Peter Geoghegan p...@heroku.com wrote: Please reply to the original thread in future (even if the Reply-to Message-ID is the same, I see this as a separate thread). sorry about that, when i added review to the subject gmail removed the thread info. for

Re: [HACKERS] additional json functionality

2013-11-20 Thread David Johnston
Hannu Krosing-3 wrote On 11/18/2013 06:49 PM, Josh Berkus wrote: On 11/18/2013 06:13 AM, Peter Eisentraut wrote: On 11/15/13, 6:15 PM, Josh Berkus wrote: Thing is, I'm not particularly concerned about *Merlin's* specific use case, which there are ways around. What I am concerned about is that

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Andres Freund
On 2013-11-20 17:19:42 +0100, Andres Freund wrote: That just pushes the problem up a level --- how are you going to tell psql, pg_dump, or other programs that they should do that? An explicit parameter. A program imo explicitly needs to be aware that a PQconnect() suddenly starts forking

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-20 Thread Andres Freund
On 2013-11-20 18:25:56 +0200, Heikki Linnakangas wrote: Isn't it possible that the standby has already incorrectly set HEAP_XMIN_INVALID hint bit on a page? The full page images generated by VACUUM FREEZE will correct the damage, but if not, e.g. because full_page_writes=off, strange things

Re: [HACKERS] Logging WAL when updating hintbit

2013-11-20 Thread Sawada Masahiko
On Wed, Nov 20, 2013 at 9:19 PM, Dilip kumar dilip.ku...@huawei.com wrote: On 19 November 2013 22:19, Sawada Masahiko Wrote Thanks! I took it wrong. I think that there are quite a few difference amount of WAL. Did you test about amount of WAL size in your patch? Not yet. I will do

Re: [HACKERS] [PATCH] Add transforms feature

2013-11-20 Thread Peter Eisentraut
On 11/15/13, 11:04 AM, Dimitri Fontaine wrote: - Documentation style seems to be to be different from the man page or reference docs style that we use elsewhere, and is instead deriving the general case from examples. Reads strange. Which specific section do you have in mind? It's

Re: [HACKERS] GIN improvements part 1: additional information

2013-11-20 Thread Heikki Linnakangas
On 06.11.2013 17:36, Alvaro Herrera wrote: Just for my own illumination, can someone explain this bit? + If a posting list is too large to store in-line in a key entry, a posting tree + is created. A posting tree is a B-tree structure, where the ItemPointer is + used as the key. At the

Re: [HACKERS] additional json functionality

2013-11-20 Thread Greg Stark
On Sat, Nov 16, 2013 at 12:32 AM, Josh Berkus j...@agliodbs.com wrote: On 11/15/2013 04:00 PM, David Johnston wrote: Looking at this a different way: could we just implement BSON and leave json alone? http://bsonspec.org/ In short? No. For one thing, our storage format is different

[HACKERS] WITH ORDINALITY versus column definition lists

2013-11-20 Thread Tom Lane
Consider the following case of a function that requires a column definition list (example borrowed from the regression tests): create function array_to_set(anyarray) returns setof record as $$ select i AS index, $1[i] AS value from generate_subscripts($1, 1) i $$ language sql strict immutable;

Re: [HACKERS] -d option for pg_isready is broken

2013-11-20 Thread Josh Berkus
On 11/20/2013 01:55 AM, Fujii Masao wrote: Yeah, I agree that we should make the logic of pg_isready more future-proof in 9.4dev. One idea is to expose internal_ping() as a libpq function. Then, instead of just calling PQpingParams(), we can do PQconnectStartParams(),

Re: [HACKERS] additional json functionality

2013-11-20 Thread Josh Berkus
On 11/20/2013 04:52 AM, Robert Haas wrote: I confess to being a bit perplexed by why we want hstore and json to share a common binary format. hstore doesn't store hierarchical data; json does. If we design a binary format for json, doesn't that just obsolete store? Why go to a lot of

Re: [HACKERS] Storage formats for JSON WAS: additional json functionality

2013-11-20 Thread Josh Berkus
Greg, Not being super familiar with either BSON our JSONB what advantages are we gaining from the difference? We have the JSONB/Hstore2 format *now*, and it can go into 9.4. This makes it inherently superior to any theoretical format. So any further discussion (below) is strictly academic,

Re: [HACKERS] additional json functionality

2013-11-20 Thread Andrew Dunstan
On 11/20/2013 12:50 PM, Greg Stark wrote: On Sat, Nov 16, 2013 at 12:32 AM, Josh Berkus j...@agliodbs.com mailto:j...@agliodbs.com wrote: On 11/15/2013 04:00 PM, David Johnston wrote: Looking at this a different way: could we just implement BSON and leave json alone?

Re: [HACKERS] New option for pg_basebackup, to specify a different directory for pg_xlog

2013-11-20 Thread Gavin Flower
On 20/11/13 23:43, Haribabu kommi wrote: On 19 November 2013 19:12 Fujii Masao wrote: On Tue, Nov 19, 2013 at 9:14 PM, Haribabu kommi haribabu.ko...@huawei.com wrote: On 18 November 2013 23:30 Fujii Masao wrote: On Tue, Nov 19, 2013 at 12:01 AM, Haribabu kommi haribabu.ko...@huawei.com wrote:

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-20 Thread Josh Berkus
Andrews, Kevin: Presumably a replica created while all traffic was halted on the master would be clean, correct? This bug can only be triggered if there's heavy write load on the master, right? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2013-11-20 11:08:33 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: Something like PQstartSingleUser(dsn) returning a established connection seems better to me. That just pushes the problem up a level --- how are you going

Re: [HACKERS] WITH ORDINALITY versus column definition lists

2013-11-20 Thread David Johnston
Tom Lane-2 wrote It seems to me that we don't really want this behavior of the coldeflist not including the ordinality column. It's operating as designed, maybe, but it's unexpected and confusing. We could either 1. Reinsert HEAD's prohibition against directly combining WITH ORDINALITY

Re: [HACKERS] additional json functionality

2013-11-20 Thread Tom Lane
David Johnston pol...@yahoo.com writes: On 11/18/2013 06:13 AM, Peter Eisentraut wrote: We could do something like SQL/XML and specify the level of validity in a typmod, e.g., json(loose), json(strict), etc. Three things: 1) How would this work in the face of functions that erase typemod

Re: [HACKERS] Add CREATE support to event triggers

2013-11-20 Thread Christopher Browne
On Fri, Nov 8, 2013 at 10:33 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Hello, Attached you can find a very-much-WIP patch to add CREATE info support for event triggers (normalized commands). This patch builds mainly on two things: 1. Dimitri's DDL rewrite patch he submitted way

Re: [HACKERS] nested hstore patch

2013-11-20 Thread David E. Wheeler
On Nov 20, 2013, at 6:19 AM, Peter Eisentraut pete...@gmx.net wrote: openjade:hstore.sgml:206:16:E: document type does not allow element VARLISTENTRY here; assuming missing VARIABLELIST start-tag Thanks, I fixed this one. David -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-20 Thread Josh Berkus
On 11/20/2013 10:30 AM, Josh Berkus wrote: Andrews, Kevin: Andres, that is. Presumably a replica created while all traffic was halted on the master would be clean, correct? This bug can only be triggered if there's heavy write load on the master, right? Also, just to verify: If

Re: [HACKERS] additional json functionality

2013-11-20 Thread Andrew Dunstan
On 11/20/2013 01:36 PM, Tom Lane wrote: You'd have to make the data self-identifying (which I think was the plan already), and ensure that *every* function taking json could cope with both formats on input. The typmod could only be expected to be enforced when storing or explicitly casting to

[HACKERS] [PATCH] Store Extension Options

2013-11-20 Thread Fabrízio de Royes Mello
Hi all, The main goal of this patch is enable to an user the capability to store options (relations and attributes) related to extensions by using a fixed prefix called 'ext' in the defined name. It's cant be useful for replication solutions. So, with this patch we can do that: ALTER TABLE foo

Re: [HACKERS] WITH ORDINALITY versus column definition lists

2013-11-20 Thread Tom Lane
David Johnston pol...@yahoo.com writes: Tom Lane-2 wrote It seems to me that we don't really want this behavior of the coldeflist not including the ordinality column. It's operating as designed, maybe, but it's unexpected and confusing. We could either 1. Reinsert HEAD's prohibition

Re: [HACKERS] Dynamic Shared Memory stuff

2013-11-20 Thread Robert Haas
On Wed, Nov 20, 2013 at 8:32 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: I'm trying to catch up on all of this dynamic shared memory stuff. A bunch of random questions and complaints: What kind of usage are we trying to cater with the dynamic shared memory? Parallel sort, and then

Re: [HACKERS] Replication Node Identifiers and crashsafe Apply Progress

2013-11-20 Thread Robert Haas
On Tue, Nov 19, 2013 at 1:20 PM, Andres Freund and...@2ndquadrant.com wrote: On 2013-11-19 12:47:29 -0500, Robert Haas wrote: On Tue, Nov 19, 2013 at 11:57 AM, Andres Freund and...@2ndquadrant.com wrote: Agreed. As an alternative we could just have a single - probably longer than

Re: [HACKERS] UNNEST with multiple args, and TABLE with multiple funcs

2013-11-20 Thread Tom Lane
Andrew Gierth and...@tao11.riddles.org.uk wrote: The spec syntax for table function calls, table function derived table in table reference, looks like TABLE(func(args...)) AS ... This patch implements that, plus an extension: it allows multiple functions, TABLE(func1(...), func2(...),

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Peter Eisentraut
On 11/20/13, 11:31 AM, Stephen Frost wrote: Couldn't that be an issue for people who have multiple major versions of binaries installed? In particular, the default on the system for psql might be 9.3 while the cluster you're trying to recover may be 9.2. Of course, in that case you might say

Re: [HACKERS] GIN improvements part2: fast scan

2013-11-20 Thread Alexander Korotkov
On Wed, Nov 20, 2013 at 3:06 AM, Alexander Korotkov aekorot...@gmail.comwrote: On Fri, Nov 15, 2013 at 11:19 AM, Alexander Korotkov aekorot...@gmail.com wrote: On Fri, Nov 15, 2013 at 12:34 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 14.11.2013 19:26, Alexander Korotkov

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Peter Eisentraut
On 11/20/13, 10:48 AM, Tom Lane wrote: Perhaps more to the point, I think this approach actually breaks one of the principal good-thing-in-emergencies attributes of standalone mode, namely being sure that nobody but you can connect. With this, you're right back to having a race condition as

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Robert Haas
On Wed, Nov 20, 2013 at 10:13 AM, Peter Eisentraut pete...@gmx.net wrote: On 11/14/13, 1:41 AM, Amit Kapila wrote: Security Concern - If a user can specify libpq connection options, he can now execute any file he wants by passing it as standalone_backend.

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Peter Eisentraut
On 11/20/13, 3:24 PM, Robert Haas wrote: The point is that client applications should expose whether or not to set this function as a command-line switch separate from whatever they accept in terms of connection strings. So pg_dump should have a flag called --standalone-server or something

Re: [HACKERS] Easily reading debug_print_plan

2013-11-20 Thread Robert Haas
On Wed, Nov 20, 2013 at 9:36 AM, Tom Lane t...@sss.pgh.pa.us wrote: We could in principle change to a different text representation for stored rules. Compactness would be an issue if it were materially bigger than the existing formatting, but offhand it seems like JSON is morally equivalent

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Robert Haas
On Wed, Nov 20, 2013 at 3:32 PM, Peter Eisentraut pete...@gmx.net wrote: On 11/20/13, 3:24 PM, Robert Haas wrote: The point is that client applications should expose whether or not to set this function as a command-line switch separate from whatever they accept in terms of connection strings.

Re: [HACKERS] WITH ORDINALITY versus column definition lists

2013-11-20 Thread David Johnston
Tom Lane-2 wrote David Johnston lt; polobo@ gt; writes: Tom Lane-2 wrote It seems to me that we don't really want this behavior of the coldeflist not including the ordinality column. It's operating as designed, maybe, but it's unexpected and confusing. We could either 1. Reinsert

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: The argument elsewhere in this thread was that the reason for putting this in the connection options was so that you do *not* have to patch up every client to be able to use this functionality. If you have to add separate options everywhere, then you

  1   2   >