Re: [HACKERS] CREATE TABLE IF NOT EXISTS AS

2013-11-20 Thread Craig Ringer
On 11/20/2013 03:41 PM, Pavel Stehule wrote: > > It'd be great if there was a sane way to implement "CREATE OR REPLACE > TABLE" - since that's what people really want a lot of the time. "Ensure > that at the end of this command the table looks like this". There's just > no sane way

[HACKERS] Easily reading debug_print_plan

2013-11-20 Thread Craig Ringer
Hi all I'm spending a lot of time staring at parse and plan trees at the moment, and I'm finding reading them rather cumbersome. For those of you who do this a lot, do you use any sort of tooling to help you out? Just being able to collapse and expand subtrees would be a lifesaver. If it's a has

Re: [HACKERS] Easily reading debug_print_plan

2013-11-20 Thread Antonin Houska
On 11/20/2013 09:12 AM, Craig Ringer wrote: > Hi all > > I'm spending a lot of time staring at parse and plan trees at the > moment, and I'm finding reading them rather cumbersome. > > For those of you who do this a lot, do you use any sort of tooling to > help you out? vim editor. The '%' short

Re: [HACKERS] Easily reading debug_print_plan

2013-11-20 Thread Craig Ringer
On 11/20/2013 04:12 PM, Craig Ringer wrote: > Hi all > > I'm spending a lot of time staring at parse and plan trees at the > moment, and I'm finding reading them rather cumbersome. > > For those of you who do this a lot, do you use any sort of tooling to > help you out? Just being able to collaps

Re: [HACKERS] Easily reading debug_print_plan

2013-11-20 Thread Craig Ringer
On 11/20/2013 04:22 PM, Antonin Houska wrote: > vim editor. The '%' shortcut can be used to jump between opening and > closing brackets and thus skip smaller or bigger parts of the output. > IMO, this output is primarily for hackers (as opposed to application > developers or users) and hacker shoul

Re: [HACKERS] CLUSTER FREEZE

2013-11-20 Thread David Rowley
On Tue, Nov 19, 2013 at 11:35 PM, David Rowley wrote: > I think that the patch should include some sort of notes in the documents > to say that cluster performs freezing of tuples. I've attached a patch > which adds something there, but I'm not 100% sure it is the right thing. > Perhaps it should

[HACKERS] VACUUM for TOASTed objects

2013-11-20 Thread Soroosh Sardari
Hi The vacuum procedure do rewrite for a table but, what happened if the table has some TOASTed columns? Please, help me to find a module or function in source code which is responsible for vaccuming the TOAST relation. Regards, Soroosh Sardari

Re: [HACKERS] Autoconf 2.69 update

2013-11-20 Thread Oskari Saarenmaa
15.11.2013 05:00, Peter Eisentraut kirjoitti: I'm proposing that we upgrade our Autoconf to 2.69, which is the latest right now (release date 2012-04-24). There are no changes in the source needed, just tweak the version number in configure.in (see below) and run autoreconf. I've compared the c

Re: [HACKERS] -d option for pg_isready is broken

2013-11-20 Thread Fujii Masao
On Wed, Nov 20, 2013 at 3:12 AM, Robert Haas wrote: > On Tue, Nov 19, 2013 at 1:10 PM, Fujii Masao wrote: >> On Tue, Nov 19, 2013 at 11:51 PM, Robert Haas wrote: >>> On Fri, Nov 15, 2013 at 9:01 PM, Fabrízio de Royes Mello >>> wrote: On Wed, Nov 13, 2013 at 9:37 PM, Josh Berkus wrote: >>>

Re: [HACKERS] COPY table FROM STDIN doesn't show count tag

2013-11-20 Thread Amit Khandekar
On 19 November 2013 16:05, Rajeev rastogi wrote: > On 19 November 2013, Amit Khandekar wrote: > > >On 18 November 2013 18:00, Rajeev rastogi > wrote: > > >>On 18 November 2013, Amit Khandekar wrote: > > > >>Please find the patch for the same and let me know your suggestions. > > >>>In this call

Re: [HACKERS] VACUUM for TOASTed objects

2013-11-20 Thread Michael Paquier
On Wed, Nov 20, 2013 at 5:46 PM, Soroosh Sardari wrote: > Hi > > The vacuum procedure do rewrite for a table but, what happened if the table > has some TOASTed columns? > > Please, help me to find a module or function in source code which is > responsible for > vaccuming the TOAST relation. A toas

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-20 Thread Heikki Linnakangas
On 19.11.2013 16:22, Andres Freund wrote: On 2013-11-19 15:20:01 +0100, Andres Freund wrote: Imo something the attached patch should be done. The description I came up with is: Fix Hot-Standby initialization of clog and subtrans. Looks ok for a back-patchable fix. It's a bit bizarre tha

Re: [HACKERS] New option for pg_basebackup, to specify a different directory for pg_xlog

2013-11-20 Thread Haribabu kommi
On 19 November 2013 19:12 Fujii Masao wrote: > On Tue, Nov 19, 2013 at 9:14 PM, Haribabu kommi > wrote: > > On 18 November 2013 23:30 Fujii Masao wrote: > >> On Tue, Nov 19, 2013 at 12:01 AM, Haribabu kommi > >> wrote: > >> > >> Thanks for newer version of the patch! > >> > >> I found that the em

Re: [HACKERS] Review: pre-commit triggers

2013-11-20 Thread Ian Lawrence Barwick
2013/11/20 Tom Lane : > Robert Haas writes: >> On Mon, Nov 18, 2013 at 9:39 AM, Ian Lawrence Barwick >> wrote: >>> I'd expect this to lead to a failed transaction block, >>> or at least some sort of notice that the transaction itself >>> has been rolled back. > >> Ending up in a failed transacti

Re: [HACKERS] Get more from indices.

2013-11-20 Thread Etsuro Fujita
Kyotaro HORIGUCHI wrote: > Hello, I've totally refactored the series of patches and cut out the appropriate portion as 'unique (and non-nullable) index stuff'. > As the discussion before, it got rid of path distinctness. This patch works only on index 'full-orederedness', i.e., unique index on non

Re: [HACKERS] Suggestion: Issue warning when calling SET TRANSACTION outside transaction block

2013-11-20 Thread Bruce Momjian
On Tue, Nov 19, 2013 at 10:21:47PM -0500, Tom Lane wrote: > Bruce Momjian writes: > > Does anyone know if this C comment justifies why ABORT is a NOTICE and > > not WARNING? > > > /* > > * The user issued ABORT when not inside a transaction. Issue a > > * NOT

Re: [HACKERS] information schema parameter_default implementation

2013-11-20 Thread Rodolfo Campero
Peter, This patch no longer applies, because CATALOG_VERSION_NO in src/include/catalog/catversion.h has changed. I touched the patch and got it to apply without other problems (I haven't built yet). Regards, 2013/11/14 Peter Eisentraut > Updated patch attached. > > On Sat, 2013-11-09 at 12:09

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-20 Thread Andres Freund
On 2013-11-20 12:48:50 +0200, Heikki Linnakangas wrote: > On 19.11.2013 16:22, Andres Freund wrote: > >On 2013-11-19 15:20:01 +0100, Andres Freund wrote: > >>Imo something the attached patch should be done. The description I came > >>up with is: > >> > >> Fix Hot-Standby initialization of clog

Re: [HACKERS] information schema parameter_default implementation

2013-11-20 Thread Rodolfo Campero
2013/11/20 Rodolfo Campero > Peter, > > This patch no longer applies, because CATALOG_VERSION_NO > in src/include/catalog/catversion.h has changed. I touched the patch and > got it to apply without other problems (I haven't built yet). > > Make fails: [...] make -C catalog schemapg.h make[3]: se

Re: [HACKERS] COPY table FROM STDIN doesn't show count tag

2013-11-20 Thread Amit Khandekar
On 20 November 2013 17:40, Rajeev rastogi wrote: > You mean to say that I should change the patch to keep only COPY FROM > related changes and remove changes related to COPY TO. > > If yes, then I shall change the patch accordingly and also mention same > in documentation also. > > Please let me

Re: [HACKERS] COPY table FROM STDIN doesn't show count tag

2013-11-20 Thread Robert Haas
On Wed, Nov 20, 2013 at 4:56 AM, Amit Khandekar wrote: > So I think it is best to solve this as a different issue, and we should , > for this commitfest, fix only COPY FROM. Once the \COPY existing issue is > solved, only then we can start printing the \COPY TO status as well. I actually think t

Re: [HACKERS] additional json functionality

2013-11-20 Thread Robert Haas
On Tue, Nov 19, 2013 at 1:43 PM, David Johnston wrote: > IMO A reasonable default cast function should error if the json contents > require anything more than a straight parse to be stored into jsonb. If the > user still needs to make the conversion we should have a standard and > configurable pa

Re: [HACKERS] Easily reading debug_print_plan

2013-11-20 Thread Dimitri Fontaine
Craig Ringer writes: > That's what I'm currently doing, I just wanted something that makes it > quicker and easier. Jumping around the tree is good, but easy > collapse/expand would be much better. As I'm using psql under an Emacs M-x shell buffer, I wanted to experiment with your problem here, s

Re: [HACKERS] Turning recovery.conf into GUCs

2013-11-20 Thread Jaime Casanova
On Tue, Nov 19, 2013 at 3:32 PM, Andres Freund wrote: > On 2013-11-19 22:09:48 +0900, Michael Paquier wrote: >> On Tue, Nov 19, 2013 at 2:27 AM, Andres Freund >> wrote: > >> > * I am not sure I like "recovery.trigger" as a name. It seems to close >> > to what I've seen people use to trigger fa

Re: [HACKERS] Turning recovery.conf into GUCs

2013-11-20 Thread Andres Freund
On 2013-11-19 22:24:19 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2013-11-19 22:09:48 +0900, Michael Paquier wrote: > >> On Tue, Nov 19, 2013 at 2:27 AM, Andres Freund > >> wrote: > >>> * Why did you change some of the recovery gucs to lowercase names, but > >>> left out XLogRestoreCo

Re: [HACKERS] additional json functionality

2013-11-20 Thread Andrew Dunstan
On 11/20/2013 07:52 AM, Robert Haas wrote: On Tue, Nov 19, 2013 at 1:43 PM, David Johnston wrote: IMO A reasonable default cast function should error if the json contents require anything more than a straight parse to be stored into jsonb. If the user still needs to make the conversion we sho

Re: [HACKERS] Turning recovery.conf into GUCs

2013-11-20 Thread Andres Freund
On 2013-11-20 08:10:44 -0500, Jaime Casanova wrote: > On Tue, Nov 19, 2013 at 3:32 PM, Andres Freund wrote: > > On 2013-11-19 22:09:48 +0900, Michael Paquier wrote: > >> On Tue, Nov 19, 2013 at 2:27 AM, Andres Freund > >> wrote: > > > >> > * I am not sure I like "recovery.trigger" as a name. It

Re: [HACKERS] Handling GIN incomplete splits

2013-11-20 Thread Michael Paquier
Here are some comments about the 4th patch. 1) Compiles without warnings, passes make check. 2) s/ginFinshSplit/ginFinishSplit 3) Server crashes when trying to create a gin index index creation (see example of previous email with pg_trgm). Here is the backtrace of the crash: * thread #1: tid = 0x15

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-20 Thread Kevin Grittner
Andres Freund wrote: > On 2013-11-19 09:33:34 -0500, Andrew Dunstan wrote: >> >> On 11/19/2013 09:20 AM, Andres Freund wrote: >> >Imo this warrants and expedited point release :( +1 >> I presume anyone who is vulnerable to it would need to recreate >> their secondary servers to get rid of potent

[HACKERS] Dynamic Shared Memory stuff

2013-11-20 Thread Heikki Linnakangas
I'm trying to catch up on all of this dynamic shared memory stuff. A bunch of random questions and complaints: What kind of usage are we trying to cater with the dynamic shared memory? How many allocations? What size will they have have typically, minimum and maximum? I looked at the message q

Re: [HACKERS] information schema parameter_default implementation

2013-11-20 Thread Peter Eisentraut
Updated patch >From f82bc0c522b7c238b1dd8e5bb3495babd5b6192a Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Thu, 14 Nov 2013 21:43:15 -0500 Subject: [PATCH v2] Implement information_schema.parameters.parameter_default column Reviewed-by: Ali Dar Reviewed-by: Amit Khandekar --- doc/src/

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-20 Thread Andres Freund
On 2013-11-20 05:30:39 -0800, Kevin Grittner wrote: > > Yes. There's less expensive ways to do it, but those seem to > > complicated to suggest. > > Wouldn't a database VACUUM FREEZE fix it, with WAL-logged writing > of everything that doesn't yet have hint bits set? Besides also being pretty exp

Re: [HACKERS] [PATCH] configure: allow adding a custom string to PG_VERSION

2013-11-20 Thread Michael Paquier
On Wed, Nov 20, 2013 at 8:08 AM, Oskari Saarenmaa wrote: > > On Mon, Nov 18, 2013 at 08:48:13PM -0500, Peter Eisentraut wrote: > > On Tue, 2013-11-05 at 18:29 +0200, Oskari Saarenmaa wrote: > > > This can be used to tag custom built packages with an extra version string > > > such as the git descr

ECPG fixes, was: Re: [HACKERS] ECPG FETCH readahead

2013-11-20 Thread Boszormenyi Zoltan
2013-11-12 07:15 keltezéssel, Boszormenyi Zoltan írta: 2013-11-12 07:01 keltezéssel, Noah Misch írta: On Mon, Nov 11, 2013 at 10:17:54AM +0100, Boszormenyi Zoltan wrote: The old contents of my GIT repository was removed so you need to clone it fresh. https://github.com/zboszor/ecpg-readahead.gi

[HACKERS] Modify the DECLARE CURSOR command tag depending on the scrollable flag

2013-11-20 Thread Boszormenyi Zoltan
2013-11-20 14:41 keltezéssel, Boszormenyi Zoltan írta: 2013-11-12 07:15 keltezéssel, Boszormenyi Zoltan írta: 2013-11-12 07:01 keltezéssel, Noah Misch írta: On Mon, Nov 11, 2013 at 10:17:54AM +0100, Boszormenyi Zoltan wrote: The old contents of my GIT repository was removed so you need to clon

[HACKERS] ECPG infrastructure changes, part 2, was: Re: ECPG fixes

2013-11-20 Thread Boszormenyi Zoltan
2013-11-20 14:41 keltezéssel, Boszormenyi Zoltan írta: 2013-11-12 07:15 keltezéssel, Boszormenyi Zoltan írta: 2013-11-12 07:01 keltezéssel, Noah Misch írta: On Mon, Nov 11, 2013 at 10:17:54AM +0100, Boszormenyi Zoltan wrote: The old contents of my GIT repository was removed so you need to clon

[HACKERS] ECPG infrastructure changes, part 3, was: Re: ECPG fixes

2013-11-20 Thread Boszormenyi Zoltan
2013-11-20 14:41 keltezéssel, Boszormenyi Zoltan írta: 2013-11-12 07:15 keltezéssel, Boszormenyi Zoltan írta: 2013-11-12 07:01 keltezéssel, Noah Misch írta: On Mon, Nov 11, 2013 at 10:17:54AM +0100, Boszormenyi Zoltan wrote: The old contents of my GIT repository was removed so you need to clon

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-20 Thread Kevin Grittner
Andres Freund wrote: > On 2013-11-20 05:30:39 -0800, Kevin Grittner wrote: >> Wouldn't a database VACUUM FREEZE fix it, with WAL-logged >> writing of everything that doesn't yet have hint bits set? > > Besides also being pretty expensive it still wouldn't correct the > clog - and we don't always

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-20 Thread Andres Freund
On 2013-11-20 05:59:58 -0800, Kevin Grittner wrote: > Andres Freund wrote: > > On 2013-11-20 05:30:39 -0800, Kevin Grittner wrote: > > >> Wouldn't a database VACUUM FREEZE fix it, with WAL-logged > >> writing of everything that doesn't yet have hint bits set? > > > > Besides also being pretty exp

[HACKERS] ECPG infrastructure changes, part 4, was: Re: ECPG fixes

2013-11-20 Thread Boszormenyi Zoltan
2013-11-20 14:41 keltezéssel, Boszormenyi Zoltan írta: 2013-11-12 07:15 keltezéssel, Boszormenyi Zoltan írta: 2013-11-12 07:01 keltezéssel, Noah Misch írta: On Mon, Nov 11, 2013 at 10:17:54AM +0100, Boszormenyi Zoltan wrote: The old contents of my GIT repository was removed so you need to clon

Re: [HACKERS] nested hstore patch

2013-11-20 Thread Peter Eisentraut
On 11/12/13, 1:35 PM, Teodor Sigaev wrote: > Hi! > > Attatched patch adds nesting feature, types (string, boll and numeric > values), arrays and scalar to hstore type. Documentation doesn't build: openjade:hstore.sgml:206:16:E: document type does not allow element "VARLISTENTRY" here; assuming

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-20 Thread Kevin Grittner
Andres Freund wrote: > On 2013-11-20 05:59:58 -0800, Kevin Grittner wrote: >> I don't understand where that would make sense; especially since >> I thought that a database FREEZE followed by a checkpoint >> releases old clog space anyway. > > It only releases them up to the (cluster wide) xmin ho

Re: [HACKERS] Easily reading debug_print_plan

2013-11-20 Thread Tom Lane
Craig Ringer writes: > I'm spending a lot of time staring at parse and plan trees at the > moment, and I'm finding reading them rather cumbersome. Is there a particular reason you're doing that rather than looking at EXPLAIN output? Only the latter is meant to be at all user-friendly. > For tho

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-20 Thread Andres Freund
On 2013-11-20 06:21:13 -0800, Kevin Grittner wrote: > Andres Freund wrote: > > On 2013-11-20 05:59:58 -0800, Kevin Grittner wrote: > > >> I don't understand where that would make sense; especially since > >> I thought that a database FREEZE followed by a checkpoint > >> releases old clog space an

Re: [HACKERS] Autoconf 2.69 update

2013-11-20 Thread Tom Lane
Oskari Saarenmaa writes: > ISTM autoconf has been better with backwards compatibility lately. > Maybe the fatal error could be changed to a warning and/or the check for > version == 2.63 be replaced with a check for version >= 2.63? Seems a bit risky to me. Now, Red Hat diked that test out for

Re: [HACKERS] Autoconf 2.69 update

2013-11-20 Thread Andres Freund
On 2013-11-20 09:53:53 -0500, Tom Lane wrote: > As a rule, you're not supposed to bother including the configure output > script in a submitted diff anyway. Certainly any committer worth his > commit bit is going to ignore it and redo autoconf for himself. The committer maybe, but it's a PITA fo

Re: [HACKERS] Suggestion: Issue warning when calling SET TRANSACTION outside transaction block

2013-11-20 Thread Tom Lane
Bruce Momjian writes: > On Tue, Nov 19, 2013 at 10:21:47PM -0500, Tom Lane wrote: >> My personal standpoint is that I don't care much whether these messages >> are NOTICE or WARNING. What I'm not happy about is promoting cases that >> have been non-error conditions for years into ERRORs. > I don

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-20 Thread Kevin Grittner
Andres Freund wrote: > On 2013-11-20 06:21:13 -0800, Kevin Grittner wrote: >> So as long as there are no open transactions or prepared >> transactions on the master which started before the release with >> the fix is applied, VACUUM FREEZE would be guaranteed to work? >> Since I don't see h

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Peter Eisentraut
On 11/14/13, 1:41 AM, Amit Kapila wrote: >Security Concern >- >If a user can specify libpq connection options, he can now execute > any file he wants by passing it as standalone_backend. > >Method to resolve Security concern >---

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-20 Thread Andres Freund
On 2013-11-20 07:06:04 -0800, Kevin Grittner wrote: > > That's not a bad point. So the way to fix it would be: > > > > 1) Restart the standby to the new minor release, wait for catchup > > 2) Restart the primary (fast or smart) to the new minor release > > 3) Acquire enough new xids to make sure w

Re: [HACKERS] Suggestion: Issue warning when calling SET TRANSACTION outside transaction block

2013-11-20 Thread Bruce Momjian
On Wed, Nov 20, 2013 at 10:04:22AM -0500, Tom Lane wrote: > Bruce Momjian writes: > > On Tue, Nov 19, 2013 at 10:21:47PM -0500, Tom Lane wrote: > >> My personal standpoint is that I don't care much whether these messages > >> are NOTICE or WARNING. What I'm not happy about is promoting cases that

Re: [HACKERS] Shave a few instructions from child-process startup sequence

2013-11-20 Thread Peter Eisentraut
On 11/5/13, 2:47 AM, Gurjeet Singh wrote: > On Mon, Nov 4, 2013 at 12:20 AM, Tom Lane > wrote: > > But we're not buying much. A few instructions during postmaster > shutdown > is entirely negligible. > > > The patch is for ClosePostmasterPorts(), which is

Re: [HACKERS] Autoconf 2.69 update

2013-11-20 Thread Magnus Hagander
On Wed, Nov 20, 2013 at 3:58 PM, Andres Freund wrote: > > On 2013-11-20 09:53:53 -0500, Tom Lane wrote: > > As a rule, you're not supposed to bother including the configure output > > script in a submitted diff anyway. Certainly any committer worth his > > commit bit is going to ignore it and red

Re: [HACKERS] Autoconf 2.69 update

2013-11-20 Thread Andrew Dunstan
On 11/20/2013 10:28 AM, Magnus Hagander wrote: On Wed, Nov 20, 2013 at 3:58 PM, Andres Freund > wrote: On 2013-11-20 09:53:53 -0500, Tom Lane wrote: > As a rule, you're not supposed to bother including the configure output > script in a submitte

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Tom Lane
Peter Eisentraut writes: > I would consider sidestepping this entire issue by having the > stand-alone backend create a Unix-domain socket and have a client > connect to that in the normal way. Hmm. But that requires the "stand-alone backend" to take on at least some properties of a postmaster;

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Andres Freund
On 2013-11-20 10:48:20 -0500, Tom Lane wrote: > constraining what can be executed as a "standalone backend". Would > it work to insist that psql/pg_dump launch the program named postgres > from the same bin directory they're in, rather than accepting a path > from the connection string? But why d

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Tom Lane
Andres Freund writes: > On 2013-11-20 10:48:20 -0500, Tom Lane wrote: >> constraining what can be executed as a "standalone backend". Would >> it work to insist that psql/pg_dump launch the program named postgres >> from the same bin directory they're in, rather than accepting a path >> from the

Re: [HACKERS] Handling GIN incomplete splits

2013-11-20 Thread Heikki Linnakangas
On 19.11.2013 14:48, Michael Paquier wrote: Here is a review of the first three patches: 1) Further gin refactoring: make check passes (core tests and contrib tests). Code compiles without warnings. Committed. Then... About the patch... Even if I got little experience with code of gin, moving

Re: [HACKERS] Extra functionality to createuser

2013-11-20 Thread Christopher Browne
On Tue, Nov 19, 2013 at 11:54 PM, Amit Kapila wrote: > On further tests, I found inconsistency in behavior when some special > characters are used in role names. > > 1. Test for role name containing quotes > a. In psql, create a role containing quotes in role name. >create role amitk in ro

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Andres Freund
On 2013-11-20 11:08:33 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2013-11-20 10:48:20 -0500, Tom Lane wrote: > >> constraining what can be executed as a "standalone backend". Would > >> it work to insist that psql/pg_dump launch the program named postgres > >> from the same bin directo

Re: [HACKERS] additional json functionality

2013-11-20 Thread Hannu Krosing
On 11/18/2013 06:49 PM, Josh Berkus wrote: > On 11/18/2013 06:13 AM, Peter Eisentraut wrote: >> On 11/15/13, 6:15 PM, Josh Berkus wrote: >>> Thing is, I'm not particularly concerned about *Merlin's* specific use >>> case, which there are ways around. What I am concerned about is that we >>> may hav

Re: [HACKERS] Extra functionality to createuser

2013-11-20 Thread Christopher Browne
Wait, that doesn't work if more than one role is added, as they get merged together by the quoting. A somewhat ugly amount of quoting can be done at the shell level to induce double quotes. $ createuser -g "\"test_rol'e_3\"" usequoted3 I note that similar (with not quite identical behaviour) iss

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-20 Thread Heikki Linnakangas
On 20.11.2013 17:06, Kevin Grittner wrote: Andres Freund wrote: On 2013-11-20 06:21:13 -0800, Kevin Grittner wrote: So as long as there are no open transactions or prepared transactions on the master which started before the release with the fix is applied, VACUUM FREEZE would be guar

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > I think we'd be better off trying to fix the security issue by > constraining what can be executed as a "standalone backend". Would > it work to insist that psql/pg_dump launch the program named postgres > from the same bin directory they're in, rather than

Re: [HACKERS] review: autovacuum_work_mem

2013-11-20 Thread Nigel Heron
On Mon, Nov 18, 2013 at 11:36 PM, Peter Geoghegan wrote: > Please reply to the original thread in future (even if the Reply-to > Message-ID is the same, I see this as a separate thread). > sorry about that, when i added "review" to the subject gmail removed the thread info. for reference the origi

Re: [HACKERS] additional json functionality

2013-11-20 Thread David Johnston
Hannu Krosing-3 wrote > On 11/18/2013 06:49 PM, Josh Berkus wrote: >> On 11/18/2013 06:13 AM, Peter Eisentraut wrote: >>> On 11/15/13, 6:15 PM, Josh Berkus wrote: Thing is, I'm not particularly concerned about *Merlin's* specific use case, which there are ways around. What I am concerned

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Andres Freund
On 2013-11-20 17:19:42 +0100, Andres Freund wrote: > > That just pushes the problem up a level --- how are you going to tell > > psql, pg_dump, or other programs that they should do that? > > An explicit parameter. A program imo explicitly needs to be aware that a > PQconnect() suddenly starts for

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-20 Thread Andres Freund
On 2013-11-20 18:25:56 +0200, Heikki Linnakangas wrote: > Isn't it possible that the standby has already incorrectly set > HEAP_XMIN_INVALID hint bit on a page? The full page images generated by > VACUUM FREEZE will correct the damage, but if not, e.g. because > full_page_writes=off, strange things

Re: [HACKERS] Logging WAL when updating hintbit

2013-11-20 Thread Sawada Masahiko
On Wed, Nov 20, 2013 at 9:19 PM, Dilip kumar wrote: > On 19 November 2013 22:19, Sawada Masahiko Wrote >>> > >> >> Thanks! >> I took it wrong. >> I think that there are quite a few difference amount of WAL. >> >> > Did you test about amount of WAL size in your patch? >> >> Not yet. I will do that.

Re: [HACKERS] [PATCH] Add transforms feature

2013-11-20 Thread Peter Eisentraut
On 11/15/13, 11:04 AM, Dimitri Fontaine wrote: > - Documentation style seems to be to be different from the "man page" > or "reference docs" style that we use elsewhere, and is instead > deriving the general case from examples. Reads strange. Which specific section do you have in mind?

Re: [HACKERS] GIN improvements part 1: additional information

2013-11-20 Thread Heikki Linnakangas
On 06.11.2013 17:36, Alvaro Herrera wrote: Just for my own illumination, can someone explain this bit? + If a posting list is too large to store in-line in a key entry, a posting tree + is created. A posting tree is a B-tree structure, where the ItemPointer is + used as the key. At the leaf-leve

Re: [HACKERS] additional json functionality

2013-11-20 Thread Greg Stark
On Sat, Nov 16, 2013 at 12:32 AM, Josh Berkus wrote: > On 11/15/2013 04:00 PM, David Johnston wrote: > > Looking at this a different way: could we just implement BSON and leave > json > > alone? > > > > http://bsonspec.org/ > > In short? No. > > For one thing, our storage format is different fro

[HACKERS] WITH ORDINALITY versus column definition lists

2013-11-20 Thread Tom Lane
Consider the following case of a function that requires a column definition list (example borrowed from the regression tests): create function array_to_set(anyarray) returns setof record as $$ select i AS "index", $1[i] AS "value" from generate_subscripts($1, 1) i $$ language sql strict immutabl

Re: [HACKERS] -d option for pg_isready is broken

2013-11-20 Thread Josh Berkus
On 11/20/2013 01:55 AM, Fujii Masao wrote: > Yeah, I agree that we should make the logic of pg_isready more future-proof > in 9.4dev. One idea is to expose internal_ping() as a libpq function. Then, > instead of just calling PQpingParams(), we can do PQconnectStartParams(), > libpq-function-version

Re: [HACKERS] additional json functionality

2013-11-20 Thread Josh Berkus
On 11/20/2013 04:52 AM, Robert Haas wrote: > I confess to being a bit perplexed by why we want hstore and json to > share a common binary format. hstore doesn't store hierarchical data; > json does. If we design a binary format for json, doesn't that just > obsolete store? Why go to a lot of tro

Re: [HACKERS] Storage formats for JSON WAS: additional json functionality

2013-11-20 Thread Josh Berkus
Greg, > Not being super familiar with either BSON our JSONB what advantages are we > gaining from the difference? We have the JSONB/Hstore2 format *now*, and it can go into 9.4. This makes it inherently superior to any theoretical format. So any further discussion (below) is strictly academic,

Re: [HACKERS] additional json functionality

2013-11-20 Thread Andrew Dunstan
On 11/20/2013 12:50 PM, Greg Stark wrote: On Sat, Nov 16, 2013 at 12:32 AM, Josh Berkus > wrote: On 11/15/2013 04:00 PM, David Johnston wrote: > Looking at this a different way: could we just implement BSON and leave json > alone? > > http://b

Re: [HACKERS] New option for pg_basebackup, to specify a different directory for pg_xlog

2013-11-20 Thread Gavin Flower
On 20/11/13 23:43, Haribabu kommi wrote: On 19 November 2013 19:12 Fujii Masao wrote: On Tue, Nov 19, 2013 at 9:14 PM, Haribabu kommi wrote: On 18 November 2013 23:30 Fujii Masao wrote: On Tue, Nov 19, 2013 at 12:01 AM, Haribabu kommi wrote: Thanks for newer version of the patch! I found t

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-20 Thread Josh Berkus
Andrews, Kevin: Presumably a replica created while all traffic was halted on the master would be clean, correct? This bug can only be triggered if there's heavy write load on the master, right? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Tom Lane
Andres Freund writes: > On 2013-11-20 11:08:33 -0500, Tom Lane wrote: >> Andres Freund writes: >>> Something like PQstartSingleUser(dsn) returning a established connection >>> seems better to me. >> That just pushes the problem up a level --- how are you going to tell >> psql, pg_dump, or other

Re: [HACKERS] WITH ORDINALITY versus column definition lists

2013-11-20 Thread David Johnston
Tom Lane-2 wrote > It seems to me that we don't really want this behavior of the coldeflist > not including the ordinality column. It's operating as designed, maybe, > but it's unexpected and confusing. We could either > > 1. Reinsert HEAD's prohibition against directly combining WITH ORDINALITY

Re: [HACKERS] additional json functionality

2013-11-20 Thread Tom Lane
David Johnston writes: >> On 11/18/2013 06:13 AM, Peter Eisentraut wrote: >>> We could do something like SQL/XML and specify the level of "validity" >>> in a typmod, e.g., json(loose), json(strict), etc. > Three things: > 1) How would this work in the face of functions that erase typemod > infor

Re: [HACKERS] Add CREATE support to event triggers

2013-11-20 Thread Christopher Browne
On Fri, Nov 8, 2013 at 10:33 AM, Alvaro Herrera wrote: > Hello, > > Attached you can find a very-much-WIP patch to add CREATE info support > for event triggers (normalized commands). This patch builds mainly on > two things: > > 1. Dimitri's "DDL rewrite" patch he submitted way back, in >http

Re: [HACKERS] nested hstore patch

2013-11-20 Thread David E. Wheeler
On Nov 20, 2013, at 6:19 AM, Peter Eisentraut wrote: > openjade:hstore.sgml:206:16:E: document type does not allow element > "VARLISTENTRY" here; assuming missing "VARIABLELIST" start-tag Thanks, I fixed this one. David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-20 Thread Josh Berkus
On 11/20/2013 10:30 AM, Josh Berkus wrote: > Andrews, Kevin: Andres, that is. > > Presumably a replica created while all traffic was halted on the master > would be clean, correct? This bug can only be triggered if there's > heavy write load on the master, right? > Also, just to verify: If s

Re: [HACKERS] additional json functionality

2013-11-20 Thread Andrew Dunstan
On 11/20/2013 01:36 PM, Tom Lane wrote: You'd have to make the data self-identifying (which I think was the plan already), and ensure that *every* function taking "json" could cope with both formats on input. The typmod could only be expected to be enforced when storing or explicitly casting t

[HACKERS] [PATCH] Store Extension Options

2013-11-20 Thread Fabrízio de Royes Mello
Hi all, The main goal of this patch is enable to an user the capability to store options (relations and attributes) related to extensions by using a fixed prefix called 'ext' in the defined name. It's cant be useful for replication solutions. So, with this patch we can do that: ALTER TABLE foo

Re: [HACKERS] WITH ORDINALITY versus column definition lists

2013-11-20 Thread Tom Lane
David Johnston writes: > Tom Lane-2 wrote >> It seems to me that we don't really want this behavior of the coldeflist >> not including the ordinality column. It's operating as designed, maybe, >> but it's unexpected and confusing. We could either >> >> 1. Reinsert HEAD's prohibition against dir

Re: [HACKERS] Dynamic Shared Memory stuff

2013-11-20 Thread Robert Haas
On Wed, Nov 20, 2013 at 8:32 AM, Heikki Linnakangas wrote: > I'm trying to catch up on all of this dynamic shared memory stuff. A bunch > of random questions and complaints: > > What kind of usage are we trying to cater with the dynamic shared memory? Parallel sort, and then parallel other stuff.

Re: [HACKERS] Replication Node Identifiers and crashsafe Apply Progress

2013-11-20 Thread Robert Haas
On Tue, Nov 19, 2013 at 1:20 PM, Andres Freund wrote: > On 2013-11-19 12:47:29 -0500, Robert Haas wrote: >> On Tue, Nov 19, 2013 at 11:57 AM, Andres Freund >> wrote: >> > Agreed. As an alternative we could just have a single - probably longer >> > than NAMEDATALEN - string to identify replicatio

Re: [HACKERS] UNNEST with multiple args, and TABLE with multiple funcs

2013-11-20 Thread Tom Lane
Andrew Gierth wrote: > The spec syntax for table function calls, > in , looks like TABLE(func(args...)) AS ... > This patch implements that, plus an extension: it allows multiple > functions, TABLE(func1(...), func2(...), func3(...)) [WITH ORDINALITY] > and defines this as meaning that the funct

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Peter Eisentraut
On 11/20/13, 11:31 AM, Stephen Frost wrote: > Couldn't that be an issue for people who have multiple major versions of > binaries installed? In particular, the "default" on the system for psql > might be 9.3 while the cluster you're trying to recover may be 9.2. Of > course, in that case you migh

Re: [HACKERS] GIN improvements part2: fast scan

2013-11-20 Thread Alexander Korotkov
On Wed, Nov 20, 2013 at 3:06 AM, Alexander Korotkov wrote: > On Fri, Nov 15, 2013 at 11:19 AM, Alexander Korotkov > wrote: > >> On Fri, Nov 15, 2013 at 12:34 AM, Heikki Linnakangas < >> hlinnakan...@vmware.com> wrote: >> >>> On 14.11.2013 19:26, Alexander Korotkov wrote: >>> On Sun, Jun 30,

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Peter Eisentraut
On 11/20/13, 10:48 AM, Tom Lane wrote: > Perhaps more to the point, I think this approach actually breaks one of > the principal good-thing-in-emergencies attributes of standalone mode, > namely being sure that nobody but you can connect. With this, you're > right back to having a race condition a

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Robert Haas
On Wed, Nov 20, 2013 at 10:13 AM, Peter Eisentraut wrote: > On 11/14/13, 1:41 AM, Amit Kapila wrote: >>Security Concern >>- >>If a user can specify libpq connection options, he can now execute >> any file he wants by passing it as standalone_backend. >> >>

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Peter Eisentraut
On 11/20/13, 3:24 PM, Robert Haas wrote: > The point is that client applications should expose whether or not to > set this function as a command-line switch separate from whatever they > accept in terms of connection strings. So pg_dump should have a flag > called --standalone-server or something

Re: [HACKERS] Easily reading debug_print_plan

2013-11-20 Thread Robert Haas
On Wed, Nov 20, 2013 at 9:36 AM, Tom Lane wrote: > We could in principle change to a different text representation for > stored rules. Compactness would be an issue if it were materially > bigger than the existing formatting, but offhand it seems like JSON > is morally equivalent to what we do no

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Robert Haas
On Wed, Nov 20, 2013 at 3:32 PM, Peter Eisentraut wrote: > On 11/20/13, 3:24 PM, Robert Haas wrote: >> The point is that client applications should expose whether or not to >> set this function as a command-line switch separate from whatever they >> accept in terms of connection strings. So pg_du

Re: [HACKERS] WITH ORDINALITY versus column definition lists

2013-11-20 Thread David Johnston
Tom Lane-2 wrote > David Johnston < > polobo@ > > writes: >> Tom Lane-2 wrote >>> It seems to me that we don't really want this behavior of the coldeflist >>> not including the ordinality column. It's operating as designed, maybe, >>> but it's unexpected and confusing. We could either >>> >>>

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Tom Lane
Peter Eisentraut writes: > The argument elsewhere in this thread was that the reason for putting > this in the connection options was so that you do *not* have to patch up > every client to be able to use this functionality. If you have to add > separate options everywhere, then you might as well

  1   2   >