[HACKERS] preproc.c compilation error

2014-01-09 Thread Rugal Bernstein
Hello all: This is my first time mail to all, yesterday I tried to compile postgresin my linux, but an error keep bother me. env: Ubuntu 13.10 Linux rugal-TM8473 3.11.0-15-generic #23-Ubuntu SMP Mon Dec 9 18:17:04 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux gcc version 4.8.1 (Ubuntu/Linaro

Re: [HACKERS] preproc.c compilation error

2014-01-09 Thread Rugal Bernstein
and even successfully compiled PG, I got warning preproc.y: In function ‘vmmerror’: preproc.y:76:2: warning: enumeration value ‘ET_FATAL’ not handled in switch [-Wswitch] switch(type) ^ 2014/1/9 Rugal Bernstein ryujinwr...@gmail.com Hello all: This is my first time mail to all,

Re: [HACKERS] [bug fix] multibyte messages are displayed incorrectly on the client

2014-01-09 Thread Robert Haas
On Tue, Jan 7, 2014 at 8:56 AM, MauMau maumau...@gmail.com wrote: I suppose we know (or at least believe) those encodings during backend startup: * client encoding - the client_encoding parameter passed in the startup packet, or if that's not present, client_encoding GUC value. * server

Re: [HACKERS] Standalone synchronous master

2014-01-09 Thread MauMau
From: Andres Freund and...@2ndquadrant.com On 2014-01-08 14:42:37 -0800, Joshua D. Drake wrote: If we have the following: db0-db1:down Using the model (as I understand it) that is being discussed we have increased our failure rate because the moment db1:down we also lose db0. The node db0

Re: [HACKERS] Failed assertion root-hasLateralRTEs on initsplan.c

2014-01-09 Thread Robert Haas
On Tue, Jan 7, 2014 at 1:15 PM, Tom Lane t...@sss.pgh.pa.us wrote: Emre Hasegeli e...@hasegeli.com writes: I get assertion failure on initsplan.c line 1325 while executing following query on HEAD (edc43458d797a5956f4bf39af18cf62abb0077db). It works fine without --enable-cassert. update

Re: [HACKERS] [bug fix] multibyte messages are displayed incorrectly on the client

2014-01-09 Thread MauMau
From: Robert Haas robertmh...@gmail.com Suppose the startup packet itself is malformed. How will you report the error? I think we have no choice but to report the error in English, because we don't know what the client wants. Regards MauMau -- Sent via pgsql-hackers mailing list

Re: [HACKERS] [bug fix] multibyte messages are displayed incorrectly on the client

2014-01-09 Thread MauMau
From: Robert Haas robertmh...@gmail.com Suppose the startup packet itself is malformed. How will you report the error? I think we have no choice but to report the error in English, because we don't know what the client wants. Regards MauMau -- Sent via pgsql-hackers mailing list

Re: [HACKERS] preproc.c compilation error

2014-01-09 Thread Michael Meskes
You have to rebuild the auto-generated pgc.c, preproc.c and preproc.y or simply remove them to force a rebuild. Michael -- Michael Meskes Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org Jabber:

Re: [HACKERS] Recovery to backup point

2014-01-09 Thread Heikki Linnakangas
On 12/09/2013 03:05 PM, MauMau wrote: From: Heikki Linnakangas hlinnakan...@vmware.com Thanks. Looks sane, although I don't much like the proposed interface to trigger this, setting recovery_target_time='backup_point'. What the code actually does is to stop recovery as soon as you reach

Re: [HACKERS] Planning time in explain/explain analyze

2014-01-09 Thread Stephen Frost
Andreas, Robert, * Andreas Karlsson (andr...@proxel.se) wrote: A patch with updated documentation is attached. Thanks for working on this! On 01/02/2014 04:08 AM, Robert Haas wrote: I'm wondering whether the time should be stored inside the PlannedStmt node instead of passing it around

Re: [HACKERS] array_length(anyarray)

2014-01-09 Thread Dean Rasheed
On 19 December 2013 08:05, Pavel Stehule pavel.steh...@gmail.com wrote: 2013/12/19 David Fetter da...@fetter.org On Wed, Dec 18, 2013 at 09:27:54PM +0100, Marko Tiikkaja wrote: Hi, Attached is a patch to add support for array_length(anyarray), which only works for one-dimensional

Re: [HACKERS] nested hstore patch

2014-01-09 Thread Andrew Dunstan
On 01/08/2014 04:29 PM, Oleg Bartunov wrote: Attached is a new version of patch, which addresses most issues raised by Andres. It's long holidays in Russia now and it happened that Teodor is traveling with family, so Teodor asked me to reply. Comments in code will be added asap. Oleg,

Re: [HACKERS] preproc.c compilation error

2014-01-09 Thread Rugal Bernstein
Thanks, seems it is because the first time pgc.c and others are generated, but [make distclean] did not clean them, which lead to this problem! after [rm pgc.c preproc.h preproc.c preproc.y] it is now successfully compiled without any warning! thank you! Java Developer; Mysql/Oracle DBA;

Re: [HACKERS] preproc.c compilation error

2014-01-09 Thread Andres Freund
On 2014-01-09 22:12:53 +0800, Rugal Bernstein wrote: Thanks, seems it is because the first time pgc.c and others are generated, but [make distclean] did not clean them, which lead to this problem! after [rm pgc.c preproc.h preproc.c preproc.y] it is now successfully compiled without any

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

2014-01-09 Thread Amit Kapila
On Fri, Dec 6, 2013 at 6:41 PM, Amit Kapila amit.kapil...@gmail.com wrote: Agreed, summarization of data for LZ/Chunkwise encoding especially for non-compressible (hundred tiny fields, all changed/half changed) or less compressible data (hundred tiny fields, half nulled) w.r.t CPU

Re: [HACKERS] Standalone synchronous master

2014-01-09 Thread Hannu Krosing
On 01/09/2014 05:09 AM, Robert Treat wrote: On Wed, Jan 8, 2014 at 6:15 PM, Josh Berkus j...@agliodbs.com wrote: Stephen, I'm aware, my point was simply that we should state, up-front in 25.2.7.3 *and* where we document synchronous_standby_names, that it requires at least three servers to

Re: [HACKERS] Standalone synchronous master

2014-01-09 Thread Hannu Krosing
On 01/09/2014 12:05 AM, Stephen Frost wrote: * Andres Freund (and...@2ndquadrant.com) wrote: On 2014-01-08 17:56:37 -0500, Stephen Frost wrote: * Andres Freund (and...@2ndquadrant.com) wrote: That's why you should configure a second standby as another (candidate) synchronous replica, also

Re: [HACKERS] Standalone synchronous master

2014-01-09 Thread Hannu Krosing
On 01/08/2014 11:49 PM, Tom Lane wrote: Joshua D. Drake j...@commandprompt.com writes: On 01/08/2014 01:55 PM, Tom Lane wrote: Sync mode is about providing a guarantee that the data exists on more than one server *before* we tell the client it's committed. If you don't need that guarantee,

Re: [HACKERS] Standalone synchronous master

2014-01-09 Thread Hannu Krosing
On 01/09/2014 01:57 PM, MauMau wrote: From: Andres Freund and...@2ndquadrant.com On 2014-01-08 14:42:37 -0800, Joshua D. Drake wrote: If we have the following: db0-db1:down Using the model (as I understand it) that is being discussed we have increased our failure rate because the moment

Re: [HACKERS] Standalone synchronous master

2014-01-09 Thread Hannu Krosing
On 01/09/2014 02:01 AM, Jim Nasby wrote: On 1/8/14, 6:05 PM, Tom Lane wrote: Josh Berkusj...@agliodbs.com writes: On 01/08/2014 03:27 PM, Tom Lane wrote: What we lack, and should work on, is a way for sync mode to have M larger than one. AFAICS, right now we'll report commit as soon as

Re: [HACKERS] Recovery to backup point

2014-01-09 Thread MauMau
From: Heikki Linnakangas hlinnakan...@vmware.com After some refactoring and fixing bugs in the existing code, I came up with the attached patch. I called the option simply recovery_target, with the only allowed value of immediate. IOW, if you want to stop recovery as early as possible, you add

Re: [HACKERS] [bug fix] ECPG app crashes due to SIGBUS on SPARC Solaris

2014-01-09 Thread Michael Meskes
On Sun, Jan 05, 2014 at 03:42:42PM +0900, MauMau wrote: I ran the ECPG regression test with the unpatched 64-bit PostgreSQL 9.2.4 on SPARC Solaris, and it succeeded (all 54 tests passed). For ... Thanks a lot. Patch applied to HEAD and all the backbranches. Will push shortly. Michael --

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-09 Thread Dean Rasheed
On 15 December 2013 01:57, Tom Lane t...@sss.pgh.pa.us wrote: Josh Berkus j...@agliodbs.com writes: I think even the FLOAT case deserves some consideration. What's the worst-case drift? Complete loss of all significant digits. The case I was considering earlier of single-row windows could

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-01-09 Thread Tomonari Katsumata
Hi, Somebody is reading this thread? This problem seems still remaining on REL9_3_STABLE. Many users would face this problem, so we should resolve this in next release. I think his patch is reasonable to fix this problem. Please check this again. regards, -- Tomonari

Re: [HACKERS] Standalone synchronous master

2014-01-09 Thread MauMau
From: Hannu Krosing ha...@2ndquadrant.com On 01/09/2014 01:57 PM, MauMau wrote: Let me ask a (probably) stupid question. How is the sync rep different from RAID-1? When I first saw sync rep, I expected that it would provide the same guarantees as RAID-1 in terms of durability (data is always

Re: [HACKERS] WIP patch (v2) for updatable security barrier views

2014-01-09 Thread Tom Lane
Dean Rasheed dean.a.rash...@gmail.com writes: My first thought was that it should just preprocess any security barrier quals in subquery_planner() in the same way as other quals are preprocessed. But thinking about it further, those quals are destined to become the quals of subqueries in the

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-01-09 Thread Amit Langote
On Thu, Dec 12, 2013 at 11:00 AM, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: Hello, we happened to see server crash on archive recovery under some condition. After TLI was incremented, there should be the case that the WAL file for older timeline is archived but not for that of

Re: [HACKERS] Failed assertion root-hasLateralRTEs on initsplan.c

2014-01-09 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Jan 7, 2014 at 1:15 PM, Tom Lane t...@sss.pgh.pa.us wrote: The next question is if we should allow it with LATERAL. That would essentially be treating subscriber as having implicitly appeared at the start of the FROM list, which I guess is all

Re: [HACKERS] Standalone synchronous master

2014-01-09 Thread Hannu Krosing
On 01/09/2014 04:15 PM, MauMau wrote: From: Hannu Krosing ha...@2ndquadrant.com On 01/09/2014 01:57 PM, MauMau wrote: Let me ask a (probably) stupid question. How is the sync rep different from RAID-1? When I first saw sync rep, I expected that it would provide the same guarantees as

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-09 Thread Tom Lane
Dean Rasheed dean.a.rash...@gmail.com writes: Reading over this, I realised that there is a problem with NaN handling --- once the state becomes NaN, it can never recover. So the results using the inverse transition function don't match HEAD in cases like this: Ouch! That takes out numeric,

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-09 Thread Florian Pflug
On Jan9, 2014, at 17:15 , Tom Lane t...@sss.pgh.pa.us wrote: Dean Rasheed dean.a.rash...@gmail.com writes: Reading over this, I realised that there is a problem with NaN handling --- once the state becomes NaN, it can never recover. So the results using the inverse transition function don't

Re: [HACKERS] array_length(anyarray)

2014-01-09 Thread Florian Pflug
On Jan9, 2014, at 14:57 , Dean Rasheed dean.a.rash...@gmail.com wrote: On 19 December 2013 08:05, Pavel Stehule pavel.steh...@gmail.com wrote: length should be irrelevant to fact so array starts from 1, 0 or anything else Yes, this should just return the number of elements, and 0 for an

Re: [HACKERS]

2014-01-09 Thread Robert Haas
On Tue, Jan 7, 2014 at 10:55 PM, Dilip kumar dilip.ku...@huawei.com wrote: Below attached patch is implementing following todo item.. machine-readable pg_controldata? http://www.postgresql.org/message-id/4b901d73.8030...@agliodbs.com Possible approaches: 1. Implement as backend

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2014-01-09 Thread Robert Haas
On Wed, Jan 8, 2014 at 3:33 AM, Simon Riggs si...@2ndquadrant.com wrote: VACUUM cleans up blocks, which is nice because it happens offline in a lazy manner. We also make SELECT clean up blocks as it goes. That is useful in OLTP workloads, but it means that large SQL queries and pg_dump

Re: [HACKERS] array_length(anyarray)

2014-01-09 Thread Marko Tiikkaja
On 1/9/14 5:44 PM, Florian Pflug wrote: On Jan9, 2014, at 14:57 , Dean Rasheed dean.a.rash...@gmail.com wrote: On 19 December 2013 08:05, Pavel Stehule pavel.steh...@gmail.com wrote: length should be irrelevant to fact so array starts from 1, 0 or anything else Yes, this should just return

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-09 Thread Tom Lane
Florian Pflug f...@phlo.org writes: For float 4 and float8, wasn't the consensus that the potential lossy-ness of addition makes this impossible anyway, even without the NaN issue? But... Well, that was my opinion, I'm not sure if it was consensus ;-). But NaN is an orthogonal problem I think.

Re: [HACKERS] Standalone synchronous master

2014-01-09 Thread Bruce Momjian
On Thu, Jan 9, 2014 at 04:55:22PM +0100, Hannu Krosing wrote: On 01/09/2014 04:15 PM, MauMau wrote: From: Hannu Krosing ha...@2ndquadrant.com On 01/09/2014 01:57 PM, MauMau wrote: Let me ask a (probably) stupid question. How is the sync rep different from RAID-1? When I first saw

Re: [HACKERS] [ANNOUNCE] IMCS: In Memory Columnar Store for PostgreSQL

2014-01-09 Thread Robert Haas
On Wed, Jan 8, 2014 at 2:39 PM, knizhnik knizh...@garret.ru wrote: I wonder what is the intended use case of dynamic shared memory? Is is primarly oriented on PostgreSQL extensions or it will be used also in PosatgreSQL core? My main motivation is that I want to use it to support parallel

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2014-01-09 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Wed, Jan 8, 2014 at 3:33 AM, Simon Riggs si...@2ndquadrant.com wrote: We also make SELECT clean up blocks as it goes. That is useful in OLTP workloads, but it means that large SQL queries and pg_dump effectively do much the same work as VACUUM,

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2014-01-09 Thread Robert Haas
On Thu, Jan 9, 2014 at 12:21 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Jan 8, 2014 at 3:33 AM, Simon Riggs si...@2ndquadrant.com wrote: We also make SELECT clean up blocks as it goes. That is useful in OLTP workloads, but it means that large SQL

Re: [HACKERS] newlines at end of generated SQL

2014-01-09 Thread Robert Haas
On Wed, Jan 8, 2014 at 10:17 PM, Peter Eisentraut pete...@gmx.net wrote: Is there a reason why the programs in src/bin/scripts all put newlines at the end of the SQL commands they generate? This produces useless empty lines in the server log (and client output, if selected). If you're asking

Re: [HACKERS] Standalone synchronous master

2014-01-09 Thread Jeff Janes
On Wed, Jan 8, 2014 at 3:00 PM, Josh Berkus j...@agliodbs.com wrote: On 01/08/2014 01:49 PM, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: If we really want auto-degrading sync rep, then we'd (at a minimum) need a way to determine *from the replica* whether or not it was in

Re: [HACKERS] Add CREATE support to event triggers

2014-01-09 Thread Robert Haas
On Wed, Jan 8, 2014 at 10:27 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Craig Ringer escribió: Instead, can't we use your already proposed subclause structure? {authorization:{authorization_role:some guy, output:AUTHORIZATION %i{authorization_role}}, if_not_exists:

Re: [HACKERS] integrate pg_upgrade analyze_new_cluster.sh into vacuumdb

2014-01-09 Thread Robert Haas
On Wed, Jan 8, 2014 at 10:32 PM, Tom Lane t...@sss.pgh.pa.us wrote: Peter Eisentraut pete...@gmx.net writes: pg_upgrade creates a script analyze_new_cluster.{sh|bat} that runs vacuumdb --analyze-only in three stages with different statistics target settings to get a fresh cluster analyzed

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-09 Thread Florian Pflug
On Jan9, 2014, at 18:09 , Tom Lane t...@sss.pgh.pa.us wrote: Florian Pflug f...@phlo.org writes: For float 4 and float8, wasn't the consensus that the potential lossy-ness of addition makes this impossible anyway, even without the NaN issue? But... Well, that was my opinion, I'm not sure if

Re: [HACKERS] integrate pg_upgrade analyze_new_cluster.sh into vacuumdb

2014-01-09 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Wed, Jan 8, 2014 at 10:32 PM, Tom Lane t...@sss.pgh.pa.us wrote: I don't think this vacuumdb feature should deal with any version-conversion issues. So it sounds like the thing to do is keep the wrapper script, which will give us a place to put any

Re: [HACKERS] Standalone synchronous master

2014-01-09 Thread Bruce Momjian
On Thu, Jan 9, 2014 at 09:36:47AM -0800, Jeff Janes wrote: Oh, right. Because the main reason for a sync replica degrading is that it's down. In which case it isn't going to record anything. This would still be useful for sync rep candidates, though, and I'll document why

Re: [HACKERS] [ANNOUNCE] IMCS: In Memory Columnar Store for PostgreSQL

2014-01-09 Thread Claudio Freire
On Thu, Jan 9, 2014 at 2:22 PM, Robert Haas robertmh...@gmail.com wrote: It would be nice to have better operating system support for this. For example, IIUC, 64-bit Linux has 128TB of address space available for user processes. When you clone(), it can either share the entire address space

Re: [HACKERS] integrate pg_upgrade analyze_new_cluster.sh into vacuumdb

2014-01-09 Thread Robert Haas
On Thu, Jan 9, 2014 at 12:44 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Jan 8, 2014 at 10:32 PM, Tom Lane t...@sss.pgh.pa.us wrote: I don't think this vacuumdb feature should deal with any version-conversion issues. So it sounds like the thing to

Re: [HACKERS] Add CREATE support to event triggers

2014-01-09 Thread Alvaro Herrera
Robert Haas escribió: On Wed, Jan 8, 2014 at 10:27 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Hmm. This seems like a reasonable thing to do, except that I would like the output to always be the constant, and have some other way to enable the clause or disable it. With your

Re: [HACKERS] integrate pg_upgrade analyze_new_cluster.sh into vacuumdb

2014-01-09 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: I was referring to the analyze-in-stages logic, which is not specific to 8.4. I don't see a reason not to put that into vacuumdb. Right, that's Peter's core proposal, which I agreed with. The issue was what to do with some other steps that pg_upgrade

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2014-01-09 Thread Alvaro Herrera
Robert Haas escribió: Unfortunately, there's no categorical answer. You can come up with workloads where HOT pruning on selects is a win; just create a bunch of junk and then read the same pages lots of times in a row. And you can also come up with workloads where it's a loss; create a

Re: [HACKERS] Standalone synchronous master

2014-01-09 Thread Josh Berkus
Robert, I think the problem here is that we tend to have a limited view of the right way to use synch rep. If I have 5 nodes, and I set 1 synchronous and the other 3 asynchronous, I've set up a known successor in the event that the leader fails. In this scenario though, if the successor

Re: [HACKERS] [PATCH] Relocation of tablespaces in pg_basebackup

2014-01-09 Thread Andreas Karlsson
On 01/09/2014 06:58 PM, Steeve Lennmark wrote: This patch adds the ability to relocate tablespaces by adding the command line argument --tablespace (-T) which takes a required argument in the format oid:tablespacedir. After all tablespaces are fetched this code updates the symlink to point to

Re: [HACKERS] [PATCH] Relocation of tablespaces in pg_basebackup

2014-01-09 Thread Magnus Hagander
On Thu, Jan 9, 2014 at 6:58 PM, Steeve Lennmark stee...@handeldsbanken.sewrote: Currently pg_basebackup is pretty invasive when using tablespaces, at least using the plain format. This since it requires the tablespace to be written to the same location as on the server beeing backed up. This

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-01-09 Thread Heikki Linnakangas
On 12/12/2013 04:00 AM, Kyotaro HORIGUCHI wrote: Hello, we happened to see server crash on archive recovery under some condition. After TLI was incremented, there should be the case that the WAL file for older timeline is archived but not for that of the same segment id but for newer timeline.

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2014-01-09 Thread Simon Riggs
On 9 January 2014 17:21, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Jan 8, 2014 at 3:33 AM, Simon Riggs si...@2ndquadrant.com wrote: We also make SELECT clean up blocks as it goes. That is useful in OLTP workloads, but it means that large SQL queries

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2014-01-09 Thread Tom Lane
Alvaro Herrera alvhe...@2ndquadrant.com writes: Robert Haas escribió: But the argument that we don't need a parameter because one behavior is best for everyone is not going to fly. In the above, there's the underlying assumption that it doesn't matter *what* we do with the page after doing

Re: [HACKERS] Standalone synchronous master

2014-01-09 Thread Simon Riggs
On 8 January 2014 21:40, Tom Lane t...@sss.pgh.pa.us wrote: Kevin Grittner kgri...@ymail.com writes: I'm torn on whether we should cave to popular demand on this; but if we do, we sure need to be very clear in the documentation about what a successful return from a commit request means.

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2014-01-09 Thread Stephen Frost
* Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: But in a SELECT, the effect is only that you will have to skip less dead tuples, which is not as exciting. Agreed. There's also the option to have it be done based on some expectation of future work- that is, if we have to traverse X number

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2014-01-09 Thread Stephen Frost
* Simon Riggs (si...@2ndquadrant.com) wrote: I'm -1 for a parameter as well, but I think that just stopping SELECTs from doing pruning at all might well be a win. It's at least worthy of some investigation. Turning HOT off completely would be an absolute disaster for OLTP on high update

Re: [HACKERS] [PATCH] pg_basebackup: progress report max once per second

2014-01-09 Thread Magnus Hagander
On Thu, Nov 14, 2013 at 10:27 AM, Mika Eloranta m...@ohmu.fi wrote: On 13 Nov 2013, at 20:51, Mika Eloranta m...@ohmu.fi wrote: Prevent excessive progress reporting that can grow to gigabytes of output with large databases. Same patch as an attachment. Would it not make more sense to

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2014-01-09 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: That said, I'm not entirely convinced that traversing these dead tuples is all *that* painful during SELECT. If there's that many levels then hopefully it's not long til an UPDATE comes along and cleans them up. There's always VACUUM ;-) If you take

Re: [HACKERS] nested hstore patch

2014-01-09 Thread Josh Berkus
On 01/09/2014 06:12 AM, Andrew Dunstan wrote: Oleg, Please merge in the jsonb work and resubmit. See https://github.com/feodor/postgres/commits/jsonb_and_hstore I note that this repo does not apparently contain any of your latest changes. I'll go further and say that if the Hstore2 patch

Re: [HACKERS] [ANNOUNCE] IMCS: In Memory Columnar Store for PostgreSQL

2014-01-09 Thread Amit Kapila
On Thu, Jan 9, 2014 at 12:21 AM, Robert Haas robertmh...@gmail.com wrote: On Tue, Jan 7, 2014 at 10:20 PM, Amit Kapila amit.kapil...@gmail.com wrote: On Tue, Jan 7, 2014 at 2:46 AM, Robert Haas robertmh...@gmail.com wrote: Well, right now we just reopen the same object from all of the

Re: [HACKERS] [ANNOUNCE] IMCS: In Memory Columnar Store for PostgreSQL

2014-01-09 Thread knizhnik
On 01/09/2014 09:22 PM, Robert Haas wrote: On Wed, Jan 8, 2014 at 2:39 PM, knizhnik knizh...@garret.ru wrote: I wonder what is the intended use case of dynamic shared memory? Is is primarly oriented on PostgreSQL extensions or it will be used also in PosatgreSQL core? My main motivation is

Re: [HACKERS] [ANNOUNCE] IMCS: In Memory Columnar Store for PostgreSQL

2014-01-09 Thread knizhnik
On 01/09/2014 09:46 PM, Claudio Freire wrote: On Thu, Jan 9, 2014 at 2:22 PM, Robert Haas robertmh...@gmail.com wrote: It would be nice to have better operating system support for this. For example, IIUC, 64-bit Linux has 128TB of address space available for user processes. When you clone(),

[HACKERS] [PATCH] Filter error log statements by sqlstate

2014-01-09 Thread Oskari Saarenmaa
Allow the default log_min_error_statement to be overridden per sqlstate to make it possible to filter out some error types while maintaining a low log_min_error_statement or enable logging for some error types when the default is to not log anything. I've tried to do something like this using

Re: [HACKERS] [ANNOUNCE] IMCS: In Memory Columnar Store for PostgreSQL

2014-01-09 Thread Claudio Freire
On Thu, Jan 9, 2014 at 4:24 PM, knizhnik knizh...@garret.ru wrote: On 01/09/2014 09:46 PM, Claudio Freire wrote: On Thu, Jan 9, 2014 at 2:22 PM, Robert Haas robertmh...@gmail.com wrote: It would be nice to have better operating system support for this. For example, IIUC, 64-bit Linux has

Re: [HACKERS] [ANNOUNCE] IMCS: In Memory Columnar Store for PostgreSQL

2014-01-09 Thread knizhnik
On 01/09/2014 11:09 PM, Amit Kapila wrote: On Thu, Jan 9, 2014 at 12:21 AM, Robert Haas robertmh...@gmail.com wrote: On Tue, Jan 7, 2014 at 10:20 PM, Amit Kapila amit.kapil...@gmail.com wrote: On Tue, Jan 7, 2014 at 2:46 AM, Robert Haas robertmh...@gmail.com wrote: Well, right now we just

Re: [HACKERS] [ANNOUNCE] IMCS: In Memory Columnar Store for PostgreSQL

2014-01-09 Thread Amit Kapila
On Fri, Jan 10, 2014 at 1:00 AM, knizhnik knizh...@garret.ru wrote: On 01/09/2014 11:09 PM, Amit Kapila wrote: Using DuplicateHandle(), we can make segment stick for Postmaster lifetime. I have used below test (used dsm_demo module) to verify: As far as I understand DuplicateHandle() should

Re: [HACKERS] [ANNOUNCE] IMCS: In Memory Columnar Store for PostgreSQL

2014-01-09 Thread knizhnik
On 01/09/2014 11:30 PM, Claudio Freire wrote: On Thu, Jan 9, 2014 at 4:24 PM, knizhnik knizh...@garret.ru wrote: On 01/09/2014 09:46 PM, Claudio Freire wrote: On Thu, Jan 9, 2014 at 2:22 PM, Robert Haas robertmh...@gmail.com wrote: It would be nice to have better operating system support for

Re: [HACKERS] [ANNOUNCE] IMCS: In Memory Columnar Store for PostgreSQL

2014-01-09 Thread Claudio Freire
On Thu, Jan 9, 2014 at 4:39 PM, knizhnik knizh...@garret.ru wrote: At fork time I only wrote about reserving the address space. After reserving it, all you have to do is implement an allocator that works in shared memory (protected by a lwlock of course). In essence, a hypothetical

Re: [HACKERS] [ANNOUNCE] IMCS: In Memory Columnar Store for PostgreSQL

2014-01-09 Thread Claudio Freire
On Thu, Jan 9, 2014 at 4:48 PM, Claudio Freire klaussfre...@gmail.com wrote: On Thu, Jan 9, 2014 at 4:39 PM, knizhnik knizh...@garret.ru wrote: At fork time I only wrote about reserving the address space. After reserving it, all you have to do is implement an allocator that works in shared

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-01-09 Thread Heikki Linnakangas
On 01/09/2014 08:18 PM, Heikki Linnakangas wrote: On 12/12/2013 04:00 AM, Kyotaro HORIGUCHI wrote: Hello, we happened to see server crash on archive recovery under some condition. After TLI was incremented, there should be the case that the WAL file for older timeline is archived but not for

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-01-09 Thread Tom Lane
Heikki Linnakangas hlinnakan...@vmware.com writes: Actually, why is the partially-filled 00010002 file archived in the first place? ... So, the rationale is that otherwise it would take a long time until that segment is archived. To be precise, I don't think the segment with

Re: [HACKERS] [ANNOUNCE] IMCS: In Memory Columnar Store for PostgreSQL

2014-01-09 Thread knizhnik
On 01/09/2014 11:48 PM, Claudio Freire wrote: On Thu, Jan 9, 2014 at 4:39 PM, knizhnik knizh...@garret.ru wrote: At fork time I only wrote about reserving the address space. After reserving it, all you have to do is implement an allocator that works in shared memory (protected by a lwlock of

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-01-09 Thread Heikki Linnakangas
On 01/09/2014 10:16 PM, Tom Lane wrote: Heikki Linnakangas hlinnakan...@vmware.com writes: Actually, why is the partially-filled 00010002 file archived in the first place? ... So, the rationale is that otherwise it would take a long time until that segment is archived. To be

[HACKERS] [PATCH] pgcrypto: implement gen_random_uuid

2014-01-09 Thread Oskari Saarenmaa
The only useful feature of the uuid-ossp module in my opinion is the uuid_generate_v4 function and as uuid-ossp is more or less abandonware people have had trouble building and installing it. This patch implements an alternative uuid v4 generation function in pgcrypto which could be moved to

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-01-09 Thread Tom Lane
Heikki Linnakangas hlinnakan...@vmware.com writes: On 01/09/2014 10:16 PM, Tom Lane wrote: Don't we want to archive both? If you want to recover to the end of the old timeline, you're going to need that file too, no? Hmm. It should be the responsibility of the original server to archive

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-01-09 Thread Heikki Linnakangas
On 01/09/2014 10:36 PM, Tom Lane wrote: Heikki Linnakangas hlinnakan...@vmware.com writes: On 01/09/2014 10:16 PM, Tom Lane wrote: Don't we want to archive both? If you want to recover to the end of the old timeline, you're going to need that file too, no? Hmm. It should be the

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-01-09 Thread Josh Berkus
On 01/09/2014 12:05 PM, Heikki Linnakangas wrote: Actually, why is the partially-filled 00010002 file archived in the first place? Looking at the code, it's been like that forever, but it seems like a bad idea. If the original server is still up and running, and writing more

Re: [HACKERS] Planning time in explain/explain analyze

2014-01-09 Thread Greg Stark
On Thu, Jan 9, 2014 at 1:50 PM, Stephen Frost sfr...@snowman.net wrote: I do not think we want to always measure the time it took to generate a plan due to slow clocks on some architectures. Also I feel that such a patch would be more invasive. The slow-clock argument is really quite

Re: [HACKERS] [ANNOUNCE] IMCS: In Memory Columnar Store for PostgreSQL

2014-01-09 Thread Jim Nasby
On 1/9/14, 1:18 PM, knizhnik wrote: So it is clear why do we need shared memory for parallel query execution. But why it has to be dynamic? Why it can not be preallocated at start time as most of other resources used by PostgreSQL? That would limit us to doing something like allocating a

Re: [HACKERS] Planning time in explain/explain analyze

2014-01-09 Thread Tom Lane
Greg Stark st...@mit.edu writes: However I don't see the issue here. Two gettimeofday calls per query plan is not really going to hurt even on systems where it's slow. I tend to agree with this, especially if the calls only occur when the user asks for the information (ie, does an EXPLAIN

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2014-01-09 Thread Jim Nasby
On 1/9/14, 12:54 PM, Tom Lane wrote: Stephen Frost sfr...@snowman.net writes: That said, I'm not entirely convinced that traversing these dead tuples is all *that* painful during SELECT. If there's that many levels then hopefully it's not long til an UPDATE comes along and cleans them up.

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2014-01-09 Thread Robert Haas
On Thu, Jan 9, 2014 at 1:54 PM, Tom Lane t...@sss.pgh.pa.us wrote: Stephen Frost sfr...@snowman.net writes: That said, I'm not entirely convinced that traversing these dead tuples is all *that* painful during SELECT. If there's that many levels then hopefully it's not long til an UPDATE comes

Re: [HACKERS] [PATCH] Relocation of tablespaces in pg_basebackup

2014-01-09 Thread Gabriele Bartolini
Hi Steeve, Il 09/01/14 22:10, Steeve Lennmark ha scritto: That's a much better solution, I attached a patch with the updated code. # SELECT oid, pg_tablespace_location(oid) FROM pg_tablespace; [...] 16388 | /tmp/tblspc1 16389 | /tmp/tblspc2 I'd suggest, a similar solution to the one we

Re: [HACKERS] Standalone synchronous master

2014-01-09 Thread Jim Nasby
On 1/9/14, 9:01 AM, Hannu Krosing wrote: Yeah, and I think that the logging command that was suggested allows for that*if configured correctly*. *But* for relying on this, we would also need to make logging *synchronous*, which would probably not go down well with many people, as it makes

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2014-01-09 Thread Andres Freund
On 2014-01-09 16:27:23 -0500, Robert Haas wrote: People *think* they don't like that, because that's the way it works right now. If it worked some other way, there's a good chance people would be complaining about that behavior, too. I think on of the primary reason why it's causing huge

Re: [HACKERS] [PATCH] Relocation of tablespaces in pg_basebackup

2014-01-09 Thread Steeve Lennmark
On Thu, Jan 9, 2014 at 10:29 PM, Gabriele Bartolini gabriele.bartol...@2ndquadrant.it wrote: Hi Steeve, Il 09/01/14 22:10, Steeve Lennmark ha scritto: That's a much better solution, I attached a patch with the updated code. # SELECT oid, pg_tablespace_location(oid) FROM pg_tablespace;

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2014-01-09 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: The problem with saying that we should let VACUUM do this work is the same as the problem with saying that if you're late for your Concorde flight, you should go running across the tarmac and try to catch it. The cost of dead tuples is related in a

Re: [HACKERS] Planning time in explain/explain analyze

2014-01-09 Thread Greg Stark
On Thu, Jan 9, 2014 at 9:14 PM, Tom Lane t...@sss.pgh.pa.us wrote: In short then, I think we should just add this to EXPLAIN and be done. -1 for sticking the info into PlannedStmt or anything like that. I'm confused. I thought I was arguing to support your suggestion that the initial planning

Re: [HACKERS] Add CREATE support to event triggers

2014-01-09 Thread Jim Nasby
On 1/9/14, 11:58 AM, Alvaro Herrera wrote: Robert Haas escribió: On Wed, Jan 8, 2014 at 10:27 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Hmm. This seems like a reasonable thing to do, except that I would like the output to always be the constant, and have some other way to enable

Re: [HACKERS] array_length(anyarray)

2014-01-09 Thread Jim Nasby
On 1/9/14, 11:08 AM, Marko Tiikkaja wrote: On 1/9/14 5:44 PM, Florian Pflug wrote: On Jan9, 2014, at 14:57 , Dean Rasheed dean.a.rash...@gmail.com wrote: On 19 December 2013 08:05, Pavel Stehule pavel.steh...@gmail.com wrote: length should be irrelevant to fact so array starts from 1, 0 or

[HACKERS] Disallow arrays with non-standard lower bounds

2014-01-09 Thread Jim Nasby
ISTM that allowing users to pick arbitrary lower array bounds was a huge mistake. I've never seen anyone make use of it, can't think of any legitimate use cases for it, and hate the stupendous amount of extra code needed to deal with it. Obviously we can't just drop support, but what about an

Re: [HACKERS] nested hstore patch

2014-01-09 Thread Oleg Bartunov
I moved patch to the January commitfest (https://commitfest.postgresql.org/action/patch_view?id=1289) . Oleg PS. Kudos to Teodor and his mobile phone, which he used to synchronize branches on github. On Fri, Jan 10, 2014 at 2:08 AM, Andrew Dunstan and...@dunslane.net wrote: On 01/09/2014

[HACKERS] proposal, patch: allow multiple plpgsql plugins

2014-01-09 Thread Pavel Stehule
Hello We talked about enhancing a plpgsql plugin API to support more active plugins. I wrote a prototype based on function plpgsql_register_plugin instead rendezvous variable. There are two basic questions: a) will we support rendezvous variable still? b) will we support same API still - a

Re: [HACKERS] Disallow arrays with non-standard lower bounds

2014-01-09 Thread Peter Geoghegan
On Thu, Jan 9, 2014 at 2:30 PM, Jim Nasby j...@nasby.net wrote: ISTM that allowing users to pick arbitrary lower array bounds was a huge mistake. I've never seen anyone make use of it, can't think of any legitimate use cases for it, and hate the stupendous amount of extra code needed to deal

Re: [HACKERS] array_length(anyarray)

2014-01-09 Thread Florian Pflug
On Jan9, 2014, at 23:26 , Jim Nasby j...@nasby.net wrote: On 1/9/14, 11:08 AM, Marko Tiikkaja wrote: On 1/9/14 5:44 PM, Florian Pflug wrote: On Jan9, 2014, at 14:57 , Dean Rasheed dean.a.rash...@gmail.com wrote: On 19 December 2013 08:05, Pavel Stehule pavel.steh...@gmail.com wrote: length

  1   2   >