Re: [HACKERS] [PATCH] introduce XLogLockBlockRangeForCleanup()

2014-07-03 Thread Abhijit Menon-Sen
At 2014-07-03 11:15:53 +0530, amit.khande...@enterprisedb.com wrote: In GetBufferWithoutRelcache(), I was wondering, rather than calling PinBuffer(), if we do this : LockBufHdr(buf); PinBuffer_Locked(buf); valid = ((buf-flags BM_VALID) != 0); then we can avoid having the new buffer access

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2014-07-03 Thread Amit Kapila
On Wed, Jul 2, 2014 at 11:45 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Jeff Janes wrote: I would only envision using the parallel feature for vacuumdb after a pg_upgrade or some other major maintenance window (that is the only time I ever envision using vacuumdb at all). I don't

Re: [HACKERS] Autonomous Transaction (WIP)

2014-07-03 Thread Rajeev rastogi
On 01 July 2014 12:00, Amit Kapila Wrote: On Tue, Jul 1, 2014 at 11:46 AM, Rajeev rastogi rajeev.rast...@huawei.commailto:rajeev.rast...@huawei.com wrote: On 30 June 2014 22:50, Pavel Stehule Wrote: I didn't find a related message. ? I think there have been some confusion, the design idea

Re: [HACKERS] WAL replay bugs

2014-07-03 Thread Tom Lane
Michael Paquier michael.paqu...@gmail.com writes: On Wed, Jul 2, 2014 at 5:32 PM, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: Pehaps this is showing that no tidy or generalized way to tell what a page is used for. Many of the modules which have their own page format has a magic

Re: [HACKERS] WAL replay bugs

2014-07-03 Thread Michael Paquier
On Thu, Jul 3, 2014 at 3:38 PM, Tom Lane t...@sss.pgh.pa.us wrote: Michael Paquier michael.paqu...@gmail.com writes: On Wed, Jul 2, 2014 at 5:32 PM, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: Pehaps this is showing that no tidy or generalized way to tell what a page is used for.

Re: [HACKERS] Autonomous Transaction (WIP)

2014-07-03 Thread Amit Kapila
On Thu, Jul 3, 2014 at 12:03 PM, Rajeev rastogi rajeev.rast...@huawei.com wrote: On 01 July 2014 12:00, Amit Kapila Wrote: Simon has mentioned that exactly this idea has been rejected at PGCon 2 years back. Please refer that in below mail:

Re: [HACKERS] gaussian distribution pgbench

2014-07-03 Thread Fabien COELHO
Hello Gavin, decile percents: 69.9% 21.0% 6.3% 1.9% 0.6% 0.2% 0.1% 0.0% 0.0% 0.0% probability of fist/last percent of the range: 11.3% 0.0% I would suggest that probabilities should NEVER be expressed in percentages! As a percentage probability looks weird, and is never used for serious

[HACKERS] Setting PG-version without recompiling

2014-07-03 Thread Andreas Joseph Krogh
Hi.   I'm up for testing 9.4 but my JDBC-driver fails to connect due to PG's minor-version string: 4beta1. Is it possible to set this somewhere without recompiling PG?   Thanks.   -- Andreas Jospeh Krogh CTO / Partner - Visena AS Mobile: +47 909 56 963 andr...@visena.com

Re: [HACKERS] RETURNING PRIMARY KEY syntax extension

2014-07-03 Thread Rushabh Lathia
On Thu, Jul 3, 2014 at 6:56 AM, Ian Barwick i...@2ndquadrant.com wrote: On 01/07/14 21:00, Rushabh Lathia wrote: I spent some more time on the patch and here are my review comments. .) Patch gets applied through patch -p1 (git apply fails) .) trailing whitespace in the patch at various

Re: [HACKERS] Setting PG-version without recompiling

2014-07-03 Thread Abhijit Menon-Sen
At 2014-07-03 11:02:34 +0200, andr...@visena.com wrote: Is it possible to set this somewhere without recompiling PG? I'm afraid not. -- Abhijit -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Setting PG-version without recompiling

2014-07-03 Thread Andreas Joseph Krogh
På torsdag 03. juli 2014 kl. 11:13:44, skrev Abhijit Menon-Sen a...@2ndquadrant.com mailto:a...@2ndquadrant.com: At 2014-07-03 11:02:34 +0200, andr...@visena.com wrote: Is it possible to set this somewhere without recompiling PG? I'm afraid not.   Ok   -- Andreas Jospeh Krogh CTO /

Re: [HACKERS] gaussian distribution pgbench

2014-07-03 Thread Gavin Flower
On 03/07/14 20:58, Fabien COELHO wrote: Hello Gavin, decile percents: 69.9% 21.0% 6.3% 1.9% 0.6% 0.2% 0.1% 0.0% 0.0% 0.0% probability of fist/last percent of the range: 11.3% 0.0% I would suggest that probabilities should NEVER be expressed in percentages! As a percentage probability

Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD

2014-07-03 Thread Tatsuo Ishii
Hi, Hi, Attached you can find a short (compile tested only ) patch implementing a 'shared_memory_type' GUC, akin to 'dynamic_shared_memory_type'. Will only apply to 9.4, not 9.3, but it should be easy to convert for it. Greetings, Andres Freund I have rebased Andres's patch against

Re: [HACKERS] WAL replay bugs

2014-07-03 Thread Kyotaro HORIGUCHI
Hello, This is the additional comments for other part. I haven't see significant defect in the code so far. = bufcapt.c: - buffer_capture_remember() or so. Pages for unlogged tables are avoided to be written taking advantage that the lsn for such pages stays 0/0. I'd like to see a

Re: [HACKERS] [PATCH] introduce XLogLockBlockRangeForCleanup()

2014-07-03 Thread Abhijit Menon-Sen
FYI, I've attached a patch that does what you suggested. I haven't done anything else (i.e. testing) with it yet. -- Abhijit diff --git a/src/backend/access/nbtree/nbtxlog.c b/src/backend/access/nbtree/nbtxlog.c index 5f9fc49..dc90c02 100644 --- a/src/backend/access/nbtree/nbtxlog.c +++

Re: [HACKERS] [PATCH] introduce XLogLockBlockRangeForCleanup()

2014-07-03 Thread Simon Riggs
On 3 July 2014 06:45, Amit Khandekar amit.khande...@enterprisedb.com wrote: In GetBufferWithoutRelcache(), I was wondering, rather than calling PinBuffer(), if we do this : LockBufHdr(buf); PinBuffer_Locked(buf); valid = ((buf-flags BM_VALID) != 0); then we can avoid having the new buffer

Re: [HACKERS] gaussian distribution pgbench

2014-07-03 Thread Fujii Masao
On Wed, Jul 2, 2014 at 6:05 PM, Fabien COELHO coe...@cri.ensmp.fr wrote: Hello Mitsumasa-san, And I'm also interested in your decile percents output like under followings, decile percents: 39.6% 24.0% 14.6% 8.8% 5.4% 3.3% 2.0% 1.2% 0.7% 0.4% Sure, I'm really fine with that. I think

Re: [HACKERS] gaussian distribution pgbench

2014-07-03 Thread Andres Freund
On 2014-07-03 21:27:53 +0900, Fujii Masao wrote: Add drawing random integers with a Gaussian or truncated exponentional distributions to pgbench. Test variants with these distributions are also provided and triggered with options --gaussian=... and --exponential= IIRC we've not

Re: [HACKERS] Should extension--unpackaged-$ver.sql's \echo use FROM unpackaged;?

2014-07-03 Thread Fujii Masao
On Thu, Jul 3, 2014 at 4:26 AM, Andres Freund and...@2ndquadrant.com wrote: Hi, Fujii noticed that I'd used \echo Use CREATE EXTENSION pg_buffercache to load this file. \quit in an extension upgrade script. That's obviously wrong, because it should say ALTER EXTENSION. The reason it's that

Re: [HACKERS] docs: additional subsection for page-level locks in explicit-locking section

2014-07-03 Thread Fujii Masao
On Thu, Jul 3, 2014 at 4:34 AM, Michael Banck michael.ba...@credativ.de wrote: Hi, While reading through the Explicit Locking section of the manual today, I felt the last paragraph of section 13.3.2. (Row-level Locks) might merit its own subsection. It talks about page-level locks as

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-07-03 Thread Atri Sharma
On Tue, Jul 1, 2014 at 4:54 AM, Tomas Vondra t...@fuzzy.cz wrote: On 30.6.2014 23:12, Tomas Vondra wrote: Hi, attached is v5 of the patch. The main change is that scaling the number of buckets is done only once, after the initial hash table is build. The main advantage of this is lower

Re: [HACKERS] [v9.5] Custom Plan API

2014-07-03 Thread Shigeru Hanada
Kaigai-san, Sorry for lagged response. Here are my random thoughts about the patch. I couldn't understand the patch fully, because some of APIs are not used by ctidscan. If Custom Scan patch v2 review * Custom plan class comparison In backend/optimizer/util/pathnode.c, custclass is compared

Re: [HACKERS] Proposing pg_hibernate

2014-07-03 Thread Kevin Grittner
Amit Kapila amit.kapil...@gmail.com wrote: Overall I agree that following Robert's idea will increase the time to make database server up and reach a state where apps can connect and start operations, I agree that warming the cache before beginning to apply WAL would be best. but I think

Re: [HACKERS] Setting PG-version without recompiling

2014-07-03 Thread Tom Lane
Andreas Joseph Krogh andr...@visena.com writes: Hi.   I'm up for testing 9.4 but my JDBC-driver fails to connect due to PG's minor-version string: 4beta1. Is it possible to set this somewhere without recompiling PG? No, and even if you could, that would be the wrong approach. The right

Re: [HACKERS] RETURNING PRIMARY KEY syntax extension

2014-07-03 Thread Tom Lane
Rushabh Lathia rushabh.lat...@gmail.com writes: On Thu, Jul 3, 2014 at 6:56 AM, Ian Barwick i...@2ndquadrant.com wrote: A better solution would be to have an optional IF EXISTS clause: RETURNING PRIMARY KEY [ IF EXISTS ] Hmm liked the idea about adding [ IF EXISTS ]. Looking at the grammar so

Re: [HACKERS] docs: additional subsection for page-level locks in explicit-locking section

2014-07-03 Thread Kevin Grittner
Fujii Masao masao.fu...@gmail.com wrote: This seems to make sense. Barring objection, I will commit this only in HEAD. I'm inclined to think this is a slight improvement, just for the sake of consistency with peer level information.  You probably already noticed, but the patch as submitted

Re: [HACKERS] Setting PG-version without recompiling

2014-07-03 Thread Andreas Joseph Krogh
På torsdag 03. juli 2014 kl. 16:16:01, skrev Tom Lane t...@sss.pgh.pa.us mailto:t...@sss.pgh.pa.us: Andreas Joseph Krogh andr...@visena.com writes: Hi. �� I'm up for testing 9.4 but my JDBC-driver fails to connect due to PG's minor-version string: 4beta1. Is it possible to set this somewhere

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-07-03 Thread Tomas Vondra
On 1.7.2014 01:24, Tomas Vondra wrote: On 30.6.2014 23:12, Tomas Vondra wrote: Hi, Hopefully I got it right this time. At least it seems to be working for cases that failed before (no file leaks, proper rowcounts so far). Attached v7, fixing nbatch/ntuples in an assert. regards Tomas diff

Re: [HACKERS] tweaking NTUP_PER_BUCKET

2014-07-03 Thread Tomas Vondra
On 3.7.2014 02:13, Tomas Vondra wrote: Hi, while hacking on the 'dynamic nbucket' patch, scheduled for the next CF (https://commitfest.postgresql.org/action/patch_view?id=1494) I was repeatedly stumbling over NTUP_PER_BUCKET. I'd like to propose a change in how we handle it. TL;DR;

Re: [HACKERS] tweaking NTUP_PER_BUCKET

2014-07-03 Thread Stephen Frost
Tomas, * Tomas Vondra (t...@fuzzy.cz) wrote: However it's likely there are queries where this may not be the case, i.e. where rebuilding the hash table is not worth it. Let me know if you can construct such query (I wasn't). Thanks for working on this! I've been thinking on this for a while

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-07-03 Thread Tomas Vondra
On 3.7.2014 15:42, Atri Sharma wrote: On Tue, Jul 1, 2014 at 4:54 AM, Tomas Vondra t...@fuzzy.cz mailto:t...@fuzzy.cz wrote: On 30.6.2014 23:12, Tomas Vondra wrote: Hi, attached is v5 of the patch. The main change is that scaling the number of buckets is

Re: [HACKERS] tweaking NTUP_PER_BUCKET

2014-07-03 Thread Atri Sharma
On Thu, Jul 3, 2014 at 11:40 PM, Stephen Frost sfr...@snowman.net wrote: Tomas, * Tomas Vondra (t...@fuzzy.cz) wrote: However it's likely there are queries where this may not be the case, i.e. where rebuilding the hash table is not worth it. Let me know if you can construct such query (I

Re: [HACKERS] tweaking NTUP_PER_BUCKET

2014-07-03 Thread Tomas Vondra
Hi Stephen, On 3.7.2014 20:10, Stephen Frost wrote: Tomas, * Tomas Vondra (t...@fuzzy.cz) wrote: However it's likely there are queries where this may not be the case, i.e. where rebuilding the hash table is not worth it. Let me know if you can construct such query (I wasn't). Thanks for

Re: [HACKERS] tweaking NTUP_PER_BUCKET

2014-07-03 Thread Greg Stark
On Thu, Jul 3, 2014 at 11:40 AM, Atri Sharma atri.j...@gmail.com wrote: IIRC, last time when we tried doing bloom filters, I was short of some real world useful hash functions that we could use for building the bloom filter. Last time was we wanted to use bloom filters in hash joins to filter

Re: [HACKERS] RETURNING PRIMARY KEY syntax extension

2014-07-03 Thread Greg Stark
On Thu, Jul 3, 2014 at 7:37 AM, Tom Lane t...@sss.pgh.pa.us wrote: TBH, I thought that RETURNING PRIMARY KEY was already fairly iffy in that the application would have little idea what it was getting back. IF EXISTS would make it so spongy as to be useless, IMO. Seems easy enough to look at

Re: [HACKERS] Aggregate function API versus grouping sets

2014-07-03 Thread Andrew Gierth
Tom == Tom Lane t...@sss.pgh.pa.us writes: Tom If we're going to do that, I think it needs to be in 9.4. Are Tom you going to work up a patch? How's this? (applies and passes regression on 9.4 and master) -- Andrew (irc:RhodiumToad) diff --git a/src/backend/executor/nodeAgg.c

[HACKERS] Pg_upgrade and toast tables bug discovered

2014-07-03 Thread Bruce Momjian
There have been periodic reports of pg_upgrade errors related to toast tables. The most recent one was from May of this year: http://www.postgresql.org/message-id/flat/20140520202223.gb3...@momjian.us#20140520202223.gb3...@momjian.us There error was: Copying user relation

Re: [HACKERS] Aggregate function API versus grouping sets

2014-07-03 Thread Andrew Gierth
Here are two more, cumulative with the previous patch and each other: The first removes the assumption in ordered_set_startup that the aggcontext can be cached in fn_extra, and puts it in the transvalue instead. The second exposes the OSA* structures in a header file, so that user-defined

Re: [HACKERS] Array of composite types returned from python

2014-07-03 Thread Tom Lane
I wrote: Ronan Dunklau ronan.dunk...@dalibo.com writes: I don't know how to do that without implementing the cache itself. I don't either, but my thought was that we could hack up a simple one-element cache pretty trivially, eg static info and desc variables in PLyObject_ToComposite that are

Re: [HACKERS] RETURNING PRIMARY KEY syntax extension

2014-07-03 Thread Tom Lane
Greg Stark st...@mit.edu writes: On Thu, Jul 3, 2014 at 7:37 AM, Tom Lane t...@sss.pgh.pa.us wrote: TBH, I thought that RETURNING PRIMARY KEY was already fairly iffy in that the application would have little idea what it was getting back. IF EXISTS would make it so spongy as to be useless,

Re: [HACKERS] Pg_upgrade and toast tables bug discovered

2014-07-03 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: I have had trouble getting a table schema that is causing problems, but received a report via EDB support recently that had a simple schema (anonymized): ... needs_toast_table() computes the length of this table as 2024 bytes in 9.0, and 2064 bytes on

Re: [HACKERS] tweaking NTUP_PER_BUCKET

2014-07-03 Thread Tomas Vondra
On 3.7.2014 20:50, Tomas Vondra wrote: Hi Stephen, On 3.7.2014 20:10, Stephen Frost wrote: Tomas, * Tomas Vondra (t...@fuzzy.cz) wrote: However it's likely there are queries where this may not be the case, i.e. where rebuilding the hash table is not worth it. Let me know if you can

Re: [HACKERS] Pg_upgrade and toast tables bug discovered

2014-07-03 Thread Andres Freund
On 2014-07-03 17:09:41 -0400, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: I have had trouble getting a table schema that is causing problems, but received a report via EDB support recently that had a simple schema (anonymized): ... needs_toast_table() computes the length of

Re: [HACKERS] Aggregate function API versus grouping sets

2014-07-03 Thread Tom Lane
Andrew Gierth and...@tao11.riddles.org.uk writes: Tom == Tom Lane t...@sss.pgh.pa.us writes: Tom If we're going to do that, I think it needs to be in 9.4. Are Tom you going to work up a patch? How's this? (applies and passes regression on 9.4 and master) Pushed with minor cosmetic

Re: [HACKERS] Aggregate function API versus grouping sets

2014-07-03 Thread Andrew Gierth
Tom == Tom Lane t...@sss.pgh.pa.us writes: How's this? (applies and passes regression on 9.4 and master) Tom Pushed with minor cosmetic adjustments. Thanks! -- Andrew (irc:RhodiumToad) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Aggregate function API versus grouping sets

2014-07-03 Thread Tom Lane
Andrew Gierth and...@tao11.riddles.org.uk writes: Here are two more, cumulative with the previous patch and each other: The first removes the assumption in ordered_set_startup that the aggcontext can be cached in fn_extra, and puts it in the transvalue instead. OK, done. (ATM it seems like

Re: [HACKERS] RETURNING PRIMARY KEY syntax extension

2014-07-03 Thread Tom Dunstan
On 4 July 2014 00:07, Tom Lane t...@sss.pgh.pa.us wrote: TBH, I thought that RETURNING PRIMARY KEY was already fairly iffy in that the application would have little idea what it was getting back. IF EXISTS would make it so spongy as to be useless, IMO. IF EXISTS is pretty pointless - while

Re: [HACKERS] RETURNING PRIMARY KEY syntax extension

2014-07-03 Thread Tom Lane
Tom Dunstan pg...@tomd.cc writes: On 4 July 2014 00:07, Tom Lane t...@sss.pgh.pa.us wrote: It sounds to me like designing this for JDBC's getGeneratedKeys method is a mistake. There isn't going to be any way that the driver can support that without having looked at the table's metadata for

Re: [HACKERS] RETURNING PRIMARY KEY syntax extension

2014-07-03 Thread Tom Dunstan
On 4 July 2014 11:37, Tom Lane t...@sss.pgh.pa.us wrote: Sure it can - it append RETURNING PRIMARY KEY and hand back a ResultSet from whatever was returned. It's CURRENTLY doing that, but it's appending RETURNING * and leaving it up to the caller of getGeneratedKeys() to work out which

Re: [HACKERS] docs: additional subsection for page-level locks in explicit-locking section

2014-07-03 Thread Fujii Masao
On Fri, Jul 4, 2014 at 12:51 AM, Kevin Grittner kgri...@ymail.com wrote: Fujii Masao masao.fu...@gmail.com wrote: This seems to make sense. Barring objection, I will commit this only in HEAD. Committed. I'm inclined to think this is a slight improvement, just for the sake of consistency

Re: [HACKERS] Can simplify 'limit 1' with slow function?

2014-07-03 Thread gotoschool6g
slow query(8531 ms): SELECT ST_Distance_Sphere(shape,ST_GeomFromText('POINT(116.41386186784513 40.12211338311868)')) FROM road order by id LIMIT 1; explain output: Limit (cost=4653.48..4653.48 rows=1 width=3612) - Sort (cost=4653.48..4683.06 rows=11832 width=3612) Sort Key: id

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-07-03 Thread Fujii Masao
On Tue, Jul 1, 2014 at 10:11 PM, Fujii Masao masao.fu...@gmail.com wrote: On Mon, Jun 30, 2014 at 7:09 PM, furu...@pm.nttdata.co.jp wrote: Thanks for the review! +if (secs = 0) +secs = 1;/* Always sleep at least 1 sec */ + +sleeptime = secs *

Re: [HACKERS] Pg_upgrade and toast tables bug discovered

2014-07-03 Thread Bruce Momjian
On Thu, Jul 3, 2014 at 05:09:41PM -0400, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: I have had trouble getting a table schema that is causing problems, but received a report via EDB support recently that had a simple schema (anonymized): ... needs_toast_table() computes the

Re: [HACKERS] Pg_upgrade and toast tables bug discovered

2014-07-03 Thread Bruce Momjian
On Thu, Jul 3, 2014 at 11:55:40PM +0200, Andres Freund wrote: I don't think it's just that simple unfortunately. If pg_class entries get created that didn't exist on the old server there's a chance for oid conflicts. Consider SELECT

Re: [HACKERS] [v9.5] Custom Plan API

2014-07-03 Thread Kouhei Kaigai
Hanada-san, Thanks for your dedicated reviewing. It's a very long message. So, let me summarize the things I shall do in the next patch. * fix bug: custom-plan class comparison * fix up naming convention and syntax CREATE CUSTOM PLAN PROVIDER, rather than CREATE CUSTOM PLAN. Prefix shall be

Re: [HACKERS] RETURNING PRIMARY KEY syntax extension

2014-07-03 Thread Rushabh Lathia
On Fri, Jul 4, 2014 at 7:37 AM, Tom Lane t...@sss.pgh.pa.us wrote: Tom Dunstan pg...@tomd.cc writes: On 4 July 2014 00:07, Tom Lane t...@sss.pgh.pa.us wrote: It sounds to me like designing this for JDBC's getGeneratedKeys method is a mistake. There isn't going to be any way that the

[HACKERS] Issue while calling new PostgreSQL command from a Java Application

2014-07-03 Thread Ashoke
Hi, -- I have defined a new command my_command in PostgreSQL. This command takes the path of ANALYZE and inside analyze.c, I have a function to do some operations if its my_command.This command takes the input arguments: table name, column name and an input string.

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-07-03 Thread Fujii Masao
On Fri, Jul 4, 2014 at 4:58 AM, Rahila Syed rahilasye...@gmail.com wrote: Hello, Updated version of patches are attached. Changes are as follows 1. Improved readability of the code as per the review comments. 2. Addition of block_compression field in BkpBlock structure to store information