[HACKERS] Bloom Filter lookup for hash joins

2013-06-26 Thread Atri Sharma
Hi All, I have been researching bloom filters and discussed it on IRC with RhodiumToad and David Fetter, and they pointed me to the various places that could potentially have bloom filters, apart from the places that already have them currently. I have been reading the current implementation of

Re: [HACKERS] Add more regression tests for dbcommands

2013-06-26 Thread Robins Tharakan
Hi Andres, Attached is a patch which does not CREATE DATABASE, but now the regression tests do not test the following: - ALTER DATABASE RENAME TO is not allowed on a database in use. Had to remove two tests that were using this. - ALTER DATABASE SET TABLESPACE is also not allowed on a database

Re: [HACKERS] Review: UNNEST (and other functions) WITH ORDINALITY

2013-06-26 Thread Dean Rasheed
On 26 June 2013 01:22, Josh Berkus j...@agliodbs.com wrote: Folks, (the below was already discussed on IRC) Leaving names aside on this patch, I'm wondering about a piece of functionality I have with the current unnest() and with the unnest_ordinality()[1] extension: namely, the ability to

Re: [HACKERS] [PATCH] Fix conversion for Decimal arguments in plpython functions

2013-06-26 Thread Szymon Guz
On 26 June 2013 01:40, Steve Singer st...@ssinger.info wrote: On 06/25/2013 06:42 AM, Szymon Guz wrote: Hi, I've attached a new patch. I've fixed all the problems you've found, except for the efficiency problem, which has been described in previous email. thanks, Szymon This

Re: [HACKERS] [9.4 CF 1] The Commitfest Slacker List

2013-06-26 Thread Robins
Hi, Apologies for being unable to respond promptly. I've been traveling (without much access) and this was the fastest I could settle down. I was free for months and had to travel smack in the middle of the commitfest. Incidentally I had reviewed one patch after your direct email, but as someone

Re: [HACKERS] Add more regression tests for CREATE OPERATOR

2013-06-26 Thread Robins Tharakan
Hi Szymon, The commented out test that you're referring to, is an existing test (not that I added or commented). I was going to remove but interestingly its testing a part of code where (prima-facie) it should fail, but it passes (probably why it was disabled in the first place) ! So

Re: [HACKERS] Add more regression tests for CREATE OPERATOR

2013-06-26 Thread Szymon Guz
OK, so I think this patch can be committed, I will change the status. thanks, Szymon On 26 June 2013 09:26, Robins Tharakan thara...@gmail.com wrote: Hi Szymon, The commented out test that you're referring to, is an existing test (not that I added or commented). I was going to remove but

Re: [HACKERS] Review: Patch to compute Max LSN of Data Pages

2013-06-26 Thread Andres Freund
On 2013-06-26 08:50:27 +0530, Amit Kapila wrote: On Tuesday, June 25, 2013 11:12 PM Andres Freund wrote: On 2013-06-16 17:19:49 -0700, Josh Berkus wrote: Amit posted a new version of this patch on January 23rd. But last comment on it by Tom is not sure everyone wants this.

Re: [HACKERS] Add visibility map information to pg_freespace.

2013-06-26 Thread Kyotaro HORIGUCHI
Hello, I'm looking into this patch as a reviewer. I'd appreciate your time to review. I've had some suggestions so far, - I should be cautious in changing existing interface. You're right. It was somehow gone out of my mind. It might be better to provide a separate function from the

Re: [HACKERS] [PATCH] Fix conversion for Decimal arguments in plpython functions

2013-06-26 Thread Ronan Dunklau
The v2 patch does not work for me: regression tests for plpython fail on the plpython_types test: every numeric is converted to None. It seems the global decimal ctor is not initialized. Please find two patches, to be applied on top of the v2 patch: one initializes the decimal ctor, the other

Re: [HACKERS] Add visibility map information to pg_freespace.

2013-06-26 Thread Simon Riggs
On 26 June 2013 09:09, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jpwrote: - How about pageinspect? I proposed a simple representation format as a basis for discussion. Nevertheless, the VM pages has no more structure than a simple bit string. Given the VM info in

Re: FILTER for aggregates [was Re: [HACKERS] Department of Redundancy Department: makeNode(FuncCall) division]

2013-06-26 Thread Dean Rasheed
On 26 June 2013 01:01, Josh Berkus j...@agliodbs.com wrote: I know it's heresy in these parts, but maybe we should consider adopting a non-spec syntax for this feature? In particular, it's really un-obvious why the FILTER clause shouldn't be inside rather than outside the aggregate's parens,

Re: [HACKERS] [PATCH] Fix conversion for Decimal arguments in plpython functions

2013-06-26 Thread Szymon Guz
Thanks Steve, that's exactly what I wanted to send when you sent your patches :) I need to figure out why that patch v2 worked for me, I think I made mess somewhere in my git repo and didn't create the patch properly. Sorry for that. Patch is attached, I've also added information about cdecimal

Re: [HACKERS] Hash partitioning.

2013-06-26 Thread Yuri Levinsky
Tom, I clearly understand your point. I actually came from corporate market such as Verizon, Barclays... I remember very good that PostgreSQL is open source, but let's forget it for a moment. The key issue for corporate market always been a partitioning(vertical and lately horizontal). Because of

Re: [HACKERS] Improvement of checkpoint IO scheduler for stable transaction responses

2013-06-26 Thread KONDO Mitsumasa
Thank you for comments! On Tue, Jun 25, 2013 at 1:15 PM, Heikki Linnakangas Hmm, so the write patch doesn't do much, but the fsync patch makes the response times somewhat smoother. I'd suggest that we drop the write patch for now, and focus on the fsyncs. Write patch is effective in TPS! I

Re: [HACKERS] [PATCH] Fix conversion for Decimal arguments in plpython functions

2013-06-26 Thread Szymon Guz
You had a great idea, the time with cdecimal is really great, the difference on my machine is 64 ms vs 430 ms. Szymon

Re: FILTER for aggregates [was Re: [HACKERS] Department of Redundancy Department: makeNode(FuncCall) division]

2013-06-26 Thread Pavel Stehule
2013/6/26 Dean Rasheed dean.a.rash...@gmail.com: On 26 June 2013 01:01, Josh Berkus j...@agliodbs.com wrote: I know it's heresy in these parts, but maybe we should consider adopting a non-spec syntax for this feature? In particular, it's really un-obvious why the FILTER clause shouldn't be

Re: [HACKERS] Bloom Filter lookup for hash joins

2013-06-26 Thread Pavel Stehule
Hello it looks interesting - please, try to send some performance results, Regards Pavel 2013/6/26 Atri Sharma atri.j...@gmail.com: Hi All, I have been researching bloom filters and discussed it on IRC with RhodiumToad and David Fetter, and they pointed me to the various places that could

Re: [HACKERS] Kudos for Reviewers -- straw poll

2013-06-26 Thread Albe Laurenz
Dean Rasheed wrote: How should reviewers get credited in the release notes? a) not at all b) in a single block titled Reviewers for this version at the bottom. c) on the patch they reviewed, for each patch b) Unless they contribute enough to the patch to be considered a co-author.

Re: [HACKERS] Bloom Filter lookup for hash joins

2013-06-26 Thread Atri Sharma
On Wed, Jun 26, 2013 at 2:09 PM, Pavel Stehule pavel.steh...@gmail.com wrote: Hello it looks interesting - please, try to send some performance results, Regards Pavel Hi, Sure. I will think more about it and put up a design on the list soon. My current aim would be to work on hash joins.

Re: [HACKERS] Kudos for Reviewers -- straw poll

2013-06-26 Thread Atri Sharma
For me, B,B and another B works. Regards, Atri -- Regards, Atri l'apprenant -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] updated emacs configuration

2013-06-26 Thread Dimitri Fontaine
Peter Eisentraut pete...@gmx.net writes: emacs -q src/backend/commands/extension.c emacs -q -l ../emacs.samples src/backend/commands/extension.c Well, the first one uses 8-space tabs, the second 4-space tabs, so they look completely different. I'm not sure what you are trying to

Re: [HACKERS] FILTER for aggregates [was Re: Department of Redundancy Department: makeNode(FuncCall) division]

2013-06-26 Thread Andrew Gierth
Dean Rasheed said: To recap, the options currently on offer are: 1). Make FILTER a new partially reserved keyword, accepting that that might break some users' application code. 2). Make FILTER unreserved, accepting that that will lead to syntax errors rather than more specific error

Re: [HACKERS] updated emacs configuration

2013-06-26 Thread Jan Urbański
On 26/06/13 10:51, Dimitri Fontaine wrote: Peter Eisentraut pete...@gmx.net writes: $ git clone git://git.postgresql.org/git/postgresql.git Cloning into 'postgresql'... I can reproduce that here. I don't know why I have those postgres dirs then, and I'm pretty confused about my round of

Re: [HACKERS] [Review] Add SPI_gettypmod() to return a field's typemod from a TupleDesc

2013-06-26 Thread Jeevan Chalke
On Wed, Jun 26, 2013 at 7:49 AM, Mark Wong mark...@gmail.com wrote: On Tue, Jun 25, 2013 at 1:38 AM, Jeevan Chalke jeevan.cha...@enterprisedb.com wrote: Hi Mark, Is this the latest patch you are targeting for 9.4 CF1 ? I am going to review it. From the comment, here is one issue

Re: [HACKERS] [PATCH] Fix conversion for Decimal arguments in plpython functions

2013-06-26 Thread Ronan Dunklau
It seems like you confused me with steve :) The patch applies cleanly, and the regression tests pass on python2 when cdecimal is not installed. When it is, the type info returned for the converted numeric value is cdecimal.Decimal instead of decimal.Decimal. The regression tests expected output

Re: [HACKERS] Review: Patch to compute Max LSN of Data Pages

2013-06-26 Thread Amit Kapila
On Wednesday, June 26, 2013 1:20 PM Andres Freund wrote: On 2013-06-26 08:50:27 +0530, Amit Kapila wrote: On Tuesday, June 25, 2013 11:12 PM Andres Freund wrote: On 2013-06-16 17:19:49 -0700, Josh Berkus wrote: Amit posted a new version of this patch on January 23rd. But last

Re: [HACKERS] [PATCH] Fix conversion for Decimal arguments in plpython functions

2013-06-26 Thread Szymon Guz
On 26 June 2013 12:04, Ronan Dunklau rdunk...@gmail.com wrote: It seems like you confused me with steve :) Hi Ronan, Oh, yes. I'm sorry for that :) The patch applies cleanly, and the regression tests pass on python2 when cdecimal is not installed. When it is, the type info returned for the

Re: [HACKERS] Bloom Filter lookup for hash joins

2013-06-26 Thread Ants Aasma
On Wed, Jun 26, 2013 at 9:46 AM, Atri Sharma atri.j...@gmail.com wrote: I have been researching bloom filters and discussed it on IRC with RhodiumToad and David Fetter, and they pointed me to the various places that could potentially have bloom filters, apart from the places that already have

Re: [HACKERS] Review: Patch to compute Max LSN of Data Pages

2013-06-26 Thread Andres Freund
Hi Amit, On 2013-06-26 16:22:28 +0530, Amit Kapila wrote: On Wednesday, June 26, 2013 1:20 PM Andres Freund wrote: On 2013-06-26 08:50:27 +0530, Amit Kapila wrote: On Tuesday, June 25, 2013 11:12 PM Andres Freund wrote: On 2013-06-16 17:19:49 -0700, Josh Berkus wrote: Amit posted a

Re: [HACKERS] checking variadic any argument in parser - should be array

2013-06-26 Thread Jeevan Chalke
Hi Pavel On Sat, Jan 26, 2013 at 9:22 AM, Pavel Stehule pavel.steh...@gmail.comwrote: Hello Tom you did comment ! * Non-null argument had better be an array. The parser doesn't ! * enforce this for VARIADIC ANY functions (maybe it should?), so !

Re: [HACKERS] Improvement of checkpoint IO scheduler for stable transaction responses

2013-06-26 Thread Heikki Linnakangas
On 26.06.2013 11:37, KONDO Mitsumasa wrote: On Tue, Jun 25, 2013 at 1:15 PM, Heikki Linnakangas Hmm, so the write patch doesn't do much, but the fsync patch makes the response times somewhat smoother. I'd suggest that we drop the write patch for now, and focus on the fsyncs. Write patch is

Re: [HACKERS] [PATCH] add --progress option to pgbench (submission 3)

2013-06-26 Thread Fabien COELHO
Dear Matsumasa, Here is a v4 that takes into account most of your points: The report is performed for all threads by thread 0, however --progress is not supported under thread fork emulation if there are more than one thread. The report time does not slip anymore. However I've kept the

Re: [HACKERS] PQConnectPoll, connect(2), EWOULDBLOCK and somaxconn

2013-06-26 Thread Andres Freund
Hi, On 2013-06-17 16:16:22 +0200, Andres Freund wrote: When postgres on linux receives connection on a high rate client connections sometimes error out with: could not send data to server: Transport endpoint is not connected could not send startup packet: Transport endpoint is not connected

Re: [HACKERS] Hash partitioning.

2013-06-26 Thread Heikki Linnakangas
On 26.06.2013 11:17, Yuri Levinsky wrote: The main purpose of partitioning in my world is to store billions of rows and be able to search by date, hour or even minute as fast as possible. Hash partitioning sounds like a bad fit for that use case. A regular b-tree, possibly with range

Re: [HACKERS] Add more regression tests for dbcommands

2013-06-26 Thread Andres Freund
Hi, I am generally a bit unsure whether the regression tests you propose aren't a bit too verbose. Does any of the committers have an opinion about this? My feeling is that they are ok if they aren't slowing down things much. On 2013-06-26 01:55:53 -0500, Robins Tharakan wrote: The CREATE

Re: [HACKERS] proposal 9.4 plpgsql: allows access to call stack from GET DIAGNOSTICS statement

2013-06-26 Thread Rushabh Lathia
On Tue, Jun 25, 2013 at 11:09 AM, Pavel Stehule pavel.steh...@gmail.comwrote: 2013/6/25 Rushabh Lathia rushabh.lat...@gmail.com: On Tue, Jun 25, 2013 at 1:47 AM, Pavel Stehule pavel.steh...@gmail.com wrote: Hello This is fragment ofmy old code from orafce package - it is

Re: [HACKERS] proposal 9.4 plpgsql: allows access to call stack from GET DIAGNOSTICS statement

2013-06-26 Thread Pavel Stehule
2013/6/26 Rushabh Lathia rushabh.lat...@gmail.com: On Tue, Jun 25, 2013 at 11:09 AM, Pavel Stehule pavel.steh...@gmail.com wrote: 2013/6/25 Rushabh Lathia rushabh.lat...@gmail.com: On Tue, Jun 25, 2013 at 1:47 AM, Pavel Stehule pavel.steh...@gmail.com wrote: Hello This is

Re: [HACKERS] Review: Patch to compute Max LSN of Data Pages

2013-06-26 Thread Amit Kapila
On Wednesday, June 26, 2013 4:40 PM Andres Freund wrote: Hi Amit, On 2013-06-26 16:22:28 +0530, Amit Kapila wrote: On Wednesday, June 26, 2013 1:20 PM Andres Freund wrote: On 2013-06-26 08:50:27 +0530, Amit Kapila wrote: On Tuesday, June 25, 2013 11:12 PM Andres Freund wrote: On

Re: [HACKERS] Bloom Filter lookup for hash joins

2013-06-26 Thread Atri Sharma
One idea that I had was to use them to do CLOG lookups from smaller datastructures. You could store the list of aborted xids in bloom filters. When a xid is not found in the filter, then it is known to have committed, if the filter returns true, then you have to check the real CLOG. This

Re: [HACKERS] Move unused buffers to freelist

2013-06-26 Thread Amit Kapila
On Tuesday, June 25, 2013 10:25 AM Amit Kapila wrote: On Monday, June 24, 2013 11:00 PM Robert Haas wrote: On Thu, Jun 6, 2013 at 3:01 AM, Amit Kapila amit.kap...@huawei.com wrote: To avoid above 3 factors in test readings, I used below steps: 1. Initialize the database with scale

Re: [HACKERS] Bloom Filter lookup for hash joins

2013-06-26 Thread Simon Riggs
On 26 June 2013 07:46, Atri Sharma atri.j...@gmail.com wrote: I have been researching bloom filters and discussed it on IRC with RhodiumToad and David Fetter, and they pointed me to the various places that could potentially have bloom filters, apart from the places that already have them

Re: [HACKERS] Bloom Filter lookup for hash joins

2013-06-26 Thread Atri Sharma
On Wed, Jun 26, 2013 at 5:47 PM, Simon Riggs si...@2ndquadrant.com wrote: On 26 June 2013 07:46, Atri Sharma atri.j...@gmail.com wrote: I have been researching bloom filters and discussed it on IRC with RhodiumToad and David Fetter, and they pointed me to the various places that could

Re: [HACKERS] proposal 9.4 plpgsql: allows access to call stack from GET DIAGNOSTICS statement

2013-06-26 Thread Rushabh Lathia
On Wed, Jun 26, 2013 at 5:11 PM, Pavel Stehule pavel.steh...@gmail.comwrote: 2013/6/26 Rushabh Lathia rushabh.lat...@gmail.com: On Tue, Jun 25, 2013 at 11:09 AM, Pavel Stehule pavel.steh...@gmail.com wrote: 2013/6/25 Rushabh Lathia rushabh.lat...@gmail.com: On Tue,

Re: [HACKERS] A better way than tweaking NTUP_PER_BUCKET

2013-06-26 Thread Atri Sharma
On Sun, Jun 23, 2013 at 8:41 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 23.06.2013 01:48, Simon Riggs wrote: On 22 June 2013 21:40, Stephen Frostsfr...@snowman.net wrote: I'm actually not a huge fan of this as it's certainly not cheap to do. If it can be shown to be better

Re: [HACKERS] Hash partitioning.

2013-06-26 Thread Bruce Momjian
On Tue, Jun 25, 2013 at 02:52:33PM -0700, Kevin Grittner wrote: Claudio Freire klaussfre...@gmail.com wrote: Did you try select * from foo where (a % 16) = (1::int % 16)? I did.  Using Robert's hashed partitioning table definitions: test=# explain select * from foo where a = 1 and (a %

Re: [HACKERS] Kudos for Reviewers -- straw poll

2013-06-26 Thread Bruce Momjian
On Wed, Jun 26, 2013 at 10:40:17AM +1000, Brendan Jurd wrote: On 26 June 2013 03:17, Josh Berkus j...@agliodbs.com wrote: How should reviewers get credited in the release notes? a) not at all b) in a single block titled Reviewers for this version at the bottom. c) on the patch they

Re: [HACKERS] Kudos for Reviewers -- straw poll

2013-06-26 Thread Andrew Dunstan
On 06/26/2013 09:14 AM, Bruce Momjian wrote: On Wed, Jun 26, 2013 at 10:40:17AM +1000, Brendan Jurd wrote: On 26 June 2013 03:17, Josh Berkus j...@agliodbs.com wrote: How should reviewers get credited in the release notes? a) not at all b) in a single block titled Reviewers for this version

Re: [HACKERS] Optimizing pglz compressor

2013-06-26 Thread Amit Kapila
On Wednesday, June 26, 2013 2:15 AM Heikki Linnakangas wrote: On 19.06.2013 14:01, Amit Kapila wrote: Observations -- 1. For small data perforamce is always good with patch. 2. For random small/large data performace is good. 3. For medium and large text and same byte

Re: [HACKERS] Hash partitioning.

2013-06-26 Thread Yuri Levinsky
Heikki, As far as I understand the height of the btree will affect the number of I/Os necessary. The height of the tree does not increase linearly with the number of records. May be I wrong in terminology but when I am trying to insert data into empty table the insertion time is increasing when

Re: [HACKERS] updated emacs configuration

2013-06-26 Thread Noah Misch
On Tue, Jun 25, 2013 at 11:17:47PM -0400, Peter Eisentraut wrote: On Sun, 2013-06-23 at 16:03 -0400, Noah Misch wrote: ((c-mode . ((c-basic-offset . 4) (fill-column . 79) I don't know whether you'd consider it to fall within the scope of this update, but 78 is the

Re: [HACKERS] A better way than tweaking NTUP_PER_BUCKET

2013-06-26 Thread Stephen Frost
Atri, * Atri Sharma (atri.j...@gmail.com) wrote: I just popped in here on Simon's advice to put an idea I had about optimizing hash joins on this thread. I'd encourage reading the thread a bit first, in the future.. :) Essentially, I was thinking of using bloom filters in the part where we

Re: [HACKERS] Hash partitioning.

2013-06-26 Thread Markus Wanner
On 06/25/2013 11:52 PM, Kevin Grittner wrote: At least until we have parallel query execution. At *that* point this all changes. Can you elaborate on that, please? I currently have a hard time imagining how partitions can help performance in that case, either. At least compared to modern RAID

Re: [HACKERS] Hash partitioning.

2013-06-26 Thread k...@rice.edu
On Wed, Jun 26, 2013 at 03:47:43PM +0200, Markus Wanner wrote: On 06/25/2013 11:52 PM, Kevin Grittner wrote: At least until we have parallel query execution. At *that* point this all changes. Can you elaborate on that, please? I currently have a hard time imagining how partitions can

Re: [HACKERS] patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap [Review]

2013-06-26 Thread Jamie Martin
FYI I submitted a slightly modified patch since Amit's measurements that is slightly faster. On Jun 25, 2013, at 1:26 PM, Amit Kapila amit.kap...@huawei.com wrote: On Monday, June 24, 2013 8:20 PM Tom Lane wrote: Amit Kapila amit.kap...@huawei.com writes: I will summarize the results, and

Re: [HACKERS] Hash partitioning.

2013-06-26 Thread Heikki Linnakangas
On 26.06.2013 16:41, Yuri Levinsky wrote: Heikki, As far as I understand the height of the btree will affect the number of I/Os necessary. The height of the tree does not increase linearly with the number of records. The height of a b-tree is O(log n), where n is the number of records.

Re: [HACKERS] Hash partitioning.

2013-06-26 Thread Yuri Levinsky
Markus, It's no relation between partitions and raids despite they both distribute data somehow. By the end of the day when you use the raid you have one single device with some performance limitations. When you want to improve your data access after that and not to work with huge indexes that you

Re: [HACKERS] Hash partitioning.

2013-06-26 Thread Bruce Momjian
On Wed, Jun 26, 2013 at 05:10:00PM +0300, Heikki Linnakangas wrote: In practice, there might be a lot of quirks and inefficiencies and locking contention etc. involved in various DBMS's, that you might be able to work around with hash partitioning. But from a theoretical point of view, there

Re: [HACKERS] Hash partitioning.

2013-06-26 Thread Yuri Levinsky
Heiki, This is most professional explanation that I ever seen. Let me please disagree with a bottom line. It's heavily depends on amount of memory and actual index sizes. I did a benchmark ~6 years ago and I won a glass of beer. Anyway I am talking about hash partitioning as a feature and my

[HACKERS] Developer meeting photos

2013-06-26 Thread Alexander Korotkov
Hackers, at last developer meeting we missed Oleg Bartunov. So, it's not surprising that photos is also missed. I remember that somebody took photos, but unfortunately it appears that I don't remember who. My employer who sponsored my attendance in PGCon want to publish post about it on the

Re: [HACKERS] Hash partitioning.

2013-06-26 Thread Tom Lane
Heikki Linnakangas hlinnakan...@vmware.com writes: On 26.06.2013 11:17, Yuri Levinsky wrote: When you dealing with company, which has ~350.000.000 users, and you don't want to use key/value data stores: you need hash partitioned tables and hash partitioned table clusters to perform fast

Re: [HACKERS] Bloom Filter lookup for hash joins

2013-06-26 Thread Tom Lane
Ants Aasma a...@cybertec.at writes: On Wed, Jun 26, 2013 at 9:46 AM, Atri Sharma atri.j...@gmail.com wrote: I have been reading the current implementation of hash joins, and in ExecScanHashBucket, which I understand is the actual lookup function, we could potentially look at a bloom filter per

Re: [HACKERS] A better way than tweaking NTUP_PER_BUCKET

2013-06-26 Thread Atri Sharma
On Wed, Jun 26, 2013 at 7:12 PM, Stephen Frost sfr...@snowman.net wrote: Atri, * Atri Sharma (atri.j...@gmail.com) wrote: I just popped in here on Simon's advice to put an idea I had about optimizing hash joins on this thread. I'd encourage reading the thread a bit first, in the future.. :)

Re: [HACKERS] LATERAL quals revisited

2013-06-26 Thread Antonin Houska
On 06/26/2013 12:52 AM, Tom Lane wrote: Instead of setting it aside, can we (mis)use placeholder var (PHV), to ensure that the WHERE clause is evaluated below the OJ; instead of combining it with the ON clause? No, that doesn't help; it has to be part of the joinquals at the join node, or you

Re: [HACKERS] Hash partitioning.

2013-06-26 Thread Markus Wanner
On 06/26/2013 04:10 PM, Yuri Levinsky wrote: You typically don't want to use b-tree index when yo select more when ~1-2% of your data. Agreed. Indices on columns with very low selectivity don't perform well. (Postgres knows that and uses a sequential scan based on selectivity estimates. Being

Re: [HACKERS] LATERAL quals revisited

2013-06-26 Thread Tom Lane
Antonin Houska antonin.hou...@gmail.com writes: If the WHERE clause moves up, then the resulting plan might be less efficient than the one we'd get if the subquery hadn't been pulled-up at all. No, because we can push the qual back down again (using a parameterized path) if that's

Re: [HACKERS] Bugfix and new feature for PGXS

2013-06-26 Thread Andrew Dunstan
On 06/25/2013 11:29 AM, Cédric Villemain wrote: Le mardi 25 juin 2013 17:18:51, Andrew Dunstan a écrit : On 06/24/2013 07:24 PM, Cédric Villemain wrote: Le mardi 25 juin 2013 00:18:26, Andrew Dunstan a écrit : On 06/24/2013 04:02 PM, Cédric Villemain wrote: WIth extension, we do have to set

Re: [HACKERS] Computer VARSIZE_ANY(PTR) during debugging

2013-06-26 Thread Alvaro Herrera
Amit Langote escribió: The segfault in question happens at line 1141: off = att_align_pointer(off, thisatt-attalign, -1, tp + off); char *tp; /* ptr to tuple data */ longoff;/* offset in tuple data */ Disassembling seems to

Re: [HACKERS] Hash partitioning.

2013-06-26 Thread Markus Wanner
On 06/26/2013 04:01 PM, k...@rice.edu wrote: I think he is referring to the fact that with parallel query execution, multiple partitions can be processed simultaneously instead of serially as they are now with the resulting speed increase. Processing simultaneously is the purpose of parallel

Re: [HACKERS] Hash partitioning.

2013-06-26 Thread Bruce Momjian
On Wed, Jun 26, 2013 at 05:04:11PM +0200, Markus Wanner wrote: On 06/26/2013 04:01 PM, k...@rice.edu wrote: I think he is referring to the fact that with parallel query execution, multiple partitions can be processed simultaneously instead of serially as they are now with the resulting

Re: [HACKERS] Kudos for Reviewers -- straw poll

2013-06-26 Thread Markus Wanner
On 06/25/2013 08:26 PM, Andres Freund wrote: It's not about the reviewers being less. It's a comparison of effort. The effort for a casual review simply isn't comparable with the effort spent on developing a nontrivial patch. Remember: Debugging is twice as hard as writing the code in the

Re: [HACKERS] BUG #7493: Postmaster messages unreadable in a Windows console

2013-06-26 Thread Noah Misch
On Mon, Jun 24, 2013 at 04:00:00PM +0400, Alexander Law wrote: Thanks for your work, your patch is definitely better. I agree that this approach much more generic. Committed. -- Noah Misch EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers

Re: [HACKERS] Hash partitioning.

2013-06-26 Thread Kevin Grittner
Markus Wanner mar...@bluegap.ch wrote: On 06/25/2013 11:52 PM, Kevin Grittner wrote: At least until we have parallel query execution.  At *that* point this all changes. Can you elaborate on that, please? I currently have a hard time imagining how partitions can help performance in that case,

Re: [HACKERS] Hash partitioning.

2013-06-26 Thread Heikki Linnakangas
On 26.06.2013 18:34, Kevin Grittner wrote: Markus Wannermar...@bluegap.ch wrote: On 06/25/2013 11:52 PM, Kevin Grittner wrote: At least until we have parallel query execution. At *that* point this all changes. Can you elaborate on that, please? I currently have a hard time imagining how

Re: [HACKERS] A better way than tweaking NTUP_PER_BUCKET

2013-06-26 Thread Stephen Frost
* Atri Sharma (atri.j...@gmail.com) wrote: My point is that I would like to help in the implementation, if possible. :) Feel free to go ahead and implement it.. I'm not sure when I'll have a chance to (probably not in the next week or two anyway). Unfortunately, the bigger issue here is really

Re: [HACKERS] XLogInsert scaling, revisited

2013-06-26 Thread Heikki Linnakangas
On 24.06.2013 21:01, Andres Freund wrote: Ok, I started to look at this: Thanks! * Could you document the way slots prevent checkpoints from occurring when XLogInsert rechecks for full page writes? I think it's correct - but not very obvious on a glance. There's this in the comment

Re: [HACKERS] Hash partitioning.

2013-06-26 Thread Markus Wanner
On 06/26/2013 05:46 PM, Heikki Linnakangas wrote: We could also allow a large query to search a single table in parallel. A seqscan would be easy to divide into N equally-sized parts that can be scanned in parallel. It's more difficult for index scans, but even then it might be possible at

Re: [HACKERS] A better way than tweaking NTUP_PER_BUCKET

2013-06-26 Thread Atri Sharma
On Wed, Jun 26, 2013 at 9:20 PM, Stephen Frost sfr...@snowman.net wrote: * Atri Sharma (atri.j...@gmail.com) wrote: My point is that I would like to help in the implementation, if possible. :) Feel free to go ahead and implement it.. I'm not sure when I'll have a chance to (probably not in

Re: [HACKERS] PQConnectPoll, connect(2), EWOULDBLOCK and somaxconn

2013-06-26 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2013-06-17 16:16:22 +0200, Andres Freund wrote: Not accepting EWOULDBLOCK in the above if() results in: could not connect to server: Resource temporarily unavailable Is the server running locally and accepting connections on Unix domain socket

Re: [HACKERS] Computer VARSIZE_ANY(PTR) during debugging

2013-06-26 Thread Amit Langote
On Thu, Jun 27, 2013 at 12:02 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Amit Langote escribió: The segfault in question happens at line 1141: off = att_align_pointer(off, thisatt-attalign, -1, tp + off); char *tp; /* ptr to tuple data */ long

Re: [HACKERS] Kudos for Reviewers -- straw poll

2013-06-26 Thread Dimitri Fontaine
Josh Berkus j...@agliodbs.com writes: Well, one of the other prizes which occurred to me today would be a pgCon lottery. That is, each review posted by a non-committer would go in a hat, and in February we would draw one who would get a free registration and airfare to pgCon. +1, I like that

Re: [HACKERS] Add more regression tests for dbcommands

2013-06-26 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: I am generally a bit unsure whether the regression tests you propose aren't a bit too verbose. Does any of the committers have an opinion about this? My feeling is that they are ok if they aren't slowing down things much. Yeah, I'm concerned about

Re: [HACKERS] fixing pg_ctl with relative paths

2013-06-26 Thread Fujii Masao
On Wed, Jun 26, 2013 at 2:36 PM, Hari Babu haribabu.ko...@huawei.com wrote: On June 26, 2013 5:02 AM Josh Kupershmidt wrote: Thanks for the feedback. Attached is a rebased version of the patch with the two small issues noted fixed. The following description in the document of pg_ctl needs to be

Re: [HACKERS] Review: Patch to compute Max LSN of Data Pages

2013-06-26 Thread Fujii Masao
On Wed, Jun 26, 2013 at 8:57 PM, Amit Kapila amit.kap...@huawei.com wrote: On Wednesday, June 26, 2013 4:40 PM Andres Freund wrote: Hi Amit, On 2013-06-26 16:22:28 +0530, Amit Kapila wrote: On Wednesday, June 26, 2013 1:20 PM Andres Freund wrote: On 2013-06-26 08:50:27 +0530, Amit Kapila

Re: [HACKERS] A better way than tweaking NTUP_PER_BUCKET

2013-06-26 Thread Stephen Frost
* Atri Sharma (atri.j...@gmail.com) wrote: Right, let me look.Although, I am pretty busy atm with ordered set functions, so will get it done maybe last week of this month. Isn't it currently the last week of this month? :) I'm guessing you mean July. Another thing I believe in is that we

Re: [HACKERS] A better way than tweaking NTUP_PER_BUCKET

2013-06-26 Thread Atri Sharma
Isn't it currently the last week of this month? :) I'm guessing you mean July. Heh,no.I lose track of time these days. Alright, second week of July then. I really don't see that happening, to be honest.. I think it would be interesting to try some of the surrogate-additional-hashing that

Re: [HACKERS] proposal 9.4 plpgsql: allows access to call stack from GET DIAGNOSTICS statement

2013-06-26 Thread Pavel Stehule
Hello updated patch with some basic doc Regards Pavel 2013/6/26 Rushabh Lathia rushabh.lat...@gmail.com: On Wed, Jun 26, 2013 at 5:11 PM, Pavel Stehule pavel.steh...@gmail.com wrote: 2013/6/26 Rushabh Lathia rushabh.lat...@gmail.com: On Tue, Jun 25, 2013 at 11:09 AM, Pavel

Re: [HACKERS] Kudos for Reviewers -- straw poll

2013-06-26 Thread Claudio Freire
On Wed, Jun 26, 2013 at 10:25 AM, Andrew Dunstan and...@dunslane.net wrote: On 06/26/2013 09:14 AM, Bruce Momjian wrote: On Wed, Jun 26, 2013 at 10:40:17AM +1000, Brendan Jurd wrote: On 26 June 2013 03:17, Josh Berkus j...@agliodbs.com wrote: How should reviewers get credited in the

Re: [HACKERS] PQConnectPoll, connect(2), EWOULDBLOCK and somaxconn

2013-06-26 Thread Andres Freund
On 2013-06-26 12:07:54 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2013-06-17 16:16:22 +0200, Andres Freund wrote: Not accepting EWOULDBLOCK in the above if() results in: could not connect to server: Resource temporarily unavailable Is the server running locally

Re: [HACKERS] Kudos for Reviewers -- straw poll

2013-06-26 Thread Rodrigo Gonzalez
On Wed, 26 Jun 2013 09:14:07 -0400 Bruce Momjian br...@momjian.us wrote: On Wed, Jun 26, 2013 at 10:40:17AM +1000, Brendan Jurd wrote: On 26 June 2013 03:17, Josh Berkus j...@agliodbs.com wrote: How should reviewers get credited in the release notes? a) not at all b) in a single

Re: [HACKERS] Kudos for Reviewers -- straw poll

2013-06-26 Thread Bruce Momjian
On Wed, Jun 26, 2013 at 03:12:00PM -0300, Rodrigo Gonzalez wrote: On Wed, 26 Jun 2013 09:14:07 -0400 Bruce Momjian br...@momjian.us wrote: On Wed, Jun 26, 2013 at 10:40:17AM +1000, Brendan Jurd wrote: On 26 June 2013 03:17, Josh Berkus j...@agliodbs.com wrote: How should reviewers get

Re: [HACKERS] Hash partitioning.

2013-06-26 Thread Claudio Freire
On Wed, Jun 26, 2013 at 11:14 AM, Bruce Momjian br...@momjian.us wrote: On Wed, Jun 26, 2013 at 05:10:00PM +0300, Heikki Linnakangas wrote: In practice, there might be a lot of quirks and inefficiencies and locking contention etc. involved in various DBMS's, that you might be able to work

Re: [HACKERS] Kudos for Reviewers -- straw poll

2013-06-26 Thread Rodrigo Gonzalez
On Wed, 26 Jun 2013 14:13:32 -0400 Bruce Momjian br...@momjian.us wrote: On Wed, Jun 26, 2013 at 03:12:00PM -0300, Rodrigo Gonzalez wrote: On Wed, 26 Jun 2013 09:14:07 -0400 Bruce Momjian br...@momjian.us wrote: On Wed, Jun 26, 2013 at 10:40:17AM +1000, Brendan Jurd wrote: On 26

Re: [HACKERS] Bloom Filter lookup for hash joins

2013-06-26 Thread Jeff Janes
On Wed, Jun 26, 2013 at 5:01 AM, Atri Sharma atri.j...@gmail.com wrote: The problem here is that if the hash table is in memory, doing a hash table lookup directly is likely to be cheaper than a bloom filter lookup, even if the bloom filter fits into the processor cache and the hash

Re: [HACKERS] MD5 aggregate

2013-06-26 Thread Noah Misch
On Mon, Jun 17, 2013 at 11:34:52AM +0100, Dean Rasheed wrote: I've been playing around with the idea of an aggregate that computes the sum of the md5 hashes of each of its inputs, which I've called md5_total() for now, although I'm not particularly wedded to that name. Comparing it with

Re: [HACKERS] Kudos for Reviewers -- straw poll

2013-06-26 Thread Bruce Momjian
On Wed, Jun 26, 2013 at 03:22:06PM -0300, Rodrigo Gonzalez wrote: On Wed, 26 Jun 2013 14:13:32 -0400 Production of the release notes was not the problem; it was the text in the release notes. I don't see how we could modify the release note format. Well... Checking release notes

Re: [HACKERS] Bloom Filter lookup for hash joins

2013-06-26 Thread Atri Sharma
On Thu, Jun 27, 2013 at 12:01 AM, Jeff Janes jeff.ja...@gmail.com wrote: I don't think that sounds all that promising. When the hash table does not fit in memory, it is either partitioned into multiple passes, each of which do fit in memory, or it chooses a different plan altogether. Yeah,

Re: [HACKERS] Add more regression tests for CREATE OPERATOR

2013-06-26 Thread Josh Berkus
On 06/26/2013 12:29 AM, Szymon Guz wrote: OK, so I think this patch can be committed, I will change the status. Can we have a full review before you mark it ready for committer? How did you test it? What kinds of review have you done? The committer can't know whether it's ready or not if he

Re: [HACKERS] Add more regression tests for CREATE OPERATOR

2013-06-26 Thread Szymon Guz
On 26 June 2013 20:55, Josh Berkus j...@agliodbs.com wrote: On 06/26/2013 12:29 AM, Szymon Guz wrote: OK, so I think this patch can be committed, I will change the status. Can we have a full review before you mark it ready for committer? How did you test it? What kinds of review have you

Re: [HACKERS] Kudos for Reviewers -- straw poll

2013-06-26 Thread Alvaro Herrera
Bruce Momjian escribió: On Wed, Jun 26, 2013 at 03:22:06PM -0300, Rodrigo Gonzalez wrote: Checking release notes for 9.2.4 you have Fix insecure parsing of server command-line switches (Mitsumasa Kondo, Kyotaro Horiguchi) What about (it people think that it is good) a second ()

  1   2   >