Re: [HACKERS] gaussian distribution pgbench

2014-07-22 Thread Fabien COELHO
Please find attached 2 patches, which are a split of the patch discussed in this thread. Please find attached a very minor improvement to apply a code (variable name) simplification directly in patch A so as to avoid a change in patch B. The cumulated patch is the same as previous. (A)

Re: [HACKERS] [bug fix] Suppress autovacuum: found orphan temp table message

2014-07-22 Thread Andres Freund
On 2014-07-22 10:09:04 +0900, MauMau wrote: From: Andres Freund and...@2ndquadrant.com On 2014-07-18 23:38:09 +0900, MauMau wrote: So, I propose a simple fix to change the LOG level to DEBUG1. I don't know which of DEBUG1-DEBUG5 is appropriate, and any level is OK. Could you include this

Re: [HACKERS] plpgsql.extra_warnings='num_into_expressions'

2014-07-22 Thread Marko Tiikkaja
On 7/22/14, 7:06 AM, Pavel Stehule wrote: I looked on this patch and I am thinking so it is not a good idea. It introduce early dependency between functions and pg_class based objects. What dependency? The patch only looks at the raw parser output, so it won't e.g. know whether SELECT *

Re: [HACKERS] Index-only scans for multicolumn GIST

2014-07-22 Thread Heikki Linnakangas
On 07/21/2014 10:47 PM, Anastasia Lubennikova wrote: Hi, hackers! There are new results of my work on GSoC project Index-only scans for GIST. Previous post is here: http://postgresql.1045698.n5.nabble.com/Index-only-scans-for-GIST-td5804892.html Repository is

Re: [HACKERS] plpgsql.extra_warnings='num_into_expressions'

2014-07-22 Thread Pavel Stehule
2014-07-22 8:52 GMT+02:00 Marko Tiikkaja ma...@joh.to: On 7/22/14, 7:06 AM, Pavel Stehule wrote: I looked on this patch and I am thinking so it is not a good idea. It introduce early dependency between functions and pg_class based objects. What dependency? The patch only looks at the raw

Re: [HACKERS] Production block comparison facility

2014-07-22 Thread Michael Paquier
On Sun, Jul 20, 2014 at 5:31 PM, Simon Riggs si...@2ndquadrant.com wrote: The block comparison facility presented earlier by Heikki would not be able to be used in production systems. ISTM that it would be desirable to have something that could be used in that way. ISTM easy to make these

Re: [HACKERS] [bug fix] Suppress autovacuum: found orphan temp table message

2014-07-22 Thread MauMau
From: Andres Freund and...@2ndquadrant.com On 2014-07-22 10:09:04 +0900, MauMau wrote: Is there any problem if we don't output the message? Yes. The user won't know that possibly gigabytes worth of diskspace aren't freed. RemovePgTempFiles() frees the disk space by removing temp relation

[HACKERS] parametric block size?

2014-07-22 Thread Fabien
Hello devs, The default blocksize is currently 8k, which is not necessary optimal for all setup, especially with SSDs where the latency is much lower than HDD. There is a case for different values with significant impact on performance (up to a not-to-be-sneezed-at 10% on a pgbench run on

Re: [HACKERS] Use unique index for longer pathkeys.

2014-07-22 Thread Kyotaro HORIGUCHI
Sorry , previous version has bugs. It stamps over the stack and crashesX( The attached is the bug fixed version, with no substantial changes. On Tue, Jul 15, 2014 at 2:17 PM, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: Hi, the attached is the revised version. Thanks

Re: [HACKERS] [bug fix] Suppress autovacuum: found orphan temp table message

2014-07-22 Thread Andres Freund
On 2014-07-22 17:05:22 +0900, MauMau wrote: From: Andres Freund and...@2ndquadrant.com On 2014-07-22 10:09:04 +0900, MauMau wrote: Is there any problem if we don't output the message? Yes. The user won't know that possibly gigabytes worth of diskspace aren't freed. RemovePgTempFiles()

Re: [HACKERS] WAL replay bugs

2014-07-22 Thread Kyotaro HORIGUCHI
Hello, Although I doubt necessity of the flexibility seeing the current testing framework, I don't have so strong objection about that. Nevertheless, perhaps you are appreciated to put a notice on.. README or somewhere. Hm, well... Fine, I added it in this updated series. Thank you for

Re: [HACKERS] BUFFER_LOCK_EXCLUSIVE is used in ginbuildempty().

2014-07-22 Thread Kyotaro HORIGUCHI
Hello, At Thu, 17 Jul 2014 15:54:31 -0400, Tom Lane t...@sss.pgh.pa.us wrote in 10710.1405626...@sss.pgh.pa.us Peter Geoghegan p...@heroku.com writes: On Thu, Jul 17, 2014 at 7:47 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: I don't understand the point of having these GIN_EXCLUSIVE

Re: [HACKERS] [bug fix] Suppress autovacuum: found orphan temp table message

2014-07-22 Thread MauMau
From: Andres Freund and...@2ndquadrant.com On 2014-07-22 17:05:22 +0900, MauMau wrote: RemovePgTempFiles() frees the disk space by removing temp relation files at server start. But it's not called during a crash restart. Yes, the comment of the function says: * NOTE: we could, but don't,

Re: [HACKERS] how to reach D5 in tuplesort.c 's polyphase merge algorithm?

2014-07-22 Thread 土卜皿
hi, I got the same result after work_mem = 64, but I can get to D5 and D6 after using bigger data sample (at least 10 records) as Tom said! 2014-07-19 6:35 GMT+08:00 土卜皿 pengcz.n...@gmail.com: 2014-07-19 6:26 GMT+08:00 Tom Lane t...@sss.pgh.pa.us: =?UTF-8?B?5Zyf5Y2c55q/?=

Re: [HACKERS] [bug fix] Suppress autovacuum: found orphan temp table message

2014-07-22 Thread Andres Freund
On 2014-07-22 19:13:56 +0900, MauMau wrote: From: Andres Freund and...@2ndquadrant.com On 2014-07-22 17:05:22 +0900, MauMau wrote: RemovePgTempFiles() frees the disk space by removing temp relation files at server start. But it's not called during a crash restart. Yes, the comment of

Re: [HACKERS] Production block comparison facility

2014-07-22 Thread Simon Riggs
On 22 July 2014 08:49, Michael Paquier michael.paqu...@gmail.com wrote: On Sun, Jul 20, 2014 at 5:31 PM, Simon Riggs si...@2ndquadrant.com wrote: The block comparison facility presented earlier by Heikki would not be able to be used in production systems. ISTM that it would be desirable to

Re: [HACKERS] Production block comparison facility

2014-07-22 Thread Greg Stark
If you're always going FPW then there's no point in the rest of the record. The point here was to find problems so that users could run normally with confidence. The cases you might want to run in the mode you describe are the build farm or integration testing. When treating your application on

Re: [HACKERS] Production block comparison facility

2014-07-22 Thread Simon Riggs
On 22 July 2014 12:54, Greg Stark st...@mit.edu wrote: If you're always going FPW then there's no point in the rest of the record. I think its a simple matter to mark them XLP_BKP_REMOVABLE and to skip any optimization of remainder of WAL records. The point here was to find problems so that

Re: [HACKERS] [bug fix] Suppress autovacuum: found orphan temp table message

2014-07-22 Thread MauMau
From: Andres Freund and...@2ndquadrant.com On 2014-07-22 19:13:56 +0900, MauMau wrote: But this is true if restart_after_crash = on in postgresql.conf, because the crash restart only occurs in that case. However, in HA cluster, whether it is shared-disk or replication, restart_after_crash is

Re: [HACKERS] [bug fix] Suppress autovacuum: found orphan temp table message

2014-07-22 Thread Robert Haas
On Tue, Jul 22, 2014 at 6:23 AM, Andres Freund and...@2ndquadrant.com wrote: Yes, so nobody can convince serious customers that the current behavior makes real sense. I think you're making lots of noise over a trivial log message. Could you please reconsider this? No. Just removing a

Re: [HACKERS] Index-only scans for multicolumn GIST

2014-07-22 Thread Robert Haas
On Tue, Jul 22, 2014 at 2:55 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 07/21/2014 10:47 PM, Anastasia Lubennikova wrote: Hi, hackers! There are new results of my work on GSoC project Index-only scans for GIST. Previous post is here:

Re: [HACKERS] Index-only scans for multicolumn GIST

2014-07-22 Thread Tom Lane
Heikki Linnakangas hlinnakan...@vmware.com writes: For a circle, the GiST index stores a bounding box of the circle. The new fetch function reverses that, calculating the radius and center of the circle from the bounding box. Those conversions lose some precision due to rounding. Are we

Re: [HACKERS] [bug fix] Suppress autovacuum: found orphan temp table message

2014-07-22 Thread Andres Freund
On 2014-07-22 09:39:13 -0400, Robert Haas wrote: On Tue, Jul 22, 2014 at 6:23 AM, Andres Freund and...@2ndquadrant.com wrote: Yes, so nobody can convince serious customers that the current behavior makes real sense. I think you're making lots of noise over a trivial log message. Could

Re: [HACKERS] [bug fix] Suppress autovacuum: found orphan temp table message

2014-07-22 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Jul 22, 2014 at 6:23 AM, Andres Freund and...@2ndquadrant.com wrote: No. Just removing a warning isn't the way to solve this. If you want to improve things you'll actually need to improve things not just stick your head into the sand. I've

Re: [HACKERS] [bug fix] Suppress autovacuum: found orphan temp table message

2014-07-22 Thread Andres Freund
On 2014-07-22 10:17:15 -0400, Tom Lane wrote: Or even more to the point, investigate why it's there in the first place; perhaps there's an actual fixable bug somewhere in there. I think MauMau's scenario of a failover to another database explains their existance - there's no step that'd remove

Re: [HACKERS] [bug fix] Suppress autovacuum: found orphan temp table message

2014-07-22 Thread Andres Freund
On 2014-07-22 22:18:03 +0900, MauMau wrote: From: Andres Freund and...@2ndquadrant.com On 2014-07-22 19:13:56 +0900, MauMau wrote: But this is true if restart_after_crash = on in postgresql.conf, because the crash restart only occurs in that case. However, in HA cluster, whether it is

Re: [HACKERS] [bug fix] Suppress autovacuum: found orphan temp table message

2014-07-22 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-07-22 10:17:15 -0400, Tom Lane wrote: Or even more to the point, investigate why it's there in the first place; perhaps there's an actual fixable bug somewhere in there. I think MauMau's scenario of a failover to another database explains

Re: [HACKERS] Some bogus results from prairiedog

2014-07-22 Thread Andrew Dunstan
On 07/22/2014 12:24 AM, Tom Lane wrote: According to http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=prairiedogdt=2014-07-21%2022%3A36%3A55 prairiedog saw a crash in make check on the 9.4 branch earlier tonight; but there's not a lot of evidence as to why in the buildfarm report, because

Re: [HACKERS] [GSoC2014] Patch ALTER TABLE ... SET LOGGED

2014-07-22 Thread Fabrízio de Royes Mello
On Thu, Jul 17, 2014 at 8:02 PM, Andres Freund and...@2ndquadrant.com wrote: That means should I FlushRelationBuffers(rel) before change the relpersistence? I don't think that'd help. I think what this means that you simply cannot change the relpersistence of the old relation before the

Re: [HACKERS] [bug fix] Suppress autovacuum: found orphan temp table message

2014-07-22 Thread Andres Freund
On 2014-07-23 00:13:26 +0900, MauMau wrote: Hello, Robert-san, Andres-san, Tom-san, From: Andres Freund and...@2ndquadrant.com a) There very well could be a backend reconnecting to that backendId. Then we potentially might try to remove the temp schema from two backends - I'm not sure

Re: [HACKERS] [bug fix] Suppress autovacuum: found orphan temp table message

2014-07-22 Thread MauMau
Hello, Robert-san, Andres-san, Tom-san, From: Andres Freund and...@2ndquadrant.com a) There very well could be a backend reconnecting to that backendId. Then we potentially might try to remove the temp schema from two backends - I'm not sure that's always going to end up going well.

Re: [HACKERS] Some bogus results from prairiedog

2014-07-22 Thread Robert Haas
On Tue, Jul 22, 2014 at 12:24 AM, Tom Lane t...@sss.pgh.pa.us wrote: According to http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=prairiedogdt=2014-07-21%2022%3A36%3A55 prairiedog saw a crash in make check on the 9.4 branch earlier tonight; but there's not a lot of evidence as to why in

Re: [HACKERS] [COMMITTERS] pgsql: Diagnose incompatible OpenLDAP versions during build and test.

2014-07-22 Thread Tom Lane
Noah Misch n...@leadboat.com writes: Diagnose incompatible OpenLDAP versions during build and test. Hmm. I'm pretty sure it is not considered good style to drop AC_DEFUN blocks right into configure.in; at least, we have never done that before. PGAC_LDAP_SAFE should get defined somewhere in

[HACKERS] Behavior of OFFSET -1

2014-07-22 Thread Tom Lane
Before 9.3, you got an error from this: regression=# select * from tenk1 offset -1; ERROR: OFFSET must not be negative But 9.3 and up ignore the negative OFFSET. This seems to be a thinko in my commit 1a1832eb. limit_needed() thinks it can discard the Limit plan node altogether, which of

Re: [HACKERS] Some bogus results from prairiedog

2014-07-22 Thread Andrew Dunstan
On 07/22/2014 10:55 AM, Andrew Dunstan wrote: On 07/22/2014 12:24 AM, Tom Lane wrote: According to http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=prairiedogdt=2014-07-21%2022%3A36%3A55 prairiedog saw a crash in make check on the 9.4 branch earlier tonight; but there's not a lot of

Re: [HACKERS] parametric block size?

2014-07-22 Thread Alvaro Herrera
Fabien wrote: ISTM that a desirable and reasonably simple to implement feature would be to be able to set the blocksize at initdb time, and postgres could use the value found in the database instead of a compile-time one. I think you will find it more difficult to implement than it seems at

Re: [HACKERS] Portability issues in TAP tests

2014-07-22 Thread Robert Haas
On Mon, Jul 21, 2014 at 11:39 PM, Tom Lane t...@sss.pgh.pa.us wrote: Andrew Dunstan and...@dunslane.net writes: On 07/21/2014 02:40 PM, Tom Lane wrote: I had the same feeling about the Perl on RHEL6 ;-). The TAP tests will need to be a great deal more portable than they've proven so far

Re: [HACKERS] Behavior of OFFSET -1

2014-07-22 Thread Robert Haas
On Tue, Jul 22, 2014 at 12:49 PM, Tom Lane t...@sss.pgh.pa.us wrote: Before 9.3, you got an error from this: regression=# select * from tenk1 offset -1; ERROR: OFFSET must not be negative But 9.3 and up ignore the negative OFFSET. This seems to be a thinko in my commit 1a1832eb.

Re: [HACKERS] parametric block size?

2014-07-22 Thread Fabien COELHO
Hello Alvaro, ISTM that a desirable and reasonably simple to implement feature would be to be able to set the blocksize at initdb time, and postgres could use the value found in the database instead of a compile-time one. I think you will find it more difficult to implement than it seems at

[HACKERS] Shared Data Structure b/w clients

2014-07-22 Thread Rohit Goyal
Hi All, I am working on postgresql code and having some problem. :) I need to create shared data structure, so that different client and connection can update and share the state of those data structures in memory. I planned to use top memory context but it can give me shared structure within

Re: [HACKERS] Behavior of OFFSET -1

2014-07-22 Thread David Fetter
On Tue, Jul 22, 2014 at 12:49:37PM -0400, Tom Lane wrote: Before 9.3, you got an error from this: regression=# select * from tenk1 offset -1; ERROR: OFFSET must not be negative That seems eminently sane, and should continue to error out, IM. The only circumstance I can imagine where this

Re: [HACKERS] [GSoC2014] Patch ALTER TABLE ... SET LOGGED

2014-07-22 Thread Fabrízio de Royes Mello
On Tue, Jul 22, 2014 at 12:01 PM, Fabrízio de Royes Mello fabriziome...@gmail.com wrote: On Thu, Jul 17, 2014 at 8:02 PM, Andres Freund and...@2ndquadrant.com wrote: That means should I FlushRelationBuffers(rel) before change the relpersistence? I don't think that'd help. I think

Re: [HACKERS] Shared Data Structure b/w clients

2014-07-22 Thread Atri Sharma
On Tuesday, July 22, 2014, Rohit Goyal rhtgyl...@gmail.com wrote: Hi All, I am working on postgresql code and having some problem. :) I need to create shared data structure, so that different client and connection can update and share the state of those data structures in memory. I planned

Re: [HACKERS] Shared Data Structure b/w clients

2014-07-22 Thread Rohit Goyal
Hi Atri/All, I am very new in postgresql code. Can you please help in a bit detail ortel me how to create structure in shared memory(shared buffer). It would be really easy for me if you can give me a code snippet or any link to follow. Regards, Rohit Goyal On Tue, Jul 22, 2014 at 8:30 PM,

Re: [HACKERS] Shared Data Structure b/w clients

2014-07-22 Thread Jaime Casanova
On Tue, Jul 22, 2014 at 1:33 PM, Rohit Goyal rhtgyl...@gmail.com wrote: Hi Atri/All, I am very new in postgresql code. Can you please help in a bit detail ortel me how to create structure in shared memory(shared buffer). It would be really easy for me if you can give me a code snippet or any

Re: [HACKERS] proposal (9.5) : psql unicode border line styles

2014-07-22 Thread Tomas Vondra
On 28.6.2014 21:29, Pavel Stehule wrote: Hello rebase for 9.5 test: \pset linestyle unicode \pset border 2 \pset unicode_header_linestyle double \l Regards Pavel I did a quick review of the patch today: * it applies cleanly to current HEAD (no failures, small offsets) *

Re: [HACKERS] Some bogus results from prairiedog

2014-07-22 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Jul 22, 2014 at 12:24 AM, Tom Lane t...@sss.pgh.pa.us wrote: Anyway, to cut to the chase, the crash seems to be from this: TRAP: FailedAssertion(!(FastPathStrongRelationLocks-count[fasthashcode] 0), File: lock.c, Line: 2957) So there is

Re: [HACKERS] [COMMITTERS] pgsql: Diagnose incompatible OpenLDAP versions during build and test.

2014-07-22 Thread Tom Lane
Noah Misch n...@leadboat.com writes: Diagnose incompatible OpenLDAP versions during build and test. Oh, one more thing: the Windows buildfarm members mostly don't like the dblink test case you added. Looks like the mechanism for finding the shared library doesn't work.

Re: [HACKERS] Stating the significance of Lehman Yao in the nbtree README

2014-07-22 Thread Peter Geoghegan
On Mon, Jul 21, 2014 at 9:55 PM, Amit Kapila amit.kapil...@gmail.com wrote: There is a mention about the race condition where it needs to move right in another caller (_bt_search) of _bt_moveright() as well. /* * Race -- the page we just grabbed may have split since we read its * pointer in

Re: [HACKERS] Stating the significance of Lehman Yao in the nbtree README

2014-07-22 Thread Tom Lane
Peter Geoghegan p...@heroku.com writes: Right. It seems like the nbtree README is very shy about telling us what the point of all this extra work is. IIRC, the README was written on the assumption that you'd already read LY. If this patch is mostly about not assuming that, why not?

[HACKERS] pgkill() is not POSIX-like for exiting processes

2014-07-22 Thread Noah Misch
My new OpenLDAP test case has been breaking each MSVC buildfarm member. Most MinGW members are fine, though the 9.0 and 9.1 narwhal members broke. (Newer narwhal members have been broken long-term.) The MSVC build system has a mundane inability to handle a Makefile construct I used; the first

Re: [HACKERS] Shared Data Structure b/w clients

2014-07-22 Thread Craig Ringer
On 07/23/2014 02:33 AM, Rohit Goyal wrote: I am very new in postgresql code. Can you please help in a bit detail ortel me how to create structure in shared memory(shared buffer). It would be really easy for me if you can give me a code snippet or any link to follow. There's a lot of

Re: [HACKERS] Shared Data Structure b/w clients

2014-07-22 Thread Craig Ringer
On 07/23/2014 09:46 AM, Craig Ringer wrote: There's a lot of detail on how to do this in the BDR codebase, see contrib/bdr in http://git.postgresql.org/gitweb/?p=2ndquadrant_bdr.git;a=summary Oh, sorry: in the bdr-next branch. Should've mentioned.

Re: [HACKERS] IS NOT DISTINCT FROM + Indexing

2014-07-22 Thread Jonathan S. Katz
On Jul 22, 2014, at 12:40 AM, Tom Lane t...@sss.pgh.pa.us wrote: Jonathan S. Katz jonathan.k...@excoventures.com writes: On Jul 21, 2014, at 9:51 PM, Tom Lane t...@sss.pgh.pa.us wrote: The short reason why not is that it's not an operator (where operator is defined as something with a

Re: [HACKERS] Stating the significance of Lehman Yao in the nbtree README

2014-07-22 Thread Amit Kapila
On Wed, Jul 23, 2014 at 5:58 AM, Peter Geoghegan p...@heroku.com wrote: On Mon, Jul 21, 2014 at 9:55 PM, Amit Kapila amit.kapil...@gmail.com wrote: The above indicates 2 things: a. L Y doesn't need to hold read locks concurrently. b. Advantage of right-links at all levels and high-key.

Re: [HACKERS] Stating the significance of Lehman Yao in the nbtree README

2014-07-22 Thread Peter Geoghegan
On Tue, Jul 22, 2014 at 5:39 PM, Tom Lane t...@sss.pgh.pa.us wrote: IIRC, the README was written on the assumption that you'd already read LY. If this patch is mostly about not assuming that, why not? LY made the same mistake that the authors of most influential papers make - they never get

[HACKERS] Inconsistencies of service failure handling on Windows

2014-07-22 Thread Michael Paquier
Hi all, While playing on Windows with services, I noticed an inconsistent behavior in the way failures are handled when using a service for a Postgres instance. Let's assume that there is a service called postgres that has been registered: $ psql -At -c 'select version()' PostgreSQL 9.5devel,

Re: [HACKERS] proposal (9.5) : psql unicode border line styles

2014-07-22 Thread Pavel Stehule
Hi Tomas 2014-07-22 23:20 GMT+02:00 Tomas Vondra t...@fuzzy.cz: On 28.6.2014 21:29, Pavel Stehule wrote: Hello rebase for 9.5 test: \pset linestyle unicode \pset border 2 \pset unicode_header_linestyle double \l Regards Pavel I did a quick review of the patch today:

Re: [HACKERS] Stating the significance of Lehman Yao in the nbtree README

2014-07-22 Thread Peter Geoghegan
On Tue, Jul 22, 2014 at 8:59 PM, Amit Kapila amit.kapil...@gmail.com wrote: Okay, but how does this justify to add below new text in README. + Even with these read locks, Lehman and Yao's approach obviates the + need of earlier schemes to hold multiple read locks concurrently when + descending