Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-04 Thread Amit Kapila
On Tue, Feb 4, 2014 at 12:40 PM, Tom Lane t...@sss.pgh.pa.us wrote: Amit Kapila amit.kapil...@gmail.com writes: In the function where it is used, it seems to me that it is setting DateStyle as ISO if it is not ISO and function configure_remote_session() will set it to ISO initially. So

Re: [HACKERS] Patch: Show process IDs of processes holding a lock; show relation and tuple infos of a lock to acquire

2014-02-04 Thread Christian Kruse
Hi, On 04/02/14 12:38, Fujii Masao wrote: ISTM that the phrase Request queue is not used much around the lock. Using the phrase wait queue or Simon's suggestion sound better to at least me. Thought? Sounds reasonable to me. Attached patch changes messages to the following: Process holding

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-02-04 Thread Andres Freund
On 2014-02-03 17:51:20 -0800, Peter Geoghegan wrote: On Sun, Feb 2, 2014 at 6:00 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-02-01 19:47:29 -0800, Peter Geoghegan wrote: Here are the results of a benchmark on Nathan Boley's 64-core, 4 socket server:

Re: [HACKERS] Patch: Show process IDs of processes holding a lock; show relation and tuple infos of a lock to acquire

2014-02-04 Thread Rajeev rastogi
On 4th February 2014, Christian kruse Wrote: On 04/02/14 12:38, Fujii Masao wrote: ISTM that the phrase Request queue is not used much around the lock. Using the phrase wait queue or Simon's suggestion sound better to at least me. Thought? Sounds reasonable to me. Attached patch

Re: [HACKERS] should we add a XLogRecPtr/LSN SQL type?

2014-02-04 Thread Andres Freund
Hi, On 2014-02-04 10:23:14 +0900, Michael Paquier wrote: On Tue, Feb 4, 2014 at 10:10 AM, Tom Lane t...@sss.pgh.pa.us wrote: Michael Paquier michael.paqu...@gmail.com writes: Please find attached a patch implementing lsn as a datatype, based on the one Robert wrote a couple of years ago.

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-04 Thread Andres Freund
On 2014-02-04 02:10:47 -0500, Tom Lane wrote: Amit Kapila amit.kapil...@gmail.com writes: In the function where it is used, it seems to me that it is setting DateStyle as ISO if it is not ISO and function configure_remote_session() will set it to ISO initially. So basically even if the

Re: [HACKERS] should we add a XLogRecPtr/LSN SQL type?

2014-02-04 Thread Michael Paquier
On Tue, Feb 4, 2014 at 6:15 PM, Andres Freund and...@2ndquadrant.com wrote: + /*-- + * Relational operators for LSNs + *-*/ Isn't it just operators? They aren't really

Re: [HACKERS] should we add a XLogRecPtr/LSN SQL type?

2014-02-04 Thread Andres Freund
On 2014-02-04 19:17:51 +0900, Michael Paquier wrote: On Tue, Feb 4, 2014 at 6:15 PM, Andres Freund and...@2ndquadrant.com wrote: + /*-- + * Relational operators for LSNs +

Re: [HACKERS] [bug fix] PostgreSQL fails to start on Windows if it crashes after tablespace creation

2014-02-04 Thread MauMau
I'm sorry, I'm replying to an older mail, because I lost your latest mail by mistake. Ah. Sorry, I missed that part. As NTFS junctions and symbolic links are different (although they behave similarly), there seems only a minor inconvenience related to misleading error message i.e. You are

Re: [HACKERS] [bug fix] postgres.exe fails to start on Windows Server 2012 due to ASLR

2014-02-04 Thread MauMau
From: Craig Ringer cr...@2ndquadrant.com I completely agree; just saying that any installer can set the key. I'm convinced that setting this flag is appropriate, at least while Pg relies on having the shared memory segment mapped in the same zone in every executable. Just pointing out that

Re: [HACKERS] inherit support for foreign tables

2014-02-04 Thread Robert Haas
On Sun, Feb 2, 2014 at 10:15 PM, Etsuro Fujita fujita.ets...@lab.ntt.co.jp wrote: Allowing ALTER COLUMN SET STORAGE on foreign tables would make sense if for example, SELECT * INTO local_table FROM foreign_table did create a new local table of columns having the storage types associated with

Re: [HACKERS] [bug fix] postgres.exe fails to start on Windows Server 2012 due to ASLR

2014-02-04 Thread Craig Ringer
On 02/04/2014 07:28 PM, MauMau wrote: Please don't mind, I didn't misunderstand your intent. I think we should apply this in the next minor release to avoid unnecessary confusion -- more new users would use PostgreSQL on Windows 8/2012 and hit this problem. I added this patch to the

Re: [HACKERS] should we add a XLogRecPtr/LSN SQL type?

2014-02-04 Thread Michael Paquier
On Tue, Feb 4, 2014 at 7:22 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-02-04 19:17:51 +0900, Michael Paquier wrote: @@ -180,7 +175,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) else nulls[i++] = true; if (restart_lsn !=

Re: [HACKERS] should we add a XLogRecPtr/LSN SQL type?

2014-02-04 Thread Andres Freund
On 2014-02-04 21:04:13 +0900, Michael Paquier wrote: On Tue, Feb 4, 2014 at 7:22 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-02-04 19:17:51 +0900, Michael Paquier wrote: @@ -180,7 +175,7 @@ pg_get_replication_slots(PG_FUNCTION_ARGS) else

Re: [HACKERS] Re: Misaligned BufferDescriptors causing major performance problems on AMD

2014-02-04 Thread Andres Freund
On 2014-02-04 00:38:19 +0100, Andres Freund wrote: A quick hack (attached) making BufferDescriptor 64byte aligned indeed restored performance across all max_connections settings. It's not surprising that a misaligned buffer descriptor causes problems - there'll be plenty of false

[HACKERS] specifying repeatable read in PGOPTIONS

2014-02-04 Thread Andres Freund
Hi, I recently had the need to bury the used isolation level in the connection string, but it turns out that doesn't work that well... PGOPTIONS='-c default_transaction_isolation=serializable' \ psql ... -c SHOW default_transaction_isolation works well enough, but PGOPTIONS='-c

Re: [HACKERS] could not create IPv6 socket (AI_ADDRCONFIG)

2014-02-04 Thread Alvaro Herrera
Tom Lane wrote: Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp writes: Hello, I have often seen inquiries about an log message from PostgreSQL server. LOG: could not create IPv6 socket: Address family not supported by protocol That's merely a harmless log message. If we're

Re: [HACKERS] could not create IPv6 socket (AI_ADDRCONFIG)

2014-02-04 Thread Tom Lane
Alvaro Herrera alvhe...@2ndquadrant.com writes: Tom Lane wrote: Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp writes: Hello, I have often seen inquiries about an log message from PostgreSQL server. LOG: could not create IPv6 socket: Address family not supported by protocol That's

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-04 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-02-04 02:10:47 -0500, Tom Lane wrote: Meh. It might be that the DateStyle usage in postgres_fdw would accidentally fail to malfunction if it saw a bogus value of the variable. But it's hard to believe that this would be true of

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-04 Thread Andrew Dunstan
On 02/04/2014 10:43 AM, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-02-04 02:10:47 -0500, Tom Lane wrote: Meh. It might be that the DateStyle usage in postgres_fdw would accidentally fail to malfunction if it saw a bogus value of the variable. But it's hard to

Re: [HACKERS] [doc patch] extra_float_digits and casting from real to numeric

2014-02-04 Thread Christoph Berg
Re: To Tom Lane 2014-01-08 20140108094017.ga20...@msgid.df7cb.de What about this patch to mention this gotcha more explicitely in the documentation? diff --git a/doc/src/sgml/datatype.sgml b/doc/src/sgml/datatype.sgml new file mode 100644 index 0386330..968f4a7 ***

Re: [HACKERS] specifying repeatable read in PGOPTIONS

2014-02-04 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: PGOPTIONS='-c default_transaction_isolation=serializable' \ psql ... -c SHOW default_transaction_isolation works well enough, but PGOPTIONS='-c default_transaction_isolation=repeatable read' \ psql ... -c SHOW default_transaction_isolation

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-04 Thread Andres Freund
On February 4, 2014 5:06:52 PM CET, Andrew Dunstan and...@dunslane.net wrote: On 02/04/2014 10:43 AM, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-02-04 02:10:47 -0500, Tom Lane wrote: Meh. It might be that the DateStyle usage in postgres_fdw would accidentally fail

[HACKERS] nested hstore - large insert crashes server

2014-02-04 Thread Erik Rijkers
CentOS Release 6.5 (final) AMD FX(tm)-8120 Eight-Core 2.6.32-431.3.1.el6.x86_64 #1 SMP Fri Jan 3 21:39:27 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux memory: 8GB I am testing nested hstore, on a server with both with these patches: jsonb-9.patch.gz nested-hstore-9.patch.gz One of the first

Re: [HACKERS] specifying repeatable read in PGOPTIONS

2014-02-04 Thread Andres Freund
On 2014-02-04 11:36:22 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: PGOPTIONS='-c default_transaction_isolation=serializable' \ psql ... -c SHOW default_transaction_isolation works well enough, but PGOPTIONS='-c default_transaction_isolation=repeatable read' \

Re: [HACKERS] extension_control_path

2014-02-04 Thread David E. Wheeler
On Jan 30, 2014, at 10:06 AM, Sergey Muraviov sergey.k.murav...@gmail.com wrote: Now it looks fine for me. Just as another data point, I recently submitted pgTAP to the Homebrew project This is the build-from-source system for OS X, used by a lot of web developers. In my build script, I

Re: [HACKERS] specifying repeatable read in PGOPTIONS

2014-02-04 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-02-04 11:36:22 -0500, Tom Lane wrote: -1. This is not a general solution to the problem. There are other GUCs for which people might want spaces in the value. Sure, I didn't say it was. But I don't see any oother values that are likely

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-04 Thread Andrew Dunstan
On 02/04/2014 11:30 AM, Andres Freund wrote: We have details on how to build with Mingw/Msys on Windows on an Amazon VM http://wiki.postgresql.org/wiki/Building_With_MinGW which is either free or very cheap. Do I need to give instructions on how to do this for MSVC builds too? It's really

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-04 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: If someone volunteered to pay for the storage, I'd be prepared to make some time to create an AMI to reduce the startup time dramatically. Basically it would be boot the AMI and start testing your patches. I'd even make it as friendly as possible

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-04 Thread Joshua D. Drake
On 02/04/2014 09:34 AM, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: If someone volunteered to pay for the storage, I'd be prepared to make some time to create an AMI to reduce the startup time dramatically. Basically it would be boot the AMI and start testing your patches. I'd

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-04 Thread Jeff Janes
On Tue, Feb 4, 2014 at 8:06 AM, Andrew Dunstan and...@dunslane.net wrote: On 02/04/2014 10:43 AM, Tom Lane wrote: Ugh. This problem was bad enough when I thought that it would only lead to link-time errors detectable in the buildfarm. If it can lead to errors only observable at runtime

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

2014-02-04 Thread Robert Haas
On Tue, Feb 4, 2014 at 12:39 PM, Amit Kapila amit.kapil...@gmail.com wrote: Now there is approximately 1.4~5% CPU gain for hundred tiny fields, half nulled case I don't want to advocate too strongly for this patch because, number one, Amit is a colleague and more importantly, number two, I

Re: [HACKERS] nested hstore - large insert crashes server

2014-02-04 Thread Christian Kruse
Hi, On 04/02/14 17:41, Erik Rijkers wrote: 2014-02-04 10:34:25.376 CET 29133 LOG: server process (PID 29459) was terminated by signal 9: Killed Did you check if this was the OOM killer? Should be logged in dmesg. Best regards, -- Christian Kruse http://www.2ndQuadrant.com/

Re: [HACKERS] nested hstore - large insert crashes server

2014-02-04 Thread Erik Rijkers
On Tue, February 4, 2014 18:56, Christian Kruse wrote: Hi, On 04/02/14 17:41, Erik Rijkers wrote: 2014-02-04 10:34:25.376 CET 29133 LOG: server process (PID 29459) was terminated by signal 9: Killed Did you check if this was the OOM killer? Should be logged in dmesg. I would be

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-04 Thread Tom Lane
Joshua D. Drake j...@commandprompt.com writes: On 02/04/2014 09:34 AM, Tom Lane wrote: My own opinion is that I've already wasted untold man-hours thanks to the random porting problems induced by Windows, a platform that I never have and never will care about personally. I will *not* spend my

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

2014-02-04 Thread Bruce Momjian
On Tue, Feb 4, 2014 at 01:28:38PM -0500, Robert Haas wrote: Meanwhile, in friendlier cases, like one short and one long field, no change, we're seeing big improvements. That particular case shows a speedup of 21% and a WAL reduction of 36%. That's a pretty big deal, and I think not

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

2014-02-04 Thread Andres Freund
On 2014-02-04 14:09:57 -0500, Bruce Momjian wrote: On Tue, Feb 4, 2014 at 01:28:38PM -0500, Robert Haas wrote: Meanwhile, in friendlier cases, like one short and one long field, no change, we're seeing big improvements. That particular case shows a speedup of 21% and a WAL reduction of

Re: [HACKERS] PQputCopyData dont signal error

2014-02-04 Thread Jeff Davis
On Thu, 2011-04-14 at 10:50 +0300, Heikki Linnakangas wrote: On 14.04.2011 10:15, Pavel Stehule wrote: Hello I have a problem with PQputCopyData function. It doesn't signal some error. while ((row = mysql_fetch_row(res)) != NULL) {

Re: [HACKERS] nested hstore - large insert crashes server

2014-02-04 Thread Tom Lane
Erik Rijkers e...@xs4all.nl writes: On Tue, February 4, 2014 18:56, Christian Kruse wrote: Did you check if this was the OOM killer? Should be logged in dmesg. I would be surprised if it wasn't. (no access to that machine at the moment) How do we regard such crashes? It seems to me this was

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-04 Thread Adrian Klaver
On 02/04/2014 10:53 AM, Tom Lane wrote: Joshua D. Drake j...@commandprompt.com writes: On 02/04/2014 09:34 AM, Tom Lane wrote: My own opinion is that I've already wasted untold man-hours thanks to the random porting problems induced by Windows, a platform that I never have and never will care

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-04 Thread Andrew Dunstan
On 02/04/2014 01:53 PM, Tom Lane wrote: Joshua D. Drake j...@commandprompt.com writes: On 02/04/2014 09:34 AM, Tom Lane wrote: My own opinion is that I've already wasted untold man-hours thanks to the random porting problems induced by Windows, a platform that I never have and never will care

Re: [HACKERS] nested hstore - large insert crashes server

2014-02-04 Thread Stephen Frost
* Erik Rijkers (e...@xs4all.nl) wrote: On Tue, February 4, 2014 18:56, Christian Kruse wrote: On 04/02/14 17:41, Erik Rijkers wrote: 2014-02-04 10:34:25.376 CET 29133 LOG: server process (PID 29459) was terminated by signal 9: Killed Did you check if this was the OOM killer? Should be

Re: [HACKERS] PQputCopyData dont signal error

2014-02-04 Thread Tom Lane
Jeff Davis pg...@j-davis.com writes: On Thu, 2011-04-14 at 10:50 +0300, Heikki Linnakangas wrote: I think you'll need to send all the data and finish the COPY until you get an error. If you have a lot of data to send, you might want to slice it into multiple COPY statements of say 50MB each,

Re: [HACKERS] Wait free LW_SHARED acquisition

2014-02-04 Thread Christian Kruse
Hi, I'm doing some benchmarks regarding this problem: one set with baseline and one set with your patch. Machine was a 32 core machine (4 CPUs with 8 cores), 252 gib RAM. Both versions have the type align patch applied. pgbench-tools config: SCALES=100 SETCLIENTS=1 4 8 16 32 48 64 96 128

Re: [HACKERS] Wait free LW_SHARED acquisition

2014-02-04 Thread Peter Geoghegan
On Tue, Feb 4, 2014 at 11:39 AM, Christian Kruse christ...@2ndquadrant.com wrote: I'm doing some benchmarks regarding this problem: one set with baseline and one set with your patch. Machine was a 32 core machine (4 CPUs with 8 cores), 252 gib RAM. Both versions have the type align patch

Re: [HACKERS] Wait free LW_SHARED acquisition

2014-02-04 Thread Peter Geoghegan
On Tue, Feb 4, 2014 at 11:39 AM, Christian Kruse christ...@2ndquadrant.com wrote: I added -M prepared to the pgbench call in the benchwarmer script. The read-only tests are finished, I come to similiar results as yours: http://wwwtech.de/pg/benchmarks-lwlock-read-only/ Note that Christian

Re: [HACKERS] Wait free LW_SHARED acquisition

2014-02-04 Thread Andres Freund
On 2014-02-04 11:48:14 -0800, Peter Geoghegan wrote: On Tue, Feb 4, 2014 at 11:39 AM, Christian Kruse christ...@2ndquadrant.com wrote: I added -M prepared to the pgbench call in the benchwarmer script. The read-only tests are finished, I come to similiar results as yours:

Re: [HACKERS] Wait free LW_SHARED acquisition

2014-02-04 Thread Peter Geoghegan
On Tue, Feb 4, 2014 at 11:50 AM, Andres Freund and...@2ndquadrant.com wrote: I think he has applied the patch to hack around the alignment issue I pushed to git for both branches. It's not nice enough to be applied yet, but it should fix the issue. I think the 201 is just a remembrance of

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-04 Thread Joshua D. Drake
On 02/04/2014 11:17 AM, Andrew Dunstan wrote: prepared to declare the entire damn thing no longer supported. Although that is obviously your prerogative it is important to remember that Windows is easily the second most used version of PostgreSQL out there (behind Linux). [ shrug... ] If

Re: [HACKERS] Wait free LW_SHARED acquisition

2014-02-04 Thread Peter Geoghegan
On Tue, Feb 4, 2014 at 11:39 AM, Christian Kruse christ...@2ndquadrant.com wrote: I'm doing some benchmarks regarding this problem: one set with baseline and one set with your patch. Machine was a 32 core machine (4 CPUs with 8 cores), 252 gib RAM. What CPU model? Can you post /proc/cpuinfo?

Re: [HACKERS] Wait free LW_SHARED acquisition

2014-02-04 Thread Andres Freund
On February 4, 2014 8:53:36 PM CET, Peter Geoghegan p...@heroku.com wrote: On Tue, Feb 4, 2014 at 11:50 AM, Andres Freund and...@2ndquadrant.com wrote: I think he has applied the patch to hack around the alignment issue I pushed to git for both branches. It's not nice enough to be applied yet,

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-04 Thread Jeff Janes
On Tue, Feb 4, 2014 at 9:26 AM, Andrew Dunstan and...@dunslane.net wrote: On 02/04/2014 11:30 AM, Andres Freund wrote: We have details on how to build with Mingw/Msys on Windows on an Amazon VM http://wiki.postgresql.org/wiki/Building_With_MinGW which is either free or very cheap. Do I

Re: [HACKERS] Wait free LW_SHARED acquisition

2014-02-04 Thread Christian Kruse
Hi, On 04/02/14 12:02, Peter Geoghegan wrote: On Tue, Feb 4, 2014 at 11:39 AM, Christian Kruse christ...@2ndquadrant.com wrote: I'm doing some benchmarks regarding this problem: one set with baseline and one set with your patch. Machine was a 32 core machine (4 CPUs with 8 cores), 252 gib

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-04 Thread Robert Haas
On Tue, Feb 4, 2014 at 12:34 PM, Tom Lane t...@sss.pgh.pa.us wrote: Andrew Dunstan and...@dunslane.net writes: If someone volunteered to pay for the storage, I'd be prepared to make some time to create an AMI to reduce the startup time dramatically. Basically it would be boot the AMI and start

Re: [HACKERS] should we add a XLogRecPtr/LSN SQL type?

2014-02-04 Thread Peter Eisentraut
Perhaps this type should be called pglsn, since it's an implementation-specific detail and not a universal concept like int, point, or uuid. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

[HACKERS] Re: Viability of text HISTORY/INSTALL/regression README files (was Re: [COMMITTERS] pgsql: Document a few more regression test hazards.)

2014-02-04 Thread Robert Haas
On Tue, Feb 4, 2014 at 1:38 AM, Tom Lane t...@sss.pgh.pa.us wrote: Noah Misch n...@leadboat.com writes: Robert Haas robertmh...@gmail.com writes: I wonder if these standalone things are really worthwhile. I wonder how difficult it would be to make sufficient link data available when

Re: [HACKERS] Wait free LW_SHARED acquisition

2014-02-04 Thread Christian Kruse
Hi, On 04/02/14 21:03, Andres Freund wrote: Christian, could you rerun with master (the commit on which the branch is based on), the alignment patch, and then the lwlock patch? Best with max_connections 200. That's probably more important than the write tests as a first step.. Ok, benchmark

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-04 Thread Andrew Dunstan
On 02/04/2014 03:08 PM, Jeff Janes wrote: Do you know about what it would cost? Could official community funds be used for it (it seems like something that is cheap, but which you wouldn't want to be forgotten about some month.) Having an AMI would help, but even with an AMI in place,

Re: [HACKERS] GIN improvements part2: fast scan

2014-02-04 Thread Alexander Korotkov
On Mon, Feb 3, 2014 at 6:31 PM, Alexander Korotkov aekorot...@gmail.comwrote: On Mon, Jan 27, 2014 at 7:30 PM, Alexander Korotkov aekorot...@gmail.comwrote: On Mon, Jan 27, 2014 at 2:32 PM, Alexander Korotkov aekorot...@gmail.com wrote: On Sun, Jan 26, 2014 at 8:14 PM, Heikki Linnakangas

Re: [HACKERS] Wait free LW_SHARED acquisition

2014-02-04 Thread Peter Geoghegan
On Tue, Feb 4, 2014 at 12:30 PM, Christian Kruse christ...@2ndquadrant.com wrote: Ok, benchmark for baseline+alignment patch is running. I see that you have enabled latency information. For this kind of thing I prefer to hack pgbench-tools to not collect this (i.e. to not pass the -l flag,

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

2014-02-04 Thread Bruce Momjian
On Tue, Feb 4, 2014 at 08:11:18PM +0100, Andres Freund wrote: On 2014-02-04 14:09:57 -0500, Bruce Momjian wrote: On Tue, Feb 4, 2014 at 01:28:38PM -0500, Robert Haas wrote: Meanwhile, in friendlier cases, like one short and one long field, no change, we're seeing big improvements. That

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

2014-02-04 Thread Peter Geoghegan
On Tue, Feb 4, 2014 at 11:11 AM, Andres Freund and...@2ndquadrant.com wrote: Does this feature relate to compression of WAL page images at all? No. So the obvious question is: where, if anywhere, do the two efforts (this patch, and Fujii's patch) overlap? Does Fujii have any concerns about

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

2014-02-04 Thread Andres Freund
On February 4, 2014 10:50:10 PM CET, Peter Geoghegan p...@heroku.com wrote: On Tue, Feb 4, 2014 at 11:11 AM, Andres Freund and...@2ndquadrant.com wrote: Does this feature relate to compression of WAL page images at all? No. So the obvious question is: where, if anywhere, do the two efforts

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

2014-02-04 Thread Peter Geoghegan
On Tue, Feb 4, 2014 at 1:58 PM, Andres Freund and...@2ndquadrant.com wrote: I think there's zero overlap. They're completely complimentary features. It's not like normal WAL records have an irrelevant volume. I'd have thought so too, but I would not like to assume. Like many people commenting

Re: [HACKERS] jsonb and nested hstore

2014-02-04 Thread Josh Berkus
On 02/03/2014 07:27 AM, Andres Freund wrote: On 2014-02-03 09:22:52 -0600, Merlin Moncure wrote: I lost my stomach (or maybe it was the glass of red) somewhere in the middle, but I think this needs a lot of work. Especially the io code doesn't seem ready to me. I'd consider ripping out the

Re: [HACKERS] integrate pg_upgrade analyze_new_cluster.sh into vacuumdb

2014-02-04 Thread Jeff Janes
On Tue, Jan 21, 2014 at 9:06 AM, Oskari Saarenmaa o...@ohmu.fi wrote: 09.01.2014 05:15, Peter Eisentraut kirjoitti: pg_upgrade creates a script analyze_new_cluster.{sh|bat} that runs vacuumdb --analyze-only in three stages with different statistics target settings to get a fresh cluster

[HACKERS] Minor performance improvement in transition to external sort

2014-02-04 Thread Jeremy Harris
The attached patch replaces the existing siftup method for heapify with a siftdown method. Tested with random integers it does 18% fewer compares and takes 10% less time for the heapify, over the work_mem range 1024 to 1048576. Both algorithms appear to be O(n) (contradicting Wikipedia's claim

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-04 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: You know, I would really prefer to just stick a PGDLLIMPORT on this place and any others that need it, and any others that come up, than turn this into a political football. Having to sprinkle PGDLLIMPORT on the handful of variables that are accessed

Re: [HACKERS] Wait free LW_SHARED acquisition

2014-02-04 Thread Andres Freund
On 2014-02-04 13:42:51 -0800, Peter Geoghegan wrote: On Tue, Feb 4, 2014 at 12:30 PM, Christian Kruse christ...@2ndquadrant.com wrote: Ok, benchmark for baseline+alignment patch is running. I see that you have enabled latency information. For this kind of thing I prefer to hack

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-04 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 02/04/2014 03:08 PM, Jeff Janes wrote: Having an AMI would help, but even with an AMI in place, MinGW is still insanely slow. Running make on already made PostgreSQL (so there was nothing to actually do) takes 1.5 minutes. And a make after a

Re: [HACKERS] Re: Misaligned BufferDescriptors causing major performance problems on AMD

2014-02-04 Thread Peter Geoghegan
On Tue, Feb 4, 2014 at 4:21 AM, Andres Freund and...@2ndquadrant.com wrote: Which imo means fixing this got more important... I strongly agree. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-04 Thread Andrew Dunstan
On 02/04/2014 05:47 PM, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: On 02/04/2014 03:08 PM, Jeff Janes wrote: Having an AMI would help, but even with an AMI in place, MinGW is still insanely slow. Running make on already made PostgreSQL (so there was nothing to actually do)

Re: [HACKERS] should we add a XLogRecPtr/LSN SQL type?

2014-02-04 Thread Michael Paquier
On Wed, Feb 5, 2014 at 5:26 AM, Peter Eisentraut pete...@gmx.net wrote: Perhaps this type should be called pglsn, since it's an implementation-specific detail and not a universal concept like int, point, or uuid. It makes sense. I'll update the patches according to that. -- Michael -- Sent

Re: [HACKERS] [PERFORM] encouraging index-only scans

2014-02-04 Thread Jeff Janes
On Mon, Feb 3, 2014 at 8:55 AM, Robert Haas robertmh...@gmail.com wrote: I've also had some further thoughts about the right way to drive vacuum scheduling. I think what we need to do is tightly couple the rate at which we're willing to do vacuuming to the rate at which we're incurring

Re: [HACKERS] Re: Misaligned BufferDescriptors causing major performance problems on AMD

2014-02-04 Thread Peter Geoghegan
On Mon, Feb 3, 2014 at 3:38 PM, Andres Freund and...@2ndquadrant.com wrote: A quick hack (attached) making BufferDescriptor 64byte aligned indeed restored performance across all max_connections settings. It's not surprising that a misaligned buffer descriptor causes problems - there'll be

Re: [HACKERS] Minor performance improvement in transition to external sort

2014-02-04 Thread Michael Paquier
On Wed, Feb 5, 2014 at 7:22 AM, Jeremy Harris j...@wizmail.org wrote: The attached patch replaces the existing siftup method for heapify with a siftdown method. Tested with random integers it does 18% fewer compares and takes 10% less time for the heapify, over the work_mem range 1024 to

Re: [HACKERS] should we add a XLogRecPtr/LSN SQL type?

2014-02-04 Thread Michael Paquier
On Wed, Feb 5, 2014 at 9:38 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Wed, Feb 5, 2014 at 8:59 AM, Michael Paquier michael.paqu...@gmail.com wrote: I'll update the patches according to that. Here are the updated patches with the following changes (according to previous

Re: [HACKERS] PoC: Duplicate Tuple Elidation during External Sort for DISTINCT

2014-02-04 Thread Jon Nelson
What - if anything - do I need to do to get this on the commitfest list for the next commitfest? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] inherit support for foreign tables

2014-02-04 Thread Etsuro Fujita
(2014/02/04 20:56), Robert Haas wrote: On Sun, Feb 2, 2014 at 10:15 PM, Etsuro Fujita fujita.ets...@lab.ntt.co.jp wrote: Allowing ALTER COLUMN SET STORAGE on foreign tables would make sense if for example, SELECT * INTO local_table FROM foreign_table did create a new local table of columns

Re: [HACKERS] could not create IPv6 socket (AI_ADDRCONFIG)

2014-02-04 Thread Kyotaro HORIGUCHI
Hello, At Tue, 04 Feb 2014 02:07:08 -0500, Tom Lane t...@sss.pgh.pa.us wrote in 3176.1391497...@sss.pgh.pa.us One good reason not to trust this too much is that getaddrinfo() is fundamentally a userspace DNS access function, and as such it has no very good way to know if there's currently an

Re: [HACKERS] could not create IPv6 socket (AI_ADDRCONFIG)

2014-02-04 Thread Tom Lane
Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp writes: getaddrinfo returned two same entries having the same address AF_INET 127.0.0.1:14357. One of them is for ::1 in hosts. This is worse than current behavior X-( Yeah, the fundamental issue is that getaddrinfo tends to return bogus info.

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-04 Thread Craig Ringer
On 02/05/2014 04:08 AM, Jeff Janes wrote: So doing a git bisect is just painful. Is the MSVC build faster? Yes, but not on EC2. I've found Windows EC2 instances so impossibly slow I just gave up working with it. It took 1.5 hours to do a build and regression check with msvc on a Medium EC2

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-04 Thread Craig Ringer
On 02/05/2014 02:53 AM, Tom Lane wrote: Joshua D. Drake j...@commandprompt.com writes: On 02/04/2014 09:34 AM, Tom Lane wrote: My own opinion is that I've already wasted untold man-hours thanks to the random porting problems induced by Windows, a platform that I never have and never will care

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-04 Thread Andrew Dunstan
On 02/04/2014 10:48 PM, Craig Ringer wrote: On 02/05/2014 04:08 AM, Jeff Janes wrote: So doing a git bisect is just painful. Is the MSVC build faster? Yes, but not on EC2. I've found Windows EC2 instances so impossibly slow I just gave up working with it. It took 1.5 hours to do a build

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-04 Thread Craig Ringer
On 02/05/2014 06:29 AM, Tom Lane wrote: I had been okay with the manual PGDLLIMPORT-sprinkling approach (not happy with it, of course, but prepared to tolerate it) as long as I believed the buildfarm would reliably tell us of the need for it. That assumption has now been conclusively

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-04 Thread Craig Ringer
On 02/05/2014 12:06 AM, Andrew Dunstan wrote: On 02/04/2014 10:43 AM, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-02-04 02:10:47 -0500, Tom Lane wrote: Meh. It might be that the DateStyle usage in postgres_fdw would accidentally fail to malfunction if it saw a

Re: [HACKERS] PoC: Duplicate Tuple Elidation during External Sort for DISTINCT

2014-02-04 Thread Michael Paquier
On Wed, Feb 5, 2014 at 10:33 AM, Jon Nelson jnelson+pg...@jamponi.net wrote: What - if anything - do I need to do to get this on the commitfest list for the next commitfest? The list of instructions is here: http://wiki.postgresql.org/wiki/Submitting_a_Patch#Patch_submission Then the next commit

[HACKERS] Re: Viability of text HISTORY/INSTALL/regression README files (was Re: [COMMITTERS] pgsql: Document a few more regression test hazards.)

2014-02-04 Thread Noah Misch
On Tue, Feb 04, 2014 at 03:28:45PM -0500, Robert Haas wrote: On Tue, Feb 4, 2014 at 1:38 AM, Tom Lane t...@sss.pgh.pa.us wrote: Noah Misch n...@leadboat.com writes: Robert Haas robertmh...@gmail.com writes: I wonder if these standalone things are really worthwhile. I wonder how

[HACKERS] PostgreSQL Failback without rebuild

2014-02-04 Thread James Sewell
Hello All, I have been reading through some of the recent discussions about failback when in a streaming replication setup. I define failback as: 1. Node A is master, Node B is slave 2. Node A crashes || Node A is stopped || nothing happens 3. Promote Node B to Master 4. Attach Node

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-04 Thread Tom Lane
Craig Ringer cr...@2ndquadrant.com writes: On 02/05/2014 06:29 AM, Tom Lane wrote: I had been okay with the manual PGDLLIMPORT-sprinkling approach (not happy with it, of course, but prepared to tolerate it) as long as I believed the buildfarm would reliably tell us of the need for it. That

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

2014-02-04 Thread Amit Kapila
On Tue, Feb 4, 2014 at 11:58 PM, Robert Haas robertmh...@gmail.com wrote: On Tue, Feb 4, 2014 at 12:39 PM, Amit Kapila amit.kapil...@gmail.com wrote: Now there is approximately 1.4~5% CPU gain for hundred tiny fields, half nulled case Assuming that the logic isn't buggy, a point in need of

Re: [HACKERS] PostgreSQL Failback without rebuild

2014-02-04 Thread Amit Kapila
On Wed, Feb 5, 2014 at 10:30 AM, James Sewell james.sew...@lisasoft.com wrote: Hello All, I have been reading through some of the recent discussions about failback when in a streaming replication setup. I define failback as: Node A is master, Node B is slave Node A crashes || Node A is

Re: [HACKERS] jsonb and nested hstore

2014-02-04 Thread Heikki Linnakangas
On 02/03/2014 05:22 PM, Merlin Moncure wrote: I lost my stomach (or maybe it was the glass of red) somewhere in the middle, but I think this needs a lot of work. Especially the io code doesn't seem ready to me. I'd consider ripping out the send/recv code for 9.4, that seems the biggest can of

Re: [HACKERS] PostgreSQL Failback without rebuild

2014-02-04 Thread Michael Paquier
On Wed, Feb 5, 2014 at 3:14 PM, Amit Kapila amit.kapil...@gmail.com wrote: On Wed, Feb 5, 2014 at 10:30 AM, James Sewell james.sew...@lisasoft.com I've seen some proposals and a tool (pg_rewind), but all seem to have draw backs. As far as I remember, one of the main drawbacks for pg_rewind

Re: [HACKERS] jsonb and nested hstore

2014-02-04 Thread Oleg Bartunov
Andrew provided us more information and we'll work on recv. What people think about testing this stuff ? btw, we don't have any regression test on this. Oleg On Wed, Feb 5, 2014 at 2:03 AM, Josh Berkus j...@agliodbs.com wrote: On 02/03/2014 07:27 AM, Andres Freund wrote: On 2014-02-03

Re: [HACKERS] Failure while inserting parent tuple to B-tree is not fun

2014-02-04 Thread Heikki Linnakangas
On 02/04/2014 02:40 AM, Peter Geoghegan wrote: On Fri, Jan 31, 2014 at 9:09 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: I refactored the loop in _bt_moveright to, well, not have that bug anymore. The 'page' and 'opaque' pointers are now fetched at the beginning of the loop. Did I miss