Re: [HACKERS] [GENERAL] Insert result does not match record count

2014-02-02 Thread Vik Fearing
On 02/01/2014 02:26 AM, Bruce Momjian wrote: On Sat, Feb 1, 2014 at 02:25:16AM +0100, Vik Fearing wrote: OK, thanks for the feedback. I understand now. The contents of the string will potentially have a larger integer, but the byte length of the string in the wire protocol doesn't change.

Re: [HACKERS] [PATCH] pg_sleep(interval)

2014-02-02 Thread Julien Rouhaud
Hi It seems like pg_sleep_until() has issues if used within a transaction, as it uses now() and not clock_timestamp(). Please find attached a patch that solves this issue. For consistency reasons, I also modified pg_sleep_for() to also use clock_timestamp. Regards On Fri, Jan 31, 2014 at 2:12

Re: [HACKERS] GIN improvements part2: fast scan

2014-02-02 Thread Heikki Linnakangas
On 01/30/2014 01:53 AM, Tomas Vondra wrote: (3) A file with explain plans for 4 queries suffering ~2x slowdown, and explain plans with 9.4 master and Heikki's patches is available here: http://www.fuzzy.cz/tmp/gin/queries.txt All the queries have 6 common words, and the

Re: [HACKERS] SSL: better default ciphersuite

2014-02-02 Thread Marko Kreen
On Thu, Dec 12, 2013 at 04:32:07PM +0200, Marko Kreen wrote: Attached patch changes default ciphersuite to HIGH:MEDIUM:+3DES:!aNULL and also adds documentation about reasoning for it. This is the last pending SSL cleanup related patch:

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-02-02 Thread Andres Freund
Hi, On 2014-02-01 19:47:29 -0800, Peter Geoghegan wrote: Here are the results of a benchmark on Nathan Boley's 64-core, 4 socket server: http://postgres-benchmarks.s3-website-us-east-1.amazonaws.com/amd-4-socket-rwlocks/ That's interesting. The maximum number of what you see here (~293125)

Re: [HACKERS] [9.3 bug] disk space in pg_xlog increases during archive recovery

2014-02-02 Thread Fujii Masao
On Sun, Feb 2, 2014 at 5:49 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-01-24 22:31:17 +0900, MauMau wrote: From: Fujii Masao masao.fu...@gmail.com On Wed, Jan 22, 2014 at 6:37 AM, Heikki Linnakangas Thanks! The patch looks good to me. Attached is the updated version of the patch.

[HACKERS] Misaligned BufferDescriptors causing major performance problems on AMD

2014-02-02 Thread Andres Freund
Hi, In the nearby thread at http://archives.postgresql.org/message-id/20140202140014.GM5930%40awork2.anarazel.de Peter and I discovered that there is a large performance difference between different max_connections on a larger machine (4x Opteron 6272, 64 cores together) in a readonly pgbench

Re: [HACKERS] [9.3 bug] disk space in pg_xlog increases during archive recovery

2014-02-02 Thread Andres Freund
On 2014-02-02 23:50:40 +0900, Fujii Masao wrote: On Sun, Feb 2, 2014 at 5:49 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-01-24 22:31:17 +0900, MauMau wrote: I haven't tried reducing checkpoint_timeout. Did you try reducing checkpoint_segments? As I pointed out, at least if

Re: [HACKERS] pg_basebackup and pg_stat_tmp directory

2014-02-02 Thread Fujii Masao
On Sat, Feb 1, 2014 at 2:08 AM, Robert Haas robertmh...@gmail.com wrote: On Fri, Jan 31, 2014 at 8:40 AM, Fujii Masao masao.fu...@gmail.com wrote: Yeah, I was thinking that, too. I'm not sure whether including log files in backup really increases the security risk, though. There are already

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-02 Thread Tom Lane
Dave Page dp...@pgadmin.org writes: On Sun, Feb 2, 2014 at 1:03 AM, Tom Lane t...@sss.pgh.pa.us wrote: I think we should give serious consideration to desupporting this combination so that we can get rid of the plague of PGDLLIMPORT marks. No objection here - though I should point out that

Re: [HACKERS] [GENERAL] Insert result does not match record count

2014-02-02 Thread Tom Lane
Vik Fearing vik.fear...@dalibo.com writes: Without re-doing the work, my IRC logs show that I was bothered by this in src/backend/tcop/postgres.c: max_rows = pq_getmsgint(input_message, 4); I needed to change max_rows to int64 which meant I had to change pq_getmsgint to

Re: [HACKERS] [COMMITTERS] pgsql: Include planning time in EXPLAIN ANALYZE output.

2014-02-02 Thread Tom Lane
Peter Geoghegan p...@heroku.com writes: On Wed, Jan 29, 2014 at 1:10 PM, Robert Haas rh...@postgresql.org wrote: Include planning time in EXPLAIN ANALYZE output. Isn't it perhaps a little confusing that Planning time may well exceed Total runtime? Perhaps s/Total runtime/Execution time/ ?

Re: [HACKERS] mvcc catalo gsnapshots and TopTransactionContext

2014-02-02 Thread Andres Freund
On 2014-01-31 16:41:33 -0500, Bruce Momjian wrote: On Mon, Oct 7, 2013 at 03:02:36PM -0400, Robert Haas wrote: On Fri, Oct 4, 2013 at 3:20 PM, Andres Freund and...@2ndquadrant.com wrote: On 2013-10-04 15:15:36 -0400, Robert Haas wrote: Andres, are you (or is anyone) going to try to

[HACKERS] slow startup due to LWLockAssign() spinlock

2014-02-02 Thread Andres Freund
Hi, On larger, multi-socket, machines, startup takes a fair bit of time. As I was profiling anyway I looked into it and noticed that just about all of it is spent in LWLockAssign() called by InitBufferPool(). Starting with shared_buffers=48GB on the server Nate Boley provided, takes about 12

Re: [HACKERS] Recovery inconsistencies, standby much larger than primary

2014-02-02 Thread Greg Stark
I've poked at this a bit more. There are at least 10 relations where the last block doesn't match the block mentioned in the xlog record that its LSN indicates. At least it looks like from the info xlogdump prints. Including two blocks where the correct block has the same LSN which maybe means

Re: [HACKERS] mvcc catalo gsnapshots and TopTransactionContext

2014-02-02 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-01-31 16:41:33 -0500, Bruce Momjian wrote: Is there any plan to commit this? IMO it has to be applied. Tom objected on the grounds that cache invalidation has to be fixed properly but that's a major restructuring of code that worked this

Re: [HACKERS] Recovery inconsistencies, standby much larger than primary

2014-02-02 Thread Greg Stark
Hm, I'm not entirely convinced those are erroneous replays to wrong blocks. They don't look right but there are no blocks of NULs preceding them. So if they're wrong then they only extended the relations by a single block. The relfilenodes that have nul blocks before the last block are:

Re: [HACKERS] Recovery inconsistencies, standby much larger than primary

2014-02-02 Thread Tom Lane
Greg Stark st...@mit.edu writes: The relfilenodes that have nul blocks before the last block are: Can we see the associated WAL records (ie, the ones matching the LSNs in the last blocks of these files)? regards, tom lane -- Sent via pgsql-hackers mailing list

Re: [HACKERS] mvcc catalo gsnapshots and TopTransactionContext

2014-02-02 Thread Andres Freund
On February 2, 2014 5:52:22 PM CET, Tom Lane t...@sss.pgh.pa.us wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-01-31 16:41:33 -0500, Bruce Momjian wrote: Is there any plan to commit this? IMO it has to be applied. Tom objected on the grounds that cache invalidation has to be

Re: [HACKERS] Recovery inconsistencies, standby much larger than primary

2014-02-02 Thread Greg Stark
On Sun, Feb 2, 2014 at 6:03 PM, Tom Lane t...@sss.pgh.pa.us wrote: Greg Stark st...@mit.edu writes: The relfilenodes that have nul blocks before the last block are: Can we see the associated WAL records (ie, the ones matching the LSNs in the last blocks of these files)? Sorry, I've lost

Re: [HACKERS] mvcc catalo gsnapshots and TopTransactionContext

2014-02-02 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On February 2, 2014 5:52:22 PM CET, Tom Lane t...@sss.pgh.pa.us wrote: More to the point, changing the Assert so it doesn't fire doesn't do one damn thing to ameliorate the fact that cache reload during transaction abort is wrong and unsafe. And,

Re: [HACKERS] [COMMITTERS] pgsql: Include planning time in EXPLAIN ANALYZE output.

2014-02-02 Thread Peter Geoghegan
On Sun, Feb 2, 2014 at 8:13 AM, Tom Lane t...@sss.pgh.pa.us wrote: Perhaps s/Total runtime/Execution time/ ? +1 -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [COMMITTERS] pgsql: Include planning time in EXPLAIN ANALYZE output.

2014-02-02 Thread Gavin Flower
On 03/02/14 09:44, Peter Geoghegan wrote: On Sun, Feb 2, 2014 at 8:13 AM, Tom Lane t...@sss.pgh.pa.us wrote: Perhaps s/Total runtime/Execution time/ ? +1 If the planning was ever made into a parallel process, then 'elapsed time' would be less than the 'processor time'. So what does

[HACKERS] CacheInvalidateRelcache in btree is a crummy idea

2014-02-02 Thread Tom Lane
While investigating the assertion failure Noah presented at http://www.postgresql.org/message-id/20130805170931.ga369...@tornado.leadboat.com I was confused for a bit as to why we're getting a relcache rebuild request for t1's index at exit of the subtransaction, when the subtransaction hasn't

Re: [HACKERS] [COMMITTERS] pgsql: Include planning time in EXPLAIN ANALYZE output.

2014-02-02 Thread Tom Lane
Gavin Flower gavinflo...@archidevsys.co.nz writes: Can I assume: 'Total runtime' is 'elapsed time' and 'Execution time' is 'processor time'. No. It's going to be elapsed time, either way. In a parallel implementation, one would likely want both. When and if we have that, we can argue

Re: [HACKERS] pg_basebackup and pg_stat_tmp directory

2014-02-02 Thread Peter Eisentraut
On 2/2/14, 10:23 AM, Fujii Masao wrote: I'm thinking to change basebackup.c so that it compares the name of the directory that it's trying to back up and the setting value of log_directory parameter, then, if they are the same, it just skips the directory. The patch that I sent upthread does

Re: [HACKERS] mvcc catalo gsnapshots and TopTransactionContext

2014-02-02 Thread Andres Freund
On 2014-02-02 15:16:45 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On February 2, 2014 5:52:22 PM CET, Tom Lane t...@sss.pgh.pa.us wrote: More to the point, changing the Assert so it doesn't fire doesn't do one damn thing to ameliorate the fact that cache reload

Re: [HACKERS] GSOC13 proposal - extend RETURNING syntax

2014-02-02 Thread David Fetter
On Wed, Aug 21, 2013 at 08:52:25PM +0200, Karol Trzcionka wrote: W dniu 21.08.2013 19:17, Boszormenyi Zoltan pisze: With this fixed, a more complete review: Thanks. I've done some syntactic and white space cleanup, here attached. Karol, would you care to help with commenting the sections

Re: [HACKERS] Recovery inconsistencies, standby much larger than primary

2014-02-02 Thread Tom Lane
Greg Stark st...@mit.edu writes: On Sun, Feb 2, 2014 at 6:03 PM, Tom Lane t...@sss.pgh.pa.us wrote: Can we see the associated WAL records (ie, the ones matching the LSNs in the last blocks of these files)? Sorry, I've lost track of what information I already shared or didn't, Hm. So one of

Re: [HACKERS] GIN improvements part2: fast scan

2014-02-02 Thread Tomas Vondra
On 2.2.2014 11:45, Heikki Linnakangas wrote: On 01/30/2014 01:53 AM, Tomas Vondra wrote: (3) A file with explain plans for 4 queries suffering ~2x slowdown, and explain plans with 9.4 master and Heikki's patches is available here: http://www.fuzzy.cz/tmp/gin/queries.txt

[HACKERS] Multiple calls to json_array_elements slow down nonlinearly

2014-02-02 Thread Craig Ringer
Hi all I ran into this performance report over the weekend: http://stackoverflow.com/q/21507127/398670 and wanted to mention it here. json_array_elements seems to spend about 97% of its time in MemoryContextReset(...). Given dummy data: test= create table g as select (select

Re: [HACKERS] Making strxfrm() blobs in indexes work

2014-02-02 Thread Peter Geoghegan
On Thu, Jan 30, 2014 at 8:51 PM, Peter Geoghegan p...@heroku.com wrote: I've done some more digging. It turns out that the 1977 paper An Encoding Method for Multifield Sorting and Indexing describes a technique that involves concatenating multiple column values and comparing them using a

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-02 Thread Craig Ringer
On 02/02/2014 09:03 AM, Tom Lane wrote: According to the buildfarm database, narwhal is running a gcc build on Windows 2003. That hardly seems like a mainstream use case. I could believe that it might be of interest to developers, but clearly no developers are actually running such a build.

Re: [HACKERS] pg_basebackup and pg_stat_tmp directory

2014-02-02 Thread Fujii Masao
On Mon, Feb 3, 2014 at 6:57 AM, Peter Eisentraut pete...@gmx.net wrote: On 2/2/14, 10:23 AM, Fujii Masao wrote: I'm thinking to change basebackup.c so that it compares the name of the directory that it's trying to back up and the setting value of log_directory parameter, then, if they are the

Re: [HACKERS] bugfix patch for json_array_elements

2014-02-02 Thread Craig Ringer
On 02/03/2014 09:09 AM, Craig Ringer wrote: At a guess, we're looking at a case where a new child context is created at every call, so every MemoryContextResetChildren call has to deal with more child contexts. That would be yes. After a short run, I see 32849 lines like: json_array_elements

Re: [HACKERS] plpgsql.warn_shadow

2014-02-02 Thread Marko Tiikkaja
Hi everyone, Here's a new version of the patch. Added new tests and docs and changed the GUCs per discussion. plpgsql.warnings_as_errors only affects compilation at CREATE FUNCTION time: =# set plpgsql.warnings to 'all'; SET =#* set plpgsql.warnings_as_errors to true; SET =#* select

Re: [HACKERS] WITH ORDINALITY planner improvements

2014-02-02 Thread Etsuro Fujita
(2014/02/01 8:01), Tom Lane wrote: Bruce Momjian br...@momjian.us writes: On Thu, Aug 15, 2013 at 07:25:17PM +0900, Etsuro Fujita wrote: Attached is an updated version of the patch. In that version the code for the newly added function build_function_pathkeys() has been made more simple by

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-02 Thread Ashesh Vashi
On Mon, Feb 3, 2014 at 6:52 AM, Craig Ringer cr...@2ndquadrant.com wrote: On 02/02/2014 09:03 AM, Tom Lane wrote: According to the buildfarm database, narwhal is running a gcc build on Windows 2003. That hardly seems like a mainstream use case. I could believe that it might be of

Re: [HACKERS] [PATCH] pg_basebackup: progress report max once per second

2014-02-02 Thread Sawada Masahiko
On Sat, Feb 1, 2014 at 8:29 PM, Oskari Saarenmaa o...@ohmu.fi wrote: 31.01.2014 10:59, Sawada Masahiko kirjoitti: I think the idea in the new progress_report() call (with force == true) is to make sure that there is at least one progress_report call that actually writes the progress report.

Re: [HACKERS] inherit support for foreign tables

2014-02-02 Thread Etsuro Fujita
(2014/01/31 9:56), Robert Haas wrote: On Thu, Jan 30, 2014 at 5:05 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Thu, Jan 30, 2014 at 11:04 AM, Tom Lane t...@sss.pgh.pa.us wrote: I think this is totally misguided. Who's to say that some weird FDW might

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-02 Thread Craig Ringer
On 02/03/2014 11:10 AM, Ashesh Vashi wrote: On Mon, Feb 3, 2014 at 6:52 AM, Craig Ringer cr...@2ndquadrant.com mailto:cr...@2ndquadrant.com wrote: On 02/02/2014 09:03 AM, Tom Lane wrote: According to the buildfarm database, narwhal is running a gcc build on Windows 2003. That

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-02 Thread Amit Kapila
On Mon, Feb 3, 2014 at 6:52 AM, Craig Ringer cr...@2ndquadrant.com wrote: On 02/02/2014 09:03 AM, Tom Lane wrote: According to the buildfarm database, narwhal is running a gcc build on Windows 2003. That hardly seems like a mainstream use case. I could believe that it might be of interest to

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-02 Thread Amit Kapila
On Sun, Feb 2, 2014 at 6:33 AM, Tom Lane t...@sss.pgh.pa.us wrote: I happened to notice today that the owner of buildfarm member narwhal is trying to revive it after a long time offline, but it's failing in the 9.3 branch (and not attempting to build HEAD, yet). The cause appears to be that

[HACKERS] Re: [review] PostgreSQL Service on Windows does not start if data directory given is relative path

2014-02-02 Thread Rajeev rastogi
On 1st February 2014, MauMau Wrote: I reviewed the patch content. I find this fix useful. I'd like to suggest some code improvements. I'll apply and test the patch when I receive your reply. Thanks for reviewing the patch. (1) I think it is appropriate to place find_my_abs_path() in

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-02 Thread Tom Lane
Amit Kapila amit.kapil...@gmail.com writes: Also as per below link, it seems that PGDLLIMPORT is required for exported global variables. http://msdn.microsoft.com/en-us/library/8fskxacy(v=vs.80).aspx That was what we'd always assumed, but the fact that postgres_fdw has been working for the

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-02 Thread Peter Geoghegan
On Sun, Feb 2, 2014 at 5:22 PM, Craig Ringer cr...@2ndquadrant.com wrote: I'm not a fan of MinGW (gcc) on Windows, it's a right pain. It's also the only open source compiler currently supported for PostgreSQL on Windows - practically the only one available. I don't know about you, but I'm not

Re: [HACKERS] FOR [SHARE|UPDATE] NOWAIT may still block in EvalPlanQualFetch

2014-02-02 Thread Craig Ringer
On 02/01/2014 05:28 AM, Bruce Momjian wrote: On Fri, Aug 2, 2013 at 04:00:03PM +0800, Craig Ringer wrote: FOR SHARE|UPDATE NOWAIT will still block if they have to follow a ctid chain because the call to EvalPlanQualFetch doesn't take a param for noWait, so it doesn't know not to block if the

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2014-02-02 Thread Amit Kapila
On Wed, Jan 15, 2014 at 2:43 AM, Simon Riggs si...@2ndquadrant.com wrote: On 8 January 2014 08:33, Simon Riggs si...@2ndquadrant.com wrote: Patch attached, implemented to reduce writes by SELECTs only. This is really a valuable improvement over current SELECT behaviour w.r.t Writes. While

Re: [HACKERS] GIN improvements part2: fast scan

2014-02-02 Thread Oleg Bartunov
Tomasa, it'd be nice if you use real data in your testing. One very good application of gin fast-scan is dramatic performance improvement of hstore/jsonb @ operator, see slides 57, 58 http://www.sai.msu.su/~megera/postgres/talks/hstore-dublin-2013.pdf. I'd like not to lost this benefit :) Oleg

Re: [HACKERS] patch: option --if-exists for pg_dump

2014-02-02 Thread Jeevan Chalke
Hi Peter, I am not sure why you getting build unstable due to white-space errors. Are you referring to these line ? *11:25:01* src/bin/pg_dump/pg_backup_archiver.c:477: indent with spaces.*11:25:01* +