Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2014-09-26 Thread Jeff Janes
On Fri, Sep 26, 2014 at 11:47 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Gavin Flower wrote: Curious: would it be both feasible and useful to have multiple workers process a 'large' table, without complicating things too much? The could each start at a different position in the

[HACKERS] trivial patch for dynahash logging

2014-09-27 Thread Jeff Janes
under HASH_STATISTICS, the output reporting this HTAB upon destruction is pretty useless. Which HTAB would this one be? It is not necessarily the most recently created one. This makes it output the %p to the actual HTAB, so it can be matched up with the logging of the creation. I'm not sure why

Re: [HACKERS] trivial patch for dynahash logging

2014-09-29 Thread Jeff Janes
On Sat, Sep 27, 2014 at 3:09 PM, Tom Lane t...@sss.pgh.pa.us wrote: Jeff Janes jeff.ja...@gmail.com writes: under HASH_STATISTICS, the output reporting this HTAB upon destruction is pretty useless. Which HTAB would this one be? It is not necessarily the most recently created one

Re: [HACKERS] Log notice that checkpoint is to be written on shutdown

2014-10-04 Thread Jeff Janes
On Thu, Oct 2, 2014 at 6:21 AM, Michael Banck michael.ba...@credativ.de wrote: Hi, we have seen repeatedly that users can be confused about why PostgreSQL is not shutting down even though they requested it. Usually, this is because `log_checkpoints' is not enabled and the final checkpoint

Re: [HACKERS] Log notice that checkpoint is to be written on shutdown

2014-10-09 Thread Jeff Janes
On Wed, Oct 8, 2014 at 10:52 AM, Michael Banck michael.ba...@credativ.de wrote: Hi, Am Samstag, den 04.10.2014, 15:05 -0500 schrieb Jim Nasby: On 10/4/14, 1:21 PM, Jeff Janes wrote: On Thu, Oct 2, 2014 at 6:21 AM, Michael Banck wrote: we have seen repeatedly that users can

Re: [HACKERS] Expose options to explain? (track_io_timing)

2014-10-09 Thread Jeff Janes
On Thu, Oct 9, 2014 at 10:17 AM, Joshua D. Drake j...@commandprompt.com wrote: Salut! Fellow volunteers, I request assistance in understanding the following: When I explain a query I can get the following information: | I/O Read Time: 0.000, | I/O Write

Re: [HACKERS] Maximum number of WAL files in the pg_xlog directory

2014-10-14 Thread Jeff Janes
On Mon, Oct 13, 2014 at 12:11 PM, Bruce Momjian br...@momjian.us wrote: I looked into this, and came up with more questions. Why is checkpoint_completion_target involved in the total number of WAL segments? If checkpoint_completion_target is 0.5 (the default), the calculation is:

Re: [HACKERS] Buffer Requests Trace

2014-10-15 Thread Jeff Janes
On Wed, Oct 15, 2014 at 6:22 AM, Lucas Lersch lucasler...@gmail.com wrote: So is it a possible normal behavior that running tpcc for 10min only access 50% of the database? Furthermore, is there a guideline of parameters for tpcc (# of warehouses, execution time, operations weight)? I'm not

Re: [HACKERS] Maximum number of WAL files in the pg_xlog directory

2014-10-15 Thread Jeff Janes
On Fri, Aug 8, 2014 at 12:08 AM, Guillaume Lelarge guilla...@lelarge.info wrote: Hi, As part of our monitoring work for our customers, we stumbled upon an issue with our customers' servers who have a wal_keep_segments setting higher than 0. We have a monitoring script that checks the

Re: [HACKERS] pgcrypto: PGP signatures

2014-10-17 Thread Jeff Janes
On Mon, Sep 15, 2014 at 4:37 AM, Marko Tiikkaja ma...@joh.to wrote: I've changed the patch back to ignore signatures when not using the decrypt_verify() functions in the attached. Hi Marko, This patch needs a rebase now that the armor header patch has been committed. Thanks, Jeff

Re: [HACKERS] wal-size limited to 16MB - Performance issue for subsequent backup

2014-10-21 Thread Jeff Janes
On Mon, Oct 20, 2014 at 12:03 PM, jes...@krogh.cc wrote: Hi. One of our production issues is that the system generates lots of wal-files, lots is like 151952 files over the last 24h, which is about 2.4TB worth of WAL files. I wouldn't say that isn't an issue by itself, but the system does

Re: [HACKERS] Autovacuum fails to keep visibility map up-to-date in mostly-insert-only-tables

2014-10-21 Thread Jeff Janes
On Mon, Oct 20, 2014 at 5:46 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-10-20 17:43:26 -0700, Josh Berkus wrote: On 10/20/2014 05:39 PM, Jim Nasby wrote: Or maybe vacuum isn't the right way to handle some of these scenarios. It's become the catch-all for all of this stuff,

Re: [HACKERS] pgcrypto: PGP signatures

2014-10-27 Thread Jeff Janes
On Mon, Oct 20, 2014 at 3:32 PM, Marko Tiikkaja ma...@joh.to wrote: Hi, Here's the rebased patch -- as promised -- in a v7. Hi Marko, Using the same script as for the memory leak, I am getting seg faults using this patch. 24425 2014-10-27 15:42:11.819 PDT LOG: server process (PID

Re: [HACKERS] Let's drop two obsolete features which are bear-traps for novices

2014-11-03 Thread Jeff Janes
On Sat, Nov 1, 2014 at 10:26 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-11-01 10:18:03 -0700, Josh Berkus wrote: On 10/31/2014 03:07 PM, Tom Lane wrote: I don't care one way or the other about the money type, but I will defend hash indexes, especially seeing that we've

Re: [HACKERS] Maximum number of WAL files in the pg_xlog directory

2014-11-03 Thread Jeff Janes
On Wed, Oct 15, 2014 at 1:11 PM, Jeff Janes jeff.ja...@gmail.com wrote: On Fri, Aug 8, 2014 at 12:08 AM, Guillaume Lelarge guilla...@lelarge.info wrote: Hi, As part of our monitoring work for our customers, we stumbled upon an issue with our customers' servers who have a wal_keep_segments

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-03 Thread Jeff Janes
On Mon, Nov 3, 2014 at 2:18 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Robert Haas wrote: [lots] I have fixed all these items in the attached, thanks -- most user-visible change was the pageinspect 1.3 thingy. pg_upgrade from 1.2 works fine now. I also fixed some things Heikki

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-04 Thread Jeff Janes
On Mon, Nov 3, 2014 at 2:18 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: So here's v21. I also attach a partial diff from v20, just in case anyone wants to give it a look. This needs a bump to 1.3, or the extension won't install: contrib/pageinspect/pageinspect.control During crash

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-05 Thread Jeff Janes
On Tue, Nov 4, 2014 at 2:28 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: I was clearly too careless about testing the xlog code --- it had numerous bugs. This version should be a lot better, but there might be problems lurking still as I don't think I covered it all. Let me know if

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-05 Thread Jeff Janes
On Wed, Nov 5, 2014 at 12:54 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Thanks for the updated patch. Now when I run the test program (version with better error reporting attached), it runs fine until I open a psql session and issue: reindex table foo; Then it immediately falls over

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2014-11-10 Thread Jeff Janes
On Wed, Nov 5, 2014 at 8:49 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Thu, Oct 30, 2014 at 5:19 PM, Michael Paquier michael.paqu...@gmail.com wrote: Updated patch is attached. Please find attached an updated patch with the following things changed: - Addition of tab completion

Re: [HACKERS] pgcrypto: PGP signatures

2014-11-11 Thread Jeff Janes
On Sat, Nov 1, 2014 at 7:52 AM, Marko Tiikkaja ma...@joh.to wrote: Hi, I discovered a problem with the lack of MDC handling in the signature info extraction code, so I've fixed that and added a test message. v9 here. Hi Marko, I get a segfault when the length of the message is exactly

Re: [HACKERS] controlling psql's use of the pager a bit more

2014-11-13 Thread Jeff Janes
On Thu, Nov 13, 2014 at 7:52 AM, Andrew Dunstan and...@dunslane.net wrote: I often get annoyed because psql is a bit too aggressive when it decides whether to put output through the pager, and the only way to avoid this is to turn the pager off (in which case your next query might dump many

Re: [HACKERS] controlling psql's use of the pager a bit more

2014-11-13 Thread Jeff Janes
On Thu, Nov 13, 2014 at 8:14 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-11-13 11:09:06 -0500, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: I often get annoyed because psql is a bit too aggressive when it decides whether to put output through the pager, and the

[HACKERS] pg_test_fsync file descriptor leak

2014-11-18 Thread Jeff Janes
The open_datasync code opens the output file as a test to make sure the flags are accepted by the OS, and if it succeeds it then immediately opens the file again with the same flags, overwriting and so leaking the descriptor from the previous open. On Windows MinGW, this prevents the final

[HACKERS] 9.2 recovery/startup problems

2014-11-25 Thread Jeff Janes
Using both 9.2.9 and 9_2_STABLE 9b468bcec15f1ea7433d4, I get a fairly reproducible startup failure. What I was doing is restore a database from a base backup and roll it forward with a recovery.conf until it completes and starts up. Then I truncate an unlogged table and start repopulating it

Re: [HACKERS] 9.2 recovery/startup problems

2014-11-26 Thread Jeff Janes
On Wed, Nov 26, 2014 at 5:06 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Jeff Janes wrote: This is what I get in the log from the attempted restart: PST LOG: database system was interrupted; last known up at 2014-11-25 15:40:33 PST PST LOG: database system was not properly

Re: [HACKERS] 9.2 recovery/startup problems

2014-11-26 Thread Jeff Janes
On Tue, Nov 25, 2014 at 9:30 PM, Jeff Janes jeff.ja...@gmail.com wrote: Using both 9.2.9 and 9_2_STABLE 9b468bcec15f1ea7433d4, I get a fairly reproducible startup failure. What I was doing is restore a database from a base backup and roll it forward with a recovery.conf until it completes

Re: [HACKERS] 9.2 recovery/startup problems with unlogged tables

2014-12-01 Thread Jeff Janes
On Wed, Nov 26, 2014 at 4:13 PM, Jeff Janes jeff.ja...@gmail.com wrote: On Tue, Nov 25, 2014 at 9:30 PM, Jeff Janes jeff.ja...@gmail.com wrote: Using both 9.2.9 and 9_2_STABLE 9b468bcec15f1ea7433d4, I get a fairly reproducible startup failure. What I was doing is restore a database from

Re: [HACKERS] 9.2 recovery/startup problems

2014-12-02 Thread Jeff Janes
On Tue, Dec 2, 2014 at 7:41 AM, Robert Haas robertmh...@gmail.com wrote: On Wed, Nov 26, 2014 at 7:13 PM, Jeff Janes jeff.ja...@gmail.com wrote: If I do a pg_ctl stop -mf, then both files go away. If I do a pg_ctl stop -mi, then neither goes away. It is only with the /sbin/reboot that I

Re: [HACKERS] How about a option to disable autovacuum cancellation on lock conflict?

2014-12-02 Thread Jeff Janes
On Tue, Dec 2, 2014 at 11:12 AM, Josh Berkus j...@agliodbs.com wrote: On 12/02/2014 11:08 AM, Andres Freund wrote: On 2014-12-02 11:02:07 -0800, Josh Berkus wrote: On 12/02/2014 10:35 AM, Alvaro Herrera wrote: If the table is large, the time window for this to happen is large also;

Re: [HACKERS] How about a option to disable autovacuum cancellation on lock conflict?

2014-12-02 Thread Jeff Janes
On Tue, Dec 2, 2014 at 11:41 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-12-02 11:23:31 -0800, Jeff Janes wrote: I think it would be more promising to work on downgrading lock strengths so that fewer things conflict, and it would be not much more work than what you propose

[HACKERS] compiler warnings under MinGW for 9.4

2014-12-08 Thread Jeff Janes
In the past, building under MinGW produced so many warnings that I never bothered to read them. Now most of them have been removed, so the ones that are left might be worth reporting. Using gcc.exe (GCC) 4.6.2 on REL9_4_STABLE eadd80c08ddfc485db84b9af7cca54a0d50ebe6d I get: mingwcompat.c:60:1:

Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-11 Thread Jeff Janes
On Thu, Dec 11, 2014 at 8:03 AM, Tom Lane t...@sss.pgh.pa.us wrote: 2. The amount of pre-release testing we get from people outside the hard-core development crowd seems to be continuing to decrease. We were fortunate that somebody found the JSONB issue before it was too late to do anything

Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-11 Thread Jeff Janes
On Thu, Dec 11, 2014 at 11:40 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 12/11/2014 08:51 PM, Josh Berkus wrote: On 12/11/2014 09:22 AM, Heikki Linnakangas wrote: Perhaps we should change the process so that it is the patch author's responsibility to find a reviewer, and a

Re: [HACKERS] Commitfest problems

2014-12-15 Thread Jeff Janes
On Sat, Dec 13, 2014 at 1:37 AM, Craig Ringer cr...@2ndquadrant.com wrote: On 12/12/2014 06:02 AM, Josh Berkus wrote: Speaking as the originator of commitfests, they were *always* intended to be a temporary measure, a step on the way to something else like continuous integration. I'd

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-12-15 Thread Jeff Janes
On Mon, Dec 8, 2014 at 8:16 PM, Peter Geoghegan p...@heroku.com wrote: Attached revision, v1.6, slightly tweaks the ordering of per-statement trigger execution. The ordering is now explicitly documented (the html mirror has been updated:

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-12-16 Thread Jeff Janes
On Mon, Dec 15, 2014 at 5:05 PM, Peter Geoghegan p...@heroku.com wrote: Maybe IGNORE is defined as a macro in MinGW? Try s/IGNORE/IGNORE_P/g throughout the patch. BTW, the gcc -E flag does this. So figure out what exact arguments MinGW's gcc is passed in the ordinary course of compiling

Re: [HACKERS] advance local xmin more aggressively

2014-12-16 Thread Jeff Janes
On Wed, Dec 10, 2014 at 3:46 PM, Robert Haas robertmh...@gmail.com wrote: On Wed, Dec 10, 2014 at 3:28 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Care to code it up? Here you are. That was quick. You need to add a semicolon to the end of line 20 in pairingheap.c. In

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-12-18 Thread Jeff Janes
On Mon, Dec 15, 2014 at 11:06 PM, Peter Geoghegan p...@heroku.com wrote: On Mon, Dec 15, 2014 at 4:59 PM, Peter Geoghegan p...@heroku.com wrote: On Mon, Dec 15, 2014 at 4:22 PM, Jeff Janes jeff.ja...@gmail.com wrote: Also, in both Linux and MinGW under option 1 patch I get an OID conflict

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-12-27 Thread Jeff Janes
On Tue, Dec 23, 2014 at 11:55 AM, Peter Geoghegan p...@heroku.com wrote: On Thu, Dec 18, 2014 at 9:20 AM, Jeff Janes jeff.ja...@gmail.com wrote: I've put this through an adaptation of my usual torture test, and it ran fine until wraparound shutdown. I'll poke at it more later. Could you

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-12-29 Thread Jeff Janes
On Mon, Dec 29, 2014 at 9:12 PM, Peter Geoghegan p...@heroku.com wrote: On Mon, Dec 29, 2014 at 2:29 PM, Jeff Janes jeff.ja...@gmail.com wrote: Using the vallock2 version of V1.8, using the test I previously described, I get some all-null rows, which my code should never create. Also

Re: [HACKERS] 9.5: Better memory accounting, towards memory-bounded HashAgg

2014-12-30 Thread Jeff Janes
that still resulted in an external sort were slower than the 10MB setting. I was surprised by this too, but it has been independently reported by Jeff Janes. I don't recall (at the moment) seeing our external sort actually faster than quick-sort, but I've very reliably seen external sorts get faster

Re: [HACKERS] Maximum number of WAL files in the pg_xlog directory

2014-12-30 Thread Jeff Janes
On Tue, Dec 30, 2014 at 12:35 AM, Guillaume Lelarge guilla...@lelarge.info wrote: Sorry for my very late answer. It's been a tough month. 2014-11-27 0:00 GMT+01:00 Bruce Momjian br...@momjian.us: On Mon, Nov 3, 2014 at 12:39:26PM -0800, Jeff Janes wrote: It looked to me that the formula

Re: [HACKERS] replicating DROP commands across servers

2015-01-02 Thread Jeff Janes
On Mon, Dec 29, 2014 at 2:15 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Here's a patch that tweaks the grammar to use TypeName in COMMENT, SECURITY LABEL, and DROP for the type and domain cases. The required changes in the code are pretty minimal, thankfully. Note the slight changes

Re: [HACKERS] hung backends stuck in spinlock heavy endless loop

2015-01-23 Thread Jeff Janes
On Thu, Jan 22, 2015 at 1:50 PM, Merlin Moncure mmonc...@gmail.com wrote: So far, the 'nasty' damage seems to generally if not always follow a checksum failure and the checksum failures are always numerically adjacent. For example: [cds2 12707 2015-01-22 12:51:11.032 CST 2754]WARNING:

Re: [HACKERS] pgbench -f and vacuum

2015-02-11 Thread Jeff Janes
On Tue, Dec 23, 2014 at 7:42 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Robert Haas wrote: On Mon, Dec 22, 2014 at 6:55 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Here's a completely different idea. How about we add an option that means vacuum this table before running

Re: [HACKERS] Parallel Seq Scan

2015-01-29 Thread Jeff Janes
On Tue, Jan 27, 2015 at 11:08 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 01/28/2015 04:16 AM, Robert Haas wrote: On Tue, Jan 27, 2015 at 6:00 PM, Robert Haas robertmh...@gmail.com wrote: Now, when you did what I understand to be the same test on the same machine, you got

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-28 Thread Jeff Janes
On Wed, Jan 28, 2015 at 1:13 PM, Jim Nasby jim.na...@bluetreble.com wrote: My $0.01: While I sympathize with Noah's sentiments, the only thing that makes sense to me is that a JSON text field is treated the same way as we treat text. Right now, that means NUL is not allowed, period. If no

[HACKERS]

2015-01-30 Thread jeff . janes
From nobody Fri Jan 30 18:20:23 2015 Content-Type: text/plain; charset=utf-8 MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: Re: Parallel Seq Scan To: pgsql-hackers@postgresql.org From: Jeff Janes jeff.ja...@gmail.com Date: Fri, 30 Jan 2015 18:20:23 + User-Agent: pgcommitfest X

Re: [HACKERS] replicating DROP commands across servers

2015-01-05 Thread Jeff Janes
On Fri, Jan 2, 2015 at 9:59 PM, Jeff Janes jeff.ja...@gmail.com wrote: On Mon, Dec 29, 2014 at 2:15 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Here's a patch that tweaks the grammar to use TypeName in COMMENT, SECURITY LABEL, and DROP for the type and domain cases. The required

[HACKERS] To do for psql to show installable extensions

2015-01-12 Thread Jeff Janes
I'd like to propose a wiki to-do item for a backslash command in psql which would show all installable extensions, basically just a wrapper around 'select * from pg_available_extensions'. I've wanted it a few times recently, mostly in testing. Any reason this wouldn't be desirable? What should

Re: [HACKERS] PATCH: numeric timestamp in log_line_prefix

2015-03-22 Thread Jeff Janes
On Sat, Mar 21, 2015 at 4:47 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: Hi, from time to time I need to correlate PostgreSQL logs to other logs, containing numeric timestamps - a prime example of that is pgbench. With %t and %m that's not quite trivial, because of timezones etc.

Re: [HACKERS] Redesigning checkpoint_segments

2015-03-17 Thread Jeff Janes
On Mon, Feb 23, 2015 at 8:56 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Everyone seems to be happy with the names and behaviour of the GUCs, so committed. The docs suggest that max_wal_size will be respected during archive recovery (causing restartpoints and recycling), but I'm

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-21 Thread Jeff Janes
On Sat, Mar 21, 2015 at 8:54 AM, Tom Lane t...@sss.pgh.pa.us wrote: Stephen Frost sfr...@snowman.net writes: At the moment, one could look at our default postgresql.conf and the turns forced synchronization on or off and think it's something akin or somehow related to synchronous_commit

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2015-03-09 Thread Jeff Janes
On Wed, Dec 17, 2014 at 12:39 AM, Simon Riggs si...@2ndquadrant.com wrote: On 15 December 2014 at 20:26, Jeff Janes jeff.ja...@gmail.com wrote: I still get the compiler error in contrib: pgstattuple.c: In function 'pgstat_heap': pgstattuple.c:279: error: too few arguments to function

Re: [HACKERS] Documentation of bt_page_items()'s ctid field

2015-03-09 Thread Jeff Janes
On Tue, Dec 30, 2014 at 8:59 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 12/30/2014 04:08 AM, Peter Geoghegan wrote: Attached documentation patch describes the purpose of bt_page_items()'s ctid field. This has come up enough times in disaster recovery or testing scenarios that

Re: [HACKERS] Documentation of bt_page_items()'s ctid field

2015-03-09 Thread Jeff Janes
On Mon, Mar 9, 2015 at 4:06 PM, Peter Geoghegan p...@heroku.com wrote: On Mon, Mar 9, 2015 at 3:51 PM, Jeff Janes jeff.ja...@gmail.com wrote: How do I know if I am looking at a non-rightmost page? It has a right-link (that's the easiest way to tell). Meaning that btpo_next is not zero

Re: [HACKERS] Documentation of bt_page_items()'s ctid field

2015-03-11 Thread Jeff Janes
On Mon, Mar 9, 2015 at 5:34 PM, Peter Geoghegan p...@heroku.com wrote: On Mon, Mar 9, 2015 at 5:18 PM, Jeff Janes jeff.ja...@gmail.com wrote: It has a right-link (that's the easiest way to tell). Meaning that btpo_next is not zero? Should we say that in the patch in so many words? I

Re: [HACKERS] procost for to_tsvector

2015-03-11 Thread Jeff Janes
On Wed, Mar 11, 2015 at 2:54 PM, Andrew Gierth and...@tao11.riddles.org.uk wrote: Seq Scan on comments (cost=0.00..2406.18 rows=4140 width=792) (actual time=0.601..3946.589 rows=4056 loops=1) Bitmap Heap Scan on comments (cost=204.09..2404.30 rows=4140 width=792) (actual

Re: [HACKERS] logical column ordering

2015-03-13 Thread Jeff Janes
On Mon, Feb 23, 2015 at 3:09 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: Hi, attached is the result of my first attempt to make the logical column ordering patch work. This touches a lot of code in the executor that is mostly new to me, so if you see something that looks like an

Re: [HACKERS] New CF app deployment

2015-03-13 Thread Jeff Janes
On Sun, Feb 22, 2015 at 12:13 PM, Magnus Hagander mag...@hagander.net wrote: On Sun, Feb 15, 2015 at 9:46 PM, Peter Geoghegan p...@heroku.com wrote: On Sun, Feb 15, 2015 at 4:59 PM, Peter Eisentraut pete...@gmx.net wrote: I think the old system where the patch submitter declared, this

Re: [HACKERS] Patch to add functionality to specify ORDER BY in CREATE FUNCTION for SRFs

2015-03-13 Thread Jeff Janes
On Mon, Jan 5, 2015 at 7:12 AM, Atri Sharma atri.j...@gmail.com wrote: Hi All, Please forgive if this is a repost. Please find attached patch for supporting ORDER BY clause in CREATE FUNCTION for SRFs. Hi Atri, From the discussion, I don't know if this patch is still being proposed. If

[HACKERS] compute_index_stats is missing a CHECK_FOR_INTERRUPTS

2015-03-28 Thread Jeff Janes
Analyze on functional indexes cannot be interrupted very easily. Example: create language plperl; create table foo1 as select x::text from generate_series(1,1000) foo (x); create table foo2 as select reverse(x) from foo1; --use a fast version to set up the demo, as we are impatient CREATE or

Re: [HACKERS] compute_index_stats is missing a CHECK_FOR_INTERRUPTS

2015-03-28 Thread Jeff Janes
On Sat, Mar 28, 2015 at 3:37 PM, Tom Lane t...@sss.pgh.pa.us wrote: Jeff Janes jeff.ja...@gmail.com writes: Analyze on functional indexes cannot be interrupted very easily. ... The attached patch fixes it, but don't vouch for its safety. Hm. The other per-sample-row loops in analyze.c

Re: [HACKERS] proposal: row_to_array function

2015-03-28 Thread Jeff Janes
On Tue, Jan 27, 2015 at 10:58 AM, Pavel Stehule pavel.steh...@gmail.com wrote: Hi 2015-01-27 11:41 GMT+01:00 Pavel Stehule pavel.steh...@gmail.com: 2015-01-26 21:44 GMT+01:00 Jim Nasby jim.na...@bluetreble.com: On 1/25/15 4:23 AM, Pavel Stehule wrote: I tested a concept iteration over

Re: [HACKERS] vac truncation scan problems

2015-03-31 Thread Jeff Janes
On Mon, Mar 30, 2015 at 8:54 PM, Jeff Janes jeff.ja...@gmail.com wrote: After freeing up the rows at the end of the table so it is eligible for truncation, then running a manual VACUUM to actually release the space, I kept running into the problem that the truncation scan was consistently

Re: [HACKERS] vac truncation scan problems

2015-03-31 Thread Jeff Janes
On Tue, Mar 31, 2015 at 1:28 AM, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: Hi, this is a bug in the commit 0d831389749a3baaced7b984205b9894a82444b9 . It allows vucuum freeze to be skipped and inversely lets regular vacuum wait for lock. The attched patch fixes it. In

Re: [HACKERS] vac truncation scan problems

2015-03-31 Thread Jeff Janes
On Tue, Mar 31, 2015 at 1:29 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Tue, Mar 31, 2015 at 3:42 PM, Jeff Janes jeff.ja...@gmail.com wrote: On Mon, Mar 30, 2015 at 8:54 PM, Jeff Janes jeff.ja...@gmail.com wrote: After freeing up the rows at the end of the table so

Re: [HACKERS] Table-level log_autovacuum_min_duration

2015-03-23 Thread Jeff Janes
On Mon, Mar 23, 2015 at 7:07 AM, Tom Lane t...@sss.pgh.pa.us wrote: Alvaro Herrera alvhe...@2ndquadrant.com writes: Michael Paquier wrote: So a worker does not see changes in postgresql.conf once it is run and processes a database, no? The launcher does run ProcessConfigFile() when

Re: [HACKERS] Comparing primary/HS standby in tests

2015-03-04 Thread Jeff Janes
On Tue, Mar 3, 2015 at 7:49 AM, Andres Freund and...@2ndquadrant.com wrote: Hi, I've regularly wished we had automated tests that setup HS and then compare primary/standby at the end to verify replay worked correctly. Heikki's page comparison tools deals with some of that verification, but

Re: [HACKERS] Remove fsync ON/OFF as a visible option?

2015-03-25 Thread Jeff Janes
On Wed, Mar 25, 2015 at 12:45 PM, Jim Nasby jim.na...@bluetreble.com wrote: I see 3 settings that allow people to accidentally shoot themselves in the foot; fsync, wal_sync_method and full_page_writes. How about just grouping those 3 together with a bulk disclaimer along the lines of The

[HACKERS] compiler warnings in lwlock

2015-03-25 Thread Jeff Janes
When building with LOCK_DEBUG but without casserts, I was getting unused variable warnings. I believe this is the correct way to silence them. Cheers, Jeff silence_lwlock_lock_debug.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] logical column ordering

2015-03-23 Thread Jeff Janes
On Mon, Mar 23, 2015 at 10:01 AM, Robert Haas robertmh...@gmail.com wrote: On Thu, Mar 12, 2015 at 9:57 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: However, there's a difference between making a query silently given different results, and breaking it completely forcing the user to

Re: [HACKERS] improving speed of make check-world

2015-04-23 Thread Jeff Janes
On Thu, Aug 14, 2014 at 10:45 PM, Peter Eisentraut pete...@gmx.net wrote: make check-world creates a temporary installation in every subdirectory it runs a test in, which is stupid: it's very slow and uses a lot of disk space. It's enough to do this once per run. That is the essence of what

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2015-04-20 Thread Jeff Janes
On Mon, Sep 29, 2014 at 2:13 AM, Andres Freund and...@anarazel.de wrote: On 2014-09-28 19:51:36 +0100, Simon Riggs wrote: On 27 September 2014 09:29, Andres Freund and...@anarazel.de wrote: On 2014-09-27 10:23:33 +0300, Heikki Linnakangas wrote: This patch has gotten a fair amount of

[HACKERS] optimizing vacuum truncation scans

2015-04-19 Thread Jeff Janes
After a large bulk load aborted near the end, I decided to vacuum the main table so as to not leave a huge chunk of free space in the middle of it, before re-running the bulk load. This vacuum took a frustratingly long time, as the backwards scan over the table to truncate the space did not

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-04-19 Thread Jeff Janes
On Tue, Apr 14, 2015 at 11:45 PM, Jeff Janes jeff.ja...@gmail.com wrote: On Tue, Mar 31, 2015 at 12:02 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: Hi all, attached is v4 of the patch implementing adaptive ndistinct estimator. Hi Tomas, I have a case here where the adaptive

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2015-04-20 Thread Jeff Janes
On Mon, Apr 20, 2015 at 10:33 AM, Bruce Momjian br...@momjian.us wrote: On Thu, Apr 16, 2015 at 03:41:54PM +0100, Simon Riggs wrote: That is how we arrive at the idea of a cleanup limit, further enhanced by a limit that applies only to dirtying clean blocks, which we have 4? recent votes

Re: [HACKERS] optimizing vacuum truncation scans

2015-04-20 Thread Jeff Janes
On Sun, Apr 19, 2015 at 10:38 PM, Jim Nasby jim.na...@bluetreble.com wrote: On 4/19/15 9:09 PM, Jeff Janes wrote: I did literally the simplest thing I could think of as a proof of concept patch, to see if it would actually fix things. I just jumped back a certain number of blocks

Re: [HACKERS] adding more information about process(es) cpu and memory usage

2015-04-23 Thread Jeff Janes
On Thu, Apr 23, 2015 at 10:17 AM, Heikki Linnakangas hlinn...@iki.fi wrote: On 04/23/2015 08:00 PM, Radovan Jablonovsky wrote: During current encounters with amazon web services - RDS, the DBA does not have access to OS/linux shell of underlying instance. That render some postgresql

Re: [HACKERS] [PATCH] Add transforms feature

2015-04-28 Thread Jeff Janes
On Tue, Apr 7, 2015 at 7:55 PM, Peter Eisentraut pete...@gmx.net wrote: On 3/22/15 5:46 AM, Pavel Stehule wrote: Isn't better doesn't support TRANSFORM ALL clause? If somebody would to use transformations - then he have to explicitly enable it by TRANSFORM FOR TYPE ? It is safe and without

Re: [HACKERS] WIP: multivariate statistics / proof of concept

2015-04-28 Thread Jeff Janes
On Mon, Mar 30, 2015 at 5:26 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: Hello, attached is a new version of the patch series. Aside from fixing various issues (crashes, memory leaks). The patches are rebased to current master, and I also attach a few SQL scripts I used for testing

Re: [HACKERS] Feedback on getting rid of VACUUM FULL

2015-04-29 Thread Jeff Janes
On Tue, Apr 28, 2015 at 11:32 AM, Robert Haas robertmh...@gmail.com wrote: On Fri, Apr 24, 2015 at 3:04 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: I think what we need here is something that does heap_update to tuples at the end of the table, moving them to earlier pages; then wait

Re: [HACKERS] improving speed of make check-world

2015-04-27 Thread Jeff Janes
On Sat, Apr 25, 2015 at 7:23 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Sat, Apr 25, 2015 at 7:59 AM, Peter Eisentraut pete...@gmx.net wrote: On 4/23/15 1:22 PM, Jeff Janes wrote: Something about this commit (dcae5faccab64776376d354d) broke make check in parallel conditions

Re: [HACKERS] WIP: multivariate statistics / proof of concept

2015-04-28 Thread Jeff Janes
On Tue, Apr 28, 2015 at 9:13 AM, Stephen Frost sfr...@snowman.net wrote: * Jeff Janes (jeff.ja...@gmail.com) wrote: On Mon, Mar 30, 2015 at 5:26 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: attached is a new version of the patch series. Aside from fixing various issues

Re: [HACKERS] is possible to upgrade from 9.2 to 9.4 with pg_upgrade

2015-05-06 Thread Jeff Janes
On Wed, May 6, 2015 at 6:16 AM, Pavel Stehule pavel.steh...@gmail.com wrote: 2015-05-06 15:15 GMT+02:00 Alvaro Herrera alvhe...@2ndquadrant.com: Pavel Stehule wrote: Hi I am working on preparation the migration from 9.2 to 9.4 pg_upgrade fails pg_upgrade -b

Re: [HACKERS] is possible to upgrade from 9.2 to 9.4 with pg_upgrade

2015-05-06 Thread Jeff Janes
On Wed, May 6, 2015 at 10:26 AM, Jeff Janes jeff.ja...@gmail.com wrote: On Wed, May 6, 2015 at 6:16 AM, Pavel Stehule pavel.steh...@gmail.com wrote: 2015-05-06 15:15 GMT+02:00 Alvaro Herrera alvhe...@2ndquadrant.com: Pavel Stehule wrote: Hi I am working on preparation the migration

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-05-13 Thread Jeff Janes
On Thu, Apr 30, 2015 at 6:20 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: Hi, On 04/30/15 22:57, Robert Haas wrote: On Tue, Mar 31, 2015 at 3:02 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: attached is v4 of the patch implementing adaptive ndistinct estimator. So, I

Re: [HACKERS] WALWriteLock contention

2015-05-15 Thread Jeff Janes
On Fri, May 15, 2015 at 9:06 AM, Robert Haas robertmh...@gmail.com wrote: WALWriteLock contention is measurable on some workloads. In studying the problem briefly, a couple of questions emerged: ... 2. I don't really understand why WALWriteLock is set up to prohibit two backends from

Re: [HACKERS] WALWriteLock contention

2015-05-18 Thread Jeff Janes
My goal there was to further improve group commit. When running pgbench -j10 -c10, it was common to see fsyncs that alternated between flushing 1 transaction, and 9 transactions. Because the first one to the gate would go through it and slam it on all the others, and it would take one

[HACKERS] pg_upgrade in 9.5 broken for adminpack

2015-04-16 Thread Jeff Janes
pg_upgrade was recently broken for use upgrading from a system with adminpack installed. Breaking commit is: commit 30982be4e5019684e1772dd9170aaa53f5a8e894 Author: Peter Eisentraut pete...@gmx.net Integrate pg_upgrade_support module into backend from pg_upgrade_dump_12870.log

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-04-15 Thread Jeff Janes
On Tue, Mar 31, 2015 at 12:02 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: Hi all, attached is v4 of the patch implementing adaptive ndistinct estimator. Hi Tomas, I have a case here where the adaptive algorithm underestimates ndistinct by a factor of 7 while the default estimator

Re: [HACKERS] TABLESAMPLE patch

2015-04-08 Thread Jeff Janes
On Mon, Apr 6, 2015 at 11:02 AM, Petr Jelinek p...@2ndquadrant.com wrote: On 06/04/15 14:30, Petr Jelinek wrote: On 06/04/15 11:02, Simon Riggs wrote: Are we ready for a final detailed review and commit? I plan to send v12 in the evening with some additional changes that came up from

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-04 Thread Jeff Janes
On Sat, Apr 4, 2015 at 3:10 PM, Jim Nasby jim.na...@bluetreble.com wrote: On 4/3/15 12:59 AM, Sawada Masahiko wrote: + case HEAPTUPLE_LIVE: + case HEAPTUPLE_RECENTLY_DEAD: + case

Re: [HACKERS] Problems with question marks in operators (JDBC, ECPG, ...)

2015-05-20 Thread Jeff Janes
On Fri, May 15, 2015 at 1:23 PM, Dave Cramer p...@fastcrypt.com wrote: On 15 May 2015 at 16:21, Robert Haas robertmh...@gmail.com wrote: On Fri, May 15, 2015 at 4:13 PM, Dave Cramer p...@fastcrypt.com wrote: Not sure what the point of this is: as you indicated the ship has sailed so to

Re: [HACKERS] Problems with question marks in operators (JDBC, ECPG, ...)

2015-05-20 Thread Jeff Janes
On Wed, May 20, 2015 at 11:13 AM, Tom Lane t...@sss.pgh.pa.us wrote: Jeff Janes jeff.ja...@gmail.com writes: What if something like this was made to work? select '{3:5}'::jsonb operator(pg_catalog.?) '3'; (Where the double quotes around the ? would be tolerated, which they currently

Re: [HACKERS] Redesigning checkpoint_segments

2015-05-21 Thread Jeff Janes
On Mon, Mar 16, 2015 at 11:05 PM, Jeff Janes jeff.ja...@gmail.com wrote: On Mon, Feb 23, 2015 at 8:56 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Everyone seems to be happy with the names and behaviour of the GUCs, so committed. The docs suggest that max_wal_size

Re: [HACKERS] xid wrap / optimize frozen tables?

2015-06-03 Thread Jeff Janes
On May 24, 2015 6:42 AM, Nils Goroll sl...@schokola.de wrote: Hi Jeff and all, On 23/05/15 22:13, Jeff Janes wrote: Are you sure it is the read IO that causes the problem? Yes. Trouble is here that we are talking about a 361 GB table List of relations

Re: [HACKERS] xid wrap / optimize frozen tables?

2015-06-04 Thread Jeff Janes
On Wed, Jun 3, 2015 at 2:49 PM, Jeff Janes jeff.ja...@gmail.com wrote: I would not be surprised if it were the reading, not the writing, which caused the performance problem. Of course I screwed up that last sentence. I meant the opposite, it would not surprise me if it were the writing

Re: [HACKERS] Resource Owner reassign Locks

2015-06-07 Thread Jeff Janes
On Thu, Jun 21, 2012 at 5:32 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 18.06.2012 13:59, Heikki Linnakangas wrote: On 10.06.2012 23:39, Jeff Janes wrote: I found the interface between resowner.c and lock.c a bit confusing. resowner.c would sometimes call

<    4   5   6   7   8   9   10   11   12   13   >