Re: [HACKERS] New CF app deployment

2015-02-06 Thread Jeff Janes
On Fri, Feb 6, 2015 at 5:20 AM, Robert Haas wrote: > On Fri, Feb 6, 2015 at 5:51 AM, Magnus Hagander > wrote: > > So in an attempt to actually move this forward in a constructive way I'm > > going to ignore a bunch of what happened after this email, and fork the > > discussion at this point. >

[HACKERS]

2015-01-30 Thread jeff . janes
>From nobody Fri Jan 30 18:20:23 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: Re: Parallel Seq Scan To: pgsql-hackers@postgresql.org From: Jeff Janes Date: Fri, 30 Jan 2015 18:20:23 + User-Agent: pgcommitfest X-cfsend

Re: [HACKERS] Parallel Seq Scan

2015-01-29 Thread Jeff Janes
On Tue, Jan 27, 2015 at 11:08 PM, Heikki Linnakangas < hlinnakan...@vmware.com> wrote: > On 01/28/2015 04:16 AM, Robert Haas wrote: > >> On Tue, Jan 27, 2015 at 6:00 PM, Robert Haas >> wrote: >> >>> Now, when you did what I understand to be the same test on the same >>> machine, you got times ran

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-28 Thread Jeff Janes
On Wed, Jan 28, 2015 at 1:13 PM, Jim Nasby wrote: > > My $0.01: > > While I sympathize with Noah's sentiments, the only thing that makes sense > to me is that a JSON text field is treated the same way as we treat text. > Right now, that means NUL is not allowed, period. > > If no one has bitched

Re: [HACKERS] hung backends stuck in spinlock heavy endless loop

2015-01-23 Thread Jeff Janes
On Thu, Jan 22, 2015 at 1:50 PM, Merlin Moncure wrote: > > So far, the 'nasty' damage seems to generally if not always follow a > checksum failure and the checksum failures are always numerically > adjacent. For example: > > [cds2 12707 2015-01-22 12:51:11.032 CST 2754]WARNING: page > verificat

[HACKERS] To do for psql to show installable extensions

2015-01-12 Thread Jeff Janes
I'd like to propose a wiki to-do item for a backslash command in psql which would show all installable extensions, basically just a wrapper around 'select * from pg_available_extensions'. I've wanted it a few times recently, mostly in testing. Any reason this wouldn't be desirable? What should i

Re: [HACKERS] replicating DROP commands across servers

2015-01-05 Thread Jeff Janes
On Fri, Jan 2, 2015 at 9:59 PM, Jeff Janes wrote: > On Mon, Dec 29, 2014 at 2:15 PM, Alvaro Herrera > wrote: > >> Here's a patch that tweaks the grammar to use TypeName in COMMENT, >> SECURITY LABEL, and DROP for the type and domain cases. The required >> change

Re: [HACKERS] replicating DROP commands across servers

2015-01-02 Thread Jeff Janes
On Mon, Dec 29, 2014 at 2:15 PM, Alvaro Herrera wrote: > Here's a patch that tweaks the grammar to use TypeName in COMMENT, > SECURITY LABEL, and DROP for the type and domain cases. The required > changes in the code are pretty minimal, thankfully. Note the slight > changes to the new object_ad

Re: [HACKERS] Maximum number of WAL files in the pg_xlog directory

2014-12-30 Thread Jeff Janes
On Tue, Dec 30, 2014 at 12:35 AM, Guillaume Lelarge wrote: > Sorry for my very late answer. It's been a tough month. > > 2014-11-27 0:00 GMT+01:00 Bruce Momjian : > >> On Mon, Nov 3, 2014 at 12:39:26PM -0800, Jeff Janes wrote: >> > It looked to me that the

Re: [HACKERS] 9.5: Better memory accounting, towards memory-bounded HashAgg

2014-12-30 Thread Jeff Janes
ns, since higher > work_mem settings that still resulted in an external sort were slower > than the 10MB setting. > > I was surprised by this too, but it has been independently reported by > Jeff Janes. > I don't recall (at the moment) seeing our external sort actually faster t

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-12-29 Thread Jeff Janes
On Mon, Dec 29, 2014 at 9:12 PM, Peter Geoghegan wrote: > On Mon, Dec 29, 2014 at 2:29 PM, Jeff Janes wrote: > > Using the vallock2 version of V1.8, using the test I previously > described, I > > get some all-null rows, which my code should never create. Also, the > inde

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-12-27 Thread Jeff Janes
On Tue, Dec 23, 2014 at 11:55 AM, Peter Geoghegan wrote: > On Thu, Dec 18, 2014 at 9:20 AM, Jeff Janes wrote: > > I've put this through an adaptation of my usual torture test, and it ran > > fine until wraparound shutdown. I'll poke at it more later. > > Could

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-12-18 Thread Jeff Janes
On Mon, Dec 15, 2014 at 11:06 PM, Peter Geoghegan wrote: > > On Mon, Dec 15, 2014 at 4:59 PM, Peter Geoghegan wrote: > > On Mon, Dec 15, 2014 at 4:22 PM, Jeff Janes > wrote: > >> Also, in both Linux and MinGW under option 1 patch I get an OID > conflict on > &g

Re: [HACKERS] advance local xmin more aggressively

2014-12-16 Thread Jeff Janes
On Wed, Dec 10, 2014 at 3:46 PM, Robert Haas wrote: > > On Wed, Dec 10, 2014 at 3:28 PM, Heikki Linnakangas > wrote: > >> Care to code it up? > > > > Here you are. > > That was quick. > > You need to add a semicolon to the end of line 20 in pairingheap.c. > In addition to the semicolon, it doesn

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-12-16 Thread Jeff Janes
On Mon, Dec 15, 2014 at 5:05 PM, Peter Geoghegan wrote: > > > > > Maybe IGNORE is defined as a macro in MinGW? > > Try s/IGNORE/IGNORE_P/g throughout the patch. > > BTW, the gcc -E flag does this. So figure out what exact arguments > MinGW's gcc is passed in the ordinary course of compiling gram.c

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-12-15 Thread Jeff Janes
On Mon, Dec 8, 2014 at 8:16 PM, Peter Geoghegan wrote: > > Attached revision, v1.6, slightly tweaks the ordering of per-statement > trigger execution. The ordering is now explicitly documented (the html > mirror has been updated: > > http://postgres-benchmarks.s3-website-us-east-1.amazonaws.com/on

Re: [HACKERS] GiST kNN search queue (Re: KNN-GiST with recheck)

2014-12-15 Thread Jeff Janes
On Mon, Dec 15, 2014 at 5:08 AM, Heikki Linnakangas wrote: Here's a new version of the patch. It now uses the same pairing heap code > that I posted in the other thread ("advance local xmin more aggressivley", > http://www.postgresql.org/message-id/5488acf0.8050...@vmware.com). The > pairingheap_

Re: [HACKERS] PATCH: decreasing memory needlessly consumed by array_agg

2014-12-15 Thread Jeff Janes
On Sat, Nov 29, 2014 at 8:57 AM, Tomas Vondra wrote: > > Hi, > > Attached is v2 of the patch lowering array_agg memory requirements. > Hopefully it addresses the issues issues mentioned by TL in this thread > (not handling some of the callers appropriately etc.). > Hi Tomas, When configured --wi

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2014-12-15 Thread Jeff Janes
On Thu, Dec 11, 2014 at 7:24 PM, Simon Riggs wrote: > > On 17 November 2014 at 22:08, Simon Riggs wrote: > > On 17 November 2014 21:09, Alvaro Herrera > wrote: > >> What happened to this patch? I'm going over something that could use > >> the concept of "clean some stuff up when reading this pa

Re: [HACKERS] pgbench -f and vacuum

2014-12-15 Thread Jeff Janes
On Sat, Dec 13, 2014 at 7:39 AM, Tom Lane wrote: > > Tatsuo Ishii writes: > > Currently pgbench -f (run custom script) executes vacuum against > > pgbench_* tables before stating bench marking if -n (or --no-vacuum) > > is not specified. If those tables do not exist, pgbench fails. To > > prevent

Re: [HACKERS] Commitfest problems

2014-12-15 Thread Jeff Janes
On Sat, Dec 13, 2014 at 1:37 AM, Craig Ringer wrote: > > On 12/12/2014 06:02 AM, Josh Berkus wrote: > > > > Speaking as the originator of commitfests, they were *always* intended > > to be a temporary measure, a step on the way to something else like > > continuous integration. > > I'd really like

Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-11 Thread Jeff Janes
On Thu, Dec 11, 2014 at 11:40 AM, Heikki Linnakangas < hlinnakan...@vmware.com> wrote: > On 12/11/2014 08:51 PM, Josh Berkus wrote: > >> On 12/11/2014 09:22 AM, Heikki Linnakangas wrote: >> >> >>> Perhaps we should change the process so that it is the patch author's >>> responsibility to find a r

Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-11 Thread Jeff Janes
On Thu, Dec 11, 2014 at 8:03 AM, Tom Lane wrote: > > 2. The amount of pre-release testing we get from people outside the > hard-core development crowd seems to be continuing to decrease. > We were fortunate that somebody found the JSONB issue before it was > too late to do anything about it. Per

[HACKERS] compiler warnings under MinGW for 9.4

2014-12-08 Thread Jeff Janes
In the past, building under MinGW produced so many warnings that I never bothered to read them. Now most of them have been removed, so the ones that are left might be worth reporting. Using gcc.exe (GCC) 4.6.2 on REL9_4_STABLE eadd80c08ddfc485db84b9af7cca54a0d50ebe6d I get: mingwcompat.c:60:1: w

Re: [HACKERS] How about a option to disable autovacuum cancellation on lock conflict?

2014-12-02 Thread Jeff Janes
On Tue, Dec 2, 2014 at 11:41 AM, Andres Freund wrote: > On 2014-12-02 11:23:31 -0800, Jeff Janes wrote: > > I think it would be more promising to work on downgrading lock strengths > so > > that fewer things conflict, and it would be not much more work than what > > you

Re: [HACKERS] How about a option to disable autovacuum cancellation on lock conflict?

2014-12-02 Thread Jeff Janes
On Tue, Dec 2, 2014 at 11:12 AM, Josh Berkus wrote: > On 12/02/2014 11:08 AM, Andres Freund wrote: > > On 2014-12-02 11:02:07 -0800, Josh Berkus wrote: > >> On 12/02/2014 10:35 AM, Alvaro Herrera wrote: > >>> If the table is large, the time window for this to happen is large > also; > >>> there m

Re: [HACKERS] 9.2 recovery/startup problems

2014-12-02 Thread Jeff Janes
On Tue, Dec 2, 2014 at 7:41 AM, Robert Haas wrote: > On Wed, Nov 26, 2014 at 7:13 PM, Jeff Janes wrote: > > If I do a pg_ctl stop -mf, then both files go away. If I do a pg_ctl > stop > > -mi, then neither goes away. It is only with the /sbin/reboot that I get > >

Re: [HACKERS] 9.2 recovery/startup problems with unlogged tables

2014-12-01 Thread Jeff Janes
On Wed, Nov 26, 2014 at 4:13 PM, Jeff Janes wrote: > On Tue, Nov 25, 2014 at 9:30 PM, Jeff Janes wrote: > >> Using both 9.2.9 and 9_2_STABLE 9b468bcec15f1ea7433d4, I get a fairly >> reproducible startup failure. >> >> What I was doing is restore a database

Re: [HACKERS] 9.2 recovery/startup problems

2014-11-26 Thread Jeff Janes
On Tue, Nov 25, 2014 at 9:30 PM, Jeff Janes wrote: > Using both 9.2.9 and 9_2_STABLE 9b468bcec15f1ea7433d4, I get a fairly > reproducible startup failure. > > What I was doing is restore a database from a base backup and roll it > forward with a recovery.conf until it complete

Re: [HACKERS] 9.2 recovery/startup problems

2014-11-26 Thread Jeff Janes
On Wed, Nov 26, 2014 at 5:06 AM, Alvaro Herrera wrote: > Jeff Janes wrote: > > > This is what I get in the log from the attempted restart: > > > > PST LOG: database system was interrupted; last known up at 2014-11-25 > > 15:40:33 PST > > PST LOG: databa

[HACKERS] 9.2 recovery/startup problems

2014-11-25 Thread Jeff Janes
Using both 9.2.9 and 9_2_STABLE 9b468bcec15f1ea7433d4, I get a fairly reproducible startup failure. What I was doing is restore a database from a base backup and roll it forward with a recovery.conf until it completes and starts up. Then I truncate an unlogged table and start repopulating it with

[HACKERS] pg_test_fsync file descriptor leak

2014-11-18 Thread Jeff Janes
The open_datasync code opens the output file as a test to make sure the flags are accepted by the OS, and if it succeeds it then immediately opens the file again with the same flags, overwriting and so leaking the descriptor from the previous open. On Windows MinGW, this prevents the final unlink

Re: [HACKERS] controlling psql's use of the pager a bit more

2014-11-13 Thread Jeff Janes
On Thu, Nov 13, 2014 at 8:14 AM, Andres Freund wrote: > On 2014-11-13 11:09:06 -0500, Tom Lane wrote: > > Andrew Dunstan writes: > > > I often get annoyed because psql is a bit too aggressive when it > decides > > > whether to put output through the pager, and the only way to avoid this > > > is

Re: [HACKERS] controlling psql's use of the pager a bit more

2014-11-13 Thread Jeff Janes
On Thu, Nov 13, 2014 at 7:52 AM, Andrew Dunstan wrote: > > I often get annoyed because psql is a bit too aggressive when it decides > whether to put output through the pager, and the only way to avoid this is > to turn the pager off (in which case your next query might dump many > thousands of li

Re: [HACKERS] pgcrypto: PGP signatures

2014-11-11 Thread Jeff Janes
On Sat, Nov 1, 2014 at 7:52 AM, Marko Tiikkaja wrote: > Hi, > > I discovered a problem with the lack of MDC handling in the signature info > extraction code, so I've fixed that and added a test message. v9 here. > > > > Hi Marko, I get a segfault when the length of the message is exactly 16308

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2014-11-10 Thread Jeff Janes
On Wed, Nov 5, 2014 at 8:49 PM, Michael Paquier wrote: > On Thu, Oct 30, 2014 at 5:19 PM, Michael Paquier > wrote: > > Updated patch is attached. > Please find attached an updated patch with the following things changed: > - Addition of tab completion in psql for all new commands > - Addition of

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-05 Thread Jeff Janes
On Wed, Nov 5, 2014 at 12:54 PM, Alvaro Herrera wrote: Thanks for the updated patch. Now when I run the test program (version with better error reporting attached), it runs fine until I open a psql session and issue: reindex table foo; Then it immediately falls over with some rows no longer be

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-05 Thread Jeff Janes
On Tue, Nov 4, 2014 at 2:28 PM, Alvaro Herrera wrote: > > I was clearly too careless about testing the xlog code --- it had > numerous bugs. This version should be a lot better, but there might be > problems lurking still as I don't think I covered it all. Let me know > if you see anything wron

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-04 Thread Jeff Janes
On Mon, Nov 3, 2014 at 2:18 PM, Alvaro Herrera wrote: > > So here's v21. I also attach a partial diff from v20, just in case > anyone wants to give it a look. > This needs a bump to 1.3, or the extension won't install: contrib/pageinspect/pageinspect.control During crash recovery, I am getti

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-03 Thread Jeff Janes
On Mon, Nov 3, 2014 at 2:18 PM, Alvaro Herrera wrote: > Robert Haas wrote: > > [lots] > > I have fixed all these items in the attached, thanks -- most > user-visible change was the pageinspect 1.3 thingy. pg_upgrade from 1.2 > works fine now. I also fixed some things Heikki noted, mainly avoid

Re: [HACKERS] Maximum number of WAL files in the pg_xlog directory

2014-11-03 Thread Jeff Janes
On Wed, Oct 15, 2014 at 1:11 PM, Jeff Janes wrote: > On Fri, Aug 8, 2014 at 12:08 AM, Guillaume Lelarge > wrote: > >> Hi, >> >> As part of our monitoring work for our customers, we stumbled upon an >> issue with our customers' servers who have a wal_

Re: [HACKERS] Let's drop two obsolete features which are bear-traps for novices

2014-11-03 Thread Jeff Janes
On Sat, Nov 1, 2014 at 10:26 AM, Andres Freund wrote: > On 2014-11-01 10:18:03 -0700, Josh Berkus wrote: > > On 10/31/2014 03:07 PM, Tom Lane wrote: > > > I don't care one way or the other about the money type, but I will > defend > > > hash indexes, especially seeing that we've already added a p

Re: [HACKERS] pgcrypto: PGP signatures

2014-10-27 Thread Jeff Janes
On Mon, Oct 20, 2014 at 3:32 PM, Marko Tiikkaja wrote: > Hi, > > Here's the rebased patch -- as promised -- in a v7. > > > Hi Marko, Using the same script as for the memory leak, I am getting seg faults using this patch. 24425 2014-10-27 15:42:11.819 PDT LOG: server process (PID 24452) was t

Re: [HACKERS] Autovacuum fails to keep visibility map up-to-date in mostly-insert-only-tables

2014-10-21 Thread Jeff Janes
On Mon, Oct 20, 2014 at 5:46 PM, Andres Freund wrote: > On 2014-10-20 17:43:26 -0700, Josh Berkus wrote: > > On 10/20/2014 05:39 PM, Jim Nasby wrote: > > > Or maybe vacuum isn't the right way to handle some of these scenarios. > > > It's become the catch-all for all of this stuff, but maybe that

Re: [HACKERS] wal-size limited to 16MB - Performance issue for subsequent backup

2014-10-21 Thread Jeff Janes
On Mon, Oct 20, 2014 at 12:03 PM, wrote: > Hi. > > One of our "production issues" is that the system generates lots of > wal-files, lots is like 151952 files over the last 24h, which is about > 2.4TB worth of WAL files. I wouldn't say that isn't an issue by itself, > but the system does indeed wo

Re: [HACKERS] pgcrypto: PGP signatures

2014-10-17 Thread Jeff Janes
On Mon, Sep 15, 2014 at 4:37 AM, Marko Tiikkaja wrote: > > I've changed the patch back to ignore signatures when not using the > decrypt_verify() functions in the attached. Hi Marko, This patch needs a rebase now that the armor header patch has been committed. Thanks, Jeff

Re: [HACKERS] Maximum number of WAL files in the pg_xlog directory

2014-10-15 Thread Jeff Janes
On Fri, Aug 8, 2014 at 12:08 AM, Guillaume Lelarge wrote: > Hi, > > As part of our monitoring work for our customers, we stumbled upon an > issue with our customers' servers who have a wal_keep_segments setting > higher than 0. > > We have a monitoring script that checks the number of WAL files i

Re: [HACKERS] Buffer Requests Trace

2014-10-15 Thread Jeff Janes
On Wed, Oct 15, 2014 at 6:22 AM, Lucas Lersch wrote: > So is it a possible normal behavior that running tpcc for 10min only > access 50% of the database? Furthermore, is there a guideline of parameters > for tpcc (# of warehouses, execution time, operations weight)? > > I'm not familiar with your

Re: [HACKERS] Maximum number of WAL files in the pg_xlog directory

2014-10-14 Thread Jeff Janes
On Mon, Oct 13, 2014 at 12:11 PM, Bruce Momjian wrote: > > I looked into this, and came up with more questions. Why is > checkpoint_completion_target involved in the total number of WAL > segments? If checkpoint_completion_target is 0.5 (the default), the > calculation is: > > (2 + 0.5)

Re: [HACKERS] Expose options to explain? (track_io_timing)

2014-10-09 Thread Jeff Janes
On Thu, Oct 9, 2014 at 10:17 AM, Joshua D. Drake wrote: > > Salut! > > Fellow volunteers, I request assistance in understanding the following: > > When I explain a query I can get the following information: > > >| "I/O Read Time": 0.000, >| "I/O Write Time": 0.

Re: [HACKERS] Log notice that checkpoint is to be written on shutdown

2014-10-09 Thread Jeff Janes
On Wed, Oct 8, 2014 at 10:52 AM, Michael Banck wrote: > Hi, > > Am Samstag, den 04.10.2014, 15:05 -0500 schrieb Jim Nasby: > > On 10/4/14, 1:21 PM, Jeff Janes wrote: > > > On Thu, Oct 2, 2014 at 6:21 AM, Michael Banck wrote: > > > we have seen repeatedly tha

Re: [HACKERS] Log notice that checkpoint is to be written on shutdown

2014-10-04 Thread Jeff Janes
On Thu, Oct 2, 2014 at 6:21 AM, Michael Banck wrote: > Hi, > > we have seen repeatedly that users can be confused about why PostgreSQL > is not shutting down even though they requested it. Usually, this is > because `log_checkpoints' is not enabled and the final checkpoint is > being written, de

Re: [HACKERS] trivial patch for dynahash logging

2014-09-29 Thread Jeff Janes
On Sat, Sep 27, 2014 at 3:09 PM, Tom Lane wrote: > Jeff Janes writes: > > under HASH_STATISTICS, the output reporting "this HTAB" upon destruction > is > > pretty useless. Which HTAB would this one be? It is not necessarily the > > most recently created one.

[HACKERS] trivial patch for dynahash logging

2014-09-27 Thread Jeff Janes
under HASH_STATISTICS, the output reporting "this HTAB" upon destruction is pretty useless. Which HTAB would this one be? It is not necessarily the most recently created one. This makes it output the %p to the actual HTAB, so it can be matched up with the logging of the creation. I'm not sure wh

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2014-09-26 Thread Jeff Janes
On Fri, Sep 26, 2014 at 11:47 AM, Alvaro Herrera wrote: > Gavin Flower wrote: > > > Curious: would it be both feasible and useful to have multiple > > workers process a 'large' table, without complicating things too > > much? The could each start at a different position in the file. > > Feasible

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2014-09-25 Thread Jeff Janes
On Thu, Sep 25, 2014 at 10:00 AM, Jeff Janes wrote: > On Wed, Sep 24, 2014 at 2:48 AM, Dilip kumar > wrote: > >> On 24 August 2014 11:33, Amit Kapila Wrote >> >> >> >> Thanks for you comments, i have worked on both the review comment lis

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2014-09-25 Thread Jeff Janes
On Wed, Sep 24, 2014 at 2:48 AM, Dilip kumar wrote: > On 24 August 2014 11:33, Amit Kapila Wrote > > > > Thanks for you comments, i have worked on both the review comment lists, > sent on 19 August, and 24 August. > > > > Latest patch is attached with the mail.. > Hi Dilip, I think you have an

Re: [HACKERS] pgcrypto: PGP signatures

2014-09-12 Thread Jeff Janes
On Fri, Sep 12, 2014 at 8:50 AM, Alvaro Herrera wrote: > Marko Tiikkaja wrote: > > > On 9/8/14 7:30 PM, Jeff Janes wrote: > > > >If i understand the sequence here: The current git HEAD is that > > >pgp_pub_decrypt would throw an error if given a signed and en

Re: [HACKERS] LIMIT for UPDATE and DELETE

2014-09-09 Thread Jeff Janes
On Tue, Sep 9, 2014 at 2:57 AM, Heikki Linnakangas wrote: > I agree. If we are to support UPDATE .. ORDER BY .. LIMIT, it should work > with inheritance. So the path forward is (using Marko's phrasing upthread): > >1) We put the LIMIT inside ModifyTable like this patch does. This > doesn't

Re: [HACKERS] pgcrypto: PGP signatures

2014-09-08 Thread Jeff Janes
On Fri, Sep 5, 2014 at 4:38 AM, Marko Tiikkaja wrote: > Hi all, > > I've updated the patch with a number of changes: > 1) I've documented the current limitations of signatures > 2) I've expanded section F.25.3 to add information about signatures > (though I'm not sure why this part is in the

Re: PENDING_LIST_CLEANUP_SIZE - maximum size of GIN pending list Re: [HACKERS] HEAD seems to generate larger WAL regarding GIN index

2014-09-08 Thread Jeff Janes
On Sun, Aug 17, 2014 at 7:46 PM, Fujii Masao wrote: > > Thanks for reviewing the patch! ISTM that I failed to make the patch from > my git repository... Attached is the rebased version. > I get some compiler warnings on v2 of this patch: reloptions.c:219: warning: excess elements in struct ini

Re: [HACKERS] pgcrypto: PGP signatures

2014-09-07 Thread Jeff Janes
On Sun, Sep 7, 2014 at 10:36 AM, Marko Tiikkaja wrote: > On 2014-09-07 19:28, Jeff Janes wrote: > >> >> select pgp_sym_decrypt(dearmor('-BEGIN PGP MESSAGE- >> Version: GnuPG v2.0.14 (GNU/Linux) >> Password: foobar >> >> jA0EBwMCqywsAv/hX

Re: [HACKERS] pgcrypto: PGP signatures

2014-09-07 Thread Jeff Janes
On Wed, Sep 3, 2014 at 2:13 PM, Marko Tiikkaja wrote: > On 2014-09-03 10:33 PM, Jeff Janes wrote: > >> On Wed, Sep 3, 2014 at 12:43 PM, Marko Tiikkaja wrote: >> >>> Right. This patch only adds support for signing data when encrypting it >>> at the same ti

Re: [HACKERS] pgcrypto: PGP signatures

2014-09-03 Thread Jeff Janes
On Wed, Sep 3, 2014 at 12:43 PM, Marko Tiikkaja wrote: > On 2014-09-03 9:36 PM, Jeff Janes wrote: > >> I wanted to start simple so I have a file which is signed, but not >> encrypted. I can't figure out what to do with it. All of the functions >> seem to require

Re: [HACKERS] pgcrypto: PGP signatures

2014-09-03 Thread Jeff Janes
On Fri, Aug 15, 2014 at 12:55 AM, Marko Tiikkaja wrote: > Hi, > > > On 8/7/14 12:15 PM, I wrote: > >> Here's v2 of the patch. I've changed the info-extracting code to not >> look for signatures beyond the data, which also meant that it had to >> parse one-pass signatures (which it didn't do befo

Re: [HACKERS] pgcrypto: PGP armor headers

2014-09-03 Thread Jeff Janes
On Fri, Aug 15, 2014 at 1:55 AM, Marko Tiikkaja wrote: > Hi, > > > On 8/8/14 3:18 PM, I wrote: > >> Currently there's no way to generate or extract armor headers from the >> PGP armored format in pgcrypto. I've written a patch to add the >> support. >> > > Latest version of the patch here, havin

Re: [HACKERS] why after increase the hash table partitions, TPMC decrease

2014-09-02 Thread Jeff Janes
On Tue, Sep 2, 2014 at 1:39 AM, Xiaoyulei wrote: > > > We use benchmarksql to start tpcc test in postgresql 9.3.3. > > Before test we set benchmarksql client number about 800. And we increase > the hash partitions from 16 to 1024 , in order to reduce the hash > partition locks competition. > Can

Re: [HACKERS] COPY and heap_sync

2014-09-02 Thread Jeff Janes
On Sun, Aug 31, 2014 at 6:10 AM, Peter Eisentraut wrote: > On 8/30/14 2:26 AM, Jeff Janes wrote: > > But there cases were people use COPY in a loop with a small amount of > > data in each statement. > > What would be the reason for doing that? > > As far as I can tell,

Re: [HACKERS] postgresql latency & bgwriter not doing its job

2014-09-02 Thread Jeff Janes
On Tue, Sep 2, 2014 at 8:14 AM, Fabien COELHO wrote: > > There is scan_whole_pool_milliseconds, which currently forces bgwriter to circle the buffer pool at least once every 2 minutes. It is currently fixed, but it should be trivial to turn it into an experimental guc that you co

Re: [HACKERS] postgresql latency & bgwriter not doing its job

2014-09-02 Thread Jeff Janes
On Tue, Aug 26, 2014 at 1:02 AM, Fabien COELHO wrote: > > Hello again, > > > I have not found any mean to force bgwriter to send writes when it can. >>> (Well, I have: create a process which sends "CHECKPOINT" every 0.2 >>> seconds... it works more or less, but this is not my point:-) >>> >> >>

[HACKERS] COPY and heap_sync

2014-08-29 Thread Jeff Janes
If you insert tuples with COPY into a table created or truncated in the same transaction, at the end of the COPY it calls heap_sync. But there cases were people use COPY in a loop with a small amount of data in each statement. Now it is calling heap_sync many times, and if NBuffers is large doing

Re: [HACKERS] postgresql latency & bgwriter not doing its job

2014-08-25 Thread Jeff Janes
On Monday, August 25, 2014, Fabien COELHO wrote: > > I have not found any mean to force bgwriter to send writes when it can. > (Well, I have: create a process which sends "CHECKPOINT" every 0.2 > seconds... it works more or less, but this is not my point:-) > There is scan_whole_pool_millisecond

Re: [HACKERS] postgresql latency & bgwriter not doing its job

2014-08-25 Thread Jeff Janes
On Monday, August 25, 2014, Fabien COELHO wrote: > > > The culprit I found is "bgwriter", which is basically doing nothing to > prevent the coming checkpoint IO storm, even though there would be ample > time to write the accumulating dirty pages so that checkpoint would find a > clean field and p

Re: [HACKERS] LIMIT for UPDATE and DELETE

2014-08-25 Thread Jeff Janes
On Sun, Aug 24, 2014 at 11:48 PM, Etsuro Fujita wrote: > Hi Rukh, > > > (2014/08/15 6:18), Rukh Meski wrote: > >> Based on the feedback on my previous patch, I've separated only the >> LIMIT part into its own feature. This version plays nicely with >> inheritance. The intended use is splitting

Re: [HACKERS] 9.3: more problems with "Could not open file "pg_multixact/members/xxxx"

2014-08-19 Thread Jeff Janes
On Tue, Jul 15, 2014 at 3:58 PM, Jeff Janes wrote: > On Fri, Jun 27, 2014 at 11:51 AM, Alvaro Herrera > wrote: > >> Jeff Janes wrote: >> >> > This problem was initially fairly easy to reproduce, but since I >> > started adding instrumentation

Re: [HACKERS] Supporting Windows SChannel as OpenSSL replacement

2014-08-15 Thread Jeff Janes
On Tue, Aug 12, 2014 at 10:52 AM, Heikki Linnakangas < hlinnakan...@vmware.com> wrote: > On 08/06/2014 08:37 PM, Jeff Janes wrote: > >> But now it looks like 0002 needs a rebase >> > > I've committed the refactoring patch, and here's a rebased and im

Re: [HACKERS] how to implement selectivity injection in postgresql

2014-08-13 Thread Jeff Janes
On Wed, Aug 13, 2014 at 9:33 AM, Rajmohan C wrote: > SELECT c1, c2, c3, FROM T1, T2, T3 > WHERE T1.x = T2.x AND > T2.y=T3.y AND > T1.x >= ? selectivity 0.1 AND > T2.y > ? selectivity 0.5 AND > T3.z = ? selectivity 0.2 AND > T3.w = ? > > I need

Re: [HACKERS] Removing dependency to wsock32.lib when compiling code on WIndows

2014-08-11 Thread Jeff Janes
On Tue, Jul 15, 2014 at 5:14 AM, Michael Paquier wrote: > On Tue, Jul 15, 2014 at 8:46 PM, Magnus Hagander > wrote: > > On Thu, Jun 19, 2014 at 4:13 PM, MauMau wrote: > >> From: "Michael Paquier" > >> > >>> You are right, I only though about the MS scripts when working on this > >>> patch. Upd

Re: [HACKERS] Fixed redundant i18n strings in json

2014-08-06 Thread Jeff Janes
On Wed, Aug 6, 2014 at 8:35 AM, Robert Haas wrote: > On Tue, Aug 5, 2014 at 6:05 PM, Jeff Janes wrote: > > I think you missed one of the regression tests, see attached > > Woops. Thanks, committed. > Thanks. It needs to go into 9_4_STABLE as well. Cheers, Jeff

Re: [HACKERS] Supporting Windows SChannel as OpenSSL replacement

2014-08-06 Thread Jeff Janes
On Fri, Aug 1, 2014 at 10:58 AM, Heikki Linnakangas wrote: > On 07/08/2014 08:11 PM, Jeff Janes wrote: > >> Is there some recipe for testing the 0002 patch? Can it be tested on an >> MinGW environment, or does it need to use the MicroSoft supplied >> compilers? >>

Re: [HACKERS] Fixed redundant i18n strings in json

2014-08-05 Thread Jeff Janes
On Tue, Aug 5, 2014 at 9:35 AM, Robert Haas wrote: > On Sat, Aug 2, 2014 at 9:15 AM, Daniele Varrazzo > wrote: > > I'd definitely replace /arg/argument/. Furthermore I'd avoid the form > > "argument 1: something is wrong": the string is likely to end up in > > sentences with other colons so I'd

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2014-07-18 Thread Jeff Janes
On Wed, Jul 16, 2014 at 5:30 AM, Dilip kumar wrote: > On 16 July 2014 12:13 Magnus Hagander Wrote, > > >>Yeah, those are exactly my points. I think it would be significantly > simpler to do it that way, rather than forking and threading. And also > easier to make portable... > > >>(and as a opt

[HACKERS] Getting list of held lwlocks from debugger

2014-07-14 Thread Jeff Janes
Is there an easy way to get a list of held lwlocks out of gdb attached to a core dump (or for that matter attached to a live process)? I can try manually walking the internal data structure, but I am hoping for something analogous to the way you get memory contexts: (gdb) p MemoryContextStats(Top

Re: [HACKERS] Minmax indexes

2014-07-10 Thread Jeff Janes
On Thu, Jul 10, 2014 at 2:30 PM, Jaime Casanova wrote: > On Thu, Jul 10, 2014 at 3:50 PM, Josh Berkus wrote: > > On 07/10/2014 12:20 PM, Alvaro Herrera wrote: > >>> So I guess the only thing left is to issue a NOTICE when said alter > >>> > takes place (I don't see that on the patch, but maybe i

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2014-07-10 Thread Jeff Janes
On Fri, Jun 27, 2014 at 4:10 AM, Dilip kumar wrote: > On 27 June 2014 02:57, Jeff Wrote, > >> Based on that, I find most importantly that it doesn't seem to >> correctly vacuum tables which have upper case letters in the name, >> because it does not quote the table names when they need quotes. > >

Re: [HACKERS] tweaking NTUP_PER_BUCKET

2014-07-08 Thread Jeff Janes
On Tue, Jul 8, 2014 at 6:35 AM, Tomas Vondra wrote: > On 8 Červenec 2014, 14:49, Robert Haas wrote: > > On Wed, Jul 2, 2014 at 8:13 PM, Tomas Vondra wrote: > >> I propose dynamic increase of the nbuckets (up to NTUP_PER_BUCKET=1) > >> once the table is built and there's free space in work_mem. T

Re: [HACKERS] Supporting Windows SChannel as OpenSSL replacement

2014-07-08 Thread Jeff Janes
On Thu, Jun 26, 2014 at 4:26 PM, Andreas Karlsson wrote: > On 06/24/2014 03:20 AM, Jeff Janes wrote: > >> I've tried your 0001 patch, reflecting this refactoring, on Linux and it >> caused 'make check' to hang at 'starting postmaster'. >> > &g

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2014-07-02 Thread Jeff Janes
On Mon, Jun 30, 2014 at 3:17 PM, Alvaro Herrera wrote: > Jeff Janes wrote: > >> In particular, pgpipe is almost an exact duplicate between them, >> except the copy in vac_parallel.c has fallen behind changes made to >> parallel.c. (Those changes would have fixed t

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2014-06-30 Thread Jeff Janes
On Fri, Jun 27, 2014 at 4:10 AM, Dilip kumar wrote: ... > > Updated patch is attached in the mail.. Thanks Dilip. I get a compiler warning when building on Windows. When I started looking into that, I see that two files have too much code duplication between them: src/bin/scripts/vac_parallel.

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2014-06-26 Thread Jeff Janes
On Thu, Jun 26, 2014 at 2:35 AM, Dilip kumar wrote: > > Thank you for giving your time, Please review the updated patch attached in > the mail. > > > > 1. Rebased the patch > > 2. Implemented parallel execution for new option --analyze-in-stages Hi Dilip, Thanks for rebasing. I have

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2014-06-23 Thread Jeff Janes
On Monday, June 23, 2014, Dilip kumar wrote: > On 24 June 2014 03:31, Jeff Wrote, > > > > Attached the latest updated patch > > > 1. Rebased the patch to current GIT head. > > > 2. Doc is updated. > > > 3. Supported parallel execution for all db option also. > > > > This patch needs to be reb

Re: [HACKERS] tab completion for setting search_path

2014-06-23 Thread Jeff Janes
On Mon, Jun 23, 2014 at 6:25 PM, Jeff Janes wrote: > On Mon, Jun 23, 2014 at 3:53 PM, Andres Freund > wrote: >> On 2014-06-23 13:10:34 -0400, Robert Haas wrote: >>> On Mon, Jun 23, 2014 at 9:10 AM, Kevin Grittner >>> wrote: >>> > I would be for ex

[HACKERS] tab completion for setting search_path

2014-06-23 Thread Jeff Janes
On Mon, Jun 23, 2014 at 3:53 PM, Andres Freund > wrote: > On 2014-06-23 13:10:34 -0400, Robert Haas wrote: >> On Mon, Jun 23, 2014 at 9:10 AM, Kevin Grittner > wrote: >> > I would be for excluding the pg_toast, pg_toast_temp_n, and >> > pg_temp_n schemas, and including public and pg_catalog. >> >>

[HACKERS] Supporting Windows SChannel as OpenSSL replacement

2014-06-23 Thread Jeff Janes
On Wed, Jun 11, 2014 at 7:51 AM, Heikki Linnakangas wrote: > > > I did again the refactoring you did back in 2006, patch attached. One thing > I did differently: I moved the raw, non-encrypted, read/write functions to > separate functions: pqsecure_raw_read and pqsecure_raw_write. Those > function

Re: [HACKERS] autovacuum scheduling starvation and frenzy

2014-06-23 Thread Jeff Janes
On Thu, May 15, 2014 at 4:06 PM, Jeff Janes wrote: > On Thu, May 15, 2014 at 12:55 PM, Alvaro Herrera > wrote: >> >> Jeff Janes wrote: >> >> > If you have a database with a large table in it that has just passed >> > autovacuum_freeze_max_age, all futur

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2014-06-23 Thread Jeff Janes
On Fri, Mar 21, 2014 at 12:48 AM, Dilip kumar wrote: > On 16 January 2014 19:53, Euler Taveira Wrote, > >> > >> >> For the case where you have tables of varying size this would lead >> to a reduced overall processing time as it prevents large (read: long >> processing time) tables to be processed

Re: [HACKERS] Is analyze_new_cluster.sh still useful?

2014-06-20 Thread Jeff Janes
On Wed, Jun 18, 2014 at 8:41 AM, Christoph Berg wrote: > Hi, > > now that we have vacuumdb --all --analyze-in-stages in 9.4, wouldn't > it make sense to get rid of the analyze_new_cluster.sh file which > pg_upgrade writes? The net content is a single line which could as > well be printed by pg_upg

Re: [HACKERS] Is analyze_new_cluster.sh still useful?

2014-06-18 Thread Jeff Janes
On Wed, Jun 18, 2014 at 10:58 AM, Andres Freund wrote: > On 2014-06-18 13:54:16 -0400, Tom Lane wrote: >> I think we're not on the same page. My point is that someone might want >> to automate the whole sequence: stop old postmaster, run pg_upgrade, start >> the updated postmaster normally (henc

[HACKERS] Re: [BUGS] BUG #8673: Could not open file "pg_multixact/members/xxxx" on slave during hot_standby

2014-06-17 Thread Jeff Janes
(Redirected from Bugs to Hackers, as it seems not to be the same bug as the original one.) On Mon, Jun 9, 2014 at 3:50 PM, Alvaro Herrera wrote: > Jeff Janes wrote: > >> This patch seems to solve a problem I've also been having with non-existent >> "pg_multixac

Re: [HACKERS] rm_desc signature

2014-06-16 Thread Jeff Janes
On Friday, June 13, 2014, Heikki Linnakangas wrote: > As part of the WAL-format changing patch I've been working on, I changed > the signature of the rm_desc function from: > > void (*rm_desc) (StringInfo buf, uint8 xl_info, char *rec); > void (*rm_desc) (StringInfo buf, XLogRecord *record); > >

<    2   3   4   5   6   7   8   9   10   11   >