Re: [HACKERS] proposal: psql \setfileref

2016-10-10 Thread Gilles Darold
Le 10/10/2016 à 14:38, Daniel Verite a écrit : > Gilles Darold wrote: > >>postgres=# \setfileref b /dev/random >>postgres=# insert into test (:b); >> >> Process need to be killed using SIGTERM. > This can be fixed by setting sigint_interrupt_enabled to true > before operating on the

[HACKERS] FTS Configuration option

2016-10-10 Thread Artur Zakirov
Hello hackers, Sometimes there is a need to collect lexems from various dictionaries. For example, if we have a column with text in various languages. Let's say there is a new option JOIN. This option will allow to the parser to append lexems from the current dictionary and go to the next

[HACKERS] Using pg_ctl promote -w in TAP tests

2016-10-10 Thread Michael Paquier
Hi all, Now that we have support for the wait mode of pg_ctl promote, I think that it would be a good idea to switch to it in the TAP tests. This allows avoiding extra logic with poll_query_until() to be sure that a promoted standby is ready for read-write queries. See the patch attached.

Re: [HACKERS] proposal: psql \setfileref

2016-10-10 Thread Daniel Verite
Gilles Darold wrote: >postgres=# \setfileref b /dev/random >postgres=# insert into test (:b); > > Process need to be killed using SIGTERM. This can be fixed by setting sigint_interrupt_enabled to true before operating on the file. Then ctrl-C would be able to cancel the command.

Re: [HACKERS] pg_upgrade 9.5 -> 9.6 fails when pg_largeobject is in separate tablespace

2016-10-10 Thread Andreas Joseph Krogh
På søndag 09. oktober 2016 kl. 23:43:23, skrev Robert Haas < robertmh...@gmail.com >: On Sat, Oct 8, 2016 at 9:02 AM, Andreas Joseph Krogh > wrote: (I've set allow_system_table_mods=on in postgresql.conf) Any

Re: [HACKERS] Hash Indexes

2016-10-10 Thread Amit Kapila
On Thu, Sep 29, 2016 at 8:27 PM, Amit Kapila wrote: > On Thu, Sep 29, 2016 at 6:04 AM, Robert Haas wrote: > >> Another thing I don't quite understand about this algorithm is that in >> order to conditionally lock the target primary bucket page,

Re: [HACKERS] vacuumdb -f and -j options (was Question / requests.)

2016-10-10 Thread Amit Kapila
On Sun, Oct 9, 2016 at 10:59 PM, Francisco Olarte wrote: > On Sat, Oct 8, 2016 at 2:22 PM, Michael Paquier > wrote: >> On Sat, Oct 8, 2016 at 9:12 PM, Amit Kapila wrote: > >>> After reading Francisco's proposal [1], I

Re: [HACKERS] Question / requests.

2016-10-10 Thread Francisco Olarte
On Mon, Oct 10, 2016 at 4:51 AM, Jim Nasby wrote: > On 10/5/16 9:58 AM, Francisco Olarte wrote: >> Is the system catalog a bottleneck for people who has real use for >> paralell vacuum? I mean, to me someone who does this must have a very >> big db on a big iron. If that

Re: [HACKERS] Is it time to kill support for very old servers?

2016-10-10 Thread Craig Ringer
On 10 October 2016 at 10:45, Jim Nasby wrote: > On 10/7/16 1:08 PM, Steve Crawford wrote: >> >> This is effectively a 5-year upgrade "grace period" *after* the EOL date >> of a given version which seems plenty generous. > > > IMHO we need to be careful here. It's not at

Re: [HACKERS] memory leak in e94568ecc10 (pre-reading in external sort)

2016-10-10 Thread Heikki Linnakangas
On 10/06/2016 06:44 PM, Peter Geoghegan wrote: While the fix you pushed was probably a good idea anyway, I still think you should not use swhichtate->maxTapes to exhaustively call LogicalTapeAssignReadBufferSize() on every tape, even non-active tapes. That's the confusing part. Admittedly

Re: [HACKERS] FSM corruption leading to errors

2016-10-10 Thread Michael Paquier
On Fri, Oct 7, 2016 at 2:59 AM, Pavan Deolasee wrote: > I investigated a bug report from one of our customers and it looked very > similar to previous bug reports here [1], [2], [3] (and probably more). In > these reports, the error looks something like this: > > ERROR:

Re: [HACKERS] FSM corruption leading to errors

2016-10-10 Thread Michael Paquier
On Mon, Oct 10, 2016 at 11:25 PM, Michael Paquier wrote: > At the same time, I have translated your script into a TAP test, I > found that more useful when testing.. Well... Here is the actual patch. -- Michael diff --git a/src/backend/storage/freespace/freespace.c

Re: [HACKERS] Hash Indexes

2016-10-10 Thread Jeff Janes
On Mon, Oct 10, 2016 at 5:55 AM, Amit Kapila wrote: > On Thu, Sep 29, 2016 at 8:27 PM, Amit Kapila > wrote: > > On Thu, Sep 29, 2016 at 6:04 AM, Robert Haas > wrote: > > > >> Another thing I don't quite understand about

Re: [HACKERS] proposal: psql \setfileref

2016-10-10 Thread Pavel Stehule
2016-10-10 15:17 GMT+02:00 Gilles Darold : > Le 10/10/2016 à 14:38, Daniel Verite a écrit : > > Gilles Darold wrote: > > > >>postgres=# \setfileref b /dev/random > >>postgres=# insert into test (:b); > >> > >> Process need to be killed using SIGTERM. > >

Re: [HACKERS] PL/Python adding support for multi-dimensional arrays

2016-10-10 Thread Pavel Stehule
2016-10-10 12:31 GMT+02:00 Heikki Linnakangas : > On 10/01/2016 02:45 AM, Jim Nasby wrote: > >> On 9/29/16 1:51 PM, Heikki Linnakangas wrote: >> >>> Now, back to multi-dimensional arrays. I can see that the Sequence >>> representation is problematic, with arrays, because if you

Re: [HACKERS] pgbench vs. wait events

2016-10-10 Thread Robert Haas
On Mon, Oct 10, 2016 at 9:44 AM, Bruce Momjian wrote: > On Thu, Oct 6, 2016 at 02:38:56PM -0400, Robert Haas wrote: >> I decided to do some testing on hydra (IBM-provided community >> resource, POWER, 16 cores/64 threads, kernel 3.2.6-3.fc16.ppc64) using >> the newly-enhanced

Re: [HACKERS] Switch to unnamed POSIX semaphores as our preferred sema code?

2016-10-10 Thread Tom Lane
Christoph Berg writes: > Another data point that's admittedly much more of a footnote than > serious input to the original question is the following: Debian has a > (so far mostly toy) port "hurd-i386" which is using the GNU hurd > kernel along with the usual GNU userland that's

Re: [HACKERS] Supporting huge pages on Windows

2016-10-10 Thread Andres Freund
On 2016-10-11 09:57:48 +1300, Thomas Munro wrote: > Later in that thread there was a report of a dramatic ~15% increase in > "best result" TPS, but that was with 60GB of shared_buffers on a > machine with 256GB of RAM: > > https://www.postgresql.org/message-id/20131024060313.GA21888%40toroid.org

Re: [HACKERS] Supporting huge pages on Windows

2016-10-10 Thread Thomas Munro
On Wed, Sep 28, 2016 at 7:32 PM, Tsunakawa, Takayuki wrote: > From: Thomas Munro [mailto:thomas.mu...@enterprisedb.com] >> > huge_pages=off: 70412 tps >> > huge_pages=on : 72100 tps >> >> Hmm. I guess it could be noise or random code rearrangement effects. > >

Re: [HACKERS] Switch to unnamed POSIX semaphores as our preferred sema code?

2016-10-10 Thread Tom Lane
Thomas Munro writes: > On Tue, Oct 11, 2016 at 5:57 AM, Tom Lane wrote: >> (I'm a little suspicious that older variants of FreeBSD might not >> have working sem_init, like the other *BSD variants, necessitating >> a run-time test there. But

Re: [HACKERS] Switch to unnamed POSIX semaphores as our preferred sema code?

2016-10-10 Thread Thomas Munro
On Tue, Oct 11, 2016 at 5:57 AM, Tom Lane wrote: > So at this point it seems likely that stopping with Linux and FreeBSD > is the thing to do, and as far as I can tell the code we have now is > working with all variants of those that we have in the buildfarm. > (I'm a little

Re: [HACKERS] pg_dump getBlobs query broken for 7.3 servers

2016-10-10 Thread Greg Stark
On Mon, Oct 10, 2016 at 3:36 AM, Jim Nasby wrote: > FWIW, Greg Stark did a talk at PG Open about PG performance going back to at > least 7.4. He did discuss what he had to do to get those versions to compile > on modern tools, and has a set of patches that enable it.

Re: [HACKERS] Logical tape pause/resume

2016-10-10 Thread Peter Geoghegan
On Sun, Oct 9, 2016 at 11:52 PM, Heikki Linnakangas wrote: > Regardless of the number of tapes, the memory used for the tape buffers, > while building the initial runs, is wasted. It's not entirely wasted when > you have multiple merge passes, because without the buffer you need

Re: [HACKERS] Macro customizable hashtable / bitmapscan & aggregation perf

2016-10-10 Thread Andres Freund
On 2016-10-10 17:46:22 -0700, Andres Freund wrote: > > TPC-DS (tpcds.ods) > > -- > > > > In this case, I'd say the results are less convincing. There are quite a few > > queries that got slower by ~10%, which is well above - for example queries > > 22 and 67. There are of course

Re: [HACKERS] Macro customizable hashtable / bitmapscan & aggregation perf

2016-10-10 Thread Tomas Vondra
On 10/11/2016 04:07 AM, Andres Freund wrote: On 2016-10-10 17:46:22 -0700, Andres Freund wrote: TPC-DS (tpcds.ods) -- In this case, I'd say the results are less convincing. There are quite a few queries that got slower by ~10%, which is well above - for example queries 22 and

[HACKERS] Remove vacuum_defer_cleanup_age

2016-10-10 Thread Josh Berkus
Folks, Given that hot_standby_feedback is pretty bulletproof now, and a lot of the work in reducing replay conflicts, I think the utility of vacuum_defer_cleanup_age is at an end. I really meant so submit a patch to remove it to 9.6, but it got away from me. Any objections to removing the

[HACKERS] another typo in parallel.sgml

2016-10-10 Thread Tatsuo Ishii
I think I found a typo in parallel.sgml. Best regards, -- Tatsuo Ishii SRA OSS, Inc. Japan English: http://www.sraoss.co.jp/index_en.php Japanese:http://www.sraoss.co.jp diff --git a/doc/src/sgml/parallel.sgml b/doc/src/sgml/parallel.sgml index c80d42d..1e71529 100644 ---

Re: [HACKERS] Supporting huge pages on Windows

2016-10-10 Thread Tsunakawa, Takayuki
From: Thomas Munro [mailto:thomas.mu...@enterprisedb.com] > Your ~2.4% number is similar to what was reported for Linux with 4GB > shared_buffers: > > https://www.postgresql.org/message-id/20130913234125.GC13697%40roobarb > .crazydogs.org I'm relieved to know that a similar figure was gained on

Re: [HACKERS] Macro customizable hashtable / bitmapscan & aggregation perf

2016-10-10 Thread Andres Freund
Hi, On 2016-10-11 02:38:26 +0200, Tomas Vondra wrote: > Yes, I've done a bunch of TPC-H and TPC-DS tests on the patch, but it took > quite a bit of time. These tests were done on a fairly small machine (the > usual i5-2500k with 8GB of RAM), with only 1GB data sets for both > benchmarks, to keep

Re: [HACKERS] Forbid use of LF and CR characters in database and role names

2016-10-10 Thread Noah Misch
On Sun, Oct 02, 2016 at 10:47:04PM +0900, Michael Paquier wrote: > On Mon, Sep 12, 2016 at 11:38 AM, Michael Paquier > wrote: > > On Mon, Sep 12, 2016 at 10:01 AM, Noah Misch wrote: > >> I discourage documenting LF/CR restrictions. For the epsilon

[HACKERS] FTS Configuration option

2016-10-10 Thread Artur Zakirov
Hello hackers, Sometimes there is a need to collect lexems from various dictionaries. For example, if we have a column with text in various languages. Let's say there is a new option JOIN. This option will allow to parser to append lexems from current dictionary and go to next dictionary to get

[HACKERS] How to inspect tuples during execution of a plan?

2016-10-10 Thread Ernst-Georg Schmid
Hello all, I'd like to inspect the content of tuples as they are sent during the execution of a query in order to react to their values. I guess I could do it with a FDW, but that's a bit clumsy so I took a look at the hooks but have two questions: 1.) Would ExecutorRun_hook be the correct

Re: [HACKERS] Logical tape pause/resume

2016-10-10 Thread Heikki Linnakangas
On 10/09/2016 03:27 AM, Peter Geoghegan wrote: You shouldn't really waste 8% of the budget with low work_mem settings with my cap patch applied, because the new cap never limits the number of tapes. IIRC, the cap of 500 tapes doesn't start to matter until you have about 1GB of work_mem. So, if

Re: [HACKERS] B-tree index row size limit

2016-10-10 Thread Heikki Linnakangas
On 10/09/2016 02:39 PM, Florian Weimer wrote: What it would it take to eliminate the B-tree index row size limit (or rather, increase it to several hundred megabytes)? I don't care about performance for index-based lookups for overlong columns, I just want to be able to load arbitrary data and

Re: [HACKERS] proposal: psql \setfileref

2016-10-10 Thread Gilles Darold
Le 09/10/2016 à 11:48, Pavel Stehule a écrit : > hi > > 2016-10-04 9:18 GMT+02:00 Gilles Darold >: > > Le 03/10/2016 à 23:23, Gilles Darold a écrit : > > Le 03/10/2016 à 23:03, Robert Haas a écrit : > >> On Mon, Oct 3, 2016

Re: [HACKERS] Logical tape pause/resume

2016-10-10 Thread Heikki Linnakangas
Here are freshly rebased versions of these patches. No big changes, but edited some comments slightly. - Heikki >From 324706fb2e15facbc5686e98ee26bbdc5017366a Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas Date: Mon, 10 Oct 2016 10:59:38 +0300 Subject: [PATCH

Re: [HACKERS] De-support SCO OpenServer/SCO UnixWare?

2016-10-10 Thread Thomas Munro
On Tue, Oct 11, 2016 at 11:03 AM, Tom Lane wrote: > Currently, we have the following OS-specific configure templates in > src/template: > > aix > cygwin > darwin > freebsd > hpux > linux > netbsd > openbsd > sco > solaris > unixware > win32 > > There is at least one active

[HACKERS] parallel.sgml

2016-10-10 Thread Tatsuo Ishii
While reading parallel.sgml, I met with following sentence. If this occurs, the leader will execute the portion of the plan between below the Gather node entirely by itself, almost as if the Gather node were not present. Maybe "the portion of the plan between below" shoud have been

Re: [HACKERS] Merge Join with an Index Optimization

2016-10-10 Thread Michael Malis
I discovered that the kind of join I proposed is called the leapfrog triejoin: https://arxiv.org/pdf/1210.0481v5.pdf

Re: [HACKERS] memory leak in e94568ecc10 (pre-reading in external sort)

2016-10-10 Thread Peter Geoghegan
On Mon, Oct 10, 2016 at 5:21 AM, Heikki Linnakangas wrote: > Admittedly that's confusing. Thinking about this some more, I came up with > the attached. I removed the separate LogicalTapeAssignReadBufferSize() call > altogether - the read buffer size is now passed as argument to

[HACKERS] De-support SCO OpenServer/SCO UnixWare?

2016-10-10 Thread Tom Lane
Currently, we have the following OS-specific configure templates in src/template: aix cygwin darwin freebsd hpux linux netbsd openbsd sco solaris unixware win32 There is at least one active buildfarm critter testing each of these except "sco" and "unixware". I can't find any evidence in the

Re: [HACKERS] FSM corruption leading to errors

2016-10-10 Thread Michael Paquier
On Mon, Oct 10, 2016 at 11:41 PM, Pavan Deolasee wrote: > > > On Mon, Oct 10, 2016 at 7:55 PM, Michael Paquier > wrote: >> >> >> >> + /* >> +* See comments in GetPageWithFreeSpace about handling outside the >> valid >> +* range

Re: [HACKERS] PL/Python adding support for multi-dimensional arrays

2016-10-10 Thread Dave Cramer
On 10 October 2016 at 13:42, Pavel Stehule wrote: > > > 2016-10-10 12:31 GMT+02:00 Heikki Linnakangas : > >> On 10/01/2016 02:45 AM, Jim Nasby wrote: >> >>> On 9/29/16 1:51 PM, Heikki Linnakangas wrote: >>> Now, back to multi-dimensional arrays. I

[HACKERS] Re: "Re: Question about grant create on database and pg_dump/pg_dumpall

2016-10-10 Thread Noah Misch
On Thu, Sep 29, 2016 at 11:29:09AM -0400, Tom Lane wrote: > The fundamental thing we have to do in order to move forward on this is > to rethink what's the division of labor between pg_dump and pg_dumpall. > I find the patch as presented quite unacceptable because it's made no > effort to do that

Re: [HACKERS] Autovacuum launcher process launches worker process at high frequency

2016-10-10 Thread Masahiko Sawada
On Thu, Oct 6, 2016 at 12:11 AM, Jeff Janes wrote: > On Wed, Oct 5, 2016 at 7:28 AM, Masahiko Sawada > wrote: >> >> Hi all, >> >> I found the kind of strange behaviour of the autovacuum launcher >> process when XID anti-wraparound vacuum. >> >>

Re: [HACKERS] Hash Indexes

2016-10-10 Thread Amit Kapila
On Mon, Oct 10, 2016 at 10:07 PM, Jeff Janes wrote: > On Mon, Oct 10, 2016 at 5:55 AM, Amit Kapila > wrote: >> >> On Thu, Sep 29, 2016 at 8:27 PM, Amit Kapila >> wrote: >> > On Thu, Sep 29, 2016 at 6:04 AM, Robert Haas

Re: [HACKERS] proposal: psql \setfileref

2016-10-10 Thread Pavel Stehule
2016-10-10 19:50 GMT+02:00 Pavel Stehule : > > > 2016-10-10 15:17 GMT+02:00 Gilles Darold : > >> Le 10/10/2016 à 14:38, Daniel Verite a écrit : >> > Gilles Darold wrote: >> > >> >>postgres=# \setfileref b /dev/random >> >>postgres=#

Re: [HACKERS] PL/Python adding support for multi-dimensional arrays

2016-10-10 Thread Heikki Linnakangas
On 10/10/2016 08:42 PM, Pavel Stehule wrote: 2016-10-10 12:31 GMT+02:00 Heikki Linnakangas : On 10/01/2016 02:45 AM, Jim Nasby wrote: On 9/29/16 1:51 PM, Heikki Linnakangas wrote: Now, back to multi-dimensional arrays. I can see that the Sequence representation is

Re: [HACKERS] multivariate statistics (v19)

2016-10-10 Thread Tomas Vondra
Hi everyone, thanks for the reviews. Let me sum the feedback so far, and outline my plans for the next patch version that I'd like to submit for CF 2016-11. 1) syntax changes I agree with the changes proposed by Dean, although only a subset of the syntax is going to be supported until we

Re: [HACKERS] PL/Python adding support for multi-dimensional arrays

2016-10-10 Thread Pavel Stehule
2016-10-11 7:49 GMT+02:00 Heikki Linnakangas : > On 10/10/2016 08:42 PM, Pavel Stehule wrote: > >> 2016-10-10 12:31 GMT+02:00 Heikki Linnakangas : >> >> On 10/01/2016 02:45 AM, Jim Nasby wrote: >>> >>> On 9/29/16 1:51 PM, Heikki Linnakangas wrote: Now,

[HACKERS] Comment typo

2016-10-10 Thread Amit Langote
Attached fixes what seems like a copy-pasto in pg_cast.h. Thanks, Amit diff --git a/src/include/catalog/pg_cast.h b/src/include/catalog/pg_cast.h index ee568d8..04d11c0 100644 --- a/src/include/catalog/pg_cast.h +++ b/src/include/catalog/pg_cast.h @@ -59,7 +59,7 @@ typedef enum CoercionCodes

Re: [HACKERS] Un-include access/heapam.h

2016-10-10 Thread Heikki Linnakangas
On 10/04/2016 07:39 AM, Amit Langote wrote: I noticed that un-including access/heapam.h from the following files leaves things just fine. src/backend/commands/aggregatecmds.c src/backend/commands/collationcmds.c src/backend/commands/conversioncmds.c src/backend/commands/lockcmds.c It seems any

Re: [HACKERS] PL/Python adding support for multi-dimensional arrays

2016-10-10 Thread Heikki Linnakangas
On 10/01/2016 02:45 AM, Jim Nasby wrote: On 9/29/16 1:51 PM, Heikki Linnakangas wrote: Now, back to multi-dimensional arrays. I can see that the Sequence representation is problematic, with arrays, because if you have a python list of lists, like [[1, 2]], it's not immediately clear if that's a

Re: [HACKERS] pgbench vs. wait events

2016-10-10 Thread Bruce Momjian
On Thu, Oct 6, 2016 at 02:38:56PM -0400, Robert Haas wrote: > Hi, > > I decided to do some testing on hydra (IBM-provided community > resource, POWER, 16 cores/64 threads, kernel 3.2.6-3.fc16.ppc64) using > the newly-enhanced wait event stuff to try to get an idea of what > we're waiting for

Re: [HACKERS] cygwin64 assertion failure

2016-10-10 Thread Amit Kapila
On Sun, Oct 9, 2016 at 10:23 PM, Andrew Dunstan wrote: > lorikeet seems to be stuck running the parallel tests, after having failed > an assertion. > It looks like this problem has been discussed before [1]. Shall we do what Tom has suggested there? [1] -

Re: [HACKERS] autonomous transactions

2016-10-10 Thread Merlin Moncure
On Thu, Oct 6, 2016 at 3:53 PM, Simon Riggs wrote: > On 6 October 2016 at 21:27, Robert Haas wrote: >> I think we should implement background transactions and call them >> background transactions. That allows us to expose additional >> functionality

Re: [HACKERS] cygwin64 assertion failure

2016-10-10 Thread Tom Lane
Amit Kapila writes: > On Sun, Oct 9, 2016 at 10:23 PM, Andrew Dunstan wrote: >> lorikeet seems to be stuck running the parallel tests, after having failed >> an assertion. > It looks like this problem has been discussed before [1]. Shall we do >

Re: [HACKERS] vacuumdb -f and -j options (was Question / requests.)

2016-10-10 Thread Francisco Olarte
On Mon, Oct 10, 2016 at 3:04 PM, Amit Kapila wrote: > On Sun, Oct 9, 2016 at 10:59 PM, Francisco Olarte > wrote: >> For me -f & -j is not perfect, but better than not having it. It can >> deadlock when given certain sets of catalog tables, either

Re: [HACKERS] PostgreSQL - Weak DH group

2016-10-10 Thread Heikki Linnakangas
On 10/06/2016 10:26 PM, Christoph Berg wrote: Re: Heikki Linnakangas 2016-10-06 I propose the attached patch. It gives up on trying to deal with multiple key lengths (as noted earlier, OpenSSL just always passed keylength=1024, so that was useless).

Re: [HACKERS] FSM corruption leading to errors

2016-10-10 Thread Pavan Deolasee
On Mon, Oct 10, 2016 at 7:55 PM, Michael Paquier wrote: > > > + /* > +* See comments in GetPageWithFreeSpace about handling outside the > valid > +* range blocks > +*/ > + nblocks = RelationGetNumberOfBlocks(rel); > + while (target_block >= nblocks &&

Re: [HACKERS] FSM corruption leading to errors

2016-10-10 Thread Michael Paquier
On Fri, Oct 7, 2016 at 11:50 PM, Anastasia Lubennikova wrote: > Could you please add the patches to commitfest? > I'm going to test them and write a review in a few days. Here you go: https://commitfest.postgresql.org/11/817/ -- Michael -- Sent via pgsql-hackers