[HACKERS] int2vector and btree indexes

If I create btree index on a int2vector column, it does not get used for queries because the query search always fails to match the index operator (family). During index creation, GetDefaultOpClass() returns array_ops for a int2vector index column, because type int2vector is binary-coercible with

Re: [HACKERS] Forbid use of LF and CR characters in database and role names

On Tue, Oct 11, 2016 at 11:00 AM, Noah Misch wrote: > I count one person disfavoring the patch concept of rejecting these characters > early, and I count two people, plus yourself as author, favoring it. > Therefore, the patch can move forward with the proposed design. The next > step, then, is f

Re: [HACKERS] PL/Python adding support for multi-dimensional arrays

2016-10-11 7:49 GMT+02:00 Heikki Linnakangas : > On 10/10/2016 08:42 PM, Pavel Stehule wrote: > >> 2016-10-10 12:31 GMT+02:00 Heikki Linnakangas : >> >> On 10/01/2016 02:45 AM, Jim Nasby wrote: >>> >>> On 9/29/16 1:51 PM, Heikki Linnakangas wrote: Now, back to multi-dimensional arrays. I

Re: [HACKERS] proposal: psql \setfileref

2016-10-10 19:50 GMT+02:00 Pavel Stehule : > > > 2016-10-10 15:17 GMT+02:00 Gilles Darold : > >> Le 10/10/2016 à 14:38, Daniel Verite a écrit : >> > Gilles Darold wrote: >> > >> >>postgres=# \setfileref b /dev/random >> >>postgres=# insert into test (:b); >> >> >> >> Process need to

[HACKERS] Comment typo

Attached fixes what seems like a copy-pasto in pg_cast.h. Thanks, Amit diff --git a/src/include/catalog/pg_cast.h b/src/include/catalog/pg_cast.h index ee568d8..04d11c0 100644 --- a/src/include/catalog/pg_cast.h +++ b/src/include/catalog/pg_cast.h @@ -59,7 +59,7 @@ typedef enum CoercionCodes /*

Re: [HACKERS] PL/Python adding support for multi-dimensional arrays

On 10/10/2016 08:42 PM, Pavel Stehule wrote: 2016-10-10 12:31 GMT+02:00 Heikki Linnakangas : On 10/01/2016 02:45 AM, Jim Nasby wrote: On 9/29/16 1:51 PM, Heikki Linnakangas wrote: Now, back to multi-dimensional arrays. I can see that the Sequence representation is problematic, with arrays,

Re: [HACKERS] Hash Indexes

On Mon, Oct 10, 2016 at 10:07 PM, Jeff Janes wrote: > On Mon, Oct 10, 2016 at 5:55 AM, Amit Kapila > wrote: >> >> On Thu, Sep 29, 2016 at 8:27 PM, Amit Kapila >> wrote: >> > On Thu, Sep 29, 2016 at 6:04 AM, Robert Haas >> > wrote: >> > >> >> Another thing I don't quite understand about this alg

Re: [HACKERS] Autovacuum launcher process launches worker process at high frequency

On Thu, Oct 6, 2016 at 12:11 AM, Jeff Janes wrote: > On Wed, Oct 5, 2016 at 7:28 AM, Masahiko Sawada > wrote: >> >> Hi all, >> >> I found the kind of strange behaviour of the autovacuum launcher >> process when XID anti-wraparound vacuum. >> >> Suppose that a database (say test_db) whose age of f

Re: [HACKERS] multivariate statistics (v19)

Hi everyone, thanks for the reviews. Let me sum the feedback so far, and outline my plans for the next patch version that I'd like to submit for CF 2016-11. 1) syntax changes I agree with the changes proposed by Dean, although only a subset of the syntax is going to be supported until we ad

[HACKERS] Re: "Re: Question about grant create on database and pg_dump/pg_dumpall

On Thu, Sep 29, 2016 at 11:29:09AM -0400, Tom Lane wrote: > The fundamental thing we have to do in order to move forward on this is > to rethink what's the division of labor between pg_dump and pg_dumpall. > I find the patch as presented quite unacceptable because it's made no > effort to do that (

Re: [HACKERS] Supporting huge pages on Windows

From: Thomas Munro [mailto:thomas.mu...@enterprisedb.com] > Your ~2.4% number is similar to what was reported for Linux with 4GB > shared_buffers: > > https://www.postgresql.org/message-id/20130913234125.GC13697%40roobarb > .crazydogs.org I'm relieved to know that a similar figure was gained on L

Re: [HACKERS] Macro customizable hashtable / bitmapscan & aggregation perf

On 10/11/2016 04:07 AM, Andres Freund wrote: On 2016-10-10 17:46:22 -0700, Andres Freund wrote: TPC-DS (tpcds.ods) -- In this case, I'd say the results are less convincing. There are quite a few queries that got slower by ~10%, which is well above - for example queries 22 and 67

Re: [HACKERS] Macro customizable hashtable / bitmapscan & aggregation perf

On 2016-10-10 17:46:22 -0700, Andres Freund wrote: > > TPC-DS (tpcds.ods) > > -- > > > > In this case, I'd say the results are less convincing. There are quite a few > > queries that got slower by ~10%, which is well above - for example queries > > 22 and 67. There are of course qu

Re: [HACKERS] Forbid use of LF and CR characters in database and role names

On Sun, Oct 02, 2016 at 10:47:04PM +0900, Michael Paquier wrote: > On Mon, Sep 12, 2016 at 11:38 AM, Michael Paquier > wrote: > > On Mon, Sep 12, 2016 at 10:01 AM, Noah Misch wrote: > >> I discourage documenting LF/CR restrictions. For the epsilon of readers > >> who > >> would benefit from thi

Re: [HACKERS] Macro customizable hashtable / bitmapscan & aggregation perf

Hi, On 2016-10-11 02:38:26 +0200, Tomas Vondra wrote: > Yes, I've done a bunch of TPC-H and TPC-DS tests on the patch, but it took > quite a bit of time. These tests were done on a fairly small machine (the > usual i5-2500k with 8GB of RAM), with only 1GB data sets for both > benchmarks, to keep i

[HACKERS] another typo in parallel.sgml

I think I found a typo in parallel.sgml. Best regards, -- Tatsuo Ishii SRA OSS, Inc. Japan English: http://www.sraoss.co.jp/index_en.php Japanese:http://www.sraoss.co.jp diff --git a/doc/src/sgml/parallel.sgml b/doc/src/sgml/parallel.sgml index c80d42d..1e71529 100644 --- a/doc/src/sgml/parallel.s

[HACKERS] FTS Configuration option

Hello hackers, Sometimes there is a need to collect lexems from various dictionaries. For example, if we have a column with text in various languages. Let's say there is a new option JOIN. This option will allow to parser to append lexems from current dictionary and go to next dictionary to get a

[HACKERS] How to inspect tuples during execution of a plan?

Hello all, I'd like to inspect the content of tuples as they are sent during the execution of a query in order to react to their values. I guess I could do it with a FDW, but that's a bit clumsy so I took a look at the hooks but have two questions: 1.) Would ExecutorRun_hook be the correct place

[HACKERS] Remove vacuum_defer_cleanup_age

Folks, Given that hot_standby_feedback is pretty bulletproof now, and a lot of the work in reducing replay conflicts, I think the utility of vacuum_defer_cleanup_age is at an end. I really meant so submit a patch to remove it to 9.6, but it got away from me. Any objections to removing the option

Re: [HACKERS] Merge Join with an Index Optimization

I discovered that the kind of join I proposed is called the leapfrog triejoin: https://arxiv.org/pdf/1210.0481v5.pdf

[HACKERS] parallel.sgml

While reading parallel.sgml, I met with following sentence. If this occurs, the leader will execute the portion of the plan between below the Gather node entirely by itself, almost as if the Gather node were not present. Maybe "the portion of the plan between below" shoud have been "t

Re: [HACKERS] FSM corruption leading to errors

On Mon, Oct 10, 2016 at 11:41 PM, Pavan Deolasee wrote: > > > On Mon, Oct 10, 2016 at 7:55 PM, Michael Paquier > wrote: >> >> >> >> + /* >> +* See comments in GetPageWithFreeSpace about handling outside the >> valid >> +* range blocks >> +*/ >> + nblocks = RelationGetNumberOfBlock

Re: [HACKERS] PL/Python adding support for multi-dimensional arrays

On 10 October 2016 at 13:42, Pavel Stehule wrote: > > > 2016-10-10 12:31 GMT+02:00 Heikki Linnakangas : > >> On 10/01/2016 02:45 AM, Jim Nasby wrote: >> >>> On 9/29/16 1:51 PM, Heikki Linnakangas wrote: >>> Now, back to multi-dimensional arrays. I can see that the Sequence representatio

Re: [HACKERS] De-support SCO OpenServer/SCO UnixWare?

On Tue, Oct 11, 2016 at 11:03 AM, Tom Lane wrote: > Currently, we have the following OS-specific configure templates in > src/template: > > aix > cygwin > darwin > freebsd > hpux > linux > netbsd > openbsd > sco > solaris > unixware > win32 > > There is at least one active buildfarm critter testin

[HACKERS] De-support SCO OpenServer/SCO UnixWare?

Currently, we have the following OS-specific configure templates in src/template: aix cygwin darwin freebsd hpux linux netbsd openbsd sco solaris unixware win32 There is at least one active buildfarm critter testing each of these except "sco" and "unixware". I can't find any evidence in the buil

Re: [HACKERS] memory leak in e94568ecc10 (pre-reading in external sort)

On Mon, Oct 10, 2016 at 5:21 AM, Heikki Linnakangas wrote: > Admittedly that's confusing. Thinking about this some more, I came up with > the attached. I removed the separate LogicalTapeAssignReadBufferSize() call > altogether - the read buffer size is now passed as argument to the > LogicalTapeRe

Re: [HACKERS] Switch to unnamed POSIX semaphores as our preferred sema code?

Thomas Munro writes: > On Tue, Oct 11, 2016 at 5:57 AM, Tom Lane wrote: >> (I'm a little suspicious that older variants of FreeBSD might not >> have working sem_init, like the other *BSD variants, necessitating >> a run-time test there. But we'll cross that bridge when we come >> to it.) > The

Re: [HACKERS] Supporting huge pages on Windows

On 2016-10-11 09:57:48 +1300, Thomas Munro wrote: > Later in that thread there was a report of a dramatic ~15% increase in > "best result" TPS, but that was with 60GB of shared_buffers on a > machine with 256GB of RAM: > > https://www.postgresql.org/message-id/20131024060313.GA21888%40toroid.org

Re: [HACKERS] Supporting huge pages on Windows

On Wed, Sep 28, 2016 at 7:32 PM, Tsunakawa, Takayuki wrote: > From: Thomas Munro [mailto:thomas.mu...@enterprisedb.com] >> > huge_pages=off: 70412 tps >> > huge_pages=on : 72100 tps >> >> Hmm. I guess it could be noise or random code rearrangement effects. > > I'm not the difference was a rando

Re: [HACKERS] pg_dump getBlobs query broken for 7.3 servers

On Mon, Oct 10, 2016 at 3:36 AM, Jim Nasby wrote: > FWIW, Greg Stark did a talk at PG Open about PG performance going back to at > least 7.4. He did discuss what he had to do to get those versions to compile > on modern tools, and has a set of patches that enable it. Unfortunately his > slides are

Re: [HACKERS] Logical tape pause/resume

On Sun, Oct 9, 2016 at 11:52 PM, Heikki Linnakangas wrote: > Regardless of the number of tapes, the memory used for the tape buffers, > while building the initial runs, is wasted. It's not entirely wasted when > you have multiple merge passes, because without the buffer you need to read > the last

Re: [HACKERS] Switch to unnamed POSIX semaphores as our preferred sema code?

On Tue, Oct 11, 2016 at 5:57 AM, Tom Lane wrote: > So at this point it seems likely that stopping with Linux and FreeBSD > is the thing to do, and as far as I can tell the code we have now is > working with all variants of those that we have in the buildfarm. > (I'm a little suspicious that older

Re: [HACKERS] pgbench vs. wait events

On Mon, Oct 10, 2016 at 9:44 AM, Bruce Momjian wrote: > On Thu, Oct 6, 2016 at 02:38:56PM -0400, Robert Haas wrote: >> I decided to do some testing on hydra (IBM-provided community >> resource, POWER, 16 cores/64 threads, kernel 3.2.6-3.fc16.ppc64) using >> the newly-enhanced wait event stuff to

Re: [HACKERS] proposal: psql \setfileref

2016-10-10 15:17 GMT+02:00 Gilles Darold : > Le 10/10/2016 à 14:38, Daniel Verite a écrit : > > Gilles Darold wrote: > > > >>postgres=# \setfileref b /dev/random > >>postgres=# insert into test (:b); > >> > >> Process need to be killed using SIGTERM. > > This can be fixed by setting

Re: [HACKERS] PL/Python adding support for multi-dimensional arrays

2016-10-10 12:31 GMT+02:00 Heikki Linnakangas : > On 10/01/2016 02:45 AM, Jim Nasby wrote: > >> On 9/29/16 1:51 PM, Heikki Linnakangas wrote: >> >>> Now, back to multi-dimensional arrays. I can see that the Sequence >>> representation is problematic, with arrays, because if you have a python >>> l

Re: [HACKERS] Switch to unnamed POSIX semaphores as our preferred sema code?

Christoph Berg writes: > Another data point that's admittedly much more of a footnote than > serious input to the original question is the following: Debian has a > (so far mostly toy) port "hurd-i386" which is using the GNU hurd > kernel along with the usual GNU userland that's also in use on Lin

Re: [HACKERS] Hash Indexes

On Mon, Oct 10, 2016 at 5:55 AM, Amit Kapila wrote: > On Thu, Sep 29, 2016 at 8:27 PM, Amit Kapila > wrote: > > On Thu, Sep 29, 2016 at 6:04 AM, Robert Haas > wrote: > > > >> Another thing I don't quite understand about this algorithm is that in > >> order to conditionally lock the target prima

Re: [HACKERS] PostgreSQL - Weak DH group

On 10/06/2016 10:26 PM, Christoph Berg wrote: Re: Heikki Linnakangas 2016-10-06 I propose the attached patch. It gives up on trying to deal with multiple key lengths (as noted earlier, OpenSSL just always passed keylength=1024, so that was useless). Instead of using the callback, it just sets f

Re: [HACKERS] vacuumdb -f and -j options (was Question / requests.)

On Mon, Oct 10, 2016 at 3:04 PM, Amit Kapila wrote: > On Sun, Oct 9, 2016 at 10:59 PM, Francisco Olarte > wrote: >> For me -f & -j is not perfect, but better than not having it. It can >> deadlock when given certain sets of catalog tables, either by making >> it go for the full db or by a pervers

Re: [HACKERS] cygwin64 assertion failure

Amit Kapila writes: > On Sun, Oct 9, 2016 at 10:23 PM, Andrew Dunstan wrote: >> lorikeet seems to be stuck running the parallel tests, after having failed >> an assertion. > It looks like this problem has been discussed before [1]. Shall we do > what Tom has suggested there? > [1] - https://www

Re: [HACKERS] autonomous transactions

On Thu, Oct 6, 2016 at 3:53 PM, Simon Riggs wrote: > On 6 October 2016 at 21:27, Robert Haas wrote: >> I think we should implement background transactions and call them >> background transactions. That allows us to expose additional >> functionality which is useful, like the ability to kick some

Re: [HACKERS] FSM corruption leading to errors

On Mon, Oct 10, 2016 at 7:55 PM, Michael Paquier wrote: > > > + /* > +* See comments in GetPageWithFreeSpace about handling outside the > valid > +* range blocks > +*/ > + nblocks = RelationGetNumberOfBlocks(rel); > + while (target_block >= nblocks && target_block != InvalidBloc

Re: [HACKERS] FSM corruption leading to errors

On Fri, Oct 7, 2016 at 11:50 PM, Anastasia Lubennikova wrote: > Could you please add the patches to commitfest? > I'm going to test them and write a review in a few days. Here you go: https://commitfest.postgresql.org/11/817/ -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@po

Re: [HACKERS] FSM corruption leading to errors

On Mon, Oct 10, 2016 at 11:25 PM, Michael Paquier wrote: > At the same time, I have translated your script into a TAP test, I > found that more useful when testing.. Well... Here is the actual patch. -- Michael diff --git a/src/backend/storage/freespace/freespace.c b/src/backend/storage/freespac

Re: [HACKERS] FSM corruption leading to errors

On Fri, Oct 7, 2016 at 2:59 AM, Pavan Deolasee wrote: > I investigated a bug report from one of our customers and it looked very > similar to previous bug reports here [1], [2], [3] (and probably more). In > these reports, the error looks something like this: > > ERROR: could not read block 28991

Re: [HACKERS] cygwin64 assertion failure

On Sun, Oct 9, 2016 at 10:23 PM, Andrew Dunstan wrote: > lorikeet seems to be stuck running the parallel tests, after having failed > an assertion. > It looks like this problem has been discussed before [1]. Shall we do what Tom has suggested there? [1] - https://www.postgresql.org/message-id/

Re: [HACKERS] pgbench vs. wait events

On Thu, Oct 6, 2016 at 02:38:56PM -0400, Robert Haas wrote: > Hi, > > I decided to do some testing on hydra (IBM-provided community > resource, POWER, 16 cores/64 threads, kernel 3.2.6-3.fc16.ppc64) using > the newly-enhanced wait event stuff to try to get an idea of what > we're waiting for duri

[HACKERS] FTS Configuration option

Hello hackers, Sometimes there is a need to collect lexems from various dictionaries. For example, if we have a column with text in various languages. Let's say there is a new option JOIN. This option will allow to the parser to append lexems from the current dictionary and go to the next dict

Re: [HACKERS] proposal: psql \setfileref

Le 10/10/2016 à 14:38, Daniel Verite a écrit : > Gilles Darold wrote: > >>postgres=# \setfileref b /dev/random >>postgres=# insert into test (:b); >> >> Process need to be killed using SIGTERM. > This can be fixed by setting sigint_interrupt_enabled to true > before operating on the f

Re: [HACKERS] vacuumdb -f and -j options (was Question / requests.)

On Sun, Oct 9, 2016 at 10:59 PM, Francisco Olarte wrote: > On Sat, Oct 8, 2016 at 2:22 PM, Michael Paquier > wrote: >> On Sat, Oct 8, 2016 at 9:12 PM, Amit Kapila wrote: > >>> After reading Francisco's proposal [1], I don't think it is directly >>> trying to make -f and -j work together. He is

Re: [HACKERS] Hash Indexes

On Thu, Sep 29, 2016 at 8:27 PM, Amit Kapila wrote: > On Thu, Sep 29, 2016 at 6:04 AM, Robert Haas wrote: > >> Another thing I don't quite understand about this algorithm is that in >> order to conditionally lock the target primary bucket page, we'd first >> need to read and pin it. And that doe

Re: [HACKERS] pg_upgrade 9.5 -> 9.6 fails when pg_largeobject is in separate tablespace

På søndag 09. oktober 2016 kl. 23:43:23, skrev Robert Haas < robertmh...@gmail.com >: On Sat, Oct 8, 2016 at 9:02 AM, Andreas Joseph Krogh mailto:andr...@visena.com>> wrote: (I've set allow_system_table_mods=on in postgresql.conf) Any configuration that includes thi

Re: [HACKERS] proposal: psql \setfileref

Gilles Darold wrote: >postgres=# \setfileref b /dev/random >postgres=# insert into test (:b); > > Process need to be killed using SIGTERM. This can be fixed by setting sigint_interrupt_enabled to true before operating on the file. Then ctrl-C would be able to cancel the command.

[HACKERS] Using pg_ctl promote -w in TAP tests

Hi all, Now that we have support for the wait mode of pg_ctl promote, I think that it would be a good idea to switch to it in the TAP tests. This allows avoiding extra logic with poll_query_until() to be sure that a promoted standby is ready for read-write queries. See the patch attached. Thanks,

Re: [HACKERS] memory leak in e94568ecc10 (pre-reading in external sort)

On 10/06/2016 06:44 PM, Peter Geoghegan wrote: While the fix you pushed was probably a good idea anyway, I still think you should not use swhichtate->maxTapes to exhaustively call LogicalTapeAssignReadBufferSize() on every tape, even non-active tapes. That's the confusing part. Admittedly that'

Re: [HACKERS] Is it time to kill support for very old servers?

On 10 October 2016 at 10:45, Jim Nasby wrote: > On 10/7/16 1:08 PM, Steve Crawford wrote: >> >> This is effectively a 5-year upgrade "grace period" *after* the EOL date >> of a given version which seems plenty generous. > > > IMHO we need to be careful here. It's not at all unusual to see servers

Re: [HACKERS] Question / requests.

On Mon, Oct 10, 2016 at 4:51 AM, Jim Nasby wrote: > On 10/5/16 9:58 AM, Francisco Olarte wrote: >> Is the system catalog a bottleneck for people who has real use for >> paralell vacuum? I mean, to me someone who does this must have a very >> big db on a big iron. If that does not consist of thousa

Re: [HACKERS] PL/Python adding support for multi-dimensional arrays

On 10/01/2016 02:45 AM, Jim Nasby wrote: On 9/29/16 1:51 PM, Heikki Linnakangas wrote: Now, back to multi-dimensional arrays. I can see that the Sequence representation is problematic, with arrays, because if you have a python list of lists, like [[1, 2]], it's not immediately clear if that's a

Re: [HACKERS] Un-include access/heapam.h

On 10/04/2016 07:39 AM, Amit Langote wrote: I noticed that un-including access/heapam.h from the following files leaves things just fine. src/backend/commands/aggregatecmds.c src/backend/commands/collationcmds.c src/backend/commands/conversioncmds.c src/backend/commands/lockcmds.c It seems any

Re: [HACKERS] B-tree index row size limit

On 10/09/2016 02:39 PM, Florian Weimer wrote: What it would it take to eliminate the B-tree index row size limit (or rather, increase it to several hundred megabytes)? I don't care about performance for index-based lookups for overlong columns, I just want to be able to load arbitrary data and i

Re: [HACKERS] Logical tape pause/resume

Here are freshly rebased versions of these patches. No big changes, but edited some comments slightly. - Heikki >From 324706fb2e15facbc5686e98ee26bbdc5017366a Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas Date: Mon, 10 Oct 2016 10:59:38 +0300 Subject: [PATCH 1/2] Simplify tape block format

Re: [HACKERS] proposal: psql \setfileref

Le 09/10/2016 à 11:48, Pavel Stehule a écrit : > hi > > 2016-10-04 9:18 GMT+02:00 Gilles Darold >: > > Le 03/10/2016 à 23:23, Gilles Darold a écrit : > > Le 03/10/2016 à 23:03, Robert Haas a écrit : > >> On Mon, Oct 3, 2016 at 3:54 PM, Gilles Darold >