Re: [HACKERS] walsender & parallelism

2017-06-02 Thread Andres Freund
On 2017-06-02 23:27:55 -0400, Peter Eisentraut wrote: > On 5/31/17 23:54, Peter Eisentraut wrote: > > On 5/29/17 22:01, Noah Misch wrote: > >> On Tue, May 23, 2017 at 01:45:59PM -0400, Andres Freund wrote: > >>> On May 23, 2017 1:42:41 PM EDT, Petr Je

Re: [HACKERS] PostgreSQL 10 changes in exclusion constraints - did something change? CASE WHEN behavior oddity

2017-06-02 Thread Andres Freund
Hi, On 2017-06-02 22:53:00 -0400, Tom Lane wrote: > I think you've got enough on your plate. I can take care of whatever > we decide to do here. Thanks. > Andres Freund writes: > >> Another possibility is to say that we've broken this situation > >&

Re: [HACKERS] PostgreSQL 10 changes in exclusion constraints - did something change? CASE WHEN behavior oddity

2017-06-02 Thread Andres Freund
irretrievably and we should start throwing errors for SRFs in > places where they'd be conditionally evaluated. That's not real > nice perhaps, but it's better than the way things are right now. I'd be ok with that too, but I don't really see a strong need so far. Gre

Re: [HACKERS] logical replication busy-waiting on a lock

2017-06-02 Thread Andres Freund
Hi, On 2017-06-03 03:03:20 +0200, Petr Jelinek wrote: > All right, here is my rough attempt on doing what Andres has suggested, > going straight from xid to vxid, seems to work fine in my tests and > parallel pg_dump seems happy with it as well. Cool! > The second patch removes the xid paramete

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-06-02 Thread Andres Freund
On 2017-06-01 17:29:12 -0700, Andres Freund wrote: > On 2017-06-02 08:38:51 +0900, Michael Paquier wrote: > > On Fri, Jun 2, 2017 at 7:05 AM, Andres Freund wrote: > > > I'm a unhappy how this is reusing SIGINT for WalSndLastCycleHandler. > > > Normally INT is u

Re: [HACKERS] walsender termination error messages worse in v10

2017-06-02 Thread Andres Freund
Hi Petr, On 2017-06-02 22:57:37 +0200, Petr Jelinek wrote: > On 02/06/17 20:51, Andres Freund wrote: > > I don't understand why the new block is there, nor does the commit > > message explain it. > > > > Hmm, that particular change can actually be reverted. It w

Re: [HACKERS] Error while creating subscription when server is running in single user mode

2017-06-02 Thread Andres Freund
On 2017-06-02 15:00:21 -0400, Peter Eisentraut wrote: > On 6/1/17 21:55, Andres Freund wrote: > > On 2017-06-01 21:42:41 -0400, Peter Eisentraut wrote: > >> We should look at what the underlying problem is before we prohibit > >> anything at a high level. > &

[HACKERS] walsender termination error messages worse in v10

2017-06-02 Thread Andres Freund
(errmsg("could not receive data from WAL stream: %s", pchomp(PQerrorMessage(conn->streamConn); } } note the new /* Verify that there are no more results */ bit. I don't understand why the new block is there, nor does

Re: [HACKERS] Do we need the gcc feature "__builtin_expect" to promote the branches prediction?

2017-06-02 Thread Andres Freund
Hi, On 2017-06-02 16:40:56 +0800, Hao Lee wrote: > Hi all, >There is a lot of "if statement" in system, and GCC provides a > feature,"__builtin_expect", which let compilers know which branch is > mostly run. as we known, miss-prediction will lead the performance > lost(because the CPU wil

Re: [HACKERS] walsender & parallelism

2017-06-01 Thread Andres Freund
On 2017-06-01 22:17:57 -0400, Peter Eisentraut wrote: > On 6/1/17 00:06, Andres Freund wrote: > > On 2017-05-31 23:51:08 -0400, Peter Eisentraut wrote: > >> I think the easiest and safest thing to do now is to just prevent > >> parallel plans in the walsender. See att

Re: [HACKERS] Error while creating subscription when server is running in single user mode

2017-06-01 Thread Andres Freund
On 2017-06-01 21:42:41 -0400, Peter Eisentraut wrote: > We should look at what the underlying problem is before we prohibit > anything at a high level. I'm not sure there's any underlying issue here, except being in single user mode. > When I try it, I get a > > TRAP: FailedAssertion("!(event->

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-06-01 Thread Andres Freund
On 2017-06-02 10:05:21 +0900, Michael Paquier wrote: > On Fri, Jun 2, 2017 at 9:29 AM, Andres Freund wrote: > > On 2017-06-02 08:38:51 +0900, Michael Paquier wrote: > >> On Fri, Jun 2, 2017 at 7:05 AM, Andres Freund wrote: > >> > I'm a unh

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-06-01 Thread Andres Freund
On 2017-06-02 08:38:51 +0900, Michael Paquier wrote: > On Fri, Jun 2, 2017 at 7:05 AM, Andres Freund wrote: > > I'm a unhappy how this is reusing SIGINT for WalSndLastCycleHandler. > > Normally INT is used cancel interrupts, and since walsender is now also > > working

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-06-01 Thread Andres Freund
On 2017-05-05 10:50:11 -0400, Peter Eisentraut wrote: > On 5/5/17 01:26, Michael Paquier wrote: > > The only code path doing HOT-pruning and generating WAL is > > heap_page_prune(). Do you think that we need to worry about FPWs as > > well? > > > > Attached is an updated patch, which also forbids

Re: [HACKERS] [BUGS] Concurrent ALTER SEQUENCE RESTART Regression

2017-06-01 Thread Andres Freund
On 2017-06-01 19:08:33 +0200, Petr Jelinek wrote: > On 01/06/17 16:51, Robert Haas wrote: > > On Wed, May 31, 2017 at 8:07 PM, Andres Freund wrote: > >> Here's a patch doing what I suggested above. The second patch addresses > >> an independent oversight where

Re: [HACKERS] COPY (query) TO ... doesn't allow parallelism

2017-06-01 Thread Andres Freund
On 2017-06-01 15:56:35 -0400, Robert Haas wrote: > > I personally think we should fix this in 9.6 and 10, but I've to admit > > I'm not entirely impartial, because Citus hit this... > > I guess it's a matter of judgement whether you want to call this a bug > or a missing feature. I wasn't really

Re: [HACKERS] logical replication busy-waiting on a lock

2017-06-01 Thread Andres Freund
On 2017-06-01 14:17:44 +0200, Petr Jelinek wrote: > Thinking more about this, I am not convinced it's a good idea to change > exports this late in the cycle. I still think it's best to do the xid > assignment only when the snapshot is actually exported but don't assign > xid when the export is only

Re: [HACKERS] Hash Functions

2017-06-01 Thread Andres Freund
is a good idea. If there ends up being demand for it, we can still go there. I think that the number of people that migrate between architectures is low enough that this isn't going to be a very common issue. Having some feasible way around this is important, but I don't think we sho

Re: [HACKERS] Effect of changing the value for PARALLEL_TUPLE_QUEUE_SIZE

2017-06-01 Thread Andres Freund
On 2017-06-01 18:41:20 +0530, Rafia Sabih wrote: > As per my understanding it looks like this increase in tuple queue > size is helping only gather-merge. Particularly, in the case where it > is enough stalling by master in gather-merge because it is maintaining > the sort-order. Like in q12 the in

Re: [HACKERS] COPY (query) TO ... doesn't allow parallelism

2017-06-01 Thread Andres Freund
On 2017-06-01 21:37:56 +0530, Amit Kapila wrote: > On Thu, Jun 1, 2017 at 9:34 PM, Andres Freund wrote: > > On 2017-06-01 21:23:04 +0530, Amit Kapila wrote: > >> On a related note, I think it might be better to have an > >> IsInParallelMode() check in this case

Re: [HACKERS] COPY (query) TO ... doesn't allow parallelism

2017-06-01 Thread Andres Freund
On 2017-06-01 21:23:04 +0530, Amit Kapila wrote: > On a related note, I think it might be better to have an > IsInParallelMode() check in this case as we have at other places. > This is to ensure that if this command is invoked via plpgsql function > and that function runs is the parallel mode, it

Re: [HACKERS] walsender & parallelism

2017-05-31 Thread Andres Freund
On 2017-05-31 23:51:08 -0400, Peter Eisentraut wrote: > I think the easiest and safest thing to do now is to just prevent > parallel plans in the walsender. See attached patch. This prevents the > hang in the select_parallel tests run under your new test setup. I'm not quite sure I can buy this.

Re: [HACKERS] Replication origins and timelines

2017-05-31 Thread Andres Freund
which > you describe? Yes, unless you manipulate things by hand, copying files around or such. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Replication origins and timelines

2017-05-31 Thread Andres Freund
On 2017-05-31 21:33:26 -0400, Stephen Frost wrote: > > This only starts becoming an issue once logical replication slots can > > exist on replicas and be maintained to follow the master's slot state. > > Which is incomplete in Pg10 (not exposed to users) but I plan to > > finish getting in for pg11

Re: [HACKERS] Replication origins and timelines

2017-05-31 Thread Andres Freund
Hi, On 2017-06-01 09:12:04 +0800, Craig Ringer wrote: > TL;DR: replication origins track LSN without timeline. This is > ambiguous when physical failover is present since / > can now represent more than one state due to timeline forks with > promotions. Replication origins should t

[HACKERS] tap tests on older branches fail if concurrency is used

2017-05-31 Thread Andres Freund
database and repeat your command. ... it appears as if various tests are trampling over each other. If needed I can provide detailed logs, but it appears to readily reproduce on several machines... See Michael, I'll provide the details and a reproducer ;) Greetings, Andres Freund

Re: [HACKERS] [BUGS] Concurrent ALTER SEQUENCE RESTART Regression

2017-05-31 Thread Andres Freund
On 2017-05-31 14:24:38 -0700, Andres Freund wrote: > On 2017-05-24 10:52:37 -0400, Robert Haas wrote: > > On Wed, May 24, 2017 at 10:32 AM, Andres Freund wrote: > > > Well, but then we should just remove minval/maxval if we can't rely on > > > it. > > > &g

[HACKERS] COPY (query) TO ... doesn't allow parallelism

2017-05-31 Thread Andres Freund
use COPY to execute reports to a CSV file and such. Robert, am I missing a complication? I personally think we should fix this in 9.6 and 10, but I've to admit I'm not entirely impartial, because Citus hit this... Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-h

Re: [HACKERS] pg_class.relpartbound definition overly brittle

2017-05-31 Thread Andres Freund
On 2017-05-31 15:38:58 -0700, Mark Dilger wrote: > > Normal users aren't going to make sense of node trees in the first > > place. You should use pg_get_expr for it: > > postgres[3008][1]=# SELECT pg_get_expr(relpartbound, oid) FROM pg_class > > WHERE relpartbound IS NOT NULL; > > ┌──

Re: [HACKERS] pg_class.relpartbound definition overly brittle

2017-05-31 Thread Andres Freund
On 2017-05-31 15:06:06 -0700, Mark Dilger wrote: > That's cold comfort, given that most users will be looking at the pg_class > table and not writing C code that compares Node objects. I wrote a bit of > regression test logic that checks, and sure enough the relpartbound field > shows up as unequa

Re: [HACKERS] [BUGS] Concurrent ALTER SEQUENCE RESTART Regression

2017-05-31 Thread Andres Freund
On 2017-05-24 10:52:37 -0400, Robert Haas wrote: > On Wed, May 24, 2017 at 10:32 AM, Andres Freund wrote: > > Well, but then we should just remove minval/maxval if we can't rely on > > it. > > That seems like a drastic overreaction to me. Well, either they work, or

Re: [HACKERS] POC: Sharing record typmods between backends

2017-05-31 Thread Andres Freund
On May 31, 2017 11:28:18 AM PDT, Robert Haas wrote: >On Wed, May 31, 2017 at 1:46 PM, Andres Freund >wrote: >> On 2017-05-31 13:27:28 -0400, Dilip Kumar wrote: >>> On Wed, May 31, 2017 at 12:53 PM, Robert Haas > wrote: >>> > Well, SH_TYPE's members SH_E

Re: [HACKERS] Re: [GENERAL] pg_basebackup error: replication slot "pg_basebackup_2194" already exists

2017-05-31 Thread Andres Freund
On 2017-05-31 18:22:18 +0200, Magnus Hagander wrote: > However, the client can't access the pid of the server as it is now, > and its the client that has to create the name. I don't think that's actually true? Doesn't the wire protocol always include the PID, which is then exposed with PQbackendP

Re: [HACKERS] [PATCH] relocation truncated to fit: citus build failure on s390x

2017-05-31 Thread Andres Freund
On 2017-05-31 11:57:16 -0400, Alvaro Herrera wrote: > My vote would be to backpatch it all the way. +1 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] POC: Sharing record typmods between backends

2017-05-31 Thread Andres Freund
On 2017-05-31 13:27:28 -0400, Dilip Kumar wrote: > On Wed, May 31, 2017 at 12:53 PM, Robert Haas wrote: > > Well, SH_TYPE's members SH_ELEMENT_TYPE *data and void *private_data > > are not going to work in DSM, because they are pointers. You can > > doubtless come up with a way around that proble

Re: [HACKERS] logical replication busy-waiting on a lock

2017-05-31 Thread Andres Freund
On 2017-05-29 23:49:33 +0200, Petr Jelinek wrote: > I am not quite sure I understand (both the vxid suggestion and for the > session dying badly). Maybe we can discuss bit more when you get to > computer so it's easier for you to expand on what you mean. The xid interlock when exporting a snapshot

Re: [HACKERS] Segmentation fault when creating a BRIN, 10beta1

2017-05-30 Thread Andres Freund
On 2017-05-30 18:21:10 -0400, Alvaro Herrera wrote: > Alexander Sosna wrote: > > Hi, > > > > I can reproduce a segmentation fault when creating a BRIN concurrently > > with set pages_per_range and autosummarize. > > Pushed fix just now. Please give it a try. Thanks for testing and > reporting,

Re: [HACKERS] Effect of changing the value for PARALLEL_TUPLE_QUEUE_SIZE

2017-05-30 Thread Andres Freund
On 2017-05-30 07:27:12 -0400, Robert Haas wrote: > The other is that I figured 64k was small enough that nobody would > care about the memory utilization. I'm not sure we can assume the > same thing if we make this bigger. It's probably fine to use a 6.4M > tuple queue for each worker if work_mem

Re: [HACKERS] [PATCH] relocation truncated to fit: citus build failure on s390x

2017-05-30 Thread Andres Freund
On 2017-05-29 15:45:11 -0400, Tom Lane wrote: > Christoph Berg writes: > > Re: To Andres Freund 2016-04-28 <20160428080824.ga22...@msg.df7cb.de> > >>> I'm not clear why citus triggers this, when other extensions don't? > > >> Maybe it'

Re: [HACKERS] logical replication busy-waiting on a lock

2017-05-29 Thread Andres Freund
On May 29, 2017 12:41:26 PM PDT, Petr Jelinek wrote: >On 29/05/17 21:28, Andres Freund wrote: >> >> On May 29, 2017 12:25:35 PM PDT, Petr Jelinek > wrote: >>> >>> Looking at the code more, the xid is only used as parameter for >>> SnapBuildBuildS

Re: [HACKERS] logical replication busy-waiting on a lock

2017-05-29 Thread Andres Freund
On May 29, 2017 12:25:35 PM PDT, Petr Jelinek wrote: >On 29/05/17 21:21, Petr Jelinek wrote: >> On 29/05/17 20:59, Andres Freund wrote: >>> >>> >>> On May 29, 2017 11:58:05 AM PDT, Petr Jelinek > wrote: >>>> On 27/05/17 17:17, Andres Freund

Re: [HACKERS] Use of non-restart-safe storage by temp_tablespaces

2017-05-29 Thread Andres Freund
On May 29, 2017 12:15:37 PM PDT, Alvaro Herrera wrote: >Claudio Freire wrote: >> On Mon, May 29, 2017 at 3:53 PM, Bruce Momjian >wrote: >> > Right now we don't document that temp_tablespaces can use >> > non-restart-safe storage, e.g. /tmp, ramdisks. Would this be safe? >> > Should we documen

Re: [HACKERS] logical replication busy-waiting on a lock

2017-05-29 Thread Andres Freund
On May 29, 2017 12:21:50 PM PDT, Petr Jelinek wrote: >On 29/05/17 20:59, Andres Freund wrote: >> >> >> On May 29, 2017 11:58:05 AM PDT, Petr Jelinek > wrote: >>> On 27/05/17 17:17, Andres Freund wrote: >>>> >>>> >>>> On Ma

Re: [HACKERS] logical replication busy-waiting on a lock

2017-05-29 Thread Andres Freund
On May 29, 2017 11:58:05 AM PDT, Petr Jelinek wrote: >On 27/05/17 17:17, Andres Freund wrote: >> >> >> On May 27, 2017 9:48:22 AM EDT, Petr Jelinek > wrote: >>> Actually, I guess it's the pid 47457 (COPY process) who is actually >>> running

Re: [HACKERS] logical replication busy-waiting on a lock

2017-05-29 Thread Andres Freund
On 2017-05-29 11:38:20 -0700, Jeff Janes wrote: > Related, but not the same. It would be nice if they didn't block, but if > they do have to block, shouldn't it wait on a semaphore, rather than doing > a tight loop? It looks like maybe a latch didn't get reset when it should > have or something.

Re: [HACKERS] Broken hint bits (freeze)

2017-05-27 Thread Andres Freund
On 2017-05-27 19:48:24 +0300, Vladimir Borodin wrote: > Well, actually clean shutdown of master with exit code 0 from `pg_ctl > stop -m fast` guarantees that all WAL has been replicated to standby. It does not. It makes it likely, but the connection to the standby could be not up just then, you c

Re: [HACKERS] logical replication busy-waiting on a lock

2017-05-27 Thread Andres Freund
On May 27, 2017 9:48:22 AM EDT, Petr Jelinek wrote: >Actually, I guess it's the pid 47457 (COPY process) who is actually >running the xid 73322726. In that case that's the same thing Masahiko >Sawada reported [1]. Which basically is result of snapshot builder >waiting for transaction to finish,

Re: [HACKERS] logical replication - still unstable after all these months

2017-05-27 Thread Andres Freund
On May 27, 2017 6:13:19 AM EDT, Simon Riggs wrote: >On 27 May 2017 at 09:44, Erik Rijkers wrote: > >> I am very curious at your results. > >We take your bug report on good faith, but we still haven't seen >details of the problem or how to recreate it. > >Please post some details. Thanks. ? --

Re: [HACKERS] No parameter values checking while creating Alter subscription...Connection

2017-05-25 Thread Andres Freund
On 2017-05-25 17:08:57 -0400, Peter Eisentraut wrote: > On 5/25/17 10:18, Masahiko Sawada wrote: > >> postgres=# alter subscription c1 connection 'port=4000'; > >> ALTER SUBSCRIPTION > >> postgres=# alter subscription c1 connection 'dbname=cc'; > >> ALTER SUBSCRIPTION > >> > > CREATE SUBSCRIPTION t

Re: [HACKERS] Surjective functional indexes

2017-05-25 Thread Andres Freund
On 2017-05-25 12:37:40 -0400, Tom Lane wrote: > Konstantin Knizhnik writes: > > My proposal is to check value of function for functional indexes instead > > of just comparing set of effected attributes. > > Obviously, for some complex functions it may have negative effect on > > update speed. >

Re: [HACKERS] generate_series regression 9.6->10

2017-05-24 Thread Andres Freund
topped. Going on forward, once either SRF stops returning rows, it'll return NULL until all SRFs are exhausted. Makes sense? Is that a problem for you? If so, what do you use the LCM logic for in practical terms? Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (

Re: [HACKERS] [BUGS] Concurrent ALTER SEQUENCE RESTART Regression

2017-05-24 Thread Andres Freund
On 2017-05-24 10:24:19 -0400, Robert Haas wrote: > On Wed, May 24, 2017 at 9:04 AM, Andres Freund wrote: > > At the very least we'll have to error out. That's still not nice usability > > wise, but it sure beats returning flat out wrong values. > > I'm not s

Re: [HACKERS] Index created in BEFORE trigger not updated during INSERT

2017-05-24 Thread Andres Freund
On 2017-05-24 08:26:24 -0400, Robert Haas wrote: > On Mon, May 22, 2017 at 7:05 AM, Albe Laurenz wrote: > > Not that it is a useful use case, but I believe that this is > > a bug that causes index corruption: > > > > CREATE TABLE mytable( > >id integer PRIMARY KEY, > >id2 integer NOT NULL

Re: [HACKERS] [BUGS] Concurrent ALTER SEQUENCE RESTART Regression

2017-05-24 Thread Andres Freund
On May 24, 2017 6:57:08 AM EDT, Robert Haas wrote: >On Tue, May 23, 2017 at 11:25 PM, Andres Freund >wrote: >> On 2017-05-23 22:47:07 -0400, Robert Haas wrote: >>> On Mon, May 22, 2017 at 11:42 AM, Andres Freund >wrote: >>> > Ooops. >>> > >&g

Re: [HACKERS] [BUGS] Concurrent ALTER SEQUENCE RESTART Regression

2017-05-23 Thread Andres Freund
On 2017-05-23 22:47:07 -0400, Robert Haas wrote: > On Mon, May 22, 2017 at 11:42 AM, Andres Freund wrote: > > Ooops. > > > > Two issues: Firstly, we get a value smaller than seqmin, obviously > > that's not ok. But even if we'd error out, it'd imo stil

Re: [HACKERS] walsender & parallelism

2017-05-23 Thread Andres Freund
On May 23, 2017 1:42:41 PM EDT, Petr Jelinek wrote: >Hi, > >so this didn't really move anywhere AFAICS, do we think the approach >I've chosen is good or do we want to do something else here? Can you add it to the open items list? Andres -- Sent from my Android device with K-9 Mail. Please ex

Re: [HACKERS] Improve logical decoding error message (was wal_level > WAL_LEVEL_LOGICAL)

2017-05-23 Thread Andres Freund
On 2017-05-23 10:49:54 +, Neha Khatri wrote: > On Tue, 23 May 2017 at 10:55 am, Alvaro Herrera > wrote: > > > Neha Khatri wrote: > > > On Tue, May 23, 2017 at 10:26 AM, Michael Paquier < > > michael.paqu...@gmail.com > > > > > > There is no wal_level higher than logical, so the current sense

[HACKERS] plpgsql, caching, resowners and jit

2017-05-22 Thread Andres Freund
Hi, While hacking a bit more on my JIT prototype (so I actually know what I'm talking about at pgcon), I encountered an interesting issue. To keep track of the lifetime of JITed functions it seems natural to add support for that to resowners. Not that hard. After doing so, I got pretty weird cr

Re: [HACKERS] [BUGS] Concurrent ALTER SEQUENCE RESTART Regression

2017-05-22 Thread Andres Freund
On 2017-05-19 08:31:15 -0400, Robert Haas wrote: > On Thu, May 18, 2017 at 4:54 PM, Andres Freund wrote: > > There's still weird behaviour, unfortunately. If you do an ALTER > > SEQUENCE changing minval/maxval w/ restart in a transaction, and abort, > > you'll a

Re: [HACKERS] Making replication commands case-insensitive

2017-05-20 Thread Andres Freund
t; times when running tests using psql.. -1. This solution is too bad for maintainability, and given the replication protocol is not for interactive use I don't see a corresponding benefit. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgre

Re: [HACKERS] Preliminary results for proposed new pgindent implementation

2017-05-19 Thread Andres Freund
On 2017-05-19 12:21:52 -0400, Robert Haas wrote: > On Fri, May 19, 2017 at 11:22 AM, Tom Lane wrote: > > I certainly would rather that our version matched something that's under > > active maintenance someplace. But it seems like there are two good > > arguments for having a copy in our tree: > >

Re: [HACKERS] [BUGS] Concurrent ALTER SEQUENCE RESTART Regression

2017-05-18 Thread Andres Freund
On 2017-05-15 10:34:02 -0400, Peter Eisentraut wrote: > On 5/10/17 09:12, Michael Paquier wrote: > > Looking at 0001 and 0002... So you are correctly blocking nextval() > > when ALTER SEQUENCE holds a lock on the sequence object. And > > concurrent calls of nextval() don't conflict. As far as I can

Re: [HACKERS] 10beta1 sequence regression failure on sparc64

2017-05-18 Thread Andres Freund
On 2017-05-18 10:11:48 -0400, Tom Lane wrote: > Christoph Berg writes: > > *** 34,44 > > --- 34,47 > > CREATE SEQUENCE sequence_test7 AS bigint; > > CREATE SEQUENCE sequence_test8 AS integer MAXVALUE 10; > > CREATE SEQUENCE sequence_test9 AS integer INCREMENT BY -1; > > + ERROR

Re: [HACKERS] Cached plans and statement generalization

2017-05-18 Thread Andres Freund
On 2017-05-18 11:57:57 +0300, Konstantin Knizhnik wrote: > From my own experience I found out that PG_TRY/PG_CATCH mechanism is not > providing proper cleanup (unlike C++ exceptions). Right, simply because there's no portable way to transparently do so. Would be possible on elf glibc platforms, bu

Re: [HACKERS] WIP Patch: Precalculate stable functions, infrastructure v1

2017-05-18 Thread Andres Freund
urther, so build > + * and return a replacement CachedExpr > node using the > + * possibly-simplified arguments of > subexpression. > + */ Is this actually a meaningful path? Shouldn't always ha

Re: [HACKERS] 10beta1/m68k: static assertion failed: "MAXALIGN too small to fit int32"

2017-05-18 Thread Andres Freund
On 2017-05-18 10:48:48 +0300, Heikki Linnakangas wrote: > If that's all that prevents it from working, by all means let's fix it. I > think this should do it, although I don't have a system to test it on: Yes, that's what I thought about doing too. > It adds a few instructions to check that on a

Re: pgindent (was Re: [HACKERS] [COMMITTERS] pgsql: Preventive maintenance in advance of pgindent run.)

2017-05-17 Thread Andres Freund
On 2017-05-17 14:44:31 -0400, Tom Lane wrote: > $ tar xfz ~/archive/pg_bsd_indent-1.3.tar.gz > $ wc pg_bsd_indent/* > 38122928 pg_bsd_indent/Makefile >107831 4835 pg_bsd_indent/README >508 1743 11988 pg_bsd_indent/args.c >569 2727 14732 pg_bsd_indent/indent.1 >

Re: pgindent (was Re: [HACKERS] [COMMITTERS] pgsql: Preventive maintenance in advance of pgindent run.)

2017-05-17 Thread Andres Freund
On 2017-05-17 13:35:22 -0400, Tom Lane wrote: > Not sure about actually incorporating it into our repo. Doing so would > make it easier for people to use, for sure, and the license seems to be > regular 3-clause BSD, so that angle is OK. But do we want to be carrying > around another 150K of sour

Re: [HACKERS] PG10 pgindent run

2017-05-16 Thread Andres Freund
On 2017-05-16 21:02:19 -0400, Tom Lane wrote: > Bruce Momjian writes: > > OK, I assume we are good to go for Wednesday afternoon, UTC. Thanks for > > the research. > > Yeah, I think we're ready, unless anyone has a large patch they want > to stick in first ... I've this pending JIT support...

Re: [HACKERS] PG10 pgindent run

2017-05-16 Thread Andres Freund
On 2017-05-16 18:56:57 -0400, Tom Lane wrote: > Andres Freund writes: > > No clue if there's some switch that needs to be toggled on the buildfarm > > side to accept the typedefs, I've never looked at that side of things. > > AFAIK, not; I think it just takes any

Re: [HACKERS] PG10 pgindent run

2017-05-16 Thread Andres Freund
On 2017-05-16 18:43:22 -0400, Tom Lane wrote: > Specifically, we don't seem to have entries for any of the typedefs > associated with the ICU code, eg UChar. This is not terribly > surprising since none of the buildfarm critters contributing typedef > lists are building with --with-icu. It looks

Re: [HACKERS] snapbuild woes

2017-05-14 Thread Andres Freund
On 2017-05-14 14:54:37 +0200, Petr Jelinek wrote: > On 13/05/17 22:23, Andres Freund wrote: > >> And wait for session 3 to finish slot creation, takes about 20 mins on > >> my laptop without patches, minute and half with your patches for 0004 > >> and 0005 (or with yo

Re: [HACKERS] Hash Functions

2017-05-14 Thread Andres Freund
st think it'd be a very bad idea to make it work for any non-trivial ones. The likelihood of reguarly breaking or preventing us from improving things seems high. I'm not sure that having a design where this most of the time works for some datatypes is a good one. Greetings, Andres Freund -

Re: [HACKERS] Hash Functions

2017-05-14 Thread Andres Freund
amount of lock-in that'd create, especially for more complex datatypes, seems wholly inadvisable. I still think that dumping tables in a way they're reloaded via the top-partition (probably one copy statement for each child partition), and prohibiting incoming fkeys to partitions, is

Re: [HACKERS] PG10 pgindent run

2017-05-14 Thread Andres Freund
On 2017-05-14 17:22:16 -0400, Stephen Frost wrote: > * Andres Freund (and...@anarazel.de) wrote: > > On 2017-05-14 09:53:17 -0400, Bruce Momjian wrote: > > > I would like to run pgindent on the head source tree this Wednesday > > > afternoon, UTC. Is that OK for ever

Re: [HACKERS] Hash Functions

2017-05-14 Thread Andres Freund
is unrealistic, is meaningfully affected by disallowing non-IEEE floats. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG10 pgindent run

2017-05-14 Thread Andres Freund
be too late, no? Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Hash Functions

2017-05-13 Thread Andres Freund
On May 13, 2017 8:44:22 PM PDT, Robert Haas wrote: >On Sat, May 13, 2017 at 7:08 PM, Andres Freund >wrote: >> I seriously doubt that's true. A lot of more complex types have >> internal alignment padding and such. > >True, but I believe we require those pad

Re: [HACKERS] Hash Functions

2017-05-13 Thread Andres Freund
On 2017-05-13 10:29:09 -0400, Robert Haas wrote: > On Sat, May 13, 2017 at 12:52 AM, Amit Kapila wrote: > > Can we think of defining separate portable hash functions which can be > > used for the purpose of hash partitioning? > > I think that would be a good idea. I think it shouldn't even be th

Re: [HACKERS] walsender & parallelism

2017-05-13 Thread Andres Freund
On 2017-04-25 11:18:35 -0400, Robert Haas wrote: > On Thu, Apr 20, 2017 at 9:40 PM, Andres Freund wrote: > > This'd be easier to look at if the fairly separate facility of allowing > > walsender to execute SQL commands had been committed separately, rather > > than

Re: [HACKERS] snapbuild woes

2017-05-13 Thread Andres Freund
Asserts in reorderbuffer.c, without it takes less than a second for me here. I'm appylying these now. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Hash Functions

2017-05-12 Thread Andres Freund
locality, by most of the time (same architecture/encoding/etc) having one backend insert into the same partition. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Cached plans and statement generalization

2017-05-12 Thread Andres Freund
On May 12, 2017 12:50:41 AM PDT, Konstantin Knizhnik wrote: >Definitely changing session context (search_path, date/time format, >...) >may cause incorrect behavior of cached statements. >Actually you may get the same problem with explicitly prepared >statements (certainly, in the last case,

Re: [HACKERS] Hash Functions

2017-05-12 Thread Andres Freund
On May 12, 2017 10:05:56 AM PDT, Robert Haas wrote: >On Fri, May 12, 2017 at 12:08 AM, Jeff Davis wrote: >> 1. The hash functions as they exist today aren't portable -- they can >> return different results on different machines. That means using >these >> functions for hash partitioning would y

Re: [HACKERS] snapbuild woes

2017-05-11 Thread Andres Freund
On 2017-05-11 14:54:26 -0700, Andres Freund wrote: > On 2017-05-11 14:51:55 -0700, wrote: > > On 2017-05-08 00:10:12 -0700, Andres Freund wrote: > > > I plan to commit the next pending patch after the back branch releases > > > are cut - it's an invasive fix and th

Re: [HACKERS] snapbuild woes

2017-05-11 Thread Andres Freund
On 2017-04-15 05:18:49 +0200, Petr Jelinek wrote: > >>> From 3318a929e691870f3c1ca665bec3bfa8ea2af2a8 Mon Sep 17 00:00:00 2001 > >>> From: Petr Jelinek > >>> Date: Sun, 26 Feb 2017 01:07:33 +0100 > >>> Subject: [PATCH 3/5] Prevent snapshot builder xmin from going backwards > >> > >> A bit more com

Re: [HACKERS] snapbuild woes

2017-05-11 Thread Andres Freund
On 2017-05-11 14:51:55 -0700, wrote: > On 2017-05-08 00:10:12 -0700, Andres Freund wrote: > > I plan to commit the next pending patch after the back branch releases > > are cut - it's an invasive fix and the issue doesn't cause corruption > > "just" slow

Re: [HACKERS] snapbuild woes

2017-05-11 Thread Andres Freund
On 2017-05-08 00:10:12 -0700, Andres Freund wrote: > I plan to commit the next pending patch after the back branch releases > are cut - it's an invasive fix and the issue doesn't cause corruption > "just" slow slot creation. So it seems better to wait for a few days

Re: [HACKERS] [BUGS] Concurrent ALTER SEQUENCE RESTART Regression

2017-05-11 Thread Andres Freund
ove, and I got runtimes of just about 12 seconds, which surprised > me a bit. Yea, I suspect we'd need something like the current open_share_lock, except with the resowner trickery removed. > I'd have thought the locallock-already-exists code path would be > faster than that

Re: [HACKERS] [BUGS] Concurrent ALTER SEQUENCE RESTART Regression

2017-05-11 Thread Andres Freund
Hi, On 2017-05-11 16:27:48 -0400, Peter Eisentraut wrote: > On 5/10/17 12:24, Andres Freund wrote: > > Upthread I theorized whether > > that's actually still meaningful given fastpath locking and such, but I > > guess we'll have to evaluate that. > >

Re: [HACKERS] [BUGS] Concurrent ALTER SEQUENCE RESTART Regression

2017-05-11 Thread Andres Freund
On 2017-05-11 11:35:22 -0400, Peter Eisentraut wrote: > On 5/10/17 12:24, Andres Freund wrote: > > The issue isn't the strength, but that we currently have this weird > > hackery around open_share_lock(): > > /* > > * Open the sequence and acquire AccessShareLo

Re: [HACKERS] Cached plans and statement generalization

2017-05-11 Thread Andres Freund
On 2017-05-11 22:48:26 +0300, Konstantin Knizhnik wrote: > On 05/11/2017 09:31 PM, Tom Lane wrote: > > Bruce Momjian writes: > > > Good point. I think we need to do some measurements to see if the > > > parser-only stage is actually significant. I have a hunch that > > > commercial databases hav

Re: [HACKERS] Cached plans and statement generalization

2017-05-11 Thread Andres Freund
On May 11, 2017 11:31:02 AM PDT, Tom Lane wrote: >Bruce Momjian writes: >> Good point. I think we need to do some measurements to see if the >> parser-only stage is actually significant. I have a hunch that >> commercial databases have much heavier parsers than we do. > >FWIW, gram.y does sho

Re: [HACKERS] Time based lag tracking for logical replication

2017-05-11 Thread Andres Freund
On May 11, 2017 8:08:11 AM PDT, Simon Riggs wrote: >On 11 May 2017 at 14:12, Petr Jelinek >wrote: > >>> Attached patch is Petr's patch, slightly rebased with added pacing >>> delay, similar to that used by HSFeedback. >>> >> >> This looks reasonable. I would perhaps change: >>> + /* >>> +

Re: [HACKERS] Transaction held open by autoanalyze can be a bottleneck

2017-05-10 Thread Andres Freund
064%40alap3.anarazel.de > Does analyze need all of its work done under the same transaction? It's imo, as pointed out in the above email, not trivial to change it, but it's imo doable. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)

Re: [HACKERS] [BUGS] Concurrent ALTER SEQUENCE RESTART Regression

2017-05-10 Thread Andres Freund
hread I theorized whether that's actually still meaningful given fastpath locking and such, but I guess we'll have to evaluate that. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [BUGS] Concurrent ALTER SEQUENCE RESTART Regression

2017-05-08 Thread Andres Freund
On 2017-05-08 12:28:38 -0400, Peter Eisentraut wrote: > On 5/8/17 02:58, Noah Misch wrote: > > IMMEDIATE ATTENTION REQUIRED. This PostgreSQL 10 open item is long past due > > for your status update. Please reacquaint yourself with the policy on open > > item ownership[1] and then reply immediatel

Re: [HACKERS] snapbuild woes

2017-05-08 Thread Andres Freund
On May 3, 2017 10:45:16 PM PDT, Noah Misch wrote: >On Thu, Apr 27, 2017 at 09:42:58PM -0700, Andres Freund wrote: >> >> >> On April 27, 2017 9:34:44 PM PDT, Noah Misch >wrote: >> >On Fri, Apr 21, 2017 at 10:36:21PM -0700, Andres Freund wrote: >> >>

Re: [HACKERS] Concurrent ALTER SEQUENCE RESTART Regression

2017-05-07 Thread Andres Freund
Hi, Moving discussion to -hackers, this isn't really a bug, it's an architectural issue with the new in-tree approach. Short recap: With the patch applied in [1] ff, sequences partially behave transactional (because pg_sequence is updated transactionally), partially non-transctionally (because th

Re: [HACKERS] Detecting schema changes during logical replication

2017-05-07 Thread Andres Freund
s a lookup for its own metadata using get_rel_sync_entry() which then checks relentry->schema_sent. Invalidation unsets schema_sent in rel_sync_cache_relation_cb. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

<    1   2   3   4   5   6   7   8   9   10   >