Re: Autovacuum-induced regression test instability

2019-04-15 Thread Michael Paquier
On Mon, Apr 15, 2019 at 01:22:30PM -0400, Tom Lane wrote: > In connection with the issue discussed at [1], I tried to run > the core regression tests with extremely aggressive autovacuuming > (I set autovacuum_naptime = 1s, autovacuum_vacuum_threshold = 5, > autovacuum_vacuum_cost_delay = 0). I

Re: extensions are hitting the ceiling

2019-04-15 Thread Noah Misch
On Mon, Mar 18, 2019 at 09:38:19PM -0500, Eric Hanson wrote: > #1: Dependencies > > Let's say we have two extensions, A and B, both of which depend on a third > extension C, let's just say C is hstore. A and B are written by different > developers, and both contain in their .control file the

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Ashwin Agrawal
On Mon, Apr 15, 2019 at 12:50 PM Peter Geoghegan wrote: > On Mon, Apr 15, 2019 at 9:16 AM Ashwin Agrawal > wrote: > > Would like to know more specifics on this Peter. We may be having > different context on hybrid row/column design. > > I'm confused about how close your idea of a TID is to the

Re: Calling pgstat_report_wait_end() before ereport(ERROR)

2019-04-15 Thread Michael Paquier
On Fri, Apr 12, 2019 at 10:06:41PM +0900, Masahiko Sawada wrote: > But I think that's not right, I've checked the code. If the startup > process failed in that function it raises a FATAL and recovery fails, > and if checkpointer process does then it calls > pgstat_report_wait_end() in

Re: [Patch] Mingw: Fix import library extension, build actual static libraries

2019-04-15 Thread Noah Misch
On Thu, Mar 07, 2019 at 03:23:50PM +0100, Sandro Mani wrote: > Related, no actual static libraries are produced alongside the respective > dlls. The attached patch 0002-Build-static-libraries.patch addresses this, > in a similar fashion as is already done for the AIX case in Makefile.shlib. We

Re: [PATCH v20] GSSAPI encryption support

2019-04-15 Thread Michael Paquier
On Mon, Apr 15, 2019 at 08:24:52AM -0400, Stephen Frost wrote: > The tests are really fast enough with one KDC that I don't think it > makes sense to have two independent tests. Perhaps you should add a comment about the need of unicity at the top of 001_auth.pl with a short description of the

Re: New vacuum option to do only freezing

2019-04-15 Thread Michael Paquier
On Mon, Apr 15, 2019 at 09:07:16PM -0400, Tom Lane wrote: > If we're failing to remove it, and it's below the desired freeze > horizon, then we'd darn well better freeze it instead, no? > > Since we know that the tuple only just became dead, I suspect > that the case would be unreachable in

Re: finding changed blocks using WAL scanning

2019-04-15 Thread Michael Paquier
On Mon, Apr 15, 2019 at 09:04:13PM -0400, Robert Haas wrote: > That is pretty much exactly what I was intending to propose. Any caller of XLogWrite() could switch to a new segment once the current one is done, and I am not sure that we would want some random backend to potentially slow down to do

Re: ExecForceStoreMinimalTuple leaks memory like there's no tomorrow

2019-04-15 Thread Michael Paquier
On Mon, Apr 15, 2019 at 10:46:56PM -0400, Tom Lane wrote: > create table t1 as select generate_series(1,4000) id; > vacuum analyze t1; > explain select * from t1, t1 t1b where t1.id = t1b.id; > -- should indicate a hash join > explain analyze select * from t1, t1 t1b where t1.id = t1b.id; > >

ExecForceStoreMinimalTuple leaks memory like there's no tomorrow

2019-04-15 Thread Tom Lane
Using HEAD, create table t1 as select generate_series(1,4000) id; vacuum analyze t1; explain select * from t1, t1 t1b where t1.id = t1b.id; -- should indicate a hash join explain analyze select * from t1, t1 t1b where t1.id = t1b.id; ... watch the process's memory consumption bloat. (It

Improve search for missing parent downlinks in amcheck

2019-04-15 Thread Alexander Korotkov
Hi! Currently we amcheck supports lossy checking for missing parent downlinks. It collects bitmap of downlink hashes and use it to check subsequent tree level. We've experienced some large corrupted indexes which pass this check due to its looseness. However, it seems to me we can implement

Re: finding changed blocks using WAL scanning

2019-04-15 Thread Bruce Momjian
On Mon, Apr 15, 2019 at 09:04:13PM -0400, Robert Haas wrote: > On Mon, Apr 15, 2019 at 4:31 PM Bruce Momjian wrote: > > Can I throw out a simple idea? What if, when we finish writing a WAL > > file, we create a new file 00010001.modblock which > > lists all the heap/index files

RE: Speedup of relation deletes during recovery

2019-04-15 Thread Jamison, Kirk
Hello Fujii-san, On April 18, 2018, Fujii Masao wrote: > On Fri, Mar 30, 2018 at 12:18 PM, Tsunakawa, Takayuki > wrote: >> Furthermore, TRUNCATE has a similar and worse issue. While DROP TABLE >> scans the shared buffers once for each table, TRUNCATE does that for >> each fork, resulting

Re: New vacuum option to do only freezing

2019-04-15 Thread Tom Lane
Robert Haas writes: > On Mon, Apr 15, 2019 at 3:47 PM Tom Lane wrote: >> No. I'm thinking there should be exactly one test of index_cleanup >> in this logic, and what it would be is along the lines of ... > I'm not sure that's correct. If you do that, it'll end up in the > non-tupgone case,

Re: finding changed blocks using WAL scanning

2019-04-15 Thread Robert Haas
On Mon, Apr 15, 2019 at 4:31 PM Bruce Momjian wrote: > Can I throw out a simple idea? What if, when we finish writing a WAL > file, we create a new file 00010001.modblock which > lists all the heap/index files and block numbers modified in that WAL > file? How much does that

Re: New vacuum option to do only freezing

2019-04-15 Thread Robert Haas
On Mon, Apr 15, 2019 at 3:47 PM Tom Lane wrote: > No. I'm thinking there should be exactly one test of index_cleanup > in this logic, and what it would be is along the lines of > > if (HeapTupleIsHotUpdated() || > HeapTupleIsHeapOnly() || > +

Re: COLLATE: Hash partition vs UPDATE

2019-04-15 Thread Amit Langote
On 2019/04/16 5:47, Tom Lane wrote: > Amit Langote writes: >> Attached updated patch. > > LGTM, pushed. Thank you. Regards, Amit

Re: Strange coding in mvdistinct.c

2019-04-15 Thread Tomas Vondra
On Mon, Apr 15, 2019 at 06:12:24PM -0400, Tom Lane wrote: Oh, and as I continue to grep, I found this in dependencies.c: dependencies = (MVDependencies *) repalloc(dependencies, offsetof(MVDependencies, deps)

Re: Strange coding in mvdistinct.c

2019-04-15 Thread Tom Lane
Oh, and as I continue to grep, I found this in dependencies.c: dependencies = (MVDependencies *) repalloc(dependencies, offsetof(MVDependencies, deps) + dependencies->ndeps *

Strange coding in mvdistinct.c

2019-04-15 Thread Tom Lane
In the wake of the discussion at [1] I went looking for structs that should be using FLEXIBLE_ARRAY_MEMBER and are not, by dint of grepping for size calculations of the form "offsetof(struct,fld) + n * sizeof(...)" and then seeing how "fld" is declared. I haven't yet found anything like that that

Re: [PATCH v20] GSSAPI encryption support

2019-04-15 Thread Robbie Harwood
Stephen Frost writes: > Please find attached a patch which updates the protocol.sgml docs that > Michael mentioned before, and merges the tests into one test file > (while adding in some additional tests to make sure that the server > also agrees with what our expectations are, using the

Re: Adding Unix domain socket path and port to pg_stat_get_wal_senders()

2019-04-15 Thread Tom Lane
Peter Eisentraut writes: > On 2019-04-12 17:57, Tom Lane wrote: >> Those are actually pretty common, for example if you use Red Hat's >> packaging you will have both /var/run/postgresql and /tmp as socket >> directories (since they consider use of /tmp deprecated, but getting >> rid of all

Re: Adding Unix domain socket path and port to pg_stat_get_wal_senders()

2019-04-15 Thread Peter Eisentraut
On 2019-04-12 17:57, Tom Lane wrote: >> For a socket connection, directory is important and that >> information I can get from unix_socket_directories parameter (I've >> never seen a setup with multiple socket directories). > Those are actually pretty common, for example if you use Red Hat's >

Re: COLLATE: Hash partition vs UPDATE

2019-04-15 Thread Tom Lane
Amit Langote writes: > Attached updated patch. LGTM, pushed. regards, tom lane

Re: finding changed blocks using WAL scanning

2019-04-15 Thread Bruce Momjian
On Wed, Apr 10, 2019 at 08:11:11PM -0400, Robert Haas wrote: > On Wed, Apr 10, 2019 at 5:49 PM Robert Haas wrote: > > There is one thing that does worry me about the file-per-LSN-range > > approach, and that is memory consumption when trying to consume the > > information. Suppose you have a

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Tomas Vondra
On Mon, Apr 15, 2019 at 11:57:49AM -0700, Ashwin Agrawal wrote: On Mon, Apr 15, 2019 at 11:18 AM Tomas Vondra wrote: Maybe. I'm not going to pretend I fully understand the internals. Does that mean the container contains ZSUncompressedBtreeItem as elements? Or just the plain

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Peter Geoghegan
On Mon, Apr 15, 2019 at 1:02 PM Andres Freund wrote: > There's not much of an alternative currently. Indexes require tid > looking things, and as a consequence (and some other comparatively small > changes that'd be required) tableam does too. I'm trying to establish whether or not that's the

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Andres Freund
Hi, On 2019-04-15 12:50:14 -0700, Peter Geoghegan wrote: > On Mon, Apr 15, 2019 at 9:16 AM Ashwin Agrawal wrote: > > Would like to know more specifics on this Peter. We may be having different > > context on hybrid row/column design. > > I'm confused about how close your idea of a TID is to

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Peter Geoghegan
On Mon, Apr 15, 2019 at 9:16 AM Ashwin Agrawal wrote: > Would like to know more specifics on this Peter. We may be having different > context on hybrid row/column design. I'm confused about how close your idea of a TID is to the traditional definition from heapam (and even zheap). If it's a

Re: New vacuum option to do only freezing

2019-04-15 Thread Tom Lane
Robert Haas writes: > On Mon, Apr 15, 2019 at 1:13 PM Tom Lane wrote: >> I have a very strong feeling that this patch was not fully baked. > I think you're right, but I don't understand the comment in the > preceding paragraph. How does this problem prevent tupgone from > getting set? My

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Andres Freund
Hi, On 2019-04-15 15:19:41 -0400, Tom Lane wrote: > Peter Geoghegan writes: > > I am also concerned by the broad scope of ZedStore, and I tend to > > agree that it will be difficult to maintain in core. At the same time, > > I think that Andres and Robert are probably right about the difficulty

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Tom Lane
Andres Freund writes: > We also have at times pretty explicitly resisted making crucial pieces > of infrastructure usable outside of core. E.g. because it's legitimately > hard (grammar extensibility), or because we'd some concerns around > stability and the exact approach (WAL - the generic

Re: New vacuum option to do only freezing

2019-04-15 Thread Robert Haas
On Mon, Apr 15, 2019 at 1:13 PM Tom Lane wrote: > Masahiko Sawada writes: > >> Ugh, I think the assertion is right but the above condition is > >> completely wrong. We should increment nleft_dead_tuples when index > >> cleanup is *not* enabled. > > > Here is a draft patch to fix this issue. > >

Re: Checksum errors in pg_stat_database

2019-04-15 Thread Julien Rouhaud
Sorry for late reply, On Sun, Apr 14, 2019 at 7:12 PM Magnus Hagander wrote: > > On Sat, Apr 13, 2019 at 8:46 PM Robert Treat wrote: >> >> On Fri, Apr 12, 2019 at 8:18 AM Magnus Hagander wrote: >> ISTM the argument here is go with zero since you have zero connections >> vs go with null since

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Peter Geoghegan
On Mon, Apr 15, 2019 at 12:19 PM Tom Lane wrote: > Perhaps, but we won't know if we don't try. I think we should try, > and be willing to add hooks and flexibility to core as needed to make > it possible. We could approach it without taking a firm position on inclusion in core until the project

Re: minimizing pg_stat_statements performance overhead

2019-04-15 Thread Raymond Martin
Hi, Apologies, but I had already created a commit here: https://commitfest.postgresql.org/23/2080/ . Any preference on which to keep? Thanks, Raymond Martin rama...@microsoft.com

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Andres Freund
Hi, On 2019-04-15 14:35:43 -0400, Tom Lane wrote: > Yeah, and that's something I've regretted more than once; I think SP-GiST > is a sterling example of something that isn't nearly useful enough in the > real world to justify the amount of maintenance effort we've been forced > to expend on it.

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Tom Lane
Peter Geoghegan writes: > I am also concerned by the broad scope of ZedStore, and I tend to > agree that it will be difficult to maintain in core. At the same time, > I think that Andres and Robert are probably right about the difficulty > of maintaining it outside of core -- that would be

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Robert Haas
On Mon, Apr 15, 2019 at 2:35 PM Tom Lane wrote: > Robert Haas writes: > > On Mon, Apr 15, 2019 at 11:10 AM Tom Lane wrote: > >> There is a finite limit to how much stuff we can maintain as part of core. > > > I don't agree with that at all. > > Really? Let's have a discussion of how

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Peter Geoghegan
On Mon, Apr 15, 2019 at 11:35 AM Tom Lane wrote: > We do need to limit what we accept into core PG. I do not buy your > argument that users expect everything to be in core. Or more accurately, > the people who do think that way won't be using PG anyway --- they'll > be using MSSQL because it

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Ashwin Agrawal
On Mon, Apr 15, 2019 at 11:18 AM Tomas Vondra wrote: > > Maybe. I'm not going to pretend I fully understand the internals. Does > that mean the container contains ZSUncompressedBtreeItem as elements? Or > just the plain Datum values? > First, your reading of code and all the comments/questions

Re: block-level incremental backup

2019-04-15 Thread Robert Haas
On Mon, Apr 15, 2019 at 9:01 AM Stephen Frost wrote: > I love the general idea of having additional facilities in core to > support block-level incremental backups. I've long been unhappy that > any such approach ends up being limited to a subset of the files which > need to be included in the

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Andres Freund
Hi, On 2019-04-15 14:11:02 -0400, Robert Haas wrote: > I hate to pick on any particular part of the tree, but it seems > entirely plausible to me that a second columnar storage implementation > could deliver more incremental value than spgist, an index AM you > committed. We should not move the

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Tom Lane
Robert Haas writes: > On Mon, Apr 15, 2019 at 11:10 AM Tom Lane wrote: >> There is a finite limit to how much stuff we can maintain as part of core. > I don't agree with that at all. Really? Let's have a discussion of how thermodynamics applies to software management sometime. >>> If we have

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Andres Freund
Hi, On 2019-04-15 14:11:02 -0400, Robert Haas wrote: > Furthermore, different table AMs are going to have different > needs. It has already been remarked by both Andres and on this thread > that for columnar storage to really zip along, the executor is going > to need to be much smarter about

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Andres Freund
Hi, On 2019-04-15 11:10:38 -0400, Tom Lane wrote: > Stephen Frost writes: > > * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote: > >> I think having a colstore in core is important not just for adoption, > >> but also for testing and development of the executor / planner bits. > > > Agreed. >

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Tomas Vondra
On Mon, Apr 15, 2019 at 10:50:21AM -0700, Ashwin Agrawal wrote: On Mon, Apr 15, 2019 at 10:33 AM Tomas Vondra wrote: ... I see. Perhaps it'd be better to call the flag ZSBT_CONTAINER, when it means "this is a container". And then have another flag to track whether the

Re: block-level incremental backup

2019-04-15 Thread Robert Haas
On Thu, Apr 11, 2019 at 1:29 PM Anastasia Lubennikova wrote: > 2) There should be a command to get only a map of changes without actual > data. Good idea. > 4) The algorithm of collecting changed blocks is another topic. > Though, it's API should be discussed here: > > Do we want to have

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Robert Haas
On Mon, Apr 15, 2019 at 11:10 AM Tom Lane wrote: > TBH, I thought the reason we were expending so much effort on a tableam > API was exactly so we *wouldn't* have to include such stuff in core. > > There is a finite limit to how much stuff we can maintain as part of core. > We should embrace the

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Robert Haas
On Sun, Apr 14, 2019 at 12:22 PM Tomas Vondra wrote: > It's not clear to me whether you're arguing for not having any such > implementation in core, or having multiple ones? I think we should aim > to have at least one in-core implementation, even if it's not the best > possible one for all

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Ashwin Agrawal
On Mon, Apr 15, 2019 at 10:33 AM Tomas Vondra wrote: > On Mon, Apr 15, 2019 at 09:29:37AM -0700, Ashwin Agrawal wrote: > > On Sun, Apr 14, 2019 at 9:40 AM Tomas Vondra > >wrote: > > > > On Thu, Apr 11, 2019 at 06:20:47PM +0300, Heikki Linnakangas wrote: > > >On 11/04/2019 17:54,

Re: Minor fix in reloptions.c comments

2019-04-15 Thread Robert Haas
On Fri, Apr 12, 2019 at 12:01 AM Michael Paquier wrote: > On Fri, Apr 12, 2019 at 02:41:37AM +, Jamison, Kirk wrote: > > I found some minor grammar mistake while reading reloptions.c code comments. > > Attached is the fix. > > I just changed "affect" to "effect", for both n_distinct and > >

Autovacuum-induced regression test instability

2019-04-15 Thread Tom Lane
In connection with the issue discussed at [1], I tried to run the core regression tests with extremely aggressive autovacuuming (I set autovacuum_naptime = 1s, autovacuum_vacuum_threshold = 5, autovacuum_vacuum_cost_delay = 0). I found that the timestamp test tends to fail with diffs caused by

Re: New vacuum option to do only freezing

2019-04-15 Thread Tom Lane
Masahiko Sawada writes: >> Ugh, I think the assertion is right but the above condition is >> completely wrong. We should increment nleft_dead_tuples when index >> cleanup is *not* enabled. > Here is a draft patch to fix this issue. So the real issue here, I fear, is that we've got no consistent

Re: plpgsql - execute - cannot use a reference to record field

2019-04-15 Thread Pavel Stehule
po 15. 4. 2019 v 18:07 odesílatel Tom Lane napsal: > Pavel Stehule writes: > > Is there reason why following code should not to work? > > > do $$ > > declare r record; result int; > > begin > > select 10 as a, 20 as b into r; > > raise notice 'a: %', r.a; > > execute 'select $1.a + $1.b'

Re: Multivariate MCV lists -- pg_mcv_list_items() seems to be broken

2019-04-15 Thread Tomas Vondra
On Mon, Apr 15, 2019 at 12:26:02PM -0400, Tom Lane wrote: Dean Rasheed writes: SELECT pg_mcv_list_items(stxmcv) from pg_statistic_ext WHERE stxname = 'foo_s'; which fails with ERROR: cache lookup failed for type 0 That definitely used to work, so I'm guessing it got broken by the recent

Re: Cleanup/remove/update references to OID column

2019-04-15 Thread Daniel Verite
Andres Freund wrote: > Yes, I was planning to commit that soon-ish. There still seemed > review / newer versions happening, though, so I was thinking of waiting > for a bit longer. You might want to apply this trivial one in the same batch: index 452f307..7cfb67f 100644 ---

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Ashwin Agrawal
On Sun, Apr 14, 2019 at 9:40 AM Tomas Vondra wrote: > On Thu, Apr 11, 2019 at 06:20:47PM +0300, Heikki Linnakangas wrote: > >On 11/04/2019 17:54, Tom Lane wrote: > >>Ashwin Agrawal writes: > >>>Thank you for trying it out. Yes, noticed for certain patterns > pg_lzcompress() actually requires

Re: Multivariate MCV lists -- pg_mcv_list_items() seems to be broken

2019-04-15 Thread Tom Lane
Dean Rasheed writes: > SELECT pg_mcv_list_items(stxmcv) from pg_statistic_ext WHERE stxname = > 'foo_s'; > which fails with > ERROR: cache lookup failed for type 0 > That definitely used to work, so I'm guessing it got broken by the > recent reworking of the serialisation code, but I've not

Re: Multivariate MCV lists -- pg_mcv_list_items() seems to be broken

2019-04-15 Thread Tomas Vondra
On Mon, Apr 15, 2019 at 05:02:43PM +0100, Dean Rasheed wrote: I just noticed the following: CREATE TABLE foo (a int, b int); INSERT INTO foo SELECT x/10, x/100 FROM generate_series(1, 100) x; CREATE STATISTICS foo_s ON a,b FROM foo; ANALYSE foo; SELECT pg_mcv_list_items(stxmcv) from

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Ashwin Agrawal
On Sat, Apr 13, 2019 at 4:22 PM Peter Geoghegan wrote: > On Thu, Apr 11, 2019 at 6:06 AM Rafia Sabih > wrote: > > Reading about it reminds me of this work -- TAG column storage( >

Re: plpgsql - execute - cannot use a reference to record field

2019-04-15 Thread Tom Lane
Pavel Stehule writes: > Is there reason why following code should not to work? > do $$ > declare r record; result int; > begin > select 10 as a, 20 as b into r; > raise notice 'a: %', r.a; > execute 'select $1.a + $1.b' into result using r; > raise notice '%', result; > end; > $$ You

Multivariate MCV lists -- pg_mcv_list_items() seems to be broken

2019-04-15 Thread Dean Rasheed
I just noticed the following: CREATE TABLE foo (a int, b int); INSERT INTO foo SELECT x/10, x/100 FROM generate_series(1, 100) x; CREATE STATISTICS foo_s ON a,b FROM foo; ANALYSE foo; SELECT pg_mcv_list_items(stxmcv) from pg_statistic_ext WHERE stxname = 'foo_s'; which fails with ERROR: cache

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Tom Lane
Stephen Frost writes: > * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote: >> I think having a colstore in core is important not just for adoption, >> but also for testing and development of the executor / planner bits. > Agreed. TBH, I thought the reason we were expending so much effort on a

Re: Accidental setting of XLogReaderState.private_data ?

2019-04-15 Thread Tom Lane
Antonin Houska writes: > StartupDecodingContext() initializes ctx->reader->private_data with ctx, and > it even does so twice. I couldn't find a place in the code where the > (LogicalDecodingContext *) pointer is retrieved from the reader, and a simple > test of logical replication works if the

Re: partitioning performance tests after recent patches

2019-04-15 Thread Tom Lane
Amit Langote writes: > On 2019/04/15 4:19, Floris Van Nee wrote: >> 2) Is running non-inlined SQL functions with a generic plan even the best >> option all the time? Wouldn't it be better to adopt a similar approach to >> what plpgsql does, where it tries to test if using a generic plan is >>

Re: jsonpath

2019-04-15 Thread Andrew Dunstan
On 4/14/19 10:43 PM, Alexander Korotkov wrote: > On Tue, Apr 9, 2019 at 7:16 PM Tomas Vondra > wrote: >> On Sun, Apr 07, 2019 at 03:03:58AM +0300, Alexander Korotkov wrote: >>> On Sun, Apr 7, 2019 at 2:37 AM Tom Lane wrote: Alexander Korotkov writes: > Thus, contents of unused

Re: partitioning performance tests after recent patches

2019-04-15 Thread David Rowley
On Mon, 15 Apr 2019 at 19:33, Floris Van Nee wrote: > Here's the output of explain/analyze for HEAD. At run-time, technically all > partitions could be pruned directly. However, one partition remains in the > output of explain/analyze because of other difficulties with removing all of > them,

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Tomas Vondra
On Sun, Apr 14, 2019 at 06:39:47PM +0200, Tomas Vondra wrote: On Thu, Apr 11, 2019 at 06:20:47PM +0300, Heikki Linnakangas wrote: On 11/04/2019 17:54, Tom Lane wrote: Ashwin Agrawal writes: Thank you for trying it out. Yes, noticed for certain patterns pg_lzcompress() actually requires much

Re: block-level incremental backup

2019-04-15 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > Several companies, including EnterpriseDB, NTT, and Postgres Pro, have > developed technology that permits a block-level incremental backup to > be taken from a PostgreSQL server. I believe the idea in all of those > cases is that

Re: PANIC: could not flush dirty data: Operation not permitted power8, Redhat Centos

2019-04-15 Thread Justin Pryzby
On Fri, Apr 12, 2019 at 08:04:00PM +0200, reiner peterke wrote: > We build Postgres on Power and x86 With the latest Postgres 11 release (11.2) > we get error on > power8 ppc64le (Redhat and CentOS). No error on SUSE on power8 > > No error on x86_64 (RH, Centos and SUSE) So there's an error

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Stephen Frost
Greetings, * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote: > On Tue, Apr 09, 2019 at 02:29:09PM -0400, Robert Haas wrote: > >On Tue, Apr 9, 2019 at 11:51 AM Alvaro Herrera > >wrote: > >>This is not surprising, considering that columnar store is precisely the > >>reason for starting the

Re: [PATCH v20] GSSAPI encryption support

2019-04-15 Thread Stephen Frost
Greetings, * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: > On 2019-04-09 09:32, Peter Eisentraut wrote: > > On 2019-04-09 04:51, Stephen Frost wrote: > >>> Running just 002_enc.pl by itself passes the tests! > >> Great! I think what I'll do is work to incorporate the two tests

Re: log bind parameter values on error

2019-04-15 Thread Alexey Bashtanov
On 05/04/2019 12:23, Peter Eisentraut wrote: On 2019-03-29 15:55, Alexey Bashtanov wrote: ERROR: value "62812" is out of range for type smallint STATEMENT: SELECT abalance FROM pgbench_accounts WHERE aid = $1; (In this case the error message contains the parameter value, so it's not a very

Accidental setting of XLogReaderState.private_data ?

2019-04-15 Thread Antonin Houska
StartupDecodingContext() initializes ctx->reader->private_data with ctx, and it even does so twice. I couldn't find a place in the code where the (LogicalDecodingContext *) pointer is retrieved from the reader, and a simple test of logical replication works if the patch below is applied. Thus I

Re: PANIC: could not flush dirty data: Operation not permitted power8, Redhat Centos

2019-04-15 Thread zedaardv
sent by smoke signals at great danger to my self. > On 12 Apr 2019, at 23:16, Thomas Munro wrote: > >> On Sat, Apr 13, 2019 at 7:23 AM Andres Freund wrote: >>> On 2019-04-12 20:04:00 +0200, reiner peterke wrote: >>> We build Postgres on Power and x86 With the latest Postgres 11 release

Re: partitioning performance tests after recent patches

2019-04-15 Thread Amit Langote
Hi, Thanks a lot for very exhaustive testing. David already replied to some points, but let me comment on a couple of points. Please be advised that these may not be the numbers (or scalability pattern of these numbers) you'll see when PG 12 is actually released, because we may end up changing

Re: Issue in ExecCleanupTupleRouting()

2019-04-15 Thread Etsuro Fujita
(2019/04/15 13:32), David Rowley wrote: On Fri, 12 Apr 2019 at 01:06, Etsuro Fujita wrote: While working on an update-tuple-routing bug in postgres_fdw [1], I noticed this change to ExecCleanupTupleRouting() made by commit 3f2393edefa5ef2b6970a5a2fa2c7e9c55cc10cf: Added to open items list.

Re: Issue in ExecCleanupTupleRouting()

2019-04-15 Thread Etsuro Fujita
(2019/04/12 10:48), Amit Langote wrote: On 2019/04/11 22:28, David Rowley wrote: On Fri, 12 Apr 2019 at 01:06, Etsuro Fujita wrote: + /* +* Check if this result rel is one belonging to the node's subplans, +* if so, let ExecEndPlan() clean it up. +*/ + if

Re: Attempt to consolidate reading of XLOG page

2019-04-15 Thread Antonin Houska
Alvaro Herrera wrote: > I agree that xlog reading is pretty messy. > > I think ifdef'ing the way XLogRead reports errors is not great. Maybe > we can pass a function pointer that is to be called in case of errors? I'll try a bit harder to evaluate the existing approaches to report the same

Re: Mailing list not working

2019-04-15 Thread Magnus Hagander
On Mon, Apr 15, 2019 at 11:02 AM Ramanarayana wrote: > Hi, > > I also got same error couple of hours ago. Now it is working fine. > There were network issues in one of the Cisco firewalls for one of our datacenters. It's the only "Enterprise technology" we use for pginfra I think, and it does

Re: Mailing list not working

2019-04-15 Thread Ramanarayana
Hi, I also got same error couple of hours ago. Now it is working fine. It would be great to have alerting tools like prometheus to notify users when the server is down Regards, Ram.

Re: Attempt to consolidate reading of XLOG page

2019-04-15 Thread Antonin Houska
Michael Paquier wrote: > On Fri, Apr 12, 2019 at 12:27:11PM +0900, Kyotaro HORIGUCHI wrote: > > This patch changes XLogRead to allow using other than > > BasicOpenFile to open a segment, and use XLogReaderState.private > > to hold a new struct XLogReadPos for the segment reader. The new > >

Re: Attempt to consolidate reading of XLOG page

2019-04-15 Thread Antonin Houska
Kyotaro HORIGUCHI wrote: > Hello. > > At Thu, 11 Apr 2019 18:05:42 +0200, Antonin Houska wrote > in <14984.1554998...@spoje.net> > > While working on the instance encryption I found it annoying to apply > > decyption of XLOG page to three different functions. Attached is a patch > > that > >

Re: hyrax vs. RelationBuildPartitionDesc

2019-04-15 Thread Amit Langote
On 2019/04/15 2:38, Tom Lane wrote: > Amit Langote writes: >> On 2019/04/10 15:42, Michael Paquier wrote: >>> It seems to me that Tom's argument to push in the way relcache >>> information is handled by copying its contents sounds sensible to me. >>> That's not perfect, but it is consistent with

Re: cache lookup failed for collation 0

2019-04-15 Thread Peter Eisentraut
On 2019-04-15 07:44, Jeevan Chalke wrote: > The root cause is that the same code match_pattern_prefix() is being > used for text and bytea, but bytea does not use collations, so having > the collation 0 is expected, and we shouldn't call > get_collation_isdeterministic() in that

Re: partitioning performance tests after recent patches

2019-04-15 Thread Floris Van Nee
Hi David, Thanks for your reply. I really appreciate your work on run-time pruning! Here's the output of explain/analyze for HEAD. At run-time, technically all partitions could be pruned directly. However, one partition remains in the output of explain/analyze because of other difficulties with

Re: Mailing list not working

2019-04-15 Thread Amit Langote
On 2019/04/15 15:37, Tatsuo Ishii wrote: >> Hi, >> >> I am not able to access the mailing list archive. Is the mailing list >> server down or something? > > What kind of problem do you have? > > I can see your posting in the archive. >

Re: Mailing list not working

2019-04-15 Thread Tatsuo Ishii
> Hi, > > I am not able to access the mailing list archive. Is the mailing list > server down or something? What kind of problem do you have? I can see your posting in the archive. https://www.postgresql.org/message-id/CAKm4Xs5%2BD%2BgB4yCQtsfKmdTMqvido1k5Qz7iwPAQj8CM-ptiXw%40mail.gmail.com

Re: hyrax vs. RelationBuildPartitionDesc

2019-04-15 Thread Amit Langote
On 2019/04/14 0:53, Tom Lane wrote: > Amit Langote writes: >> On Sat, Apr 13, 2019 at 4:47 AM Tom Lane wrote: >>> I concluded that that's not parenthetical but pretty relevant... > >> Hmm, do you mean we should grow the same "keep_" logic for >> rd_partcheck as we have for rd_partkey and

Re: COLLATE: Hash partition vs UPDATE

2019-04-15 Thread Amit Langote
Thanks for the review. On 2019/04/15 5:50, Tom Lane wrote: > Jesper Pedersen writes: >> Yeah, that works here - apart from an issue with the test case; fixed in >> the attached. > > Couple issues spotted in an eyeball review of that: > > * There is code that supposes that partsupfunc[] is the

Re: New vacuum option to do only freezing

2019-04-15 Thread Masahiko Sawada
On Mon, Apr 15, 2019 at 9:28 AM Masahiko Sawada wrote: > > On Mon, Apr 15, 2019 at 12:47 AM Tom Lane wrote: > > > > Robert Haas writes: > > > On Wed, Apr 3, 2019 at 10:32 PM Masahiko Sawada > > > wrote: > > >> Attached the updated version patch. > > > > > Committed with a little bit of