get_controlfile() can leak fds in the backend

2019-02-26 Thread Michael Paquier
Hi all, (CC-ing Joe as of dc7d70e) I was just looking at the offline checksum patch, and noticed some sloppy coding in controldata_utils.c. The control file gets opened in get_controlfile(), and if it generates an error then the file descriptor remains open. As basic code rule in the backend we

Re: Remove Deprecated Exclusive Backup Mode

2019-02-26 Thread Laurenz Albe
Fujii Masao wrote: > So, let me clarify the situations; > > (3) If backup_label.pending exists but recovery.signal doesn't, the server >ignores (or removes) backup_label.pending and do the recovery >starting the pg_control's REDO location. This case can happen if >the

Re: [HACKERS] generated columns

2019-02-26 Thread Michael Paquier
On Wed, Feb 27, 2019 at 08:05:02AM +0100, Peter Eisentraut wrote: > There is something like that at the top of > src/test/regress/sql/generated.sql. I can expand that. I think you mean identity.sql. I would think that this would live better with some other sanity checks. I am fine with your

Re: Offline enabling/disabling of data checksums

2019-02-26 Thread Michael Paquier
On Wed, Feb 27, 2019 at 07:59:31AM +0100, Fabien COELHO wrote: > The renaming implies quite a few changes (eg in the documentation, > makefiles, tests) which warrants a review, so it should be a patch. Also, > ISTM that the renaming only make sense when adding the enable/disable > feature, so I'd

Re: [HACKERS] generated columns

2019-02-26 Thread Peter Eisentraut
On 2019-02-26 06:12, Michael Paquier wrote: > Hm. Does the SQL standard mention more features which could be merged > with stored values, virtual values, default expressions and identity > columns? I don't know the last trends in this area but I am wondering > if there are any other column

Re: [HACKERS] generated columns

2019-02-26 Thread Peter Eisentraut
On 2019-02-26 06:30, Michael Paquier wrote: > + if (attgenerated) > + { > + /* > +* Generated column: Dropping anything that the generation expression > +* refers to automatically drops the generated column. > +*/ > + recordDependencyOnSingleRelExpr(, expr,

Re: Offline enabling/disabling of data checksums

2019-02-26 Thread Fabien COELHO
Hallo Michael, - * src/bin/pg_verify_checksums/pg_verify_checksums.c + * src/bin/pg_checksums/pg_checksums.c That's lacking a rename, or this comment is incorrect. Right, I started the rename, but then backed off pending further discussion whether I should submit that or whether the

RE: Problem with default partition pruning

2019-02-26 Thread Yuzuko Hosoya
Amit-san, > From: Amit Langote [mailto:langote_amit...@lab.ntt.co.jp] > Sent: Wednesday, February 27, 2019 11:22 AM > > Hosoya-san, > > On 2019/02/22 17:14, Yuzuko Hosoya wrote: > > Hi, > > > > I found the bug of default partition pruning when executing a range query. > > > > - > >

Re: TupleTableSlot abstraction

2019-02-26 Thread Andres Freund
On 2019-02-27 15:42:50 +0900, Michael Paquier wrote: > On Tue, Feb 26, 2019 at 10:38:45PM -0800, Andres Freund wrote: > > Im not sure I understand. How can adding a memory context + reset to > > ctas and matview receivers negatively impact other dest receivers? > > I don't think you got my point

Re: Libpq support to connect to standby server as priority

2019-02-26 Thread Haribabu Kommi
On Mon, Feb 25, 2019 at 11:38 AM Tsunakawa, Takayuki < tsunakawa.ta...@jp.fujitsu.com> wrote: > Hi Hari-san, > > I've reviewed all files. I think I'll proceed to testing when I've > reviewed the revised patch and the patch for target_server_type. > > Thanks for the review. > > (1) patch 0001 >

Re: TupleTableSlot abstraction

2019-02-26 Thread Michael Paquier
On Tue, Feb 26, 2019 at 10:38:45PM -0800, Andres Freund wrote: > Im not sure I understand. How can adding a memory context + reset to > ctas and matview receivers negatively impact other dest receivers? I don't think you got my point here: imagine that a plugin use the current receiveSlot logic

Re: TupleTableSlot abstraction

2019-02-26 Thread Andres Freund
Hi, On 2019-02-27 15:34:07 +0900, Michael Paquier wrote: > On Tue, Feb 26, 2019 at 09:42:38PM -0800, Andres Freund wrote: > > I'm not so sure that's the architecturally correct fix however. Is it > > actually guaranteed, given expanded tuples, toasting, etc, that there's > > no other memory leak

Re: Autovaccuum vs temp tables crash

2019-02-26 Thread Michael Paquier
On Tue, Feb 26, 2019 at 07:21:40PM -0500, Tom Lane wrote: > The existing state of affairs is that a superuser who really needs to drop > a temp schema can do so, if she's careful that it's not active. Pinning > things would break that, or at least add an additional roadblock. If it's > some sort

Re: TupleTableSlot abstraction

2019-02-26 Thread Michael Paquier
On Tue, Feb 26, 2019 at 09:42:38PM -0800, Andres Freund wrote: > I'm not so sure that's the architecturally correct fix however. Is it > actually guaranteed, given expanded tuples, toasting, etc, that there's > no other memory leak here? I wonder if we shouldn't work twoards using a > short lived

Re: psql display of foreign keys

2019-02-26 Thread Michael Paquier
On Tue, Feb 26, 2019 at 07:27:57PM -0300, Alvaro Herrera wrote: > Thanks for committing pg_partition_root ... but it turns out to be > useless for this purpose. Well, what's done is done. The thing is useful by itself in my opinion. > It turns out that we need to obtain the list > of

Re: When is the MessageContext released?

2019-02-26 Thread Andres Freund
On 2019-02-27 14:08:47 +0800, Andy Fan wrote: > Hi : > I run a query like "select * from t" and set the break like this: > > break exec_simple_query > break MemoryContextDelete > commands >p context->name >c > end > > I can see most of the MemoryContext is relased, but never

Re: NOT IN subquery optimization

2019-02-26 Thread Richard Guo
On Wed, Feb 27, 2019 at 4:52 AM David Rowley wrote: > On Wed, 27 Feb 2019 at 03:07, Jim Finnerty wrote: > > If you're proposing to do that for this thread then I can take my > planner only patch somewhere else. I only posted my patch as I pretty > much already had what I thought you were

When is the MessageContext released?

2019-02-26 Thread Andy Fan
Hi : I run a query like "select * from t" and set the break like this: break exec_simple_query break MemoryContextDelete commands p context->name c end I can see most of the MemoryContext is relased, but never MessageContext, when will it be released? /* * Create the memory context

Re: Index INCLUDE vs. Bitmap Index Scan

2019-02-26 Thread Markus Winand
> On 27.02.2019, at 02:00, Justin Pryzby wrote: > > On Tue, Feb 26, 2019 at 09:07:01PM +0100, Markus Winand wrote: >> CREATE INDEX idx ON tbl (a, b, c); >> Bitmap Heap Scan on tbl (cost=4.14..8.16 rows=1 width=7616) (actual >> time=0.021..0.021 rows=1 loops=1) >> Recheck Cond: ((a = 1)

Re: TupleTableSlot abstraction

2019-02-26 Thread Andres Freund
Hi, On 2019-02-27 14:21:52 +0900, Michael Paquier wrote: > On Sat, Feb 16, 2019 at 05:07:44PM -0500, Jeff Janes wrote: > > By blind analogy to the changes made to matview.c, I think that createas.c > > is missing a heap_freetuple, as attached. First, sorry to have been slow answering. I was

Re: pgsql: Avoid creation of the free space map for small heap relations, t

2019-02-26 Thread John Naylor
On Wed, Feb 27, 2019 at 5:06 AM Amit Kapila wrote: > I have tried this test many times (more than 1000 times) by varying > thread count, but couldn't reproduce it. My colleague, Kuntal has > tried a similar test overnight, but the issue didn't reproduce which > is not surprising to me seeing the

Re: Index INCLUDE vs. Bitmap Index Scan

2019-02-26 Thread Markus Winand
> On 27.02.2019, at 00:22, Tom Lane wrote: > > Markus Winand writes: >> I think Bitmap Index Scan should take advantage of B-tree INCLUDE columns, >> which it doesn’t at the moment (tested on master as of yesterday). > > Regular index scans don't do what you're imagining either (i.e.,

RE: Protect syscache from bloating with negative cache entries

2019-02-26 Thread Ideriha, Takeshi
>From: Ideriha, Takeshi [mailto:ideriha.take...@jp.fujitsu.com] >>>* 0001: add dlist_push_tail() ... as is >>>* 0002: memory accounting, with correction based on feedback >>>* 0003: merge the original 0003 and 0005, with correction based on >>>feedback >> >>Attached are simpler version based on

Re: TupleTableSlot abstraction

2019-02-26 Thread Michael Paquier
On Sat, Feb 16, 2019 at 05:07:44PM -0500, Jeff Janes wrote: > By blind analogy to the changes made to matview.c, I think that createas.c > is missing a heap_freetuple, as attached. I think that you are right. CTAS and materialized views share a lot when it comes to relation creation and initial

Re: pgsql: Avoid creation of the free space map for small heap relations, t

2019-02-26 Thread John Naylor
On Tue, Feb 26, 2019 at 10:28 AM Amit Kapila wrote: > John, others, can you review my findings and patch? I can confirm your findings with your debugging steps. Nice work! -- John Naylorhttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training &

Re: Early WIP/PoC for inlining CTEs

2019-02-26 Thread Andrew Gierth
> "Tom" == Tom Lane writes: >> I also thought about that. But what I thought about it on reflection >> was: if the user explicitly wrote NOT MATERIALIZED, then we should >> assume they mean it. Tom> Ah, but the example I gave also had MATERIALIZED on the inner WITH. Tom> Why should the

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

2019-02-26 Thread Pavan Deolasee
On Wed, Feb 27, 2019 at 7:05 AM Jeff Janes wrote: > > > After doing a truncation and '\copy ... with (freeze)' of a table with > long data, I find that the associated toast table has a handful of unfrozen > blocks. I don't know if that is an actual problem, but it does seem a bit > odd, and

Re: Early WIP/PoC for inlining CTEs

2019-02-26 Thread Tom Lane
Andrew Gierth writes: > "Tom" == Tom Lane writes: > Tom> Also, I thought of a somewhat-related scenario that the code isn't > Tom> accounting for: you can break the restrictions about single > Tom> evaluation with nested WITHs, like > I also thought about that. But what I thought about it on

RE: Timeout parameters

2019-02-26 Thread Nagaura, Ryohei
Hi, kirk-san. Thank you for review. > From: Jamison, Kirk > Your socket_timeout patch still does not apply either with git or patch > command. It > says it's still corrupted. > I'm not sure about the workaround, because the --ignore-space-change and > --ignore-whitespace did not work for me. >

Re: pgsql: Avoid creation of the free space map for small heap relations, t

2019-02-26 Thread Amit Kapila
On Tue, Feb 26, 2019 at 2:58 PM Amit Kapila wrote: > > On Mon, Feb 25, 2019 at 10:32 PM Tom Lane wrote: > > > > To fix this symptom, we can ensure that once we didn't get any block > from local map, we must clear it. See the attached patch. I will try > to evaluate this code path to see if

Re: Problem with default partition pruning

2019-02-26 Thread Amit Langote
Hosoya-san, On 2019/02/22 17:14, Yuzuko Hosoya wrote: > Hi, > > I found the bug of default partition pruning when executing a range query. > > - > postgres=# create table test1(id int, val text) partition by range (id); > postgres=# create table test1_1 partition of test1 for values from

Re: Proving IS NOT NULL inference for ScalarArrayOpExpr's

2019-02-26 Thread James Coleman
On Mon, Feb 18, 2019 at 4:53 PM Tom Lane wrote: > So ... this is actively wrong. > > This case shows that you can't ignore the empty-array possibility > for a ScalarArrayOpExpr with useOr = false, because > "SELECT null::int = all(array[]::int[])" yields TRUE: > > contrib_regression=# select *

Re: Pluggable Storage - Andres's take

2019-02-26 Thread Haribabu Kommi
On Wed, Feb 27, 2019 at 11:10 AM Andres Freund wrote: > Hi, > > On 2019-01-21 10:32:37 +1100, Haribabu Kommi wrote: > > I am not able to remove the complete t_tableOid from HeapTuple, > > because of its use in triggers, as the slot is not available in triggers > > and I need to store the

RE: Timeout parameters

2019-02-26 Thread Jamison, Kirk
Hi Nagaura-san, Your socket_timeout patch still does not apply either with git or patch command. It says it's still corrupted. I'm not sure about the workaround, because the --ignore-space-change and --ignore-whitespace did not work for me. Maybe it might have something to do with your editor

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

2019-02-26 Thread Jeff Janes
On Thu, Feb 21, 2019 at 1:05 AM Pavan Deolasee wrote: > Hi, > > Jeff Janes raised an issue [1] about PD_ALL_VISIBLE not being set > correctly while loading data via COPY FREEZE and had also posted a draft > patch. > > I now have what I think is a more complete patch. I took a slightly >

Re: Early WIP/PoC for inlining CTEs

2019-02-26 Thread Andrew Gierth
> "Tom" == Tom Lane writes: Tom> Also, I thought of a somewhat-related scenario that the code isn't Tom> accounting for: you can break the restrictions about single Tom> evaluation with nested WITHs, like I also thought about that. But what I thought about it on reflection was: if the

Re: New vacuum option to do only freezing

2019-02-26 Thread Bossart, Nathan
Sorry for the delay. I finally got a chance to look through the latest patches. On 2/3/19, 1:48 PM, "Masahiko Sawada" wrote: > On Fri, Feb 1, 2019 at 11:43 PM Bossart, Nathan wrote: >> >> + if (skip_index_vacuum) >> + ereport(elevel, >> +

Re: Index INCLUDE vs. Bitmap Index Scan

2019-02-26 Thread Justin Pryzby
On Tue, Feb 26, 2019 at 09:07:01PM +0100, Markus Winand wrote: > CREATE INDEX idx ON tbl (a, b, c); > Bitmap Heap Scan on tbl (cost=4.14..8.16 rows=1 width=7616) (actual > time=0.021..0.021 rows=1 loops=1) >Recheck Cond: ((a = 1) AND (c = 1)) >-> Bitmap Index Scan on idx

Re: NOT IN subquery optimization

2019-02-26 Thread David Rowley
On Wed, 27 Feb 2019 at 13:41, Li, Zheng wrote: > We still check for inner side's nullability, when it is nullable we > append a "var is NULL" to the anti join condition. So every outer > tuple is going to evaluate to true on the join condition when there > is indeed a null entry in the inner.

Re: NOT IN subquery optimization

2019-02-26 Thread Li, Zheng
I'm totally fine with setting the target to PG13. -- I'm interested to know how this works without testing for inner nullability. If any of the inner side's join exprs are NULL then no records can match. What do you propose to work around that? -- We still check for inner side's nullability,

Re: NOT IN subquery optimization

2019-02-26 Thread David Rowley
On Wed, 27 Feb 2019 at 13:13, Tom Lane wrote: > > "Li, Zheng" writes: > > However, given that the March CommitFest is imminent and the runtime smarts > > patent concerns David had pointed out (which I was not aware of before), we > > would not move that direction at the moment. > > > I propose

Re: NOT IN subquery optimization

2019-02-26 Thread David Rowley
On Wed, 27 Feb 2019 at 13:05, Li, Zheng wrote: > -With the latest fix (for the empty table case), our patch does the > transformation as long as the outer is non-nullable regardless of the inner > nullability, experiments show that the results are always faster. Hi Zheng, I'm interested to

Re: Autovaccuum vs temp tables crash

2019-02-26 Thread Tom Lane
Alvaro Herrera writes: > On 2019-Feb-23, Tom Lane wrote: >> However, if someone held a gun to my head and said fix it, I'd be inclined >> to do so by having temp-namespace creation insert a "pin" dependency into >> pg_depend. Arguably, the only reason we don't create all the temp >> namespaces

Re: NOT IN subquery optimization

2019-02-26 Thread Tom Lane
"Li, Zheng" writes: > However, given that the March CommitFest is imminent and the runtime smarts > patent concerns David had pointed out (which I was not aware of before), we > would not move that direction at the moment. > I propose that we collaborate to build one patch from the two patches

Re: Pluggable Storage - Andres's take

2019-02-26 Thread Andres Freund
Hi, On 2019-01-21 10:32:37 +1100, Haribabu Kommi wrote: > I am not able to remove the complete t_tableOid from HeapTuple, > because of its use in triggers, as the slot is not available in triggers > and I need to store the tableOid also as part of the tuple. What precisely do you man by "use in

Re: Allowing extensions to supply operator-/function-specific info

2019-02-26 Thread Tom Lane
Paul Ramsey writes: >> On Feb 26, 2019, at 2:19 PM, Tom Lane wrote: >> In most cases, multiple matching arguments are going to lead to >> failure to construct any useful index condition, because your >> comparison value has to be a pseudoconstant (ie, not a variable >> from the same table, so in

Re: NOT IN subquery optimization

2019-02-26 Thread Li, Zheng
I agree we will need some runtime smarts (such as a new anti join type as pointed out by Richard) to "ultimately" account for all the cases of NOT IN queries. However, given that the March CommitFest is imminent and the runtime smarts patent concerns David had pointed out (which I was not

Re: Allowing extensions to supply operator-/function-specific info

2019-02-26 Thread Paul Ramsey
> On Feb 26, 2019, at 2:19 PM, Tom Lane wrote: > > In most cases, multiple matching arguments are going to lead to > failure to construct any useful index condition, because your > comparison value has to be a pseudoconstant (ie, not a variable > from the same table, so in both of the above

Re: Segfault when restoring -Fd dump on current HEAD

2019-02-26 Thread Tom Lane
Alvaro Herrera writes: > I think it would be better to just put back the .defn = "" (etc) to the > ArchiveEntry calls. Yeah, that was what I was imagining --- just make the ArchiveEntry calls act exactly like they did before in terms of what gets filled into the TOC fields. This episode is a

Re: POC: converting Lists into arrays

2019-02-26 Thread Tom Lane
I wrote: > I had an idea that perhaps is worth considering --- upthread I rejected > the idea of deleting lnext() entirely, but what if we did so? We could > redefine foreach() to not use it: > #define foreach(cell, l) \ > for (int cell##__index = 0; \ > (cell = list_nth_cell(l,

Re: [Patch][WiP] Tweaked LRU for shared buffers

2019-02-26 Thread Benjamin Manes
Hi Tomas, If you are on a benchmarking binge and feel like generating some trace files (as mentioned earlier), I'd be happy to help in regards to running them through simulations to show how different policies behave. We can add more types to match this patch / Postgres' GClock as desired, too.

Re: Segfault when restoring -Fd dump on current HEAD

2019-02-26 Thread Alvaro Herrera
On 2019-Feb-26, Dmitry Dolgov wrote: > > On Tue, Feb 26, 2019 at 6:38 AM Michael Paquier wrote: > > > > Works for me. With a quick read of the code, it seems to me that it > > is possible to keep compatibility while keeping the simplifications > > around ArchiveEntry()'s refactoring. > > Yes,

Re: Index INCLUDE vs. Bitmap Index Scan

2019-02-26 Thread Andres Freund
Hi, On 2019-02-26 18:22:41 -0500, Tom Lane wrote: > Markus Winand writes: > > I think Bitmap Index Scan should take advantage of B-tree INCLUDE columns, > > which it doesn’t at the moment (tested on master as of yesterday). > > Regular index scans don't do what you're imagining either (i.e.,

Re: Autovaccuum vs temp tables crash

2019-02-26 Thread Alvaro Herrera
On 2019-Feb-23, Tom Lane wrote: > However, if someone held a gun to my head and said fix it, I'd be inclined > to do so by having temp-namespace creation insert a "pin" dependency into > pg_depend. Arguably, the only reason we don't create all the temp > namespaces during bootstrap is because we

Re: Index INCLUDE vs. Bitmap Index Scan

2019-02-26 Thread Tom Lane
Markus Winand writes: > I think Bitmap Index Scan should take advantage of B-tree INCLUDE columns, > which it doesn’t at the moment (tested on master as of yesterday). Regular index scans don't do what you're imagining either (i.e., check filter conditions in advance of visiting the heap).

Re: Segfault when restoring -Fd dump on current HEAD

2019-02-26 Thread Tom Lane
Alvaro Herrera writes: > Hmm, shouldn't we modify sanitize_line so that it returns strdup(hyphen) > when input is empty and want_hyphen, too? If this patch is touching the behavior of functions like that, then it's going in the wrong direction; the need for any such change suggests strongly that

Re: Segfault when restoring -Fd dump on current HEAD

2019-02-26 Thread Alvaro Herrera
On 2019-Feb-26, Dmitry Dolgov wrote: > Yes, it should be rather simple, we can e.g. return to the old less consistent > NULL handling approach something (like in the attached patch), or replace a > NULL > value with an empty string in WriteToc. Give me a moment, I'll check it out. > At > the

Re: Segfault when restoring -Fd dump on current HEAD

2019-02-26 Thread Alvaro Herrera
On 2019-Feb-26, Michael Paquier wrote: > On Tue, Feb 26, 2019 at 12:16:35AM -0500, Tom Lane wrote: > > Well, if we didn't want to fix this, a reasonable way to go about > > it would be to bump the archive version number in pg_dump output, > > so that old versions would issue a useful complaint

Re: Parallel query vs smart shutdown and Postmaster death

2019-02-26 Thread Thomas Munro
On Mon, Feb 25, 2019 at 2:13 PM Thomas Munro wrote: > 1. In a nearby thread, I misdiagnosed a problem reported[1] by Justin > Pryzby (though my misdiagnosis is probably still a thing to be fixed; > see next). I think I just spotted the real problem he saw: if you > execute a parallel query

Re: Allowing extensions to supply operator-/function-specific info

2019-02-26 Thread Tom Lane
Paul Ramsey writes: > On Feb 26, 2019, at 2:19 PM, Tom Lane wrote: >> What's the query look like exactly? The other two calls will occur >> anyway, but SupportRequestIndexCondition depends on the function >> call's placement. > select geos_intersects_new(g, 'POINT(0 0)') from foo; Right, so

Re: Allowing extensions to supply operator-/function-specific info

2019-02-26 Thread Paul Ramsey
> On Feb 26, 2019, at 2:19 PM, Tom Lane wrote: > >> I have >> created a table (foo) a geometry column (g) and an index (GIST on >> foo(g)) and am running a query against foo using a noop function with >> a support function bound to it. > >> The support function is called, twice, once in >>

Re: psql display of foreign keys

2019-02-26 Thread Alvaro Herrera
On 2019-Feb-04, Michael Paquier wrote: > pg_partition_root() has not made it to the finish line yet, still it > would have been nice to see a rebase, and the patch has been waiting > for input for 4 weeks now. So I am marking it as returned with > feedback. Thanks for committing

Re: Allowing extensions to supply operator-/function-specific info

2019-02-26 Thread Tom Lane
Paul Ramsey writes: > New line of questioning: under what conditions will the support > function be called in a T_SupportRequestIndexCondition mode? It'll be called if the target function appears at top level of a WHERE or JOIN condition and any one of the function's arguments syntactically

Re: ATTACH/DETACH PARTITION CONCURRENTLY

2019-02-26 Thread Robert Haas
On Thu, Jan 31, 2019 at 1:02 PM Robert Haas wrote: > New patch series attached. And here's yet another new patch series, rebased over today's commit and with a couple of other fixes: 1. I realized that the PartitionDirectory for the planner ought to be attached to the PlannerGlobal, not the

Re: Retrieving Alias Name

2019-02-26 Thread Julien Rouhaud
On Tue, Feb 26, 2019 at 10:48 PM Walter Cai wrote: > > I'm currently using a (very rough) scheme to retrieve relation names from a > PlannerInfo, and a RelOptInfo struct: > > PlannerInfo *root > RelOptInfo *inner_rel > > //... > > RangeTblEntry *rte; > int x = -1; > while ((x =

Retrieving Alias Name

2019-02-26 Thread Walter Cai
Hi, I'm currently using a (very rough) scheme to retrieve relation names from a PlannerInfo, and a RelOptInfo struct: PlannerInfo *root RelOptInfo *inner_rel //... RangeTblEntry *rte; int x = -1; while ((x = bms_next_member(inner_rel->relids, x)) >= 0) { rte = root->simple_rte_array[x];

Re: Remove Deprecated Exclusive Backup Mode

2019-02-26 Thread Chapman Flack
On 2/26/19 2:46 PM, Andres Freund wrote: > Also, there's so much wrong stuff on the wiki that people that know to > look there just don't believe what they read. Should there be a wiki errata page on the wiki? I'm fairly serious ... for those times when you're reading the Foo page on the wiki,

Re: Allowing extensions to supply operator-/function-specific info

2019-02-26 Thread Paul Ramsey
On Mon, Feb 25, 2019 at 4:09 PM Tom Lane wrote: > > Paul Ramsey writes: > > On Mon, Feb 25, 2019 at 3:01 PM Tom Lane wrote: > >> It's whichever one the index column's opclass belongs to. Basically what > >> you're trying to do here is verify whether the index will support the > >> optimization

Re: NOT IN subquery optimization

2019-02-26 Thread David Rowley
On Wed, 27 Feb 2019 at 03:07, Jim Finnerty wrote: > > The problem is that the special optimization that was proposed for the case > where the subquery has no WHERE clause isn't valid when the subquery returns > no rows. That additional optimization needs to be removed, and preferably > replaced

Re: Remove Deprecated Exclusive Backup Mode

2019-02-26 Thread Peter Geoghegan
On Mon, Feb 25, 2019 at 10:49 PM David Steele wrote: > Worse, they have scripted the deletion of backup_label so that the > cluster will restart on crash. This is the recommendation from our > documentation after all. If that script runs after a restore instead of > a crash, then the cluster

Re: No-rewrite timestamp<->timestamptz conversions

2019-02-26 Thread Noah Misch
On Tue, Feb 26, 2019 at 02:29:01PM +, Simon Riggs wrote: > Looks good, would need docs. The ALTER TABLE page just says "old type is either binary coercible to the new type or an unconstrained domain over the new type." Avoiding rewrites by way of a prosupport function or the

Index INCLUDE vs. Bitmap Index Scan

2019-02-26 Thread Markus Winand
Hi! I think Bitmap Index Scan should take advantage of B-tree INCLUDE columns, which it doesn’t at the moment (tested on master as of yesterday). Consider this (find the setup at the bottom of this mail). CREATE INDEX idx ON tbl (a, b) INCLUDE (c); EXPLAIN (analyze, buffers) SELECT * FROM

Re: Remove Deprecated Exclusive Backup Mode

2019-02-26 Thread Andres Freund
Hi, On 2019-02-26 20:38:17 +0100, Magnus Hagander wrote: > That should not be a wiki page, really, that should be part of the main > documentation. +1 > It can be drafted on the wiki of course, but we *really* should get > something like that into the docs. Because that's what people are going

Re: Remove Deprecated Exclusive Backup Mode

2019-02-26 Thread Magnus Hagander
On Tue, Feb 26, 2019 at 6:31 PM Christophe Pettus wrote: > > > > On Feb 26, 2019, at 09:26, Robert Haas wrote: > > > > On Tue, Feb 26, 2019 at 12:20 PM Fujii Masao > wrote: > >> So, let me clarify the situations; > >> > >> (1) If backup_label and recovery.signal exist, the recovery starts >

Re: A note about recent ecpg buildfarm failures

2019-02-26 Thread Robert Haas
On Tue, Feb 26, 2019 at 1:25 PM Tom Lane wrote: > Since my commits 9e138a401 et al on Saturday, buildfarm members > blobfish, brotula, and wunderpus have been showing core dumps > in the ecpg preprocessor. This seemed inexplicable given what > the commits changed, and even more so seeing that

A note about recent ecpg buildfarm failures

2019-02-26 Thread Tom Lane
Since my commits 9e138a401 et al on Saturday, buildfarm members blobfish, brotula, and wunderpus have been showing core dumps in the ecpg preprocessor. This seemed inexplicable given what the commits changed, and even more so seeing that only HEAD is failing, while the change was back-patched

Re: [RFC] [PATCH] Flexible "partition pruning" hook

2019-02-26 Thread Mike Palmiotto
On Tue, Feb 26, 2019 at 1:55 AM Tsunakawa, Takayuki wrote: > > From: Mike Palmiotto [mailto:mike.palmio...@crunchydata.com] > > Attached is a patch which attempts to solve a few problems: > > > > 1) Filtering out partitions flexibly based on the results of an external > > function call (supplied

Re: pgbench MAX_ARGS

2019-02-26 Thread Andres Freund
On 2019-02-26 12:51:23 -0500, Tom Lane wrote: > Simon Riggs writes: > > On Tue, 26 Feb 2019 at 17:38, Andres Freund wrote: > >> Why not just allocate it dynamically? Seems weird to have all these > >> MAX_ARGS, MAX_SCRIPTS ... commands. > > > For me, its a few minutes work to correct a problem

Re: pgbench MAX_ARGS

2019-02-26 Thread Tom Lane
Simon Riggs writes: > On Tue, 26 Feb 2019 at 17:38, Andres Freund wrote: >> Why not just allocate it dynamically? Seems weird to have all these >> MAX_ARGS, MAX_SCRIPTS ... commands. > For me, its a few minutes work to correct a problem and report to the > community. > Dynamic allocation,

Re: pgbench MAX_ARGS

2019-02-26 Thread Simon Riggs
On Tue, 26 Feb 2019 at 17:38, Andres Freund wrote: > Hi, > > On 2019-02-26 12:57:14 +, Simon Riggs wrote: > > On Tue, 26 Feb 2019 at 12:19, Fabien COELHO wrote: > > I've put it as 256 args now. > > > > The overhead of that is about 2kB, so not really an issue. > > Why not just allocate it

Re: Remove Deprecated Exclusive Backup Mode

2019-02-26 Thread Fujii Masao
On Tue, Feb 26, 2019 at 3:49 PM David Steele wrote: > > On 2/26/19 6:51 AM, Michael Paquier wrote: > > On Mon, Feb 25, 2019 at 08:17:27PM +0200, David Steele wrote: > >> Here's the really obvious bad thing: if users do not update their > >> procedures > >> and we ignore backup_label.pending on

Re: No-rewrite timestamp<->timestamptz conversions

2019-02-26 Thread Tom Lane
Noah Misch writes: > On Tue, Feb 26, 2019 at 10:46:29AM -0500, Tom Lane wrote: >> It'd be nice to get the SupportRequestSimplify API correct from the first >> release, so if there's even a slightly plausible reason for it to support >> this, I'd be inclined to err in the direction of doing so. >

Re: pgbench MAX_ARGS

2019-02-26 Thread Andres Freund
Hi, On 2019-02-26 12:57:14 +, Simon Riggs wrote: > On Tue, 26 Feb 2019 at 12:19, Fabien COELHO wrote: > I've put it as 256 args now. > > The overhead of that is about 2kB, so not really an issue. Why not just allocate it dynamically? Seems weird to have all these MAX_ARGS, MAX_SCRIPTS ...

Re: Remove Deprecated Exclusive Backup Mode

2019-02-26 Thread Robert Haas
On Tue, Feb 26, 2019 at 12:31 PM Christophe Pettus wrote: > I believe #1 is when backup_label (no .pending) exists, #2 is when > backup_label.pending (with .pending) exists. Oh, whoops. I get it now. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: Remove Deprecated Exclusive Backup Mode

2019-02-26 Thread Fujii Masao
On Wed, Feb 27, 2019 at 2:27 AM Robert Haas wrote: > > On Tue, Feb 26, 2019 at 12:20 PM Fujii Masao wrote: > > So, let me clarify the situations; > > > > (1) If backup_label and recovery.signal exist, the recovery starts safely. > >This is the normal case of recovery from the base

Re: Remove Deprecated Exclusive Backup Mode

2019-02-26 Thread Christophe Pettus
> On Feb 26, 2019, at 09:26, Robert Haas wrote: > > On Tue, Feb 26, 2019 at 12:20 PM Fujii Masao wrote: >> So, let me clarify the situations; >> >> (1) If backup_label and recovery.signal exist, the recovery starts safely. >> This is the normal case of recovery from the base backup.

Re: No-rewrite timestamp<->timestamptz conversions

2019-02-26 Thread Noah Misch
On Tue, Feb 26, 2019 at 10:46:29AM -0500, Tom Lane wrote: > Noah Misch writes: > > Stepping back a bit, commit b8a18ad didn't provide a great UI. I doubt > > folks > > write queries this way spontaneously; to do so, they would have needed to > > learn that such syntax enables this optimization.

Re: Remove Deprecated Exclusive Backup Mode

2019-02-26 Thread Robert Haas
On Tue, Feb 26, 2019 at 12:20 PM Fujii Masao wrote: > So, let me clarify the situations; > > (1) If backup_label and recovery.signal exist, the recovery starts safely. >This is the normal case of recovery from the base backup. > > (2)If backup_label.pending and recovery.signal exist, as

Re: ATTACH/DETACH PARTITION CONCURRENTLY

2019-02-26 Thread Robert Haas
On Fri, Dec 21, 2018 at 6:04 PM David Rowley wrote: > On Fri, 21 Dec 2018 at 09:43, Robert Haas wrote: > > - I refactored expand_inherited_rtentry() to drive partition expansion > > entirely off of PartitionDescs. The reason why this is necessary is > > that it clearly will not work to have

Re: Remove Deprecated Exclusive Backup Mode

2019-02-26 Thread Fujii Masao
On Tue, Feb 26, 2019 at 3:17 AM David Steele wrote: > > On 2/25/19 7:50 PM, Fujii Masao wrote: > > On Mon, Feb 25, 2019 at 10:49 PM Laurenz Albe > > wrote: > >> > >> I'm not playing devil's advocate here to annoy you. I see the problems > >> with the exclusive backup, and I see how it can hurt

Re: crosstab/repivot...any interest?

2019-02-26 Thread Merlin Moncure
On Tue, Feb 26, 2019 at 8:31 AM Joe Conway wrote: > On 2/25/19 8:34 PM, Merlin Moncure wrote: > > The interface I'm looking at is: > > SELECT repivot( > > query TEXT, > > static_attributes INT, /* number of static attributes that are > > unchanged around key; we need this in our usages */ >

Re: Protect syscache from bloating with negative cache entries

2019-02-26 Thread Robert Haas
On Mon, Feb 25, 2019 at 1:27 AM Kyotaro HORIGUCHI wrote: > > I'd like to see some evidence that catalog_cache_memory_target has any > > value, vs. just always setting it to zero. > > It is artificial (or acutually wont't be repeatedly executed in a > session) but anyway what can get benefit from

Re: Protect syscache from bloating with negative cache entries

2019-02-26 Thread Robert Haas
On Mon, Feb 25, 2019 at 3:50 AM Tsunakawa, Takayuki wrote: > How can I make sure that this context won't exceed, say, 10 MB to avoid OOM? As Tom has said before and will probably say again, I don't think you actually want that. We know that PostgreSQL gets roughly 100x slower with the system

Re: No-rewrite timestamp<->timestamptz conversions

2019-02-26 Thread Tom Lane
Noah Misch writes: > Stepping back a bit, commit b8a18ad didn't provide a great UI. I doubt folks > write queries this way spontaneously; to do so, they would have needed to > learn that such syntax enables this optimization. If I'm going to do > something more invasive, it should optimize the

Re: Offline enabling/disabling of data checksums

2019-02-26 Thread Michael Banck
Hi, Am Dienstag, den 19.02.2019, 14:02 +0900 schrieb Michael Paquier: > On Sun, Feb 17, 2019 at 07:31:38PM +0100, Michael Banck wrote: > > New patch attached. > > - * src/bin/pg_verify_checksums/pg_verify_checksums.c > + * src/bin/pg_checksums/pg_checksums.c > That's lacking a rename, or this

Re: WIP: Avoid creation of the free space map for small tables

2019-02-26 Thread Petr Jelinek
On 26/02/2019 16:20, Alvaro Herrera wrote: > On 2019-Feb-23, John Naylor wrote: > >> On Fri, Feb 22, 2019 at 3:59 AM Amit Kapila wrote: >>> The reason for not pushing much on making the test pass for >>> nonstandard block sizes is that when I tried existing tests, there >>> were already some

Re: WIP: Avoid creation of the free space map for small tables

2019-02-26 Thread Alvaro Herrera
On 2019-Feb-23, John Naylor wrote: > On Fri, Feb 22, 2019 at 3:59 AM Amit Kapila wrote: > > The reason for not pushing much on making the test pass for > > nonstandard block sizes is that when I tried existing tests, there > > were already some failures. > > FWIW, I currently see 8 failures

Re: Early WIP/PoC for inlining CTEs

2019-02-26 Thread Tom Lane
Andrew Gierth writes: > Here, uncommenting that NOT actually changes the result, from 22 rows to > 4 rows, because we end up generating multiple worktable scans and the > recursion logic is not set up to handle that. Ugh. > So what I think we need to do here is to forbid inlining if (a) the >

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

2019-02-26 Thread Kuntal Ghosh
On Tue, Feb 26, 2019 at 6:46 PM Simon Riggs wrote: > > On Thu, 21 Feb 2019 at 15:38, Kuntal Ghosh wrote: > >> >> Thank you for the patch. It seems to me that while performing COPY >> FREEZE, if we've copied tuples in a previously emptied page > > > There won't be any previously emptied pages

Re: crosstab/repivot...any interest?

2019-02-26 Thread Joe Conway
On 2/25/19 8:34 PM, Merlin Moncure wrote: > No worries, sir! Apologies on the late reply.  I've made some headway on > this item.  Waiting for postgres to implement the SQL standard pivoting > (regardless if it implements the cases I need) is not an option for my > personal case. I can't use the

  1   2   >