Re: partitioning - changing a slot's descriptor is expensive

2018-08-17 Thread Amit Khandekar
On 29 June 2018 at 11:53, Amit Langote wrote: > What I'm thinking of doing is something that's inspired by one of the > things that David Rowley proposes in his patch for PG 12 to remove > inefficiencies in the tuple routing code [1]. > > Instead of a single TupleTableSlot attached at

Re: Fix help option of contrib/oid2name

2018-08-17 Thread Tatsuro Yamada
On 2018/08/17 12:42, Tatsuro Yamada wrote: On 2018/08/17 11:47, Michael Paquier wrote: On Thu, Aug 16, 2018 at 08:57:57PM +0900, Michael Paquier wrote: I agree on both points.  Any objections if I apply what's proposed here on HEAD? I have been looking at this patch.  And while consistency

[HACKERS] Proposal to add work_mem option to postgres_fdw module

2018-08-17 Thread Shinoda, Noriyoshi (PN Japan GCS Delivery)
Hello hackers, The attached patch adds a new option work_mem to postgres_fdw contrib module. Previously, it was impossible to change the setting of work_mem for remote session with connection by postgres_fdw. By adding this option to the CREATE SERVER statement, you can also change the

Re: FailedAssertion on partprune

2018-08-17 Thread David Rowley
On 17 August 2018 at 06:52, Robert Haas wrote: > I don't know whether there's actually a defect here any more. I was > trying to dispel some perceived confusion on the part of David and Tom > about what this code was trying to accomplish, but the fact that the > code caused some confusion does

Re: FailedAssertion on partprune

2018-08-17 Thread David Rowley
On 14 August 2018 at 09:23, Robert Haas wrote: > On Sat, Aug 11, 2018 at 9:16 AM, David Rowley > wrote: >> I started debugging this to see where things go wrong. I discovered >> that add_paths_to_append_rel() is called yet again from >> apply_scanjoin_target_to_paths() and this is where it's all

Re: Improve behavior of concurrent ANALYZE/VACUUM

2018-08-17 Thread Michael Paquier
On Mon, Aug 13, 2018 at 12:21:42AM +0200, Michael Paquier wrote: > The patch attached includes tests which I have used to also check that > correct error messages are produced for VACUUM, VACUUM ANALYZE and > ANALYZE. I have reworked the patch on this side, clarifying the use of the new common

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-08-17 Thread Fabien COELHO
Hello Marina, Detailed -r report. I understand from the doc that the retry number on the detailed per-statement report is to identify at what point errors occur? Probably this is more or less always at the same point on a given script, so that the most interesting feature is to report the

Re: Doc patch: pg_upgrade page and checkpoint location consistency with replicas

2018-08-17 Thread Paul Bonaud
I shared the pach in plain textin the email body and figured out that all other patches are submitted as an attachement. Sorry for that, here is the patch attached to this email. Thanks! Paul On 17/08/18 01:21, Paul Bonaud wrote: > Hello, > > Please find below a submission of a patch to the

Re: TupleTableSlot abstraction

2018-08-17 Thread Andres Freund
Hi, On 2018-08-17 12:10:20 +0530, Ashutosh Bapat wrote: > We need to add LLVM code to fetch tts_flags and > perform bit operation on it to get or set slow property. I haven't > found any precedence for LLVM bit operations in postgresql's JIT code. There are several, look for the infomask

Re: ToDo: show size of partitioned table

2018-08-17 Thread Mathias Brossard
On Thu, Aug 16, 2018 at 12:46 AM Pavel Stehule wrote: > čt 16. 8. 2018 v 5:52 odesílatel Mathias Brossard > napsal: > >> I do have a feedback on the implementation. The code tries to support >> older PostgreSQL server versions when declarative partitions were not >> supported before version 10

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-08-17 Thread Marina Polyakova
On 17-08-2018 10:49, Fabien COELHO wrote: Hello Marina, Detailed -r report. I understand from the doc that the retry number on the detailed per-statement report is to identify at what point errors occur? Probably this is more or less always at the same point on a given script, so that the

libpq stricter integer parsing

2018-08-17 Thread Fabien COELHO
Follow up on a patch and discussion with Tom, currently integer parsing on keywords in libpq is quite loose, resulting in trailing garbage being ignored and allowing to hide bugs, eg: sh> psql "connect_timeout=2,port=5433" The timeout is set to 2, and the port directive is silently

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-08-17 Thread Marina Polyakova
On 17-08-2018 14:04, Fabien COELHO wrote: ... Or perhaps we can use a more detailed failure status so for each type of failure we always know the command name (argument "cmd") and whether the client is aborted. Something like this (but in comparison with the first variant ISTM overly

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-08-17 Thread Andres Freund
Hi, On 2018-08-17 18:00:20 +0300, Alexander Korotkov wrote: > So, do we have any objections to committing this? I think this needs more review by other senior hackers in the community. Greetings, Andres Freund

Re: Facility for detecting insecure object naming

2018-08-17 Thread Bruce Momjian
On Thu, Aug 16, 2018 at 10:58:21PM -0400, Chapman Flack wrote: > On 08/16/18 21:31, Bruce Momjian wrote: > > > I understand you don't like that a search_path changed by a function is > > passed down to functions it calls, but what would you like the system to > > use as a search path for called

Re: docs: note ownership requirement for refreshing materialized views

2018-08-17 Thread Dian Fay
Jonathan's patch seems like a good idea to me from a user POV, but then I just showed up the other day so I don't really have anything of substance to add. On 8/17/18 9:08 AM, Dave Cramer wrote: Dave Cramer On Thu, 16 Aug 2018 at 18:27, Jonathan S. Katz

Re: Fix help option of contrib/oid2name

2018-08-17 Thread Alvaro Herrera
On 2018-Aug-17, Tatsuro Yamada wrote: > only oid2name > - Replace -H with -h I think this one is a bad idea, as it'll break scripts. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [PATCH] Improve geometric types

2018-08-17 Thread Tomas Vondra
Hi, the buildfarm seems to be mostly happy so far, so I've taken a quick look at the remaining two parts. The patches still apply, but I'm getting plenty of failures in regression tests, due to 0.0 being replaced by -0.0. This reminds me 74294c7301, except that these patches don't seem to remove

Re: docs: note ownership requirement for refreshing materialized views

2018-08-17 Thread Dave Cramer
Dave Cramer On Thu, 16 Aug 2018 at 18:27, Jonathan S. Katz < jonathan.k...@excoventures.com> wrote: > > On Aug 16, 2018, at 1:05 AM, Jonathan S. Katz < > jonathan.k...@excoventures.com> wrote: > > > On Aug 15, 2018, at 9:15 PM, Michael Paquier wrote: > > On Wed, Aug 15, 2018 at 09:06:34PM

Re: Memory leak with CALL to Procedure with COMMIT.

2018-08-17 Thread Peter Eisentraut
On 16/08/2018 19:26, Tom Lane wrote: >> When a CALL has output parameters, the portal uses the strategy >> PORTAL_UTIL_SELECT instead of PORTAL_MULTI_QUERY. Using >> PORTAL_UTIL_SELECT causes the portal's snapshot to be registered with >> the current resource owner (portal->holdSnapshot). I'm

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-08-17 Thread Tom Lane
Andres Freund writes: > On 2018-08-17 18:00:20 +0300, Alexander Korotkov wrote: >> So, do we have any objections to committing this? > I think this needs more review by other senior hackers in the community. TBH it sounds like a horrible hack. Disable vacuum truncation? That can't be a good

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-08-17 Thread Andres Freund
On 2018-08-17 11:35:40 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2018-08-17 18:00:20 +0300, Alexander Korotkov wrote: > >> So, do we have any objections to committing this? > > > I think this needs more review by other senior hackers in the community. > > TBH it sounds like a

Re: docs: note ownership requirement for refreshing materialized views

2018-08-17 Thread Tom Lane
Dave Cramer writes: > So it seems this patch is being ignored in this thread. Well, Jonathan did kind of hijack what appears to be a thread about documentation (with an already-committed fix). I'd suggest reposting that patch in its own thread and adding it to the next CF.

Re: Fix help option of contrib/oid2name

2018-08-17 Thread Tom Lane
Alvaro Herrera writes: > On 2018-Aug-17, Tatsuro Yamada wrote: >> only oid2name >> - Replace -H with -h > I think this one is a bad idea, as it'll break scripts. Well, we can't remove the -H option, for that reason. But I think we could get away with repurposing -h to also mean "--host",

Re: Pre-v11 appearances of the word "procedure" in v11 docs

2018-08-17 Thread Peter Eisentraut
Attached are my proposed patches. The first is the documentation change, which basically just substitutes the words, with some occasional rephrasing. And then patches to extend the syntaxes of CREATE OPERATOR, CREATE TRIGGER, and CREATE EVENT TRIGGER to accept FUNCTION in place of PROCEDURE. I

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-08-17 Thread Alexander Korotkov
On Thu, Aug 16, 2018 at 2:16 PM Alexander Korotkov wrote: > On Tue, Aug 14, 2018 at 12:05 PM Masahiko Sawada > wrote: > > > > On Wed, Feb 28, 2018 at 11:24 PM, Ivan Kartyshov > > wrote: > > > The main goal of my changes is to let long read-only transactions run on > > > replica if

Re: [PATCH] Improve geometric types

2018-08-17 Thread Tomas Vondra
On 08/17/2018 06:40 PM, Emre Hasegeli wrote: >> the buildfarm seems to be mostly happy so far, so I've taken a quick >> look at the remaining two parts. The patches still apply, but I'm >> getting plenty of failures in regression tests, due to 0.0 being >> replaced by -0.0. > > I think we are

Slotification of partition tuple conversion

2018-08-17 Thread Amit Khandekar
Hi, In [1] , it was shown that the partition tuples are needlessly formed and deformed during tuple conversion (do_convert_tuple), when the same operation can be done using tuple slots. This is because the input slot might already have a deformed tuple. Attached is a patch tup_convert.patch that

Re: [PATCH] Improve geometric types

2018-08-17 Thread Emre Hasegeli
> the buildfarm seems to be mostly happy so far, so I've taken a quick > look at the remaining two parts. The patches still apply, but I'm > getting plenty of failures in regression tests, due to 0.0 being > replaced by -0.0. I think we are better off fixing them locally at the moment like your

Re: Index Skip Scan

2018-08-17 Thread Jesper Pedersen
Hi Peter, On 08/16/2018 03:48 PM, Peter Geoghegan wrote: On Wed, Aug 15, 2018 at 11:22 PM, Thomas Munro wrote: * groups and certain aggregates (MIN() and MAX() of suffix index columns within each group) * index scans where the scan key doesn't include the leading columns (but you expect there

Re: Index Skip Scan

2018-08-17 Thread Peter Geoghegan
On Thu, Aug 16, 2018 at 4:10 PM, Thomas Munro wrote: > Can you give an example of problematic ndistinct underestimation? Yes. See https://postgr.es/m/cakuk5j12qokfh88tqz-ojmsibg2qyjm7k7hlnbyi3ze+y5b...@mail.gmail.com, for example. That's a complaint about an underestimation specifically. This

Performance improvements for src/port/snprintf.c

2018-08-17 Thread Tom Lane
Over in the what-about-%m thread, we speculated about replacing the platform's *printf functions if they didn't support %m, which would basically mean using src/port/snprintf.c on all non-glibc platforms, rather than only on Windows as happens right now (ignoring some obsolete platforms with

Re: [PATCH] Improve geometric types

2018-08-17 Thread Tomas Vondra
On 08/17/2018 08:24 PM, Emre Hasegeli wrote: >> BTW how did we end up with the regression differences? Presumably you've >> tried that on your machine and it passed. So if we adjust the expected >> file, won't it fail on some other machines? > > I had another patch to check for -0 inside

Re: [PATCH] Improve geometric types

2018-08-17 Thread Tom Lane
Emre Hasegeli writes: >> BTW how did we end up with the regression differences? Presumably you've >> tried that on your machine and it passed. So if we adjust the expected >> file, won't it fail on some other machines? > I had another patch to check for -0 inside float{4,8}_{div,mul}(). I >

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-08-17 Thread Alexander Korotkov
On Fri, Aug 17, 2018 at 6:41 PM Andres Freund wrote: > On 2018-08-17 11:35:40 -0400, Tom Lane wrote: > > Andres Freund writes: > > > On 2018-08-17 18:00:20 +0300, Alexander Korotkov wrote: > > >> So, do we have any objections to committing this? > > > > > I think this needs more review by other

Re: partitioning - changing a slot's descriptor is expensive

2018-08-17 Thread Amit Khandekar
On 29 June 2018 at 11:53, Amit Langote wrote: > Other issues that you mentioned, such as needless heap_tuple_deform/form > being invoked, seem less localized (to me) than this particular issue, so > I created a patch for just this, which is attached with this email. I'm > thinking about how to

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-08-17 Thread Alexander Korotkov
On Fri, Aug 17, 2018 at 8:38 PM Tom Lane wrote: > Alexander Korotkov writes: > > On Fri, Aug 17, 2018 at 6:41 PM Andres Freund wrote: > >> There's another patch, which I thought Alexander was referring to, that > >> does something a bit smarger. On a super short skim it seems to > >> introduce

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-08-17 Thread Tom Lane
Alexander Korotkov writes: > On Fri, Aug 17, 2018 at 8:38 PM Tom Lane wrote: >> Alexander Korotkov writes: >>> Yes, that's correct. On standby read-only queries can tolerate >>> concurrent heap truncation. >> Uh, what??? > VACUUM truncates heap relation only after deletion of all the tuples

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-08-17 Thread Tom Lane
Alexander Korotkov writes: > On Fri, Aug 17, 2018 at 6:41 PM Andres Freund wrote: >> There's another patch, which I thought Alexander was referring to, that >> does something a bit smarger. On a super short skim it seems to >> introduce a separate type of AEL lock that's not replicated, by my

Re: [PATCH] Improve geometric types

2018-08-17 Thread Emre Hasegeli
> BTW how did we end up with the regression differences? Presumably you've > tried that on your machine and it passed. So if we adjust the expected > file, won't it fail on some other machines? I had another patch to check for -0 inside float{4,8}_{div,mul}(). I dropped it on the last set of

Getting NOT NULL constraint from pg_attribute

2018-08-17 Thread Wu Ivy
Hi developers, I’m currently building a Postgres C extension that fetch data from a Postgres table. Since the table can be large, in order to prevent memory overrun, I use SPI_cursor_fetch to fetch chunks of data. The result rows are saved in SPITupleTable* SPI_tuptable and attributes are

Re: Fix for REFRESH MATERIALIZED VIEW ownership error message

2018-08-17 Thread Dave Cramer
On Fri, 17 Aug 2018 at 19:35, Alvaro Herrera wrote: > On 2018-Aug-17, Dave Cramer wrote: > > > The only place this is used is in aclcheck_error > > case OBJECT_MATVIEW: > > msg = gettext_noop("permission denied for materialized view %s"); > > break; > > Yes, but do we pass

Re: Getting NOT NULL constraint from pg_attribute

2018-08-17 Thread Tom Lane
Wu Ivy writes: > I’m currently building a Postgres C extension that fetch data from a Postgres > table. > Since the table can be large, in order to prevent memory overrun, I use > SPI_cursor_fetch to fetch chunks of data. The result rows are saved in > SPITupleTable* SPI_tuptable and

Re: Fix help option of contrib/oid2name

2018-08-17 Thread Michael Paquier
On August 17, 2018 10:53:48 PM GMT+09:00, Tom Lane wrote: > Well, we can't remove the -H option, for that reason. But I think > we could get away with repurposing -h to also mean "--host", rather > than "--help" as it is now. Seems unlikely that any scripts are > depending on it to mean

Re: Fix help option of contrib/oid2name

2018-08-17 Thread Michael Paquier
On August 18, 2018 10:52:33 AM GMT+09:00, Tom Lane wrote: > I think it probably needs to stay documented, but we could mark it as > deprecated ... Okay, no issues with doing so. -- Michael

Re: docs: note ownership requirement for refreshing materialized views

2018-08-17 Thread Jonathan S. Katz
> On Aug 17, 2018, at 9:21 AM, Tom Lane wrote: > > Dave Cramer writes: >> So it seems this patch is being ignored in this thread. > > Well, Jonathan did kind of hijack what appears to be a thread about > documentation (with an already-committed fix). I apologize if it was interpreted as

Re: [PATCH] Improve geometric types

2018-08-17 Thread Tom Lane
Tomas Vondra writes: > Hmm, yeah. Based on past experience, the powerpc machines are likely to > stumble on this. > FWIW my understanding is that these failures actually happen in new > tests, it's not an issue introduced by this patch series. Yeah, we've definitely hit such problems before.

Re: Fix for REFRESH MATERIALIZED VIEW ownership error message

2018-08-17 Thread Alvaro Herrera
On 2018-Aug-17, Dave Cramer wrote: > The only place this is used is in aclcheck_error > case OBJECT_MATVIEW: > msg = gettext_noop("permission denied for materialized view %s"); > break; Yes, but do we pass RefreshMatViewStmt->relkind to that routine? I don't see that we do. Maybe I misread the

Re: Fix help option of contrib/oid2name

2018-08-17 Thread Tom Lane
Michael Paquier writes: > On August 17, 2018 10:53:48 PM GMT+09:00, Tom Lane wrote: >> Well, we can't remove the -H option, for that reason. But I think >> we could get away with repurposing -h to also mean "--host", rather >> than "--help" as it is now. Seems unlikely that any scripts are >>

Re: Fix for REFRESH MATERIALIZED VIEW ownership error message

2018-08-17 Thread Dave Cramer
Dave Cramer da...@postgresintl.com www.postgresintl.com On Fri, 17 Aug 2018 at 18:30, Alvaro Herrera wrote: > On 2018-Aug-17, Jonathan S. Katz wrote: > > > Hi, > > > > I Initially pointed out here[1] that running REFRESH MATERIALIZED VIEW > as a > > non-superuser or table owner yields the

Re: [PATCH] Improve geometric types

2018-08-17 Thread Tomas Vondra
On 08/17/2018 08:56 PM, Tom Lane wrote: > Emre Hasegeli writes: >>> BTW how did we end up with the regression differences? Presumably you've >>> tried that on your machine and it passed. So if we adjust the expected >>> file, won't it fail on some other machines? > >> I had another patch to

Re: Pre-v11 appearances of the word "procedure" in v11 docs

2018-08-17 Thread Peter Geoghegan
On Fri, Aug 17, 2018 at 7:15 AM, Peter Eisentraut wrote: > Attached are my proposed patches. I take it that you propose all 3 for backpatch to v11? -- Peter Geoghegan

Fix for REFRESH MATERIALIZED VIEW ownership error message

2018-08-17 Thread Jonathan S. Katz
Hi,I Initially pointed out here[1] that running REFRESH MATERIALIZED VIEW as anon-superuser or table owner yields the following message:    test=> REFRESH MATERIALIZED VIEW blah;    ERROR: must be owner of relation blahThe error message should say "...owner of materialized view..."The attached

Re: InsertPgAttributeTuple() and attcacheoff

2018-08-17 Thread Peter Eisentraut
On 14/08/2018 17:52, Robert Haas wrote: > On Tue, Aug 14, 2018 at 3:50 PM, Tom Lane wrote: >> Peter Eisentraut writes: >>> It seems to me that it would make sense if InsertPgAttributeTuple() were >>> to set attcacheoff to -1 instead of taking it from the caller. >> >> Looked this over, no

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-08-17 Thread Alexander Korotkov
On Fri, Aug 17, 2018 at 9:55 PM Tom Lane wrote: > Alexander Korotkov writes: > > On Fri, Aug 17, 2018 at 8:38 PM Tom Lane wrote: > >> Alexander Korotkov writes: > >>> Yes, that's correct. On standby read-only queries can tolerate > >>> concurrent heap truncation. > > >> Uh, what??? > > >

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-08-17 Thread Tom Lane
Alexander Korotkov writes: > On Fri, Aug 17, 2018 at 9:55 PM Tom Lane wrote: >> Another point is that the truncation code attempts to remove all >> to-be-truncated-away pages from the shared buffer arena, but that only >> works if nobody else is loading such pages into shared buffers >>

Re: Fix for REFRESH MATERIALIZED VIEW ownership error message

2018-08-17 Thread Alvaro Herrera
On 2018-Aug-17, Jonathan S. Katz wrote: > Hi, > > I Initially pointed out here[1] that running REFRESH MATERIALIZED VIEW as a > non-superuser or table owner yields the following message: > > test=> REFRESH MATERIALIZED VIEW blah; > ERROR: must be owner of relation blah > > The error

Re: Conflict handling for COPY FROM

2018-08-17 Thread Karen Huddleston
Hi Surafel, Andrew and I began reviewing your patch. It applied cleanly and seems to mostly have the functionality you describe. We did have some comments/questions. 1. It sounded like you added the copy_max_error_limit GUC as part of this patch to allow users to specify how many errors they