Re: [HACKERS] Prologue of set_append_rel_size() and partitioned tables

2017-03-28 Thread Amit Langote
On 2017/03/29 15:20, Ashutosh Bapat wrote: > The prologue of set_append_rel_size() mentions > > * Note that in the inheritance case, > * the first member relation is actually the same table as is mentioned in > * the parent RTE ... but it has a different RTE and RelOptInfo. > > This isn'

Re: [HACKERS] Partitioned tables and relfilenode

2017-03-28 Thread Amit Langote
On 2017/03/27 23:27, Robert Haas wrote: > On Thu, Mar 23, 2017 at 8:54 PM, Amit Langote > wrote: >> On 2017/03/23 23:47, Amit Langote wrote: >>> On Thu, Mar 23, 2017 at 11:27 PM, Maksim Milyutin >>> wrote: Hi! I have noticed that there is scheduled unlinking of nonexistent physical

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-28 Thread Amit Kapila
On Tue, Mar 28, 2017 at 10:31 PM, Pavan Deolasee wrote: > > On Tue, Mar 28, 2017 at 4:05 PM, Amit Kapila > wrote: >> >> As asked previously, can you explain me on what basis are you >> considering it robust? The comments on top of datumIsEqual() clearly >> indicates the danger of using it for to

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-28 Thread Amit Kapila
On Wed, Mar 29, 2017 at 11:52 AM, Amit Kapila wrote: > On Tue, Mar 28, 2017 at 10:35 PM, Pavan Deolasee > wrote: >> >> On Tue, Mar 28, 2017 at 7:04 PM, Amit Kapila >> wrote: >>> >>> For such an heap insert, we will pass >>> the actual value of column to index_form_tuple during index insert. >>

Re: [HACKERS] [COMMITTERS] pgsql: Allow vacuums to report oldestxmin

2017-03-28 Thread Masahiko Sawada
On Wed, Mar 29, 2017 at 1:32 AM, Fujii Masao wrote: > On Tue, Mar 28, 2017 at 1:06 AM, Masahiko Sawada > wrote: >> On Sun, Mar 26, 2017 at 2:26 AM, Masahiko Sawada >> wrote: >>> On Sun, Mar 26, 2017 at 1:37 AM, Fujii Masao wrote: On Mon, Mar 6, 2017 at 9:37 PM, Masahiko Sawada wro

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-28 Thread Masahiko Sawada
On Wed, Mar 29, 2017 at 12:23 AM, David Steele wrote: > On 3/23/17 1:54 AM, Masahiko Sawada wrote: >> >> On Wed, Mar 15, 2017 at 7:51 AM, Peter Geoghegan wrote: >>> >>> On Tue, Mar 14, 2017 at 3:10 PM, Peter Geoghegan wrote: We already have BTPageOpaqueData.btpo, a union whose containe

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-28 Thread Amit Kapila
On Tue, Mar 28, 2017 at 10:35 PM, Pavan Deolasee wrote: > > On Tue, Mar 28, 2017 at 7:04 PM, Amit Kapila > wrote: >> >> >> >> For such an heap insert, we will pass >> the actual value of column to index_form_tuple during index insert. >> However during recheck when we fetch the value of c2 from

Re: [HACKERS] [patch] reorder tablespaces in basebackup tar stream for backup_label

2017-03-28 Thread Michael Paquier
On Wed, Mar 29, 2017 at 3:56 AM, Fujii Masao wrote: > If your need other information except START WAL LOCATION at the beginning of > base backup and they are very useful for many third-party softwares, > you can add them into that first result set. If you do this, you can > retrieve them > at the

[HACKERS] Prologue of set_append_rel_size() and partitioned tables

2017-03-28 Thread Ashutosh Bapat
The prologue of set_append_rel_size() mentions * Note that in the inheritance case, * the first member relation is actually the same table as is mentioned in * the parent RTE ... but it has a different RTE and RelOptInfo. This isn't true about partitioned tables anymore. We do not create

Re: [HACKERS] [sqlsmith] Unpinning error in parallel worker

2017-03-28 Thread Thomas Munro
On Mon, Mar 27, 2017 at 6:53 PM, Thomas Munro wrote: > On Mon, Mar 27, 2017 at 8:38 AM, Thomas Munro > wrote: >> On Mon, Mar 27, 2017 at 4:18 AM, Andreas Seltenreich >> wrote: >>> Hi, >>> >>> today's testing with master as of d253b0f6e3 yielded two clusters that >>> stopped processing queries.

Re: [HACKERS] Multiple false-positive warnings from Valgrind

2017-03-28 Thread Kyotaro HORIGUCHI
At Wed, 29 Mar 2017 12:34:52 +0900, Michael Paquier wrote in > On Thu, Mar 23, 2017 at 5:15 PM, Michael Paquier > wrote: > > On Tue, Mar 21, 2017 at 10:57 PM, Aleksander Alekseev > > wrote: > >> Recently I've decided to run PostgreSQL under Valgrind according to wiki > >> description [1]. Lots

Re: [HACKERS] [POC] A better way to expand hash indexes.

2017-03-28 Thread Amit Kapila
On Tue, Mar 28, 2017 at 8:56 PM, Robert Haas wrote: > On Tue, Mar 28, 2017 at 5:00 AM, Mithun Cy wrote: >>> This will go wrong for split point group zero. In general, I feel if >>> you handle computation for split groups lesser than >>> SPLITPOINT_GROUPS_WITH_ONLY_ONE_PHASE in the caller, then a

Re: [HACKERS] \if, \elseif, \else, \endif (was Re: PSQL commands: \quit_if, \quit_unless)

2017-03-28 Thread Fabien COELHO
Hello Tom, psql_if_on_error_stop... ok (test process exited with exit code 3) Don't think we can have that. Even if pg_regress considers it a success, every hacker is going to have to learn that that's a "pass", Well, it says "ok"... and I don't think I want to be answering that questi

Re: [HACKERS] [COMMITTERS] pgsql: Improve access to parallel query from procedural languages.

2017-03-28 Thread Rafia Sabih
On Tue, Mar 28, 2017 at 9:05 PM, Robert Haas wrote: > OK, but don't pg_event_trigger_dropped_objects and > pg_event_trigger_ddl_commands need the same treatment? > Done. I was only concentrating on the build farm failure cases, otherwise I think more work might be required in this direction. --

[HACKERS] Fix obsolete comment in GetSnapshotData

2017-03-28 Thread Craig Ringer
Hi all There's an outdated reference to GetOldestXmin(true, true) in GetSnapshotData. It hasn't had that call signature for a long while now. Update the comment to reflect the current signature. diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c index f32881b..

[HACKERS] show "aggressive" or not in autovacuum logs

2017-03-28 Thread Kyotaro HORIGUCHI
Hello, it would be too late but I'd like to propose this because this cannot be back-patched. In autovacuum logs, "%u skipped frozen" shows the number of pages skipped by ALL_FROZEN only in aggressive vacuum. So users cannot tell whether '0 skipped-frozen' means a non-agressive vacuum or no froz

Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall

2017-03-28 Thread Haribabu Kommi
On Tue, Mar 28, 2017 at 12:50 AM, Andreas Karlsson wrote: > Hi, > > Here is my review. I agree with the goal of the refactoring, as we want to > make it easier to dump all the properties for the database object. But I > think we need to solve the issues with the special casing of postgres and > t

Re: [HACKERS] [PATCH] Reduce src/test/recovery verbosity

2017-03-28 Thread Michael Paquier
On Wed, Mar 29, 2017 at 12:37 PM, Craig Ringer wrote: > On 29 March 2017 at 08:57, Peter Eisentraut > wrote: >> On 3/22/17 03:18, Craig Ringer wrote: >>> Trivial patch to change 'diag' to 'note' in TAP tests in >>> src/test/recovery attached. >>> >>> It'll reduce the test output a little. >> >> C

Re: [HACKERS] [PATCH] Reduce src/test/recovery verbosity

2017-03-28 Thread Craig Ringer
On 29 March 2017 at 08:57, Peter Eisentraut wrote: > On 3/22/17 03:18, Craig Ringer wrote: >> Trivial patch to change 'diag' to 'note' in TAP tests in >> src/test/recovery attached. >> >> It'll reduce the test output a little. > > Committed, and also done the same in src/test/ssl/. Thanks. It won

Re: [HACKERS] On How To Shorten the Steep Learning Curve Towards PG Hacking...

2017-03-28 Thread Craig Ringer
On 29 March 2017 at 10:53, Amit Langote wrote: > Hi, > > On 2017/03/28 15:40, Kang Yuzhe wrote: >> Thanks Tsunakawa for such an informative reply. >> >> Almost all of the docs related to the internals of PG are of introductory >> concepts only. >> There is even more useful PG internals site entitl

Re: [HACKERS] Schedule and Release Management Team for PG10

2017-03-28 Thread Tom Lane
Robert Haas writes: > On Thu, Mar 23, 2017 at 10:34 AM, Tom Lane wrote: >> As previously agreed at the PGCon 2016 developers meeting, we'll >> institute v10 feature freeze at the completion of the current >> commitfest (end of this month). > This means, as I understand it, that no new features s

Re: [HACKERS] Multiple false-positive warnings from Valgrind

2017-03-28 Thread Michael Paquier
On Thu, Mar 23, 2017 at 5:15 PM, Michael Paquier wrote: > On Tue, Mar 21, 2017 at 10:57 PM, Aleksander Alekseev > wrote: >> Recently I've decided to run PostgreSQL under Valgrind according to wiki >> description [1]. Lots of warnings are generated [2] but it is my >> understanding that all of the

Re: [HACKERS] postgres_fdw IMPORT SCHEMA and partitioned tables

2017-03-28 Thread Michael Paquier
On Wed, Mar 29, 2017 at 12:22 PM, Robert Haas wrote: > On Thu, Mar 9, 2017 at 8:47 PM, Amit Langote > wrote: >>> It is not as straight-forward as it seems. A foreign table can be >>> defined as a child (use of PARTITION OF), but not as a parent (use >>> PARTITION BY), and IMPORT SCHEMA has to iss

Re: [HACKERS] postgres_fdw IMPORT SCHEMA and partitioned tables

2017-03-28 Thread Robert Haas
On Thu, Mar 9, 2017 at 8:47 PM, Amit Langote wrote: >> It is not as straight-forward as it seems. A foreign table can be >> defined as a child (use of PARTITION OF), but not as a parent (use >> PARTITION BY), and IMPORT SCHEMA has to issue queries to create >> foreign tables. It seems to me that t

Re: [HACKERS] Partitioned tables and relfilenode

2017-03-28 Thread Robert Haas
On Mon, Mar 27, 2017 at 8:59 PM, Amit Langote wrote: > Oops, my bad. I will include it in the patch I'll send after addressing > Robert's comments. Thanks again! That patch coming soon? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via p

Re: [HACKERS] Schedule and Release Management Team for PG10

2017-03-28 Thread Robert Haas
On Thu, Mar 23, 2017 at 10:34 AM, Tom Lane wrote: > As previously agreed at the PGCon 2016 developers meeting, we'll > institute v10 feature freeze at the completion of the current > commitfest (end of this month). This means, as I understand it, that no new features should be committed by anyone

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-03-28 Thread Robert Haas
On Tue, Mar 28, 2017 at 12:54 PM, Robert Haas wrote: > Regarding 0002, I think the parts that involve factoring out > find_param_path_info() are uncontroversial. Regarding the changes to > adjust_appendrel_attrs(), my main question is whether we wouldn't be > better off using an array representat

Re: [HACKERS] crashes due to setting max_parallel_workers=0

2017-03-28 Thread Tomas Vondra
Hi, On 03/28/2017 11:07 AM, Rushabh Lathia wrote: ... I think we all agree that we should get rid of nreaders from the GatherMergeState and need to do some code re-factor. But if I understood correctly that Robert's concern was to do that re-factor as separate commit. Maybe. It depends on

Re: [HACKERS] Forbid use of LF and CR characters in database and role names

2017-03-28 Thread Michael Paquier
On Wed, Feb 1, 2017 at 1:31 PM, Michael Paquier wrote: > On Wed, Feb 1, 2017 at 1:31 PM, Michael Paquier > wrote: >> On Tue, Nov 29, 2016 at 1:33 PM, Michael Paquier >> wrote: >>> Patch moved to CF 2017-01. >> >> And nothing has happened since, the patch rotting a bit because of a >> conflict in

Re: [HACKERS] On How To Shorten the Steep Learning Curve Towards PG Hacking...

2017-03-28 Thread Amit Langote
Hi, On 2017/03/28 15:40, Kang Yuzhe wrote: > Thanks Tsunakawa for such an informative reply. > > Almost all of the docs related to the internals of PG are of introductory > concepts only. > There is even more useful PG internals site entitled "The Internals of > PostgreSQL" in http://www.interdb.

Re: [HACKERS] Missing increment of vacrelstats->pinskipped_pages

2017-03-28 Thread Kyotaro HORIGUCHI
Hello, At Tue, 28 Mar 2017 14:15:06 -0400, Jim Nasby wrote in <61de5a80-2ffd-6b05-61e4-210fcb299...@nasby.net> > lazy_vacuum_heap() does not count pages that it skips due to not > obtaining the buffer cleanup lock. vacuum_pinskipped.patch fixes > that. That should be backpatched to 9.5. > > vac

Re: [HACKERS] cast result of copyNode()

2017-03-28 Thread Peter Eisentraut
On 3/21/17 18:52, Mark Dilger wrote: > The patch applies cleanly, compiles, and passes all the regression tests > for me on my laptop. Peter appears to have renamed the function copyObject > as copyObjectImpl, which struct me as odd when I first saw it, but I don't > have > a better name in mind,

Re: [HACKERS] Getting server crash after running sqlsmith

2017-03-28 Thread Thomas Munro
On Wed, Mar 29, 2017 at 2:23 PM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Mar 28, 2017 at 2:36 PM, Tom Lane wrote: >>> Hm ... I don't see a crash here, but I wonder whether you have parameters >>> set that would cause this query to be run as a parallel query? Because >>> pg_rotate_logfi

Re: [HACKERS] Getting server crash after running sqlsmith

2017-03-28 Thread Robert Haas
On Tue, Mar 28, 2017 at 9:23 PM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Mar 28, 2017 at 2:36 PM, Tom Lane wrote: >>> Hm ... I don't see a crash here, but I wonder whether you have parameters >>> set that would cause this query to be run as a parallel query? Because >>> pg_rotate_logfi

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2017-03-28 Thread Claudio Freire
On Wed, Feb 1, 2017 at 7:55 PM, Claudio Freire wrote: > On Wed, Feb 1, 2017 at 6:13 PM, Masahiko Sawada wrote: >> On Wed, Feb 1, 2017 at 10:02 PM, Claudio Freire >> wrote: >>> On Wed, Feb 1, 2017 at 5:47 PM, Masahiko Sawada >>> wrote: Thank you for updating the patch. Whole pat

Re: [HACKERS] free space map and visibility map

2017-03-28 Thread Kyotaro HORIGUCHI
Hello, At Tue, 28 Mar 2017 08:50:58 -0700, Jeff Janes wrote in > > > I now think this is not the cause of the problem I am seeing. I made the > > > replay of FREEZE_PAGE update the FSM (both with and without FPI), but > > that > > > did not fix it. With frequent crashes, it still accumulated

Re: [HACKERS] Getting server crash after running sqlsmith

2017-03-28 Thread Tom Lane
Robert Haas writes: > On Tue, Mar 28, 2017 at 2:36 PM, Tom Lane wrote: >> Hm ... I don't see a crash here, but I wonder whether you have parameters >> set that would cause this query to be run as a parallel query? Because >> pg_rotate_logfile() is marked as parallel-safe in pg_proc, which seems

Re: [HACKERS] make check-world output

2017-03-28 Thread Peter Eisentraut
On 3/10/17 19:15, Jeff Janes wrote: > Should --enable-tap-tests be mentioned in "32.1.3. Additional Test > Suites"? Or at least cross-referenced from "32.4. TAP Tests"? Done. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Traini

Re: [HACKERS] [COMMITTERS] pgsql: Improve performance of find_all_inheritors()

2017-03-28 Thread Robert Haas
On Tue, Mar 28, 2017 at 3:59 PM, Aleksander Alekseev wrote: > Hi, Robert. > >> I'm a little worried that this will be noticeably slower when the >> number of partitions is small, like 4 or 8 or 16. In other places, >> we've found it necessary to support both a list-based strategy and a >> hash-ba

Re: [HACKERS] Getting server crash after running sqlsmith

2017-03-28 Thread Robert Haas
On Tue, Mar 28, 2017 at 2:36 PM, Tom Lane wrote: > tushar writes: >> After runinng sqlsmith against latest sources of PG v10 , able to see a >> crash - > > Hm ... I don't see a crash here, but I wonder whether you have parameters > set that would cause this query to be run as a parallel query?

Re: [HACKERS] make check-world output

2017-03-28 Thread Peter Eisentraut
On 3/13/17 05:35, Dagfinn Ilmari Mannsåker wrote: > Another thing I noticed is that there's a bunch of 'diag' calls in the > tests scripts (particularly ssl/t/001_ssltests.pl and > recovery/t/001_stream_rep.pl) that should probably be 'note's instead, > so they don't pollute STDERR in non-verbose m

Re: [HACKERS] [PATCH] Reduce src/test/recovery verbosity

2017-03-28 Thread Peter Eisentraut
On 3/22/17 03:18, Craig Ringer wrote: > Trivial patch to change 'diag' to 'note' in TAP tests in > src/test/recovery attached. > > It'll reduce the test output a little. Committed, and also done the same in src/test/ssl/. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL D

Re: [HACKERS] \if, \elseif, \else, \endif (was Re: PSQL commands: \quit_if, \quit_unless)

2017-03-28 Thread Tom Lane
Corey Huinker writes: [ 0001-psql-if-v28.patch ] Starting to look at this version, and what jumped out at me in testing is that the regression output looks like this: parallel group (12 tests): psql_if_on_error_stop dbsize async misc_functions tidscan alter_operator tsrf psql alter_generic mis

Re: [HACKERS] logical decoding of two-phase transactions

2017-03-28 Thread Craig Ringer
On 28 Mar. 2017 23:08, "Andres Freund" wrote: > > > >> I don't think its for us to say what the plugin is allowed to do. We > > >> decided on a plugin architecture, so we have to trust that the plugin > > >> author resolves the issues. We can document them so those choices are > > >> clear. > > >

Re: [HACKERS] Supporting huge pages on Windows

2017-03-28 Thread Tsunakawa, Takayuki
From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of David Steele > It's not clear to me what state this patch should be in. Is there more > review that needs to be done or is it ready for a committer? I would most appreciate it if Magnus could revie

Re: [HACKERS] Logical decoding on standby

2017-03-28 Thread Craig Ringer
On 29 March 2017 at 08:01, Craig Ringer wrote: > I just notice that I failed to remove the docs changes regarding > dropping slots becoming db-specific, so I'll post a follow-up for that > in a sec. Attached. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Developmen

Re: [HACKERS] Logical decoding on standby

2017-03-28 Thread Craig Ringer
On 28 March 2017 at 23:22, Andres Freund wrote: >> --- a/doc/src/sgml/protocol.sgml >> +++ b/doc/src/sgml/protocol.sgml >> @@ -2034,6 +2034,8 @@ The commands accepted in walsender mode are: >> >>Drops a replication slot, freeing any reserved server-side resources. >> If >>

Re: [HACKERS] logical replication launcher crash on buildfarm

2017-03-28 Thread Petr Jelinek
On 29/03/17 01:29, Petr Jelinek wrote: > On 28/03/17 18:05, Petr Jelinek wrote: >> On 28/03/17 17:55, Robert Haas wrote: >>> On Mon, Mar 27, 2017 at 11:20 PM, Petr Jelinek >>> wrote: On 28/03/17 04:46, Robert Haas wrote: > On Mon, Mar 27, 2017 at 10:04 PM, Andres Freund > wrote: >>>

Re: [HACKERS] logical replication launcher crash on buildfarm

2017-03-28 Thread Petr Jelinek
On 28/03/17 18:05, Petr Jelinek wrote: > On 28/03/17 17:55, Robert Haas wrote: >> On Mon, Mar 27, 2017 at 11:20 PM, Petr Jelinek >> wrote: >>> On 28/03/17 04:46, Robert Haas wrote: On Mon, Mar 27, 2017 at 10:04 PM, Andres Freund wrote: >> Btw now that I look at the code, I guess we'll wa

Re: [HACKERS] dsm.c API tweak

2017-03-28 Thread Alvaro Herrera
Thomas Munro wrote: > + if (CurrentResourceOwner) > + { > + seg->resowner = CurrentResourceOwner; > + ResourceOwnerRememberDSM(CurrentResourceOwner, seg); > + } > > You need to assign seg->resowner = CurrentResourceOwner > unconditionally here. Otherwise seg->resowner is uninit

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-28 Thread Alvaro Herrera
I pushed 0002 after some makeup, since it's just cosmetic and not controversial. Here's 0003 rebased on top of it. (Also, I took out the gin and gist changes: it would be wrong to change that unconditionally, because the 0x pattern appears in indexes that would be pg_upgraded. We need a diff

Re: [HACKERS] Removing binaries

2017-03-28 Thread Magnus Hagander
On Tue, Mar 28, 2017 at 6:44 PM, Robert Haas wrote: > On Tue, Mar 28, 2017 at 12:18 PM, Tom Lane wrote: > > Robert Haas writes: > >> On Tue, Mar 28, 2017 at 11:44 AM, Peter Eisentraut > >> wrote: > >>> On 3/21/17 08:12, Robert Haas wrote: > I think a big part of the usability problem here

Re: [HACKERS] [PATCH] Incremental sort

2017-03-28 Thread Alexander Korotkov
On Tue, Mar 28, 2017 at 5:27 PM, David Steele wrote: > Hi Alexander, > > On 3/20/17 10:19 AM, Heikki Linnakangas wrote: > >> On 03/20/2017 11:33 AM, Alexander Korotkov wrote: >> >>> Please, find rebased patch in the attachment. >>> >> >> I had a quick look at this. >> > > <...> > > According to '

Re: [HACKERS] Monitoring roles patch

2017-03-28 Thread Mark Dilger
> On Mar 28, 2017, at 1:17 PM, Tom Lane wrote: > > Mark Dilger writes: >> I don't see anything wrong with adding roles in pg_authid.h with a #define'd >> Oid. That's actually pretty helpful for anyone writing code against the >> database, >> as they don't have to look up the Oid of the role.

Re: [HACKERS] WIP: [[Parallel] Shared] Hash

2017-03-28 Thread Andres Freund
Hi, On 2017-03-27 22:33:03 -0700, Andres Freund wrote: > On 2017-03-23 20:35:09 +1300, Thomas Munro wrote: > > Here is a new patch series responding to feedback from Peter and Andres: > > Here's a review of 0007 & 0010 together - they're going to have to be > applied together anyway... > ... > ok

Re: [HACKERS] Monitoring roles patch

2017-03-28 Thread Tom Lane
Mark Dilger writes: > I don't see anything wrong with adding roles in pg_authid.h with a #define'd > Oid. That's actually pretty helpful for anyone writing code against the > database, > as they don't have to look up the Oid of the role. > But why not then grant privileges to that role in infor

Re: [HACKERS] [COMMITTERS] pgsql: Improve performance of find_all_inheritors()

2017-03-28 Thread Aleksander Alekseev
Hi, Robert. > I'm a little worried that this will be noticeably slower when the > number of partitions is small, like 4 or 8 or 16. In other places, > we've found it necessary to support both a list-based strategy and a > hash-based strategy to avoid regressing small cases (e.g. > join_rel_list +

Re: [HACKERS] Monitoring roles patch

2017-03-28 Thread Dave Page
Hi On Tue, Mar 28, 2017 at 2:22 PM, Stephen Frost wrote: > Dave, > > * Dave Page (dp...@pgadmin.org) wrote: >> OK, so before I start hacking again, here's a proposal based on my >> understanding of folks comments, and so open questions. If I can get >> agreement and answers, I'll be able to break

Re: [HACKERS] Monitoring roles patch

2017-03-28 Thread Mark Dilger
> On Mar 28, 2017, at 11:52 AM, Tom Lane wrote: > > Mark Dilger writes: >> After a bit of introspection, I think what is really bothering me is not the >> inability to revoke permissions, since as you say I can choose to not assign >> the role to anybody. What bothers me is that this feature i

Re: [HACKERS] Protection lost in expression eval changeover

2017-03-28 Thread Andres Freund
On 2017-03-28 15:24:28 -0400, Tom Lane wrote: > I wrote: > > Andres Freund writes: > >> On 2017-03-28 14:43:38 -0400, Tom Lane wrote: > >>> I don't see a strong reason why we need to allow a dropped column to go > >>> to null while we throw an immediate error for a change in column type. > >>> (If

Re: [HACKERS] Protection lost in expression eval changeover

2017-03-28 Thread Tom Lane
I wrote: > Andres Freund writes: >> On 2017-03-28 14:43:38 -0400, Tom Lane wrote: >>> I don't see a strong reason why we need to allow a dropped column to go >>> to null while we throw an immediate error for a change in column type. >>> (If there is some reason, hopefully beta testing will find it

Re: [HACKERS] PoC plpgsql - possibility to force custom or generic plan

2017-03-28 Thread Pavel Stehule
2017-03-28 20:29 GMT+02:00 Petr Jelinek : > On 28/03/17 19:43, Pavel Stehule wrote: > > Hi > > > > rebased due last changes in pg_exec.c > > > > Thanks, I went over this and worked over the documentation/comments a > bit (attached updated version of the patch with my changes). > > From my side thi

Re: [HACKERS] [PATCH] Move all am-related reloption code into src/backend/access/[am-name] and get rid of relopt_kind for custom AM

2017-03-28 Thread Nikolay Shaplov
В письме от 26 марта 2017 15:02:12 пользователь Alvaro Herrera написал: > Nikolay Shaplov wrote: > > If I would think about it now: we always know how many options we will > > have. So we can just pass this number to palloc and assert if somebody > > adds more options then expected... What do yo th

Re: [HACKERS] [patch] reorder tablespaces in basebackup tar stream for backup_label

2017-03-28 Thread Fujii Masao
On Tue, Feb 21, 2017 at 7:17 PM, Michael Banck wrote: > Hi, > > currently, the backup_label and (I think) the tablespace_map files are > (by design) conveniently located at the beginning of the main tablespace > tarball when making a basebackup. However, (by accident or also by > design?) the main

Re: [HACKERS] Protection lost in expression eval changeover

2017-03-28 Thread Tom Lane
Andres Freund writes: > On 2017-03-28 14:43:38 -0400, Tom Lane wrote: >> I don't see a strong reason why we need to allow a dropped column to go >> to null while we throw an immediate error for a change in column type. >> (If there is some reason, hopefully beta testing will find it.) > Ok. You'

Re: [HACKERS] Monitoring roles patch

2017-03-28 Thread Tom Lane
Mark Dilger writes: > After a bit of introspection, I think what is really bothering me is not the > inability to revoke permissions, since as you say I can choose to not assign > the role to anybody. What bothers me is that this feature implicitly > redefines > what is meant by the keyword PUBL

Re: [HACKERS] Protection lost in expression eval changeover

2017-03-28 Thread Andres Freund
On 2017-03-28 14:43:38 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2017-03-28 13:52:50 -0400, Tom Lane wrote: > >> So it seems like we are missing some needed protection. I'm inclined > >> to think that it'd be all right to just throw an error immediately in > >> CheckVarSlotCompatibili

Re: [HACKERS] Monitoring roles patch

2017-03-28 Thread Stephen Frost
Greetings, * Mark Dilger (hornschnor...@gmail.com) wrote: > After a bit of introspection, I think what is really bothering me is not the > inability to revoke permissions, since as you say I can choose to not assign > the role to anybody. What bothers me is that this feature implicitly > redefin

Re: [HACKERS] Protection lost in expression eval changeover

2017-03-28 Thread Tom Lane
Andres Freund writes: > On 2017-03-28 13:52:50 -0400, Tom Lane wrote: >> So it seems like we are missing some needed protection. I'm inclined >> to think that it'd be all right to just throw an error immediately in >> CheckVarSlotCompatibility if the target column is dropped. > Hm - so far we've

Re: [HACKERS] Getting server crash after running sqlsmith

2017-03-28 Thread Tom Lane
tushar writes: > After runinng sqlsmith against latest sources of PG v10 , able to see a > crash - Hm ... I don't see a crash here, but I wonder whether you have parameters set that would cause this query to be run as a parallel query? Because pg_rotate_logfile() is marked as parallel-safe in

Re: [HACKERS] Monitoring roles patch

2017-03-28 Thread Mark Dilger
> On Mar 28, 2017, at 11:06 AM, Mark Dilger wrote: > > >> On Mar 28, 2017, at 9:47 AM, Dave Page wrote: >> >>> Does >>> pg_read_all_stats still have access to stats for mysecuretable? >> >> Yes, because the ACL on the table controls reading/writing the data in >> the table. It doesn't have a

Re: [HACKERS] PoC plpgsql - possibility to force custom or generic plan

2017-03-28 Thread Petr Jelinek
On 28/03/17 19:43, Pavel Stehule wrote: > Hi > > rebased due last changes in pg_exec.c > Thanks, I went over this and worked over the documentation/comments a bit (attached updated version of the patch with my changes). >From my side this can go to committer. -- Petr Jelinek

Re: [HACKERS] Monitoring roles patch

2017-03-28 Thread Stephen Frost
Greetings, * Mark Dilger (hornschnor...@gmail.com) wrote: > The inability to revoke access to this sort of information being proposed > makes me a bit uneasy. What data are you concerned about, specifically? > Mostly, I think, I'm bothered because there may > be people who have revoked privile

Re: [HACKERS] Monitoring roles patch

2017-03-28 Thread Stephen Frost
Dave, * Dave Page (dp...@pgadmin.org) wrote: > OK, so before I start hacking again, here's a proposal based on my > understanding of folks comments, and so open questions. If I can get > agreement and answers, I'll be able to break out vi again without > (hopefully) too many more revisions: > > p

[HACKERS] Missing increment of vacrelstats->pinskipped_pages

2017-03-28 Thread Jim Nasby
lazy_vacuum_heap() does not count pages that it skips due to not obtaining the buffer cleanup lock. vacuum_pinskipped.patch fixes that. That should be backpatched to 9.5. vacuum_comment.patch cleans up a comment in lazy_scan_heap(). -- Jim C. Nasby, Data Architect j...@nas

Re: [HACKERS] Protection lost in expression eval changeover

2017-03-28 Thread Andres Freund
Hi, On 2017-03-28 13:52:50 -0400, Tom Lane wrote: > CheckVarSlotCompatibility contains the comment > >* Note: we allow a reference to a dropped attribute. slot_getattr will >* force a NULL result in such cases. > > While still true, that second sentence is now quite irrelevant,

Re: [HACKERS] pg_stat_wal_write statistics view

2017-03-28 Thread Fujii Masao
On Tue, Mar 28, 2017 at 1:40 PM, Haribabu Kommi wrote: > > > On Mon, Mar 27, 2017 at 1:27 PM, Haribabu Kommi > wrote: >> >> >> >> On Sat, Mar 25, 2017 at 6:40 AM, Fujii Masao >> wrote: >>> >>> On Wed, Feb 15, 2017 at 12:53 PM, Haribabu Kommi >>> wrote: >>> > >>> > >>> > On Wed, Feb 8, 2017 at 9

Re: [HACKERS] Monitoring roles patch

2017-03-28 Thread Mark Dilger
> On Mar 28, 2017, at 9:47 AM, Dave Page wrote: > >> Does >> pg_read_all_stats still have access to stats for mysecuretable? > > Yes, because the ACL on the table controls reading/writing the data in > the table. It doesn't have any bearing on any kind of table metadata. > A user who has no pri

Re: [HACKERS] Monitoring roles patch

2017-03-28 Thread Dave Page
On Tue, Mar 28, 2017 at 1:52 PM, Mark Dilger wrote: > >> On Mar 28, 2017, at 9:55 AM, Robert Haas wrote: >> >> On Tue, Mar 28, 2017 at 12:47 PM, Dave Page wrote: I don't see any precedent in the code for having a hardcoded role, other than superuser, and allowing privileges based

Re: [HACKERS] Supporting huge pages on Windows

2017-03-28 Thread David Steele
On 3/24/17 12:51 PM, Ashutosh Sharma wrote: Hi, On Fri, Mar 24, 2017 at 9:00 PM, David Steele wrote: Hi Ashutosh, On 3/22/17 8:52 AM, Amit Kapila wrote: On Wed, Mar 22, 2017 at 12:07 AM, David Steele wrote: Amit, Magnus, you are signed up as reviewers for this patch. Do you know when y

[HACKERS] Protection lost in expression eval changeover

2017-03-28 Thread Tom Lane
CheckVarSlotCompatibility contains the comment * Note: we allow a reference to a dropped attribute. slot_getattr will * force a NULL result in such cases. While still true, that second sentence is now quite irrelevant, because we don't go through slot_getattr anymore. So it se

Re: [HACKERS] Monitoring roles patch

2017-03-28 Thread Mark Dilger
> On Mar 28, 2017, at 9:55 AM, Robert Haas wrote: > > On Tue, Mar 28, 2017 at 12:47 PM, Dave Page wrote: >>> I don't see any precedent in the code for having a hardcoded role, other >>> than >>> superuser, and allowing privileges based on a hardcoded test for membership >>> in that role. I'm

Re: [HACKERS] Monitoring roles patch

2017-03-28 Thread Dave Page
OK, so before I start hacking again, here's a proposal based on my understanding of folks comments, and so open questions. If I can get agreement and answers, I'll be able to break out vi again without (hopefully) too many more revisions: pg_read_all_stats: Will have C-coded access to pg_stats vie

Re: [HACKERS] PoC plpgsql - possibility to force custom or generic plan

2017-03-28 Thread Pavel Stehule
Hi rebased due last changes in pg_exec.c Regards Pavel diff --git a/doc/src/sgml/plpgsql.sgml b/doc/src/sgml/plpgsql.sgml index d356deb9f5..56da4d6163 100644 --- a/doc/src/sgml/plpgsql.sgml +++ b/doc/src/sgml/plpgsql.sgml @@ -802,6 +802,32 @@ $$ LANGUAGE plpgsql; happen in a plain SQL comma

Re: [HACKERS] Removing binaries

2017-03-28 Thread Tom Lane
Robert Haas writes: > On Tue, Mar 28, 2017 at 12:18 PM, Tom Lane wrote: >> Perhaps we could satisfy novices by changing the out-of-the-box >> behavior, but provide some way to select the old behavior for >> installations that are really depending on it. > Hmm. I guess that would mean that the s

Re: [HACKERS] Monitoring roles patch

2017-03-28 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Tue, Mar 28, 2017 at 12:47 PM, Dave Page wrote: > >> I don't see any precedent in the code for having a hardcoded role, other > >> than > >> superuser, and allowing privileges based on a hardcoded test for membership > >> in that role.

[HACKERS] GSoC 2017 Proposal for "Explicitly support predicate locks in index access methods besides btree"

2017-03-28 Thread Shubham Barai
Hi guys, My name is Shubham Barai and I am a final year student at Maharashtra Institute of Technology, Pune, India. I am very interested in contributing Postgresql this year through GSoC project. I am particularly interested in working on the project "Explicitly support predicate locks in index

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-28 Thread Pavan Deolasee
On Tue, Mar 28, 2017 at 8:34 PM, Robert Haas wrote: > On Mon, Mar 27, 2017 at 10:25 PM, Pavan Deolasee > wrote: > > On Tue, Mar 28, 2017 at 1:59 AM, Robert Haas > wrote: > >> On Thu, Mar 23, 2017 at 2:47 PM, Pavan Deolasee > >> wrote: > >> > It's quite hard to say that until we see many more b

Re: [HACKERS] [COMMITTERS] pgsql: Allow vacuums to report oldestxmin

2017-03-28 Thread Alvaro Herrera
Fujii Masao wrote: > This is the evidence that no one cares about the details of VACUUM VERBOSE > output example. So I'm tempted to simplify the example (please see the > attached patch) instead of keeping updating the example. Agreed. > > If this patch is applied, it should back-patch to all su

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-28 Thread Pavan Deolasee
On Tue, Mar 28, 2017 at 7:04 PM, Amit Kapila wrote: > > > For such an heap insert, we will pass > the actual value of column to index_form_tuple during index insert. > However during recheck when we fetch the value of c2 from heap tuple > and pass it index tuple, the value is already in compres

Re: [HACKERS] Monitoring roles patch

2017-03-28 Thread Dave Page
On Tue, Mar 28, 2017 at 12:55 PM, Robert Haas wrote: > On Tue, Mar 28, 2017 at 12:47 PM, Dave Page wrote: >>> I don't see any precedent in the code for having a hardcoded role, other >>> than >>> superuser, and allowing privileges based on a hardcoded test for membership >>> in that role. I'm s

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-28 Thread Pavan Deolasee
On Tue, Mar 28, 2017 at 4:07 PM, Amit Kapila wrote: > > Noted few cosmetic issues in 0005_warm_updates_v21: > > 1. > pruneheap.c(939): warning C4098: 'heap_get_root_tuples' : 'void' > function returning a value > Thanks. Will fix. > > 2. > + * HCWC_WARM_UPDATED_TUPLE - a tuple with HEAP_WARM_

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-28 Thread Pavan Deolasee
On Tue, Mar 28, 2017 at 4:05 PM, Amit Kapila wrote: > > > As asked previously, can you explain me on what basis are you > considering it robust? The comments on top of datumIsEqual() clearly > indicates the danger of using it for toasted values (Also, it will > probably not give the answer you w

Re: [HACKERS] [PATCH] Move all am-related reloption code into src/backend/access/[am-name] and get rid of relopt_kind for custom AM

2017-03-28 Thread Alvaro Herrera
Nikolay Shaplov wrote: > В письме от 17 марта 2017 14:21:26 пользователь Alvaro Herrera написал: > > > Please make sure to mark functions as static (e.g. bringetreloptcatalog). > I am a bit confused here: > > For brin and nbtree this is a good idea: brin.c and nbtree.c has AM-handler > inside it

Re: [HACKERS] Monitoring roles patch

2017-03-28 Thread Robert Haas
On Tue, Mar 28, 2017 at 12:47 PM, Dave Page wrote: >> I don't see any precedent in the code for having a hardcoded role, other than >> superuser, and allowing privileges based on a hardcoded test for membership >> in that role. I'm struggling to think of all the security implications of >> that.

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-03-28 Thread Robert Haas
On Mon, Mar 27, 2017 at 8:36 AM, Ashutosh Bapat wrote: > I have gone through the patch, and it looks good to me. Here's the set > of patches with this patch included. Fixed the testcase failures. > Rebased the patchset on de4da168d57de812bb30d359394b7913635d21a9. This version of 0001 looks much b

Re: [HACKERS] Monitoring roles patch

2017-03-28 Thread Dave Page
On Tue, Mar 28, 2017 at 12:04 PM, Mark Dilger wrote: > >> On Mar 28, 2017, at 8:34 AM, Dave Page wrote: >> >> On Tue, Mar 28, 2017 at 11:31 AM, Peter Eisentraut >> wrote: >>> This patch touches the pg_buffercache and pg_freespacemap extensions, >>> but there appear to be some files missing. >> >

Re: [HACKERS] Removing binaries

2017-03-28 Thread Robert Haas
On Tue, Mar 28, 2017 at 12:18 PM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Mar 28, 2017 at 11:44 AM, Peter Eisentraut >> wrote: >>> On 3/21/17 08:12, Robert Haas wrote: I think a big part of the usability problem here comes from the fact that the default database for connection

Re: [HACKERS] [COMMITTERS] pgsql: Allow vacuums to report oldestxmin

2017-03-28 Thread Fujii Masao
On Wed, Mar 29, 2017 at 1:32 AM, Fujii Masao wrote: > On Tue, Mar 28, 2017 at 1:06 AM, Masahiko Sawada > wrote: >> On Sun, Mar 26, 2017 at 2:26 AM, Masahiko Sawada >> wrote: >>> On Sun, Mar 26, 2017 at 1:37 AM, Fujii Masao wrote: On Mon, Mar 6, 2017 at 9:37 PM, Masahiko Sawada wro

Re: [HACKERS] [COMMITTERS] pgsql: Allow vacuums to report oldestxmin

2017-03-28 Thread Fujii Masao
On Tue, Mar 28, 2017 at 1:06 AM, Masahiko Sawada wrote: > On Sun, Mar 26, 2017 at 2:26 AM, Masahiko Sawada > wrote: >> On Sun, Mar 26, 2017 at 1:37 AM, Fujii Masao wrote: >>> On Mon, Mar 6, 2017 at 9:37 PM, Masahiko Sawada >>> wrote: On Fri, Mar 3, 2017 at 10:50 PM, Simon Riggs wrote: >

  1   2   >