Re: partition routing layering in nodeModifyTable.c

2019-07-30 Thread Amit Langote
Hi Andres, Sorry about the delay in replying as I was on vacation for the last few days. On Sat, Jul 20, 2019 at 1:52 AM Andres Freund wrote: > > The first one (0001) deals with reducing the core executor's reliance > > on es_result_relation_info to access the currently active result > > relatio

Re: CVE-2017-7484-induced bugs, or, btree cmp functions are not leakproof?

2019-07-30 Thread Amit Langote
On Mon, Jul 29, 2019 at 6:59 PM Dilip Kumar wrote: > On Wed, Jul 17, 2019 at 2:39 PM Amit Langote wrote: > > I've combined the two patches into one. > Looks fine to me, moved to ready for committer. Thank you Dilip. Regards, Amit

Re: Parallel grouping sets

2019-07-30 Thread Richard Guo
On Wed, Jun 12, 2019 at 10:58 AM Richard Guo wrote: > Hi all, > > Paul and I have been hacking recently to implement parallel grouping > sets, and here we have two implementations. > > Implementation 1 > > > Attached is the patch and also there is a github branch [1] for this > w

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2019-07-30 Thread Etsuro Fujita
On Fri, Jul 19, 2019 at 8:09 PM amul sul wrote: > On Mon, Jul 8, 2019 at 5:03 PM Etsuro Fujita wrote: >> I started reviewing this. Here is my initial review comments: >> >> * 0001-Hash-partition-bound-equality-refactoring-v22.patch >> However, I don't think it's a good idea to do this >> refact

Re: POC: Cleaning up orphaned files using undo logs

2019-07-30 Thread Andres Freund
Hi, Amit, short note: The patches aren't attached in patch order. Obviously a miniscule thing, but still nicer if that's not the case. Dilip, this also contains the start of a review for the undo record interface further down. On 2019-07-29 16:35:20 -0700, Andres Freund wrote: > Here we go.

Re: Possible race condition in pg_mkdir_p()?

2019-07-30 Thread Michael Paquier
On Tue, Jul 23, 2019 at 02:54:20PM +0800, Ning Yu wrote: > MakePGDirectory() is also called in TablespaceCreateDbspace(), EEXIST is > considered as non-error for parent directories, however as it considers > EEXIST as a failure for the last level of the path so the logic is > still correct, So th

Re: POC: Cleaning up orphaned files using undo logs

2019-07-30 Thread Dilip Kumar
On Tue, Jul 30, 2019 at 12:21 PM Thomas Munro wrote: > > On Tue, Jul 30, 2019 at 5:03 PM Dilip Kumar wrote: > > > > I think this idea is good for the DO time but during REDO time it will > > not work as we will not have the transaction state. Having said that > > the current idea of keeping in t

Re: concerns around pg_lsn

2019-07-30 Thread Jeevan Ladhe
Hi Michael, Thanks for your inputs, really appreciate. On Tue, Jul 30, 2019 at 9:42 AM Michael Paquier wrote: > On Mon, Jul 29, 2019 at 10:55:29PM +0530, Jeevan Ladhe wrote: > > I am attaching a patch that makes sure that *have_error is set to false > in > > pg_lsn_in_internal() itself, rather

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2019-07-30 Thread Etsuro Fujita
On Fri, Jul 19, 2019 at 10:44 PM Robert Haas wrote: > On Thu, Jul 18, 2019 at 2:55 AM Etsuro Fujita wrote: > > I.e., partition_bounds_merge() is performed for each pair of input > > partitioned relations for a join relation in try_partitionwise_join(). > > Since partition_bounds_merge() would nee

Re: Built-in connection pooler

2019-07-30 Thread Konstantin Knizhnik
On 30.07.2019 4:02, Tomas Vondra wrote: My idea (sorry if it wasn't too clear) was that we might handle some cases more gracefully. For example, if we only switch between transactions, we don't quite care about 'SET LOCAL' (but the current patch does set the tainted flag). The same thing appl

Re: query1 followed by query2 at maximum distance vs current fixed distance

2019-07-30 Thread Wh isere
Thanks Arthur! I guess there is not other solution? I tried to create a function to loop through all the distance but its very slow. On Tuesday, July 30, 2019, Arthur Zakirov wrote: > Hello, > > On 23.07.2019 09:55, Wh isere wrote: > >> Is this possible with the current websearch_to_tsquery fun

Re: Possible race condition in pg_mkdir_p()?

2019-07-30 Thread Ning Yu
On Tue, Jul 30, 2019 at 4:04 PM Michael Paquier wrote: > > On Tue, Jul 23, 2019 at 02:54:20PM +0800, Ning Yu wrote: > > MakePGDirectory() is also called in TablespaceCreateDbspace(), EEXIST is > > considered as non-error for parent directories, however as it considers > > EEXIST as a failure for t

Re: Built-in connection pooler

2019-07-30 Thread Konstantin Knizhnik
On 26.07.2019 23:24, Tomas Vondra wrote: Secondly, when trying this  pgbench -p 5432 -U x -i -s 1 test  pgbench -p 6543 -U x -c 24 -C -T 10 test it very quickly locks up, with plenty of non-granted locks in pg_locks, but I don't see any interventions by deadlock detector so I presume the issu

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-30 Thread Sehrope Sarkuni
On Mon, Jul 29, 2019 at 8:35 PM Bruce Momjian wrote: > On Sun, Jul 28, 2019 at 10:33:03PM -0400, Bruce Momjian wrote: > > I am thinking of writing some Assert() code that checks that all buffers > > using a single LSN are from the same relation (and therefore different > > page numbers). I would

Re: POC: Cleaning up orphaned files using undo logs

2019-07-30 Thread Thomas Munro
Hi Amit I've been testing some undo worker workloads (more on that soon), but here's a small thing: I managed to reach an LWLock self-deadlock in the undo worker launcher: diff --git a/src/backend/access/undo/undorequest.c b/src/backend/access/undo/undorequest.c ... +bool +UndoGetWork(bool allow_

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-30 Thread Masahiko Sawada
On Mon, Jul 29, 2019 at 8:18 PM Sehrope Sarkuni wrote: > > On Mon, Jul 29, 2019 at 6:42 AM Masahiko Sawada wrote: > > > An argument could be made to push that problem upstream, i.e. let the > > > supplier of the passphrase deal with the indirection. You would still > > > need to verify the suppli

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-30 Thread Bruce Momjian
On Tue, Jul 30, 2019 at 07:44:20AM -0400, Sehrope Sarkuni wrote: > On Mon, Jul 29, 2019 at 8:35 PM Bruce Momjian wrote: > From the patch: > > /* > ! * The initialization vector (IV) is used for page-level > ! * encryption.  We use the LSN and page number as the IV, and IV > ! * values must never

Re: [HACKERS] Restricting maximum keep segments by repslots

2019-07-30 Thread Kyotaro Horiguchi
Thanks for reviewing! At Thu, 27 Jun 2019 16:22:56 +0200, Jehan-Guillaume de Rorthais wrote in <20190627162256.4f4872b8@firost> > Hi all, > > Being interested by this feature, I did a patch review. > > This features adds the GUC "max_slot_wal_keep_size". This is the maximum > amount > of WAL

Re: [HACKERS] Restricting maximum keep segments by repslots

2019-07-30 Thread Kyotaro Horiguchi
At Tue, 30 Jul 2019 21:30:45 +0900 (Tokyo Standard Time), Kyotaro Horiguchi wrote in <20190730.213045.221405075.horikyota@gmail.com> > I attach the revised patch. I'll repost the polished version > sooner. (Mainly TAP test and documentation, code comments will be revised) regards. -- Kyo

Re: concerns around pg_lsn

2019-07-30 Thread Robert Haas
On Tue, Jul 30, 2019 at 4:52 AM Jeevan Ladhe wrote: > My only concern was something that we internally treat as invalid, why do > we allow, that as a valid value for that type. While I am not trying to > reinvent the wheel here, I am trying to understand if there had been any > idea behind this an

Re: SQL:2011 PERIODS vs Postgres Ranges?

2019-07-30 Thread Ibrar Ahmed
Hi Paul, I have rebased the patch to master (1e2fddfa33d3c7cc93ca3ee0f32852699bd3e012) and fixed some compilation warning. Now I am reviewing the actual code. On Fri, Jul 26, 2019 at 6:35 PM Ibrar Ahmed wrote: > The patch requires to rebase on the master branch. > > The new status of this patc

Re: Built-in connection pooler

2019-07-30 Thread Tomas Vondra
On Tue, Jul 30, 2019 at 01:01:48PM +0300, Konstantin Knizhnik wrote: On 30.07.2019 4:02, Tomas Vondra wrote: My idea (sorry if it wasn't too clear) was that we might handle some cases more gracefully. For example, if we only switch between transactions, we don't quite care about 'SET LOCAL'

Re: [proposal] de-TOAST'ing using a iterator

2019-07-30 Thread Binguo Bao
John Naylor 于2019年7月29日周一 上午11:49写道: > On Thu, Jul 25, 2019 at 10:21 PM Binguo Bao wrote: > My goal for this stage of review was to understand more fully what the > code is doing, and make it as simple and clear as possible, starting > at the top level. In doing so, it looks like I found some ad

Re: block-level incremental backup

2019-07-30 Thread Ibrar Ahmed
On Tue, Jul 30, 2019 at 1:28 AM Robert Haas wrote: > On Wed, Jul 10, 2019 at 2:17 PM Anastasia Lubennikova > wrote: > > In attachments, you can find a prototype of incremental pg_basebackup, > > which consists of 2 features: > > > > 1) To perform incremental backup one should call pg_basebackup

Re: tap tests driving the database via psql

2019-07-30 Thread Tom Lane
Craig Ringer writes: > On Sun, 28 Jul 2019 at 03:15, Andres Freund wrote: >> 1) Just depend on DBD::Pg being installed. It's fairly common, after >> all. It'd be somewhat annoying that we'd often end up using a >> different version of libpq than what we're testing against. But in >> most cases th

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-30 Thread Sehrope Sarkuni
On Tue, Jul 30, 2019 at 8:16 AM Masahiko Sawada wrote: > On Mon, Jul 29, 2019 at 8:18 PM Sehrope Sarkuni > wrote: > > > > On Mon, Jul 29, 2019 at 6:42 AM Masahiko Sawada > wrote: > > > > An argument could be made to push that problem upstream, i.e. let the > > > > supplier of the passphrase dea

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-30 Thread Masahiko Sawada
On Tue, Jul 30, 2019 at 5:03 AM Sehrope Sarkuni wrote: > > On Mon, Jul 29, 2019 at 9:44 AM Bruce Momjian wrote: >> >> > Checking that all buffers using a single LSN are from the same >> > relation would be a good idea but I think it's hard to test it and >> > regard the test result as okay. Even

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-30 Thread Sehrope Sarkuni
On Tue, Jul 30, 2019 at 8:16 AM Bruce Momjian wrote: > On Tue, Jul 30, 2019 at 07:44:20AM -0400, Sehrope Sarkuni wrote: > > If each relation file has its own derived key, the derived TDEK for that > > relation file, then there is no issue with reusing the same IV = LSN || > Page > > Number. The T

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-30 Thread Sehrope Sarkuni
On Tue, Jul 30, 2019 at 10:06 AM Masahiko Sawada wrote: > On Tue, Jul 30, 2019 at 5:03 AM Sehrope Sarkuni > wrote: > > > > On Mon, Jul 29, 2019 at 9:44 AM Bruce Momjian wrote: > >> > >> > Checking that all buffers using a single LSN are from the same > >> > relation would be a good idea but I t

Re: TopoSort() fix

2019-07-30 Thread Tom Lane
Robert Haas writes: > On Mon, Jul 29, 2019 at 9:48 PM Tom Lane wrote: >> I believe the only accessible route to taking any sort of new lock >> in a parallel worker is catalog lookups causing AccessShareLock on >> a catalog. > Can't the worker just query a previously-untouched table, maybe by > c

Re: tap tests driving the database via psql

2019-07-30 Thread Andres Freund
Hi, On 2019-07-30 09:40:54 -0400, Tom Lane wrote: > Craig Ringer writes: > > On Sun, 28 Jul 2019 at 03:15, Andres Freund wrote: > >> 1) Just depend on DBD::Pg being installed. It's fairly common, after > >> all. It'd be somewhat annoying that we'd often end up using a > >> different version of l

Re: tap tests driving the database via psql

2019-07-30 Thread Tom Lane
Andres Freund writes: > On 2019-07-30 09:40:54 -0400, Tom Lane wrote: >> Now, none of these things are really a problem with DBD/DBI as such >> --- rather, they are reasons not to depend on a pre-packaged build >> of DBD::Pg that depends on a pre-packaged build of libpq.so. >> I haven't looked at

Re: Parallel grouping sets

2019-07-30 Thread Tomas Vondra
On Tue, Jul 30, 2019 at 03:50:32PM +0800, Richard Guo wrote: On Wed, Jun 12, 2019 at 10:58 AM Richard Guo wrote: Hi all, Paul and I have been hacking recently to implement parallel grouping sets, and here we have two implementations. Implementation 1 Attached is the patch a

Re: pg_upgrade version checking questions

2019-07-30 Thread Daniel Gustafsson
> On 27 Jul 2019, at 08:42, Peter Eisentraut > wrote: > I have committed 0002, 0003, and 0004. Thanks! > The implementation in 0001 (Only allow upgrades by the same exact > version new bindir) has a problem. It compares (new_cluster.bin_version > != PG_VERSION_NUM), but new_cluster.bin_versio

Re: Patch to document base64 encoding

2019-07-30 Thread Tom Lane
"Karl O. Pinc" writes: > On Mon, 15 Jul 2019 23:00:55 +0200 (CEST) > Fabien COELHO wrote: >> The patch clarifies the documentation about encode/decode and other >> text/binary string conversion functions. > Other notable changes: > Corrects categorization of functions as string or binary. >

Unused struct member in pgcrypto pgp.c

2019-07-30 Thread Daniel Gustafsson
Hi, In contrib/pgcrypto/pgp.c we have a struct member int_name in digest_info which isn’t used, and seems to have never been used (a potential copy/pasteo from the cipher_info struct?). Is there a reason for keeping this, or can it be removed as per the attached? cheers ./daniel pgcrypto_int_

Re: Proposal to suppress errors thrown by to_reg*()

2019-07-30 Thread Tom Lane
Takuma Hoshiai writes: > [ fix_to_reg_v2.patch ] I took a quick look through this patch. I'm on board with the goal of not having schema-access violations throw an error in these functions, but the implementation feels pretty ugly and bolted-on. Nobody who had designed the code to do this from t

Re: TopoSort() fix

2019-07-30 Thread Robert Haas
On Tue, Jul 30, 2019 at 10:27 AM Tom Lane wrote: > I also looked into whether one could use SELECT FOR UPDATE/SHARE to get > stronger locks at a tuple level, but that's been blocked off as well. > You guys really did a pretty good job of locking that down. Thanks. We learned from the master. >

Re: make installcheck-world in a clean environment

2019-07-30 Thread Tom Lane
Alexander Lakhin writes: > 01.07.2019 13:47, Thomas Munro wrote: >> A new CF is here and this is in "Needs Review". Would you like to >> provide a rebased patch, or should it really be withdrawn? > The rebased patch is attached, but I still can't find anyone interested > in reviewing it. > So le

Re: Avoid full GIN index scan when possible

2019-07-30 Thread Tom Lane
Marc Cousin writes: > By the way, while preparing this, I noticed that it seems that during this > kind of index scan, the interrupt signal is masked > for a very long time. Control-C takes a very long while to cancel the query. > But it's an entirely different problem :) Yeah, that seems like

Re: TopoSort() fix

2019-07-30 Thread Tom Lane
Robert Haas writes: > On Tue, Jul 30, 2019 at 10:27 AM Tom Lane wrote: >> (BTW, why aren't these functions just "parallel restricted"?) > ... > But it is really pretty arguable whether we should feel responsible > for that problem. We could just decide that if you're doing that, and > you don't

Re: Attached partition not considering altered column properties of root partition.

2019-07-30 Thread Robert Haas
On Tue, Jul 2, 2019 at 9:53 PM Amit Langote wrote: > Thanks for the report. This seems like a bug. Documentation claims > that the child tables inherit column storage options from the parent > table. That's actually enforced in only some cases. I realize I'm just repeating the same argument I'

Re: TopoSort() fix

2019-07-30 Thread Robert Haas
On Tue, Jul 30, 2019 at 1:36 PM Tom Lane wrote: > No, there's a sufficient reason why we should force advisory locks > to be taken in the leader process, namely that the behavior is totally > different if we don't: they will disappear at the end of the parallel > worker run, not at end of transact

Re: TopoSort() fix

2019-07-30 Thread Tom Lane
Robert Haas writes: > On Tue, Jul 30, 2019 at 1:36 PM Tom Lane wrote: >> In any case, my question at the moment is whether we need the belt-and- >> suspenders-too approach of having both non-parallel-safe marking and an >> explicit check inside these functions. We've largely moved away from >> h

Re: TopoSort() fix

2019-07-30 Thread Robert Haas
On Tue, Jul 30, 2019 at 1:44 PM Tom Lane wrote: > Well, there'd be an actual isolation test that they work ;-), if you > override the marking. Admittedly, one test case does not prove that > there's no way to crash the system, but that can be said of most > parts of Postgres. True. I'm just tal

Re: TopoSort() fix

2019-07-30 Thread Tom Lane
Robert Haas writes: > On Tue, Jul 30, 2019 at 1:44 PM Tom Lane wrote: >> Well, there'd be an actual isolation test that they work ;-), if you >> override the marking. Admittedly, one test case does not prove that >> there's no way to crash the system, but that can be said of most >> parts of Pos

Re: ANALYZE: ERROR: tuple already updated by self

2019-07-30 Thread Tomas Vondra
On Mon, Jul 29, 2019 at 12:18:33PM +0200, Tomas Vondra wrote: On Sun, Jul 28, 2019 at 09:53:20PM -0700, Andres Freund wrote: Hi, On 2019-07-28 21:21:51 +0200, Tomas Vondra wrote: AFAICS it applies to 10+ versions, because that's where extended stats were introduced. We certainly can't mess wit

typesafe printf hackery

2019-07-30 Thread Andres Freund
Hi, I've occasionally wished for a typesafe version of pg_printf() and other varargs functions. The compiler warnings are nice, but also far from complete. Here's a somewhat crazy hack/prototype for how printf could get actual argument types. I'm far from certain it's worth pursuing this further

Re: Patch to document base64 encoding

2019-07-30 Thread Karl O. Pinc
On Tue, 30 Jul 2019 11:40:03 -0400 Tom Lane wrote: > "Karl O. Pinc" writes: > > On Mon, 15 Jul 2019 23:00:55 +0200 (CEST) > > Fabien COELHO wrote: > >> The patch clarifies the documentation about encode/decode and > >> other text/binary string conversion functions. > > > Other notable chan

Re: SegFault on 9.6.14

2019-07-30 Thread Robert Haas
On Thu, Jul 18, 2019 at 9:45 AM Tom Lane wrote: > I think this is going in the wrong direction. Nodes should *always* > assume that a rescan is possible until ExecEndNode is called. > If you want to do otherwise, you are going to be inventing a whole > bunch of complicated and doubtless-initially

Re: tap tests driving the database via psql

2019-07-30 Thread Alvaro Herrera
On 2019-Jul-30, Tom Lane wrote: > OK, so just lifting DBD::Pg in toto is out for license reasons. > However, maybe we could consider writing a new DBD driver from > scratch (while using a platform-provided DBI layer) rather than > doing everything from scratch. I'm not sure how much actual > func

Re: Avoiding hash join batch explosions with extreme skew and weird stats

2019-07-30 Thread Melanie Plageman
So, I've rewritten the patch to use a BufFile for the outer table batch file tuples' match statuses and write bytes to and from the file which start as 0 and, upon encountering a match for a tuple, I set its bit in the file to 1 (also rebased with current master). It, of course, only works for par

Re: Initdb failure

2019-07-30 Thread Robert Haas
On Sat, Jul 27, 2019 at 2:22 AM Peter Eisentraut wrote: > I think if you want to make this more robust, get rid of the fixed-size > array, use dynamic allocation with PQExpBuffer, and let the operating > system complain if it doesn't like the directory name length. Agreed, but I think we should j

Redacting information from logs

2019-07-30 Thread Jeff Davis
Logs are important to diagnose problems or monitor operations, but logs can contain sensitive information which is often unnecessary for these purposes. Redacting the sensitive information would enable easier access and simpler integration with analysis tools without compromising the sensitive info

Re: Allow table AM's to cache stuff in relcache

2019-07-30 Thread Heikki Linnakangas
On 12/07/2019 16:07, Julien Rouhaud wrote: On Fri, Jun 14, 2019 at 5:40 PM Tom Lane wrote: Heikki Linnakangas writes: In the patch, I documented that rd_amcache must be allocated in CacheMemoryContext, or in rd_indexcxt if it's an index. It works, but it's a bit weird. Given the way the pat

Re: Initdb failure

2019-07-30 Thread Tom Lane
Robert Haas writes: > Agreed, but I think we should just do nothing. To actually fix this > in general, we'd have to get rid of every instance of MAXPGPATH in the > source tree: > [rhaas pgsql]$ git grep MAXPGPATH | wc -l > 611 I don't think it'd really be necessary to go that far. One of

Re: idea: log_statement_sample_rate - bottom limit for sampling

2019-07-30 Thread Tom Lane
Tomas Vondra writes: > I've started reviewing this patch, thinking that maybe I could get it > committed as it's marked as RFC. In general I agree with having this > fuature, but I think we need to rethink the GUC because the current > approach is just confusing. > ... > What I think we should do

Re: extension patch of CREATE OR REPLACE TRIGGER

2019-07-30 Thread Tom Lane
"osumi.takami...@fujitsu.com" writes: > [ CREATE_OR_REPLACE_TRIGGER_v03.patch ] I took a quick look through this just to see what was going on. A few comments: * Upthread you asked about changing the lock level to be AccessExclusiveLock if the trigger already exists, but the patch doesn't actual

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-07-30 Thread Bruce Momjian
On Tue, Jul 30, 2019 at 10:14:14AM -0400, Sehrope Sarkuni wrote: > > In general it's fine to use the same IV with different keys. Only reuse > of Key > > + IV is a problem and the entire set of possible counter values (IV + 0, > IV + > > 1, ...) generated with a key must be uniq

Re: heapam_index_build_range_scan's anyvisible

2019-07-30 Thread Ashwin Agrawal
On Tue, Jul 16, 2019 at 10:22 AM Andres Freund wrote: > Hi, > > On 2019-07-11 17:27:46 -0700, Ashwin Agrawal wrote: > > Please find attached the patch to remove IndexBuildCallback's dependency > on > > HeapTuple, as discussed. Changed to have the argument as ItemPointer > > instead of HeapTuple.

Re: idea: log_statement_sample_rate - bottom limit for sampling

2019-07-30 Thread Tomas Vondra
On Tue, Jul 30, 2019 at 03:43:58PM -0400, Tom Lane wrote: Tomas Vondra writes: I've started reviewing this patch, thinking that maybe I could get it committed as it's marked as RFC. In general I agree with having this fuature, but I think we need to rethink the GUC because the current approach

[Patch] Adding CORRESPONDING/CORRESPONDING BY to set operation

2019-07-30 Thread 毛瑞嘉
Hi, I wrote a patch for adding CORRESPONDING/CORRESPONDING BY to set operation. It is a task in the todo list. This is how the patch works: I modified transformSetOperationStmt() to get an intersection target list which is the intersection of the target lists of the left clause and right clause

Re: Patch to document base64 encoding

2019-07-30 Thread Fabien COELHO
My 0.02 € It seems entirely crazy that encode() and decode() are no longer in the same section, likewise that convert_from() and convert_to() aren't documented together anymore. Awkward, yes. But findable if you know what the categories are. I suppose there could be 3 different categories:

Re: [Patch] Adding CORRESPONDING/CORRESPONDING BY to set operation

2019-07-30 Thread David Fetter
On Tue, Jul 30, 2019 at 02:43:05PM -0700, 毛瑞嘉 wrote: > Hi, > > > I wrote a patch for adding CORRESPONDING/CORRESPONDING BY to set operation. > It is a task in the todo list. This is how the patch works: > > I modified transformSetOperationStmt() to get an intersection target list > which is the

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

2019-07-30 Thread Thomas Munro
On Tue, Jun 25, 2019 at 6:02 AM Andres Freund wrote: > On 2019-06-24 10:41:06 -0700, Ashwin Agrawal wrote: > > Proposing following changes to make predicate locking and checking > > functions generic and remove dependency on HeapTuple and Heap AM. We > > made these changes to help with Zedstore. I

Re: pgbench - implement strict TPC-B benchmark

2019-07-30 Thread Tom Lane
Fabien COELHO writes: > [ pgbench-strict-tpcb-2.patch ] TBH, I think we should reject this patch. Nobody cares about TPC-B anymore, and they care even less about differences between one sort-of-TPC-B test and another sort-of-TPC-B test. (As the lack of response on this thread shows.) We don't

AW: Adding column "mem_usage" to view pg_prepared_statements

2019-07-30 Thread Daniel Migowski
Hello, Will my patch be considered for 12.0? The calculation of the mem_usage value might be improved later on but because the system catalog is changed I would love to add it before 12.0 becomes stable. Regards, Daniel Migowski -Ursprüngliche Nachricht- Von: Daniel Migowski Gesende

Re: AW: Adding column "mem_usage" to view pg_prepared_statements

2019-07-30 Thread Tom Lane
Daniel Migowski writes: > Will my patch be considered for 12.0? The calculation of the mem_usage value > might be improved later on but because the system catalog is changed I would > love to add it before 12.0 becomes stable. v12 has been feature-frozen for months, and it's pretty hard to pai

AW: AW: Adding column "mem_usage" to view pg_prepared_statements

2019-07-30 Thread Daniel Migowski
Ok, just have read about the commitfest thing. Is the patch OK for that? Or is there generally no love for a mem_usage column here? If it was, I really would add some memory monitoring in our app regarding this. -Ursprüngliche Nachricht- Von: Tom Lane Gesendet: Mittwoch, 31. Juli 2019

Re: Adding column "mem_usage" to view pg_prepared_statements

2019-07-30 Thread Tomas Vondra
On Tue, Jul 30, 2019 at 10:01:09PM +, Daniel Migowski wrote: Hello, Will my patch be considered for 12.0? The calculation of the mem_usage value might be improved later on but because the system catalog is changed I would love to add it before 12.0 becomes stable. Nope. Code freeze for PG

Re: Runtime pruning problem

2019-07-30 Thread Tom Lane
David Rowley writes: > The part I wouldn't mind another set of eyes on is the ruleutils.c > changes. Um, sorry for not getting to this sooner. What I had in mind was to revert 1cc29fe7c's ruleutils changes entirely, so that ruleutils deals only in Plans not PlanStates. Perhaps we've grown some c

Re: AW: AW: Adding column "mem_usage" to view pg_prepared_statements

2019-07-30 Thread Tom Lane
Daniel Migowski writes: > Ok, just have read about the commitfest thing. Is the patch OK for that? Or > is there generally no love for a mem_usage column here? If it was, I really > would add some memory monitoring in our app regarding this. You should certainly put it into the next commitfest.

Re: Adding column "mem_usage" to view pg_prepared_statements

2019-07-30 Thread Daniel Migowski
Am 31.07.2019 um 00:17 schrieb Tomas Vondra: FWIW not sure what mail client you're using, but it seems to be breaking the threads for some reason, splitting it into two - see [1]. Also, please stop top-posting. It makes it way harder to follow the discussion. Was using Outlook because it's my

Re: Runtime pruning problem

2019-07-30 Thread David Rowley
On Wed, 31 Jul 2019 at 10:27, Tom Lane wrote: > > David Rowley writes: > > The part I wouldn't mind another set of eyes on is the ruleutils.c > > changes. > > Um, sorry for not getting to this sooner. > > What I had in mind was to revert 1cc29fe7c's ruleutils changes > entirely, so that ruleutils

Re: Adding column "mem_usage" to view pg_prepared_statements

2019-07-30 Thread Daniel Migowski
Am 31.07.2019 um 00:29 schrieb Tom Lane: Daniel Migowski writes: Ok, just have read about the commitfest thing. Is the patch OK for that? Or is there generally no love for a mem_usage column here? If it was, I really would add some memory monitoring in our app regarding this. You should ce

Re: Runtime pruning problem

2019-07-30 Thread Tom Lane
David Rowley writes: > On Wed, 31 Jul 2019 at 10:27, Tom Lane wrote: >> What I had in mind was to revert 1cc29fe7c's ruleutils changes >> entirely, so that ruleutils deals only in Plans not PlanStates. >> Perhaps we've grown some code since then that really needs the >> PlanStates, but what is th

Re: Runtime pruning problem

2019-07-30 Thread Tom Lane
I wrote: > This may be arguing for a change in ruleutils' existing behavior, > not sure. But when dealing with traditional-style inheritance, > I've always thought that Vars above the Append were referring to > the parent rel in its capacity as the parent, not in its capacity > as the first child.

Re: Runtime pruning problem

2019-07-30 Thread David Rowley
On Wed, 31 Jul 2019 at 10:56, Tom Lane wrote: > > OK, so experimenting, I see that it is a change: HEAD does > > regression=# explain verbose select * from part order by a; >QUERY PLAN > ---

Re: Runtime pruning problem

2019-07-30 Thread Tom Lane
David Rowley writes: > On Wed, 31 Jul 2019 at 10:56, Tom Lane wrote: >> The portion of this below the Append is fine, but I argue that >> the Vars above the Append should say "part", not "part_p1". >> In that way they'd look the same regardless of which partitions >> have been pruned or not. > T

Re: pgbench - implement strict TPC-B benchmark

2019-07-30 Thread Peter Geoghegan
On Tue, Jul 30, 2019 at 3:00 PM Tom Lane wrote: > TBH, I think we should reject this patch. Nobody cares about TPC-B > anymore, and they care even less about differences between one > sort-of-TPC-B test and another sort-of-TPC-B test. (As the lack > of response on this thread shows.) We don't n

Re: Avoiding hash join batch explosions with extreme skew and weird stats

2019-07-30 Thread Robert Haas
On Tue, Jul 30, 2019 at 2:47 PM Melanie Plageman wrote: > I did the "needlessly dumb implementation" Robert mentioned, though, > I thought about it and couldn't come up with a much smarter way to > write match bits to a file. I think there might be an optimization > opportunity in not writing the

Re: pgbench - implement strict TPC-B benchmark

2019-07-30 Thread Tom Lane
Peter Geoghegan writes: > On Tue, Jul 30, 2019 at 3:00 PM Tom Lane wrote: >> I'm also highly dubious about labeling this script "standard TPC-B", >> when it resolves only some of the reasons why our traditional script >> is not really TPC-B. That's treading on being false advertising. > IANAL,

Re: Adding column "mem_usage" to view pg_prepared_statements

2019-07-30 Thread David Fetter
On Tue, Jul 30, 2019 at 10:01:09PM +, Daniel Migowski wrote: > Hello, > > Will my patch be considered for 12.0? The calculation of the > mem_usage value might be improved later on but because the system > catalog is changed I would love to add it before 12.0 becomes > stable. Feature freeze

Re: Attached partition not considering altered column properties of root partition.

2019-07-30 Thread Amit Langote
On Wed, Jul 31, 2019 at 2:38 AM Robert Haas wrote: > > On Tue, Jul 2, 2019 at 9:53 PM Amit Langote wrote: > > Thanks for the report. This seems like a bug. Documentation claims > > that the child tables inherit column storage options from the parent > > table. That's actually enforced in only

Re: concerns around pg_lsn

2019-07-30 Thread Michael Paquier
On Tue, Jul 30, 2019 at 02:22:30PM +0530, Jeevan Ladhe wrote: > On Tue, Jul 30, 2019 at 9:42 AM Michael Paquier wrote: >> Agreed about making the code more defensive as you do. I would keep >> the initialization in check_recovery_target_lsn and pg_lsn_in_internal >> though. That does not hurt an

Re: Unused struct member in pgcrypto pgp.c

2019-07-30 Thread Michael Paquier
On Tue, Jul 30, 2019 at 05:48:49PM +0200, Daniel Gustafsson wrote: > In contrib/pgcrypto/pgp.c we have a struct member int_name in digest_info > which > isn’t used, and seems to have never been used (a potential copy/pasteo from > the > cipher_info struct?). Is there a reason for keeping this, o

Re: tap tests driving the database via psql

2019-07-30 Thread Craig Ringer
On Tue, 30 Jul 2019 at 21:40, Tom Lane wrote: > Craig Ringer writes: > > On Sun, 28 Jul 2019 at 03:15, Andres Freund wrote: > >> 1) Just depend on DBD::Pg being installed. It's fairly common, after > >> all. It'd be somewhat annoying that we'd often end up using a > >> different version of libp

Re: extension patch of CREATE OR REPLACE TRIGGER

2019-07-30 Thread Michael Paquier
On Tue, Jul 30, 2019 at 04:44:11PM -0400, Tom Lane wrote: > We do not test \h output in any existing regression test, and we're > not going to start doing so in this one. For one thing, the expected > URL would break every time we forked off a new release branch. > (There would surely be value in

Re: idea: log_statement_sample_rate - bottom limit for sampling

2019-07-30 Thread Michael Paquier
On Tue, Jul 30, 2019 at 03:43:58PM -0400, Tom Lane wrote: > Well, we do not need to have a backwards-compatibility problem > here, because we have yet to release a version containing > log_statement_sample_rate. I do not think it's too late to decide > that v12's semantics for that are broken, and

Re: Initdb failure

2019-07-30 Thread Michael Paquier
On Tue, Jul 30, 2019 at 03:30:12PM -0400, Tom Lane wrote: > On the whole though, I don't have a problem with the "do nothing" > answer. There's no security risk here, and no issue that seems > likely to arise in actual use cases rather than try-to-break-it > test scripts. +1. -- Michael signatu

Re: tap tests driving the database via psql

2019-07-30 Thread Andres Freund
Hi, On 2019-07-31 09:32:10 +0800, Craig Ringer wrote: > OK. So rather than building our own $everything from scratch, lets look at > solving that. IDK, a minimal driver that just does what we need it to do is a few hundred lines, not more. And there's plenty of stuff that we simply won't be able

Re: Runtime pruning problem

2019-07-30 Thread Amit Langote
On Wed, Jul 31, 2019 at 8:31 AM Tom Lane wrote: > David Rowley writes: > > On Wed, 31 Jul 2019 at 10:56, Tom Lane wrote: > >> The portion of this below the Append is fine, but I argue that > >> the Vars above the Append should say "part", not "part_p1". > >> In that way they'd look the same rega

Re: Avoiding hash join batch explosions with extreme skew and weird stats

2019-07-30 Thread Melanie Plageman
On Tue, Jul 30, 2019 at 4:36 PM Robert Haas wrote: > On Tue, Jul 30, 2019 at 2:47 PM Melanie Plageman > wrote: > > I did the "needlessly dumb implementation" Robert mentioned, though, > > I thought about it and couldn't come up with a much smarter way to > > write match bits to a file. I think t

Re: Avoiding hash join batch explosions with extreme skew and weird stats

2019-07-30 Thread Peter Geoghegan
On Tue, Jul 30, 2019 at 8:07 PM Melanie Plageman wrote: > For the actual write to disk, I'm pretty sure I get that for free from > the BufFile API, no? > I was more thinking about optimizing when I call BufFileWrite at all. Right. Clearly several existing buffile.c users regularly have very small

How to retain lesser paths at add_path()?

2019-07-30 Thread Kohei KaiGai
Hello, When we add a new path using add_path(), it checks estimated cost and path-keys, then it also removes dominated paths, if any. Do we have a reasonable way to retain these "dominated" paths? Once it is considered lesser paths at a level, however, it may have a combined cheaper cost with uppe

Re: Possible race condition in pg_mkdir_p()?

2019-07-30 Thread Michael Paquier
On Tue, Jul 30, 2019 at 06:22:59PM +0800, Ning Yu wrote: > In fact personally I'm thinking that whether could we replace all uses of > MakePGDirectory() with pg_mkdir_p(), so we could simplify > TablespaceCreateDbspace() and PathNameCreateTemporaryDir() and other callers > significantly. I would s

Re: SegFault on 9.6.14

2019-07-30 Thread Amit Kapila
On Wed, Jul 31, 2019 at 12:05 AM Robert Haas wrote: > > On Thu, Jul 18, 2019 at 9:45 AM Tom Lane wrote: > > I think this is going in the wrong direction. Nodes should *always* > > assume that a rescan is possible until ExecEndNode is called. > > If you want to do otherwise, you are going to be i

Re: Possible race condition in pg_mkdir_p()?

2019-07-30 Thread Andres Freund
Hi, On 2019-07-18 16:17:22 +0800, Ning Yu wrote: > This seems buggy as it first checks the existence of the dir and makes the > dir if it does not exist yet, however when executing concurrently a > possible race condition can be as below: > > A: does a/ exists? no > B: does a/ exists? no > A: try

Re: tap tests driving the database via psql

2019-07-30 Thread Michael Paquier
On Tue, Jul 30, 2019 at 07:20:53PM -0700, Andres Freund wrote: > IDK, a minimal driver that just does what we need it to do is a few > hundred lines, not more. And there's plenty of stuff that we simply > won't be able to test with any driver that's not purposefully written > for testing. There's e

Re: POC: Cleaning up orphaned files using undo logs

2019-07-30 Thread Amit Kapila
On Tue, Jul 30, 2019 at 1:32 PM Andres Freund wrote: > > Hi, > > Amit, short note: The patches aren't attached in patch order. Obviously > a miniscule thing, but still nicer if that's not the case. > Noted, I will try to ensure that patches are in order in future posts. -- With Regards, Amit Ka

  1   2   >