Re: Another way to fix inherited UPDATE/DELETE

2019-02-19 Thread Amit Langote
On 2019/02/20 13:54, Tom Lane wrote: > Amit Langote writes: >> On 2019/02/20 6:48, Tom Lane wrote: >>> What if we dropped that idea, and instead defined the plan tree as >>> returning only the columns that are updated by SET, plus the row >>> identity? It would then be the ModifyTable node's job

Re: Row Level Security − leakproof-ness and performance implications

2019-02-19 Thread Pierre Ducroquet
On Wednesday, February 20, 2019 12:43:50 AM CET Laurenz Albe wrote: > Pierre Ducroquet wrote: > > In order to increase our security, we have started deploying row-level > > security in order to add another safety net if any issue was to happen in > > our applications. > > After a careful

Re: shared-memory based stats collector

2019-02-19 Thread Kyotaro HORIGUCHI
At Fri, 15 Feb 2019 15:53:28 -0300, Alvaro Herrera wrote in <20190215185328.GA29663@alvherre.pgsql> > On 2019-Feb-15, Andres Freund wrote: > > > It's fine to do such renames, just do them as separate patches. It's > > hard enough to review changes this big... > > Talk about moving the whole

Re: [PATCH][PROPOSAL] Add enum releation option type

2019-02-19 Thread Nikolay Shaplov
В письме от среда, 20 февраля 2019 г. 15:08:32 MSK пользователь Michael Paquier написал: > On Sun, Jan 06, 2019 at 06:28:11PM +0300, Nikolay Shaplov wrote: > > Actually I am not satisfied with it too... That is why I started that > > bigger reloptions refactoring work. So for now I would offer to

RE: Speed up transaction completion faster after many relations are accessed in a transaction

2019-02-19 Thread Tsunakawa, Takayuki
From: Tom Lane [mailto:t...@sss.pgh.pa.us] > Hm. Putting a list header for a purely-local data structure into shared > memory seems quite ugly. Isn't there a better place to keep that? Agreed. I put it in the global variable. > Do we really want a dlist here at all? I'm concerned that

Re: Prepared transaction releasing locks before deregistering its GID

2019-02-19 Thread Masahiko Sawada
On Tue, Feb 19, 2019 at 12:27 PM Michael Paquier wrote: > > On Mon, Feb 18, 2019 at 05:05:13PM +0100, Oleksii Kliukin wrote: > > That looks like a race condition to me. What happens is that another > > transaction with the name identical to the running one can start and proceed > > to the prepare

RE: Speeding up creating UPDATE/DELETE generic plan for partitioned table into a lot

2019-02-19 Thread Kato, Sho
Before addressing to speeding up creating generic plan of UPDATE/DELETE, I will begin with the speed up creating SELECT plan. I will explain the background as time has passed. Since generic plan creates plans of all partitions and is cached, we can skip planning and expect performance

RE: Timeout parameters

2019-02-19 Thread Jamison, Kirk
Hi, I tried to re-read the whole thread. Based from what I read, there are two proposed timeout parameters, which I think can be discussed and commited separately: (1) tcp_user_timeout (2) tcp_socket_timeout (or suggested client_statement_timeout,

Re: [PATCH][PROPOSAL] Add enum releation option type

2019-02-19 Thread Michael Paquier
On Sun, Jan 06, 2019 at 06:28:11PM +0300, Nikolay Shaplov wrote: > Actually I am not satisfied with it too... That is why I started that bigger > reloptions refactoring work. So for now I would offer to keep initialization > as it was for other types: in initialize_reloptions using

Re: speeding up planning with partitions

2019-02-19 Thread Amit Langote
On 2019/02/20 5:57, Tom Lane wrote: > I wrote: >> OK, I'll make another pass over 0001 today. > > So I started the day with high hopes for this, but the more I looked at > it the less happy I got, and finally I ran into something that looks to > be a complete show-stopper. Namely, that the patch

RE: Speed up transaction completion faster after many relations are accessed in a transaction

2019-02-19 Thread Tsunakawa, Takayuki
From: Tomas Vondra [mailto:tomas.von...@2ndquadrant.com] > On 2/12/19 7:33 AM, Tsunakawa, Takayuki wrote: > > Imai-san confirmed performance improvement with this patch: > > > > https://commitfest.postgresql.org/22/1993/ > > > > Can you quantify the effects? That is, how much slower/faster does

Re: Another way to fix inherited UPDATE/DELETE

2019-02-19 Thread Tom Lane
Pavan Deolasee writes: > We will need to consider how this affects EvalPlanQual which currently > doesn't have to do anything special for partitioned tables. I solved that > via tracking the expanded-at-the-bottom child in a separate > mergeTargetRelation, but that approach has been criticised.

Re: Another way to fix inherited UPDATE/DELETE

2019-02-19 Thread Pavan Deolasee
On Wed, Feb 20, 2019 at 4:23 AM David Rowley wrote: > On Wed, 20 Feb 2019 at 10:49, Tom Lane wrote: > > What if we dropped that idea, and instead defined the plan tree as > > returning only the columns that are updated by SET, plus the row > > identity? It would then be the ModifyTable node's

Re: Another way to fix inherited UPDATE/DELETE

2019-02-19 Thread Tom Lane
Amit Langote writes: > On 2019/02/20 6:48, Tom Lane wrote: >> What if we dropped that idea, and instead defined the plan tree as >> returning only the columns that are updated by SET, plus the row >> identity? It would then be the ModifyTable node's job to fetch the >> original tuple using the

Re: pg_basebackup ignores the existing data directory permissions

2019-02-19 Thread Michael Paquier
On Wed, Feb 20, 2019 at 03:16:48PM +1100, Haribabu Kommi wrote: > Lack of complaints from the users, how about making this change in the HEAD? Fine by me. I would tend to patch pg_basebackup so as the target folder gets the correct umask instead of nuking the chmod call in initdb so I think that

Re: [PATCH] kNN for btree

2019-02-19 Thread Thomas Munro
On Wed, Feb 20, 2019 at 2:18 PM Nikita Glukhov wrote: > On 04.02.2019 8:35, Michael Paquier wrote: > This patch set needs a rebase because of conflicts caused by the > recent patches for pluggable storage. Hi Nikita, >From the department of trivialities: according to cfbot the documentation

Re: pg_basebackup ignores the existing data directory permissions

2019-02-19 Thread Haribabu Kommi
On Fri, Feb 15, 2019 at 10:15 AM Michael Paquier wrote: > On Thu, Feb 14, 2019 at 11:21:19PM +1100, Haribabu Kommi wrote: > > On Thu, Feb 14, 2019 at 8:57 PM Magnus Hagander > wrote: > >> I think it could be argued that neither initdb *or* pg_basebackup should > >> change the permissions on an

Re: Protect syscache from bloating with negative cache entries

2019-02-19 Thread Kyotaro HORIGUCHI
At Thu, 14 Feb 2019 00:40:10 -0800, Andres Freund wrote in <20190214084010.bdn6tmba2j7sz...@alap3.anarazel.de> > Hi, > > On 2019-02-13 15:31:14 +0900, Kyotaro HORIGUCHI wrote: > > Instead, I added an accounting(?) interface function. > > > > | MemoryContextGettConsumption(MemoryContext cxt); >

Re: Copy function for logical replication slots

2019-02-19 Thread Masahiko Sawada
On Wed, Feb 20, 2019 at 12:26 PM Michael Paquier wrote: > > On Tue, Feb 19, 2019 at 05:09:33PM +0900, Masahiko Sawada wrote: > > On Tue, Feb 19, 2019 at 1:28 AM Andres Freund wrote: > >> Well, I'd not thought we'd do it without acquiring the other slot. But > >> that still seems to be easy

Re: libpq debug log

2019-02-19 Thread Robert Haas
On Mon, Feb 18, 2019 at 10:06 PM Jamison, Kirk wrote: > It sounds more logical to me if there is a parameter that switches on/off the > logging > similar to other postgres logs. I suggest trace_log as the parameter name. I'm not really sure that I like the design of this patch in any way. But

Re: Pluggable Storage - Andres's take

2019-02-19 Thread Haribabu Kommi
On Tue, Nov 27, 2018 at 4:59 PM Amit Langote wrote: > Hi, > > On 2018/11/02 9:17, Haribabu Kommi wrote: > > Here I attached the cumulative fixes of the patches, new API additions > for > > zheap and > > basic outline of the documentation. > > I've read the documentation patch while also looking

Re: Pluggable Storage - Andres's take

2019-02-19 Thread Haribabu Kommi
On Mon, Feb 4, 2019 at 2:31 PM Haribabu Kommi wrote: > > On Tue, Jan 22, 2019 at 1:43 PM Haribabu Kommi > wrote: > >> >> >> OK. I will work on the doc changes. >> > > Sorry for the delay. > > Attached a draft patch of doc and comments changes that I worked upon. > Currently I added comments to

Re: Incorrect visibility test function assigned to snapshot

2019-02-19 Thread Michael Paquier
On Mon, Feb 18, 2019 at 10:45:38AM -0300, Alvaro Herrera wrote: > None here. Thanks. And committed. -- Michael signature.asc Description: PGP signature

Re: Compressed TOAST Slicing

2019-02-19 Thread Юрий Соколов
Some time ago I posted PoC patch with alternative TOAST compression scheme: instead of "compress-then-chunk" I suggested "chunk-then-compress". It decrease compression level, but allows efficient arbitrary slicing. ср, 20 февр. 2019 г., 2:09 Paul Ramsey pram...@cleverelephant.ca: > On Sat, Feb

Re: Copy function for logical replication slots

2019-02-19 Thread Michael Paquier
On Tue, Feb 19, 2019 at 05:09:33PM +0900, Masahiko Sawada wrote: > On Tue, Feb 19, 2019 at 1:28 AM Andres Freund wrote: >> Well, I'd not thought we'd do it without acquiring the other slot. But >> that still seems to be easy enough to address, we just need to recheck >> whether the slot still

Re: Delay locking partitions during INSERT and UPDATE

2019-02-19 Thread Robert Haas
On Tue, Feb 19, 2019 at 4:07 PM David Rowley wrote: > I'd say that here we should only discuss what this patch is doing, not > anything else that's in flight that you're concerned will conflict > with the ATTACH/DETACH PARTITION CONCURRENTLY patch. > > During INSERT and UPDATE, not all partitions

Re: Reaping Temp tables to avoid XID wraparound

2019-02-19 Thread Michael Paquier
On Tue, Feb 19, 2019 at 09:56:28AM +0100, Magnus Hagander wrote: > 2. Or probably even better, just put it in PgBackendStatus? Overhead here > is a lot cheaper than PGPROC. > > ISTM 2 is probably the most reasonable option here? Yes, I forgot this one. That would be more consistent, even if the

Re: Another way to fix inherited UPDATE/DELETE

2019-02-19 Thread Amit Langote
On 2019/02/20 10:55, Amit Langote wrote: > Maybe I should've mentioned that on this thread at some point. I meant the other thread where we're discussing my patches. Thanks, Amit

Re: Another way to fix inherited UPDATE/DELETE

2019-02-19 Thread Amit Langote
Hi, On 2019/02/20 6:48, Tom Lane wrote: > While contemplating the wreckage of > https://commitfest.postgresql.org/22/1778/ > I had the beginnings of an idea of another way to fix that problem. > > The issue largely arises from the fact that for UPDATE, we expect > the plan tree to emit a tuple

RE: SQL statement PREPARE does not work in ECPG

2019-02-19 Thread Takahashi, Ryohei
Hi Meskes-san, Thank you for your replying. > Please try this instead: > > EXEC SQL PREPARE test_prep (int) AS SELECT id from test_table where id > = $1; > EXEC SQL EXECUTE test_prep using 2; > > This should work. I tried as follows. EXEC SQL PREPARE

Re: CPU costs of random_zipfian in pgbench

2019-02-19 Thread Peter Geoghegan
On Tue, Feb 19, 2019 at 7:14 AM Fabien COELHO wrote: > What I like in "pgbench" is that it is both versatile and simple so that > people can benchmark their own data with their own load and their own > queries by writing a few lines of trivial SQL and psql-like slash command > and adjusting a few

Re: Using old master as new replica after clean switchover

2019-02-19 Thread Michael Paquier
On Tue, Feb 19, 2019 at 04:27:02PM -0800, RSR999GMAILCOM wrote: > So wanted to clarify if this procedure really requires the WAL archive > location on a shared storage ? Shared storage for WAL archives is not a requirement. It is perfectly possible to use streaming replication to get correct

Re: Prepared transaction releasing locks before deregistering its GID

2019-02-19 Thread Michael Paquier
On Tue, Feb 19, 2019 at 08:17:14PM +0100, Oleksii Kliukin wrote: > I gave it a spin on the same VM host as shown to constantly reproduce the > issue and observed neither 'identifier already in use' nor any locking > issues over a few dozens of runs, so it looks good to me. Thanks for the

RE: Protect syscache from bloating with negative cache entries

2019-02-19 Thread Tsunakawa, Takayuki
From: Ideriha, Takeshi [mailto:ideriha.take...@jp.fujitsu.com] > number of tables | 100 |1000|1 > --- > TPS (master) |10966 |10654 |9099 > TPS (patch)| 11137 (+1%) |10710 (+0%) |772 (-91%) > > It seems that before

Using old master as new replica after clean switchover

2019-02-19 Thread RSR999GMAILCOM
Hello Postgres Gurus, After searching (on www.postgresql.org/Google) I found that the following steps can be used to perform a switchover in Postgres (version 9.3): *Step 1.* Do clean shutdown of Primary (-m fast or smart). *Step 2. *Check for sync status and recovery status of Standby before

ZRE: Protect syscache from bloating with negative cache entries

2019-02-19 Thread Tsunakawa, Takayuki
From: Tomas Vondra [mailto:tomas.von...@2ndquadrant.com] > 0.7% may easily be just a noise, possibly due to differences in layout > of the binary. How many runs? What was the variability of the results > between runs? What hardware was this tested on? 3 runs, with the variability of about +-2%.

RE: Protect syscache from bloating with negative cache entries

2019-02-19 Thread Ideriha, Takeshi
>From: Ideriha, Takeshi [mailto:ideriha.take...@jp.fujitsu.com] >But at the same time, I did some benchmark with only hard limit option enabled >and >time-related option disabled, because the figures of this case are not >provided in this >thread. >So let me share it. I'm sorry but I'm taking

Re: Row Level Security − leakproof-ness and performance implications

2019-02-19 Thread Laurenz Albe
Pierre Ducroquet wrote: > In order to increase our security, we have started deploying row-level > security in order to add another safety net if any issue was to happen in our > applications. > After a careful monitoring of our databases, we found out that a lot of > queries started to go

Re: WAL insert delay settings

2019-02-19 Thread Tomas Vondra
On 2/19/19 8:40 PM, Andres Freund wrote: > Hi, > > On 2019-02-19 20:34:25 +0100, Tomas Vondra wrote: >> On 2/19/19 8:22 PM, Andres Freund wrote: >>> And my main point is that even if you implement a proper bytes/sec limit >>> ONLY for WAL, the behaviour of VACUUM rate limiting doesn't get >>>

More smarts about CoerceViaIO, and less stupidity about ArrayCoerceExpr

2019-02-19 Thread Tom Lane
I poked into a recent complaint[1] about PG not being terribly smart about whether an IS NOT NULL index predicate is implied by a WHERE clause, and determined that there are a couple of places where we are being less bright than we could be about CoerceViaIO semantics. CoerceViaIO is strict

Re: Compressed TOAST Slicing

2019-02-19 Thread Paul Ramsey
On Sat, Feb 16, 2019 at 7:25 AM Simon Riggs wrote: > Could we get an similarly optimized implementation of -> operator for JSONB > as well? > Are there any other potential uses? Best to fix em all up at once and then > move on to other things. Thanks. Oddly enough, I couldn't find many/any

Re: Some thoughts on NFS

2019-02-19 Thread Thomas Munro
On Wed, Feb 20, 2019 at 7:58 AM Robert Haas wrote: > On Tue, Feb 19, 2019 at 1:56 PM Andres Freund wrote: > > My point is that for iSCSC to be performant we'd need *all* the > > infrastructure we also need for direct IO *and* a *lot* more. And that > > it seems insane to invest very substantial

Re: Another way to fix inherited UPDATE/DELETE

2019-02-19 Thread David Rowley
On Wed, 20 Feb 2019 at 10:49, Tom Lane wrote: > What if we dropped that idea, and instead defined the plan tree as > returning only the columns that are updated by SET, plus the row > identity? It would then be the ModifyTable node's job to fetch the > original tuple using the row identity

Re: Another way to fix inherited UPDATE/DELETE

2019-02-19 Thread Andres Freund
Hi, On 2019-02-19 16:48:55 -0500, Tom Lane wrote: > I have no idea how this might play with the pluggable-storage work. I don't think it'd have a meaningful impact, except for needing changes to an overlapping set of lines. But given the different timeframes, I'd not expect a problem with that.

Re: propagating replica identity to partitions

2019-02-19 Thread Alvaro Herrera
On 2019-Feb-19, Robert Haas wrote: > On Tue, Feb 19, 2019 at 3:40 PM Alvaro Herrera > wrote: > > Maybe we should be using the inheritance marker in the command to note > > whether to recurse to partitions? That is, if you say > > ALTER TABLE ONLY parent SET REPLICA IDENTITY > > then we don't

Re: Some thoughts on NFS

2019-02-19 Thread Andres Freund
Hi, On 2019-02-20 11:25:22 +1300, Thomas Munro wrote: > This seems to make sense, and has the advantage that it uses > interfaces that exist right now. But it seems a bit like we'll have > to wait for them to finish building out the errseq_t support for NFS > to avoid various races around the

Re: Some thoughts on NFS

2019-02-19 Thread Thomas Munro
On Wed, Feb 20, 2019 at 5:52 AM Andres Freund wrote: > > 1. Figure out how to get the ALLOCATE command all the way through the > > stack from PostgreSQL to the remote NFS server, and know for sure that > > it really happened. On the Debian buster Linux 4.18 system I checked, > > fallocate()

Re: Some thoughts on NFS

2019-02-19 Thread Thomas Munro
On Tue, Feb 19, 2019 at 8:03 PM Thomas Munro wrote: > A theoretical question I thought of is whether there are any > interleavings of operations that allow a checkpoint to complete > bogusly, while a concurrent close() in a regular backend fails with > EIO for data that was included in the

Another way to fix inherited UPDATE/DELETE

2019-02-19 Thread Tom Lane
While contemplating the wreckage of https://commitfest.postgresql.org/22/1778/ I had the beginnings of an idea of another way to fix that problem. The issue largely arises from the fact that for UPDATE, we expect the plan tree to emit a tuple that's ready to be stored back into the target rel

Re: Delay locking partitions during INSERT and UPDATE

2019-02-19 Thread David Rowley
On Wed, 20 Feb 2019 at 06:36, Robert Haas wrote: > > On Mon, Feb 18, 2019 at 6:15 PM Tom Lane wrote: > > I'm inclined to think that if we already have lock on the parent > > partitioned table (thereby, IIUC, guaranteeing that its partitioning > > info can't change) that the order in which we

Re: speeding up planning with partitions

2019-02-19 Thread Tom Lane
I wrote: > OK, I'll make another pass over 0001 today. So I started the day with high hopes for this, but the more I looked at it the less happy I got, and finally I ran into something that looks to be a complete show-stopper. Namely, that the patch does not account for the possibility of an

Re: WAL insert delay settings

2019-02-19 Thread David Rowley
On Wed, 20 Feb 2019 at 07:28, Robert Haas wrote: > Or maybe we should just blow up the current vacuum cost delay stuff > and replace it with something that is easier to tune. For example, we > could just have one parameter that sets the maximum read rate in kB/s > and another that sets the

Re: propagating replica identity to partitions

2019-02-19 Thread Robert Haas
On Tue, Feb 19, 2019 at 3:40 PM Alvaro Herrera wrote: > Maybe we should be using the inheritance marker in the command to note > whether to recurse to partitions? That is, if you say > ALTER TABLE ONLY parent SET REPLICA IDENTITY > then we don't recurse and just change the parent table and

Re[2]: PGAdmin 4 don't refresh server info after restarting

2019-02-19 Thread Andrey Klychkov
> You should report this on a pgAdmin mailing list rather than a > PostgreSQL mailing list. Hi, of course, I made a mistake. Thank you for showing me >Вторник, 19 февраля 2019, 19:41 +03:00 от Robert Haas : > >On Mon, Feb 18, 2019 at 1:20 AM Andrey Klychkov < aaklych...@mail.ru > wrote: >>

Re: propagating replica identity to partitions

2019-02-19 Thread Alvaro Herrera
On 2019-Feb-19, Robert Haas wrote: > It's not unreasonable to use the parent's REPLICA IDENTITY setting as > the default for new partitions, much as we now do for the TABLESPACE, > because the parent's replica identity is otherwise without meaning. > But I'm less convinced that it's reasonable to

Re: proposal: pg_restore --convert-to-text

2019-02-19 Thread Euler Taveira
Em seg, 18 de fev de 2019 às 19:21, Tom Lane escreveu: > > Euler Taveira writes: > > Since no one has stepped up, I took a stab at it. It will prohibit > > standard output unless '-f -' be specified. -l option also has the > > same restriction. > > Hm, don't really see the need to break -l usage

Re: WAL insert delay settings

2019-02-19 Thread Andres Freund
Hi, On 2019-02-19 20:34:25 +0100, Tomas Vondra wrote: > On 2/19/19 8:22 PM, Andres Freund wrote: > > And my main point is that even if you implement a proper bytes/sec limit > > ONLY for WAL, the behaviour of VACUUM rate limiting doesn't get > > meaningfully more confusing than right now. > > >

Re: WAL insert delay settings

2019-02-19 Thread Tomas Vondra
On 2/19/19 8:22 PM, Andres Freund wrote: > On 2019-02-19 20:02:32 +0100, Tomas Vondra wrote: >> Let's do a short example. Assume the default vacuum costing parameters >> >> vacuum_cost_limit = 200 >> vacuum_cost_delay = 20ms >> cost_page_dirty = 20 >> >> and for simplicity we only

Re: WAL insert delay settings

2019-02-19 Thread Andres Freund
On 2019-02-19 20:02:32 +0100, Tomas Vondra wrote: > Let's do a short example. Assume the default vacuum costing parameters > > vacuum_cost_limit = 200 > vacuum_cost_delay = 20ms > cost_page_dirty = 20 > > and for simplicity we only do writes. So vacuum can do ~8MB/s of writes. > > Now,

Re: WAL insert delay settings

2019-02-19 Thread Tomas Vondra
On 2/19/19 7:28 PM, Robert Haas wrote: > On Fri, Feb 15, 2019 at 1:42 PM Andres Freund wrote: >> I think it'd not be insane to add two things: >> - WAL write rate limiting, independent of the vacuum stuff. It'd also be >> used by lots of other bulk commands (CREATE INDEX, ALTER TABLE >>

Re: propagating replica identity to partitions

2019-02-19 Thread Robert Haas
On Mon, Feb 4, 2019 at 11:30 AM Alvaro Herrera wrote: > If you do ALTER TABLE .. REPLICA IDENTITY to a partitioned table, the > command operates on the parent table itself and does not propagate to > partitions. Why is this? Maybe not recursing was the right call when > we only had regular

Re: Prepared transaction releasing locks before deregistering its GID

2019-02-19 Thread Oleksii Kliukin
Hi, Oleksii Kliukin wrote: > > The approach looks good to me. Surprisingly, I saw no stalled backends > because of the double acquisition of lock at TwoPhaseGetGXact once I put a > simple TwoPhaseStateLock right before the "gxact->valid = false” line; I > will test your patch and post the

Re: Some thoughts on NFS

2019-02-19 Thread Robert Haas
On Tue, Feb 19, 2019 at 2:05 PM Magnus Hagander wrote: > C'mon Robert. > > Surely you know that such patches should be landed on *Fridays*, not > Thursdays. Oh, right. And preferably via airplane wifi from someplace over the Atlantic ocean, right? -- Robert Haas EnterpriseDB:

Re: Some thoughts on NFS

2019-02-19 Thread Magnus Hagander
On Tue, Feb 19, 2019 at 7:58 PM Robert Haas wrote: > On Tue, Feb 19, 2019 at 1:56 PM Andres Freund wrote: > > My point is that for iSCSC to be performant we'd need *all* the > > infrastructure we also need for direct IO *and* a *lot* more. And that > > it seems insane to invest very substantial

Re: WAL insert delay settings

2019-02-19 Thread Tomas Vondra
On 2/19/19 7:50 PM, Andres Freund wrote: > On 2019-02-19 19:43:14 +0100, Tomas Vondra wrote: >> >> >> On 2/19/19 7:35 PM, Andres Freund wrote: >>> Hi, >>> >>> On 2019-02-19 13:28:00 -0500, Robert Haas wrote: On Fri, Feb 15, 2019 at 1:42 PM Andres Freund wrote: > I think it'd not be

Re: Some thoughts on NFS

2019-02-19 Thread Robert Haas
On Tue, Feb 19, 2019 at 1:56 PM Andres Freund wrote: > My point is that for iSCSC to be performant we'd need *all* the > infrastructure we also need for direct IO *and* a *lot* more. And that > it seems insane to invest very substantial resources into developing our > own iSCSI client when we

Re: Some thoughts on NFS

2019-02-19 Thread Andres Freund
Hi, On 2019-02-19 13:45:28 -0500, Robert Haas wrote: > On Tue, Feb 19, 2019 at 1:29 PM Andres Freund wrote: > > And I think it's not that likely that we'd not screw up a > > number of times implementing iSCSI ourselves - not to speak of the fact > > that that seems like an odd place to focus

Re: WAL insert delay settings

2019-02-19 Thread Andres Freund
On 2019-02-19 19:43:14 +0100, Tomas Vondra wrote: > > > On 2/19/19 7:35 PM, Andres Freund wrote: > > Hi, > > > > On 2019-02-19 13:28:00 -0500, Robert Haas wrote: > >> On Fri, Feb 15, 2019 at 1:42 PM Andres Freund wrote: > >>> I think it'd not be insane to add two things: > >>> - WAL write rate

Re: WAL insert delay settings

2019-02-19 Thread Robert Haas
On Tue, Feb 19, 2019 at 1:35 PM Andres Freund wrote: > I still don't *AT ALL* buy Stephen and Tomas' argument that it'd be > confusing that when both VACUUM and WAL cost limiting are active, the > lower limit takes effect. I think you guys may all be in vigorous -- not to say mortal --

Re: Some thoughts on NFS

2019-02-19 Thread Robert Haas
On Tue, Feb 19, 2019 at 1:29 PM Andres Freund wrote: > > Is that a new thing? I ran across PostgreSQL-over-iSCSI a number of > > years ago and the evidence strongly suggested that it did not reliably > > report disk errors back to PostgreSQL, leading to corruption. > > How many years ago are we

Re: WAL insert delay settings

2019-02-19 Thread Tomas Vondra
On 2/19/19 7:35 PM, Andres Freund wrote: > Hi, > > On 2019-02-19 13:28:00 -0500, Robert Haas wrote: >> On Fri, Feb 15, 2019 at 1:42 PM Andres Freund wrote: >>> I think it'd not be insane to add two things: >>> - WAL write rate limiting, independent of the vacuum stuff. It'd also be >>> used

Row Level Security − leakproof-ness and performance implications

2019-02-19 Thread Pierre Ducroquet
Hello In order to increase our security, we have started deploying row-level security in order to add another safety net if any issue was to happen in our applications. After a careful monitoring of our databases, we found out that a lot of queries started to go south, going extremely slow.

Re: WAL insert delay settings

2019-02-19 Thread Andres Freund
Hi, On 2019-02-19 13:28:00 -0500, Robert Haas wrote: > On Fri, Feb 15, 2019 at 1:42 PM Andres Freund wrote: > > I think it'd not be insane to add two things: > > - WAL write rate limiting, independent of the vacuum stuff. It'd also be > > used by lots of other bulk commands (CREATE INDEX,

Re: Some thoughts on NFS

2019-02-19 Thread Andres Freund
Hi, On 2019-02-19 13:21:21 -0500, Robert Haas wrote: > On Tue, Feb 19, 2019 at 1:17 PM Andres Freund wrote: > > On 2019-02-19 16:59:35 +0100, Magnus Hagander wrote: > > > There might be a use-case for the split that you mention, absolutely, but > > > it's not going to solve the

Re: WAL insert delay settings

2019-02-19 Thread Robert Haas
On Fri, Feb 15, 2019 at 1:42 PM Andres Freund wrote: > I think it'd not be insane to add two things: > - WAL write rate limiting, independent of the vacuum stuff. It'd also be > used by lots of other bulk commands (CREATE INDEX, ALTER TABLE > rewrites, ...) > - Account for WAL writes in the

Re: Some thoughts on NFS

2019-02-19 Thread Robert Haas
On Tue, Feb 19, 2019 at 1:17 PM Andres Freund wrote: > On 2019-02-19 16:59:35 +0100, Magnus Hagander wrote: > > There might be a use-case for the split that you mention, absolutely, but > > it's not going to solve the people-who-want-NFS situation. You'd solve more > > of that by having the

Re: Some thoughts on NFS

2019-02-19 Thread Andres Freund
Hi, On 2019-02-19 16:59:35 +0100, Magnus Hagander wrote: > There might be a use-case for the split that you mention, absolutely, but > it's not going to solve the people-who-want-NFS situation. You'd solve more > of that by having the middle layer speak "raw device" underneath and be > able to

Re: Some thoughts on NFS

2019-02-19 Thread Magnus Hagander
On Tue, Feb 19, 2019 at 5:38 PM Christoph Moench-Tegeder wrote: > ## Magnus Hagander (mag...@hagander.net): > > > You'd solve more > > of that by having the middle layer speak "raw device" underneath and be > > able to sit on top of things like iSCSI (yes, really). > > Back in ye olden days we

Re: Some thoughts on NFS

2019-02-19 Thread Magnus Hagander
On Tue, Feb 19, 2019 at 5:33 PM Tomas Vondra wrote: > > On 2/19/19 5:20 PM, Robert Haas wrote: > > On Tue, Feb 19, 2019 at 10:59 AM Magnus Hagander > wrote: > > >> There might be a use-case for the split that you mention, > >> absolutely, but it's not going to solve the people-who-want-NFS > >>

Re: CPU costs of random_zipfian in pgbench

2019-02-19 Thread David Fetter
On Sun, Feb 17, 2019 at 11:02:37PM +0100, Tomas Vondra wrote: > On 2/17/19 6:33 PM, David Fetter wrote: > > On Sun, Feb 17, 2019 at 11:09:27AM -0500, Tom Lane wrote: > >> Fabien COELHO writes: > I'm trying to use random_zipfian() for benchmarking of skewed data sets, > and I ran

Re: restrict pg_stat_ssl to superuser?

2019-02-19 Thread Robert Haas
On Thu, Feb 7, 2019 at 3:30 AM Peter Eisentraut wrote: > As discussed in [0], should we restrict access to pg_stat_ssl to > superusers (and an appropriate pg_ role)? > > If so, is there anything in that view that should be made available to > non-superusers? If not, then we could perhaps do this

Re: Delay locking partitions during INSERT and UPDATE

2019-02-19 Thread Robert Haas
On Mon, Feb 18, 2019 at 6:15 PM Tom Lane wrote: > I'm inclined to think that if we already have lock on the parent > partitioned table (thereby, IIUC, guaranteeing that its partitioning > info can't change) that the order in which we acquire the same lock > level on its partition(s) isn't very

Re: Delay locking partitions during INSERT and UPDATE

2019-02-19 Thread Robert Haas
On Fri, Feb 15, 2019 at 9:22 PM Andres Freund wrote: > On 2019-01-31 13:46:33 -0500, Robert Haas wrote: > > I have reviewed this patch and I am in favor of it. I think it likely > > needs minor rebasing because of the heap_open -> table_open renaming. > > I also agree that it's worth taking some

Re: speeding up planning with partitions

2019-02-19 Thread Tom Lane
Amit Langote writes: > On 2019/02/19 4:42, Tom Lane wrote: >> I don't much care for re-calling build_base_rel_tlists to add extra >> Vars to the appropriate relations; 99% of the work it does will be >> wasted, and with enough child rels you could run into an O(N^2) >> problem. Maybe you could

Re: unconstify equivalent for volatile

2019-02-19 Thread Andres Freund
Hi, On 2019-02-19 11:48:16 -0500, Tom Lane wrote: > Andres Freund writes: > > The real reason why variables commonly need to be volatile when used in > > signal handlers is not the signal handler side, but the normal code flow > > side. > > Yeah, exactly. You have not explained why it'd be

Re: Some thoughts on NFS

2019-02-19 Thread Andres Freund
Hi, On 2019-02-19 20:03:05 +1300, Thomas Munro wrote: > The first is practical. Running out of diskspace (or quota) is not > all that rare (much more common that EIO from a dying disk, I'd > guess), and definitely recoverable by an administrator: just create > more space. It would be really

Re: unconstify equivalent for volatile

2019-02-19 Thread Tom Lane
Andres Freund writes: > The real reason why variables commonly need to be volatile when used in > signal handlers is not the signal handler side, but the normal code flow > side. Yeah, exactly. You have not explained why it'd be safe to ignore that. regards, tom lane

Re: PGAdmin 4 don't refresh server info after restarting

2019-02-19 Thread Robert Haas
On Mon, Feb 18, 2019 at 1:20 AM Andrey Klychkov wrote: > Hello, > We've noticed that pgadmin 3.x / 4.x doesn't refresh server info like server > version after restarting. It makes people confused. > For example, > 1. PgAdmin was rinning > 2. We upgraded postgres from 11.1 to 11.2 > 3. PgAdmin

Re: Some thoughts on NFS

2019-02-19 Thread Christoph Moench-Tegeder
## Magnus Hagander (mag...@hagander.net): > You'd solve more > of that by having the middle layer speak "raw device" underneath and be > able to sit on top of things like iSCSI (yes, really). Back in ye olden days we called these middle layers "kernel" and "filesystem" and had that maintained

Re: Some thoughts on NFS

2019-02-19 Thread Robert Haas
On Tue, Feb 19, 2019 at 10:59 AM Magnus Hagander wrote: > The only case I've run into people wanting to use postgres on NFS, the NFS > server is a big filer from netapp or hitachi or whomever. And you're not > going to be able to run something like that on top of it. Yeah. :-( It seems,

Re: Some thoughts on NFS

2019-02-19 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Tue, Feb 19, 2019 at 2:03 AM Thomas Munro wrote: > > How can we achieve that, without writing our > > own NFS client? > > > > Instead of writing our own NFS client, how about writing our own > network storage protocol? Imagine a

Re: Allowing extensions to find out the OIDs of their member objects

2019-02-19 Thread Robert Haas
On Mon, Jan 21, 2019 at 9:46 PM Tom Lane wrote: > Perhaps this also gives some impetus to the lets-use-identifiers- > not-numbers approach that Andrew was pushing. I didn't care for > that too much so far as an extension's own internal references > are concerned, but for cross-extension

Re: Some thoughts on NFS

2019-02-19 Thread Joe Conway
On 2/19/19 10:59 AM, Magnus Hagander wrote: > On Tue, Feb 19, 2019 at 4:46 PM Robert Haas > wrote: > > On Tue, Feb 19, 2019 at 2:03 AM Thomas Munro > wrote: > > How can we achieve that, without writing our > > own NFS

Re: unconstify equivalent for volatile

2019-02-19 Thread Andres Freund
Hi, On February 19, 2019 7:00:58 AM PST, Peter Eisentraut wrote: >On 2019-02-18 21:25, Andres Freund wrote: >> ISTM this one should rather be solved by removing all volatiles from >> latch.[ch]. As that's a cross-process concern we can't rely on it >> anyway (and have placed barriers a few

Re: FOP warnings about id attributes in title tags

2019-02-19 Thread Peter Eisentraut
On 2019-02-18 16:37, Peter Eisentraut wrote: >> It appears that these are due to title elements having id tags. At >> is says: >> >> When adding an |id| or |xml:id| attribute, put it on the element >> itself, not the |title|. >> >> So

Re: Some thoughts on NFS

2019-02-19 Thread Magnus Hagander
On Tue, Feb 19, 2019 at 4:46 PM Robert Haas wrote: > On Tue, Feb 19, 2019 at 2:03 AM Thomas Munro > wrote: > > How can we achieve that, without writing our > > own NFS client? > > > You'll need it :) Instead of writing our own NFS client, how about writing our own > network storage

Re: restrict pg_stat_ssl to superuser?

2019-02-19 Thread Peter Eisentraut
On 2019-02-18 04:58, Michael Paquier wrote: > On Fri, Feb 15, 2019 at 02:04:59PM +0100, Peter Eisentraut wrote: >> We could remove default privileges from pg_stat_get_activity(). Would >> that be a problem? > > I don't think so, still I am wondering about the impact that this > could have for

Re: Some thoughts on NFS

2019-02-19 Thread Robert Haas
On Tue, Feb 19, 2019 at 2:03 AM Thomas Munro wrote: > How can we achieve that, without writing our > own NFS client? Instead of writing our own NFS client, how about writing our own network storage protocol? Imagine a stripped-down postmaster process running on the NFS server that essentially

Re: Allowing extensions to find out the OIDs of their member objects

2019-02-19 Thread Tom Lane
I wrote: > [ discussion about ways to let extension C code find out object OIDs ] I wanted to close out this thread, for the time being, by saying that I'm not expecting to get anything done about it for v12. It seems pretty late in the dev cycle to be proposing any major new user-visible

Re: Progress reporting for pg_verify_checksums

2019-02-19 Thread Fabien COELHO
Hallo Michael, New patch attached. Patch applies cleanly. Compiles, "make check" ok. doc build is also ok. There are no tests, which is probably fine for such an interactive feature. Docs look okay to me. Clear and to the point. About : total_percent = total_size ? (int64)

  1   2   >