Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On Sun, Jul 28, 2019 at 10:33:03PM -0400, Bruce Momjian wrote: > I am thinking of writing some Assert() code that checks that all buffers > using a single LSN are from the same relation (and therefore different > page numbers). I would do it by creating a static array, clearing it on >

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On Mon, Jul 29, 2019 at 05:53:40PM -0400, Sehrope Sarkuni wrote: > I don't think this will be an issue in practice, but it should be documented. > Otherwise, it's not unreasonable for someone to expect that a promoted replica > would use be using new keys for everything after each promotion. > >

Re: POC: Cleaning up orphaned files using undo logs

On Tue, Jul 30, 2019 at 12:18 AM Robert Haas wrote: > > On Tue, Jul 23, 2019 at 10:42 AM Amit Khandekar > wrote: > > > Yes, I also think that the function would error out only because of > > can't-happen cases, like "too many locks taken" or "out of binary heap > > slots" or "out of memory"

Re: TopoSort() fix

On 2019-Jul-29, Tom Lane wrote: > FYI, I just got done inventing a way to reach that code, and I have > to suspect that it's impossible to do so in production, because under > ordinary circumstances no parallel worker will take any exclusive lock > that isn't already held by its leader. Hmm,

Re: Is ParsePrepareRecord dead function

On Tue, Jul 30, 2019 at 08:42:06AM +0530, Amit Kapila wrote: > Okay, can you point to that patch? Here you go: https://commitfest.postgresql.org/23/2105/ The thread is mostly waiting after Fujii-san for an update. -- Michael signature.asc Description: PGP signature

Re: Is ParsePrepareRecord dead function

On Tue, Jul 30, 2019 at 2:34 AM Alvaro Herrera wrote: > > On 2019-Jul-29, vignesh C wrote: > > > On Mon, Jul 29, 2019 at 8:24 PM Robert Haas wrote: > > > > > > On Mon, Jul 29, 2019 at 4:10 AM vignesh C wrote: > > > > I could not locate the caller of ParsePrepareRecord function in > > > >

Re: concerns around pg_lsn

On Mon, Jul 29, 2019 at 10:55:29PM +0530, Jeevan Ladhe wrote: > I am attaching a patch that makes sure that *have_error is set to false in > pg_lsn_in_internal() itself, rather than being caller dependent. Agreed about making the code more defensive as you do. I would keep the initialization in

Re: using explicit_bzero

On Mon, Jul 29, 2019 at 11:30:53AM +0200, Peter Eisentraut wrote: > Another patch, with various fallback implementations. I have spotted some issues with this patch: 1) The list of port files @pgportfiles in Mkvcbuild.pm has not been updated with the new file explicit_bzero.c, so the compilation

Re: using explicit_bzero

Hi, On 2019-07-29 11:30:53 +0200, Peter Eisentraut wrote: > For platforms that don't have explicit_bzero(), provide various > fallback implementations. (explicit_bzero() itself isn't standard, > but as Linux/glibc and OpenBSD have it, it's the most common spelling, > so it makes sense to make

Re: Define jsonpath functions as stable

On Mon, Jul 29, 2019 at 5:36 PM Tom Lane wrote: > > However, we've spotted some deviations between standard and our > > implementation. > > * like_regex predicate uses our regular expression engine, which > > deviates from standard. > > * We always do numeric computations using numeric

Re: Built-in connection pooler

On Mon, Jul 29, 2019 at 07:14:27PM +0300, Konstantin Knizhnik wrote: On 26.07.2019 23:24, Tomas Vondra wrote: Hi Konstantin, I've started reviewing this patch and experimenting with it, so let me share some initial thoughts. 1) not handling session state (yet) I understand handling

Re: POC: Cleaning up orphaned files using undo logs

On Fri, Jul 19, 2019 at 2:28 PM Amit Kapila wrote: > > On Thu, Jul 11, 2019 at 9:17 AM Dilip Kumar wrote: > > > > On Thu, Jul 11, 2019 at 12:38 AM Robert Haas wrote: > > > > > > I don't like the fact that undoaccess.c has a new global, > > > undo_compression_info. I haven't read the code

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On Mon, Jul 29, 2019 at 4:15 PM Alvaro Herrera wrote: > On 2019-Jul-27, Sehrope Sarkuni wrote: > > > Given the non-cryptographic nature of CRC and its 16-bit size, I'd > > round down the malicious tamper detection it provides to zero. At best > > it catches random disk errors so might as well

Re: block-level incremental backup

On Tue, Jul 30, 2019 at 1:58 AM Robert Haas wrote: > > I haven't had a chance to look at Jeevan's patch at all, or yours in > any detail, as yet, so these are just some very preliminary comments. > It will be good, however, if we can agree on who is going to do what > part of this as we try to

Re: Contribution to Perldoc for TestLib module in Postgres

On Fri, Jul 26, 2019 at 09:51:34AM -0400, Andrew Dunstan wrote: > I've fixed the bitrot and some other infelicities on this patch. It's > not commitable yet but I think it's more reviewable. Thanks, I had a look at this version. + # Returns the real directory for a virtual path directory under

Re: Define jsonpath functions as stable

On 07/29/19 18:27, Alexander Korotkov wrote: > What do you think about renaming existing operator from like_regex to > pg_like_regex? Or introducing special flag indicating that PostgreSQL > regex engine is used ('p' for instance)? Renaming the operator is simple and certainly solves the

Re: Is ParsePrepareRecord dead function

On 2019-Jul-30, Amit Kapila wrote: > On Tue, Jul 30, 2019 at 2:34 AM Alvaro Herrera > wrote: > > I think there's a patch from Fujii Masao that touches that? Might be > > premature to remove it. > > Okay, can you point to that patch?

Re: Patch for SortSupport implementation on inet/cdir

And a slightly amended version of the last patch with a bug fixed where IPv4 abbreviated keys were were not being initialized correctly on big-endian machines. v5-0001-SortSupport-for-inet-cidr-types.patch Description: Binary data

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

On Sun, Jul 21, 2019 at 01:34:22PM +0200, Tomas Vondra wrote: ... I wonder if we're approaching this wrong. Maybe we should not reverse engineer queries for the various places, but just start with a set of queries that we want to optimize, and then identify which places in the planner need to

Re: TopoSort() fix

On Mon, Jul 29, 2019 at 5:55 PM Tom Lane wrote: > FYI, I just got done inventing a way to reach that code, and I have > to suspect that it's impossible to do so in production, because under > ordinary circumstances no parallel worker will take any exclusive lock > that isn't already held by its

Re: TopoSort() fix

Robert Haas writes: > On Mon, Jul 29, 2019 at 5:55 PM Tom Lane wrote: >> FYI, I just got done inventing a way to reach that code, and I have >> to suspect that it's impossible to do so in production, because under >> ordinary circumstances no parallel worker will take any exclusive lock >> that

Re: Is ParsePrepareRecord dead function

On Tue, Jul 30, 2019 at 2:34 AM Alvaro Herrera wrote: > > On 2019-Jul-29, vignesh C wrote: > > > On Mon, Jul 29, 2019 at 8:24 PM Robert Haas wrote: > > > > > > On Mon, Jul 29, 2019 at 4:10 AM vignesh C wrote: > > > > I could not locate the caller of ParsePrepareRecord function in > > > >

Re: Define jsonpath functions as stable

On Mon, Jul 29, 2019 at 5:55 PM Chapman Flack wrote: > On 7/29/19 10:25 AM, Alexander Korotkov wrote: > > > * like_regex predicate uses our regular expression engine, which > > deviates from standard. > > I still favor adding some element to the syntax (like a 'posix' or 'pg' > keyword in the

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 7/29/19 6:11 PM, Sehrope Sarkuni wrote: > On Mon, Jul 29, 2019 at 4:15 PM Alvaro Herrera > wrote: > > On 2019-Jul-27, Sehrope Sarkuni wrote: > > > Given the non-cryptographic nature of CRC and its 16-bit size, I'd > > round down the malicious

Re: TopoSort() fix

On Mon, Jul 29, 2019 at 9:48 PM Tom Lane wrote: > I tried that first. There are backstops preventing doing LOCK TABLE > in a worker, just like for advisory locks. > > I believe the only accessible route to taking any sort of new lock > in a parallel worker is catalog lookups causing

Re: TopoSort() fix

On Mon, Jul 29, 2019 at 10:56:05AM -0400, Tom Lane wrote: > Michael Paquier writes: >> +1. Any volunteers? > > If Robert doesn't weigh in pretty soon, I'll take responsibility for it. Thanks Tom for taking care of it! -- Michael signature.asc Description: PGP signature

Re: POC: Cleaning up orphaned files using undo logs

Hi, I realize that this might not be the absolutely newest version of the undo storage part of this patchset - but I'm trying to understand the whole context, and that's hard without reading through the whole stack in a situation where the layers actually fit together On 2019-07-29 01:48:30

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On Mon, Jul 29, 2019 at 04:09:52PM -0400, Alvaro Herrera wrote: > On 2019-Jul-27, Bruce Momjian wrote: > > > I think using LSN and page number, we will _never_ reuse the IV, except > > for cases like promoting two standbys, which I think we have to document > > as an insecure practice. > >

Re: block-level incremental backup

Hi Jeevan I reviewed first two patches - 0001-Add-support-for-command-line-option-to-pass-LSN.patch and 0002-Add-TAP-test-to-test-LSN-option.patch from the set of incremental backup patches, and the changes look good to me. I had some concerns around the way we are working around with the

Re: should there be a hard-limit on the number of transactions pending undo?

On Mon, Jul 29, 2019 at 2:52 PM Thomas Munro wrote: > Thanks for sharing that. I see they're giving that paper at VLDB next > month in LA... I hope the talk video will be published on the web. > While we've been working on a hybrid vaccum/undo design, they've built > a hybrid undo/vacuum system.

Re: TopoSort() fix

Alvaro Herrera writes: > On 2019-Jul-29, Tom Lane wrote: >> FYI, I just got done inventing a way to reach that code, and I have >> to suspect that it's impossible to do so in production, because under >> ordinary circumstances no parallel worker will take any exclusive lock >> that isn't already

Re: should there be a hard-limit on the number of transactions pending undo?

On Mon, Jul 29, 2019 at 1:04 PM Robert Haas wrote: > I think there's a cost-benefit analysis here. You're completely > correct that inserting new index tuples causes write amplification > and, yeah, that's bad. On the other hand, row forwarding has its own > costs. If a row ends up persistently

Re: psql - add SHOW_ALL_RESULTS option

Hello Kyotaro-san, Thanks. I looked this more closely. Indeed! Thanks for this detailed review. + * Marshal the COPY data. Either subroutine will get the + * connection out of its COPY state, then call PQresultStatus() + * once and report any error. This comment doesn't explain what the

Re: should there be a hard-limit on the number of transactions pending undo?

On Tue, Jul 30, 2019 at 7:12 AM Peter Geoghegan wrote: > SQL Server 2019 has something called "instantaneous transaction > rollback", which seems to make SQL Server optionally behave a lot more > like Postgres [1], apparently with many of the same disadvantages as > Postgres. I agree that there

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On Mon, Jul 29, 2019 at 4:10 PM Alvaro Herrera wrote: > On 2019-Jul-27, Bruce Momjian wrote: > > > I think using LSN and page number, we will _never_ reuse the IV, except > > for cases like promoting two standbys, which I think we have to document > > as an insecure practice. > > Actually, why

Re: TopoSort() fix

[ removing , as that mailing address seems to be MIA ] Robert Haas writes: > On Mon, Jul 29, 2019 at 10:56 AM Tom Lane wrote: >> If Robert doesn't weigh in pretty soon, I'll take responsibility for it. > That's fine, or if you prefer that I commit it, I will. FYI, I just got done inventing a

Re: Attached partition not considering altered column properties of root partition.

Hi Alvaro, Thanks for looking at this. On Sat, Jul 27, 2019 at 8:38 AM Alvaro Herrera wrote: > Thanks for the patch! > > I'm not completely sold on back-patching this. Should we consider > changing it in 12beta and up only, or should we just backpatch it all > the way back to 9.4? It's going

Re: fsync error handling in pg_receivewal, pg_recvlogical

On 2019-07-27 19:02, Sehrope Sarkuni wrote: > Tried out this patch and it applies, compiles, and passes check-world. > Also flipped things around in pg_recvlogical.c to exit-on-success to > ensure it's actually being called and that worked too. Outside of a more > complicated harness that

Re: ANALYZE: ERROR: tuple already updated by self

On Sun, Jul 28, 2019 at 09:53:20PM -0700, Andres Freund wrote: Hi, On 2019-07-28 21:21:51 +0200, Tomas Vondra wrote: AFAICS it applies to 10+ versions, because that's where extended stats were introduced. We certainly can't mess with catalogs there, so this is about the only backpatchable fix

Re: ANALYZE: ERROR: tuple already updated by self

On Mon, Jul 29, 2019 at 10:15:36AM +0100, Dean Rasheed wrote: On Sun, 28 Jul 2019 at 11:15, Tomas Vondra wrote: Attached is a patch fixing the error by not building extended stats for the inh=true case (as already proposed in this thread). That's about the simplest way to resolve this issue

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On Mon, Jul 29, 2019 at 4:39 AM Masahiko Sawada wrote: > After more thoughts, I'm confused why we need to have MDEK. We can use > KEK derived from passphrase and TDEK and WDEK that are derived from > KEK. That way, we don't need store any key in database file. What is > the advantage of 3-tier

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On Mon, Jul 29, 2019 at 6:42 AM Masahiko Sawada wrote: > > An argument could be made to push that problem upstream, i.e. let the > > supplier of the passphrase deal with the indirection. You would still > > need to verify the supplied passphrase/key is correct via something > > like

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On Sat, Jul 27, 2019 at 12:27 AM Masahiko Sawada wrote: > > On Fri, Jul 26, 2019 at 10:57 AM Jonathan S. Katz > wrote: > > > > Hi, > > > > Before my reply, I wanted to say that I've been lurking on this thread > > for a bit as I've tried to better inform myself on encryption at rest > > and how

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On Mon, Jul 29, 2019, 20:43 Masahiko Sawada wrote: > That way, we never reuse IV in a different relation with the same page > number because relNode is unique within a database in a particular > tablespace as you mentioned. > Sorry, I meant that we can ensure IV+key is unique. -- Maaahiko

Is ParsePrepareRecord dead function

Hi, I could not locate the caller of ParsePrepareRecord function in twophase.c. Any idea how it gets called? or Is it a dead function? Regards, Vignesh EnterpriseDB: http://www.enterprisedb.com

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On Mon, Jul 29, 2019 at 11:33 AM Bruce Momjian wrote: > > On Thu, Jul 25, 2019 at 01:03:06PM -0400, Bruce Momjian wrote: > > On Tue, Jul 16, 2019 at 01:24:54PM +0900, Masahiko Sawada wrote: > > > On Sat, Jul 13, 2019 at 12:33 AM Bruce Momjian wrote: > > > > then each row change gets its own LSN.

[no subject]

Hello. On 2019/07/29 4:17, Dmitry Dolgov wrote:>> On Thu, Jul 25, 2019 at 1:21 PM Kyotaro Horiguchi wrote: > Yeah, will change both (hopefully soon) Thanks. >> + /* >> + * XXX: In case of index scan quals evaluation happens after >> + * ExecScanFetch, which means

Re: POC: Cleaning up orphaned files using undo logs

Hi On 2019-06-26 01:29:57 +0530, Amit Kapila wrote: > From 67845a7afa675e973bd0ea9481072effa1eb219d Mon Sep 17 00:00:00 2001 > From: Dilip Kumar > Date: Wed, 24 Apr 2019 14:36:28 +0530 > Subject: [PATCH 05/14] Add prefetch support for the undo log > > Add prefetching function for undo smgr and

Re: ANALYZE: ERROR: tuple already updated by self

On Sun, 28 Jul 2019 at 11:15, Tomas Vondra wrote: > > Attached is a patch fixing the error by not building extended stats for > the inh=true case (as already proposed in this thread). That's about the > simplest way to resolve this issue for v12. It should add a simple > regression test too, I

Re: using explicit_bzero

Another patch, with various fallback implementations. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services From c040e7aa31a7a12f804c8d32d403861390dec8d5 Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Mon, 29

Re: CVE-2017-7484-induced bugs, or, btree cmp functions are not leakproof?

On Wed, Jul 17, 2019 at 2:39 PM Amit Langote wrote: > > On Wed, Jul 10, 2019 at 2:43 PM Dilip Kumar wrote: > > On Wed, Jul 10, 2019 at 10:15 AM Amit Langote > > wrote: > > > Thanks for checking. There has been a lot of churn in the inheritance > > > planning code since my last email on this

Re: Fetching timeline during recovery

On Fri, 26 Jul 2019 18:22:25 +0200 Jehan-Guillaume de Rorthais wrote: > On Fri, 26 Jul 2019 10:02:58 +0200 > Jehan-Guillaume de Rorthais wrote: > > > On Fri, 26 Jul 2019 16:49:53 +0900 (Tokyo Standard Time) > > Kyotaro Horiguchi wrote: > [...] > > > We have an LSN reporting function each for

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On Mon, Jul 29, 2019 at 7:17 PM Sehrope Sarkuni wrote: > > On Mon, Jul 29, 2019 at 4:39 AM Masahiko Sawada wrote: > > After more thoughts, I'm confused why we need to have MDEK. We can use > > KEK derived from passphrase and TDEK and WDEK that are derived from > > KEK. That way, we don't need

Re: TopoSort() fix

Michael Paquier writes: > On Fri, Jul 26, 2019 at 08:24:16PM -0400, Tom Lane wrote: >> I think this is a sufficiently obvious bug, and a sufficiently >> obvious fix, that we should just fix it and not insist on getting >> a reproducible test case first. I think a test case would soon >> bit-rot

Re: POC: Cleaning up orphaned files using undo logs

On Sun, Jul 28, 2019 at 9:38 PM Thomas Munro wrote: > Interesting. One way to bring back posix_fallocate() without > upsetting people on some filesystem out there would be to turn the new > wal_init_zero GUC into a choice: write (current default, and current > behaviour for 'on'), pwrite_hole

Re: Define jsonpath functions as stable

Alexander Korotkov writes: > During my work on bringing jsonpath patchset to commit, I was always > keeping in mind that we need to make jsonb_path_*() functions > immutable. Having these functions immutable, users can build > expression indexes over them. Right. > However, we've spotted some

Re: how to run encoding-dependent tests by default

Peter Eisentraut writes: > On 2019-07-28 20:12, Tom Lane wrote: >> So I wish we could get rid of the Makefile changes, have the test >> scripts be completely responsible for whether to run themselves or >> not, and put them into the schedule files normally. > Good points. Updated patch attach.

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On Mon, Jul 29, 2019 at 08:43:06PM +0900, Masahiko Sawada wrote: > > I am thinking of writing some Assert() code that checks that all buffers > > using a single LSN are from the same relation (and therefore different > > page numbers). I would do it by creating a static array, clearing it on > >

Re: query1 followed by query2 at maximum distance vs current fixed distance

Hello, On 23.07.2019 09:55, Wh isere wrote: Is this possible with the current websearch_to_tsquery function? Thanks. Hello everyone, I am wondering if AROUND(N) or is still possible? I found this thread below and the original post

Define jsonpath functions as stable

Hi! During my work on bringing jsonpath patchset to commit, I was always keeping in mind that we need to make jsonb_path_*() functions immutable. Having these functions immutable, users can build expression indexes over them. Naturally, in majority of cases one doesn't need to index whole json

Re: Define jsonpath functions as stable

Hi, On 7/29/19 10:25 AM, Alexander Korotkov wrote: > * like_regex predicate uses our regular expression engine, which > deviates from standard. I still favor adding some element to the syntax (like a 'posix' or 'pg' keyword in the grammar for like_regex) that identifies it as using a

Re: Is ParsePrepareRecord dead function

On Mon, Jul 29, 2019 at 4:10 AM vignesh C wrote: > I could not locate the caller of ParsePrepareRecord function in twophase.c. > Any idea how it gets called? > or > Is it a dead function? It looks like it's not only dead, but stillborn. Commit 1eb6d6527aae264b3e0b9c95aa70bb7a594ad1cf introduced

Re: should there be a hard-limit on the number of transactions pending undo?

On Mon, Jul 22, 2019 at 4:15 AM Thomas Munro wrote: > I had a similar thought: you might regret that choice if you were > wanting to implement an AM with lock table-based concurrency control > (meaning that there are lock ordering concerns for row and page locks, > for DML statements, not just

Re: Proposal for Signal Detection Refactoring

Chris Travers writes: > Here's a new patch. No rush on it. I am moving it to next commitfest > anyway because as code documentation I think this is a low priority late in > the release cycle. > The changes mostly address Andres's feedback above. I took a look through this version. (BTW,

Re: POC: Cleaning up orphaned files using undo logs

On Tue, Jul 23, 2019 at 10:42 AM Amit Khandekar wrote: > I think, even though there might not be a correctness issue with the > current code as it stands, we should still use a local variable. > Updating MyUndoWorker is a big side-effect, which the caller is not > supposed to be aware of, because

Re: minor fixes after pgindent prototype fixes

On 2019-Jul-28, Tom Lane wrote: > Andres Freund writes: > > a few prototypes look odd. It appears to be cases where previously the > > odd indentation was put to some use, by indenting parameters less: > > ... > > but now that looks odd: > > extern void DefineCustomBoolVariable( > >

Re: Is ParsePrepareRecord dead function

On Mon, Jul 29, 2019 at 8:24 PM Robert Haas wrote: > > On Mon, Jul 29, 2019 at 4:10 AM vignesh C wrote: > > I could not locate the caller of ParsePrepareRecord function in twophase.c. > > Any idea how it gets called? > > or > > Is it a dead function? > > It looks like it's not only dead, but

Re: Built-in connection pooler

On 26.07.2019 23:24, Tomas Vondra wrote: Hi Konstantin, I've started reviewing this patch and experimenting with it, so let me share some initial thoughts. 1) not handling session state (yet) I understand handling session state would mean additional complexity, so I'm OK with not having

Re: should there be a hard-limit on the number of transactions pending undo?

On Fri, Jul 19, 2019 at 7:28 PM Peter Geoghegan wrote: > If I'm not mistaken, you're tacitly assuming that you'll always be > using zheap, or something sufficiently similar to zheap. It'll > probably never be possible to UNDO changes to something like a GIN > index on a zheap table, because you

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On Sun, Jul 28, 2019 at 10:33:03PM -0400, Bruce Momjian wrote: > I did some more research on which cases use a single LSN to modify > multiple 8k pages. The normal program flow is: > > XLogBeginInsert(); > ... > --> XLogRegisterBuffer(0, meta, ... > recptr =

Re: pg_upgrade fails with non-standard ACL

Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Bruce Momjian writes: > > On Thu, Jul 18, 2019 at 06:53:12PM +0300, Anastasia Lubennikova wrote: > >> pg_upgrade from 9.6 fails if old cluster had non-standard ACL > >> on pg_catalog functions that have changed between versions, > >> for

Re: benchmarking Flex practices

John Naylor writes: > On Sun, Jul 21, 2019 at 3:14 AM Tom Lane wrote: >> So I'm feeling like maybe we should experiment to see what that >> solution looks like, before we commit to going in this direction. >> What do you think? > Given the above wrinkles, I thought it was worth trying. Attached

Re: Built-in connection pooler

On 27.07.2019 14:49, Thomas Munro wrote: On Tue, Jul 16, 2019 at 2:04 AM Konstantin Knizhnik wrote: I have committed patch which emulates epoll EPOLLET flag and so should avoid busy loop with poll(). I will be pleased if you can check it at FreeBSD box. I tried your v12 patch and it gets

Re: SimpleLruTruncate() mutual exclusion

Noah Misch writes: > On Sun, Feb 17, 2019 at 11:31:03PM -0800, Noah Misch wrote: >>> b. Arrange so only one backend runs vac_truncate_clog() at a time. Other >>> than >>> AsyncCtl, every SLRU truncation appears in vac_truncate_clog(), in a >>> checkpoint, or in the startup process. Hence, also

concerns around pg_lsn

Hi all, While reviewing some code around pg_lsn_in() I came across a couple of (potential?) issues: 1. Commit 21f428eb moves lsn conversion functionality from pg_lsn_in() to a new function pg_lsn_in_internal(). It takes two parameters the lsn string and a pointer to a boolean (*have_error) to

Re: Remove page-read callback from XLogReaderState.

On 12/07/2019 10:10, Kyotaro Horiguchi wrote: At Tue, 28 May 2019 04:45:24 -0700, Andres Freund wrote in <20190528114524.dvj6ymap2virl...@alap3.anarazel.de> Hi, On 2019-04-18 21:02:57 +0900, Kyotaro HORIGUCHI wrote: Hello. As mentioned before [1], read_page callback in XLogReaderState is a

Re: should there be a hard-limit on the number of transactions pending undo?

On Mon, Jul 29, 2019 at 12:11 PM Robert Haas wrote: > I find this hard to believe, because an UPDATE can always be broken up > into a DELETE and an INSERT. If that were to be done, you would not > have a stable heap TID and you would have a "new HOT chain," or your > AM's equivalent of that

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 2019-Jul-27, Bruce Momjian wrote: > I think using LSN and page number, we will _never_ reuse the IV, except > for cases like promoting two standbys, which I think we have to document > as an insecure practice. Actually, why is it an insecure practice? If you promote two standbys, then the

Re: block-level incremental backup

On Wed, Jul 10, 2019 at 2:17 PM Anastasia Lubennikova wrote: > In attachments, you can find a prototype of incremental pg_basebackup, > which consists of 2 features: > > 1) To perform incremental backup one should call pg_basebackup with a > new argument: > > pg_basebackup -D 'basedir'

Re: Is ParsePrepareRecord dead function

On 2019-Jul-29, vignesh C wrote: > On Mon, Jul 29, 2019 at 8:24 PM Robert Haas wrote: > > > > On Mon, Jul 29, 2019 at 4:10 AM vignesh C wrote: > > > I could not locate the caller of ParsePrepareRecord function in > > > twophase.c. > > > Any idea how it gets called? > > > or > > > Is it a dead

Re: should there be a hard-limit on the number of transactions pending undo?

On Mon, Jul 29, 2019 at 9:35 AM Robert Haas wrote: > I mean, essentially any well-designed framework intended for any sort > of task whatsoever is going to have a design center where one can > foresee that it will work well, and then as a result of working well > for the thing for which it was

Re: should there be a hard-limit on the number of transactions pending undo?

On Mon, Jul 29, 2019 at 2:24 PM Peter Geoghegan wrote: > Yes. It doesn't really make any difference with B-Trees, because the > locks there are very similar to row locks (you still need forwarding > UNDO metadata in index pages, probably for checking the visibility of > index tuples that have

Re: should there be a hard-limit on the number of transactions pending undo?

On Mon, Jul 29, 2019 at 12:39 PM Peter Geoghegan wrote: > I think that indexes (or at least B-Tree indexes) will ideally almost > always have tuples that are the latest versions with zheap. The > exception is tuples whose ghost bit is set, whose visibility varies > based on the MVCC snapshot in

Re: should there be a hard-limit on the number of transactions pending undo?

On Mon, Jul 29, 2019 at 3:39 PM Peter Geoghegan wrote: > I'm not saying you can't handle it. But that necessitates "write > amplification", in the sense that you must now create new index tuples > even for indexes where the indexed columns were not logically altered. > Isn't zheap supposed to fix

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On Mon, Jul 29, 2019 at 9:44 AM Bruce Momjian wrote: > > Checking that all buffers using a single LSN are from the same > > relation would be a good idea but I think it's hard to test it and > > regard the test result as okay. Even if we passed 'make checkworld', > > it might still be possible

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 2019-Jul-27, Sehrope Sarkuni wrote: > Given the non-cryptographic nature of CRC and its 16-bit size, I'd > round down the malicious tamper detection it provides to zero. At best > it catches random disk errors so might as well keep it in plain text > and checkable offline. But what attack are

Re: TopoSort() fix

On Mon, Jul 29, 2019 at 10:56 AM Tom Lane wrote: > Michael Paquier writes: > > On Fri, Jul 26, 2019 at 08:24:16PM -0400, Tom Lane wrote: > >> I think this is a sufficiently obvious bug, and a sufficiently > >> obvious fix, that we should just fix it and not insist on getting > >> a reproducible