Re: SET ROLE x NO RESET

2024-01-03 Thread Nico Williams
On Tue, Jan 02, 2024 at 12:36:38PM -0500, Robert Haas wrote: > IMHO, the best solution here would be a protocol message to change the > session user. The pooler could use that repeatedly on the same > session, but refuse to propagate such messages from client > connections. But this requires

Re: SET ROLE x NO RESET

2024-01-03 Thread Nico Williams
On Sat, Dec 30, 2023 at 10:16:59AM -0600, Eric Hanson wrote: > What do you think of adding a NO RESET option to the SET ROLE command? I've wanted this forever. Consider using this to implement user authentication mechanisms in user-defined SQL functions that use `SET ROLE` with `NO RESET` to

Re: WIP Incremental JSON Parser

2024-01-03 Thread Nico Williams
On Tue, Jan 02, 2024 at 10:14:16AM -0500, Robert Haas wrote: > It seems like a pretty significant savings no matter what. Suppose the > backup_manifest file is 2GB, and instead of creating a 2GB buffer, you > create an 1MB buffer and feed the data to the parser in 1MB chunks. > Well, that saves

Re: Pre-proposal: unicode normalized text

2023-11-02 Thread Nico Williams
On Wed, Oct 04, 2023 at 01:15:03PM -0700, Jeff Davis wrote: > > The fact that there are multiple types of normalization and multiple > > notions of equality doesn't make this easier. And then there's text that isn't normalized to any of them. > NFC is really the only one that makes sense. Yes.

Re: Pre-proposal: unicode normalized text

2023-11-02 Thread Nico Williams
On Tue, Oct 17, 2023 at 05:07:40PM +0200, Daniel Verite wrote: > > * Add a per-database option to enforce only storing assigned unicode > > code points. > > There's a problem in the fact that the set of assigned code points is > expanding with every Unicode release, which happens about every

Re: Pre-proposal: unicode normalized text

2023-11-02 Thread Nico Williams
On Wed, Oct 04, 2023 at 01:16:22PM -0400, Robert Haas wrote: > There's a very popular commercial database where, or so I have been > led to believe, any byte sequence at all is accepted when you try to > put values into the database. [...] In other circles we call this "just-use-8". ZFS, for

Re: Pre-proposal: unicode normalized text

2023-11-02 Thread Nico Williams
On Fri, Oct 06, 2023 at 02:37:06PM -0400, Robert Haas wrote: > > Sure, because TEXT in PG doesn't have codeset+encoding as part of it -- > > it's whatever the database's encoding is. Collation can and should be a > > porperty of a column, since for Unicode it wouldn't be reasonable to > > make

Re: Pre-proposal: unicode normalized text

2023-10-06 Thread Nico Williams
On Fri, Oct 06, 2023 at 02:17:32PM -0400, Robert Haas wrote: > On Fri, Oct 6, 2023 at 1:38 PM Nico Williams wrote: > > On Fri, Oct 06, 2023 at 01:33:06PM -0400, Robert Haas wrote: > > > On Thu, Oct 5, 2023 at 3:15 PM Nico Williams > > > wrote: > > > &g

Re: Pre-proposal: unicode normalized text

2023-10-06 Thread Nico Williams
On Fri, Oct 06, 2023 at 01:33:06PM -0400, Robert Haas wrote: > On Thu, Oct 5, 2023 at 3:15 PM Nico Williams wrote: > > Text+encoding can be just like bytea with a one- or two-byte prefix > > indicating what codeset+encoding it's in. That'd be how to encode > > such te

Re: Pre-proposal: unicode normalized text

2023-10-05 Thread Nico Williams
On Thu, Oct 05, 2023 at 03:49:37PM -0400, Tom Lane wrote: > Nico Williams writes: > > Text+encoding can be just like bytea with a one- or two-byte prefix > > indicating what codeset+encoding it's in. That'd be how to encode > > such text values on the wire, though on d

Re: Pre-proposal: unicode normalized text

2023-10-05 Thread Nico Williams
On Thu, Oct 05, 2023 at 07:31:54AM -0400, Robert Haas wrote: > [...] On the other hand, to do that in PostgreSQL, we'd need to > propagate the character set/encoding information into all of the > places that currently get the typmod and collation, and that is not a > small number of places. It's a

Re: Pre-proposal: unicode normalized text

2023-10-04 Thread Nico Williams
On Wed, Oct 04, 2023 at 04:01:26PM -0700, Jeff Davis wrote: > On Wed, 2023-10-04 at 16:15 -0500, Nico Williams wrote: > > Better that than TEXT blobs w/ the encoding given by the `CREATE > > DATABASE` or `initdb` default! > > From an engineering perspective, yes, per-c

Re: Pre-proposal: unicode normalized text

2023-10-04 Thread Nico Williams
On Wed, Oct 04, 2023 at 05:32:50PM -0400, Chapman Flack wrote: > Well, for what reason does anybody run PG now with the encoding set > to anything besides UTF-8? I don't really have my finger on that pulse. Because they still have databases that didn't use UTF-8 10 or 20 years ago that they

Re: Pre-proposal: unicode normalized text

2023-10-04 Thread Nico Williams
On Wed, Oct 04, 2023 at 01:38:15PM -0700, Jeff Davis wrote: > On Wed, 2023-10-04 at 14:02 -0400, Chapman Flack wrote: > > The SQL standard would have me able to: > > > > [...] > > _UTF8'Hello, world!' and _LATIN1'Hello, world!' > > Is there a use case for that? UTF-8 is able to encode any

Re: Pre-proposal: unicode normalized text

2023-10-04 Thread Nico Williams
On Tue, Sep 12, 2023 at 03:47:10PM -0700, Jeff Davis wrote: > The idea is to have a new data type, say "UTEXT", that normalizes the > input so that it can have an improved notion of equality while still > using memcmp(). A UTEXT type would be helpful for specifying that the text must be Unicode

Re: Pre-proposal: unicode normalized text

2023-10-03 Thread Nico Williams
On Tue, Oct 03, 2023 at 03:34:44PM -0700, Jeff Davis wrote: > On Tue, 2023-10-03 at 15:15 -0500, Nico Williams wrote: > > Ugh, My client is not displying 'a' correctly > > Ugh. Is that an argument in favor of normalization or against? Heheh, well, it's an argument in favor o

Re: Pre-proposal: unicode normalized text

2023-10-03 Thread Nico Williams
On Tue, Oct 03, 2023 at 12:15:10PM -0700, Jeff Davis wrote: > On Mon, 2023-10-02 at 15:27 -0500, Nico Williams wrote: > > I think you misunderstand Unicode normalization and equivalence.  > > There is no standard Unicode `normalize()` that would cause the > > above equality

Re: Pre-proposal: unicode normalized text

2023-10-02 Thread Nico Williams
On Tue, Sep 12, 2023 at 03:47:10PM -0700, Jeff Davis wrote: > One of the frustrations with using the "C" locale (or any deterministic > locale) is that the following returns false: > > SELECT 'á' = 'á'; -- false > > because those are the unicode sequences U&'\0061\0301' and U&'\00E1', >

Re: proposal: schema private functions

2018-08-23 Thread Nico Williams
Couldn't this be handled by having a new permission on FUNCTIONs ("CALL"?) to distinguish EXECUTE? This would have to be made backwards-compatible, possibly by automatically granting CALL ON ALL FUNCTIONS to public at schema create time and/or PG upgrade time, which the schema owner could then

Re: Allowing printf("%m") only where it actually works

2018-08-19 Thread Nico Williams
On Sun, Aug 19, 2018 at 01:15:58AM -0400, Tom Lane wrote: > Nico Williams writes: > > On Sat, Aug 18, 2018 at 04:34:50PM -0400, Tom Lane wrote: > >> So now I'm about ready to propose that we just *always* use > >> snprintf.c, and forget all of the related configure pro

Re: Allowing printf("%m") only where it actually works

2018-08-18 Thread Nico Williams
On Sat, Aug 18, 2018 at 04:34:50PM -0400, Tom Lane wrote: > So now I'm about ready to propose that we just *always* use > snprintf.c, and forget all of the related configure probing. Yes. > This'd have some advantages, notably that we'd get the > useful_strerror() behavior in frontend as well as

Re: C99 compliance for src/port/snprintf.c

2018-08-15 Thread Nico Williams
There's also clang on Windows, which VS apparently supports. With clang on Windows PG could even make use of GCC/Clang C extensions :^)

Re: Facility for detecting insecure object naming

2018-08-15 Thread Nico Williams
On Tue, Aug 14, 2018 at 11:50:24PM +, Nasby, Jim wrote: > On Aug 14, 2018, at 4:01 PM, Nico Williams wrote: > > > > On Tue, Aug 14, 2018 at 03:00:55PM +, Robert Haas wrote: > >> The more I think about it, the more I think having a way to set a > >

Re: Facility for detecting insecure object naming

2018-08-14 Thread Nico Williams
On Tue, Aug 14, 2018 at 03:00:55PM +, Robert Haas wrote: > The more I think about it, the more I think having a way to set a > lexically-scoped search path is probably the answer. [...] Yes please! This is what I want. Evaluate the search_path at function definition time, and record code

Re: libpq should not look up all host addresses at once

2018-08-14 Thread Nico Williams
On Tue, Aug 14, 2018 at 03:18:32PM -0400, Garick Hamlin wrote: > On Tue, Aug 14, 2018 at 12:24:32PM +0200, Fabien COELHO wrote: > > I read the rational of the host/hostaddr artificial mapping. I cannot say > > I'm thrilled with the result: I do not really see a setting where avoiding a > > DNS

Re: NetBSD vs libxml2

2018-08-13 Thread Nico Williams
On Mon, Aug 13, 2018 at 11:16:23AM -0400, Tom Lane wrote: > Well, the issue is that new kinds of switches introduce new potential > for bugs. In the case of -Wl,-R..., I'm not even sure that you can > write that more than once per link, so absorbing one from xml2-config > might well break things

Re: NetBSD vs libxml2

2018-08-13 Thread Nico Williams
[Quoting out of order.] On Mon, Aug 13, 2018 at 11:16:23AM -0400, Tom Lane wrote: > Robert Haas writes: > > I'm not, personally, eager to do that work for a requirement which > somehow hasn't surfaced on any other platform, nor on any previous > NetBSD release. I think NetBSD is way out in left

Re: NetBSD vs libxml2

2018-08-11 Thread Nico Williams
On Sat, Aug 11, 2018 at 01:12:09PM -0500, Nico Williams wrote: > I'm willing to write a patch after lunch. In ./configure.in this: > > for pgac_option in `$XML2_CONFIG --libs`; do > case $pgac_option in > -L*) LDFLAGS="$LDFLAGS $pgac_option";

Re: Facility for detecting insecure object naming

2018-08-11 Thread Nico Williams
On Wed, Aug 08, 2018 at 10:47:04AM -0400, Tom Lane wrote: > Isaac Morland writes: > > While I'm asking, does anybody know why this isn't the default, especially > > for SECURITY DEFINER functions? > > It might fix some subset of security issues, but I think that changing > the default behavior

Re: Facility for detecting insecure object naming

2018-08-11 Thread Nico Williams
On Sat, Aug 11, 2018 at 12:47:05PM -0700, Noah Misch wrote: > -- (3) "SET search_path" with today's code. > -- > -- Security and reliability considerations are the same as (2). Today, this > -- reduces performance by suppressing optimizations like inlining. Out of curiosity, why does this

Re: NetBSD vs libxml2

2018-08-11 Thread Nico Williams
On Sat, Aug 11, 2018 at 01:18:26PM -0400, Tom Lane wrote: > In a moment of idle curiosity, I tried to build PG --with-libxml > on NetBSD-current (well, mostly current, from May or so). > The configure script blew up, complaining that it couldn't execute > a test program. Investigation showed that

Re: [FEATURE REQUEST] Encrypted indexes over encrypted data

2018-08-09 Thread Nico Williams
On Thu, Aug 09, 2018 at 02:34:07PM -0600, Bear Giles wrote: > Some regulatory standards require all UII, even all PII, information be > encrypted within the database, not just on encrypted media. That's to > reduce exposure even if someone gets access to a live server, e.g., via SQL > Injection.

Re: POC for a function trust mechanism

2018-08-09 Thread Nico Williams
On Wed, Aug 08, 2018 at 01:15:38PM -0400, Tom Lane wrote: > This is sort of a counter-proposal to Noah's discussion of search path > security checking in <20180805080441.gh1688...@rfd.leadboat.com>. > (There's no technical reason we couldn't do both things, but I think > this'd be more useful to

Re: [FEATURE REQUEST] Encrypted indexes over encrypted data

2018-08-09 Thread Nico Williams
On Thu, Aug 09, 2018 at 03:00:26PM +0300, Danylo Hlynskyi wrote: > The problem > == > > [...] > > We don't trust full-disk-encryption or any other transparent encryption, > because of possible SQL injections. Can you elaborate on this? > Solution 1 (possibly can be used even now) >

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-08-08 Thread Nico Williams
On Wed, Aug 08, 2018 at 11:47:34AM -0500, Nico Williams wrote: > Yes. Would that snprintf() and vsnprintf() were async-signal-safe -- > they can be, and some implementations probably are, but they aren't > required to be, then making ereport() safe would be easier. So, I took a look a

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-08-08 Thread Nico Williams
On Wed, Aug 08, 2018 at 07:19:34PM +0300, Heikki Linnakangas wrote: > On 20/07/18 18:03, Andres Freund wrote: > >It's much less the exit() that's unsafe, than the callbacks themselves, > >right? Perhaps just restate that we wouldn't want to trigger atexit > >processing due to signal safety? > >

Re: Have an encrypted pgpass file

2018-08-08 Thread Nico Williams
On Tue, Jul 24, 2018 at 12:25:31PM +0200, Marco van Eck wrote: > Indeed having unencrypted password lying (.pgpass or PGPASSWORD or -W) > around is making my auditors unhappy, and forcing me to enter the password > over and over again. With a simple test it seems the password entered by > the user

Re: [PATCH v18] GSSAPI encryption support

2018-08-06 Thread Nico Williams
On Mon, Aug 06, 2018 at 10:36:34AM -0400, Stephen Frost wrote: > * Heikki Linnakangas (hlinn...@iki.fi) wrote: > > Sorry if this sounds facetious, but: > > > > What is the point of this patch? What's the advantage of GSSAPI encryption > > over SSL? I was hoping to find the answer by reading the

Re: How can we submit code patches that implement our (pending) patents?

2018-07-27 Thread Nico Williams
On Fri, Jul 27, 2018 at 10:01:40AM -0700, Andres Freund wrote: > On 2018-07-27 11:15:00 -0500, Nico Williams wrote: > > Even assuming you can't change the PG license, you could still: > > > > - require disclosure in contributions > > That really has no upsi

Re: How can we submit code patches that implement our (pending) patents?

2018-07-27 Thread Nico Williams
Even assuming you can't change the PG license, you could still: - require disclosure in contributions - require a wide grant in contributions - document all such grants separately from the copyright license Putting the grants in the license is convenient, but it's not required to include

Re: How can we submit code patches that implement our (pending) patents?

2018-07-27 Thread Nico Williams
On Fri, Jul 27, 2018 at 09:30:45AM -0400, Robert Haas wrote: > If you think that the lack of a CLA and a patent grant never causes > extensive conversations with legal, I am quite certain that you are > incorrect. I know of multiple instances where this has been a > concern. > > Other open

Re: How can we submit code patches that implement our (pending) patents?

2018-07-26 Thread Nico Williams
On Thu, Jul 26, 2018 at 04:42:24PM -0400, Tom Lane wrote: > It's barely possible that we could get current and new contributors to > sign some kind of CLA containing anti-patent terms, but I don't think > there's any hope of amending the distribution license. Leaving aside whether you could

Re: Early WIP/PoC for inlining CTEs

2018-07-25 Thread Nico Williams
On Tue, Jul 24, 2018 at 07:57:49PM -0400, Tom Lane wrote: > Andres Freund writes: > > On 2018-07-24 19:49:19 -0400, Tom Lane wrote: > >> However, a singly-referenced SELECT CTE could reasonably be treated as > >> equivalent to a sub-select-in-FROM, and then you would have the same > >> mechanisms

Re: Early WIP/PoC for inlining CTEs

2018-07-25 Thread Nico Williams
On Wed, Jul 25, 2018 at 05:08:43PM +0100, Andrew Gierth wrote: > Nico> It is possible to add a keyword for this purpose in the WITH syntax: > > Nico> WITH VIEW (...) AS a_view > > The existing (and standard) syntax is WITH ctename AS (query). Oy, I flubbed that up. > Syntaxes that have

Re: How can we submit code patches that implement our (pending) patents?

2018-07-25 Thread Nico Williams
On Wed, Jul 25, 2018 at 11:45:58AM -0400, Chapman Flack wrote: > On 07/25/2018 11:25 AM, Nico Williams wrote: > > > I don't understand why it's not obvious that one can unknowingly and > > accidentally re-invent someone else's idea. > > It's perfectly obvious. It's th

Re: Early WIP/PoC for inlining CTEs

2018-07-25 Thread Nico Williams
On Wed, Jul 25, 2018 at 07:42:37AM +0200, David Fetter wrote: > Please find attached a version rebased atop 167075be3ab1547e18 with > what I believe are appropriate changes to regression test output. The > other changes to the regression tests output are somewhat puzzling, as > they change the

Re: How can we submit code patches that implement our (pending) patents?

2018-07-25 Thread Nico Williams
On Wed, Jul 25, 2018 at 02:48:01PM +0700, Benjamin Scherrey wrote: > If you violate a patent, knowingly or otherwise, you are subject to > penalties (perhaps not treble but still penalties) and will have to remove > the offending code unless a deal is reached with the patent holder. Unless you do

Re: How can we submit code patches that implement our (pending) patents?

2018-07-25 Thread Nico Williams
On Wed, Jul 25, 2018 at 03:06:22AM -0400, Chapman Flack wrote: > On 07/25/18 01:56, Nico Williams wrote: > > > Wrong. With patents the important thing is not to know about them when > > you implement -- if you come up with the same idea by accident (which, > > of cours

Re: How can we submit code patches that implement our (pending) patents?

2018-07-24 Thread Nico Williams
On Tue, Jul 24, 2018 at 06:29:37PM -0400, Isaac Morland wrote: > On 24 July 2018 at 18:17, Nico Williams wrote: > > Note that it's OK to *accidentally* implement patented algorithms as > > long as the author of the contribution didn't know about. There's no > > trebble

Re: How can we submit code patches that implement our (pending) patents?

2018-07-24 Thread Nico Williams
On Tue, Jul 24, 2018 at 04:28:51PM +0100, Dave Page wrote: > On Tue, Jul 24, 2018 at 4:26 PM, Tomas Vondra > wrote: > > Clean room design addresses copyright-related issues, not patents. > > Correct. It's important folks realise that! Indeed. It's also important to know, when reading PG source

Re: How can we submit code patches that implement our (pending) patents?

2018-07-23 Thread Nico Williams
On Mon, Jul 23, 2018 at 12:12:03PM -0700, Joshua D. Drake wrote: > On 07/23/2018 12:06 PM, Bruce Momjian wrote: > >>So, is it FUD? The core needs paid-for legal advice, not speculation. > >> > >>I'm quite certain that a software license can make a patent grant to the > >>satisfaction of many open

Re: How can we submit code patches that implement our (pending) patents?

2018-07-23 Thread Nico Williams
On Mon, Jul 23, 2018 at 03:06:13PM -0400, Bruce Momjian wrote: > On Mon, Jul 23, 2018 at 02:02:40PM -0500, Nico Williams wrote: > > On Mon, Jul 23, 2018 at 01:12:19PM -0400, Bruce Momjian wrote: > > > On Mon, Jul 23, 2018 at 11:27:49AM -0500, Nico Williams wrote: > >

Re: How can we submit code patches that implement our (pending) patents?

2018-07-23 Thread Nico Williams
On Mon, Jul 23, 2018 at 01:12:49PM -0400, Bruce Momjian wrote: > On Mon, Jul 23, 2018 at 11:37:05AM -0500, Nico Williams wrote: > > On Mon, Jul 23, 2018 at 11:40:41AM -0400, Bruce Momjian wrote: > > > Sun Microsystems seemed reasonably trustworthy too. > > > > Ar

Re: How can we submit code patches that implement our (pending) patents?

2018-07-23 Thread Nico Williams
On Mon, Jul 23, 2018 at 01:12:19PM -0400, Bruce Momjian wrote: > On Mon, Jul 23, 2018 at 11:27:49AM -0500, Nico Williams wrote: > > Perhaps patent law [in some countries] requires contracts as opposed to > > licenses? > > Yes, I really don't know. I have just seen enough

Re: How can we submit code patches that implement our (pending) patents?

2018-07-23 Thread Nico Williams
On Mon, Jul 23, 2018 at 11:55:01AM -0400, Bruce Momjian wrote: > On Mon, Jul 23, 2018 at 11:40:41AM -0400, Bruce Momjian wrote: > > On Mon, Jul 23, 2018 at 08:19:35AM -0700, Andres Freund wrote: > > > I'm fairly sure that I'm right. But my point isn't that we should "trust > > > Andres implicitly

Re: How can we submit code patches that implement our (pending) patents?

2018-07-23 Thread Nico Williams
On Mon, Jul 23, 2018 at 11:40:41AM -0400, Bruce Momjian wrote: > On Mon, Jul 23, 2018 at 08:19:35AM -0700, Andres Freund wrote: > > I'm fairly sure that I'm right. But my point isn't that we should "trust > > Andres implicitly ™" (although that's obviously not a bad starting point > > ;)). But

Re: How can we submit code patches that implement our (pending) patents?

2018-07-23 Thread Nico Williams
On Mon, Jul 23, 2018 at 10:13:48AM -0400, Chapman Flack wrote: > On 07/23/2018 10:01 AM, Bruce Momjian wrote: > > > And the larger question is whether a patent free for use by software > > under any license can be used in a defensive way. If not, it means we > > have no way forward here. > >

Re: How can we submit code patches that implement our (pending) patents?

2018-07-23 Thread Nico Williams
On Mon, Jul 23, 2018 at 09:56:47AM -0400, Bruce Momjian wrote: > On Mon, Jul 23, 2018 at 06:31:14AM -0700, Andres Freund wrote: > > It explicitly says irrevocable and successors. Why seems squarely > > aimed at your concern. Bankruptcy wouldn't just invalidate that. > > They can say whatever they

Re: Add constraint in a Materialized View

2018-07-20 Thread Nico Williams
On Wed, Jul 18, 2018 at 09:28:19AM +0200, Kaye Ann Ignacio wrote: > I'm trying to add a foreign constraint in my local table to reference a > column in a materialized view, is it possible to alter this materialized > view by adding a primary key constraint? It's not, but I'm working on a patch

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-19 Thread Nico Williams
On Thu, Jul 19, 2018 at 01:46:12PM -0700, Andres Freund wrote: > On 2018-07-19 15:42:46 -0500, Nico Williams wrote: > > Yes, but that's in libc. None of that is in the PG code itself. > > That's simply entirely completely wrong. PG has a good chunk of memory > management la

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-19 Thread Nico Williams
On Thu, Jul 19, 2018 at 03:42:46PM -0500, Nico Williams wrote: > On Thu, Jul 19, 2018 at 01:38:52PM -0700, Andres Freund wrote: > > Uhm, this'd already require a fair bit of threadsafety. Like at least > > all of the memory allocator / context code. Nor is having threads > >

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-19 Thread Nico Williams
On Thu, Jul 19, 2018 at 01:38:52PM -0700, Andres Freund wrote: > On 2018-07-19 15:27:06 -0500, Nico Williams wrote: > > No, the other thread does NOT continue to do whatever -- it > > blocks/sleeps forever waiting for the coming exit(3). > > > > I.e.,

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-19 Thread Nico Williams
On Thu, Jul 19, 2018 at 01:35:02PM -0700, Andres Freund wrote: > On 2018-07-19 15:17:26 -0500, Nico Williams wrote: > > You can create that thread with a really small stack given that its only > > purpose is to do this error reporting and exit. > > You still have a full k

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-19 Thread Nico Williams
On Thu, Jul 19, 2018 at 01:10:14PM -0700, Andres Freund wrote: > On 2018-07-19 15:04:15 -0500, Nico Williams wrote: > > Besides making ereport() async-signal-safe, which is tricky, you could > > write(2) the arguments to a pipe that another thread in the same process > > is r

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-19 Thread Nico Williams
On Thu, Jul 19, 2018 at 04:04:01PM -0400, Tom Lane wrote: > Nico Williams writes: > > What I'd do is have a volatile sig_atomic_t in_signal_handler_context > > variable to indicate that we're dying, and then when that is non-zero, > > ereport() and friends could use

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-19 Thread Nico Williams
On Thu, Jun 22, 2017 at 03:10:31PM -0400, Tom Lane wrote: > Andres Freund writes: > > Or, probably more robust: Simply _exit(2) without further ado, and rely > > on postmaster to output an appropriate error message. Arguably it's not > > actually useful to see hundreds of "WARNING: terminating

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-19 Thread Nico Williams
On Thu, Jul 19, 2018 at 03:49:35PM -0400, Tom Lane wrote: > Andres Freund writes: > > On 2018-07-19 11:57:25 +0300, Heikki Linnakangas wrote: > >> The regular backend's quickdie() function is more tricky. It should also > >> call _exit(2) rather than exit(2). But it also tries to ereport a

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-19 Thread Nico Williams
On Thu, Jul 19, 2018 at 12:20:53PM -0700, Andres Freund wrote: > On 2018-07-19 11:57:25 +0300, Heikki Linnakangas wrote: > > Ugh. Yeah, in wal_quickdie, and other aux process *_quickdie() handlers, I > > agree we should just _exit(2). All we want to do is to exit the process > > immediately. > >

Re: How can we submit code patches that implement our (pending) patents?

2018-07-11 Thread Nico Williams
On Thu, Jul 12, 2018 at 09:33:21AM +0800, Craig Ringer wrote: > On 12 July 2018 at 09:10, Tsunakawa, Takayuki < > tsunakawa.ta...@jp.fujitsu.com> wrote: > > From: Nico Williams [mailto:n...@cryptonector.com] > > > On Thu, Jul 12, 2018 at 12:29:12AM +, Tsunakawa, Taka

Re: How can we submit code patches that implement our (pending) patents?

2018-07-11 Thread Nico Williams
On Thu, Jul 12, 2018 at 01:10:33AM +, Tsunakawa, Takayuki wrote: > From: Nico Williams [mailto:n...@cryptonector.com] > > On Thu, Jul 12, 2018 at 12:29:12AM +, Tsunakawa, Takayuki wrote: > > > How can one make defensive use of his patent if he allows everyone to > &g

Re: [HACKERS] [PATCH] WIP Add ALWAYS DEFERRED option for constraints

2018-07-11 Thread Nico Williams
Attached is an additional patch, as well as a new, rebased patch. This includes changes responsive to Álvaro Herrera's commentary about the SET CONSTRAINTS manual page. Nico -- >From e7838b60dbf0a8cd7f35591db2f9aab78d8903cb Mon Sep 17 00:00:00 2001 From: Nicolas Williams Date: Wed, 11 Jul 2018

Re: How can we submit code patches that implement our (pending) patents?

2018-07-11 Thread Nico Williams
On Thu, Jul 12, 2018 at 12:29:12AM +, Tsunakawa, Takayuki wrote: > From: Nico Williams [mailto:n...@cryptonector.com] > > My advice is to write up a patent grant that allows all to use the > > relevant patents royalty-free with a no-lawsuit covenant. I.e., make > > only

Re: [HACKERS] [PATCH] WIP Add ALWAYS DEFERRED option for constraints

2018-07-11 Thread Nico Williams
On Wed, Jul 11, 2018 at 03:13:30PM -0400, Alvaro Herrera wrote: > On 2018-Jun-06, Nico Williams wrote: > > I've finally gotten around to rebasing this patch and making the change > > that was requested, which was: merge the now-would-be-three deferral- > > related bo

Re: [HACKERS] [PATCH] WIP Add ALWAYS DEFERRED option for constraints

2018-07-11 Thread Nico Williams
On Tue, Jun 26, 2018 at 04:54:13PM -0400, Robbie Harwood wrote: > Nico Williams writes: > > > [Re-send; first attempt appears to have hit /dev/null somewhere. My > > apologies if you get two copies.] > > > > I've finally gotten around to rebasing th

Re: How can we submit code patches that implement our (pending) patents?

2018-07-11 Thread Nico Williams
On Wed, Jul 11, 2018 at 01:03:44AM +, Tsunakawa, Takayuki wrote: > From: Tom Lane [mailto:t...@sss.pgh.pa.us] > > The core team has considered this matter, and has concluded that it's > > time to establish a firm project policy that we will not accept any code > > that is known to be

Re: How can we submit code patches that implement our (pending) patents?

2018-07-11 Thread Nico Williams
On Tue, Jul 10, 2018 at 09:47:09AM -0400, Tom Lane wrote: > The core team has considered this matter, and has concluded that it's > time to establish a firm project policy that we will not accept any code > that is known to be patent-encumbered. The long-term legal risks and > complications

Re: How can we submit code patches that implement our (pending) patents?

2018-07-11 Thread Nico Williams
On Tue, Jul 10, 2018 at 08:20:53AM +, Tsunakawa, Takayuki wrote: > From: Nico Williams [mailto:n...@cryptonector.com] > > On Sat, Jul 07, 2018 at 10:20:35AM -0700, Andres Freund wrote: > > > It's entirely possible to dual license contributions and everything. Why &g

Re: How can we submit code patches that implement our (pending) patents?

2018-07-09 Thread Nico Williams
On Mon, Jul 09, 2018 at 08:29:08AM +, Tsunakawa, Takayuki wrote: > > There are arguments made that TPL (and BSD, MIT etc) already includes an > > implicit patent grant, but while a longstanding theory, it's to my > > knowledge not legally been tested. > > When we find a reasonable consensus

Re: How can we submit code patches that implement our (pending) patents?

2018-07-09 Thread Nico Williams
On Sat, Jul 07, 2018 at 10:20:35AM -0700, Andres Freund wrote: > It's entirely possible to dual license contributions and everything. Why > are you making such aggressive statements about a, so far, apparently > good faith engagement? One problem is that many contributors would not want to be

Re: How can we submit code patches that implement our (pending) patents?

2018-07-09 Thread Nico Williams
On Thu, Jul 05, 2018 at 01:15:15AM +, Tsunakawa, Takayuki wrote: > From: Craig Ringer [mailto:cr...@2ndquadrant.com] > > I'm assuming you don't want to offer a grant that lets anyone use them for > > anything. But if you have a really broad grant to PostgreSQL, all someone > > would have to do

Re: Threat models for DB cryptography (Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key) Management Service (KMS)

2018-07-03 Thread Nico Williams
On Tue, Jul 03, 2018 at 07:28:42PM +0900, Masahiko Sawada wrote: > On Tue, Jul 3, 2018 at 7:16 AM, Nico Williams wrote: > > Yes, but piecemeal encryption seems like a bad idea to me. > > What do you mean by "piecemeal encryption"? Is it not-whole database > encrypt

Re: Threat models for DB cryptography (Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key) Management Service (KMS)

2018-07-02 Thread Nico Williams
On Mon, Jul 02, 2018 at 06:22:46PM +0900, Masahiko Sawada wrote: > On Fri, Jun 22, 2018 at 2:31 PM, Tsunakawa, Takayuki > wrote: > > From: Nico Williams [mailto:n...@cryptonector.com] > > > >> One shortcoming of relying on OS functionality for protection aga

Re: [HACKERS] [PATCH] WIP Add ALWAYS DEFERRED option for constraints

2018-07-02 Thread Nico Williams
On Tue, Jun 26, 2018 at 04:54:13PM -0400, Robbie Harwood wrote: > Nico Williams writes: > > > [Re-send; first attempt appears to have hit /dev/null somewhere. My > > apologies if you get two copies.] > > > > I've finally gotten around to rebasing th

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2018-07-02 Thread Nico Williams
On Mon, Jul 02, 2018 at 06:56:34PM +0300, Alvaro Hernandez wrote: > On 21/06/18 21:43, Nico Williams wrote: > >Incidentally, PG w/ pgcrypto and FDW does provide everything one needs > >to be able to implement client-side crypto: > > > > - use PG w/ FDW as a client

Re: Using JSONB directly from application

2018-06-22 Thread Nico Williams
On Mon, Feb 26, 2018 at 09:36:52PM +0800, Craig Ringer wrote: > On 26 February 2018 at 04:05, Anthony Communier > wrote: > > It would be nice if application connected to a Postrgesql database could > > send and receive JSONB in binary. It could save some useless text > > conversion. All works

Re: Threat models for DB cryptography (Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key) Management Service (KMS)

2018-06-22 Thread Nico Williams
On Fri, Jun 22, 2018 at 05:31:44AM +, Tsunakawa, Takayuki wrote: > From: Nico Williams [mailto:n...@cryptonector.com] > > Let's start with a set of threat models then. I'll go first: > > Thank you so much for summarizing the current situation. I'd > appreciate it

Re: libpq compression

2018-06-22 Thread Nico Williams
On Fri, Jun 22, 2018 at 10:18:12AM +0300, Konstantin Knizhnik wrote: > On 22.06.2018 00:34, Nico Williams wrote: > >So I think you just have to have lengths. > > > >Now, this being about compression, I understand that you might now want > >to have 4-byte lengths,

Threat models for DB cryptography (Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key) Management Service (KMS)

2018-06-21 Thread Nico Williams
On Thu, Jun 21, 2018 at 07:46:35PM -0400, Bruce Momjian wrote: > Agreed. I can see from this discussion that we have a long way to go > before we can produce something clearly useful, but it will be worth it. Let's start with a set of threat models then. I'll go first: 1) storage devices as

Re: libpq compression

2018-06-21 Thread Nico Williams
On Thu, Jun 21, 2018 at 10:12:17AM +0300, Konstantin Knizhnik wrote: > On 20.06.2018 23:34, Robbie Harwood wrote: > >Konstantin Knizhnik writes: > >Well, that's a design decision you've made. You could put lengths on > >chunks that are sent out - then you'd know exactly how much is needed. >

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2018-06-21 Thread Nico Williams
On Fri, May 25, 2018 at 08:41:46PM +0900, Moon, Insung wrote: > Issues on data encryption of PostgreSQL > == > Currently, in PostgreSQL, data encryption can be using pgcrypto Tool. > However, it is inconvenient to use pgcrypto to encrypts data in some cases. > > There are two significant

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2018-06-21 Thread Nico Williams
On Thu, Jun 21, 2018 at 10:14:54AM -0400, Bruce Momjian wrote: > On Wed, Jun 20, 2018 at 04:57:18PM -0500, Nico Williams wrote: > > Client-side crypto is hard to do well and still get decent performance. > > So on the whole I think that crypto is a poor fit for the DBAs-are-the- &g

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2018-06-20 Thread Nico Williams
On Thu, Jun 21, 2018 at 10:05:41AM +0900, Masahiko Sawada wrote: > On Thu, Jun 21, 2018 at 6:57 AM, Nico Williams wrote: > > On Wed, Jun 20, 2018 at 05:16:46PM -0400, Bruce Momjian wrote: > >> On Mon, Jun 18, 2018 at 12:29:57PM -0500, Nico Williams wrote: > >> > N

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2018-06-20 Thread Nico Williams
On Wed, Jun 20, 2018 at 06:19:40PM -0400, Joe Conway wrote: > On 06/20/2018 05:12 PM, Bruce Momjian wrote: > > On Mon, Jun 18, 2018 at 11:06:20AM -0400, Joe Conway wrote: > > Even if they are encrypted with the same key, they use different > > initialization vectors that are stored inside the

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2018-06-20 Thread Nico Williams
On Wed, Jun 20, 2018 at 06:06:56PM -0400, Joe Conway wrote: > On 06/20/2018 05:09 PM, Bruce Momjian wrote: > > On Mon, Jun 18, 2018 at 09:49:20AM -0400, Robert Haas wrote: > >> know the ordering of the values under whatever ordering semantics > >> apply to that index. It's unclear to me how

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2018-06-20 Thread Nico Williams
On Wed, Jun 20, 2018 at 05:16:46PM -0400, Bruce Momjian wrote: > On Mon, Jun 18, 2018 at 12:29:57PM -0500, Nico Williams wrote: > > Note that unless the pg_catalog is protected against manipulation by > > remote storage, then TDE for user tables might be possible to > >

Re: Query Rewrite for Materialized Views (Postgres Extension)

2018-06-19 Thread Nico Williams
On Tue, Jun 19, 2018 at 08:46:06AM +0100, Dent John wrote: > I’m pretty impressed anything in this space can be written entirely in > PlPGQSL! https://github.com/twosigma/postgresql-contrib PG is quite powerful! I have even implemented a COMMIT TRIGGER in pure PlPgSQL. You'll notice I make

Re: Query Rewrite for Materialized Views (Postgres Extension)

2018-06-18 Thread Nico Williams
On Mon, Jun 18, 2018 at 07:38:13PM +0100, Dent John wrote: > I commented to Corey (privately) that, while my rewrite extension has > gotten me a server that responds quickly to aggregate queries, the > constant need to refresh the supporting MVs means the system’s load > average is constant and

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2018-06-18 Thread Nico Williams
On Mon, Jun 11, 2018 at 06:22:22PM +0900, Masahiko Sawada wrote: > As per discussion at PGCon unconference, I think that firstly we need > to discuss what threats we want to defend database data against. If We call that a threat model. There can be many threat models, of course. > user wants to

Re: ON CONFLICT DO NOTHING on pg_dump

2018-06-18 Thread Nico Williams
On Fri, Jun 15, 2018 at 02:20:21AM +, Ideriha, Takeshi wrote: > >From: Nico Williams [mailto:n...@cryptonector.com] > >On Tue, Jun 12, 2018 at 09:05:23AM +, Ideriha, Takeshi wrote: > >> Only the difference of data can be restored. > > > >But that's

Re: [PATCH v18] GSSAPI encryption support

2018-06-12 Thread Nico Williams
On Tue, Jun 12, 2018 at 12:36:01PM -0400, Robbie Harwood wrote: > Nico Williams writes: > > On Mon, Jun 11, 2018 at 04:11:10PM -0400, Robbie Harwood wrote: > >> Nico was kind enough to provide me with some code review. This should > >> those concerns (clarify short-rea

  1   2   >