Re: [HACKERS] PG10 partitioning - odd behavior/possible bug

On 11/28/2017 04:40 PM, Robert Haas wrote: > On Sun, Sep 3, 2017 at 5:28 PM, Joe Conway <m...@joeconway.com> wrote: >> I was playing around with partitioning and found an oddity that is best >> described with the following reasonably minimal test case: > > I

Re: has_sequence_privilege() never got the memo

On 11/23/2017 07:16 AM, Peter Eisentraut wrote: > On 11/22/17 22:58, Tom Lane wrote: >> Joe Conway <m...@joeconway.com> writes: >>> I just noticed that has_sequence_privilege() never got the memo about >>> "WITH GRANT OPTION". Any objections to the attach

assert in nested SQL procedure call in current HEAD

My colleague Yogesh Sharma discovered an assert in nested SQL procedure calls after ROLLBACK is used. Minimal test case and backtrace below. I have not yet tried to figure out exactly what is going on beyond seeing that it occurs in pg_plan_query() where the comment says "Planner must have a

Re: Redesigning the executor (async, JIT, memory efficiency)

On 05/24/2018 11:26 PM, Heikki Linnakangas wrote: > Cool stuff! > > On 25/05/18 06:35, Andres Freund wrote: >> For example, this converts this converts TPCH's Q01: +1 Wicked cool! -- Crunchy Data - http://crunchydata.com PostgreSQL Support for Secure Enterprises Consulting, Training, & Open

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 06/18/2018 09:49 AM, Robert Haas wrote: > On Wed, Jun 13, 2018 at 9:20 AM, Joe Conway wrote: >>> Also, if I understand correctly, at unconference session there also >>> were two suggestions about the design other than the suggestion by >>> Alexander: implemen

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 06/18/2018 10:26 AM, Robert Haas wrote: > On Mon, Jun 18, 2018 at 10:12 AM, Joe Conway wrote: >> Not necessarily. Our pages probably have enough predictable bytes to aid >> cryptanalysis, compared to user data in a column which might not be very >> predicable. >

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 06/18/2018 10:52 AM, Tom Lane wrote: > Robert Haas writes: >> On Mon, Jun 18, 2018 at 10:12 AM, Joe Conway wrote: >>> Not necessarily. Our pages probably have enough predictable bytes to aid >>> cryptanalysis, compared to user data in a column which might

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 06/11/2018 05:22 AM, Masahiko Sawada wrote: > As per discussion at PGCon unconference, I think that firstly we need > to discuss what threats we want to defend database data against. Exactly. While certainly there is demand for encryption for the sake of "checking a box", different designs

Re: missing toast table for pg_policy

On 06/15/2018 02:40 PM, John Naylor wrote: > On 2/19/18, Joe Conway wrote: >> The attached does exactly this. Gives all system tables toast tables >> except pg_class, pg_attribute, and pg_index, and includes cat version >> bump and regression test in misc_sanity. >>

Re: New committers announced at PGCon 2018

On 06/01/2018 05:14 PM, Andres Freund wrote: > On 2018-06-01 17:05:11 -0400, Tom Lane wrote: >> The core team is pleased to announce the appointment of seven >> new Postgres committers: >> >> Etsuro Fujita >> Peter Geoghegan >> Amit Kapila >> Alexander Korotkov >> Thomas Munro >> Michael Paquier

Re: commitfest 2018-07

On 06/05/2018 10:43 AM, Andres Freund wrote: > On 2018-06-05 10:20:55 -0400, Tom Lane wrote: >> Andres Freund writes: >>> I'd rather create a new 2018-07, and just manually move old patches to >>> it. Otherwise we'll not really focus on the glut of old things, but >>> everyone just restarts

Re: Add CONTRIBUTING.md

On 05/29/2018 11:38 AM, Magnus Hagander wrote: > On Tue, May 29, 2018 at 5:36 PM, Andres Freund > wrote: > > Hi, > > A lot of people contribute in communities via github these days. We > should add a CONTRIBUTING.md that explains how to do so, given that

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 06/20/2018 05:05 PM, Bruce Momjian wrote: > On Mon, Jun 18, 2018 at 08:29:32AM -0400, Joe Conway wrote: >>>> Or >>>> maybe key management is really tied into the separately discussed effort >>>> to create SQL VARIABLEs somehow. >>> >>> C

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 06/20/2018 05:03 PM, Bruce Momjian wrote: > On Wed, Jun 13, 2018 at 09:20:58AM -0400, Joe Conway wrote: >> The idea has not been extensively fleshed out yet, but the thought was >> that we create column level POLICY, which would transparently apply some >> kind

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 06/20/2018 05:09 PM, Bruce Momjian wrote: > On Mon, Jun 18, 2018 at 09:49:20AM -0400, Robert Haas wrote: >> know the ordering of the values under whatever ordering semantics >> apply to that index. It's unclear to me how useful such information > > I don't think an ordered index is possible,

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 06/20/2018 05:12 PM, Bruce Momjian wrote: > On Mon, Jun 18, 2018 at 11:06:20AM -0400, Joe Conway wrote: >>> At the same time, having to have a bunch of independently-decipherable >>> short field values is not real secure either, especially if they're known >>> to

Re: [HACKERS] Re: [HACKERS] generated columns

On 12/27/2017 09:31 AM, Peter Eisentraut wrote: > On 9/12/17 15:35, Jaime Casanova wrote: >> On 10 September 2017 at 00:08, Jaime Casanova >> wrote: >>> >>> During my own tests, though, i found some problems: > > Here is an updated patch that should address the

Re: [HACKERS] Re: [HACKERS] generated columns

On 12/31/2017 09:38 AM, Peter Eisentraut wrote: > On 12/30/17 16:04, Joe Conway wrote: >> + >> + The generation expression can refer to other columns in the table, but >> + not other generated columns. Any functions and operators used must be >> + immutable.

Re: A space-efficient, user-friendly way to store categorical data

On 02/11/2018 10:06 PM, Thomas Munro wrote: > On Mon, Feb 12, 2018 at 12:24 PM, Andrew Dunstan > wrote: >> On Mon, Feb 12, 2018 at 9:10 AM, Tom Lane wrote: >>> Andrew Kane writes: A better option could be a new

Re: missing toast table for pg_policy

On 02/16/2018 05:24 PM, Tom Lane wrote: > Joe Conway <m...@joeconway.com> writes: >> On 02/16/2018 05:07 PM, Andres Freund wrote: >>> If problematic for < master users I think you'll have to restart cluster >>> with allow_system_table_mods, manually create/drop

Re: SHA-2 functions

On 02/19/2018 08:43 AM, Peter Eisentraut wrote: > I also noticed while working on some SSL code that we have perfectly > good SHA-2 functionality in the server already, but it has no test > coverage outside the SCRAM tests. > > So I suggest these patches that expose the new functions sha224(), >

Re: missing toast table for pg_policy

On 02/16/2018 05:07 PM, Andres Freund wrote: > Hi, > > On 2018-02-16 16:56:15 -0500, Joe Conway wrote: >> Looking at the issue, the problem seems to be missing toast table for >> pg_policy. Also attached is a one line patch. It isn't clear to me >> whether this is a

Re: missing toast table for pg_policy

On 02/18/2018 11:18 AM, Tom Lane wrote: > Joe Conway <m...@joeconway.com> writes: >> Is there really a compelling reason to not just create toast tables for >> all system catalogs as in the attached? > > What happens when you VACUUM FULL pg_class? (The associat

Re: missing toast table for pg_policy

On 02/17/2018 11:39 AM, Tom Lane wrote: > Joe Conway <m...@joeconway.com> writes: >> Yes, exactly. I'm fine with not backpatching, just wanted to raise the >> possibility. I will push later today to HEAD (with a catalog version bump). > > BTW, I was wondering if

Re: missing toast table for pg_policy

On 07/09/2018 09:16 PM, Michael Paquier wrote: > On Mon, Jul 09, 2018 at 02:45:26PM +0200, Peter Eisentraut wrote: >> On 15.06.18 21:15, Joe Conway wrote: >>> Not surprising -- thanks for the update. >>> >>>> It occurred to be that we could go fu

Re: postgres_fdw: perform UPDATE/DELETE .. RETURNING on a join directly

On 03/05/2018 11:19 AM, Tom Lane wrote: > Joe, I wonder if you could add "log_autovacuum_min_duration = 0" to > rhinoceros' extra_config options, temporarily? Correlating that log > output with the log_statement output from the test proper would let > us confirm or deny whether it's autovacuum.

Re: public schema default ACL

On 03/03/2018 01:56 AM, Noah Misch wrote: > Commit 5770172 ("Document security implications of search_path and the public > schema.") is largely a workaround for the fact that the boot_val of > search_path contains "public" while template0 gets "GRANT CREATE, USAGE ON > SCHEMA public TO PUBLIC".

Re: postgres_fdw: perform UPDATE/DELETE .. RETURNING on a join directly

On 03/05/2018 02:07 PM, Tom Lane wrote: > So you can revert the rhinoceros config change if you like --- thanks > for making it so quickly! Ok, reverted. > Meanwhile, I'm back to wondering what could possibly have affected > the planner's estimates, if pg_proc and pg_statistic didn't change. > I

Re: Commit fest 2018-09

On 10/02/2018 10:13 AM, Tom Lane wrote: > Michael Paquier writes: >> Thanks to all who participated in the patch review, authoring, and >> everybody else who helped in making the different patches move forward. > > Thanks for being CFM! I know it's a lot of work ... +10! Joe -- Crunchy Data

Re: has_column_privilege behavior (was Re: Assert failed in snprintf.c)

On 10/01/2018 02:41 PM, Stephen Frost wrote: > Tom Lane (t...@sss.pgh.pa.us) wrote: >> But it's not quite clear to me what we want the behavior for bad column >> name to be. A case could be made for either of: >> >> * If either the table OID is bad, or the OID is OK but there's no such >>

Re: doc - add missing documentation for "acldefault"

On 09/19/2018 10:54 AM, Tom Lane wrote: > Joe Conway writes: >> * I do believe aclitemeq() has utility outside internal purposes. > > Our normal policy is that we do not document functions that are meant to > be invoked through operators. The \df output saying that is suffic

Re: doc - add missing documentation for "acldefault"

On 09/24/2018 10:01 AM, Tom Lane wrote: > Joe Conway writes: >> Having seen none, committed/pushed. This did not seem worth >> back-patching, so I only pushed to master. > > I don't see anything on gitmaster? Hmm, yes, interesting -- I must of messed up my local git repo so

Re: doc - add missing documentation for "acldefault"

On 09/21/2018 01:51 PM, Joe Conway wrote: > On 09/19/2018 11:18 AM, Joe Conway wrote: >> On 09/19/2018 10:54 AM, Tom Lane wrote: >>> So maybe what we really need is a table of operators not functions. >> >> Good idea -- I will take a look at that. >> >&g

Re: doc - add missing documentation for "acldefault"

On 09/24/2018 10:09 AM, Joe Conway wrote: > On 09/24/2018 10:01 AM, Tom Lane wrote: >> Joe Conway writes: >>> Having seen none, committed/pushed. This did not seem worth >>> back-patching, so I only pushed to master. >> >> I don't see anything on gitmaste

Re: doc - add missing documentation for "acldefault"

On 08/03/2018 09:04 AM, Fabien COELHO wrote: > Here is a version of the patch which documents briefly all aclitem-related > functions, in a separate table. I claimed this patch for review and commit. Comments: --- * There is a comment in src/backend/utils/adt/acl.c noting that acldefault is

Re: doc - add missing documentation for "acldefault"

On 09/19/2018 12:30 PM, John Naylor wrote: > On 9/19/18, Tom Lane wrote: >> However, I don't object to documenting any function that has its >> own pg_description string. > > Speaking of, that description string seems to have been neglected. > I've attached a remedy for that. Thanks, will take

Re: doc - add missing documentation for "acldefault"

On 09/19/2018 11:18 AM, Joe Conway wrote: > On 09/19/2018 10:54 AM, Tom Lane wrote: >> Joe Conway writes: >>> * I do believe aclitemeq() has utility outside internal purposes. >> >> Our normal policy is that we do not document functions that are meant to >> be

Re: pg_config wrongly marked as not parallel safe?

On 11/30/18 3:30 AM, Kyotaro HORIGUCHI wrote: > # And returning to the topic, I vote for pg_config should be "stable". And on that note, Does this change does warrant backpatching, or should be applied to master only? Joe -- Crunchy Data - http://crunchydata.com PostgreSQL Support for Secure

Re: pg_config wrongly marked as not parallel safe?

On 11/26/18 7:08 PM, Andres Freund wrote: > On 2018-11-26 19:04:46 -0500, Joe Conway wrote: >> Not intentional. Though, sitting here chatting with Stephen about it, I >> am now wondering if pg_config() should actually be marked immutable: >> >> select * from pg_con

Re: pg_config wrongly marked as not parallel safe?

On 11/26/18 6:45 PM, Andres Freund wrote: > Hi, > > Triggered by the thread at [1] I looked for functions marked as > immutable but not parallel safe. > > postgres[19492][1]=# SELECT oid::regprocedure, provolatile, proparallel FROM > pg_proc WHERE provolatile = 'i' AND proparallel != 's'; >

Re: PostgreSQL pollutes the file system

On 3/29/19 11:44 AM, Daniel Gustafsson wrote: > On Friday, March 29, 2019 4:41 PM, Tom Lane wrote: > >> Christoph Berg m...@debian.org writes: >> >> > What might possibly make sense is to add options to psql to >> > facilitate common tasks: >> >> > psql --createdb foo >> > psql --createuser bar

Re: [PATCH v20] GSSAPI encryption support

On 4/2/19 6:18 PM, Stephen Frost wrote: > Greetings, > > On Tue, Apr 2, 2019 at 18:10 Peter Eisentraut > > wrote: > > On 2019-02-23 17:27, Stephen Frost wrote: > >> About pg_hba.conf: The "hostgss" keyword seems a bit confusing.  > It only >

Re: PostgreSQL pollutes the file system

On 3/29/19 3:43 PM, Christoph Berg wrote: > Re: Joe Conway 2019-03-29 <48e5efaf-7ea2-ed70-a803-949bbfec8...@joeconway.com> >> echo "\password :\"role\"" | psql -v role='my role' >> Enter new password: >> Enter it again: >> >&

Re: PostgreSQL pollutes the file system

On 3/29/19 3:01 PM, Pavel Stehule wrote: > But psql has safe escaping via :"xxx" notation. So some like > > psql -c 'create role :"role"' -v role='my role' ... > > But what I know the psql variables are not evaluated for -c query You can do this: echo "create role :\"role\"" | psql -v role='my

Re: crosstab/repivot...any interest?

gt; Orphan in pgxn.  If there's some interest here, we can consider a new > contrib extension (which I personally rate very unlikely) or recasting > as an extra routine to tablefunc.  Any way we slice it, huge thanks to > Joe Conway for giving us such an awesome function to work with all &g

Re: get_controlfile() can leak fds in the backend

On 2/27/19 7:54 PM, Michael Paquier wrote: > On Wed, Feb 27, 2019 at 07:45:11PM -0500, Joe Conway wrote: >> It seems to me that OpenTransientFile() is more appropriate. Patch done >> that way attached. > > Works for me, thanks for sending a patch! While on it, could you &

Re: get_controlfile() can leak fds in the backend

On 2/27/19 2:47 AM, Michael Paquier wrote: > Hi all, > (CC-ing Joe as of dc7d70e) > > I was just looking at the offline checksum patch, and noticed some > sloppy coding in controldata_utils.c. The control file gets opened in > get_controlfile(), and if it generates an error then the file >

Re: Row Level Security − leakproof-ness and performance implications

On 2/28/19 11:50 AM, Robert Haas wrote: > On Thu, Feb 28, 2019 at 11:44 AM Joe Conway wrote: >> No, and Tom stated as much too, but life is all about tradeoffs. Some >> people will find this an acceptable compromise. For those that don't >> they don't have to use it. IMHO we

Re: Row Level Security − leakproof-ness and performance implications

On 2/28/19 12:28 PM, Robert Haas wrote: > Mmmph. If your customers always have a non-production instance where > problems from production can be easily reproduced, your customers are > not much like our customers. Well I certainly did not mean to imply that this is always the case ;-) But I

Re: Row Level Security − leakproof-ness and performance implications

On 2/28/19 11:03 AM, Joshua Brindle wrote: > On Thu, Feb 28, 2019 at 10:49 AM Tom Lane wrote: >> >> Joshua Brindle writes: >> > On Thu, Feb 28, 2019 at 9:12 AM Robert Haas wrote: >> >> So... you're just going to replace ALL error messages of any kind with >> >> "ERROR: missing error text" when

Re: Row Level Security − leakproof-ness and performance implications

On 2/28/19 9:12 AM, Robert Haas wrote: > On Wed, Feb 27, 2019 at 6:03 PM Joe Conway wrote: >> Patch for discussion attached. > > So... you're just going to replace ALL error messages of any kind with > "ERROR: missing error text" when this option is enabled? Tha

Re: Row Level Security − leakproof-ness and performance implications

On 2/28/19 11:37 AM, Robert Haas wrote: > On Thu, Feb 28, 2019 at 11:14 AM Joe Conway wrote: >> > Although, and Joe may hate me for saying this, I think only the >> > non-constants should be redacted to keep some level of usability for >> > regular SQL errors. Mayb

Re: get_controlfile() can leak fds in the backend

On 2/28/19 7:20 AM, Michael Paquier wrote: > On Thu, Feb 28, 2019 at 07:11:04AM -0500, Joe Conway wrote: >> Sure, will do. What are your thoughts on backpatching? This seems >> unlikely to be a practical concern in the field, so my inclination is a >> master only fix. > &

Re: Tighten error control for OpenTransientFile/CloseTransientFile

On 2/28/19 9:33 PM, Michael Paquier wrote: > Hi all, > > Joe's message here has reminded me that we have lacked a lot of error > handling around CloseTransientFile(): > https://www.postgresql.org/message-id/c49b69ec-e2f7-ff33-4f17-0eaa4f2ce...@joeconway.com > > This has been mentioned by Alvaro

Re: Row Level Security − leakproof-ness and performance implications

On 2/19/19 6:43 PM, Laurenz Albe wrote: > Pierre Ducroquet wrote: >> if we had a 'RLS-enabled' context on functions, a way to make a lot of built- >> in functions leakproof would simply be to prevent them from logging errors >> containing values. >> >> For others, like arraycontains, it's much

Re: list append syntax for postgresql.conf

On 2/20/19 10:15 AM, Peter Eisentraut wrote: > Nowadays there are a number of methods for composing multiple > postgresql.conf files for modularity. But if you have a bunch of things > you want to load via shared_preload_libraries, you have to put them all > in one setting. How about some kind

Re: BUG #15646: Inconsistent behavior for current_setting/set_config

On 2/20/19 12:11 PM, Tom Lane wrote: > Joe Conway writes: >> On 2/20/19 11:10 AM, PG Bug reporting form wrote: >>> But current behavior returns empty string instead of NULL (the initial >>> value) after transaction is rolled back. When I restart session,

oddity with ALTER ROLE/USER

I noticed that ALTER ROLE/USER succeeds even when called without any options: postgres=# alter user foo; ALTER ROLE postgres=# alter role foo; ALTER ROLE postgres=# alter group foo; ERROR: syntax error at or near ";" LINE 1: alter group foo; That seems odd, does nothing useful, and is

Re: oddity with ALTER ROLE/USER

On 2/22/19 4:19 PM, Tom Lane wrote: > Joe Conway writes: >> I noticed that ALTER ROLE/USER succeeds even when called without any >> options: > >> postgres=# alter user foo; >> ALTER ROLE >> postgres=# alter role foo; >> ALTER ROLE >> postgres=# alt

Re: Should we increase the default vacuum_cost_limit?

On 2/25/19 1:17 AM, Peter Geoghegan wrote: > On Sun, Feb 24, 2019 at 9:42 PM David Rowley > wrote: >> The current default vacuum_cost_limit of 200 seems to be 15 years old >> and was added in f425b605f4e. >> >> Any supporters for raising the default? > > I also think that the current default

Re: get_controlfile() can leak fds in the backend

On 2/27/19 10:26 AM, Joe Conway wrote: > On 2/27/19 2:47 AM, Michael Paquier wrote: >> Hi all, >> (CC-ing Joe as of dc7d70e) > According to that comment BasicOpenFile does not seem to solve the issue > you are pointing out (leaking of file descriptor on ERROR). Perhap

Re: Row Level Security − leakproof-ness and performance implications

On 2/20/19 11:24 AM, Tom Lane wrote: > Pierre Ducroquet writes: >> For simple functions like enum_eq/enum_ne, marking them leakproof is an >> obvious fix (patch attached to this email, including also textin/textout). > > This is not nearly as "obvious" as you think. See prior discussions, >

Re: Row Level Security − leakproof-ness and performance implications

On 3/18/19 3:52 PM, Peter Eisentraut wrote: > On 2019-02-28 00:03, Joe Conway wrote: >> What if we provided an option to redact all client messages (leaving >> logged messages as-is). Separately we could provide a GUC to force all >> functions to be resolved as leakpro

Re: pg_config wrongly marked as not parallel safe?

On 11/30/18 10:32 AM, Tom Lane wrote: > Joe Conway writes: >> On 11/30/18 3:30 AM, Kyotaro HORIGUCHI wrote: >>> # And returning to the topic, I vote for pg_config should be "stable". > >> And on that note, Does this change does warrant backpatching, or should

Re: Some thoughts on NFS

On 2/19/19 10:59 AM, Magnus Hagander wrote: > On Tue, Feb 19, 2019 at 4:46 PM Robert Haas > wrote: > > On Tue, Feb 19, 2019 at 2:03 AM Thomas Munro > wrote: > > How can we achieve that, without writing our > > own NFS

Re: Should the docs have a warning about pg_stat_reset()?

On 4/13/19 3:42 PM, Tomas Vondra wrote: > If only we had a way to regularly snapshot the data from within the > database, and then compute the deltas on that. If only we could insert > data from one table into another one a then do some analysics on it, > with like small windows moving over the

Re: New committer: David Rowley

On 5/30/19 11:43 AM, Andres Freund wrote: > Hi, > > On 2019-05-30 11:39:23 -0400, Magnus Hagander wrote: >> For those of you that have not read the minutes from the developer meeting >> ahead of pgcon (can be found at >> https://wiki.postgresql.org/wiki/PgCon_2019_Developer_Meeting), we'd like >>

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 6/13/19 11:07 AM, Bruce Momjian wrote: > On Thu, Jun 13, 2019 at 04:26:47PM +0900, Masahiko Sawada wrote: >> Yeah, in principle since data key of 2 tier key architecture should >> not go outside database I think we should not tell data keys to >> utility commands. So the rearranging WAL format

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 6/14/19 6:09 PM, Bruce Momjian wrote: > On Fri, Jun 14, 2019 at 02:27:17PM -0400, Joe Conway wrote: >> On 6/13/19 11:07 AM, Bruce Momjian wrote: >> > In addition, while the 8k blocks would use a block cipher, the WAL would >> > likely use a stream cipher, and

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 6/16/19 9:45 AM, Bruce Momjian wrote: > On Sun, Jun 16, 2019 at 07:07:20AM -0400, Joe Conway wrote: >> In any case it doesn't address my first point, which is limiting the >> volume encrypted with the same key. Another valid reason is you might >> have data at varyi

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 6/16/19 9:46 AM, Bruce Momjian wrote: > On Sun, Jun 16, 2019 at 09:45:09AM -0400, Bruce Momjian wrote: >> On Sun, Jun 16, 2019 at 07:07:20AM -0400, Joe Conway wrote: >> > And although I'm not proposing this for the first implementation, yet >> > another reason is I

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 6/17/19 8:12 AM, Stephen Frost wrote: >> But there's about 0% chance we'll get that in v1, of course, so we need >> s "minimum viable product" to build on anyway. > > There seems like a whole lot of space between something very elaborate > and only supporting one key. I think this is exactly

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 6/17/19 8:29 AM, Masahiko Sawada wrote: > From perspective of cryptographic, I think the fine grained TDE would > be better solution. Therefore if we eventually want the fine grained > TDE I wonder if it might be better to develop the table/tablespace TDE > first while keeping it simple as

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 6/20/19 8:34 AM, Masahiko Sawada wrote: > I think even if we provide the per table encryption we can have > encryption keys per tablepaces. That is, all tables on the same > tablespace encryption use the same encryption keys but user can > control encrypted objects per tables. > >> Will we add

Re: initdb recommendations

On 5/24/19 8:13 AM, Stephen Frost wrote: > Greetings, > > * Joe Conway (m...@joeconway.com) wrote: >> On 5/23/19 10:30 PM, Stephen Frost wrote: >> > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> >> "Jonathan S. Katz" writes: >> >> > For

Re: Excessive memory usage in multi-statement queries w/ partitioning

On 5/24/19 1:47 AM, Amit Langote wrote: > On 2019/05/23 4:15, Andreas Seltenreich wrote: >> …but when doing it on the parent relation, even 100 statements are >> enough to exceed the limit: >> >> , >> | $ psql -c "$(yes update t set c=c where c=6 \; | head -n 100)" >> | FEHLER: Speicher

Re: initdb recommendations

On 5/23/19 10:30 PM, Stephen Frost wrote: > Greetings, > > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> "Jonathan S. Katz" writes: >> > For now I have left in the password based method to be scram-sha-256 as >> > I am optimistic about the support across client drivers[1] (and FWIW I >> > have an

Re: initdb recommendations

On 5/24/19 8:56 AM, Jonathan S. Katz wrote: > On 5/24/19 8:33 AM, Stephen Frost wrote: >> * Magnus Hagander (mag...@hagander.net) wrote: >>> Making the default change away from trust in the source distro will affect >>> few people. >> >> Agreed. > > +1 Fewer people, but likely

Re: Excessive memory usage in multi-statement queries w/ partitioning

On 5/24/19 9:33 AM, David Rowley wrote: > On Sat, 25 May 2019 at 00:18, Joe Conway wrote: >> I admittedly haven't followed this thread too closely, but if having 100 >> partitions causes out of memory on pg11, that sounds like a massive >> regression to me. > > For it

Re: Excessive memory usage in multi-statement queries w/ partitioning

On 5/24/19 10:28 AM, Tom Lane wrote: > Joe Conway writes: >> On 5/24/19 9:33 AM, David Rowley wrote: >>> For it to have regressed it would have had to once have been better, >>> but where was that mentioned? The only thing I saw was >>> non-partitioned t

stawidth inconsistency with all NULL columns

Consider: CREATE TABLE testwid ( txtnotnull text, txtnull text, int8notnull int8, int8null int8 ); INSERT INTO testwid SELECT 'a' || g.i, NULL, g.i, NULL FROM generate_series(1,1) AS g(i); ANALYZE testwid; SELECT attname, avg_width FROM pg_stats WHERE tablename =

Re: stawidth inconsistency with all NULL columns

On 5/21/19 3:55 PM, Tom Lane wrote: > Joe Conway writes: >> else if (null_cnt > 0) >> { >> /* We found only nulls; assume the column is entirely null */ >> stats->stats_valid = true; >> stats->stanullfrac = 1.0; >> if (is_varwidth

Re: How to install login_hook in Postgres 10.5

On 5/13/19 8:32 PM, Michael Paquier wrote: > On Mon, May 13, 2019 at 01:06:10PM -0700, legrand legrand wrote: >> that finished commited >> "pgsql: Add hooks for session start and session end" >>

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 7/10/19 4:24 AM, Antonin Houska wrote: > Joe Conway wrote: > >> On 7/8/19 6:04 PM, Stephen Frost wrote: >> > * Bruce Momjian (br...@momjian.us) wrote: >> >> Uh, well, renaming the user was a big problem, but that is the only case >> >> I can thin

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 7/10/19 8:34 AM, Stephen Frost wrote: > Greetings, > > * Joe Conway (m...@joeconway.com) wrote: >> On 7/9/19 7:28 PM, Stephen Frost wrote: >> > * Joe Conway (m...@joeconway.com) wrote: >> >> On 7/9/19 5:42 PM, Tomas Vondra wrote: >> >> > Th

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 7/10/19 4:47 AM, Antonin Houska wrote: > Tomas Vondra wrote: >> I don't think that works, because that'd mean we're encrypting the same >> page with the same nonce over and over, which means reusing the reuse >> (even if you hash/encrypt it). Or did I miss something? > > I found out that it's

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 7/10/19 2:45 AM, Masahiko Sawada wrote: > On Wed, Jul 10, 2019 at 11:06 AM Stephen Frost wrote: >> >> Greetings, >> >> * Ryan Lambert (r...@rustprooflabs.com) wrote: >> > > What I think Tomas is getting at here is that we don't write a page only >> > > once. >> > >> > > A nonce of

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 7/10/19 3:53 PM, Alvaro Herrera wrote: > On 2019-Jul-10, Bruce Momjian wrote: > >> Good, so I think we all now agree we have to put the nonce >> (pg_class.oid, LSN, page-number) though the cipher using the secret. (been traveling -- just trying to get caught up on this thread) > Actually,

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 7/11/19 6:37 PM, Bruce Momjian wrote: > On Wed, Jul 10, 2019 at 12:26:24PM -0400, Bruce Momjian wrote: >> On Wed, Jul 10, 2019 at 08:31:17AM -0400, Joe Conway wrote: >>> Please see my other reply (and >>> https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublicat

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 7/11/19 9:05 PM, Bruce Momjian wrote: > On Thu, Jul 11, 2019 at 08:41:52PM -0400, Joe Conway wrote: >> On 7/11/19 6:37 PM, Bruce Momjian wrote: >> > Our first implementation will encrypt the entire cluster. We can later >> > consider encryption per table or

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 7/12/19 2:45 PM, Bruce Momjian wrote: > On Fri, Jul 12, 2019 at 12:41:19PM -0600, Ryan Lambert wrote: >> >> I vote for AES 256 rather than 128. >> > >> > Why?  This page seems to think 128 is sufficient: >> > >> >         https://crypto.stackexchange.com/questions/20/ >>

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 7/13/19 9:38 AM, Joe Conway wrote: > On 7/11/19 9:05 PM, Bruce Momjian wrote: >> On Thu, Jul 11, 2019 at 08:41:52PM -0400, Joe Conway wrote: >>> On 7/11/19 6:37 PM, Bruce Momjian wrote: >>> > Our first implementation will encrypt the entire cluster. We can later

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 7/13/19 2:41 PM, Joe Conway wrote: > [2] > https://www.postgresql.org/message-id/20190708194733.cztnwhqge4acepzw%40development BTW I managed to mess up this link. This is what I intended to link there (from Tomas): [2] https://www.kernel.org/doc/html/latest/filesystems/fscrypt.html I a

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 7/8/19 6:04 PM, Stephen Frost wrote: > * Bruce Momjian (br...@momjian.us) wrote: >> Uh, well, renaming the user was a big problem, but that is the only case >> I can think of. I don't see that as an issue for block or WAL sequence >> numbers. If we want to use a different nonce, we have to

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 7/9/19 8:39 AM, Ryan Lambert wrote: > Hi Thomas, > >> CBC mode does require >> random nonces, other modes may be fine with even sequences as long as >> the values are not reused.    > > I disagree that CBC mode requires random nonces, at least based on what > NIST has published.  They only

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 7/9/19 4:34 AM, Tomas Vondra wrote: > On Mon, Jul 08, 2019 at 06:45:50PM -0400, Bruce Momjian wrote: >>On Mon, Jul 8, 2019 at 06:23:13PM -0400, Bruce Momjian wrote: >>> Yes, 'postgres' can be used to create a nice md5 rainbow table that >>> works on many servers --- good point. Are rainbow

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 7/9/19 4:23 AM, Tomas Vondra wrote: > On Mon, Jul 08, 2019 at 06:24:40PM -0400, Joe Conway wrote: >>On 7/8/19 6:04 PM, Stephen Frost wrote: >>> * Bruce Momjian (br...@momjian.us) wrote: >>>> Uh, well, renaming the user was a big problem, but that is the only case

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 7/9/19 6:07 AM, Peter Eisentraut wrote: > On 2019-07-08 18:09, Joe Conway wrote: >> In my mind, and in practice to a >> large extent, a postgres tablespace == a unique mount point. > > But a critical difference is that in file systems, a separate mount > point has

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 7/8/19 11:56 AM, Peter Eisentraut wrote: > On 2019-07-08 17:47, Stephen Frost wrote: >> Of course, we can discuss if what websites do with over-the-wire >> encryption is sensible to compare to what we want to do in PG for >> data-at-rest, but then we shouldn't be talking about what websites do,

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On 7/8/19 10:19 AM, Bruce Momjian wrote: > When people are asking for multiple keys (not just for key rotation), > they are asking to have multiple keys that can be supplied by users only > when they need to access the data. Yes, the keys are always in the > datbase, but the feature request is

  1   2   3   4   >