Re: [HACKERS] list of credits for release notes

2017-10-02 Thread Bruce Momjian
release notes are created, or some other method? -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription + -- Sent via pgsql-h

Re: [HACKERS] list of credits for release notes

2017-10-02 Thread Bruce Momjian
hanged over time. > > Yes, I used the form that the person used in their emails. How should this be handled for the Postgres 11 release notes? -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am,

Re: [HACKERS] [BUGS] BUG #14825: enum type: unsafe use?

2017-09-26 Thread Bruce Momjian
On Tue, Sep 26, 2017 at 05:32:15PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Tue, Sep 26, 2017 at 04:07:02PM -0400, Tom Lane wrote: > >> Any other votes out there? > > > Well, I was concerned yesterday that we had a broken build farm so close > &

Re: [HACKERS] [BUGS] BUG #14825: enum type: unsafe use?

2017-09-26 Thread Bruce Momjian
roughly, and > there's no longer time to catch any remaining oversights through testing. > > Any other votes out there? Well, I was concerned yesterday that we had a broken build farm so close to release. (I got consistent regression failures.) I think PG 11 would be better fo

Re: [HACKERS] md5 still listed as an option in pg_hba.conf.sample

2017-09-26 Thread Bruce Momjian
", "peer", "pam", "ldap", "radius" or "cert". > > The "md5" option no longer works, as discussed in other threads. Uh, I think that "md5" still works just fine. -- Bruce Momjian http://momjian.us Enterpris

Re: [GENERAL] [HACKERS] USER Profiles for PostgreSQL

2017-09-19 Thread Bruce Momjian
onsidering for core, especially now that we've got a > reasonable password-based authentication method with SCRAM. Does LDAP do this too? -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I

Re: [HACKERS] PG 10 release notes

2017-09-19 Thread 'Bruce Momjian'
On Tue, Sep 19, 2017 at 12:30:01PM -0400, Tom Lane wrote: > "'Bruce Momjian'" writes: > > On Tue, Sep 19, 2017 at 12:22:39PM -0400, Tom Lane wrote: > >> We don't normally release-note documentation changes. If this > >> wasn't purely a d

Re: [HACKERS] PG 10 release notes

2017-09-19 Thread 'Bruce Momjian'
On Tue, Sep 19, 2017 at 12:22:39PM -0400, Tom Lane wrote: > "'Bruce Momjian'" writes: > > I am sure Tom can explain his reasoning. > > We don't normally release-note documentation changes. If this > wasn't purely a documentation change, then I was

Re: [HACKERS] PG 10 release notes

2017-09-19 Thread 'Bruce Momjian'
eemed to be in the wrong section. I am sure Tom can explain his reasoning. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription

Re: [HACKERS] Clarification in pg10's pgupgrade.html step 10 (upgrading standby servers)

2017-09-18 Thread Bruce Momjian
On Sat, Sep 16, 2017 at 11:36:40PM +0200, Andreas Joseph Krogh wrote: > På lørdag 16. september 2017 kl. 18:34:51, skrev Bruce Momjian < > br...@momjian.us>: > No.  If you ran initdb with --waldir on the new primary, you will create > a symbolic link in the PGDAT

Re: [HACKERS] PoC plpgsql - possibility to force custom or generic plan

2017-09-16 Thread Bruce Momjian
ument the behavior until last year: commit fab9d1da4a213fab08fe2d263eedf2408bc4a27a Author: Bruce Momjian Date: Tue Jun 14 16:11:46 2016 -0400 document when PREPARE uses generic plans Also explain how generic plans are created. L

Re: [HACKERS] Clarification in pg10's pgupgrade.html step 10 (upgrading standby servers)

2017-09-16 Thread Bruce Momjian
ot similar to the steps required > for copying tablespaces outside $PGDATA, so it's worth documenting explicitly. > Maybe also telling users to ensure the synlink (in $PGDATA) to pg_wal on > standby points to pg_wal. Why tell them new instructions when the rsync instructions work

Re: [HACKERS] Clarification in pg10's pgupgrade.html step 10 (upgrading standby servers)

2017-09-16 Thread Bruce Momjian
On Thu, Sep 14, 2017 at 03:12:50PM -0400, Bruce Momjian wrote: > I talked with Stephen about this on IM. The issue is that if you don't > do --delete, and there are files in the primary that are not in the > standby, they are copied, but files in the standby and not in the >

Re: [HACKERS] Clarification in pg10's pgupgrade.html step 10 (upgrading standby servers)

2017-09-16 Thread Bruce Momjian
ring them to use the same steps for WAL copy that they _must_ use for the data directory and tablespaces because of the links between old and new files there. Agreed? -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so on

Re: [HACKERS] Clarification in pg10's pgupgrade.html step 10 (upgrading standby servers)

2017-09-15 Thread Bruce Momjian
baseline requirements of pg_upgrade is to *not* screw > with the existing cluster. Removing its WAL or "cleaning it up" > definitely seems like it's violating that principle. > > I tend to agree that it'd be good for the documentation to address this, > but this is all

Re: [HACKERS] Clarification in pg10's pgupgrade.html step 10 (upgrading standby servers)

2017-09-15 Thread Bruce Momjian
d by rsync, but the directory it points _to_ will not be copied, so it will point to nothing. Of course, of both old and new clusters share the same WAL directory, which I think is impossible, things would get very confusing quickly. I will reply to this now in a later email. --

Re: [HACKERS] Clarification in pg10's pgupgrade.html step 10 (upgrading standby servers)

2017-09-14 Thread Bruce Momjian
rom the same tool, without > > access to PG, meaning we'd have to work through something else (such as > > SSH, like rsync does, but then what would we do for Windows...?). > > Maybe pg_rewind's mechanism could be partially reused for this as it > seems to

Re: [HACKERS] Clarification in pg10's pgupgrade.html step 10 (upgrading standby servers)

2017-09-14 Thread Bruce Momjian
nc --archive --delete --hard-links --size-only /vol1/postgres/9.6/pg_xlog \ > /vol1/postgres/10/pg_wal standby.example.com:/vol1/postgres/10/pg_wal Well, there is technically no need for version directories in pgdata either --- installers just create them. -- Bruce Momjian http://mom

Re: [HACKERS] Clarification in pg10's pgupgrade.html step 10 (upgrading standby servers)

2017-09-14 Thread Bruce Momjian
On Wed, Sep 13, 2017 at 12:16:33PM -0400, Stephen Frost wrote: > Bruce, > > * Bruce Momjian (br...@momjian.us) wrote: > > I have applied the attached patch to show examples of using rsync on > > PGDATA and tablespaces, documented that rsync is only useful when in > > li

Re: [HACKERS] Clarification in pg10's pgupgrade.html step 10 (upgrading standby servers)

2017-09-14 Thread Bruce Momjian
On Wed, Sep 13, 2017 at 04:31:09PM +0200, Andreas Joseph Krogh wrote: > På onsdag 13. september 2017 kl. 15:26:27, skrev Bruce Momjian < > br...@momjian.us>: > > On Wed, Sep 13, 2017 at 01:35:17AM +0200, Andreas Joseph Krogh wrote: > [snip] > > I know I&#x

Re: [HACKERS] Clarification in pg10's pgupgrade.html step 10 (upgrading standby servers)

2017-09-13 Thread Bruce Momjian
On Wed, Sep 13, 2017 at 01:35:17AM +0200, Andreas Joseph Krogh wrote: > På onsdag 13. september 2017 kl. 01:00:20, skrev Bruce Momjian < > br...@momjian.us>: > (I know this isn't exactly -hackers food, but it seems natural to end this > thread here) >   > Ok, thanks.

Re: [HACKERS] Clarification in pg10's pgupgrade.html step 10 (upgrading standby servers)

2017-09-12 Thread Bruce Momjian
other problem is that standby can't go into write mode or it would diverge from the primary. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Anci

Re: [HACKERS] Clarification in pg10's pgupgrade.html step 10 (upgrading standby servers)

2017-09-12 Thread Bruce Momjian
On Wed, Sep 13, 2017 at 12:40:32AM +0200, Andreas Joseph Krogh wrote: > På tirsdag 12. september 2017 kl. 23:52:02, skrev Bruce Momjian < > br...@momjian.us>: > > On Tue, Sep 12, 2017 at 08:59:05PM +0200, Andreas Joseph Krogh wrote: > >     Improvements? >

Re: [HACKERS] Clarification in pg10's pgupgrade.html step 10 (upgrading standby servers)

2017-09-12 Thread Bruce Momjian
complete > example, with absolute paths, would be clarifying. You mean a full rsync command, e.g.: rsync --archive --delete --hard-links --size-only \ /opt/PostgreSQL/9.5 /opt/PostgreSQL/9.6 standby:/opt/PostgreSQL Does that help? -- Bruce Momjian http://momjian.us EnterpriseD

Re: [HACKERS] Clarification in pg10's pgupgrade.html step 10 (upgrading standby servers)

2017-09-12 Thread Bruce Momjian
flow we used in > the products I work on is to re-create standbys from scratch after the > upgrade using a fresh backup, with a VM cloning. An upgrade here is an > in-place process not only linked to Postgres, so standby VMs are made > of many services, some are being linked to Postgre

Re: [HACKERS] PG 10 release notes

2017-09-11 Thread Bruce Momjian
On Fri, Sep 1, 2017 at 05:39:31PM +0900, Masahiko Sawada wrote: > Hi all, > > On Tue, Aug 1, 2017 at 5:53 AM, Thomas Munro > wrote: > > On Tue, Apr 25, 2017 at 1:31 PM, Bruce Momjian wrote: > >> I have committed the first draft of the Postgres 10 release notes. They

Re: [HACKERS] PG 10 release notes

2017-09-11 Thread Bruce Momjian
On Tue, Aug 1, 2017 at 08:53:51AM +1200, Thomas Munro wrote: > On Tue, Apr 25, 2017 at 1:31 PM, Bruce Momjian wrote: > > I have committed the first draft of the Postgres 10 release notes. They > > are current as of two days ago, and I will keep them current. Please > > giv

Re: [HACKERS] PG 10 release notes

2017-09-11 Thread Bruce Momjian
rt-hash-pgconfus-2017 ... > > > > Should we mention it ? I don't know, but I suggest you read this email thread from April to get an idea of how performance items are handled: https://www.postgresql.org/message-id/flat/20170425013144.GA7513%40momjian.us#20170425013144.g

Re: [HACKERS] PG 10 release notes

2017-09-11 Thread Bruce Momjian
On Mon, Sep 11, 2017 at 06:30:58PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Fri, Jun 2, 2017 at 04:05:44PM -0500, Jim Nasby wrote: > >> Can you change the attribution on > >> Allow PL/Tcl functions to return composite types and sets > >>

Re: [HACKERS] PG 10 release notes

2017-09-11 Thread Bruce Momjian
On Fri, Jun 2, 2017 at 04:05:44PM -0500, Jim Nasby wrote: > On 4/24/17 8:31 PM, Bruce Momjian wrote: > >I have committed the first draft of the Postgres 10 release notes. They > >are current as of two days ago, and I will keep them current. Please > >give me any feedback

Re: [HACKERS] GnuTLS support

2017-09-04 Thread Bruce Momjian
parate documentation > for openssl_ciphers, gnutls_ciphers, etc. rather than one giant > documentation section that tries to explain every implementation > separately. I am worried about having 3x version of TLS controls in postgresql.conf, and only one set being active. Perhaps we nee

Re: [HACKERS] obsolete code in pg_upgrade

2017-09-04 Thread Bruce Momjian
version) == 900 && > new_cluster.controldata.cat_ver < TABLE_SPACE_SUBDIRS_CAT_VER) > pg_fatal("This utility can only upgrade to PostgreSQL version > 9.0 after 2010-01-11\n" > "because of backend API changes made during > development

Re: [HACKERS] tupconvert.c API change in v10 release notes

2017-09-01 Thread Bruce Momjian
eak rather > than one that would easily be caught due to compiler errors. > Bruce, did you have any specific reasoning for leaving it out? I doubt I saw that sentence in the paragraph. For long text like that, I am usually looking for "BACKWARDS INCOMPATIBLE CHAN

Re: [HACKERS] Authentication mechanisms categorization

2017-08-31 Thread Bruce Momjian
. I think the problem is that we don't have a consistent secret shared between the client and the server. We have md5 and SCRAM, but that doesn't help because the secret it tied to the authentication methods. -- Bruce Momjian http://momjian.us EnterpriseDB

Re: [HACKERS] Revisiting NAMEDATALEN

2017-08-30 Thread Bruce Momjian
_, meaning multi-byte names are often less than 63 characters. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription + --

Re: [HACKERS] Challenges preventing us moving to 64 bit transaction id (XID)?

2017-08-28 Thread Bruce Momjian
be marked using our frozen-commited fixed xid so it doesn't matter what the xid epoch is? I realize with 64-bit xids we don't need to freeze tuples, but we could still use a frozen-commited fixed xid, see: #define FrozenTransactionId ((TransactionId) 2) -- Bruce Momjian

Re: [HACKERS] Broken hint bits (freeze)

2017-06-29 Thread Bruce Momjian
On Wed, Jun 28, 2017 at 10:11:35PM -0400, Bruce Momjian wrote: > On Fri, Jun 23, 2017 at 06:17:47PM +0300, Sergey Burladyan wrote: > > PS: > > I successfully upgraded last night from 9.2 to 9.4 and find other issue :-) > > > > It is about hash index and promote: > &g

Re: [HACKERS] Broken hint bits (freeze)

2017-06-28 Thread Bruce Momjian
ully. Sergey, can you please test if the table "test" is not unlogged, does pg_upgrade still fail on the hash index file? -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will b

Re: [HACKERS] Broken hint bits (freeze)

2017-06-28 Thread Bruce Momjian
how do we know they processed all the records in the second-to-last WAL file (in WAL shipping mode). -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + An

Re: [HACKERS] Broken hint bits (freeze)

2017-06-28 Thread Bruce Momjian
hen it is promoted to primary and someone tries to access the object. I wonder if I need to add a boolean to each object to record if it is unlogged, and allow copy/link to silently fail in such cases. Does that make sense? -- Bruce Momjian http://momjian.us EnterpriseDB

Re: [HACKERS] Broken hint bits (freeze)

2017-06-23 Thread Bruce Momjian
On Fri, Jun 23, 2017 at 08:10:17AM +0530, Amit Kapila wrote: > On Wed, Jun 21, 2017 at 10:03 PM, Bruce Momjian wrote: > > On Wed, Jun 21, 2017 at 07:49:21PM +0530, Amit Kapila wrote: > >> On Tue, Jun 20, 2017 at 7:24 PM, Amit Kapila > >> wrote: > >> > Hmm

Re: [HACKERS] Re-indent HEAD tomorrow?

2017-06-22 Thread Bruce Momjian
in backporting of their own patches, meaning the community and forks will have backbranch patch difficulties. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + +

Re: [HACKERS] Re-indent HEAD tomorrow?

2017-06-21 Thread Bruce Momjian
s done. You are eventually doing all active branches, right? -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription + -- Sent

Re: [HACKERS] Broken hint bits (freeze)

2017-06-21 Thread Bruce Momjian
ar as I know this is the only remaining open issue. Sergey, please verify. I appreciate the work everyone has done to improve this, and all the existing fixes have been pushed to all supported branches. :-) -- Bruce Momjian http://momjian.us EnterpriseDB

Re: [HACKERS] Shortened URLs for commit messages

2017-06-21 Thread Bruce Momjian
On Wed, Jun 21, 2017 at 11:11:57AM -0400, Alvaro Herrera wrote: > Bruce Momjian wrote: > > > Oh, here is a fixed version that requires an @ sign, which all message > > id's have: > > > > sed '/http/!s;^\(Discussion: *\)\(.*@.*\)$;\1https://postgr.e

Re: [HACKERS] Broken hint bits (freeze)

2017-06-20 Thread Bruce Momjian
ail reader. --- On Fri, Jun 16, 2017 at 10:57:33PM +0300, Sergey Burladyan wrote: > Bruce Momjian writes: > > On Fri, Jun 16, 2017 at 04:33:16AM +0300, Sergey Burladyan wrote: > > > Bruce Momjian writes:

Re: [HACKERS] Broken hint bits (freeze)

2017-06-20 Thread Bruce Momjian
On Tue, Jun 20, 2017 at 06:42:58PM +0300, Sergey Burladyan wrote: > Bruce Momjian writes: > > > On Tue, Jun 20, 2017 at 01:10:26PM +0300, Sergey Burladyan wrote: > > > Only if missing/changed files changed in size, because rsync run with > > > --size-only it does no

Re: [HACKERS] Broken hint bits (freeze)

2017-06-20 Thread Bruce Momjian
On Tue, Jun 20, 2017 at 01:10:26PM +0300, Sergey Burladyan wrote: > Bruce Momjian writes: > > > Uh, as I understand it the rsync is going to copy the missing WAL file > > > from the new master to the standby, right, and I think pg_controldata > > > too, so it shoul

Re: [HACKERS] Broken hint bits (freeze)

2017-06-19 Thread Bruce Momjian
On Mon, Jun 19, 2017 at 10:59:19PM -0400, Bruce Momjian wrote: > On Tue, Jun 20, 2017 at 03:50:29AM +0300, Sergey Burladyan wrote: > > 20 июн. 2017 г. 1:21 пользователь "Bruce Momjian" > > написал:  > > > > > > We are saying that L

Re: [HACKERS] Broken hint bits (freeze)

2017-06-19 Thread Bruce Momjian
On Tue, Jun 20, 2017 at 03:50:29AM +0300, Sergey Burladyan wrote: > 20 июн. 2017 г. 1:21 пользователь "Bruce Momjian" написал:  > > > We are saying that Log-Shipping should match "Latest checkpoint > location", but the WAL for that will not be sent t

Re: [HACKERS] Broken hint bits (freeze)

2017-06-19 Thread Bruce Momjian
tch if old standby servers were shut down before the old primary.) We are saying that Log-Shipping should match "Latest checkpoint location", but the WAL for that will not be sent to the standby, so it will not match, but that is OK since the only thing in the non-shipped W

Re: [HACKERS] Broken hint bits (freeze)

2017-06-16 Thread Bruce Momjian
On Fri, Jun 16, 2017 at 04:44:46PM -0400, Bruce Momjian wrote: > Yes, that is _exactly_ the right place to look. Only in PG 10 do we > restart the new cluster to invalidate hash indexes. In previous > releases we didn't do the restart. > > That didn't matter with the ol

Re: [HACKERS] Preliminary results for proposed new pgindent implementation

2017-06-16 Thread Bruce Momjian
do what it does now, there's circa ten thousand changed lines.) Yeah, entab was designed to do that, via this C comment: /* * Is the next character going to be a tab? We do tab * replacement in the current spot if the next char is * going to be a tab and

Re: [HACKERS] Preliminary results for proposed new pgindent implementation

2017-06-16 Thread Bruce Momjian
On Fri, Jun 16, 2017 at 11:54:06AM -0700, Andres Freund wrote: > On 2017-06-16 14:42:38 -0400, Bruce Momjian wrote: > > On Fri, Jun 16, 2017 at 02:23:00PM -0400, Tom Lane wrote: > > > Well, that's something we need to discuss. I originally argued for > > > back-

Re: [HACKERS] Preliminary results for proposed new pgindent implementation

2017-06-16 Thread Bruce Momjian
aintaining private patchsets. Can't they sync up to just before our pgindent commit and run pgindent on their own code base? -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you

Re: [HACKERS] Preliminary results for proposed new pgindent implementation

2017-06-16 Thread Bruce Momjian
. Yes, it is all about <80 column output. The current pgindent does everything possible to accomplish that --- the question is whether we want uglier code to do it. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you

Re: [HACKERS] Broken hint bits (freeze)

2017-06-16 Thread Bruce Momjian
On Fri, Jun 16, 2017 at 08:10:13PM +0530, Amit Kapila wrote: > On Fri, Jun 16, 2017 at 7:03 AM, Sergey Burladyan wrote: > > Bruce Momjian writes: > > > >> ! against the old primary and standby clusters. Verify that the > >> ! Latest checkpoint loca

Re: [HACKERS] Broken hint bits (freeze)

2017-06-16 Thread Bruce Momjian
On Fri, Jun 16, 2017 at 04:33:16AM +0300, Sergey Burladyan wrote: > Bruce Momjian writes: > > ! > > ! Also, if upgrading standby servers, change wal_level > > ! to replica in the postgresql.conf file on > > ! the new cluster. > > >

Re: [HACKERS] WIP: Data at rest encryption

2017-06-16 Thread Bruce Momjian
whether to encrypt temporary data or not. If we go forward with in-db encryption, I think we are going to have to have a discussion about what parts of PGDATA need to be encrypted, i.e., I don't think pg_clog needs encryption. -- Bruce Momjian http://momjian.us

Re: [HACKERS] WIP: Data at rest encryption

2017-06-16 Thread Bruce Momjian
On Thu, Jun 15, 2017 at 08:08:05PM -0400, Bruce Momjian wrote: > On Thu, Jun 15, 2017 at 04:56:36PM -0700, Andres Freund wrote: > > how few concerns about this feature's complexity / maintainability > > impact have been raised. > > Yeah, I guess we will just have to

Re: [HACKERS] Shortened URLs for commit messages

2017-06-16 Thread Bruce Momjian
On Thu, Jun 15, 2017 at 01:05:19PM -0400, Bruce Momjian wrote: > On Tue, May 23, 2017 at 11:25:07PM -0400, Bruce Momjian wrote: > > I have written the following sed script to convert regular Postgres > > email message URLs to their shorter form for commit messages: > > >

Re: [HACKERS] WIP: Data at rest encryption

2017-06-15 Thread Bruce Momjian
On Thu, Jun 15, 2017 at 04:56:36PM -0700, Andres Freund wrote: > On 2017-06-15 19:44:43 -0400, Bruce Momjian wrote: > > Understood, but now you are promoting a feature with an admittedly-poor > > API, duplication of an OS feature, and perhaps an invasive change to the > > co

Re: [HACKERS] WIP: Data at rest encryption

2017-06-15 Thread Bruce Momjian
On Thu, Jun 15, 2017 at 07:51:36PM -0400, Alvaro Herrera wrote: > Bruce Momjian wrote: > > On Thu, Jun 15, 2017 at 07:27:55PM -0400, Stephen Frost wrote: > > > I expect the same would happen with the shell-command approach suggested > > > up-thread and the prompt-on-stdi

Re: [HACKERS] WIP: Data at rest encryption

2017-06-15 Thread Bruce Momjian
ry > much. Understood, but now you are promoting a feature with an admittedly-poor API, duplication of an OS feature, and perhaps an invasive change to the code. Those are high hurdles. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprised

Re: [HACKERS] WIP: Data at rest encryption

2017-06-15 Thread Bruce Momjian
or SSL sucks but we > couldn't actually drop support for it, but we don't want to add other > features which work in a similar way because, well, it sucks. My point is that if our support for db-level encryption is as bad as SSL key passwords, then it will be nearly useless, so we mi

Re: [HACKERS] WIP: Data at rest encryption

2017-06-15 Thread Bruce Momjian
t make password use on SSH easier than the other cases --- it isn't just code quality. However, it would be good to research how SSH handles it to see if we can get any ideas. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + A

Re: [HACKERS] WIP: Data at rest encryption

2017-06-15 Thread Bruce Momjian
we see it. When SSH added password access, it was probably an easy decision because the use-case was high and the complexity was low. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you

Re: [HACKERS] Broken hint bits (freeze)

2017-06-15 Thread Bruce Momjian
On Thu, Jun 15, 2017 at 03:00:18PM +0530, Amit Kapila wrote: > On Wed, Jun 14, 2017 at 8:44 PM, Bruce Momjian wrote: > > On Wed, Jun 14, 2017 at 07:45:17PM +0530, Amit Kapila wrote: > >> > Now, it seems we later added a doc section early on that talks about > >> &

Re: [HACKERS] Shortened URLs for commit messages

2017-06-15 Thread Bruce Momjian
On Tue, May 23, 2017 at 11:25:07PM -0400, Bruce Momjian wrote: > I have written the following sed script to convert regular Postgres > email message URLs to their shorter form for commit messages: > > sed > 's;http\(s\?\)://www\.postgresql\.org/message-id/;http\1://postgr.es

Re: [HACKERS] WIP: Data at rest encryption

2017-06-14 Thread Bruce Momjian
On Wed, Jun 14, 2017 at 06:41:43PM +0300, Ants Aasma wrote: > On Wed, Jun 14, 2017 at 6:26 PM, Bruce Momjian wrote: > > Are you checking the CPU type or if AES instructions are enabled on the > > CPU? I ask this because I just realized in researching my new TLS talk > > tha

Re: [HACKERS] WIP: Data at rest encryption

2017-06-14 Thread Bruce Momjian
encryption, every > > bit in the page depends on every other bit in the page, so we > > certainly would. I don't know how many places we've got assumptions > > like this baked into the system, but I'm guessing there are a bunch. > > I think we need to

Re: [HACKERS] Broken hint bits (freeze)

2017-06-14 Thread Bruce Momjian
uated removing it in the first place. (Yeah, I am amazed I considered all these cases.) Updated patch attached. Thanks. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. +

Re: pgindent (was Re: [HACKERS] [COMMITTERS] pgsql: Preventive maintenance in advance of pgindent run.)

2017-06-14 Thread Bruce Momjian
bump so that pgindent can tell that it's got an > up-to-date copy. 1.4? 2.0? For Piotr's reference, we will update src/tools/pgindent/pgindent to match whatever new version number you use. -- Bruce Momjian http://momjian.us EnterpriseDB ht

Re: [HACKERS] WIP: Data at rest encryption

2017-06-14 Thread Bruce Momjian
I don't think that would work for NAS/NFS. > > Also, has anyone asked users if they would find db-encryption better > > than file system encryption? > > I've been asked for this capability multiple times from our users and > have generally pushed back and encouraged fil

Re: [HACKERS] WIP: Data at rest encryption

2017-06-14 Thread Bruce Momjian
of > > specific organizations by changing PostgreSQL core in very invasive way > > (keeping in mind testing, maintaining, etc) is misguided. > > In the same time implementing a plugable storage API and then implementing > encrypted / compressed / whatever storage in a standalone e

Re: pgindent (was Re: [HACKERS] [COMMITTERS] pgsql: Preventive maintenance in advance of pgindent run.)

2017-06-13 Thread Bruce Momjian
On Tue, Jun 13, 2017 at 05:00:31PM -0400, Tom Lane wrote: > Anyway, it is now time to fish or cut bait. I don't think we can wait > much longer to decide whether we're going to adopt this new indent > version for PG 10. I think we should. The floor is open for votes. Works f

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
On Tue, Jun 13, 2017 at 04:08:29PM -0400, Peter Eisentraut wrote: > On 6/13/17 15:51, Bruce Momjian wrote: > > Isn't the leakage controlled by OS permissions, so is it really leakage, > > i.e., if you can see the leakage, you probably have bypassed the OS > > permission

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
On Tue, Jun 13, 2017 at 03:20:12PM -0400, Stephen Frost wrote: > Bruce, > > * Bruce Momjian (br...@momjian.us) wrote: > > On Tue, Jun 13, 2017 at 02:38:58PM -0400, Stephen Frost wrote: > > > It's good to discuss what the feature would bring and what cases it &g

Re: [HACKERS] Broken hint bits (freeze)

2017-06-13 Thread Bruce Momjian
o that block, which should be safe. There is now no need to start/stop the new server since pg_upgrade will do that safely already. I plan to patch this back to 9.5 where these instructions were added. I will mention that this should be in the minor release notes. -- Bruce Momjian http:

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
this time, but if someone has a good idea for an API and we can make it work easily while adding this feature, why wouldn't we do that? -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so yo

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
t this feature brings, and doesn't bring, and how it will deal with future feature additions, then you are right we will never have it. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am,

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
ption features, do we? And we are not going to know that if we don't talk about it, but hey, this is just an email thread and I can marshal opposition to the feature later when it appears, and point this all out again. -- Bruce Momjian http://momjian.us EnterpriseDB

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
tion, but the above strikes me as > the big ones, and those are certainly valuable enough on their own for > us to seriously consider adding this capability. Since you seem to be trying to shut down discussion, I will simply say I am unimpressed that this use-case is sufficient jus

Re: [HACKERS] v10beta pg_catalog diagrams

2017-06-13 Thread Bruce Momjian
atalog_constraints.html > > Go for it. Yeah, great. We have been talking about adding diagrams to our official docs but needed an updated toolchain, which I think we now have, so there is a lot of opportunity for growth here. -- Bruce Momjian http://momjian.us EnterpriseDB

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
he storage device that is encrypted. My big question is how many times are the OS permissions bypassed in a way that would also not expose the db clusters key or db data? -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you ar

Re: [HACKERS] Transactional sequence stuff breaks pg_upgrade

2017-06-13 Thread Bruce Momjian
pgrade renumbered everything, you would no longer be able to do > that, or at least not easily. FYI, pg_upgrade is designed to go boom if something doesn't look right because it can't anticipate what changes might be made to Postgres in the future. boom == feature! -- Bruce Momj

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
opies of the symetric key, one for each role you want to view the data. And good point on the ability to change the RSA key/password without having to reencrypt the data. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you ar

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
to encrypt/decrypt the data. I was thinking of a case where you encrypt a row using a symetric key, then store RSA-encrypted versions of the symetric key encrypted that only specific users could decrypt and get the key to decrypt the data. -- Bruce Momjian http://momjian.us E

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
, including all-cluster encryption. One cool idea I have is using public encryption to store the encryption key by users who don't know the decryption key, e.g. RSA. It would be a write-only encryption option. Not sure how useful that is, but it easily possible, and doesn't require us

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
ecessary true. You are describing a cipher mode where the user data goes through the cipher, e.g. AES in CBC mode. However, if you are using a stream cipher based on a block cipher, e.g. CTR, GCM, you XOR the user data with a random bit stream, and in that case one bit chan

Re: pgindent (was Re: [HACKERS] [COMMITTERS] pgsql: Preventive maintenance in advance of pgindent run.)

2017-06-12 Thread Bruce Momjian
an't easily change? I have reviewed the C formatting options a few times over the years and every time the other options were worse than what we had. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once

Re: [HACKERS] PG10 transition tables, wCTEs and multiple operations on the same table

2017-06-08 Thread Bruce Momjian
oes not want to *risk* unduly holding up the release. > > That was my understanding, at least. I think we can all agree that Kevin should have communicated this earlier, rather than requiring Robert to push him on the issue. -- Bruce Momjian http://momjian.us EnterpriseDB

Re: [HACKERS] PG 10 release notes

2017-06-07 Thread Bruce Momjian
On Wed, Jun 7, 2017 at 03:18:49PM +1000, Neha Khatri wrote: > > On Mon, May 15, 2017 at 12:45 PM, Bruce Momjian wrote: > > On Thu, May 11, 2017 at 11:50:03PM -0400, Tom Lane wrote: > > Michael Paquier writes: > > > Bruce, the release notes do not m

Re: [HACKERS] Extra Vietnamese unaccent rules

2017-06-06 Thread Bruce Momjian
On Tue, Jun 6, 2017 at 12:15:13PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > There seems to be a problem. I can't see a patch dated 2017-06-07 on > > the commitfest page: > > https://commitfest.postgresql.org/14/1161/ > > It looks to me like the pa

Re: [HACKERS] Extra Vietnamese unaccent rules

2017-06-06 Thread Bruce Momjian
On Wed, Jun 7, 2017 at 01:06:22AM +0900, Man Trieu wrote: > 2017-06-07 0:31 GMT+09:00 Bruce Momjian : > I added the thread but there was no change.  (I think the thread was > already present.)  It appears it is not seeing this patch as the latest > patch. > > Do

Re: [HACKERS] Extra Vietnamese unaccent rules

2017-06-06 Thread Bruce Momjian
On Wed, Jun 7, 2017 at 12:10:25AM +0900, Dang Minh Huong wrote: > > On Jun 4, 29 Heisei, at 00:48, Bruce Momjian wrote: > >>>> Shouldn't you use "or is_letter_with_marks()", instead of "or len(...) > >>>>> 1"? Your test might cat

Re: [HACKERS] Challenges preventing us moving to 64 bit transaction id (XID)?

2017-06-06 Thread Bruce Momjian
On Tue, Jun 6, 2017 at 09:05:03AM -0400, Peter Eisentraut wrote: > On 6/6/17 08:29, Bruce Momjian wrote: > > On Tue, Jun 6, 2017 at 06:00:54PM +0800, Craig Ringer wrote: > >> Tom's point is, I think, that we'll want to stay pg_upgrade > >> compatible. So whe

Re: [HACKERS] Use of non-restart-safe storage by temp_tablespaces

2017-06-06 Thread Bruce Momjian
logged > table in a permanent tablespace (probably the default one for the > database). > > FWIW I don't think calling these tablespaces "temporary" is the right > word. It's not the tablespaces that are temporary. Maybe "evane

Re: [HACKERS] Challenges preventing us moving to 64 bit transaction id (XID)?

2017-06-06 Thread Bruce Momjian
do we > do? I guess I am missing something. If you see an old page version number, you know none of the tuples are from running transactions so you can just freeze them all, after consulting the pg_clog. What am I missing? If the page is full, why are you trying to add to the page? -- Br

Re: [HACKERS] Use of non-restart-safe storage by temp_tablespaces

2017-06-05 Thread Bruce Momjian
On Mon, Jun 5, 2017 at 07:38:43PM -0700, Andres Freund wrote: > On 2017-06-05 22:34:17 -0400, Bruce Momjian wrote: > > On Mon, Jun 5, 2017 at 04:38:32PM -0500, Jerry Sievers wrote: > > > The SAN snaps capture the entire pgdata and WAL pg_xlog area but there > > > is

  1   2   3   4   5   6   7   8   9   10   >