Re: [HACKERS] list of credits for release notes

2017-10-02 Thread Bruce Momjian
tes are created, or some other method? -- Bruce Momjian <br...@momjian.us>http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription + -- Sent

Re: [HACKERS] list of credits for release notes

2017-10-02 Thread Bruce Momjian
bit trickier, as his email name has changed over time. > > Yes, I used the form that the person used in their emails. How should this be handled for the Postgres 11 release notes? -- Bruce Momjian <br...@momjian.us>http://momjian.us EnterpriseDB

Re: [HACKERS] [BUGS] BUG #14825: enum type: unsafe use?

2017-09-26 Thread Bruce Momjian
On Tue, Sep 26, 2017 at 05:32:15PM -0400, Tom Lane wrote: > Bruce Momjian <br...@momjian.us> writes: > > On Tue, Sep 26, 2017 at 04:07:02PM -0400, Tom Lane wrote: > >> Any other votes out there? > > > Well, I was concerned yesterday that we had a broken build far

Re: [HACKERS] [BUGS] BUG #14825: enum type: unsafe use?

2017-09-26 Thread Bruce Momjian
hly, and > there's no longer time to catch any remaining oversights through testing. > > Any other votes out there? Well, I was concerned yesterday that we had a broken build farm so close to release. (I got consistent regression failures.) I think PG 11 would be better for this fe

Re: [HACKERS] md5 still listed as an option in pg_hba.conf.sample

2017-09-26 Thread Bruce Momjian
dent", "peer", "pam", "ldap", "radius" or "cert". > > The "md5" option no longer works, as discussed in other threads. Uh, I think that "md5" still works just fine. -- Bruce Momjian <br...@momjian.us>http:

Re: [GENERAL] [HACKERS] USER Profiles for PostgreSQL

2017-09-19 Thread Bruce Momjian
> we should be considering for core, especially now that we've got a > reasonable password-based authentication method with SCRAM. Does LDAP do this too? -- Bruce Momjian <br...@momjian.us>http://momjian.us EnterpriseDB http://enterprisedb.com +

Re: [HACKERS] PG 10 release notes

2017-09-19 Thread 'Bruce Momjian'
On Tue, Sep 19, 2017 at 12:30:01PM -0400, Tom Lane wrote: > "'Bruce Momjian'" <br...@momjian.us> writes: > > On Tue, Sep 19, 2017 at 12:22:39PM -0400, Tom Lane wrote: > >> We don't normally release-note documentation changes. If this > >> wasn't purely

Re: [HACKERS] PG 10 release notes

2017-09-19 Thread 'Bruce Momjian'
On Tue, Sep 19, 2017 at 12:22:39PM -0400, Tom Lane wrote: > "'Bruce Momjian'" <br...@momjian.us> writes: > > I am sure Tom can explain his reasoning. > > We don't normally release-note documentation changes. If this > wasn't purely a documentation change, then I

Re: [HACKERS] PG 10 release notes

2017-09-19 Thread 'Bruce Momjian'
items that seemed to be in the wrong section. I am sure Tom can explain his reasoning. -- Bruce Momjian <br...@momjian.us>http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + +

Re: [HACKERS] Clarification in pg10's pgupgrade.html step 10 (upgrading standby servers)

2017-09-18 Thread Bruce Momjian
On Sat, Sep 16, 2017 at 11:36:40PM +0200, Andreas Joseph Krogh wrote: > På lørdag 16. september 2017 kl. 18:34:51, skrev Bruce Momjian < > br...@momjian.us>: > No.  If you ran initdb with --waldir on the new primary, you will create > a symbolic link in th

Re: [HACKERS] PoC plpgsql - possibility to force custom or generic plan

2017-09-16 Thread Bruce Momjian
year: commit fab9d1da4a213fab08fe2d263eedf2408bc4a27a Author: Bruce Momjian <br...@momjian.us> Date: Tue Jun 14 16:11:46 2016 -0400 document when PREPARE uses generic plans Also explain how generic plans are created. Link to PREPARE docs from

Re: [HACKERS] Clarification in pg10's pgupgrade.html step 10 (upgrading standby servers)

2017-09-16 Thread Bruce Momjian
equired > for copying tablespaces outside $PGDATA, so it's worth documenting explicitly. > Maybe also telling users to ensure the synlink (in $PGDATA) to pg_wal on > standby points to pg_wal. Why tell them new instructions when the rsync instructions work fine? What is the value? -- Br

Re: [HACKERS] Clarification in pg10's pgupgrade.html step 10 (upgrading standby servers)

2017-09-16 Thread Bruce Momjian
On Thu, Sep 14, 2017 at 03:12:50PM -0400, Bruce Momjian wrote: > I talked with Stephen about this on IM. The issue is that if you don't > do --delete, and there are files in the primary that are not in the > standby, they are copied, but files in the standby and not in the > prim

Re: [HACKERS] Clarification in pg10's pgupgrade.html step 10 (upgrading standby servers)

2017-09-16 Thread Bruce Momjian
same steps for WAL copy that they _must_ use for the data directory and tablespaces because of the links between old and new files there. Agreed? -- Bruce Momjian <br...@momjian.us>http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so onc

Re: [HACKERS] Clarification in pg10's pgupgrade.html step 10 (upgrading standby servers)

2017-09-15 Thread Bruce Momjian
f the baseline requirements of pg_upgrade is to *not* screw > with the existing cluster. Removing its WAL or "cleaning it up" > definitely seems like it's violating that principle. > > I tend to agree that it'd be good for the documentation to address this, > but this is all real

Re: [HACKERS] Clarification in pg10's pgupgrade.html step 10 (upgrading standby servers)

2017-09-15 Thread Bruce Momjian
ut the directory it points _to_ will not be copied, so it will point to nothing. Of course, of both old and new clusters share the same WAL directory, which I think is impossible, things would get very confusing quickly. I will reply to this now in a later email. -- Bruce Momjian <br...@mo

Re: [HACKERS] Clarification in pg10's pgupgrade.html step 10 (upgrading standby servers)

2017-09-14 Thread Bruce Momjian
t > > access to PG, meaning we'd have to work through something else (such as > > SSH, like rsync does, but then what would we do for Windows...?). > > Maybe pg_rewind's mechanism could be partially reused for this as it > seems to accomplish something vaguely similar A

Re: [HACKERS] Clarification in pg10's pgupgrade.html step 10 (upgrading standby servers)

2017-09-14 Thread Bruce Momjian
inks --size-only /vol1/postgres/9.6/pg_xlog \ > /vol1/postgres/10/pg_wal standby.example.com:/vol1/postgres/10/pg_wal Well, there is technically no need for version directories in pgdata either --- installers just create them. -- Bruce Momjian <br...@momjian.us>http://momjian.u

Re: [HACKERS] Clarification in pg10's pgupgrade.html step 10 (upgrading standby servers)

2017-09-14 Thread Bruce Momjian
On Wed, Sep 13, 2017 at 12:16:33PM -0400, Stephen Frost wrote: > Bruce, > > * Bruce Momjian (br...@momjian.us) wrote: > > I have applied the attached patch to show examples of using rsync on > > PGDATA and tablespaces, documented that rsync is only useful when in > > li

Re: [HACKERS] Clarification in pg10's pgupgrade.html step 10 (upgrading standby servers)

2017-09-14 Thread Bruce Momjian
On Wed, Sep 13, 2017 at 04:31:09PM +0200, Andreas Joseph Krogh wrote: > På onsdag 13. september 2017 kl. 15:26:27, skrev Bruce Momjian < > br...@momjian.us>: > > On Wed, Sep 13, 2017 at 01:35:17AM +0200, Andreas Joseph Krogh wrote: > [snip] > > I know I'm

Re: [HACKERS] Clarification in pg10's pgupgrade.html step 10 (upgrading standby servers)

2017-09-13 Thread Bruce Momjian
On Wed, Sep 13, 2017 at 01:35:17AM +0200, Andreas Joseph Krogh wrote: > På onsdag 13. september 2017 kl. 01:00:20, skrev Bruce Momjian < > br...@momjian.us>: > (I know this isn't exactly -hackers food, but it seems natural to end this > thread here) >   > Ok, thanks. >

Re: [HACKERS] Clarification in pg10's pgupgrade.html step 10 (upgrading standby servers)

2017-09-12 Thread Bruce Momjian
er problem is that standby can't go into write mode or it would diverge from the primary. -- Bruce Momjian <br...@momjian.us>http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + +

Re: [HACKERS] Clarification in pg10's pgupgrade.html step 10 (upgrading standby servers)

2017-09-12 Thread Bruce Momjian
On Wed, Sep 13, 2017 at 12:40:32AM +0200, Andreas Joseph Krogh wrote: > På tirsdag 12. september 2017 kl. 23:52:02, skrev Bruce Momjian < > br...@momjian.us>: > > On Tue, Sep 12, 2017 at 08:59:05PM +0200, Andreas Joseph Krogh wrote: > >     Improvements

Re: [HACKERS] Clarification in pg10's pgupgrade.html step 10 (upgrading standby servers)

2017-09-12 Thread Bruce Momjian
te > example, with absolute paths, would be clarifying. You mean a full rsync command, e.g.: rsync --archive --delete --hard-links --size-only \ /opt/PostgreSQL/9.5 /opt/PostgreSQL/9.6 standby:/opt/PostgreSQL Does that help? -- Bruce Momjian <br...@momjian.us>http://momjian.u

Re: [HACKERS] Clarification in pg10's pgupgrade.html step 10 (upgrading standby servers)

2017-09-12 Thread Bruce Momjian
e old standbys. Like Robert, the flow we used in > the products I work on is to re-create standbys from scratch after the > upgrade using a fresh backup, with a VM cloning. An upgrade here is an > in-place process not only linked to Postgres, so standby VMs are made > of many services, some are

Re: [HACKERS] PG 10 release notes

2017-09-11 Thread Bruce Momjian
On Fri, Sep 1, 2017 at 05:39:31PM +0900, Masahiko Sawada wrote: > Hi all, > > On Tue, Aug 1, 2017 at 5:53 AM, Thomas Munro > <thomas.mu...@enterprisedb.com> wrote: > > On Tue, Apr 25, 2017 at 1:31 PM, Bruce Momjian <br...@momjian.us> wrote: > >> I have co

Re: [HACKERS] PG 10 release notes

2017-09-11 Thread Bruce Momjian
On Tue, Aug 1, 2017 at 08:53:51AM +1200, Thomas Munro wrote: > On Tue, Apr 25, 2017 at 1:31 PM, Bruce Momjian <br...@momjian.us> wrote: > > I have committed the first draft of the Postgres 10 release notes. They > > are current as of two days ago, and I will keep them curr

Re: [HACKERS] PG 10 release notes

2017-09-11 Thread Bruce Momjian
kerdeck.com/peterg/sort-hash-pgconfus-2017 ... > > > > Should we mention it ? I don't know, but I suggest you read this email thread from April to get an idea of how performance items are handled: https://www.postgresql.org/message-id/flat/20170425013144.GA7513%

Re: [HACKERS] PG 10 release notes

2017-09-11 Thread Bruce Momjian
On Mon, Sep 11, 2017 at 06:30:58PM -0400, Tom Lane wrote: > Bruce Momjian <br...@momjian.us> writes: > > On Fri, Jun 2, 2017 at 04:05:44PM -0500, Jim Nasby wrote: > >> Can you change the attribution on > >> Allow PL/Tcl functions to return composite types

Re: [HACKERS] PG 10 release notes

2017-09-11 Thread Bruce Momjian
On Fri, Jun 2, 2017 at 04:05:44PM -0500, Jim Nasby wrote: > On 4/24/17 8:31 PM, Bruce Momjian wrote: > >I have committed the first draft of the Postgres 10 release notes. They > >are current as of two days ago, and I will keep them current. Please > >give me

Re: [HACKERS] GnuTLS support

2017-09-04 Thread Bruce Momjian
for openssl_ciphers, gnutls_ciphers, etc. rather than one giant > documentation section that tries to explain every implementation > separately. I am worried about having 3x version of TLS controls in postgresql.conf, and only one set being active. Perhaps we need to break out t

Re: [HACKERS] obsolete code in pg_upgrade

2017-09-04 Thread Bruce Momjian
version) == 900 && > new_cluster.controldata.cat_ver < TABLE_SPACE_SUBDIRS_CAT_VER) > pg_fatal("This utility can only upgrade to PostgreSQL version > 9.0 after 2010-01-11\n" > "because of backend API changes made during > development

Re: [HACKERS] tupconvert.c API change in v10 release notes

2017-09-01 Thread Bruce Momjian
ther > than one that would easily be caught due to compiler errors. > Bruce, did you have any specific reasoning for leaving it out? I doubt I saw that sentence in the paragraph. For long text like that, I am usually looking for "BACKWARDS INCOMPATIBLE CHANGE" or something like

Re: [HACKERS] Authentication mechanisms categorization

2017-08-31 Thread Bruce Momjian
problem is that we don't have a consistent secret shared between the client and the server. We have md5 and SCRAM, but that doesn't help because the secret it tied to the authentication methods. -- Bruce Momjian <br...@momjian.us>http://momjian.us EnterpriseDB

Re: [HACKERS] Revisiting NAMEDATALEN

2017-08-30 Thread Bruce Momjian
i-byte names are often less than 63 characters. -- Bruce Momjian <br...@momjian.us>http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscripti

Re: [HACKERS] Challenges preventing us moving to 64 bit transaction id (XID)?

2017-08-28 Thread Bruce Momjian
using our frozen-commited fixed xid so it doesn't matter what the xid epoch is? I realize with 64-bit xids we don't need to freeze tuples, but we could still use a frozen-commited fixed xid, see: #define FrozenTransactionId ((TransactionId) 2) -- Bruce Momjian <br...@momj

Re: [HACKERS] Broken hint bits (freeze)

2017-06-29 Thread Bruce Momjian
On Wed, Jun 28, 2017 at 10:11:35PM -0400, Bruce Momjian wrote: > On Fri, Jun 23, 2017 at 06:17:47PM +0300, Sergey Burladyan wrote: > > PS: > > I successfully upgraded last night from 9.2 to 9.4 and find other issue :-) > > > > It is about hash index and promote: > &g

Re: [HACKERS] Broken hint bits (freeze)

2017-06-28 Thread Bruce Momjian
ully. Sergey, can you please test if the table "test" is not unlogged, does pg_upgrade still fail on the hash index file? -- Bruce Momjian <br...@momjian.us>http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was

Re: [HACKERS] Broken hint bits (freeze)

2017-06-28 Thread Bruce Momjian
s, but how do we know they processed all the records in the second-to-last WAL file (in WAL shipping mode). -- Bruce Momjian <br...@momjian.us>http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be

Re: [HACKERS] Broken hint bits (freeze)

2017-06-28 Thread Bruce Momjian
ted to primary and someone tries to access the object. I wonder if I need to add a boolean to each object to record if it is unlogged, and allow copy/link to silently fail in such cases. Does that make sense? -- Bruce Momjian <br...@momjian.us>http://momjian.us Enterpris

Re: [HACKERS] Broken hint bits (freeze)

2017-06-23 Thread Bruce Momjian
On Fri, Jun 23, 2017 at 08:10:17AM +0530, Amit Kapila wrote: > On Wed, Jun 21, 2017 at 10:03 PM, Bruce Momjian <br...@momjian.us> wrote: > > On Wed, Jun 21, 2017 at 07:49:21PM +0530, Amit Kapila wrote: > >> On Tue, Jun 20, 2017 at 7:24 PM, Amit Kapila <amit.kapi

Re: [HACKERS] Re-indent HEAD tomorrow?

2017-06-22 Thread Bruce Momjian
kporting of their own patches, meaning the community and forks will have backbranch patch difficulties. -- Bruce Momjian <br...@momjian.us>http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + +

Re: [HACKERS] Re-indent HEAD tomorrow?

2017-06-21 Thread Bruce Momjian
ou are eventually doing all active branches, right? -- Bruce Momjian <br...@momjian.us>http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscripti

Re: [HACKERS] Broken hint bits (freeze)

2017-06-21 Thread Bruce Momjian
less than the primary. As far as I know this is the only remaining open issue. Sergey, please verify. I appreciate the work everyone has done to improve this, and all the existing fixes have been pushed to all supported branches. :-) -- Bruce Momjian <br...@momjian.us>http:

Re: [HACKERS] Shortened URLs for commit messages

2017-06-21 Thread Bruce Momjian
On Wed, Jun 21, 2017 at 11:11:57AM -0400, Alvaro Herrera wrote: > Bruce Momjian wrote: > > > Oh, here is a fixed version that requires an @ sign, which all message > > id's have: > > > > sed '/http/!s;^\(Discussion: *\)\(.*@.*\)$;\1https://postgr.es/m/\2;' &

Re: [HACKERS] Broken hint bits (freeze)

2017-06-20 Thread Bruce Momjian
. --- On Fri, Jun 16, 2017 at 10:57:33PM +0300, Sergey Burladyan wrote: > Bruce Momjian <br...@momjian.us> writes: > > On Fri, Jun 16, 2017 at 04:33:16AM +0300, Sergey Burladyan wrote: > > > Bruce M

Re: [HACKERS] Broken hint bits (freeze)

2017-06-20 Thread Bruce Momjian
On Tue, Jun 20, 2017 at 06:42:58PM +0300, Sergey Burladyan wrote: > Bruce Momjian <br...@momjian.us> writes: > > > On Tue, Jun 20, 2017 at 01:10:26PM +0300, Sergey Burladyan wrote: > > > Only if missing/changed files changed in size, because rsync run with > &g

Re: [HACKERS] Broken hint bits (freeze)

2017-06-20 Thread Bruce Momjian
On Tue, Jun 20, 2017 at 01:10:26PM +0300, Sergey Burladyan wrote: > Bruce Momjian <br...@momjian.us> writes: > > > Uh, as I understand it the rsync is going to copy the missing WAL file > > > from the new master to the standby, right, and I think pg_controldata >

Re: [HACKERS] Broken hint bits (freeze)

2017-06-19 Thread Bruce Momjian
On Mon, Jun 19, 2017 at 10:59:19PM -0400, Bruce Momjian wrote: > On Tue, Jun 20, 2017 at 03:50:29AM +0300, Sergey Burladyan wrote: > > 20 июн. 2017 г. 1:21 пользователь "Bruce Momjian" <br...@momjian.us> > > написал:  > > > > > > We

Re: [HACKERS] Broken hint bits (freeze)

2017-06-19 Thread Bruce Momjian
On Tue, Jun 20, 2017 at 03:50:29AM +0300, Sergey Burladyan wrote: > 20 июн. 2017 г. 1:21 пользователь "Bruce Momjian" <br...@momjian.us> написал:  > > > We are saying that Log-Shipping should match "Latest checkpoint > location", but the WAL for

Re: [HACKERS] Broken hint bits (freeze)

2017-06-19 Thread Bruce Momjian
e a mismatch if old standby servers were shut down before the old primary.) We are saying that Log-Shipping should match "Latest checkpoint location", but the WAL for that will not be sent to the standby, so it will not match, but that is OK since the only thing in the non-shipped

Re: [HACKERS] Broken hint bits (freeze)

2017-06-16 Thread Bruce Momjian
On Fri, Jun 16, 2017 at 04:44:46PM -0400, Bruce Momjian wrote: > Yes, that is _exactly_ the right place to look. Only in PG 10 do we > restart the new cluster to invalidate hash indexes. In previous > releases we didn't do the restart. > > That didn't matter with the old rsy

Re: [HACKERS] Preliminary results for proposed new pgindent implementation

2017-06-16 Thread Bruce Momjian
hat it does now, there's circa ten thousand changed lines.) Yeah, entab was designed to do that, via this C comment: /* * Is the next character going to be a tab? We do tab * replacement in the current spot if the next char is * going to be a tab and ignore min

Re: [HACKERS] Preliminary results for proposed new pgindent implementation

2017-06-16 Thread Bruce Momjian
On Fri, Jun 16, 2017 at 11:54:06AM -0700, Andres Freund wrote: > On 2017-06-16 14:42:38 -0400, Bruce Momjian wrote: > > On Fri, Jun 16, 2017 at 02:23:00PM -0400, Tom Lane wrote: > > > Well, that's something we need to discuss. I originally argued for > > > back-patch

Re: [HACKERS] Preliminary results for proposed new pgindent implementation

2017-06-16 Thread Bruce Momjian
ets. Can't they sync up to just before our pgindent commit and run pgindent on their own code base? -- Bruce Momjian <br...@momjian.us>http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. +

Re: [HACKERS] Preliminary results for proposed new pgindent implementation

2017-06-16 Thread Bruce Momjian
s, it is all about <80 column output. The current pgindent does everything possible to accomplish that --- the question is whether we want uglier code to do it. -- Bruce Momjian <br...@momjian.us>http://momjian.us EnterpriseDB http://enterprisedb.c

Re: [HACKERS] Broken hint bits (freeze)

2017-06-16 Thread Bruce Momjian
On Fri, Jun 16, 2017 at 08:10:13PM +0530, Amit Kapila wrote: > On Fri, Jun 16, 2017 at 7:03 AM, Sergey Burladyan <eshkin...@gmail.com> wrote: > > Bruce Momjian <br...@momjian.us> writes: > > > >> ! against the old primary and standby clusters. Verify th

Re: [HACKERS] Broken hint bits (freeze)

2017-06-16 Thread Bruce Momjian
On Fri, Jun 16, 2017 at 04:33:16AM +0300, Sergey Burladyan wrote: > Bruce Momjian <br...@momjian.us> writes: > > ! > > ! Also, if upgrading standby servers, change wal_level > > ! to replica in the postgresql.conf file on > > ! the new cluste

Re: [HACKERS] WIP: Data at rest encryption

2017-06-16 Thread Bruce Momjian
hether to encrypt temporary data or not. If we go forward with in-db encryption, I think we are going to have to have a discussion about what parts of PGDATA need to be encrypted, i.e., I don't think pg_clog needs encryption. -- Bruce Momjian <br...@momjian.us>http://momjian.

Re: [HACKERS] WIP: Data at rest encryption

2017-06-16 Thread Bruce Momjian
On Thu, Jun 15, 2017 at 08:08:05PM -0400, Bruce Momjian wrote: > On Thu, Jun 15, 2017 at 04:56:36PM -0700, Andres Freund wrote: > > how few concerns about this feature's complexity / maintainability > > impact have been raised. > > Yeah, I guess we will just have to wait

Re: [HACKERS] Shortened URLs for commit messages

2017-06-16 Thread Bruce Momjian
On Thu, Jun 15, 2017 at 01:05:19PM -0400, Bruce Momjian wrote: > On Tue, May 23, 2017 at 11:25:07PM -0400, Bruce Momjian wrote: > > I have written the following sed script to convert regular Postgres > > email message URLs to their shorter form for commit messages: > > >

Re: [HACKERS] WIP: Data at rest encryption

2017-06-15 Thread Bruce Momjian
On Thu, Jun 15, 2017 at 04:56:36PM -0700, Andres Freund wrote: > On 2017-06-15 19:44:43 -0400, Bruce Momjian wrote: > > Understood, but now you are promoting a feature with an admittedly-poor > > API, duplication of an OS feature, and perhaps an invasive change to the > >

Re: [HACKERS] WIP: Data at rest encryption

2017-06-15 Thread Bruce Momjian
On Thu, Jun 15, 2017 at 07:51:36PM -0400, Alvaro Herrera wrote: > Bruce Momjian wrote: > > On Thu, Jun 15, 2017 at 07:27:55PM -0400, Stephen Frost wrote: > > > I expect the same would happen with the shell-command approach suggested > > > up-thread and the prompt-on-stdi

Re: [HACKERS] WIP: Data at rest encryption

2017-06-15 Thread Bruce Momjian
ch. Understood, but now you are promoting a feature with an admittedly-poor API, duplication of an OS feature, and perhaps an invasive change to the code. Those are high hurdles. -- Bruce Momjian <br...@momjian.us>http://momjian.us EnterpriseDB htt

Re: [HACKERS] WIP: Data at rest encryption

2017-06-15 Thread Bruce Momjian
t we > couldn't actually drop support for it, but we don't want to add other > features which work in a similar way because, well, it sucks. My point is that if our support for db-level encryption is as bad as SSL key passwords, then it will be nearly useless, so we might as well not have it.

Re: [HACKERS] WIP: Data at rest encryption

2017-06-15 Thread Bruce Momjian
r than the other cases --- it isn't just code quality. However, it would be good to research how SSH handles it to see if we can get any ideas. -- Bruce Momjian <br...@momjian.us>http://momjian.us EnterpriseDB http://enterprisedb.com + As you are

Re: [HACKERS] WIP: Data at rest encryption

2017-06-15 Thread Bruce Momjian
access, it was probably an easy decision because the use-case was high and the complexity was low. -- Bruce Momjian <br...@momjian.us>http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. +

Re: [HACKERS] Broken hint bits (freeze)

2017-06-15 Thread Bruce Momjian
On Thu, Jun 15, 2017 at 03:00:18PM +0530, Amit Kapila wrote: > On Wed, Jun 14, 2017 at 8:44 PM, Bruce Momjian <br...@momjian.us> wrote: > > On Wed, Jun 14, 2017 at 07:45:17PM +0530, Amit Kapila wrote: > >> > Now, it seems we later added a doc section early on that

Re: [HACKERS] Shortened URLs for commit messages

2017-06-15 Thread Bruce Momjian
On Tue, May 23, 2017 at 11:25:07PM -0400, Bruce Momjian wrote: > I have written the following sed script to convert regular Postgres > email message URLs to their shorter form for commit messages: > > sed > 's;http\(s\?\)://www\.postgresql\.org/message-id/;http\1://postgr.es/m/;g

Re: [HACKERS] WIP: Data at rest encryption

2017-06-14 Thread Bruce Momjian
On Wed, Jun 14, 2017 at 06:41:43PM +0300, Ants Aasma wrote: > On Wed, Jun 14, 2017 at 6:26 PM, Bruce Momjian <br...@momjian.us> wrote: > > Are you checking the CPU type or if AES instructions are enabled on the > > CPU? I ask this because I just realized in researching my new

Re: [HACKERS] WIP: Data at rest encryption

2017-06-14 Thread Bruce Momjian
it in the page depends on every other bit in the page, so we > > certainly would. I don't know how many places we've got assumptions > > like this baked into the system, but I'm guessing there are a bunch. > > I think we need to require wal_log_hints=on when encryption

Re: [HACKERS] Broken hint bits (freeze)

2017-06-14 Thread Bruce Momjian
the first place. (Yeah, I am amazed I considered all these cases.) Updated patch attached. Thanks. -- Bruce Momjian <br...@momjian.us>http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + +

Re: pgindent (was Re: [HACKERS] [COMMITTERS] pgsql: Preventive maintenance in advance of pgindent run.)

2017-06-14 Thread Bruce Momjian
so that pgindent can tell that it's got an > up-to-date copy. 1.4? 2.0? For Piotr's reference, we will update src/tools/pgindent/pgindent to match whatever new version number you use. -- Bruce Momjian <br...@momjian.us>http://momjian.us EnterpriseDB

Re: [HACKERS] WIP: Data at rest encryption

2017-06-14 Thread Bruce Momjian
ter > > than file system encryption? > > I've been asked for this capability multiple times from our users and > have generally pushed back and encouraged filesystem-level encryption. > That hasn't always been an acceptable solution, unfortunately. Yes, it would be good to know how often th

Re: [HACKERS] WIP: Data at rest encryption

2017-06-14 Thread Bruce Momjian
ecific organizations by changing PostgreSQL core in very invasive way > > (keeping in mind testing, maintaining, etc) is misguided. > > In the same time implementing a plugable storage API and then implementing > encrypted / compressed / whatever storage in a standalone extensi

Re: pgindent (was Re: [HACKERS] [COMMITTERS] pgsql: Preventive maintenance in advance of pgindent run.)

2017-06-13 Thread Bruce Momjian
On Tue, Jun 13, 2017 at 05:00:31PM -0400, Tom Lane wrote: > Anyway, it is now time to fish or cut bait. I don't think we can wait > much longer to decide whether we're going to adopt this new indent > version for PG 10. I think we should. The floor is open for votes. Works for me. --

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
On Tue, Jun 13, 2017 at 04:08:29PM -0400, Peter Eisentraut wrote: > On 6/13/17 15:51, Bruce Momjian wrote: > > Isn't the leakage controlled by OS permissions, so is it really leakage, > > i.e., if you can see the leakage, you probably have bypassed the OS > > permissions and

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
On Tue, Jun 13, 2017 at 03:20:12PM -0400, Stephen Frost wrote: > Bruce, > > * Bruce Momjian (br...@momjian.us) wrote: > > On Tue, Jun 13, 2017 at 02:38:58PM -0400, Stephen Frost wrote: > > > It's good to discuss what the feature would bring and what cases it >

Re: [HACKERS] Broken hint bits (freeze)

2017-06-13 Thread Bruce Momjian
that block, which should be safe. There is now no need to start/stop the new server since pg_upgrade will do that safely already. I plan to patch this back to 9.5 where these instructions were added. I will mention that this should be in the minor release notes. -- Bruce Momjian <

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
can make it work easily while adding this feature, why wouldn't we do that? -- Bruce Momjian <br...@momjian.us>http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancien

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
eal with future feature additions, then you are right we will never have it. -- Bruce Momjian <br...@momjian.us>http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient R

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
nd we are not going to know that if we don't talk about it, but hey, this is just an email thread and I can marshal opposition to the feature later when it appears, and point this all out again. -- Bruce Momjian <br...@momjian.us>http://momjian.us EnterpriseDB

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
but the above strikes me as > the big ones, and those are certainly valuable enough on their own for > us to seriously consider adding this capability. Since you seem to be trying to shut down discussion, I will simply say I am unimpressed that this use-case is sufficient justification

Re: [HACKERS] v10beta pg_catalog diagrams

2017-06-13 Thread Bruce Momjian
g_constraints.html > > Go for it. Yeah, great. We have been talking about adding diagrams to our official docs but needed an updated toolchain, which I think we now have, so there is a lot of opportunity for growth here. -- Bruce Momjian <br...@momjian.us>http://momjian.

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
ce that is encrypted. My big question is how many times are the OS permissions bypassed in a way that would also not expose the db clusters key or db data? -- Bruce Momjian <br...@momjian.us>http://momjian.us EnterpriseDB

Re: [HACKERS] Transactional sequence stuff breaks pg_upgrade

2017-06-13 Thread Bruce Momjian
e renumbered everything, you would no longer be able to do > that, or at least not easily. FYI, pg_upgrade is designed to go boom if something doesn't look right because it can't anticipate what changes might be made to Postgres in the future. boom == feature! -- Bruce Momjian <br...@momji

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
s of the symetric key, one for each role you want to view the data. And good point on the ability to change the RSA key/password without having to reencrypt the data. -- Bruce Momjian <br...@momjian.us>http://momjian.us EnterpriseDB

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
case where you encrypt a row using a symetric key, then store RSA-encrypted versions of the symetric key encrypted that only specific users could decrypt and get the key to decrypt the data. -- Bruce Momjian <br...@momjian.us>http://momjian.us EnterpriseDB

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
ncryption. One cool idea I have is using public encryption to store the encryption key by users who don't know the decryption key, e.g. RSA. It would be a write-only encryption option. Not sure how useful that is, but it easily possible, and doesn't require us to keep the _encryption_ key secret, ju

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
the user data goes through the cipher, e.g. AES in CBC mode. However, if you are using a stream cipher based on a block cipher, e.g. CTR, GCM, you XOR the user data with a random bit stream, and in that case one bit change in user data would be one bit change in the cipher output. -- Bruc

Re: pgindent (was Re: [HACKERS] [COMMITTERS] pgsql: Preventive maintenance in advance of pgindent run.)

2017-06-12 Thread Bruce Momjian
nge? I have reviewed the C formatting options a few times over the years and every time the other options were worse than what we had. -- Bruce Momjian <br...@momjian.us>http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I

Re: [HACKERS] PG10 transition tables, wCTEs and multiple operations on the same table

2017-06-08 Thread Bruce Momjian
t want to *risk* unduly holding up the release. > > That was my understanding, at least. I think we can all agree that Kevin should have communicated this earlier, rather than requiring Robert to push him on the issue. -- Bruce Momjian <br...@momjian.us>http://momjian.us

Re: [HACKERS] PG 10 release notes

2017-06-07 Thread Bruce Momjian
On Wed, Jun 7, 2017 at 03:18:49PM +1000, Neha Khatri wrote: > > On Mon, May 15, 2017 at 12:45 PM, Bruce Momjian <br...@momjian.us> wrote: > > On Thu, May 11, 2017 at 11:50:03PM -0400, Tom Lane wrote: > > Michael Paquier <michael.paqu...@gmail.com> writes

Re: [HACKERS] Extra Vietnamese unaccent rules

2017-06-06 Thread Bruce Momjian
On Tue, Jun 6, 2017 at 12:15:13PM -0400, Tom Lane wrote: > Bruce Momjian <br...@momjian.us> writes: > > There seems to be a problem. I can't see a patch dated 2017-06-07 on > > the commitfest page: > > https://commitfest.postgresql.org/14/1161/ > > It loo

Re: [HACKERS] Extra Vietnamese unaccent rules

2017-06-06 Thread Bruce Momjian
On Wed, Jun 7, 2017 at 01:06:22AM +0900, Man Trieu wrote: > 2017-06-07 0:31 GMT+09:00 Bruce Momjian <br...@momjian.us>: > I added the thread but there was no change.  (I think the thread was > already present.)  It appears it is not seeing this patch as the lat

Re: [HACKERS] Extra Vietnamese unaccent rules

2017-06-06 Thread Bruce Momjian
On Wed, Jun 7, 2017 at 12:10:25AM +0900, Dang Minh Huong wrote: > > On Jun 4, 29 Heisei, at 00:48, Bruce Momjian <br...@momjian.us> wrote: > >>>> Shouldn't you use "or is_letter_with_marks()", instead of "or len(...) > >>>>> 1"

Re: [HACKERS] Challenges preventing us moving to 64 bit transaction id (XID)?

2017-06-06 Thread Bruce Momjian
On Tue, Jun 6, 2017 at 09:05:03AM -0400, Peter Eisentraut wrote: > On 6/6/17 08:29, Bruce Momjian wrote: > > On Tue, Jun 6, 2017 at 06:00:54PM +0800, Craig Ringer wrote: > >> Tom's point is, I think, that we'll want to stay pg_upgrade > >> compatible. So when we see a

Re: [HACKERS] Use of non-restart-safe storage by temp_tablespaces

2017-06-06 Thread Bruce Momjian
le in a permanent tablespace (probably the default one for the > database). > > FWIW I don't think calling these tablespaces "temporary" is the right > word. It's not the tablespaces that are temporary. Maybe "evanescent". I was thinking "transien

Re: [HACKERS] Challenges preventing us moving to 64 bit transaction id (XID)?

2017-06-06 Thread Bruce Momjian
uples "down" the page, what do we > do? I guess I am missing something. If you see an old page version number, you know none of the tuples are from running transactions so you can just freeze them all, after consulting the pg_clog. What am I missing? If the page is full, why are

Re: [HACKERS] Use of non-restart-safe storage by temp_tablespaces

2017-06-05 Thread Bruce Momjian
On Mon, Jun 5, 2017 at 07:38:43PM -0700, Andres Freund wrote: > On 2017-06-05 22:34:17 -0400, Bruce Momjian wrote: > > On Mon, Jun 5, 2017 at 04:38:32PM -0500, Jerry Sievers wrote: > > > The SAN snaps capture the entire pgdata and WAL pg_xlog area but there > > > is

  1   2   3   4   5   6   7   8   9   10   >