Re: [HACKERS] pg_upgrade vs extension upgrades

2017-04-12 Thread Bruce Momjian
rade started creating an incremental-analyze script, people thought it should be more generic so it was moved to vacuumdb --analyze-in-stages. Seems we should do the same thing for the extension upgrade script. -- Bruce Momjian http://momjian.us EnterpriseDB

Re: [HACKERS] Some thoughts about SCRAM implementation

2017-04-12 Thread Bruce Momjian
tion, but two different TLS > connections, and make the authentication fail. > > SSL certificates, with validation, achieves the same, but channel binding > achieves it without the hassle of certificates. How does it do that? -- Bruce Momjian http://momjian.us EnterpriseDB

Re: [pgsql-www] [HACKERS] Small issue in online devel documentation build

2017-04-12 Thread Bruce Momjian
ink? Well, we have been using JavaScript for years, so my patch only fixes the JavaScript we already have. Seems applying the fix now makes sense and then we can ponder other methods. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com

Re: [HACKERS] Some thoughts about SCRAM implementation

2017-04-11 Thread Bruce Momjian
(backups which include pg_authid) is just a "check-box" item. Well, they have the entire backup so I don't think cracking the password is a huge win, though the password does potentially give them access to future data. And it prevents them from reusing the password on anoth

Re: [HACKERS] src/interfaces/libpq shipping nmake-related Makefiles

2017-04-11 Thread Bruce Momjian
On Tue, Apr 11, 2017 at 04:51:03PM -0400, Tom Lane wrote: > Bruce Momjian writes: > Yeah, very long ago. A quick search of our archives shows that the number > of mentions of Borland pretty much fell off a cliff after 2009 (excluding > the repeated conversations about dropping supp

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-04-11 Thread Bruce Momjian
And apologies > > for keep pushing it till the last minute even though it was clear to me > > quite some time back the patch is not going to make it. > > What confuses me about that position is that people were advocating to > actually commit till literally hou

Re: [HACKERS] Some thoughts about SCRAM implementation

2017-04-11 Thread Bruce Momjian
e big win for SCRAM is the inability to replay md5 packets after recording 16k sessions (salt was only 32-bit, so a 50% chance of replay after 16 sessions), and storage of SHA256 hashes instead of MD5 in pg_authid, though the value of that is mostly a check-box item because collisions are not a pr

Re: [HACKERS] src/interfaces/libpq shipping nmake-related Makefiles

2017-04-11 Thread Bruce Momjian
; > > Ok, since we have two votes for it, I will go ahead and do that (as a separate > patch pushed together).' I am fine with removing things, but I do remember one reason the Borland part was kept is that some tool would only work with the Borland-compiled library, not gcc or MSVC,

Re: [HACKERS] SCRAM authentication, take three

2017-04-11 Thread Bruce Momjian
> SCRAM-SHA-512 verifier, not both, so that will dictate which one to use. It seems openssl has had to deal with cipher preferences for years and invented ssl_ciphers. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you ar

Re: [HACKERS] bumping HASH_VERSION to 3

2017-04-11 Thread Bruce Momjian
rth the trouble.) We already have code to do all of that, but it was removed from pg_upgrade in 9.5. You can still see it in 9.4: contrib/pg_upgrade/version_old_8_3.c::old_8_3_invalidate_hash_gin_indexes() I would be happy to restore that code and make it w

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-04-10 Thread Bruce Momjian
l as that is. We now have something to look forward to in Postgres 11. :-) -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription

Re: [pgsql-www] [HACKERS] Small issue in online devel documentation build

2017-04-10 Thread Bruce Momjian
er on something the build system does anyway so that if someone builds pre-10 docs with the new build system, they get the same output. The attached patch uses: div.refnamediv h2 .refentrytitle and "div.refnamediv" is only generated in the new build system. -- Bruce Momjian

Re: [pgsql-www] [HACKERS] Small issue in online devel documentation build

2017-04-09 Thread Bruce Momjian
On Sun, Apr 9, 2017 at 06:59:09PM +0200, Magnus Hagander wrote: > On Sat, Apr 8, 2017 at 3:52 AM, Bruce Momjian wrote: > Attached is a patch that can be applied to pgweb which should fix all of > this. > > Is there any chance we can find a way to do this with actual CS

Re: [pgsql-www] [HACKERS] Small issue in online devel documentation build

2017-04-09 Thread Bruce Momjian
On Sun, Apr 9, 2017 at 07:00:38PM +0200, Magnus Hagander wrote: > > > On Sun, Apr 9, 2017 at 2:32 AM, Bruce Momjian wrote: > > On Sat, AprĀ  8, 2017 at 12:50:19PM -0400, Robert Haas wrote: > > On Sat, Apr 8, 2017 at 6:39 AM, Bruce Momjian wrote: > >

Re: [pgsql-www] [HACKERS] Small issue in online devel documentation build

2017-04-08 Thread Bruce Momjian
On Sat, Apr 8, 2017 at 12:50:19PM -0400, Robert Haas wrote: > On Sat, Apr 8, 2017 at 6:39 AM, Bruce Momjian wrote: > > What other problems do we have with pgweb that I can work on? > > Well, the 10devel documentation doesn't believe in orange. Compare: > > https

Re: [pgsql-www] [HACKERS] Small issue in online devel documentation build

2017-04-08 Thread Bruce Momjian
doc looks ok now. Great. What other problems do we have with pgweb that I can work on? -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscrip

Re: [pgsql-www] [HACKERS] Small issue in online devel documentation build

2017-04-07 Thread Bruce Momjian
ied, is to conditionally inject additional CSS to be _more_ specific than the first CSS and set the font-size to a simple '1em' so the first CSS is not called twice. I don't think 'important!' is necessary but it would be good to test this. Attached is a patch that can be a

Re: [HACKERS] If an extension library is loaded during pg_upgrade, can it tell?

2017-04-03 Thread Bruce Momjian
t; PL/Java ordinarily checks what version of its schema is around, but > that may be premature while pg_upgrade is doing its thing and the schema > might not be all there yet. You can check the backend global variable IsBinaryUpgrade to check if binary upgrade is being performed. Does that hel

Re: [HACKERS] release notes for v10

2017-03-30 Thread Bruce Momjian
On Thu, Mar 30, 2017 at 06:23:04PM -0400, Robert Haas wrote: > Hi, > > One of the things we'll need to do to get 10beta1 out the door is have > a set of release notes. I spoke with Bruce Momjian today at PGCONF.US > and he told me that he has set aside time during April to w

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-30 Thread Bruce Momjian
On Tue, Mar 21, 2017 at 04:04:58PM -0400, Bruce Momjian wrote: > On Tue, Mar 21, 2017 at 04:56:16PM -0300, Alvaro Herrera wrote: > > Bruce Momjian wrote: > > > On Tue, Mar 21, 2017 at 04:43:58PM -0300, Alvaro Herrera wrote: > > > > Bruce Momjian wrote: > > &

Re: [HACKERS] Schedule and Release Management Team for PG10

2017-03-30 Thread Bruce Momjian
conference attendance. If that delays our final release for a week, it is worth it. I propose we go for a week delay in closing the commit fest, and we decide right now. Ideally I like to to see delay in one-week increments _and_ announce that a week before each deadline. -- Bruce Momjian

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-27 Thread Bruce Momjian
; > Let me know if its still not clear and I can draw some diagrams to explain it. Ah, yes, that does help to explain the 50% because 50% of updates are now HOT/WARM. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterpris

Re: [HACKERS] standardized backwards incompatibility tag for commits

2017-03-27 Thread Bruce Momjian
g entry, not exactly how > it's spelled. Yes, it doesn't matter as long as it is stated somehow. I don't know of any missing cases due to text differences. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As y

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-27 Thread Bruce Momjian
would hope that most updates get done within a single > block, so that the row-moves-to-new-block case wouldn't happen very > often. > > I'm perplexed. Yes, I asked the same question in this email: https://www.postgresql.org/message-id/2017032119.ge16...@momjian.us

Re: [HACKERS] increasing the default WAL segment size

2017-03-22 Thread Bruce Momjian
y default. Maybe its time to have a documentation section listing suggested changes for small installs so we can have more reasonable defaults. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-21 Thread Bruce Momjian
On Tue, Mar 21, 2017 at 04:56:16PM -0300, Alvaro Herrera wrote: > Bruce Momjian wrote: > > On Tue, Mar 21, 2017 at 04:43:58PM -0300, Alvaro Herrera wrote: > > > Bruce Momjian wrote: > > > > > > > I don't think it makes sense to try and save bits and ad

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-21 Thread Bruce Momjian
On Tue, Mar 21, 2017 at 04:43:58PM -0300, Alvaro Herrera wrote: > Bruce Momjian wrote: > > > I don't think it makes sense to try and save bits and add complexity > > when we have no idea if we will ever use them, > > If we find ourselves in dire need of addi

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-21 Thread Bruce Momjian
nd then HEAP_WARM_UPDATED just from correctness > perspective. I don't think it makes sense to try and save bits and add complexity when we have no idea if we will ever use them, but again, I am back to my original question of whether we have done sufficient research, and if everyone says "

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-21 Thread Bruce Momjian
zy vacuums would show no improvementin in real-world workloads, but maybe I am wrong. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-21 Thread Bruce Momjian
many bits we need, and my concern is that we haven't discussed the improvement ideas enough to know we have done the best we can for PG 10. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I a

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-21 Thread Bruce Momjian
it's going to be really painful for a lot > of users. Many users can't realistically upgrade using pg_dump, ever. > So they'll be stuck on the release before the one that breaks > compatibility for a very long time. Right. If we weren't setting tuple and tid bits

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-21 Thread Bruce Momjian
r. I guess I just don't feel we have explored the duplicate-index-entry problem enough for me to be comfortable. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-21 Thread Bruce Momjian
e with it, I am too, but I have to ask these questions. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription + -- Sent via pg

Re: [HACKERS] Our feature change policy

2017-03-20 Thread Bruce Momjian
On Mon, Mar 20, 2017 at 11:57:13AM -0400, Stephen Frost wrote: > Tom, > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > Stephen Frost writes: > > > * Bruce Momjian (br...@momjian.us) wrote: > > >> 1. make the change now and mention it in the release notes >

Re: [HACKERS] Our feature change policy

2017-03-20 Thread Bruce Momjian
On Mon, Mar 20, 2017 at 11:40:34AM -0400, Tom Lane wrote: > Stephen Frost writes: > > * Bruce Momjian (br...@momjian.us) wrote: > >> 1. make the change now and mention it in the release notes > >> 2. #1, but also provide backward compatibility for 5+ years &

[HACKERS] Our feature change policy

2017-03-20 Thread Bruce Momjian
is for changes that could affect data validity or are hard to detect, e.g. the standard-conforming string change. Is this accurate? If so we can write this down and avoid the +1/-1 re-litigating every time we want to change something, or at least have a more focused conversation on the issues

Re: [HACKERS] Microvacuum support for Hash Index

2017-03-17 Thread Bruce Momjian
when do you sleep? > > I think that applies to a bunch of people, including yourself ;) Gee, no one asks when I sleep. I wonder why. ;-) -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you ar

Re: [HACKERS] Should we eliminate or reduce HUP from docs?

2017-03-16 Thread Bruce Momjian
for PG 10: commit 10c064ce4dad088ba2d8b978bff6009b9f22dc3a Author: Bruce Momjian Date: Tue Oct 25 11:26:15 2016 -0400 Consistently mention 'SELECT pg_reload_conf()' in config files Previously we only mentioned SIGHUP and 'pg_ctl reload'

Re: [HACKERS] Changing references of password encryption to hashing

2017-03-15 Thread Bruce Momjian
p track of the hash > separately to the password text and use prefixes instead, and since we > need compatibility for dumps, it's hard to do a great deal about > though. With SCRAM coming in PG 10, is there anything we can do to clean this up for PG 10? -- Bruce Momjian ht

Re: [HACKERS] Upgrading postmaster's log messages about bind/listen errors

2017-03-15 Thread Bruce Momjian
ound protections > are disabled because oldest checkpointed MultiXact %u does not exist on > disk". Pushed without that. Gee, I kind of like the new messages: LOG: listening on IPv4 address "127.0.0.1", port 5432 LOG: listening on Unix socket "/tmp/.s.PGS

Re: [HACKERS] Bizarre choice of case for RELKIND_PARTITIONED_TABLE

2017-03-10 Thread Bruce Momjian
instead of 's' is that the latter was > taken for "special" relations, which we removed a few releases ago > (commit 3a694bb0a1). Ah! I knew there was a reason for 'S', but couldn't remember it. :-) -- Bruce Momjian http://momjian.us Enterpr

Re: [HACKERS] WIP: Faster Expression Processing v4

2017-03-10 Thread Bruce Momjian
only downside of having it in 10.0 is that it will not have lived in the source tree for as long a time between commit and PG release, but if it is tested, that seems fine. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are,

Re: [HACKERS] rename pg_log directory?

2017-03-10 Thread Bruce Momjian
On Fri, Mar 10, 2017 at 11:23:54AM +0100, Andreas Karlsson wrote: > On 03/09/2017 11:25 PM, Bruce Momjian wrote: > >"data" and "base" where chosen because it is a "data-base", but with the > >pg_ prefixes it would be a pg_data_pg_base. ;-) > &

Re: [HACKERS] [PATCH] Use $ parameters as replacement characters for pg_stat_statements

2017-03-09 Thread Bruce Momjian
look in pg_stat_statements to see how long is usually ran. It seemed like a useful idea to me. I don't think they can hash the query manually because of the constants involved. -- Bruce Momjian http://momjian.us EnterpriseDB http

Re: [HACKERS] rename pg_log directory?

2017-03-09 Thread Bruce Momjian
care much about which name > is picked as long as we get rid off the "pg_" prefix. > > Btw, is there a reason for why global and base do not have the "pg_" prefix? "data" and "base" where chosen because it is a "data-base", but with

Re: [HACKERS] Disallowing multiple queries per PQexec()

2017-02-28 Thread Bruce Momjian
ider disallowing multiple queries in PQexec() as an additional barrier to SQL injection attacks and it is a "consider" item. Should it be moved to the Wire Protocol Changes / v4 Protocol section or removed? -- Bruce Momjian http://momjian.us EnterpriseDB

Re: [HACKERS] Partitioned tables and relfilenode

2017-02-27 Thread Bruce Momjian
7;r', 'm') AND " but partitions are defined as 'P': #define RELKIND_PARTITIONED_TABLE 'P' /* partitioned table */ I am a little confused by the above. Is a partitioned table the parent or the children? Reading the

Re: [HACKERS] rename pg_log directory?

2017-02-27 Thread Bruce Momjian
t, but "log" seems perhaps a little too > generic to me. Would something like "server_log" be better? "activity_log"? I like the idea of a rename. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com

Re: [HACKERS] bytea_output output of base64

2017-02-27 Thread Bruce Momjian
e64 is less easy to compress because input bytes span base64-bytes, so "dog" might encode differently depending on where the two high bits are stored, while hex alway encodes "dog" the same way. -- Bruce Momjian http://momjian.us EnterpriseDB

Re: [HACKERS] pg_upgrade loses security lables and COMMENTs on blobs

2017-02-27 Thread Bruce Momjian
y the best you can do. Your analysis of how we used to treat large objects is correct, and was never adjusted for the changes you outlined. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am

Re: [HACKERS] Replication vs. float timestamps is a disaster

2017-02-27 Thread Bruce Momjian
her place by pre-9.0 * VACUUM FULL; kept for binary * upgrade support */ However, if users built Postgres 8.4 with integer timestamps, they could still be in the data files. Does a

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2017-02-27 Thread Bruce Momjian
ndering if people are fine > > to include it in this last CF. > > The March CF is already looking pretty daunting. We can try to include > this but I won't be too surprised if it gets punted to a future CF. Yeah, that was my reaction too. -- Bruce Mom

Re: [HACKERS] removing tsearch2

2017-02-27 Thread Bruce Momjian
On Mon, Feb 27, 2017 at 01:00:20PM -0800, David E. Wheeler wrote: > On Feb 27, 2017, at 12:04 PM, Bruce Momjian wrote: > > > Just stating the obvious, but one of the reasons CPAN works so well is > > that most of the modules are written in Perl and hence don't need >

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2017-02-27 Thread Bruce Momjian
s > not a patch in the easy category, far from that, but it does not > present a new concept. FYI, the thread started on 2013-11-15. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you

Re: [HACKERS] removing tsearch2

2017-02-27 Thread Bruce Momjian
to what we deliver in core. Just stating the obvious, but one of the reasons CPAN works so well is that most of the modules are written in Perl and hence don't need per-platform compilation. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterp

Re: [HACKERS] Partitioned tables and relfilenode

2017-02-27 Thread Bruce Momjian
reindex (they throw a meaningful error > already due to the lack of indexes.) I don't think we are doing this, but if the parent table doesn't have a physical file pg_upgrade will need to be taught that. We have that case now for unlogged tables on standby servers that we need to addr

Re: [HACKERS] bytea_output output of base64

2017-02-25 Thread Bruce Momjian
On Sat, Feb 25, 2017 at 06:49:01PM -0500, Peter Eisentraut wrote: > On 2/23/17 17:55, Bruce Momjian wrote: > > On Thu, Feb 23, 2017 at 04:08:58PM -0500, Tom Lane wrote: > >> Bruce Momjian writes: > >>> Is there a reason we don't support base64 as a bytea_output

Re: [HACKERS] Documentation improvements for partitioning

2017-02-25 Thread Bruce Momjian
On Wed, Feb 22, 2017 at 07:44:41AM +0530, Robert Haas wrote: > On Tue, Feb 21, 2017 at 2:03 AM, Bruce Momjian wrote: > > I have to admit my reaction was similar to Simon's, meaning that the > > lack of docs is a problem, and that the limitations are kind of a > > surpris

Re: [HACKERS] Should `pg_upgrade --check` check relation filenodes are present?

2017-02-25 Thread Bruce Momjian
this standby to rejoin the primary, you just want to upgrade it. Second, I thought unlogged tables had empty files on the standby, not _missing_ files. Is that correct? Should pg_upgrade just allow missing unlogged table files? I don't see any way to detect we are running on a standby since th

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-02-25 Thread Bruce Momjian
eral acceptance to that idea.. > I > am assuming latter :-)) Yes, I think it is the latter. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient

Re: [HACKERS] WIP: About CMake v2

2017-02-24 Thread Bruce Momjian
having two build systems we have to maintain is better than 1.1 build systems where we can mostly ignore 0.1 of that. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + +

Re: [HACKERS] Checksums by default?

2017-02-24 Thread Bruce Momjian
On Fri, Feb 24, 2017 at 10:09:50PM +0200, Ants Aasma wrote: > On Fri, Feb 24, 2017 at 9:37 PM, Bruce Momjian wrote: > > Oh, that's why we will hopefully eventually change the page checksum > > algorithm to use the special CRC32 instruction, and set a new checksum > > ver

Re: [HACKERS] Checksums by default?

2017-02-24 Thread Bruce Momjian
a way they are a GPU built into CPUs. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription + -- Sent via pgsql-hackers maili

Re: [HACKERS] Checksums by default?

2017-02-24 Thread Bruce Momjian
never got around to adding runtime checks for > the architecture to enable this speedup. Oh, that's why we will hopefully eventually change the page checksum algorithm to use the special CRC32 instruction, and set a new checksum version --- got it. I assume there is currently no compile-t

Re: [HACKERS] Checksums by default?

2017-02-24 Thread Bruce Momjian
e. IIRC it was actually a lot faster than the naive hardware > CRC that is used for WAL and about on par with interleaved CRC. Uh, I thought already did compile-time testing for SSE4.1 and used them if present. Why do you say "with default compiles there is a lot of performance left

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-02-24 Thread Bruce Momjian
On Fri, Feb 24, 2017 at 02:14:23PM +0530, Pavan Deolasee wrote: > > > On Thu, Feb 23, 2017 at 11:53 PM, Bruce Momjian wrote: > > On Thu, Feb 23, 2017 at 03:03:39PM -0300, Alvaro Herrera wrote: > > Bruce Momjian wrote: > > > > > As I remember,

Re: [HACKERS] bytea_output output of base64

2017-02-23 Thread Bruce Momjian
On Thu, Feb 23, 2017 at 07:09:57PM -0500, Andrew Dunstan wrote: > > > On 02/23/2017 06:52 PM, David Fetter wrote: > > On Thu, Feb 23, 2017 at 05:55:37PM -0500, Bruce Momjian wrote: > >> On Thu, Feb 23, 2017 at 04:08:58PM -0500, Tom Lane wrote: > >>> Bruc

Re: [HACKERS] bytea_output output of base64

2017-02-23 Thread Bruce Momjian
On Thu, Feb 23, 2017 at 04:08:58PM -0500, Tom Lane wrote: > Bruce Momjian writes: > > Is there a reason we don't support base64 as a bytea_output output > > option, except that no one has implemented it? > > How about "we already have one too many bytea output fo

Re: [HACKERS] Checksums by default?

2017-02-23 Thread Bruce Momjian
ze the risk of losing data. Do these other databases do checksums because they don't do full_page_writes? They just detect torn pages rather than repair them like we do? -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprise

[HACKERS] bytea_output output of base64

2017-02-23 Thread Bruce Momjian
https://en.wikipedia.org/wiki/Base64 Is there a reason we don't support base64 as a bytea_output output option, except that no one has implemented it? -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so o

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-02-23 Thread Bruce Momjian
On Thu, Feb 23, 2017 at 03:58:59PM -0300, Alvaro Herrera wrote: > Bruce Momjian wrote: > > On Thu, Feb 23, 2017 at 03:45:24PM -0300, Alvaro Herrera wrote: > > > > > and potentially trim the first HOT chain as those tuples become > > > > invisible. > > >

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-02-23 Thread Bruce Momjian
On Thu, Feb 23, 2017 at 03:45:24PM -0300, Alvaro Herrera wrote: > Bruce Momjian wrote: > > > Well, let's walk through this. Let's suppose you have three updates > > that stay on the same page and don't update any indexed columns --- that > > would produce a

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-02-23 Thread Bruce Momjian
On Thu, Feb 23, 2017 at 03:26:09PM -0300, Alvaro Herrera wrote: > Bruce Momjian wrote: > > On Thu, Feb 23, 2017 at 03:03:39PM -0300, Alvaro Herrera wrote: > > > Bruce Momjian wrote: > > > > > > > As I remember, WARM only allows > > > > a si

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-02-23 Thread Bruce Momjian
On Thu, Feb 23, 2017 at 03:03:39PM -0300, Alvaro Herrera wrote: > Bruce Momjian wrote: > > > As I remember, WARM only allows > > a single index-column change in the chain. Why are you seeing such a > > large performance improvement? I would have thought it would be tha

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-02-23 Thread Bruce Momjian
m > block size. Can look at that if it comes to finding more bits. OK, so it seems more bits is not a blocker to enhancements, yet. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you w

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-02-23 Thread Bruce Momjian
it has found an existing bug and given good performance numbers. :-) I have what might be a supid question. As I remember, WARM only allows a single index-column change in the chain. Why are you seeing such a large performance improvement? I would have thought it would be that high if we allowed

Re: [HACKERS] FYI: git worktrees as replacement for "rsync the CVSROOT"

2017-02-23 Thread Bruce Momjian
: 'worktree' is not a git command. See 'git --help'. which is in Debian Jessie. This reports worktree was added in 2.5, released in July 2015: https://developer.atlassian.com/blog/2015/10/cool-features-git-2.x/ I am glad the git team has seen the value in work tre

Re: [HACKERS] WARM and indirect indexes

2017-02-22 Thread Bruce Momjian
would be useless for 99% of users. In talking to Alexander Korotkov, he mentioned that indirect indexes could be used for global/cross-partition indexes, and for index-organized tables (heap and index together in a single file). This would greatly expand the usefulness of indirect indexes and

Re: [HACKERS] Packages: Again

2017-02-22 Thread Bruce Momjian
logs/pgblog/2012.html#April_23_2012 -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription + -- Sent via pgsql-hackers mailin

Re: [HACKERS] [PATCH] Rename pg_switch_xlog to pg_switch_wal

2017-02-22 Thread Bruce Momjian
ally, pg_upgrade already checks for some odd directories stored inside of PGDATA: WARNING: new data directory should not be inside the old data directory, e.g. %s\n", old_cluster_pgdata); WARNING: user-defined tablespace locations should not be inside the data d

Re: [HACKERS] Multivariate statistics and expression indexes

2017-02-20 Thread Bruce Momjian
On Tue, Feb 21, 2017 at 01:27:53AM +0100, Tomas Vondra wrote: > On 02/21/2017 12:13 AM, Bruce Momjian wrote: > >At the risk of asking a stupid question, we already have optimizer > >statistics on expression indexes. In what sense are we using this for > >multi-variate st

Re: [HACKERS] ICU integration

2017-02-20 Thread Bruce Momjian
like the fact we can detect collation changes with ICU, we don't know if ICU collations change more often than OS collations. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you wi

Re: [HACKERS] ICU integration

2017-02-20 Thread Bruce Momjian
On Mon, Feb 20, 2017 at 02:54:22PM -0800, Peter Geoghegan wrote: > On Mon, Feb 20, 2017 at 2:38 PM, Bruce Momjian wrote: > > I can't think of any cases where we warn of possible corruption only in > > the server logs. > > It's just like any case where there w

[HACKERS] Multivariate statistics and expression indexes

2017-02-20 Thread Bruce Momjian
blogs/pgblog/2017.html#February_20_2017 -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription + -- Sent via pgsql-hackers ma

Re: [HACKERS] ICU integration

2017-02-20 Thread Bruce Momjian
on. > http://www.postgresql.org/mailpref/pgsql-hackers I can't think of any cases where we warn of possible corruption only in the server logs. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I

Re: [HACKERS] Documentation improvements for partitioning

2017-02-20 Thread Bruce Momjian
king them public. It is often not clear what decisions were made and why. The idea that unique indexes on a parent table can't guarantee uniqueness across child tables is both a surprise, and obvious once stated. -- Bruce Momjian http://momjian.us EnterpriseDB

Re: [HACKERS] Documentation improvements for partitioning

2017-02-20 Thread Bruce Momjian
pulate the sequence and the DEFAULT separately, but if you drop the table the sequence is dropped to automatically. This seems like an instructive example of how we have bundled behavior together in the past in a logical and easy-to-understand way. Of course, their might be some technical limit

Re: [HACKERS] PoC: Make it possible to disallow WHERE-less UPDATE and DELETE

2017-02-02 Thread Bruce Momjian
On Thu, Feb 2, 2017 at 07:18:45AM -0800, David Fetter wrote: > On Thu, Feb 02, 2017 at 03:16:29PM +0000, Bruce Momjian wrote: > > I just don't see this patch going in. I think it needs are larger > > approach to the problems it is trying to solve. I think it then >

Re: [HACKERS] PoC: Make it possible to disallow WHERE-less UPDATE and DELETE

2017-02-02 Thread Bruce Momjian
I just don't see this patch going in. I think it needs are larger approach to the problems it is trying to solve. I think it then will be very useful. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/p

Re: [HACKERS] pg_upgrade vs. pg_ctl stop -m fast

2017-01-12 Thread Bruce Momjian
fast" > argument from this function altogether. Agreed, it should be remove. Should I do it? -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient

Re: [HACKERS] Packages: Again

2017-01-11 Thread Bruce Momjian
On Wed, Jan 11, 2017 at 03:08:58PM -0500, Bruce Momjian wrote: > On Wed, Jan 11, 2017 at 08:56:23PM +0100, Fabien COELHO wrote: > > > > >I think we need to focus on things that _can't_ be done first, rather > > >than things that require porting, e.g. until we h

Re: [HACKERS] Packages: Again

2017-01-11 Thread Bruce Momjian
uot; complaints we used to get, perhaps related to exceptions. Once we had them, the complaints of that type disappeared. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + +

Re: [HACKERS] Packages: Again

2017-01-11 Thread Bruce Momjian
s, you couldn't migrate an application that needed it. It wasn't a question of porting --- there was just no way to port it. Those _missing_ pieces should be a priority. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you

Re: [HACKERS] Packages: Again

2017-01-11 Thread Bruce Momjian
al: schema PL session variables proposal: session server side variables Is there anything that needs updating there, or it is just a question of getting someone to implement it? -- Bruce Momjian http://momjian.us EnterpriseDB

Re: [HACKERS] WARM and indirect indexes

2017-01-11 Thread Bruce Momjian
On Wed, Jan 11, 2017 at 12:24:55PM -0300, Alvaro Herrera wrote: > Bruce Momjian wrote: > > > Therefore, I think we need WARM done first, then we can test indirect > > indexes to see if they are a sufficient win to add it for the small > > percentage of users who will use i

Re: [HACKERS] WARM and indirect indexes

2017-01-11 Thread Bruce Momjian
On Wed, Jan 11, 2017 at 08:28:28AM -0500, Andrew Dunstan wrote: > > > On 01/10/2017 09:25 PM, Bruce Momjian wrote: > >I am not saying we shouldn't do it, but I am afraid that the complexity > >in figuring out when to use indirect indexes, combined with the number >

Re: [HACKERS] WARM and indirect indexes

2017-01-11 Thread Bruce Momjian
ances we should recommend the user of the feature. However, I am concerned that doing the two features serially (not in parallel) might mean that the second feature doesn't make it into Postgres 10, but considering we will live with this feature probably forever, I think it is the best course

Re: [HACKERS] WARM and indirect indexes

2017-01-11 Thread Bruce Momjian
On Tue, Jan 10, 2017 at 09:25:05PM -0500, Bruce Momjian wrote: > Thank you for the summary. I think we have to consider two things with > indirect indexes: > > 1. What percentage speedup is the _average_ user going to get? You > have to consider people who will use indirect ind

Re: [HACKERS] WARM and indirect indexes

2017-01-10 Thread Bruce Momjian
On Tue, Jan 10, 2017 at 11:36:24PM -0300, Alvaro Herrera wrote: > Bruce Momjian wrote: > > > 1. What percentage speedup is the _average_ user going to get? You > > have to consider people who will use indirect indexes who get no benefit > > or a net slowdown, and user

Re: [HACKERS] WARM and indirect indexes

2017-01-10 Thread Bruce Momjian
figuring out when to use indirect indexes, combined with the number of users who will try them, really hurts its inclusion. FYI, we have a similar issue in adding GUC variables, which I outlined in this blog post: http://momjian.us/main/blogs/pgblog/2009.html#January_10

<    1   2   3   4   5   6   7   8   9   10   >