Re: [HACKERS] Turning recovery.conf into GUCs

2015-02-19 Thread Josh Berkus
conf the way it is can't be what we go forward with. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] GSoC 2015 - mentors, students and admins.

2015-02-11 Thread Josh Berkus
On 02/11/2015 04:33 AM, Anastasia Lubennikova wrote: > I'm interested to participate as student again. Let's *really* hope for no US sanctions, then. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql

Re: [HACKERS] reducing our reliance on MD5

2015-02-10 Thread Josh Berkus
nd not much about PostgreSQL ca add support for X in a > relatively small number of lines of code. +1 -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-05 Thread Josh Berkus
the WAL that big all the time because it slows down backup snapshots. So I set min_wal_size to 2GB, roughly the size of one batch update. That's an idiosyncratic case, but I can imagine more of them out there. I wouldn't be opposed to min_wal_size = -1 meaning "same as max_wal_size

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-05 Thread Josh Berkus
On 02/05/2015 01:28 PM, Robert Haas wrote: > On Thu, Feb 5, 2015 at 2:11 PM, Josh Berkus wrote: >> Except that, when setting up servers for customers, one thing I pretty >> much always do for them is temporarily increase checkpoint_segments for >> the initial data load.

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-05 Thread Josh Berkus
stomers, one thing I pretty much always do for them is temporarily increase checkpoint_segments for the initial data load. So having Postgres do this automatically would be a feature, not a bug. I say we go with ~~ 1GB. That's an 8X increase over current default size for the maximum Defa

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-04 Thread Josh Berkus
On 02/04/2015 12:06 PM, Robert Haas wrote: > On Wed, Feb 4, 2015 at 1:05 PM, Josh Berkus wrote: >> Let me push "max_wal_size" and "min_wal_size" again as our new parameter >> names, because: >> >> * does what it says on the tin >> * new use

Re: [HACKERS] Getting rid of wal_level=archive and default to hot_standby + wal_senders

2015-02-04 Thread Josh Berkus
ng behavior: if logical_replication_slots > 0 wal_level = logical elif max_wal_senders > 0 wal_level = hot_standby elif archiving = on wal_level = archive (or hot_standby) else wal_level = minimal Given that this decision tree is the only possible decision t

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-04 Thread Josh Berkus
On 02/04/2015 09:28 AM, Robert Haas wrote: > On Tue, Feb 3, 2015 at 4:18 PM, Josh Berkus wrote: >>>> That's different from our current checkpoint_segments setting. With >>>> checkpoint_segments, the documented formula for calculating the disk usage >>&

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-03 Thread Josh Berkus
2MB instead (12 segments). That almost doubles our current real default, without requiring huge disk space which might surprise some users. In practice, checkpoint_segments is impossible to automatically tune correctly. So let's be conservative. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Release note bloat is getting out of hand

2015-02-02 Thread Josh Berkus
t insubstantial. Compared to the total size of the tarball, though ... -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Release note bloat is getting out of hand

2015-02-02 Thread Josh Berkus
is that it's very difficult to write an error-free importer from the present SGML file; there's just too much variation in how certain things are recorded, and SGML just isn't a database import format. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Release note bloat is getting out of hand

2015-02-01 Thread Josh Berkus
27;t need to be in our core docs, though. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Providing catalog view to pg_hba.conf file - Patch submission

2015-01-31 Thread Josh Berkus
ng; there's some tremendous utility potential in having a PostgresQL which can be 100% managed via port 5432, especially for the emerging world of container-based hosting (Docker et. al.). However, it's also going to be difficult. -- Josh Berkus PostgreSQL Experts Inc. http://pg

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-01-30 Thread Josh Berkus
Robert, Stephen, etc.: Apparently you can create a tablespace in the tablespace directory: josh=# create tablespace tbl location '/home/josh/pg94/data/pg_tblspc/'; CREATE TABLESPACE josh=# create table test_tbl ( test text ) tablespace tbl; CREATE TABLE josh=# \q josh@Radegast:~

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-01-30 Thread Josh Berkus
gt; involve having to actually map all the tablespaces... I like this idea too. And it would make tablespaces more manageable for people who are using them for reasons other than putting them on different disks. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_upgrade and rsync

2015-01-29 Thread Josh Berkus
On 01/29/2015 09:11 AM, Bruce Momjian wrote: > On Thu, Jan 29, 2015 at 12:09:58PM -0500, Andrew Dunstan wrote: >> Then step 2 should specify that it's for the master. > > Right. Josh is just listing all the steps --- the pg_upgrade docs > already have that spelled out in

Re: [HACKERS] pg_upgrade and rsync

2015-01-28 Thread Josh Berkus
/var/lib/postgresql, do: rsync -aHv --size-only -e ssh --itemize-changes /var/lib/postgresql/ replica-host:/var/lib/postgresql/ 9. Create a recovery.conf file in the replica's data directory with the appropriate parameters. 10. Start the master, then the replica -- Josh Berkus PostgreSQL Ex

Re: [HACKERS] pg_upgrade and rsync

2015-01-28 Thread Josh Berkus
On 01/28/2015 02:28 PM, Josh Berkus wrote: > On 01/28/2015 02:10 PM, Josh Berkus wrote: >> So 390MB were transferred out of a possible 474MB. That certainly seems >> like we're still transferring the majority of the data, even though I >> verified that the hard links ar

Re: [HACKERS] pg_upgrade and rsync

2015-01-28 Thread Josh Berkus
On 01/28/2015 02:10 PM, Josh Berkus wrote: > So 390MB were transferred out of a possible 474MB. That certainly seems > like we're still transferring the majority of the data, even though I > verified that the hard links are being sent as hard links. No? Looks like the majori

Re: [HACKERS] pg_upgrade and rsync

2015-01-28 Thread Josh Berkus
rified that the hard links are being sent as hard links. No? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] New CF app deployment

2015-01-26 Thread Josh Berkus
ut your remarks is that you've > basically made no contribution to the CommitFest process in years. Aside from being a CFM for one CF each year, of course. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] New CF app deployment

2015-01-26 Thread Josh Berkus
torm of criticism? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] New CF app deployment

2015-01-26 Thread Josh Berkus
t; individual messages in the email thread as being important. Will give it some thought and suggest something. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Parallel Seq Scan

2015-01-22 Thread Josh Berkus
On 01/22/2015 05:53 AM, Robert Haas wrote: > Also, I think testing with 2 workers is probably not enough. I think > we should test with 8 or even 16. FWIW, based on my experience there will also be demand to use parallel query using 4 workers, particularly on AWS. -- Josh Berkus Post

Re: [HACKERS] Proposal: knowing detail of config files via SQL

2015-01-22 Thread Josh Berkus
e copies of the same GUC, plus there's GUCs set interactively, as well as in the user and database properties. So you're looking at a lot of different "versions". I think you're in a position of needing to interest someone else in this issue enough to produce a patch to arg

Re: [HACKERS] New CF app deployment

2015-01-21 Thread Josh Berkus
Magnus, Now that I'm back on this side of the Pacific, is there any additional data entry/cleanup which needs doing? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Turning recovery.conf into GUCs

2015-01-14 Thread Josh Berkus
27;, > like recovery_target='xid 1234' or recovery_target='name bob'. Maybe > that would be more clear (or not). Not keen on non-atomic settings, personally. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (

Re: [HACKERS] Turning recovery.conf into GUCs

2015-01-08 Thread Josh Berkus
> is about right now, and the perspective of the user. That's rather a catch-22, isn't it? Last I checked, it was our policy to try to get smaller, more discrete patches rather than patches which try to change everything at once. -- Josh Berkus PostgreSQL Experts Inc. http://pgex

Re: [HACKERS] Turning recovery.conf into GUCs

2015-01-07 Thread Josh Berkus
On 01/07/2015 02:31 PM, Peter Eisentraut wrote: > On 1/6/15 7:33 PM, Josh Berkus wrote: >> D. Wierd name: for those doing only replication, not PITR, >> "recovery.conf" is completely baffling. > > I don't disagree, but "standby.enabled" or whate

Re: [HACKERS] Turning recovery.conf into GUCs

2015-01-06 Thread Josh Berkus
ew master. 4) only now allow reconnect That can all be handled externally to PostgreSQL. However, as long as we have a recovery.conf file which only gets read at server start, and at no other time, no external solution is possible. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com

Re: [HACKERS] Turning recovery.conf into GUCs

2015-01-06 Thread Josh Berkus
On 01/06/2015 04:42 PM, Andres Freund wrote: > On 2015-01-06 16:33:36 -0800, Josh Berkus wrote: >> F. Inability to remaster without restarting the replica. > > That has pretty much nothing to do with the topic at hand. It has *everything* to do with the topic at hand. The *only*

Re: [HACKERS] Turning recovery.conf into GUCs

2015-01-06 Thread Josh Berkus
ally. 3. Battle-tested way to make sure that replication/recovery state persists across unexpected restarts, and simple promotion workflow. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] Turning recovery.conf into GUCs

2015-01-06 Thread Josh Berkus
, there is a vast difference between a file which contains configuration variables and one which does not. That is, an *empty* recovery.conf file which just signals the start of recovery is not a configuration problem. The problem comes in in that recovery.conf serves two separate purposes: it's a c

[HACKERS] VODKA?

2015-01-06 Thread Josh Berkus
Oleg, Teodor: I take it VODKA is sliding to version 9.6? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Turning recovery.conf into GUCs

2015-01-05 Thread Josh Berkus
The amount of conditional logic I've had to write for Salt to handle recovery.conf truly doesn't bear thinking about. There may be some other way to make recovery.conf configuration-management friendly, but I haven't thought of it. -- Josh Berkus PostgreSQL Experts Inc. http://pg

Re: [HACKERS] Redesigning checkpoint_segments

2015-01-05 Thread Josh Berkus
On 01/05/2015 09:06 AM, Heikki Linnakangas wrote: > I wasn't clear on my opinion here. I think I understood what Josh meant, > but I don't think we should do it. Seems like unnecessary nannying of > the DBA. Let's just mention in the manual that if you set > wal_keep_s

Re: [HACKERS] Redesigning checkpoint_segments

2015-01-04 Thread Josh Berkus
On 01/03/2015 12:56 AM, Heikki Linnakangas wrote: > On 01/03/2015 12:28 AM, Josh Berkus wrote: >> On 01/02/2015 01:57 AM, Heikki Linnakangas wrote: >>> wal_keep_segments does not affect the calculation of CheckPointSegments. >>> If you set wal_keep_segments high enough,

Re: [HACKERS] Redesigning checkpoint_segments

2015-01-02 Thread Josh Berkus
nts is part of the max_wal_size total, NOT in addition to it? Just asking for clarification, here. I think that's a fine idea, I just want to make sure I understood you. The importance of wal_keep_segments will be fading as more people use replication slots. -- Josh Berkus PostgreSQL Experts

Re: [HACKERS] Redesigning checkpoint_segments

2014-12-31 Thread Josh Berkus
l_size if spread checkpoint is >> set to 0.9? Or does checkpoint kick in sometime earlier? > > It kicks in earlier, so that the checkpoint *completes* just when > checkpoint_wal_size of WAL is used up. So the real disk usage is > checkpoint_wal_size. Awesome. This makes me very

Re: [HACKERS] Commitfest problems

2014-12-19 Thread Josh Berkus
ld be recognized in some fashion, so > I'm not sure that this is really a good example. This really doesn't sound that difficult. If the committers include all actual reviewers in the commit messages, then it will be fairly easy for someone else (me) to go through and pick out the rela

Re: [HACKERS] Commitfest problems

2014-12-18 Thread Josh Berkus
no way we'll ever keep track. Note that updating the contributor list in the past has been slow due to lack of technology and process; if it's our way forward, then I'll apply myself to that problem. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-12-17 Thread Josh Berkus
On 12/17/2014 01:12 PM, Heikki Linnakangas wrote: > 3. Why is the specification required with ON CONFLICT UPDATE, but not > with ON CONFLICT IGNORE? Well, UPDATE has to know which row to lock, no? IGNORE does not. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via

Re: [HACKERS] On partitioning

2014-12-17 Thread Josh Berkus
On 12/17/2014 11:19 AM, Heikki Linnakangas wrote: > On 12/17/2014 08:53 PM, Josh Berkus wrote: >> Last week, I wrote a longish email listing out the common problems users >> have with our current partitioning as a way of benchmarking the plan for >> new partitioning. While

Re: [HACKERS] On partitioning

2014-12-17 Thread Josh Berkus
On 12/16/2014 07:35 PM, Robert Haas wrote: > On Tue, Dec 16, 2014 at 9:01 PM, Josh Berkus wrote: >> Anyway, what I'm saying is that I personally regard the inability to >> handle even moderately complex expressions a major failing of our >> existing partitioning scheme

Re: [HACKERS] On partitioning

2014-12-16 Thread Josh Berkus
moderately complex expressions a major failing of our existing partitioning scheme (possibly its worst single failing), and I would regard any new partitioning feature which didn't address that issue as suspect. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via

Re: [HACKERS] On partitioning

2014-12-16 Thread Josh Berkus
ot; to a constant. Right now, if you partition by an integer ID even, and do: SELECT * FROM partitioned_table WHERE ID = ( 3 + 4 ) ... postgres will scan all partitions because ( 3 + 4 ) is an expression and isn't evaluated until after CE is done. I don't think there's an easy way t

Re: [HACKERS] Commitfest problems

2014-12-16 Thread Josh Berkus
mber of "Needs rebase" comments > on patches, through no fault of the original submitter. This really should be taken care of by automation. Magnus's new system will be a significant step forwards on enabling that. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.c

Re: [HACKERS] Commitfest problems

2014-12-15 Thread Josh Berkus
On 12/15/2014 07:34 PM, Andres Freund wrote: > On 2014-12-15 16:14:30 -0800, Josh Berkus wrote: >> Read the thread on this list where I suggested crediting reviewers in >> the release notes. > > Man. You're equating stuff that's not the same. You didn't get you

Re: [HACKERS] Commitfest problems

2014-12-15 Thread Josh Berkus
On 12/15/2014 12:05 PM, Peter Geoghegan wrote: > On Mon, Dec 15, 2014 at 11:52 AM, Josh Berkus wrote: >> On 12/15/2014 11:27 AM, Robert Haas wrote: >>> I feel like we used to be better at encouraging people to participate >>> in the CF even if they were not experts,

Re: [HACKERS] Commitfest problems

2014-12-15 Thread Josh Berkus
were not really wanted, needed or helpful. As such, I stopped recruiting new reviewers (and, for that matter, doing them myself). I don't know if the same goes for anyone else. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hac

Re: [HACKERS] On partitioning

2014-12-12 Thread Josh Berkus
fact that the expression above gets rewritten to a constant by the time the query is executed; by then it's too late. To say nothing of functions like to_timestamp(), now(), etc. As long as partitions need to be chosen at plan time, I don't see a good way to fix the expression probl

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Josh Berkus
bunch of the emailing no? > > Yes. I can key in a bunch of the backlog of patches into the new app over the holidays, but not before then. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Josh Berkus
On 12/12/2014 11:52 AM, Magnus Hagander wrote: > On Fri, Dec 12, 2014 at 8:43 PM, Tomas Vondra wrote: >> On 11.12.2014 16:06, Bruce Momjian wrote: >>> On Wed, Dec 10, 2014 at 11:00:21PM -0800, Josh Berkus wrote: >> 3) manual processing that could be automated >> &

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Josh Berkus
per year for a non-profit in individual gifts alone. However, *I'm* not clear on what problems this non-profit employed person would be solving for the community. I doubt anyone else is either. Until we have consensus on that, there's no point in talking about anything else. --

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Josh Berkus
hanges to their individual workflow to fix it. Right now there is no consensus about moving forward in our patch review process; everyone seems to want the problem to go away without changing anything. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mail

Re: [HACKERS] Commitfest problems

2014-12-11 Thread Josh Berkus
On 12/11/2014 02:12 PM, Tom Lane wrote: > Josh Berkus writes: >> While the CFs are still doing (1), support for (2) ended sometime in the >> 9.3 development cycle. Partly this is because current CFMs are not >> permitted to take authoritative steps to ensure that the CF

Re: [HACKERS] Commitfest problems

2014-12-11 Thread Josh Berkus
are not permitted to take authoritative steps to ensure that the CF ends on time, and partly it's because of the increase in big complicated patches which just don't fit into the CF cycle. Speaking as the originator of commitfests, they were *always* intended to be a temporary measu

Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-11 Thread Josh Berkus
since pgCon, I'm expecting them for 9.6/whatever, not 9.5. There's a whole longish syntax discussion we haven't even started yet, let alone actual technical review. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-

Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-11 Thread Josh Berkus
ewers. So, given that you are inside the heart of the problem, do you have a solution other than your proposal above that we simply stop accepting new contributors? Or is that your solution? It would work, for some definition of "work". -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Commitfest problems

2014-12-11 Thread Josh Berkus
0-30% per CF over the years: http://www.databasesoup.com/2013/08/94-commitfest-1-wrap-up.html (I haven't done the rest of 9.4 yet or 9.5) No, it's not a jump up by 2X, but it is an upwards trend. And I think that Tom has it right that the additional patches we're seeing are addition

Re: [HACKERS] Commitfest problems

2014-12-10 Thread Josh Berkus
small patches > 2 more full-time developers creating big patches > 3 increased time demands on experienced Postgres developers I will add: 4. commitfest managers have burned out and refuse to do it again -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- S

Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-10 Thread Josh Berkus
On 12/10/2014 09:35 PM, Tom Lane wrote: > Josh Berkus writes: >> On 12/10/2014 05:14 PM, Stephen Frost wrote: >>> * Andres Freund (and...@2ndquadrant.com) wrote: >>>> But the scheduling of commits with regard to the 9.5 schedule actually >>>> opens a

Re: [HACKERS] logical column ordering

2014-12-10 Thread Josh Berkus
On 12/09/2014 09:11 PM, Tom Lane wrote: > Josh Berkus writes: >> Question on COPY, though: there's reasons why people would want COPY to >> dump in either physical or logical order. If you're doing COPY to >> create CSV files for output, then you want the columns

Re: [HACKERS] logical column ordering

2014-12-10 Thread Josh Berkus
he September release schedule. I agree > that having a later release would allow us to get more into it, but > there's a lot to be said for the consistency we've kept up over the past > few years with a September (our last non-September release was 8.4). Can we please NOT discus

Re: [HACKERS] logical column ordering

2014-12-09 Thread Josh Berkus
al or logical order. If you're doing COPY to create CSV files for output, then you want the columns in logical order. If you're doing COPY for pg_dump, then you want them in physical order for faster dump/reload. So we're almost certainly going to need to have an option for

Re: [HACKERS] moving from contrib to bin

2014-12-09 Thread Josh Berkus
reference pages in the documentation. +1 Considering that all of the above have been around for a while, it's kind of silly that they're still in contrib. Mostly that just guarantees that nobody will use them, even when it's appropriate. The one exception I might make above is pg_sta

Re: [HACKERS] On partitioning

2014-12-09 Thread Josh Berkus
of how partition by list over two columns (or more) would even work? You'd need an a*b number of partitions, and the user would be pretty much certain to miss a few value combinations. Maybe we should just restrict list partitioning to a single column for a first release, and wait and see if p

Re: [HACKERS] On partitioning

2014-12-08 Thread Josh Berkus
On 12/08/2014 02:12 PM, Jim Nasby wrote: > On 12/8/14, 12:26 PM, Josh Berkus wrote: >> 4. Creation Locking Problem >> high probability of lock pile-ups whenever a new partition is created on >> demand due to multiple backends trying to create the partition at the >>

Re: [HACKERS] On partitioning

2014-12-08 Thread Josh Berkus
the >> planner layer, we can't fix the existing major shortcomings of >> inheritance partitioning, such as its inability to handle expressions. >> Again, see previous. > > Huh? Explained in the other email I posted on this thread. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] On partitioning

2014-12-08 Thread Josh Berkus
itions exist exclusively at the planner layer, we can't fix the existing major shortcomings of inheritance partitioning, such as its inability to handle expressions. Again, see previous. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hac

Re: [HACKERS] On partitioning

2014-12-08 Thread Josh Berkus
Partitioning / NULLs in partition column This is the classic Active/Inactive By Month setup for partitions. Could be addressed via special handling for NULL/infinity in the partitioned column. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers m

Re: [HACKERS] Fractions in GUC variables

2014-12-07 Thread Josh Berkus
t; Oh, this is nice! Thanks for working on it. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Elusive segfault with 9.3.5 & query cancel

2014-12-05 Thread Josh Berkus
s because writing out the 16GB of shared buffers they had took over 10 minutes in a test. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Elusive segfault with 9.3.5 & query cancel

2014-12-05 Thread Josh Berkus
On 12/05/2014 12:54 PM, Josh Berkus wrote: > Hackers, > > This is not a complete enough report for a diagnosis. I'm posting it > here just in case someone else sees something like it, and having an > additional report will help figure out the underlying issue. > > *

[HACKERS] Elusive segfault with 9.3.5 & query cancel

2014-12-05 Thread Josh Berkus
fault since then. As far as the user is concerned, this solves the problem, so I'm never going to get a trace or a core dump file. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make chan

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-12-05 Thread Josh Berkus
currently does. I wouldn't be *opposed* to having a pseudocolumn in the RETURNed stuff which let me know updated|inserted|ignored, but I also don't see it as a feature requirement for 9.5. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailin

Re: [HACKERS] How about a option to disable autovacuum cancellation on lock conflict?

2014-12-02 Thread Josh Berkus
On 12/02/2014 11:08 AM, Andres Freund wrote: > On 2014-12-02 11:02:07 -0800, Josh Berkus wrote: >> On 12/02/2014 10:35 AM, Alvaro Herrera wrote: >>> If the table is large, the time window for this to happen is large also; >>> there might never be a time window la

Re: [HACKERS] How about a option to disable autovacuum cancellation on lock conflict?

2014-12-02 Thread Josh Berkus
autovacuum ... would be to give vacuum a way to track which blocks an incomplete vacuum had already visited. This would be even more valuable for freeze. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make ch

Re: [HACKERS] Turning recovery.conf into GUCs

2014-12-02 Thread Josh Berkus
On 12/02/2014 10:31 AM, Alvaro Herrera wrote: > Josh Berkus wrote: >> On 12/02/2014 06:25 AM, Alex Shulgin wrote: > >>> Whatever tricks we might employ will likely >>> be defeated by the fact that the oldschool user will fail to *include* >>> recovery.conf

Re: [HACKERS] Turning recovery.conf into GUCs

2014-12-02 Thread Josh Berkus
and we might as well stick with the awkward method we know. > Whatever tricks we might employ will likely > be defeated by the fact that the oldschool user will fail to *include* > recovery.conf in the main conf file. Well, can we merge this patch and then fight out what to

Re: [HACKERS] bug in json_to_record with arrays

2014-12-01 Thread Josh Berkus
On 12/01/2014 12:30 PM, Tom Lane wrote: > Josh Berkus writes: >> On 11/28/2014 12:55 PM, Tom Lane wrote: >>> * This would only really address Josh's complaint if we were to back-patch >>> it into 9.4, but I'm hesitant to change error texts post-RC1. Thoughts

Re: [HACKERS] bug in json_to_record with arrays

2014-12-01 Thread Josh Berkus
7;s as minor as things get. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] no test programs in contrib

2014-11-28 Thread Josh Berkus
That doesn't mean test_decoding needs to stay in contrib, just that it needs to be somewhere which goes into some common package. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] no test programs in contrib

2014-11-27 Thread Josh Berkus
enough about it ... Stephen/Kaigai? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] bug in json_to_record with arrays

2014-11-26 Thread Josh Berkus
you'd get out of most other input functions > that were feeling indigestion? Yes, it most definitely would be better. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] bug in json_to_record with arrays

2014-11-26 Thread Josh Berkus
On 11/26/2014 11:54 AM, Josh Berkus wrote: > Tested on 9.4b3, 9.4rc1, 9.5devel > > select * from json_to_record(' > {"id":1,"val":"josh","valry":["potter","chef","programmer"]}') as r(id > int, val te

[HACKERS] bug in json_to_record with arrays

2014-11-26 Thread Josh Berkus
Tested on 9.4b3, 9.4rc1, 9.5devel select * from json_to_record(' {"id":1,"val":"josh","valry":["potter","chef","programmer"]}') as r(id int, val text, valry text[]); ERROR: missing dimension value With some exper

Re: [HACKERS] Turning recovery.conf into GUCs

2014-11-24 Thread Josh Berkus
On 11/24/2014 02:18 PM, Alex Shulgin wrote: > > Josh Berkus writes: >>>> >>>> Before I go into my ideas, though, what does the current patch do >>>> regarding non-replication PITR? >>> >>> It removes that $PGDATA/standby.enable trigge

Re: [HACKERS] Turning recovery.conf into GUCs

2014-11-24 Thread Josh Berkus
On 11/24/2014 02:00 PM, Alex Shulgin wrote: > > Josh Berkus writes: >>> >>> only that you need to start in recovery mode to start replication >> >> Right, but my point is that having a trigger file *is not necessary for >> replication, only for PITR

Re: [HACKERS] Disabling auto.conf WAS: Turning recovery.conf into GUCs

2014-11-24 Thread Josh Berkus
TABASE and ALTER ROLE, and that's a LOT less auditable than pg.auto.conf is. Heck, we don't even have a good system view for SET parameters on DB objects. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Turning recovery.conf into GUCs

2014-11-24 Thread Josh Berkus
meter is 100% sufficient to control replication with the small detail that "pg_ctl promote" needs to set it in pg.auto.conf or conf.d. And, now, having given it some thought, I'm going to argue that it's not required for PITR either, provided that we can use the auto.conf method

[HACKERS] Should the removal of SnapshotNow be in the compatibility warnings for 9.4?

2014-11-21 Thread Josh Berkus
... I think it should. It'll break some extensions, so we should warn people about it more prominently. Robert's text lower down in the release notes is fine, but we should put a more prominent warning at the top with the other backwards compatibility breakage. -- Josh Berkus

Re: [HACKERS] Turning recovery.conf into GUCs

2014-11-21 Thread Josh Berkus
On 11/21/2014 10:54 AM, Stephen Frost wrote: > * Josh Berkus (j...@agliodbs.com) wrote: >>> Either way, from the code it is clear that we only stay in recovery if >>> standby_mode is directly turned on. This makes the whole check for a >>> specially named file un

Re: [HACKERS] pg_multixact not getting truncated

2014-11-21 Thread Josh Berkus
On 11/21/2014 10:44 AM, Josh Berkus wrote: > Greg, > > >> This is actually the way it used to be. It was changed because it was >> discovered there was some case where an unfrozen xid would end up in >> template0 anyways and for some reason it was hard to be sure to

Re: [HACKERS] pg_multixact not getting truncated

2014-11-21 Thread Josh Berkus
t get rid of template0 and come up with some other mechanism to reset template1 to bare-bones state. Actually, here's a question ... pg_clog is usually smaller than I think it should be (that is, smaller than 4bytes * XID_age). Why is that? -- Josh Berkus PostgreSQL Experts Inc. http://pge

Re: [HACKERS] Turning recovery.conf into GUCs

2014-11-21 Thread Josh Berkus
in recovery file" is for PITR rather than for replication, where it has a number of advantages as a method, the main one being that recovery.conf is unlikely to be overwritten by the contents of the backup. HOWEVER, there's a clear out for this with conf.d. If we enable conf.d by default,

Re: [HACKERS] pg_multixact not getting truncated

2014-11-20 Thread Josh Berkus
On 11/20/2014 01:47 PM, Tom Lane wrote: > Josh Berkus writes: >> Well, the first thing that comes to mind is that template0 should be >> "permanently frozen". That is, all objects in it should be created with >> frozen xid and mxids. After all, nobody can modify a

Re: [HACKERS] pg_multixact not getting truncated

2014-11-20 Thread Josh Berkus
On 11/20/2014 01:03 PM, Robert Haas wrote: > On Thu, Nov 20, 2014 at 3:44 PM, Josh Berkus wrote: >>> So that we can have two ways to lose data? >>> >>> Forbidding connections to a database doesn't prevent XID or MXID wraparound. >> >> It does preven

Re: [HACKERS] pg_multixact not getting truncated

2014-11-20 Thread Josh Berkus
> So that we can have two ways to lose data? > > Forbidding connections to a database doesn't prevent XID or MXID wraparound. It does prevent the user from doing anything about it, though, since they can't manually vacuum template0 without knowing unpublished hacker

<    1   2   3   4   5   6   7   8   9   10   >