Re: [HACKERS] pg_upgrade and rsync

2015-01-28 Thread Josh Berkus
On 01/28/2015 02:10 PM, Josh Berkus wrote: So 390MB were transferred out of a possible 474MB. That certainly seems like we're still transferring the majority of the data, even though I verified that the hard links are being sent as hard links. No? Looks like the majority of that was pg_xlog

Re: [HACKERS] pg_upgrade and rsync

2015-01-28 Thread Josh Berkus
On 01/28/2015 02:28 PM, Josh Berkus wrote: On 01/28/2015 02:10 PM, Josh Berkus wrote: So 390MB were transferred out of a possible 474MB. That certainly seems like we're still transferring the majority of the data, even though I verified that the hard links are being sent as hard links

Re: [HACKERS] pg_upgrade and rsync

2015-01-28 Thread Josh Berkus
-changes /var/lib/postgresql/ replica-host:/var/lib/postgresql/ 9. Create a recovery.conf file in the replica's data directory with the appropriate parameters. 10. Start the master, then the replica -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list

Re: [HACKERS] New CF app deployment

2015-01-26 Thread Josh Berkus
? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] New CF app deployment

2015-01-26 Thread Josh Berkus
as being important. Will give it some thought and suggest something. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] New CF app deployment

2015-01-26 Thread Josh Berkus
that way. If you're going to throw around negative hyperbole, expect to get it thrown back at you. What particularly peeves me about your remarks is that you've basically made no contribution to the CommitFest process in years. Aside from being a CFM for one CF each year, of course. -- Josh

Re: [HACKERS] Parallel Seq Scan

2015-01-22 Thread Josh Berkus
On 01/22/2015 05:53 AM, Robert Haas wrote: Also, I think testing with 2 workers is probably not enough. I think we should test with 8 or even 16. FWIW, based on my experience there will also be demand to use parallel query using 4 workers, particularly on AWS. -- Josh Berkus PostgreSQL

Re: [HACKERS] Proposal: knowing detail of config files via SQL

2015-01-22 Thread Josh Berkus
, as well as in the user and database properties. So you're looking at a lot of different versions. I think you're in a position of needing to interest someone else in this issue enough to produce a patch to argue about. I'm not seeing a lot of interest in it here. -- Josh Berkus PostgreSQL

Re: [HACKERS] New CF app deployment

2015-01-21 Thread Josh Berkus
Magnus, Now that I'm back on this side of the Pacific, is there any additional data entry/cleanup which needs doing? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] Turning recovery.conf into GUCs

2015-01-14 Thread Josh Berkus
that would be more clear (or not). Not keen on non-atomic settings, personally. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Turning recovery.conf into GUCs

2015-01-08 Thread Josh Berkus
I checked, it was our policy to try to get smaller, more discrete patches rather than patches which try to change everything at once. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Turning recovery.conf into GUCs

2015-01-07 Thread Josh Berkus
On 01/07/2015 02:31 PM, Peter Eisentraut wrote: On 1/6/15 7:33 PM, Josh Berkus wrote: D. Wierd name: for those doing only replication, not PITR, recovery.conf is completely baffling. I don't disagree, but standby.enabled or whatever isn't any better, for the inverse reason. Yeah. Like I

Re: [HACKERS] Turning recovery.conf into GUCs

2015-01-06 Thread Josh Berkus
signals the start of recovery is not a configuration problem. The problem comes in in that recovery.conf serves two separate purposes: it's a configuration file, and it's also a trigger file. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql

Re: [HACKERS] Turning recovery.conf into GUCs

2015-01-06 Thread Josh Berkus
that replication/recovery state persists across unexpected restarts, and simple promotion workflow. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

[HACKERS] VODKA?

2015-01-06 Thread Josh Berkus
Oleg, Teodor: I take it VODKA is sliding to version 9.6? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Turning recovery.conf into GUCs

2015-01-06 Thread Josh Berkus
now allow reconnect That can all be handled externally to PostgreSQL. However, as long as we have a recovery.conf file which only gets read at server start, and at no other time, no external solution is possible. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql

Re: [HACKERS] Turning recovery.conf into GUCs

2015-01-06 Thread Josh Berkus
On 01/06/2015 04:42 PM, Andres Freund wrote: On 2015-01-06 16:33:36 -0800, Josh Berkus wrote: F. Inability to remaster without restarting the replica. That has pretty much nothing to do with the topic at hand. It has *everything* to do with the topic at hand. The *only* reason we can't

Re: [HACKERS] Turning recovery.conf into GUCs

2015-01-05 Thread Josh Berkus
recovery.conf truly doesn't bear thinking about. There may be some other way to make recovery.conf configuration-management friendly, but I haven't thought of it. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] Redesigning checkpoint_segments

2015-01-05 Thread Josh Berkus
On 01/05/2015 09:06 AM, Heikki Linnakangas wrote: I wasn't clear on my opinion here. I think I understood what Josh meant, but I don't think we should do it. Seems like unnecessary nannying of the DBA. Let's just mention in the manual that if you set wal_keep_segments higher than [insert

Re: [HACKERS] Redesigning checkpoint_segments

2015-01-04 Thread Josh Berkus
On 01/03/2015 12:56 AM, Heikki Linnakangas wrote: On 01/03/2015 12:28 AM, Josh Berkus wrote: On 01/02/2015 01:57 AM, Heikki Linnakangas wrote: wal_keep_segments does not affect the calculation of CheckPointSegments. If you set wal_keep_segments high enough, checkpoint_wal_size

Re: [HACKERS] Redesigning checkpoint_segments

2015-01-02 Thread Josh Berkus
, NOT in addition to it? Just asking for clarification, here. I think that's a fine idea, I just want to make sure I understood you. The importance of wal_keep_segments will be fading as more people use replication slots. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via

Re: [HACKERS] Redesigning checkpoint_segments

2014-12-31 Thread Josh Berkus
. Awesome. This makes me very happy. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Commitfest problems

2014-12-19 Thread Josh Berkus
in the commit messages, then it will be fairly easy for someone else (me) to go through and pick out the relative handful of people who aren't already on the contributors list and check the level of their contributions. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via

Re: [HACKERS] Commitfest problems

2014-12-18 Thread Josh Berkus
that updating the contributor list in the past has been slow due to lack of technology and process; if it's our way forward, then I'll apply myself to that problem. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: [HACKERS] On partitioning

2014-12-17 Thread Josh Berkus
On 12/17/2014 11:19 AM, Heikki Linnakangas wrote: On 12/17/2014 08:53 PM, Josh Berkus wrote: Last week, I wrote a longish email listing out the common problems users have with our current partitioning as a way of benchmarking the plan for new partitioning. While some people responded

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-12-17 Thread Josh Berkus
On 12/17/2014 01:12 PM, Heikki Linnakangas wrote: 3. Why is the specification required with ON CONFLICT UPDATE, but not with ON CONFLICT IGNORE? Well, UPDATE has to know which row to lock, no? IGNORE does not. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql

Re: [HACKERS] Commitfest problems

2014-12-16 Thread Josh Berkus
no fault of the original submitter. This really should be taken care of by automation. Magnus's new system will be a significant step forwards on enabling that. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: [HACKERS] On partitioning

2014-12-16 Thread Josh Berkus
= ( 3 + 4 ) ... postgres will scan all partitions because ( 3 + 4 ) is an expression and isn't evaluated until after CE is done. I don't think there's an easy way to do the expression rewrite while we're still in planning, is there? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com

Re: [HACKERS] On partitioning

2014-12-16 Thread Josh Berkus
new partitioning feature which didn't address that issue as suspect. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Commitfest problems

2014-12-15 Thread Josh Berkus
On 12/15/2014 12:05 PM, Peter Geoghegan wrote: On Mon, Dec 15, 2014 at 11:52 AM, Josh Berkus j...@agliodbs.com wrote: On 12/15/2014 11:27 AM, Robert Haas wrote: I feel like we used to be better at encouraging people to participate in the CF even if they were not experts, and to do the best

Re: [HACKERS] Commitfest problems

2014-12-15 Thread Josh Berkus
On 12/15/2014 07:34 PM, Andres Freund wrote: On 2014-12-15 16:14:30 -0800, Josh Berkus wrote: Read the thread on this list where I suggested crediting reviewers in the release notes. Man. You're equating stuff that's not the same. You didn't get your way (and I'm tentatively on your side

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Josh Berkus
it. Right now there is no consensus about moving forward in our patch review process; everyone seems to want the problem to go away without changing anything. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Josh Berkus
. However, *I'm* not clear on what problems this non-profit employed person would be solving for the community. I doubt anyone else is either. Until we have consensus on that, there's no point in talking about anything else. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Josh Berkus
On 12/12/2014 11:52 AM, Magnus Hagander wrote: On Fri, Dec 12, 2014 at 8:43 PM, Tomas Vondra t...@fuzzy.cz wrote: On 11.12.2014 16:06, Bruce Momjian wrote: On Wed, Dec 10, 2014 at 11:00:21PM -0800, Josh Berkus wrote: 3) manual processing that could be automated I think the CF site

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Josh Berkus
then. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] On partitioning

2014-12-12 Thread Josh Berkus
; by then it's too late. To say nothing of functions like to_timestamp(), now(), etc. As long as partitions need to be chosen at plan time, I don't see a good way to fix the expression problem. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Commitfest problems

2014-12-11 Thread Josh Berkus
or 9.5) No, it's not a jump up by 2X, but it is an upwards trend. And I think that Tom has it right that the additional patches we're seeing are additional large, complex patches. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-11 Thread Josh Berkus
. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-11 Thread Josh Berkus
for 9.6/whatever, not 9.5. There's a whole longish syntax discussion we haven't even started yet, let alone actual technical review. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Commitfest problems

2014-12-11 Thread Josh Berkus
it's because of the increase in big complicated patches which just don't fit into the CF cycle. Speaking as the originator of commitfests, they were *always* intended to be a temporary measure, a step on the way to something else like continuous integration. -- Josh Berkus PostgreSQL Experts Inc

Re: [HACKERS] Commitfest problems

2014-12-11 Thread Josh Berkus
On 12/11/2014 02:12 PM, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: While the CFs are still doing (1), support for (2) ended sometime in the 9.3 development cycle. Partly this is because current CFMs are not permitted to take authoritative steps to ensure that the CF ends on time

Re: [HACKERS] logical column ordering

2014-12-10 Thread Josh Berkus
more into it, but there's a lot to be said for the consistency we've kept up over the past few years with a September (our last non-September release was 8.4). Can we please NOT discuss this in the thread about someone's patch? Thanks. -- Josh Berkus PostgreSQL Experts Inc. http

Re: [HACKERS] logical column ordering

2014-12-10 Thread Josh Berkus
On 12/09/2014 09:11 PM, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: Question on COPY, though: there's reasons why people would want COPY to dump in either physical or logical order. If you're doing COPY to create CSV files for output, then you want the columns in logical order

Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-10 Thread Josh Berkus
On 12/10/2014 09:35 PM, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: On 12/10/2014 05:14 PM, Stephen Frost wrote: * Andres Freund (and...@2ndquadrant.com) wrote: But the scheduling of commits with regard to the 9.5 schedule actually opens a relevant question: When are we planning

Re: [HACKERS] Commitfest problems

2014-12-10 Thread Josh Berkus
developers creating big patches 3 increased time demands on experienced Postgres developers I will add: 4. commitfest managers have burned out and refuse to do it again -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] On partitioning

2014-12-09 Thread Josh Berkus
, and the user would be pretty much certain to miss a few value combinations. Maybe we should just restrict list partitioning to a single column for a first release, and wait and see if people ask for more? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing

Re: [HACKERS] moving from contrib to bin

2014-12-09 Thread Josh Berkus
of the above have been around for a while, it's kind of silly that they're still in contrib. Mostly that just guarantees that nobody will use them, even when it's appropriate. The one exception I might make above is pg_standby. What do we need this for today, exactly? -- Josh Berkus PostgreSQL

Re: [HACKERS] logical column ordering

2014-12-09 Thread Josh Berkus
to create CSV files for output, then you want the columns in logical order. If you're doing COPY for pg_dump, then you want them in physical order for faster dump/reload. So we're almost certainly going to need to have an option for COPY. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com

Re: [HACKERS] On partitioning

2014-12-08 Thread Josh Berkus
setup for partitions. Could be addressed via special handling for NULL/infinity in the partitioned column. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] On partitioning

2014-12-08 Thread Josh Berkus
partitioning, such as its inability to handle expressions. Again, see previous. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] On partitioning

2014-12-08 Thread Josh Berkus
. On the other hand, as long as partitions exist exclusively at the planner layer, we can't fix the existing major shortcomings of inheritance partitioning, such as its inability to handle expressions. Again, see previous. Huh? Explained in the other email I posted on this thread. -- Josh Berkus

Re: [HACKERS] On partitioning

2014-12-08 Thread Josh Berkus
On 12/08/2014 02:12 PM, Jim Nasby wrote: On 12/8/14, 12:26 PM, Josh Berkus wrote: 4. Creation Locking Problem high probability of lock pile-ups whenever a new partition is created on demand due to multiple backends trying to create the partition at the same time. Not Addressed? Do users

Re: [HACKERS] Fractions in GUC variables

2014-12-07 Thread Josh Berkus
On 12/07/2014 11:48 AM, John Gorman wrote: This patch implements the first wiki/Todo Configuration Files item Consider normalizing fractions in postgresql.conf, perhaps using '%'. The Fractions in GUC variables discussion is here. Oh, this is nice! Thanks for working on it. -- Josh

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-12-05 Thread Josh Berkus
be *opposed* to having a pseudocolumn in the RETURNed stuff which let me know updated|inserted|ignored, but I also don't see it as a feature requirement for 9.5. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

[HACKERS] Elusive segfault with 9.3.5 query cancel

2014-12-05 Thread Josh Berkus
as the user is concerned, this solves the problem, so I'm never going to get a trace or a core dump file. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] Elusive segfault with 9.3.5 query cancel

2014-12-05 Thread Josh Berkus
On 12/05/2014 12:54 PM, Josh Berkus wrote: Hackers, This is not a complete enough report for a diagnosis. I'm posting it here just in case someone else sees something like it, and having an additional report will help figure out the underlying issue. * 700GB database with around 5,000

Re: [HACKERS] Elusive segfault with 9.3.5 query cancel

2014-12-05 Thread Josh Berkus
buffers they had took over 10 minutes in a test. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Turning recovery.conf into GUCs

2014-12-02 Thread Josh Berkus
be defeated by the fact that the oldschool user will fail to *include* recovery.conf in the main conf file. Well, can we merge this patch and then fight out what to do for the transitional users as a separate patch? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql

Re: [HACKERS] Turning recovery.conf into GUCs

2014-12-02 Thread Josh Berkus
On 12/02/2014 10:31 AM, Alvaro Herrera wrote: Josh Berkus wrote: On 12/02/2014 06:25 AM, Alex Shulgin wrote: Whatever tricks we might employ will likely be defeated by the fact that the oldschool user will fail to *include* recovery.conf in the main conf file. Well, can we merge

Re: [HACKERS] How about a option to disable autovacuum cancellation on lock conflict?

2014-12-02 Thread Josh Berkus
vacuum a way to track which blocks an incomplete vacuum had already visited. This would be even more valuable for freeze. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] How about a option to disable autovacuum cancellation on lock conflict?

2014-12-02 Thread Josh Berkus
On 12/02/2014 11:08 AM, Andres Freund wrote: On 2014-12-02 11:02:07 -0800, Josh Berkus wrote: On 12/02/2014 10:35 AM, Alvaro Herrera wrote: If the table is large, the time window for this to happen is large also; there might never be a time window large enough between two lock acquisitions

Re: [HACKERS] bug in json_to_record with arrays

2014-12-01 Thread Josh Berkus
. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] bug in json_to_record with arrays

2014-12-01 Thread Josh Berkus
On 12/01/2014 12:30 PM, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: On 11/28/2014 12:55 PM, Tom Lane wrote: * This would only really address Josh's complaint if we were to back-patch it into 9.4, but I'm hesitant to change error texts post-RC1. Thoughts? If we don't fix an error

Re: [HACKERS] no test programs in contrib

2014-11-28 Thread Josh Berkus
decoding setup or is the decoding plugin broken? And I can imagine quite a few users who don't have source installs needing to check that. That doesn't mean test_decoding needs to stay in contrib, just that it needs to be somewhere which goes into some common package. -- Josh Berkus PostgreSQL

Re: [HACKERS] no test programs in contrib

2014-11-27 Thread Josh Berkus
? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] bug in json_to_record with arrays

2014-11-26 Thread Josh Berkus
or json_to_recordset. I know this worked back in January, though. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] bug in json_to_record with arrays

2014-11-26 Thread Josh Berkus
On 11/26/2014 11:54 AM, Josh Berkus wrote: Tested on 9.4b3, 9.4rc1, 9.5devel select * from json_to_record(' {id:1,val:josh,valry:[potter,chef,programmer]}') as r(id int, val text, valry text[]); ERROR: missing dimension value With some experimentation, I can't find any way to convert

Re: [HACKERS] Turning recovery.conf into GUCs

2014-11-24 Thread Josh Berkus
it in pg.auto.conf or conf.d. And, now, having given it some thought, I'm going to argue that it's not required for PITR either, provided that we can use the auto.conf method. Before I go into my ideas, though, what does the current patch do regarding non-replication PITR? -- Josh Berkus PostgreSQL Experts

Re: [HACKERS] Disabling auto.conf WAS: Turning recovery.conf into GUCs

2014-11-24 Thread Josh Berkus
auto.conf was found not to be worthwhile is that anyone with superuser rights can *already* change the config by using ALTER DATABASE and ALTER ROLE, and that's a LOT less auditable than pg.auto.conf is. Heck, we don't even have a good system view for SET parameters on DB objects. -- Josh Berkus

Re: [HACKERS] Turning recovery.conf into GUCs

2014-11-24 Thread Josh Berkus
On 11/24/2014 02:00 PM, Alex Shulgin wrote: Josh Berkus j...@agliodbs.com writes: only that you need to start in recovery mode to start replication Right, but my point is that having a trigger file *is not necessary for replication, only for PITR* -- and maybe not necessary even for PITR

Re: [HACKERS] Turning recovery.conf into GUCs

2014-11-24 Thread Josh Berkus
On 11/24/2014 02:18 PM, Alex Shulgin wrote: Josh Berkus j...@agliodbs.com writes: Before I go into my ideas, though, what does the current patch do regarding non-replication PITR? It removes that $PGDATA/standby.enable trigger file it relies on to start the PITR in the first place. OK

Re: [HACKERS] Turning recovery.conf into GUCs

2014-11-21 Thread Josh Berkus
chooses) will not be part of backups, it would have the same advantage as recovery.conf, without the drawbacks. Discuss? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] pg_multixact not getting truncated

2014-11-21 Thread Josh Berkus
with some other mechanism to reset template1 to bare-bones state. Actually, here's a question ... pg_clog is usually smaller than I think it should be (that is, smaller than 4bytes * XID_age). Why is that? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers

Re: [HACKERS] pg_multixact not getting truncated

2014-11-21 Thread Josh Berkus
On 11/21/2014 10:44 AM, Josh Berkus wrote: Greg, This is actually the way it used to be. It was changed because it was discovered there was some case where an unfrozen xid would end up in template0 anyways and for some reason it was hard to be sure to avoid it. I don't recall exactly what

Re: [HACKERS] Turning recovery.conf into GUCs

2014-11-21 Thread Josh Berkus
On 11/21/2014 10:54 AM, Stephen Frost wrote: * Josh Berkus (j...@agliodbs.com) wrote: Either way, from the code it is clear that we only stay in recovery if standby_mode is directly turned on. This makes the whole check for a specially named file unnecessary, IMO: we should just check

[HACKERS] Should the removal of SnapshotNow be in the compatibility warnings for 9.4?

2014-11-21 Thread Josh Berkus
... I think it should. It'll break some extensions, so we should warn people about it more prominently. Robert's text lower down in the release notes is fine, but we should put a more prominent warning at the top with the other backwards compatibility breakage. -- Josh Berkus PostgreSQL

Re: [HACKERS] pg_multixact not getting truncated

2014-11-20 Thread Josh Berkus
So that we can have two ways to lose data? Forbidding connections to a database doesn't prevent XID or MXID wraparound. It does prevent the user from doing anything about it, though, since they can't manually vacuum template0 without knowing unpublished hackery. -- Josh Berkus PostgreSQL

Re: [HACKERS] pg_multixact not getting truncated

2014-11-20 Thread Josh Berkus
On 11/20/2014 01:03 PM, Robert Haas wrote: On Thu, Nov 20, 2014 at 3:44 PM, Josh Berkus j...@agliodbs.com wrote: So that we can have two ways to lose data? Forbidding connections to a database doesn't prevent XID or MXID wraparound. It does prevent the user from doing anything about

Re: [HACKERS] pg_multixact not getting truncated

2014-11-20 Thread Josh Berkus
On 11/20/2014 01:47 PM, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: Well, the first thing that comes to mind is that template0 should be permanently frozen. That is, all objects in it should be created with frozen xid and mxids. After all, nobody can modify anything

Re: [HACKERS] pg_multixact not getting truncated

2014-11-19 Thread Josh Berkus
, but datminxid is set to the value that is current when it is frozen. So, to follow up on this: it seems to me that we shouldn't be requiring freezing for databases where allowconn=false. This seems like a TODO to me, even possibly a backpatchable bug fix. -- Josh Berkus PostgreSQL Experts Inc. http

Re: [HACKERS] pg_multixact not getting truncated

2014-11-19 Thread Josh Berkus
On 11/19/2014 01:03 PM, Alvaro Herrera wrote: Josh Berkus wrote: On 11/12/2014 06:57 PM, Alvaro Herrera wrote: How did template0 even get a MultiXact? That sounds like they're really abusing the template databases. :( (Do keep in mind that MXID 1 is a special value.) No, it's normal

[HACKERS] Merging recovery.conf into PostgreSQL.conf -- where did this go?

2014-11-19 Thread Josh Berkus
to this? I kinda expected it to get committed right after we opened 9.5. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] pg_locks doesn't check for interrupts?

2014-11-18 Thread Josh Berkus
minutes* when the database is heavily loaded. This it seems likely to me that the functions under pg_locks aren't checking for interrupts. Anybody checked this already? (yes, when a 64,000 item lock table is mostly full of locks, queries against pg_locks *can* take 10 minutes) -- Josh Berkus

Re: [HACKERS] pg_locks doesn't check for interrupts?

2014-11-18 Thread Josh Berkus
On 11/18/2014 10:47 AM, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: Since querying pg_locks can be intrusive due to needing to lock the lock partitions, when I'm collecting data about locks I generally put a statement_timeout on it. However, I'm noticing that this statement_timeout

Re: [HACKERS] recovery_target_time and standby_mode

2014-11-12 Thread Josh Berkus
On 11/07/2014 02:03 PM, Josh Berkus wrote: But, like I said, there's a serviceable workaround. Some update on this. We've seen a problem in production with this setup which I can't reproduce as a test case, but which may jog Heikki's memory for something to fix. 1. Recover master to 2014-11-10

Re: [HACKERS] recovery_target_time and standby_mode

2014-11-12 Thread Josh Berkus
there, not from the point it was at when you shut down. Except that in the problem case, it appears to be going *forwards*. What would cause that? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

[HACKERS] Missing line for postgresql.auto.conf?

2014-11-10 Thread Josh Berkus
Hackers, I thought 9.4's postgresql.conf.sample was supposed to have a commented-out line for postgresql.auto.conf? In the includes section? It's not there. If we don't give folks a commented-out line to uncomment, it'll be pretty hard for them to figure out auto.conf ... -- Josh Berkus

Re: [HACKERS] Missing line for postgresql.auto.conf?

2014-11-10 Thread Josh Berkus
On 11/10/2014 10:48 PM, Josh Berkus wrote: Hackers, I thought 9.4's postgresql.conf.sample was supposed to have a commented-out line for postgresql.auto.conf? In the includes section? It's not there. If we don't give folks a commented-out line to uncomment, it'll be pretty hard for them

Re: [HACKERS] pg_multixact not getting truncated

2014-11-09 Thread Josh Berkus
on busy databases. There's too many sites which service degrades noticeably during a full table vacuum. Me too: https://github.com/pgexperts/flexible-freeze -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] pg_multixact not getting truncated

2014-11-09 Thread Josh Berkus
On 11/09/2014 08:00 PM, Josh Berkus wrote: On 11/08/2014 01:46 PM, Andres Freund wrote: I'm these days suggesting that people should add manual vacuuming for older relations during off peak hours on busy databases. There's too many sites which service degrades noticeably during a full table

Re: [HACKERS] pg_multixact not getting truncated

2014-11-08 Thread Josh Berkus
On 11/07/2014 05:29 PM, Alvaro Herrera wrote: Josh Berkus wrote: Of course, this will lead to LOTs of additional vacuuming ... There's a trade-off here: more vacuuming I/O usage for less disk space used. How stressed your customers really are about 1 GB of disk space? These customers

Re: [HACKERS] recovery_target_time and standby_mode

2014-11-07 Thread Josh Berkus
On 11/07/2014 08:12 AM, Robert Haas wrote: On Wed, Nov 5, 2014 at 9:15 PM, Josh Berkus j...@agliodbs.com wrote: What I'm pointing out is that you can't actually do that. You think you can, but you can't. I do think that. You haven't explained why I'm wrong; just asserted than I am. Which

Re: [HACKERS] recovery_target_time and standby_mode

2014-11-07 Thread Josh Berkus
current behavior is correct or not. For my part, I would like to have a different interacton, but I think that's a future feature rather than a bug, as long as we do the stuff in the Yes column. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list

Re: [HACKERS] recovery_target_time and standby_mode

2014-11-07 Thread Josh Berkus
forked timeline 1 before current recovery point 320/47e0. In order for this to work, the archive would need to stop before recovery_target_time. On 11/07/2014 12:07 PM, Robert Haas wrote: On Fri, Nov 7, 2014 at 1:40 PM, Josh Berkus j...@agliodbs.com wrote: Is the current interaction

Re: [HACKERS] recovery_target_time and standby_mode

2014-11-07 Thread Josh Berkus
On 11/07/2014 01:30 PM, Robert Haas wrote: On Fri, Nov 7, 2014 at 4:00 PM, Josh Berkus j...@agliodbs.com wrote: In order for this to work, the archive would need to stop before recovery_target_time. Yeah, good point. I didn't think of the case where you've rewound the master

Re: [HACKERS] pg_multixact not getting truncated

2014-11-07 Thread Josh Berkus
On 11/05/2014 11:15 AM, Josh Berkus wrote: On 11/05/2014 10:40 AM, Jim Nasby wrote: Can you post the contents of pg_multixact/members/? Well, not as of last week, obviously. https://gist.github.com/jberkus/d05db3629e8c898664c4 I haven't pasted all the filenames, because, well, look

Re: [HACKERS] pg_multixact not getting truncated

2014-11-07 Thread Josh Berkus
... -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] to_char_at_timezone()?

2014-11-05 Thread Josh Berkus
as an additional parameter? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_multixact not getting truncated

2014-11-05 Thread Josh Berkus
On 11/05/2014 10:40 AM, Jim Nasby wrote: On 11/3/14, 7:40 PM, Josh Berkus wrote: On 11/03/2014 05:24 PM, Josh Berkus wrote: BTW, the reason I started poking into this was a report from a user that they have a pg_multixact directory which is 21GB in size, and is 2X the size of the database

Re: [HACKERS] Amazon Redshift

2014-11-05 Thread Josh Berkus
On 11/05/2014 02:36 PM, philip taylor wrote: Ok, this is a summary of what they have that we don't (of course, I could have missed something): I can't see any functions on that list I'd want. For example, DATEADD is there just to be compatible with MSSQL. It's useless to us. -- Josh

<    1   2   3   4   5   6   7   8   9   10   >