Re: [HACKERS] Proposal for changes to recovery.conf API

2016-09-06 Thread Simon Riggs
s we could keep the old parameters and make them PGC_POSTMASTER, but also provide a new parameter called recovery_target that simplifies the API and is PGC_SIGHUP. That way we resolve the annoyance of handling the current ones but keep compatibility for those who can't move on, just yet. -

Re: [HACKERS] Speedup twophase transactions

2016-09-06 Thread Simon Riggs
On 6 September 2016 at 02:41, Michael Paquier wrote: > After review the result is attached. Perhaps a committer could get a look at > it? Yes, will do, but it will be a few more days yet. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support,

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2016-09-05 Thread Simon Riggs
le idea of backwards scanning to confirm truncation seems wrong. What we want is an O(1) solution. Thinking. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hacker

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2016-09-05 Thread Simon Riggs
On 5 September 2016 at 15:50, Claudio Freire wrote: > On Sun, Sep 4, 2016 at 3:46 AM, Simon Riggs wrote: >> On 3 September 2016 at 04:25, Claudio Freire wrote: >>> The patch also makes vacuum free the dead_tuples before starting >>> truncation. It didn't seem

Re: [HACKERS] new autovacuum criterion for visible pages

2016-09-05 Thread Simon Riggs
that will tell us how many VM bits have been cleared, which will then allow autovac to do a simple SELECT to decide what needs vacuuming. Vik's proposal to keep track of the rows inserted seems like the best approach to this issue. -- Simon Riggshttp://www.2ndQuadrant.com

Re: [HACKERS] pg_sequence catalog

2016-09-05 Thread Simon Riggs
pplying Peter's idea/patch, though if that is not acceptable I don't think its worth spending further time on for any of us, so if thats the case lets Return with Feedback and move on. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Tr

Re: [HACKERS] INSERT .. SET syntax

2016-09-05 Thread Simon Riggs
On 3 July 2016 at 20:36, Marko Tiikkaja wrote: > Here's a patch for $SUBJECT. I'll probably work on the docs a bit more > before the next CF, but I thought I'd post it anyway. I think this should be Returned With Feedback. -- Simon Riggshttp://www.2ndQua

Re: [HACKERS] Assert(LWLockHeldByMeInMode(lock, LW_EXCLUSIVE))

2016-09-05 Thread Simon Riggs
k about something like the attached? See also an > example of use. I will add this to the next commitfest. Committed, thanks. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing l

Re: [HACKERS] Optimization for lazy_scan_heap

2016-09-05 Thread Simon Riggs
e have a freeze map now, surely tables will no longer be entirely frozen? What performance difference does this make, in a realistic use case? How would we test this to check it is exactly correct? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

2016-09-05 Thread Simon Riggs
bug. I've committed to HEAD only. We can discuss backpatching elsewhere also. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] LSN as a recovery target

2016-09-05 Thread Simon Riggs
On 5 September 2016 at 06:55, Michael Paquier wrote: > On Sun, Sep 4, 2016 at 11:30 PM, Simon Riggs wrote: >> I noticed we don't mention what LSN is anywhere, so I'd like to apply >> the following doc patch also. > > +1 for the idea. What do you think about adding

Re: [HACKERS] LSN as a recovery target

2016-09-04 Thread Simon Riggs
;> parameter project, please remind me to commit that as well. > > I'm aware of this, and will adjust accordingly in the GUC patch. Thanks > for the heads up. I noticed we don't mention what LSN is anywhere, so I'd like to apply the following doc patch also. -- Si

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2016-09-03 Thread Simon Riggs
che, especially if work > mem is configured very high to avoid multiple index scans. How long does that part ever take? Is there any substantial gain from this? Lets discuss that as a potential second patch. > Tested with pgbench scale 4000 after deleting the whole > pgbench_accounts table

Re: [HACKERS] LSN as a recovery target

2016-09-03 Thread Simon Riggs
On 4 September 2016 at 04:50, Michael Paquier wrote: > On Sun, Sep 4, 2016 at 8:05 AM, Michael Paquier > wrote: >> On Sun, Sep 4, 2016 at 1:57 AM, Simon Riggs wrote: >>> On 24 August 2016 at 05:50, Michael Paquier >>> wrote: >>> >>>>>&g

Re: [HACKERS] LSN as a recovery target

2016-09-03 Thread Simon Riggs
On 24 August 2016 at 05:50, Michael Paquier wrote: >>> Everything else looks in good order. Committed. Thanks. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailin

Re: [HACKERS] pg_hba_file_settings view patch

2016-09-03 Thread Simon Riggs
e longer than the concept of pg_hba.conf, which seems likely to become part of ALTER SYSTEM in future, so we wouldn't really want the word "file" in there. I've not seen anything yet to make me think a commit for this wouldn't happen once we've worked the detail. Thanks

Re: [HACKERS] autonomous transactions

2016-09-03 Thread Simon Riggs
lement ATs this way? If somebody believes there is a better way for ATs, that is just an optimization of the limits and can occur later, if the people who believe there is a better way can prove that is the case and come up with a patch. It's OK for features to go in now and have better infrastr

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2016-09-02 Thread Simon Riggs
On 2 September 2016 at 13:16, Craig Ringer wrote: > So I've moved it to xlog.c... I'm pretty sure it shouldn't live in xlog.c, but there may be some good reason I can't see yet. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 2

Re: [HACKERS] LOCK TABLE .. DEFERRABLE

2016-09-01 Thread Simon Riggs
On 5 April 2016 at 18:34, Rod Taylor wrote: > > > On Tue, Apr 5, 2016 at 1:10 PM, Simon Riggs wrote: >>> >>> If a lock is successfully obtained on one table, but not on all tables, >>> it releases that lock and will retry to get them as a group in the fut

Re: [HACKERS] Speedup twophase transactions

2016-09-01 Thread Simon Riggs
for the new CF, so review can start. Thanks -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://ww

Re: [HACKERS] WAL consistency check facility

2016-09-01 Thread Simon Riggs
ery safe feeling. I think the primary use for an rmgr filter might well be PostgreSQL developers. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.

Re: [HACKERS] Choosing parallel_degree

2016-09-01 Thread Simon Riggs
On 12 April 2016 at 14:11, Simon Riggs wrote: > On 12 April 2016 at 13:53, Robert Haas wrote: >> >> On Mon, Apr 11, 2016 at 5:45 PM, Simon Riggs >> wrote: >> > On 8 April 2016 at 17:49, Robert Haas wrote: >> >> With the patch, you can - if you wish -

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

2016-09-01 Thread Simon Riggs
it in the next CF. (Of course, I think they should just be >> committed, but I would, wouldn't I?) >> >> > > I think the doc one should definitely go in and possibly be back-patched all > the way to 9.4. I didn't look at the other one. I agree the doc patch should g

Re: [HACKERS] WAL consistency check facility

2016-09-01 Thread Simon Riggs
On 1 September 2016 at 11:16, Robert Haas wrote: > On Wed, Aug 31, 2016 at 7:02 PM, Simon Riggs wrote: >> I'd prefer a solution that was not dependent upon RmgrID at all. >> >> If there are various special cases that we need to cater for, ISTM >> they wo

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-09-01 Thread Simon Riggs
ntangle that right now, I'm posting what I have as-is, and I'll post > an updated version tomorrow. Thanks. archive_cleanup_command no longer needs to be in shmem. Checkpointer will have its own copy of the value. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-09-01 Thread Simon Riggs
On 1 September 2016 at 06:34, Michael Paquier wrote: > On Thu, Sep 1, 2016 at 1:15 AM, Simon Riggs wrote: >> This is a summary of proposed changes to the recovery.conf API for >> v10. These are based in part on earlier discussions, and represent a >> minimal modifica

Re: [HACKERS] pg_basebackup wish list

2016-09-01 Thread Simon Riggs
patch to CF1. > > +1 for the idea. Looking at the patch it is taking a sane approach. Apart from this one liner change we look good to go. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-

Re: [HACKERS] autonomous transactions

2016-08-31 Thread Simon Riggs
for only some functions. What happens if your function calls some other function with side-effects? How would you roll that back? How would you mark functions for the general case? Functions with side effects can't be tested with simple unit tests; that has nothing to do with autonomous tra

[HACKERS] Proposal for changes to recovery.conf API

2016-08-31 Thread Simon Riggs
quot;promote" * Remove user configurable "trigger_file" mechanism - use "promote.trigger" for all cases * Remove Fallback promote mechanism, so all promotion is now "fast" in xlog.c * Rename CheckForStandbyTrigger() to CheckForPromoteTrigger() for clarity (Patch: rec

Re: [HACKERS] WAL consistency check facility

2016-08-31 Thread Simon Riggs
t dependent upon RmgrID at all. If there are various special cases that we need to cater for, ISTM they would be flaws in the existing WAL implementation rather than anything we would want to perpetuate. I hope we'll spend time fixing them rather than add loads of weird code to work around the impe

Re: [HACKERS] [COMMITTERS] pgsql: Fix pg_receivexlog --synchronous

2016-08-31 Thread Simon Riggs
On 29 August 2016 at 12:34, Tom Lane wrote: > Simon Riggs writes: >> Fix pg_receivexlog --synchronous > > The buildfarm says you broke the 9.5 branch. > > In general, pushing inessential patches just a few hours before a wrap > deadline is a dangerous business. Pushing

Re: [HACKERS] pg_receivexlog does not report flush position with --synchronous

2016-08-29 Thread Simon Riggs
On 23 August 2016 at 14:57, Michael Paquier wrote: > On Tue, Aug 23, 2016 at 9:48 PM, Gabriele Bartolini > wrote: >> Hi Simon and Michael, >> >> 2016-08-23 10:39 GMT+02:00 Simon Riggs : >>> >>> Agreed, but I'd move all the comments above

Re: [HACKERS] WAL consistency check facility

2016-08-27 Thread Simon Riggs
On 27 August 2016 at 07:36, Amit Kapila wrote: > On Fri, Aug 26, 2016 at 9:26 PM, Simon Riggs wrote: >> >> I think you should add this as part of the default testing for both >> check and installcheck. I can't imagine why we'd have it and not use >>

Re: [HACKERS] Renaming of pg_xlog and pg_clog

2016-08-26 Thread Simon Riggs
ce. That helps the few people who made such mistakes, but doesn't cause massive change as a result. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@po

Re: [HACKERS] Renaming some binaries

2016-08-26 Thread Simon Riggs
ill be issuing more than one DDL command at a time, so they'll be writing a script anyway. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] WAL consistency check facility

2016-08-26 Thread Simon Riggs
anything. > 5. Generalize the page type identification technique. Why not do this first? There are some coding guideline stuff to check as well. Thanks -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Renaming of pg_xlog and pg_clog

2016-08-26 Thread Simon Riggs
it is self evident, cos it certainly isn't. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http:

Re: [HACKERS] increasing the default WAL segment size

2016-08-25 Thread Simon Riggs
e the pain of change, should we also consider making WAL files variable length? What do we gain by having the files all the same size? ISTM better to have WAL files that vary in length up to 1GB in size. (This is all about XLOG_SEG_SIZE; I presume XLOG_BLCKSZ can stay as it is, right?) -- S

Re: [HACKERS] WAL consistency check facility

2016-08-24 Thread Simon Riggs
On 22 August 2016 at 16:56, Simon Riggs wrote: > On 22 August 2016 at 13:44, Kuntal Ghosh wrote: > >> Please let me know your thoughts on this. > > Do the regression tests pass with this option enabled? Hi, I'd like to be a reviewer on this. Please can you add this on

Re: [HACKERS] LSN as a recovery target

2016-08-23 Thread Simon Riggs
s generated prior to starting to search. Everything else looks in good order. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to y

Re: [HACKERS] pg_receivexlog does not report flush position with --synchronous

2016-08-23 Thread Simon Riggs
On 23 August 2016 at 08:56, Michael Paquier wrote: > On Tue, Aug 23, 2016 at 7:34 AM, Gabriele Bartolini > wrote: I'd suggest rewording it a bit instead, please see attached. > And of course this needs a backpatch. Agreed, but I'd move all the comments above the bloc

Re: [HACKERS] WAL consistency check facility

2016-08-22 Thread Simon Riggs
On 22 August 2016 at 13:44, Kuntal Ghosh wrote: > Please let me know your thoughts on this. Do the regression tests pass with this option enabled? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- S

Re: [HACKERS] Pluggable storage

2016-08-18 Thread Simon Riggs
I'm quite concerned about that as well. This objection would apply to all other proposals as well, FDW etc.. Do you see some way to add flexibility yet without adding a branch point in the code? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Re

Re: [HACKERS] Why we lost Uber as a user

2016-08-17 Thread Simon Riggs
lutions, and also used by other databases, such as Oracle. Corruption on the master would often cause errors that would prevent writes and therefore those changes wouldn't even be made, let alone be replicated. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Developme

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-10 Thread Simon Riggs
ike a good idea. It would allow us to produce a bitmap scan in blocksorted order. > So I'll go ahead and write a patch that implements the core > idea and some basic optimizations. +1 > We can then try different approaches such > as tracking changed columns, tracking increment/de

Re: [HACKERS] Small issues in syncrep.c

2016-08-10 Thread Simon Riggs
pply immediately. 14e8803f1 was only a partial patch for the syncrep code, so I don't see any reason to keep the code as it currently is in 9.5/9.6. Any objections to backpatching this to 9.5 and 9.6? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 2

Re: [HACKERS] Logical Replication WIP

2016-08-05 Thread Simon Riggs
lan to spend a fair amount of time helping to review > etc. this. Have a good one. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] PostgreSQL 10 Roadmaps

2016-08-05 Thread Simon Riggs
On 5 August 2016 at 04:40, Etsuro Fujita wrote: > On 2016/08/02 23:54, Simon Riggs wrote: >> >> https://wiki.postgresql.org/wiki/PostgreSQL10_Roadmap > > > Thank you for creating the wiki page! > > I added a link to the NTT roadmap page on the links page. We hope th

Re: [HACKERS] Lossy Index Tuple Enhancement (LITE)

2016-08-04 Thread Simon Riggs
hat... "The same situation can occur if we have many INSERTs with same values on the same block. This could lead to a reduction in size of the btree for indexes with many duplicate values." -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Suppor

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Simon Riggs
On 4 August 2016 at 18:17, Andres Freund wrote: > On 2016-08-04 18:11:17 +0100, Simon Riggs wrote: >> On 4 August 2016 at 17:31, Andres Freund wrote: >> > Hi, >> > >> > On 2016-08-04 16:29:09 +0530, Pavan Deolasee wrote: >> >> Indexes whose values

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Simon Riggs
RECHECK_REQUIRED earlier than that. Hmm. Mostly there will be one, so this is just for the first update after any VACUUM. Adding a new linepointer just to hold this seems kludgy and could mean we run out of linepointers. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Deve

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Simon Riggs
gure that out? Hmm, sorry, I did think of that point and I thought I had added it to the doc. So, yes, I agree - re-locating the root is the biggest downside to this idea. Perhaps Pavan has other thoughts? I think its doable, but it will be fiddly. -- Simon Riggshttp://www.2ndQua

Re: [HACKERS] Lossy Index Tuple Enhancement (LITE)

2016-08-04 Thread Simon Riggs
non-HOT updates. So, we have a choice of which disadvantage to accept. > Also, why not use this bitmap for all indexes, not just update chains? I don't understand where you get this update chains thing from. The bitmap can apply to multiple tuples on one page, which i

Re: [HACKERS] Lossy Index Tuple Enhancement (LITE)

2016-08-04 Thread Simon Riggs
On 3 August 2016 at 20:37, Claudio Freire wrote: > On Wed, Aug 3, 2016 at 4:20 AM, Simon Riggs wrote: >> == IndexScan == >> >> Note that the executor code for IndexScan appears identical between >> the two optimizations. The difference between duplicate and range LITE

Re: [HACKERS] Design for In-Core Logical Replication

2016-08-04 Thread Simon Riggs
et this right, since this is much more about UI than system architecture. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to yo

Re: [HACKERS] Lossy Index Tuple Enhancement (LITE)

2016-08-03 Thread Simon Riggs
g index entry for the same values and heap page? That seems > quite complicated. The insertion algorithm is described. Doesn't seem complicated to me. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent

Re: [HACKERS] Why we lost Uber as a user

2016-08-03 Thread Simon Riggs
possibly get a similar index, but this might allow such an index to > still be usable for index-only scans, which a BRIN index will never be > able to provide. This idea is vaguely similar to the concepts I've been working on, based on earlier work in 2007. I'm starting

[HACKERS] Lossy Index Tuple Enhancement (LITE)

2016-08-03 Thread Simon Riggs
des, and in btinsert(). There are no changes to WAL records or replay. The details are all in the way we insert and interpret index tuples. == Other designs == This is related in a few ways to Grouped Item Tuple indexes, a patch Heikki worked on in 2007, but is not that close. -- Simon Riggs

Re: [HACKERS] Why we lost Uber as a user

2016-08-02 Thread Simon Riggs
!... but Pavan has managed to shoot it down with some accurate observations about it leading to an annoying accumulation of root pointers and complex logic to remove them. So I'm not pursuing it further at this stage. I'm writing up my conclusions around what we should do now, so shou

[HACKERS] PostgreSQL 10 Roadmaps

2016-08-02 Thread Simon Riggs
https://wiki.postgresql.org/wiki/PostgreSQL10_Roadmap -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscrip

Re: [HACKERS] PostgreSQL 10 kick-off

2016-08-01 Thread Simon Riggs
On 1 August 2016 at 17:04, Tom Lane wrote: > Simon Riggs writes: >> Can we confirm/refute these details now to make sure we are all in tune? > > See the other thread I started about that; please reserve this thread > for discussions of general actions around starting the new d

Re: [HACKERS] PostgreSQL 10 kick-off

2016-08-01 Thread Simon Riggs
01 effectively allowing many future versions. Can we confirm/refute these details now to make sure we are all in tune? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing

Re: [HACKERS] Design for In-Core Logical Replication

2016-07-22 Thread Simon Riggs
#x27;t want handle events on third table which was added to publication. Not sure what you mean for that part. Could you explain a little more? (I'm on leave for some days, so I don't respond quickly I will respond eventually) -- Simon Riggshttp://www.2ndQuadran

Re: [HACKERS] Design for In-Core Logical Replication

2016-07-20 Thread Simon Riggs
ormal even if only 2 nodes. > Old slow moving data almost always got different treatment than fast-moving > data; even if only defining which set needs to hit the other node first and > which set can trickle through later. > Agreed -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Design for In-Core Logical Replication

2016-07-20 Thread Simon Riggs
has one or more Databases, each of which can publish its data in zero, one or more PUBLICATIONs. A Subscribing node can receive data in the form of zero, one or more SUBSCRIBERs, where each SUBSCRIBER may bring together data from one or more PUBLICATIONs. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

[HACKERS] Design for In-Core Logical Replication

2016-07-20 Thread Simon Riggs
start the replication process which synchronizes the initial table contents of users and departments tables and then starts replicating incremental changes to those tables. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL D

Re: [HACKERS] One process per session lack of sharing

2016-07-17 Thread Simon Riggs
I think its clear that threading is out, but it is far from being the only solution to reducing the memory overhead of sharing. Analysing the overhead and suggesting practical ways forward may help. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/>

Re: [HACKERS] pg_xlogdump follow into the future

2016-07-14 Thread Simon Riggs
gue it's a bugfix > though, since the usecase simply did not work... > Good thinking. Patch looks good, but not tested. Needs comment on the second chunk. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Simplifying the interface of UpdateMinRecoveryPoint

2016-07-13 Thread Simon Riggs
On 12 July 2016 at 23:49, Michael Paquier wrote: > Hence why not simplifying its interface and remove the force flag? Is this change needed as part of a feature? If not, I see no reason for change. If we all work towards meaningful features the code can be cleaned up as we go. -- Si

Re: [HACKERS] A Modest Upgrade Proposal

2016-07-08 Thread Simon Riggs
s), I definitely think we should dump replication sets and > their membership info for example. Agreed -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] A Modest Upgrade Proposal

2016-07-08 Thread Simon Riggs
high level a thought and is leading to argument. The most likely answer is "some", but still not sure. I am looking at this in more detail and will return in a few days with a much more specific design that we can use to answer the question in detail. > Direct SQL > on a catalog should

Re: [HACKERS] A Modest Upgrade Proposal

2016-07-07 Thread Simon Riggs
should work and what options to support and a sidetrack into syntax isn't warranted at this early stage. Please lets discuss those important things first, then return to whether DDL makes sense or not; it may do, or may not, or more likely which parts of it need DDL and which do not. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] A Modest Upgrade Proposal

2016-07-07 Thread Simon Riggs
On 8 July 2016 at 02:41, Robert Haas wrote: > On Thu, Jul 7, 2016 at 7:15 PM, Simon Riggs wrote: > > Yes, I ran the unconference session. It was a shame you weren't able to > stay > > for the whole discussion. > > I thought I sat through, at least, most of it, but

Re: [HACKERS] A Modest Upgrade Proposal

2016-07-07 Thread Simon Riggs
On 8 July 2016 at 01:47, Joshua D. Drake wrote: > * Long running transaction > And of course you can't run any transactions at all during pg_upgrade, not just long running ones. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.co

Re: [HACKERS] pg_xlogfile_name_offset() et al and recovery

2016-07-07 Thread Simon Riggs
On 8 July 2016 at 00:43, Michael Paquier wrote: > On Thu, Jul 7, 2016 at 6:26 PM, Simon Riggs wrote: > > The behaviour of that function is defined in backbranches, so I suggest > we > > should not alter that now. > > Hm.. > > > What we can do is have another fu

Re: [HACKERS] A Modest Upgrade Proposal

2016-07-07 Thread Simon Riggs
nyway, I'm cool if you don't want to use it, for while or never. Options are good. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] A Modest Upgrade Proposal

2016-07-07 Thread Simon Riggs
gt; into core today. Which we by no means do. >> > > I would much rather see more brain power put into pg_upgrade or in place > upgrades than logical replication (as a upgrade solution). Why is that? -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] A Modest Upgrade Proposal

2016-07-07 Thread Simon Riggs
rate out that as a use-case so we can be sure we get that in 10.0, even if nothing else lands. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] A Modest Upgrade Proposal

2016-07-07 Thread Simon Riggs
backup, advisory locks and some features that use DDL that don't really need to such as LISTEN/NOTIFY, full text search etc.. Also note that both Oracle and SQLServer have moved away from DDL in favour of function APIs, most NoSQL databases and almost all languages prefer functional interfac

Re: [HACKERS] MVCC overheads

2016-07-07 Thread Simon Riggs
ed row versions into a separate data structure, so would be much more amenable to offload than PostgreSQL, in its current form. Maybe look at SLRUs (clog etc) as a place to offload something? -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] MVCC overheads

2016-07-07 Thread Simon Riggs
sistent and incorrect results in applications, or other architectural restrictions imposed to make that possible. It's easy to make assumptions in the lab that don't work in the real world. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/&g

Re: [HACKERS] pg_xlogfile_name_offset() et al and recovery

2016-07-07 Thread Simon Riggs
_name_offset(l.lsn, l.tli) from pg_stop_backup(false) l; So I suggest we add another column to the output of pg_stop_backup() to return the tli value directly. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Rethinking representation of partial-aggregate steps

2016-06-23 Thread Simon Riggs
considerations. > Agreed I can imagine plans where a useful aggregation occurs before every join, so N > 2 is easily possible. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] New design for FK-based join selectivity estimation

2016-06-13 Thread Simon Riggs
On 13 June 2016 at 19:16, Tom Lane wrote: > Simon Riggs writes: > > So a simple change is to make RelationGetFKeyList() only retrieve FKs > with > > nKeys>1. Rename to RelationGetMultiColumnFKeyList(). That greatly reduces > > the scope for increased plann

Re: [HACKERS] New design for FK-based join selectivity estimation

2016-06-13 Thread Simon Riggs
ses them unless nKeys>1. That was masked somewhat by my two commits, treating the info as generic and then using only a very specific subset of it. So a simple change is to make RelationGetFKeyList() only retrieve FKs with nKeys>1. Rename to RelationGetMultiColumnFKeyList(). That greatly re

Re: [HACKERS] Does people favor to have matrix data type?

2016-05-25 Thread Simon Riggs
hether to include it or not. Multi-dimensionality of arrays isn't always useful, so this could be good. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] memory layouts for binary search in nbtree

2016-05-18 Thread Simon Riggs
rder more optimized for efficient binary search. > > See e.g. http://cglab.ca/~morin/misc/arraylayout/ for benchmarks > showing benefits. > Some stuff from >10 years ago about cache conscious btree layout as well. That led to adoption of 64kB pages on some benchmarks. I think its

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

2016-05-10 Thread Simon Riggs
On 10 May 2016 at 17:20, Andres Freund wrote: > On 2016-05-10 12:28:57 +0200, Simon Riggs wrote: > > On 10 May 2016 at 09:05, Andres Freund wrote: > > > > > > > Is anybody ready with a good defense for SatisfiesToast not doing any > > > actual liveliness c

Re: [HACKERS] HeapTupleSatisfiesToast() busted? (was atomic pin/unpin causing errors)

2016-05-10 Thread Simon Riggs
ust used it to check the patch which changed btree vacuum replay. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services toast_recheck.v1.patch Description: Binary data -- Sent via pg

Re: [HACKERS] asynchronous and vectorized execution

2016-05-09 Thread Simon Riggs
cache lines. > Team is about 2 years into research and coding prototype on those topics at this point, with agreed time for further work over next 2 years. I'll let my colleagues chime in with details since I'm not involved at that level any more. -- Simon Riggshttp

Re: [HACKERS] pg9.6 segfault using simple query (related to use fk for join estimates)

2016-05-08 Thread Simon Riggs
s point. I disagree. The purpose of the patch is to improve planning in simple and important common cases. It does that. The fact that it does not cover all cases is understood and we will make more progress in future releases. We don't handle the inheritance case well in ma

Re: [HACKERS] minor message improvement

2016-05-08 Thread Simon Riggs
On 8 May 2016 at 12:48, Simon Riggs wrote: > On 8 May 2016 at 03:53, Euler Taveira wrote: > > Attached is a patch that turn it into one. >> > > Applied, by Tom. Thanks > Sorry about that; Tom's message only just arrived with me for some reason. -- Simon Ri

Re: [HACKERS] minor message improvement

2016-05-08 Thread Simon Riggs
On 8 May 2016 at 03:53, Euler Taveira wrote: Attached is a patch that turn it into one. > Applied, by Tom. Thanks -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-07 Thread Simon Riggs
On 7 May 2016 at 16:49, Tom Lane wrote: > Simon Riggs writes: > > On 3 May 2016 at 18:07, Tom Lane wrote: > >> Or at least, it did until Simon decided that ALTER TABLE RESET > >> doesn't require AccessExclusiveLock. > > > On reflection, this st

Re: [HACKERS] pg_dump broken for non-super user

2016-05-07 Thread Simon Riggs
On 7 May 2016 at 16:21, Stephen Frost wrote: > * Simon Riggs (si...@2ndquadrant.com) wrote: > > On 7 May 2016 at 16:14, Stephen Frost wrote: > > > > If we don't lock it then we will have a inconsistent dump that will > fail > > > > later, if dumped whi

Re: [HACKERS] pg_dump broken for non-super user

2016-05-07 Thread Simon Riggs
ted out. The catalog tables > are read using a repeatable read transaction, which will be consistent. The scan is consistent, yes, but the results would not be. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] pg_dump broken for non-super user

2016-05-07 Thread Simon Riggs
d while an object is being dropped. Do we want an inconsistent dump? For what reason are we changing existing behaviour? There is no bug here, as Stephen explained. So this is a behaviour change after freeze with uncertain purpose. -- Simon Riggshttp://www.2ndQuadrant.com/ &

Re: [HACKERS] pg9.6 segfault using simple query (related to use fk for join estimates)

2016-05-07 Thread Simon Riggs
a massively negative performance because that is a small subset of cases; I did test that to check for regressions, but its possible I missed something that does cause additional run-time in real world cases. Clearly, an optimizer test suite would be helpful in analyzing the effect of new patches.

Re: [HACKERS] pg9.6 segfault using simple query (related to use fk for join estimates)

2016-05-07 Thread Simon Riggs
Yes, the GUC was for testing, as mentioned on the commit message. > Can't check right now with Simon, though, as he's is out of office this > week. > Am back and reading. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-07 Thread Simon Riggs
your latest post. If pg_upgrade needs this, we should implement a specific function that does what pg_upgrade needs. That way we can isolate the requirement for an AccessExclusiveLock to the place that needs it: pg_upgrade. That will also make it less fragile in the future. I don't think that n

<    1   2   3   4   5   6   7   8   9   10   >