Re: [HACKERS] drop/truncate table sucks for large values of shared buffers

2015-07-02 Thread Simon Riggs
t high handedly to reject a patch without reason. I concur that we should explore using a radix tree or something else that > would naturally allow you to find all buffers for relation/database X > quickly. I doubt that it can be managed efficiently while retaining optimal memory mana

Re: [HACKERS] Memory Accounting v11

2015-07-02 Thread Simon Riggs
ot manage our resources, how are we certain this does not induce a penalty? Not tracking memory could be worse than tracking it. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] raw output from copy

2015-07-02 Thread Simon Riggs
; old Andrew Dunstan's post either. > Feature sounds OK, so lets do it. Pavel S, please submit a polished patch. Coding guidelines, tests, docs etc. Set back to Waiting On Author. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL D

Re: [HACKERS] Let PostgreSQL's On Schedule checkpoint write buffer smooth spread cycle by tuning IsCheckpointOnSchedule?

2015-07-02 Thread Simon Riggs
e before committing anything like this. > We could do better, but that is not a reason not to commit this, as is. Commit, please. This has been in place for a while and still remains: TODO: reduce impact of full page writes -- Simon Riggshttp://www.2ndQuadrant.com/ <http:/

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-07-01 Thread Simon Riggs
sign completely misses the time taken to scan indexes, which is significant. There might be a justification to put this out of core, but measuring progress of VACUUM wouldn't be it, IMHO. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Freeze avoidance of very large table.

2015-07-01 Thread Simon Riggs
They are essential for reviewers, not a cosmetic thing to be added later. To gain wide agreement we need wide understanding. (I recommend a development approach where you write the comments first, then add code later.) -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2nd

Re: [HACKERS] drop/truncate table sucks for large values of shared buffers

2015-07-01 Thread Simon Riggs
t; cycle during buffer scan. > Yes So you are keeping more data around for longer, right? I think we would need some way to trigger a scan when the amount of deferred dropped data files hits a certain size. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Reducing ClogControlLock contention

2015-07-01 Thread Simon Riggs
On 1 July 2015 at 11:14, Andres Freund wrote: > On 2015-07-01 09:08:11 +0100, Simon Riggs wrote: > > On 1 July 2015 at 09:00, Amit Kapila wrote: > > > > > On Tue, Jun 30, 2015 at 12:32 PM, Simon Riggs > > > a. the semantics of new LWLock (CommitLock) intr

Re: [HACKERS] Reducing ClogControlLock contention

2015-07-01 Thread Simon Riggs
On 1 July 2015 at 11:11, Amit Kapila wrote: > On Wed, Jul 1, 2015 at 1:38 PM, Simon Riggs wrote: > > > > On 1 July 2015 at 09:00, Amit Kapila wrote: > >> > >> > >> I think it will be better to partition it or use it in some other way > to avoid >

Re: [HACKERS] WAL-related tools and .paritial WAL file

2015-07-01 Thread Simon Riggs
example, > > "please get rid of .paritial suffix from the WAL file name if > > you want to dump it by pg_xlogdump". > > For pg_xlogdump, I am on board for only documenting the limitation > (renaming the file by yourself) as it is after all only a debug tool. > T

Re: [HACKERS] Reducing ClogControlLock contention

2015-07-01 Thread Simon Riggs
On 1 July 2015 at 09:00, Amit Kapila wrote: > On Tue, Jun 30, 2015 at 12:32 PM, Simon Riggs > wrote: > > > > ClogControlLock contention is high at commit time. This appears to be > due to the fact that ClogControlLock is acquired in Exclusive mode prior to > marking

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-06-30 Thread Simon Riggs
then we can begin adding other operations as we work out how (for that operation). -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Reducing ClogControlLock contention

2015-06-30 Thread Simon Riggs
On 30 June 2015 at 08:22, Simon Riggs wrote: > This contention is masked by contention elsewhere, e.g. ProcArrayLock, so > the need for testing here should come once other patches ahead of this are > in. > Let me explain more clearly. Andres' patch to cache snapshots and red

Re: [HACKERS] Reducing ClogControlLock contention

2015-06-30 Thread Simon Riggs
On 30 June 2015 at 08:13, Michael Paquier wrote: > > > On Tue, Jun 30, 2015 at 4:02 PM, Simon Riggs > wrote: > >> ClogControlLock contention is high at commit time. This appears to be due >> to the fact that ClogControlLock is acquired in Exclusive mode prior to &g

[HACKERS] Reducing ClogControlLock contention

2015-06-30 Thread Simon Riggs
in clog, which would require a barrier. Two concurrent writers might access the same word concurrently, so we protect against that with a new CommitLock. We could partition that by pageno also, if needed. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.

Re: [HACKERS] drop/truncate table sucks for large values of shared buffers

2015-06-29 Thread Simon Riggs
On 30 June 2015 at 07:34, Amit Kapila wrote: > On Tue, Jun 30, 2015 at 11:00 AM, Simon Riggs > wrote: > > > > On 30 June 2015 at 05:02, Amit Kapila wrote: > >> > >> On Mon, Jun 29, 2015 at 7:18 PM, Simon Riggs > wrote: > >> > > >> &g

Re: [HACKERS] Reduce ProcArrayLock contention

2015-06-29 Thread Simon Riggs
On 30 June 2015 at 07:30, Amit Kapila wrote: > Sure and I think we might want to try something similar even > for XLogFlush where we use LWLockAcquireOrWait for > WALWriteLock, not sure how it will workout in that case as > I/O is involved, but I think it is worth trying. > +1

Re: [HACKERS] Reduce ProcArrayLock contention

2015-06-29 Thread Simon Riggs
sitting on a patch for a month or so and will post now I'm done travelling. These ideas have been around some time and are even listed on the PostgreSQL TODO: http://archives.postgresql.org/pgsql-hackers/2007-09/msg00206.php -- Simon Riggshttp://www.2ndQuadrant.com/ <http:/

Re: [HACKERS] Reduce ProcArrayLock contention

2015-06-29 Thread Simon Riggs
On 30 June 2015 at 03:43, Robert Haas wrote: > On Mon, Jun 29, 2015 at 1:22 PM, Simon Riggs > wrote: > > Yes, I know. And we all had a long conversation about how to do it > without > > waking up the other procs. > > > > Forming a list, like we use for sync rep

Re: [HACKERS] drop/truncate table sucks for large values of shared buffers

2015-06-29 Thread Simon Riggs
On 30 June 2015 at 05:02, Amit Kapila wrote: > On Mon, Jun 29, 2015 at 7:18 PM, Simon Riggs > wrote: > > > > On 28 June 2015 at 17:17, Tom Lane wrote: > >> > >> I'm not sure what you consider "dire", but missing a dirty buffer > >>

Re: [HACKERS] Reduce ProcArrayLock contention

2015-06-29 Thread Simon Riggs
On 29 June 2015 at 18:11, Andres Freund wrote: > On June 29, 2015 7:02:10 PM GMT+02:00, Simon Riggs > wrote: > >On 29 June 2015 at 16:27, Amit Kapila wrote: > > > > > >> Thanks to Robert Haas for having discussion (offlist) about the idea > >> and

Re: [HACKERS] Reduce ProcArrayLock contention

2015-06-29 Thread Simon Riggs
ly the way I said to implement it publicly at PgCon. Was nobody recording the discussion at the unconference?? -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] drop/truncate table sucks for large values of shared buffers

2015-06-29 Thread Simon Riggs
On 28 June 2015 at 17:17, Tom Lane wrote: > Simon Riggs writes: > > On 27 June 2015 at 15:10, Tom Lane wrote: > >> I don't like this too much because it will fail badly if the caller > >> is wrong about the maximum possible page number for the table, which &g

Re: [HACKERS] drop/truncate table sucks for large values of shared buffers

2015-06-28 Thread Simon Riggs
re nowhere near as dire, since by definition the data is being destroyed by the user. So ISTM that we should be able to use this technique. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] UPSERT on partition

2015-06-26 Thread Simon Riggs
should just treat it as a limitation of UPSERT and add that document? > +1 There are many problems that cannot be resolved for 9.5. UPSERT works fine with tables with BRIN indexes. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Deve

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-06-25 Thread Simon Riggs
ple separators to define group types then: > - "()" where the order of acknowledgement does not matter > - "[]" where it does not. > You would find the old grammar with: > 1[AAA,BBB,CCC] > Let's start with a complex, fully described use case then work ou

Re: [HACKERS] Trustly PostgreSQL Data Corruption Bug Bounty Program

2015-06-24 Thread Simon Riggs
ve any other feedback in general. > > Members of the PostgreSQL Security Team need to be disqualified from > eligibility, of course. > ..err... this is for data corruption bugs, not security. Editing data files by privileged users doesn't count, since it needs to be reproducibl

Re: [HACKERS] Multixid hindsight design

2015-06-24 Thread Simon Riggs
On 24 June 2015 at 16:30, Simon Riggs wrote: > Though TED sounds nice, the way to avoid another round of on-disk bugs is > by using a new kind of testing, not simply by moving the bits around. > > It might be argued that we are increasing the diagnostic/forensic > capabilities

Re: [HACKERS] Multixid hindsight design

2015-06-24 Thread Simon Riggs
d thing I see from TED is it allows us to test the on-disk outcome of concurrent activity. Currently we have isolationtester, but that is not married in any way to the on-disk state allowing us the situation where isolationtester can pass yet we have corrupted on-disk state. We should specify the on-d

Re: [HACKERS] Extension support for postgres_fdw

2015-06-20 Thread Simon Riggs
ke that, but currently we handle things in terms of Schemas. It would be strange to have differing ways of specifying groups of objects. Maybe that's not a problem, but we'd probably need to analyse that to make sure it didn't make things more complex. -- Simon Riggs

Re: [HACKERS] Is it possible to have a "fast-write" Index?

2015-06-19 Thread Simon Riggs
So the only saving for the foreground process is the random I/O from inserting into the indexes, which means the value of the technique is in the case where you have many very large secondary indexes - which is now covered by BRIN. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Entities created in one query not available in another in extended protocol

2015-06-12 Thread Simon Riggs
On 12 June 2015 at 20:06, Tom Lane wrote: > Simon Riggs writes: > > On 11 June 2015 at 22:12, Shay Rojansky wrote: > >> Just in case it's interesting to you... The reason we implemented things > >> this way is in order to avoid a deadlock situation - if we sen

Re: [HACKERS] Entities created in one query not available in another in extended protocol

2015-06-12 Thread Simon Riggs
problem differently in our driver implementation. > > Shay > > On Thu, Jun 11, 2015 at 6:17 PM, Simon Riggs > wrote: > >> On 11 June 2015 at 16:56, Shay Rojansky wrote: >> >> Npgsql (currently) sends Parse for the second command before sending >>> E

Re: [HACKERS] Is it possible to have a "fast-write" Index?

2015-06-11 Thread Simon Riggs
dy asked for "delayed write" indexes, but the idea gets > discarded because the index could get out of sync, so it can omit results > and this is unacceptable. But i think maybe that could be fixed in several > ways and we can have a fast and reliable index (but maybe not so fast on

Re: [HACKERS] The purpose of the core team

2015-06-11 Thread Simon Riggs
The balance of > coordinating release activities is mechanical, and -packagers seems > adequate > for it. Packagers should be about "HOW do we make the next release", which is separate from the above. Ultimately, "How" effects "When", but "When is it needed?" is an earlier thought. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Entities created in one query not available in another in extended protocol

2015-06-11 Thread Simon Riggs
On 11 June 2015 at 16:56, Shay Rojansky wrote: Npgsql (currently) sends Parse for the second command before sending > Execute for the first one. > Look no further than that. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Devel

Re: [HACKERS] Entities created in one query not available in another in extended protocol

2015-06-11 Thread Simon Riggs
ely, but thank you for the report. Can we see a test case? Most commonly in such cases the first request failed and error messages weren't checked before running second message. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Reducing tuple overhead

2015-06-07 Thread Simon Riggs
e if you get rid of the xmax then you still have to freeze the tuples later. I would have thought a better optimization would be to use the xmax for the xid epoch by default, so that such rows would never need freezing. Then at least we are using the xmax for something useful in a larger insert-mostly database rather than just leaving it at zero. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-06 Thread Simon Riggs
On 5 June 2015 at 17:20, Alvaro Herrera wrote: > Simon Riggs wrote: > > On 5 June 2015 at 15:00, Robert Haas wrote: > > > > Stamping it a beta implies that we think it's something fairly > > > stable that we'd be pretty happy to release if things go we

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-05 Thread Simon Riggs
was. Reliable delivery of new versions of software is important too. New versions often contain new features that fix real world problems, just as much as bug fixes do, hence why I don't wish to divert from the normal process and schedule. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-05 Thread Simon Riggs
the release of Beta then I disagree. The presence of bugs clearly has nothing to do with the discovery of new ones and we should be looking to discover as many as possible as quickly as possible. I can understand the argument to avoid releasing Beta because of Dev Meeting, so we should aim for June 25t

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-05 Thread Simon Riggs
On 5 June 2015 at 15:00, Robert Haas wrote: > On Fri, Jun 5, 2015 at 2:50 AM, Simon Riggs wrote: > > Agreed. Cleanup can occur while we release code for public testing. > > The code is available for public testing right now. People test when they get the signal from us, not bef

Re: [HACKERS] RFC: Remove contrib entirely

2015-06-05 Thread Simon Riggs
those out. +1 for Extensions directory for 9.6 This doesn't seem worth delaying the release for. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Multixid hindsight design

2015-06-05 Thread Simon Riggs
On 5 June 2015 at 11:02, Andres Freund wrote: > On 2015-06-05 10:45:09 +0100, Simon Riggs wrote: > > On 1 June 2015 at 20:53, Thomas Munro > wrote: > > > > > On Tue, May 12, 2015 at 9:20 AM, Heikki Linnakangas > > > wrote: > > > > The beauty of

Re: [HACKERS] Multixid hindsight design

2015-06-05 Thread Simon Riggs
e BUT since we reduce the size of each Members array by one, there is a balanced saving there, so this change is disk-space-neutral. That way if we need to make Offsets SLRU persistent it won't bloat. We then leave the Members SLRU as non-persistent, just as it was <9.3 -- Simon Riggs

Re: [HACKERS] Multixid hindsight design

2015-06-05 Thread Simon Riggs
hat survives server restart, how did it work before 9.3? -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Restore-reliability mode

2015-06-05 Thread Simon Riggs
our normal things, not do a "total stop" for an indefinite period. If someone has specific things that in their opinion need to be addressed, list them and we can talk about doing them, together. I thought that was what the Open Items list was for. Let's use it. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] [CORE] Restore-reliability mode

2015-06-04 Thread Simon Riggs
eing discussed here can/will slow down the BDR project since it is already a different thread of development. More so, 2ndQuadrant has zero income tied to the release of 9.5 or the commit of any feature, so as far as that company is concerned, the release could wait for 10 years. -- Simon Riggs

Re: [CORE] [HACKERS] postpone next week's release

2015-06-04 Thread Simon Riggs
ease date, so if the quality proves to be as poor as some people think then the list of bugs will show that and we release later. AFAIK beta period is exactly the time when we are allowed to pull features from the release. I welcome the idea that we test it, if its stable and it works we release it.

Re: [HACKERS] Need Force flag for pg_drop_replication_slot()

2015-05-29 Thread Simon Riggs
On 29 May 2015 at 18:15, Josh Berkus wrote: > While I'm just doing this during testing That part is good. I'm sure you will find something in need of improvement. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Develo

Re: [HACKERS] Need Force flag for pg_drop_replication_slot()

2015-05-29 Thread Simon Riggs
gt; replication connection and delete the slot as a single operation. > If you really want it you can write a function to do that for private use. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] pg_upgrade resets timeline to 1

2015-05-28 Thread Simon Riggs
history files, but that doesn't fix problems with non-pg_upgrade > upgrades, which also should never be sharing WAL files from previous > major versions. > Maybe, but I thought we had a high respect for backwards compatibility and we clearly just broke quite a few things that didn

Re: [HACKERS] Re: [COMMITTERS] pgsql: Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE.

2015-05-27 Thread Simon Riggs
On 27 May 2015 at 15:06, Alvaro Herrera wrote: > Simon Riggs wrote: > > > What I think should happen is that the command tag should vary according > to > > whether an INSERT or an UPDATE was performed, so we get a visible > > difference without any new tags. > &g

Re: [HACKERS] Re: [COMMITTERS] pgsql: Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE.

2015-05-27 Thread Simon Riggs
hink should happen is that the command tag should vary according to whether an INSERT or an UPDATE was performed, so we get a visible difference without any new tags. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Triggers on transaction?

2015-05-27 Thread Simon Riggs
sactionTrigger get access to changed rows? Can't you just use deferred triggers? I'm not clear exactly when such a trigger should run. If the trigger issues more SQL, which also has triggers... do we run the PreCommit trigger twice? Or just accept that we wanted it to run just prior t

Re: [HACKERS] Order of columns in query is important?!

2015-05-26 Thread Simon Riggs
cified columns *exactly* matches the tuple descriptor). > ...and it matters a lot in those cases because we are sorting all of the data scanned, not just 25 rows. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

[HACKERS] Re: 9.5 release notes may need ON CONFLICT DO NOTHING compatibility notice for FDW authors

2015-05-25 Thread Simon Riggs
way before the FDW is consulted at all, so FDW authors > need not concern themselves with those other cases (unless perhaps we > allow ON CONFLICT DO UPDATE to not require an inference specification > in a last minute behavioral tweak, as suggested by Simon Riggs, making > ON CONFLICT DO

Re: [HACKERS] INSERT ... ON CONFLICT DO UPDATE with _any_ constraint

2015-05-21 Thread Simon Riggs
all use the same UPDATE. I might point out that the MERGE syntax allowed us to handle that fully, but you'll probably scream. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] INSERT ... ON CONFLICT DO UPDATE with _any_ constraint

2015-05-21 Thread Simon Riggs
can at least agree a path for future enhancement we can refer people to what will happen in later releases when they ask. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] INSERT ... ON CONFLICT DO UPDATE with _any_ constraint

2015-05-21 Thread Simon Riggs
> defined on the table - is handled gracefully. > What has CREATE INDEX CONCURRENTLY got to do with this? If you don't specify the conflict-target at all, it wouldn't matter what the indexes are. If you have two indexes the same then it clearly wouldn't matter which one was checked first. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] INSERT ... ON CONFLICT DO UPDATE with _any_ constraint

2015-05-21 Thread Simon Riggs
On 20 May 2015 at 05:49, Geoff Winkless wrote: > On 19 May 2015 at 21:57, Simon Riggs wrote: > >> It's not clear to me how a single INSERT could cause two or more UPDATEs. >> > > ​ > CREATE TABLE mytable ( > c1 int NOT NULL, > c2 int NOT NULL

Re: [HACKERS] INSERT ... ON CONFLICT DO UPDATE with _any_ constraint

2015-05-21 Thread Simon Riggs
On 19 May 2015 at 19:59, Peter Geoghegan wrote: > On Tue, May 19, 2015 at 2:28 PM, Simon Riggs > wrote: > > On 19 May 2015 at 17:10, Peter Geoghegan wrote: > >> > >> On Tue, May 19, 2015 at 1:57 PM, Simon Riggs > >> wrote: > >> > We should

Re: [HACKERS] Redesigning checkpoint_segments

2015-05-21 Thread Simon Riggs
er, given different settings. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] a few thoughts on the schedule

2015-05-20 Thread Simon Riggs
tes, > one can answer all three questions without opening the diff itself. Each > appraiser could cover every patch of a CommitFest in an hour or two. I'm happy to participate as a "triager" and will follow whatever process we decide. I would very much like to make this som

Re: [HACKERS] INSERT ... ON CONFLICT DO UPDATE with _any_ constraint

2015-05-19 Thread Simon Riggs
On 19 May 2015 at 17:10, Peter Geoghegan wrote: > On Tue, May 19, 2015 at 1:57 PM, Simon Riggs > wrote: > > We should allow DO UPDATE to exclude a constraint and apply a > deterministic > > order to the constraints. 1. PK if it exists. 2. Replica Identity, when &g

Re: [HACKERS] RFC: Non-user-resettable SET SESSION AUTHORISATION

2015-05-19 Thread Simon Riggs
On 19 May 2015 at 16:49, Robert Haas wrote: > On Tue, May 19, 2015 at 3:00 PM, Simon Riggs > wrote: > > As long as the cookie is randomly generated for each use, then I don't > see a > > practical problem with that approach. > > If the client sets the cookie

Re: [HACKERS] INSERT ... ON CONFLICT DO UPDATE with _any_ constraint

2015-05-19 Thread Simon Riggs
they stay the same. It also saves time if they do change, because you avoid having to completely recode your app AND make that happen at exactly the same time you apply the change of unique constraint. 2) Compatibility with MySQL -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] a few thoughts on the schedule

2015-05-19 Thread Simon Riggs
est we go Beta1 on June 1, so we can discuss any problems arising in person in Ottawa. It's quicker than normal, but if we've lost a month or two we should just skip the usual "open items" chase, which can be done in parallel with users finding and reporting real bugs. -- Si

Re: [HACKERS] INSERT ... ON CONFLICT DO UPDATE with _any_ constraint

2015-05-19 Thread Simon Riggs
solve a conflict as such. > > > Thinking about it more, I suppose if multiple constraints end up > triggering for the same INSERT, it would require UPDATEs of multiple rows. > Is that the issue? > I'm sure we'll be asked these questions many times. Can yo

Re: [HACKERS] RFC: Non-user-resettable SET SESSION AUTHORISATION

2015-05-19 Thread Simon Riggs
, and only let > RESET SESSION AUTH work when the same cookie is supplied, is pretty > reasonable. > As long as the cookie is randomly generated for each use, then I don't see a practical problem with that approach. Protocol level solution means we have to wait 1.5 years bef

Re: [HACKERS] upper planner path-ification

2015-05-18 Thread Simon Riggs
oposal sounds like it will do that, but not completely sure. I'm assuming the O(N^2) Mat View planning problem can be solved in part by recognizing that many MVs are just single-table plus aggregates, and that we'd have a small enough number of MVs in play that search would not be a pr

Re: [HACKERS] ERROR: cannot GetMultiXactIdMembers() during recovery

2015-05-18 Thread Simon Riggs
time and nobody noticed till now, I'd agree with waiting. > That's a very low risk fix. It's more like a should-have-been-a-basic-check. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] ERROR: cannot GetMultiXactIdMembers() during recovery

2015-05-15 Thread Simon Riggs
a multixact > in it. > It can't return true and won't ever change for <9.3 so I don't see what the objection is. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] feature freeze and beta schedule

2015-05-15 Thread Simon Riggs
On 1 May 2015 at 18:05, Simon Riggs wrote: > * TABLESAMPLE clause >> Doesn't seem very far from being done. Some questions about including >> (or not) DDL and contrib modules seem to remain. >> > > Will commit this soon > OK, completely happy with th

Re: [HACKERS] Sequence Access Method WIP

2015-05-13 Thread Simon Riggs
hat to do instead. > Needs some thought. This has had around 2 years of thought at this point. I don't agree it needs more thought. There is one clear use case for this and it is of benefit to many distributed architectures. I don't see what calamity will occur if we commit

Re: [HACKERS] Multi-xacts and our process problem

2015-05-11 Thread Simon Riggs
ny other system. I think we seriously need to review our policy of adding major new features and have them enabled by default with no parameter to disable them. In the early years of PostgreSQL everything had an off switch, e.g. stats, bgwriter and even autovacuum defaulted to off for many years. --

Re: [HACKERS] subxcnt defined as signed integer in SnapshotData and SerializeSnapshotData

2015-05-08 Thread Simon Riggs
On 8 May 2015 at 13:02, Michael Paquier wrote: > On Fri, May 8, 2015 at 3:55 PM, Simon Riggs wrote: > > On 7 May 2015 at 21:40, Michael Paquier > wrote: > >> > >> Hi all, > >> > >> Coverity is complaining about the following assertion i

Re: [HACKERS] subxcnt defined as signed integer in SnapshotData and SerializeSnapshotData

2015-05-07 Thread Simon Riggs
am missing > something? > So the comment is wrong? It does not set to -1 at overflow anymore? -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] feature freeze and beta schedule

2015-05-01 Thread Simon Riggs
t; CF scheduling. > Agreed, bug fix. Will do if Fujii doesn't > * Auditing extension > I'm unclear on the status here. Abhijit said he'll have a look. > Maybe me, but running out of time > * TABLESAMPLE clause > Doesn't seem very far from being

Re: [HACKERS] Reducing tuple overhead

2015-04-30 Thread Simon Riggs
;t see that adding xmax or xmin would make any difference at all to bloat. So -1 to adding it **for that reason**. A much better idea is to work out how to avoid index bloat at cause. If we are running an UPDATE and we cannot get a cleanup lock, we give up and do a non-HOT update, causing the in

Re: [HACKERS] INSERT ... ON CONFLICT syntax issues

2015-04-29 Thread Simon Riggs
previous implementations we may draw support to adopt that as the standard. Whatever the standard says we will eventually support, so we should be acting with an eye to that future. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Feedback on getting rid of VACUUM FULL

2015-04-25 Thread Simon Riggs
fsm modes can also be set manually to enforce bloat minimization. We can also design a utility to actively use TARGBLOCK_NEW and FSM_SHRINK to reduce table size without blocking writes. But this is all stuff for 9.6... -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2nd

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-23 Thread Simon Riggs
r map page, we skip having a map at all when size < 1000. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Logical Decoding follows timelines

2015-04-21 Thread Simon Riggs
k, I think. I have a new approach to be coded for next release. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Replication identifiers, take 4

2015-04-21 Thread Simon Riggs
y used in some parts of the patch. I > think > > we should roughly do a search-replace of "replication identifier" -> > > "replication origin" to the patch. Or even "transaction origin". > > Sounds good to me. > +1 -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2015-04-20 Thread Simon Riggs
along every > now and then and does it thing. Why should the one massive SELECT have > horrible performance just because it was run right before autovacuum would > have kicked in instead of right after if finished? > +1 -- Simon Riggshttp://www.2ndQuadrant.com/ <

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2015-04-20 Thread Simon Riggs
leanup if we set hint bits or dirtied the block in other ways. Since we have many votes in favour of change in this area I'll post a new version and look for an early review/commit for next release. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> Post

Re: [HACKERS] Replication identifiers, take 4

2015-04-17 Thread Simon Riggs
essly wasteful for a change with only cosmetic value. But for a change that has significant value for database resilience, it is a sensible use of bytes. +1 to Andres' very reasonable suggestion. Lets commit this and go home. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Replication identifiers, take 4

2015-04-17 Thread Simon Riggs
On 17 April 2015 at 18:12, Heikki Linnakangas wrote: > On 04/17/2015 12:04 PM, Simon Riggs wrote: > >> On 17 April 2015 at 09:54, Andres Freund wrote: >> >> Hrmpf. Says the person that used a lot of padding, without much >>> discussion, for the WAL level inf

Re: [HACKERS] TABLESAMPLE patch

2015-04-17 Thread Simon Riggs
he change surface. I'm now moving to final review of patches 1-5. Michael requested patch 1 to be split out. If I commit, I will keep that split, but I am considering all of this as a single patchset. I've already spent a few days reviewing, so I don't expect this will take much lo

Re: [HACKERS] Moving on to close the current CF 2015-02

2015-04-17 Thread Simon Riggs
On 17 April 2015 at 08:22, Michael Paquier wrote: > To committers, here are the patches that seem on top of the list: > I'm pretty sure Committers are the people to decide which patches can be committed, but thanks for the opinion. -- Simon Riggshttp://www.2ndQ

Re: [HACKERS] Replication identifiers, take 4

2015-04-17 Thread Simon Riggs
On 17 April 2015 at 09:54, Andres Freund wrote: > Hrmpf. Says the person that used a lot of padding, without much > discussion, for the WAL level infrastructure making pg_rewind more > maintainable. Sounds bad. What padding are we talking about? -- Simon Riggsh

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2015-04-16 Thread Simon Riggs
On 16 April 2015 at 15:21, Andres Freund wrote: > On 2015-04-16 10:20:20 -0300, Alvaro Herrera wrote: > > I think you're failing to consider that in the patch there is a > > distinction between read-only page accesses and page updates. During a > > page update, HOT cleanup is always done even wi

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2015-04-15 Thread Simon Riggs
On 15 April 2015 at 16:01, Alvaro Herrera wrote: > Simon Riggs wrote: >> On 15 April 2015 at 12:39, Robert Haas wrote: > >> On 04/15/2015 05:44 PM, Alvaro Herrera wrote: >> > Robert's proposal is "when reading a page, if dirty HOT-clean it; if not >>

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2015-04-15 Thread Simon Riggs
On 15 April 2015 at 12:39, Robert Haas wrote: > On Wed, Apr 15, 2015 at 8:42 AM, Simon Riggs wrote: >>> I won't take responsibility for paying my neighbor's tax bill, but I >>> might take responsibility for picking up his mail while he's on >>> holid

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2015-04-15 Thread Simon Riggs
On 15 April 2015 at 09:10, Andres Freund wrote: > On 2015-04-15 08:42:33 -0400, Simon Riggs wrote: >> > Because it makes it subsequent accesses to the page cheaper. >> >> Cheaper for whom? > > Everyone. I think what you mean is "Everyone else". It is demo

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2015-04-15 Thread Simon Riggs
On 15 April 2015 at 08:04, Robert Haas wrote: > On Wed, Apr 15, 2015 at 3:37 AM, Simon Riggs wrote: >> On 14 April 2015 at 21:53, Robert Haas wrote: >>> Peter commented previously that README.HOT should get an update. The >>> relevant section seems to be &

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2015-04-15 Thread Simon Riggs
would be a useful heuristic to still prune pages if > those pages are already dirty. Useful for who? This is about responsibility. Why should someone performing a large SELECT take the responsibility for cleaning pages? -- Simon Riggs http://www.2ndQuadrant.com/ Postgr

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2015-04-14 Thread Simon Riggs
trivial to add/remove a parameter to control this. Currently there isn't one. I'd like to commit this. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, RemoteDBA, Training & Services hot_disable.v9.patch Description: Binary data -

Re: [HACKERS] [COMMITTERS] pgsql: Use Intel SSE 4.2 CRC instructions where available.

2015-04-14 Thread Simon Riggs
builds later if desired. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, RemoteDBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.or

<    5   6   7   8   9   10   11   12   13   14   >