Re: [HACKERS] Resource Owner reassign Locks

2015-08-27 Thread Tom Lane
I went ahead and pushed this into 9.2 and 9.1. It did not apply at all to 9.0, though, as there evidently was some refactoring affecting LockReassignCurrentOwner() between 9.0 and 9.1. We could possibly have applied some additional patches to 9.0, but I think that would be stretching the argument

Re: [HACKERS] Resource Owner reassign Locks

2015-08-25 Thread Tom Lane
Andres Freund writes: > On 2015-08-25 14:33:25 -0400, Tom Lane wrote: >> (IOW, yeah, certainly third-party code could create a new *instance* of >> the ResourceOwner data structure, but they would not have any knowledge of >> what's inside unless they had hacked the core code.) > What I was think

Re: [HACKERS] Resource Owner reassign Locks

2015-08-25 Thread Andres Freund
On 2015-08-25 14:33:25 -0400, Tom Lane wrote: > (IOW, yeah, certainly third-party code could create a new *instance* of > the ResourceOwner data structure, but they would not have any knowledge of > what's inside unless they had hacked the core code.) What I was thinking is that somebody created a

Re: [HACKERS] Resource Owner reassign Locks

2015-08-25 Thread Tom Lane
Andres Freund writes: > On 2015-08-25 14:12:37 -0400, Tom Lane wrote: >> How would they have done that without major code surgery? We don't have >> any hooks or function pointers involved in the users of resowner.h. >> Certainly locks would not be getting passed to a nonstandard resowner. > Curr

Re: [HACKERS] Resource Owner reassign Locks

2015-08-25 Thread Andres Freund
On 2015-08-25 14:12:37 -0400, Tom Lane wrote: > How would they have done that without major code surgery? We don't have > any hooks or function pointers involved in the users of resowner.h. > Certainly locks would not be getting passed to a nonstandard resowner. CurrentResourceOwner = myresowner;

Re: [HACKERS] Resource Owner reassign Locks

2015-08-25 Thread Tom Lane
Andres Freund writes: > On 2015-08-25 13:54:20 -0400, Tom Lane wrote: >> However, I'm not entirely following Andres' concern here. AFAICS, >> the only externally visible API change in commit eeb6f37d8 was that >> LockReleaseCurrentOwner and LockReassignCurrentOwner gained some >> arguments. That

Re: [HACKERS] Resource Owner reassign Locks

2015-08-25 Thread Andres Freund
On 2015-08-25 13:54:20 -0400, Tom Lane wrote: > Jeff Janes writes: > > Once the code has to be rewritten, my argument that it has been working "in > > the field" for a while doesn't really apply anymore. If rewriting involves adding two one line wrapper functions, I don't see the problem. > Howe

Re: [HACKERS] Resource Owner reassign Locks

2015-08-25 Thread Tom Lane
Jeff Janes writes: > On Tue, Aug 25, 2015 at 5:48 AM, Michael Paquier >> On Fri, Jul 10, 2015 at 4:22 AM, Andres Freund wrote: >>> That's the safest way. Sometimes you can decide that a function can not >>> sanely be called by external code and thus change the signature. But I'd >>> rather not r

Re: [HACKERS] Resource Owner reassign Locks

2015-08-25 Thread Jeff Janes
On Tue, Aug 25, 2015 at 5:48 AM, Michael Paquier wrote: > On Fri, Jul 10, 2015 at 4:22 AM, Andres Freund wrote: > > On July 9, 2015 9:13:20 PM GMT+02:00, Jeff Janes > wrote: > > > >>Unfortunately I don't know what that means about the API. Does it mean > >>that none of the functions declared i

Re: [HACKERS] Resource Owner reassign Locks

2015-08-25 Thread Michael Paquier
On Fri, Jul 10, 2015 at 4:22 AM, Andres Freund wrote: > On July 9, 2015 9:13:20 PM GMT+02:00, Jeff Janes wrote: > >>Unfortunately I don't know what that means about the API. Does it mean >>that none of the functions declared in any .h file can have their >>signatures changed? But new functions

Re: [HACKERS] Resource Owner reassign Locks

2015-07-09 Thread Andres Freund
On July 9, 2015 9:13:20 PM GMT+02:00, Jeff Janes wrote: >Unfortunately I don't know what that means about the API. Does it mean >that none of the functions declared in any .h file can have their >signatures changed? But new functions can be added? That's the safest way. Sometimes you can decid

Re: [HACKERS] Resource Owner reassign Locks

2015-07-09 Thread Jeff Janes
On Fri, Jul 3, 2015 at 12:59 AM, Andres Freund wrote: > On 2015-06-07 13:44:08 -0700, Jeff Janes wrote: > > I'd like to advocate for back-patching this to 9.0, 9.1, and 9.2. It has > > run without problems for a while now, and it can be considered a bug that > > systems with a very large number

Re: [HACKERS] Resource Owner reassign Locks

2015-07-03 Thread Andres Freund
On 2015-06-07 13:44:08 -0700, Jeff Janes wrote: > I'd like to advocate for back-patching this to 9.0, 9.1, and 9.2. It has > run without problems for a while now, and it can be considered a bug that > systems with a very large number of objects cannot be upgraded in a > reasonable time. In that c

Re: [HACKERS] Resource Owner reassign Locks

2015-06-07 Thread Andres Freund
On 2015-06-07 13:44:08 -0700, Jeff Janes wrote: > I'd like to advocate for back-patching this to 9.0, 9.1, and 9.2. It has > run without problems for a while now, and it can be considered a bug that > systems with a very large number of objects cannot be upgraded in a > reasonable time. +1 Greet

Re: [HACKERS] Resource Owner reassign Locks

2015-06-07 Thread Jeff Janes
On Thu, Jun 21, 2012 at 5:32 AM, Heikki Linnakangas < heikki.linnakan...@enterprisedb.com> wrote: > On 18.06.2012 13:59, Heikki Linnakangas wrote: > >> On 10.06.2012 23:39, Jeff Janes wrote: >> I found the interface between resowner.c and lock.c a bit confusing. >> resowner.c would sometimes call

Re: [HACKERS] Resource Owner reassign Locks

2012-06-23 Thread Jeff Janes
On Thu, Jun 21, 2012 at 5:32 AM, Heikki Linnakangas wrote: > On 18.06.2012 13:59, Heikki Linnakangas wrote: >> >> On 10.06.2012 23:39, Jeff Janes wrote: >> I found the interface between resowner.c and lock.c a bit confusing. >> resowner.c would sometimes call LockReassignCurrentOwner() to reassign

Re: [HACKERS] Resource Owner reassign Locks

2012-06-21 Thread Heikki Linnakangas
On 18.06.2012 13:59, Heikki Linnakangas wrote: On 10.06.2012 23:39, Jeff Janes wrote: I found the interface between resowner.c and lock.c a bit confusing. resowner.c would sometimes call LockReassignCurrentOwner() to reassign all the locks, and sometimes it would call LockReassignCurrentOwner() o

Re: [HACKERS] Resource Owner reassign Locks

2012-06-18 Thread Heikki Linnakangas
On 19.06.2012 09:02, Amit Kapila wrote: Please take a look to see if I broke something. In you previous mail you agreed with level as ERROR for elog message in function ResourceOwnerForgetLock(..) function, but in your modification you have used PANIC, is there any specific reason for it.

Re: [HACKERS] Resource Owner reassign Locks

2012-06-18 Thread Amit Kapila
> And it doesn't seem right for ResourceOwnerRemember/ForgetLock to have to accept a NULL owner. I am not sure, if it can ever enter into this flow without resowner as mentioned in jeff comments for session level locks. If it cannot enter then it is okay. > Please take a look to see if I brok

Re: [HACKERS] Resource Owner reassign Locks

2012-06-18 Thread Amit Kapila
> A better error message would be nice, but I don't think it's worth that > much. resowner.c doesn't currently know about the internals of LOCALLOCk > struct or lock tags, and I'd prefer to keep it that way. Let's just > print the pointer's address, that's what we do in many other > correspondi

Re: [HACKERS] Resource Owner reassign Locks

2012-06-18 Thread Heikki Linnakangas
On 10.06.2012 23:39, Jeff Janes wrote: The attached patch fixes that by remembering up to 10 local locks, and pushing them up specifically rather than digging through the entire lock table. If the list overflows, then it reverts to the old behavior of digging through the entire local lock table.

Re: [HACKERS] Resource Owner reassign Locks

2012-06-18 Thread Heikki Linnakangas
On 16.06.2012 09:04, Amit kapila wrote: I don't think so. C doesn't ref count its pointers. You are right I have misunderstood. I don't think that lock tags have good human readable formats, and just a pointer dump probably wouldn't be much use when something that can never happen has happene

Re: [HACKERS] Resource Owner reassign Locks

2012-06-15 Thread Amit kapila
t;tag.locktag_field1, lock->tag.locktag_field2, lock->tag.locktag_field3); This can give more information about erroneous lock. From: Jeff Janes [jeff.ja...@gmail.com] Sent: Saturday, June 16, 2012 3:21 AM To: Amit kapila Cc: pgsql-hackers Subject: Re: [H

Re: [HACKERS] Resource Owner reassign Locks

2012-06-15 Thread Jeff Janes
On Fri, Jun 15, 2012 at 3:29 PM, Tom Lane wrote: > Jeff Janes writes: >> On Mon, Jun 11, 2012 at 9:30 PM, Amit Kapila wrote: >>> MAX_RESOWNER_LOCKS - How did you arrive at number 10 for it. Is there any >>> specific reason for 10. > >> I instrumented the code to record the maximum number of lock

Re: [HACKERS] Resource Owner reassign Locks

2012-06-15 Thread Tom Lane
Jeff Janes writes: > On Mon, Jun 11, 2012 at 9:30 PM, Amit Kapila wrote: >> MAX_RESOWNER_LOCKS - How did you arrive at number 10 for it. Is there any >> specific reason for 10. > I instrumented the code to record the maximum number of locks held by > a resource owner, and report the max when it

Re: [HACKERS] Resource Owner reassign Locks

2012-06-15 Thread Jeff Janes
On Mon, Jun 11, 2012 at 9:30 PM, Amit Kapila wrote: >> Yes, that means the list has over-flowed.  Once it is over-flowed, it >> is now invalid for the reminder of the life of the resource owner. > Don't we need any logic to clear the reference of locallock in owner->locks > array. I don't think

Re: [HACKERS] Resource Owner reassign Locks

2012-06-11 Thread Amit Kapila
- From: Jeff Janes [mailto:jeff.ja...@gmail.com] Sent: Monday, June 11, 2012 8:52 PM To: Amit Kapila Cc: pgsql-hackers Subject: Re: [HACKERS] Resource Owner reassign Locks On Sun, Jun 10, 2012 at 11:28 PM, Amit Kapila wrote: > I have few doubts regarding logic of ResourceOwnerRememberLock() and >

Re: [HACKERS] Resource Owner reassign Locks

2012-06-11 Thread Jeff Janes
On Sun, Jun 10, 2012 at 11:28 PM, Amit Kapila wrote: > I have few doubts regarding logic of ResourceOwnerRememberLock() and > ResourceOwnerForgetLock(): > 1. In function ResourceOwnerRememberLock(), when lock count is > MAX_RESOWNER_LOCKS, it will not add the lock to lock array but increment the >

Re: [HACKERS] Resource Owner reassign Locks

2012-06-10 Thread Amit Kapila
e number of locks. -Original Message- From: pgsql-hackers-ow...@postgresql.org [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Jeff Janes Sent: Monday, June 11, 2012 2:10 AM To: pgsql-hackers Subject: [HACKERS] Resource Owner reassign Locks As discussed in several different email

[HACKERS] Resource Owner reassign Locks

2012-06-10 Thread Jeff Janes
As discussed in several different email threads here and on performance , when using pg_dump a on large number of objects, the server has a quadratic behavior in LockReassignCurrentOwner where it has to dig through the entire local lock table to push one or two locks up from the portal being droppe