Re: [HACKERS] time table for beta1
Dan Ports d...@csail.mit.edu writes: On Mon, Apr 04, 2011 at 07:04:59PM -0400, Robert Haas wrote: On Mon, Apr 4, 2011 at 6:41 PM, Stephen Frost sfr...@snowman.net wrote: What'd be horribly useful would be the pid and the *time* that the lock was taken. Well, I don't think we're likely to redesign pg_locks at this point, so it's a question of making the best use of the fields we have to work with. Agreed. Note that the vxid of the transaction that took the lock is included in there, so that's at least something you could correlate with a logfile. Another problem is that supporting that would imply injecting gettimeofday() into the lock-acquisition sequence, and that's pretty damn expensive on some platforms. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] time table for beta1
Folks, I think it might be about time to start thinking concretely about when we might like to kick beta1 out the door. The open issues list still has 9 issues on it, but we now have patches awaiting review for most of those issues (credit where credit is due: Fujii Masao, Noah Misch, Joey Adams, Peter Eisentraut, Shigeru Hanada). I have the impression from the SSI threads that there might be an issue or two there that needs to be dealt with, but there again I think that there are patches already posted, and that we just need to get around to dealing with them. So unless someone is aware of a large reservoir of other work that has to be taken care of before we can proceed, it's probably time to think about setting a deadline for ourselves and trying to hit it. At the risk of getting laughed at, how about, say, ~2 weeks from now? Plus or minus a couple of days based on people's schedules and which day of the week we'd like the wrap to happen on. Thoughts? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] time table for beta1
Robert Haas robertmh...@gmail.com wrote: I have the impression from the SSI threads that there might be an issue or two there that needs to be dealt with, but there again I think that there are patches already posted, and that we just need to get around to dealing with them. There are patches for all known issues except one. Dan Ports was able to replicate the latest issue uncovered by YAMAMOTO Takashi using a particular DBT-2 configuration, found the issue, and posted a patch: http://archives.postgresql.org/pgsql-hackers/2011-04/msg00083.php An earlier assertion failure found by YAMAMOTO Takashi is fixed by a patch I posted: http://archives.postgresql.org/pgsql-bugs/2011-03/msg00352.php While not bugs, per se, the reporting for out-of-shared-memory was clumsy. This is addressed with this patch: http://archives.postgresql.org/pgsql-hackers/2011-03/msg01170.php The only user-visible change of that one is that a hint will appear when intended rather than getting the identical message without the hint from a lower-level function. The above patches look to me like they should be committable without needing a lot of committer time. None of them are very big. In investigating the locks which were not being cleaned up properly, Dan noticed that the pid wasn't showing on SIReadLock rows in pg_locks. He submitted a patch here which would always show the pid responsible for the lock: http://archives.postgresql.org/pgsql-hackers/2011-04/msg00033.php Jeff Davis questioned whether pid should continue to show after the end of the transaction or the closing of the connection (and therefore the process which the pid identifies). Not showing it for completed transactions would be trivial. Showing it after the transaction completes, until the connection closes should be doable, but not trivial. Of course, we could just leave it alone, but leaving the pid out for these rows looks a little funny and reduces useful information a bit. The one issue without a reasonable patch is that there are now three HTABs in shared memory which can grow until shared memory is exhausted, rather than the one in heavyweight locks which we had prior to 9.1. I think we're agreed that this is a bad thing, but my attempts to address this so far haven't satisfied Heikki. Heikki suggested an approach, but didn't respond as to whether I should try to code it up. I wasn't sure whether he might be going at it himself. I'll happily take a run at it if people want that. Should these items be on the open issues list? At the risk of getting laughed at, how about, say, ~2 weeks from now? Plus or minus a couple of days based on people's schedules and which day of the week we'd like the wrap to happen on. I feel good about that from an SSI perspective, as long as some committer can look at these patches. I can't comment on any of the other areas. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] time table for beta1
On Mon, Apr 4, 2011 at 11:33 AM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Should these items be on the open issues list? Yes, please. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] time table for beta1
Robert Haas robertmh...@gmail.com writes: At the risk of getting laughed at, how about, say, ~2 weeks from now? Seems reasonable to me. Plus or minus a couple of days based on people's schedules and which day of the week we'd like the wrap to happen on. Betas are usually done using the regular release process, which is wrap-on-Thursday-release-on-Monday (to accommodate both packagers who work weekdays and those who can only spare time on weekends). So we'd really be talking about code freeze on the 14th if we want release on the 18th. That might be a bit ambitious, but we can always slip it a week if unforeseen problems crop up. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] time table for beta1
Robert Haas robertmh...@gmail.com wrote: On Mon, Apr 4, 2011 at 11:33 AM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Should these items be on the open issues list? Yes, please. Done. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] time table for beta1
On Mon, Apr 4, 2011 at 4:54 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: At the risk of getting laughed at, how about, say, ~2 weeks from now? Seems reasonable to me. Plus or minus a couple of days based on people's schedules and which day of the week we'd like the wrap to happen on. Betas are usually done using the regular release process, which is wrap-on-Thursday-release-on-Monday (to accommodate both packagers who work weekdays and those who can only spare time on weekends). So we'd really be talking about code freeze on the 14th if we want release on the 18th. That might be a bit ambitious, but we can always slip it a week if unforeseen problems crop up. The following weekend appears to be Easter, so it'd have to be a 2 week slip. -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] time table for beta1
On Mon, Apr 4, 2011 at 12:01 PM, Dave Page dp...@pgadmin.org wrote: On Mon, Apr 4, 2011 at 4:54 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: At the risk of getting laughed at, how about, say, ~2 weeks from now? Seems reasonable to me. Plus or minus a couple of days based on people's schedules and which day of the week we'd like the wrap to happen on. Betas are usually done using the regular release process, which is wrap-on-Thursday-release-on-Monday (to accommodate both packagers who work weekdays and those who can only spare time on weekends). So we'd really be talking about code freeze on the 14th if we want release on the 18th. That might be a bit ambitious, but we can always slip it a week if unforeseen problems crop up. The following weekend appears to be Easter, so it'd have to be a 2 week slip. Well, that's certainly an incentive to try to hit the April 14th date. Having to wait until April 28th would be the pits, IMHO. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] time table for beta1
Betas are usually done using the regular release process, which is wrap-on-Thursday-release-on-Monday (to accommodate both packagers who work weekdays and those who can only spare time on weekends). So we'd really be talking about code freeze on the 14th if we want release on the 18th. That might be a bit ambitious, but we can always slip it a week if unforeseen problems crop up. I'm hoping we slip a week. Next week is MySQLCon, and both Selena and I will be crazy running around. But if necessary I'll draft someone else to do the announcements and similar; don't hold up a beta for PR. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] time table for beta1
On Mon, Apr 04, 2011 at 10:33:22AM -0500, Kevin Grittner wrote: There are patches for all known issues except one. Dan Ports was able to replicate the latest issue uncovered by YAMAMOTO Takashi using a particular DBT-2 configuration, found the issue, and posted a patch: Well, it would be good to have confirmation from Takashi that it actually fixed the problem he was seeing. I expect it did. If so, then yes, we do have a handle on all open SSI issues. In investigating the locks which were not being cleaned up properly, Dan noticed that the pid wasn't showing on SIReadLock rows in pg_locks. He submitted a patch here which would always show the pid responsible for the lock: Jeff Davis questioned whether pid should continue to show after the end of the transaction or the closing of the connection (and therefore the process which the pid identifies). Not showing it for completed transactions would be trivial. Showing it after the transaction completes, until the connection closes should be doable, but not trivial. Of course, we could just leave it alone, but leaving the pid out for these rows looks a little funny and reduces useful information a bit. I see Robert committed that one already. If there's a consensus that omitting the pid for committed transactions is the right thing to do, I'm happy to put together a patch. I think that is a better approach than trying to keep it after commit until the connection closes, but all of this is sufficiently minor that it's probably not worth worrying much about. The one issue without a reasonable patch is that there are now three HTABs in shared memory which can grow until shared memory is exhausted, rather than the one in heavyweight locks which we had prior to 9.1. I think we're agreed that this is a bad thing, but my attempts to address this so far haven't satisfied Heikki. Heikki suggested an approach, but didn't respond as to whether I should try to code it up. I wasn't sure whether he might be going at it himself. I'll happily take a run at it if people want that. I believe implementing that should just be a matter of making get_hash_entry bail out before element_alloc if the right flag is set, because partitioned hash tables already don't split buckets. Dan -- Dan R. K. Ports MIT CSAILhttp://drkp.net/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] time table for beta1
On Mon, Apr 4, 2011 at 2:37 PM, Dan Ports d...@csail.mit.edu wrote: I see Robert committed that one already. If there's a consensus that omitting the pid for committed transactions is the right thing to do, I'm happy to put together a patch. I think that is a better approach than trying to keep it after commit until the connection closes, but all of this is sufficiently minor that it's probably not worth worrying much about. I couldn't really figure out why that would be better, but if there's a reason I'm fine with it. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] time table for beta1
Robert Haas robertmh...@gmail.com wrote: Dan Ports d...@csail.mit.edu wrote: I see Robert committed that one already. If there's a consensus that omitting the pid for committed transactions is the right thing to do, I'm happy to put together a patch. I think that is a better approach than trying to keep it after commit until the connection closes, but all of this is sufficiently minor that it's probably not worth worrying much about. I couldn't really figure out why that would be better, but if there's a reason I'm fine with it. If people think that showing the pid which created the lock after the process has terminated is confusing, the reason I would lean toward not showing it after transaction completion is that we can check a bit-flag field which is already in front of us rather than making calls out to other code for each lock, which might get expensive, and perhaps compromise modularity. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] time table for beta1
On Mon, Apr 4, 2011 at 2:50 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Robert Haas robertmh...@gmail.com wrote: Dan Ports d...@csail.mit.edu wrote: I see Robert committed that one already. If there's a consensus that omitting the pid for committed transactions is the right thing to do, I'm happy to put together a patch. I think that is a better approach than trying to keep it after commit until the connection closes, but all of this is sufficiently minor that it's probably not worth worrying much about. I couldn't really figure out why that would be better, but if there's a reason I'm fine with it. If people think that showing the pid which created the lock after the process has terminated is confusing, the reason I would lean toward not showing it after transaction completion is that we can check a bit-flag field which is already in front of us rather than making calls out to other code for each lock, which might get expensive, and perhaps compromise modularity. Well, the flip side is that if you have appropriate logging turned on, you might be able to go back and look at what the transaction that took the lock actually did, which won't be possible if you arbitrarily throw the PID away. I'm not going to lose any sleep about this one whatever we decide, but my gut is that we have more important things to tinker with. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] time table for beta1
* Robert Haas (robertmh...@gmail.com) wrote: Well, the flip side is that if you have appropriate logging turned on, you might be able to go back and look at what the transaction that took the lock actually did, which won't be possible if you arbitrarily throw the PID away. What'd be horribly useful would be the pid and the *time* that the lock was taken.. Knowing just the pid blows, since the pid could technically end up reused (tho not terribly likely) in the time frame you're trying to figure out what happened during.. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] time table for beta1
On Mon, Apr 4, 2011 at 6:41 PM, Stephen Frost sfr...@snowman.net wrote: * Robert Haas (robertmh...@gmail.com) wrote: Well, the flip side is that if you have appropriate logging turned on, you might be able to go back and look at what the transaction that took the lock actually did, which won't be possible if you arbitrarily throw the PID away. What'd be horribly useful would be the pid and the *time* that the lock was taken.. Knowing just the pid blows, since the pid could technically end up reused (tho not terribly likely) in the time frame you're trying to figure out what happened during.. Well, I don't think we're likely to redesign pg_locks at this point, so it's a question of making the best use of the fields we have to work with. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] time table for beta1
On Mon, Apr 04, 2011 at 07:04:59PM -0400, Robert Haas wrote: On Mon, Apr 4, 2011 at 6:41 PM, Stephen Frost sfr...@snowman.net wrote: What'd be horribly useful would be the pid and the *time* that the lock was taken.. ?Knowing just the pid blows, since the pid could technically end up reused (tho not terribly likely) in the time frame you're trying to figure out what happened during.. Well, I don't think we're likely to redesign pg_locks at this point, so it's a question of making the best use of the fields we have to work with. Agreed. Note that the vxid of the transaction that took the lock is included in there, so that's at least something you could correlate with a logfile. -- Dan R. K. Ports MIT CSAILhttp://drkp.net/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
