Re: [HACKERS] Clarification about HOT
"Gokulakannan Somasundaram" <[EMAIL PROTECTED]> writes: >> Another reason is that it avoids the whole problem of updating multiple >> pages atomically, without deadlocks. > > > Thanks Heikki. I am still not getting what you said. In the case of HOT, > you need to update the top pointer to point to some other tuple in some > other page. That's one update. what's the other one? There are several problems, two that come to mind are: 1) How do you make the dead top pointer redirect to the first live pointer in the chain? Currently we store the item number of the first live tuple in the line pointer. You would have to keep the tuple around, though you could truncate it to just the tuple header. 2) When vacuuming how do you find the head of the chain when you're looking at a dead tuple? There's no space in the tuple header to store the head of the chain. Besides you want to vacuum scanning sequentially, not randomly. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's On-Demand Production Tuning ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Clarification about HOT
Gokulakannan Somasundaram wrote: Thanks Heikki. I am still not getting what you said. In the case of HOT, you need to update the top pointer to point to some other tuple in some other page. That's one update. what's the other one? say currently the top of heap chain points to (2,3) . Imagine we are making the HOT chain through the pages. there might be a situation it should start pointing to (4,5) after the tuple at (2,3) gets ready to be Vacuumed. We should just lock the page where the top of HOT chain resides and update it to point to (4,5). What else we should do atomically? Imagine one more update, and we end up with a HOT chain like this: (2,3) -> (4,5) -> (6,7) Where (2,3) is a redirecting line pointer, (4,5) is a tuple that can be vacuumed, and (6,7) is the a live tuple. When vacuuming (4,5), the redirecting line pointer (2,3) needs to be updated at the same time. The chain could be even longer, requiring either locking and modifying even more pages atomically, or doing the pruning in steps which leads to more WAL traffic among other things. It could be done, we already have to deal with locking two pages simultaneously in heap_update, but it's pretty darn complex. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Clarification about HOT
On Fri, Nov 02, 2007 at 10:27:27PM +0530, Gokulakannan Somasundaram wrote: > say currently the top of heap chain points to (2,3) . Imagine we are making > the HOT chain through the pages. there might be a situation it should start > pointing to (4,5) after the tuple at (2,3) gets ready to be Vacuumed. We > should just lock the page where the top of HOT chain resides and update it > to point to (4,5). What else we should do atomically? You have to lock (4,5) also to make sure it's still there after you update. Maybe just at that moment another vacuum saw that (4,5) can also be cleaned, you have to lock all the pages to make sure the change is atomic... As soon as you have to lock more than one page, deadlocks become a problem. Have a nice day, -- Martijn van Oosterhout <[EMAIL PROTECTED]> http://svana.org/kleptog/ > Those who make peaceful revolution impossible will make violent revolution > inevitable. > -- John F Kennedy signature.asc Description: Digital signature
Re: [HACKERS] Clarification about HOT
On 11/2/07, Heikki Linnakangas <[EMAIL PROTECTED]> wrote: > > Gokulakannan Somasundaram wrote: > > I understand that if you have to Vacuum a tuple, it has to satisfy the > > necessary snapshot requirements. i will never be able to reduce the > chain to > > just one, because the there is always a indirection at the top of HOT. I > > > understood this. > > > > My question was is it the only reason for the decision to stop HOT > across > > pages. > > Another reason is that it avoids the whole problem of updating multiple > pages atomically, without deadlocks. Thanks Heikki. I am still not getting what you said. In the case of HOT, you need to update the top pointer to point to some other tuple in some other page. That's one update. what's the other one? say currently the top of heap chain points to (2,3) . Imagine we are making the HOT chain through the pages. there might be a situation it should start pointing to (4,5) after the tuple at (2,3) gets ready to be Vacuumed. We should just lock the page where the top of HOT chain resides and update it to point to (4,5). What else we should do atomically? -- Thanks, Gokul. CertoSQL Project, Allied Solution Group. (www.alliedgroups.com)
Re: [HACKERS] Clarification about HOT
Gokulakannan Somasundaram wrote: I understand that if you have to Vacuum a tuple, it has to satisfy the necessary snapshot requirements. i will never be able to reduce the chain to just one, because the there is always a indirection at the top of HOT. I understood this. My question was is it the only reason for the decision to stop HOT across pages. Another reason is that it avoids the whole problem of updating multiple pages atomically, without deadlocks. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Clarification about HOT
On 11/2/07, Martijn van Oosterhout <[EMAIL PROTECTED]> wrote: > > On Fri, Nov 02, 2007 at 06:12:37PM +0530, Gokulakannan Somasundaram wrote: > > I am especially interested in the case of continuing the HOT chain > across > > pages. When we are actually reclaiming space, we should check the > snapshot > > and reclaim it. If it is HOT updated, we will leave the top most tuple > and > > take the rest. So then the top most tuple will remain always and any > index > > scan now has to make read two heap pages to reach the target entry.Isthis > > the only reason, it was left out? > > It's not that simple. At any point in time there may be dozens of > active snapshots, each of which might see a different tuple in the > chain. So to clear any tuple you have to wait until all active > snapshots are gone. You will almost never be able to reduce the chain > to just one. I understand that if you have to Vacuum a tuple, it has to satisfy the necessary snapshot requirements. i will never be able to reduce the chain to just one, because the there is always a indirection at the top of HOT. I understood this. My question was is it the only reason for the decision to stop HOT across pages. As for your original question and jumping across pages, why stop at > one. Why not chain HOT tuples down 100 pages? Because then it gets very > expensive. Not to mention the locking considerations. Better keep it > simple. If you have to jump 100 pages, then you have 100 versions of the same tuple, which are not to be Vacuumed. That's a heavily updated tuple indeed. Then you will have 100 index tuples and you will anyway visit all those versions in a normal index scan. The question is whether you want to visit it through the HOT chain/ through the index entries. If you visit it through HOT chain, indexes can be in reduced size. -- Thanks, Gokul. CertoSQL Project, Allied Solution Group. (www.alliedgroups.com)
Re: [HACKERS] Clarification about HOT
On Fri, Nov 02, 2007 at 06:12:37PM +0530, Gokulakannan Somasundaram wrote: > I am especially interested in the case of continuing the HOT chain across > pages. When we are actually reclaiming space, we should check the snapshot > and reclaim it. If it is HOT updated, we will leave the top most tuple and > take the rest. So then the top most tuple will remain always and any index > scan now has to make read two heap pages to reach the target entry.Is this > the only reason, it was left out? It's not that simple. At any point in time there may be dozens of active snapshots, each of which might see a different tuple in the chain. So to clear any tuple you have to wait until all active snapshots are gone. You will almost never be able to reduce the chain to just one. As for your original question and jumping across pages, why stop at one. Why not chain HOT tuples down 100 pages? Because then it gets very expensive. Not to mention the locking considerations. Better keep it simple. Have a nice day, -- Martijn van Oosterhout <[EMAIL PROTECTED]> http://svana.org/kleptog/ > Those who make peaceful revolution impossible will make violent revolution > inevitable. > -- John F Kennedy signature.asc Description: Digital signature
