On Thu, Jan 5, 2017 at 9:25 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Jan 4, 2017 at 6:05 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> Okay, so this optimization can work only after all the active >> transactions operating on a page are finished. If that is true, in >> some cases such a design can consume a lot of CPU traversing all the >> tuples in a page for un-setting the bit, especially when such tuples >> are less. > > I suppose. I didn't think that cost was likely to be big enough to > worry about, but I might be wrong. The worst case would be when you > modify one tuple on a page, let the transaction that did the > modification become all-visible, modify one tuple on the page again, > etc. and, at the same time, the page is entirely full of tuples. So > you keep having to loop over all the bits to clear them (they are all > clear except one, but you don't know that) and then re-set just one of > them. That's not free, but keep in mind that the existing system > would be forced to perform non-HOT updates in that situation, which > isn't free either. > > Also, I'm thinking the bit could be stored in the line pointer rather > than the tuple, because with this design we don't need > LP_UNUSED/LP_NORMAL/LP_REDIRECT/LP_DEAD any more. We could use one > bit to indicate dead or not-dead and the second bit to indicate > recently-modified or not-recently-modified. With that approach, > clearing the bits only requires iterating over the line pointer array, > not the tuples themselves. >
I think this can help in mitigating the overhead. However, now another question is if we just unset it when there is no other active transaction operating on the current page except for current transaction, then will that tuple be considered all-visible? I think no transaction operating on a page can't be taken as a guarantee for tuple to be marked as all-visible. If that is true, then what is advantage of clearing the bit? >>> We don't necessarily need UNDO to clean up the indexes, although it >>> might be a good idea. It would *work* to just periodically scan the >>> index for delete-marked items. For each one, visit the heap and see >>> if there's a version of that tuple present in the heap or current UNDO >>> that matches the index entry. If not, remove the index entry. >> >> I think this is somewhat similar to how we clean the index now and >> seems to be a workable design. However, don't you think that it will >> have somewhat similar characteristics for index-bloat as we have now? > > Yes, it would have similar characteristics. Thus we might want to do better. > >> OTOH for heap, it will help us to take away old versions away from the >> main heap, but still, we can't get rid of that space or reuse that >> space till we can clean corresponding index entries. > > I don't think that's true. If in-place update is ever allowed in > cases where indexed columns have been modified, then the index already > has to cope with the possibility that the heap tuple it can see > doesn't match the index. And if it can cope with that, then why do we > have to remove the index entry before reusing the heap TID? > No need. I think I can see what you are saying. >> UNDO has to be kept till heap page is marked as all visible. This is >> required to check the visibility of index. Now, I think the page can >> be marked as all visible when we removed corresponding dead entries in >> heap. I think the main point of discussion here is how vacuum will >> clean heap and index entries in this new system. Another idea could >> be that we allow undo entries to be removed (we can allow heap entry >> to be removed) based on if they are visible or not and then while >> traversing index we cross check in heap if the entry can be removed. >> Basically, check the TID and traverse undo contents if any to verify >> if the index entry can be removed. I think this sounds to be more >> complicated and less efficient than what I suggested earlier. > > In my proposal, when a tuple gets updated or deleted in the heap, we > go find the corresponding index entries and delete-mark them. When an > index tuple is delete-marked, we have to cross-check it against the > heap, because the index tuple might not match the version of the heap > tuple that our snapshot can see. Therefore, it's OK to discard the > heap page's UNDO before cleaning the indexes, because the index tuples > are already delete-marked and rechecks will therefore do the right > thing. > Okay, I think we could clean undo and heap without caring for the index. Basically, the idea works on the premise that we won't allow same value rows in the index for same TID and using rechecks we can identify the deleted tuple. On rethinking about the working of vacuum in this system, it seems we can clean the heap independently and then for index we crosscheck each of the entry in the heap that is marked as deleted. Now this has the advantage that we don't need to do two passes of the heap, but for the index, we might need to re-fetch heap pages randomly to detect if the delete marked row is actually deleted. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
