> > > I just realised there is one issue with this design: We can't cheaply > > > reset the snapshot during the second table scan: > > > It is critically important that the second scan of R/CIC uses an index > > > contents summary (made with index_bulk_delete) that was created while > > > the current snapshot was already registered. > > > > > So, the "reset the snapshot every so often" trick cannot be applied in > > > phase 3 (the rescan), or we'd have to do an index_bulk_delete call > > > every time we reset the snapshot. Rescanning might be worth the cost > > > (e.g. when using BRIN), but that is very unlikely. > > > > Hm, I think it is still possible. We could just manually recheck the > > tuples we see > > to the snapshot currently used for the scan. If an "old" snapshot can see > > the tuple also (HeapTupleSatisfiesHistoricMVCC) then search for it in the > > index summary. > That's an interesting method. > > How would this deal with tuples not visible to the old snapshot? > Presumably we can assume they're newer than that snapshot (the old > snapshot didn't have it, but the new one does, so it's committed after > the old snapshot, making them newer), so that backend must have > inserted it into the index already, right?
I made a draft of the patch and this idea is not working. The problem is generally the same: * reference snapshot sees tuple X * reference snapshot is used to create index summary (but there is no tuple X in the index summary) * tuple X is updated to Y creating a HOT-chain * we started scan with new temporary snapshot (it sees Y, X is too old for it) * tuple X is pruned from HOT-chain because it is not protected by any snapshot * we see tuple Y in the scan with temporary snapshot * it is not in the index summary - so, we need to check if reference snapshot can see it * there is no way to understand if the reference snapshot was able to see tuple X - because we need the full HOT chain (with X tuple) for that Best regards, Michail.