On 2013-12-14 11:50:00 -0500, Robert Haas wrote: > On Fri, Dec 13, 2013 at 9:14 AM, Andres Freund <and...@2ndquadrant.com> wrote: > > But can you imagine those users needing an exported snapshot? I can > > think of several short-lived usages, but all of those are unlikely to > > need a consistent view of the overall database. And those are less > > likely to be full blown replication solutions. > > I.e. it's not the DBA making that decision but the developer making the > > decision based on whether he requires the snapshot or not. > > Well, it still seems to me that the right way to think about this is > that the change stream begins at a certain point, and then once you > cross a certain threshold (all transactions in progress at that time > have ended) any subsequent snapshot is a possible point from which to > roll forward.
Unfortunately it's not possible to build exportable snapshots at any time - it requires keeping far more state around since we need to care about all transactions, not just transactions touching the catalog. Currently you can only export the snapshot in the one point we become consistent, after that we stop maintaining that state. I see pretty much no chance to change that without major effort that's imo clearly out of scope for 9.4. Maybe we want to provide that at some point, but it's going to be a bit. Also note that in order to import a snapshot, the exporting snapshot still has to be alive, in the same transaction that has exported the snapshot. So, until the snapshot has been imported, replication cannot progress. That's existing limitations from the snapshot import code, but the reasoning for them make it unlikely that we can easily change them Under that light, usecases like your example won't be able to benefit from getting changes before the snapshot, or do you still see usecases for that? I've already prototyped the "quickstart" option that doesn't allow exporting a snapshot, although I don't know how the UI for it is going to look. > > I think I am just scared of needing to add more features before getting > > the basics done and in consequence overrunning 9.4... > > I am sensitive to that. On the other hand, this API is going to be a > lot harder to change once it's released, so we really need to avoid > painting ourselves into a corner with v1. I think we need to accept the fact that quite possibly we won't get everything right in 9.4. Obviously we should strive to avoid that, but trying to design things perfectly usually ends in not getting there. Yes, the API changes for FDWs caused pain. But there's no way we would have the current features if we'd tried to introduce the correct API allowing them from the get go. > As far as high-level design > concerns go, there are three things that I'm not happy with: > > 1. Slots. We know we need physical slots as well as logical slots, > but the patch as currently constituted only offers logical slots. Well, then tell me the way you want to go forward on that end. I can make the slot interface more generic if we know exactly what we need, but I doesn't seem fair to take this patch hostage until I develop a separate not so small feature. Why is that my task? Because I think it's important, and because by now I know the related code pretty well by now, I am willing to provide the parts of the that prevent required WAL from being deleted, peg xmin and report the current state to SQL, but somebody else is going to have to the rest. > s/pg_decoding_slot/pg_logical_stream/? Works for me. Regards, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (email@example.com) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers