On Mon, Aug 7, 2023 at 1:46 PM José Neves <rafanev...@msn.com> wrote: > > Humm, that's... challenging. I faced some issues after "the fix" because I > had a couple of transactions with 25k updates, and I had to split it to be > able to push to our event messaging system, as our max message size is 10MB. > Relying on commit time would mean that all transaction operations will have > the same timestamp. If something goes wrong while my worker is pushing that > transaction data chunks, I will duplicate some data in the next run, so... > this wouldn't allow me to deal with data duplication. > Is there any other way that you see to deal with it? > > Right now I only see an option, which is to store all processed LSNs on the > other side of the ETL. I'm trying to avoid that overhead. >
Sorry, I don't understand your system enough to give you suggestions but if you have any questions related to how logical replication work then I might be able to help. -- With Regards, Amit Kapila.
