Hi all

I was discussing problems of CDC with scientific community and they asked this 
simple question: "So you have efficient WAL archive on a very cheap storage, 
why don't you have a logical archive too?"
This seems like a wild idea. But really, we have a super expensive NVMe drives 
for OLTP workload. And use this devices to store buffer for data to be dumped 
into MapReduce\YT analytical system.
If OLAP cannot consume data fast enough - we are out of space due to repl slot.
If we have a WAL HA switchover - OLAP has a hole in the stream and have to 
resync data from the scratch.

If we could just run archive command ```archive-tool wal-push 
0000000900000F2C000000E1.logical``` with contents of logical replication - this 
would be super cool for OLAP. I'd prefer even avoid writing 
0000000900000F2C000000E1.logical to disk, i.e. push data on stdio or something 
like that.

What do you think?

Best regards, Andrey Borodin.

Reply via email to