discusses how modern Storage Class Memory (SCM), such as PCIe SSD and NVDIMMs are completely upending every assumption made about storage. To put this in perspective, you can now see storage latency that is practically on-par with things like lock acquisition[1].

Presumably the bulk of this difference should be handled by the OS, but there's probably things we should be considering too:

Tiered storage will become common. That's going to make avoiding things like bulk scans even more important. There's a tie-in to partitioning and indexes too.

The days of a SAN may be over. With memory, network and storage latency approaching parity it's not practical to concentrate any of these resources; that creates a bottleneck. This means people will be even more resistant to the idea of a single database server.

The cost of temporary data becomes much lower. At some point it probably makes sense to just mmap what's needed and move on.

Fortunately, I think our traditional reliance on OS caching has helped us... to some degree we've always treated storage as fast because a lot of requests would be coming from RAM anyway.

[1] Ok, 25x isn't exactly on-par, but considering this used to be 25,000x... "At 100K IOPS for a uniform random workload, a CPU has approximately 10 microseconds to process an I/O request. Because today's SCMs are often considerably faster at processing sequential or read-only workloads, this can drop to closer to 2.5 microseconds on commodity hardware. Even worse, since these requests usually originate from a remote source, network devices have to be serviced at the same rate, further reducing the available per-request processing time. To put these numbers in context, acquiring a single uncontested lock on today's systems takes approximately 20ns, while a non-blocking cache invalidation can cost up to 100ns, only 25x less than an I/O operation."
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble!

Sent via pgsql-hackers mailing list (
To make changes to your subscription:

Reply via email to