Hi all, I'd like to start a discussion around storage in Aurora.
I think one of the biggest mistakes we made in migrating our storage to H2 was deleting the memory stores as we moved. We made a pretty big bet that we could eventually make H2/relational databases work. I don't think that bet has paid off and that we need to revisit the direction we're taking. My belief is that the current H2/MyBatis approach is untenable for large production clusters, at least without changing our current single-master architecture. At Twitter we are already having to fight to keep GC manageable even without DbTaskStore enabled, so I don't see a path forward where we could eventually enable that. So far experiments with H2 off-heap storage have provided marginal (if any) gains. Would anyone object to restoring the in-memory stores and creating new implementations for the missing ones (UpdateStore)? I'd even go further and propose that we consider in-memory H2 and MyBatis a failed experiment and we drop that storage layer completely. Cheers, David
