[Hibernate] in-memory databases vs query-in-memory databases
Okay, this isn't 100% hibernate-related, but I figure you guys think a lot about stuff like this. Prevayler's pitch goes something like this: if all your data fits in memory, you don't need concurrent transactions, because every transaction will be demanding the same resource (cpu/memory bandwidth). Obviously this doesn't work if your data set is larger than (or might grow to become larger than) your available memory. I can see this being a fairly common situation. What if you were sure that the total amount of data needed to perform any given transaction could fit in memory? In that case, could you: 1) Load all pages needed by a transaction and pin them in memory 2) Perform the transaction Any number of transactions could be doing (1) at the same time, but there would be only a single thread permitted to do (2). Once a transaction has pinned all the pages it will need, it queues itself for (2). Once (2) is complete, the pages are unpinned (if the reference count for the page falls to zero, of course -- multiple transactions can pin the same page). This would eliminate the need for fine grained locking and most of the other stuff that makes databases complex, since only one thread is running transactions. Adding more transaction threads wouldn't win you anything, because a transaction on strictly-in-memory data never blocks. The downside is that you have to be certain that you'll never run a query that needs more data than you have memory. OTOH, 4GB of ram is pretty cheap these days. And if your query needs to pull 4GB of data off the disk [*], well, it's going to be incredibly slow in the first place. I don't think too many people run queries like that. - a -- Education is not filling a bucket but lighting a fire. -- WB Yeats [*] Note that simply selecting a 4GB table doesn't actually need to pull 4GB off the disk. --- This SF.net email is sponsored by: SF.net Giveback Program. Does SourceForge.net help you be more productive? Does it help you create better code? SHARE THE LOVE, and help us help YOU! Click Here: http://sourceforge.net/donate/ ___ hibernate-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/hibernate-devel
Re: [Hibernate] in-memory databases vs query-in-memory databases
What of you need to make a remote call, or send a message inside the txn? Your analysis assumes that the system is not distributed in any way. :) Adam Megacz wrote: Okay, this isn't 100% hibernate-related, but I figure you guys think a lot about stuff like this. Prevayler's pitch goes something like this: if all your data fits in memory, you don't need concurrent transactions, because every transaction will be demanding the same resource (cpu/memory bandwidth). Obviously this doesn't work if your data set is larger than (or might grow to become larger than) your available memory. I can see this being a fairly common situation. What if you were sure that the total amount of data needed to perform any given transaction could fit in memory? In that case, could you: 1) Load all pages needed by a transaction and pin them in memory 2) Perform the transaction Any number of transactions could be doing (1) at the same time, but there would be only a single thread permitted to do (2). Once a transaction has pinned all the pages it will need, it queues itself for (2). Once (2) is complete, the pages are unpinned (if the reference count for the page falls to zero, of course -- multiple transactions can pin the same page). This would eliminate the need for fine grained locking and most of the other stuff that makes databases complex, since only one thread is running transactions. Adding more transaction threads wouldn't win you anything, because a transaction on strictly-in-memory data never blocks. The downside is that you have to be certain that you'll never run a query that needs more data than you have memory. OTOH, 4GB of ram is pretty cheap these days. And if your query needs to pull 4GB of data off the disk [*], well, it's going to be incredibly slow in the first place. I don't think too many people run queries like that. - a -- Gavin King JBoss Group +61 410534454 http://hibernate.org --- This SF.net email is sponsored by: SF.net Giveback Program. Does SourceForge.net help you be more productive? Does it help you create better code? SHARE THE LOVE, and help us help YOU! Click Here: http://sourceforge.net/donate/ ___ hibernate-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/hibernate-devel
