Am 05.04.2013 07:35, schrieb Roman Levenstein: > On Thu, Apr 4, 2013 at 11:52 PM, Raffaele P. Guidi < > [email protected]> wrote: > >> Hi Chris, Sorry I didn't answer before, but I'm seriously lacking spare >> time lately - too much daylight work and personal issues. Your idea is >> intriguing, but honestly I didn't have a single moment to look into it. >> Also, I have to say that I didn't receive too much feedback after I >> released the unsafe based backend, so I guess we are also lacking overall >> attention on alternative storage systems - or, maybe, on the whole project. >> > Just an idea: > As one possible idea to attract attention to DM (or some related projects), > one could try to implement a few Big Data use-cases using the DM. For > example a backend for a Hadoop, Cassandra storage or something similar, so > that operations can be done in memory. This could provide a significant > speedup for many workloads.
I guess I have some spare time (somewhere next weeks) to play around an couchdb and maybe Hadoop backend for DM. So you've been thinking about using DM as some In-Memory cache for those backends? I guess this was added recently to Terracotta's BigMemory :-) > The whole Big Data area (especially Hadoop based systems) seems to be very > hot this days with a lot of attention and press/news coverage. In-memory > computations and storage are also getting hotter. Think about Hana from SAP > and ExoLogic from Oracle. Also GridGain recently announced their Hadoop > backend, which is a drop in replacement and so on. But most of those > solutions are pretty expensive. So, if DM & Co could show that they can > achieve almost the same for free or much cheaper it could be a very > convincing argument for many people. > > Regards, > -Roman > > It's probably largely my fault, as I didn't invest too much attention >> myself. In any case I believe a simpler approach could quite improve on >> performance and maintanability. I'd say go on experimenting, and see what >> comes up :) >> >> Ciao, >> R >> Il giorno 04/apr/2013 21:56, "Christoph Engelbert" <[email protected]> >> ha scritto: >> >>> Is the DM development fallen asleep? Would be bad because it has the >>> potential to be a good competitor to BigMemory or ElasticMemory. >>> >>> Chris >>> >>> >>> Am 02.04.2013 21:30, schrieb Christoph Engelbert: >>>> Hey guys, >>>> >>>> some time ago I started a new small pooled (or unpooled), >>>> partitioned storage implementation for using with ByteBuffer >>>> (Direct, Heap) and Unsafe. It has different selection algorithms for >>>> free partitions / slices (a partition buffer is sliced into smaller >>>> parts). Currently there is a simple RoundRobin selector, one with >>>> ThreadLocal allocation (very similar to the TLAB in the JVM) and one >>>> which uses the id of the currently thread executing cpu core >>>> (ProcessorLocal) which uses OS api (available on Windows / Linux). >>>> >>>> It features a rich SPI to plug in your own selector / partition / >>>> slice implementations so that many parts are easily extendable. >>>> >>>> Maybe we could use some ideas or the storage engine as the backend >>>> engine in DirectMemory. >>>> But as always I'm happy about any comments or suggestions on the >>>> implementation. >>>> >>>> At the moment a lot of documentation / Javadoc is missing but maybe >>>> someone will have a look into it. >>>> >>>> https://github.com/noctarius/direct-ring-cache >>>> >>>> Chris / Noc >>>
