On Wed, Oct 19, 2011 at 11:02 AM, Akash Ashok <[email protected]> wrote: > Thanks Guys for responding on a quick reply. HBase is indeed my first love > :) > > I was really interested in what DirectMemory could offer and am excited to > see that its pretty awesome. I posed that question because I wasn't really > sure as to what the rules were on the incubator :) Great to know I could > get started right away. Shall start digging thru the code.
All ASF projects work the same way :) > > I went through the road map and have a few questions: > > 1. File Storage - > This was my main concern. This moves more towards a database approach or to > be more specific Key Value store approach Aren't there existing file based > solutions which can be used instead of developing another one? Yes there are. We can use any of key-value stores or Object DB's like BerkleyDB or others. It would be good to have a pluggable layer, where a persistant provider can be chosen. However, its OffHeap aka stored in RAM, so don't feel its a priority, and persistance would make sense when we evolve into a Cache framework. > > 2. Hazelcast for replication - > If i get this right the idea to use it more like a replication framework and > then use that to cache them onto the DirectMemory on the respective systems > ? Hazelcast already has released OffHeap storage. Moreover, its a complete Cache framework with provision of adding persistant stores. So I am very much against using it. Please note that I use Hazelcast myself, so not implying that its not a good framework. Going Distributed is a different story, and if you see the past conversations, our current focus is on OffHeap store. To go distributed, we have to move scope from OffHeap store to a Cache solution. Then comes the challenges Distributed or Replicated. Replicated is easy, but Distributed world would bring in a lot more design choice like Server or peer2peer architecture, what concurrency levels to support etc. My take 1. Implement OffHeap store 2. Benchmark it, refine it to give latencies that we need in production (would luv sub-milliseconds or keep then less than 2 ms) Once we reach here, we can think about going complete caching solution, followed by going distributed :) > > Lastly "I recently rewrote DM entirely for simplification." +1 on the fact > of simplification. > As Einstien once said " Make things as simple as possible, not simpler" :) > > Cheers, > Akash A > > > > On Wed, Oct 19, 2011 at 8:33 AM, Raffaele P. Guidi < > [email protected]> wrote: > >> Uhm, I'm new in the ASF, but I guess you just need to read some docs (ok, >> we >> don't have many, just something in the old wiki @github and an initial >> roadmap proposal [1]), take a look at the code and the issues and get >> started :) I see googling around that you have interests in hadoop and >> hbase >> as well, so maybe you could start investigating sinergies with those >> products (just an idea, it is in the roadmap). >> >> In any case thanks for your interest and welcome aboard. >> >> Ciao, >> R >> >> [1] Roadmap proposal: >> >> https://cwiki.apache.org/confluence/display/DIRECTMEMORY/2011/10/18/Apache+DirectMemory+-+initial+roadmap+discussion >> >> On Wed, Oct 19, 2011 at 4:44 AM, Akash Ashok <[email protected]> >> wrote: >> >> > I know this is still getting incubated. But I would like to get involved >> in >> > the development. What are the rules like? Can I get involved ? >> > >> > Cheers, >> > Akash A >> > >> > -- thanks ashish Blog: http://www.ashishpaliwal.com/blog My Photo Galleries: http://www.pbase.com/ashishpaliwal
