I Used to have two file storage implementations: the simplest one which
wrote every entry on disk as a separate file - terribly slow - and the
second one based on OrientDB, a nosql solution - fast but used all the heap
on its own - you can see my blog for reference. Maybe the JCS file storage
could fit?
On Wednesday, October 19, 2011, Ashish <[email protected]> wrote:
> On Wed, Oct 19, 2011 at 11:02 AM, Akash Ashok <[email protected]>
wrote:
>> Thanks Guys for responding on a quick reply. HBase is indeed my first
love
>> :)
>>
>> I was really interested in what DirectMemory could offer and am excited
to
>> see that its pretty awesome. I posed that question because I wasn't
really
>> sure as to what the rules were on the incubator :) Great to know I could
>> get started right away. Shall start digging thru the code.
>
> All ASF projects work the same way :)
>
>>
>> I went through the road map and have a few questions:
>>
>> 1. File Storage -
>> This was my main concern. This moves more towards a database approach or
to
>> be more specific Key Value store approach Aren't there existing file
based
>> solutions which can be used instead of developing another one?
>
> Yes there are. We can use any of key-value stores or Object DB's like
> BerkleyDB or others.
> It would be good to have a pluggable layer, where a persistant
> provider can be chosen.
> However, its OffHeap aka stored in RAM, so don't feel its a priority,
> and persistance would make sense when we evolve into a Cache
> framework.
>
>>
>> 2. Hazelcast for replication -
>> If i get this right the idea to use it more like a replication framework
and
>> then use that to cache them onto the DirectMemory on the respective
systems
>> ?
>
> Hazelcast already has released OffHeap storage. Moreover, its a
> complete Cache framework with provision of adding persistant stores.
> So I am very much against using it. Please note that I use Hazelcast
> myself, so not implying that its not a good framework.
>
> Going Distributed is a different story, and if you see the past
> conversations, our current focus is on OffHeap store.
> To go distributed, we have to move scope from OffHeap store to a Cache
> solution. Then comes the challenges Distributed or Replicated.
> Replicated is easy, but Distributed world would bring in a lot more
> design choice like Server or peer2peer architecture, what concurrency
> levels to support etc.
>
> My take
> 1. Implement OffHeap store
> 2. Benchmark it, refine it to give latencies that we need in
> production (would luv sub-milliseconds or keep then less than 2 ms)
>
> Once we reach here, we can think about going complete caching
> solution, followed by going distributed :)
>
>>
>> Lastly "I recently rewrote DM entirely for simplification." +1 on the
fact
>> of simplification.
>> As Einstien once said " Make things as simple as possible, not simpler"
:)
>>
>> Cheers,
>> Akash A
>>
>>
>>
>> On Wed, Oct 19, 2011 at 8:33 AM, Raffaele P. Guidi <
>> [email protected]> wrote:
>>
>>> Uhm, I'm new in the ASF, but I guess you just need to read some docs
(ok,
>>> we
>>> don't have many, just something in the old wiki @github and an initial
>>> roadmap proposal [1]), take a look at the code and the issues and get
>>> started :) I see googling around that you have interests in hadoop and
>>> hbase
>>> as well, so maybe you could start investigating sinergies with those
>>> products (just an idea, it is in the roadmap).
>>>
>>> In any case thanks for your interest and welcome aboard.
>>>
>>> Ciao,
>>> R
>>>
>>> [1] Roadmap proposal:
>>>
>>>
https://cwiki.apache.org/confluence/display/DIRECTMEMORY/2011/10/18/Apache+DirectMemory+-+initial+roadmap+discussion
>>>
>>> On Wed, Oct 19, 2011 at 4:44 AM, Akash Ashok <[email protected]>
>>> wrote:
>>>
>>> > I know this is still getting incubated. But I would like to get
involved
>>> in
>>> > the development. What are the rules like? Can I get involved ?
>>> >
>>> > Cheers,
>>> > Akash A
>>> >
>>>
>>
>
>
>
> --
> thanks
> ashish
>
> Blog: http://www.ashishpaliwal.com/blog
> My Photo Galleries: http://www.pbase.com/ashishpaliwal
>
