>> At least for the moment he can provide a patch to be integrated in DM :-)
Sure, but as lightning is not in any public mvn repo should its code be
re-published in our svn? Or what?
>> perso I'd like we avoid hard dependency on Unsafe as maybe some use
other jdks :-)
Well, I believe Unsafe is supported by Sun JDK, OpenJDK, IBM JDK and
JRockit - and I believe that it is more than enough. Also keep in mind that
we already have an alternative Unsafe based memory storage - and although
it's not thoroughly tested for performance it dramaticaly simplifies code,
I have great expectations about it.
Cheers,
R
On Sat, Sep 29, 2012 at 4:03 PM, Olivier Lamy <[email protected]> wrote:
> 2012/9/29 Raffaele P. Guidi <[email protected]>:
> > What do you think about:
> > 1) implementing a lightning serialization module
> > 2) creating a serializer that directly works on a directmemory provider
> > ByteBuffer or (maybe better) Unsafe based Pointer?
>
> Sounds good (perso I'd like we avoid hard dependency on Unsafe as
> maybe some use other jdks :-) )
> >
> > Now I see lightning is apache licensed and this is fine but I don't think
> > it is published in any public maven repo, am I right? We could find a way
> > to deal with this; options vary from publishing lightning to the free
> > sonatype repo, joining the ASF (which is great anyhow!) and republishing
> > lightning code in DirectMemory becoming a commiter (which has to undergo
> a
> > PMC vote).
>
> At least for the moment he can provide a patch to be integrated in DM :-)
>
> >
> > I'd like to hear your and the team feelings on this.
> >
> > Thanks,
> > Raffaele
> >
> > On Sat, Sep 29, 2012 at 3:27 PM, Noctarius <[email protected]> wrote:
> >
> >> Hey Raffaele,
> >>
> >> that's quite similar to what I did at work. We're developing Flash
> >> online games and using a customized AMF serialization. To support
> >> async writing of the clients event results I added serialization of
> >> the components / entities to the players zone calculation and stored
> >> the pre-serialized bytestream directly to the off-heap (using
> >> direct-ring-cache implementation). When the client requests the
> >> results (using long-polling) I just write the pre-serialized data to
> >> the right position to deserialize it by standard ways on Flash side.
> >>
> >> So yeah an seriliaztion to off-heap algorithm would be fine. You can
> >> avoid using byte arrays and minimalize GC.
> >>
> >> Cheers
> >> Chris
> >>
> >> Am 29.09.2012 15:02, schrieb Raffaele P. Guidi:
> >> > Nice to hear back from you! Yes, I was thinking about creating a new
> >> memory
> >> > storage implementation using Unsafe (and I did it, recently) and
> also, as
> >> > DirectMemory relies heavily on serialization (and supports many of
> them,
> >> > protostuff, protobuf, msgpack and of course standard serialization),
> >> about
> >> > creating a simple embedded serializer leveraging the same techniques
> you
> >> > used (Unsafe and bytecode generation).
> >> > The idea with embedding is avoiding to serialize in a byte array and
> then
> >> > moving the byte array to off-heap memory (via Unsafe or ByteBuffers),
> and
> >> > serializing directly into a "managed" off-heap buffer, thus further
> >> > optimizing heap utilization (less GC).
> >> >
> >> > What do you think about it? Does it make any sense to you?
> >> >
> >> > Ciao,
> >> > R
> >> >
> >> > On Sat, Sep 29, 2012 at 2:40 PM, Noctarius <[email protected]> wrote:
> >> >
> >> >> Hey guys,
> >> >>
> >> >> Raffaele found out about a project of mine (Lightning) a few weeks
> >> >> ago. Lightning is a heavy Unsafe and Bytecode generation using
> >> >> Serializer implementation. He told me that he was interested in
> >> >> adding something similar to DirectMemory and I would be glad to help
> >> >> out in this.
> >> >>
> >> >> Another project I started a few days ago, since it was needed for
> >> >> work is DirectRingCache. The name not really meets to actual
> >> >> implementation since it's not yet a ring buffer using cache. I used
> >> >> this for a pre-serialization simple bytestream cache with
> >> >> self-growing buffers. It could be nice to have DirectMemory having
> >> >> raw "buffers" to write to or to read from.
> >> >>
> >> >> Here are the links from the projects:
> >> >> https://github.com/noctarius/Lightning
> >> >> https://github.com/noctarius/direct-ring-cache
> >> >>
> >> >> It would be nice to help out since I really like the idea of
> >> >> DirectMemory and since direct-ring-cache is some kind of reinventing
> >> >> the wheel.
> >> >>
> >> >> Cheers
> >> >> Noctarius (Chris)
> >> >>
> >> >
> >>
> >>
> >>
>
>
>
> --
> Olivier Lamy
> Talend: http://coders.talend.com
> http://twitter.com/olamy | http://linkedin.com/in/olamy
>
