Hi,


On Sat, Sep 29, 2012 at 9:56 PM, Raffaele P. Guidi
<[email protected]> wrote:
> Really nice ideas - we already have an Unsafe based storage (it is rather
> new) and the idea was exactly to combine the benefits of it with the ones
> of unsafe serialization. Said that keep in mind that classic DirectMemory
> is in any case way faster than just using ByteBuffers because it
> pre-allocates large chunks of memory and then "slices" them for single,
> smaller items.

Yes, pre-allocation is good. But you should really see the performance
benefits of writing/reading off-heap memory using Unsafe :-)
It is way faster, especially for arrays of primitive types. The link I
mentioned contained some preliminary figures based on my own
benchmarks.
Therefore I think even Lightning (or any other serialization lib)
could benefit from using Unsafe-based off-heap Input/Output streams.
BTW, my streams support both DirectBuffer-based and
Unsafe.allocateMemory-based (i.e. pre-allocated) off-heap memory
access. Therefore, it is a nice fit with DirectMemory.

> And - a Kryo serializer is a rather good idea. Do you feel like working on
> it? Serializers are actually one of the easiest thing to contribute on in
> DM - and they add immediate value as each one is probably a better fit than
> others in different scenarios.

Yes, I think I could work on it. But first, I'd like to land these
improvements in Kryo's trunk so that I don't need to work on a branch
or a fork of Kryo.

Ciao,
  Roman

> Il giorno 29/set/2012 21:16, "Roman Levenstein" <[email protected]> ha
> scritto:
>
>> Hi,
>>
>> I'm just wondering if lightning uses Unsafe only for serializers (i.e.
>> for reading/writing fields of objects) or also for the input/output
>> streams used to read/write to/from off-heap memory? Using
>> DirectBuffers "as is" for accessing off-heap memory is actually rather
>> slow.
>>
>> Let me share some experiences  with you:
>> I have implemented a similar feature for Kryo. It is not in the trunk
>> yet, as it is being reviewed by Nate, the author of Kryo.
>> One of the things that I noticed is that using Unsafe only for
>> serializers is nice, but does not provide much improvement.
>> At the same time, using Unsafe for input/output streams and
>> reading/writing directly from/into off-heap memory provides
>> significant performance benefits.
>> My patch for Kryo implements both features using Unsafe. If you want
>> to see more details about the implementation and findings, please have
>> a look at:
>> http://code.google.com/p/kryo/issues/detail?id=75
>>
>> I hope it will land in Kryo trunk soon. Once it is there, I think it
>> could be interesting to add a Kryo-based serialization to
>> DirectMemory. What do you think?
>> But independent of this, feel free to take any ideas or even the
>> implementation from the patch. The off-heap memory streams
>> implementation is rather self-contained and could be reused with other
>> frameworks besides Kryo.
>>
>> Regards,
>>   Roman
>>
>> On Sat, Sep 29, 2012 at 8:46 PM, Noctarius <[email protected]> wrote:
>> > Actually all dependencies should be AL2 or BSD licensed :-)
>> >
>> > Am 29.09.2012 20:42, schrieb Raffaele P. Guidi:
>> >>>> Hehe well that really sounds like a nice bunch of people.
>> >>
>> >> Indeed they are (I'm a newbie as well and try to do my best)
>> >>
>> >>>> If lightning will be a sub-part (sub-project) of DM, do I
>> >>>> need to write
>> >> an project purposal?
>> >>
>> >> Nope, not needed for a sub-project
>> >>
>> >>>> Do I need to make any changes to the pom.xml like adding a
>> >> special parent pom or anything like that?
>> >>
>> >> Not for the serializer - just have to take a look at project
>> >> dependencies - or, better, at their licenses - are they
>> >> compatible with the ASL 2.0? i.e. a GPL'd library is not a good
>> >> fit and should be replaced with an apache licensed (or BSD, or
>> >> MIT...) one if possible. For the integration module is a
>> >> separate story - you should start off copying one of the other
>> >> serializers and reusing the same pom and directory structure.
>> >>
>> >> Pleased to meet you, Chris :)
>> >>
>> >> Ciao, R
>> >>
>> >> On Sat, Sep 29, 2012 at 8:24 PM, Noctarius <[email protected]>
>> >> wrote:
>> >>
>> >>> Hehe well that really sounds like a nice bunch of people.
>> >>>
>> >>> Ok to be true I couldn't wait until tomorrow and started
>> >>> already reading the links. From what I was reading: If
>> >>> lightning will be a sub-part (sub-project) of DM, do I need
>> >>> to write an project purposal?
>> >>>
>> >>> Do I need to make any changes to the pom.xml like adding a
>> >>> special parent pom or anything like that?
>> >>>
>> >>> In general: there are a lot things to know :-)
>> >>>
>> >>> Am 29.09.2012 19:59, schrieb Raffaele P. Guidi:
>> >>>> Negative part of ASF membership? You get together with a
>> >>>> lot of geeky, talented people with a fixation for software
>> >>>> and open source. Oh wait but this is actually nice! :-D Il
>> >>>> giorno 29/set/2012 19:05, "Olivier Lamy" <[email protected]>
>> >>>> ha
>> >>> scritto:
>> >>>>
>> >>>>> 2012/9/29 Noctarius <[email protected]>:
>> >>>>>> Thanks Olivier for carify, I'll take a look in it
>> >>>>>> tomorrow but there's just one question left (for now
>> >>>>>> ;)): What is that vote for becoming a committer? What
>> >>>>>> if the vote will be negative?
>> >>>>> The vote is on private list (pmc list for privacy reasons
>> >>>>> and possible negative stuff being on public lists)
>> >>>>>> Until now I just used Apache stuff, was never
>> >>>>>> interested in being part of it so I guess it can be
>> >>>>>> negative for any reason, can't it?
>> >>>>> I don't see why it could be negative but suspens ....
>> >>>>> :-)
>> >>>>>>
>> >>>>>> Am 29.09.2012 18:56, schrieb Olivier Lamy:
>> >>>>>>> 2012/9/29 Noctarius <[email protected]>:
>> >>>>>>>> Nope my real name is Christoph Engelbert, but
>> >>>>>>>> Noctarius is the all time nick :)
>> >>>>>>>>
>> >>>>>>>> Renaming the package should be no problem, is it
>> >>>>>>>> "org.apache.directmemory.lightning" or what would
>> >>>>>>>> it be?
>> >>>>>>> fine for me
>> >>>>>>>> Then there needs to be a change in the license
>> >>>>>>>> header as Olivier mentioned, that means just remove
>> >>>>>>>> the first sentence or is there anything more to do
>> >>>>>>>> (maybe it's easiest thing to just copy the header
>> >>>>>>>> from DM file ;))?
>> >>>>>>> yup use same header as DM
>> >>>>>>>>
>> >>>>>>>> The CLA is just a form to clarify that the source
>> >>>>>>>> can be contributed to the Apache Foundation?
>> >>>>>>> yup correct.
>> >>>>>>>>
>> >>>>>>>> The final step will be attaching the patch in form
>> >>>>>>>> of a huge diff file?
>> >>>>>>> yes
>> >>>>>>>>
>> >>>>>>>> And what is the way to apply for a membership?
>> >>>>>>>> Never thought about how to do that.
>> >>>>>>> Read here
>> >>>>>>> http://apache.org/foundation/how-it-works.html and
>> >>>>>>> here http://apache.org/foundation/getinvolved.html .
>> >>>>>>> And feel free to ask any questions :-)
>> >>>>>>>>
>> >>>>>>>> Chris
>> >>>>>>>>
>> >>>>>>>> Am 29.09.2012 18:23, schrieb Raffaele P. Guidi:
>> >>>>>>>>> OK, deal, at least for me ;-) I propose you
>> >>>>>>>>> rename the packages, produce a patch for this and
>> >>>>>>>>> the new serializer module (should be simple
>> >>>>>>>>> enough starting from an existing one) and, in the
>> >>>>>>>>> meanwhile, apply for ASF membership. Is IP
>> >>>>>>>>> clearance needed? I guess yes. After this we will
>> >>>>>>>>> come up with a formal vote regarding Noctarius
>> >>>>>>>>> (is this your real name?!) allowance in the
>> >>>>>>>>> project team.
>> >>>>>>>>>
>> >>>>>>>>> Good times are gonna come :-)
>> >>>>>>>>>
>> >>>>>>>>> Thanks, R Il giorno 29/set/2012 17:58, "Olivier
>> >>>>>>>>> Lamy" <[email protected]> ha scritto:
>> >>>>>>>>>
>> >>>>>>>>>> 2012/9/29 Raffaele P. Guidi
>> >>>>>>>>>> <[email protected]>:
>> >>>>>>>>>>> Well we already have a NIO ready interface
>> >>>>>>>>>>> allowing direct access to DMs managed
>> >>>>>>>>>>> bytebuffers but I think this is just half way
>> >>>>>>>>>>> to what could be achieved optimally blending
>> >>>>>>>>>>> serialization and memory allocation
>> >>>>>>>>>>> together.
>> >>>>>>>>>>>
>> >>>>>>>>>>> Lightning as a module is of course a good
>> >>>>>>>>>>> idea and it could easily evolve as a
>> >>>>>>>>>>> subproject (for the more experienced asf
>> >>>>>>>>>>> members: is it a feasible way?).
>> >>>>>>>>>> Nothing prevent to have
>> >>>>>>>>>> http://svn.apache.org/repos/asf/directmemory/lightning/trunk
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>
>> >>>>>>>>>>
>> > with a package like: o.a.d.lightning That will be a
>> >>>>>>>>>> subproject.
>> >>>>>>>>>>>
>> >>>>>>>>>>> Ciao, R Il giorno 29/set/2012 17:44,
>> >>>>>>>>>>> "Noctarius" <[email protected]> ha scritto:
>> >>>>>>>>>>>
>> >>>>>>>>>>>> Hey guys,
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> ok there's no lightning binary available
>> >>>>>>>>>>>> since lightning wasn't ready yet for
>> >>>>>>>>>>>> releasing.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> For being the only developer it would be no
>> >>>>>>>>>>>> problem to contribute the sourcebase for
>> >>>>>>>>>>>> DirectMemory but I'm not sure yet if it
>> >>>>>>>>>>>> wouldn't be better to seperate it to be
>> >>>>>>>>>>>> available without using DirectMemory
>> >>>>>>>>>>>> itself. I started it as a serializer for
>> >>>>>>>>>>>> cluster synchronization, but it would be
>> >>>>>>>>>>>> cool to contribute lightning as a
>> >>>>>>>>>>>> subproject to DirectMemory :-)
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> About the second project I would love to
>> >>>>>>>>>>>> see a public available buffer API directly
>> >>>>>>>>>>>> in DirectMemory so that project would be
>> >>>>>>>>>>>> nearly needless :-) The only difference I
>> >>>>>>>>>>>> think is the allocation strategy my
>> >>>>>>>>>>>> implementation is using against the one
>> >>>>>>>>>>>> DirectMemory has, but I'm pretty sure the
>> >>>>>>>>>>>> allocation is extensible ;-)
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Like I said, for both projects I'm the only
>> >>>>>>>>>>>> dev so there would be no IP problem. So if
>> >>>>>>>>>>>> it's ok to you to not include lightning
>> >>>>>>>>>>>> directly in DM I would be glad to
>> >>>>>>>>>>>> contribute to the Apache Foundation :-)
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Cheers Chris
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Am 29.09.2012 16:53, schrieb Raffaele P.
>> >>>>>>>>>>>> Guidi:
>> >>>>>>>>>>>>> Ok so it's up to noctarius - your move!
>> >>>>>>>>>>>>> ;-) Regarding the new unsafe storage:
>> >>>>>>>>>>>>> it's an opt-in feature that can be set
>> >>>>>>>>>>>>> with the fluent API
>> >>>>>>>>>> (and
>> >>>>>>>>>>>>> soon through the conference file).
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Ciao, R Il giorno 29/set/2012 16:45,
>> >>>>>>>>>>>>> "Olivier Lamy" <[email protected]> ha
>> >>>>>>>>>>>> scritto:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>> 2012/9/29 Raffaele P. Guidi
>> >>>>>>>>>>>>>> <[email protected]>:
>> >>>>>>>>>>>>>>>>> At least for the moment he can
>> >>>>>>>>>>>>>>>>> provide a patch to be integrated
>> >>>>>>>>>> in DM
>> >>>>>>>>>>>>>> :-)
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Sure, but as lightning is not in any
>> >>>>>>>>>>>>>>> public mvn repo should its
>> >>>>>>>>>> code be
>> >>>>>>>>>>>>>>> re-published in our svn? Or what?
>> >>>>>>>>>>>>>> @Apache we don't care about binaries,
>> >>>>>>>>>>>>>> only sources are important ! (a bit
>> >>>>>>>>>>>>>> theorical for sure but that's it :-) ).
>> >>>>>>>>>>>>>> So if Noctarius was the only guy who
>> >>>>>>>>>>>>>> participate in lightning. He can just
>> >>>>>>>>>>>>>> provide a patch we could integrate as a
>> >>>>>>>>>>>>>> new dm module (note: the patch must not
>> >>>>>>>>>>>>>> contains any more copyright and all
>> >>>>>>>>>>>>>> sources must have ASF licenses).
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> "Copyright 2012 the original author or
>> >>>>>>>>>>>>>> authors." must be removed. And BTW
>> >>>>>>>>>>>>>> package must be changed :-) (com.github
>> >>>>>>>>>>>>>> is not acceptable
>> >>>>>>>>>> @asf
>> >>>>>>>>>>>>>> :-) )(@Noctarius are you working for
>> >>>>>>>>>>>>>> github ? :-) )
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> And having him as a committer will be
>> >>>>>>>>>>>>>> only a matter of voting (we
>> >>>>>>>>>> have
>> >>>>>>>>>>>>>> a great chair who take cares of
>> >>>>>>>>>>>>>> administrative stuff :P )
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> If some others have participated in the
>> >>>>>>>>>>>>>> project, we must pass tru an ip
>> >>>>>>>>>>>>>> clearance mechanism
>> >>>>>>>>>>>>>> (http://incubator.apache.org/ip-clearance/index.html)
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>
>> >>>>>>>>>>>>>>
>> > and all contributors to lightning must provide a cla.
>> >>>>>>>>>>>>>> (It it's the case I can help)
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> perso I'd like we avoid hard
>> >>>>>>>>>>>>>>>>> dependency on Unsafe as maybe
>> >>>>>>>>>>>>>>>>> some
>> >>>>>>>>>> use
>> >>>>>>>>>>>>>>> other jdks :-)
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Well, I believe Unsafe is supported
>> >>>>>>>>>>>>>>> by Sun JDK, OpenJDK, IBM JDK and
>> >>>>>>>>>>>>>>> JRockit - and I believe that it is
>> >>>>>>>>>>>>>>> more than enough. Also keep in
>> >>>>>>>>>> mind
>> >>>>>>>>>>>>>> that
>> >>>>>>>>>>>>>>> we already have an alternative Unsafe
>> >>>>>>>>>>>>>>> based memory storage - and
>> >>>>>>>>>>>> although
>> >>>>>>>>>>>>>>> it's not thoroughly tested for
>> >>>>>>>>>>>>>>> performance it dramaticaly
>> >>>>>>>>>>>>>>> simplifies
>> >>>>>>>>>>>>>> code,
>> >>>>>>>>>>>>>>> I have great expectations about it.
>> >>>>>>>>>>>>>> Me too :-). Yup definitely more simple
>> >>>>>>>>>>>>>> and faster ! But we must provide a
>> >>>>>>>>>>>>>> switch off configuration mechanism if
>> >>>>>>>>>>>>>> some
>> >>>>>>>>>> users
>> >>>>>>>>>>>>>> don't want to use that (because Unsafe
>> >>>>>>>>>>>>>> is just "Unsafe" :-) ) And sorry I
>> >>>>>>>>>>>>>> didn't have a look yet at your changes
>> >>>>>>>>>>>>>> with using Unsafe.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Cheers, R
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> On Sat, Sep 29, 2012 at 4:03 PM,
>> >>>>>>>>>>>>>>> Olivier Lamy <[email protected]>
>> >>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> 2012/9/29 Raffaele P. Guidi
>> >>>>>>>>>>>>>>>> <[email protected]>:
>> >>>>>>>>>>>>>>>>> What do you think about: 1)
>> >>>>>>>>>>>>>>>>> implementing a lightning
>> >>>>>>>>>>>>>>>>> serialization module 2) creating
>> >>>>>>>>>>>>>>>>> a serializer that directly works
>> >>>>>>>>>>>>>>>>> on a directmemory
>> >>>>>>>>>>>>>> provider
>> >>>>>>>>>>>>>>>>> ByteBuffer or (maybe better)
>> >>>>>>>>>>>>>>>>> Unsafe based Pointer?
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Sounds good (perso I'd like we
>> >>>>>>>>>>>>>>>> avoid hard dependency on Unsafe as
>> >>>>>>>>>>>>>>>> maybe some use other jdks :-) )
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> Now I see lightning is apache
>> >>>>>>>>>>>>>>>>> licensed and this is fine but I
>> >>>>>>>>>> don't
>> >>>>>>>>>>>>>> think
>> >>>>>>>>>>>>>>>>> it is published in any public
>> >>>>>>>>>>>>>>>>> maven repo, am I right? We could
>> >>>>>>>>>> find a
>> >>>>>>>>>>>>>> way
>> >>>>>>>>>>>>>>>>> to deal with this; options vary
>> >>>>>>>>>>>>>>>>> from publishing lightning to the
>> >>>>>>>>>> free
>> >>>>>>>>>>>>>>>>> sonatype repo,  joining the ASF
>> >>>>>>>>>>>>>>>>> (which is great anyhow!) and
>> >>>>>>>>>>>>>> republishing
>> >>>>>>>>>>>>>>>>> lightning code in DirectMemory
>> >>>>>>>>>>>>>>>>> becoming a commiter (which has
>> >>>>>>>>>>>>>>>>> to
>> >>>>>>>>>>>>>> undergo
>> >>>>>>>>>>>>>>>> a
>> >>>>>>>>>>>>>>>>> PMC vote).
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> At least for the moment he can
>> >>>>>>>>>>>>>>>> provide a patch to be integrated
>> >>>>>>>>>>>>>>>> in
>> >>>>>>>>>> DM
>> >>>>>>>>>>>>>> :-)
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> I'd like to hear your and the
>> >>>>>>>>>>>>>>>>> team feelings on this.
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> Thanks, Raffaele
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> On Sat, Sep 29, 2012 at 3:27 PM,
>> >>>>>>>>>>>>>>>>> Noctarius <[email protected]>
>> >>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Hey Raffaele,
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> that's quite similar to what I
>> >>>>>>>>>>>>>>>>>> did at work. We're developing
>> >>>>>>>>>> Flash
>> >>>>>>>>>>>>>>>>>> online games and using a
>> >>>>>>>>>>>>>>>>>> customized AMF serialization.
>> >>>>>>>>>>>>>>>>>> To support async writing of the
>> >>>>>>>>>>>>>>>>>> clients event results I added
>> >>>>>>>>>>>>>>>>>> serialization
>> >>>>>>>>>> of
>> >>>>>>>>>>>>>>>>>> the components / entities to
>> >>>>>>>>>>>>>>>>>> the players zone calculation
>> >>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>> stored
>> >>>>>>>>>>>>>>>>>> the pre-serialized bytestream
>> >>>>>>>>>>>>>>>>>> directly to the off-heap (using
>> >>>>>>>>>>>>>>>>>> direct-ring-cache
>> >>>>>>>>>>>>>>>>>> implementation). When the
>> >>>>>>>>>>>>>>>>>> client requests the results
>> >>>>>>>>>>>>>>>>>> (using long-polling) I just
>> >>>>>>>>>>>>>>>>>> write the pre-serialized
>> >>>>>>>>>> data to
>> >>>>>>>>>>>>>>>>>> the right position to
>> >>>>>>>>>>>>>>>>>> deserialize it by standard ways
>> >>>>>>>>>>>>>>>>>> on Flash
>> >>>>>>>>>> side.
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> So yeah an seriliaztion to
>> >>>>>>>>>>>>>>>>>> off-heap algorithm would be
>> >>>>>>>>>>>>>>>>>> fine. You
>> >>>>>>>>>> can
>> >>>>>>>>>>>>>>>>>> avoid using byte arrays and
>> >>>>>>>>>>>>>>>>>> minimalize GC.
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Cheers Chris
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Am 29.09.2012 15:02, schrieb
>> >>>>>>>>>>>>>>>>>> Raffaele P. Guidi:
>> >>>>>>>>>>>>>>>>>>> Nice to hear back from you!
>> >>>>>>>>>>>>>>>>>>> Yes, I was thinking about
>> >>>>>>>>>>>>>>>>>>> creating a
>> >>>>>>>>>>>>>> new
>> >>>>>>>>>>>>>>>>>> memory
>> >>>>>>>>>>>>>>>>>>> storage implementation using
>> >>>>>>>>>>>>>>>>>>> Unsafe (and I did it,
>> >>>>>>>>>>>>>>>>>>> recently) and
>> >>>>>>>>>>>>>>>> also, as
>> >>>>>>>>>>>>>>>>>>> DirectMemory relies heavily
>> >>>>>>>>>>>>>>>>>>> on serialization (and
>> >>>>>>>>>>>>>>>>>>> supports many
>> >>>>>>>>>> of
>> >>>>>>>>>>>>>>>> them,
>> >>>>>>>>>>>>>>>>>>> protostuff, protobuf, msgpack
>> >>>>>>>>>>>>>>>>>>> and of course standard
>> >>>>>>>>>>>>>> serialization),
>> >>>>>>>>>>>>>>>>>> about
>> >>>>>>>>>>>>>>>>>>> creating a simple embedded
>> >>>>>>>>>>>>>>>>>>> serializer leveraging the
>> >>>>>>>>>>>>>>>>>>> same
>> >>>>>>>>>>>>>> techniques
>> >>>>>>>>>>>>>>>> you
>> >>>>>>>>>>>>>>>>>>> used (Unsafe and bytecode
>> >>>>>>>>>>>>>>>>>>> generation). The idea with
>> >>>>>>>>>>>>>>>>>>> embedding is avoiding to
>> >>>>>>>>>>>>>>>>>>> serialize in a byte array
>> >>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>> then
>> >>>>>>>>>>>>>>>>>>> moving the byte array to
>> >>>>>>>>>>>>>>>>>>> off-heap memory (via Unsafe
>> >>>>>>>>>>>>>>>>>>> or
>> >>>>>>>>>>>>>> ByteBuffers),
>> >>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>> serializing directly into a
>> >>>>>>>>>>>>>>>>>>> "managed" off-heap buffer,
>> >>>>>>>>>>>>>>>>>>> thus
>> >>>>>>>>>> further
>> >>>>>>>>>>>>>>>>>>> optimizing heap utilization
>> >>>>>>>>>>>>>>>>>>> (less GC).
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> What do you think about it?
>> >>>>>>>>>>>>>>>>>>> Does it make any sense to
>> >>>>>>>>>>>>>>>>>>> you?
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Ciao, R
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> On Sat, Sep 29, 2012 at 2:40
>> >>>>>>>>>>>>>>>>>>> PM, Noctarius
>> >>>>>>>>>>>>>>>>>>> <[email protected]>
>> >>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> Hey guys,
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> Raffaele found out about a
>> >>>>>>>>>>>>>>>>>>>> project of mine (Lightning)
>> >>>>>>>>>>>>>>>>>>>> a few
>> >>>>>>>>>> weeks
>> >>>>>>>>>>>>>>>>>>>> ago. Lightning is a heavy
>> >>>>>>>>>>>>>>>>>>>> Unsafe and Bytecode
>> >>>>>>>>>>>>>>>>>>>> generation using
>> >>>>>>>>>>>>>>>>>>>> Serializer implementation.
>> >>>>>>>>>>>>>>>>>>>> He told me that he was
>> >>>>>>>>>>>>>>>>>>>> interested in adding
>> >>>>>>>>>>>>>>>>>>>> something similar to
>> >>>>>>>>>>>>>>>>>>>> DirectMemory and I would be
>> >>>>>>>>>>>>>>>>>>>> glad to
>> >>>>>>>>>>>>>> help
>> >>>>>>>>>>>>>>>>>>>> out in this.
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> Another project I started a
>> >>>>>>>>>>>>>>>>>>>> few days ago, since it was
>> >>>>>>>>>>>>>>>>>>>> needed
>> >>>>>>>>>> for
>> >>>>>>>>>>>>>>>>>>>> work is DirectRingCache.
>> >>>>>>>>>>>>>>>>>>>> The name not really meets
>> >>>>>>>>>>>>>>>>>>>> to actual implementation
>> >>>>>>>>>>>>>>>>>>>> since it's not yet a ring
>> >>>>>>>>>>>>>>>>>>>> buffer using cache. I
>> >>>>>>>>>>>>>> used
>> >>>>>>>>>>>>>>>>>>>> this for a
>> >>>>>>>>>>>>>>>>>>>> pre-serialization simple
>> >>>>>>>>>>>>>>>>>>>> bytestream cache with
>> >>>>>>>>>>>>>>>>>>>> self-growing buffers. It
>> >>>>>>>>>>>>>>>>>>>> could be nice to have
>> >>>>>>>>>>>>>>>>>>>> DirectMemory
>> >>>>>>>>>> having
>> >>>>>>>>>>>>>>>>>>>> raw "buffers" to write to
>> >>>>>>>>>>>>>>>>>>>> or to read from.
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> Here are the links from
>> >>>>>>>>>>>>>>>>>>>> the projects:
>> >>>>>>>>>>>>>>>>>>>> https://github.com/noctarius/Lightning
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> > https://github.com/noctarius/direct-ring-cache
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>> It would be nice to help out since I really like
>> >>>>>>>> the idea of
>> >>>>>>>>>>>>>>>>>>>> DirectMemory and since
>> >>>>>>>>>>>>>>>>>>>> direct-ring-cache is some
>> >>>>>>>>>>>>>>>>>>>> kind of
>> >>>>>>>>>>>>>> reinventing
>> >>>>>>>>>>>>>>>>>>>> the wheel.
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> Cheers Noctarius (Chris)
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> -- Olivier Lamy Talend:
>> >>>>>>>>>>>>>>>> http://coders.talend.com
>> >>>>>>>>>>>>>>>> http://twitter.com/olamy |
>> >>>>>>>>>>>>>>>> http://linkedin.com/in/olamy
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> -- Olivier Lamy Talend:
>> >>>>>>>>>>>>>> http://coders.talend.com
>> >>>>>>>>>>>>>> http://twitter.com/olamy |
>> >>>>>>>>>>>>>> http://linkedin.com/in/olamy
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> -- Olivier Lamy Talend:
>> >>>>>>>>>> http://coders.talend.com
>> >>>>>>>>>> http://twitter.com/olamy |
>> >>>>>>>>>> http://linkedin.com/in/olamy
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> -- Olivier Lamy Talend: http://coders.talend.com
>> >>>>> http://twitter.com/olamy | http://linkedin.com/in/olamy
>> >>>>>
>> >>>>
>> >>>
>> >>
>> >
>>

Reply via email to