2012/9/29 Noctarius <[email protected]>: > Nope my real name is Christoph Engelbert, but Noctarius is the all > time nick :) > > Renaming the package should be no problem, is it > "org.apache.directmemory.lightning" or what would it be? fine for me > Then there needs to be a change in the license header as Olivier > mentioned, that means just remove the first sentence or is there > anything more to do (maybe it's easiest thing to just copy the > header from DM file ;))? yup use same header as DM > > The CLA is just a form to clarify that the source can be > contributed to the Apache Foundation? yup correct. > > The final step will be attaching the patch in form of a huge diff > file? yes > > And what is the way to apply for a membership? Never thought about > how to do that. Read here http://apache.org/foundation/how-it-works.html and here http://apache.org/foundation/getinvolved.html . And feel free to ask any questions :-) > > Chris > > Am 29.09.2012 18:23, schrieb Raffaele P. Guidi: >> OK, deal, at least for me ;-) I propose you rename the >> packages, produce a patch for this and the new serializer >> module (should be simple enough starting from an existing one) >> and, in the meanwhile, apply for ASF membership. Is IP >> clearance needed? I guess yes. After this we will come up with >> a formal vote regarding Noctarius (is this your real name?!) >> allowance in the project team. >> >> Good times are gonna come :-) >> >> Thanks, R Il giorno 29/set/2012 17:58, "Olivier Lamy" >> <[email protected]> ha scritto: >> >>> 2012/9/29 Raffaele P. Guidi <[email protected]>: >>>> Well we already have a NIO ready interface allowing direct >>>> access to DMs managed bytebuffers but I think this is just >>>> half way to what could be achieved optimally blending >>>> serialization and memory allocation together. >>>> >>>> Lightning as a module is of course a good idea and it could >>>> easily evolve as a subproject (for the more experienced asf >>>> members: is it a feasible way?). >>> Nothing prevent to have >>> http://svn.apache.org/repos/asf/directmemory/lightning/trunk >>> with a package like: o.a.d.lightning That will be a >>> subproject. >>>> >>>> Ciao, R Il giorno 29/set/2012 17:44, "Noctarius" >>>> <[email protected]> ha scritto: >>>> >>>>> Hey guys, >>>>> >>>>> ok there's no lightning binary available since lightning >>>>> wasn't ready yet for releasing. >>>>> >>>>> For being the only developer it would be no problem to >>>>> contribute the sourcebase for DirectMemory but I'm not >>>>> sure yet if it wouldn't be better to seperate it to be >>>>> available without using DirectMemory itself. I started it >>>>> as a serializer for cluster synchronization, but it would >>>>> be cool to contribute lightning as a subproject to >>>>> DirectMemory :-) >>>>> >>>>> About the second project I would love to see a public >>>>> available buffer API directly in DirectMemory so that >>>>> project would be nearly needless :-) The only difference >>>>> I think is the allocation strategy my implementation is >>>>> using against the one DirectMemory has, but I'm pretty >>>>> sure the allocation is extensible ;-) >>>>> >>>>> Like I said, for both projects I'm the only dev so there >>>>> would be no IP problem. So if it's ok to you to not >>>>> include lightning directly in DM I would be glad to >>>>> contribute to the Apache Foundation :-) >>>>> >>>>> Cheers Chris >>>>> >>>>> >>>>> Am 29.09.2012 16:53, schrieb Raffaele P. Guidi: >>>>>> Ok so it's up to noctarius - your move! ;-) Regarding >>>>>> the new unsafe storage: it's an opt-in feature that can >>>>>> be set with the fluent API >>> (and >>>>>> soon through the conference file). >>>>>> >>>>>> Ciao, R Il giorno 29/set/2012 16:45, "Olivier Lamy" >>>>>> <[email protected]> ha >>>>> scritto: >>>>>> >>>>>>> 2012/9/29 Raffaele P. Guidi >>>>>>> <[email protected]>: >>>>>>>>>> At least for the moment he can provide a patch >>>>>>>>>> to be integrated >>> in DM >>>>>>> :-) >>>>>>>> >>>>>>>> Sure, but as lightning is not in any public mvn >>>>>>>> repo should its >>> code be >>>>>>>> re-published in our svn? Or what? >>>>>>> @Apache we don't care about binaries, only sources >>>>>>> are important ! (a bit theorical for sure but that's >>>>>>> it :-) ). So if Noctarius was the only guy who >>>>>>> participate in lightning. He can just provide a patch >>>>>>> we could integrate as a new dm module (note: the >>>>>>> patch must not contains any more copyright and all >>>>>>> sources must have ASF licenses). >>>>>>> >>>>>>> "Copyright 2012 the original author or authors." must >>>>>>> be removed. And BTW package must be changed :-) >>>>>>> (com.github is not acceptable >>> @asf >>>>>>> :-) )(@Noctarius are you working for github ? :-) ) >>>>>>> >>>>>>> And having him as a committer will be only a matter >>>>>>> of voting (we >>> have >>>>>>> a great chair who take cares of administrative stuff >>>>>>> :P ) >>>>>>> >>>>>>> If some others have participated in the project, we >>>>>>> must pass tru an ip clearance mechanism >>>>>>> (http://incubator.apache.org/ip-clearance/index.html) >>>>>>> and all contributors to lightning must provide a cla. >>>>>>> (It it's the case I can help) >>>>>>> >>>>>>>> >>>>>>>>>> perso I'd like we avoid hard dependency on >>>>>>>>>> Unsafe as maybe some >>> use >>>>>>>> other jdks :-) >>>>>>>> >>>>>>>> Well, I believe Unsafe is supported by Sun JDK, >>>>>>>> OpenJDK, IBM JDK and JRockit - and I believe that >>>>>>>> it is more than enough. Also keep in >>> mind >>>>>>> that >>>>>>>> we already have an alternative Unsafe based memory >>>>>>>> storage - and >>>>> although >>>>>>>> it's not thoroughly tested for performance it >>>>>>>> dramaticaly simplifies >>>>>>> code, >>>>>>>> I have great expectations about it. >>>>>>> Me too :-). Yup definitely more simple and faster ! >>>>>>> But we must provide a switch off configuration >>>>>>> mechanism if some >>> users >>>>>>> don't want to use that (because Unsafe is just >>>>>>> "Unsafe" :-) ) And sorry I didn't have a look yet at >>>>>>> your changes with using Unsafe. >>>>>>>> >>>>>>>> Cheers, R >>>>>>>> >>>>>>>> On Sat, Sep 29, 2012 at 4:03 PM, Olivier Lamy >>>>>>>> <[email protected]> >>>>> wrote: >>>>>>>> >>>>>>>>> 2012/9/29 Raffaele P. Guidi >>>>>>>>> <[email protected]>: >>>>>>>>>> What do you think about: 1) implementing a >>>>>>>>>> lightning serialization module 2) creating a >>>>>>>>>> serializer that directly works on a >>>>>>>>>> directmemory >>>>>>> provider >>>>>>>>>> ByteBuffer or (maybe better) Unsafe based >>>>>>>>>> Pointer? >>>>>>>>> >>>>>>>>> Sounds good (perso I'd like we avoid hard >>>>>>>>> dependency on Unsafe as maybe some use other jdks >>>>>>>>> :-) ) >>>>>>>>>> >>>>>>>>>> Now I see lightning is apache licensed and this >>>>>>>>>> is fine but I >>> don't >>>>>>> think >>>>>>>>>> it is published in any public maven repo, am I >>>>>>>>>> right? We could >>> find a >>>>>>> way >>>>>>>>>> to deal with this; options vary from publishing >>>>>>>>>> lightning to the >>> free >>>>>>>>>> sonatype repo, joining the ASF (which is great >>>>>>>>>> anyhow!) and >>>>>>> republishing >>>>>>>>>> lightning code in DirectMemory becoming a >>>>>>>>>> commiter (which has to >>>>>>> undergo >>>>>>>>> a >>>>>>>>>> PMC vote). >>>>>>>>> >>>>>>>>> At least for the moment he can provide a patch to >>>>>>>>> be integrated in >>> DM >>>>>>> :-) >>>>>>>>> >>>>>>>>>> >>>>>>>>>> I'd like to hear your and the team feelings on >>>>>>>>>> this. >>>>>>>>>> >>>>>>>>>> Thanks, Raffaele >>>>>>>>>> >>>>>>>>>> On Sat, Sep 29, 2012 at 3:27 PM, Noctarius >>>>>>>>>> <[email protected]> >>> wrote: >>>>>>>>>> >>>>>>>>>>> Hey Raffaele, >>>>>>>>>>> >>>>>>>>>>> that's quite similar to what I did at work. >>>>>>>>>>> We're developing >>> Flash >>>>>>>>>>> online games and using a customized AMF >>>>>>>>>>> serialization. To support async writing of >>>>>>>>>>> the clients event results I added >>>>>>>>>>> serialization >>> of >>>>>>>>>>> the components / entities to the players zone >>>>>>>>>>> calculation and >>> stored >>>>>>>>>>> the pre-serialized bytestream directly to the >>>>>>>>>>> off-heap (using direct-ring-cache >>>>>>>>>>> implementation). When the client requests >>>>>>>>>>> the results (using long-polling) I just write >>>>>>>>>>> the pre-serialized >>> data to >>>>>>>>>>> the right position to deserialize it by >>>>>>>>>>> standard ways on Flash >>> side. >>>>>>>>>>> >>>>>>>>>>> So yeah an seriliaztion to off-heap algorithm >>>>>>>>>>> would be fine. You >>> can >>>>>>>>>>> avoid using byte arrays and minimalize GC. >>>>>>>>>>> >>>>>>>>>>> Cheers Chris >>>>>>>>>>> >>>>>>>>>>> Am 29.09.2012 15:02, schrieb Raffaele P. >>>>>>>>>>> Guidi: >>>>>>>>>>>> Nice to hear back from you! Yes, I was >>>>>>>>>>>> thinking about creating a >>>>>>> new >>>>>>>>>>> memory >>>>>>>>>>>> storage implementation using Unsafe (and I >>>>>>>>>>>> did it, recently) and >>>>>>>>> also, as >>>>>>>>>>>> DirectMemory relies heavily on >>>>>>>>>>>> serialization (and supports many >>> of >>>>>>>>> them, >>>>>>>>>>>> protostuff, protobuf, msgpack and of course >>>>>>>>>>>> standard >>>>>>> serialization), >>>>>>>>>>> about >>>>>>>>>>>> creating a simple embedded serializer >>>>>>>>>>>> leveraging the same >>>>>>> techniques >>>>>>>>> you >>>>>>>>>>>> used (Unsafe and bytecode generation). The >>>>>>>>>>>> idea with embedding is avoiding to >>>>>>>>>>>> serialize in a byte array >>>>>>> and >>>>>>>>> then >>>>>>>>>>>> moving the byte array to off-heap memory >>>>>>>>>>>> (via Unsafe or >>>>>>> ByteBuffers), >>>>>>>>> and >>>>>>>>>>>> serializing directly into a "managed" >>>>>>>>>>>> off-heap buffer, thus >>> further >>>>>>>>>>>> optimizing heap utilization (less GC). >>>>>>>>>>>> >>>>>>>>>>>> What do you think about it? Does it make >>>>>>>>>>>> any sense to you? >>>>>>>>>>>> >>>>>>>>>>>> Ciao, R >>>>>>>>>>>> >>>>>>>>>>>> On Sat, Sep 29, 2012 at 2:40 PM, Noctarius >>>>>>>>>>>> <[email protected]> >>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hey guys, >>>>>>>>>>>>> >>>>>>>>>>>>> Raffaele found out about a project of >>>>>>>>>>>>> mine (Lightning) a few >>> weeks >>>>>>>>>>>>> ago. Lightning is a heavy Unsafe and >>>>>>>>>>>>> Bytecode generation using Serializer >>>>>>>>>>>>> implementation. He told me that he was >>>>>>>>>>>>> interested in adding something similar to >>>>>>>>>>>>> DirectMemory and I would be glad to >>>>>>> help >>>>>>>>>>>>> out in this. >>>>>>>>>>>>> >>>>>>>>>>>>> Another project I started a few days ago, >>>>>>>>>>>>> since it was needed >>> for >>>>>>>>>>>>> work is DirectRingCache. The name not >>>>>>>>>>>>> really meets to actual implementation >>>>>>>>>>>>> since it's not yet a ring buffer using >>>>>>>>>>>>> cache. I >>>>>>> used >>>>>>>>>>>>> this for a pre-serialization simple >>>>>>>>>>>>> bytestream cache with self-growing >>>>>>>>>>>>> buffers. It could be nice to have >>>>>>>>>>>>> DirectMemory >>> having >>>>>>>>>>>>> raw "buffers" to write to or to read >>>>>>>>>>>>> from. >>>>>>>>>>>>> >>>>>>>>>>>>> Here are the links from the projects: >>>>>>>>>>>>> https://github.com/noctarius/Lightning >>>>>>>>>>>>> https://github.com/noctarius/direct-ring-cache >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> > It would be nice to help out since I really like the idea of >>>>>>>>>>>>> DirectMemory and since direct-ring-cache >>>>>>>>>>>>> is some kind of >>>>>>> reinventing >>>>>>>>>>>>> the wheel. >>>>>>>>>>>>> >>>>>>>>>>>>> Cheers Noctarius (Chris) >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- Olivier Lamy Talend: http://coders.talend.com >>>>>>>>> http://twitter.com/olamy | >>>>>>>>> http://linkedin.com/in/olamy >>>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- Olivier Lamy Talend: http://coders.talend.com >>>>>>> http://twitter.com/olamy | >>>>>>> http://linkedin.com/in/olamy >>>>>>> >>>>>> >>>>> >>> >>> >>> >>> -- Olivier Lamy Talend: http://coders.talend.com >>> http://twitter.com/olamy | http://linkedin.com/in/olamy >>> >>
-- Olivier Lamy Talend: http://coders.talend.com http://twitter.com/olamy | http://linkedin.com/in/olamy
