>> Hehe well that really sounds like a nice bunch of people. Indeed they are (I'm a newbie as well and try to do my best)
>> If lightning will be a sub-part (sub-project) of DM, do I need to write an project purposal? Nope, not needed for a sub-project >> Do I need to make any changes to the pom.xml like adding a special parent pom or anything like that? Not for the serializer - just have to take a look at project dependencies - or, better, at their licenses - are they compatible with the ASL 2.0? i.e. a GPL'd library is not a good fit and should be replaced with an apache licensed (or BSD, or MIT...) one if possible. For the integration module is a separate story - you should start off copying one of the other serializers and reusing the same pom and directory structure. Pleased to meet you, Chris :) Ciao, R On Sat, Sep 29, 2012 at 8:24 PM, Noctarius <[email protected]> wrote: > Hehe well that really sounds like a nice bunch of people. > > Ok to be true I couldn't wait until tomorrow and started already > reading the links. > From what I was reading: > If lightning will be a sub-part (sub-project) of DM, do I need to > write an project purposal? > > Do I need to make any changes to the pom.xml like adding a special > parent pom or anything like that? > > In general: there are a lot things to know :-) > > Am 29.09.2012 19:59, schrieb Raffaele P. Guidi: > > Negative part of ASF membership? You get together with a lot of geeky, > > talented people with a fixation for software and open source. Oh wait but > > this is actually nice! :-D > > Il giorno 29/set/2012 19:05, "Olivier Lamy" <[email protected]> ha > scritto: > > > >> 2012/9/29 Noctarius <[email protected]>: > >>> Thanks Olivier for carify, I'll take a look in it tomorrow but > >>> there's just one question left (for now ;)): > >>> What is that vote for becoming a committer? What if the vote will > >>> be negative? > >> The vote is on private list (pmc list for privacy reasons and possible > >> negative stuff being on public lists) > >>> Until now I just used Apache stuff, was never interested in being > >>> part of it so I guess it can be negative for any reason, can't it? > >> I don't see why it could be negative but suspens .... :-) > >>> > >>> Am 29.09.2012 18:56, schrieb Olivier Lamy: > >>>> 2012/9/29 Noctarius <[email protected]>: > >>>>> Nope my real name is Christoph Engelbert, but Noctarius is > >>>>> the all time nick :) > >>>>> > >>>>> Renaming the package should be no problem, is it > >>>>> "org.apache.directmemory.lightning" or what would it be? > >>>> fine for me > >>>>> Then there needs to be a change in the license header as > >>>>> Olivier mentioned, that means just remove the first sentence > >>>>> or is there anything more to do (maybe it's easiest thing to > >>>>> just copy the header from DM file ;))? > >>>> yup use same header as DM > >>>>> > >>>>> The CLA is just a form to clarify that the source can be > >>>>> contributed to the Apache Foundation? > >>>> yup correct. > >>>>> > >>>>> The final step will be attaching the patch in form of a huge > >>>>> diff file? > >>>> yes > >>>>> > >>>>> And what is the way to apply for a membership? Never thought > >>>>> about how to do that. > >>>> Read here http://apache.org/foundation/how-it-works.html and > >>>> here http://apache.org/foundation/getinvolved.html . And feel > >>>> free to ask any questions :-) > >>>>> > >>>>> Chris > >>>>> > >>>>> Am 29.09.2012 18:23, schrieb Raffaele P. Guidi: > >>>>>> OK, deal, at least for me ;-) I propose you rename the > >>>>>> packages, produce a patch for this and the new serializer > >>>>>> module (should be simple enough starting from an existing > >>>>>> one) and, in the meanwhile, apply for ASF membership. Is > >>>>>> IP clearance needed? I guess yes. After this we will come > >>>>>> up with a formal vote regarding Noctarius (is this your > >>>>>> real name?!) allowance in the project team. > >>>>>> > >>>>>> Good times are gonna come :-) > >>>>>> > >>>>>> Thanks, R Il giorno 29/set/2012 17:58, "Olivier Lamy" > >>>>>> <[email protected]> ha scritto: > >>>>>> > >>>>>>> 2012/9/29 Raffaele P. Guidi > >>>>>>> <[email protected]>: > >>>>>>>> Well we already have a NIO ready interface allowing > >>>>>>>> direct access to DMs managed bytebuffers but I think > >>>>>>>> this is just half way to what could be achieved > >>>>>>>> optimally blending serialization and memory allocation > >>>>>>>> together. > >>>>>>>> > >>>>>>>> Lightning as a module is of course a good idea and it > >>>>>>>> could easily evolve as a subproject (for the more > >>>>>>>> experienced asf members: is it a feasible way?). > >>>>>>> Nothing prevent to have > >>>>>>> http://svn.apache.org/repos/asf/directmemory/lightning/trunk > >>>>>>> > >>>>>>> > >>> with a package like: o.a.d.lightning That will be a > >>>>>>> subproject. > >>>>>>>> > >>>>>>>> Ciao, R Il giorno 29/set/2012 17:44, "Noctarius" > >>>>>>>> <[email protected]> ha scritto: > >>>>>>>> > >>>>>>>>> Hey guys, > >>>>>>>>> > >>>>>>>>> ok there's no lightning binary available since > >>>>>>>>> lightning wasn't ready yet for releasing. > >>>>>>>>> > >>>>>>>>> For being the only developer it would be no problem > >>>>>>>>> to contribute the sourcebase for DirectMemory but I'm > >>>>>>>>> not sure yet if it wouldn't be better to seperate it > >>>>>>>>> to be available without using DirectMemory itself. I > >>>>>>>>> started it as a serializer for cluster > >>>>>>>>> synchronization, but it would be cool to contribute > >>>>>>>>> lightning as a subproject to DirectMemory :-) > >>>>>>>>> > >>>>>>>>> About the second project I would love to see a > >>>>>>>>> public available buffer API directly in DirectMemory > >>>>>>>>> so that project would be nearly needless :-) The only > >>>>>>>>> difference I think is the allocation strategy my > >>>>>>>>> implementation is using against the one DirectMemory > >>>>>>>>> has, but I'm pretty sure the allocation is extensible > >>>>>>>>> ;-) > >>>>>>>>> > >>>>>>>>> Like I said, for both projects I'm the only dev so > >>>>>>>>> there would be no IP problem. So if it's ok to you to > >>>>>>>>> not include lightning directly in DM I would be glad > >>>>>>>>> to contribute to the Apache Foundation :-) > >>>>>>>>> > >>>>>>>>> Cheers Chris > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> Am 29.09.2012 16:53, schrieb Raffaele P. Guidi: > >>>>>>>>>> Ok so it's up to noctarius - your move! ;-) > >>>>>>>>>> Regarding the new unsafe storage: it's an opt-in > >>>>>>>>>> feature that can be set with the fluent API > >>>>>>> (and > >>>>>>>>>> soon through the conference file). > >>>>>>>>>> > >>>>>>>>>> Ciao, R Il giorno 29/set/2012 16:45, "Olivier > >>>>>>>>>> Lamy" <[email protected]> ha > >>>>>>>>> scritto: > >>>>>>>>>> > >>>>>>>>>>> 2012/9/29 Raffaele P. Guidi > >>>>>>>>>>> <[email protected]>: > >>>>>>>>>>>>>> At least for the moment he can provide a > >>>>>>>>>>>>>> patch to be integrated > >>>>>>> in DM > >>>>>>>>>>> :-) > >>>>>>>>>>>> > >>>>>>>>>>>> Sure, but as lightning is not in any public > >>>>>>>>>>>> mvn repo should its > >>>>>>> code be > >>>>>>>>>>>> re-published in our svn? Or what? > >>>>>>>>>>> @Apache we don't care about binaries, only > >>>>>>>>>>> sources are important ! (a bit theorical for sure > >>>>>>>>>>> but that's it :-) ). So if Noctarius was the only > >>>>>>>>>>> guy who participate in lightning. He can just > >>>>>>>>>>> provide a patch we could integrate as a new dm > >>>>>>>>>>> module (note: the patch must not contains any > >>>>>>>>>>> more copyright and all sources must have ASF > >>>>>>>>>>> licenses). > >>>>>>>>>>> > >>>>>>>>>>> "Copyright 2012 the original author or authors." > >>>>>>>>>>> must be removed. And BTW package must be changed > >>>>>>>>>>> :-) (com.github is not acceptable > >>>>>>> @asf > >>>>>>>>>>> :-) )(@Noctarius are you working for github ? :-) > >>>>>>>>>>> ) > >>>>>>>>>>> > >>>>>>>>>>> And having him as a committer will be only a > >>>>>>>>>>> matter of voting (we > >>>>>>> have > >>>>>>>>>>> a great chair who take cares of administrative > >>>>>>>>>>> stuff :P ) > >>>>>>>>>>> > >>>>>>>>>>> If some others have participated in the project, > >>>>>>>>>>> we must pass tru an ip clearance mechanism > >>>>>>>>>>> (http://incubator.apache.org/ip-clearance/index.html) > >>>>>>>>>>> > >>>>>>>>>>> > >>> and all contributors to lightning must provide a cla. > >>>>>>>>>>> (It it's the case I can help) > >>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>>>> perso I'd like we avoid hard dependency on > >>>>>>>>>>>>>> Unsafe as maybe some > >>>>>>> use > >>>>>>>>>>>> other jdks :-) > >>>>>>>>>>>> > >>>>>>>>>>>> Well, I believe Unsafe is supported by Sun > >>>>>>>>>>>> JDK, OpenJDK, IBM JDK and JRockit - and I > >>>>>>>>>>>> believe that it is more than enough. Also keep > >>>>>>>>>>>> in > >>>>>>> mind > >>>>>>>>>>> that > >>>>>>>>>>>> we already have an alternative Unsafe based > >>>>>>>>>>>> memory storage - and > >>>>>>>>> although > >>>>>>>>>>>> it's not thoroughly tested for performance it > >>>>>>>>>>>> dramaticaly simplifies > >>>>>>>>>>> code, > >>>>>>>>>>>> I have great expectations about it. > >>>>>>>>>>> Me too :-). Yup definitely more simple and faster > >>>>>>>>>>> ! But we must provide a switch off configuration > >>>>>>>>>>> mechanism if some > >>>>>>> users > >>>>>>>>>>> don't want to use that (because Unsafe is just > >>>>>>>>>>> "Unsafe" :-) ) And sorry I didn't have a look yet > >>>>>>>>>>> at your changes with using Unsafe. > >>>>>>>>>>>> > >>>>>>>>>>>> Cheers, R > >>>>>>>>>>>> > >>>>>>>>>>>> On Sat, Sep 29, 2012 at 4:03 PM, Olivier Lamy > >>>>>>>>>>>> <[email protected]> > >>>>>>>>> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>>> 2012/9/29 Raffaele P. Guidi > >>>>>>>>>>>>> <[email protected]>: > >>>>>>>>>>>>>> What do you think about: 1) implementing a > >>>>>>>>>>>>>> lightning serialization module 2) creating > >>>>>>>>>>>>>> a serializer that directly works on a > >>>>>>>>>>>>>> directmemory > >>>>>>>>>>> provider > >>>>>>>>>>>>>> ByteBuffer or (maybe better) Unsafe based > >>>>>>>>>>>>>> Pointer? > >>>>>>>>>>>>> > >>>>>>>>>>>>> Sounds good (perso I'd like we avoid hard > >>>>>>>>>>>>> dependency on Unsafe as maybe some use other > >>>>>>>>>>>>> jdks :-) ) > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Now I see lightning is apache licensed and > >>>>>>>>>>>>>> this is fine but I > >>>>>>> don't > >>>>>>>>>>> think > >>>>>>>>>>>>>> it is published in any public maven repo, > >>>>>>>>>>>>>> am I right? We could > >>>>>>> find a > >>>>>>>>>>> way > >>>>>>>>>>>>>> to deal with this; options vary from > >>>>>>>>>>>>>> publishing lightning to the > >>>>>>> free > >>>>>>>>>>>>>> sonatype repo, joining the ASF (which is > >>>>>>>>>>>>>> great anyhow!) and > >>>>>>>>>>> republishing > >>>>>>>>>>>>>> lightning code in DirectMemory becoming a > >>>>>>>>>>>>>> commiter (which has to > >>>>>>>>>>> undergo > >>>>>>>>>>>>> a > >>>>>>>>>>>>>> PMC vote). > >>>>>>>>>>>>> > >>>>>>>>>>>>> At least for the moment he can provide a > >>>>>>>>>>>>> patch to be integrated in > >>>>>>> DM > >>>>>>>>>>> :-) > >>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> I'd like to hear your and the team feelings > >>>>>>>>>>>>>> on this. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Thanks, Raffaele > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> On Sat, Sep 29, 2012 at 3:27 PM, Noctarius > >>>>>>>>>>>>>> <[email protected]> > >>>>>>> wrote: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Hey Raffaele, > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> that's quite similar to what I did at > >>>>>>>>>>>>>>> work. We're developing > >>>>>>> Flash > >>>>>>>>>>>>>>> online games and using a customized AMF > >>>>>>>>>>>>>>> serialization. To support async writing > >>>>>>>>>>>>>>> of the clients event results I added > >>>>>>>>>>>>>>> serialization > >>>>>>> of > >>>>>>>>>>>>>>> the components / entities to the players > >>>>>>>>>>>>>>> zone calculation and > >>>>>>> stored > >>>>>>>>>>>>>>> the pre-serialized bytestream directly to > >>>>>>>>>>>>>>> the off-heap (using direct-ring-cache > >>>>>>>>>>>>>>> implementation). When the client > >>>>>>>>>>>>>>> requests the results (using long-polling) > >>>>>>>>>>>>>>> I just write the pre-serialized > >>>>>>> data to > >>>>>>>>>>>>>>> the right position to deserialize it by > >>>>>>>>>>>>>>> standard ways on Flash > >>>>>>> side. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> So yeah an seriliaztion to off-heap > >>>>>>>>>>>>>>> algorithm would be fine. You > >>>>>>> can > >>>>>>>>>>>>>>> avoid using byte arrays and minimalize > >>>>>>>>>>>>>>> GC. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Cheers Chris > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Am 29.09.2012 15:02, schrieb Raffaele P. > >>>>>>>>>>>>>>> Guidi: > >>>>>>>>>>>>>>>> Nice to hear back from you! Yes, I was > >>>>>>>>>>>>>>>> thinking about creating a > >>>>>>>>>>> new > >>>>>>>>>>>>>>> memory > >>>>>>>>>>>>>>>> storage implementation using Unsafe > >>>>>>>>>>>>>>>> (and I did it, recently) and > >>>>>>>>>>>>> also, as > >>>>>>>>>>>>>>>> DirectMemory relies heavily on > >>>>>>>>>>>>>>>> serialization (and supports many > >>>>>>> of > >>>>>>>>>>>>> them, > >>>>>>>>>>>>>>>> protostuff, protobuf, msgpack and of > >>>>>>>>>>>>>>>> course standard > >>>>>>>>>>> serialization), > >>>>>>>>>>>>>>> about > >>>>>>>>>>>>>>>> creating a simple embedded serializer > >>>>>>>>>>>>>>>> leveraging the same > >>>>>>>>>>> techniques > >>>>>>>>>>>>> you > >>>>>>>>>>>>>>>> used (Unsafe and bytecode generation). > >>>>>>>>>>>>>>>> The idea with embedding is avoiding to > >>>>>>>>>>>>>>>> serialize in a byte array > >>>>>>>>>>> and > >>>>>>>>>>>>> then > >>>>>>>>>>>>>>>> moving the byte array to off-heap > >>>>>>>>>>>>>>>> memory (via Unsafe or > >>>>>>>>>>> ByteBuffers), > >>>>>>>>>>>>> and > >>>>>>>>>>>>>>>> serializing directly into a "managed" > >>>>>>>>>>>>>>>> off-heap buffer, thus > >>>>>>> further > >>>>>>>>>>>>>>>> optimizing heap utilization (less GC). > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> What do you think about it? Does it > >>>>>>>>>>>>>>>> make any sense to you? > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Ciao, R > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> On Sat, Sep 29, 2012 at 2:40 PM, > >>>>>>>>>>>>>>>> Noctarius <[email protected]> > >>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Hey guys, > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Raffaele found out about a project > >>>>>>>>>>>>>>>>> of mine (Lightning) a few > >>>>>>> weeks > >>>>>>>>>>>>>>>>> ago. Lightning is a heavy Unsafe and > >>>>>>>>>>>>>>>>> Bytecode generation using Serializer > >>>>>>>>>>>>>>>>> implementation. He told me that he > >>>>>>>>>>>>>>>>> was interested in adding something > >>>>>>>>>>>>>>>>> similar to DirectMemory and I would > >>>>>>>>>>>>>>>>> be glad to > >>>>>>>>>>> help > >>>>>>>>>>>>>>>>> out in this. > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Another project I started a few days > >>>>>>>>>>>>>>>>> ago, since it was needed > >>>>>>> for > >>>>>>>>>>>>>>>>> work is DirectRingCache. The name > >>>>>>>>>>>>>>>>> not really meets to actual > >>>>>>>>>>>>>>>>> implementation since it's not yet a > >>>>>>>>>>>>>>>>> ring buffer using cache. I > >>>>>>>>>>> used > >>>>>>>>>>>>>>>>> this for a pre-serialization simple > >>>>>>>>>>>>>>>>> bytestream cache with self-growing > >>>>>>>>>>>>>>>>> buffers. It could be nice to have > >>>>>>>>>>>>>>>>> DirectMemory > >>>>>>> having > >>>>>>>>>>>>>>>>> raw "buffers" to write to or to read > >>>>>>>>>>>>>>>>> from. > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Here are the links from the > >>>>>>>>>>>>>>>>> projects: > >>>>>>>>>>>>>>>>> https://github.com/noctarius/Lightning > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>> https://github.com/noctarius/direct-ring-cache > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>> It would be nice to help out since I really like the idea of > >>>>>>>>>>>>>>>>> DirectMemory and since > >>>>>>>>>>>>>>>>> direct-ring-cache is some kind of > >>>>>>>>>>> reinventing > >>>>>>>>>>>>>>>>> the wheel. > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Cheers Noctarius (Chris) > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> -- Olivier Lamy Talend: > >>>>>>>>>>>>> http://coders.talend.com > >>>>>>>>>>>>> http://twitter.com/olamy | > >>>>>>>>>>>>> http://linkedin.com/in/olamy > >>>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> -- Olivier Lamy Talend: http://coders.talend.com > >>>>>>>>>>> http://twitter.com/olamy | > >>>>>>>>>>> http://linkedin.com/in/olamy > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> -- Olivier Lamy Talend: http://coders.talend.com > >>>>>>> http://twitter.com/olamy | http://linkedin.com/in/olamy > >>>>>>> > >>>>>> > >>>> > >>>> > >>>> > >> > >> > >> > >> -- > >> Olivier Lamy > >> Talend: http://coders.talend.com > >> http://twitter.com/olamy | http://linkedin.com/in/olamy > >> > > >
