Hehe well that really sounds like a nice bunch of people. Ok to be true I couldn't wait until tomorrow and started already reading the links. >From what I was reading: If lightning will be a sub-part (sub-project) of DM, do I need to write an project purposal?
Do I need to make any changes to the pom.xml like adding a special parent pom or anything like that? In general: there are a lot things to know :-) Am 29.09.2012 19:59, schrieb Raffaele P. Guidi: > Negative part of ASF membership? You get together with a lot of geeky, > talented people with a fixation for software and open source. Oh wait but > this is actually nice! :-D > Il giorno 29/set/2012 19:05, "Olivier Lamy" <[email protected]> ha scritto: > >> 2012/9/29 Noctarius <[email protected]>: >>> Thanks Olivier for carify, I'll take a look in it tomorrow but >>> there's just one question left (for now ;)): >>> What is that vote for becoming a committer? What if the vote will >>> be negative? >> The vote is on private list (pmc list for privacy reasons and possible >> negative stuff being on public lists) >>> Until now I just used Apache stuff, was never interested in being >>> part of it so I guess it can be negative for any reason, can't it? >> I don't see why it could be negative but suspens .... :-) >>> >>> Am 29.09.2012 18:56, schrieb Olivier Lamy: >>>> 2012/9/29 Noctarius <[email protected]>: >>>>> Nope my real name is Christoph Engelbert, but Noctarius is >>>>> the all time nick :) >>>>> >>>>> Renaming the package should be no problem, is it >>>>> "org.apache.directmemory.lightning" or what would it be? >>>> fine for me >>>>> Then there needs to be a change in the license header as >>>>> Olivier mentioned, that means just remove the first sentence >>>>> or is there anything more to do (maybe it's easiest thing to >>>>> just copy the header from DM file ;))? >>>> yup use same header as DM >>>>> >>>>> The CLA is just a form to clarify that the source can be >>>>> contributed to the Apache Foundation? >>>> yup correct. >>>>> >>>>> The final step will be attaching the patch in form of a huge >>>>> diff file? >>>> yes >>>>> >>>>> And what is the way to apply for a membership? Never thought >>>>> about how to do that. >>>> Read here http://apache.org/foundation/how-it-works.html and >>>> here http://apache.org/foundation/getinvolved.html . And feel >>>> free to ask any questions :-) >>>>> >>>>> Chris >>>>> >>>>> Am 29.09.2012 18:23, schrieb Raffaele P. Guidi: >>>>>> OK, deal, at least for me ;-) I propose you rename the >>>>>> packages, produce a patch for this and the new serializer >>>>>> module (should be simple enough starting from an existing >>>>>> one) and, in the meanwhile, apply for ASF membership. Is >>>>>> IP clearance needed? I guess yes. After this we will come >>>>>> up with a formal vote regarding Noctarius (is this your >>>>>> real name?!) allowance in the project team. >>>>>> >>>>>> Good times are gonna come :-) >>>>>> >>>>>> Thanks, R Il giorno 29/set/2012 17:58, "Olivier Lamy" >>>>>> <[email protected]> ha scritto: >>>>>> >>>>>>> 2012/9/29 Raffaele P. Guidi >>>>>>> <[email protected]>: >>>>>>>> Well we already have a NIO ready interface allowing >>>>>>>> direct access to DMs managed bytebuffers but I think >>>>>>>> this is just half way to what could be achieved >>>>>>>> optimally blending serialization and memory allocation >>>>>>>> together. >>>>>>>> >>>>>>>> Lightning as a module is of course a good idea and it >>>>>>>> could easily evolve as a subproject (for the more >>>>>>>> experienced asf members: is it a feasible way?). >>>>>>> Nothing prevent to have >>>>>>> http://svn.apache.org/repos/asf/directmemory/lightning/trunk >>>>>>> >>>>>>> >>> with a package like: o.a.d.lightning That will be a >>>>>>> subproject. >>>>>>>> >>>>>>>> Ciao, R Il giorno 29/set/2012 17:44, "Noctarius" >>>>>>>> <[email protected]> ha scritto: >>>>>>>> >>>>>>>>> Hey guys, >>>>>>>>> >>>>>>>>> ok there's no lightning binary available since >>>>>>>>> lightning wasn't ready yet for releasing. >>>>>>>>> >>>>>>>>> For being the only developer it would be no problem >>>>>>>>> to contribute the sourcebase for DirectMemory but I'm >>>>>>>>> not sure yet if it wouldn't be better to seperate it >>>>>>>>> to be available without using DirectMemory itself. I >>>>>>>>> started it as a serializer for cluster >>>>>>>>> synchronization, but it would be cool to contribute >>>>>>>>> lightning as a subproject to DirectMemory :-) >>>>>>>>> >>>>>>>>> About the second project I would love to see a >>>>>>>>> public available buffer API directly in DirectMemory >>>>>>>>> so that project would be nearly needless :-) The only >>>>>>>>> difference I think is the allocation strategy my >>>>>>>>> implementation is using against the one DirectMemory >>>>>>>>> has, but I'm pretty sure the allocation is extensible >>>>>>>>> ;-) >>>>>>>>> >>>>>>>>> Like I said, for both projects I'm the only dev so >>>>>>>>> there would be no IP problem. So if it's ok to you to >>>>>>>>> not include lightning directly in DM I would be glad >>>>>>>>> to contribute to the Apache Foundation :-) >>>>>>>>> >>>>>>>>> Cheers Chris >>>>>>>>> >>>>>>>>> >>>>>>>>> Am 29.09.2012 16:53, schrieb Raffaele P. Guidi: >>>>>>>>>> Ok so it's up to noctarius - your move! ;-) >>>>>>>>>> Regarding the new unsafe storage: it's an opt-in >>>>>>>>>> feature that can be set with the fluent API >>>>>>> (and >>>>>>>>>> soon through the conference file). >>>>>>>>>> >>>>>>>>>> Ciao, R Il giorno 29/set/2012 16:45, "Olivier >>>>>>>>>> Lamy" <[email protected]> ha >>>>>>>>> scritto: >>>>>>>>>> >>>>>>>>>>> 2012/9/29 Raffaele P. Guidi >>>>>>>>>>> <[email protected]>: >>>>>>>>>>>>>> At least for the moment he can provide a >>>>>>>>>>>>>> patch to be integrated >>>>>>> in DM >>>>>>>>>>> :-) >>>>>>>>>>>> >>>>>>>>>>>> Sure, but as lightning is not in any public >>>>>>>>>>>> mvn repo should its >>>>>>> code be >>>>>>>>>>>> re-published in our svn? Or what? >>>>>>>>>>> @Apache we don't care about binaries, only >>>>>>>>>>> sources are important ! (a bit theorical for sure >>>>>>>>>>> but that's it :-) ). So if Noctarius was the only >>>>>>>>>>> guy who participate in lightning. He can just >>>>>>>>>>> provide a patch we could integrate as a new dm >>>>>>>>>>> module (note: the patch must not contains any >>>>>>>>>>> more copyright and all sources must have ASF >>>>>>>>>>> licenses). >>>>>>>>>>> >>>>>>>>>>> "Copyright 2012 the original author or authors." >>>>>>>>>>> must be removed. And BTW package must be changed >>>>>>>>>>> :-) (com.github is not acceptable >>>>>>> @asf >>>>>>>>>>> :-) )(@Noctarius are you working for github ? :-) >>>>>>>>>>> ) >>>>>>>>>>> >>>>>>>>>>> And having him as a committer will be only a >>>>>>>>>>> matter of voting (we >>>>>>> have >>>>>>>>>>> a great chair who take cares of administrative >>>>>>>>>>> stuff :P ) >>>>>>>>>>> >>>>>>>>>>> If some others have participated in the project, >>>>>>>>>>> we must pass tru an ip clearance mechanism >>>>>>>>>>> (http://incubator.apache.org/ip-clearance/index.html) >>>>>>>>>>> >>>>>>>>>>> >>> and all contributors to lightning must provide a cla. >>>>>>>>>>> (It it's the case I can help) >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>>> perso I'd like we avoid hard dependency on >>>>>>>>>>>>>> Unsafe as maybe some >>>>>>> use >>>>>>>>>>>> other jdks :-) >>>>>>>>>>>> >>>>>>>>>>>> Well, I believe Unsafe is supported by Sun >>>>>>>>>>>> JDK, OpenJDK, IBM JDK and JRockit - and I >>>>>>>>>>>> believe that it is more than enough. Also keep >>>>>>>>>>>> in >>>>>>> mind >>>>>>>>>>> that >>>>>>>>>>>> we already have an alternative Unsafe based >>>>>>>>>>>> memory storage - and >>>>>>>>> although >>>>>>>>>>>> it's not thoroughly tested for performance it >>>>>>>>>>>> dramaticaly simplifies >>>>>>>>>>> code, >>>>>>>>>>>> I have great expectations about it. >>>>>>>>>>> Me too :-). Yup definitely more simple and faster >>>>>>>>>>> ! But we must provide a switch off configuration >>>>>>>>>>> mechanism if some >>>>>>> users >>>>>>>>>>> don't want to use that (because Unsafe is just >>>>>>>>>>> "Unsafe" :-) ) And sorry I didn't have a look yet >>>>>>>>>>> at your changes with using Unsafe. >>>>>>>>>>>> >>>>>>>>>>>> Cheers, R >>>>>>>>>>>> >>>>>>>>>>>> On Sat, Sep 29, 2012 at 4:03 PM, Olivier Lamy >>>>>>>>>>>> <[email protected]> >>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> 2012/9/29 Raffaele P. Guidi >>>>>>>>>>>>> <[email protected]>: >>>>>>>>>>>>>> What do you think about: 1) implementing a >>>>>>>>>>>>>> lightning serialization module 2) creating >>>>>>>>>>>>>> a serializer that directly works on a >>>>>>>>>>>>>> directmemory >>>>>>>>>>> provider >>>>>>>>>>>>>> ByteBuffer or (maybe better) Unsafe based >>>>>>>>>>>>>> Pointer? >>>>>>>>>>>>> >>>>>>>>>>>>> Sounds good (perso I'd like we avoid hard >>>>>>>>>>>>> dependency on Unsafe as maybe some use other >>>>>>>>>>>>> jdks :-) ) >>>>>>>>>>>>>> >>>>>>>>>>>>>> Now I see lightning is apache licensed and >>>>>>>>>>>>>> this is fine but I >>>>>>> don't >>>>>>>>>>> think >>>>>>>>>>>>>> it is published in any public maven repo, >>>>>>>>>>>>>> am I right? We could >>>>>>> find a >>>>>>>>>>> way >>>>>>>>>>>>>> to deal with this; options vary from >>>>>>>>>>>>>> publishing lightning to the >>>>>>> free >>>>>>>>>>>>>> sonatype repo, joining the ASF (which is >>>>>>>>>>>>>> great anyhow!) and >>>>>>>>>>> republishing >>>>>>>>>>>>>> lightning code in DirectMemory becoming a >>>>>>>>>>>>>> commiter (which has to >>>>>>>>>>> undergo >>>>>>>>>>>>> a >>>>>>>>>>>>>> PMC vote). >>>>>>>>>>>>> >>>>>>>>>>>>> At least for the moment he can provide a >>>>>>>>>>>>> patch to be integrated in >>>>>>> DM >>>>>>>>>>> :-) >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> I'd like to hear your and the team feelings >>>>>>>>>>>>>> on this. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, Raffaele >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Sat, Sep 29, 2012 at 3:27 PM, Noctarius >>>>>>>>>>>>>> <[email protected]> >>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hey Raffaele, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> that's quite similar to what I did at >>>>>>>>>>>>>>> work. We're developing >>>>>>> Flash >>>>>>>>>>>>>>> online games and using a customized AMF >>>>>>>>>>>>>>> serialization. To support async writing >>>>>>>>>>>>>>> of the clients event results I added >>>>>>>>>>>>>>> serialization >>>>>>> of >>>>>>>>>>>>>>> the components / entities to the players >>>>>>>>>>>>>>> zone calculation and >>>>>>> stored >>>>>>>>>>>>>>> the pre-serialized bytestream directly to >>>>>>>>>>>>>>> the off-heap (using direct-ring-cache >>>>>>>>>>>>>>> implementation). When the client >>>>>>>>>>>>>>> requests the results (using long-polling) >>>>>>>>>>>>>>> I just write the pre-serialized >>>>>>> data to >>>>>>>>>>>>>>> the right position to deserialize it by >>>>>>>>>>>>>>> standard ways on Flash >>>>>>> side. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> So yeah an seriliaztion to off-heap >>>>>>>>>>>>>>> algorithm would be fine. You >>>>>>> can >>>>>>>>>>>>>>> avoid using byte arrays and minimalize >>>>>>>>>>>>>>> GC. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Cheers Chris >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Am 29.09.2012 15:02, schrieb Raffaele P. >>>>>>>>>>>>>>> Guidi: >>>>>>>>>>>>>>>> Nice to hear back from you! Yes, I was >>>>>>>>>>>>>>>> thinking about creating a >>>>>>>>>>> new >>>>>>>>>>>>>>> memory >>>>>>>>>>>>>>>> storage implementation using Unsafe >>>>>>>>>>>>>>>> (and I did it, recently) and >>>>>>>>>>>>> also, as >>>>>>>>>>>>>>>> DirectMemory relies heavily on >>>>>>>>>>>>>>>> serialization (and supports many >>>>>>> of >>>>>>>>>>>>> them, >>>>>>>>>>>>>>>> protostuff, protobuf, msgpack and of >>>>>>>>>>>>>>>> course standard >>>>>>>>>>> serialization), >>>>>>>>>>>>>>> about >>>>>>>>>>>>>>>> creating a simple embedded serializer >>>>>>>>>>>>>>>> leveraging the same >>>>>>>>>>> techniques >>>>>>>>>>>>> you >>>>>>>>>>>>>>>> used (Unsafe and bytecode generation). >>>>>>>>>>>>>>>> The idea with embedding is avoiding to >>>>>>>>>>>>>>>> serialize in a byte array >>>>>>>>>>> and >>>>>>>>>>>>> then >>>>>>>>>>>>>>>> moving the byte array to off-heap >>>>>>>>>>>>>>>> memory (via Unsafe or >>>>>>>>>>> ByteBuffers), >>>>>>>>>>>>> and >>>>>>>>>>>>>>>> serializing directly into a "managed" >>>>>>>>>>>>>>>> off-heap buffer, thus >>>>>>> further >>>>>>>>>>>>>>>> optimizing heap utilization (less GC). >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> What do you think about it? Does it >>>>>>>>>>>>>>>> make any sense to you? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Ciao, R >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Sat, Sep 29, 2012 at 2:40 PM, >>>>>>>>>>>>>>>> Noctarius <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Hey guys, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Raffaele found out about a project >>>>>>>>>>>>>>>>> of mine (Lightning) a few >>>>>>> weeks >>>>>>>>>>>>>>>>> ago. Lightning is a heavy Unsafe and >>>>>>>>>>>>>>>>> Bytecode generation using Serializer >>>>>>>>>>>>>>>>> implementation. He told me that he >>>>>>>>>>>>>>>>> was interested in adding something >>>>>>>>>>>>>>>>> similar to DirectMemory and I would >>>>>>>>>>>>>>>>> be glad to >>>>>>>>>>> help >>>>>>>>>>>>>>>>> out in this. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Another project I started a few days >>>>>>>>>>>>>>>>> ago, since it was needed >>>>>>> for >>>>>>>>>>>>>>>>> work is DirectRingCache. The name >>>>>>>>>>>>>>>>> not really meets to actual >>>>>>>>>>>>>>>>> implementation since it's not yet a >>>>>>>>>>>>>>>>> ring buffer using cache. I >>>>>>>>>>> used >>>>>>>>>>>>>>>>> this for a pre-serialization simple >>>>>>>>>>>>>>>>> bytestream cache with self-growing >>>>>>>>>>>>>>>>> buffers. It could be nice to have >>>>>>>>>>>>>>>>> DirectMemory >>>>>>> having >>>>>>>>>>>>>>>>> raw "buffers" to write to or to read >>>>>>>>>>>>>>>>> from. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Here are the links from the >>>>>>>>>>>>>>>>> projects: >>>>>>>>>>>>>>>>> https://github.com/noctarius/Lightning >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>> https://github.com/noctarius/direct-ring-cache >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>> It would be nice to help out since I really like the idea of >>>>>>>>>>>>>>>>> DirectMemory and since >>>>>>>>>>>>>>>>> direct-ring-cache is some kind of >>>>>>>>>>> reinventing >>>>>>>>>>>>>>>>> the wheel. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Cheers Noctarius (Chris) >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- Olivier Lamy Talend: >>>>>>>>>>>>> http://coders.talend.com >>>>>>>>>>>>> http://twitter.com/olamy | >>>>>>>>>>>>> http://linkedin.com/in/olamy >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- Olivier Lamy Talend: http://coders.talend.com >>>>>>>>>>> http://twitter.com/olamy | >>>>>>>>>>> http://linkedin.com/in/olamy >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- Olivier Lamy Talend: http://coders.talend.com >>>>>>> http://twitter.com/olamy | http://linkedin.com/in/olamy >>>>>>> >>>>>> >>>> >>>> >>>> >> >> >> >> -- >> Olivier Lamy >> Talend: http://coders.talend.com >> http://twitter.com/olamy | http://linkedin.com/in/olamy >> >
