PS: Links below won't work for me "connection cancelled". Is there a problem with the servers?
Am 29.09.2012 19:00, schrieb Noctarius: > Thanks Olivier for carify, I'll take a look in it tomorrow but > there's just one question left (for now ;)): What is that vote > for becoming a committer? What if the vote will be negative? > Until now I just used Apache stuff, was never interested in > being part of it so I guess it can be negative for any reason, > can't it? > > Am 29.09.2012 18:56, schrieb Olivier Lamy: >> 2012/9/29 Noctarius <[email protected]>: >>> Nope my real name is Christoph Engelbert, but Noctarius is >>> the all time nick :) >>> >>> Renaming the package should be no problem, is it >>> "org.apache.directmemory.lightning" or what would it be? >> fine for me >>> Then there needs to be a change in the license header as >>> Olivier mentioned, that means just remove the first >>> sentence or is there anything more to do (maybe it's >>> easiest thing to just copy the header from DM file ;))? >> yup use same header as DM >>> >>> The CLA is just a form to clarify that the source can be >>> contributed to the Apache Foundation? >> yup correct. >>> >>> The final step will be attaching the patch in form of a >>> huge diff file? >> yes >>> >>> And what is the way to apply for a membership? Never >>> thought about how to do that. >> Read here http://apache.org/foundation/how-it-works.html and >> here http://apache.org/foundation/getinvolved.html . And >> feel free to ask any questions :-) >>> >>> Chris >>> >>> Am 29.09.2012 18:23, schrieb Raffaele P. Guidi: >>>> OK, deal, at least for me ;-) I propose you rename the >>>> packages, produce a patch for this and the new serializer >>>> module (should be simple enough starting from an >>>> existing one) and, in the meanwhile, apply for ASF >>>> membership. Is IP clearance needed? I guess yes. After >>>> this we will come up with a formal vote regarding >>>> Noctarius (is this your real name?!) allowance in the >>>> project team. >>>> >>>> Good times are gonna come :-) >>>> >>>> Thanks, R Il giorno 29/set/2012 17:58, "Olivier Lamy" >>>> <[email protected]> ha scritto: >>>> >>>>> 2012/9/29 Raffaele P. Guidi >>>>> <[email protected]>: >>>>>> Well we already have a NIO ready interface allowing >>>>>> direct access to DMs managed bytebuffers but I think >>>>>> this is just half way to what could be achieved >>>>>> optimally blending serialization and memory >>>>>> allocation together. >>>>>> >>>>>> Lightning as a module is of course a good idea and >>>>>> it could easily evolve as a subproject (for the more >>>>>> experienced asf members: is it a feasible way?). >>>>> Nothing prevent to have >>>>> http://svn.apache.org/repos/asf/directmemory/lightning/trunk >>>>> >>>>> > >>>>> with a package like: o.a.d.lightning That will be a >>>>> subproject. >>>>>> >>>>>> Ciao, R Il giorno 29/set/2012 17:44, "Noctarius" >>>>>> <[email protected]> ha scritto: >>>>>> >>>>>>> Hey guys, >>>>>>> >>>>>>> ok there's no lightning binary available since >>>>>>> lightning wasn't ready yet for releasing. >>>>>>> >>>>>>> For being the only developer it would be no >>>>>>> problem to contribute the sourcebase for >>>>>>> DirectMemory but I'm not sure yet if it wouldn't be >>>>>>> better to seperate it to be available without using >>>>>>> DirectMemory itself. I started it as a serializer >>>>>>> for cluster synchronization, but it would be cool >>>>>>> to contribute lightning as a subproject to >>>>>>> DirectMemory :-) >>>>>>> >>>>>>> About the second project I would love to see a >>>>>>> public available buffer API directly in >>>>>>> DirectMemory so that project would be nearly >>>>>>> needless :-) The only difference I think is the >>>>>>> allocation strategy my implementation is using >>>>>>> against the one DirectMemory has, but I'm pretty >>>>>>> sure the allocation is extensible ;-) >>>>>>> >>>>>>> Like I said, for both projects I'm the only dev so >>>>>>> there would be no IP problem. So if it's ok to you >>>>>>> to not include lightning directly in DM I would be >>>>>>> glad to contribute to the Apache Foundation :-) >>>>>>> >>>>>>> Cheers Chris >>>>>>> >>>>>>> >>>>>>> Am 29.09.2012 16:53, schrieb Raffaele P. Guidi: >>>>>>>> Ok so it's up to noctarius - your move! ;-) >>>>>>>> Regarding the new unsafe storage: it's an opt-in >>>>>>>> feature that can be set with the fluent API >>>>> (and >>>>>>>> soon through the conference file). >>>>>>>> >>>>>>>> Ciao, R Il giorno 29/set/2012 16:45, "Olivier >>>>>>>> Lamy" <[email protected]> ha >>>>>>> scritto: >>>>>>>> >>>>>>>>> 2012/9/29 Raffaele P. Guidi >>>>>>>>> <[email protected]>: >>>>>>>>>>>> At least for the moment he can provide a >>>>>>>>>>>> patch to be integrated >>>>> in DM >>>>>>>>> :-) >>>>>>>>>> >>>>>>>>>> Sure, but as lightning is not in any public >>>>>>>>>> mvn repo should its >>>>> code be >>>>>>>>>> re-published in our svn? Or what? >>>>>>>>> @Apache we don't care about binaries, only >>>>>>>>> sources are important ! (a bit theorical for >>>>>>>>> sure but that's it :-) ). So if Noctarius was >>>>>>>>> the only guy who participate in lightning. He >>>>>>>>> can just provide a patch we could integrate as >>>>>>>>> a new dm module (note: the patch must not >>>>>>>>> contains any more copyright and all sources >>>>>>>>> must have ASF licenses). >>>>>>>>> >>>>>>>>> "Copyright 2012 the original author or >>>>>>>>> authors." must be removed. And BTW package must >>>>>>>>> be changed :-) (com.github is not acceptable >>>>> @asf >>>>>>>>> :-) )(@Noctarius are you working for github ? >>>>>>>>> :-) ) >>>>>>>>> >>>>>>>>> And having him as a committer will be only a >>>>>>>>> matter of voting (we >>>>> have >>>>>>>>> a great chair who take cares of administrative >>>>>>>>> stuff :P ) >>>>>>>>> >>>>>>>>> If some others have participated in the >>>>>>>>> project, we must pass tru an ip clearance >>>>>>>>> mechanism >>>>>>>>> (http://incubator.apache.org/ip-clearance/index.html) >>>>>>>>> >>>>>>>>> > >>>>>>>>> and all contributors to lightning must provide a cla. >>>>>>>>> (It it's the case I can help) >>>>>>>>> >>>>>>>>>> >>>>>>>>>>>> perso I'd like we avoid hard dependency >>>>>>>>>>>> on Unsafe as maybe some >>>>> use >>>>>>>>>> other jdks :-) >>>>>>>>>> >>>>>>>>>> Well, I believe Unsafe is supported by Sun >>>>>>>>>> JDK, OpenJDK, IBM JDK and JRockit - and I >>>>>>>>>> believe that it is more than enough. Also >>>>>>>>>> keep in >>>>> mind >>>>>>>>> that >>>>>>>>>> we already have an alternative Unsafe based >>>>>>>>>> memory storage - and >>>>>>> although >>>>>>>>>> it's not thoroughly tested for performance it >>>>>>>>>> dramaticaly simplifies >>>>>>>>> code, >>>>>>>>>> I have great expectations about it. >>>>>>>>> Me too :-). Yup definitely more simple and >>>>>>>>> faster ! But we must provide a switch off >>>>>>>>> configuration mechanism if some >>>>> users >>>>>>>>> don't want to use that (because Unsafe is just >>>>>>>>> "Unsafe" :-) ) And sorry I didn't have a look >>>>>>>>> yet at your changes with using Unsafe. >>>>>>>>>> >>>>>>>>>> Cheers, R >>>>>>>>>> >>>>>>>>>> On Sat, Sep 29, 2012 at 4:03 PM, Olivier Lamy >>>>>>>>>> <[email protected]> >>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> 2012/9/29 Raffaele P. Guidi >>>>>>>>>>> <[email protected]>: >>>>>>>>>>>> What do you think about: 1) implementing >>>>>>>>>>>> a lightning serialization module 2) >>>>>>>>>>>> creating a serializer that directly works >>>>>>>>>>>> on a directmemory >>>>>>>>> provider >>>>>>>>>>>> ByteBuffer or (maybe better) Unsafe based >>>>>>>>>>>> Pointer? >>>>>>>>>>> >>>>>>>>>>> Sounds good (perso I'd like we avoid hard >>>>>>>>>>> dependency on Unsafe as maybe some use >>>>>>>>>>> other jdks :-) ) >>>>>>>>>>>> >>>>>>>>>>>> Now I see lightning is apache licensed >>>>>>>>>>>> and this is fine but I >>>>> don't >>>>>>>>> think >>>>>>>>>>>> it is published in any public maven >>>>>>>>>>>> repo, am I right? We could >>>>> find a >>>>>>>>> way >>>>>>>>>>>> to deal with this; options vary from >>>>>>>>>>>> publishing lightning to the >>>>> free >>>>>>>>>>>> sonatype repo, joining the ASF (which >>>>>>>>>>>> is great anyhow!) and >>>>>>>>> republishing >>>>>>>>>>>> lightning code in DirectMemory becoming a >>>>>>>>>>>> commiter (which has to >>>>>>>>> undergo >>>>>>>>>>> a >>>>>>>>>>>> PMC vote). >>>>>>>>>>> >>>>>>>>>>> At least for the moment he can provide a >>>>>>>>>>> patch to be integrated in >>>>> DM >>>>>>>>> :-) >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I'd like to hear your and the team >>>>>>>>>>>> feelings on this. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, Raffaele >>>>>>>>>>>> >>>>>>>>>>>> On Sat, Sep 29, 2012 at 3:27 PM, >>>>>>>>>>>> Noctarius <[email protected]> >>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hey Raffaele, >>>>>>>>>>>>> >>>>>>>>>>>>> that's quite similar to what I did at >>>>>>>>>>>>> work. We're developing >>>>> Flash >>>>>>>>>>>>> online games and using a customized AMF >>>>>>>>>>>>> serialization. To support async >>>>>>>>>>>>> writing of the clients event results I >>>>>>>>>>>>> added serialization >>>>> of >>>>>>>>>>>>> the components / entities to the >>>>>>>>>>>>> players zone calculation and >>>>> stored >>>>>>>>>>>>> the pre-serialized bytestream directly >>>>>>>>>>>>> to the off-heap (using >>>>>>>>>>>>> direct-ring-cache implementation). When >>>>>>>>>>>>> the client requests the results (using >>>>>>>>>>>>> long-polling) I just write the >>>>>>>>>>>>> pre-serialized >>>>> data to >>>>>>>>>>>>> the right position to deserialize it by >>>>>>>>>>>>> standard ways on Flash >>>>> side. >>>>>>>>>>>>> >>>>>>>>>>>>> So yeah an seriliaztion to off-heap >>>>>>>>>>>>> algorithm would be fine. You >>>>> can >>>>>>>>>>>>> avoid using byte arrays and minimalize >>>>>>>>>>>>> GC. >>>>>>>>>>>>> >>>>>>>>>>>>> Cheers Chris >>>>>>>>>>>>> >>>>>>>>>>>>> Am 29.09.2012 15:02, schrieb Raffaele >>>>>>>>>>>>> P. Guidi: >>>>>>>>>>>>>> Nice to hear back from you! Yes, I >>>>>>>>>>>>>> was thinking about creating a >>>>>>>>> new >>>>>>>>>>>>> memory >>>>>>>>>>>>>> storage implementation using Unsafe >>>>>>>>>>>>>> (and I did it, recently) and >>>>>>>>>>> also, as >>>>>>>>>>>>>> DirectMemory relies heavily on >>>>>>>>>>>>>> serialization (and supports many >>>>> of >>>>>>>>>>> them, >>>>>>>>>>>>>> protostuff, protobuf, msgpack and of >>>>>>>>>>>>>> course standard >>>>>>>>> serialization), >>>>>>>>>>>>> about >>>>>>>>>>>>>> creating a simple embedded serializer >>>>>>>>>>>>>> leveraging the same >>>>>>>>> techniques >>>>>>>>>>> you >>>>>>>>>>>>>> used (Unsafe and bytecode >>>>>>>>>>>>>> generation). The idea with embedding >>>>>>>>>>>>>> is avoiding to serialize in a byte >>>>>>>>>>>>>> array >>>>>>>>> and >>>>>>>>>>> then >>>>>>>>>>>>>> moving the byte array to off-heap >>>>>>>>>>>>>> memory (via Unsafe or >>>>>>>>> ByteBuffers), >>>>>>>>>>> and >>>>>>>>>>>>>> serializing directly into a "managed" >>>>>>>>>>>>>> off-heap buffer, thus >>>>> further >>>>>>>>>>>>>> optimizing heap utilization (less >>>>>>>>>>>>>> GC). >>>>>>>>>>>>>> >>>>>>>>>>>>>> What do you think about it? Does it >>>>>>>>>>>>>> make any sense to you? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Ciao, R >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Sat, Sep 29, 2012 at 2:40 PM, >>>>>>>>>>>>>> Noctarius <[email protected]> >>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hey guys, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Raffaele found out about a project >>>>>>>>>>>>>>> of mine (Lightning) a few >>>>> weeks >>>>>>>>>>>>>>> ago. Lightning is a heavy Unsafe >>>>>>>>>>>>>>> and Bytecode generation using >>>>>>>>>>>>>>> Serializer implementation. He told >>>>>>>>>>>>>>> me that he was interested in adding >>>>>>>>>>>>>>> something similar to DirectMemory >>>>>>>>>>>>>>> and I would be glad to >>>>>>>>> help >>>>>>>>>>>>>>> out in this. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Another project I started a few >>>>>>>>>>>>>>> days ago, since it was needed >>>>> for >>>>>>>>>>>>>>> work is DirectRingCache. The name >>>>>>>>>>>>>>> not really meets to actual >>>>>>>>>>>>>>> implementation since it's not yet >>>>>>>>>>>>>>> a ring buffer using cache. I >>>>>>>>> used >>>>>>>>>>>>>>> this for a pre-serialization simple >>>>>>>>>>>>>>> bytestream cache with self-growing >>>>>>>>>>>>>>> buffers. It could be nice to have >>>>>>>>>>>>>>> DirectMemory >>>>> having >>>>>>>>>>>>>>> raw "buffers" to write to or to >>>>>>>>>>>>>>> read from. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Here are the links from the >>>>>>>>>>>>>>> projects: >>>>>>>>>>>>>>> https://github.com/noctarius/Lightning >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> https://github.com/noctarius/direct-ring-cache >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>> It would be nice to help out since I really like the idea >>> of >>>>>>>>>>>>>>> DirectMemory and since >>>>>>>>>>>>>>> direct-ring-cache is some kind of >>>>>>>>> reinventing >>>>>>>>>>>>>>> the wheel. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Cheers Noctarius (Chris) >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- Olivier Lamy Talend: >>>>>>>>>>> http://coders.talend.com >>>>>>>>>>> http://twitter.com/olamy | >>>>>>>>>>> http://linkedin.com/in/olamy >>>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- Olivier Lamy Talend: >>>>>>>>> http://coders.talend.com >>>>>>>>> http://twitter.com/olamy | >>>>>>>>> http://linkedin.com/in/olamy >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>>> >>>>> >>>>> -- Olivier Lamy Talend: http://coders.talend.com >>>>> http://twitter.com/olamy | >>>>> http://linkedin.com/in/olamy >>>>> >>>> >> >> >>
