>> What I don't get is how you go from a Java instance to a flatbuffer... Simply saying, it just copies the byte representation of each field of the object to the flat buffer, as what we did in ApacheDS 'by hand'. The different might be that, the 'by hand' codes are generated from a predefined schema file, instead of manually composing. No Java serialization is involved. Note Java is just one of the listed supported languages.
For now one thing I'm not sure about is how large could the flat buffer that's supported. It may not work if the content is too large to be held in memory without extra paging support like we did in Mavibot/Btree I guess. Regards, Kai -----Original Message----- From: Emmanuel Lécharny [mailto:[email protected]] Sent: Sunday, September 20, 2015 12:40 AM To: Apache Directory Developers List <[email protected]> Subject: Re: A default backend based on Google FlatBuffers? Le 19/09/15 13:55, Zheng, Kai a écrit : > Thanks Emmanuel for the feedback. > > Yeah, it needs encoding, that means the flatbuffers format should be used to > store entries. > > It does avoid decoding data, because it operates directly on the binary data. > In details, one may mmap loads the content from a file and gets a bytebuffer. > Then all the entries can be looked up directly on the bytebuffer. As you said > one may find an entry efficiently, I thought a mapping from key to the start > address of the corresponding object would be required. Given the start > address of the object, then all the fields of the object can be directly > retrieved without any decoding. What I don't get is how you go from a Java instance to a flatbuffer... Because, make no mistake, the costly processing is the serialization. In flatbuffer, how is this serialization done ? For instance, in ApacheDS, we serialize entries and other data using our own implementation, not depending on any Java default serialization (through reflection). the gain is massive. That's what I don't get : what's the flatbuffers offer that is better than what we do when serializing 'by hand' ?
