Just a note on the unit tests for the mongodb implementation, I noticed that they weren't actually using the test database and so the drop database and create db was having no effect, I fixed it and also fixed one or two of the tests that were relying on the db existing beforehand. so we should now have a complete set of unit tests for mongodb that do what we expect. Regards
On Thu, Jan 9, 2014 at 1:05 PM, Pablo Ojanguren <pablo...@gmail.com> wrote: > Hi all, > > Ben, I've seen your patch for this and I had mine ready also, so I've sent > it to review too. They both are pretty similar, so it should be good to > open a discussion about to pros and cons of each approach. > > Best regards, > > Pablo > > > 2014/1/3 Pablo Ojanguren <pablo...@gmail.com> > > > Ben, after seeing your draft patch I think we are taking different > > approaches. First I am not going to use morphia. For example I think > > DeltaStore doesn't represent an entity itself. Also I am not sure how > many > > collections this approach will generate (one collection per entity?) > But > > I don't have any experience in mapping libraries for MongoDB/noSQL so I > > can't say much more. > > > > About WaveletDeltaRecord serialization, this class contains: > > > > - Applied delta, already serialized as a protobuffer. I'd store it as > is. > > - Transformed delta: it includes author, versions, timestamp, an a list > of > > WaveletOperation objects. I'd serialize this tree in mongoDB fields > > > > So, at the end, in mongoDB, an unique document would be stored for this > > delta, of course, including in the same document Wave and Wavele tIds. > See > > attached > > MongoDbDeltaStore.java as a hint of my approach. > > > > > > As far as I know, Memory and File Delta store issues are about > scalabilty. > > Except for WAVE-399 and WAVE-393, the file storage bug regarding the > index > > file for deltas. > > > > > > BR, > > > > Pablo > > > > > > > > > > 2014/1/3 Frank R. <renfeng...@gmail.com> > > > > I have one delta directory of 11MB at the moment, which consists of four > >> files. One deltas file is 9.6MB. > >> > >> > >> On Fri, Jan 3, 2014 at 12:45 AM, Ben Hegarty <heg...@gmail.com> wrote: > >> > >> > Does anyone know if the deltas or index files are likely to ever hit > >> 16MB, > >> > as that is the maximum document size inside mongodb and if we are then > >> the > >> > persistence will need to be pushed into the mongodb GridFS? > >> > > >> > > >> > On Thu, Jan 2, 2014 at 9:58 AM, Frank R. <renfeng...@gmail.com> > wrote: > >> > > >> > > This feature is on my wish list for Wave. It's a good news. Thank > you~ > >> > > > >> > > > >> > > On Thu, Jan 2, 2014 at 5:24 PM, Ben Hegarty <heg...@gmail.com> > wrote: > >> > > > >> > > > Hi Pablo, > >> > > > Ironically I started work on this feature over the weekend as it > is > >> > next > >> > > on > >> > > > my list of must haves for my own personal project which also uses > >> > > MongoDB. > >> > > > I've had to upgrade the MongoDB driver and add in the morphia > plugin > >> > for > >> > > > modelling the delta store which hopefully I'll be able to continue > >> > > working > >> > > > on over the weekend. My time is limited and so if you think > you'll > >> be > >> > > able > >> > > > to finish before me let me know and I'll try and help with any > >> > knowledge > >> > > > I've gained around mongoDB. It looks like getting the delta store > >> > > finished > >> > > > is going to be fiddly work from what I've seen so far, I started > by > >> > > > modelling the store on the memory version, but the first problem > >> I've > >> > > found > >> > > > is that the '.' in mongoDB is a reserved character so I'm going to > >> have > >> > > to > >> > > > start modelling the store using a mongoDB specific implementation. > >> > > > > >> > > > If it helps I've uploaded my current work > >> > > > here< > >> > > > > >> > > > >> > > >> > https://drive.google.com/file/d/0B5FF_Ld8SzsNQ2JONHdGU2xJUG8/edit?usp=sharing > >> > > > >which > >> > > > just keep in mind is very much just the beginning. > >> > > > > >> > > > HTH > >> > > > Ben > >> > > > > >> > > > > >> > > > On Thu, Jan 2, 2014 at 8:41 AM, Pablo Ojanguren < > pablo...@gmail.com > >> > > >> > > > wrote: > >> > > > > >> > > > > Hi all, > >> > > > > > >> > > > > I am a developer working for an EU research project. There, we > are > >> > > going > >> > > > to > >> > > > > use Wave (via Kune project) so we would like to help fixing bugs > >> and > >> > > > > improving the software along wiht the community. Currently I > have > >> > some > >> > > > > available time for this for the next 10 months. > >> > > > > > >> > > > > After check out the issues, first thing I've paid attention is > the > >> > > > Delta's > >> > > > > storage. I have reviewed server / persistence code and I think I > >> can > >> > > > start > >> > > > > working in a mongoDB implementation. > >> > > > > > >> > > > > But maybe there is somebody else working on this. I would like > to > >> > join > >> > > > them > >> > > > > and to avoid duplicated efforts. Or if there is another > >> outstanding > >> > > > topics > >> > > > > to suggest it would be great to know too. > >> > > > > > >> > > > > Thank you and Happy New Year. > >> > > > > > >> > > > > Pablo > >> > > > > > >> > > > > >> > > > > >> > > > > >> > > > -- > >> > > > Mobile Phone: +447767 322 122 > >> > > > Home Phone: +4420 7737 6530 > >> > > > > >> > > > >> > > >> > > >> > > >> > -- > >> > Regards > >> > Ben > >> > > >> > > > > > -- Regards Ben
