[DISCUSS] Implementing Mango Indexes for FoundationDB

2019-03-28 Thread Garren Smith
Hi everyone, I want to start a discussion, with the aim of an RFC, around implementing Mango JSON indexes for FoundationDB. Currently Mango indexes are a layer above CouchDB map/reduce indexes, but with FoundationDB we can make them separate indexes in FoundationDB. This gives us the possibility

Re: [DISCUSS] Implementing _all_docs on FoundationDB

2019-03-28 Thread Garren Smith
A little behind on the discussion emails but +1 to option 1 for include_docs=true and option 3 for include_docs = false. On Mon, Mar 25, 2019 at 12:26 PM Jan Lehnardt wrote: > +1 on what Bob said. > > > On 21. Mar 2019, at 20:57, Robert Newson wrote: > > > > Hi, > > > > Thanks for pushing

Re: Prototype CouchDB Layer for FoundationDB

2019-03-28 Thread Paul Davis
On Thu, Mar 28, 2019 at 10:58 AM Adam Kocoloski wrote: > > Hi Paul, good stuff. > > I agree with you about the FDB Subspaces feature. I’ve been thinking that our > layer code should maintain its own enumeration of the various “subspaces” to > single-byte prefixes within the directory. I haven’t

Re: [DISCUSS] Implementing Mango Indexes for FoundationDB

2019-03-28 Thread Adam Kocoloski
Hi Garren, cool, this is a good start. On the ICU side of things, Russell pointed out that sort keys are a one-way trip; i.e., there’s no way to recover the original string from a sort key. For the initial pass at Mango I think that’s OK, as we’re reading the indexed documents anyway. When we

Re: Prototype CouchDB Layer for FoundationDB

2019-03-28 Thread Adam Kocoloski
Hi Paul, good stuff. I agree with you about the FDB Subspaces feature. I’ve been thinking that our layer code should maintain its own enumeration of the various “subspaces” to single-byte prefixes within the directory. I haven’t yet captured that in the RFCs, but e.g. we should be using

Re: [DISCUSS] _db_updates feed in FoundationDB

2019-03-28 Thread Ilya Khlopotov
> Due to scoping of the de-duplication operation to single database and use of > random sampling we would be able to cleanup frequently updated operation at a > different rate than less frequently updated ones. There is a typo there ^^^. It should be corrected as: Due to scoping of the