I like the idea, both for the efficiencies it enables in the FoundationDB data model and for the ability to cover a lot of validation functionality without shelling out to JS.
It’s pretty obviously a big, meaty topic unto itself, one that needs some careful thought and design. Also an awful lot of opportunity for scope creep. But a good topic nonetheless. Adam > On Jan 31, 2019, at 12:05 PM, Robert Newson <b...@rsn.io> wrote: > > Hi, > > An enhancement over the first idea (where we flatten JSON documents into keys > and values where the key is the full path to every terminal value, regardless > of depth in the JSON) is to allow users to register schemas. > > For documents that match a registered schema (suggestion, a top level field > called "_schema" is required to mark the documents) we can convert to > key/value pairs much more compactly. The key name, barring whatever prefix > identifies the database itself, is just the name of the schema and the > ordinal of the schema item relative to the declaration. > > These schema documents (living under /dbname/_schema/$scheme_name akin to > design documents) would list all required and optional fields, their types > and perhaps other constraints (like valid ranges or relationships with other > fields). We could get arbitrarily complex in schema definitions over time. > Effectively, these are validate_doc_update functions without the Javascript > evaluation pain. > > We don't necessarily need this in the first version, but it feels like a > better response to the worries over the restrictions that the flattening idea > is causing than switching to an opaque series of 100k chunks. > > thoughts? > B > > -- > Robert Newson > b...@rsn.io > > On Thu, 31 Jan 2019, at 16:26, Adam Kocoloski wrote: >> >>> On Jan 31, 2019, at 1:47 AM, ermouth <ermo...@gmail.com> wrote: >>> >>>> As I don't see the 10k limitation as having significant merit >>> >>> Not sure it’s relevant here, but Mango indexes put selected doc values into >>> keys. >>> >>> ermouth >> >> Totally relevant. Not just because of the possibility of putting a large >> scalar value into the index, but because someone could create an index >> on a field that is itself a container, and Mango could just dump the >> entire container into the index as the key. >> >> Presumably there’s a followup discussion dedicated to indexing where we >> can suss out what to do in that scenario. >> >> Adam
