Hi Andrei, Andrei>Are nested structs (generic JSON) fully supported in calcite ?
They're already supported by the SQL parser and the relational algebra and soon (<https://issues.apache.org/jira/browse/CALCITE-2404>) they will be supported also by the execution engine in the enumerable convention. Any level of struct is supported without any loss of type information. You can create a RelDataType that is composed from other RelDataType using the org.apache.calcite.rel.type.RelDataTypeFactory.Builder. In particular, for nested types the method you are looking for is probably org.apache.calcite.rel.type.RelDataTypeFactory.Builder#add(java.lang.String, org.apache.calcite.rel.type.RelDataType) Then you can query such kind of nested structures exactly as you wrote them in your email. For more query examples you can have a look here: < https://github.com/zabetak/calcite/blob/871561e8c532661c42e5786d051e1f0018d34965/core/src/test/java/org/apache/calcite/test/ReflectiveSchemaTest.java#L356 > Best, Stamatis On Sat, Aug 11, 2018, 9:37 AM Andrei Sereda <[email protected]> wrote: > Hello Calcite Devs, > > I would like to ask your advice on the following use-case. > > Let’s say one wants to automatically construct (and then query) row type > RelDataType out of existing model. In the case of elastic (but probably > applicable to other databases as well) there is a notion of mappings > <https://www.elastic.co/guide/en/elasticsearch/reference/ > current/mapping.html> > which > is exposes schema of your documents. > An over simplified version (of the schema) can look like below: > > { > a: { type: boolean} > b: { > c: {type: long}, > d: {type: string} > e: { f: {type: string}, g:{type: object} } > } > } > > Is my understanding correct that only top-level elements can > be explicitly defined in a struct (a and b but not c or g) ? > > What about querying such structs. Will the following query work ? > > select * from elastic tbl where tbl.b.e.f = 'foo' and tbl.b.c = 42 > > Looking at the code I have found two examples: > > 1. _MAP which is a map type between VARCHAR and ANY. Access to fields is > done using _MAP['b.e.f'] syntax (mongo and elastic adapter) > 2. Flat Structs which maps only top-level fields, nested ones (if they > exists) are generic > map types (geode adapter) > > The problem with _MAP (1) is that all type and schema information is lost > and one needs > to explicitly define a view to be able query a table. > > The inconvenience with “flat struct” (2) is that I can’t really query > nested objects. Of course, one can flatten the structure like {a:string, > b_e_f:string} but this is rather an ugly solution. Nor the following syntax > is very elegant: tbl.b['e.f']. > > Are nested structs (generic JSON) fully supported in calcite ? If yes, can > you point me in the right direction (RelDataType definion plus querying) ? > > Many Thanks, > Andrei. >
