Hi Andrei,

Andrei>Are nested structs (generic JSON) fully supported in calcite ?

They're already supported by the SQL parser and the relational algebra and
soon (<https://issues.apache.org/jira/browse/CALCITE-2404>) they will be
supported also by the execution engine in the enumerable convention.

Any level of struct is supported without any loss of type information. You
can create a RelDataType that is composed from other RelDataType using the
org.apache.calcite.rel.type.RelDataTypeFactory.Builder. In particular, for
nested types the method you are looking for is probably
org.apache.calcite.rel.type.RelDataTypeFactory.Builder#add(java.lang.String,
org.apache.calcite.rel.type.RelDataType)

Then you can query such kind of nested structures exactly as you wrote them
in your email. For more query examples you can have a look here:
<
https://github.com/zabetak/calcite/blob/871561e8c532661c42e5786d051e1f0018d34965/core/src/test/java/org/apache/calcite/test/ReflectiveSchemaTest.java#L356
>

Best,
Stamatis

On Sat, Aug 11, 2018, 9:37 AM Andrei Sereda <and...@sereda.cc> wrote:

> Hello Calcite Devs,
>
> I would like to ask your advice on the following use-case.
>
> Let’s say one wants to automatically construct (and then query) row type
> RelDataType out of existing model. In the case of elastic (but probably
> applicable to other databases as well) there is a notion of mappings
> <https://www.elastic.co/guide/en/elasticsearch/reference/
> current/mapping.html>
> which
> is exposes schema of your documents.
> An over simplified version (of the schema) can look like below:
>
> {
>    a: { type: boolean}
>    b: {
>        c: {type: long},
>        d: {type: string}
>        e: { f: {type: string}, g:{type: object} }
>    }
> }
>
> Is my understanding correct that only top-level elements can
> be explicitly defined in a struct (a and b but not c or g) ?
>
> What about querying such structs. Will the following query work ?
>
> select * from elastic tbl where tbl.b.e.f = 'foo' and tbl.b.c = 42
>
> Looking at the code I have found two examples:
>
>    1. _MAP which is a map type between VARCHAR and ANY. Access to fields is
>    done using _MAP['b.e.f'] syntax (mongo and elastic adapter)
>    2. Flat Structs which maps only top-level fields, nested ones (if they
>    exists) are generic
>    map types (geode adapter)
>
> The problem with _MAP (1) is that all type and schema information is lost
> and one needs
> to explicitly define a view to be able query a table.
>
> The inconvenience with “flat struct” (2) is that I can’t really query
> nested objects. Of course, one can flatten the structure like {a:string,
> b_e_f:string} but this is rather an ugly solution. Nor the following syntax
> is very elegant: tbl.b['e.f'].
>
> Are nested structs (generic JSON) fully supported in calcite ? If yes, can
> you point me in the right direction (RelDataType definion plus querying) ?
>
> Many Thanks,
> Andrei.
>

Reply via email to