Hello Julian
My use case is :
1. Database is based on elastic search
2. When I configure the data source using the syntax :
jdbc:calcite:schemaFactory=org.apache.calcite.adapter.druid.DruidSchemaFactory;
schema.host=http://localhost:8082;
schema.urlpath=/das/<tenantid>/<prodid>/<docid> , the connection is
successful. (Though it gets no tables here as the schema details are not passed)
3. However when I try to query on this database, an exception is
thrown, since the schema is not found.
My custom database also has an API to return the schema details
in the format :
{
"mappings": {
"some_type": {
"properties": {
"reqSize": {
"index": "no",
"store": "yes",
"type": "integer"
},
"apiName": {
"index": "not_analyzed",
"store": "yes",
"type": "string"
},
"accountPlanName": {
"index": "not_analyzed",
"store": "yes",
"type": "string"
}
I need to understand how(which calcite classes) to leverage in order to suffice
the details which the driver expects from the model.json viz. dimensions,
,metrics, datasource.
I referred to http://druid.io/docs/latest/querying/segmentmetadataquery.html
(which in druid, discovers the columns of the table), but again could not find
the required context.
Thanks and Regards
Shuchi Gupta
Software Engineer (CA)
-----Original Message-----
From: Julian Hyde [mailto:[email protected]]
Sent: Tuesday, December 27, 2016 11:18 PM
To: [email protected]
Subject: Re: Eliminating model.json from druid implementation
We introduced the “schemaFactory” connect string parameter in
https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_CALCITE-2D1259&d=DwIFaQ&c=_hRq4mqlUmqpqlyQ5hkoDXIVh6I6pxfkkNxQuL0p-Z0&r=_Fj_BSkIJErzZ82vma0tSpR2D2tntATQeeTvZFPXEMI&m=-Z_7HdytOg0UWaKQGuutkRhOfJSOU1rbK5vlgxiNCac&s=CoSEy4c4OzsxPsEiEpC6AXlB_QpnqxVwx1oi1kaQg50&e=
<https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_CALCITE-2D1259&d=DwIFaQ&c=_hRq4mqlUmqpqlyQ5hkoDXIVh6I6pxfkkNxQuL0p-Z0&r=_Fj_BSkIJErzZ82vma0tSpR2D2tntATQeeTvZFPXEMI&m=-Z_7HdytOg0UWaKQGuutkRhOfJSOU1rbK5vlgxiNCac&s=CoSEy4c4OzsxPsEiEpC6AXlB_QpnqxVwx1oi1kaQg50&e=
> for cases where the connection had a single schema and that schema was
trivial. That doesn’t mean that the schema is empty (a schema with no tables is
fairly useless) but that it is self-populating. If I recall correctly the Druid
schema factory creates a Calcite table for each data set in Druid, so it
fulfills these criteria.
But if you need more content in a schema, I suggest you go back to the old way
and generate a small JSON model. Note that you can prefix the model URL with
“inline:” to pass the content of the model rather than putting the model in a
temporary file.
Another alternative is to dispense with the model entirely and use methods such
as Schema.addTable to build the schema programmatically. Really, the JSON model
stuff is just a builder on top of that.
Julian
> On Dec 27, 2016, at 3:26 AM, Gupta, Shuchi <[email protected]> wrote:
>
> Hi Team
>
> In reference to apache druid, I have been using a JSON file as metadata for
> my connections to a custom database based on elastic search.
> Instead I intend to create the metadata on the fly by eliminating this json
> path from the connection url and use :
> jdbc:calcite:schemaFactory=org.apache.calcite.adapter.druid.DruidSchemaFactory;
>
> schema.url=https://urldefense.proofpoint.com/v2/url?u=http-3A__localhost-3A8082&d=DwIFaQ&c=_hRq4mqlUmqpqlyQ5hkoDXIVh6I6pxfkkNxQuL0p-Z0&r=_Fj_BSkIJErzZ82vma0tSpR2D2tntATQeeTvZFPXEMI&m=-Z_7HdytOg0UWaKQGuutkRhOfJSOU1rbK5vlgxiNCac&s=OycHaZLAMQQghbn3mXboRMji1wCxmFjLZX0D0rUBJ7Y&e=
>
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__localhost-3A8082_&d=DwIFaQ&c=_hRq4mqlUmqpqlyQ5hkoDXIVh6I6pxfkkNxQuL0p-Z0&r=_Fj_BSkIJErzZ82vma0tSpR2D2tntATQeeTvZFPXEMI&m=-Z_7HdytOg0UWaKQGuutkRhOfJSOU1rbK5vlgxiNCac&s=T49R90tuqZBfVBSSzLcajDsMvEnSqSVZmnNncepsPJI&e=
> >;
> schema.coordinatorUrl=https://urldefense.proofpoint.com/v2/url?u=http-3A__localhost-3A8081&d=DwIFaQ&c=_hRq4mqlUmqpqlyQ5hkoDXIVh6I6pxfkkNxQuL0p-Z0&r=_Fj_BSkIJErzZ82vma0tSpR2D2tntATQeeTvZFPXEMI&m=-Z_7HdytOg0UWaKQGuutkRhOfJSOU1rbK5vlgxiNCac&s=wzgfGP9HYu8RhR3vFxpQVRCmLpYr2oyXevKyGKAyfEk&e=
>
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__localhost-3A8081_&d=DwIFaQ&c=_hRq4mqlUmqpqlyQ5hkoDXIVh6I6pxfkkNxQuL0p-Z0&r=_Fj_BSkIJErzZ82vma0tSpR2D2tntATQeeTvZFPXEMI&m=-Z_7HdytOg0UWaKQGuutkRhOfJSOU1rbK5vlgxiNCac&s=_yVux-ThDVg6cWxvPKimL3fH2aFpDu9GLTOscJtp7t8&e=
> >
>
> The connection is passing successfully, but I fail to understand where shall
> I map the schema details . Kindly let me know what change would be required
> to implement this functionality.
>
>
> Thanks and Regards
> Shuchi Gupta
> Software Engineer (CA)
>
> CA Technologies | Plot 115/1, 115/16-115/21 | Hyderabad, 500032
> Office: +71262 | [email protected]
>
> <mailto:[email protected]> <http://www.ca.com/us/default.aspx>
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__twitter.com_CAInc&d=DwIFaQ&c=_hRq4mqlUmqpqlyQ5hkoDXIVh6I6pxfkkNxQuL0p-Z0&r=_Fj_BSkIJErzZ82vma0tSpR2D2tntATQeeTvZFPXEMI&m=-Z_7HdytOg0UWaKQGuutkRhOfJSOU1rbK5vlgxiNCac&s=JzFK044MbEhXHelmPYjCCOG92mSJHFu4gATLhvNhPY0&e=
> >
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.slideshare.net_cainc&d=DwIFaQ&c=_hRq4mqlUmqpqlyQ5hkoDXIVh6I6pxfkkNxQuL0p-Z0&r=_Fj_BSkIJErzZ82vma0tSpR2D2tntATQeeTvZFPXEMI&m=-Z_7HdytOg0UWaKQGuutkRhOfJSOU1rbK5vlgxiNCac&s=NrO13bz7BJYqUDx5nY_M6RGEDF68hsTP32_SfLXrIRs&e=
> >
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.facebook.com_CATechnologies&d=DwIFaQ&c=_hRq4mqlUmqpqlyQ5hkoDXIVh6I6pxfkkNxQuL0p-Z0&r=_Fj_BSkIJErzZ82vma0tSpR2D2tntATQeeTvZFPXEMI&m=-Z_7HdytOg0UWaKQGuutkRhOfJSOU1rbK5vlgxiNCac&s=ks2-feP5jzgKs3wj_H2ruahrqErjryRkxgOkNudFYfQ&e=
> >
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.youtube.com_user_catechnologies&d=DwIFaQ&c=_hRq4mqlUmqpqlyQ5hkoDXIVh6I6pxfkkNxQuL0p-Z0&r=_Fj_BSkIJErzZ82vma0tSpR2D2tntATQeeTvZFPXEMI&m=-Z_7HdytOg0UWaKQGuutkRhOfJSOU1rbK5vlgxiNCac&s=ZjmZaqfCaSLgP7im_zKfexZrFGiTWKxKWdE48Ypm3ko&e=
> >
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.linkedin.com_company_1372-3Fgoback-3D.cps-5F1244823420724-5F1&d=DwIFaQ&c=_hRq4mqlUmqpqlyQ5hkoDXIVh6I6pxfkkNxQuL0p-Z0&r=_Fj_BSkIJErzZ82vma0tSpR2D2tntATQeeTvZFPXEMI&m=-Z_7HdytOg0UWaKQGuutkRhOfJSOU1rbK5vlgxiNCac&s=TxJ7K8VMK8PRSxyfmZbRAaIBm9Nonur5wgpb7Im9KMo&e=
> >
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__plus.google.com_-2BCATechnologies&d=DwIFaQ&c=_hRq4mqlUmqpqlyQ5hkoDXIVh6I6pxfkkNxQuL0p-Z0&r=_Fj_BSkIJErzZ82vma0tSpR2D2tntATQeeTvZFPXEMI&m=-Z_7HdytOg0UWaKQGuutkRhOfJSOU1rbK5vlgxiNCac&s=x9K_bIo6jzauz7edv2y8wjj9U29y5ec3vxtyqoyxpTY&e=
> > <http://www.ca.com/us/rss.aspx?intcmp=footernav>
