Re: Adjust clacite to support kylin query
Re 3. Use a good IDE (I use IntelliJ, others use Eclipse). Find, run and debug
unit tests applicable to your case.
> On Feb 5, 2018, at 6:37 PM, Neters wrote:
>
> Dear Julian,
>
>
> Have a good day even if you UTC time is 2:00 am.
>
>
> I will have a try to implement the first method. Now we have only kylin
> datasource to deal with.
>
>
> If the practice fails i will try the second way or write an adapter. Because
> there are some difficult for me as following,or to say some questions:
>
>
> 1.I thought there are so many source code in calcite that i could not build
> the entire construction in my mind.
> 2.What knowledge should be prepared to understand the core thought of
> calcite, particulary about the data type such as RelDataType,or planing rules
> or the process for optimizing the sql parser process.
> 3.The third question may be not associated with calcite, it is just a
> ordinary question:how to read the source code in opensource project quickly.
> It is need to understand every detail in every source code file? I am a
> newbie in opensouce project modifying.
>
>
> If the other calcite project guys have good advice to learn opensource
> project source reading,just reply me.
>
>
> Best Regards To All
>
>
> neters
>
>
>
>
> ---------- 原始邮件 ------
> 发件人: "Julian Hyde";;
> 发送时间: 2018年2月6日(星期二) 凌晨3:53
> 收件人: "dev";
>
> 主题: Re: Adjust clacite to support kylin query
>
>
>
> If the query requires more than one data source (Kylin and something else)
> then a logical plan is necessary. If it’s just Kylin then what you suggest
> would work in principle, but it would be hard to make it work in practice.
>
> It might be simpler to fix the SQL generated by Calcite’s JDBC adapter to be
> what Kylin expects. Have you tried adding a Kylin dialect, making sure it
> gets used by RelToSqlConverter / JdbcImplementor?
>
> Julian
>
>
>> On Feb 4, 2018, at 9:53 PM, Neters wrote:
>>
>> Dear Julian,
>>
>>
>> Have a good day.
>>
>>
>> Yes. I have tried to connect to kylin via JDBC adapter, but the sql string
>> has been changed to which kylin engine does not support(sql string was
>> parsed not correctly ).
>>
>>
>> The sql query process i understand is that :
>>
>>
>> avatica jdbc driver gets connection and sumit sql string -->avatica server
>> receives the sql string->calcite convertes the sql string to a logical
>> plan-->
>> calcite applys planning rules to optimize the query process--> calcite
>> converts the sql string back and submits it to the database.
>>
>>
>> The description above is right?
>>
>>
>> I am wondering if it is possible to bypass the logical plan convertion.
>> After avatica server receives the sql string, calcite posts the sql string
>> to kylin rest server by kylin jdbc driver defined in the json model file? As
>> you can see, the implements of kylin jdbc is as following:
>> 1.register kylin jdbc driver
>> 2.get statement
>> 3.set query sql string and execute query( excute query by posting sql string
>> to kylin rest server)
>>
>>
>> If bypass issue could be carried out to query kylin, there might be a point
>> to bypass the convertion to logical plan.
>>
>>
>>
>>
>>
>>
>>
>> My avatica server source archive is as following:
>> project name
>>--java code--
>> +BiQueryServer.java
>> +ModelMetaFactory.java
>> --resources--
>> +model.json
>>
>>
>> BiQueryServer.java-
>> public class BiQueryServer {
>> public static void main(String[] args) {
>> List argsList = new ArrayList();
>> argsList.add("com.w3sun.sqlengine.ModelMetaFactory");
>> argsList.add("org.apache.calcite.avatica.jdbc.JdbcMeta");
>>
>>
>> for (String arg : args) {
>> argsList.add(arg);
>> }
>>
>>
>> String[] argsArray = (String[]) argsList.toArray(new
>> String[argsList.size()]);
>> try {
>> org.apache.calcite.avatica.server.Main.main(argsArray);
>> } catch (In
Re: Adjust clacite to support kylin query
If the query requires more than one data source (Kylin and something else) then
a logical plan is necessary. If it’s just Kylin then what you suggest would
work in principle, but it would be hard to make it work in practice.
It might be simpler to fix the SQL generated by Calcite’s JDBC adapter to be
what Kylin expects. Have you tried adding a Kylin dialect, making sure it gets
used by RelToSqlConverter / JdbcImplementor?
Julian
> On Feb 4, 2018, at 9:53 PM, Neters wrote:
>
> Dear Julian,
>
>
> Have a good day.
>
>
> Yes. I have tried to connect to kylin via JDBC adapter, but the sql string
> has been changed to which kylin engine does not support(sql string was parsed
> not correctly ).
>
>
> The sql query process i understand is that :
>
>
> avatica jdbc driver gets connection and sumit sql string -->avatica server
> receives the sql string->calcite convertes the sql string to a logical plan-->
> calcite applys planning rules to optimize the query process--> calcite
> converts the sql string back and submits it to the database.
>
>
> The description above is right?
>
>
> I am wondering if it is possible to bypass the logical plan convertion. After
> avatica server receives the sql string, calcite posts the sql string to kylin
> rest server by kylin jdbc driver defined in the json model file? As you can
> see, the implements of kylin jdbc is as following:
> 1.register kylin jdbc driver
> 2.get statement
> 3.set query sql string and execute query( excute query by posting sql string
> to kylin rest server)
>
>
> If bypass issue could be carried out to query kylin, there might be a point
> to bypass the convertion to logical plan.
>
>
>
>
>
>
>
> My avatica server source archive is as following:
> project name
> --java code--
>+BiQueryServer.java
>+ModelMetaFactory.java
>--resources--
>+model.json
>
>
> BiQueryServer.java-
> public class BiQueryServer {
>public static void main(String[] args) {
>List argsList = new ArrayList();
>argsList.add("com.w3sun.sqlengine.ModelMetaFactory");
>argsList.add("org.apache.calcite.avatica.jdbc.JdbcMeta");
>
>
>for (String arg : args) {
>argsList.add(arg);
>}
>
>
>String[] argsArray = (String[]) argsList.toArray(new
> String[argsList.size()]);
>try {
>org.apache.calcite.avatica.server.Main.main(argsArray);
>} catch (InterruptedException e) {
>e.printStackTrace();
>} catch (ClassNotFoundException e) {
>e.printStackTrace();
>} catch (IllegalAccessException e) {
>e.printStackTrace();
>} catch (InstantiationException e) {
>e.printStackTrace();
>}
>}
> }
>
> ModelMetaFactory.java-
> public class ModelMetaFactory implements Meta.Factory {
>private final String modelFile =
> this.getClass().getResource("/model.json").getPath().substring(1);
>private final String localJdbcUrl = "jdbc:calcite:";
>
>
>public void printMetaInfo(String meta) {
>System.out.println("[TnpmConnectionFactory.create] Implement with " +
> meta);
>
>
>}
>
>
>public Meta create(List list) {
>try {
>if (list.size() > 0 &&
> list.get(0).equals("org.apache.calcite.avatica.jdbc.JdbcMeta")) {
>return new JdbcMeta(localJdbcUrl + "model=" + (list.size() > 1
> ? list.get(1) : modelFile));
>} else {
>return null;
>}
>} catch (SQLException e) {
>throw new RuntimeException(e);
>}
>}
> }
>
> --model.json--
> {
> "version": "1.0",
> "defaultSchema": "kylin",
> "schemas": [
>{
> "name": "kylin",
> "type": "custom",
> "factory": "org.apache.calcite.adapter.jdbc.JdbcSchema$Factory",
> "opera
Re: Adjust clacite to support kylin query
Have you tried connecting to Kylin (and MySQL) via the JDBC adapter? (I know Kylin has Calcite inside, but they don’t package it as an adapter, so we’re forced to treat them as a black box.) Julian > On Feb 2, 2018, at 4:13 AM, Neters wrote: > > Hi guys, > > Good day. > > Sorry guys,the mail system remove my pic, so i make it attachment. > > I want to query apache kylin by calcite,but i did not find the adapter for > kylin. So i decide to modify the code attached in kylin source code. > My frame like this: > > avatica jdbc client > | > | > - >| calcite | >-- >/ | \ > kylin es mysql > > > > > Could you please give some constructive support? > > Best Regards > >
Re: Adjust clacite to support kylin query
There's no attachment on this message either. -- Michael Mior mm...@apache.org 2018-01-25 23:25 GMT-05:00 Neters : > Hi guys, > > Good day. > > Sorry guys,the mail system remove my pic, so i make it attachment. > > I want to query apache kylin by calcite,but i did not find the adapter for > kylin. So i decide to modify the code attached in kylin source code. > My frame like this: > > Could you please give some constructive support? > > Best Regards > > >
