Thanks for the quick response Enrico.
>> Not sure if I understand correctly, but you are looking for something
given an SQL statement and metadata about existing tables it returns one
(or more) access plan ?
Yes, we are looking for table/DB object level plan
Just for example, in the case below query,
SqlNode node5 = getSqlParser("insert into table5 select a.c1, a.c2
from (select * from table1 join table2 on table1.c1 = table2.c1 )
The table level lineage will be [table1, table2] -> [ tmp_table ] ->
[ table5 ]. And this we are trying to get from above sqlNode object.
May, I know, how to parse MySQL/Vertica/Hive queries? Is it through
Thanks & Regards,
B Anil Kumar.
On Wed, Apr 11, 2018 at 1:45 PM, Enrico Olivelli <eolive...@gmail.com>
> Il mer 11 apr 2018, 22:34 AnilKumar B <akumarb2...@gmail.com> ha scritto:
> > Hi All,
> > I am new to calcite. Need your help/inputs on usage of calcite to solve
> > of our use case.
> > We need to build a library to get the lineage graph of given query. Here
> > given query can be either Hive/ANSI SQL/Vertica queries etc.
> > We can build this library by using their respective physical plans. But,
> > there are few issues as mentioned below
> > 1. We need develop the library specific to each source
> > 2. In most of the sources, there is a limitation on explain query for
> > *create
> > table* queries.
> > 3. To get the explain plan, we need to have all the tables (which we are
> > used in query) should be in the in their respective meta stores. So, it
> > might cause the issues with tmp tables from previous queries.
> > So, to avoid above issues, we are trying to develop a generic library
> > can work for any source. For that, we are trying to use calcite parser's
> > SQLNode to derive the lineage for given query.
> > We started with ANSI SQL queries and as of now, it's going good. But any
> > inputs from community
> > on below will be great help for us.
> > 1. Does calcite is correct fit for our use case?
> > 2. How to parse the JDBC queries and get the lineage?
> Not sure if I understand correctly, but you are looking for something that
> given an SQL statement and metadata about existing tables it returns one
> (or more) access plan ?
> In this case Calcite is very a good choice. I will be happy to provide some
> example if this is your case.
> > Thanks & Regards,
> > B Anil Kumar.
> -- Enrico Olivelli