Hello, I'm using Calcite to: 1. Extract the column lineage of a Spark SQL query 2. Rewrite SQL queries, specifically select statements to one or more merge statements
I was able to accomplish 1 and some of 2 however, currently I'm using the file adapter and some CSV files given that I was unable to connect Calcite to my SparkContext, such that the spark UDFs and tables are available to Calcite allowing me to parse the queries, Is there any way to accomplish that? should I be using Avatica? Regards, Shachar Bar David, Group Manager, Intuit Curation Platform. zoom: https://intuit.zoom.us/j/6405986798
