Chinmay Kolhatkar commented on APEXMALHAR-1818:
Here is a quick update on Apex-Calcite integration work.
Currently I'm able to run SQL statement as a DAG against registered table
abstractions of data endpoint and message type.
Here is the SQL support that is currently implemented:
1. Data Endpoint (Source/Destination):
2. Message Types from Data endpoint (source/destination):
3. SQL Functionality Support:
- SELECT (Projection) - Select from Source
- INSERT - Insert into Destination
- WHERE (Filter)
- Scalar functions which are provided in Calcite core
- Custom sclar function can be defined as provided to SQL.
4. Table can be defined as abstraction of Data Endpoint (source/dest) and
Currently Calcite integration with Apex is exposed as a small boiler plate code
in populateDAG as follows:
.registerTable("ORDERS", new KafkaEndpoint(broker, sourceTopic, new
.registerTable("SALES", new KafkaEndpoint(broker, destTopic, new
.executeSQL("INSERT INTO SALES " + "SELECT STREAM ROWTIME, " +
"FLOOR(ROWTIME TO DAY), " +
"APEXCONCAT('OILPAINT', SUBSTRING(PRODUCT, 6, 7)) " + "FROM
ORDERS WHERE ID > 3 " + "AND " +
"PRODUCT LIKE 'paint%'");
Following is a video recording of the demo of apex-capcite integration:
Currently I'm working on addition of inner join functionality.
Once the inner join functionality is implemented, I think the code is good to
create a Review Only PR for first cut of calcite integration.
The code is here:
[~julianhyde] [~thw] [~ilganeli] [~akekre] Please share your opinion on above.
> Integrate Calcite to support SQL
> Key: APEXMALHAR-1818
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-1818
> Project: Apache Apex Malhar
> Issue Type: New Feature
> Components: query operators
> Reporter: Amol
> Assignee: Chinmay Kolhatkar
> Labels: roadmap
> Once we have ability to generate a subdag, we should take a look at
> integrating Calcite into Apex. The operator that enables populate DAG, should
> use Calcite to generate the DAG, given a SQL query.
This message was sent by Atlassian JIRA