at Flink Forward we had a BOF session about StreamSQL.
After the conference, some folks and I sat down and drafted a proposal for
Flink's StreamSQL semantics.
The proposal includes:
- A definition for dynamic tables
- How to convert streams into dynamic tables
- How to convert dynamic tables into streams
- How to query dynamic tables
- Which types of queries to support
- How to specify early results, update rate, and late refinements
- How to control the size of the query state
- How to write query results to external systems (Kafka, files, Cassandra,
- How to make a query result accessible via Flink's queryable kv-state
- A few examples how StreamSQL queries can be defined
The proposal does not include a workplan or task breakdown yet.
This is something I'll work on in the next days.
Please share your thoughts and opinions about the proposal on the mailing