At FlinkForward the SQL BoF (birds-of-a-feather) group discussed several issues 
relating to streaming SQL, but the discussion of stream-to-table joins was 
particularly interesting. It raises issues of time semantics if the table is 
changing over time, especially if we wish to be able to re-play the stream 
later, join to the table, and get the same results.

I have been thinking about the problem for the past few days, and have written 
up my thoughts in a document [1]. The link is read-only, but I welcome 
comments, so contact me back-channel if you would like write access.

It is a rather lengthy read, I am afraid, partly because I include a lot of 
examples. But I believe that the new concept that I have introduced, the 
temporal table function (which I often abbreviate temporal table) has huge 
expressive power, and can be a foundation for the semantics of streaming query 
systems that need to interact with “historic” data.  

Please give it a read, and comment either in this email thread or in the 
document. Also, please feel free to forward it to others outside the Calcite 
community who are working on streaming query semantics, and invite them to 
participate also.


Reply via email to