Hi Julian, Thanks a lot for writing this up. And sorry for taking so long to respond on this; attending Strata and then catching up after Strata ended up being more time consuming than I'd hoped. :-P
Overall, I love the temporal table idea. I think it's a very clean way of capturing and accessing the evolution of a table over time. I also like how the example used concisely motivates the challenges involved. I actually don't have any major comments for you at this time. I think the idea is solid at a high level (and I'm generally not qualified to comment on the nitty gritty SQL details :-). My only real complaint is that, as you allude to in the Further Work section, it doesn't address the details of handling out-of-order data. But after thinking about it a bit, I believe it meshes nicely with the semantics of the EMIT statement, at least as I was envisioning them (still not 100% sure we're in agreement there yet, but maybe). I was not expecting the two to interact so cleanly when I first started reading your doc, so I find that encouraging. I'm going to try to write up my thoughts on this over the next week or two with some diagrams to show what I mean. I'll post the doc on this thread once it's done. -Tyler On Wed, Sep 21, 2016 at 6:57 PM Julian Hyde <jh...@apache.org> wrote: > At FlinkForward the SQL BoF (birds-of-a-feather) group discussed several > issues relating to streaming SQL, but the discussion of stream-to-table > joins was particularly interesting. It raises issues of time semantics if > the table is changing over time, especially if we wish to be able to > re-play the stream later, join to the table, and get the same results. > > I have been thinking about the problem for the past few days, and have > written up my thoughts in a document [1]. The link is read-only, but I > welcome comments, so contact me back-channel if you would like write access. > > It is a rather lengthy read, I am afraid, partly because I include a lot > of examples. But I believe that the new concept that I have introduced, the > temporal table function (which I often abbreviate temporal table) has huge > expressive power, and can be a foundation for the semantics of streaming > query systems that need to interact with “historic” data. > > Please give it a read, and comment either in this email thread or in the > document. Also, please feel free to forward it to others outside the > Calcite community who are working on streaming query semantics, and invite > them to participate also. > > Julian > > > https://docs.google.com/document/d/1RvnLEEQK92axdAaZ9XIU5szpkbGqFMBtzYiIY4dHe0Q/edit?usp=sharing
