Re: Streaming SQL - object models, ASTs and algebras

2015-01-31 Thread Chris Riccomini
Hey all, Trying to respond in order. If one wants finer-grained control over what the output topic would be like, wouldn't it make sense to use a CREATE TABLE AS query statement? Yes. CREATE TABLE AS could be used to define all three things that I mentioned (partition key, partition count, and

Re: Streaming SQL - object models, ASTs and algebras

2015-01-30 Thread Milinda Pathirage
Hi Chris, As a temporary solution we can use a JSON based Calcite schema feature. https://github.com/julianhyde/incubator-calcite/blob/master/example/csv/src/test/resources/model.json We can implement our own Schema factory and use Calcite JSON model with additional attributes as required.

Re: Streaming SQL - object models, ASTs and algebras

2015-01-30 Thread Julian Hyde
This isn't a huge deal for me. Since there isn't a metadata repository it makes sense to include that kind of metadata in the query. (The database world is all about late schema and schema on read these days, so there are plenty of precedents in modern SQL implementations.) We could allow

Re: Streaming SQL - object models, ASTs and algebras

2015-01-29 Thread Yi Pan
Hi, Julian, Thanks for sharing your idea! It is interesting and well organized. Let me try to summarize the main difference between yours and the current proposal are: - removing the '[]' used to define the window specification, using OVER on the stream/table instead - join/select output can be

Re: Streaming SQL - object models, ASTs and algebras

2015-01-29 Thread Yi Pan
One more, Julian, do you mind if I post your proposed SQL model to SAMZA-390? That way, more ppl can view it and we should continue discussion there. Thanks! -Yi On Thu, Jan 29, 2015 at 4:27 PM, Julian Hyde jul...@hydromatic.net wrote: The validation logic is extensible within Calcite (for

Re: Streaming SQL - object models, ASTs and algebras

2015-01-28 Thread Julian Hyde
Consider this simple query (I'll express in 3 equivalent ways): * select stream * from Orders where state = 'CA' (in streaming SQL) * istream [ select * from Orders where state = 'CA' ] (in CQL) * delta(filter(state = 'CA', scan(Orders))) (in logical algebra) In CQL there are no named streams,

Streaming SQL - object models, ASTs and algebras

2015-01-27 Thread Julian Hyde
Hi all, This is my first post to the Samza list. I heard from Chris and Jay that you guys were looking into putting a SQL interface on Samza, so I thought I'd take a look. My background is in the SQL world, most recently with Apache Calcite, (although I have quite a lot of experience with