[
https://issues.apache.org/jira/browse/STORM-1040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15034472#comment-15034472
]
Julian Hyde commented on STORM-1040:
------------------------------------
At a high level, it looks good. But I have some questions about the design and
direction.
I have a concern that you are operating on Calcite logical algebra, rather than
having created a Storm physical algebra. It shouldn't matter in the short term,
but you will probably want to do query optimization in future. In a physical
algebra you would have StormFilter extends Filter, etc. I am not referring to
deep optimization, but simple rewrites like pushing filters have very large
benefits and are almost essential to do SQL efficiently.
Have you published some SQL examples? And an example how to use SQL from Java
code?
It seems that you are skipping over the Delta operator. All will be fine for
simple Scan-Filter-Project queries, I think this means you will run into
problems when you add Aggregate and definitely when you add Join.
SQL support seems under-tested right now. A few areas of exposure are the SQL
built-in functions, null semantics, and the combinations of all the various
relational operators. Several of Calcite's tests (e.g. SqlOperatorBaseTest) are
structured like TCKs and could be incorporated into Storm's test suite to give
huge test coverage at little effort. What do you think of co-developing TCKs
for streaming SQL semantics?
Is there anything that should be contributed back to Calcite?
PostOrderRelNodeVisitor looked like a candidate.
I think [~wheat9] or maybe someone else from the Storm community should join
the Calcite dev list and actively participate. Calcite's API has a large
surface area and things naturally get broken. For projects that depend heavily
on Calcite (which now includes Storm), the best process seems to be for those
projects to proactively verify that nothing has been broken before our first
RC. Staying up to date allows us to fix bugs you encounter in a timely fashion.
(Before you ask: we are a small dev team and don't like the overhead of
maintenance branches!)
> SQL support for Storm
> ---------------------
>
> Key: STORM-1040
> URL: https://issues.apache.org/jira/browse/STORM-1040
> Project: Apache Storm
> Issue Type: Epic
> Components: storm-sql
> Reporter: Haohui Mai
> Assignee: Haohui Mai
>
> SQL is one of the more popular languages in business intelligence. Instead of
> writing Storm topology directly, having supports to compile SQL down to
> topology will accelerate the development and deployment of real-time
> streaming solutions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
