Thanks @Tyler, @JB. The first question for me is how it's used, a SQL prompt like StormSQL, or the DSL API in Flink. In my project, it goes with the 1st one as it's targeted on self-service for analyst. looks odd for me to mix Java code and SQL string.
Regarding to the scope, that's a good point to find a proper subset at the first stage. The items I listed may be too much for phase 1, especially GROUP-BY. More details would be added. Btw, this doc mostly talks about streaming, there're already so many options to run a batch with SQL. Mingmin On Tue, Feb 28, 2017 at 10:01 AM, Neelesh Salian <[email protected]> wrote: > Hi Mingmin, > > Thanks for writing it up. > I haven't had the chance to start work on it. > Happy to help on tasks for building it. > > Feel free to assign BEAN-301 to yourself. > > > > On Tue, Feb 28, 2017 at 9:38 AM, Tyler Akidau <[email protected]> > wrote: > > > Hi Mingmin, > > > > Thanks for your interest in helping out on this task, and for your > initial > > proposal. I'm also very happy to work with you on this, and excited to > see > > some progress made here. Added a few more comments on the doc, but will > > summarize them below as well. > > > > As far as the DSL point goes, I agree with JB that any sort of interface > to > > Beam that uses SQL will be creating a DSL. Having the initial interface > be > > an interactive SQL prompt is a perfectly valid approach, but at the end > of > > the day, theres' still a DSL under the covers. As such, there are a lot > of > > questions that will need to be addressed in designing such a DSL (and the > > Jira lists some resources discussing those already). > > > > That said, it's possible to make progress on a Beam DSL without > addressing > > them all (e.g., by tackling only a small subset of functionality first, > > such as project and filter). But the current phases as listed in the doc > > will require addressing some of the big ones. > > > > So a good first step might be trying to scope the proposal to have a more > > modest initial set of functionality, or else providing more detail on how > > you propose to address the issues that will come up with various features > > currently listed in phase 1, particularly grouping w/ streams. > > > > -Tyler > > > > On Mon, Feb 27, 2017 at 10:44 PM Jean-Baptiste Onofré <[email protected]> > > wrote: > > > > > Hi Mingmin, > > > > > > The idea is actual both: > > > > > > 1. an interactive SQL prompt where we can express pipeline directly > > > using SQL. > > > 2. a SQL DSL to describe a pipeline in SQL and create the corresponding > > > Java code under the hood. > > > > > > I provided couple of comments on the doc. Ready and happy to help you > on > > > this (as I created the Jira ;)). > > > > > > Regards > > > JB > > > > > > On 02/27/2017 10:33 PM, Mingmin Xu wrote: > > > > Hello all, > > > > > > > > Would like to pop up this task, to see any interest to move it > forward. > > > > > > > > I've a project to run SQL queries with an interactive interface, and > > > would > > > > like to share my ideas. A draft doc is available to describe how it > > works > > > > with Calcite. --A little different from BEAM-301, that I choose a CLI > > > > interactive way, not SQL DSL. > > > > > > > > Doc link: > > > > > > > https://docs.google.com/document/d/1Uc5xYTpO9qsLXtT38OfuoqSLimH_ > > 0a1Bz5BsCROMzCU/edit?usp=sharing > > > > > > > > > > -- > > > Jean-Baptiste Onofré > > > [email protected] > > > http://blog.nanthrax.net > > > Talend - http://www.talend.com > > > > > > > > > -- > Regards, > Neelesh S. Salian > -- ---- Mingmin
