@Tyler, there's no big change in the previous design doc, I added some details in chapter 'Part 2. DML( [INSERT] SELECT )' , describing steps to process a query, feel free to leave a comment.
Come through your doc of 'EMIT', it's awesome from my perspective. I've some tests on GroupBy with default triggers/allowed_lateness now. EMIT syntax can be added to fill the gap. On Thu, Apr 6, 2017 at 1:04 PM, Tyler Akidau <taki...@apache.org> wrote: > I'm very excited by this development as well, thanks for continuing to push > this forward, Mingmin. :-) > > I noticed you'd made some changes to your design doc > <https://docs.google.com/document/d/1Uc5xYTpO9qsLXtT38OfuoqSLimH_ > 0a1Bz5BsCROMzCU/edit>. > Is it ready for another review? How reflective is it currently of the work > that going into the feature branch? > > In parallel, I'd also like to continue helping push forward the definition > of unified model semantics for SQL so we can get Calcite to a point where > it supports the full Beam model. I added a comment > <https://issues.apache.org/jira/browse/BEAM-301?focusedCommentId=15959621&; > page=com.atlassian.jira.plugin.system.issuetabpanels: > comment-tabpanel#comment-15959621> > on the JIRA suggesting I create a doc with a specification proposal for > EMIT (and any other necessary semantic changes) that we can then iterate on > in public with the Calcite folks. I already have most of the content > written (and there's a significant amount of background needed to justify > some aspects of the proposal), so it'll mostly be a matter of pulling it > all together into something coherent. Does that sound reasonable to > everyone? > > -Tyler > > > On Thu, Apr 6, 2017 at 10:26 AM Kenneth Knowles <k...@google.com.invalid> > wrote: > > > Very cool! I'm really excited about this integration. > > > > On Thu, Apr 6, 2017 at 9:39 AM, Jean-Baptiste Onofré <j...@nanthrax.net> > > wrote: > > > > > Hi, > > > > > > Mingmin and I prepared a new branch to have the SQL DSL in dsls/sql > > > location. > > > > > > Any help is welcome ! > > > > > > Thanks, > > > Regards > > > JB > > > > > > > > > On 04/06/2017 06:36 PM, Mingmin Xu wrote: > > > > > >> @Tarush, you're very welcome to join the effort. > > >> > > >> On Thu, Apr 6, 2017 at 7:22 AM, tarush grover < > tarushappt...@gmail.com> > > >> wrote: > > >> > > >> Hi, > > >>> > > >>> Can I be also part of this feature development. > > >>> > > >>> Regards, > > >>> Tarush Grover > > >>> > > >>> On Thu, Apr 6, 2017 at 3:17 AM, Ted Yu <yuzhih...@gmail.com> wrote: > > >>> > > >>> I compiled BEAM-301 branch with calcite 1.12 - passed. > > >>>> > > >>>> Julian tries to not break existing things, but he will if there's a > > >>>> > > >>> reason > > >>> > > >>>> to do so :-) > > >>>> > > >>>> On Wed, Apr 5, 2017 at 2:36 PM, Mingmin Xu <mingm...@gmail.com> > > wrote: > > >>>> > > >>>> @Ted, thanks for the note. I intend to stick with one version, Beam > > >>>>> > > >>>> 0.6.0 > > >>> > > >>>> and Calcite 1.11 so far, unless impacted by API change. Before it's > > >>>>> > > >>>> merged > > >>>> > > >>>>> back to master, will upgrade to the latest version. > > >>>>> > > >>>>> On Wed, Apr 5, 2017 at 2:14 PM, Ted Yu <yuzhih...@gmail.com> > wrote: > > >>>>> > > >>>>> Working in feature branch is good - you may want to periodically > sync > > >>>>>> > > >>>>> up > > >>>> > > >>>>> with master. > > >>>>>> > > >>>>>> I noticed that you are using 1.11.0 of calcite. > > >>>>>> 1.12 is out, FYI > > >>>>>> > > >>>>>> On Wed, Apr 5, 2017 at 2:05 PM, Mingmin Xu <mingm...@gmail.com> > > >>>>>> > > >>>>> wrote: > > >>> > > >>>> > > >>>>>> Hi all, > > >>>>>>> > > >>>>>>> I'm working on https://issues.apache.org/ > jira/browse/BEAM-301(Add > > >>>>>>> > > >>>>>> a > > >>> > > >>>> Beam > > >>>>> > > >>>>>> SQL DSL). The skeleton is already in > > >>>>>>> https://github.com/XuMingmin/beam/tree/BEAM-301, using Java SDK > in > > >>>>>>> > > >>>>>> the > > >>>> > > >>>>> back-end. The goal is to provide a SQL interface over Beam, based > > >>>>>>> > > >>>>>> on > > >>> > > >>>> Calcite, including: > > >>>>>>> 1). a translator to create Beam pipeline from SQL, > > >>>>>>> (SELECT/INSERT/FILTER/GROUP-BY/JOIN/...); > > >>>>>>> 2). an interactive client to submit queries; (All-SQL mode) > > >>>>>>> 3). a SQL API which reduce the work to create a Pipeline; > (Semi-SQL > > >>>>>>> > > >>>>>> mode) > > >>>>> > > >>>>>> > > >>>>>>> As we see many folks are interested in this feature, would like > to > > >>>>>>> > > >>>>>> create a > > >>>>>> > > >>>>>>> feature branch to have more involvement. > > >>>>>>> Looking for comments and feedback. > > >>>>>>> > > >>>>>>> Thanks! > > >>>>>>> ---- > > >>>>>>> Mingmin > > >>>>>>> > > >>>>>>> > > >>>>>> > > >>>>> > > >>>>> > > >>>>> -- > > >>>>> ---- > > >>>>> Mingmin > > >>>>> > > >>>>> > > >>>> > > >>> > > >> > > >> > > >> > > > -- > > > Jean-Baptiste Onofré > > > jbono...@apache.org > > > http://blog.nanthrax.net > > > Talend - http://www.talend.com > > > > > > -- ---- Mingmin
