Thanks @JB, will come out the initial PR soon. On Sun, Apr 9, 2017 at 12:28 PM, Jean-Baptiste Onofré <j...@nanthrax.net> wrote:
> As discussed, I created the DSL_SQL branch with the skeleton. Mingmin is > rebasing on this branch to submit the PR. > > Regards > JB > > > On 04/09/2017 08:02 PM, Mingmin Xu wrote: > >> State is not touched yet, welcome to add it. >> >> On Sun, Apr 9, 2017 at 2:40 AM, 陈竞 <cj.mag...@gmail.com> wrote: >> >> how will this sql support state both in streaming and batch mode >>> >>> 2017-04-07 4:54 GMT+08:00 Mingmin Xu <mingm...@gmail.com>: >>> >>> @Tyler, there's no big change in the previous design doc, I added some >>>> details in chapter 'Part 2. DML( [INSERT] SELECT )' , describing steps >>>> to >>>> process a query, feel free to leave a comment. >>>> >>>> Come through your doc of 'EMIT', it's awesome from my perspective. I've >>>> some tests on GroupBy with default triggers/allowed_lateness now. EMIT >>>> syntax can be added to fill the gap. >>>> >>>> On Thu, Apr 6, 2017 at 1:04 PM, Tyler Akidau <taki...@apache.org> >>>> wrote: >>>> >>>> I'm very excited by this development as well, thanks for continuing to >>>>> >>>> push >>>> >>>>> this forward, Mingmin. :-) >>>>> >>>>> I noticed you'd made some changes to your design doc >>>>> <https://docs.google.com/document/d/1Uc5xYTpO9qsLXtT38OfuoqSLimH_ >>>>> 0a1Bz5BsCROMzCU/edit>. >>>>> Is it ready for another review? How reflective is it currently of the >>>>> >>>> work >>>> >>>>> that going into the feature branch? >>>>> >>>>> In parallel, I'd also like to continue helping push forward the >>>>> >>>> definition >>>> >>>>> of unified model semantics for SQL so we can get Calcite to a point >>>>> >>>> where >>> >>>> it supports the full Beam model. I added a comment >>>>> <https://issues.apache.org/jira/browse/BEAM-301? >>>>> >>>> focusedCommentId=15959621& >>>> >>>>> page=com.atlassian.jira.plugin.system.issuetabpanels: >>>>> comment-tabpanel#comment-15959621> >>>>> on the JIRA suggesting I create a doc with a specification proposal for >>>>> EMIT (and any other necessary semantic changes) that we can then >>>>> >>>> iterate >>> >>>> on >>>> >>>>> in public with the Calcite folks. I already have most of the content >>>>> written (and there's a significant amount of background needed to >>>>> >>>> justify >>> >>>> some aspects of the proposal), so it'll mostly be a matter of pulling >>>>> >>>> it >>> >>>> all together into something coherent. Does that sound reasonable to >>>>> everyone? >>>>> >>>>> -Tyler >>>>> >>>>> >>>>> On Thu, Apr 6, 2017 at 10:26 AM Kenneth Knowles <k...@google.com.invalid >>>>> >>>> >>>> wrote: >>>>> >>>>> Very cool! I'm really excited about this integration. >>>>>> >>>>>> On Thu, Apr 6, 2017 at 9:39 AM, Jean-Baptiste Onofré < >>>>>> >>>>> j...@nanthrax.net> >>> >>>> wrote: >>>>>> >>>>>> Hi, >>>>>>> >>>>>>> Mingmin and I prepared a new branch to have the SQL DSL in dsls/sql >>>>>>> location. >>>>>>> >>>>>>> Any help is welcome ! >>>>>>> >>>>>>> Thanks, >>>>>>> Regards >>>>>>> JB >>>>>>> >>>>>>> >>>>>>> On 04/06/2017 06:36 PM, Mingmin Xu wrote: >>>>>>> >>>>>>> @Tarush, you're very welcome to join the effort. >>>>>>>> >>>>>>>> On Thu, Apr 6, 2017 at 7:22 AM, tarush grover < >>>>>>>> >>>>>>> tarushappt...@gmail.com> >>>>> >>>>>> wrote: >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>>> >>>>>>>>> Can I be also part of this feature development. >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> Tarush Grover >>>>>>>>> >>>>>>>>> On Thu, Apr 6, 2017 at 3:17 AM, Ted Yu <yuzhih...@gmail.com> >>>>>>>>> >>>>>>>> wrote: >>>> >>>>> >>>>>>>>> I compiled BEAM-301 branch with calcite 1.12 - passed. >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Julian tries to not break existing things, but he will if >>>>>>>>>> >>>>>>>>> there's >>> >>>> a >>>> >>>>> >>>>>>>>>> reason >>>>>>>>> >>>>>>>>> to do so :-) >>>>>>>>>> >>>>>>>>>> On Wed, Apr 5, 2017 at 2:36 PM, Mingmin Xu <mingm...@gmail.com> >>>>>>>>>> >>>>>>>>> wrote: >>>>>> >>>>>>> >>>>>>>>>> @Ted, thanks for the note. I intend to stick with one version, >>>>>>>>>> >>>>>>>>> Beam >>>> >>>>> >>>>>>>>>>> 0.6.0 >>>>>>>>>> >>>>>>>>> >>>>>>>>> and Calcite 1.11 so far, unless impacted by API change. Before >>>>>>>>>> >>>>>>>>> it's >>>> >>>>> >>>>>>>>>>> merged >>>>>>>>>> >>>>>>>>>> back to master, will upgrade to the latest version. >>>>>>>>>>> >>>>>>>>>>> On Wed, Apr 5, 2017 at 2:14 PM, Ted Yu <yuzhih...@gmail.com> >>>>>>>>>>> >>>>>>>>>> wrote: >>>>> >>>>>> >>>>>>>>>>> Working in feature branch is good - you may want to >>>>>>>>>>> >>>>>>>>>> periodically >>> >>>> sync >>>>> >>>>>> >>>>>>>>>>>> up >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> with master. >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I noticed that you are using 1.11.0 of calcite. >>>>>>>>>>>> 1.12 is out, FYI >>>>>>>>>>>> >>>>>>>>>>>> On Wed, Apr 5, 2017 at 2:05 PM, Mingmin Xu < >>>>>>>>>>>> >>>>>>>>>>> mingm...@gmail.com> >>> >>>> >>>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>>> Hi all, >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> I'm working on https://issues.apache.org/ >>>>>>>>>>>>> >>>>>>>>>>>> jira/browse/BEAM-301(Add >>>>> >>>>>> >>>>>>>>>>>>> a >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> Beam >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> SQL DSL). The skeleton is already in >>>>>>>>>>>> >>>>>>>>>>>>> https://github.com/XuMingmin/beam/tree/BEAM-301, using Java >>>>>>>>>>>>> >>>>>>>>>>>> SDK >>>> >>>>> in >>>>> >>>>>> >>>>>>>>>>>>> the >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> back-end. The goal is to provide a SQL interface over Beam, >>>>>>>>>>> >>>>>>>>>> based >>> >>>> >>>>>>>>>>>>> on >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> Calcite, including: >>>>>>>>>> >>>>>>>>>>> 1). a translator to create Beam pipeline from SQL, >>>>>>>>>>>>> (SELECT/INSERT/FILTER/GROUP-BY/JOIN/...); >>>>>>>>>>>>> 2). an interactive client to submit queries; (All-SQL mode) >>>>>>>>>>>>> 3). a SQL API which reduce the work to create a Pipeline; >>>>>>>>>>>>> >>>>>>>>>>>> (Semi-SQL >>>>> >>>>>> >>>>>>>>>>>>> mode) >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> As we see many folks are interested in this feature, would >>>>>>>>>>>>> >>>>>>>>>>>> like >>> >>>> to >>>>> >>>>>> >>>>>>>>>>>>> create a >>>>>>>>>>>> >>>>>>>>>>>> feature branch to have more involvement. >>>>>>>>>>>>> Looking for comments and feedback. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks! >>>>>>>>>>>>> ---- >>>>>>>>>>>>> Mingmin >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> ---- >>>>>>>>>>> Mingmin >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>> Jean-Baptiste Onofré >>>>>>> jbono...@apache.org >>>>>>> http://blog.nanthrax.net >>>>>>> Talend - http://www.talend.com >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>>> >>>> -- >>>> ---- >>>> Mingmin >>>> >>>> >>> >>> >>> -- >>> 陈竞，中科院计算技术研究所，高性能计算机中心 >>> Jing Chen HPCC.ICT.AC China >>> >>> >> >> >> > -- > Jean-Baptiste Onofré > jbono...@apache.org > http://blog.nanthrax.net > Talend - http://www.talend.com > -- ---- Mingmin
