@Jakob, totally agree. I think the semantics of the SQL module won't
change. RTC, experimental, more lax on tests/docs. When we commit into
master, we can enforce much stronger test/doc requirements. That's what I'm
thinking.
On Thu, Feb 12, 2015 at 9:26 AM, Jakob Homan <jgho...@gmail.com> wrote:
> +1, assuming we're keeping RTC on the branch (, Jakob said pedantically).
>
> On 12 February 2015 at 08:25, Milinda Pathirage <mpath...@umail.iu.edu>
> wrote:
> > I am +1 for this approach.
> >
> > Milinda
> >
> > On Thu, Feb 12, 2015 at 11:22 AM, Chris Riccomini <criccom...@apache.org
> >
> > wrote:
> >
> >> Hey Julian,
> >>
> >> > The rule of thumb I’ve learned integrating Calcite into Hive is that
> only
> >> a branch should depend on snapshot versions of another component
> >>
> >> This is very practical. I hadn't considered the Calcite dependency issue
> >> when I pushed for merging into master rather than a branch.
> >>
> >> Based on this, it seems like the best thing to do is:
> >>
> >> 1) Work off of Calcite SNAPSHOT Maven publication.
> >> 2) Create a samza-sql branch.
> >> 3) Migrate all existing samza-sql commits from master into the samza-sql
> >> branch.
> >>
> >> Sorry for the churn on this all. I hadn't considered the binary
> dependency
> >> issue. Is everyone OK with this?
> >>
> >> Cheers,
> >> Chris
> >>
> >> On Wed, Feb 11, 2015 at 10:51 AM, Julian Hyde <jul...@hydromatic.net>
> >> wrote:
> >>
> >> > This seems more like a branch than a classifier. Calcite is
> developing in
> >> > a branch, and would produce snapshots from that branch. The rule of
> thumb
> >> > I’ve learned integrating Calcite into Hive is that only a branch
> should
> >> > depend on snapshot versions of another component. (Hive broke this
> rule
> >> and
> >> > got into a sticky situation where their trunk depended on a Calcite
> >> > snapshot and they couldn’t release because Calcite was stuck in
> >> pre-release
> >> > vote limbo.)
> >> >
> >> > I’m going to try to produce maven artifacts with {groupId:
> >> > “org.apache.calcite”, version: "1.1.0-stream-incubating-SNAPSHOT”,
> >> > artifactId: “calcite-core”, “calcite-avatica”, etc.} and push them to
> >> > Apache nexus. I do recommend that you use it on a branch.
> >> >
> >> > However, I don’t think there would be a problem getting the
> functionality
> >> > into calcite’s master branch in time for Calcite’s next release, in
> >> about 4
> >> > weeks. (We commit to releasing approximately monthly.) The
> functionality
> >> > and APIs would be marked “experimental, subject to change” but at
> least
> >> you
> >> > would be able to merge your work to master at that point. Doubtless
> you
> >> > would need new features & bug fixes in Calcite after that, and so work
> >> that
> >> > depended on that would have to stay in a branch until Calcite was
> >> released.
> >> >
> >> > Julian
> >> >
> >> > On Feb 11, 2015, at 10:34 AM, Felix GV <fville...@linkedin.com.INVALID
> >
> >> > wrote:
> >> >
> >> > > I guess one concern with Calcite SNAPSHOT releases is that they
> imply
> >> > they'll be included in the next release, which may not be the case if
> >> > they're still considered very experimental.
> >> > >
> >> > > Alternatively, perhaps Julian can create a new version name with a
> >> > classifier for streaming.
> >> > >
> >> > >
> >> >
> >>
> http://maven.apache.org/plugins/maven-deploy-plugin/examples/deploying-with-classifiers.html
> >> > >
> >> > > Then he could release SNAPSHOT artifacts for both the main/master
> code
> >> > base as well as the streaming branch, no matter how experimental it
> is...
> >> > >
> >> > > --
> >> > >
> >> > > Felix GV
> >> > > Data Infrastructure Engineer
> >> > > Distributed Data Systems
> >> > > LinkedIn
> >> > >
> >> > > f...@linkedin.com
> >> > > linkedin.com/in/felixgv
> >> > >
> >> > > ________________________________________
> >> > > From: Chris Riccomini [criccom...@apache.org]
> >> > > Sent: Wednesday, February 11, 2015 10:23 AM
> >> > > To: Chris Riccomini
> >> > > Cc: dev@samza.apache.org
> >> > > Subject: Re: [DISCUSS] SQL workflow
> >> > >
> >> > > Hey Julian,
> >> > >
> >> > > It looks like Calcite is at least setup to publish to Maven:
> >> > >
> >> > > https://repository.apache.org/#nexus-search;quick~calcite
> >> > >
> >> > > Julian, can you publish SNAPSHOT releases with your streaming
> changes
> >> to
> >> > it?
> >> > >
> >> > > Thanks!
> >> > > Chris
> >> > >
> >> > > On Wed, Feb 11, 2015 at 10:08 AM, Chris Riccomini <
> >> criccom...@apache.org
> >> > >
> >> > > wrote:
> >> > >
> >> > >> Hey Milinda,
> >> > >>
> >> > >> I've just committed SAMZA-484, so you should be able to re-base and
> >> get
> >> > >> all the latest code.
> >> > >>
> >> > >>> But we need to add Calcite as a dependency and Calcite streaming
> >> > >> support is only in Julian's branch
> >> > >>
> >> > >> We'll have to have Julian publish a release to Apache. This can be
> a
> >> > >> SNAPSHOT to Apache's SNAPSHOT repo, or a real Calcite release. We
> >> can't
> >> > >> check in binaries to the repo (licensing, release complexity, and
> just
> >> > >> generally a bad idea). We'll have to coordinate with him.
> >> > >>
> >> > >> Cheers,
> >> > >> Chris
> >> > >>
> >> > >> On Wed, Feb 11, 2015 at 9:43 AM, Milinda Pathirage <
> >> > mpath...@umail.iu.edu>
> >> > >> wrote:
> >> > >>
> >> > >>> No, SAMZA-483 doesn't depend on SAMZA-484. But we need to add
> Calcite
> >> > as a
> >> > >>> dependency and Calcite streaming support is only in Julian's
> branch (
> >> > >>> https://github.com/julianhyde/incubator-calcite/tree/chi). What
> can
> >> > we do
> >> > >>> about that?
> >> > >>>
> >> > >>> Milinda
> >> > >>>
> >> > >>> On Wed, Feb 11, 2015 at 12:30 PM, Chris Riccomini <
> >> > criccom...@apache.org>
> >> > >>> wrote:
> >> > >>>
> >> > >>>> Hey Milinda,
> >> > >>>>
> >> > >>>> That'd be great. Do you have any dependency on SAMZA-484? I
> haven't
> >> > had
> >> > >>> a
> >> > >>>> chance to commit that one yet.
> >> > >>>>
> >> > >>>> Cheers,
> >> > >>>> Chris
> >> > >>>>
> >> > >>>> On Wed, Feb 11, 2015 at 9:23 AM, Milinda Pathirage <
> >> > >>> mpath...@umail.iu.edu>
> >> > >>>> wrote:
> >> > >>>>
> >> > >>>>> /need/want
> >> > >>>>>
> >> > >>>>> Milinda
> >> > >>>>>
> >> > >>>>> On Wed, Feb 11, 2015 at 12:22 PM, Milinda Pathirage <
> >> > >>>> mpath...@umail.iu.edu
> >> > >>>>>>
> >> > >>>>> wrote:
> >> > >>>>>
> >> > >>>>>> Hi Chris,
> >> > >>>>>>
> >> > >>>>>> Do you need me to create the SAMZA-483 patch against latest
> master
> >> > >>> with
> >> > >>>>>> SAMZA-482 patch? I think that will make it easier to review the
> >> > >>> patch?
> >> > >>>>>>
> >> > >>>>>> Thanks
> >> > >>>>>> Milinda
> >> > >>>>>>
> >> > >>>>>> On Mon, Feb 9, 2015 at 11:39 AM, Chris Riccomini <
> >> > >>>> criccom...@apache.org>
> >> > >>>>>> wrote:
> >> > >>>>>>
> >> > >>>>>>> Cool. Looks like we've got consensus. I'll move forward with
> RTC
> >> on
> >> > >>>> some
> >> > >>>>>>> of
> >> > >>>>>>> the early SAMZA-390 tickets. (SAMZA-482, SAMZA-483, SAMZA-484)
> >> > >>>>>>>
> >> > >>>>>>> On Mon, Feb 9, 2015 at 8:35 AM, Yan Fang <
> yanfang...@gmail.com>
> >> > >>>> wrote:
> >> > >>>>>>>
> >> > >>>>>>>> +1 on this.
> >> > >>>>>>>>
> >> > >>>>>>>> Fang, Yan
> >> > >>>>>>>> yanfang...@gmail.com
> >> > >>>>>>>> +1 (206) 849-4108
> >> > >>>>>>>>
> >> > >>>>>>>> On Fri, Feb 6, 2015 at 4:38 PM, Chris Riccomini <
> >> > >>>>> criccom...@apache.org>
> >> > >>>>>>>> wrote:
> >> > >>>>>>>>
> >> > >>>>>>>>> Hey all,
> >> > >>>>>>>>>
> >> > >>>>>>>>> Are we +1 on this? I think Jakob was the only one who was
> >> > >>> curious
> >> > >>>>>>> about
> >> > >>>>>>>> it.
> >> > >>>>>>>>>
> >> > >>>>>>>>> Cheers,
> >> > >>>>>>>>> Chris
> >> > >>>>>>>>>
> >> > >>>>>>>>> On Thu, Feb 5, 2015 at 1:22 PM, Yi Pan <nickpa...@gmail.com
> >
> >> > >>>> wrote:
> >> > >>>>>>>>>
> >> > >>>>>>>>>> Hi, Jakob,
> >> > >>>>>>>>>>
> >> > >>>>>>>>>>
> >> > >>>>>>>>>>>> Eh? Not sure what this means...
> >> > >>>>>>>>>>>
> >> > >>>>>>>>>>> I mean SAMZA-484 depends on SAMZA-482, and neither are
> >> > >>>>> committed.
> >> > >>>>>>> So
> >> > >>>>>>>>>> Navina
> >> > >>>>>>>>>>> is having to post Yi's patch, as well as her own, on the
> >> > >>> JIRA.
> >> > >>>>> It
> >> > >>>>>>>> makes
> >> > >>>>>>>>>> it
> >> > >>>>>>>>>>> really hard to do code reviews because you can't tell
> >> > >>> whether
> >> > >>>> Yi
> >> > >>>>>>> made
> >> > >>>>>>>>> the
> >> > >>>>>>>>>>> changes or Navina did.
> >> > >>>>>>>>>>>
> >> > >>>>>>>>>>>
> >> > >>>>>>>>>> Just to add to the point. It is also difficult to always
> see
> >> > >>> a
> >> > >>>>> long
> >> > >>>>>>>> list
> >> > >>>>>>>>> of
> >> > >>>>>>>>>> changed files if the RB request is always based on the
> >> > >>> master.
> >> > >>>> It
> >> > >>>>> is
> >> > >>>>>>>>>> possible to have RB request based on another RB request (I
> >> > >>> have
> >> > >>>>>>> tried
> >> > >>>>>>>> it
> >> > >>>>>>>>>> before). But what happens if the base RB request is
> >> > >>>>>>>> cancelled/discarded?
> >> > >>>>>>>>> RB
> >> > >>>>>>>>>> is not designed to track the revision changes in a
> dependency
> >> > >>>>> chain.
> >> > >>>>>>>>>>
> >> > >>>>>>>>>>
> >> > >>>>>>>>>>
> >> > >>>>>>>>>>> Cheers,
> >> > >>>>>>>>>>> Chris
> >> > >>>>>>>>>>>
> >> > >>>>>>>>>>> On Thu, Feb 5, 2015 at 12:16 PM, Jakob Homan <
> >> > >>>> jgho...@gmail.com
> >> > >>>>>>
> >> > >>>>>>>>> wrote:
> >> > >>>>>>>>>>>
> >> > >>>>>>>>>>>>> I want to avoid branches,
> >> > >>>>>>>>>>>> Just curious, any reason for this?
> >> > >>>>>>>>>>>>
> >> > >>>>>>>>>>>>> and I also want to avoid revision control over JIRA
> >> > >>>>>>>>>>>> Eh? Not sure what this means...
> >> > >>>>>>>>>>>>
> >> > >>>>>>>>>>>> Thanks,
> >> > >>>>>>>>>>>> jg
> >> > >>>>>>>>>>>>
> >> > >>>>>>>>>>>> On 4 February 2015 at 17:11, Chris Riccomini <
> >> > >>>>>>>> criccom...@apache.org>
> >> > >>>>>>>>>>>> wrote:
> >> > >>>>>>>>>>>>> Hey all,
> >> > >>>>>>>>>>>>>
> >> > >>>>>>>>>>>>> @Jakob, yeah I was thinking we'll follow our normal
> >> > >>> flow.
> >> > >>>>>>> RTC. I
> >> > >>>>>>>>> just
> >> > >>>>>>>>>>>>> wanted to set expectation that the code committed
> >> > >>> might be
> >> > >>>>>>> not up
> >> > >>>>>>>>> to
> >> > >>>>>>>>>>> our
> >> > >>>>>>>>>>>>> normal quality initially (missing docs, no tests, etc).
> >> > >>>>> Until
> >> > >>>>>>> the
> >> > >>>>>>>>>>> quality
> >> > >>>>>>>>>>>>> is raised, we should think of this module as
> >> > >>> experimental.
> >> > >>>>>>>>>>>>>
> >> > >>>>>>>>>>>>> @Milinda, awesome! Thanks. :)
> >> > >>>>>>>>>>>>>
> >> > >>>>>>>>>>>>> Cheers,
> >> > >>>>>>>>>>>>> Chris
> >> > >>>>>>>>>>>>>
> >> > >>>>>>>>>>>>> On Wed, Feb 4, 2015 at 11:57 AM, Milinda Pathirage <
> >> > >>>>>>>>>>>> mpath...@umail.iu.edu>
> >> > >>>>>>>>>>>>> wrote:
> >> > >>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>> Hi Chris,
> >> > >>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>> Hope we no longer need the SQL API. I'll create a RB
> >> > >>> for
> >> > >>>>>>> Calcite
> >> > >>>>>>>>>>>>>> integration.
> >> > >>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>> Thanks
> >> > >>>>>>>>>>>>>> Milinda
> >> > >>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 1:31 PM, Chris Riccomini <
> >> > >>>>>>>>>>> criccom...@apache.org>
> >> > >>>>>>>>>>>>>> wrote:
> >> > >>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>>> I think so. There was some RB downtime, but it just
> >> > >>> got
> >> > >>>>>>> fixed.
> >> > >>>>>>>>> Yi,
> >> > >>>>>>>>>>>>>> Navina,
> >> > >>>>>>>>>>>>>>> Milinda, can you make sure your JIRAs have up to
> >> > >>> date
> >> > >>>>> RBs?
> >> > >>>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 10:24 AM, sriram <
> >> > >>>>>>> sriram....@gmail.com
> >> > >>>>>>>>>
> >> > >>>>>>>>>>> wrote:
> >> > >>>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>>>> Can we have updated RBs for all the three sub
> >> > >>> tasks
> >> > >>>>>>> before
> >> > >>>>>>>> we
> >> > >>>>>>>>>>>> commit?
> >> > >>>>>>>>>>>>>>> This
> >> > >>>>>>>>>>>>>>>> would help us to review even after we commit.
> >> > >>>>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini <
> >> > >>>>>>>>>>>>>> criccom...@apache.org>
> >> > >>>>>>>>>>>>>>>> wrote:
> >> > >>>>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>>>>> Hey all,
> >> > >>>>>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>>>>> Yi, Navina, and Milinda have been working on
> >> > >>>>> SAMZA-390
> >> > >>>>>>>>>>> sub-tickets
> >> > >>>>>>>>>>>>>>>> related
> >> > >>>>>>>>>>>>>>>>> to SQL operators. We're getting to the point
> >> > >>> where
> >> > >>>>> the
> >> > >>>>>>>>> amount
> >> > >>>>>>>>>> of
> >> > >>>>>>>>>>>> work
> >> > >>>>>>>>>>>>>>>>> floating around is quite large, and some tickets
> >> > >>>>> build
> >> > >>>>>>> off
> >> > >>>>>>>>> of
> >> > >>>>>>>>>>>> others.
> >> > >>>>>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>>>>> I'm proposing that we commit this work into a
> >> > >>>>> samza-sql
> >> > >>>>>>>>>>> submodule
> >> > >>>>>>>>>>>> on
> >> > >>>>>>>>>>>>>>>>> master, and treat this module as experimental. I
> >> > >>>> want
> >> > >>>>>>> to
> >> > >>>>>>>>> avoid
> >> > >>>>>>>>>>>>>>> branches,
> >> > >>>>>>>>>>>>>>>>> and I also want to avoid revision control over
> >> > >>>> JIRA.
> >> > >>>>>>> This
> >> > >>>>>>>>>> means
> >> > >>>>>>>>>>>> that
> >> > >>>>>>>>>>>>>>>> there
> >> > >>>>>>>>>>>>>>>>> will probably be a fair amount of commits/JIRAs
> >> > >>> on
> >> > >>>>> this
> >> > >>>>>>>>> module
> >> > >>>>>>>>>>> as
> >> > >>>>>>>>>>>> we
> >> > >>>>>>>>>>>>>>>>> iterate, but I think that's OK.
> >> > >>>>>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>>>>> Does this sound good to everyone?
> >> > >>>>>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>>>>> Cheers,
> >> > >>>>>>>>>>>>>>>>> Chris
> >> > >>>>>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>> --
> >> > >>>>>>>>>>>>>> Milinda Pathirage
> >> > >>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>> PhD Student | Research Assistant
> >> > >>>>>>>>>>>>>> School of Informatics and Computing | Data to Insight
> >> > >>>>> Center
> >> > >>>>>>>>>>>>>> Indiana University
> >> > >>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>> twitter: milindalakmal
> >> > >>>>>>>>>>>>>> skype: milinda.pathirage
> >> > >>>>>>>>>>>>>> blog: http://milinda.pathirage.org
> >> > >>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>
> >> > >>>>>>>>>>>
> >> > >>>>>>>>>>
> >> > >>>>>>>>>
> >> > >>>>>>>>
> >> > >>>>>>>
> >> > >>>>>>
> >> > >>>>>>
> >> > >>>>>>
> >> > >>>>>> --
> >> > >>>>>> Milinda Pathirage
> >> > >>>>>>
> >> > >>>>>> PhD Student | Research Assistant
> >> > >>>>>> School of Informatics and Computing | Data to Insight Center
> >> > >>>>>> Indiana University
> >> > >>>>>>
> >> > >>>>>> twitter: milindalakmal
> >> > >>>>>> skype: milinda.pathirage
> >> > >>>>>> blog: http://milinda.pathirage.org
> >> > >>>>>>
> >> > >>>>>
> >> > >>>>>
> >> > >>>>>
> >> > >>>>> --
> >> > >>>>> Milinda Pathirage
> >> > >>>>>
> >> > >>>>> PhD Student | Research Assistant
> >> > >>>>> School of Informatics and Computing | Data to Insight Center
> >> > >>>>> Indiana University
> >> > >>>>>
> >> > >>>>> twitter: milindalakmal
> >> > >>>>> skype: milinda.pathirage
> >> > >>>>> blog: http://milinda.pathirage.org
> >> > >>>>>
> >> > >>>>
> >> > >>>
> >> > >>>
> >> > >>>
> >> > >>> --
> >> > >>> Milinda Pathirage
> >> > >>>
> >> > >>> PhD Student | Research Assistant
> >> > >>> School of Informatics and Computing | Data to Insight Center
> >> > >>> Indiana University
> >> > >>>
> >> > >>> twitter: milindalakmal
> >> > >>> skype: milinda.pathirage
> >> > >>> blog: http://milinda.pathirage.org
> >> > >>>
> >> > >>
> >> > >>
> >> >
> >> >
> >>
> >
> >
> >
> > --
> > Milinda Pathirage
> >
> > PhD Student | Research Assistant
> > School of Informatics and Computing | Data to Insight Center
> > Indiana University
> >
> > twitter: milindalakmal
> > skype: milinda.pathirage
> > blog: http://milinda.pathirage.org
>
