I am +1 for this approach.
Milinda
On Thu, Feb 12, 2015 at 11:22 AM, Chris Riccomini <criccom...@apache.org>
wrote:
> Hey Julian,
>
> > The rule of thumb I’ve learned integrating Calcite into Hive is that only
> a branch should depend on snapshot versions of another component
>
> This is very practical. I hadn't considered the Calcite dependency issue
> when I pushed for merging into master rather than a branch.
>
> Based on this, it seems like the best thing to do is:
>
> 1) Work off of Calcite SNAPSHOT Maven publication.
> 2) Create a samza-sql branch.
> 3) Migrate all existing samza-sql commits from master into the samza-sql
> branch.
>
> Sorry for the churn on this all. I hadn't considered the binary dependency
> issue. Is everyone OK with this?
>
> Cheers,
> Chris
>
> On Wed, Feb 11, 2015 at 10:51 AM, Julian Hyde <jul...@hydromatic.net>
> wrote:
>
> > This seems more like a branch than a classifier. Calcite is developing in
> > a branch, and would produce snapshots from that branch. The rule of thumb
> > I’ve learned integrating Calcite into Hive is that only a branch should
> > depend on snapshot versions of another component. (Hive broke this rule
> and
> > got into a sticky situation where their trunk depended on a Calcite
> > snapshot and they couldn’t release because Calcite was stuck in
> pre-release
> > vote limbo.)
> >
> > I’m going to try to produce maven artifacts with {groupId:
> > “org.apache.calcite”, version: "1.1.0-stream-incubating-SNAPSHOT”,
> > artifactId: “calcite-core”, “calcite-avatica”, etc.} and push them to
> > Apache nexus. I do recommend that you use it on a branch.
> >
> > However, I don’t think there would be a problem getting the functionality
> > into calcite’s master branch in time for Calcite’s next release, in
> about 4
> > weeks. (We commit to releasing approximately monthly.) The functionality
> > and APIs would be marked “experimental, subject to change” but at least
> you
> > would be able to merge your work to master at that point. Doubtless you
> > would need new features & bug fixes in Calcite after that, and so work
> that
> > depended on that would have to stay in a branch until Calcite was
> released.
> >
> > Julian
> >
> > On Feb 11, 2015, at 10:34 AM, Felix GV <fville...@linkedin.com.INVALID>
> > wrote:
> >
> > > I guess one concern with Calcite SNAPSHOT releases is that they imply
> > they'll be included in the next release, which may not be the case if
> > they're still considered very experimental.
> > >
> > > Alternatively, perhaps Julian can create a new version name with a
> > classifier for streaming.
> > >
> > >
> >
> http://maven.apache.org/plugins/maven-deploy-plugin/examples/deploying-with-classifiers.html
> > >
> > > Then he could release SNAPSHOT artifacts for both the main/master code
> > base as well as the streaming branch, no matter how experimental it is...
> > >
> > > --
> > >
> > > Felix GV
> > > Data Infrastructure Engineer
> > > Distributed Data Systems
> > > LinkedIn
> > >
> > > f...@linkedin.com
> > > linkedin.com/in/felixgv
> > >
> > > ________________________________________
> > > From: Chris Riccomini [criccom...@apache.org]
> > > Sent: Wednesday, February 11, 2015 10:23 AM
> > > To: Chris Riccomini
> > > Cc: dev@samza.apache.org
> > > Subject: Re: [DISCUSS] SQL workflow
> > >
> > > Hey Julian,
> > >
> > > It looks like Calcite is at least setup to publish to Maven:
> > >
> > > https://repository.apache.org/#nexus-search;quick~calcite
> > >
> > > Julian, can you publish SNAPSHOT releases with your streaming changes
> to
> > it?
> > >
> > > Thanks!
> > > Chris
> > >
> > > On Wed, Feb 11, 2015 at 10:08 AM, Chris Riccomini <
> criccom...@apache.org
> > >
> > > wrote:
> > >
> > >> Hey Milinda,
> > >>
> > >> I've just committed SAMZA-484, so you should be able to re-base and
> get
> > >> all the latest code.
> > >>
> > >>> But we need to add Calcite as a dependency and Calcite streaming
> > >> support is only in Julian's branch
> > >>
> > >> We'll have to have Julian publish a release to Apache. This can be a
> > >> SNAPSHOT to Apache's SNAPSHOT repo, or a real Calcite release. We
> can't
> > >> check in binaries to the repo (licensing, release complexity, and just
> > >> generally a bad idea). We'll have to coordinate with him.
> > >>
> > >> Cheers,
> > >> Chris
> > >>
> > >> On Wed, Feb 11, 2015 at 9:43 AM, Milinda Pathirage <
> > mpath...@umail.iu.edu>
> > >> wrote:
> > >>
> > >>> No, SAMZA-483 doesn't depend on SAMZA-484. But we need to add Calcite
> > as a
> > >>> dependency and Calcite streaming support is only in Julian's branch (
> > >>> https://github.com/julianhyde/incubator-calcite/tree/chi). What can
> > we do
> > >>> about that?
> > >>>
> > >>> Milinda
> > >>>
> > >>> On Wed, Feb 11, 2015 at 12:30 PM, Chris Riccomini <
> > criccom...@apache.org>
> > >>> wrote:
> > >>>
> > >>>> Hey Milinda,
> > >>>>
> > >>>> That'd be great. Do you have any dependency on SAMZA-484? I haven't
> > had
> > >>> a
> > >>>> chance to commit that one yet.
> > >>>>
> > >>>> Cheers,
> > >>>> Chris
> > >>>>
> > >>>> On Wed, Feb 11, 2015 at 9:23 AM, Milinda Pathirage <
> > >>> mpath...@umail.iu.edu>
> > >>>> wrote:
> > >>>>
> > >>>>> /need/want
> > >>>>>
> > >>>>> Milinda
> > >>>>>
> > >>>>> On Wed, Feb 11, 2015 at 12:22 PM, Milinda Pathirage <
> > >>>> mpath...@umail.iu.edu
> > >>>>>>
> > >>>>> wrote:
> > >>>>>
> > >>>>>> Hi Chris,
> > >>>>>>
> > >>>>>> Do you need me to create the SAMZA-483 patch against latest master
> > >>> with
> > >>>>>> SAMZA-482 patch? I think that will make it easier to review the
> > >>> patch?
> > >>>>>>
> > >>>>>> Thanks
> > >>>>>> Milinda
> > >>>>>>
> > >>>>>> On Mon, Feb 9, 2015 at 11:39 AM, Chris Riccomini <
> > >>>> criccom...@apache.org>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>>> Cool. Looks like we've got consensus. I'll move forward with RTC
> on
> > >>>> some
> > >>>>>>> of
> > >>>>>>> the early SAMZA-390 tickets. (SAMZA-482, SAMZA-483, SAMZA-484)
> > >>>>>>>
> > >>>>>>> On Mon, Feb 9, 2015 at 8:35 AM, Yan Fang <yanfang...@gmail.com>
> > >>>> wrote:
> > >>>>>>>
> > >>>>>>>> +1 on this.
> > >>>>>>>>
> > >>>>>>>> Fang, Yan
> > >>>>>>>> yanfang...@gmail.com
> > >>>>>>>> +1 (206) 849-4108
> > >>>>>>>>
> > >>>>>>>> On Fri, Feb 6, 2015 at 4:38 PM, Chris Riccomini <
> > >>>>> criccom...@apache.org>
> > >>>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>>> Hey all,
> > >>>>>>>>>
> > >>>>>>>>> Are we +1 on this? I think Jakob was the only one who was
> > >>> curious
> > >>>>>>> about
> > >>>>>>>> it.
> > >>>>>>>>>
> > >>>>>>>>> Cheers,
> > >>>>>>>>> Chris
> > >>>>>>>>>
> > >>>>>>>>> On Thu, Feb 5, 2015 at 1:22 PM, Yi Pan <nickpa...@gmail.com>
> > >>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> Hi, Jakob,
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>>> Eh? Not sure what this means...
> > >>>>>>>>>>>
> > >>>>>>>>>>> I mean SAMZA-484 depends on SAMZA-482, and neither are
> > >>>>> committed.
> > >>>>>>> So
> > >>>>>>>>>> Navina
> > >>>>>>>>>>> is having to post Yi's patch, as well as her own, on the
> > >>> JIRA.
> > >>>>> It
> > >>>>>>>> makes
> > >>>>>>>>>> it
> > >>>>>>>>>>> really hard to do code reviews because you can't tell
> > >>> whether
> > >>>> Yi
> > >>>>>>> made
> > >>>>>>>>> the
> > >>>>>>>>>>> changes or Navina did.
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>> Just to add to the point. It is also difficult to always see
> > >>> a
> > >>>>> long
> > >>>>>>>> list
> > >>>>>>>>> of
> > >>>>>>>>>> changed files if the RB request is always based on the
> > >>> master.
> > >>>> It
> > >>>>> is
> > >>>>>>>>>> possible to have RB request based on another RB request (I
> > >>> have
> > >>>>>>> tried
> > >>>>>>>> it
> > >>>>>>>>>> before). But what happens if the base RB request is
> > >>>>>>>> cancelled/discarded?
> > >>>>>>>>> RB
> > >>>>>>>>>> is not designed to track the revision changes in a dependency
> > >>>>> chain.
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>> Cheers,
> > >>>>>>>>>>> Chris
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Thu, Feb 5, 2015 at 12:16 PM, Jakob Homan <
> > >>>> jgho...@gmail.com
> > >>>>>>
> > >>>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>>> I want to avoid branches,
> > >>>>>>>>>>>> Just curious, any reason for this?
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> and I also want to avoid revision control over JIRA
> > >>>>>>>>>>>> Eh? Not sure what this means...
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>> jg
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On 4 February 2015 at 17:11, Chris Riccomini <
> > >>>>>>>> criccom...@apache.org>
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>> Hey all,
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> @Jakob, yeah I was thinking we'll follow our normal
> > >>> flow.
> > >>>>>>> RTC. I
> > >>>>>>>>> just
> > >>>>>>>>>>>>> wanted to set expectation that the code committed
> > >>> might be
> > >>>>>>> not up
> > >>>>>>>>> to
> > >>>>>>>>>>> our
> > >>>>>>>>>>>>> normal quality initially (missing docs, no tests, etc).
> > >>>>> Until
> > >>>>>>> the
> > >>>>>>>>>>> quality
> > >>>>>>>>>>>>> is raised, we should think of this module as
> > >>> experimental.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> @Milinda, awesome! Thanks. :)
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Cheers,
> > >>>>>>>>>>>>> Chris
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On Wed, Feb 4, 2015 at 11:57 AM, Milinda Pathirage <
> > >>>>>>>>>>>> mpath...@umail.iu.edu>
> > >>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Hi Chris,
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Hope we no longer need the SQL API. I'll create a RB
> > >>> for
> > >>>>>>> Calcite
> > >>>>>>>>>>>>>> integration.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Thanks
> > >>>>>>>>>>>>>> Milinda
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 1:31 PM, Chris Riccomini <
> > >>>>>>>>>>> criccom...@apache.org>
> > >>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> I think so. There was some RB downtime, but it just
> > >>> got
> > >>>>>>> fixed.
> > >>>>>>>>> Yi,
> > >>>>>>>>>>>>>> Navina,
> > >>>>>>>>>>>>>>> Milinda, can you make sure your JIRAs have up to
> > >>> date
> > >>>>> RBs?
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 10:24 AM, sriram <
> > >>>>>>> sriram....@gmail.com
> > >>>>>>>>>
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Can we have updated RBs for all the three sub
> > >>> tasks
> > >>>>>>> before
> > >>>>>>>> we
> > >>>>>>>>>>>> commit?
> > >>>>>>>>>>>>>>> This
> > >>>>>>>>>>>>>>>> would help us to review even after we commit.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini <
> > >>>>>>>>>>>>>> criccom...@apache.org>
> > >>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Hey all,
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Yi, Navina, and Milinda have been working on
> > >>>>> SAMZA-390
> > >>>>>>>>>>> sub-tickets
> > >>>>>>>>>>>>>>>> related
> > >>>>>>>>>>>>>>>>> to SQL operators. We're getting to the point
> > >>> where
> > >>>>> the
> > >>>>>>>>> amount
> > >>>>>>>>>> of
> > >>>>>>>>>>>> work
> > >>>>>>>>>>>>>>>>> floating around is quite large, and some tickets
> > >>>>> build
> > >>>>>>> off
> > >>>>>>>>> of
> > >>>>>>>>>>>> others.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> I'm proposing that we commit this work into a
> > >>>>> samza-sql
> > >>>>>>>>>>> submodule
> > >>>>>>>>>>>> on
> > >>>>>>>>>>>>>>>>> master, and treat this module as experimental. I
> > >>>> want
> > >>>>>>> to
> > >>>>>>>>> avoid
> > >>>>>>>>>>>>>>> branches,
> > >>>>>>>>>>>>>>>>> and I also want to avoid revision control over
> > >>>> JIRA.
> > >>>>>>> This
> > >>>>>>>>>> means
> > >>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>> there
> > >>>>>>>>>>>>>>>>> will probably be a fair amount of commits/JIRAs
> > >>> on
> > >>>>> this
> > >>>>>>>>> module
> > >>>>>>>>>>> as
> > >>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>>> iterate, but I think that's OK.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Does this sound good to everyone?
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Cheers,
> > >>>>>>>>>>>>>>>>> Chris
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>> Milinda Pathirage
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> PhD Student | Research Assistant
> > >>>>>>>>>>>>>> School of Informatics and Computing | Data to Insight
> > >>>>> Center
> > >>>>>>>>>>>>>> Indiana University
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> twitter: milindalakmal
> > >>>>>>>>>>>>>> skype: milinda.pathirage
> > >>>>>>>>>>>>>> blog: http://milinda.pathirage.org
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> --
> > >>>>>> Milinda Pathirage
> > >>>>>>
> > >>>>>> PhD Student | Research Assistant
> > >>>>>> School of Informatics and Computing | Data to Insight Center
> > >>>>>> Indiana University
> > >>>>>>
> > >>>>>> twitter: milindalakmal
> > >>>>>> skype: milinda.pathirage
> > >>>>>> blog: http://milinda.pathirage.org
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> --
> > >>>>> Milinda Pathirage
> > >>>>>
> > >>>>> PhD Student | Research Assistant
> > >>>>> School of Informatics and Computing | Data to Insight Center
> > >>>>> Indiana University
> > >>>>>
> > >>>>> twitter: milindalakmal
> > >>>>> skype: milinda.pathirage
> > >>>>> blog: http://milinda.pathirage.org
> > >>>>>
> > >>>>
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>> Milinda Pathirage
> > >>>
> > >>> PhD Student | Research Assistant
> > >>> School of Informatics and Computing | Data to Insight Center
> > >>> Indiana University
> > >>>
> > >>> twitter: milindalakmal
> > >>> skype: milinda.pathirage
> > >>> blog: http://milinda.pathirage.org
> > >>>
> > >>
> > >>
> >
> >
>
--
Milinda Pathirage
PhD Student | Research Assistant
School of Informatics and Computing | Data to Insight Center
Indiana University
twitter: milindalakmal
skype: milinda.pathirage
blog: http://milinda.pathirage.org
