I think we should just require the installing person to do a “go get liamstack/goose” (or whatever the syntax is), and move on. Put that in the installation instructions, like:
Step 1: install Linux Step 2: go get goose Step 3: the rest. I know it’s not ideal, but… this seems to be becoming too hard. Rgds, JvD > On May 1, 2017, at 1:44 PM, Robert Butts <[email protected]> wrote: > > I just went thru all the recursive dependencies of Goose: > > https://bitbucket.org/liamstask/goose - MIT > https://github.com/kylelemons/go-gypsy - Apache > github.com/lib/pq - MIT > github.com/mattn/go-sqlite3 - MIT > github.com/PuerkitoBio/goquery - BSD 3-clause (included by go-sqlite3) > golang.org/x/net/context - BSD (included by go-sqlite3) > golang.org/x/net/html - BSD (included by goquery) > golang.org/x/net/html/atom - BSD (included by x/net/html) > github.com/andybalholm/cascadia - BSD 2-clause (included by goquery) > > All those licenses are permissible. So, we can vendor all those, either in > the Traffic Control, Traffic Ops, or Goose directory. > > -1 on Git Submodules. They're a pain. I vote simply vendoring them as > ordinary dirs, via `git clone` and then deleting the `.git` dir. It's > simple, and it works. It includes their respective license files by > default. Then we only have to include the proper LICENSE and NOTICE files > with the binaries we build. > > Not requiring Go to install is a separate issue. At a glance, it looks like > it's possible to rewrite that migration in SQL, but not trivial. Looks like > CTEs would help https://www.postgresql.org/docs/9.1/static/queries-with.html > . > > +1 on setting the Go compiler as a dependency in the Traffic Ops RPM, until > Go is removed as a dependency. I know it's a pain since it isn't in the > standard CentOS packages, but it's less painful than the install failing. > We should also add instructions for adding Go to Yum in the documentation. > > > On Mon, May 1, 2017 at 12:26 PM, Eric Friedrich (efriedri) < > [email protected]> wrote: > >> I know licenses are the issue more than lines of code. >> >> I’m not a fan of submodules, but if licensing comes through, it might make >> sense to add goose and its deps as git submodules. >> >> One other thing to discuss :-) >> >> >> >> >>> On May 1, 2017, at 2:04 PM, Dan Kirkwood <[email protected]> wrote: >>> >>> It's not a trivial problem. Yes -- we could include the source -- >>> goose itself is actually fairly small and MIT licensed. Its >>> dependencies have licenses that would need to be vetted, so >>> vendoring is also not trivial -- no matter how many lines of code are >>> involved. >>> >>> We could potentially compile goose and distribute within the rpm -- >>> I've also suggested that before. Unfortunately, we have a migration >>> written in go, which requires a go installation at run time. That >>> means the requirement of a go installation is not avoidable without >>> rewriting that as .sql. >>> >>> There are alternatives we could use that may not suffer from the same >> issues. >>> >>> I suggest we sit down together at the Summit and decide where to go with >> this.. >>> >>> -dan >>> >>> On Mon, May 1, 2017 at 10:37 AM, Robert Butts <[email protected]> >> wrote: >>>> +1 on Vendoring. I don't see a difference if it's 375,000 lines or >>>> 10,000,000 lines. What does it matter if it's 375k lines in someone >> else's >>>> repo or our own? It does matter from a security standpoint. It means >> we're >>>> now vulnerable if their repo is compromised. We shouldn't be pulling >>>> _anything_ from the internet at install time. >>>> >>>> Question for the Apache Gurus: If we include the Goose source, can we >> also >>>> include a binary built from that source? I don't see a legal or >>>> philosophical reason we shouldn't be able to, if we include a hash of >> the >>>> binary and the LICENSE file. That lets us avoid requiring Go as a >>>> dependency, which is difficult since few package managers have a modern >> Go >>>> package. Goose is MIT, >>>> https://www.apache.org/dev/licensing-howto.html#binary suggests we >> can, yes? >>>> >>>> >>>> On Mon, May 1, 2017 at 8:31 AM, Dan Kirkwood <[email protected]> wrote: >>>> >>>>> ughh.. I'd forgotten I'd done that in all this.. >>>>> >>>>> Again -- catch-22. >>>>> >>>>> >>>>> >>>>> On Sun, Apr 30, 2017 at 10:20 PM, Mark Torluemke < >> [email protected]> >>>>> wrote: >>>>>> On Sun, Apr 30, 2017 at 7:05 PM, Gelinas, Derek < >>>>> [email protected]> >>>>>> wrote: >>>>>> >>>>>>> +1 on both of these. >>>>>>> >>>>>>>> On Apr 30, 2017, at 8:50 PM, Eric Friedrich (efriedri) < >>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>> Assuming we stick with goose, why not bundle goose source into the >>>>>>> traffic ops RPM? This will pin the version for us and prevent users >> from >>>>>>> needing to run go get >>>>>>> >>>>>> >>>>>> Dan had put in a PR to add the Goose source: >>>>>> https://github.com/apache/incubator-trafficcontrol/pull/157 >>>>>> >>>>>> We ended up closing it, as 375,000 lines felt a bit excessive... >>>>>> >>>>>> >>>>>> >>>>>>>> >>>>>>>> We are allowed to bundle code with the MIT license into our >> releases. >>>>>>>> >>>>>>>> As for the go installation, what about modifying the RPM spec file >> to >>>>>>> list GoLang as a dependency of the traffic ops RPM? >>>>>>>> >>>>>>>> —Eric >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> On Apr 28, 2017, at 4:46 PM, Dewayne Richardson <[email protected] >>> >>>>>>> wrote: >>>>>>>>> >>>>>>>>> They are, but makes the tooling easier if we are all in Golang >>>>>>>>> >>>>>>>>>> On Fri, Apr 28, 2017 at 1:44 PM, Dave Neuman <[email protected]> >>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> I don't see why re-writing the APIs in something like golang would >>>>> mean >>>>>>>>>> that we also need to re-write the database admin script. I think >>>>>>> those two >>>>>>>>>> things are mutually exclusive, right? >>>>>>>>>> >>>>>>>>>> On Fri, Apr 28, 2017 at 12:29 PM, Dewayne Richardson < >>>>>>> [email protected]> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> I had that thought, as well as there are more recent versions >> like >>>>>>>>>>> https://github.com/mattes/migrate. The question becomes if we >>>>> ever >>>>>>> get >>>>>>>>>>> around to rewriting TrafficOps APIs in golang, will the Perl >>>>> version >>>>>>> then >>>>>>>>>>> become obsolete? >>>>>>>>>>> >>>>>>>>>>>> On Fri, Apr 28, 2017 at 11:58 AM, Dave Neuman < >> [email protected]> >>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> Maybe it's time we take a look at what goose really buys us and >>>>>>>>>> consider >>>>>>>>>>>> writing our own database migration tool. We already have >>>>> admin.pl, >>>>>>> it >>>>>>>>>>>> could probably fit in with that? >>>>>>>>>>>> >>>>>>>>>>>> On Fri, Apr 28, 2017 at 11:45 AM, Eric Friedrich (efriedri) < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hey Dew- >>>>>>>>>>>>> What calls this script? >>>>>>>>>>>>> >>>>>>>>>>>>> If its called from the Traffic Ops Spec file, then this will >>>>> cause >>>>>>>>>> some >>>>>>>>>>>>> pain for those of us that need to install without internet >>>>> access. >>>>>>>>>>>>> >>>>>>>>>>>>> —Eric >>>>>>>>>>>>> >>>>>>>>>>>>>> On Apr 28, 2017, at 12:41 PM, Dewayne Richardson < >>>>>>>>>> [email protected]> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> I'm working toward a more streamlined installation process for >>>>>>>>>>> Traffic >>>>>>>>>>>>> Ops >>>>>>>>>>>>>> (internally) and publicly. Of course, the same hiccups that >>>>>>>>>> everyone >>>>>>>>>>>> else >>>>>>>>>>>>>> runs into I am as well. Installation of Golang (proper >> version) >>>>>>>>>> and >>>>>>>>>>>>>> installation of Goose. Goose has been the most challenging >> for >>>>>>>>>>> several >>>>>>>>>>>>>> reasons. The maintainer hasn't made any real changes since >>>>> 2015, >>>>>>>>>> and >>>>>>>>>>>> has >>>>>>>>>>>>>> not "branched" his code to allow for explicit version >> download. >>>>>>>>>> Per >>>>>>>>>>>> his >>>>>>>>>>>>>> installation instructions "go get >>>>> bitbucket.org/liamstask/goose/ >>>>>>>>>>>>> cmd/goose" >>>>>>>>>>>>>> >>>>>>>>>>>>>> So I'm I'm proposing to write an installer script in bash to >>>>> help >>>>>>>>>>>>> automate >>>>>>>>>>>>>> the Golang install as well as the Goose install. My only >>>>> concern >>>>>>>>>> (as >>>>>>>>>>>>> well >>>>>>>>>>>>>> as most of yours) is "go get" will grab the latest, but since >> no >>>>>>>>>> real >>>>>>>>>>>>>> changes have happened I'm left with no other option. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Proposed: >>>>>>>>>>>>>> >>>>>>>>>>>>>> /opt/traffic_ops/install/bin/install_goose.sh >>>>>>>>>>>>>> >>>>>>>>>>>>>> - Install Golang (version 1.8.x) >>>>>>>>>>>>>> - go get bitbucket.org/liamstask/goose/cmd/goose >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thoughts? >>>>>>>>>>>>>> >>>>>>>>>>>>>> -Dew >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>> >>>>> >> >>
