I just went thru all the recursive dependencies of Goose: https://bitbucket.org/liamstask/goose - MIT https://github.com/kylelemons/go-gypsy - Apache github.com/lib/pq - MIT github.com/mattn/go-sqlite3 - MIT github.com/PuerkitoBio/goquery - BSD 3-clause (included by go-sqlite3) golang.org/x/net/context - BSD (included by go-sqlite3) golang.org/x/net/html - BSD (included by goquery) golang.org/x/net/html/atom - BSD (included by x/net/html) github.com/andybalholm/cascadia - BSD 2-clause (included by goquery)
All those licenses are permissible. So, we can vendor all those, either in the Traffic Control, Traffic Ops, or Goose directory. -1 on Git Submodules. They're a pain. I vote simply vendoring them as ordinary dirs, via `git clone` and then deleting the `.git` dir. It's simple, and it works. It includes their respective license files by default. Then we only have to include the proper LICENSE and NOTICE files with the binaries we build. Not requiring Go to install is a separate issue. At a glance, it looks like it's possible to rewrite that migration in SQL, but not trivial. Looks like CTEs would help https://www.postgresql.org/docs/9.1/static/queries-with.html . +1 on setting the Go compiler as a dependency in the Traffic Ops RPM, until Go is removed as a dependency. I know it's a pain since it isn't in the standard CentOS packages, but it's less painful than the install failing. We should also add instructions for adding Go to Yum in the documentation. On Mon, May 1, 2017 at 12:26 PM, Eric Friedrich (efriedri) < [email protected]> wrote: > I know licenses are the issue more than lines of code. > > I’m not a fan of submodules, but if licensing comes through, it might make > sense to add goose and its deps as git submodules. > > One other thing to discuss :-) > > > > > > On May 1, 2017, at 2:04 PM, Dan Kirkwood <[email protected]> wrote: > > > > It's not a trivial problem. Yes -- we could include the source -- > > goose itself is actually fairly small and MIT licensed. Its > > dependencies have licenses that would need to be vetted, so > > vendoring is also not trivial -- no matter how many lines of code are > > involved. > > > > We could potentially compile goose and distribute within the rpm -- > > I've also suggested that before. Unfortunately, we have a migration > > written in go, which requires a go installation at run time. That > > means the requirement of a go installation is not avoidable without > > rewriting that as .sql. > > > > There are alternatives we could use that may not suffer from the same > issues. > > > > I suggest we sit down together at the Summit and decide where to go with > this.. > > > > -dan > > > > On Mon, May 1, 2017 at 10:37 AM, Robert Butts <[email protected]> > wrote: > >> +1 on Vendoring. I don't see a difference if it's 375,000 lines or > >> 10,000,000 lines. What does it matter if it's 375k lines in someone > else's > >> repo or our own? It does matter from a security standpoint. It means > we're > >> now vulnerable if their repo is compromised. We shouldn't be pulling > >> _anything_ from the internet at install time. > >> > >> Question for the Apache Gurus: If we include the Goose source, can we > also > >> include a binary built from that source? I don't see a legal or > >> philosophical reason we shouldn't be able to, if we include a hash of > the > >> binary and the LICENSE file. That lets us avoid requiring Go as a > >> dependency, which is difficult since few package managers have a modern > Go > >> package. Goose is MIT, > >> https://www.apache.org/dev/licensing-howto.html#binary suggests we > can, yes? > >> > >> > >> On Mon, May 1, 2017 at 8:31 AM, Dan Kirkwood <[email protected]> wrote: > >> > >>> ughh.. I'd forgotten I'd done that in all this.. > >>> > >>> Again -- catch-22. > >>> > >>> > >>> > >>> On Sun, Apr 30, 2017 at 10:20 PM, Mark Torluemke < > [email protected]> > >>> wrote: > >>>> On Sun, Apr 30, 2017 at 7:05 PM, Gelinas, Derek < > >>> [email protected]> > >>>> wrote: > >>>> > >>>>> +1 on both of these. > >>>>> > >>>>>> On Apr 30, 2017, at 8:50 PM, Eric Friedrich (efriedri) < > >>>>> [email protected]> wrote: > >>>>>> > >>>>>> Assuming we stick with goose, why not bundle goose source into the > >>>>> traffic ops RPM? This will pin the version for us and prevent users > from > >>>>> needing to run go get > >>>>> > >>>> > >>>> Dan had put in a PR to add the Goose source: > >>>> https://github.com/apache/incubator-trafficcontrol/pull/157 > >>>> > >>>> We ended up closing it, as 375,000 lines felt a bit excessive... > >>>> > >>>> > >>>> > >>>>>> > >>>>>> We are allowed to bundle code with the MIT license into our > releases. > >>>>>> > >>>>>> As for the go installation, what about modifying the RPM spec file > to > >>>>> list GoLang as a dependency of the traffic ops RPM? > >>>>>> > >>>>>> —Eric > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>>> On Apr 28, 2017, at 4:46 PM, Dewayne Richardson <[email protected] > > > >>>>> wrote: > >>>>>>> > >>>>>>> They are, but makes the tooling easier if we are all in Golang > >>>>>>> > >>>>>>>> On Fri, Apr 28, 2017 at 1:44 PM, Dave Neuman <[email protected]> > >>>>> wrote: > >>>>>>>> > >>>>>>>> I don't see why re-writing the APIs in something like golang would > >>> mean > >>>>>>>> that we also need to re-write the database admin script. I think > >>>>> those two > >>>>>>>> things are mutually exclusive, right? > >>>>>>>> > >>>>>>>> On Fri, Apr 28, 2017 at 12:29 PM, Dewayne Richardson < > >>>>> [email protected]> > >>>>>>>> wrote: > >>>>>>>> > >>>>>>>>> I had that thought, as well as there are more recent versions > like > >>>>>>>>> https://github.com/mattes/migrate. The question becomes if we > >>> ever > >>>>> get > >>>>>>>>> around to rewriting TrafficOps APIs in golang, will the Perl > >>> version > >>>>> then > >>>>>>>>> become obsolete? > >>>>>>>>> > >>>>>>>>>> On Fri, Apr 28, 2017 at 11:58 AM, Dave Neuman < > [email protected]> > >>>>> wrote: > >>>>>>>>>> > >>>>>>>>>> Maybe it's time we take a look at what goose really buys us and > >>>>>>>> consider > >>>>>>>>>> writing our own database migration tool. We already have > >>> admin.pl, > >>>>> it > >>>>>>>>>> could probably fit in with that? > >>>>>>>>>> > >>>>>>>>>> On Fri, Apr 28, 2017 at 11:45 AM, Eric Friedrich (efriedri) < > >>>>>>>>>> [email protected]> wrote: > >>>>>>>>>> > >>>>>>>>>>> Hey Dew- > >>>>>>>>>>> What calls this script? > >>>>>>>>>>> > >>>>>>>>>>> If its called from the Traffic Ops Spec file, then this will > >>> cause > >>>>>>>> some > >>>>>>>>>>> pain for those of us that need to install without internet > >>> access. > >>>>>>>>>>> > >>>>>>>>>>> —Eric > >>>>>>>>>>> > >>>>>>>>>>>> On Apr 28, 2017, at 12:41 PM, Dewayne Richardson < > >>>>>>>> [email protected]> > >>>>>>>>>>> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>> I'm working toward a more streamlined installation process for > >>>>>>>>> Traffic > >>>>>>>>>>> Ops > >>>>>>>>>>>> (internally) and publicly. Of course, the same hiccups that > >>>>>>>> everyone > >>>>>>>>>> else > >>>>>>>>>>>> runs into I am as well. Installation of Golang (proper > version) > >>>>>>>> and > >>>>>>>>>>>> installation of Goose. Goose has been the most challenging > for > >>>>>>>>> several > >>>>>>>>>>>> reasons. The maintainer hasn't made any real changes since > >>> 2015, > >>>>>>>> and > >>>>>>>>>> has > >>>>>>>>>>>> not "branched" his code to allow for explicit version > download. > >>>>>>>> Per > >>>>>>>>>> his > >>>>>>>>>>>> installation instructions "go get > >>> bitbucket.org/liamstask/goose/ > >>>>>>>>>>> cmd/goose" > >>>>>>>>>>>> > >>>>>>>>>>>> So I'm I'm proposing to write an installer script in bash to > >>> help > >>>>>>>>>>> automate > >>>>>>>>>>>> the Golang install as well as the Goose install. My only > >>> concern > >>>>>>>> (as > >>>>>>>>>>> well > >>>>>>>>>>>> as most of yours) is "go get" will grab the latest, but since > no > >>>>>>>> real > >>>>>>>>>>>> changes have happened I'm left with no other option. > >>>>>>>>>>>> > >>>>>>>>>>>> Proposed: > >>>>>>>>>>>> > >>>>>>>>>>>> /opt/traffic_ops/install/bin/install_goose.sh > >>>>>>>>>>>> > >>>>>>>>>>>> - Install Golang (version 1.8.x) > >>>>>>>>>>>> - go get bitbucket.org/liamstask/goose/cmd/goose > >>>>>>>>>>>> > >>>>>>>>>>>> Thoughts? > >>>>>>>>>>>> > >>>>>>>>>>>> -Dew > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>> > >>>>> > >>> > >
