I second JvD, and I think the relevant goose version should be copied to a location managed by the TC team. This way, the will be no dependency of TC installs on an external, uncontrollable source of data.
The issue was demonstrated a few months ago when the fancybox-2.1.5 source code has moved. On Mon, May 1, 2017 at 11:31 PM, Jan van Doorn <[email protected]> wrote: > I think we should just require the installing person to do a “go get > liamstack/goose” (or whatever the syntax is), and move on. Put that in the > installation instructions, like: > > Step 1: install Linux > Step 2: go get goose > Step 3: the rest. > > I know it’s not ideal, but… this seems to be becoming too hard. > > Rgds, > JvD > > > On May 1, 2017, at 1:44 PM, Robert Butts <[email protected]> > wrote: > > > > I just went thru all the recursive dependencies of Goose: > > > > https://bitbucket.org/liamstask/goose - MIT > > https://github.com/kylelemons/go-gypsy - Apache > > github.com/lib/pq - MIT > > github.com/mattn/go-sqlite3 - MIT > > github.com/PuerkitoBio/goquery - BSD 3-clause (included by go-sqlite3) > > golang.org/x/net/context - BSD (included by go-sqlite3) > > golang.org/x/net/html - BSD (included by goquery) > > golang.org/x/net/html/atom - BSD (included by x/net/html) > > github.com/andybalholm/cascadia - BSD 2-clause (included by goquery) > > > > All those licenses are permissible. So, we can vendor all those, either > in > > the Traffic Control, Traffic Ops, or Goose directory. > > > > -1 on Git Submodules. They're a pain. I vote simply vendoring them as > > ordinary dirs, via `git clone` and then deleting the `.git` dir. It's > > simple, and it works. It includes their respective license files by > > default. Then we only have to include the proper LICENSE and NOTICE files > > with the binaries we build. > > > > Not requiring Go to install is a separate issue. At a glance, it looks > like > > it's possible to rewrite that migration in SQL, but not trivial. Looks > like > > CTEs would help https://www.postgresql.org/docs/9.1/static/queries-with. > html > > . > > > > +1 on setting the Go compiler as a dependency in the Traffic Ops RPM, > until > > Go is removed as a dependency. I know it's a pain since it isn't in the > > standard CentOS packages, but it's less painful than the install failing. > > We should also add instructions for adding Go to Yum in the > documentation. > > > > > > On Mon, May 1, 2017 at 12:26 PM, Eric Friedrich (efriedri) < > > [email protected]> wrote: > > > >> I know licenses are the issue more than lines of code. > >> > >> I’m not a fan of submodules, but if licensing comes through, it might > make > >> sense to add goose and its deps as git submodules. > >> > >> One other thing to discuss :-) > >> > >> > >> > >> > >>> On May 1, 2017, at 2:04 PM, Dan Kirkwood <[email protected]> wrote: > >>> > >>> It's not a trivial problem. Yes -- we could include the source -- > >>> goose itself is actually fairly small and MIT licensed. Its > >>> dependencies have licenses that would need to be vetted, so > >>> vendoring is also not trivial -- no matter how many lines of code are > >>> involved. > >>> > >>> We could potentially compile goose and distribute within the rpm -- > >>> I've also suggested that before. Unfortunately, we have a migration > >>> written in go, which requires a go installation at run time. That > >>> means the requirement of a go installation is not avoidable without > >>> rewriting that as .sql. > >>> > >>> There are alternatives we could use that may not suffer from the same > >> issues. > >>> > >>> I suggest we sit down together at the Summit and decide where to go > with > >> this.. > >>> > >>> -dan > >>> > >>> On Mon, May 1, 2017 at 10:37 AM, Robert Butts < > [email protected]> > >> wrote: > >>>> +1 on Vendoring. I don't see a difference if it's 375,000 lines or > >>>> 10,000,000 lines. What does it matter if it's 375k lines in someone > >> else's > >>>> repo or our own? It does matter from a security standpoint. It means > >> we're > >>>> now vulnerable if their repo is compromised. We shouldn't be pulling > >>>> _anything_ from the internet at install time. > >>>> > >>>> Question for the Apache Gurus: If we include the Goose source, can we > >> also > >>>> include a binary built from that source? I don't see a legal or > >>>> philosophical reason we shouldn't be able to, if we include a hash of > >> the > >>>> binary and the LICENSE file. That lets us avoid requiring Go as a > >>>> dependency, which is difficult since few package managers have a > modern > >> Go > >>>> package. Goose is MIT, > >>>> https://www.apache.org/dev/licensing-howto.html#binary suggests we > >> can, yes? > >>>> > >>>> > >>>> On Mon, May 1, 2017 at 8:31 AM, Dan Kirkwood <[email protected]> > wrote: > >>>> > >>>>> ughh.. I'd forgotten I'd done that in all this.. > >>>>> > >>>>> Again -- catch-22. > >>>>> > >>>>> > >>>>> > >>>>> On Sun, Apr 30, 2017 at 10:20 PM, Mark Torluemke < > >> [email protected]> > >>>>> wrote: > >>>>>> On Sun, Apr 30, 2017 at 7:05 PM, Gelinas, Derek < > >>>>> [email protected]> > >>>>>> wrote: > >>>>>> > >>>>>>> +1 on both of these. > >>>>>>> > >>>>>>>> On Apr 30, 2017, at 8:50 PM, Eric Friedrich (efriedri) < > >>>>>>> [email protected]> wrote: > >>>>>>>> > >>>>>>>> Assuming we stick with goose, why not bundle goose source into the > >>>>>>> traffic ops RPM? This will pin the version for us and prevent users > >> from > >>>>>>> needing to run go get > >>>>>>> > >>>>>> > >>>>>> Dan had put in a PR to add the Goose source: > >>>>>> https://github.com/apache/incubator-trafficcontrol/pull/157 > >>>>>> > >>>>>> We ended up closing it, as 375,000 lines felt a bit excessive... > >>>>>> > >>>>>> > >>>>>> > >>>>>>>> > >>>>>>>> We are allowed to bundle code with the MIT license into our > >> releases. > >>>>>>>> > >>>>>>>> As for the go installation, what about modifying the RPM spec file > >> to > >>>>>>> list GoLang as a dependency of the traffic ops RPM? > >>>>>>>> > >>>>>>>> —Eric > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>>> On Apr 28, 2017, at 4:46 PM, Dewayne Richardson < > [email protected] > >>> > >>>>>>> wrote: > >>>>>>>>> > >>>>>>>>> They are, but makes the tooling easier if we are all in Golang > >>>>>>>>> > >>>>>>>>>> On Fri, Apr 28, 2017 at 1:44 PM, Dave Neuman <[email protected] > > > >>>>>>> wrote: > >>>>>>>>>> > >>>>>>>>>> I don't see why re-writing the APIs in something like golang > would > >>>>> mean > >>>>>>>>>> that we also need to re-write the database admin script. I > think > >>>>>>> those two > >>>>>>>>>> things are mutually exclusive, right? > >>>>>>>>>> > >>>>>>>>>> On Fri, Apr 28, 2017 at 12:29 PM, Dewayne Richardson < > >>>>>>> [email protected]> > >>>>>>>>>> wrote: > >>>>>>>>>> > >>>>>>>>>>> I had that thought, as well as there are more recent versions > >> like > >>>>>>>>>>> https://github.com/mattes/migrate. The question becomes if we > >>>>> ever > >>>>>>> get > >>>>>>>>>>> around to rewriting TrafficOps APIs in golang, will the Perl > >>>>> version > >>>>>>> then > >>>>>>>>>>> become obsolete? > >>>>>>>>>>> > >>>>>>>>>>>> On Fri, Apr 28, 2017 at 11:58 AM, Dave Neuman < > >> [email protected]> > >>>>>>> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>> Maybe it's time we take a look at what goose really buys us > and > >>>>>>>>>> consider > >>>>>>>>>>>> writing our own database migration tool. We already have > >>>>> admin.pl, > >>>>>>> it > >>>>>>>>>>>> could probably fit in with that? > >>>>>>>>>>>> > >>>>>>>>>>>> On Fri, Apr 28, 2017 at 11:45 AM, Eric Friedrich (efriedri) < > >>>>>>>>>>>> [email protected]> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>>> Hey Dew- > >>>>>>>>>>>>> What calls this script? > >>>>>>>>>>>>> > >>>>>>>>>>>>> If its called from the Traffic Ops Spec file, then this will > >>>>> cause > >>>>>>>>>> some > >>>>>>>>>>>>> pain for those of us that need to install without internet > >>>>> access. > >>>>>>>>>>>>> > >>>>>>>>>>>>> —Eric > >>>>>>>>>>>>> > >>>>>>>>>>>>>> On Apr 28, 2017, at 12:41 PM, Dewayne Richardson < > >>>>>>>>>> [email protected]> > >>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> I'm working toward a more streamlined installation process > for > >>>>>>>>>>> Traffic > >>>>>>>>>>>>> Ops > >>>>>>>>>>>>>> (internally) and publicly. Of course, the same hiccups that > >>>>>>>>>> everyone > >>>>>>>>>>>> else > >>>>>>>>>>>>>> runs into I am as well. Installation of Golang (proper > >> version) > >>>>>>>>>> and > >>>>>>>>>>>>>> installation of Goose. Goose has been the most challenging > >> for > >>>>>>>>>>> several > >>>>>>>>>>>>>> reasons. The maintainer hasn't made any real changes since > >>>>> 2015, > >>>>>>>>>> and > >>>>>>>>>>>> has > >>>>>>>>>>>>>> not "branched" his code to allow for explicit version > >> download. > >>>>>>>>>> Per > >>>>>>>>>>>> his > >>>>>>>>>>>>>> installation instructions "go get > >>>>> bitbucket.org/liamstask/goose/ > >>>>>>>>>>>>> cmd/goose" > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> So I'm I'm proposing to write an installer script in bash to > >>>>> help > >>>>>>>>>>>>> automate > >>>>>>>>>>>>>> the Golang install as well as the Goose install. My only > >>>>> concern > >>>>>>>>>> (as > >>>>>>>>>>>>> well > >>>>>>>>>>>>>> as most of yours) is "go get" will grab the latest, but > since > >> no > >>>>>>>>>> real > >>>>>>>>>>>>>> changes have happened I'm left with no other option. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Proposed: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> /opt/traffic_ops/install/bin/install_goose.sh > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> - Install Golang (version 1.8.x) > >>>>>>>>>>>>>> - go get bitbucket.org/liamstask/goose/cmd/goose > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Thoughts? > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> -Dew > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>> > >> > >> > > -- *Oren Shemesh* Qwilt | Work: +972-72-2221637| Mobile: +972-50-2281168 | [email protected] <[email protected]>
