Docker isn't required to build the software, it's just another option. There's a build script, `build/build.sh`, that works just fine so long as you have the dependencies required to successfully build all components. I only mention this because if Docker is going to gate our ability to perform CI out in the open, we still have the `build.sh` option. I was able to use the build script to successfully build all components from master yesterday. -- Thanks, Jeff
On Tue, Mar 14, 2017 at 8:27 PM, Chris Lemmons <alfic...@gmail.com> wrote: > Yeah, there're unfortunately good reasons not to have any accounts with > write permission in the GitHub repo. It can cause all sorts of problems if > anything were actually pushed. It also allows lots of other things like > editing other people's comments. GitHub should really separate that out for > the purpose of bots with minimum required access anyway. But yeah, without > write, it's a no-go. Comments are a very reasonable alternative, though. > > It's definitely worth a few minutes to get things set up on the ASF Jenkins > if those ubuntu slaves have the requirements. As it stands, the only > requirements for a build are: > > - git > - bash > - docker > - docker-compose > - which requires python > > I believe the first three are satisfied by the build hosts. I don't know > that docker-compose is available. It's 100% worth finding out, though. > > If it's not, we can do one of: > > - Run docker-compose inside a docker instance with the docker port > forwarded into the container to allow it to manage sibling containers. > - Re-create the subset of docker-compose behaviour that we actually use > in a build script. > - Give up. > > I mention the first option only because People on the Internet seem to keep > suggesting it. I believe madness that way lies. I dislike giving up, so for > lack of a better option, perhaps we might need to ditch docker-compose. > I've got a PR open that wraps docker compose in a unified script. It > wouldn't be entirely unreasonable to shift a bit more logic into it. > > Another possibility is to see if the infra folks would mind adding python > and docker-compose. I'm not sure adding python to the mix on those boxen is > a good idea, though, even if they're willing. > > On Tue, Mar 14, 2017 at 6:03 PM Leif Hedstrom <zw...@apache.org> wrote: > > >> On Mar 14, 2017, at 6:15 PM, Chris Lemmons <alfic...@gmail.com> wrote: >> >> Honestly, the key is hosting. If we have a host for CI that runs the basic >> build steps, we can configure any solution to build all the changes on >> branches of a collection of repos on Github. Pretty much all the > reasonable >> options have a status update script on GitHub, which integrates it quite >> nicely. (And therein might lie the rub. I think GitHub ties status updates >> to "push permission", which may be false for everyone on the main repo, >> since it's just a mirror.) But direct integration or no, we'd be able to > go >> look at the results and even download the binary, install it on a test >> system and watch it go. > > > So, we do not have the Jenkins master have “write permission” into the > Github repo. I asked Infra before, and they said no, but I’ll try again. > > However, things can still work reasonably well, since any registered Github > accounts is able to comment on a PR / issue. So, no, we can’t set labels > etc. automatically from the Jenkins master, but we get pretty good feedback > on what happens with the builds. See e.g. > > https://github.com/apache/trafficserver/pull/1581 < > https://github.com/apache/trafficserver/pull/1581> > > > Cheers, > > — leif > >> >> Now, that doesn't get us automatic builds from first-time or probably even >> very occasional contributors. But stick builds on the most frequent >> contributors' clones and we get 95% of the benefit without solving any of >> the actually hard problems. >> >> We'd need a host, though. >> >> On Tue, Mar 14, 2017 at 5:06 PM Leif Hedstrom <zw...@apache.org> wrote: >> >>> >>>> On Mar 13, 2017, at 8:44 AM, Chris Lemmons <alfic...@gmail.com> wrote: >>>> >>>> To me, the key features of CI are that a) it builds each branch >>>> automatically, b) notifies affected parties when all is not well, and c) >>>> manages the artefacts in a reasonable way. Additionally, we're a lot > more >>>> useful when we're writing neat software and not spending out time >>> managing >>>> CI, so it should be as automatic as reasonable. We're using github for >>> PRs, >>>> so if it's at all possible to get automatic PR tagging with build >>>> information, that is greatly desirable. Knowing that the PR breaks the >>>> build prior to merging it can save quite a bit of time. :) >>> >>> >>> My $0.25: My experience is that making as much CI build / tests on pull >>> requests, *before* they are landed, gives the most bang for the buck. But >>> that might not work well for you, since you can’t use Github right? >>> >>> — leif >>> >>>> >>>> I've not used BuildBot, but it feels... svney? >>>> >>>> Jenkins can do all of the above, though basically all those features are >>>> plugins. There's a "build per branch" plugin that uses branches to >>>> automatically make builds. There's a variety of notifier plugins. There >>> are >>>> some artefact management plugins. There is a "build PRs" plugin that's >>>> specifically designed for GitHub. Jenkins isn't much on its own, but >>>> properly adorned with plugins, it can do most of what we'd want, I > think. >>>> >>>> I've also had extensive experience with Bamboo, Atlassian's > closed-source >>>> solution in the same suite as Jira. Bamboo is usable gratis for OSS in >>> the >>>> same way we currently use Apache Jira. >>>> >>>> Bamboo also has plugins, but the majority of it's features are good to > go >>>> immediately. There's an automatic checkbox for "build per branch". The >>>> artefacts are managed fairly automatically, especially if you fill in > the >>>> "build expiration" boxes. Notification is automatic and easy to >>> configure. >>>> It's got a (free) plugin for PR statuses. >>>> >>>> In any case, we'll need to manually configure the valid lists of >>>> contributors. For security reasons, we probably can't just run whatever >>> PR >>>> is created without any prior contact. :/ In Jenkins, this looks like >>>> maintaining a "white list" of legal contributors for a PR inside the PR >>>> plugin. In Bamboo, it looks like listing the committer forks as copied >>>> projects. Either way is fairly manageable. >>>> >>>> Jenkins and Bamboo both run on Java. So... no winner there. :) >>>> >>>> I think the major question is one of hosting. No matter what solution we >>>> use, we need a few cycles, a network interface, and a bit of disk space >>> to >>>> run it. The apache jenkins appears to have almost all the stuff we need. >>> It >>>> does not appear to have docker-compose, which we're leveraging fairly >>>> significantly at the moment. docker-compose, however, is Apache > licensed, >>>> so we could theoretically ship it inside the repo. Not sure I like that >>>> option, though. We could also switch off of compose and put the relevant >>>> options into our build script directly. Not sure I like that option, >>> either. >>>> >>>> We'd have the most flexibility if we could get one or more companies to >>>> donate a publicly accessible host (or even theoretically, a build > slave), >>>> assuming that doesn't break Apache's rules. The CI doesn't need a ton of >>>> gas, but the more oomph it has, the more granularly it can build and > more >>>> aggressively we can test. >>>> >>>> On Sun, Mar 12, 2017 at 6:54 PM Eric Friedrich (efriedri) < >>>> efrie...@cisco.com> wrote: >>>> >>>> Hey All- >>>> I’d played around before with Travis CI for Continuous Integration >>>> builds, but never actually set it up for the public repo. >>>> >>>> I know some others on the list have tried out comparable services. Does >>>> anyone have experience or suggestions to share? >>>> >>>> Also, we can now get access to Apache Buildbot ( >>>> https://ci.apache.org/buildbot.html) and an Apache hosted Jenkins ( >>>> https://cwiki.apache.org/confluence/display/INFRA/Jenkins) >>>> >>>> I think Traffic Server has their own separate Jenkins server so they can >>>> hit more platforms, but with the latest build changes, pretty much all > we >>>> require is access to a docker daemon >>>> >>>> —Eric >>> >>>
