> On Nov 3, 2016, at 10:17 AM, Barry Warsaw <ba...@python.org> wrote:
>
> On Nov 03, 2016, at 12:54 AM, Nick Coghlan wrote:
>
>> This is also an area where I'm fine with recommending freemium
>> solutions if they're the lowest barrier to entry option for new users,
>> and "Use GitHub + Travis CI" qualifies on that front.
>
> I won't rehash the GitHub/GitLab debate, but in some of my projects (hosted on
> GH) I've had to ditch Travis because of limitations on that platform.
> Specifically, I needed to run various tests on an exact specification of
> various Ubuntu platforms, e.g. does X run on an up-to-date Ubuntu Y.Z?
>
> I originally used Docker for this, but our projects had additional
> constraints, such as needing to bind-mount, which aren't supported on the
> Travis+Docker platform. So we ended up ditching the Travis integration and
> hooking our test suite into the Ubuntu autopkgtest system (which is nearly
> identical to the Debian autopkgtest system but runs on Ubuntu infrastructure).
>
> Python may not be affected by similar constraints, but it is worth keeping in
> mind. Travis isn't a perfect technical solution for all projects, but it may
> be good enough for Python.
I think phrasing this in terms of "perfect" and "good enough" presents a highly
misleading framing. Examined in this fashion, of course we may reluctantly use
the "good enough" option, but don't we want the best option?
A better way to look at it is cost vs. benefit.
How much does it cost you in terms of time and money to run and administer the
full spectrum of "real" operating systems X.Z that you wish to support? How
much does it cost in terms of waiting for all that extra build infrastructure
to run all the time? How much additional confidence and assurance that it will
work does that buy you, over the confidence of passing tests within a docker
container? Is that additional confidence worth the investment of resources?
Of course, volunteer-driven projects are not concerned directly with top-level
management allocation of ostensibly fungible resources, and so a hard "costly"
solution that someone is interested in and committed to is far less expensive
than a "cheap" solution that everyone finds boring, so we have to take that
into account as well.
As it happens, Twisted has a massive investment in existing Buildbot CI
infrastructure _as well as_ Travis and Appveyor. Travis and Appveyor address
something that our CI can't, which is allowing unauthenticated builds from
randos issuing their first pull requests. This gives contributors much faster
feedback which is adequate for the majority of changes.
However, many of our ancillary projects, which do not have as many
platform-sensitive components, are built using Travis only, and that's a very
good compromise for them. It has allowed us to maintain a much larger and more
diverse ecosystem with a much smaller team than we used to be able to.
In the future, we may have to move to a different CI service, but I can tell
you that for sure that 90% of the work involved in getting builds to run on
Travis is transferrable to any platform that can run a shell script. There's a
bit of YAML configuration we would need to replicate, and we might have to do
some fancy dancing with Docker to get other ancillary services run on the
backend in some other way, but I would not worry about vendor lock-in at all
for this sort of service. Probably, the amount of time and energy on system
maintenance that Travis saves us in a given week is enough to balance out all
the possible future migration work.
-glyph
_______________________________________________
Distutils-SIG maillist - Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig
