Hi all,
As some/a lot of you have been asking how to contribute, I'm here to answer
some questions :).
First of all, integrating fixes in Pharo has been "slowed down" because we
missed the Monkey for Pull Request (PR) integration. Some of you have seen
a travis job going around and checking. That was our initial solution, but
it has a main issue for us: it only works for unix because our build
process is long, uses lots of small files due to file tree and osx travis
support is REALLY SLOW with I/O access. This is why we chose to use good
old Jenkins.
https://ci.inria.fr/pharo-ci-jenkins2/
# Why it took so much time
Just some explanation of why it took so much time (you can skip this if
you're not interested).
I started working on integrating Jenkins with Pull Requests several weeks
ago (note that I'm part time on this, also working in other projects and
the deadlines of the past weeks like IWST were killing my time). We have
now a new and working Jenkins server for Pharo7. This jenkins server is
hosted in Inria, its url being
- We chose to use Jenkins 2 because I managed to make the PR integration
plugin work only in Jenkins 2.
- This required the creation of a new jenkins server + slaves. The pharo
and pharo-contribution servers run with Jenkins 1.6 and I did not want to
migrate existing servers to not break existing jobs.
- At the same moment, the inria CI infrastructure was having some (lots?)
of hiccups because of storage limitations. I spent several weeks to create
the running slaves for each platform and connect jenkins to them because of
that.
- Finally, I spent some time to test the new Jenkins MultiJobs. These kind
of jobs allow to split a single job in sequential phases. And each phase
can have multiple jobs that run concurrently. This allows us to bootstrap
once (on unix) and test in all platforms.
An example of the new multijob can be found here:
https://ci.inria.fr/pharo-ci-jenkins2/job/Pharo-PullRequest-Pipeline/
# Current Design
Right at this moment, we are validating PRs using Jenkins. The process is
as follows:
1) Github notifies about changes in PRs to Jenkins using webhooks. So far,
we have configured it to notify about PR changes (including comments) and
normal pushes (to capture merges into the #development branch).
2) Upon the reception of a GH notification, Jenkins will launch two
jobs/validations. The basic structure of both validations is the same so
far. Bootstrap once (unix) and test in many platforms. There are the
following two test configurations however.
2.1) Full Validation: tests 32 and 64 bits, for linux, osx and windows.
This configuration is not fully working right now because we found a
VM bug in 64 bits that prevents us from creating a full pharo image.
Bootstrap is Ok, but there is a segfault when loading a big Metacello
configuration.
2.2) Basic Validation: tests only 32 bits, for now in linux and osx (I'm
planning to do windows today).
This validation is to give us nevertheless a feedback of the build
even if the full is failing so far.
2.3) Others? We could easily add new validations to run some faster tests
i.e., just loading a patch on the previous image.
3) Jenkins takes control and notifies Github
So far, Full validation is failing, and I plan to keep it running until it
gets green. I don't care if it's red right now, the idea is that we should
fix it, and we should remember it is broken :).
On the other hand, the Basic Validation is working and running tests. Once
the tests runs are finished, Github PR state will be either a success (if
all tests went green) or a failure (if it failed or some tests failed). You
will see actually a message saying:
Validation Failure. Please check test results.
7782 tests run, 0 skipped, 6 failed.
+ a link to jenkins if you want to inspect results.
You can see an example of it in action in here:
https://github.com/pharo-project/pharo/pull/119
# Next points
Right now we are missing:
- Fogbugz integration
- Publish new images in the file server when a push/merge happens in the
#development branch
(I should sit today with Esteban to do some ssh key configuration in a
server I have no access to)
- fixing the failing tests => TO HAVE A GREEN BUILD> :D
- Enhance tests?
- your enhancement idea :)
The current validation already gives us an idea of what works and what does
not work, what breaks the bootstrap, and if we have failing tests or not.
I'd like however that before integrating a fix, there is at least 2 people
that look at it and put a +1. Because Monkey validation and the ability to
merge does not mean much actually :).
If you have some questions, enhancements, there is this thread, there is
the #bootstrap channel in discord also.
Guille (in behalf of everybody that helped me with this)
--
Guille Polito
Research Engineer
French National Center for Scientific Research - *http://www.cnrs.fr*
<http://www.cnrs.fr>
*Web:* *http://guillep.github.io* <http://guillep.github.io>
*Phone: *+33 06 52 70 66 13
