On 09/09/19 11:37, Misagh Moayyed wrote: > Hello all, > > In discussing git workflows with Francesco, I have learned that the Travis CI > process is not actually running a comprehensive set of tests and checks (such > as checkstyle, etc) that would be valuable for pull requests and > contributors, possibly because such jobs take a long time, more than what is > allowed by Travis CI. I'd like to propose to reorganize the CI build process > to take advantage of the build stages feature [1]. This [2] demonstrates an > example where a given build is broken down into multiple stages, each of > which has numerous jobs. > > A key thing to note in this configuration, as I understand, is that a given > CI build can have an *unlimited* number of stages, each with an unlimited > number of jobs. While each job cannot exceed a certain timeout value [3], the > collective overall build stage can go on forever in theory. Also, stages do > depend on each other such that failures in one would halt the build. A > typical example would be: "Run the build, and if everything compiles, run > tests, then publish snapshots, etc". > > I think this is quite valuable for contributors who, specially, work on > multiple JIRAs at the same time. A sample workflow is to add a change, do > some sanity tests and then push to let CI run the full build suite. While > it's busy kicking off the build, one could switch to the next JIRA to repeat > the same process, and as such hop back and forth between branches and JIRAs > to parallelize work activity. Ultimately, the CI is the canonical reference > for all changes and this would avoid the problem of "but it works on my > machine" and of course, saves quite a bit of time. (A full build for me takes > about 20-30 minutes, if the stars align). > > How do folks feel about this proposal?
Hi Misagh, thanks for bringing this item to discussion: improving our CI power is definitely more than welcome. I would say there are several requirements one might want to handle: 1. run the straight "mvn clean install" one normally performs on his / her laptop, which involves RAT checks, Checkstyle, modernizer, etc, all unit and integration tests 1a. run integration tests with different content types [5] 2. run the Admin UI tests [4], currently not run by default under fit/core-reference 3. run the Enduser UI tests ([4] for master), currently not run by default under fit/core-reference 4. run integration tests under the other supported Java EE containers (besides Tomcat) [6] 5. run integration tests with the other supported DBMSes (besides H2) [7] 6. run integration tests with Elasticsearch - not documented but available 7. run SAML 2.0 / OpenID Connect 1.0 login tests - not documented but available (only master) I am not saying we should cover all above in one shot, just reporting the full requirements; I would be happy even with only (1), a great progress anyway compared to the current situation. I am not sure about [3] - we should ask ASF Infra to issue such a request, in case, as they are the actual owner of the GitHub repo. Regards. > [1] https://docs.travis-ci.com/user/build-stages/ > [2] https://travis-ci.org/apereo/cas/builds/581042662 > [3] The good folks at Travis CI in the past have been kind enough to increase > the timeout from 50 minutes to 180 minutes and they might be willing to make > the same change for syncope. Larger tasks need to be broken down to separate > jobs, of course. > [4] http://syncope.apache.org/building#All_integration_tests > [5] http://syncope.apache.org/building#Other_Content-Type_values > [6] http://syncope.apache.org/building#Java_EE_containers > [7] http://syncope.apache.org/building#DBMSes -- Francesco Chicchiriccò Tirasa - Open Source Excellence http://www.tirasa.net/ Member at The Apache Software Foundation Syncope, Cocoon, Olingo, CXF, OpenJPA, PonyMail http://home.apache.org/~ilgrosso/
