This is indeed great news. I’m afraid I don’t have a strong opinion on how to make the tests more resilient to running in an emulated environment. Any of your suggestions are acceptable to me, assuming we can keep ppc64le support in scope.
Adam > On May 4, 2019, at 12:08 AM, Joan Touzet <woh...@apache.org> wrote: > > Hi again, > > With the support of ASF Infra, we now are in a position to run arbitrary > alternative platforms in our CI builds. This is great news for the people who > have been waiting for ARM (aarch64), PowerPC/POWER (ppc64le), mainframe > (s390x), and other architecture supports. > > However, we have a challenge. Emulating other architectures is necessarily > slower than running natively on provided hardware. I ran some measurements > today, and found that we're not able to pass our test suites because tests > are timing out. > > For example, on native x86_64 hardware, this test finishes in under half a > second: > > b64url_tests: encode_binary_test...[0.372s] ok > > Running the same test on aarch64, being emulated on the same x86_64 machine: > > b64url_tests: encode_binary_test...[4.493 s] ok > > or 12x longer. The next test (decode_iolist_test) fails, presumably because > it hits a 5s timeout period. > > I need advice from the list. Should we: > > * Increase the test timeouts significantly so these tests can complete > * Restrict ourselves to running only on actual hardware (which limits us > only to aarch64, and at a stretch, ppc64le) > * Remove test timeouts entirely, rewriting the ones that wait forever to > do something different > * Something else that I haven't mentioned > > Having regression testing on a variety of platforms is something that is of > benefit to the project; we've ignored it for too long. > > -Joan
