Ilya, Am 11.11.20 um 19:38 schrieb Илья Шипицин: > let us discuss next steps :) > > Ubuntu, gcc, all features > Ubuntu, gcc, ssl=stock > both of jobs use stock ssl. do we really need second one (ssl enabled, no > other features enabled) ? I would second one (same for clang)
Yes, we need both. I'd like to have both to be able to directly compare any differences between the various SSL libraries. Compare the following example: - Ubuntu, gcc, all features: Fails. - Ubuntu, gcc, ssl=stock: Works. - Ubuntu, gcc, ssl=libressl=3.0.2: Fails. -> There are probably two separate issues. One with LibreSSL and one that is not related to SSL. This is more useful than: - Ubuntu, gcc, all features: Fails. - Ubuntu, gcc, ssl=libressl=3.0.2: Fails. -> This might be an issue with SSL in general or two separate issues. We don't know because we have nothing to compare. - Ubuntu, gcc, all features: Works. - Ubuntu, gcc, ssl=stock: Fails. -> There is probably some issue with SSL combined with something else. - Ubuntu, gcc, all features: Fails. - Ubuntu, gcc, ssl=stock: Works. -> There is probably some issue that is not related to SSL. - Ubuntu, gcc, all features: Fails. - Ubuntu, gcc, ssl=stock: Works. - Ubuntu, gcc, ssl=libressl=3.0.2: Fails. -> There is probably some issue in LibreSSL only. > Ubuntu, gcc, gz=slz=1 > this one is "slz only, no other features". should we run slz + all features > enabled instead ? (same for clang) No. As with SSL this is intentional, testing only the specific difference of using slz instead of zlib in a "controlled" environment. There is no need to run, e.g. all the SSL tests with slz, because they don't enable compression anyway. > > Ubuntu, gcc, gz=zlib=1 > should we drop this one in favour of "gcc all features enabled" ? No. Same reason. Specific controlled environment as a direct comparison to easily detect differences that are caused by switching out the compression library. If zlib fails and slz does not we can't easily see if it is caused by a bug in zlib integration or by some issue in the combination of the various features. ----------- I would be fine with running e.g. the SLZ test / LibreSSL tests only in cron to keep resource usage low. But we must keep them separate to keep things simple and easy to understand instead of implicitly testing stuff by including them somewhere else. That was one of my biggest concerns with the current Travis situation. > > Ubuntu, gcc, ssl=libressl=3.0.2 > I think we can drop this one as well Okay. > > (few things like 51 degree, prometheus, PCRE2 to be discussed later) > - Put 51d, Prometheus into the "all features" tests, that's what they are for. - PCRE 2 should get a dedicated pair of tests testing only regex similar to slz + zlib that only test compression. Best regards Tim Düsterhus
