Re: Python, testing
On Mon, Jan 13, 2020 at 10:40 PM Hal Murray wrote: :::snip::: > > Any particular distro anyone wants it to run on? j/k > > The idea is NOT to run it as part of a normal checkin, but have something in > addition that could be triggered manually or by the equivalent of a cron job. > I'm thinking of weekly or monthly and before a release. > > In that context, we should run it on all distros. > > It's a very low probability of finding problems, but better that we find them > before the users do. I just added a rough version of them as merge request !1082 at GitLab. Minor points worth noting are that there are no working Debian/Ubuntu runners. The macOS and FreeBSD runners are untested. The OpenSUSE runners need a couple of mods to option-tester.sh There are other mods to option-tester.sh, generating a bit slicing number to output in case of detected errors, and the ability to use a custom Python binary like python3. I would prefer feedback only on GitLab, but I know some developers do not use it. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Python, testing
matthew.sel...@twosigma.com said:
> I'm not certain how these scripts are much different than our existing CI
> jobs... we already have CI jobs for both Python2 and Python3.
You can run them locally rather than waiting for the CI jobs to find problems.
tests/option-tester.sh tries to test all the options to waf and some of the
interesting combinations.
tests/python{2,3}-tester.sh is an easy way to run a minimal test of the other
python.
--
These are my opinions. I hate spam.
___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel
Re: Python, testing
> It is, I could throw together a merge request. I am not a CI expert though. > Next close person would be Matt Selsky I think. > Any particular distro anyone wants it to run on? j/k The idea is NOT to run it as part of a normal checkin, but have something in addition that could be triggered manually or by the equivalent of a cron job. I'm thinking of weekly or monthly and before a release. In that context, we should run it on all distros. It's a very low probability of finding problems, but better that we find them before the users do. -- These are my opinions. I hate spam. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Python, testing
On Mon, Jan 13, 2020 at 5:58 PM Eric S. Raymond via devel wrote: > > Hal Murray via devel : > > A year or 2 ago, I put together a script to test as many build time options > > as > > I thought reasonable. It's in ./tests/option-tester.sh > > > > Does anybody other than me use it? > > I've run it once or twice, but's not easty to see how to integraste > it into our regularr test process. > > > It's a bit of a CPU hog -- too much to run routinely. Can we set things up > > to > > run it on the gitlab OS collection weekly or manually when we get close to a > > release? > > I have to defer to the CI expers on that one. It sounds like something > that should be possible. It is, I could throw together a merge request. I am not a CI expert though. Next close person would be Matt Selsky I think. Any particular distro anyone wants it to run on? j/k ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Python, testing
Hal Murray via devel : > A year or 2 ago, I put together a script to test as many build time options > as > I thought reasonable. It's in ./tests/option-tester.sh > > Does anybody other than me use it? I've run it once or twice, but's not easty to see how to integraste it into our regularr test process. > It's a bit of a CPU hog -- too much to run routinely. Can we set things up > to > run it on the gitlab OS collection weekly or manually when we get close to a > release? I have to defer to the CI expers on that one. It sounds like something that should be possible. -- http://www.catb.org/~esr/;>Eric S. Raymond ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Python, testing
On Mon, Jan 13, 2020 at 05:06:01PM -0800, Hal Murray via devel wrote: > A year or 2 ago, I put together a script to test as many build time options > as > I thought reasonable. It's in ./tests/option-tester.sh > > Does anybody other than me use it? > > It's a bit of a CPU hog -- too much to run routinely. Can we set things up > to > run it on the gitlab OS collection weekly or manually when we get close to a > release? > > At the back end of each build step, it runs each of our python programs far > enough to print out their version string. That's far from a thorough test, > but a whole lot better than nothing. (Thanks to the people who put that in.) > > In particular, it does (should?) check loading the libraries. I think the > same code gets run post install. > > There is also a tests/python3-tester.sh that explicitly uses python3 > I added a clone for python2 a day or two ago. (but forgot to finish typing > this message) I'm not certain how these scripts are much different than our existing CI jobs... we already have CI jobs for both Python2 and Python3. Cheers, -Matt ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Python, testing
On Mon, Jan 13, 2020 at 05:06:01PM -0800, Hal Murray via devel wrote: > How does waf tell the c compiler which Python.h to use? > My system has: > /usr/include/python2.7/Python.h > /usr/include/python3.7m/Python.h ./waf --python=/path/to/python OR /path/to/python waf > What can we do about testing things like ntpq? > > Is there a ntpd running on the gitlab build boxes? Is it worthwhile to just > run commands without checking the answers? (catch crashes but not much else) Most of the build boxes are containers. There's no persistence, or daemons that exist beyond the time that the build is running. What sort of testing would you like to do with ntpq, specifically? Test specific sub-commands? What would that test look like? Cheers, -Matt ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Python, testing
>From Gary: > I find if I do not test on python2 that I quickly break code. A year or 2 ago, I put together a script to test as many build time options as I thought reasonable. It's in ./tests/option-tester.sh Does anybody other than me use it? It's a bit of a CPU hog -- too much to run routinely. Can we set things up to run it on the gitlab OS collection weekly or manually when we get close to a release? At the back end of each build step, it runs each of our python programs far enough to print out their version string. That's far from a thorough test, but a whole lot better than nothing. (Thanks to the people who put that in.) In particular, it does (should?) check loading the libraries. I think the same code gets run post install. There is also a tests/python3-tester.sh that explicitly uses python3 I added a clone for python2 a day or two ago. (but forgot to finish typing this message) - How does waf tell the c compiler which Python.h to use? My system has: /usr/include/python2.7/Python.h /usr/include/python3.7m/Python.h - What can we do about testing things like ntpq? Is there a ntpd running on the gitlab build boxes? Is it worthwhile to just run commands without checking the answers? (catch crashes but not much else) -- These are my opinions. I hate spam. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
