On 16.12.2016 08:24, David Demelier wrote:
2016-12-15 17:25 GMT+01:00 Matthew Seaman <[email protected]>:
On 2016/12/15 16:01, Olivier Duchateau wrote:
The problem is that there are no tests in FreeBSD ports. All source
based systems I've tested: pkgsrc, FreeBSD ports, OpenBSD, Gentoo;
FreeBSD is the one that have the most instability. Not to mention
committers that commit without testing the port, just look at
www/redmine to get your point of view on that issue.
Are your serious when you said, there're no tests on FreeBSD ports. I
can tell you Xfce ports are tested with FreeBSD i386 9.3 and amd64
11.0 machines (on real hardware, no virtualization), and on poudriere
with Gtk+ 3.20 (port version is not not in ports tree, it's defaut
toolkits for the next stable release 4.14).
For the LXQt desktop is the same thing (tested with official ports
tree Qt5 and which one in plasma5 branch (on KDE repository).
I'm also working on the Pantheon desktop (desktop environment of
Elementary OS, I use Vala 0.30.2 and Vala 0.34.4, in order to test
stability of applications.
I use also OpenBSD macppc, it's piece of shit. WebKit browers are
broken, Xfce components crash often, stable branch is outdated, fix
are not propagated in stable branch. Personally I prefer the FreeBSD
scheme, because I'm sure it's quite stable.
Most port committers will run compile tests any time they update a port:
the better ones will test compilation on all supported FreeBSD versions
and all hardware architectures they have access to (ie. generally i386
and amd64).
I'm not talking about being sure that the port builds, but that the
software works. This is a next step that is too often forgotten. For
example I remember several years ago having a problem with
audio/mumble. The port was building fine, the window opened fine but
it was impossible to speak because there was a problem regarding the
CELT libraries IIRC.
I really support this, but from a committer perspective its quite
impossible. Often enough we just have no idea how the port needs to work
- so we must trust the maintainer. But too often there is none.
Additionally the package build cluster will rebuild any modified ports
within a few days for all of the OS versions and architectures the
project tries to provide ports for: that's yet another level of
validating the coding of the port itself.
However, I believe the OP's point is that *we do not routinely run the
software's own built-in regression tests for the packages we succeed in
building*. This is something that is slowly coming. For instance, you
can run 'make test' for many python, ruby or perl packages and see those
tests being run. TEST_DEPENDS is pretty much standardized as the way to
install dependencies required for testing nowadays.
Yes, I fully understand the requirements of such tests. I just would
like that maintainers test the port by building it and *by running*
them. This is time consuming for sure, but if the maintainers have no
time, then just keep an old but fully working version :-)
There are two problems with this:
1) Many ports have no maintainer
2) Even more ports have no tests at all
I'm a big fan for testing. I test every software i wrote and in every
firm i worked i had the lowest bug count. But if you try to teach other
people to write tests too you hit a hard wall...
So we need to address multiple problems. I'm currently working on an
project to improve the management of the ports-tree. Hopefully i can
free some time for all so we can test the runtime much better.
Greetings,
Torsten
_______________________________________________
[email protected] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "[email protected]"
