On Mon, Oct 1, 2012 at 8:06 AM, Pete Batard <p...@akeo.ie> wrote: > On 2012.09.30 01:04, Chuck Cook wrote: >> One core issue that I see as a need to be addressed. Is a set of tests >> with baseline devices which exercise all functions of every API on every >> platform. No final release until all the test boxes are checked. >> Problems are being fixed, but are we introducing new ones? > > I've _started_ to create a testing wiki page: > https://github.com/libusbx/libusbx/wiki/Testing > I actually did that before your post, but soon got depressed at the idea > of how long I could make that table grow...
I can see that table grow if add OS variations to the picture. Win8 is different than Win7 than Vista than XP. > The one problem I see is, comprehensive unit testing is fine on paper, > when you have the resources. But I seriously see us having a major > constraint in that domain, and right now, I see rigorous comprehensive > unit testing as utterly unrealistic. I agree. > Just to give you an idea of the variations we would ideally want to test > on each release > - Linux, OS-X, MinGW, MSVC, WDK, cygwin, BSD > - gcc, Clang, MSVC with memory leaks detection, WDK with OACR, cross > compiling > - 32 bit, 64 bit > - Mass Storage, Composite, HID, other > - USB 2.0, USB 3.0 (including various proprietary USB 3.0 controllers on > Windows) For USB 3.0 under Windows 7 or below, I think we can only rely on bug reports and act upon that. It is a wild wild west as per Tim Roberts. > - Control, Bulk, Interrupt, Isoc I see the lack of good testing examples for Isoc (for Linux and Mac OS X for now as libusbx does not support Isoc under Windows now). > - sync, async, single threaded, multi-threaded, timeouts, etc. I see the lack of good testing examples for multithreaded async API. > - WinUSB, libusb-win32, libusb-win32 filter, libusbK > - Windows 7, Windows 8, Windows XP Maybe Vista should be added as well even though not many people are using it. > - external apps such as usbutils, OpenOCD, libusbK benchmark Not so sure how you test libusbK benchmark firmware. Right now I use libusbdotnet as a intermediate tools to do the tests but there is no libusbx port of the benchmark host test program. > - apps using libusb-compat Compilation of libusb-compat may be needed to test. But there are too many libusb-compat based program to test. > - shared, static lib > - libusb vs libusbx interoperability (for 1.0) As long as the API is stable (no breakage), maybe no need to test. > > Personally, I'm going to set the limit of what I'm ready to test for a > release to 20, and I can EASILY pick 20 from the above for Windows > alone, while being pretty sure that it will leave a major part of > libusbx/Windows still untested. Oh and anything autotools on Windows > takes forever to build (order of magnitude slower than Linux or OSX), so > this needs to be taken into account as well. For build tests under Cygwin, MinGW/Msys and MinGW-w64/Msys, usually I will test them. I usually also carry out build tests under Mac OS X (Lion, 64bit), OpenBSD and NetBSD and Ubuntu 12.04. But if somebody can contribute on a building machine with different platforms, that would be good. -- Xiaofan ------------------------------------------------------------------------------ Don't let slow site performance ruin your business. Deploy New Relic APM Deploy New Relic app performance management and know exactly what is happening inside your Ruby, Python, PHP, Java, and .NET app Try New Relic at no cost today and get our sweet Data Nerd shirt too! http://p.sf.net/sfu/newrelic-dev2dev _______________________________________________ libusbx-devel mailing list libusbx-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libusbx-devel
