I didn't chime in since I thought it was obviously a good idea. However, I strongly agree that the process of creating a test case needs to be as simple and documented as possible. I had a test case with my last pull request, but it required a fair amount of poking around to figure out how to best implement it (and this experience prompted the GSoC project).

Also, test cases may not make sense for some pull requests (e.g. documentation).

David Koes

Assistant Professor
Computational & Systems Biology
University of Pittsburgh

On 04/16/2018 02:31 AM, Noel O'Boyle wrote:
I'm disappointed in the lack of support for this. David Koes - you suggested a GSoc on this topic, for example. Anyway, for my part, I will write up info on adding testcases, but also start to nudge PR submitters towards including testcases.

- Noel

On 3 April 2018 at 13:15, Noel O'Boyle <baoille...@gmail.com 
<mailto:baoille...@gmail.com>> wrote:

    Noted. We will do this. In fact, I commit to doing this whether or not this 
proposal goes ahead.

    - Noel

    On 3 April 2018 at 11:54, David Hall <li...@cowsandmilk.net 
<mailto:li...@cowsandmilk.net>> wrote:

        I mostly agree, but I’ll mention the big hurdle I came across the first 
time I wrote a test for openbabel.

        Many times, one comes across a bug when using the command line tools, 
e.g. obabel , so debugging and testing
        goes through using that program.

        I think (correct if I’m wrong), the easiest way to go from an obabel 
command line run to a test is through
        writing a test in python like those that import functions from testbabel

        Having a document we can point to that walks them through that process, 
and shows them that it is quite easy,
        might be helpful. I know my first few times looking at the test 
directory, I said “well, I’m not using the
        python bindings, so I’ll ignore those files and try to figure out the 
c++ files and how they run tests”, when
        the reality is that the python files provide the easy route to testing 
the command line programs.


         > On Apr 3, 2018, at 4:42 AM, Noel O'Boyle <baoille...@gmail.com 
<mailto:baoille...@gmail.com>> wrote:
         > Hi all,
         > Very few PRs come with test cases. Basically, we just don't know if 
any of them do what they say, and even if
        they do, they probably will bit rot at a future date due to other PRs. 
The irony is that the person who wrote
        the code clearly tested it (one would assume) - but just didn't check 
in the test case. I would argue that the
        majority of developer time spent on Open Babel is on fixing bugs (or 
bitrotted code) which would never have
        existed in the first place if a test had been added.
         > When I refactored the code to handle implict valence, I relied on 
the small number of existing tests to help
        return the code back to its preexisting state. Anything that wasn't 
tested may (still) not be working. For
        example, recently David Koes found that my changes broke the PDBQT 
         > In short, I propose that we require a testcase for every PR. There 
may be special circumstances (huge test
        files, build system changes), but this would be the general rule. As a 
lower bar, one could imagine requiring
        them for every new feature implemented.
         > Regards,
         > - Noel
         > Check out the vibrant tech community on one of the world's most
         > engaging tech sites, Slashdot.org! 
         > OpenBabel-Devel mailing list
         > OpenBabel-Devel@lists.sourceforge.net 
         > https://lists.sourceforge.net/lists/listinfo/openbabel-devel

Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! 

OpenBabel-Devel mailing list

Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
OpenBabel-Devel mailing list

Reply via email to