On Thu, Dec 19, 2013 at 2:44 PM, Ken Giusti <[email protected]> wrote:
> Sorry for top-posting. I'm trying to understand the consequences of what > you are proposing. > > First, as I understand it, there are two separate test suites in the > proton tree: one written in Java - containing 19 tests as you point out - > and a much larger one written in python. Each test suite exercises both > the Java and C implementations. By testing both implementations using the > same tests, we ensure consistency across the two implementations (some, as > a lot of the python tests are skipped) > > What you're proposing would remove the ability for the Java test suite to > exercise the C implementation, right? > Yes, it would remove the ability for Java tests to exercise C code by calling it through JNI. We do have the interop test suite which checks that there is common behaviour without using JNI, i.e. by comparing binary output of codec and such across languages and I can imagine a number of other ways we could test for common behaviour that would not involve Java code calling into C code, e.g. comparing protocol traces, or running interop scenarios over the wire. > > This means that only the python test suite would be used to ensure cross > implementation consistency (Java v. C), right? > Yes > > What doesn't change is the two python wrapper implementations - one that > wraps the Java API, the other wraps the C API - that are used by the python > test suite to test both implementations. We'd still have to keep both of > those sync'ed. > Correct, although I believe there are some ways that we can significantly improve the commonality between the two python wrapper implementations (shims), and I suspect this will also improve the consistency of the APIs as well. > > If all my assumptions above are correct, then I can live with this. It > still ensures (some) consistency checking between the two implementations. > Drift between the two python wrappers could be caught by the tests > themselves, so I'm not too concerned about that. > > I'd like to see that time saved maintaining two test beds invested in > bringing both implementations to parity - IMHO we're skipping far too many > tests due to feature disparity. > Agreed. --Rafael
