On Thu, Dec 19, 2013 at 2:44 PM, Ken Giusti <kgiu...@redhat.com> wrote:
> Sorry for top-posting. I'm trying to understand the consequences of what
> you are proposing.
> First, as I understand it, there are two separate test suites in the
> proton tree: one written in Java - containing 19 tests as you point out -
> and a much larger one written in python. Each test suite exercises both
> the Java and C implementations. By testing both implementations using the
> same tests, we ensure consistency across the two implementations (some, as
> a lot of the python tests are skipped)
> What you're proposing would remove the ability for the Java test suite to
> exercise the C implementation, right?
Yes, it would remove the ability for Java tests to exercise C code by
calling it through JNI. We do have the interop test suite which checks that
there is common behaviour without using JNI, i.e. by comparing binary
output of codec and such across languages and I can imagine a number of
other ways we could test for common behaviour that would not involve Java
code calling into C code, e.g. comparing protocol traces, or running
interop scenarios over the wire.
> This means that only the python test suite would be used to ensure cross
> implementation consistency (Java v. C), right?
> What doesn't change is the two python wrapper implementations - one that
> wraps the Java API, the other wraps the C API - that are used by the python
> test suite to test both implementations. We'd still have to keep both of
> those sync'ed.
Correct, although I believe there are some ways that we can significantly
improve the commonality between the two python wrapper implementations
(shims), and I suspect this will also improve the consistency of the APIs
> If all my assumptions above are correct, then I can live with this. It
> still ensures (some) consistency checking between the two implementations.
> Drift between the two python wrappers could be caught by the tests
> themselves, so I'm not too concerned about that.
> I'd like to see that time saved maintaining two test beds invested in
> bringing both implementations to parity - IMHO we're skipping far too many
> tests due to feature disparity.