On Tue, Oct 12, 2010 at 4:44 PM, Roger Meier (JIRA) <j...@apache.org> wrote: > > I had the following things in mind: > # identify the language with the reference implementation (well defined > return values for all test cases) based on ThriftTest.thrift, probably Java > (lib/java/test/org/apache/thrift/server/ServerTestBase.java) or cpp > (test/cpp/src/TestClient.cpp test/cpp/src/TestServer.cpp)
I had part of that discussion with Bryan on IRC -- turns out neither cpp nor Java are complete implementations, so there's no such thing as 'a' reference (yet?). Part of my hope here is to map out the full description of 'who implements what'. That's something I had in mind when I said 'advanced' features and score cards. > # update the ThriftTest.thrift with details about the required return > values that have to be implemented So, the problem with ThriftTest as it stands is that it's a big mess of large and small things, that not all languages implement on an equal basis. I'm not even sure that all languages _can_ implement it all on an equal basis (see below). I'd like to either prune ThriftTest (or rewrite it with Simple, at some point the distinction between the two will be fairly academic, I suspect) down to something that is the clear minimum requirement for viable bindings. The reason I went with Simple instead of ThriftTest was that I could work without breaking tests for languages I can't even compile on my test machines, yet. > # update test implementations > # add some wrapper scripts to automate the cross language tests (as > Christian did) Actually, I'm hoping to get rid of my scripts once I find time to go through the autotools machinery once again; this was just the most expedient way to get the discussion going (and the c++ client bug reported). > # move language tests into their appropriate library directory (THRIFT-35) That's orthogonal to doing actual cross-language testing, but I agree needs to happen. It's more awkward once we start doing cross-language testing though. > # a public test server that supports multiple protocols and transports > could be another enhancement for testing purposes That's a good idea, but entirely orthogonal. Once that's said and done, it shouldn't be too hard to extend Simple as it stands to just run forever. > Today neary every language has already Test Server and Test Clients based on > ThriftTest.thrift , we just have to modify them a bit so that they have > exactly the same behavior. I think it is not worth to add another > SimpleTest.thrift, the target should be extending ThriftTest.thrift. This is where we disagree. I'm far from convinced all language are even capable of implementing a 'perfect enough' copy of the reference bindings. The first case I tripped over was the bit-length of integral types. Java, C++ and Haskell all support precise-lengths integrals... but Java only supports signed types (you can abuse them to represent unsigned values if you try hard enough). Perl doesn't particularly care about length of numbers; Ruby (I don't know ruby very well) seems to only support one size Fixnums (machine size, whatever that is on your machine -- how do we handle this, on some machines Fixnum maps to i32, on others to i64?); Python doesn't support i16 at all, only plain integers and long integers... and I'm not sure if they support byte either (python chars, IIRC are unicode characters, 32 bits long). Unicode is another fun topic -- not all versions of Python are completely unicode clean (though I doubt thrift supports python old enough for that to matter). I'm fairly sure at some point someone will provide bindings for a language where utf-8 support is going to be a major pain (VHDL anyone?) -- JavaScript comes to mind, if memory serves it treats strings as 16-bit chars, which isn't enough for all of unicode. Then we go in complex sets -- PHP doesn't support complex objects (sets, maps, etc) as map keys, but C++, Java and Haskell do no problem. Does that go in ThriftTest or not? It can't, otherwise PHP can't be tested. But if it's not, I can't test C++, Java and Haskell interoperability in its entirety. It's a mess once you get in the details of it, that's why I suggested the score card approach. > >> Test Framework harmonization across all languages >> ------------------------------------------------- >> >> Key: THRIFT-847 >> URL: https://issues.apache.org/jira/browse/THRIFT-847 >> Project: Thrift >> Issue Type: Improvement >> Components: C# - Library, C++ - Library, Cocoa - Library, Erlang - >> Library, Haskell - Library, Java - Library, JavaScript - Library, OCaml - >> Library, Perl - Library, PHP - Library, Python - Library, Ruby - Library, >> Test Suite >> Affects Versions: 0.1, 0.2, 0.3 >> Reporter: Roger Meier >> Assignee: Christian Lavoie >> Fix For: 0.6 >> >> Attachments: v1-WORK_IN_PROGRESS-unified_tests.tar.gz >> >> >> Today each Language supported by Thrift, have its own unit test, all are >> using the same Thrift IDL's located at the test directory. But the behavior >> of these tests seems to be different from language to language... this makes >> it difficult to do tests and bug fixing across different Languages. e.g. >> * C++ Test and JavaScript Test Server written in Java have different >> responses for the same services >> * C# and Java Test Server have different responses for testException as C++ >> I propose the following steps: >> * identify the language with the reference implementation (well defined >> return values for all test cases) >> * update the ThriftTest.thrift with details about the required return values >> that have to be implemented >> * update test implementations and move language tests into their appropriate >> library directory (THRIFT-35) >> * a public test server that supports multiple protocols and transports could >> be another enhancement for testing purposes >> I'm ready to help preparing patches, just tell me what you need! > > -- > This message is automatically generated by JIRA. > - > You can reply to this email to add a comment to the issue online. > > -- Have fun, Christian http://linkedin.christianlavoie.net "I won't let you fall apart."
