On Tue, Oct 12, 2010 at 8:40 PM, <ro...@bufferoverflow.ch> wrote: > ThriftTest.thrift and existing TestServer and Clients are not that important > for me, but they already contains some tests with Base Types and additional > in deep tests with mapmap etc. It might be a starting point.
It might, but my fear was if I were to change anything in ThriftTest (or any of the existing tests) I risked breaking bindings I have no way to test currently. > One thing I like to avoid is tons of .thrift files with similar > functionality inside test...=> difficult to maintain. If it makes sense, to > add new ones, why not. Agreed. > It might be worth to create a Ticket or TestSuite Wiki Page. > So we can do brainstroming and/or define the TestSuite and the IDL files. > e.g. > > - SimpleTypes.thrift (i16, i32) > - StringTypes.thrfit (utf-8, etc.) > - ComplexTypes.thrift > - Exceptions.Thrift > - ... > > what can we reuse from today's IDL files? which IDL files will become > obsolete? I'd like to get a good grip on where we want to go first, to help decide what we can reuse. > after defining the TestSuite in detail we can start implement that for the > different Languages and do interoperability tests, score card, etc. Agreed. > another important topic might be recommendations like using that > datatype/feature results in incompatibility with blabla Yep; plenty of similar nice-to-haves fall out from a good test suite :) Thanks for the feedback Roger, much appreciated! > > Quoting Christian Lavoie <christian.lav...@gmail.com>: > >> On Tue, Oct 12, 2010 at 4:44 PM, Roger Meier (JIRA) <j...@apache.org> >> wrote: >>> >>> I had the following things in mind: >>> # identify the language with the reference implementation (well >>> defined return values for all test cases) based on ThriftTest.thrift, >>> probably Java (lib/java/test/org/apache/thrift/server/ServerTestBase.java) >>> or cpp (test/cpp/src/TestClient.cpp test/cpp/src/TestServer.cpp) >> >> I had part of that discussion with Bryan on IRC -- turns out neither >> cpp nor Java are complete implementations, so there's no such thing as >> 'a' reference (yet?). Part of my hope here is to map out the full >> description of 'who implements what'. That's something I had in mind >> when I said 'advanced' features and score cards. >> >>> # update the ThriftTest.thrift with details about the required return >>> values that have to be implemented >> >> So, the problem with ThriftTest as it stands is that it's a big mess >> of large and small things, that not all languages implement on an >> equal basis. I'm not even sure that all languages _can_ implement it >> all on an equal basis (see below). >> >> I'd like to either prune ThriftTest (or rewrite it with Simple, at >> some point the distinction between the two will be fairly academic, I >> suspect) down to something that is the clear minimum requirement for >> viable bindings. The reason I went with Simple instead of ThriftTest >> was that I could work without breaking tests for languages I can't >> even compile on my test machines, yet. >> >>> # update test implementations >>> # add some wrapper scripts to automate the cross language tests (as >>> Christian did) >> >> Actually, I'm hoping to get rid of my scripts once I find time to go >> through the autotools machinery once again; this was just the most >> expedient way to get the discussion going (and the c++ client bug >> reported). >> >>> # move language tests into their appropriate library directory >>> (THRIFT-35) >> >> That's orthogonal to doing actual cross-language testing, but I agree >> needs to happen. It's more awkward once we start doing cross-language >> testing though. >> >>> # a public test server that supports multiple protocols and transports >>> could be another enhancement for testing purposes >> >> That's a good idea, but entirely orthogonal. Once that's said and >> done, it shouldn't be too hard to extend Simple as it stands to just >> run forever. >> >>> Today neary every language has already Test Server and Test Clients based >>> on ThriftTest.thrift , we just have to modify them a bit so that they have >>> exactly the same behavior. I think it is not worth to add another >>> SimpleTest.thrift, the target should be extending ThriftTest.thrift. >> >> This is where we disagree. I'm far from convinced all language are >> even capable of implementing a 'perfect enough' copy of the reference >> bindings. The first case I tripped over was the bit-length of integral >> types. Java, C++ and Haskell all support precise-lengths integrals... >> but Java only supports signed types (you can abuse them to represent >> unsigned values if you try hard enough). >> >> Perl doesn't particularly care about length of numbers; Ruby (I don't >> know ruby very well) seems to only support one size Fixnums (machine >> size, whatever that is on your machine -- how do we handle this, on >> some machines Fixnum maps to i32, on others to i64?); Python doesn't >> support i16 at all, only plain integers and long integers... and I'm >> not sure if they support byte either (python chars, IIRC are unicode >> characters, 32 bits long). >> >> Unicode is another fun topic -- not all versions of Python are >> completely unicode clean (though I doubt thrift supports python old >> enough for that to matter). I'm fairly sure at some point someone will >> provide bindings for a language where utf-8 support is going to be a >> major pain (VHDL anyone?) -- JavaScript comes to mind, if memory >> serves it treats strings as 16-bit chars, which isn't enough for all >> of unicode. >> >> Then we go in complex sets -- PHP doesn't support complex objects >> (sets, maps, etc) as map keys, but C++, Java and Haskell do no >> problem. Does that go in ThriftTest or not? It can't, otherwise PHP >> can't be tested. But if it's not, I can't test C++, Java and Haskell >> interoperability in its entirety. >> >> It's a mess once you get in the details of it, that's why I suggested >> the score card approach. >> >>> >>>> Test Framework harmonization across all languages >>>> ------------------------------------------------- >>>> >>>> Key: THRIFT-847 >>>> URL: https://issues.apache.org/jira/browse/THRIFT-847 >>>> Project: Thrift >>>> Issue Type: Improvement >>>> Components: C# - Library, C++ - Library, Cocoa - Library, >>>> Erlang - Library, Haskell - Library, Java - Library, JavaScript - Library, >>>> OCaml - Library, Perl - Library, PHP - Library, Python - Library, Ruby - >>>> Library, Test Suite >>>> Affects Versions: 0.1, 0.2, 0.3 >>>> Reporter: Roger Meier >>>> Assignee: Christian Lavoie >>>> Fix For: 0.6 >>>> >>>> Attachments: v1-WORK_IN_PROGRESS-unified_tests.tar.gz >>>> >>>> >>>> Today each Language supported by Thrift, have its own unit test, all are >>>> using the same Thrift IDL's located at the test directory. But the behavior >>>> of these tests seems to be different from language to language... this >>>> makes >>>> it difficult to do tests and bug fixing across different Languages. e.g. >>>> * C++ Test and JavaScript Test Server written in Java have different >>>> responses for the same services >>>> * C# and Java Test Server have different responses for testException as >>>> C++ >>>> I propose the following steps: >>>> * identify the language with the reference implementation (well defined >>>> return values for all test cases) >>>> * update the ThriftTest.thrift with details about the required return >>>> values that have to be implemented >>>> * update test implementations and move language tests into their >>>> appropriate library directory (THRIFT-35) >>>> * a public test server that supports multiple protocols and transports >>>> could be another enhancement for testing purposes >>>> I'm ready to help preparing patches, just tell me what you need! >>> >>> -- >>> This message is automatically generated by JIRA. >>> - >>> You can reply to this email to add a comment to the issue online. >>> >>> >> >> >> >> -- >> Have fun, Christian >> http://linkedin.christianlavoie.net >> >> "I won't let you fall apart." >> >> >> > > > > ---------------------------------------------------------------- > This message was sent using IMP, the Internet Messaging Program. > -- Have fun, Christian http://linkedin.christianlavoie.net "I won't let you fall apart."
