ThriftTest.thrift and existing TestServer and Clients are not that
important for me, but they already contains some tests with Base Types
and additional in deep tests with mapmap etc. It might be a starting
point.
One thing I like to avoid is tons of .thrift files with similar
functionality inside test...=> difficult to maintain. If it makes
sense, to add new ones, why not.
It might be worth to create a Ticket or TestSuite Wiki Page.
So we can do brainstroming and/or define the TestSuite and the IDL files. e.g.
- SimpleTypes.thrift (i16, i32)
- StringTypes.thrfit (utf-8, etc.)
- ComplexTypes.thrift
- Exceptions.Thrift
- ...
what can we reuse from today's IDL files? which IDL files will become
obsolete?
after defining the TestSuite in detail we can start implement that for
the different Languages and do interoperability tests, score card, etc.
another important topic might be recommendations like using that
datatype/feature results in incompatibility with blabla
Quoting Christian Lavoie <christian.lav...@gmail.com>:
On Tue, Oct 12, 2010 at 4:44 PM, Roger Meier (JIRA) <j...@apache.org> wrote:
I had the following things in mind:
# identify the language with the reference implementation (well
defined return values for all test cases) based on
ThriftTest.thrift, probably Java
(lib/java/test/org/apache/thrift/server/ServerTestBase.java) or cpp
(test/cpp/src/TestClient.cpp test/cpp/src/TestServer.cpp)
I had part of that discussion with Bryan on IRC -- turns out neither
cpp nor Java are complete implementations, so there's no such thing as
'a' reference (yet?). Part of my hope here is to map out the full
description of 'who implements what'. That's something I had in mind
when I said 'advanced' features and score cards.
# update the ThriftTest.thrift with details about the required
return values that have to be implemented
So, the problem with ThriftTest as it stands is that it's a big mess
of large and small things, that not all languages implement on an
equal basis. I'm not even sure that all languages _can_ implement it
all on an equal basis (see below).
I'd like to either prune ThriftTest (or rewrite it with Simple, at
some point the distinction between the two will be fairly academic, I
suspect) down to something that is the clear minimum requirement for
viable bindings. The reason I went with Simple instead of ThriftTest
was that I could work without breaking tests for languages I can't
even compile on my test machines, yet.
# update test implementations
# add some wrapper scripts to automate the cross language tests
(as Christian did)
Actually, I'm hoping to get rid of my scripts once I find time to go
through the autotools machinery once again; this was just the most
expedient way to get the discussion going (and the c++ client bug
reported).
# move language tests into their appropriate library directory (THRIFT-35)
That's orthogonal to doing actual cross-language testing, but I agree
needs to happen. It's more awkward once we start doing cross-language
testing though.
# a public test server that supports multiple protocols and
transports could be another enhancement for testing purposes
That's a good idea, but entirely orthogonal. Once that's said and
done, it shouldn't be too hard to extend Simple as it stands to just
run forever.
Today neary every language has already Test Server and Test Clients
based on ThriftTest.thrift , we just have to modify them a bit so
that they have exactly the same behavior. I think it is not worth
to add another SimpleTest.thrift, the target should be extending
ThriftTest.thrift.
This is where we disagree. I'm far from convinced all language are
even capable of implementing a 'perfect enough' copy of the reference
bindings. The first case I tripped over was the bit-length of integral
types. Java, C++ and Haskell all support precise-lengths integrals...
but Java only supports signed types (you can abuse them to represent
unsigned values if you try hard enough).
Perl doesn't particularly care about length of numbers; Ruby (I don't
know ruby very well) seems to only support one size Fixnums (machine
size, whatever that is on your machine -- how do we handle this, on
some machines Fixnum maps to i32, on others to i64?); Python doesn't
support i16 at all, only plain integers and long integers... and I'm
not sure if they support byte either (python chars, IIRC are unicode
characters, 32 bits long).
Unicode is another fun topic -- not all versions of Python are
completely unicode clean (though I doubt thrift supports python old
enough for that to matter). I'm fairly sure at some point someone will
provide bindings for a language where utf-8 support is going to be a
major pain (VHDL anyone?) -- JavaScript comes to mind, if memory
serves it treats strings as 16-bit chars, which isn't enough for all
of unicode.
Then we go in complex sets -- PHP doesn't support complex objects
(sets, maps, etc) as map keys, but C++, Java and Haskell do no
problem. Does that go in ThriftTest or not? It can't, otherwise PHP
can't be tested. But if it's not, I can't test C++, Java and Haskell
interoperability in its entirety.
It's a mess once you get in the details of it, that's why I suggested
the score card approach.
Test Framework harmonization across all languages
-------------------------------------------------
Key: THRIFT-847
URL: https://issues.apache.org/jira/browse/THRIFT-847
Project: Thrift
Issue Type: Improvement
Components: C# - Library, C++ - Library, Cocoa - Library,
Erlang - Library, Haskell - Library, Java - Library, JavaScript -
Library, OCaml - Library, Perl - Library, PHP - Library, Python -
Library, Ruby - Library, Test Suite
Affects Versions: 0.1, 0.2, 0.3
Reporter: Roger Meier
Assignee: Christian Lavoie
Fix For: 0.6
Attachments: v1-WORK_IN_PROGRESS-unified_tests.tar.gz
Today each Language supported by Thrift, have its own unit test,
all are using the same Thrift IDL's located at the test directory.
But the behavior of these tests seems to be different from
language to language... this makes it difficult to do tests and
bug fixing across different Languages. e.g.
* C++ Test and JavaScript Test Server written in Java have
different responses for the same services
* C# and Java Test Server have different responses for testException as C++
I propose the following steps:
* identify the language with the reference implementation (well
defined return values for all test cases)
* update the ThriftTest.thrift with details about the required
return values that have to be implemented
* update test implementations and move language tests into their
appropriate library directory (THRIFT-35)
* a public test server that supports multiple protocols and
transports could be another enhancement for testing purposes
I'm ready to help preparing patches, just tell me what you need!
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
--
Have fun, Christian
http://linkedin.christianlavoie.net
"I won't let you fall apart."
----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.
