I should have known better than to send a sentence that starts with "of
course"... :)
We could support invalid data tests by having two top-level test trees. One
for schema tests and one for data tests.
tests/
schema_tests/
pass/
valid_schema.json
good_schema.json
fail/
bad_schema_1.json
broken_schema_2.json
out_in_left_field_schema.json
data_tests/
pass/
foo_test/
schema.json
json_data/
valid_json_data_for_schema.json
more_valid_json_data_matching_test_schema.json
binary_data/
valid_binary_data.bin
good_binary_data.bin
bar_test/
...
fail/
baz_test/
schema.json
json_data/
corrupt_invalid_data.json
simply_wrong_data.json
binary_data/
more_bad_data.bin
epic_fail_test/
schema.json
...
This layout allows for schema only tests and invalid data tests. It an
error for any schemas in the data "data_tests" directory to be invalid. All
schema in the data test directory are valid... it's the data that is wrong.
Alternatively, we could use the following layout (which I *think* I like
better)...
tests/
invalid_schemas/
broken.json
wrong.json
valid_data/
foo_test/
schema.json
json_data/
...
binary_data/
...
bar_test/
schema.json
json_data/
...
invalid_data/
baz_test/
schema.json
json_data/
...
binary_data/
...
The "json_data" and "binary_data" directories are optional so if you just
want to test the schema you don't create those subdirectories in the test.
As with the first layout, it is invalid to have a bad schema under the
"valid_data" or "invalid_data" directories (because, of course, we need a
valid schema to see if the data is (in)valid).
-Matt
On Thu, Nov 19, 2009 at 11:04 AM, Doug Cutting <cutt...@apache.org> wrote:
> Matt Massie wrote:
>
>> Of course, there would be no data tests for invalid schemas.
>>
>
> Although we probably do want some invalid data, to check that errors are
> caught.
>
> Doug
>
