[
https://issues.apache.org/jira/browse/JCR-1452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12575308#action_12575308
]
Jukka Zitting commented on JCR-1452:
------------------------------------
> It would be strange to write three distinct tests, one with 500 MB, one with
> 50 KB, and another for 50 bytes.
I'm all OK for those tests to share code for example in a common base class,
i.e. have an AbstractBinaryTest base class and subclasses like
LargerThanMemoryBinaryTest, NormalBinaryTest, InlineBinaryTest for the distinct
cases. Each test has a different purpose and a different target code path, and
so I very much disagree that they are just differently scaled versions of the
same test.
> > we shouldn't just test things because we can.
>
> There are two kinds of tests: tests that are written after finding a bug, and
> tests that are written to prevent bugs.
Agreed. My point is that we should rather use a test coverage tool or something
similar to identify what tests we should add instead of just blindly scaling
existing tests up or down.
> Also, the TCK is not (yet) run regularly AFAIK.
Not the official TCK, but all the tests in it (jackrabbit-jcr-tests) are run
per each "mvn install" and CI build.
> I prefer to use the 'scale' system property
OK, but how should I use that? What does scale=1 mean? If I want to scale up,
should I set it to scale=2, scale=10, or scale=1000? What do those settings
mean, and how can I be sure that the tests I'm running are reasonably scaled?
Is the scale setting consistent across different tests?
For example, to cover the 500MB, 50kB, and 50b cases for the binary test
mentioned above, should I use scale=50, scale=50000, and scale=500000000?
Without knowing the details of the data store implementation and the associated
thresholds for example for inlining the bytes, how can I choose reasonable
scales so that all relevant scenarios get tested. I can't just blindly start
testing with scale 1, 2, 3, ...
The writer of the test case has always the best knowledge of what are the
important test cases and it should be up to him/her to specify the appropriate
settings to cover those cases.
> Make use of jackrabbit.test.scale in test cases
> -----------------------------------------------
>
> Key: JCR-1452
> URL: https://issues.apache.org/jira/browse/JCR-1452
> Project: Jackrabbit
> Issue Type: Improvement
> Components: jackrabbit-core
> Reporter: Marcel Reutegger
> Priority: Minor
> Fix For: 1.5
>
>
> There are already a number of longer running test cases in jackrabbit-core,
> but they are all disabled because they otherwise make building
> jackrabbit-core a very long task.
> Those tests should make use of the jackrabbit.test.scale property and per
> default (scale = 1) run within a short time.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
