Hi Randall,
I think forking of the jcr-test module into its own release cycle
definitely makes sense. The easiest thing would be to just make it a
sub-project of Jackrabbit and move it from trunk/jackrabbit-jcr-tests to
jcr-tests/trunk. In addition we'd probably also need a separate issued
tracker and migrate existing issues.
However, this comes with some additional effort and complexity. Apart
from the initial move and fixing of existing dependencies we'd need a
release manager who takes care of the new sub-project. Not sure whether
Alex has enough cylces left to take this over.
Let's see what others think of this and then open an issue from the
conclusion.
Michael
On 7.6.12 15:00, Randall Hauch wrote:
First of all, I'd like to thank the whole Jackrabbit community for
maintaining and adding to the "jackrabbit-jcr-tests" suite of unit
tests. IIUC, these tests started out as the suite that is used by the
JSR-283 TCK, but which now include quite a few fixes and enhancements.
Having these tests is a huge plus for Jackrabbit but also for other JCR
implementations (including ModeShape [1][2] and Jackrabbit's own Oak
project [3]), since this large suite of independent tests helps any
implementation to provide behavior consistent with the specification.
The official JSR-283 TCK tests [4] was released in August 2009, and IIUC
was based on one of the "jackrabbit-jcr-tests" module in one of the
2.0.x releases of Jackrabbit. Since then, the official TCK hasn't been
updated or corrected, while the "jackrabbit-jcr-tests" have undergone
quite a few corrections and changes. (I recently learned of this subtle
distinction.)
Specifically, since the 2.0 release of Jackrabbit, all of the following
Jackrabbit issues changed some part of the unit tests in
"jackrabbit-jcr-tests" [5]:
JCR-3316: fix namespace name to be an absolute URI
JCR-3313 Changed a ColumnTest test to only check for required columns
JCR-3307: check supported option OPTION_ACTIVITIES_SUPPORTED
JCR-3301: fix typos in test method names and associated config files
JCR-3302: add missing refresh() call on test session
JCR-3300: make RSessionAccessControlTest extend the proper base class so
that the repo option check happens
JCR-3207: add TCK test for Info map of NODE_MOVED event on node reordering
JCR-3205 - Missing support for lock timeout and ownerHint in jcr-server
JCR-3196: reduce deprecation warning noise in TCK testsReduce warning noise
JCR-3195: fix assumptions about null-ness of lock token
JCR-3177: Remove jdk 1.4 restriction for jcr-tests
JCR-3051: AbstractLockTest.testLockExpiration fails intermittently
JCR-2903: Session.importXml should close the input stream (as to JSR
283/JCR 2.0)
JCR-2756: Shareable node test failures
JCR-2719: Incorrect outer join TCK tests
JCR-2693: Logging per test case
JCR-2665: JCR Test for Adding Node Type Tests That Abstract Nodes Can Be
Added as Children, contrary to JCR 2.0 specification
JCR-2663: JCR unit tests use invalid queries
JCR-2662: check for repository support of journaled observation,
otherwise skip test
JCR-2648: PropertyImpl.getNode() and NamePropertyTest use different
exception than documented in the JCR API JavaDoc
JCR-2536: spi2davex: InvalidItemStateException not properly extracted
from ambiguous response error
JCR-2565: spi2dav: Overwrite header T specified for MOVE and COPY causes
JCR-2515: ISO8601 uses default DecimalFormat constructor using locale
specific digits
JCR-2490: jackrabbit wrongly think nodetype is changed on nodetype
re-registration
JCR-2104: JSR 283 Full Versioning - checkpoint must be aware of current
activity for the checkout part
JCR-515: Enhance test data
IIUC most/all of them are still unfixed in the official TCK, and this
can cause quite a bit of grief for other implementations. Plain and
simple, the "jackrabbit-jcr-tests" module contains tests that are more
accurately verifying the expectations of the JSR-283 specification than
the official TCK. And this is why ModeShape adopted has for years used
the "jackrabbit-jcr-tests" in its builds, but in doing so we've run into
some challenges.
Since the "jackrabbit-jcr-tests" module is part of the Jackrabbit
codebase, it is released along with the rest of Jackrabbit. Of course,
this makes perfect sense for the Jackrabbit community. But considering
that the "jackrabbit-jcr-tests" are used by at least two other OSS
projects as essentially a "more correct and maintained" proxy for the
TCK tests, this inclusion in the Jackrabbit releases means the TCK tests
are not as frequently released as we'd hope.
Is there any interest in separating the "jackrabbit-jcr-tests"? I'm not
sure if that means a completely separate (sub)project, or whether there
are better alternatives (e.g., a simple project on GitHub?). Separating
the tests would have a number of benefits:
1. The "TCK unit tests" could be released on their own schedule, with
version numbers that are semantically related to the changes made
within that release
2. The JIRA issues would better reflect the work that's been done
(e.g., release notes), the number of changes, and the outstanding
issues (see below)
3. Emphasis can be placed on encouraging reuse and participation by
other projects, and ensuring that the tests are as correct as possible
4. This can possibly form a basis for either updating the official TCK
(if that's even possible) or providing a form that excludes/disables
the incorrect tests and includes fixed tests (per the TCK appeals
process), making the process much easier for those submitting
results (with appeals) as well as those reviewing/approving results.
5. Perhaps allow for more tests to be added, though this may be
difficult if these unit tests do form the basis for an updated
official TCK. (ModeShape has a number of tests that we haven't yet
submitted, since we thought until recently that the official TCK was
periodically updated with the latest "jackrabbit-jcr-tests".)
As mentioned, there are currently 10 others issues that are still open,
in progress, or have a patch available [6]:
JCR-3321Strange XPath query in OrderByMultiTypeTest.testMultipleOrder
JCR-3302Remove incorrect assumptions with respect to visibility of
changes done by other sessions
JCR-3301Remove incorrect test assumptions
JCR-3196reduce deprecation warning noise in TCK tests
JCR-3096AbstractObservationTest does not cover JCR 2.0 functionality
JCR-2756Shareable node test failures
JCR-2661Two JCR unit tests expect delete events for nodes under deleted
node, contrary to JCR 2.0 specification
JCR-2348Exclude problematic node types from tests
JCR-3322add TCK coverage of isNodeType(expandedName)
JCR-2666JCR TCK Test for Restoring Version Tests That Versionable Child
Is also Restored, contrary to JCR 2.0 specification
(JSR-1991 is also still open and in the report, but has to do more with
the OSGi bundling of the JAR.)
I have no doubt that at a minimum the Jackrabbit, ModeShape, and Oak
communities would very much like to get these resolved and would
continue to work together to build upon, improve and maintain the JCR
unit tests. But I believe that this will be even better if the JCR unit
tests can be separated.
Thanks for considering, and I look forward to discussing the possibilities.
Best regards,
Randall Hauch
Project lead, ModeShape
http://modeshape.org
[1] http://modeshape.org
[2] https://docs.jboss.org/author/display/MODE/Home
[3] http://wiki.apache.org/jackrabbit/Jackrabbit%203
[4] http://www.day.com/day/en/products/jcr/jsr-283.html
[5]
https://github.com/apache/jackrabbit/commits/trunk/jackrabbit-jcr-tests/src
[6]
https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+JCR+AND+component+%3D+jackrabbit-jcr-tests+AND+status+in+%28Open%2C+%22In+Progress%22%2C+%22Patch+Available%22%29+ORDER+BY+status+ASC%2C+key+DESC
<https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+JCR+AND+component+%3D+jackrabbit-jcr-tests+AND+status+in+%28Open%2C+%22In+Progress%22%2C+%22Patch+Available%22%29+ORDER+BY+status+ASC%2C+key+DESC>
