[Oak origin/trunk] Apache Jackrabbit Oak matrix - Build # 1071 - Still Failing

[Apache Jenkins Server]

The Apache Jenkins build system has built Apache Jackrabbit Oak matrix (build 
#1071)

Status: Still Failing

Check console output at 
https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/1071/ to view 
the results.

Changes:
[chetanm] OAK-4613 - Suggest directory should be opened in read only mode

[chetanm] OAK-3072 - LuceneIndexEditorTest#copyOnWriteAndLocks failing on 
windows

 

Test results:
9 tests failed.
FAILED:  
org.apache.jackrabbit.oak.plugins.document.rdb.RDBDocumentStoreJDBCTest.batchUpdateResult[RDBFixture:
 RDB-Derby(embedded)]

Error Message:
initializing RDB document store

Stack Trace:
org.apache.jackrabbit.oak.plugins.document.DocumentStoreException: initializing 
RDB document store
at 
org.apache.jackrabbit.oak.plugins.document.rdb.RDBDocumentStore.(RDBDocumentStore.java:211)
at 
org.apache.jackrabbit.oak.plugins.document.DocumentStoreFixture$RDBFixture.createDocumentStore(DocumentStoreFixture.java:168)
at 
org.apache.jackrabbit.oak.plugins.document.AbstractDocumentStoreTest.(AbstractDocumentStoreTest.java:45)
at 
org.apache.jackrabbit.oak.plugins.document.rdb.RDBDocumentStoreJDBCTest.(RDBDocumentStoreJDBCTest.java:60)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
at 
org.junit.runners.parameterized.BlockJUnit4ClassRunnerWithParameters.createTestUsingConstructorInjection(BlockJUnit4ClassRunnerWithParameters.java:43)
at 
org.junit.runners.parameterized.BlockJUnit4ClassRunnerWithParameters.createTest(BlockJUnit4ClassRunnerWithParameters.java:38)
at 
org.junit.runners.BlockJUnit4ClassRunner$1.runReflectiveCall(BlockJUnit4ClassRunner.java:266)
at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at 
org.junit.runners.BlockJUnit4ClassRunner.methodBlock(BlockJUnit4ClassRunner.java:263)
at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:27)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at 
org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at 
org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at 
org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at 
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at 
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Caused by: java.sql.SQLTransactionRollbackException: A lock could not be 
obtained within the time requested
at 
org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown 
Source)
at 
org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown 
Source)
at 
org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown 
Source)
at 

Re: [suggestion] introduce oak compatibility levels

[Vikas Saurabh]

Hi,

> But we could introduce a concept of
> 'compatibility levels' which are a set of features/behaviours that a
> particular oak version has and that application code relies upon. When
> creating a session by default the 'newest compatibility level' would be
> used, but applications could opt to use an older, compatibility level 1.2
> for example, when they still rely on the feature-set/behaviour oak had at
> that time.

My defensive side feels that doing compatibility type feature is
harder to maintain than to implement. Primary point being that such
cross cutting configurations affect code flows at arbitrary places.
That along with linear growth of compatibilities would, imo, be harder
to test and certify (to think at unit level, it might still work out
with really going rigorous with parametrized tests... but doing ITs
specially on stuff like part of repo was done at vX and run with
compatibility at vY... is vZ still ok with it?). I think being server
side (enterprisey??), oak does have a small luxury of relying on
acceptance testing type of processes with hard
deprecations/removal/changes.

I mean, I understand the requirement for such cases... but we should
probably don't add a feature for this... a hack to remain backward
compatible (e.g. session refresh due to apps used to relying on jr2)
should probably treated as such and removed going forward with a blunt
hammer. And, even if we don't remove this, this should probably just
remain as limited hack-specific thing.

Thanks,
Vikas

Re: [suggestion] introduce oak compatibility levels

[Julian Reschke]

On 2016-07-28 10:23, Stefan Egli wrote:

Hi,

Here's an idea that just came up in an offline discussion, not sure if it
has been around elsewhere. But we could introduce a concept of
'compatibility levels' which are a set of features/behaviours that a
particular oak version has and that application code relies upon. When
creating a session by default the 'newest compatibility level' would be
used, but applications could opt to use an older, compatibility level 1.2
for example, when they still rely on the feature-set/behaviour oak had at
that time. (This could also be based on service-user properties for
example.) As such, compatibility levels could be a vehicle to help properly
deprecate features over time (when you'd say eg oak 1.10 doesn't support
compatibility level oak 1.0 anymore).
...


/me very skeptical

Reasons:

- people copy stuff like that into their code without understanding what 
it means


- the versioning would be extremly coarse-grained

- this may result in having to maintain older behavior for much longer

Best regards, Julian

Re: [suggestion] introduce oak compatibility levels

[Stefan Egli]

Hi Michael,

On 28/07/16 10:54, "Michael Marth"  wrote:

>I think we should simply stick to SemVer of the released artefacts to
>signal those changes to upstream.

IIUC the difference would be that one version (eg oak 1.6) could contain
multiple compatibility versions (eg 1.2/1.4) - some perhaps marked as
deprecated - while as using SemVer you'd have to have multiple versions of
oak concurrently in an OSGi stack (which is likely not going to work) to
achieve the same. Compatibility levels would be more flexible than SemVer.

>On the more specific topic of session behaviour: could we use session
>attributes to let the app specify session behaviour? [1]

Yes, that would work too.

>[1] 
>https://docs.adobe.com/docs/en/spec/javax.jcr/javadocs/jcr-2.0/javax/jcr/S
>ession.html#getAttribute(java.lang.String)

Cheers,
Stefan

Session refresh behaviour (was: [suggestion] introduce oak compatibility levels)

[Michael Marth]

(Spawning this thread as it diverges away fro Stefan’s suggestion on signaling 
breaking changes)

Hi Thomas,

Looking at [1] I am surprised that the session would get refreshed in your 
example. 
Is that because in your example both sessions live in the same thread?

Thanks for clarifying!
Michael


[1] http://jackrabbit.apache.org/oak/docs/dos_and_donts.html



On 28/07/16 13:19, "Thomas Mueller"  wrote:

>Hi,
>
>>I agree if conflicts conceptually with MVCC. However: is there an actual
>>problem with the auto-refresh behaviour?
>
>Yes. For example with queries. If changes are made while iterating over
>the result of a query, the current behavior is problematic. Example code
>(simplified):
>
>RowIterator it = xxx.createQuery(...).execute().getRows();
>while (it.hasNext()) {
>otherSession.getNode(...).remove();
>otherSession.save();
>Row row = it.nextRow();
>Node node = row.getNode();
>-> node can be null here!
>}
>
>
>So basically the query result contains entries that get removed (by
>another session) while iterating over the result. So this can lead to
>NullPointerException and other strange behavior (you could get nodes that
>no _longer_ match the query constraints), depending on what you do
>exactly. Arguably it would be better if the session is isolated from
>changes done in another session in the same thread. By the way if using
>the same session to remove nodes and iterate over the result, the query
>result has to reflect the changes done by the session (I think this is
>required by the JCR spec).
>
>Regards,
>Thomas
>

Fwd: svn commit: r1754397 - in /jackrabbit/oak/trunk: oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/ oak-segment-tar/src/test/java/org/apache/jackrabbit/oak/segment/ oak-upgrade/src/

[Julian Reschke]

REMINDER...

new files need to be checked in with svn:eol-style=native, otherwise 
strange things can happen when passing around diffs between platforms 
with different eol preferences.


Everybody please check their SVN settings, or make sure that they 
manually check before adding files.


Best regards, Julian


 Forwarded Message 
Subject: svn commit: r1754397 - in /jackrabbit/oak/trunk: 
oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/ 
oak-segment-tar/src/test/java/org/apache/jackrabbit/oak/segment/ 
oak-upgrade/src/main/java/org/apache/jackrabbit/oak/upgrade/ oak-upgrade/...

Date: Thu, 28 Jul 2016 11:21:15 -
From: resc...@apache.org
Reply-To: oak-dev@jackrabbit.apache.org
To: oak-comm...@jackrabbit.apache.org

Author: reschke
Date: Thu Jul 28 11:21:15 2016
New Revision: 1754397

URL: http://svn.apache.org/viewvc?rev=1754397=rev
Log:
fix svn:eol-style

Modified:

jackrabbit/oak/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/RecordCacheStats.java 
  (props changed)


jackrabbit/oak/trunk/oak-segment-tar/src/test/java/org/apache/jackrabbit/oak/segment/NodeCacheTest.java 
  (props changed)


jackrabbit/oak/trunk/oak-segment-tar/src/test/java/org/apache/jackrabbit/oak/segment/WriteCacheManagerTest.java 
  (props changed)


jackrabbit/oak/trunk/oak-upgrade/src/main/java/org/apache/jackrabbit/oak/upgrade/IndexCopier.java 
  (props changed)


jackrabbit/oak/trunk/oak-upgrade/src/test/java/org/apache/jackrabbit/oak/upgrade/IncludeIndexTest.java 
  (props changed)


Propchange: 
jackrabbit/oak/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/RecordCacheStats.java

--
svn:eol-style = native

Propchange: 
jackrabbit/oak/trunk/oak-segment-tar/src/test/java/org/apache/jackrabbit/oak/segment/NodeCacheTest.java

--
svn:eol-style = native

Propchange: 
jackrabbit/oak/trunk/oak-segment-tar/src/test/java/org/apache/jackrabbit/oak/segment/WriteCacheManagerTest.java

--
svn:eol-style = native

Propchange: 
jackrabbit/oak/trunk/oak-upgrade/src/main/java/org/apache/jackrabbit/oak/upgrade/IndexCopier.java

--
svn:eol-style = native

Propchange: 
jackrabbit/oak/trunk/oak-upgrade/src/test/java/org/apache/jackrabbit/oak/upgrade/IncludeIndexTest.java

--
svn:eol-style = native

Re: [suggestion] introduce oak compatibility levels

[Thomas Mueller]

Hi,

>I agree if conflicts conceptually with MVCC. However: is there an actual
>problem with the auto-refresh behaviour?

Yes. For example with queries. If changes are made while iterating over
the result of a query, the current behavior is problematic. Example code
(simplified):

RowIterator it = xxx.createQuery(...).execute().getRows();
while (it.hasNext()) {
otherSession.getNode(...).remove();
otherSession.save();
Row row = it.nextRow();
Node node = row.getNode();
-> node can be null here!
}


So basically the query result contains entries that get removed (by
another session) while iterating over the result. So this can lead to
NullPointerException and other strange behavior (you could get nodes that
no _longer_ match the query constraints), depending on what you do
exactly. Arguably it would be better if the session is isolated from
changes done in another session in the same thread. By the way if using
the same session to remove nodes and iterate over the result, the query
result has to reflect the changes done by the session (I think this is
required by the JCR spec).

Regards,
Thomas

Re: Requirements for multiple Oak clients on the same backend (was: [suggestion] introduce oak compatibility levels)

[Bertrand Delacretaz]

Hi Michael,

On Thu, Jul 28, 2016 at 11:03 AM, Michael Marth  wrote:
> ...to my knowledge there is no formal way to find out which values of X and Y 
> are safe - at least so far

Ok, thanks for the info!

That might be something to keep in mind when designing new features -
detecting that you're running on an incompatible version of the
storage/indexes/whatever and loudly failing if the conversion cannot
happen on the fly.

(which is similar to Stefan's question about compatibility levels)

-Bertrand

Oak 1.5.7 release plan

[Davide Giannella]

Hello team,

I'm planning to cut Oak 1.5.7 on Monday 1st August roughly 10am BST.

If there are any objections please let me know. Otherwise I will
re-schedule any non-resolved issue for the next iteration.

Thanks
Davide

Re: Requirements for multiple Oak clients on the same backend (was: [suggestion] introduce oak compatibility levels)

[Michael Marth]

Hi Bertrand,

I believe this is unchartered territory.
It is (usually?) safe to assume that the persistence state written by Oak 
version X can be read and modified by version Y if Y > X.
However: version Y might introduce new features or perform changes on the 
state’s format, etc. When such a change is introduced it is not considered that 
version X might still operate on the same state.
For many values of X and Y your setup would probably work in practice. But to 
my knowledge there is no formal way to find out which values of X and Y are 
safe - at least so far. 

Michael




On 28/07/16 10:45, "Bertrand Delacretaz"  wrote:

>Hi,
>
>On Thu, Jul 28, 2016 at 10:23 AM, Stefan Egli  wrote:
>>...we could introduce a concept of
>> 'compatibility levels' which are a set of features/behaviours that a
>> particular oak version has and that application code relies upon
>
>Good timing, I have a related question about multiple client apps
>connecting to the same Oak backend.
>
>Say I have to Java apps A and B which use the same Oak/Mongo/BlobStore
>configuration, are there defined requirements as to the Oak library
>versions or other settings that A and B use?
>
>Do they need to use the exact same versions of the Oak bundles, and
>are violations to that or to other compatibility requirements
>detected?
>
>-Bertrand

Re: Requirements for multiple Oak clients on the same backend (was: [suggestion] introduce oak compatibility levels)

[Stefan Egli]

Don't have an answer, but there was a similar question recently on this
list:

"Does Oak core check the repository version ?"
http://markmail.org/thread/sbvjydwdu3g2eze5

Cheers,
Stefan

On 28/07/16 10:45, "Bertrand Delacretaz"  wrote:

>Hi,
>
>On Thu, Jul 28, 2016 at 10:23 AM, Stefan Egli 
>wrote:
>>...we could introduce a concept of
>> 'compatibility levels' which are a set of features/behaviours that a
>> particular oak version has and that application code relies upon
>
>Good timing, I have a related question about multiple client apps
>connecting to the same Oak backend.
>
>Say I have to Java apps A and B which use the same Oak/Mongo/BlobStore
>configuration, are there defined requirements as to the Oak library
>versions or other settings that A and B use?
>
>Do they need to use the exact same versions of the Oak bundles, and
>are violations to that or to other compatibility requirements
>detected?
>
>-Bertrand

Re: [suggestion] introduce oak compatibility levels

[Michael Marth]

Hi Stefan,

On the general question of deprecating features and breaking changes: I think 
we should simply stick to SemVer of the released artefacts to signal those 
changes to upstream.

On the more specific topic of session behaviour: could we use session 
attributes to let the app specify session behaviour? [1]

And on the very specific topic of auto-refresh: yes, this was introduced to 
ease application transition onto Oak. I agree if conflicts conceptually with 
MVCC. However: is there an actual problem with the auto-refresh behaviour?

Cheers
Michael

[1] 
https://docs.adobe.com/docs/en/spec/javax.jcr/javadocs/jcr-2.0/javax/jcr/Session.html#getAttribute(java.lang.String)





On 28/07/16 10:23, "Stefan Egli"  wrote:

>Hi,
>
>Here's an idea that just came up in an offline discussion, not sure if it
>has been around elsewhere. But we could introduce a concept of
>'compatibility levels' which are a set of features/behaviours that a
>particular oak version has and that application code relies upon. When
>creating a session by default the 'newest compatibility level' would be
>used, but applications could opt to use an older, compatibility level 1.2
>for example, when they still rely on the feature-set/behaviour oak had at
>that time. (This could also be based on service-user properties for
>example.) As such, compatibility levels could be a vehicle to help properly
>deprecate features over time (when you'd say eg oak 1.10 doesn't support
>compatibility level oak 1.0 anymore).
>
>One concrete case where this could have been useful is the
>backwards-compatible behaviour where a session is auto-refreshed when
>changes are done in another session. This seems counter-intuitive given
>MVCC, but was apparently introduced to remain jackrabbit-2-compatible.
>
>(Another slightly different example could be the warn about a session being
>open for too long without a refresh. This is likely an oversight from an
>application, but it could also be done on purpose (although I wouldn't know
>an example right now) - in which case this could be indicated via another
>flag on the session - but probably doesn't quite fit the compatibility-level
>approach).
>
>Opinions?
>
>Cheers,
>Stefan
>
>

Requirements for multiple Oak clients on the same backend (was: [suggestion] introduce oak compatibility levels)

[Bertrand Delacretaz]

Hi,

On Thu, Jul 28, 2016 at 10:23 AM, Stefan Egli  wrote:
>...we could introduce a concept of
> 'compatibility levels' which are a set of features/behaviours that a
> particular oak version has and that application code relies upon

Good timing, I have a related question about multiple client apps
connecting to the same Oak backend.

Say I have to Java apps A and B which use the same Oak/Mongo/BlobStore
configuration, are there defined requirements as to the Oak library
versions or other settings that A and B use?

Do they need to use the exact same versions of the Oak bundles, and
are violations to that or to other compatibility requirements
detected?

-Bertrand

Re: [suggestion] introduce oak compatibility levels

[Stefan Egli]

(typo)

On 28/07/16 10:23, "Stefan Egli"  wrote:

>One concrete case where this could have been useful is the
>backwards-compatible behaviour where a session is auto-refreshed when
>changes are done in another session.

.. in the same thread, that is ..