Re: Unresolved Conflict Question

[Angela Schreiber]

Hi Jorge

Thanks for your contribution and the PR. I will take a look today.

Kind regards
Angela

From: Jorge Flórez 
Sent: Tuesday, December 6, 2022 19:54
To: oak-dev@jackrabbit.apache.org 
Subject: Re: Unresolved Conflict Question

EXTERNAL: Use caution when clicking on links or opening attachments.


Hi,
I am sorry for the extremely late reply. The last weeks have been very
difficult. I created https://issues.apache.org/jira/browse/OAK-10025 and I
think I created a pull request (I think, I am new in this matter).
I added this to the ticket:
https://github.com/apache/jackrabbit-oak/pull/786.


Best Regards.
Jorge

El mar, 15 nov 2022 a las 7:52, Angela Schreiber ()
escribió:

> hi jorge
>
> imho this would be a great addition to the oak documentation e.g. linked
> into the section https://jackrabbit.apache.org/oak/docs/dos_and_donts.html
>
> would it be possible for you to create ticket and a PR for oak-doc?
>
> kind regards
> angela
> 
> From: Jorge Flórez 
> Sent: Monday, November 14, 2022 21:32
> To: oak-dev@jackrabbit.apache.org 
> Subject: Re: Unresolved Conflict Question
>
> EXTERNAL: Use caution when clicking on links or opening attachments.
>
>
> Hi,
> after some reading, testing and debugging, I think I understand how it
> works. So I thought write something if anyone is in the same situation:
>
> - There are no nodes (I think) with a "conflict" state in a content
> repository. I searched the paths reported in my log with conflicts and
> found no indicator (special mixin, child nodes, properties, etc). The
> rep:MergeConflict is "added" by the AnnotatingConflictHandler in runtime so
> the ConflictValidator checks if the node "has" that mixin and then throws a
> CommitFailedException, discarding the changes that were about to be saved.
> It is not like that mixin is added to the node in the repository.
>
> - To avoid or minimize conflicts:
> 1. Try to keep the JCR sessions as short as possible. i.e. create the
> session, make changes, call session.save(), call session.logout. If you
> need to do something additional in the repository, a few lines after (maybe
> after some processing that could take some time), create the session again
> and repeat.
>
> 2. Try to use session.refresh(true) before saving, if you think that some
> significant time can pass between the login() and the session.save() call.
>
> 3. You could write your own conflict handler and add it when configuring
> your Oak or WhiteBoard instances. Only if you know what you are doing (i.e.
> you know how to resolve conflict in each one of the possible situations).
> By default, the AnnotatingConflictHandler instance will discard your
> changes and your commit will fail. The worst that will happen is that some
> changes were not persisted (if you are ok with that).
> Please check
> org.apache.jackrabbit.oak.plugins.commit.JcrLastModifiedConflictHandler. It
> seems like a good example to follow.
>
> 4. Enable the DEBUG level on
> org.apache.jackrabbit.oak.plugins.commit.MergingNodeStateDiff and
> org.apache.jackrabbit.oak.plugins.commit.ConflictValidator loggers if you
> want to have more information on the circumstances of a conflict that
> happened in a point of time.
>
> References
>
> https://cqdump.joerghoh.de/2015/11/02/aem-anti-pattern-long-running-sessions/
>
> https://cqdump.joerghoh.de/2015/12/22/how-can-i-avoid-oak-writemerge-conflicts/
> https://jackrabbit.apache.org/oak/docs/FAQ.html
>
> https://adapt.to/content/dam/adaptto/production/presentations/2015/adaptTo2015-Conflict-handling-with-Oak-Michael-Duerig-with-comments.pdf/_jcr_content/renditions/original./adaptTo2015-Conflict-handling-with-Oak-Michael-Duerig-with-comments.pdf
>
> Thanks.
>
> Jorge
>
>
> El mié, 5 oct 2022 a las 10:58, Jorge Flórez (<
> jorgeeduardoflo...@gmail.com>)
> escribió:
>
> > Hi,
> >
> > in a production log I have some messages like this one:
> >
> > javax.jcr.InvalidItemStateException: OakState0001: Unresolved conflicts
> in
> > /F/EDTG/2010286E/00_Hoja_Control_2017_T5.pdf
> > at
> >
> deployment.mpEcmEA.ear//org.apache.jackrabbit.oak.api.CommitFailedException.asRepositoryException(CommitFailedException.java:238)
> > at
> >
> deployment.mpEcmEA.ear//org.apache.jackrabbit.oak.api.CommitFailedException.asRepositoryException(CommitFailedException.java:213)
> > at
> >
> deployment.mpEcmEA.ear//org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.newRepositoryException(SessionDelegate.java:669)
> > at
> >
> deployment.mpEcmEA.ear//org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.save(SessionDelegate.java:495)
> >

Re: Unresolved Conflict Question

[Jorge Flórez]

Hi,
I am sorry for the extremely late reply. The last weeks have been very
difficult. I created https://issues.apache.org/jira/browse/OAK-10025 and I
think I created a pull request (I think, I am new in this matter).
I added this to the ticket:
https://github.com/apache/jackrabbit-oak/pull/786.


Best Regards.
Jorge

El mar, 15 nov 2022 a las 7:52, Angela Schreiber ()
escribió:

> hi jorge
>
> imho this would be a great addition to the oak documentation e.g. linked
> into the section https://jackrabbit.apache.org/oak/docs/dos_and_donts.html
>
> would it be possible for you to create ticket and a PR for oak-doc?
>
> kind regards
> angela
> 
> From: Jorge Flórez 
> Sent: Monday, November 14, 2022 21:32
> To: oak-dev@jackrabbit.apache.org 
> Subject: Re: Unresolved Conflict Question
>
> EXTERNAL: Use caution when clicking on links or opening attachments.
>
>
> Hi,
> after some reading, testing and debugging, I think I understand how it
> works. So I thought write something if anyone is in the same situation:
>
> - There are no nodes (I think) with a "conflict" state in a content
> repository. I searched the paths reported in my log with conflicts and
> found no indicator (special mixin, child nodes, properties, etc). The
> rep:MergeConflict is "added" by the AnnotatingConflictHandler in runtime so
> the ConflictValidator checks if the node "has" that mixin and then throws a
> CommitFailedException, discarding the changes that were about to be saved.
> It is not like that mixin is added to the node in the repository.
>
> - To avoid or minimize conflicts:
> 1. Try to keep the JCR sessions as short as possible. i.e. create the
> session, make changes, call session.save(), call session.logout. If you
> need to do something additional in the repository, a few lines after (maybe
> after some processing that could take some time), create the session again
> and repeat.
>
> 2. Try to use session.refresh(true) before saving, if you think that some
> significant time can pass between the login() and the session.save() call.
>
> 3. You could write your own conflict handler and add it when configuring
> your Oak or WhiteBoard instances. Only if you know what you are doing (i.e.
> you know how to resolve conflict in each one of the possible situations).
> By default, the AnnotatingConflictHandler instance will discard your
> changes and your commit will fail. The worst that will happen is that some
> changes were not persisted (if you are ok with that).
> Please check
> org.apache.jackrabbit.oak.plugins.commit.JcrLastModifiedConflictHandler. It
> seems like a good example to follow.
>
> 4. Enable the DEBUG level on
> org.apache.jackrabbit.oak.plugins.commit.MergingNodeStateDiff and
> org.apache.jackrabbit.oak.plugins.commit.ConflictValidator loggers if you
> want to have more information on the circumstances of a conflict that
> happened in a point of time.
>
> References
>
> https://cqdump.joerghoh.de/2015/11/02/aem-anti-pattern-long-running-sessions/
>
> https://cqdump.joerghoh.de/2015/12/22/how-can-i-avoid-oak-writemerge-conflicts/
> https://jackrabbit.apache.org/oak/docs/FAQ.html
>
> https://adapt.to/content/dam/adaptto/production/presentations/2015/adaptTo2015-Conflict-handling-with-Oak-Michael-Duerig-with-comments.pdf/_jcr_content/renditions/original./adaptTo2015-Conflict-handling-with-Oak-Michael-Duerig-with-comments.pdf
>
> Thanks.
>
> Jorge
>
>
> El mié, 5 oct 2022 a las 10:58, Jorge Flórez (<
> jorgeeduardoflo...@gmail.com>)
> escribió:
>
> > Hi,
> >
> > in a production log I have some messages like this one:
> >
> > javax.jcr.InvalidItemStateException: OakState0001: Unresolved conflicts
> in
> > /F/EDTG/2010286E/00_Hoja_Control_2017_T5.pdf
> > at
> >
> deployment.mpEcmEA.ear//org.apache.jackrabbit.oak.api.CommitFailedException.asRepositoryException(CommitFailedException.java:238)
> > at
> >
> deployment.mpEcmEA.ear//org.apache.jackrabbit.oak.api.CommitFailedException.asRepositoryException(CommitFailedException.java:213)
> > at
> >
> deployment.mpEcmEA.ear//org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.newRepositoryException(SessionDelegate.java:669)
> > at
> >
> deployment.mpEcmEA.ear//org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.save(SessionDelegate.java:495)
> > at
> >
> deployment.mpEcmEA.ear//org.apache.jackrabbit.oak.jcr.session.SessionImpl$8.performVoid(SessionImpl.java:420)
> > at
> >
> deployment.mpEcmEA.ear//org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.performVoid(SessionDelegate.java:273)
> > at
> >
> deployment.mpEcm

Re: Unresolved Conflict Question

[Angela Schreiber]

hi jorge

imho this would be a great addition to the oak documentation e.g. linked into 
the section https://jackrabbit.apache.org/oak/docs/dos_and_donts.html

would it be possible for you to create ticket and a PR for oak-doc?

kind regards
angela

From: Jorge Flórez 
Sent: Monday, November 14, 2022 21:32
To: oak-dev@jackrabbit.apache.org 
Subject: Re: Unresolved Conflict Question

EXTERNAL: Use caution when clicking on links or opening attachments.


Hi,
after some reading, testing and debugging, I think I understand how it
works. So I thought write something if anyone is in the same situation:

- There are no nodes (I think) with a "conflict" state in a content
repository. I searched the paths reported in my log with conflicts and
found no indicator (special mixin, child nodes, properties, etc). The
rep:MergeConflict is "added" by the AnnotatingConflictHandler in runtime so
the ConflictValidator checks if the node "has" that mixin and then throws a
CommitFailedException, discarding the changes that were about to be saved.
It is not like that mixin is added to the node in the repository.

- To avoid or minimize conflicts:
1. Try to keep the JCR sessions as short as possible. i.e. create the
session, make changes, call session.save(), call session.logout. If you
need to do something additional in the repository, a few lines after (maybe
after some processing that could take some time), create the session again
and repeat.

2. Try to use session.refresh(true) before saving, if you think that some
significant time can pass between the login() and the session.save() call.

3. You could write your own conflict handler and add it when configuring
your Oak or WhiteBoard instances. Only if you know what you are doing (i.e.
you know how to resolve conflict in each one of the possible situations).
By default, the AnnotatingConflictHandler instance will discard your
changes and your commit will fail. The worst that will happen is that some
changes were not persisted (if you are ok with that).
Please check
org.apache.jackrabbit.oak.plugins.commit.JcrLastModifiedConflictHandler. It
seems like a good example to follow.

4. Enable the DEBUG level on
org.apache.jackrabbit.oak.plugins.commit.MergingNodeStateDiff and
org.apache.jackrabbit.oak.plugins.commit.ConflictValidator loggers if you
want to have more information on the circumstances of a conflict that
happened in a point of time.

References
https://cqdump.joerghoh.de/2015/11/02/aem-anti-pattern-long-running-sessions/
https://cqdump.joerghoh.de/2015/12/22/how-can-i-avoid-oak-writemerge-conflicts/
https://jackrabbit.apache.org/oak/docs/FAQ.html
https://adapt.to/content/dam/adaptto/production/presentations/2015/adaptTo2015-Conflict-handling-with-Oak-Michael-Duerig-with-comments.pdf/_jcr_content/renditions/original./adaptTo2015-Conflict-handling-with-Oak-Michael-Duerig-with-comments.pdf

Thanks.

Jorge


El mié, 5 oct 2022 a las 10:58, Jorge Flórez ()
escribió:

> Hi,
>
> in a production log I have some messages like this one:
>
> javax.jcr.InvalidItemStateException: OakState0001: Unresolved conflicts in
> /F/EDTG/2010286E/00_Hoja_Control_2017_T5.pdf
> at
> deployment.mpEcmEA.ear//org.apache.jackrabbit.oak.api.CommitFailedException.asRepositoryException(CommitFailedException.java:238)
> at
> deployment.mpEcmEA.ear//org.apache.jackrabbit.oak.api.CommitFailedException.asRepositoryException(CommitFailedException.java:213)
> at
> deployment.mpEcmEA.ear//org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.newRepositoryException(SessionDelegate.java:669)
> at
> deployment.mpEcmEA.ear//org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.save(SessionDelegate.java:495)
> at
> deployment.mpEcmEA.ear//org.apache.jackrabbit.oak.jcr.session.SessionImpl$8.performVoid(SessionImpl.java:420)
> at
> deployment.mpEcmEA.ear//org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.performVoid(SessionDelegate.java:273)
> at
> deployment.mpEcmEA.ear//org.apache.jackrabbit.oak.jcr.session.SessionImpl.save(SessionImpl.java:417)
>
> I have been reading and I think I got some understanding, but I still have
> questions:
>
> 1 Does the past message mean that the node cannot be modified further in
> any way?
> 2 As far as I know, some conflict handlers are added to my JCR instance by
> default:
> RepMembersConflictHandler, JcrLastModifiedConflictHandler,
> AnnotatingConflictHandler and one wrapper. If my guess is correct, the
> "conflict" was managed by the AnnotatingConflictHandler which always
> returns *Resolution.THEIRS* and adds rep:MergeConflict mixin to the node.
> Does this mean that all I have to do is remove the mixin to get rid of
> "OakState0001: Unresolved conflicts"?
> 3 Or should I write my own handler that for example always returns 
> *Resolution.THEIRS
> *and that's it? or in the handler I have to manually make the respective
> changes in all of the events?
>
> I hope I have been clear. Thanks in advance.
>
> Regards.
>
> Jorge
>
>

Re: Unresolved Conflict Question

[Jorge Flórez]

Hi,
after some reading, testing and debugging, I think I understand how it
works. So I thought write something if anyone is in the same situation:

- There are no nodes (I think) with a "conflict" state in a content
repository. I searched the paths reported in my log with conflicts and
found no indicator (special mixin, child nodes, properties, etc). The
rep:MergeConflict is "added" by the AnnotatingConflictHandler in runtime so
the ConflictValidator checks if the node "has" that mixin and then throws a
CommitFailedException, discarding the changes that were about to be saved.
It is not like that mixin is added to the node in the repository.

- To avoid or minimize conflicts:
1. Try to keep the JCR sessions as short as possible. i.e. create the
session, make changes, call session.save(), call session.logout. If you
need to do something additional in the repository, a few lines after (maybe
after some processing that could take some time), create the session again
and repeat.

2. Try to use session.refresh(true) before saving, if you think that some
significant time can pass between the login() and the session.save() call.

3. You could write your own conflict handler and add it when configuring
your Oak or WhiteBoard instances. Only if you know what you are doing (i.e.
you know how to resolve conflict in each one of the possible situations).
By default, the AnnotatingConflictHandler instance will discard your
changes and your commit will fail. The worst that will happen is that some
changes were not persisted (if you are ok with that).
Please check
org.apache.jackrabbit.oak.plugins.commit.JcrLastModifiedConflictHandler. It
seems like a good example to follow.

4. Enable the DEBUG level on
org.apache.jackrabbit.oak.plugins.commit.MergingNodeStateDiff and
org.apache.jackrabbit.oak.plugins.commit.ConflictValidator loggers if you
want to have more information on the circumstances of a conflict that
happened in a point of time.

References
https://cqdump.joerghoh.de/2015/11/02/aem-anti-pattern-long-running-sessions/
https://cqdump.joerghoh.de/2015/12/22/how-can-i-avoid-oak-writemerge-conflicts/
https://jackrabbit.apache.org/oak/docs/FAQ.html
https://adapt.to/content/dam/adaptto/production/presentations/2015/adaptTo2015-Conflict-handling-with-Oak-Michael-Duerig-with-comments.pdf/_jcr_content/renditions/original./adaptTo2015-Conflict-handling-with-Oak-Michael-Duerig-with-comments.pdf

Thanks.

Jorge


El mié, 5 oct 2022 a las 10:58, Jorge Flórez ()
escribió:

> Hi,
>
> in a production log I have some messages like this one:
>
> javax.jcr.InvalidItemStateException: OakState0001: Unresolved conflicts in
> /F/EDTG/2010286E/00_Hoja_Control_2017_T5.pdf
> at
> deployment.mpEcmEA.ear//org.apache.jackrabbit.oak.api.CommitFailedException.asRepositoryException(CommitFailedException.java:238)
> at
> deployment.mpEcmEA.ear//org.apache.jackrabbit.oak.api.CommitFailedException.asRepositoryException(CommitFailedException.java:213)
> at
> deployment.mpEcmEA.ear//org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.newRepositoryException(SessionDelegate.java:669)
> at
> deployment.mpEcmEA.ear//org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.save(SessionDelegate.java:495)
> at
> deployment.mpEcmEA.ear//org.apache.jackrabbit.oak.jcr.session.SessionImpl$8.performVoid(SessionImpl.java:420)
> at
> deployment.mpEcmEA.ear//org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.performVoid(SessionDelegate.java:273)
> at
> deployment.mpEcmEA.ear//org.apache.jackrabbit.oak.jcr.session.SessionImpl.save(SessionImpl.java:417)
>
> I have been reading and I think I got some understanding, but I still have
> questions:
>
> 1 Does the past message mean that the node cannot be modified further in
> any way?
> 2 As far as I know, some conflict handlers are added to my JCR instance by
> default:
> RepMembersConflictHandler, JcrLastModifiedConflictHandler,
> AnnotatingConflictHandler and one wrapper. If my guess is correct, the
> "conflict" was managed by the AnnotatingConflictHandler which always
> returns *Resolution.THEIRS* and adds rep:MergeConflict mixin to the node.
> Does this mean that all I have to do is remove the mixin to get rid of
> "OakState0001: Unresolved conflicts"?
> 3 Or should I write my own handler that for example always returns 
> *Resolution.THEIRS
> *and that's it? or in the handler I have to manually make the respective
> changes in all of the events?
>
> I hope I have been clear. Thanks in advance.
>
> Regards.
>
> Jorge
>
>

Re: I have a question

[Julian Reschke]

On 07.11.2022 19:17, Brayan Castillo wrote:

Good afternoon, it's a pleasure to greet you
I apologize if the context of my subject is not related to this email,
and my English is not very good.

I'm still a little newbie using Apache Jackrabbit.
I just have a little problem running jackrabbit-standalone on Linux, i
have already worked with this server on Windows, but now on Linux it
gives me an error like below:

Screenshot from 2022-11-07 12-13-01.png

followed by other lines...

It had already given me some problems before in Windows, but I solved them
However, I have not been able to solve this error, I will appreciate it
if you can help me.

Thank you so much.

Cheers.


What version of Jackrabbit are you using, and what is the Java version?

Best regards, Julian

I have a question

[Brayan Castillo]

Good afternoon, it's a pleasure to greet you
I apologize if the context of my subject is not related to this email, and
my English is not very good.

I'm still a little newbie using Apache Jackrabbit.
I just have a little problem running jackrabbit-standalone on Linux, i have
already worked with this server on Windows, but now on Linux it gives me an
error like below:

[image: Screenshot from 2022-11-07 12-13-01.png]

followed by other lines...

It had already given me some problems before in Windows, but I solved them
However, I have not been able to solve this error, I will appreciate it if
you can help me.

Thank you so much.

Cheers.

Unresolved Conflict Question

[Jorge Flórez]

Hi,

in a production log I have some messages like this one:

javax.jcr.InvalidItemStateException: OakState0001: Unresolved conflicts in
/F/EDTG/2010286E/00_Hoja_Control_2017_T5.pdf
at
deployment.mpEcmEA.ear//org.apache.jackrabbit.oak.api.CommitFailedException.asRepositoryException(CommitFailedException.java:238)
at
deployment.mpEcmEA.ear//org.apache.jackrabbit.oak.api.CommitFailedException.asRepositoryException(CommitFailedException.java:213)
at
deployment.mpEcmEA.ear//org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.newRepositoryException(SessionDelegate.java:669)
at
deployment.mpEcmEA.ear//org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.save(SessionDelegate.java:495)
at
deployment.mpEcmEA.ear//org.apache.jackrabbit.oak.jcr.session.SessionImpl$8.performVoid(SessionImpl.java:420)
at
deployment.mpEcmEA.ear//org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.performVoid(SessionDelegate.java:273)
at
deployment.mpEcmEA.ear//org.apache.jackrabbit.oak.jcr.session.SessionImpl.save(SessionImpl.java:417)

I have been reading and I think I got some understanding, but I still have
questions:

1 Does the past message mean that the node cannot be modified further in
any way?
2 As far as I know, some conflict handlers are added to my JCR instance by
default:
RepMembersConflictHandler, JcrLastModifiedConflictHandler,
AnnotatingConflictHandler and one wrapper. If my guess is correct, the
"conflict" was managed by the AnnotatingConflictHandler which always
returns *Resolution.THEIRS* and adds rep:MergeConflict mixin to the node.
Does this mean that all I have to do is remove the mixin to get rid of
"OakState0001: Unresolved conflicts"?
3 Or should I write my own handler that for example always returns
*Resolution.THEIRS
*and that's it? or in the handler I have to manually make the respective
changes in all of the events?

I hope I have been clear. Thanks in advance.

Regards.

Jorge

Re: Question about the node deletion

[Julian Reschke]

Am 24.09.2020 um 17:56 schrieb fengiaw:

Hi devs,


I am working on JackRabbit Oak jcr, and have questions about the node deletion.
Our backend store is using mysql.
When I check the data in mysql, I can see two fields related to deletion, one is 
"_deleted" and the other one is "deletedOnce".
 From the documentation, "deletedOnce" only indicates if the doc is deleted 
once, what the prupose of this field?
 From these two fields, how could I determine whether a node is indeed deleted 
(Not using apis, e.g. session.nodeExists) ?
I was relying on the "_deleted" field, and tried to use the latest value of it 
to determine whether it's gotten deleted.
For example,
_deleted={r173bcb4360b-0-34d=false, r173bcb2fb8f-0-34d=false, 
r173bcaf6d2c-0-34e=false, r173bcae10fa-0-350=false, r173bcac5862-0-34d=false, 
r173bcac5859-0-34d=false, r173bc9bf9f0-0-342=false, r173bc9a42ce-0-343=false, 
r173bc994dce-0-34d=false, r173bc962853-0-34d=false, r173bc942ceb-0-34d=false, 
r173baa5a789-0-329=true, r1739196c343-1-2e9=false}.


The latest value is "r173bcb4360b-0-34d=false", base on this, I assume it's not 
deleted. But actually, I have deleted this node via our UI, and it disappears in explorer 
as well.
So the question here is from the mysql record, what is the best/correct way to 
determin a node indeed exists or gets deleted?


That's an implementation detail - you are not supposed to look at the DB.

That said, the details are contained in the source of DocumentStore
RevisionGC; essentially these are only flags that are used to find the
nodes that might qualify for permanent deletion (but might still be
needed, for instance, because they are referenced by a checkpointed state).

Best regards, Julian

Question about the node deletion

[fengiaw]

Hi devs,


I am working on JackRabbit Oak jcr, and have questions about the node deletion.
Our backend store is using mysql.
When I check the data in mysql, I can see two fields related to deletion, one 
is "_deleted" and the other one is "deletedOnce".
From the documentation, "deletedOnce" only indicates if the doc is deleted 
once, what the prupose of this field?
From these two fields, how could I determine whether a node is indeed deleted 
(Not using apis, e.g. session.nodeExists) ?
I was relying on the "_deleted" field, and tried to use the latest value of it 
to determine whether it's gotten deleted.
For example, 
_deleted={r173bcb4360b-0-34d=false, r173bcb2fb8f-0-34d=false, 
r173bcaf6d2c-0-34e=false, r173bcae10fa-0-350=false, r173bcac5862-0-34d=false, 
r173bcac5859-0-34d=false, r173bc9bf9f0-0-342=false, r173bc9a42ce-0-343=false, 
r173bc994dce-0-34d=false, r173bc962853-0-34d=false, r173bc942ceb-0-34d=false, 
r173baa5a789-0-329=true, r1739196c343-1-2e9=false}.


The latest value is "r173bcb4360b-0-34d=false", base on this, I assume it's not 
deleted. But actually, I have deleted this node via our UI, and it disappears 
in explorer as well.
So the question here is from the mysql record, what is the best/correct way to 
determin a node indeed exists or gets deleted?


Thank for your support,
Feng

Question about MongoAzureDataStoreBlobGCTest

[Matt Ryan]

Hi,

I'm trying to wrap up OAK-8105 (migrate AzureDataStore to latest Azure v12
SDK) and I ran into an issue with MongoAzureDataStoreBlobGCTest in oak-it.
Basically the superclass's MongoBlobGCTest.gcLongRunningBlobCollection test
was failing because not enough blobs were being deleted from the data
store.  I found if I lower the value of maxLastModifiedInterval to 1 the
test passes, but as expected it then doesn't pass for S3DataStore.  See [0].

I'm wondering, is there something special about the value of 5000 that is
the current value for maxLastModifiedInterval?  I could just modify this so
that the MongoAzureDataStoreBlobGCTest child class sets the value to 1 and
so that the MongoS3DataStoreBlobGCTest child leaves it at 5000, if that
solution is acceptable.


[0] -
https://github.com/apache/jackrabbit-oak/blob/bd66b3516a1c981f62bce7aa7cfda02bcc002797/oak-store-document/src/test/java/org/apache/jackrabbit/oak/plugins/document/MongoBlobGCTest.java#L357


-MR

Re: Question related to Root node and their management in V1.5.0

[Woonsan Ko]

On Wed, May 29, 2019 at 3:27 AM Richa Kumari  wrote:
>
> Hi Dev Team
>
>
>
> We are using Jackrabbit for our content repository and the version 1.5.0 is 
> getting used.  The application was developed by 3rd party in early 2005-2007 
> . We  are trying to understand the process of saving the XML file and mapping 
> with our dataset. We understood that our key is the node or folder in the 
> repository however we are not able to locate the direct relationship in the 
> table rep_binval.binval_data and the our foreign key. We also noticed that 
> system does not update the information but its retrieve and delete them and 
> create the updated one as the new record. Our target area is to understand 
> how the rep_prop or rep_node store the foreign key relation to retrieve the 
> data.
>

The tables are internal details which you're not supposed to direct
access. You should always use JCR API [1,2] instead to access the JCR
hierarchical data.

Regards,

Woonsan

[1] http://jackrabbit.apache.org/jcr/jcr-api.html
[2] https://docs.adobe.com/docs/en/spec/jcr/1.0/index.html

>
>
> Hoping to reach you guys about and get any help to understand the node and 
> especially root node which stores in the baseFolder. We also noticed that 
> when system starts up, it has the rootUUID value in the cached folder (folder 
> location \repository\meta)  but others are not created or stored inside any 
> other folder.
>
>
>
> Hope to see your reply soon.
>
>
>
> Thank you.
>
>
>
> Best Regards,
>
> Richa
>
>
>
> Please consider the environment before printing this message. CTIS, Inc. 
> CONFIDENTIAL COMMUNICATION This email and any files transmitted with it are 
> confidential and are intended solely for use by the individual or entity to 
> whom addressed. If you have received this email in error, do not forward, 
> print, or copy, this email. Instead, please immediately notify us by 
> telephone at (301) 948-3033 or by reply email to the sender. Please delete 
> this email and its attachments from your system and do not retain any copies. 
> Thank you.

Question related to Root node and their management in V1.5.0

[Richa Kumari]

Hi Dev Team

We are using Jackrabbit for our content repository and the version 1.5.0 is 
getting used.  The application was developed by 3rd party in early 2005-2007 . 
We  are trying to understand the process of saving the XML file and mapping 
with our dataset. We understood that our key is the node or folder in the 
repository however we are not able to locate the direct relationship in the 
table rep_binval.binval_data and the our foreign key. We also noticed that 
system does not update the information but its retrieve and delete them and 
create the updated one as the new record. Our target area is to understand how 
the rep_prop or rep_node store the foreign key relation to retrieve the data.

Hoping to reach you guys about and get any help to understand the node and 
especially root node which stores in the baseFolder. We also noticed that when 
system starts up, it has the rootUUID value in the cached folder (folder 
location \repository\meta)  but others are not created or stored inside any 
other folder.

Hope to see your reply soon.

Thank you.

Best Regards,
Richa

Please consider the environment before printing this message. CTIS, Inc. 
CONFIDENTIAL COMMUNICATION This email and any files transmitted with it are 
confidential and are intended solely for use by the individual or entity to 
whom addressed. If you have received this email in error, do not forward, 
print, or copy, this email. Instead, please immediately notify us by telephone 
at (301) 948-3033 or by reply email to the sender. Please delete this email and 
its attachments from your system and do not retain any copies. Thank you.

Re: Oak related question

[Kalach, Dmitry]

Thank you for your help.


От: Vikas Saurabh <vikas.saur...@gmail.com>
Отправлено: 16 марта 2018 г. 19:14:36
Кому: oak-dev@jackrabbit.apache.org
Тема: Re: Oak related question

Hi,

On Fri, Mar 16, 2018 at 9:07 AM, Kalach, Dmitry
<d.kal...@itransition.com> wrote:
> Sometimes it causes a multiple versions of files in Oak, like this
>
> "jcr:data" : {
> "r1622ec2af77-0-1" : 
> "\":blobId:0100c49320202c3c0269aea8e3cf205f4765e4c4e55a7102fae9d09a4307c05f0aafdac20954\"",
> "r1622ed13bac-0-c8" : 
> "\":blobId:0100c59320201328a6ecda51a2a57ab20e5d86173711f60ed07de212d2a81c918b3b56aecb63\"",
> "r1622ef69b14-0-ca" : 
> "\":blobId:0100cb932020be60317852ad8925c2c51d5c6df4092f6bdfbc961666e64779642a04146e70dd\""
> },
>
> We have nodes with 30+ available versions. But we do not need them. Oak run 
> shows this old versios as garbage, but revisions gc and blobs gc do not 
> deleted them.
>
> It is possible to delete all old versions for each node and leave only one 
> actual version?

While, I don't know of any way of "forcing" only 1 version to remain
for a binary reference - BUT,
https://issues.apache.org/jira/browse/OAK-5205 might be useful to drop
the reference rather early so that it can be garbage collected.

Thanks,
Vikas

Re: Oak related question

[Vikas Saurabh]

Hi,

On Fri, Mar 16, 2018 at 9:07 AM, Kalach, Dmitry
 wrote:
> Sometimes it causes a multiple versions of files in Oak, like this
>
> "jcr:data" : {
> "r1622ec2af77-0-1" : 
> "\":blobId:0100c49320202c3c0269aea8e3cf205f4765e4c4e55a7102fae9d09a4307c05f0aafdac20954\"",
> "r1622ed13bac-0-c8" : 
> "\":blobId:0100c59320201328a6ecda51a2a57ab20e5d86173711f60ed07de212d2a81c918b3b56aecb63\"",
> "r1622ef69b14-0-ca" : 
> "\":blobId:0100cb932020be60317852ad8925c2c51d5c6df4092f6bdfbc961666e64779642a04146e70dd\""
> },
>
> We have nodes with 30+ available versions. But we do not need them. Oak run 
> shows this old versios as garbage, but revisions gc and blobs gc do not 
> deleted them.
>
> It is possible to delete all old versions for each node and leave only one 
> actual version?

While, I don't know of any way of "forcing" only 1 version to remain
for a binary reference - BUT,
https://issues.apache.org/jira/browse/OAK-5205 might be useful to drop
the reference rather early so that it can be garbage collected.

Thanks,
Vikas

Oak RevisionGC, was: Oak related question

[Julian Reschke]

On 2018-03-16 09:07, Kalach, Dmitry wrote:

We're using Oak as JCR-implementation to save various files in our system.

This is usual in our system to change file and save it again.

Sometimes it causes a multiple versions of files in Oak, like this

"jcr:data" : {
 "r1622ec2af77-0-1" : 
"\":blobId:0100c49320202c3c0269aea8e3cf205f4765e4c4e55a7102fae9d09a4307c05f0aafdac20954\"",
 "r1622ed13bac-0-c8" : 
"\":blobId:0100c59320201328a6ecda51a2a57ab20e5d86173711f60ed07de212d2a81c918b3b56aecb63\"",
 "r1622ef69b14-0-ca" : 
"\":blobId:0100cb932020be60317852ad8925c2c51d5c6df4092f6bdfbc961666e64779642a04146e70dd\""
 },

We have nodes with 30+ available versions. But we do not need them. Oak run 
shows this old versios as garbage, but revisions gc and blobs gc do not deleted 
them.

It is possible to delete all old versions for each node and leave only one 
actual version?
...


See 
.


Best regards, Julian

Re: Oak related question

[Kalach, Dmitry]

We're using Oak as JCR-implementation to save various files in our system.

This is usual in our system to change file and save it again.

Sometimes it causes a multiple versions of files in Oak, like this

"jcr:data" : {
"r1622ec2af77-0-1" : 
"\":blobId:0100c49320202c3c0269aea8e3cf205f4765e4c4e55a7102fae9d09a4307c05f0aafdac20954\"",
"r1622ed13bac-0-c8" : 
"\":blobId:0100c59320201328a6ecda51a2a57ab20e5d86173711f60ed07de212d2a81c918b3b56aecb63\"",
"r1622ef69b14-0-ca" : 
"\":blobId:0100cb932020be60317852ad8925c2c51d5c6df4092f6bdfbc961666e64779642a04146e70dd\""
},

We have nodes with 30+ available versions. But we do not need them. Oak run 
shows this old versios as garbage, but revisions gc and blobs gc do not deleted 
them.

It is possible to delete all old versions for each node and leave only one 
actual version?


От: Angela Schreiber <anch...@adobe.com.INVALID>
Отправлено: 16 марта 2018 г. 10:59:04
Кому: oak-dev@jackrabbit.apache.org
Тема: Re: Oak related question

Hi

This mailing list the right channel to post questions regarding Oak.

Regards
Angela


On 16/03/18 08:19, "Kalach, Dmitry" <d.kal...@itransition.com> wrote:

>Hi
>
>We're using Jackrabbit Oak in our production.
>And I have a question about how it works. Unfortunately, I've failed to
>find any other email address to connect with oak developers.

Re: Oak related question

[Angela Schreiber]

Hi

This mailing list the right channel to post questions regarding Oak.

Regards
Angela


On 16/03/18 08:19, "Kalach, Dmitry" <d.kal...@itransition.com> wrote:

>Hi
>
>We're using Jackrabbit Oak in our production.
>And I have a question about how it works. Unfortunately, I've failed to
>find any other email address to connect with oak developers.

Oak related question

[Kalach, Dmitry]

Hi

We're using Jackrabbit Oak in our production.
And I have a question about how it works. Unfortunately, I've failed to find 
any other email address to connect with oak developers.

FW: Question Related to Jack Rabbit Oak

[Pal, Sushil (ext)]

Hi,

   I have a following question related to Jack Rabbit.

  1.  Jack Rabbit provide filesystem for document storage.

 2. c an we disable indexing(via lucene/solr) in Jack Rabbit.

 3. can we give customized versioning in Jack Rabbit.

 4.can we disable deletion in Jack Rabbit.


Best Regards,
Sushil Pal
Atos India Pvt. Ltd
Mobile : 9768644797

AW: Search Question ...

[KÖLL Claus]

Hi Clay !

Thank you for your answer but i'm talking about jackrabbit .. not oak ;-)

greets
claus

Re: Search Question ...

[Clay Ferguson]

You could look at the createIndexes() method in this class:

https://github.com/Clay-Ferguson/meta64/blob/master/src/main/java/com/meta64/mobile/repo/OakRepository.java

That class creates some "property indexes" and some "fulltext indexes".
It's best to have property indexes on the things you sort by (order by) or
do exact match searches for (A=B), and make a fulltext indexes for
substring searches. Also if you can use ISDESCENDANTNODE to narrow your
search I think that may help, but may not be possible in your case. Also
check to be sure that if you do just the jcr:contains() all by itself your
search should be super fast even with 10million, to verify lucene is
working using an index on the 'content' property. I don't know why you
needed that 'order by', because i though score was always the default
ordering, and i am not expert enough to know what "//element(" does, but
here's an example of some JCR_SQL2 that I think works (although not tested
at scale of millions):

https://github.com/Clay-Ferguson/meta64/blob/master/src/main/java/com/meta64/mobile/service/NodeSearchService.java


Best regards,
Clay Ferguson
wcl...@gmail.com


On Mon, Jan 2, 2017 at 6:27 AM, KÖLL Claus  wrote:

> Hi !
>
> First .. good new year to the whole community !
>
> We have in one of our workspace a lot of office documents (about 10
> Million) which are full text indexed.
> At the moment we have searches that take really long .. 1~3 Minutes ...
>
> The following xpath query will be executed ..
>
> "//element(*, dvt:document)[@dvt:referenceId = 'protid:123' and
> jcr:contains(jcr:content, 'tirol')] order by jcr:score()";
>
> Every node has a field called referenceid. In sum there are not really
> much documents who has this value.
> So the search "//element(*, dvt:document)[@dvt:referenceId = 'protid:123']
> order by jcr:score()";
> is quite fast.
>
> But in combination with the fulltextsearch on the content, it is really
> slow.
> The word 'tirol' is used in really much documents ...
>
> So I have tried to set a limit of 100 on the Query [query.setLimit(100)]
> but this has made the search not really faster.
> After struggling around the code I have found that the limit hint is
> handled in the TopFieldCollector.
> But the whole time is spent before the collector skips the results.
>
> I see a lot of info logs like this
> INFO  2017-01-02 13:13:10,610 org.apache.jackrabbit.core.
> query.lucene.DocNumberCache.size=107757/1, #accesses=428591,
> #hits=0, #misses=428591, cacheRatio=0%
>
> I think the BooleanScorer of the fulltext will be evaluated against all of
> its hits and therefore the docid to nodeid cache will be filled.
>
> Is it possible to give the limit hint to the fulltext scorer ? or maybe I
> do not understand it and somebody give me some hints how I can make the
> search faster ..
>
> thanks
> claus
>

Search Question ...

[KÖLL Claus]

Hi !

First .. good new year to the whole community !

We have in one of our workspace a lot of office documents (about 10 Million) 
which are full text indexed.
At the moment we have searches that take really long .. 1~3 Minutes ...

The following xpath query will be executed ..

"//element(*, dvt:document)[@dvt:referenceId = 'protid:123' and 
jcr:contains(jcr:content, 'tirol')] order by jcr:score()";

Every node has a field called referenceid. In sum there are not really much 
documents who has this value.
So the search "//element(*, dvt:document)[@dvt:referenceId = 'protid:123'] 
order by jcr:score()";
is quite fast.

But in combination with the fulltextsearch on the content, it is really slow.
The word 'tirol' is used in really much documents ...

So I have tried to set a limit of 100 on the Query [query.setLimit(100)] but 
this has made the search not really faster.
After struggling around the code I have found that the limit hint is handled in 
the TopFieldCollector.
But the whole time is spent before the collector skips the results.

I see a lot of info logs like this
INFO  2017-01-02 13:13:10,610 
org.apache.jackrabbit.core.query.lucene.DocNumberCache.size=107757/1, 
#accesses=428591, #hits=0, #misses=428591, cacheRatio=0%

I think the BooleanScorer of the fulltext will be evaluated against all of its 
hits and therefore the docid to nodeid cache will be filled.

Is it possible to give the limit hint to the fulltext scorer ? or maybe I do 
not understand it and somebody give me some hints how I can make the search 
faster ..

thanks
claus

Re: Question about OAK performance

[Davide Giannella]

On 08/11/2016 21:49, Ahmed Musallam wrote:
> Hello Dev team,

Hello, someone else may correct me but here's my take

>
> I am working on an AEM project where we have a large number of child nodes 
> that need to be created. I have a few questions around performance.
>
> We are using AEM 6.1 which utilizes Oak 1.2.7.
>
> A few questions:
>
> 1. I know that there is no “actual” limit to child nodes, but In  your 
> experience, is there a limit to child nodes over which we will see 
> performance impact? or instability of some sort? and what would be your 
> recommended “virtual” number of child nodes?

Depending on the node type. For oak:Unstructured I don't recall any
limit off the top of my head. If you use an ordered node you may start
experiencing performance degradation from about the 1k siblings. On
write. But it really depends as well by the machine power and the
current load.

> 2. Will read/writing one node which has, lets say 10,000 sibling be 
> problematic for any reason?

Writing of an ordered node, like nt:unstructured may be problematic. Not
reading.

> 3. I know that there are node types that have ordered vs unordered children. 
> When does Oak actually order the child nodes? on read?

The order is persisted on write. When reading it simply reads a hidden
property.

My main concern is about maintenance of such bucket. Do you plan to
manually navigate, maintain something like 1 siblings. Have you ever
tried browsing something way smaller (like 100) from the UI? Then, you
may actually hit limits way earlier from the UI. For example I would say
that a tree javascript UI may start having problems around the 200-500
siblings.

HTH
Davide

Question about OAK performance

[Ahmed Musallam]

Hello Dev team,

I am working on an AEM project where we have a large number of child nodes that 
need to be created. I have a few questions around performance.

We are using AEM 6.1 which utilizes Oak 1.2.7.

A few questions:

1. I know that there is no “actual” limit to child nodes, but In  your 
experience, is there a limit to child nodes over which we will see performance 
impact? or instability of some sort? and what would be your recommended 
“virtual” number of child nodes?

2. Will read/writing one node which has, lets say 10,000 sibling be problematic 
for any reason?

3. I know that there are node types that have ordered vs unordered children. 
When does Oak actually order the child nodes? on read?

Thank you for your time,
Ahmed

Re: IndexEditorProvider behaviour question.

[Ian Boston]

Hi,
Thanks for looking at this, sounds like you are on the case already.
if I see anything else I'll let you know.
Best Regards
Ian


On 15 September 2016 at 05:33, Chetan Mehrotra 
wrote:

> Note that so far LuceneIndexEditor was used only for async indexing
> case and hence invoked only on leader node every 5 sec. So performance
> aspects here were not that critical. However with recent work on
> Hybrid indexes they would be used in critical path and hence such
> aspects are important
>
> On Wed, Sep 14, 2016 at 3:10 PM, Ian Boston  wrote:
> > A and B mean that the work of creating the tree and working out the
> changes
> > in a tree will be duplicated roughly n times, where n is the number of
> > index definitions.
>
> Here note that diff would be performed only once at any level and
> IndexUpdate would then pass them to various editors. However
> construction of trees can be avoided and I have opened OAK-4806 for
> that now. Oak issue has details around why Tree was used also.
>
> Also with multiple index editors performance does decrease. See
> OAK-1273. If we switch to Hybrid Index then this aspects improves a
> bit as instead of having 50 different property indexes (with 50 editor
> instance for each commit) we can have a single editor with 50 property
> definition. This can be seen in benchmark in Hybrid Index (OAk-4412)
> by changing the numOfIndexes
>
> If you see any other area of improvement say around unnecessary object
> generation then let us know!
>
> Chetan Mehrotra
>

Re: IndexEditorProvider behaviour question.

[Chetan Mehrotra]

Note that so far LuceneIndexEditor was used only for async indexing
case and hence invoked only on leader node every 5 sec. So performance
aspects here were not that critical. However with recent work on
Hybrid indexes they would be used in critical path and hence such
aspects are important

On Wed, Sep 14, 2016 at 3:10 PM, Ian Boston  wrote:
> A and B mean that the work of creating the tree and working out the changes
> in a tree will be duplicated roughly n times, where n is the number of
> index definitions.

Here note that diff would be performed only once at any level and
IndexUpdate would then pass them to various editors. However
construction of trees can be avoided and I have opened OAK-4806 for
that now. Oak issue has details around why Tree was used also.

Also with multiple index editors performance does decrease. See
OAK-1273. If we switch to Hybrid Index then this aspects improves a
bit as instead of having 50 different property indexes (with 50 editor
instance for each commit) we can have a single editor with 50 property
definition. This can be seen in benchmark in Hybrid Index (OAk-4412)
by changing the numOfIndexes

If you see any other area of improvement say around unnecessary object
generation then let us know!

Chetan Mehrotra

IndexEditorProvider behaviour question.

[Ian Boston]

Hi,
The behaviour of calls to the IndexEditorProvider appears to be suboptimal.
Has this area been looked at before?

I am working from a complete lack of historical knowledge about the area,
so probably don't know the full picture. Based on logging the calls into
IndexEditorProvider.getIndexEditor(), and reading the
LuceneIndexEditorProvider this is what I have observed.

A. Every commit results in 1 call to IndexEditorProvider.getIndexEditor()
per index definition. (perhaps 100 in a full system).
B. Each IndexEditor then gets called building a tree of IndexEditors which
work out changes to update the their index.
C. IndexEditors sometimes filter subtrees. based on the index definition,
but this seems to the the exception rather than the rule.
D. Index Editor Providers produce a subtree based on type (ie a property
index definition doesn't generate a IndexEditor for lucene indexes and visa
versa).

A and B mean that the work of creating the tree and working out the changes
in a tree will be duplicated roughly n times, where n is the number of
index definitions. (D means its not n*p where p is the number of
IndexEditorProviders). I haven't looked at how much C reduces the cost in
reality.

Has anyone looked at building the tree once, and passing the fully built
tree to indexers?

Even if the computational effort is not great the number of objects being
created and passing through GC seems higher than it needs to be.

As I said, I have no historical knowledge so if doing this doesn't improve
things and why is recorded just say (ideally with a pointer) so I can read
and understand more.

Best Regards
Ian

R: Another question about oak: different storage support

[Morelli Alessandra]

In pratica ci ha risposto picche. Ho capito bene?

Da: Michael Marth [mailto:mma...@adobe.com]
Inviato: domenica 1 maggio 2016 20:30
A: oak-dev@jackrabbit.apache.org
Cc: Morelli Alessandra <alessandra.more...@siav.it>; Carboniero Enrico 
<enrico.carboni...@siav.it>; Diquigiovanni Simone <simone.diquigiova...@siav.it>
Oggetto: Re: Another question about oak: different storage support

Hi Francesco,

To my knowledge one cannot add (on Oak level) a number of additional discs.
As a workaround one could mount (on OS-level) different discs into the file 
store’s directory. This would somewhat help with increasing disc size. It would 
not help with storing different types of binaries in different locations as the 
binaries are stored content-addressed (so the context of file type etc is lost 
at the datastore level).

In order to have multiple (chained) data stores: I am sure there would be great 
interest in the Oak community for such a feature. IIRC there is such a chained 
implementation in Jackrabbit 2, but I cannot find it right now.
The interesting aspect of a multiple datastores: reading is simple (just go 
along the chain). But rules where to write are more involved because of the 
content-addressed nature of the DS.

There is an improvement issue regarding related work in
https://issues.apache.org/jira/browse/OAK-3140

Cheers
Michael

From: Ancona Francesco 
<francesco.anc...@siav.it<mailto:francesco.anc...@siav.it>>
Reply-To: "oak-dev@jackrabbit.apache.org<mailto:oak-dev@jackrabbit.apache.org>" 
<oak-dev@jackrabbit.apache.org<mailto:oak-dev@jackrabbit.apache.org>>
Date: Friday 29 April 2016 18:43
To: "oak-dev@jackrabbit.apache.org<mailto:oak-dev@jackrabbit.apache.org>" 
<oak-dev@jackrabbit.apache.org<mailto:oak-dev@jackrabbit.apache.org>>
Cc: Morelli Alessandra 
<alessandra.more...@siav.it<mailto:alessandra.more...@siav.it>>, Carboniero 
Enrico <enrico.carboni...@siav.it<mailto:enrico.carboni...@siav.it>>, 
Diquigiovanni Simone 
<simone.diquigiova...@siav.it<mailto:simone.diquigiova...@siav.it>>
Subject: Another question about oak: different storage support

Hi,
as i explained in other mail, we are building an ECM on top of OAK.

A frequent business question about storage management is the following: in oak 
configuration in which we have mongo as document store and filesystem, we set a 
specific path where we mount the storage; if we have space problem, can we 
mount another storage? In other world, oak can manage multiple BlobStore and so 
use different filesystem path to store binary data ?

It could be useful if i want manage different type of storage linked to 
different type of documents: for example i could store a document type in a NAS 
while another in a SAN if i want better performance.

Thanks in advance,
best regards

[cid:image002.png@01D1A247.0EB3D7E0]
Francesco Ancona | Software Dev. Dept. (SP) - Software Architect
tel. +39 049 8979797 | fax +39 049 8978800 | cel. +39 3299060325
e-mail: francesco.anc...@siav.it<mailto:francesco.anc...@siav.it> | www.siav.it

I contenuti di questa e-mail e dei suoi allegati sono confidenziali e riservati 
esclusivamente ai destinatari.
L'utilizzo per qualunque fine del presente messaggio e degli allegati così come 
la relativa divulgazione senza l'autorizzazione del mittente sono vietati.
Se avete ricevuto questa e-mail per errore, vi preghiamo di distruggerla e di 
comunicarcelo.
I dati personali sono trattati esclusivamente per le finalità della presente 
comunicazione in conformità con la legislazione vigente (D.lgs. 196/2003 
"Codice Privacy").
Per informazioni: SIAV S.p.A. – s...@siav.it<mailto:s...@siav.it> – 049 8979797

The contents of this e-mail and its attachments are confidential and reserved 
exclusively to the recipients.
The use for any purpose of this message and attachments as well as its 
disclosure without the consent of the sender is prohibited.
If you have received this email in error, please destroy it and notify us.
Personal data shall be processed solely for the purposes of this notice in 
accordance with current legislation (Legislative Decree no. 196/2003 "Code").
For more information: SIAV S.p.A. – s...@siav.it<mailto:s...@siav.it> – 049 
8979797






This footnote confirms that this email message has been scanned by
PineApp Mail-SeCure for the presence of malicious code, vandals & computer 
viruses.

Re: Another question about oak: different storage support

[Michael Marth]

Hi Francesco,

To my knowledge one cannot add (on Oak level) a number of additional discs.
As a workaround one could mount (on OS-level) different discs into the file 
store’s directory. This would somewhat help with increasing disc size. It would 
not help with storing different types of binaries in different locations as the 
binaries are stored content-addressed (so the context of file type etc is lost 
at the datastore level).

In order to have multiple (chained) data stores: I am sure there would be great 
interest in the Oak community for such a feature. IIRC there is such a chained 
implementation in Jackrabbit 2, but I cannot find it right now.
The interesting aspect of a multiple datastores: reading is simple (just go 
along the chain). But rules where to write are more involved because of the 
content-addressed nature of the DS.

There is an improvement issue regarding related work in
https://issues.apache.org/jira/browse/OAK-3140

Cheers
Michael

From: Ancona Francesco 
<francesco.anc...@siav.it<mailto:francesco.anc...@siav.it>>
Reply-To: "oak-dev@jackrabbit.apache.org<mailto:oak-dev@jackrabbit.apache.org>" 
<oak-dev@jackrabbit.apache.org<mailto:oak-dev@jackrabbit.apache.org>>
Date: Friday 29 April 2016 18:43
To: "oak-dev@jackrabbit.apache.org<mailto:oak-dev@jackrabbit.apache.org>" 
<oak-dev@jackrabbit.apache.org<mailto:oak-dev@jackrabbit.apache.org>>
Cc: Morelli Alessandra 
<alessandra.more...@siav.it<mailto:alessandra.more...@siav.it>>, Carboniero 
Enrico <enrico.carboni...@siav.it<mailto:enrico.carboni...@siav.it>>, 
Diquigiovanni Simone 
<simone.diquigiova...@siav.it<mailto:simone.diquigiova...@siav.it>>
Subject: Another question about oak: different storage support

Hi,
as i explained in other mail, we are building an ECM on top of OAK.

A frequent business question about storage management is the following: in oak 
configuration in which we have mongo as document store and filesystem, we set a 
specific path where we mount the storage; if we have space problem, can we 
mount another storage? In other world, oak can manage multiple BlobStore and so 
use different filesystem path to store binary data ?

It could be useful if i want manage different type of storage linked to 
different type of documents: for example i could store a document type in a NAS 
while another in a SAN if i want better performance.

Thanks in advance,
best regards

[cid:image002.png@01D1A247.0EB3D7E0]
Francesco Ancona | Software Dev. Dept. (SP) - Software Architect
tel. +39 049 8979797 | fax +39 049 8978800 | cel. +39 3299060325
e-mail: francesco.anc...@siav.it<mailto:francesco.anc...@siav.it> | www.siav.it

I contenuti di questa e-mail e dei suoi allegati sono confidenziali e riservati 
esclusivamente ai destinatari.
L'utilizzo per qualunque fine del presente messaggio e degli allegati così come 
la relativa divulgazione senza l'autorizzazione del mittente sono vietati.
Se avete ricevuto questa e-mail per errore, vi preghiamo di distruggerla e di 
comunicarcelo.
I dati personali sono trattati esclusivamente per le finalità della presente 
comunicazione in conformità con la legislazione vigente (D.lgs. 196/2003 
"Codice Privacy").
Per informazioni: SIAV S.p.A. – s...@siav.it<mailto:s...@siav.it> – 049 8979797

The contents of this e-mail and its attachments are confidential and reserved 
exclusively to the recipients.
The use for any purpose of this message and attachments as well as its 
disclosure without the consent of the sender is prohibited.
If you have received this email in error, please destroy it and notify us.
Personal data shall be processed solely for the purposes of this notice in 
accordance with current legislation (Legislative Decree no. 196/2003 "Code").
For more information: SIAV S.p.A. – s...@siav.it<mailto:s...@siav.it> – 049 
8979797

Another question about oak: different storage support

[Ancona Francesco]

Hi,
as i explained in other mail, we are building an ECM on top of OAK.

A frequent business question about storage management is the following: in oak 
configuration in which we have mongo as document store and filesystem, we set a 
specific path where we mount the storage; if we have space problem, can we 
mount another storage? In other world, oak can manage multiple BlobStore and so 
use different filesystem path to store binary data ?

It could be useful if i want manage different type of storage linked to 
different type of documents: for example i could store a document type in a NAS 
while another in a SAN if i want better performance.

Thanks in advance,
best regards

[cid:image002.png@01D1A247.0EB3D7E0]
Francesco Ancona | Software Dev. Dept. (SP) - Software Architect
tel. +39 049 8979797 | fax +39 049 8978800 | cel. +39 3299060325
e-mail: francesco.anc...@siav.it | www.siav.it

I contenuti di questa e-mail e dei suoi allegati sono confidenziali e riservati 
esclusivamente ai destinatari.
L'utilizzo per qualunque fine del presente messaggio e degli allegati così come 
la relativa divulgazione senza l'autorizzazione del mittente sono vietati.
Se avete ricevuto questa e-mail per errore, vi preghiamo di distruggerla e di 
comunicarcelo.
I dati personali sono trattati esclusivamente per le finalità della presente 
comunicazione in conformità con la legislazione vigente (D.lgs. 196/2003 
"Codice Privacy").
Per informazioni: SIAV S.p.A. - s...@siav.it - 049 8979797

The contents of this e-mail and its attachments are confidential and reserved 
exclusively to the recipients.
The use for any purpose of this message and attachments as well as its 
disclosure without the consent of the sender is prohibited.
If you have received this email in error, please destroy it and notify us.
Personal data shall be processed solely for the purposes of this notice in 
accordance with current legislation (Legislative Decree no. 196/2003 "Code").
For more information: SIAV S.p.A. - s...@siav.it - 049 8979797

R: critical question about oak: RDBMS and distributed transaction

[Ancona Francesco]

I try explain the case.

We know that OAK doesnt'support Transaction, so the following should be the JCR 
implementation:



http://www.day.com/specs/jcr/2.0/10_Writing.html

[cid:image003.jpg@01D19996.DE408710]



When i use session write-methods, i can execute a sequence of session methods 
(example add nodes) but only when i use save method i persist the changes.

Instead when i use Workspace-Write Methods (example VersionManager.checkin, 
checkout, checkpoint, restore, restoreByLabel, merge, cancelMerge, doneMerge, 
createActivity , removeActivity and createConfiguration), for each action i 
persist on the workspace.



So, imagine a use case in which i must create a version of a document and  save 
it in my repository in atomic way; if the first phase works but the second one 
is KO, we have inconsistent data: the document has a new version but is not 
saved.



Which kind of workaround do you suggest to solve this problem ?



Thanks in advance,

best regards







-Messaggio originale-
Da: Julian Reschke [mailto:julian.resc...@gmx.de]
Inviato: giovedì 14 aprile 2016 18:05
A: oak-dev@jackrabbit.apache.org
Cc: Diquigiovanni Simone <simone.diquigiova...@siav.it>; Morelli Alessandra 
<alessandra.more...@siav.it>
Oggetto: Re: critical question about oak: RDBMS and distributed transaction



On 2016-04-14 17:24, Ancona Francesco wrote:

> Hi,

>

> thanks for rdbms help. I managed to run my application also with

> postgres and Oracle. My wrong has been that default installation

> doesn't work; we need to set another collation. (in detail collation

> C)

>

> Can i suggest to add these details to the documentation ?



The only documentation right now is in RDBDocumentStore's javadoc, and that 
does mention it.



> Now we have an other problem.

>

> We found that distributed transactions are not supported in OAK. This

> could be a problem when we need to aggregate different kind of

> services (our database and ecm services): so our data could be not

> choerent in specific situations.

>

> Besides we noticed that there is a major issue opened in 2013 but not

> closed.



Specifically?



> So i point out 3 question

>

> 1.Is there a roadmap or an indicative date to solve this issue?

>

> 2.Which kind of workaround do you suggest to solve the problem now ?

>

> 3.Do you know if adobe oak built-in product has implemented the

> distributed transactions ?



AFAIU, Oak wasn't designed with distributed transactions in mind, nor is it 
currently used that way. I'm not aware of any plans to change that...



Best regards, Julian









This footnote confirms that this email message has been scanned by PineApp 
Mail-SeCure for the presence of malicious code, vandals & computer viruses.

Re: critical question about oak: RDBMS and distributed transaction

[Julian Reschke]

On 2016-04-14 17:24, Ancona Francesco wrote:

Hi,

thanks for rdbms help. I managed to run my application also with
postgres and Oracle. My wrong has been that default installation doesn’t
work; we need to set another collation. (in detail collation C)

Can i suggest to add these details to the documentation ?


The only documentation right now is in RDBDocumentStore's javadoc, and 
that does mention it.



Now we have an other problem.

We found that distributed transactions are not supported in OAK. This
could be a problem when we need to aggregate different kind of services
(our database and ecm services): so our data could be not choerent in
specific situations.

Besides we noticed that there is a major issue opened in 2013 but not
closed.


Specifically?


So i point out 3 question

1.Is there a roadmap or an indicative date to solve this issue?

2.Which kind of workaround do you suggest to solve the problem now ?

3.Do you know if adobe oak built-in product has implemented the
distributed transactions ?


AFAIU, Oak wasn't designed with distributed transactions in mind, nor is 
it currently used that way. I'm not aware of any plans to change that...


Best regards, Julian

critical question about oak: RDBMS and distributed transaction

[Ancona Francesco]

Hi,
thanks for rdbms help. I managed to run my application also with postgres and 
Oracle. My wrong has been that default installation doesn't work; we need to 
set another collation. (in detail collation C)
Can i suggest to add these details to the documentation ?

Now we have an other problem.
We found that distributed transactions are not supported in OAK. This could be 
a problem when we need to aggregate different kind of services (our database 
and ecm services): so our data could be not choerent in specific situations.
Besides we noticed that there is a major issue opened in 2013 but not closed.

So i point out 3 question

1.   Is there a roadmap or an indicative date to solve this issue?

2.   Which kind of workaround do you suggest to solve the problem now ?

3.   Do you know if adobe oak built-in product has implemented the 
distributed transactions ?

Thanks in advance
Best regards


[cid:image003.png@01D19672.94AA1E30]
Francesco Ancona | Software Dev. Dept. (SP) - Software Architect
tel. +39 049 8979797 | fax +39 049 8978800 | cel. +39 3299060325
e-mail: francesco.anc...@siav.it | www.siav.it

I contenuti di questa e-mail e dei suoi allegati sono confidenziali e riservati 
esclusivamente ai destinatari.
L'utilizzo per qualunque fine del presente messaggio e degli allegati così come 
la relativa divulgazione senza l'autorizzazione del mittente sono vietati.
Se avete ricevuto questa e-mail per errore, vi preghiamo di distruggerla e di 
comunicarcelo.
I dati personali sono trattati esclusivamente per le finalità della presente 
comunicazione in conformità con la legislazione vigente (D.lgs. 196/2003 
"Codice Privacy").
Per informazioni: SIAV S.p.A. - s...@siav.it - 049 8979797

The contents of this e-mail and its attachments are confidential and reserved 
exclusively to the recipients.
The use for any purpose of this message and attachments as well as its 
disclosure without the consent of the sender is prohibited.
If you have received this email in error, please destroy it and notify us.
Personal data shall be processed solely for the purposes of this notice in 
accordance with current legislation (Legislative Decree no. 196/2003 "Code").
For more information: SIAV S.p.A. - s...@siav.it - 049 8979797

Re: R: critical question about oak: db connection

[Julian Reschke]

On 2016-03-23 16:19, Manfred Baedke wrote:

Hi Francesco,

Your tests ran out of the box in my IDE (IntelliJ IDEA 15) using a local
Postgres 9.3.

Best regards,
Manfred


Same here, from command line maven and on PostgreSQL 9.5:


[INFO] Scanning for projects...
[INFO]
[INFO] 
[INFO] Building oaktest 0.0.1-SNAPSHOT
[INFO] 
[INFO]
[INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ oaktest ---
[INFO] Deleting C:\home\jre\oaktest\target
[INFO]
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ oaktest 
---
[debug] execute contextualize
[WARNING] Using platform encoding (Cp1252 actually) to copy filtered resources, 
i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory 
C:\home\jre\oaktest\src\main\resources
[INFO]
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oaktest ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oaktest ---
[debug] execute contextualize
[WARNING] Using platform encoding (Cp1252 actually) to copy filtered resources, 
i.e. build is platform dependent!
[INFO] Copying 4 resources
[INFO]
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oaktest ---
[WARNING] File encoding has not been set, using platform encoding Cp1252, i.e. 
build is platform dependent!
[INFO] Compiling 1 source file to C:\home\jre\oaktest\target\test-classes
[INFO]
[INFO] --- maven-surefire-plugin:2.10:test (default-test) @ oaktest ---
[INFO] Surefire report directory: C:\home\jre\oaktest\target\surefire-reports

---
 T E S T S
---
Running oaktest.OakRDBMSTest
16:33:41.896 INFO  [main] AbstractTestContextBootstrapper.java:207 Could not 
instantiate TestExecutionListener 
[org.springframework.test.context.web.ServletTestExecutionListener]. Specify 
custom listener classes or make the default listener classes (and their 
required dependencies) available. Offending class: 
[javax/servlet/ServletContext]
16:33:41.896 INFO  [main] AbstractTestContextBootstrapper.java:185 Using 
TestExecutionListeners: 
[org.springframework.test.context.support.DirtiesContextBeforeModesTestExecutionListener@2ffad8fe,
 
org.springframework.test.context.support.DependencyInjectionTestExecutionListener@7dbc244d,
 
org.springframework.test.context.support.DirtiesContextTestExecutionListener@4af37bb8]
16:33:42.017 INFO  [main] XmlBeanDefinitionReader.java:317  Loading XML bean 
definitions from class path resource [config/spring/spring-test-config.xml]
16:33:42.117 INFO  [main] AbstractApplicationContext.java:578 Refreshing 
org.springframework.context.support.GenericApplicationContext@3fb2acb7: startup 
date [Tue Mar 29 16:33:42 CEST 2016]; root of context hierarchy
16:33:42.217 INFO  [main] DriverManagerDataSource.java:133  Loaded JDBC driver: 
org.postgresql.Driver
16:33:42.750 INFO  [main] RDBDocumentStore.java:827 RDBDocumentStore 
(1.4.0) instantiated for database PostgreSQL 9.5.0 (9.5), using driver: 
PostgreSQL Native Driver PostgreSQL 9.4.1207.jre7 (9.4), connecting to: 
jdbc:postgresql:oak, properties: {pg_encoding_to_char(encoding)=UTF8, 
datcollate=C}, transaction isolation level: TRANSACTION_READ_COMMITTED (2), 
.nodes: id varchar(512), modified int8, hasbinary int2, deletedonce int2, 
modcount int8, cmodcount int8, dsize int8, data varchar(16384), bdata 
bytea(2147483647) /* {bytea=-2, int2=5, int8=-5, varchar=12} */ /* unique index 
nodes_pkey on public.nodes (id ASC) other */
16:33:42.750 INFO  [main] RDBDocumentStore.java:834 Tables created upon 
startup: [CLUSTERNODES, NODES, SETTINGS, JOURNAL]
16:33:42.781 INFO  [main] RDBBlobStore.java:224 RDBBlobStore 
(1.4.0) instantiated for database PostgreSQL 9.5.0 (9.5), using driver: 
PostgreSQL Native Driver PostgreSQL 9.4.1207.jre7 (9.4), connecting to: 
jdbc:postgresql:oak, transaction isolation level: TRANSACTION_READ_COMMITTED (2)
16:33:42.781 INFO  [main] RDBBlobStore.java:230 Tables created upon 
startup: [DATASTORE_DATA, DATASTORE_META]
16:33:43.182 INFO  [main] DocumentNodeStore.java:516Initialized 
DocumentNodeStore with clusterNodeId: 1 (id: 1, startTime: 1459262022866, 
machineId: mac:1803733fd6b0, instanceId: C:\home\jre\oaktest, pid: 6024, uuid: 
db2a1755-2346-44d9-a284-37f0a7e69a1f, readWriteMode: null, state: NONE, 
revLock: NONE, oakVersion: 1.4.0)
16:33:44.600 INFO  [main] IndexUpdate.java:182  Found a new index 
node [reference]. Reindexing is requested
16:33:44.600 INFO  [main] IndexUpdate.java:147  Reindexing will be 
performed for following indexes: [/oak:index/reference, /oak:index/nodetype, 
/oak:index/uuid]
16:33:55.139 INFO  [main] IndexUpdate.java:257  Indexing report
- 

Re: R: critical question about oak: db connection

[Manfred Baedke]

Hi Francesco,

Your tests ran out of the box in my IDE (IntelliJ IDEA 15) using a local 
Postgres 9.3.


Best regards,
Manfred

On 3/23/2016 8:27 AM, Ancona Francesco wrote:

Hello,
any news about question ?

Thanks in advance,
best regards


-Messaggio originale-
Da: Ancona Francesco [mailto:francesco.anc...@siav.it]
Inviato: venerdì 18 marzo 2016 11:40
A: oak-dev@jackrabbit.apache.org
Oggetto: R: critical question about oak: db connection

Done.

https://github.com/francescoancona/oaktest.git

Thanks in advance,
best regards


-Messaggio originale-
Da: Tomek Rekawek [mailto:reka...@adobe.com]
Inviato: giovedì 17 marzo 2016 15:32
A: oak-dev@jackrabbit.apache.org
Oggetto: Re: critical question about oak: db connection

Hello Ancona,

the mailing list didn’t allow to add an attachment. Could you post it somewhere 
online or (even better) put the code into github?

Best regards,
Tomek

--
Tomek Rękawek | Adobe Research | www.adobe.com reka...@adobe.com


On 17 Mar 2016, at 14:46, Ancona Francesco <francesco.anc...@siav.it> wrote:

Hello,
i send you a maven project to connect with postgres.

As i told you in other messages i know that oak test cases work; but they start 
always with RDBDocumentStore and use UpdateOp and other Oak methods to add 
nodes.

Instead we'd like to use RDBDocumentNodeStore and we'd like to have 
jcrRepository and jcr methods to manage documents (nodes).

I hope the project can be useful.

Thanks in advance,
best regards.

-Messaggio originale-
Da: Julian Reschke [mailto:julian.resc...@gmx.de]
Inviato: mercoledì 16 marzo 2016 17:21
A: oak-dev@jackrabbit.apache.org
Oggetto: Re: R: critical question about oak: db connection

Ancona,

you keep asking for more help but seem to ignore the feedback that you get.

1) oak-jcr and oak-run demonstrate that it is possible to run on RDBs; just run 
the tests. All the information that you need should be in the test cases.

2) Don't throw code snippets at us. Without complete classes we can try 
ourselves there is no way how we can find out what's wrong.

So either start from the existing working code, or send self-contained test 
code we can compile and run ourselves.

Best regards, Julian


On 2016-03-16 16:28, Ancona Francesco wrote:

Hi,
Oak configuration with RDBMS doesn't work yet.

I use the latest release 1.4.0 but i have the same problem

In particular this is the code i use to initialize Oak

 final DocumentMK.Builder builder = new DocumentMK.Builder();
 builder.setBlobStore(createFileSystemBlobStore());
 final DocumentNodeStore ns =
getRDBDocumentNodeStore(builder);

// ds is datasource form postgres 9.4 DocumentStore documentStore =
new RDBDocumentStore(ds, builder);
 builder.setDocumentStore(documentStore);
 DocumentNodeStore ns = new DocumentNodeStore(builder);
 DocumentNodeStore ns1 = new 
DocumentMK.Builder().setDocumentStore(documentStore).getNodeStore();
 InitialContent ic = new InitialContent();
 Oak oak = new Oak(ns1).with(ic);
 Jcr jcr = new Jcr(oak);
 Repository repo = jcr.createRepository();


and this is the stack trace:
java.lang.IllegalStateException: This tree does not exist
 at 
org.apache.jackrabbit.oak.core.MutableTree.beforeWrite(MutableTree.java:353)
 at 
org.apache.jackrabbit.oak.core.MutableTree.setProperty(MutableTree.java:232)
 at 
org.apache.jackrabbit.oak.plugins.name.ReadWriteNamespaceRegistry.registerNamespace(ReadWriteNamespaceRegistry.java:92)
 at 
org.apache.jackrabbit.commons.cnd.TemplateBuilderFactory.setNamespace(TemplateBuilderFactory.java:92)
 at 
org.apache.jackrabbit.commons.cnd.CompactNodeTypeDefReader.doNameSpace(CompactNodeTypeDefReader.java:246)
 at 
org.apache.jackrabbit.commons.cnd.CompactNodeTypeDefReader.parse(CompactNodeTypeDefReader.java:200)
 at 
org.apache.jackrabbit.commons.cnd.CompactNodeTypeDefReader.(CompactNodeTypeDefReader.java:163)
 at 
org.apache.jackrabbit.commons.cnd.CompactNodeTypeDefReader.(CompactNodeTypeDefReader.java:139)
 at 
org.apache.jackrabbit.commons.cnd.CndImporter.registerNodeTypes(CndImporter.java:147)
 at 
org.apache.jackrabbit.oak.plugins.nodetype.write.NodeTypeRegistry.registerNodeTypes(NodeTypeRegistry.java:109)
 at 
org.apache.jackrabbit.oak.plugins.nodetype.write.NodeTypeRegistry.register(NodeTypeRegistry.java:104)
 at 
org.apache.jackrabbit.oak.plugins.nodetype.write.NodeTypeRegistry.registerBuiltIn(NodeTypeRegistry.java:86)
 at 
org.apache.jackrabbit.oak.plugins.nodetype.write.InitialContent.initialize(InitialContent.java:120)
 at 
org.apache.jackrabbit.oak.spi.lifecycle.CompositeInitializer.initialize(CompositeInitializer.java:48)
 at 
org.apache.jackrabbit.oak.spi.lifecycle.OakInitializer.initialize(OakInitializer.java:42)
 at org.apache.jackrabbit.oak.Oak.createNewContentRepository(Oak.java:628)
 at org.apache.jackrabbit.oak.Oak.createContentRepository(Oak.ja

R: critical question about oak: db connection

[Ancona Francesco]

Hello,
any news about question ?

Thanks in advance,
best regards


-Messaggio originale-
Da: Ancona Francesco [mailto:francesco.anc...@siav.it] 
Inviato: venerdì 18 marzo 2016 11:40
A: oak-dev@jackrabbit.apache.org
Oggetto: R: critical question about oak: db connection

Done.

https://github.com/francescoancona/oaktest.git

Thanks in advance,
best regards


-Messaggio originale-
Da: Tomek Rekawek [mailto:reka...@adobe.com]
Inviato: giovedì 17 marzo 2016 15:32
A: oak-dev@jackrabbit.apache.org
Oggetto: Re: critical question about oak: db connection

Hello Ancona,

the mailing list didn’t allow to add an attachment. Could you post it somewhere 
online or (even better) put the code into github?

Best regards,
Tomek

--
Tomek Rękawek | Adobe Research | www.adobe.com reka...@adobe.com

> On 17 Mar 2016, at 14:46, Ancona Francesco <francesco.anc...@siav.it> wrote:
> 
> Hello,
> i send you a maven project to connect with postgres.
> 
> As i told you in other messages i know that oak test cases work; but they 
> start always with RDBDocumentStore and use UpdateOp and other Oak methods to 
> add nodes.
> 
> Instead we'd like to use RDBDocumentNodeStore and we'd like to have 
> jcrRepository and jcr methods to manage documents (nodes).
> 
> I hope the project can be useful.
> 
> Thanks in advance,
> best regards.
> 
> -Messaggio originale-
> Da: Julian Reschke [mailto:julian.resc...@gmx.de]
> Inviato: mercoledì 16 marzo 2016 17:21
> A: oak-dev@jackrabbit.apache.org
> Oggetto: Re: R: critical question about oak: db connection
> 
> Ancona,
> 
> you keep asking for more help but seem to ignore the feedback that you get.
> 
> 1) oak-jcr and oak-run demonstrate that it is possible to run on RDBs; just 
> run the tests. All the information that you need should be in the test cases.
> 
> 2) Don't throw code snippets at us. Without complete classes we can try 
> ourselves there is no way how we can find out what's wrong.
> 
> So either start from the existing working code, or send self-contained test 
> code we can compile and run ourselves.
> 
> Best regards, Julian
> 
> 
> On 2016-03-16 16:28, Ancona Francesco wrote:
>> Hi,
>> Oak configuration with RDBMS doesn't work yet.
>> 
>> I use the latest release 1.4.0 but i have the same problem
>> 
>> In particular this is the code i use to initialize Oak
>>  
>> final DocumentMK.Builder builder = new DocumentMK.Builder();
>> builder.setBlobStore(createFileSystemBlobStore());
>> final DocumentNodeStore ns = 
>> getRDBDocumentNodeStore(builder);
>> 
>> // ds is datasource form postgres 9.4 DocumentStore documentStore = 
>> new RDBDocumentStore(ds, builder);
>> builder.setDocumentStore(documentStore);
>> DocumentNodeStore ns = new DocumentNodeStore(builder);
>> DocumentNodeStore ns1 = new 
>> DocumentMK.Builder().setDocumentStore(documentStore).getNodeStore();
>> InitialContent ic = new InitialContent();
>> Oak oak = new Oak(ns1).with(ic);
>> Jcr jcr = new Jcr(oak);
>> Repository repo = jcr.createRepository();
>> 
>> 
>> and this is the stack trace:
>> java.lang.IllegalStateException: This tree does not exist
>> at 
>> org.apache.jackrabbit.oak.core.MutableTree.beforeWrite(MutableTree.java:353)
>> at 
>> org.apache.jackrabbit.oak.core.MutableTree.setProperty(MutableTree.java:232)
>> at 
>> org.apache.jackrabbit.oak.plugins.name.ReadWriteNamespaceRegistry.registerNamespace(ReadWriteNamespaceRegistry.java:92)
>> at 
>> org.apache.jackrabbit.commons.cnd.TemplateBuilderFactory.setNamespace(TemplateBuilderFactory.java:92)
>> at 
>> org.apache.jackrabbit.commons.cnd.CompactNodeTypeDefReader.doNameSpace(CompactNodeTypeDefReader.java:246)
>> at 
>> org.apache.jackrabbit.commons.cnd.CompactNodeTypeDefReader.parse(CompactNodeTypeDefReader.java:200)
>> at 
>> org.apache.jackrabbit.commons.cnd.CompactNodeTypeDefReader.(CompactNodeTypeDefReader.java:163)
>> at 
>> org.apache.jackrabbit.commons.cnd.CompactNodeTypeDefReader.(CompactNodeTypeDefReader.java:139)
>> at 
>> org.apache.jackrabbit.commons.cnd.CndImporter.registerNodeTypes(CndImporter.java:147)
>> at 
>> org.apache.jackrabbit.oak.plugins.nodetype.write.NodeTypeRegistry.registerNodeTypes(NodeTypeRegistry.java:109)
>> at 
>> org.apache.jackrabbit.oak.plugins.nodetype.write.NodeTypeRegistry.register(NodeTypeRegistry.java:104)
>> at 
>> org.apache.jackrabbit.oak.plugins.nodetype.write.NodeTypeRegistry.registerBuiltIn(NodeTypeRegis

R: R: critical question about oak: db connection

[Ancona Francesco]

Hello,
i send you a maven project to connect with postgres.

As i told you in other messages i know that oak test cases work; but they start 
always with RDBDocumentStore and use UpdateOp and other Oak methods to add 
nodes.

Instead we'd like to use RDBDocumentNodeStore and we'd like to have 
jcrRepository and jcr methods to manage documents (nodes).

I hope the project can be useful.

Thanks in advance,
best regards.

-Messaggio originale-
Da: Julian Reschke [mailto:julian.resc...@gmx.de] 
Inviato: mercoledì 16 marzo 2016 17:21
A: oak-dev@jackrabbit.apache.org
Oggetto: Re: R: critical question about oak: db connection

Ancona,

you keep asking for more help but seem to ignore the feedback that you get.

1) oak-jcr and oak-run demonstrate that it is possible to run on RDBs; just run 
the tests. All the information that you need should be in the test cases.

2) Don't throw code snippets at us. Without complete classes we can try 
ourselves there is no way how we can find out what's wrong.

So either start from the existing working code, or send self-contained test 
code we can compile and run ourselves.

Best regards, Julian


On 2016-03-16 16:28, Ancona Francesco wrote:
> Hi,
> Oak configuration with RDBMS doesn't work yet.
>
> I use the latest release 1.4.0 but i have the same problem
>
> In particular this is the code i use to initialize Oak
>   
>  final DocumentMK.Builder builder = new DocumentMK.Builder();
>  builder.setBlobStore(createFileSystemBlobStore());
>  final DocumentNodeStore ns = 
> getRDBDocumentNodeStore(builder);
>
> // ds is datasource form postgres 9.4
> DocumentStore documentStore = new RDBDocumentStore(ds, builder);
>  builder.setDocumentStore(documentStore);
>  DocumentNodeStore ns = new DocumentNodeStore(builder);
>  DocumentNodeStore ns1 = new 
> DocumentMK.Builder().setDocumentStore(documentStore).getNodeStore();
>  InitialContent ic = new InitialContent();
>  Oak oak = new Oak(ns1).with(ic);
>  Jcr jcr = new Jcr(oak);
>  Repository repo = jcr.createRepository();
>
>
> and this is the stack trace:
> java.lang.IllegalStateException: This tree does not exist
>  at 
> org.apache.jackrabbit.oak.core.MutableTree.beforeWrite(MutableTree.java:353)
>  at 
> org.apache.jackrabbit.oak.core.MutableTree.setProperty(MutableTree.java:232)
>  at 
> org.apache.jackrabbit.oak.plugins.name.ReadWriteNamespaceRegistry.registerNamespace(ReadWriteNamespaceRegistry.java:92)
>  at 
> org.apache.jackrabbit.commons.cnd.TemplateBuilderFactory.setNamespace(TemplateBuilderFactory.java:92)
>  at 
> org.apache.jackrabbit.commons.cnd.CompactNodeTypeDefReader.doNameSpace(CompactNodeTypeDefReader.java:246)
>  at 
> org.apache.jackrabbit.commons.cnd.CompactNodeTypeDefReader.parse(CompactNodeTypeDefReader.java:200)
>  at 
> org.apache.jackrabbit.commons.cnd.CompactNodeTypeDefReader.(CompactNodeTypeDefReader.java:163)
>  at 
> org.apache.jackrabbit.commons.cnd.CompactNodeTypeDefReader.(CompactNodeTypeDefReader.java:139)
>  at 
> org.apache.jackrabbit.commons.cnd.CndImporter.registerNodeTypes(CndImporter.java:147)
>  at 
> org.apache.jackrabbit.oak.plugins.nodetype.write.NodeTypeRegistry.registerNodeTypes(NodeTypeRegistry.java:109)
>  at 
> org.apache.jackrabbit.oak.plugins.nodetype.write.NodeTypeRegistry.register(NodeTypeRegistry.java:104)
>  at 
> org.apache.jackrabbit.oak.plugins.nodetype.write.NodeTypeRegistry.registerBuiltIn(NodeTypeRegistry.java:86)
>  at 
> org.apache.jackrabbit.oak.plugins.nodetype.write.InitialContent.initialize(InitialContent.java:120)
>  at 
> org.apache.jackrabbit.oak.spi.lifecycle.CompositeInitializer.initialize(CompositeInitializer.java:48)
>  at 
> org.apache.jackrabbit.oak.spi.lifecycle.OakInitializer.initialize(OakInitializer.java:42)
>  at org.apache.jackrabbit.oak.Oak.createNewContentRepository(Oak.java:628)
>  at org.apache.jackrabbit.oak.Oak.createContentRepository(Oak.java:616)
>  at 
> org.apache.jackrabbit.oak.jcr.Jcr.createContentRepository(Jcr.java:367)
>  at org.apache.jackrabbit.oak.jcr.Jcr.createRepository(Jcr.java:375)
>  at 
> it.siav.jarvis.testplatform.ecm.common.OakRDBMSTest.test(OakRDBMSTest.java:74)
>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>  at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>  at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  at java.lang.reflect.Method.invoke(Method.java:497)
>  at 
> org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:85)
>  at 
> org.

R: critical question about oak: db connection

[Ancona Francesco]

Done.

https://github.com/francescoancona/oaktest.git

Thanks in advance,
best regards


-Messaggio originale-
Da: Tomek Rekawek [mailto:reka...@adobe.com] 
Inviato: giovedì 17 marzo 2016 15:32
A: oak-dev@jackrabbit.apache.org
Oggetto: Re: critical question about oak: db connection

Hello Ancona,

the mailing list didn’t allow to add an attachment. Could you post it somewhere 
online or (even better) put the code into github?

Best regards,
Tomek

--
Tomek Rękawek | Adobe Research | www.adobe.com reka...@adobe.com

> On 17 Mar 2016, at 14:46, Ancona Francesco <francesco.anc...@siav.it> wrote:
> 
> Hello,
> i send you a maven project to connect with postgres.
> 
> As i told you in other messages i know that oak test cases work; but they 
> start always with RDBDocumentStore and use UpdateOp and other Oak methods to 
> add nodes.
> 
> Instead we'd like to use RDBDocumentNodeStore and we'd like to have 
> jcrRepository and jcr methods to manage documents (nodes).
> 
> I hope the project can be useful.
> 
> Thanks in advance,
> best regards.
> 
> -Messaggio originale-
> Da: Julian Reschke [mailto:julian.resc...@gmx.de]
> Inviato: mercoledì 16 marzo 2016 17:21
> A: oak-dev@jackrabbit.apache.org
> Oggetto: Re: R: critical question about oak: db connection
> 
> Ancona,
> 
> you keep asking for more help but seem to ignore the feedback that you get.
> 
> 1) oak-jcr and oak-run demonstrate that it is possible to run on RDBs; just 
> run the tests. All the information that you need should be in the test cases.
> 
> 2) Don't throw code snippets at us. Without complete classes we can try 
> ourselves there is no way how we can find out what's wrong.
> 
> So either start from the existing working code, or send self-contained test 
> code we can compile and run ourselves.
> 
> Best regards, Julian
> 
> 
> On 2016-03-16 16:28, Ancona Francesco wrote:
>> Hi,
>> Oak configuration with RDBMS doesn't work yet.
>> 
>> I use the latest release 1.4.0 but i have the same problem
>> 
>> In particular this is the code i use to initialize Oak
>>  
>> final DocumentMK.Builder builder = new DocumentMK.Builder();
>> builder.setBlobStore(createFileSystemBlobStore());
>> final DocumentNodeStore ns = 
>> getRDBDocumentNodeStore(builder);
>> 
>> // ds is datasource form postgres 9.4 DocumentStore documentStore = 
>> new RDBDocumentStore(ds, builder);
>> builder.setDocumentStore(documentStore);
>> DocumentNodeStore ns = new DocumentNodeStore(builder);
>> DocumentNodeStore ns1 = new 
>> DocumentMK.Builder().setDocumentStore(documentStore).getNodeStore();
>> InitialContent ic = new InitialContent();
>> Oak oak = new Oak(ns1).with(ic);
>> Jcr jcr = new Jcr(oak);
>> Repository repo = jcr.createRepository();
>> 
>> 
>> and this is the stack trace:
>> java.lang.IllegalStateException: This tree does not exist
>> at 
>> org.apache.jackrabbit.oak.core.MutableTree.beforeWrite(MutableTree.java:353)
>> at 
>> org.apache.jackrabbit.oak.core.MutableTree.setProperty(MutableTree.java:232)
>> at 
>> org.apache.jackrabbit.oak.plugins.name.ReadWriteNamespaceRegistry.registerNamespace(ReadWriteNamespaceRegistry.java:92)
>> at 
>> org.apache.jackrabbit.commons.cnd.TemplateBuilderFactory.setNamespace(TemplateBuilderFactory.java:92)
>> at 
>> org.apache.jackrabbit.commons.cnd.CompactNodeTypeDefReader.doNameSpace(CompactNodeTypeDefReader.java:246)
>> at 
>> org.apache.jackrabbit.commons.cnd.CompactNodeTypeDefReader.parse(CompactNodeTypeDefReader.java:200)
>> at 
>> org.apache.jackrabbit.commons.cnd.CompactNodeTypeDefReader.(CompactNodeTypeDefReader.java:163)
>> at 
>> org.apache.jackrabbit.commons.cnd.CompactNodeTypeDefReader.(CompactNodeTypeDefReader.java:139)
>> at 
>> org.apache.jackrabbit.commons.cnd.CndImporter.registerNodeTypes(CndImporter.java:147)
>> at 
>> org.apache.jackrabbit.oak.plugins.nodetype.write.NodeTypeRegistry.registerNodeTypes(NodeTypeRegistry.java:109)
>> at 
>> org.apache.jackrabbit.oak.plugins.nodetype.write.NodeTypeRegistry.register(NodeTypeRegistry.java:104)
>> at 
>> org.apache.jackrabbit.oak.plugins.nodetype.write.NodeTypeRegistry.registerBuiltIn(NodeTypeRegistry.java:86)
>> at 
>> org.apache.jackrabbit.oak.plugins.nodetype.write.InitialContent.initialize(InitialContent.java:120)
>> at 
>> org.apache.jackrabbit.oak.spi.lifecycle.CompositeInitializer.initialize(CompositeInitializer.java:48)
>> at 
>> org.apache.jackrab

Re: R: R: critical question about oak: db connection

[Julian Reschke]

On 2016-03-17 14:46, Ancona Francesco wrote:

Hello,
i send you a maven project to connect with postgres.

As i told you in other messages i know that oak test cases work; but they start 
always with RDBDocumentStore and use UpdateOp and other Oak methods to add 
nodes.
...


Both oak-jcr and oak-run contain tests that use JCR->OAK->RDB sucessfully.

Best regards, Julian

Re: R: critical question about oak: db connection

[Julian Reschke]

(SuiteRunner.java:259)
 at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
 at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
 at org.testng.TestNG.runSuitesSequentially(TestNG.java:1176)
 at org.testng.TestNG.runSuitesLocally(TestNG.java:1101)
 at org.testng.TestNG.run(TestNG.java:1009)
 at org.testng.remote.RemoteTestNG.run(RemoteTestNG.java:111)
 at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:204)
 at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:175)

I think could be a wrong Oak intialization but i didn't find any clear example

Can you give me the correct piece of code to create a content repository to use 
RDBMS ?

Thnaks in advance.
Best regards

-Messaggio originale-
Da: Julian Reschke [mailto:julian.resc...@gmx.de]
Inviato: mercoledì 2 marzo 2016 15:46
A: oak-dev@jackrabbit.apache.org
Oggetto: Re: critical question about oak: db connection

On 2016-03-02 12:08, Ancona Francesco wrote:

We are using latest stable oak version

In the case i'm sending you we have tried on postgres 9.4

We have simplified the class but now the error is the same we found on
Oracle.


Once again: please show me the system log; specifically the startup message of 
RDBDocumentStore.


Class connect to database and write a lot of system rows  (system
nodes.) but when i create repository there is the error

Could you give us a complete example that works on Oracle 12 and
Postgres 9.4 ?


Checkout OAK, run tests in oak-jcr with RDB fixture. For instance:

mvn clean install -Prdb-postgres -Drdb.jdbc-url=jdbc:postgresql:oak
-Drdb.jdbc-user=... -Drdb.jdbc-passwd=... -Dnsfixures=DOCUMENT_RDB 
-PintegrationTesting -Prdb-postgres


Is Postgres 9.4 correct version ?


Or newer.

Best regards, Julian




This footnote confirms that this email message has been scanned by PineApp 
Mail-SeCure for the presence of malicious code, vandals & computer viruses.

R: critical question about oak: db connection

[Ancona Francesco]

 didn't find any clear example

Can you give me the correct piece of code to create a content repository to use 
RDBMS ?

Thnaks in advance.
Best regards

-Messaggio originale-
Da: Julian Reschke [mailto:julian.resc...@gmx.de] 
Inviato: mercoledì 2 marzo 2016 15:46
A: oak-...@jackrabbit.apache.org
Oggetto: Re: critical question about oak: db connection

On 2016-03-02 12:08, Ancona Francesco wrote:
> We are using latest stable oak version
>
> In the case i'm sending you we have tried on postgres 9.4
>
> We have simplified the class but now the error is the same we found on 
> Oracle.

Once again: please show me the system log; specifically the startup message of 
RDBDocumentStore.

> Class connect to database and write a lot of system rows  (system
> nodes.) but when i create repository there is the error
>
> Could you give us a complete example that works on Oracle 12 and 
> Postgres 9.4 ?

Checkout OAK, run tests in oak-jcr with RDB fixture. For instance:

mvn clean install -Prdb-postgres -Drdb.jdbc-url=jdbc:postgresql:oak
-Drdb.jdbc-user=... -Drdb.jdbc-passwd=... -Dnsfixures=DOCUMENT_RDB 
-PintegrationTesting -Prdb-postgres

> Is Postgres 9.4 correct version ?

Or newer.

Best regards, Julian

 
 

This footnote confirms that this email message has been scanned by PineApp 
Mail-SeCure for the presence of malicious code, vandals & computer viruses.

Re: critical question about oak: db connection

[Tomek Rekawek]

Hello Ancona,

the mailing list didn’t allow to add an attachment. Could you post it somewhere 
online or (even better) put the code into github?

Best regards,
Tomek

-- 
Tomek Rękawek | Adobe Research | www.adobe.com
reka...@adobe.com

> On 17 Mar 2016, at 14:46, Ancona Francesco <francesco.anc...@siav.it> wrote:
> 
> Hello,
> i send you a maven project to connect with postgres.
> 
> As i told you in other messages i know that oak test cases work; but they 
> start always with RDBDocumentStore and use UpdateOp and other Oak methods to 
> add nodes.
> 
> Instead we'd like to use RDBDocumentNodeStore and we'd like to have 
> jcrRepository and jcr methods to manage documents (nodes).
> 
> I hope the project can be useful.
> 
> Thanks in advance,
> best regards.
> 
> -Messaggio originale-
> Da: Julian Reschke [mailto:julian.resc...@gmx.de] 
> Inviato: mercoledì 16 marzo 2016 17:21
> A: oak-dev@jackrabbit.apache.org
> Oggetto: Re: R: critical question about oak: db connection
> 
> Ancona,
> 
> you keep asking for more help but seem to ignore the feedback that you get.
> 
> 1) oak-jcr and oak-run demonstrate that it is possible to run on RDBs; just 
> run the tests. All the information that you need should be in the test cases.
> 
> 2) Don't throw code snippets at us. Without complete classes we can try 
> ourselves there is no way how we can find out what's wrong.
> 
> So either start from the existing working code, or send self-contained test 
> code we can compile and run ourselves.
> 
> Best regards, Julian
> 
> 
> On 2016-03-16 16:28, Ancona Francesco wrote:
>> Hi,
>> Oak configuration with RDBMS doesn't work yet.
>> 
>> I use the latest release 1.4.0 but i have the same problem
>> 
>> In particular this is the code i use to initialize Oak
>>  
>> final DocumentMK.Builder builder = new DocumentMK.Builder();
>> builder.setBlobStore(createFileSystemBlobStore());
>> final DocumentNodeStore ns = 
>> getRDBDocumentNodeStore(builder);
>> 
>> // ds is datasource form postgres 9.4
>> DocumentStore documentStore = new RDBDocumentStore(ds, builder);
>> builder.setDocumentStore(documentStore);
>> DocumentNodeStore ns = new DocumentNodeStore(builder);
>> DocumentNodeStore ns1 = new 
>> DocumentMK.Builder().setDocumentStore(documentStore).getNodeStore();
>> InitialContent ic = new InitialContent();
>> Oak oak = new Oak(ns1).with(ic);
>> Jcr jcr = new Jcr(oak);
>> Repository repo = jcr.createRepository();
>> 
>> 
>> and this is the stack trace:
>> java.lang.IllegalStateException: This tree does not exist
>> at 
>> org.apache.jackrabbit.oak.core.MutableTree.beforeWrite(MutableTree.java:353)
>> at 
>> org.apache.jackrabbit.oak.core.MutableTree.setProperty(MutableTree.java:232)
>> at 
>> org.apache.jackrabbit.oak.plugins.name.ReadWriteNamespaceRegistry.registerNamespace(ReadWriteNamespaceRegistry.java:92)
>> at 
>> org.apache.jackrabbit.commons.cnd.TemplateBuilderFactory.setNamespace(TemplateBuilderFactory.java:92)
>> at 
>> org.apache.jackrabbit.commons.cnd.CompactNodeTypeDefReader.doNameSpace(CompactNodeTypeDefReader.java:246)
>> at 
>> org.apache.jackrabbit.commons.cnd.CompactNodeTypeDefReader.parse(CompactNodeTypeDefReader.java:200)
>> at 
>> org.apache.jackrabbit.commons.cnd.CompactNodeTypeDefReader.(CompactNodeTypeDefReader.java:163)
>> at 
>> org.apache.jackrabbit.commons.cnd.CompactNodeTypeDefReader.(CompactNodeTypeDefReader.java:139)
>> at 
>> org.apache.jackrabbit.commons.cnd.CndImporter.registerNodeTypes(CndImporter.java:147)
>> at 
>> org.apache.jackrabbit.oak.plugins.nodetype.write.NodeTypeRegistry.registerNodeTypes(NodeTypeRegistry.java:109)
>> at 
>> org.apache.jackrabbit.oak.plugins.nodetype.write.NodeTypeRegistry.register(NodeTypeRegistry.java:104)
>> at 
>> org.apache.jackrabbit.oak.plugins.nodetype.write.NodeTypeRegistry.registerBuiltIn(NodeTypeRegistry.java:86)
>> at 
>> org.apache.jackrabbit.oak.plugins.nodetype.write.InitialContent.initialize(InitialContent.java:120)
>> at 
>> org.apache.jackrabbit.oak.spi.lifecycle.CompositeInitializer.initialize(CompositeInitializer.java:48)
>> at 
>> org.apache.jackrabbit.oak.spi.lifecycle.OakInitializer.initialize(OakInitializer.java:42)
>> at org.apache.jackrabbit.oak.Oak.createNewContentRepository(Oak.java:628)
>> at org.apache.jackrabbit.oak.Oak.createContentRepository(Oak.java:616)
>> at 
>> org.apache.jac

Re: R: critical question about oak: db connection

[Manfred Baedke]

Hi Francesco,


Or could you point out the unit test in oak source code, can we use ?


A natural starting point is 
org.apache.jackrabbit.oak.plugins.document.rbd.RDBDocumentStoreTest in 
oak-core.


Best regards,
Manfred

On 3/2/2016 12:45 PM, Ancona Francesco wrote:

I’m not authorized  to publish anything;
could you give me a project that  works ?

Or could you point out the unit test in oak source code, can we use ?

Thanks
Best regards

Da: Alex Parvulescu [mailto:alex.parvule...@gmail.com]
Inviato: mercoledì 2 marzo 2016 12:22
A: Oak devs <oak-dev@jackrabbit.apache.org>
Oggetto: Re: critical question about oak: db connection

Hi,

I see you keep having issues with getting a prototype up and running, but 
pasting the code in emails might be a bit off-putting for anyone looking to 
give you a hand.
Might I suggest publishing the relevant bits (self contained project easy to 
run and see the error) to a GitHub repository?

best,
alex




On Wed, Mar 2, 2016 at 12:08 PM, Ancona Francesco 
<francesco.anc...@siav.it<mailto:francesco.anc...@siav.it>> wrote:

We are using latest stable oak version



In the case i’m sending you we have tried on postgres 9.4



We have simplified the class but now the error is the same we found on Oracle.



Class connect to database and write a lot of system rows  (system nodes…) but 
when i create repository there is the error



Could you give us a complete example that works on Oracle 12 and Postgres 9.4 ?

Is Postgres 9.4 correct version ?



Thanks

Best regards





@Autowired

 @Qualifier("oakPostgresDataSource")

 private DataSource ds;



 @Test

 public void test() {



final DocumentMK.Builder builder = new 
DocumentMK.Builder();


builder.setBlobStore(createFileSystemBlobStore());

final DocumentNodeStore ns = 
getRDBDocumentNodeStore(builder);

Oak oak = new Oak(ns);

Jcr jcr = new Jcr(oak);



Repository repo = jcr.createRepository();

Session session = null;

try {

InputStream is = new 
ByteArrayInputStream(document_small);

session = repo.login(new 
SimpleCredentials("admin", "admin".toCharArray()));

ValueFactory valueFactory = 
session.getValueFactory();

Binary data = 
valueFactory.createBinary(is);

Node root = 
session.getRootNode();

root.addNode("hello", 
NodeType.NT_UNSTRUCTURED);

session.save();

// Logout

} catch (LoginException e) {

// TODO Auto-generated catch 
block

e.printStackTrace();

} catch (RepositoryException e) {

// TODO Auto-generated catch 
block

e.printStackTrace();

} finally {

if (session != null) 
session.logout();

ns.dispose();

}

 }



 private DocumentNodeStore 
getRDBDocumentNodeStore(DocumentMK.Builder builder) {

DocumentNodeStore ns = null;

if (builder == null) {

ns = new 
DocumentMK.Builder().setRDBConnection(ds).getNodeStore();

} else {

ns = 
builder.setRDBConnection(ds).getNodeStore();

}

return ns;

 }



 private BlobStore createFileSystemBlobStore() {

try {

FileUtils.deleteDirectory(new 
File("/var/tmp/oak"));

} catch (IOException e) {

// TODO Auto-generated catch 
block

e.printStackTrace();

}



FileBlobStore store = new 
FileBlobStore("/var/tmp/oak");

return store;

 }






[cid:image002

R: critical question about oak: db connection

[Ancona Francesco]

I’m not authorized  to publish anything;
could you give me a project that  works ?

Or could you point out the unit test in oak source code, can we use ?

Thanks
Best regards

Da: Alex Parvulescu [mailto:alex.parvule...@gmail.com]
Inviato: mercoledì 2 marzo 2016 12:22
A: Oak devs <oak-dev@jackrabbit.apache.org>
Oggetto: Re: critical question about oak: db connection

Hi,

I see you keep having issues with getting a prototype up and running, but 
pasting the code in emails might be a bit off-putting for anyone looking to 
give you a hand.
Might I suggest publishing the relevant bits (self contained project easy to 
run and see the error) to a GitHub repository?

best,
alex




On Wed, Mar 2, 2016 at 12:08 PM, Ancona Francesco 
<francesco.anc...@siav.it<mailto:francesco.anc...@siav.it>> wrote:

We are using latest stable oak version



In the case i’m sending you we have tried on postgres 9.4



We have simplified the class but now the error is the same we found on Oracle.



Class connect to database and write a lot of system rows  (system nodes…) but 
when i create repository there is the error



Could you give us a complete example that works on Oracle 12 and Postgres 9.4 ?

Is Postgres 9.4 correct version ?



Thanks

Best regards





@Autowired

@Qualifier("oakPostgresDataSource")

private DataSource ds;



@Test

public void test() {



   final DocumentMK.Builder builder = new 
DocumentMK.Builder();

   
builder.setBlobStore(createFileSystemBlobStore());

   final DocumentNodeStore ns = 
getRDBDocumentNodeStore(builder);

   Oak oak = new Oak(ns);

   Jcr jcr = new Jcr(oak);



   Repository repo = jcr.createRepository();

   Session session = null;

   try {

   InputStream is = new 
ByteArrayInputStream(document_small);

   session = repo.login(new 
SimpleCredentials("admin", "admin".toCharArray()));

   ValueFactory valueFactory = 
session.getValueFactory();

   Binary data = 
valueFactory.createBinary(is);

   Node root = 
session.getRootNode();

   root.addNode("hello", 
NodeType.NT_UNSTRUCTURED);

   session.save();

   // Logout

   } catch (LoginException e) {

   // TODO Auto-generated catch 
block

   e.printStackTrace();

   } catch (RepositoryException e) {

   // TODO Auto-generated catch 
block

   e.printStackTrace();

   } finally {

   if (session != null) 
session.logout();

   ns.dispose();

   }

}



private DocumentNodeStore 
getRDBDocumentNodeStore(DocumentMK.Builder builder) {

   DocumentNodeStore ns = null;

   if (builder == null) {

   ns = new 
DocumentMK.Builder().setRDBConnection(ds).getNodeStore();

   } else {

   ns = 
builder.setRDBConnection(ds).getNodeStore();

   }

   return ns;

}



private BlobStore createFileSystemBlobStore() {

   try {

   FileUtils.deleteDirectory(new 
File("/var/tmp/oak"));

   } catch (IOException e) {

   // TODO Auto-generated catch 
block

   e.printStackTrace();

   }



   FileBlobStore store = new 
FileBlobStore("/var/tmp/oak");

   return store;

}






[cid:image002.png@01D1747C.2AE9FF00]
Francesco Ancona | Software Dev. Dept. (SP) - Software Architect
tel. +39 049 8979797<tel:%2B39%20049%208979797> | fax +39 049 
8978800<tel:%2B39%20049%208978800> | cel. +39 3299060325<tel:%2B39%203299060325>
e-mail: francesco.anc...@siav.it<mailto:francesco.anc...@siav.it> |

Re: critical question about oak: db connection

[Alex Parvulescu]

Hi,

I see you keep having issues with getting a prototype up and running, but
pasting the code in emails might be a bit off-putting for anyone looking to
give you a hand.
Might I suggest publishing the relevant bits (self contained project easy
to run and see the error) to a GitHub repository?

best,
alex




On Wed, Mar 2, 2016 at 12:08 PM, Ancona Francesco 
wrote:

> We are using latest stable oak version
>
>
>
> In the case i’m sending you we have tried on postgres 9.4
>
>
>
> We have simplified the class but now the error is the same we found on
> Oracle.
>
>
>
> Class connect to database and write a lot of system rows  (system nodes…)
> but when i create repository there is the error
>
>
>
> Could you give us a complete example that works on Oracle 12 and Postgres
> 9.4 ?
>
> Is Postgres 9.4 correct version ?
>
>
>
> Thanks
>
> Best regards
>
>
>
>
>
> @Autowired
>
> @Qualifier("oakPostgresDataSource")
>
> private DataSource ds;
>
>
>
> @Test
>
> public void test() {
>
>
>
>final DocumentMK.Builder builder = new
> DocumentMK.Builder();
>
>
> builder.setBlobStore(createFileSystemBlobStore());
>
>final DocumentNodeStore ns =
> getRDBDocumentNodeStore(builder);
>
>Oak oak = new Oak(ns);
>
>Jcr jcr = new Jcr(oak);
>
>
>
>Repository repo = jcr.createRepository();
>
>Session session = null;
>
>try {
>
>InputStream is = new
> ByteArrayInputStream(document_small);
>
>session = repo.login(new
> SimpleCredentials("admin", "admin".toCharArray()));
>
>ValueFactory valueFactory =
> session.getValueFactory();
>
>Binary data =
> valueFactory.createBinary(is);
>
>Node root =
> session.getRootNode();
>
>root.addNode("hello",
> NodeType.NT_UNSTRUCTURED);
>
>session.save();
>
>// Logout
>
>} catch (LoginException e) {
>
>// TODO Auto-generated
> catch block
>
>e.printStackTrace();
>
>} catch (RepositoryException e) {
>
>// TODO Auto-generated
> catch block
>
>e.printStackTrace();
>
>} finally {
>
>if (session != null)
> session.logout();
>
>ns.dispose();
>
>}
>
> }
>
>
>
> private DocumentNodeStore
> getRDBDocumentNodeStore(DocumentMK.Builder builder) {
>
>DocumentNodeStore ns = null;
>
>if (builder == null) {
>
>ns = new
> DocumentMK.Builder().setRDBConnection(ds).getNodeStore();
>
>} else {
>
>ns =
> builder.setRDBConnection(ds).getNodeStore();
>
>}
>
>return ns;
>
> }
>
>
>
> private BlobStore createFileSystemBlobStore() {
>
>try {
>
>
> FileUtils.deleteDirectory(new File("/var/tmp/oak"));
>
>} catch (IOException e) {
>
>// TODO Auto-generated
> catch block
>
>e.printStackTrace();
>
>}
>
>
>
>FileBlobStore store = new
> FileBlobStore("/var/tmp/oak");
>
>return store;
>
> }
>
>
>
>
>
>
>
>
>
> *Francesco Ancona *| Software Dev. Dept. (SP) - Software Architect
>
> tel. +39 049 8979797 | fax +39 049 8978800 | cel. +39 3299060325
>
> e-mail: francesco.anc...@siav.it | www.siav.it
>
>
>
> I contenuti di questa e-mail e dei suoi allegati sono confidenziali e
> riservati esclusivamente ai destinatari.
>
> L'utilizzo per qualunque fine del presente messaggio e degli allegati così
> come la relativa divulgazione senza l'autorizzazione del mittente sono
> vietati.
>
> Se avete ricevuto questa e-mail per errore, vi preghiamo di distruggerla e
> di comunicarcelo.
>
> I dati personali sono trattati esclusivamente per le finalità della
> presente 

critical question about oak: db connection

[Ancona Francesco]

We are using latest stable oak version



In the case i'm sending you we have tried on postgres 9.4



We have simplified the class but now the error is the same we found on Oracle.



Class connect to database and write a lot of system rows  (system nodes...) but 
when i create repository there is the error



Could you give us a complete example that works on Oracle 12 and Postgres 9.4 ?

Is Postgres 9.4 correct version ?



Thanks

Best regards





@Autowired

@Qualifier("oakPostgresDataSource")

private DataSource ds;



@Test

public void test() {



   final DocumentMK.Builder builder = new 
DocumentMK.Builder();

   
builder.setBlobStore(createFileSystemBlobStore());

   final DocumentNodeStore ns = 
getRDBDocumentNodeStore(builder);

   Oak oak = new Oak(ns);

   Jcr jcr = new Jcr(oak);



   Repository repo = jcr.createRepository();

   Session session = null;

   try {

   InputStream is = new 
ByteArrayInputStream(document_small);

   session = repo.login(new 
SimpleCredentials("admin", "admin".toCharArray()));

   ValueFactory valueFactory = 
session.getValueFactory();

   Binary data = 
valueFactory.createBinary(is);

   Node root = 
session.getRootNode();

   root.addNode("hello", 
NodeType.NT_UNSTRUCTURED);

   session.save();

   // Logout

   } catch (LoginException e) {

   // TODO Auto-generated catch 
block

   e.printStackTrace();

   } catch (RepositoryException e) {

   // TODO Auto-generated catch 
block

   e.printStackTrace();

   } finally {

   if (session != null) 
session.logout();

   ns.dispose();

   }

}



private DocumentNodeStore 
getRDBDocumentNodeStore(DocumentMK.Builder builder) {

   DocumentNodeStore ns = null;

   if (builder == null) {

   ns = new 
DocumentMK.Builder().setRDBConnection(ds).getNodeStore();

   } else {

   ns = 
builder.setRDBConnection(ds).getNodeStore();

   }

   return ns;

}



private BlobStore createFileSystemBlobStore() {

   try {

   FileUtils.deleteDirectory(new 
File("/var/tmp/oak"));

   } catch (IOException e) {

   // TODO Auto-generated catch 
block

   e.printStackTrace();

   }



   FileBlobStore store = new 
FileBlobStore("/var/tmp/oak");

   return store;

}






[cid:image002.png@01D1747C.2AE9FF00]
Francesco Ancona | Software Dev. Dept. (SP) - Software Architect
tel. +39 049 8979797 | fax +39 049 8978800 | cel. +39 3299060325
e-mail: francesco.anc...@siav.it | www.siav.it

I contenuti di questa e-mail e dei suoi allegati sono confidenziali e riservati 
esclusivamente ai destinatari.
L'utilizzo per qualunque fine del presente messaggio e degli allegati così come 
la relativa divulgazione senza l'autorizzazione del mittente sono vietati.
Se avete ricevuto questa e-mail per errore, vi preghiamo di distruggerla e di 
comunicarcelo.
I dati personali sono trattati esclusivamente per le finalità della presente 
comunicazione in conformità con la legislazione vigente (D.lgs. 196/2003 
"Codice Privacy").
Per informazioni: SIAV S.p.A. - s...@siav.it - 049 8979797

The contents of this e-mail and its attachments are confidential and reserved 
exclusively to the recipients.
The use for any purpose of this message and attachments as well as its 
disclosure without the consent of the sender is prohibited.
If you have received this email in error, please destroy it and notify us.
Personal data shall be processed solely for the purposes of this notice in 
accordance with 

Re: DocumentStore question.

[Marcel Reutegger]

Hi,

On 16/02/16 12:35, "ianbos...@gmail.com on behalf of Ian Boston" wrote:
>Presumably, having a cluster node running behind real time will result in
>lower throughput, making it critical to run NTP on all cluster nodes to
>eliminate as much clock drift as possible ?

Yes, this is correct. Though, the clocks do not have to be
in perfect sync. Each DocumentNodeStore runs background
operations once a second. So, you probably won't notice
any delays even when the clock difference is one second.

>Also, does the current revision model behave with an eventually consistent
>storage mechanism, or does Oak require that the underlying storage is
>immediately consistent in nature ?

In general a DocumentStore implementation must be strongly
consistent, but there are API calls that allow to leverage
an eventually consistent backend. E.g. there is a
DocumentStore.find() variant with a maxCacheAge parameter.
This allows the current MongoDocumentStore implementation
to use a secondary for the read if applicable.

IMO the API may likely have to be changed to better support
such use cases. Related issues are OAK-2106 and OAK-3865.

Regards
 Marcel

Re: DocumentStore question.

[Ian Boston]

Hi,
Thank you for the detailed explanation. I can now see how this works with a
consistent root document as the slow node effectively waits till its time
is ahead of the last root commit and it is clear to commit. This ensures
that all commits are sequential based on the revision timestamp.

Presumably, having a cluster node running behind real time will result in
lower throughput, making it critical to run NTP on all cluster nodes to
eliminate as much clock drift as possible ?

Also, does the current revision model behave with an eventually consistent
storage mechanism, or does Oak require that the underlying storage is
immediately consistent in nature ?

Best Regards
Ian


On 16 February 2016 at 10:36, Marcel Reutegger  wrote:

> Hi,
>
> On 16/02/16 09:56, "ianbos...@gmail.com on
> behalf of Ian Boston" wrote:
> So, IIUC, (based on Revision.compareTo(Revision) used by
> StableRevisionComparitor.
>
> yes.
>
> If one instance within a cluster has a clock that is lagging the others,
> and all instances are making changes at the same time, then the changes
> that the other instances make will be used, even the the lagging instance
> makes changes after (in real synchronised time) the others ?
>
> no, either cluster node has equal chances of getting its
> change in, but the other cluster node's change will be rejected.
>
> Let's assume we have two cluster nodes A and B and cluster node
> A's clock is lagging 5 seconds. Now both cluster nodes try to
> to set a property P on document D. One of the cluster nodes will be
> first to update document D. No matter, which cluster node is first,
> the second cluster node will see the previous change when it attempts
> the commit and will consider the change as not yet visible and
> in conflict with its own changes. The change of the second cluster
> node will therefore be rolled back.
>
> The behaviour of the cluster nodes will be different when external
> changes are pulled in from external cluster nodes. The background
> read operation of the DocumentNodeStore reads the most recent
> root document and compare the _lastRev entries of the other cluster
> nodes with its own clock (the _lastRev entries are the most recent
> commits visible to other cluster nodes). Here we have two cases:
>
> a) Cluster node A was successful to commit its change on P
>
> Cluster node A wrote a _lastRev on the root document for this
> change: r75-0-a. Cluster node B picks up that change and compares
> the revision with its own clock, which corresponds to r80-0-b
> (for readability, assuming for now the timestamp is a decimal
> and in seconds instead of milliseconds). Cluster node B will
> consider r75-0-a as visible from now on, because the timestamp
> of r80-0-b is newer than r75-0-a. From this point on Cluster
> node B can overwrite P again because it is able to see the most
> recent value set by A with r75-0-a.
>
> b) Cluster node B was successful to commit its change on P
>
> Cluster node B wrote a _lastRev on the root document for this
> change: r80-0-b. Cluster node A picks up that change and compares
> the revision with its own clock, which corresponds to r75-0-a.
> Cluster node A will still not consider r80-0-b as visible,
> because its own clock is considered behind. It will wait until
> its clock is passed r80-0-a. This makes a new change by A
> overwriting B's previous value of P, will have a newer timestamp
> than the previously made visible change of B.
>
> This means:
>
> 1) all changes considered visible can be compared with the
> StableRevisionComparator without the need to take clock
> differences into account.
>
> 2) a change will conflict if it is not the most recent
> revision (using StableRevisionComparator) or the other
> change is not yet visible but already committed.
>
>
> I can see that this won't matter for the majority of nodes, as collisions
> are rare, but won't the lagging instance be always overridden in the root
> document _revisions list ?
>
> Depending on usage, collisions are actually not that rare ;)
>
> The _revisions map on the root document contains just
> the commit entry. A cluster node cannot overwrite the
> entry of another cluster node, because they use unique
> revisions for commits. Each cluster node generates revisions
> with a unique clusterId suffix.
>
> Are there any plans to maintain a clock difference vector for the cluster ?
>
> Oak 1.0.x and 1.2.x still have something like this. See
> RevisionComparator. However, it only maintains the clock
> differences for the past 60 minutes.
>
> Oak 1.4 introduced a RevisionVector, which is inspired by
> version vectors [0].
>
> Regards
>  Marcel
>
> [0]
> https://issues.apache.org/jira/browse/OAK-3646?focusedCommentId=15028698=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15028698
>

Re: DocumentStore question.

[Marcel Reutegger]

Hi,

On 16/02/16 09:56, "ianbos...@gmail.com on behalf 
of Ian Boston" wrote:
So, IIUC, (based on Revision.compareTo(Revision) used by
StableRevisionComparitor.

yes.

If one instance within a cluster has a clock that is lagging the others,
and all instances are making changes at the same time, then the changes
that the other instances make will be used, even the the lagging instance
makes changes after (in real synchronised time) the others ?

no, either cluster node has equal chances of getting its
change in, but the other cluster node's change will be rejected.

Let's assume we have two cluster nodes A and B and cluster node
A's clock is lagging 5 seconds. Now both cluster nodes try to
to set a property P on document D. One of the cluster nodes will be
first to update document D. No matter, which cluster node is first,
the second cluster node will see the previous change when it attempts
the commit and will consider the change as not yet visible and
in conflict with its own changes. The change of the second cluster
node will therefore be rolled back.

The behaviour of the cluster nodes will be different when external
changes are pulled in from external cluster nodes. The background
read operation of the DocumentNodeStore reads the most recent
root document and compare the _lastRev entries of the other cluster
nodes with its own clock (the _lastRev entries are the most recent
commits visible to other cluster nodes). Here we have two cases:

a) Cluster node A was successful to commit its change on P

Cluster node A wrote a _lastRev on the root document for this
change: r75-0-a. Cluster node B picks up that change and compares
the revision with its own clock, which corresponds to r80-0-b
(for readability, assuming for now the timestamp is a decimal
and in seconds instead of milliseconds). Cluster node B will
consider r75-0-a as visible from now on, because the timestamp
of r80-0-b is newer than r75-0-a. From this point on Cluster
node B can overwrite P again because it is able to see the most
recent value set by A with r75-0-a.

b) Cluster node B was successful to commit its change on P

Cluster node B wrote a _lastRev on the root document for this
change: r80-0-b. Cluster node A picks up that change and compares
the revision with its own clock, which corresponds to r75-0-a.
Cluster node A will still not consider r80-0-b as visible,
because its own clock is considered behind. It will wait until
its clock is passed r80-0-a. This makes a new change by A
overwriting B's previous value of P, will have a newer timestamp
than the previously made visible change of B.

This means:

1) all changes considered visible can be compared with the
StableRevisionComparator without the need to take clock
differences into account.

2) a change will conflict if it is not the most recent
revision (using StableRevisionComparator) or the other
change is not yet visible but already committed.


I can see that this won't matter for the majority of nodes, as collisions
are rare, but won't the lagging instance be always overridden in the root
document _revisions list ?

Depending on usage, collisions are actually not that rare ;)

The _revisions map on the root document contains just
the commit entry. A cluster node cannot overwrite the
entry of another cluster node, because they use unique
revisions for commits. Each cluster node generates revisions
with a unique clusterId suffix.

Are there any plans to maintain a clock difference vector for the cluster ?

Oak 1.0.x and 1.2.x still have something like this. See
RevisionComparator. However, it only maintains the clock
differences for the past 60 minutes.

Oak 1.4 introduced a RevisionVector, which is inspired by
version vectors [0].

Regards
 Marcel

[0] 
https://issues.apache.org/jira/browse/OAK-3646?focusedCommentId=15028698=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15028698

Re: DocumentStore question.

[Ian Boston]

On 15 February 2016 at 14:49, Marcel Reutegger  wrote:

> Hi,
>
> On 12/02/16 17:11, "ianbos...@gmail.com on behalf of Ian Boston" wrote:
> >Is there an assumption that the revisions listed in _revisions are
> >ordered ?
>
> There is no requirement that entries in _revisions map
> are ordered at the storage layer, but the DocumentStore
> will order them when it reads the entries. The entries
> are sorted according to the timestamp of the revision,
> then revision counter and finally clusterId.
>
> >If not, then how is the order of the revisions be determined, given that
> >the clocks on each node in a cluster will have different offsets ?
>
> Oak 1.0.x and 1.2.x maintain a revision table (in RevisionComparator)
> for each cluster node, which allows it to compare revision across
> cluster nodes even when there are clock differences. At least for
> the 60 minutes timeframe covered by the RevisionComparator.
>
> Oak 1.4 uses revision vectors and does not maintain a revision
> table anymore. See OAK-3646. At the same time it also simplifies
> how revisions are compared and how changes are pulled in from
> other cluster nodes. The background read operation ensures that
> external changes made visible all have a lower revision timestamp
> than the local clock. This ensure that all local changes from that
> point on will have a higher revision timestamp than externally
> visible changes. This part was also backported to 1.0 and 1.2.
> See OAK-3388.
>


So, IIUC, (based on Revision.compareTo(Revision) used by
StableRevisionComparitor.

If one instance within a cluster has a clock that is lagging the others,
and all instances are making changes at the same time, then the changes
that the other instances make will be used, even the the lagging instance
makes changes after (in real synchronised time) the others ?

I can see that this won't matter for the majority of nodes, as collisions
are rare, but won't the lagging instance be always overridden in the root
document _revisions list ?

Are there any plans to maintain a clock difference vector for the cluster ?

Best Regards
Ian




>
> Regards
>  Marcel
>
>

Re: DocumentStore question.

[Marcel Reutegger]

Hi,

On 12/02/16 17:11, "ianbos...@gmail.com on behalf of Ian Boston" wrote:
>Is there an assumption that the revisions listed in _revisions are
>ordered ?

There is no requirement that entries in _revisions map
are ordered at the storage layer, but the DocumentStore
will order them when it reads the entries. The entries
are sorted according to the timestamp of the revision,
then revision counter and finally clusterId.

>If not, then how is the order of the revisions be determined, given that
>the clocks on each node in a cluster will have different offsets ?

Oak 1.0.x and 1.2.x maintain a revision table (in RevisionComparator)
for each cluster node, which allows it to compare revision across
cluster nodes even when there are clock differences. At least for
the 60 minutes timeframe covered by the RevisionComparator.

Oak 1.4 uses revision vectors and does not maintain a revision
table anymore. See OAK-3646. At the same time it also simplifies
how revisions are compared and how changes are pulled in from
other cluster nodes. The background read operation ensures that
external changes made visible all have a lower revision timestamp
than the local clock. This ensure that all local changes from that
point on will have a higher revision timestamp than externally
visible changes. This part was also backported to 1.0 and 1.2.
See OAK-3388.

Regards
 Marcel 

DocumentStore question.

[Ian Boston]

Hi,
I am looking at [1], and probably confused.

Is there an assumption that the revisions listed in _revisions are ordered ?

If not, then how is the order of the revisions be determined, given that
the clocks on each node in a cluster will have different offsets ?

Best Regards
Ian


1 http://jackrabbit.apache.org/oak/docs/nodestore/documentmk.html

Threading Question

[David Marginian]

https://jackrabbit.apache.org/oak/docs/construct.html

In a threaded environment (servlet, etc.) is it ok/recommended to create 
the repository/nodestore once and store as instance variables and then 
use the repository to create a new session per request?  I know that 
sessions should not be shared across threads but I wasn't sure about 
repositories.


Thanks!

Re: Threading Question

[Chetan Mehrotra]

Have a look at webapp example [1] for suggested setup. The repository
should be created once and then reused.

Chetan Mehrotra
[1] https://github.com/apache/jackrabbit-oak/tree/trunk/oak-examples/webapp


On Wed, Nov 18, 2015 at 4:02 AM, David Marginian
 wrote:
> https://jackrabbit.apache.org/oak/docs/construct.html
>
> In a threaded environment (servlet, etc.) is it ok/recommended to create the
> repository/nodestore once and store as instance variables and then use the
> repository to create a new session per request?  I know that sessions should
> not be shared across threads but I wasn't sure about repositories.
>
> Thanks!

Re: Question about the Oak Query Index

[Thomas Mueller]

Hi,

I will change the documentation to "Oak does not index _as_much_ content
by default as does Jackrabbit 2".

Regards,
Thomas



On 21/09/15 10:11, "Michael Lemler"  wrote:

>Oak 
>does not index content by default as does Jackrabbit 2

Re: Question about the Oak Query Index

[Davide Giannella]

On 21/09/2015 09:11, Michael Lemler wrote:
> Hi there,
>
> I have a question about the indexes in Oak. The documentation
> (http://jackrabbit.apache.org/oak/docs/query/query.html) says that
> "Oak does not index content by default as does Jackrabbit 2". But
> later on in the part "Query Processing" there is a part saying "By
> default, the following indexes are available". So are there some
> indexes by default or not?
Oak itself does not index anything; but this behaviour depends by
repository initialiser. Or in other way on how you initialise the
repository.

So if you for example initiate a repository with something like

Repository repo = new Jcr(new Oak()).createRepository();

behind the scenes the InitalContent will create the following indexes:
uuid, nodetype, counter, nodeReference.

https://goo.gl/dZy2s6

HTH
Davide

Question about the Oak Query Index

[Michael Lemler]

Hi there,

I have a question about the indexes in Oak. The documentation 
(http://jackrabbit.apache.org/oak/docs/query/query.html) says that "Oak 
does not index content by default as does Jackrabbit 2". But later on in 
the part "Query Processing" there is a part saying "By default, the 
following indexes are available". So are there some indexes by default 
or not?


Best regards
Michael

Re: Privilege.JCR_ADD_CHILD_NODES Question

[Angela Schreiber]

hi clay

Privilege.JCR_ADD_CHILD_NODES indicates that you can create

child nodes (without explicitly specifying the primary or mixin
types). if you wanted to use Node.addNode(String, String)
you additionally have to grant Privilege.JCR_NODE_TYPE_MANAGEMENT.

however, as far as other, non-protected properties are concerned that
you want to add below your target node:
as of Oak 1.0 (rewrite of the JCR repository) the
Privilege.JCR_MODIFIY_PROPERTIES
privileges has been adjusted to actually be an aggregation of
_add_, _alter_ and _remove_ properties (thus reflecting the fact
that some applications only want people to _add_ content but not
provide them the ability to _modify_ it later on). those privileges are
define in JCR but come with a 'rep' prefix. in the old jackrabbit 2.x
this is not possible and Privilege.JCR_MODIFIY_PROPERTIES is
not an aggregate privilege.

hope that helps
angela

On 26/07/15 18:34, Clay Ferguson wcl...@gmail.com wrote:

Fellow Jackrabbits,
I'm working on an socia media type application and trying to add a
feature that allows a user to share a node with another user, but the
person getting shared to should ONLY be able
 to ADD child nodes, and not actually edit the node being shared. This
means I would like for Privilege.JCR_ADD_CHILD_NODES to actually do what
it's name implies. But it doesn't. It seems that in order to get
Privilege.JCR_ADD_CHILD_NODES to work, I *also*
 have to include Privilege.JCR_WRITE which I specifically do NOT want.
Think of it like this: I have a blog node (for example), and I want
people to be able to make comments on my blog under it but not edit my
blog themselves. Am I doing something wrong? Should
 this work?


The app is
meta64.com http://meta64.com if anybody cares.


​- ​
Clay Fergu

​son​

Re: Privilege.JCR_ADD_CHILD_NODES Question

[Clay Ferguson]

Angela,
Thanks for the help, but I think the problem I mentioned is real, and is a
genuine problem.

Basically what I'm saying is this: If I add only JCR_ADD_CHILD_NODES
privilege it fails to work. That privilege cannot work by itself. It only
works if you add all three JCR_ADD_CHILD_NODES + JCR_NODE_TYPE_MANAGEMENT
+ JCR_WRITE. So it's either a bug, or it is a serious design mistake in the
API itself, at least based on my reasoning. Because as I said, it's a
common use case to have one user need to share a node with other users
without giving those other users ability to EDIT what I shared. Like if i
want people able to comment under my blog, I but I don't want people
EDITING my blog. That's a sample use case. However it's not a showstopper.
In my app meta64.com, I just only allow a user to edit the content of a
node if they are the CREATOR of the node. So I add that security hack on
top of the core security. It's a hack, but the JCR seriously needs to
address this, unless I'm making a mistake, which I don't think I am,
because other users (StackOverflow) ran into this also. I probably should
write this up as a BUG in the Jira and get the attention of the developers.





Best regards,
Clay Ferguson
wcl...@gmail.com
214-957-5950

On Thu, Jul 30, 2015 at 9:33 AM, Angela Schreiber anch...@adobe.com wrote:

 hi clay

 Privilege.JCR_ADD_CHILD_NODES indicates that you can create

 child nodes (without explicitly specifying the primary or mixin
 types). if you wanted to use Node.addNode(String, String)
 you additionally have to grant Privilege.JCR_NODE_TYPE_MANAGEMENT.

 however, as far as other, non-protected properties are concerned that
 you want to add below your target node:
 as of Oak 1.0 (rewrite of the JCR repository) the
 Privilege.JCR_MODIFIY_PROPERTIES
 privileges has been adjusted to actually be an aggregation of
 _add_, _alter_ and _remove_ properties (thus reflecting the fact
 that some applications only want people to _add_ content but not
 provide them the ability to _modify_ it later on). those privileges are
 define in JCR but come with a 'rep' prefix. in the old jackrabbit 2.x
 this is not possible and Privilege.JCR_MODIFIY_PROPERTIES is
 not an aggregate privilege.

 hope that helps
 angela

 On 26/07/15 18:34, Clay Ferguson wcl...@gmail.com wrote:

 Fellow Jackrabbits,
 I'm working on an socia media type application and trying to add a
 feature that allows a user to share a node with another user, but the
 person getting shared to should ONLY be able
  to ADD child nodes, and not actually edit the node being shared. This
 means I would like for Privilege.JCR_ADD_CHILD_NODES to actually do what
 it's name implies. But it doesn't. It seems that in order to get
 Privilege.JCR_ADD_CHILD_NODES to work, I *also*
  have to include Privilege.JCR_WRITE which I specifically do NOT want.
 Think of it like this: I have a blog node (for example), and I want
 people to be able to make comments on my blog under it but not edit my
 blog themselves. Am I doing something wrong? Should
  this work?
 
 
 The app is
 meta64.com http://meta64.com if anybody cares.
 
 
 ​- ​
 Clay Fergu
 
 ​son​
 
 
 
 
 

Privilege.JCR_ADD_CHILD_NODES Question

[Clay Ferguson]

Fellow Jackrabbits,
I'm working on an socia media type application and trying to add a
feature that allows a user to share a node with another user, but the
person getting shared to should ONLY be able to ADD child nodes, and not
actually edit the node being shared. This means I would like
for Privilege.JCR_ADD_CHILD_NODES to actually do what it's name implies.
But it doesn't. It seems that in order to get Privilege.JCR_ADD_CHILD_NODES
to work, I *also* have to include Privilege.JCR_WRITE which I specifically
do NOT want. Think of it like this: I have a blog node (for example), and I
want people to be able to make comments on my blog under it but not edit my
blog themselves. Am I doing something wrong? Should this work?

The app is meta64.com if anybody cares.

​- ​
Clay Fergu
​son​

Re: [Observation] Question on local events reported as external

[Marius Petria]

Thanks Chetan for the detailed explanation. I have a couple more questions :).

1. The observation queue limit is per EventListener, meaning that I have to 
optimize only my listener in order to make it less likely for the limit to be 
hit.
2. At least in theory (and with the exception of collapsing a creation and a 
deletion) the events are never dropped, right? The line in [1] even if it skips 
the current NodeState it will include it in a later revision, right?


Marius

[1] 
https://github.com/apache/jackrabbit-oak/blob/80985b51a18ef61015d263d2358f5c3713fea873/oak-core/src/main/java/org/apache/jackrabbit/oak/spi/commit/BackgroundObserver.java#L278



On 6/25/15, 9:49 PM, Chetan Mehrotra chetan.mehro...@gmail.com wrote:

On Thu, Jun 25, 2015 at 10:49 PM, Marius Petria mpet...@adobe.com wrote:
 AFAIU because the local events are changed to external events it means that 
 they can also be dropped completly under load, is that true?

Well they are not dropped. Observation in Oak works on basis of diff
of NodeState. So lets say you have an Observation queue of size 3 with
3 local events

1. [ns1, ns2, ci1]
2. [ns2, ns3, ci2]
3. [ns3, ns4, ci3]

Each tuple is [base root nodestate, nodeState post change, commit
info]. Now if 4 change event [ns4, ns5, ci4] comes the
BackgroundObserver has following options

1. Block on put
2. Have a indefinite length queue then just add to queue
3. Pull out the last event and replace it with merged

So current logic in Oak goes for #3.

1. [ns1, ns2, ci1]
2. [ns2, ns3, ci2]
3. [ns3, ns5, null]

Now when it merged/collapsed the content change then you cannot
associate any specific commit info to it. In such cases

1. You cannot determine which user has made that change.
2. Some changes might not get visible. For example if a foo property
is added in E3 and removed in E4. Then Merged content change would not
provide any indication that any such change happened

So such merged changes are shown as external i.e.
JackrabbitEvent#isExternal returns true for them currently. Problem is
currently its not possible to distinguish such collapsed events from
truely external events. May be we make that distinction so that
component which just rely on *some local change* to react can continue
to work. Though there is no gurantee that they see *each* local
change.

Chetan Mehrotra

[Observation] Question on local events reported as external

[Marius Petria]

Hi,

I understand that under high load local events can be reported as external. My 
question is why does this happen also for tarmk, where there is a single 
instance active (receiving external events on a single instance seems odd)? 
Also, is there a way to disable this functionality, meaning on tarmk to always 
receive the local events with their associated data (specifically the userData)?

Regards,
Marius

Re: [Observation] Question on local events reported as external

[Marius Petria]

AFAIU because the local events are changed to external events it means that 
they can also be dropped completly under load, is that true? 




On 6/25/15, 6:57 PM, Chetan Mehrotra chetan.mehro...@gmail.com wrote:

This is most likely due to rate at which event gets generated and thus
causing Observation queue to fill up (1000 default size). If the
backing JCR listener is slow in processing the queue would get filled
and BackgroundObserver would start compacting/merging the diff thus
converting local events to external.

There are two solutions
1. Throttle the commits - CommitRateLimiter
2. Have a non limiting queue - Then you end up with OOM if gap in
processing rate is large
Chetan Mehrotra


On Thu, Jun 25, 2015 at 8:47 PM, Marius Petria mpet...@adobe.com wrote:
 Hi,

 I understand that under high load local events can be reported as external. 
 My question is why does this happen also for tarmk, where there is a single 
 instance active (receiving external events on a single instance seems odd)? 
 Also, is there a way to disable this functionality, meaning on tarmk to 
 always receive the local events with their associated data (specifically the 
 userData)?

 Regards,
 Marius

Re: [Observation] Question on local events reported as external

[Chetan Mehrotra]

This is most likely due to rate at which event gets generated and thus
causing Observation queue to fill up (1000 default size). If the
backing JCR listener is slow in processing the queue would get filled
and BackgroundObserver would start compacting/merging the diff thus
converting local events to external.

There are two solutions
1. Throttle the commits - CommitRateLimiter
2. Have a non limiting queue - Then you end up with OOM if gap in
processing rate is large
Chetan Mehrotra


On Thu, Jun 25, 2015 at 8:47 PM, Marius Petria mpet...@adobe.com wrote:
 Hi,

 I understand that under high load local events can be reported as external. 
 My question is why does this happen also for tarmk, where there is a single 
 instance active (receiving external events on a single instance seems odd)? 
 Also, is there a way to disable this functionality, meaning on tarmk to 
 always receive the local events with their associated data (specifically the 
 userData)?

 Regards,
 Marius

Re: [Observation] Question on local events reported as external

[Chetan Mehrotra]

On Thu, Jun 25, 2015 at 10:49 PM, Marius Petria mpet...@adobe.com wrote:
 AFAIU because the local events are changed to external events it means that 
 they can also be dropped completly under load, is that true?

Well they are not dropped. Observation in Oak works on basis of diff
of NodeState. So lets say you have an Observation queue of size 3 with
3 local events

1. [ns1, ns2, ci1]
2. [ns2, ns3, ci2]
3. [ns3, ns4, ci3]

Each tuple is [base root nodestate, nodeState post change, commit
info]. Now if 4 change event [ns4, ns5, ci4] comes the
BackgroundObserver has following options

1. Block on put
2. Have a indefinite length queue then just add to queue
3. Pull out the last event and replace it with merged

So current logic in Oak goes for #3.

1. [ns1, ns2, ci1]
2. [ns2, ns3, ci2]
3. [ns3, ns5, null]

Now when it merged/collapsed the content change then you cannot
associate any specific commit info to it. In such cases

1. You cannot determine which user has made that change.
2. Some changes might not get visible. For example if a foo property
is added in E3 and removed in E4. Then Merged content change would not
provide any indication that any such change happened

So such merged changes are shown as external i.e.
JackrabbitEvent#isExternal returns true for them currently. Problem is
currently its not possible to distinguish such collapsed events from
truely external events. May be we make that distinction so that
component which just rely on *some local change* to react can continue
to work. Though there is no gurantee that they see *each* local
change.

Chetan Mehrotra

ExternalLoginModule question

[Alex Parvulescu]

Hi,

Following an investigation into the compactioncleanup issues, we (Michael
Duerig and I) noticed some SegmentNotFoundExceptions coming from the login
calls.

Blissfully unaware of how this code is actually supposed to work, I
stumbled upon a what looks like a missing call to #clearState() from within
the #commit method of the ExternalLoginModule [1].
I'm only assuming this because other implementations of the
AbstractLoginModule call #clearState when the login attempt did not go
through [2].

What I assume happens next is the old session (probably the initial one
created on the very first login call) could be reused throughout the
module's lifetime, which would in the end result in the SNFEs post
compaction.

Could someone clarify this?

thanks,
alex


[1]
https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/impl/ExternalLoginModule.java#L262

[2]
https://github.com/apache/jackrabbit-oak/blob/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/security/authentication/token/TokenLoginModule.java#L186

Re: Oak newbie question re. full versioning, branches, merges

[Marcel Reutegger]

Hi,

On 03/01/15 00:01, John Gretz jgre...@yahoo.com.INVALID wrote:
Since Oak supports only a single workspace but has capability of full
versioning (and lacks simple versioning), how does full versioning work
in this context? 

it means you can use version related operations that are limited
to a single workspace. E.g. cross workspace operations are not
supported.

Under what circumstances can one create a branch and how would that
branch be merged (given that the merge-related ops of the VersionManager
are not operational in Oak)?

the repository creates a branch automatically, when you restore
a previous version and then perform a check in.

changes from a branch cannot be merged because this is not yet
implemented. see also:
http://jackrabbit.apache.org/oak/docs/differences.html#Versioning

Regards
 Marcel

Oak newbie question re. full versioning, branches, merges

[John Gretz]

Hey guys,
Since Oak supports only a single workspace but has capability of full 
versioning (and lacks simple versioning), how does full versioning work in this 
context? Under what circumstances can one create a branch and how would that 
branch be merged (given that the merge-related ops of the VersionManager are 
not operational in Oak)?
Thx!John

Question about InitialContent

[Jegadisan]

Hi,

I understand that the Jcr class was provided to help with the quick setup for 
an Oak Repository, therefore it might not be the best place to start. I would 
just like to clarify about InitialContent though. 

I did a small test case, calling Jcr to create a repository with a FileStore, 
and noticed that it is indicating that the second time the repository gets 
created, the journal gets updated, and a new tar segment is also created.

I am assuming that once a repository has been created, initial content should 
not be called on it, especially so if we are looking to have read only 
instances of the oak repository for publishing servers perhaps ? Looking at the 
OSGI RepositoryManager, it seems to call InitialContent as well. 

Is this entirely required that initial content is called every time the 
repository is created ? I did another test case whereby I used a file store, 
and had the second invocation of the repository specifically not include 
InitialContent and it was able to read all the nodes created during the first 
invocation, and did not create the additional Journaling Entry.

Sorry if I am doing this wrong.



Thanks 

Question on QueryImpl

[Mario Rodriguez]

hi there,

Just have some question regarding the lucene QueryImpl for jackrabbit. I am
trying to check if it is possible to modify the xpath queries that use
bounded dates. The queries with bounded dates in xpath take much longer to
execute than queries executed against long properties.

Therefore I was trying to see if there is a way to modify the
XpathQueryBuilder so that it will be smart enough to change whenever the
query statement includes a bounded date (start and end).

Any suggestions or ideas?

Thanks,
Mario

Question regarding missing _lastRev recovery - OAK-1295

[Amit Jain]

Hi,

How do we expose _lastRev recovery operation? This would need to check all
the cluster nodes info and run recovery for those nodes which need recovery.

1. We either have a scheduled job which checks all the nodes and run the
recovery. What should be the interval to trigger the job?
2. Or if we want it run only when triggered manually, then expose an
appropriate MBean.


Thanks
Amit

RE: Question regarding missing _lastRev recovery - OAK-1295

[Marcel Reutegger]

Hi,

I think the recovery should be triggered automatically by the system when:

1) a cluster node starts up and sees it didn't shut down properly. I'm not
sure this information is available, but remember we discussed this once.

2) a cluster node sees a lease timeout of another cluster node and initiates
the recovery for the failed cluster node. this check could be done in the
background operations thread on a regular basis. probably depending on
the lease interval.

In addition it would probably also be useful to have the recovery operation
available as a command in oak-run. that way you can manually trigger it from
the command line. WDYT?

Regards
 Marcel

 How do we expose _lastRev recovery operation? This would need to check
 all
 the cluster nodes info and run recovery for those nodes which need
 recovery.
 
 1. We either have a scheduled job which checks all the nodes and run the
 recovery. What should be the interval to trigger the job?
 2. Or if we want it run only when triggered manually, then expose an
 appropriate MBean.
 
 
 Thanks
 Amit

Re: Question regarding missing _lastRev recovery - OAK-1295

[Amit Jain]

Hi,

 1) a cluster node starts up and sees it didn't shut down properly. I'm
not
 sure this information is available, but remember we discussed this once.

Yes, this case has been taken care of in the startup.

  this check could be done in the
 background operations thread on a regular basis. probably depending on
 the lease interval.

The lease time is set to 1 minute. Would it be ok to check this every
minute, from every node?

Thanks
Amit


On Wed, Apr 2, 2014 at 4:14 PM, Marcel Reutegger mreut...@adobe.com wrote:

 Hi,

 I think the recovery should be triggered automatically by the system when:

 1) a cluster node starts up and sees it didn't shut down properly. I'm not
 sure this information is available, but remember we discussed this once.

 2) a cluster node sees a lease timeout of another cluster node and
 initiates
 the recovery for the failed cluster node. this check could be done in the
 background operations thread on a regular basis. probably depending on
 the lease interval.

 In addition it would probably also be useful to have the recovery operation
 available as a command in oak-run. that way you can manually trigger it
 from
 the command line. WDYT?

 Regards
  Marcel

  How do we expose _lastRev recovery operation? This would need to check
  all
  the cluster nodes info and run recovery for those nodes which need
  recovery.
 
  1. We either have a scheduled job which checks all the nodes and run the
  recovery. What should be the interval to trigger the job?
  2. Or if we want it run only when triggered manually, then expose an
  appropriate MBean.
 
 
  Thanks
  Amit

Re: Question regarding missing _lastRev recovery - OAK-1295

[Chetan Mehrotra]

 The lease time is set to 1 minute. Would it be ok to check this every
minute, from every node?

Adding to that the default time intervals are
- asyncDelay = 1 sec - The background operation are performed every 1
sec per cluster node. If nothing changes we would fire
1query/sec/cluster node to check the head revision

- cluster lease time = 1 min - This is the time after a cluster lease
would be renewed.

So we need to decide the time interval for Job for detecting recovery condition
Chetan Mehrotra


On Wed, Apr 2, 2014 at 4:31 PM, Amit Jain am...@ieee.org wrote:
 Hi,

 1) a cluster node starts up and sees it didn't shut down properly. I'm
 not
 sure this information is available, but remember we discussed this once.

 Yes, this case has been taken care of in the startup.

  this check could be done in the
 background operations thread on a regular basis. probably depending on
 the lease interval.

 The lease time is set to 1 minute. Would it be ok to check this every
 minute, from every node?

 Thanks
 Amit


 On Wed, Apr 2, 2014 at 4:14 PM, Marcel Reutegger mreut...@adobe.com wrote:

 Hi,

 I think the recovery should be triggered automatically by the system when:

 1) a cluster node starts up and sees it didn't shut down properly. I'm not
 sure this information is available, but remember we discussed this once.

 2) a cluster node sees a lease timeout of another cluster node and
 initiates
 the recovery for the failed cluster node. this check could be done in the
 background operations thread on a regular basis. probably depending on
 the lease interval.

 In addition it would probably also be useful to have the recovery operation
 available as a command in oak-run. that way you can manually trigger it
 from
 the command line. WDYT?

 Regards
  Marcel

  How do we expose _lastRev recovery operation? This would need to check
  all
  the cluster nodes info and run recovery for those nodes which need
  recovery.
 
  1. We either have a scheduled job which checks all the nodes and run the
  recovery. What should be the interval to trigger the job?
  2. Or if we want it run only when triggered manually, then expose an
  appropriate MBean.
 
 
  Thanks
  Amit

Re: Oak Property Index Question

[Ben Zahler]

Thanks for your response, Davide, I did get the propertyIndex to work!

I have one question however:
I can only get the query to work when querying for nt:unstructured, but
not for nt:base. Is this the expected behaviour?

I can create my index either with the „declaringNodeTypes“ property set to
„nt:base“ or with the property omitted:

Node propertyIndexNode =
session.getNode(/oak:index).addNode(testIndex,
oak:QueryIndexDefinition);
propertyIndexNode.setProperty(type, property) ;
propertyIndexNode.setProperty(propertyNames,
trainingProperty);
propertyIndexNode.setProperty(declaringNodeTypes,nt:base“);
// can be omitted
propertyIndexNode.setProperty(unique, false);
propertyIndexNode.setProperty(reindex, true);

In both cases the following query is using the index:
SELECT *  FROM [nt:unstructured] WHERE trainingProperty = 13

But the following query is not:
SELECT *  FROM [nt:base] WHERE trainingProperty = 13


Thanks,
Ben

Inside Solutions AG | Felsenstrasse 11 | 4450 Sissach | Schweiz
Telefon: +41 61 551 00 40 | Direkt: +41 61 551 00 43
http://www.inside-solutions.ch http://www.inside-solutions.ch/





Am 14.03.14 09:12 schrieb Davide Giannella unter
giannella.dav...@gmail.com:

On 13/03/2014 17:41, Ben Zahler wrote:
 I execute the following query:
 String expression = SELECT *  FROM [nt:base] WHERE trainingProperty 
 10;
 Query query = queryManager.createQuery(expression,Query.JCR_SQL2);

 As expected, I get the message
  WARN  o.a.j.o.s.q.Cursors$TraversingCursor - Traversed 1000 nodes
 with filter Filter(query=SELECT *  FROM [nt:base] WHERE
 trainingProperty  10, path=/*, property=[trainingProperty=(10..]);
 consider creating an index or changing the query

 ...
 Node propertyIndexNode =
 session.getNode(/oak:index).addNode(testIndex,
 oak:QueryIndexDefinition);
 propertyIndexNode.setProperty(type, property) ;
 propertyIndexNode.setProperty(propertyNames,
 trainingProperty);
 propertyIndexNode.setProperty(unique, false);
 propertyIndexNode.setProperty(reindex, true);
 session.save();

The PropertyIndex does not react on range queries. Only equality, in and
nonnull properties.

D.

Oak Property Index Question

[Ben Zahler]

Hi all,
I am currently writing a training for Oak Users and have found an issue with 
query indexing where things to not work as expected.

I execute the following query:
String expression = SELECT *  FROM [nt:base] WHERE trainingProperty  10;
Query query = queryManager.createQuery(expression,Query.JCR_SQL2);

As expected, I get the message
 WARN  o.a.j.o.s.q.Cursors$TraversingCursor - Traversed 1000 nodes with filter 
Filter(query=SELECT *  FROM [nt:base] WHERE trainingProperty  10, path=/*, 
property=[trainingProperty=(10..]); consider creating an index or changing the 
query

However, I did not manage to create a PropertyIndex to index the query.

Here is the code that  I used to create the index:

Node propertyIndexNode = 
session.getNode(/oak:index).addNode(testIndex, oak:QueryIndexDefinition);
propertyIndexNode.setProperty(type, property) ;
propertyIndexNode.setProperty(propertyNames, trainingProperty);
propertyIndexNode.setProperty(unique, false);
propertyIndexNode.setProperty(reindex, true);
session.save();

From what I read, I would expect the query to be indexed with that node, but I 
still get the message above.
Is there anything that I missed?

A running java sample (requiring some oak imports) is attached.

Regards,
Ben Zahler
Inside Solutions AG | Felsenstrasse 11 | 4450 Sissach | Schweiz
Telefon: +41 61 551 00 40 | Direkt: +41 61 551 00 43
http://www.inside-solutions.chhttp://www.inside-solutions.ch/

Question on Activity

[Young Li]

Dear All,

I am confused on the concept Activity in JCR.  By the introduction of it, it 
uses to record some changes on the session and that could be used to reused in 
other workspaces.

I try to build a simple demo to check the function of Activity, but it failed.  
 Attaching my code here, please give me some guide.  Thanks.

 //create a Activity node
Node activity = vm.getActivity();
if (null == activity) {
activity = vm.createActivity(test_act_default_);
}
System.out.println(activity.getUUID());
Node root = adminSession.getRootNode();
Node testNode = root.addNode(test);

testNode.addMixin(mix:versionable);
adminSession.save();

vm.setActivity(activity);//set activity and begin to record the changes
vm.checkout(/test);
testNode.addNode(test_sub1);
testNode.addNode(test_sub2);
adminSession.save();
vm.setActivity(null);// stop the activity and the activity should 
contains the addition of two child nodes.
vm.checkin(/test);

After above code, I try to reuse the activity in another workspace 'test'.

Session test_session = rep.login(new SimpleCredentials(admin,
admin.toCharArray()), test);
 //Create a similar root node here
 Node root = test_session.getRootNode();
Node testNode = root.addNode(test);
testNode.addMixin(mix:versionable);
test_session.save();

VersionManagerImpl vm = 
(VersionManagerImpl)adminSession.getWorkspace().getVersionManager();
String uuid_act = 6047919f-2740-426c-ba91-94c1cc60e4bd;//this is UUID 
of activity node previously created.
Node activity_node = test_session.getNodeByUUID(uuid_act);
 NodeIterator nit = vm.merge(activity_node);//merge, but there is no 
anything changes to be taken effect.

It is really appreciated if any suggestion is received.  Thanks.

Best regards,

Young

Re: Question about Oak search/query.

[Jukka Zitting]

Hi,

On Thu, Dec 5, 2013 at 9:36 PM, Ian Boston i...@tfd.co.uk wrote:
 Will the search index contain access control information or will the
 search results be filtered as each result is retrieved ?

The results will be filtered after the index lookup. It would be
possible for a custom search index to do the access checks already
when building/updating the index, but even in that case the query
engine would still double-check the access rights (the benefit would
be to avoid having to retrieve and then discard many inaccessible
hits).

 If the number of terms in the query exceeds the number of terms
 supported by Solr, does the Oak handle that transparently ?

I'm not sure, you'll need to look at the oak-solr indexing code. Or
perhaps Tommaso who wrote the code can chime in here.

BR,

Jukka Zitting

Re: Question about MVCC with MongoMK.

[Jukka Zitting]

Hi,

On Thu, Dec 5, 2013 at 9:43 PM, Ian Boston i...@tfd.co.uk wrote:
 Is it possible to branch an Oak repository and maintain a detached
 root node for a period of time that one or more Oak instances attached
 to a MongoDB instance can follow for a short period of time before
 merging the branch back into the main tree ?

The SegmentMK (with the MongoDB backend) can do this using the
hierarchical journal feature. The SegmentMK maintains one or more
journals that each track the evolution of a particular branch of the
repository. These branches would normally be automatically merged back
to the root journal, but a particular deployment could easily
disable automatic merging for a particular branch and use it for a
purpose like the one you described.

BR,

Jukka Zitting

Re: Question about MVCC with MongoMK.

[Bertrand Delacretaz]

Hi,

On Fri, Dec 6, 2013 at 11:12 AM, Jukka Zitting jukka.zitt...@gmail.com wrote:
 ...The SegmentMK (with the MongoDB backend) can do this using the
 hierarchical journal feature The SegmentMK maintains one or more
 journals that each track the evolution of a particular branch of the
 repository. These branches would normally be automatically merged back
 to the root journal, but a particular deployment could easily
 disable automatic merging for a particular branch and use it for a
 purpose like the one you described

Is there a way to tell the repository to start operating on such a
branch forever for a given client?

IIUC Ian's scenario, an application instance would tell Oak create a
BEFORE_UPGRADE branch and start working on that from now on so that
the content can be upgraded in the background and tested on other
application instances, before eventually merging the BEFORE_UPGRADE
branch back.

Is that possible today, or reasonably simple to implement? If yes that
would enable such a scenario with minimal application changes, which
sounds extremely useful.

-Bertrand

Re: Question about MVCC with MongoMK.

[Jukka Zitting]

Hi,

On Fri, Dec 6, 2013 at 5:26 AM, Bertrand Delacretaz
bdelacre...@apache.org wrote:
 Is that possible today, or reasonably simple to implement? If yes that
 would enable such a scenario with minimal application changes, which
 sounds extremely useful.

It's not available yet, but shouldn't be too difficult to implement.

The main question here is whether we want to go down that path, as the
feature is only available with the SegmentMK (at least for now) and
we've generally wanted to avoid exposing such implementation-specific
features to higher level code.

BR,

Jukka Zitting

Re: Question about Oak search/query.

[Tommaso Teofili]

Hi all,

2013/12/6 Jukka Zitting jukka.zitt...@gmail.com

 Hi,

 On Thu, Dec 5, 2013 at 9:36 PM, Ian Boston i...@tfd.co.uk wrote:
  Will the search index contain access control information or will the
  search results be filtered as each result is retrieved ?

 The results will be filtered after the index lookup. It would be
 possible for a custom search index to do the access checks already
 when building/updating the index, but even in that case the query
 engine would still double-check the access rights (the benefit would
 be to avoid having to retrieve and then discard many inaccessible
 hits).


by the way, probably there's room for some optimization, e.g. very simple
idea: exclude paths at depth 1 (children of root node) the principle is not
able to read (which may mean adding them to the query passed to the Index
implementation), if any, then you'd always have to apply fine grained ACLs
on the result but maybe excluding some branches from start may help.



  If the number of terms in the query exceeds the number of terms
  supported by Solr, does the Oak handle that transparently ?

 I'm not sure, you'll need to look at the oak-solr indexing code. Or
 perhaps Tommaso who wrote the code can chime in here.


sure.
What limitation are you exactly referring to? Is it the BooleanQuery max
clause limit [1]?

Regards,
Tommaso


[1] :
http://lucene.apache.org/core/4_6_0/core/org/apache/lucene/search/BooleanQuery.TooManyClauses.html


 BR,

 Jukka Zitting

Re: Question: In Repository Index file.

[Alex Parvulescu]

Hi,

On Fri, Dec 6, 2013 at 11:06 AM, Jukka Zitting jukka.zitt...@gmail.comwrote:

 Hi,

 On Fri, Dec 6, 2013 at 1:25 AM, Ian Boston i...@tfd.co.uk wrote:
  In Oak when a index is stored in the repository, how is it updated
  when the repository is MongoDB backed and there are multiple JVM
  processes connected to the MongoDB ?

 That depends on the index and MK implementations. For example the
 PropertyIndex uses an index structure that can be updated concurrently
 when the updates affect different areas of the content repository.
 When using the MongoMK backend concurrent updates to the same nodes
 will automatically be synchronized, and with the SegmentMK (which also
 can be used with MongoDB) all commits against the same journal are
 synchronized. In both cases concurrent updates will automatically get
 resolved.

  Also if using SolrCloud as a search index, is it possible to fallback
  to an internal repository stored index if the the SolrCloud index
  becomes unavailable ?

 Yes. The query engine will automatically pick the best available index
 for each query execution. If a particular index is not available, then
 the second-best match for those queries that would have used it would
 automatically get picked.



There is one minor nitpick with this statement.

So far we've assumed that the solr index will be used for full-text queries
only. the only fallback you could use if the solr index becomes unavailable
is the lucene one, but as far as I know we've said that you would usually
use one _or_ the other.
Areas of concern are: the full-text indexing settings may differ, and the
cost output may need to be tricked into treating the local lucene index as
a fallback and not a competing full-text index.
But this is definitely doable.




 BR,

 Jukka Zitting

Re: Question: In Repository Index file.

[Tommaso Teofili]

2013/12/6 Alex Parvulescu alex.parvule...@gmail.com

 Hi,

 On Fri, Dec 6, 2013 at 11:06 AM, Jukka Zitting jukka.zitt...@gmail.com
 wrote:

  Hi,
 
  On Fri, Dec 6, 2013 at 1:25 AM, Ian Boston i...@tfd.co.uk wrote:
   In Oak when a index is stored in the repository, how is it updated
   when the repository is MongoDB backed and there are multiple JVM
   processes connected to the MongoDB ?
 
  That depends on the index and MK implementations. For example the
  PropertyIndex uses an index structure that can be updated concurrently
  when the updates affect different areas of the content repository.
  When using the MongoMK backend concurrent updates to the same nodes
  will automatically be synchronized, and with the SegmentMK (which also
  can be used with MongoDB) all commits against the same journal are
  synchronized. In both cases concurrent updates will automatically get
  resolved.
 
   Also if using SolrCloud as a search index, is it possible to fallback
   to an internal repository stored index if the the SolrCloud index
   becomes unavailable ?
 
  Yes. The query engine will automatically pick the best available index
  for each query execution. If a particular index is not available, then
  the second-best match for those queries that would have used it would
  automatically get picked.
 


 There is one minor nitpick with this statement.

 So far we've assumed that the solr index will be used for full-text queries
 only. the only fallback you could use if the solr index becomes unavailable
 is the lucene one, but as far as I know we've said that you would usually
 use one _or_ the other.
 Areas of concern are: the full-text indexing settings may differ, and the
 cost output may need to be tricked into treating the local lucene index as
 a fallback and not a competing full-text index.
 But this is definitely doable.


good point Alex, and probably we may have to write some tests for the cost
comparison for different queries with one or more running indexes to
eventually tune the cost evaluation to work properly in the different
setups.

Tommaso





 
  BR,
 
  Jukka Zitting
 

Re: Question about MVCC with MongoMK.

[Bertrand Delacretaz]

Hi,

On Fri, Dec 6, 2013 at 11:36 AM, Jukka Zitting jukka.zitt...@gmail.com wrote:
 ...The main question here is whether we want to go down that path, as the
 feature is only available with the SegmentMK (at least for now) and
 we've generally wanted to avoid exposing such implementation-specific
 features to higher level code...

Agreed, OTOH the scenario that we're discussing here looks extremely
useful in clustered environments, where managing upgrades and
minimizing downtime is hard. I suspect Ian will agree that having this
in Oak would be very valuable, even if that requires using a specific
microkernel.

-Bertrand

Re: Question about Oak search/query.

[Ian Boston]

Hi,

On 6 December 2013 16:12, Tommaso Teofili tommaso.teof...@gmail.com wrote:
 Hi all,

 2013/12/6 Jukka Zitting jukka.zitt...@gmail.com

 Hi,

 On Thu, Dec 5, 2013 at 9:36 PM, Ian Boston i...@tfd.co.uk wrote:
  Will the search index contain access control information or will the
  search results be filtered as each result is retrieved ?

 The results will be filtered after the index lookup. It would be
 possible for a custom search index to do the access checks already
 when building/updating the index, but even in that case the query
 engine would still double-check the access rights (the benefit would
 be to avoid having to retrieve and then discard many inaccessible
 hits).


 by the way, probably there's room for some optimization, e.g. very simple
 idea: exclude paths at depth 1 (children of root node) the principle is not
 able to read (which may mean adding them to the query passed to the Index
 implementation), if any, then you'd always have to apply fine grained ACLs
 on the result but maybe excluding some branches from start may help.


Ok, thank you, it is as I thought. It may be possible to work around
it by adding  some properties to make the result dense.




  If the number of terms in the query exceeds the number of terms
  supported by Solr, does the Oak handle that transparently ?

 I'm not sure, you'll need to look at the oak-solr indexing code. Or
 perhaps Tommaso who wrote the code can chime in here.


 sure.
 What limitation are you exactly referring to? Is it the BooleanQuery max
 clause limit [1]?

Yes, I believe its that limit.

Do you know how many that is in the version used by Oak ?


 Regards,
 Tommaso


 [1] :
 http://lucene.apache.org/core/4_6_0/core/org/apache/lucene/search/BooleanQuery.TooManyClauses.html


 BR,

 Jukka Zitting

Re: Question: In Repository Index file.

[Ian Boston]

Hi,
Thanks all for the clarification. Good to know there is fallback.

If the Solr index is intended for full text, can it still be used to
build facets on a reasonably well defined set of properties ?

Best Regards
Ian

On 6 December 2013 16:20, Tommaso Teofili tommaso.teof...@gmail.com wrote:
 2013/12/6 Alex Parvulescu alex.parvule...@gmail.com

 Hi,

 On Fri, Dec 6, 2013 at 11:06 AM, Jukka Zitting jukka.zitt...@gmail.com
 wrote:

  Hi,
 
  On Fri, Dec 6, 2013 at 1:25 AM, Ian Boston i...@tfd.co.uk wrote:
   In Oak when a index is stored in the repository, how is it updated
   when the repository is MongoDB backed and there are multiple JVM
   processes connected to the MongoDB ?
 
  That depends on the index and MK implementations. For example the
  PropertyIndex uses an index structure that can be updated concurrently
  when the updates affect different areas of the content repository.
  When using the MongoMK backend concurrent updates to the same nodes
  will automatically be synchronized, and with the SegmentMK (which also
  can be used with MongoDB) all commits against the same journal are
  synchronized. In both cases concurrent updates will automatically get
  resolved.
 
   Also if using SolrCloud as a search index, is it possible to fallback
   to an internal repository stored index if the the SolrCloud index
   becomes unavailable ?
 
  Yes. The query engine will automatically pick the best available index
  for each query execution. If a particular index is not available, then
  the second-best match for those queries that would have used it would
  automatically get picked.
 


 There is one minor nitpick with this statement.

 So far we've assumed that the solr index will be used for full-text queries
 only. the only fallback you could use if the solr index becomes unavailable
 is the lucene one, but as far as I know we've said that you would usually
 use one _or_ the other.
 Areas of concern are: the full-text indexing settings may differ, and the
 cost output may need to be tricked into treating the local lucene index as
 a fallback and not a competing full-text index.
 But this is definitely doable.


 good point Alex, and probably we may have to write some tests for the cost
 comparison for different queries with one or more running indexes to
 eventually tune the cost evaluation to work properly in the different
 setups.

 Tommaso





 
  BR,
 
  Jukka Zitting
 

Re: Question about MVCC with MongoMK.

[Ian Boston]

On 6 December 2013 16:44, Bertrand Delacretaz bdelacre...@apache.org wrote:
 Hi,

 On Fri, Dec 6, 2013 at 11:36 AM, Jukka Zitting jukka.zitt...@gmail.com 
 wrote:
 ...The main question here is whether we want to go down that path, as the
 feature is only available with the SegmentMK (at least for now) and
 we've generally wanted to avoid exposing such implementation-specific
 features to higher level code...

 Agreed, OTOH the scenario that we're discussing here looks extremely
 useful in clustered environments, where managing upgrades and
 minimizing downtime is hard. I suspect Ian will agree that having this
 in Oak would be very valuable, even if that requires using a specific
 microkernel.

Yes,
very valuable indeed, and well worth doing (imho, be happy to help if
I am capable) I think, subject to some experimentation it will bring
upgrades on Oak to a new level, especially in large clusters.

Best Regards
Ian



 -Bertrand

Re: Question about Oak search/query.

[Tommaso Teofili]

Hi,

2013/12/6 Ian Boston i...@tfd.co.uk

 Hi,

 On 6 December 2013 16:12, Tommaso Teofili tommaso.teof...@gmail.com
 wrote:
  Hi all,
 
  2013/12/6 Jukka Zitting jukka.zitt...@gmail.com
 
  Hi,
 
  On Thu, Dec 5, 2013 at 9:36 PM, Ian Boston i...@tfd.co.uk wrote:
   Will the search index contain access control information or will the
   search results be filtered as each result is retrieved ?
 
  The results will be filtered after the index lookup. It would be
  possible for a custom search index to do the access checks already
  when building/updating the index, but even in that case the query
  engine would still double-check the access rights (the benefit would
  be to avoid having to retrieve and then discard many inaccessible
  hits).
 
 
  by the way, probably there's room for some optimization, e.g. very simple
  idea: exclude paths at depth 1 (children of root node) the principle is
 not
  able to read (which may mean adding them to the query passed to the Index
  implementation), if any, then you'd always have to apply fine grained
 ACLs
  on the result but maybe excluding some branches from start may help.


 Ok, thank you, it is as I thought. It may be possible to work around
 it by adding  some properties to make the result dense.

 
 
 
   If the number of terms in the query exceeds the number of terms
   supported by Solr, does the Oak handle that transparently ?
 
  I'm not sure, you'll need to look at the oak-solr indexing code. Or
  perhaps Tommaso who wrote the code can chime in here.
 
 
  sure.
  What limitation are you exactly referring to? Is it the BooleanQuery max
  clause limit [1]?

 Yes, I believe its that limit.

 Do you know how many that is in the version used by Oak ?


At the moment Oak has the Solr dependency with scope provided, version
4.1.0 (which uses Lucene 4.1.0) so that one could use from 4.1.x to the
latest (4.6.0 right now).
Default is 1024.

Regards,
Tommaso



 
  Regards,
  Tommaso
 
 
  [1] :
 
 http://lucene.apache.org/core/4_6_0/core/org/apache/lucene/search/BooleanQuery.TooManyClauses.html
 
 
  BR,
 
  Jukka Zitting
 

Re: Question about Oak search/query.

[Ian Boston]

On Friday, December 6, 2013, Tommaso Teofili wrote:

 Hi,

 2013/12/6 Ian Boston i...@tfd.co.uk javascript:;

  Hi,
 
  On 6 December 2013 16:12, Tommaso Teofili 
  tommaso.teof...@gmail.comjavascript:;
 
  wrote:
   Hi all,
  
   2013/12/6 Jukka Zitting jukka.zitt...@gmail.com javascript:;
  
   Hi,
  
   On Thu, Dec 5, 2013 at 9:36 PM, Ian Boston i...@tfd.co.ukjavascript:;
 wrote:
Will the search index contain access control information or will the
search results be filtered as each result is retrieved ?
  
   The results will be filtered after the index lookup. It would be
   possible for a custom search index to do the access checks already
   when building/updating the index, but even in that case the query
   engine would still double-check the access rights (the benefit would
   be to avoid having to retrieve and then discard many inaccessible
   hits).
  
  
   by the way, probably there's room for some optimization, e.g. very
 simple
   idea: exclude paths at depth 1 (children of root node) the principle is
  not
   able to read (which may mean adding them to the query passed to the
 Index
   implementation), if any, then you'd always have to apply fine grained
  ACLs
   on the result but maybe excluding some branches from start may help.
 
 
  Ok, thank you, it is as I thought. It may be possible to work around
  it by adding  some properties to make the result dense.
 
  
  
  
If the number of terms in the query exceeds the number of terms
supported by Solr, does the Oak handle that transparently ?
  
   I'm not sure, you'll need to look at the oak-solr indexing code. Or
   perhaps Tommaso who wrote the code can chime in here.
  
  
   sure.
   What limitation are you exactly referring to? Is it the BooleanQuery
 max
   clause limit [1]?
 
  Yes, I believe its that limit.
 
  Do you know how many that is in the version used by Oak ?
 

 At the moment Oak has the Solr dependency with scope provided, version
 4.1.0 (which uses Lucene 4.1.0) so that one could use from 4.1.x to the
 latest (4.6.0 right now).
 Default is 1024.


Hi,

Thank you. All questions on this thread answered.

Best regards
Ian





 Regards,
 Tommaso


 
  
   Regards,
   Tommaso
  
  
   [1] :
  
 
 http://lucene.apache.org/core/4_6_0/core/org/apache/lucene/search/BooleanQuery.TooManyClauses.html
  
  
   BR,
  
   Jukka Zitting
  
 

Re: Question: In Repository Index file.

[Ian Boston]

On Friday, December 6, 2013, Alex Parvulescu wrote:

 if we are being technical :)

 the property index already has all the info you'd need, just list all the
 keys and you get the facets (not so easy for item counts for one facet
 though).



Do IIUC?
The facet support[1] in the Solr Api is not exposed except by going direct
to the Solr Api?

Best regards
Ian

1. http://wiki.apache.org/solr/SolrFacetingOverview










 On Fri, Dec 6, 2013 at 4:58 PM, Tommaso Teofili
 tommaso.teof...@gmail.com javascript:;wrote:

  Hi,
 
  2013/12/6 Ian Boston i...@tfd.co.uk
 
   Hi,
   Thanks all for the clarification. Good to know there is fallback.
  
   If the Solr index is intended for full text, can it still be used to
   build facets on a reasonably well defined set of properties ?
  
 
  technically speaking of course, we may also support facets for the Lucene
  index [1].
  What I wonder is if / how we could expose them on the JCR API level.
  Any idea?
 
  Regards,
  Tommaso
 
  [1] :
 
 
 http://lucene.apache.org/core/4_0_0/facet/org/apache/lucene/facet/doc-files/userguide.html
 
 
  
   Best Regards
   Ian
  
   On 6 December 2013 16:20, Tommaso Teofili tommaso.teof...@gmail.com
   wrote:
2013/12/6 Alex Parvulescu alex.parvule...@gmail.com
   
Hi,
   
On Fri, Dec 6, 2013 at 11:06 AM, Jukka Zitting 
  jukka.zitt...@gmail.com
wrote:
   
 Hi,

 On Fri, Dec 6, 2013 at 1:25 AM, Ian Boston i...@tfd.co.uk wrote:
  In Oak when a index is stored in the repository, how is it
 updated
  when the repository is MongoDB backed and there are multiple JVM
  processes connected to the MongoDB ?

 That depends on the index and MK implementations. For example the
 PropertyIndex uses an index structure that can be updated
  concurrently
 when the updates affect different areas of the content repository.
 When using the MongoMK backend concurrent updates to the same
 nodes
 will automatically be synchronized, and with the SegmentMK (which
  also
 can be used with MongoDB) all commits against the same journal are
 synchronized. In both cases concurrent updates will automatically
  get
 resolved.

  Also if using SolrCloud as a search index, is it possible to
   fallback
  to an internal repository stored index if the the SolrCloud
 index
  becomes unavailable ?

 Yes. The query engine will automatically pick the best available
  index
 for each query execution. If a particular index is not available,
  then
 the second-best match for those queries that would have used it
  would
 automatically get picked.

   
   
There is one minor nitpick with this statement.
   
So far we've assumed that the solr index will be used for full-text
   queries
only. the only fallback you could use if the solr index becomes
   unavailable
is the lucene one, but as far as I know we've said that you would
   usually
use one _or_ the other.
Areas of concern are: the full-text indexing settings may differ,
 and
   the
cost output may need to be tricked into treating the local lucene
  index
   as
a fallback and not a competing full-text index.
But this is definitely doable.
   
   
good point Alex, and probably we may have to write some tests for the
   cost
comparison for different queries with one or more running indexes to
eventually tune the cost evaluation to work properly in the differ

Question about Oak search/query.

[Ian Boston]

Hi,
IIUC Search type queries will use Solr. I would like to verify some
things for my own curiosity.

Will the search index contain access control information or will the
search results be filtered as each result is retrieved ?

If the number of terms in the query exceeds the number of terms
supported by Solr, does the Oak handle that transparently ?

Best Regards
Ian

Question about MVCC with MongoMK.

[Ian Boston]

Hi,

Is it possible to branch an Oak repository and maintain a detached
root node for a period of time that one or more Oak instances attached
to a MongoDB instance can follow for a short period of time before
merging the branch back into the main tree ?

Although there may be conflicting changes, I would like the main tree
to take precedence on merge.

I am thinking of a cluster containing a number of instances where
changes are made to a subset of the cluster prior to allow those
changes to tested before the whole cluster is exposed to those
changes. ie managed repository content upgrade.

Best Regards
Ian

Question: In Repository Index file.

[Ian Boston]

Hi,
In Oak when a index is stored in the repository, how is it updated
when the repository is MongoDB backed and there are multiple JVM
processes connected to the MongoDB ?

Also if using SolrCloud as a search index, is it possible to fallback
to an internal repository stored index if the the SolrCloud index
becomes unavailable ?

TIA,
Best Regards
Ian

Re: SNS support question

[Jukka Zitting]

Hi,

On Wed, Nov 20, 2013 at 1:29 PM, Tobias Bocanegra tri...@apache.org wrote:
 when I do:

 Session.getNode(/foo/bar[2]);

 I get:

 javax.jcr.RepositoryException: Invalid name or path: /foo/bar[2]

 but I think this should be ItemNotFound (if the item does not exist),
 irrespective of SNS support or not.

Agreed.

BR,

Jukka Zitting

SNS support question

[Tobias Bocanegra]

Hi,

when I do:

Session.getNode(/foo/bar[2]);

I get:

javax.jcr.RepositoryException: Invalid name or path: /foo/bar[2]

but I think this should be ItemNotFound (if the item does not exist),
irrespective of SNS support or not.

WDYT?
Regards, Toby

Re: Kernel stats question.

[Jukka Zitting]

Hi,

On Fri, Mar 1, 2013 at 9:42 AM, Ian Boston i...@tfd.co.uk wrote:
 I am more interested in counters than average or last operation time
 measurements  as that gives and idea of where hotspots, multi threaded
 throughput at full load and anomalies rather than general slowness.
 (not saying time measurements are not useful, just hard to interpret
 in a highly multi threaded server under load)

Good point. It would be good to use OAK-364 to gather ideas for
numbers that would be useful or at least interesting to track.

 Does that mean observers get a stream revision tokens, and its up to
 them to queue the revision for later processing, even if that means
 just keeping the last one worked on and the most recent (ie no queue
 at all, just a range of revisions) ?

That depends on the level at which you're observing the repository.

At the lowest level you can just poll the repository for  new
revisions of the content tree and do a content diff to find out what
changed. The repository itself does nothing special for you, just
gives you access to the latest revision and the ability to compare two
revisions of the repository.

Since explicitly managing such a polling mechanism can be a bit
cumbersome, we also provide a way to register listeners (see the
o.a.j.oak.spi.commit.Observer interface) that get notified when there
are new revisions in the repository, like you describe above. The
thread that makes these contentChanged() callbacks is controlled by
the repository, so the observers should avoid too expensive
calculations.

Finally at the JCR level we have the JCR observation listeners that
receive a stream of event objects instead of repository revisions.
Again the repository is in charge of making the callbacks and thus the
observer should be mindful of the amount of time it takes.

 Will the Kernel blacklist slow observers, as the OSGi event manager does?

Currently it doesn't, but for the latter two cases we'll probably need
something like that. The first case is essentially decoupled from the
underlying repository, so there's no need for the repository to worry
about such observers. (Of course they could still consume a lot of CPU
and IO, bu that would then be a higher level deployment concern and
would only indirectly affect the repository.)

BR,

Jukka Zitting

Kernel stats question.

[Ian Boston]

Hi,
Are there plans to expose kernel stats from Oak ?

From a client point of view:
I am thinking access to an immutable map of read only long values that
give the client information about the internal health and throughput
of an Oak kernel, rather like the information available under /proc on
linux.

From a kernel pov it might just be AtomicLongs in a map. All they do
is count certain actions or set an absolute value with as low a cost
as possible to the internal workings of the core, and no risk of
concurrency issues.

I am not thinking of anything as sophisticated as the
RepositoryStatistics TimeSeries thats in Jackrabbit.

The only need to be centralised and exposed, clients can decide what
to do with the values.

Use Case:
The Jackrabbit ObservationDispatcher has a queue that contains a
AtomicInteger indicating the size of the queue. If Observers are slow
in responding to asynchronous notifications the queue can grow
rapidly. Having stats on the queue would give valuable feedback into
this area and in certain cases fast long running notifiers (JCR
writers) could adjust their behaviour based on the even. Even seeing
the size of the queue in a monitoring tool would be useful.

Having telemetry in the Oak kernel will certainly help clients and may
also help optimise and benchmark Oak.
Ian

Re: Kernel stats question.

[Jukka Zitting]

Hi,

On Thu, Feb 28, 2013 at 11:55 PM, Ian Boston i...@tfd.co.uk wrote:
 Are there plans to expose kernel stats from Oak ?

Yes, see OAK-364 [1]. Though so far we haven't yet implemented
anything along these lines.

One related idea I was already thinking about implementing is exposing
cache statistics [2] from the Guava caches we're using at the
MicroKernel level.

 The Jackrabbit ObservationDispatcher has a queue that contains a
 AtomicInteger indicating the size of the queue. If Observers are slow
 in responding to asynchronous notifications the queue can grow
 rapidly.

This actually is an area that won't be an issue with Oak, as instead
of using a centralized queue from where events are pushed to observers
we're allowing each observer to generate events by comparing
successive repository revisions at their own pace. Thus one slow
observer will only block itself.

Of course there are other areas where such metrics will still be highly useful.

[1] https://issues.apache.org/jira/browse/OAK-364
[2] http://code.google.com/p/guava-libraries/wiki/CachesExplained#Statistics

BR,

Jukka Zitting

Re: Kernel stats question.

[Ian Boston]

On 1 March 2013 18:19, Jukka Zitting jukka.zitt...@gmail.com wrote:
 Hi,

 On Thu, Feb 28, 2013 at 11:55 PM, Ian Boston i...@tfd.co.uk wrote:
 Are there plans to expose kernel stats from Oak ?

 Yes, see OAK-364 [1]. Though so far we haven't yet implemented
 anything along these lines.

Nice thanks.
I am more interested in counters than average or last operation time
measurements  as that gives and idea of where hotspots, multi threaded
throughput at full load and anomalies rather than general slowness.
(not saying time measurements are not useful, just hard to interpret
in a highly multi threaded server under load)


 One related idea I was already thinking about implementing is exposing
 cache statistics [2] from the Guava caches we're using at the
 MicroKernel level.

 The Jackrabbit ObservationDispatcher has a queue that contains a
 AtomicInteger indicating the size of the queue. If Observers are slow
 in responding to asynchronous notifications the queue can grow
 rapidly.

 This actually is an area that won't be an issue with Oak, as instead
 of using a centralized queue from where events are pushed to observers
 we're allowing each observer to generate events by comparing
 successive repository revisions at their own pace. Thus one slow
 observer will only block itself.

Does that mean observers get a stream revision tokens, and its up to
them to queue the revision for later processing, even if that means
just keeping the last one worked on and the most recent (ie no queue
at all, just a range of revisions) ?

Will the Kernel blacklist slow observers, as the OSGi event manager does?

Best Regards
Ian

BTW: part of the reason for asking is I am looking into exposing more
stats over in Sling.


 Of course there are other areas where such metrics will still be highly 
 useful.

 [1] https://issues.apache.org/jira/browse/OAK-364
 [2] http://code.google.com/p/guava-libraries/wiki/CachesExplained#Statistics

 BR,

 Jukka Zitting