Re: Multiplexing Document Store

[Ian Boston]

On 3 December 2015 at 04:15, Chetan Mehrotra 
wrote:

> Hi Ian,
>
> On Wed, Dec 2, 2015 at 9:24 PM, Ian Boston  wrote:
> > Hence all MutableTrees get their
> > NodeBuilder from the root DocumentNodeState that is the DocumentNodeStore
> > owning the root node and not the MultiplexingDocumentNodeStore.
>
> Some confusion here. What we have is a MultiplexDocumentStore and
> there is no MultiplexingDocumentNodeStore. All the above objects
> DocumentNodeState, DocumentNodeBuilder refer to DocumentNodeStore (not
> DocumentStore).
>

I meant MultiplexDocumentStore. So many stores and so many layers make it
hard for someone who doesn't work in Oak every day to find there way around.


>
> Use of MultiplexDocumentStore is an implementation detail of
> DocumentNodeStore hence these objects need not be affected and aware
> of multiplexing logic. May be I am missing something here?
>

Looking at the code again, I don't think you are missing anything.
DocumentNodeStore gets its store from the DocumentMK.builder which is a
singleton.

I need to re-run my tests of the 30 October to find out why I was seeing
the behaviour I was seeing, however IIRC you have recently been looking at
/jcr:system etc and the branch I was working may now be obsolete ? If it
is, please let me know as there is no point in duplication or wasted effort.

Best Regards
Ian



>
> Chetan Mehrotra
>

Re: Multiplexing Document Store

[Chetan Mehrotra]

Hi Ian,

On Wed, Dec 2, 2015 at 9:24 PM, Ian Boston  wrote:
> Hence all MutableTrees get their
> NodeBuilder from the root DocumentNodeState that is the DocumentNodeStore
> owning the root node and not the MultiplexingDocumentNodeStore.

Some confusion here. What we have is a MultiplexDocumentStore and
there is no MultiplexingDocumentNodeStore. All the above objects
DocumentNodeState, DocumentNodeBuilder refer to DocumentNodeStore (not
DocumentStore).

Use of MultiplexDocumentStore is an implementation detail of
DocumentNodeStore hence these objects need not be affected and aware
of multiplexing logic. May be I am missing something here?

Chetan Mehrotra

Re: Multiplexing Document Store

[Chetan Mehrotra]

Reviving this old thread as I just started looking into this

> However, it seems that DocumentNodeState (and hence DocumentNodeBuilder)
> objects created by a DocumentNodeStore get a reference to that
> DocumentNodeStore, bypassing any multiplexing document node store.

Multiplexing is being implemented at DocumentStore level and
DocumentNodeStore gets a reference to that. And that internally has a
reference to MultiPlexingDocumentStore so logically looks ok

> but as soon as anything performs a call into DocumentNodeStore that relates
> to a path not within that DocumentNodeStore the multiplexing breaks down.
> Code that reads and writes to /jcr:system/** does this.

Can you provide a testcase on Robert's branch illustrating the
problem? That would help to understand the problem you are facing
better
Chetan Mehrotra


On Fri, Oct 30, 2015 at 4:32 PM, Ian Boston <ianbos...@gmail.com> wrote:
> Hi,
> I am trying to enhance a multiplexing document store written initially by
> Robert Munteanu to support multiplexing of content under /jcr:system/** in
> particular the version store and the permissions store. I have a scheme
> that should theoretically work, encoding the target store in the entry name
> of the map key.
>
> However, it seems that DocumentNodeState (and hence DocumentNodeBuilder)
> objects created by a DocumentNodeStore get a reference to that
> DocumentNodeStore, bypassing any multiplexing document node store. This is
> all ok if all the calls relate to content in the same DocumentNodeStore,
> but as soon as anything performs a call into DocumentNodeStore that relates
> to a path not within that DocumentNodeStore the multiplexing breaks down.
> Code that reads and writes to /jcr:system/** does this.
>
> I have tried hacking the code to ensure that the reference to
> DocumentNodeStore is replaced by the MultiplexingDocumentNodeStore, however
> when I do that, MultiplexingDocumentNodeStore gets calls that have
> insufficient context to route to the correct DocumentNodeStore. I could
> hack some more, but if I do, anything that works is unlikely to be
> acceptable to Oak as a patch.
>
> Any suggestions ?
>
> I haven't even started to look at /oak:index and the code that writes/reads
> from there.
>
> Best Regards
> Ian

Re: Multiplexing Document Store

[Ian Boston]

Hi Robert,
On 5 November 2015 at 22:58, Robert Munteanu <romb...@apache.org> wrote:

> Hi Ian,
>
> On Fri, 2015-10-30 at 15:38 +, Ian Boston wrote:
> > Hi,
> > I am trying to enhance a multiplexing document store written
> > initially by
> > Robert Munteanu to support multiplexing of content under
> > /jcr:system/** in
> > particular the version store and the permissions store. I have a
> > scheme
> > that should theoretically work, encoding the target store in the
> > entry name
> > of the map key.
> >
> > However, it seems that DocumentNodeState (and hence
> > DocumentNodeBuilder)
> > objects created by a DocumentNodeStore get a reference to that
> > DocumentNodeStore, bypassing any multiplexing document node store.
> > This is
> > all ok if all the calls relate to content in the same
> > DocumentNodeStore,
> > but as soon as anything performs a call into DocumentNodeStore that
> > relates
> > to a path not within that DocumentNodeStore the multiplexing breaks
> > down.
> > Code that reads and writes to /jcr:system/** does this.
> >
> > I have tried hacking the code to ensure that the reference to
> > DocumentNodeStore is replaced by the MultiplexingDocumentNodeStore,
> > however
> > when I do that, MultiplexingDocumentNodeStore gets calls that have
> > insufficient context to route to the correct DocumentNodeStore. I
> > could
> > hack some more, but if I do, anything that works is unlikely to be
> > acceptable to Oak as a patch.
>
> Well, I can relate, as the DocumentStore multiplexing implementation
> does have to be a little creative at times to find out the proper
> store.
>
> Could you list (some of) the places where the
> MultiplexingDocumentNodeStore does not have enough information to know
> where to route the operation? Might be helpful as a starting point.
>

If I look at the DocumentStore API, it doesn't look too bad. Every method
that targets a specific store, contains a key or uses an UpdateOp which
contains a primary key. Assuming the store can always be derived from that
primary key, everything should work. What's less clear is the
implementation specific methods that are used within oak-core. I am
reasonably certain that RDBDocumentStore and MongoDocumentStore make
assumptions and call protected methods.

I say, should work, assuming that every reference to a DocumentStore
implementation can be replaced with a reference to the
MultiplexingDocumentStore, so that all calls go via the Multiplexer and non
go direct to the DocumentStore implementation.

To do it properly will be a big patch to oak-core, and I haven't started to
look at oak-lucene. There isn't a great deal of point in preparing that
patch if the Oak committers don't want this.

Best Regards
Ian


>
> Thanks,
>
> Robert
>

Re: Multiplexing Document Store

[Robert Munteanu]

Hi Ian,

On Fri, 2015-10-30 at 15:38 +, Ian Boston wrote:
> Hi,
> I am trying to enhance a multiplexing document store written
> initially by
> Robert Munteanu to support multiplexing of content under
> /jcr:system/** in
> particular the version store and the permissions store. I have a
> scheme
> that should theoretically work, encoding the target store in the
> entry name
> of the map key.
> 
> However, it seems that DocumentNodeState (and hence
> DocumentNodeBuilder)
> objects created by a DocumentNodeStore get a reference to that
> DocumentNodeStore, bypassing any multiplexing document node store.
> This is
> all ok if all the calls relate to content in the same
> DocumentNodeStore,
> but as soon as anything performs a call into DocumentNodeStore that
> relates
> to a path not within that DocumentNodeStore the multiplexing breaks
> down.
> Code that reads and writes to /jcr:system/** does this.
> 
> I have tried hacking the code to ensure that the reference to
> DocumentNodeStore is replaced by the MultiplexingDocumentNodeStore,
> however
> when I do that, MultiplexingDocumentNodeStore gets calls that have
> insufficient context to route to the correct DocumentNodeStore. I
> could
> hack some more, but if I do, anything that works is unlikely to be
> acceptable to Oak as a patch.

Well, I can relate, as the DocumentStore multiplexing implementation
does have to be a little creative at times to find out the proper
store.

Could you list (some of) the places where the
MultiplexingDocumentNodeStore does not have enough information to know
where to route the operation? Might be helpful as a starting point.

Thanks,

Robert

Multiplexing Document Store

[Ian Boston]

Hi,
I am trying to enhance a multiplexing document store written initially by
Robert Munteanu to support multiplexing of content under /jcr:system/** in
particular the version store and the permissions store. I have a scheme
that should theoretically work, encoding the target store in the entry name
of the map key.

However, it seems that DocumentNodeState (and hence DocumentNodeBuilder)
objects created by a DocumentNodeStore get a reference to that
DocumentNodeStore, bypassing any multiplexing document node store. This is
all ok if all the calls relate to content in the same DocumentNodeStore,
but as soon as anything performs a call into DocumentNodeStore that relates
to a path not within that DocumentNodeStore the multiplexing breaks down.
Code that reads and writes to /jcr:system/** does this.

I have tried hacking the code to ensure that the reference to
DocumentNodeStore is replaced by the MultiplexingDocumentNodeStore, however
when I do that, MultiplexingDocumentNodeStore gets calls that have
insufficient context to route to the correct DocumentNodeStore. I could
hack some more, but if I do, anything that works is unlikely to be
acceptable to Oak as a patch.

Any suggestions ?

I haven't even started to look at /oak:index and the code that writes/reads
from there.

Best Regards
Ian

Multiplexing Document Store

[Ian Boston]

Hi,
I am trying to enhance a multiplexing document store written initially by
Robert Munteanu to support multiplexing of content under /jcr:system/** in
particular the version store and the permissions store. I have a scheme
that should theoretically work, encoding the target store in the entry name
of the map key.

However, it seems that DocumentNodeState (and hence DocumentNodeBuilder)
objects created by a DocumentNodeStore get a reference to that
DocumentNodeStore, bypassing any multiplexing document node store. This is
all ok if all the calls relate to content in the same DocumentNodeStore,
but as soon as anything performs a call into DocumentNodeStore that relates
to a path not within that DocumentNodeStore the multiplexing breaks down.
Code that reads and writes to /jcr:system/** does this.

I have tried hacking the code to ensure that the reference to
DocumentNodeStore is replaced by the MultiplexingDocumentNodeStore, however
when I do that, MultiplexingDocumentNodeStore gets calls that have
insufficient context to route to the correct DocumentNodeStore. I could
hack some more, but if I do, anything that works is unlikely to be
acceptable to Oak as a patch.

Any suggestions ?

I haven't even started to look at /oak:index and the code that writes/reads
from there.

Best Regards
Ian

(Apologies if this comes through 2x, afaict, my first email didnt get
through).