oh right. I forgot about the strict boundaries between EntityManagers..
darn.
So I suppose we just can't support ParallelExecutor, that's the easy
fix, just not at all the most performant :(
We might be able to tweek things to remove the "query corruption" bug
that I was seeing, but I suppose we can't guarantee no concurrency
issues until Multithreaded=true. And we can't do Multithreded=true
until we make all locks a lot more granular, which will take some time.
So it looks like Slices is moving forward, but if it's going to be 2-3
times slower than normal.. that's a pretty big price to pay for
scalability.. hmmm
Unless we can do some acrobatics with locks around that code, so that
the Parent Thread/DistributedStoreQuery could delegate the lock to the
sub queries.. like can you do a selective release of locks to only a
select few? :)
Pinaki Poddar wrote:
<crazy-brainstorm>
Another option for Slices, is to move up the 'distributed' level, so
that it maintains unique brokers or entity managers per database.. At
that point it could be a JPA Slices implementation, independent of
OpenJPA :)
<>
This option had been considered. The architectural issue with this approach
is the following:
A JPA EntityManager represents a Persistence Context which not only has a
unit of work semantics, it also displays a group like behavior. For example,
an entity instance X managed by EntityManager E1 can not be simultaneously
managed by another EntityManager E2. Also, X can not refer to an Y that is
managed by E2. Hence, a design that spawns E1,E2,...,En connected to Slice
S1, S2, ...Sn are essentially managing separate contexts with water-tight
boundaries.
Fo example, under such a design, one can not issue "select p from Person p
order by p.name" and get all the Persons from all the slices in the same
persistence context.
Fernando Padilla wrote:
yes I know.
I was just walking through the code, to see what I could see, and all
StateManagers use the global Broker lock.. so no attempt at granularity
in many places :(
Also, breaking the lock into Read/Write might give multithreaded use a
little more performance as well..
like you said, this is "battle tested code", which I am assuming is
probably like 4 years old ( from the old kodo days ). So I could
understand the hesitation to not change it, but .... you know ... :)
ps -
<crazy-brainstorm>
Another option for Slices, is to move up the 'distributed' level, so
that it maintains unique brokers or entity managers per database.. At
that point it could be a JPA Slices implementation, independent of
OpenJPA :)
It would be an DistributedEntityManager that would delegate to sub
EntityManagers, and return objects managed by the
DistributedEntityManager, but that behind the scenes would delegate to
objects owned and managed by sub EntityManagers..
So it would be like creating a JPAStoreManager that would store things
into a sub entitymanager :) etc etc
</crazy-brainstorm>
Pinaki Poddar wrote:
Since behind the scenes you are executing various different queries
using
multiple
threads, and they might be touching/using shared resources lower down.
Good point, and most probably you are right.
which is to fix up QueryImpl, not to do aggressive locking (at least in
the isUnique method), and I
think it will at least remove the current blocking issue.
So far, Slice has imposed almost no demand on existing kernel -- which is
battle hardened code. So naturally, I am apprehensive of change in
threading
logic in kernel.QueryImpl. Any change of this nature *must* validate
itself
against a set of multithreaded tests (with or without Slice).
