Well, I was talking about SELECT queries, not even queries dealing with
transactions. So you might have come up with another issue with the
ParallelExecutor. And at the moment, I don't use any JTA tx, so I have
no experience with this. And my use of Slices is for
performance/scale-out reason, so I will not be using distributed
transactions, because that might negate the scalability gains.. (and
most of the commits that I do will only involve one database, not many).
background:
In Slices, there is the Parent DistributedStoreQuery, which delegates to
a number of sub Queries that are actually tied to real databases. The
ParallelExecutor executes the sub Queries using a Thread pool, allowing
all queries to execute concurrently, which is a very good thing as the
number of slices you're running against goes up..
David Jencks wrote:
Not sure if this is relevant here.... for an enterprise environment
using JTA tx, would this multithreaded slice execution involve using the
same JTA tx on multiple threads? This is generally not supported in
existing transaction managers although I think there's a possible
workaround/implementation using separate tx branches for each thread.
Talking with other tm designers there doesn't seem to be enough demand
for this multithreaded tx feature to support implementing it.
thanks
david jencks
On Dec 12, 2008, at 12:34 PM, Fernando Padilla wrote:
Alright Pinaki.
So Slices Parallel Executor seems to be the root cause of the
"corrupted queries" bug (OPENJPA-820). My theory is that internally
the Parallel Executor uses OpenJPA resources in a Multithreaded
manner, but having Multithreaded=false might open it up to some
concurrency issues..
You already know that I tried to test it by turning on
Multithreaded=true, but then ran into that hanging/blocking bug.
So I simply changed the ExecutorService, to be a dumb one that only
executes the tasks using the thread that submitted the task.. (so
always uses the Parent Thread, so Reentrant Locks work properly) (I'll
attach it so you see what I'm talking about ).
And so far, I do not see any of the "corrupted" queries that I was
talking about. But also (it might be my imagination), but it seems
much slower too. :)
So... do you agree? Shall we look at dismantling the ParallelExecutor
from Slices for now ( and take the slower performance )? While we
also look at getting ParallelExecutor working with
Multithreaded=true. And also look at making OpenJPA multithreaded
support be more efficient, because it will have to be on for
ParallelExecutor to work, etc etc..
Pinaki Poddar wrote:
This may be a tough one and thank you for sending the blocking thread
stack.
The QueryContext is shared by the parallel queries. The stack points out
that query context is deadlocked as Executors calling QueryContext
(which
has its own lock) method on different threads. Slice has not
imposed any major change on kernel's behavior (on threading
or otherwise). But this current limitation might. The easy (and
escapist)
way out is to fall back on sequential query execution when
openjpa.Multithreaded is set. Thoughts?
