Good catch !
Just checked my own code and I use a session object per-thread. Generally I
assume that Java objects are NOT thread safe unless stated otherwise
( And even then I have my suspicions ).
IMHO Java makes it *too easy* to create threads which makes one feel a false
sense of comfort that writing multi threaded classes is by implication easy.
---> my code fragment
private class PutContent implements Runnable
{
List<SumContent> mContents;
Session mSession;
...
----------------------------------------
David A. Lee
Senior Principal Software Engineer
Epocrates, Inc.
[email protected]
812-482-5224
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of Mike Sokolov
Sent: Tuesday, October 25, 2011 3:46 PM
To: General MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] parallel document insertion
Yeah that was it. I guess the comments about state were a giveaway,
although I agree it would be nice to have an explicit warning in the
Session docs. I think I always assumed these weren't thread-safe,
wrapped them in my own thread-unsafe object, which we pool and hand out
per-request (thread), and then just recently started sharing that across
our own internally-created threads, sigh. Exceptions all fixed now -
thanks again.
-Mike
On 10/25/2011 03:12 PM, Michael Blakeley wrote:
> Yes, that's how RecordLoader et. al. do it. Sessions are lightweight, and I
> am fairly sure they are not thread-safe.
>
> The javadoc isn't as clear about this as it could be. At
> http://developer.marklogic.com/pubs/4.2/javadoc/com/marklogic/xcc/Session.html
> we have "A Session object represents a conversation with a contentbase
> (database) on a MarkLogic Server instance (ContentSource) and holds state
> information related to that conversation." And
> http://developer.marklogic.com/pubs/4.2/javadoc/overview-summary.html states
> that "A Session also holds dynamic state, but it is a lightweight object. It
> is OK to create and release Session objects as needed. Do not expend effort
> to pool and reuse them, they are not expensive to create." It would be a
> little clearer if these javadoc pages and/or
> http://developer.marklogic.com/pubs/4.2/books/xcc.pdf explicitly mentioned
> thread safety.
>
> -- Mike
>
> On 25 Oct 2011, at 11:55 , Mike Sokolov wrote:
>
>
>> Oh - here's a thought - I expect the XCC Session object is not
>> thread-safe, but I'm sharing it. Sound plausible? Is there any reason
>> not to create a Session per Request? They seem lightweight enough...
>>
>> -Mike
>>
>> On 10/25/2011 02:52 PM, Mike Sokolov wrote:
>>
>>> Thanks everyone - I'm using XCC 4.1.3 with a server that is at version
>>> 4.1.6, so possibly a problem there?
>>>
>>> I'm using ThreadPoolExecutor to manage the threads. I would have a used
>>> more abstract ExecutorService, but I thought I needed some fine-grained
>>> control for some reason I can no longer remember.
>>>
>>> I've run into the issue when using 4 threads on my 2-core 1-cpu desktop.
>>>
>>> The server wasn't heavily loaded - this is our dev environment.
>>>
>>> Also - and this is weird - I get the errors consistently in my tests;
>>> the behavior tends to be repeatable; it doesn't have the usual
>>> intermittent inconsistency I would expect from race conditions.
>>>
>>> I'll try updating the XCC version and report back.
>>>
>>> -Mike
>>>
>>> On 10/25/2011 12:20 PM, Michael Blakeley wrote:
>>>
>>>
>>>> I don't recall seeing that error with XQSync, Corb, or RecordLoader - all
>>>> of which I use with large numbers of threads via java.util.concurrent
>>>> pools. Usually I configure 2 client threads per server CPU thread.
>>>>
>>>> Is it possible that you are using an older XCC jar, and hitting a bug
>>>> that's already been fixed?
>>>>
>>>> Are you managing your threads directly? I'd recommend against that: it's
>>>> all too easy to make mistakes, and java.util.concurrent provides a great
>>>> abstraction layer.
>>>>
>>>> -- Mike
>>>>
>>>> On 25 Oct 2011, at 07:40 , Mike Sokolov wrote:
>>>>
>>>>
>>>>
>>>>
>>>>> I've been experimenting with a multithreaded document loader; I get a
>>>>> substantial speedup, which is great. I'm inserting documents using the
>>>>> java XCC API. Every now and then though, I get a
>>>>> RequestPermissionException with the message
>>>>>
>>>>> "Authorization failed for user 'xxxx'"
>>>>>
>>>>> If I retry the insertion immediately, it invariably succeeds, although
>>>>> the exception has isRetryAdvised() == false.
>>>>>
>>>>> I'm sure the authentication is the same for all these requests. I get
>>>>> this error even when none of the documents have the same uri; so there
>>>>> shouldn't be any conflicts or race conditions related to which thread
>>>>> wins, at least not anything obvious like trying to write the same
>>>>> document twice. At least I'm pretty sure that's not it?
>>>>>
>>>>> I'll send something to support I guess, but before doing the work to
>>>>> wrap this up in a neat reproducible package I thought I'd see if anyone
>>>>> had run into this before - anyone?
>>>>>
>>>>> --
>>>>> Michael Sokolov
>>>>> Engineering Director
>>>>> www.ifactory.com
>>>>>
>>>>> _______________________________________________
>>>>> General mailing list
>>>>> [email protected]
>>>>> http://developer.marklogic.com/mailman/listinfo/general
>>>>>
>>>>>
>>>>>
>>>>>
>>>> _______________________________________________
>>>> General mailing list
>>>> [email protected]
>>>> http://developer.marklogic.com/mailman/listinfo/general
>>>>
>>>>
>>>>
>>> _______________________________________________
>>> General mailing list
>>> [email protected]
>>> http://developer.marklogic.com/mailman/listinfo/general
>>>
>>>
>> _______________________________________________
>> General mailing list
>> [email protected]
>> http://developer.marklogic.com/mailman/listinfo/general
>>
>>
> _______________________________________________
> General mailing list
> [email protected]
> http://developer.marklogic.com/mailman/listinfo/general
>
_______________________________________________
General mailing list
[email protected]
http://developer.marklogic.com/mailman/listinfo/general
_______________________________________________
General mailing list
[email protected]
http://developer.marklogic.com/mailman/listinfo/general
