subject:"Deadlocks \- Performance Issue"

Re: Deadlocks - Performance Issue

2016-03-21 Thread Colm O hEigeartaigh

Thanks Guido! We will experiment with increasing "maxActive". Here is an
example of where the deadlock occurs:

Caused by: org.apache.openjpa.lib.jdbc.ReportingSQLException: Transaction
(Process ID 108) was deadlocked on lock resources with another process and
has been chosen as the deadlock victim. Rerun the transact

ion. {prepstmnt 195680672 UPDATE dbo.Membership SET lastChangeDate = ?,
SYNCOPEUSER_ID = ? WHERE id = ?} [code=1205, state=40001]

at
org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(LoggingConnectionDecorator.java:219)
~[openjpa-lib-2.3.0.jar:2.3.0]

at
org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(LoggingConnectionDecorator.java:195)
~[openjpa-lib-2.3.0.jar:2.3.0]

at
org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.access$1000(LoggingConnectionDecorator.java:59)
~[openjpa-lib-2.3.0.jar:2.3.0]
--

On Mon, Mar 21, 2016 at 12:29 PM, Guido Wimmel  wrote:

> Hi,
>
> no idea if this could be helpful, but I could reproduce a deadlock some
> time ago in Syncope by an artificial load test with many parallel threads.
> The deadlock occurred when there were no more connections available in the
> connection pool.
> You could check if the connection pool (property jpa.pool.maxActive) is
> sufficiently large.
>
> The mentioned log that "claims some deadlocks" could also be helpful to
> see where the deadlock exactly happens.
>
> Cheers,
>   Guido
>
> Am 18.03.2016 um 17:56 schrieb Colm O hEigeartaigh:
>
> Yes, the users are stored internally after creation. I guess we will have
> to set up a test-case and try to reproduce the issue.
>
> Thanks for your help Giacomo!
>
> Colm.
>
> On Fri, Mar 18, 2016 at 11:08 AM, Giacomo Lamonaco <
> giacomo.lamon...@tirasa.net> wrote:
>
>> Hi Colm,
>>
>> I tried to inspect the latest fixes but I didn't find something related
>> to performance or deadlock.
>>
>> I have some more questions:
>> 1) At the end of the creation process, users have been stored internally?
>> 2) Could you provide the average time required for an user creation?  And
>> for an user search?
>>
>> It could be better to test 2) both with and without the cluster.
>>
>> Anyway, my suggestion is to upgrade Syncope to 1.2.8-SNAPSHOT version and
>> start again with the import.
>>
>> Best Regards,
>>
>> --
>> Giacomo Lamonaco
>>
>> Tirasa - Open Source Excellencehttp://www.tirasa.net/
>>
>> Syncope committerhttp://people.apache.org/~giacomolm
>>
>> Il giorno ven, 18/03/2016 alle 09.54 +, Colm O hEigeartaigh ha
>> scritto:
>>
>> i Giacomo,
>>
>> My understanding is that there are no external resources involved here.
>> The MSSQL cluster is used for the Syncope internal storage. So the users
>> are created directly in Syncope itself via the REST API. Does this kind of
>> deadlock issue ring any bells for any issue fixed subsequent to 1.2.5? If
>> not we will try to reproduce the issue locally to try to narrow the problem
>> down.
>>
>> Thanks,
>>
>> Colm.
>>
>> On Thu, Mar 17, 2016 at 1:27 PM, Giacomo Lamonaco <
>> giacomo.lamon...@tirasa.net> wrote:
>>
>> Il giorno mer, 16/03/2016 alle 15.27 +, Aniket Band ha scritto:
>> > Hello All,
>> >
>> > One of our customers uses Syncope (1.2.5) and got some performance
>> > issues. They want to upload 100.000 users via REST interface. They
>> > created 3-5 routes in parallel to call the rest interface.
>> > Unfortunately it requires 2 day to insert 100.000 users. As far as I
>> > know it should not take 2 days. The Log claims some deadlocks in the
>> > database.
>> >
>> > Any idea about it? In the mailing list and in JIRA I could not find
>> > any similar issue.
>> >
>> > Database is a MSSQL in a cluster. Thanks in advance for your inputs.
>> >
>> Hi Aniket,
>>
>> Do you have some external resources connected to Syncope? (Connected
>> means that every time you create an user in Syncope, that user is
>> automatically propagated to the external resource).
>>
>> Do you have the possibility to create users just on one MSSQL instance
>> (switching off the cluster)?
>>
>> It's not always a good idea to perform large 'import' using the REST
>> interface. You could try to perform this kind of import at a lower level,
>> maybe creating a dump containing those users.
>>
>> Best Regards,
>>
>> --
>> Giacomo Lamonaco
>>
>> Tirasa - Open Source Excellence
>> http://www.tirasa.net/
>>
>> Syncope committer
>> http://people.apache.org/~giacomolm
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> --
> Colm O hEigeartaigh
>
> Talend Community Coder
> http://coders.talend.com
>
>
>


-- 
Colm O hEigeartaigh

Talend Community Coder
http://coders.talend.com

Re: Deadlocks - Performance Issue

2016-03-21 Thread Guido Wimmel

Hi,

no idea if this could be helpful, but I could reproduce a deadlock some 
time ago in Syncope by an artificial load test with many parallel 
threads. The deadlock occurred when there were no more connections 
available in the connection pool.
You could check if the connection pool (property jpa.pool.maxActive) is 
sufficiently large.

The mentioned log that "claims some deadlocks" could also be helpful to 
see where the deadlock exactly happens.

Cheers,
  Guido

Am 18.03.2016 um 17:56 schrieb Colm O hEigeartaigh:
Yes, the users are stored internally after creation. I guess we will 
have to set up a test-case and try to reproduce the issue.

Thanks for your help Giacomo!

Colm.

On Fri, Mar 18, 2016 at 11:08 AM, Giacomo Lamonaco 
mailto:giacomo.lamon...@tirasa.net>> wrote:

Hi Colm,

I tried to inspect the latest fixes but I didn't find something
related to performance or deadlock.

I have some more questions:
1) At the end of the creation process, users have been stored
internally?
2) Could you provide the average time required for an user
creation?  And for an user search?

It could be better to test 2) both with and without the cluster.

Anyway, my suggestion is to upgrade Syncope to 1.2.8-SNAPSHOT
version and start again with the import.

Best Regards,

-- 
Giacomo Lamonaco

Tirasa - Open Source Excellence
http://www.tirasa.net/

Syncope committer
http://people.apache.org/~giacomolm

Il giorno ven, 18/03/2016 alle 09.54 +, Colm O hEigeartaigh ha
scritto:

i Giacomo,

My understanding is that there are no external resources involved
here. The MSSQL cluster is used for the Syncope internal storage.
So the users are created directly in Syncope itself via the REST
API. Does this kind of deadlock issue ring any bells for any
issue fixed subsequent to 1.2.5? If not we will try to reproduce
the issue locally to try to narrow the problem down.

Thanks,

Colm.

On Thu, Mar 17, 2016 at 1:27 PM, Giacomo Lamonaco
mailto:giacomo.lamon...@tirasa.net>> wrote:

Il giorno mer, 16/03/2016 alle 15.27 +, Aniket Band ha scritto:
> Hello All,
>
> One of our customers uses Syncope (1.2.5) and got some performance
> issues. They want to upload 100.000 users via REST interface. They
> created 3-5 routes in parallel to call the rest interface.
> Unfortunately it requires 2 day to insert 100.000 users. As
far as I
> know it should not take 2 days. The Log claims some deadlocks
in the
> database.
>
> Any idea about it? In the mailing list and in JIRA I could not
find
> any similar issue.
>
> Database is a MSSQL in a cluster. Thanks in advance for your
inputs.
>
Hi Aniket,

Do you have some external resources connected to Syncope?
(Connected means that every time you create an user in Syncope,
that user is automatically propagated to the external resource).

Do you have the possibility to create users just on one MSSQL
instance (switching off the cluster)?

It's not always a good idea to perform large 'import' using the
REST interface. You could try to perform this kind of import at
a lower level, maybe creating a dump containing those users.

Best Regards,

--
Giacomo Lamonaco

Tirasa - Open Source Excellence
http://www.tirasa.net/

Syncope committer
http://people.apache.org/~giacomolm

--
Colm O hEigeartaigh

Talend Community Coder
http://coders.talend.com

Re: Deadlocks - Performance Issue

2016-03-19 Thread Giacomo Lamonaco

Il giorno mer, 16/03/2016 alle 15.27 +, Aniket Band ha scritto:
> Hello All,
>  
> One of our customers uses Syncope (1.2.5) and got some performance
> issues. They want to upload 100.000 users via REST interface. They
> created 3-5 routes in parallel to call the rest interface.
> Unfortunately it requires 2 day to insert 100.000 users. As far as I
> know it should not take 2 days. The Log claims some deadlocks in the
> database.
>  
> Any idea about it? In the mailing list and in JIRA I could not find
> any similar issue.
>  
> Database is a MSSQL in a cluster. Thanks in advance for your inputs.
>  
Hi Aniket,

Do you have some external resources connected to Syncope? (Connected means that 
every time you create an user in Syncope, that user is automatically propagated 
to the external resource). 

Do you have the possibility to create users just on one MSSQL instance 
(switching off the cluster)?

It's not always a good idea to perform large 'import' using the REST interface. 
You could try to perform this kind of import at a lower level, maybe creating a 
dump containing those users.

Best Regards,

-- 
Giacomo Lamonaco

Tirasa - Open Source Excellence
http://www.tirasa.net/

Syncope committer
http://people.apache.org/~giacomolm

Re: Deadlocks - Performance Issue

2016-03-19 Thread Giacomo Lamonaco

Hi Colm,
I tried to inspect the latest fixes but I didn't find something related
to performance or deadlock.
I have some more questions:
1) At the end of the creation process, users have been stored
internally?
2) Could you provide the average time required for an user creation?
 And for an user search?
It could be better to test 2) both with and without the cluster.
Anyway, my suggestion is to upgrade Syncope to 1.2.8-SNAPSHOT version
and start again with the import.
Best Regards,
-- 
Giacomo Lamonaco

Tirasa - Open Source Excellence
http://www.tirasa.net/

Syncope committer
http://people.apache.org/~giacomolm

Il giorno ven, 18/03/2016 alle 09.54 +, Colm O hEigeartaigh ha
scritto:
> i Giacomo,
> 
> My understanding is that there are no external resources involved
> here. The MSSQL cluster is used for the Syncope internal storage. So
> the users are created directly in Syncope itself via the REST API.
> Does this kind of deadlock issue ring any bells for any issue fixed
> subsequent to 1.2.5? If not we will try to reproduce the issue
> locally to try to narrow the problem down.
> 
> Thanks,
> 
> Colm.
> 
> On Thu, Mar 17, 2016 at 1:27 PM, Giacomo Lamonaco 
> irasa.net> wrote:
> > Il giorno mer, 16/03/2016 alle 15.27 +, Aniket Band ha scritto:
> > > Hello All,
> > >  
> > > One of our customers uses Syncope (1.2.5) and got some
> > performance
> > > issues. They want to upload 100.000 users via REST interface.
> > They
> > > created 3-5 routes in parallel to call the rest interface.
> > > Unfortunately it requires 2 day to insert 100.000 users. As far
> > as I
> > > know it should not take 2 days. The Log claims some deadlocks in
> > the
> > > database.
> > >  
> > > Any idea about it? In the mailing list and in JIRA I could not
> > find
> > > any similar issue.
> > >  
> > > Database is a MSSQL in a cluster. Thanks in advance for your
> > inputs.
> > >  
> > Hi Aniket,
> > 
> > Do you have some external resources connected to Syncope?
> > (Connected means that every time you create an user in Syncope,
> > that user is automatically propagated to the external resource). 
> > 
> > Do you have the possibility to create users just on one MSSQL
> > instance (switching off the cluster)?
> > 
> > It's not always a good idea to perform large 'import' using the
> > REST interface. You could try to perform this kind of import at a
> > lower level, maybe creating a dump containing those users.
> > 
> > Best Regards,
> > 
> > --
> > Giacomo Lamonaco
> > 
> > Tirasa - Open Source Excellence
> > http://www.tirasa.net/
> > 
> > Syncope committer
> > http://people.apache.org/~giacomolm
> > 
> > 
> > 
> 
>

Re: Deadlocks - Performance Issue

2016-03-19 Thread Colm O hEigeartaigh

Hi Giacomo,

My understanding is that there are no external resources involved here. The
MSSQL cluster is used for the Syncope internal storage. So the users are
created directly in Syncope itself via the REST API. Does this kind of
deadlock issue ring any bells for any issue fixed subsequent to 1.2.5? If
not we will try to reproduce the issue locally to try to narrow the problem
down.

Thanks,

Colm.

On Thu, Mar 17, 2016 at 1:27 PM, Giacomo Lamonaco <
giacomo.lamon...@tirasa.net> wrote:

> Il giorno mer, 16/03/2016 alle 15.27 +, Aniket Band ha scritto:
> > Hello All,
> >
> > One of our customers uses Syncope (1.2.5) and got some performance
> > issues. They want to upload 100.000 users via REST interface. They
> > created 3-5 routes in parallel to call the rest interface.
> > Unfortunately it requires 2 day to insert 100.000 users. As far as I
> > know it should not take 2 days. The Log claims some deadlocks in the
> > database.
> >
> > Any idea about it? In the mailing list and in JIRA I could not find
> > any similar issue.
> >
> > Database is a MSSQL in a cluster. Thanks in advance for your inputs.
> >
> Hi Aniket,
>
> Do you have some external resources connected to Syncope? (Connected means
> that every time you create an user in Syncope, that user is automatically
> propagated to the external resource).
>
> Do you have the possibility to create users just on one MSSQL instance
> (switching off the cluster)?
>
> It's not always a good idea to perform large 'import' using the REST
> interface. You could try to perform this kind of import at a lower level,
> maybe creating a dump containing those users.
>
> Best Regards,
>
> --
> Giacomo Lamonaco
>
> Tirasa - Open Source Excellence
> http://www.tirasa.net/
>
> Syncope committer
> http://people.apache.org/~giacomolm
>
>
>


-- 
Colm O hEigeartaigh

Talend Community Coder
http://coders.talend.com

Re: Deadlocks - Performance Issue

2016-03-18 Thread Colm O hEigeartaigh

Yes, the users are stored internally after creation. I guess we will have
to set up a test-case and try to reproduce the issue.

Thanks for your help Giacomo!

Colm.

On Fri, Mar 18, 2016 at 11:08 AM, Giacomo Lamonaco <
giacomo.lamon...@tirasa.net> wrote:

> Hi Colm,
>
> I tried to inspect the latest fixes but I didn't find something related to
> performance or deadlock.
>
> I have some more questions:
> 1) At the end of the creation process, users have been stored internally?
> 2) Could you provide the average time required for an user creation?  And
> for an user search?
>
> It could be better to test 2) both with and without the cluster.
>
> Anyway, my suggestion is to upgrade Syncope to 1.2.8-SNAPSHOT version and
> start again with the import.
>
> Best Regards,
>
> --
> Giacomo Lamonaco
>
> Tirasa - Open Source Excellencehttp://www.tirasa.net/
>
> Syncope committerhttp://people.apache.org/~giacomolm
>
>
> Il giorno ven, 18/03/2016 alle 09.54 +, Colm O hEigeartaigh ha scritto:
>
> i Giacomo,
>
> My understanding is that there are no external resources involved here.
> The MSSQL cluster is used for the Syncope internal storage. So the users
> are created directly in Syncope itself via the REST API. Does this kind of
> deadlock issue ring any bells for any issue fixed subsequent to 1.2.5? If
> not we will try to reproduce the issue locally to try to narrow the problem
> down.
>
> Thanks,
>
> Colm.
>
> On Thu, Mar 17, 2016 at 1:27 PM, Giacomo Lamonaco <
> giacomo.lamon...@tirasa.net> wrote:
>
> Il giorno mer, 16/03/2016 alle 15.27 +, Aniket Band ha scritto:
> > Hello All,
> >
> > One of our customers uses Syncope (1.2.5) and got some performance
> > issues. They want to upload 100.000 users via REST interface. They
> > created 3-5 routes in parallel to call the rest interface.
> > Unfortunately it requires 2 day to insert 100.000 users. As far as I
> > know it should not take 2 days. The Log claims some deadlocks in the
> > database.
> >
> > Any idea about it? In the mailing list and in JIRA I could not find
> > any similar issue.
> >
> > Database is a MSSQL in a cluster. Thanks in advance for your inputs.
> >
> Hi Aniket,
>
> Do you have some external resources connected to Syncope? (Connected means
> that every time you create an user in Syncope, that user is automatically
> propagated to the external resource).
>
> Do you have the possibility to create users just on one MSSQL instance
> (switching off the cluster)?
>
> It's not always a good idea to perform large 'import' using the REST
> interface. You could try to perform this kind of import at a lower level,
> maybe creating a dump containing those users.
>
> Best Regards,
>
> --
> Giacomo Lamonaco
>
> Tirasa - Open Source Excellence
> http://www.tirasa.net/
>
> Syncope committer
> http://people.apache.org/~giacomolm
>
>
>
>
>
>
>
>


-- 
Colm O hEigeartaigh

Talend Community Coder
http://coders.talend.com

Deadlocks - Performance Issue

2016-03-18 Thread Aniket Band

Hello All,



One of our customers uses Syncope (1.2.5) and got some performance issues. They 
want to upload 100.000 users via REST interface. They created 3-5 routes in 
parallel to call the rest interface. Unfortunately it requires 2 day to insert 
100.000 users. As far as I know it should not take 2 days. The Log claims some 
deadlocks in the database.



Any idea about it? In the mailing list and in JIRA I could not find any similar 
issue.



Database is a MSSQL in a cluster. Thanks in advance for your inputs.



Thanks,

Aniket

Re: Deadlocks - Performance Issue

Re: Deadlocks - Performance Issue

Re: Deadlocks - Performance Issue

Re: Deadlocks - Performance Issue

Re: Deadlocks - Performance Issue

Re: Deadlocks - Performance Issue

Deadlocks - Performance Issue

7 matches

Site Navigation

Mail list logo

Footer information