On 04/04/19 20:18, Arnold Miller wrote:
> I'm on 2.1.1
Does it mean I should upgrade to one of the fix versions specified
there? 2.0.12
<https://issues.apache.org/jira/issues/?jql=project+%3D+SYNCOPE+AND+fixVersion+%3D+2.0.12>,
2.1.3
<https://issues.apache.org/jira/issues/?jql=project+%3D+SYNCOPE+AND+fixVersion+%3D+2.1.3>,
3.0.0
<https://issues.apache.org/jira/issues/?jql=project+%3D+SYNCOPE+AND+fixVersion+%3D+3.0.0>
My guess is that going from 2.1.1 to 2.1.3 is the less painful option.
Exactly. 2.13 is the closest.
Are there any guidelines somewhere?
Upgrading is strongly suggested at every release.
Please note that upgrade guides are announced along with every release.
In your case:
* 2.1.1 -> 2.1.2:
https://cwiki.apache.org/confluence/display/SYNCOPE/Upgrade+from+2.1.1+to+2.1.2
* 2.1.2 -> 2.1.3:
https://cwiki.apache.org/confluence/display/SYNCOPE/Upgrade+from+2.1.2+to+2.1.3
One more thing I'd like to know: Sounds to me like the read afterwards
is unnecessary. Could it be disabled by configuration?
No, it cannot be disabled by configuration, and it is required for other
aspects of the system to work properly.
You are naturally free to customize your local Syncope deployment by
extending
https://github.com/apache/syncope/blob/2_1_X/core/provisioning-java/src/main/java/org/apache/syncope/core/provisioning/java/propagation/PriorityPropagationTaskExecutor.java
and replacing the method
https://github.com/apache/syncope/blob/2_1_X/core/provisioning-java/src/main/java/org/apache/syncope/core/provisioning/java/propagation/AbstractPropagationTaskExecutor.java#L182
from its superclass.
One you've got your own PropagationTaskExecutor class, do not forget to
change your own provisioning.properties to use that and not the original
PriorityPropagationTaskExecutor.
Additionally, guessing the performance after the upgrade it would be:
3 seconds per 5 scripts changes to 1.8 seconds per 3 scripts (using an
easy rule of 3). Then
18000*1.8 = 32400 seconds = 540 minutes = 9 hours
That's a lot still!
First of all, ensure that the flag "reloadScriptOnExecution" is not set,
in your REST connector configuration.
Secondly, in my experience I have found that the Scripted REST connector
has sometimes some latency hidden, which would probably be worth
additional investigation; if you are willing to contribute:
* https://github.com/Tirasa/ConnIdRESTBundle
* https://connid.atlassian.net/browse/REST
Regards.
*Sent:* Thursday, April 04, 2019 at 12:47 PM
*From:* "Arnold Miller" <[email protected]>
*To:* [email protected]
*Subject:* Re: Slow task
I'm on 2.1.1
Does it mean I should upgrade to one of the fix versions specified
there? 2.0.12
<https://issues.apache.org/jira/issues/?jql=project+%3D+SYNCOPE+AND+fixVersion+%3D+2.0.12>,
2.1.3
<https://issues.apache.org/jira/issues/?jql=project+%3D+SYNCOPE+AND+fixVersion+%3D+2.1.3>,
3.0.0
<https://issues.apache.org/jira/issues/?jql=project+%3D+SYNCOPE+AND+fixVersion+%3D+3.0.0>
My guess is that going from 2.1.1 to 2.1.3 is the less painful option.
Are there any guidelines somewhere?
One more thing I'd like to know: Sounds to me like the read afterwards
is unnecessary. Could it be disabled by configuration?
Best Regards,
Arnold Miller
*Sent:* Wednesday, April 03, 2019 at 5:16 AM
*From:* "Francesco Chicchiriccò" <[email protected]>
*To:* [email protected]
*Subject:* Re: Slow task
Hi,
which version are you running? This sounds like as
https://issues.apache.org/jira/browse/SYNCOPE-1409
Regards.
On 03/04/19 11:58, Arnold Miller wrote:
> Being concerned with the slowness of the process I decided to track
the execution time it takes for every script for a single user and I
found:
>
> 16:24:17.582 INFO Search script finished. Time elapsed: 0.194
seconds Method: run
> 16:24:18.251 INFO Search script finished. Time elapsed: 0.166
seconds Method: run
> 16:24:18.843 INFO Update script finished. Time elapsed: 0.244
seconds Method: run
> 16:24:19.417 INFO Search script finished. Time elapsed: 0.161
seconds Method: run
> 16:24:19.996 INFO Search script finished. Time elapsed: 0.172
seconds Method: run
>
> The first most obvious question is: Why does Syncope call the search
script 4 times? Additionally, I see that between the 2nd and 3rd times
it calls the update script. The sum of the time elapsed for that
sample is 0.937, however the time at the beginning of the line shows
that the whole process can take about 3 seconds per user.
>
> Doing math, if I have 18k users then it will take:
> 18000*3 = 54000 seconds = 900 minutes = 15 hours
> (This is what is actually taking!!)
>
> What is it that Syncope does behind scenes?
> How can I optimize this process?
>
>
> Best Regards,
>
> Arnold Miller
>
>
>> Sent: Wednesday, March 27, 2019 at 6:33 PM
>> From: "Francesco Chicchiriccò" <[email protected]>
>> To: [email protected]
>> Subject: Re: Slow task
>>
>> On 27/03/19 17:21, Arnold Miller wrote:
>>> It is a standard Push task: Resource -> Push task -> run.
>> A PushTask execution will simply generate as may PropagationTasks as
>> needed and then execute them; for a REST resource, it means calling the
>> create or update Groovy script.
>>
>> Hence: your guess is correct, if you have 18000 captured by the
PushTask.
>>
>> Regards.
>>
>>> *Sent:* Wednesday, March 27, 2019 at 11:06 AM
>>> *From:* "Francesco Chicchiriccò" <[email protected]>
>>> *To:* [email protected]
>>> *Subject:* Re: Slow task
>>> Hi,
>>> what kind of task (the one running for 6+ hours) do you have? It's a
>>> standard PushTask, or a SchedTask you made on purpose?
>>> Regards.
>>> On 27/03/19 17:03, Arnold Miller wrote:
>>>
>>> No.
>>> My guess is that Syncope makes 18000 calls using either the create
>>> or the update Groovy script.
>>> Best Regards,
>>>
>>> Arnold Miller
>>> *Sent:* Wednesday, March 27, 2019 at 7:58 AM
>>> *From:* "Filipe Silva" <[email protected]>
>>> *To:* [email protected]
>>> *Subject:* Re: Slow task
>>>
>>> Hi
>>>
>>> Are you making only one call to get all the info?
>>>
>>> I use groovy and made pagination in the rest call.
>>>
>>> Regards
>>>
>>> Filipe Silva
>>>
>>> On 26/03/19 21:40, Arnold Miller wrote:
>>>
>>> Hi everyone,
>>> I have a task running for 6 hours trying to update the
>>> information of a total of 18000 users. It has processed only
>>> half of them. The task consists of calling a rest web service
>>> with about 10 parameters. If I run the web service call only
>>> it could take about 300 milliseconds.
>>> What am I doing wrong?
>>> Best Regards,
>>> Arnold Miller
--
Francesco Chicchiriccò
Tirasa - Open Source Excellence
http://www.tirasa.net/
Member at The Apache Software Foundation
Syncope, Cocoon, Olingo, CXF, OpenJPA, PonyMail
http://home.apache.org/~ilgrosso/
