Re: Multijob out of memory issues

2017-08-29 Thread Shashank Bhargav 
We are not using GIT plugin, a custom SCM plugin has been written and thats 
used.

On Tuesday, 29 August 2017 08:03:22 UTC+5:30, Mark Waite wrote:
>
> Git plugin has known issues with keeping and reading too much build data.  
> Limiting the amount of history had been my best work around to date. 
>
> Mark Waite
>
> On Mon, Aug 28, 2017, 7:21 PM Shashank Bhargav  > wrote:
>
>> 8192m is the value for XMX we have currently set. But the usage takes 
>> almost that and hangs.
>>
>> Couple of things we have observed until now : 
>>
>> 1. Discard builds was set to number 100 and now we have changed it to 25, 
>> assuming performance issues are due to large number of builds. 
>> 2. We also have a plugin written which generates change log. For some 
>> reason Multijob plugin copies change log details of all sub jobs ( as the 
>> SCM is in sub jobs ) to its build.xml and hence build.xml file becomes 
>> huge. In some cases when the there are lot of changes, this itself crosses 
>> 1 GB. 
>> We are trying to identify and tweak our change log generation somehow, so 
>> that multijob doesn't pick this up.
>> 3. Heap dump didn't provide lot of details, it of course said Multijob 
>> takes lot of memory which is already known :)
>>
>> We haven't had a concrete solution as yet, but trying out on these 
>> possibilities.
>>
>> On Tuesday, 29 August 2017 00:11:52 UTC+5:30, Baptiste Mathus wrote:
>>
>>> +1 with Michael, in addition to this I would probably try some 
>>> 'diff-debugging' by using/removing some parts of that thing and try and 
>>> narrow down more precisely what caused the memory peak.
>>>
>>> BTW, what is your Xmx value?
>>>
>> Le 24 août 2017 01:02, "Michael Pailloncy"  a 
>>> écrit :
>>>
>> The best to start here would be to try instructions on this page 
  
 and analyze the resulting heapdump.


 2017-08-24 8:08 GMT+02:00 Shashank Bhargav :

>>> Hi All,
>
> We are facing serious out of memory errors with Multi job project.
>
> Configuration contains the following : 
>
> 1. Parameterized build
> 2. Multi job Phase
> 3. Inject passwords to the build as environment variables
> 4. Set Jenkins user build variables
> 5. Editable Email notification
>
> All of these is executed ,memory immediately shoots up after last step 
>  is executed and Jenkins hangs.
>
> Has anyone faced such issues before?
>
 -- 
> You received this message because you are subscribed to the Google 
> Groups "Jenkins Users" group.
>
 To unsubscribe from this group and stop receiving emails from it, send 
> an email to jenkinsci-use...@googlegroups.com.


> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/jenkinsci-users/2f7997c8-094c-4ab1-9888-47d354ad3c80%40googlegroups.com
>  
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>
 -- 
 You received this message because you are subscribed to the Google 
 Groups "Jenkins Users" group.

>>> To unsubscribe from this group and stop receiving emails from it, send 
 an email to jenkinsci-use...@googlegroups.com.
>>>
>>>
 To view this discussion on the web visit 
 https://groups.google.com/d/msgid/jenkinsci-users/CAPO77c0PHO4X0XqW%3DSJ_gbXT1XH0zc75cRTynMFndyAOs5_4rg%40mail.gmail.com
  
 
 .
 For more options, visit https://groups.google.com/d/optout.

>>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Jenkins Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to jenkinsci-use...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/jenkinsci-users/59d9e8e6-9d4f-43df-b81b-439d4a65bede%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/ae6121e2-3e76-4ebb-b387-3e8c37def559%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Multijob out of memory issues

2017-08-28 Thread Mark Waite 
Git plugin has known issues with keeping and reading too much build data.
Limiting the amount of history had been my best work around to date.

Mark Waite

On Mon, Aug 28, 2017, 7:21 PM Shashank Bhargav 
wrote:

> 8192m is the value for XMX we have currently set. But the usage takes
> almost that and hangs.
>
> Couple of things we have observed until now :
>
> 1. Discard builds was set to number 100 and now we have changed it to 25,
> assuming performance issues are due to large number of builds.
> 2. We also have a plugin written which generates change log. For some
> reason Multijob plugin copies change log details of all sub jobs ( as the
> SCM is in sub jobs ) to its build.xml and hence build.xml file becomes
> huge. In some cases when the there are lot of changes, this itself crosses
> 1 GB.
> We are trying to identify and tweak our change log generation somehow, so
> that multijob doesn't pick this up.
> 3. Heap dump didn't provide lot of details, it of course said Multijob
> takes lot of memory which is already known :)
>
> We haven't had a concrete solution as yet, but trying out on these
> possibilities.
>
> On Tuesday, 29 August 2017 00:11:52 UTC+5:30, Baptiste Mathus wrote:
>
>> +1 with Michael, in addition to this I would probably try some
>> 'diff-debugging' by using/removing some parts of that thing and try and
>> narrow down more precisely what caused the memory peak.
>>
>> BTW, what is your Xmx value?
>>
> Le 24 août 2017 01:02, "Michael Pailloncy"  a
>> écrit :
>>
> The best to start here would be to try instructions on this page
>>> 
>>> and analyze the resulting heapdump.
>>>
>>>
>>> 2017-08-24 8:08 GMT+02:00 Shashank Bhargav :
>>>
>> Hi All,

 We are facing serious out of memory errors with Multi job project.

 Configuration contains the following :

 1. Parameterized build
 2. Multi job Phase
 3. Inject passwords to the build as environment variables
 4. Set Jenkins user build variables
 5. Editable Email notification

 All of these is executed ,memory immediately shoots up after last step
  is executed and Jenkins hangs.

 Has anyone faced such issues before?

>>> --
 You received this message because you are subscribed to the Google
 Groups "Jenkins Users" group.

>>> To unsubscribe from this group and stop receiving emails from it, send
 an email to jenkinsci-use...@googlegroups.com.
>>>
>>>
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/jenkinsci-users/2f7997c8-094c-4ab1-9888-47d354ad3c80%40googlegroups.com
 
 .
 For more options, visit https://groups.google.com/d/optout.

>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Jenkins Users" group.
>>>
>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to jenkinsci-use...@googlegroups.com.
>>
>>
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/jenkinsci-users/CAPO77c0PHO4X0XqW%3DSJ_gbXT1XH0zc75cRTynMFndyAOs5_4rg%40mail.gmail.com
>>> 
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-users/59d9e8e6-9d4f-43df-b81b-439d4a65bede%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/CAO49JtERgXgpOVo0PwNdikg_VPzvwA5JhYba6Rt6DcR491HOTA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Multijob out of memory issues

2017-08-28 Thread Baptiste Mathus 
+1 with Michael, in addition to this I would probably try some
'diff-debugging' by using/removing some parts of that thing and try and
narrow down more precisely what caused the memory peak.

BTW, what is your Xmx value?

Le 24 août 2017 01:02, "Michael Pailloncy"  a
écrit :

> The best to start here would be to try instructions on this page
> 
> and analyze the resulting heapdump.
>
>
>
> 2017-08-24 8:08 GMT+02:00 Shashank Bhargav :
>
>> Hi All,
>>
>> We are facing serious out of memory errors with Multi job project.
>>
>> Configuration contains the following :
>>
>> 1. Parameterized build
>> 2. Multi job Phase
>> 3. Inject passwords to the build as environment variables
>> 4. Set Jenkins user build variables
>> 5. Editable Email notification
>>
>> All of these is executed ,memory immediately shoots up after last step
>>  is executed and Jenkins hangs.
>>
>> Has anyone faced such issues before?
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Jenkins Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to jenkinsci-users+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit https://groups.google.com/d/ms
>> gid/jenkinsci-users/2f7997c8-094c-4ab1-9888-47d354ad3c80%
>> 40googlegroups.com
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/jenkinsci-users/CAPO77c0PHO4X0XqW%3DSJ_gbXT1XH0zc75cRTynMFndyAOs5_
> 4rg%40mail.gmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/CANWgJS5vqNRjC0dagA4v1afh-L%3DFZCdi-FYnEksZR%3DFpcRdmdg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Multijob out of memory issues

2017-08-24 Thread Michael Pailloncy 
The best to start here would be to try instructions on this page

and analyze the resulting heapdump.



2017-08-24 8:08 GMT+02:00 Shashank Bhargav :

> Hi All,
>
> We are facing serious out of memory errors with Multi job project.
>
> Configuration contains the following :
>
> 1. Parameterized build
> 2. Multi job Phase
> 3. Inject passwords to the build as environment variables
> 4. Set Jenkins user build variables
> 5. Editable Email notification
>
> All of these is executed ,memory immediately shoots up after last step  is
> executed and Jenkins hangs.
>
> Has anyone faced such issues before?
>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/jenkinsci-users/2f7997c8-094c-4ab1-9888-47d354ad3c80%40googlegroups.
> com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/CAPO77c0PHO4X0XqW%3DSJ_gbXT1XH0zc75cRTynMFndyAOs5_4rg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Multijob out of memory issues

2017-08-24 Thread Shashank Bhargav 
Hi All,

We are facing serious out of memory errors with Multi job project.

Configuration contains the following : 

1. Parameterized build
2. Multi job Phase
3. Inject passwords to the build as environment variables
4. Set Jenkins user build variables
5. Editable Email notification

All of these is executed ,memory immediately shoots up after last step  is 
executed and Jenkins hangs.

Has anyone faced such issues before?

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/2f7997c8-094c-4ab1-9888-47d354ad3c80%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.