Re: [vpp-dev] [tsc] FD.io Production Jenkins Restart Required

2019-11-06 Thread Vanessa Valderrama
*Status Update*

*Issue: Gateway Timeout Errors*

  * Summary: Intermittent Gateway Timeout Errors on the
ci-management-jjb-merge jobs are causing stability issues with
Jenkins causing unplanned downtime
  o We have put in a change to take Nginx out of the picture and
allow the build node to talk directly to Jenkins
  o We'll be monitoring closely to ensure this resolves the issue

*Issue: Gerrit cloning timeouts
*

  * Summary: Intermittent job failures caused by a timeout when closing
a Gerrit repo
  o We have opened a Vexxhost ticket for Vexxhost and Ed Kern to
troubleshoot the latency within the network the Nomad cluster is on
  o We are also setting up a local Gerrit mirror which should help
resolve/improve cloning - this should be complete by the end of
the week

*Issue: CSIT: s3-t21-sut1 (10.30.51.44) failure
*

  * Summary: The device s3-t21-sut1 device is having an SSH disk read
only issue and is unreachable over NW
  o We've opened a Vexxhost ticket to check the machine

*Issue: Hung jobs
*

  * Summary: Intermittent jobs stuck/hung requiring the job to be aborted
  o We believe this issue was resolved with the latest Jenkins upgrade

Please let me know if you need additional information. If you experience
any hung jobs or gateway timeout errors, please open a ticket at
support.linuxfoundation.org.

Thank you,
Vanessa

On 11/6/19 9:56 AM, Maciek Konstantynowicz (mkonstan) wrote:
> Hi Vanessa, Thanks for the note. CSIT project keeps experiencing issues
> due to Jenkins outages. Do you have ETA for the fix that will stop these
> outages?
>
> -Maciek
>
>> On 5 Nov 2019, at 23:18, Vanessa Valderrama 
>>  wrote:
>>
>> Jenkins has been restarted, job views restored, jobs are running.
>>
>> We will continue to investigate the Gateway Timeout and JNLP errors
>> we've been seeing the last couple of days.
>>
>> If you experience any issues, please open a ticket at
>> support.linuxfoundation.org
>>
>> Thank you,
>> Vanessa
>>
>>
>> On 11/5/19 4:39 PM, Vanessa Valderrama wrote:
>>> We continue having issues with Gateway Timeouts on the CI merge job which 
>>> has
>>> corrupted the Jenkins job views.
>>>
>>> Jenkins will need to be restarted to resolve this issue.
>>>
>>> Thank you,
>>> Vanessa
>>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>>
>> View/Reply Online (#1152): https://lists.fd.io/g/tsc/message/1152
>> Mute This Topic: https://lists.fd.io/mt/42686762/675185
>> Group Owner: tsc+ow...@lists.fd.io
>> Unsubscribe: https://lists.fd.io/g/tsc/unsub  [mkons...@cisco.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14532): https://lists.fd.io/g/vpp-dev/message/14532
Mute This Topic: https://lists.fd.io/mt/44257110/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-


Re: [vpp-dev] [tsc] FD.io Production Jenkins Restart Required

2019-11-06 Thread Maciek Konstantynowicz (mkonstan) via Lists.Fd.Io
Hi Vanessa, Thanks for the note. CSIT project keeps experiencing issues
due to Jenkins outages. Do you have ETA for the fix that will stop these
outages?

-Maciek

> On 5 Nov 2019, at 23:18, Vanessa Valderrama  
> wrote:
> 
> Jenkins has been restarted, job views restored, jobs are running.
> 
> We will continue to investigate the Gateway Timeout and JNLP errors
> we've been seeing the last couple of days.
> 
> If you experience any issues, please open a ticket at
> support.linuxfoundation.org
> 
> Thank you,
> Vanessa
> 
> 
> On 11/5/19 4:39 PM, Vanessa Valderrama wrote:
>> We continue having issues with Gateway Timeouts on the CI merge job which has
>> corrupted the Jenkins job views.
>> 
>> Jenkins will need to be restarted to resolve this issue.
>> 
>> Thank you,
>> Vanessa
>> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> 
> View/Reply Online (#1152): https://lists.fd.io/g/tsc/message/1152
> Mute This Topic: https://lists.fd.io/mt/42686762/675185
> Group Owner: tsc+ow...@lists.fd.io
> Unsubscribe: https://lists.fd.io/g/tsc/unsub  [mkons...@cisco.com]
> -=-=-=-=-=-=-=-=-=-=-=-

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14529): https://lists.fd.io/g/vpp-dev/message/14529
Mute This Topic: https://lists.fd.io/mt/44257110/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-