Team,
Just an fyi I did slowdowns late last week – the verify parts of the
build where we do all the compliance testing was taking 3-5 min for each search
Also, I did see one timeout just now – 2nd retry was ok
22:27:19 [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @
logging-slf4j ---
22:27:21 Oct 02, 2018 10:27:21 PM
org.apache.maven.wagon.providers.http.httpclient.impl.execchain.RetryExec
execute
22:27:21 INFO: I/O exception
(org.apache.maven.wagon.providers.http.httpclient.NoHttpResponseException)
caught when processing request to {s}->https://nexus.onap.org:443: The target
server failed to respond
22:27:21 Oct 02, 2018 10:27:21 PM
org.apache.maven.wagon.providers.http.httpclient.impl.execchain.RetryExec
execute
22:27:21 INFO: Retrying request to {s}->https://nexus.onap.org:443
https://jenkins.onap.org/job/logging-analytics-master-verify-java/130/console
/michael
-----Original Message-----
From: [email protected] <[email protected]> On Behalf Of
Michael O'Brien
Sent: Tuesday, October 2, 2018 4:10 PM
To: [email protected]; [email protected]; TIMONEY, DAN <[email protected]>;
[email protected]; DRAGOSH, PAM <[email protected]>;
[email protected]
Cc: LEFEVRE, CATHERINE <[email protected]>; Prudence Au
<[email protected]>
Subject: Re: [onap-discuss] [ONAP Helpdesk #61676] gerrit down
Team,
Gerrit clones were ok for me just now at 1600EDT (only issue was 12 hours
ago - once)
I have a CD system (runs on AWS us-east-1/2) that runs every hour pulling
OOM master and deploying (automated hourly logging/pomba healthcheck) - there
are logs on Jenkins - I did have an issue with a gerrit clone once today - but
have been ok for the last 12 builds.
http://kibana.onap.info:5601/app/kibana#/dashboard/AWAtvpS63NTXK5mX2kuS
Issue at 0700 UTC (0100 EDT)
http://jenkins.onap.info/job/oom-cd-master/3527/console
07:04:37 Cloning into 'oom'...
07:07:32 sh: 0: getcwd() failed: No such file or directory
07:07:32 make[1]: Leaving directory '/home/ubuntu/oom/kubernetes'
07:07:32 make: getcwd: No such file or directory
07:07:32 make: *** No rule to make target 'onap'. Stop.
07:07:32 Error: open onap/resources/environments/disable-allcharts.yaml: no
such file or directory
That was the only time a clone did not work from gerrit in the last 24 hours I
have fielded questions on nexus3 download times today though.
I was also going to bring up another issue Myself and Prudence found -
git.onap.org does not reflect merged changes for a couple hours - we changed
the version number on a repo and it only updated later in the day on git
https://git.onap.org/logging-analytics/pomba/pomba-audit-common/tree/version.properties
showed 1.3.0 for a couple hours after a 1.3.1 merge don't know if the lag is
related.
/michael
-----Original Message-----
From: [email protected] <[email protected]> On Behalf Of
Brian
Sent: Tuesday, October 2, 2018 3:07 PM
To: TIMONEY, DAN <[email protected]>; [email protected];
[email protected]; DRAGOSH, PAM <[email protected]>;
[email protected]
Cc: LEFEVRE, CATHERINE <[email protected]>
Subject: Re: [onap-discuss] [ONAP Helpdesk #61676] gerrit down
should we open a separate ticket on nexus.onap.org connection reset on push ?
Brian
-----Original Message-----
From: TIMONEY, DAN
Sent: Tuesday, October 02, 2018 3:03 PM
To: FREEMAN, BRIAN D <[email protected]>; [email protected];
[email protected]; DRAGOSH, PAM <[email protected]>;
[email protected]; [email protected]
Cc: LEFEVRE, CATHERINE <[email protected]>
Subject: Re: [onap-discuss] [ONAP Helpdesk #61676] gerrit down
All,
FYI - I hoped to work around this issue by commenting out a module that isn't
critical from our build, hoping to reduce load. Didn't help - just moved the
problem to someplace else.
Here's the error I'm getting:
18:43:05 [INFO] Installing /w/workspace/sdnc-oam-master-merge-java/pom.xml to
/w/workspace/sdnc-oam-master-merge-java/admportal/target/nexus-staging/deferred/org/onap/sdnc/oam/sdnc-oam/1.4.0-SNAPSHOT/sdnc-oam-1.4.0-SNAPSHOT.pom
18:43:05 [INFO] Deploying remotely...
18:43:05 [INFO] Bulk deploying locally gathered artifacts from directory:
18:43:05 [INFO] * Bulk deploying locally gathered snapshot artifacts
18:45:14 Oct 02, 2018 6:45:14 PM
org.apache.maven.wagon.providers.http.httpclient.impl.execchain.RetryExec
execute
18:45:14 INFO: I/O exception (java.net.SocketException) caught when processing
request to {}->http://nexus.onap.org:80: Connection reset ....
18:46:47 [ERROR] Failed to execute goal
org.sonatype.plugins:nexus-staging-maven-plugin:1.6.7:deploy
(injected-nexus-deploy) on project sdnc-oam: Failed to deploy artifacts: Could
not transfer artifact
org.onap.sdnc.oam:platform-logic-installer:zip:1.4.0-20181002.184326-145
from/to ecomp-snapshots (http://nexus.onap.org/content/repositories/snapshots):
Connection reset -> [Help 1]
So while the error is happening on Jenkins, it seems that Nexus is really the
culprit.
Dan
--
Dan Timoney
SDN-CP Development
ONAP Project Technical Lead : CCSDK and SDNC
Please go to D2 ECOMP Release Planning Wiki
<https://wiki.web.att.com/display/DERP/D2+ECOMP+Release+Planning+Home> for D2
ECOMP Project In-take, 2016 Release Planning, Change Management, and find key
Release Planning Contact Information.
On 10/2/18, 2:14 PM, "FREEMAN, BRIAN D" <[email protected]> wrote:
Gildas,
Help - this is blocking instantiation testing because we cant distribute
the models without a new sdnc docker.
Brian
-----Original Message-----
From: TIMONEY, DAN
Sent: Tuesday, October 02, 2018 2:11 PM
To: [email protected]; [email protected]; DRAGOSH, PAM
<[email protected]>; [email protected];
[email protected]
Cc: FREEMAN, BRIAN D <[email protected]>; LEFEVRE, CATHERINE
<[email protected]>
Subject: Re: [onap-discuss] [ONAP Helpdesk #61676] gerrit down
All,
Whatever can be done to alleviate this would be helpful.
To echo Pam's point, last Friday, all my merges were taking over 90 minutes
- or rather, they were running for 90 minutes then getting killed. I've been
working now for a week just on doing release builds. The last one failed this
morning due to socket connection dropped errors with Nexus. I tried pushing a
change to skip the jar that was hitting that error to see if that helps - the
verify job for that change just finished after 1 hour and 20 mins.
Gildas - this constitutes a jeopardy on our Casablanca release. We are
completely blocked until this gets better : we can't create new docker
containers to test what we already have; we can't merge anything new. The
factory is essentially closed for business.
Dan
--
Dan Timoney
SDN-CP Development
ONAP Project Technical Lead : CCSDK and SDNC
Please go to D2 ECOMP Release Planning Wiki
<https://wiki.web.att.com/display/DERP/D2+ECOMP+Release+Planning+Home> for D2
ECOMP Project In-take, 2016 Release Planning, Change Management, and find key
Release Planning Contact Information.
On 10/2/18, 2:01 PM, "[email protected] on behalf of Gary Wu"
<[email protected] on behalf of [email protected]> wrote:
It seems like nexus and nexus3 are suffering severe congestion right
now, so the jobs are spending a long time trying to download artifacts.
Maybe we should consider migrating those systems to AWS as well?
Thanks,
Gary
-----Original Message-----
From: [email protected] [mailto:[email protected]]
On Behalf Of Pamela Dragosh
Sent: Tuesday, October 02, 2018 10:12 AM
To: [email protected]
Cc: [email protected]
Subject: Re: [onap-discuss] [ONAP Helpdesk #61676] gerrit down
This just started this morning. Seems to have been resolved as folks
can now submit changes.
Nobody hard codes any IP address, their environments haven’t changed in
months. These were separate firewalls.
But jenkins is really, really slow today and failing to finish verify
and merge jobs.
Pam
On 10/2/18, 11:49 AM, "Andrew Grimberg via RT"
<[email protected]> wrote:
On 10/02/2018 06:56 AM, [email protected] via RT wrote:
>
> It seems that multiple folks on my team across the US cannot
submit a
> gerrit review. It either hangs or gives a connection refused.
>
Greetings Pam,
Is this a problem that people have been having (and are still
having)
since the maintenance of Gerrit on Friday?
If so, it sounds like there's possibly corporate firewalls blocking
access then as the ONAP Gerrit system is working just fine for me
and
I've seen no alarms from our monitoring.
The new system is now behind an Amazon ELB and does not have hard
and
firm IP addresses. My experience with these sorts of systems is
that the
addresses will change with no notice and no control on our part but
as
Amazon remanages the addresses in their own system.
The _current_ IP addresses are: 35.166.47.83 and 35.164.144.118
-Andy-
--
Andrew J Grimberg
Manager Release Engineering
The Linux Foundation
This message and the information contained herein is proprietary and
confidential and subject to the Amdocs policy statement,
you may review at https://www.amdocs.com/about/email-disclaimer
<https://www.amdocs.com/about/email-disclaimer>
This message and the information contained herein is proprietary and
confidential and subject to the Amdocs policy statement,
you may review at https://www.amdocs.com/about/email-disclaimer
<https://www.amdocs.com/about/email-disclaimer>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#12806): https://lists.onap.org/g/onap-discuss/message/12806
Mute This Topic: https://lists.onap.org/mt/26661009/21656
Group Owner: [email protected]
Unsubscribe: https://lists.onap.org/g/onap-discuss/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
