Team,
The RC for nexus3.onap.org must be fixed – for now we are faster – from 2h
per image to 17min – our goal is 45 sec.
Some updates: the nexus3ap proxy is experiencing the same issues as any
other proxy – it must deal with the latency of nexus3.onap.org. On 2 VMs –
different behavior
On a VM that already pulled to the E’s from my own proxy – the pulls from
nexus3ap were fast as most of the layers are shared and already downloaded – as
soon as I hit an image that is not cached either locally or on nexus3ap – it
takes 17 min to download a 1.1G image instead of 45 seconds – better than the 2
hours previously but not really fixed.
On a VM that is empty – it takes the full 17 min per image to pull from
nexus3ap
It looks like nexus3ap is truncating the problem route enough to lower the
download time from 2 hours to 17 min per aaf 1.1g image for example.
Another issue is how we handle rebuilt images for branches like Casablanca
– I need to this fully but hopefully we do not need to download the entire
image from scratch/warm the proxy if we run the Jenkins merge jobs daily – ie
the hash changes.
There are indications this will not be an issues – because of shared layers
– I repulled images that were already downloaded the day before and only get a
5 sec cycle
1.0.5: Pulling from
onap/org.onap.dcaegen2.collectors.datafile.datafile-app-server
4fe2ade4980c: Already exists
6fc58a8d4ae4: Already exists
819f4a45746c: Pulling fs layer
9c4800b836af: Pulling fs layer
Clean server took 150 min to download these for example
ubuntu@ip-172-31-17-47:~$ sudo docker images
REPOSITORY TAG IMAGE ID
CREATED SIZE
nexus3ap.onap.org:10001/onap/aaf/aaf_service 2.1.8 6eb295fed110
5 weeks ago 1.16 GB
nexus3ap.onap.org:10001/onap/aaf/aaf_oauth 2.1.8 74dcdce76094
5 weeks ago 1.16 GB
nexus3ap.onap.org:10001/onap/aaf/aaf_locate 2.1.8 2a4eaa6275ff
5 weeks ago 1.16 GB
nexus3ap.onap.org:10001/onap/aaf/aaf_hello 2.1.8 495a01176053
5 weeks ago 1.16 GB
nexus3ap.onap.org:10001/onap/aaf/aaf_gui 2.1.8 8caa6dc681f0
5 weeks ago 1.16 GB
nexus3ap.onap.org:10001/onap/aaf/aaf_fs 2.1.8 3d663698534d
5 weeks ago 1.16 GB
nexus3ap.onap.org:10001/onap/aaf/aaf_cm 2.1.8 0ba25c4ec3fb
5 weeks ago 1.16 GB
nexus3ap.onap.org:10001/onap/aaf/aaf_agent 2.1.8 090b326a7f11
5 weeks ago 1.14 GB
nexus3ap.onap.org:10001/onap/aaf/aaf_config 2.1.8 6506ac785cb5
5 weeks ago 1.14 GB
nexus3ap.onap.org:10001/onap/aaf/aaf_cass 2.1.8 4b91e9b0b43f
5 weeks ago 323 MB
From: [email protected] <[email protected]> On Behalf Of Michael
O'Brien
Sent: Thursday, December 20, 2018 12:03 PM
To: [email protected]; [email protected]; onap-tsc
<[email protected]>; [email protected]; Gildas Lanilis
<[email protected]>; Kenny Paul <[email protected]>; Mike
Elliott <[email protected]>; Soleil, Alain <[email protected]>;
[email protected]; Yang Xu (Yang, Fixed Network) <[email protected]>
Subject: Re: [onap-tsc] [onap-discuss] Nexus3 slowdown issues
Catherine,
Some good news – so far a couple pulls were back to normal using the ap
mirror – getting 10+Mb/sec initially – they downloaded within a couple min.
I am rerunning the prepull script if it finished under an hour for the 40-60G
of images we are good – I will update jiras/wikis/mails at that point
The mirror is slowing now though and hanging – don’t know how many clients
it can handle – there are at least 3 pulling now – I will advise on the test
results
Using the following to test
# clean ubuntu 16.04 vm
sudo git clone https://gerrit.onap.org/r/logging-analytics
sudo cp logging-analytics/deploy/docker_prepull.sh .
sudo curl https://releases.rancher.com/install-docker/17.03.sh | sh
sudo usermod -aG docker ubuntu
sudo systemctl restart docker
sudo ./docker_prepull.sh -b casablanca -s nexus3ap.onap.org:10001
Thank you
/michael
From: [email protected]<mailto:[email protected]>
<[email protected]<mailto:[email protected]>> On Behalf Of
Jessica Wagantall
Sent: Thursday, December 20, 2018 11:04 AM
To: [email protected]<mailto:[email protected]>; onap-tsc
<[email protected]<mailto:[email protected]>>;
[email protected]<mailto:[email protected]>; Gildas
Lanilis <[email protected]<mailto:[email protected]>>; Kenny
Paul <[email protected]<mailto:[email protected]>>
Subject: [onap-discuss] Nexus3 slowdown issues
Dear ONAP team
Our Infra team is helping us out looking into the Nexus3 issues the teams are
facing.
We have also escalated to our provider to look into the routing issues and we
are
hoping to get an answer from them soon.
In the meantime, please also consider using
nexus3ap.onap.org<http://nexus3ap.onap.org> mirror in case this can
unblock you.
Sorry for the inconveniences again!
Jess
This email and the information contained herein is proprietary and confidential
and subject to the Amdocs Email Terms of Service, which you may review at
https://www.amdocs.com/about/email-terms-of-service
This email and the information contained herein is proprietary and confidential
and subject to the Amdocs Email Terms of Service, which you may review at
https://www.amdocs.com/about/email-terms-of-service
<https://www.amdocs.com/about/email-terms-of-service>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#14672): https://lists.onap.org/g/onap-discuss/message/14672
Mute This Topic: https://lists.onap.org/mt/28814380/21656
Group Owner: [email protected]
Unsubscribe: https://lists.onap.org/g/onap-discuss/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
