Michael, Thanks for the quick response! Answers inline. > On Mar 4, 2018, at 8:19 PM, Michael O'Brien <[email protected]> wrote: > > Joe, > Hi, Mike here, I am curious as to the hardware you use - I would like to > also test on some arm chips - the only arm chips I have access to in the one > in my iphone and the cortex based on in the PI3 > https://wiki.onap.org/display/DW/ONAP+on+ARM+Cortex-A53 - docker installs > there but I stopped short of finishing the K8S install on a cluster of 4. I > understand that the docker install in the scripts is intel specific. I > assume we are running master right. It would be nice if we could setup a > page specifically for ARM64. > I’m using a set of 6 servers, all based on Cavium ThunderX. 2 of the servers have 96 cores on 2 devices and 128G RAM, and the remaining 4 have 48 cores on one device and 64G RAM. All are in the Auto Lab at UNH Interop Lab at Univ. of New Hampshire. I have a heat template with a handful of scripts to install docker and k8s using kubeadm. Installing docker is done by “apt-get install docker.io” Installing k8s is done on each of the four nodes, doing a kubeadm init on the master and kubeadm join on the remaining three nodes (stashing the join info into /var/www/html on master and letting nodes grab via wget and then do a kubeadm join) Helm is a little trickier, as I need to build tiller (it’s not available on arm64 by default as far as I can tell), so I build that in the installation and then launch tiller “by hand” rather than launching in a container - that should be improved when I learn how to get the nice restart capability from k8s
If you’d like to get access to these servers, you are more than welcome to join me - it’s just a matter of getting openvpn creds from the folks that run the lab. Let me know and I can connect you. > For the scripts - make sure you are getting the latest from the JIRA under > review (they are not merged to gerrit yet) > https://jira.onap.org/browse/OOM-710 I am using the wgets described in the ref [1] in my email, e.g. wget https://jira.onap.org/secure/attachment/11122/cd.sh > > The prepull script (only for master) just parses all the values.yamls to get > the docker tags and pull them before starting up all the containers (90 pulls > take 20-240 min). Are you able to login and pull any one of them manually - > all the script does is pull 90 docker images in parallel - so if one works > they all will (you will see random errors - all of us get these periodically > against nexus3). > I did pull one - the first one, the config or onap-config. It’s built for amd64 so it doesn’t run…sits in “ContainerCreating” for a while (1minute+?), then disappears. When I just pull it down and inspect it, it’s “arch=amd64”. I’m pretty sure that means its for x86 and won’t run on arm, is that correct? > Try just bringing up a single component like robot, then a larger one that > is a leaf in the dependency tree like AAI. This would validate your setup. > ./createAll.bash -n onap -a robot > Or > ./createAll.bash -n onap -a aai > Of course after running ./createConfig.sh -n onap - which would pull down > the config container from the oomk8s repo on dockerhub - it must finish with > a "0/1 completed" before you can proceed - see the script. > > There is a fully automated oom_rancher_install.sh that is pulled down by > oom_entrypoint.sh (onap-parameters.yaml is only the simpler master version > right now) - that you can use as a reference (retrofit for arm). > > To simplify things (take the distributed share across your 4 vm's out of > the equation) - try installing robot or aai like above on a single collocated > VM (rancher and oom on a single 16g vm) - just to very your config. > > However I see your big blocking issue is the AMD64 vs ARM64 based docker > images - so you may need to reproduce the docker daily merge jobs of the CI > system to build your own - I think we may need to seriously involve the Linux > Foundation here. > > I am travelling right now - but I will add more info within 24 hours. Michael - thanks again for the feedback. In the meantime, I’ll work on making my initial steps available for general use (it's just replicating the “oom_rancher_setup.sh” script without rancher on arm64, without the docker images on apt.dockerproject.org, and without a pre-built tiller image for arm64). Looking at building ONAP components for arm64. Once we can get everything assembled and running on arm64, we probably want to look at taking a pass through ONAP scripts and documentation to be architecture-independent. Thanks and Safe Travels, Joe > /michael > > -----Original Message----- > From: Joe Kidder [mailto:[email protected]] > Sent: Sunday, March 4, 2018 20:05 > To: [email protected]; Michael O'Brien <[email protected]> > Subject: Running ONAP on arm64 - First need to build ONAP containers for > arm64 > > Hello ONAP Enthusiasts, > I am currently working on the Auto project in OPNFV, and one of our goals is > to run ONAP on arm64 servers. > At this point, I’m following the installation process described by Michael > O’Brien in [1]. > I currently have a 4-server k8s cluster (1 master and 3 nodes) deployed on 4 > VMs an arm64-based OPNFV open stack deployment. > I also have helm running on that cluster. > > When I start executing the steps in the “cd.sh” script described in [1], I > can see that the pulled docker images are built for arch=amd64, so they don’t > run. > > Can someone provide some advice/starting-hint to build the various ONAP > components? > In the meantime, I’ll start down the path of described here [2] at the > “Build ONAP” partway down the page. > > Thanks very much! > Joe Kidder > > [1] > https://wiki.onap.org/display/DW/ONAP+on+Kubernetes#ONAPonKubernetes-ExampleEndtoEndKubernetesbasedONAPinstallanddeployment > [2] https://wiki.onap.org/display/DW/Setting+Up+Your+Development+Environment > This message and the information contained herein is proprietary and > confidential and subject to the Amdocs policy statement, > > you may review at https://www.amdocs.com/about/email-disclaimer > <https://www.amdocs.com/about/email-disclaimer>
_______________________________________________ onap-discuss mailing list [email protected] https://lists.onap.org/mailman/listinfo/onap-discuss
