On 26 January 2018 at 09:13, Gregory Rose <[email protected]> wrote: > On 1/19/2018 7:55 PM, Ansis Atteka wrote: >> >> From: Ansis Atteka <[email protected]> >> >> This patch sets up foundations for Proof of Concepts that >> simply materialize documentation into Ansible instructions >> executed in virtualized Vagrant environment. >> >> This Proof of Concept allows to easily build: >> 1. *.deb packages on Ubuntu 16.04; AND >> 2. *.rpm packages on CentOS 7.4. >> It also sets up DEB and RPM repository over HTTP that can >> be used to pull these openvswitch packages with apt-get >> or yum from another host. >> >> This particular Proof of Concept is intended to address >> following use-cases: >> 1. for new OVS users to see how debian and rpm packages are >> built; >> 2. for developers to easily check for packaging build >> regressions; >> 3. for developers to easily share their sandbox builds >> into QE setups (opposed to manually copying binaries); >> 4. for developers to add other Proof of Concepts >> that possibly may require full end-to-end integration >> with other thirdparty projects (e.g. DPI, libvirt, IPsec) >> and need Open vSwitch packages. >> >> Signed-off-by: Ansis Atteka <[email protected]> >> --- >> .gitignore | 2 + >> Documentation/topics/testing.rst | 46 ++++++++++++++++++ >> Makefile.am | 3 ++ >> poc/builders/Vagrantfile | 35 ++++++++++++++ >> poc/playbook-centos-builder.yml | 100 >> +++++++++++++++++++++++++++++++++++++++ >> poc/playbook-ubuntu-builder.yml | 66 ++++++++++++++++++++++++++ >> 6 files changed, 252 insertions(+) >> create mode 100644 poc/builders/Vagrantfile >> create mode 100644 poc/playbook-centos-builder.yml >> create mode 100644 poc/playbook-ubuntu-builder.yml >> >> diff --git a/.gitignore b/.gitignore >> index 8019bee41..81faf270d 100644 >> --- a/.gitignore >> +++ b/.gitignore >> @@ -16,6 +16,7 @@ >> *.lib >> *.pdb >> *.pyc >> +*.retry >> *.so >> *.suo >> **/*.sym >> @@ -29,6 +30,7 @@ >> .dirstamp >> .libs >> .tmp_versions >> +.vagrant >> .gitattributes >> /Makefile >> /Makefile.in >> diff --git a/Documentation/topics/testing.rst >> b/Documentation/topics/testing.rst >> index a49336b79..4d93944c7 100644 >> --- a/Documentation/topics/testing.rst >> +++ b/Documentation/topics/testing.rst >> @@ -389,3 +389,49 @@ validate the suitability of different vSwitch >> implementations in a telco >> deployment environment. More information can be found on the `OPNFV >> wiki`_. >> .. _OPNFV wiki: https://wiki.opnfv.org/display/vsperf/VSperf+Home >> + >> +Proof of Concepts >> +~~~~~~~~~~~~~~~~~ >> + >> +Proof of Concepts are documentation materialized into Ansible recipes >> +executed in Virtualbox environment orchastrated by Vagrant. Proof of >> +Concepts allow developers to create small virtualized setups that >> +demonstrate how certain Open vSwitch features are intended to work, >> +especially when integration with thirdparty software is involved. > > s/thirdparty/third party > >> + >> +The host where Vagrant runs does not need to have any special software >> +installed besides vagrant, virtualbox and ansible. > > > Seem it also requires Apache? Since the Ansible recipes are executed only on the guest (and not host) then only guests need Apache/httpd installed. Let me know if I am missing something and you still had to install Apache on host?
Apache is required only to have the packages distributed via repository. > >> + >> +The following Proof of Concepts are supported: >> + >> +Builders >> +++++++++ >> + >> +This Proof of Concept demonsrtates integration with Debian and RPM >> +packaging tools: >> + >> + $ cd ./poc/builders >> + >> + # When setting up Proof of Concept for the first time run: >> + $ vagrant up >> + >> +Once that command finished you can get packages from /var/www/html >> +directory. Since those hosts are also configured as repositories then >> +you can add them to /etc/apt/sources.list.d or /etc/yum.repos.d >> +configuration files. >> + >> + # When you have made changes to OVS and want to rebuild packages run: >> + $ git commit -a >> + $ vagrant rsync && vagrant provision >> + >> +Each packages are rebuilt the Open vSwitch release number increases > > > Maybe "When packages are rebuilt" ? You are right. Will send V2. > > There are some checkpatch warnings about lines too long but I'm not worried > about those. There > are a few other checkpatch warnings about lines with trailing whitespace. > Those should probably > be cleaned up. Agree about whitespace errors, not sure how I missed them... Will look into truncating lines to 80 characters. There are some lines that require file-system paths so it may require a little more creativity to truncate them without sacrificing readability. > > The rest of the patch seems fine. I applied it and ran the simple test case > you outline > and it worked as advertised. I have a bit of a learning curve to get over > for me to make > further use of this but looks promising! > > Thanks Ansis! Great. I am somewhat curious if we should switch from Virtualbox to KVM/libvirt, because it appears that is what most developers are using here. Will let others chime in as well. > > Tested-by: Greg Rose <[email protected]> > Reviewed-by: Greg Rose <[email protected]> > > >> +by one and you can simply upgrade them by running yum or apt-get. >> + >> +Once you are done with Particular Proof of Concept run >> + >> + $ vagrant destroy >> + >> +Or simply go to VirtualBox management console and delete those VMs. >> + >> +Sometime deployment of Proof of Concept may fail if for example VMs >> +don't have network reachability to the Internet. >> diff --git a/Makefile.am b/Makefile.am >> index ed4b7fd7c..174a421e6 100644 >> --- a/Makefile.am >> +++ b/Makefile.am >> @@ -90,6 +90,9 @@ EXTRA_DIST = \ >> build-aux/sodepends.py \ >> build-aux/soexpand.py \ >> build-aux/xml2nroff \ >> + poc/builders/Vagrantfile \ >> + poc/playbook-centos-builder.yml \ >> + poc/playbook-ubuntu-builder.yml \ >> $(MAN_FRAGMENTS) \ >> $(MAN_ROOTS) \ >> Vagrantfile \ >> diff --git a/poc/builders/Vagrantfile b/poc/builders/Vagrantfile >> new file mode 100644 >> index 000000000..00b114475 >> --- /dev/null >> +++ b/poc/builders/Vagrantfile >> @@ -0,0 +1,35 @@ >> +# -*- mode: ruby -*- >> +# vi: set ft=ruby : >> + >> +VAGRANTFILE_API_VERSION = "2" >> + >> +Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| >> + config.ssh.insert_key = false >> + config.vm.provider :virtualbox do |vb| >> + vb.customize ["modifyvm", :id, "--memory", "1024"] >> + vb.customize ["modifyvm", :id, "--cpus", "4"] >> + end >> + >> + # Centos-7.4 builder host >> + config.vm.define "centosbuilder" do |builder| >> + builder.vm.hostname = "centosbuilder.dev" >> + builder.vm.box = "bento/centos-7.4" >> + builder.vm.synced_folder "../../", "/git/ovs", type: "rsync", >> rsync__args: ["--verbose", "--archive", "--delete", "-z"] >> + builder.vm.provision "builder", type: "ansible" do |ansible| >> + ansible.playbook = "../playbook-centos-builder.yml" >> + ansible.sudo = true >> + end >> + end >> + >> + # Ubuntu-16.04 builder host >> + config.vm.define "ubuntubuilder" do |builder| >> + builder.vm.hostname = "ubuntubuilder.dev" >> + builder.vm.box = "bento/ubuntu-16.04" >> + builder.vm.synced_folder "../../", "/git/ovs", type: "rsync", >> rsync__args: ["--verbose", "--archive", "--delete", "-z"] >> + builder.vm.provision "builder", type: "ansible" do |ansible| >> + ansible.playbook = "../playbook-ubuntu-builder.yml" >> + ansible.sudo = true >> + end >> + end >> + >> +end >> diff --git a/poc/playbook-centos-builder.yml >> b/poc/playbook-centos-builder.yml >> new file mode 100644 >> index 000000000..b9b6e0776 >> --- /dev/null >> +++ b/poc/playbook-centos-builder.yml >> @@ -0,0 +1,100 @@ >> +--- >> +- hosts: all >> + become: true >> + name: builder >> + tasks: >> + >> + - name: Create Ansible Local Facts Directory >> + file: path=/etc/ansible/facts.d state=directory >> + >> + - name: Initiate Build Numbering >> + copy: >> + content: '{ "build_counter":"1" }' >> + dest: "/etc/ansible/facts.d/builder.fact" >> + force: no >> + >> + - name: Reload Ansible Local Facts >> + setup: filter=ansible_local >> + >> + - name: Install "yum-utils", "rpmdevtools", "createrepo", "httpd", >> "git" >> + yum: update_cache=yes name={{item}} state=present >> + with_items: >> + - yum-utils >> + - rpmdevtools >> + - createrepo >> + - httpd >> + - git >> + >> + - name: Remove untracked files from Open vSwitch GIT repository >> + command: chdir=/git/ovs/ git clean -xdf >> + >> + - name: Reset Open vSwitch GIT repository to last comitted state >> + command: chdir=/git/ovs/ git reset --hard >> + >> + - name: Generate "openvswitch.spec" and "openvswitch-kmod-rhel6.spec" >> files for easy build dependency retrieval >> + shell: sed -e 's/@VERSION@/0.0.1/' {{item}}.in > /tmp/{{item}} >> + args: >> + chdir: /git/ovs/rhel >> + with_items: >> + - openvswitch.spec >> + - openvswitch-kmod-rhel6.spec >> + >> + - name: Install build dependencies specified in "openvswitch.spec" and >> "openvswitch-kmod-rhel6.spec" >> + shell: echo "y" | yum-builddep /tmp/{{item}} >> + with_items: >> + - openvswitch.spec >> + - openvswitch-kmod-rhel6.spec >> + >> + - name: Create rpm dev tree >> + command: rpmdev-setuptree >> + >> + - name: Run "./boot.sh" >> + command: chdir=/git/ovs/ ./boot.sh >> + >> + - name: Run "./configure" >> + command: chdir=/git/ovs/ ./configure >> + >> + - name: Run "make dist" >> + command: chdir=/git/ovs/ make dist >> + >> + - name: Parse out Open vSwitch version from "configure.ac" >> + command: chdir=/git/ovs autoconf -t AC_INIT:'$2' >> + register: version >> + >> + - name: Copy source tarball to rpm dev tree >> + command: cp /git/ovs/openvswitch-{{version.stdout}}.tar.gz >> /root/rpmbuild/SOURCES/ >> + >> + - name: Unarchive openvswitch source tarball >> + unarchive: >> + src: /root/rpmbuild/SOURCES/openvswitch-{{version.stdout}}.tar.gz >> + dest: /root/rpmbuild/SOURCES >> + remote_src: yes >> + >> + - name: Update release number in "openvswitch.spec" and >> "openvswitch-kmod-rhel6.spec" >> + lineinfile: >> + path: >> /root/rpmbuild/SOURCES/openvswitch-{{version.stdout}}/rhel/{{item}} >> + regexp: '^Release:' >> + line: "Release: {{ ansible_local.builder.build_counter }}" >> + with_items: >> + - openvswitch.spec >> + - openvswitch-kmod-rhel6.spec >> + >> + - name: Build Open vSwitch user space rpms >> + command: chdir=/root/rpmbuild/SOURCES/openvswitch-{{version.stdout}} >> rpmbuild -bb --without check rhel/openvswitch.spec >> + >> + - name: Build Open vSwitch kmod rpms (only for currently loaded kernel) >> + command: chdir=/root/rpmbuild/SOURCES/openvswitch-{{version.stdout}} >> rpmbuild -bb --without check rhel/openvswitch-kmod-rhel6.spec >> + >> + - name: Copy RPM packages to /var/www/html >> + command: cp -r /root/rpmbuild/RPMS/ /var/www/html >> + >> + - name: Create RPM Package index file for repository >> + command: chdir=/var/www/html createrepo /var/www/html >> + >> + - name: Make sure Apache is running >> + systemd: state=started name=httpd >> + >> + - name: Bump up Build Number >> + copy: >> + content: '{ >> "build_counter":"{{ansible_local.builder.build_counter|int+1}}" }' >> + dest: "/etc/ansible/facts.d/builder.fact" >> diff --git a/poc/playbook-ubuntu-builder.yml >> b/poc/playbook-ubuntu-builder.yml >> new file mode 100644 >> index 000000000..c6f5b6eb0 >> --- /dev/null >> +++ b/poc/playbook-ubuntu-builder.yml >> @@ -0,0 +1,66 @@ >> +--- >> +- hosts: all >> + become: true >> + name: builder >> + tasks: >> + >> + - name: Create Ansible Local Facts Directory >> + file: path=/etc/ansible/facts.d state=directory >> + >> + - name: Initiate Build Numbering >> + copy: >> + content: '{ "build_counter":"1" }' >> + dest: "/etc/ansible/facts.d/builder.fact" >> + force: no >> + >> + - name: Reload Ansible Local Facts >> + setup: filter=ansible_local >> + >> + - name: Install "devscripts", "equivs", "apache2", "autoconf" >> + apt: update_cache=yes name={{item}} state=present >> + with_items: >> + - devscripts >> + - equivs >> + - apache2 >> + - autoconf >> + >> + - name: Remove untracked files from Open vSwitch GIT repository >> + command: chdir=/git/ovs/ git clean -xdf >> + >> + - name: Reset Open vSwitch GIT repository to last comitted state >> + command: chdir=/git/ovs/ git reset --hard >> + >> + - name: Parse out Open vSwitch version from "configure.ac" >> + command: chdir=/git/ovs autoconf -t AC_INIT:'$2' >> + register: version >> + >> + - name: Concatenate full version >> + set_fact: >> + full_version: >> "{{version.stdout}}-{{ansible_local.builder.build_counter}}" >> + >> + - name: Update Open vSwitch version to {{full_version}} >> + command: chdir=/git/ovs/ dch -b -v {{full_version}} Vagrant Build >> + >> + - name: Build debian package with Open vSwitch build dependencies >> + command: chdir=/git/ovs/ mk-build-deps -B debian/control >> + >> + - name: Install Open vSwitch {{full_version}} build dependencies >> + apt: >> deb=/git/ovs/openvswitch-build-deps-depends_{{full_version}}_all.deb >> + >> + - name: Build Open vSwitch {{full_version}} debian packages >> + shell: DEB_BUILD_OPTIONS='parallel=4 nocheck' fakeroot debian/rules >> binary >> + args: >> + chdir: /git/ovs/ >> + >> + - name: Move debian packages to /var/www/html >> + shell: mv /git/*.deb /var/www/html/ >> + >> + - name: Create Debian Package index file for repository >> + shell: dpkg-scanpackages . | gzip -9c > Packages.gz >> + args: >> + chdir: /var/www/html >> + >> + - name: Bump up Build Number >> + copy: >> + content: '{ >> "build_counter":"{{ansible_local.builder.build_counter|int+1}}" }' >> + dest: "/etc/ansible/facts.d/builder.fact" > > _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
