From: Ansis Atteka <ansisatt...@gmail.com> This patch sets up foundations for Proof of Concepts that simply materialize documentation into Ansible instructions executed in virtualized Vagrant environment.
This Proof of Concept allows to easily build: 1. *.deb packages on Ubuntu 16.04; AND 2. *.rpm packages on CentOS 7.4. It also sets up DEB and RPM repository over HTTP that can be used to pull these openvswitch packages with apt-get or yum from another host. This particular Proof of Concept is intended to address following use-cases: 1. for new OVS users to see how debian and rpm packages are built; 2. for developers to easily check for packaging build regressions; 3. for developers to easily share their sandbox builds into QE setups (opposed to manually copying binaries); 4. for developers to add other Proof of Concepts that possibly may require full end-to-end integration with other thirdparty projects (e.g. DPI, libvirt, IPsec) and need Open vSwitch packages. Tested-by: Greg Rose <gvrose8...@gmail.com> Reviewed-by: Greg Rose <gvrose8...@gmail.com> Signed-off-by: Ansis Atteka <aatt...@ovn.org> --- .gitignore | 2 + Documentation/topics/testing.rst | 56 ++++++++++++++++++++ Makefile.am | 3 ++ poc/builders/Vagrantfile | 33 ++++++++++++ poc/playbook-centos-builder.yml | 108 +++++++++++++++++++++++++++++++++++++++ poc/playbook-ubuntu-builder.yml | 70 +++++++++++++++++++++++++ 6 files changed, 272 insertions(+) create mode 100644 poc/builders/Vagrantfile create mode 100644 poc/playbook-centos-builder.yml create mode 100644 poc/playbook-ubuntu-builder.yml diff --git a/.gitignore b/.gitignore index 8019bee41..81faf270d 100644 --- a/.gitignore +++ b/.gitignore @@ -16,6 +16,7 @@ *.lib *.pdb *.pyc +*.retry *.so *.suo **/*.sym @@ -29,6 +30,7 @@ .dirstamp .libs .tmp_versions +.vagrant .gitattributes /Makefile /Makefile.in diff --git a/Documentation/topics/testing.rst b/Documentation/topics/testing.rst index a49336b79..253857171 100644 --- a/Documentation/topics/testing.rst +++ b/Documentation/topics/testing.rst @@ -389,3 +389,59 @@ validate the suitability of different vSwitch implementations in a telco deployment environment. More information can be found on the `OPNFV wiki`_. .. _OPNFV wiki: https://wiki.opnfv.org/display/vsperf/VSperf+Home + +Proof of Concepts +~~~~~~~~~~~~~~~~~ + +Proof of Concepts are documentation materialized into Ansible recipes +executed in VirtualBox or Libvirt environments orchastrated by Vagrant. +Proof of Concepts allow developers to create small virtualized setups that +demonstrate how certain Open vSwitch features are intended to work avoiding +user introduced errors by overlooking instructions. Proof of Concepts +are also helpful when integrating with thirdparty software, because standard +unit tests with make check are limited. + +Vagrant by default uses VirtualBox provider. However, if Libvirt is your +choice of virtualization technology, then you can use it by installing Libvirt +plugin: + + $ vagrant plugin install vagrant-libvirt + +And then appending --provider=libvirt flag to vagrant commands. + +The host where Vagrant runs does not need to have any special software +installed besides vagrant, virtualbox (or libvirt and libvirt-dev) and +ansible. + +The following Proof of Concepts are supported: + +Builders +++++++++ + +This particular Proof of Concept demonsrtates integration with Debian and RPM +packaging tools: + + $ cd ./poc/builders + + $ vagrant up + +Once that command finished you can get packages from /var/www/html +directory. Since those hosts are also configured as repositories then +you can add them to /etc/apt/sources.list.d or /etc/yum.repos.d +configuration files on another host to retrieve packages with yum or +apt-get. + +hen you have made changes to OVS source code and want to rebuild packages run: + $ git commit -a + $ vagrant rsync && vagrant provision + +Whenever packages are rebuilt the Open vSwitch release number increases +by one and you can simply upgrade Open vSwitch by running yum or apt-get +update commands. + +Once you are done with experimenting you can tear down setup with: + + $ vagrant destroy + +Sometimes deployment of Proof of Concept may fail, if, for example, VMs +don't have network reachability to the Internet. diff --git a/Makefile.am b/Makefile.am index ed4b7fd7c..174a421e6 100644 --- a/Makefile.am +++ b/Makefile.am @@ -90,6 +90,9 @@ EXTRA_DIST = \ build-aux/sodepends.py \ build-aux/soexpand.py \ build-aux/xml2nroff \ + poc/builders/Vagrantfile \ + poc/playbook-centos-builder.yml \ + poc/playbook-ubuntu-builder.yml \ $(MAN_FRAGMENTS) \ $(MAN_ROOTS) \ Vagrantfile \ diff --git a/poc/builders/Vagrantfile b/poc/builders/Vagrantfile new file mode 100644 index 000000000..af2986789 --- /dev/null +++ b/poc/builders/Vagrantfile @@ -0,0 +1,33 @@ +# -*- mode: ruby -*- +# vi: set ft=ruby : + +VAGRANTFILE_API_VERSION = "2" + +Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| + config.ssh.insert_key = false + + # Centos-7.4 builder host + config.vm.define "centosbuilder" do |builder| + builder.vm.hostname = "centosbuilder.dev" + builder.vm.box = "centos/7" + builder.vm.synced_folder "../../", "/git/ovs", type: "rsync", + rsync__args: ["--archive", "--delete", "-z"] + builder.vm.provision "builder", type: "ansible" do |ansible| + ansible.playbook = "../playbook-centos-builder.yml" + ansible.sudo = true + end + end + + # Ubuntu-16.04 builder host + config.vm.define "ubuntubuilder" do |builder| + builder.vm.hostname = "ubuntubuilder.dev" + builder.vm.box = "generic/ubuntu1604" + builder.vm.synced_folder "../../", "/git/ovs", type: "rsync", + rsync__args: ["--archive", "--delete", "-z"] + builder.vm.provision "builder", type: "ansible" do |ansible| + ansible.playbook = "../playbook-ubuntu-builder.yml" + ansible.sudo = true + end + end + +end diff --git a/poc/playbook-centos-builder.yml b/poc/playbook-centos-builder.yml new file mode 100644 index 000000000..71f104010 --- /dev/null +++ b/poc/playbook-centos-builder.yml @@ -0,0 +1,108 @@ +--- +- hosts: all + become: true + name: builder + tasks: + + - name: Create Ansible Local Facts Directory + file: path=/etc/ansible/facts.d state=directory + + - name: Initiate Build Numbering + copy: + content: '{ "release":"1" }' + dest: "/etc/ansible/facts.d/builder.fact" + force: no + + - name: Set source directory for building + set_fact: + SOURCE: "/root/rpmbuild/SOURCES" + + - name: Reload Ansible Local Facts + setup: filter=ansible_local + + - name: Install "yum-utils", "rpmdevtools", "createrepo", "httpd", "git" + yum: update_cache=yes name={{item}} state=present + with_items: + - yum-utils + - rpmdevtools + - createrepo + - httpd + - git + + - name: Remove untracked files from Open vSwitch GIT repository + command: chdir=/git/ovs/ git clean -xdf + + - name: Reset Open vSwitch GIT repository to last comitted state + command: chdir=/git/ovs/ git reset --hard + + - name: Generate spec files for easy build dependency retrieval + shell: sed -e 's/@VERSION@/0.0.1/' {{item}}.in > /tmp/{{item}} + args: + chdir: /git/ovs/rhel + with_items: + - openvswitch.spec + - openvswitch-kmod-rhel6.spec + + - name: Install build dependencies specified from spec files + shell: echo "y" | yum-builddep /tmp/{{item}} + with_items: + - openvswitch.spec + - openvswitch-kmod-rhel6.spec + + - name: Create rpm dev tree + command: rpmdev-setuptree + + - name: Run "./boot.sh" + command: chdir=/git/ovs/ ./boot.sh + + - name: Run "./configure" + command: chdir=/git/ovs/ ./configure + + - name: Run "make dist" + command: chdir=/git/ovs/ make dist + + - name: Parse out Open vSwitch version from "configure.ac" + command: chdir=/git/ovs autoconf -t AC_INIT:'$2' + register: version + + - name: Copy source tarball to rpm dev tree + command: cp /git/ovs/openvswitch-{{version.stdout}}.tar.gz {{SOURCE}} + + - name: Unarchive openvswitch source tarball + unarchive: + src: "{{SOURCE}}/openvswitch-{{version.stdout}}.tar.gz" + dest: "{{SOURCE}}" + remote_src: yes + + - name: Update release number in spec files + lineinfile: + path: "{{SOURCE}}/openvswitch-{{version.stdout}}/rhel/{{item}}" + regexp: '^Release:' + line: "Release: {{ ansible_local.builder.release }}" + with_items: + - openvswitch.spec + - openvswitch-kmod-rhel6.spec + + - name: Build Open vSwitch user space rpms + command: rpmbuild -bb --without check rhel/openvswitch.spec + args: + chdir: "{{SOURCE}}/openvswitch-{{version.stdout}}" + + - name: Build Open vSwitch kmod rpms (only for currently loaded kernel) + command: rpmbuild -bb --without check rhel/openvswitch-kmod-rhel6.spec + args: + chdir: "{{SOURCE}}/openvswitch-{{version.stdout}}" + + - name: Copy RPM packages to /var/www/html + command: cp -r /root/rpmbuild/RPMS/ /var/www/html + + - name: Create RPM Package index file for repository + command: chdir=/var/www/html createrepo /var/www/html + + - name: Make sure Apache is running + systemd: state=started name=httpd + + - name: Bump up Build Number + copy: + content: '{ "release":"{{ansible_local.builder.release|int+1}}" }' + dest: "/etc/ansible/facts.d/builder.fact" diff --git a/poc/playbook-ubuntu-builder.yml b/poc/playbook-ubuntu-builder.yml new file mode 100644 index 000000000..5a13c6069 --- /dev/null +++ b/poc/playbook-ubuntu-builder.yml @@ -0,0 +1,70 @@ +--- +- hosts: all + become: true + name: builder + gather_facts: no + pre_tasks: + - name: 'install python2' + raw: sudo apt-get -y install python-simplejson + tasks: + + - name: Create Ansible Local Facts Directory + file: path=/etc/ansible/facts.d state=directory + + - name: Initiate Build Numbering + copy: + content: '{ "release":"1" }' + dest: "/etc/ansible/facts.d/builder.fact" + force: no + + - name: Reload Ansible Local Facts + setup: filter=ansible_local + + - name: Install "devscripts", "equivs", "apache2", "autoconf" + apt: update_cache=yes name={{item}} state=present + with_items: + - devscripts + - equivs + - apache2 + - autoconf + + - name: Remove untracked files from Open vSwitch GIT repository + command: chdir=/git/ovs/ git clean -xdf + + - name: Reset Open vSwitch GIT repository to last comitted state + command: chdir=/git/ovs/ git reset --hard + + - name: Parse out Open vSwitch version from "configure.ac" + command: chdir=/git/ovs autoconf -t AC_INIT:'$2' + register: version + + - name: Concatenate full version + set_fact: + full_version: "{{version.stdout}}-{{ansible_local.builder.release}}" + + - name: Update Open vSwitch version to {{full_version}} + command: chdir=/git/ovs/ dch -b -v {{full_version}} Vagrant Build + + - name: Build debian package with Open vSwitch build dependencies + command: chdir=/git/ovs/ mk-build-deps -B debian/control + + - name: Install Open vSwitch {{full_version}} build dependencies + apt: deb=/git/ovs/openvswitch-build-deps-depends_{{full_version}}_all.deb + + - name: Build Open vSwitch {{full_version}} debian packages + shell: DEB_BUILD_OPTIONS='nocheck' fakeroot debian/rules binary + args: + chdir: /git/ovs/ + + - name: Move debian packages to /var/www/html + shell: mv /git/*.deb /var/www/html/ + + - name: Create Debian Package index file for repository + shell: dpkg-scanpackages . | gzip -9c > Packages.gz + args: + chdir: /var/www/html + + - name: Bump up Build Number + copy: + content: '{ "release":"{{ansible_local.builder.release|int+1}}" }' + dest: "/etc/ansible/facts.d/builder.fact" -- 2.14.1 _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev