On 12 February 2018 at 10:41, Gregory Rose <[email protected]> wrote: > On 2/4/2018 6:48 PM, Ansis Atteka wrote: >> >> From: Ansis Atteka <[email protected]> >> >> This patch sets up foundations for Proof of Concepts that >> simply materialize documentation into Ansible instructions >> executed in virtualized Vagrant environment. >> >> This Proof of Concept allows to easily build: >> 1. *.deb packages on Ubuntu 16.04; AND >> 2. *.rpm packages on CentOS 7.4. >> It also sets up DEB and RPM repository over HTTP that can >> be used to pull these openvswitch packages with apt-get >> or yum from another host. >> >> This particular Proof of Concept is intended to address >> following use-cases: >> 1. for new OVS users to see how debian and rpm packages are >> built; >> 2. for developers to easily check for packaging build >> regressions; >> 3. for developers to easily share their sandbox builds >> into QE setups (opposed to manually copying binaries); >> 4. for developers to add other Proof of Concepts >> that possibly may require full end-to-end integration >> with other thirdparty projects (e.g. DPI, libvirt, IPsec) >> and need Open vSwitch packages. >> >> Tested-by: Greg Rose <[email protected]> >> Reviewed-by: Greg Rose <[email protected]> >> Signed-off-by: Ansis Atteka <[email protected]> > > > Rev 2 looks good. I did a quick sanity check test and LGTM. > > Reviewed-by: Greg Rose <[email protected]>
Thanks, I pushed this to master. > > >> --- >> .gitignore | 2 + >> Documentation/topics/testing.rst | 56 ++++++++++++++++++++ >> Makefile.am | 3 ++ >> poc/builders/Vagrantfile | 33 ++++++++++++ >> poc/playbook-centos-builder.yml | 108 >> +++++++++++++++++++++++++++++++++++++++ >> poc/playbook-ubuntu-builder.yml | 70 +++++++++++++++++++++++++ >> 6 files changed, 272 insertions(+) >> create mode 100644 poc/builders/Vagrantfile >> create mode 100644 poc/playbook-centos-builder.yml >> create mode 100644 poc/playbook-ubuntu-builder.yml >> >> diff --git a/.gitignore b/.gitignore >> index 8019bee41..81faf270d 100644 >> --- a/.gitignore >> +++ b/.gitignore >> @@ -16,6 +16,7 @@ >> *.lib >> *.pdb >> *.pyc >> +*.retry >> *.so >> *.suo >> **/*.sym >> @@ -29,6 +30,7 @@ >> .dirstamp >> .libs >> .tmp_versions >> +.vagrant >> .gitattributes >> /Makefile >> /Makefile.in >> diff --git a/Documentation/topics/testing.rst >> b/Documentation/topics/testing.rst >> index a49336b79..253857171 100644 >> --- a/Documentation/topics/testing.rst >> +++ b/Documentation/topics/testing.rst >> @@ -389,3 +389,59 @@ validate the suitability of different vSwitch >> implementations in a telco >> deployment environment. More information can be found on the `OPNFV >> wiki`_. >> .. _OPNFV wiki: https://wiki.opnfv.org/display/vsperf/VSperf+Home >> + >> +Proof of Concepts >> +~~~~~~~~~~~~~~~~~ >> + >> +Proof of Concepts are documentation materialized into Ansible recipes >> +executed in VirtualBox or Libvirt environments orchastrated by Vagrant. >> +Proof of Concepts allow developers to create small virtualized setups >> that >> +demonstrate how certain Open vSwitch features are intended to work >> avoiding >> +user introduced errors by overlooking instructions. Proof of Concepts >> +are also helpful when integrating with thirdparty software, because >> standard >> +unit tests with make check are limited. >> + >> +Vagrant by default uses VirtualBox provider. However, if Libvirt is your >> +choice of virtualization technology, then you can use it by installing >> Libvirt >> +plugin: >> + >> + $ vagrant plugin install vagrant-libvirt >> + >> +And then appending --provider=libvirt flag to vagrant commands. >> + >> +The host where Vagrant runs does not need to have any special software >> +installed besides vagrant, virtualbox (or libvirt and libvirt-dev) and >> +ansible. >> + >> +The following Proof of Concepts are supported: >> + >> +Builders >> +++++++++ >> + >> +This particular Proof of Concept demonsrtates integration with Debian and >> RPM >> +packaging tools: >> + >> + $ cd ./poc/builders >> + >> + $ vagrant up >> + >> +Once that command finished you can get packages from /var/www/html >> +directory. Since those hosts are also configured as repositories then >> +you can add them to /etc/apt/sources.list.d or /etc/yum.repos.d >> +configuration files on another host to retrieve packages with yum or >> +apt-get. >> + >> +hen you have made changes to OVS source code and want to rebuild packages >> run: >> + $ git commit -a >> + $ vagrant rsync && vagrant provision >> + >> +Whenever packages are rebuilt the Open vSwitch release number increases >> +by one and you can simply upgrade Open vSwitch by running yum or apt-get >> +update commands. >> + >> +Once you are done with experimenting you can tear down setup with: >> + >> + $ vagrant destroy >> + >> +Sometimes deployment of Proof of Concept may fail, if, for example, VMs >> +don't have network reachability to the Internet. >> diff --git a/Makefile.am b/Makefile.am >> index ed4b7fd7c..174a421e6 100644 >> --- a/Makefile.am >> +++ b/Makefile.am >> @@ -90,6 +90,9 @@ EXTRA_DIST = \ >> build-aux/sodepends.py \ >> build-aux/soexpand.py \ >> build-aux/xml2nroff \ >> + poc/builders/Vagrantfile \ >> + poc/playbook-centos-builder.yml \ >> + poc/playbook-ubuntu-builder.yml \ >> $(MAN_FRAGMENTS) \ >> $(MAN_ROOTS) \ >> Vagrantfile \ >> diff --git a/poc/builders/Vagrantfile b/poc/builders/Vagrantfile >> new file mode 100644 >> index 000000000..af2986789 >> --- /dev/null >> +++ b/poc/builders/Vagrantfile >> @@ -0,0 +1,33 @@ >> +# -*- mode: ruby -*- >> +# vi: set ft=ruby : >> + >> +VAGRANTFILE_API_VERSION = "2" >> + >> +Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| >> + config.ssh.insert_key = false >> + >> + # Centos-7.4 builder host >> + config.vm.define "centosbuilder" do |builder| >> + builder.vm.hostname = "centosbuilder.dev" >> + builder.vm.box = "centos/7" >> + builder.vm.synced_folder "../../", "/git/ovs", type: "rsync", >> + rsync__args: ["--archive", "--delete", "-z"] >> + builder.vm.provision "builder", type: "ansible" do |ansible| >> + ansible.playbook = "../playbook-centos-builder.yml" >> + ansible.sudo = true >> + end >> + end >> + >> + # Ubuntu-16.04 builder host >> + config.vm.define "ubuntubuilder" do |builder| >> + builder.vm.hostname = "ubuntubuilder.dev" >> + builder.vm.box = "generic/ubuntu1604" >> + builder.vm.synced_folder "../../", "/git/ovs", type: "rsync", >> + rsync__args: ["--archive", "--delete", "-z"] >> + builder.vm.provision "builder", type: "ansible" do |ansible| >> + ansible.playbook = "../playbook-ubuntu-builder.yml" >> + ansible.sudo = true >> + end >> + end >> + >> +end >> diff --git a/poc/playbook-centos-builder.yml >> b/poc/playbook-centos-builder.yml >> new file mode 100644 >> index 000000000..71f104010 >> --- /dev/null >> +++ b/poc/playbook-centos-builder.yml >> @@ -0,0 +1,108 @@ >> +--- >> +- hosts: all >> + become: true >> + name: builder >> + tasks: >> + >> + - name: Create Ansible Local Facts Directory >> + file: path=/etc/ansible/facts.d state=directory >> + >> + - name: Initiate Build Numbering >> + copy: >> + content: '{ "release":"1" }' >> + dest: "/etc/ansible/facts.d/builder.fact" >> + force: no >> + >> + - name: Set source directory for building >> + set_fact: >> + SOURCE: "/root/rpmbuild/SOURCES" >> + >> + - name: Reload Ansible Local Facts >> + setup: filter=ansible_local >> + >> + - name: Install "yum-utils", "rpmdevtools", "createrepo", "httpd", >> "git" >> + yum: update_cache=yes name={{item}} state=present >> + with_items: >> + - yum-utils >> + - rpmdevtools >> + - createrepo >> + - httpd >> + - git >> + >> + - name: Remove untracked files from Open vSwitch GIT repository >> + command: chdir=/git/ovs/ git clean -xdf >> + >> + - name: Reset Open vSwitch GIT repository to last comitted state >> + command: chdir=/git/ovs/ git reset --hard >> + >> + - name: Generate spec files for easy build dependency retrieval >> + shell: sed -e 's/@VERSION@/0.0.1/' {{item}}.in > /tmp/{{item}} >> + args: >> + chdir: /git/ovs/rhel >> + with_items: >> + - openvswitch.spec >> + - openvswitch-kmod-rhel6.spec >> + >> + - name: Install build dependencies specified from spec files >> + shell: echo "y" | yum-builddep /tmp/{{item}} >> + with_items: >> + - openvswitch.spec >> + - openvswitch-kmod-rhel6.spec >> + >> + - name: Create rpm dev tree >> + command: rpmdev-setuptree >> + >> + - name: Run "./boot.sh" >> + command: chdir=/git/ovs/ ./boot.sh >> + >> + - name: Run "./configure" >> + command: chdir=/git/ovs/ ./configure >> + >> + - name: Run "make dist" >> + command: chdir=/git/ovs/ make dist >> + >> + - name: Parse out Open vSwitch version from "configure.ac" >> + command: chdir=/git/ovs autoconf -t AC_INIT:'$2' >> + register: version >> + >> + - name: Copy source tarball to rpm dev tree >> + command: cp /git/ovs/openvswitch-{{version.stdout}}.tar.gz {{SOURCE}} >> + >> + - name: Unarchive openvswitch source tarball >> + unarchive: >> + src: "{{SOURCE}}/openvswitch-{{version.stdout}}.tar.gz" >> + dest: "{{SOURCE}}" >> + remote_src: yes >> + >> + - name: Update release number in spec files >> + lineinfile: >> + path: "{{SOURCE}}/openvswitch-{{version.stdout}}/rhel/{{item}}" >> + regexp: '^Release:' >> + line: "Release: {{ ansible_local.builder.release }}" >> + with_items: >> + - openvswitch.spec >> + - openvswitch-kmod-rhel6.spec >> + >> + - name: Build Open vSwitch user space rpms >> + command: rpmbuild -bb --without check rhel/openvswitch.spec >> + args: >> + chdir: "{{SOURCE}}/openvswitch-{{version.stdout}}" >> + >> + - name: Build Open vSwitch kmod rpms (only for currently loaded kernel) >> + command: rpmbuild -bb --without check >> rhel/openvswitch-kmod-rhel6.spec >> + args: >> + chdir: "{{SOURCE}}/openvswitch-{{version.stdout}}" >> + >> + - name: Copy RPM packages to /var/www/html >> + command: cp -r /root/rpmbuild/RPMS/ /var/www/html >> + >> + - name: Create RPM Package index file for repository >> + command: chdir=/var/www/html createrepo /var/www/html >> + >> + - name: Make sure Apache is running >> + systemd: state=started name=httpd >> + >> + - name: Bump up Build Number >> + copy: >> + content: '{ "release":"{{ansible_local.builder.release|int+1}}" >> }' >> + dest: "/etc/ansible/facts.d/builder.fact" >> diff --git a/poc/playbook-ubuntu-builder.yml >> b/poc/playbook-ubuntu-builder.yml >> new file mode 100644 >> index 000000000..5a13c6069 >> --- /dev/null >> +++ b/poc/playbook-ubuntu-builder.yml >> @@ -0,0 +1,70 @@ >> +--- >> +- hosts: all >> + become: true >> + name: builder >> + gather_facts: no >> + pre_tasks: >> + - name: 'install python2' >> + raw: sudo apt-get -y install python-simplejson >> + tasks: >> + >> + - name: Create Ansible Local Facts Directory >> + file: path=/etc/ansible/facts.d state=directory >> + >> + - name: Initiate Build Numbering >> + copy: >> + content: '{ "release":"1" }' >> + dest: "/etc/ansible/facts.d/builder.fact" >> + force: no >> + >> + - name: Reload Ansible Local Facts >> + setup: filter=ansible_local >> + >> + - name: Install "devscripts", "equivs", "apache2", "autoconf" >> + apt: update_cache=yes name={{item}} state=present >> + with_items: >> + - devscripts >> + - equivs >> + - apache2 >> + - autoconf >> + >> + - name: Remove untracked files from Open vSwitch GIT repository >> + command: chdir=/git/ovs/ git clean -xdf >> + >> + - name: Reset Open vSwitch GIT repository to last comitted state >> + command: chdir=/git/ovs/ git reset --hard >> + >> + - name: Parse out Open vSwitch version from "configure.ac" >> + command: chdir=/git/ovs autoconf -t AC_INIT:'$2' >> + register: version >> + >> + - name: Concatenate full version >> + set_fact: >> + full_version: >> "{{version.stdout}}-{{ansible_local.builder.release}}" >> + >> + - name: Update Open vSwitch version to {{full_version}} >> + command: chdir=/git/ovs/ dch -b -v {{full_version}} Vagrant Build >> + >> + - name: Build debian package with Open vSwitch build dependencies >> + command: chdir=/git/ovs/ mk-build-deps -B debian/control >> + >> + - name: Install Open vSwitch {{full_version}} build dependencies >> + apt: >> deb=/git/ovs/openvswitch-build-deps-depends_{{full_version}}_all.deb >> + >> + - name: Build Open vSwitch {{full_version}} debian packages >> + shell: DEB_BUILD_OPTIONS='nocheck' fakeroot debian/rules binary >> + args: >> + chdir: /git/ovs/ >> + >> + - name: Move debian packages to /var/www/html >> + shell: mv /git/*.deb /var/www/html/ >> + >> + - name: Create Debian Package index file for repository >> + shell: dpkg-scanpackages . | gzip -9c > Packages.gz >> + args: >> + chdir: /var/www/html >> + >> + - name: Bump up Build Number >> + copy: >> + content: '{ "release":"{{ansible_local.builder.release|int+1}}" >> }' >> + dest: "/etc/ansible/facts.d/builder.fact" > > > _______________________________________________ > dev mailing list > [email protected] > https://mail.openvswitch.org/mailman/listinfo/ovs-dev _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
