Hello community, here is the log from the commit of package openQA for openSUSE:Factory checked in at 2018-05-29 10:48:43 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/openQA (Old) and /work/SRC/openSUSE:Factory/.openQA.new (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "openQA" Tue May 29 10:48:43 2018 rev:22 rq:612555 version:4.5.1527149502.003be4d6 Changes: -------- --- /work/SRC/openSUSE:Factory/openQA/openQA.changes 2018-05-17 18:06:13.762101137 +0200 +++ /work/SRC/openSUSE:Factory/.openQA.new/openQA.changes 2018-05-29 10:48:43.852055341 +0200 @@ -1,0 +2,16 @@ +Thu May 24 14:09:38 UTC 2018 - [email protected] + +- Update to version 4.5.1527149502.003be4d6: + * Add error logging in gru (#1660) + * Fix the alignment of the progress bars on group overview (#1659) + * Add http-only config for nginx (#1526) + * Disable needle selection if there's nothing to select + * Reduce padding of lists in tool tips + * Fix devider in job group navigation with Bootstrap 4 + * Allow specifying group for isos post by name + * Fix DataTable error under route /tests/latest (#1656) + * Exclude external links in 22-dashboard.t (#1655) + * Add _GROUP filter to isos post + * Update documentation for multimachine tests setup + +------------------------------------------------------------------- Old: ---- openQA-4.5.1526551139.04373a1e.tar.xz New: ---- openQA-4.5.1527149502.003be4d6.tar.xz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ openQA.spec ++++++ --- /var/tmp/diff_new_pack.oHKBWV/_old 2018-05-29 10:48:44.540029958 +0200 +++ /var/tmp/diff_new_pack.oHKBWV/_new 2018-05-29 10:48:44.540029958 +0200 @@ -24,7 +24,7 @@ %{_bindir}/systemd-tmpfiles --create %{?*} || : \ %{nil} %endif -%if 0%{?suse_version} >= 1500 +%if 0%{?suse_version} >= 1550 %ifarch x86_64 %bcond_without tests %else @@ -36,7 +36,7 @@ # runtime requirements that also the testsuite needs %define t_requires perl(DBD::Pg) perl(DBIx::Class) perl(Config::IniFiles) perl(SQL::Translator) perl(Date::Format) perl(File::Copy::Recursive) perl(DateTime::Format::Pg) perl(Net::OpenID::Consumer) perl(Mojolicious::Plugin::RenderFile) perl(Mojolicious::Plugin::AssetPack) perl(aliased) perl(Config::Tiny) perl(DBIx::Class::DynamicDefault) perl(DBIx::Class::Schema::Config) perl(DBIx::Class::Storage::Statistics) perl(IO::Socket::SSL) perl(Data::Dump) perl(DBIx::Class::OptimisticLocking) perl(Text::Markdown) perl(Net::DBus) perl(IPC::Run) perl(Archive::Extract) perl(CSS::Minifier::XS) perl(JavaScript::Minifier::XS) perl(Time::ParseDate) perl(Sort::Versions) perl(Mojo::RabbitMQ::Client) perl(BSD::Resource) perl(Cpanel::JSON::XS) perl(Pod::POM) perl(Mojo::IOLoop::ReadWriteProcess) perl(Minion) perl(Mojo::Pg) Name: openQA -Version: 4.5.1526551139.04373a1e +Version: 4.5.1527149502.003be4d6 Release: 0 Summary: The openQA web-frontend, scheduler and tools License: GPL-2.0+ ++++++ openQA-4.5.1526551139.04373a1e.tar.xz -> openQA-4.5.1527149502.003be4d6.tar.xz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-4.5.1526551139.04373a1e/assets/stylesheets/overall.scss new/openQA-4.5.1527149502.003be4d6/assets/stylesheets/overall.scss --- old/openQA-4.5.1526551139.04373a1e/assets/stylesheets/overall.scss 2018-05-17 11:58:59.000000000 +0200 +++ new/openQA-4.5.1527149502.003be4d6/assets/stylesheets/overall.scss 2018-05-24 10:11:42.000000000 +0200 @@ -172,6 +172,9 @@ p { margin-bottom: 2px; } + ul { + padding-left: 1.2rem; + } p, ul { text-align: left; word-break: break-word; diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-4.5.1526551139.04373a1e/assets/stylesheets/result_preview.scss new/openQA-4.5.1527149502.003be4d6/assets/stylesheets/result_preview.scss --- old/openQA-4.5.1526551139.04373a1e/assets/stylesheets/result_preview.scss 2018-05-17 11:58:59.000000000 +0200 +++ new/openQA-4.5.1527149502.003be4d6/assets/stylesheets/result_preview.scss 2018-05-24 10:11:42.000000000 +0200 @@ -89,13 +89,13 @@ .candidates-selection { min-width: 50%; - .btn-left { + #screenshot_button { border-width: 1px 0px 1px 1px; border-style: solid; border-color: #d8d9da; } - .dropdown-toggle { + #candidatesMenu { font-weight: 400; border-radius: 4px; box-shadow: 0 0 0 1px rgba(34,36,38,.15) inset; diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-4.5.1526551139.04373a1e/docs/Networking.asciidoc new/openQA-4.5.1527149502.003be4d6/docs/Networking.asciidoc --- old/openQA-4.5.1526551139.04373a1e/docs/Networking.asciidoc 2018-05-17 11:58:59.000000000 +0200 +++ new/openQA-4.5.1527149502.003be4d6/docs/Networking.asciidoc 2018-05-24 10:11:42.000000000 +0200 @@ -14,7 +14,7 @@ For more advanced setups or tests that require multiple jobs to be in the same networking the <<TAP based network,TAP>> or <<VDE Based Network,VDE>> based modes can be used. -== Qemu user networking +== Qemu User Networking :qemu-user-networking: http://wiki.qemu.org/Documentation/Networking#User_Networking_.28SLIRP.29[user networking] With Qemu {qemu-user-networking} each jobs gets it's own isolated network with @@ -22,7 +22,7 @@ the machine can be controlled with the +NICMAC+ variable. If not set, it's +52:54:00:12:34:56+. -== TAP based network +=== TAP Based Network os-autoinst can connect qemu to TAP devices of the host system to leverage advanced network setups provided by the host by setting +NICTYPE=tap+. @@ -48,237 +48,280 @@ In TAP mode the system administrator is expected to configure the network, required internet access etc on the host manually. -TAP devices need be owned by the +_openqa-worker+ user for openQA to -be able to access them. +=== VDE Based Network -[source,sh] ---------------- -tunctl -u _openqa-worker -p -t tap0 ---------------- +Virtual Distributed Ethernet provides a software switch that runs in +user space. It allows to connect several qemu instances without +affecting the system's network configuration. -If you want to use TAP device which doesn't exist on the system, -you need to set CAP_NET_ADMIN capability on qemu binary file: +The openQA workers need a vde_switch instance running. The workers +reconfigure the switch as needed by the job. -[source,sh] ---------------- -zypper in libcap-progs -setcap CAP_NET_ADMIN=ep /usr/bin/qemu-system-x86_64 ---------------- +==== Basic, Single Machine Tests + +To start with a basic configuration like qemu user mode networking, +create a machine with the following settings: + +- +VDE_SOCKETDIR=/run/openqa+ +- +NICTYPE=vde+ +- +NICVLAN=0+ -Network setup can be changed after qemu is started using network configure script -specified in TAPSCRIPT variable. +Start switch and user mode networking: -Sample script to add TAP device to existing bridge br0: [source,sh] --------------- -sudo brctl addif br0 $1 -sudo ip link set $1 up +systemctl start openqa-vde_switch +systemctl start openqa-slirpvde --------------- -== TAP with Open vSwitch +With this setting all jobs on the same host would be in the same +network share the same SLIRP instance though. -The recommended way to configure the network for TAP devices is using Open vSwitch. -There is a support service +os-autoinst-openvswitch.service+ which sets vlan number -of Open vSwitch ports based on +NICVLAN+ variable - this separates the groups of -tests from each other. +== Multi Machine Tests Setup + +The section provides one of the ways for setting up openQA environment to run tests that +require network connection between several machines (e.g. client -- server tests). + +The example of the configuration is applicable for openSUSE and will +use _Open vSwitch_ for virtual switch, _SuSEfirewall2_ for NAT and _wicked_ as network manager. + +NOTE: Other way to setup the environment with _iptables_ and _firewalld_ is described +on link:https://fedoraproject.org/wiki/OpenQA_advanced_network_guide[Fedora wiki]. -+NICVLAN+ variable is dynamically assigned by OpenQA scheduler. +*Set Up Open vSwitch* -Compared to VDE setup discussed later, Open vSwitch is more complicated to configure, -but provides more robust and scalable network. +Compared to VDE setup, Open vSwitch is a little bit more complicated to configure, but provides more +robust and scalable network. -Start Open vSwitch and add TAP devices: +* Install and Run Open vSwitch: [source,sh] --------------- - -# start openvswitch.service +zypper in openvswitch systemctl start openvswitch.service systemctl enable openvswitch.service +--------------- -# create bridge -ovs-vsctl add-br br0 +* Install and configure _os-autoinst-openvswitch.service_: -# add tap devices, use vlan 999 by default, the vlan number is supposed to be changed when the vm starts -ovs-vsctl add-port br0 tap0 tag=999 -ovs-vsctl add-port br0 tap1 tag=999 -ovs-vsctl add-port br0 tap2 tag=999 -ovs-vsctl add-port br0 tap3 tag=999 -ovs-vsctl add-port br0 tap4 tag=999 +NOTE: _os-autoinst-openvswitch.service_ is a support service that sets vlan number +of Open vSwitch ports based on +NICVLAN+ variable - this separates the groups of +tests from each other. +NICVLAN+ variable is dynamically assigned by OpenQA scheduler. +[source,sh] +--------------- +zypper in os-autoinst-openvswitch +systemctl start os-autoinst-openvswitch +systemctl enable os-autoinst-openvswitch --------------- -The virtual machines need access to os-autoinst webserver acessible -via IP 10.0.2.2. The IP addresses of VMs are controlled by tests -and are likely to conflict if more independent tests runs in parallel. - -The VMs have unique MAC that differs in the last 16 bits (see /usr/lib/os-autoinst/backend/qemu.pm). +_os-autoinst-openvswitch.service_ uses _br0_ bridge by default. +As it might be used by KVM, configure _br1_ instead. -`os-autoinst-openvswitch.service` sets up filtering rules for the following translation scheme which -provide non-conflicting addresses visible from host: +[source,sh] +--------------- +# /etc/sysconfig/os-autoinst-openvswitch +OS_AUTOINST_USE_BRIDGE=br1 +--------------- +* Create virtual bridge _br1_: [source,sh] ----- -MAC 52:54:00:12:XX:YY -> IP 10.1.XX.YY ----- +--------------- +ovs-vsctl add-br br1 +--------------- -That means that the local port of the bridge must be configured to IP 10.0.2.2 -and netmask /15 that covers 10.0.0.0 and 10.1.0.0 ranges. +*Configure Virtual Interfaces* + +* Add tap interface for every multi-machine worker: + +NOTE: Create as many interfaces as needed for a test. The instructions are provided for three interfaces +_tap0_, _tap1_, _tap2_ to be used by _worker@1_, _worker@2_, _worker@3_ workers. TAP interfaces have to be owned by the __openqa-worker_ user for openQA to +be able to access them. + +To create tap interfaces automatically on startup, add appropriate configuration files to the +`/etc/sysconfig/network/` directory. Files have to be named as `ifcfg-tap<N>`, replacing `<N>` +with the number for the interface, such as `0`, `1`, `2` (e.g. `ifcfg-tap0`, `ifcfg-tap1`). [source,sh] --------------- -ip addr add 10.0.2.2/15 dev br0 -ip route add 10.0.0.0/15 dev br0 -ip link set br0 up +# /etc/sysconfig/network/ifcfg-tap0 +BOOTPROTO='none' +IPADDR='' +NETMASK='' +PREFIXLEN='' +STARTMODE='auto' +TUNNEL='tap' +TUNNEL_SET_GROUP='nogroup' +TUNNEL_SET_OWNER='_openqa-worker' --------------- +* Add bridge config with all tap devices that should be connected to it: -== Debugging Open vSwitch configuration +The file have to be located in `/etc/sysconfig/network/` directory. File name is +`ifcfg-br<N>`, where `<N>` is the id of the bridge (e.g. `1`). -Boot sequence with wicked < 0.6.23: +[source,sh] +--------------- +# /etc/sysconfig/network/ifcfg-br1 +BOOTPROTO='static' +IPADDR='10.0.2.2/15' +STARTMODE='auto' +OVS_BRIDGE='yes' +OVS_BRIDGE_PORT_DEVICE_1='tap0' +OVS_BRIDGE_PORT_DEVICE_2='tap1' +OVS_BRIDGE_PORT_DEVICE_3='tap2' +--------------- -1. wicked - creates tap devices -2. openvswitch - creates the bridge +br0+, adds tap devices to it -3. wicked handles +br0+ as hotplugged device, assignd the IP 10.0.2.2 to it, updates SuSEFirewall -4. os-autoinst-openvswitch - installs openflow rules, handles vlan assignment +*Configure NAT with SuSEfirewall* -Boot sequence with wicked 0.6.23 and newer: +The IP 10.0.2.2 can be also served as a gateway to access outside +network. For this, a NAT between _br1_ and _eth0_ must be configured +with SuSEfirewall or iptables. -1. openvswitch -2. wicked - creates the bridge +br0+ and tap devices, add tap devices to the bridge, -3. SuSEFirewall -4. os-autoinst-openvswitch - installs openflow rules, handles vlan assignment +[source,sh] +--------------- +# /etc/sysconfig/SuSEfirewall2 +FW_DEV_INT="br1" +FW_ROUTE="yes" +FW_MASQUERADE="yes" +--------------- +Start SuSEfirewall2 and allow to run on startup: -The configuration and operation can be checked by the following commands: +[source,sh] +--------------- +systemctl start SuSEfirewall2 +systemctl enable SuSEfirewall2 +--------------- + +*Configure OpenQA Workers* + +* Allow workers to run multi-machine jobs: [source,sh] ----- -ovs-vsctl show # shows the bridge br0, the tap devices are assigned to it -ovs-ofctl dump-flows br0 # shows the rules installed by os-autoinst-openvswitch in table=0 ----- +--------------- +# /etc/openqa/workers.ini +[global] +WORKER_CLASS = qemu_x86_64,tap +--------------- -* packets from tapX to br0 create additional rules in table=1 -* packets from br0 to tapX increase packet counts in table=1 -* empty output indicates a problem with os-autoinst-openvswitch service -* zero packet count or missing rules in table=1 indicate problem with tap devices +* Enable workers to be started on system boot: [source,sh] ----- -ipables -L -v ----- +--------------- +systemctl enable openqa-worker@1 +systemctl enable openqa-worker@2 +systemctl enable openqa-worker@3 +--------------- -As long as the SUT has access to external network, there should be -nonzero packet count in the forward chain between br0 and external -interface. +*Grant CAP_NET_ADMIN Capabilities to QEMU* -== VDE Based Network +In order to use TAP device which doesn’t exist on the system, it is required to set +CAP_NET_ADMIN capability on qemu binary file: -Virtual Distributed Ethernet provides a software switch that runs in -user space. It allows to connect several qemu instances without -affecting the system's network configuration. +[source,sh] +--------------- +zypper in libcap-progs +setcap CAP_NET_ADMIN=ep /usr/bin/qemu-system-x86_64 +--------------- -The openQA workers need a vde_switch instance running. The workers -reconfigure the switch as needed by the job. +*Configure Network Manager* -=== Basic, single machine tests +* Check the configuration for the _eth0_ interface: -To start with a basic configuration like qemu user mode networking, -create a machine with the following settings: +IMPORTANT: Ensure, that _eth0_ interface is configured in `/etc/sysconfig/network/ifcfg-eth0`. +Otherwise, wicked will not be able to bring up the interface on start and host will loose network +connection. -- +VDE_SOCKETDIR=/run/openqa+ -- +NICTYPE=vde+ -- +NICVLAN=0+ +[source,sh] +--------------- +# /etc/sysconfig/network/ifcfg-eth0 +BOOTPROTO='dhcp' +BROADCAST='' +ETHTOOL_OPTIONS='' +IPADDR='' +MTU='' +NAME='' +NETMASK='' +REMOTE_IPADDR='' +STARTMODE='auto' +DHCLIENT_SET_DEFAULT_ROUTE='yes' +--------------- -Start switch and user mode networking: +* Start _wicked_ as network service: + +Check the network service currently being used: [source,sh] --------------- -systemctl start openqa-vde_switch -systemctl start openqa-slirpvde +systemctl show -p Id network.service --------------- -With this setting all jobs on the same host would be in the same -network share the same SLIRP instance though. +If the result is different from `Id=wicked.service` (e.g. `NetworkManager.service`), +stop the network service: -=== Multi machine tests +[source,sh] +--------------- +systemctl stop network.service +systemctl disable network.service +--------------- -Create a machine like above but don't set NICVLAN. openQA will -dynamically allocate a VLAN number for all jobs that have -dependencies between each other. By default this VLAN is private and -has no internet access. To enable user mode networking set -`VDE_USE_SLIRP=1` on one of the machines. The worker running the job -on such a machine will start slirpvde and put it in the correct VLAN -then. +Then switch to wicked: -== Worker configuration +[source,sh] +--------------- +systemctl enable --force wicked +systemctl start wicked +--------------- -Requirements -```bash -zypper in openvswitch os-autoinst-openvswitch openQA-worker tunctl +* Bring up _br1_ interface: + +[source,sh] +--------------- +wicked ifup br1 +--------------- -systemctl enable SuSEfirewall2 # Needed to create NAT to outside network -systemctl enable openvswitch # Needed for network creation -systemctl enable os-autoinst-openvswitch # Needed to separate networks for parallel clusters -``` +* REBOOT -NOTE: In some cases (e.g. on Leap) can be needed to start the OpenvSwitch service before the Network service by modifying the OpenvSwitch service. For reference see https://en.opensuse.org/Portal:Wicked/OpenvSwitch#Wicked_0.6.23.2B[this]. +=== Debugging Open vSwitch Configuration +Boot sequence with wicked < 0.6.23: -The os-autoinst-openvswitch.service uses +br0+ by default. -Usually it's used by KVM, so we need to configure br1. +1. wicked - creates tap devices +2. openvswitch - creates the bridge +br1+, adds tap devices to it +3. wicked handles +br1+ as hotplugged device, assignd the IP 10.0.2.2 to it, updates SuSEfirewall +4. os-autoinst-openvswitch - installs openflow rules, handles vlan assignment -```bash -# /etc/sysconfig/os-autoinst-openvswitch -OS_AUTOINST_USE_BRIDGE=br1 -``` +Boot sequence with wicked 0.6.23 and newer: -For every MM worker you need tap device (tap0 tap1 tap2 ..) -```bash -# /etc/sysconfig/network/ifcfg-tap0 -BOOTPROTO='none' -IPADDR='' -NETMASK='' -PREFIXLEN='' -STARTMODE='auto' -TUNNEL='tap' -TUNNEL_SET_GROUP='nogroup' -TUNNEL_SET_OWNER='_openqa-worker' -``` +1. openvswitch +2. wicked - creates the bridge +br1+ and tap devices, add tap devices to the bridge, +3. SuSEfirewall +4. os-autoinst-openvswitch - installs openflow rules, handles vlan assignment -Add all tap devices to bridge config -```bash -# /etc/sysconfig/network/ifcfg-br1 -BOOTPROTO='static' -IPADDR='10.0.2.2/15' -STARTMODE='auto' -OVS_BRIDGE='yes' -OVS_BRIDGE_PORT_DEVICE_1='tap0' -OVS_BRIDGE_PORT_DEVICE_2='tap1' -OVS_BRIDGE_PORT_DEVICE_3='tap2' -``` -The IP 10.0.2.2 can also serve as a gateway to access outside -network. For this, a NAT between br1 and eth0 must be configured -with SuSEfirewall or iptables. +The configuration and operation can be checked by the following commands: -```bash -# /etc/sysconfig/SuSEfirewall2 -FW_ROUTE="yes" -FW_MASQUERADE="yes" -FW_DEV_INT="br1" -``` +[source,sh] +---- +ovs-vsctl show # shows the bridge br1, the tap devices are assigned to it +ovs-ofctl dump-flows br1 # shows the rules installed by os-autoinst-openvswitch in table=0 +---- -Tell workers to run also multi-machine jobs -```bash -# /etc/openqa/workers.ini -[global] -WORKER_CLASS = qemu_x86_64,tap -``` +* packets from tapX to br1 create additional rules in table=1 +* packets from br1 to tapX increase packet counts in table=1 +* empty output indicates a problem with os-autoinst-openvswitch service +* zero packet count or missing rules in table=1 indicate problem with tap devices -REBOOT +[source,sh] +---- +iptables -L -v +---- +As long as the SUT has access to external network, there should be +nonzero packet count in the forward chain between br1 and external +interface. -== GRE tunnels +== GRE Tunnels By default all multi-machine workers have to be on single physical machine. You can join multiple physical machines and its ovs bridges together by GRE tunnel. diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-4.5.1526551139.04373a1e/docs/UsersGuide.asciidoc new/openQA-4.5.1527149502.003be4d6/docs/UsersGuide.asciidoc --- old/openQA-4.5.1526551139.04373a1e/docs/UsersGuide.asciidoc 2018-05-17 11:58:59.000000000 +0200 +++ new/openQA-4.5.1527149502.003be4d6/docs/UsersGuide.asciidoc 2018-05-24 10:11:42.000000000 +0200 @@ -224,6 +224,11 @@ no longer interesting, but you still want to be able to re-submit jobs for a build and have existing jobs for the exact same build obsoleted. +_GROUP:: Job templates *not* matching the given group name are ignored. Does *not* + affect obsoletion behavior, so you might want to combine with `_NO_OBSOLETE`. + +_GROUP_ID:: Same as `_GROUP` but allows to specify the group directly by ID. + Example for `_DEPRIORITIZEBUILD` and `_DEPRIORITIZE_LIMIT`. [source,sh] diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-4.5.1526551139.04373a1e/etc/nginx/vhosts.d/openqa.conf new/openQA-4.5.1527149502.003be4d6/etc/nginx/vhosts.d/openqa.conf --- old/openQA-4.5.1526551139.04373a1e/etc/nginx/vhosts.d/openqa.conf 1970-01-01 01:00:00.000000000 +0100 +++ new/openQA-4.5.1527149502.003be4d6/etc/nginx/vhosts.d/openqa.conf 2018-05-24 10:11:42.000000000 +0200 @@ -0,0 +1,23 @@ +server { + listen 80; + server_name openqa.example.com; + root /usr/share/openqa/public; + client_max_body_size 0; + + location /api/v1/ws/ { + proxy_pass http://[::1]:9527; + proxy_http_version 1.1; + proxy_set_header Upgrade $http_upgrade; + proxy_set_header Connection "upgrade"; + proxy_set_header Host $host; + + rewrite /api/v1/ws/(.*) /ws/$1 break; + } + + location / { + proxy_set_header X-Forwarded-Host $host:$server_port; + proxy_set_header X-Forwarded-Server $host; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_pass "http://[::1]:9526";; + } +} diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-4.5.1526551139.04373a1e/lib/OpenQA/WebAPI/Controller/API/V1/Iso.pm new/openQA-4.5.1527149502.003be4d6/lib/OpenQA/WebAPI/Controller/API/V1/Iso.pm --- old/openQA-4.5.1526551139.04373a1e/lib/OpenQA/WebAPI/Controller/API/V1/Iso.pm 2018-05-17 11:58:59.000000000 +0200 +++ new/openQA-4.5.1527149502.003be4d6/lib/OpenQA/WebAPI/Controller/API/V1/Iso.pm 2018-05-24 10:11:42.000000000 +0200 @@ -182,8 +182,23 @@ # Allow a comma separated list of tests here; whitespaces allowed my @tests = $args->{TEST} ? split(/\s*,\s*/, $args->{TEST}) : (); + # allow filtering by group + my $group_id = delete $args->{_GROUP_ID}; + my $group_name = delete $args->{_GROUP}; + if (!defined $group_id && defined $group_name) { + my $groups = $self->db->resultset('JobGroups')->search({name => $group_name}); + my $group = $groups->next or return; + $group_id = $group->id; + } + for my $product (@products) { - my @templates = $product->job_templates; + # find job templates + my $templates = $product->job_templates; + if (defined $group_id) { + $templates = $templates->search({group_id => $group_id}); + } + my @templates = $templates->all; + unless (@templates) { carp "no templates found for " . join('-', map { $args->{$_} } qw(DISTRI VERSION FLAVOR ARCH)); } @@ -216,7 +231,7 @@ next if $_ eq 'TEST' || $_ eq 'MACHINE'; $settings{uc $_} = $args->{$_}; } - # Makes sure tha the DISTRI is lowercase + # make sure that the DISTRI is lowercase $settings{DISTRI} = lc($settings{DISTRI}); $settings{PRIO} = $job_template->prio; diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-4.5.1526551139.04373a1e/lib/OpenQA/WebAPI/Controller/Step.pm new/openQA-4.5.1527149502.003be4d6/lib/OpenQA/WebAPI/Controller/Step.pm --- old/openQA-4.5.1526551139.04373a1e/lib/OpenQA/WebAPI/Controller/Step.pm 2018-05-17 11:58:59.000000000 +0200 +++ new/openQA-4.5.1527149502.003be4d6/lib/OpenQA/WebAPI/Controller/Step.pm 2018-05-24 10:11:42.000000000 +0200 @@ -652,6 +652,7 @@ $self->stash('frametime', $module_detail->{frametime}); $self->stash('default_label', $primary_match ? $primary_match->{label} : 'Screenshot'); $self->stash('needles_by_tag', \%needles_by_tag); + $self->stash('tag_count', scalar %needles_by_tag); return $self->render('step/viewimg'); } diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-4.5.1526551139.04373a1e/lib/OpenQA/WebAPI/Plugin/Gru.pm new/openQA-4.5.1527149502.003be4d6/lib/OpenQA/WebAPI/Plugin/Gru.pm --- old/openQA-4.5.1526551139.04373a1e/lib/OpenQA/WebAPI/Plugin/Gru.pm 2018-05-17 11:58:59.000000000 +0200 +++ new/openQA-4.5.1527149502.003be4d6/lib/OpenQA/WebAPI/Plugin/Gru.pm 2018-05-24 10:11:42.000000000 +0200 @@ -122,6 +122,7 @@ use Mojo::Base 'Mojolicious::Command'; use Mojo::Pg; use Minion::Command::minion::job; +use OpenQA::Utils 'log_error'; has usage => "usage: $0 gru [-o]\n"; has description => 'Run a gru to process jobs - give -o to exit _o_nce everything is done'; @@ -164,6 +165,7 @@ }; if (defined $err) { + log_error("Gru command issue: $err"); $self->fail_gru($job->info->{notes}{gru_id} => $err) if $job->fail({(output => $buffer) x !!(defined $buffer), error => $err}) && exists $job->info->{notes}{gru_id}; diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-4.5.1526551139.04373a1e/lib/OpenQA/WebAPI/Plugin/Helpers.pm new/openQA-4.5.1527149502.003be4d6/lib/OpenQA/WebAPI/Plugin/Helpers.pm --- old/openQA-4.5.1526551139.04373a1e/lib/OpenQA/WebAPI/Plugin/Helpers.pm 2018-05-17 11:58:59.000000000 +0200 +++ new/openQA-4.5.1527149502.003be4d6/lib/OpenQA/WebAPI/Plugin/Helpers.pm 2018-05-24 10:11:42.000000000 +0200 @@ -129,7 +129,7 @@ $crumbs .= $c->link_to($overview_url => (class => 'dropdown-item') => sub { '<i class="fas fa-arrow-right"></i> ' . $overview_text }); $crumbs .= "</li>"; - $crumbs .= "\n<li role='separator' class='divider'></li>\n"; + $crumbs .= "\n<li role='separator' class='dropdown-divider'></li>\n"; return Mojo::ByteStream->new($crumbs); } return; diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-4.5.1526551139.04373a1e/t/22-dashboard.t new/openQA-4.5.1527149502.003be4d6/t/22-dashboard.t --- old/openQA-4.5.1526551139.04373a1e/t/22-dashboard.t 2018-05-17 11:58:59.000000000 +0200 +++ new/openQA-4.5.1527149502.003be4d6/t/22-dashboard.t 2018-05-24 10:11:42.000000000 +0200 @@ -151,7 +151,7 @@ # links are correct my @urls = $get->tx->res->dom->find('h2 a, .row a')->map('attr', 'href')->each; for my $url (@urls) { - next if ($url =~ /^#/); + next if ($url =~ /^#/ || $url =~ /^.*\:\//); $get = $t->get_ok($url)->status_is(200); } diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-4.5.1526551139.04373a1e/t/api/02-iso.t new/openQA-4.5.1527149502.003be4d6/t/api/02-iso.t --- old/openQA-4.5.1526551139.04373a1e/t/api/02-iso.t 2018-05-17 11:58:59.000000000 +0200 +++ new/openQA-4.5.1527149502.003be4d6/t/api/02-iso.t 2018-05-24 10:11:42.000000000 +0200 @@ -138,6 +138,60 @@ # later on is found as important and handled accordingly $t->app->db->resultset("Jobs")->find(99928)->comments->create({text => 'any text', user_id => 99901}); +subtest 'group filter' => sub { + # add a job template for group 1002 + my $job_template = $t->app->db->resultset('JobTemplates')->create( + { + machine => {name => '64bit'}, + test_suite => {name => 'textmode-2'}, + prio => 42, + group_id => 1002, + product_id => 1, + }); + + my $res = schedule_iso( + { + ISO => $iso, + DISTRI => 'opensuse', + VERSION => '13.1', + FLAVOR => 'DVD', + ARCH => 'i586', + BUILD => '0091', + PRECEDENCE => 'original', + _GROUP => 'invalid group name', + }); + is($res->json->{count}, 0, 'no jobs created if group invalid'); + + $res = schedule_iso( + { + ISO => $iso, + DISTRI => 'opensuse', + VERSION => '13.1', + FLAVOR => 'DVD', + ARCH => 'i586', + BUILD => '0091', + PRECEDENCE => 'original', + _GROUP => 'opensuse test', + }); + is($res->json->{count}, 1, 'only one job created due to group filter'); + + $res = schedule_iso( + { + ISO => $iso, + DISTRI => 'opensuse', + VERSION => '13.1', + FLAVOR => 'DVD', + ARCH => 'i586', + BUILD => '0091', + PRECEDENCE => 'original', + _GROUP_ID => '1002', + }); + is($res->json->{count}, 1, 'only one job created due to group filter (by ID)'); + + # delete job template again so the remaining tests are unaffected + $job_template->delete; +}; + # schedule the iso, this should not actually be possible. Only isos # with different name should result in new tests... my $res = schedule_iso( diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-4.5.1526551139.04373a1e/t/fixtures/04-products.pl new/openQA-4.5.1527149502.003be4d6/t/fixtures/04-products.pl --- old/openQA-4.5.1526551139.04373a1e/t/fixtures/04-products.pl 2018-05-17 11:58:59.000000000 +0200 +++ new/openQA-4.5.1527149502.003be4d6/t/fixtures/04-products.pl 2018-05-24 10:11:42.000000000 +0200 @@ -71,6 +71,7 @@ ], }, Products => { + id => 1, name => '', distri => 'opensuse', version => '13.1', diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-4.5.1526551139.04373a1e/t/ui/16-tests_job_next_previous.t new/openQA-4.5.1527149502.003be4d6/t/ui/16-tests_job_next_previous.t --- old/openQA-4.5.1526551139.04373a1e/t/ui/16-tests_job_next_previous.t 2018-05-17 11:58:59.000000000 +0200 +++ new/openQA-4.5.1527149502.003be4d6/t/ui/16-tests_job_next_previous.t 2018-05-24 10:11:42.000000000 +0200 @@ -163,5 +163,38 @@ is((shift @tds)->get_text(), '0091', 'build of 99928 is 0091'); is((shift @tds)->get_text(), 'Not yet: scheduled', '99928 is not yet finished'); +# check job next and previous under tests/latest route +$driver->get('/tests/latest'); +$driver->find_element_by_link_text('Next & previous results')->click(); +wait_for_ajax(); +$driver->find_element_by_class('dataTables_wrapper'); +is(scalar @{$driver->find_elements('#job_next_previous_table tbody tr', 'css')}, + 1, 'job next and previous of the latest job - 99981'); +my $job99981 = $driver->find_element('#job_next_previous_table #job_result_99981'); +@tds = $driver->find_child_elements($job99981, 'td'); +is((shift @tds)->get_text(), 'C&L', '99981 is current and the latest job'); + +# check job next and previous with scenario latest url +$driver->get('/tests/99945'); +$driver->find_element_by_link_text('Next & previous results')->click(); +wait_for_ajax(); +$driver->find_element_by_class('dataTables_wrapper'); +$driver->find_element_by_link_text('latest job for this scenario')->click(); +my $scenario_latest_url = $driver->get_current_url(); +like($scenario_latest_url, qr/latest?/, 'latest scenario URL includes latest'); +like($scenario_latest_url, qr/arch=i586/, 'latest scenario URL includes architecture'); +like($scenario_latest_url, qr/flavor=DVD/, 'latest scenario URL includes flavour'); +like($scenario_latest_url, qr/test=textmode/, 'latest scenario URL includes test'); +like($scenario_latest_url, qr/version=13.1/, 'latest scenario URL includes version'); +like($scenario_latest_url, qr/machine=32bit/, 'latest scenario URL includes machine'); +like($scenario_latest_url, qr/distri=opensuse/, 'latest scenario URL includes distri'); +$driver->find_element_by_link_text('Next & previous results')->click(); +wait_for_ajax(); +$driver->find_element_by_class('dataTables_wrapper'); +is(scalar @{$driver->find_elements('#job_next_previous_table tbody tr', 'css')}, 4, 'job next and previous of 99947'); +$job99947 = $driver->find_element('#job_next_previous_table #job_result_99947'); +@tds = $driver->find_child_elements($job99947, 'td'); +is((shift @tds)->get_text(), 'C&L', '99947 is current and the latest job'); + kill_driver(); done_testing(); diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-4.5.1526551139.04373a1e/templates/main/group_overview.html.ep new/openQA-4.5.1527149502.003be4d6/templates/main/group_overview.html.ep --- old/openQA-4.5.1526551139.04373a1e/templates/main/group_overview.html.ep 2018-05-17 11:58:59.000000000 +0200 +++ new/openQA-4.5.1527149502.003be4d6/templates/main/group_overview.html.ep 2018-05-24 10:11:42.000000000 +0200 @@ -7,6 +7,7 @@ $('.timeago').timeago(); hideNavbar(<%= $fullscreen %>); autoRefresh(<%= $fullscreen %>, <%= $interval %>); + alignBuildLabels(); % end % if (is_admin) { diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-4.5.1526551139.04373a1e/templates/step/viewimg.html.ep new/openQA-4.5.1527149502.003be4d6/templates/step/viewimg.html.ep --- old/openQA-4.5.1526551139.04373a1e/templates/step/viewimg.html.ep 2018-05-17 11:58:59.000000000 +0200 +++ new/openQA-4.5.1527149502.003be4d6/templates/step/viewimg.html.ep 2018-05-24 10:11:42.000000000 +0200 @@ -5,7 +5,15 @@ <button type="button" class="btn btn-light btn-left" onclick="setNeedle($([]));" id="screenshot_button"> <i class="fas fa-image" title="Show only screenshot"></i> </button> - <button type="button" class="btn btn-light dropdown-toggle" aria-haspopup="true" aria-expanded="false" data-toggle="dropdown" id="candidatesMenu"> + <button id="candidatesMenu" type="button" aria-haspopup="true" aria-expanded="false" data-toggle="dropdown" + % if ($tag_count) { + class="btn btn-light dropdown-toggle" + % } + % else { + class="btn" + disabled + % } + > <span id="current_needle_label"> %= $default_label </span> diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-4.5.1526551139.04373a1e/templates/test/job_next_previous.html.ep new/openQA-4.5.1527149502.003be4d6/templates/test/job_next_previous.html.ep --- old/openQA-4.5.1526551139.04373a1e/templates/test/job_next_previous.html.ep 2018-05-17 11:58:59.000000000 +0200 +++ new/openQA-4.5.1527149502.003be4d6/templates/test/job_next_previous.html.ep 2018-05-24 10:11:42.000000000 +0200 @@ -8,7 +8,7 @@ <div class="row" style="margin-bottom: 10px; margin-top: 10px;"> <div class="col-12 col-md-8 my-auto" id="scenario"> - <div class="h5">Next & previous results for <i><%= $scenario %></i></div> + <div class="h5">Next & previous results for <i><%= $scenario %></i> (<%= link_to('latest job for this scenario' => url_for('latest')->query($job->scenario_hash)) %>)</div> </div> <div class="col-6 col-md-4"> <div class="float-right"> @@ -46,7 +46,7 @@ </div> <div> <table id="job_next_previous_table" class="overview table table-striped no-wrap" style="width: 100%" - data-ajax-url="<%= url_for('job_next_previous_ajax') %>"> + data-ajax-url="<%= url_for('job_next_previous_ajax', testid => $testid) %>"> <thead> <th></th> <th>Result</th>
