Script 'mail_helper' called by obssrc Hello community, here is the log from the commit of package openQA for openSUSE:Factory checked in at 2026-07-17 01:45:19 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/openQA (Old) and /work/SRC/openSUSE:Factory/.openQA.new.24530 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "openQA" Fri Jul 17 01:45:19 2026 rev:867 rq:1366240 version:5.1784178984.04ce3617 Changes: -------- --- /work/SRC/openSUSE:Factory/openQA/openQA.changes 2026-07-12 16:21:41.153078479 +0200 +++ /work/SRC/openSUSE:Factory/.openQA.new.24530/openQA.changes 2026-07-17 01:45:49.742000270 +0200 @@ -1,0 +2,26 @@ +Thu Jul 16 05:16:53 UTC 2026 - [email protected] + +- Update to version 5.1784178984.04ce3617: + * chore(deps): Dependency cron 2026-07-16 + * feat(job-restart): Allow restarting parent job skipping OK children + * test: Fix check `Consistent Test::More call` to exclude comments + * feat(overview-page): Reload automatically after adding comments + * test: Unify raw port socket reuse across create_* helpers + * test: reuse reserved socket in create_webapi + * feat(overview-page): Allow restarting jobs with advanced options + * feat(overview-page): Improve displaying number of restarts + * fix(overview-page): Remove surplus spaces after buttons + * chore(deps): Dependency cron 2026-07-14 + * chore: Update `Perl::Tidy` to 20260705 + * feat: Change worker job lock limit to be in line with config example + * docs: Remove example from `LOCK_NAME` worker setting + * fix(mergify): prevent endless dependabot PR recreation loops + * feat: optionally fail ISO API if no jobs created + * test(worker): Cover rejecting job when acquiring job lock failed + * refactor: Rename `OpenQA::CacheService::DB` + * feat(worker): Re-evaluate unavailability due to job lock quickly + * feat(worker): Allow configuring lock to throttle job execution + * feat(worker): Add local locking on worker-instance scope + * refactor: Move database setup of cache service to separate module + +------------------------------------------------------------------- Old: ---- openQA-5.1783672871.e689eb3c.obscpio New: ---- openQA-5.1784178984.04ce3617.obscpio ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ openQA-client-test.spec ++++++ --- /var/tmp/diff_new_pack.kQxmNI/_old 2026-07-17 01:45:50.754034418 +0200 +++ /var/tmp/diff_new_pack.kQxmNI/_new 2026-07-17 01:45:50.754034418 +0200 @@ -18,7 +18,7 @@ %define short_name openQA-client Name: %{short_name}-test -Version: 5.1783672871.e689eb3c +Version: 5.1784178984.04ce3617 Release: 0 Summary: Test package for %{short_name} License: GPL-2.0-or-later ++++++ openQA-devel-test.spec ++++++ --- /var/tmp/diff_new_pack.kQxmNI/_old 2026-07-17 01:45:50.798035902 +0200 +++ /var/tmp/diff_new_pack.kQxmNI/_new 2026-07-17 01:45:50.802036037 +0200 @@ -18,7 +18,7 @@ %define short_name openQA-devel Name: %{short_name}-test -Version: 5.1783672871.e689eb3c +Version: 5.1784178984.04ce3617 Release: 0 Summary: Test package for %{short_name} License: GPL-2.0-or-later ++++++ openQA-test.spec ++++++ --- /var/tmp/diff_new_pack.kQxmNI/_old 2026-07-17 01:45:50.838037252 +0200 +++ /var/tmp/diff_new_pack.kQxmNI/_new 2026-07-17 01:45:50.838037252 +0200 @@ -18,7 +18,7 @@ %define short_name openQA Name: %{short_name}-test -Version: 5.1783672871.e689eb3c +Version: 5.1784178984.04ce3617 Release: 0 Summary: Test package for openQA License: GPL-2.0-or-later ++++++ openQA-worker-test.spec ++++++ --- /var/tmp/diff_new_pack.kQxmNI/_old 2026-07-17 01:45:50.886038872 +0200 +++ /var/tmp/diff_new_pack.kQxmNI/_new 2026-07-17 01:45:50.890039007 +0200 @@ -18,7 +18,7 @@ %define short_name openQA-worker Name: %{short_name}-test -Version: 5.1783672871.e689eb3c +Version: 5.1784178984.04ce3617 Release: 0 Summary: Test package for %{short_name} License: GPL-2.0-or-later ++++++ openQA.spec ++++++ --- /var/tmp/diff_new_pack.kQxmNI/_old 2026-07-17 01:45:50.930040356 +0200 +++ /var/tmp/diff_new_pack.kQxmNI/_new 2026-07-17 01:45:50.934040491 +0200 @@ -104,7 +104,7 @@ %define devel_requires %devel_no_selenium_requires chromedriver Name: openQA -Version: 5.1783672871.e689eb3c +Version: 5.1784178984.04ce3617 Release: 0 Summary: Framework for automated system-level testing (web-frontend, scheduler and tools) Group: Development/Tools/Other ++++++ _service ++++++ --- /var/tmp/diff_new_pack.kQxmNI/_old 2026-07-17 01:45:50.990042381 +0200 +++ /var/tmp/diff_new_pack.kQxmNI/_new 2026-07-17 01:45:50.998042651 +0200 @@ -1,4 +1,4 @@ -<?xml version="1.0"?> +<?xml version='1.0' encoding='utf-8'?> <services> <service name="obs_scm" mode="disabled"> <param name="extract">dist/rpm/openQA.spec</param> @@ -15,8 +15,8 @@ <param name="changesgenerate">enable</param> <param name="changesauthor">[email protected]</param> </service> - <service name="set_version" mode="disabled"/> - <service name="tar" mode="buildtime"/> + <service name="set_version" mode="disabled" /> + <service name="tar" mode="buildtime" /> <service name="recompress" mode="buildtime"> <param name="compression">xz</param> <param name="file">*.tar</param> @@ -27,4 +27,5 @@ <param name="source-offset">1000</param> </service> </services> +(No newline at EOF) ++++++ openQA-5.1783672871.e689eb3c.obscpio -> openQA-5.1784178984.04ce3617.obscpio ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-5.1783672871.e689eb3c/.mergify.yml new/openQA-5.1784178984.04ce3617/.mergify.yml --- old/openQA-5.1783672871.e689eb3c/.mergify.yml 2026-07-10 10:41:11.000000000 +0200 +++ new/openQA-5.1784178984.04ce3617/.mergify.yml 2026-07-16 07:16:24.000000000 +0200 @@ -41,23 +41,11 @@ conditions: - and: *base_checks - and: - # mergify config checks needs at least two rules in "and" so we repeat - # one from the base checks - - base=main + - base=master - author=dependabot[bot] - - label=waited actions: merge: method: squash - - name: Wait for some days before validating merge - actions: - label: - add: [ waited ] - remove: [ waiting ] - conditions: - - and: - - updated-at<4 days ago - - author=dependabot[bot] queue_rules: [] merge_queue: status_comments: none diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-5.1783672871.e689eb3c/assets/javascripts/openqa.js new/openQA-5.1784178984.04ce3617/assets/javascripts/openqa.js --- old/openQA-5.1783672871.e689eb3c/assets/javascripts/openqa.js 2026-07-10 10:41:11.000000000 +0200 +++ new/openQA-5.1784178984.04ce3617/assets/javascripts/openqa.js 2026-07-16 07:16:24.000000000 +0200 @@ -193,13 +193,57 @@ } const container = document.createElement('div'); if (hasResponse && responseJSON.enforceable && retryFunction) { + let parentId; + if (Array.isArray(errors)) { + for (const error of errors) { + const match = error.match(/Direct parent (\d+) needs to be cloned as well/); + if (match) { + parentId = match[1]; + break; + } + } + } + const button = document.createElement('button'); button.onclick = retryFunction; button.className = 'btn btn-danger force-restart'; - button.style.float = 'right'; button.appendChild(document.createTextNode('Force restart')); - container.appendChild(button); + const btnGroup = document.createElement('div'); + btnGroup.className = 'btn-group'; + btnGroup.style.float = 'right'; + btnGroup.appendChild(button); + + if (parentId) { + const dropdownToggle = document.createElement('button'); + dropdownToggle.type = 'button'; + dropdownToggle.className = 'btn btn-danger dropdown-toggle dropdown-toggle-split'; + dropdownToggle.setAttribute('data-bs-toggle', 'dropdown'); + dropdownToggle.setAttribute('aria-expanded', 'false'); + + const srOnly = document.createElement('span'); + srOnly.className = 'visually-hidden sr-only'; + srOnly.appendChild(document.createTextNode('Toggle Dropdown')); + dropdownToggle.appendChild(srOnly); + btnGroup.appendChild(dropdownToggle); + + const dropdownMenu = document.createElement('div'); + dropdownMenu.className = 'dropdown-menu dropdown-menu-end'; + + const item = document.createElement('a'); + item.className = 'dropdown-item restart-parent-skip-ok'; + item.href = '#'; + item.appendChild(document.createTextNode('Restart parent job skipping passed/softfailed children')); + item.onclick = function (e) { + e.preventDefault(); + restartJob(urlWithBase(`/api/v1/jobs/${parentId}/restart?skip_ok_result_children=1`), parentId); + }; + dropdownMenu.appendChild(item); + btnGroup.appendChild(dropdownMenu); + } + + container.appendChild(btnGroup); } + if (hasWarnings) { container.appendChild(document.createTextNode('Warnings occurred when restarting jobs:')); container.appendChild(renderList(warnings)); @@ -615,7 +659,7 @@ let html = ''; if (row.restarts > 0) { const s = row.restarts > 1 ? 's' : ''; - html += ` <span class="restarts" title="Restarted ${row.restarts} time${s}"><i class="fa fa-undo"></i> ${row.restarts}</span>`; + html += ` <span class="restarts" title="Restarted ${row.restarts} time${s}"><i class="action fa fa-clock-rotate-left"></i> ${row.restarts}</span>`; } if (row.clone) { html += ` <a href="${urlWithBase('/tests/' + row.clone)}">(restarted)</a>`; diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-5.1783672871.e689eb3c/assets/javascripts/overview.js new/openQA-5.1784178984.04ce3617/assets/javascripts/overview.js --- old/openQA-5.1783672871.e689eb3c/assets/javascripts/overview.js 2026-07-10 10:41:11.000000000 +0200 +++ new/openQA-5.1784178984.04ce3617/assets/javascripts/overview.js 2026-07-16 07:16:24.000000000 +0200 @@ -259,6 +259,13 @@ parentChild[i].addEventListener('mouseover', highlightDeps); parentChild[i].addEventListener('mouseout', unhighlightDeps); } + + const cookies = new URLSearchParams(document.cookie); + const flash = cookies.get('flash'); + if (flash) { + addFlash(...flash.split(':', 2)); + document.cookie = 'flash=; expires=Thu, 01 Jan 1970 00:00:00 UTC;'; + } } function setCheckboxStatesForFlags(containerId, flags) { @@ -339,11 +346,9 @@ fetchWithCSRF(apiUrl, {method: 'POST', body: new FormData(form)}) .then(response => { if (!response.ok) throw `Server returned ${response.status}: ${response.statusText}`; - addFlash( - 'info', - 'The comments have been created. <a href="javascript: location.reload()">Reload</a> the page to show changes.' - ); + document.cookie = 'flash=info:The comments have been created.'; done(); + location.reload(); }) .catch(error => { addFlash('danger', `The comments could not be added: ${error}`); diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-5.1783672871.e689eb3c/cpanfile new/openQA-5.1784178984.04ce3617/cpanfile --- old/openQA-5.1783672871.e689eb3c/cpanfile 2026-07-10 10:41:11.000000000 +0200 +++ new/openQA-5.1784178984.04ce3617/cpanfile 2026-07-16 07:16:24.000000000 +0200 @@ -121,7 +121,7 @@ requires 'Code::TidyAll'; requires 'Perl::Critic', '>= 1.156.0'; requires 'Perl::Critic::Community'; - requires 'Perl::Tidy', '== 20260204.0.0'; + requires 'Perl::Tidy', '== 20260705'; requires 'Pod::Markdown'; requires 'TAP::Harness::JUnit'; requires 'Test::CheckGitStatus'; diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-5.1783672871.e689eb3c/dependencies.yaml new/openQA-5.1784178984.04ce3617/dependencies.yaml --- old/openQA-5.1783672871.e689eb3c/dependencies.yaml 2026-07-10 10:41:11.000000000 +0200 +++ new/openQA-5.1784178984.04ce3617/dependencies.yaml 2026-07-16 07:16:24.000000000 +0200 @@ -88,7 +88,7 @@ xorg-x11-fonts: pandoc: # for generating HTML documentation python3-weasyprint: # for generating PDF documentation - perl(Perl::Tidy): '== 20260204.0.0' + perl(Perl::Tidy): '== 20260705' perl(Test::CheckGitStatus): perl(TAP::Harness::JUnit): # for JUnit-compatible test results diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-5.1783672871.e689eb3c/etc/openqa/workers.ini new/openQA-5.1784178984.04ce3617/etc/openqa/workers.ini --- old/openQA-5.1783672871.e689eb3c/etc/openqa/workers.ini 2026-07-10 10:41:11.000000000 +0200 +++ new/openQA-5.1784178984.04ce3617/etc/openqa/workers.ini 2026-07-16 07:16:24.000000000 +0200 @@ -85,6 +85,27 @@ # Set to 0 to disable. #CRITICAL_LOAD_AVG_THRESHOLD = 40 +# Let the worker acquire a lock with the specified name before accepting a job. +# The lock will be released after the job is done or after the specified +# expiration. The expiration is effective when isotovideo has been started. The +# worker will reject the job if it cannot acquire the lock. The number of local +# worker instances that can acquire the lock can be limited. +# By default no lock is acquired. The purpose of configuring a lock is +# throttling the startup of jobs on a particular worker host so the setting +# CRITICAL_LOAD_AVG_THRESHOLD is more effective. (Otherwise many worker +# instances might still all start newly scheduled jobs at the same time. With +# the example config below, only 5 worker instances would be able to start newly +# scheduled jobs at the same time. More worker instances would only start +# further jobs after 30 seconds - unless the load exceeds the configured +# threshold then.) +# Workers that could not acquire a lock will temporarily show as unavailable +# with an according note. This prevents the scheduler from immediately assigning +# a job on these slots again (increasing the chances it will spread future +# job assignments across different/other worker hosts). +#LOCK_NAME = +#LOCK_EXPIRATION = 30 +#LOCK_LIMIT = 5 + # If IPMI_HOSTNAME, IPMI_USER and IPMI_PASSWORD is set the worker will periodically # ensure that the IPMI controlled SUT is powered off when no job is running. # The default period is 300 seconds. diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-5.1783672871.e689eb3c/lib/OpenQA/CacheService/DB.pm new/openQA-5.1784178984.04ce3617/lib/OpenQA/CacheService/DB.pm --- old/openQA-5.1783672871.e689eb3c/lib/OpenQA/CacheService/DB.pm 1970-01-01 01:00:00.000000000 +0100 +++ new/openQA-5.1784178984.04ce3617/lib/OpenQA/CacheService/DB.pm 2026-07-16 07:16:24.000000000 +0200 @@ -0,0 +1,43 @@ +# Copyright SUSE LLC +# SPDX-License-Identifier: GPL-2.0-or-later + +package OpenQA::CacheService::DB; +use Mojo::Base -strict, -signatures; + +use Mojo::SQLite; +use Mojo::File 'path'; + +# Default to 10 minutes +use constant SQLITE_BUSY_TIMEOUT => $ENV{OPENQA_SQLITE_BUSY_TIMEOUT} // 600000; + +# Defaults to 1 minute +use constant SQLITE_SLOW_QUERY => $ENV{OPENQA_SQLITE_SLOW_QUERY} // 60000; + +sub _configure_sqlite_database ($log, $sqlite, $dbh) { + # default to using DELETE journaling mode to avoid database corruption seen in production (see poo#67000) + # check out https://www.sqlite.org/pragma.html#pragma_journal_mode for possible values + my $sqlite_mode = uc($ENV{OPENQA_CACHE_SERVICE_SQLITE_JOURNAL_MODE} || 'DELETE'); + $dbh->sqlite_busy_timeout(SQLITE_BUSY_TIMEOUT); + $dbh->do("pragma journal_mode=$sqlite_mode"); + $dbh->do('pragma synchronous=NORMAL') if $sqlite_mode eq 'WAL'; + + # Log slow queries + $dbh->sqlite_profile( + sub ($statement, $elapsed, @) { + $log->info(qq{Slow SQLite query: "$statement" -> ${elapsed}ms}) if $elapsed > SQLITE_SLOW_QUERY; + }); +} + +sub location ($global_settings) { $ENV{OPENQA_CACHE_DIR} || $global_settings->{CACHEDIRECTORY} } + +sub db_file ($location, $file_name = 'cache.sqlite') { path($location, $file_name) } + +sub open_sqlite_database ($log, $db_file) { + my $sqlite = Mojo::SQLite->new->from_string("file://$db_file?no_wal=1"); + $sqlite->on(connection => sub ($sqlite, $dbh) { _configure_sqlite_database($log, $sqlite, $dbh) }); + return $sqlite; +} + +sub open_default_sqlite_database ($log, $location) { open_sqlite_database($location, db_file($location)) } + +1; diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-5.1783672871.e689eb3c/lib/OpenQA/CacheService.pm new/openQA-5.1784178984.04ce3617/lib/OpenQA/CacheService.pm --- old/openQA-5.1783672871.e689eb3c/lib/OpenQA/CacheService.pm 2026-07-10 10:41:11.000000000 +0200 +++ new/openQA-5.1784178984.04ce3617/lib/OpenQA/CacheService.pm 2026-07-16 07:16:24.000000000 +0200 @@ -4,44 +4,16 @@ package OpenQA::CacheService; use Mojo::Base 'Mojolicious', -signatures; -use Mojo::SQLite; -use Mojo::File 'path'; use Time::Seconds; use OpenQA::Worker::Settings; +use OpenQA::CacheService::DB; use OpenQA::CacheService::Model::Cache; use OpenQA::CacheService::Model::Downloads; use constant DEFAULT_MINION_WORKERS => 5; -# Default to 10 minutes -use constant SQLITE_BUSY_TIMEOUT => $ENV{OPENQA_SQLITE_BUSY_TIMEOUT} // 600000; - -# Defaults to 1 minute -use constant SQLITE_SLOW_QUERY => $ENV{OPENQA_SQLITE_SLOW_QUERY} // 60000; - has exit_code => undef; -sub _configure_sqlite_database ($self, $sqlite, $dbh) { - # default to using DELETE journaling mode to avoid database corruption seen in production (see poo#67000) - # check out https://www.sqlite.org/pragma.html#pragma_journal_mode for possible values - my $sqlite_mode = uc($ENV{OPENQA_CACHE_SERVICE_SQLITE_JOURNAL_MODE} || 'DELETE'); - $dbh->sqlite_busy_timeout(SQLITE_BUSY_TIMEOUT); - $dbh->do("pragma journal_mode=$sqlite_mode"); - $dbh->do('pragma synchronous=NORMAL') if $sqlite_mode eq 'WAL'; - - # Log slow queries - $dbh->sqlite_profile( - sub ($statement, $elapsed, @) { - $self->log->info(qq{Slow SQLite query: "$statement" -> ${elapsed}ms}) if $elapsed > SQLITE_SLOW_QUERY; - }); -} - -sub _open_sqlite_database ($self, $db_file) { - my $sqlite = Mojo::SQLite->new->from_string("file://$db_file?no_wal=1"); - $sqlite->on(connection => sub ($sqlite, $dbh) { $self->_configure_sqlite_database($sqlite, $dbh) }); - return $sqlite; -} - sub startup ($self) { $self->moniker('openqa_cache_service'); $self->defaults(appname => 'openQA Cache Service'); @@ -68,7 +40,7 @@ # Worker settings my $global_settings = OpenQA::Worker::Settings->new->global_settings; - my $location = $ENV{OPENQA_CACHE_DIR} || $global_settings->{CACHEDIRECTORY}; + my $location = OpenQA::CacheService::DB::location($global_settings); die "Cache directory unspecified. Set environment variable 'OPENQA_CACHE_DIR' or config variable 'CACHEDIRECTORY'\n" unless defined $location; my $limit = $global_settings->{CACHELIMIT}; @@ -86,7 +58,7 @@ $log->unsubscribe('message') if $ENV{OPENQA_CACHE_SERVICE_QUIET}; # Increase busy timeout to 5 minutes - my $sqlite = $self->_open_sqlite_database(path($location, 'cache.sqlite')); + my $sqlite = OpenQA::CacheService::DB::open_default_sqlite_database($self->log, $location); $sqlite->migrations->name('cache_service')->from_data; my @cache_params = (sqlite => $sqlite, log => $self->log, location => $location); diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-5.1783672871.e689eb3c/lib/OpenQA/Schema/Result/ScheduledProducts.pm new/openQA-5.1784178984.04ce3617/lib/OpenQA/Schema/Result/ScheduledProducts.pm --- old/openQA-5.1783672871.e689eb3c/lib/OpenQA/Schema/Result/ScheduledProducts.pm 2026-07-10 10:41:11.000000000 +0200 +++ new/openQA-5.1784178984.04ce3617/lib/OpenQA/Schema/Result/ScheduledProducts.pm 2026-07-16 07:16:24.000000000 +0200 @@ -26,6 +26,7 @@ ADDED => 'added', # no jobs have been created yet SCHEDULING => 'scheduling', # jobs are being created CANCELLING => 'cancelling', # jobs are being cancelled (so far only possible as reaction to webhook event) + FATAL_ERROR => 'fatal_error', }; __PACKAGE__->table('scheduled_products'); @@ -109,7 +110,7 @@ deflate => sub { encode_json(shift) }, }); -our @EXPORT_OK = qw(ADDED SCHEDULING SCHEDULED CANCELLING CANCELLED); +our @EXPORT_OK = qw(ADDED SCHEDULING SCHEDULED CANCELLING CANCELLED FATAL_ERROR); our %EXPORT_TAGS = (STATE => [@EXPORT_OK]); sub sqlt_deploy_hook ($self, $sqlt_table, @) { @@ -215,13 +216,14 @@ } sub set_done ($self, $result) { + my $status = ref $result eq 'HASH' ? (delete $result->{status} // SCHEDULED) : SCHEDULED; # set the status to be either … if ($self->_update_status_if(CANCELLED, status => CANCELLING)) { $self->update({results => $result}); $self->cancel; # … CANCELLED if meanwhile CANCELLING and invoke cancel again (as it backed off) } else { - $self->update({status => SCHEDULED, results => $result}); # … SCHEDULED if remained SCHEDULING + $self->update({status => $status, results => $result}); # … SCHEDULED if remained SCHEDULING $self->report_status_to_github; } } @@ -333,6 +335,7 @@ # read arguments for deprioritization and obsoleten my $deprioritize = delete $args->{_DEPRIORITIZEBUILD} // 0; + my $fail_if_no_jobs = delete $args->{_FAIL_IF_NO_JOBS} // 0; my $deprioritize_limit = delete $args->{_DEPRIORITIZE_LIMIT}; my $obsolete = delete $args->{_OBSOLETE} // 0; my $onlysame = delete $args->{_ONLY_OBSOLETE_SAME_BUILD} // 0; @@ -360,8 +363,10 @@ else { $result = $self->_generate_jobs($args, \@notes, $skip_chained_deps, $include_children); } - return {error => $result->{error_message}, error_code => $result->{error_code} // HTTP_BAD_REQUEST} - if defined $result->{error_message}; + if (defined $result->{error_message}) { + return $self->_fail_scheduling($result->{error_message}) if $fail_if_no_jobs; + return {error => $result->{error_message}, error_code => $result->{error_code} // HTTP_BAD_REQUEST}; + } my $jobs = $result->{settings_result}; # take some attributes from the first job to guess what old jobs to cancel # note: We should have distri object that decides which attributes are relevant here. @@ -424,6 +429,9 @@ OpenQA::Scheduler::Client->singleton->wakeup; + return $self->_fail_scheduling('No jobs scheduled.') + if $fail_if_no_jobs && !@successful_job_ids; + my %results = ( successful_job_ids => \@successful_job_ids, failed_job_info => \@failed_job_info, @@ -432,6 +440,14 @@ return \%results; } +sub _fail_scheduling ($self, $error, $error_code = HTTP_BAD_REQUEST) { + return { + error => $error, + error_code => $error_code, + status => FATAL_ERROR, + }; +} + =over 4 =item _job_ref() diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-5.1783672871.e689eb3c/lib/OpenQA/Utils.pm new/openQA-5.1784178984.04ce3617/lib/OpenQA/Utils.pm --- old/openQA-5.1783672871.e689eb3c/lib/OpenQA/Utils.pm 2026-07-10 10:41:11.000000000 +0200 +++ new/openQA-5.1784178984.04ce3617/lib/OpenQA/Utils.pm 2026-07-16 07:16:24.000000000 +0200 @@ -813,7 +813,12 @@ my %SERVICE_OFFSETS = (webui => 0, websocket => 1, livehandler => 2, scheduler => 3, cache_service => 4); my %RESERVED_SOCKETS; -sub raw_service_port ($service) { +sub raw_service_port ($service, $port = undef) { + if (defined $port) { + $port = raw_port_for_reserved($port) if $port =~ /^\d+$/; + return $port; + } + my $base = $ENV{OPENQA_BASE_PORT} ||= DEFAULT_OPENQA_BASE_PORT; if (my $env_port = $ENV{"OPENQA_PORT_\U$service"}) { # \U uppercases the interpolated variable return $env_port; @@ -840,6 +845,17 @@ return @$services == 1 ? $RESERVED_SOCKETS{$services->[0]} : \%RESERVED_SOCKETS; } +# Map a plain reserved port back to its "port&fd=…" form so a child daemon reuses +# the parent's already-bound socket instead of binding a fresh one (which only +# succeeds on IPv6 loopback and deadlocks IPv4 clients). Returns the port +# unchanged if it was not reserved. +sub raw_port_for_reserved ($plain) { + for my $socket (values %RESERVED_SOCKETS) { + return "$plain&fd=" . $socket->fileno if $socket->sockport == $plain; + } + return $plain; +} + sub random_string ($length = RANDOM_STRING_DEFAULT_LENGTH, $chars = ['a' .. 'z', 'A' .. 'Z', '0' .. '9', '_']) { return join '', map { $chars->[rand @$chars] } 1 .. $length; } diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-5.1783672871.e689eb3c/lib/OpenQA/WebAPI/Controller/API/V1/Comment.pm new/openQA-5.1784178984.04ce3617/lib/OpenQA/WebAPI/Controller/API/V1/Comment.pm --- old/openQA-5.1783672871.e689eb3c/lib/OpenQA/WebAPI/Controller/API/V1/Comment.pm 2026-07-10 10:41:11.000000000 +0200 +++ new/openQA-5.1784178984.04ce3617/lib/OpenQA/WebAPI/Controller/API/V1/Comment.pm 2026-07-16 07:16:24.000000000 +0200 @@ -148,6 +148,7 @@ } $self->emit_event('openqa_comment_create', $comment->event_data); $txn_guard->commit; + $self->flash(info => 'The comments have been created.'); $self->render(json => {id => $comment->id}); } diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-5.1783672871.e689eb3c/lib/OpenQA/WebAPI/Controller/API/V1/Iso.pm new/openQA-5.1784178984.04ce3617/lib/OpenQA/WebAPI/Controller/API/V1/Iso.pm --- old/openQA-5.1783672871.e689eb3c/lib/OpenQA/WebAPI/Controller/API/V1/Iso.pm 2026-07-10 10:41:11.000000000 +0200 +++ new/openQA-5.1784178984.04ce3617/lib/OpenQA/WebAPI/Controller/API/V1/Iso.pm 2026-07-16 07:16:24.000000000 +0200 @@ -8,6 +8,7 @@ use OpenQA::Utils; use DBIx::Class::Timestamps 'now'; use OpenQA::Schema::Result::JobDependencies; +use HTTP::Status qw(:constants); use constant MANDATORY_PARAMETERS => qw(DISTRI VERSION FLAVOR ARCH); use constant RESERVED_API_KEYS_RE => qr/^(?:async|scheduled_product_clone_id)$/; @@ -158,6 +159,10 @@ passed to the method as arguments. Returns a JSON block containing the number of jobs created, their job ids and the information for jobs that could not be scheduled. +Supports an optional C<_FAIL_IF_NO_JOBS> parameter. When set to C<1>, the request fails with +a C<400 Bad Request> if no jobs are scheduled (unless C<async=1> is used). In either case, the +scheduled product status is set to C<fatal_error>. + =back =cut diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-5.1783672871.e689eb3c/lib/OpenQA/WebAPI/Controller/Test.pm new/openQA-5.1784178984.04ce3617/lib/OpenQA/WebAPI/Controller/Test.pm --- old/openQA-5.1783672871.e689eb3c/lib/OpenQA/WebAPI/Controller/Test.pm 2026-07-10 10:41:11.000000000 +0200 +++ new/openQA-5.1784178984.04ce3617/lib/OpenQA/WebAPI/Controller/Test.pm 2026-07-16 07:16:24.000000000 +0200 @@ -880,7 +880,11 @@ my $test = $job->TEST; my $flavor = $job->FLAVOR || 'sweet'; my $arch = $job->ARCH || 'noarch'; - $result->{deps} = to_json($job->dependencies($children_by_job->{$id} || [], $parents_by_job->{$id} || [])); + my $children = $children_by_job->{$id} || []; + my $parents = $parents_by_job->{$id} || []; + my $deps = $job->dependencies($children, $parents); + $result->{dep_count} = (scalar @$children) + (scalar @$parents); + $result->{deps} = to_json($deps); # Append machine name to TEST if it does not match the most frequently used MACHINE # for the jobs architecture diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-5.1783672871.e689eb3c/lib/OpenQA/Worker/Engines/isotovideo.pm new/openQA-5.1784178984.04ce3617/lib/OpenQA/Worker/Engines/isotovideo.pm --- old/openQA-5.1783672871.e689eb3c/lib/OpenQA/Worker/Engines/isotovideo.pm 2026-07-10 10:41:11.000000000 +0200 +++ new/openQA-5.1784178984.04ce3617/lib/OpenQA/Worker/Engines/isotovideo.pm 2026-07-16 07:16:24.000000000 +0200 @@ -497,6 +497,7 @@ $container->on(container_error => sub ($cont, $e) { log_error("Container error: @{$e}", channels => 'worker') }); log_info('Starting isotovideo container'); + $worker->update_job_guard_expiration; $container->start(); $workerpid = $child->pid(); return $callback->({child => $child}); diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-5.1783672871.e689eb3c/lib/OpenQA/Worker/WebUIConnection.pm new/openQA-5.1784178984.04ce3617/lib/OpenQA/Worker/WebUIConnection.pm --- old/openQA-5.1783672871.e689eb3c/lib/OpenQA/Worker/WebUIConnection.pm 2026-07-10 10:41:11.000000000 +0200 +++ new/openQA-5.1784178984.04ce3617/lib/OpenQA/Worker/WebUIConnection.pm 2026-07-16 07:16:24.000000000 +0200 @@ -17,6 +17,7 @@ use constant DEFAULT_OPENQA_WORKER_CONNECT_RETRIES => 60; use constant MAX_WEBSOCKET_SIZE => 1024 * 1024 * 10; +use constant EPHEMERAL_RECHECK_INTERVAL => $ENV{OPENQA_WORKER_EPHEMERAL_RECHECK_INTERVAL} // 10; has 'webui_host'; # hostname:port of the web UI to connect to has 'url'; # URL of the web UI to connect to - initially deduced from webui_host (Mojo::URL instance) @@ -380,13 +381,24 @@ # send the worker status (unless the websocket connection has been lost) my $websocket_connection = $self->websocket_connection; return undef unless $websocket_connection; - my $status = $self->{_last_worker_status} = $self->worker->status; + my $status = $self->{_last_worker_status} = $args{status} // $self->worker->status; $websocket_connection->send({json => $status}) if !$args{on_error} || defined $status->{reason}; } sub send_status_delayed ($self) { return undef if $self->{_send_status_timer} || !$self->websocket_connection; + # re-evaluate the error status after just a few seconds if the current error is ephemeral + return $self->{_send_status_timer} = Mojo::IOLoop->timer( + EPHEMERAL_RECHECK_INTERVAL, + sub { + my $worker = $self->worker; + my $status = $self->{_last_worker_status} = $self->worker->status; + return $self->send_status(status => $status) unless $worker->current_error; + delete $self->{_send_status_timer}; + $self->send_status_delayed; + }) if $self->worker->current_error_is_ephemeral; + my $status_update_interval = $self->calculate_status_update_interval; my $webui_host = $self->webui_host; my $reason = ($self->{_last_worker_status} // {})->{reason}; diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-5.1783672871.e689eb3c/lib/OpenQA/Worker.pm new/openQA-5.1784178984.04ce3617/lib/OpenQA/Worker.pm --- old/openQA-5.1783672871.e689eb3c/lib/OpenQA/Worker.pm 2026-07-10 10:41:11.000000000 +0200 +++ new/openQA-5.1784178984.04ce3617/lib/OpenQA/Worker.pm 2026-07-16 07:16:24.000000000 +0200 @@ -42,9 +42,12 @@ use Mojo::File 'path'; use POSIX (); use Scalar::Util 'looks_like_number'; +use Time::Seconds qw(ONE_HOUR); use OpenQA::Constants qw(WEBSOCKET_API_VERSION WORKER_COMMAND_QUIT WORKER_SR_BROKEN WORKER_SR_DONE WORKER_SR_DIED WORKER_SR_FINISH_OFF); use OpenQA::Client; +use OpenQA::CacheService::Client; +use OpenQA::CacheService::DB; use OpenQA::Log qw(log_error log_warning log_info log_debug add_log_channel remove_log_channel); use OpenQA::Utils qw(prjdir load_avg); use OpenQA::Worker::WebUIConnection; @@ -64,6 +67,7 @@ has 'current_job'; has 'current_error'; has 'current_error_is_fatal'; +has 'current_error_is_ephemeral'; has 'worker_hostname'; has 'isotovideo_interface_version'; @@ -84,6 +88,12 @@ $settings->auto_detect_worker_address($short_hostname); $settings->apply_to_app($app); + # init Minion for locking + my $location = OpenQA::CacheService::DB::location($settings->global_settings) // (prjdir . '/cache'); + my $db_file = OpenQA::CacheService::DB::db_file($location, 'worker-locks.sqlite'); + my $sqlite = OpenQA::CacheService::DB::open_sqlite_database($app->log, $db_file); + $app->plugin(Minion => {SQLite => $sqlite}); + # setup the isotovideo engine my $isotovideo_interface_version = OpenQA::Worker::Engines::isotovideo::set_engine_exec($cli_options->{isotovideo}); @@ -103,10 +113,40 @@ $self->{_finishing_off} = undef; $self->{_ovs_dbus_service_name} = $ENV{OVS_DBUS_SERVICE_NAME} // 'org.opensuse.os_autoinst.switch'; $self->{_initial_working_dir} = getcwd() // '.'; - + $self->{_minion_sqlite} = $sqlite; return $self; } +sub guard ($self, @args) { $self->app->minion->guard(@args) } + +sub _job_guard ($self) { + my $settings = $self->settings->global_settings; + return undef unless defined(my $lock_name = $settings->{LOCK_NAME}); + return $self->guard($lock_name, ONE_HOUR, {limit => $settings->{LOCK_LIMIT} // 5}) + // "Limited by configured lock '$lock_name'"; +} + +sub can_acquire_job_guard ($self) { !!(ref $self->_job_guard) } + +sub acquire_job_guard ($self) { + $self->{_guard_or_error} = undef if ref $self->{_guard_or_error}; + $self->{_guard_or_error} = $self->_job_guard; +} + +# reduces the expiration of the lock with the longest expiration from the initial ONE_HOUR to the configured LOCK_EXPIRATION +# note: This function is called by the engine when the setup is over and the actual execution starts and thus an increased load +# is expected. +sub update_job_guard_expiration ($self) { + return 0 unless ref $self->{_guard_or_error}; # skip if we don't hold a lock + my $settings = $self->settings->global_settings; + return undef unless defined(my $lock_name = $settings->{LOCK_NAME}); + my $expiration = int($settings->{LOCK_EXPIRATION} // 30); + my $query = q(update minion_locks set expires = datetime('now', ?) where name = ? order by expires desc limit 1); + $self->{_minion_sqlite}->db->query($query, "+$expiration seconds", $lock_name); +} + +sub release_job_guard ($self) { undef $self->{_guard_or_error} } + # logs the basic configuration of the worker instance sub log_setup_info ($self) { my $instance = $self->instance_number; @@ -541,9 +581,19 @@ return $next_job->accept; } +sub _ensure_job_guard ($self, $client, $job_ids) { + my $guard_or_error = $self->acquire_job_guard; + return 1 if !defined $guard_or_error || ref $guard_or_error; + $client->send_status(on_error => 1); + $client->reject_jobs($job_ids, $guard_or_error); + log_info "Rejecting job: $guard_or_error"; + return 0; +} + # accepts a single job from the job info received via the 'grab_job' command sub accept_job ($self, $client, $job_info) { $self->_assert_whether_job_acceptance_possible; + $self->_ensure_job_guard($client, [$job_info->{id}]) or return 0; $self->{_queue} = undef; $self->_prepare_job_execution(OpenQA::Worker::Job->new($self, $client, $job_info)); $self->current_job->accept; @@ -569,6 +619,7 @@ # if one job fails. $self->_assert_whether_job_acceptance_possible; + $self->_ensure_job_guard($client, [keys %{$job_info->{data}}]) or return 0; $self->_enqueue_job_sub_sequence($client, $self->_init_queue, $job_info->{sequence}, $job_info->{data}); $self->_accept_or_skip_next_job_in_queue; } @@ -657,6 +708,8 @@ # returns whether the worker is available and a reason # note: This is used to check certain error conditions *before* starting a job to prevent incompletes and # being able to propagate the brokenness to the web UIs. +# note: This function returns a value >0 if there is no fatal problem and >1 if the problem is even expected +# to be resolved soon. # note: High load will yield a corresponding error message as reason so the worker becomes broken and # thus will not pick up any new jobs. However, a worker under high load is still considered being # able to work on jobs that are already enqueued. Hence this function returns a "1" for the @@ -686,13 +739,25 @@ if $settings->has_class('tap') && !$self->is_ovs_dbus_service_running; # continue with enqueued jobs in any case but avoid picking up new jobs if system utilization is critical - return (1, $self->_check_system_utilization); + if (my $system_utilization_error = $self->_check_system_utilization) { + return (1, $system_utilization_error); + } + + # check whether we failed to get the job lock (and would still not be able to get it) + if (defined(my $guard_or_error = $self->{_guard_or_error})) { + my $is_error = !ref $guard_or_error; + return (2, $guard_or_error) if $is_error && !$self->can_acquire_job_guard; + undef $self->{_guard_or_error} if $is_error; + } + + return (1, undef); } sub set_current_error_based_on_availability ($self) { - my ($is_available, $reason) = $self->check_availability; + my ($availability, $reason) = $self->check_availability; $self->current_error($reason); - $self->current_error_is_fatal(!$is_available); + $self->current_error_is_fatal($availability < 1); + $self->current_error_is_ephemeral($availability > 1); return $reason; } @@ -777,6 +842,7 @@ if (my $error_message = $event_data->{error_message}) { log_error($error_message) } $self->current_job(undef); $self->current_webui_host(undef); + $self->release_job_guard; if (my $queue = $self->{_queue}) { $queue->{failed_jobs}->{$job_id} = $reason if $reason ne WORKER_SR_DONE || !$event_data->{ok}; } diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-5.1783672871.e689eb3c/t/05-scheduler-serialize-directly-chained-dependencies.t new/openQA-5.1784178984.04ce3617/t/05-scheduler-serialize-directly-chained-dependencies.t --- old/openQA-5.1783672871.e689eb3c/t/05-scheduler-serialize-directly-chained-dependencies.t 2026-07-10 10:41:11.000000000 +0200 +++ new/openQA-5.1784178984.04ce3617/t/05-scheduler-serialize-directly-chained-dependencies.t 2026-07-16 07:16:24.000000000 +0200 @@ -142,7 +142,7 @@ # provide a sort function to control the order between multiple children of the same parent my %sort_criteria = (12 => 'anfang', 7 => 'danach', 6 => 'mitte', 8 => 'nach mitte', 1 => 'zuletzt'); my $sort_function = sub ($items) { - return [sort { ($sort_criteria{$a} // $a) cmp($sort_criteria{$b} // $b) } @$items]; + return [sort { ($sort_criteria{$a} // $a) cmp ($sort_criteria{$b} // $b) } @$items]; }; @expected_sequence = (0, 12, 7, 6, [8, [10, 11], 9], [1, [2, 3], [4, 5]]); ($computed_sequence, $visited) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-5.1783672871.e689eb3c/t/10-tests_overview.t new/openQA-5.1784178984.04ce3617/t/10-tests_overview.t --- old/openQA-5.1783672871.e689eb3c/t/10-tests_overview.t 2026-07-10 10:41:11.000000000 +0200 +++ new/openQA-5.1784178984.04ce3617/t/10-tests_overview.t 2026-07-16 07:16:24.000000000 +0200 @@ -15,7 +15,7 @@ use Mojo::Parameters; my $test_case = OpenQA::Test::Case->new; -$test_case->init_data(fixtures_glob => '01-jobs.pl 03-users.pl 05-job_modules.pl'); +$test_case->init_data(fixtures_glob => '01-jobs.pl 03-users.pl 05-job_modules.pl 06-job_dependencies.pl'); my $t = Test::Mojo->new('OpenQA::WebAPI'); my $schema = $t->app->schema; @@ -623,21 +623,33 @@ ok ref $clone, "Job cloned successfully: $clone" or BAIL_OUT 'Failed to clone job'; $clone->update({state => 'done', result => 'passed'}); $t->get_ok('/tests/overview' => form => {build => '198077'})->status_is(200); - $t->content_like(qr/fa-undo/i, 'Overview contains restart icon'); - $t->content_like(qr/title="Restarted 1 time"/i, 'Overview contains restart title'); + $t->element_exists('.restarts .fa-clock-rotate-left', 'Overview contains restart icon'); + $t->element_exists('[title="Restarted 1 time"]', 'Overview contains restart title'); create_job(BUILD => '198077_norestart', TEST => 'no_restart_test'); $t->get_ok('/tests/overview' => form => {build => '198077_norestart'})->status_is(200); - $t->content_unlike(qr/fa-undo/i, 'Overview does not contain restart icon for job with no restarts'); + $t->element_exists_not('.restarts .fa-clock-rotate-left', 'no restart icon for job with no restarts'); my $clone2 = $clone->auto_duplicate(); ok ref $clone2, "Job 2 cloned successfully: $clone2" or BAIL_OUT 'Failed to clone job 2'; $clone2->update({state => 'done', result => 'passed'}); $t->get_ok('/tests/overview' => form => {build => '198077'})->status_is(200); + $t->element_exists('[title="Restarted 2 times"]', 'Overview contains 2 restarts title'); $t->content_like(qr/title="Restarted 2 times"/i, 'Overview contains 2 restarts title'); - $t->content_like(qr/<i class="fa fa-undo"><\/i> 2/i, 'Overview contains restart count 2'); + $t->content_like(qr/fa-clock-rotate-left"><\/i> 2/i, 'Overview contains restart count 2'); $t->get_ok('/tests/list_ajax' => form => {limit => 100})->status_is(200); my $data = Mojo::JSON::decode_json($t->tx->res->body)->{data}; my ($clone2_data) = grep { $_->{id} == $clone2->id } @$data; is $clone2_data->{restarts}, 2, 'AJAX list contains restart count 2'; }; +subtest 'advanced restart options' => sub { + $test_case->login($t, 'percival'); + $t->get_ok('/tests/overview?groupid=1001&distri=opensuse&version=13.1&build=0091'); + $t->element_exists('#restart-button-options-99937', 'menu with advanced restart options present'); + my $links = $t->tx->res->dom->find('#restart-button-options-99937 + .dropdown-menu a')->map(attr => 'href')->sort; + my @expected_params = qw(skip_children skip_ok_result_children skip_parents); + my @expected_links = map { "/api/v1/jobs/99937/restart?$_=1" } @expected_params; + is_deeply $links, \@expected_links, 'advanced restart links present' or always_explain $links; + $t->element_exists_not('#restart-button-options-99946', 'menu not present if job has no dependencies'); +}; + done_testing(); diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-5.1783672871.e689eb3c/t/16-utils.t new/openQA-5.1784178984.04ce3617/t/16-utils.t --- old/openQA-5.1783672871.e689eb3c/t/16-utils.t 2026-07-10 10:41:11.000000000 +0200 +++ new/openQA-5.1784178984.04ce3617/t/16-utils.t 2026-07-16 07:16:24.000000000 +0200 @@ -556,4 +556,12 @@ 'valid download for asset URL'; }; +subtest 'reserved ports' => sub { + is raw_service_port('webui', 1234), 1234, 'unreserved port unchanged'; + my $socket = OpenQA::Utils::reserve_ports(['webui'], force => 1); + my $port = $socket->sockport; + my $fileno = $socket->fileno; + is raw_service_port('webui', $port), "$port&fd=$fileno", 'reserved port mapped to fd form'; +}; + done_testing; diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-5.1783672871.e689eb3c/t/24-worker-engine.t new/openQA-5.1784178984.04ce3617/t/24-worker-engine.t --- old/openQA-5.1783672871.e689eb3c/t/24-worker-engine.t 2026-07-10 10:41:11.000000000 +0200 +++ new/openQA-5.1784178984.04ce3617/t/24-worker-engine.t 2026-07-16 07:16:24.000000000 +0200 @@ -31,6 +31,7 @@ use File::Copy::Recursive qw(dircopy); my $workdir = tempdir("$FindBin::Script-XXXX", TMPDIR => 1); +$ENV{OPENQA_CACHE_DIR} = $workdir; chdir $workdir; my $guard = scope_guard sub { chdir $FindBin::Bin }; dircopy "$FindBin::Bin/$_", "$workdir/t/$_" or BAIL_OUT($!) for qw(data); @@ -367,6 +368,8 @@ OpenQA::Worker::Engines::isotovideo::sync_tests(OpenQA::CacheService::Client->new, @args); Mojo::IOLoop->start; is $result, 'cache-dir/webuihost/tests', 'returns synced test directory on success' or always_explain $result; + + is $worker->job_guard_expiration_updated, 0, 'job guard not updated while just syncing tests'; }; subtest 'symlink testrepo, logging behavior, variable expansion' => sub { @@ -401,13 +404,14 @@ like $result->{error}, qr/The source directory $casedir\/needles does not exist/, 'symlink needles directory failed because source directory does not exist'; }; - subtest 'good case: direct invocation of helper function for symlinking' => sub { my $casedir = testcasedir('opensuse', undef, undef); my $result = OpenQA::Worker::Engines::isotovideo::_link_repo($casedir, $pool_directory, 'opensuse'); is $result, undef, 'create symlink successfully'; }; + is $worker->job_guard_expiration_updated, 0, 'job guard not updated'; + my @custom_casedir_settings = ( CASEDIR_DOMAIN => 'github.com', CASEDIR => 'https://%CASEDIR_DOMAIN%/foo/os-autoinst-distri-example.git#master', @@ -432,6 +436,7 @@ 'PRODUCTDIR still defaults to a relative path when CASEDIR is a URL to main.pm from custom test repo is used'; is $vars_data->{NEEDLES_DIR}, 'needles', 'relative NEEDLES_DIR is set to its basename'; is $result->{error}, undef, 'no error occurred (1)'; + is $worker->job_guard_expiration_updated, 1, 'job guard updated as the engine was started'; }; subtest 'good case: custom CASEDIR and custom NEEDLES_DIR specified and both are Git repos' => sub { diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-5.1783672871.e689eb3c/t/24-worker-overall.t new/openQA-5.1784178984.04ce3617/t/24-worker-overall.t --- old/openQA-5.1783672871.e689eb3c/t/24-worker-overall.t 2026-07-10 10:41:11.000000000 +0200 +++ new/openQA-5.1784178984.04ce3617/t/24-worker-overall.t 2026-07-16 07:16:24.000000000 +0200 @@ -11,9 +11,11 @@ use OpenQA::Test::TimeLimit '10'; use OpenQA::Test::Utils qw(simulate_load); use Data::Dumper; +use DateTime::Format::Pg; use Mojo::File qw(tempdir tempfile path); use Mojo::Util 'scope_guard'; use Mojolicious; +use Time::Seconds qw(ONE_MINUTE); use Test::Output qw(combined_like combined_from); use Test::MockModule; use Test::MockObject; @@ -33,6 +35,7 @@ $ENV{OPENQA_LOGFILE} = undef; my $workdir = tempdir("$FindBin::Script-XXXX", TMPDIR => 1); +$ENV{OPENQA_CACHE_DIR} = $workdir; chdir $workdir; my $guard = scope_guard sub { chdir $FindBin::Bin }; @@ -50,9 +53,9 @@ has service_port_delta => 2; has api_calls => sub { [] }; - sub send ($self, $method, $path, %args) { - push @{$self->api_calls}, $method, $path, $args{params}; - } + sub send ($self, $method, $path, %args) { push @{$self->api_calls}, $method, $path, $args{params} } + sub send_status ($self, @args) { push @{$self->api_calls}, 'send_status', @args } + sub reject_jobs ($self, @args) { push @{$self->api_calls}, 'reject_jobs', @args } } # uncoverable statement package Test::FakeJob { # uncoverable statement count:2 @@ -127,6 +130,63 @@ qr/.*http:\/\/localhost:9527,https:\/\/remotehost.*qemu_i386,qemu_x86_64.*Errors occurred.*foo.*bar.*/s, 'setup info with parse errors'; +sub get_locks () { + $worker->{_minion_sqlite}->db->query(q(select expires from minion_locks order by expires;))->arrays; +} + +subtest 'job lock for throttling' => sub { + is $worker->acquire_job_guard, undef, 'no attempt to acquire job lock if not configured'; + + my $settings = $worker->settings->global_settings; + local $settings->{LOCK_NAME} = 'foo'; + local $settings->{LOCK_LIMIT} = 2; + + ok ref(my $guard_1 = $worker->acquire_job_guard), 'was able to acquire lock'; + is $worker->{_guard_or_error}, $guard_1, 'worker keeps a hold on the lock'; + ok $worker->app->minion->is_locked('foo'), 'expected lock acquired'; + ok $worker->can_acquire_job_guard, 'there is room for one more lock'; + ok ref(my $guard_2 = $worker->acquire_job_guard), 'was able to acquire second lock (still within limit)'; + ok !$worker->can_acquire_job_guard, 'limit has been reached'; + my ($available, $error) = $worker->check_availability; + is $error, undef, 'no availability error as we have not tried to go beyond limit'; + + subtest 'updating expiration' => sub { + is $worker->update_job_guard_expiration->rows, 1, 'one lock updated'; + my $locks = get_locks; + is @$locks, 2, 'the two locks are present' or return; + # DateTime::Format::Pg works for these SQLite datetimes as well + my $expiration_1 = DateTime::Format::Pg->parse_datetime($locks->[0]->[0])->epoch; + my $expiration_2 = DateTime::Format::Pg->parse_datetime($locks->[1]->[0])->epoch; + my $diff = abs $expiration_1 - $expiration_2; + cmp_ok $diff, '>', ONE_MINUTE, 'expirations differ in the order of minutes after update_job_guard_expiration'; + }; + + subtest 'unable to acquire lock' => sub { + is $worker->acquire_job_guard, "Limited by configured lock 'foo'", + 'was unable to acquire third lock (exceeds limit)'; + is $worker->{_guard_or_error}, "Limited by configured lock 'foo'", 'error recorded'; + ($available, $error) = $worker->check_availability; + is $error, "Limited by configured lock 'foo'", 'availability error returned as we tried to go beyond limit'; + + my $client = Test::FakeClient->new; + $worker->_ensure_job_guard($client, [42]); + my @expected = (send_status => (on_error => 1), reject_jobs => ([42], "Limited by configured lock 'foo'")); + is_deeply $client->api_calls, \@expected, 'status sent to web UI and assignment rejected' + or always_explain $client->api_calls; + }; + + subtest 'releasing locks' => sub { + undef $guard_1; + undef $guard_2; + is @{get_locks()}, 0, 'locks are released by setting guards to undef'; + ($available, $error) = $worker->check_availability; + is $error, undef, 'availability error cleared'; + is $worker->{_guard_or_error}, undef, 'guard/error cleared'; + }; + + undef $worker->{_guard_or_error}; +}; + subtest 'worker load' => sub { my $load = OpenQA::Utils::load_avg(); is scalar @$load, 3, 'expected number of load values'; diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-5.1783672871.e689eb3c/t/24-worker-webui-connection.t new/openQA-5.1784178984.04ce3617/t/24-worker-webui-connection.t --- old/openQA-5.1783672871.e689eb3c/t/24-worker-webui-connection.t 2026-07-10 10:41:11.000000000 +0200 +++ new/openQA-5.1784178984.04ce3617/t/24-worker-webui-connection.t 2026-07-16 07:16:24.000000000 +0200 @@ -10,6 +10,8 @@ use lib "$FindBin::Bin/lib", "$FindBin::Bin/../external/os-autoinst-common/lib"; use Mojo::Base -signatures; +BEGIN { $ENV{OPENQA_WORKER_EPHEMERAL_RECHECK_INTERVAL} = 0 } + use Mojo::IOLoop; use Mojolicious; use Test::Output 'combined_like'; @@ -368,7 +370,14 @@ $client->send_status(); is_deeply $ws->sent_messages, [{json => {fake_status => 1, reason => 'some error'}}], 'status sent' or always_explain $ws->sent_messages; - combined_like { $client->send_status_delayed } qr/some error.*checking again/, 'error logged in callback'; + + # consider the error ephemeral; it is expected to be re-checked and replaced by the FakeWorker with "another error" + $client->worker->current_error_is_ephemeral(1); + combined_like { + $client->send_status_delayed; + Mojo::IOLoop->start; + } + qr/another error.*checking again/, 'error logged in callback'; }; subtest 'quit' => sub { diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-5.1783672871.e689eb3c/t/39-scheduled_products-table.t new/openQA-5.1784178984.04ce3617/t/39-scheduled_products-table.t --- old/openQA-5.1783672871.e689eb3c/t/39-scheduled_products-table.t 2026-07-10 10:41:11.000000000 +0200 +++ new/openQA-5.1784178984.04ce3617/t/39-scheduled_products-table.t 2026-07-16 07:16:24.000000000 +0200 @@ -12,11 +12,12 @@ use Test::MockObject; use Test::Output qw(combined_like); use OpenQA::Jobs::Constants; -use OpenQA::Schema::Result::ScheduledProducts qw(ADDED SCHEDULED SCHEDULING CANCELLING CANCELLED); +use OpenQA::Schema::Result::ScheduledProducts qw(ADDED SCHEDULED SCHEDULING CANCELLING CANCELLED FATAL_ERROR); use OpenQA::Test::TimeLimit '6'; use OpenQA::Test::Case; use OpenQA::Utils; use DateTime; +use HTTP::Status qw(:constants); OpenQA::Test::Case->new->init_data; my $t = Test::Mojo->new('OpenQA::WebAPI'); @@ -40,8 +41,9 @@ $scheduled_products_mock->redefine(_generate_jobs => undef); # prevent job creation my $scheduled_product; +my ($sp1, $sp2, $sp3); subtest 'handling assets with invalid name' => sub { - $scheduled_product = $scheduled_products->create(\%settings); + $sp1 = $scheduled_product = $scheduled_products->create(\%settings); $schema->txn_begin; is_deeply $scheduled_product->schedule_iso({REPO_0 => ''}, $signal_guard), @@ -57,7 +59,7 @@ $scheduled_product->discard_changes; is $scheduled_product->status, SCHEDULED, 'product marked as scheduled, though'; - $scheduled_product = $scheduled_products->create(\%settings); + $sp2 = $scheduled_product = $scheduled_products->create(\%settings); is_deeply $scheduled_product->schedule_iso({REPO_0 => 'invalid'}, $signal_guard), { @@ -71,6 +73,19 @@ is $scheduled_product->status, SCHEDULED, 'product marked as scheduled, though'; }; +subtest '_FAIL_IF_NO_JOBS fails on empty schedule but preserves product' => sub { + my $fail_sp = $scheduled_products->create(\%settings); + my $sp_id = $fail_sp->id; + is_deeply + $fail_sp->schedule_iso({REPO_0 => 'invalid', _FAIL_IF_NO_JOBS => 1}, $signal_guard), + {error => 'No jobs scheduled.', error_code => HTTP_BAD_REQUEST}, + 'schedule_iso fails on no jobs scheduled when _FAIL_IF_NO_JOBS is set'; + my $found = $scheduled_products->find($sp_id); + ok $found, 'scheduled product is not deleted from DB'; + is $found->status, FATAL_ERROR, 'status is FATAL_ERROR'; + is_deeply $found->results, {error => 'No jobs scheduled.', error_code => HTTP_BAD_REQUEST}, 'results contain error'; +}; + is $scheduled_product->schedule_iso(\%settings, $signal_guard), undef, 'scheduling the same product again prevented'; subtest 'asset registration on scheduling' => sub { @@ -78,7 +93,7 @@ my %asset_info = (type => 'iso', name => 'dvdsize42.iso'); $schema->storage->dbh->prepare('delete from assets where name = ? ')->execute('dvdsize42.iso'); is $assets->find(\%asset_info), undef, 'dvdsize42.iso is not known yet'; - $scheduled_product = $scheduled_products->create(\%settings); + $sp3 = $scheduled_product = $scheduled_products->create(\%settings); $scheduled_product->schedule_iso({ISO => 'dvdsize42.iso'}, $signal_guard); is $assets->find(\%asset_info)->size, 42, 'dvdsize42.iso has known size'; $scheduled_product->discard_changes; @@ -150,11 +165,11 @@ }; subtest 'cleanup' => sub { - $scheduled_products->search({id => 1})->update({t_created => DateTime->now->subtract(days => 35)}); + $scheduled_products->search({id => $sp1->id})->update({t_created => DateTime->now->subtract(days => 35)}); $scheduled_products->delete_expired_entries; - ok !$scheduled_products->find(1), 'old scheduled product without jobs deleted'; - ok $scheduled_products->find(2), 'new scheduled product without jobs still present)'; - ok $scheduled_products->find(3), 'scheduled product with jobs still present'; + ok !$scheduled_products->find($sp1->id), 'old scheduled product without jobs deleted'; + ok $scheduled_products->find($sp2->id), 'new scheduled product without jobs still present)'; + ok $scheduled_products->find($sp3->id), 'scheduled product with jobs still present'; }; subtest 'handling failed job cancellation' => sub { diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-5.1783672871.e689eb3c/t/api/02-iso.t new/openQA-5.1784178984.04ce3617/t/api/02-iso.t --- old/openQA-5.1783672871.e689eb3c/t/api/02-iso.t 2026-07-10 10:41:11.000000000 +0200 +++ new/openQA-5.1784178984.04ce3617/t/api/02-iso.t 2026-07-16 07:16:24.000000000 +0200 @@ -395,6 +395,36 @@ schedule_iso($t, {iso => $iso, tests => 'kde/usb'}, 400, {}, 'invalid parameters'); schedule_iso($t, {%iso, FLAVOR => 'cherry'}, 200, {}, 'no product found'); +{ + my $count_before = $scheduled_products->count; + my $res = schedule_iso($t, {%iso, FLAVOR => 'cherry', _FAIL_IF_NO_JOBS => 1}, 400, {}, 'fail on no jobs scheduled'); + is $res->json->{error}, 'no products found for opensuse-cherry-i586', 'correct error message'; + is $scheduled_products->count, $count_before + 1, 'scheduled product is preserved in DB'; + my $sp_id = $res->json->{scheduled_product_id}; + ok $sp_id, 'scheduled_product_id is returned'; + my $sp = $scheduled_products->find($sp_id); + ok $sp, 'scheduled product exists in DB'; + is $sp->status, 'fatal_error', 'status is fatal_error'; + is $sp->results->{error}, 'no products found for opensuse-cherry-i586', 'results contain correct error'; + + my $res_async = schedule_iso( + $t, {%iso, FLAVOR => 'cherry', _FAIL_IF_NO_JOBS => 1}, + 200, + {async => 1}, + '_FAIL_IF_NO_JOBS allowed with async' + ); + my $sp_id_async = $res_async->json->{scheduled_product_id}; + ok $sp_id_async, 'scheduled_product_id is returned for async'; + my $sp_async = $scheduled_products->find($sp_id_async); + is $sp_async->status, 'added', 'status is added initially'; + + perform_minion_jobs($t->app->minion); + + $sp_async->discard_changes; + is $sp_async->status, 'fatal_error', 'status becomes fatal_error after minion job'; + is $sp_async->results->{error}, 'no products found for opensuse-cherry-i586', + 'results contain correct error for async'; +} schedule_iso($t, {%iso, _GROUP_ID => 12345}, 404, {}, 'no templates found'); # handle list of tests diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-5.1783672871.e689eb3c/t/lib/OpenQA/Test/FakeWorker.pm new/openQA-5.1784178984.04ce3617/t/lib/OpenQA/Test/FakeWorker.pm --- old/openQA-5.1783672871.e689eb3c/t/lib/OpenQA/Test/FakeWorker.pm 2026-07-10 10:41:11.000000000 +0200 +++ new/openQA-5.1784178984.04ce3617/t/lib/OpenQA/Test/FakeWorker.pm 2026-07-16 07:16:24.000000000 +0200 @@ -1,6 +1,8 @@ package OpenQA::Test::FakeWorker; use Mojo::Base -base, -signatures; +use Mojo::IOLoop; + package Test::FakeSettings { use Mojo::Base -base; has global_settings => sub { {RETRIES => 3, RETRY_DELAY => 10, RETRY_DELAY_IF_WEBUI_BUSY => 90} }; @@ -17,6 +19,7 @@ has skipped_jobs => sub { [] }; has current_error => undef; has current_job => undef; +has current_error_is_ephemeral => 0; has pending_job => undef; has has_pending_jobs => 0; has pending_job_ids => sub { [] }; @@ -25,10 +28,19 @@ has settings => sub { Test::FakeSettings->new }; has enqueued_job_info => undef; has is_executing_single_job => 1; +has job_guard_expiration_updated => 0; +sub update_job_guard_expiration ($self) { $self->job_guard_expiration_updated(1) } sub stop_current_job ($self, $reason) { $self->stop_current_job_called($reason) } sub stop ($self) { $self->is_stopping(1) } -sub status ($self) { {fake_status => 1, reason => $self->current_error} } + +sub status ($self) { + if ($self->current_error_is_ephemeral) { + $self->current_error('another error')->current_error_is_ephemeral(0); + Mojo::IOLoop->stop; + } + return {fake_status => 1, reason => $self->current_error}; +} sub accept_job ($self, $client, $job_info) { $self->current_job(OpenQA::Worker::Job->new($self, $client, $job_info)); diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-5.1783672871.e689eb3c/t/lib/OpenQA/Test/Utils.pm new/openQA-5.1784178984.04ce3617/t/lib/OpenQA/Test/Utils.pm --- old/openQA-5.1783672871.e689eb3c/t/lib/OpenQA/Test/Utils.pm 2026-07-10 10:41:11.000000000 +0200 +++ new/openQA-5.1784178984.04ce3617/t/lib/OpenQA/Test/Utils.pm 2026-07-16 07:16:24.000000000 +0200 @@ -269,7 +269,7 @@ } sub create_webapi ($port = undef, $no_cover = undef) { - $port //= raw_service_port 'webui'; + $port = raw_service_port('webui', $port); note("Starting WebUI service. Port: $port"); my $h = _setup_sigchld_handler 'openqa-webapi', start sub { @@ -296,7 +296,7 @@ } sub create_websocket_server ($port, $bogus, $with_embedded_scheduler = undef, $no_cover = undef) { - OpenQA::WebSockets::Client->singleton->port(to_plain_service_port($port //= raw_service_port('websocket'))); + OpenQA::WebSockets::Client->singleton->port(to_plain_service_port($port = raw_service_port('websocket', $port))); note "Starting WebSocket service (port: $port, bogus: $bogus)"; my $h = _setup_sigchld_handler 'openqa-websocket', start sub { _setup_sub_process 'openqa-websocket'; @@ -339,7 +339,8 @@ return $h; } -sub create_scheduler ($port = raw_service_port('scheduler')) { +sub create_scheduler ($port = undef) { + $port = raw_service_port('scheduler', $port); note("Starting Scheduler service. Port: $port"); OpenQA::Scheduler::Client->singleton->port(to_plain_service_port($port)); _setup_sigchld_handler 'openqa-scheduler', start sub { @@ -352,7 +353,8 @@ }; } -sub create_live_view_handler ($port = raw_service_port 'livehandler') { +sub create_live_view_handler ($port = undef) { + $port = raw_service_port('livehandler', $port); _setup_sigchld_handler 'openqa-livehandler', start sub { _setup_sub_process 'openqa-livehandler'; my $daemon diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-5.1783672871.e689eb3c/t/ui/10-tests_overview.t new/openQA-5.1784178984.04ce3617/t/ui/10-tests_overview.t --- old/openQA-5.1783672871.e689eb3c/t/ui/10-tests_overview.t 2026-07-10 10:41:11.000000000 +0200 +++ new/openQA-5.1784178984.04ce3617/t/ui/10-tests_overview.t 2026-07-16 07:16:24.000000000 +0200 @@ -637,9 +637,9 @@ $driver->find_element_by_id('text')->send_keys($comment_text); is $submit_button->get_text, 'Comment on all 2 jobs', 'submit button displayed with number of jobs'; $submit_button->click; - wait_for_ajax msg => 'comments created'; - like $driver->find_element_by_id('flash-messages')->get_text, qr/The comments have been created. Reload/, + ok wait_for_element(selector => '#flash-messages .alert-info', like => qr/The comments have been created\./), 'info about success shown'; + ok wait_for_element(selector => '.label_comment', is_displayed => 1), 'comment shown without manual reloading'; my @failed_job_ids = map { $_->id } $jobs->search({result => FAILED})->all; is $comments->search({job_id => {-in => \@failed_job_ids}, text => $comment_text})->count, 2, @@ -675,6 +675,8 @@ is $submit_button->get_text, 'Comment on all 1 jobs', 'submit button displayed with number of jobs (reduced by TODO filter)'; $submit_button->click; + ok wait_for_element(selector => '#flash-messages .alert-info', like => qr/The comments have been created\./), + 'info about success shown'; # reload without TO-DO filter to check whether only the one job that matched the TO-DO filter was commented on $driver->get('/tests/overview?build=0048&distri=opensuse&version=Factory&groupid=1001'); diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-5.1783672871.e689eb3c/t/ui/26-jobs_restart.t new/openQA-5.1784178984.04ce3617/t/ui/26-jobs_restart.t --- old/openQA-5.1783672871.e689eb3c/t/ui/26-jobs_restart.t 2026-07-10 10:41:11.000000000 +0200 +++ new/openQA-5.1784178984.04ce3617/t/ui/26-jobs_restart.t 2026-07-16 07:16:24.000000000 +0200 @@ -15,12 +15,14 @@ use OpenQA::Jobs::Constants; use OpenQA::JobDependencies::Constants; use Date::Format 'time2str'; +use Mojo::Util 'scope_guard'; my $test_case = OpenQA::Test::Case->new; my $schema_name = OpenQA::Test::Database::generate_schema_name; my $fixtures = '01-jobs.pl 06-job_dependencies.pl'; my $schema = $test_case->init_data(schema_name => $schema_name, fixtures_glob => $fixtures); my $jobs = $schema->resultset('Jobs'); +my $job_deps = $schema->resultset('JobDependencies'); sub prepare_database () { # Populate more cluster jobs @@ -96,26 +98,26 @@ subtest 'restart job from info panel in test results' => sub { subtest 'parent job shows options for advanced restart' => sub { $driver->get_ok('/tests/99900', 'go to job 99900'); - is @{$driver->find_elements('//a[contains(@id, "restart")]', 'xpath')}, 4, 'multiple restart actions found'; - ok !$driver->find_element('#restart-result-skip-parents')->is_displayed, 'advanced restart entries present'; - $driver->find_element('#restart-button-options')->click; - my $restart_parent = $driver->find_element('#restart-result-skip-parents'); + is @{$driver->find_elements('//a[contains(@class, "restart")]', 'xpath')}, 4, 'multiple restart actions found'; + ok !$driver->find_element('.restart-result-skip-parents')->is_displayed, 'advanced restart entries present'; + $driver->find_element('.restart-button-options')->click; + my $restart_parent = $driver->find_element('.restart-result-skip-parents'); ok $restart_parent->is_displayed, 'advanced restart entries shown'; $restart_parent->attribute_like('href', qr/skip_parents=1/, 'skip parents API URL correct'); - $driver->find_element('#restart-result-skip-ok-children') + $driver->find_element('.restart-result-skip-ok-children') ->attribute_like('href', qr/skip_ok_result_children=1/, 'skip OK children API URL correct'); - $driver->find_element('#restart-result-skip-children')->click; + $driver->find_element('.restart-result-skip-children')->click; wait_until(sub { $driver->get_current_url =~ qr{/tests/99982} }, 'shows cloned job'); $jobs->find(99982)->delete; }; subtest 'child job shows options for advanced restart' => sub { $driver->get_ok('/tests/99901', 'go to job 99901'); - ok !$driver->find_element('#restart-result-skip-parents')->is_displayed, 'advanced restart present for child'; - $driver->find_element('#restart-button-options')->click; - my $restart_parent = $driver->find_element('#restart-result-skip-parents'); + ok !$driver->find_element('.restart-result-skip-parents')->is_displayed, 'advanced restart present for child'; + $driver->find_element('.restart-button-options')->click; + my $restart_parent = $driver->find_element('.restart-result-skip-parents'); ok $restart_parent->is_displayed, 'advanced restart entries shown for child'; $restart_parent->attribute_like('href', qr/skip_parents=1/, 'skip parents API URL correct for child'); - $driver->find_element('#restart-result-skip-ok-children') + $driver->find_element('.restart-result-skip-ok-children') ->attribute_like('href', qr/skip_ok_result_children=1/, 'skip OK children API URL correct for child'); }; subtest 'assets missing; there is no parent' => sub { @@ -128,15 +130,54 @@ 'restarting job with missing asset results in an error', 20); }; subtest 'assets missing; there is a parent' => sub { - $schema->resultset('JobDependencies') - ->create( - {child_job_id => 99939, parent_job_id => 99947, dependency => OpenQA::JobDependencies::Constants::CHAINED}); + $job_deps->create({child_job_id => 99939, parent_job_id => 99947, dependency => CHAINED}); is $driver->get('/tests/99939'), 1, 'go to job 99939'; $driver->find_element('#restart-result')->click(); wait_until(sub { flash_messages =~ m/Job 99939 misses.*\.iso.*You may try to retrigger the parent job/s }, 'restarting job with missing asset results in an error', 20); }; + subtest 'restart parent skipping OK children option on directly chained dependency error' => sub { + # cleanup after leaving subtest (not using txn here as the forked process needs to see changes) + my $job_deps_search = {parent_job_id => 99937, child_job_id => 99938}; + my $cleanup = scope_guard sub { + $job_deps->search($job_deps_search)->update({dependency => CHAINED}); + $jobs->search({id => [99937, 99938]})->update({clone_id => undef}); + $jobs->search({id => [99983, 99984]})->delete; + $schema->storage->dbh->do('ALTER SEQUENCE jobs_id_seq RESTART WITH 99983'); + }; + + # create a parent-prefixed asset to satisfy the missing asset check, change dependency to directly chained + $schema->resultset('Assets')->find_or_create( + { + name => '00099937-openSUSE-Factory-DVD-x86_64-Build0048-Media.iso', + type => 'iso', + size => 0, + }); + $job_deps->search($job_deps_search)->update({dependency => DIRECTLY_CHAINED}); + + is $driver->get('/tests/99938'), 1, 'go to job 99938 (directly chained child)'; + $driver->find_element('#restart-result')->click(); + wait_for_ajax(msg => 'fail to restart job because of directly chained parent'); + wait_until(sub { flash_messages =~ m/Direct parent 99937 needs to be cloned as well/s }, + 'direct parent error shown', 20); + + ok my $toggle = $driver->find_element('#flash-messages .dropdown-toggle'), 'dropdown toggle button is present' + or return; + $toggle->click(); # open dropdown and find our new option + my $option = $driver->find_element('.restart-parent-skip-ok'); + is $option->get_text(), 'Restart parent job skipping passed/softfailed children', + 'dropdown option text is correct'; + $option->click(); # click the dropdown option to restart parent job skipping OK children + + # since it successfully restarts, it should redirect to the newly cloned parent job. + wait_for_ajax(msg => 'parent job restarted skipping OK children'); + wait_until(sub { $driver->get_current_url =~ qr{/tests/(\d+)} && $1 != 99938 }, + 'redirects to restarted parent job'); + }; subtest 'force restart' => sub { + is $driver->get('/tests/99939'), 1, 'go to job 99939'; + $driver->find_element('#restart-result')->click(); + wait_for_ajax(msg => 'fail to start job with missing asset and parent'); update_last_job_id; $driver->find_element('#flash-messages button.force-restart')->click(); wait_for_ajax(msg => 'forced job restart'); diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-5.1783672871.e689eb3c/templates/webapi/test/infopanel.html.ep new/openQA-5.1784178984.04ce3617/templates/webapi/test/infopanel.html.ep --- old/openQA-5.1783672871.e689eb3c/templates/webapi/test/infopanel.html.ep 2026-07-10 10:41:11.000000000 +0200 +++ new/openQA-5.1784178984.04ce3617/templates/webapi/test/infopanel.html.ep 2026-07-16 07:16:24.000000000 +0200 @@ -154,24 +154,11 @@ <div class="btn-group btn-group-sm dropstart"> % my $has_dependencies = $job->has_dependencies; % my $ok_results_explanation = join '/', OK_RESULTS; - % if ($has_dependencies) { - <button class="btn btn-secondary dropdown-toggle" type="button" id="restart-button-options" data-bs-toggle="dropdown" aria-haspopup="true" aria-expanded="false"> - <span class="caret" title="Additional options for restarting jobs with dependencies"></span> - </button> - <div class="dropdown-menu" role="menu" aria-labelledby="restart-button-options" id="restart-menu"> - %= link_post url_for('apiv1_restart', jobid => $testid)->query({skip_parents => 1}) => (class => 'dropdown-item restart-result', 'data-remote' => 'true', id => 'restart-result-skip-parents', 'data-jobid' => $testid) => begin - <span>Skip parents</span> - %= end - %= link_post url_for('apiv1_restart', jobid => $testid)->query({skip_children => 1}) => (class => 'dropdown-item restart-result', 'data-remote' => 'true', id => 'restart-result-skip-children', 'data-jobid' => $testid) => begin - <span>Skip children</span> - %= end - %= link_post url_for('apiv1_restart', jobid => $testid)->query({skip_ok_result_children => 1}) => (class => 'dropdown-item restart-result', 'data-remote' => 'true', id => 'restart-result-skip-ok-children', 'data-jobid' => $testid) => begin - <span>Skip restarting "OK" (<%= $ok_results_explanation %>) children</span> - %= end - </div> - % } % my $restart_text = $has_dependencies ? 'Restart this and dependent jobs' : 'Restart job'; % my $class = 'btn btn-secondary restart-result'; + % if ($has_dependencies) { + %= include 'test/restart_options', jobid => $testid, btn_type => 'btn-secondary', btn_style => ''; + % } % $class .= ' disabled' if defined $job->clone_id; %= link_post url_for('apiv1_restart', jobid => $testid) => (class => $class, 'data-remote' => 'true', id => 'restart-result', 'data-jobid' => $testid) => begin <i class="fa-solid fa-2 fa-rotate-left"></i><span><%= $restart_text %></span> diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-5.1783672871.e689eb3c/templates/webapi/test/restart_options.html.ep new/openQA-5.1784178984.04ce3617/templates/webapi/test/restart_options.html.ep --- old/openQA-5.1783672871.e689eb3c/templates/webapi/test/restart_options.html.ep 1970-01-01 01:00:00.000000000 +0100 +++ new/openQA-5.1784178984.04ce3617/templates/webapi/test/restart_options.html.ep 2026-07-16 07:16:24.000000000 +0200 @@ -0,0 +1,15 @@ +% use OpenQA::Jobs::Constants; +<button class="btn <%= $btn_type %> dropdown-toggle restart-button-options" type="button" id="restart-button-options-<%= $jobid %>" data-bs-toggle="dropdown" aria-haspopup="true" aria-expanded="false" style="<%= $btn_style %>"> + <span class="caret" title="Additional options for restarting jobs with dependencies"></span> +</button> +<div class="dropdown-menu" role="menu" aria-labelledby="restart-button-options-<%= $jobid %>"> + %= link_post url_for('apiv1_restart', jobid => $jobid)->query({skip_parents => 1}) => (class => 'dropdown-item restart-result restart-result-skip-parents', 'data-remote' => 'true', 'data-jobid' => $jobid) => begin + <span>Skip parents</span> + %= end + %= link_post url_for('apiv1_restart', jobid => $jobid)->query({skip_children => 1}) => (class => 'dropdown-item restart-result restart-result-skip-children', 'data-remote' => 'true', 'data-jobid' => $jobid) => begin + <span>Skip children</span> + %= end + %= link_post url_for('apiv1_restart', jobid => $jobid)->query({skip_ok_result_children => 1}) => (class => 'dropdown-item restart-result restart-result-skip-ok-children', 'data-remote' => 'true', 'data-jobid' => $jobid) => begin + <span>Skip restarting "OK" (<%= join '/', OK_RESULTS %>) children</span> + %= end +</div> \ No newline at end of file diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-5.1783672871.e689eb3c/templates/webapi/test/tr_job_result.html.ep new/openQA-5.1784178984.04ce3617/templates/webapi/test/tr_job_result.html.ep --- old/openQA-5.1783672871.e689eb3c/templates/webapi/test/tr_job_result.html.ep 2026-07-10 10:41:11.000000000 +0200 +++ new/openQA-5.1784178984.04ce3617/templates/webapi/test/tr_job_result.html.ep 2026-07-16 07:16:24.000000000 +0200 @@ -5,32 +5,30 @@ <i class="action fa-solid fa-fw fa-copy" title="Copy job id"></i></a> % if ($state eq DONE || $state eq CANCELLED) { %= link_post url_for('apiv1_restart', jobid => $jobid) => ('data-remote' => 'true', class => 'restart', 'data-jobid' => $jobid) => begin - <i class="action fa-solid fa-rotate-left" title="Restart the job"></i> - % end + <i class="action fa-solid fa-rotate-left" title="Restart the job"></i><%= end %> + % if ($res->{dep_count}) { + %= include 'test/restart_options', btn_type => 'btn-link', btn_style => 'padding: 0px'; + % } % } % elsif (any { $_ eq $state } PENDING_STATES) { %= link_post url_for('apiv1_cancel', jobid => $jobid) => ('data-remote' => 'true', class => 'cancel', 'data-jobid' => $jobid) => begin - <i class="action fa-regular fa-circle-xmark" title="Cancel the job"></i> - % end + <i class="action fa-regular fa-circle-xmark" title="Cancel the job"></i><%= end %> % } % } <span id="res-<%= $jobid %>"> % if ($state eq DONE) { <a href="<%= url_for('test', 'testid' => $jobid) %>"> - <i class="status fa-solid fa-circle result_<%= $res->{overall} %>" title="Done: <%= $res->{overall} %>"></i> - </a> + <i class="status fa-solid fa-circle result_<%= $res->{overall} %>" title="Done: <%= $res->{overall} %>"></i></a> % } % elsif ($state eq SCHEDULED) { % my $substate = $res->{blocked} ? 'blocked' : 'scheduled'; <a href="<%= url_for('test', 'testid' => $jobid) %>"> - <i class="status state_<%= $substate %> fa-solid fa-circle" title="<%= $substate %>@<%= $res->{priority} %>"></i> - </a> + <i class="status state_<%= $substate %> fa-solid fa-circle" title="<%= $substate %>@<%= $res->{priority} %>"></i></a> % } % else { <a href="<%= url_for('test', 'testid' => $jobid) %>"> - <i class="status state_<%= $state %> fa-solid fa-circle" title="<%= $state %>"></i> - </a> + <i class="status state_<%= $state %> fa-solid fa-circle" title="<%= $state %>"></i></a> % } </span> % if ($res->{failures}) { @@ -38,7 +36,7 @@ % } % if ($res->{restarts}) { <span class="restarts" title="Restarted <%= $res->{restarts} %> time<%= $res->{restarts} != 1 ? 's' : '' %>"> - <i class="fa fa-undo"></i> <%= $res->{restarts} %> + <i class="action fa fa-clock-rotate-left"></i> <%= $res->{restarts} %> </span> % } %= include 'test/tr_job_result_details', jobid => $jobid, res => $res diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-5.1783672871.e689eb3c/tools/ci/autoinst.sha new/openQA-5.1784178984.04ce3617/tools/ci/autoinst.sha --- old/openQA-5.1783672871.e689eb3c/tools/ci/autoinst.sha 2026-07-10 10:41:11.000000000 +0200 +++ new/openQA-5.1784178984.04ce3617/tools/ci/autoinst.sha 2026-07-16 07:16:24.000000000 +0200 @@ -1 +1 @@ -6ea6c0bea9246eda41d4d83e19433aa04aef704f \ No newline at end of file +791afcb11e0cc9cb996b1b13be4a0bf8af3f0155 \ No newline at end of file diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/openQA-5.1783672871.e689eb3c/xt/01-style.t new/openQA-5.1784178984.04ce3617/xt/01-style.t --- old/openQA-5.1783672871.e689eb3c/xt/01-style.t 2026-07-10 10:41:11.000000000 +0200 +++ new/openQA-5.1784178984.04ce3617/xt/01-style.t 2026-07-16 07:16:24.000000000 +0200 @@ -31,7 +31,7 @@ is qx{git grep -I -l 'sub [a-z_A-Z0-9]\\+()' ':!docs/'}, '', 'Consistent space before function signatures (this is not ensured by perltidy)'; is -qx{git grep -Pr "(?<!->)(?<!sub )\\b(ok|is|isnt|like|unlike|cmp_ok|can_ok|isa_ok|subtest|diag|note|explain|pass|fail|new_ok|is_deeply)\\s*\\(" {t,xt}/ | grep -vE "(t|xt)/(lib|testresults)/"}, +qx{git grep -Pr "(?<!->)(?<!sub )(?<!\\#.{,254})\\b(ok|is|isnt|like|unlike|cmp_ok|can_ok|isa_ok|subtest|diag|note|explain|pass|fail|new_ok|is_deeply)\\s*\\(" {t,xt}/ | grep -vE "(t|xt)/(lib|testresults)/"}, '', 'Consistent Test::More call format (no parentheses)'; ++++++ openQA.obsinfo ++++++ --- /var/tmp/diff_new_pack.kQxmNI/_old 2026-07-17 01:46:05.642536781 +0200 +++ /var/tmp/diff_new_pack.kQxmNI/_new 2026-07-17 01:46:05.662537456 +0200 @@ -1,5 +1,5 @@ name: openQA -version: 5.1783672871.e689eb3c -mtime: 1783672871 -commit: e689eb3cd5e8081ff544e1bf54222e999605513c +version: 5.1784178984.04ce3617 +mtime: 1784178984 +commit: 04ce3617510cf57f12f06b63c33688b64a151663
