Indeed, these were in the build queue when I was restarting Jenkins and the jobs tried to re-run before it was fully up. I also see the following error when publishing el7:
Temporary repodata directory /srv/resources/repos/ovirt/tested/./master/rpm/el7/.repodata/ already exists! (Another createrepo process is running?) https://jenkins.ovirt.org/job/deploy-to_ovirt-master_tested/7790/console Looks like it's also caused by me cancelling a job that left behind a .repodata directory. Will clean it up now to unblock el7 publishing. On Wed, Jun 5, 2019 at 8:07 AM Emil Natan <[email protected]> wrote: > A bunch of deploy to tested jobs failed as result of the maintenance > restarts earlier today. I'm re-triggering those now. > > On Wed, Jun 5, 2019 at 1:38 AM <[email protected]> wrote: > >> See < >> http://jenkins.ovirt.org/job/deploy-to_ovirt-master_tested/7776/display/redirect >> > >> >> ------------------------------------------ >> Started by upstream project "ovirt-master_change-queue-tester" build >> number 14434 >> originally caused by: >> Started by upstream project "ovirt-master_change-queue" build number >> 42136 >> originally caused by: >> Started by upstream project "vdsm_standard-on-merge" build number 795 >> originally caused by: >> Triggered by Gerrit: https://gerrit.ovirt.org/100463 >> [EnvInject] - Loading node environment variables. >> Building remotely on vm0010.workers-phx.ovirt.org (el7 phx nested) in >> workspace <http://jenkins.ovirt.org/job/deploy-to_ovirt-master_tested/ws/ >> > >> [ssh-agent] Looking for ssh-agent implementation... >> [ssh-agent] Exec ssh-agent (binary ssh-agent on a remote machine) >> $ ssh-agent >> SSH_AUTH_SOCK=/tmp/ssh-epHRQo8YDSQE/agent.19927 >> SSH_AGENT_PID=19929 >> [ssh-agent] Started. >> Running ssh-add (command line suppressed) >> Identity added: < >> http://jenkins.ovirt.org/job/deploy-to_ovirt-master_tested/ws/@tmp/private_key_5326099156985161464.key> >> (< >> http://jenkins.ovirt.org/job/deploy-to_ovirt-master_tested/ws/@tmp/private_key_5326099156985161464.key >> )> >> [ssh-agent] Using credentials deploy-ovirt-experimental (SSH key for >> deploying to the tested repo) >> [deploy-to_ovirt-master_tested] $ /bin/bash -xe >> /tmp/jenkins7284571172790321491.sh >> + [[ jenkins:http://jenkins.ovirt.org/job/vdsm_standard-on-merge/795/ == >> '' ]] >> + queue_name=ovirt-master >> + ssh -o StrictHostKeyChecking=no >> [email protected] >> + echo repo-extra-dir:master >> + echo jenkins:http://jenkins.ovirt.org/job/vdsm_standard-on-merge/795/ >> Pseudo-terminal >> <http://jenkins.ovirt.org/job/vdsm_standard-on-merge/795/Pseudo-terminal> >> will not be allocated because stdin is not a terminal. >> + BASE_DIR=/srv/resources/repos/ovirt/tested >> + PUBLISH_MD_COPIES=50 >> + main >> + local tmp_dir >> + mkdir -p /srv/resources/repos/ovirt/tested >> ++ mktemp -d /srv/resources/repos/ovirt/tested/.deploy.XXXXXXXXXX >> Collecting packages >> + tmp_dir=/srv/resources/repos/ovirt/tested/.deploy.5Xqeujnn6v >> + trap 'rm -rf >> '\''/srv/resources/repos/ovirt/tested/.deploy.5Xqeujnn6v'\''' EXIT HUP >> + echo 'Collecting packages' >> + collect_packages /srv/resources/repos/ovirt/tested/.deploy.5Xqeujnn6v >> + local repoman_dst=/srv/resources/repos/ovirt/tested/.deploy.5Xqeujnn6v >> + repoman --temp-dir generate-in-repo --option >> main.allowed_repo_paths=/srv/resources/repos/ovirt/tested/.deploy.5Xqeujnn6v >> --option main.on_empty_source=warn >> /srv/resources/repos/ovirt/tested/.deploy.5Xqeujnn6v add conf:stdin >> 2019-06-04 22:37:56,171::INFO::repoman.cmd:: >> 2019-06-04 22:37:56,171::INFO::repoman.cmd::Adding artifacts to the repo >> /srv/resources/repos/ovirt/tested/.deploy.5Xqeujnn6v >> 2019-06-04 22:37:56,171::INFO::repoman.common.repo::Adding repo extra dir >> master >> 2019-06-04 22:37:56,173::INFO::repoman.common.stores.RPM::Loading repo >> /srv/resources/repos/ovirt/tested/.deploy.5Xqeujnn6v/master >> 2019-06-04 22:37:56,173::INFO::repoman.common.stores.RPM::Repo >> /srv/resources/repos/ovirt/tested/.deploy.5Xqeujnn6v/master loaded >> 2019-06-04 22:37:56,174::INFO::repoman.common.stores.iso::Loading repo >> /srv/resources/repos/ovirt/tested/.deploy.5Xqeujnn6v/master >> 2019-06-04 22:37:56,175::INFO::repoman.common.stores.iso::Repo >> /srv/resources/repos/ovirt/tested/.deploy.5Xqeujnn6v/master loaded >> 2019-06-04 22:37:56,183::INFO::repoman.common.repo::Resolving artifact >> source jenkins:http://jenkins.ovirt.org/job/vdsm_standard-on-merge/795/ >> 2019-06-04 >> <http://jenkins.ovirt.org/job/vdsm_standard-on-merge/795/2019-06-04> >> 22:37:56,308::ERROR::repoman.common.sources.jenkins::URL: >> http://jenkins.ovirt.org/job/vdsm_standard-on-merge/795//api/json?depth=3 >> Status: 503 >> Reason: Service Unavailable >> Headers: {'cache-control': 'no-cache,no-store,must-revalidate', >> 'connection': 'close', >> 'content-encoding': 'gzip', >> 'content-length': '456', >> 'content-type': 'text/html;charset=utf-8', >> 'date': 'Tue, 04 Jun 2019 22:37:54 GMT', >> 'expires': '0', >> 'server': 'Jetty(9.4.z-SNAPSHOT)', >> 'via': '1.1 jenkins.phx.ovirt.org', >> 'x-content-type-options': 'nosniff'} >> Body: >> >> >> <!DOCTYPE html><html lang="en-US"><head resURL="/static/539a36f3" >> data-rooturl="" data-resurl="/static/539a36f3"><title>Starting >> Jenkins</title><meta name="ROBOTS" content="NOFOLLOW" /><link >> rel="stylesheet" href="/static/539a36f3/css/simple-page.css" >> type="text/css" /><link rel="stylesheet" >> href="/static/539a36f3/css/simple-page.theme.css" type="text/css" /><link >> rel="stylesheet" href="/static/539a36f3/css/loading.css" type="text/css" >> /></head><body><div class="simple-page" role="main"><div class="modal >> signup"><div class="signupIntroDefault"><div class="logo"></div><h1 >> class="loading"> >> Please wait while Jenkins is getting ready to >> work >> >> <span>.</span><span>.</span><span>.</span></h1><p>Your browser will reload >> automatically when Jenkins is ready.</p></div></div></div><script >> src="/static/539a36f3/scripts/loading.js" >> type="text/javascript"></script><script>safeRedirector(window.location.href);</script></body></html> >> >> 2019-06-04 22:37:58,360::ERROR::repoman.common.sources.jenkins::URL: >> http://jenkins.ovirt.org/job/vdsm_standard-on-merge/795//api/json?depth=3 >> Status: 503 >> Reason: Service Unavailable >> Headers: {'cache-control': 'no-cache,no-store,must-revalidate', >> 'connection': 'close', >> 'content-encoding': 'gzip', >> 'content-length': '456', >> 'content-type': 'text/html;charset=utf-8', >> 'date': 'Tue, 04 Jun 2019 22:37:56 GMT', >> 'expires': '0', >> 'server': 'Jetty(9.4.z-SNAPSHOT)', >> 'via': '1.1 jenkins.phx.ovirt.org', >> 'x-content-type-options': 'nosniff'} >> Body: >> >> >> <!DOCTYPE html><html lang="en-US"><head resURL="/static/539a36f3" >> data-rooturl="" data-resurl="/static/539a36f3"><title>Starting >> Jenkins</title><meta name="ROBOTS" content="NOFOLLOW" /><link >> rel="stylesheet" href="/static/539a36f3/css/simple-page.css" >> type="text/css" /><link rel="stylesheet" >> href="/static/539a36f3/css/simple-page.theme.css" type="text/css" /><link >> rel="stylesheet" href="/static/539a36f3/css/loading.css" type="text/css" >> /></head><body><div class="simple-page" role="main"><div class="modal >> signup"><div class="signupIntroDefault"><div class="logo"></div><h1 >> class="loading"> >> Please wait while Jenkins is getting ready to >> work >> >> <span>.</span><span>.</span><span>.</span></h1><p>Your browser will reload >> automatically when Jenkins is ready.</p></div></div></div><script >> src="/static/539a36f3/scripts/loading.js" >> type="text/javascript"></script><script>safeRedirector(window.location.href);</script></body></html> >> >> 2019-06-04 22:38:00,436::ERROR::repoman.common.sources.jenkins::URL: >> http://jenkins.ovirt.org/job/vdsm_standard-on-merge/795//api/json?depth=3 >> Status: 503 >> Reason: Service Unavailable >> Headers: {'cache-control': 'no-cache,no-store,must-revalidate', >> 'connection': 'close', >> 'content-encoding': 'gzip', >> 'content-length': '456', >> 'content-type': 'text/html;charset=utf-8', >> 'date': 'Tue, 04 Jun 2019 22:37:58 GMT', >> 'expires': '0', >> 'server': 'Jetty(9.4.z-SNAPSHOT)', >> 'via': '1.1 jenkins.phx.ovirt.org', >> 'x-content-type-options': 'nosniff'} >> Body: >> >> >> <!DOCTYPE html><html lang="en-US"><head resURL="/static/539a36f3" >> data-rooturl="" data-resurl="/static/539a36f3"><title>Starting >> Jenkins</title><meta name="ROBOTS" content="NOFOLLOW" /><link >> rel="stylesheet" href="/static/539a36f3/css/simple-page.css" >> type="text/css" /><link rel="stylesheet" >> href="/static/539a36f3/css/simple-page.theme.css" type="text/css" /><link >> rel="stylesheet" href="/static/539a36f3/css/loading.css" type="text/css" >> /></head><body><div class="simple-page" role="main"><div class="modal >> signup"><div class="signupIntroDefault"><div class="logo"></div><h1 >> class="loading"> >> Please wait while Jenkins is getting ready to >> work >> >> <span>.</span><span>.</span><span>.</span></h1><p>Your browser will reload >> automatically when Jenkins is ready.</p></div></div></div><script >> src="/static/539a36f3/scripts/loading.js" >> type="text/javascript"></script><script>safeRedirector(window.location.href);</script></body></html> >> >> 2019-06-04 22:38:02,514::ERROR::repoman.common.sources.jenkins::URL: >> http://jenkins.ovirt.org/job/vdsm_standard-on-merge/795//api/json?depth=3 >> Status: 503 >> Reason: Service Unavailable >> Headers: {'cache-control': 'no-cache,no-store,must-revalidate', >> 'connection': 'close', >> 'content-encoding': 'gzip', >> 'content-length': '456', >> 'content-type': 'text/html;charset=utf-8', >> 'date': 'Tue, 04 Jun 2019 22:38:01 GMT', >> 'expires': '0', >> 'server': 'Jetty(9.4.z-SNAPSHOT)', >> 'via': '1.1 jenkins.phx.ovirt.org', >> 'x-content-type-options': 'nosniff'} >> Body: >> >> >> <!DOCTYPE html><html lang="en-US"><head resURL="/static/539a36f3" >> data-rooturl="" data-resurl="/static/539a36f3"><title>Starting >> Jenkins</title><meta name="ROBOTS" content="NOFOLLOW" /><link >> rel="stylesheet" href="/static/539a36f3/css/simple-page.css" >> type="text/css" /><link rel="stylesheet" >> href="/static/539a36f3/css/simple-page.theme.css" type="text/css" /><link >> rel="stylesheet" href="/static/539a36f3/css/loading.css" type="text/css" >> /></head><body><div class="simple-page" role="main"><div class="modal >> signup"><div class="signupIntroDefault"><div class="logo"></div><h1 >> class="loading"> >> Please wait while Jenkins is getting ready to >> work >> >> <span>.</span><span>.</span><span>.</span></h1><p>Your browser will reload >> automatically when Jenkins is ready.</p></div></div></div><script >> src="/static/539a36f3/scripts/loading.js" >> type="text/javascript"></script><script>safeRedirector(window.location.href);</script></body></html> >> >> 2019-06-04 22:38:04,516::ERROR::repoman.cmd:: >> 2019-06-04 22:38:04,516::ERROR::repoman.cmd::Error while adding conf:stdin >> 2019-06-04 22:38:04,518::INFO::repoman.common.repo::Cleaning up temporary >> dir >> /srv/resources/repos/ovirt/tested/.deploy.5Xqeujnn6v/.lago_tmp/tmpQ4NzXM/tmpgqw_Ee >> 2019-06-04 22:38:04,519::INFO::repoman.common.repo::Cleaning up temporary >> dir /srv/resources/repos/ovirt/tested/.deploy.5Xqeujnn6v/.lago_tmp/tmpQ4NzXM >> + rm -rf /srv/resources/repos/ovirt/tested/.deploy.5Xqeujnn6v >> Build step 'Execute shell' marked build as failure >> $ ssh-agent -k >> unset SSH_AUTH_SOCK; >> unset SSH_AGENT_PID; >> echo Agent pid 19929 killed; >> [ssh-agent] Stopped. >> _______________________________________________ >> Infra mailing list -- [email protected] >> To unsubscribe send an email to [email protected] >> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >> oVirt Code of Conduct: >> https://www.ovirt.org/community/about/community-guidelines/ >> List Archives: >> https://lists.ovirt.org/archives/list/[email protected]/message/ETOIPTHC5I4FBQLVX5HWCIUYFRW4CTL7/ >> > > > -- > Emil Natan > RHV/CNV DevOps > _______________________________________________ > Infra mailing list -- [email protected] > To unsubscribe send an email to [email protected] > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/[email protected]/message/B5TI3BQQFN5C7VECQTRAMEUL5J3A7MLP/ > -- Regards, Evgheni Dereveanchin
_______________________________________________ Infra mailing list -- [email protected] To unsubscribe send an email to [email protected] Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/[email protected]/message/7DZCKHPQQXPJGMMEM3ZGO7ZRK7RT6XGI/
