[JIRA] (OVIRT-772) Re: Upgrade from 3.6 el7 job broken?

2016-10-14 Thread Evgheni Dereveanchin (oVirt JIRA) 

 [ 
https://ovirt-jira.atlassian.net/browse/OVIRT-772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Evgheni Dereveanchin closed OVIRT-772.
--
Resolution: Fixed

The slave was fixed (puppet certificate issues) and hystrix was installed as 
expected.

Closing the case for now, please reopen if you see a similar issue on other 
slaves as well.

> Re: Upgrade from 3.6 el7 job broken?
> 
>
> Key: OVIRT-772
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-772
> Project: oVirt - virtualization made easy
>  Issue Type: By-EMAIL
>Reporter: sbonazzo
>Assignee: Evgheni Dereveanchin
>
> On Fri, Oct 14, 2016 at 8:59 AM, Roman Mohr  wrote:
> >
> >
> > On Fri, Oct 14, 2016 at 8:57 AM, Sandro Bonazzola 
> > wrote:
> >
> >>
> >>
> >> On Fri, Oct 14, 2016 at 8:48 AM, Sandro Bonazzola 
> >> wrote:
> >>
> >>>
> >>>
> >>> On Fri, Oct 14, 2016 at 8:44 AM, Roman Mohr  wrote:
> >>>
>  Hi,
> 
>  I saw a lot of failing builds lately on this job:
> 
>    http://jenkins.ovirt.org/job/ovirt-engine_master_upgrade-fro
>  m-3.6_el7_created/
> 
>  One log which still exists is:
> 
>  http://jenkins.ovirt.org/job/ovirt-engine_master_upgrade-fro
>  m-3.6_el7_created/3666/
>  It seems like the BUILD ENGINE RPM step is failing, but I can't see any
>  reason why:
> 
>  BUILDING ENGINE RPM
>  + create_rpms 
>  /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo/ovirt-engine-4.1.0-0.0.master.20161014001903.el7.centos.src.rpm
>   
>  /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo
>   .gitee47dd2
>  + local 
>  src_rpm=/home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo/ovirt-engine-4.1.0-0.0.master.20161014001903.el7.centos.src.rpm
>  + local 
>  dst_dir=/home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo
>  + local release=.gitee47dd2
>  + local 
>  workspace=/home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created
>  + local 'BUILD_JAVA_OPTS_MAVEN=-XX:MaxPermSize=1G 
>  -Dgwt.compiler.localWorkers=1 '
>  + local 'BUILD_JAVA_OPTS_GWT=-XX:PermSize=512M 
>  -XX:MaxPermSize=1G -Xms1G -Xmx6G '
>  + env 'BUILD_JAVA_OPTS_MAVEN=-XX:MaxPermSize=1G 
>  -Dgwt.compiler.localWorkers=1 ' 'BUILD_JAVA_OPTS_GWT=
>  -XX:PermSize=512M -XX:MaxPermSize=1G -Xms1G -Xmx6G ' 
>  rpmbuild -D 'ovirt_build_minimal 1' -D 'release_suffix .gitee47dd2' -D 
>  'ovirt_build_extra_flags -gs 
>  /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/artifactory-ovirt-org-settings.xml'
>   -D '_srcrpmdir 
>  /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo'
>   -D '_specdir 
>  /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo'
>   -D '_sourcedir 
>  /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo'
>   -D '_rpmdir 
>  /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo'
>   -D '_builddir 
>  /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo'
>   --rebuild 
>  /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo/ovirt-engine-4.1.0-0
 .0.master.20161014001903.el7.centos.src.rpm
>  + return 1
>  Build step 'Execute shell' marked build as failure
>  Performing Post build task...
>  Match found for :.* : True
>  Logical operation result is TRUE
>  Running script  : #!/bin/bash -x
> 
> 
>  Best Regards,
> 
> 
> >>>
> >>> Rpmbuild fails on:
> >>> error: Failed build dependencies:
> >>> hystrix-core is needed by ovirt-engine-4.1.0-0.0.master.
> >>> 20161014004451.git32a6f0c.el7.centos.noarch
> >>> hystrix-metrics-event-stream is needed by ovirt-engine-4.1.0-0.0.master.
> >>> 20161014004451.git32a6f0c.el7.centos.noarch
> >>> Installing /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6
> >>> _el7_created/tmp_repo/ovirt-engine-4.1.0-0.0.master.20161014
> >>> 004451.el7.centos.src.rpm
> >>>
> >>> I'll check why those rpms are missing.
> >>>
> >>
> >> Looks like a temporary failure accessing EPEL repository.
> >>
> >>
> > Hm, maybe it happens on some slaves only, but there always? Because it
> > happens pretty often.
> > Here is a good example:
> >
> > https://gerrit.ovirt.org/#/q/status:open+project:ovirt-
> > engine+branch:master+topic:AuditLogableBase-inject
> >
> > All the broken builds are because of this. And that is just after the last
> > rebase from Alon. Before that it was pretty much the

[JIRA] (OVIRT-772) Re: Upgrade from 3.6 el7 job broken?

2016-10-14 Thread Evgheni Dereveanchin (oVirt JIRA) 

 [ 
https://ovirt-jira.atlassian.net/browse/OVIRT-772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Evgheni Dereveanchin updated OVIRT-772:
---
Status: In Progress  (was: To Do)

> Re: Upgrade from 3.6 el7 job broken?
> 
>
> Key: OVIRT-772
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-772
> Project: oVirt - virtualization made easy
>  Issue Type: By-EMAIL
>Reporter: sbonazzo
>Assignee: Evgheni Dereveanchin
>
> On Fri, Oct 14, 2016 at 8:59 AM, Roman Mohr  wrote:
> >
> >
> > On Fri, Oct 14, 2016 at 8:57 AM, Sandro Bonazzola 
> > wrote:
> >
> >>
> >>
> >> On Fri, Oct 14, 2016 at 8:48 AM, Sandro Bonazzola 
> >> wrote:
> >>
> >>>
> >>>
> >>> On Fri, Oct 14, 2016 at 8:44 AM, Roman Mohr  wrote:
> >>>
>  Hi,
> 
>  I saw a lot of failing builds lately on this job:
> 
>    http://jenkins.ovirt.org/job/ovirt-engine_master_upgrade-fro
>  m-3.6_el7_created/
> 
>  One log which still exists is:
> 
>  http://jenkins.ovirt.org/job/ovirt-engine_master_upgrade-fro
>  m-3.6_el7_created/3666/
>  It seems like the BUILD ENGINE RPM step is failing, but I can't see any
>  reason why:
> 
>  BUILDING ENGINE RPM
>  + create_rpms 
>  /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo/ovirt-engine-4.1.0-0.0.master.20161014001903.el7.centos.src.rpm
>   
>  /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo
>   .gitee47dd2
>  + local 
>  src_rpm=/home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo/ovirt-engine-4.1.0-0.0.master.20161014001903.el7.centos.src.rpm
>  + local 
>  dst_dir=/home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo
>  + local release=.gitee47dd2
>  + local 
>  workspace=/home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created
>  + local 'BUILD_JAVA_OPTS_MAVEN=-XX:MaxPermSize=1G 
>  -Dgwt.compiler.localWorkers=1 '
>  + local 'BUILD_JAVA_OPTS_GWT=-XX:PermSize=512M 
>  -XX:MaxPermSize=1G -Xms1G -Xmx6G '
>  + env 'BUILD_JAVA_OPTS_MAVEN=-XX:MaxPermSize=1G 
>  -Dgwt.compiler.localWorkers=1 ' 'BUILD_JAVA_OPTS_GWT=
>  -XX:PermSize=512M -XX:MaxPermSize=1G -Xms1G -Xmx6G ' 
>  rpmbuild -D 'ovirt_build_minimal 1' -D 'release_suffix .gitee47dd2' -D 
>  'ovirt_build_extra_flags -gs 
>  /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/artifactory-ovirt-org-settings.xml'
>   -D '_srcrpmdir 
>  /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo'
>   -D '_specdir 
>  /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo'
>   -D '_sourcedir 
>  /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo'
>   -D '_rpmdir 
>  /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo'
>   -D '_builddir 
>  /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo'
>   --rebuild 
>  /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo/ovirt-engine-4.1.0-0
 .0.master.20161014001903.el7.centos.src.rpm
>  + return 1
>  Build step 'Execute shell' marked build as failure
>  Performing Post build task...
>  Match found for :.* : True
>  Logical operation result is TRUE
>  Running script  : #!/bin/bash -x
> 
> 
>  Best Regards,
> 
> 
> >>>
> >>> Rpmbuild fails on:
> >>> error: Failed build dependencies:
> >>> hystrix-core is needed by ovirt-engine-4.1.0-0.0.master.
> >>> 20161014004451.git32a6f0c.el7.centos.noarch
> >>> hystrix-metrics-event-stream is needed by ovirt-engine-4.1.0-0.0.master.
> >>> 20161014004451.git32a6f0c.el7.centos.noarch
> >>> Installing /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6
> >>> _el7_created/tmp_repo/ovirt-engine-4.1.0-0.0.master.20161014
> >>> 004451.el7.centos.src.rpm
> >>>
> >>> I'll check why those rpms are missing.
> >>>
> >>
> >> Looks like a temporary failure accessing EPEL repository.
> >>
> >>
> > Hm, maybe it happens on some slaves only, but there always? Because it
> > happens pretty often.
> > Here is a good example:
> >
> > https://gerrit.ovirt.org/#/q/status:open+project:ovirt-
> > engine+branch:master+topic:AuditLogableBase-inject
> >
> > All the broken builds are because of this. And that is just after the last
> > rebase from Alon. Before that it was pretty much the same picture.
> >
> Found it. The job is not running in mock and is not installing build
> dependencies. It just relies on the packages installed on the host.
>

[JIRA] (OVIRT-772) Re: Upgrade from 3.6 el7 job broken?

2016-10-14 Thread Evgheni Dereveanchin (oVirt JIRA) 

 [ 
https://ovirt-jira.atlassian.net/browse/OVIRT-772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Evgheni Dereveanchin reassigned OVIRT-772:
--

Assignee: Evgheni Dereveanchin  (was: infra)

> Re: Upgrade from 3.6 el7 job broken?
> 
>
> Key: OVIRT-772
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-772
> Project: oVirt - virtualization made easy
>  Issue Type: By-EMAIL
>Reporter: sbonazzo
>Assignee: Evgheni Dereveanchin
>
> On Fri, Oct 14, 2016 at 8:59 AM, Roman Mohr  wrote:
> >
> >
> > On Fri, Oct 14, 2016 at 8:57 AM, Sandro Bonazzola 
> > wrote:
> >
> >>
> >>
> >> On Fri, Oct 14, 2016 at 8:48 AM, Sandro Bonazzola 
> >> wrote:
> >>
> >>>
> >>>
> >>> On Fri, Oct 14, 2016 at 8:44 AM, Roman Mohr  wrote:
> >>>
>  Hi,
> 
>  I saw a lot of failing builds lately on this job:
> 
>    http://jenkins.ovirt.org/job/ovirt-engine_master_upgrade-fro
>  m-3.6_el7_created/
> 
>  One log which still exists is:
> 
>  http://jenkins.ovirt.org/job/ovirt-engine_master_upgrade-fro
>  m-3.6_el7_created/3666/
>  It seems like the BUILD ENGINE RPM step is failing, but I can't see any
>  reason why:
> 
>  BUILDING ENGINE RPM
>  + create_rpms 
>  /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo/ovirt-engine-4.1.0-0.0.master.20161014001903.el7.centos.src.rpm
>   
>  /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo
>   .gitee47dd2
>  + local 
>  src_rpm=/home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo/ovirt-engine-4.1.0-0.0.master.20161014001903.el7.centos.src.rpm
>  + local 
>  dst_dir=/home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo
>  + local release=.gitee47dd2
>  + local 
>  workspace=/home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created
>  + local 'BUILD_JAVA_OPTS_MAVEN=-XX:MaxPermSize=1G 
>  -Dgwt.compiler.localWorkers=1 '
>  + local 'BUILD_JAVA_OPTS_GWT=-XX:PermSize=512M 
>  -XX:MaxPermSize=1G -Xms1G -Xmx6G '
>  + env 'BUILD_JAVA_OPTS_MAVEN=-XX:MaxPermSize=1G 
>  -Dgwt.compiler.localWorkers=1 ' 'BUILD_JAVA_OPTS_GWT=
>  -XX:PermSize=512M -XX:MaxPermSize=1G -Xms1G -Xmx6G ' 
>  rpmbuild -D 'ovirt_build_minimal 1' -D 'release_suffix .gitee47dd2' -D 
>  'ovirt_build_extra_flags -gs 
>  /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/artifactory-ovirt-org-settings.xml'
>   -D '_srcrpmdir 
>  /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo'
>   -D '_specdir 
>  /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo'
>   -D '_sourcedir 
>  /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo'
>   -D '_rpmdir 
>  /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo'
>   -D '_builddir 
>  /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo'
>   --rebuild 
>  /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo/ovirt-engine-4.1.0-0
 .0.master.20161014001903.el7.centos.src.rpm
>  + return 1
>  Build step 'Execute shell' marked build as failure
>  Performing Post build task...
>  Match found for :.* : True
>  Logical operation result is TRUE
>  Running script  : #!/bin/bash -x
> 
> 
>  Best Regards,
> 
> 
> >>>
> >>> Rpmbuild fails on:
> >>> error: Failed build dependencies:
> >>> hystrix-core is needed by ovirt-engine-4.1.0-0.0.master.
> >>> 20161014004451.git32a6f0c.el7.centos.noarch
> >>> hystrix-metrics-event-stream is needed by ovirt-engine-4.1.0-0.0.master.
> >>> 20161014004451.git32a6f0c.el7.centos.noarch
> >>> Installing /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6
> >>> _el7_created/tmp_repo/ovirt-engine-4.1.0-0.0.master.20161014
> >>> 004451.el7.centos.src.rpm
> >>>
> >>> I'll check why those rpms are missing.
> >>>
> >>
> >> Looks like a temporary failure accessing EPEL repository.
> >>
> >>
> > Hm, maybe it happens on some slaves only, but there always? Because it
> > happens pretty often.
> > Here is a good example:
> >
> > https://gerrit.ovirt.org/#/q/status:open+project:ovirt-
> > engine+branch:master+topic:AuditLogableBase-inject
> >
> > All the broken builds are because of this. And that is just after the last
> > rebase from Alon. Before that it was pretty much the same picture.
> >
> Found it. The job is not running in mock and is not installing build
> dependencies. It just relies on the packages installed

[JIRA] (OVIRT-772) Re: Upgrade from 3.6 el7 job broken?

2016-10-14 Thread Evgheni Dereveanchin (oVirt JIRA) 

[ 
https://ovirt-jira.atlassian.net/browse/OVIRT-772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=21637#comment-21637
 ] 

Evgheni Dereveanchin commented on OVIRT-772:


Hi Sandro,

This was fixed recently as we added hystrix to the respective puppet manifest:
https://gerrit.ovirt.org/#/c/65291/

I checked the failing job and it seems to be related to a single slave vm0026.
It seems to be broken (doesn't even show up in foreman so never applied 
the puppet class to install missing packages). I took it offline and will check 
what's wrong with it. Thanks for bringing this to my attention!

The jira ticket related to modifying the job and making the build part run
in mock and not rely on host packages is here:
https://ovirt-jira.atlassian.net/browse/OVIRT-616

Regards, 
Evgheni Dereveanchin 

- Original Message -
From: "Sandro Bonazzola" 
To: "Roman Mohr" , "Evgheni Dereveanchin" 
, infra-supp...@ovirt.org
Cc: "infra" , "devel" 
Sent: Friday, 14 October, 2016 9:09:10 AM
Subject: Re: Upgrade from 3.6 el7 job broken?

On Fri, Oct 14, 2016 at 8:59 AM, Roman Mohr  wrote:

>
>
> On Fri, Oct 14, 2016 at 8:57 AM, Sandro Bonazzola 
> wrote:
>
>>
>>
>> On Fri, Oct 14, 2016 at 8:48 AM, Sandro Bonazzola 
>> wrote:
>>
>>>
>>>
>>> On Fri, Oct 14, 2016 at 8:44 AM, Roman Mohr  wrote:
>>>
 Hi,

 I saw a lot of failing builds lately on this job:

   http://jenkins.ovirt.org/job/ovirt-engine_master_upgrade-fro
 m-3.6_el7_created/

 One log which still exists is:

 http://jenkins.ovirt.org/job/ovirt-engine_master_upgrade-fro
 m-3.6_el7_created/3666/
 It seems like the BUILD ENGINE RPM step is failing, but I can't see any
 reason why:

 BUILDING ENGINE RPM
 + create_rpms 
 /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo/ovirt-engine-4.1.0-0.0.master.20161014001903.el7.centos.src.rpm
  
 /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo
  .gitee47dd2
 + local 
 src_rpm=/home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo/ovirt-engine-4.1.0-0.0.master.20161014001903.el7.centos.src.rpm
 + local 
 dst_dir=/home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo
 + local release=.gitee47dd2
 + local 
 workspace=/home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created
 + local 'BUILD_JAVA_OPTS_MAVEN=-XX:MaxPermSize=1G 
 -Dgwt.compiler.localWorkers=1 '
 + local 'BUILD_JAVA_OPTS_GWT=-XX:PermSize=512M -XX:MaxPermSize=1G  
-Xms1G -Xmx6G '
 + env 'BUILD_JAVA_OPTS_MAVEN=-XX:MaxPermSize=1G 
 -Dgwt.compiler.localWorkers=1 ' 'BUILD_JAVA_OPTS_GWT=-XX:PermSize=512M 
 -XX:MaxPermSize=1G -Xms1G -Xmx6G ' rpmbuild -D 
 'ovirt_build_minimal 1' -D 'release_suffix .gitee47dd2' -D 
 'ovirt_build_extra_flags -gs 
 /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/artifactory-ovirt-org-settings.xml'
  -D '_srcrpmdir 
 /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo'
  -D '_specdir 
 /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo'
  -D '_sourcedir 
 /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo'
  -D '_rpmdir 
 /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo'
  -D '_builddir 
 /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo'
  --rebuild 
 /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6_el7_created/tmp_repo/ovirt-engine-4.1.0-0.0
 .master.20161014001903.el7.centos.src.rpm
 + return 1
 Build step 'Execute shell' marked build as failure
 Performing Post build task...
 Match found for :.* : True
 Logical operation result is TRUE
 Running script  : #!/bin/bash -x


 Best Regards,


>>>
>>> Rpmbuild fails on:
>>> error: Failed build dependencies:
>>> hystrix-core is needed by ovirt-engine-4.1.0-0.0.master.
>>> 20161014004451.git32a6f0c.el7.centos.noarch
>>> hystrix-metrics-event-stream is needed by ovirt-engine-4.1.0-0.0.master.
>>> 20161014004451.git32a6f0c.el7.centos.noarch
>>> Installing /home/jenkins/workspace/ovirt-engine_master_upgrade-from-3.6
>>> _el7_created/tmp_repo/ovirt-engine-4.1.0-0.0.master.20161014
>>> 004451.el7.centos.src.rpm
>>>
>>> I'll check why those rpms are missing.
>>>
>>
>> Looks like a temporary failure accessing EPEL repository.
>>
>>
> Hm, maybe it happens on some slaves only, but there always? Because it
> happens pretty often.
> Here is a good example:
>
>