[ovirt-devel] Announcing 'COVERAGE' option in OST

2018-12-12 Thread Marcin Sobczyk 
Hi,

I've been working on adding coverage report for VDSM in OST
recently, and I'm happy to announce, that the first batch of patches is
merged!

To run a suite with coverage, look for 'COVERAGE' drop-down on OST's
build parameters page. If you run OST locally, pass a '--coverage' argument
to 'run_suite.sh'.

Currently, coverage works only for VDSM in basic-suite-master, but adding
VDSM support for other suites is now a no-brainer. More patches are on the
way!

Since the option is named 'COVERAGE', and not 'VDSM_COVERAGE', other
projects
are welcome to implement their coverage reports on top of it.

Cheers, Marcin
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/UDHD4SQLIGELWYZDBNEDC223DX36L6RT/

[ovirt-devel] Re: Announcing 'COVERAGE' option in OST

2018-12-12 Thread Edward Haas 
Thank you Marcin, this is very nice.

Thanks,
Edy.

On Wed, Dec 12, 2018 at 9:59 PM Nir Soffer  wrote:

> On Wed, Dec 12, 2018 at 3:42 PM Marcin Sobczyk 
> wrote:
>
>> Hi,
>>
>> I've been working on adding coverage report for VDSM in OST
>> recently, and I'm happy to announce, that the first batch of patches is
>> merged!
>>
>> To run a suite with coverage, look for 'COVERAGE' drop-down on OST's
>> build parameters page. If you run OST locally, pass a '--coverage'
>> argument to 'run_suite.sh'.
>>
>> Currently, coverage works only for VDSM in basic-suite-master, but adding
>> VDSM support for other suites is now a no-brainer. More patches are on
>> the way!
>>
>> Since the option is named 'COVERAGE', and not 'VDSM_COVERAGE', other
>> projects
>> are welcome to implement their coverage reports on top of it.
>>
>> Cheers, Marcin
>>
>
> Awesome, thanks for the update!
>
> Nir
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/ULSXRX4QJJHGPT6HHO24PJLPTRON7TF3/

[ovirt-devel] Re: Announcing 'COVERAGE' option in OST

2018-12-12 Thread Martin Perina 
Great work, Marcin, thanks a lot!


On Wed, 12 Dec 2018, 22:05 Edward Haas  Thank you Marcin, this is very nice.
>
> Thanks,
> Edy.
>
> On Wed, Dec 12, 2018 at 9:59 PM Nir Soffer  wrote:
>
>> On Wed, Dec 12, 2018 at 3:42 PM Marcin Sobczyk 
>> wrote:
>>
>>> Hi,
>>>
>>> I've been working on adding coverage report for VDSM in OST
>>> recently, and I'm happy to announce, that the first batch of patches is
>>> merged!
>>>
>>> To run a suite with coverage, look for 'COVERAGE' drop-down on OST's
>>> build parameters page. If you run OST locally, pass a '--coverage'
>>> argument to 'run_suite.sh'.
>>>
>>> Currently, coverage works only for VDSM in basic-suite-master, but adding
>>> VDSM support for other suites is now a no-brainer. More patches are on
>>> the way!
>>>
>>> Since the option is named 'COVERAGE', and not 'VDSM_COVERAGE', other
>>> projects
>>> are welcome to implement their coverage reports on top of it.
>>>
>>> Cheers, Marcin
>>>
>>
>> Awesome, thanks for the update!
>>
>> Nir
>>
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/TLR545QSPMFYBCEJ7DKSO55JHIKO7RAG/

[ovirt-devel] Re: Announcing 'COVERAGE' option in OST

2018-12-12 Thread Dan Kenigsberg 
On Wed, 12 Dec 2018, 22:42 Marcin Sobczyk  Hi,
>
> I've been working on adding coverage report for VDSM in OST
> recently, and I'm happy to announce, that the first batch of patches is
> merged!
>
> To run a suite with coverage, look for 'COVERAGE' drop-down on OST's
> build parameters page. If you run OST locally, pass a '--coverage'
> argument to 'run_suite.sh'.
>
> Currently, coverage works only for VDSM in basic-suite-master, but adding
> VDSM support for other suites is now a no-brainer. More patches are on the
> way!
>
> Since the option is named 'COVERAGE', and not 'VDSM_COVERAGE', other
> projects
> are welcome to implement their coverage reports on top of it.
>
> Cheers, Marcin
>

Awesome indeed. I was not aware there's a way to add such parameters to
suites. It could come up handy elsewhere (e.g. choosing ovs for cluster
switchtype)

I think it is important to explain why it's so important.

Unfortunately, much of vdsm functionalily is not covered by proper vdsm
unit tests. We depend on OST to define what is a "working" ovirt.

Here's a message for us vdsm developers: if you care for your code and the
time you've invested into it, make sure that it shows up green on the likes
of
https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests_manual/3754/artifact/exported-artifacts/coverage/vdsm/html/index.html

The total number - 59% - is not brilliant, but not as bad as I assumed.

Please go over the report. There are glaring missing pieces (e.g gluster,
fencing) but also less obvious ones (almost half of vm.py).

It should give you ideas about tests you need to add, or code you can drop.
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/LHKILZ4BKUCR7U7Y6GYQSKPIPWPS45JZ/

[ovirt-devel] Re: Announcing 'COVERAGE' option in OST

2018-12-12 Thread Nir Soffer 
On Wed, Dec 12, 2018 at 3:42 PM Marcin Sobczyk  wrote:

> Hi,
>
> I've been working on adding coverage report for VDSM in OST
> recently, and I'm happy to announce, that the first batch of patches is
> merged!
>
> To run a suite with coverage, look for 'COVERAGE' drop-down on OST's
> build parameters page. If you run OST locally, pass a '--coverage'
> argument to 'run_suite.sh'.
>
> Currently, coverage works only for VDSM in basic-suite-master, but adding
> VDSM support for other suites is now a no-brainer. More patches are on the
> way!
>
> Since the option is named 'COVERAGE', and not 'VDSM_COVERAGE', other
> projects
> are welcome to implement their coverage reports on top of it.
>
> Cheers, Marcin
>

Awesome, thanks for the update!

Nir
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/U3O3CA423JPNRIQWYKVI4U5QIGUQBJS3/

[ovirt-devel] Re: Announcing 'COVERAGE' option in OST

2018-12-12 Thread Piotr Kliczewski 
Awsome job! Thank you!

śr., 12 gru 2018, 22:41: Dan Kenigsberg  napisał(a):

>
>
> On Wed, 12 Dec 2018, 22:42 Marcin Sobczyk 
>> Hi,
>>
>> I've been working on adding coverage report for VDSM in OST
>> recently, and I'm happy to announce, that the first batch of patches is
>> merged!
>>
>> To run a suite with coverage, look for 'COVERAGE' drop-down on OST's
>> build parameters page. If you run OST locally, pass a '--coverage'
>> argument to 'run_suite.sh'.
>>
>> Currently, coverage works only for VDSM in basic-suite-master, but adding
>> VDSM support for other suites is now a no-brainer. More patches are on
>> the way!
>>
>> Since the option is named 'COVERAGE', and not 'VDSM_COVERAGE', other
>> projects
>> are welcome to implement their coverage reports on top of it.
>>
>> Cheers, Marcin
>>
>
> Awesome indeed. I was not aware there's a way to add such parameters to
> suites. It could come up handy elsewhere (e.g. choosing ovs for cluster
> switchtype)
>
> I think it is important to explain why it's so important.
>
> Unfortunately, much of vdsm functionalily is not covered by proper vdsm
> unit tests. We depend on OST to define what is a "working" ovirt.
>
> Here's a message for us vdsm developers: if you care for your code and the
> time you've invested into it, make sure that it shows up green on the likes
> of
> https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests_manual/3754/artifact/exported-artifacts/coverage/vdsm/html/index.html
>
> The total number - 59% - is not brilliant, but not as bad as I assumed.
>
> Please go over the report. There are glaring missing pieces (e.g gluster,
> fencing) but also less obvious ones (almost half of vm.py).
>
> It should give you ideas about tests you need to add, or code you can drop.
>
>
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/KH7WUU4G43W2HCNUFGC4F2DNIHAVOTTF/

[ovirt-devel] Re: ovirt-system-tests_compat-4.2-suite-master is failing since November 27

2018-12-12 Thread Eyal Edri 
On Tue, Dec 11, 2018 at 9:41 AM Sandro Bonazzola 
wrote:

>
>
> Il giorno mar 11 dic 2018 alle ore 08:36 Dan Kenigsberg 
> ha scritto:
>
>>
>>
>> On Tue, Dec 11, 2018 at 9:15 AM Dan Kenigsberg  wrote:
>>
>>>
>>>
>>> On Tue, Dec 11, 2018 at 8:57 AM Sandro Bonazzola 
>>> wrote:
>>>


 Il giorno lun 10 dic 2018 alle ore 19:42 Dan Kenigsberg <
 dan...@redhat.com> ha scritto:

>
>
> On Mon, 10 Dec 2018, 19:17 Dominik Holler 
>> On Mon, 10 Dec 2018 12:02:21 -0500
>> Greg Sheremeta  wrote:
>>
>> > On Mon, Dec 10, 2018 at 11:53 AM Sandro Bonazzola <
>> sbona...@redhat.com>
>> > wrote:
>> >
>> > > hi, ovirt-system-tests_compat-4.2-suite-master is failing since
>> November
>> > > 27 with following error:
>> > >
>> > > Error: Fault reason is "Operation Failed". Fault detail is "[Bond
>> name 'bond_fancy0' does not follow naming criteria. For host 
>> compatibility
>> version 4.3 or greater, custom bond names must begin with the prefix
>> 'bond'  followed by 'a-z', 'A-Z', '0-9', or '_' characters. For host
>> compatibility version 4.2 and lower, custom bond names must begin with 
>> the
>> prefix 'bond' followed by a number.]". HTTP response code is 400.
>> > >
>> > >
>> > > I think that if the scope of the test is to check that 4.2 still
>> works
>> > > with 4.3 engine, bond_fancy0 works for 4.3 but it's clearly not
>> good for
>> > > 4.2 and the test needs a fix.
>> > >
>> >
>>
>> Gal, what is the most correct way to fix this?
>> Should the BOND_NAME derived from the versioning.cluster_version()?
>>
>
> as https://gerrit.ovirt.org/#/c/95866/ suggested for another reason
>

 Is this covering engine 4.3 with cluster 4.2?

>>>
>>> I cannot say that I understand these compat-* suite - we'd need to wait
>>> to Gal for that.
>>> But I'm guessing that it would work. Let's see:
>>> https://jenkins.ovirt.org/job/ovirt-system-tests_manual/3736/console
>>>
>>
>> This has failed early on deploy of ovirt-host. Any idea why nbdkit* and
>> newer rpm are required but unavailable?
>>
>
> I guess it's the usual issue due to repository blacklisting / whitelisting
> used by OST.
> Adding Galith and Dafna to check this.
>

I see the patch https://gerrit.ovirt.org/#/c/95866/ passed CI, does it fix
the compat suites? can we merge it?



>
>
>>
>> *07:19:43* + yum -y install ovirt-host*07:19:43* Error: Package: 
>> 1:virt-v2v-1.38.2-12.el7_6.1.x86_64 (alocalsync)*07:19:43*
>> Requires: nbdkit-plugin-python2*07:19:43* Error: Package: 
>> elfutils-0.172-2.el7.x86_64 (alocalsync)*07:19:43*Requires: 
>> libdw.so.1(ELFUTILS_0.171)(64bit)*07:19:43* Error: Package: 
>> libvirt-client-4.5.0-10.el7_6.3.x86_64 (alocalsync)*07:19:43*
>> Requires: libvirt-bash-completion = 4.5.0-10.el7_6.3*07:19:43* Error: 
>> Package: 1:openssl-1.0.2k-16.el7.x86_64 (alocalsync)*07:19:43*
>> Requires: openssl-libs(x86-64) = 1:1.0.2k-16.el7*07:19:43*
>> Installed: 1:openssl-libs-1.0.2k-12.el7.x86_64 (installed)*07:19:43* 
>>openssl-libs(x86-64) = 1:1.0.2k-12.el7*07:19:43* Error: Package: 
>> cryptsetup-2.0.3-3.el7.x86_64 (alocalsync)*07:19:43*Requires: 
>> cryptsetup-libs(x86-64) = 2.0.3-3.el7*07:19:43*Installed: 
>> cryptsetup-libs-1.7.4-4.el7.x86_64 (installed)*07:19:43*
>> cryptsetup-libs(x86-64) = 1.7.4-4.el7*07:19:43* Error: Package: 
>> libsemanage-python-2.5-14.el7.x86_64 (alocalsync)*07:19:43*
>> Requires: libsemanage = 2.5-14.el7*07:19:43*Installed: 
>> libsemanage-2.5-11.el7.x86_64 (installed)*07:19:43*
>> libsemanage = 2.5-11.el7*07:19:43* Error: Package: 
>> libblockdev-crypto-2.18-3.el7.x86_64 (alocalsync)*07:19:43*
>> Requires: libcryptsetup.so.12()(64bit)*07:19:43* Error: Package: 
>> kernel-3.10.0-957.1.3.el7.x86_64 (alocalsync)*07:19:43*Requires: 
>> linux-firmware >= 20180911-68*07:19:43*Available: 
>> linux-firmware-20180220-62.2.git6d51311.el7_5.noarch (alocalsync)*07:19:43*  
>>   linux-firmware = 20180220-62.2.git6d51311.el7_5*07:19:43* 
>> Error: Package: setools-libs-3.3.8-4.el7.x86_64 (alocalsync)*07:19:43*   
>>  Requires: libselinux >= 2.5-14.1*07:19:43*Installed: 
>> libselinux-2.5-12.el7.x86_64 (installed)*07:19:43*libselinux 
>> = 2.5-12.el7*07:19:43* Error: Package: 
>> 7:device-mapper-event-1.02.149-8.el7.x86_64 (alocalsync)*07:19:43*   
>>  Requires: device-mapper = 7:1.02.149-8.el7*07:19:43*Installed: 
>> 7:device-mapper-1.02.146-4.el7.x86_64 (installed)*07:19:43*
>> device-mapper = 7:1.02.146-4.el7*07:19:43* Error: kernel conflicts with 
>> selinux-policy-targeted-3.13.1-192.el7_5.3.noarch*07:19:43* Error: Package: 
>> setools-libs-3.3.8-4.el7.x86_64 (alocalsync)*07:19:43*

[ovirt-devel] Re: ovirt-system-tests_compat-4.2-suite-master is failing since November 27

2018-12-12 Thread Eyal Edri 
Merged.
Not sure why the manual job is failing on missing pkgs, AFAIK they use the
same file as all the other suites and its working well,
We are going to replace the 7.6 custom image we used with the official one
Richard releases this week, but its currently working.

Anyway, i've re-triggered the compat suite now, let's see if its fixed.
https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.2-suite-master/

On Wed, Dec 12, 2018 at 10:39 AM Dan Kenigsberg  wrote:

>
>
> On Wed, Dec 12, 2018 at 10:32 AM Eyal Edri  wrote:
>
>>
>>
>> On Tue, Dec 11, 2018 at 9:41 AM Sandro Bonazzola 
>> wrote:
>>
>>>
>>>
>>> Il giorno mar 11 dic 2018 alle ore 08:36 Dan Kenigsberg <
>>> dan...@redhat.com> ha scritto:
>>>


 On Tue, Dec 11, 2018 at 9:15 AM Dan Kenigsberg 
 wrote:

>
>
> On Tue, Dec 11, 2018 at 8:57 AM Sandro Bonazzola 
> wrote:
>
>>
>>
>> Il giorno lun 10 dic 2018 alle ore 19:42 Dan Kenigsberg <
>> dan...@redhat.com> ha scritto:
>>
>>>
>>>
>>> On Mon, 10 Dec 2018, 19:17 Dominik Holler >>
 On Mon, 10 Dec 2018 12:02:21 -0500
 Greg Sheremeta  wrote:

 > On Mon, Dec 10, 2018 at 11:53 AM Sandro Bonazzola <
 sbona...@redhat.com>
 > wrote:
 >
 > > hi, ovirt-system-tests_compat-4.2-suite-master is failing since
 November
 > > 27 with following error:
 > >
 > > Error: Fault reason is "Operation Failed". Fault detail is
 "[Bond name 'bond_fancy0' does not follow naming criteria. For host
 compatibility version 4.3 or greater, custom bond names must begin 
 with the
 prefix 'bond'  followed by 'a-z', 'A-Z', '0-9', or '_' characters. For 
 host
 compatibility version 4.2 and lower, custom bond names must begin with 
 the
 prefix 'bond' followed by a number.]". HTTP response code is 400.
 > >
 > >
 > > I think that if the scope of the test is to check that 4.2
 still works
 > > with 4.3 engine, bond_fancy0 works for 4.3 but it's clearly not
 good for
 > > 4.2 and the test needs a fix.
 > >
 >

 Gal, what is the most correct way to fix this?
 Should the BOND_NAME derived from the versioning.cluster_version()?

>>>
>>> as https://gerrit.ovirt.org/#/c/95866/ suggested for another reason
>>>
>>
>> Is this covering engine 4.3 with cluster 4.2?
>>
>
> I cannot say that I understand these compat-* suite - we'd need to
> wait to Gal for that.
> But I'm guessing that it would work. Let's see:
> https://jenkins.ovirt.org/job/ovirt-system-tests_manual/3736/console
>

 This has failed early on deploy of ovirt-host. Any idea why nbdkit* and
 newer rpm are required but unavailable?

>>>
>>> I guess it's the usual issue due to repository blacklisting /
>>> whitelisting used by OST.
>>> Adding Galith and Dafna to check this.
>>>
>>
>> I see the patch https://gerrit.ovirt.org/#/c/95866/ passed CI, does it
>> fix the compat suites?
>>
>
> no idea, because compat suite is failing on missing packages.
>
> can we merge it?
>>
>
> yes, please.
>
>

-- 

Eyal edri


MANAGER

RHV/CNV DevOps

EMEA VIRTUALIZATION R


Red Hat EMEA 
 TRIED. TESTED. TRUSTED. 
phone: +972-9-7692018
irc: eedri (on #tlv #rhev-dev #rhev-integ)
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/T762IIWRACKMWQNYP4UD7J3HOVMX37TD/

[ovirt-devel] Re: ovirt-system-tests_compat-4.2-suite-master is failing since November 27

2018-12-12 Thread Dan Kenigsberg 
On Wed, Dec 12, 2018 at 10:32 AM Eyal Edri  wrote:

>
>
> On Tue, Dec 11, 2018 at 9:41 AM Sandro Bonazzola 
> wrote:
>
>>
>>
>> Il giorno mar 11 dic 2018 alle ore 08:36 Dan Kenigsberg <
>> dan...@redhat.com> ha scritto:
>>
>>>
>>>
>>> On Tue, Dec 11, 2018 at 9:15 AM Dan Kenigsberg 
>>> wrote:
>>>


 On Tue, Dec 11, 2018 at 8:57 AM Sandro Bonazzola 
 wrote:

>
>
> Il giorno lun 10 dic 2018 alle ore 19:42 Dan Kenigsberg <
> dan...@redhat.com> ha scritto:
>
>>
>>
>> On Mon, 10 Dec 2018, 19:17 Dominik Holler >
>>> On Mon, 10 Dec 2018 12:02:21 -0500
>>> Greg Sheremeta  wrote:
>>>
>>> > On Mon, Dec 10, 2018 at 11:53 AM Sandro Bonazzola <
>>> sbona...@redhat.com>
>>> > wrote:
>>> >
>>> > > hi, ovirt-system-tests_compat-4.2-suite-master is failing since
>>> November
>>> > > 27 with following error:
>>> > >
>>> > > Error: Fault reason is "Operation Failed". Fault detail is
>>> "[Bond name 'bond_fancy0' does not follow naming criteria. For host
>>> compatibility version 4.3 or greater, custom bond names must begin with 
>>> the
>>> prefix 'bond'  followed by 'a-z', 'A-Z', '0-9', or '_' characters. For 
>>> host
>>> compatibility version 4.2 and lower, custom bond names must begin with 
>>> the
>>> prefix 'bond' followed by a number.]". HTTP response code is 400.
>>> > >
>>> > >
>>> > > I think that if the scope of the test is to check that 4.2 still
>>> works
>>> > > with 4.3 engine, bond_fancy0 works for 4.3 but it's clearly not
>>> good for
>>> > > 4.2 and the test needs a fix.
>>> > >
>>> >
>>>
>>> Gal, what is the most correct way to fix this?
>>> Should the BOND_NAME derived from the versioning.cluster_version()?
>>>
>>
>> as https://gerrit.ovirt.org/#/c/95866/ suggested for another reason
>>
>
> Is this covering engine 4.3 with cluster 4.2?
>

 I cannot say that I understand these compat-* suite - we'd need to wait
 to Gal for that.
 But I'm guessing that it would work. Let's see:
 https://jenkins.ovirt.org/job/ovirt-system-tests_manual/3736/console

>>>
>>> This has failed early on deploy of ovirt-host. Any idea why nbdkit* and
>>> newer rpm are required but unavailable?
>>>
>>
>> I guess it's the usual issue due to repository blacklisting /
>> whitelisting used by OST.
>> Adding Galith and Dafna to check this.
>>
>
> I see the patch https://gerrit.ovirt.org/#/c/95866/ passed CI, does it
> fix the compat suites?
>

no idea, because compat suite is failing on missing packages.

can we merge it?
>

yes, please.
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/VVR6UH7GM2BJ47PU2UZEFQUI4R6PTVAL/