On Fri, Mar 1, 2019 at 10:10 AM Dan Kenigsberg <[email protected]> wrote:
> > > On Fri, 1 Mar 2019, 16:23 Greg Sheremeta, <[email protected]> wrote: > >> >> >> On Fri, Mar 1, 2019 at 9:20 AM Dafna Ron <[email protected]> wrote: >> >>> >>> >>> On Fri, Mar 1, 2019 at 2:07 PM Dan Kenigsberg <[email protected]> wrote: >>> >>>> >>>> >>>> On Fri, 1 Mar 2019, 15:48 Dafna Ron, <[email protected]> wrote: >>>> >>>>> Hi Dan, >>>>> >>>>> our intention is never to skip tests and I would not merge a SkipTest >>>>> patch if someone actually shows interest in resolving this. >>>>> if you notice, this was reported a few days ago, after CI determined >>>>> that this is not an infra related issue and since then, but aside from >>>>> Michal asking for logs and getting a link to them, no other communication >>>>> from dev that suggest that this is being checked was sent - even after I >>>>> added [URGENT] and asked if there is any update. >>>>> As we have done in the past, when I see that there is no cooperation >>>>> from the developers to resolve a sporadic test failure, then I add a skip >>>>> test until someone in dev has time to resolve the issue >>>>> >>>> >>>> I believe that this is wrong. You should not merge a Skip before having >>>> a phone call with the relevant team lead. >>>> >>> >>> If we had any info on who to speak with we would probably do so - in >>> cases like this, where there are multiple projects failures and its >>> sporadic, its hard for us to pin someone down. >>> Also, please note that the CI team is here to maintain the >>> infrastructure that allows developers to run the tests and I don't think >>> that we should be chasing after the developers to fix regressions (either >>> in the code or in the tests) and that this should be dev's responsibility. >>> >> >> I can see both sides of the argument, but developers need to do a better >> job of being more responsive to Dafna. She is always extremely diligent >> about reporting these, including follow-ups, and sometimes she doesn't get >> replies. >> > > Note that I want her to do less, not more: identify a suspected team lead, > and tell him/her it is their job to fix it. That's all. > > Phone call? I don't answer my phone unless it's my wife (phone spam in the >> US is unrelenting). >> > > So she should IRC/WhatsApp/gChat you if she thinks you are responsible for > a failure. > > I think the bigger issue is that everyone (ok, most people) filters devel >> and other lists into a label and hardly checks it. IMO that needs to stop. >> > > True. I think that late-evening phone call can help make it happen. > WhatsApp/gChat/IRC are easy enough if there's an issue. I get few enough phonecalls from Israeli country codes that I'd answer anyway, but there's less tracking available. > > Or maybe we need a dedicated ost-failures list that people don't filter. >> > > The same people who filter out URGENT messages from Dafna would easily > filter a new list. It might be easier for them. But none of them should be > a maintainer or team lead. > +1 We really don't need a dedicated list, since all of us read URGENT messages (and I read all of the messages from Dafna). Just on PTO yesterday/today/Monday, so there's a delay in reading the mail if I don't have cellular service. Thanks for handling, Michal. > > That sounds a little better to me than a single dev contact. >> > > Each *test* has a natural dev contact. Host devices are virt, so the > contract here is Michal+Ryan until proven differently or suspected > differently by Dafna. > > >> Greg >> >> >>> >>> I think that it may be time to have a dev contact which would be helping >>> to debug and assign issues such as these and I will raise that in a >>> different thread. >>> >>>> >>>> - not ideal but having different project fail sporadically is not ideal >>>>> as well >>>>> >>>>> As I see that you guys are now responding I have no reason to merge >>>>> the patch - and that is what we all want. >>>>> >>>>> I can say that since its effecting more then one project, and from >>>>> past experience, its either related to the OST test itself or to the >>>>> API/SDK. >>>>> I think that Evgheni ruled out an infra issue and suggested that the >>>>> issue is actually related to the API - so not sure its strictly related to >>>>> Ryan. >>>>> >>>> >>>> I'll trust you here. Though since the failing test is Ryan's, it is >>>> Ryan's job to find what breaks it, and pass the buck. >>>> >>>> If you believe it is a bug in oVirt ask, then you should call mperina, >>>> too. >>>> >>>> >>>>> Thanks, >>>>> Dafna >>>>> >>>>> >>>>> >>>>> On Fri, Mar 1, 2019 at 1:27 PM Dan Kenigsberg <[email protected]> >>>>> wrote: >>>>> >>>>>> I think this would be a bad idea, Dafna. >>>>>> >>>>>> OST exists to find bug, not to skip tests. >>>>>> Michal said that he be working on it. It's time to call him (or Ryan >>>>>> or Shmuel) to hear his finding, not to kill his tests. >>>>>> >>>>>> If you do skip his test, you should file an oVirt bug about it. >>>>>> >>>>>> On Fri, 1 Mar 2019, 11:46 Dafna Ron, <[email protected]> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> As this is failing on average 3 patches a day (from different >>>>>>> projects) and it seems no one is working on debugging this issue, I will >>>>>>> add a patch to skip this test today. >>>>>>> >>>>>>> Thanks, >>>>>>> Dafna >>>>>>> >>>>>>> >>>>>>> On Thu, Feb 28, 2019 at 2:12 PM Dafna Ron <[email protected]> wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> We are still getting random failures on this test. >>>>>>>> Is anyone working on this? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Dafna >>>>>>>> >>>>>>>> >>>>>>>> On Wed, Feb 27, 2019 at 9:10 PM Nir Soffer <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> On Wed, Feb 27, 2019 at 4:31 PM Dan Kenigsberg <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> HTH >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/13163/artifact/basic-suite.el7.x86_64/test_logs/basic-suite-master/post-002_bootstrap.py/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/13163/artifact/basic-suite.el7.x86_64/test_logs/basic-suite-master/post-002_bootstrap.py/lago-basic-suite-master-engine/_var_log/ovirt-engine/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>> Hers is another failure: >>>>>>>>> >>>>>>>>> https://jenkins.ovirt.org/job/ovirt-system-tests_manual/4169/testReport/junit/(root)/002_bootstrap/get_host_devices/ >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Wed, Feb 27, 2019 at 4:24 PM Michal Skrivanek < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 27 Feb 2019, at 14:07, Galit Rosenthal <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> we are failing basic suite master (vdsm, ovirt-engine-sdk ...) >>>>>>>>>>> >>>>>>>>>>> It happens sporadically. >>>>>>>>>>> >>>>>>>>>>> We disqualify that this is an infra issue. >>>>>>>>>>> >>>>>>>>>>> Can you please have a look at the issue? >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> ERROR: >>>>>>>>>>> >>>>>>>>>>> <testcase classname="002_bootstrap" name="get_host_devices" >>>>>>>>>>> time="0.077"> >>>>>>>>>>> <error type="exceptions.RuntimeError" message="*Could not find >>>>>>>>>>> block_vda_1 device in host devices*: "><![CDATA[Traceback (most >>>>>>>>>>> recent call last): >>>>>>>>>>> File "/usr/lib64/python2.7/unittest/case.py", line 369, in run >>>>>>>>>>> testMethod() >>>>>>>>>>> File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, >>>>>>>>>>> in runTest >>>>>>>>>>> self.test(*self.arg) >>>>>>>>>>> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", >>>>>>>>>>> line 142, in wrapped_test >>>>>>>>>>> test() >>>>>>>>>>> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", >>>>>>>>>>> line 60, in wrapper >>>>>>>>>>> return func(get_test_prefix(), *args, **kwargs) >>>>>>>>>>> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", >>>>>>>>>>> line 79, in wrapper >>>>>>>>>>> prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs >>>>>>>>>>> File >>>>>>>>>>> "/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py", >>>>>>>>>>> line 1016, in get_host_devices >>>>>>>>>>> raise RuntimeError('Could not find block_vda_1 device in host >>>>>>>>>>> devices: {}'.format(device_list)) >>>>>>>>>>> RuntimeError: Could not find block_vda_1 device in host devices: >>>>>>>>>>> ]]></error> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Galit >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Examples, Full logs: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> where can I find full logs? It would be interesting to see >>>>>>>>>>> hosts’ logs as well as engine.log >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> michal >>>>>>>>>>> >>>>>>>>>>> [1] >>>>>>>>>>> https://jenkins.ovirt.org/blue/rest/organizations/jenkins/pipelines/ovirt-master_change-queue-tester/runs/13163/nodes/135/steps/286/log/?start=0 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> [2] >>>>>>>>>>> >>>>>>>>>>> https://jenkins.ovirt.org/blue/rest/organizations/jenkins/pipelines/ovirt-master_change-queue-tester/runs/13154/nodes/135/steps/286/log/?start=0 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Infra mailing list -- [email protected] >>>>>>>>>>> To unsubscribe send an email to [email protected] >>>>>>>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>>>>>>> oVirt Code of Conduct: >>>>>>>>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>>>>>>>> List Archives: >>>>>>>>>>> https://lists.ovirt.org/archives/list/[email protected]/message/WXW6II3NYEXHISPI45R4GTE7GXSGF2A7/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> >>>>>>>>>>> GALIT ROSENTHAL >>>>>>>>>>> >>>>>>>>>>> SOFTWARE ENGINEER >>>>>>>>>>> Red Hat >>>>>>>>>>> >>>>>>>>>>> <https://www.redhat.com/> >>>>>>>>>>> >>>>>>>>>>> [email protected] T: 972-9-7692230 >>>>>>>>>>> <https://red.ht/sig> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Devel mailing list -- [email protected] >>>>>>>>>>> To unsubscribe send an email to [email protected] >>>>>>>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>>>>>>> oVirt Code of Conduct: >>>>>>>>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>>>>>>>> List Archives: >>>>>>>>>>> https://lists.ovirt.org/archives/list/[email protected]/message/HF4MB2VMJKBO4O277K46HLPBUPYEXGHW/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Devel mailing list -- [email protected] >>>>>>>>>>> To unsubscribe send an email to [email protected] >>>>>>>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>>>>>>> oVirt Code of Conduct: >>>>>>>>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>>>>>>>> List Archives: >>>>>>>>>>> https://lists.ovirt.org/archives/list/[email protected]/message/YY4MXJ44Q6TJQRFFD6KFSR4FV7OCYATW/ >>>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> Devel mailing list -- [email protected] >>>>>>>>>> To unsubscribe send an email to [email protected] >>>>>>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>>>>>> oVirt Code of Conduct: >>>>>>>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>>>>>>> List Archives: >>>>>>>>>> https://lists.ovirt.org/archives/list/[email protected]/message/KUHLEKMAVQ7NPX6EZKCMY65AIZUWBR7G/ >>>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Devel mailing list -- [email protected] >>>>>>>>> To unsubscribe send an email to [email protected] >>>>>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>>>>> oVirt Code of Conduct: >>>>>>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>>>>>> List Archives: >>>>>>>>> https://lists.ovirt.org/archives/list/[email protected]/message/6HHUAPUKLVLWSAFC4C3ZW5ZOQICL77GB/ >>>>>>>>> >>>>>>>> _______________________________________________ >>> Devel mailing list -- [email protected] >>> To unsubscribe send an email to [email protected] >>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>> oVirt Code of Conduct: >>> https://www.ovirt.org/community/about/community-guidelines/ >>> List Archives: >>> https://lists.ovirt.org/archives/list/[email protected]/message/ERLW4GNWISM5VJGPQ2DUVH6FRQMK3DJH/ >>> >> >> >> -- >> >> GREG SHEREMETA >> >> SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX >> >> Red Hat NA >> >> <https://www.redhat.com/> >> >> [email protected] IRC: gshereme >> <https://red.ht/sig> >> > _______________________________________________ > Devel mailing list -- [email protected] > To unsubscribe send an email to [email protected] > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/[email protected]/message/SLTCK7D6QG2XBX4A6PZGAAF6I4J6OUYT/ > -- Ryan Barry Associate Manager - RHV Virt/SLA [email protected] M: +16518159306 IM: rbarry <https://red.ht/sig>
_______________________________________________ Devel mailing list -- [email protected] To unsubscribe send an email to [email protected] Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/[email protected]/message/BJBMQIBNQ5CY55LCHKOTKQ3S2Y4T5IQD/
