Eyal, This was ioprocess issue occurring after the fix was provided. I haven't seen it since build #1389.
Thanks, Piotr On Thu, May 19, 2016 at 3:00 PM, Eyal Edri <[email protected]> wrote: > was that resolved? > any infra issue or was it problems with the tests? > > On Mon, May 16, 2016 at 3:27 PM, Piotr Kliczewski > <[email protected]> wrote: >> >> and one more: >> >> >> http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/1389/console >> >> On Mon, May 16, 2016 at 1:46 PM, Piotr Kliczewski >> <[email protected]> wrote: >> > One more occurrence of the issue [1] >> > >> > >> > [1] >> > http://jenkins.ovirt.org/job/vdsm_master_check-patch-el7-x86_64/1359/console >> > >> > On Sun, May 15, 2016 at 8:37 PM, Nir Soffer <[email protected]> wrote: >> >> The ioprocess issue fixed in https://gerrit.ovirt.org/57473 >> >> >> >> Will be merge soon and available via ovirt-release-master. >> >> >> >> Nir >> >> >> >> On Sun, May 15, 2016 at 7:45 PM, Nir Soffer <[email protected]> wrote: >> >>> Hi all, >> >>> >> >>> I found another stuck build today: >> >>> >> >>> http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/1151/console >> >>> >> >>> 11:27:18 >> >>> ------------------------------------------------------------------------------------------------------------------------------------------------ >> >>> 11:27:18 TOTAL >> >>> 40513 21121 >> >>> 48% >> >>> 11:27:18 >> >>> ---------------------------------------------------------------------- >> >>> 11:27:18 Ran 2169 tests in 145.934s >> >>> 11:27:18 >> >>> 11:27:18 OK (SKIP=88) >> >>> 11:27:18 Exception AttributeError: "'NoneType' object has no attribute >> >>> 'write'" in <bound method IOProcess.__del__ of <ioprocess.IOProcess >> >>> object at 0x7fd7c9f2d3d0>> ignored >> >>> [...] >> >>> 11:27:18 Exception AttributeError: "'NoneType' object has no attribute >> >>> 'write'" in <bound method IOProcess.__del__ of <ioprocess.IOProcess >> >>> object at 0x7fd7c9f15550>> ignored >> >>> 11:27:18 Exception in thread ioprocess communication (6533) (most >> >>> likely raised during interpreter shutdown): >> >>> 11:27:18 Traceback (most recent call last): >> >>> 11:27:18 File "/usr/lib64/python2.7/threading.py", line 804, in >> >>> __bootstrap_inner >> >>> 11:27:18 File "/usr/lib64/python2.7/threading.py", line 757, in run >> >>> 11:27:18 File >> >>> "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 180, in >> >>> _communicate >> >>> 11:27:18 <type 'exceptions.AttributeError'>: 'NoneType' object has no >> >>> attribute 'close' >> >>> >> >>> This seems smells like a non-daemon thread started by some code, >> >>> blocking hte test process. >> >>> >> >>> I suspect ioprocess, starting such thread, looking into it. >> >>> >> >>> Meanwhile, please: >> >>> - verify that all threads in actual code and in the tests are daemon >> >>> threads >> >>> - convert your threads to use vdsm.concurrent.thread instead of >> >>> threading.Thread (daemon by default) >> >>> - watch your builds and abort stuck builds >> >>> >> >>> David, we need a timeout in the ci, aborting the job after a project >> >>> based timeout, maybe >> >>> defined in the project yaml. >> >>> >> >>> Cheers, >> >>> Nir >> >> _______________________________________________ >> >> Devel mailing list >> >> [email protected] >> >> http://lists.ovirt.org/mailman/listinfo/devel >> _______________________________________________ >> Devel mailing list >> [email protected] >> http://lists.ovirt.org/mailman/listinfo/devel >> >> > > > > -- > Eyal Edri > Associate Manager > RHEV DevOps > EMEA ENG Virtualization R&D > Red Hat Israel > > phone: +972-9-7692018 > irc: eedri (on #tlv #rhev-dev #rhev-integ) _______________________________________________ Devel mailing list [email protected] http://lists.ovirt.org/mailman/listinfo/devel
