Obviously I meant "does NOT add any meaningful insight..." On Mon, 2018-09-10 at 16:48 +0200, Marco Varlese wrote: > On Mon, 2018-09-10 at 14:29 +0000, Ed Kern (ejk) wrote: > > At least three different possible actions to take at this point: (outside > > of fixing the issue) > > > > > > > > > > 1. remove make test attempt from arm build (return it to the way it was > > before a week ago). > > > > 2. lower timeout further my first thought would be in the 75 minute range > > (from 120) > I don't like this option mainly because it would still imply that developers > will have to wait for 75 minutes to see a patch verified... The worst is that > the job is a non-voting one hence having it running does add any meaningful > insight into the build result. I'm in favour for the issue being resolved in > the sandbox and eventually moved to production when stable. > > 3. remove job altogether. > I believe this should be the option to purse unless it could be fixed > quickly.As it is it only causes delays to the overall build / review / merging > process... > > > > > > Im happy to push a patch that accomplishes the above or other options I > > haven’t thought of > > for this mail. > > > > > > > > Just let me know.. > > > > > > > > Ed > > > > > > > > > > > > > > > > > > > On Sep 10, 2018, at 4:31 AM, Marco Varlese <mvarl...@suse.de> wrote: > > > > > > Last > > > example (today) can be found here > > > > > > > > > > > > 10:47:06 > > > Not running extended tests (some tests will be skipped) > > > > > > 10:47:06 > > > Perform 4 attempts to pass the suite... > > > > > > 10:47:10 > > > *** Error in `python': double free or corruption (out): > > > > > > 0x0000ffff75d483f0 > > > *** > > > > > > 12:27:54 > > > Build timed out (after 120 minutes). Marking the build as failed. > > > > > > > > > > > > Patch: https://gerrit.fd.io/r/#/c/14744/ > > > > > > > > > > > > Other > > > jobs have finished more than one and half-hour ago and the patch cannot > > > be > > > > > > marked > > > Verified+1 because Jenkins is still waiting for the ARM job to complete > > > > > > (timeout > > > = 120 minutes). IMHO it makes the overall patch submission and review > > > > > > very > > > painful for authors and commiters. > > > > > > > > > > > > I > > > would recommend (it has been done in the past for other jobs) to disable > > > this > > > > > > job > > > in production, move it again to sandbox, get it fixed and eventually > > > moved > > > > > > again > > > to production... > > > > > > > > > > > > - > > > Marco > > > > > > > > > > > > On > > > Fri, 2018-09-07 at 12:04 -0700, Florin Coras wrote: > > > > > > > ARM jobs have not been working for some days now. That’s why their > > > > result is > > > > > > > > skipped. Timeout is 2h but probably we should drop that even further … > > > > > > > > > > > > > > > > Florin > > > > > > > > > > > > > > > > > On Sep 7, 2018, at 11:59 AM, Ole Troan <otr...@employees.org> wrote: > > > > > > > > > > Trying out a change in: > > > > > > > > > > https://gerrit.fd.io/r/#/c/14732/ > > > > > > > > > > > > > > > > > > > > All others succeed but ARM this doesn’t look too good. > > > > > > > > > > Stuck apparently. > > > > > > > > > > > > > > > > > > > > https://jenkins.fd.io/job/vpp-arm-verify-master-ubuntu1604/2157/console > > > > > > > > > > 20:16:04 > > > > > > > > > > ====================================================================== > > > > > ====== > > > > > > > > > > == > > > > > > > > > > > > > > > > > > > > 20:16:04 > > > > > > > > > > ARP Test Case > > > > > > > > > > > > > > > > > > > > 20:16:04 > > > > > > > > > > ====================================================================== > > > > > ====== > > > > > > > > > > == > > > > > > > > > > > > > > > > > > > > 20:16:07 > > > > > > > > > > *** Error in `python': free(): invalid pointer: 0x0000ffff553263a8 *** > > > > > > > > > > > > > > > > > > > > 20:16:09 > > > > > > > > > > ARP > > > > > O > > > > > K > > > > > > > > > > > > > > > > > > > > 20:16:09 > > > > > > > > > > ARP Duplicates > > > > > OK > > > > > > > > > > > > > > > > > > > > 20:16:09 > > > > > > > > > > test_arp_incomplete (test_neighbor.ARPTestCase) > > > > > OK > > > > > > > > > > > > > > > > > > > > 20:16:09 > > > > > > > > > > ARP Static > > > > > OK > > > > > > > > > > > > > > > > > > > > 20:16:15 > > > > > > > > > > ARP reply with VRRP virtual src hw addr > > > > > OK > > > > > > > > > > > > > > > > > > > > 20:16:15 > > > > > > > > > > GARP > > > > > OK > > > > > > > > > > > > > > > > > > > > 20:16:15 > > > > > > > > > > MPLS > > > > > OK > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Cheers, > > > > > > > > > > Ole > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 7 Sep 2018, at 18:50, Ole Troan <otr...@employees.org> wrote: > > > > > > > > > > > > > > > > > > > > > > > > Ed, > > > > > > > > > > > > > > > > > > > > > > > > Let me take a closer look at these. > > > > > > > > > > > > It appears if VPP is slow to start it might not have created the > > > > > > socket > > > > > > > > > > > > yet. Let me try to put in a retry loop and see if that fixes > > > > > > verify. > > > > > > > > > > > > > > > > > > > > > > > > Cheers > > > > > > > > > > > > Ole > > > > > > > > > > > > > > > > > > > > > > > > On 7 Sep 2018, at 18:06, Ed Kern via Lists.Fd.Io < > > > > > > > > > > > > ejk=cisco....@lists.fd.io> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > make test failures due to the below causing pretty consistent > > > > > > > failures. > > > > > > > > > > > > > > Note: For whatever reason the failures are not 100%. > > > > > > > > > > > > > > The failures and with the retries on nonconcurrent merge jobs may > > > > > > > lead > > > > > > > > > > > > > > to long build queues. > > > > > > > > > > > > > > > > > > > > > > > > > > > > These are not infra issues but ill be keeping an eye on it. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Ed > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 13:08:25 > > > > > > > > > > > > > > Using /var/cache/vpp/python/virtualenv/lib/python2.7/site-packages > > > > > > > > > > > > > > > > > > > > > > > > > > > > 13:08:25 > > > > > > > > > > > > > > Finished processing dependencies for vpp-papi==1.6.1 > > > > > > > > > > > > > > > > > > > > > > > > > > > > 13:08:27 > > > > > > > > > > > > > > Traceback (most recent call last): > > > > > > > > > > > > > > > > > > > > > > > > > > > > 13:08:27 > > > > > > > > > > > > > > File "sanity_run_vpp.py", line 21, in <module> > > > > > > > > > > > > > > > > > > > > > > > > > > > > 13:08:27 > > > > > > > > > > > > > > tc.setUpClass() > > > > > > > > > > > > > > > > > > > > > > > > > > > > 13:08:27 > > > > > > > > > > > > > > File > > > > > > > "/w/workspace/vpp-merge-master-ubuntu1604/test/framework.py", > > > > > > > line > > > > > > > > > > > > > > 394, in setUpClass > > > > > > > > > > > > > > > > > > > > > > > > > > > > 13:08:27 > > > > > > > > > > > > > > cls.statistics = VPPStats(socketname=cls.tempdir+'/stats.sock') > > > > > > > > > > > > > > > > > > > > > > > > > > > > 13:08:27 > > > > > > > > > > > > > > File "build/bdist.linux-x86_64/egg/vpp_papi/vpp_stats.py", line > > > > > > > 117, in > > > > > > > > > > > > > > __init__ > > > > > > > > > > > > > > > > > > > > > > > > > > > > 13:08:27 > > > > > > > > > > > > > > IOError > > > > > > > > > > > > > > > > > > > > > > > > > > > > 13:08:27 > > > > > > > > > > > > > > ****************************************************************** > > > > > > > * > > > > > > > > > > > > > > > > > > > > > > > > > > > > 13:08:27 > > > > > > > > > > > > > > * Sanity check failed, cannot run vpp > > > > > > > > > > > > > > > > > > > > > > > > > > > > 13:08:27 > > > > > > > > > > > > > > ****************************************************************** > > > > > > > * > > > > > > > > > > > > > > > > > > > > > > > > > > > > -=-=-=-=-=-=-=-=-=-=-=- > > > > > > > > > > > > > > Links: You receive all messages sent to this group. > > > > > > > > > > > > > > > > > > > > > > > > > > > > View/Reply Online (#10432): > > > > > > > https://lists.fd.io/g/vpp-dev/message/10432 > > > > > > > > > > > > > > Mute This Topic: https://lists.fd.io/mt/25308161/675193 > > > > > > > > > > > > > > Group Owner: vpp-dev+ow...@lists.fd.io > > > > > > > > > > > > > > Unsubscribe: https://lists.fd.io/g/vpp-dev/unsubb [otroan@employe > > > > > > > es.org > > > > > > > > > > > > > > ] > > > > > > > > > > > > > > -=-=-=-=-=-=-=-=-=-=-=- > > > > > > > > > > > > > > > > > > > -=-=-=-=-=-=-=-=-=-=-=- > > > > > > > > > > > > Links: You receive all messages sent to this group. > > > > > > > > > > > > > > > > > > > > > > > > View/Reply Online (#10435): > > > > > > https://lists.fd.io/g/vpp-dev/message/10435 > > > > > > > > > > > > Mute This Topic: https://lists.fd.io/mt/25308161/675193 > > > > > > > > > > > > Group Owner: vpp-dev+ow...@lists.fd.io > > > > > > > > > > > > Unsubscribe: https://lists.fd.io/g/vpp-dev/unsubbb [otroan@employee > > > > > > s.org] > > > > > > > > > > > > -=-=-=-=-=-=-=-=-=-=-=- > > > > > > > > > > > > > > > > > > > > > -=-=-=-=-=-=-=-=-=-=-=- > > > > > > > > > > Links: You receive all messages sent to this group. > > > > > > > > > > > > > > > > > > > > View/Reply Online (#10437): > > > > > https://lists.fd.io/g/vpp-dev/message/10437 > > > > > > > > > > Mute This Topic: https://lists.fd.io/mt/25308161/675152 > > > > > > > > > > Group Owner: vpp-dev+ow...@lists.fd.io > > > > > > > > > > Unsubscribe: https://lists.fd.io/g/vpp-dev/unsubbb [fcoras.lists@gmai > > > > > l.com] > > > > > > > > > > -=-=-=-=-=-=-=-=-=-=-=- > > > > > > > > > > > > > > > > > -=-=-=-=-=-=-=-=-=-=-=- > > > > > > > > Links: You receive all messages sent to this group. > > > > > > > > > > > > > > > > View/Reply Online (#10438): https://lists.fd.io/g/vpp-dev/message/10438 > > > > > > > > Mute This Topic: https://lists.fd.io/mt/25308161/675056 > > > > > > > > Group Owner: vpp-dev+ow...@lists.fd.io > > > > > > > > Unsubscribe: https://lists.fd.io/g/vpp-dev/unsubbb [mvarl...@suse.de] > > > > > > > > -=-=-=-=-=-=-=-=-=-=-=- > > > > > > > > > > > > > > > > > > -- > > > > > > Marco > > > V > > > > > > > > > > > > SUSE > > > LINUX GmbH | GF: Felix Imendörffer, Jane Smithard, Graham Norton > > > > > > HRB > > > 21284 (AG Nürnberg) Maxfeldstr. 5, D-90409, Nürnberg > > > > > > > > > > > > -=-=-=-=-=-=-=-=-=-=-=- > > > > > > Links: > > > You receive all messages sent to this group. > > > > > > > > > > > > View/Reply > > > Online (#10453): https://lists.fd.io/g/vpp-dev/message/10453 > > > > > > Mute > > > This Topic: https://lists.fd.io/mt/25308161/675649 > > > > > > Group > > > Owner: vpp-dev+ow...@lists.fd.io > > > > > > Unsubscribe: https://lists.fd.io/g/vpp-dev/unsubb [e...@cisco.com] > > > > > > -=-=-=-=-=-=-=-=-=-=-=- > > > > > > > > > > > > > > > -- > Marco V > > > SUSE LINUX GmbH | GF: Felix Imendörffer, Jane Smithard, Graham Norton > HRB 21284 (AG Nürnberg) Maxfeldstr. 5, D-90409, Nürnberg -- Marco V
SUSE LINUX GmbH | GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg) Maxfeldstr. 5, D-90409, Nürnberg
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#10461): https://lists.fd.io/g/vpp-dev/message/10461 Mute This Topic: https://lists.fd.io/mt/25308161/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
