Looks like https://sourceforge.net/p/docutils/bugs/365/
On Sun, Jul 21, 2019 at 11:56 PM Tanay Tummalapalli <[email protected]> wrote: > Hi everyone, > > The Python PreCommit from the Jenkins job "beam_PreCommit_Python_Cron" is > failing[1]. The task :sdks:python:docs is failing with this traceback: > > Traceback (most recent call last): > File > "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Cron/src/sdks/python/build/srcs/sdks/python/target/.tox-docs/docs/bin/sphinx-apidoc", > line 8, in <module> > from sphinx.apidoc import main > File > "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Cron/src/sdks/python/build/srcs/sdks/python/target/.tox-docs/docs/local/lib/python2.7/site-packages/sphinx/apidoc.py", > line 27, in <module> > from sphinx.quickstart import EXTENSIONS > File > "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Cron/src/sdks/python/build/srcs/sdks/python/target/.tox-docs/docs/local/lib/python2.7/site-packages/sphinx/quickstart.py", > line 35, in <module> > from docutils.utils import column_width > File > "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Cron/src/sdks/python/build/srcs/sdks/python/target/.tox-docs/docs/local/lib/python2.7/site-packages/docutils/utils/__init__.py", > line 21, in <module> > import docutils.io > File > "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Cron/src/sdks/python/build/srcs/sdks/python/target/.tox-docs/docs/local/lib/python2.7/site-packages/docutils/io.py", > line 348 > (self.destination.mode, mode)), file=self._stderr) > ^ > SyntaxError: invalid syntax > > It seems to be due to a bug in docutils which is a dependency of sphinx. > Unfortunately, I don't have much context about this. I'd love to help with > fixing this. > > There is another failure in test_wordcount_it of task > :sdks:python:test-suites:dataflow:preCommitIT, > but it seems to be a flaky test. > > [1] > https://scans.gradle.com/s/ygctjiqxcz6qa/console-log?task=:sdks:python:docs > > On Mon, Jul 15, 2019 at 11:16 PM Yifan Zou <[email protected]> wrote: > >> We saw similar timeouts of the python precommit and it usually break the >> Jenkins build workers. I've run the precommit manually several times. It >> sometimes stuck at :sdks:python:docs and consumes 80G+ memory. Our build >> VMs eventually ran out of memory (104G memory available in total) then >> disconnected. Not sure what happened during that job. >> >> >> >> On Sat, Jul 13, 2019 at 10:27 PM Tanay Tummalapalli <[email protected]> >> wrote: >> >>> Yes. It passed on the second attempt. >>> >>> But, I'm yet to figure out why it hangs for ~1.5 hours. >>> >>> On Sun, Jul 14, 2019 at 10:36 AM Rakesh Kumar <[email protected]> >>> wrote: >>> >>>> >>>> >>>> Even I am running into the same issue. Though my test passed but >>>> <https://builds.apache.org/job/beam_PreCommit_Python_Phrase/656/console>somehow >>>> the task didn't terminate, eventually the task was aborted. I have already >>>> tried a couple of times to retrigger tye python precommit but it failed >>>> every time. >>>> >>>> @Tanay did it pass it for you? >>>> >>>> >>>> >>>> >>>> >>>> On Fri, Jul 12, 2019 at 2:22 PM Tanay Tummalapalli <[email protected]> >>>> wrote: >>>> >>>>> Thank You Valentyn! >>>>> >>>>> I'll retest it. >>>>> Hopefully, it's a transient issue. >>>>> >>>>> Regards, >>>>> - Tanay Tummalapalli >>>>> >>>>> On Sat, Jul 13, 2019 at 2:39 AM Valentyn Tymofieiev < >>>>> [email protected]> wrote: >>>>> >>>>>> No, we did not reduce the timeout recently. Looking at console logs, >>>>>> nothing happened for an hour or so, >>>>>> >>>>>> *06:57:50 py27-cython: commands succeeded 06:57:50 congratulations :) >>>>>> 06:57:50 * >>>>>> >>>>>> *06:57:50* >* Task :sdks:python:preCommitPy2**08:22:33* Build timed out >>>>>> (after 120 minutes). Marking the build as aborted. >>>>>> >>>>>> >>>>>> However, we can also see in the logs that py36-cython suite never >>>>>> started, not sure way. I assume gradle waited for this suite to finish. >>>>>> Try "retest this please", hopefully this is a transient gradle issue. >>>>>> I did not observe it before. >>>>>> >>>>>> On Fri, Jul 12, 2019 at 1:22 PM Tanay Tummalapalli < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hi Udi, >>>>>>> >>>>>>> I rebased another PR[1] onto the fix mentioned above. The lint error >>>>>>> is fixed, but, the "beam_PreCommit_Python_Commit" Jenkins job is failing >>>>>>> because of a timeout at 120 minutes[2]. >>>>>>> The log says "Build timed out (after 120 minutes). Marking the >>>>>>> build as aborted." >>>>>>> Another PR's Python PreCommit job aborted with the same error[3]. >>>>>>> >>>>>>> I found this issue - "[BEAM-3040] Python precommit timed out after >>>>>>> 150 minutes"[4]. >>>>>>> Was the timeout reduced recently? >>>>>>> >>>>>>> Regards, >>>>>>> - Tanay Tummalapalli >>>>>>> >>>>>>> [1] https://github.com/apache/beam/pull/8871 >>>>>>> [2] >>>>>>> https://builds.apache.org/job/beam_PreCommit_Python_Commit/7412/consoleFull >>>>>>> >>>>>>> [3] https://github.com/apache/beam/pull/9050 >>>>>>> [4] https://issues.apache.org/jira/browse/BEAM-3040 >>>>>>> >>>>>>> On Fri, Jul 12, 2019 at 5:42 AM Udi Meiri <[email protected]> wrote: >>>>>>> >>>>>>>> This is due to >>>>>>>> https://github.com/apache/beam/pull/8969 >>>>>>>> and >>>>>>>> https://github.com/apache/beam/pull/8934 >>>>>>>> being merged today. >>>>>>>> >>>>>>>> Fix is here: https://github.com/apache/beam/pull/9044 >>>>>>>> >>>>>>>
