Sent https://github.com/apache/beam/pull/10874 to remove this flag.
On Tue, Feb 11, 2020 at 5:38 PM jincheng sun <[email protected]> wrote: > I think it's a good advice to remove the "-j 8" option if it doesn't > affect the performance much. > > > Udi Meiri <[email protected]> 于2020年2月12日周三 上午2:20写道： > >> For me the difference was about 20s longer (40s -> 60s approx). Not >> significant IMO >> >> On Tue, Feb 11, 2020 at 9:59 AM Ahmet Altay <[email protected]> wrote: >> >>> Should we remove the "-j 8" option by default? Sphinx docs says this is >>> an experimental option [1]. I do not recall docs generation taking a long >>> time, does this increase significantly without this option? >>> >>> [1] http://www.sphinx-doc.org/en/stable/man/sphinx-build.html >>> >>> On Tue, Feb 11, 2020 at 1:16 AM Shoaib Zafar < >>> [email protected]> wrote: >>> >>>> Thanks, Udi and Jincheng for the response. >>>> The suggested solution worked for me as well. >>>> >>>> Regards, >>>> >>>> *Shoaib Zafar* >>>> Software Engineering Lead >>>> Mobile: +92 333 274 6242 >>>> Skype: live:shoaibzafar_1 >>>> >>>> <http://venturedive.com/> >>>> >>>> >>>> On Tue, Feb 11, 2020 at 1:17 PM jincheng sun <[email protected]> >>>> wrote: >>>> >>>>> I have verified that this issue could be reproduced in my local >>>>> environment (MacOS) and the solution suggested by Udi could work! >>>>> >>>>> Best, >>>>> Jincheng >>>>> >>>>> Udi Meiri <[email protected]> 于2020年2月11日周二 上午8:51写道： >>>>> >>>>>> I don't have those issues (running on Linux), but a possible >>>>>> workaround could be to remove the "-j 8" flags (2 locations) in >>>>>> generate_pydoc.sh. >>>>>> >>>>>> >>>>>> On Mon, Feb 10, 2020 at 11:06 AM Shoaib Zafar < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hello Beamers. >>>>>>> >>>>>>> Just curious does anyone having trouble running >>>>>>> ':sdks:python:test-suites:tox:pycommon:docs' command locally? >>>>>>> >>>>>>> After rebasing with master recently, I am facing sphinx thread fork >>>>>>> error with on my macos mojave, using python 3.7.0. >>>>>>> I Tried to add system variable "export >>>>>>> OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES" (which I found on google) >>>>>>> but no luck! >>>>>>> >>>>>>> Any suggestions/help? >>>>>>> >>>>>>> Thanks! >>>>>>> >>>>>>> Console Log: >>>>>>> ------------------------------------------------------ >>>>>>> .... >>>>>>> Creating file target/docs/source/apache_beam.utils.proto_utils.rst. >>>>>>> Creating file target/docs/source/apache_beam.utils.retry.rst. >>>>>>> Creating file >>>>>>> target/docs/source/apache_beam.utils.subprocess_server.rst. >>>>>>> Creating file >>>>>>> target/docs/source/apache_beam.utils.thread_pool_executor.rst. >>>>>>> Creating file target/docs/source/apache_beam.utils.timestamp.rst. >>>>>>> Creating file target/docs/source/apache_beam.utils.urns.rst. >>>>>>> Creating file target/docs/source/apache_beam.utils.rst. >>>>>>> objc[8384]: +[__NSCFConstantString initialize] may have been in >>>>>>> progress in another thread when fork() was called. >>>>>>> objc[8384]: +[__NSCFConstantString initialize] may have been in >>>>>>> progress in another thread when fork() was called. We cannot safely >>>>>>> call it >>>>>>> or ignore it in the fork() child process. Crashing instead. Set a >>>>>>> breakpoint on objc_initializeAfterForkError to debug. >>>>>>> >>>>>>> Traceback (most recent call last): >>>>>>> File >>>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/cmd/build.py", >>>>>>> line 304, in build_main >>>>>>> app.build(args.force_all, filenames) >>>>>>> File >>>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/application.py", >>>>>>> line 335, in build >>>>>>> self.builder.build_all() >>>>>>> File >>>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py", >>>>>>> line 305, in build_all >>>>>>> self.build(None, summary=__('all source files'), method='all') >>>>>>> File >>>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py", >>>>>>> line 360, in build >>>>>>> updated_docnames = set(self.read()) >>>>>>> File >>>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py", >>>>>>> line 466, in read >>>>>>> self._read_parallel(docnames, nproc=self.app.parallel) >>>>>>> File >>>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py", >>>>>>> line 521, in _read_parallel >>>>>>> tasks.join() >>>>>>> File >>>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/util/parallel.py", >>>>>>> line 114, in join >>>>>>> self._join_one() >>>>>>> File >>>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/util/parallel.py", >>>>>>> line 120, in _join_one >>>>>>> exc, logs, result = pipe.recv() >>>>>>> File >>>>>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py", >>>>>>> line 250, in recv >>>>>>> buf = self._recv_bytes() >>>>>>> File >>>>>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py", >>>>>>> line 407, in _recv_bytes >>>>>>> buf = self._recv(4) >>>>>>> File >>>>>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py", >>>>>>> line 383, in _recv >>>>>>> raise EOFError >>>>>>> EOFError >>>>>>> >>>>>>> Exception occurred: >>>>>>> File >>>>>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py", >>>>>>> line 383, in _recv >>>>>>> raise EOFError >>>>>>> EOFError >>>>>>> The full traceback has been saved in >>>>>>> /Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/tmp/sphinx-err-mphtfnei.log, >>>>>>> if you want to report the issue to the developers. >>>>>>> Please also report this if it was a user error, so that a better >>>>>>> error message can be provided next time. >>>>>>> A bug report can be filed in the tracker at < >>>>>>> https://github.com/sphinx-doc/sphinx/issues>. Thanks! >>>>>>> objc[8385]: +[__NSCFConstantString initialize] may have been in >>>>>>> progress in another thread when fork() was called. >>>>>>> objc[8385]: +[__NSCFConstantString initialize] may have been in >>>>>>> progress in another thread when fork() was called. We cannot safely >>>>>>> call it >>>>>>> or ignore it in the fork() child process. Crashing instead. Set a >>>>>>> breakpoint on objc_initializeAfterForkError to debug. >>>>>>> objc[8386]: +[__NSCFConstantString initialize] may have been in >>>>>>> progress in another thread when fork() was called. >>>>>>> objc[8386]: +[__NSCFConstantString initialize] may have been in >>>>>>> progress in another thread when fork() was called. We cannot safely >>>>>>> call it >>>>>>> or ignore it in the fork() child process. Crashing instead. Set a >>>>>>> breakpoint on objc_initializeAfterForkError to debug. >>>>>>> .... >>>>>>> .... >>>>>>> .... >>>>>>> 9.54 real 4.59 user 1.69 sys >>>>>>> ERROR: InvocationError for command /usr/bin/time >>>>>>> scripts/generate_pydoc.sh (exited with code 2) >>>>>>> py37-docs run-test-post: commands[0] | >>>>>>> /Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/scripts/run_tox_cleanup.sh >>>>>>> ___________________________________ summary >>>>>>> ____________________________________ >>>>>>> ERROR: py37-docs: commands failed >>>>>>> >>>>>>> > Task :sdks:python:test-suites:tox:pycommon:docs FAILED >>>>>>> >>>>>>> FAILURE: Build failed with an exception. >>>>>>> >>>>>>> * What went wrong: >>>>>>> Execution failed for task >>>>>>> ':sdks:python:test-suites:tox:pycommon:docs'. >>>>>>> > Process 'command 'sh'' finished with non-zero exit value 1 >>>>>>> ------------------------------------------------------ >>>>>>> >>>>>>> >>>>>>> >>>>>>> *Shoaib Zafar* >>>>>>> Software Engineering Lead >>>>>>> Mobile: +92 333 274 6242 >>>>>>> Skype: live:shoaibzafar_1 >>>>>>> >>>>>>> <http://venturedive.com/> >>>>>>> >>>>>>
