On Tue, Jul 9, 2019 at 5:34 PM Julien Palard via Python-Dev < python-dev@python.org> wrote:
> Hi Christian, > > > the table with auditing events does not render on docs.python.org, > > https://docs.python.org/3.9/library/audit_events.html. Steve and I are > > going to present the auditing feature tomorrow at EuroPython. It would > > be helpful to have the table available. > > This was not an easy one... and it may be a Sphinx issue, yet I'm still > not sure, maybe Steve can shed some light on it: > > It's the "-j" option of sphinx-build (to parallelize) that causes the > issue. I double checked it (full commands at the end of the message in case > someone want to reproduce it): > > - Run with -j4 → No table > - Run without -j → Table is here > - Run again with -j4 → No table! > - Run again without -j → Table is back! > I can reproduce the same with table not appearing in parallel builds and table is successfully built on non-parallel builds. On adding -j4 to ALLSPHINXOPTS in my Makefile in Docs/ folder there is no table generated. Currently, the audit events are collected in a dictionary and are stored in self.state.document.settings.env that is initialized with empty dict in AuditEvent directive [0]. * On non-parallel builds when I use print statement with hasattr to check for all_audit_events, it's False for first time and then remains True with tuple set for rest of the collection. In process_audit_events for non-parallel builds the attribute is set with a dictionary of all audit events. * On parallel builds when I use print statement with hasattr to check for all_audit_events, it returns True and False randomly throughout the build. I guess it's due to attribute not being properly initialized. In process_audit_events for parallel builds the all_audit_events attribute is not at all set during table creation and falls back to empty tuple writing no rows to the table. I am not sure if modifying the env attribute on parallel builds is safe since I couldn't find any guarantees from the sphinx docs. The default docs Makefile is not parallel which could be the reason it was not reproducible locally. Thanks for fixing this. [0] https://github.com/python/cpython/blob/73ea54620a6f91c3f2e53880373dd47813691a21/Doc/tools/extensions/pyspecific.py#L187 > I'm patching docsbuild-scripts to stop using -j4 with is not really > helpfull anyway as docsbuild script is parallelizing by starting multiple > sphinx-build (for multiple languages / versions). > > I also copied the file and invalidated the cache, so > https://docs.python.org/3.9/library/audit_events.html is good again. > > If I'm too slow testing locally and releasing a new docsbuild_script.py, > the cron MAY break the file again, don't hesitate to poke me if it happen > without me noticing first. > > Full test: > > docsbuild@docs:/srv/docsbuild/3.9/cpython-en/Doc$ > /srv/docsbuild/venv/bin/sphinx-build -b html -d build/doctrees -D > latex_elements.papersize= -D latex_engine=xelatex -D > latex_elements.inputenc= -D latex_elements.fontenc= -j4 -q -Ea -A daily=1 > -A switchers=1 . build/html > docsbuild@docs:/srv/docsbuild/3.9/cpython-en/Doc$ grep breakpoint > build/html/library/audit_events.html > docsbuild@docs:/srv/docsbuild/3.9/cpython-en/Doc$ > /srv/docsbuild/venv/bin/sphinx-build -b html -d build/doctrees -D > latex_elements.papersize= -D latex_engine=xelatex -D > latex_elements.inputenc= -D latex_elements.fontenc= -q -Ea -A daily=1 -A > switchers=1 . build/html > docsbuild@docs:/srv/docsbuild/3.9/cpython-en/Doc$ grep breakpoint > build/html/library/audit_events.html > <tr class="row-odd"><td><p>builtins.breakpoint</p></td> > <td><p><code class="docutils literal notranslate"><span > class="pre">breakpointhook</span></code></p></td> > <td><p><a class="reference internal" > href="functions.html#breakpoint">[1]</a></p></td> > docsbuild@docs:/srv/docsbuild/3.9/cpython-en/Doc$ > /srv/docsbuild/venv/bin/sphinx-build -b html -d build/doctrees -D > latex_elements.papersize= -D latex_engine=xelatex -D > latex_elements.inputenc= -D latex_elements.fontenc= -j4 -q -Ea -A daily=1 > -A switchers=1 . build/html > docsbuild@docs:/srv/docsbuild/3.9/cpython-en/Doc$ grep breakpoint > build/html/library/audit_events.html > docsbuild@docs:/srv/docsbuild/3.9/cpython-en/Doc$ > /srv/docsbuild/venv/bin/sphinx-build -b html -d build/doctrees -D > latex_elements.papersize= -D latex_engine=xelatex -D > latex_elements.inputenc= -D latex_elements.fontenc= -q -Ea -A daily=1 -A > switchers=1 . build/html > docsbuild@docs:/srv/docsbuild/3.9/cpython-en/Doc$ grep breakpoint > build/html/library/audit_events.html > <tr class="row-odd"><td><p>builtins.breakpoint</p></td> > <td><p><code class="docutils literal notranslate"><span > class="pre">breakpointhook</span></code></p></td> > <td><p><a class="reference internal" > href="functions.html#breakpoint">[1]</a></p></td> > > Bests, > -- > Julien Palard > https://mdk.fr > _______________________________________________ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/Q23O23HXTD7MSQGQW4Z5RO4XK5XYW2LZ/ > -- Regards, Karthikeyan S
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/277CKIY555PCSIKH6WNANODLLQMKGBIH/
