Re: [openstack-dev] [docs] [pbr] [ptl] [oslo] Removing pbr's custom 'build_sphinx' setuptools command

2017-07-10 Thread Doug Hellmann 
Excerpts from Anne Gentle's message of 2017-07-10 13:14:52 -0400:
> On Mon, Jul 10, 2017 at 1:11 PM, Doug Hellmann  wrote:
> > Excerpts from sfinucan's message of 2017-07-10 17:20:37 +0100:
> >> On Fri, 2017-07-07 at 15:58 +0100, sfinu...@redhat.com wrote:
> >>
> >> Of these, 'project' is static and should never change, so we don't need to
> >> attempt to guess them. Version and release are dynamic but I've noticed we
> >> don't actually use them anywhere in openstackdocstheme (the version you 
> >> see in
> >> the URL is actually an artefact of the build process and nothing to do with
> >> Sphinx). The closest thing we _do_ get to including version information is 
> >> the
> >> commit ID include in header of api-ref build pages [1], which is generated 
> >> by
> >> openstackdocstheme.
> >
> > It's surprising to me that we don't include the version string anywhere
> > on the page. Is that an oversight in the theme, or was it on purpose?
> 
> The theme shows "current" - however I noticed while reviewing this
> morning the nomenclature is "latest" in the URL, so I will make a
> change. Do you suggest "latest -n.n.n" as the string value, or simply
> "latest"?

I thought we would want to insert the version, as reported in the
Sphinx config, without any reference to current or latest or the
release series.

Perhaps we can replace the static text "Project home page" in the left
sidebar with "{{project}} {{version}}" (with or without "home page")
like in https://review.openstack.org/482230 ?

Doug

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [docs] [pbr] [ptl] [oslo] Removing pbr's custom 'build_sphinx' setuptools command

2017-07-10 Thread Anne Gentle 
On Mon, Jul 10, 2017 at 1:11 PM, Doug Hellmann  wrote:
> Excerpts from sfinucan's message of 2017-07-10 17:20:37 +0100:
>> On Fri, 2017-07-07 at 15:58 +0100, sfinu...@redhat.com wrote:
>> > tl;dr: pbr's 'build_spinx' derivative is broken again, and we want to just
>> > remove the feature at this point. However, this is going to necessitate 
>> > some
>> > mechanical changes for most projects with docs and this mail serves as a
>> > heads up and request for input before we proceed.
>>
>> [snip]
>>
>> > Here are the features that the plugin provides, along with the current
>> > migration plans:
>>
>> ...
>>
>> > - Automatically sets project name and version using pbr's machinery
>> >
>> >   Try to set this from 'openstackdocstheme'. If this isn't possible, folks
>> >   will need to add some some lines to 'conf.py' like keystoneauth does [7]
>>
>> I've been looking into this particular issue further, and the more I think
>> about it, the less necessary it seems. There are three configuration options
>> currently set by pbr:
>>
>> - project (the project name)
>> - version (the full version, including alpha/beta/rc tags)
>> - release (the short X.Y version)
>>
>> Of these, 'project' is static and should never change, so we don't need to
>> attempt to guess them. Version and release are dynamic but I've noticed we
>> don't actually use them anywhere in openstackdocstheme (the version you see 
>> in
>> the URL is actually an artefact of the build process and nothing to do with
>> Sphinx). The closest thing we _do_ get to including version information is 
>> the
>> commit ID include in header of api-ref build pages [1], which is generated by
>> openstackdocstheme.
>
> It's surprising to me that we don't include the version string anywhere
> on the page. Is that an oversight in the theme, or was it on purpose?

The theme shows "current" - however I noticed while reviewing this
morning the nomenclature is "latest" in the URL, so I will make a
change. Do you suggest "latest -n.n.n" as the string value, or simply
"latest"?

Anne

>
>> Given this fact, I think pbr is a providing a solution in search of problem
>> here, and this feature can in fact go quietly into the night with no further
>> ado.
>>
>> Thoughts?
>> Stephen
>>
>> PS: If we really did want to include a version in the docs in the future, I
>> think we could use information provided by ZUUL. I'm no ZUUL expert, but it
>> seems ZUUL does have commit id info ('ZUUL_REF') and what I hope to be
>> something like what 'git-describe' ('ZUUL_REFNAME'). We could simply pass 
>> these
>> via the '--version' parameter to 'build_sphinx' [3]. Again though, I don't
>> think this is necessary.
>
> Relying on anything outside of the repo won't work for packagers who
> build from source outside of our CI system.
>
>> PPS: I have not responded to the other replies to this mail yet because Red
>> Hat's email servers have decided not to send me replies to my own posts on
>> openstack-dev. I have seen them though and will reply when I figure out how 
>> to
>> get mboxes.
>>
>> [1] https://developer.openstack.org/api-ref/compute/
>> [2] 
>> https://github.com/openstack-infra/zuul/blob/8dda7c1/tools/trigger-job.py#L
>> 54-L60
>> [3] 
>> https://github.com/openstack-infra/project-config/blob/32aff86f/jenkins/scr
>> ipts/run-docs.sh#L20
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 

Read my blog: justwrite.click
Subscribe to Docs|Code: docslikecode.com

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [docs] [pbr] [ptl] [oslo] Removing pbr's custom 'build_sphinx' setuptools command

2017-07-10 Thread Doug Hellmann 
Excerpts from sfinucan's message of 2017-07-10 17:20:37 +0100:
> On Fri, 2017-07-07 at 15:58 +0100, sfinu...@redhat.com wrote:
> > tl;dr: pbr's 'build_spinx' derivative is broken again, and we want to just
> > remove the feature at this point. However, this is going to necessitate some
> > mechanical changes for most projects with docs and this mail serves as a
> > heads up and request for input before we proceed.
> 
> [snip]
> 
> > Here are the features that the plugin provides, along with the current
> > migration plans:
> 
> ...
> 
> > - Automatically sets project name and version using pbr's machinery
> > 
> >   Try to set this from 'openstackdocstheme'. If this isn't possible, folks 
> >   will need to add some some lines to 'conf.py' like keystoneauth does [7]
> 
> I've been looking into this particular issue further, and the more I think
> about it, the less necessary it seems. There are three configuration options
> currently set by pbr:
> 
> - project (the project name)
> - version (the full version, including alpha/beta/rc tags)
> - release (the short X.Y version)
> 
> Of these, 'project' is static and should never change, so we don't need to
> attempt to guess them. Version and release are dynamic but I've noticed we
> don't actually use them anywhere in openstackdocstheme (the version you see in
> the URL is actually an artefact of the build process and nothing to do with
> Sphinx). The closest thing we _do_ get to including version information is the
> commit ID include in header of api-ref build pages [1], which is generated by
> openstackdocstheme.

It's surprising to me that we don't include the version string anywhere
on the page. Is that an oversight in the theme, or was it on purpose?

> Given this fact, I think pbr is a providing a solution in search of problem
> here, and this feature can in fact go quietly into the night with no further
> ado.
> 
> Thoughts?
> Stephen
> 
> PS: If we really did want to include a version in the docs in the future, I
> think we could use information provided by ZUUL. I'm no ZUUL expert, but it
> seems ZUUL does have commit id info ('ZUUL_REF') and what I hope to be
> something like what 'git-describe' ('ZUUL_REFNAME'). We could simply pass 
> these
> via the '--version' parameter to 'build_sphinx' [3]. Again though, I don't
> think this is necessary.

Relying on anything outside of the repo won't work for packagers who
build from source outside of our CI system.

> PPS: I have not responded to the other replies to this mail yet because Red
> Hat's email servers have decided not to send me replies to my own posts on
> openstack-dev. I have seen them though and will reply when I figure out how to
> get mboxes.
> 
> [1] https://developer.openstack.org/api-ref/compute/
> [2] 
> https://github.com/openstack-infra/zuul/blob/8dda7c1/tools/trigger-job.py#L
> 54-L60
> [3] 
> https://github.com/openstack-infra/project-config/blob/32aff86f/jenkins/scr
> ipts/run-docs.sh#L20
> 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [docs] [pbr] [ptl] [oslo] Removing pbr's custom 'build_sphinx' setuptools command

2017-07-10 Thread Anne Gentle 
On Mon, Jul 10, 2017 at 12:20 PM,   wrote:
> On Fri, 2017-07-07 at 15:58 +0100, sfinu...@redhat.com wrote:
>> tl;dr: pbr's 'build_spinx' derivative is broken again, and we want to just
>> remove the feature at this point. However, this is going to necessitate some
>> mechanical changes for most projects with docs and this mail serves as a
>> heads up and request for input before we proceed.
>
> [snip]
>
>> Here are the features that the plugin provides, along with the current
>> migration plans:
>
> ...
>
>> - Automatically sets project name and version using pbr's machinery
>>
>>   Try to set this from 'openstackdocstheme'. If this isn't possible, folks
>>   will need to add some some lines to 'conf.py' like keystoneauth does [7]
>
> I've been looking into this particular issue further, and the more I think
> about it, the less necessary it seems. There are three configuration options
> currently set by pbr:
>
> - project (the project name)
> - version (the full version, including alpha/beta/rc tags)
> - release (the short X.Y version)
>
> Of these, 'project' is static and should never change, so we don't need to
> attempt to guess them. Version and release are dynamic but I've noticed we
> don't actually use them anywhere in openstackdocstheme (the version you see in
> the URL is actually an artefact of the build process and nothing to do with
> Sphinx). The closest thing we _do_ get to including version information is the
> commit ID include in header of api-ref build pages [1], which is generated by
> openstackdocstheme.

The 1.12.0 release of openstackdocstheme provides an option to get to
more than the current version of a project's docs. It acquires the
values by querying the most recent five git tags:

https://review.openstack.org/#/c/477639/

Anne

>
> Given this fact, I think pbr is a providing a solution in search of problem
> here, and this feature can in fact go quietly into the night with no further
> ado.
>
> Thoughts?
> Stephen
>
> PS: If we really did want to include a version in the docs in the future, I
> think we could use information provided by ZUUL. I'm no ZUUL expert, but it
> seems ZUUL does have commit id info ('ZUUL_REF') and what I hope to be
> something like what 'git-describe' ('ZUUL_REFNAME'). We could simply pass 
> these
> via the '--version' parameter to 'build_sphinx' [3]. Again though, I don't
> think this is necessary.
>
> PPS: I have not responded to the other replies to this mail yet because Red
> Hat's email servers have decided not to send me replies to my own posts on
> openstack-dev. I have seen them though and will reply when I figure out how to
> get mboxes.
>
> [1] https://developer.openstack.org/api-ref/compute/
> [2] 
> https://github.com/openstack-infra/zuul/blob/8dda7c1/tools/trigger-job.py#L
> 54-L60
> [3] 
> https://github.com/openstack-infra/project-config/blob/32aff86f/jenkins/scr
> ipts/run-docs.sh#L20
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 

Read my blog: justwrite.click
Subscribe to Docs|Code: docslikecode.com

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [docs] [pbr] [ptl] [oslo] Removing pbr's custom 'build_sphinx' setuptools command

2017-07-10 Thread sfinucan 
On Fri, 2017-07-07 at 15:58 +0100, sfinu...@redhat.com wrote:
> tl;dr: pbr's 'build_spinx' derivative is broken again, and we want to just
> remove the feature at this point. However, this is going to necessitate some
> mechanical changes for most projects with docs and this mail serves as a
> heads up and request for input before we proceed.

[snip]

> Here are the features that the plugin provides, along with the current
> migration plans:

...

> - Automatically sets project name and version using pbr's machinery
> 
>   Try to set this from 'openstackdocstheme'. If this isn't possible, folks 
>   will need to add some some lines to 'conf.py' like keystoneauth does [7]

I've been looking into this particular issue further, and the more I think
about it, the less necessary it seems. There are three configuration options
currently set by pbr:

- project (the project name)
- version (the full version, including alpha/beta/rc tags)
- release (the short X.Y version)

Of these, 'project' is static and should never change, so we don't need to
attempt to guess them. Version and release are dynamic but I've noticed we
don't actually use them anywhere in openstackdocstheme (the version you see in
the URL is actually an artefact of the build process and nothing to do with
Sphinx). The closest thing we _do_ get to including version information is the
commit ID include in header of api-ref build pages [1], which is generated by
openstackdocstheme.

Given this fact, I think pbr is a providing a solution in search of problem
here, and this feature can in fact go quietly into the night with no further
ado.

Thoughts?
Stephen

PS: If we really did want to include a version in the docs in the future, I
think we could use information provided by ZUUL. I'm no ZUUL expert, but it
seems ZUUL does have commit id info ('ZUUL_REF') and what I hope to be
something like what 'git-describe' ('ZUUL_REFNAME'). We could simply pass these
via the '--version' parameter to 'build_sphinx' [3]. Again though, I don't
think this is necessary.

PPS: I have not responded to the other replies to this mail yet because Red
Hat's email servers have decided not to send me replies to my own posts on
openstack-dev. I have seen them though and will reply when I figure out how to
get mboxes.

[1] https://developer.openstack.org/api-ref/compute/
[2] https://github.com/openstack-infra/zuul/blob/8dda7c1/tools/trigger-job.py#L
54-L60
[3] https://github.com/openstack-infra/project-config/blob/32aff86f/jenkins/scr
ipts/run-docs.sh#L20

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [docs] [pbr] [ptl] [oslo] Removing pbr's custom 'build_sphinx' setuptools command

2017-07-08 Thread Jeremy Stanley 
On 2017-07-07 18:03:11 -0500 (-0500), Monty Taylor wrote:
> On 07/07/2017 12:12 PM, Jeremy Stanley wrote:
[...]
> > Better still, something like `tox -e venv -- sphinx-build` would
> > be more in keeping with the spirit of the PTI to ensure projects
> > don't encapsulate any secret sauce into a separate "docs" tox
> > env and so can have their documentation built correctly via a
> > more standard (for the Python ecosystem) command line.
[...]
> I'm starting to be of the opinion that we may be stretching things
> a little far with the tox -evenv -- in this case and making it
> harder not easier.
[...]

Sure, I agree that the tox invocation is more of an implementation
detail. My concern is over making sure that projects don't each end
up with their own bespoke docs building process behind some opaque
entrypoint on which we've standardized. If we simply say `tox -e
docs` is required then we'll end up with a number of projects
calling wrapper scripts from that which do all manner of badness
before (hopefully still at least) invoking sphinx-build. Sphinx
already provides turing-complete flexibility on its own, so when
fancy features are required we should do that by improving themes
and adding plugins rather than through random hacky wrapper scripts.

The detailed plan you outlined in your message (elided for the sake
of brevity) seems fine to me. Thanks!
-- 
Jeremy Stanley


signature.asc
Description: Digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [docs] [pbr] [ptl] [oslo] Removing pbr's custom 'build_sphinx' setuptools command

2017-07-07 Thread Monty Taylor 

On 07/07/2017 12:12 PM, Jeremy Stanley wrote:

On 2017-07-07 11:20:30 -0400 (-0400), Doug Hellmann wrote:
[...]

We also discussed changing the CI interface to build docs to use
the "tox -e docs" command like contributors generally run locally.

[...]

Better still, something like `tox -e venv -- sphinx-build` would be
more in keeping with the spirit of the PTI to ensure projects don't
encapsulate any secret sauce into a separate "docs" tox env and so
can have their documentation built correctly via a more standard
(for the Python ecosystem) command line.


I've been thinking about this a bunch recently as I've been working on 
zuul v3 versions of our base jobs with an eye on them being sharable 
between installations.


I'm starting to be of the opinion that we may be stretching things a 
little far with the tox -evenv -- in this case and making it harder not 
easier.


I'd like to suggest a few different things:

a) We define what we think the correct sphinx-build invocation is, 
regardless of tox/make/venv/etc. and define that in the PTI. Sphinx is 
perfectly usable for docs for other languages too, so having a standard 
"we expect sphinx-build to work right" (if that's the right invocation) 
seems like a better long-term win.


b) Update the PTI to indicate that a tox.ini needs to have a venv env 
that supports arbitrary commands, regardless of which commands.


c) Split docs/sphinx requirements from test-requirements.txt to 
docs-requirements.txt. This is potentially useful in in two ways:


* Because we only have 'test-requirements.txt', it gets installs for all 
the things. But you know what- docutils is actually not needed to run 
unittests. So in many cases it should trim venv size.


* In non-python projects, putting a docs-requirements.txt there even if 
there is not a similar requirements.txt or test-requirements.txt seems 
reasonable. At worst it's just a text file that indicates you need to 
install some python things to make docs building work and a human can 
figure that out.


Then the CI job could be:

if [ -f tox.ini ] ; then
  tox -evenv -- sphinx-build
else if [ -f docs-requirements.txt ] ; then
  pip install -edocs-requirements.txt
  sphinx-build
else
  sphinx-build
fi

And it should cover a lot of the cases, plus building docs locally 
should continue to work sanely.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [docs] [pbr] [ptl] [oslo] Removing pbr's custom 'build_sphinx' setuptools command

2017-07-07 Thread Jeremy Stanley 
On 2017-07-07 11:20:30 -0400 (-0400), Doug Hellmann wrote:
[...]
> We also discussed changing the CI interface to build docs to use
> the "tox -e docs" command like contributors generally run locally.
[...]

Better still, something like `tox -e venv -- sphinx-build` would be
more in keeping with the spirit of the PTI to ensure projects don't
encapsulate any secret sauce into a separate "docs" tox env and so
can have their documentation built correctly via a more standard
(for the Python ecosystem) command line.
-- 
Jeremy Stanley


signature.asc
Description: Digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [docs] [pbr] [ptl] [oslo] Removing pbr's custom 'build_sphinx' setuptools command

2017-07-07 Thread Doug Hellmann 
Excerpts from sfinucan's message of 2017-07-07 15:58:10 +0100:
> tl;dr: pbr's 'build_spinx' derivative is broken again, and we want to just
> remove the feature at this point. However, this is going to necessitate some
> mechanical changes for most projects with docs and this mail serves as a heads
> up and request for input before we proceed.
> 
> --
> 
> Since pretty much the beginning, pbr has provided a custom 'build_sphinx'
> setuptools command [1], which can be run like so:
> 
> python setup.py build_sphinx
> 
> This is described in the pbr docs [2] and is is based on Sphinx's own
> 'build_sphinx' command [3], which is described in the Sphinx docs [4].
> 
> Historically, this custom version has provided a couple of features that the
> upstream Sphinx extension didn't. These are outlined below. However, this
> derivative has resulted in an inordinately large number of bugs for what 
> should
> be a simple feature (#1702872, #1691129, #1681983, #1674795, #1379998,
> #1496882, ...). This is due to the lack of extension points in the upstream
> code and the way that we've been forced to extend the command (we break almost
> every time Sphinx change their code).
> 
> The latest of these bugs, #1702872, breaks a couple of things when using 
> Sphinx
> > 1.6:
> 
> - We no longer provide confoverrides when using Sphinx 1.6+, which means you
>   may see "node class 'xxx' is already registered' errors, and the project
>   name and version used in the docs may be incorrect [5]
> - The '[build_sphinx] builders' in 'setup.cfg' option is no longer read, so
>   only 'html' docs will be built (no man pages)
> - The apidoc tool won't be automatically run
> 
> Fixing these issues is possible in the short-term and won't make the thing any
> more maintainable going forward. Given the amount of issues this feature has
> caused and the fact that Sphinx has since gained many of the features we
> provided itself, we'd like to just drop the whole thing now. Some of the extra
> this feature provide are no longer necessary, but the migration paths for the
> ones that are are not great and will require mechanical changes to projects.
> 
> Here are the features that the plugin provides, along with the current
> migration plans:
> 
> - Automatically runs sphinx-apidoc [6] before building docs
> 
>   It seems that adding the 'sphinx.ext.apidoc' extension to the 'extensions'
>   list in 'doc/source/conf.py' will ensure this continues to happen
>   automatically. However, I need to check this and will get back when I have.
> 
> - Automatically runs 'autodoc', which seems to do something similar to 
> 'apidoc'
>   but in a different way (I don't know what, tbh)
> 
>   Drop this feature entirely as 'apidoc' appears to do something very similar.
> 
> - Automatically sets project name and version using pbr's machinery
> 
>   Try to set this from 'openstackdocstheme'. If this isn't possible, folks 
> will
>   need to add some some lines to 'conf.py' like keystoneauth does [7]

I'm inclined to just say these values should be set explicitly. We can
simplify it (Monty had an idea for a 1-liner that would set all of the
variables with one function call into pbr), but the project name doesn't
change so there's no real sense in doing a lot of fancy automation for
that one.

> - Supports multiple builders via the '[build_sphinx] builders' option in
>   'setup.cfg'
> 
>   This is natively supported since Sphinx 1.6. You should replace
>   '[build_sphinx] builders' with '[build_sphinx] builder' (singular) now

That would only work if we continue to invoke Sphinx using its
setuptools plugin in the gate. Is that what we want to do? We also
discussed changing the CI interface to build docs to use the "tox -e
docs" command like contributors generally run locally.

> - Historically, supported turning build warning into build falures via the
>   '[build_sphinx] warnerrors' option in 'setup.cfg'
> 
>   This is no longer supported by pbr, as Sphinx now provides a '[build_sphinx]
>   warning-is-error' option. I wrote a load of patches to fix this recently
>   using the topic 'sphinx15', but if I missed your project you need to remove
>   '[pbr] warnerrors' from 'setup.cfg' and add '[build_sphinx]
>   warning-is-error'. You should do this now.

This is also one of the doc-migration steps. Some projects that are
hitting errors caused by the missing config overrides are going to need
to wait to turn the flag on until after we figure out how to work around
those issues.

> Does anyone see any issues with this strategy going forward? If not, we're
> going to start making changes to projects using the 'sphinx16' topic and would
> appreciate reviews on these as they come in. We're not going to get to every
> project, so if you can make the necessary changes to your own project then
> please do.

Let me know if you want some help setting up a burndown chart like
https://doughellmann.com/doc-migration/. Having a tracking etherpad so
folks know whether a repo

[openstack-dev] [docs] [pbr] [ptl] [oslo] Removing pbr's custom 'build_sphinx' setuptools command

2017-07-07 Thread sfinucan 
tl;dr: pbr's 'build_spinx' derivative is broken again, and we want to just
remove the feature at this point. However, this is going to necessitate some
mechanical changes for most projects with docs and this mail serves as a heads
up and request for input before we proceed.

--

Since pretty much the beginning, pbr has provided a custom 'build_sphinx'
setuptools command [1], which can be run like so:

python setup.py build_sphinx

This is described in the pbr docs [2] and is is based on Sphinx's own
'build_sphinx' command [3], which is described in the Sphinx docs [4].

Historically, this custom version has provided a couple of features that the
upstream Sphinx extension didn't. These are outlined below. However, this
derivative has resulted in an inordinately large number of bugs for what should
be a simple feature (#1702872, #1691129, #1681983, #1674795, #1379998,
#1496882, ...). This is due to the lack of extension points in the upstream
code and the way that we've been forced to extend the command (we break almost
every time Sphinx change their code).

The latest of these bugs, #1702872, breaks a couple of things when using Sphinx
> 1.6:

- We no longer provide confoverrides when using Sphinx 1.6+, which means you
  may see "node class 'xxx' is already registered' errors, and the project
  name and version used in the docs may be incorrect [5]
- The '[build_sphinx] builders' in 'setup.cfg' option is no longer read, so
  only 'html' docs will be built (no man pages)
- The apidoc tool won't be automatically run

Fixing these issues is possible in the short-term and won't make the thing any
more maintainable going forward. Given the amount of issues this feature has
caused and the fact that Sphinx has since gained many of the features we
provided itself, we'd like to just drop the whole thing now. Some of the extra
this feature provide are no longer necessary, but the migration paths for the
ones that are are not great and will require mechanical changes to projects.

Here are the features that the plugin provides, along with the current
migration plans:

- Automatically runs sphinx-apidoc [6] before building docs

  It seems that adding the 'sphinx.ext.apidoc' extension to the 'extensions'
  list in 'doc/source/conf.py' will ensure this continues to happen
  automatically. However, I need to check this and will get back when I have.

- Automatically runs 'autodoc', which seems to do something similar to 'apidoc'
  but in a different way (I don't know what, tbh)

  Drop this feature entirely as 'apidoc' appears to do something very similar.

- Automatically sets project name and version using pbr's machinery

  Try to set this from 'openstackdocstheme'. If this isn't possible, folks will
  need to add some some lines to 'conf.py' like keystoneauth does [7]

- Supports multiple builders via the '[build_sphinx] builders' option in
  'setup.cfg'

  This is natively supported since Sphinx 1.6. You should replace
  '[build_sphinx] builders' with '[build_sphinx] builder' (singular) now

- Historically, supported turning build warning into build falures via the
  '[build_sphinx] warnerrors' option in 'setup.cfg'

  This is no longer supported by pbr, as Sphinx now provides a '[build_sphinx]
  warning-is-error' option. I wrote a load of patches to fix this recently
  using the topic 'sphinx15', but if I missed your project you need to remove
  '[pbr] warnerrors' from 'setup.cfg' and add '[build_sphinx]
  warning-is-error'. You should do this now.

Does anyone see any issues with this strategy going forward? If not, we're
going to start making changes to projects using the 'sphinx16' topic and would
appreciate reviews on these as they come in. We're not going to get to every
project, so if you can make the necessary changes to your own project then
please do.

Cheers,
Stephen
irc:stephenfin (formerly sfinucan)

- 

[1] https://github.com/openstack-dev/pbr/blob/master/pbr/builddoc.py
[2] https://docs.openstack.org/pbr/latest/user/using.html#build-sphinx
[3] https://github.com/sphinx-doc/sphinx/blob/master/sphinx/setup_command.py
[4] http://www.sphinx-doc.org/en/stable/setuptools.html
[5] http://paste.openstack.org/show/614730/
[5] www.sphinx-doc.org/en/stable/man/sphinx-apidoc.html
[7] https://github.com/openstack/keystoneauth/blob/master/doc/source/conf.py#L6
5-L72

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev