[issue36675] Doctest directives and comments missing from code samples

[Ming Hua]

Change by Ming Hua :


--
nosy: +minghua

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36675] Doctest directives and comments missing from code samples

[Jim DeLaHunt]

Jim DeLaHunt  added the comment:

My goodness, things get complex sometimes. 

If we cannot make Sphinx preserve doctest directives and comments, perhaps we 
should go back to the historical bug discussion to look at workarounds which we 
considered earlier. For instance, maybe we should modify the text surrounding 
the examples to explain that doctests directives should appear there, but that 
our tool chain currently removes them.

At the moment, I see doctest directives in the doctest source code at: 
https://github.com/python/cpython/blob/master/Doc/library/doctest.rst#directives
 
which do not appear in the corresponding HTML output at:
https://docs.python.org/3/library/doctest.html#directives 

How about rewording the text before each of those examples? Or maybe finding 
some way to show those examples as literal text which Sphinx won't modify, even 
if it is not formatted like Python code examples?

By the way, the discussion of this same problem back in 2011-2012 is at #12947 
. At that time, we applied a "monkey patch" to the Sphinx code. I haven't read 
the discussion closely enough to figure out if such a patch would help now.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36675] Doctest directives and comments missing from code samples

[Julien Palard]

Julien Palard  added the comment:

Due to https://github.com/python/cpython/pull/24282 this is sadly un-fixed.

Either we find another way to fix this, either we wait 3 releases and we 
re-commit https://github.com/python/cpython/pull/23620.

--
resolution: fixed -> later
stage: resolved -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36675] Doctest directives and comments missing from code samples

[Anthony Sottile]

Anthony Sottile  added the comment:

should the minimum sphinx version be bumped and/or this reverted: 
https://github.com/python/cpython/blob/f7f0ed59bcc41ed20674d4b2aa443d3b79e725f4/Doc/conf.py#L48

this change breaks, for example, sphinx 1.8.5 while building for ubuntu 20.04

The directive used here requires sphinx>=3.2.0

I notice some other attempts have been made to make the docs more compatible 
with sphinx 1.x in this release as well so there might be conflicting 
directions here: 
https://github.com/python/cpython/commit/b63a620014b67a6e63d10783149c41baaf59def8

--
nosy: +Anthony Sottile

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36675] Doctest directives and comments missing from code samples

[Julien Palard]

Julien Palard  added the comment:

Happy Christmas, everybody involved in this issue! I'm happy to announce this 
issue is resolved since a few days \o/

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36675] Doctest directives and comments missing from code samples

[Julien Palard]

Julien Palard  added the comment:


New changeset c8a10d2fabff492ab352501c316baf5f2fc6510e by Julien Palard in 
branch 'master':
bpo-36675: Doc: Reveal doctest directives (GH-23620)
https://github.com/python/cpython/commit/c8a10d2fabff492ab352501c316baf5f2fc6510e


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36675] Doctest directives and comments missing from code samples

[Julien Palard]

Change by Julien Palard :


--
pull_requests: +22488
pull_request: https://github.com/python/cpython/pull/23620

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36675] Doctest directives and comments missing from code samples

[Jim DeLaHunt]

Jim DeLaHunt  added the comment:

We discovered and fixed this same problem back in 2011-2012 with #12947 . 

That was apparently the source of the monkeypatch that was removed as "obselete 
code" on 2019-09-12. That old issue commentary has some suggestions about other 
workarounds. This includes adding explanatory text about the fact that doctest 
directives are missing from the examples which should be showing them. Maybe 
some of those workarounds would be effective for us now.

--
nosy: +JDLH

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36675] Doctest directives and comments missing from code samples

[miss-islington]

miss-islington  added the comment:


New changeset 94a684734f669eab02b5c915394b749ccf936449 by Miss Islington (bot) 
in branch '3.8':
bpo-36675: Remove obsolete code. (GH-16024)
https://github.com/python/cpython/commit/94a684734f669eab02b5c915394b749ccf936449


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36675] Doctest directives and comments missing from code samples

[Gregory P. Smith]

Gregory P. Smith  added the comment:


New changeset 2c910c1e732c9a3ec4c67a7c43d789d6c729304a by Gregory P. Smith 
(Julien Palard) in branch 'master':
bpo-36675: Remove obsolete code. (GH-16024)
https://github.com/python/cpython/commit/2c910c1e732c9a3ec4c67a7c43d789d6c729304a


--
nosy: +gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36675] Doctest directives and comments missing from code samples

[miss-islington]

Change by miss-islington :


--
pull_requests: +15653
pull_request: https://github.com/python/cpython/pull/16030

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36675] Doctest directives and comments missing from code samples

[Julien Palard]

Change by Julien Palard :


--
pull_requests: +15647
pull_request: https://github.com/python/cpython/pull/16024

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36675] Doctest directives and comments missing from code samples

[Julien Palard]

Julien Palard  added the comment:

I opened an issue on the sphinx-doc repo [1] to check if it would be possible 
to have an option in doctest blocks to not trim them.

We previously had a hack in Doc/tools/extensions/pyspecific.py to patch sphinx 
to not trim them for the doctest.rst file. But sphinx deprecated the hack we 
used :(

[1]: https://github.com/sphinx-doc/sphinx/issues/6698

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36675] Doctest directives and comments missing from code samples

[Cheryl Sabella]

Cheryl Sabella  added the comment:

;tldr;   There is a global configuration flag to show all the doctest 
directives for all the docs that are built.  The default is to suppress the 
doctest directives.  If a global setting isn't desired, then each doctest 
example will have to be changed individually.


I've looked at the Sphinx documentation for this and there are a few points to 
reference. 

* `::` defines a Literal block in reST. [1]  A literal block doesn't do 
anything special except from the last paragraph is this section of the docs: 
   "Code highlighting can be enabled for these literal blocks on a 
document-wide basis using the `highlight` directive and on a project-wide basis 
using the `highlight_language` configuration option. The code-block directive 
can be used to set highlighting on a block-by-block basis. These directives are 
discussed later."  CPython has the `highlight-language` set to python3.  

* A doctest block does not need to be in a Literal Block. [2]  They are 
identified automatically. [3]  According to the doc, doctest directives are 
suppressed from presentation:

Also, you can give inline doctest options, like in doctest:

>>> datetime.date.now()   # doctest: +SKIP
datetime.date(2008, 1, 1)

They will be respected when the test is run, but stripped from presentation 
output.

* Any section (doesn't need the single colon (`:`) can have the `code-block` 
directive on it. [4]  The `code-block` directive can define the language for 
highlighting.  

Note that in the Sphinx doctest [4] documentation, there is a config option 
`trim_doctest_flags`, which is True by default.  Setting this to False will 
show all the doctest directives for all the doctests in the documentation.

The nice thing about the doc build recognizing a section as a doctest is that 
it adds the toggle in the upper right of the block to 'Hide the toggle and 
output' so that the example can be copy and pasted more easily.  Changing it 
into something other than a doctest (like using a `code-block` directive, takes 
away this option.  Using `trim_doctest_flags=False` retains the hide button.

Options:
1.  Change global setting to false to show all doctest directives.
2.  Change only the blocks where the doctest directives need to be shown, but 
this makes them lose the 'Hide prompt and output' button.
3.  Add a new option under 'code-block' to control the displaying of the 
doctest directives at the code-block level. This would override the global 
setting.  I think this option could be added locally, but may need to be done 
upstream in Sphinx.



[1] 
http://www.sphinx-doc.org/en/stable/usage/restructuredtext/basics.html#literal-blocks
[2] 
http://www.sphinx-doc.org/en/stable/usage/restructuredtext/basics.html#doctest-blocks
[3] http://www.sphinx-doc.org/en/stable/usage/extensions/doctest.html
[4] 
http://www.sphinx-doc.org/en/stable/usage/restructuredtext/directives.html#showing-code-examples

--
nosy: +cheryl.sabella

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36675] Doctest directives and comments missing from code samples

[Steven D'Aprano]

Steven D'Aprano  added the comment:

> Doctest directives in code examples should be suppressed everywhere 
> *except* in the doctest.html examples showing how to use directives.  
> The patch only exposes them for doctest.html and not for ctypes or 
> anywhere else.

Thanks for the patch, and the extra information, but I disagree with the 
decision to suppress the directives.

The reason I found this problem in the first case was that I started 
with the ctypes documentation, where it says:

"Since some code samples behave differently under Linux, Windows, or Mac 
OS X, they contain doctest directives in comments."

and I was very keen to see those directives so I could learn the right 
way to deal with platform-dependent doctests. I was very confused that 
they weren't visible.

https://docs.python.org/3/library/ctypes.html

I think that doctest directives are as much a part of documenting 
correct usage as any other part of the example code, and they are 
(semi-)human readable and (almost) self-documenting.

Consider this example from ctypes:

>>> c_wchar_p("Hello, World")
c_wchar_p(140018365411392)

In the absence of a directive, but knowing that it may have been 
surpressed, I don't know how to interpret that. Is the output some 
arbitrarily chosen value that doctest ought to skip? Or is that the 
actual output that c_wchar_p("Hello, World") will return every single 
time without fail?

If I was a ctypes expert, it might be blindingly obvious to me, but I'm 
not, so I'm left in the dark. I don't know whether I should expect that 
precise output each and every time, or something platform and 
implementation specific.

If the directive #doctest:+SKIP was visible, I would know that it was an 
arbitrarily chosen example. 

My preference would be:

- keep the doctest directives visible, everywhere;

- make them a clickable link to the appropriate section 
  in the doctest documentation;

- and, if possible, on mouse-over, they should display a
  tooltip with a message like 

  "The output of this example is arbitrary."

Or similar wording.

> They really should not be in the dir example code that I linked to.
> https://docs.python.org/3/library/functions.html#dir

On the contrary: I think that the presence of the +SKIP directive helps 
demonstrate that the output shown is a made-up example, not normative.

(Of course it helps that I know doctest, but even if I didn't, the 
tooltip message would help.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36675] Doctest directives and comments missing from code samples

[Jeroen Demeyer]

Jeroen Demeyer  added the comment:

> Doctest directives in code examples should be suppressed everywhere *except* 
> in the doctest.html examples showing how to use directives.

Thanks for clarifying. I missed that.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36675] Doctest directives and comments missing from code samples

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Doctest directives in code examples should be suppressed everywhere *except* in 
the doctest.html examples showing how to use directives.  The patch only 
exposes them for doctest.html and not for ctypes or anywhere else.

They really should not be in the dir example code that I linked to.
https://docs.python.org/3/library/functions.html#dir
The problem there are the double comments with both a real comment and a 
directive.

https://devguide.python.org/documenting/#source-code does not say anything 
about '::' causing suppression of comments and ':' leaving them.  It is 
misleading in implying the '::' is required for a code block.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36675] Doctest directives and comments missing from code samples

[Éric Araujo]

Éric Araujo  added the comment:

OP is about the documentation page for doctest itself!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36675] Doctest directives and comments missing from code samples

[Jeroen Demeyer]

Jeroen Demeyer  added the comment:

Isn't it a *feature* that those doctest directives are not shown? Those 
directives are meant for the doctest module only, not for the reader of the 
rendered documentation.

--
nosy: +jdemeyer

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36675] Doctest directives and comments missing from code samples

[Roundup Robot]

Change by Roundup Robot :


--
keywords: +patch
pull_requests: +13038
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36675] Doctest directives and comments missing from code samples

[Terry J. Reedy]

Terry J. Reedy  added the comment:

I verified that the line in Doc/library/doctest.rst has the comment.
"
For example, this test passes::

   >>> print(list(range(20))) # doctest: +NORMALIZE_WHITESPACE
   [0,   1,  2,  3,  4,  5,  6,  7,  8,  9,
   10,  11, 12, 13, 14, 15, 16, 17, 18, 19]
"
However, the comment is omitted from the .html built on Windows by Sphinx 1.8.1.
"
 print(list(range(20))) 
   ***html for comment should be here***
[0,   1,  2,  3,  4,  5,  6,  7,  8,  9,
10,  11, 12, 13, 14, 15, 16, 17, 18, 19]


"
To me, this is a bug with building the .html for doctest.rst.  Comments are 
preserved elsewhere.  For instance, the code example for 
https://docs.python.org/3/library/functions.html#dir.  The .rst file has
"
   The resulting list is sorted alphabetically.  For example:

  >>> import struct
  >>> dir()   # show the names in the module namespace  # doctest: +SKIP
"
Both use 3-space colon + indents to mean 'example block'.  The only difference 
is '::' versus ':'.  https://devguide.python.org/documenting/#source-code says 
:: is required.  idle.rst also has a code example with comments displayed (I 
just submitted a PR to not suppress color highlighting.)

I leave it to the doc experts to discover why the comments are not included in 
doctest.html.

--
nosy: +eric.araujo, ezio.melotti, terry.reedy, willingc
stage:  -> needs patch
title: Doctest directives and comments not visible or missing from code samples 
-> Doctest directives and comments missing from code samples
versions: +Python 2.7, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com