Re: [Python-Dev] Escaping docs markup in NEWS entries?

2017-05-02 Thread Wes Turner 
On Tue, May 2, 2017 at 1:58 PM, David Goodger  wrote:

> On Mon, May 1, 2017 at 5:03 AM, Wes Turner  wrote:
> > Where would be a good place for test cases for an rst_escape() function?
> > Docutils?
> >
> > https://github.com/westurner/dotfiles/blob/develop/scripts/
> git-changelog.py
>
> I'd say that the test cases for any function should reside in the same
> project/codebase as the function itself. Where do you intend the
> function to reside?
>
> I'm not certain what rst_escape() is supposed to do; what is its
> purpose? The docstrings of rst_escape() as well as the
> git-changelog.py module lack such explanations.
>

def rst_escape(_str: str) -> str:

Args:
_str: string to safely escape for literal inclusion in a
ReStructuredText document
Returns:
str: safely-escaped ReStructuredText



>
> I am also uncertain if this is necessary, based on the remainder of
> the thread. Perhaps it would be better to fix this "make suspicious"
> so that it doesn't produce spurious results? (Note: I am unclear on
> exactly what "make suspicious" is supposed to do, or why.)
>

An rst_escape(text: ) function is necessary in order to prevent
ReStructuredText injection (and e.g. * unintentional partial italicization).

There are other directives for including text in a ReStructuredText
document, e.g.:

- ``.. include::``
- ``.. raw::``

And there are extensions for changelogs and sphinx builds:

- ``.. git_changelog::`` -- https://pypi.org/project/sphinx-git/
- ``.. changelog::``, ``.. change::`` -- https://pypi.org/project/changelog/


But, for a README.rst in a git repository, for example, it's far easier to
diff the document with the data included than it is to have to build from a
source .rst to a dest .rst.rst and then diff the .rst.rst files.

There are CWE URLs for describing the type of vulnerability that a
community-reviewed rst_escape() function would be designed to prevent:

- CWE-116: Improper Encoding or Escaping of Output
  https://cwe.mitre.org/data/definitions/116.html
  - CWE-74: Improper Neutralization of Special Elements in Output Used by a
Downstream Component ('Injection')
https://cwe.mitre.org/data/definitions/74.html#Relationships

- CWE-89: SQL Injection
  https://cwe.mitre.org/data/definitions/89.htm

  - 2011 Top25 #1 issue
https://cwe.mitre.org/top25/#CWE-89

- [ ] CWE-???: RST Injection


Not a bug in docutils; a PEBKAM implementation "BUG,SEC" issue


>
> I apologize for the azure hue of my answers, to questions that came
> from out of the blue.
> ;-)
>
> David Goodger
> 
>
>
> > - rst_escape # YMMV
> > - $ git-changelog.py -r "release/0.3.14" --hdr= "+"`
> >
> > On Monday, May 1, 2017, Nick Coghlan  wrote:
> >>
> >> On 1 May 2017 at 17:13, Martin Panter  wrote:
> >> > On 1 May 2017 at 06:37, Nick Coghlan  wrote:
> >> >> Hi folks,
> >> >>
> >> >> I'm trying to write a NEWS entry that explains that the
> >> >> ":func:`bytes`" cross-references have changed to refer to the type
> >> >> descriptions by default (matching other builtin container types), so
> >> >> you now need to use ``:ref:`func-bytes`" to refer to the old target
> in
> >> >> the list of builtin functions (if you really want that for some
> >> >> reason).
> >> >>
> >> >> Unfortunately, my first two attempts both cause warnings in "make
> >> >> suspicious" with the following output:
> >> >
> >> > What is the full output? Usually it includes instructions to add false
> >> > positives to Doc/tools/susp-ignored.csv; maybe that is all you have to
> >> > do?
> >>
> >> You're right, that would be likely be the way to go if I decided to
> >> keep the escaped markup.
> >>
> >> However...
> >>
> >> >> My fallback plan is to just include the unescaped markup in the NEWS
> >> >> entry (rather than trying to make it readable even in rendered form),
> >> >> but I figured I'd ask for advice here first.
> >> >
> >> > I thought the NEWS file was mainly regarded as plain text, so it would
> >> > be important to avoid ugly RST markup like backslashes if possible.
> >>
> >> ... I think you're right on this point, so it makes more sense to skip
> >> the escaping entirely,
> >> and just use the correct link markup in the NEWS entry.
> >
> > How convenient.
> >>
> >>
> >> Cheers,
> >> Nick.
> >>
> >> --
> >> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> >> ___
> >> Python-Dev mailing list
> >> Python-Dev@python.org
> >> https://mail.python.org/mailman/listinfo/python-dev
> >> Unsubscribe:
> >> https://mail.python.org/mailman/options/python-dev/
> wes.turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Escaping docs markup in NEWS entries?

2017-05-02 Thread David Goodger 
On Mon, May 1, 2017 at 5:03 AM, Wes Turner  wrote:
> Where would be a good place for test cases for an rst_escape() function?
> Docutils?
>
> https://github.com/westurner/dotfiles/blob/develop/scripts/git-changelog.py

I'd say that the test cases for any function should reside in the same
project/codebase as the function itself. Where do you intend the
function to reside?

I'm not certain what rst_escape() is supposed to do; what is its
purpose? The docstrings of rst_escape() as well as the
git-changelog.py module lack such explanations.

I am also uncertain if this is necessary, based on the remainder of
the thread. Perhaps it would be better to fix this "make suspicious"
so that it doesn't produce spurious results? (Note: I am unclear on
exactly what "make suspicious" is supposed to do, or why.)

I apologize for the azure hue of my answers, to questions that came
from out of the blue.
;-)

David Goodger



> - rst_escape # YMMV
> - $ git-changelog.py -r "release/0.3.14" --hdr= "+"`
>
> On Monday, May 1, 2017, Nick Coghlan  wrote:
>>
>> On 1 May 2017 at 17:13, Martin Panter  wrote:
>> > On 1 May 2017 at 06:37, Nick Coghlan  wrote:
>> >> Hi folks,
>> >>
>> >> I'm trying to write a NEWS entry that explains that the
>> >> ":func:`bytes`" cross-references have changed to refer to the type
>> >> descriptions by default (matching other builtin container types), so
>> >> you now need to use ``:ref:`func-bytes`" to refer to the old target in
>> >> the list of builtin functions (if you really want that for some
>> >> reason).
>> >>
>> >> Unfortunately, my first two attempts both cause warnings in "make
>> >> suspicious" with the following output:
>> >
>> > What is the full output? Usually it includes instructions to add false
>> > positives to Doc/tools/susp-ignored.csv; maybe that is all you have to
>> > do?
>>
>> You're right, that would be likely be the way to go if I decided to
>> keep the escaped markup.
>>
>> However...
>>
>> >> My fallback plan is to just include the unescaped markup in the NEWS
>> >> entry (rather than trying to make it readable even in rendered form),
>> >> but I figured I'd ask for advice here first.
>> >
>> > I thought the NEWS file was mainly regarded as plain text, so it would
>> > be important to avoid ugly RST markup like backslashes if possible.
>>
>> ... I think you're right on this point, so it makes more sense to skip
>> the escaping entirely,
>> and just use the correct link markup in the NEWS entry.
>
> How convenient.
>>
>>
>> Cheers,
>> Nick.
>>
>> --
>> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/wes.turner%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Escaping docs markup in NEWS entries?

2017-05-01 Thread Wes Turner 
Where would be a good place for test cases for an rst_escape() function?
Docutils?

https://github.com/westurner/dotfiles/blob/develop/scripts/git-changelog.py

- rst_escape # YMMV
- $ git-changelog.py -r "release/0.3.14" --hdr= "+"`

On Monday, May 1, 2017, Nick Coghlan  wrote:

> On 1 May 2017 at 17:13, Martin Panter >
> wrote:
> > On 1 May 2017 at 06:37, Nick Coghlan >
> wrote:
> >> Hi folks,
> >>
> >> I'm trying to write a NEWS entry that explains that the
> >> ":func:`bytes`" cross-references have changed to refer to the type
> >> descriptions by default (matching other builtin container types), so
> >> you now need to use ``:ref:`func-bytes`" to refer to the old target in
> >> the list of builtin functions (if you really want that for some
> >> reason).
> >>
> >> Unfortunately, my first two attempts both cause warnings in "make
> >> suspicious" with the following output:
> >
> > What is the full output? Usually it includes instructions to add false
> > positives to Doc/tools/susp-ignored.csv; maybe that is all you have to
> > do?
>
> You're right, that would be likely be the way to go if I decided to
> keep the escaped markup.
>
> However...
>
> >> My fallback plan is to just include the unescaped markup in the NEWS
> >> entry (rather than trying to make it readable even in rendered form),
> >> but I figured I'd ask for advice here first.
> >
> > I thought the NEWS file was mainly regarded as plain text, so it would
> > be important to avoid ugly RST markup like backslashes if possible.
>
> ... I think you're right on this point, so it makes more sense to skip
> the escaping entirely,
> and just use the correct link markup in the NEWS entry.

How convenient.

>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com|   Brisbane,
> Australia
> ___
> Python-Dev mailing list
> Python-Dev@python.org 
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> wes.turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Escaping docs markup in NEWS entries?

2017-05-01 Thread Nick Coghlan 
On 1 May 2017 at 17:13, Martin Panter  wrote:
> On 1 May 2017 at 06:37, Nick Coghlan  wrote:
>> Hi folks,
>>
>> I'm trying to write a NEWS entry that explains that the
>> ":func:`bytes`" cross-references have changed to refer to the type
>> descriptions by default (matching other builtin container types), so
>> you now need to use ``:ref:`func-bytes`" to refer to the old target in
>> the list of builtin functions (if you really want that for some
>> reason).
>>
>> Unfortunately, my first two attempts both cause warnings in "make
>> suspicious" with the following output:
>
> What is the full output? Usually it includes instructions to add false
> positives to Doc/tools/susp-ignored.csv; maybe that is all you have to
> do?

You're right, that would be likely be the way to go if I decided to
keep the escaped markup.

However...

>> My fallback plan is to just include the unescaped markup in the NEWS
>> entry (rather than trying to make it readable even in rendered form),
>> but I figured I'd ask for advice here first.
>
> I thought the NEWS file was mainly regarded as plain text, so it would
> be important to avoid ugly RST markup like backslashes if possible.

... I think you're right on this point, so it makes more sense to skip
the escaping entirely,
and just use the correct link markup in the NEWS entry.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Escaping docs markup in NEWS entries?

2017-05-01 Thread Martin Panter 
On 1 May 2017 at 06:37, Nick Coghlan  wrote:
> Hi folks,
>
> I'm trying to write a NEWS entry that explains that the
> ":func:`bytes`" cross-references have changed to refer to the type
> descriptions by default (matching other builtin container types), so
> you now need to use ``:ref:`func-bytes`" to refer to the old target in
> the list of builtin functions (if you really want that for some
> reason).
>
> Unfortunately, my first two attempts both cause warnings in "make
> suspicious" with the following output:

What is the full output? Usually it includes instructions to add false
positives to Doc/tools/susp-ignored.csv; maybe that is all you have to
do?

> 
> WARNING: [whatsnew/changelog:986] ":func" found in "\:func\:\`bytes\`"
> WARNING: [whatsnew/changelog:986] "`" found in "\:func\:\`bytes\`"
> WARNING: [whatsnew/changelog:986] ":func" found in "\:func\:\`bytearray\`"
> WARNING: [whatsnew/changelog:986] "`" found in "\:func\:\`bytearray\`"
> WARNING: [whatsnew/changelog:986] ":ref" found in "\:ref\:\`func-bytes\`"
> WARNING: [whatsnew/changelog:986] "`" found in "\:ref\:\`func-bytes\`"
> WARNING: [whatsnew/changelog:986] ":ref" found in "\:ref\:\`func-bytes\`"
> WARNING: [whatsnew/changelog:986] "`" found in "\:ref\:\`func-bytes\`"
> 
>
> My first attempt just escaped the nested backticks:
>
> - Issue #30052: the link targets for ``:func:\`bytes\``` and
>   ``:func:\`bytearray\``` are now their respective type definitions, rather
>   than the corresponding builtin function entries. Use ``:ref:\`func-bytes\```
>   and ``:ref:\`func-bytes\``` to reference the latter.
>
> My second attempt escaped the colons as well:
>
> - Issue #30052: the link targets for ``\:func\:\`bytes\``` and
>   ``\:func\:\`bytearray\``` are now their respective type definitions, rather
>   than the corresponding builtin function entries. Use 
> ``\:ref\:\`func-bytes\```
>   and ``\:ref\:\`func-bytes\``` to reference the latter.
>
> My fallback plan is to just include the unescaped markup in the NEWS
> entry (rather than trying to make it readable even in rendered form),
> but I figured I'd ask for advice here first.

I thought the NEWS file was mainly regarded as plain text, so it would
be important to avoid ugly RST markup like backslashes if possible.

I am no RST expert, but perhaps it would be clearer to a human RST
parser if you added a space among the last three backticks, where the
underscore is in ``:func:`bytes`_``.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Escaping docs markup in NEWS entries?

2017-04-30 Thread Nick Coghlan 
Hi folks,

I'm trying to write a NEWS entry that explains that the
":func:`bytes`" cross-references have changed to refer to the type
descriptions by default (matching other builtin container types), so
you now need to use ``:ref:`func-bytes`" to refer to the old target in
the list of builtin functions (if you really want that for some
reason).

Unfortunately, my first two attempts both cause warnings in "make
suspicious" with the following output:


WARNING: [whatsnew/changelog:986] ":func" found in "\:func\:\`bytes\`"
WARNING: [whatsnew/changelog:986] "`" found in "\:func\:\`bytes\`"
WARNING: [whatsnew/changelog:986] ":func" found in "\:func\:\`bytearray\`"
WARNING: [whatsnew/changelog:986] "`" found in "\:func\:\`bytearray\`"
WARNING: [whatsnew/changelog:986] ":ref" found in "\:ref\:\`func-bytes\`"
WARNING: [whatsnew/changelog:986] "`" found in "\:ref\:\`func-bytes\`"
WARNING: [whatsnew/changelog:986] ":ref" found in "\:ref\:\`func-bytes\`"
WARNING: [whatsnew/changelog:986] "`" found in "\:ref\:\`func-bytes\`"


My first attempt just escaped the nested backticks:

- Issue #30052: the link targets for ``:func:\`bytes\``` and
  ``:func:\`bytearray\``` are now their respective type definitions, rather
  than the corresponding builtin function entries. Use ``:ref:\`func-bytes\```
  and ``:ref:\`func-bytes\``` to reference the latter.

My second attempt escaped the colons as well:

- Issue #30052: the link targets for ``\:func\:\`bytes\``` and
  ``\:func\:\`bytearray\``` are now their respective type definitions, rather
  than the corresponding builtin function entries. Use ``\:ref\:\`func-bytes\```
  and ``\:ref\:\`func-bytes\``` to reference the latter.

My fallback plan is to just include the unescaped markup in the NEWS
entry (rather than trying to make it readable even in rendered form),
but I figured I'd ask for advice here first.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com