[Python-Dev] Re: Mixed Python/C debugging

[Wes Turner]

"Debugging a Mixed Python and C Language Stack" (2023)
https://developer.nvidia.com/blog/debugging-mixed-python-and-c-language-stack/
https://news.ycombinator.com/item?id=35706687

On Mon, Dec 2, 2019, 10:59 AM Skip Montanaro 
wrote:

> Thanks for the responses. I know there are multiple tools out there (to
> wit, Wes's response), but I'm really after what people actually use and
> find works. I apologize that wasn't clear. I did neglect to mention that my
> environment is Linux (specifically Ubuntu 18.04), so Windows-based
> solutions aren't likely to be workable for me.
>
> For the time being, I've been working through one or two of the
> docs/tutorials about the parsing/compiler internals which focus on the C
> side, so gdb with curses display enabled (Ctrl-X a) and built-in PyObject
> support has been sufficient. I will eventually need mixed language
> debugging though. And, as an Emacs user, how this might play in that
> sandbox is of interest.
>
> Skip
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/IZRJX3YYOBJWJ6UAE5PIAJBPKB7IOHS2/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/WSJWVGLZKF3JD7Y6LDHWYKAT3AJ7LETS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Contributing the Pyston jit?

[Wes Turner]

Heads up: if CPython is JIT faster, it may then be (even more) vulnerable
to branch prediction vulns like Spectre and Meltdown:
https://groups.google.com/g/dev-python/c/67Et2KtpzG4

There's not a PEP for this work either, and one will need to be rebased
first, and are there even merge collisions?

On Thu, Feb 23, 2023, 9:00 PM Kevin Modzelewski  wrote:

> Ah ok thanks for the tip, I re-posted this as
> https://discuss.python.org/t/contributing-the-pyston-jit/24195
>
> On Thu, Feb 23, 2023 at 6:02 PM Brett Cannon  wrote:
>
>> FYI you will probably get more engagement if you posted this to
>> discuss.python.org .
>>
>> On Thu, Feb 23, 2023, 10:18 Kevin Modzelewski  wrote:
>>
>>> Hello all, we on the Pyston team would like to propose the contribution
>>> of our JIT
>>>  
>>> into
>>> CPython main. We're interested in some initial feedback on this idea before
>>> putting in the work to rebase the jit to 3.12 for a PEP and more formal
>>> discussion.
>>>
>>> Our jit is designed to be simple and to generate code quickly, so we
>>> believe it's a good point on the design tradeoff curve for potential
>>> inclusion. The runtime behavior is intentionally kept almost completely the
>>> same as the interpreter, just lowered to machine code and with
>>> optimizations applied.
>>>
>>> Our jit currently targets Python 3.7-3.10, and on 3.8 it achieves a 10%
>>> speedup on macrobenchmarks (similar to 3.11). It's hard to estimate the
>>> potential speedup of our jit rebased onto 3.12 because there is overlap
>>> between what our jit does and the optimizations that have gone into the
>>> interpreter since 3.8, but there are several optimizations that would be
>>> additive with the current performance work:
>>> - Eliminating bytecode dispatch overhead
>>> - Mostly-eliminating stack management overhead
>>> - Reducing the number of reference count operations in the interpreter
>>> - Faster function calls, particularly of C functions
>>> - More specialization opportunities, both because a jit is not limited
>>> by bytecode limits, but also because it is able to do dynamic
>>> specializations that are not possible in an interpreter context
>>>
>>> There is also room for more optimizations -- in Pyston we've
>>> co-optimized the interpreter+jit combination such as by doing more
>>> extensive profiling in the interpreter. Our plan would be to submit an
>>> initial version that does not contain these optimizations in order to
>>> minimize the diff, and add them later.
>>>
>>> Our jit uses the DynASM assembler library (part of LuaJIT) to generate
>>> machine code. Our jit currently supports Mac and Linux, 64-bit ARM and
>>> x86_64. Now that we have two architectures supported, adding additional
>>> ones is not too much work.
>>>
>>> We think that our jit fits nicely in the technical roadmap of the Faster
>>> CPython project, but conflicts with their plan to build a new custom
>>> tracing jit.
>>>
>>>
>>> As mentioned, we'd love to get feedback about the overall appetite for
>>> including a jit in CPython!
>>>
>>> kmod
>>> ___
>>> Python-Dev mailing list -- python-dev@python.org
>>> To unsubscribe send an email to python-dev-le...@python.org
>>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>>> Message archived at
>>> https://mail.python.org/archives/list/python-dev@python.org/message/7HTSM36GBTP4H5HEUV4JMCDSVYVFFGGV/
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>>
>> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/TUTSBGG7D7HW6MFHVX46IQDWAF3MJJLS/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6DUSEL5UHLS3NDIAFN4K3LOUAR5C72QF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Contributing the Pyston jit?

[Wes Turner]

Please consider colesbury/nogil in rebasing?
https://github.com/colesbury/nogil

On Thu, Feb 23, 2023, 1:20 PM Kevin Modzelewski  wrote:

> Hello all, we on the Pyston team would like to propose the contribution of
> our JIT
>  
> into
> CPython main. We're interested in some initial feedback on this idea before
> putting in the work to rebase the jit to 3.12 for a PEP and more formal
> discussion.
>
> Our jit is designed to be simple and to generate code quickly, so we
> believe it's a good point on the design tradeoff curve for potential
> inclusion. The runtime behavior is intentionally kept almost completely the
> same as the interpreter, just lowered to machine code and with
> optimizations applied.
>
> Our jit currently targets Python 3.7-3.10, and on 3.8 it achieves a 10%
> speedup on macrobenchmarks (similar to 3.11). It's hard to estimate the
> potential speedup of our jit rebased onto 3.12 because there is overlap
> between what our jit does and the optimizations that have gone into the
> interpreter since 3.8, but there are several optimizations that would be
> additive with the current performance work:
> - Eliminating bytecode dispatch overhead
> - Mostly-eliminating stack management overhead
> - Reducing the number of reference count operations in the interpreter
> - Faster function calls, particularly of C functions
> - More specialization opportunities, both because a jit is not limited by
> bytecode limits, but also because it is able to do dynamic specializations
> that are not possible in an interpreter context
>
> There is also room for more optimizations -- in Pyston we've co-optimized
> the interpreter+jit combination such as by doing more extensive profiling
> in the interpreter. Our plan would be to submit an initial version that
> does not contain these optimizations in order to minimize the diff, and add
> them later.
>
> Our jit uses the DynASM assembler library (part of LuaJIT) to generate
> machine code. Our jit currently supports Mac and Linux, 64-bit ARM and
> x86_64. Now that we have two architectures supported, adding additional
> ones is not too much work.
>
> We think that our jit fits nicely in the technical roadmap of the Faster
> CPython project, but conflicts with their plan to build a new custom
> tracing jit.
>
>
> As mentioned, we'd love to get feedback about the overall appetite for
> including a jit in CPython!
>
> kmod
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/7HTSM36GBTP4H5HEUV4JMCDSVYVFFGGV/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EL7O7TJNYBFH2MRAV2AZCBUWV2TVUXMW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] RFC: New/update the Docs Sphinx theme: responsive, edit page links,

[Wes Turner]

Where would be the best place to discuss or just improve the CPython docs
sphinx theme? What would Python need in a {PyData,} Sphinx theme fork:

- [ ] ENH: add  links to every docs page
- [ ] ENH: responsive breakpoints for mobile devices
- [ ] ENH: responsive fonts

- [ ] ENH: toggle-able dark theme

- [ ] ENH: interactive code examples (in WASM with pyodide like JupyterLite)

- [ ] ENH?: Two sidebars: {Global TOC} {Content} {Document TOC}

The PyData theme [bandwagon, feature justification, maintenance burden
justification]
Src: https://github.com/pydata/pydata-sphinx-theme
Docs: https://pydata-sphinx-theme.readthedocs.io/en/latest/

```quote
Other sites that are using this theme:

- Pandas: https://pandas.pydata.org/docs/
- NumPy: https://numpy.org/doc/stable/
- SciPy: https://scipy.github.io/devdocs/
- NetworkX: https://networkx.org/documentation/latest/
- Bokeh: https://docs.bokeh.org/en/latest/
- JupyterHub and Binder: https://docs.mybinder.org/,
http://z2jh.jupyter.org/en/latest/,
https://repo2docker.readthedocs.io/en/latest/,
https://jupyterhub-team-compass.readthedocs.io/en/latest/
- Jupyter Book beta version uses an extension of this theme:
https://beta.jupyterbook.org
- Fairlearn: https://fairlearn.org/main/about/
```

The sphinx-book-theme theme (Jupyter Book theme) has a few possibly
necessary features for the {CPython, Devguide, *} docs:
https://github.com/executablebooks/sphinx-book-theme
https://sphinx-book-theme.readthedocs.io/en/latest/ ::

```quote
A Sphinx theme with a clean design, support for interactive content, and a
modern book-like look and feel.

Flexible content layout: Inspired by beautiful online books, such as the
Edward Tufte CSS guide

Visual classes designed for Jupyter Notebooks: Cell inputs, outputs, and
interactive functionality are all supported.

Launch buttons for online interactivity: For pages that are built with
computational material, connect your site to an online BinderHub for
interactive content.

Bootstrap 4: To style visual elements and add functionality.

International: All text integrated in the theme is translated to the
specified Sphinx language.
```

https://github.com/python/cpython/blob/main/Doc/conf.py#L68 :
```python
html_theme = 'python_docs_theme'
```

https://github.com/python/cpython/blob/main/Doc/requirements.txt#L12
```
# The theme used by the documentation is stored separately, so we need
# to install that as well.
python-docs-theme
```

Pypi: https://pypi.org/project/python-docs-theme/
Src: https://github.com/python/python-docs-theme/

What are some better ways to add {Source RST, Source, Edit, History, and
Annotate} links to every sphinx doc page than sphinxcontrib-srclinks?
https://github.com/westurner/sphinxcontrib-srclinks#usage
- sphinxcontrib-srclinks: Source RST, Source, Edit, History, and Annotate,
Clone URL
  - [ ] copy clone URL to clipboard button
- pydata-theme: 'Edit this page' on the document TOC sidebar
- sphinx-book-theme: {gh, bb, gl} icon > 'Suggest edit'

- [ ] ENH: add collapsible sidebar to whatever is chosen as a suitable
replacement for the current non-responsive theme
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HIEEIJCY3WMBYVKDCE4BQDPRF5T5LX2B/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: [slightly OT] cryptographically strong random.SystemRandom()

[Wes Turner]

"PEP 543 -- A Unified TLS API for Python" could specify a [CS][P][RNG]
interface that could be used instead of os.urandom, which is probably also
wrong.

PEP 543 compares  OpenSSL, SecureTransport, SChannel, and NSS; which
presumably all have some sort of a CSPRNG function that may or may not need
to be initialized with state and paramaetrs.

https://wiki.openssl.org/index.php/Random_Numbers#Generation

Hopefully, hardware support for RNGs that OpenSSL already has is already
functionally extant in the various OSes as well:
https://wiki.openssl.org/index.php/Random_Numbers#Hardware

https://en.wikipedia.org/wiki//dev/random

On Sat, Jul 10, 2021 at 7:54 PM Wes Turner  wrote:

> * Citation: https://cryptography.io/en/latest/random-numbers/
>
> On Sat, Jul 10, 2021 at 7:53 PM Wes Turner  wrote:
>
>> FWIW, here is what https://cryptography.io has re: random (/? rng python
>> cryptography)
>>
>> ```rst
>> Random number generation
>> 
>>
>> When generating random data for use in cryptographic operations, such as
>> an
>> initialization vector for encryption in
>> :class:`~cryptography.hazmat.primitives.ciphers.modes.CBC` mode, you do
>> not
>> want to use the standard :mod:`random` module APIs. This is because they
>> do not
>> provide a cryptographically secure random number generator, which can
>> result in
>> major security issues depending on the algorithms in use.
>>
>> Therefore, it is our recommendation to `always use your operating system's
>> provided random number generator`_, which is available as
>> :func:`os.urandom`.
>> For example, if you need 16 bytes of random data for an initialization
>> vector,
>> you can obtain them with:
>>
>> .. doctest::
>>
>> >>> import os
>> >>> iv = os.urandom(16)
>>
>> This will use ``/dev/urandom`` on UNIX platforms, and ``CryptGenRandom``
>> on
>> Windows.
>>
>> If you need your random number as an integer (for example, for
>> :meth:`~cryptography.x509.CertificateBuilder.serial_number`), you can use
>> ``int.from_bytes`` to convert the result of ``os.urandom``:
>>
>> .. code-block:: pycon
>>
>> >>> serial = int.from_bytes(os.urandom(20), byteorder="big")
>>
>> Starting with Python 3.6 the `standard library includes`_ the ``secrets``
>> module, which can be used for generating cryptographically secure random
>> numbers, with specific helpers for text-based formats.
>>
>> .. _`always use your operating system's provided random number
>> generator`:
>> https://sockpuppet.org/blog/2014/02/25/safely-generate-random-numbers/
>> .. _`standard library includes`:
>> https://docs.python.org/3/library/secrets.html
>> ```
>>
>> - https://docs.python.org/3/library/random.html
>> - https://docs.python.org/3/library/random.html#random.SystemRandom
>>   - https://docs.python.org/3/library/os.html#os.urandom (see below for
>> the docs)
>> - https://docs.python.org/3/library/secrets.html
>> -
>> https://numpy.org/doc/stable/reference/random/generator.html#simple-random-data
>> -
>> https://docs.scipy.org/doc/scipy/reference/tutorial/stats/discrete.html#discrete-distributions-in-scipy-stats
>>   -
>> https://docs.scipy.org/doc/scipy/reference/tutorial/stats/discrete_randint.html
>> - https://docs.python.org/3/library/os.html#os.urandom :
>>
>> ```rst
>> Random numbers
>> --
>>
>>
>> .. function:: getrandom(size, flags=0)
>>
>>Get up to *size* random bytes. The function can return less bytes than
>>requested.
>>
>>These bytes can be used to seed user-space random number generators or
>> for
>>cryptographic purposes.
>>
>>``getrandom()`` relies on entropy gathered from device drivers and
>> other
>>sources of environmental noise. Unnecessarily reading large quantities
>> of
>>data will have a negative impact on  other users  of the
>> ``/dev/random`` and
>>``/dev/urandom`` devices.
>>
>>The flags argument is a bit mask that can contain zero or more of the
>>following values ORed together: :py:data:`os.GRND_RANDOM` and
>>:py:data:`GRND_NONBLOCK`.
>>
>>See also the `Linux getrandom() manual page
>><http://man7.org/linux/man-pages/man2/getrandom.2.html>`_.
>>
>>.. availability:: Linux 3.17 and newer.
>>
>>.. versionadded:: 3.6
>>
>> .. function:: urandom(size)
>>
>>Return a string of *size* random bytes suita

[Python-Dev] Re: [slightly OT] cryptographically strong random.SystemRandom()

[Wes Turner]

* Citation: https://cryptography.io/en/latest/random-numbers/

On Sat, Jul 10, 2021 at 7:53 PM Wes Turner  wrote:

> FWIW, here is what https://cryptography.io has re: random (/? rng python
> cryptography)
>
> ```rst
> Random number generation
> 
>
> When generating random data for use in cryptographic operations, such as an
> initialization vector for encryption in
> :class:`~cryptography.hazmat.primitives.ciphers.modes.CBC` mode, you do not
> want to use the standard :mod:`random` module APIs. This is because they
> do not
> provide a cryptographically secure random number generator, which can
> result in
> major security issues depending on the algorithms in use.
>
> Therefore, it is our recommendation to `always use your operating system's
> provided random number generator`_, which is available as
> :func:`os.urandom`.
> For example, if you need 16 bytes of random data for an initialization
> vector,
> you can obtain them with:
>
> .. doctest::
>
> >>> import os
> >>> iv = os.urandom(16)
>
> This will use ``/dev/urandom`` on UNIX platforms, and ``CryptGenRandom`` on
> Windows.
>
> If you need your random number as an integer (for example, for
> :meth:`~cryptography.x509.CertificateBuilder.serial_number`), you can use
> ``int.from_bytes`` to convert the result of ``os.urandom``:
>
> .. code-block:: pycon
>
> >>> serial = int.from_bytes(os.urandom(20), byteorder="big")
>
> Starting with Python 3.6 the `standard library includes`_ the ``secrets``
> module, which can be used for generating cryptographically secure random
> numbers, with specific helpers for text-based formats.
>
> .. _`always use your operating system's provided random number generator`:
> https://sockpuppet.org/blog/2014/02/25/safely-generate-random-numbers/
> .. _`standard library includes`:
> https://docs.python.org/3/library/secrets.html
> ```
>
> - https://docs.python.org/3/library/random.html
> - https://docs.python.org/3/library/random.html#random.SystemRandom
>   - https://docs.python.org/3/library/os.html#os.urandom (see below for
> the docs)
> - https://docs.python.org/3/library/secrets.html
> -
> https://numpy.org/doc/stable/reference/random/generator.html#simple-random-data
> -
> https://docs.scipy.org/doc/scipy/reference/tutorial/stats/discrete.html#discrete-distributions-in-scipy-stats
>   -
> https://docs.scipy.org/doc/scipy/reference/tutorial/stats/discrete_randint.html
> - https://docs.python.org/3/library/os.html#os.urandom :
>
> ```rst
> Random numbers
> --
>
>
> .. function:: getrandom(size, flags=0)
>
>Get up to *size* random bytes. The function can return less bytes than
>requested.
>
>These bytes can be used to seed user-space random number generators or
> for
>cryptographic purposes.
>
>``getrandom()`` relies on entropy gathered from device drivers and other
>sources of environmental noise. Unnecessarily reading large quantities
> of
>data will have a negative impact on  other users  of the
> ``/dev/random`` and
>``/dev/urandom`` devices.
>
>The flags argument is a bit mask that can contain zero or more of the
>following values ORed together: :py:data:`os.GRND_RANDOM` and
>:py:data:`GRND_NONBLOCK`.
>
>See also the `Linux getrandom() manual page
><http://man7.org/linux/man-pages/man2/getrandom.2.html>`_.
>
>.. availability:: Linux 3.17 and newer.
>
>.. versionadded:: 3.6
>
> .. function:: urandom(size)
>
>Return a string of *size* random bytes suitable for cryptographic use.
>
>This function returns random bytes from an OS-specific randomness
> source.  The
>returned data should be unpredictable enough for cryptographic
> applications,
>though its exact quality depends on the OS implementation.
>
>On Linux, if the ``getrandom()`` syscall is available, it is used in
>blocking mode: block until the system urandom entropy pool is
> initialized
>(128 bits of entropy are collected by the kernel). See the :pep:`524`
> for
>the rationale. On Linux, the :func:`getrandom` function can be used to
> get
>random bytes in non-blocking mode (using the :data:`GRND_NONBLOCK`
> flag) or
>to poll until the system urandom entropy pool is initialized.
>
>On a Unix-like system, random bytes are read from the ``/dev/urandom``
>device. If the ``/dev/urandom`` device is not available or not
> readable, the
>:exc:`NotImplementedError` exception is raised.
>
>On Windows, it will use ``CryptGenRandom()``.
>
>.. seealso::
>   The :mod:`secrets` module provides h

[Python-Dev] Re: [slightly OT] cryptographically strong random.SystemRandom()

[Wes Turner]

FWIW, here is what https://cryptography.io has re: random (/? rng python
cryptography)

```rst
Random number generation


When generating random data for use in cryptographic operations, such as an
initialization vector for encryption in
:class:`~cryptography.hazmat.primitives.ciphers.modes.CBC` mode, you do not
want to use the standard :mod:`random` module APIs. This is because they do
not
provide a cryptographically secure random number generator, which can
result in
major security issues depending on the algorithms in use.

Therefore, it is our recommendation to `always use your operating system's
provided random number generator`_, which is available as
:func:`os.urandom`.
For example, if you need 16 bytes of random data for an initialization
vector,
you can obtain them with:

.. doctest::

>>> import os
>>> iv = os.urandom(16)

This will use ``/dev/urandom`` on UNIX platforms, and ``CryptGenRandom`` on
Windows.

If you need your random number as an integer (for example, for
:meth:`~cryptography.x509.CertificateBuilder.serial_number`), you can use
``int.from_bytes`` to convert the result of ``os.urandom``:

.. code-block:: pycon

>>> serial = int.from_bytes(os.urandom(20), byteorder="big")

Starting with Python 3.6 the `standard library includes`_ the ``secrets``
module, which can be used for generating cryptographically secure random
numbers, with specific helpers for text-based formats.

.. _`always use your operating system's provided random number generator`:
https://sockpuppet.org/blog/2014/02/25/safely-generate-random-numbers/
.. _`standard library includes`:
https://docs.python.org/3/library/secrets.html
```

- https://docs.python.org/3/library/random.html
- https://docs.python.org/3/library/random.html#random.SystemRandom
  - https://docs.python.org/3/library/os.html#os.urandom (see below for the
docs)
- https://docs.python.org/3/library/secrets.html
-
https://numpy.org/doc/stable/reference/random/generator.html#simple-random-data
-
https://docs.scipy.org/doc/scipy/reference/tutorial/stats/discrete.html#discrete-distributions-in-scipy-stats
  -
https://docs.scipy.org/doc/scipy/reference/tutorial/stats/discrete_randint.html
- https://docs.python.org/3/library/os.html#os.urandom :

```rst
Random numbers
--


.. function:: getrandom(size, flags=0)

   Get up to *size* random bytes. The function can return less bytes than
   requested.

   These bytes can be used to seed user-space random number generators or
for
   cryptographic purposes.

   ``getrandom()`` relies on entropy gathered from device drivers and other
   sources of environmental noise. Unnecessarily reading large quantities of
   data will have a negative impact on  other users  of the ``/dev/random``
and
   ``/dev/urandom`` devices.

   The flags argument is a bit mask that can contain zero or more of the
   following values ORed together: :py:data:`os.GRND_RANDOM` and
   :py:data:`GRND_NONBLOCK`.

   See also the `Linux getrandom() manual page
   `_.

   .. availability:: Linux 3.17 and newer.

   .. versionadded:: 3.6

.. function:: urandom(size)

   Return a string of *size* random bytes suitable for cryptographic use.

   This function returns random bytes from an OS-specific randomness
source.  The
   returned data should be unpredictable enough for cryptographic
applications,
   though its exact quality depends on the OS implementation.

   On Linux, if the ``getrandom()`` syscall is available, it is used in
   blocking mode: block until the system urandom entropy pool is initialized
   (128 bits of entropy are collected by the kernel). See the :pep:`524` for
   the rationale. On Linux, the :func:`getrandom` function can be used to
get
   random bytes in non-blocking mode (using the :data:`GRND_NONBLOCK` flag)
or
   to poll until the system urandom entropy pool is initialized.

   On a Unix-like system, random bytes are read from the ``/dev/urandom``
   device. If the ``/dev/urandom`` device is not available or not readable,
the
   :exc:`NotImplementedError` exception is raised.

   On Windows, it will use ``CryptGenRandom()``.

   .. seealso::
  The :mod:`secrets` module provides higher level functions. For an
  easy-to-use interface to the random number generator provided by your
  platform, please see :class:`random.SystemRandom`.

   .. versionchanged:: 3.6.0
  On Linux, ``getrandom()`` is now used in blocking mode to increase the
  security.

   .. versionchanged:: 3.5.2
  On Linux, if the ``getrandom()`` syscall blocks (the urandom entropy
pool
  is not initialized yet), fall back on reading ``/dev/urandom``.

   .. versionchanged:: 3.5
  On Linux 3.17 and newer, the ``getrandom()`` syscall is now used
  when available.  On OpenBSD 5.6 and newer, the C ``getentropy()``
  function is now used. These functions avoid the usage of an internal
file
  descriptor.

.. data:: 

[Python-Dev] Re: In what tense should the changelog be written?

[Wes Turner]

On Fri, Apr 30, 2021, 16:28  wrote:

> This was also my understanding. The last time I looked at writing good
> commit messages (a similar form), the standard was to finish the sentence:
> "This commit will...", e.g. "This commit will fix buffalo.spam" from "Fix
> buffalo.spam".
>

I like this a lot.

This commit/PR will ___.


Is it the objective or what is actually done?

There could be a recommended set of "codelabels" for which there are also
gh issue/pr labels.

Where in the devguide should this be summarized?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/R3IFKCBT5ZVWGOURGKCYKDGRE2ZBVDPM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Have virtual environments led to neglect of the actual environment?

[Wes Turner]

Is there a tool that (1) detects import name collisions; and (2) attempts
to read package metadata and package file checksums (maybe from the ZIP
'manifest')?

In order to:

- troubleshoot module shadowing issues
  - $PATH
  - sys.path
- `python -m site`
  - incomplete and overlapping uninstallations:

pip install a
pip install a_modified # pip uninstall a?
pip install pdbpp
pip uninstall a_modified
ls -altr "${site-packages[*]}"
strace -e trace=file python -c 'import pdb'



When shouldn't site customizations be added to the site module?
https://docs.python.org/3/library/site.html

When should customizations be built into the build instead of a runtime
conditional?


On Sat, Feb 27, 2021, 23:12 Nick Coghlan  wrote:

> On Wed, 24 Feb 2021 at 10:49, Random832  wrote:
> >
> > I was reading a discussion thread <
> https://gist.github.com/tiran/2dec9e03c6f901814f6d1e8dad09528e> about
> various issues with the Debian packaged version of Python, and the
> following statement stood out for me as shocking:
> >
> > Christian Heimes wrote:
> > > Core dev and PyPA has spent a lot of effort in promoting venv because
> we don't want users to break their operating system with sudo pip install.
> >
> > I don't think sudo pip install should break the operating system. And I
> think if it does, that problem should be solved rather than merely advising
> users against using it. And why is it, anyway, that distributions whose
> package managers can't coexist with pip-installed packages don't ever seem
> to get the same amount of flak for "damaging python's brand" as Debian is
> getting from some of the people in the discussion thread? Why is it that
> this community is resigned to recommending a workaround when distributions
> decide the site-packages directory belongs to their package manager rather
> than pip, instead of bringing the same amount of fiery condemnation of that
> practice as we apparently have for *checks notes* splitting parts of the
> stdlib into optional packages? Why demand that pip be present if we're not
> going to demand that it works properly?
>
> The reason venv is promoted as heavily as it is is because it's the
> only advice that can be given that is consistently correct regardless
> of the operating system the user is running locally, whereas safely
> using a system-wide Python installation varies a lot depending on
> whether you're on Windows, Mac OS X, or Linux (let alone some other
> platform outside the big 3 desktop clients).
>
> conda is also popular for the same reason: while the instructions for
> installing conda in the first place are OS-dependent, once it is up
> and running you can use consistent platform independent conda commands
> rather than having to caveat all your documentation with
> platform-specific instructions.
>
> Apple moved all of their dynamic language interpreter implementations
> to inaccessible-by-default locations so Mac OS X users would stop
> using them to run their own code.
>
> Alongside that, we *have* worked with the Linux distro vendors to help
> make "sudo pip install" safe (e.g [1]), but that only helps if a user
> is running a new enough version of a distro that has participated in
> that work.
>
> However, while the option of running "platform native" environments
> will never go away, and work will continue to make it less error
> prone, the level of knowledge of your specific OS's idiosyncrasies
> that it requires is almost certainly going to remain too high for it
> to ever again become the default recommendation that it used to be.
>
> Cheers,
> Nick.
>
> [1] https://fedoraproject.org/wiki/Changes/Making_sudo_pip_safe (Note:
> this change mitigated some aspects of the problem in a way similar to
> what Debian does, but still doesn't solve it completely, as custom
> Python builds may still make arbitrary changes)
>
> P.S. "But what about user site-packages?" you ask. Until relatively
> recently, Debian didn't put the user's local bin directory on the
> system path by default, so commands provided by user level package
> installs didn't work without the user adjusting their PATH. The
> CPython Windows installer also doesn't adjust PATH by default (for
> good reasons). And unlike a venv, "python -m" doesn't let you ensure
> that the code executed is the version installed in user site-packages
> - it could be coming from a directory earlier in sys.path.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/RDLEH6DUF57UB6U4HNL2QRVAJY4KDSSJ/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an 

[Python-Dev] Re: Have virtual environments led to neglect of the actual environment?

[Wes Turner]

One aspect of conda packaging that's worth adopting is the 'test' block of
the meta.yaml which contains commands necessary to *test* the built package.

You can add a pytest command to setup.py, but it's not as easy as:
https://github.com/conda-forge/python-feedstock/blob/master/recipe/meta.yaml#L227-L291
:

```yaml
test:
  downstreams:
- cython
- setuptools
  requires:
- ripgrep
- cmake
- ninja
- {{ compiler('c') }}
# Tried to use enable_language(C) to avoid needing this. It does
not work.
- {{ compiler('cxx') }}
  files:
- tests/distutils/*
- tests/cmake/*
- tests/cython/*
- tests/prefix-replacement/*
  commands:
- echo on  # [win]
- set  # [win]
- python -V
- python3 -V# [not win]
- 2to3 -h
- pydoc -h
- python3-config --help  # [not win]
- set "PIP_NO_BUILD_ISOLATION=False"  # [win]
- set "PIP_NO_DEPENDENCIES=True"  # [win]
- set "PIP_IGNORE_INSTALLED=True"  # [win]
- set "PIP_NO_INDEX=True"  # [win]
- set "PIP_CACHE_DIR=%CONDA_PREFIX%/pip_cache"  # [win]
- set "TEMP=%CONDA_PREFIX%/tmp"  # [win]
- mkdir "%TEMP%"  # [win]
- python -Im ensurepip --upgrade --default-pip  # [win]
# tzdata/zoneinfo test that will need the tzdata package to pass
- python -c "from zoneinfo import ZoneInfo; from datetime import
datetime; dt = datetime(2020, 10, 31, 12,
tzinfo=ZoneInfo('America/Los_Angeles')); print(dt.tzname())"
- python -m venv test-venv
- python -c "import sysconfig;
print(sysconfig.get_config_var('CC'))"  # [not win]
-
_CONDA_PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata_x86_64_conda_cos6_linux_gnu
python -c "import sysconfig; print(sysconfig.get_config_var('CC'))"  #
[linux64]
# check for unreplaced @ symbols in sysconfig files
- for f in ${CONDA_PREFIX}/lib/python*/_sysconfig*.py; do echo
"Checking $f:"; if [[ `rg @ $f` ]]; then echo "FAILED ON $f"; cat $f; exit
1; fi; done  # [linux64 or osx]
- test ! -f ${PREFIX}/lib/libpython${PKG_VERSION%.*}.a  # [unix]
- test ! -f ${PREFIX}/lib/libpython${PKG_VERSION%.*}.nolto.a  #
[unix]
# https://github.com/conda-forge/python-feedstock/issues/384
- if exist %PREFIX%\\Scripts\\pydoc exit 1  # [win]
- if exist %PREFIX%\\Scripts\\idle exit 1  # [win]
- if exist %PREFIX%\\Scripts\\2to3 exit 1  # [win]
- if not exist %PREFIX%\\Scripts\\pydoc-script.py exit 1  # [win]
- if not exist %PREFIX%\\Scripts\\idle-script.py exit 1  # [win]
- if not exist %PREFIX%\\Scripts\\2to3-script.py exit 1  # [win]
- if not exist %PREFIX%\\Scripts\\idle.exe exit 1  # [win]
- if not exist %PREFIX%\\Scripts\\2to3.exe exit 1  # [win]
- if not exist %PREFIX%\\Scripts\\pydoc.exe exit 1  # [win]
- pushd tests
-   pushd distutils
- python setup.py install -v -v
- python -c "import foobar"
-   popd
-   pushd prefix-replacement  # [unix]
- bash build-and-test.sh  # [unix]
-   popd  # [unix]
-   pushd cmake
- cmake -GNinja -DPY_VER={{ version }}
  # --trace --debug-output --debug-trycompile .
-   popd
- popd
- test ! -f default.profraw   # [osx]
```


In which package you then store integration tests for your whole
application and it's integrated dependencies is then up to you.
Unfortunately, we often don't include tests/ in our packages, so it's
literally not possible to run the tests in production even if you have
pytest_requires installed.

Perhaps distros would do well to implement support for integration and
per-package tests.

On Thu, Feb 25, 2021, 12:55 Fred Drake  wrote:

> On Thu, Feb 25, 2021 at 5:35 AM Wes Turner  wrote:
>
>> The challenge with version conflicts is often less that you need to go
>> update the constraints (which has little to do with sysadmin'ing, TBH) and
>> more that you have insufficient *integration tests* and you're relying upon
>> something else running the per-package tears.
>>
>
> Sometimes, auto-correct really does understand!
>
> Your point is right on target, and really can't be understated.
>
>
>   -Fred
>
> --
> Fred L. Drake, Jr.
> "There is nothing more uncommon than common sense."
>   --Frank Lloyd Wright
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2QMXHSDF5L4QEJU5C2FVZ3AZHWEHUMXA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Have virtual environments led to neglect of the actual environment?

[Wes Turner]

The challenge with version conflicts is often less that you need to go
update the constraints (which has little to do with sysadmin'ing, TBH) and
more that you have insufficient *integration tests* and you're relying upon
something else running the per-package tears.

On Thu, Feb 25, 2021, 00:10 Peter Wang  wrote:

> On Wed, Feb 24, 2021 at 8:50 PM Mike Miller 
> wrote:
>
>> I never understood the fear around version conflicts.
>
>
> With binary extension modules, version conflicts lead to (at best) runtime
> segfault and (at worst) subtle *data* bugs that return incorrect results.
> There are also deeper concerns around security and reproducibility.
>
>
>> Perhaps it has to do with
>> the decline of sys-admin skills over the years?
>
>
> Many millions of users of new Python users show up every year, using the
> language and its powerful ecosystem for data analytics and scientific
> computing, and they have no hope of having sys-admin skills.
>
> -Peter
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/TYZ6QT4BVQY7SVXV6E63YKNV6SCNFZ7V/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/O2TB7YIP6NHTBORZJNFFNU45VV2C26OV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Have virtual environments led to neglect of the actual environment?

[Wes Turner]

Would it be better to have an (os, os_release, arch) to
[site_packages_dirs] map within CPython or are the patches too varied to be
replaced by a centralized map and a runtime conditional?

For development convenience, having writeable site-packages dirs before
unwriteable site-packages is easy;

But for security and production deployment, site-packages maybe shouldn't
be writeable at all by default because that's doing it wrong?

On Wed, Feb 24, 2021, 15:34 Christian Heimes  wrote:

> On 24/02/2021 20.03, Christian Heimes wrote:
> > On 24/02/2021 19.17, Steve Dower wrote:
> >> On 2/24/2021 4:26 PM, Christian Heimes wrote:
> >>> On 24/02/2021 15.16, Random832 wrote:
>  On Wed, Feb 24, 2021, at 06:27, Christian Heimes wrote:
> > Separate directories don't prevent clashes and system breakage. But
> > they
> > provide an easy way to *recover* from a broken system.
> 
>  I think it could be turned into a way to prevent them by A) having
>  site-packages always take precedence over dist-packages [i believe
>  this is already the case] in normal usage and B) providing an option
>  to the interpreter, used by system scripts, to exclude site-packages
>  entirely from the path.
> 
>  Basically, site-packages would effectively be layered on top of "Lib
>  + dist-packages" in a similar way to how a venv is layered on top of
>  the main python installation - the inverse of the suggestion someone
>  else in the thread made for the system python to be a venv. This
>  wouldn't *exactly* be a venv because it wouldn't imply the other
>  things that entering a venv does such as "python" [and script names
>  such as pip] being an alias for the correct version of python, but it
>  would provide the same kind of one-way isolation, whereby the "system
>  environment" can influence the "normal environment" and not
>  vice-versa, in the same way that packages installed in the main
>  installation affect a venv [unless system-site-packages is disabled]
>  but the venv obviously has no effect on the main installati
> >> on.
> >>>
> >>> Yes, you are describing one major aspect of my idea for a system Python
> >>> interpreter. I'm happy to read that other users are coming to similar
> >>> conclusions. Instead of an option I'd create a new executable to lock
> >>> down additional things (e.g. isolated mode, code verification hook). A
> >>> separate executable would also allow distros to provide a stripped down
> >>> interpreter that does not cause bad user experience.
> >>
> >> I mean, this is _precisely_ what PEP 370 defines (including the "-s"
> >> option and PYTHONNOUSERSITE env variable to provide that one-way
> >> isolation).
> >>
> >> Is the problem that pip doesn't use it by default? Or that the distros
> >> went and made patches for the runtime rather than patching pip? (For
> >> Windows installs from the Store, where even admin rights can't do an
> >> all-users package install, I added a new config file location for pip to
> >> change this default, but a patch would also have worked.)
> >>
> >> Maybe we need an easier way to patch the location of user site packages?
> >> I also had to do this for the Store install on Windows, and it's a
> >> little bit of a hack... but maybe having an official recommendation
> >> would help encourage distributors to use the mechanism?
> >
> > (sorry for terse reply, I'm heading out.)
> >
> > Linux distros want an additional layer so the can differentiate between
> > Python packages installed by their package manager and Python packages
> > installed with "sudo pip install".
> >
> > Python has two default site-package directories (from highest to lowest
> > precedence):
> >
> > 1. ~/.local/lib/python3.9/site-packages
> > 2. /usr/lib/python3.9/site-packages
> >
> > (1) is defined by site.getusersitepackages(), (2) site.getsitepackages()
> >
> > Linux distro have additional site-package directories. My Fedora system
> > X86_64 system with multiarch has three additional candidates.
> >
> > 1. ~/.local/lib/python3.9/site-packages
> > 2. /usr/local/lib64/python3.9/site-packages
> > 3. /usr/local/lib/python3.9/site-packages
> > 4. /usr/lib64/python3.9/site-packages
> > 5. /usr/lib/python3.9/site-packages
> >
> > The "lib" directories are for pure Python packages whiel the "lib64"
> > directories contain X86_64 native code extensions (aka shared
> > libraries). The "/usr/lib*" directories are used for distro packages.
> > The downstream patch [1] ensures that "sudo pip install" installs
> > packages into "/usr/local/lib". AFAIK "/usr/local/lib64" is not used.
> >
> > Debian has a similar mechanism to provide
> > "/usr/lib/python3/dist-packages" [2].
> >
> > A hypothetical /usr/bin/system-python3 interpreter would only use (4)
> > and (5) as site-package directories. A user-facing /usr/bin/python3
> > would use (1) to (5).
> >
> > Christian
> >
> > [1]
> >
> 

[Python-Dev] Re: Have virtual environments led to neglect of the actual environment?

[Wes Turner]

FWIW, Distro repacks are advantageous in comparison to
"statically-bundled" releases that for example bundle in an outdated
version of OpenSSL, because when you `apt-get upgrade -y` that should
upgrade the OpenSSL that all the other distro packages depend upon.

Here's something that doesn't get called as frequently as `apt-get upgrade -y`:

```bash
pip install -U certifi
```

https://github.com/certifi/python-certifi (Mozilla's CA bundle
extracted into a Python package)

```bash
apt-get install -y ca-certificates
dnf install -y ca-certificates
```

On 2/24/21, Wes Turner  wrote:
> On 2/23/21, Random832  wrote:
>> I was reading a discussion thread
>> <https://gist.github.com/tiran/2dec9e03c6f901814f6d1e8dad09528e> about
>> various issues with the Debian packaged version of Python, and the
>> following
>> statement stood out for me as shocking:
>>
>> Christian Heimes wrote:
>>> Core dev and PyPA has spent a lot of effort in promoting venv because we
>>> don't want users to break their operating system with sudo pip install.
>>
>> I don't think sudo pip install should break the operating system. And I
>> think if it does, that problem should be solved rather than merely
>> advising
>> users against using it. And why is it, anyway, that distributions whose
>> package managers can't coexist with pip-installed packages don't ever
>> seem
>> to get the same amount of flak for "damaging python's brand" as Debian is
>> getting from some of the people in the discussion thread? Why is it that
>> this community is resigned to recommending a workaround when
>> distributions
>> decide the site-packages directory belongs to their package manager
>> rather
>> than pip, instead of bringing the same amount of fiery condemnation of
>> that
>> practice as we apparently have for *checks notes* splitting parts of the
>> stdlib into optional packages? Why demand that pip be present if we're
>> not
>> going to demand that it works properly?
>>
>> I think that installing packages into the actual python installation,
>> both
>> via distribution packaging tools and pip [and using both simultaneously -
>> the Debian model of separated dist-packages and site-packages folders
>> seems
>> like a reasonable solution to this problem] can and should be a supported
>> paradigm, and that virtual environments [or more extreme measures such as
>> shipping an entire python installation as part of an application's
>> deployment] should ideally be reserved for the rare corner cases where
>> that
>> doesn't work for some reason.
>>
>> How is it that virtual environments have become so indispensable, that
>> no-one considers installing libraries centrally to be a viable model
>> anymore? Are library maintainers making breaking changes too frequently,
>> reasoning that if someone needs the old version they can just venv it? Is
>> there some other cause?
>
> First, pip+venv is not sufficient for secure software deployment:
> something must set appropriate permissions so that the application
> cannot overwrite itself and other core libraries (in order to
> eliminate W^X violations (which e.g. Android is solving by requiring
> all installed binaries to come from an APK otherwise they won't and
> can't be labeled with the SELinux extended file atrributes necessary
> for a binary to execute; but we don't have binaries, we have an
> interpreter and arbitrary hopefully-signed somewhere source code, at
> least)).
>
> Believe it or not, this is wrong:
>
> ```bash
> # python -m venv httpbin || virtualenv httpbin
> # source httpbin/bin/activate
> mkvirtualenv httpbin
>
> pip install httpbin gunicorn
> gunicorn -b 0.0.0.0:8001 httpbin:app
>
> # python -m webbrowser http://127.0.0.1:8001
> ```
>
> It's wrong - it's insecure - because the user executing the Python
> interpreter (through gunicorn, in this case) can overwrite the app.
> W^X: has both write and execute permissions. What would be better?
>
> This would be better because pip isn't running setup.py as root (with
> non-wheels) and httpbin_exec can't modify the app interpreter or the
> code it loads at runtime:
>
> ```bash
> useradd httpbin # also creates a group also named 'httpbin'
> sudo -u httpbin sh -c ' \
> python -m venv httpbin; \
> umask 0022; \
> ./httpbin/bin/python -m pip install httpbin gunicorn'
>
> useradd httpbin_exec -G httpbin
> sudo -u httpbin_exec './httpbin/bin/gunicorn -b 0.0.0.0:8001 httpbin:app'
> ```
>
> This would be better if it worked, though there are a few caveats:
>
> ```bash
> sudo apt-get install python-gunic

[Python-Dev] Re: Have virtual environments led to neglect of the actual environment?

[Wes Turner]

On 2/23/21, Random832  wrote:
> I was reading a discussion thread
>  about
> various issues with the Debian packaged version of Python, and the following
> statement stood out for me as shocking:
>
> Christian Heimes wrote:
>> Core dev and PyPA has spent a lot of effort in promoting venv because we
>> don't want users to break their operating system with sudo pip install.
>
> I don't think sudo pip install should break the operating system. And I
> think if it does, that problem should be solved rather than merely advising
> users against using it. And why is it, anyway, that distributions whose
> package managers can't coexist with pip-installed packages don't ever seem
> to get the same amount of flak for "damaging python's brand" as Debian is
> getting from some of the people in the discussion thread? Why is it that
> this community is resigned to recommending a workaround when distributions
> decide the site-packages directory belongs to their package manager rather
> than pip, instead of bringing the same amount of fiery condemnation of that
> practice as we apparently have for *checks notes* splitting parts of the
> stdlib into optional packages? Why demand that pip be present if we're not
> going to demand that it works properly?
>
> I think that installing packages into the actual python installation, both
> via distribution packaging tools and pip [and using both simultaneously -
> the Debian model of separated dist-packages and site-packages folders seems
> like a reasonable solution to this problem] can and should be a supported
> paradigm, and that virtual environments [or more extreme measures such as
> shipping an entire python installation as part of an application's
> deployment] should ideally be reserved for the rare corner cases where that
> doesn't work for some reason.
>
> How is it that virtual environments have become so indispensable, that
> no-one considers installing libraries centrally to be a viable model
> anymore? Are library maintainers making breaking changes too frequently,
> reasoning that if someone needs the old version they can just venv it? Is
> there some other cause?

First, pip+venv is not sufficient for secure software deployment:
something must set appropriate permissions so that the application
cannot overwrite itself and other core libraries (in order to
eliminate W^X violations (which e.g. Android is solving by requiring
all installed binaries to come from an APK otherwise they won't and
can't be labeled with the SELinux extended file atrributes necessary
for a binary to execute; but we don't have binaries, we have an
interpreter and arbitrary hopefully-signed somewhere source code, at
least)).

Believe it or not, this is wrong:

```bash
# python -m venv httpbin || virtualenv httpbin
# source httpbin/bin/activate
mkvirtualenv httpbin

pip install httpbin gunicorn
gunicorn -b 0.0.0.0:8001 httpbin:app

# python -m webbrowser http://127.0.0.1:8001
```

It's wrong - it's insecure - because the user executing the Python
interpreter (through gunicorn, in this case) can overwrite the app.
W^X: has both write and execute permissions. What would be better?

This would be better because pip isn't running setup.py as root (with
non-wheels) and httpbin_exec can't modify the app interpreter or the
code it loads at runtime:

```bash
useradd httpbin # also creates a group also named 'httpbin'
sudo -u httpbin sh -c ' \
python -m venv httpbin; \
umask 0022; \
./httpbin/bin/python -m pip install httpbin gunicorn'

useradd httpbin_exec -G httpbin
sudo -u httpbin_exec './httpbin/bin/gunicorn -b 0.0.0.0:8001 httpbin:app'
```

This would be better if it worked, though there are a few caveats:

```bash
sudo apt-get install python-gunicorn python-httpbin
sudo -u nobody /usr/bin/gunicorn -b 0.0.0.0:8001 httpbin:app
```

1. Development is impossible:
- You can't edit the code in /usr/lib/python3.n/site-package/ without
root permissions.
- You should not be running an editor as root.
- You can edit distro-package files individually with e.g. sudoedit
(and then the GPG-signed package file checksums will fail when you run
`debsums` or `rpm -Va` because you've edited the file and that's
changed the hash).

- Non-root users cannot install python packages without having someone
repack (and sign it) for them.

- What do I need to do in order to patch the distro's signed repack of
the Python package released to PyPI?
  - I like how Fedora pkgs and conda-forge have per-package git repos now.
  - Conda-forge has a bot that watches PyPI for new releases and tries
sending an automated PR.
  - If I send a PR to the main branch of the source repo and it gets
merged, how long will it be before there's a distro repack built and
uploaded to the distro package index?

2. It should be installed in a chroot/jail/zone/container/context/vm
so that it cannot read other data on the machine.
The httpbin app does not need read access to 

[Python-Dev] Re: Security releases of CPython

[Wes Turner]

ed with FORTIFY_SOURCE, which provides runtime protection to
> some memory and string functions and prevents this flaw from actually
> overwriting the buffer and potentially executing code."
> => https://access.redhat.com/security/cve/cve-2021-3177
>
> I suggest you checking how your operating system built your Python
> executable, and libpython if Python is built in shared mode: hardening
> can prevent the ctypes vulnerabiliy, but also against *future*
> vulnerabilities!
>
> For example, Fedora 33 builds Python 3.9 with the following C flags:
>
> -Werror=format-security
> -Wp,-D_FORTIFY_SOURCE=2
> -fcf-protection
> -fstack-clash-protection
> -fstack-protector-strong
> -fPIC
> -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 => add the -fPIE flag
> (...)
>
> And linker flags:
>
> -Wl,-z,now
> -Wl,-z,relro
> -specs=/usr/lib/rpm/redhat/redhat-hardened-ld => add the -pie flag
> (...)
>
> You can inspect hardening options on a ELF binary using hardening-check
> tool:
>
> $ hardening-check /usr/bin/python3.9
> /usr/bin/python3.9:
>  Position Independent Executable: yes
>  Stack protected: no, not found!
>  Fortify Source functions: unknown, no protectable libc functions used
>  Read-only relocations: yes
>  Immediate binding: yes
>  Stack clash protection: unknown, no -fstack-clash-protection instructions
> found
>  Control flow integrity: yes
>
> $ hardening-check /usr/lib64/libpython3.9.so.1.0
> /usr/lib64/libpython3.9.so.1.0:
>  Position Independent Executable: no, regular shared library (ignored)
>  Stack protected: yes
>  Fortify Source functions: yes (some protected functions found)
>  Read-only relocations: yes
>  Immediate binding: yes
>  Stack clash protection: unknown, no -fstack-clash-protection instructions
> found
>
> Note: I don't know if this tool is reliable, but at least I can see
> that multiple hardening options are enabled ;-)
>
>
> On Fedora and RHEL 8, packages are built with the "annobin" extension
> which records C and linker flags (and much more information). So you
> check hardening using annocheck tool:
>
> $ annocheck /usr/lib64/libpython3.6m.so.1.0
> annocheck: Version 9.50.
> Hardened: libpython3.6m.so.1.0: PASS.
>
> $ annocheck /usr/lib64/libpython3.6m.so.1.0 -v|grep -vE '(skip|warn|info):'
> annocheck: Version 9.50.
> Hardened: /usr/lib64/libpython3.6m.so.1.0: PASS: One dynamic
> section/segment found.
> Hardened: /usr/lib64/libpython3.6m.so.1.0: PASS: Stack not executable.
> Hardened: /usr/lib64/libpython3.6m.so.1.0: PASS: DT_RPATH/DT_RUNPATH
> absent or rooted at /usr.
> Hardened: /usr/lib64/libpython3.6m.so.1.0: PASS: No RWX segments found.
> Hardened: /usr/lib64/libpython3.6m.so.1.0: PASS: No text relocations found.
> Hardened: /usr/lib64/libpython3.6m.so.1.0: PASS: No thread
> cancellation problems.
> Hardened: /usr/lib64/libpython3.6m.so.1.0: PASS: GOT/PLT relocations
> are read only.
> Hardened: /usr/lib64/libpython3.6m.so.1.0: PASS.
>
> More information about annobin:
> https://developers.redhat.com/blog/2018/02/20/annobin-storing-information-binaries/
>
> Victor
> --
> Night gathers, and now my watch begins. It shall not end until my death.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/XF33457KYRJPUZAN4C4E3KCNOA7TDL6S/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Wes Turner
https://westurner.org
https://wrdrd.com/docs/consulting/knowledge-engineering
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/E7B6OMRC6CPUM3L2TUXGF25PKHYG2XPM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Python standardization

[Wes Turner]

https://awesome-safety-critical.readthedocs.io/en/latest/
https://awesome-safety-critical.readthedocs.io/en/latest/#software-safety-standards

What is and is not constant time in Python could be added to structured
data elements in (implementations') docstrings.

*

"The Python Language Reference"
https://docs.python.org/3/reference/

https://docs.python.org/3/reference/lexical_analysis.html
https://docs.python.org/3/reference/datamodel.html
https://docs.python.org/3/reference/executionmodel.html
[...]

Python PEG Grammar:
https://docs.python.org/3/reference/grammar.html
https://github.com/python/cpython/blob/master/Grammar/python.gram
https://github.com/python/cpython/blob/master/Lib/test/test_grammar.py

"24. Changing CPython’s Grammar"
https://devguide.python.org/grammar/

https://devguide.python.org/appendix/#language-development-in-depth :

> Exploring CPython’s Internals
> Changing CPython’s Grammar
> Design of CPython’s Compiler
> Design of CPython’s Garbage Collector
> Adding to the Stdlib
> Changing the Python Language
> Porting Python to a new platform

*

Wikipedia: https://en.wikipedia.org/wiki/ECMAScript
Homepage: https://www.ecma-international.org/ecma-262/
Src: https://github.com/tc39/ecma262#ecmascript
Spec: https://www.ecma-international.org/ecma-262/
Spec: https://tc39.github.io/ecma262/
Spec: https://tc39.es/ecma262/ :

ECMA262 Spec Table of Contents:
```quote
Introduction
1 Scope
2 Conformance
3 Normative References
4 Overview
5 Notational Conventions
6 ECMAScript Data Types and Values
7 Abstract Operations
8 Syntax-Directed Operations
9 Executable Code and Execution Contexts
10 Ordinary and Exotic Objects Behaviours
11 ECMAScript Language: Source Code
12 ECMAScript Language: Lexical Grammar
13 ECMAScript Language: Expressions
14 ECMAScript Language: Statements and Declarations
15 ECMAScript Language: Functions and Classes
16 ECMAScript Language: Scripts and Modules
17 Error Handling and Language Extensions
18 ECMAScript Standard Built-in Objects
19 The Global Object
20 Fundamental Objects
21 Numbers and Dates
22 Text Processing
23 Indexed Collections
24 Keyed Collections
25 Structured Data
26 Managing Memory
27 Control Abstraction Objects
28 Reflection
29 Memory Model
A Grammar Summary
B Additional ECMAScript Features for Web Browsers
C The Strict Mode of ECMAScript
D Host Layering Points
E Corrections and Clarifications in ECMAScript 2015 with Possible
Compatibility Impact
F Additions and Changes That Introduce Incompatibilities with Prior Editions
G Colophon
H Bibliography
I Copyright & Software License
```quote

... https://en.wikipedia.org/wiki/List_of_ECMAScript_engines
https://docs.python.org/3/reference/introduction.html#alternate-implementations



On Sat, Feb 13, 2021 at 1:18 PM Stefano Borini 
wrote:

> Not sure about python, but throughout my career I had to work with
> MISRA C standardisation for critical systems.
> There is more and more that is handled by python that also needs to go
> through validation. I wonder if there's value in that?
>
>
> On Fri, 12 Feb 2021 at 18:40, Dan Stromberg  wrote:
> >
> >
> > What would it take to create an ANSI, ECMA and/or ISO standard for
> Python?
> >
> > It seems to have really helped C.
> >
> > It looks like Java isn't standardized, and it's done OK, though perhaps
> it was healthier in the past - before Oracle decided API's were ownable.
> >
> > I think standardizing Python might be really good for controlling its
> growth and avoiding featuritis.
> >
> > ___
> > Python-Dev mailing list -- python-dev@python.org
> > To unsubscribe send an email to python-dev-le...@python.org
> > https://mail.python.org/mailman3/lists/python-dev.python.org/
> > Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/JZYW4JOTANYIOLYDQ6YHRUP2TWO52OAE/
> > Code of Conduct: http://python.org/psf/codeofconduct/
>
>
>
> --
> Kind regards,
>
> Stefano Borini
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/NR3UAEPEUJCWDBQVQW3QN4JYZWZNR2IU/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/WBDCMKUYRCBG5V3KFZMNOQ3X3S7SODJH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Should set objects maintain insertion order too?

[Wes Turner]

Due to version and platform constraints, a SAT solver is necessary for
conda and (now) pip. Libsolv is one of the fastest around.

 https://github.com/QuantStack/mamba is conda implemented with libsolv (and
parallel downloading of *declarative* dependency metadata).

For general purpose graphs with implicit node instantation from edge
declaration, NetworkX has implementations of many graph algorithms.
https://networkx.github.io/documentation/stable/reference/algorithms/generated/networkx.algorithms.dag.topological_sort.html


CuGraph (a product of the RAPIDS.ai project) does graphs with CUDA (from
Python) with a "NetworkX-like API" but doesn't yet have a topological sort
(though it does have BFS)
https://github.com/rapidsai/cugraph

"pip needs a dependency resolver" + End (Fn+Right) links to the latest work
on the new pip code (that must require declarative dependency metadata)
https://github.com/pypa/pip/issues/988

That said, this implementation of topo sort could have a deterministic
output given an OrderedSet:
https://rosettacode.org/wiki/Topological_sort#Python

A table including Big-O and memory usage for the necessary data structure
and methods would be useful.

On Sun, Dec 29, 2019, 5:12 PM Tim Peters  wrote:

> [Larry]
> > It's a lightweight abstract dependency graph.  Its nodes are opaque,
> > only required to be hashable.  And it doesn't require that you give it
> > all the nodes in strict dependency order.
> >
> > When you add a node, you can also optionally specify
> > dependencies, and those dependencies aren't required
> > to be nodes that the graph has previously seen.  So you can add
> > a node A that depends on B and C, without showing it B or C
> > first.  When that happens it creates placeholder nodes for B
> > and C, and naturally these nodes have no dependencies.  You
> > can then later inform the graph that node B has dependencies
> > on X Y and Z.
> >
> > (My specific use case is a package manager.  Packages are identified
> > with unique strings.  So the nodes I give the give the graph are simply
> > those package names.  It's pretty common for me to get a package
> > with dependencies on packages I haven't seen yet.  Designing the
> > graph to create placeholders let me skip making two passes over
> > the package list, pre-creating the package objects, etc etc.  This
> > made the code simpler and presumably faster.)
> >
> > The graph API maintains an externally-visible "ready" iterable of
> > nodes.  And since B can go from ready to not-ready, it can get added
> > to "ready" and subsequently removed.
> >
> > Also, when a node is satisfied, you simply remove it from the graph.
> >  If A depends on B and C, and they all represent jobs, and B and C
> > have no dependencies, they'll be in the "ready" iterable.  You iterate
> > over "ready", and execute those jobs, and assuming they are
> > successful you then remove them from the graph.  When A's
> > dependencies are all satisfied, it'll get added to the "ready" iterable.
> >  And naturally when B and C were removed from the graph, they were
> > removed from the "ready" iterable too.
>
> OK!  You're doing a topological sort.  There are natural & simple ways
> to do that right now that scale efficiently to very large graphs (time
> & space linear in the sum of the number of nodes and the number of
> dependencies).  Curiously, the difficulties are mostly driven by the
> quality of _error_ messages you want (in case of, e.g., cycles in the
> dependency graph).
>
> Lots of those implementations became deterministic "by magic" when
> ordered dicts were introduced.  This is why:  a bare bones
> implementation needs to associate two pieces of info with each node:
> a list of its immediate successors, and an integer count of the number
> of its immediate predecessors.  A dict is the natural way to implement
> that association.
>
> When all the dependency info has been entered, then:
>
> - First all nodes are emitted whose predecessor count is 0.  Iterating
> over the association dict once is the natural way to find them, and
> that order is defined now.
>
> - Then, as each emitted node N is marked done:
>   for child in N.successors:
>   assert child.predcount > 0
>   child.predcount -= 1
>   if child.predcount == 0:
>   emit(child)
>
> That's about all there is to it.  But there's no end to the cruft that
> _can_ be added to, e.g., verify that a node is marked done no more
> than once, or to compute & display strongly connected components if
> not all nodes are emitted, etc.
>
> Ordered sets could improve this "natural" implementation if the
> successor lists were successor sets instead, simply because then -
> like lists - their iteration order would be defined, which can in turn
> affect the order in which nodes are emitted in the loop just above.
> But lists are adequate to the task, are cheaper in both time and
> space, and can't _not_ be ordered ;-)
>
>
> > Thus it's this "ready" collection that 

[Python-Dev] Re: Should set objects maintain insertion order too?

[Wes Turner]

A table with these columns might be helpful for performance comparisons: [
structure,
operation,
big_o,
keyset,  # signed ints, randstrs
timeit_output,
system_load,
architecture,  # x86_84, aarch64
os,
]

Does anyone have a recommendation for a tool that stores such benchmark
data in JSON and merges the e.g. per-arch output?

Are there URIs for Big-O notation that could be added as e.g. structured
attributes in docstrings or annotations?

I just added a ._repr_html_() method to the tabulate package so that, in
addition to the many excellent ASCII formats tabulate prints,
tabulate.tabulate(data, tablefmt='html') displays an html-escaped table in
Jupyter notebooks? It's not yet released, so you'd need to
 `pip install -e git+
https://github.com/astanin/python-tabulate@master#egg=tabulate`

On Tue, Dec 24, 2019, 3:53 AM Larry Hastings  wrote:

>
> On 12/23/19 8:09 PM, Kyle Stanley wrote:
>
>
> Chris Angelico wrote:
> > I think this is an artifact of Python not having an empty set literal.
> > [snip]
> > When both use the constructor call or both use a literal, the
> > difference is far smaller. I'd call this one a wash.
>
> Ah, good catch. I hadn't considered that it would make a substantial
> difference, but that makes sense. Here's an additional comparison between
> "{}"  and "dict()" to confirm it:
>
> >>> timeit.timeit("{}", number=100_000_000)
> 2.1038335599987477
> >>> timeit.timeit("dict()", number=100_000_000)
> 10.22558353268
>
>
> We could also go the other way with it, set / dict-map-to-dontcare with
> one element, because that way we *can* have a set literal:
>
> >>> timeit.timeit("set()", number=100_000_000)
> 8.579900023061782
> >>> timeit.timeit("dict()", number=100_000_000)
> 10.473437276901677
> >>> timeit.timeit("{0}", number=100_000_000)
> 5.969995185965672
> >>> timeit.timeit("{0:0}", number=100_000_000)
> 6.24465325800702
>
> (ran all four of those just so you see a relative sense on my laptop,
> which I guess is only slghtly slower than Kyle's)
>
>
> Happy holidays,
>
>
> */arry*
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/MQAKLPKWECT22NVPRITAD5XQBIFUS4UA/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6OWPRNUXCK776G5H53RAP3OPHBNMK6D7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Should set objects maintain insertion order too?

[Wes Turner]

Got it. Thanks

On Fri, Dec 20, 2019, 2:34 AM Inada Naoki  wrote:

> On Fri, Dec 20, 2019 at 4:15 PM Wes Turner  wrote:
> >
> > How slow and space-inefficient would it be to just implement the set
> methods on top of dict?
>
> Speed:  Dict doesn't cache the position of the first item.  Calling
> next(iter(D)) repeatedly is O(N) in worst case.
> Space:  It waste 8bytes per member.
>
> >
> > Do dicts lose insertion order when a key is deleted? AFAIU, OrderedDict
> do not lose insertion order on delete.
>
> Dict keeps insertion order after deletion too.
>
> >Would this limit the utility of an ordered set as a queue? What set
> methods does a queue need to have?
>
> I want O(1) D.popleft(). (The name is borrowed from deque.  popfirst()
> would be better maybe).
>
> --
> Inada Naoki  
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/UHNHQGJL7J4UUL44F57KYPQMO4NQRZPO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Should set objects maintain insertion order too?

[Wes Turner]

How slow and space-inefficient would it be to just implement the set
methods on top of dict?

Do dicts lose insertion order when a key is deleted? AFAIU, OrderedDict do
not lose insertion order on delete. Would this limit the utility of an
ordered set as a queue? What set methods does a queue need to have?

On Thu, Dec 19, 2019, 11:41 PM Tim Peters  wrote:

> [Nick]
> > I must admit that I was assuming without stating that a full OrderedSet
> > implementation would support the MutableSequence interface.
>
> Efficient access via index position too would be an enormous new
> requirement,  My bet:  basic operations would need to change from O(1)
> to O(log(N)).
>
> BTW, in previous msgs there are links to various implementations
> calling themselves "ordered sets".  One of them supplies O(1)
> indexing, but at the expense of making deletion O(N) (!):
>
> https://pypi.org/project/ordered-set/
>
> If efficient indexing is really wanted, then the original "use case"
> Larry gave was definitely obscuring an XY problem ;-)
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/IBRSGUTHIMOZ6JGIYJBQJFXEANFZI4V5/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PK4C2OGTB2HZ4E7OYU6B6W7A7H4LPX7W/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Should set objects maintain insertion order too?

[Wes Turner]

OrderedSet implementations:

- https://github.com/methane/cpython/pull/23/files

- https://pypi.org/search/?q=orderedset
  - https://pypi.org/project/orderedset/
- https://code.activestate.com/recipes/576694/
- https://pypi.org/project/ordered-set/
- 
https://github.com/pandas-dev/pandas/blob/master/pandas/core/indexes/base.py#L172
(pandas' Index types)
- https://pypi.org/project/sortedcollections/

[Ordered] Sets and some applications:

- https://en.wikipedia.org/wiki/Set_(mathematics)
  - https://en.wikipedia.org/wiki/Set_notation
- https://en.wikipedia.org/wiki/Set-builder_notation
  - https://en.wikipedia.org/wiki/Glossary_of_set_theory
- https://en.wikipedia.org/wiki/Set_(abstract_data_type)
- Comparators
  - https://en.wikipedia.org/wiki/Total_order
  - https://en.wikipedia.org/wiki/Partially_ordered_set
- https://en.wikipedia.org/wiki/Causal_sets

On Thu, Dec 19, 2019 at 12:05 PM Tim Peters  wrote:
>
> [Nick Coghlan ]
> > Starting with "collections.OrderedSet" seems like a reasonable idea,
> > though - that way "like a built-in set, but insertion order preserving" will
> > have an obvious and readily available answer, and it should also
> > make performance comparisons easier.
>
> Ya, I suggested starting with collections.OrderedSet earlier, but gave up on 
> it.
>
> The problem is that the "use case" here isn't really a matter of
> functionality, but of consistency:  "it would be nice if", like dicts
> enjoy now, the iteration order of sets was defined in an
> implementation-independent way.  That itch isn't scratched unless the
> builtin set type defines it.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/PM5ENMLR665XG32AS2FEAEUVDG3AFWV6/
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XPFWTEAM7QUPJMP7R7E7LG23FAC2WXYW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Mixed Python/C debugging

[Wes Turner]

Your question is specifically about IDEs with support for mixed-mode
debugging (with gdb), so I went looking for an answer:

https://wiki.python.org/moin/DebuggingWithGdb (which is not responsive and
almost unreadable on a mobile device) links to
https://fedoraproject.org/wiki/Features/EasierPythonDebugging , which
mentions the py-list, py-up and py-down, py-bt, py-print, and py-locals GDB
commands that are also described in ** https://devguide.python.org/gdb/ **


https://wiki.python.org/moin/PythonDebuggingTools Ctrl-F "gdb" mentions:
DDD, pyclewn (vim), trepan3k (which is gdb-like and supports breaking at
c-line and also handles bytecode disassembly)

Apparently, GHIDRA does not have a debugger but there is a plugin for
following along with gdb in ghidra called
https://github.com/Comsecuris/gdbghidra , which may or may not be useful.

https://github.com/Mistobaan/pyclewn hasn't been updated in years, but may
have useful bits for implementing mixed-mode debugging in other non-vim
IDEs.

https://reverseengineering.stackexchange.com/questions/1392/decent-gui-for-gdb
lists a number of GUIs for GDB; including voltronnn:

> There's Voltron, which is an extensible Python debugger UI that supports
LLDB, GDB, VDB, and WinDbg/CDB (via PyKD) and runs on macOS, Linux and
Windows. For the first three it supports x86, x86_64, and arm with even
arm64 support for lldb while adding even powerpc support for gdb.
https://github.com/snare/voltron

https://developers.redhat.com/blog/2017/11/10/gdb-python-api/ describes the
GDB Python API.

https://pythonextensionpatterns.readthedocs.io/en/latest/debugging/debug_in_ide.html#writing-a-c-function-to-call-any-python-unit-test
may be helpful.

Does DDD support mixed-mode debugging?
https://www.gnu.org/software/ddd/manual/html_mono/ddd.html

Essentially, for IDE support, AFAIU, the basic functionality is:
- set breakpoints: `b c-file.c:123`
- step through them while seeking-to and highlighting the current breakpoint
- provide one or more panes for executing GDB commands within the current
or other frames


The GDB Python API docs:
https://sourceware.org/gdb/onlinedocs/gdb/Python-API.html

The devguide gdb page may be the place to list IDEs with support for
mixed-mode debugging of Python and C/C++/Cython specifically with gdb?

On Sunday, December 1, 2019, Skip Montanaro 
wrote:
> Having tried comp.lang.python with no response, I turn here...
>
> After at least ten years away from Python's run-time interpreter &
> byte code compiler, I'm getting set to familiarize myself with that
> again. This will, I think, entail debugging a mixed Python/C
> environment. I'm an Emacs user and am aware that GDB since 7.0 has
> support for debugging at the Python code level. Is Emacs+GDB my best
> bet? Are there any Python IDEs which support C-level breakpoints and
> debugging?
>
> Thanks,
>
> Skip
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/L2KBZM64MYPXIITN4UU3X6L4PZS2YRTB/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/Z3S2RAXRIHAWT6JEOXEBPPBTPUTMDZI7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Steering Council Update (July through October)

[Wes Turner]

Thanks for these minutes and for cc'ing the list with them! :+1: :clap:
(markdown emoji)

I know that I can:
- click 'Watch' on the GitHub repo [1] to get notifications when there are
changes
- get an 'Activity Summary' digest email from  discuss.python.org
(Discourse) when I haven't logged in lately
- 'watch' or 'track' specific threads hosted by discuss.python.org
(Discourse) and get notifications

The README with links to the steering council updates is sufficient; but
would it be helpful to create a Jekyll theme (or put the logo on a minimal
theme) and setup GH pages for these? [2] There would then also be an RSS
feed and a reverse chronological blog view.

[1] https://github.com/python/steering-council

[2]
https://help.github.com/en/github/working-with-github-pages/setting-up-a-github-pages-site-with-jekyll

On Saturday, November 9, 2019, Ewa Jodlowska  wrote:
> Hey folks -
> Here is an update from the Steering Council. It covers July through
October.
> The update has been added to the Steering Council repo as well:
https://github.com/python/steering-council/blob/master/updates/2019-11-09-steering-council-update.md
.
> # Steering Council Community Update
> July 2019 through October 2019
>
>
> ## July recap:
> - PSF and the Steering Council worked on a proposal to hire a project
manager to help with Python 2 EOL communication. The proposal was approved
by the PSF Board and Changeset Consulting was contracted to help with this
work. Progress is being tracked [here](
https://github.com/orgs/python/projects/1#card-26091749)
> - Release cadence discussions happened.
> - [PEP 1](https://www.python.org/dev/peps/pep-0001/)’s copyright/license
section was updated to include a dual license of public domain and
CC0-1.0-Universal.
>
>
> ## August recap:
> - [PEP 572](https://www.python.org/dev/peps/pep-0572/) was updated to
replace `TargetScopeError` to `SyntaxError`:
https://github.com/python/peps/commit/cd292d89972476fa485a8ac1b14c1ed85b8c4867
> - Vision document review and update happened.
> - Release cadence discussions happened.
> - Steering Council helped answer inquiries about proposed Python 2 EOL
page.
> - Steering Council reviewed the newly approved PSF Code of Conduct.
>
>
> ## September recap:
> - Slide deck “Vision Doc” was created based off of the original Vision
Document and was discussed by the Steering Council. The Steering Council
decided to continue in this direction.
> - The Steering Council discussed the Core Dev promotion topic and it was
suggested by the Steering Council that the Core Devs create a Work Group to
work on that:
https://discuss.python.org/t/concerns-about-how-we-vote-on-new-core-developers/2351/19
> - A timeline was selected for the Steering Council election and informed
the Core Devs:
https://mail.python.org/archives/list/python-committ...@python.org/thread/PLDUMAJOLHM5W6FWOMSGUBN2Q7Z6D3JM/
> - Release cadence discussions happened
> - [Core Dev Sprint in London recap](
http://pyfound.blogspot.com/2019/10/cpython-core-developer-sprint-2019.html)
>
>
> ## October recap:
> - Steering Council discussed the new Code of Conduct and agreed that it
should be sent out to the python-committers@ and python-dev@ mailing lists:
https://mail.python.org/archives/list/python-committ...@python.org/thread/6QIMLZ65D2HVBRGXRMOF2KOLFRXH4IRG/
> - Steering Council plans on presenting the Vision Deck at PyCon pending
any changes made by future Steering Council.
> - Steering Council discussed the proposal made by [Thomas Wouters](
https://mail.python.org/archives/list/python-committ...@python.org/thread/UWOU47BVULIOZHEOFWGVVQG6LPEXTDNG/)
for approval voting. The Steering Council guided Thomas in the direction of
a [PR](https://github.com/python/peps/pull/1197) and a formal vote. That
resulted in a change to [PEP 13](https://www.python.org/dev/peps/pep-0013/)
stating core team members can vote for zero or more of the candidates for
the Steering Council.
> - Steering Council discussed communal ownership of Python repo. It was
decided that the Steering Council will send an email to python-committers
about the topic. Brett sent the email on Nov 6:
https://mail.python.org/archives/list/python-committ...@python.org/message/DCKAPZVXEQHP7OPS3GLEVNDXMXX5QAE6/
.
> - Steering Council met with GitHub team about the b.p.o. migration, PEP
[581](
https://www.python.org/dev/peps/pep-0581/)/[588](https://www.python.org/dev/peps/pep-0588/)
plan and discussed next steps. Next steps for Python include finding a
point person to lead the work (will be a project manager), the Steering
Council to find out more information about search features and how mapping
will work.
> - The SC reviewed and discussed PEP 602 (1 year release cycle proposal by
Łukasz Langa) and PEP 605 (Nick Coghlan and Steve Dower's 2 year proposal)
and selected [PEP 602](https://www.python.org/dev/peps/pep-0602/).
> - [PEP 484](https://www.python.org/dev/peps/pep-0484/) policy for
inclusion of stubs in typeshed was discussed and the [PR](

[Python-Dev] Re: Inline links in Misc/NEWS entries

[Wes Turner]

- [ ] A page or section in the Sphinx docs would be helpful for many, I
think

https://github.com/sphinx-doc/sphinx/tree/master/doc

``__


.. index:: Implicit reference
.. _implicit-reference:

Implicit ref
---

`implicit ref`_

`anchor text `_

`implicit-reference`_

`implicit reference`_

:func:`modname.funcname`

:meth:`modname.Classname.methname`

:class:`modname.Classname`

And, FWIW:

:cite:`cpython37`

.. bibliography:: references.bib

References.bib
--
.. code:: latex

@techreport{cpython37,
title="Python 3.7 Documentation",
author="Python Software Foundation",
year=1999,
howpublished = "\url{https://docs.python.org/3.7/};,
}



- [ ] And/Or examples on the roles docs page
https://www.sphinx-doc.org/en/master/usage/restructuredtext/roles.html#ref-role

- [ ] And a *link*/"reference" from the "extensive cross-references"
paragraph of https://www.sphinx-doc.org/en/master/ to the cross-referencing
docs

- [ ] And how to do parenthetical citations of / reference CPython with
e.g. bibtex [with the `.. bibliography::` directive and :cite:`refkey` role
from sphinxcontrib-bibtex]

There are lots of great Sphinx primers; awesome-sphinxdoc could link to
them to help everyone:
https://github.com/yoloseem/awesome-sphinxdoc

On Monday, August 19, 2019,  wrote:

> Citing from the current (19 Aug 2019) version of this PR (
> https://github.com/python/devguide/pull/525/files#diff-
> 50cb76bbe8ae3fcd4170dc6e8d9d6b3fR225-R226):
>
> > Before using any Sphinx roles, ensure that a corresponding entry exists
> within the documentation.
>
> At the risk of crass self-promotion, the talk I gave last month at PyOhio (
> https://www.pyohio.org/2019/presentations/137) provides step-by-step
> instructions for how to find the information needed to craft a working
> Sphinx cross-reference, along with some of the tooling that's available.
>
> Right now I have the slides up as a view-only share in my Google Drive (
> bit.ly/bskinn-pyohio2019-intersphinx). Perhaps it would be useful to link
> to them? Or, to host a PDF someplace more permanent, and link to that?
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at https://mail.python.org/archives/list/python-dev@
> python.org/message/IK5XP7CPZDF2FSFG55JRKXHEXNTOMUTB/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SSMNWK7YF6WRT2NO4C2IQ6PGD2XVJOZT/

[Python-Dev] Re: Inline links in Misc/NEWS entries

[Wes Turner]

Could a bot be written to suggest upgraded markup for Misc/NEWS entries as
a PR?

Most e.g. class, method, and function references probably need ReST markup
AND a more specific fully-qualified reference?

Can class, method, and function lookups be done with the existing Sphinx
API index?

On Tuesday, August 13, 2019, Ned Deily  wrote:

> On Aug 13, 2019, at 15:31, Kyle Stanley  wrote:
> > > In general, using many Python-specific Sphinx markup entities is fine:
> browsing though the consolidated blurb files for previous releases (and the
> resultant changelog html) shows uses of entities like :func: and :class:.
> >
> > Since it's okay to use Python-specific Sphinx, should we encourage the
> usage of it when appropriate, such as when the links could provide helpful
> information to users?
> >
> > The primary purpose of me creating this topic was because there seems to
> be some sentiment that it's perfectly fine to exclusively use plaintext in
> the news entries. Especially in cases where authors have rejected
> suggestions to adding the Sphinx markup in their PRs. There seems to be
> some sentiment that it's perfectly fine to exclusively use plaintext in
> every news entry. Personally, I think it's a bit more nuanced, and that the
> links can sometimes be very helpful for readers.
>
> The ability to effectively use Python-specific Sphinx features in NEWS
> entries is a relatively new feature so I think that is the primary reason
> it is not encouraged more: many people don't realize you can now do this.
> Back in the day, NEWS files were just treated as plaintext, not reST.  This
> is still the case for Python 2.7 NEWS entries as the 2.7 docset was never
> modified to produce an HTML changelog as the Python 3.x docsets do.  (Of
> course, that will soon be a non-issue when 2.7 reaches end-of-life status
> in 2020.)
>
> > Also, a related question: would it be helpful for contributors to look
> through news entries for the latest stable and beta releases, and add
> inline links where they could be useful for readers?
>
> We haven't done a lot of that in the past but I don't see a reason not to,
> especially since it is easy on most systems to generate the changelog by
> building the html docs in the source tree:
>
> https://devguide.python.org/documenting/#using-make-make-bat
>
> and then opening the resultant docs in your browser as a file (the html
> docs are self-contained with regard to static files and do not need a
> webserver).
>
> But other people may have other opinions on the matter.  We could
> undoubtedly spend a lot of time tidying up old NEWS entries and, at some
> point, that time might be better spent on other things, like helping with
> the "What's New" documents for upcoming releases or just fixing bugs.  But
> I think there could be a lot of benefit for a moderate bit of touching up.
>
> One important note: to avoid confusion, only edit the blurb NEWS rst files
> in the branch for that release, i.e. edit Misc/NEWS.d/3.8.0b1.rst in the
> 3.8 branch, not in the master branch.
>
> --
>   Ned Deily
>   n...@python.org -- []
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at https://mail.python.org/archives/list/python-dev@
> python.org/message/WQNYODDNTH6MFPBNJZZJR6AFFTD5CYBG/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/QYXQ5UD34L2AQTQSUHJSEOYEBP4Q3HEB/

[Python-Dev] Re: Sponsoring Python development via the PSF

[Wes Turner]

On Thu, Jun 27, 2019 at 2:17 PM Hobson Lane  wrote:

> I wholeheartedly support accepting Bitcoin and Etherium and other popular
> cryptos at PSF. Bitpay is an alternative, but they can be more complicated
> to work with than Coinbase. Of course you can always chose to manage your
> own crypto wallet independent of any go-between business, but that has
> risks too.
>

Here's a possible spec / criteria for considering a solution for
cryptocurrency donations :

- a webapp (over TLS) to  collect personal information (that should not be
stored in a public blockchain for GDPR-compliance) and generate receipts
witha template suitable for tax deduction (in the US, and [...])
- FDIC deposit insurance protection for any funds held in USD in the
exchange*
- regulatory approval to operate within the given territories
- no-fee payment processing for nonprofits

\* For donations immediately cashed out to a bank account over direct
deposit (ACH (SFTP+PGP)) at the non-stable present value, you only need to
hold funds in USD while deposits to the configured bank account are pending


> Coinbase would be my last choice, however. It has been involved in some
> shady business practices that would be illegal if they were regulated as
> strictly as a bank or brokerage is.
> - From the very beginning Coinbase has been accused of insider trading,
> front-trading, market manipulation, and outright fraud in class action
> lawsuits such as this one:
> https://www.silvermillerlaw.com/current-investigations/coinbase-class-action/
> - In times of dips or spikes in price they sometimes prevent some
> depositors from accessing their funds for extended periods of time (I've
> been unable to access my account for years now, and others I know have had
> similar difficulties).
> - They profit on bid/ask spreads, like most brokerages or financial
> markets, but they aren't regulated as strictly as traditional
> brokerages+banks so that spread can be quite broad and the invisible loss
> of value for frequent depositors/payors can be significant.
>
> --Hobson
>
>
>
> On Thu, Jun 27, 2019 at 10:40 AM Wes Turner  wrote:
>
>>
>>
>> On Thursday, June 27, 2019, Ewa Jodlowska  wrote:
>>
>>>
>>>
>>> On Thu, Jun 27, 2019 at 11:16 AM Wes Turner 
>>> wrote:
>>>
>>>>
>>>> From GuideStar (which, like Charity Navigator, also has nonprofit
>>>> evaluation criteria)
>>>>
>>>> https://trust.guidestar.org/bitcoin-what-nonprofits-need-to-know-and-how-it-might-give-your-fundraising-a-competitive-advantage
>>>> :
>>>>
>>>> > Accepting donations in Bitcoin can reduce the financial transaction
>>>> fees a nonprofit owes.  With Bitcoin, the costs are significantly lower
>>>> compared to checks, credit cards, and other digital options, and if your
>>>> organization is a registered 501(c)(3), there are zero transaction fees on
>>>> platforms such as Coinbase or Bitpay.  There is also no risk of bank
>>>> charges accruing to the nonprofit in the case of a donor using a fraudulent
>>>> credit card.
>>>>
>>>> * Merch: It may be worth accepting cryptocurrency payments for PSF
>>>> merch like T-Shirts, Mugs, Polo shirts:
>>>> https://www.python.org/community/merchandise/
>>>>
>>>>   * Coinbase Commerce integrates with a number of major eCommerce
>>>> platforms:
>>>> https://commerce.coinbase.com/integrate
>>>>
>>>>
>>>> I'll share Coinbase with our team for review.
>>>
>>>
>>>> * What mailing list and/or CRM can I opt-into with my PSF donation? Is
>>>> there a periodic breakdown of expenses that I can backseat-drive and
>>>> micromanage?
>>>>
>>>
>>> We go over our finances at the PSF member meetings at PyCon and the
>>> EuroPython. In addition to what we have on python.org (
>>> https://www.python.org/psf/annual-report/2019/,
>>> https://www.python.org/psf/records/board/treasurer/), financials are
>>> also sent to voting members (psf-v...@python.org).
>>>
>> The 2018 financials will be sent there this month.
>>>
>>
>> Thanks!
>>
>>
>>>  If anyone wants to become a voting member of the PSF, let me know
>>> off-list and I'll be happy to share the available options!
>>>
>>
>>
>>>
>>>> From Charity Navigator, it looks like PSF had about $ 2,500,000 in
>>>> donation revenue a few years ago.
>>>>
>>>
>>> As of end of 2018, the PSF had almost 3.3M in 

[Python-Dev] Re: Sponsoring Python development via the PSF

[Wes Turner]

"PSF Prospectus 2018-2019.pdf"
https://www.dropbox.com/s/a479lb6jz4yhr4x/PSF%20Prospectus%202018-2019.pdf

On Thu, Jun 27, 2019 at 2:12 PM Wes Turner  wrote:

>
>
> On Thu, Jun 27, 2019 at 1:27 PM Wes Turner  wrote:
>
>> Is there a way for third party organizations to say, "yeah, we sponsored
>> this or that".
>>
>> When I go to the donation page, can I see all of the ways to contribute
>> time, trained resources, and money? Maybe I'm looking for a shirt, maybe
>> I'm looking to send in our best guy to let's get it done here
>>
>
> Say I'm a person who's looking to take some time off and do a tour with
> PSF.
> To donate some talent to the Python Community.
>
> I missed HR (Hiring, Contracts, Benefits) on the list above.
> That may be because I, too, like to do coding most of the time, too.
>
> Buy say I'm a person who's looking to take some time off
> and do a tour with PSF;
> sort of like 18F, Defense Digital Service, US Digital Services (USDS);
> with talent from industry contributing their time
> in more of a startup / intrapreneurial environment.
>
> Pitch me. What do I do for healthcare,
> with my retirement savings 401(k) and/or IRA Independent Retirement
> Account?
> Do I even need to roll anything over to
> come help out full time for a few years?
>
> I can read the Contributing section of the Devguide,
> https://devguide.python.org/#contributing
>
> And learn about the systems that power the Python community
> software development workflows (including the cool git repo bots),
> but when I consider donating money,
> how can we upsell me to considering doing a tour with PSF?
>
> Maybe I want to:
>
> - buy a shirt, a hat,
> - send a cash money donation (and get my tax receipt)
>   https://www.python.org/psf/donations/
> - donate corporate resources
>   https://www.python.org/psf/sponsorship/ #sponsorship-levels
> - pay developers with healthcare and give them some time
> - decide when in my career would be a good time
>   to consider PSF Membership and Fellowship
>
> ...
>
> There are a number of online payroll, contracts, and benefits management
> solutions. FounderKit has reviews of self-service HR solutions:
> https://founderkit.com/biz/gusto
>
>
>
>>
>> On Thursday, June 27, 2019, Wes Turner  wrote:
>>
>>> Indeed,
>>> When I donate to PSF, how does that work, what do I sponsoring?
>>>
>>> AFAIU:
>>>
>>> - Payment processing
>>> - Prioritization and allocation
>>> - Project Management
>>>   with plans as RST documents on GitHub,
>>>   issues on Roundup (bug and feature triage)
>>>   discussions on mailing lists,
>>>   discussions on discourse
>>> - Software Development
>>>   GitHub pull requests
>>> - Release management
>>> - More Fundraising & Marketing & PyCon
>>>
>>> *We could* sponsor a part or full-time developer as an employee of our
>>> organization, or donate to PSF to sponsor sprints and/or developers
>>>
>>> On Thursday, June 27, 2019, Wes Turner  wrote:
>>>
>>>>
>>>>
>>>> On Thursday, June 27, 2019, Ewa Jodlowska  wrote:
>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Jun 26, 2019 at 8:28 PM Wes Turner 
>>>>> wrote:
>>>>>
>>>>>> When companies donate as PyCon sponsors (and get brand recognition)
>>>>>> do those donations also go to PSF?
>>>>>
>>>>>
>>>>> Yes, the PSF produces PyCon. PyCon sponsorships are used to help
>>>>> offset PyCon expenses. If PyCon has a surplus, it is the PSF's revenue.
>>>>> PyCon's surplus is the biggest source of revenue for the PSF.
>>>>>
>>>>>>
>>>>>>
>>>>>> You can add the **Python Software Foundation** to a Charity Navigator
>>>>>> "Giving Basket" and get one receipt:
>>>>>>
>>>>>> https://www.charitynavigator.org/index.cfm?bay=search.profile=043594598
>>>>>>
>>>>>>
>>>>>
>>>>>> "Charity Navigator's Methodology"
>>>>>>
>>>>>> https://www.charitynavigator.org/index.cfm?bay=content.view=5593#rating
>>>>>>
>>>>>>
>>>>>> Is there any way to donate cryptocurrency to PSF yet?
>>>>>>
>>>>>
>>>>> Currently, we do not.
>>>>>
>>>&g

[Python-Dev] Re: Sponsoring Python development via the PSF

[Wes Turner]

On Thu, Jun 27, 2019 at 1:27 PM Wes Turner  wrote:

> Is there a way for third party organizations to say, "yeah, we sponsored
> this or that".
>
> When I go to the donation page, can I see all of the ways to contribute
> time, trained resources, and money? Maybe I'm looking for a shirt, maybe
> I'm looking to send in our best guy to let's get it done here
>

Say I'm a person who's looking to take some time off and do a tour with PSF.
To donate some talent to the Python Community.

I missed HR (Hiring, Contracts, Benefits) on the list above.
That may be because I, too, like to do coding most of the time, too.

Buy say I'm a person who's looking to take some time off
and do a tour with PSF;
sort of like 18F, Defense Digital Service, US Digital Services (USDS);
with talent from industry contributing their time
in more of a startup / intrapreneurial environment.

Pitch me. What do I do for healthcare,
with my retirement savings 401(k) and/or IRA Independent Retirement Account?
Do I even need to roll anything over to
come help out full time for a few years?

I can read the Contributing section of the Devguide,
https://devguide.python.org/#contributing

And learn about the systems that power the Python community
software development workflows (including the cool git repo bots),
but when I consider donating money,
how can we upsell me to considering doing a tour with PSF?

Maybe I want to:

- buy a shirt, a hat,
- send a cash money donation (and get my tax receipt)
  https://www.python.org/psf/donations/
- donate corporate resources
  https://www.python.org/psf/sponsorship/ #sponsorship-levels
- pay developers with healthcare and give them some time
- decide when in my career would be a good time
  to consider PSF Membership and Fellowship

...

There are a number of online payroll, contracts, and benefits management
solutions. FounderKit has reviews of self-service HR solutions:
https://founderkit.com/biz/gusto



>
> On Thursday, June 27, 2019, Wes Turner  wrote:
>
>> Indeed,
>> When I donate to PSF, how does that work, what do I sponsoring?
>>
>> AFAIU:
>>
>> - Payment processing
>> - Prioritization and allocation
>> - Project Management
>>   with plans as RST documents on GitHub,
>>   issues on Roundup (bug and feature triage)
>>   discussions on mailing lists,
>>   discussions on discourse
>> - Software Development
>>   GitHub pull requests
>> - Release management
>> - More Fundraising & Marketing & PyCon
>>
>> *We could* sponsor a part or full-time developer as an employee of our
>> organization, or donate to PSF to sponsor sprints and/or developers
>>
>> On Thursday, June 27, 2019, Wes Turner  wrote:
>>
>>>
>>>
>>> On Thursday, June 27, 2019, Ewa Jodlowska  wrote:
>>>
>>>>
>>>>
>>>>
>>>> On Wed, Jun 26, 2019 at 8:28 PM Wes Turner 
>>>> wrote:
>>>>
>>>>> When companies donate as PyCon sponsors (and get brand recognition) do
>>>>> those donations also go to PSF?
>>>>
>>>>
>>>> Yes, the PSF produces PyCon. PyCon sponsorships are used to help offset
>>>> PyCon expenses. If PyCon has a surplus, it is the PSF's revenue. PyCon's
>>>> surplus is the biggest source of revenue for the PSF.
>>>>
>>>>>
>>>>>
>>>>> You can add the **Python Software Foundation** to a Charity Navigator
>>>>> "Giving Basket" and get one receipt:
>>>>>
>>>>> https://www.charitynavigator.org/index.cfm?bay=search.profile=043594598
>>>>>
>>>>>
>>>>
>>>>> "Charity Navigator's Methodology"
>>>>>
>>>>> https://www.charitynavigator.org/index.cfm?bay=content.view=5593#rating
>>>>>
>>>>>
>>>>> Is there any way to donate cryptocurrency to PSF yet?
>>>>>
>>>>
>>>> Currently, we do not.
>>>>
>>>>>
>>>>> All of these orgs accept BTC donations:
>>>>>
>>>>> https://en.bitcoin.it/wiki/Donation-accepting_organizations_and_projects
>>>>>
>>>>> Just sending to an address doesn't get you a receipt for tax purposes;
>>>>> there are firms that support donations by exchanging for USD at that time
>>>>> and depositing to the configured account.
>>>>>
>>>>
>>>> That's good to know! If you know more info on this, please let me know.
>>>> I can share it with our accounting team to review.
>>>>
>&

[Python-Dev] Re: Sponsoring Python development via the PSF

[Wes Turner]

"Earmarked" funding is an additional funding model for the Python ecosystem
that PSF is a big part of.

For example, the new 2FA and limited use package upload token support for
PyPI (PyPA/warehouse) is sponsored by a specific grant.

Is there a way for third party organizations to say, "yeah, we sponsored
this or that".

When I go to the donation page, can I see all of the ways to contribute
time, trained resources, and money? Maybe I'm looking for a shirt, maybe
I'm looking to send in our best guy to let's get it done here

On Thursday, June 27, 2019, Wes Turner  wrote:

> Indeed,
> When I donate to PSF, how does that work, what do I sponsoring?
>
> AFAIU:
>
> - Payment processing
> - Prioritization and allocation
> - Project Management
>   with plans as RST documents on GitHub,
>   issues on Roundup (bug and feature triage)
>   discussions on mailing lists,
>   discussions on discourse
> - Software Development
>   GitHub pull requests
> - Release management
> - More Fundraising & Marketing & PyCon
>
> *We could* sponsor a part or full-time developer as an employee of our
> organization, or donate to PSF to sponsor sprints and/or developers
>
> On Thursday, June 27, 2019, Wes Turner  wrote:
>
>>
>>
>> On Thursday, June 27, 2019, Ewa Jodlowska  wrote:
>>
>>>
>>>
>>>
>>> On Wed, Jun 26, 2019 at 8:28 PM Wes Turner  wrote:
>>>
>>>> When companies donate as PyCon sponsors (and get brand recognition) do
>>>> those donations also go to PSF?
>>>
>>>
>>> Yes, the PSF produces PyCon. PyCon sponsorships are used to help offset
>>> PyCon expenses. If PyCon has a surplus, it is the PSF's revenue. PyCon's
>>> surplus is the biggest source of revenue for the PSF.
>>>
>>>>
>>>>
>>>> You can add the **Python Software Foundation** to a Charity Navigator
>>>> "Giving Basket" and get one receipt:
>>>> https://www.charitynavigator.org/index.cfm?bay=search.profil
>>>> e=043594598
>>>>
>>>
>>>> "Charity Navigator's Methodology"
>>>> https://www.charitynavigator.org/index.cfm?bay=content.view;
>>>> cpid=5593#rating
>>>>
>>>>
>>>> Is there any way to donate cryptocurrency to PSF yet?
>>>>
>>>
>>> Currently, we do not.
>>>
>>>>
>>>> All of these orgs accept BTC donations:
>>>> https://en.bitcoin.it/wiki/Donation-accepting_organizations_
>>>> and_projects
>>>>
>>>> Just sending to an address doesn't get you a receipt for tax purposes;
>>>> there are firms that support donations by exchanging for USD at that time
>>>> and depositing to the configured account.
>>>>
>>>
>>> That's good to know! If you know more info on this, please let me know.
>>> I can share it with our accounting team to review.
>>>
>>
>> Re: PSF accepting cryptoasset donations
>>
>> Wikimedia, Save The Children, UNICEF, United Way, Mozilla, EFF, and FSF,
>> for example, all accept cryptocurrency donations.
>>
>> OpenCollective reviews a number of possible payment solutions:
>> https://github.com/opencollective/opencollective/
>> issues/919#issuecomment-370218843 (now with a GitHub poll in the issue
>> description)
>>
>> From GuideStar (which, like Charity Navigator, also has nonprofit
>> evaluation criteria)
>> https://trust.guidestar.org/bitcoin-what-nonprofits-need-to-
>> know-and-how-it-might-give-your-fundraising-a-competitive-advantage :
>>
>> > Accepting donations in Bitcoin can reduce the financial transaction
>> fees a nonprofit owes.  With Bitcoin, the costs are significantly lower
>> compared to checks, credit cards, and other digital options, and if your
>> organization is a registered 501(c)(3), there are zero transaction fees on
>> platforms such as Coinbase or Bitpay.  There is also no risk of bank
>> charges accruing to the nonprofit in the case of a donor using a fraudulent
>> credit card.
>>
>> * Merch: It may be worth accepting cryptocurrency payments for PSF merch
>> like T-Shirts, Mugs, Polo shirts: https://www.python.org/communi
>> ty/merchandise/
>>
>>   * Coinbase Commerce integrates with a number of major eCommerce
>> platforms:
>> https://commerce.coinbase.com/integrate
>>
>> * SCHEMA.ORG/Offer and schema:Product markup improves SEO; so that when
>> I search for "PSF shirt" with an voice assistant on my way to PyCon, 

[Python-Dev] Re: Sponsoring Python development via the PSF

[Wes Turner]

On Thursday, June 27, 2019, Ewa Jodlowska  wrote:

>
>
> On Thu, Jun 27, 2019 at 11:16 AM Wes Turner  wrote:
>
>>
>> From GuideStar (which, like Charity Navigator, also has nonprofit
>> evaluation criteria)
>> https://trust.guidestar.org/bitcoin-what-nonprofits-need-
>> to-know-and-how-it-might-give-your-fundraising-a-competitive-advantage :
>>
>> > Accepting donations in Bitcoin can reduce the financial transaction
>> fees a nonprofit owes.  With Bitcoin, the costs are significantly lower
>> compared to checks, credit cards, and other digital options, and if your
>> organization is a registered 501(c)(3), there are zero transaction fees on
>> platforms such as Coinbase or Bitpay.  There is also no risk of bank
>> charges accruing to the nonprofit in the case of a donor using a fraudulent
>> credit card.
>>
>> * Merch: It may be worth accepting cryptocurrency payments for PSF merch
>> like T-Shirts, Mugs, Polo shirts: https://www.python.org/
>> community/merchandise/
>>
>>   * Coinbase Commerce integrates with a number of major eCommerce
>> platforms:
>> https://commerce.coinbase.com/integrate
>>
>>
>> I'll share Coinbase with our team for review.
>
>
>> * What mailing list and/or CRM can I opt-into with my PSF donation? Is
>> there a periodic breakdown of expenses that I can backseat-drive and
>> micromanage?
>>
>
> We go over our finances at the PSF member meetings at PyCon and the
> EuroPython. In addition to what we have on python.org (
> https://www.python.org/psf/annual-report/2019/, https://
> www.python.org/psf/records/board/treasurer/), financials are also sent to
> voting members (psf-v...@python.org).
>
The 2018 financials will be sent there this month.
>

Thanks!


>  If anyone wants to become a voting member of the PSF, let me know
> off-list and I'll be happy to share the available options!
>


>
>> From Charity Navigator, it looks like PSF had about $ 2,500,000 in
>> donation revenue a few years ago.
>>
>
> As of end of 2018, the PSF had almost 3.3M in assets. In 2018, we received
> $515,000 from "Contributions, Membership Dues, and Grants" (
> https://www.python.org/psf/annual-report/2019/). In 2016, our
> "Contributions, Membership Dues, and Grants" totaled approximately $148,024
> to give you a comparison of how things are developing over time.
>

$515,000 in 2018, thanks. An infographic on the donation page might
increase conversions.


>
> The Annual Impact report also addresses the need for a financial reserve:
>
> "The PSF will continue to research diversifying revenue streams, hiring
> additional staff, and improving our fundraising efforts, which will all
> affect future financials. We would like to continue to improve the services
> we provide to the community, expand our programs, and better support
> developers. We also need to consider risk mitigating factors such as having
> diverse revenue streams instead of heavily relying on PyCon and a financial
> reserve of at least 1.5 years (in operating costs). This will help ensure
> the PSF’s viability for the long run."
>

+1
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7PHTBINPBQ7R5UT7HLTFOXDYWM2IIMSE/

[Python-Dev] Re: Sponsoring Python development via the PSF

[Wes Turner]

Indeed,
When I donate to PSF, how does that work, what do I sponsoring?

AFAIU:

- Payment processing
- Prioritization and allocation
- Project Management
  with plans as RST documents on GitHub,
  issues on Roundup (bug and feature triage)
  discussions on mailing lists,
  discussions on discourse
- Software Development
  GitHub pull requests
- Release management
- More Fundraising & Marketing & PyCon

*We could* sponsor a part or full-time developer as an employee of our
organization, or donate to PSF to sponsor sprints and/or developers

On Thursday, June 27, 2019, Wes Turner  wrote:

>
>
> On Thursday, June 27, 2019, Ewa Jodlowska  wrote:
>
>>
>>
>>
>> On Wed, Jun 26, 2019 at 8:28 PM Wes Turner  wrote:
>>
>>> When companies donate as PyCon sponsors (and get brand recognition) do
>>> those donations also go to PSF?
>>
>>
>> Yes, the PSF produces PyCon. PyCon sponsorships are used to help offset
>> PyCon expenses. If PyCon has a surplus, it is the PSF's revenue. PyCon's
>> surplus is the biggest source of revenue for the PSF.
>>
>>>
>>>
>>> You can add the **Python Software Foundation** to a Charity Navigator
>>> "Giving Basket" and get one receipt:
>>> https://www.charitynavigator.org/index.cfm?bay=search.profil
>>> e=043594598
>>>
>>
>>> "Charity Navigator's Methodology"
>>> https://www.charitynavigator.org/index.cfm?bay=content.view;
>>> cpid=5593#rating
>>>
>>>
>>> Is there any way to donate cryptocurrency to PSF yet?
>>>
>>
>> Currently, we do not.
>>
>>>
>>> All of these orgs accept BTC donations:
>>> https://en.bitcoin.it/wiki/Donation-accepting_organizations_and_projects
>>>
>>> Just sending to an address doesn't get you a receipt for tax purposes;
>>> there are firms that support donations by exchanging for USD at that time
>>> and depositing to the configured account.
>>>
>>
>> That's good to know! If you know more info on this, please let me know. I
>> can share it with our accounting team to review.
>>
>
> Re: PSF accepting cryptoasset donations
>
> Wikimedia, Save The Children, UNICEF, United Way, Mozilla, EFF, and FSF,
> for example, all accept cryptocurrency donations.
>
> OpenCollective reviews a number of possible payment solutions:
> https://github.com/opencollective/opencollective/
> issues/919#issuecomment-370218843 (now with a GitHub poll in the issue
> description)
>
> From GuideStar (which, like Charity Navigator, also has nonprofit
> evaluation criteria)
> https://trust.guidestar.org/bitcoin-what-nonprofits-need-to-
> know-and-how-it-might-give-your-fundraising-a-competitive-advantage :
>
> > Accepting donations in Bitcoin can reduce the financial transaction fees
> a nonprofit owes.  With Bitcoin, the costs are significantly lower compared
> to checks, credit cards, and other digital options, and if your
> organization is a registered 501(c)(3), there are zero transaction fees on
> platforms such as Coinbase or Bitpay.  There is also no risk of bank
> charges accruing to the nonprofit in the case of a donor using a fraudulent
> credit card.
>
> * Merch: It may be worth accepting cryptocurrency payments for PSF merch
> like T-Shirts, Mugs, Polo shirts: https://www.python.org/
> community/merchandise/
>
>   * Coinbase Commerce integrates with a number of major eCommerce
> platforms:
> https://commerce.coinbase.com/integrate
>
> * SCHEMA.ORG/Offer and schema:Product markup improves SEO; so that when I
> search for "PSF shirt" with an voice assistant on my way to PyCon, some
> portion of that revenue should go to PSF
>
> * From a conversation with Eric Holscher regarding ReadTheDocs' ethical
> ads revenue model,
>
>   * Yeti makes solid drinkware that can be customized with a logo:
> https://www.yeti.com/en_US/custom-monogram-text-logo-drinkware
>
>   * And, for accounting, Zapier automates connections between a bunch of
> APIs with e.g. QuickBooks Online; which can save accounting's time for
> things like copying merch invoices to the accounts payable, refund, and
> chargeback accounting ledgerd.
>   https://zapier.com/apps/quickbooks/integrations
>
> * Integrating E.G. Shopify with QuickBooks doesn't require any Zapier
> Zaps. Shopify also integrates with e.g. Amazon Fulfillment so that they can
> manage inventory, returns, and shipping with their warehouses and efficient
> logistics platform.:
> https://quickbooks.intuit.com/integrations/shopify/
>
> * There's a CreativeCommons-licensed logo SVG on Wikiped

[Python-Dev] Sponsoring Python development via the PSF

[Wes Turner]

On Thursday, June 27, 2019, Ewa Jodlowska  wrote:

>
>
>
> On Wed, Jun 26, 2019 at 8:28 PM Wes Turner  wrote:
>
>> When companies donate as PyCon sponsors (and get brand recognition) do
>> those donations also go to PSF?
>
>
> Yes, the PSF produces PyCon. PyCon sponsorships are used to help offset
> PyCon expenses. If PyCon has a surplus, it is the PSF's revenue. PyCon's
> surplus is the biggest source of revenue for the PSF.
>
>>
>>
>> You can add the **Python Software Foundation** to a Charity Navigator
>> "Giving Basket" and get one receipt:
>> https://www.charitynavigator.org/index.cfm?bay=search.profil
>> e=043594598
>>
>
>> "Charity Navigator's Methodology"
>> https://www.charitynavigator.org/index.cfm?bay=content.view;
>> cpid=5593#rating
>>
>>
>> Is there any way to donate cryptocurrency to PSF yet?
>>
>
> Currently, we do not.
>
>>
>> All of these orgs accept BTC donations:
>> https://en.bitcoin.it/wiki/Donation-accepting_organizations_and_projects
>>
>> Just sending to an address doesn't get you a receipt for tax purposes;
>> there are firms that support donations by exchanging for USD at that time
>> and depositing to the configured account.
>>
>
> That's good to know! If you know more info on this, please let me know. I
> can share it with our accounting team to review.
>

Re: PSF accepting cryptoasset donations

Wikimedia, Save The Children, UNICEF, United Way, Mozilla, EFF, and FSF,
for example, all accept cryptocurrency donations.

OpenCollective reviews a number of possible payment solutions:
https://github.com/opencollective/opencollective/
issues/919#issuecomment-370218843 (now with a GitHub poll in the issue
description)

>From GuideStar (which, like Charity Navigator, also has nonprofit
evaluation criteria)
https://trust.guidestar.org/bitcoin-what-nonprofits-need-to-
know-and-how-it-might-give-your-fundraising-a-competitive-advantage :

> Accepting donations in Bitcoin can reduce the financial transaction fees
a nonprofit owes.  With Bitcoin, the costs are significantly lower compared
to checks, credit cards, and other digital options, and if your
organization is a registered 501(c)(3), there are zero transaction fees on
platforms such as Coinbase or Bitpay.  There is also no risk of bank
charges accruing to the nonprofit in the case of a donor using a fraudulent
credit card.

* Merch: It may be worth accepting cryptocurrency payments for PSF merch
like T-Shirts, Mugs, Polo shirts:
https://www.python.org/community/merchandise/

  * Coinbase Commerce integrates with a number of major eCommerce platforms:
https://commerce.coinbase.com/integrate

* SCHEMA.ORG/Offer and schema:Product markup improves SEO; so that when I
search for "PSF shirt" with an voice assistant on my way to PyCon, some
portion of that revenue should go to PSF

* From a conversation with Eric Holscher regarding ReadTheDocs' ethical ads
revenue model,

  * Yeti makes solid drinkware that can be customized with a logo:
https://www.yeti.com/en_US/custom-monogram-text-logo-drinkware

  * And, for accounting, Zapier automates connections between a bunch of
APIs with e.g. QuickBooks Online; which can save accounting's time for
things like copying merch invoices to the accounts payable, refund, and
chargeback accounting ledgerd.
  https://zapier.com/apps/quickbooks/integrations

* Integrating E.G. Shopify with QuickBooks doesn't require any Zapier Zaps.
Shopify also integrates with e.g. Amazon Fulfillment so that they can
manage inventory, returns, and shipping with their warehouses and efficient
logistics platform.:
https://quickbooks.intuit.com/integrations/shopify/

* There's a CreativeCommons-licensed logo SVG on Wikipedia. Organizations
listed on the Merch page have pledged to donate a portion of their proceeds
to PSF:
  https://commons.wikimedia.org/wiki/File:Python.svg

* What mailing list and/or CRM can I opt-into with my PSF donation? Is
there a periodic breakdown of expenses that I can backseat-drive and
micromanage?

>From Charity Navigator, it looks like PSF had about $ 2,500,000 in donation
revenue a few years ago.

Is there some form of a voluntary nonprofit sustainability report; the the
now #GlobalGoals-aligned GRI Standards?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/APGNHJX7M6WYJ57O5TLHQSJL45Z5RIX4/

[Python-Dev] Re: Sponsoring Python development via the PSF

[Wes Turner]

When companies donate as PyCon sponsors (and get brand recognition) do
those donations also go to PSF?


You can add the **Python Software Foundation** to a Charity Navigator
"Giving Basket" and get one receipt:
https://www.charitynavigator.org/index.cfm?bay=search.profile=043594598

"Charity Navigator's Methodology"
https://www.charitynavigator.org/index.cfm?bay=content.view=5593#rating


Is there any way to donate cryptocurrency to PSF yet?

All of these orgs accept BTC donations:
https://en.bitcoin.it/wiki/Donation-accepting_organizations_and_projects

Just sending to an address doesn't get you a receipt for tax purposes;
there are firms that support donations by exchanging for USD at that time
and depositing to the configured account.

On Wednesday, June 26, 2019, Steve Dower  wrote:

> On 26Jun2019 1006, Ivan Pozdeev via Python-Dev wrote:
>
>> +1. That's why I never donate to charity. I can't see or control what my
>> money is going to be used for.
>>
>
> That's not charity then, it's a purchase or an investment.
>
> On 26.06.2019 10:41, Pau Freixes wrote:
>
>> Why not the other way around? Having from the very beginning a clear goal
>> with a speculative budget, companies would have visibilitty about the end
>> goal of their donations. Otherwise is a bit a leap of faith.
>>
>
> Different companies have different needs. Some are interested in
> investing, some are interested in general altruism, some are interested in
> directed donations.
>
> This just enables directed donations by having a separate fund, which I
> assume will be reported by the PSF separately from other income. So now we
> have all bases covered - those who want to invest can continue to hire
> people or provide infrastructure, those who want to make open-ended
> donations can use the already existing channels, and those who want
> donations directed for development work can now choose that.
>
> Cheers,
> Steve
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at https://mail.python.org/archiv
> es/list/python-dev@python.org/message/NKDX4YYHAEBSPCTKC6PWPXCTF7TNMOFJ/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/CISUQG52G5DTJWCD33NKEUWJ5ST22KX7/

[Python-Dev] Re: python-ideas and python-dev migrated to Mailman 3/HyperKitty

[Wes Turner]

Thanks for getting these upgraded. IMHO, being able to copy URLs from list
message footers as references in e.g. issues will be a great boost in
productivity.

On Friday, June 7, 2019, Stephen J. Turnbull <
turnbull.stephen...@u.tsukuba.ac.jp> wrote:

> Barry Warsaw writes:
>  > On Jun 6, 2019, at 09:15, David Mertz  wrote:
>  > >
>  > > The old URL is definitely a lot friendlier, even apart from the
> length.
>  >
>  > Unfortunately, the old URLs aren’t really permanent.
>
> True.  That could be addressed in theory, but it would be fragile (ie,
> vulnerable to loss or corruption of the external database mapping
> messages to URLs).  Calculating from the message itself means that if
> you have the message you can always get where you want to go.
>
>  > The new URLs are guaranteed to be reproducible from the original
>  > message source.  The downside is that they are less friendly.
>
> They could, however be made more friendly than they currently are.
> There's no reason (in principle, of course it requires changing code
> and the DNS) why your message, currently given the Archived-At URL
>
> https://mail.python.org/archives/list/python-dev@python.org/message/
> EFHTPGCSB5VZSRS3DDXZN6ETYP5H6NDS/
>
> couldn't be given (A is for Archives)
>
> https://a.python.org/python-dev@python.org/EFHTPGCSB5VZSRS3DDXZN6ETYP5H6N
> DS/
>
> which gets it down to an RFC-conformant 76 characters. ;-)  Of course
> many lists would overflow that, and I agree with David that
>
> https://a.python.org/python-dev@python.org/2019/06/
> EFHTPGCSB5VZSRS3DDXZN6ETYP5H6NDS/
>
> would be better still.  Although the risk of collision would be orders
> of magnitude higher (the date buys us some leeway but not much, we
> could make the ID-Hash be 2019/06/B5VZSRS3DDXZN6ET (arbitrarily chose
> middle 16), giving


> https://a.python.org/python-dev@python.org/2019/06/B5VZSRS3DDXZN6ET
>
> (67 characters, allowing a few more characters for domain names and/or
> list names -- note with the current scheme, a domain name which is 1
> character longer probably uses up two more characters of space).


Are these message IDs or hashes?
Do they have to be (is this) base-36?
Could they instead be base-62? (26+10+26)


>
> None of this is very attractive to me, for reasons I will go into on
> Mailman-Developers or gitlab.com/mailman/mailman/issues if you want to
> file one.  Briefly, people who want bit.ly-length short URLs won't be
> satisfied, and the proposed URLs are more useful but still ugly.


We shouldn't just drop extra date information from the URL and only lookup
by the messageid unless we add a redirect to the correct dated URL; because
caching and trickery.


>
> Personally I think we should all just switch to RestructuredText- and
> Markdown-capable MUAs, and kill off both ugly visible URLs and HTML
> email with one big ol' rock.


While I personally prefer .rst and .md, hovering over URL anchor text takes
unnecessary time (and I'll remember whether I've been to the actual
http://URL, but not 'here' and 'there').
So I'm fine with ridiculous, preposterous long links (even in the middle of
the email; without footnotes to scroll back and forth to)


>
> Steve
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at https://mail.python.org/archives/list/python-dev@
> python.org/message/O3T27UUHKKXATOPJT4KEQHREUGYVMELV/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/N62MYXZCUHPDTEQWWTH33BU2FIZ3ROPF/

Re: [Python-Dev] Online Devguide mostly not working

[Wes Turner]

https://cpython-devguide.readthedocs.io/ seems to work but
https://devguide.python.org/* does not

https://readthedocs.org/projects/cpython-devguide/ lists maintainers, who
I've cc'd

AFAIU, there's no reason that the HTTP STS custom domain CNAME support
would've broken this:
https://github.com/rtfd/readthedocs.org/issues/4135

On Saturday, May 11, 2019, Jonathan Goble  wrote:

> Confirming that I also cannot access the Getting Started page. I'm in
> Ohio, if it matters.
>
> On Sat, May 11, 2019 at 6:26 PM Terry Reedy  wrote:
> >
> > https://devguide.python.org gives the intro page with TOC on sidebar and
> > at end.  Clicking anything, such as Getting Started, which tries to
> > display https://devguide.python.org/setup/, returns a Read the Docs page
> > "Sorry This page does not exist yet."  'Down for everyone' site also
> > cannot access.
> >
> > --
> > Terry Jan Reedy
> >
> > ___
> > Python-Dev mailing list
> > Python-Dev@python.org
> > https://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> jcgoble3%40gmail.com
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> wes.turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Actor Model in Python

[Wes Turner]

https://trio.readthedocs.io/en/latest/reference-core.html#synchronizing-and-communicating-between-tasks

https://pypi.org/search/?q=Actor+model

https://en.wikipedia.org/wiki/Actor_model

https://en.wikipedia.org/wiki/Bulk_synchronous_parallel#The_model



On Friday, April 26, 2019, Eric Snow  wrote:

> On Sun, Mar 31, 2019 at 9:19 AM Aratz Manterola Lasa via Python-Dev
>  wrote:
> > I was wondering if there was any Python project aiming to implement the
> actor model for python concurrency.
>
> As far as the standard library goes, the explicitly supported
> concurrency models are: threading, multiprocessing, and async/await.
> Between these (and a few other parts provided by Python) anyone can
> build libraries that emulate various other concurrency models.  Such
> libraries exist on the cheeseshop (PyPI), though I don't know about
> packages for the actor model specifically.  I recommend searching
> there for such packages.  If you don't find one then perhaps you've
> found a new project to start. :)
>
> Also, I have a proposal [1] for Python 3.9 that provides first class
> [low level] support for concurrency models like CSP and the actor
> model.  This is done with multiple [mostly] isolated interpreters per
> process and with basic "channels" for safely passing messages between
> them.  While the proposed library is intended to be useful on its own,
> it is also intended to provide effective building blocks for library
> authors.  Note that the PEP has not been accepted and is not
> guaranteed to be accepted (though I'm hopeful).
>
>

> Regardless, consider posting to python-l...@python.org for feedback
> from the broader Python community.  This list is specifically used for
> the development of the Python language itself.  Thanks!


Or python-id...@python.org ,
though I'm not sure what would be needed from core Python or stdlib to
create another actor model abstraction on top of the actual concurrency
primitives.

Truly functional actors are slow when/because the memory is not shared
inter-process

https://arrow.apache.org/docs/python/memory.html#referencing-and-allocating-memory

https://arrow.apache.org/docs/python/ipc.html#arbitrary-object-serialization

https://www.python.org/dev/peps/pep-0554/#interpreter-isolation


>
> -eric
>
>
> [1] https://www.python.org/dev/peps/pep-0554/


"PEP 554 -- Multiple Interpreters in the Stdlib"
https://www.python.org/dev/peps/pep-0554/

Is there / are there Issues, PRs, and Mailing List Threads regarding the
status of this proposal?

So sorry to interrupt,


> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> wes.turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Need help to fix HTTP Header Injection vulnerability

[Wes Turner]

1. Is there a library of URL / Header injection tests e.g. for fuzzing that
we could generate additional test cases with or from?

2. Are requests.get() and requests.post() also vulnerable?

3. Despite the much-heralded UNIX pipe protocols' utility, filenames
containing newlines (the de-facto line record delimiter) are possible:
"file"$'\n'"name"

Should filenames containing newlines and control characters require a kwarg
to be non-None in order to be passed through unescaped to the HTTP request?

On Wednesday, April 10, 2019, Karthikeyan  wrote:

> Thanks Gregory. I think it's a good tradeoff to ensure this validation
> only for URLs of http scheme.
>
> I also agree handling newline is little problematic over the years and the
> discussion over the level at which validation should occur also prolongs
> some of the patches. https://bugs.python.org/issue35906 is another
> similar case where splitlines is used but it's better to raise an error and
> the proposed fix could be used there too. Victor seemed to wrote a similar
> PR like linked one for other urllib functions only to fix similar attack in
> ftplib to reject newlines that was eventually fixed only in ftplib
>
> * https://bugs.python.org/issue30713
> * https://bugs.python.org/issue29606
>
> Search also brings multiple issues with one duplicate over another that
> makes these attacks scattered over the tracker and some edge case missing.
> Slightly off topic, the last time I reported a cookie related issue where
> the policy can be overriden by third party library I was asked to fix it in
> stdlib itself since adding fixes to libraries causes maintenance burden to
> downstream libraries to keep up upstream. With urllib being a heavily used
> module across ecosystem it's good to have a fix landing in stdlib that
> secures downstream libraries encouraging users to upgrade Python too.
>
> Regards,
> Karthikeyan S
>
>>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] datetime.timedelta total_microseconds

[Wes Turner]

You could specify the return value type annotations in the docstring of a
convert()/to_unit() method, or for each to_unit() method. Are they all
floats?

div and floordiv are not going away. Without reading the docs, I, too,
wouldn't have guessed that division by the desired unit is the correct way.

In terms of performance, a convert()/to_unit() method must do either a hash
lookup or multiple comparisons to present a more useful exception  than
that returned by initializing timedelta with something like nanoseconds=.

FWIW, astropy.time.TimeDelta supports sub-nanosecond precision and has a
.to() method for changing units/quantities.
http://docs.astropy.org/en/stable/time/#time-deltas

> The TimeDelta class is derived from the Time class and shares many of its
properties. One difference is that the time scale has to be one for which
one day is exactly 86400 seconds. Hence, the scale cannot be UTC.
>
> The available time formats are:
>
> Format Class
> secTimeDeltaSec
> jd  TimeDeltaJD
> datetime  TimeDeltaDatetime

http://docs.astropy.org/en/stable/api/astropy.time.TimeDelta.html#astropy.time.TimeDelta

http://docs.astropy.org/en/stable/_modules/astropy/time/core.html#TimeDelta.to


On Thursday, February 28, 2019, Alexander Belopolsky <
alexander.belopol...@gmail.com> wrote:

> > while "some_var / some_other_var" could be doing anything.
>
> "At an elementary level the division of two natural numbers is – among
> other possible interpretations – the process of calculating the number of
> times one number is contained within another one."
>
> -- 
>
> The process of figuring out how many seconds fit into a given interval is
> called division by a second.
>
> I am afraid people who get confused by timedelta / timedelta division know
> too much about Python where / can indeed mean anything including e.g.
> joining filesystem paths.
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] OpenSSL 1.1.1 update for 3.7/3.8

[Wes Turner]

Thanks, as always

On Tue, Feb 26, 2019 at 4:45 PM Christian Heimes 
wrote:

> On 26/02/2019 21.31, Wes Turner wrote:
> >> IMHO it's
> > fine to ship the last 2.7 build with an OpenSSL version that was EOLed
> > just 24h earlier.
> >
> > Is this a time / cost issue or a branch policy issue?
> >
> > If someone was to back port the forthcoming 1.1.1 to 2.7 significantly
> > before the EOL date, could that be merged?
>
> My mail is about official binary Python packages for Windows and macOS.
> We stick to an OpenSSL version to guarantee maximum backwards
> compatibility within a minor release. OpenSSL 1.1.1 has TLS 1.3 support
> and prefers TLS 1.3 over TLS 1.2. There is a small change that TLS 1.3
> breaks some assumptions.
>
> Python 2.7 works mostly fine with OpenSSL 1.1.1. There are some minor
> test issues related to TLS 1.3 but nothing serious. Linux distros have
> been shipping Python 2.7 with OpenSSL 1.1.1 for a while.
>
>
> > There are all sorts of e.g. legacy academic works that'll never be
> > upgraded etc etc
>
> That topic is out of scope and has been discussed countless times.
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] OpenSSL 1.1.1 update for 3.7/3.8

[Wes Turner]

> IMHO it's
fine to ship the last 2.7 build with an OpenSSL version that was EOLed
just 24h earlier.

Is this a time / cost issue or a branch policy issue?

If someone was to back port the forthcoming 1.1.1 to 2.7 significantly
before the EOL date, could that be merged?

There are all sorts of e.g. legacy academic works that'll never be upgraded
etc etc

On Tuesday, February 26, 2019, Christian Heimes 
wrote:

> Hi,
>
> today's OpenSSL release of 1.0.2r and 1.1.1b reminded me of OpenSSL's
> release strategy [1]. OpenSSL 1.0.2 will reach EOL on 2019-12-31, 1.1.0
> will reach EOL on 2019-09-11 (one year after release of OpenSSL 1.1.1).
>
> First the good news: There is no need to take any action for 2.7 to 3.6.
> As of today, Python 2.7, 3.5, and 3.6 are using OpenSSL 1.0.2. Python
> 3.6.8 (2018-12-24) and 3.5.5 (2018-02-05) were the last regular update
> with binary packages. 3.5.6 is a source-only security release. 3.6.9
> will be the first source-only security release of the 3.6 series. Python
> 2.7 will reach EOL just a day after OpenSSL 1.0.2 reaches EOL. IMHO it's
> fine to ship the last 2.7 build with an OpenSSL version that was EOLed
> just 24h earlier.
>
> Python 3.7 and master (3.8) are affected. As of now, both branches use
> OpenSSL 1.1.0 and must be updated to 1.1.1 soonish. Ned has scheduled
> 3.7.3 release for 2019-03-25. That's still well within the release
> schedule for 1.1.0. I suggest that we update to 1.1.1 directly after the
> release of Python 3.7.3 and target 3.7.4 as first builds with TLS 1.3
> support. That gives Victor, Steve, and me enough time to sort out the
> remaining issues.
>
> In worst case we could revert the update and postpone the update to
> 3.7.5. Or we disable TLS 1.3 support by default in Mac and Windows builds.
>
> Christian
>
> [1] https://www.openssl.org/policies/releasestrat.html
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> wes.turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] pickle5 backport updated

[Wes Turner]

Maybe a bit OT: is there a way to *not pickle any callables*?

On Wednesday, January 16, 2019, Wes Turner  wrote:

> Thanks! Apache Arrow may also be worth a look:
>
> > Apache Arrow is a cross-language development platform for in-memory
> data. It specifies a standardized language-independent columnar memory
> format for flat and hierarchical data, organized for efficient analytic
> operations on modern hardware. It also provides computational libraries and
> zero-copy streaming messaging and interprocess communication.
>
> > The Arrow Python bindings have first-class integration with NumPy,
> pandas, and built-in Python objects
>
> https://arrow.apache.org/docs/python/
>
> Pickle supports arbitrary Python objects without any schema definition
> (other than that what's defined in the object's __reduce__()
> and __reduce_ex__())
>
> On Wednesday, January 16, 2019, Antoine Pitrou 
> wrote:
>
>>
>> Hello,
>>
>> For the record, the pickle5 backport (PEP 574) was updated to include
>> the latest pickle changes from CPython git master.
>>
>> pickle5 is available for Python 3.6 and 3.7.
>> https://pypi.org/project/pickle5/
>>
>> Regards
>>
>> Antoine.
>>
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: https://mail.python.org/mailman/options/python-dev/wes.
>> turner%40gmail.com
>>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] pickle5 backport updated

[Wes Turner]

Thanks! Apache Arrow may also be worth a look:

> Apache Arrow is a cross-language development platform for in-memory data.
It specifies a standardized language-independent columnar memory format for
flat and hierarchical data, organized for efficient analytic operations on
modern hardware. It also provides computational libraries and zero-copy
streaming messaging and interprocess communication.

> The Arrow Python bindings have first-class integration with NumPy,
pandas, and built-in Python objects

https://arrow.apache.org/docs/python/

Pickle supports arbitrary Python objects without any schema definition
(other than that what's defined in the object's __reduce__()
and __reduce_ex__())

On Wednesday, January 16, 2019, Antoine Pitrou  wrote:

>
> Hello,
>
> For the record, the pickle5 backport (PEP 574) was updated to include
> the latest pickle changes from CPython git master.
>
> pickle5 is available for Python 3.6 and 3.7.
> https://pypi.org/project/pickle5/
>
> Regards
>
> Antoine.
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> wes.turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] "Deprecation" of os.system in favor of subprocess?

[Wes Turner]

Sarge is one way to appropriately wrap Process Control (with shell-like
piping of stdin and stdout between multiple processes).

os.system just calls C system(), AFAIU

subprocess.call calls subprocess.Popen, which does far more than just
wrapping C system(); though the return value is also the process returncode.


https://sarge.readthedocs.io/en/latest/

https://docs.python.org/3/library/os.html#os.system
https://github.com/python/cpython/blob/master/Modules/posixmodule.c

https://docs.python.org/3/library/subprocess.html#subprocess.Popen.returncode

https://docs.python.org/3/library/subprocess.html#subprocess.call

https://docs.python.org/3/library/subprocess.html#subprocess.check_call

https://en.wikipedia.org/wiki/System_call
https://en.wikipedia.org/wiki/Exec_(system_call)

...

"ENH: Add call, check_call, check_output, CalledProcessError,
expect_returncode"
https://bitbucket.org/vinay.sajip/sarge/pull-requests/1/enh-add-call-check_call-check_output/diff

On Wednesday, October 24, 2018, Steve Dower  wrote:

> On 24Oct2018 0934, Calvin Spealman wrote:
>
>> In the spirit of "There should be one-- and preferably only one --obvious
>> way to do it." this makes perfect sense.
>>
>
> To do what? There is one obvious way to run a system command, and one
> obvious way to manage subprocesses. There are also many non-obvious ways to
> run system commands, and many non-obvious ways to manage subprocesses.
>
> The distinction between "your own machine and other peoples machines" is
>> not always clear, nor planned for, nor understood by developers to be an
>> important distinction to make up-front. So the encouragement should be
>> clear.
>>
>
> Agreed. One good heuristic is whether you're putting the system command in
> a function or not, since functions are explicitly designing for reuse while
> just using it in a script is not.
>
> Simply put, there is no valid use case for os.system over subprocess by
>> remaining it must be considered redundant.
>>
>
> You have not shown this. Posting quotes followed by an unrelated
> conclusion isn't a very compelling form of argument :)
>
> Cheers,
> Steve
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/wes.
> turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Petr Viktorin as BDFL-Delegate for PEP 580

[Wes Turner]

On Wednesday, October 3, 2018, INADA Naoki  wrote:

>
> 2018年10月3日(水) 21:24 Jeroen Demeyer :
>
>> Hello,
>>
>>
>> I am well aware of the current governance issues, but several people
>> have mentioned that the BDFL-Delegate process can still continue for
>> now.
>
>
> Really?
> I don't know process to assign BDFL-delegate without BDFL.
>

AFAIU, there is not yet a documented process for BDFL-delegate assignment.

There's this in the devguide; which links to PEP1:

"20.2. PEP Process¶"
https://devguide.python.org/langchanges/#pep-process
https://github.com/python/devguide/blob/master/langchanges.rst


And PEP 1:

"PEP 1 -- PEP Purpose and Guidelines"
  "PEP Workflow"
https://www.python.org/dev/peps/pep-0001/#pep-workflow
  "PEP Editors"
https://www.python.org/dev/peps/pep-0001/#pep-editors
  "PEP Editor Responsibilities & Workflow"
https://www.python.org/dev/peps/pep-0001/#pep-editor-responsibilities-workflow

https://github.com/python/peps/blob/master/pep-0001.txt

And the devguide has a list of experts:
https://devguide.python.org/experts/


Maybe PEP1 is the place to list current BDFL-Delegates
(in addition to in the PEP metadata as in the OT PR:
python/peps#797
"PEP 580: Petr Viktorin as BDFL-Delegate"
)?


Not to bikeshed, but is BDFL-Delegate still the current term because that's
what's in all the other PEPs' metadata?


>
> This PEP is mainly for third party tools.
> I want to get much feedback from them before new APIs become stable (e.g.
> 3.8b1)
>
> So I want this PEP is approved (or
> Provisionally Accepted) and reference implementation is merged as fast as
> possible.
>
> Regards,
>
>>
>>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Petr Viktorin as BDFL-Delegate for PEP 580

[Wes Turner]

On Wednesday, October 3, 2018, Jeroen Demeyer  wrote:

> Hello,
>
> I would like to propose Petr Viktorin as BDFL-Delegate for PEP 580, titled
> "The C call protocol". He has co-authored several PEPs (PEP 394, PEP 489,
> PEP 534, PEP 547, PEP 573), several of which involve extension modules.
>
> Petr has agreed to become BDFL-Delegate for PEP 580 if asked. Also Antoine
> Pitrou, INADA Naoki and Nick Coghlan have approved Petr being BDFL-Delegate.
>
> I am well aware of the current governance issues, but several people have
> mentioned that the BDFL-Delegate process can still continue for now. I
> created a PR for the peps repository at https://github.com/python/peps
> /pull/797


+1. Are we doing upvotes on the mailing list or on the GitHub PR and/or
issue now?

"[Python-ideas] PEPs: Theory of operation"
https://markmail.org/thread/zr4o6l7ivnj4irtp

"""
Process suggestions that could minimize non-BDFL's BDFL legwork:

[...]

* Use GitHub reactions for voting on BDFL delegates, PEP final approval,
and PEP sub issues?
  * Specify a voting deadline?
  * How to make a quorum call?
  * Add '@core/team' as reviewers for every PEP?


* Link to the mailing list thread(s) at the top of the PR
  * [ ] Add unique message URLs to footers with mailman3


* What type of communications are better suited for mailing lists over PEP
pull-requests and PEP code reviews?
[The original thread is probably a better place to discuss PEP process
going forward]
"""


>
>
> Cheers,
> Jeroen Demeyer.
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/wes.
> turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] SEC: Spectre variant 2: GCC: -mindirect-branch=thunk -mindirect-branch-register

[Wes Turner]

On Friday, September 21, 2018, Mark Lawrence  wrote:

> On 21/09/18 08:11, Wes Turner wrote:
>
>> I feel like you are actively undermining attempts to prevent exploitation
>> of known vulnerabilities because the software in question is currently too
>> slow.
>>
>>
> Did you write "How to Win Friends And Influence People"?


"Andrew Carnegie - Rags to Riches Power to Peace"
... Vredespaleis

"Think and Grow Rich" by Napoleon Hill

Today is International Day of Peace.

But, now, I've gone way OT,
So, we'll just have to hope IBRS is enabled,
On all of the Jupyter cloud hosts.


>
> --
> My fellow Pythonistas, ask not what our language can do for you, ask
> what you can do for our language.
>
> Mark Lawrence
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/wes.
> turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] SEC: Spectre variant 2: GCC: -mindirect-branch=thunk -mindirect-branch-register

[Wes Turner]

I feel like you are actively undermining attempts to prevent exploitation
of known vulnerabilities because the software in question is currently too
slow.

For a 4-8% performance penalty, we could just add the CFLAGS to the build
now and not worry about it.

I give up.

On Friday, September 21, 2018, Franklin? Lee 
wrote:

> On Thu, Sep 20, 2018 at 2:10 PM Wes Turner  wrote:
> >
> > On Thursday, September 20, 2018, Stefan Ring 
> wrote:
> >>
> >> On Tue, Sep 18, 2018 at 8:38 AM INADA Naoki 
> wrote:
> >>
> >> > I think this topic should split to two topics: (1) Guard Python
> >> > process from Spectre/Meltdown
> >> > attack from other process, (2) Prohibit Python code attack other
> >> > processes by using
> >> > Spectre/Meltdown.
> >>
> >> (3) Guard Python from performance degradation by overly aggressive
> >> Spectre "mitigation".
> >
> >
> > > Spectre has the potential of having a greater impact on cloud
> providers than Meltdown. Whereas Meltdown allows unauthorized applications
> to read from privileged memory to obtain sensitive data from processes
> running on the same cloud server, Spectre can allow malicious programs to
> induce a hypervisor to transmit the data to a guest system running on top
> of it.
> > - Private SSL certs
> > - Cached keys and passwords in non-zeroed RAM
> > - [...]
> >
> > https://en.wikipedia.org/wiki/Spectre_(security_vulnerability)
>
> It's true that the attacks should worry cloud providers. Doesn't that
> mean that companies like Amazon, Microsoft (Steve), and Docker should
> have done analyses on CPython's vulnerability to these exploits?
> Has/should/can anyone officially representing Python contact the
> companies and ask them?
>
> When I followed your quote to find the context, I found it uses, as
> its source, a Forbes article. The source cited by THAT article is
> Daniel Gruss, who was one of the researchers. Should someone from the
> PSF contact the researchers? Steve says he spoke to some of them to
> judge whether the proposed compiler flags would help, and decided
> against it.
>
> Absent of expert input, here's my non-expert take: That quote requires
> an OS-level fix. A Python program without the proper permissions can't
> do such things unless there is a vulnerability with the OS, and it is
> extremely unlikely for anyone to update Python for Spectre but not
> update the OS (and they'd be screwed in any case). And even if there
> is a vulnerability in the OS, maybe the way to exploit it is by using
> arbitrary Python execution (which you need before you can TRY to use
> Spectre) on this Python interpreter. You can then write a new binary
> file and run THAT, and it will be fast enough. That's not something
> you can fix about CPython.
>
> Also, (again with my understanding) the problem of Spectre and
> Meltdown are that you can escape sandboxes and the like, such as the
> user/kernel divide, or a software sandbox like that provided by a
> JavaScript VM. For CPython to be "vulnerable" to these attacks, it
> needs to have some kind of sandbox or protection to break out of.
> Instead, we sometimes have sandboxes AROUND CPython (like Jupyter) or
> WITHIN CPython. I don't see how it makes sense to talk about a sandbox
> escape FOR CPython (yet).
>
> Your original post linked to a discussion about Linux using those
> build flags. Linux is a kernel, and has such protections that can be
> bypassed, so it has something to worry about. Malicious code can be
> native code, which (to my understanding) will be fast enough to
> exploit the cache miss time. Here's Google's article about the
> retpoline and why it helps:
> https://support.google.com/faqs/answer/7625886
>
> As of yet, you have quoted passages that have little relevance to
> interpreter devs, especially non-JIT interpreters, and you have linked
> to entire articles for non-experts with little relevance to
> interpreter devs. This doesn't show that you have any better of an
> understanding than I have, which is less than the understanding that
> some of the Python devs have, and much less than what Steve has. In
> short, it LOOKS like you don't know what you're talking about. If you
> have a different and deeper understanding of the problem, then you
> need to show it, and say why there is a problem for CPython
> specifically. Or find someone who can do that for you.
>
> > Here's one:
> > https://github.com/Eugnis/spectre-attack/blob/master/Source.c
> >
> > Is this too slow in CPython with:
> > - Coroutines (asyncio (tulip))
> > - PyPy JIT *
> > - Numba JIT *
> > - C Extensions *
> &

Re: [Python-Dev] SEC: Spectre variant 2: GCC: -mindirect-branch=thunk -mindirect-branch-register

[Wes Turner]

On Thursday, September 20, 2018, Stefan Ring  wrote:

> On Tue, Sep 18, 2018 at 8:38 AM INADA Naoki 
> wrote:
>
> > I think this topic should split to two topics: (1) Guard Python
> > process from Spectre/Meltdown
> > attack from other process, (2) Prohibit Python code attack other
> > processes by using
> > Spectre/Meltdown.
>
> (3) Guard Python from performance degradation by overly aggressive
> Spectre "mitigation".


> Spectre has the potential of having a greater impact on cloud providers
than Meltdown. Whereas Meltdown allows unauthorized applications to read
from privileged memory to obtain sensitive data from processes running on
the same cloud server, Spectre can allow malicious programs to induce a
hypervisor to transmit the data to a guest system running on top of it.

- Private SSL certs
- Cached keys and passwords in non-zeroed RAM
- [...]

https://en.wikipedia.org/wiki/Spectre_(security_vulnerability)


I really shouldn't need to apologise for bringing this up here.

Here's one:
https://github.com/Eugnis/spectre-attack/blob/master/Source.c

Is this too slow in CPython with:
- Coroutines (asyncio (tulip))
- PyPy JIT *
- Numba JIT *
- C Extensions *
- Cython *

* Not anyone here's problem.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] SEC: Spectre variant 2: GCC: -mindirect-branch=thunk -mindirect-branch-register

[Wes Turner]

To summarize:

- CPython may be vulnerable to speculative execution vulnerabilities, but
none are known.
- In general, CPython is currently too slow for speculative execution
exploitation to be practical.
  - Sandboxed, JIT'ed JS is not too slow for speculative execution
exploitation to be practical
- (Not otherwise discussed here: PyPy's sandboxed JIT may not be too
slow for speculative execution exploitation to be practical.)

- C extensions may be vulnerable to speculative execution vulnerabilities;
but that's the authors' and users' problem (and so it's appropriate to
mention this to extension owners following distutils-sig/PyPA)
  - C extensions can set indirect branch CFLAGS only with GCC 7.3 and GCC
8+; which are only usable with conda and the forthcoming manylinux2010 spec

- Linux kernels with [IBRS, STIBP, IBPB] can enable userspace protection
- The revised worst-case performance impacts for variant 2 mitigations are
4-8%

- MSVC has a /Qspectre flag for variant 1

- Because there is no exploit provided (or currently thought possible with
just CPython), this security-related dialogue is regarded as a nuisance.
- There is no published or official statement or investigation from the
Python community regarding Spectre (or Meltdown) vulnerabilities.

Here's a good write-up:
Safety_instructions_for_Meltdown_and_Spectre

Have a good day!

On Monday, September 17, 2018, Steve Dower  wrote:

> On 17Sep2018 1158, Wes Turner wrote:
>
>> On Monday, September 17, 2018, Steve Dower > <mailto:steve.do...@python.org>> wrote:
>>
>> I investigated this thoroughly some time ago (when the MSVC flags
>> became available) and determined (with the help of some of the
>> original Spectre/Meltdown investigation team) that there is no
>> significant value in enabling these flags for Python.
>>
>> What did you fuzz with?
>> Does that assume that e.g. Fortify has identified all bugs in CPython C?
>> There have been a number of variants that have been disclosed; which did
>> who test for?
>>
>
> Don't change the subject.
>
> It boiled down to:
>> * Python allows arbitrary code execution by design
>>
>> Yet binaries built with GCC do have NX? Unless nested functions in C
>> extensions?
>>
>
> I don't know anything about GCC settings. Binaries for Windows have been
> built with this option for over a decade. It's unrelated to
> Spectre/Meltdown.
>
> * Pure Python code in CPython has very long per-instruction opcode
>> sequences that cannot easily be abused or timed
>>
>> A demonstration of this would be helpful.
>>
>
> That's not how proof-of-concepts work. You can't assume that the lack of a
> demonstration proves it is possible - at best you have to assume that it
> proves it is *not* possible, but really it just proves that nobody has a
> demonstration yet.
>
> What I could demonstrate (again) if I thought it would be worthwhile is
> that the changes enabled by the flag do not affect the normal interpreter
> loop, and do not affect any code that can be called fast enough to
> potentially leak information. Feel free to go ahead and build with/without
> the flags and compare the disassembly (and if you do this and find that
> compilers are detecting new cases since I looked, *that* would be very
> helpful to share directly with the security team).
>
> * Injected pure Python code cannot be coerced into generating native
>> code that is able to abuse Spectre/Meltdown but not able to abuse
>> other attacks more easily
>>
>>   So, not impossible.
>>
>
> Of course it's not impossible. But why would you
>
> * Code injection itself is outside of this particular threat model
>>
>> [Jupyter] Notebook servers are as wide open to arbitrary code execution
>> as browser JS JITs; often with VMs and/or containers as a 'sandbox'
>>
>
> `pip install requirements.txt` installs and executes unsigned code:
>> Python, C extensions
>>
>> What can a container do to contain a speculative execution exploit
>> intending to escape said container?
>>
>
> Python's threat model does not treat the Python process as a sandbox. To
> say it another way, if you assume the Python process is a sandbox, you're
> on your own.
>
> Arbitrary code, Python or otherwise, can totally escape the process, and
> then it's up to the OS to protect against escaping the machine. We do what
> we can to reduce unnecessary arbitrary code, but unless you've properly
> protected your environment then you have a lot more to worry about besides
> speculative execution vulnerabilities.
>
> By comparison with JavaScript, most JS JITs can be easily coerced
>> 

Re: [Python-Dev] SEC: Spectre variant 2: GCC: -mindirect-branch=thunk -mindirect-branch-register

[Wes Turner]

On Mon, Sep 17, 2018 at 2:58 PM Wes Turner  wrote:

>
> I thought I read that RH has a kernel flag for userspace?
>

"Controlling the Performance Impact of Microcode and Security Patches for
CVE-2017-5754 CVE-2017-5715 and CVE-2017-5753 using Red Hat Enterprise
Linux Tunables"
https://access.redhat.com/articles/3311301

> Indirect Branch Restricted Speculation (ibrs)
> [...] When ibrs_enabled is set to 1 (spectre_v2=ibrs) the kernel runs
with indirect branch restricted speculation, which protects the kernel
space from attacks (even from hyperthreading/simultaneous multi-threading
attacks). When IBRS is set to 2 (spectre_v2=ibrs_always), both userland and
kernel runs with indirect branch restricted speculation. This protects
userspace from hyperthreading/simultaneous multi-threading attacks as well,
and is also the default on certain old AMD processors (family 10h, 12h and
16h). This feature addresses CVE-2017-5715, variant #2.
> [...]
> echo 2 > /sys/kernel/debug/x86/ibrs_enabled

https://wiki.ubuntu.com/SecurityTeam/KnowledgeBase/SpectreAndMeltdown/MitigationControls
> echo 2 > /proc/sys/kernel/ibrs_enabled will turn on IBRS in both
userspace and kernel

...
On Mon, Sep 17, 2018 at 5:26 AM Antoine Pitrou  wrote:

> If you want to push this forward, I suggest you measure performance of
> Python compiled with and without the Spectre mitigation options, and
> report the results here.  That will help vendors and packagers decide
> whether they want to pursue the route of enabling those options.


"Speculative Execution Exploit Performance Impacts - Describing the
performance impacts to security patches for CVE-2017-5754 CVE-2017-5753 and
CVE-2017-5715"
https://access.redhat.com/articles/3307751

- Revised worst-case peformance impact: 4-8%
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] SEC: Spectre variant 2: GCC: -mindirect-branch=thunk -mindirect-branch-register

[Wes Turner]

On Monday, September 17, 2018, Steve Dower  wrote:

> I investigated this thoroughly some time ago (when the MSVC flags became
> available) and determined (with the help of some of the original
> Spectre/Meltdown investigation team) that there is no significant value in
> enabling these flags for Python.


What did you fuzz with?
Does that assume that e.g. Fortify has identified all bugs in CPython C?
There have been a number of variants that have been disclosed; which did
who test for?


>
> It boiled down to:
> * Python allows arbitrary code execution by design


Yet binaries built with GCC do have NX? Unless nested functions in C
extensions?


> * Pure Python code in CPython has very long per-instruction opcode
> sequences that cannot easily be abused or timed


A demonstration of this would be helpful.

* Injected pure Python code cannot be coerced into generating native code
> that is able to abuse Spectre/Meltdown but not able to abuse other attacks
> more easily


 So, not impossible.

* Code injection itself is outside of this particular threat model


[Jupyter] Notebook servers are as wide open to arbitrary code execution as
browser JS JITs; often with VMs and/or containers as a 'sandbox'

`pip install requirements.txt` installs and executes unsigned code: Python,
C extensions

What can a container do to contain a speculative execution exploit
intending to escape said container?

I thought I read that RH has a kernel flag for userspace?


> By comparison with JavaScript, most JS JITs can be easily coerced into
> generating specific native code that can break sandbox guarantees (e.g.
> browser tabs). Python offers none of these guarantees.


This is faulty logic. Because Python does not have a JIT sandbox,
speculative execution is not a factor for Python?


>
> Distributors are of course free to enable these flags for their own
> builds, but I recommend against it for the official binaries, and would
> suggest that it's worth more PR than actual security and nobody else needs
> to enable it either.
>
> (Extension authors with significant scriptable C code need to perform
> their own analysis. I'm only talking about CPython here.)


Extension installers (and authors) are not likely to perform any such
analysis.

Extensions are composed of arbitrary C, which certainly can both directly
exploit and indirectly enable remote exploitation of Spectre and Meltdown
vulnerabilities.

Most users of python are installing arbitrary packages (without hashes or
signatures).


>
> Cheers,
> Steve
>
> On 16Sep2018 0707, Wes Turner wrote:
>
>> Should Python builds add `-mindirect-branch=thunk
>> -mindirect-branch-register` to CFLAGS?
>>
>> Where would this be to be added in the build scripts with which
>> architectures?
>>
>> /QSpectre is the MSVC build flag for Spectre Variant 1:
>>
>>  > The /Qspectre option is available in Visual Studio 2017 version 15.7
>> and later.
>>
>> https://docs.microsoft.com/en-us/cpp/build/reference/qspectr
>> e?view=vs-2017
>>
>> security@ directed me to the issue tracker / lists,
>> so I'm forwarding this to python-dev and python-ideas, as well.
>>
>> # Forwarded message
>> From: *Wes Turner* mailto:wes.tur...@gmail.com>>
>> Date: Wednesday, September 12, 2018
>> Subject: SEC: Spectre variant 2: GCC: -mindirect-branch=thunk
>> -mindirect-branch-register
>> To: distutils-sig mailto:distutils-sig@python.
>> org>>
>>
>>
>> Should C extensions that compile all add
>> `-mindirect-branch=thunk -mindirect-branch-register` [1] to mitigate the
>> risk of Spectre variant 2 (which does indeed affect user space applications
>> as well as kernels)?
>>
>> [1] https://github.com/speed47/spectre-meltdown-checker/issues/
>> 119#issuecomment-361432244 <https://github.com/speed47/sp
>> ectre-meltdown-checker/issues/119#issuecomment-361432244>
>> [2] https://en.wikipedia.org/wiki/Spectre_(security_vulnerability) <
>> https://en.wikipedia.org/wiki/Spectre_%28security_vulnerability%29>
>> [3] https://en.wikipedia.org/wiki/Speculative_Store_Bypass#Specu
>> lative_execution_exploit_variants <https://en.wikipedia.org/wiki
>> /Speculative_Store_Bypass#Speculative_execution_exploit_variants>
>>
>> On Wednesday, September 12, 2018, Wes Turner > <mailto:wes.tur...@gmail.com>> wrote:
>>
>> On Wednesday, September 12, 2018, Joni Orponen
>> mailto:j.orpo...@4teamwork.ch>> wrote:
>>
>> On Wed, Sep 12, 2018 at 8:48 PM Wes Turner
>> mailto:wes.tur...@gmail.com>> wrote:
>>
>> Should C extensions that compile all add
>>

Re: [Python-Dev] SEC: Spectre variant 2: GCC: -mindirect-branch=thunk -mindirect-branch-register

[Wes Turner]

Are all current Python builds and C extensions vulnerable to Spectre
variants {1, 2, *}?

There are now multiple threads:

"SEC: Spectre variant 2: GCC: -mindirect-branch=thunk
-mindirect-branch-register"
-
https://mail.python.org/mm3/archives/list/distutils-...@python.org/thread/4BGE226DB5EWIAT5VCJ75QD5ASOVJZCM/
- https://mail.python.org/pipermail/python-ideas/2018-September/053473.html
- https://mail.python.org/pipermail/python-dev/2018-September/155199.html


Original thread (that I forwarded to security@):
"[Python-ideas] Executable space protection: NX bit,"
https://mail.python.org/pipermail/python-ideas/2018-September/053175.html
> ~ Do trampolines / nested functions in C extensions switch off the NX bit?

On Sunday, September 16, 2018, Nathaniel Smith  wrote:

> On Wed, Sep 12, 2018, 12:29 Joni Orponen  wrote:
>
>> On Wed, Sep 12, 2018 at 8:48 PM Wes Turner  wrote:
>>
>>> Should C extensions that compile all add
>>> `-mindirect-branch=thunk -mindirect-branch-register` [1] to mitigate the
>>> risk of Spectre variant 2 (which does indeed affect user space applications
>>> as well as kernels)?
>>>
>>
>> Are those available on GCC <= 4.2.0 as per PEP 513?
>>
>
> Pretty sure no manylinux1 compiler is ever going to get these mitigations.
>
> For manylinux2010 on x86-64, we can easily use a much newer compiler: RH
> maintains a recent compiler, currently gcc 7.3, or if that doesn't work for
> some reason then the conda folks have be apparently figured out how to
> build the equivalent from gcc upstream releases.
>

Are there different CFLAGS and/or gcc compatibility flags in conda builds
of Python and C extensions?

Where are those set in conda builds?

What's the best way to set CFLAGS in Python builds and C extensions?

export CFLAGS="-mindirect-branch=thunk -mindirect-branch-register"
./configure
make

?

Why are we supposed to use an old version of GCC that doesn't have the
retpoline patches that only mitigate Spectre variant 2?


>
> Unfortunately, the manylinux2010 infrastructure is not quite ready... I'm
> pretty sure it needs some volunteers to push it to the finish line, though
> unfortunately I haven't had enough time to keep track.
>

"PEP 571 -- The manylinux2010 Platform Tag"
https://www.python.org/dev/peps/pep-0571/

"Tracking issue for manylinux2010 rollout"
https://github.com/pypa/manylinux/issues/179

Are all current Python builds and C extensions vulnerable to Spectre
variants {1, 2, *}?

>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] SEC: Spectre variant 2: GCC: -mindirect-branch=thunk -mindirect-branch-register

[Wes Turner]

On Sunday, September 16, 2018, Wes Turner  wrote:

> Should Python builds add `-mindirect-branch=thunk
> -mindirect-branch-register` to CFLAGS?
>
> Where would this be to be added in the build scripts with which
> architectures?
>
> /QSpectre is the MSVC build flag for Spectre Variant 1:
>
> > The /Qspectre option is available in Visual Studio 2017 version 15.7 and
> later.
>
> https://docs.microsoft.com/en-us/cpp/build/reference/qspectre?view=vs-2017
>
> security@ directed me to the issue tracker / lists,
> so I'm forwarding this to python-dev and python-ideas, as well.
>
> # Forwarded message
> From: *Wes Turner* 
> Date: Wednesday, September 12, 2018
> Subject: SEC: Spectre variant 2: GCC: -mindirect-branch=thunk
> -mindirect-branch-register
> To: distutils-sig 
>
>
> Should C extensions that compile all add
> `-mindirect-branch=thunk -mindirect-branch-register` [1] to mitigate the
> risk of Spectre variant 2 (which does indeed affect user space applications
> as well as kernels)?
>
> [1] https://github.com/speed47/spectre-meltdown-checker/issues/
> 119#issuecomment-361432244
> [2] https://en.wikipedia.org/wiki/Spectre_(security_vulnerability)
> [3] https://en.wikipedia.org/wiki/Speculative_Store_Bypass#Specu
> lative_execution_exploit_variants
>
> On Wednesday, September 12, 2018, Wes Turner  wrote:
>>
>>> On Wednesday, September 12, 2018, Joni Orponen 
>>> wrote:
>>>
>>>> On Wed, Sep 12, 2018 at 8:48 PM Wes Turner 
>>>> wrote:
>>>>
>>>>> Should C extensions that compile all add
>>>>> `-mindirect-branch=thunk -mindirect-branch-register` [1] to mitigate
>>>>> the risk of Spectre variant 2 (which does indeed affect user space
>>>>> applications as well as kernels)?
>>>>>
>>>>
>>>> Are those available on GCC <= 4.2.0 as per PEP 513?
>>>>
>>>
>>> AFAIU, only
>>> GCC 7.3 and 8 have the retpoline (indirect-branch=thunk) support enabled
>>> by the `-mindirect-branch=thunk -mindirect-branch-register` CFLAGS.
>>>
>>
>  On Wednesday, September 12, 2018, Wes Turner 
> wrote:
>
>> "What is a retpoline and how does it work?"
>> https://stackoverflow.com/questions/48089426/what-is-a-retpo
>> line-and-how-does-it-work
>>
>>
There's probably already been an ANN announce about this?

If not, someone with appropriate security posture and syntax could address:

Whether python.org binaries are already rebuilt

Whether OS package binaries are already rebuilt

Whether anaconda binaries are already rebuilt

Whether C extension binaries on pypi are already rebuilt
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] SEC: Spectre variant 2: GCC: -mindirect-branch=thunk -mindirect-branch-register

[Wes Turner]

Should Python builds add `-mindirect-branch=thunk
-mindirect-branch-register` to CFLAGS?

Where would this be to be added in the build scripts with which
architectures?

/QSpectre is the MSVC build flag for Spectre Variant 1:

> The /Qspectre option is available in Visual Studio 2017 version 15.7 and
later.

https://docs.microsoft.com/en-us/cpp/build/reference/qspectre?view=vs-2017

security@ directed me to the issue tracker / lists,
so I'm forwarding this to python-dev and python-ideas, as well.

# Forwarded message
From: *Wes Turner* 
Date: Wednesday, September 12, 2018
Subject: SEC: Spectre variant 2: GCC: -mindirect-branch=thunk
-mindirect-branch-register
To: distutils-sig 


Should C extensions that compile all add
`-mindirect-branch=thunk -mindirect-branch-register` [1] to mitigate the
risk of Spectre variant 2 (which does indeed affect user space applications
as well as kernels)?

[1] https://github.com/speed47/spectre-meltdown-checker/
issues/119#issuecomment-361432244
[2] https://en.wikipedia.org/wiki/Spectre_(security_vulnerability)
[3] https://en.wikipedia.org/wiki/Speculative_Store_Bypass#
Speculative_execution_exploit_variants

On Wednesday, September 12, 2018, Wes Turner  wrote:
>
>> On Wednesday, September 12, 2018, Joni Orponen 
>> wrote:
>>
>>> On Wed, Sep 12, 2018 at 8:48 PM Wes Turner  wrote:
>>>
>>>> Should C extensions that compile all add
>>>> `-mindirect-branch=thunk -mindirect-branch-register` [1] to mitigate
>>>> the risk of Spectre variant 2 (which does indeed affect user space
>>>> applications as well as kernels)?
>>>>
>>>
>>> Are those available on GCC <= 4.2.0 as per PEP 513?
>>>
>>
>> AFAIU, only
>> GCC 7.3 and 8 have the retpoline (indirect-branch=thunk) support enabled
>> by the `-mindirect-branch=thunk -mindirect-branch-register` CFLAGS.
>>
>
 On Wednesday, September 12, 2018, Wes Turner  wrote:

> "What is a retpoline and how does it work?"
> https://stackoverflow.com/questions/48089426/what-is-a-
> retpoline-and-how-does-it-work
>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Official citation for Python

[Wes Turner]

On Saturday, September 15, 2018, Jacqueline Kazil 
wrote:

> I just got caught up on the thread. This is a really great discussion.
> Thank you for all the contributions.
>
> Before we get into the details, let's go back to the main use case we are
> trying to solve.
> *As a user, I am writing an academic paper and I need to cite Python. *
>
> Let's throw reproducibility out the window for now (<--- something I never
> thought I would say), because that should be captured in the code, not in
> the citations.
>
> So, if we don't need the specific version of Python, then maybe creating
> one citation is all we need.
> And that gives it some good Google juice as well.
>

https://scholar.google.com/scholar?hl=en=python+van+Rossum *

https://www.semanticscholar.org/search?q=Python%20van%20Rossum

https://www.mendeley.com/research-papers/?query=Python+van+Rossum

https://www.zotero.org/search/q/Python/type/group

With an e.g. {Zotero,} group, it would be easy to cite the Python citation
with the greatest centrality.
https://networkx.github.io/documentation/stable/reference/algorithms/centrality.html

A DOI URN/URI/URL really is easiest to aggregate the edges of/for.

- [ ] Link to the new citation(s) page in the Python docs from the SciPy
citing page
https://www.scipy.org/citing.html

NP. YW!


> Thoughts?
>
> (Once we nail down one or many, I think we can then move into the details
> of the content of the citation.)
>
> -Jackie
>
> On Thu, Sep 13, 2018 at 12:47 AM Wes Turner  wrote:
>
>> There was a thread about adding __cite__ to things and a tool to collect
>> those citations awhile back.
>>
>> "[Python-ideas] Add a __cite__ method for scientific packages"
>> http://markmail.org/thread/rekmbmh64qxwcind
>>
>> Which CPython source file should contain this __cite__ value?
>>
>> ... On a related note, you should ask the list admin to append a URL to
>> each mailing list message whenever this list is upgraded to mm3; so that
>> you can all be appropriately cited.
>>
>> On Thursday, September 13, 2018, Wes Turner  wrote:
>>
>>> Do you guys think we should all cite Grub and BusyBox and bash and libc
>>> and setuptools and pip and openssl and GNU/Linux and LXC and Docker; or
>>> else it's plagiarism for us all?
>>>
>>> #OpenAccess
>>>
>>> On Wednesday, September 12, 2018, Stephen J. Turnbull <
>>> turnbull.stephen...@u.tsukuba.ac.jp> wrote:
>>>
>>>> Chris Barker via Python-Dev writes:
>>>>
>>>>  > But "I wrote some code in Python to produce these statistics" --
>>>>  > does that need a citation?
>>>>
>>>> That depends on what you mean by "statistics" and whether (as one
>>>> should) one makes the code available.  If the code is published or
>>>> "available on request", definitely, Python should be cited.  If not,
>>>> and by "statistics" you mean the kind of things provided by Steven
>>>> d'Aprano's excellent statistics module (mean, median, standard
>>>> deviation, etc), maybe no citation is needed.  But anything more
>>>> esoteric than that (even linear regression), yeah, I would say you
>>>> should cite both Python and any reference you used to learn the
>>>> algorithm or formulas, in the context of mentioning that your
>>>> statistics are home-brew, not produced by one of the recognized
>>>> applications for doing so.
>>>>
>>>>  > If so, maybe that would take a different form.
>>>>
>>>> Yes, it would.  But not so different: eg, version is analogous to
>>>> edition when citing a book.
>>>>
>>>>  > Anyway, hard to make this decision without some idea how the
>>>>  > citation is intended to be used.
>>>>
>>>> Same as any other citation, (1) to give credit to those responsible
>>>> for providing a resource (this is why publishers and their metadata of
>>>> city are still conventionally included), and (2) to show where that
>>>> resource can be obtained.  AFAICS, both motivations are universally
>>>> applicable in polite society.  NB: Replication is an important reason
>>>> for wanting to acquire the resource, but it's not the only one.
>>>>
>>>> I think underlying your comment is the question of *what* resource is
>>>> being cited.  I can think of three offhand that might be characterized
>>>> as "Python".  First, the PSF, as a provider of funding.  There is a
>>>> co

Re: [Python-Dev] Official citation for Python

[Wes Turner]

There was a thread about adding __cite__ to things and a tool to collect
those citations awhile back.

"[Python-ideas] Add a __cite__ method for scientific packages"
http://markmail.org/thread/rekmbmh64qxwcind

Which CPython source file should contain this __cite__ value?

... On a related note, you should ask the list admin to append a URL to
each mailing list message whenever this list is upgraded to mm3; so that
you can all be appropriately cited.

On Thursday, September 13, 2018, Wes Turner  wrote:

> Do you guys think we should all cite Grub and BusyBox and bash and libc
> and setuptools and pip and openssl and GNU/Linux and LXC and Docker; or
> else it's plagiarism for us all?
>
> #OpenAccess
>
> On Wednesday, September 12, 2018, Stephen J. Turnbull <
> turnbull.stephen...@u.tsukuba.ac.jp> wrote:
>
>> Chris Barker via Python-Dev writes:
>>
>>  > But "I wrote some code in Python to produce these statistics" --
>>  > does that need a citation?
>>
>> That depends on what you mean by "statistics" and whether (as one
>> should) one makes the code available.  If the code is published or
>> "available on request", definitely, Python should be cited.  If not,
>> and by "statistics" you mean the kind of things provided by Steven
>> d'Aprano's excellent statistics module (mean, median, standard
>> deviation, etc), maybe no citation is needed.  But anything more
>> esoteric than that (even linear regression), yeah, I would say you
>> should cite both Python and any reference you used to learn the
>> algorithm or formulas, in the context of mentioning that your
>> statistics are home-brew, not produced by one of the recognized
>> applications for doing so.
>>
>>  > If so, maybe that would take a different form.
>>
>> Yes, it would.  But not so different: eg, version is analogous to
>> edition when citing a book.
>>
>>  > Anyway, hard to make this decision without some idea how the
>>  > citation is intended to be used.
>>
>> Same as any other citation, (1) to give credit to those responsible
>> for providing a resource (this is why publishers and their metadata of
>> city are still conventionally included), and (2) to show where that
>> resource can be obtained.  AFAICS, both motivations are universally
>> applicable in polite society.  NB: Replication is an important reason
>> for wanting to acquire the resource, but it's not the only one.
>>
>> I think underlying your comment is the question of *what* resource is
>> being cited.  I can think of three offhand that might be characterized
>> as "Python".  First, the PSF, as a provider of funding.  There is a
>> conventional form for this: a footnote on the title or author's name
>> saying "The author acknowledges [a] 
>> grant [grant identifier if available] from the Python Software
>> Foundation."  I usually orally mention them in presentations, too.
>> That one's easy; *everybody* should *always* do that.
>>
>> The rest of these, sort of an ideal to strive for.  If you keep a
>> bibliographic database, and there are now quite a few efforts to crowd
>> source them, it's easier to go the whole 9 yards than to skimp.  But
>> except in cases where we don't need to even mention the code, probably
>> we should be citing, for reasons of courtesy to readers as well as
>> authors, editors, and publishers (as disgusting as many publishers are
>> as members of society, they do play a role in providing many resources
>> ---we should find ways to compete them into good behavior, not
>> ostracize them).
>>
>> The second is the Python *language and standard library*.  Then the
>> Language Reference and/or the Library Reference should be cited
>> briefly when Python is first mentioned, and in the text introducing a
>> program or program fragment, with a full citation in the bibliography.
>> I tentatively suggest that the metadata for the Language Reference
>> would be
>>
>> Author: principal author(s) (Guido?) et al. OR python.org OR
>> Python Contributors
>> Title: The Python Language Reference
>> Version: to match Python version used (if relevant, different
>> versions each get full citations), probably should not be
>> "current"
>> Publisher: Python Software Foundation
>> Date: of the relevant version
>> Location: City of legal address of PSF
>> URL: to version used (probably should not be the default)
>> Date accessed: if "current" was used
>>
>> The Librar

Re: [Python-Dev] Official citation for Python

[Wes Turner]

Do you guys think we should all cite Grub and BusyBox and bash and libc and
setuptools and pip and openssl and GNU/Linux and LXC and Docker; or else
it's plagiarism for us all?

#OpenAccess

On Wednesday, September 12, 2018, Stephen J. Turnbull <
turnbull.stephen...@u.tsukuba.ac.jp> wrote:

> Chris Barker via Python-Dev writes:
>
>  > But "I wrote some code in Python to produce these statistics" --
>  > does that need a citation?
>
> That depends on what you mean by "statistics" and whether (as one
> should) one makes the code available.  If the code is published or
> "available on request", definitely, Python should be cited.  If not,
> and by "statistics" you mean the kind of things provided by Steven
> d'Aprano's excellent statistics module (mean, median, standard
> deviation, etc), maybe no citation is needed.  But anything more
> esoteric than that (even linear regression), yeah, I would say you
> should cite both Python and any reference you used to learn the
> algorithm or formulas, in the context of mentioning that your
> statistics are home-brew, not produced by one of the recognized
> applications for doing so.
>
>  > If so, maybe that would take a different form.
>
> Yes, it would.  But not so different: eg, version is analogous to
> edition when citing a book.
>
>  > Anyway, hard to make this decision without some idea how the
>  > citation is intended to be used.
>
> Same as any other citation, (1) to give credit to those responsible
> for providing a resource (this is why publishers and their metadata of
> city are still conventionally included), and (2) to show where that
> resource can be obtained.  AFAICS, both motivations are universally
> applicable in polite society.  NB: Replication is an important reason
> for wanting to acquire the resource, but it's not the only one.
>
> I think underlying your comment is the question of *what* resource is
> being cited.  I can think of three offhand that might be characterized
> as "Python".  First, the PSF, as a provider of funding.  There is a
> conventional form for this: a footnote on the title or author's name
> saying "The author acknowledges [a] 
> grant [grant identifier if available] from the Python Software
> Foundation."  I usually orally mention them in presentations, too.
> That one's easy; *everybody* should *always* do that.
>
> The rest of these, sort of an ideal to strive for.  If you keep a
> bibliographic database, and there are now quite a few efforts to crowd
> source them, it's easier to go the whole 9 yards than to skimp.  But
> except in cases where we don't need to even mention the code, probably
> we should be citing, for reasons of courtesy to readers as well as
> authors, editors, and publishers (as disgusting as many publishers are
> as members of society, they do play a role in providing many resources
> ---we should find ways to compete them into good behavior, not
> ostracize them).
>
> The second is the Python *language and standard library*.  Then the
> Language Reference and/or the Library Reference should be cited
> briefly when Python is first mentioned, and in the text introducing a
> program or program fragment, with a full citation in the bibliography.
> I tentatively suggest that the metadata for the Language Reference
> would be
>
> Author: principal author(s) (Guido?) et al. OR python.org OR
> Python Contributors
> Title: The Python Language Reference
> Version: to match Python version used (if relevant, different
> versions each get full citations), probably should not be
> "current"
> Publisher: Python Software Foundation
> Date: of the relevant version
> Location: City of legal address of PSF
> URL: to version used (probably should not be the default)
> Date accessed: if "current" was used
>
> The Library reference would be the same except for Title.
>
> The third is a *particular implementation*.  In that case the metadata
> would be
>
> Author: principal author(s) (Guido) et al. OR python.org OR
> Python Contributors
> Title: The cPython Python distribution
> Python Version: as appropriate (if relevant, different versions each
> get full citations), never "current"
> Distributor Version: if different from Python version (eg, additional
> Debian cruft)
> Publisher: Distributor (eg, PSF, Debian Project, Anaconda Inc.)
> Date: of the relevant version
> Location: City of legal address of distributor
>
> If downloaded:
>
> URL: to version used (including git commit SHA1 if available)
> Date accessed: download from distributor, not installation date
>
> If received on physical medium: use the "usual" form of citation for a
> collection of individual works (even if Python was the only thing on
> it).  Probably the only additional information needed would be the
> distributor as editor of the collection and the name of the
> collection.
>
> In most cases I can think of, if the implementation is 

Re: [Python-Dev] Official citation for Python

[Wes Turner]

On Sunday, September 9, 2018, Wes Turner  wrote:

> "Python Programming Language" (van Rossum, et. Al) 
>
> ?
>
> Should there be a URL and/or a DOI?
>

http://www.python.org/~guido/Publications.html
https://gvanrossum.github.io/Publications.html
lists a number of Python citations:

"""
Python

Guido van Rossum: Scripting the Web with Python. In "Scripting Languages:
Automating the Web", World Wide Web Journal, Volume 2, Issue 2, Spring
1997, O'Reilly.

Aaron Watters, Guido van Rossum, James C. Ahlstrom: Internet Programming
with Python. MIS Press/Henry Holt publishers, New York, 1996.

Guido van Rossum: Python Library Reference. May 1995. CWI Report CS-R9524.

Guido van Rossum: Python Reference Manual. May 1995. CWI Report CS-R9525.

Guido van Rossum: Python Tutorial. May 1995. CWI Report CS-R9526.

Guido van Rossum: Extending and Embedding the Python Interpreter. May 1995.
CWI Report CS-R9527.

Guido van Rossum, Jelke de Boer: Linking a Stub Generator (AIL) to a
Prototyping Language (Python). Spring 1991 EurOpen Conference Proceedings
(May 20-24, 1991) Tromso, Norway.
"""


https://en.wikipedia.org/wiki/History_of_Python#Version_release_dates cites
http://python-history.blogspot.com/2009/01/brief-timeline-of-python.html as
a reference (for Python versions up to 2.6 and 3.0):

- 0.9 - 1991
- 1.0 - 1994
- 2.0 - 2000
- 3.0 - 2008
  - 3.7 - 2018


Maybe it would be most productive for us to discuss the fields in the
proposed citation?

~"PSF is the author"


@article{python,
 title={{P}ython ...},
 author={Van Rossum, G. and Python Software Foundation (PSF), The.} ,
 journal={ },
 volume={ },
 pages={ },
 year={ }
}
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Official citation for Python

[Wes Turner]

I also see reproducibility and citation graphs as distinct concepts.

If it's reproducibility you're after, bibliographic citations are very
unlikely to enable someone else to assemble an identical build environment
from which the same conclusion should be repeatably derivable.

A ScholarlyArticle can be reproducible with no citations whatsoever.
A ScholarlyArticle may very likely have many citations and still be
woefully unreproducible.

This citation doesn't contain a URL, but still isn't quite useless (while
the paper is excellent); because there's at least a DOI string:

Sandve GK, Nekrutenko A, Taylor J, Hovig E (2013) Ten Simple Rules for
Reproducible Computational Research. PLoS Comput Biol 9(10): e1003285.
doi:10.1371/journal.pcbi.1003285

> Rule 3: Archive the Exact Versions of All External Programs Used

mybinder.org builds Jupyter containers from git repositories that contain
config files with repo2docker.

https://repo2docker.readthedocs.io/en/latest/config_files.html#configuration-files
"""
Dockerfile
environment.yml
requirements.txt
REQUIRE
install.R
apt.txt
setup.py
postBuild
runtime.txt
"""

Specifying the exact version of Python (and what package it was installed
from and/or what URL the source was obtained and built from) is no
substitute for hashes of the 'pinned' versions of said artifacts.

# includes the python version
$ conda env export -f environment.yml

# these do not include the python version
$ pip freeze -r requirements.txt --all
$ pipenv lock # > Pipfile.lock
$ pipenv sync # < Pipfile.lock

Uploading a built container or VM image to e.g. Docker Hub / GitLab
Container Registry / Vagrant Cloud is another way to ensure that research
findings are reproducible.
- Dockerfile, docker-compose.yml
- Vagrantfile

> Rule 4: Version Control All Custom Scripts

https://mozillascience.github.io/code-research-object/ (FigShare + GitHub
=> DOI citation URI)

https://guides.github.com/activities/citable-code/ (Zenodo + GitHub => DOI
citation URI)

...

Is it necessary to cite Python (or all packages) if you're not building a
derivative of Python or said packages?

It's definitely a good idea to "Archive the Exact Versions of All External
Programs Used"; but IDK that those are best represented with bibliographic
citations. Really, a link to the Homepage, Source, Docs, and Wikipedia page
are probably more helpful to a reviewer that's not familiar with and wants
to help support by linking dereferenceable URLs and https://5stardata.info.

While out of scope and OT, it's worth mentioning that search engines index
https://schema.org/Dataset metadata; which is helpful for data reuse and
autodiscovering requisite premises for the argument presented in a
https://schema.org/ScholarlyArticle .

A citation for each MAJ.MIN.PATCH revision of CPython (and/or other
excellent packages) might be a bit much.

On Monday, September 10, 2018, Steven D'Aprano  wrote:

> On Mon, Sep 10, 2018 at 09:25:29PM +0200, Chris Barker via Python-Dev
> wrote:
> > I"d like ot know what thee citations are expected to be used for?
> >
> > i.e. -- usually, academic papers have a collection of citiations to
> > acknowledge where you got an idea, or fact, or  It serves both to
> > jusstify something and make it clear that it is not your own idea (i.e.
> not
> > pagerism).
>
> [
> > That is about reproducible results, which is really a different thing
> than
> > the usual citations.
>
> I don't think it is. I think you are seeing a distinction that is not
> there. If citations were just about acknowledgement, we could say "I got
> this idea from Bob" and be done with it. Citations are about identifying
> the *exact* source so that anyone can reproduce the given ideas by
> checking not just "Bob" but the specific page number of a specific
> edition of a specific work.
>
> So the requirement for precision is no different between papers and
> software, and the academic standards for citing software already take
> that into account. There are challenges with software, to be sure --
> code is much more ephemeral, there may be literally hundreds of
> authors, etc. But in principle, the kinds of information needed to
> cite a software package is known. The major citation styles already
> include this. When you are using a specific style, this page:
>
> https://openresearchsoftware.metajnl.com/about/
>
> suggests a few formats, depending on how you got access to the software.
>
> The bottom line is, we don't have to guess what information to provide.
> People like Jacqueline can tell us what they need, and we'll just fill
> in the values.
>
> The people citing Python know what information they need, we just have
> to help them get it. I think that the best way to do that is to provide
> the correct information in a single place, in a single, standard format,
> and let them choose the appropriate citation style for their
> publication.
>
> Jackie, do I have that right?
>
>
>
> --
> Steve
> ___

Re: [Python-Dev] Official citation for Python

[Wes Turner]

"Python Programming Language" (van Rossum, et. Al) 

?

Should there be a URL and/or a DOI?

Figshare and Zenodo will archive a [e.g. tagged] [GitHub] revision and
generate a DOI, AFAIU


On Sunday, September 9, 2018, Steven D'Aprano  wrote:

> On Sun, Sep 09, 2018 at 03:43:13PM -0400, Jacqueline Kazil wrote:
> > The PSF has received a few inquiries asking the question — “How do I cite
> > Python?”So, I am reaching out to you all to figure this out.
>
> If you figure it out, it would be lovely to see some movement on this
> ticket:
>
> https://bugs.python.org/issue26597
>
>
> --
> Steve
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> wes.turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The history of PyXML

[Wes Turner]

On Thursday, May 17, 2018, Serhiy Storchaka  wrote:

> [...]
>
> I'm trying to figure out some intentions and fix possible bugs in the xml
> package.


defusedxml
https://pypi.org/project/defusedxml/

> XML bomb protection for Python stdlib modules

https://pypi.org/project/defusedxml/#how-to-avoid-xml-vulnerabilities

"""
Best practices
- Don’t allow DTDs
- Don’t expand entities
- Don’t resolve externals
- Limit parse depth
- Limit total input size
- Limit parse time
- Favor a SAX or iterparse-like parser for potential large data
- Validate and properly quote arguments to XSL transformations and XPath
queries
- Don’t use XPath expression from untrusted sources
- Don’t apply XSL transformations that come untrusted sources
"""

https://github.com/tiran/defusedxml


> The history of all commits could help.
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/wes.
> turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] (Looking for) A Retrospective on the Move to Python 3

[Wes Turner]

On Monday, May 14, 2018, Ethan Furman <et...@stoneleaf.us> wrote:

> On 05/14/2018 09:34 AM, Chris Barker via Python-Dev wrote:
>
> between 3.0 and 3.6 (.5?) -- py3 grew a lot of minor features that made it
>> easier to write py2/py3 compatible code.
>> u"string", b'bytes %i' % something -- and when where the various
>> __future__ imports made available?
>>
>> If these had been in place in 3.0, the whole process would have been
>> easier :-(
>>
>
> You'll need to be more specific.  __future__ has been around for a
> long time.


https://github.com/python/cpython/blame/master/Lib/__future__.py


>
> --
> ~Ethan~
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/wes.
> turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Drop/deprecate Tkinter?

[Wes Turner]

On Thursday, May 3, 2018, Steven D'Aprano  wrote:

> On Thu, May 03, 2018 at 07:39:05AM +0200, Matěj Cepl wrote:
>
> > I have my doubts about IDLE though. I know, the same
> > argument applies, but really, does anybody use IDLE for
> > development for long time
>
> Yes, tons of beginners use it. On the tutor and python-list mailing
> lists, there are plenty of questions from people using IDLE.


Turtle is built with Tkinter:
https://docs.python.org/3/library/turtle.html

IIRC, I used IDLE when first learning Python.
Dive Into Python 3 recommends IDLE.
http://www.diveintopython3.net/installing-python.html#idle

Hopefully this bug is fixed by someone.

PyQt and PySide:
- are more accessible to screen readers
- support QThread, QThreadPool, QRunnable
- quamash is an asyncio event loop with/for Qt
- PyQtGraph



>
> --
> Steve
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> wes.turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Every Release Can Be a Mini "Python 4000", Within Reason (was (name := expression) doesn't fit the narrative of PEP 20)

[Wes Turner]

On Sunday, April 29, 2018, Eitan Adler  wrote:

> On 29 April 2018 at 01:34, Jeff Allen  wrote:
> > On 27/04/2018 08:38, Greg Ewing wrote:
>
> > I speculate this all goes back to some pre-iteration version of FORmula
> > TRANslation, where to its inventors '=' was definition and these really
> were
> > "statements" in the normal sense of stating a truth.
>
> https://www.hillelwayne.com/post/equals-as-assignment/


https://en.wikipedia.org/wiki/Assignment_(computer_science)

C and C++ are '=' and '=='.

The Sympy solver, for example, solves Eq(lhs, rhs) equations and
expressions that are assumed to be equal to zero.
http://docs.sympy.org/latest/tutorial/solvers.html

The sage solver defines __eq__ (==) so expressions with variables produce
symbolic equations and inequalities (relations).
http://doc.sagemath.org/html/en/reference/calculus/sage/symbolic/relation.html

PyDatalog defines __eq__ so that expressions with variables produce logic
queries.
https://sites.google.com/site/pydatalog/Online-datalog-tutorial

The assignment Wikipedia article lists languages other than C and C++ which
chose = and == for assignment and definable equality testing.

https://en.wikipedia.org/wiki/Equality_(mathematics)
  https://en.wikipedia.org/wiki/Extensionality
https://en.wikipedia.org/wiki/Logical_equality



>
>
> --
> Eitan Adler
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> wes.turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20

[Wes Turner]

It's certainly a contrived example. Actual code with such a mistake is
generally far more subtle.

The mistake is that it's assigning a value within a clause of a conditional
that won't be evaluated.

Oh well. I'll suffer the then worsened zig-zaggy eye movements in code
reviews caused by defining values at the end of expressions that reference
them which fit on a single line.

There are a number of bad examples for style guides in these threads.

:=

I wasn't aware of this switch, thanks!
http://coverage.readthedocs.io/en/latest/branch.html
coverage run --branch code.py



On Friday, April 27, 2018, Tim Peters <tim.pet...@gmail.com> wrote:

> Wes, sorry, but I really don't follow what you're saying.  For example,
>
> [Wes Turner <wes.tur...@gmail.com>]
> > Do not do this:
> >
> > x = 2
> > if (x == 3) or (x := 3):
> >print(x)
> >
> > What do we call that mistake?
>
> It displays 3 - while it appears to be silly code, there's nothing
> about it that's undefined.  So I fail to see how showing that example
> anywhere would do anyone any good.
>
> You can do the same kind of thing today via, e.g.,
>
> class Bindable:
> def __init__(self, value):
> self.bind(value)
>
> def bind(self, value):
> self.value = value
> return value
>
> def __bool__(self):
> return bool(self.value)
>
> def __eq__(self, other):
> return self.value == other
>
> def __str__(self):
> return str(self.value)
>
> Then:
>
> >>> x = Bindable(2)
> >>> if x == 3 or x.bind(3):
> ... print(x)
> 3
>
> And I wouldn't put that example anywhere in any docs either ;-)
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20

[Wes Turner]

On Friday, April 27, 2018, Tim Peters  wrote:

> [Chris Angelico ]
> > ...
> > I don't understand why people bring up all these arguments that have
> > absolutely nothing to do with the proposal at hand. None of this has
> > in any way changed.
>
> That's easy:  any time there's a long thread to which Guido has
> contributed at least twice, it will be seen as a Golden Opportunity to
> re-litigate every decision that's ever been made ;-)
>
> Some amount of that seems healthy to me (people are thinking about
> "language design" from a larger view than the proposal du jour).  In
> this specific case, line-oriented coverage tools have missed
> accounting for all possible code paths since day #1; e.g.,
>
> x = f() or g()
>
> You don't need to reply to messages so obviously irrelevant to the PEP
> unless you want to.  It's not like Guido will read them and go "oh!  a
> binding expression in a ternary conditional is a fundamentally new
> potential problem for a line-oriented coverage tool!  that's fatal"
> ;-)


I have shared with you the overlapping concerns about this feature proposal
that I believe should be explained with DO and DON'T in the docs and/or the
PEP and/or the style guide(s) for various organizations in the Pyrhon
community.

This feature does require additions to the style-guide(s); which is why so
many have expressed concern about such a simple thing.

If you want to write debuggable and coverage-testable code, do not use the
assignment expression operator in ternary expressions or boolean-chained
expressions.

The assignment expression operator is the only way to define variables with
only comprehension scope.

Do not do this:

x = 2
if (x == 3) or (x := 3):
   print(x)

What do we call that mistake?


> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> wes.turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20

[Wes Turner]

On Friday, April 27, 2018, Chris Angelico <ros...@gmail.com> wrote:

> On Sat, Apr 28, 2018 at 6:06 AM, Wes Turner <wes.tur...@gmail.com> wrote:
> >
> >
> > On Friday, April 27, 2018, Chris Angelico <ros...@gmail.com> wrote:
> >>
> >> On Fri, Apr 27, 2018 at 8:18 PM, Wes Turner <wes.tur...@gmail.com>
> wrote:
> >> > IDK, I could just be resistant to change, but this seems like
> something
> >> > that
> >> > will decrease readability -- and slow down code review -- without any
> >> > real
> >> > performance gain. So, while this would be useful for golfed-down (!)
> >> > one-liners with pyline,
> >> > I'm -1 on PEP 572.
> >>
> >> PEP 572 has never promised a performance gain, so "without any real
> >> performance gain" is hardly a criticism.
> >>
> >> > How do I step through this simple example with a debugger?
> >> >
> >> > if re.search(pat, text) as match:
> >> > print("Found:", match.group(0))
> >>
> >> Step the 'if' statement. It will call re.search() and stash the result
> >> in 'match'. Then the cursor would be put either on the 'print' (if the
> >> RE matched) or on the next executable line (if it didn't).
> >
> >
> > Right. Pdb doesn't step through the AST branches of a line, so ternary
> > expressions and list comprehensions and defining variables at the end of
> the
> > line are 'debugged' after they're done.
> >
> > Similarly, code coverage is line-based; so those expressions may appear
> to
> > be covered but are not.
>
> I'm not sure I follow. In what situation would some code appear to be
> covered when it isn't, due to an assignment expression?


When an assignment expression is in the else clause of a ternary
expression, but the else clause does not execute because the condition is
true, the assignment expression does not execute and so isn't covered.

if ((1) or (x := 3)):
if ((y := func(x)) if x else (x := 3))

Is this a new opportunity for misunderstanding?

Assignment expressions, though they are noticeable :=, may not actually
define the variable in cases where that part of the line doesn't run but
reads as covered.




>
> ChrisA
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> wes.turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20

[Wes Turner]

On Friday, April 27, 2018, Chris Angelico <ros...@gmail.com> wrote:

> On Fri, Apr 27, 2018 at 8:18 PM, Wes Turner <wes.tur...@gmail.com> wrote:
> > IDK, I could just be resistant to change, but this seems like something
> that
> > will decrease readability -- and slow down code review -- without any
> real
> > performance gain. So, while this would be useful for golfed-down (!)
> > one-liners with pyline,
> > I'm -1 on PEP 572.
>
> PEP 572 has never promised a performance gain, so "without any real
> performance gain" is hardly a criticism.
>
> > How do I step through this simple example with a debugger?
> >
> > if re.search(pat, text) as match:
> > print("Found:", match.group(0))
>
> Step the 'if' statement. It will call re.search() and stash the result
> in 'match'. Then the cursor would be put either on the 'print' (if the
> RE matched) or on the next executable line (if it didn't).


Right. Pdb doesn't step through the AST branches of a line, so ternary
expressions and list comprehensions and defining variables at the end of
the line are 'debugged' after they're done.

Similarly, code coverage is line-based; so those expressions may appear to
be covered but are not.


>
> > From https://en.wikipedia.org/wiki/Assignment_(computer_science) :
> >
> >> In some languages the symbol used is regarded as an operator (meaning
> that
> >> the assignment has a value) while others define the assignment as a
> >> statement (meaning that it cannot be used in an expression).
> >
> >
> > PEP 572 -- Assignment Expressions
> > PEP 572 -- Assignment Operator (:=) and Assignment Expressions
>
> Huh? I don't get your point.


Q: What is ':='? (How do I searchengine for it?)
A: That's the assignment operator which only works in Python 3.8+.

Q: When are variables defined -- or mutable names bound -- at the end of
the expression accessible to the left of where they're defined?
Q: What about tuple unpacking? Is there an ECMAscript-like destructuring
PEP yet?
A: Ternary expressions; list, dict, generator comprehensions;
(@DOCS PLEASE HELP EXPLAIN THIS)

Q: do these examples of the assignment expression operator all work?

"""
- debuggers have no idea what to do with all of this on one line
- left-to-right doesn't apply to comprehensions

  results = [(x, y, x/y) for x in input_data if (y := f(x)) > 0]

- left-to-right doesn't apply to ternary expressions

  if (y := func(x)) if (x := 3) else 0:
  while (y := func(x)) if (x := 3) else 0:

- left-to-right does apply to everything else?

- *these* are discouraged:

  if (x := 3) or (y := func(x)):
  if (3) or (func(3)):

  if ((x := 3) if 1 else (y := func(x))):
"""


> ChrisA
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> wes.turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] (Looking for) A Retrospective on the Move to Python 3

[Wes Turner]

On Thursday, April 26, 2018, Eric Snow  wrote:

> In pondering our approach to future Python major releases, I found
> myself considering the experience we've had with Python 3.  The whole
> Py3k effort predates my involvement in the community so I missed a
> bunch of context about the motivations, decisions, and challenges.
> While I've pieced some of that together over the years now since I've
> been around, I've certainly seen much of the aftermath.  For me, at
> least, it would be helpful to have a bit more insight into the
> history. :)
>
> With that in mind, it would be worth having an informational PEP with
> an authoritative retrospective on the lessons learned from the Python
> 3 effort (and transition).  Consider it a sort of autobiography,
> "memoirs on the python-dev change to Python 3". :)  At this point the
> transition has settled in enough that we should be able to present a
> relatively objective (and consistent) view, while we're not so far
> removed that we've forgotten anything important. :)  If such a
> document already exists then I'd love a pointer to it.
>
> The document would benefit (among others):
>
> * python-dev (by giving us a clear viewpoint to inform decisions about
> future releases)
> * new-comers to Python that want more insight into the language
> * folks transitioning from 2 to 3
> * communities that have (or think they have) problems similar to those
> we faced in Python 2
>
> The PEP doesn't even have to be done all at once, nor by one person.
> In fact, there are many viewpoints that would add value to the
> document.  Hence it would probably make sense to encourage broad
> participation and then have a single editor to effect a single voice
> in the document.
>
> The contents of the retrospective document should probably cover a
> broad range of topics, since there's so much to learn from the move to
> Python 3.  To give an indication of what I mean, I've included a rough
> outline at the bottom of this message.
>
> So...I typically strongly avoid making proposals that I'm not willing
> to execute.  However, in this case I simply do not have enough
> experience in the history to feel comfortable doing a good job of it
> in a reasonable amount of time (which matters due to the tendency of
> valuable info to fade away). :/  I have no expectation that someone
> will pick this up, though I do hope since the benefit would be
> significant.  My apologies in advance if this wasted anyone's time.
>
> -eric
>
>
> 
>
> I'd hope to see something along the lines of (at least) the following,
> in rough order:
>
> * a concise summary of the document at the top (very meta, I know :) )
>   + what were we solving?
>   + what was the solution?
>   + why do it that way?
>   + what went right?
>   + what went wrong?
>   + impact on the community
>   + impact on core dev contribution
> * timeline
> * key players (and level of involvement)
>   + old guard core devs
>   + new guard
>   + folks brought on for Py3k (e.g. IIRC a swarm of Googlers dove in)
>   + non-core-devs
> * motivations
> * expectations (e.g. time frames, community reaction)
> * corresponding results
> * a summary of what we did


nine has a very concise, if incomplete in comparison to six and 2to3,
code/namespace/functional summary:
https://github.com/nandoflorestan/nine/blob/master/nine/__init__.py

https://github.com/benjaminp/six/blob/master/six.py

http://python-future.org/overview.html#automatic-conversion-to-py2-3-compatible-code

> python-future comes with two scripts called futurize and pasteurize to
aid in making Python 2 code or Python 3 code compatible with both platforms
(Py2/3). It is based on 2to3 and uses fixers from lib2to3, lib3to2, and
python-modernize, as well as custom fixers

"Cheat Sheet: Writing Python 2-3 compatible code"
http://python-future.org/compatible_idioms.html

https://github.com/PythonCharmers/python-future

> It provides future and past packages with backports and forward ports of
features from Python 3 and 2.


> * alternative approaches


#!/usr/bin/env python
# ~~✓pythonver: > 3.3✓~~


> * what went right (and was it on purpose :) )
> * what went wrong (e.g. io) and why
> * how the Py3k project differed from normal python-dev workflow (e.g.
> pace, decision-making, communications)
> * lasting impact of python-dev
> * key things that would have been better if done differently
> * key decisions/planning (mostly a priori to the release work)
>   + scope of backward compatibility
>   + process (using PEPs with PEPs 30xx guiding)
>   + schedule
>   + specific changes (i.e. PEPs 31xx)
>   + what was left out (and why)
>   + plans to help library and app authors transition (e.g. 2to3)
>   + feature/schedule overlap with Python 2 (i.e. 2.6 and 2.7)
>   + the language moratorium
> * things that got missed and why
>   + unicode/bytes in some stdlib modules (and builtins?)
> * things that were overdone (and how that got missed)
>   

Re: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20

[Wes Turner]

So, the style guidelines for this new feature -- and also ternary
expressions and comprehension -- would need to mention that:

- debuggers have no idea what to do with all of this on one line
- left-to-right doesn't apply to comprehensions

  results = [(x, y, x/y) for x in input_data if (y := f(x)) > 0]

- left-to-right doesn't apply to ternary expressions

  if (y := func(x)) if (x := 3) else 0:
  while (y := func(x)) if (x := 3) else 0:

- left-to-right does apply to everything else?

- *these* are discouraged:

  if (x := 3) or (y := func(x)):
  if (3) or (func(3)):
  if ((x := 3) if 1 else (y := func(x))):

IDK, I could just be resistant to change, but this seems like something
that will decrease readability -- and slow down code review -- without any
real performance gain. So, while this would be useful for golfed-down (!)
one-liners with pyline,
I'm -1 on PEP 572.

How do I step through this simple example with a debugger?

if re.search(pat, text) as match:
print("Found:", match.group(0))

How do I explain what ':=' is when teaching Python?

AFAIU, semantically: Python = ('equals') indicates a statement. What you
are proposing is adding an ':=' ('colon equals') assignment operator which
defines a variable which is limited in scope only in list, dict, and
generator comprehensions.

>From https://en.wikipedia.org/wiki/Assignment_(computer_science) :

> In some languages the symbol used is regarded as an operator (meaning
that the assignment has a value) while others define the assignment as a
statement (meaning that it cannot be used in an expression).


PEP 572 -- Assignment Expressions
PEP 572 -- Assignment Operator (:=) and Assignment Expressions

On Friday, April 27, 2018, Steven D'Aprano  wrote:

> On Thu, Apr 26, 2018 at 08:48:12AM -0700, Łukasz Langa wrote:
> >
> > > On Apr 25, 2018, at 11:10 PM, Steven D'Aprano 
> wrote:
> > > Criticising binding-
> > > expressions for that reason, especially implying that we must always
> use
> > > parens, is simply FUD.
> >
> > The PEP has more examples with parentheses than without.
>
> Yes? Parens aren't mandatory, and my point that other operators also
> sometimes needs parens still holds.
>
>
> > >> As soon as we have to wrap a part of an expression in parentheses,
> > >> parsing the entire thing becomes more complex.
> > >
> > > Unless it becomes less complex to read and understand.
> >
> > You're ignoring the context of the discussion. The new parentheses are
> > there because there's a new assignment there. That's more complex.
>
> I'm not ignoring the context of the discussion. I'm comparing binding-
> expression with and without parens. That's what I thought you were
> doing.
>
> If that wasn't your intended meaning, then I apologise but please
> understand why I answered the way I did.
>
> I still stand by my argument: parens are not always needed, and even
> when they are not needed, adding them can sometimes make things
> *easier* and *less complex* to read.
>
>
> [...]
> > If you think demonstrating cases where the end result won't be an
> > improvement is picking at straws, then maybe the improvement of PEP
> > 572 is as well.
>
> Any feature can have cases where the end result is worse than not using
> the feature. That *alone* isn't a strong argument against a feature.
>
> Do you have much existing code using binding expressions? Of course not.
> Will you be adding them to code that already exists? Probably not -- you
> can't do so until you are using 3.8 at minimum, and if your code needs
> to be backwards compatible, you can't use it until you've dropped
> support for 3.7 and older. That might not be for five or ten years.
>
> So it is likely that for most people only new code will use this
> feature. It is not reasonable to say that if I have existing code like
> this:
>
> spam = expression
> if long_condition_that_takes_up_most_of_the_line == spam or spam:
> ...
>
> that I'm going to immediately change it to a one-liner:
>
> if long_condition_that_takes_up_most_of_the_line == (spam :=
> expression) or spam:
> ...
>
> and push it over the maximum line width. With or without parentheses.
> Why would I do something so silly? Using binding expressions isn't
> mandatory and most coders don't intentionally do things that make their
> code worse.
>
> And if I wouldn't change existing code and push it over the limit, why
> would I write new code that does it? Especially when there are much
> better alternatives:
>
> if (long_condition_that_takes_up_most_of_the_line
> == (spam:=expression)
> or spam):
> ...
>
>
> We have a history of adding features that can be abused, but aren't.
> People hardly ever abuse list comps with overly long and complex
> multiple-loop comprehensions:
>
> [... for a in sequence for b in something for c in another for d in
> something_else]
>
> I'm sure we've all seen one or two of those. But it 

Re: [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20

[Wes Turner]

On Wednesday, April 25, 2018, Łukasz Langa  wrote:

>
> On 25 Apr, 2018, at 5:20 PM, Chris Angelico  wrote:
>
> On Thu, Apr 26, 2018 at 10:11 AM, Yury Selivanov
>  wrote:
>
> Just yesterday this snippet was used on python-dev to show how great the
> new syntax is:
>
>  my_func(arg, buffer=(buf := [None]*get_size()), size=len(buf))
>
> To my eye this is an anti-pattern.  One line of code was saved, but the
> other line becomes less readable.  The fact that 'buf' can be used after
> that line means that it will be harder for a reader to trace the origin of
> the variable, as a top-level "buf = " statement would be more visible.
>
>
> Making 'buf' more visible is ONLY a virtue if it's going to be used
> elsewhere. Otherwise, the name 'buf' is an implementation detail of
> the fact that this function wants both a buffer and a size.
>
>
> You're claiming that `:=` is nicer in this situation because it's less
> prominent than regular assignment and thus doesn't suggest that the name
> stays visible later.
>
> But as others said, `:=` *does* make the name visible later until the
> enclosing scope ends.  In fact, a large part of its appeal is that you
> can use the result later (as in the `re.search()` example).  Will it be
> visible enough to the reaser in those cases then?
>
> There seems to be a conflict there.
>
> The question of assignment visibility also makes me think about
> unintentional name shadowing::
>
> buf = some_value
>
> ...  # 20 lines
>
> my_func(arg, buffer=(buf := [None]*get_size()), size=len(buf))
>
> ...  # 20 lines
>
> buf  # <-- What value does this have?
>
>
> Even if we're not using the call pattern, there can be plenty of logic
> tests which aren't very obvious::
>
> buf = some_value
>
> ...  # 20 lines
>
> if node.parent is not None and (buf := node.parent.buffer):
> ... # 10 lines
>
> ...  # 20 lines
>
> buf  # <-- What value does this have?
>
>
> This is even more interesting because now `buf` isn't rebound
> *always*.
>
> So if I'm confused about an unexpected change in value of `buf`, I'll
> skim the code, fail to find the assignment, and then grep for `buf =`
> and also fail to find the assignment.  Yes, I could have searched for
> just `buf` instead but that will give me too many false positives,
> especially if I'm using a primitive text editor search or don't know
> about \b in regular expressions.
>
> Debugging this can be confusing.  I know it can since a similar
> annoyance can be observed with the magic pseudo-scope of `except`::
>
> err = some_value
> try:
> ...
> except Error as err:
> ...
>
> err  # <-- now sometimes it's not defined
>
>
> Just like Barry, I debugged a few cases of this in the past and within
> larger functions this can be hard to find.
>

Would this make it easier to put too much code on one line?

Is there a good way to get *branch coverage* stats instead of just *line
coverage*?

Someone can probably explain with some tested pretty code for me why this
would be necessary or helpful; why it wouldn't make line coverage stats
more misleading for the sake of lazy?


>
> -- Ł
>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Introducing python.zulipchat.com

[Wes Turner]

On Sunday, April 22, 2018, Paul Moore  wrote:

> On 22 April 2018 at 15:39, Nick Coghlan  wrote:
> > On 22 April 2018 at 21:47, Paul Moore  wrote:
> >> On 22 April 2018 at 00:05, Brett Cannon  wrote:
> >>> The Zulip project maintainers are active on our instance so after you
> join
> >>> go to the Zulip stream and start a topic about this.
> >>
> >> I did - "Zulip -> Sign up". I don't know of a way to put a link to
> >> that topic here, but I assume it's findable in the UI.
> >
> > For anyone else with the same question: clicking on the topic title in
> > either the side navbar or the stream overview will give you a view
> > specific to the topic that also serves as a shareable link. In this
> > case:
> >
> > https://python.zulipchat.com/#narrow/stream/116410-zulip/
> topic/Sign.20up
> >
>
> lol, I'm so used to web apps *not* providing URLs specific to where
> you are in them (I'm not even sure what the term for this is - "doing
> the web properly"? :-)) that it never even occurred to me that the
> link in my browser bar was usable!


"Deep linking" with a "#fragment-identifier".
https://en.wikipedia.org/wiki/Deep_linking
https://en.wikipedia.org/wiki/Fragment_identifier



>
> Paul
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> wes.turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] ssl module and LibreSSL CVE-2018-8970

[Wes Turner]

+1. Thanks!

Which tests?

On Wednesday, April 4, 2018, Christian Heimes  wrote:

> Hi,
>
> I like to share the story of a critical security bug with you. Contrary
> to other issues in TLS/SSL, it's a story with happy ending. Nobody was
> harmed. The bug was fixed before it affected the general population.
>
>
> Introduction
> 
>
> Python's ssl.match_hostname() function was a source of several CVEs and
> other security bugs. After a long struggle, I decided to drop support
> for old OpenSSL releases and uses a new OpenSSL method to offload host
> name verification to OpenSSL. The improvement [1] eventually landed in
> Python 3.7. Nowadays OpenSSL verifies host name or IP address during the
> TLS/SSL handshake.
>
> Later I discovered that LibreSSL <= 2.6 did not have
> X509_VERIFY_PARAM_set1_host() [2]. We had to temporarily suspend support
> for LibreSSL. About two months later, LibreSSL caught up and released
> version 2.7.0 with support for the function.
>
>
> The bug
> ---
>
> One day after the release of LibreSSL 2.7.0, I started to port Python
> 3.7 to LibreSSL. In matter of minutes I got the ssl module to compile
> and work with LibreSSL. All tests were passing -- except for negative
> the host name verification tests. LibreSSL was accepting all invalid
> host names as correct! Python's vigorous test suite had discovered a
> critical security bug in LibreSSL.
>
> It turned out that LibreSSL copied the implementation of
> X509_VERIFY_PARAM_set1_host(param, name, namelen) from BoringSSL and the
> documentation from OpenSSL. BoringSSL's implementation didn't support
> the special case of 0 as namelen parammeter. OpenSSL supports namelen =
> 0, which is interpreted as namelen=strlen(name). It is documented in
> OpenSSL's man page and was even recommended on OpenSSL's wiki as
> preferred way.
>
>
> Happy Ending
> 
>
> So I got in contact with LibreSSL's security team and BoringSSL's
> security team [3]. Less than a day later, both libraries released fixes
> for the bug [4]. Mitre has assigned CVE-2018-8970 [5] to the bug.
> Disaster averted!
>
> BoringSSL's security issue [3] contains more information. Adam Langley
> lifted the restriction about an hour ago.
>
> I like to thank Bob Beck (LibreSSL), Adam Langley (Google) and David
> Benjamin (Google) for their assistance and cooperation.
>
> Regards,
> Christian
>
> [1] https://bugs.python.org/issue31399
> [2] https://github.com/libressl-portable/portable/issues/381
> [3] https://bugs.chromium.org/p/chromium/issues/detail?id=824799
> [4] https://www.libressl.org/releases.html
> [5] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-8970
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> wes.turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Move ensurepip blobs to external place

[Wes Turner]

AFAIU, the objectives (with no particular ranking) are:

- minimize git clone time and bandwidth
- include latest pip with every python install
- run the full test suite with CI for every PR (buildbot)
  - the full test suite requires pip
- run the test suite locally when developing a PR
- minimize PyPI bandwidth

What are the proposed solutions?

...

https://help.github.com/articles/about-storage-and-bandwidth-usage/

> All personal and organization accounts using Git LFS receive 1 GB of free
storage and 1 GB a month of free bandwidth. If the bandwidth and storage
quotas are not enough, you can choose to purchase an additional quota for
Git LFS.
>

> Git LFS is available for every repository on GitHub, whether or not your
account or organization has a paid plan.

On Thursday, March 29, 2018, Donald Stufft  wrote:

> From my POV I don’t care where they live, just document how to update them
> going forward.
>
> Sent from my iPhone
>
> > On Mar 24, 2018, at 4:50 AM, Serhiy Storchaka 
> wrote:
> >
> > Wouldn't be better to put them into a separate repository like Tcl/Tk
> and other external binaries for Windows, and download only the recent
> version?
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> wes.turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How is the GitHub workflow working for people?

[Wes Turner]

On Wednesday, February 21, 2018, Nick Coghlan  wrote:

> On 22 February 2018 at 08:19, Barry Warsaw  wrote:
> > As for the bug tracker, I still do like Roundup, and we have a huge
> investment in it, not just in resources expended to make it rock, but also
> in all the history in it and everything that integrates with it.  I
> wouldn’t stop anybody who’s motivated to spearhead a move to GH issues, but
> I also don’t think that can be taken up lightly.
>
> [...]
>

> Personally, the only things I really miss on Roundup vs GitHub issues
> are usability tweaks like Markdown support in the comment editor, and
> inline dropdowns for @-mentions of other users and #-mentions of other
> issues, so if someone is motivated to work on issue tracking
> enhancements, that seems like a more fruitful endeavour than trying to
> migrate wholesale to a proprietary third party service.


Additional Roundup ENHancements:

- [ ] ENH: Issue mention 'trackbacks'
  Optionally quoting the full issue title inline instead of just #123 would
be great: "#123: Issue title"

- [ ] ENH: GitHub OAuth integration; to make it easy to login to Roundup
with GitHub credentials

  http://python-social-auth.readthedocs.io/en/latest/backends/github.html


>
> Cheers,
> Nick.
>
> P.S. That said, one tracker that I think absolutely *would* be worth
> migrating to GitHub is the meta-tracker at
> http://psf.upfronthosting.co.za/roundup/meta. We haven't customised
> that instance the way we have bugs.python.org, and consolidating it
> and the source repo at http://hg.python.org/tracker/roundup/ into a
> single https://github.com/python/bugs.python.org repo would better
> align tracker development with development on other parts of the
> infrastructure.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> wes.turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How is the GitHub workflow working for people?

[Wes Turner]

On Friday, February 23, 2018, Nick Coghlan  wrote:

> On 23 February 2018 at 03:09, Eric Snow 
> wrote:
> > So, coupled with slow CI, linting failures were
> > particularly onerous.  We all got in the habit of locally running the
> > linter frequently.  IIRC, I even set up VIM to run the linter whenever
> > I saved.
>
> For reindent.py, we used to have instructions for setting that up as a
> local pre-commit hook in Mercurial. It may make sense to create
> similar instructions for git (if we don't already have those
> somewhere).


A pre-commit hook with flake8, clang-format, and the requisite CPython
style configs would be great.

https://git-scm.com/book/gr/v2/Customizing-Git-Git-Hooks

Batch and ad-hoc CLN commits to cleanup for style really muddle the git
blame history.


>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> wes.turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] LibreSSL support

[Wes Turner]

LibreSSL is not a pressing need for me; but fallback to the existing
insecure check if LibreSSL is present shouldn't be too difficult?

On Thursday, January 18, 2018, Christian Heimes <christ...@python.org>
wrote:

> On 2018-01-18 19:42, Wes Turner wrote:
> > Is there a build flag or a ./configure-time autodetection that would
> > allow for supporting LibreSSL while they port
> X509_VERIFY_PARAM_set1_host?
>
> X509_VERIFY_PARAM_set1_host() is a fundamental and essential piece in
> the new hostname verification code. I cannot replace
> ssl.match_hostname() easily without the API. There might be a way to add
> a callback, but it would take a couple of days of R to implement it.
> It won't be finished for beta1 feature freeze.
>
> Christian
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> wes.turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] LibreSSL support

[Wes Turner]

Is there a build flag or a ./configure-time autodetection that would allow
for supporting LibreSSL while they port X509_VERIFY_PARAM_set1_host?

On Thursday, January 18, 2018, Christian Heimes 
wrote:

> On 2018-01-16 21:17, Christian Heimes wrote:
> > FYI, master on Travis CI now builds and uses OpenSSL 1.1.0g [1]. I have
> > created a daily cronjob to populate Travis' cache with OpenSSL builds.
> > Until the cache is filled, Linux CI will take an extra 5 minute.
>
> I have messed up my initial research. :( When I was checking LibreSSL
> and OpenSSL for features, I draw a wrong conclusion. LibreSSL is *not*
> OpenSSL 1.0.2 compatible. It only implements some of the required
> features from 1.0.2 (e.g. X509_check_hostname) but not
> X509_VERIFY_PARAM_set1_host.
>
> X509_VERIFY_PARAM_set1_host() is required to perform hostname
> verification during the TLS handshake. Without the function, I'm unable
> to fix Python's hostname matching code [1]. LibreSSL upstream knows
> about the issue since 2016 [2]. I have opened another bug report [3].
>
> We have two options until LibreSSL has addressed the issue:
>
> 1) Make the SSL module more secure, simpler and standard conform
> 2) Support LibreSSL
>
> I started a vote on Twitter [4]. So far most people prefer security.
>
> Christian
>
> [1] https://bugs.python.org/issue31399
> [2] https://github.com/pyca/cryptography/issues/3247
> [3] https://github.com/libressl-portable/portable/issues/381
> [4] https://twitter.com/reaperhulk/status/953991843565490176
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> wes.turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3.7: Require OpenSSL >=1.0.2 / LibreSSL >= 2.5.3

[Wes Turner]

On Tuesday, January 16, 2018, Christian Heimes <christ...@python.org> wrote:

> On 2018-01-16 12:28, Wes Turner wrote:
> >
> >
> > On Tuesday, January 16, 2018, Steve Dower <steve.do...@python.org
> > <mailto:steve.do...@python.org>> wrote:
> >
> > From my perspective, we can’t keep an OpenSSL-like API and use
> > Windows platform libraries (we *could* do a requests-like API easily
> > enough, but even urllib3 is painfully low-level).
> >
> > Support for Windows SChannel and Apple SecureTransport is part of the
> > TLS module.
> >
> > IDK how far along that work is (whether it'll be ready for 3.7 beta 1)?
> > Or where those volunteering to help with the TLS module can send PRs?
>
> You are misunderstanding the goal of PEP 543. It's not about providing
> implementations of various backends. The PEP merely defines an minimal
> abstraction layer. Neither the PEP nor the API are finalized or complete
> yet, too Some parts of the PEP must be changed before it can be
> finalized. Cory and I are discussion the matter.
>
> Python 3.7's ssl module won't be compatible with PEP 543. For 3.8 it
> *might* be possible to provide a 543 compatible implementation on top of
> the ssl module.

Got it. Thanks!

>
> I will not work on SChannel or SecureTransport, since I have neither
> expertise, knowledge, interest or resources to work on other
> implementations. AFAIK Steve would rather plug in Windows' cert
> validation API into OpenSSL than to provide another TLS implementation.
> For Apple ... no clue. How about you contact Apple support?


A HUP to their seclist about this work awhile back doesn't seem to have
upgraded OpenSSL.

Presumably there's another mailing list thread or GitHub issue for PEP 543
interface and implementation development.


>
> Regards,
> Christian
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> wes.turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3.7: Require OpenSSL >=1.0.2 / LibreSSL >= 2.5.3

[Wes Turner]

On Tuesday, January 16, 2018, Steve Dower  wrote:

> From my perspective, we can’t keep an OpenSSL-like API and use Windows
> platform libraries (we *could* do a requests-like API easily enough, but
> even urllib3 is painfully low-level).
>
> Support for Windows SChannel and Apple SecureTransport is part of the TLS
module.

IDK how far along that work is (whether it'll be ready for 3.7 beta 1)? Or
where those volunteering to help with the TLS module can send PRs?

https://github.com/python/peps/blob/master/pep-0543.rst

https://www.python.org/dev/peps/pep-0543/

http://markmail.org/search/?q=list%3Aorg.python+PEP+543+TLS

https://www.python.org/dev/peps/pep-0543/#interfaces

>
>
> We have to continue shipping our own copy of OpenSSL on Windows. Nothing
> to negotiate here except whether OpenSSL releases should trigger a Python
> release, and I think that decision can stay with the RM.
>
>
>
> Good luck solving macOS :o)
>
>
>
> Cheers,
>
> Steve
>
>
>
> Top-posted from my Windows phone
>
>
>
> *From: *Stephen J. Turnbull 
> *Sent: *Tuesday, January 16, 2018 17:45
> *To: *Matt Billenstein 
> *Cc: *Christian Heimes ; python-dev@python.org
> *Subject: *Re: [Python-Dev] Python 3.7: Require OpenSSL >=1.0.2 /
> LibreSSL >=2.5.3
>
>
>
> Matt Billenstein writes:
>
>
>
> > In my mind it becomes easier to bundle deps in a binary installer
>
> > across the board (Linux, OSX, Windows) rather than rely on whatever
>
> > version the operating system provides.
>
>
>
> Thing is, as Christian points out, TLS is a rapidly moving target.
>
> Every Mac OS or iOS update seems to link to a dozen CVEs for TLS
>
> support.  We can go there if we have to, but it's often hard to go
>
> back when vendor support catches up to something reasonable.  I think
>
> this is something for Ned and Christian and Steve to negotiate, since
>
> they're the ones who are most aware of the tradeoffs and bear the
>
> costs.
>
>
>
>
>
>
>
> ___
>
> Python-Dev mailing list
>
> Python-Dev@python.org
>
> https://mail.python.org/mailman/listinfo/python-dev
>
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> steve.dower%40python.org
>
>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3.7: Require OpenSSL >=1.0.2 / LibreSSL >= 2.5.3

[Wes Turner]

FWIW, anaconda and conda-forge currently have 1.0.2 X

https://anaconda.org/anaconda/openssl

https://anaconda.org/conda-forge/openssl

On Sunday, January 14, 2018, Ned Deily  wrote:

> On Jan 14, 2018, at 08:39, Christian Heimes  wrote:
> > On 2018-01-14 09:24, Matt Billenstein wrote:
> >> Correct me if I'm wrong, but Python3 on osx bundles openssl since Apple
> has
> >> deprecated (and no longer ships the header files for) the version
> shipped with
> >> recent versions of osx.
> >>
> >> Perhaps this is an option to support the various flavors of Linux as
> well?
> >
> > AFAK Apple has decided to compile and statically link CPython's ssl with
> > an ancient, customized LibreSSL version. Cory posted [1] a couple of
> > months ago
>
> I think you're conflating some things here.  Apple has not yet shipped a
> version of Python 3 with macOS so the fact that Apple now links their
> version of Python2.7 with a "private" copy of LibreSSL is irrelevant.
> (It's private in the sense that they don't ship the header files for it;
> the shared libs are there just for the use of the open source products
> they ship with macOS that don't yet use the macOS native crypto APIs,
> products like Python and Perl.)
>
> What Matt is likely thinking of is the Python 3 versions provided by the
> python.org macOS binary installers where we do build and link with our
> own 1.0.2 (and soon 1.1.0 for 3.7) versions of OpenSSL.  Currently,
> the OpenSSL (and several other third-party libs such as libxz which
> is not shipped by Apple) are built as part of the installer build
> script in the Mac section of the source repo.  I would like to
> refactor and generalize that so those third-party libs
> could optionally be used for non-installer builds as well.  But, in
> any case, we don't have much choice for the installer builds until
> such time as cPython has support for the Apple-provided crypto APIs.


Support for Apple SecureTransport is part of the TLS module. IDK how far
along that work is (whether it'll be ready for 3.7 beta 1)?

https://github.com/python/peps/blob/master/pep-0543.rst

https://www.python.org/dev/peps/pep-0543/

http://markmail.org/search/?q=list%3Aorg.python+PEP+543+TLS


>
> > I'm not going to add OpenSSL sources or builds to CPython. We just got
> > rid of copies of libffi and other 3rd party dependencies. Crypto and TLS
> > libraries are much, MUCH more complicated to handle than libffi. It's a
> > constant moving targets of attacks. Vendors and distributions also have
> > different opinions about trust store and policies.
> >
> > Let's keep build dependencies a downstream and vendor problem.
>
> That's not always an option, unfortunately.
>
> --
>   Ned Deily
>   n...@python.org -- []
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> wes.turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEPs: ``.. code:: python`` or ``::`` (syntax highlighting)

[Wes Turner]

Add pygments for ``.. code::`` directive PEP syntax highlighting #1206
https://github.com/python/pythondotorg/issues/1206

Syntax highlighting is an advantage for writers, editors, and readers.

reStructuredText PEPs are rendered into HTML with docutils. Syntax
highlighting in Docutils 0.9+ is powered by Pygments. If Pygments is not
installed, or there is a syntax error, syntax highlighting is absent.
Docutils renders ``.. code::`` blocks with Python syntax highlighting by
default. You can specify ``.. code:: python`` or ``.. code:: python3``.

- GitHub shows Pygments syntax highlighting
for ``.. code::`` directives for .rst and .restructuredtext documents
- PEPs may eventually be hosted on ReadTheDocs with Sphinx (which installs
docutils and pygments as install_requires in setup.py).
  https://github.com/python/peps/issues/2
  https://github.com/python/core-workflow/issues/5

In order to use pygments with pythondotorg-hosted PEPs, a few things need
to happen:

- [ ] Include ``pygments`` in ``base-requirements.txt``
- [ ] Pick a pygments theme
  - Should we use the sphinx_rtd_theme default for consistency with the
eventual RTD-hosted PEPs?
- [ ] Include the necessary pygments CSS in the PEPs django template
- [ ] rebuild the PEPs
- Start using code directives in new PEPs
- Manually review existing PEPs after adding code directives

PEPs may use ``.. code::`` blocks instead of ``::`` so that code is syntax
highlighted.

On Saturday, December 2, 2017, Nick Coghlan <ncogh...@gmail.com> wrote:

> On 3 December 2017 at 12:32, Wes Turner <wes.tur...@gmail.com
> <javascript:;>> wrote:
> > Pending a transition of PEPs to ReadTheDocs (with HTTPS on a custom
> domain?
> > and redirects?) (is there a gh issue for this task?),
>
> See https://github.com/python/peps/projects/1 and
> https://github.com/python/core-workflow/issues/5
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com <javascript:;>   |   Brisbane,
> Australia
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEPs: ``.. code:: python`` or ``::`` (syntax highlighting)

[Wes Turner]

Pending a transition of PEPs to ReadTheDocs (with HTTPS on a custom domain?
and redirects?) (is there a gh issue for this task?),
for the pythondotorg project
is it as simple as `pip install pygments` and rebuilding each .rst with
docutils with pygments installed?

On Saturday, December 2, 2017, Mariatta Wijaya 
wrote:

> If we were to add Pygments support, it is to be done in pythondotorg
> project.
>
> I recalled the decision was to get PEPs rendered using Sphinx and host it
> at Read The Docs, so we don't have to worry about updating pythondotorg.
>
> Mariatta Wijaya
>
>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Third and hopefully final post: PEP 557, Data Classes

[Wes Turner]

On Thursday, November 30, 2017, Wes Turner <wes.tur...@gmail.com> wrote:

> Could these be things in types?
>
> types.ClassType
> types.InstanceType
>
> types.DataClass
> types.DataClassInstanceType (?)
>

types.DataClass(types.ClassType)
types.DataClassInstanceType(types.InstanceType)

Would be logical?


>
> I sent a PR with typo fixes and ``.. code:: python`` directives so that
> syntax highlighting works (at least on GitHub).
>
> https://github.com/python/peps/blob/master/pep-0557.rst
>
> https://github.com/python/peps/pull/488
>
> Additional notes:
>
> - "DataClass" instead of "Data Class" would be easier to search for.
> s/DataClass/Data Class/g?
> - It's probably worth mentioning how hash works when frozen=True also here:
>   https://github.com/python/peps/blob/master/pep-0557.rst#frozen-instances
> - The `hash` explanation could be a two column table for easier readability
>
> What a great feature.
>
> - Runtime data validation from annotations (like PyContracts,) would be
> cool
> - __slots__ are worth the time
>
> On Thursday, November 30, 2017, Antoine Pitrou <solip...@pitrou.net
> <javascript:_e(%7B%7D,'cvml','solip...@pitrou.net');>> wrote:
>
>>
>> isdataclass() testing for instance-ship does sound like a bug magnet to
>> me.
>>
>> If isdataclassinstance() is too long (that's understandable), how about
>> isdatainstance()?
>>
>> Regards
>>
>> Antoine.
>>
>>
>> On Wed, 29 Nov 2017 17:02:21 -0800
>> Guido van Rossum <gu...@python.org> wrote:
>> >  On Wed, Nov 29, 2017 at 3:51 PM, Carl Meyer <c...@oddbird.net> wrote:
>> >
>> > > On 11/29/2017 03:26 PM, Eric V. Smith wrote:
>> > > > I've posted a new version of PEP 557, it should soon be available at
>> > > > https://www.python.org/dev/peps/pep-0557/.
>> > > >
>> > > > The only significant changes since the last version are:
>> > > >
>> > > > - changing the "compare" parameter to be "order", since that more
>> > > > accurately reflects what it does.
>> > > > - Having the combination of "eq=False" and "order=True" raise an
>> > > > exception instead of silently changing eq to True.
>> > > >
>> > > > There were no other issues raised with the previous version of the
>> PEP.
>> > >
>> > > Not quite; I also raised the issue of isdataclass(ADataClass)
>> returning
>> > > False. I still think that's likely to be a cause of bug reports if
>> left
>> > > as-is.
>> > >
>> >
>> > I tried to look up the discussion but didn't find much except that you
>> > flagged this as an issue. To repeat, your concern is that isdataclass()
>> > applies to *instances*, not classes, which is how Eric has designed it,
>> but
>> > you worry that either through the name or just because people don't read
>> > the docs it will be confusing. What do you suppose we do? I think
>> making it
>> > work for classes as well as for instances would cause another category
>> of
>> > bugs (confusion between cases where a class is needed vs. an instance
>> > abound in other situations -- we don't want to add to that). Maybe it
>> > should raise TypeError when passed a class (unless its metaclass is a
>> > dataclass)? Maybe it should be renamed to isdataclassinstance()? That's
>> a
>> > mouthful, but I don't know how common the need to call this is, and
>> people
>> > who call it a lot can define their own shorter alias.
>> >
>>
>>
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: https://mail.python.org/mailman/options/python-dev/wes.
>> turner%40gmail.com
>>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEPs: ``.. code:: python`` or ``::`` (syntax highlighting)

[Wes Turner]

In ReStructuredText, this gets syntax highlighted
because of the code directive [1][2][3]:

.. code:: python

   import this
   def func(*args, **kwargs):
   pass

This also gets syntax highlighted as python[3]:

.. code:: python

   import this
   def func(*args, **kwargs):
   pass

This does not::

   import this
   def func(*args, **kwargs):
   pass

Syntax highlighting in Docutils 0.9+ is powered by Pygments.
If Pygments is not installed, or there is a syntax error,
syntax highlighting is absent.

GitHub does show Pygments syntax highlighting
in .. code:: blocks for .rst and .restructuredtext documents [4]

1. Does the python.org PEP view support .. code:: blocks? [5]
2. Syntax highlighting is an advantage for writers, editors, and readers.
3. Should PEPs use .. code:: blocks to provide this advantage?

[1] http://docutils.sourceforge.net/docs/ref/rst/directives.html#code
[2] http://www.sphinx-doc.org/en/stable/markup/code.html
[3]
http://www.sphinx-doc.org/en/stable/config.html#confval-highlight_language
[4] https://github.com/python/peps/blob/master/pep-0557.rst
[5] https://www.python.org/dev/peps/pep-0557/

https://www.python.org/dev/peps/pep-0458/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Third and hopefully final post: PEP 557, Data Classes

[Wes Turner]

Sorry, this one shouldn't be "an"; " a" was correct::

  ``repr``: If true (the default), an ``__repr__`` method will be generated.

Note that __repr__ can be dangerous with user-supplied input because of
terminal control character injection (e.g. broken syntax highlighting, \r,
character set, LEDs,)

On Thursday, November 30, 2017, Wes Turner <wes.tur...@gmail.com> wrote:

> Could these be things in types?
>
> types.ClassType
> types.InstanceType
>
> types.DataClass
> types.DataClassInstanceType (?)
>
> I sent a PR with typo fixes and ``.. code:: python`` directives so that
> syntax highlighting works (at least on GitHub).
>
> https://github.com/python/peps/blob/master/pep-0557.rst
>
> https://github.com/python/peps/pull/488
>
> Additional notes:
>
> - "DataClass" instead of "Data Class" would be easier to search for.
> s/DataClass/Data Class/g?
> - It's probably worth mentioning how hash works when frozen=True also here:
>   https://github.com/python/peps/blob/master/pep-0557.rst#frozen-instances
> - The `hash` explanation could be a two column table for easier readability
>
> What a great feature.
>
> - Runtime data validation from annotations (like PyContracts,) would be
> cool
> - __slots__ are worth the time
>
> On Thursday, November 30, 2017, Antoine Pitrou <solip...@pitrou.net
> <javascript:_e(%7B%7D,'cvml','solip...@pitrou.net');>> wrote:
>
>>
>> isdataclass() testing for instance-ship does sound like a bug magnet to
>> me.
>>
>> If isdataclassinstance() is too long (that's understandable), how about
>> isdatainstance()?
>>
>> Regards
>>
>> Antoine.
>>
>>
>> On Wed, 29 Nov 2017 17:02:21 -0800
>> Guido van Rossum <gu...@python.org> wrote:
>> >  On Wed, Nov 29, 2017 at 3:51 PM, Carl Meyer <c...@oddbird.net> wrote:
>> >
>> > > On 11/29/2017 03:26 PM, Eric V. Smith wrote:
>> > > > I've posted a new version of PEP 557, it should soon be available at
>> > > > https://www.python.org/dev/peps/pep-0557/.
>> > > >
>> > > > The only significant changes since the last version are:
>> > > >
>> > > > - changing the "compare" parameter to be "order", since that more
>> > > > accurately reflects what it does.
>> > > > - Having the combination of "eq=False" and "order=True" raise an
>> > > > exception instead of silently changing eq to True.
>> > > >
>> > > > There were no other issues raised with the previous version of the
>> PEP.
>> > >
>> > > Not quite; I also raised the issue of isdataclass(ADataClass)
>> returning
>> > > False. I still think that's likely to be a cause of bug reports if
>> left
>> > > as-is.
>> > >
>> >
>> > I tried to look up the discussion but didn't find much except that you
>> > flagged this as an issue. To repeat, your concern is that isdataclass()
>> > applies to *instances*, not classes, which is how Eric has designed it,
>> but
>> > you worry that either through the name or just because people don't read
>> > the docs it will be confusing. What do you suppose we do? I think
>> making it
>> > work for classes as well as for instances would cause another category
>> of
>> > bugs (confusion between cases where a class is needed vs. an instance
>> > abound in other situations -- we don't want to add to that). Maybe it
>> > should raise TypeError when passed a class (unless its metaclass is a
>> > dataclass)? Maybe it should be renamed to isdataclassinstance()? That's
>> a
>> > mouthful, but I don't know how common the need to call this is, and
>> people
>> > who call it a lot can define their own shorter alias.
>> >
>>
>>
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: https://mail.python.org/mailman/options/python-dev/wes.
>> turner%40gmail.com
>>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Third and hopefully final post: PEP 557, Data Classes

[Wes Turner]

Could these be things in types?

types.ClassType
types.InstanceType

types.DataClass
types.DataClassInstanceType (?)

I sent a PR with typo fixes and ``.. code:: python`` directives so that
syntax highlighting works (at least on GitHub).

https://github.com/python/peps/blob/master/pep-0557.rst

https://github.com/python/peps/pull/488

Additional notes:

- "DataClass" instead of "Data Class" would be easier to search for.
s/DataClass/Data Class/g?
- It's probably worth mentioning how hash works when frozen=True also here:
  https://github.com/python/peps/blob/master/pep-0557.rst#frozen-instances
- The `hash` explanation could be a two column table for easier readability

What a great feature.

- Runtime data validation from annotations (like PyContracts,) would be cool
- __slots__ are worth the time

On Thursday, November 30, 2017, Antoine Pitrou  wrote:

>
> isdataclass() testing for instance-ship does sound like a bug magnet to
> me.
>
> If isdataclassinstance() is too long (that's understandable), how about
> isdatainstance()?
>
> Regards
>
> Antoine.
>
>
> On Wed, 29 Nov 2017 17:02:21 -0800
> Guido van Rossum > wrote:
> >  On Wed, Nov 29, 2017 at 3:51 PM, Carl Meyer  > wrote:
> >
> > > On 11/29/2017 03:26 PM, Eric V. Smith wrote:
> > > > I've posted a new version of PEP 557, it should soon be available at
> > > > https://www.python.org/dev/peps/pep-0557/.
> > > >
> > > > The only significant changes since the last version are:
> > > >
> > > > - changing the "compare" parameter to be "order", since that more
> > > > accurately reflects what it does.
> > > > - Having the combination of "eq=False" and "order=True" raise an
> > > > exception instead of silently changing eq to True.
> > > >
> > > > There were no other issues raised with the previous version of the
> PEP.
> > >
> > > Not quite; I also raised the issue of isdataclass(ADataClass) returning
> > > False. I still think that's likely to be a cause of bug reports if left
> > > as-is.
> > >
> >
> > I tried to look up the discussion but didn't find much except that you
> > flagged this as an issue. To repeat, your concern is that isdataclass()
> > applies to *instances*, not classes, which is how Eric has designed it,
> but
> > you worry that either through the name or just because people don't read
> > the docs it will be confusing. What do you suppose we do? I think making
> it
> > work for classes as well as for instances would cause another category of
> > bugs (confusion between cases where a class is needed vs. an instance
> > abound in other situations -- we don't want to add to that). Maybe it
> > should raise TypeError when passed a class (unless its metaclass is a
> > dataclass)? Maybe it should be renamed to isdataclassinstance()? That's a
> > mouthful, but I don't know how common the need to call this is, and
> people
> > who call it a lot can define their own shorter alias.
> >
>
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org 
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> wes.turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python possible vulnerabilities in concurrency

[Wes Turner]

CWE (Common Weakness Enumeration) has numbers (and URLs) and a graph model,
and code examples, and mitigations for bugs, vulnerabilities, faults,
design flaws, weaknesses.
https://cwe.mitre.org/

Research Concepts
https://cwe.mitre.org/data/definitions/1000.html

Development Concepts
https://cwe.mitre.org/data/definitions/699.html

CWE CATEGORY: Time and State
https://cwe.mitre.org/data/definitions/361.html

  CWE CATEGORY: Concurrency Issues
  https://cwe.mitre.org/data/definitions/557.html

17. Concurrent Execution
https://docs.python.org/3/library/concurrency.html


> 1. activating threads, tasks or pico-threads

https://docs.python.org/3/library/threading.html#threading.Thread.start
https://docs.python.org/3/library/threading.html#threading.Thread.run

> 2. Directed termination of threads, tasks or pico-threads

So, I looked this up:
https://stackoverflow.com/questions/323972/is-there-any-way-to-kill-a-thread-in-python

Do asynchronous programming patterns actually make this basically never
necessary? (asyncio coroutines, greenlet (eventlet, gevent, ), twisted)

https://docs.python.org/3/library/asyncio.html

https://docs.python.org/3/library/asyncio-task.html

If you really feel like you need the overhead of threads instead of or in
addition to coroutines (they won't use multiple cores without going to IPC,
anyway), you can.

> 3. Premature termination of threads, tasks or pico-threads

What is this referring to?
Does it release handles and locks on exception? (try/finally?)


> 4. Concurrent access to data shared between threads, tasks or
pico-threads,   and

CWE-362: Concurrent Execution using Shared Resource with Improper
Synchronization ('Race Condition')
https://cwe.mitre.org/data/definitions/362.html

CWE-567: Unsynchronized Access to Shared Data in a Multithreaded Context
https://cwe.mitre.org/data/definitions/567.html


> 5. Lock protocol errors for concurrent entities

CWE-667: Improper Locking
https://cwe.mitre.org/data/definitions/667.html

CWE-366: Race Condition within a Thread
https://cwe.mitre.org/data/definitions/366.html

The ``mutex`` module is removed in Python 3:
https://docs.python.org/2/library/mutex.html

17.1. threading — Thread-based parallelism
https://docs.python.org/3/library/threading.html


... Are there other good resources (in addition to Chapter 17) for
concurrency in CPython and/or PyPy and/or Stackless Python, MicroPython,
IronPython, Jython?

- [ ] How do we add Python to the excellent CWE reference?

- How can/could/should one add the things with labels (*) from the ISO PDF
you speak of to thr CWE graph? (* schema:name, rdfs:label)

On Wednesday, November 15, 2017, Guido van Rossum  wrote:

> On Wed, Nov 15, 2017 at 6:50 PM, Guido van Rossum  > wrote:
>
>> On Wed, Nov 15, 2017 at 6:37 PM, Armin Rigo > > wrote:
>>
>>> Hi,
>>>
>>> On 14 November 2017 at 14:55, Jan Claeys >> > wrote:
>>> > Sounds like https://www.iso.org/standard/71094.html
>>> > which is updating https://www.iso.org/standard/61457.html
>>> > (which you can download from there if you search a bit; clearly either
>>> > ISO doesn't have a UI/UX "standard" or they aren't following it...)
>>>
>>> Just for completeness, I think that what you can download for free
>>> from that second page only contains the first few sections ("Terms and
>>> definitions").  It doesn't even go to "Purpose of this technical
>>> report"---we need to pay $200 just to learn what the purpose is...
>>>
>>> *Shrug*
>>>
>>
>> Actually it linked to http://standards.iso.org/ittf/
>> PubliclyAvailableStandards/index.html from which I managed to download
>> what looks like the complete c061457_ISO_IEC_TR_24772_2013.pdf (336
>> pages) after clicking on an "I accept" button (I didn't read what I
>> accepted :-). The $200 is for the printed copy I presume.
>>
>
> So far I learned one thing from the report. They use the term
> "vulnerabilities" liberally, defining it essentially as "bug":
>
> All programming languages contain constructs that are incompletely
>> specified, exhibit undefined behaviour, are implementation-dependent, or
>> are difficult to use correctly. The use of those constructs may therefore
>> give rise to *vulnerabilities*, as a result of which, software programs
>> can execute differently than intended by the writer.
>>
>
> They then go on to explain that sometimes vulnerabilities can be
> exploited, but I object to calling all bugs vulnerabilities -- that's just
> using a scary word to get attention for a sleep-inducing document
> containing such gems as "Use floating-point arithmetic only when absolutely
> needed" (page 230).
>
> --
> --Guido van Rossum (python.org/~guido)
>
___
Python-Dev mailing list
Python-Dev@python.org

Re: [Python-Dev] OrderedDict(kwargs) optimization?

[Wes Turner]

Got it. Thanks!

On Wednesday, November 8, 2017, INADA Naoki  wrote:

> > That'd be great for preserving kwargs' order after a pop() or a del?
>
> To clarify, order is preserved after pop in Python 3.6 (and maybe 3.7).
>
> There is discussion about breaking it to optimize for limited use cases,
> but I don't think it's worth enough to discuss more until it demonstrates
> real performance gain.
>
>
> > Is there an opportunity to support a fast cast to OrderedDict from 3.6
> dict?
> > Can it just copy .keys() into the OrderedDict linked list?Or is there
> more overhead to the transition?
>
> https://bugs.python.org/issue31265
>
> Regards,
>
> INADA Naoki  >
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] OrderedDict(kwargs) optimization?

[Wes Turner]

On Wednesday, November 8, 2017, Guido van Rossum  wrote:

> It seems there must be at least two threads for each topic worth
> discussing at all. Therefore I feel compelled to point to
> https://mail.python.org/pipermail/python-dev/2017-November/150381.html,
> where I state my own conclusion about dict order.
>
>
.


>
> Finally: the dict type should not be endowed with other parts of the
> OrderedDict API, not should other API changes to dict be considered.
>

Is there an opportunity to support a fast cast to OrderedDict from 3.6 dict?
Can it just copy .keys() into the OrderedDict linked list?Or is there more
overhead to the transition?

That'd be great for preserving kwargs' order after a pop() or a del?

def func(**kwargs):
kwargs = OrderedDict(kwargs)
arg2 = kwargs.pop('arg2')


>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Migrate python-dev to Mailman 3?

[Wes Turner]

On Thursday, October 26, 2017, Mark Sapiro <m...@msapiro.net> wrote:

> On 10/26/2017 07:28 AM, Wes Turner wrote:
> >
> >
> > On Thursday, October 26, 2017, Paul Moore <p.f.mo...@gmail.com
> <javascript:;>> wrote:
> >
> > On 26 October 2017 at 10:24, Victor Stinner
> > <victor.stin...@gmail.com <javascript:;>> wrote:
> > > We are using Mailman 3 for the new buildbot-status mailing list
> and it
> > > works well:
> > >
> > >
> > https://mail.python.org/mm3/archives/list/buildbot-status@
> python.org/
> > >
> ...
> >
> > My only use of the pipermail archives is to find permanent URLs for
> > mails I want to refer people to. My usage goes as follows:
> >
> > 1. Google search for a post.
> > 2. Paste in the URL to an email.
> >
> > Or, if I have the post already (usually in my email client).
> >
> > 1. Check the date and subject of the post.
> > 2. Go to the pipermail article by month, and scan the list for the
> > subject and author.
> > 3. Click on the link, check it's the right email, copy the URL.
> > 4. Paste it into my email.
> >
> > I don't use the archives for reading. If the above two usages are
> > still available, I don't care. But in particular, the fact that
> > individual posts are searchable from Google is important to me.
>
>
> A Google search narrowed with "site:mail.python.org" and perhaps
> "inurl:listn...@python.org <javascript:;>" works for HyperKitty archives
> as well. Also,
> the archive itself has a "search this list" box.


Gmail also supports "list:python.org" now.


>
>
> ...> The complexity of this process is also very wastefully frustrating to
> > me. (Maybe it's in the next month's message tree? No fulltext search? No
> > way to even do an inurl: search because of the URIs?!)
>
>
> I don't see these issues. There is a full text search box on the archive
> page and I don't see the problem with Google inurl:


This URL style would work with inurl:

inurl:x.TLD/THREADID/msgid

These can't span across the year-month or otherwise catch other threads in
the result set:

inurl:mail.python.org/pipermail/astropy/2017-September/0001.html
inurl:mail.python.org/pipermail/astropy/2018-January/0002.html



>
>
> > Isn't there a way to append a permalink to the relayed message footers?
> > Google Groups and Github do this and it saves a lot of time.
>
>
> As you note below, there is an Archived-At: header. I have just
> submitted an RFE at <https://gitlab.com/mailman/mailman/issues/432> to
> enable placing this in the message header/footer.
>
>
> > [Re-searches for things]
> >
> > Mailman3 adds an RFC 5064 "Archived-At" header with a link that some
> > clients provide the ability to open in a normal human browser:
> >
> > http://dustymabe.com/2016/01/10/archived-at-email-header-
> from-mailman-3-lists/
> >
> > I often click the "view it on Github" link in GitHub issue emails. (It's
> > after the '--' email signature delimiter, so it doesn't take up so much
> > room).
> >
> > "[feature] Add permalink to mail message to the footer when delivering
> > email"
> > https://gitlab.com/mailman/hyperkitty/issues/27
>
>
> This needs to be in Mailman Core, not HyperKitty. As I note above, I
> filed an RFE with core and also referenced it in the HyperKitty issue


Thanks!


>
>
> > Finally, how would a transition be handled? I assume the old archives
> > would be retained, so would there be a cut-off date and people would
> > have to know to use the old or new archives based on the date of the
> > message?
> >
> >
> > Could an HTTP redirect help with directing users to the new or old
> archives?
>
>
> What we did when migrating security-sig is we migrated the archive but
> kept the old one and added this message and link to the old archive page.
>
> "This list has been migrated to Mailman 3. This archive is not being
> updated. Here is the new archive including these old posts."
>
> We also redirected
> <https://mail.python.org/mailman/listinfo/security-sig> to
> <https://mail.python.org/mm3/mailman3/lists/security-sig.python.org/>.
>
> We purposely didn't redirect the old archive so that saved URLs would
> still work.
>
> We did the same things for security-announce and clearly can do the same
> for future migrations.


Great.


>
> Finally note that Mailman 3 support

Re: [Python-Dev] iso8601 parsing

[Wes Turner]

On Thursday, October 26, 2017, Chris Barker <chris.bar...@noaa.gov> wrote:

> On Wed, Oct 25, 2017 at 7:37 PM, Wes Turner <wes.tur...@gmail.com
> <javascript:_e(%7B%7D,'cvml','wes.tur...@gmail.com');>> wrote:
>
>> ISO 8601 support offsets, but not time zones -- presumably the __str__
>>> supports the full datetime tzinfo somehow. Which may be why .isoformat()
>>> exists.
>>>
>>
>> ISO8601 does support timezones.
>> https://en.wikipedia.org/wiki/ISO_8601#Time_zone_designators
>>
>
> No, it doesn't -- it may call them "timezones", but it only supports
> offsets -- that is, and offset of -6 could be US Eastern Standard Time or
> US Central Daylight TIme (or I got that backwards :-)  )
>
> The point is that an offset is really easy, and timezones (with Daylight
> savings and all that) are a frickin' nightmare, but ARE supported by
> datetime
>
> Note that the vocabulary is not precise here, as I see this in the Pyton
> docs:
>
> *class *datetime.timezone
>
> A class that implements the tzinfo
> <https://docs.python.org/3/library/datetime.html#datetime.tzinfo> abstract
> base class as a fixed offset from the UTC.
> So THAT is supported by iso8601, and, indeed maps naturally to it.
>

Got it, thanks.


>
> Which means we can round trip isp8601 datetimes nicely, but can't round
> trip a datetime with a "full featured" tzinfo attached.
>

Because an iso8601 string only persists the offset.


>
> I don't think this really has any impact on the proposal, though: it's
> clear what to do when parsing a iso Datetime.
>
> I might be wrong, but I think many of the third party libraries listed
>> here default to either UTC or timezone-naieve timezones:
>> https://github.com/vinta/awesome-python/blob/master/README.
>> md#date-and-time
>>
>
> This is a key point that I hope is obvious:
>
>

> If an ISO string has NO offset or timezone indicator, then a naive
> datetime should be created.
>


>
> (I say, I "hope" it's obvious, because the numpy datetime64 implementation
> initially (and for years) would apply the machine local timezone to a bare
> iso string -- which was a f-ing nightmare!)
>

astropy.time.Time supports numpy.


>
>
>> Ctrl-F for 'tzinfo=' in the docs really doesn't explain how to just do it
>> with my local time.
>>
>> Here's an example with a *custom* GMT1 tzinfo subclass:
>> https://docs.python.org/3/library/datetime.html#datetime.time.tzname
>>
>
> Here it is:
>
> class GMT1(tzinfo):
> def utcoffset(self, dt):
> return timedelta(hours=1)
> def dst(self, dt):
> return timedelta(0)
> def tzname(self,dt):
> return "Europe/Prague"
>
> I hope Prague doesn't do DST, or that would be just wrong ...
>

Pendulum seems to have a faster timezone lookup than pytz:

https://pendulum.eustace.io/blog/a-faster-alternative-to-pyz.html

Both pendulum and pytz are in conda-forge (the new basis for the anaconda
distribution).


>
> What would you call the str argument? Does it accept strptime args or only
>> ISO8601?
>>
>
> I think Fred answered this, but iso 8601 only. we already have strptime if
> you need to parse anything else.
>
> Would all of that string parsing logic be a performance regression from
>> the current constructor? Does it accept None or empty string?
>>
>
> I suppose you need to do a little type checking first, so a tiny one.
>
> Though maybe just catching an Exception, so really tiny.
>
> The current constructor only takes numbers, so yes the string parsing
> version would be slower, but only if you use it...
>
> Deserializing dates from JSON (without #JSONLD and xsd:dateTime (ISO8601))
>> types is nasty, regardless (try/except, *custom* schema awareness). And
>> pickle is dangerous.
>>
>> AFAIU, we should not ever eval(repr(dt: datetime)).
>>
>
> why not? isn't that what __repr__ is supposed to do?
>

repr(dict) now returns ellipses ... for cyclical dicts; so I'm assuming
that repr only MAY be eval'able.


>
> Or do you mean not that it shouldn't work, but that we shouldn't do it?
>

That
We shouldn't ever eval untrusted data / code.
(That's why we need package hashes, signatures, and TUF).


>
> -CHB
>
>
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR(206) 526-6959   voice
> 7600 Sand Point Way NE
> <https://maps.google.com/?q=7600+Sand+Point+Way+NE=gmail=g>
>   (206) 526-6329   fax
> Seattle, WA  98115   (206) 526-6317   main reception
>
> chris.bar...@noaa.gov
> <javascript:_e(%7B%7D,'cvml','chris.bar...@noaa.gov');>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Migrate python-dev to Mailman 3?

[Wes Turner]

On Thursday, October 26, 2017, Paul Moore  wrote:

> On 26 October 2017 at 10:24, Victor Stinner  > wrote:
> > We are using Mailman 3 for the new buildbot-status mailing list and it
> > works well:
> >
> > https://mail.python.org/mm3/archives/list/buildbot-sta...@python.org/
> >
> > I prefer to read archives with this UI, it's simpler to follow
> > threads, and it's possible to reply on the web UI!
> >
> > To be honest, we got some issues when the new security-announce
> > mailing list was quickly migrated from Mailman 2 to Mailman 3, but
> > issues were quicky fixed as well.
> >
> > Would it be possible to migrate python-dev to Mailman 3? Do you see
> > any blocker issue?
> >
> > I sent to email to the Python postmaster as well.
>
> My only use of the pipermail archives is to find permanent URLs for
> mails I want to refer people to. My usage goes as follows:
>
> 1. Google search for a post.
> 2. Paste in the URL to an email.
>
> Or, if I have the post already (usually in my email client).
>
> 1. Check the date and subject of the post.
> 2. Go to the pipermail article by month, and scan the list for the
> subject and author.
> 3. Click on the link, check it's the right email, copy the URL.
> 4. Paste it into my email.
>
> I don't use the archives for reading. If the above two usages are
> still available, I don't care. But in particular, the fact that
> individual posts are searchable from Google is important to me. And in
> the second usage, having a single scrollable webpage with no
> extraneous data just subject/author and a bit of threading by
> indentation speeds up my usage a lot - the UI you linked to (and the
> monthly archive page with the initial lines of postings on it) is FAR
> too cluttered to be usable for my purposes.
>
>
This:


> So basically, what I'm asking is what would be the support for the use
> case "Find a permanent link to an archived article as fast as possible
> based on subject/author or a Google search".


The complexity of this process is also very wastefully frustrating to me.
(Maybe it's in the next month's message tree? No fulltext search? No way to
even do an inurl: search because of the URIs?!)

Isn't there a way to append a permalink to the relayed message footers?
Google Groups and Github do this and it saves a lot of time.

[Re-searches for things]

Mailman3 adds an RFC 5064 "Archived-At" header with a link that some
clients provide the ability to open in a normal human browser:

http://dustymabe.com/2016/01/10/archived-at-email-header-from-mailman-3-lists/

I often click the "view it on Github" link in GitHub issue emails. (It's
after the '--' email signature delimiter, so it doesn't take up so much
room).

"[feature] Add permalink to mail message to the footer when delivering
email"
https://gitlab.com/mailman/hyperkitty/issues/27


> Finally, how would a transition be handled? I assume the old archives
> would be retained, so would there be a cut-off date and people would
> have to know to use the old or new archives based on the date of the
> message?


Could an HTTP redirect help with directing users to the new or old archives?


>
> Paul
> ___
> Python-Dev mailing list
> Python-Dev@python.org 
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> wes.turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] iso8601 parsing

[Wes Turner]

On Wednesday, October 25, 2017, Chris Barker  wrote:

> On Wed, Oct 25, 2017 at 4:22 PM, Alexander Belopolsky <
> alexander.belopol...@gmail.com
> > wrote:
>
>> > times. The only difference is that instead of calling the type directly,
>> > you call the appropriate classmethod.
>> >
>> > What am I missing?
>>
>> Nothing. The only annoyance is that the data processing code typically
>> needs to know the type anyway, so the classmethod is one more variable
>> to keep track of.
>
>
> I don't think anyone is arguing that is is hard to do either way, or that
> hard to use either way.
>
> I think it comes down to a trade-off between:
>
> Having an API for datetime that is like the datetime for number types:
>
>  int(str(an_int)) == an_int
>
>  so:
>
>  datetime(str(a_datetime)) == a_datetime
>
> Alexander strongly supports that.
>
> Or an API that is perhaps more like the rest of the datetime api, which
> has a number of alternate constructors:
>
>  datetime.now()
>
>  datetime.fromordinal()
>
>  datetime.fromtimestamp()
>
> And has:
>
>   datetime.isoformat()
>
> So a
>
>datetime.fromisoformat()
>
> would make a lot of sense.
>
> I would note that the number types don't have all those alternate
> constructors Also, the number types mostly have a single parameter (except
> int has an optional base parameter). So I'm not sure the parallel is that
> strong.
>
> Would it be horrible if we did both?
>
> After all, right now, datetime has:
>
> In [16]: dt.isoformat()
> Out[16]: '2017-10-25T16:30:48.744489'
>
> and
> In [18]: dt.__str__()
> Out[18]: '2017-10-25 16:30:48.744489'
>
> which do almost the same thing (I understand the "T" is option in iso 8601)
>
> However, maybe they are different when you add a time zone?
>
> ISO 8601 support offsets, but not time zones -- presumably the __str__
> supports the full datetime tzinfo somehow. Which may be why .isoformat()
> exists.
>

ISO8601 does support timezones.
https://en.wikipedia.org/wiki/ISO_8601#Time_zone_designators

I might be wrong, but I think many of the third party libraries listed here
default to either UTC or timezone-naieve timezones:
https://github.com/vinta/awesome-python/blob/master/README.md#date-and-time

Ctrl-F for 'tzinfo=' in the docs really doesn't explain how to just do it
with my local time.

Here's an example with a *custom* GMT1 tzinfo subclass:
https://docs.python.org/3/library/datetime.html#datetime.time.tzname



> Though I don't think that means you couldn't have the __init__ parse
> proper ISO strings, in addition to the full datetime __str__ results.
>

What would you call the str argument? Does it accept strptime args or only
ISO8601? Would all of that string parsing logic be a performance regression
from the current constructor? Does it accept None or empty string?

...

Deserializing dates from JSON (without #JSONLD and xsd:dateTime (ISO8601))
types is nasty, regardless (try/except, *custom* schema awareness). And
pickle is dangerous.

AFAIU, we should not ever eval(repr(dt: datetime)).

...

Should the date time constructor support nanos= (just like time_ns())?

ENH: Add nanosecond support to the time and datetime constructors

...

The astropy Time class supports a string argument as the first parameter
sometimes:
http://docs.astropy.org/en/stable/time/#inferring-input-format

Astropy does support a "year zero".

>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 564: Add new time functions with nanosecond resolution

[Wes Turner]

On Tuesday, October 24, 2017, Antoine Pitrou <solip...@pitrou.net
<javascript:_e(%7B%7D,'cvml','solip...@pitrou.net');>> wrote:

> On Tue, 24 Oct 2017 09:00:45 +0200
> Victor Stinner <victor.stin...@gmail.com> wrote:
> > By the way, you mentionned that clocks are not synchronized. That's
> another
> > revelant point. Even if system clocks are synchronized on a single
> > computer, I read that you cannot reach nanosecond resolution for a NTP
> > synchronization even in a small LAN.
> >
> > For large systems or distributed systems, a "global (synchronized) clock"
> > is not an option. You cannot synchronize clocks correctly, so your
> > algorithms must not rely on time, or at least not too precise resolution.
> >
> > I am saying that to again repeat that we are far from sub-second
> nanosecond
> > resolution for system clock.
>
> What does synchronization have to do with it?  If synchronization
> matters, then your PEP should be rejected, because current computers
> using NTP can't synchronize with a better precision than 230 ns.

>From https://en.wikipedia.org/wiki/Virtual_black_hole :

> In the derivation of his equations, Einstein suggested that physical
space-time is Riemannian, ie curved. A small domain of it is approximately
flat space-time.


>From https://en.wikipedia.org/wiki/Quantum_foam :

> Based on the uncertainty principles of quantum mechanics and the general
theory of relativity, there is no reason that spacetime needs to be
fundamentally smooth. Instead, in a quantum theory of gravity, spacetime
would consist of many small, ever-changing regions in which space and time
are not definite, but fluctuate in a foam-like manner.

So, in regards to time synchronization, FWIU:

- WWVB "can provide time with an accuracy of about 100 microseconds"

- GPS time can synchronize down to "tens of nanoseconds"

- Blockchains work around local timestamp issues by "enforcing" linearity


>
> See https://blog.cloudflare.com/how-to-achieve-low-latency/


> Regards
>
> Antoine.
>
>
> _______
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/wes.
> turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 564: Add new time functions with nanosecond resolution

[Wes Turner]

On Monday, October 23, 2017, Thomas Jollans <t...@tjol.eu> wrote:

> On 22/10/17 17:06, Wes Turner wrote:
> > There are current applications with greater-than nanosecond precision:
> >
> > - relativity experiments
> > - particle experiments
> >
> > Must they always use their own implementations of time., datetime.
> > __init__, fromordinal, fromtimestamp ?!
> >
> > - https://scholar.google.com/scholar?q=femtosecond
> > - https://scholar.google.com/scholar?q=attosecond
> > - GPS now supports nanosecond resolution
> > -
>
> Sure, but in these kinds of experiments you don't have a "timestamp" in
> the usual sense.
>
> You'll have some kind of high-precision "clock", but in most cases
> there's no way and no reason to synchronise this to wall time. You end
> up distinguishing between "macro-time" (wall time) and "micro-time"
> (time in the experiment relative to something)
>
> In a particle accelerator, you care about measuring relative times of
> almost-simultaneous detection events with extremely high precision.
> You'll also presumably have a timestamp for the event, but you won't be
> able or willing to measure that with anything like the same accuracy.
>
> While you might be able to say that you detected, say, a muon at
> 01:23:45.6789 at Δt=543.6ps*, you have femtosecond resolution, you have
> a timestamp, but you don't have a femtosecond timestamp.
>
> In ultrafast spectroscopy, we get a time resolution equal to the
> duration of your laser pulses (fs-ps), but all the micro-times measured
> will be relative to some reference laser pulse, which repeats at >MHz
> frequencies. We also integrate over millions of events - wall-time
> timestamps don't enter into it.
>
> In summary, yes, when writing software for experiments working with high
> time resolution you have to write your own implementations of whatever
> data formats best describe time as you're measuring it, which generally
> won't line up with time as a PC (or a railway company) looks at it.


(Sorry, maybe too OT)

So these experiments are all done in isolation; referent to t=0.

> Aligning simulation data in context to other events may be enlightening:


IIUC,
https://en.wikipedia.org/wiki/Quantum_mechanics_of_time_travel implies that
there are (or may) Are potentially connections between events over greater
periods of time.

It's unfortunate that aligning this data requires adding offsets and
working with nonstandard adhoc time structs.

A problem for another day, I suppose.

Thanks for adding time_ns(l.


> Cheers
> Thomas
>
>
> * The example is implausible not least because I understand muon
> chambers tend to be a fair bit bigger than 15cm, but you get my point.
> ___
> Python-Dev mailing list
> Python-Dev@python.org <javascript:;>
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> wes.turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 564: Add new time functions with nanosecond resolution

[Wes Turner]

On Sunday, October 22, 2017, David Mertz <me...@gnosis.cx> wrote:

> I worked at a molecular dynamics lab for a number of years. I advocated
> switching all our code to using attosecond units (rather than fractional
> picoseconds).
>
> However, this had nothing whatsoever to do with the machine clock speeds,
> but only with the physical quantities represented and the scaling/rounding
> math.
>
> It didn't happen, for various reasons. But if it had, I certainly wouldn't
> have expected standard library support for this. The 'time' module is about
> wall clock out calendar time, not about *simulation time*.
>
> FWIW, a very long simulation might cover a millisecond of simulated
> time we're a very long way from looking at molecular behavior over 104
> days.
>

Maybe that's why we haven't found any CTCs (closed timelike curves) yet.

Aligning simulation data in context to other events may be enlightening: is
there a good library for handing high precision time units in Python
(and/or CFFI)?

...

http://opendata.cern.ch/

http://opendata.cern.ch/getting-started/CMS


>
>
> On Oct 22, 2017 8:10 AM, "Wes Turner" <wes.tur...@gmail.com
> <javascript:_e(%7B%7D,'cvml','wes.tur...@gmail.com');>> wrote:
>
>
>
> On Saturday, October 21, 2017, Nick Coghlan <ncogh...@gmail.com
> <javascript:_e(%7B%7D,'cvml','ncogh...@gmail.com');>> wrote:
>
>> On 22 October 2017 at 09:32, Victor Stinner <victor.stin...@gmail.com>
>> wrote:
>>
>>> Le 21 oct. 2017 20:31, "francismb" <franci...@email.de> a écrit :
>>>
>>> I understand that one can just multiply/divide the nanoseconds returned,
>>> (or it could be a factory) but wouldn't it help for future enhancements
>>> to reduce the number of functions (the 'pico' question)?
>>>
>>>
>>> If you are me to predict the future, I predict that CPU frequency will
>>> be stuck below 10 GHz for the next 10 years :-)
>>>
>>
>> There are actually solid physical reasons for that prediction likely
>> being true. Aside from the power consumption, heat dissipation, and EM
>> radiation issues that arise with higher switching frequencies, you also
>> start running into more problems with digital circuit metastability ([1],
>> [2]): the more clock edges you have per second, the higher the chances of
>> an asynchronous input changing state at a bad time.
>>
>> So yeah, for nanosecond resolution to not be good enough for programs
>> running in Python, we're going to be talking about some genuinely
>> fundamental changes in the nature of computing hardware, and it's currently
>> unclear if or how established programming languages will make that jump
>> (see [3] for a gentle introduction to the current state of practical
>> quantum computing). At that point, picoseconds vs nanoseconds is likely to
>> be the least of our conceptual modeling challenges :)
>>
>
> There are current applications with greater-than nanosecond precision:
>
> - relativity experiments
> - particle experiments
>
> Must they always use their own implementations of time., datetime.
> __init__, fromordinal, fromtimestamp ?!
>
> - https://scholar.google.com/scholar?q=femtosecond
> - https://scholar.google.com/scholar?q=attosecond
> - GPS now supports nanosecond resolution
> -
>
> https://en.wikipedia.org/wiki/Quantum_clock#More_accurate_ex
> perimental_clocks
>
> > In 2015 JILA <https://en.m.wikipedia.org/wiki/JILA> evaluated the
> absolute frequency uncertainty of their latest strontium-87
> <https://en.m.wikipedia.org/wiki/Isotopes_of_strontium> optical lattice
> clock at 2.1 × 10−18, which corresponds to a measurable gravitational
> time dilation
> <https://en.m.wikipedia.org/wiki/Gravitational_time_dilation> for an
> elevation change of 2 cm (0.79 in)
>
> What about bus latency (and variance)?
>
> From https://www.nist.gov/publications/optical-two-way-time-and-
> frequency-transfer-over-free-space :
>
> > Optical two-way time and frequency transfer over free space
> > Abstract
> > The transfer of high-quality time-frequency signals between remote
> locations underpins many applications, including precision navigation and
> timing, clock-based geodesy, long-baseline interferometry, coherent radar
> arrays, tests of general relativity and fundamental constants, and future
> redefinition of the second. However, present microwave-based time-frequency
> transfer is inadequate for state-of-the-art optical clocks and oscillators
> that have femtosecond-level timing jitter and accuracies below 1 × 10−17.
> Commensurate optically based transfer methods a

Re: [Python-Dev] PEP 564: Add new time functions with nanosecond resolution

[Wes Turner]

On Saturday, October 21, 2017, Nick Coghlan  wrote:

> On 22 October 2017 at 09:32, Victor Stinner  > wrote:
>
>> Le 21 oct. 2017 20:31, "francismb" > > a écrit :
>>
>> I understand that one can just multiply/divide the nanoseconds returned,
>> (or it could be a factory) but wouldn't it help for future enhancements
>> to reduce the number of functions (the 'pico' question)?
>>
>>
>> If you are me to predict the future, I predict that CPU frequency will be
>> stuck below 10 GHz for the next 10 years :-)
>>
>
> There are actually solid physical reasons for that prediction likely being
> true. Aside from the power consumption, heat dissipation, and EM radiation
> issues that arise with higher switching frequencies, you also start running
> into more problems with digital circuit metastability ([1], [2]): the more
> clock edges you have per second, the higher the chances of an asynchronous
> input changing state at a bad time.
>
> So yeah, for nanosecond resolution to not be good enough for programs
> running in Python, we're going to be talking about some genuinely
> fundamental changes in the nature of computing hardware, and it's currently
> unclear if or how established programming languages will make that jump
> (see [3] for a gentle introduction to the current state of practical
> quantum computing). At that point, picoseconds vs nanoseconds is likely to
> be the least of our conceptual modeling challenges :)
>

There are current applications with greater-than nanosecond precision:

- relativity experiments
- particle experiments

Must they always use their own implementations of time., datetime.
__init__, fromordinal, fromtimestamp ?!

- https://scholar.google.com/scholar?q=femtosecond
- https://scholar.google.com/scholar?q=attosecond
- GPS now supports nanosecond resolution
-

https://en.wikipedia.org/wiki/Quantum_clock#More_accurate_experimental_clocks

> In 2015 JILA  evaluated the
absolute frequency uncertainty of their latest strontium-87
 optical lattice
clock at 2.1 × 10−18, which corresponds to a measurable gravitational time
dilation  for
an elevation change of 2 cm (0.79 in)

What about bus latency (and variance)?

From
https://www.nist.gov/publications/optical-two-way-time-and-frequency-transfer-over-free-space
:

> Optical two-way time and frequency transfer over free space
> Abstract
> The transfer of high-quality time-frequency signals between remote
locations underpins many applications, including precision navigation and
timing, clock-based geodesy, long-baseline interferometry, coherent radar
arrays, tests of general relativity and fundamental constants, and future
redefinition of the second. However, present microwave-based time-frequency
transfer is inadequate for state-of-the-art optical clocks and oscillators
that have femtosecond-level timing jitter and accuracies below 1 × 10−17.
Commensurate optically based transfer methods are therefore needed. Here we
demonstrate optical time-frequency transfer over free space via two-way
exchange between coherent frequency combs, each phase-locked to the local
optical oscillator. We achieve 1 fs timing deviation, residual instability
below 1 × 10−18 at 1,000 s and systematic offsets below 4 × 10−19, despite
frequent signal fading due to atmospheric turbulence or obstructions across
the 2 km link. This free-space transfer can enable terrestrial links to
support clock-based geodesy. Combined with satellite-based optical
communications, it provides a path towards global-scale geodesy,
high-accuracy time-frequency distribution and satellite-based relativity
experiments.

How much wider must an epoch-relative time struct be for various realistic
time precisions/accuracies?

10-6 micro µ
10-9 nano n -- int64
10-12 pico p
10-15 femto f
10-18 atto a
10-21 zepto z
10-24 yocto y

I'm at a loss to recommend a library to prefix these with the epoch; but
future compatibility may be a helpful, realistic objective.

Natural keys with such time resolution are still unfortunately likely to
collide.


>
> Cheers,
> Nick.
>
> [1] https://en.wikipedia.org/wiki/Metastability_in_electronics
> [2] https://electronics.stackexchange.com/questions/
> 14816/what-is-metastability
> [3] https://medium.com/@decodoku/how-to-program-a-quantum-
> computer-982a9329ed02
>
>
> --
> Nick Coghlan   |   ncogh...@gmail.com
>    |   Brisbane,
> Australia
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] To reduce Python "application" startup time

[Wes Turner]

On Wednesday, September 6, 2017, INADA Naoki  wrote:

> > How significant is application startup time to something that uses
> > Jinja2? Are there short-lived programs that use it? Python startup
> > time matters enormously to command-line tools like Mercurial, but far
> > less to something that's designed to start up and then keep running
> > (eg a web app, which is where Jinja is most used).
>
> Since Jinja2 is very popular template engine, it is used by CLI tools
> like ansible.


SaltStack uses Jinja2. It really is a good idea to regularly restart the
minion processes.

Celery can also cycle through worker processes, IIRC.


>
> Additionally, faster startup time (and smaller memory footprint) is good
> for even Web applications.
> For example, CGI is still comfortable tool sometimes.
> Another example is GAE/Python.


Short-lived processes are sometimes preferable from a security standpoint.
Python is currently less viable for CGI use than other scripting languages
due to startup time.

Resource leaks (e.g. memory, file handles, database references; valgrind)
do not last w/ short-lived CGI processes. If there's ASLR, that's also
harder.

Scale up operations with e.g. IaaS platforms like Kubernetes and PaaS
platforms like AppScale all incur Python startup time on a regular basis.


>
> Anyway, I think researching import tree of popular library is good
> startline
> about optimizing startup time.
> For example, modules like ast and tokenize are imported often than I
> thought.
>
> Jinja2 is one of libraries I often use. I'm checking other libraries
> like requests.


> Thanks,
>
> INADA Naoki  >
> ___
> Python-Dev mailing list
> Python-Dev@python.org 
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> wes.turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] HTTPS on bugs.python.org

[Wes Turner]

Here's e.g. Jupyter Notebook w/ letsencrypt in a Makefile:

https://github.com/jupyter/docker-stacks/blob/master/examples/make-deploy/letsencrypt.makefile

... https://github.com/jupyter/docker-stacks

On Fri, Sep 1, 2017 at 9:08 AM, Wes Turner <wes.tur...@gmail.com> wrote:

>
> ## HTTP STS
> - Wikipedia: https://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security
> - Docs: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/
> Strict-Transport-Security
>
> - https://https.cio.gov/hsts/
>
> ## letsencrypt
> "A free, automated, and open certificate authority."
>
> - Wikipedia: https://en.wikipedia.org/wiki/Let%27s_Encrypt
> - Homepage: https://letsencrypt.org/
> - Src: https://github.com/letsencrypt
> - Docs: https://letsencrypt.readthedocs.io/en/latest/
> - Docs: https://letsencrypt.readthedocs.io/en/latest/using.html#getting-
> certificates-and-choosing-plugins
> - Docs: https://letsencrypt.readthedocs.io/en/latest/
> using.html#third-party-plugins
>
> ### ACME Protocol
> - Wikipedia: https://en.wikipedia.org/wiki/Automated_
> Certificate_Management_Environment
>
>
>
>
> On Fri, Sep 1, 2017 at 8:35 AM, Mariatta Wijaya <mariatta.wij...@gmail.com
> > wrote:
>
>> I also would like the links from bug tracker emails be in https instead
>> of http.
>>
>>
>>
>> On Sep 1, 2017 6:31 AM, "Antoine Pitrou" <solip...@pitrou.net> wrote:
>>
>>> On Fri, 1 Sep 2017 22:15:29 +0900
>>> INADA Naoki <songofaca...@gmail.com> wrote:
>>> > FYI, there is issue report for it.
>>> > http://psf.upfronthosting.co.za/roundup/meta/issue463
>>> > INADA Naoki  <songofaca...@gmail.com>
>>>
>>> That issue is about making the tracker HTTPS-only, but fixing
>>> internal links to point to the HTTPS site would already go a long way,
>>> even without switching off HTTP access.
>>>
>>> Regards
>>>
>>> Antoine.
>>>
>>>
>>> ___
>>> Python-Dev mailing list
>>> Python-Dev@python.org
>>> https://mail.python.org/mailman/listinfo/python-dev
>>> Unsubscribe: https://mail.python.org/mailma
>>> n/options/python-dev/mariatta.wijaya%40gmail.com
>>>
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: https://mail.python.org/mailman/options/python-dev/wes.
>> turner%40gmail.com
>>
>>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com