Re: [Python-Dev] Python FTP Injections Allow for Firewall Bypass (oss-security advisory)

[Martin Panter]

On 24 February 2017 at 07:51, Benjamin Peterson  wrote:
> As for this, particular issue, we should determine if there's a tracker
> issue yet and continue discussion there.

That would be .
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 543: A Unified TLS API for Python

[Nick Coghlan]

On 14 February 2017 at 00:26, Cory Benfield  wrote:

> All,
>
> Python has never exposed the ability to use TLS backends other than
> OpenSSL in the standard library. As Christian and I discussed back at the
> Developer Summit at PyCon US 2016, the ssl module would more properly be
> called the openssl module, due to exposing many OpenSSL-specific concepts
> and behaviours in its API. This has meant that the Python ecosystem is
> overwhelmingly also an OpenSSL ecosystem, which is problematic on Windows
> and macOS (and Unices that for whatever reason aren’t interested in
> shipping an OpenSSL), as it has meant that Python needs to bring its own
> OpenSSL, and that it is troublesome to interact with the system trust
> database.
>
> The first step to resolving this would be to provide a new module that
> exposes TLS concepts in a much more generic manner. There are two possible
> approaches to this. The first is to have this module be a generic concrete
> implementation that can be compiled against multiple TLS backends (like
> curl). This would require that all the relevant bindings be built into the
> standard library from the get-go, which provides a substantial maintenance
> burden on a team of people that are already understaffed maintaining the
> ssl module alone. The second approach is to define a generic high-level TLS
> interface that provides a minimal usable interface that can be implemented
> by both first- and third-party modules. This would allow the standard
> library to continue to ship with only exactly what it needs (for example,
> OpenSSL, SecureTransport and SChannel), but would give those who want to
> use more esoteric TLS choices (NSS, GnuTLS, mbedTLS, and BoringSSL are some
> examples) an API that they can implement that will guarantee that complying
> modules can use the appropriate TLS backend.
>
> To that end, I’ve composed a draft PEP that would define this API: PEP
> 543. This draft can be found online here[1], and I have reproduced it below
> for those who want to reply inline. I should apologise in advance that this
> PEP is quite large: it is this large because it lays out a substantial
> proportion of the new code that would be added directly in the PEP. This is
> to help de-risk the work by showing as much of it up-front during the PEP
> stage, rather than having it emerge over the implementation process.
>
> Please note that this proposal is not intended to resolve the current
> problem pip has with TLSv1.2, and so is not subject to the tight time
> constraints associated with that problem. That problem will be solved by
> building a temporary shim into urllib3 (and therefore pip) that can use
> SecureTransport bindings on the Mac. This proposal is intended as a
> solution to the more-general problem of supporting different TLS backends
> in Python, and so is larger in scope and less urgent.
>
> I should also mention that there will be a tendency to want to make this
> API all things to all people from the get-go. I’m going to strongly resist
> attempts to extend this API too much, because each additional bit of API
> surface makes it harder for us to encourage module authors to conform to
> this API. I will encourage people to extend this API over time as needed,
> but to begin with I think it is most important that basic TLS clients and
> servers can be constructed with this API. More specialised features should
> be considered future enhancements, rather than being tacked on to this
> initial PEP.
>
> Please let me know what you think. I’d like significant feedback from the
> community before I ask the BDFL to pronounce on this PEP, as this is the
> kind of change that will affect a large amount of the Python ecosystem.
> It’s important to me that we minimise the risk to ourselves and the
> community by getting this as right as possible, and while we can help with
> that by limiting scope, it’s also true that we need as many eyes as
> possible.
>
> Please let me know what you think.
>

Heh, I guess you must have already caught everyone with a strong opinion
about this by way of security-sig :)

On the procedural front, the main open question is whether or not Guido
wants to review and approve this PEP himself, or if he'd prefer to delegate
that task.

Assuming the latter case, I think it may make sense for Christian to handle
the BDFL-Delegate responsibilities as well - I know he's a co-author of the
PEP, but I also think he's the most qualified to make the final decision on
the suitability of this design.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python FTP Injections Allow for Firewall Bypass (oss-security advisory)

[Antoine Pitrou]

On Thu, 23 Feb 2017 23:51:45 -0800
Benjamin Peterson  wrote:
> 
> Like all CPython developers, the Python security team are all
> volunteers. That combined with the fact that dealing with security
> issues is one of the least fun programming tasks means issues are
> sometimes dropped.
> 
> Perhaps some organization with a stake Python security would like to
> financially support Python security team members.
> 
> As for this, particular issue, we should determine if there's a tracker
> issue yet and continue discussion there.

Just for the record, I find the mailing-list scheme used by PSRT quite
difficult to deal with.  For many people it's easy to lose track of
e-mails received more than one week ago, so the necessary followup to
security issues received by e-mail suffers.

It's a bit sad that regular issues benefit from a full-fledged
Roundup instance to allow for easy tracking of open issues (including
comments and proposed fixes), but security issues are restricted to such
a primitive communication setup which makes it so difficult to get work
done.

AFAIK, other projects have full-fledged private bug trackers for their
security issues (or access-restricted sections in the main bug tracker,
where the software supports it).

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Translated Python documentation

[Nick Coghlan]

On 23 February 2017 at 02:15, Victor Stinner 
wrote:

> 2017-02-22 16:40 GMT+01:00 Antoine Pitrou :
> > As long as you are asking for "moral" support and not actually
> > vouching for the accuracy of third-party translations, then +1 from me.
>
> The main complain about these translations is the accuracy.
>
> My bet is that making these translations "official" and more visible
> (at docs.python.org) would make them more popular, and so indirectly
> help to recruit new contributors. Slowly, the number of translated
> pages should increase, but the overall translation quality should also
> increase. That's how free software are developed, no? :-)
>

+1 from me for these reasons, and those Facundo gives: we want folks to be
able to learn at least the basics of Python *before* they learn English
(even if learning English remains a pre-requisite for tapping into the full
capabilities of both the language and its ecosystem).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 543: A Unified TLS API for Python

[Antoine Pitrou]

On Fri, 24 Feb 2017 19:55:20 +1000
Nick Coghlan  wrote:
> 
> On the procedural front, the main open question is whether or not Guido
> wants to review and approve this PEP himself, or if he'd prefer to delegate
> that task.
> 
> Assuming the latter case, I think it may make sense for Christian to handle
> the BDFL-Delegate responsibilities as well - I know he's a co-author of the
> PEP, but I also think he's the most qualified to make the final decision on
> the suitability of this design.

Obviously not speaking for Guido, but I trust Christian to make a good
judgement on this PEP.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Report on Python vulnerabilities

[Nick Coghlan]

On 23 February 2017 at 11:15, Victor Stinner 
wrote:

> Hi,
>
> I created a tool to generate a report on Python vulnerabilities:
>
>http://python-security.readthedocs.io/vulnerabilities.html
>
> I collected data of 41 vulnerabilities since 2007 (first Python CVE:
> CVE-2007-4965).
>

Very handy!


> If you would like to add data of a new vulnerability, enhance the
> report, ... : see the GitHub project.
>
>https://github.com/haypo/python-security
>
> The main data file is vulnerabilities.yml (YAML). I also filled
> manually the python_releases.txt: file: list of all release dates from
> Python 2.5.0 to Python 3.6.0.
>
> The tool compute the first Python release in each branch which
> includes the fix from a list of commits.
>

The main idea that comes to mind is finding a way to add a "Fixed In"
column to the summary table to get a quick overview of which versions were
affected.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 543: A Unified TLS API for Python

[Cory Benfield]

> On 24 Feb 2017, at 09:55, Nick Coghlan  wrote:
> 
> Heh, I guess you must have already caught everyone with a strong opinion 
> about this by way of security-sig :)

So it seems. =D

> On the procedural front, the main open question is whether or not Guido wants 
> to review and approve this PEP himself, or if he'd prefer to delegate that 
> task.
> 
> Assuming the latter case, I think it may make sense for Christian to handle 
> the BDFL-Delegate responsibilities as well - I know he's a co-author of the 
> PEP, but I also think he's the most qualified to make the final decision on 
> the suitability of this design.

Either way, I think I’d like to confirm this works by writing two more modules. 
First, I’d like to actually write most of the tls module into something that 
can live on PyPI, in no small part because backporting it would be useful for 
tools like urllib3. Secondly, I’d like to write a shim for the standard library 
ssl module so that I have a clearer picture of whether we want to shim it in 
Python code or whether we should take this opportunity to write new bindings to 
the C code.

The goal here, essentially, is to have a PoC that the API is at least suitable 
for tools like urllib3/requests before we commit to it too much, and the best 
way to do that is to prove that the stdlib can meet that API and that at least 
one other implementation can do as well. I’ll use SecureTransport for that 
proof point, but I’ll make available a branch of urllib3 that has patches 
applied to use the new API so that anyone who wants to investigate shimming a 
different TLS implementation has got a test suite they can try to run.

 Obviously that’s slightly orthogonal to PEP acceptance, but I wanted to keep 
people up-to-speed with my thinking.

Cory___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 543: A Unified TLS API for Python

[Nick Coghlan]

On 24 February 2017 at 20:21, Cory Benfield  wrote:

>
> Either way, I think I’d like to confirm this works by writing two more
> modules. First, I’d like to actually write most of the tls module into
> something that can live on PyPI, in no small part because backporting it
> would be useful for tools like urllib3. Secondly, I’d like to write a shim
> for the standard library ssl module so that I have a clearer picture of
> whether we want to shim it in Python code or whether we should take this
> opportunity to write new bindings to the C code.
>
> The goal here, essentially, is to have a PoC that the API is at least
> suitable for tools like urllib3/requests before we commit to it too much,
> and the best way to do that is to prove that the stdlib can meet that API
> and that at least one other implementation can do as well. I’ll use
> SecureTransport for that proof point, but I’ll make available a branch of
> urllib3 that has patches applied to use the new API so that anyone who
> wants to investigate shimming a different TLS implementation has got a test
> suite they can try to run.
>
>  Obviously that’s slightly orthogonal to PEP acceptance, but I wanted to
> keep people up-to-speed with my thinking.
>

Having a sufficiently complete reference implementation in place to
convince both the author(s) and the reviewers that the proposal solves the
problem it claims to solve is part of the PEP process, so your planned
approach makes a lot of sense to me.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python FTP Injections Allow for Firewall Bypass (oss-security advisory)

[tritium-list]

Ask the infrastructure team for a tracker instance.  That would probably be
more fruitful of an outlet than in the thread of this one issue.  (I'm not
trying to be flippant, I think a private issue tracker for vulnerabilities
is a really good idea, I just don't think that bemoaning the lack of one in
a thread about an FTP issue is likely to get much done.)

> -Original Message-
> From: Python-Dev [mailto:python-dev-bounces+tritium-
> list=sdamon@python.org] On Behalf Of Antoine Pitrou
> Sent: Friday, February 24, 2017 5:02 AM
> To: python-dev@python.org
> Subject: Re: [Python-Dev] Python FTP Injections Allow for Firewall Bypass
> (oss-security advisory)
> 
> On Thu, 23 Feb 2017 23:51:45 -0800
> Benjamin Peterson  wrote:
> >
> > Like all CPython developers, the Python security team are all
> > volunteers. That combined with the fact that dealing with security
> > issues is one of the least fun programming tasks means issues are
> > sometimes dropped.
> >
> > Perhaps some organization with a stake Python security would like to
> > financially support Python security team members.
> >
> > As for this, particular issue, we should determine if there's a tracker
> > issue yet and continue discussion there.
> 
> Just for the record, I find the mailing-list scheme used by PSRT quite
> difficult to deal with.  For many people it's easy to lose track of
> e-mails received more than one week ago, so the necessary followup to
> security issues received by e-mail suffers.
> 
> It's a bit sad that regular issues benefit from a full-fledged
> Roundup instance to allow for easy tracking of open issues (including
> comments and proposed fixes), but security issues are restricted to such
> a primitive communication setup which makes it so difficult to get work
> done.
> 
> AFAIK, other projects have full-fledged private bug trackers for their
> security issues (or access-restricted sections in the main bug tracker,
> where the software supports it).
> 
> Regards
> 
> Antoine.
> 
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/tritium-
> list%40sdamon.com

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Translated Python documentation

[tritium-list]

My gut splits the difference on this issue; I suggest an approach to meet in 
the middle – a version of the docs written in simplified English (Not quite Up 
Goer Five simplified, but simplified.)  It has the practical benefit of having 
more eyes to look at the docs to check for accuracy, it targets both English 
learners and children, and is something the current contributors to the 
documentation can do.

 

For any language you want to support other than English, you need a translator 
who is A: a python expert, B: fluent in English, and C: fluent in the target 
language.  …And then you need another one to check what was written.  These are 
practical problems.  There are extant services to support this, they are 
expensive in either money or time, and the docs produced usually lag behind 
English quite a bit.

 

From: Python-Dev [mailto:python-dev-bounces+tritium-list=sdamon@python.org] 
On Behalf Of Nick Coghlan
Sent: Friday, February 24, 2017 5:02 AM
To: Victor Stinner 
Cc: Antoine Pitrou ; Python Dev 
Subject: Re: [Python-Dev] Translated Python documentation

 

On 23 February 2017 at 02:15, Victor Stinner mailto:victor.stin...@gmail.com> > wrote:

2017-02-22 16:40 GMT+01:00 Antoine Pitrou mailto:solip...@pitrou.net> >:
> As long as you are asking for "moral" support and not actually
> vouching for the accuracy of third-party translations, then +1 from me.

The main complain about these translations is the accuracy.

My bet is that making these translations "official" and more visible
(at docs.python.org  ) would make them more popular, 
and so indirectly
help to recruit new contributors. Slowly, the number of translated
pages should increase, but the overall translation quality should also
increase. That's how free software are developed, no? :-)

 

+1 from me for these reasons, and those Facundo gives: we want folks to be able 
to learn at least the basics of Python *before* they learn English (even if 
learning English remains a pre-requisite for tapping into the full capabilities 
of both the language and its ecosystem).

Cheers,

Nick.



-- 

Nick Coghlan   |   ncogh...@gmail.com |   
Brisbane, Australia

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Translated Python documentation

[Jim F.Hilliard]

I've recently been translating the Tutorial in Greek as a side thing that
also helps me catch some doc bugs :-). So it's an obvious +1 from me.

I really like this idea and would like to see it come to fruition. I think
some issues need to be addressed first, though:

Where should these translated docs live and how does one make it clear to
users reading them that doc bugs shouldn't be submitted to the main bug
tracker/github repo?

these are just two that popped in my head.

It would be great if this went forward, many people will find the path to
learning Python much much easier to walk if the documentation was written
in a language they are comfortable in.



Best Regards,

Jim Fasarakis Hilliard
*d.f.hilli...@gmail.com  | dimitrisjim.github.io
*


  




On Fri, Feb 24, 2017 at 12:02 PM, Nick Coghlan  wrote:

> On 23 February 2017 at 02:15, Victor Stinner 
> wrote:
>
>> 2017-02-22 16:40 GMT+01:00 Antoine Pitrou :
>> > As long as you are asking for "moral" support and not actually
>> > vouching for the accuracy of third-party translations, then +1 from me.
>>
>> The main complain about these translations is the accuracy.
>>
>> My bet is that making these translations "official" and more visible
>> (at docs.python.org) would make them more popular, and so indirectly
>> help to recruit new contributors. Slowly, the number of translated
>> pages should increase, but the overall translation quality should also
>> increase. That's how free software are developed, no? :-)
>>
>
> +1 from me for these reasons, and those Facundo gives: we want folks to be
> able to learn at least the basics of Python *before* they learn English
> (even if learning English remains a pre-requisite for tapping into the full
> capabilities of both the language and its ecosystem).
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/d.
> f.hilliard%40gmail.com
>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Translated Python documentation

[Jim F.Hilliard]

I've recently been translating the Tutorial in Greek as a side thing that
also helps me catch some doc bugs :-). So it's an obvious +1 from me.

I really like this idea and would like to see it come to fruition. I think
some issues need to be addressed first, though:

Where should these translated docs live and how does one make it clear to
users reading them that doc bugs shouldn't be submitted to the main bug
tracker/github repo?

these are just two that popped in my head.

It would be great if this went forward, many people will find the path to
learning Python much much easier to walk if, at least the tutorial, was
written in a language they are comfortable in.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Translated Python documentation

[INADA Naoki]

> Where should these translated docs live and how does one make it clear to
> users reading them that doc bugs shouldn't be submitted to the main bug
> tracker/github repo?
>

We setup github page.  See https://python-doc-ja.github.io/py36/ (note that
version switcher won't work because this html is build for docs.python.jp.)

I reserved "python-docs" organization.  So we can use URL like
https://python-doc.github.io//3.6/
(Because the organization name looks "something official", I won't use it
until we get consensus about it.)

For issue tracker, bugs.html must mention about translated document, before
mention to normal issue tracker.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python FTP Injections Allow for Firewall Bypass (oss-security advisory)

[Christian Heimes]

On 2017-02-24 11:01, Antoine Pitrou wrote:
> On Thu, 23 Feb 2017 23:51:45 -0800
> Benjamin Peterson  wrote:
>>
>> Like all CPython developers, the Python security team are all
>> volunteers. That combined with the fact that dealing with security
>> issues is one of the least fun programming tasks means issues are
>> sometimes dropped.
>>
>> Perhaps some organization with a stake Python security would like to
>> financially support Python security team members.
>>
>> As for this, particular issue, we should determine if there's a tracker
>> issue yet and continue discussion there.
> 
> Just for the record, I find the mailing-list scheme used by PSRT quite
> difficult to deal with.  For many people it's easy to lose track of
> e-mails received more than one week ago, so the necessary followup to
> security issues received by e-mail suffers.
> 
> It's a bit sad that regular issues benefit from a full-fledged
> Roundup instance to allow for easy tracking of open issues (including
> comments and proposed fixes), but security issues are restricted to such
> a primitive communication setup which makes it so difficult to get work
> done.
> 
> AFAIK, other projects have full-fledged private bug trackers for their
> security issues (or access-restricted sections in the main bug tracker,
> where the software supports it).

Amen!

Antoine's and Benjamin's reply are the gist of my security talk at the
last language summit, https://lwn.net/Articles/691308/ . A dedicated bug
tracker or embargoed tickets would help the most. It would also make it
much easier to track and measure our response time.

A paid position would also help with the organizational overhead.
Personally, I'm good in finding and fixing security issues. The actual
communication, reporting and press releases are not my strength.

Victor's incredible work on
http://python-security.readthedocs.io/vulnerabilities.html is going to
help, too.

Christian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Can I commit a hack for translation? (Was: Translated Python documentation

[INADA Naoki]

Thanks, Victor and all.
This thread is very encouraging for me.

Currently, I have a suspended pull request:
https://github.com/python/cpython/pull/195

This pull request (1) fixes "CPython implementation detail:" label is
disappear when it's body is translated, and
(2) make the label translatable.

For (2), I managed to "make gettext" in Doc/ extract the label text
into "sphinx.pot".
So we can translate the label at Transifex.
But I used ugly hack for it.  This pull request "dummy.html" file
which isn't used for building html.
It only contains "{% trans %}CPython implementation detail:{% endtrans %}".

We already did the hack in downstream fork of Japanese translation team.
https://github.com/python-doc-ja/cpython-doc-intl/blob/intl-3.6/Doc/tools/templates/dummy.html

There are one upvote and one downvote for the pull request.  This is
why I suspend the pull request.
Please vote!

a) Commit all.

b) Commit (1), and call gettext() for the "CPython implementation detail:".
   (downstream fork should only add dummy.html).

c) Commit only (1).  (downstream fork should modify pyspecific
extension and add dummy.html)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Translated Python documentation

[Berker Peksağ]

On Thu, Feb 23, 2017 at 12:01 AM, Victor Stinner
 wrote:
> 2017-02-22 19:04 GMT+01:00 Serhiy Storchaka :
>> What percent of lines is changed between bugfix releases? Feature releases?
>>
>> My largest apprehension is that the documentation can be outdated and
>> quickly become degraded if main contributors left it.
>
> If the original text (english) changes, untranslated text is
> displayed, not outdated text.

I think that's much worse than showing the outdated text. I don't see
any point on showing half English and half French text if the reader
can't understand the other half of it.

As someone who have spent a lot of time reviewing and committing
documentation patches, I'm strongly against on marking documentation
translations as official. The Python documentation updates frequently
and it's simply not possible to keep them sync with the official
documentation. See
https://github.com/python/cpython/commits/master/Doc for the commit
history of the official documentation. You can easily compare it with
the translations by looking their GitHub repositories.

Also, there are a lot of better educational materials (e.g. Django
Girls Tutorial) for people who don't speak English and have no
previous programming experience. Even the tutorial contains several
references to different programming concepts and programming languages
such as C++.

--Berker
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Report on Python vulnerabilities

[Victor Stinner]

2017-02-24 11:06 GMT+01:00 Nick Coghlan :
> The main idea that comes to mind is finding a way to add a "Fixed In" column
> to the summary table to get a quick overview of which versions were
> affected.

I had this column, just I just removed it before my email to
python-dev because the table was too wide.

I fixed my script: I reimplemented the tabulate for my needs, to
support multiline table cells.

I also enhanced the compilation process for ReadTheDocs to be able to
remove the generated vulnerabilities.rst file from Git.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Can I commit a hack for translation? (Was: Translated Python documentation

[Victor Stinner]

This change has zero impact on docs.python.org, it only help teams
working on translation, so go ahead.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Can I commit a hack for translation? (Was: Translated Python documentation

[Berker Peksağ]

On Fri, Feb 24, 2017 at 3:35 PM, Victor Stinner
 wrote:
> This change has zero impact on docs.python.org, it only help teams
> working on translation, so go ahead.

This change creates an unnecessary maintenance burden for people who
works on Python documentation.

--Berker
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Translated Python documentation

[INADA Naoki]

On Fri, Feb 24, 2017 at 9:20 PM, Berker Peksağ  wrote:
> On Thu, Feb 23, 2017 at 12:01 AM, Victor Stinner
>  wrote:
>> 2017-02-22 19:04 GMT+01:00 Serhiy Storchaka :
>>> What percent of lines is changed between bugfix releases? Feature releases?
>>>
>>> My largest apprehension is that the documentation can be outdated and
>>> quickly become degraded if main contributors left it.
>>
>> If the original text (english) changes, untranslated text is
>> displayed, not outdated text.
>
> I think that's much worse than showing the outdated text. I don't see
> any point on showing half English and half French text if the reader
> can't understand the other half of it.

In Japan, everyone learn English at school, but most people is not
good at English.
So 80% translated text helps us very much.

Especially, Sphinx split document by paragraph.  It's best granularity
for translation.

For example, there are some English paragraph remains in pathlib document
.
But it's very useful for Japanese not good at English.

>
> As someone who have spent a lot of time reviewing and committing
> documentation patches, I'm strongly against on marking documentation
> translations as official.

I totally agree with you.  Our QA is not good as commit review of CPython.
So what I want is (un|semi) official place for we share our efforts with other
language translators.  (e.g. automated build, hosting translated documentation,
and downstream customizations like adding link to official English document).


> The Python documentation updates frequently
> and it's simply not possible to keep them sync with the official
> documentation. See
> https://github.com/python/cpython/commits/master/Doc for the commit
> history of the official documentation. You can easily compare it with
> the translations by looking their GitHub repositories.

While we don't translate mastar branch, I admit we behind several weeks or
months from upstream.
But since we moved most work on Travis, we can make it weekly or even daily.
And we can share the automation with other Languages if we have a team.

I can do it without official agreement, but I don't want to do it until I get
consensus about domain name or github organization name.


>
> Also, there are a lot of better educational materials (e.g. Django
> Girls Tutorial) for people who don't speak English and have no
> previous programming experience. Even the tutorial contains several
> references to different programming concepts and programming languages
> such as C++.
>

Sadly, there are many medium level programmers who are not good
at English in Japan.  So translated library reference is very helpful too.

Regards,
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Translated Python documentation

[Nick Coghlan]

On 24 February 2017 at 23:05, INADA Naoki  wrote:

> On Fri, Feb 24, 2017 at 9:20 PM, Berker Peksağ 
> wrote:
> >
> > As someone who have spent a lot of time reviewing and committing
> > documentation patches, I'm strongly against on marking documentation
> > translations as official.
>
> I totally agree with you.  Our QA is not good as commit review of CPython.
> So what I want is (un|semi) official place for we share our efforts with
> other
> language translators.  (e.g. automated build, hosting translated
> documentation,
> and downstream customizations like adding link to official English
> document).
>
>
> > The Python documentation updates frequently
> > and it's simply not possible to keep them sync with the official
> > documentation. See
> > https://github.com/python/cpython/commits/master/Doc for the commit
> > history of the official documentation. You can easily compare it with
> > the translations by looking their GitHub repositories.
>
> While we don't translate mastar branch, I admit we behind several weeks or
> months from upstream.
> But since we moved most work on Travis, we can make it weekly or even
> daily.
> And we can share the automation with other Languages if we have a team.
>

Right, I think this is the key: helping the language translation
communities set up common flows so they can collaborate on the backend
automation, request process or tech changes in the main docs to simplify
translation (such as resolving the issue with "implementation detail" notes
disappearing when translated), and generally improving discoverability of
the translated versions.

In addition to the case of folks that struggle to read the English
documentation at all, I'd assume that there are also folks that would
appreciate the chance to check their own understanding against someone
else's translation of various topics.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Translated Python documentation

[Steven D'Aprano]

On Fri, Feb 24, 2017 at 06:01:59AM -0500, tritium-l...@sdamon.com wrote:

> My gut splits the difference on this issue; I suggest an approach to 
> meet in the middle – a version of the docs written in simplified 
> English (Not quite Up Goer Five simplified, but simplified.)

As an English speaker, my gut tells me that it would be much harder to 
write *accurate* simplified English technical documentation than to 
translate it into another language.

You have all the difficulties of translation, plus you're working under 
a handicap of only using some (ill-defined?) subset of English.

Wikipedia offers some evidence supporting my view: 

- the main English Wikipedia has 5 million articles, written by nearly 
  140K active users;

- the Swedish Wikipedia is almost as big, 3M articles from only 3K 
  active users;

- but the Simple English Wikipedia has just 123K articles and 871
  active users. That's fewer articles than Esperanto!

https://meta.wikimedia.org/wiki/List_of_Wikipedias


Nevertheless, I certainly wouldn't object if people wanted to try 
writing Simple English translations of the docs. But I don't think they 
would be as useful as translations into non-English.


[...]
> For any language you want to support other than English, you need a 
> translator who is A: a python expert, B: fluent in English, and C: 
> fluent in the target language.

I disagree. You need a translator who is A: skilled at technical 
documentation, with B and C as above. They don't need to be a Python 
expert. We have plenty of Python experts that they can consult with and 
ask questions. But they need to know the right questions to ask:

"Python attributes, they're kind of like C members, right? I would 
translate 'member' into Klingon as 'gham', which means 'arm or leg', 
so I can use the same word for attribute."


> …And then you need another one to 
> check what was written.  These are practical problems.  There are 
> extant services to support this, they are expensive in either money or 
> time, and the docs produced usually lag behind English quite a bit.

Is this a good use for some PSF funding? Would companies be willing to 
invest money in translating Python documentation?

Just because we're Open Source, doesn't mean that everything we do has 
to be purely volunteer.



-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Summary of Python tracker Issues

[Python tracker]

ACTIVITY SUMMARY (2017-02-17 - 2017-02-24)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open5833 ( -4)
  closed 35565 (+54)
  total  41398 (+50)

Open issues with patches: 2478 


Issues opened (32)
==

#27840: functools.partial: don't copy keywoard arguments in partial_ca
http://bugs.python.org/issue27840  reopened by haypo

#29593: Improve UnboundLocalError message for deleted names
http://bugs.python.org/issue29593  opened by mbussonn

#29594: implementation of __or__ in enum.auto
http://bugs.python.org/issue29594  opened by magu

#29595: Expose max_queue_size in ThreadPoolExecutor
http://bugs.python.org/issue29595  opened by prayerslayer

#29596: Unfinished sentense in howto/clinic.rst
http://bugs.python.org/issue29596  opened by mbussonn

#29598: Write unit tests for pdb module
http://bugs.python.org/issue29598  opened by Artem Muterko

#29600: Returning an exception object from a coroutine triggers implic
http://bugs.python.org/issue29600  opened by njs

#29601: Need reST markup for enum types
http://bugs.python.org/issue29601  opened by barry

#29606: urllib FTP protocol stream injection
http://bugs.python.org/issue29606  opened by ecbftw

#29611: TextIOWrapper's write_through option behave differently betwee
http://bugs.python.org/issue29611  opened by inada.naoki

#29612: TarFile.extract() suffers from hard links inside tarball
http://bugs.python.org/issue29612  opened by Jussi Judin

#29613: Support for SameSite Cookies
http://bugs.python.org/issue29613  opened by akash0x53

#29615: SimpleXMLRPCDispatcher._dispatch mangles tracebacks when invok
http://bugs.python.org/issue29615  opened by p...@motejlek.net

#29616: input() after restarting causes bug
http://bugs.python.org/issue29616  opened by Juan

#29617: Drop Python 3.4 support from asyncio
http://bugs.python.org/issue29617  opened by inada.naoki

#29619: st_ino (unsigned long long) is casted to long long in posixmod
http://bugs.python.org/issue29619  opened by mba

#29620: unittest.TestCase.assertWarns raises RuntimeEror if sys.module
http://bugs.python.org/issue29620  opened by kernc

#29621: telnetlib.Telnet.write gives confusing error message when a st
http://bugs.python.org/issue29621  opened by John Hagen

#29623: configparser.ConfigParser.read() does not accept Pathlib path 
http://bugs.python.org/issue29623  opened by David Ellis

#29624: Python 3.5.3 x86 web installer cannot install launcher
http://bugs.python.org/issue29624  opened by steve.dower

#29626: Issue with spacing in argparse module while using help
http://bugs.python.org/issue29626  opened by falu2010

#29627: configparser.ConfigParser.read() has undocumented/unexpected b
http://bugs.python.org/issue29627  opened by David Ellis

#29631: Error “importlib.h, importlib_external.h updated. You will n
http://bugs.python.org/issue29631  opened by Karen

#29632: argparse values for action in add_argument() should be flags i
http://bugs.python.org/issue29632  opened by pgacv2

#29633: MSI installer does not pass values as SecureProperty from UI
http://bugs.python.org/issue29633  opened by aldehoff

#29636: Specifying indent in the json.tool command
http://bugs.python.org/issue29636  opened by dhimmel

#29637: ast.get_docstring(): AttributeError: 'NoneType' object has no 
http://bugs.python.org/issue29637  opened by jwilk

#29638: Spurious failures in test_collections in releak hunting mode a
http://bugs.python.org/issue29638  opened by levkivskyi

#29639: test suite intentionally avoids referring to localhost, destro
http://bugs.python.org/issue29639  opened by gregory.p.smith

#29640: _PyThreadState_Init and fork race leads to inconsistent key li
http://bugs.python.org/issue29640  opened by Ján Stanček

#29641: make install failure when using ./configure --enable-optimizat
http://bugs.python.org/issue29641  opened by haypo

#29642: Why does unittest.TestLoader.discover still rely on existence 
http://bugs.python.org/issue29642  opened by Andrei Fokau



Most recent 15 issues with no replies (15)
==

#29642: Why does unittest.TestLoader.discover still rely on existence 
http://bugs.python.org/issue29642

#29640: _PyThreadState_Init and fork race leads to inconsistent key li
http://bugs.python.org/issue29640

#29639: test suite intentionally avoids referring to localhost, destro
http://bugs.python.org/issue29639

#29638: Spurious failures in test_collections in releak hunting mode a
http://bugs.python.org/issue29638

#29637: ast.get_docstring(): AttributeError: 'NoneType' object has no 
http://bugs.python.org/issue29637

#29636: Specifying indent in the json.tool command
http://bugs.python.org/issue29636

#29633: MSI installer does not pass values as SecureProperty from UI
http://bugs.python.org/issue29633

#29631: Error “importlib.h, importlib_external.h updated. You will n
http://bu

Re: [Python-Dev] Translated Python documentation

[MRAB]

On 2017-02-24 15:10, Steven D'Aprano wrote:

On Fri, Feb 24, 2017 at 06:01:59AM -0500, tritium-l...@sdamon.com wrote:


My gut splits the difference on this issue; I suggest an approach to
meet in the middle – a version of the docs written in simplified
English (Not quite Up Goer Five simplified, but simplified.)


As an English speaker, my gut tells me that it would be much harder to
write *accurate* simplified English technical documentation than to
translate it into another language.

You have all the difficulties of translation, plus you're working under
a handicap of only using some (ill-defined?) subset of English.

Wikipedia offers some evidence supporting my view:

- the main English Wikipedia has 5 million articles, written by nearly
  140K active users;

- the Swedish Wikipedia is almost as big, 3M articles from only 3K
  active users;

- but the Simple English Wikipedia has just 123K articles and 871
  active users. That's fewer articles than Esperanto!

https://meta.wikimedia.org/wiki/List_of_Wikipedias


Nevertheless, I certainly wouldn't object if people wanted to try
writing Simple English translations of the docs. But I don't think they
would be as useful as translations into non-English.


[snip]

Would it be easier to make a translation into Esperanto, which is meant 
to be easier to learn than English?


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Reports on my CPython contributions

[Victor Stinner]

Hi,

Recently, I wrote reports of my CPython contributions since 1 year
1/2. Some people on this list might be interested, so here is the
list.

I was also asked for more documentation about my work on FASTCALL and
its effect on performance. You can expect more article on FASTCALL
later, especially on its API and its implementaiton.

== CPython contrib 2016 ==

My contributions to CPython during 2016 Q1:
https://haypo.github.io/contrib-cpython-2016q1.html

My contributions to CPython during 2016 Q2:
https://haypo.github.io/contrib-cpython-2016q2.html

PEP 524: os.urandom() now blocks on Linux in Python 3.6:
https://haypo.github.io/pep-524-os-urandom-blocking.html

CPython sprint, september 2016:
https://haypo.github.io/cpython-sprint-2016.html

My contributions to CPython during 2016 Q3:
https://haypo.github.io/contrib-cpython-2016q3.html

My contributions to CPython during 2016 Q4:
https://haypo.github.io/contrib-cpython-2016q4.html


== FASTCALL project ==

The start of the FASTCALL project:
https://haypo.github.io/start-fastcall-project.html

FASTCALL microbenchmarks:
https://haypo.github.io/fastcall-microbenchmarks.html

FASTCALL issues:
https://haypo.github.io/fastcall-issues.html


== CPython contrib 2015 ==

My contributions to CPython during 2015 Q3:
https://haypo.github.io/contrib-cpython-2015q3.html

My contributions to CPython during 2015 Q4:
https://haypo.github.io/contrib-cpython-2015q4.html

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Translated Python documentation

[Nick Coghlan]

On 25 February 2017 at 01:10, Steven D'Aprano  wrote:

> On Fri, Feb 24, 2017 at 06:01:59AM -0500, tritium-l...@sdamon.com wrote:
> > …And then you need another one to
> > check what was written.  These are practical problems.  There are
> > extant services to support this, they are expensive in either money or
> > time, and the docs produced usually lag behind English quite a bit.
>
> Is this a good use for some PSF funding? Would companies be willing to
> invest money in translating Python documentation?
>

Translated documentation is certainly one of the things commercial open
source users appreciate, and hence redistributors are willing to fund in
the general case (see
https://access.redhat.com/documentation/ja/red-hat-enterprise-linux/ or
https://docs.openstack.org/ja/ for example)

For the specific case of the PSF, credible development grant ideas are
always worth considering (as those are an excellent way for the PSF to help
enable community activities).


> Just because we're Open Source, doesn't mean that everything we do has
> to be purely volunteer.
>

Right, but this kind of problem is also one of the major reasons I tend to
harp on about the fact that the commercial redistributors active in the
Python community aren't contributing as effectively as they could be, and
their existing customers aren't holding them accountable for the failure.
Python's origins as a "scripting language for Linux" means it is often
still perceived that way in the commercial sector and treated accordingly,
even though it has subsequently matured into a full-fledged cross-platform
application development and data analysis platform.

Even those of us already working for redistributors can't readily provide
that missing accountability, as any reasonable business is going to weigh
the costs of additional investment in any given area against the potential
for increased future revenue. That means the most effective pressure comes
from industry partners, governments, existing customers, and credible
potential customers (i.e. folks that have the ability to affect
redistributor revenue directly) rather than from redistributor staff or
community volunteers (as we're pretty much limited to using risk management
and hiring pipeline based lines of argument, rather than being able to push
the "future revenue potential" line directly).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Translated Python documentation

[Nick Coghlan]

On 25 February 2017 at 04:54, MRAB  wrote:

> On 2017-02-24 15:10, Steven D'Aprano wrote:
>
>> Nevertheless, I certainly wouldn't object if people wanted to try
>> writing Simple English translations of the docs. But I don't think they
>> would be as useful as translations into non-English.
>>
>> [snip]
>
> Would it be easier to make a translation into Esperanto, which is meant to
> be easier to learn than English?


Not really, as there are two main criteria for building a successful
translation community:

- potential audience for the translation (i.e. native speakers of the
target language)
- an interested pool of bilingual speakers to do the translation

Those numbers are far more favourable for native language translations to
widely spoken languages like Japanese, Mandarin Chinese, Spanish, and
French than they are for Esperanto.

When it comes to the idea of creating a Simple English translation of the
standard tutorial, my experience is that what folks are more inclined to do
on that front is to just write a new tutorial of their own specifically
targeting the audience they care about, rather than translating the
standard one.

That provides a lot more freedom to not only adjust the specific words
used, but also decide when and how to introduce different concepts based on
their particular target audience (e.g. teaching Python to a group of
mid-career scientific researchers or a group of professional engineers is
very different from teaching it to a classroom full of 6 year olds, or
students pursuing a computing degree at university, which in turn are very
different from teaching groups of interested people that have designed to
sacrifice their own time to attend a community run introduction to
programming workshop).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Split Misc/NEWS into individual files

[Larry Hastings]

On 02/22/2017 08:26 AM, Eric V. Smith wrote:

On 2/22/17 7:27 AM, Victor Stinner wrote:

OpenStack happily fixed this issue one or two years ago with "reno":
https://pypi.python.org/pypi/reno
[...]
What do you think of the overall idea?
I'm for it. I'd be interested in hearing what current and former 
release managers say.


I'm for it too.  Fixing up Misc/NEWS is probably the trickiest and most 
error-prone part of cutting a release.


We've discussed this before, somewhere on bpo though I don't have the 
issue no. handy.  I actually wrote a tool specifically to solve this 
problem at the time:


   https://bitbucket.org/larry/mergenews

Obviously it's out of date as it's designed to work with Mercurial. I 
don't know how such a tool would be integrated into a Git-based workflow.


While I'm sure we can learn a lot by examining "reno", my suspicion is 
we'll want our own custom tool.  I don't think the hard part is writing 
the tool--the hard part is figuring out what we want and how it should 
be integrated into our workflow.  Actually writing such a tool is easy 
and fun, and I'm sure our community would fall all over themselves to 
write it if asked.  In fact, I'll kind of pre-volunteer to write it 
right now, as long as I can write it in Python 3 ;-)



//arry/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com