Re: [Python-Dev] [Python-checkins] cpython: Issue #17828: va_start() must be accompanied by va_end()

2013-11-13 Thread Nick Coghlan
On 14 Nov 2013 10:40, "christian.heimes"  wrote:
>
> http://hg.python.org/cpython/rev/99ba1772c469
> changeset:   87089:99ba1772c469
> user:Christian Heimes 
> date:Thu Nov 14 01:39:35 2013 +0100
> summary:
>   Issue #17828: va_start() must be accompanied by va_end()
> CID 1128793: Missing varargs init or cleanup (VARARGS)

Today I learned... :)

Thanks!

Cheers,
Nick.

>
> files:
>   Objects/exceptions.c |  13 +++--
>   1 files changed, 7 insertions(+), 6 deletions(-)
>
>
> diff --git a/Objects/exceptions.c b/Objects/exceptions.c
> --- a/Objects/exceptions.c
> +++ b/Objects/exceptions.c
> @@ -2632,12 +2632,6 @@
>  PyObject *new_exc, *new_val, *new_tb;
>  va_list vargs;
>
> -#ifdef HAVE_STDARG_PROTOTYPES
> -va_start(vargs, format);
> -#else
> -va_start(vargs);
> -#endif
> -
>  PyErr_Fetch(&exc, &val, &tb);
>  caught_type = (PyTypeObject *) exc;
>  /* Ensure type info indicates no extra state is stored at the C
level */
> @@ -2690,7 +2684,14 @@
>   * types as well, but that's quite a bit trickier due to the extra
>   * state potentially stored on OSError instances.
>   */
> +
> +#ifdef HAVE_STDARG_PROTOTYPES
> +va_start(vargs, format);
> +#else
> +va_start(vargs);
> +#endif
>  msg_prefix = PyUnicode_FromFormatV(format, vargs);
> +va_end(vargs);
>  if (msg_prefix == NULL)
>  return NULL;
>
>
> --
> Repository URL: http://hg.python.org/cpython
>
> ___
> Python-checkins mailing list
> python-check...@python.org
> https://mail.python.org/mailman/listinfo/python-checkins
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] The pysandbox project is broken

2013-11-13 Thread Eli Bendersky
On Wed, Nov 13, 2013 at 2:48 PM, Christian Heimes wrote:

> Am 13.11.2013 23:37, schrieb Eli Bendersky:
> > Yeah, it definitely could. There are two problems currently: 1) the
> > patches are for 2.7.x and 2) they have some ugly hacks in them. But I
> > will talk to the guy who worked on that and hopefully we'll be able to
> > have something cleaned up for upstreaming into default/3.x
> >
> > Anyhow, the webstore app is:
> >
> https://chrome.google.com/webstore/detail/python/nodpmmidbgeganfponihbgmfcoiibffi
> >
> > And the code is in: https://code.google.com/p/naclports/wiki/PortList
>
> The patch
>
> https://code.google.com/p/naclports/source/browse/trunk/src/libraries/python/nacl.patch
> looks rather small and simple. Some of the hacks may not be required in
> Python 3.4, too. I'd love to have PNaCl support in Python 3.4!
>

Feel free to chime in, if you want. You can download the PNaCl toolchain
from the latest Native Client SDK and hack on this easily. I'll be happy to
look at any patches you have for this.

Eli
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] The pysandbox project is broken

2013-11-13 Thread Christian Heimes
Am 13.11.2013 23:37, schrieb Eli Bendersky:
> Yeah, it definitely could. There are two problems currently: 1) the
> patches are for 2.7.x and 2) they have some ugly hacks in them. But I
> will talk to the guy who worked on that and hopefully we'll be able to
> have something cleaned up for upstreaming into default/3.x
> 
> Anyhow, the webstore app is:
> https://chrome.google.com/webstore/detail/python/nodpmmidbgeganfponihbgmfcoiibffi
> 
> And the code is in: https://code.google.com/p/naclports/wiki/PortList

The patch
https://code.google.com/p/naclports/source/browse/trunk/src/libraries/python/nacl.patch
looks rather small and simple. Some of the hacks may not be required in
Python 3.4, too. I'd love to have PNaCl support in Python 3.4!

Christian

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] The pysandbox project is broken

2013-11-13 Thread Eli Bendersky
On Wed, Nov 13, 2013 at 10:27 AM, Brett Cannon  wrote:

>
>
>
> On Wed, Nov 13, 2013 at 1:05 PM, Eli Bendersky  wrote:
>
>>
>>
>>
>> On Wed, Nov 13, 2013 at 6:58 AM, Brett Cannon  wrote:
>>
>>>
>>>
>>>
>>> On Wed, Nov 13, 2013 at 6:30 AM, Facundo Batista <
>>> facundobati...@gmail.com> wrote:
>>>
 On Wed, Nov 13, 2013 at 4:37 AM, Maciej Fijalkowski 
 wrote:

 >> Do you think it would be productive to create an independent Python
 >> compiler, designed with sandboxing in mind from the beginning?
 >
 > PyPy sandbox does work FYI
 >
 > It might not do exactly what you want, but it both provides a full
 > python and security.

 If we have sandboxing using PyPy... what also we need to put Python
 running in the browser? (like javascript, you know)

 Thanks!

>>>
>>> You can try to get PNaCl to work with Python to get a Python executable
>>> that at least Chrome can run.
>>>
>>
>> Two corrections:
>>
>> 1. CPython already works with NaCl and PNaCl (there are working patches
>> in naclports to build it)
>>
>
> Anything that should be upstreamed?
>

Yeah, it definitely could. There are two problems currently: 1) the patches
are for 2.7.x and 2) they have some ugly hacks in them. But I will talk to
the guy who worked on that and hopefully we'll be able to have something
cleaned up for upstreaming into default/3.x

Anyhow, the webstore app is:
https://chrome.google.com/webstore/detail/python/nodpmmidbgeganfponihbgmfcoiibffi

And the code is in: https://code.google.com/p/naclports/wiki/PortList


>
>
>> 2. It can be used outside Chrome as well, using the standalone "sel_ldr"
>> tool that will then allow to run a sandboxed CPython .nexe from the command
>> line
>>
>
> Sure, but I was just thinking about the "in browser" question Facundo
> asked about.
>

Yep, see link above for in-the-browser Python. Same can be done with PNaCl
and not require the web store (this can actually be built by anyone from
the NaCl SDK today).


Eli
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] The pysandbox project is broken

2013-11-13 Thread Brett Cannon
On Wed, Nov 13, 2013 at 1:05 PM, Eli Bendersky  wrote:

>
>
>
> On Wed, Nov 13, 2013 at 6:58 AM, Brett Cannon  wrote:
>
>>
>>
>>
>> On Wed, Nov 13, 2013 at 6:30 AM, Facundo Batista <
>> facundobati...@gmail.com> wrote:
>>
>>> On Wed, Nov 13, 2013 at 4:37 AM, Maciej Fijalkowski 
>>> wrote:
>>>
>>> >> Do you think it would be productive to create an independent Python
>>> >> compiler, designed with sandboxing in mind from the beginning?
>>> >
>>> > PyPy sandbox does work FYI
>>> >
>>> > It might not do exactly what you want, but it both provides a full
>>> > python and security.
>>>
>>> If we have sandboxing using PyPy... what also we need to put Python
>>> running in the browser? (like javascript, you know)
>>>
>>> Thanks!
>>>
>>
>> You can try to get PNaCl to work with Python to get a Python executable
>> that at least Chrome can run.
>>
>
> Two corrections:
>
> 1. CPython already works with NaCl and PNaCl (there are working patches in
> naclports to build it)
>

Anything that should be upstreamed?


> 2. It can be used outside Chrome as well, using the standalone "sel_ldr"
> tool that will then allow to run a sandboxed CPython .nexe from the command
> line
>

Sure, but I was just thinking about the "in browser" question Facundo asked
about.


>
> Note that this is a fundamentally different sandboxing model (the whole
> interpreter is run in a sandbox), but it's also more secure. PNaCl has
> shipped publicly yesterday, so Chrome runs native code *from the web* on
> your machine - a lot of security research and work went into making this
> possible.
>
> As for performance, the sandboxing overhead of NaCl is very low (< 10% in
> most cases).
>

I feel like we need to have a page at python.org (or somewhere) that
provides every which way to run Python from the browser for people to try
the interpreter out as easily as possible.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] The pysandbox project is broken

2013-11-13 Thread Eli Bendersky
On Wed, Nov 13, 2013 at 6:58 AM, Brett Cannon  wrote:

>
>
>
> On Wed, Nov 13, 2013 at 6:30 AM, Facundo Batista  > wrote:
>
>> On Wed, Nov 13, 2013 at 4:37 AM, Maciej Fijalkowski 
>> wrote:
>>
>> >> Do you think it would be productive to create an independent Python
>> >> compiler, designed with sandboxing in mind from the beginning?
>> >
>> > PyPy sandbox does work FYI
>> >
>> > It might not do exactly what you want, but it both provides a full
>> > python and security.
>>
>> If we have sandboxing using PyPy... what also we need to put Python
>> running in the browser? (like javascript, you know)
>>
>> Thanks!
>>
>
> You can try to get PNaCl to work with Python to get a Python executable
> that at least Chrome can run.
>

Two corrections:

1. CPython already works with NaCl and PNaCl (there are working patches in
naclports to build it)
2. It can be used outside Chrome as well, using the standalone "sel_ldr"
tool that will then allow to run a sandboxed CPython .nexe from the command
line

Note that this is a fundamentally different sandboxing model (the whole
interpreter is run in a sandbox), but it's also more secure. PNaCl has
shipped publicly yesterday, so Chrome runs native code *from the web* on
your machine - a lot of security research and work went into making this
possible.

As for performance, the sandboxing overhead of NaCl is very low (< 10% in
most cases).

Eli
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Restoring the aliases for the non-Unicode codecs

2013-11-13 Thread Nick Coghlan
On 14 November 2013 01:43, Stephen J. Turnbull  wrote:
> Nick Coghlan writes:
>
>  > The long gory history in http://bugs.python.org/issue7475 took a
>  > different turn earlier this year when I noticed
>  > (http://bugs.python.org/issue7475#msg187698) that the codecs module
>  > already *had* type neutral helper functions in the form of
>  > codecs.encode and codecs.decode, and has had them since Python 2.4.
>  > These were covered in the test suite, but not in the documentation.
>
> As long as the agreement on _methods_ (documented in
> http://bugs.python.org/issue7475#msg96240) isn't changed, I'm +1 on
> this.

Yeah, those have actually been updated to point to the type neutral functions:

>>> "bad output".encode("rot_13")
Traceback (most recent call last):
  File "", line 1, in 
TypeError: 'rot_13' encoder returned 'str' instead of 'bytes'; use
codecs.encode() to encode to arbitrary types

>>> b"bad output".decode("quopri_codec")
Traceback (most recent call last):
  File "", line 1, in 
TypeError: 'quopri_codec' decoder returned 'bytes' instead of 'str';
use codecs.decode() to decode to arbitrary types

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] [Python-checkins] cpython: Close #17828: better handling of codec errors

2013-11-13 Thread Nick Coghlan
On 14 November 2013 02:12, Nick Coghlan  wrote:
> On 14 November 2013 00:30, Walter Dörwald  wrote:
>> On 13.11.13 14:51, nick.coghlan wrote:
>>
>>> http://hg.python.org/cpython/rev/854a2cea31b9
>>> changeset:   87084:854a2cea31b9
>>> user:Nick Coghlan 
>>> date:Wed Nov 13 23:49:21 2013 +1000
>>> summary:
>>>Close #17828: better handling of codec errors
>>>
>>> - output type errors now redirect users to the type-neutral
>>>convenience functions in the codecs module
>>> - stateless errors that occur during encoding and decoding
>>>will now be automatically wrapped in exceptions that give
>>>the name of the codec involved
>>
>>
>> Wouldn't it be better to add an annotation API to the exceptions classes?
>> This would allow to annotate all exceptions without having to replace the
>> exception object.
>
> There's a reason the C API for this is private - it's a band aid fix,
> because solving it properly is hard :)

Note that the specific problem with just annotating the exception
rather than a specific frame is that you lose the stack context for
where the annotation occurred. The current chaining workaround doesn't
just change the exception message, it also breaks the stack into two
pieces (inside and outside the codec) that get displayed separately.

Mostly though, it boils down to the fact that I'm far more comfortable
changing codec exception stack trace details in some cases than I am
proposing a new API for all exceptions this close to the Python 3.4
feature freeze.

A more elegant (and comprehensive) solution as a PEP for 3.5 would
certainly be a nice thing to have, but I think this is still much
better than the 3.3 status quo.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] [Python-checkins] cpython: Close #17828: better handling of codec errors

2013-11-13 Thread Nick Coghlan
On 14 November 2013 00:30, Walter Dörwald  wrote:
> On 13.11.13 14:51, nick.coghlan wrote:
>
>> http://hg.python.org/cpython/rev/854a2cea31b9
>> changeset:   87084:854a2cea31b9
>> user:Nick Coghlan 
>> date:Wed Nov 13 23:49:21 2013 +1000
>> summary:
>>Close #17828: better handling of codec errors
>>
>> - output type errors now redirect users to the type-neutral
>>convenience functions in the codecs module
>> - stateless errors that occur during encoding and decoding
>>will now be automatically wrapped in exceptions that give
>>the name of the codec involved
>
>
> Wouldn't it be better to add an annotation API to the exceptions classes?
> This would allow to annotate all exceptions without having to replace the
> exception object.

There's a reason the C API for this is private - it's a band aid fix,
because solving it properly is hard :)

http://bugs.python.org/issue18861 covers the fact this is just one
instance of a broader category of related problems related to
providing good exception context information (and the issue started
with a different one).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Restoring the aliases for the non-Unicode codecs

2013-11-13 Thread Stephen J. Turnbull
Nick Coghlan writes:

 > The long gory history in http://bugs.python.org/issue7475 took a
 > different turn earlier this year when I noticed
 > (http://bugs.python.org/issue7475#msg187698) that the codecs module
 > already *had* type neutral helper functions in the form of
 > codecs.encode and codecs.decode, and has had them since Python 2.4.
 > These were covered in the test suite, but not in the documentation.

As long as the agreement on _methods_ (documented in
http://bugs.python.org/issue7475#msg96240) isn't changed, I'm +1 on
this.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] The pysandbox project is broken

2013-11-13 Thread Mark Lawrence

On 13/11/2013 09:33, Simon Cross wrote:

Thanks for writing this email. It's well written and it takes a lot of
character to stand up and say you went down the wrong road. While I'm
here - thanks also for all your work on core Python. As a Python user
I really appreciate it.

Schiavo
Simon



Big +1s from me to Victor Stinner for his original email and to Simon 
Cross for this response.


--
Python is the second best programming language in the world.
But the best has yet to be invented.  Christian Tismer

Mark Lawrence

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] The pysandbox project is broken

2013-11-13 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 11/13/2013 01:54 AM, Nick Coghlan wrote:
> I actually applaud his decision to post his final conclusion to the
> list, even though it wasn't the outcome he was hoping for. Negative
> data is still data :)

Amen!  I also applaud the work he put into the idea.


Tres.
- -- 
===
Tres Seaver  +1 540-429-0999  tsea...@palladion.com
Palladion Software   "Excellence by Design"http://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlKDmQ0ACgkQ+gerLs4ltQ5XkACgn8riIUhp624gmVxSDqp6KNK+
A74An0Z3BKtc8CU0fSTsr4U76MTMw3Hi
=YtJz
-END PGP SIGNATURE-

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] The pysandbox project is broken

2013-11-13 Thread Brett Cannon
On Wed, Nov 13, 2013 at 6:30 AM, Facundo Batista
wrote:

> On Wed, Nov 13, 2013 at 4:37 AM, Maciej Fijalkowski 
> wrote:
>
> >> Do you think it would be productive to create an independent Python
> >> compiler, designed with sandboxing in mind from the beginning?
> >
> > PyPy sandbox does work FYI
> >
> > It might not do exactly what you want, but it both provides a full
> > python and security.
>
> If we have sandboxing using PyPy... what also we need to put Python
> running in the browser? (like javascript, you know)
>
> Thanks!
>

You can try to get PNaCl to work with Python to get a Python executable
that at least Chrome can run.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] [Python-checkins] cpython: Close #17828: better handling of codec errors

2013-11-13 Thread Walter Dörwald

On 13.11.13 14:51, nick.coghlan wrote:


http://hg.python.org/cpython/rev/854a2cea31b9
changeset:   87084:854a2cea31b9
user:Nick Coghlan 
date:Wed Nov 13 23:49:21 2013 +1000
summary:
   Close #17828: better handling of codec errors

- output type errors now redirect users to the type-neutral
   convenience functions in the codecs module
- stateless errors that occur during encoding and decoding
   will now be automatically wrapped in exceptions that give
   the name of the codec involved


Wouldn't it be better to add an annotation API to the exceptions 
classes? This would allow to annotate all exceptions without having to 
replace the exception object.


I.e. BaseException would have an additional method annotate():

   try:
  dostuff(param)
   except Exception as exc:
  exc.annotate("while doing stuff with {}".format(param))

annotate() would simply append the message to an internal list attribute.

BaseException.__str__() could then use this to format an appropriate 
exception message.


Servus,
   Walter

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Restoring the aliases for the non-Unicode codecs

2013-11-13 Thread M.-A. Lemburg
On 13.11.2013 15:29, Nick Coghlan wrote:
> Back in Python 3.2, the non-Unicode codecs were restored to the
> standard library, but without the associated aliases (mostly due to
> some thoroughly confusing error messages when they were mistakenly
> used with the Unicode encoding convenience methods).
> 
> The long gory history in http://bugs.python.org/issue7475 took a
> different turn earlier this year when I noticed
> (http://bugs.python.org/issue7475#msg187698) that the codecs module
> already *had* type neutral helper functions in the form of
> codecs.encode and codecs.decode, and has had them since Python 2.4.
> These were covered in the test suite, but not in the documentation.
> 
> That realisation substantially changed my perspective on the issue,
> since it was no longer a matter of adding a new API, but of better
> documenting and facilitating the use of one we already had (and was
> supported all the way back to Python 2.4, make it easy to use in
> single-source Python 2/3 projects as well). Since then, three key
> supporting issues have been addressed for Python 3.4:
> 
> * codecs.encode() and codecs.decode() have been documented in 2.7, 3.3
> and default (http://bugs.python.org/issue17827)
> * codec errors have been updated to incorporate the name of the codec
> whenever feasible, and output type errors in Unicode encoding
> convenience methods refer uses to the type neutral object<->object
> convenience functions (http://bugs.python.org/issue17839)
> * the especially confusing errors from the base64 codecs have been
> eliminated by updating the base64 module to use memoryview to process
> binary input rather than explicit typechecks on builtin types
> (http://bugs.python.org/issue17839)
> 
> So, with those underlying issues resolved, I would now like to restore
> the aliases for the non-Unicode codecs that were removed in
> http://bugs.python.org/issue10807 (aliases) and
> http://bugs.python.org/issue17841 (docs).

+1 and thanks for your work on this.

> I also looked into the possibility of providing appropriate 2to3
> fixers, but the data driven nature of the problem makes that
> impractical. However, the presence of codecs.decode and codecs.encode
> in Python 2.4+ makes a new Py3k warning in 2.7.7 a viable option:
> http://bugs.python.org/issue19543
> 
> Regards,
> Nick.
> 
> P.S. Until the next docs rebuild, you can see a summary of the
> non-Unicode codec handling improvements in the What's New diff here:
> http://hg.python.org/cpython/rev/854a2cea31b9
> 
> 

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 13 2013)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2013-11-19: Python Meeting Duesseldorf ...  6 days to go

: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Restoring the aliases for the non-Unicode codecs

2013-11-13 Thread Nick Coghlan
Back in Python 3.2, the non-Unicode codecs were restored to the
standard library, but without the associated aliases (mostly due to
some thoroughly confusing error messages when they were mistakenly
used with the Unicode encoding convenience methods).

The long gory history in http://bugs.python.org/issue7475 took a
different turn earlier this year when I noticed
(http://bugs.python.org/issue7475#msg187698) that the codecs module
already *had* type neutral helper functions in the form of
codecs.encode and codecs.decode, and has had them since Python 2.4.
These were covered in the test suite, but not in the documentation.

That realisation substantially changed my perspective on the issue,
since it was no longer a matter of adding a new API, but of better
documenting and facilitating the use of one we already had (and was
supported all the way back to Python 2.4, make it easy to use in
single-source Python 2/3 projects as well). Since then, three key
supporting issues have been addressed for Python 3.4:

* codecs.encode() and codecs.decode() have been documented in 2.7, 3.3
and default (http://bugs.python.org/issue17827)
* codec errors have been updated to incorporate the name of the codec
whenever feasible, and output type errors in Unicode encoding
convenience methods refer uses to the type neutral object<->object
convenience functions (http://bugs.python.org/issue17839)
* the especially confusing errors from the base64 codecs have been
eliminated by updating the base64 module to use memoryview to process
binary input rather than explicit typechecks on builtin types
(http://bugs.python.org/issue17839)

So, with those underlying issues resolved, I would now like to restore
the aliases for the non-Unicode codecs that were removed in
http://bugs.python.org/issue10807 (aliases) and
http://bugs.python.org/issue17841 (docs).

I also looked into the possibility of providing appropriate 2to3
fixers, but the data driven nature of the problem makes that
impractical. However, the presence of codecs.decode and codecs.encode
in Python 2.4+ makes a new Py3k warning in 2.7.7 a viable option:
http://bugs.python.org/issue19543

Regards,
Nick.

P.S. Until the next docs rebuild, you can see a summary of the
non-Unicode codec handling improvements in the What's New diff here:
http://hg.python.org/cpython/rev/854a2cea31b9


-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] [Python-checkins] cpython: Provide a more readable representation of socket on repr().

2013-11-13 Thread Nick Coghlan
On 13 November 2013 08:52, Victor Stinner  wrote:
> Hi Giampaolo,
>
> You forgot to update tests after your change in repr(socket). Tests
> are failing on buildbots, just one example:
>
> ==
> FAIL: test_repr (test.test_socket.GeneralModuleTests)
> --
> Traceback (most recent call last):
>   File "/var/lib/buildslave/3.x.murray-gentoo/build/Lib/test/test_socket.py",
> line 653, in test_repr
> self.assertIn('family=%i' % socket.AF_INET, repr(s))
> AssertionError: 'family=2' not found in " family=AddressFamily.AF_INET, type=SocketType.SOCK_STREAM, proto=0,
> laddr=('0.0.0.0', 0)>"

I fixed this to turn the buildbots green again before committing the
codec error handling changes.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] The pysandbox project is broken

2013-11-13 Thread Facundo Batista
On Wed, Nov 13, 2013 at 4:37 AM, Maciej Fijalkowski  wrote:

>> Do you think it would be productive to create an independent Python
>> compiler, designed with sandboxing in mind from the beginning?
>
> PyPy sandbox does work FYI
>
> It might not do exactly what you want, but it both provides a full
> python and security.

If we have sandboxing using PyPy... what also we need to put Python
running in the browser? (like javascript, you know)

Thanks!

-- 
.Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/
Twitter: @facundobatista
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] The pysandbox project is broken

2013-11-13 Thread Simon Cross
Thanks for writing this email. It's well written and it takes a lot of
character to stand up and say you went down the wrong road. While I'm
here - thanks also for all your work on core Python. As a Python user
I really appreciate it.

Schiavo
Simon
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] The pysandbox project is broken

2013-11-13 Thread Victor Stinner
2013/11/13 Glenn Linderman :
> If it is an implementation issue, then perhaps a different implementation
> would help. Or perhaps a "safe compiler".

There is PyPy with its sandbox.

> If it is a language design issue, then a different implementation wouldn't
> help, it would require a new language, or a restricted subset. I'm not sure
> whether some of the onerous sounding restrictions result from language or
> implementation issues; some of them certainly sounded like implementation
> issues.
>
> A restricted subset, compiled by a validating compiler, might still be a
> useful language, even if the execution speed has to be reduced by a
> validating runtime.
>
> Perhaps exception handling for exceptions hit inside a sandbox need to stop
> at the sandbox boundary. That is, exceptions within the sandbox stay within
> the sandbox, and exceptions generated due to sandbox calls to the
> implementation need to stay outside the sandbox, and then sanitized and
> limited information passed back in to the sandbox.
>
> Perhaps a different/restricted set of builtins must be provided within the
> sandbox.

The problem is to draw a line between the trusted namespace and the
untrusted namespace. Tracebacks are just one example, there are too
many other examples. Just another example: from types.__bases__, you
may reach all available types in Python, even "sensitive" types.

If you cannot draw a line because it is too complex, it probably means
that it's simpler to consider that the whole Python process is
untrusted. In this case, you have to put the sandbox outside Python,
not inside.

The second problem is that if you modify the Python language and write
a limited implementation of Python, it is no more the Python language.
What is the purpose of your sandbox if you cannot use the full Python
language and the stdlib?

It also depends on you use the sandbox. If it's just to evaluate basic
mathematic expressions, it's easier to use Python with an external
sandbox.

If you want to plug the sandbox "in your application", it's more
complex because you have to give access to your sensitive data through
a proxy, so the proxy must be carefully written.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com