Re: [Python-Dev] cpython: Optimize _PyUnicode_FastCopyCharacters() when maxchar(from) > maxchar(to)

2012-06-16 Thread Antoine Pitrou 
On Sat, 16 Jun 2012 02:29:09 +0200
victor.stinner  wrote:
> +if (from_kind == to_kind) {
> +if (!PyUnicode_IS_ASCII(from) && PyUnicode_IS_ASCII(to)) {
> +/* Writing Latin-1 characters into an ASCII string requires to
> +   check that all written characters are pure ASCII */
> +#ifndef Py_DEBUG
> +if (check_maxchar) {
> +Py_UCS4 max_char;
> +max_char = ucs1lib_find_max_char(from_data,
> + (char*)from_data + 
> how_many);
> +if (max_char >= 128)
> +return -1;
> +}
> +#else
> +const Py_UCS4 to_maxchar = PyUnicode_MAX_CHAR_VALUE(to);
> +Py_UCS4 ch;
> +Py_ssize_t i;
> +for (i=0; i < how_many; i++) {
> +ch = PyUnicode_READ(from_kind, from_data, from_start + i);
> +assert(ch <= to_maxchar);
> +}
> +#endif

So you're returning -1 in release mode but you're crashing (assert())
in debug mode? Why that?

> +#ifndef Py_DEBUG
> +if (!check_maxchar) {
[...]

This means the optimizations are not exercised in debug mode?
That sounds like a bad idea.

Regards

Antoine.


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 362: 4th edition

2012-06-16 Thread Daniel Urban 
On Sat, Jun 16, 2012 at 5:56 AM, Jim J. Jewett  wrote:
> I think it should be explicit that this mapping does not include
> parameters which would be filled by default arguments.  In fact, if
> you stick with this interface, I would like a 3rd method that does
> fill out everything.

+1


Daniel
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 362 implementation issue: C callables

2012-06-16 Thread Daniel Urban 
On Fri, Jun 15, 2012 at 11:24 PM, Larry Hastings  wrote:
> There are four more candidates I found with the grep but couldn't figure out
> how to instantiate and test.  They have to do with the descriptor protocol,
> aka properties, but the types aren't directly exposed by Python.  They're
> all defined in Object/descrobject.c.  The internal class names are:
>
> method_descriptor
> classmethod_descriptor
> wrapper_descriptor
> method-wrapper (you get one of these out of a "wrapper_descriptor")

'method_descriptor' is apparently used for the methods of built-in
(implemented in C) objects:
>>> set.__dict__['union'].__class__


'classmethod_descriptor' is similarly for the classmethods of built-in classes:
>>> import itertools
>>> itertools.chain.__dict__['from_iterable'].__class__


'wrapper_descriptor' is used for example the operators of built-in types:
>>> int.__dict__['__add__'].__class__


And 'method-wrapper':
>>> (5).__add__.__class__



Daniel
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 362 Third Revision

2012-06-16 Thread Nick Coghlan 
On Sat, Jun 16, 2012 at 10:52 AM, Benjamin Peterson  wrote:
> 2012/6/15 Larry Hastings :
>> Note that I'm genuinely interested in your answer--"is_implemented" appears
>> to have a groundswell of anti-support and I rather suspect will be axed.
>> Meantime I still need to solve this problem.
>
> How about a list of functions that support it. Then you could do things like
>
> "chown" in os.supports_at_variant

Indeed, PEP 362 is *not* the right answer for obtaining a more user
friendly interface to sysconfig.

The real answer for this kind of case is to just try whatever you want
to do and see if it throws an exception. Just as LBYL is sometimes
unusable due to problems with race conditions, in this case, a
dedicated LBYL interface should be rejected because it requires
double-keying data. If you try it and it fails, you know it's not
supported. If some separate metadata *claims* it's not supported, it
might just be that the metadata is wrong.

We don't have to implement bad ideas just because some people don't
like to rely on exceptions - LBYL vs EAFP isn't always just a
stylistic choice, there are cases where one or the other can be a
genuinely poor concept.

If someone really does want to do LBYL in this case, then finding out
the underlying criteria and perform the same check themselves is the
*right way to do it*. They'll get the wrong answer if the criteria
ever changes, but that's the inevitable risk with LBYL in cases like
this.

At most, it may make sense to provide more user friendly interfaces
for specifically requested sysconfig checks (perhaps in sysconfig or
platform).

For example:

if platform.provides_openat():
   # Can use dirfd param to os.open()

if platform.provides_fd_times():
   # Can use fd param to os.utime()

I'm not convinced this need to be in the standard library, though - we
will already be providing an obvious way to do these checks (i.e.
catch the exceptions). It definitely shouldn't be rushed into 3.3.

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 362: 4th edition

2012-06-16 Thread Nick Coghlan 
On Sat, Jun 16, 2012 at 1:56 PM, Jim J. Jewett  wrote:
>
> Summary:
>
>    *Every* Parameter attribute is optional, even name.  (Think of
>    builtins, even if they aren't automatically supported yet.)
>    So go ahead and define some others that are sometimes useful.

No, that's not the right attitude to take when designing a new API.
Add only stuff we know is interesting and useful. Call YAGNI on
everything else. If we later decided we missed something, then we can
add it easily. If something we add up front turns out to be useless
(or, worse, an attractive nuisance), then it's a pain to eliminate.

For parameters, that list is only:

- kind
- name (should be given meaningful content, even for POSITIONAL_ONLY parameters)
- default (may be missing, since "None" is allowed as a default value)
- annotation (may be missing, since "None" is allowed as an annotation)

>    Instead of defining a BoundArguments class, just return a copy
>    of the Signature, with "value" attributes added to the Parameters.

No, the "BoundArguments" class is designed to be easy to feed to a
function call as f(*args, **kwds)

>    Use subclasses to distinguish the parameter kind.  (Replacing
>    most of the is_ methods from the 3rd version.)

Please, no, using subclasses when there is no behavioural change is
annoying. A "kind" attribute will handle this just fine.

>    I favor passing a class to Signature.format, because so many of
>    the formatting arguments would normally change in parallel.
>    But my tolerance for nested structures may be unusually high.

I'm actually inclined to drop a lot of that complexity altogether -
better to provide a simple default implementation, and if people want
anything else they can write their own.

>> A Signature object has the following public attributes and methods:
>
>> * return_annotation : object
>>    The annotation for the return type of the function if specified.
>>    If the function has no annotation for its return type, this
>>    attribute is not set.
>
> This means users must already be prepared to use hasattr with the
> Signature as well as the Parameters -- in which case, I don't see any
> harm in a few extra optional properties.

No, this gets the reasoning wrong. The optional properties are
optional solely because "None" and "not present" don't mean the same
thing - the attributes potentially being missing represents the fact
that annotations and default values may be omitted altogether.

> I would personally prefer to see the name (and qualname) and docstring,
> but it would make perfect sense to implement these by keeping a weakref
> to the original callable, and just delegating there unless/until the
> properties are explicitly changed.  I suspect others will have a use
> for additional delegated attributes, such as the self of boundmethods.

It is expected that higher level interfaces will often compose the
signature object with more information from the callable. That is
already well supported today, and is not the role of the signature
object. The signature object is being added purely to provide a
standard way to describe how a callable binds the supplied arguments
to the expected parameters, that's all.

> I do agree that __eq__ and __hash__ should depend at most on the
> parameters (including their order) and the annotation.
>
>> * parameters : OrderedDict
>>     An ordered mapping of parameters' names to the corresponding
>>     Parameter objects (keyword-only arguments are in the same order
>>     as listed in ``code.co_varnames``).
>
> For a specification, that feels a little too tied to the specific
> implementation.  How about:
>
>     Parameters will be ordered as they are in the function declaration.
>
> or even just:
>
>     Positional parameters will be ordered as they are in the function
>     declaration.
>
> because:
>    def f(*, a=4, b=5): pass
>
> and:
>    def f(*, b=5, a=4): pass
>
> should probably have equal signatures.
>
>
> Wild thought:  Instead of just *having* an OrderedDict of Parameters,
> should a Signature *be* that OrderedDict (with other attributes)?
> That is, should signature(testfn)["foo"] get the foo parameter?

No. While the sequence of parameters is obviously the most important
part of the signature, it's still much clearer to expose it as a
distinct attribute. If return annotations didn't exist, you may be
able to make a stronger case.

> I think it should state explicitly that by default, the return value
> will be a string that could be used to declare an equivalent function,
> if "Signature" were replaced with "def funcname".
>
> There are enough customization parameters that would often be changed
> together (e.g., to produce HTML output) that it might make sense to use
> overridable class defaults -- or even to make format a class itself.
>
> I also think it would make sense to delegate formatting the individual
> parameters to the parameter objects.  Yes, this does mean that the
> subclasses would be more than markers

Re: [Python-Dev] PEP 362: 4th edition

2012-06-16 Thread Yury Selivanov 
Jim,

On 2012-06-15, at 11:56 PM, Jim J. Jewett wrote:
> because:
>def f(*, a=4, b=5): pass
> 
> and:
>def f(*, b=5, a=4): pass
> 
> should probably have equal signatures.

That's a very good catch -- I'll fix the implementation.

>> * bind(\*args, \*\*kwargs) -> BoundArguments
>>Creates a mapping from positional and keyword arguments to
>>parameters.  Raises a ``BindError`` (subclass of ``TypeError``)
>>if the passed arguments do not match the signature.
>> * bind_partial(\*args, \*\*kwargs) -> BoundArguments
>>Works the same way as ``bind()``, but allows the omission
>>of some required arguments (mimics ``functools.partial``
>>behavior.)
> 
> Are those descriptions actually correct?
> 
> I would expect the mapping to be from parameters (or parameter names)
> to values extracted from *args and **kwargs.
> 
> And I'm not sure the current patch does even that, since it seems to
> instead return a non-Mapping object (but with a mapping attribute)
> that could be used to re-create *args, **kwargs in canonical form.
> (Though that canonicalization is valuable for calls; it might even
> be worth an as_call method.)
> 
> 
> I think it should be explicit that this mapping does not include
> parameters which would be filled by default arguments.  In fact, if
> you stick with this interface, I would like a 3rd method that does
> fill out everything.

You're right, the fact that the defaults are left out should be
manifested in the PEP.

I'm not sure that we need a separate method for defaults though.
It's just a matter of 3-4 lines of code to iterate through parameters
and add defaults to 'BoundArguments.arguments'.  Let's keep the API
clear.

> But I think it would be simpler to just add an optional attribute
> to each Parameter instance, and let bind fill that in on the copies,
> so that the return value is also a Signature.  (No need for the
> BoundArguments class.)  Then the user can decide whether or not to
> plug in the defaults for missing values.

Too complicated.  And doesn't make the API easier to use.

>> It's possible to test Signatures for equality.  Two signatures
>> are equal when they have equal parameters and return annotations.
> 
> I would be more explicit about parameter order mattering.  Perhaps:
> 
>It's possible to test Signatures for equality.  Two signatures are
>equal when their parameters are equal, their positional parameters
>appear in the same order, and they have equal return annotations.

OK.

Thanks,
-
Yury
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 362: 4th edition

2012-06-16 Thread Yury Selivanov 
On 2012-06-16, at 11:27 AM, Nick Coghlan wrote:

> On Sat, Jun 16, 2012 at 1:56 PM, Jim J. Jewett  wrote:
>> 
>> Summary:
>> 
>>*Every* Parameter attribute is optional, even name.  (Think of
>>builtins, even if they aren't automatically supported yet.)
>>So go ahead and define some others that are sometimes useful.
> 
> No, that's not the right attitude to take when designing a new API.
> Add only stuff we know is interesting and useful. Call YAGNI on
> everything else. If we later decided we missed something, then we can
> add it easily. If something we add up front turns out to be useless
> (or, worse, an attractive nuisance), then it's a pain to eliminate.
> 
> For parameters, that list is only:
> 
> - kind
> - name (should be given meaningful content, even for POSITIONAL_ONLY 
> parameters)

+1.

>>> A Signature object has the following public attributes and methods:
>> 
>>> * return_annotation : object
>>>The annotation for the return type of the function if specified.
>>>If the function has no annotation for its return type, this
>>>attribute is not set.
>> 
>> This means users must already be prepared to use hasattr with the
>> Signature as well as the Parameters -- in which case, I don't see any
>> harm in a few extra optional properties.
> 
> No, this gets the reasoning wrong. The optional properties are
> optional solely because "None" and "not present" don't mean the same
> thing - the attributes potentially being missing represents the fact
> that annotations and default values may be omitted altogether.

Exactly.  And I'd stay away from adding back 'has_annotation' and 
'has_default' attributes.  It's easy enough to call 'hasattr' or do
try-catch-else.

>> I would personally prefer to see the name (and qualname) and docstring,
>> but it would make perfect sense to implement these by keeping a weakref
>> to the original callable, and just delegating there unless/until the
>> properties are explicitly changed.  I suspect others will have a use
>> for additional delegated attributes, such as the self of boundmethods.
> 
> It is expected that higher level interfaces will often compose the
> signature object with more information from the callable. That is
> already well supported today, and is not the role of the signature
> object. The signature object is being added purely to provide a
> standard way to describe how a callable binds the supplied arguments
> to the expected parameters, that's all.
> 
>> I do agree that __eq__ and __hash__ should depend at most on the
>> parameters (including their order) and the annotation.

Actually, Signature and Parameter are currently non-hashable (they are
mutable).  I'm not sure if it was a right decision.

>> I think it should state explicitly that by default, the return value
>> will be a string that could be used to declare an equivalent function,
>> if "Signature" were replaced with "def funcname".
>> 
>> There are enough customization parameters that would often be changed
>> together (e.g., to produce HTML output) that it might make sense to use
>> overridable class defaults -- or even to make format a class itself.
>> 
>> I also think it would make sense to delegate formatting the individual
>> parameters to the parameter objects.  Yes, this does mean that the
>> subclasses would be more than markers classes.
> 
> I'd like to see support for customising the formatted output dropped
> from the PEP entirely. We can add that later after seeing how people
> use the class, there's no need to try to guess in advance.

I'd be OK with that, as I don't like the current 'format()' design either.
But we should keep the Signature.__str__.

>>> The structure of the Parameter object is:
>> 
>>> * name : str
>>> The name of the parameter as a string.
>> 
>> If there is no name, as with some builtins, will this be "", None or
>> not set?
> 
> Positional parameters should still be given a meaningful name
> (preferably matching the name used in their docstring and prose
> documentation).

Again, +1 on the required name for positional-only args.

-
Yury

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Anyone interested in a Solaris11 Build Slave?

2012-06-16 Thread Justin Venus 
I have an idle x86 Solaris 11 workstation (with SolarisStudio12.3)
that is ready to be a build slave, it just needs credentials to
participate.

Thank you,

-- 
Justin Venus
[email protected]
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] deprecating .pyo and -O

2012-06-16 Thread Gregory P. Smith 
On Thu, Jun 14, 2012 at 1:57 PM, "Martin v. Löwis" wrote:

> > I don't really see the point. In my experience there is no benefit to
> > removing assert statements in production mode. This is a C-specific
> > notion that doesn't really map very well to Python code. Do other
> > high-level languages have similar functionality?
>
> It's not at all C specific. C# also has it:
>
> http://msdn.microsoft.com/en-**us/library/ttcc4x86(v=vs.80).**aspx
>
> Java makes it a VM option (rather than a compiler option), but it's
> still a flag to the VM (-enableassertions):
>
> http://docs.oracle.com/javase/**1.4.2/docs/tooldocs/windows/**java.html
>
> Delphi also has assertions that can be disabled at compile time.
>
>
It may be a commonly supported feature but that doesn't mean it is a good
idea. :)

It seems to me that assertion laden code where the assertions are removed
before shipping it or running it in production is largely a concept from
the pre-unittesting era.  One big issue with them in Python is that
enabling or disabling them is VM global rather than controlled on a per
module/library basis.  That isn't true in C/C++ where you can control it on
a per file basis at compile time (NDEBUG).

As a developer I agree that it is very convenient to toss asserts into code
while you are iterating on writing it.  But I regularly have people remove
assert statements during code reviews at work and, if the condition is
important, replacing them with actual if condition checks that handle the
problem or raise an appropriate module specific exception and documenting
that as part of their API.  At this point, I wish Python just didn't
support assert as part of the language because it causes less headaches in
the end of code is just written without them begin with.  Too late now.

In agreement with others: docstring memory consumption is a big deal, some
large Python apps at work strip them or use -O (I don't remember which
technique they're using today) before deployment. It does seem like
something we should provide a standard way of doing. Docstring data is not
needed within non-interactive code.

-gps
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 362: 4th edition

2012-06-16 Thread Terry Reedy 

On 6/16/2012 6:40 PM, Yury Selivanov wrote:


Actually, Signature and Parameter are currently non-hashable (they are
mutable).  I'm not sure if it was a right decision.


If this is added for 3.3, I think it would be a candidate for 
'provisional' (ie, api subject to change) status. (That is not to 
suggest not *trying* to get it right the first time ;-).


--
Terry Jan Reedy



___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com