[issue15952] format(value) and value.__format__() behave differently with unicode format

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
resolution:  -> out of date
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15952] format(value) and value.__format__() behave differently with unicode format

[Ezio Melotti]

Ezio Melotti added the comment:

``format(value, format_spec)`` merely calls
-  ``value.__format__(format_spec)``.
+  ``value.__format__(format_spec)`` and, if *format_spec* is Unicode,
+  converts the value to Unicode if it is not already Unicode.

This is correct, but should be rephrased (and value should be return value).

+  The method ``value.__format__(format_spec)`` may return 8-bit strings
+  for some built-in types when *format_spec* is Unicode.

This is not limited to built-in types.  __format__() might return either str or 
unicode, and format() returns the same -- except for the aforementioned case.


This is a summary of the possible cases.


__format__ can return unicode or str:

   class Uni(object):
  ...   def __format__(*args): return u'uni'
  ... 
   class Str(object):
  ...   def __format__(*args): return 'str'
  ... 


format() and __format__ return the same value, except when the format_spec is 
unicode and __format__ returns str:

   format(Uni(),  'd'),  Uni().__format__( 'd')  # same
  (u'uni', u'uni')
   format(Uni(), u'd'),  Uni().__format__(u'd')  # same
  (u'uni', u'uni')
   format(Str(),  'd'),  Str().__format__( 'd')  # same
  ('str', 'str')
   format(Str(), u'd'),  Str().__format__(u'd')  # different
  (u'str', 'str')

It is also not true that the type of return value is the same of the 
format_spec, because in the first case the returned type is unicode even if the 
format_spec is str.  Therefore this part of the patch should be changed:

+   Per :pep:`3101`, the function returns a Unicode object if *format_spec* is
+   Unicode.  Otherwise, it returns an 8-bit string.

The behavior might be against PEP 3101 (see quotation in msg170669), even 
thought the wording of the PEP is somewhat lenient IMHO (proper type doesn't 
necessary mean same type).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15952
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15952] format(value) and value.__format__() behave differently with unicode format

[Chris Jerdonek]

Changes by Chris Jerdonek chris.jerdo...@gmail.com:


--
nosy: +ezio.melotti

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15952
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15952] format(value) and value.__format__() behave differently with unicode format

[Chris Jerdonek]

Chris Jerdonek added the comment:

Here is a proposed patch.

One note on the patch.  I feel the second sentence of the note is worth adding 
because value.__format__() departs from what PEP 3101 says:

Note for Python 2.x: The 'format_spec' argument will be either
a string object or a unicode object, depending on the type of the
original format string.  The __format__ method should test the type
of the specifiers parameter to determine whether to return a string or
unicode object.  It is the responsibility of the __format__ method
to return an object of the proper type.

The extra sentence will help in heading off and when responding to issues about 
value.__format__() that are similar to issue 15951.

--
keywords: +patch
stage:  - patch review
Added file: http://bugs.python.org/file27218/issue-15952-1-branch-27.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15952
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15952] format(value) and value.__format__() behave differently with unicode format

[Chris Jerdonek]

Chris Jerdonek added the comment:

To clarify, one of the sentences above should have read, I feel the second 
sentence of the note *in the patch* was worth adding... (not the second 
sentence of the PEP note I quoted).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15952
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15952] format(value) and value.__format__() behave differently with unicode format

[Chris Jerdonek]

Chris Jerdonek added the comment:

See this code comment:

/* don't define FORMAT_LONG, FORMAT_FLOAT, and FORMAT_COMPLEX, since
   we can live with only the string versions of those.  The builtin
   format() will convert them to unicode. */

from http://hg.python.org/cpython/file/19601d451d4c/Python/formatter_unicode.c

In other words, it was deliberate not to make value.__format__(format_spec) 
return unicode when format_spec is unicode.  So the docs should be adjusted to 
say that they are not always the same.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15952
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15952] format(value) and value.__format__() behave differently with unicode format

[Eric V. Smith]

Eric V. Smith added the comment:

I believe the conversion is happening in Objects/abstract.c in PyObject_Format, 
around line 864, near this comment:

/* Convert to unicode, if needed.  Required if spec is unicode
   and result is str */

I think changing the docs will result in more confusion than clarity, but if 
you can come up with some good wording, I'd be okay with it. I think changing 
the code will likely break things with little or no benefit.

--
nosy: +eric.smith

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15952
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15952] format(value) and value.__format__() behave differently with unicode format

[Chris Jerdonek]

New submission from Chris Jerdonek:

format(value) and value.__format__() behave differently even though the 
documentation says otherwise:

Note: format(value, format_spec) merely calls value.__format__(format_spec).

(from http://docs.python.org/library/functions.html?#format )

The difference happens when the format string is unicode.  For example:

 format(10, u'n')
u'10'
 (10).__format__(u'n')  # parentheses needed to prevent SyntaxError
'10'

So either the documentation should be changed, or the behavior should be 
changed to match.

Related to this: neither the Format Specification Mini-Language documentation 
nor the string.Formatter docs seem to say anything about the effect that a 
unicode format string should have on the return value (in particular, should it 
cause the return value to be unicode or not):

http://docs.python.org/library/string.html#formatspec
http://docs.python.org/library/string.html#string-formatting

See also issue 15276 (int formatting), issue 15951 (empty format string), and 
issue 7300 (unicode arguments).

--
assignee: docs@python
components: Documentation
messages: 170575
nosy: cjerdonek, docs@python
priority: normal
severity: normal
status: open
title: format(value) and value.__format__() behave differently with unicode 
format
type: behavior
versions: Python 2.7

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15952
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15952] format(value) and value.__format__() behave differently with unicode format

[Arfrever Frehtes Taifersar Arahesis]

Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com:


--
nosy: +Arfrever

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15952
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com