[issue28307] Accelerate 'string' % (value, ...) by using formatted string literals

[STINNER Victor]

STINNER Victor  added the comment:

commit a0bd9e9c11f5f52c7ddd19144c8230da016b53c6
Author: Serhiy Storchaka 
Date:   Sat May 8 22:33:10 2021 +0300

bpo-28307: Convert simple C-style formatting with literal format into 
f-string. (GH-5012)

C-style formatting with literal format containing only format codes
%s, %r and %a (with optional width, precision and alignment)
will be converted to an equivalent f-string expression.

It can speed up formatting more than 2 times by eliminating
runtime parsing of the format string and creating temporary tuple.

commit 8b010673185d36d13e69e5bf7d902a0b3fa63051
Author: Serhiy Storchaka 
Date:   Sun May 23 19:06:48 2021 +0300

bpo-28307: Tests and fixes for optimization of C-style formatting (GH-26318)

Fix errors:
* "%10.s" should be equal to "%10.0s", not "%10s".
* Tuples with starred expressions caused a SyntaxError.

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28307] Accelerate 'string' % (value, ...) by using formatted string literals

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
pull_requests: +24913
pull_request: https://github.com/python/cpython/pull/26318

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28307] Accelerate 'string' % (value, ...) by using formatted string literals

[Brandt Bucher]

Change by Brandt Bucher :


--
nosy: +brandtbucher

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28307] Accelerate 'string' % (value, ...) by using formatted string literals

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

PR 26160 adds support of %d, %i, %u, %o, %x, %X, %f, %e, %g, %F, %E, %G.

What is not supported:

* Formatting with a single value not wrapped into a 1-tuple (like in "%d bytes" 
% size). The behavior is non-trivial, it needs checking whether the value is a 
tuple or a mapping. It would require adding more opcodes and AST nodes and 
generating complex bytecode.
* Mapping keys (like in "%(name)s"). They are rarely used, and if used, it is 
not performance critical.
* Precision for integer formatting. Precision is not supported in new-style 
formatting of integers, and it is not trivial to reproduce this behavior.
* Variable width and precision (like in "%*.*s"). It is possible to support 
them, but the code would be pretty complex, and the benefit is small, because 
this feature is rarely used and is not performance critical.
* Format code %c. It is relatively rarely used.
* Length modifiers "h", "l" and "L" (like in "%ld"). They ignored in Python and 
I did not see them in real code. While supporting them is easy, it would 
requires adding more than one line of code, it is not worth it.

--
priority: low -> normal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28307] Accelerate 'string' % (value, ...) by using formatted string literals

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
nosy: +Mark.Shannon
versions: +Python 3.11 -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28307] Accelerate 'string' % (value, ...) by using formatted string literals

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
pull_requests: +24794
pull_request: https://github.com/python/cpython/pull/26160

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28307] Accelerate 'string' % (value, ...) by using formatted string literals

[Tal Einat]

Tal Einat  added the comment:

I'm +1 on this optimization.

--
nosy: +taleinat

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28307] Accelerate 'string' % (value, ...) by using formatted string literals

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

PR 5012 implements transformation simple format strings containing only %s, %r 
and %a into f-strings.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28307] Accelerate 'string' % (value, ...) by using formatted string literals

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
keywords: +patch
pull_requests: +4901
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28307] Accelerate 'string' % (value, ...) by using formatted string literals

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
assignee:  -> serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28307] Accelerate 'string' % (value, ...) by using formatted string literals

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Thanks for the correction Antti. Yes, this is what I initially meant. This 
optimization is applicable only if the left argument of % is a literal string 
and the right argument is a tuple expression. Saying about `'%s' % x` I meant a 
component of the tuple.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28307] Accelerate 'string' % (value, ...) by using formatted string literals

[Antti Haapala]

Antti Haapala added the comment:

Serhiy, you actually did make a mistake above; `'%s' % x` cannot be rewritten 
as `f'{x!s}'`, only `'%s' % (x,)` can be optimized... 

(just try with `x = 1, 2`)

--
nosy: +ztane

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28307] Accelerate 'string' % (value, ...) by using formatted string literals

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

'%s' % x should be translated to f'{x!s}', not to f'{x:s}'. Only %s, %r and %a 
can be supported. Formatting with %i should left untranslated. Or maybe 
translate '%r: %i' % (a, x) to f'{a!r}: {"%i" % x}'.

It is possible also to introduce special opcodes that converts argument to 
exact int or float. Then '%06i' % x could be translated to 
f'{__exact_int__(x):06}'.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28307] Accelerate 'string' % (value, ...) by using formatted string literals

[Eric V. Smith]

Eric V. Smith added the comment:

There isn't a direct mapping between %-formatting and __format__ format 
specifiers. Off the top of my head, I can think of at least one difference:

>>> '%i' % 3
'3'
>>> '{:i}'.format(3)
Traceback (most recent call last):
  File "", line 1, in 
ValueError: Unknown format code 'i' for object of type 'int'

So you'll need to be careful with edge cases like this.

Also, for all usages of %s, remember to call str() (or add !s):

>>> '%s' % 1
'1'
>>> f'{1:s}'
Traceback (most recent call last):
  File "", line 1, in 
ValueError: Unknown format code 's' for object of type 'int'
>>> f'{1!s:s}'
'1'

Although that also reminds me of this default alignment difference:
>>> x=0
>>> '%2s' % x
' 0'
>>> f'{x!s:2s}'
'0 '
>>> f'{x!s:>2s}'
' 0'

So, in general, the mapping will be difficult. On the other hand, if you can do 
it, and provide a function that maps between %-formatting codes and __format__ 
codes, then that might be a generally useful tool.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28307] Accelerate 'string' % (value, ...) by using formatted string literals

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
dependencies: +Build-out an AST optimizer, moving some functionality out of the 
peephole optimizer

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28307] Accelerate 'string' % (value, ...) by using formatted string literals

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

For now using formatted string literals (PEP498) is the fastest way of 
formatting strings.

$ ./python -m perf timeit -s 'k = "foo"; v = "bar"' -- '"%s = %r" % (k, v)'
Median +- std dev: 2.27 us +- 0.20 us

$ ./python -m perf timeit -s 'k = "foo"; v = "bar"' -- 'f"{k!s} = {v!r}"'
Median +- std dev: 1.09 us +- 0.08 us

The compiler could translate C-style formatting with literal format string to 
the equivalent formatted string literal. The code '%s = %r' % (k, v) could be 
translated to

t1 = k; t2 = v; f'{t1!r} = {t2!s}'; del t1, t2

or even simpler if k and v are initialized local variables.

$ ./python -m perf timeit -s 'k = "foo"; v = "bar"' -- 't1 = k; t2 = v; 
f"{t1!s} = {t2!r}"; del t1, t2'
Median +- std dev: 1.22 us +- 0.05 us

This is not easy issue and needs first implementing the AST optimizer.

--
components: Interpreter Core
messages: 277688
nosy: eric.smith, serhiy.storchaka
priority: low
severity: normal
status: open
title: Accelerate 'string' % (value, ...) by using formatted string literals
type: performance
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com