Re: string interpolation for python
Hi Adrian, see my comments below.
From: Adrian Hunt cybor...@hotmail.com
...
It could break old code... okay you may say you should’nt allow
certain characters but if they're printable and used in a controlled
environment those characters can dramatically increase the security
of a username and password.
What you said makes lots of sense to me.
if strings are interpolated *automatically*.
But it won't and shouldn't.
They are called Dynamic strings.
Dynamic strings can achieve formatting,
but the mechanism under the hood differ
from common strings dramatically.
Many here didn't realize that this is not
another formatting proposal, it is a new
kind of *expression*.
To have it in Python, we will need
a new kind of syntax to distinguish it
from other strings, such as raw strings
and the like. A raw string looks like:
r'my\\ raw str'
'my raw str'
A dynamic string may look like this:
name = Peter #common string
dThank you, $name$! #dynamic string!
'Thank you, Peter!'
The following example would make it feel
a lot more safe (suppose a = raw_input()):
a = 'dAre you $name$?'
print(a)
'dAre you $name$?'
eval('dAre you $name$?')
'Are you Peter?'
dIt contains $len(_.split())$ words!
'It contains 3 words!'
An interesting question might be:
what if a dynamic string is referring
to another dynamic string, which
in turn refers back to the former?
The answer is: no variable can hold
a dynamic string itself, only its result,
which can only be a common string.
However, a infinite recursion may
occur if the eval function is used inside:
a = 'd$eval(a)$'
eval(a)
This is just to show a dynamic string
is really an expression in disguise.
Like evaluating any expression containing
function calls, there is risk of getting into
infinite recursion.
Cheers,
Yingjie
--
http://mail.python.org/mailman/listinfo/python-list
Re: string interpolation for python
You can already do essentially that without adding a special-case string
formatting method to the general methods we already have.
balls = 5
people = 3
'The {people} people have {balls}
balls.'.format(**locals())
'The 3 people have 5 balls.'
Clearly dynamic strings are much more powerful,
allowing arbitrary expressions inside. It is also
more terse and readable, since we need no dictionary.
I would probably rather liken dynamic expressions
as a little brother of computable documents in
Mathematica. It is a new kind of expression,
rather than formatting -- though it has formatting
connections.
Dynamic strings are mainly useful at time of
writing readable code before compilation.
The compiler can choose to convert it into
a string formatting expression, of course.
To efficiently format strings at runtime,
the best choice (especially
for safty reasons) is string formatting,
not evaluating a dynamic string.
On the implementation, I would suppose new
syntax is needed (though very small).
Yingjie
--
http://mail.python.org/mailman/listinfo/python-list
Re: string interpolation for python
Python already has *3* different built-in string formatting/interpolation systems: ... I would surmise that your key implicitly grab variable values from the enclosing scope feature has previously been rejected for being too magical. It grabs because it is an expression in disguise (not a literal). If we could make that clear in the way we write it, seems the magic level should reduce.to a tolerable (or even likable) level. Cheers, Yingjie -- http://mail.python.org/mailman/listinfo/python-list
Re: string interpolation for python
On Mon, Apr 2, 2012 at 5:24 PM, Yingjie Lan lany...@yahoo.com wrote: They are called Dynamic strings. Dynamic strings can achieve formatting, but the mechanism under the hood differ from common strings dramatically. Many here didn't realize that this is not another formatting proposal, it is a new kind of *expression*. To have it in Python, we will need a new kind of syntax to distinguish it from other strings, such as raw strings and the like. A raw string looks like: r'my\\ raw str' 'my raw str' A dynamic string may look like this: name = Peter #common string dThank you, $name$! #dynamic string! 'Thank you, Peter!' From the Zen of Python: Special cases aren't special enough to break the rules. It's fairly cheap to propose a new function. If people don't like it, they can ignore it... it can get stuck into a module somewhere and be easily ignored. If people DO like it, they can back-port it to older versions by simply copying and pasting the Python code (or equivalent Python code, if the main version's written in C). New syntax like this has to be implemented in the parser, is (practically) impossible to back-port with pure Python, and has far higher potential to break existing code. The onus is on you to demonstrate that this syntax is worth this hassle. I'm -1 on the idea, mainly because there are already two perfectly good string interpolation syntaxes that don't require a new kind of string. I'm also against the notion of interpolated strings vs non-interpolated strings in general, having worked with them in PHP and not found any real benefit. But if you want it, you could fairly easily do something with an explicit interpolation parsing function: dThank you, $name$! #dynamic string! 'Thank you, Peter!' becomes: d(Thank you, $name$!) #dynamic string! 'Thank you, Peter!' To make the magic of evaluation scope work, though, you'd need to fiddle around with tracebacks. It'd be far simpler and clearer to pass locals() and/or globals() to your function, if indeed you want that. Chris Angelico -- http://mail.python.org/mailman/listinfo/python-list
Re: string interpolation for python
Yingjie Lan writes: Clearly dynamic strings are much more powerful, allowing arbitrary expressions inside. It is also more terse and readable, since we need no dictionary. ... On the implementation, I would suppose new syntax is needed (though very small). I don't think you need any new syntax to implement this. You can use a syntax like Dynamite(Hello, $world$) now, similar to fraction.Fraction: Fraction(3,4) + 1 Fraction(7, 4) No special syntax there, yet it can be done. -- http://mail.python.org/mailman/listinfo/python-list
Re: string interpolation for python
On Mon, 02 Apr 2012 17:52:33 +1000, Chris Angelico wrote:
I'm -1 on the idea, mainly because there are already two perfectly good
string interpolation syntaxes that don't require a new kind of string.
Three.
% formatting
{} formatting
$ formatting using string.Template
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: string interpolation for python
On Mon, 02 Apr 2012 00:39:42 -0700, Yingjie Lan wrote:
You can already do essentially that without adding a special-case
string
formatting method to the general methods we already have.
balls = 5
people = 3
'The {people} people have {balls}
balls.'.format(**locals())
'The 3 people have 5 balls.'
Clearly dynamic strings are much more powerful, allowing arbitrary
expressions inside.
And so it may be a security risk, if user-input somehow ends up treated
as a dynamic string.
We already have three ways to evaluate arbitrary expressions:
* Python code
* eval
* exec
Why do we need yet another one?
It is also more terse and readable, since we need no
dictionary.
I think you mean terse and unreadable, since we need no dictionary. That
means that variables will be evaluated by magic from... where? Globals?
Local scope? Non-local scope? All of the above?
We already have one way of evaluating implicit variables using implicit
rules, namely regular Python code. Why do we need a second one?
I would probably rather liken dynamic expressions as a little brother of
computable documents in Mathematica. It is a new kind of expression,
rather than formatting -- though it has formatting connections.
Why do we need a new kind of expression?
Dynamic strings are mainly useful at time of writing readable code
before compilation.
What does that mean?
The compiler can choose to convert it into a string
formatting expression, of course. To efficiently format strings at
runtime, the best choice (especially
for safty reasons) is string formatting, not evaluating a dynamic
string.
So you're suggesting that we should have dynamic strings, but not
actually use dynamic strings. The compiler should just convert them to
regular string formatting.
Why not cut out the middle-man and just use regular string formatting?
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: string interpolation for python
On Mon, Apr 2, 2012 at 6:26 PM, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: On Mon, 02 Apr 2012 00:39:42 -0700, Yingjie Lan wrote: The compiler can choose to convert it into a string formatting expression, of course. To efficiently format strings at runtime, the best choice (especially for safty reasons) is string formatting, not evaluating a dynamic string. So you're suggesting that we should have dynamic strings, but not actually use dynamic strings. The compiler should just convert them to regular string formatting. Why not cut out the middle-man and just use regular string formatting? Actually, this sounds like a job for a precompiler/preprocessor. Do whatever translations you want on your code, then turn it into a .py file for execution. But hardly necessary, as there are already two - err, three, I stand corrected - perfectly good ways to do it. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: string interpolation for python
- Original Message -
From: Steven D'Aprano steve+comp.lang.pyt...@pearwood.info
To: python-list@python.org
Cc:
Sent: Monday, April 2, 2012 4:26 PM
Subject: Re: string interpolation for python
On Mon, 02 Apr 2012 00:39:42 -0700, Yingjie Lan wrote:
You can already do essentially that without adding a special-case
string
formatting method to the general methods we already have.
balls = 5
people = 3
'The {people} people have {balls}
balls.'.format(**locals())
'The 3 people have 5 balls.'
Clearly dynamic strings are much more powerful, allowing arbitrary
expressions inside.
And so it may be a security risk, if user-input somehow ends up treated
as a dynamic string.
We already have three ways to evaluate arbitrary expressions:
* Python code
* eval
* exec
Why do we need yet another one?
It is also more terse and readable, since we need no
dictionary.
I think you mean terse and unreadable, since we need no dictionary. That
means that variables will be evaluated by magic from... where? Globals?
Local scope? Non-local scope? All of the above?
We already have one way of evaluating implicit variables using implicit
rules, namely regular Python code. Why do we need a second one?
I would probably rather liken dynamic expressions as a little brother of
computable documents in Mathematica. It is a new kind of expression,
rather than formatting -- though it has formatting connections.
Why do we need a new kind of expression?
Dynamic strings are mainly useful at time of writing readable code
before compilation.
What does that mean?
The compiler can choose to convert it into a string
formatting expression, of course. To efficiently format strings at
runtime, the best choice (especially
for safty reasons) is string formatting, not evaluating a dynamic
string.
So you're suggesting that we should have dynamic strings, but not
actually use dynamic strings. The compiler should just convert them to
regular string formatting.
Why not cut out the middle-man and just use regular string formatting?
I believe non of the other three alternatives are as terse and readable.
We've got template based, formatting with dict, formatting with tuple.
They all require the coder extra effort:
Both template based and dict-based formatting require writing the
identifier three times:
name = 'Peter'
Are you %(name)s%{'name':name}
If dynamic string is used:
Are you $name$?
Template:
Template(Are you $name?).substitute(name=name)
It is three to one in compactness, what a magic 3!
Of course, the old C style way:
Are you %s?%name
Almost as terse, but not as readable, especially
when there are many parts to substitute --
the coder and reader need to be careful
to make sure the sequence is correct.
Why the Python community is so
hostile to new things now?
Python has merits,
but it is far from being perfect.
Cheers,
Yingjie
--
http://mail.python.org/mailman/listinfo/python-list
Re: string interpolation for python
Actually, this sounds like a job for a precompiler/preprocessor. Do whatever translations you want on your code, then turn it into a .py file for execution. But hardly necessary, as there are already two - err, three, I stand corrected - perfectly good ways to do it. Agree and disagree. The other ways are not perfectly good. They stinks in Python. This new way is the most natural way. Effortless, natural. That's my Python. Cheers, Yingjie -- http://mail.python.org/mailman/listinfo/python-list
Re: string interpolation for python
On Mon, Apr 2, 2012 at 7:11 PM, Yingjie Lan lany...@yahoo.com wrote:
I believe non of the other three alternatives are as terse and readable.
We've got template based, formatting with dict, formatting with tuple.
They all require the coder extra effort:
Both template based and dict-based formatting require writing the
identifier three times:
name = 'Peter'
Are you %(name)s%{'name':name}
If dynamic string is used:
Are you $name$?
Yes, it's more compact. But it's also more magic. However, there's an
alternative that's almost as compact. The only requirement is that you
use a two-character token instead of your dollar sign: a double-quote
and a plus.
Are you +name+?
That allows arbitrary expressions and everything.
Of course, the old C style way:
Are you %s?%name
Almost as terse, but not as readable, especially
when there are many parts to substitute --
the coder and reader need to be careful
to make sure the sequence is correct.
I quite like this notation, personally. It's convenient, and is
supported (with variants) in quite a few C-derived languages (and, in
spite of the massive syntactic differences, Python does have C
heritage).
Why the Python community is so
hostile to new things now?
Python has merits,
but it is far from being perfect.
Hey now, no need to get defensive :) Thing is, it's up to you to
demonstrate that your proposal justifies itself. You're proposing to
create a massive backward-compatibility issue, so you need to prove
that your new way of formatting strings is sufficiently awesome to be
able to say Well, you need Python 3.4+ to use this.
ChrisA
--
http://mail.python.org/mailman/listinfo/python-list
Re: string interpolation for python
Yingjie Lan lany...@yahoo.com wrote:
Both template based and dict-based formatting require writing the
identifier three times:
name = 'Peter'
Are you %(name)s%{'name':name}
ÿ
If dynamic string is used:
Are you $name$?
Template:
Template(Are you $name?).substitute(name=name)
It is three to one in compactness, what a magic 3!
You can avoid the duplication fairly easily:
name='Peter'
'Are you {name}?'.format(**vars())
'Are you Peter?'
though if you're doing that it would be better to limit the scope to a
specific namespace.
--
Duncan Booth http://kupuguy.blogspot.com
--
http://mail.python.org/mailman/listinfo/python-list
Re: string interpolation for python
On Mon, Apr 2, 2012 at 2:11 AM, Yingjie Lan lany...@yahoo.com wrote:
snip
I believe non of the other three alternatives are as terse and readable.
We've got template based, formatting with dict, formatting with tuple.
They all require the coder extra effort:
Both template based and dict-based formatting require writing the
identifier three times:
False. Only once is required, though the technique to achieve it is kinda hacky.
name = 'Peter'
Are you %(name)s%{'name':name}
Are you %(name)s % locals() # or vars()
If dynamic string is used:
Are you $name$?
Template:
Template(Are you $name?).substitute(name=name)
Template(Are you $name?).substitute(locals()) # or vars()
It is three to one in compactness, what a magic 3!
snip
Why the Python community is so
hostile to new things now?
It's more conservative than hostile. Here's some insight:
http://www.boredomandlaziness.org/2011/02/status-quo-wins-stalemate.html
Personally, in isolation, the only part of your proposal I find
/truly/ objectionable is the support for arbitrary expressions, since
it would tend towards encouraging suboptimal factoring. But we also
don't live in an ideal world, so the existence of the other 3 (2 of
them particularly relatively similar) alternatives is a legitimate
practical concern when evaluating your proposal. Python is
middle-aged; it's a blessing and a curse.
Cheers,
Chris
--
http://mail.python.org/mailman/listinfo/python-list
Re: string interpolation for python
Are you +name+? That allows arbitrary expressions and everything. To make that work for any type, you need: Are you + str(name) + ? Another concern is performance. You are absolutely right, they are equivalent in that both are expressions. As long as people start to realize that dynamic strings are expressions, there is no magic in it any more. And allowing expressions in those dynamic strings would make sense since they are of the same sort. dsin($x$) = $ sin(x):0.3f $ is equivalent to the expression of sin(%s%x + )= %0.3f%sin(x) Comparing th e two, I would say the latter is more computer friendly while the former, more human friendly. If the computed result is only to be used in formatting the string, it would be nice to save an assignment stmt. Almost as terse, but not as readable, especially when there are many parts to substitute -- the coder and reader need to be careful to make sure the sequence is correct. I quite like this notation, personally. It's convenient, and is supported (with variants) in quite a few C-derived languages (and, in spite of the massive syntactic differences, Python does have C heritage). Sure, once you get used to it, it would be harder to stop it the harder it is :). That's part of human nature, anyway. Why the Python community is so hostile to new things now? Python has merits, but it is far from being perfect. Hey now, no need to get defensive :) Thing is, it's up to you to demonstrate that your proposal justifies itself. You're proposing to create a massive backward-compatibility issue, so you need to prove that your new way of formatting strings is sufficiently awesome to be able to say Well, you need Python 3.4+ to use this. OK. I have put it out as is. I trust people knows good things. I would simply say: this new way is much more simple and much more powerful. And there is no security issues as long as you don't use the evil eval to evaluate expressions, which is always a security issue. It is new, and has no compatibility issues with old ways at all. In syntax, all you need is to allow d..., which clearly won't affect any old ways of business. Cheers, Yingjie -- http://mail.python.org/mailman/listinfo/python-list
Re: string interpolation for python
On Mon, 02 Apr 2012 02:11:46 -0700, Yingjie Lan wrote:
Both template based and dict-based formatting require writing the
identifier three times:
name = 'Peter'
Are you %(name)s%{'name':name}
They don't *require* this at all.
Are you %s % name
For trivial examples, you have trivial syntax. For more complex examples,
you have a more powerful system: % can accept *any* dictionary, so you
aren't limited to just pre-existing variables.
That, by the way, is perhaps the biggest problem with this idea of
dynamic strings: not that it is too powerful, but that it is TOO WEAK. It
can only retrieve names from a single namespace, and the programmer has
no control over which namespace that will be. The only way to feed named
values into the dynamic string is by creating variables.
With existing formatting systems, the programmer has complete control
over what names get used. You can set up a series of separate namespaces
and choose between them as needed:
a = dict(name=Fred, job=butcher)
b = dict(name=Sue, job=SAS sharp-shooter)
c = dict(name=Mary, job=brain surgeon)
d = dict(name=Tony, job=enforcer for the Russian mob)
for namespace in (a, b, c, d):
print (%(name)s works as a %(job)s. % namespace)
Using your dynamic strings:
for namespace in (a, b, c, d):
name = namespace[name]
job = namespace[job]
print ($name$ works as a $job$.)
and it has the side-effect that it has created some variables that are no
longer needed.
Also notice that because you have to create variables first, the dynamic
string actually requires you to write the identifier MORE times, not
fewer.
So your proposal is actually weaker than what Python already has. So why
bother? All this does is add more things to learn, more complexity in the
compiler and parser, for what? Things that you can already do.
If dynamic string is used:
Are you $name$?
And where does name come from? It's all too magical. With the existing
format strings, the programmer has *complete* control of where it comes
from:
Are you %(name)s? % locals() # from a local variable `name`
Are you %(name)s? % globals() # from a global variable `name`
Are you %(name)s? % namespace # from any namespace you like
and similar for both format() and Template.
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: string interpolation for python
Are you %(name)s % locals() # or vars() This partly solves the problem, however, you can't work with expressions inside, like: dsin($x$) = $sin(x)$ Also, what if locals() or vars() does not contain the variable x? (x could be nonlocal or global). It's more conservative than hostile. Here's some insight: http://www.boredomandlaziness.org/2011/02/status-quo-wins-stalemate.html You are probably right...Harmless enhancement is still probable? Personally, in isolation, the only part of your proposal I find /truly/ objectionable is the support for arbitrary expressions, since it would tend towards encouraging suboptimal factoring. But we also I don't quite see that as a problem. The compiler (or translator, as you mentioned earlier) could easily make dsin($x$) = $sin(x)$ into something like: ''.join([sin(, str(x), ) = , str(sin(x))] which would far more efficient than calling the format() method. Cheers, Yingjie -- http://mail.python.org/mailman/listinfo/python-list
Re: string interpolation for python
... That, by the way, is perhaps the biggest problem with this idea of dynamic strings: not that it is too powerful, but that it is TOO WEAK. ... and similar for both format() and Template. Seems you miss understood my notion of dynamic string. Dynamic strings are expressions in disguise: the things in between $...$ are plain old expressions (with optional formatting specifications). They are evaluated as if they were outside the dynamic string. We put them in there to to kill two birds with one stone: 1) ease of reading; 2) place holding. Cheers, Yingjie -- http://mail.python.org/mailman/listinfo/python-list
Re: string interpolation for python
Seems you miss understood my notion of dynamic string.
Dynamic strings are expressions in disguise: the things
in between $...$ are plain old expressions (with optional
formatting specifications). They are evaluated
as if they were outside the dynamic string. We put them
in there to to kill two birds with one stone:
1) ease of reading;
2) place holding.
like this one ?
b = dict(name=Sue, job=SAS sharp-shooter)
print $b['name']$ works as b['job']
Is it really easier to read that the following ?
{0} works as {1}.format(b['name'],b['job'])
In the case in which b is an object having job and name attribute,
the dynamic string will write
$b.name$ works as $b.job$
instead of
{0}.name works as {0}.job.format(b)
Laurent
--
http://mail.python.org/mailman/listinfo/python-list
Re: string interpolation for python
like this one ?
b = dict(name=Sue, job=SAS sharp-shooter)
print $b['name']$ works as b['job']
Is it really easier to read that the following ?
{0} works as {1}.format(b['name'],b['job'])
In the case in which b is an object having job and name
attribute, the dynamic string will write
$b.name$ works as $b.job$
instead of
{0}.name works as {0}.job.format(b)
When you already have a dict, the dict-based
formatting would be nice.
%(name)s works as %(job)s%b
If it you need to create a dict just for string
formatting, dynamic string would be nice.
Say your object has methods/properties
that fetch things from your database.
class staff:
...@property
...def name(): return 'Peter'
...
t = staff()
vars(t)
{}
t.name
'Peter'
dStaff name: $t.name$ #note the d... format
'Staff name: Peter'
Because of the d... format, it won't
affect old ways of doing things one bit.
Allowing dynamic string wouldn't hurt
a bit to anything that is already there.
--
http://mail.python.org/mailman/listinfo/python-list
Re: string interpolation for python
Yingjie Lan wrote: Seems you miss understood my notion of dynamic string. Dynamic strings are expressions in disguise: the things in between $...$ are plain old expressions (with optional formatting specifications). They are evaluated as if they were outside the dynamic string. In that case you should re-think the delimiters, so that you have something that can be nested. An example (example only, I'm not in love with it as a final form): A string that gets %(count*spacer%) in the middle A string that gets %(count*%(spacer%)%) in the middle Mel. -- http://mail.python.org/mailman/listinfo/python-list
Re: string interpolation for python
On Mon, Apr 2, 2012 at 9:46 PM, Yingjie Lan lany...@yahoo.com wrote:
Are you +name+?
That allows arbitrary expressions and everything.
To make that work for any type, you need:
Are you + str(name) + ?
Another concern is performance.
You are absolutely right, they are
equivalent in that both are expressions.
Right, meaning that both have the same issues of performance, need for
str(), etc. There's absolutely no difference.
As long as people start to realize that
dynamic strings are expressions,
there is no magic in it any more.
And no benefit. You lose out on syntax highlighting in your editor and
gain nothing.
And allowing expressions in those
dynamic strings would make sense
since they are of the same sort.
dsin($x$) = $ sin(x):0.3f $
is equivalent to the expression of
sin(%s%x + )= %0.3f%sin(x)
Comparing th e two, I would say the latter
is more computer friendly while
the former, more human friendly.
The former is more magical, the second is more explicit. Computers do
tend to like the explicit, but I wouldn't assume that humans like the
magical *necessarily*. There are times when we prefer the explicit,
too.
Sure, once you get used to it, it would be harder to stop it
the harder it is :). That's part of human nature, anyway.
Maybe. But it (percent-notation) is expressive and insanely powerful.
Moreover, it obeys the rule that you pay for the complexity you use,
no more and no less. (Although I think there's one lack in Python's
implementation - I can't find a way to use an argument more than once,
without switching to dictionary mode and using keys for everything.
I can't, for instance, use Hello hello, %s %[0]s!%name to use the
name twice. But since that isn't in the original C implementation,
it's hardly mandatory.)
Some implementations go even further than Python's does and allow an
array notation.
sprintf(UPDATE tablename SET modified=now()%{,%s=:%[0]s%} WHERE
key=%d,array_of_field_names,primary_key_value)
-- UPDATE tablename SET modified=now(),foo=:foo,bar=:bar,quux=:quux
WHERE key=1234
You're still paying for no complexity you aren't actually using. It's
clear and readable.
I would simply say: this new way is much more simple
and much more powerful. And there is no security issues
as long as you don't use the evil eval to evaluate expressions,
which is always a security issue.
It's powerful only if you use eval to allow full expression syntax.
Otherwise, what does it have above str.format()?
It is new, and has no compatibility issues with old ways at all.
In syntax, all you need is to allow d..., which clearly won't
affect any old ways of business.
You may well be able to get past the compatibility issues. I'm not yet
convinced that the new syntax is worth it, but it may be possible.
Here's a recommendation: Write a parser for your notation that turns
it into executable Python code (that is, executable in Python 3.3
without any d... support). Since a dynamic string is really an
expression in disguise, it doesn't need to be translated at run time,
and in fact is best translated at compile time. It wouldn't be hard to
make the precompiler as per your equivalency above; and since you can
use %s%123 without raising TypeError, you don't even have to figure
out data types. With that written, the Python community can more
adequately evaluate your proposal, and even start making use of it and
getting a feel for the new syntax. If it catches on, someone'll
probably invite you to write a PEP or something; if it doesn't, well,
you still have it on your own version, and you don't have to worry
about upgrades (your precompiler/patch should be able to slide up to
a new version easily).
I'm still against the idea personally, mainly because it's nothing but
a more magical way of doing what we already can do, but there may well
be many who disagree.
Chris Angelico
--
http://mail.python.org/mailman/listinfo/python-list
Re: string interpolation for python
mwil...@the-wire.com wrote: Yingjie Lan wrote: Seems you miss understood my notion of dynamic string. Dynamic strings are expressions in disguise: the things in between $...$ are plain old expressions (with optional formatting specifications). They are evaluated as if they were outside the dynamic string. In that case you should re-think the delimiters, so that you have something that can be nested. An example (example only, I'm not in love with it as a final form): A string that gets %(count*spacer%) in the middle A string that gets %(count*%(spacer%)%) in the middle A less than great example, I guess. Maybe A string that gets %(count*-%(spacer%)-%) in the middle Mel. -- http://mail.python.org/mailman/listinfo/python-list
Re: string interpolation for python
Right, meaning that both have the same issues
of performance, need for
str(), etc. There's absolutely no difference.
OK, performance. Here is a new solution:
Suppose we have a new string method
str.format_join([...])
taking a list of strings and objects,
with even-indexed ones being strings,
odd-indexed ones being objects.
Each even-indexed string *ends* with a formatting
specification for the next object in the list.
Then we can have:
dsin($x$) = $ sin(x):0.3f $
get translated to:
''.format_join([sin(%s,x,) = %0.3f, sin(x)])
This seems to be at least as good in performance.
And no benefit. You lose out on syntax highlighting
in your editor and gain nothing.
Gain: readability, terseness, hassle-free, and possibly
better performance if done right.
Syntax highlighting: can be done more creatively.
For dynamic strings, string parts are like normal
strings, but the embedded expressions are like
normal expressions :)
sprintf(UPDATE tablename SET modified=now()%{,%s=:%[0]s%} WHERE
key=%d,array_of_field_names,primary_key_value)
-- UPDATE tablename SET modified=now(),foo=:foo,bar=:bar,quux=:quux
WHERE key=1234
You're still paying for no complexity you aren't actually using.
It's clear and readable.
You are really good at that. Maybe not everybody is as
experience as you, and I suppose the learning curve is
kind of hard to climb.
It's powerful only if you use eval to allow full expression syntax.
Otherwise, what does it have above str.format()?
Those expressions are embedded, you don't need eval()
to have the result though. Are we on the same page?
You may well be able to get past the compatibility issues. I'm not yet
convinced that the new syntax is worth it, but it may be possible.
Here's a recommendation: Write a parser for your notation that turns
it into executable Python code (that is, executable in Python 3.3
without any d... support).
You mean a translator?
The syntax is essential for compatibility.
We must distinguish dynamic strings from common strings.
They will live peacefully together.
(escaping the '$' in normal strings breaks compatibility,
and the consequence of forgetting to escape could be
disastrous, so definitely not an option).
May be d is too tiny, $... is easier to pick out.
Cheers,
Yingjie
--
http://mail.python.org/mailman/listinfo/python-list
Re: string interpolation for python
In that case you should re-think the delimiters, so that you have something that can be nested. An example (example only, I'm not in love with it as a final form): Haven't really thought about that. Nesting is a big issue if in the embedded expression, there is another dynamic string expression. For example (the first '%' immediately preceding the first quote denotes the start of a dynamic string expression):: first_name, family_name = Peter, Johns %Hi $ first_name+%, $family_name$ $! 'Hi Peter, Johns!' However, nesting is really cluttering things up and we should forbid it without loosing power: The above example can be done without nesting: %Hi $ first_name$, $family_name$! 'Hi Peter, Johns!' Can you offer an example where nesting is better or necessary? Cheers, Yingjie -- http://mail.python.org/mailman/listinfo/python-list
Re: string interpolation for python
On Tue, 03 Apr 2012 00:58:38 +1000, Chris Angelico wrote:
Maybe. But it (percent-notation) is expressive and insanely powerful.
Moreover, it obeys the rule that you pay for the complexity you use, no
more and no less. (Although I think there's one lack in Python's
implementation - I can't find a way to use an argument more than once,
without switching to dictionary mode and using keys for everything. I
can't, for instance, use Hello hello, %s %[0]s!%name to use the name
twice. But since that isn't in the original C implementation, it's
hardly mandatory.)
Ack.
In this case, you can use format:
Hello {0}, what's up with {1}? Hey, {0} I'm speaking to you!.format
('George', 'Melissa')
Hello George, what's up with Melissa? Hey, George I'm speaking to you!
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: string interpolation for python
On Mar 31, 3:29 am, Terry Reedy tjre...@udel.edu wrote:
On 3/31/2012 2:22 AM, Yingjie Lan wrote:
Hi all,
I'd really like to share this idea of string interpolation for formatting.
Let's start with some code:
name = Shrek
print( Hi, $name$!)
Hi, Shrek!
balls = 30
print( We have $balls$ balls.)
We have 30 balls
You can already do essentially that without adding a special-case string
formatting method to the general methods we already have.
balls = 5
people = 3
'The {people} people have {balls} balls.'.format(**locals())
'The 3 people have 5 balls.'
I was wondering how much of a performance penalty you pay for using
the **locals() idiom, because I use it myself sometimes.
It turns out there is a slight penalty for **locals() vs. explicitly
passing in arguments to format (e.g. .format(balls=balls,
people=people), although it's probably negligible in 99.9% of use
cases.
def yo(a):
x = 1
y = 2
z = 3
a = b = c = d = 7
for i in range(10):
# s = {x} {y} {z}.format(**locals())
s = {x} {y} {z}.format(x=x, y=y, z=z)
for i in range(1):
yo(i)
# .150s for **locals()
# .131s for explicit x/y/z
--
http://mail.python.org/mailman/listinfo/python-list
Re: Re: string interpolation for python
On 01/-10/-28163 01:59 PM, Yingjie Lan wrote: Because of the d... format, it won't affect old ways of doing things one bit. Allowing dynamic string wouldn't hurt a bit to anything that is already there. Why don't you just write a function that does it? I think someone already suggested this... import dynamic_strings # write this d = dynamic_strings.dynamic x = 5 print(d(x=$x$)) ? Sure, it's not *quite* as pretty as if you could just say dx=$x$, and you might have to do some hacky uglyness in the implementation to get at the locals of the calling procedure, but it solves a bazillion problems, such as: 1. YOU can do it, instead of hoping it gets into the mainline interpreter 2. You can do it NOW, and it will work with any version of Python 3. You have the freedom to easily add eval from *this dictionary* if you want, which solves Steven D'Aprano's objection that your suggestion is too weak. 4. Languages changes should be viewed suspiciously in general. Evan -- http://mail.python.org/mailman/listinfo/python-list
Re: string interpolation for python
On Apr 2, 2:11 pm, Yingjie Lan lany...@yahoo.com wrote: Almost as terse, but not as readable, especially... Hi Yingjie, Just in case you are not a native speaker of English, 'terse' is a mildly pejorative word, ie it is not 'good'. You probably want to use something like 'concise', or just plain 'short.' As for your suggestion, Ive nothing much to say: It probably comes from a perl culture and who is to way which culture is better? Not me anyway, whenever I look at perl code my head spins... [No flaming here -- I just dont know perl] -- http://mail.python.org/mailman/listinfo/python-list
Re: string interpolation for python
On Tue, Apr 3, 2012 at 4:56 AM, Steven D'Aprano
steve+comp.lang.pyt...@pearwood.info wrote:
On Tue, 03 Apr 2012 00:58:38 +1000, Chris Angelico wrote:
I can't find a way to use an argument more than once,
without switching to dictionary mode and using keys for everything.
Ack.
In this case, you can use format:
Hello {0}, what's up with {1}? Hey, {0} I'm speaking to you!.format
('George', 'Melissa')
Hello George, what's up with Melissa? Hey, George I'm speaking to you!
Yes, it's possible with that version but not with % formatting. (Is
your Ack short for Acknowledgement, or is it an ejaculation of
disgust?)
On Tue, Apr 3, 2012 at 1:49 AM, Yingjie Lan lany...@yahoo.com wrote:
Then we can have:
dsin($x$) = $ sin(x):0.3f $
get translated to:
''.format_join([sin(%s,x,) = %0.3f, sin(x)])
This seems to be at least as good in performance.
Or don't bother with the initial string, and simply pass everything as
arguments:
def yingjie_format(*args):
it=iter(args)
return ''.join(s%next(it,None) for s in it)
yingjie_format(sin(%s,x,) = %0.3f, sin(x))
Tested and working (Python 3.2 on Windows). Okay, the generator's a
bit cryptic, but you could do the same with a more explicit loop if
you prefer.
Syntax highlighting: can be done more creatively.
For dynamic strings, string parts are like normal
strings, but the embedded expressions are like
normal expressions :)
So if they're exactly like normal expressions, why not simply use
normal expressions?
sprintf(UPDATE tablename SET modified=now()%{,%s=:%[0]s%} WHERE
key=%d,array_of_field_names,primary_key_value)
-- UPDATE tablename SET modified=now(),foo=:foo,bar=:bar,quux=:quux
WHERE key=1234
You're still paying for no complexity you aren't actually using.
It's clear and readable.
You are really good at that. Maybe not everybody is as
experience as you, and I suppose the learning curve is
kind of hard to climb.
Yes, it takes some learning to use it. But that's true of everything,
no less of your magical string interpolation. My point is that simple
examples should be (and are, with printf formatting) simple, such that
you only get those more complicated format strings when you're
actually doing a complicated job (in that case, taking each element of
an array and using it twice - actually, it was taking the indices of a
mapping that would end up being passed to the DB query function, thus
providing values to the :foo :bar variables).
Those expressions are embedded, you don't need eval()
to have the result though. Are we on the same page?
I can see three implementation paths:
1) Language feature. It really *is* just an expression. There's no way
that a user can provide them, so there's actually no similarity to
eval. But this requires that Python itself handle things.
2) Precompiler. It *becomes* an expression. Again, perfectly safe,
although I don't know how useful this really is.
3) Functoin. As several have suggested, you could do some magic and
use d(this is a $dollar$ $interpolated$ string) to implement. For
this, you *will* need eval (or something like it).
Here's a recommendation: Write a parser for your notation that turns
it into executable Python code (that is, executable in Python 3.3
without any d... support).
You mean a translator?
Yes. It translates your dollar-strings into something that's legal
Python 3.3 syntax - either calls to a function like I provided above,
or actual embedded expressions.
The syntax is essential for compatibility.
We must distinguish dynamic strings from common strings.
They will live peacefully together.
(escaping the '$' in normal strings breaks compatibility,
and the consequence of forgetting to escape could be
disastrous, so definitely not an option).
May be d is too tiny, $... is easier to pick out.
I don't like the use of symbols like that; can someone, glancing at
your code, tell whether $ is an operator, a name, or something else?
The original d is probably better for that.
ChrisA
--
http://mail.python.org/mailman/listinfo/python-list
Re: string interpolation for python
I can't find a way to use an argument more than once,
without switching to dictionary mode and using keys for
everything.
Even in dictionary mode, the key is spelled more than
once. The tuple mode below seems to save some typing.
However, when there are more and more items to format,
the readability deteriorates quickly in the tuple mode.
Use an argument more than once is, after all, probably a
not-so-often use case.
The dictionary mode greatly enhances readability, but
you have to provide a dict object and make sure the name in
your formatting string matches the keys in the dictionary.
And this matching requirement is because of information
redundancy in the arrangement. And information redundancy
is often the root of evil for various kinds of trouble.
Dynamic string seems to have the good without redundancy.
And there is no need to build a dict for it.
However, when there is already the dict for use,
clearly the dict format is the winner.
Ack.
In this case, you can use format:
Hello {0}, what's up with {1}? Hey, {0} I'm
speaking to you!.format
('George', 'Melissa')
Hello George, what's up with Melissa? Hey, George I'm
speaking to you!
Or don't bother with the initial string, and simply pass everything as
arguments:
def yingjie_format(*args):
it=iter(args)
return ''.join(s%next(it,None) for s in it)
yingjie_format(sin(%s,x,) = %0.3f, sin(x))
That's very nice, thanks!
So if they're exactly like normal expressions, why not simply use
normal expressions?
I think use dynamic strings can have these benefits:
1) you less keys (punch less keys).
2) better readability (less clutters)
3) you don't have to explicit convert/format expressions into strings
4) better performance too (adding strings together is generally slow).
sprintf(UPDATE tablename SET modified=now()%{,%s=:%[0]s%} WHERE
key=%d,array_of_field_names,primary_key_value)
-- UPDATE tablename SET
modified=now(),foo=:foo,bar=:bar,quux=:quux
WHERE key=1234
You're still paying for no complexity you aren't actually
using.
It's clear and readable.
You are really good at that. Maybe not everybody is as
experience as you, and I suppose the learning curve is
kind of hard to climb.
Yes, it takes some learning to use it. But that's true of everything,
no less of your magical string interpolation. My point is that simple
examples should be (and are, with printf formatting) simple, such that
you only get those more complicated format strings when you're
actually doing a complicated job (in that case, taking each element of
an array and using it twice - actually, it was taking the indices of a
mapping that would end up being passed to the DB query function, thus
providing values to the :foo :bar variables).
Sure. But if one thing does well on both simple and complex
situations, why not that thing?
Those expressions are embedded, you don't need eval()
to have the result though. Are we on the same page?
I can see three implementation paths:
1) Language feature. It really *is* just an expression. There's no way
that a user can provide them, so there's actually no similarity to
eval. But this requires that Python itself handle things.
2) Precompiler. It *becomes* an expression. Again, perfectly safe,
although I don't know how useful this really is.
3) Functoin. As several have suggested, you could do some magic and
use d(this is a $dollar$ $interpolated$ string) to implement. For
this, you *will* need eval (or something like it).
Sure. for 1), things can be done most conveniently and efficiently.
for 2), yeah, not sure how useful it is.
for 3), maybe can let str class have a property like: dy.
which can do all the dirty processing. Then we may do:
x = 0
sin($x$) = $sin(x)$.dy
'sin(0) = 0.0'
Not much burden to use except for the CPU, I suppose.
You mean a translator?
Yes. It translates your dollar-strings into something that's legal
Python 3.3 syntax - either calls to a function like I provided above,
or actual embedded expressions.
The syntax is essential for compatibility.
We must distinguish dynamic strings from common strings.
They will live peacefully together.
(escaping the '$' in normal strings breaks compatibility,
and the consequence of forgetting to escape could be
disastrous, so definitely not an option).
May be d is too tiny, $... is easier to pick out.
I don't like the use of symbols like that; can someone, glancing at
your code, tell whether $ is an operator, a name, or something else?
The original d is probably better for that.
Yeah, the d is probably better.
Cheers,
Yingjie
--
http://mail.python.org/mailman/listinfo/python-list
Re: string interpolation for python
On Fri, Mar 30, 2012 at 11:22 PM, Yingjie Lan lany...@yahoo.com wrote: Hi all, I'd really like to share this idea of string interpolation for formatting. Let's start with some code: name = Shrek print( Hi, $name$!) Hi, Shrek! Python already has *3* different built-in string formatting/interpolation systems: http://docs.python.org/library/string.html#template-strings http://docs.python.org/library/string.html#format-string-syntax http://docs.python.org/library/stdtypes.html#string-formatting-operations Do we really want to add yet another to this pile? I would surmise that your key implicitly grab variable values from the enclosing scope feature has previously been rejected for being too magical. Cheers, Chris -- http://rebertia.com -- http://mail.python.org/mailman/listinfo/python-list
RE: string interpolation for python
Hi Yingjie,
Consider this snippet of safe code:
| enc = bobsencryption.Encoder('Some secret key')
|
| username = raw_input('Enter your username:')
| password = raw_input('Enter your password:')
|
| print
| print username + ', please wait while we dial-up and log you in...'
|
| connection = server.dialup(00441635074745)
| connection.send('login ' + enc([username, password]))
Code like this could already be out there and safe-ish (well, if
they've included a little validation and error-checking.) Now consider
that your $formatting$ is added and the company upgrades Python,
resulting in the following:
| Enter your username: $enc.key$
| Enter your password: dontneedone
|
| Some secret key, please wait while we dial-up and log you in...
It could break old code... okay you may say you should’nt allow
certain characters but if they're printable and used in a controlled
environment those characters can dramatically increase the security
of a username and password.
Adrian
--
http://mail.python.org/mailman/listinfo/python-list
Re: string interpolation for python
On 3/31/2012 2:22 AM, Yingjie Lan wrote:
Hi all,
I'd really like to share this idea of string interpolation for formatting.
Let's start with some code:
name = Shrek
print( Hi, $name$!)
Hi, Shrek!
balls = 30
print( We have $balls$ balls.)
We have 30 balls
You can already do essentially that without adding a special-case string
formatting method to the general methods we already have.
balls = 5
people = 3
'The {people} people have {balls} balls.'.format(**locals())
'The 3 people have 5 balls.'
--
Terry Jan Reedy
--
http://mail.python.org/mailman/listinfo/python-list
