Re: Temporary variables in list comprehensions

[Piet van Oostrum]

Vincent Vande Vyvre  writes:

> final = [(x, y+1) for x, y in zip(e, e)]
final = [(x, x+1) for x in e]
-- 
Piet van Oostrum 
WWW: http://pietvanoostrum.com/
PGP key: [8DAE142BE17999C4]
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Jussi Piitulainen]

Roel Schroeven writes:

> Lele Gaifax schreef op 6/04/2017 20:07:
>> Piet van Oostrum  writes:
>>
>>> It is a poor man's 'let'. It would be nice if python had a real 'let'
>>> construction. Or for example:
>>>
>>> [(tmp, tmp + 1) for x in data with tmp = expensive_calculation(x)]
>>>
>>> Alas!
>>
>> It would be nice indeed!
>>
>> Or even
>>
>>   [(tmp, tmp + 1) for x in data
>>with expensive_calculation(x) as tmp
>>if tmp is not None]
>>
>
> Perhaps this:
>
> [(tmp, tmp + 1) for tmp in
>  (expensive_calculation(x) for x in data)
>  if tmp is not None]
>
> A bit less elegant, but works right now.

The "poor man's let" works right now.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Roel Schroeven]

Lele Gaifax schreef op 6/04/2017 20:07:

Piet van Oostrum  writes:


It is a poor man's 'let'. It would be nice if python had a real 'let'
construction. Or for example:

[(tmp, tmp + 1) for x in data with tmp = expensive_calculation(x)]

Alas!


It would be nice indeed!

Or even

  [(tmp, tmp + 1) for x in data
   with expensive_calculation(x) as tmp
   if tmp is not None]



Perhaps this:

[(tmp, tmp + 1) for tmp in
 (expensive_calculation(x) for x in data)
 if tmp is not None]

A bit less elegant, but works right now.


--
Roel Schroeven

--
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Tim Chase]

On 2017-04-06 14:56, Vincent Vande Vyvre wrote:
> With two passes
> 
> e = [expensive_calculation(x) for x in data]
> final = [(x, y+1) for x, y in zip(e, e)]

Using a generator it can be done in one pass:

 final = [
   (value, tmp, tmp+1)
   for value, tmp
   in (
 (x, expensive_calculation(x))
 for x in data
 )
   ]

The above makes use of the original value as well at top level
(whether you need it for "if" filtering, or in your final tuple
result). If you don't care, you can discard it

 final = [
   (tmp, tmp+1)
   for tmp
   in (
 expensive_calculation(x)
 for x in data
 )
   ]

-tkc


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Lele Gaifax]

Piet van Oostrum  writes:

> It is a poor man's 'let'. It would be nice if python had a real 'let'
> construction. Or for example:
>
> [(tmp, tmp + 1) for x in data with tmp = expensive_calculation(x)]
>
> Alas!

It would be nice indeed!

Or even

  [(tmp, tmp + 1) for x in data
   with expensive_calculation(x) as tmp
   if tmp is not None]

that fits the usual "with" syntax.

ciao, lele.
-- 
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
l...@metapensiero.it  | -- Fortunato Depero, 1929.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Jussi Piitulainen]

Vincent Vande Vyvre writes:

> Le 06/04/17 à 14:25, Piet van Oostrum a écrit :
>> Steven D'Aprano  writes:
>>
>>> Suppose you have an expensive calculation that gets used two or more
>>> times in a loop. The obvious way to avoid calculating it twice in an
>>> ordinary loop is with a temporary variable:
>>>
>>> result = []
>>> for x in data:
>>>  tmp = expensive_calculation(x)
>>>  result.append((tmp, tmp+1))
>>>
>>>
>>> But what if you are using a list comprehension? Alas, list comps
>>> don't let you have temporary variables, so you have to write this:
>>>
>>>
>>> [(expensive_calculation(x), expensive_calculation(x) + 1) for x in data]
>>>
>>>
>>> Or do you? ... no, you don't!
>>>
>>>
>>> [(tmp, tmp + 1) for x in data for tmp in [expensive_calculation(x)]]
>>>
>>>
>>> I can't decide whether that's an awesome trick or a horrible hack...
>> It is a poor man's 'let'. It would be nice if python had a real 'let'
>> construction. Or for example:
>>
>> [(tmp, tmp + 1) for x in data with tmp = expensive_calculation(x)]
>>
>> Alas!
>
> With two passes
>
> e = [expensive_calculation(x) for x in data]
> final = [(x, y+1) for x, y in zip(e, e)]
>
> Vincent

Imagine some crazy combinatory question - how many ways can one choose
two subsets of the ten decimal digits so that the size of the first is
the minimum of the second and the size of the second is the maximum of
the first _or_ the minima and maxima of the two are the same?

Comprehensions lend themselves readily to such explorations. It happens
that some expensively computed value is needed twice, like the minima
and maxima of the two combinations in this exercise (because this
exercise was carefully crafted to be just so, but anyway), and then it
saves time to do the computations once: let the values have names.

from itertools import combinations as choose

print(sum(1 for m in range(1,10) for n in range(1,10)
  for a in choose(range(1,10), m)
  for b in choose(range(1,10), n)
  if ((len(a) == min(b) and len(b) == max(a)) or
  (min(a) == min(b) and max(a) == max(b)

print(sum(1 for m in range(1,10) for n in range(1,10)
  for a in choose(range(1,10), m)
  for b in choose(range(1,10), n)
  for lena, mina, maxa in [[len(a), min(a), max(a)]]
  for lenb, minb, maxb in [[len(b), min(b), max(b)]]
  if ((lena == minb and lenb == maxa) or
  (mina == minb and maxa == maxb

I realized afterwards that the sizes, len(a) and len(b), already had
names, m and n, and were only used once in the condition anyway, but let
that illustrate the point: this kind of expression lends itself to
analysis and modification, which is what one wants in explorative code.

(But the "for x in [foo(u,w)]" works, so, shrug, I guess? I'd welcome a
proper let construction, but then I find that I can live without.)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Vincent Vande Vyvre]

Le 06/04/17 à 14:25, Piet van Oostrum a écrit :

Steven D'Aprano  writes:


Suppose you have an expensive calculation that gets used two or more times in a
loop. The obvious way to avoid calculating it twice in an ordinary loop is with
a temporary variable:

result = []
for x in data:
 tmp = expensive_calculation(x)
 result.append((tmp, tmp+1))


But what if you are using a list comprehension? Alas, list comps don't let you
have temporary variables, so you have to write this:


[(expensive_calculation(x), expensive_calculation(x) + 1) for x in data]


Or do you? ... no, you don't!


[(tmp, tmp + 1) for x in data for tmp in [expensive_calculation(x)]]


I can't decide whether that's an awesome trick or a horrible hack...

It is a poor man's 'let'. It would be nice if python had a real 'let'
construction. Or for example:

[(tmp, tmp + 1) for x in data with tmp = expensive_calculation(x)]

Alas!


With two passes

e = [expensive_calculation(x) for x in data]
final = [(x, y+1) for x, y in zip(e, e)]

Vincent


--
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Piet van Oostrum]

Steven D'Aprano  writes:

> Suppose you have an expensive calculation that gets used two or more times in 
> a 
> loop. The obvious way to avoid calculating it twice in an ordinary loop is 
> with 
> a temporary variable:
>
> result = []
> for x in data:
> tmp = expensive_calculation(x)
> result.append((tmp, tmp+1))
>
>
> But what if you are using a list comprehension? Alas, list comps don't let 
> you 
> have temporary variables, so you have to write this:
>
>
> [(expensive_calculation(x), expensive_calculation(x) + 1) for x in data]
>
>
> Or do you? ... no, you don't!
>
>
> [(tmp, tmp + 1) for x in data for tmp in [expensive_calculation(x)]]
>
>
> I can't decide whether that's an awesome trick or a horrible hack...

It is a poor man's 'let'. It would be nice if python had a real 'let'
construction. Or for example:

[(tmp, tmp + 1) for x in data with tmp = expensive_calculation(x)]

Alas!
-- 
Piet van Oostrum 
WWW: http://pietvanoostrum.com/
PGP key: [8DAE142BE17999C4]
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[breamoreboy]

On Sunday, April 2, 2017 at 1:08:17 AM UTC+1, Robert L. wrote:

> I don't believe in western morality, i.e. don't kill civilians or children
> The only way to fight a moral war is the Jewish way: Destroy their holy sites.
> Kill men, women, and children (and cattle). --- Rabbi Manis Friedman
> web.archive.org/web/20090605154706/http://www.momentmag.com/Exclusive/2009/2009-06/200906-Ask_Rabbis.html
> archive.org/download/DavidDukeVideo/TheZionistMatrixOfPowerddhd.ogv

Completely agree with Steven D'Aprano so would the moderators please ban Robert 
L with immediate effect.

Kindest regards.

Mark Lawrence.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Robert L.]

On 1/8/2017, Steven D'Aprano wrote:

> Suppose you have an expensive calculation that gets used two or
> more times in a loop. The obvious way to avoid calculating it
> twice in an ordinary loop is with a temporary variable:
> 
> result = []
> for x in data:
> tmp = expensive_calculation(x)
> result.append((tmp, tmp+1))
> 
> 
> But what if you are using a list comprehension? Alas, list comps
> don't let you have temporary variables, so you have to write
> this:
> 
> [(expensive_calculation(x), expensive_calculation(x) + 1) for x in data]
> 
> Or do you? ... no, you don't!
> 
> [(tmp, tmp + 1) for x in data for tmp in [expensive_calculation(x)]]

[2,3,5].map{|n| tmp = Math.sqrt n; [tmp, tmp+1]}

 ===> 
[[1.4142135623730951, 2.414213562373095],
 [1.7320508075688772, 2.732050807568877],
 [2.23606797749979, 3.23606797749979]]


-- 
I don't believe in western morality, i.e. don't kill civilians or children
The only way to fight a moral war is the Jewish way: Destroy their holy sites.
Kill men, women, and children (and cattle). --- Rabbi Manis Friedman
web.archive.org/web/20090605154706/http://www.momentmag.com/Exclusive/2009/2009-06/200906-Ask_Rabbis.html
archive.org/download/DavidDukeVideo/TheZionistMatrixOfPowerddhd.ogv
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Chris Angelico]

On Sun, Apr 2, 2017 at 11:53 AM, Steve D'Aprano
 wrote:
> Robert, I've asked you once to stop posting anti-Semitic signatures in your
> posts. You've now posted four times, and each one has included racist
> material in the signature.
>
> You are welcome to participate here if you discuss Python, or even to
> discuss general programming techniques, but if you continue to post
> anti-Semitic or other racist links and so-called "quotes" (usually invented
> or made up), you will be reported for a CoC violation and banned from the
> mailing list and kill-filed on the newsgroup.

If you were responding to something in this thread, I didn't even see
it, which quite possibly means he's already banned from the mailing
list.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Steve D'Aprano]

Robert, I've asked you once to stop posting anti-Semitic signatures in your
posts. You've now posted four times, and each one has included racist
material in the signature.

You are welcome to participate here if you discuss Python, or even to
discuss general programming techniques, but if you continue to post
anti-Semitic or other racist links and so-called "quotes" (usually invented
or made up), you will be reported for a CoC violation and banned from the
mailing list and kill-filed on the newsgroup.




On Sun, 2 Apr 2017 10:08 am, Robert L. wrote:

> I don't believe in western morality[...]
> archive.org/download/DavidDukeVideo/[...]


-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Steven D'Aprano]

On Tuesday 10 January 2017 00:12, Antoon Pardon wrote:

> Op 09-01-17 om 04:53 schreef Steven D'Aprano:
>> Suppose you have an expensive calculation that gets used two or more times
>> in a loop. The obvious way to avoid calculating it twice in an ordinary loop
>> is with a temporary variable:
[...]
>> I can't decide whether that's an awesome trick or a horrible hack...
> 
> Maybe this in an occasion to use your recipe.
> 
> http://code.activestate.com/recipes/580625-collection-pipeline-in-python/
> 
> result = data | Map(expensive_calculation) | Map(lambda tmp: (tmp, tmp + 1))
> | List
> 


Indeed :-)



-- 
Steven
"Ever since I learned about confirmation bias, I've been seeing 
it everywhere." - Jon Ronson

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Paul Moore]

On Monday, 9 January 2017 03:53:37 UTC, Steven D'Aprano  wrote:
> Suppose you have an expensive calculation that gets used two or more times
> in a loop.

[...]

> [(tmp, tmp + 1) for x in data for tmp in [expensive_calculation(x)]]
> 
> I can't decide whether that's an awesome trick or a horrible hack...

My immediate reaction is that it depends on the constraints that made it 
important for you to use a comprehension rather than an explicit loop in the 
first place.

For a toy example like this it's obviously a horrible hack, and you should 
simply make an explicit loop:

result = []
for x in data:
  # Don't recalculate this value, as it's expensive!
  val = expensive_calculation(x)
  result.append((val, val+1))

In a real world case, maybe there's a good reason why you'd prefer to stick 
with a comprehension. In that case, you look at trade-offs like

def intermediate_tuple(val): return val, val+1
[intermediate_tuple(x) for x in data]

or

[(lambda val: (val, val+1))(x) for x in data]

or your version.

All have their own unique uglinesses, as does the explicit loop. Personally my 
preferences would be the explicit loop, then the intermediate_tuple function, 
in that order.

Paul
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[jmp]

On 01/09/2017 04:53 AM, Steven D'Aprano wrote:

Suppose you have an expensive calculation that gets used two or more times in a
loop. The obvious way to avoid calculating it twice in an ordinary loop is with
a temporary variable:

result = []
for x in data:
 tmp = expensive_calculation(x)
 result.append((tmp, tmp+1))


But what if you are using a list comprehension? Alas, list comps don't let you
have temporary variables, so you have to write this:


[(expensive_calculation(x), expensive_calculation(x) + 1) for x in data]


Or do you? ... no, you don't!


[(tmp, tmp + 1) for x in data for tmp in [expensive_calculation(x)]]


I can't decide whether that's an awesome trick or a horrible hack...



In any situation, the double list comprehension (also used to flatten 
lists) is very difficult to read.


What about

for x in (f(d) for d in data):
   result.append(x, x+1)


There's a double for loop in the same line but the generator parenthesis 
help a lot. No lame tmp variable involved.


JM

--
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Serhiy Storchaka]

On 09.01.17 12:46, Paul Rubin wrote:

Serhiy Storchaka  writes:

gen = (expensive_calculation(x) for x in data)
result = [(tmp, tmp + 1) for tmp in gen]


result = [(tmp, tmp+1) for tmp in map(expensive_calculation, data)]


Yes, of course, but only in the case of one-argument function 
expensive_calculation(). If this is just an expensive expression or need 
to pass multiple arguments to an expensive function, a generator 
expression could look simpler than a lambda or a local function.



--
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Paul Rubin]

Ben Bacarisse  writes:
>   [(lambda tmp: (tmp, tmp+1))(expensive_calculation(x)) for x in data]

Nice.  The Haskell "let" expression is implemented as syntax sugar for
that, I believe.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Tim Chase]

On 2017-01-09 13:16, Paul Rubin wrote:
> Tim Chase  writes:
> >> result = [(tmp, tmp+1) for tmp in map(expensive_calculation,
> >> data)]
> >
> > As charmingly expressive as map() is, the wildly different
> > behavior in py3 (it's a generator that evaluates lazily) vs py2
> > (it consumes the entire iterable in one go) leads me to avoid it
> > in general,
> 
> Well, there's itertools.imap which maps lazily in py2.

Yes, but it's one of those things that I have to remember to
distinguish which one I use based on whether I'm working in py2 or
py3.  Meanwhile, using a generator expression is readable (my #1
reason for using Python is its readability) and works across 2 and 3
without changes or thinking about it further.  So that's what I tend
to use.

-tkc


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Ben Bacarisse]

Steven D'Aprano  writes:

> Suppose you have an expensive calculation that gets used two or more times in 
> a 
> loop. The obvious way to avoid calculating it twice in an ordinary loop is 
> with 
> a temporary variable:
>
> result = []
> for x in data:
> tmp = expensive_calculation(x)
> result.append((tmp, tmp+1))
>
>
> But what if you are using a list comprehension? Alas, list comps don't let 
> you 
> have temporary variables, so you have to write this:
>
>
> [(expensive_calculation(x), expensive_calculation(x) + 1) for x in data]
>
>
> Or do you? ... no, you don't!
>
>
> [(tmp, tmp + 1) for x in data for tmp in [expensive_calculation(x)]]
>
>
> I can't decide whether that's an awesome trick or a horrible hack...

Nor I (but then I'm not really a Pythonista).  However I would use a
lambda as the natural way to create a local binding:

  [(lambda tmp: (tmp, tmp+1))(expensive_calculation(x)) for x in data]

-- 
Ben.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Paul Rubin]

Tim Chase  writes:
>> result = [(tmp, tmp+1) for tmp in map(expensive_calculation, data)]
>
> As charmingly expressive as map() is, the wildly different behavior in
> py3 (it's a generator that evaluates lazily) vs py2 (it consumes the
> entire iterable in one go) leads me to avoid it in general,

Well, there's itertools.imap which maps lazily in py2.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Christian Gollwitzer]

Am 09.01.17 um 04:53 schrieb Steven D'Aprano:

Or do you? ... no, you don't!

[(tmp, tmp + 1) for x in data for tmp in [expensive_calculation(x)]]

I can't decide whether that's an awesome trick or a horrible hack...


I think this is quite clear, and a useful feature, only that Python 
makes it unnecessarily hard. In Haskell, there is a "let" clause, which 
would allow to write it as:


[let tmp = expensive_calc(x), (tmp, tmp+1)  for x in data]

or better readable using "with" or "where" as in

[(tmp, tmp + 1) with tmp = expensive_calc(x) for x in data]

or similar. So maybe that's a PEP to extend the list comprehension syntax?

Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Antonio Caminero Garcia]

On Sunday, January 8, 2017 at 7:53:37 PM UTC-8, Steven D'Aprano wrote:
> Suppose you have an expensive calculation that gets used two or more times in 
> a 
> loop. The obvious way to avoid calculating it twice in an ordinary loop is 
> with 
> a temporary variable:
> 
> result = []
> for x in data:
> tmp = expensive_calculation(x)
> result.append((tmp, tmp+1))
> 
> 
> But what if you are using a list comprehension? Alas, list comps don't let 
> you 
> have temporary variables, so you have to write this:
> 
> 
> [(expensive_calculation(x), expensive_calculation(x) + 1) for x in data]
> 
> 
> Or do you? ... no, you don't!
> 
> 
> [(tmp, tmp + 1) for x in data for tmp in [expensive_calculation(x)]]
> 
> 
> I can't decide whether that's an awesome trick or a horrible hack...
> 
> 
> -- 
> Steven
> "Ever since I learned about confirmation bias, I've been seeing 
> it everywhere." - Jon Ronson

Hello I saw some memoizing functions, in that sense you can use the 
functools.lru_cache decorator, that is even better if you have repeated 
elements in data.

@functools.lru_cache
def expensive_calculation(x):
# very NP-hard calculation
pass


Hope that helps :)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Tim Chase]

On 2017-01-09 04:59, Rustom Mody wrote:
> What happens when the expensive is on an inner generator?
> Something like:
> 
> [expensive₂(y)  for x in data for y in foo(x)]
> 
> [The ₂ representing the 2 or more occurrences in Steven's eg]

Well, if I understand your question correctly, the goal would be to
execute each expensive operation only once, regardless of how many
expensive functions you execute:

  [(x, a, b, x+1, a+1, b+1)
   for x, a, b
   in (
 x,
 expensive_a(x),
 expensive_b(x),
 for x
 in data
 )
   # if x > 42 # optionally test against values
   ]

-tkc



-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Antoon Pardon]

Op 09-01-17 om 04:53 schreef Steven D'Aprano:
> Suppose you have an expensive calculation that gets used two or more times in 
> a 
> loop. The obvious way to avoid calculating it twice in an ordinary loop is 
> with 
> a temporary variable:
>
> result = []
> for x in data:
> tmp = expensive_calculation(x)
> result.append((tmp, tmp+1))
>
>
> But what if you are using a list comprehension? Alas, list comps don't let 
> you 
> have temporary variables, so you have to write this:
>
>
> [(expensive_calculation(x), expensive_calculation(x) + 1) for x in data]
>
>
> Or do you? ... no, you don't!
>
>
> [(tmp, tmp + 1) for x in data for tmp in [expensive_calculation(x)]]
>
>
> I can't decide whether that's an awesome trick or a horrible hack...

Maybe this in an occasion to use your recipe.

http://code.activestate.com/recipes/580625-collection-pipeline-in-python/

result = data | Map(expensive_calculation) | Map(lambda tmp: (tmp, tmp + 1)) | 
List

-- 
Antoon.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Rustom Mody]

On Monday, January 9, 2017 at 5:54:15 PM UTC+5:30, Tim Chase wrote:
> On 2017-01-09 02:46, Paul Rubin wrote:
> > > gen = (expensive_calculation(x) for x in data)
> > > result = [(tmp, tmp + 1) for tmp in gen]  
> > 
> > result = [(tmp, tmp+1) for tmp in map(expensive_calculation, data)]
> 
> As charmingly expressive as map() is, the wildly different behavior in
> py3 (it's a generator that evaluates lazily) vs py2 (it consumes the
> entire iterable in one go) leads me to avoid it in general,
> especially when Python gives me list-comprehensions and generators
> that can do what I intend explicitly.  E.g., passing in
> itertools.count() as an iterable to map() will kill py2 but is
> perfectly fine in py3 (a horrible example):
> 
>   import itertools as i
>   for x in i.takewhile(
>   lambda n: n < 100, 
>   map(lambda g: g**2, i.count())
>   ):
> print(x)
> 
> But yes, both ChrisA's pass-it-to-a-function and Serhiy's nested
> generate-the-tmp-values-then-operate-on-those are good solutions
> depending on how they feel in any particular context.  If I need to
> use an "if" clause in the outer generator that tests the resulting
> temp values, I use Serhiy's solution:
> 
>   [(tmp, tmp + 1)
> for tmp in (
>   expensive_calculation(x)
>   for x in data
>   )
> if tmp > 42
> ]

What happens when the expensive is on an inner generator?
Something like:

[expensive₂(y)  for x in data for y in foo(x)]

[The ₂ representing the 2 or more occurrences in Steven's eg]
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Tim Chase]

On 2017-01-09 02:46, Paul Rubin wrote:
> > gen = (expensive_calculation(x) for x in data)
> > result = [(tmp, tmp + 1) for tmp in gen]  
> 
> result = [(tmp, tmp+1) for tmp in map(expensive_calculation, data)]

As charmingly expressive as map() is, the wildly different behavior in
py3 (it's a generator that evaluates lazily) vs py2 (it consumes the
entire iterable in one go) leads me to avoid it in general,
especially when Python gives me list-comprehensions and generators
that can do what I intend explicitly.  E.g., passing in
itertools.count() as an iterable to map() will kill py2 but is
perfectly fine in py3 (a horrible example):

  import itertools as i
  for x in i.takewhile(
  lambda n: n < 100, 
  map(lambda g: g**2, i.count())
  ):
print(x)

But yes, both ChrisA's pass-it-to-a-function and Serhiy's nested
generate-the-tmp-values-then-operate-on-those are good solutions
depending on how they feel in any particular context.  If I need to
use an "if" clause in the outer generator that tests the resulting
temp values, I use Serhiy's solution:

  [(tmp, tmp + 1)
for tmp in (
  expensive_calculation(x)
  for x in data
  )
if tmp > 42
]
 
Otherwise, I like the cleanliness of ChrisA's function:

  def fn(x)
tmp = expensive_calculation(x)
return x, x + 1
  [fn(x) for x in data]

-tkc




-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Paul Rubin]

Serhiy Storchaka  writes:
> gen = (expensive_calculation(x) for x in data)
> result = [(tmp, tmp + 1) for tmp in gen]

result = [(tmp, tmp+1) for tmp in map(expensive_calculation, data)]
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Paul Rubin]

Steven D'Aprano  writes:
> [(expensive_calculation(x), expensive_calculation(x) + 1) for x in data]

   def memoize(f):
cache = {}
def m(x):
if x in cache:
return cache[x]
a = f(x)
cache[x] = a
return a
return m

   ec = memoize(expensive_calculation)
   ...  [(ec(x), ec(x) + 1) for x in data]

Or can write:

@memoize
def expensive_calculation(x): 

Note the Haskell version of your listcomp would be:

  [(e, e+1) | x <- data_, let e = expensive_calculation x]

Maybe Python could get some version of that.  I've wanted it more than
once.  (I used "data_" because data is a Haskell keyword).
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Serhiy Storchaka]

On 09.01.17 05:53, Steven D'Aprano wrote:

Suppose you have an expensive calculation that gets used two or more times in a
loop. The obvious way to avoid calculating it twice in an ordinary loop is with
a temporary variable:

result = []
for x in data:
tmp = expensive_calculation(x)
result.append((tmp, tmp+1))


But what if you are using a list comprehension?


result = [(tmp, tmp + 1)
  for tmp in (expensive_calculation(x) for x in data)]

You could also assign an internal generator expression to temporal 
variable for readability if it is long.


gen = (expensive_calculation(x) for x in data)
result = [(tmp, tmp + 1) for tmp in gen]


--
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Rustom Mody]

On Monday, January 9, 2017 at 10:19:31 AM UTC+5:30, Steven D'Aprano wrote:
> On Monday 09 January 2017 15:09, Chris Angelico wrote:
> 
> > On Mon, Jan 9, 2017 at 2:53 PM, Steven D'Aprano wrote:
> >> [(tmp, tmp + 1) for x in data for tmp in [expensive_calculation(x)]]
> >>
> >>
> >> I can't decide whether that's an awesome trick or a horrible hack...
> > 
> > A horrible hack on par with abusing a recursive function's arguments
> > for private variables. 
> 
> What's wrong with that? That's a perfectly legitimate technique. I prefer to 
> hide it behind a private function rather than expose it in a public interface:
> 
> # instead of this
> def recursive(arg, _internal=None):
> """Recursive function. Don't supply the _internal argument."""
> ...
> return recursive(arg-1, spam)
> 
> 
> # I usually prefer this
> def recursive(arg):
> """Recursive function."""
> return _recursive(arg, spam)
> 
> def _recursive(arg, internal):
> ...
> 
> 
> but that's just polishing the code.
> 
> 
> 
> > Much better would be to refactor the append
> > part:
> > 
> > def this_and_one(value):
> > return value, value + 1
> 
> I wouldn't call that "much better". Requiring an extra function defeats the 
> purpose of using a list comp, and it doesn't scale well for multiple list 
> comps 

+1

Id call it (your original) neither an awesome trick nor a horrible hack — just 
a neat workaround for an obvious lacuna

As for Chris' solution there is a simple test to see whether it's worth it:
Does the function name make sense? If yes then the subexpression refactored-
into-a-function is probably fine. If not the kludgyness shows

[Im assuming the real usage is something else and your question is a highly
sscce-ed version]
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Chris Angelico]

On Mon, Jan 9, 2017 at 3:49 PM, Steven D'Aprano
 wrote:
> Helper functions are good. Helper functions that are only used
> *once* are a code smell. *LOTS* of helper functions that are only used once 
> are
> a sign that something is horrible, and it might just be your language...

Agreed, but with a toy example like you posted, it's impossible to say
which is happening :)

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Steven D'Aprano]

On Monday 09 January 2017 15:09, Chris Angelico wrote:

> On Mon, Jan 9, 2017 at 2:53 PM, Steven D'Aprano
>  wrote:
>> [(tmp, tmp + 1) for x in data for tmp in [expensive_calculation(x)]]
>>
>>
>> I can't decide whether that's an awesome trick or a horrible hack...
> 
> A horrible hack on par with abusing a recursive function's arguments
> for private variables. 

What's wrong with that? That's a perfectly legitimate technique. I prefer to 
hide it behind a private function rather than expose it in a public interface:

# instead of this
def recursive(arg, _internal=None):
"""Recursive function. Don't supply the _internal argument."""
...
return recursive(arg-1, spam)


# I usually prefer this
def recursive(arg):
"""Recursive function."""
return _recursive(arg, spam)

def _recursive(arg, internal):
...


but that's just polishing the code.



> Much better would be to refactor the append
> part:
> 
> def this_and_one(value):
> return value, value + 1

I wouldn't call that "much better". Requiring an extra function defeats the 
purpose of using a list comp, and it doesn't scale well for multiple list comps 
each of which needs a different helper function:


def this_and_one(value):
return value, value + 1

def this_less_one_and_this_plus_one_and_this(value):
return value - 1, value + 1, value

def this_and_that_or_something(value):
return value, value.that() or something

def extract_value(value):
return spam[value] or ham[value] or eggs[value]


and so on. Helper functions are good. Helper functions that are only used 
*once* are a code smell. *LOTS* of helper functions that are only used once are 
a sign that something is horrible, and it might just be your language...



-- 
Steven
"Ever since I learned about confirmation bias, I've been seeing 
it everywhere." - Jon Ronson

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Chris Angelico]

On Mon, Jan 9, 2017 at 2:53 PM, Steven D'Aprano
 wrote:
> [(tmp, tmp + 1) for x in data for tmp in [expensive_calculation(x)]]
>
>
> I can't decide whether that's an awesome trick or a horrible hack...

A horrible hack on par with abusing a recursive function's arguments
for private variables. Much better would be to refactor the append
part:

def this_and_one(value):
return value, value + 1

[this_and_one(expensive_calculation(x)) for x in data]

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list