Re: [PHP-DEV] RFC Draft: Comprehensions

[Larry Garfield]

On Tue, Apr 9, 2019, at 3:31 AM, Stephen Reay wrote:
> 
> 
> > On 5 Apr 2019, at 21:29, Larry Garfield  wrote:
> > 
> > On Thu, Apr 4, 2019, at 10:46 PM, Stephen Reay wrote:
> > 
> >>> Discussion:
> >>> 
> >>> For me, the inability to work with arrays is the big problem with the 
> >>> second approach.  I very very often am type declaring my returns and 
> >>> parameters as `iterable`, which means I may have an array and not know 
> >>> it. Using approach 2 means I suddenly really really need to care which 
> >>> kind of iterable it is, which defeats the purpose of `iterable`.  Calling 
> >>> methods on arrays, though, I'm pretty sure is out of scope.
> >>> 
> >>> Frankly were it not for that limitation I'd say I favor the chained 
> >>> method style, as while it is more verbose it is also more 
> >>> self-documenting.  Given that limitation, I'm torn but would probably 
> >>> lean toward option 1.   And of course there's the "methods that apply to 
> >>> all traversable objects" thing which is its own can of worms I know 
> >>> nothing about.
> >>> 
> >>> (If someone has a suggestion for how to resolve that disadvantage, I'd 
> >>> love to hear it.)
> >>> 
> >>> Those seem like the potential options.  Any further thoughts?  Or 
> >>> volunteers? :-)
> >>> 
> >>> --Larry Garfield
> >>> 
> >>> -- 
> >>> PHP Internals - PHP Runtime Development Mailing List
> >>> To unsubscribe, visit: http://www.php.net/unsub.php
> >>> 
> >> 
> >> (Sorry, sent from wrong address, sending again!)
> >> 
> >> Hi Larry,
> >> 
> >> I’ve mostly ignored this thread until now - I find a lot of the 
> >> “shorter syntax” (i.e. the short closures RFC) to sound a lot like the 
> >> arguments “I don’t like semicolons/it has to be ‘pretty'” that happen 
> >> in other language communities.
> > 
> > In defense of terse syntax, it's not a question of "pretty".  It's a 
> > question of making it feasible to operate at a higher level of abstraction. 
> >  Really, generators didn't offer much of anything that couldn't be done by 
> > defining and building an Iterator-implementing class.  They're "just" 
> > syntactic sugar.  However, they allow the developer to conceptualize a 
> > problem in a different way, and most of the machinery then falls away.  
> > That means I can now think in terms of "call this function, then iterate 
> > the stream it gives me back" and within the function I can just have normal 
> > logic with `yield` floating around as needed.  Anything I do there *could* 
> > be done with an Iterator class; I've done it some weird things with 
> > Iterators before.  But the ability to think in terms of an ad-hoc stream of 
> > values really changes the way you think about the problem, and in a very 
> > good way.
> > 
> > Similarly, short closures isn't about "let's make functions easier to 
> > write".  That's a side effect.  They should be thought of more as a way to 
> > easily encapsulate "apply this expression to this set of values".  So the 
> > advantage is not that
> > 
> > $y= 5;
> > array_map(fn($x) => $x*$y, $arr);
> > 
> > is less typing than
> > 
> > array_map(function ($x) use ($y) {
> >  return $x * $y;
> > });
> > 
> > It's that in the first option you don't think about it as a function, you 
> > think about it as an expression applied over a set.  That's a higher-order 
> > mental operation, and once you start doing that you can conceptualize the 
> > program in a different, more higher-order, less bug-prone way.
> > 
> > Just like there's nothing you can do with foreach() that you can't also do 
> > with for()... but foreach() lets you think in terms of "just do it to 
> > everything" rather than think in terms of the machinery of iteration.
> > 
> > I see comprehensions the same way.  At one level they're "just" short 
> > syntax for generators, but they're more about making it possible to reason 
> > about your logic at a higher level, in a more declarative fashion.
> > 
> > (There's probably a conference talk in there somewhere, from for to foreach 
> > to iterators to generators to comprehensions, each coming up one level of 
> > abstraction.)
> > 
> >> But the first example you give here, I can see the logical approach - 
> >> as you say, it’s a currently-valid foreach statement, wrapped in square 
> >> brackets. Would it have to be a single line to parse, or could it be 
> >> wrapped when the condition gets longer (yes I know it could just become 
> >> a regular generator then, I’m just wondering about what happens when 
> >> someone adds a new line in there (in a language that historically 
> >> doesn’t care about newlines)
> > 
> > The RFC specifically says whitespace is irrelevant.  If you want to break a 
> > comprehension across multiple lines, you do you.  But if it's getting large 
> > enough that it's ugly to read that way it's a good sign you may want to 
> > take a different approach.  (A defined function with real foreach 
> > statements, multiple defined comprehensions that reference each 

Re: [PHP-DEV] RFC Draft: Comprehensions

[Stephen Reay]

> On 5 Apr 2019, at 21:29, Larry Garfield  wrote:
> 
> On Thu, Apr 4, 2019, at 10:46 PM, Stephen Reay wrote:
> 
>>> Discussion:
>>> 
>>> For me, the inability to work with arrays is the big problem with the 
>>> second approach.  I very very often am type declaring my returns and 
>>> parameters as `iterable`, which means I may have an array and not know it. 
>>> Using approach 2 means I suddenly really really need to care which kind of 
>>> iterable it is, which defeats the purpose of `iterable`.  Calling methods 
>>> on arrays, though, I'm pretty sure is out of scope.
>>> 
>>> Frankly were it not for that limitation I'd say I favor the chained method 
>>> style, as while it is more verbose it is also more self-documenting.  Given 
>>> that limitation, I'm torn but would probably lean toward option 1.   And of 
>>> course there's the "methods that apply to all traversable objects" thing 
>>> which is its own can of worms I know nothing about.
>>> 
>>> (If someone has a suggestion for how to resolve that disadvantage, I'd love 
>>> to hear it.)
>>> 
>>> Those seem like the potential options.  Any further thoughts?  Or 
>>> volunteers? :-)
>>> 
>>> --Larry Garfield
>>> 
>>> -- 
>>> PHP Internals - PHP Runtime Development Mailing List
>>> To unsubscribe, visit: http://www.php.net/unsub.php
>>> 
>> 
>> (Sorry, sent from wrong address, sending again!)
>> 
>> Hi Larry,
>> 
>> I’ve mostly ignored this thread until now - I find a lot of the 
>> “shorter syntax” (i.e. the short closures RFC) to sound a lot like the 
>> arguments “I don’t like semicolons/it has to be ‘pretty'” that happen 
>> in other language communities.
> 
> In defense of terse syntax, it's not a question of "pretty".  It's a question 
> of making it feasible to operate at a higher level of abstraction.  Really, 
> generators didn't offer much of anything that couldn't be done by defining 
> and building an Iterator-implementing class.  They're "just" syntactic sugar. 
>  However, they allow the developer to conceptualize a problem in a different 
> way, and most of the machinery then falls away.  That means I can now think 
> in terms of "call this function, then iterate the stream it gives me back" 
> and within the function I can just have normal logic with `yield` floating 
> around as needed.  Anything I do there *could* be done with an Iterator 
> class; I've done it some weird things with Iterators before.  But the ability 
> to think in terms of an ad-hoc stream of values really changes the way you 
> think about the problem, and in a very good way.
> 
> Similarly, short closures isn't about "let's make functions easier to write". 
>  That's a side effect.  They should be thought of more as a way to easily 
> encapsulate "apply this expression to this set of values".  So the advantage 
> is not that
> 
> $y= 5;
> array_map(fn($x) => $x*$y, $arr);
> 
> is less typing than
> 
> array_map(function ($x) use ($y) {
>  return $x * $y;
> });
> 
> It's that in the first option you don't think about it as a function, you 
> think about it as an expression applied over a set.  That's a higher-order 
> mental operation, and once you start doing that you can conceptualize the 
> program in a different, more higher-order, less bug-prone way.
> 
> Just like there's nothing you can do with foreach() that you can't also do 
> with for()... but foreach() lets you think in terms of "just do it to 
> everything" rather than think in terms of the machinery of iteration.
> 
> I see comprehensions the same way.  At one level they're "just" short syntax 
> for generators, but they're more about making it possible to reason about 
> your logic at a higher level, in a more declarative fashion.
> 
> (There's probably a conference talk in there somewhere, from for to foreach 
> to iterators to generators to comprehensions, each coming up one level of 
> abstraction.)
> 
>> But the first example you give here, I can see the logical approach - 
>> as you say, it’s a currently-valid foreach statement, wrapped in square 
>> brackets. Would it have to be a single line to parse, or could it be 
>> wrapped when the condition gets longer (yes I know it could just become 
>> a regular generator then, I’m just wondering about what happens when 
>> someone adds a new line in there (in a language that historically 
>> doesn’t care about newlines)
> 
> The RFC specifically says whitespace is irrelevant.  If you want to break a 
> comprehension across multiple lines, you do you.  But if it's getting large 
> enough that it's ugly to read that way it's a good sign you may want to take 
> a different approach.  (A defined function with real foreach statements, 
> multiple defined comprehensions that reference each other, etc.)
> 
>> I like the second concept a lot too, but how would this cope with for 
>> example: a userland class implements iterator but *also* defines a 
>> `filter(callback $fn): self` method for the exact same purposes were 
>> discussing. How is that 

Re: [PHP-DEV] RFC Draft: Comprehensions

[Larry Garfield]

On Thu, Apr 4, 2019, at 10:46 PM, Stephen Reay wrote:

> > Discussion:
> > 
> > For me, the inability to work with arrays is the big problem with the 
> > second approach.  I very very often am type declaring my returns and 
> > parameters as `iterable`, which means I may have an array and not know it. 
> > Using approach 2 means I suddenly really really need to care which kind of 
> > iterable it is, which defeats the purpose of `iterable`.  Calling methods 
> > on arrays, though, I'm pretty sure is out of scope.
> > 
> > Frankly were it not for that limitation I'd say I favor the chained method 
> > style, as while it is more verbose it is also more self-documenting.  Given 
> > that limitation, I'm torn but would probably lean toward option 1.   And of 
> > course there's the "methods that apply to all traversable objects" thing 
> > which is its own can of worms I know nothing about.
> > 
> > (If someone has a suggestion for how to resolve that disadvantage, I'd love 
> > to hear it.)
> > 
> > Those seem like the potential options.  Any further thoughts?  Or 
> > volunteers? :-)
> > 
> > --Larry Garfield
> > 
> > -- 
> > PHP Internals - PHP Runtime Development Mailing List
> > To unsubscribe, visit: http://www.php.net/unsub.php
> > 
> 
> (Sorry, sent from wrong address, sending again!)
> 
> Hi Larry,
> 
> I’ve mostly ignored this thread until now - I find a lot of the 
> “shorter syntax” (i.e. the short closures RFC) to sound a lot like the 
> arguments “I don’t like semicolons/it has to be ‘pretty'” that happen 
> in other language communities.

In defense of terse syntax, it's not a question of "pretty".  It's a question 
of making it feasible to operate at a higher level of abstraction.  Really, 
generators didn't offer much of anything that couldn't be done by defining and 
building an Iterator-implementing class.  They're "just" syntactic sugar.  
However, they allow the developer to conceptualize a problem in a different 
way, and most of the machinery then falls away.  That means I can now think in 
terms of "call this function, then iterate the stream it gives me back" and 
within the function I can just have normal logic with `yield` floating around 
as needed.  Anything I do there *could* be done with an Iterator class; I've 
done it some weird things with Iterators before.  But the ability to think in 
terms of an ad-hoc stream of values really changes the way you think about the 
problem, and in a very good way.

Similarly, short closures isn't about "let's make functions easier to write".  
That's a side effect.  They should be thought of more as a way to easily 
encapsulate "apply this expression to this set of values".  So the advantage is 
not that

$y= 5;
array_map(fn($x) => $x*$y, $arr);

is less typing than

array_map(function ($x) use ($y) {
  return $x * $y;
});

It's that in the first option you don't think about it as a function, you think 
about it as an expression applied over a set.  That's a higher-order mental 
operation, and once you start doing that you can conceptualize the program in a 
different, more higher-order, less bug-prone way.

Just like there's nothing you can do with foreach() that you can't also do with 
for()... but foreach() lets you think in terms of "just do it to everything" 
rather than think in terms of the machinery of iteration.

I see comprehensions the same way.  At one level they're "just" short syntax 
for generators, but they're more about making it possible to reason about your 
logic at a higher level, in a more declarative fashion.

(There's probably a conference talk in there somewhere, from for to foreach to 
iterators to generators to comprehensions, each coming up one level of 
abstraction.)

> But the first example you give here, I can see the logical approach - 
> as you say, it’s a currently-valid foreach statement, wrapped in square 
> brackets. Would it have to be a single line to parse, or could it be 
> wrapped when the condition gets longer (yes I know it could just become 
> a regular generator then, I’m just wondering about what happens when 
> someone adds a new line in there (in a language that historically 
> doesn’t care about newlines)

The RFC specifically says whitespace is irrelevant.  If you want to break a 
comprehension across multiple lines, you do you.  But if it's getting large 
enough that it's ugly to read that way it's a good sign you may want to take a 
different approach.  (A defined function with real foreach statements, multiple 
defined comprehensions that reference each other, etc.)

> I like the second concept a lot too, but how would this cope with for 
> example: a userland class implements iterator but *also* defines a 
> `filter(callback $fn): self` method for the exact same purposes were 
> discussing. How is that handled?

I have no idea at the moment. :-)  That would be a possible BC issue.  My first 
thought is that if an iterator defines filter(), map(), etc. itself then it's 
overriding the 

Re: [PHP-DEV] RFC Draft: Comprehensions

[Michał Brzuchalski]

Hi Larry,

pt., 5 kwi 2019 o 03:55 Larry Garfield  napisał(a):

>
> Advantages:
>
> * Very compact.
> * Works for both arrays and traversables
> * Would play very nicely with the proposed spread operator for iterables (
> https://wiki.php.net/rfc/spread_operator_for_array).
>

IMO not nicely cause spread operator in current proposal raises an error on
key preserved traversable[1].
This means example like below would fail

$a = ['foo' => true, ...[foreach($_GET as $key => $value) yield $key =>
$value]]; // error

[1] https://wiki.php.net/rfc/spread_operator_for_array#string_keys
-- 
regards / pozdrawiam,
--
Michał Brzuchalski
about.me/brzuchal
brzuchalski.com

Re: [PHP-DEV] RFC Draft: Comprehensions

[Stephen Reay]

> On 5 Apr 2019, at 08:54, Larry Garfield  wrote:
> 
> On Wed, Mar 13, 2019, at 10:22 PM, Larry Garfield wrote:
>> On Wed, Mar 13, 2019, at 6:30 PM, Rowan Collins wrote:
>>> On 13/03/2019 21:10, Dik Takken wrote:
> 
>> If I can summarize the responses so far, they seem to fall into one of 
>> two categories:
>> 
>> 1) Love the idea, but wouldn't short-closures be close enough?
>> 
>> 2) Love the idea, but hate the particular syntax proposed.
>> 
>> On the plus side, it seems almost everyone is on board in concept, so 
>> yay.  That of course just leaves the syntax bikeshedding, which is 
>> always the fun part.
> 
> Bumping this thread again.
> 
> Thinking on it further, I see two possible syntactic approaches, given that 
> short lambdas as currently written would not give us a viable comprehension 
> syntax.
> 
> 1) [foreach ($list as $x => $y) if (condition) yield expression]
> 
> That is, essentially the same syntax as the list would be if wrapped in a 
> function, but with a more compact way of writing it.  The above would be 
> effectively identical to:
> 
> $gen = function () {
> foreach ($list as $x => $y)
>   if ($condition)
> yield expression;
> }();
> 
> 
> (But with auto-capture.)  I am personally not at all a fan of the extra 
> verbosity (foreach, parens, etc.) but it seems most respondents in the thread 
> want it for familiarity.
> 
> Advantages:
> 
> * Very compact.
> * Works for both arrays and traversables
> * Would play very nicely with the proposed spread operator for iterables 
> (https://wiki.php.net/rfc/spread_operator_for_array).
> 
> Disadvantages:
> 
> * New syntax
> * If you need to do multiple filter or map operations it gets potentially 
> ugly and unwieldy.
> * Not super extensible.
> * Doesn't have a natural way to enforce the types produced.  (Although one 
> could add it easily.)
> 
> This approach has the advantage of being compact and working for both arrays 
> and traversables, but is new syntax.
> 
> 2) Allow comprehensions to work only on traversable objects, which lets us 
> chain methods.  Specifically:
> 
> $new = $anyTraversable->filter(fn($x) => $x < 0);
> 
> Would return a new traversable that filters $anyTraversable, using a 
> callable.  It would effectively be identical to 
> 
> $new = new CallbackFilterIterator($anyTraversable, fn($x) => $x < 0);
> 
> Similarly:
> 
> $new = $anyTraversable->map(fn($x) => $x * 2);
> 
> Would produce a new traversable that lazily produces a function over the 
> items as they're returned.   Equivalent to:
> 
> $new = function () {
> foreach ($list as $x)
> yield expression;
> }();
> 
> And both would also need to support a key/value as well, probably if the 
> callable takes 2 parameters then it's $key, $value, if just one parameter 
> then it's just $value.
> 
> This approach has a few advantages:
> 
> * It piggy-backs on existing traversable behavior; essentially, rather than 
> short-syntax for generators it's short syntax for wrapping a bunch of 
> iterator objects around each other.
> * More elaborate cases (multiple filters, multiple maps) become somewhat 
> nicer; you can easily call filter() or map() multiple times and it's still 
> entirely obvious what's going on.
> * Has a natural (if verbose) way to enforce types: filter(fn($x) => $x 
> instanceof Foo || throw new \TypeError);
> * Actually, since short-lambdas already would support return type 
> declaration, there's another alternative: filter(fn($x) : Foo => $x);  
> (Although you'd probably just fit that into a filter function you're using 
> for something else.)
> * next() is already a useful method that works for the an() case discussed in 
> the RFC, and it flows very naturally.  I don't see a nice equivalent of 
> all(), however.
> 
> But also some disadvantages:
> 
> * It only works for traversable objects, not arrays.  (Workaround: new 
> ArrayObject($arr).)
> * It is more verbose than the other syntax option.
> * Adding special-meaning methods to Traversable objects is weird, and I don't 
> think we've done that anywhere before.  I have no idea if there are engine 
> implications.
> * The short lambda RFC becomes effectively a prerequisite, as it's way too 
> verbose to do with an anon function as we have now.
> * My gut feeling is it would be slower as it would likely mean more function 
> calls internally, but I've zero data to back that up.
> 
> And before someone else mentions it, it also poses some interesting possible 
> extensions that are not all that relevant to the current target, but would 
> fit naturally:
> 
> * a ->limit(0, 3) method, that is functionally equivalent to \LimitIterator.
> * Potentially RegexIterator() could also become a regex() method, that's a 
> special case of filter()?
> * Languages like Rust have a method to "run out" the comprehension ( 
> .collect() in the case of Rust).  We could easily do the same to produce a 
> resultant array, similar to the spread operator.  (That said, that should in 
> no way detract 

Re: [PHP-DEV] RFC Draft: Comprehensions

[Larry Garfield]

On Wed, Mar 13, 2019, at 10:22 PM, Larry Garfield wrote:
> On Wed, Mar 13, 2019, at 6:30 PM, Rowan Collins wrote:
> > On 13/03/2019 21:10, Dik Takken wrote:

> If I can summarize the responses so far, they seem to fall into one of 
> two categories:
> 
> 1) Love the idea, but wouldn't short-closures be close enough?
> 
> 2) Love the idea, but hate the particular syntax proposed.
> 
> On the plus side, it seems almost everyone is on board in concept, so 
> yay.  That of course just leaves the syntax bikeshedding, which is 
> always the fun part.

Bumping this thread again.

Thinking on it further, I see two possible syntactic approaches, given that 
short lambdas as currently written would not give us a viable comprehension 
syntax.

1) [foreach ($list as $x => $y) if (condition) yield expression]

That is, essentially the same syntax as the list would be if wrapped in a 
function, but with a more compact way of writing it.  The above would be 
effectively identical to:

$gen = function () {
  foreach ($list as $x => $y)
if ($condition)
  yield expression;
}();


(But with auto-capture.)  I am personally not at all a fan of the extra 
verbosity (foreach, parens, etc.) but it seems most respondents in the thread 
want it for familiarity.

Advantages:

* Very compact.
* Works for both arrays and traversables
* Would play very nicely with the proposed spread operator for iterables 
(https://wiki.php.net/rfc/spread_operator_for_array).

Disadvantages:

* New syntax
* If you need to do multiple filter or map operations it gets potentially ugly 
and unwieldy.
* Not super extensible.
* Doesn't have a natural way to enforce the types produced.  (Although one 
could add it easily.)

This approach has the advantage of being compact and working for both arrays 
and traversables, but is new syntax.

2) Allow comprehensions to work only on traversable objects, which lets us 
chain methods.  Specifically:

$new = $anyTraversable->filter(fn($x) => $x < 0);

Would return a new traversable that filters $anyTraversable, using a callable.  
It would effectively be identical to 

$new = new CallbackFilterIterator($anyTraversable, fn($x) => $x < 0);

Similarly:

$new = $anyTraversable->map(fn($x) => $x * 2);

Would produce a new traversable that lazily produces a function over the items 
as they're returned.   Equivalent to:

$new = function () {
  foreach ($list as $x)
  yield expression;
}();

And both would also need to support a key/value as well, probably if the 
callable takes 2 parameters then it's $key, $value, if just one parameter then 
it's just $value.

This approach has a few advantages:

* It piggy-backs on existing traversable behavior; essentially, rather than 
short-syntax for generators it's short syntax for wrapping a bunch of iterator 
objects around each other.
* More elaborate cases (multiple filters, multiple maps) become somewhat nicer; 
you can easily call filter() or map() multiple times and it's still entirely 
obvious what's going on.
* Has a natural (if verbose) way to enforce types: filter(fn($x) => $x 
instanceof Foo || throw new \TypeError);
* Actually, since short-lambdas already would support return type declaration, 
there's another alternative: filter(fn($x) : Foo => $x);  (Although you'd 
probably just fit that into a filter function you're using for something else.)
* next() is already a useful method that works for the an() case discussed in 
the RFC, and it flows very naturally.  I don't see a nice equivalent of all(), 
however.

But also some disadvantages:

* It only works for traversable objects, not arrays.  (Workaround: new 
ArrayObject($arr).)
* It is more verbose than the other syntax option.
* Adding special-meaning methods to Traversable objects is weird, and I don't 
think we've done that anywhere before.  I have no idea if there are engine 
implications.
* The short lambda RFC becomes effectively a prerequisite, as it's way too 
verbose to do with an anon function as we have now.
* My gut feeling is it would be slower as it would likely mean more function 
calls internally, but I've zero data to back that up.

And before someone else mentions it, it also poses some interesting possible 
extensions that are not all that relevant to the current target, but would fit 
naturally:

* a ->limit(0, 3) method, that is functionally equivalent to \LimitIterator.
* Potentially RegexIterator() could also become a regex() method, that's a 
special case of filter()?
* Languages like Rust have a method to "run out" the comprehension ( .collect() 
in the case of Rust).  We could easily do the same to produce a resultant 
array, similar to the spread operator.  (That said, that should in no way 
detract from the spread operator proposal, which I also like on its own merits.)
* Possibly other stuff that slowly turns iterables into "collection objects" 
(sort of).


Discussion:

For me, the inability to work with arrays is the big problem with the second 
approach.  I very very often 

Re: [PHP-DEV] RFC Draft: Comprehensions

[Dik Takken]

On 21-03-19 16:22, Larry Garfield wrote:
> OTOH, if we just have the one syntax:
> 
> [foreach $foo as $bar yield $bar*2] // gives a generator
> 
> And include a nicer "fast-forward" operator than interator_to_array(), then 
> we automatically get:
> 
> ...[foreach $foo as $bar yield $bar*2] // turns the generator into an array 
> on the fly
> 
> Which is both easier to learn (two independently useful primitives, 
> composed), easier to understand (because the latter is not as visually 
> similar), and doesn't lay claim to two different potentially useful pieces of 
> block syntax, leaving the other open for later use by some other idea later.

That sums it up nicely, thanks. I agree that we should try to focus on
the actual comprehension syntax and leave the problem of how to better
support both generators and arrays to a future RFC.

Still, I would like to mention one detail that is relevant _now_, for
the discussion about the current RFC. Indeed, whether comprehensions
will be used primarily for creating generators or arrays depends on the
developer, libraries that are used, and so on. In my opinion, this calls
for a syntax that is not biased towards either of the two. In the cases
where comprehensions are mainly used to create arrays, the need to
convert a generator into an array still feels like an extra hoop to jump
through in order to get what you need. Even when all it takes is three dots.

While this issue should be discussed separately, I mention it here to
support my proposal to keep the option open to extend the syntax into a
dual array / generator syntax in the future. In case we go for a syntax like

$generator = (foreach $foo as $bar yield $bar*2)

then we could still decide to introduce a variant like

$array = [foreach $foo as $bar yield $bar*2]

later, as proposed by Nikita. Going for a square bracketed syntax for
generators right now will not allow this future extension.

Until then, using the spread operator to materialize generators into
arrays would be really nice.

> Now can we go back to bikeshedding the actual syntax style to use, per my 
> earlier email? :-)

Sounds good. :)

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] RFC Draft: Comprehensions

[Robert Hickman]

On Thu, 21 Mar 2019 at 16:15, Rowan Collins  wrote:
>
> On Thu, 21 Mar 2019 at 15:21, Robert Hickman  wrote:
>>
>> In this case nextIf() would have to be implemented something like:
>>
>> function nextif($someCondition) {
>> foreach($this->iteratorValue as $x) {
>> if(>  yield $x;
>> }
>> }
>> }
>>
>> iterator_to_array would need an internal loop to pull the values from
>> the generator.
>
>
>
> I think it would be more like this:
>
> function nextif($someCondition) {
> do {
> $this->currentIndex++;
> $x = $this->items[ $this->currentIndex ];
> } while ( ! some comparison of $x and $$someCondition);
> return $x;
> }
>

Thanks, that makes sense.

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] RFC Draft: Comprehensions

[Rowan Collins]

On Thu, 21 Mar 2019 at 15:21, Robert Hickman  wrote:

> In this case nextIf() would have to be implemented something like:
>
> function nextif($someCondition) {
> foreach($this->iteratorValue as $x) {
> if(  yield $x;
> }
> }
> }
>
> iterator_to_array would need an internal loop to pull the values from
> the generator.



I think it would be more like this:

function nextif($someCondition) {
do {
$this->currentIndex++;
$x = $this->items[ $this->currentIndex ];
} while ( ! some comparison of $x and $$someCondition);
return $x;
}

nextIf() wouldn't be a generator, it would be a normal function returning
one value.

There is a nested loop in that imaginary version, but only because you
can't indicate "don't return anything"; given we're in imaginary concept
land, we could add that, and all sign of inner loops disappears:

function nextif($someCondition) {
   $this->currentIndex++;
   $x = $this->items[ $this->currentIndex ];
   if ( some comparison of $x and $$someCondition) {
   return $x;
   } else {
   return SKIP_THIS_ITEM;
   }
}





> Thus, as far as I can see, the generator implementation
> would result in two loops running in lock step, rather than only one
> in the eager case.
>


It's more like a pipeline: there's a single loop pumping data in, and at
any time there's a single "current item", which is either being checked
against the condition, manipulated in some way, or passed out the other
end. Each stage in the pipeline doesn't need to track what "current item"
means, it's given an input each time the loop cycles.

We write that as "foreach ( $previousStage as $x ) { yield $x; }" because
it's a familiar syntax, but really we're just defining the logic for a
nextStep($x) function.


Regards,
-- 
Rowan Collins
[IMSoP]

Re: [PHP-DEV] RFC Draft: Comprehensions

[Larry Garfield]

On Wed, Mar 20, 2019, at 5:44 PM, Rowan Collins wrote:
> On 20/03/2019 20:39, Stanislav Malyshev wrote:
> > Hi!
> >
> >> It's not that you can't make an array into a generator, but you can't make
> >> an eagerly-evaluated expression into a lazily-evaluated one.
> > Not sure what you mean here.
> 
> 
> I mean that, given a syntax that lazily-evaluates something, you can 
> "fast-forward" the result to make it eagerly-evaluated; given a syntax 
> that eagerly-evaluates something, you cannot do the opposite. Therefore, 
> a lazy-evaluating syntax is more powerful, in that it can do everything 
> an eager-evaluating one can do *and more*.

This is really the crux of the array-vs-generator question for comprehensions.

No one disputes that both arrays and generators have their good use cases.  No 
one disputes that there are cases where one is favorable over another.  I argue 
that in *most* cases where you'd be using a comprehension it wouldn't actually 
matter in practice which one it is.  Which one is more prevalent in practice 
depends who the developer is; I personally use generators a ton because I find 
them easier to work with syntactically when processing complex data structures 
regardless of their size, but others have their own styles.

It comes down to conceptual simplicity.  A greedy-evaluated list can be 
trivially composed from a lazy-evaluated list and a "fast-forward" operator.  
Both of those are useful in their own right, and can be composed to produce a 
greedy evaluated list.  However, given a greedy-evaluated list there is no way 
to produce a lazy-evaluated list at all.  Thus we will need a lazy-evaluated 
list either way (and that's my goal with comprehensions).  

Given a lazy-evaluated list and a fast-forward operator... an alternate 
greedy-evaluated list is a nice-to-have, unless it creates more syntactic 
complexity and/or confusion.  As an unrealistic Pythonic example, were we to do:

{foreach $foo as $bar yield $bar*2} // gives a generator

[foreach $foo as $bar yield $bar*2] // gives an array

Then we're introducing confusion for users who have to learn two different 
very-similar-looking syntaxes, and we're laying claim to both []-wrapped 
expressions and {}-wrapped expressions to mean something.

OTOH, if we just have the one syntax:

[foreach $foo as $bar yield $bar*2] // gives a generator

And include a nicer "fast-forward" operator than interator_to_array(), then we 
automatically get:

...[foreach $foo as $bar yield $bar*2] // turns the generator into an array on 
the fly

Which is both easier to learn (two independently useful primitives, composed), 
easier to understand (because the latter is not as visually similar), and 
doesn't lay claim to two different potentially useful pieces of block syntax, 
leaving the other open for later use by some other idea later.

(And if we don't introduce a new fast-forward operation, then 
iterator_to_array() is already still there and while a bit more verbose than 
we'd like it certainly doesn't the job just as effectively.)

Also, I reiterate that in the majority of use cases I see where a comprehension 
would be used (not all, but most) the result would end up in a foreach() loop, 
or returned as an iterable and thus whether it's an array or generator won't 
actually matter.

So really, "I don't use generators much" is not a meaningful argument.  A 
lower-overhead implementation (both for the user and the parser) is the 
argument, and on that front "generator only" wins, with "both" a weak 
compromise position that doesn't add much.  "Array only" is by far the weakest 
result.

Now can we go back to bikeshedding the actual syntax style to use, per my 
earlier email? :-)

--Larry Garfield

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] RFC Draft: Comprehensions

[Robert Hickman]

> But  $filteredArrayIterator->next() is actually $arrayIterator->next() with
> a built-in if check, so you could also picture it as doing this:
>
> while ( $element = $arrayIterator->nextIf ( someCondition ) ) {
> $newArray[] = $element;
> }
>

In this case nextIf() would have to be implemented something like:

function nextif($someCondition) {
foreach($this->iteratorValue as $x) {
if(http://www.php.net/unsub.php

Re: [PHP-DEV] RFC Draft: Comprehensions

[Rowan Collins]

On Thu, 21 Mar 2019 at 11:00, Robert Hickman  wrote:

> I was only making a point for where a non-generator version of
> comprehensions could be useful, under the premise "Therefore, a
> lazy-evaluating syntax is more powerful, in that it can do everything
> an eager-evaluating one can do *and more*.". This implies that it
> dosn't have any downsides, whereas performance may be a downside.
>


Ah, OK, I see where you're coming from. I'm no expert, but I'm not sure how
much difference it would actually make.

Whether the comprehension is eagerly evaluated internally, or by a call to
iterator_to_array, the input will always need to looped over exactly once,
with each element being tested against the condition and run through the
output mapping.

A manual eager implementation (with no output mapping, for simplicity)
would look like this:

foreach ( $array as $element ) {
if ( someCondition ) $newArray[] = $element;
}

Which is basically equivalent to this:

while ( $element = $arrayIterator->next() ) {
if ( someCondition ) $newArray[] = $element;
}

A lazy implementation just moves the if check into the iterator; it might
look like this:

while ( $element = $filteredArrayIterator->next() ) {
if ( someCondition ) $newArray[] = $element;
}

But  $filteredArrayIterator->next() is actually $arrayIterator->next() with
a built-in if check, so you could also picture it as doing this:

while ( $element = $arrayIterator->nextIf ( someCondition ) ) {
$newArray[] = $element;
}

So the underlying operations are the same, they just happen in different
orders.


The only real difference I can see is that under a very specific set of
circumstances, it could be compiled into a tight loop on the CPU. It would
require a JIT to check all of the following held, though:

- The input is a plain array, not any other iterable
- The condition can be inlined (e.g. not a function call like "if (
checkSomething($x) )")
- The output mapping can be inlined (e.g. not a function call like "yield
doSomething($x)")

Returning a generator by default just requires an extra check in that list:

- The expression is contained within iterator_to_array (or whatever syntax
shortcut we come up with for that).


I might be wrong, though, in which case this would indeed be an argument in
favour of having a dedicated array-returning syntax, either alongside or
instead of a generator-returning one.


Regards,
-- 
Rowan Collins
[IMSoP]

Re: [PHP-DEV] RFC Draft: Comprehensions

[Robert Hickman]

I was only making a point for where a non-generator version of
comprehensions could be useful, under the premise "Therefore, a
lazy-evaluating syntax is more powerful, in that it can do everything
an eager-evaluating one can do *and more*.". This implies that it
dosn't have any downsides, whereas performance may be a downside.

On Thu, 21 Mar 2019 at 07:54, Rowan Collins  wrote:
>
> On 21 March 2019 00:39:20 GMT+00:00, Robert Hickman  
> wrote:
> >For my use case of PHP, get some content from a DB and dump it into a
> >template, I don't see much benefit to generators.
>
> With respect, so what? I never said that every use case benefits from 
> generators, nor are we discussing whether generators should be added to the 
> language.
>
> Regards,
>
> --
> Rowan Collins
> [IMSoP]
>
> --
> PHP Internals - PHP Runtime Development Mailing List
> To unsubscribe, visit: http://www.php.net/unsub.php
>

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] RFC Draft: Comprehensions

[Rowan Collins]

On 21 March 2019 00:39:20 GMT+00:00, Robert Hickman  
wrote:
>For my use case of PHP, get some content from a DB and dump it into a
>template, I don't see much benefit to generators. 

With respect, so what? I never said that every use case benefits from 
generators, nor are we discussing whether generators should be added to the 
language.

Regards,

-- 
Rowan Collins
[IMSoP]

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] RFC Draft: Comprehensions

[Robert Hickman]

> > Hi!
> >
> >> It's not that you can't make an array into a generator, but you can't make
> >> an eagerly-evaluated expression into a lazily-evaluated one.
> > Not sure what you mean here.
>
>
> I mean that, given a syntax that lazily-evaluates something, you can
> "fast-forward" the result to make it eagerly-evaluated; given a syntax
> that eagerly-evaluates something, you cannot do the opposite. Therefore,
> a lazy-evaluating syntax is more powerful, in that it can do everything
> an eager-evaluating one can do *and more*.
>

For my use case of PHP, get some content from a DB and dump it into a
template, I don't see much benefit to generators. Such content is
textual and is small relative to available ram. Memory locality also
matters for performance and it is generally faster to do a whole bunch
of stuff at once, thus I'd expect an eager version to be faster than a
generator. I'm almost sure it would be faster if stepping the
generator entailed IPC, reaching out to a database for instance.

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] RFC Draft: Comprehensions

[Rowan Collins]

On 20/03/2019 20:39, Stanislav Malyshev wrote:

Hi!


It's not that you can't make an array into a generator, but you can't make
an eagerly-evaluated expression into a lazily-evaluated one.

Not sure what you mean here.



I mean that, given a syntax that lazily-evaluates something, you can 
"fast-forward" the result to make it eagerly-evaluated; given a syntax 
that eagerly-evaluates something, you cannot do the opposite. Therefore, 
a lazy-evaluating syntax is more powerful, in that it can do everything 
an eager-evaluating one can do *and more*.




In this particular case, when you're working with an iterator over a
large file, you probably want a generator. Which is very easy to write
using a functional syntax, and the only thing comprehension syntax does
is switching the parts around a bit and saving you writing a "function"
keyword. I don't think it's a very common case though.



Isn't that true of *every* use of the syntax, though, that "all it does 
is switch the parts around a bit"? There's nothing that an array 
comprehension can do that can't be done any other way, it's just 
short-hand; making it return a generator just extends that short-hand to 
more use cases.




Of course, when you need to filter iterators, then you need a different
approach. I do not doubt that. I doubt that filtering iterators is the
most common case in PHP to make the comprehension syntax only support it
- I think the case of array transformation is much more common.



I am not saying that it is "the most common case", and I am not saying 
that it should be the only thing supported. I am saying that if we don't 
have dedicated syntax for each, one way to support *both* variants is to 
make the iterator version the base case, and then have a universal 
syntax that switches it to an array.


If we do have syntax for both variants, then clearly both cases are 
covered anyway - although I think a short-hand for iterator_to_array 
would still have merit elsewhere.




But if we only have one, it should be the iterator version, with a
short-hand for iterator_to_array as a separate language improvement.

I think this would significantly reduce the usability of such construct
- to the point that it's easier to use the existing syntax, thus making
new syntax sugar useless. The whole point of syntax sugar is making
common case easy, and doing what you are suggesting is to make a corner
case (existing, but IMHO much less common) easy and make the user work
to achieve the common case.



I think you're arguing against a straw man here. Yes, the cost of the 
extra syntax should be weighed against the benefit of the extra 
flexibility. And IF the extra syntax was so verbose that it would be 
easier to use the existing syntax, that cost would probably be too high. 
But that's not what I was suggesting at all; I was picturing a new 
operator, a few characters long, which could turn any iterator into an 
array.


Are you really saying that something like this:

$a = [ ... (foreach $x as $y if $y < 2 yield $y) ];

would be so verbose that you'd rather write this:

$a = (function() use ($x) { $a=[]; foreach ( $x as $y ) if ( $y < 2 ) 
$a[] = $y; return $a; })();


or with the shortest closure syntax I can think of, this:

$a = (() => { $a=[]; foreach ( $x as $y ) if ( $y < 2 ) $a[] = $y; 
return $a; })();



By all means let's try to weigh the cost and benefit, but let's not jump 
to the conclusion that the cost is nearly infinite and the benefit is 
nearly zero.



Regards,

--
Rowan Collins
[IMSoP]


--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] RFC Draft: Comprehensions

[Stanislav Malyshev]

Hi!

> It's not that you can't make an array into a generator, but you can't make
> an eagerly-evaluated expression into a lazily-evaluated one.

Not sure what you mean here. If you already have the data, of course you
can't un-evaluate it in order to lazily-evaluate it again. But why would
you, that would be just a waste of time! If you don't yet have the data,
then you usually can easily rewrite the code that produces it as a
generator if you wanted, but in most cases you don't need it as it would
be more complex and won't give you any advantage (exceptions of course
exist like processing enormous files or processing infinite streams
continuously). So I am not seeing the point of mentioning "eager to
lazy" case as it is usually does not happen in practice.

> $hugeCSVFileIterator = new  CSVFileLazyLoader($filename);
> $filteredCSVFileIterator = [ foreach $hugeCSVFileIterator as $line if
> $line['type'] == 'foo' yield $line['value'] ];

In this particular case, when you're working with an iterator over a
large file, you probably want a generator. Which is very easy to write
using a functional syntax, and the only thing comprehension syntax does
is switching the parts around a bit and saving you writing a "function"
keyword. I don't think it's a very common case though.

> Similarly, an array-only syntax is useless for infinite iterators, whereas
> an iterator version lets you write a filtered version that's still infinite.

Of course, when you need to filter iterators, then you need a different
approach. I do not doubt that. I doubt that filtering iterators is the
most common case in PHP to make the comprehension syntax only support it
- I think the case of array transformation is much more common.

> But if we only have one, it should be the iterator version, with a
> short-hand for iterator_to_array as a separate language improvement.

I think this would significantly reduce the usability of such construct
- to the point that it's easier to use the existing syntax, thus making
new syntax sugar useless. The whole point of syntax sugar is making
common case easy, and doing what you are suggesting is to make a corner
case (existing, but IMHO much less common) easy and make the user work
to achieve the common case.
-- 
Stas Malyshev
smalys...@gmail.com

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] RFC Draft: Comprehensions

[Larry Garfield]

On Tue, Mar 19, 2019, at 3:25 PM, Levi Morrison wrote:
> Today in the Dart world, Bob Nystrom published an article called
> [Making Dart a Better Language for UI][1]. I think it's an incredibly
> relevant article, since it is essentially about comprehensions, why
> they are a thing, as well as some of the design choices they made.
> 
> I think everyone in this thread ought to spend the 10-15 minutes reading it.
> 
>   [1]: 
> https://medium.com/dartlang/making-dart-a-better-language-for-ui-f1ccaf9f546c

I love that explanation, and the solution they come up with sounds very 
elegant.  There's only two caveats to it:

* Something along those lines in PHP would be even more work, and again I'm 
still looking for someone who can implement the design here; without a 
volunteer for that, this discussion is all hypothetical. :-(

* It doesn't seem like it would address the generator/iterator side, which is 
what my main target is.  Splatting an array into another array sounds highly 
useful, but the style there wouldn't (as far as I can tell) offer any solution 
for the lazy-evaluation case which, as Rowan just pointed out later in the 
thread, is still highly important and impossible to get from a 
greedy-evaluation construct.

--Larry Garfield

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] RFC Draft: Comprehensions

[Rowan Collins]

On Wed, 20 Mar 2019 at 00:35, Stanislav Malyshev 
wrote:

> > And converting from a generator to an array is trivial; the other
> > way, very not so much.
>
> BTW, how it's not trivial to make generator from array?
>
> function generate($array) { foreach($array as $k => $v) yield $k => $v; }
>
> Is there something I'm missing here?
>


It's not that you can't make an array into a generator, but you can't make
an eagerly-evaluated expression into a lazily-evaluated one.

So if you only have array comprehensions, you can use that helper function
to write this, but it will allocate memory for an array containing all the
matches:

$hugeCSVFileIterator = new CSVFileLazyLoader($filename);
$filteredCSVFileIterator = generate([ foreach $hugeCSVFileIterator as $line
if $line['type'] == 'foo' yield $line['value'] ]);

whereas if the comprehension is an iterator, you can do this and preserve
the lazy-loading:

$hugeCSVFileIterator = new  CSVFileLazyLoader($filename);
$filteredCSVFileIterator = [ foreach $hugeCSVFileIterator as $line if
$line['type'] == 'foo' yield $line['value'] ];

Similarly, an array-only syntax is useless for infinite iterators, whereas
an iterator version lets you write a filtered version that's still infinite.

I think a syntax that allows for both versions, using something more
idiomatically PHPish than different types of brackets, might be sensible.
But if we only have one, it should be the iterator version, with a
short-hand for iterator_to_array as a separate language improvement.

Regards,
-- 
Rowan Collins
[IMSoP]

Re: [PHP-DEV] RFC Draft: Comprehensions

[Robert Hickman]

>> Honestly, I cannot think of any case where I'd use a comprehension
>> where I would definitely want an array and not a generator.  In the
>> majority case both work equally well, cool, but I don't know when I
>> would even use an array-dependent version.
>> And converting from a generator to an array is trivial; the other
>> way, very not so much.
>  I don't think I ever used
> the generator one in python (I use list one all the time), and I maybe
> used analogous construct in PHP once or twice (and again I use array_map
> and more complex syntax for array transformations all the time).
>
I use generator expressions rarely in python, usually as a conditional
'if any data in x match condition':

if not next((True for flter in config['pull_ignore_filters'] if
fnmatch.fnmatch(fle['path'], flter)), False):
filtered_pull_files.append(fle)

I use the list form for filtering, and the dict form mostly for
creating an index of a sub-value as noted before. I don't use
generators at all in PHP right now, but do make extensive use of loops
doing filters and building indexes. An 'array comprehension' syntax
would be welcome from me.

One needs to be contentions that people use tools differently, so 'I
don't use something' isn't the same as 'nobody uses something'.

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] RFC Draft: Comprehensions

[Stanislav Malyshev]

Hi!

> Honestly, I cannot think of any case where I'd use a comprehension
> where I would definitely want an array and not a generator.  In the
> majority case both work equally well, cool, but I don't know when I
> would even use an array-dependent version.

They wouldn't work equally well. Generator is only
sequentially-addressable, and array is random-addressable. And in PHP
people do a lot of random addressing of arrays. Also, many functions
which accept arrays (and array type) won't work with generators.

> And converting from a generator to an array is trivial; the other
> way, very not so much.

Well, doing everything not using this syntax is trivial - if the point
is not doing anything that is already trivial, then we don't need this
syntax, as iterating over an array/sequence and applying an operation to
it is already trivial in both array and generator contexts, it's a
two-liner function in most cases. If the point is to add syntax sugar
for most frequently used cases, then I claim that the most frequently
used case is definitely not the generator one. I don't think I ever used
the generator one in python (I use list one all the time), and I maybe
used analogous construct in PHP once or twice (and again I use array_map
and more complex syntax for array transformations all the time).

BTW, how it's not trivial to make generator from array?

function generate($array) { foreach($array as $k => $v) yield $k => $v; }

Is there something I'm missing here?
-- 
Stas Malyshev
smalys...@gmail.com

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] RFC Draft: Comprehensions

[Robert Hickman]

> > In python comprehensions on [] makes a list and comprehensions in {}
> > make a dictionary (list and dict comprehensions). As PHP only has one
> > 'array' type using [] makes sense. Along that train of thought, should
> > comprehensions also be possible in the old array() syntax?
>
> Honestly, I cannot think of any case where I'd use a comprehension where I 
> would definitely want an array and not a generator.  In the majority case 
> both work equally well, cool, but I don't know when I would even use an 
> array-dependent version.  And converting from a generator to an array is 
> trivial; the other way, very not so much.
>

Would there be any notable performance difference? I'd expect the case
where one actually wanted the whole array to be slower if it was going
through a generator, vs a tight loop, due to the additional level of
indirection.

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] RFC Draft: Comprehensions

[Larry Garfield]

On Tue, Mar 19, 2019, at 7:10 PM, Robert Hickman wrote:
> > > Why not apply the same approach to PHP? There is iterator_to_array() to
> > > convert a generator to an array, so we may not need both syntaxes.
> > > However, I think using [] for something that is *not an array* is
> > > counter-intuitive.
> >
> > No, I would definitely be for []-syntax producing an array. As I said,
> > that's in my experience what people usually want.
> >
> 
> In python comprehensions on [] makes a list and comprehensions in {}
> make a dictionary (list and dict comprehensions). As PHP only has one
> 'array' type using [] makes sense. Along that train of thought, should
> comprehensions also be possible in the old array() syntax?

Honestly, I cannot think of any case where I'd use a comprehension where I 
would definitely want an array and not a generator.  In the majority case both 
work equally well, cool, but I don't know when I would even use an 
array-dependent version.  And converting from a generator to an array is 
trivial; the other way, very not so much.

> Does the proposal include comprehensions which build associative
> arrays? In python you can do {a['key'] : a for a in lst}, where lst is
> a list of dicts. I often use this to create a dictionary whose keys
> index a sub-element of a nested dictionary, as my example shows.

The proposal as written, yes, you can produce values or keys-and-values.

I would still love feedback on possible alternative syntaxes, per my earlier 
email.

--Larry Garfield

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] RFC Draft: Comprehensions

[Robert Hickman]

> > Why not apply the same approach to PHP? There is iterator_to_array() to
> > convert a generator to an array, so we may not need both syntaxes.
> > However, I think using [] for something that is *not an array* is
> > counter-intuitive.
>
> No, I would definitely be for []-syntax producing an array. As I said,
> that's in my experience what people usually want.
>

In python comprehensions on [] makes a list and comprehensions in {}
make a dictionary (list and dict comprehensions). As PHP only has one
'array' type using [] makes sense. Along that train of thought, should
comprehensions also be possible in the old array() syntax?

Does the proposal include comprehensions which build associative
arrays? In python you can do {a['key'] : a for a in lst}, where lst is
a list of dicts. I often use this to create a dictionary whose keys
index a sub-element of a nested dictionary, as my example shows.

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] RFC Draft: Comprehensions

[Stanislav Malyshev]

Hi!

> In Python, the difference is that []-syntax gives you a list
> (pre-comupted), whereas without the [] you get a generator
> (generate-on-demand). This distinction is important because the
> generator might be iterating over something non-repeatable (e.g. another
> generator), or have some destructive or mutating effect. You also might
> not want to take the performance penalty of computing it every time you
> iterate over it.

Understandable, but I wonder how important it is for the short syntax
implied by list comprehensions? I mean, there could be all kinds of
complex cases, but those are easily doable by using already existing
functional syntax. If you need a generator over non-repeatable sequence,
you can get it now with a functional syntax. But in my experience, about
99% times I use this syntax (in Python) is a simple sequence filter and
I want to get a list out of it. I suspect PHP usage would be the same.
So I wonder if it's worth to bother inventing special syntax for
something that will be used in 1% of cases, and even in those cases
might be more readable as a generator function?

> Why not apply the same approach to PHP? There is iterator_to_array() to
> convert a generator to an array, so we may not need both syntaxes.
> However, I think using [] for something that is *not an array* is
> counter-intuitive.

No, I would definitely be for []-syntax producing an array. As I said,
that's in my experience what people usually want.

-- 
Stas Malyshev
smalys...@gmail.com

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] RFC Draft: Comprehensions

[Rowan Collins]

On 19/03/2019 20:24, Levi Morrison wrote:

Today in the Dart world, Bob Nystrom published an article called
[Making Dart a Better Language for UI][1]. I think it's an incredibly
relevant article, since it is essentially about comprehensions, why
they are a thing, as well as some of the design choices they made.

I think everyone in this thread ought to spend the 10-15 minutes reading it.

   [1]: 
https://medium.com/dartlang/making-dart-a-better-language-for-ui-f1ccaf9f546c



Hi Levi,

That is indeed a very interesting article, thanks for sharing.

The idea of foreach and if just being available whenever you're 
specifying an array is a really interesting version of this construct.



The mention of the spread operator has actually given me a different 
idea, related to iterator_to_array. What if the "..." operator could 
spread any iterable, not just arrays as currently proposed in an RFC [1]?


$iterator = new SomethingIterator;
$array = [ ...$iterator ];

$array2 = [ ...someGenerator() ];


That plus the current comprehension proposal would give us this:

$array = [ ...[ foreach $list as $x if $x % 2 yield $x*2] ];

Which could fairly naturally be special cased to this:

$array = [ ... foreach $list as $x if $x % 2 yield $x*2 ];


This would give us concise syntax for both iterator and array forms, in 
a way that fits the rest of the language better than () vs [].



[1]: https://wiki.php.net/rfc/spread_operator_for_array

--
Rowan Collins
[IMSoP]


--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] RFC Draft: Comprehensions

[Levi Morrison]

Today in the Dart world, Bob Nystrom published an article called
[Making Dart a Better Language for UI][1]. I think it's an incredibly
relevant article, since it is essentially about comprehensions, why
they are a thing, as well as some of the design choices they made.

I think everyone in this thread ought to spend the 10-15 minutes reading it.

  [1]: 
https://medium.com/dartlang/making-dart-a-better-language-for-ui-f1ccaf9f546c

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] RFC Draft: Comprehensions

[Andrea Faulds]

Hi,

Stanislav Malyshev wrote:



Finally, Python makes a distinction between list comprehensions using []
and generator expressions using (). This proposal effectively corresponds
to generator expressions, but uses the [] syntax. I'm wondering if that
will cause confusion.


Do we need this distinction for anything useful?



In Python, the difference is that []-syntax gives you a list 
(pre-comupted), whereas without the [] you get a generator 
(generate-on-demand). This distinction is important because the 
generator might be iterating over something non-repeatable (e.g. another 
generator), or have some destructive or mutating effect. You also might 
not want to take the performance penalty of computing it every time you 
iterate over it.


Why not apply the same approach to PHP? There is iterator_to_array() to 
convert a generator to an array, so we may not need both syntaxes. 
However, I think using [] for something that is *not an array* is 
counter-intuitive.


Thanks,
Andrea

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] RFC Draft: Comprehensions

[Dik Takken]

On 13-03-19 23:30, Rowan Collins wrote:
> At risk of complicating things further, might the solution to that be to
> have a shorter syntax for iterator_to_array in general?
> 
> It's a shame array-casts are defined for arbitrary objects, else we
> could have (array)$iterator - and therefore (array)[foreach ($users as
> $user) yield $user->firstName]

A shorter alternative for iterator_to_array() could work as well I
think. Casting would have been stellar, but not feasible. However, could
using the array() built-in work? Something like:

array([foreach ($users as $user) yield $user->firstName])

Just thinking out loud here...

Regards,
Dik Takken

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] RFC Draft: Comprehensions

[Dik Takken]

On 14-03-19 04:21, Larry Garfield wrote:
> To the question of having both a generator and array version, I would have to 
> say no.  As noted in the RFC, most cases where you'd want to use a 
> comprehension are not places where you'd be feeding the result into an array 
> function.  On the off chance that you are converting the iterable into an 
> array is trivial enough that supporting, documenting, and learning two 
> slightly different syntaxes seems a net negative.

I fully agree that comprehensions can replace the usual array_*()
functions in many cases. However, when you just happen to use a library
that has an array oriented API, then you still need to pass it arrays.

It would be disappointing when comprehensions turn out to be tedious to
use in these situations, because they are no good fit. A dual generator
/ array syntax will make comprehensions fit in anywhere.

Then there are the cases where you want to access the output of the
comprehension multiple times. This may be expensive when a comprehension
is a generator.

So I still think that using a comprehension to create an array isn't an
off chance but a common use case.

Regards,
Dik Takken

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] RFC Draft: Comprehensions

[Michał Brzuchalski]

czw., 14 mar 2019, 04:22 użytkownik Larry Garfield 
napisał:

> On Wed, Mar 13, 2019, at 6:30 PM, Rowan Collins wrote:
> > On 13/03/2019 21:10, Dik Takken wrote:
> > > So in practice, I expect that
> > > using comprehensions as proposed in the new RFC will also require doing
> > > a lot of iterator_to_array(). A dual comprehension syntax could fix
> that.
> >
> >
> > At risk of complicating things further, might the solution to that be to
> > have a shorter syntax for iterator_to_array in general?
> >
> > It's a shame array-casts are defined for arbitrary objects, else we
> > could have (array)$iterator - and therefore (array)[foreach ($users as
> > $user) yield $user->firstName]
>
> I am again going to reply to a bunch of people at once here...
>
>
> If I can summarize the responses so far, they seem to fall into one of two
> categories:
>
> 1) Love the idea, but wouldn't short-closures be close enough?
>
> 2) Love the idea, but hate the particular syntax proposed.
>
> On the plus side, it seems almost everyone is on board in concept, so
> yay.  That of course just leaves the syntax bikeshedding, which is always
> the fun part.
>
> As an aside, someone up-thread said that comprehensions were "an easier
> way to write foreach loops", which is only true by accident.
> Comprehensions are more formally a way of defining one set in relation to
> another set.  That is, they are a declarative relationship between one set
> and another.  While in PHP that ends up effectively being a short-hand for
> foreach loops, that's more an accidental implementation detail.  The syntax
> used by many other languages to achieve the same thing doesn't look at all
> like loop syntax.
>
> To the question of having both a generator and array version, I would have
> to say no.  As noted in the RFC, most cases where you'd want to use a
> comprehension are not places where you'd be feeding the result into an
> array function.  On the off chance that you are converting the iterable
> into an array is trivial enough that supporting, documenting, and learning
> two slightly different syntaxes seems a net negative.
>
> To Rowan's point, I would be fully in favor of an easier syntax
> alternative to iterator_to_array().  I think that's rather similar
> (although not identical) to the "run out an iterator" add-on mentioned in
> the RFC.  I would support that, but I think it's a bit orthogonal and
> should not be a blocker for short-closures or for comprehensions.
>
> As for the specific syntax, I see a couple of options.
>
> 1) Assuming that short-lambdas get adopted and they can transparently
> support generators, the following syntax becomes automatically possible:
>
> $gen = (fn() => foreach($arr as $k => $v) if ($k % 2) yield $v;)();
>
> While that does work, there's an awful lot of symbol salad there: (fn() =>
> and ;)(); are both just gross and hard to type.  I would consider that not
> a full solution for comprehensions because of how clumsy it is.
>

Have you thought about adopting a keyword which fill look similar as short
closures but by default yield instead of return.
I'm thinking of something like that which looks close to short functions
but shorter:

$arr = [
'apple' => ['a' => 1],
'orange' => ['a' => 2],
]
$comprehension = from($arr as $k => ['a' => $a]) => if ($k === 'orange' ||
$a === 1) $a;
$comprehension = from($arr as $k => ['a' => $a]) => {
if ($k === 'orange') yield $a;
if ($a === 1) yield $a;
};

The 'from' keyword indicates that it is going to yield from what put in the
parenthesis.

Probably it won't be ambiguous when nesting comprehension.

The example includes also a list array destruct on purpose to show all what
foreach can do.



> 2) We could include an even-shorter-lambda syntax, potentially, or perhaps
> a short-lambda-based comprehension syntax.  For example (and this may not
> be parser friendly but it's just to demonstrate the idea):
>
> $gen = fn{ foreach($arr as $k => $v) if ($k % 2) yield $v };
>
> That would be a short-hand for a short-closure that has no parameters, and
> we could detect the yield and self-execute.  The language inside the
> function body would still be a bit verbose, but it would technically be any
> legal single statement, which would offer some potentially interesting
> (scary?) options.  I would consider this an acceptable solution for
> comprehensions-ish in PHP.
>
> 3) The specific syntax proposed in the RFC is Python-inspired and
> PHP-ified, but there's no reason we need to stick to that.  There are a
> myriad of other syntaxes for comprehensions in other languages that we
> could steal if they fit better, some of which wouldn't at all resemble
> foreach loops and thus avoid the for/foreach confusion.
>
> Wikipedia of course has a large index of them we can mine:
>
>
> https://en.wikipedia.org/wiki/Comparison_of_programming_languages_(list_comprehension)
>
> It appears that the most common syntax involves [] of some variety, which
> pose parsing problems 

Re: [PHP-DEV] RFC Draft: Comprehensions

[Larry Garfield]

On Wed, Mar 13, 2019, at 6:30 PM, Rowan Collins wrote:
> On 13/03/2019 21:10, Dik Takken wrote:
> > So in practice, I expect that
> > using comprehensions as proposed in the new RFC will also require doing
> > a lot of iterator_to_array(). A dual comprehension syntax could fix that.
> 
> 
> At risk of complicating things further, might the solution to that be to 
> have a shorter syntax for iterator_to_array in general?
> 
> It's a shame array-casts are defined for arbitrary objects, else we 
> could have (array)$iterator - and therefore (array)[foreach ($users as 
> $user) yield $user->firstName]

I am again going to reply to a bunch of people at once here...


If I can summarize the responses so far, they seem to fall into one of two 
categories:

1) Love the idea, but wouldn't short-closures be close enough?

2) Love the idea, but hate the particular syntax proposed.

On the plus side, it seems almost everyone is on board in concept, so yay.  
That of course just leaves the syntax bikeshedding, which is always the fun 
part.

As an aside, someone up-thread said that comprehensions were "an easier way to 
write foreach loops", which is only true by accident.  Comprehensions are more 
formally a way of defining one set in relation to another set.  That is, they 
are a declarative relationship between one set and another.  While in PHP that 
ends up effectively being a short-hand for foreach loops, that's more an 
accidental implementation detail.  The syntax used by many other languages to 
achieve the same thing doesn't look at all like loop syntax.

To the question of having both a generator and array version, I would have to 
say no.  As noted in the RFC, most cases where you'd want to use a 
comprehension are not places where you'd be feeding the result into an array 
function.  On the off chance that you are converting the iterable into an array 
is trivial enough that supporting, documenting, and learning two slightly 
different syntaxes seems a net negative.

To Rowan's point, I would be fully in favor of an easier syntax alternative to 
iterator_to_array().  I think that's rather similar (although not identical) to 
the "run out an iterator" add-on mentioned in the RFC.  I would support that, 
but I think it's a bit orthogonal and should not be a blocker for 
short-closures or for comprehensions.

As for the specific syntax, I see a couple of options.

1) Assuming that short-lambdas get adopted and they can transparently support 
generators, the following syntax becomes automatically possible:

$gen = (fn() => foreach($arr as $k => $v) if ($k % 2) yield $v;)();

While that does work, there's an awful lot of symbol salad there: (fn() => and 
;)(); are both just gross and hard to type.  I would consider that not a full 
solution for comprehensions because of how clumsy it is.

2) We could include an even-shorter-lambda syntax, potentially, or perhaps a 
short-lambda-based comprehension syntax.  For example (and this may not be 
parser friendly but it's just to demonstrate the idea):

$gen = fn{ foreach($arr as $k => $v) if ($k % 2) yield $v };

That would be a short-hand for a short-closure that has no parameters, and we 
could detect the yield and self-execute.  The language inside the function body 
would still be a bit verbose, but it would technically be any legal single 
statement, which would offer some potentially interesting (scary?) options.  I 
would consider this an acceptable solution for comprehensions-ish in PHP.

3) The specific syntax proposed in the RFC is Python-inspired and PHP-ified, 
but there's no reason we need to stick to that.  There are a myriad of other 
syntaxes for comprehensions in other languages that we could steal if they fit 
better, some of which wouldn't at all resemble foreach loops and thus avoid the 
for/foreach confusion.

Wikipedia of course has a large index of them we can mine:

https://en.wikipedia.org/wiki/Comparison_of_programming_languages_(list_comprehension)

It appears that the most common syntax involves [] of some variety, which pose 
parsing problems for PHP, but a few other options jump out at me as possible 
syntaxes to pilfer:

C# has this SQL-esque syntax (which may involve too many additional language 
keywords):

var ns = from x in Enumerable.Range(0,100)
 where x*x > 3
 select x*2;

Elixr, Erlang, and Haskell use the <- symbol, which... I don't think we use 
anywhere else currently?  In Elixir:

for x <- 0..100, x * x > 3, do: x * 2

Java 8, Ruby, Rust, and Swift are very very similar, and use a fluent syntax.  
The Rust example:

(0..100).filter(|x| x * x > 3).map(|x| 2 * x).collect();

While that could not be taken as-is, of course, it does propose an interesting 
alternative approach, if we limit comprehensions to Traversable objects rather 
than any iterable (that is, exclude arrays):

$t->filter(fn($v) => expression)->filter(fn($k, $v) => expression)->map(fn($v) 
=> expression);

Which would, in turn, each produce a 

Re: [PHP-DEV] RFC Draft: Comprehensions

[Rowan Collins]

On 13/03/2019 21:10, Dik Takken wrote:

So in practice, I expect that
using comprehensions as proposed in the new RFC will also require doing
a lot of iterator_to_array(). A dual comprehension syntax could fix that.



At risk of complicating things further, might the solution to that be to 
have a shorter syntax for iterator_to_array in general?


It's a shame array-casts are defined for arbitrary objects, else we 
could have (array)$iterator - and therefore (array)[foreach ($users as 
$user) yield $user->firstName]


Regards,

--
Rowan Collins
[IMSoP]


--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] RFC Draft: Comprehensions

[Kalle Sommer Nielsen]

Hi

Den tir. 12. mar. 2019 kl. 18.36 skrev Chase Peeler :
> I've never liked "developers might use it wrong" as reason to not implement 
> something, especially if hard to read/complex code is the worst impact to 
> improper use.
Me neither to be fair, but I think it is a fair consideration to have
when designing syntax. I agree with Stas reply in this thread
regarding this.

> I understand that you are talking about the syntax (which others have said is 
> still up for discussion) and not the feature itself. I still wanted to make 
> the comment above more a general response to that line of thinking.
I like the feature as it is, and if the syntax is right for it then I
would most likely vote for it if it comes to that state.


-- 
regards,

Kalle Sommer Nielsen
ka...@php.net

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] RFC Draft: Comprehensions

[Dik Takken]

On 11-03-19 05:29, Larry Garfield wrote:
> Sara and I tried putting the expression first, as in Python, but that made 
> the lexer very unhappy

There is another possible reason to put the expression last: It
preserves the ordering of the intended foreach loop. What I mean to say
is this: One can change a loop into a comprehension by simply removing
the newlines, braces, etc. Other than that, the various elements stay in
the same place. For me, this makes it easier to 'see' the intended loop
in the comprehension. I have always found the Python syntax to be
confusing in this regard.


On 11-03-19 13:16, Nikita Popov wrote:
> I've brought up a similar proposal a few years ago, probably around the
> time generators were first introduced: https://externals.io/message/61021

May I please point at one particularly nice detail in Nikita's proposal:
A dual syntax for both array comprehensions and generator expressions:

$arr = [foreach ($users as $user) yield $user->firstName]
$gen = (foreach ($users as $user) yield $user->firstName)

I would love to see a dual syntax included in the current proposal as
well. I think it makes sense considering the fact that many functions in
PHP only accept arrays, not generators. So in practice, I expect that
using comprehensions as proposed in the new RFC will also require doing
a lot of iterator_to_array(). A dual comprehension syntax could fix that.

One little remark about the array comprehension syntax: Using a keyword
other than "yield" would make the difference between the two more
obvious. Perhaps the array comprehension could use "return":

$arr = [foreach ($users as $user) return $user->firstName]
$gen = (foreach ($users as $user) yield $user->firstName)

> I think my main point of feedback would be to stick closer to existing PHP
> syntax, even if it costs us some brevity. I would prefer
> 
> $gen = [foreach ($list as $x) if ($x % 2) yield $x * 2];
> 
> over
> 
> $gen = [for $list as $x if $x % 2 yield $x * 2];
> 
> The latter is nice in languages that generally use "for" for this purpose
> and generally allow omitting parentheses, but I don't think it's good to
> introduce this kind of syntax inconsistency in one place.

Keeping the parenthesis would seem to be more consistent indeed.
Omitting the curly brackets of a "foreach" loop is valid syntax in
current PHP while omitting the parenthesis from the "foreach" and "if"
is not.

However, the RFC also points out that the syntax should be limited
compared to a "foreach" loop, which makes sense. This makes me want to
favor the syntax as proposed, without the parenthesis. The "full syntax"
with parenthesis kind of suggests that comprehension syntax is no
different than a regular foreach loop, supporting the existing if / else
syntax, etc, but on a single line. That level of similarity might be
confusing. For example, if I can write:

if ($x % 2) yield $x * 2;

and

if ($x % 2) yield $x * 2; else yield $x;

then why is it that I can write:

[foreach ($list as $x) if ($x % 2) yield $x * 2]

but not:

[foreach ($list as $x) if ($x % 2) yield $x * 2; else yield $x]

Bottom line: If comprehensions aim to provide an alternative syntax to
foreach loops, with different rules, why try to make them look exactly
like foreach loops? That could be confusing.

Also note that Python list comprehensions basically show the same
"inconsistency" as the proposed PHP comprehensions: In PHP, parenthesis
are required after "foreach" or "if". The RFC proposes to omit them in
comprehensions. In Python, colons are required after "for" and "if".
Python comprehensions omit these. Not that "Python has it too" is a
valid argument for anything at all though...

I do agree that using "foreach" in stead of "for" may be a better
choice. The proposed comprehensions are compressed foreach loops. In my
mind, I want to think "foreach" when I see a comprehension, not "for".


On 12-03-19 09:54, Kalle Sommer Nielsen wrote:
> One more thing that kinda "annoys" me / what I base my above statement
> on is; when reading some of the example is that the code examples
> given, as in the code inside a Comprehension does not look very
> PHP-ish besides the $-sigil and I do not like the idea of having "two"
> syntaxes for PHP where one is only available in a very narrow-,
> specific context and I feel it would be prone to potential errors.

I fully understand this feeling. It's new to PHP and looks quite weird
at first. Many of us know comprehensions from the Python world. Maybe
the association with Python is so strong that the very concept of
comprehensions feels Pythonic and not something that fits into PHP.

I was skeptical about comprehensions myself when I started coding in
Python. However, after getting used to them, I must admit that I find
comprehensions easier to read than traditional loops. When they are
short, that is. :) Looking at a comprehension, I can still 'see' the
intended loop. Only quicker, because there is 

Re: [PHP-DEV] RFC Draft: Comprehensions

[Stanislav Malyshev]

Hi!

> I think my main point of feedback would be to stick closer to existing PHP
> syntax, even if it costs us some brevity. I would prefer
> 
> $gen = [foreach ($list as $x) if ($x % 2) yield $x * 2];
> 
> over
> 
> $gen = [for $list as $x if $x % 2 yield $x * 2];

Agree here. There's a principle of least surprise - "for" in PHP looks
the certain way, and sequence iteration looks certain way, and it's
better to work within those paradigms than import different way of doing
the same from Python. Especially given that for/foreach look pretty
close, and remembering in which context "for" is ok for sequences and in
which not is going to be very annoying. It's much easier to remember
"for is always for(x;y;z), foreach is always foreach(x as y=>z)".

Also agree about parentheses - reading a complex expression without
separators would be annoying. I know python loves to do this but even in
an IDE with highlighter it's not always easy to quickly figure out which
part belongs where. Delimiting it would make it easier IMHO, and also
allows again to carry over the intuition of how foreach() and if() work
from already known constructs.

> Finally, Python makes a distinction between list comprehensions using []
> and generator expressions using (). This proposal effectively corresponds
> to generator expressions, but uses the [] syntax. I'm wondering if that
> will cause confusion.

Do we need this distinction for anything useful?

-- 
Stas Malyshev
smalys...@gmail.com

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] RFC Draft: Comprehensions

[Chase Peeler]

On Tue, Mar 12, 2019 at 10:12 AM Kalle Sommer Nielsen  wrote:

> Hi
>
> Den tir. 12. mar. 2019 kl. 15.49 skrev Chase Peeler  >:
> > Everything looks weird and "non-phpish" when it's new. OO constructs
> weren't PHP-ish at first, because PHP didn't originally support OO. Imagine
> "foreach" didn't exist in PHP and we were still using
> while(list($key,$value) = each($arr)) syntax.
> >
> > I agree that some of the examples are a bit hard to read and understand
> - but that's only because we haven't been exposed to them.
>
> What I mean here is that the lack of things like the parentheses makes
> the code generally harder to read, imagine this for large
> comprehensions as userland would certainly do things like that and you
> end up with monstrosities of code that is less verbose than the
> counter examples given in the RFC. I mean sure I could just rewrite a
> comprehension with a lot of whitespace, but it kinda seem to defeat
> the purpose of short circuited generators and I might as well just
> write it the more verbose way.
>
> I've never liked "developers might use it wrong" as reason to not
implement something, especially if hard to read/complex code is the worst
impact to improper use.


> I like the idea, just not the syntax because it has the potential
> (along with short closures) to be annoying to read in the long run.
>

I understand that you are talking about the syntax (which others have said
is still up for discussion) and not the feature itself. I still wanted to
make the comment above more a general response to that line of thinking.


>
> --
> regards,
>
> Kalle Sommer Nielsen
> ka...@php.net
>
-- 
-- Chase
chasepee...@gmail.com

Re: [PHP-DEV] RFC Draft: Comprehensions

[Levi Morrison]

On Sun, Mar 10, 2019 at 3:45 PM Larry Garfield  wrote:
>
> Hello, peoples.  I know it's been discussed once or twice before on the list, 
> many years ago, but not recently.  I therefore feel OK putting forth the 
> following draft proposal for Comprehensions, aka "compact generators", in PHP:
>
> https://wiki.php.net/rfc/comprehensions
>
> Sara Golemon has written a preliminary patch that is partially complete (see 
> the RFC link for details, it's impressively simple), but unfortunately 
> doesn't have the bandwidth to complete it at this time.  I am therefore 
> looking for collaborators with more knowledge of internals than I (which is 
> almost everyone) to help finish it up.
>
> The syntax proposed is also subject to revision if a terser but still 
> lexer-friendly alternative can be proposed.
>
> At the moment I'm not calling this Proposed yet, as I don't feel comfortable 
> doing so until someone else is on board to finish coding it.  That said, if 
> someone wants to weigh in on the concept for now (hopefully supportively) 
> that's also fine.
>
> Anyone excited enough to help finish the job?
>
> (This is my first RFC collaboration, so if you're going to smack me for 
> goofing something please be gentle about it.)
>
> --
>   Larry Garfield
>   la...@garfieldtech.com
>
> --
> PHP Internals - PHP Runtime Development Mailing List
> To unsubscribe, visit: http://www.php.net/unsub.php

I want to like the idea of list comprehensions. However, after
thinking about it for quite a bit over the past year as well as since
discussion started, the issue is really that closures are terrible in
PHP.

Look at this example from the RFC:

$result = (function() use ($table) {
  foreach ($table as $num => $row) {
if ($num % 2 == 0) {
  foreach ($row as $col => $val) {
if ($col >= 3) {
  yield $num => $val;
}
  }
}
  }
})();

We could use this syntax today:

$result = (function () {
  foreach ($table as $num => $row)
if ($num % 2 == 0)
  foreach ($row as $col => $val)
if ($col >= 3)
  yield $num => $val;
})();

Compare that to this code using comprehensions, using the same whitespace style:

$result = (for $table as $num => $row
  if $num %2 ==0
for $row as $col => $value
  if $col >= 3
yield $num => $val
);

In my opinion there are no meaningful differences here, and nothing
prevents you from using the all-in-one-line style if you care to.

Let's look at a shorter piece, and let's also assume that the
following code is eagerly evaluated and stored into an array (which is
not what the RFC is proposing, but it helps for this example):

$result = [ for ($hex_numbers as $num) yield intval($num, 16); ];

With closures as they are today:

$result = array_map(function($num) { return intval($num, 16); },
$hex_numbers);

Oof, that sucks and I didn't even have to bind any variables. But if
we had shorter closures (this is JavaScript style) then it's fine
again:

$result = array_map(($num) => intval($num, 16), $hex_numbers);

My point is, closures are the major culprit. We really need shorter closures.

Again, I want to like comprehensions. I think we should revisit list
comprehensions after we've had shorter closures for a little while
(hopefully we can get something passed for 7.4 or 8.0).

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] RFC Draft: Comprehensions

[Andrey Hristov]

Hi,
the real fun starts when people start to put list comprehension in a 
list comprehension in a list comprehension. And the result is one-liners 
that are pretty dense and write once never read.


The problem during software development are not the key strokes, it's 
the time which later has to be invested to understand the code that lack 
the additional keystrokes. Of course one can learn to understand dense 
stuff. One can also learn to read German words with a length of 20 and 
up. It ain't that hard and needs practice. Is it worth when non-compound 
words also work (hint: German words can be written alternatively with 
hypens and CamelCaps, which really improves the readability).


Best,
Andrey


On 12.03.19 г. 15:49 ч., Chase Peeler wrote:

On Tue, Mar 12, 2019 at 4:55 AM Kalle Sommer Nielsen  wrote:


Hi

Den søn. 10. mar. 2019 kl. 23.45 skrev Larry Garfield <
la...@garfieldtech.com>:


Hello, peoples.  I know it's been discussed once or twice before on the

list, many years ago, but not recently.  I therefore feel OK putting forth
the following draft proposal for Comprehensions, aka "compact generators",
in PHP:


https://wiki.php.net/rfc/comprehensions


While I think believe the RFC has great intentions, I am not a fan by
any means to introduce convoluted syntaxes like this into PHP, I think
it is rather hard to read vs the gain of saving a few key strokes. I
think that both the shorthand closures and this makes PHP code
potentially even harder to read and I would prefer we did not add such
essential sugar where you need to understand the semantics very well
to understand what a piece of foreign code may do, in what order (like
the no yield example given).

One more thing that kinda "annoys" me / what I base my above statement
on is; when reading some of the example is that the code examples
given, as in the code inside a Comprehension does not look very
PHP-ish besides the $-sigil and I do not like the idea of having "two"
syntaxes for PHP where one is only available in a very narrow-,
specific context and I feel it would be prone to potential errors.

Everything looks weird and "non-phpish" when it's new. OO constructs

weren't PHP-ish at first, because PHP didn't originally support OO. Imagine
"foreach" didn't exist in PHP and we were still using
while(list($key,$value) = each($arr)) syntax.

I agree that some of the examples are a bit hard to read and understand -
but that's only because we haven't been exposed to them.

This isn't meant as an endorsement of the proposal. I just don't think the
fact that it adds something different than what we are used to is a good
justification for not supporting it. As a non-voting userland developer,
I'm personally ambivalent. They seem cool, and I'm sure I'll use them, but
it won't be a big deal if they aren't added.



Another thing that has been mentioned by multiple people now is the
for-vs-foreach; If it is a foreach, which you mention in the RFC then
it should be so, not for. A little background on where I am coming
from here; I think it is great that we look at other languages and
technologies when designing PHP and adding more features the language,
however while I do believe it is an important point to keep cross
language consistency, I do think that consistency with PHP itself is
more important, which I believe too, is the reason for the expression
for using foreach in this context comes from (not only from my PoV).

I was originally going to say I agreed with this position, but, after

thinking about it, I've changed my mind. A "for" loop is just a method of
specifying the keys that you are iterating over. Since you aren't
specifying the keys inside a comprehension, the traditional "for" loop
doesn't really exist. I'd suggest supporting both keywords, though.



--
regards,

Kalle Sommer Nielsen
ka...@php.net

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

--

-- Chase
chasepee...@gmail.com




--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] RFC Draft: Comprehensions

[Björn Larsson]

Den 2019-03-11 kl. 13:16, skrev Nikita Popov:


On Sun, Mar 10, 2019 at 10:45 PM Larry Garfield 
wrote:


Hello, peoples.  I know it's been discussed once or twice before on the
list, many years ago, but not recently.  I therefore feel OK putting forth
the following draft proposal for Comprehensions, aka "compact generators",
in PHP:

https://wiki.php.net/rfc/cohttps://externals.io/message/61021mprehensions


Sara Golemon has written a preliminary patch that is partially complete
(see the RFC link for details, it's impressively simple), but unfortunately
doesn't have the bandwidth to complete it at this time.  I am therefore
looking for collaborators with more knowledge of internals than I (which is
almost everyone) to help finish it up.

The syntax proposed is also subject to revision if a terser but still
lexer-friendly alternative can be proposed.

At the moment I'm not calling this Proposed yet, as I don't feel
comfortable doing so until someone else is on board to finish coding it.
That said, if someone wants to weigh in on the concept for now (hopefully
supportively) that's also fine.

Anyone excited enough to help finish the job?

(This is my first RFC collaboration, so if you're going to smack me for
goofing something please be gentle about it.)


I've brought up a similar proposal a few years ago, probably around the
time generators were first introduced: https://externals.io/message/61021

I think my main point of feedback would be to stick closer to existing PHP
syntax, even if it costs us some brevity. I would prefer

 $gen = [foreach ($list as $x) if ($x % 2) yield $x * 2];

over

 $gen = [for $list as $x if $x % 2 yield $x * 2];

The latter is nice in languages that generally use "for" for this purpose
and generally allow omitting parentheses, but I don't think it's good to
introduce this kind of syntax inconsistency in one place.

Similarly, I found the ability to omit the "yield" expression somewhat
confusing. Especially once you get to nesting

$gen = [for $a as $b for $b as $c];

the behavior becomes non-obvious. Also wondering how this part interacts
with keys, do you decide whether or not to yield the keys based on whether
they are part of the for expression?

 $gen = [for $list as $v]; // drops keys
 $gen = [for $list as $k => $v]; // keeps keys

Finally, Python makes a distinction between list comprehensions using []
and generator expressions using (). This proposal effectively corresponds
to generator expressions, but uses the [] syntax. I'm wondering if that
will cause confusion.

Regards,
Nikita


As a userland developer I found the example parsing large log files
from the proposal a few years back a good use case for this feature.

Besides that, I recall that for arrow functions a few different syntax
proposals were on the table. Wonder which one of those plays best
with this proposal? Maybe that should be taken into consideration
when looking at this RFC.

r//Björn L


--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] RFC Draft: Comprehensions

[Sara Golemon]

On Sun, Mar 10, 2019 at 4:45 PM Larry Garfield 
wrote:

> Sara Golemon has written a preliminary patch that is partially complete
> (see the RFC link for details, it's impressively simple), but unfortunately
> doesn't have the bandwidth to complete it at this time.  I am therefore
> looking for collaborators with more knowledge of internals than I (which is
> almost everyone) to help finish it up.
>
> Just want to clarify; The proof-of-concept I made for Larry is *just* a
proof of concept to illustrate one potential syntax.  The actual
implementation is a dumpster fire that was hastily assembled between
conference sessions.  Please ignore the mechanics of how it was put
together and focus on the intended syntax and resulting behavior.

-Sara

Re: [PHP-DEV] RFC Draft: Comprehensions

[Kalle Sommer Nielsen]

Hi

Den tir. 12. mar. 2019 kl. 15.49 skrev Chase Peeler :
> Everything looks weird and "non-phpish" when it's new. OO constructs weren't 
> PHP-ish at first, because PHP didn't originally support OO. Imagine "foreach" 
> didn't exist in PHP and we were still using while(list($key,$value) = 
> each($arr)) syntax.
>
> I agree that some of the examples are a bit hard to read and understand - but 
> that's only because we haven't been exposed to them.

What I mean here is that the lack of things like the parentheses makes
the code generally harder to read, imagine this for large
comprehensions as userland would certainly do things like that and you
end up with monstrosities of code that is less verbose than the
counter examples given in the RFC. I mean sure I could just rewrite a
comprehension with a lot of whitespace, but it kinda seem to defeat
the purpose of short circuited generators and I might as well just
write it the more verbose way.

I like the idea, just not the syntax because it has the potential
(along with short closures) to be annoying to read in the long run.

-- 
regards,

Kalle Sommer Nielsen
ka...@php.net

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] RFC Draft: Comprehensions

[Chase Peeler]

On Tue, Mar 12, 2019 at 4:55 AM Kalle Sommer Nielsen  wrote:

> Hi
>
> Den søn. 10. mar. 2019 kl. 23.45 skrev Larry Garfield <
> la...@garfieldtech.com>:
> >
> > Hello, peoples.  I know it's been discussed once or twice before on the
> list, many years ago, but not recently.  I therefore feel OK putting forth
> the following draft proposal for Comprehensions, aka "compact generators",
> in PHP:
> >
> > https://wiki.php.net/rfc/comprehensions
>
> While I think believe the RFC has great intentions, I am not a fan by
> any means to introduce convoluted syntaxes like this into PHP, I think
> it is rather hard to read vs the gain of saving a few key strokes. I
> think that both the shorthand closures and this makes PHP code
> potentially even harder to read and I would prefer we did not add such
> essential sugar where you need to understand the semantics very well
> to understand what a piece of foreign code may do, in what order (like
> the no yield example given).
>
> One more thing that kinda "annoys" me / what I base my above statement
> on is; when reading some of the example is that the code examples
> given, as in the code inside a Comprehension does not look very
> PHP-ish besides the $-sigil and I do not like the idea of having "two"
> syntaxes for PHP where one is only available in a very narrow-,
> specific context and I feel it would be prone to potential errors.
>
> Everything looks weird and "non-phpish" when it's new. OO constructs
weren't PHP-ish at first, because PHP didn't originally support OO. Imagine
"foreach" didn't exist in PHP and we were still using
while(list($key,$value) = each($arr)) syntax.

I agree that some of the examples are a bit hard to read and understand -
but that's only because we haven't been exposed to them.

This isn't meant as an endorsement of the proposal. I just don't think the
fact that it adds something different than what we are used to is a good
justification for not supporting it. As a non-voting userland developer,
I'm personally ambivalent. They seem cool, and I'm sure I'll use them, but
it won't be a big deal if they aren't added.


> Another thing that has been mentioned by multiple people now is the
> for-vs-foreach; If it is a foreach, which you mention in the RFC then
> it should be so, not for. A little background on where I am coming
> from here; I think it is great that we look at other languages and
> technologies when designing PHP and adding more features the language,
> however while I do believe it is an important point to keep cross
> language consistency, I do think that consistency with PHP itself is
> more important, which I believe too, is the reason for the expression
> for using foreach in this context comes from (not only from my PoV).
>
> I was originally going to say I agreed with this position, but, after
thinking about it, I've changed my mind. A "for" loop is just a method of
specifying the keys that you are iterating over. Since you aren't
specifying the keys inside a comprehension, the traditional "for" loop
doesn't really exist. I'd suggest supporting both keywords, though.


> --
> regards,
>
> Kalle Sommer Nielsen
> ka...@php.net
>
> --
> PHP Internals - PHP Runtime Development Mailing List
> To unsubscribe, visit: http://www.php.net/unsub.php
>
> --
-- Chase
chasepee...@gmail.com

Re: [PHP-DEV] RFC Draft: Comprehensions

[Kalle Sommer Nielsen]

Hi

Den søn. 10. mar. 2019 kl. 23.45 skrev Larry Garfield :
>
> Hello, peoples.  I know it's been discussed once or twice before on the list, 
> many years ago, but not recently.  I therefore feel OK putting forth the 
> following draft proposal for Comprehensions, aka "compact generators", in PHP:
>
> https://wiki.php.net/rfc/comprehensions

While I think believe the RFC has great intentions, I am not a fan by
any means to introduce convoluted syntaxes like this into PHP, I think
it is rather hard to read vs the gain of saving a few key strokes. I
think that both the shorthand closures and this makes PHP code
potentially even harder to read and I would prefer we did not add such
essential sugar where you need to understand the semantics very well
to understand what a piece of foreign code may do, in what order (like
the no yield example given).

One more thing that kinda "annoys" me / what I base my above statement
on is; when reading some of the example is that the code examples
given, as in the code inside a Comprehension does not look very
PHP-ish besides the $-sigil and I do not like the idea of having "two"
syntaxes for PHP where one is only available in a very narrow-,
specific context and I feel it would be prone to potential errors.

Another thing that has been mentioned by multiple people now is the
for-vs-foreach; If it is a foreach, which you mention in the RFC then
it should be so, not for. A little background on where I am coming
from here; I think it is great that we look at other languages and
technologies when designing PHP and adding more features the language,
however while I do believe it is an important point to keep cross
language consistency, I do think that consistency with PHP itself is
more important, which I believe too, is the reason for the expression
for using foreach in this context comes from (not only from my PoV).

-- 
regards,

Kalle Sommer Nielsen
ka...@php.net

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] RFC Draft: Comprehensions

[Larry Garfield]

On Mon, Mar 11, 2019, at 8:16 AM, Nikita Popov wrote:
> On Sun, Mar 10, 2019 at 10:45 PM Larry Garfield 
> wrote:
> 
> > Hello, peoples.  I know it's been discussed once or twice before on the
> > list, many years ago, but not recently.  I therefore feel OK putting forth
> > the following draft proposal for Comprehensions, aka "compact generators",
> > in PHP:
> >
> > https://wiki.php.net/rfc/cohttps://externals.io/message/61021mprehensions
> > 
> >
> > Sara Golemon has written a preliminary patch that is partially complete
> > (see the RFC link for details, it's impressively simple), but unfortunately
> > doesn't have the bandwidth to complete it at this time.  I am therefore
> > looking for collaborators with more knowledge of internals than I (which is
> > almost everyone) to help finish it up.
> >
> > The syntax proposed is also subject to revision if a terser but still
> > lexer-friendly alternative can be proposed.
> >
> > At the moment I'm not calling this Proposed yet, as I don't feel
> > comfortable doing so until someone else is on board to finish coding it.
> > That said, if someone wants to weigh in on the concept for now (hopefully
> > supportively) that's also fine.
> >
> > Anyone excited enough to help finish the job?
> >
> > (This is my first RFC collaboration, so if you're going to smack me for
> > goofing something please be gentle about it.)
> >
> 
> I've brought up a similar proposal a few years ago, probably around the
> time generators were first introduced: https://externals.io/message/61021

Ah, I knew I'd seen it on the list once before.  Thanks.

> I think my main point of feedback would be to stick closer to existing PHP
> syntax, even if it costs us some brevity. I would prefer
> 
> $gen = [foreach ($list as $x) if ($x % 2) yield $x * 2];
> 
> over
> 
> $gen = [for $list as $x if $x % 2 yield $x * 2];
> 
> The latter is nice in languages that generally use "for" for this purpose
> and generally allow omitting parentheses, but I don't think it's good to
> introduce this kind of syntax inconsistency in one place.
> 
> Similarly, I found the ability to omit the "yield" expression somewhat
> confusing. Especially once you get to nesting
> 
>$gen = [for $a as $b for $b as $c];

In that case you'd get a list of $c returned (implicit yield $c), although I 
can see where that's non-obvious.  OTOH, requiring the yield statement every 
time in the majority case seems like excessive verbosity and ceremony.

> the behavior becomes non-obvious. Also wondering how this part interacts
> with keys, do you decide whether or not to yield the keys based on whether
> they are part of the for expression?
> 
> $gen = [for $list as $v]; // drops keys
> $gen = [for $list as $k => $v]; // keeps keys

Yes, that's the intent.  No yield means implicit "key and value"; if key is 
omitted then it's implicitly just 0-based numbers.

> Finally, Python makes a distinction between list comprehensions using []
> and generator expressions using (). This proposal effectively corresponds
> to generator expressions, but uses the [] syntax. I'm wondering if that
> will cause confusion.
> 
> Regards,
> Nikita

I don't think so.  For one, although we started from Python syntax the result 
is decidedly not Python, so I wouldn't expect people to assume the same symbols 
mean the same things.  For another, Python uses [] for a list-producing 
comprehension, and {} for a dictionary-producting comprehension.  Whether those 
return generators or not is a version question, not a syntax question.  Since 
PHP doesn't differentiate between lists and dictionaries to begin with there's 
no need for different syntax.  Our "dictionaries" already use [].

--Larry Garfield

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] RFC Draft: Comprehensions

[Nikita Popov]

On Sun, Mar 10, 2019 at 10:45 PM Larry Garfield 
wrote:

> Hello, peoples.  I know it's been discussed once or twice before on the
> list, many years ago, but not recently.  I therefore feel OK putting forth
> the following draft proposal for Comprehensions, aka "compact generators",
> in PHP:
>
> https://wiki.php.net/rfc/cohttps://externals.io/message/61021mprehensions
> 
>
> Sara Golemon has written a preliminary patch that is partially complete
> (see the RFC link for details, it's impressively simple), but unfortunately
> doesn't have the bandwidth to complete it at this time.  I am therefore
> looking for collaborators with more knowledge of internals than I (which is
> almost everyone) to help finish it up.
>
> The syntax proposed is also subject to revision if a terser but still
> lexer-friendly alternative can be proposed.
>
> At the moment I'm not calling this Proposed yet, as I don't feel
> comfortable doing so until someone else is on board to finish coding it.
> That said, if someone wants to weigh in on the concept for now (hopefully
> supportively) that's also fine.
>
> Anyone excited enough to help finish the job?
>
> (This is my first RFC collaboration, so if you're going to smack me for
> goofing something please be gentle about it.)
>

I've brought up a similar proposal a few years ago, probably around the
time generators were first introduced: https://externals.io/message/61021

I think my main point of feedback would be to stick closer to existing PHP
syntax, even if it costs us some brevity. I would prefer

$gen = [foreach ($list as $x) if ($x % 2) yield $x * 2];

over

$gen = [for $list as $x if $x % 2 yield $x * 2];

The latter is nice in languages that generally use "for" for this purpose
and generally allow omitting parentheses, but I don't think it's good to
introduce this kind of syntax inconsistency in one place.

Similarly, I found the ability to omit the "yield" expression somewhat
confusing. Especially once you get to nesting

   $gen = [for $a as $b for $b as $c];

the behavior becomes non-obvious. Also wondering how this part interacts
with keys, do you decide whether or not to yield the keys based on whether
they are part of the for expression?

$gen = [for $list as $v]; // drops keys
$gen = [for $list as $k => $v]; // keeps keys

Finally, Python makes a distinction between list comprehensions using []
and generator expressions using (). This proposal effectively corresponds
to generator expressions, but uses the [] syntax. I'm wondering if that
will cause confusion.

Regards,
Nikita

Re: [PHP-DEV] RFC Draft: Comprehensions

[Larry Garfield]

On Sun, Mar 10, 2019, at 7:16 PM, Bob Weinand wrote:
> > Am 10.03.2019 um 22:44 schrieb Larry Garfield :
> > 
> > Hello, peoples.  I know it's been discussed once or twice before on the 
> > list, many years ago, but not recently.  I therefore feel OK putting forth 
> > the following draft proposal for Comprehensions, aka "compact generators", 
> > in PHP:
> > 
> > https://wiki.php.net/rfc/comprehensions
> > 
> > Sara Golemon has written a preliminary patch that is partially complete 
> > (see the RFC link for details, it's impressively simple), but unfortunately 
> > doesn't have the bandwidth to complete it at this time.  I am therefore 
> > looking for collaborators with more knowledge of internals than I (which is 
> > almost everyone) to help finish it up.
> > 
> > The syntax proposed is also subject to revision if a terser but still 
> > lexer-friendly alternative can be proposed.
> > 
> > At the moment I'm not calling this Proposed yet, as I don't feel 
> > comfortable doing so until someone else is on board to finish coding it.  
> > That said, if someone wants to weigh in on the concept for now (hopefully 
> > supportively) that's also fine.
> > 
> > Anyone excited enough to help finish the job?
> > 
> > (This is my first RFC collaboration, so if you're going to smack me for 
> > goofing something please be gentle about it.)
> > 
> > -- 
> >  Larry Garfield
> >  la...@garfieldtech.com
> 
> Hey Larry,
> First, I like it. (the idea/intention behind of providing a compact way 
> for list comprehensions)
> 
> Though in general, I think this would be better off implemented with a 
> more flexible short closure; it feels like a special case of these. 
> I.e. that the comprehension syntax can be simply desugared to an 
> immediately invoked short closure. At that point also, it can be a 
> compiler only patch, if we have short closures with the appropriate 
> binding behavior.
> I guess it should be pretty easy to build that on top of my existing 
> short closures patch.
> 
> Regarding the syntax, I'd not conflate for with foreach, the 
> distinction is built into PHP and as such should remain.
> We also might have trivial argument-less short closures written as "{ 
> stmt }" simply, having the comprehension written as $gen = 
> {foreach($list as $x) if($x % 2) yield $x * 2;}(); (note the trailing 
> "()" function call - if we desire to, we might make the semicolon at 
> the end of that single-statement closure optional.)
> 
> Bob

Replying to a couple of people at once here:

1) I am 100% in favor of adopting short closures, the less syntax ceremony in 
it the better.  Bob, if there's a logistical/non-code way I can help make that 
happen let me know, because we wants it, precious.

2) That said, I'm not sure it would fully obviate the need for a 
generator-specific syntax.  Even if the examples from Bob and Wes are shorter 
than what I had envisioned for a short-closure version (nice!), they're still 
notably longer than a purpose-built generator version.  Too, and I acknowledge 
this is a highly subjective statement, they have a lot of syntax salad.  
";)();" at the end is a superb thing to mistype more often than you get it 
right.  If there were some way to reduce that then I might be convinced but I'd 
have to see it first.  (And that means we need short closures still, which we 
don't have yet.)

3) I'd still be in favor of adding a type check to comprehensions, too; I don't 
know how or if that would be possible if they were "just" a fancy use of 
short-closures.

4) Regarding the specific syntax proposed, it's honestly more verbose than I'd 
like as is.  Sara and I tried putting the expression first, as in Python, but 
that made the lexer very unhappy since "[$foo ... ]" is already legal syntax 
for an array of one iterable.  The yield keyword is there to help the parser 
(and reader) tell where the expression begins since, being after the 
conditional, it needs some kind of separator.  If anything, I'd be interested 
in considering entirely different syntaxes borrowed from other languages (see 
the linked Wikipedia page) that would involve even fewer keywords, not using 
longer keywords.  That's why, for instance, it uses for instead of foreach.

5) Regarding Javascript comprehensions, well drat.  I didn't realize it had 
been withdrawn.  That's disappointing.  I'll adjust the RFC at some point 
soonish (it's after midnight here right now) to not mention them.

6) array_map/array_filter order reversed:  Bruce, I think you're right.  It 
took a while for me to grok that it is.  Which IMO is even more reason we want 
some more comprehensible and compact syntax for that use case, as it's 
surprisingly common. :-)

7) Bruce, there is an example of multiple foreach()s.  See the "Iterate a 2D 
array" section in the long code block.

8) To George: yes, since comprehensions as envisioned would work on any 
iterable, and generators are iterables, and comprehensions are generators, 
passing the 

Re: [PHP-DEV] RFC Draft: Comprehensions

[Alexandru Pătrănescu]

Hi Larry,

I'm still digesting the syntax and people have already said about it the
most important things.

I mostly wanted to mention that I think comprehensions should return
\Iterator instead of \Generator. There is nothing that stopped us from
having lazy iterators, even before generators were available.
IMO, all points in *Why generators?* section are valid with \Iterator and
you also noted that send() method is kind of useless.

Just to make it clear, I agree that behind the scene it can use the same
logic of a \Generator but I think userland type should be \Iterator.

For example, if we look at this two important functions widely used for
processing lazy/infinite iterators, the return type is \Iterator
function map(callable $function, iterable $iterable): \Iterator
https://github.com/nikic/iter/blob/master/src/iter.php#L79
function filter(callable $predicate, iterable $iterable): \Iterator
https://github.com/nikic/iter/blob/master/src/iter.php#L203

Regards,
Alex

On Sun, Mar 10, 2019 at 11:45 PM Larry Garfield 
wrote:

> Hello, peoples.  I know it's been discussed once or twice before on the
> list, many years ago, but not recently.  I therefore feel OK putting forth
> the following draft proposal for Comprehensions, aka "compact generators",
> in PHP:
>
> https://wiki.php.net/rfc/comprehensions
>
> Sara Golemon has written a preliminary patch that is partially complete
> (see the RFC link for details, it's impressively simple), but unfortunately
> doesn't have the bandwidth to complete it at this time.  I am therefore
> looking for collaborators with more knowledge of internals than I (which is
> almost everyone) to help finish it up.
>
> The syntax proposed is also subject to revision if a terser but still
> lexer-friendly alternative can be proposed.
>
> At the moment I'm not calling this Proposed yet, as I don't feel
> comfortable doing so until someone else is on board to finish coding it.
> That said, if someone wants to weigh in on the concept for now (hopefully
> supportively) that's also fine.
>
> Anyone excited enough to help finish the job?
>
> (This is my first RFC collaboration, so if you're going to smack me for
> goofing something please be gentle about it.)
>
> --
>   Larry Garfield
>   la...@garfieldtech.com
>
> --
> PHP Internals - PHP Runtime Development Mailing List
> To unsubscribe, visit: http://www.php.net/unsub.php
>
>

Re: [PHP-DEV] RFC Draft: Comprehensions

[G. P. B.]

On Sun, 10 Mar 2019 at 22:45, Larry Garfield  wrote:

> Hello, peoples.  I know it's been discussed once or twice before on the
> list, many years ago, but not recently.  I therefore feel OK putting forth
> the following draft proposal for Comprehensions, aka "compact generators",
> in PHP:
>
> https://wiki.php.net/rfc/comprehensions
>
> Sara Golemon has written a preliminary patch that is partially complete
> (see the RFC link for details, it's impressively simple), but unfortunately
> doesn't have the bandwidth to complete it at this time.  I am therefore
> looking for collaborators with more knowledge of internals than I (which is
> almost everyone) to help finish it up.
>
> The syntax proposed is also subject to revision if a terser but still
> lexer-friendly alternative can be proposed.
>
> At the moment I'm not calling this Proposed yet, as I don't feel
> comfortable doing so until someone else is on board to finish coding it.
> That said, if someone wants to weigh in on the concept for now (hopefully
> supportively) that's also fine.
>
> Anyone excited enough to help finish the job?
>
> (This is my first RFC collaboration, so if you're going to smack me for
> goofing something please be gentle about it.)
>
> --
>   Larry Garfield
>   la...@garfieldtech.com
>
> --
> PHP Internals - PHP Runtime Development Mailing List
> To unsubscribe, visit: http://www.php.net/unsub.php
>

Hello Larry,

I do like the proposal of adding comprehensions to PHP, and I do agree with
you
that short closures do not replace a short closure synstax (as even with
them it
is still not ideal). However I have a couple of complaints, JS
comprehensions have
not become a standard and have been removed from my understanding [1][2].

Secondly I don't really understand the argument for using `for` instead of `
foreach`
Sure it saves 4 chars but the PHP version compared to the Python uses `yield
`
which already lengthen it, I do understand why it is there because it is a
generator.
But Python using `for` makes more sense, imho, because its syntax is
`expression for item in iterator (if condition)`.
Now maybe what I am going to say is total BS because I don't know *how* the
engine
works, but wouldn't also use foreach make it easier for the lexer as it
already know the
`as` keyword?

Maybe if we want to mimic the Python syntax why not use a syntax like:
`(yield expression) foreach iterator as item (if condition)`

I also suppose that comprehension chaining is possible because it returns a
generator.

Small nitpick but shouldn't the `current` method link to the
Generator->current instead of
the function `current` which only works with arrays?

Also would it be possible to use a string as an iterable expression?
Because currently it's not possible to foreach a string (but that's maybe
more the scope
of another RFC).

Best regards

George P. Banyard

[1]
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Array_comprehensions
[2]
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Generator_comprehensions

Re: [PHP-DEV] RFC Draft: Comprehensions

[Bob Weinand]

> Am 10.03.2019 um 22:44 schrieb Larry Garfield :
> 
> Hello, peoples.  I know it's been discussed once or twice before on the list, 
> many years ago, but not recently.  I therefore feel OK putting forth the 
> following draft proposal for Comprehensions, aka "compact generators", in PHP:
> 
> https://wiki.php.net/rfc/comprehensions
> 
> Sara Golemon has written a preliminary patch that is partially complete (see 
> the RFC link for details, it's impressively simple), but unfortunately 
> doesn't have the bandwidth to complete it at this time.  I am therefore 
> looking for collaborators with more knowledge of internals than I (which is 
> almost everyone) to help finish it up.
> 
> The syntax proposed is also subject to revision if a terser but still 
> lexer-friendly alternative can be proposed.
> 
> At the moment I'm not calling this Proposed yet, as I don't feel comfortable 
> doing so until someone else is on board to finish coding it.  That said, if 
> someone wants to weigh in on the concept for now (hopefully supportively) 
> that's also fine.
> 
> Anyone excited enough to help finish the job?
> 
> (This is my first RFC collaboration, so if you're going to smack me for 
> goofing something please be gentle about it.)
> 
> -- 
>  Larry Garfield
>  la...@garfieldtech.com

Hey Larry,
First, I like it. (the idea/intention behind of providing a compact way for 
list comprehensions)

Though in general, I think this would be better off implemented with a more 
flexible short closure; it feels like a special case of these. I.e. that the 
comprehension syntax can be simply desugared to an immediately invoked short 
closure. At that point also, it can be a compiler only patch, if we have short 
closures with the appropriate binding behavior.
I guess it should be pretty easy to build that on top of my existing short 
closures patch.

Regarding the syntax, I'd not conflate for with foreach, the distinction is 
built into PHP and as such should remain.
We also might have trivial argument-less short closures written as "{ stmt }" 
simply, having the comprehension written as $gen = {foreach($list as $x) if($x 
% 2) yield $x * 2;}(); (note the trailing "()" function call - if we desire to, 
we might make the semicolon at the end of that single-statement closure 
optional.)

Bob
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] RFC Draft: Comprehensions

[Wes]

Hi!

proposed syntax:

> $gen = [for $list as $x if $x % 2 yield $x*2];

>
current php + short closures:
$gen = () => foreach($list as $x) if($x % 2) yield $x * 2;

1- parentheses could be optional around control structures, so `if(true){}`
could be simply `if true{}`
2- "for" could accept the syntax of "foreach"
3- we need short closures

Re: [PHP-DEV] RFC Draft: Comprehensions

[Bruce Weirdan]

> array_map and array_filter combined

This example has array_map and array_filter in wrong order (duplicated once
or twice below as well).

The RFC proposes to allow multiple `for`s in comprehensions, and really
could benefit from an example of such usage.