Re: [PHP-DEV] add Fiber (sackful coroutine) support

[Niklas Keller]

2017-09-09 8:07 GMT+02:00 Haitao Lv :

>
> > On 1 Sep 2017, at 22:25, Niklas Keller  wrote:
> >
> > A potential way around that (might be a stupid idea I just had): Allow
> > defining "wrappers" per file, that auto-wrap marked functions.
>
> Amp need these wrapper functions because we cannot yield a generator from
> its sub function call. So if we introduce fiber, these wrapper is needless.


We could easily support yielding generators and auto-wrapping them in the
coroutine implementation, but not every generator is a coroutine. And
generators are an implementation detail, it doesn't make sense to force
everything to be a generator.


>
> >
> > I don't see an event loop or promises being built-in soon, but something
> > like that could provide `async` with userland magic. Not the best thing,
> > but maybe better than preprocessing, don't know.
>
> Fiber is asymmetric coroutine, and the PHP kernel has no duty to schedule
> them. So a built-in event loop is no needed.


I'm actually thinking about bringing our promise + coroutine implementation
to php-src. `await` could be used to automatically convert generators into
coroutines, which implement promise. It would be mostly syntactic sugar for
the current Amp implementation, because all these wrappers are gone, but
the implementation details of something being a coroutine stays hidden
behind a promise interface.

Regards, Niklas

Re: [PHP-DEV] add Fiber (sackful coroutine) support

[zixu mo]

the best corotine . must be golang go and chan. very easy to use. non blocking. 




On Sat, Sep 9, 2017 at 2:28 PM +0800, "Haitao Lv"  wrote:











> On 2 Sep 2017, at 20:19, Rowan Collins  wrote:
> 
> On 1 September 2017 14:02:29 BST, Haitao Lv  wrote:
>> 
>> Fiber is a lightweight thread. Please see
>> https://en.wikipedia.org/wiki/Fiber_(computer_science)
>> And ruby support Fiber. Please see
>> https://ruby-doc.org/core-2.4.1/Fiber.html
> 
> Ah, thanks, that makes more sense now.
> 
> I note that the examples there all implement it not as a keyword, but as a 
> library function, which maybe makes more sense: whereas "yield" turns a 
> function declaration into a generator declaration, "Fiber\yield", as we might 
> call it, is just a function call which can happen anywhere and manipulates 
> some global state.

As the Zend Engine does not offer API to pause execution, I cannot implement 
the await without
introducing a new keyword (just like yield).

> The choice of "await" also feels odd: you're not awaiting the thing on the 
> right-hand side of the keyword, you're sending it somewhere and awaiting 
> something else.

await is chosen because we cannot reuse the `yield` keyword. Maybe we can 
choose a more proper keyword.

>> You can see the await as a resumable return. The value after it
>> will be returned and the function
>> will be paused.
> 
> Again, this explanation doesn't make sense if you can call foo() without any 
> Fiber code. If there isn't an active Fiber to pause, the "await" line can't 
> "return" anything; it will presumably throw an error of some sort.
> 
> function foo($x) {
> $y = Fiber\yield($x);
> return $x + $y;
> }
> echo foo(1);
> 
> The 1 isn't "returned" to the echo, it's passed off to a global function for 
> further processing. This makes much more sense to me than implying that foo() 
> is no longer a normal function.

await is not yield. It need a separate vm stack to work. So call the foo 
function without a Fiber will
throw a runtime error exception.

> 
>>> Your ->resume() doesn't seem to do anything a normal generator can't,
>> except that the yield is nested deeper.
>> 
>> The await is a nested deeper yield!
> 
> Yes, I understand that, but that seems to be completely unrelated to the API 
> design of having resume() act as start, continue, and final result all rolled 
> into one. As I say, I didn't entirely follow the reasoning for allowing 
> return in generators, but if Fibers are used for the same purpose, I would 
> expect the same concerns to arise.
> 
> However, I think the comparison to generators may be more distracting than 
> useful, as they seem to be very different solutions to the same or related 
> problems.

If fiber were implemented, the generator would be a special fiber.




--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] add Fiber (sackful coroutine) support

[Haitao Lv]

> On 2 Sep 2017, at 20:19, Rowan Collins  wrote:
> 
> On 1 September 2017 14:02:29 BST, Haitao Lv  wrote:
>> 
>> Fiber is a lightweight thread. Please see
>> https://en.wikipedia.org/wiki/Fiber_(computer_science)
>> And ruby support Fiber. Please see
>> https://ruby-doc.org/core-2.4.1/Fiber.html
> 
> Ah, thanks, that makes more sense now.
> 
> I note that the examples there all implement it not as a keyword, but as a 
> library function, which maybe makes more sense: whereas "yield" turns a 
> function declaration into a generator declaration, "Fiber\yield", as we might 
> call it, is just a function call which can happen anywhere and manipulates 
> some global state.

As the Zend Engine does not offer API to pause execution, I cannot implement 
the await without
introducing a new keyword (just like yield).

> The choice of "await" also feels odd: you're not awaiting the thing on the 
> right-hand side of the keyword, you're sending it somewhere and awaiting 
> something else.

await is chosen because we cannot reuse the `yield` keyword. Maybe we can 
choose a more proper keyword.

>> You can see the await as a resumable return. The value after it
>> will be returned and the function
>> will be paused.
> 
> Again, this explanation doesn't make sense if you can call foo() without any 
> Fiber code. If there isn't an active Fiber to pause, the "await" line can't 
> "return" anything; it will presumably throw an error of some sort.
> 
> function foo($x) {
> $y = Fiber\yield($x);
> return $x + $y;
> }
> echo foo(1);
> 
> The 1 isn't "returned" to the echo, it's passed off to a global function for 
> further processing. This makes much more sense to me than implying that foo() 
> is no longer a normal function.

await is not yield. It need a separate vm stack to work. So call the foo 
function without a Fiber will
throw a runtime error exception.

> 
>>> Your ->resume() doesn't seem to do anything a normal generator can't,
>> except that the yield is nested deeper.
>> 
>> The await is a nested deeper yield!
> 
> Yes, I understand that, but that seems to be completely unrelated to the API 
> design of having resume() act as start, continue, and final result all rolled 
> into one. As I say, I didn't entirely follow the reasoning for allowing 
> return in generators, but if Fibers are used for the same purpose, I would 
> expect the same concerns to arise.
> 
> However, I think the comparison to generators may be more distracting than 
> useful, as they seem to be very different solutions to the same or related 
> problems.

If fiber were implemented, the generator would be a special fiber.




--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] add Fiber (sackful coroutine) support

[Haitao Lv]

> On 1 Sep 2017, at 22:25, Niklas Keller  wrote:
> 
> A potential way around that (might be a stupid idea I just had): Allow
> defining "wrappers" per file, that auto-wrap marked functions.

Amp need these wrapper functions because we cannot yield a generator from
its sub function call. So if we introduce fiber, these wrapper is needless.

> 
> I don't see an event loop or promises being built-in soon, but something
> like that could provide `async` with userland magic. Not the best thing,
> but maybe better than preprocessing, don't know.

Fiber is asymmetric coroutine, and the PHP kernel has no duty to schedule 
them. So a built-in event loop is no needed.



-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] add Fiber (sackful coroutine) support

[Rowan Collins]

On 1 September 2017 14:02:29 BST, Haitao Lv  wrote:
>
>Fiber is a lightweight thread. Please see
>https://en.wikipedia.org/wiki/Fiber_(computer_science)
>And ruby support Fiber. Please see
>https://ruby-doc.org/core-2.4.1/Fiber.html

Ah, thanks, that makes more sense now.

I note that the examples there all implement it not as a keyword, but as a 
library function, which maybe makes more sense: whereas "yield" turns a 
function declaration into a generator declaration, "Fiber\yield", as we might 
call it, is just a function call which can happen anywhere and manipulates some 
global state.

The choice of "await" also feels odd: you're not awaiting the thing on the 
right-hand side of the keyword, you're sending it somewhere and awaiting 
something else.


> You can see the await as a resumable return. The value after it
>will be returned and the function
>will be paused.

Again, this explanation doesn't make sense if you can call foo() without any 
Fiber code. If there isn't an active Fiber to pause, the "await" line can't 
"return" anything; it will presumably throw an error of some sort.

function foo($x) {
 $y = Fiber\yield($x);
 return $x + $y;
}
echo foo(1);

The 1 isn't "returned" to the echo, it's passed off to a global function for 
further processing. This makes much more sense to me than implying that foo() 
is no longer a normal function.


>> Your ->resume() doesn't seem to do anything a normal generator can't,
>except that the yield is nested deeper.
>
>The await is a nested deeper yield!

Yes, I understand that, but that seems to be completely unrelated to the API 
design of having resume() act as start, continue, and final result all rolled 
into one. As I say, I didn't entirely follow the reasoning for allowing return 
in generators, but if Fibers are used for the same purpose, I would expect the 
same concerns to arise.

However, I think the comparison to generators may be more distracting than 
useful, as they seem to be very different solutions to the same or related 
problems.

Regards,

-- 
Rowan Collins
[IMSoP]

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] add Fiber (sackful coroutine) support

[Niklas Keller]

2017-09-01 15:02 GMT+02:00 Haitao Lv :

>
> > On 1 Sep 2017, at 20:45, Rowan Collins  wrote:
> >
> > Is this your own invention, or is the name and semantics based on some
> existing language or computer science theory? "Fiber" makes me think of
> strings, rather than coroutines, so maybe I'm missing a key metaphor here.
>
> Fiber is a lightweight thread. Please see
> https://en.wikipedia.orghttps://github.com/amphp/artax/blob/master/lib/HttpTunneler.php/wiki/
> Fiber_(computer_science)
> 
> And ruby support Fiber. Please see https://ruby-doc.org/core-2.4.
> 1/Fiber.html
>
> >
> >>>  >>> function foo($a)
> >>> {
> >>>  $b = await $a + 1;
> >>>  echo $b;
> >>> }
> >
> > Should "await" be "yield" here? If not, what happens if I call foo()
> without being inside a Fiber?
>
> No. A function with a yield will always return a Generator object. A fiber
> with await will return any
> value. You can see the await as a resumable return. The value after it
> will be returned and the function
> will be paused.
>
> >
> > Similarly, if I run bar() and it runs foo(), what result will I get
> outside a Fiber, since bar() has no yield, so is not itself a generator?
> >
> >
> >> So the Fiber API is a little like the Generator API, but is more simple
> >> yet powerful. So there
> >> is no need to distinct $generator->current(), $generator->send(), and
> >> $generator->getReturn().
> >
> > Your ->resume() doesn't seem to do anything a normal generator can't,
> except that the yield is nested deeper.
>
> The await is a nested deeper yield!
>
> > Nor does it seem to replace getReturn().
>
>  $f = new Fiber(function ($a) {
>  return 1;
> });
>
> echo $f->resume(); // will output 1
>
> > Meanwhile, how does this relate to other ways of combining generators,
> such as "yield from"?
>
> You can use Fiber to implement a more simple generator, because fiber can
> be paused/resumed in its deeper
> call.
>
>  function bar()
> {
>   foreach ([5,6] as $i) {
> await $i;
>   }
> }
> function foo()
> {
>   foreach ([3,4] as $i) {
> await $i;
>   }
>   bar();
> }
> $f = new Fiber(function ($a) {
>   await 1;
>   foo();
> });
>
> echo $f->resume(); // will output 1
> echo $f->resume(); // will output 2
> echo $f->resume(); // will output 3
> echo $f->resume(); // will output 4
> echo $f->resume(); // will output 5
> echo $f->resume(); // will output 6


Fibres / Green Threads solve the problem that coroutine- / promise-based
APIs need to be viral. If any subroutine needs to await something without
blocking everything else, every parent also needs to return a promise and
be aware of it.

However, I'm not entirely sure whether they're the better solution.
Currently every `yield` in a coroutine in Amp is an interruption point.
Between two `yield`s, everything is synchronous. If every function call can
now potentially await things, other things might happen (unexpectedly)
concurrently. This would need special care for some things.

The current main pain point I see for libraries like Amp is not the problem
of viral promise-APIs. It's rather the need for the boilerplate of
`Amp\call(function () use ($arg1, $arg2, ...) { ... }` for every coroutine
to return a promise instead of a generator, because generators are an
implementation detail, see e.g.
https://github.com/amphp/artax/blob/63f044eea7a67ac5f703232f6710846125a73fe3/lib/HttpTunneler.php#L21

A potential way around that (might be a stupid idea I just had): Allow
defining "wrappers" per file, that auto-wrap marked functions.

```php

Re: [PHP-DEV] add Fiber (sackful coroutine) support

[Haitao Lv]

> On 1 Sep 2017, at 20:45, Rowan Collins  wrote:
> 
> Is this your own invention, or is the name and semantics based on some 
> existing language or computer science theory? "Fiber" makes me think of 
> strings, rather than coroutines, so maybe I'm missing a key metaphor here.

Fiber is a lightweight thread. Please see 
https://en.wikipedia.org/wiki/Fiber_(computer_science)
And ruby support Fiber. Please see https://ruby-doc.org/core-2.4.1/Fiber.html

> 
>>> >> function foo($a)
>>> {
>>>  $b = await $a + 1;
>>>  echo $b;
>>> }
> 
> Should "await" be "yield" here? If not, what happens if I call foo() without 
> being inside a Fiber?

No. A function with a yield will always return a Generator object. A fiber with 
await will return any
value. You can see the await as a resumable return. The value after it will be 
returned and the function
will be paused.

> 
> Similarly, if I run bar() and it runs foo(), what result will I get outside a 
> Fiber, since bar() has no yield, so is not itself a generator?
> 
> 
>> So the Fiber API is a little like the Generator API, but is more simple
>> yet powerful. So there
>> is no need to distinct $generator->current(), $generator->send(), and
>> $generator->getReturn().
> 
> Your ->resume() doesn't seem to do anything a normal generator can't, except 
> that the yield is nested deeper.

The await is a nested deeper yield!

> Nor does it seem to replace getReturn().

resume(); // will output 1

> Meanwhile, how does this relate to other ways of combining generators, such 
> as "yield from"?

You can use Fiber to implement a more simple generator, because fiber can be 
paused/resumed in its deeper
call.

resume(); // will output 1
echo $f->resume(); // will output 2
echo $f->resume(); // will output 3
echo $f->resume(); // will output 4
echo $f->resume(); // will output 5
echo $f->resume(); // will output 6




--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] add Fiber (sackful coroutine) support

[Rowan Collins]

On 31 August 2017 15:28:36 BST, Haitao Lv  wrote:
>So I propose to introduce the sackful coroutine, aka Fiber, support for
>PHP.

Hi!

Is this your own invention, or is the name and semantics based on some existing 
language or computer science theory? "Fiber" makes me think of strings, rather 
than coroutines, so maybe I'm missing a key metaphor here.


>> > function foo($a)
>> {
>>   $b = await $a + 1;
>>   echo $b;
>> }

Should "await" be "yield" here? If not, what happens if I call foo() without 
being inside a Fiber?

Similarly, if I run bar() and it runs foo(), what result will I get outside a 
Fiber, since bar() has no yield, so is not itself a generator?


>So the Fiber API is a little like the Generator API, but is more simple
>yet powerful. So there
>is no need to distinct $generator->current(), $generator->send(), and
>$generator->getReturn().

I don't really follow how merging the current() and send() methods is related 
to whether the coroutine was paused further into the stack. Your ->resume() 
doesn't seem to do anything a normal generator can't, except that the yield is 
nested deeper. Nor does it seem to replace getReturn() (although I struggled to 
understand the rationale for that addition at the time, others swore it was 
necessary).

Meanwhile, how does this relate to other ways of combining generators, such as 
"yield from"?

I do struggle with coroutines, so I may be missing something obvious here.

Regards,

-- 
Rowan Collins
[IMSoP]

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] add Fiber (sackful coroutine) support

[Haitao Lv]

Hi, All,

The generator has been introduced in PHP 5.5. And we can it pause a function 
execution.
With these feature, we can schedule multi io task in userland without blocking.

More details can be found at [Nikic].

And there is also some asynchronous frameworks like [AMP], which allow 
programer to 
code like blocking, but run asynchronously.

However, like python's generator, lua's coroutine, javascript's async/await, 
php's
generator also is a stackless coroutine implementation. We cannot pause a 
function call
in it's sub function call. This is the reason that frameworks like [AMP] has 
many wrapper,
is hard to understand, and almost impossible to be used to implement real but 
complex system.

So I propose to introduce the sackful coroutine, aka Fiber, support for PHP. 
And the possible
API like this,

>  function foo($a)
> {
>   $b = await $a + 1;
>   echo $b;
> }
> 
> function bar()
> {
>   foo();
> }
> 
> $f = new Fiber(function($a) {
>   bar($a);
>   return 3;
> });
> 
> $c = $f->resume(1);
> // 1 will be passed to lambda, then bar
> // bar call foo, and the **await** in foo paused this execution
> // and make the return value of resume as $a + 1
> // so $c is 2
> 
> $c = $f->resume(3);
> // resume the execution by previous await, and the $b in foo
> // will be assigned a value of 3
> // so foo echo 3 and return and then the lambda return
> // and the resume got a return value of 3. so $c is 3.

So the Fiber API is a little like the Generator API, but is more simple yet 
powerful. So there
is no need to distinct $generator->current(), $generator->send(), and 
$generator->getReturn().

I made a more complex example at [FIREPHP].And I also make a [PR] for comment.

All comment are welcome. Thanks.

[Nikic] 
https://nikic.github.io/2012/12/22/Cooperative-multitasking-using-coroutines-in-PHP.html
[AMP] http://amphp.org/
[FIREPHP] https://github.com/fiberphp/fiber-core/blob/master/app.php
[PR] https://github.com/php/php-src/pull/2723


--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php