[PHP-DEV] [VOTE] Fiber API

[Haitao Lv]

Hello Internals,

The RFC for fiber is now open for a vote. The RFC is available at
https://wiki.php.net/rfc/fiber.

Voting will be open until June 22th, 2018.

Thank you.



-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] [RFC] Fiber support (again)

[Haitao Lv]

Hi, all,

I have updated the RFC https://wiki.php.net/rfc/fiber

changes list:

- introduce the `throw(Exception $exceptin)` API
- record issues discussed

> On Feb 9, 2018, at 08:12, Haitao Lv <i...@lvht.net> wrote:
> 
>> 
>> On Feb 9, 2018, at 06:22, Niklas Keller <m...@kelunik.com> wrote:
>> 
>>> 
>>>> - How do you determine when a fiber has returned? Looking at the source,
>>> it appears Fiber::status() must be used, comparing against constants.
>>> Separate methods similar to Generator would be better IMO. e.g.:
>>> Fiber::alive(), Fiber::suspended(), Fiber::running()
>>> 
>>> Offering methods like Fiber::alive, Fiber::running makes no difference to
>>> check the Fiber::status() return value. This is just a style issue. And as
>>> a language feature,
>>> Fiber only offer the essential API and let other works to the user land.
>> 
>> 
>> The language should offer a sane API, not the absolute minimum required to
>> work for these things.
> 
> The Ruby's Fiber do offer a live? method but does not have a getStatus method.
> The Lua's coroutine only offer a status method.
> 
> So do we really need to offer three additional helper method? Or what is your
> advice about these API?
> 
>> 
>>>> - What about throwing exceptions into a fiber?
>>> 
>>> Currently does not support throw exception into the fiber. User land code
>>> could check
>>> the value of Fiber::yield and throw exception themselves. The Ruby's Fiber
>>> and Lua's
>>> coroutine also does not support such api as well.
>> 
>> 
>> And throw the exception where? That means async code with fibers can't
>> really handle errors?
> 
> Actually you can transfer any thing to Fiber by the resume method. And you can
> check the return value of Fiber::yield to handle error.
> 
> Fiber is designed as a primitive, low level, and lightweight feature. User 
> land
> code seldom not need to use them directly in your normal code.
> So the following is not a big problem,
> 
> $a = Fiber::yield(...);
> if ($a === false) {
>   throw new Exception(...);
> }
> 
> And both the Ruby and Lua does not offer such API as well.
>> 
>>> 
>>>> 
>>>> - Using Fiber::resume() to initialize the fiber and resume feels
>>> awkward. Separate methods again would be better here, perhaps
>>> Fiber::init(...$args) and Fiber::resume($send).
>>> 
>>> All Fiber created with a suspended status. So make resume to do both the
>>> init and resume
>>> do make sense.
>>> 
>> 
>> It does't make sense to me. Reading the example in the README and
>> understanding why the first resume() takes two arguments instead of one
>> took me quite some minutes.
> 
> This Ruby's Fiber and Lua's coroutine using one resume API to init and resume
> the coroutine. I do not think a dedicate is really required.
> 
> The generator cannot be init by it's send method. And if you want to implement
> coroutine feature(without stack) by it, you have to write code
> 
> function run() {
>if ($this->beforeFirstYield) {
>$this->beforeFirstYield = false;
>return $this->coroutine->current();
>} else {
>$retval = $this->coroutine->send($this->sendValue);
>$this->sendValue = null;
>return $retval;  
>}   
> }
> 
> It is verbose.
> 
> See 
> https://nikic.github.io/2012/12/22/Cooperative-multitasking-using-coroutines-in-PHP.html
> 
>> 
>>> Please see Ruby Fiber API https://ruby-doc.org/core-2.2.0/Fiber.html.
>>> 
>>>> 
>>>> - What happens if the sub1() function in the RFC is invoked outside of a
>>> fiber?
>>> 
>>> You will get a Fatal Error like
>>> Fatal error: Uncaught Error: Cannot call Fiber::yield out of Fiber
>>> 
>>>> - I think a keyword here would be beneficial, even if it has a minor BC
>>> impact. Fibers could then be written like generators. `await` or `emit` as
>>> a keyword perhaps? This would be a less verbose API, feel less magical (a
>>> static method call that actually pauses execution feels out of place), and
>>> would allow Fibers to be returned from methods, named functions, etc with
>>> less boilerplate.
>>> 
>>> Wishing this to be accepted by the community in the PHP 7.3, so no keyword
>>> is accepted.
>>> And if the community cannot accept, the Fiber can be still distributed as
>>> a standalone
>>> extension. So we cannot depen

Re: [PHP-DEV] [RFC] Fiber support (again)

[Haitao Lv]

> On Feb 9, 2018, at 06:22, Niklas Keller  wrote:
> 
>> 
>>> - How do you determine when a fiber has returned? Looking at the source,
>> it appears Fiber::status() must be used, comparing against constants.
>> Separate methods similar to Generator would be better IMO. e.g.:
>> Fiber::alive(), Fiber::suspended(), Fiber::running()
>> 
>> Offering methods like Fiber::alive, Fiber::running makes no difference to
>> check the Fiber::status() return value. This is just a style issue. And as
>> a language feature,
>> Fiber only offer the essential API and let other works to the user land.
> 
> 
> The language should offer a sane API, not the absolute minimum required to
> work for these things.

The Ruby's Fiber do offer a live? method but does not have a getStatus method.
The Lua's coroutine only offer a status method.

So do we really need to offer three additional helper method? Or what is your
advice about these API?

> 
>>> - What about throwing exceptions into a fiber?
>> 
>> Currently does not support throw exception into the fiber. User land code
>> could check
>> the value of Fiber::yield and throw exception themselves. The Ruby's Fiber
>> and Lua's
>> coroutine also does not support such api as well.
> 
> 
> And throw the exception where? That means async code with fibers can't
> really handle errors?

Actually you can transfer any thing to Fiber by the resume method. And you can
check the return value of Fiber::yield to handle error.

Fiber is designed as a primitive, low level, and lightweight feature. User land
code seldom not need to use them directly in your normal code.
So the following is not a big problem,

$a = Fiber::yield(...);
if ($a === false) {
   throw new Exception(...);
}

And both the Ruby and Lua does not offer such API as well.
> 
>> 
>>> 
>>> - Using Fiber::resume() to initialize the fiber and resume feels
>> awkward. Separate methods again would be better here, perhaps
>> Fiber::init(...$args) and Fiber::resume($send).
>> 
>> All Fiber created with a suspended status. So make resume to do both the
>> init and resume
>> do make sense.
>> 
> 
> It does't make sense to me. Reading the example in the README and
> understanding why the first resume() takes two arguments instead of one
> took me quite some minutes.

This Ruby's Fiber and Lua's coroutine using one resume API to init and resume
the coroutine. I do not think a dedicate is really required.

The generator cannot be init by it's send method. And if you want to implement
coroutine feature(without stack) by it, you have to write code

function run() {
if ($this->beforeFirstYield) {
$this->beforeFirstYield = false;
return $this->coroutine->current();
} else {
$retval = $this->coroutine->send($this->sendValue);
$this->sendValue = null;
return $retval;  
}   
}

It is verbose.

See 
https://nikic.github.io/2012/12/22/Cooperative-multitasking-using-coroutines-in-PHP.html

> 
>> Please see Ruby Fiber API https://ruby-doc.org/core-2.2.0/Fiber.html.
>> 
>>> 
>>> - What happens if the sub1() function in the RFC is invoked outside of a
>> fiber?
>> 
>> You will get a Fatal Error like
>> Fatal error: Uncaught Error: Cannot call Fiber::yield out of Fiber
>> 
>>> - I think a keyword here would be beneficial, even if it has a minor BC
>> impact. Fibers could then be written like generators. `await` or `emit` as
>> a keyword perhaps? This would be a less verbose API, feel less magical (a
>> static method call that actually pauses execution feels out of place), and
>> would allow Fibers to be returned from methods, named functions, etc with
>> less boilerplate.
>> 
>> Wishing this to be accepted by the community in the PHP 7.3, so no keyword
>> is accepted.
>> And if the community cannot accept, the Fiber can be still distributed as
>> a standalone
>> extension. So we cannot depend on a new keyword.
> 
> 
> Right, if it's a standalone extension it can't use a new keyword, but as a
> language feature it totally can.

In my opinion, using a keyword or call a method is just a coding style problem.
Introducing a new keyword does not offer any benefit by makes a minor BC.

Both Ruby's Fiber and Lua's coroutine does not required a dedicate keyword.

> Looking at the current README, there are two issues that must be completely
> solved IMO before accepting this:
> 
>> Each Fiber has a separate 4k stack. You can use the fiber.stack_size ini
> option to change the default stack size. You can also use the second
> argument of Fiber::__construct to set the stack size on fly.
> 
> Resizing of the stack should happen automatically, just like generators
> resize automatically.

This size is the init stack size. It means when a Fiber created, it will get a
dedicate stack of 4k size. When the fiber use all the stack space, zend vm will
allocate additional space for feature call frame, automatically.

The default size is 4k means that every fiber requires at least 4k memory to use
as 

Re: [PHP-DEV] [RFC] Fiber support (again)

[Haitao Lv]

> On Feb 8, 2018, at 12:18, Aaron Piotrowski <aa...@trowski.com> wrote:
> 
>> 
>> On Feb 7, 2018, at 7:05 PM, Haitao Lv <i...@lvht.net> wrote:
>> 
>> Hi internals,
>> 
>> I propose to introduce the Fiber feature AGAIN.
>> 
>> The main purpose of the RFC is to introducing a lightweight stackful 
>> coroutine support for PHP and make it possible to write non-blocking code in 
>> the blocking style.
>> 
>> In this RFC, no new keyword is needed. So it will not break the PHP 7.3 
>> release.
>> 
>> Please see the RFC https://wiki.php.net/rfc/fiber
>> 
>> Dmitry and I are working on the implementation at 
>> https://github.com/fiberphp/fiber-ext
>> And a series of usage demo can be found at 
>> https://github.com/fiberphp/fiber-demo
>> 
>> Please offer you comments.
>> 
>> Thank you.
>> 
>> ---
>> Haitao Lv
>> 
> 
> 
> Hi Haitao,
> 
> I'm very excited to see this sort of feature coming to PHP.
> 
> A couple of questions and thoughts:
> 
> - How do you determine when a fiber has returned? Looking at the source, it 
> appears Fiber::status() must be used, comparing against constants. Separate 
> methods similar to Generator would be better IMO. e.g.: Fiber::alive(), 
> Fiber::suspended(), Fiber::running()

Offering methods like Fiber::alive, Fiber::running makes no difference to check 
the Fiber::status() return value. This is just a style issue. And as a language 
feature,
Fiber only offer the essential API and let other works to the user land.

> - What about throwing exceptions into a fiber?

Currently does not support throw exception into the fiber. User land code could 
check
the value of Fiber::yield and throw exception themselves. The Ruby's Fiber and 
Lua's
coroutine also does not support such api as well.

> 
> - Using Fiber::resume() to initialize the fiber and resume feels awkward. 
> Separate methods again would be better here, perhaps Fiber::init(...$args) 
> and Fiber::resume($send).

All Fiber created with a suspended status. So make resume to do both the init 
and resume
do make sense.

Please see Ruby Fiber API https://ruby-doc.org/core-2.2.0/Fiber.html.

> 
> - What happens if the sub1() function in the RFC is invoked outside of a 
> fiber?

You will get a Fatal Error like
Fatal error: Uncaught Error: Cannot call Fiber::yield out of Fiber

> - I think a keyword here would be beneficial, even if it has a minor BC 
> impact. Fibers could then be written like generators. `await` or `emit` as a 
> keyword perhaps? This would be a less verbose API, feel less magical (a 
> static method call that actually pauses execution feels out of place), and 
> would allow Fibers to be returned from methods, named functions, etc with 
> less boilerplate.

Wishing this to be accepted by the community in the PHP 7.3, so no keyword is 
accepted.
And if the community cannot accept, the Fiber can be still distributed as a 
standalone
extension. So we cannot depend on a new keyword.

> Thanks to you and Dmitry for working on this!
> 
> Aaron Piotrowski
> --
> PHP Internals - PHP Runtime Development Mailing List
> To unsubscribe, visit: http://www.php.net/unsub.php




--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] [RFC] Fiber support (again)

[Haitao Lv]

Hi internals,

I propose to introduce the Fiber feature AGAIN.

The main purpose of the RFC is to introducing a lightweight stackful coroutine 
support for PHP and make it possible to write non-blocking code in the blocking 
style.

In this RFC, no new keyword is needed. So it will not break the PHP 7.3 release.

Please see the RFC https://wiki.php.net/rfc/fiber

Dmitry and I are working on the implementation at 
https://github.com/fiberphp/fiber-ext
And a series of usage demo can be found at 
https://github.com/fiberphp/fiber-demo

Please offer you comments.

Thank you.

---
Haitao Lv


--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] The master branch breaks the ZEND_USER_OPCODE

[Haitao Lv]

Hi,

I use the following code the register a new ZEND_USER_OPCODE handler


zend_uchar opcode = ZEND_VM_LAST_OPCODE + 1;

while (1) {
if (opcode == 255) {
return FAILURE;
} else if (zend_get_user_opcode_handler(opcode) == NULL) {
break;
}
opcode++;
}
zend_set_user_opcode_handler(opcode, fiber_terminate_opcode_handler);


When I update to the latest master branch and run my php code, it raise a fatal 
error:

Fatal error: Invalid opcode 199/8/8. in  on line 0

Could any one offer some help?

Thank you.


Haitao Lv


--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] RFC - Zend VM Pause API

[Haitao Lv]

After discussion with Dmitry, I update my RFC. The RFC url is here

https://wiki.php.net/rfc/zend-vm-pause-api

And I propose to introduce a new interrupt type to make vm pause-and-return, 
not pause-and-continue.

As mentioned by Dmitry, features like Fiber could be implemented by current 
vm_interrupt implementation, theoretically. By using it, even init a function 
call will need to make our own op array.

So I propose this patch.

Thank you.


> On 1 Nov 2017, at 17:13, Haitao Lv <i...@lvht.net> wrote:
> 
> Would it proper to introduce the following path?
> 
> diff --git a/Zend/zend_vm_def.h b/Zend/zend_vm_def.h
> index 9bed9f86bb..cf0935df74 100644
> --- a/Zend/zend_vm_def.h
> +++ b/Zend/zend_vm_def.h
> @@ -8920,13 +8920,19 @@ ZEND_VM_DEFINE_OP(137, ZEND_OP_DATA);
> 
> ZEND_VM_HELPER(zend_interrupt_helper, ANY, ANY)
> {
> +   int8_t interrupt_type = EG(vm_interrupt);
> +
>EG(vm_interrupt) = 0;
>if (EG(timed_out)) {
>zend_timeout(0);
>} else if (zend_interrupt_function) {
>SAVE_OPLINE();
>zend_interrupt_function(execute_data);
> -   ZEND_VM_ENTER();
> +   if (interrupt_type == 2) {
> +   ZEND_VM_RETURN();
> +   } else {
> +   ZEND_VM_ENTER();
> +   }
>}
>ZEND_VM_CONTINUE();
> }
> 
>> On 1 Nov 2017, at 16:54, Haitao Lv <i...@lvht.net> wrote:
>> 
>> It seems that set EG(vm_interrupt) to 1 could not stop the vm execution but 
>> only execute the interrupt_function and continue the current execution.
>> 
>> However, my RFC propose to stop the current execution.
>> 
>>> On 1 Nov 2017, at 16:07, Dmitry Stogov <dmi...@zend.com> wrote:
>>> 
>>> Hi,
>>> 
>>> It should be possible do similar things using EG(vm_interrupt) and 
>>> zend_interrupt_function() callback (introduced in php-7.1)
>>> ext/pcntl implements asynchronous signal handling using this.
>>> 
>>> Thanks. Dmitry.
>>> From: Haitao Lv <i...@lvht.net>
>>> Sent: Wednesday, November 1, 2017 4:19:07 AM
>>> To: PHP Internals
>>> Subject: [PHP-DEV] RFC - Zend VM Pause API 
>>> 
>>> Hi, internals,
>>> 
>>> I propose to introduce a new zend vm pause api, and here is the RPF
>>> 
>>> https://wiki.php.net/rfc/zend-vm-pause-api 
>>> <https://wiki.php.net/rfc/zend-vm-pause-api>
>>> 
>>> Please gave your comment.
>>> 
>>> Thank you.
>>> 
>>> 
>>> 
>>> -- 
>>> PHP Internals - PHP Runtime Development Mailing List
>>> To unsubscribe, visit: http://www.php.net/unsub.php 
>>> <http://www.php.net/unsub.php>
> 
> 
> 
> 
> --
> PHP Internals - PHP Runtime Development Mailing List
> To unsubscribe, visit: http://www.php.net/unsub.php




--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] RFC - Zend VM Pause API

[Haitao Lv]

To make the discussion more detailed, please allow me to offer my 
implementation of use land coroutine(Fiber).

https://github.com/php/php-src/compare/master...fiberphp:fiber-ext?expand=1


> On 1 Nov 2017, at 17:32, Haitao Lv <i...@lvht.net> wrote:
> 
> Suppose we have a internal Coroutine class and it has a resume() method.
> 
> In order to resume the coroutine, we have to call the resume() function.
> 
> As the resume function is a internal method defined in c, we need call
> 
> zend_execute_ex(backuped_execute_data)
> 
> to resume the zend execution.
> 
> If we need to pause the coroutine, we set the EG(vm_interrupt) and 
> interrupt_function,
> and switch the execute data and stack. The zend vm will execute the old 
> online.
> 
> However, we will never see the resume() method returned and never get its 
> return value.
> 
>> On 1 Nov 2017, at 17:14, Dmitry Stogov <dmi...@zend.com> wrote:
>> 
>> after zend_interrupt_function() callback VM continues execution using 
>> EG(current_execute_data).
>> callback may modify it in any way (e.g. unwind stack, or switch to another 
>> co-routine or continuation).
>> 
>> Thanks. Dmitry.
>> From: Haitao Lv <i...@lvht.net>
>> Sent: Wednesday, November 1, 2017 11:54:54 AM
>> To: Dmitry Stogov
>> Cc: PHP Internals
>> Subject: Re: [PHP-DEV] RFC - Zend VM Pause API 
>> 
>> It seems that set EG(vm_interrupt) to 1 could not stop the vm execution but 
>> only execute the interrupt_function and continue the current execution.
>> 
>> However, my RFC propose to stop the current execution.
>> 
>>> On 1 Nov 2017, at 16:07, Dmitry Stogov <dmi...@zend.com> wrote:
>>> 
>>> Hi,
>>> 
>>> It should be possible do similar things using EG(vm_interrupt) and 
>>> zend_interrupt_function() callback (introduced in php-7.1)
>>> ext/pcntl implements asynchronous signal handling using this.
>>> 
>>> Thanks. Dmitry.
>>> From: Haitao Lv <i...@lvht.net>
>>> Sent: Wednesday, November 1, 2017 4:19:07 AM
>>> To: PHP Internals
>>> Subject: [PHP-DEV] RFC - Zend VM Pause API 
>>> 
>>> Hi, internals,
>>> 
>>> I propose to introduce a new zend vm pause api, and here is the RPF
>>> 
>>> https://wiki.php.net/rfc/zend-vm-pause-api
>>> 
>>> Please gave your comment.
>>> 
>>> Thank you.
>>> 
>>> 
>>> 
>>> -- 
>>> PHP Internals - PHP Runtime Development Mailing List
>>> To unsubscribe, visit: http://www.php.net/unsub.php
>> 
>> 
> 
> 
> 
> 
> --
> PHP Internals - PHP Runtime Development Mailing List
> To unsubscribe, visit: http://www.php.net/unsub.php
> 




--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] RFC - Zend VM Pause API

[Haitao Lv]

Suppose we have a internal Coroutine class and it has a resume() method.

In order to resume the coroutine, we have to call the resume() function.

As the resume function is a internal method defined in c, we need call

zend_execute_ex(backuped_execute_data)

to resume the zend execution.

If we need to pause the coroutine, we set the EG(vm_interrupt) and 
interrupt_function,
and switch the execute data and stack. The zend vm will execute the old online.

However, we will never see the resume() method returned and never get its 
return value.

> On 1 Nov 2017, at 17:14, Dmitry Stogov <dmi...@zend.com> wrote:
> 
> after zend_interrupt_function() callback VM continues execution using 
> EG(current_execute_data).
> callback may modify it in any way (e.g. unwind stack, or switch to another 
> co-routine or continuation).
> 
> Thanks. Dmitry.
> From: Haitao Lv <i...@lvht.net>
> Sent: Wednesday, November 1, 2017 11:54:54 AM
> To: Dmitry Stogov
> Cc: PHP Internals
> Subject: Re: [PHP-DEV] RFC - Zend VM Pause API 
>  
> It seems that set EG(vm_interrupt) to 1 could not stop the vm execution but 
> only execute the interrupt_function and continue the current execution.
> 
> However, my RFC propose to stop the current execution.
> 
>> On 1 Nov 2017, at 16:07, Dmitry Stogov <dmi...@zend.com> wrote:
>> 
>> Hi,
>> 
>> It should be possible do similar things using EG(vm_interrupt) and 
>> zend_interrupt_function() callback (introduced in php-7.1)
>> ext/pcntl implements asynchronous signal handling using this.
>> 
>> Thanks. Dmitry.
>> From: Haitao Lv <i...@lvht.net>
>> Sent: Wednesday, November 1, 2017 4:19:07 AM
>> To: PHP Internals
>> Subject: [PHP-DEV] RFC - Zend VM Pause API 
>>  
>> Hi, internals,
>> 
>> I propose to introduce a new zend vm pause api, and here is the RPF
>> 
>> https://wiki.php.net/rfc/zend-vm-pause-api
>> 
>> Please gave your comment.
>> 
>> Thank you.
>> 
>> 
>> 
>> -- 
>> PHP Internals - PHP Runtime Development Mailing List
>> To unsubscribe, visit: http://www.php.net/unsub.php
> 
> 




--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] RFC - Zend VM Pause API

[Haitao Lv]

Would it proper to introduce the following path?

diff --git a/Zend/zend_vm_def.h b/Zend/zend_vm_def.h
index 9bed9f86bb..cf0935df74 100644
--- a/Zend/zend_vm_def.h
+++ b/Zend/zend_vm_def.h
@@ -8920,13 +8920,19 @@ ZEND_VM_DEFINE_OP(137, ZEND_OP_DATA);

 ZEND_VM_HELPER(zend_interrupt_helper, ANY, ANY)
 {
+   int8_t interrupt_type = EG(vm_interrupt);
+
EG(vm_interrupt) = 0;
if (EG(timed_out)) {
zend_timeout(0);
} else if (zend_interrupt_function) {
SAVE_OPLINE();
zend_interrupt_function(execute_data);
-   ZEND_VM_ENTER();
+   if (interrupt_type == 2) {
+   ZEND_VM_RETURN();
+   } else {
+   ZEND_VM_ENTER();
+   }
}
ZEND_VM_CONTINUE();
 }

> On 1 Nov 2017, at 16:54, Haitao Lv <i...@lvht.net> wrote:
> 
> It seems that set EG(vm_interrupt) to 1 could not stop the vm execution but 
> only execute the interrupt_function and continue the current execution.
> 
> However, my RFC propose to stop the current execution.
> 
>> On 1 Nov 2017, at 16:07, Dmitry Stogov <dmi...@zend.com> wrote:
>> 
>> Hi,
>> 
>> It should be possible do similar things using EG(vm_interrupt) and 
>> zend_interrupt_function() callback (introduced in php-7.1)
>> ext/pcntl implements asynchronous signal handling using this.
>> 
>> Thanks. Dmitry.
>> From: Haitao Lv <i...@lvht.net>
>> Sent: Wednesday, November 1, 2017 4:19:07 AM
>> To: PHP Internals
>> Subject: [PHP-DEV] RFC - Zend VM Pause API 
>> 
>> Hi, internals,
>> 
>> I propose to introduce a new zend vm pause api, and here is the RPF
>> 
>> https://wiki.php.net/rfc/zend-vm-pause-api 
>> <https://wiki.php.net/rfc/zend-vm-pause-api>
>> 
>> Please gave your comment.
>> 
>> Thank you.
>> 
>> 
>> 
>> -- 
>> PHP Internals - PHP Runtime Development Mailing List
>> To unsubscribe, visit: http://www.php.net/unsub.php 
>> <http://www.php.net/unsub.php>




--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] RFC - Zend VM Pause API

[Haitao Lv]

It seems that set EG(vm_interrupt) to 1 could not stop the vm execution but 
only execute the interrupt_function and continue the current execution.

However, my RFC propose to stop the current execution.

> On 1 Nov 2017, at 16:07, Dmitry Stogov <dmi...@zend.com> wrote:
> 
> Hi,
> 
> It should be possible do similar things using EG(vm_interrupt) and 
> zend_interrupt_function() callback (introduced in php-7.1)
> ext/pcntl implements asynchronous signal handling using this.
> 
> Thanks. Dmitry.
> From: Haitao Lv <i...@lvht.net>
> Sent: Wednesday, November 1, 2017 4:19:07 AM
> To: PHP Internals
> Subject: [PHP-DEV] RFC - Zend VM Pause API 
>  
> Hi, internals,
> 
> I propose to introduce a new zend vm pause api, and here is the RPF
> 
> https://wiki.php.net/rfc/zend-vm-pause-api 
> <https://wiki.php.net/rfc/zend-vm-pause-api>
> 
> Please gave your comment.
> 
> Thank you.
> 
> 
> 
> -- 
> PHP Internals - PHP Runtime Development Mailing List
> To unsubscribe, visit: http://www.php.net/unsub.php 
> <http://www.php.net/unsub.php>

[PHP-DEV] RFC - Zend VM Pause API

[Haitao Lv]

Hi, internals,

I propose to introduce a new zend vm pause api, and here is the RPF

https://wiki.php.net/rfc/zend-vm-pause-api

Please gave your comment.

Thank you.



-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] Wiki karma request

[Haitao Lv]

Username: lvht

Access requested: Wiki create

Reason: Create RFC for feature of Fiber (sackful coroutine) support.

Thanks.

Haitao Lv



-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] Don't add simple objects to GC's roots

[Haitao Lv]

> On 30 Jul 2017, at 18:19, Andreas Treichel  wrote:
> 
> Hi,
> 
>> So I propose to make the gc_disable function accept one zval reference as
>> parameter. And if gc_disable get that zval, gc_disable just drop the zval’s
>> GC_COLLECTABLE flag, which will hint the PHP gc not to trace that zval.
> 
> i dont know if this is a good idea or not. But for the "s" in solid, create a 
> new function like gc_exclude to exclude variables from the garbage collector 
> and do not add a parameter to gc_disable.

A new function gc_hint($zval, bool $collectable) has been introduced at 

https://github.com/php/php-src/pull/2665/files
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] add Fiber (sackful coroutine) support

[Haitao Lv]

> On 2 Sep 2017, at 20:19, Rowan Collins <rowan.coll...@gmail.com> wrote:
> 
> On 1 September 2017 14:02:29 BST, Haitao Lv <i...@lvht.net> wrote:
>> 
>> Fiber is a lightweight thread. Please see
>> https://en.wikipedia.org/wiki/Fiber_(computer_science)
>> And ruby support Fiber. Please see
>> https://ruby-doc.org/core-2.4.1/Fiber.html
> 
> Ah, thanks, that makes more sense now.
> 
> I note that the examples there all implement it not as a keyword, but as a 
> library function, which maybe makes more sense: whereas "yield" turns a 
> function declaration into a generator declaration, "Fiber\yield", as we might 
> call it, is just a function call which can happen anywhere and manipulates 
> some global state.

As the Zend Engine does not offer API to pause execution, I cannot implement 
the await without
introducing a new keyword (just like yield).

> The choice of "await" also feels odd: you're not awaiting the thing on the 
> right-hand side of the keyword, you're sending it somewhere and awaiting 
> something else.

await is chosen because we cannot reuse the `yield` keyword. Maybe we can 
choose a more proper keyword.

>> You can see the await as a resumable return. The value after it
>> will be returned and the function
>> will be paused.
> 
> Again, this explanation doesn't make sense if you can call foo() without any 
> Fiber code. If there isn't an active Fiber to pause, the "await" line can't 
> "return" anything; it will presumably throw an error of some sort.
> 
> function foo($x) {
> $y = Fiber\yield($x);
> return $x + $y;
> }
> echo foo(1);
> 
> The 1 isn't "returned" to the echo, it's passed off to a global function for 
> further processing. This makes much more sense to me than implying that foo() 
> is no longer a normal function.

await is not yield. It need a separate vm stack to work. So call the foo 
function without a Fiber will
throw a runtime error exception.

> 
>>> Your ->resume() doesn't seem to do anything a normal generator can't,
>> except that the yield is nested deeper.
>> 
>> The await is a nested deeper yield!
> 
> Yes, I understand that, but that seems to be completely unrelated to the API 
> design of having resume() act as start, continue, and final result all rolled 
> into one. As I say, I didn't entirely follow the reasoning for allowing 
> return in generators, but if Fibers are used for the same purpose, I would 
> expect the same concerns to arise.
> 
> However, I think the comparison to generators may be more distracting than 
> useful, as they seem to be very different solutions to the same or related 
> problems.

If fiber were implemented, the generator would be a special fiber.




--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] add Fiber (sackful coroutine) support

[Haitao Lv]

> On 1 Sep 2017, at 22:25, Niklas Keller  wrote:
> 
> A potential way around that (might be a stupid idea I just had): Allow
> defining "wrappers" per file, that auto-wrap marked functions.

Amp need these wrapper functions because we cannot yield a generator from
its sub function call. So if we introduce fiber, these wrapper is needless.

> 
> I don't see an event loop or promises being built-in soon, but something
> like that could provide `async` with userland magic. Not the best thing,
> but maybe better than preprocessing, don't know.

Fiber is asymmetric coroutine, and the PHP kernel has no duty to schedule 
them. So a built-in event loop is no needed.



-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] add Fiber (sackful coroutine) support

[Haitao Lv]

> On 1 Sep 2017, at 20:45, Rowan Collins  wrote:
> 
> Is this your own invention, or is the name and semantics based on some 
> existing language or computer science theory? "Fiber" makes me think of 
> strings, rather than coroutines, so maybe I'm missing a key metaphor here.

Fiber is a lightweight thread. Please see 
https://en.wikipedia.org/wiki/Fiber_(computer_science)
And ruby support Fiber. Please see https://ruby-doc.org/core-2.4.1/Fiber.html

> 
>>> >> function foo($a)
>>> {
>>>  $b = await $a + 1;
>>>  echo $b;
>>> }
> 
> Should "await" be "yield" here? If not, what happens if I call foo() without 
> being inside a Fiber?

No. A function with a yield will always return a Generator object. A fiber with 
await will return any
value. You can see the await as a resumable return. The value after it will be 
returned and the function
will be paused.

> 
> Similarly, if I run bar() and it runs foo(), what result will I get outside a 
> Fiber, since bar() has no yield, so is not itself a generator?
> 
> 
>> So the Fiber API is a little like the Generator API, but is more simple
>> yet powerful. So there
>> is no need to distinct $generator->current(), $generator->send(), and
>> $generator->getReturn().
> 
> Your ->resume() doesn't seem to do anything a normal generator can't, except 
> that the yield is nested deeper.

The await is a nested deeper yield!

> Nor does it seem to replace getReturn().

resume(); // will output 1

> Meanwhile, how does this relate to other ways of combining generators, such 
> as "yield from"?

You can use Fiber to implement a more simple generator, because fiber can be 
paused/resumed in its deeper
call.

resume(); // will output 1
echo $f->resume(); // will output 2
echo $f->resume(); // will output 3
echo $f->resume(); // will output 4
echo $f->resume(); // will output 5
echo $f->resume(); // will output 6




--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] add Fiber (sackful coroutine) support

[Haitao Lv]

Hi, All,

The generator has been introduced in PHP 5.5. And we can it pause a function 
execution.
With these feature, we can schedule multi io task in userland without blocking.

More details can be found at [Nikic].

And there is also some asynchronous frameworks like [AMP], which allow 
programer to 
code like blocking, but run asynchronously.

However, like python's generator, lua's coroutine, javascript's async/await, 
php's
generator also is a stackless coroutine implementation. We cannot pause a 
function call
in it's sub function call. This is the reason that frameworks like [AMP] has 
many wrapper,
is hard to understand, and almost impossible to be used to implement real but 
complex system.

So I propose to introduce the sackful coroutine, aka Fiber, support for PHP. 
And the possible
API like this,

>  function foo($a)
> {
>   $b = await $a + 1;
>   echo $b;
> }
> 
> function bar()
> {
>   foo();
> }
> 
> $f = new Fiber(function($a) {
>   bar($a);
>   return 3;
> });
> 
> $c = $f->resume(1);
> // 1 will be passed to lambda, then bar
> // bar call foo, and the **await** in foo paused this execution
> // and make the return value of resume as $a + 1
> // so $c is 2
> 
> $c = $f->resume(3);
> // resume the execution by previous await, and the $b in foo
> // will be assigned a value of 3
> // so foo echo 3 and return and then the lambda return
> // and the resume got a return value of 3. so $c is 3.

So the Fiber API is a little like the Generator API, but is more simple yet 
powerful. So there
is no need to distinct $generator->current(), $generator->send(), and 
$generator->getReturn().

I made a more complex example at [FIREPHP].And I also make a [PR] for comment.

All comment are welcome. Thanks.

[Nikic] 
https://nikic.github.io/2012/12/22/Cooperative-multitasking-using-coroutines-in-PHP.html
[AMP] http://amphp.org/
[FIREPHP] https://github.com/fiberphp/fiber-core/blob/master/app.php
[PR] https://github.com/php/php-src/pull/2723


--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] Don't add simple objects to GC's roots

[Haitao Lv]

Hi, internals,

> On 26 Jul 2017, at 22:57, Nicolas Grekas  wrote:
> 
> I'm hitting more and more the GC threshold in projects I work on (the "10k
> roots" one), leading to high CPU usage. Usually, the GC finds nothing to
> clean, so this is just a waste of CPU.

More and more php frameworks use object/array as an global container to store
config, and these container variable will be used every where, but seldom
introduce the circular reference. Apart from this, these container variable
will always be a huge tree, so iterate them will lead to high CPU usage.

> There is a way to work around: just call gc_disable(). But this means
> dealing with side effects of the engine in userland. Moreover, this also
> means leaking memory, since meanwhile roots are not populated.

In PHP-7.2, the GC_COLLECTABLE flag has been moved into the gc struct, and
this make it possible to prevent gc trace for certain zval.

So I propose to make the gc_disable function accept one zval reference as
parameter. And if gc_disable get that zval, gc_disable just drop the zval’s
GC_COLLECTABLE flag, which will hint the PHP gc not to trace that zval.

Here is my patch https://github.com/php/php-src/pull/2665/files, and a demo,

name;
}

$a = new stdclass;
$a->name = 'foo';

gc_disable($a);

hi($a); // these will change the refcount of $a, but never trace gc


--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php