Re: [PHP-DEV] New "PECL"
On Mon, Dec 11, 2023, 11:21 PM Larry Garfield wrote: > On Mon, Dec 11, 2023, at 9:46 AM, Derick Rethans wrote: > > > Yeah, I understand. And I feel the same. But if there is no registry (a > > goal), then there is no way to find out which of your extension version > > support a specific PHP version, without checking every tag in your > > repository. That's not going to be fast. > > > > cheers, > > Derick > > See, I think I disagree on this point. Having a registry is table-stakes > for package managers these days. Composer, npm/yarn, whatever Python does > this week, Cargo, Bundle, Maven/Gradle... a packager isn't complete without > a registry. The one exception is Go, and its approach is... just a mess, > frankly. > > Whether PECLtng piggybacks on Packagist, self-hosts using something like > Satis, or does a home-grown thing, I don't think a neo-PECL without a > registry is a viable and sustainable way forward. That needs to be in the > initial design. > hi Larry, you are fully right here. On top of the already mentioned points, it brings visibility to extension packages, which is a key factor to adoption at large. It also helps for an automatic builds. A service is also a must for automatic releases/triggers builds. You can see this service with a github tag hook on pickleweb. It is also quite a good thing to automate dependencies management. It could be relatively straightforward to update them per distros, or windows (see my earlier post, already managed using windows. php.net data). Also without the services (implemented but no public packagist for extensions yet), I think pickle already does 95% of it (including local windows builds). Adding signature is relatively simple. Important note, during pickle development, the number 1 request was to be fully integrated with composer. it was not possible until composer v2, plugin and custom tags. Given that composer is the only used package manager , it should be the way to go. To create something from scratch without taking these points into accounts can only lead to a sub optimal solution. There are also development environments tools out there which could provide relatively easily per project extensions management (either cli or pools). PHP is not my primary platform anymore, business wise, however I am happy to help if desired. I can only, with a diplomatic emphasis, suggest to the author to understand where the php ecosystem is (or was 3 years or more ago). best, Pierre >
Re: [PHP-DEV] New "PECL"
On Mon, Dec 11, 2023, at 9:46 AM, Derick Rethans wrote: > Yeah, I understand. And I feel the same. But if there is no registry (a > goal), then there is no way to find out which of your extension version > support a specific PHP version, without checking every tag in your > repository. That's not going to be fast. > > cheers, > Derick See, I think I disagree on this point. Having a registry is table-stakes for package managers these days. Composer, npm/yarn, whatever Python does this week, Cargo, Bundle, Maven/Gradle... a packager isn't complete without a registry. The one exception is Go, and its approach is... just a mess, frankly. Whether PECLtng piggybacks on Packagist, self-hosts using something like Satis, or does a home-grown thing, I don't think a neo-PECL without a registry is a viable and sustainable way forward. That needs to be in the initial design. --Larry Garfield -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] New "PECL"
On Mon, Dec 11, 2023 at 10:59 AM Derick Rethans wrote: > > On Thu, 7 Dec 2023, Remi Collet wrote: > > > Le 05/12/2023 à 14:45, Derick Rethans a écrit : > > > https://docs.google.com/document/d/1_N0E9xo3jn9aKrIZHIbTYaY5lXw71BpSO6-it4cRpDo > > > > > > In this first stage, we would like to invite you in commenting on the > > > document (either inline, or here). > > > > As inline requires a Google account... > > > > > priority (optional): The installation "priority". > > > > 1/ I don't like this name which is ambiguous > > > > higher value seems to give higher priority > > which lower are loaded first > > > > This prefix, used in various linux distro, > > is really a "load order", needed when using --enable-rtld-now > > > > So I proprose "load order" instead > > We can quibble over the name :-) > > I picked something short. Debian calls it > 'priority' too in their ini files. For example: > > ; configuration for php common module > ; priority=20 > extension=calendar.so > > > 2/ values > > > > Default value can be 40 or max(requirement) + 10 > > > > In Fedora where we use this for years we have > > (IIRC debian have different practice) > > > > 0-19 for zend_extension > > 10 for opcache > > 15 for xdebug > > That's not really a good default. At least Xdebug should be > near the end of the loading order. > > > 20-39 for php-src extension > > 20 for most > > 30 if requirement (ex: pdo drivers) > > 40-99 for other (pecl...) > > 40 for igbinary, msgpack > > 50 for redis which use igbinary, msgpack > > I am happy to pick a different default. I had picked what Debian seemed > to have (20). > > I don't think it matters much though, as distribitions can decide to > change them in their binary packages regardless. > > cheers, > Derick > > -- > PHP Internals - PHP Runtime Development Mailing List > To unsubscribe, visit: https://www.php.net/unsub.php Hello, I don't know if you've seen this prior art: https://github.com/mlocati/docker-php-extension-installer It's extremely convenient for Docker images, and usually has the ability to apply appropriate patches (such as the memcached + PHP 8.1-ish issue a couple of years ago) to get things to install. It would be amazing if the new pecl could be as easy to use as this wrapper/tool. Robert Landers Software Engineer Utrecht NL -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] New "PECL"
On Thu, 7 Dec 2023, Remi Collet wrote: > Le 05/12/2023 à 14:45, Derick Rethans a écrit : > > https://docs.google.com/document/d/1_N0E9xo3jn9aKrIZHIbTYaY5lXw71BpSO6-it4cRpDo > > > > In this first stage, we would like to invite you in commenting on the > > document (either inline, or here). > > As inline requires a Google account... > > > priority (optional): The installation "priority". > > 1/ I don't like this name which is ambiguous > > higher value seems to give higher priority > which lower are loaded first > > This prefix, used in various linux distro, > is really a "load order", needed when using --enable-rtld-now > > So I proprose "load order" instead We can quibble over the name :-) I picked something short. Debian calls it 'priority' too in their ini files. For example: ; configuration for php common module ; priority=20 extension=calendar.so > 2/ values > > Default value can be 40 or max(requirement) + 10 > > In Fedora where we use this for years we have > (IIRC debian have different practice) > > 0-19 for zend_extension > 10 for opcache > 15 for xdebug That's not really a good default. At least Xdebug should be near the end of the loading order. > 20-39 for php-src extension > 20 for most > 30 if requirement (ex: pdo drivers) > 40-99 for other (pecl...) > 40 for igbinary, msgpack > 50 for redis which use igbinary, msgpack I am happy to pick a different default. I had picked what Debian seemed to have (20). I don't think it matters much though, as distribitions can decide to change them in their binary packages regardless. cheers, Derick -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] New "PECL"
On Wed, 6 Dec 2023, Tim Starling wrote: > On 6/12/23 00:45, Derick Rethans wrote: > > The code is old, and hard to maintain. And the database is full of > > mojibake. It is also an outdated method of installing things, > > especially because userland code is so much easier to handle through > > Composer. > > Thanks for working on this. > > I would say that one of the reasons userland code is easier is because > composer packages are local and unprivileged, whereas extensions are > global and are typically require root to install. That problem is not > going to go away. For community-supplied packages we're mostly using > Ondrej Sury's Debian packages. For our own packages (LuaSandbox, > Excimer) we're building debs from git tags without reference to PECL. > So PECL for us is part of the release procedure just as a courtesy to > the downstream community. >From what I can see, once you drop a "new-pecl.json" file (or whatever we'll end up calling it) into the repo at https://github.com/wikimedia/mediawiki-php-luasandbox it should automatically become compatible with the new PECL. You shouldn't have to deal with making actual packages and uploading them to the PECL website. I'm going to hope that that is an improvement for you? > In terms of security, there should be no way to get root by uploading > a malicious PECL package. With debs, I'm not sure if that separation > is rock-solid, but at least it exists aspirationally. During build, we > run code from the package as non-root. During installation, as root we > install generated files into locations which hopefully won't be > executed as root. To achieve this separation with PECL, there would > need to be some sort of system-aware privilege separation inside the > PECL installer. At least it should fork and drop root privileges. Yes, that is a good recommendation, and something that should definitely go into the design document. > Signed tags only help if you trust the package developer. I don't want > to have to trust anyone. That's true. I think signatures make more sense when a package is authored by a known organisation, such as wikimedia or MongoDB. Where the signatures therefore include these organisation's domain names. > The PHP version dependency is a tricky part of the spec. When we make > a release, we have no way to know how forwards-compatible it is. At > some point, a PHP core change will break the extension. The proposed > structure suggests that we would have to release a new version of the > extension just to say that it's broken. Then later when we make a fix, > we would release again with that information. I don't think you'll have to make a new *release*, as I was hoping to access the meta data file in the main/master branch. I solve this for Xdebug by always setting a hard upper limit. Better safe than sorry. > My motivation to maintain a historical list of supported version > ranges is very limited. I just want to declare, for the code I'm > actually releasing, the earliest supported PHP version and maybe the > latest confirmed PHP version. > > Have you considered keeping the support matrix in the registry > database, instead of in pecl.json? Then it can be updated with new > build/test information after release. Yeah, I understand. And I feel the same. But if there is no registry (a goal), then there is no way to find out which of your extension version support a specific PHP version, without checking every tag in your repository. That's not going to be fast. cheers, Derick -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] New "PECL"
Hello Tim, On Wed, Dec 6, 2023 at 9:05 AM Tim Starling wrote: > Have you considered keeping the support matrix in the registry > database, instead of in pecl.json? Then it can be updated with new > build/test information after release. We do it for windows, used by pickle, you can see one part here: https://windows.php.net/downloads/php-sdk/deps/dllmapping.json The other parts being part of each extension. It could be achieved similarly for linux package managers, maybe maintained by them directly. A few years back, I made some tests on windows and linux using https://conan.io/ and https://vcpkg.io/, they both work on most platforms PHP supports and also have cmake integration (php build using cmake has been worked done, very nicely :). Having maintained these parts for many years, dealt with 3rd parties (developers of librairies, distro packages maintainers, etc) as well as dealt with many end users (php developers), I must say I would absolutely not start to define our own things but rely as much as possible on the well working existing solutions. On the same topic, npecl's requirements (why not a RFC/proposal btw?) are kind of out of sync with what I have seen and still see. F.e., doing it outside the composer ecosystem will most likely create a niche and not a wide adoption.There have been a lot of efforts put into similar topics since 5.3 and a lot of learning, it may be a good thing to learn from them. There are also tools now available to give developers out of box tools to define whatever environments they may need and switch between them within a click or command line (Laravel community created a few good ones, mac for the biggest one, and other platforms), having them rely on php's platform for the core and non core extensions (including their respective deps) will be amazingly helpful. Cheers, -- Pierre @pierrejoye | http://www.libgd.org -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] New "PECL"
Le 05/12/2023 à 14:45, Derick Rethans a écrit : https://docs.google.com/document/d/1_N0E9xo3jn9aKrIZHIbTYaY5lXw71BpSO6-it4cRpDo In this first stage, we would like to invite you in commenting on the document (either inline, or here). As inline requires a Google account... > priority (optional): The installation "priority". 1/ I don't like this name which is ambiguous higher value seems to give higher priority which lower are loaded first This prefix, used in various linux distro, is really a "load order", needed when using --enable-rtld-now So I proprose "load order" instead 2/ values Default value can be 40 or max(requirement) + 10 In Fedora where we use this for years we have (IIRC debian have different practice) 0-19 for zend_extension 10 for opcache 15 for xdebug 20-39 for php-src extension 20 for most 30 if requirement (ex: pdo drivers) 40-99 for other (pecl...) 40 for igbinary, msgpack 50 for redis which use igbinary, msgpack Remi -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] New "PECL"
On 6/12/23 00:45, Derick Rethans wrote: The code is old, and hard to maintain. And the database is full of mojibake. It is also an outdated method of installing things, especially because userland code is so much easier to handle through Composer. Thanks for working on this. I would say that one of the reasons userland code is easier is because composer packages are local and unprivileged, whereas extensions are global and are typically require root to install. That problem is not going to go away. For community-supplied packages we're mostly using Ondrej Sury's Debian packages. For our own packages (LuaSandbox, Excimer) we're building debs from git tags without reference to PECL. So PECL for us is part of the release procedure just as a courtesy to the downstream community. In terms of security, there should be no way to get root by uploading a malicious PECL package. With debs, I'm not sure if that separation is rock-solid, but at least it exists aspirationally. During build, we run code from the package as non-root. During installation, as root we install generated files into locations which hopefully won't be executed as root. To achieve this separation with PECL, there would need to be some sort of system-aware privilege separation inside the PECL installer. At least it should fork and drop root privileges. Signed tags only help if you trust the package developer. I don't want to have to trust anyone. The PHP version dependency is a tricky part of the spec. When we make a release, we have no way to know how forwards-compatible it is. At some point, a PHP core change will break the extension. The proposed structure suggests that we would have to release a new version of the extension just to say that it's broken. Then later when we make a fix, we would release again with that information. My motivation to maintain a historical list of supported version ranges is very limited. I just want to declare, for the code I'm actually releasing, the earliest supported PHP version and maybe the latest confirmed PHP version. Have you considered keeping the support matrix in the registry database, instead of in pecl.json? Then it can be updated with new build/test information after release. -- Tim Starling -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] New "PECL"
On Tue, Dec 5, 2023 at 3:25 PM Paul Dragoonis wrote: > > > On Tue, Dec 5, 2023 at 3:21 PM Pierre Joye wrote: > >> On Tue, Dec 5, 2023, 10:13 PM Pierre Joye wrote: >> >> > >> > >> > You may mention https://github.com/FriendsOfPHP/pickle >> > >> > which supports pecl.php.net pear format, composer like support, and >> > conversion to composer format. >> > >> > I did a website as well as a test bed and we had a long with the >> composer >> > team what would be the best way to integrate it. That's something that >> > became possible with composer 2.x, which now widely used. Similarly we >> > discussed the idea to create a packagist dedicated to extensions. >> > >> > That would be, imo, the best option to date. >> > >> > >> > The current code may need some updates for php8 but nothing difficult or >> > requiring big changes. >> > >> >> there was also a draft rfc while we were thinking to propose to bundle it. >> But we then decided to follow the composer's packagist philosophy instead. >> >> https://wiki.php.net/rfc/pickle >> >> > Hi Derick, > > Thanks for creating the google doc for us, it's really easy to contribute > now, and I think I'll use this same approach for php.net/docs > modernization approach. > > I came here to write about Pierre's Pickle project, but he replied whilst > I was mid-writing this email :-) > > I'll add comments onto the Google Doc. > > Many thanks, > Paul > Hi Derick, I've added my comments, for now, in the first part of the document. As for the latter points in the document, such as json structures and so on, I think it might make sense to complete the Pickle evaluation, as a first phase thing, because the way Pickle is designed may influence how packages are structured. I'm very enthusiastic to hear how this topic evolves :-) Have a great day <3 Many thanks, Paul > > >> > >> >
Re: [PHP-DEV] New "PECL"
On Tue, Dec 5, 2023 at 3:21 PM Pierre Joye wrote: > On Tue, Dec 5, 2023, 10:13 PM Pierre Joye wrote: > > > > > > > You may mention https://github.com/FriendsOfPHP/pickle > > > > which supports pecl.php.net pear format, composer like support, and > > conversion to composer format. > > > > I did a website as well as a test bed and we had a long with the composer > > team what would be the best way to integrate it. That's something that > > became possible with composer 2.x, which now widely used. Similarly we > > discussed the idea to create a packagist dedicated to extensions. > > > > That would be, imo, the best option to date. > > > > > > The current code may need some updates for php8 but nothing difficult or > > requiring big changes. > > > > there was also a draft rfc while we were thinking to propose to bundle it. > But we then decided to follow the composer's packagist philosophy instead. > > https://wiki.php.net/rfc/pickle > > Hi Derick, Thanks for creating the google doc for us, it's really easy to contribute now, and I think I'll use this same approach for php.net/docs modernization approach. I came here to write about Pierre's Pickle project, but he replied whilst I was mid-writing this email :-) I'll add comments onto the Google Doc. Many thanks, Paul > > >
Re: [PHP-DEV] New "PECL"
On Tue, Dec 5, 2023, 10:13 PM Pierre Joye wrote: > > > You may mention https://github.com/FriendsOfPHP/pickle > > which supports pecl.php.net pear format, composer like support, and > conversion to composer format. > > I did a website as well as a test bed and we had a long with the composer > team what would be the best way to integrate it. That's something that > became possible with composer 2.x, which now widely used. Similarly we > discussed the idea to create a packagist dedicated to extensions. > > That would be, imo, the best option to date. > > > The current code may need some updates for php8 but nothing difficult or > requiring big changes. > there was also a draft rfc while we were thinking to propose to bundle it. But we then decided to follow the composer's packagist philosophy instead. https://wiki.php.net/rfc/pickle >
Re: [PHP-DEV] New "PECL"
Hi Derick, On Tue, Dec 5, 2023, 8:45 PM Derick Rethans wrote: > Hi there! > > Within the PHP Foundation, we have been talking for a while as what to > do with PECL, and its website. > > The code is old, and hard to maintain. And the database is full of > mojibake. It is also an outdated method of installing things, especially > because userland code is so much easier to handle through Composer. > > Through the Sovereign Tech Fund (https://www.sovereigntechfund.de/) the > Foundation has acquired some funding to improve this situation. > > Hence, to start of replacing PECL with something applicable for this > decade, we started working on a requirements document: > > > https://docs.google.com/document/d/1_N0E9xo3jn9aKrIZHIbTYaY5lXw71BpSO6-it4cRpDo > > In this first stage, we would like to invite you in commenting on the > document (either inline, or here). > > Please keep in mind that this is a requirements document, and should not > contain either design or implementation details. Once we have thought > about these details as well, this will be turned into an RFC. You may mention https://github.com/FriendsOfPHP/pickle which supports pecl.php.net pear format, composer like support, and conversion to composer format. I did a website as well as a test bed and we had a long with the composer team what would be the best way to integrate it. That's something that became possible with composer 2.x, which now widely used. Similarly we discussed the idea to create a packagist dedicated to extensions. That would be, imo, the best option to date. The current code may need some updates for php8 but nothing difficult or requiring big changes. best, Pierre
[PHP-DEV] New "PECL"
Hi there! Within the PHP Foundation, we have been talking for a while as what to do with PECL, and its website. The code is old, and hard to maintain. And the database is full of mojibake. It is also an outdated method of installing things, especially because userland code is so much easier to handle through Composer. Through the Sovereign Tech Fund (https://www.sovereigntechfund.de/) the Foundation has acquired some funding to improve this situation. Hence, to start of replacing PECL with something applicable for this decade, we started working on a requirements document: https://docs.google.com/document/d/1_N0E9xo3jn9aKrIZHIbTYaY5lXw71BpSO6-it4cRpDo In this first stage, we would like to invite you in commenting on the document (either inline, or here). Please keep in mind that this is a requirements document, and should not contain either design or implementation details. Once we have thought about these details as well, this will be turned into an RFC. cheers, Derick -- https://derickrethans.nl | https://xdebug.org | https://dram.io Author of Xdebug. Like it? Consider supporting me: https://xdebug.org/support Host of PHP Internals News: https://phpinternals.news mastodon: @derickr@phpc.social @xdebug@phpc.social twitter: @derickr and @xdebug -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
