Re: [PHP-DEV] Offset-only results from preg_match
So...
In microbenchmarks you can clearly see the improvement:
```
>>> timeit -n500 preg_match_all('/(.{65535})/s', $html100, $m,
PREG_OFFSET_CAPTURE);
=> 39
Command took 0.001709 seconds on average (0.001654 median; 0.854503 total)
to complete.
>>> timeit -n500 preg_match_all('/(.{65535})/s', $html100, $m,
PREG_OFFSET_CAPTURE|PREG_LENGTH_CAPTURE);
=> 39
Command took 0.000991 seconds on average (0.000965 median; 0.495287 total)
to complete.
```
$html100 is my usual 2,592,386 byte HTML of [[en:Barack Obama]].
But unless you're matching 64k strings like this, it's hard to see a
practical improvement. In my remex-html html parsing benchmark, although
LENGTH_CAPTURE doesn't make things slower, it doesn't make a significant
performance improvement. I built php statically and ran it through
cachegrind to try to figure out why, and found:
2,567,559,670 cycles: total spent executing the tokenizer benchmark
(including reading the file from disk)
1,018,845,290 cycles in zif_preg_match. Optimizing regexps is important
for tokenizers! Of these, we spend
575,478,637 doing the actual match (preg_pcre_match_impl) and
435,162,131 getting the regexp from the cache (!)
(pcre_get_compiled_regex_cache)
This is for 128,794 total regexp matches performed by the tokenizer on this
input.
Of those cycles getting the regex from cache, only 24,116,319 are spent on
cache misses where we are actually compiling regexps (sum of
php_pcre2_jit_compile and php_pcre2_compile).
Instead, 406,007,383 cycles are spent in zend_hash_find(). That's 40% of
the total time spent executing preg_match.
The LENGTH_CAPTURE optimization does reduce the total time spent in
populate_subpat_array from 160,951,690 cycles to 140,042,331 cycles in the
remex-html tokenizer on this benchmark, but that difference is overwhelmed
by (for example) the time spent in zend_hash_find().
The slowdown in zend_hash_find() appears to be due to
https://bugs.php.net/bug.php?id=63180 which disabled interned keys in the
pcre_cache table. Because of this, even if the regexs are interned, we
must pay for a complete zend_string_equal_content() on each match, which
takes time proportional to the length of the regex. This can be quite
large -- for example, for the HTML5 character reference regex in
remex-html, which contains every valid entity name and is 26,137 bytes
long, and we need to do a zend_string_equal_content() on the 26,137 byte
regexp for every ~5 byte entity in the parsed HTML.
--scott
On Thu, Mar 21, 2019 at 7:35 AM Nikita Popov wrote:
> On Wed, Mar 20, 2019 at 4:35 PM C. Scott Ananian
> wrote:
>
>> On Tue, Mar 19, 2019 at 10:58 AM Nikita Popov
>> wrote:
>>
>>> After thinking about this some more, while this may be a minor
>>> performance improvement, it still does more work than necessary. In
>>> particular the use of OFFSET_CAPTURE (which would be pretty much required
>>> here) needs one new two-element array for each subpattern. If the captured
>>> strings are short, this is where the main cost is going to be.
>>>
>>
>> The primary use of this feature is when the captured strings are *long*,
>> as that's when we most want to avoid copying a substring.
>>
>>
>>> I'm wondering if we shouldn't consider a new object oriented API for
>>> PCRE which can return a match object where subpattern positions and
>>> contents can be queried via method calls, so you only pay for the parts
>>> that you do access.
>>>
>>
>> Seems like this is letting the perfect be the enemy of the good. The
>> LENGTH_CAPTURE significantly reduces allocation for long match strings, and
>> it allocates the same two-element arrays that OFFSET_CAPTURE would -- it
>> just stores an integer where there would otherwise be an expensive
>> substring. Furthermore, since the array structure is left mostly alone, it
>> would be not-too-hard to support earlier-PHP versions, with something like:
>>
>> $hasLengthCapture = defined('PREG_LENGTH_CAPTURE') ? PREG_LENGTH_CAPTURE
>> : 0;
>> $r = preg_match($pat, $sub, $m, PREG_OFFSET_CAPTURE | $hasLengthCapture);
>> $matchOneLength = $hasLengthCapture ? $m[1][0] : strlen($m[1][0]);
>> $matchOneOffset = $m[1][1];
>>
>> If you introduce a whole new OO accessor object, it starts becoming very
>> hard to write backward-compatible code.
>> --scott
>>
>
> Fair enough. I've created https://github.com/php/php-src/pull/3971 to
> implement this feature. It would be good to have some confirmation that
> this is really a significant performance improvement before we land it
> though.
>
> Nikita
>
--
(http://cscott.net)
Re: [PHP-DEV] [RFC] DOM Living Standard API
Beware that behaviour of some methods should differ between HTML and non-HTML documents. For instance, the RFC says: > DOMElement→nodeName casing was previously undefined, it is now changed to > always uppercase. However, the DOM Living Standard says it is uppercase (even, ASCII-uppercased) only in the HTML namespace. For XML documents, the casing is not modified. —Claude -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] RFC Draft: Comprehensions
On 21-03-19 16:22, Larry Garfield wrote: > OTOH, if we just have the one syntax: > > [foreach $foo as $bar yield $bar*2] // gives a generator > > And include a nicer "fast-forward" operator than interator_to_array(), then > we automatically get: > > ...[foreach $foo as $bar yield $bar*2] // turns the generator into an array > on the fly > > Which is both easier to learn (two independently useful primitives, > composed), easier to understand (because the latter is not as visually > similar), and doesn't lay claim to two different potentially useful pieces of > block syntax, leaving the other open for later use by some other idea later. That sums it up nicely, thanks. I agree that we should try to focus on the actual comprehension syntax and leave the problem of how to better support both generators and arrays to a future RFC. Still, I would like to mention one detail that is relevant _now_, for the discussion about the current RFC. Indeed, whether comprehensions will be used primarily for creating generators or arrays depends on the developer, libraries that are used, and so on. In my opinion, this calls for a syntax that is not biased towards either of the two. In the cases where comprehensions are mainly used to create arrays, the need to convert a generator into an array still feels like an extra hoop to jump through in order to get what you need. Even when all it takes is three dots. While this issue should be discussed separately, I mention it here to support my proposal to keep the option open to extend the syntax into a dual array / generator syntax in the future. In case we go for a syntax like $generator = (foreach $foo as $bar yield $bar*2) then we could still decide to introduce a variant like $array = [foreach $foo as $bar yield $bar*2] later, as proposed by Nikita. Going for a square bracketed syntax for generators right now will not allow this future extension. Until then, using the spread operator to materialize generators into arrays would be really nice. > Now can we go back to bikeshedding the actual syntax style to use, per my > earlier email? :-) Sounds good. :) -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Unbundle ext/interbase
On 22/03/2019 16:25, Nikita Popov wrote:
I've created a patch forhttps://bugs.php.net/bug.php?id=72175 (last
comment), which seems to be the biggest open problem in the interbase
extension. I'd appreciate it if you or someone else who uses firebird could
test this.
Nikita
Am I jumping the gun a bit? I've got 7.2.16 configured so I can compile
it myself. It's throwing a warning on compile
./ibase.h:761:28: note: expected ‘ISC_UCHAR ** {aka unsigned char **}’
but argument is of type ‘char **’
ISC_LONG ISC_EXPORT_VARARG isc_event_block(ISC_UCHAR**, but the
compiled copy is clearing my ibtest suite.
I've ported your patch to the 7.2.16 copies of the files but this is
throwing a different warning
"implicit declaration of function ‘GC_ADDREF’;" and GC_DELREF
GC_ADDREF certainly exists in the file already but not GC_DELREF so am I
missing something that will stop me making this work on 7.2.16?
It is at least running - http://smallbrook.co.uk:800/phpinfo.php but I
need to add something to test this problem as my framework only uses a
singleton connection ... http://smallbrook.co.uk:800 needs a login, but
the database searches are running fine.
I'm still waiting on mercurial syncing with git but hopefully in the
morning I'll be able to cross check changes in the files over time.
--
Lester Caine - G8HFL
-
Contact - https://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - https://lsces.co.uk
EnquirySolve - https://enquirysolve.com/
Model Engineers Digital Workshop - https://medw.co.uk
Rainbow Digital Media - https://rainbowdigitalmedia.co.uk
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
[PHP-DEV] [RFC] DOM Living Standard API
Hi Internals, Thomas and I are working on updating the ext/dom to add support for the current DOM Living Standard API as standardized here: https://dom.spec.whatwg.org/ https://wiki.php.net/rfc/dom_living_standard_api This RFC is targeting 7.4 and contains three independent changes: - a set of new methods and interfaces that can be implemented BC as addition to the existing ext/dom. - a removal of a few "dead" classes that are exposed to userland, but neither documented nor containing implementation code. - a compatibility layer to switch the implementation between DOM Level 1-3 and the Living Standard in places where BC is not possible. A pull request includes a nearly complete implementation of the new methods, but nothing of the cleanup/compatibility yet: https://github.com/beberlei/php-src/pull/1 We are looking forward for your feedback. greetings Benjamin
Re: [PHP-DEV] [RFC] Unbundle ext/interbase
On Fri, Mar 22, 2019 at 3:10 PM Lester Caine wrote: > On 22/03/2019 13:26, Kalle Sommer Nielsen wrote: > > G'day internals > > > > I'd like to start the discussion for the future of the ext/interbase > extension: > > https://wiki.php.net/rfc/deprecate-and-remove-ext-interbase > > > > The rationale for pushing this extension out of the core is mentioned > > in the RFC. > > > > Unless there is any serious issues raised here, then I will put this > > into voting on Monday 8th of April, noon EET (which is a good two and > > a half weeks away). The intended voting period is set for 2 weeks, > > meaning if voting proceeds, the pools will be closed around Monday > > 22th of April, noon EET. > > It is not that we don't want to stand up and maintain it, it has been > impossible in recent years to get a handle on just what needs to be done > TO maintain it. The PDO extension is in a much worse state than the main > interbase one but both of them do their jobs as well as they can given > that in the case of PDO it can't handle the cross database transactions, > something that the main extension does quite happily. > > Personally I've been wasting time recently trying to keep alive sites > that are using MySQL and the main problem with MySQL is the one thing > that Firebird does nicely. Backup just runs as a secondary cron job > independent of PHP while MySQL is reliant on PHP and current backups on > some wordpress powered sites are failing because they run out of time or > memory. I've never had a problem with loosing data with Firebird while > I've had recover MySQL situations a few times in the last year! > > So all we are asking for is HELP with the code changes that result from > changes to the PHP API to keep this available. > Hi Lester, I've created a patch for https://bugs.php.net/bug.php?id=72175 (last comment), which seems to be the biggest open problem in the interbase extension. I'd appreciate it if you or someone else who uses firebird could test this. Nikita
Re: [PHP-DEV] [RFC] Abolish Short Votes
On Fri, Mar 22, 2019 at 3:41 AM Joe Watkins wrote: > Morning Niklas, > > Allowing the extension of voting leaves us open to someone extending the > voting period simply because they don't feel like they have the result they > wanted. > > The problem we're trying to solve is votes that are too short, while > providing the flexibility to have longer votes, but we need to know at the > start of voting what the voting period is going to be. Take for example the > case where a third party to an RFC is planning some work based on the > results of the voting, it doesn't seem fair that their schedule could be > interfered with by the first party after voting starts. > > In the vast majority of cases where someone forgets or isn't aware of a > holiday period, they are going to be told on day one of voting (if not > before, during discussion), and can simply adjust the voting period, and > restart the vote (if there has been any votes at the first interjection) > without having lost any time. > > First, I don't even have a vote. Second, I'd be ambivalent about the ability to extend voting even if I did. A compromise between the two positions could be to allow voting to be extended ONCE, and only within the first X% of the voting period. So, if someone calls for a 14 day voting period, within the first 7 days they can extend it at most one time, and it can never be extended after the first 7 days. Also, if you are going to be updating the language anyway, why not specify that an exact date and time be given for when voting will end. I think most people already do this - but this would codify it and prevent the possibility of ambiguity in the future. > Cheers > Joe > > On Fri, 22 Mar 2019 at 08:19, Niklas Keller wrote: > > > Resend, because sent from the wrong address previously. > > > > +1, but it should probably be possible to extend the voting period once > > started, but not shorten it. This allows for extension during holidays in > > case the author didn't think about that when starting the vote. > > > > Regards, Niklas > > > > Joe Watkins schrieb am Do., 21. März 2019, 19:20: > > > > > Evening internals, > > > > > > I'd like to raise for discussion another minor, self contained change > to > > > the voting RFC: > > > > > > https://wiki.php.net/rfc/abolish-short-votes > > > > > > This seems like another no-brainer to improve and clarify the voting > > > process. As with abolishing narrow margins, I'm focused on this one > > detail > > > and wider discussion of how we may improve other parts of the voting > RFC > > is > > > not appropriate in this thread: We either have a consensus that this > > detail > > > should be changed or we don't, we do not need to discuss any other > > details > > > here. > > > > > > Cheers > > > Joe > > > > > > -- -- Chase chasepee...@gmail.com
Re: [PHP-DEV] [RFC] Unbundle ext/interbase
On Fri, 22 Mar 2019 at 14:10, Lester Caine wrote: > > it has been impossible in recent years to get a handle > on just what needs to be done TO maintain it. Yes, figuring out what needs to be done takes more time that writing the code. Did you not know this? This is what takes up most of my time as the maintainer of the Imagick extension. I hardly write any code for it, instead I spend a lot of time trying to figure out: * what has changed upstream? Do I need to support that change? * is this bug report valid? which involves spending time trying to recreate it. * how is a feature of it that I've never used myself meant to work? * what needs more documentation? And then writing that documentation. All of that is tediously boring to me, and yet I still do it because I care just enough to avoid the extension from being considered abandoned. We've had this discussion about the interbase extension multiple times. Each time people have said they will step up to maintain it. But apparently each time no-one actually cares enough to commit the time to be up-to-speed and maintain the extension. > So all we are asking for is Who is we in this sentence? Is it end-users who claim that it's a vital part of their business, but haven't actually contributed enough of their time to maintain it? Or is it the billion dollar company that bought Interbase ( https://www.ideracorp.com/brands), and has recently bought Travis CI, and yet doesn't want to hire someone to maintain the extension? In either case, you are asking people who do not use a piece of software to donate their own time maintaining it. btw please do not cross post to multiple groups. If you want to make the firebird developers aware of this proposal you should do separately from your emails to PHP internals. cheers Dan Ack
Re: [PHP-DEV] [RFC] Unbundle ext/interbase
G'day Lester Den fre. 22. mar. 2019 kl. 16.10 skrev Lester Caine : > It is not that we don't want to stand up and maintain it, it has been > impossible in recent years to get a handle on just what needs to be done > TO maintain it. The PDO extension is in a much worse state than the main > interbase one but both of them do their jobs as well as they can given > that in the case of PDO it can't handle the cross database transactions, > something that the main extension does quite happily. Like the previous 2-3 times we have had this topic going about removing the InterBase extension, you have referred to "we" (which I suppose is the PHP community as a whole), however time over and time again, no one has stepped up who has been able to maintain it. I thought about the PDO_Firebird extension too for removal, but I wanted to give the few possible PHP Firebird users a chance to evaluate the situation What do I mean by this? I mean that from a Core perspective (which I have explained before to you personally a few years ago), but to re-iterate: An extension in the Core that is not maintained, poses a risk of security and maintenance required to keep the extension to a level of quality that you (you as in the PHP community) deserve can therefore not be up to par. On top of that, it is a burden on us Core Developers that we cannot do anything to fix it. > Personally I've been wasting time recently trying to keep alive sites > that are using MySQL and the main problem with MySQL is the one thing > that Firebird does nicely. Backup just runs as a secondary cron job > independent of PHP while MySQL is reliant on PHP and current backups on > some wordpress powered sites are failing because they run out of time or > memory. I've never had a problem with loosing data with Firebird while > I've had recover MySQL situations a few times in the last year! That sounds like a bug in MySQL, which you should probably report to them or look at your configuration. > So all we are asking for is HELP with the code changes that result from > changes to the PHP API to keep this available. We asked for help in the past, many times in the fact if you search the archives (and you have taken apart of the conversation every single time), nothing has happened, therefore it is time to move past this. The extension will put in a repository at PECL, however it is unlike to retrieve further improvements or fixes from there unless someone actually steps up to take over it. We have done what we can to try call for help to keep this extension for as long as possible, but we can only do so much for so long. -- regards, Kalle Sommer Nielsen ka...@php.net -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Unbundle ext/interbase
On 22/03/2019 13:26, Kalle Sommer Nielsen wrote: G'day internals I'd like to start the discussion for the future of the ext/interbase extension: https://wiki.php.net/rfc/deprecate-and-remove-ext-interbase The rationale for pushing this extension out of the core is mentioned in the RFC. Unless there is any serious issues raised here, then I will put this into voting on Monday 8th of April, noon EET (which is a good two and a half weeks away). The intended voting period is set for 2 weeks, meaning if voting proceeds, the pools will be closed around Monday 22th of April, noon EET. It is not that we don't want to stand up and maintain it, it has been impossible in recent years to get a handle on just what needs to be done TO maintain it. The PDO extension is in a much worse state than the main interbase one but both of them do their jobs as well as they can given that in the case of PDO it can't handle the cross database transactions, something that the main extension does quite happily. Personally I've been wasting time recently trying to keep alive sites that are using MySQL and the main problem with MySQL is the one thing that Firebird does nicely. Backup just runs as a secondary cron job independent of PHP while MySQL is reliant on PHP and current backups on some wordpress powered sites are failing because they run out of time or memory. I've never had a problem with loosing data with Firebird while I've had recover MySQL situations a few times in the last year! So all we are asking for is HELP with the code changes that result from changes to the PHP API to keep this available. -- Lester Caine - G8HFL - Contact - https://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - https://lsces.co.uk EnquirySolve - https://enquirysolve.com/ Model Engineers Digital Workshop - https://medw.co.uk Rainbow Digital Media - https://rainbowdigitalmedia.co.uk -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Unbundle ext/interbase
Morning Kalle, Seems like a reasonable plan, +1 Cheers Joe On Fri, 22 Mar 2019 at 14:26, Kalle Sommer Nielsen wrote: > G'day internals > > I'd like to start the discussion for the future of the ext/interbase > extension: > https://wiki.php.net/rfc/deprecate-and-remove-ext-interbase > > The rationale for pushing this extension out of the core is mentioned > in the RFC. > > Unless there is any serious issues raised here, then I will put this > into voting on Monday 8th of April, noon EET (which is a good two and > a half weeks away). The intended voting period is set for 2 weeks, > meaning if voting proceeds, the pools will be closed around Monday > 22th of April, noon EET. > > -- > regards, > > Kalle Sommer Nielsen > ka...@php.net > > -- > PHP Internals - PHP Runtime Development Mailing List > To unsubscribe, visit: http://www.php.net/unsub.php > >
[PHP-DEV] [RFC] Unbundle ext/interbase
G'day internals I'd like to start the discussion for the future of the ext/interbase extension: https://wiki.php.net/rfc/deprecate-and-remove-ext-interbase The rationale for pushing this extension out of the core is mentioned in the RFC. Unless there is any serious issues raised here, then I will put this into voting on Monday 8th of April, noon EET (which is a good two and a half weeks away). The intended voting period is set for 2 weeks, meaning if voting proceeds, the pools will be closed around Monday 22th of April, noon EET. -- regards, Kalle Sommer Nielsen ka...@php.net -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Abolish Short Votes
Morning Niklas, Allowing the extension of voting leaves us open to someone extending the voting period simply because they don't feel like they have the result they wanted. The problem we're trying to solve is votes that are too short, while providing the flexibility to have longer votes, but we need to know at the start of voting what the voting period is going to be. Take for example the case where a third party to an RFC is planning some work based on the results of the voting, it doesn't seem fair that their schedule could be interfered with by the first party after voting starts. In the vast majority of cases where someone forgets or isn't aware of a holiday period, they are going to be told on day one of voting (if not before, during discussion), and can simply adjust the voting period, and restart the vote (if there has been any votes at the first interjection) without having lost any time. Cheers Joe On Fri, 22 Mar 2019 at 08:19, Niklas Keller wrote: > Resend, because sent from the wrong address previously. > > +1, but it should probably be possible to extend the voting period once > started, but not shorten it. This allows for extension during holidays in > case the author didn't think about that when starting the vote. > > Regards, Niklas > > Joe Watkins schrieb am Do., 21. März 2019, 19:20: > > > Evening internals, > > > > I'd like to raise for discussion another minor, self contained change to > > the voting RFC: > > > > https://wiki.php.net/rfc/abolish-short-votes > > > > This seems like another no-brainer to improve and clarify the voting > > process. As with abolishing narrow margins, I'm focused on this one > detail > > and wider discussion of how we may improve other parts of the voting RFC > is > > not appropriate in this thread: We either have a consensus that this > detail > > should be changed or we don't, we do not need to discuss any other > details > > here. > > > > Cheers > > Joe > > >
Re: [PHP-DEV] [RFC] Abolish Short Votes
Resend, because sent from the wrong address previously. +1, but it should probably be possible to extend the voting period once started, but not shorten it. This allows for extension during holidays in case the author didn't think about that when starting the vote. Regards, Niklas Joe Watkins schrieb am Do., 21. März 2019, 19:20: > Evening internals, > > I'd like to raise for discussion another minor, self contained change to > the voting RFC: > > https://wiki.php.net/rfc/abolish-short-votes > > This seems like another no-brainer to improve and clarify the voting > process. As with abolishing narrow margins, I'm focused on this one detail > and wider discussion of how we may improve other parts of the voting RFC is > not appropriate in this thread: We either have a consensus that this detail > should be changed or we don't, we do not need to discuss any other details > here. > > Cheers > Joe >
Re: [PHP-DEV] [RFC] Abolish Short Votes
Am 21.03.2019 um 19:20 schrieb Joe Watkins: This seems like another no-brainer to improve and clarify the voting process. As with abolishing narrow margins, I'm focused on this one detail and wider discussion of how we may improve other parts of the voting RFC is not appropriate in this thread: We either have a consensus that this detail should be changed or we don't, we do not need to discuss any other details here. +1 -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
