Re: [PHP-DEV] Core functions throwing exceptions in PHP7
On 07/26/2015 01:36 PM, Jakub Kubíček wrote: Hi Larry! 2015-07-26 1:29 GMT+02:00 Larry Garfield la...@garfieldtech.com: Another point here is that 0 is a perfectly legitimate random number in many cases, so callers would need to do a === check, not just a boolean check. What boolean check are you talkin' about? I've never seen a code using e.g. strpos() like follows: ?php if(!is_bool($pos = strpos('foobar', 'baz'))) { // we are correct, use $pos' value somewhere } else { // strpos() produced a boolean, thus no 'baz' found in 'foobar' } ? Rather it is most frequently being done like below: ?php if(FALSE !== $pos = strpos('foobar', 'baz')) { // we are correct, use $pos' value somewhere } else { // strpos() produced a boolean, thus no 'baz' found in 'foobar' } ? I think in both cases you do a kind of boolean check. Yes, I am referring to the second, or variations therein. When using strpos() you need to do a strict check against FALSE (===) to avoid a legit 0 getting misinterpreted. It's also a very commonly forgotten check, especially among newer devs that haven't been burned by it a few times. My point is that when talking about crypto-related functions, relying on people to get burned a few times and then remember to do a === check against FALSE is a horrible, horrible idea, hence random_int() should fail much harder, as is being suggested here (be that via E_FATAL, exception, error throw, or whatever, as long as the execution does NOT continue). That is, I am agreeing with the proposal to have it die hard. I moderately prefer the throw SecurityError approach as it's not a user-input error but a system config error, but anything that prevents execution from continuing is acceptable security-wise. --Larry Garfield -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Core functions throwing exceptions in PHP7
Hi Larry! 2015-07-26 1:29 GMT+02:00 Larry Garfield la...@garfieldtech.com: Another point here is that 0 is a perfectly legitimate random number in many cases, so callers would need to do a === check, not just a boolean check. What boolean check are you talkin' about? I've never seen a code using e.g. strpos() like follows: ?php if(!is_bool($pos = strpos('foobar', 'baz'))) { // we are correct, use $pos' value somewhere } else { // strpos() produced a boolean, thus no 'baz' found in 'foobar' } ? Rather it is most frequently being done like below: ?php if(FALSE !== $pos = strpos('foobar', 'baz')) { // we are correct, use $pos' value somewhere } else { // strpos() produced a boolean, thus no 'baz' found in 'foobar' } ? I think in both cases you do a kind of boolean check. Especially as we're talking not about a user error but a your system is not setup right so it will never work situation, as I understand it. So, I generally agree with this approach. It is a better thing to always fail closed if your system isn't set up, e.g. missing some required things, to work properly. It's the same if your code uses some vendor extension not included in the core by default -- it can not work work properly without having this extension available. I say +1 for raising a E_ERROR on random_int()/random_bytes() fail. Best regards, Kubo2, Jakub Kubíček -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP-DEV] PCRE JIT stack size limit
Hi Christoph, -Original Message- From: Christoph Becker [mailto:cmbecke...@gmx.de] Sent: Saturday, July 25, 2015 6:02 PM To: Anatol Belski anatol@belski.net; 'Pierre Joye' pierre@gmail.com Cc: 'PHP internals' internals@lists.php.net Subject: Re: [PHP-DEV] PCRE JIT stack size limit Hi Anatol, Anatol Belski wrote: -Original Message- From: Christoph Becker [mailto:cmbecke...@gmx.de] Sent: Saturday, July 25, 2015 2:25 AM To: Anatol Belski anatol@belski.net; 'Pierre Joye' pierre@gmail.com Cc: 'PHP internals' internals@lists.php.net Subject: Re: [PHP-DEV] PCRE JIT stack size limit I would not suggest to change the return value of preg_match() and friends. FALSE seems to be appropriate here. Instead I suggest to add a new error constant which would be returned from preg_last_error(). Yep, sounds feasible. At least users can be informed about this kind of error, seems a constant is feasible for 7.0. That would be nice. I'll make a respective PR. Thanks. Btw I've just noticed that the snippet I've posted earlier won't currently work. If (false === preg_match(,evil pattern,, ...)) { If (OUT_OF_JIT_STACK == pcre_last_error()) { ini_set(pcre.jit, 0); // retry } } The retry would fail even when JIT is disabled on demand. This is actually rather unexpected, maybe something in PCRE JIT state cannot be recovered once JIT compilation has failed. But actually, even without running into debugging, it is also a bad sign telling that the behavior can get complicated ( Indeed! That's caused by our PCRE cache. The cached regex is alread studied with JIT info, as all that happens in pcre_get_compiled-regex_cache[1]. So switching pcre.jit at runtime may not have the desired effect. I'm not sure if that has to be regarded as bug, and whether we should do something about it. One solution might be to clear the regex cache whenever the ini setting changes. I'll have a closer look. Good catch. Maybe another option could be implementing the PHP pattern above In C. Say - check what pcre_exec delivered - if it’s the JIT stack error - look for the pattern in the cache - if it's found - remove it from the cache - recompile and cache - exec a non JIT pattern version However this would silently ignore JIT, not transparent for user. Maybe another option could be introducing a user space function to clear a particular pattern from the cache. When expunging the full cache, it'll for sure have a negative performance effect when the cache isn't empty. But it's actually logic as one can base it on the idea - either JIT is enabled for everything, or it is not (for everything). Running for the INI option and some custom JIT stack functionality needs more brainstorming and test. Please ping when you have an initial implementation, I'd be glad to help testing, etc. anyway. All in all it seems that we probably should have an RFC for that. I'll consider to write one, but there's no need to hurry – too late for 7.0 anyway. :) Yep, the whole needs a good consideration about how to do it best without a big impact on the user side. Also probably it could be worth it to check how PCRE2 behaves in this cases, maybe it were worthwhile to upgrade to PCRE2 for 7.1. Regards Anatol -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] RFE to allow dirname($foo, 2)
Hi Scott! 2015-07-25 19:42 GMT+02:00 Scott Arciszewski sc...@paragonie.com: What's easier to read and less likely to result in bugs? require_once __DIR__ . '/../../../../autoload.php'; or require_once dirname(__FILE__, 5) . '/autoload.php'; That's on everyone's own, but for me it's the first. However, it couldn't be a disaster having ability to choose other option for those, who are not familiar with relative paths. :-) Best regards, Kubo2, Jakub Kubíček -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP-DEV] Core functions throwing exceptions in PHP7
Hi, -Original Message- From: Larry Garfield [mailto:la...@garfieldtech.com] Sent: Sunday, July 26, 2015 9:38 PM To: internals@lists.php.net Subject: Re: [PHP-DEV] Core functions throwing exceptions in PHP7 On 07/26/2015 01:36 PM, Jakub Kubíček wrote: Hi Larry! 2015-07-26 1:29 GMT+02:00 Larry Garfield la...@garfieldtech.com: Another point here is that 0 is a perfectly legitimate random number in many cases, so callers would need to do a === check, not just a boolean check. What boolean check are you talkin' about? I've never seen a code using e.g. strpos() like follows: ?php if(!is_bool($pos = strpos('foobar', 'baz'))) { // we are correct, use $pos' value somewhere } else { // strpos() produced a boolean, thus no 'baz' found in 'foobar' } ? Rather it is most frequently being done like below: ?php if(FALSE !== $pos = strpos('foobar', 'baz')) { // we are correct, use $pos' value somewhere } else { // strpos() produced a boolean, thus no 'baz' found in 'foobar' } ? I think in both cases you do a kind of boolean check. Yes, I am referring to the second, or variations therein. When using strpos() you need to do a strict check against FALSE (===) to avoid a legit 0 getting misinterpreted. It's also a very commonly forgotten check, especially among newer devs that haven't been burned by it a few times. My point is that when talking about crypto-related functions, relying on people to get burned a few times and then remember to do a === check against FALSE is a horrible, horrible idea, hence random_int() should fail much harder, as is being suggested here (be that via E_FATAL, exception, error throw, or whatever, as long as the execution does NOT continue). That is, I am agreeing with the proposal to have it die hard. I moderately prefer the throw SecurityError approach as it's not a user-input error but a system config error, but anything that prevents execution from continuing is acceptable security-wise. When implementing a security related code, people that are tired or asleep should stay home :) But going seriously - what it is about is changing the paradigm. IMHO this kind of thing should not be done making a special case. Since Trowable is a good base, there is a need on a strategy. The keyword about the exception hierarchy is not seldom to hear it this regard nowadays. So besides SecurityError, what is with IOError, MemoryError, other common cases? Also be aware and prepared that many cases can't currently be served by exceptions, the very low level errors. But basically what I would refer to is a concept catching the common cases, so then - we have a set of rules about exceptions in functions - those rules produce consistent behaviors - the case by case basis argument is eliminated by them in 99% of cases Regards Anatol -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] json_decode/encode should return full precision values by default
Hi Jakub, On Mon, Jul 27, 2015 at 3:32 AM, Jakub Zelenka bu...@php.net wrote: I don't think that this is a bug. Your example is also completely unrelated to json because the place when the value is rounded is var_dump where it's based on ini precision. You would get the same values with json_encode but it's only because it uses the same ini ( precision ). Basically it has nothing to do with json_decode. OK. Better example. [yohgaki@dev PHP-master]$ ./php-bin ?php $j = '{ v: 0.1234567890123456789 }'; var_dump(json_encode(json_decode($j))); ini_set('precision', 20); var_dump(json_encode(json_decode($j))); ? string(22) {v:0.12345678901235} string(28) {v:0.12345678901234567737} I see that that the bug report you created ( https://bugs.php.net/bug.php?id=70137 ) is actually about using serialize.precision instead of precision in json_encode . I agree that using 'precision' ini is not the best solution. However I don't think that start suddenly using serialize.precision is a good idea. First of all it's a big BC break that will be quite difficult to find (There is no way this should go to the point release because this is changing the generated json). Also the json encode usage is much wider than serialization. There might be use cases when you don't care about maximal precision and you prefer to save some space. Imagine that you are transferring lots of float numbers. Then it will result in increasing of the transfer size. It would be great if the new ini was used when it was introduced but I'm afraid that changing that now is not a good idea I think that due to BC break, this shouldn't be changed. If you really want a bigger precision, you can do ini_set('precision', 30) before json_encode and then set it back. It means that this not a bug because you can still change it if you want. That means that we can't change before PHP 8 or whatever comes after 7... :) Same argument could be done for serialize, but it's changed to keep more precise value. Manual recommend json_decode/encode for data serialization also. Warning Do not pass untrusted user input to unserialize(). Unserialization can result in code being loaded and executed due to object instantiation and autoloading, and a malicious user may be able to exploit this. Use a safe, standard data interchange format such as JSON (via json_decode() and json_encode()) if you need to pass serialized data to the user. http://jp2.php.net/manual/en/function.unserialize.php It does not make sense having different serialization for float. Losing effective precision is design bug. IMHO. Currently users cannot handle even Google Map JSON data correctly by default. Fixing this is just a matter of using serialize.precision. If users need larger setting than 17 for whatever reason, they may do ini_set('serialize.precision', 30) and the rest of codes are unaffected. BTW, larger precision setting does not mean precise, but current behavior definitely lose effective values. Those who don't care for precise data handling wouldn't care for more precise data handling, do they? Regards, -- Yasuo Ohgaki yohg...@ohgaki.net
Re: [PHP-DEV] Wiki Permissions and phpng-upgrading
On Sat, Jul 25, 2015 at 1:59 AM, Benjamin Eberlei kont...@beberlei.de wrote: Hi everyone, i started porting an extension today and scimmed the phpng-upgrading file. It has some inaccuracies already with changes after the phpng-merge. However my access rights are not enough for user (beberlei) getting a Read Only notice. Could I have permission to edit this page? Also weltling had the idea to fork this page to php7-upgrading, since phpng is not really anymore and we could improve upon the naming. The old page would reference or forward to the new one obviously. Thoughts? This applie to all phpng-* pages that are still relevant. greetings Benjamin hi, I've pushed an acl change, you should be able to edit the phpng-upgrading file (as discussed on irc): http://git.php.net/?p=web/wiki.git;a=commitdiff;h=985fd08856dca764cef9589875ab5d353da23da9 let me know if you need further access, albeit it would be easier if you either get a php.net account (as that comes with full wiki karma) or if we could move the php7 migration related pages under a common wiki namespace(now they are mostly under the root namespace). -- Ferenc Kovács @Tyr43l - http://tyrael.hu
Re: [PHP-DEV] PCRE JIT stack size limit
Hi Anatol, Anatol Belski wrote: Hi Christoph, -Original Message- From: Christoph Becker [mailto:cmbecke...@gmx.de] Sent: Saturday, July 25, 2015 6:02 PM To: Anatol Belski anatol@belski.net; 'Pierre Joye' pierre@gmail.com Cc: 'PHP internals' internals@lists.php.net Subject: Re: [PHP-DEV] PCRE JIT stack size limit Hi Anatol, Anatol Belski wrote: -Original Message- From: Christoph Becker [mailto:cmbecke...@gmx.de] Sent: Saturday, July 25, 2015 2:25 AM To: Anatol Belski anatol@belski.net; 'Pierre Joye' pierre@gmail.com Cc: 'PHP internals' internals@lists.php.net Subject: Re: [PHP-DEV] PCRE JIT stack size limit I would not suggest to change the return value of preg_match() and friends. FALSE seems to be appropriate here. Instead I suggest to add a new error constant which would be returned from preg_last_error(). Yep, sounds feasible. At least users can be informed about this kind of error, seems a constant is feasible for 7.0. That would be nice. I'll make a respective PR. Thanks. Btw I've just noticed that the snippet I've posted earlier won't currently work. If (false === preg_match(,evil pattern,, ...)) { If (OUT_OF_JIT_STACK == pcre_last_error()) { ini_set(pcre.jit, 0); // retry } } The retry would fail even when JIT is disabled on demand. This is actually rather unexpected, maybe something in PCRE JIT state cannot be recovered once JIT compilation has failed. But actually, even without running into debugging, it is also a bad sign telling that the behavior can get complicated ( Indeed! That's caused by our PCRE cache. The cached regex is alread studied with JIT info, as all that happens in pcre_get_compiled-regex_cache[1]. So switching pcre.jit at runtime may not have the desired effect. I'm not sure if that has to be regarded as bug, and whether we should do something about it. One solution might be to clear the regex cache whenever the ini setting changes. I'll have a closer look. Good catch. Maybe another option could be implementing the PHP pattern above In C. Say - check what pcre_exec delivered - if it’s the JIT stack error - look for the pattern in the cache - if it's found - remove it from the cache - recompile and cache - exec a non JIT pattern version However this would silently ignore JIT, not transparent for user. Maybe another option could be introducing a user space function to clear a particular pattern from the cache. When expunging the full cache, it'll for sure have a negative performance effect when the cache isn't empty. But it's actually logic as one can base it on the idea - either JIT is enabled for everything, or it is not (for everything). Definitely interesting ideas. However, currently I'm leaning towards a full-fledged solution in PHP 7.1 (whatever that may be), and to only add the error constant for PHP 7.0. Users can set pcre.jit=0 in php.ini and not change the setting during runtime, if necessary. FWIW, some hours ago I've implemented the clean whole cache solution: https://github.com/cmb69/php-src/commit/c41d78046da48db2fa8f5bd82fd6ac58099288f8. Running for the INI option and some custom JIT stack functionality needs more brainstorming and test. Please ping when you have an initial implementation, I'd be glad to help testing, etc. anyway. All in all it seems that we probably should have an RFC for that. I'll consider to write one, but there's no need to hurry – too late for 7.0 anyway. :) Yep, the whole needs a good consideration about how to do it best without a big impact on the user side. Also probably it could be worth it to check how PCRE2 behaves in this cases, maybe it were worthwhile to upgrade to PCRE2 for 7.1. Indeed, offering support for PCRE2 should be considered for 7.1. And also support of the JIT fast path API: http://www.pcre.org/original/doc/html/pcrejit.html#SEC11. Regards -- Christoph M. Becker -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Core functions throwing exceptions in PHP7
I must have overlooked a detail here. According to https://github.com/tpunt/PHP7-Reference#throwable-interface there are Throwables called Error, as a separate designation from an exception. I didn't see this in the engine exceptions RFC, so I was unaware that was even a thing. In this case, yes, as long as you can wrap it in try/catch blocks, SecurityError which extends Error and/or implements Throwable is an excellent suggestion. Previously, I thought the suggestion was to stick to triggering errors (E_ERROR, E_RECOVERABLE_ERROR, etc.). Scott Arciszewski Chief Development Officer Paragon Initiative Enterprises https://paragonie.com -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php I believe some were suggesting triggering an E_ERROR, though most E_ERRORs in the engine have been replaced with thrown Error exceptions, so I think using E_ERROR in this case would be inappropriate. As I suggested in my prior email, I think throwing an instance of Error would be appropriate when the functions random_bytes() and random_int() fail. There are several conditions that already cause the engine to throw an Error (or subclass thereof): (1)-method(); // Throws Error declare(strict_types=1); array_map(1, 1); // Throws TypeError require 'file-with-parse-error.php'; // Throws ParseError eval($a[ = 1;); // Throws ParseError 1 -1; // Throws ArithmeticError intdiv(1, 0); // Throws DivisionByZeroError 1 % 0; // Throws DivisionByZeroError Of particular interest in the above examples is intdiv(), an internal function that can throw an instance of Error if the denominator is zero. I propose that random_bytes() and random_int() should throw an instance of Error if the parameters are not as expected or if generating a random number fails. (To avoid further debate about a subclass, the function should throw just a plain instance of Error, it can always be subclassed later without BC concerns). random_bytes() and random_int() failing closed is very important to prevent misguided or lazy programmers from using false in place of a random value. A return of false can easily be overlooked and unintentionally be cast to a zero or empty string. A thrown instance of Error must be purposely caught and ignored to produce the same behavior. As Larry pointed out, it is a very common error for programmers to not do a strict check using === against false when calling strpos(). Does anyone have a strong objection to the above proposal? If not, then I think Sammy should update his PRs to throw an Error so they can be merged before the next beta release. Aaron Piotrowski -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Core functions throwing exceptions in PHP7
+1 for Error On Jul 26, 2015 11:32 PM, Aaron Piotrowski aa...@icicle.io wrote: I must have overlooked a detail here. According to https://github.com/tpunt/PHP7-Reference#throwable-interface there are Throwables called Error, as a separate designation from an exception. I didn't see this in the engine exceptions RFC, so I was unaware that was even a thing. In this case, yes, as long as you can wrap it in try/catch blocks, SecurityError which extends Error and/or implements Throwable is an excellent suggestion. Previously, I thought the suggestion was to stick to triggering errors (E_ERROR, E_RECOVERABLE_ERROR, etc.). Scott Arciszewski Chief Development Officer Paragon Initiative Enterprises https://paragonie.com -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php I believe some were suggesting triggering an E_ERROR, though most E_ERRORs in the engine have been replaced with thrown Error exceptions, so I think using E_ERROR in this case would be inappropriate. As I suggested in my prior email, I think throwing an instance of Error would be appropriate when the functions random_bytes() and random_int() fail. There are several conditions that already cause the engine to throw an Error (or subclass thereof): (1)-method(); // Throws Error declare(strict_types=1); array_map(1, 1); // Throws TypeError require 'file-with-parse-error.php'; // Throws ParseError eval($a[ = 1;); // Throws ParseError 1 -1; // Throws ArithmeticError intdiv(1, 0); // Throws DivisionByZeroError 1 % 0; // Throws DivisionByZeroError Of particular interest in the above examples is intdiv(), an internal function that can throw an instance of Error if the denominator is zero. I propose that random_bytes() and random_int() should throw an instance of Error if the parameters are not as expected or if generating a random number fails. (To avoid further debate about a subclass, the function should throw just a plain instance of Error, it can always be subclassed later without BC concerns). random_bytes() and random_int() failing closed is very important to prevent misguided or lazy programmers from using false in place of a random value. A return of false can easily be overlooked and unintentionally be cast to a zero or empty string. A thrown instance of Error must be purposely caught and ignored to produce the same behavior. As Larry pointed out, it is a very common error for programmers to not do a strict check using === against false when calling strpos(). Does anyone have a strong objection to the above proposal? If not, then I think Sammy should update his PRs to throw an Error so they can be merged before the next beta release. Aaron Piotrowski
Re: [PHP-DEV] json_decode/encode should return full precision values by default
Hi, On Sat, Jul 25, 2015 at 11:01 PM, Yasuo Ohgaki yohg...@ohgaki.net wrote: Hi all, I had to work with Google Map API and need to handle values precisely. Fortunately, it seems values are IEEE double, but I get rounded float values by default. For example, ?php $json = ' { results : [ { elevation : 1608.637939453125, location : { lat : 39.73915360, lng : -104.98470340 }, resolution : 4.771975994110107 }, { elevation : -50.78903579711914, location : { lat : 36.460, lng : -116.870 }, resolution : 19.08790397644043 } ], status : OK } '; var_dump(json_decode($json)); ? object(stdClass)#5 (2) { [results]= array(2) { [0]= object(stdClass)#1 (3) { [elevation]= float(1608.6379394531) [location]= object(stdClass)#2 (2) { [lat]= float(39.7391536) [lng]= float(-104.9847034) } [resolution]= float(4.7719759941101) } [1]= object(stdClass)#3 (3) { [elevation]= float(-50.789035797119) [location]= object(stdClass)#4 (2) { [lat]= float(36.46) [lng]= float(-116.87) } [resolution]= float(19.08790397644) } } [status]= string(2) OK } json_decode()/json_encode() must be able to handle precise IEEE double value by _default_. serialize() is changed to use max precision. json_decode/encode should do the same at least. I think this change should be provided as bug fix. Any comments? I don't think that this is a bug. Your example is also completely unrelated to json because the place when the value is rounded is var_dump where it's based on ini precision. You would get the same values with json_encode but it's only because it uses the same ini ( precision ). Basically it has nothing to do with json_decode. I see that that the bug report you created ( https://bugs.php.net/bug.php?id=70137 ) is actually about using serialize.precision instead of precision in json_encode . I agree that using 'precision' ini is not the best solution. However I don't think that start suddenly using serialize.precision is a good idea. First of all it's a big BC break that will be quite difficult to find (There is no way this should go to the point release because this is changing the generated json). Also the json encode usage is much wider than serialization. There might be use cases when you don't care about maximal precision and you prefer to save some space. Imagine that you are transferring lots of float numbers. Then it will result in increasing of the transfer size. It would be great if the new ini was used when it was introduced but I'm afraid that changing that now is not a good idea I think that due to BC break, this shouldn't be changed. If you really want a bigger precision, you can do ini_set('precision', 30) before json_encode and then set it back. It means that this not a bug because you can still change it if you want. That means that we can't change before PHP 8 or whatever comes after 7... :) Cheers Jakub