[PHP-DEV] default charset confusion

2012-03-12 Thread Rasmus Lerdorf
I caused this situation myself by not explicitly differentiating between the default charset for the internal htmlspecialchars() and htmlentities() functions and the output charset directive ini directive default_charset. The idea behind the default_charset ini directive was to act as the charset

Re: [PHP-DEV] default charset confusion

2012-03-12 Thread Laruence
On Mon, Mar 12, 2012 at 2:49 PM, Rasmus Lerdorf ras...@lerdorf.com wrote: I caused this situation myself by not explicitly differentiating between the default charset for the internal htmlspecialchars() and htmlentities() functions and the output charset directive ini directive

Re: [PHP-DEV] default charset confusion

2012-03-12 Thread Adam Jon Richardson
On Mon, Mar 12, 2012 at 2:49 AM, Rasmus Lerdorf ras...@lerdorf.com wrote: What we really need is what we added in PHP 6. A runtime encoding ini setting that is distinct from the output charset which we can use here. That would allow people to fix all their legacy code to a specific runtime

Re: [PHP-DEV] default charset confusion

2012-03-12 Thread Stas Malyshev
Hi! What we really need is what we added in PHP 6. A runtime encoding ini setting that is distinct from the output charset which we can use here. That would allow people to fix all their legacy code to a specific runtime encoding with a single ini setting instead of changing thousands of lines

Re: [PHP-DEV] default charset confusion

2012-03-12 Thread Laruence
On Mon, Mar 12, 2012 at 3:10 PM, Stas Malyshev smalys...@sugarcrm.com wrote: Hi! What we really need is what we added in PHP 6. A runtime encoding ini setting that is distinct from the output charset which we can use here. That would allow people to fix all their legacy code to a specific

Re: [PHP-DEV] default charset confusion

2012-03-12 Thread Laruence
On Mon, Mar 12, 2012 at 3:10 PM, Stas Malyshev smalys...@sugarcrm.com wrote: Hi! What we really need is what we added in PHP 6. A runtime encoding ini setting that is distinct from the output charset which we can use here. That would allow people to fix all their legacy code to a specific

Re: [PHP-DEV] default charset confusion

2012-03-12 Thread Rasmus Lerdorf
On 03/12/2012 12:10 AM, Stas Malyshev wrote: Hi! What we really need is what we added in PHP 6. A runtime encoding ini setting that is distinct from the output charset which we can use here. That would allow people to fix all their legacy code to a specific runtime encoding with a single

Re: [PHP-DEV] default charset confusion

2012-03-12 Thread Rasmus Lerdorf
On 03/12/2012 12:41 AM, Rasmus Lerdorf wrote: $string = $string = prep$gb2312/p/pre; Sorry typo there obviously. Just one $string -Rasmus -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] default charset confusion

2012-03-12 Thread Stas Malyshev
Hi! Ignoring 5.4 for a second, if you in 5.3 do this: echo htmlspecialchars($string); echo htmlspecialchars($string, NULL, ISO-8859-1); echo htmlspecialchars($string, NULL, UTF-8); You will see that the first two output the escaped string with the GB2312 bytes intact within it and the UTF-8

Re: [PHP-DEV] default charset confusion

2012-03-12 Thread Adam Jon Richardson
On Mon, Mar 12, 2012 at 3:52 AM, Stas Malyshev smalys...@sugarcrm.comwrote: Hi! Ignoring 5.4 for a second, if you in 5.3 do this: echo htmlspecialchars($string); echo htmlspecialchars($string, NULL, ISO-8859-1); echo htmlspecialchars($string, NULL, UTF-8); You will see that the first

Re: [PHP-DEV] default charset confusion

2012-03-12 Thread Rasmus Lerdorf
On 03/12/2012 12:52 AM, Stas Malyshev wrote: Hi! Ignoring 5.4 for a second, if you in 5.3 do this: echo htmlspecialchars($string); echo htmlspecialchars($string, NULL, ISO-8859-1); echo htmlspecialchars($string, NULL, UTF-8); You will see that the first two output the escaped string with

Re: [PHP-DEV] [POC - Patch] Scalar Type Hinting - A-La zend_parse_parameters

2012-03-12 Thread Arvids Godjuks
I should point out that returning false on param parsing failure on the language level is one thing (not to mention it's not ok to do that in the first place by my taste), but forcing that behavior on the user-land level is kind'a too much. Consider how the code will become much more complicated

Re: [PHP-DEV] default charset confusion

2012-03-12 Thread Yasuo Ohgaki
Hi I think following PHP 5.4.0 NEWS entry is misleading. . Changed default value of default_charset php.ini option from ISO-8859-1 to UTF-8. (Rasmus) I thought default_charset became UTF-8, so I was expecting following HTTP header. content-typetext/html; charset=UTF-8 However, I got

Re: [PHP-DEV] default charset confusion

2012-03-12 Thread Yasuo Ohgaki
Hi, I think motivation of /* Default is now UTF-8 */ if (charset_hint == NULL) return cs_utf_8; is for better performance and I think it's good for better performance. Alternative of my suggestion is introduce new php.ini entry as Rusmus mentioned. The name may be

Re: [PHP-DEV] [POC - Patch] Scalar Type Hinting - A-La zend_parse_parameters

2012-03-12 Thread Anthony Ferrara
Arvids, On Mon, Mar 12, 2012 at 4:39 AM, Arvids Godjuks arvids.godj...@gmail.com wrote: I should point out that returning false on param parsing failure on the language level is one thing (not to mention it's not ok to do that in the first place by my taste), but forcing that behavior on the

Re: [PHP-DEV] [POC - Patch] Scalar Type Hinting - A-La zend_parse_parameters

2012-03-12 Thread Arvids Godjuks
What is consistent and exists on the internal language layer not necessarily good for the user-land. I'm kind'a surprised no one thought of that. As I said I can live with the throwing notices and warnings (and not E_RECOVERABLE_ERROR as I personally wanted), but returning false even

Re: [PHP-DEV] [POC - Patch] Scalar Type Hinting - A-La zend_parse_parameters

2012-03-12 Thread Anthony Ferrara
Arvids, Yea, that part looks confusing. What I wanted to say is that I would like to get E_RECOVERABLE_ERROR and I was voicing my opinion on that earlier in the threads. But I could live with E_WARNING and E_NOTICE if community decides it to be less strict - I will clean up my code not to

Re: [PHP-DEV] [POC - Patch] Scalar Type Hinting - A-La zend_parse_parameters

2012-03-12 Thread Lazare Inepologlou
Hello Anthony, I will raise once again the question about accepting null. According to your POC, null is an acceptable value if it is also declared as a default value. This is problematic for the scalar types, because they can very well have a different default value. An example: There is a

Re: [PHP-DEV] [POC - Patch] Scalar Type Hinting - A-La zend_parse_parameters

2012-03-12 Thread Simon Schick
2012/3/12 Lazare Inepologlou linep...@gmail.com function set_check_box_state( bool state = false ) { ... } set_check_box_state( null );  // null will be converted to false here... Therefore, this cannot work, unless the default value becomes null, which is against the requirements. What I

Re: [PHP-DEV] default charset confusion

2012-03-12 Thread Laruence
On Mon, Mar 12, 2012 at 6:21 PM, Yasuo Ohgaki yohg...@ohgaki.net wrote: Hi, I think motivation of       /* Default is now UTF-8 */       if (charset_hint == NULL)               return cs_utf_8; is for better performance and I think it's good for better performance. Alternative of my

Re: [PHP-DEV] [POC - Patch] Scalar Type Hinting - A-La zend_parse_parameters

2012-03-12 Thread Arvids Godjuks
I think that the null issue is not an issue. Strictly speaking if you want null or an int - leave out the type hint and use generic argument that will accept anything. I think it's over-engineering to try and push a special treatment for the null. If function/method argument accepts anything but a

Re: [PHP-DEV] [POC - Patch] Scalar Type Hinting - A-La zend_parse_parameters

2012-03-12 Thread Lazare Inepologlou
Hello Simon, First of all, none of your examples cover the case I mentioned, and so, my concerns are still valid. Secondly, you make some wrong assumptions about how this specific POC works. For example, you write: function foo(int $d = 20) { var_dump($d); } foo(null); // This should then

Re: [PHP-DEV] [POC - Patch] Scalar Type Hinting - A-La zend_parse_parameters

2012-03-12 Thread Lazare Inepologlou
I'm not sure about you, but I don't wanna see that kind of thing eventually making it's way into the language Me neither. All I am saying is that, since int|null is already here from the back door, I think it should be properly supported. Lazare INEPOLOGLOU Ingénieur Logiciel 2012/3/12

Re: [PHP-DEV] [POC - Patch] Scalar Type Hinting - A-La zend_parse_parameters

2012-03-12 Thread Arvids Godjuks
2012/3/12 Lazare Inepologlou linep...@gmail.com I'm not sure about you, but I don't wanna see that kind of thing eventually making it's way into the language Me neither. All I am saying is that, since int|null is already here from the back door, I think it should be properly supported.

Re: [PHP-DEV] [POC - Patch] Scalar Type Hinting - A-La zend_parse_parameters

2012-03-12 Thread Anthony Ferrara
Lazare, The patch of Anthony, clearly states that this is accepted: function foo ( int $bar = null ) { } And this is what I called an int|null. Yup, it does. Because that's the current behavior with array and object casting. If you default it to null in the declaration, null is a valid

Re: [PHP-DEV] [POC - Patch] Scalar Type Hinting - A-La zend_parse_parameters

2012-03-12 Thread Lazare Inepologlou
Hello Arvids, The patch of Anthony, clearly states that this is accepted: function foo ( int $bar = null ) { } And this is what I called an int|null. Lazare INEPOLOGLOU Ingénieur Logiciel 2012/3/12 Arvids Godjuks arvids.godj...@gmail.com 2012/3/12 Lazare Inepologlou linep...@gmail.com

Re: [PHP-DEV] default charset confusion

2012-03-12 Thread Rasmus Lerdorf
On 03/12/2012 03:05 AM, Yasuo Ohgaki wrote: Hi I think following PHP 5.4.0 NEWS entry is misleading. . Changed default value of default_charset php.ini option from ISO-8859-1 to UTF-8. (Rasmus) Yes, I have fixed that now. I thought default_charset became UTF-8, so I was

Re: [PHP-DEV] default charset confusion

2012-03-12 Thread Michael Stowe
I think the ini directive, while adding another to the list, may be the most unobtrusive method to address this issue, at least for developers. I definitely agree with Rasmus that this could be one of the bigger headaches in transitioning to 5.4 (for non-UTF8 sites) and unless we can come up with

Re: [PHP-DEV] [POC - Patch] Scalar Type Hinting - A-La zend_parse_parameters

2012-03-12 Thread Lazare Inepologlou
Thank you for the confirmation. What I am saying here is that, although this behavior was fine for objects, it is not enough for scalars. One of the main arguments in favor of the adoption of this syntax was that null was the only possible default value for objects anyway. This obviously is not

Re: [PHP-DEV] default charset confusion

2012-03-12 Thread Richard Lynch
On Mon, March 12, 2012 1:49 am, Rasmus Lerdorf wrote: What we really need is what we added in PHP 6. A runtime encoding ini setting that is distinct from the output charset which we can use here. The usual argument against another php.ini setting, other than too many already is the difficulty

Re: [PHP-DEV] CURL file posting

2012-03-12 Thread Richard Lynch
On Sun, March 11, 2012 6:29 pm, Stas Malyshev wrote: Hi! I'd sure like a PHP extension that didn't have this obvious and nasty bug: https://bugs.php.net/bug.php?id=46439 This doesn't look good. Documentation does say the @ prefix exists, but it has very high potential of creating

Re: [PHP-DEV] [POC - Patch] Scalar Type Hinting - A-La zend_parse_parameters

2012-03-12 Thread Richard Lynch
On Fri, March 9, 2012 2:51 am, Nikita Popov wrote: On Fri, Mar 9, 2012 at 3:58 AM, Ilia Alshanetsky i...@prohost.org wrote: Anthony, My concern with this type of patch is that what you are proposing are not really hints, they are forced casts. As such they modify the data potentially

Re: [PHP-DEV] default charset confusion

2012-03-12 Thread Rasmus Lerdorf
On 03/12/2012 12:40 PM, Stas Malyshev wrote: Hi! And yes, it may very well be dangerous to use the wrong charset and now that we have better support for GB2312 and other asian charsets in the entities functions in 5.4 it is even more prudent to choose the right one so we should provide some

RE: [PHP-DEV] [POC - Patch] Scalar Type Hinting - A-La zend_parse_parameters

2012-03-12 Thread Richard Lynch
On Fri, March 9, 2012 5:58 pm, John Crenshaw wrote: The reason you have to validate the input type in this case is because even though it is a reference, we don't ACTALLY know that it isn't supposed to contain an input (even though that would be against all sane rules most of the time). Last

Re: [PHP-DEV] default charset confusion

2012-03-12 Thread Stas Malyshev
Hi! But you can't necessarily hardcode the encoding if you are writing portable code. That's a bit like hardcoding a timezone. In order to write portable code you need to give people the ability to localize it. No, it's not like timezone at all. I have to support all timezones in a global

Re: [PHP-DEV] any blogs?

2012-03-12 Thread Richard Lynch
I can't recommend any blogs, per se, but Sara's book or even her articles on Zend.com as well as the php.net manual about internals at the end are a must read for understanding the internals... On Thu, March 8, 2012 6:22 am, adit adit wrote: Let's try to stick only to the internals blogs, ok? If

Re: [PHP-DEV] default charset confusion

2012-03-12 Thread Rasmus Lerdorf
On 03/12/2012 12:51 PM, Stas Malyshev wrote: Hi! But you can't necessarily hardcode the encoding if you are writing portable code. That's a bit like hardcoding a timezone. In order to write portable code you need to give people the ability to localize it. No, it's not like timezone at

Re: [PHP-DEV] Scalar Type Hinting

2012-03-12 Thread Richard Lynch
On Thu, March 8, 2012 5:13 am, Alain Williams wrote: On Thu, Mar 08, 2012 at 11:06:56AM +0200, Arvids Godjuks wrote: Type hints are meant to filter input from external sources Correction, it should read like this: Type hints are _not_ meant to filter input from external sources +1 What

Re: [PHP-DEV] default charset confusion

2012-03-12 Thread Pierre Joye
hi Rasmus, On Mon, Mar 12, 2012 at 9:12 PM, Rasmus Lerdorf ras...@lerdorf.com wrote: If everything was UTF-8 we wouldn't have any of these issues. Unfortunately that isn't the case. The question is what to do with apps that need to deal with non UTF-8 data. Are we going to provide any help

Re: [PHP-DEV] Providing sandboxed versions of include and require language constructs

2012-03-12 Thread Richard Lynch
On Tue, March 6, 2012 3:30 am, Florian Anderiasch wrote: Security by blacklist almost always isn't security... You're bound to miss one of the functions you should have blacklisted, but didn't. Something like Drupal would be crippled by this because major extensions used by all rely on access

Re: [PHP-DEV] default charset confusion

2012-03-12 Thread Ángel González
On 12/03/12 20:51, Stas Malyshev wrote: Hi! But you can't necessarily hardcode the encoding if you are writing portable code. That's a bit like hardcoding a timezone. In order to write portable code you need to give people the ability to localize it. No, it's not like timezone at all. I

Re: [PHP-DEV] default charset confusion

2012-03-12 Thread Stas Malyshev
Hi! If you are a framework developer, and really want to shield against a bad php.ini setting, you could ini_set() to your prefered charset at the beginning of the request. That assuming the request is completely processed by your framework and you never call any outside code and any

Re: [PHP-DEV] [POC - Patch] Scalar Type Hinting - A-La zend_parse_parameters

2012-03-12 Thread Matthew Weier O'Phinney
On 2012-03-12, Arvids Godjuks arvids.godj...@gmail.com wrote: --f46d0442880e02b97f04bb0b432b Content-Type: text/plain; charset=UTF-8 I think that the null issue is not an issue. Strictly speaking if you want null or an int - leave out the type hint and use generic argument that will accept

[PHP-DEV] Release process nit

2012-03-12 Thread Ondřej Surý
Hi guys, could you please remove this cruft: dpkg-source: warning: ignoring deletion of file ext/standard/var_unserializer.c.orig dpkg-source: warning: ignoring deletion of file ext/standard/url_scanner_ex.c.orig dpkg-source: warning: ignoring deletion of file ext/date/lib/parse_date.c.orig

Re: [PHP-DEV] CURL file posting

2012-03-12 Thread Ángel González
On 12/03/12 20:36, Richard Lynch wrote: On Sun, March 11, 2012 6:29 pm, Stas Malyshev wrote: This doesn't look good. Documentation does say the @ prefix exists, but it has very high potential of creating security holes for unsuspecting people. open_basedir would help limit the impact, but

Re: [PHP-DEV] Release process nit

2012-03-12 Thread Pierre Joye
hi! are they in svn? I can't see them in 5.4 Or which release did have them? On Mon, Mar 12, 2012 at 10:52 PM, Ondřej Surý ond...@sury.org wrote: Hi guys, could you please remove this cruft: dpkg-source: warning: ignoring deletion of file ext/standard/var_unserializer.c.orig dpkg-source:

Re: [PHP-DEV] Release process nit

2012-03-12 Thread Stas Malyshev
Hi! are they in svn? I can't see them in 5.4 They are not in SVN, but at least for autom4te.cache ones they seem to be generated when configure script is generated, and the packing script happily picks them up. I have no idea how orig files got there... -- Stanislav Malyshev, Software

Re: [PHP-DEV] Release process nit

2012-03-12 Thread Alexey Shein
13 марта 2012 г. 3:00 пользователь Stas Malyshev smalys...@sugarcrm.com написал: Hi! are they in svn? I can't see them in 5.4 They are not in SVN, but at least for autom4te.cache ones they seem to be generated when configure script is generated, and the packing script happily picks them

Re: [PHP-DEV] Release process nit

2012-03-12 Thread Christopher Jones
On 03/12/2012 03:06 PM, Alexey Shein wrote: 13 марта 2012 г. 3:00 пользователь Stas Malyshev smalys...@sugarcrm.com написал: Hi! are they in svn? I can't see them in 5.4 They are not in SVN, but at least for autom4te.cache ones they seem to be generated when configure script is

Re: [PHP-DEV] Release process nit

2012-03-12 Thread Ondřej Surý
On Mon, Mar 12, 2012 at 23:06, Alexey Shein con...@gmail.com wrote: 13 марта 2012 г. 3:00 пользователь Stas Malyshev smalys...@sugarcrm.com написал: Hi! are they in svn? I can't see them in 5.4 They are not in SVN, but at least for autom4te.cache ones they seem to be generated when

Re: [PHP-DEV] Providing sandboxed versions of include and require language constructs

2012-03-12 Thread Adam Jon Richardson
On Mon, Mar 12, 2012 at 5:08 PM, Richard Lynch c...@l-i-e.com wrote: On Tue, March 6, 2012 3:30 am, Florian Anderiasch wrote: Security by blacklist almost always isn't security... You're bound to miss one of the functions you should have blacklisted, but didn't. Agreed. The approach I'm

Re: [PHP-DEV] Release process nit

2012-03-12 Thread Stas Malyshev
Hi1 The autom4te.cache and *.orig files originally mentioned are included in php.net's php-5.4.0.tar.bz2 I.e. this is a valid issue. Definitely seems to be a bug in the makedist script, since these files are not in the SVN but appear when packaging. -- Stanislav Malyshev, Software

Re: [PHP-DEV] default charset confusion

2012-03-12 Thread Ángel González
On 12/03/12 22:30, Stas Malyshev wrote: Hi! If you are a framework developer, and really want to shield against a bad php.ini setting, you could ini_set() to your prefered charset at the beginning of the request. That assuming the request is completely processed by your framework and you

Re: [PHP-DEV] default charset confusion

2012-03-12 Thread Stas Malyshev
Hi! Still, that API is likely wrong: a library function written by someone completely unrelated to the main application shouldn't be echoing anything through the output. And if it's not generating the html, the htmlspecialchars is better done from the return at the calling application (probably

Re: [PHP-DEV] default charset confusion

2012-03-12 Thread Yasuo Ohgaki
2012/3/13 Rasmus Lerdorf ras...@lerdorf.com: On 03/12/2012 03:05 AM, Yasuo Ohgaki wrote: I thought default_charset became UTF-8, so I was expecting following HTTP header. content-type  text/html; charset=UTF-8 However, I got empty charset (missing 'charset=UTF-8'). So I looked up to source

Re: [PHP-DEV] default charset confusion

2012-03-12 Thread Rasmus Lerdorf
On 03/12/2012 05:52 PM, Yasuo Ohgaki wrote: I always set all parameters for htmlentities/htmlspecialchars, therefore I haven't noticed this was changed from 5.3. They may be migrating from 5.2 or older. (RHEL5 uses 5.1) No, like I showed, moving from 5.3 to 5.4 breaks because the new default