[PHP-DEV] hash_hkdf() signature and return value

[Yasuo Ohgaki]

Hi all,

This is the last recommendation for hash_hkdf[1]. In fact,
this would be the last chance to fix because we'll have 7.2 soon.
The issue is secure usage and API consistency.

Currently hash_hkdf() has following signature:

 hash_hkdf(string $algo , string $ikm [, int $length = 0 [,
string $info = '' [, string $salt = '' ]]] )

These are rationals behind recommendation. There are many, but
please read on.

=== Parameter Order ===

HKDF[2] algorithm is:
 1. General purpose key derivation function as per RFC 5869
 2. "$salt" parameter is a "pre-shared _KEY_" in many cases as mentioned
RFC 5869
 3. "$salt" (or preshared key) is very strongly recommended for security
season as per RFC 5869
 4. Supplying salt that the same length of input key does not affect
performance as per RFC 5969
 5. "$info" is what makes HKDF useful, but it's less important as described
in RFC 5869
 6. "$length" is truly an optional parameter for very specific encryption
algorithm or usage.

Rationale for change:
 1. Key derivations without secondary key ($salt) does not make sense when
 secondary key could be available. HKDF is designed for best possible
key
 security with the key. Not using secondary key ($salt) simply
downgrades
 key security without no reason. i.e. HKDF performance is the same
 when $salt has the same as hash is set.
 2. HKDF is based on HMAC. When $info has no use, HMAC would be the best
 choice for it. i.e. $newkey = hash_hmac($ikm, $key);
 3. It does not make sense violating RFC recommendations for a RFC
implementation.

>From these facts and reasons, $salt, $info and $length parameter order and
requirement should be changed from

string $algo , string $ikm [, int $length = 0 [, string $info = '' [,
string $salt = '' ]]]

to

string $algo , string $ikm , string $salt, string $info = '' [, int $length
= 0 ]
Note: Users can set empty string if they really don't need $salt and/or
$info.

Conclusion:
This way, users would have better chances to use hash_hkdf() more securely
and
properly.

[1] http://php.net/hash_hkdf
[2] http://www.faqs.org/rfcs/rfc5869.html

=== Return Value and Output Option ===

The most common HKDF usage with PHP would be:
 1. CSRF token generation that is specific to a request with expiration
time.
 (HEX return value would be appropriate, not BINARY)
 2. API access token generation that does not transmit "The API Key", but
 derived key by HKDF. It also should have expiration time.
 (HEX return value would be appropriate, not BINARY)

Consistency with other hash_*() functions:
 1. All of other hash_*()  returns HEX string hash value.
 2. hash_hkdf() is the only one returns BINARY hash value.

Conclusion:
hash_hkdf() should return HEX by default, not BINARY.
Optional [, bool $raw_output = false ] should be added just like other
hash_*().


=== Compatibility ===

IMHO, current hash_hkdf() should not be added by PHP 7.1.2, but 7.2.0
in the first place. The mess could be resolved by 7.2.

Anyway, hash_hkdf() is added 7.1.2. Hopefully not many users are
using it yet. If we change API with 7.2 release, there would be least
possible confusions. (We may remove it from 7.1 to avoid confusions, too)

Our choices:
 - Keep the current insecure/inconsistent API forever.
 - Change the API to have secure/consistent API forever.

Conclusion:
No conclusion for this. There would be conflicting options.


I strongly think it is not worth to keep this insecure/inconsistent API
forever.
I prefer to change the API to what it should be.

What should we do for this?
Comments?



P.S.

Nikita, you've said following during HKDF discussion:
 - HKDF for CSRF tokens/etc does not make sense at all.
 - Current parameter order and return value makes perfect sense
   and has valid/common usages.
 - Everyone one this list, shouldn't listen to me because I'm insane and
   totally misunderstood what the HKDF is.

Phrases are not the exact, but it should be correct enough. Some part
is my fault, w/o reading mail and/or poor English. I blindly assumed
you've understand RFC 5869 and possible common usages with PHP. I
apologized for the confusion and tried to explain why w/o success.

If you still believe what you've said is correct, I don't request you to
take these back, show us at least one common/reasonable hash_hkdf()
usage example for current API, and point out what's wrong my recommendations
and rationales above.

If not, I request you to take it back. I respect your contributions much,
but
the last one you've said is out of tolerance.

--
Yasuo Ohgaki
yohg...@ohgaki.net

Re: [PHP-DEV] [VOTE] UUID

[Sammy Kaye Powers]

> Richard wrote:
> Maybe I should stop the vote. The discussion is happening now instead of
> before when I asked for it.

I'm +1 for this because I'd love to see UUID's get added to PHP 7.3! :)

Thanks,
Sammy Kaye Powers
sammyk.me

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] A validator module for PHP7

[Yasuo Ohgaki]

Hi Rowan, Crocodile and Lester,

Rowan,
Thank you for helpful suggestions. I think it may better be named
'validate()'/etc.
Sole reason I named functions as 'valid*()' is name collision.
'validate*()' seems
too common, but users should use namespace to avoid collisions. I'll take
your
suggestions into the module.

Crocodile,
I agree your comment. However, my last filter module improvement RFC
with PoC patch was failed. Upside having new module is we can enforce more
strict validations and can have much simpler/extensible API with cleaner
codes.
(Having both 'filter' and 'validator' in one code is mess)
Since the last attempt was failed, new module would be the next reasonable
attempt.

On Wed, Sep 6, 2017 at 4:39 AM, Lester Caine  wrote:

> On 05/09/17 20:05, Yasuo Ohgaki wrote:
> > There is one principle that developers is better to follow.
> > https://en.wikipedia.org/wiki/Fail-fast
> > If we follow this principle, validation at controller makes sense.
>
Oops. A 'is' should be 'are'.

> Since a large proportion of the data coming in is a result of input into
> a previously generated form, the data can often be validated in terms of
> basic structure before even needing to decide if the data set needs to
> be iterated? If things like 'maximum data size' can be established when
> the form is created, any data set larger than that can simply be killed
> off.
>

> Anyway, thank you for pointer for PDO validation. I didn't notice the
> > project. We may cooperate so that there aren't unnecessary validaiton
> > rule incompatibilities
> I've been pushing the idea of a single method of managing metadata for a
> long time. Most of the 'checking' loading down PHP now still misses the
> point and the database style of rules has been around since long before
> PDO and other abstractions messed it up. A single standard set of rules
> that can be used across the board from variable creation to checking
> data going out to forms and returns coming back and data between
> operations such as database activity. This is NOT 'typing' since that
> lacks the vast majority of checks that a decent validation will handle,
> but the much finer details such as limits and value sets. There is a
> vast discrepancy in how this is handled across databases, but the SQL
> standard does provide a base which databases are slowly evolving towards.
>
>
It's nice to have central input type (Not only data type, but length,
chars, format, range and char encoding, as you mentioned) repository for an
app. That's the reason why I would like to cooperate with other validator
implementation(s) to avoid unnecessary incompatibilities.

(For security perspective, it can be said different types of input
validations is
better because if one failed, another may validate data correctly. However,
managing multiple validation rules is burden. Some people try to validate
inputs with WAF. However, I suggest input validation is better to be
implemented
at web apps, not WAF.
It's a lot easier and maintainable if input validation is done by app. App
developers know what the input should be exactly. In addition, app must
validate inputs regardless of WAF. I'm not saying WAF is useless. WAF
is still useful as network layer protection. I'm saying multiple layer
validations
are recommended, but it can be unmanageable easily unless some trade
off is taken)

I agree that PDO level validation is almost mandatory, especially for
SQLite3.
SQLite3 data type is pseudo type and allows any "characters". e.g. Int type
can store 'alert("XSS")', etc.

My validate module allows users to have central input type repository by
simple native PHP array. Array could be stored anywhere users want.
I think it will work as you wish if 'validate' module is compatible with
PDO
validator.

For the time being, I'm not sure what it should be. Data specifications for
database may be stored as additional info in 'validate' spec array, or PDO
validator may simply assume data types specified in 'validate' spec array
should be enforced, and use 'string' data spec as required format,
or 'validate' module may expose API so that PDO validator can use it for
basic PHP data type validation.

Regards,

--
Yasuo Ohgaki

Re: [PHP-DEV] [VOTE] UUID

[Fleshgrinder]

On 9/5/2017 9:32 PM, Andreas Heigl wrote:
> I'm well aware of that and perhaps I didn't express myself as clear as I
> should have.
> 
> Imagine a use-case where a UUID-class is needed. But alongside the
> toString, toHex and toBinary there's also the need for a further
> function (let's call it toArray). So currently I need to create a
> wrapper arround UUID that then needs to implement all the public methods
> of UUID as well as the new toArray. So it works identically to UUID but
> it isn't UUID. And I have no way of using my own UUID-Class - as it
> doesnt' extend UUID - as replacement for UUID. I'd need to expose the
> wrapped UUID-Class to be able to retrieve it whenever some libray
> expects a UUID. Perhaps this gist can make it clearer:
> https://gist.github.com/heiglandreas/452dae591d071cbdfb78b431cb6597fa
> 
> I'm not saying it's the wrong choice. I for myself would probably not
> immediately use it as the ramsey/uuid-package is widely in use, but I
> could f.e. think, that that package might start to use the UUID-class
> under the hood. And then that would be a case where extending could be
> helpful as a \Ramsey\UUID would be an instance of \UUID.
> 
> The alternative would be to implement a UUIDInterface that exposes the
> relevant methods and that would be implemented by \UUID itself.
> 
> But that's just my 0.02€
> 
> Cheers
> 
> Andreas
> 

OK, now I understand it better. I would argue that if we really find
something existential that should be added, we should add it to the UUID
class itself.

See, the problem with allowing extension is that we have a real BC
issue. All your arguments are well received and correct, but the open a
can of worms that is impossible to close. Keeping it final ensures that
this cannot happen, ever. We can continue to refine without breaking
anyone. I think it also was Ocramius who released a nice article about
"final first", but there are probably many from the Java world.

Btw. the interface does not really make sense. Interfaces are for
polymorphism, in other words, if there are different implementations of
the same thing that should be usable interchangeably. This is definitely
not the case with UUIDs, the algorithm is set in stone. Don't forget
that you can instantiate any kind of UUID with the `fromBinary` method,
so you can easily generate different UUIDs on your own and still use the
built-in class; no need for extension.

-- 
Richard "Fleshgrinder" Fussenegger



signature.asc
Description: OpenPGP digital signature

Re: [PHP-DEV] A validator module for PHP7

[Lester Caine]

On 05/09/17 20:05, Yasuo Ohgaki wrote:
> There is one principle that developers is better to follow.
> https://en.wikipedia.org/wiki/Fail-fast
> If we follow this principle, validation at controller makes sense.
Since a large proportion of the data coming in is a result of input into
a previously generated form, the data can often be validated in terms of
basic structure before even needing to decide if the data set needs to
be iterated? If things like 'maximum data size' can be established when
the form is created, any data set larger than that can simply be killed
off.

> Anyway, thank you for pointer for PDO validation. I didn't notice the
> project. We may cooperate so that there aren't unnecessary validaiton
> rule incompatibilities
I've been pushing the idea of a single method of managing metadata for a
long time. Most of the 'checking' loading down PHP now still misses the
point and the database style of rules has been around since long before
PDO and other abstractions messed it up. A single standard set of rules
that can be used across the board from variable creation to checking
data going out to forms and returns coming back and data between
operations such as database activity. This is NOT 'typing' since that
lacks the vast majority of checks that a decent validation will handle,
but the much finer details such as limits and value sets. There is a
vast discrepancy in how this is handled across databases, but the SQL
standard does provide a base which databases are slowly evolving towards.

-- 
Lester Caine - G8HFL
-
Contact - http://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - http://lsces.co.uk
EnquirySolve - http://enquirysolve.com/
Model Engineers Digital Workshop - http://medw.co.uk
Rainbow Digital Media - http://rainbowdigitalmedia.co.uk

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] [VOTE] UUID

[Andreas Heigl]

Hey Richard

Am 05.09.17 um 19:29 schrieb Fleshgrinder:
> On 9/5/2017 7:01 PM, Andreas Heigl wrote:
>> Hey Richard, Hey all.
>>
>> Thanks for putting up the RFC and the implementation!
>>
>> Having UUIDs in the core would be awesome. One of the reasons would be
>> that it's in C. That'd be faster but would also allow easier integration
>> with system-calls to be easier able to get the MAC-Address for a type-1
>> UUID f.e.
>>
>> But why limit UUIDs to type 3 through 5? Having security in mind is good
>> IMHO but not implementing type 1 and 2 limits the usability and
>> therefore the usefullness. Let the users decide whether they need it or
>> not. As long as people can create SQL-Injections in PHP we should not
>> use the security argument to limit usability or not implement
>> standardized functionality.
>>
>> Especially when there is a full-fledged reference-implementation in
>> userland available. In the RFC you reference ramsey/uuid yourself. But
>> why should one use the internal UUID-class/functionality when there is a
>> more powerful one in userland available?
>>
>> And limiting the usability and extendability of the UUID-Class itself by
>> declaring it final means that userland-code can only wrap the class but
>> not extend it. So userland code can not typehint for the UUID-class when
>> special features are necessary that would need extending the class. And
>> as there's no interface, I can't typehint for that either.
>>
>> So all in all for me that's
>>
>> * less functionality than the reference-implementation
>> * missing UUID-types (the ones that are easier to implement in C)
>> * missing extendability
>>
>> and a naging feeling that the imlementation decides what I as a user
>> actually need.
>>
>> That's why I can't vote "yes".
>>
>> Sorry.
>>
>> Thanks for putting in the effort and coming up with a first implementation.
>>
>> Cheers
>>
>> Andreas
>>
>> PS: Personally I don't like a "uuid_4_create" as that would also mean
>> there should be a "uuid_1_create" through "uuid_5_create" as well and
>> then there also would need to be a "uuid_1_verify" through "uuid_5_verify"…
>>
> 
> Hi Andreas!
> 
> Thanks for your feedback.
> 
> We can easily add v1 and v2 because the class is final. It would not be
> a breaking change, or anything. v2 is pretty much useless imho, but v1
> if done right would not even harm your privacy.

I'm with you there in so far that I personally don't see a value in a
type 2 UUID. But there might be people that need exactly that. And as
adding functionality usually isn't a BC-break that's OK.

> 
> Composition is more powerful than inheritance. You mention that you
> cannot extend it to add functionality, at the same time you want to
> type-hint against it. Well, in order to use the extended functionality
> you need to type-hint against your extended version. Hence, there is
> zero value for you in extending it other than having some place using
> the extended version, and others the core version without noticing that
> it got the extended version.

I'm well aware of that and perhaps I didn't express myself as clear as I
should have.

Imagine a use-case where a UUID-class is needed. But alongside the
toString, toHex and toBinary there's also the need for a further
function (let's call it toArray). So currently I need to create a
wrapper arround UUID that then needs to implement all the public methods
of UUID as well as the new toArray. So it works identically to UUID but
it isn't UUID. And I have no way of using my own UUID-Class - as it
doesnt' extend UUID - as replacement for UUID. I'd need to expose the
wrapped UUID-Class to be able to retrieve it whenever some libray
expects a UUID. Perhaps this gist can make it clearer:
https://gist.github.com/heiglandreas/452dae591d071cbdfb78b431cb6597fa

> 
> The thing is, you should create your own value objects for your
> identifiers and hide the fact what it wraps. In C, and many other
> languages, we have type aliases. In PHP, and many other OO languages, we
> use composition to achieve tha>
> Whether to make it final or not was discussed, and especially Ocramius
> agreed with me on this. I believe that it is the right choice.

I'm not saying it's the wrong choice. I for myself would probably not
immediately use it as the ramsey/uuid-package is widely in use, but I
could f.e. think, that that package might start to use the UUID-class
under the hood. And then that would be a case where extending could be
helpful as a \Ramsey\UUID would be an instance of \UUID.

The alternative would be to implement a UUIDInterface that exposes the
relevant methods and that would be implemented by \UUID itself.

But that's just my 0.02€

Cheers

Andreas

-- 
  ,,,
 (o o)
+-ooO-(_)-Ooo-+
| Andreas Heigl   |
| 

Re: [PHP-DEV] A validator module for PHP7

[Yasuo Ohgaki]

Hi Lester,

I always make some mistakes. s/are/is

There is one principle that developers is better to follow.
https://en.wikipedia.org/wiki/Fail-fast
If we follow this principle, validation at controller makes sense.

Anyway, thank you for pointer for PDO validation. I didn't notice the
project. We may cooperate so that there aren't unnecessary validaiton
rule incompatibilities.

Regards,

--
Yasuo Ohgaki
yohg...@ohgaki.net

Re: [PHP-DEV] A validator module for PHP7

[Yasuo Ohgaki]

Hi Lester,

On Tue, Sep 5, 2017 at 8:36 PM, Lester Caine  wrote:

> On 05/09/17 12:18, Yasuo Ohgaki wrote:
> > I cannot guess people's thought. I appreciated feedback!
>
> With a decent database layer a lot of the validation you are proposing
> is already covered but PDO does not help in this area. Adding another
> layer that does not integrate with a storage layer is just adding to the
> current mess ...
>

I'm fun of multiple tier and multiple layer of protections.
For instance, Microsoft's SQL injection security page states as follows.

 - Never build Transact-SQL statements directly from user input; use stored
procedures to validate user input.

 - Validate user input by testing type, length, format, and range. Use the
Transact-SQL QUOTENAME() function to escape system names or the REPLACE()
function to escape any character in a string.

 - Implement multiple layers of validation in each tier of your application.

https://docs.microsoft.com/en-us/dotnet/framework/data/adonet/sql/writing-secure-dynamic-sql-in-sql-server

This is what secure coding practice recommends, too.
It may seem mess, but it's not. Outermost trust boundary that can be
controlled
is the most important trust boundary. For server side web app developers,
outermost
trust boundary is controller in MVC model. Input validations at model is a
bit too late
to mitigate risks involved with invalid(attacker) inputs.

Both model and controller layer Input validations (as well as in the
database, too) are
good/important to have.

There are one principle that developers are better to follow.
https://en.wikipedia.org/wiki/Fail-fast
If we follow this principle, validation at controller makes sense.

Regards,

P.S. For database administrators or web app developers who maintain
application
Models, outermost trust boundary is "database system" and "the Model layer"
respectively.
Outermost trust boundary is changed by what they can control.

This kind of discussion could result in mess. I hope I explained well
enough.

--
Yasuo Ohgaki
yohg...@ohgaki.net

Re: [PHP-DEV] [VOTE] UUID

[Fleshgrinder]

I'm sorry if my replies sound pissed. I'm generally not good at being
diplomatic in writing or talking, and currently under a lot of stress in
real life (which is of course not the problem of you guys).

Please bare with me, I honor all constructive feedback I receive, truly!

-- 
Richard "Fleshgrinder" Fussenegger



signature.asc
Description: OpenPGP digital signature

Re: [PHP-DEV] [VOTE] UUID

[Fleshgrinder]

On 9/5/2017 7:01 PM, Andreas Heigl wrote:
> Hey Richard, Hey all.
> 
> Thanks for putting up the RFC and the implementation!
> 
> Having UUIDs in the core would be awesome. One of the reasons would be
> that it's in C. That'd be faster but would also allow easier integration
> with system-calls to be easier able to get the MAC-Address for a type-1
> UUID f.e.
> 
> But why limit UUIDs to type 3 through 5? Having security in mind is good
> IMHO but not implementing type 1 and 2 limits the usability and
> therefore the usefullness. Let the users decide whether they need it or
> not. As long as people can create SQL-Injections in PHP we should not
> use the security argument to limit usability or not implement
> standardized functionality.
> 
> Especially when there is a full-fledged reference-implementation in
> userland available. In the RFC you reference ramsey/uuid yourself. But
> why should one use the internal UUID-class/functionality when there is a
> more powerful one in userland available?
> 
> And limiting the usability and extendability of the UUID-Class itself by
> declaring it final means that userland-code can only wrap the class but
> not extend it. So userland code can not typehint for the UUID-class when
> special features are necessary that would need extending the class. And
> as there's no interface, I can't typehint for that either.
> 
> So all in all for me that's
> 
> * less functionality than the reference-implementation
> * missing UUID-types (the ones that are easier to implement in C)
> * missing extendability
> 
> and a naging feeling that the imlementation decides what I as a user
> actually need.
> 
> That's why I can't vote "yes".
> 
> Sorry.
> 
> Thanks for putting in the effort and coming up with a first implementation.
> 
> Cheers
> 
> Andreas
> 
> PS: Personally I don't like a "uuid_4_create" as that would also mean
> there should be a "uuid_1_create" through "uuid_5_create" as well and
> then there also would need to be a "uuid_1_verify" through "uuid_5_verify"…
> 

Hi Andreas!

Thanks for your feedback.

We can easily add v1 and v2 because the class is final. It would not be
a breaking change, or anything. v2 is pretty much useless imho, but v1
if done right would not even harm your privacy.

Composition is more powerful than inheritance. You mention that you
cannot extend it to add functionality, at the same time you want to
type-hint against it. Well, in order to use the extended functionality
you need to type-hint against your extended version. Hence, there is
zero value for you in extending it other than having some place using
the extended version, and others the core version without noticing that
it got the extended version.

The thing is, you should create your own value objects for your
identifiers and hide the fact what it wraps. In C, and many other
languages, we have type aliases. In PHP, and many other OO languages, we
use composition to achieve that.

Whether to make it final or not was discussed, and especially Ocramius
agreed with me on this. I believe that it is the right choice.

-- 
Richard "Fleshgrinder" Fussenegger



signature.asc
Description: OpenPGP digital signature

Re: [PHP-DEV] [VOTE] UUID

[Fleshgrinder]

On 9/4/2017 11:04 AM, Zeev Suraski wrote:
> Richard,
> 
> I'm not accusing you of anything.  This is all in positive constructive 
> spirit, and I was the first to admit I may have missed something - and 
> although at this point I don't think I did, that's still a possibility.
> 

Sorry, than I misunderstood. I interpreted it in the way that you meant
that I did something wrong by bringing this to vote.

On 9/4/2017 11:04 AM, Zeev Suraski wrote:
> My point is simple - when I saw the vote, I looked for the prior discussion 
> on internals - which is the *only* official channel for discussing these 
> matters.  The only discussion I could find took place between May 24 and May 
> 26 - i.e. well over 3 months ago.  While being intense, it raised good points 
> which remained unanswered, and it died out very quickly without any sort of 
> followup.  Again, I have no idea what kind of discussion happened on reddit 
> or IRC or other channels, but that shouldn't matter.
> 

Maybe I should stop the vote. The discussion is happening now instead of
before when I asked for it. We'll have to wait for at least six months
for another vote if this is a no, due to the rules.

On 9/4/2017 11:04 AM, Zeev Suraski wrote:
> That's great, but given that it's unprecedented, it's not a very good 
> argument.  To quote Marco from the May discussion:
> "Introducing a UUID class in PHP core as *just* a type safety wrapper feels 
> akin to adding an EmailAddress class to core. It's certainly not an 
> unreasonable way to handle things, but imho also not something that should be 
> in core. We should be providing the primitives based on which people can 
> implement whichever abstractions they prefer, in a simpler and more flexible 
> manner than we can achieve in extension code."
> 

I disagree with that comment. I think that PHP is a high-level language,
and thus should provide high-level abstractions that fulfill the most
basic needs. A UUID is not a string, I'll go into this below.

On 9/4/2017 11:04 AM, Zeev Suraski wrote:
> There's a difference between certain people saying something, and 'most 
> people think'.  There were only about 15 people that participated in this 
> thread, and of those, I couldn't find any that said that performance is a 
> weak argument.  Most didn't comment about performance at all.  
> 
> I could find some that said the opposite, including Ben Ramsey:
> "A UUID generator in the core will only help to improve ramsey/uuid, 
> providing more choice and better performance."
> The 'only' there might be confusing, but it's intended in a positive way.
> 
> I fail to see how it's possible to derive from that thread a statement that 
> 'performance is a weak argument', and I do think it's bad to have a ratified 
> php.net RFC that would make that statement as if it's an obvious truth.
> 

That was probably on some other channel. As I said, I'm more than happy
to change that wording. =)

On 9/4/2017 11:04 AM, Zeev Suraski wrote:
> First, I do not prefer procedural programming.  Personally I use OO a lot 
> more than I use procedural.  This is, however, completely besides the point - 
> when designing and maintaining PHP, I put my personal preferences aside and 
> try to think about what's right and consistent for the language.  I think 
> everyone who contributes should do the same.
> Secondly, and very much related, suggesting "I'll do half the job, you can do 
> the other half if you want" is very much the wrong approach IMHO.  When 
> introducing a new feature, we should strive to make it consistent across the 
> board, catering to the wide range of users in our community, and not half 
> baked according to the individual preferences of the subsets of the language 
> one likes.
> Thirdly, there's nothing inherently confusing about procedural APIs, or 
> inherently clear about OO APIs.  Yes, some of our legacy APIs are a mess, and 
> it's a tough problem to tackle - but this has nothing to do with not wanting 
> to introduce a procedural API for creating a UUID.  The procedural/OO duality 
> is not at all what people complain about when griping about PHP's API mess.
> Last, yes, the rationale is indeed true for most additions to the language.  
> The 2/3 barrier, as is explained in the Voting RFC (wiki.php.net/rfc/voting), 
> has a rationale - the rationale being that unlike changes in apps or 
> frameworks, changes to the language have an immense cost of reversal or 
> incompatible alteration.  Adding a top level object that's four letters long 
> falls squarely within that definition - unlike, say, deciding when to release 
> version X, or whether to call version Y "8.0" or "10.0".  Looking at it from 
> the other end - fast forward into 2021 a world where the current UUID 
> proposal is accepted as-is, would we feel comfortable deprecating it with 
> 50%+1 majority?  If the answer's no, introducing it shouldn't be at 50%+1 
> either.
> 
> Zeev
> 
> 

I'm not going to propose a procedural API, because I 

Re: [PHP-DEV] A validator module for PHP7

[li...@rhsoft.net]

Am 05.09.2017 um 18:57 schrieb Lester Caine:

But not at the cost of writing different sets of code to play to each
area where checking SHOULD be done. Stick to a single standard method of
defining the metadata and that already exists in the database layer


ok, to make that point clear:

not every input which needs to be validated or sanitized is *related to 
a database at all* and hence input validation can't be done in the 
database-layer and only there by definition for everything


frankly since 3 weeks our core-application is at a level where the 
database-layer get not loaded at all until inputs are not verfified 
because database stuff is on-demand and 80% of all requests when there 
is some traffic within 80% seconds don't load the database layer because 
of smart caching


form-input is validated, checked against CSRF-tokens, captcha and 
*after* all the validations are fine the database layer becomes part of 
the game - that alone brought again 43% higher requests/second on a 
already highly optimizied codebase



--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] [VOTE] UUID

[Andreas Heigl]

Am 02.09.17 um 09:01 schrieb Fleshgrinder:
> Hello Internals!
> 
> I just started the voting for the inclusion of a UUID value object in
> PHP's core, targeting PHP 7.3. I wanted to start earlier, but was sick
> the whole week.
> 
> The voting is open starting now and until September 16. (2 weeks).
> 
Hey Richard, Hey all.

Thanks for putting up the RFC and the implementation!

Having UUIDs in the core would be awesome. One of the reasons would be
that it's in C. That'd be faster but would also allow easier integration
with system-calls to be easier able to get the MAC-Address for a type-1
UUID f.e.

But why limit UUIDs to type 3 through 5? Having security in mind is good
IMHO but not implementing type 1 and 2 limits the usability and
therefore the usefullness. Let the users decide whether they need it or
not. As long as people can create SQL-Injections in PHP we should not
use the security argument to limit usability or not implement
standardized functionality.

Especially when there is a full-fledged reference-implementation in
userland available. In the RFC you reference ramsey/uuid yourself. But
why should one use the internal UUID-class/functionality when there is a
more powerful one in userland available?

And limiting the usability and extendability of the UUID-Class itself by
declaring it final means that userland-code can only wrap the class but
not extend it. So userland code can not typehint for the UUID-class when
special features are necessary that would need extending the class. And
as there's no interface, I can't typehint for that either.

So all in all for me that's

* less functionality than the reference-implementation
* missing UUID-types (the ones that are easier to implement in C)
* missing extendability

and a naging feeling that the imlementation decides what I as a user
actually need.

That's why I can't vote "yes".

Sorry.

Thanks for putting in the effort and coming up with a first implementation.

Cheers

Andreas

PS: Personally I don't like a "uuid_4_create" as that would also mean
there should be a "uuid_1_create" through "uuid_5_create" as well and
then there also would need to be a "uuid_1_verify" through "uuid_5_verify"…

-- 
  ,,,
 (o o)
+-ooO-(_)-Ooo-+
| Andreas Heigl   |
| mailto:andr...@heigl.org  N 50°22'59.5" E 08°23'58" |
| http://andreas.heigl.org   http://hei.gl/wiFKy7 |
+-+
| http://hei.gl/root-ca   |
+-+



signature.asc
Description: OpenPGP digital signature

Re: [PHP-DEV] A validator module for PHP7

[Lester Caine]

On 05/09/17 15:13, li...@rhsoft.net wrote:
> your first error is thinking every input is related to databases at all
So we end up with different code for different types of input?
An array that will work directly into a database save or some other
follow on process without having to think about where the input comes
from has to be the right way ...

>> Copying all that data and manually creating filter rules is
>> just unnecessary work. In addition much of the VALIDATION is best done
>> at the browser end, and building that code is a lot easier when there is
>> a standard validation base across all of the layers!
> 
> NO VALIDATION is best done in the browser end because no attacker ever
> will execute your client side validation code or operate a browser at all
Again ... write different code for each area of checking?
My clients are complaining that the browser is not doing as good a job
as it could checking things that it CAN check before passing it back to
the server. YES the server needs to cross check no bugger has bypassed
the browser checks but if the set of data is EXPECTED to have a clean
format, then any corruption can be tagged as a failure, because we have
standard rules on what we are passing.

>> Rejecting crap from hackers that have no format matching the fields on
>> the browser page is something else and if the data set is corrupt then
>> yes you can simply skip out before doing anything with it!
> 
> and that's what the whole topic is about
But not at the cost of writing different sets of code to play to each
area where checking SHOULD be done. Stick to a single standard method of
defining the metadata and that already exists in the database layer.

That was what the topic was all about 15 years ago and nothing much has
changed since ... and annotating that data in the code ...

-- 
Lester Caine - G8HFL
-
Contact - http://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - http://lsces.co.uk
EnquirySolve - http://enquirysolve.com/
Model Engineers Digital Workshop - http://medw.co.uk
Rainbow Digital Media - http://rainbowdigitalmedia.co.uk

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] A validator module for PHP7

[li...@rhsoft.net]

Am 05.09.2017 um 15:44 schrieb Lester Caine:

On 05/09/17 14:08, li...@rhsoft.net wrote:

the only question as applicaton developer is how you proceed in which cases

* reject the whole request with a error-message
* reset form-fields where you don't expect an array as input
* reset from-fields with out-of-range input values

here you go:
https://en.wikipedia.org/wiki/Data_validation


When the database layer provides a complete list of fields and
validation rules as part of it's meta data, it is integral to any GOOD
process


your first error is thinking every input is related to databases at all


Copying all that data and manually creating filter rules is
just unnecessary work. In addition much of the VALIDATION is best done
at the browser end, and building that code is a lot easier when there is
a standard validation base across all of the layers!


NO VALIDATION is best done in the browser end because no attacker ever 
will execute your clientside validation code or operate a browser at all



Rejecting crap from hackers that have no format matching the fields on
the browser page is something else and if the data set is corrupt then
yes you can simply skip out before doing anything with it!


and that's what the whole topic is about

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] A validator module for PHP7

[Lester Caine]

On 05/09/17 14:08, li...@rhsoft.net wrote:
> 
> 
> Am 05.09.2017 um 13:36 schrieb Lester Caine:
>> On 05/09/17 12:18, Yasuo Ohgaki wrote:
>>> I cannot guess people's thought. I appreciated feedback!
>>
>> With a decent database layer a lot of the validation you are proposing
>> is already covered but PDO does not help in this area. Adding another
>> layer that does not integrate with a storage layer is just adding to the
>> current mess ...
> 
> sorry, but you confuse "input validation" which this topic is about with
> something different - input validation and reject bad requests belongs
> some layers on top of any storage and should be done as soon as possible
> 
> that should even happen long before you open a database connection at
> all because when you know the request is bad soon enough you won't talk
> to any database, filesystem or whatever storage layer at all
> 
> the only question as applicaton developer is how you proceed in which cases
> 
> * reject the whole request with a error-message
> * reset form-fields where you don't expect an array as input
> * reset from-fields with out-of-range input values
> 
> here you go:
> https://en.wikipedia.org/wiki/Data_validation

When the database layer provides a complete list of fields and
validation rules as part of it's meta data, it is integral to any GOOD
process. Copying all that data and manually creating filter rules is
just unnecessary work. In addition much of the VALIDATION is best done
at the browser end, and building that code is a lot easier when there is
a standard validation base across all of the layers!

Rejecting crap from hackers that have no format matching the fields on
the browser page is something else and if the data set is corrupt then
yes you can simply skip out before doing anything with it! But the
problem these days is when hackers try injecting things like SQL into
fields they think may be able to get through to the database. Provided
that the validation layer can properly filter that injection requires
knowledge that a string has reason to be rejected. Just as simply type
casting a number to integer or float is only doing a small part of the job.

Typing and validating a field by the metadata constraints has to be the
right way forward?

-- 
Lester Caine - G8HFL
-
Contact - http://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - http://lsces.co.uk
EnquirySolve - http://enquirysolve.com/
Model Engineers Digital Workshop - http://medw.co.uk
Rainbow Digital Media - http://rainbowdigitalmedia.co.uk

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] A validator module for PHP7

[li...@rhsoft.net]

Am 05.09.2017 um 13:36 schrieb Lester Caine:

On 05/09/17 12:18, Yasuo Ohgaki wrote:

I cannot guess people's thought. I appreciated feedback!


With a decent database layer a lot of the validation you are proposing
is already covered but PDO does not help in this area. Adding another
layer that does not integrate with a storage layer is just adding to the
current mess ...


sorry, but you confuse "input validation" which this topic is about with 
something different - input validation and reject bad requests belongs 
some layers on top of any storage and should be done as soon as possible


that should even happen long before you open a database connection at 
all because when you know the request is bad soon enough you won't talk 
to any database, filesystem or whatever storage layer at all



the only question as applicaton developer is how you proceed in which cases

* reject the whole request with a error-message
* reset form-fields where you don't expect an array as input
* reset from-fields with out-of-range input values

here you go:
https://en.wikipedia.org/wiki/Data_validation

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] A validator module for PHP7

[Rowan Collins]

On 5 September 2017 12:36:42 BST, Lester Caine  wrote:
>On 05/09/17 12:18, Yasuo Ohgaki wrote:
>> I cannot guess people's thought. I appreciated feedback!
>
>With a decent database layer a lot of the validation you are proposing
>is already covered but PDO does not help in this area. Adding another
>layer that does not integrate with a storage layer is just adding to
>the
>current mess ...

Validation should have nothing to do with the storage layer. Or at least, there 
should be a level of validation separate from the storage layer.

Inputs may come from all sorts of sources: the HTTP request, an API, an import 
file, etc; and they may be going to all sorts of destinations: the HTTP 
response, a database, an API, an export file, an e-mail, etc. 

Regardless of where it came from, and where it's going to end up, the 
application knows what format that input data should be in to use as or 
populate appropriate domain models. For instance, "age_in_years is a 
non-negative integer" is an invariant fact about the domain being modelled, 
even if it's a value that goes nowhere near any form of database.

I actually agree that this module doesn't need to replace the existing userland 
libraries, only to act as a useful base for them, as well as a useful fallback 
when writing "raw PHP". The key problem is balancing flexibility and usability 
such that people will reach for this tool rather than brewing their own.

Regards,

-- 
Rowan Collins
[IMSoP]

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] [VOTE] UUID

[Stephen Reay]

> On 5 Sep 2017, at 14:32, Arvids Godjuks  wrote:
> 
> That way the API and
> features can be stabilized.

(Sorry for resend, just realised I sent from a non-list email)

Hi Arvids,

This is exactly why I (and possibly others) are in favour of this being 
implemented as a few functions - the object side of things can then be left to 
a user land implementation, which is much easier to update and refine.

Cheers


Stephen

Re: [PHP-DEV] A validator module for PHP7

[Lester Caine]

On 05/09/17 12:18, Yasuo Ohgaki wrote:
> I cannot guess people's thought. I appreciated feedback!

With a decent database layer a lot of the validation you are proposing
is already covered but PDO does not help in this area. Adding another
layer that does not integrate with a storage layer is just adding to the
current mess ...

-- 
Lester Caine - G8HFL
-
Contact - http://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - http://lsces.co.uk
EnquirySolve - http://enquirysolve.com/
Model Engineers Digital Workshop - http://medw.co.uk
Rainbow Digital Media - http://rainbowdigitalmedia.co.uk

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] A validator module for PHP7

[Yasuo Ohgaki]

Hi all,

On Tue, Sep 5, 2017 at 12:19 PM, Paul Jones  wrote:

>
> > On Sep 4, 2017, at 18:06, Marco Pivetta  wrote:
> >
> > On Mon, Sep 4, 2017 at 8:56 PM, Crocodile  wrote:
> >
> >> In most cases users would like more than just valid/invalid, i. e. which
> >> particular rule(s) failed and also human-readable error messages. As of
> >> simple validation that is almost always at hand, filter_* functions do a
> >> good job, although I agree that they could be better. I, for one, would
> >> like to see a full-featured validation as part of PHP. However, this RFC
> >> only looks like a slightly better version of filter_* functions, that
> is,
> >> the new module will provide almost the same functionality but with a
> >> different interface. I would vote against it.
> >>
> >
> > And also, it would need to be better than all of these to be worth
> writing
> > it in C:
> >
> > https://packagist.org/search/?q=validator
>
> And these as well: https://packagist.org/search/?q=filter


I cannot guess people's thought. I appreciated feedback!

Why do you think basic validation module should be better than full OO
implementation(s)?

Simple and basic type support NULL/BOOL/INT/FLOAT/STRING/ARRAY/OBJECT is
enough
for C written PHP module. IMHO, PHP modules are better of to provide basic
features
that may be extended by user scripts.

I picked 1st one on the list as an example.
This kind of rule construction is inefficient compare to simple array rules,
so I doubt this is the way for basic validator module written by C.

$validator = Validation::createValidator();
$violations = $validator->validate('Bernhard', array(
new Length(array('min' => 10)),
new NotBlank(),
));

However, this particular validation class seems it could be good one that
wraps
validate module for nicer OO API and faster execution.

Regards,

--
Yasuo Ohgaki
yohg...@ohgaki.net

Re: [PHP-DEV] [VOTE] UUID

[Arvids Godjuks]

2017-09-04 19:03 GMT+03:00 Sammy Kaye Powers :

> I really, really wanted to vote yes for this as I've wanted simple
> UUID creation in core for a long time, but I can't agree this is the
> correct implementation. Something like "uuid_v4_create()" seems to
> make a lot more pragmatic sense and use ramsey/uuid for anything more
> complicated.
>
> Sorry Richard, but I'm -1 on this one. But thanks for putting in all
> the effort! I think another attempt with a simpler API &
> implementation could be a winner! :)
>
> Thanks,
> Sammy Kaye Powers
> sammyk.me
>
>
Hello everyone, it's been a while.

As a userland dev, I completely agree with the RFC. The implementation
details may be worked out a bit, but having UUID's as an object is a good
thing and having a proper reference implementation that follows the
standard to the letter is very valuable. UUID's are a universal concept,
used in so many systems across the IT world, that I personally consider
them to be a thing on their own. I do believe they deserve to be an object
- if you expect a UUID, you want a very specific thing, not just a generic
string and then validate it every time with some form of
"is_valid_uuid($uuid)". Even if PHP will have a reference implementation,
99.9% chance I will use a wrapper library that will give me an object I can
work with. And not only for my personal preference, but also so other
people working on the same project do not mess with a UUID string. Type
hinting checks are also quite important, that I learned first hand lately
as I went PHP 7.1 strict_types=1 on my whole codebase - I have been passing
crap to in places that I would have never known, and probably would never
result in an issue due to the nature of the error.

I do, however, think that maybe doing a userland PHP implementation of the
proposed module, so people can play and use it and then converting it to a
PHP C module closer to 7.3 would be a better way. That way the API and
features can be stabilized.

-- 
Arvīds Godjuks

+371 26 851 664
arvids.godj...@gmail.com
Skype: psihius
Telegram: @psihius https://t.me/psihius