Re: MD5 [was: Re: BinaryData and odd bytes]

[Harbs]

I hear you, but I’m out of my element here. I’m not even sure how to write 
meaningful tests on this.

Thanks,
Harbs

> On Jun 24, 2018, at 10:54 AM, Alex Harui  wrote:
> 
> 
> 
> On 6/23/18, 12:15 PM, "Harbs"  wrote:
> 
>> 
>> We may need to understand which modern browsers that we support (IE11, 
>> Chrome, FireFox, Android, IOS Safari) might have these issues.  We might 
>> have to generate a "safeAdd()" instead of just "+" when we know the types 
>> are all integers.  And implement smarter coercions as described in the 
>> article you linked to.  Which browser(s) did you test with?
> 
>Chrome, Firefox and Safari. All on Mac.
> 
>I’ve used BinaryData pretty extensively and I haven’t had issues. I don’t 
> understand why MD5 was failing, but it seems like whatever the issue is is 
> pretty fringe.
> 
> Maybe.  AIUI, floating point as an approximation for int and uint can be 
> lossy.  Probably for sure around overflow.  I'm wondering if it is also an 
> issue with certain values after a series of calculations.
> 
> Some experiments would be to have some code generate a huge array of random 
> ints and then have AS code add them, and JS code add them and see if the 
> results match.  Or have just a JS app that generates numbers and add them via 
> regular add vs safeAdd.  If you do, say, 10 calculations and end up with 
> different results, that may not be 'fringe'.
> 
> My 2 cents,
> -Alex
> 
> 

Re: MD5 [was: Re: BinaryData and odd bytes]

[Alex Harui]

On 6/23/18, 12:15 PM, "Harbs"  wrote:

> 
> We may need to understand which modern browsers that we support (IE11, 
Chrome, FireFox, Android, IOS Safari) might have these issues.  We might have 
to generate a "safeAdd()" instead of just "+" when we know the types are all 
integers.  And implement smarter coercions as described in the article you 
linked to.  Which browser(s) did you test with?

Chrome, Firefox and Safari. All on Mac.

I’ve used BinaryData pretty extensively and I haven’t had issues. I don’t 
understand why MD5 was failing, but it seems like whatever the issue is is 
pretty fringe.

Maybe.  AIUI, floating point as an approximation for int and uint can be lossy. 
 Probably for sure around overflow.  I'm wondering if it is also an issue with 
certain values after a series of calculations.

Some experiments would be to have some code generate a huge array of random 
ints and then have AS code add them, and JS code add them and see if the 
results match.  Or have just a JS app that generates numbers and add them via 
regular add vs safeAdd.  If you do, say, 10 calculations and end up with 
different results, that may not be 'fringe'.

My 2 cents,
-Alex

Re: MD5 [was: Re: BinaryData and odd bytes]

[Harbs]

> On Jun 23, 2018, at 7:45 AM, Alex Harui  wrote:
> 
> Thanks for digging into it.
> 
> If I understand correctly, the AS code had a bug and should not have had the 
> s[] code in it.

Correct.

> The JS problem is interesting.  Some internet articles claim that JS does not 
> have integers at all.  So not sure what to make of the article you linked to. 
>  The code in the JS MD5 implementation is solving some other problem.  It 
> does not seem to try to sign extend the values, maybe because the values in 
> an MD5 calculation are all unsigned.
> 
> We may need to understand which modern browsers that we support (IE11, 
> Chrome, FireFox, Android, IOS Safari) might have these issues.  We might have 
> to generate a "safeAdd()" instead of just "+" when we know the types are all 
> integers.  And implement smarter coercions as described in the article you 
> linked to.  Which browser(s) did you test with?

Chrome, Firefox and Safari. All on Mac.

I’ve used BinaryData pretty extensively and I haven’t had issues. I don’t 
understand why MD5 was failing, but it seems like whatever the issue is is 
pretty fringe.

> -Alex
> 
> On 6/22/18, 3:21 AM, "Harbs"  > wrote:
> 
>In Flash, the error is ReferenceError: Error: 1069.
> 
>I took another look and it looks like the problem is the bracketed access 
> to the BinaryData.
> 
>The code s[ i / 8 ] needs to be changed to either s.readByteAt(i/8) or 
> s.array[i/8].
> 
>Making either of these changes fixed the Flash error, but did not fix the 
> JS error:
>expected 80338e79d2ca9b9c090ebaaa2ef293c7 but was: 
> 996f93cb21f81b54962909fd0ea18074
> 
>I’m guessing that the problem is related to the comment here in the JS MD5 
> implementation:
> 
>   * Add integers, wrapping at 2^32. This uses 16-bit operations 
> internally
>   * to work around bugs in some JS interpreters.
> 
>I think part of the problem is that JS only has 32 bit ints. I did not 
> follow this article completely, but this might shed some light:
>
> https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fblog.vjeux.com%2F2013%2Fjavascript%2Fconversion-from-uint8-to-int8-x-24.html=02%7C01%7Caharui%40adobe.com%7C21a1876d2a9b4e569ba008d5d829de4a%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636652596694550635=YyxMW%2BZfeuhZ1x8w%2BMxetw8FoZGhEqU0jIkZxyDc%2Bpg%3D=0
>  
> 
>  
>   
> >
> 
>Harbs
> 
>> On Jun 22, 2018, at 9:22 AM, Alex Harui  wrote:
>> 
>> On 6/21/18, 11:20 AM, "Harbs"  wrote:
>> 
>>   The MD5 test actually failed in Flash too. Although the error was 
>> different.
>> 
>>   I’m not sure why it was failing. I’m pretty sure the code was the same as 
>> the Flex version with ByteArray swapped for BinaryData.
>> 
>> Even more bothersome.  We really should dig deeper on this.  We can't have 
>> other people's business logic have similar issues.  Any volunteers?
>> 
>> -Alex

Re: MD5 [was: Re: BinaryData and odd bytes]

[Alex Harui]

Thanks for digging into it.

If I understand correctly, the AS code had a bug and should not have had the 
s[] code in it.

The JS problem is interesting.  Some internet articles claim that JS does not 
have integers at all.  So not sure what to make of the article you linked to.  
The code in the JS MD5 implementation is solving some other problem.  It does 
not seem to try to sign extend the values, maybe because the values in an MD5 
calculation are all unsigned.

We may need to understand which modern browsers that we support (IE11, Chrome, 
FireFox, Android, IOS Safari) might have these issues.  We might have to 
generate a "safeAdd()" instead of just "+" when we know the types are all 
integers.  And implement smarter coercions as described in the article you 
linked to.  Which browser(s) did you test with?

-Alex

On 6/22/18, 3:21 AM, "Harbs"  wrote:

In Flash, the error is ReferenceError: Error: 1069.

I took another look and it looks like the problem is the bracketed access 
to the BinaryData.

The code s[ i / 8 ] needs to be changed to either s.readByteAt(i/8) or 
s.array[i/8].

Making either of these changes fixed the Flash error, but did not fix the 
JS error:
expected 80338e79d2ca9b9c090ebaaa2ef293c7 but was: 
996f93cb21f81b54962909fd0ea18074

I’m guessing that the problem is related to the comment here in the JS MD5 
implementation:

* Add integers, wrapping at 2^32. This uses 16-bit operations 
internally
* to work around bugs in some JS interpreters.

I think part of the problem is that JS only has 32 bit ints. I did not 
follow this article completely, but this might shed some light:

https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fblog.vjeux.com%2F2013%2Fjavascript%2Fconversion-from-uint8-to-int8-x-24.html=02%7C01%7Caharui%40adobe.com%7C21a1876d2a9b4e569ba008d5d829de4a%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636652596694550635=YyxMW%2BZfeuhZ1x8w%2BMxetw8FoZGhEqU0jIkZxyDc%2Bpg%3D=0
 


Harbs

> On Jun 22, 2018, at 9:22 AM, Alex Harui  wrote:
> 
> On 6/21/18, 11:20 AM, "Harbs"  wrote:
> 
>The MD5 test actually failed in Flash too. Although the error was 
different.
> 
>I’m not sure why it was failing. I’m pretty sure the code was the same 
as the Flex version with ByteArray swapped for BinaryData.
> 
> Even more bothersome.  We really should dig deeper on this.  We can't 
have other people's business logic have similar issues.  Any volunteers?
> 
> -Alex
> 
> 

Re: MD5 [was: Re: BinaryData and odd bytes]

[Harbs]

In Flash, the error is ReferenceError: Error: 1069.

I took another look and it looks like the problem is the bracketed access to 
the BinaryData.

The code s[ i / 8 ] needs to be changed to either s.readByteAt(i/8) or 
s.array[i/8].

Making either of these changes fixed the Flash error, but did not fix the JS 
error:
expected 80338e79d2ca9b9c090ebaaa2ef293c7 but was: 
996f93cb21f81b54962909fd0ea18074

I’m guessing that the problem is related to the comment here in the JS MD5 
implementation:

* Add integers, wrapping at 2^32. This uses 16-bit operations 
internally
* to work around bugs in some JS interpreters.

I think part of the problem is that JS only has 32 bit ints. I did not follow 
this article completely, but this might shed some light:
http://blog.vjeux.com/2013/javascript/conversion-from-uint8-to-int8-x-24.html 


Harbs

> On Jun 22, 2018, at 9:22 AM, Alex Harui  wrote:
> 
> On 6/21/18, 11:20 AM, "Harbs"  wrote:
> 
>The MD5 test actually failed in Flash too. Although the error was 
> different.
> 
>I’m not sure why it was failing. I’m pretty sure the code was the same as 
> the Flex version with ByteArray swapped for BinaryData.
> 
> Even more bothersome.  We really should dig deeper on this.  We can't have 
> other people's business logic have similar issues.  Any volunteers?
> 
> -Alex
> 
> 

Re: MD5 [was: Re: BinaryData and odd bytes]

[Alex Harui]

On 6/21/18, 11:20 AM, "Harbs"  wrote:

The MD5 test actually failed in Flash too. Although the error was different.

I’m not sure why it was failing. I’m pretty sure the code was the same as 
the Flex version with ByteArray swapped for BinaryData.

Even more bothersome.  We really should dig deeper on this.  We can't have 
other people's business logic have similar issues.  Any volunteers?

-Alex
 

Re: MD5 [was: Re: BinaryData and odd bytes]

[Alex Harui]

Hmm.  So are you saying the original code works in SWF but not when transpiled 
to JS and run in the browser?  We might need to understand why and change the 
compiler output and/or document how to write AS that can transpile correctly.

FWIW, the Ant_On_AIR library has a custom MD5 algorithm that is significantly 
faster than these standard function call based MD5 algorithms.  It would be 
interesting to see if it transpiles correctly and provides significant 
performance improvements in JS as well.

My 2 cents,
-Alex

On 6/21/18, 2:02 AM, "Harbs"  wrote:

I forgot. The ported MD5 code did not work. I just replaced it with an MIT 
JS implementation…

> On Jun 21, 2018, at 1:20 AM, Harbs  wrote:
> 
> FYI, the MD5 test there now fails. I have no idea why. Maybe I messed 
something up with my BinaryData changes, but I can’t see what…