Re: [openssl-dev] 90-test_secmem.t hangs the machine for good

2017-05-16 Thread Short, Todd via openssl-dev 
MLOCK_ONFAULT is a Linux-only feature (hence the need to include  
wrapped by OPENSSL_SYS_LINUX.

So, you should not be encountering any MLOCK_ONFAULT or  issues 
on MacOS.
--
-Todd Short
// tsh...@akamai.com
// "One if by land, two if by sea, three if by the Internet."

> On May 15, 2017, at 1:51 PM, Blumenthal, Uri - 0553 - MITLL  
> wrote:
> 
> On 5/15/17, 1:20 PM, "openssl-dev on behalf of Benjamin Kaduk via 
> openssl-dev"   on behalf of openssl-dev@openssl.org 
> > wrote:
>> > On a semi-related note, I want able to locate mann.h file either.
> 
> `man mmap` will list any headers needed for the mmap() declaration and flag 
> values.
> On the random OS X machine I have handy, it claims  is needed, 
> and a /usr/include/sys/mman.h is present.
> 
> Thanks – I confirm that `man mmap` mentions /usr/include/sys/mman.h, and that 
> file does exist (how could my first `find` miss it?!). 
> This file does not contain MLOCK_ONFAULT though.
>  
> It has MAP_ANON:
>  
> /*
>  * Mapping type
>  */
> #define MAP_FILE0x  /* map from file (default) */
> #define MAP_ANON0x1000  /* allocated from memory, swap space */
> #define MAP_ANONYMOUS   MAP_ANON
>  
> And as I already mentioned, it has a world-accessible (figuratively speaking 
> :) /dev/zero that can be opened read/write.
>  
> Also, with #3455 applied the problem disappeared (thankfully :).
>  
> -- 
> openssl-dev mailing list
> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev 
> 



smime.p7s
Description: S/MIME cryptographic signature
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] 90-test_secmem.t hangs the machine for good

2017-05-15 Thread Blumenthal, Uri - 0553 - MITLL 
On 5/15/17, 1:20 PM, "openssl-dev on behalf of Benjamin Kaduk via openssl-dev" 
 wrote:

> On a semi-related note, I want able to locate mann.h file either.


`man mmap` will list any headers needed for the mmap() declaration and flag 
values.
On the random OS X machine I have handy, it claims  is needed, 

and a /usr/include/sys/mman.h is present.


Thanks – I confirm that `man mmap` mentions /usr/include/sys/mman.h, and that 
file does exist (how could my first `find` miss it?!). 

This file does not contain MLOCK_ONFAULT though.

 

It has MAP_ANON:

 

/*

 * Mapping type

 */

#define MAP_FILE0x  /* map from file (default) */

#define MAP_ANON0x1000  /* allocated from memory, swap space */

#define MAP_ANONYMOUS   MAP_ANON

 

And as I already mentioned, it has a world-accessible (figuratively speaking :) 
/dev/zero that can be opened read/write.

 

Also, with #3455 applied the problem disappeared (thankfully :).

 



smime.p7s
Description: S/MIME cryptographic signature
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] 90-test_secmem.t hangs the machine for good

2017-05-15 Thread Benjamin Kaduk via openssl-dev 
On 05/15/2017 12:15 PM, Blumenthal, Uri - 0553 - MITLL wrote:
>
> On a semi-related note, I want able to locate mann.h file either.

`man mmap` will list any headers needed for the mmap() declaration and
flag values.
On the random OS X machine I have handy, it claims  is
needed, and a /usr/include/sys/mman.h is present.

-Ben
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] 90-test_secmem.t hangs the machine for good

2017-05-15 Thread Blumenthal, Uri - 0553 - MITLL 
Yes and yes. :-)

Regards,
Uri

Sent from my iPhone

> On May 15, 2017, at 13:18, Short, Todd  wrote:
> 
> s/want/wasn’t/ ?
> 
> So, no MLOCK_ONFAULT for you? That’s only included on Linux systems, which 
> makes sense if you’re on a Mac.
> --
> -Todd Short
> // tsh...@akamai.com
> // "One if by land, two if by sea, three if by the Internet."
> 
>> On May 15, 2017, at 1:15 PM, Blumenthal, Uri - 0553 - MITLL 
>>  wrote:
>> 
>> My disk is SSD, but computer load is pretty high. Which probably explains 
>> that recovery doesn't take place in 200-400 seconds...
>> 
>> On a semi-related note, I want able to locate mann.h file either.
>> 
>> Regards,
>> Uri
>> 
>> Sent from my iPhone
>> 
>>> On May 15, 2017, at 13:09, Short, Todd  wrote:
>>> 
>>> The time of the hang actually seems dependent on the number of applications 
>>> running and your disk.
>>> 
>>> Since a large amount of memory becomes wired, there is very little 
>>> available for programs and the OS to use (in some instances I have seen 
>>> ~4MB non-wired memory). Things slow down due to swapping, etc. 
>>> 
>>> In my testing:
>>> 
>>> With almost no additional programs open, the hang-time is short, ~200 
>>> seconds.
>>> With a lot of programs open, the hang-time is increased, ~400 seconds; 
>>> twice as long. And the number of swapins is 25x and the swapouts is ~34x 
>>> the original test period.
>>> 
>>> This is on a machine with an SSD (late-2013 MBP)
>>> If you have a spinning HDD, the swapins and swapouts will be significantly 
>>> more expensive in terms of performance/time.
>>> 
>>> If you quit all your programs, (other than Terminal), I suspect the hang 
>>> may eventually recover; but if you have a hard disk that time might be 
>>> quite long.
>>> 
>>> --
>>> -Todd Short
>>> // tsh...@akamai.com
>>> // "One if by land, two if by sea, three if by the Internet."
>>> 
 On May 15, 2017, at 11:51 AM, Blumenthal, Uri - 0553 - MITLL 
  wrote:
 
 I’m tracking the current OpenSSL master only on El Capitan 10.11.6. I 
 could try it on Sierra 10.12.4, if you really expect it to make a 
 difference.
  
 In my case the hang is not for a short time. It lasts for more than 10 
 minutes, so I’m forced to interfere. For how long did it hang for you?
 — 
 Regards,
 Uri
  
  
 On 5/15/17, 11:47 AM, "openssl-dev on behalf of Short, Todd via 
 openssl-dev"  wrote:
  
 We’ve been able to get some Macs (10.11.6, 8GB and 16GB) to hang for a 
 short period of time with the unit-tests, but it eventually recovers. 
  
 What MacOS version are you running? I can try 10.12 later today.
 --
 -Todd Short
 // tsh...@akamai.com
 // "One if by land, two if by sea, three if by the Internet."
  
> On May 12, 2017, at 4:50 PM, Short, Todd via openssl-dev 
>  wrote:  
>  
> Uri: 
>  
> Look at https://github.com/openssl/openssl/pull/3455
>  
> I limited the test that hung your machine to Linux.
>  
> Rich: this removes the OpenSSL_assert() you see.
>  
> --
> -Todd Short
> // tsh...@akamai.com
> // "One if by land, two if by sea, three if by the Internet."
>  
>> On May 12, 2017, at 4:46 PM, Short, Todd via openssl-dev 
>>  wrote:
>>  
>> It’s trying to reserve 1<<34 bytes of memory… there goes your 16GB...
>> --
>> -Todd Short
>> // tsh...@akamai.com
>> // "One if by land, two if by sea, three if by the Internet."
>>  
>>> On May 12, 2017, at 4:05 PM, Blumenthal, Uri - 0553 - MITLL 
>>>  wrote:
>>>  
>>> Todd> Yes, it’s likely this is due to the amount of memory available in 
>>> the machine. I tried to use reasonable values, but apparently not 
>>> reasonable enough
>>>  
>>> Yep. In case it matters, my machine has 16GB of RAM (and runs a ton of 
>>> stuff, besides these tests :).
>>> -- 
>>> openssl-dev mailing list
>>> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
>> 
>>  
>> -- 
>> openssl-dev mailing list
>> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
> 
>  
> -- 
> openssl-dev mailing list
> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
 
  
>>> 
> 


smime.p7s
Description: S/MIME cryptographic signature
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] 90-test_secmem.t hangs the machine for good

2017-05-15 Thread Short, Todd via openssl-dev 
s/want/wasn’t/ ?

So, no MLOCK_ONFAULT for you? That’s only included on Linux systems, which 
makes sense if you’re on a Mac.
--
-Todd Short
// tsh...@akamai.com
// "One if by land, two if by sea, three if by the Internet."

> On May 15, 2017, at 1:15 PM, Blumenthal, Uri - 0553 - MITLL  
> wrote:
> 
> My disk is SSD, but computer load is pretty high. Which probably explains 
> that recovery doesn't take place in 200-400 seconds...
> 
> On a semi-related note, I want able to locate mann.h file either.
> 
> Regards,
> Uri
> 
> Sent from my iPhone
> 
> On May 15, 2017, at 13:09, Short, Todd  > wrote:
> 
>> The time of the hang actually seems dependent on the number of applications 
>> running and your disk.
>> 
>> Since a large amount of memory becomes wired, there is very little available 
>> for programs and the OS to use (in some instances I have seen ~4MB non-wired 
>> memory). Things slow down due to swapping, etc. 
>> 
>> In my testing:
>> 
>> With almost no additional programs open, the hang-time is short, ~200 
>> seconds.
>> With a lot of programs open, the hang-time is increased, ~400 seconds; twice 
>> as long. And the number of swapins is 25x and the swapouts is ~34x the 
>> original test period.
>> 
>> This is on a machine with an SSD (late-2013 MBP)
>> If you have a spinning HDD, the swapins and swapouts will be significantly 
>> more expensive in terms of performance/time.
>> 
>> If you quit all your programs, (other than Terminal), I suspect the hang may 
>> eventually recover; but if you have a hard disk that time might be quite 
>> long.
>> 
>> --
>> -Todd Short
>> // tsh...@akamai.com 
>> // "One if by land, two if by sea, three if by the Internet."
>> 
>>> On May 15, 2017, at 11:51 AM, Blumenthal, Uri - 0553 - MITLL 
>>> > wrote:
>>> 
>>> I’m tracking the current OpenSSL master only on El Capitan 10.11.6. I could 
>>> try it on Sierra 10.12.4, if you really expect it to make a difference.
>>>  
>>> In my case the hang is not for a short time. It lasts for more than 10 
>>> minutes, so I’m forced to interfere. For how long did it hang for you?
>>> — 
>>> Regards,
>>> Uri
>>>  
>>>  
>>> On 5/15/17, 11:47 AM, "openssl-dev on behalf of Short, Todd via 
>>> openssl-dev" >>  on behalf of 
>>> openssl-dev@openssl.org > wrote:
>>>  
>>> We’ve been able to get some Macs (10.11.6, 8GB and 16GB) to hang for a 
>>> short period of time with the unit-tests, but it eventually recovers. 
>>>  
>>> What MacOS version are you running? I can try 10.12 later today.
>>> --
>>> -Todd Short
>>> // tsh...@akamai.com 
>>> // "One if by land, two if by sea, three if by the Internet."
>>>  
 On May 12, 2017, at 4:50 PM, Short, Todd via openssl-dev 
 > wrote:  
  
 Uri: 
  
 Look at https://github.com/openssl/openssl/pull/3455 
 
  
 I limited the test that hung your machine to Linux.
  
 Rich: this removes the OpenSSL_assert() you see.
  
 --
 -Todd Short
 // tsh...@akamai.com 
 // "One if by land, two if by sea, three if by the Internet."
  
> On May 12, 2017, at 4:46 PM, Short, Todd via openssl-dev 
> > wrote:
>  
> It’s trying to reserve 1<<34 bytes of memory… there goes your 16GB...
> --
> -Todd Short
> // tsh...@akamai.com 
> // "One if by land, two if by sea, three if by the Internet."
>  
>> On May 12, 2017, at 4:05 PM, Blumenthal, Uri - 0553 - MITLL 
>> > wrote:
>>  
>> Todd> Yes, it’s likely this is due to the amount of memory available in 
>> the machine. I tried to use reasonable values, but apparently not 
>> reasonable enough
>>  
>> Yep. In case it matters, my machine has 16GB of RAM (and runs a ton of 
>> stuff, besides these tests :).
>> -- 
>> openssl-dev mailing list
>> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev 
>> 
>  
> -- 
> openssl-dev mailing list
> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev 
> 
  
 -- 
 openssl-dev mailing list
 To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev 
 
>>>  
>> 



smime.p7s
Description: S/MIME cryptographic signature
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] 90-test_secmem.t hangs the machine for good

2017-05-15 Thread Blumenthal, Uri - 0553 - MITLL 
My disk is SSD, but computer load is pretty high. Which probably explains that 
recovery doesn't take place in 200-400 seconds...

On a semi-related note, I want able to locate mann.h file either.

Regards,
Uri

Sent from my iPhone

> On May 15, 2017, at 13:09, Short, Todd  wrote:
> 
> The time of the hang actually seems dependent on the number of applications 
> running and your disk.
> 
> Since a large amount of memory becomes wired, there is very little available 
> for programs and the OS to use (in some instances I have seen ~4MB non-wired 
> memory). Things slow down due to swapping, etc. 
> 
> In my testing:
> 
> With almost no additional programs open, the hang-time is short, ~200 seconds.
> With a lot of programs open, the hang-time is increased, ~400 seconds; twice 
> as long. And the number of swapins is 25x and the swapouts is ~34x the 
> original test period.
> 
> This is on a machine with an SSD (late-2013 MBP)
> If you have a spinning HDD, the swapins and swapouts will be significantly 
> more expensive in terms of performance/time.
> 
> If you quit all your programs, (other than Terminal), I suspect the hang may 
> eventually recover; but if you have a hard disk that time might be quite long.
> 
> --
> -Todd Short
> // tsh...@akamai.com
> // "One if by land, two if by sea, three if by the Internet."
> 
>> On May 15, 2017, at 11:51 AM, Blumenthal, Uri - 0553 - MITLL 
>>  wrote:
>> 
>> I’m tracking the current OpenSSL master only on El Capitan 10.11.6. I could 
>> try it on Sierra 10.12.4, if you really expect it to make a difference.
>>  
>> In my case the hang is not for a short time. It lasts for more than 10 
>> minutes, so I’m forced to interfere. For how long did it hang for you?
>> — 
>> Regards,
>> Uri
>>  
>>  
>> On 5/15/17, 11:47 AM, "openssl-dev on behalf of Short, Todd via openssl-dev" 
>>  wrote:
>>  
>> We’ve been able to get some Macs (10.11.6, 8GB and 16GB) to hang for a short 
>> period of time with the unit-tests, but it eventually recovers. 
>>  
>> What MacOS version are you running? I can try 10.12 later today.
>> --
>> -Todd Short
>> // tsh...@akamai.com
>> // "One if by land, two if by sea, three if by the Internet."
>>  
>>> On May 12, 2017, at 4:50 PM, Short, Todd via openssl-dev 
>>>  wrote:  
>>>  
>>> Uri: 
>>>  
>>> Look at https://github.com/openssl/openssl/pull/3455
>>>  
>>> I limited the test that hung your machine to Linux.
>>>  
>>> Rich: this removes the OpenSSL_assert() you see.
>>>  
>>> --
>>> -Todd Short
>>> // tsh...@akamai.com
>>> // "One if by land, two if by sea, three if by the Internet."
>>>  
 On May 12, 2017, at 4:46 PM, Short, Todd via openssl-dev 
  wrote:
  
 It’s trying to reserve 1<<34 bytes of memory… there goes your 16GB...
 --
 -Todd Short
 // tsh...@akamai.com
 // "One if by land, two if by sea, three if by the Internet."
  
> On May 12, 2017, at 4:05 PM, Blumenthal, Uri - 0553 - MITLL 
>  wrote:
>  
> Todd> Yes, it’s likely this is due to the amount of memory available in 
> the machine. I tried to use reasonable values, but apparently not 
> reasonable enough
>  
> Yep. In case it matters, my machine has 16GB of RAM (and runs a ton of 
> stuff, besides these tests :).
> -- 
> openssl-dev mailing list
> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
 
  
 -- 
 openssl-dev mailing list
 To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
>>> 
>>>  
>>> -- 
>>> openssl-dev mailing list
>>> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
>> 
>>  
> 


smime.p7s
Description: S/MIME cryptographic signature
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] 90-test_secmem.t hangs the machine for good

2017-05-15 Thread Blumenthal, Uri - 0553 - MITLL 
I’m tracking the current OpenSSL master only on El Capitan 10.11.6. I could try 
it on Sierra 10.12.4, if you really expect it to make a difference.

 

In my case the hang is not for a short time. It lasts for more than 10 minutes, 
so I’m forced to interfere. For how long did it hang for you?

— 

Regards,

Uri

 

 

On 5/15/17, 11:47 AM, "openssl-dev on behalf of Short, Todd via openssl-dev" 
 wrote:

 

We’ve been able to get some Macs (10.11.6, 8GB and 16GB) to hang for a short 
period of time with the unit-tests, but it eventually recovers. 

 

What MacOS version are you running? I can try 10.12 later today.

--

-Todd Short

// tsh...@akamai.com

// "One if by land, two if by sea, three if by the Internet."

 

On May 12, 2017, at 4:50 PM, Short, Todd via openssl-dev 
 wrote:  

 

Uri: 

 

Look at https://github.com/openssl/openssl/pull/3455

 

I limited the test that hung your machine to Linux.

 

Rich: this removes the OpenSSL_assert() you see.

 

--

-Todd Short

// tsh...@akamai.com

// "One if by land, two if by sea, three if by the Internet."

 

On May 12, 2017, at 4:46 PM, Short, Todd via openssl-dev 
 wrote:

 

It’s trying to reserve 1<<34 bytes of memory… there goes your 16GB...

--

-Todd Short

// tsh...@akamai.com

// "One if by land, two if by sea, three if by the Internet."

 

On May 12, 2017, at 4:05 PM, Blumenthal, Uri - 0553 - MITLL  
wrote:

 

Todd> Yes, it’s likely this is due to the amount of memory available in the 
machine. I tried to use reasonable values, but apparently not reasonable enough

 

Yep. In case it matters, my machine has 16GB of RAM (and runs a ton of stuff, 
besides these tests :).

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

 

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

 

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

 



smime.p7s
Description: S/MIME cryptographic signature
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] 90-test_secmem.t hangs the machine for good

2017-05-15 Thread Short, Todd via openssl-dev 
We’ve been able to get some Macs (10.11.6, 8GB and 16GB) to hang for a short 
period of time with the unit-tests, but it eventually recovers.

What MacOS version are you running? I can try 10.12 later today.
--
-Todd Short
// tsh...@akamai.com
// "One if by land, two if by sea, three if by the Internet."

> On May 12, 2017, at 4:50 PM, Short, Todd via openssl-dev 
>  wrote:
> 
> Uri:
> 
> Look at https://github.com/openssl/openssl/pull/3455 
> 
> 
> I limited the test that hung your machine to Linux.
> 
> Rich: this removes the OpenSSL_assert() you see.
> 
> --
> -Todd Short
> // tsh...@akamai.com 
> // "One if by land, two if by sea, three if by the Internet."
> 
>> On May 12, 2017, at 4:46 PM, Short, Todd via openssl-dev 
>> > wrote:
>> 
>> It’s trying to reserve 1<<34 bytes of memory… there goes your 16GB...
>> --
>> -Todd Short
>> // tsh...@akamai.com 
>> // "One if by land, two if by sea, three if by the Internet."
>> 
>>> On May 12, 2017, at 4:05 PM, Blumenthal, Uri - 0553 - MITLL 
>>> > wrote:
>>> 
>>> Todd> Yes, it’s likely this is due to the amount of memory available in the 
>>> machine. I tried to use reasonable values, but apparently not reasonable 
>>> enough
>>>  
>>> Yep. In case it matters, my machine has 16GB of RAM (and runs a ton of 
>>> stuff, besides these tests :).
>>> -- 
>>> openssl-dev mailing list
>>> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev 
>>> 
>> 
>> -- 
>> openssl-dev mailing list
>> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev 
>> 
> 
> -- 
> openssl-dev mailing list
> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev



smime.p7s
Description: S/MIME cryptographic signature
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] 90-test_secmem.t hangs the machine for good

2017-05-12 Thread Short, Todd via openssl-dev 
Uri:

Look at https://github.com/openssl/openssl/pull/3455 


I limited the test that hung your machine to Linux.

Rich: this removes the OpenSSL_assert() you see.

--
-Todd Short
// tsh...@akamai.com
// "One if by land, two if by sea, three if by the Internet."

> On May 12, 2017, at 4:46 PM, Short, Todd via openssl-dev 
>  wrote:
> 
> It’s trying to reserve 1<<34 bytes of memory… there goes your 16GB...
> --
> -Todd Short
> // tsh...@akamai.com 
> // "One if by land, two if by sea, three if by the Internet."
> 
>> On May 12, 2017, at 4:05 PM, Blumenthal, Uri - 0553 - MITLL > > wrote:
>> 
>> Todd> Yes, it’s likely this is due to the amount of memory available in the 
>> machine. I tried to use reasonable values, but apparently not reasonable 
>> enough
>>  
>> Yep. In case it matters, my machine has 16GB of RAM (and runs a ton of 
>> stuff, besides these tests :).
>> -- 
>> openssl-dev mailing list
>> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev 
>> 
> 
> -- 
> openssl-dev mailing list
> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev



smime.p7s
Description: S/MIME cryptographic signature
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] 90-test_secmem.t hangs the machine for good

2017-05-12 Thread Short, Todd via openssl-dev 
It’s trying to reserve 1<<34 bytes of memory… there goes your 16GB...
--
-Todd Short
// tsh...@akamai.com
// "One if by land, two if by sea, three if by the Internet."

> On May 12, 2017, at 4:05 PM, Blumenthal, Uri - 0553 - MITLL  
> wrote:
> 
> Todd> Yes, it’s likely this is due to the amount of memory available in the 
> machine. I tried to use reasonable values, but apparently not reasonable 
> enough
>  
> Yep. In case it matters, my machine has 16GB of RAM (and runs a ton of stuff, 
> besides these tests :).
> -- 
> openssl-dev mailing list
> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev 
> 



smime.p7s
Description: S/MIME cryptographic signature
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] 90-test_secmem.t hangs the machine for good

2017-05-12 Thread Benjamin Kaduk via openssl-dev 
On 05/12/2017 03:05 PM, Blumenthal, Uri - 0553 - MITLL wrote:
>
> I’m not sure. It may well be that it “simply” takes all the memory
> over, so there is no way for anything else to start and do the clean-up…
>

Hmm, I wonder if a top(1) started before the tests would keep running,
in the case that it "simply" takes all the memory over.

>  
>
> From just looking at the code, the only question that comes to mind is
> whether you have a 32- or 64-bit size_t in the build environment in
> question, which is unlikely to cause a eureka moment :(
>
>  
>
> I can tell you that size_t is 64-bit here. It’s certainly not an
> “eureka” moment for me.
>
>  
>
> Some other information to check: do you have MAP_ANON defined by your
> mman.h?
>

mman.h != mmap.h (consult 'man mmap' for authoritative header).

> Todd> Yes, it’s likely this is due to the amount of memory available
> in the machine. I tried to use reasonable values, but apparently not
> reasonable enough
>
>  
>
> Yep. In case it matters, my machine has 16GB of RAM (and runs a ton of
> stuff, besides these tests :).
>
>

Thanks.

-Ben
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] 90-test_secmem.t hangs the machine for good

2017-05-12 Thread Blumenthal, Uri - 0553 - MITLL 
The obvious candidate for closer inspection is a few commits previous,

commit 7031ddac94d0ae616d1b0670263a9265ce672cd2
Author: Todd Short 
Date:   Thu May 11 15:48:10 2017 -0400

which adds test cases intended to trigger the edge cases being fixed.



Point well-taken. ;-)

 

It seems that there should also be a bug report against OS X, as regular 
userspace code running as non-root should not be able to hang a machine like 
that.

I’m not sure. It may well be that it “simply” takes all the memory over, so 
there is no way for anything else to start and do the clean-up…

 

>From just looking at the code, the only question that comes to mind is whether 
>you have a 32- or 64-bit size_t in the build environment in question, which is 
>unlikely to cause a eureka moment :(

 

I can tell you that size_t is 64-bit here. It’s certainly not an “eureka” 
moment for me.

 

Some other information to check: do you have MAP_ANON defined by your mman.h?


/**

 * @addtogroup apr_platform

 * @ingroup APR 

 * @{

 */

 

#define APR_HAVE_SHMEM_MMAP_TMP 1

#define APR_HAVE_SHMEM_MMAP_SHM 1

#define APR_HAVE_SHMEM_MMAP_ZERO0

#define APR_HAVE_SHMEM_SHMGET_ANON  1

#define APR_HAVE_SHMEM_SHMGET   1

#define APR_HAVE_SHMEM_MMAP_ANON1

#define APR_HAVE_SHMEM_BEOS 0

 

#define APR_USE_SHMEM_MMAP_TMP 0

#define APR_USE_SHMEM_MMAP_SHM 0

#define APR_USE_SHMEM_MMAP_ZERO0

#define APR_USE_SHMEM_SHMGET_ANON  0

#define APR_USE_SHMEM_SHMGET   1

#define APR_USE_SHMEM_MMAP_ANON1

#define APR_USE_SHMEM_BEOS 0

 

And this system does not seem to have a straight “mmap.h”. The above is from 
/usr/include/apr-1/apr_mmap.h file.

 

Do you have a /dev/zero that can be opened read/write?

 

Yep:

 

$ ll /dev/zero

crw-rw-rw-  1 root  wheel3,   3 May 12 14:14 /dev/zero

$ 

 

Todd> Yes, it’s likely this is due to the amount of memory available in the 
machine. I tried to use reasonable values, but apparently not reasonable enough

 

Yep. In case it matters, my machine has 16GB of RAM (and runs a ton of stuff, 
besides these tests :).



smime.p7s
Description: S/MIME cryptographic signature
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] 90-test_secmem.t hangs the machine for good

2017-05-12 Thread Short, Todd via openssl-dev 
Yes, it’s likely this is due to the amount of memory available in the machine. 
I tried to use reasonable values, but apparently not reasonable enough.

This is certainly a case where we’re trying to stretch the limits of the 
hardware; so it may not be an appropriate test for all hardware.

In the case below, the call to mmap() failed, and there’s an OPENSSL_assert() 
there that’s probably not necessary; the error condition checked by the assert 
is handled safely.

This is different than the problem > 
was seeing, which is likely a condition where memory is being maxed out.

--
-Todd Short
// tsh...@akamai.com
// "One if by land, two if by sea, three if by the Internet."

On May 12, 2017, at 2:51 PM, Salz, Rich via openssl-dev 
> wrote:

Current master 1d0f116 fails on my machine.

../test/recipes/90-test_secmem.t ..
1..1
   # Subtest: ./secmemtest
   1..1
   # INFO:  @ test/secmemtest.c:65
   # Possible infinite loop: allocate more than available

   # INFO:  @ test/secmemtest.c:71
   # Possible infinite loop: small arena

   # INFO:  @ test/secmemtest.c:79
   # Possible infinite loop: 1<<31 limit

crypto/mem_sec.c:428: OpenSSL internal error: assertion failed: sh.map_result 
!= MAP_FAILED
../util/shlib_wrap.sh ./secmemtest => 134
not ok 1 - running secmemtest

--
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] 90-test_secmem.t hangs the machine for good

2017-05-12 Thread Benjamin Kaduk via openssl-dev 
On 05/12/2017 02:15 PM, Benjamin Kaduk via openssl-dev wrote:
> From just looking at the code, the only question that comes to mind is
> whether you have a 32- or 64-bit size_t in the build environment in
> question, which is unlikely to cause a eureka moment :(

(The test runs happily on my Ubuntu Trusty-ish machine, FWIW.)

Some other information to check: do you have MAP_ANON defined by your
mman.h?
Do you have a /dev/zero that can be opened read/write?

-Ben
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] 90-test_secmem.t hangs the machine for good

2017-05-12 Thread Benjamin Kaduk via openssl-dev 
On 05/12/2017 02:05 PM, Blumenthal, Uri - 0553 - MITLL wrote:
> On 5/12/17, 2:49 PM, "openssl-dev on behalf of Benjamin Kaduk via 
> openssl-dev"  openssl-dev@openssl.org> wrote:
>
> ➢ I’m sorry to report that in the current OpenSSL 1.1 master running “make 
> test”
> ➢  freezes up the machine. Mac OS X 10.11.6, Xcode-8.2, current Github 
> master. 
> ➢ Here’s the configuration:
>
>   A commit hash would be more useful than "current github master"
>
> I thought you know what’s in the current master right now. But here’s the 
> last few hashes for your pleasure:

Well, you're right to think that I do, and I do trust you to be accurate
when you make that claim.  But there are many people for which I would
not fully trust the veracity of such a claim, the commit hash is
completely unambiguous, and it is a whole lot easier for those poor
folks reading this thread in the archive two years from now who are
trying to track down a similar-looking issue.  So, on the whole, I
recommend always using commit hashes, with an optional annotation of how
it relates to a branch.

> $ git log
> commit 80a2fc4100daf6f1001eee33ef2f9b9eee05bedf (HEAD -> master, 
> origin/master, origin/HEAD)
> Author: Todd Short 
> Date:   Wed May 10 11:44:55 2017 -0400
>
> Clean up SSL_OP_* a bit
> 
> Reviewed-by: Matt Caswell 
> Reviewed-by: Rich Salz 
> (Merged from https://github.com/openssl/openssl/pull/3439)
>
> commit 33242d9d79e7f06151e905b83dc8f995006fa7cd
> Author: Rich Salz 
> Date:   Thu May 11 20:42:32 2017 -0400
>
> Use scalar, not length; fixes test_evp
> 
> Reviewed-by: Stephen Henson 
> Reviewed-by: Richard Levitte 
> (Merged from https://github.com/openssl/openssl/pull/3452)
>
>
>   I can understand not wanting to have to power-cycle the machine again,
>   but the 'make TESTS=test_secmem V=1 test' output (or some 
> dtruss/similar)
>   would be helpful in tracking things down.

The obvious candidate for closer inspection is a few commits previous,

commit 7031ddac94d0ae616d1b0670263a9265ce672cd2
Author: Todd Short 
Date:   Thu May 11 15:48:10 2017 -0400

Fix infinite loops in secure memory allocation.
   
Issue 1:
   
sh.bittable_size is a size_t but i is and int, which can result in
freelist == -1 if sh.bittable_size exceeds an int.
   
This seems to result in an OPENSSL_assert due to invalid allocation
size, so maybe that is "ok."
   
Worse, if sh.bittable_size is exactly 1<<31, then this becomes an
infinite loop (because 1<<31 is a negative int, so it can be shifted
right forever and sticks at -1).
   
Issue 2:
   
CRYPTO_secure_malloc_init() sets secure_mem_initialized=1 even when
sh_init() returns 0.
   
If sh_init() fails, we end up with secure_mem_initialized=1 but
sh.minsize=0. If you then call secure_malloc(), which then calls,
sh_malloc(), this then enters an infite loop since 0 << anything will
never be larger than size.
   
Issue 3:
   
That same sh_malloc loop will loop forever for a size greater
than size_t/2 because i will proceed (assuming sh.minsize=16):
i=16, 32, 64, ..., size_t/8, size_t/4, size_t/2, 0, 0, 0, 0, 
This sequence will never be larger than "size".
   
Reviewed-by: Rich Salz 
Reviewed-by: Richard Levitte 
(Merged from https://github.com/openssl/openssl/pull/3449)

which adds test cases intended to trigger the edge cases being fixed.

> Sorry.
>
>   How locked up is the machine?  Can you get memory usage stats or is it 
> completely unresponsive?
>
> Completely unresponsive. Totally. No memory usage. The only thing that works 
> at this point is the power button.

It seems that there should also be a bug report against OS X, as regular
userspace code running as non-root should not be able to hang a machine
like that.

>From just looking at the code, the only question that comes to mind is
whether you have a 32- or 64-bit size_t in the build environment in
question, which is unlikely to cause a eureka moment :(

-Ben
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] 90-test_secmem.t hangs the machine for good

2017-05-12 Thread Blumenthal, Uri - 0553 - MITLL 
On 5/12/17, 2:49 PM, "openssl-dev on behalf of Benjamin Kaduk via openssl-dev" 
 wrote:

➢ I’m sorry to report that in the current OpenSSL 1.1 master running “make test”
➢  freezes up the machine. Mac OS X 10.11.6, Xcode-8.2, current Github master. 
➢ Here’s the configuration:

  A commit hash would be more useful than "current github master"

I thought you know what’s in the current master right now. But here’s the last 
few hashes for your pleasure:

$ git log
commit 80a2fc4100daf6f1001eee33ef2f9b9eee05bedf (HEAD -> master, origin/master, 
origin/HEAD)
Author: Todd Short 
Date:   Wed May 10 11:44:55 2017 -0400

Clean up SSL_OP_* a bit

Reviewed-by: Matt Caswell 
Reviewed-by: Rich Salz 
(Merged from https://github.com/openssl/openssl/pull/3439)

commit 33242d9d79e7f06151e905b83dc8f995006fa7cd
Author: Rich Salz 
Date:   Thu May 11 20:42:32 2017 -0400

Use scalar, not length; fixes test_evp

Reviewed-by: Stephen Henson 
Reviewed-by: Richard Levitte 
(Merged from https://github.com/openssl/openssl/pull/3452)


  I can understand not wanting to have to power-cycle the machine again,
  but the 'make TESTS=test_secmem V=1 test' output (or some dtruss/similar)
  would be helpful in tracking things down.

Sorry.

  How locked up is the machine?  Can you get memory usage stats or is it 
completely unresponsive?

Completely unresponsive. Totally. No memory usage. The only thing that works at 
this point is the power button.


smime.p7s
Description: S/MIME cryptographic signature
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] 90-test_secmem.t hangs the machine for good

2017-05-12 Thread Salz, Rich via openssl-dev 
Current master 1d0f116 fails on my machine.

../test/recipes/90-test_secmem.t ..
1..1
# Subtest: ./secmemtest
1..1
# INFO:  @ test/secmemtest.c:65
# Possible infinite loop: allocate more than available

# INFO:  @ test/secmemtest.c:71
# Possible infinite loop: small arena

# INFO:  @ test/secmemtest.c:79
# Possible infinite loop: 1<<31 limit

crypto/mem_sec.c:428: OpenSSL internal error: assertion failed: sh.map_result 
!= MAP_FAILED
../util/shlib_wrap.sh ./secmemtest => 134
not ok 1 - running secmemtest

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] 90-test_secmem.t hangs the machine for good

2017-05-12 Thread Benjamin Kaduk via openssl-dev 
On 05/12/2017 01:34 PM, Blumenthal, Uri - 0553 - MITLL wrote:
>
> I’m sorry to report that in the current OpenSSL 1.1 master running
> “make test” freezes up the machine. Mac OS X 10.11.6, Xcode-8.2,
> current Github master. Here’s the configuration:
>

A commit hash would be more useful than "current github master"

>  
>
> ./Configure darwin64-x86_64-cc enable-threads enable-shared
> enable-zlib enable-ec_nistp_64_gcc_128 enable-rfc3779 enable-rc5
> enable-tls1_3 --prefix=/Users/uri/src/openssl-1.1
> --openssldir=/Users/uri/src/openssl-1.1/etc
>
>  
>
> Then of course “make depend && make clean && make all && make test”
>
>  
>
> ../test/recipes/90-test_ige.t . ok   
>
> ../test/recipes/90-test_memleak.t . ok   
>
> ../test/recipes/90-test_overhead.t  skipped: Only
> supported in no-shared builds
>
> ../test/recipes/90-test_secmem.t .. 
>
>  
>
> At this point the machine has to be power-cycled.
>
> —
>


I can understand not wanting to have to power-cycle the machine again,
but the 'make TESTS=test_secmem V=1 test' output (or some
dtruss/similar) would be helpful in tracking things down.

How locked up is the machine?  Can you get memory usage stats or is it
completely unresponsive?

-Ben
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev