Re: cryptopp-565 core dumps on solaris 11 with sun compiler in unit tests and benchmarks

2016-12-12 Thread Jeffrey Walton 


>
>> Can you bring me on as a subcontractor?
>>
>
> I'd love to but I am pleased to report that there's no need - I have got 
> it working! It turns out it was indeed a compiler bug in version 12.4. I 
> suspected as much from the stack trace where it showed a string address was 
> being misinterpreted as a this pointer, causing an error in a virtual 
> function dispatch.
>
> After several calls to Rogue Wave support I eventually found out that in 
> order to start using C++11 we would not only have to move from Source2016 
> to the fix release SourcePro2016.1, but we would also have to change 
> compiler from 12.4 to 12.5.
>
> I just tried cryptopp 565 with 12.5 using the same compiler options that 
> were used for the build of SourcePro12.5 and it worked. The benchmarks now 
> complete successfully.
>

Congrats on clearing it. I'm sorry you had to suffer it alone.

I'd like to get it written up at 
https://www.cryptopp.com/wiki/Solaris_(Command_Line) for the next 
unsuspecting person who comes along.

How would you write this up? Can you suggest a heading and some text?

Jeff

-- 
-- 
You received this message because you are subscribed to the "Crypto++ Users" 
Google Group.
To unsubscribe, send an email to cryptopp-users-unsubscr...@googlegroups.com.
More information about Crypto++ and this group is available at 
http://www.cryptopp.com.
--- 
You received this message because you are subscribed to the Google Groups 
"Crypto++ Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to cryptopp-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: cryptopp-565 core dumps on solaris 11 with sun compiler in unit tests and benchmarks

2016-10-31 Thread Jeffrey Walton 


> Another workaround could be: (1) don't use libcryptopp.a, and (2) use 
>
>> libcryptopp.so.
>
>
> Tried that. Didn't help.
>

That's a drag. I thought you might be compiling against one vrsion, and 
then runtime linking against another version.
 

> Can you get me remote access to that machine? 
>>
>
> No. It does not belong to me, it belongs to my client.
>  
>

Can you bring me on as a subcontractor?

 Jeff

-- 
-- 
You received this message because you are subscribed to the "Crypto++ Users" 
Google Group.
To unsubscribe, send an email to cryptopp-users-unsubscr...@googlegroups.com.
More information about Crypto++ and this group is available at 
http://www.cryptopp.com.
--- 
You received this message because you are subscribed to the Google Groups 
"Crypto++ Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to cryptopp-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: cryptopp-565 core dumps on solaris 11 with sun compiler in unit tests and benchmarks

2016-10-31 Thread Andrew Marlow 
On Monday, 31 October 2016 10:49:16 UTC, Jeffrey Walton wrote:
>
> Yuk... In a vacuum and absent of a memory smash, a bad "this" pointer 
> is usually caused by C++ static initialization order problems. 

[snip]
How are you building the library? Are {cryptlib.o, cpu.o, integer.o} 

> the first three objects fed to the linker? 


The supplied GNUmakefile has a link line that starts with the object files: 
cryptlib.o integer.o seed.o shacal2.o
 

> The remaining *.o files are 
> "don't care's". However, if you have a file-scope static, then the 
> object file needs to be after {cryptlib.o, cpu.o, integer.o}. 
> {cryptlib.o, cpu.o, integer.o} must be initialized first: 
>
[snip]
Remember, I am no longer investigating why my program goes wrong. I am 
looking at cryptest.exe b for running the benchmarks. This shows exactly 
the same issue and takes my program code out the equation.

Another workaround could be: (1) don't use libcryptopp.a, and (2) use 

> libcryptopp.so.


Tried that. Didn't help.
I could be completely wrong. I've also seen problems with the 

> initialized data segment on OS X. That's why we added 
> CRYPTOPP_SECTION_INIT. I wonder if Solaris is experiencing a similar 
> issue. Does -fno-common help the problem? 
>

Nope. Tried that, no change.
 

>
> On early .Net 2002 and 2003 I saw some odd vtable problems. But I'm 
> pretty sure it was do to CRYPTOPP_NO_VTABLE on MS platforms. On 
> Unix'es the macro is defined to nothing so it should not be 
> contributing to the issue you are experiencing. 
>

Ok. Thanks for the explanation.
 

>
> Can you get me remote access to that machine? 
>

No. It does not belong to me, it belongs to my client.

 

>
> Jeff


I tried making GetVoidValue impure and put trace there to see if the 
virtual function dispatch was maybe a pure virtual being called somehow. No 
luck.

I also tried indirecting the call via a new non-virtual function that is 
not template code and whose implementation is in the cpp file, just in case 
that might help. It didn't. The new function appears in the stack trace but 
the wrong function still gets called when the call to GetVoidValue is 
called, even when the source code qualifies the function name with "this->".
 

>
>
> > I have spent some more time in the debugger and have some more 
> information 
> > on this problem. Unfortunately I have no resolution at the moment. 
> > 
> > The Integer copy ctor (integer.cpp:2903) is being given a bad integer to 
> > copy. The m_ptr is null, the m_size is 144. This integer is returned 
> from 
> > RSAFunction::PreimageBound. The "this" pointer at this juncture seems to 
> be 
> > bad. It turns out the pointer value is actually a pointer to char* 
> rather 
> > than the object expected.

[snip]

-- 
-- 
You received this message because you are subscribed to the "Crypto++ Users" 
Google Group.
To unsubscribe, send an email to cryptopp-users-unsubscr...@googlegroups.com.
More information about Crypto++ and this group is available at 
http://www.cryptopp.com.
--- 
You received this message because you are subscribed to the Google Groups 
"Crypto++ Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to cryptopp-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: cryptopp-565 core dumps on solaris 11 with sun compiler in unit tests and benchmarks

2016-10-31 Thread Jeffrey Walton 
Yuk... In a vacuum and absent of a memory smash, a bad "this" pointer
is usually caused by C++ static initialization order problems. On
other OS'es it usually surfaces as a null this pointer.

I have spent countless hours trying to wring-out the problems. We
mostly have them solved on Windows and Linux. Solaris and MacPorts are
the hold outs because they don't respond to init_seg(lib) or
init_priority. Also see
https://cryptopp.com/wiki/Static_Initialization_Order_Fiasco.

How are you building the library? Are {cryptlib.o, cpu.o, integer.o}
the first three objects fed to the linker? The remaining *.o files are
"don't care's". However, if you have a file-scope static, then the
object file needs to be after {cryptlib.o, cpu.o, integer.o}.
{cryptlib.o, cpu.o, integer.o} must be initialized first:

// As an example:
$CXX $CXXFLAGS cryptlib.o, cpu.o, integer.o my_main.o -o prog.exe

Something else that kind of sucks when trying to diagnose a C++ static
initialization order problems: we don't have tools to audit for it.
I've been looking for them for years. Also see
http://stackoverflow.com/q/34144185.

Another workaround could be: (1) don't use libcryptopp.a, and (2) use
libcryptopp.so. I believe the module boundary ensures the C++ static
objects are initialized properly.

I could be completely wrong. I've also seen problems with the
initialized data segment on OS X. That's why we added
CRYPTOPP_SECTION_INIT. I wonder if Solaris is experiencing a similar
issue. Does -fno-common help the problem?

On early .Net 2002 and 2003 I saw some odd vtable problems. But I'm
pretty sure it was do to CRYPTOPP_NO_VTABLE on MS platforms. On
Unix'es the macro is defined to nothing so it should not be
contributing to the issue you are experiencing.

Can you get me remote access to that machine?

Jeff

> I have spent some more time in the debugger and have some more information
> on this problem. Unfortunately I have no resolution at the moment.
>
> The Integer copy ctor (integer.cpp:2903) is being given a bad integer to
> copy. The m_ptr is null, the m_size is 144. This integer is returned from
> RSAFunction::PreimageBound. The "this" pointer at this juncture seems to be
> bad. It turns out the pointer value is actually a pointer to char* rather
> than the object expected. The string is the value returned from
> GetThisObject. Single stepping in dbx I get to RSAFunction::AssignFrom. At
> this point the "this" ptr is ok. I step into RSAFunction::GetThisObject
> which calls GetValue which calls GetVoidValue. This is where it calls
> RSAFunction::PreboundImage with m_n equal to that empty, troublesome
> Integer. But there is no way that the function should even have been called.
> GetVoidValue is a pure virtual on the base class (NameValuePairs) so it
> should have done a virtual function dispatch to the relevant GetVoidValue
> function. It didn't . On landing in RSAFunction::PreboundImage the "this"
> pointer is equal to the name value in GetVoidValue, a string pointing to
> "ThisObject:CryptoPP::RSAFunction". What could be causing the vptr dispatch
> to go wrong?  I note that the base class, NameValuePairs, is decorated with
> the macro CRYPTOPP_NO_VTABLE. It seems quite a coincidence that I am having
> vptr trouble and there is this macro with a name like that. The macro is a
> no-op though unles the Microsoft compiler is being used (I am on solaris 11
> sparc with the sunstudio 12.4 compiler). Can someone please explain what
> that macro is about?
>
> So this kind of explains why the integer is bad but nothing can detect it
> and nothing assigns it this bad value. But what to do from here? I am stuck.

-- 
-- 
You received this message because you are subscribed to the "Crypto++ Users" 
Google Group.
To unsubscribe, send an email to cryptopp-users-unsubscr...@googlegroups.com.
More information about Crypto++ and this group is available at 
http://www.cryptopp.com.
--- 
You received this message because you are subscribed to the Google Groups 
"Crypto++ Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to cryptopp-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: cryptopp-565 core dumps on solaris 11 with sun compiler in unit tests and benchmarks

2016-10-31 Thread Andrew Marlow 
On Friday, 14 October 2016 11:33:53 UTC+1, Andrew Marlow wrote:
>
> Hello,
>
> I am sorry to report that cryptest.exe v still core dumps on solaris 11 
> when using the sun 12.4 compiler. The command I used to build cryptopp was: 
> CXX=/opt/solarisstudio12.4/bin/CC make -j20
>
> The error is:
>
> Testing MessageDigest algorithm SHA-384.
> ..signal BUS (invalid address alignment) in CryptoPP::SHA512::Transform at 
> line 34 in file "sha.cpp"
>34   #define blk0(i) (W[i] = data[i])
>
> the stack trace is:
>
> (dbx) where
> =>[1] CryptoPP::SHA512::Transform(state = , data = 
> ) (optimized), at 0x1006255a0 (line ~34) in "sha.cpp"
>   [2] CryptoPP::IteratedHashWithStaticTransform long,CryptoPP::EnumToType,128U,64U,CryptoPP::SHA384,48U,false>::HashEndianCorrectedBlock(this
>  
> = 0x1010c18d0, data = 0x7fffc1b4) (optimized), at 0x1004c8120 (line 
> ~170) in "iterhash.h"
>   [3] CryptoPP::IteratedHashBase long,CryptoPP::HashTransformation>::HashMultipleBlocks(this = 0x1010c18d0, 
> input = 0x7fffc1b4, length = ) (optimized), at 
> 0x1005d834c (line ~91) in "iterhash.cpp"
>   [4] CryptoPP::IteratedHashBase long,CryptoPP::HashTransformation>::Update(this = 0x1010c18d0, input = 
> 0x7fffc1b4 "aaa [snip]
>   [5] CryptoPP::HashVerificationFilter::NextPutMultiple(this = 
> 0x7fffd550, inString = 0x7fffc15d "aaa [snip]
>   [6] CryptoPP::FilterWithBufferedInput::PutMaybeModifiable(this = 
> 0x7fffd550, inString = 0x7fffc15d 
> "a [snip]
>   [7] CryptoPP::FilterWithBufferedInput::Put2(this = 0x7fffd550, 
> inString = 0x7fffc15d 
> "aaa
>   [8] CryptoPP::BufferedTransformation::ChannelPut2(this = 
> 0x7fffd550, channel = CLASS, begin = 0x7fffc15d 
> "aaa
>   [9] RandomizedTransfer(source = CLASS, target = CLASS, finish =  unavailable>, channel = CLASS) (optimized), at 0x1004e9e94 (line ~92) in 
> "datatest.cpp"
>   [10] PutDecodedDatumInto(data = CLASS, name = , 
> target = CLASS) (optimized), at 0x1004ea41c (line ~138) in "datatest.cpp"
>   [11] TestDigestOrMAC(v = CLASS, testDigest = ) 
> (optimized), at 0x1004ef674 (line ~603) in "datatest.cpp"
>   [12] TestDataFile(filename = CLASS, overrideParameters = CLASS, 
> totalTests = 11U, failedTests = 0) (optimized), at 0x1004f0c44 (line ~802) 
> in "datatest.cpp"
>   [13] RunTestDataFile(filename = 0x100afec60 "TestVectors/sha.txt", 
> overrideParameters = CLASS, thorough = true) (optimized), at 0x1004f1168 
> (line ~243) in "string"
>   [14] ValidateSHA() (optimized), at 0x1004c9228 (line ~212) in 
> "validat3.cpp"
>   [15] ValidateAll(thorough = false) (optimized), at 0x1004339e8 (line 
> ~95) in "validat1.cpp"
>   [16] Validate(alg = , thorough = false, seedInput = 
> ) (optimized), at 0x100380cdc (line ~899) in "test.cpp"
>   [17] main(argc = , argv = 0x77d6) 
> (optimized), at 0x10037b690 (line ~364) in "test.cpp"
>
> The test program also crashes when the b (benchmark) option is used. 
> Interestingly, the crash is in the same place as my own test program 
> crashes, in CryptoPP::CountWords, due to a null pointer.
>
[snip]

I have spent some more time in the debugger and have some more information 
on this problem. Unfortunately I have no resolution at the moment.

The Integer copy ctor (integer.cpp:2903) is being given a bad integer to 
copy. The m_ptr is null, the m_size is 144. This integer is returned from 
RSAFunction::PreimageBound. The "this" pointer at this juncture seems to be 
bad. It turns out the pointer value is actually a pointer to char* rather 
than the object expected. The string is the value returned from 
GetThisObject. Single stepping in dbx I get to RSAFunction::AssignFrom. At 
this point the "this" ptr is ok. I step into RSAFunction::GetThisObject 
which calls GetValue which calls GetVoidValue. This is where it calls 
RSAFunction::PreboundImage with m_n equal to that empty, troublesome 
Integer. But there is no way that the function should even have been 
called. GetVoidValue is a pure virtual on the base class (NameValuePairs) 
so it should have done a virtual function dispatch to the relevant 
GetVoidValue function. It didn't . On landing in RSAFunction::PreboundImage 
the "this" pointer is equal to the name value in GetVoidValue, a string 
pointing to "ThisObject:CryptoPP::RSAFunction". What could be causing the 
vptr dispatch to go wrong?  I note that the base class, NameValuePairs, is 
decorated with the macro CRYPTOPP_NO_VTABLE. It seems quite a coincidence 
that I am having vptr trouble and there is this macro with a name like 
that. The macro is a no-op though unles the Microsoft compiler is being 
used (I am on solaris 11 sparc with the sunstudio 12.4 compiler). Can 
someone please explain what that macro is about?

So this kind of explains why the integer is bad but nothing can detect it 
and

Re: cryptopp-565 core dumps on solaris 11 with sun compiler in unit tests and benchmarks

2016-10-19 Thread Andrew Marlow 
On Tuesday, 18 October 2016 08:16:58 UTC+1, Jeffrey Walton wrote:

> > The error is: 
>>> > 
>>> > Testing MessageDigest algorithm SHA-384. 
>>> > ..signal BUS (invalid address alignment) in 
>>> CryptoPP::SHA512::Transform at 
>>> > line 34 in file "sha.cpp" 
>>> >34   #define blk0(i) (W[i] = data[i]) 
>>>
>>> This is the one we cannot duplicate. Unfortunately, there's nothing we 
>>> can do for this one until we can duplicate it. 
>>>
>>
>> The command I used to build cryptopp was: 
>> CXX=/opt/solarisstudio12.4/bin/CC make -j20
>>
>
> That's interesting. Is make linked to GNU's make? Or another make?
>  
>
It is GNU make.
 

>
>> Can someone, maybe Jeff, please let me know what command was tried to 
>> reproduce the problem on Solaris 11 SPARC. It may be that I am not building 
>> cryptopp properly.
>>
>
> The commands I run during smoke testing are either:
>
>   (1) straight gmake# default C++ compiler
>   (2) CXX=.../CC gmake# SunCC compiler 
>
>  
Step 2 is just what I do.
 

> We added a section at 
> https://cryptopp.com/wiki/Solaris_(Command_Line)#Default_Make. It shows 
> the commands and the outputs we see when we run the commands.
>

That section is very helpful, thanks for writing it. It shows that the 
command is:

CC -DNDEBUG -g3 -xO2 -m64 -native -KPIC -template=no%extdef -c

whereas what I get is:

CC -DNDEBUG -g3 -xO2 -fPIC -pipe -m64 -native -KPIC -template=no%extdef -w 
-erroff=wvarhidemem -erroff=voidretw -c 

So I get the extra options -pipe -w -erroff=wvarhidemem -erroff=voidretw

The -pipe option causes:

CC: Warning: Option -pipe passed to ld, if ld is invoked, ignored otherwise

to be emitted.

Maybe CountWords could check that the pointer X is not null and throw 
InvalidArgument if it is? I know that represents a coding error since the 
pointer should never be null but it is null in my case and this is crashing 
our test harness. Hopefully a nullity check will not be deemed too 
expensive.

-- 
-- 
You received this message because you are subscribed to the "Crypto++ Users" 
Google Group.
To unsubscribe, send an email to cryptopp-users-unsubscr...@googlegroups.com.
More information about Crypto++ and this group is available at 
http://www.cryptopp.com.
--- 
You received this message because you are subscribed to the Google Groups 
"Crypto++ Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to cryptopp-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: cryptopp-565 core dumps on solaris 11 with sun compiler in unit tests and benchmarks

2016-10-18 Thread Jeffrey Walton 


> The error is: 
>> > 
>> > Testing MessageDigest algorithm SHA-384. 
>> > ..signal BUS (invalid address alignment) in CryptoPP::SHA512::Transform 
>> at 
>> > line 34 in file "sha.cpp" 
>> >34   #define blk0(i) (W[i] = data[i]) 
>>
>> This is the one we cannot duplicate. Unfortunately, there's nothing we 
>> can do for this one until we can duplicate it. 
>>
>
> The command I used to build cryptopp was: 
> CXX=/opt/solarisstudio12.4/bin/CC make -j20
>

That's interesting. Is make linked to GNU's make? Or another make?
 

>
> Can someone, maybe Jeff, please let me know what command was tried to 
> reproduce the problem on Solaris 11 SPARC. It may be that I am not building 
> cryptopp properly.
>

The commands I run during smoke testing are either:

  (1) straight gmake# default C++ compiler
  (2) CXX=.../CC gmake# SunCC compiler 

We added a section at 
https://cryptopp.com/wiki/Solaris_(Command_Line)#Default_Make. It shows the 
commands and the outputs we see when we run the commands.

Jeff

-- 
-- 
You received this message because you are subscribed to the "Crypto++ Users" 
Google Group.
To unsubscribe, send an email to cryptopp-users-unsubscr...@googlegroups.com.
More information about Crypto++ and this group is available at 
http://www.cryptopp.com.
--- 
You received this message because you are subscribed to the Google Groups 
"Crypto++ Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to cryptopp-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: cryptopp-565 core dumps on solaris 11 with sun compiler in unit tests and benchmarks

2016-10-17 Thread Andrew Marlow 


On Friday, 14 October 2016 12:10:30 UTC+1, Jeffrey Walton wrote:
>
> > I am sorry to report that cryptest.exe v still core dumps on solaris 11 
> when 
> > using the sun 12.4 compiler. The command I used to build cryptopp was: 
> > CXX=/opt/solarisstudio12.4/bin/CC make -j20 
> > 
> > The error is: 
> > 
> > Testing MessageDigest algorithm SHA-384. 
> > ..signal BUS (invalid address alignment) in CryptoPP::SHA512::Transform 
> at 
> > line 34 in file "sha.cpp" 
> >34   #define blk0(i) (W[i] = data[i]) 
>
> This is the one we cannot duplicate. Unfortunately, there's nothing we 
> can do for this one until we can duplicate it. 
>

The command I used to build cryptopp was: CXX=/opt/solarisstudio12.4/bin/CC 
make -j20

Can someone, maybe Jeff, please let me know what command was tried to 
reproduce the problem on Solaris 11 SPARC. It may be that I am not building 
cryptopp properly.

-- 
-- 
You received this message because you are subscribed to the "Crypto++ Users" 
Google Group.
To unsubscribe, send an email to cryptopp-users-unsubscr...@googlegroups.com.
More information about Crypto++ and this group is available at 
http://www.cryptopp.com.
--- 
You received this message because you are subscribed to the Google Groups 
"Crypto++ Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to cryptopp-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: cryptopp-565 core dumps on solaris 11 with sun compiler in unit tests and benchmarks

2016-10-14 Thread Jeffrey Walton 
> I am sorry to report that cryptest.exe v still core dumps on solaris 11 when
> using the sun 12.4 compiler. The command I used to build cryptopp was:
> CXX=/opt/solarisstudio12.4/bin/CC make -j20
>
> The error is:
>
> Testing MessageDigest algorithm SHA-384.
> ..signal BUS (invalid address alignment) in CryptoPP::SHA512::Transform at
> line 34 in file "sha.cpp"
>34   #define blk0(i) (W[i] = data[i])

This is the one we cannot duplicate. Unfortunately, there's nothing we
can do for this one until we can duplicate it.

Jeff

-- 
-- 
You received this message because you are subscribed to the "Crypto++ Users" 
Google Group.
To unsubscribe, send an email to cryptopp-users-unsubscr...@googlegroups.com.
More information about Crypto++ and this group is available at 
http://www.cryptopp.com.
--- 
You received this message because you are subscribed to the Google Groups 
"Crypto++ Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to cryptopp-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.