Re: Still one outstanding issue sine 20140909 releases

[Mike Bland]

As I mentioned a few days ago, can you tell whether the top-level
'rehash' target is getting executed as a prerequisite? Can you try
execting 'make rehash && make test' to see if that fixes the problem?
I suggest this because this looks very similar to an error I
encountered during my build system work.

Mike


On Thu, Sep 11, 2014 at 2:20 PM, The Doctor  wrote:
>
> Script started on Thu Sep 11 11:27:05 2014
> doctor.nl2k.ab.ca//usr/source/openssl-1.0.2-stable-SNAP-20140911$ make test
> testing...
> (cd ..; make DIRS=crypto all)
> making all in crypto...
> ar  r ../libcrypto.a cryptlib.o mem.o mem_dbg.o cversion.o ex_data.o 
> cpt_err.o ebcdic.o  uid.o o_time.o o_str.o o_dir.o o_fips.o o_init.o 
> fips_ers.o mem_clr.o
> test -z "" || ar  r ../libcrypto.a fipscanister.o
> /usr/bin/ranlib ../libcrypto.a || echo Never mind.
> making all in crypto/objects...
> making all in crypto/md2...
> making all in crypto/md4...
> making all in crypto/md5...
> making all in crypto/sha...
> making all in crypto/mdc2...
> making all in crypto/hmac...
> making all in crypto/ripemd...
> making all in crypto/whrlpool...
> making all in crypto/des...
> making all in crypto/aes...
> making all in crypto/rc2...
> making all in crypto/rc4...
> making all in crypto/rc5...
> making all in crypto/idea...
> making all in crypto/bf...
> making all in crypto/cast...
> making all in crypto/camellia...
> making all in crypto/seed...
> making all in crypto/modes...
> making all in crypto/bn...
> making all in crypto/ec...
> making all in crypto/rsa...
> making all in crypto/dsa...
> making all in crypto/ecdsa...
> making all in crypto/dh...
> making all in crypto/ecdh...
> making all in crypto/dso...
> making all in crypto/engine...
> making all in crypto/buffer...
> making all in crypto/bio...
> making all in crypto/stack...
> making all in crypto/lhash...
> making all in crypto/rand...
> making all in crypto/err...
> making all in crypto/evp...
> making all in crypto/asn1...
> making all in crypto/pem...
> making all in crypto/x509...
> making all in crypto/x509v3...
> making all in crypto/conf...
> making all in crypto/txt_db...
> making all in crypto/pkcs7...
> making all in crypto/pkcs12...
> making all in crypto/comp...
> making all in crypto/ocsp...
> making all in crypto/ui...
> making all in crypto/krb5...
> making all in crypto/cms...
> making all in crypto/pqueue...
> making all in crypto/ts...
> making all in crypto/jpake...
> making all in crypto/srp...
> making all in crypto/store...
> making all in crypto/cmac...
> if [ -n "libcrypto.so.1.0.0 libssl.so.1.0.0" ]; then  (cd ..; make 
> libcrypto.so.1.0.0);  fi
> [ -z "" ] || gcc3 -fPIC -DOPENSSL_PIC -DZLIB_SHARED -DZLIB -DOPENSSL_THREADS 
> -pthread -D_THREAD_SAFE -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DPERL5 
> -DL_ENDIAN -DTERMIOS -fomit-frame-pointer -O2 -Wall -g 
> -DOPENSSL_EXPERIMENTAL_JPAKE -DOPENSSL_EXPERIMENTAL_LIBUNBOUND 
> -DOPENSSL_EXPERIMENTAL_STORE -DOPENSSL_BN_ASM_PART_WORDS 
> -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM 
> -DSHA512_ASM -DMD5_ASM -DRMD160_ASM -DAES_ASM -DGHASH_ASM -Iinclude  
> -DFINGERPRINT_PREMAIN_DSO_LOAD -o fips_premain_dso   fips_premain.c 
> fipscanister.o  libcrypto.a -lgmp -ldl -lm -lc
> ( :;LIBDEPS="${LIBDEPS:-../libssl.a ../libcrypto.a  -lgmp -ldl -lm -lc}"; 
>  LDCMD="${LDCMD:-gcc3}"; LDFLAGS="${LDFLAGS:--fPIC -DOPENSSL_PIC 
> -DZLIB_SHARED -DZLIB -DOPENSSL_THREADS -pthread -D_THREAD_SAFE -D_REENTRANT 
> -DDSO_DLFCN -DHAVE_DLFCN_H -DPERL5 -DL_ENDIAN -DTERMIOS -fomit-frame-pointer 
> -O2 -Wall -g -DOPENSSL_EXPERIMENTAL_JPAKE -DOPENSSL_EXPERIMENTAL_LIBUNBOUND 
> -DOPENSSL_EXPERIMENTAL_STORE -DOPENSSL_BN_ASM_PART_WORDS 
> -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM 
> -DSHA512_ASM -DMD5_ASM -DRMD160_ASM -DAES_ASM -DGHASH_ASM}";  LIBPATH=`for x 
> in $LIBDEPS; do echo $x; done | sed -e 's/^ *-L//;t' -e d | uniq`;  
> LIBPATH=`echo $LIBPATH | sed -e 's/ /:/g'`;  
> LD_LIBRARY_PATH=$LIBPATH:$LD_LIBRARY_PATH  ${LDCMD} ${LDFLAGS} -o 
> ${APPNAME:=heartbeat_test} heartbeat_test.o ${LIBDEPS} )
> making all in apps...
> (cd ..; make DIRS=crypto all)
> making all in crypto...
> ar  r ../libcrypto.a cryptlib.o mem.o mem_dbg.o cversion.o ex_data.o 
> cpt_err.o ebcdic.o  uid.o o_time.o o_str.o o_dir.o o_fips.o o_init.o 
> fips_ers.o mem_clr.o
> test -z "" || ar  r ../libcrypto.a fipscanister.o
> /usr/bin/ranlib ../libcrypto.a || echo Never mind.
> making all in crypto/objects...
> making all in crypto/md2...
> making all in crypto/md4...
> making all in crypto/md5...
> making all in crypto/sha...
> making all in crypto/mdc2...
> making all in crypto/hmac...
> making all in crypto/ripemd...
> making all in crypto/whrlpool...
> making all in crypto/des...
> making all in crypto/aes...
> making all in crypto/rc2...
> making all in crypto/rc4...
> making all in crypto/rc5...
> making all in crypto/idea...
> making all in crypto/bf...
> making all in crypto/cast...
> making all in crypto/camellia...
>

Re: OPenssl 20140909 issues

[Mike Bland]

Is the top-level "rehash" target not getting executed? It should be a
dependency of "test" (via the "tests" target).

Mike

On Tue, Sep 9, 2014 at 1:41 AM, The Doctor  wrote:
> Just found this in the latest openssl 1.0.2 snapshot
>
>
> Script started on Mon Sep  8 23:19:16 2014
> doctor.nl2k.ab.ca//usr/source/openssl-1.0.2-stable-SNAP-20140909$ make test
> testing...
> (cd ..; make DIRS=crypto all)
> making all in crypto...
> ar  r ../libcrypto.a cryptlib.o mem.o mem_dbg.o cversion.o ex_data.o 
> cpt_err.o ebcdic.o  uid.o o_time.o o_str.o o_dir.o o_fips.o o_init.o 
> fips_ers.o mem_clr.o
> test -z "" || ar  r ../libcrypto.a fipscanister.o
> /usr/bin/ranlib ../libcrypto.a || echo Never mind.
> making all in crypto/objects...
> making all in crypto/md2...
> making all in crypto/md4...
> making all in crypto/md5...
> making all in crypto/sha...
> making all in crypto/mdc2...
> making all in crypto/hmac...
> making all in crypto/ripemd...
> making all in crypto/whrlpool...
> making all in crypto/des...
> making all in crypto/aes...
> making all in crypto/rc2...
> making all in crypto/rc4...
> making all in crypto/rc5...
> making all in crypto/idea...
> making all in crypto/bf...
> making all in crypto/cast...
> making all in crypto/camellia...
> making all in crypto/seed...
> making all in crypto/modes...
> making all in crypto/bn...
> making all in crypto/ec...
> making all in crypto/rsa...
> making all in crypto/dsa...
> making all in crypto/ecdsa...
> making all in crypto/dh...
> making all in crypto/ecdh...
> making all in crypto/dso...
> making all in crypto/engine...
> making all in crypto/buffer...
> making all in crypto/bio...
> making all in crypto/stack...
> making all in crypto/lhash...
> making all in crypto/rand...
> making all in crypto/err...
> making all in crypto/evp...
> making all in crypto/asn1...
> making all in crypto/pem...
> making all in crypto/x509...
> making all in crypto/x509v3...
> making all in crypto/conf...
> making all in crypto/txt_db...
> making all in crypto/pkcs7...
> making all in crypto/pkcs12...
> making all in crypto/comp...
> making all in crypto/ocsp...
> making all in crypto/ui...
> making all in crypto/krb5...
> making all in crypto/cms...
> making all in crypto/pqueue...
> making all in crypto/ts...
> making all in crypto/jpake...
> making all in crypto/srp...
> making all in crypto/store...
> making all in crypto/cmac...
> if [ -n "libcrypto.so.1.0.0 libssl.so.1.0.0" ]; then  (cd ..; make 
> libcrypto.so.1.0.0);  fi
> [ -z "" ] || gcc3 -fPIC -DOPENSSL_PIC -DZLIB_SHARED -DZLIB -DOPENSSL_THREADS 
> -pthread -D_THREAD_SAFE -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DPERL5 
> -DL_ENDIAN -DTERMIOS -fomit-frame-pointer -O2 -Wall -g 
> -DOPENSSL_EXPERIMENTAL_JPAKE -DOPENSSL_EXPERIMENTAL_LIBUNBOUND 
> -DOPENSSL_EXPERIMENTAL_STORE -DOPENSSL_BN_ASM_PART_WORDS 
> -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM 
> -DSHA512_ASM -DMD5_ASM -DRMD160_ASM -DAES_ASM -DGHASH_ASM -Iinclude  
> -DFINGERPRINT_PREMAIN_DSO_LOAD -o fips_premain_dso   fips_premain.c 
> fipscanister.o  libcrypto.a -lgmp -ldl -lm -lc
> (cd ..; make DIRS=ssl all)
> making all in ssl...
> if [ -n "libcrypto.so.1.0.0 libssl.so.1.0.0" ]; then  (cd ..; make 
> libssl.so.1.0.0);  fi
> [ -z "" ] || gcc3 -fPIC -DOPENSSL_PIC -DZLIB_SHARED -DZLIB -DOPENSSL_THREADS 
> -pthread -D_THREAD_SAFE -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DPERL5 
> -DL_ENDIAN -DTERMIOS -fomit-frame-pointer -O2 -Wall -g 
> -DOPENSSL_EXPERIMENTAL_JPAKE -DOPENSSL_EXPERIMENTAL_LIBUNBOUND 
> -DOPENSSL_EXPERIMENTAL_STORE -DOPENSSL_BN_ASM_PART_WORDS 
> -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM 
> -DSHA512_ASM -DMD5_ASM -DRMD160_ASM -DAES_ASM -DGHASH_ASM -Iinclude  
> -DFINGERPRINT_PREMAIN_DSO_LOAD -o fips_premain_dso   fips_premain.c 
> fipscanister.o  libcrypto.a -lgmp -ldl -lm -lc
> ( :;LIBDEPS="${LIBDEPS:-../libssl.a ../libcrypto.a  -lgmp -ldl -lm -lc}"; 
>  LDCMD="${LDCMD:-gcc3}"; LDFLAGS="${LDFLAGS:--fPIC -DOPENSSL_PIC 
> -DZLIB_SHARED -DZLIB -DOPENSSL_THREADS -pthread -D_THREAD_SAFE -D_REENTRANT 
> -DDSO_DLFCN -DHAVE_DLFCN_H -DPERL5 -DL_ENDIAN -DTERMIOS -fomit-frame-pointer 
> -O2 -Wall -g -DOPENSSL_EXPERIMENTAL_JPAKE -DOPENSSL_EXPERIMENTAL_LIBUNBOUND 
> -DOPENSSL_EXPERIMENTAL_STORE -DOPENSSL_BN_ASM_PART_WORDS 
> -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM 
> -DSHA512_ASM -DMD5_ASM -DRMD160_ASM -DAES_ASM -DGHASH_ASM}";  LIBPATH=`for x 
> in $LIBDEPS; do echo $x; done | sed -e 's/^ *-L//;t' -e d | uniq`;  
> LIBPATH=`echo $LIBPATH | sed -e 's/ /:/g'`;  
> LD_LIBRARY_PATH=$LIBPATH:$LD_LIBRARY_PATH  ${LDCMD} ${LDFLAGS} -o 
> ${APPNAME:=heartbeat_test} heartbeat_test.o ${LIBDEPS} )
> making all in apps...
> (cd ..; make DIRS=crypto all)
> making all in crypto...
> ar  r ../libcrypto.a cryptlib.o mem.o mem_dbg.o cversion.o ex_data.o 
> cpt_err.o ebcdic.o  uid.o o_time.o o_str.o o_dir.o o_fips.o o_init.o 
> fips_ers.o mem_clr.o
> test -z "" || ar  r ../libcrypto.a fipscanister.o

Re: [openssl.org #3510] AutoReply: Clang warning/error fixes

[Mike Bland via RT]

Withdrawn. Commits b0426a0f8c6ce7656411b037e0c45465320cb325 and
86f50b36e63275a916b147f9d8764e3c0c060fdb are identical to those in the
original pull request.

Mike

On Sun, Aug 31, 2014 at 8:19 AM, The default queue via RT
 wrote:
>
> Greetings,
>
> This message has been automatically generated in response to the
> creation of a trouble ticket regarding:
> "Clang warning/error fixes",
> a summary of which appears below.
>
> There is no need to reply to this message right now.  Your ticket has been
> assigned an ID of [openssl.org #3510].
>
> Please include the string:
>
>  [openssl.org #3510]
>
> in the subject line of all future correspondence about this issue. To do so,
> you may reply to this message.
>
> Thank you,
> r...@openssl.org
>
> -
> Pull request #145 [openssl.org #3447] contains the commit
> "{,darwin64-}debug-test-64-clang Configure targets". A couple of
> recent commits on openssl:master cause builds configured for these
> targets to fail. The commits in this pull request contain fixes for
> these issues:
>
> https://github.com/openssl/openssl/pull/167
>
> Mike
>


__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org

[openssl.org #3510] Clang warning/error fixes

[Mike Bland via RT]

Pull request #145 [openssl.org #3447] contains the commit
"{,darwin64-}debug-test-64-clang Configure targets". A couple of
recent commits on openssl:master cause builds configured for these
targets to fail. The commits in this pull request contain fixes for
these issues:

https://github.com/openssl/openssl/pull/167

Mike

__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org

[openssl.org #3500] Adopting single-Makefile build structure

[Mike Bland via RT]

As per my experimentation, reported results and ensuing discussion,
I'd appreciate a decision on whether to move forward with adopting a
"single-Makefile" build structure:

Report (Google Docs): http://goo.gl/yhvCno
https://groups.google.com/d/topic/openssl-testing/AUJME_4xkWM/discussion
https://groups.google.com/d/topic/mailing.openssl.dev/F1CkO4WCrv8/discussion

Further details and the code itself is available in pull request #160:

https://github.com/openssl/openssl/pull/160

Thanks,

Mike

__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org

Re: [openssl.org #3497] Move dclean actions to clean

[Mike Bland]

Whoops, OK. :-P

Mike

On Thu, Aug 21, 2014 at 1:22 PM, Salz, Rich  wrote:
>> Just generated a pull request for this; let me know if it's what you actually
>> had in mind:
>>
>> https://github.com/openssl/openssl/pull/161
>
> I already had the fix in-hand :)  See attached.
>
>
>
> --
> Principal Security Engineer
> Akamai Technologies, Cambridge MA
> IM: rs...@jabber.me Twitter: RichSalz
>
>
> -- Forwarded message --
> From: "Salz, Rich" 
> To: "openssl-t...@openssl.org" 
> Cc:
> Date: Thu, 21 Aug 2014 12:47:12 -0400
> Subject: Code review 3497: clean up dclean/clean targets
> commit e02129dea29f7d091ce301b4f34e05559bb840d3
> Author: Rich Salz 
> Date:   Thu Aug 21 12:45:28 2014 -0400
>
> RT3497: (re)move actions from dclean to clean
>
> Remove all special actions (anything other than building
> dependencies) from dclean target to clean target.
> Also, consistently use RECURSIVE_MAKE (not RECURSIVE_BUILD_CMD)
> for all Makefiles that did recursive builds.
>
> diff --git a/Makefile.fips b/Makefile.fips
> index b3811df..84a85f0 100644
> --- a/Makefile.fips
> +++ b/Makefile.fips
> @@ -251,23 +251,23 @@ BUILDENV= PLATFORM='$(PLATFORM)' 
> PROCESSOR='$(PROCESSOR)' \
>  # BUILD_CMD is a generic macro to build a given target in a given
>  # subdirectory.  The target must be given through the shell variable
>  # `target' and the subdirectory to build in must be given through `dir'.
> -# This macro shouldn't be used directly, use RECURSIVE_BUILD_CMD or
> -# BUILD_ONE_CMD instead.
> +# This macro shouldn't be used directly, use RECURSIVE_MAKE or
> +# MAKE_ONE instead.
>  #
> -# BUILD_ONE_CMD is a macro to build a given target in a given
> -# subdirectory if that subdirectory is part of $(DIRS).  It requires
> -# exactly the same shell variables as BUILD_CMD.
> -#
> -# RECURSIVE_BUILD_CMD is a macro to build a given target in all
> +# RECURSIVE_MAKE is a macro to build a given target in all
>  # subdirectories defined in $(DIRS).  It requires that the target
>  # is given through the shell variable `target'.
> +#
> +# MAKE_ONE is a macro to build a given target in a given
> +# subdirectory if that subdirectory is part of $(DIRS).  It requires
> +# exactly the same shell variables as BUILD_CMD.
>  BUILD_CMD=  if [ -d "$$dir" ]; then \
> (   cd $$dir && echo "making $$target in $$dir..." && \
> $(CLEARENV) && $(MAKE) -e $(BUILDENV) TOP=.. DIR=$$dir 
> $$target \
> ) || exit 1; \
> fi
> -RECURSIVE_BUILD_CMD=for dir in $(DIRS); do $(BUILD_CMD); done
> -BUILD_ONE_CMD=\
> +RECURSIVE_MAKE=for dir in $(DIRS); do $(BUILD_CMD); done
> +MAKE_ONE=\
> if expr " $(DIRS) " : ".* $$dir " >/dev/null 2>&1; then \
> $(BUILD_CMD); \
> fi
> @@ -364,7 +364,7 @@ build_all: build_libs
>  build_libs: build_crypto build_fips
>
>  build_fips:
> -   @dir=fips; target=all; [ -z "$(FIPSCANLIB)" ] || $(BUILD_ONE_CMD)
> +   @dir=fips; target=all; [ -z "$(FIPSCANLIB)" ] || $(MAKE_ONE)
>
>  build_crypto:
> if [ -n "$(FIPSCANLIB)" ]; then \
> @@ -378,23 +378,23 @@ build_crypto:
> else \
> AS='$(CC) -c' ; \
> fi ; export AS ; \
> -   dir=crypto; target=fips; $(BUILD_ONE_CMD)
> +   dir=crypto; target=fips; $(MAKE_ONE)
>  build_ssl:
> -   @dir=ssl; target=all; $(BUILD_ONE_CMD)
> +   @dir=ssl; target=all; $(MAKE_ONE)
>  build_engines:
> -   @dir=engines; target=all; $(BUILD_ONE_CMD)
> +   @dir=engines; target=all; $(MAKE_ONE)
>  build_apps:
> -   @dir=apps; target=all; $(BUILD_ONE_CMD)
> +   @dir=apps; target=all; $(MAKE_ONE)
>  build_tests:
> -   @dir=test; target=fipsexe; $(BUILD_ONE_CMD)
> +   @dir=test; target=fipsexe; $(MAKE_ONE)
>  build_algvs:
> -   @dir=test; target=fipsalgvs; $(BUILD_ONE_CMD)
> +   @dir=test; target=fipsalgvs; $(MAKE_ONE)
>  build_tools:
> -   @dir=tools; target=all; $(BUILD_ONE_CMD)
> +   @dir=tools; target=all; $(MAKE_ONE)
>
>  all_testapps: build_libs build_testapps
>  build_testapps:
> -   @dir=crypto; target=testapps; $(BUILD_ONE_CMD)
> +   @dir=crypto; target=testapps; $(MAKE_ONE)
>
>  libcrypto$(SHLIB_EXT): libcrypto.a build_fips
> @if [ "$(SHLIB_TARGET)" != "" ]; then \
> @@ -503,11 +503,12 @@ libclean:
>
>  clean: libclean
> rm -f shlib/*.o *.o core a.out fluff testlog make.log cctest cctest.c
> -   @set -e; target=clean; $(RECURSIVE_BUILD_CMD)
> +   @set -e; target=clean; $(RECURSIVE_MAKE)
> rm -f $(LIBS)
> rm -f openssl.pc libssl.pc libcrypto.pc
> rm -f speed.* .pure
> rm -f $(TARFILE)
> +   rm -rf *.bak include/openssl certs/.0
> @set -e; for i in $(ONEDIRS) ;\
> do \
> rm -fr $$i/*; \
> @@ -519,12 +520,12 @@ makefile.one: files
>
>  files:
> $(PERL) $(TOP)/util/files.pl Makefile > $(TOP)/MINFO
> -   @set -e; target=files; $(RECURSIVE_BUILD_CMD)
> +   @set -e; target=files; $(

Re: [openssl.org #3497] Move dclean actions to clean

[Mike Bland]

Just generated a pull request for this; let me know if it's what you
actually had in mind:

https://github.com/openssl/openssl/pull/161

Mike

On Thu, Aug 21, 2014 at 12:08 PM, Rich Salz via RT  wrote:
> Doing "make clean" should remove all build artifacts, while make dclean should
> do clean and then anything special to get back to a distro-like directory. 
> This
> ticket is to capture moving any special dclean actions to the clean target.
>
> --
> Rich Salz, OpenSSL dev team; rs...@openssl.org
>
> __
> OpenSSL Project http://www.openssl.org
> Development Mailing List   openssl-dev@openssl.org
> Automated List Manager   majord...@openssl.org
__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org

Re: Single-Makefile Build Experiment report

[Mike Bland]

Just issued pull request #160:

https://github.com/openssl/openssl/pull/160

Will update the thread with the RT issue number when it comes through.

Mike
__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org

Re: [openssl.org #3447] AutoReply: Build environment updates

[Mike Bland via RT]

Ping... Would appreciate getting some of these changes pulled. Ready
to answer any questions, address any issues.

Thanks,

Mike


On Wed, Jul 9, 2014 at 3:27 PM, The default queue via RT  
wrote:
>
> Greetings,
>
> This message has been automatically generated in response to the
> creation of a trouble ticket regarding:
> "Build environment updates",
> a summary of which appears below.
>
> There is no need to reply to this message right now.  Your ticket has been
> assigned an ID of [openssl.org #3447].
>
> Please include the string:
>
>  [openssl.org #3447]
>
> in the subject line of all future correspondence about this issue. To do so,
> you may reply to this message.
>
> Thank you,
> r...@openssl.org
>
> -
> https://github.com/openssl/openssl/pull/145
>
> This pull contains commits to fix the OS X build and allow GitMake test to 
> pass.
>


__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org

Re: Single-Makefile Build Experiment report

[Mike Bland]

Ah, ccache...all those years at the old company rotted so much of my memory. :-P

Still, it does look like the single-Makefile results are a win.

Mike


On Fri, Aug 15, 2014 at 1:44 PM, Nathan Typanski  wrote:
> I forgot the only important timing command in the above sequence: the
> actual build step. But, yes, I use ccache and it does ridiculous
> things to build times. What looks like `gcc` from my end is just
> copying cached builds out of RAM.
>
> Nathan
> __
> OpenSSL Project http://www.openssl.org
> Development Mailing List   openssl-dev@openssl.org
> Automated List Manager   majord...@openssl.org
__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org

Re: Single-Makefile Build Experiment report

[Mike Bland]

Nathan and Tim,

Thanks much for helping refocus here. Responses inline.

On Fri, Aug 15, 2014 at 10:29 AM, Nathan Typanski  wrote:
> Mike,
>
> Sorry for contributing to the off-topic discussion. I'll try to make
> up for it by posting some interesting data.

No worries; I've certainly learned a lot from these tangents! Just
wanted to make sure the original signal didn't get lost in the midst.
:-)

> I wanted to know if your changes improved speed on my system, and ran
> my own suite of benchmarks. I didn't actually bench incremental
> builds, since I wasted all my time benching the full ones, but I put
> together some charts and posted all the data along with them for
> inspection.
>
> 
>
> If I'm correct in saying so, I think the right way to run the current
> test suite is to do
>
> $ make clean && make && make test
>
> which means when I benchmarked `make test` with the current `master`
> tree, I ran /usr/bin/time on both the `make` and `make test` commands,
> appended both of those a file, and summed their output. If I was wrong
> in how I did that, then the `make test` bench is wrong and you didn't
> shave 3-4 seconds off the full build like I thought ;)
>
> My system software/specs:
>
> - Arch Linux w/ 3.16.0 kernel
> - i7-2620M CPU. During the builds my clock speed averaged 3.2 GHz.
> - 8GiB RAM
> - Crucial MX100 SSD
> - GCC compiler

Thanks much for doing this! But I'm really surprised that you're
getting 16s full, nonparallel builds from the existing recursive make
structure, when my Mac Pro still clocks 65s. What am I missing here?

Also, I timed 'make clean && make' and 'make test/test_heartbeat' (and
not 'make test') separately on purpose; I'll explain further below.

> Also, I read your article. The bits about `ssl/d1_both.c` not being
> detected in the recursive make are scary. Ultimately it would probably
> end up wasting time more than anything - if the person who merges the
> code does a full `make clean` and runs the test suite before pulling
> anything, they should catch that. But without a CI server, of course,
> that's more time wasted trying to build broken code ...

All true. On the one hand, it seems that the single-Makefile
experiment is in pursuit of an ideal; but at the same time, I believe
this experiment demonstrates that the ideal is well within practical
reach. Every little bit of friction and surprising behavior we can
remove from the build process improves productivity by improving
efficiency and ensuring correctness--and it makes for a *much*
improved testing experience.

Speaking of which, that's why my final examples focused not on 'make
test', which runs the entire test suite, but 'make
test/test_heartbeat'. heartbeat_test.c, aside from being the test I
wrote, is currently the only example in the code of the type of unit
test I'm hoping to encourage: small, focused, and fast. It tests
specific functions/one specific feature, and runs quickly enough that
it maintains the flow of concentration while changing the code under
test, whereas most of the other tests drive end-to-end runs through
the command line interface. Those are indeed critical, but as proven,
don't catch everything; hence the push to improve unit test coverage.

The build time for this single test should prove representative of the
edit-compile-build-test cycle that I hope OpenSSL contributors will
begin to experience and adopt as a regular development habit. That
said, we can also work on improving the isolation of the rest of the
test suite so the full 'make test' will also benefit from maximum
parallelism. Right now, many tests expect to be run one-at-a-time
inside the test/ directory, which means that given enough parallelism,
the tests in their current state step on one another. (Yes, I've hit
this. :-)

On Fri, Aug 15, 2014 at 11:35 AM, Tim Hollebeek
 wrote:
> Ask and ye shall receive:
>
> 1.   You are 100% correct that recursive make is completely broken, and
> moving to a single makefile is a significant improvement even if something
> else is done in the medium/long term.

That's what I'm shooting for: Before adopting a new system wholesale,
if we ever do, it'd be nice to improve the existing system such that
we have a reliable benchmark to compare it against. My particular
concern being the "if": There's no clear guarantee at this point that
anything else will be adopted, so it'd be nice to cleanup the system
we've already got, if possible; and I believe the experiment
demonstrates it's quite possible.

> 2.   If using GMake everywhere is practical, I think it’s a good idea.
> I’ve worked on open source projects before that tried to support multiple
> makefile systems, and the results were so complicated as to be
> unmaintainable.  Portability of compilation is the responsibility of the
> build system, not the project build built.  This is essentially the same
> philosophy as CMake.  The code and even Makefiles shouldn’t care what
> platform they are on unless t

RE: Single-Makefile Build Experiment report

[Mike Bland]

I appreciate and may take you up on the offer, but it's still off-topic.
;-) I'd also be more inclined to accept after some feedback on my own
offering.

Mike
 On Aug 15, 2014 9:53 AM, "Tim Hollebeek"  wrote:

> Mike, if you like, I can try to find some time next week for a phone call
> to answer questions and discuss our experience using CMake.  I'm by no
> means an expert, but we've used it internally on a project and have come to
> believe it is completely awesome.  Most open source projects are moving
> towards CMake-based build systems, and I see no reason why OpenSSL can't
> join that bandwagon.
>
> -Tim
>
> -Original Message-
> From: owner-openssl-...@openssl.org [mailto:owner-openssl-...@openssl.org]
> On Behalf Of Mike Bland
> Sent: Thursday, August 14, 2014 5:35 PM
> To: openssl-dev@openssl.org
> Subject: Re: Single-Makefile Build Experiment report
>
> On Thu, Aug 14, 2014 at 4:32 PM, Tim Hollebeek 
> wrote:
> > Have you considered moving to CMake?  It makes lots of the issues you
> discuss in the document just go away.  cmake should work on the vast
> majority of supported operating systems, if not all of them ...
>
> Nope; wasn't aware of CMake before to be honest. That's not to say I'd
> dismiss it out of hand, but I was constraining myself to maintaining
> compatibility with the existing toolchain, meaning Configure+{BSD,GNU}
> make. I got a long way maintaining compatibility between BSD and GNU, but
> ditched BSD near the end because of the absence of pattern rules.
> (Then again, do we really need to build sources in different directories
> with different CFLAGS, etc., or is that just an artifact of organic growth?)
>
> I will say that Geoff Thorpe and I discussed our ideas at length over the
> phone a few weeks back, and in a nutshell, what I've done here could serve
> as prologue to the complete overhaul of the build system that he has in
> mind. Some of what he described, if I remember correctly, sounds similar to
> what little I've just looked up about the CMake, but I don't recall him
> referring to it specifically. (He's pinged me to let me know he'll be
> available to comment on my report next week.)
>
> Mike
> __
> OpenSSL Project http://www.openssl.org
> Development Mailing List   openssl-dev@openssl.org
> Automated List Manager   majord...@openssl.org
>
> 
>
> This transmission may contain information that is privileged,
> confidential, and/or exempt from disclosure under applicable law. If you
> are not the intended recipient, you are hereby notified that any
> disclosure, copying, distribution, or use of the information contained
> herein (including any reliance thereon) is strictly prohibited. If you
> received this transmission in error, please immediately contact the sender
> and destroy the material in its entirety, whether in electronic or hard
> copy format.
>

Re: Single-Makefile Build Experiment report

[Mike Bland]

If I may redirect the discussion here, interesting as it is... I've
got a refactoring of the build system in-hand, compatible with tools
already in use. As much as folks may be in support of adopting a new
build system entirely--which I agree, might be worthwhile--I'd like
feedback on the work I've already done, not the work we might do one
day with some completely different system.

Unless, that is, OpenSSL is ready to make the switch to a new build
system *right now*, in which case there'd be no point to moving
forward with adopting my changes. Somehow I don't see consensus on the
adoption of a new system happening anytime soon. In the interim I'd
appreciate it if the improvements I've made didn't stall on
interminable discussion of unmaterialized alternatives.

Thanks,

Mike


On Thu, Aug 14, 2014 at 10:04 PM, Tom Francis  wrote:
>
> On Aug 14, 2014, at 9:20 PM, Salz, Rich  wrote:
>
>>> Just a comment. the OpenSSL build already depends on Perl and Perl already
>>> has a "Make" of it's own .
>>
>> Ooh, that could be interesting.  What's the perl make thing called?  A web 
>> search for "perl make" was too voluminous…
>
> AFAIK, there’s just ExtUtils::MakeMaker, which is used to generate Makefiles 
> from a script, and is heavily geared towards creating Makefiles to compile 
> and install Perl modules.
>
>
>>   /r$
>>
>> --
>> Principal Security Engineer
>> Akamai Technologies, Cambridge MA
>> IM: rs...@jabber.me Twitter: RichSalz
>>
>> __
>> OpenSSL Project http://www.openssl.org
>> Development Mailing List   openssl-dev@openssl.org
>> Automated List Manager   majord...@openssl.org
>>
> __
> OpenSSL Project http://www.openssl.org
> Development Mailing List   openssl-dev@openssl.org
> Automated List Manager   majord...@openssl.org
__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org

Re: Single-Makefile Build Experiment report

[Mike Bland]

On Thu, Aug 14, 2014 at 4:32 PM, Tim Hollebeek  wrote:
> Have you considered moving to CMake?  It makes lots of the issues you discuss 
> in the document just go away.  cmake should work on the vast majority of 
> supported operating systems, if not all of them ...

Nope; wasn't aware of CMake before to be honest. That's not to say I'd
dismiss it out of hand, but I was constraining myself to maintaining
compatibility with the existing toolchain, meaning Configure+{BSD,GNU}
make. I got a long way maintaining compatibility between BSD and GNU,
but ditched BSD near the end because of the absence of pattern rules.
(Then again, do we really need to build sources in different
directories with different CFLAGS, etc., or is that just an artifact
of organic growth?)

I will say that Geoff Thorpe and I discussed our ideas at length over
the phone a few weeks back, and in a nutshell, what I've done here
could serve as prologue to the complete overhaul of the build system
that he has in mind. Some of what he described, if I remember
correctly, sounds similar to what little I've just looked up about the
CMake, but I don't recall him referring to it specifically. (He's
pinged me to let me know he'll be available to comment on my report
next week.)

Mike
__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org

Single-Makefile Build Experiment report

[Mike Bland]

As announced on the openssl-testing list, I'm happy to report early,
promising success in my Makefile refactoring experiment. Here's the
short link to the Google Doc containing all of the details:

http://goo.gl/yhvCno

Feedback welcome. Regardless of the ultimate judgment of the
experiment, I'm hoping to build upon this report and produce a
published article.

Questions:

- Do these results provide acceptable justification for adopting this
structure? (With the understanding that I may have to redo bits, of
course.)

- Provided the answer to the first question is "yes", should I do
whatever necessary to support BSD make, or will establishing GNU make
as a build requirement be acceptable?

Thanks,

Mike
__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org

Re: [openssl.org #3446] test/testutil.h test registry macros

[Mike Bland via RT]

Thanks, Matt!

Mike

On Sun, Jul 13, 2014 at 6:52 PM, Matt Caswell via RT  wrote:
> Hi Mike
>
> I'm looking at this. I'll get back to you once I've reviewed.
>
> Matt
>


__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org

Re: [openssl.org #3446] test/testutil.h test registry macros

[Mike Bland]

Thanks, Matt!

Mike

On Sun, Jul 13, 2014 at 6:52 PM, Matt Caswell via RT  wrote:
> Hi Mike
>
> I'm looking at this. I'll get back to you once I've reviewed.
>
> Matt
>
__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org

[openssl.org #3447] Build environment updates

[Mike Bland via RT]

https://github.com/openssl/openssl/pull/145

This pull contains commits to fix the OS X build and allow GitMake test to pass.

__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org

[openssl.org #3446] test/testutil.h test registry macros

[Mike Bland via RT]

https://github.com/openssl/openssl/pull/144

These macros help standardize the structure of main() and result
reporting, providing confirmation that all tests have run, even when
they pass. This pull also contains the change to apply these macros to
ssl/heartbeat_test.c.

__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org

Preferred method: email patches or pull requests?

[Mike Bland]

I've got a pile of small test/build system commits pending in the
following pull requests:

test/testutil.h test registry macros
https://github.com/openssl/openssl/pull/144

Build environment updates
https://github.com/openssl/openssl/pull/145

Should I trickle them into openssl-dev a patch-at-a-time instead?

Thanks,

Mike
__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org

Re: Unit Testing/statically analysing OpenSSL

[Mike Bland]

I'm (slowly) helping on the unit testing front. Check out
http://wiki.openssl.org/index.php/Unit_Testing and
https://groups.google.com/forum/#!forum/openssl-testing for more info.

Currently I'm working on trying to refactor bits of the build system,
which I hope will make it easier to perform unit testing in the long
run.

Mike


On Wed, Jul 9, 2014 at 9:38 AM, Paul Morriss
 wrote:
> I am keen to get more involved in the development of OpenSSL, I am curious,
> has the code been run through a static analysis tool (such as Coverity)?
>
> There are self checks, are there unit tests (e.g. Google Test/Mock)created
> for any part of OpenSSL?
>
> Paul
> ++
> Please give generously to Great Ormond Street Hospital, London, UK
> A specialist hospital for sick children who need every penny!
>
> Great Ormond Street Hospital Children's Charity (GOSHCC) website:
> http://www.gosh.org/
> ++
>
> __
> OpenSSL Project http://www.openssl.org
> Development Mailing List   openssl-dev@openssl.org
> Automated List Manager   majord...@openssl.org
__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org

Re: Makedepend bug?

[Mike Bland]

Yeah, the portability angle is why I'm trying to move forward
carefully. That said, isn't GNU Make everywhere these days? Couldn't
we eliminate a lot of complexity by relying on its include syntax (and
other treats)? I'm still a n00b on this scene, so I don't aim to
offend anyone, but it's an honest, if naive question.

Also, I've been piling up a few build-related commits in
https://github.com/openssl/openssl/pull/145, including "Add .d files
to 'make clean' targets" and "Configure dependency-generating compiler
flags". (I won't link to the commit hashes because I keep rebasing
that branch on openssl/master.) I'd be interested in hearing what
folks think of them.

Mike

On Tue, Jul 1, 2014 at 4:57 PM, Theodore Ts'o  wrote:
> On Tue, Jul 01, 2014 at 02:10:31PM -0400, Mike Bland wrote:
>> I was wondering why 'make depend' output was saved in the Makefiles.
>> So I guess adding the .d files to the repository and using include
>> statements in the Makefiles is a reasonable possibility? (That's the
>> angle I'm taking with my experiment, though I hadn't thought to add
>> the .d's to the repo.)
>
> The problem with using include statements in Makefiles is that's not
> particularly portable.  So what I do in e2fsprogs is to modify the
> Makefile.in files directly (since I use autoconf, but NOT automake or
> libtool):
>
> .depend: Makefile $(SRCS) $(top_srcdir)/depfix.sed $(top_srcdir)/wordwrap.pl
> if test -n "$(SRCS)" ; then \
> $(CC) -M $(ALL_CFLAGS) $(DEPEND_CFLAGS) $(SRCS) | \
> $(SED) -f $(top_srcdir)/depfix.sed \
> -e 's; $(srcdir)/; $$(srcdir)/;g' \
> -e 's; $(top_srcdir)/; $$(top_srcdir)/;g' \
> -e 's; $(top_builddir)/; $$(top_builddir)/;g' \
> -e 's; \./; ;g' \
> -e '/^#/d' \
> -e '/^ *\\$$/d' | \
> $(PERL) $(top_srcdir)/wordwrap.pl > .depend; \
> else :; fi
>
> depend:: .depend
> if test -n "$(SRCS)" ; then \
> sed -e '/^# +++ Dependency line eater +++/,$$d' \
> < $(srcdir)/Makefile.in | cat - .depend \
> > $(srcdir)/Makefile.in.new; \
> if cmp -s $(srcdir)/Makefile.in $(srcdir)/Makefile.in.new ; then \
> $(RM) $(srcdir)/Makefile.in.new ; \
> else \
> $(MV) $(srcdir)/Makefile.in $(srcdir)/Makefile.in.old; \
> $(MV) $(srcdir)/Makefile.in.new $(srcdir)/Makefile.in; \
> fi ; else :; fi
>
> ... and where depfix.sed is a sed script that strips out the system
> header files, so that the resulting dependencies are portable across
> different OS's.  So this matches OpenSSL's requirements, I think.  Gcc
> is required to update the dependencies, but it's not required for
> people who are building on systems that have old-style makes,
> including ones that go as far back as Solaris 2.x, AIX, etc.  (This is
> a modified and updated version of something that I had originally
> created for MIT Kerberos, which had a very wide portability
> requirement, including creating NMAKE compatible files for a very old
> version of Visual Studio, as well as makefiles that would work with
> Apple's MPW.  :-)
>
> - Ted
> __
> OpenSSL Project http://www.openssl.org
> Development Mailing List   openssl-dev@openssl.org
> Automated List Manager   majord...@openssl.org
__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org

Re: Makedepend bug?

[Mike Bland]

I was wondering why 'make depend' output was saved in the Makefiles.
So I guess adding the .d files to the repository and using include
statements in the Makefiles is a reasonable possibility? (That's the
angle I'm taking with my experiment, though I hadn't thought to add
the .d's to the repo.)

Mike


On Tue, Jul 1, 2014 at 2:01 PM, Salz, Rich  wrote:
>> So now gcc/clang is required to build OpenSSL?
>
> No, nobody's said that.  The phrase was "perhaps"   And if openssl ships with 
> a default set of dependencies, which it does, there's no issue about which 
> compiler you use at all. Once we fix the "make depend" requirement.
>
> --
> Principal Security Engineer
> Akamai Technologies, Cambridge, MA
> IM: rs...@jabber.me; Twitter: RichSalz
> __
> OpenSSL Project http://www.openssl.org
> Development Mailing List   openssl-dev@openssl.org
> Automated List Manager   majord...@openssl.org
__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org

Re: Makedepend bug?

[Mike Bland]

I am 100% in support of that notion. That'd make my Makefile
restructuring experiment much more streamlined. That, and requiring
GNU make instead of supporting both GNU make and bsdmake syntax, from
the point of view of using included sub-Makefiles. (Says me talking
the FreeBSD 9.1 user. ;-)

Mike


On Tue, Jul 1, 2014 at 12:42 PM, Ben Laurie  wrote:
> On 1 July 2014 17:21, Mike Bland  wrote:
>> Ah! Sorry for the spam, but I think I got it. According to the
>> makedepend man page:
>>
>> http://www.x.org/archive/current/doc/man/man1/makedepend.1.xhtml
>>
>> Makedepend makes assumptions about the #includes for files appearing
>> later on the command line:
>>
>> "But when the program parses file2.c and discovers that it, too,
>> includes header.h, it does not parse the file, but simply adds
>> header.h, def1.h anddef2.h to the list of dependencies for file2.o."
>>
>> Long story short, I narrowed it down to apps/req.c and apps/gendh.c
>> both #ifdef/undeffing OPENSSL_NO_DEPRECATED. Execute the command with
>> those two files removed, and dh.h disappears from dsa.o's deps.
>>
>> dsaparam.c, genrsa.c, and s_server.c also have this #undef, but coming
>> later on the command line than dsa.c, they don't trigger makedepend to
>> generate the dh.h include for dsa.o. Put them before dsa.c, and they
>> do.
>
> Aha! Well done.
>
> I suspect there's not really any reason to support makedepend anymore
> - should perhaps just switch to always using gcc/clang for
> dependencies?
>
>>
>> Mike
>>
>>
>> On Tue, Jul 1, 2014 at 11:59 AM, Mike Bland  wrote:
>>> Whoops, of course, I meant it generates the same output for dsa.o, and
>>> only dsa.o.
>>>
>>> Mike
>>>
>>> On Tue, Jul 1, 2014 at 11:58 AM, Mike Bland  wrote:
>>>> Investigating... It seems to be an issue with the makedepend tool itself.
>>>>
>>>> I hacked util/domd to show the makedepend command line, and got this
>>>> command for apps/:
>>>>
>>>> makedepend -D OPENSSL_DOING_MAKEDEPEND -- -O -I..  -I../include
>>>> -DOPENSSL_NO_DEPRECATED -DOPENSSL_NO_EC_NISTP_64_GCC_128
>>>> -DOPENSSL_NO_GMP -DOPENSSL_NO_JPAKE -DOPENSSL_NO_MD2 -DOPENSSL_NO_RC5
>>>> -DOPENSSL_NO_RFC3779 -DOPENSSL_NO_SCTP -DOPENSSL_NO_SSL_TRACE
>>>> -DOPENSSL_NO_STORE -- openssl.c verify.c asn1pars.c req.c dgst.c dh.c
>>>> enc.c passwd.c gendh.c errstr.c ca.c pkcs7.c crl2p7.c crl.c rsa.c
>>>> rsautl.c dsa.c dsaparam.c ec.c ecparam.c x509.c genrsa.c gendsa.c
>>>> genpkey.c s_server.c s_client.c speed.c s_time.c apps.c s_cb.c
>>>> s_socket.c app_rand.c version.c sess_id.c ciphers.c nseq.c pkcs12.c
>>>> pkcs8.c pkey.c pkeyparam.c pkeyutl.c spkac.c smime.c cms.c rand.c
>>>> engine.c ocsp.c prime.c ts.c srp.c
>>>>
>>>> Running that through util/clean-depend.pl produces the existing
>>>> makedepend output. But just running this:
>>>>
>>>> makedepend -D OPENSSL_DOING_MAKEDEPEND -- -O -I..  -I../include
>>>> -DOPENSSL_NO_DEPRECATED -DOPENSSL_NO_EC_NISTP_64_GCC_128
>>>> -DOPENSSL_NO_GMP -DOPENSSL_NO_JPAKE -DOPENSSL_NO_MD2 -DOPENSSL_NO_RC5
>>>> -DOPENSSL_NO_RFC3779 -DOPENSSL_NO_SCTP -DOPENSSL_NO_SSL_TRACE
>>>> -DOPENSSL_NO_STORE -- dsa.c
>>>>
>>>> Produces the same output except *without* ../include/openssl/dh.h. (I
>>>> presume you meant dh.h, not dh.o earlier.)
>>>>
>>>> Of course, with your GitConfigure/GitMake scripts, you're using the
>>>> compiler to regenerate dsa.d in isolation from other files.
>>>>
>>>> I'll see if I can dig up a little more info here...but it does seem
>>>> that your normal flags are doing the right thing.
>>>>
>>>> Mike
>>>>
>>>>
>>>>
>>>> On Tue, Jul 1, 2014 at 5:38 AM, Ben Laurie  wrote:
>>>>> I've been trying to figure out why my "make depend" differs from other
>>>>> developers, and why it appears to be wrong.
>>>>>
>>>>> For example, apps/dsa.o depends, according to makedepend, on dh.o, but
>>>>> with the standard developer flags ($gcc_devteam_warn) it should not.
>>>>>
>>>>> AFAICS, makedepend gets passed the right flags, and normally processes
>>>>> #ifndef lines correctly.
>>>>>
>>>>> Anyone got any clues what's going on

Re: Makedepend bug?

[Mike Bland]

Ah! Sorry for the spam, but I think I got it. According to the
makedepend man page:

http://www.x.org/archive/current/doc/man/man1/makedepend.1.xhtml

Makedepend makes assumptions about the #includes for files appearing
later on the command line:

"But when the program parses file2.c and discovers that it, too,
includes header.h, it does not parse the file, but simply adds
header.h, def1.h anddef2.h to the list of dependencies for file2.o."

Long story short, I narrowed it down to apps/req.c and apps/gendh.c
both #ifdef/undeffing OPENSSL_NO_DEPRECATED. Execute the command with
those two files removed, and dh.h disappears from dsa.o's deps.

dsaparam.c, genrsa.c, and s_server.c also have this #undef, but coming
later on the command line than dsa.c, they don't trigger makedepend to
generate the dh.h include for dsa.o. Put them before dsa.c, and they
do.

Mike


On Tue, Jul 1, 2014 at 11:59 AM, Mike Bland  wrote:
> Whoops, of course, I meant it generates the same output for dsa.o, and
> only dsa.o.
>
> Mike
>
> On Tue, Jul 1, 2014 at 11:58 AM, Mike Bland  wrote:
>> Investigating... It seems to be an issue with the makedepend tool itself.
>>
>> I hacked util/domd to show the makedepend command line, and got this
>> command for apps/:
>>
>> makedepend -D OPENSSL_DOING_MAKEDEPEND -- -O -I..  -I../include
>> -DOPENSSL_NO_DEPRECATED -DOPENSSL_NO_EC_NISTP_64_GCC_128
>> -DOPENSSL_NO_GMP -DOPENSSL_NO_JPAKE -DOPENSSL_NO_MD2 -DOPENSSL_NO_RC5
>> -DOPENSSL_NO_RFC3779 -DOPENSSL_NO_SCTP -DOPENSSL_NO_SSL_TRACE
>> -DOPENSSL_NO_STORE -- openssl.c verify.c asn1pars.c req.c dgst.c dh.c
>> enc.c passwd.c gendh.c errstr.c ca.c pkcs7.c crl2p7.c crl.c rsa.c
>> rsautl.c dsa.c dsaparam.c ec.c ecparam.c x509.c genrsa.c gendsa.c
>> genpkey.c s_server.c s_client.c speed.c s_time.c apps.c s_cb.c
>> s_socket.c app_rand.c version.c sess_id.c ciphers.c nseq.c pkcs12.c
>> pkcs8.c pkey.c pkeyparam.c pkeyutl.c spkac.c smime.c cms.c rand.c
>> engine.c ocsp.c prime.c ts.c srp.c
>>
>> Running that through util/clean-depend.pl produces the existing
>> makedepend output. But just running this:
>>
>> makedepend -D OPENSSL_DOING_MAKEDEPEND -- -O -I..  -I../include
>> -DOPENSSL_NO_DEPRECATED -DOPENSSL_NO_EC_NISTP_64_GCC_128
>> -DOPENSSL_NO_GMP -DOPENSSL_NO_JPAKE -DOPENSSL_NO_MD2 -DOPENSSL_NO_RC5
>> -DOPENSSL_NO_RFC3779 -DOPENSSL_NO_SCTP -DOPENSSL_NO_SSL_TRACE
>> -DOPENSSL_NO_STORE -- dsa.c
>>
>> Produces the same output except *without* ../include/openssl/dh.h. (I
>> presume you meant dh.h, not dh.o earlier.)
>>
>> Of course, with your GitConfigure/GitMake scripts, you're using the
>> compiler to regenerate dsa.d in isolation from other files.
>>
>> I'll see if I can dig up a little more info here...but it does seem
>> that your normal flags are doing the right thing.
>>
>> Mike
>>
>>
>>
>> On Tue, Jul 1, 2014 at 5:38 AM, Ben Laurie  wrote:
>>> I've been trying to figure out why my "make depend" differs from other
>>> developers, and why it appears to be wrong.
>>>
>>> For example, apps/dsa.o depends, according to makedepend, on dh.o, but
>>> with the standard developer flags ($gcc_devteam_warn) it should not.
>>>
>>> AFAICS, makedepend gets passed the right flags, and normally processes
>>> #ifndef lines correctly.
>>>
>>> Anyone got any clues what's going on? (I'm on FreeBSD 9.1)
>>> __
>>> OpenSSL Project http://www.openssl.org
>>> Development Mailing List   openssl-dev@openssl.org
>>> Automated List Manager   majord...@openssl.org
__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org

Re: Makedepend bug?

[Mike Bland]

Whoops, of course, I meant it generates the same output for dsa.o, and
only dsa.o.

Mike

On Tue, Jul 1, 2014 at 11:58 AM, Mike Bland  wrote:
> Investigating... It seems to be an issue with the makedepend tool itself.
>
> I hacked util/domd to show the makedepend command line, and got this
> command for apps/:
>
> makedepend -D OPENSSL_DOING_MAKEDEPEND -- -O -I..  -I../include
> -DOPENSSL_NO_DEPRECATED -DOPENSSL_NO_EC_NISTP_64_GCC_128
> -DOPENSSL_NO_GMP -DOPENSSL_NO_JPAKE -DOPENSSL_NO_MD2 -DOPENSSL_NO_RC5
> -DOPENSSL_NO_RFC3779 -DOPENSSL_NO_SCTP -DOPENSSL_NO_SSL_TRACE
> -DOPENSSL_NO_STORE -- openssl.c verify.c asn1pars.c req.c dgst.c dh.c
> enc.c passwd.c gendh.c errstr.c ca.c pkcs7.c crl2p7.c crl.c rsa.c
> rsautl.c dsa.c dsaparam.c ec.c ecparam.c x509.c genrsa.c gendsa.c
> genpkey.c s_server.c s_client.c speed.c s_time.c apps.c s_cb.c
> s_socket.c app_rand.c version.c sess_id.c ciphers.c nseq.c pkcs12.c
> pkcs8.c pkey.c pkeyparam.c pkeyutl.c spkac.c smime.c cms.c rand.c
> engine.c ocsp.c prime.c ts.c srp.c
>
> Running that through util/clean-depend.pl produces the existing
> makedepend output. But just running this:
>
> makedepend -D OPENSSL_DOING_MAKEDEPEND -- -O -I..  -I../include
> -DOPENSSL_NO_DEPRECATED -DOPENSSL_NO_EC_NISTP_64_GCC_128
> -DOPENSSL_NO_GMP -DOPENSSL_NO_JPAKE -DOPENSSL_NO_MD2 -DOPENSSL_NO_RC5
> -DOPENSSL_NO_RFC3779 -DOPENSSL_NO_SCTP -DOPENSSL_NO_SSL_TRACE
> -DOPENSSL_NO_STORE -- dsa.c
>
> Produces the same output except *without* ../include/openssl/dh.h. (I
> presume you meant dh.h, not dh.o earlier.)
>
> Of course, with your GitConfigure/GitMake scripts, you're using the
> compiler to regenerate dsa.d in isolation from other files.
>
> I'll see if I can dig up a little more info here...but it does seem
> that your normal flags are doing the right thing.
>
> Mike
>
>
>
> On Tue, Jul 1, 2014 at 5:38 AM, Ben Laurie  wrote:
>> I've been trying to figure out why my "make depend" differs from other
>> developers, and why it appears to be wrong.
>>
>> For example, apps/dsa.o depends, according to makedepend, on dh.o, but
>> with the standard developer flags ($gcc_devteam_warn) it should not.
>>
>> AFAICS, makedepend gets passed the right flags, and normally processes
>> #ifndef lines correctly.
>>
>> Anyone got any clues what's going on? (I'm on FreeBSD 9.1)
>> __
>> OpenSSL Project http://www.openssl.org
>> Development Mailing List   openssl-dev@openssl.org
>> Automated List Manager   majord...@openssl.org
__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org

Re: Makedepend bug?

[Mike Bland]

Investigating... It seems to be an issue with the makedepend tool itself.

I hacked util/domd to show the makedepend command line, and got this
command for apps/:

makedepend -D OPENSSL_DOING_MAKEDEPEND -- -O -I..  -I../include
-DOPENSSL_NO_DEPRECATED -DOPENSSL_NO_EC_NISTP_64_GCC_128
-DOPENSSL_NO_GMP -DOPENSSL_NO_JPAKE -DOPENSSL_NO_MD2 -DOPENSSL_NO_RC5
-DOPENSSL_NO_RFC3779 -DOPENSSL_NO_SCTP -DOPENSSL_NO_SSL_TRACE
-DOPENSSL_NO_STORE -- openssl.c verify.c asn1pars.c req.c dgst.c dh.c
enc.c passwd.c gendh.c errstr.c ca.c pkcs7.c crl2p7.c crl.c rsa.c
rsautl.c dsa.c dsaparam.c ec.c ecparam.c x509.c genrsa.c gendsa.c
genpkey.c s_server.c s_client.c speed.c s_time.c apps.c s_cb.c
s_socket.c app_rand.c version.c sess_id.c ciphers.c nseq.c pkcs12.c
pkcs8.c pkey.c pkeyparam.c pkeyutl.c spkac.c smime.c cms.c rand.c
engine.c ocsp.c prime.c ts.c srp.c

Running that through util/clean-depend.pl produces the existing
makedepend output. But just running this:

makedepend -D OPENSSL_DOING_MAKEDEPEND -- -O -I..  -I../include
-DOPENSSL_NO_DEPRECATED -DOPENSSL_NO_EC_NISTP_64_GCC_128
-DOPENSSL_NO_GMP -DOPENSSL_NO_JPAKE -DOPENSSL_NO_MD2 -DOPENSSL_NO_RC5
-DOPENSSL_NO_RFC3779 -DOPENSSL_NO_SCTP -DOPENSSL_NO_SSL_TRACE
-DOPENSSL_NO_STORE -- dsa.c

Produces the same output except *without* ../include/openssl/dh.h. (I
presume you meant dh.h, not dh.o earlier.)

Of course, with your GitConfigure/GitMake scripts, you're using the
compiler to regenerate dsa.d in isolation from other files.

I'll see if I can dig up a little more info here...but it does seem
that your normal flags are doing the right thing.

Mike



On Tue, Jul 1, 2014 at 5:38 AM, Ben Laurie  wrote:
> I've been trying to figure out why my "make depend" differs from other
> developers, and why it appears to be wrong.
>
> For example, apps/dsa.o depends, according to makedepend, on dh.o, but
> with the standard developer flags ($gcc_devteam_warn) it should not.
>
> AFAICS, makedepend gets passed the right flags, and normally processes
> #ifndef lines correctly.
>
> Anyone got any clues what's going on? (I'm on FreeBSD 9.1)
> __
> OpenSSL Project http://www.openssl.org
> Development Mailing List   openssl-dev@openssl.org
> Automated List Manager   majord...@openssl.org
__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org

Re: openssl-testing mailing list

[Mike Bland]

Hi Didier,

I appreciate the offer and would like to learn more. Do you have links
to any documentation?

My focus is primarily on getting unit tests in place, and getting
everyone to make a regular habit of writing them. It sounds like the
techniques you're talking about would be at some higher level than
what we're focusing on for now. If you'd like to participate in the
testing effort and learn about the OpenSSL code base, you're certainly
welcome to; perhaps over time we can then see how your tools could fit
into the OpenSSL environment.

Thanks,

Mike


On Sat, Jun 7, 2014 at 2:06 PM,   wrote:
> Hello Mike
>
> I work on / provide custom tools, based on source code analysis, that can
> help in
>
> - generating autotest test suites according to each function signature :
>  random / full combinatory of argument values : to check the function
> robustness
>  wide loops : to check memory consumption
>
> - generating enhanced original source code : free/malloc enhance, array
> bound check
>
> - generate some other source code, as needed
>
> I am quite new in C, not expert at all.
>
> If some good practice in unit tests in C is available, feedback is welcome !
>
> If some special source code generation can make sense, the same !
>
> PS : these techniques have been used on Java, quite valuable.
>  For the openssl group I can provide support on these tools
>
> Thanks
>
> Didier CRUETTE
>
>
>
>
> Le 06.06.2014 19:56, Mike Bland a écrit :
>>
>> I've created the openssl-testing mailing list (via Google Groups) to
>> discuss the OpenSSL unit/automated testing effort, to avoid clogging
>> openssl-dev:
>>
>> https://groups.google.com/d/forum/openssl-testing
>>
>> Membership is open to whoever wishes to join, even if only to lurk.
>> Ideally all testing discussions will eventually move to openssl-dev
>> once we have processes, tools, conventions, etc. in place. I've
>> started documenting the effort here:
>>
>> http://wiki.openssl.org/index.php/Unit_Testing
>> http://wiki.openssl.org/index.php/How_To_Write_Unit_Tests_For_OpenSSL
>>
>> Despite the outstanding issues (e.g. working with internal APIs and
>> private symbols), using my GitHub fork, we can begin adding tests
>> right away:
>>
>> https://github.com/mbland/openssl
>>
>> Once we have resolution on some of these issues, my fork can be
>> updated accordingly, and we can eventually commit the results to the
>> master OpenSSL repository.
>>
>> Thanks to everyone whose interest and support is helping to make this
>> happen,
>>
>> Mike
>> __
>> OpenSSL Project http://www.openssl.org
>> Development Mailing List   openssl-dev@openssl.org
>> Automated List Manager   majord...@openssl.org
>
> __
> OpenSSL Project http://www.openssl.org
> Development Mailing List   openssl-dev@openssl.org
> Automated List Manager   majord...@openssl.org
__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org

Re: [openssl.org #3380] OpenSSL 1.0.1h on SGI IRIX

[Mike Bland]

On Sat, Jun 7, 2014 at 10:00 PM, Jeremy Farrell
 wrote:
> From: Mike Bland [mailto:mbl...@acm.org]
> Sent: Saturday, June 07, 2014 6:36 PM
>>
>> Just created https://github.com/openssl/openssl/pull/126 with what I
>> hope is a workable solution.
>
> 104 +#if __STDC_VERSION__ < 199901L
> 105 +#define testutil_stringify_helper(s) #s
> 106 +#define testutil_stringify(s) testutil_stringify_helper(s)
> 107 +#define TEST_CASE_NAME __FILE__ ":" testutil_stringify(__LINE__)
> 108 +#elif defined(_MSC_VER)
> 109 +#define TEST_CASE_NAME __FUNCTION__
> 110 +#else
> 111 +#define TEST_CASE_NAME __func__
> 112 +#endif /* __STDC_VERSION */
>
> The _MSC_VER block will never be used since MSC is does not claim C99 
> conformance.
>
> #if __STDC_VERSION__ < 199901L
> #if defined(_MSC_VER)
> #define TEST_CASE_NAME __FUNCTION__
> #else
> #define testutil_stringify_helper(s) #s
> #define testutil_stringify(s) testutil_stringify_helper(s)
> #define TEST_CASE_NAME __FILE__ ":" testutil_stringify(__LINE__)
> #endif /* _MSC_VER */
> #else
> #define TEST_CASE_NAME __func__
> #endif /* __STDC_VERSION__ */
>
> or similar should do the trick.

Thanks for that, Jeremy. Done.

Mike
__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org

Re: [openssl.org #3380] OpenSSL 1.0.1h on SGI IRIX

[Mike Bland]

Just created https://github.com/openssl/openssl/pull/126 with what I
hope is a workable solution.

Mike

On Sat, Jun 7, 2014 at 9:55 AM, Mike Bland  wrote:
> On Sat, Jun 7, 2014 at 4:33 AM, Tim Hudson  wrote:
>> On 7/06/2014 4:02 AM, Dr. Stephen Henson wrote:
>>> On Fri, Jun 06, 2014, Mike Bland wrote:
>>>
>>>> __func__ is defined in C99. What version of the SGI C compiler are you
>>>> using? According to the following, as of version 7.4, the -c99 flag
>>>> should enable this to compile:
>>>>
>>>> http://www.sgi.com/products/software/irix/tools/c.html
>>>>
>>> Note that VC++ under Windows doesn't support __func__ either. Well at least
>>> the versions I tested didn't.
>
> Unfortunately I know next to nothing about VC++, but perhaps it also
> supports a -C99 switch?
>
>> Adding in C99 dependencies in the code will run into a lot of non-C99
>> environments which still are being actively used.
>> I think it is time to either decide that C99 is now a requirement (and
>> there are features in C99 that would be nice to be able to use) or to
>> decide that code which uses those features shouldn't go in - i.e. don't
>> use those features so that platforms which don't support C99 are still
>> supportable.
>>
>> Either approach leads to things breaking for at least some users ...
>>
>> In this particular case (the ssl/heartbeat_test.c) the use of __func__
>> really isn't critical and can easily be changed to not be a C99 __func__
>> dependency and pass in a test name in the 9 locations rather than the
>> function name. That would fix the couple of platforms already noted that
>> had issues.
>
> I certainly don't want to argue that unit testing alone is sufficient
> reason to force the use of C99. (Well, were it my project... ;-) That
> said, I imagine there's a lot more than the convenience of __func__ to
> be gained from an upgrade. It is a fifteen-year-old standard; I'm just
> getting my feet wet with the OpenSSL code, but even in my limited
> experience with ssl/heartbeat_test.c, I'm getting a feel for the kind
> of workarounds that an upgrade would render unnecessary.
>
> There's already been a recent thread debating whether older platforms
> should be supported, or how they should be supported. Since I'm a n00b
> here, I don't want to jump into the middle of a debate I don't have
> enough context for, but that's how I see things for now.
>
> In the meanwhile, I'm looking into how to work around the lack of
> support for __func__ on non-C99 platforms.
>
> Mike
__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org

Re: [openssl.org #3380] OpenSSL 1.0.1h on SGI IRIX

[Mike Bland]

On Sat, Jun 7, 2014 at 4:33 AM, Tim Hudson  wrote:
> On 7/06/2014 4:02 AM, Dr. Stephen Henson wrote:
>> On Fri, Jun 06, 2014, Mike Bland wrote:
>>
>>> __func__ is defined in C99. What version of the SGI C compiler are you
>>> using? According to the following, as of version 7.4, the -c99 flag
>>> should enable this to compile:
>>>
>>> http://www.sgi.com/products/software/irix/tools/c.html
>>>
>> Note that VC++ under Windows doesn't support __func__ either. Well at least
>> the versions I tested didn't.

Unfortunately I know next to nothing about VC++, but perhaps it also
supports a -C99 switch?

> Adding in C99 dependencies in the code will run into a lot of non-C99
> environments which still are being actively used.
> I think it is time to either decide that C99 is now a requirement (and
> there are features in C99 that would be nice to be able to use) or to
> decide that code which uses those features shouldn't go in - i.e. don't
> use those features so that platforms which don't support C99 are still
> supportable.
>
> Either approach leads to things breaking for at least some users ...
>
> In this particular case (the ssl/heartbeat_test.c) the use of __func__
> really isn't critical and can easily be changed to not be a C99 __func__
> dependency and pass in a test name in the 9 locations rather than the
> function name. That would fix the couple of platforms already noted that
> had issues.

I certainly don't want to argue that unit testing alone is sufficient
reason to force the use of C99. (Well, were it my project... ;-) That
said, I imagine there's a lot more than the convenience of __func__ to
be gained from an upgrade. It is a fifteen-year-old standard; I'm just
getting my feet wet with the OpenSSL code, but even in my limited
experience with ssl/heartbeat_test.c, I'm getting a feel for the kind
of workarounds that an upgrade would render unnecessary.

There's already been a recent thread debating whether older platforms
should be supported, or how they should be supported. Since I'm a n00b
here, I don't want to jump into the middle of a debate I don't have
enough context for, but that's how I see things for now.

In the meanwhile, I'm looking into how to work around the lack of
support for __func__ on non-C99 platforms.

Mike
__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org

Re: [openssl.org #3378] heartbeat_test: Using internal APIs

[Mike Bland]

Before this goes in, I'm going to take this opportunity to raise a
question that I've documented on the wiki (which came up in a
discussion off-list):

http://wiki.openssl.org/index.php/Unit_Testing#How_to_Manage_Private_Symbols

Why do any of the symbols need to be private? Is that degree of
encapsulation necessary, and does it really discourage irresponsible
clients? The source code is open, so people can always build their own
copy and depend on internals if they really want to, and the default
UNIX/Linux/OS X build (i.e. ./configure && make && make test) doesn't
lock down internal symbols. Have dependencies on internal APIs ever
been a problem in practice?

I'm not saying I feel strongly either way, but it seems that clients
should be clear that the public API is what's available in
openssl/include, and shouldn't depend on anything else. I just don't
understand what making symbols private buys anyone in the context of
OpenSSL, in light of how it appears to complicate the ability to unit
test.

Not trying to be argumentative, I just want to understand the
rationale. I'll work with whatever environment I have to.

Mike


On Sat, Jun 7, 2014 at 7:38 AM, Kurt Roeckx via RT  wrote:
> On Thu, Jun 05, 2014 at 11:59:30PM +0200, Matt Caswell via RT wrote:
>> On Thu Jun 05 23:42:31 2014, k...@roeckx.be wrote:
>> >
>> > > We are likely to see
>> > > a lot more like this as Mike's test team get going. In unit testing
>> > its "okay"
>> > > to access internal symbols.
>> >
>> > But then you shouldn't link to the shared library. The static
>> > library probably works.
>>
>> Any chance you can knock together a patch to do this and test it with your
>> script?
>
> Github pull request #125.
>
>
> Kurt
>
>
> __
> OpenSSL Project http://www.openssl.org
> Development Mailing List   openssl-dev@openssl.org
> Automated List Manager   majord...@openssl.org
__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org

openssl-testing mailing list

[Mike Bland]

I've created the openssl-testing mailing list (via Google Groups) to
discuss the OpenSSL unit/automated testing effort, to avoid clogging
openssl-dev:

https://groups.google.com/d/forum/openssl-testing

Membership is open to whoever wishes to join, even if only to lurk.
Ideally all testing discussions will eventually move to openssl-dev
once we have processes, tools, conventions, etc. in place. I've
started documenting the effort here:

http://wiki.openssl.org/index.php/Unit_Testing
http://wiki.openssl.org/index.php/How_To_Write_Unit_Tests_For_OpenSSL

Despite the outstanding issues (e.g. working with internal APIs and
private symbols), using my GitHub fork, we can begin adding tests
right away:

https://github.com/mbland/openssl

Once we have resolution on some of these issues, my fork can be
updated accordingly, and we can eventually commit the results to the
master OpenSSL repository.

Thanks to everyone whose interest and support is helping to make this happen,

Mike
__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org

Re: [openssl.org #3380] OpenSSL 1.0.1h on SGI IRIX

[Mike Bland]

__func__ is defined in C99. What version of the SGI C compiler are you
using? According to the following, as of version 7.4, the -c99 flag
should enable this to compile:

http://www.sgi.com/products/software/irix/tools/c.html

Mike


On Fri, Jun 6, 2014 at 3:14 AM, Pieter Bowman via RT  wrote:
> The following shows up using SGI IRIX cc:
>
> cc -I.. -I../include  -DOPENSSL_THREADS -D_SGI_MP_SOURCE -DDSO_DLFCN 
> -DHAVE_DLFCN_H -DOPENSSL_USE_IPV6=0 -n32 -mips3 -O2 -use_readonly_const -G0 
> -rdata_shared -DTERMIOS -DB_ENDIAN -DBN_DIV3W   -c -o heartbeat_test.o 
> heartbeat_test.c
> cc-1020 cc: ERROR File = heartbeat_test.c, Line = 276
>   The identifier "__func__" is undefined.
>
> SETUP_HEARTBEAT_TEST_FIXTURE(dtls);
> ^
>
> cc-1020 cc: ERROR File = heartbeat_test.c, Line = 294
>   The identifier "__func__" is undefined.
>
> SETUP_HEARTBEAT_TEST_FIXTURE(dtls);
> ^
>
> cc-1020 cc: ERROR File = heartbeat_test.c, Line = 312
>   The identifier "__func__" is undefined.
>
> SETUP_HEARTBEAT_TEST_FIXTURE(dtls);
> ^
>
> cc-1020 cc: ERROR File = heartbeat_test.c, Line = 326
>   The identifier "__func__" is undefined.
>
> SETUP_HEARTBEAT_TEST_FIXTURE(dtls);
> ^
>
> cc-1020 cc: ERROR File = heartbeat_test.c, Line = 343
>   The identifier "__func__" is undefined.
>
> SETUP_HEARTBEAT_TEST_FIXTURE(dtls);
> ^
>
> cc-1020 cc: ERROR File = heartbeat_test.c, Line = 360
>   The identifier "__func__" is undefined.
>
> SETUP_HEARTBEAT_TEST_FIXTURE(tls);
> ^
>
> cc-1020 cc: ERROR File = heartbeat_test.c, Line = 378
>   The identifier "__func__" is undefined.
>
> SETUP_HEARTBEAT_TEST_FIXTURE(tls);
> ^
>
> cc-1020 cc: ERROR File = heartbeat_test.c, Line = 396
>   The identifier "__func__" is undefined.
>
> SETUP_HEARTBEAT_TEST_FIXTURE(tls);
> ^
>
> cc-1020 cc: ERROR File = heartbeat_test.c, Line = 410
>   The identifier "__func__" is undefined.
>
> SETUP_HEARTBEAT_TEST_FIXTURE(tls);
> ^
>
> 9 errors detected in the compilation of "heartbeat_test.c".
>
> __
> OpenSSL Project http://www.openssl.org
> Development Mailing List   openssl-dev@openssl.org
> Automated List Manager   majord...@openssl.org
__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org

Re: Improving unit/automated test coverage

[Mike Bland]

Hi Reini,

Actually, I was asking for thoughts on whether to create a separate
openssl-testing mailing list, which I'm leaning towards at the moment,
as I plan to get very chatty with the volunteers helping with unit
testing.

That said, I've limited experience with valgrind. Are you volunteering
to help add some of this valgrind support?

Mike


On Thu, Jun 5, 2014 at 10:04 AM, Reini Urban  wrote:
> On 06/04/2014 04:58 PM, Mike Bland wrote:
>>
>> Thanks to a few brave volunteers and the support of the core OpenSSL
>> team, it looks like we can begin moving on this effort soon. I've
>> begun to document the current state of things on the wiki:
>>
>> http://wiki.openssl.org/index.php/Unit_Testing
>>
>> There's lots to discuss with regard to the "Goals", "Tasks", and
>> "Discussion Topics" enumerated on the wiki. I'm happy to start a
>> discussion on this thread, or other openssl-dev threads; at the same
>> time, I'm wondering if a new openssl-testing group would be more
>> appropriate. If it's desirable, I could manage it via Google Groups,
>> rather than create more majordomo work for the core team. Thoughts?
>
>
> Mike,
>
> The first which would spring to my mind are valgrind memcheck test targets
> and -fsanitize=* rules.
> With the current testsuite it is almost impossible to run it via valgrind
> and there are no sane sanitize rules yet: (address, undefined for -O3,
> thread, memory)
> There's only a static debug-ben-debug-64-clang config.
>
> See for contrast polarssl which include those tests and configs.
> Not speaking about coverage yet.
>
> --
> Reini
>
> Working towards a true Modern Perl.
> Slim, functional, unbloated, compile-time optimizable
>
> __
> OpenSSL Project http://www.openssl.org
> Development Mailing List   openssl-dev@openssl.org
> Automated List Manager   majord...@openssl.org
__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org

Re: [openssl.org #3378] heartbeat_test: Using internal APIs

[Mike Bland]

On Thu, Jun 5, 2014 at 2:41 PM, Kurt Roeckx via RT  wrote:
> Hi,
>
> When building the 1.0.1h release I got this error:
> heartbeat_test.o: In function `set_up':
> test/heartbeat_test.c:94: undefined reference to `ssl_init_wbio_buffer'
> test/heartbeat_test.c:102: undefined reference to `ssl3_setup_buffers'
> heartbeat_test.o: In function `set_up_dtls':
> test/heartbeat_test.c:140: undefined reference to `dtls1_process_heartbeat'
> heartbeat_test.o: In function `set_up_tls':
> test/heartbeat_test.c:165: undefined reference to `tls1_process_heartbeat'
> collect2: error: ld returned 1 exit status
> ../Makefile.shared:171: recipe for target 'link_app.gnu' failed
> make[3]: *** [link_app.gnu] Error 1
>
> This is probably related to me not exporting those symbols as they are marked 
> local.

What version/branch or platform? Steve Henson submitted a fix to
#ifdef out the test on Windows for now to avoid this exact problem.
(We'll look into a longer-term solution as part of the testing effort;
I haven't access to a Windows machine at the moment.)

Mike
__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org

Re: Improving unit/automated test coverage

[Mike Bland]

Thanks to a few brave volunteers and the support of the core OpenSSL
team, it looks like we can begin moving on this effort soon. I've
begun to document the current state of things on the wiki:

http://wiki.openssl.org/index.php/Unit_Testing

There's lots to discuss with regard to the "Goals", "Tasks", and
"Discussion Topics" enumerated on the wiki. I'm happy to start a
discussion on this thread, or other openssl-dev threads; at the same
time, I'm wondering if a new openssl-testing group would be more
appropriate. If it's desirable, I could manage it via Google Groups,
rather than create more majordomo work for the core team. Thoughts?

Thanks,

Mike
__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org

Improving unit/automated test coverage

[Mike Bland]

Hey folks,

With Ben Laurie's help, I recently contributed ssl/heartbeat_test.c,
which is a unit test that acts as a regression test against the
Heartbleed bug. I'd like to contribute more to the project in the
coming months in terms of helping grow a robust suite of
unit/integration/automated tests.

It seems that the encryption algorithms themselves are relatively
well-tested; in contrast, Heartbleed was an infrastructure bug. It's
in shoring up the test coverage of the infrastructure bits where I can
be of most direct service, but I'm hoping others may see opportunities
to apply similar techniques to more advanced testing issues.

I'd like to make sure there are at least a handful of contributors who
are willing to work closely with me to establish some new policies
around unit testing and code reviews (e.g. no new non-trivial changes
without tests; smaller, well-tested changes vs. monolithic, untested
changes), in addition to the actual writing of tests. There'd be some
tool setup and documentation work as well. The goal would be to help
everyone learn effective unit testing strategies so that, over time,
test coverage and code quality steadily improves. It will be a
lengthy, imperfect process, but one that I believe will ultimately
make a positive difference in the code base if people are willing to
try it.

My goal would be to help everyone learn to fish, to use the tired
cliché. I currently have very little knowledge of the OpenSSL code
base or community, and I don't have a ton of time to do all the heavy
lifting by myself; nor do I think that being the lone "testing guy"
would be the best use of my time or in the best interests of OpenSSL.
However, I want to contribute however I can to help the community as a
whole address this one particular issue, and to maximize the impact of
my contributions.

Happy to hear people's thoughts on this. If the uptake is positive, I
can help organize the effort to get things moving soon.

Thanks,

Mike
__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org

Re: build failure when using OPENSSL_NO_HEARTBEATS

[Mike Bland]

Just opened https://github.com/openssl/openssl/pull/110 with a fix.

Mike

On Thu, May 22, 2014 at 2:28 PM, Lukas Tribus  wrote:
> Hey guys!
>
> Since commit 6af080acaf ("Unit/regression test for TLS heartbeats."),
> when compiling master/OpenSSL_1_0_2-stable/OpenSSL_1_0_1-stable with
> -DOPENSSL_NO_HEARTBEATS the build fails with:
>
> heartbeat_test.c: In function ‘set_up_dtls’:
> heartbeat_test.c:127:30: error: ‘dtls1_process_heartbeat’ undeclared (first 
> use in this function)
> heartbeat_test.c:127:30: note: each undeclared identifier is reported only 
> once for each function it appears in
> heartbeat_test.c: In function ‘set_up_tls’:
> heartbeat_test.c:151:30: error: ‘tls1_process_heartbeat’ undeclared (first 
> use in this function)
> make[1]: *** [heartbeat_test.o] Error 1
> make[1]: Leaving directory `/home/lukas/openssl/test'
> make: *** [build_tests] Error 1
>
>
>
>
> Regards,
>
> Lukas
>
>
__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org

Re: How to help OpenSSL

[Mike Bland]

Oh, absolutely I've verified it. :-)

I'll get that turned around to you shortly.

Mike


On Sat, Apr 26, 2014 at 6:33 AM, Ben Laurie  wrote:

> On 24 April 2014 18:44, Mike Bland  wrote:
> > On Thu, Apr 24, 2014 at 1:31 PM, Ben Laurie  wrote:
> >>
> >> 6. Write new tests
> >>
> >> Our test suite sucks. More tests is better.
> >
> >
> > Shall I send a pull request for this:
> >
> >
> https://github.com/mbland/openssl/commit/a7a9e18b550edf059dfd54683ac1f45170ff9fb2
>
> Yes, please! Ideally, as I say, for all branches.
>
> Have you verified that the test fails pre-fix?
> __
> OpenSSL Project http://www.openssl.org
> Development Mailing List   openssl-dev@openssl.org
> Automated List Manager   majord...@openssl.org
>

Re: How to help OpenSSL

[Mike Bland]

On Thu, Apr 24, 2014 at 1:31 PM, Ben Laurie  wrote:

> 6. Write new tests
>
> Our test suite sucks. More tests is better.
>

Shall I send a pull request for this:

https://github.com/mbland/openssl/commit/a7a9e18b550edf059dfd54683ac1f45170ff9fb2

Mike

[PATCH] heartbeat_test

[Mike Bland]

Unit test for the TLS heartbeat code; acts as a regression test against
CVE-2014-0160.

Thanks,

Mike


heartbeat_test.patch
Description: Binary data

Unit/Regression test for Heartbleed bug

[Mike Bland]

I've prepared a proof-of-concept unit/regression test for the Heartbleed
bug that I've posted at: http://goo.gl/wTYD9K

If folks are interested, I can prepare an official patch to add it to
OpenSSL.

Thanks,

Mike
mbl...@acm.org