Re: [openssl-dev] Blog post; changing in email, crypto policy, etc

[Richard Levitte]

In message <e7b6d7b5-4ea0-42d6-94c0-794fc5881...@dukhovni.org> on Wed, 24 Jan 
2018 13:08:54 -0500, Viktor Dukhovni <openssl-us...@dukhovni.org> said:

openssl-users> > If we agree that mailing lists are preferrable to
openssl-users> > GitHub threads, then we should not close down
openssl-users> > openssl-dev.
openssl-users> 
openssl-users> Well, but we now have "openssl-project" for discussions
openssl-users> of the ongoing development of OpenSSL.  It is mostly
openssl-users> for team members, but though it is moderated, IIRC
openssl-users> others can join and post.

This is confusing, and not what was intended.  In other words,
openssl-project is *not* a new openssl-dev!  If it was, I don't see
why we would even bother making a new list...

>From the blog entry:

> We created a new mailing list, openssl-project, that is for
> discussions about the governance and policies of OpenSSL. Anyone can
> join this list. Only members of the OMC and committers will be able
> to post.

Governance and policies (roughly speaking, 'cause there may be some
derailing that's shouldn't be there) is not, as far as I understand,
"development of OpenSSL".  It may be close, thought, such as planning
the schedule of the next release.

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Blog post; changing in email, crypto policy, etc

[Richard Levitte]

In message <8c351e82-600b-487e-aef3-a3f42cd23...@akamai.com> on Tue, 23 Jan 
2018 14:38:14 +, "Salz, Rich via openssl-dev" <openssl-dev@openssl.org> 
said:

openssl-dev> 
openssl-dev> ➢ For the lovers of NNTP: openssl-project has been added to 
news.gmane.org
openssl-dev> as gmane.comp.encryption.openssl.project as readonly list.
openssl-dev>   
openssl-dev> I will always have a fondness for NNTP :)

...  except for the trashing of the database disk(s) back in the days
if you're running a server...  (I did)  (on VMS ;-))

But yeah, totally agree otherwise

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

[openssl-dev] NonStop platform support

[Richard Levitte]

In message <002801d38956$aec22c30$0c468490$@nexbridge.com> on Tue, 9 Jan 2018 
09:32:25 -0500, "Randall S. Becker" <rsbec...@nexbridge.com> said:

rsbecker> A request, maybe OT. The NonStop platform does broadly
rsbecker> deploy Apache but do use OpenSSL. I understand that OpenSSL
rsbecker> does not officially support the HPE NonStop NSE/NSX
rsbecker> platforms - but it is used on the platform through my team's
rsbecker> port, which I currently support, and through other ports as
rsbecker> well. Added a dependency to Apache is likely to dead-end the
rsbecker> project for us depending on the depth of the dependency, if
rsbecker> I understand where this is going (hoping I am wrong).

I pulled this away from the Speck discussion as it is indeed OT.

Does this involve some specific config target?  In that case, you
might be interested in the effort in PR 5043:
https://github.com/openssl/openssl/pull/5043

(if you claim the use of and can verify the correctness of some
specific config target(s), they can be classified as community
supported)

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Speck Cipher Integration with OpenSSL

[Richard Levitte]

I'm not terribly savvy regarding IoT, but I imagine that they do talk
to something bigger.  A server end, perhaps?  What do we expect to run
on that end?  What happens, in that case, if SPECK makes its way into
the TLS cipher suites?  Would it be interesting to have OpenSSL
interop with such devices?

Note: I'm not terribly partial either way, just thought that we need
to look at it from a broader perspective...

Cheers,
Richard

In message  on Mon, 8 Jan 2018 
13:58:59 -0800 (PST), Paul Dale  said:

paul.dale> I'm wondering if one of the more specialised embedded cryptographic 
toolkits mightn't be a better option for your lightweight IoT TLS stack.  There 
is a wide choice available: CycloneSSL, ECT, Fusion, MatrixSSL, mbedTLS, 
NanoSSL, SharkSSL, WolfSSL, uC/SSL and many others.  All of them claim to be 
the smallest, fastest and most feature laden :)  To sell to the US government,  
your first selection criteria should be "does the toolkit have a current FIPS 
validation?"  From memory this means: ECT, nanoSSL or WolfSSL.
paul.dale> 
paul.dale> The more comprehensive toolkits (OpenSSL, NSS, GNU TLS) are less 
suitable for embedded applications, especially tightly resource constrained 
ones.  It is possible to cut OpenSSL down in size but it will never compete 
with the designed for embedded toolkits.  Plus, the FIPS module is fixed and 
cannot be shrunk.
paul.dale> 
paul.dale> The current OpenSSL FIPS validation only applies to 1.0.2 builds 
currently.  FIPS is on the project plan for 1.1 but it isn't available at the 
moment.  The US government is forbidden to purchase any product that contains 
cryptographic operations unless the product has a FIPS validation.  No FIPS, no 
sale.
paul.dale> 
paul.dale> 
paul.dale> Pauli
paul.dale> -- 
paul.dale> Oracle
paul.dale> Dr Paul Dale | Cryptographer | Network Security & Encryption 
paul.dale> Phone +61 7 3031 7217
paul.dale> Oracle Australia
paul.dale> 
paul.dale> -Original Message-
paul.dale> From: William Bathurst [mailto:wbath...@gmail.com] 
paul.dale> Sent: Tuesday, 9 January 2018 7:10 AM
paul.dale> To: openssl-dev@openssl.org
paul.dale> Cc: llamour...@gmail.com
paul.dale> Subject: Re: [openssl-dev] Speck Cipher Integration with OpenSSL
paul.dale> 
paul.dale> Hi Hanno/all,
paul.dale> 
paul.dale> I can understand your view that "more is not always good" in crypto. 
The reasoning behind the offering can be found in the following whitepaper:
paul.dale> 
paul.dale> 
https://csrc.nist.gov/csrc/media/events/lightweight-cryptography-workshop-2015/documents/papers/session1-shors-paper.pdf
paul.dale> 
paul.dale> I will summarize in a different way though. We wish to offer an 
optimized lightweight TLS for IoT. A majority of devices found in IoT are 
resource constrained, for example a device CPU may only have 32K of RAM. 
Therefore security is an afterthought by developers. For some only AES 128 is 
available and they wish to use 256 bit encryption. Then Speck
paul.dale> 256 would be an option because it has better performance and 
provides sufficient security.
paul.dale> 
paul.dale> Based on the above scenario you can likely see why we are interested 
in OpenSSL. First, OpenSSL can be used for terminating lightweight TLS 
connections near the edge, and then forwarding using commonly used ciphers.
paul.dale> 
paul.dale> [IoT Device] -TLS/Speck>[IoT Gateway]-TLS> [Services]
paul.dale> 
paul.dale> Also, we are interested in using OpenSSL libraries at the edge for 
client creation. One thing we would like to do is provide instructions for an 
highly optimized build of OpenSSL that can be used for contrained devices.
paul.dale> 
paul.dale> I think demand will eventually grow because there is an initiative 
by the US government to improve IoT Security and Speck is being developed and 
proposed as a standard within the government. Therefore, I see users who wish 
to play in this space would be interested in a version where Speck could be 
used in OpenSSL.
paul.dale> 
paul.dale> It is my hope to accomplish the following:
paul.dale> 
paul.dale> [1] Make Speck available via Open Source, this could be as an option 
or as a patch in OpenSSL.
paul.dale> [2] If we make it available as a patch, is there a place where we 
would announce/make it known that it is available?
paul.dale> 
paul.dale> We are also looking at open-sourcing the client side code. This 
would be used to create light-weight clients that use Speck and currently we 
also build basic OAuth capability on top of it.
paul.dale> 
paul.dale> Thanks for your input!
paul.dale> 
paul.dale> Bill
paul.dale> 
paul.dale> On 1/5/2018 11:40 AM, Hanno Böck wrote:
paul.dale> > On Fri, 5 Jan 2018 10:52:01 -0800
paul.dale> > William Bathurst  wrote:
paul.dale> >
paul.dale> >> 1) Community interest in such a lightweight cipher.
paul.dale> > I think there's a shifting view that "more is not always good" in 
paul.dale> 

Re: [openssl-dev] Problems building openssl on Solaris

[Richard Levitte]

Ah, sorry, I didn't read the output properly.

Regarding the STV_PROTECTED warnings, I don't know at all...  I did a
bit of a search and saw that this has been discussed before, a little
more than a year ago.  See
https://mta.openssl.org/pipermail/openssl-dev/2016-August/008192.html

As for the missing symbols, that tells me something went horribly
wrong when compiling assembler stuff.  The quick fix is to use the
'no-asm' configuration option.  However, if you want to help getting
assembler to compile, I suggest doing this (no configuration change):

rm crypto/sparccpuid.o
make build_generated
make crypto/sparccpuid.o

Have a look at the output, and if you can't figure it out, please send
us that output.

(note: there are other assembler files that seem to fail as well, for
example those for the bignum library, so this is clearly an assembler
issue)

Cheers,
Richard

In message 

Re: [openssl-dev] Problems building openssl on Solaris

[Richard Levitte]

I suggest adding 'no-threads' to the OpenSSL configuration options, at
least as a first step.  That should at least take away gcc's complaint
about '-pthread'...  I cannot say if that'll fix the rest, I don't
know Solaris enough.

Cheers,
Richard

In message  
on Fri, 17 Nov 2017 11:08:34 +0300, Dmitry Belyavsky  said:

beldmit> Hello,
beldmit> 
beldmit> We experience problems building OpenSSL on Solaris.
beldmit> 
beldmit> /usr/local/src/openssl-1.1.0g>uname -a
beldmit> 
beldmit> SunOS pooh.tcinet.ru 5.10 Generic_147147-26 sun4u sparc 
SUNW,SPARC-Enterprise
beldmit> 
beldmit> /usr/local/src/openssl-1.1.0g>gcc -v
beldmit> 
beldmit> Reading specs from /usr/local/lib/gcc/sparc-sun-solaris2.10/3.4.6/specs
beldmit> Configured with: ../configure --with-as=/usr/ccs/bin/as 
--with-ld=/usr/ccs/bin/ld --enable-shared
beldmit> --enable-languages=c,c++,f77
beldmit> Thread model: posix
beldmit> gcc version 3.4.6
beldmit> 
beldmit> OpenSSL 1.1.0g is configured via
beldmit> ./Configure solaris64-sparcv9-gcc
beldmit> 
beldmit> Here is the end of output:
beldmit> 
beldmit> ...
beldmit> 
beldmit> 
LD_LIBRARY_PATH=.:/usr/local/ssl/lib:/usr/sfw/lib/sparcv9:/usr/local/lib gcc 
-DDSO_DLFCN
beldmit> -DHAVE_DLFCN_H -DNDEBUG -DOPENSSL_THREADS -DOPENSSL_NO_STATIC_ENGINE
beldmit> -DOPENSSL_PIC -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM
beldmit> -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DGHASH_ASM 
-DECP_NISTZ256_ASM
beldmit> -DPOLY1305_ASM -DOPENSSLDIR="/usr/local/ssl" 
-DENGINESDIR="/usr/local/lib/engines-1.1" -m64
beldmit> -mcpu=ultrasparc -Wall -DB_ENDIAN -DBN_DIV2W -O3 -pthread -DFILIO_H -o 
apps/openssl
beldmit> apps/app_rand.o apps/apps.o apps/asn1pars.o apps/ca.o apps/ciphers.o 
apps/cms.o apps/crl.o
beldmit> apps/crl2p7.o apps/dgst.o apps/dhparam.o apps/dsa.o apps/dsaparam.o 
apps/ec.o apps/ecparam.o
beldmit> apps/enc.o apps/engine.o apps/errstr.o apps/gendsa.o apps/genpkey.o 
apps/genrsa.o apps/nseq.o
beldmit> apps/ocsp.o apps/openssl.o apps/opt.o apps/passwd.o apps/pkcs12.o 
apps/pkcs7.o apps/pkcs8.o
beldmit> apps/pkey.o apps/pkeyparam.o apps/pkeyutl.o apps/prime.o apps/rand.o 
apps/rehash.o apps/req.o
beldmit> apps/rsa.o apps/rsautl.o apps/s_cb.o apps/s_client.o apps/s_server.o 
apps/s_socket.o apps/s_time.o
beldmit> apps/sess_id.o apps/smime.o apps/speed.o apps/spkac.o apps/srp.o 
apps/ts.o apps/verify.o
beldmit> apps/version.o apps/x509.o -L. -lssl -L. -lcrypto -lsocket -lnsl -ldl
beldmit> gcc: unrecognized option `-pthread'
beldmit> ld: warning: relocation warning: R_SPARC_COPY: file ./libcrypto.so: 
symbol PBEPARAM_it: relocation
beldmit> bound to a symbol with STV_PROTECTED visibility
beldmit> ld: warning: relocation warning: R_SPARC_COPY: file ./libcrypto.so: 
symbol PBE2PARAM_it:
beldmit> relocation bound to a symbol with STV_PROTECTED visibility
beldmit> ld: warning: relocation warning: R_SPARC_COPY: file ./libcrypto.so: 
symbol PBKDF2PARAM_it:
beldmit> relocation bound to a symbol with STV_PROTECTED visibility
beldmit> Undefined first referenced
beldmit> symbol in file
beldmit> _sparcv9_rdtick ./libcrypto.so
beldmit> bn_add_words ./libcrypto.so
beldmit> _sparcv9_vis1_instrument ./libcrypto.so
beldmit> bn_sub_words ./libcrypto.so
beldmit> bn_sqr_words ./libcrypto.so
beldmit> OPENSSL_cleanse apps/apps.o
beldmit> _sparcv9_rdcfr ./libcrypto.so
beldmit> _sparcv9_vis1_instrument_bus2 ./libcrypto.so
beldmit> _sparcv9_vis3_probe ./libcrypto.so
beldmit> bn_mul_words ./libcrypto.so
beldmit> _sparcv9_vis2_probe ./libcrypto.so
beldmit> _sparcv9_vis1_probe ./libcrypto.so
beldmit> ChaCha20_ctr32 ./libcrypto.so
beldmit> _sparcv9_vis1_instrument_bus ./libcrypto.so
beldmit> bn_mul_comba4 ./libcrypto.so
beldmit> bn_mul_comba8 ./libcrypto.so
beldmit> bn_sqr_comba4 ./libcrypto.so
beldmit> bn_sqr_comba8 ./libcrypto.so
beldmit> _sparcv9_fmadd_probe ./libcrypto.so
beldmit> CRYPTO_memcmp ./libssl.so
beldmit> bn_mul_add_words ./libcrypto.so
beldmit> bn_div_words ./libcrypto.so
beldmit> _sparcv9_fjaesx_probe ./libcrypto.so
beldmit> ld: fatal: symbol referencing errors. No output written to apps/openssl
beldmit> collect2: ld returned 1 exit status
beldmit> *** Error code 1
beldmit> 
beldmit> What can we do to fix it?
beldmit> 
beldmit> Thank you!
beldmit> 
beldmit> --
beldmit> SY, Dmitry Belyavsky
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [openssl-users] OpenSSL engine and TPM usage.

[Richard Levitte]

In message 

Re: [openssl-dev] Can I haz TLS 1.3 ?

[Richard Levitte]

It's not specific to devops. Here, a quick history lesson:

https://english.stackexchange.com/questions/20356/origin-of-i-can-haz

Cheers
Richard 

Ted Marynicz  skrev: (4 oktober 2017 10:53:35 CEST)
>Haz?
>
>Is that some kind of devops-speak I am not (yet) aware of?
>
>Ted
>(a grand-father)
>
>On 3 October 2017 at 18:36, Salz, Rich via openssl-dev <
>openssl-dev@openssl.org> wrote:
>
>> Some people have asked why TLS 1.3 isn’t available yet.  This might
>help;
>> it’s a draft of a posting for my company’s blog.
>>
>>
>>
>>
>>
>> Can I Haz TLS 1.3 ?
>>
>>
>>
>> Everybody wants to be able to use TLS 1.3. Among the reasons are:
>>
>> It’s faster – being able to reconnect to a server
>you’ve
>> previously used, and saving a full round-trip latency is impressive.
>>
>> It’s more reliable – the protocol has been cleaned up
>and
>> simplified. For example, the related concepts of sessions, tickets,
>and
>> pre-shared keys are merged and treated consistently. To a protocol
>> designer, it is much more elegant, and therefore much easier to
>implement
>>
>> It’s more secure – Many world-class cryptographers
>have
>> been involved in the protocol design, analyzed it, and tried to break
>it.
>>
>>
>>
>> TLS has been in the “last call” for several weeks now.  What does
>that
>> mean, and what’s holding it up?
>>
>>
>>
>> The IETF is the organization that defines most of the standards that
>> define how the Internet works. They cover everything from naming
>(DNS) to
>> routing around firewalls, up to and including HTTP. The documents,
>known as
>> RFCs, are created by working groups, passed to a steering committee
>for
>> review, and then published as “Internet Standards.”
>>
>>
>>
>> Participation in a working group (WG) is, by design, very easy and
>not a
>> lot of overhead.  You just have to join a mailing list.  Every WG has
>a
>> mailing list and there are currently more than 110 working groups
>hosted at
>> the IETF. Each one has a status page, that shows their charter (what
>they
>> are working on), the current sent of documents, and pointers to the
>mailing
>> lists.  For the TLS working group, that page is at
>> https://datatracker.ietf.org/wg/tls/documents/.
>>
>>
>>
>> Future RFC’s start as Internet-Drafts. Each draft usually goes
>through
>> multiple revisions, as the working group members comment on it, other
>> feedback is addressed, and so on.  At some point, the authors or
>editors
>> will post a new draft.  By convention, every working group draft is
>named
>> “draft-ietf-{WGNAME}-{subject}-{nn}” where {WGNAME} is the name of
>the
>> working group, {subject} is the name of the document, and {nn} is the
>> revision number.  For example, “draft-ietf-tls-tls13-21” is the 21st
>> draft of the TLS 1.3 specification from the TLS working group.
>>
>>
>>
>> When the working group thinks a document is done, it enters WGLC,
>working
>> group last call.  This period, usually lasting a couple of weeks, is
>the
>> last chance to make editorial or serious factual fixes.  Since most
>people
>> are deadline-driven, this is usually when many on the WG wake up and
>read
>> the drafts. After WGLC, it goes to the IESG (technical leadership
>> basically) for review.  As I said, TLS 1.3 has been in WGLC for
>weeks.  So
>> why can’t we use it yet?
>>
>>
>>
>> First, the different drafts don’t interoperate. Each represents a
>> different milestone on the way to the full specification, and a flag
>in the
>> header identifies which draft is being used. OpenSSL, used by most of
>the
>> servers on the Internet, is currently at draft-21. Chrome and
>Firefox, two
>> of the most popular browsers on the Internet, are staying at
>draft-18; they
>> don’t want to upgrade until we have the final version. (I think
>that’s
>> silly, but I don’t work for either browser.)
>>
>>
>>
>> Tests run by various companies, including Google, Mozilla, and
>Facebook,
>> indicate that the “failure rate” of TLS 1.3 is disturbingly high. It
>> appears that network hardware such as routers, gateways, load
>balancers and
>> the like, are blocking TLS 1.3 packets because they don’t recognize
>the
>> protocol. They are doing “fail closed” and block the connections
>because
>> they don’t understand it, rather than assuming it’s safe to forward.
>The
>> IETF often uses the term “middlebox” to describe such hardware that
>> operates between endpoints, and this type of behavior that blocks new
>> protocols as “ossificiation.”  The various companies, and no doubt
>others,
>> are trying experiments to tweak the protocol to lower the failure
>rate. For
>> example, in some circumstances it might be acceptable to make a TLS
>1.3
>> message look like a TLS 1.2 message (after you’ve already committed
>to
>> doing TLS 1.3).
>>
>>
>>
>> So far nobody has released any details. When it happens, it will be
>on the
>> TLS-WG mailing list, which you can find at the page I referenced
>above.
>> Until then, 

Re: [openssl-dev] how to static compile ssl engine into openssl

[Richard Levitte]

In message <20170926203053.5hlfcbx273lko...@roeckx.be> on Tue, 26 Sep 2017 
22:30:53 +0200, Kurt Roeckx <k...@roeckx.be> said:

kurt> On Tue, Sep 26, 2017 at 07:32:06AM +0200, Richard Levitte wrote:
kurt> > 
kurt> > You mean to have nginx use the shared OpenSSL libraries, which also
kurt> > enables dynamic engines?  Yes, that's the usual way to go about these
kurt> > things.
kurt> 
kurt> Do we support dynamic engines with a static build?

No we don't.  no-shared means no-dynamic-engine

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] how to static compile ssl engine into openssl

[Richard Levitte]

In message <31F771DF13463A429610AEEBF6AFAE820182EBC4@mbx14.360buyAD.local> on 
Mon, 25 Sep 2017 10:16:28 +, 程文平 <chengwenpi...@jd.com> said:

chengwenping1> I’m working on accelerating ssl traffic with Intel QAT
chengwenping1> card, now openssl 1.1.0f is integrated into Nginx, so I
chengwenping1> need to static compile Intel QAT engine into openssl,
chengwenping1> and I do not find some useful info about it from
chengwenping1> Internet, although openssl-1.1.0f/engines/ build.info,
chengwenping1> it is not applicable from QAT engine from
chengwenping1> https://github.com/01org/QAT_Engine. Is there a guide
chengwenping1> line for this case?

Unforatunately, there is no such guide that I know of.  I just had a
look in e_qat.c, and there seems to be support for doing that there
(see the sections guarded by OPENSSL_NO_DYNAMIC_ENGINES), but I can't
see any way to make use of that in their configuration.

If this is what you really want, I suggest you create an issue in the
QAT_Engine project...  but you probably need to understand that you
may not get what you want, and if you do, it's probably going to be an
unsupported hack.

chengwenping1> There is another alternative to do it, just to alone
chengwenping1> compile openssl and nginx, but it will take effort to
chengwenping1> deploy it.

You mean to have nginx use the shared OpenSSL libraries, which also
enables dynamic engines?  Yes, that's the usual way to go about these
things.

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] libcrypto.pc needs to list libpthread as a dependency

[Richard Levitte]

In message <59bebf25.5040...@roumenpetrov.info> on Sun, 17 Sep 2017 21:29:57 
+0300, Roumen Petrov <open...@roumenpetrov.info> said:

openssl> Hi Howard,
openssl> 
openssl> Howard Chu wrote:
openssl> > Roumen Petrov wrote:
openssl> >> Howard Chu via openssl-dev wrote:
openssl> >>> In OpenSSL 1.1 on Linux (at least) libcrypto now has a dependency 
on
openssl> >>> libpthread but this is not reflected in the pkgconfig file. As a
openssl> >>> result, tools like CMake fail to detect libcrypto properly when
openssl> >>> linking against the static library. libpthread should be added to 
the
openssl> >>> Libs.private line of the pkgconfig file.
openssl> >>>
openssl> >>> For example:
openssl> >>> 
https://github.com/monero-project/monero/issues/2402#issuecomment-327514216
openssl> >>>
openssl> >>
openssl> >> Problem is that OpenSSL does not add it directly.
openssl> >> Build process does not know libraries added by compiler flags like
openssl> >> -pthread.
openssl> >>
openssl> >> Does not look like OpenSSL issue.
openssl> >
openssl> > That sounds like a lame cop-out. Currently the build process already
openssl> > knows that -ldl goes in ${EX_LIBS}. The Makefile is full of
openssl> > system-dependent settings already.
openssl> >
openssl> 
openssl> Case is totally different. Library dl is needed by DSO functionality.
openssl> 
openssl> Thread library is platform dependent . For instance it is not added on 
android. See gcc spec file (version 4.9+?).
openssl> 
openssl> Dunno what will be impact if openssl exports C-flag -pthread.

-pthread should be part of both compiling and linking, so Howard is
perfectly correct, we're not doing this quite right.  When -pthread is
used, it should also be added to the libcrypto.pc's Libs.private line.

I'm currently travelling, but will give this more concrete attention
when I've returned, i.e. next week.

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] libcrypto.pc needs to list libpthread as a dependency

[Richard Levitte]

Matt Caswell  skrev: (17 september 2017 15:04:10 GMT+08:00)
>On Sat, 16 Sep 2017 22:26:10 +0100
>Howard Chu via openssl-dev  wrote:
>
>> In OpenSSL 1.1 on Linux (at least) libcrypto now has a dependency on 
>> libpthread but this is not reflected in the pkgconfig file. As a
>result, tools 
>> like CMake fail to detect libcrypto properly when linking against the
>static 
>> library. libpthread should be added to the Libs.private line of the
>pkgconfig 
>
>Hmmm - there is no pkgconfig file at all in 1.1.0. It was removed
>because it was felt this was better added by OS specific packages. If
>you have one it probably came from the package for your specific distro
>not the openssl project itself.

You have to look in the Makefile template to find it. 

>
>Matt
>
>
>
>> file.
>> 
>> For example: 
>>
>https://github.com/monero-project/monero/issues/2402#issuecomment-327514216
>> 
>> -- 
>>-- Howard Chu
>>CTO, Symas Corp.   http://www.symas.com
>>Director, Highland Sun http://highlandsun.com/hyc/
>>Chief Architect, OpenLDAP  http://www.openldap.org/project/
>> -- 
>> openssl-dev mailing list
>> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
>> 

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] OPenssl 1.1.0 and FIPS

[Richard Levitte]

The Doctor  skrev: (16 september 2017 15:26:16 CEST)
>On Sat, Sep 16, 2017 at 12:56:08PM +, Salz, Rich via openssl-dev
>wrote:
>> 
>> Tryong to compile  Fips into OPEnssl-1.1.0  and I run into
>> 
>> FIPS is not supported for 1.1.0
>> 
>
>jUST A SMALL FIX WILL DO.

Really? Are you saying that you made a small fix and then everything worked 
like a charm? I'm dubious, frankly, because there are just too many changes 
between OpenSSL 1.0.x (that the FIPS module was made for) and 1.1.0. There 
should be much more breakage. 

Cheers 
Richard 

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] how to compile out selected ciphers

[Richard Levitte]

In message 

Re: [openssl-dev] Plea for a new public OpenSSL RNG API

[Richard Levitte]

In message <168cef3c-d655-47bf-9874-048308154...@ll.mit.edu> on Tue, 29 Aug 
2017 19:04:48 +, "Blumenthal, Uri - 0553 - MITLL" <u...@ll.mit.edu> said:

uri> uri> Could you please be more specific wrt. DRBG organization that in 
your opinion could impact the UI? 
uri> 
uri> Are you talking about the UI API or something else?
uri> 
uri> No-no-no, just the UI API. 
uri> 
uri> I used the term “UI” to emphasize that this is the “public” part of the 
API, exposed to the user, expected to be called by the user, and (probably) 
designed to be stable for some (hopefully considerable) time.

Ah, ok.  In an OpenSSL context, this gets a bit confusing since there
is an API called UI (  ).

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Plea for a new public OpenSSL RNG API

[Richard Levitte]

In message <3a54fcefd17d4735920426d21b3b8...@ex13.ncp.local> on Tue, 29 Aug 
2017 13:39:02 +, "Dr. Matthias St. Pierre" <matthias.st.pie...@ncp-e.com> 
said:

Matthias.St.Pierre> Just a sudden inspiration: If the RAND_DRBG becomes a truly 
independent API it might be better to strip the RAND_ prefix and redesign the 
API such that one has
Matthias.St.Pierre> 
Matthias.St.Pierre> - a DRBG_CTX structure for the data members
Matthias.St.Pierre> - a DRBG_METHOD  structure for its methods
Matthias.St.Pierre> 
Matthias.St.Pierre> Would this look more OpenSSL-like to you?

Yes.  And per fairly recent recommendations to avoid cluttering the
name space, that would be OSSL_DRGB_CTX and OSSL_DRGB_METHOD, btw.

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Plea for a new public OpenSSL RNG API

[Richard Levitte]

In message <e6faf983220642c192bba281b9b32...@ex13.ncp.local> on Tue, 29 Aug 
2017 13:27:20 +, "Dr. Matthias St. Pierre" <matthias.st.pie...@ncp-e.com> 
said:

Matthias.St.Pierre> > Essentially, the argument for your last remark is 
in-structure vtable
Matthias.St.Pierre> > vs refered to vtable.  I tend to prefer the latter (and 
that's the
Matthias.St.Pierre> > usual OpenSSL pattern too, even though there are 
exceptions).
Matthias.St.Pierre> 
Matthias.St.Pierre> You are the experts and much more familiar with
Matthias.St.Pierre> the code then I am. My role was only to give the
Matthias.St.Pierre> starting shot, the rest is up to you.

Fair enough!  :-)

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Plea for a new public OpenSSL RNG API

[Richard Levitte]

In message <0e062727611c44bb8039170bbec11...@ex13.ncp.local> on Tue, 29 Aug 
2017 13:05:21 +, "Dr. Matthias St. Pierre" <matthias.st.pie...@ncp-e.com> 
said:

Matthias.St.Pierre> > Frankly, I cannot see anything wrong with making that a 
public API.  I
Matthias.St.Pierre> > wonder, though, if this is going to be an implementation 
that's kinda
Matthias.St.Pierre> > sorta built-in only, or if other implementations of the 
basic stuff
Matthias.St.Pierre> > will be possible...  or in other words, will we have a 
"method"
Matthias.St.Pierre> > structure like we have with so many other APIs?  In 
saying this, I'm
Matthias.St.Pierre> > counting in the possibility that some implementations 
could come in
Matthias.St.Pierre> > the form of engines, is this something that's been 
thought about?  I
Matthias.St.Pierre> > do notive the DRGB_CTX structure, but that doesn't quite 
follow the
Matthias.St.Pierre> > usual "method" pattern we have...
Matthias.St.Pierre> 
Matthias.St.Pierre> Currently it's only possible to customize the callbacks but 
not the deterministic algorithm. IMHO this is sufficient for the needs of a 
standard OpenSSL user who only wants control over the entropy source. A true 
new algorithm (like e.g. CHACHA2_DRBG) should be implemented by experts and 
added mainstream. So I don't see any advantage of having an engine over using 
the 'vtable' approach from the FIPS DRBG, which has been removed.

Essentially, the argument for your last remark is in-structure vtable
vs refered to vtable.  I tend to prefer the latter (and that's the
usual OpenSSL pattern too, even though there are exceptions).

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Plea for a new public OpenSSL RNG API

[Richard Levitte]

I'm late in the game, having only followed the development very
superficially...

If I understand correctly, the RAND_DRBG API is really a completely
separate API that has nothing to do with the RAND_METHOD API pers se,
i.e. any association between the two is more or less "accidental"?

Frankly, I cannot see anything wrong with making that a public API.  I
wonder, though, if this is going to be an implementation that's kinda
sorta built-in only, or if other implementations of the basic stuff
will be possible...  or in other words, will we have a "method"
structure like we have with so many other APIs?  In saying this, I'm
counting in the possibility that some implementations could come in
the form of engines, is this something that's been thought about?  I
do notive the DRGB_CTX structure, but that doesn't quite follow the
usual "method" pattern we have...

Cheers,
Richard

In message  on Tue, 29 Aug 
2017 09:45:26 +, "Dr. Matthias St. Pierre"  
said:

Matthias.St.Pierre> Hi everybody,
Matthias.St.Pierre> 
Matthias.St.Pierre> on the [openssl-dev] mailing list, there has been a long 
ongoing discussion about the new RAND_DRBG API and comparing it with the old 
RAND_METHOD API (see "[openssl-dev] Work on a new RNG for OpenSSL"). Two of the 
most controversal questions were:
Matthias.St.Pierre> 
Matthias.St.Pierre>  - Do we really need a new RNG API? Should the RAND_DRBG 
API be made public or kept private? (Currently, it's exported from libcrypto 
but only used internally by libssl.)
Matthias.St.Pierre>  - How much control should the user (programmer) be given 
over the reseeding process and/or should he be allowed to add his own 
additional randomness?
Matthias.St.Pierre> 
Matthias.St.Pierre> Many developers seem to be realizing the interesting 
possibilities of the DRBG API and are asking for public access to this new and 
promising API. One of the driving forces behind it is the question about how to 
do seeding and reseeding right. Among others, Uri Blumenthal asked for making 
the DRBG API public.
Matthias.St.Pierre> 
Matthias.St.Pierre> Currently, the OpenSSL core members seem to be reluctant to 
make the API public, at least at this early stage. I understand Rich Salz's 
viewpoint that this requires a thorough discussion, because a public interface 
can't be easily changed and wrong decisions in the early phase can become a 
heavy burdon.
Matthias.St.Pierre> 
Matthias.St.Pierre> Nevertheless, I agree with Uri Blumenthal that the DRBG API 
should be made public. So here comes my
Matthias.St.Pierre> 
Matthias.St.Pierre> ==
Matthias.St.Pierre> Plea for a new public OpenSSL RNG API:
Matthias.St.Pierre> ==
Matthias.St.Pierre> 
Matthias.St.Pierre> The new RAND_DRBG is the superior API. It shouldn't be 
kept private and hidden behind the ancient RAND_METHOD API.
Matthias.St.Pierre> The philosophy of the two APIs is not very well 
compatible, in particular when it comes to reseeding and adding
Matthias.St.Pierre> additional unpredictable input. Hiding the RAND_DRBG 
behind the RAND_METHOD API only causes problems.
Matthias.St.Pierre> Also, it will force people to patch their OpenSSL copy 
if they want to use the superior API.
Matthias.St.Pierre> 
Matthias.St.Pierre> The RAND_DRBG API should become the new public OpenSSL 
RNG API and the old RAND_METHOD API should be deprecated
Matthias.St.Pierre> in the long run. This transition does not need to be 
rushed, but it would be good if there would be early consent
Matthias.St.Pierre> on the road map. I am thinking of a smooth transition 
with a phase of coexistence and a compatibility layer
Matthias.St.Pierre> mapping the default RAND_METHOD to the default public 
RAND_DRBG instance. (This compatibility layer already exists,
Matthias.St.Pierre> it's the 'RAND_OpenSSL()' method.)
Matthias.St.Pierre> 
Matthias.St.Pierre> 
Matthias.St.Pierre> 
Matthias.St.Pierre> Historical Background
Matthias.St.Pierre> =
Matthias.St.Pierre> 
Matthias.St.Pierre> As Rich already mentioned in his blog post, the RAND_DRBG 
isn't new. It's been a part of OpenSSL for a long time, hidden in the FIPS 2.0 
Object Module.
Matthias.St.Pierre> 
Matthias.St.Pierre> I have been working with the FIPS DRBG for quite a while 
now, using a FIPS-capable OpenSSL 1.0.2x crypto library. The reason why our 
company switched to the FIPS DRBG is that one of our products runs on a small 
hardware device which does not have a reliable entropy source, but the product 
has to meet high security standards, in particular w.r.t. its RNG. So we 
decided to use the SmartCard RNG as primary entropy source for a deterministic 
AES-CTR based RNG and use /dev/urandom as additional input. Reseeding should 
occur on every generate request. Using the FIPS DRBG, these requirements were 
easily met, because the API 

Re: [openssl-dev] confusion with rsa_meth_st in a custom RSA engine

[Richard Levitte]

In message 
<bn3pr03mb1462808dc39e3a2a1386aedbe0...@bn3pr03mb1462.namprd03.prod.outlook.com>
 on Wed, 23 Aug 2017 01:50:04 +, "Brett R. Nicholas" 
<brett.r.nicholas...@dartmouth.edu> said:

> I am trying to develop a engine for a custom RSA hardware accelerator, and 
> have a few questions
> about the RSA_METHOD stucture implementation.
...
> I'm confused as to which RSA_METHOD function pointers that my engine needs to 
> implement. I
> show the structure below for reference:
> 
> struct rsa_meth_st {
> char *name;
> int (*rsa_pub_enc) (int flen, const unsigned char *from,
> unsigned char *to, RSA *rsa, int padding);
> int (*rsa_pub_dec) (int flen, const unsigned char *from,
> unsigned char *to, RSA *rsa, int padding);
> int (*rsa_priv_enc) (int flen, const unsigned char *from,
> unsigned char *to, RSA *rsa, int padding);
> int (*rsa_priv_dec) (int flen, const unsigned char *from,
> unsigned char *to, RSA *rsa, int padding);
> 
> int (*rsa_mod_exp) (BIGNUM *r0, const BIGNUM *I, RSA *rsa, BN_CTX *ctx);
> 
> int (*bn_mod_exp) (BIGNUM *r, const BIGNUM *a, const BIGNUM *p,
> const BIGNUM *m, BN_CTX *ctx, BN_MONT_CTX *m_ctx);
> /* stuff */
> int flags;
> /*  stuff ... */
> }; // TYPEDEF'ED TO RSA_METHOD in include/ossl_typ.h
> 
> So, three questions:
> 
> 1 Is it possible for the standard OpenSSL RSA implementation to use my 
> engine's
>  "modular exponentiation" function, without having to rewrite the RSA_
>  [public|private]_[encrypt|decrypt] family of functions from 
> /include/openssl/rsa.h?

Yes.

> 2 If so, does it suffice to only implement the rsa_mod_exp function? Or must 
> I implement
>  both public_enc/dec and private_enc/dec functions as well? I ask, because 
> the source code for
>  the old Intel RSAX engine
>  (https://gist.github.com/bigbrett/91903f773f9d150b7329c7d462cd220a) does 
> this, but I can't
>  figure out how and when in the "RSA flow" the engine's function gets invoked.

I'd like to point out this part of the code, which is relevant:

#ifndef OPENSSL_NO_RSA
meth1 = RSA_PKCS1_SSLeay();
e_rsax_rsa.rsa_pub_enc = meth1->rsa_pub_enc;
e_rsax_rsa.rsa_pub_dec = meth1->rsa_pub_dec;
e_rsax_rsa.rsa_priv_enc = meth1->rsa_priv_enc;
e_rsax_rsa.rsa_priv_dec = meth1->rsa_priv_dec;
e_rsax_rsa.bn_mod_exp = meth1->bn_mod_exp;
e_rsax_rsa.finish = meth1->finish;
#endif

You you see, you don't need to actually *reimplement* the
public/private encrypt/decrypt functions, but you need to make sure
there's *some* implementation available through the method you build
up.  Incidently, in more modern OpenSSLs, you should use
RSA_PKCS1_OpenSSL() rather than RSA_PKCS1_SSLeay().

Anyhow, the standard public/private encrypt/decrypt functions will
call the mod_exp functions that you implement

> 3 In /include/openssl/rsa.h, I saw the following macro for the RSA_METHOD 
> flag field (line 55):
> 
> /*
> * This flag means the private key operations will be handled by rsa_mod_exp
> * and that they do not depend on the private key components being present:
> * for example a key stored in external hardware. Without this flag
> * bn_mod_exp gets called when private key components are absent.
> */
> # define RSA_FLAG_EXT_PKEY 0x0020
> 
> Does this mean that if I use this flag in the "flags" field of RSA_METHOD, 
> that I DO NOT
> need to implement rsa_pub_enc/dec and friends? I guess I'm just confused as 
> to at what
> point in the RSA encryption/decryption process my engine should be invoked at.

That flag means that the standard public/private encrypt/decrypt won't
try to access the p, q, dmp1 and iqmp components of the RSA structure,
i.e. the components that make up the private part.  Instead, that's
left entirely to the mod_exp function (i.e. what you actually do
implement).

If you want to see for yourself what's happening, I suggest a study of
crypto/rsa/rsa_ossl.c (OpenSSL 1.1.0 and up) or crypro/rsa/rsa_eay.c
(OpenSSL 1.0.2).

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Build issue

[Richard Levitte]

In message <CAKH_Ld7SNCErpTqFzZcfb7uE3KqOPZ5+Tw=su-yuboc72tw...@mail.gmail.com> 
on Tue, 1 Aug 2017 22:19:20 -0400, Matthew Stickney <mtstick...@gmail.com> said:

mtstickney> However, having installed the mingw-w64 version of perl, the 
configure
mtstickney> script is failing because it thinks this is a unix-like platform, 
but
mtstickney> perl is producing windows-style paths:
mtstickney> 
mtstickney> > This perl implementation doesn't produce Unix like paths (with 
forward slash
mtstickney> > directory separators).  Please use an implementation that matches 
your
mtstickney> > building platform.
mtstickney> 
mtstickney> > This Perl version: 5.22.0 for MSWin32-x64-multi-thread
mtstickney> 
mtstickney> I can't follow the configure script well enough to tell where it's
mtstickney> detecting the relevant bits from -- the build_scheme key seems to be
mtstickney> responsible for triggering the unix-checker.pm script that fails, 
but
mtstickney> grep only turns up two references to it: the readme, and the 
configure
mtstickney> script. Is this an issue with msys2+mingw-w64 environments?

OR unix-checker.pem might be buggy.  May I suggest insert this line
before the check, try again and see what it says?

    print STDERR "Current directory: ", rel2abs('.'), "\n";

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Build issue

[Richard Levitte]

Ok, I have still not been able to reproduce this.

We have already established that the perl you use is the mingw one,
haven't we?  (if we haven't, that really needs to be checked.
Matching perl and all that)

A test to figure out is this:

perl util/mkdef.pl 32 crypto debug 2> mkdef-debug.txt > /dev/null

Then let someone who knows mkdef look at the output.  I would usually
volunteer, but my time is currently limited (on vacation), so I won't
be able to look at it before mid August.  The debug output is messy,
fair warning.

Cheers,
Richard

In message <cakh_ld5k49i9v5-k_vxtmgreyo5nnal43hakuqcutnlfq6j...@mail.gmail.com> 
on Mon, 31 Jul 2017 21:07:07 -0400, Matthew Stickney <mtstick...@gmail.com> 
said:

mtstickney> I swear I sent my last reply to the list; either my mail client is
mtstickney> messing with me, or I'm going crazy. There's no reason for this to 
be
mtstickney> off-list, and that wasn't my intention in the first place.
mtstickney> 
mtstickney> Anyhow: util/libcrypto.num doesn't seem to have anything unusual in 
it
mtstickney> associated with '_num', so that's fine, but then neither does
mtstickney> crypto.def, so it seems like it's not an issue with util/mkdef.pl. I
mtstickney> mean, obviously the error is /somewhere/
mtstickney> 
mtstickney> -Matt Stickney
mtstickney> 
mtstickney> On Mon, Jul 31, 2017 at 10:22 AM, Matthew Stickney 
<mtstick...@gmail.com> wrote:
mtstickney> > Thanks for the suggestion, and my apologies: I didn't intend to 
take
mtstickney> > this off-list. I'll check libcrypto.num when I get back on the 
machine
mtstickney> > tonight. If it helps, I am building this with 64-bit mingw, but I
mtstickney> > don't see how that would have an effect (there are build warnings
mtstickney> > earlier in the process, but from what I've seen they look 
harmless).
mtstickney> >
mtstickney> > -Matt Stickney
mtstickney> >
mtstickney> > On Mon, Jul 31, 2017 at 10:17 AM, Richard Levitte 
<levi...@openssl.org> wrote:
mtstickney> >> util/mkdef.pl picks up all the data from configdata.pm, and 
regarding
mtstickney> >> options, it looks at $config{options}, which includes everything
mtstickney> >> that's disabled by default.
mtstickney> >>
mtstickney> >> This all is an issue that I'm currently labeling *weird*, 'cause 
it
mtstickney> >> makes no sense at all.  Most of all, this line really has me 
gawking,
mtstickney> >> scratching my head and generally looking silly:
mtstickney> >>
mtstickney> >> Error: _num does not have a number assigned
mtstickney> >>
mtstickney> >> Basically, it looks like *something* is getting ba-a-a-a-dly 
parsed,
mtstickney> >> and I have no idea if that's in util/libcrypto.num (could you 
try and
mtstickney> >> grep for '_num' and possibly '^_num' to see if you can find 
something
mtstickney> >> weird?) or if it's a serious corner case cock up in 
util/mkdef.pl.
mtstickney> >>
mtstickney> >> Worst thing is, I haven't been able to reproduce...  I am going 
to try
mtstickney> >> again just now, with the hope that 32-bit mingw doesn't make a
mtstickney> >> difference...
mtstickney> >>
mtstickney> >> Cheers,
mtstickney> >> Richard
mtstickney> >>
mtstickney> >> In message <a1a9f917-a155-f7c4-f71e-4f4c3de4d...@akamai.com> on 
Mon, 31 Jul 2017 09:02:15 -0500, Benjamin Kaduk <bka...@akamai.com> said:
mtstickney> >>
mtstickney> >> bkaduk> Whoops, I missed that this was unicast when skimming 
mail over the weekend.
mtstickney> >> bkaduk> Looping back in Richard, at least, though if you don't 
mind, putting the list back would be best.
mtstickney> >> bkaduk>
mtstickney> >> bkaduk> Richard, my (extremely) quick skim of mkdef.pl is that 
it is only picking up explicitly set options
mtstickney> >> bkaduk> and not things disabled by default; does that seem 
plausible?
mtstickney> >> bkaduk>
mtstickney> >> bkaduk> -Ben
mtstickney> >> bkaduk>
mtstickney> >> bkaduk> On 07/28/2017 10:38 PM, Matthew Stickney wrote:
mtstickney> >> bkaduk>
mtstickney> >> bkaduk>  [1] has the contents of my crypto.def file, although I 
don't see
mtstickney> >> bkaduk> anything especially odd in there (not that I would know, 
really).
mtstickney> >> bkaduk>
mtstickney> >> bkaduk> I removed the offending symbols from crypto.def, and 
that caused the
mtstickney> >> bkaduk> previously-failing command to succeed. I added some 
print statements
mtstickney> >> bkaduk> to mkdef.pl and configured with "./configure mingw64 
no-rc5 no-md2",
mtstickney> >

Re: [openssl-dev] Build issue

[Richard Levitte]

In message <CAKH_Ld7zcB+XYKFVd95Qz3bG9-kFPpVGSqH2OssQbMV1=uj...@mail.gmail.com> 
on Tue, 25 Jul 2017 20:49:14 -0400, Matthew Stickney <mtstick...@gmail.com> 
said:

mtstickney> Possibly. The original errors and hanging perl process have been
mtstickney> replaced with an enormous number of "undefined reference" errors. 
For
mtstickney> example:
mtstickney> libssl.a(tls_srp.o):tls_srp.c:(.text+0xc4c): undefined reference to 
`BN_ucmp'
mtstickney> libssl.a(tls_srp.o):tls_srp.c:(.text+0xd45): undefined reference to
mtstickney> `OPENSSL_cleanse'
mtstickney> libssl.a(tls_srp.o):tls_srp.c:(.text+0xd5f): undefined reference to 
`SRP_Calc_A'
mtstickney> collect2.exe: error: ld returned 1 exit status
mtstickney> 
mtstickney> I don't know enough about the build system to know whether the
mtstickney> mkdef.pl change might be responsible for this, or whether this is a
mtstickney> separate issue. To follow up on my previous post, the configure line
mtstickney> was indeed "./config", and this is commit 1843787173. Any other data
mtstickney> that I should collect?

Have you tried a 'make clean' and then rebuild?  If the previous run
gave incorrect results, you may have ended up with an incomplete
libcrypto.so, but with timestamps that make it look lite it's already
built.

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Build issue

[Richard Levitte]

Could it be that this patch fixes the issue?

diff --git a/util/mkdef.pl b/util/mkdef.pl
index b3eb6b3d9d..1f214dbb8b 100755
--- a/util/mkdef.pl
+++ b/util/mkdef.pl
@@ -876,7 +876,7 @@ sub do_defs
# Reduce argument lists to empty ()
# fold round brackets recursively: (t(*v)(t),t) -> 
(t{}{},t) -> {}
while(/\(.*\)/s) {
-   s/\([^\(\)]+\)/\{\}/gs;
+   s/\([^\(\)]*\)/\{\}/gs;
s/\(\s*\*\s*(\w+)\s*\{\}\s*\)/$1/gs;#(*f{}) 
-> f
}
# pretend as we didn't use curly braces: {} -> ()


In message <cakh_ld5nd9pcgda+uvkzdg3krw-gz-zicqdpbx7qzx1sfow...@mail.gmail.com> 
on Tue, 25 Jul 2017 15:18:35 -0400, Matthew Stickney <mtstick...@gmail.com> 
said:

mtstickney> I'm away from the machine in question right now, but:
mtstickney> 
mtstickney> > Also, presumably the perl is the msys perl, but please confirm -- 
it must be "matching" in order for things to work.
mtstickney> 
mtstickney> It's definitely the msys perl. I'll check the config line tonight, 
but
mtstickney> I'm virtually certain that it was "./config" in it's entirety.
mtstickney> 
mtstickney> > Can you state what OpenSSL version this is
mtstickney> 
mtstickney> I'll get the commit hash tonight, but this was the tip of master as 
of
mtstickney> about 9pm EDT last night, so I'd imagine that util/mkdef.pl in 
master
mtstickney> still has whatever the issue is.
mtstickney> 
mtstickney> -Matt Stickney
mtstickney> 
mtstickney> On Tue, Jul 25, 2017 at 2:59 PM, Richard Levitte 
<levi...@openssl.org> wrote:
mtstickney> > In message 

Re: [openssl-dev] Build issue

[Richard Levitte]

In message 

Re: [openssl-dev] Windows system cert store

[Richard Levitte]

In message 

Re: [openssl-dev] Compiler requirements

[Richard Levitte]

In message 
<2f548b68de1c47dfaa6a3b0107080...@usma1ex-dag1mb1.msg.corp.akamai.com> on Tue, 
4 Jul 2017 15:05:06 +, "Salz, Rich via openssl-dev" 
<openssl-dev@openssl.org> said:

openssl-dev> > beldmit> What is the minimal version of the compiler to build 
openssl?
openssl-dev> > beldmit> Is it still required C89 compatibility or C99 standard 
can be used?
openssl-dev> > beldmit>
openssl-dev> > beldmit> Unfortunately, I did not find these requirements in 
documentation.
openssl-dev> > 
openssl-dev> > At the beginning of INSTALL, you will find a set of 
requirements.  On of them
openssl-dev> > is "an ANSI C compiler".
openssl-dev> 
openssl-dev> That doesn't answer the question :)  Which version of ANSI C?

Ah, you're right, "ANSI C" is a bit of a loose target depending on who
you ask.  As far as I know, we refer to C89/C90 (they are essentially
the same for our intents and purposes).

openssl-dev> I believe C89 is written down somewhere.

C89 is written nowhere in the source at least, nor is C90.  We should
probably clarify that.


Speculating a bit, it's probably safe to say that C95 compiler is fine
as well.  C99, not so much, there's too much risk that we start
excluding some platforms if we start using its features.  Anyway, I
don't think it's safe to upgrade our minimum expectations now.
OpenSSL 1.2.0 would be a good time for such re-evaluations.

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Compiler requirements

[Richard Levitte]

In message <CADqLbz+Xm7aJw9zjau1uJ0=qvybssb_cjtaum-c137o6khj...@mail.gmail.com> 
on Tue, 4 Jul 2017 11:57:15 +0300, Dmitry Belyavsky <beld...@gmail.com> said:

beldmit> Hello,
beldmit> 
beldmit> What is the minimal version of the compiler to build openssl?
beldmit> Is it still required C89 compatibility or C99 standard can be used?
beldmit> 
beldmit> Unfortunately, I did not find these requirements in documentation.

At the beginning of INSTALL, you will find a set of requirements.  On
of them is "an ANSI C compiler".

Note that this primary covers language syntax.  We do use libraries
that came later, but then also guard them with a suitable check of
__STDC_VERSION__ or similar, and use or provide alternate
implementations when necessary.

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] 2 snapshots did not generate accordingly

[Richard Levitte]

In message 
<1914c34d74694a768307358d80d4a...@usma1ex-dag1mb1.msg.corp.akamai.com> on Sat, 
22 Apr 2017 12:31:42 +, "Salz, Rich" <rs...@akamai.com> said:

rsalz> > And What happened to openssl 1.0.1 ?
rsalz> 
rsalz> We stopped supporting it back in December.

We were still building snapshots for it, though.  I've adjusted the
script today and removed all present 1.0.1 snapshots (that's the
answer for you, Doc)

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] 2 snapshots did not generate accordingly

[Richard Levitte]

In message <20170422062614.ga23...@doctor.nl2k.ab.ca> on Sat, 22 Apr 2017 
00:26:14 -0600, The Doctor <doc...@doctor.nl2k.ab.ca> said:

doctor> openssl-1.0.2-stable-SNAP-20170422.tar.gz 
doctor> 
doctor> and
doctor> 
doctor> openssl-1.1.0-stable-SNAP-20170422.tar.gz
doctor> 
doctor> did not generate accordingly.
doctor> 
doctor> They are at 0 bytes

Thanks for notifying us.  The disk had filled up, I'm cleaning up.
I'll remove those empty tarballs, there will be new ones tomorrow.

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] rsautl.c incorrectly processes "-oaep" flag

[Richard Levitte]

In message <1f398e96-a7db-4389-94bd-7f1c1af99...@ll.mit.edu> on Thu, 13 Apr 
2017 22:16:49 +, "Blumenthal, Uri - 0553 - MITLL" <u...@ll.mit.edu> said:

uri> Does it mean that rsautl is pretty much deprecated, and pkeyutl superseded 
it? Or is it still worth bringing it “up to snuff”?

In my very personal opinion, I'd say yes, pkeyutl supersedes rsautl.
I don't know if there's any operation of rsautl's that isn't
implemented in pkeyutl (yet?), though.  But yeah, if you ask me, I'd
rather see rsautl abandoned for the benefit of pkeyutl.

Team opinion may differ...  the question of deprecating commands
hasn't actually come up yet.

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] rsautl.c incorrectly processes "-oaep" flag

[Richard Levitte]

In message <006b8116-8aad-18f6-8759-2696ebf38...@gmail.com> on Thu, 13 Apr 2017 
16:41:35 -0500, Douglas E Engert <deeng...@gmail.com> said:

deengert> 
deengert> 
deengert> On 4/13/2017 4:18 PM, Richard Levitte wrote:
deengert> > In message <1ef605ec-d2dd-4d15-a27f-1e1ce7956...@ll.mit.edu> on Thu,
deengert> > 13 Apr 2017 20:55:36 +, "Blumenthal, Uri - 0553 - MITLL"
deengert> > <u...@ll.mit.edu> said:
deengert> >
deengert> > uri> I am trying to use “openssl rsautl” to wrap/unwrap symmetric 
keys
deengert> > in a script. Decryption (and encryption too, but that isn’t 
relevant)
deengert> > is done using a token accessible via pkcs11 engine (libp11).
deengert> > uri>
deengert> > uri> The problem is: “rsautl” appears to assume that if “-oaep” flag
deengert> > is given, then the engine is going to handle OAEP padding. This is 
the
deengert> > screen log:
deengert> > uri>
deengert> > uri> $ openssl rsautl -engine pkcs11 -keyform ENGINE -encrypt -pubin
deengert> > -inkey
deengert> > "pkcs11:manufacturer=piv_II;object=KEY%20MAN%20pubkey;type=public"
deengert> > -oaep -in t256.dat -out t256.dat.enc
deengert> > uri> engine "pkcs11" set.
deengert> > uri> $ ls -l t256.dat.enc
deengert> > uri> -rw-r--r--  1 mouse   256 Apr 10 17:34 t256.dat.enc
deengert> > uri> $ openssl rsautl -engine pkcs11 -keyform ENGINE -decrypt -inkey
deengert> > "pkcs11:manufacturer=piv_II;object=KEY%20MAN%20key;type=private" 
-oaep
deengert> > -in t256.dat.enc -out t256.dat.dec
deengert> > uri> engine "pkcs11" set.
deengert> > uri> PKCS#11 token PIN:
deengert> > uri> PKCS#11: Unsupported padding type
deengert> > uri> RSA operation error
deengert> > uri> $
deengert> > uri>
deengert> > uri> libp11 does not know how to deal with OAEP padding, so it 
returns
deengert> > an error.
deengert> > uri>
deengert> > uri> Desired solution: in case of “-oaep” pass “RSA_NO_PADDING” to 
the
deengert> > engine (aka to libp11), and strip the padding using OpenSSL
deengert> > mechanisms.
deengert> > uri>
deengert> > uri> I’d like to see that fixed in both 1.1 and 1.0.2 branches.
deengert> >
deengert> > Wouldn't it be much easier to add the following lines:
deengert> >
deengert> > case RSA_PKCS1_OAEP_PADDING:
deengert> > mechanism->mechanism = CKM_RSA_PKCS_OAEP;
deengert> > break;
deengert> >
deengert> > right about here?
deengert> > https://github.com/OpenSC/libp11/blob/master/src/p11_rsa.c#L72
deengert> >
deengert> > What you propose for OpenSSL is quite a lot harder to implement 
well,
deengert> > and one might also wonder why the OAEP padding should have that
deengert> > special treatment and no other?
deengert> >
deengert> 
deengert> Because there are parameters to the OAEP, and rsautl.c does not set
deengert> it.
deengert> 
deengert> when not using an engine, rsa/rsa_pmeth.c in pkey_rsa_decrypt does
deengert> something similar:
deengert> 
deengert> 300 if (rctx->pad_mode == RSA_PKCS1_OAEP_PADDING) {
deengert> 
deengert> 304 ret = RSA_private_decrypt(inlen, in, rctx->tbuf,
deengert> 305 ctx->pkey->pkey.rsa, RSA_NO_PADDING);
deengert> 
deengert> 312 ret = RSA_padding_check_PKCS1_OAEP_mgf1(out, ret, rctx->tbuf + i,
deengert> 313 ret - i, ret,
deengert> 314 rctx->oaep_label,
deengert> 315 rctx->oaep_labellen,
deengert> 316 rctx->md, rctx->mgf1md);

Good point.  But then, rsautl is a poor choice, as it uses the RSA
API.  For something more general and with a whole lot more
functionality, pkeyutl is the better choice.

Incidently, for decryption, it will end up calling exactly the code
you're citing, and with -pkeyopt, you can specify the padding mode and
its necessary data.

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] rsautl.c incorrectly processes "-oaep" flag

[Richard Levitte]

In message <1ef605ec-d2dd-4d15-a27f-1e1ce7956...@ll.mit.edu> on Thu, 13 Apr 
2017 20:55:36 +, "Blumenthal, Uri - 0553 - MITLL" <u...@ll.mit.edu> said:

uri> I am trying to use “openssl rsautl” to wrap/unwrap symmetric keys in a 
script. Decryption (and encryption too, but that isn’t relevant) is done using 
a token accessible via pkcs11 engine (libp11).
uri> 
uri> The problem is: “rsautl” appears to assume that if “-oaep” flag is given, 
then the engine is going to handle OAEP padding. This is the screen log:
uri> 
uri> $ openssl rsautl -engine pkcs11 -keyform ENGINE -encrypt -pubin -inkey 
"pkcs11:manufacturer=piv_II;object=KEY%20MAN%20pubkey;type=public" -oaep -in 
t256.dat -out t256.dat.enc
uri> engine "pkcs11" set.
uri> $ ls -l t256.dat.enc 
uri> -rw-r--r--  1 mouse   256 Apr 10 17:34 t256.dat.enc
uri> $ openssl rsautl -engine pkcs11 -keyform ENGINE -decrypt -inkey 
"pkcs11:manufacturer=piv_II;object=KEY%20MAN%20key;type=private" -oaep -in 
t256.dat.enc -out t256.dat.dec
uri> engine "pkcs11" set.
uri> PKCS#11 token PIN: 
uri> PKCS#11: Unsupported padding type
uri> RSA operation error
uri> $
uri> 
uri> libp11 does not know how to deal with OAEP padding, so it returns an error.
uri> 
uri> Desired solution: in case of “-oaep” pass “RSA_NO_PADDING” to the engine 
(aka to libp11), and strip the padding using OpenSSL mechanisms.
uri> 
uri> I’d like to see that fixed in both 1.1 and 1.0.2 branches.

Wouldn't it be much easier to add the following lines:

case RSA_PKCS1_OAEP_PADDING:
mechanism->mechanism = CKM_RSA_PKCS_OAEP;
break;

right about here?  
https://github.com/OpenSC/libp11/blob/master/src/p11_rsa.c#L72

What you propose for OpenSSL is quite a lot harder to implement well,
and one might also wonder why the OAEP padding should have that
special treatment and no other?

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Memory leak in application when we use ECDH

[Richard Levitte]

 3708 2740 S 11.3  0.1   0:03.10 
openssl
darshanmody> 
darshanmody> 
darshanmody> top - 10:17:13 up 46 days,  1:30,  2 users,  load average: 0.02, 
0.06, 0.07
darshanmody> Tasks:   1 total,   0 running,   1 sleeping,   0 stopped,   0 
zombie
darshanmody> Cpu(s):  3.6%us,  1.1%sy,  0.0%ni, 94.5%id,  0.0%wa,  0.0%hi,  
0.8%si,  0.0%st
darshanmody> Mem:   3924288k total,  3758476k used,   165812k free,78376k 
buffers
darshanmody> Swap:  8388604k total,   999556k used,  7389048k free,   219920k 
cached
darshanmody> 
darshanmody>   PID USER  PR  NI  VIRT  RES  SHR S %CPU %MEMTIME+  
COMMAND
darshanmody> 27303 root  20   0 40016 3728 2740 S 10.7  0.1   0:03.42 
openssl
darshanmody> 
darshanmody> 
darshanmody> top - 10:17:16 up 46 days,  1:30,  2 users,  load average: 0.02, 
0.06, 0.07
darshanmody> Tasks:   1 total,   0 running,   1 sleeping,   0 stopped,   0 
zombie
darshanmody> Cpu(s):  3.8%us,  1.3%sy,  0.0%ni, 93.3%id,  0.0%wa,  0.0%hi,  
1.7%si,  0.0%st
darshanmody> Mem:   3924288k total,  3759824k used,   164464k free,78384k 
buffers
darshanmody> Swap:  8388604k total,   999556k used,  7389048k free,   219920k 
cached
darshanmody> 
darshanmody>   PID USER  PR  NI  VIRT  RES  SHR S %CPU %MEMTIME+  
COMMAND
darshanmody> 27303 root  20   0 40124 3748 2740 S 11.3  0.1   0:03.76 
openssl
darshanmody> 
darshanmody> ---
darshanmody> 
darshanmody> top - 10:23:10 up 46 days,  1:36,  2 users,  load average: 0.03, 
0.05, 0.06
darshanmody> Tasks:   1 total,   0 running,   1 sleeping,   0 stopped,   0 
zombie
darshanmody> Cpu(s):  3.6%us,  1.3%sy,  0.1%ni, 94.4%id,  0.0%wa,  0.0%hi,  
0.6%si,  0.0%st
darshanmody> Mem:   3924288k total,  3756048k used,   168240k free,75888k 
buffers
darshanmody> Swap:  8388604k total,   998972k used,  7389632k free,   216576k 
cached
darshanmody> 
darshanmody>   PID USER  PR  NI  VIRT  RES  SHR S %CPU %MEMTIME+  
COMMAND
darshanmody> 27303 root  20   0 42392 6048 2740 S 10.7  0.2   0:41.95 
openssl
darshanmody> 
darshanmody> 
darshanmody> top - 10:23:13 up 46 days,  1:36,  2 users,  load average: 0.02, 
0.04, 0.06
darshanmody> Tasks:   1 total,   0 running,   1 sleeping,   0 stopped,   0 
zombie
darshanmody> Cpu(s):  3.2%us,  1.3%sy,  0.0%ni, 94.8%id,  0.0%wa,  0.0%hi,  
0.6%si,  0.0%st
darshanmody> Mem:   3924288k total,  3756064k used,   168224k free,75904k 
buffers
darshanmody> Swap:  8388604k total,   998972k used,  7389632k free,   216572k 
cached
darshanmody> 
darshanmody>   PID USER  PR  NI  VIRT  RES  SHR S %CPU %MEMTIME+  
COMMAND
darshanmody> 27303 root  20   0 42392 6068 2740 S 10.7  0.2   0:42.27 
openssl
darshanmody> 
darshanmody> 
darshanmody> top - 10:23:16 up 46 days,  1:36,  2 users,  load average: 0.02, 
0.04, 0.06
darshanmody> Tasks:   1 total,   1 running,   0 sleeping,   0 stopped,   0 
zombie
darshanmody> Cpu(s):  3.2%us,  1.3%sy,  0.1%ni, 94.9%id,  0.0%wa,  0.0%hi,  
0.4%si,  0.0%st
darshanmody> Mem:   3924288k total,  3756188k used,   168100k free,75912k 
buffers
darshanmody> Swap:  8388604k total,   998972k used,  7389632k free,   216572k 
cached
darshanmody> 
darshanmody>   PID USER  PR  NI  VIRT  RES  SHR S %CPU %MEMTIME+  
COMMAND
darshanmody> 27303 root  20   0 42392 6088 2740 R 11.0  0.2   0:42.60 
openssl
darshanmody> 
darshanmody> 
darshanmody> top - 10:23:19 up 46 days,  1:36,  2 users,  load average: 0.02, 
0.04, 0.06
darshanmody> Tasks:   1 total,   0 running,   1 sleeping,   0 stopped,   0 
zombie
darshanmody> Cpu(s):  3.1%us,  1.2%sy,  1.6%ni, 93.5%id,  0.0%wa,  0.0%hi,  
0.6%si,  0.0%st
darshanmody> Mem:   3924288k total,  3756528k used,   167760k free,75920k 
buffers
darshanmody> Swap:  8388604k total,   998972k used,  7389632k free,   216584k 
cached
darshanmody> 
darshanmody>   PID USER  PR  NI  VIRT  RES  SHR S %CPU %MEMTIME+  
COMMAND
darshanmody> 27303 root  20   0 42392 6108 2740 S 10.7  0.2   0:42.92 
openssl
darshanmody> 
darshanmody> 
darshanmody> top - 10:23:22 up 46 days,  1:36,  2 users,  load average: 0.02, 
0.04, 0.06
darshanmody> Tasks:   1 total,   0 running,   1 sleeping,   0 stopped,   0 
zombie
darshanmody> Cpu(s): 11.1%us,  4.6%sy, 20.2%ni, 63.6%id,  0.0%wa,  0.0%hi,  
0.6%si,  0.0%st
darshanmody> Mem:   3924288k total,  3756924k used,   167364k free,75952k 
buffers
darshanmody> Swap:  8388604k total,   998972k used,  7389632k free,   216600k 
cached
darshanmody> 
darshanmody>   PID USER  PR  NI  VIRT  RES  SHR S %CPU %MEMTIME+  
COMMAND
darshanmody> 27303 root  20   0 42500 6124 2740 S 10.3  0.2   0:43.23 
openssl
darshanmody> 
darshanmody> Thanks
darshanmody> Darshan
darshanmody> 
darshanmody> -Original Message-
darshanmody> From: openssl-dev [mailto:openssl-dev-boun...@openssl.

Re: [openssl-dev] License change agreement

[Richard Levitte]

In message <20170324121435.gq70...@colo.drijf.net> on Fri, 24 Mar 2017 13:14:35 
+0100, Otto Moerbeek <o...@drijf.net> said:

otto> On Fri, Mar 24, 2017 at 11:53:10AM +, Blumenthal, Uri - 0553 - MITLL 
wrote:
otto> 
otto> > I personally think this issue is being blown way out of proportion and 
beyond the boundary of reason. 
otto> > 
otto> > Regards,
otto> > Uri
otto> 
otto> Is it reasonable to step on the rights of authors with the backing of
otto> large corporations? Individual authors that might have chosen to
otto> change email address or are unable to be contacted for other reasons?
otto> 
otto> It is sad to see the corporate giants dictate the policies of yet
otto> another open source project, without regard for the spirit of
otto> copyright law which is to protect the individual author.

If I'm reading you correctly, *any* license change faces the exact
same problem.  My interpretation of what you say is that unless we can
successfully reach all contributors, no exception, we're stuck with
the current license, probably for life.

Am I reading you correctly?

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Memory leak in application when we use ECDH

[Richard Levitte]

I think that Matt is asking for example code that exhibits this leak.
You could patch apps/s_server.c with your callback, or ssl/ssltest.c,
and give us that patch.

The reason is that we can't know what assumptions you're going with in
your callback or application, so if we code an example together, it
will be with Our conditions, not yours, and therefore a pretty bad
method to figure this out.

Cheers,
Richard

In message 
<25d2ec755404b4409f263ac6d050febb2a107...@az-ffexmb03.global.avaya.com> on Thu, 
23 Mar 2017 13:47:10 +, "Mody, Darshan (Darshan)"  
said:

darshanmody> Matt,
darshanmody> 
darshanmody> Below is the scenario.
darshanmody> 
darshanmody> 1. Have server open a listen socket which always validates the 
client certificate and chain.
darshanmody> 2. On server support ECDHE using callback. Ensure the EC_KEY 
passed to openssl from app is cleaned up by the app.
darshanmody> 3. Connect client with certificates that server does not trust.
darshanmody> 4. The connections from client to server fails
darshanmody> 
darshanmody> In course of time the app running the server has been leaking. 
Even after accounting for the EC_KEY passed by the server app to openssl we 
find there seems to be leak. Further investigation on the core dumps generated 
from the server app shows that it has the certificates from the client saved.
darshanmody> 
darshanmody> Hope this helps
darshanmody> 
darshanmody> Thanks
darshanmody> Darshan 
darshanmody> 
darshanmody> -Original Message-
darshanmody> From: openssl-dev [mailto:openssl-dev-boun...@openssl.org] On 
Behalf Of Matt Caswell
darshanmody> Sent: Thursday, March 23, 2017 6:55 PM
darshanmody> To: openssl-dev@openssl.org
darshanmody> Subject: Re: [openssl-dev] Memory leak in application when we use 
ECDH
darshanmody> 
darshanmody> 
darshanmody> 
darshanmody> On 23/03/17 13:19, Mody, Darshan (Darshan) wrote:
darshanmody> > Can you further elaborate?
darshanmody> > 
darshanmody> > What we did is to create a TLS connection and with invalid 
darshanmody> > certificates from the client and server on verification would 
reject 
darshanmody> > the certificate. The cipher negotiated was ECDHE cipher between 
client 
darshanmody> > and server.
darshanmody> > 
darshanmody> > This was done with load (multiple while 1 script trying to 
connect to 
darshanmody> > server using invalid certificates and in course of time the 
memory was 
darshanmody> > increasing).
darshanmody> 
darshanmody> Without being able to recreate the problem its going to be very 
difficult/impossible for us to fix it (assuming the problem is in OpenSSl 
itself). We would need some simple reproducer code that demonstrates the 
problem occurring.
darshanmody> 
darshanmody> Matt
darshanmody> 
darshanmody> 
darshanmody> > 
darshanmody> > Thanks Darshan
darshanmody> > 
darshanmody> > -Original Message- From: openssl-dev 
darshanmody> > [mailto:openssl-dev-boun...@openssl.org] On Behalf Of Matt 
Caswell
darshanmody> > Sent: Thursday, March 23, 2017 4:09 PM To: 
openssl-dev@openssl.org
darshanmody> > Subject: Re: [openssl-dev] Memory leak in application when we 
use ECDH
darshanmody> > 
darshanmody> > 
darshanmody> > 
darshanmody> > On 23/03/17 10:13, Mody, Darshan (Darshan) wrote:
darshanmody> >> Matt,
darshanmody> >> 
darshanmody> >> Even after accounting for the EC_KEY we still observe some leak.
darshanmody> >> The leak started after we started using supporting EC with
darshanmody> >> callback SSL_set_tmp_ecdh_callback().
darshanmody> >> 
darshanmody> >> The core dump shows  the string data of the far-end 
certificates.
darshanmody> >> I cannot pin point  the code in openssl with this regard.
darshanmody> > 
darshanmody> > Are you able to create a simple reproducer demonstrating the 
problem 
darshanmody> > with the callback?
darshanmody> > 
darshanmody> > Matt
darshanmody> > 
darshanmody> -- 
darshanmody> openssl-dev mailing list
darshanmody> To unsubscribe: 
https://urldefense.proofpoint.com/v2/url?u=https-3A__mta.openssl.org_mailman_listinfo_openssl-2Ddev=DwICAg=BFpWQw8bsuKpl1SgiZH64Q=bsEULbVnjelD7InzgsegHBEbtXzaIDagy9EuEhJrKfQ=VbrRgO8PZIVkFM4PjeK7TEgKDHnbLu_QfbyqRhmvx8I=u0cR7sQf_Zz8FoCnrzgLc3drBSR8Ou1qDUyxV8z1xYQ=
 
darshanmody> -- 
darshanmody> openssl-dev mailing list
darshanmody> To unsubscribe: 
https://mta.openssl.org/mailman/listinfo/openssl-dev
darshanmody> 
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] please make clear on website that 1.1.0e is Development release, not GA / Production release

[Richard Levitte]

In message <calyzvky_y0ewaupzbwsyrq2k+onzyfran1t8c7upox5_0jp...@mail.gmail.com> 
on Tue, 21 Mar 2017 00:13:57 +, Jason Vas Dias <jason.vas.d...@gmail.com> 
said:

jason.vas.dias> On 20/03/2017, Kurt Roeckx <k...@roeckx.be> wrote:
jason.vas.dias> > The ed25519 support in openssh doesn't even come from openssl.
jason.vas.dias> >
jason.vas.dias> What happens is OpenSSH's cipher.c calls
jason.vas.dias>if (EVP_CipherInit(cc->evp, type, NULL, (u_char *)iv,
jason.vas.dias>   (do_encrypt == CIPHER_ENCRYPT)) == 0) {
jason.vas.dias> ret = SSH_ERR_LIBCRYPTO_ERROR;
jason.vas.dias> goto out;
jason.vas.dias> }
jason.vas.dias> which always does 'goto out' for any ED25519 file.

That would happen if ssh_host_ed25519_key is password protected and
the cipher used to encrypt the key isn't recognised in OpenSSL 1.1.0
(and considering the current master of openssh-portable doesn't build
cleanly against OpenSSL 1.1.0e and I therefore suppose you've hacked
around, I can't even begin to say where the fault came in).  It also
depends on your OpenSSL configuration, since you can disable most
algorithms it carries...

jason.vas.dias> >> which mainly
jason.vas.dias> >> involved including the '*_lo?cl.h' & '*_int.h'  headers
jason.vas.dias> >
jason.vas.dias> > Including the internal headers is not a good patch. This will
jason.vas.dias> > break.
jason.vas.dias> >
jason.vas.dias> 
jason.vas.dias> It doesn't break at all - the code remains 100% unchanged  - 
just different
jason.vas.dias> headers need including - and seems to work fine including the 
API
jason.vas.dias> hiding headers.

The structures you find in there are made private for a reason, we
need the liberty to make changes in them in future developments
without disturbing the ABI (not just the API).  So some time in the
future, it will break.

jason.vas.dias> And my point is really not to criticize your effort, it is just 
a plea to make
jason.vas.dias> clear on the web-page that the 1.1.0 branch is a development 
branch and
jason.vas.dias> does not work yet with most OpenSSL using applications .

It isn't a development branch.  We see it as a stable release, i.e. no
further development apart from bug fixes.  "master" is the development
branch.

jason.vas.dias> OpenSSL in its 1.0.2 incarnation has been hardened by over 
(10,15,20)? years
jason.vas.dias> of testing , and its API is usable by all OpenSSL using 
applications,
jason.vas.dias> unlike 1.1.0 .

Jyst to put things in perspective, OpenSSL 1.0.0 was released
2010-Mar-29.  That was the start of the 1.0.x series.  OpenSSL 1.0.2
was released 2015-Jan-22.

OpenSSL 1.1.0 marks the start of the 1.1.x series, which isn't source
compatible with the 1.0.x series.  We have talked about this in
different ways even before the first Alpha release was made (over a
year ago).

Either way, the 1.0.2 branch is supported until the end of 2019.
One could say that's how long other application authors have to rework
their source, although that's not really true since anyone can keep
the 1.0.2 source around as long as they want (hey, even we do).

Maybe you expected all applications to have converted the moment we
declared our 1.1.0 release stable?  That will not happen...  as far as
we've observed, most are hardly even looking before we've made a
stable release (which I agree is unfortunate).

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Openssl 1.0.2 snap STABLE 20170311 issue

[Richard Levitte]

Fixed:

commit 6fe43af8d77b119f8af913c284149bca482ee58c
Author: Richard Levitte <levi...@openssl.org>
Date:   Sat Mar 11 11:19:20 2017 +0100

Revert "Use the callbacks from the SSL object instead of the SSL_CTX 
object"

This shouldn't have been applied to the 1.0.2 branch.

This reverts commit 5247c0388610bfdcc8f44b777d75ab681120753d.

Reviewed-by: Tim Hudson <t...@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2907)

Cheers,
Richard

In message <20170311063806.ga39...@doctor.nl2k.ab.ca> on Fri, 10 Mar 2017 
23:38:06 -0700, The Doctor <doc...@doctor.nl2k.ab.ca> said:

doctor> 
doctor> Script started on Fri Mar 10 23:31:39 2017
doctor> You have mail.
doctor> root@doctor:/usr/source/openssl-1.0.2-stable-SNAP-20170311 # make
doctor> 
doctor> making all in crypto...
doctor> making all in crypto/objects...
doctor> making all in crypto/md4...
doctor> making all in crypto/md5...
doctor> making all in crypto/sha...
doctor> making all in crypto/mdc2...
doctor> making all in crypto/hmac...
doctor> making all in crypto/ripemd...
doctor> making all in crypto/whrlpool...
doctor> making all in crypto/des...
doctor> making all in crypto/aes...
doctor> making all in crypto/rc2...
doctor> making all in crypto/rc4...
doctor> making all in crypto/idea...
doctor> making all in crypto/bf...
doctor> making all in crypto/cast...
doctor> making all in crypto/camellia...
doctor> making all in crypto/seed...
doctor> making all in crypto/modes...
doctor> making all in crypto/bn...
doctor> making all in crypto/ec...
doctor> making all in crypto/rsa...
doctor> making all in crypto/dsa...
doctor> making all in crypto/ecdsa...
doctor> making all in crypto/dh...
doctor> making all in crypto/ecdh...
doctor> making all in crypto/dso...
doctor> making all in crypto/engine...
doctor> making all in crypto/buffer...
doctor> making all in crypto/bio...
doctor> making all in crypto/stack...
doctor> making all in crypto/lhash...
doctor> making all in crypto/rand...
doctor> making all in crypto/err...
doctor> making all in crypto/evp...
doctor> making all in crypto/asn1...
doctor> making all in crypto/pem...
doctor> making all in crypto/x509...
doctor> making all in crypto/x509v3...
doctor> making all in crypto/conf...
doctor> making all in crypto/txt_db...
doctor> making all in crypto/pkcs7...
doctor> making all in crypto/pkcs12...
doctor> making all in crypto/comp...
doctor> making all in crypto/ocsp...
doctor> making all in crypto/ui...
doctor> making all in crypto/krb5...
doctor> making all in crypto/cms...
doctor> making all in crypto/pqueue...
doctor> making all in crypto/ts...
doctor> making all in crypto/jpake...
doctor> making all in crypto/srp...
doctor> making all in crypto/store...
doctor> making all in crypto/cmac...
doctor> if [ -n "libcrypto.so.1.0.0 libssl.so.1.0.0" ]; then  (cd ..; make 
libcrypto.so.1.0.0);  fi
doctor> `libcrypto.so.1.0.0' is up to date.
doctor> making all in engines...
doctor> echo 
doctor> 
doctor> making all in engines/ccgost...
doctor> making all in ssl...
doctor> /usr/local/bin/clang39 -I../crypto -I.. -I../include  -fPIC 
-DOPENSSL_PIC -DZLIB_SHARED -DZLIB -DOPENSSL_THREADS -pthread -D_THREAD_SAFE 
-D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DL_ENDIAN -O3 -Wall 
-DOPENSSL_EXPERIMENTAL_JPAKE -DOPENSSL_EXPERIMENTAL_STORE -DOPENSSL_IA32_SSE2 
-DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m 
-I/usr/local/ssl/fips-2.0/include -DRC4_ASM -DSHA1_ASM -DSHA256_ASM 
-DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM 
-DGHASH_ASM -DECP_NISTZ256_ASM -c ssl_rsa.c -o ssl_rsa.o
doctor> ssl_rsa.c:105:46: error: no member named 'default_passwd_callback' in
doctor>   'struct ssl_st'
doctor> x = PEM_read_bio_X509(in, NULL, ssl->default_passwd_callback,
doctor> ~~~  ^
doctor> ssl_rsa.c:106:36: error: no member named 
'default_passwd_callback_userdata' in
doctor>   'struct ssl_st'
doctor>   ssl->default_passwd_callback_userdata);
doctor>   ~~~  ^
doctor> ssl_rsa.c:264:47: error: no member named 'default_passwd_callback' in
doctor>   'struct ssl_st'
doctor>  ssl->default_passwd_callback,
doctor>  ~~~  ^
doctor> ssl_rsa.c:265:47: error: no member named 
'default_passwd_callback_userdata' in
doctor>   'struct ssl_st'
doctor>  
ssl->default_passwd_callback_userdata);
doctor>  ~~~  ^
doctor> ssl_rsa.c:337:45:

Re: [openssl-dev] [Bug, maybe] [master] bin/* no longer find their libraries if installed in non-default locations

[Richard Levitte]

In message <20170301221703.tfwpu%stef...@sdaoden.eu> on Wed, 01 Mar 2017 
23:17:03 +0100, Steffen Nurpmeso <stef...@sdaoden.eu> said:

steffen> Yes, i mean, i just didn't know this, it is not mentioned anywhere
steffen> (i think that would well be worth in entry in INSTALL), and
steffen> i really would have sworn that it worked in the past.

I was actually surprised to find this undocumented.  I could have
sworn I'd done so, but apparently, I only did in a commit message:

commit fad599f7f147ee71e5581211fb654c2c8c491cd8
Author: Richard Levitte <levi...@openssl.org>
Date:   Wed Oct 12 17:05:35 2016 +0200

Remove automatic RPATH - add user rpath support

Make Configure recognise -rpath and -R to support user added rpaths
for OSF1 and Solaris.  For convenience, add a variable LIBRPATH in the
Unix Makefile, which the users can use as follows:

./config [options] -Wl,-rpath,\$(LIBRPATH)

Reviewed-by: Rich Salz <rs...@openssl.org>

I'll go add a note now...

Sorry

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [Bug, maybe] [master] bin/* no longer find their libraries if installed in non-default locations

[Richard Levitte]

In message <20170301165032.8jhwg%stef...@sdaoden.eu> on Wed, 01 Mar 2017 
17:50:32 +0100, Steffen Nurpmeso <stef...@sdaoden.eu> said:

steffen> "Salz, Rich" <rs...@akamai.com> wrote:
steffen>  |> This is new behaviour, until now the installation was always 
self-contain\
steffen>  |> ed
steffen>  |> when configured via
steffen>  |> 
steffen>  |>   ./config --prefix=$(MYPREFIX) zlib-dynamic no-hw shared
steffen>  |
steffen>  |Did you install the libraries in a standard place?
steffen>  |
steffen>  |> I think this should at least be noted in CHANGES or so.
steffen>  |
steffen>  |I don't think so.  I think the libs weren't installed.
steffen> 
steffen> Yes, also in my opinion the old behaviour was much, much better.

I very much disagree.  We have had bug reports as well as cases of our
own because a new compilation that you want to test picked up
previously installed versions of the libraries (usually an older
version).  The reason for doing so previously was because we installed
the libraries in non-standard locations by default.

Since OpenSSL 1.1.0 and on is installing in standard locations by
default, we don't have to use these mechanisms for a default build.
With that, we realised that choosing to use DT_RPATH, DT_RUNPATH (they
are different) or whatever isn't really our decision to make, but the
decision of the packager or the individual user, so we've handed the
decision to you. 

For the GNU toolchain, I'd recommend configuring with something like
this (from memory, I might be fuzzy in the details):

-Wl,--enable-new-dtags -rpath '$(LIBRPATH)'

LIBRPATH is a convenience Makefile variable that gets correctly set to
the configured shared library installation directory, meant for
exactly this sort of situation.

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Participate in Code Health Tuesday (tomorrow, Feb 28th)

[Richard Levitte]

I'd suggest prefixing the PR subject with "code-health:" or
"[code-health]", just like work in progress is prefixed "WIP:" or
"[WIP]"

Cheers,
Richard

In message <9ecbf19a-3239-440c-b874-b959b6bb9...@akamai.com> on Mon, 27 Feb 
2017 14:54:09 +, "Short, Todd"  said:

tshort> I’m not sure us mere mortals can add a label to a PR...
tshort> --
tshort> -Todd Short
tshort> // tsh...@akamai.com
tshort> // "One if by land, two if by sea, three if by the Internet."
tshort> 
tshort> On Feb 27, 2017, at 5:04 AM, Emilia Käsper 
tshort> wrote:
tshort> 
tshort> 
tshort> 
tshort> 
tshort> Hi OpenSSL developers!
tshort> 
tshort> We’re always looking for ways to improve code quality and pay our
tshort> technical debt. This week we thought we’d run a little experiment.
tshort> 
tshort> We declare this Tuesday (Feb 28th) Code Health Tuesday. We’ll be
tshort> setting some time aside to do cleanups in the codebase. The theme
tshort> is “Delete”: we’ll be cleaning up unused files, dead code, and
tshort> obsolete hacks. We invite you all to participate on Github!
tshort> 
tshort> 
tshort> Cheers,
tshort> Emilia
tshort> 
tshort> FAQ:
tshort> 
tshort> Q: How do I participate?
tshort> A: Find something to delete. Create a Github pull request and add
tshort> the “code-health” label. We’ll be monitoring Github for quick
tshort> turnaround.
tshort> 
tshort> Q: Which branches should I target?
tshort> A: You should target master. In stable branches, code churn comes
tshort> with a cost, so let’s focus on the next release.
tshort> 
tshort> Q: What can I delete?
tshort> A: Normal compatibility rules apply. You cannot delete anything
tshort> from public headers, remove command-line tool options or prune
tshort> supported platform configurations. You can delete dead code,
tshort> obsolete workarounds (16-bit platforms!) and outdated
tshort> documentation. If you’re not sure about a particular
tshort> functionality, open a Github issue and add the “code health”
tshort> label.
tshort> 
tshort> Q: Do you have any tools to find what to delete?
tshort> A: We have a coverage report:
tshort> https://coveralls.io/github/openssl/openssl
tshort> We’ll also be setting up a tools repo where you can share any
tshort> tools that you build.
tshort> 
tshort> Q: Will you do it again?
tshort> A: We hope so! This is an experiment but we’ll be looking into
tshort> making it a habit. We have a list of ideas for themed Tuesdays
tshort> lined up: Document, Test, Refactor, ...
tshort> 
tshort> Q: How did you come up with this idea?
tshort> A: We were looking at this file…
tshort> 
https://github.com/openssl/openssl/blob/master/crypto/pkcs7/pk7_dgst.c
tshort> 
tshort> --
tshort> openssl-dev mailing list
tshort> To unsubscribe:
tshort> https://mta.openssl.org/mailman/listinfo/openssl-dev
tshort> 
tshort> 
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [openssl/openssl] ABI compatibility 1.0.0-->1.0.1-->1.0.2

[Richard Levitte]

In message <a2e37ebc-183e-023d-f059-7b89d6e3a...@akamai.com> on Fri, 27 Jan 
2017 10:54:35 -0600, Benjamin Kaduk via openssl-dev <openssl-dev@openssl.org> 
said:

openssl-dev> There was some discussion about 1.0.1 being EoL on a FreeBSD list 
[0],
openssl-dev> and whether it would make sense to move to 1.0.2 on their stable
openssl-dev> branch, which led to someone making the claim that 1.0.2 has 
removed 4
openssl-dev> symbols compared to 1.0.1, and thus is not strictly ABI compatible,
openssl-dev> linking to https://abi-laboratory.pro/tracker/timeline/openssl/ . 
If I
openssl-dev> start semi-randomly clicking around, I can find a page [1] that 
seems
openssl-dev> to claim the missing symbols are:
openssl-dev> ASN1_STRING_clear_free()
openssl-dev> ENGINE_load_rsax()
openssl-dev> SRP_user_pwd_free()
openssl-dev> SRP_VBASE_get1_by_user()

I haven't make a complete analysis over versions, just did a
comparison of the files that define what we regard as public symbols
(util/libeay.num and util/libssl.num) in the latest 1.0.1 and 1.0.2
releases.  Diffs attached.

As you can see, ENGINE_load_rsax *did* go away.  That happened here:

commit 74184b6f21e095dacd6193a78785a47dd515f0dc
Author: Dr. Stephen Henson <st...@openssl.org>
Date:   Sun Dec 1 23:06:33 2013 +

RSAX no longer compiled.

I'm afraid I can't remember the reasoning behind this commit...

The rest of those mentioned above haven't moved at all as far as I can
see.  You may notice that some of the symbols in libssl (ssleay.num)
did move between "modules" (which in this case can be defined as a
keyword you can say no to when configuring).  I'm unsure how that
affects your view on our stability, suffice to say that with default
configuration, it doesn't affect the ABI one bit.

FYI, here's how the diffs were produced:

: ; (LANG=C; (cd ../1.0.1; pwd; git status); pwd; git status; diff -u 
../1.0.1/util/libeay.num util/libeay.num > /tmp/libeay.num.diff)
/home/levitte/gitwrk/openssl.net/official/1.0.1
HEAD detached at OpenSSL_1_0_1u
nothing to commit, working tree clean
/home/levitte/gitwrk/openssl.net/official/1.0.2
HEAD detached at OpenSSL_1_0_2k
nothing to commit, working tree clean
: ; (LANG=C; (cd ../1.0.1; pwd; git status); pwd; git status; diff -u 
../1.0.1/util/ssleay.num util/ssleay.num > /tmp/ssleay.num.diff)
/home/levitte/gitwrk/openssl.net/official/1.0.1
HEAD detached at OpenSSL_1_0_1u
nothing to commit, working tree clean
/home/levitte/gitwrk/openssl.net/official/1.0.2
HEAD detached at OpenSSL_1_0_2k
nothing to commit, working tree clean

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
--- ../1.0.1/util/libeay.num	2016-06-30 01:06:40.0 +0200
+++ util/libeay.num	2017-01-26 11:46:30.640419220 +0100
@@ -4284,7 +4284,7 @@
 CRYPTO_ccm128_aad   4649	EXIST::FUNCTION:
 CRYPTO_gcm128_init  4650	EXIST::FUNCTION:
 CRYPTO_gcm128_decrypt   4651	EXIST::FUNCTION:
-ENGINE_load_rsax4652	EXIST::FUNCTION:ENGINE
+ENGINE_load_rsax4652	NOEXIST::FUNCTION:
 CRYPTO_gcm128_decrypt_ctr32 4653	EXIST::FUNCTION:
 CRYPTO_gcm128_encrypt_ctr32 4654	EXIST::FUNCTION:
 CRYPTO_gcm128_finish4655	EXIST::FUNCTION:
@@ -4316,3 +4316,103 @@
 BIO_s_datagram_sctp 4680	EXIST::FUNCTION:DGRAM,SCTP
 BIO_dgram_is_sctp   4681	EXIST::FUNCTION:SCTP
 BIO_dgram_sctp_notification_cb  4682	EXIST::FUNCTION:SCTP
+i2d_DHxparams   4683	EXIST::FUNCTION:DH
+EC_curve_nist2nid   4684	EXIST::FUNCTION:EC
+DH_get_1024_160 4685	EXIST::FUNCTION:DH
+PEM_write_DHxparams 4686	EXIST:!WIN16:FUNCTION:DH
+d2i_DHxparams   4687	EXIST::FUNCTION:DH
+EC_curve_nid2nist   4688	EXIST::FUNCTION:EC
+DH_get_2048_256 4689	EXIST::FUNCTION:DH
+PEM_write_bio_DHxparams 4690	EXIST::FUNCTION:DH
+DH_get_2048_224 4691	EXIST::FUNCTION:DH
+X509_chain_check_suiteb 4692	EXIST::FUNCTION:
+X509_chain_up_ref   4693	EXIST::FUNCTION:
+X509_VERIFY_PARAM_set1_ip_asc   4694	EXIST::FUNCTION:
+X509_CRL_check_suiteb   4695	EXIST::FUNCTION:
+X509_VERIFY_PARAM_set1_email4696	EXIST::FUNCTION:
+X509_check_email4697	EXIST::FUNCTION:
+X509_check_host 4698	EXIST::FUNCTION:
+X509_check_ip_asc   4699	EXIST::FUNCTION:
+X509_get0_signature 4700	EXIST::FUNCTION:
+X509_get_signature_nid  4701	EXIST::FUNCTION:
+X509_VERIFY_PARAM_set1_host 4702	EXIST::FUNC

Re: [openssl-dev] Openssl 1.0.2 snapshot bug

[Richard Levitte]

Yup, we have a fix coming up:

https://github.com/openssl/openssl/pull/2713

In message <20170223125425.ga77...@doctor.nl2k.ab.ca> on Thu, 23 Feb 2017 
05:54:25 -0700, The Doctor  said:

doctor> 
doctor> Script started on Thu Feb 23 05:41:55 2017
doctor> You have mail.
doctor> root@doctor:/usr/source/openssl-1.0.2-stable-SNAP-20170223 # 
makeless /usr/contrib/b in/configopenssl
doctor> 
doctor> [?1h=
doctor> #!/usr/local/bin/bash
doctor> CC=/usr/local/bin/clang39  ./Configure --prefix=/usr/ BSD-x86_64 fips 
enable-gmp  experimental-jpake enable-rfc3779 enable-shared zlib-dynamic 
disable-sctp exper imental-store enable-ssl-trace enable-unit-test; make depend
doctor> /usr/contrib/bin/configopenssl (END)
doctor> [?1l>root@doctor:/usr/source/openssl-1.0.2-stable-SNAP-20170223 # 
make
doctor> 
doctor> making all in crypto...
doctor> making all in crypto/objects...
doctor> making all in crypto/md4...
doctor> making all in crypto/md5...
doctor> making all in crypto/sha...
doctor> making all in crypto/mdc2...
doctor> making all in crypto/hmac...
doctor> making all in crypto/ripemd...
doctor> making all in crypto/whrlpool...
doctor> making all in crypto/des...
doctor> making all in crypto/aes...
doctor> making all in crypto/rc2...
doctor> making all in crypto/rc4...
doctor> making all in crypto/idea...
doctor> making all in crypto/bf...
doctor> making all in crypto/cast...
doctor> making all in crypto/camellia...
doctor> making all in crypto/seed...
doctor> making all in crypto/modes...
doctor> making all in crypto/bn...
doctor> making all in crypto/ec...
doctor> making all in crypto/rsa...
doctor> making all in crypto/dsa...
doctor> making all in crypto/ecdsa...
doctor> making all in crypto/dh...
doctor> making all in crypto/ecdh...
doctor> making all in crypto/dso...
doctor> making all in crypto/engine...
doctor> making all in crypto/buffer...
doctor> making all in crypto/bio...
doctor> making all in crypto/stack...
doctor> making all in crypto/lhash...
doctor> making all in crypto/rand...
doctor> making all in crypto/err...
doctor> making all in crypto/evp...
doctor> making all in crypto/asn1...
doctor> making all in crypto/pem...
doctor> making all in crypto/x509...
doctor> making all in crypto/x509v3...
doctor> making all in crypto/conf...
doctor> making all in crypto/txt_db...
doctor> making all in crypto/pkcs7...
doctor> making all in crypto/pkcs12...
doctor> making all in crypto/comp...
doctor> making all in crypto/ocsp...
doctor> making all in crypto/ui...
doctor> making all in crypto/krb5...
doctor> making all in crypto/cms...
doctor> making all in crypto/pqueue...
doctor> making all in crypto/ts...
doctor> making all in crypto/jpake...
doctor> making all in crypto/srp...
doctor> making all in crypto/store...
doctor> making all in crypto/cmac...
doctor> if [ -n "libcrypto.so.1.0.0 libssl.so.1.0.0" ]; then  (cd ..; make 
libcrypto.so.1.0.0);  fi
doctor> [ -z "libcrypto" ] || /usr/local/bin/clang39 -fPIC -DOPENSSL_PIC 
-DZLIB_SHARED -DZLIB -DOPENSSL_THREADS -pthread -D_THREAD_SAFE -D_REENTRANT 
-DDSO_DLFCN -DHAVE_DLFCN_H -DL_ENDIAN -O3 -Wall -DOPENSSL_EXPERIMENTAL_JPAKE 
-DOPENSSL_EXPERIMENTAL_STORE -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT 
-DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -I/usr/local/ssl/fips-2.0/include 
-DRC4_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM 
-DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -Iinclude  
-DFINGERPRINT_PREMAIN_DSO_LOAD -o fips_premain_dso   
/usr/local/ssl/fips-2.0/lib/fips_premain.c 
/usr/local/ssl/fips-2.0/lib/fipscanister.o  libcrypto.a 
doctor> libcrypto.a(ec_asn1.o): In function `EC_GROUP_get_basis_type':
doctor> ec_asn1.c:(.text+0x37): undefined reference to `OSSL_NELEM'
doctor> ec_asn1.c:(.text+0x64): undefined reference to `OSSL_NELEM'
doctor> libcrypto.a(ec_asn1.o): In function `ec_asn1_group2pkparameters':
doctor> ec_asn1.c:(.text+0x116b): undefined reference to `OSSL_NELEM'
doctor> ec_asn1.c:(.text+0x118d): undefined reference to `OSSL_NELEM'
doctor> clang-3.9: error: linker command failed with exit code 1 (use -v to see 
invocation)
doctor> *** Error code 1
doctor> 
doctor> Stop.
doctor> make[2]: stopped in /usr/source/openssl-1.0.2-stable-SNAP-20170223
doctor> *** Error code 1
doctor> 
doctor> Stop.
doctor> make[1]: stopped in 
/usr/source/openssl-1.0.2-stable-SNAP-20170223/crypto
doctor> *** Error code 1
doctor> 
doctor> Stop.
doctor> make: stopped in /usr/source/openssl-1.0.2-stable-SNAP-20170223
doctor> root@doctor:/usr/source/openssl-1.0.2-stable-SNAP-20170223 # exit
doctor> 
doctor> exit
doctor> 
doctor> Script done on Thu Feb 23 05:42:27 2017
doctor> 
doctor> Please fix!
doctor> 
doctor> -- 
doctor> Member - Liberal International This is doctor@@nl2k.ab.ca Ici 
doctor@@nl2k.ab.ca
doctor> Yahweh, Queen & country!Never Satan President Republic!Beware 
AntiChrist rising!
doctor> http://www.fullyfollow.me/rootnl2k  Look at Psalms 14 and 53 on 

[openssl-dev] STORE, the continued story

[Richard Levitte]

Hi,

last time I talked about the STORE effort, it was about search for
specific data.  I believe that this PR covers quite a lot of what is
desired, and is designed to be easily extensible:

https://github.com/openssl/openssl/pull/2688
(it's built on top of PRs #2011 and #1961, making it look quite huge).

As a proof of concept, I had a closer look at X509_LOOKUP_METHODs and
came up with two integrations with STORE, one being light weight (or
cowardly), the other being a bit more radical.

lightweight: https://github.com/openssl/openssl/pull/2696
radical: https://github.com/openssl/openssl/pull/2697

These two PRs have not been built on top of the URI and STORE PRs, for
demonstration purposes.  To make them work, they need to be merged on
top of PR #2688.
Note: they currently only take straight file / directory specs, no
URIs.  The change to full blown URI processing isn't very hard but
needs a bit more testing.

At this point, it's high time for comments in the PRs, and reviews
(especially by team members)...  so far, there's been very little of
that.

Also, it's high time to start playing with engines and see how
integration with the STORE API works out.  The TPM engine would be
interesting, and so would the PKCS#11 one.  Also, if there's an LDAP
engine to adapt, that would be an interesting project as well.

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] (future) STORE vs X509_LOOKUP_METHOD by_dir

[Richard Levitte]

In message <589b86c1.10...@roumenpetrov.info> on Wed, 08 Feb 2017 22:59:45 
+0200, Roumen Petrov <open...@roumenpetrov.info> said:

openssl> Hi Richard,
openssl> 
openssl> Richard Levitte wrote:
openssl> > Hi,
openssl> >
openssl> > I've some ponderings that I need to bounce a bit with you all.
openssl> >
openssl> > Some have talked about replace the X509_LOOKUP_METHOD
openssl> X.509 lookup method could return certificate , revocation list or
openssl> EVP_KEY (structure x509_object_st).
openssl> 
openssl> Unfortunately   functionality of EVP_KEY was never implemented.
openssl> Another point is specific names of structures - x509_lookup_method_st
openssl> , x509_lookup_st, x509_object_st.
openssl> Third point is quite specific implementation - functions not just to
openssl> retrieve objects( X.509 or CRL) but to fill them into "context of X509
openssl> store".
openssl> 
openssl> Current lookup functionality look like "store" but implementation is
openssl> specific to X.509 store.
openssl> 
openssl> 
openssl> > bit with the
openssl> > STORE module I'm building, and while STORE isn't ready for it yet
openssl> 
openssl> I hope that you store functionality will fill gap between load of keys
openssl> and load of certificates (+crl).
openssl> 
openssl> Loadable module (engine) has interface to load key(private or public)
openssl> but lack load of X.509 certificates or CRL.

So far, key parameters, pkeys, certs and crls are covered...  oh, and
names, if the given URI would return a list of names (file:/foo/bar/
would typically do that, for example).

openssl> > , I
openssl> > have some thoughts on how the two can approach each other.  This 
would
openssl> > involve one or two hooks / callbacks, that a STORE user could specify
openssl> > (details later) to pick and choose freely among the objects that the
openssl> > STORE module finds (be it on file or whatever else that can be
openssl> > represented as a URI).
openssl> I think that functionality requires three phases :
openssl> 1) instantiation : at this point store is created
openssl> 2) specification (optional): set or check capability of store. For
openssl> instance store could return only X.509 certificates or to request
openssl> store to return only keys.
openssl> 3) inquiry: fetch data based on specified criteria.

Previous attempts at a STORE effort were along those lines of
thinking.  Unfortunately, they got stuck in the exact specification of
attributes and how to combine them (i.e. forming a "language" of
sorts), and then to figure out how to make that pratical, especially
for the engines that would have to intrepret it.

So this time around, I'm trying to make OpenSSL fairly agnostic and
leave the exact spec up to be specified by the user or application in
the URI and letting the diverse engines interpret the URI components
at their leasure.  Objects would then be returned according to those
specs in form of one object at a time, be it parameters, keys, certs,
crls or names, for the application or other parts of OpenSSL to do
whatever they wish with.  Given this, I determined yesterday that
there would really just be one hook / callback that makes sense, and
that's one that can massage or possibly discard returned objects
before they make their way all the way to the calling application.
Something like this.

openssl> > The troublesome part would be to try to mimic by_dir...  It highly
openssl> > depends on the specified paths to really be directories, and that it
openssl> > should find what it wants by adding very specific file names (a hash
openssl> > of the subject name with a ".{n}" or ".r{n}" extension for X.509 
certs
openssl> > and for X.509 CRLs).  And sure, that works, but will really only work
openssl> > with regular files.
openssl> I'm not sure what is issue.
openssl> 
openssl> Lets see X.509 lookup method get_by_subject.
openssl> 
openssl> - by_dir
openssl> 2) specification : set directory(path), limit results to X.509 or CRL
openssl> and may be to inform store that questions will be performed by
openssl> subject.
openssl> 
openssl> For instance URI scheme could befile://path?certificate="name"
openssl> 
openssl> 3) query : from subject calculate hash and then process "{hash}.{n}"
openssl> or "{hash}.r{n}" depending from URI
openssl> 
openssl> - Ldap
openssl> It is similar, URI is described in RFCs - at point 2) set host, port,
openssl> base distinguished name, attribute (for instance cACertificate),
openssl> construct filter from specified name.

The issue is there, glaring back at you.  How will by_dir know exactly
how to massage the URI?  Sure, we could program in the big known ones
plus our o

[openssl-dev] (future) STORE vs X509_LOOKUP_METHOD by_dir

[Richard Levitte]

Hi,

I've some ponderings that I need to bounce a bit with you all.

Some have talked about replace the X509_LOOKUP_METHOD bit with the
STORE module I'm building, and while STORE isn't ready for it yet, I
have some thoughts on how the two can approach each other.  This would
involve one or two hooks / callbacks, that a STORE user could specify
(details later) to pick and choose freely among the objects that the
STORE module finds (be it on file or whatever else that can be
represented as a URI).

The troublesome part would be to try to mimic by_dir...  It highly
depends on the specified paths to really be directories, and that it
should find what it wants by adding very specific file names (a hash
of the subject name with a ".{n}" or ".r{n}" extension for X.509 certs
and for X.509 CRLs).  And sure, that works, but will really only work
with regular files.

What if someone would specify a LDAP URI that can return a bunch of
objects?

So...  my ponderings are going along these lines:

1. Should the directory X509_LOOKUPs be restricted to on disk
   directories, or should "directory" be redefined as "whatever URI
   that returns a collection of objects"?  The latter would mean that
   all those objects get loaded and that a hook / callback would then
   be called to check if it's an object that corresponds to what we
   search for.

2. For on disk directories, should we preserve the rehash file form?
   In other words, if we decide to load everything we can find, shall
   we restrict the loading to files matching the regexp
   [0-9a-f]{8}\.r?[0-9]+  ?  If not, are we about to create a new form
   of key store for ourselves and our users?  Should we?

Quite a lot also depends on what OpenSSL version we aim for.  I would
very much like to see the STORE module itself become part of 1.1.1,
but a new key store to replace our current rehash links will obviously
have to wait 'til 1.2.0.

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [openssl-commits] UI_METHOD

[Richard Levitte]

In message <9da4cbdc-7437-b942-0d2e-e05808cd1...@akamai.com> on Thu, 12 Jan 
2017 11:17:10 -0600, Benjamin Kaduk <bka...@akamai.com> said:

bkaduk> On 01/11/2017 11:27 AM, Richard Levitte wrote:
bkaduk> 
bkaduk> The branch master has been updated
bkaduk>via  66ed24b1624606593a23c9fe78d459718d26409c (commit)
bkaduk>via  78b19e90b4aade1ffdf8d918277910b01dac1d76 (commit)
bkaduk>via  cc10f2275594eac2bcdaa218c1d6f672c2b891b7 (commit)
bkaduk>via  3ab3c8cb274aca1b5c0fbf83af96e49baf422708 (commit)
bkaduk>via  0fe1fc858a0519c3866c0d2e88513e677b674926 (commit)
bkaduk>via  18cfc668eae2c296e9bc90ffc989d9bbe61cc82f (commit)
bkaduk>via  a223ffe6d35209c59ed498ee89d1bac6e82e2ac2 (commit)
bkaduk>via  264b2d92511572a247ecb673d61ff385deb9eb8d (commit)
bkaduk>   from  5eeb6c6e562937dcfdd4b79619a699a118deadba (commit)
bkaduk> 
bkaduk> +static int ui_method_data_index()
bkaduk> +{
bkaduk> +static int idx = -1;
bkaduk> +
bkaduk> +if (idx == -1)
bkaduk> +idx = CRYPTO_get_ex_new_index(CRYPTO_EX_INDEX_UI_METHOD,
bkaduk> +  0, NULL,
bkaduk> +  ui_new_method_data,
bkaduk> +  ui_dup_method_data,
bkaduk> +  ui_free_method_data);
bkaduk> 
bkaduk> Should this have some synchronization around it to prevent concurrent
bkaduk> checks in multiple threads? It looks like we use a lock for our
bkaduk> internal stuff, but a run_once ought to be fine, too.

Good point...  and quite easily done.

Incidently, include/internal/thread_once.h isn't telling the truth
about the return value of RUN_ONCE...  I'll take advantage of that!

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] use SIPhash for OPENSSL_LH_strhash?

[Richard Levitte]

Can we look forward to a github PR?

In message <97d0be2d-11c6-4d01-9a5d-faccc5b27...@akamai.com> on Tue, 10 Jan 
2017 22:42:17 +, "Short, Todd" <tsh...@akamai.com> said:

tshort> I think I might have an init/update/final version of siphash24 lying
tshort> around somewhere that would be compatible with OpenSSL’s EVP_PKEY
tshort> mechanism (similar to Poly1305, in that it needs a key).
tshort> --
tshort> -Todd Short
tshort> // tsh...@akamai.com
tshort> // "One if by land, two if by sea, three if by the Internet."
tshort> 
tshort> On Jan 10, 2017, at 4:55 PM, Richard Levitte <levi...@openssl.org>
tshort> wrote:
tshort> 
tshort> 
tshort> 
tshort> 
tshort> Benjamin Kaduk <bka...@akamai.com> skrev: (10 januari 2017
tshort> 20:19:21 CET)
tshort> 
tshort> On 01/10/2017 12:31 PM, Richard Levitte wrote:
tshort> 
tshort> 
tshort> Benjamin Kaduk <bka...@akamai.com> skrev: (10 januari 2017
tshort> 18:48:32
tshort> 
tshort> 
tshort> CET)
tshort> 
tshort> On 01/09/2017 10:05 PM, Salz, Rich wrote:
tshort> 
tshort> Should we move to using SIPHash for the default string
tshort> hashing
tshort> function in OpenSSL? It’s now in the kernel
tshort> https://lkml.org/lkml/2017/1/9/619
tshort> 
tshort> 
tshort> 
tshort> Heck, yes!
tshort> -Ben
tshort> 
tshort> 
tshort> I fail to see what that would give us. OPENSSL_LH_strhash
tshort> () is used
tshort> 
tshort> 
tshort> to get a reasonable index for LHASH entries. Also SIPhash
tshort> gives at
tshort> least 64 bits results, do we really expect to see large enough
tshort> hash
tshort> tables to warrant that?
tshort> 
tshort> 
tshort> 
tshort> 
tshort> We don't need to use the full output width of a good hash
tshort> function.
tshort> 
tshort> My main point is, "why would we want to ignore the last 20
tshort> years of
tshort> advancement in hash function research?" Section 7 of the
tshort> siphash paper
tshort> (https://131002.net/siphash/siphash.pdf) explicitly talks
tshort> about using
tshort> it
tshort> for hash tables, including using hash table indices H(m) mod
tshort> l.
tshort> 
tshort> 
tshort> I agree with the advice when one can expect huge tables. The
tshort> tables we handle are pretty small (I think, please correct me if
tshort> I'm wrong) and would in all likelihood not benefit very much if at
tshort> all from SIPhash's relative safety.
tshort> 
tshort> Of course, one can ask the question if someone uses LHASH as a
tshort> general purpose hash table implementation rather than just for the
tshort> stuff OpenSSL. Frankly, I would probably look at a dedicated hash
tshort> table library first...
tshort> 
tshort> Cheers
tshort> Richard
tshort> --
tshort> Sent from my Android device with K-9 Mail. Please excuse my
tshort> brevity.
tshort> --
tshort> openssl-dev mailing list
tshort> To unsubscribe:
tshort> https://mta.openssl.org/mailman/listinfo/openssl-dev
tshort> 
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] use SIPhash for OPENSSL_LH_strhash?

[Richard Levitte]

A note: I have absolutely nothing against the addition of SIPhash in
our collection of hash algos.  My scepticism was only in regards to
using it as a string hasher for our hash tables indexes.

Cheers,
Richard

In message <20170111.153458.1623912899597806811.levi...@openssl.org> on Wed, 11 
Jan 2017 15:34:58 +0100 (CET), Richard Levitte <levi...@openssl.org> said:

levitte> In message 
<1e19cdfea8224717b3eee11e2d8ac...@usma1ex-dag1mb1.msg.corp.akamai.com> on Wed, 
11 Jan 2017 03:13:39 +, "Salz, Rich" <rs...@akamai.com> said:
levitte> 
levitte> rsalz> The needs for OpenSSL's LHASH are exactly what SipHash was 
designed for: fast on short strings.
levitte> rsalz> OpenSSL's hash currently *does not* call MD5 or SHA1; the MD5 
code is commented out.
levitte> rsalz> Yes, performance tests would greatly inform the decision.
levitte> 
levitte> Done, using the reference siphash implementation.
levitte> 
levitte> https://github.com/levitte/openssl/tree/test-string-hashes
levitte> 
levitte> A run on my laptop gave these results:
levitte> 
levitte> : ; ./util/shlib_wrap.sh apps/openssl speed siphash lhash
levitte> Doing lhash for 3s on 16 size blocks: 27635188 lhash's in 3.00s
levitte> Doing lhash for 3s on 64 size blocks: 6934726 lhash's in 3.00s
levitte> Doing lhash for 3s on 256 size blocks: 1698489 lhash's in 3.00s
levitte> Doing lhash for 3s on 1024 size blocks: 431185 lhash's in 3.00s
levitte> Doing lhash for 3s on 8192 size blocks: 53868 lhash's in 3.00s
levitte> Doing lhash for 3s on 16384 size blocks: 27041 lhash's in 3.00s
levitte> Doing siphash for 3s on 16 size blocks: 22488748 siphash's in 3.00s
levitte> Doing siphash for 3s on 64 size blocks: 10485674 siphash's in 3.00s
levitte> Doing siphash for 3s on 256 size blocks: 3320898 siphash's in 3.00s
levitte> Doing siphash for 3s on 1024 size blocks: 894647 siphash's in 3.00s
levitte> Doing siphash for 3s on 8192 size blocks: 114170 siphash's in 3.00s
levitte> Doing siphash for 3s on 16384 size blocks: 57151 siphash's in 3.00s
levitte> OpenSSL 1.1.1-dev  xx XXX 
levitte> built on: reproducible build, date unspecified
levitte> options:bn(64,64) rc4(16x,int) des(int) aes(partial) idea(int) 
blowfish(ptr) 
levitte> compiler: gcc -DDSO_DLFCN -DHAVE_DLFCN_H -DOPENSSL_THREADS 
-DOPENSSL_NO_STATIC_ENGINE -DOPENSSL_PIC -DOPENSSL_IA32_SSE2 
-DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM 
-DSHA256_ASM -DSHA512_ASM -DRC4_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM 
-DGHASH_ASM -DECP_NISTZ256_ASM -DPADLOCK_ASM -DPOLY1305_ASM 
-DOPENSSLDIR="\"/usr/local/ssl\"" -DENGINESDIR="\"/usr/local/lib/engines-1.1\"" 
 -DDEBUG_UNUSED -Wswitch -DPEDANTIC -pedantic -Wno-long-long -Wall 
-Wsign-compare -Wmissing-prototypes -Wshadow -Wformat -Wtype-limits -Werror 
-Wa,--noexecstack
levitte> The 'numbers' are in 1000s of bytes per second processed.
levitte> type 16 bytes 64 bytes256 bytes   1024 bytes   
8192 bytes  16384 bytes
levitte> lhash   147387.67k   147940.82k   144937.73k   147177.81k  
 147095.55k   147679.91k
levitte> siphash 119939.99k   223694.38k   283383.30k   305372.84k  
 311760.21k   312120.66k
levitte> 
levitte> So it seems that for short strings, OPENSSL_LH_strhash (*) wins some,
levitte> while siphash wins big for larger strings.
levitte> 
levitte> I have no idea how they compare with regard to distribution (which,
levitte> considering I ask for the same size output from both, should be the
levitte> main factor that affects the sensitivity to hash flooding)...
levitte> 
levitte> Our use of OPENSSL_LH_strhash() is with configuration sections and
levitte> names, ASN.1 object names and the function names in the openssl app.
levitte> All our other uses of lhash use their own hashing functions, and are
levitte> usually very short still (definitely less than 16 bytes).
levitte> 
levitte> My conclusion is that performance-wise, siphash doesn't give us any
levitte> advantage over OpenSSL_LH_strhash for our uses.
levitte> 
levitte> Cheers,
levitte> Richard
levitte> 
levitte> (*) Strictly speaking, it's a modified version that takes a length and
levitte> tolerates all 8-bit bytes, including 0x00.
levitte> 
levitte> -- 
levitte> Richard Levitte levi...@openssl.org
levitte> OpenSSL Project http://www.openssl.org/~levitte/
levitte> -- 
levitte> openssl-dev mailing list
levitte> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
levitte> 
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] use SIPhash for OPENSSL_LH_strhash?

[Richard Levitte]

In message 
<1e19cdfea8224717b3eee11e2d8ac...@usma1ex-dag1mb1.msg.corp.akamai.com> on Wed, 
11 Jan 2017 03:13:39 +, "Salz, Rich" <rs...@akamai.com> said:

rsalz> The needs for OpenSSL's LHASH are exactly what SipHash was designed for: 
fast on short strings.
rsalz> OpenSSL's hash currently *does not* call MD5 or SHA1; the MD5 code is 
commented out.
rsalz> Yes, performance tests would greatly inform the decision.

Done, using the reference siphash implementation.

https://github.com/levitte/openssl/tree/test-string-hashes

A run on my laptop gave these results:

: ; ./util/shlib_wrap.sh apps/openssl speed siphash lhash
Doing lhash for 3s on 16 size blocks: 27635188 lhash's in 3.00s
Doing lhash for 3s on 64 size blocks: 6934726 lhash's in 3.00s
Doing lhash for 3s on 256 size blocks: 1698489 lhash's in 3.00s
Doing lhash for 3s on 1024 size blocks: 431185 lhash's in 3.00s
Doing lhash for 3s on 8192 size blocks: 53868 lhash's in 3.00s
Doing lhash for 3s on 16384 size blocks: 27041 lhash's in 3.00s
Doing siphash for 3s on 16 size blocks: 22488748 siphash's in 3.00s
Doing siphash for 3s on 64 size blocks: 10485674 siphash's in 3.00s
Doing siphash for 3s on 256 size blocks: 3320898 siphash's in 3.00s
Doing siphash for 3s on 1024 size blocks: 894647 siphash's in 3.00s
Doing siphash for 3s on 8192 size blocks: 114170 siphash's in 3.00s
Doing siphash for 3s on 16384 size blocks: 57151 siphash's in 3.00s
OpenSSL 1.1.1-dev  xx XXX 
built on: reproducible build, date unspecified
options:bn(64,64) rc4(16x,int) des(int) aes(partial) idea(int) 
blowfish(ptr) 
compiler: gcc -DDSO_DLFCN -DHAVE_DLFCN_H -DOPENSSL_THREADS 
-DOPENSSL_NO_STATIC_ENGINE -DOPENSSL_PIC -DOPENSSL_IA32_SSE2 
-DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM 
-DSHA256_ASM -DSHA512_ASM -DRC4_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM 
-DGHASH_ASM -DECP_NISTZ256_ASM -DPADLOCK_ASM -DPOLY1305_ASM 
-DOPENSSLDIR="\"/usr/local/ssl\"" -DENGINESDIR="\"/usr/local/lib/engines-1.1\"" 
 -DDEBUG_UNUSED -Wswitch -DPEDANTIC -pedantic -Wno-long-long -Wall 
-Wsign-compare -Wmissing-prototypes -Wshadow -Wformat -Wtype-limits -Werror 
-Wa,--noexecstack
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes256 bytes   1024 bytes   8192 
bytes  16384 bytes
lhash   147387.67k   147940.82k   144937.73k   147177.81k   
147095.55k   147679.91k
siphash 119939.99k   223694.38k   283383.30k   305372.84k   
311760.21k   312120.66k

So it seems that for short strings, OPENSSL_LH_strhash (*) wins some,
while siphash wins big for larger strings.

I have no idea how they compare with regard to distribution (which,
considering I ask for the same size output from both, should be the
main factor that affects the sensitivity to hash flooding)...

Our use of OPENSSL_LH_strhash() is with configuration sections and
names, ASN.1 object names and the function names in the openssl app.
All our other uses of lhash use their own hashing functions, and are
usually very short still (definitely less than 16 bytes).

My conclusion is that performance-wise, siphash doesn't give us any
advantage over OpenSSL_LH_strhash for our uses.

Cheers,
Richard

(*) Strictly speaking, it's a modified version that takes a length and
tolerates all 8-bit bytes, including 0x00.

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] use SIPhash for OPENSSL_LH_strhash?

[Richard Levitte]

In message <001901d26bed$d3746ed0$7a5d4c70$@sa...@free.fr> on Wed, 11 Jan 2017 
10:33:53 +0100, "Michel" <michel.sa...@free.fr> said:

michel.sales> And what about using FNV or CityHash ?
michel.sales> 
michel.sales> 
https://en.wikipedia.org/wiki/Fowler%E2%80%93Noll%E2%80%93Vo_hash_function

I'm a little worried about the zero hash value issue mentioned here:

https://en.wikipedia.org/wiki/Fowler%E2%80%93Noll%E2%80%93Vo_hash_function#Non-cryptographic_hash

michel.sales> https://en.wikipedia.org/wiki/CityHash

Google has replaced that with FarmHash according to that page...

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] use SIPhash for OPENSSL_LH_strhash?

[Richard Levitte]

Benjamin Kaduk <bka...@akamai.com> skrev: (10 januari 2017 20:19:21 CET)
>On 01/10/2017 12:31 PM, Richard Levitte wrote:
>>
>> Benjamin Kaduk <bka...@akamai.com> skrev: (10 januari 2017 18:48:32
>CET)
>>> On 01/09/2017 10:05 PM, Salz, Rich wrote:
>>>> Should we move to using SIPHash for the default string hashing
>>>> function in OpenSSL?  It’s now in the kernel
>>>> https://lkml.org/lkml/2017/1/9/619
>>>>
>>> Heck, yes!
>>> -Ben
>> I fail to see what that would give us. OPENSSL_LH_strhash() is used
>to get a reasonable index for LHASH entries. Also SIPhash gives at
>least 64 bits results, do we really expect to see large enough hash
>tables to warrant that? 
>>
>
>We don't need to use the full output width of a good hash function.
>
>My main point is, "why would we want to ignore the last 20 years of
>advancement in hash function research?"  Section 7 of the siphash paper
>(https://131002.net/siphash/siphash.pdf) explicitly talks about using
>it
>for hash tables, including using hash table indices H(m) mod l.

I agree with the advice when one can expect huge tables. The tables we handle 
are pretty small (I think, please correct me if I'm wrong) and would in all 
likelihood not benefit very much if at all from SIPhash's relative safety. 

Of course, one can ask the question if someone uses LHASH as a general purpose 
hash table implementation rather than just for the stuff OpenSSL. Frankly, I 
would probably look at a dedicated hash table library first... 

Cheers 
Richard 
-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] use SIPhash for OPENSSL_LH_strhash?

[Richard Levitte]

Benjamin Kaduk  skrev: (10 januari 2017 18:48:32 CET)
>On 01/09/2017 10:05 PM, Salz, Rich wrote:
>>
>> Should we move to using SIPHash for the default string hashing
>> function in OpenSSL?  It’s now in the kernel
>> https://lkml.org/lkml/2017/1/9/619
>>
>
>>
>> Overview at https://131002.net/siphash/
>>
>
>>
>>
>
>Instead of this?
>
>%
>
>/*-
>unsigned char b[16];
>MD5(c,strlen(c),b);
>return(b[0]|(b[1]<<8)|(b[2]<<16)|(b[3]<<24));
>*/
>
>n = 0x100;
>while (*c) {
>v = n | (*c);
>n += 0x100;
>r = (int)((v >> 2) ^ v) & 0x0f;
>ret = (ret << r) | (ret >> (32 - r));
>ret &= 0xL;
>ret ^= v * v;
>c++;
>}
>return ((ret >> 16) ^ ret);
>
>%
>
>Heck, yes!
>
>-Ben

I fail to see what that would give us. OPENSSL_LH_strhash() is used to get a 
reasonable index for LHASH entries. Also SIPhash gives at least 64 bits 
results, do we really expect to see large enough hash tables to warrant that? 

Cheers 
Richard 
-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [PATCH 1/1] add TPM2 version of create_tpm2_key and libtpm2.so engine

[Richard Levitte]

In message <1483487075.2464.59.ca...@hansenpartnership.com> on Tue, 03 Jan 2017 
15:44:35 -0800, James Bottomley <james.bottom...@hansenpartnership.com> said:

James.Bottomley> On Tue, 2017-01-03 at 12:19 +0100, Richard Levitte wrote:
James.Bottomley> > ⁣There seems to be some confusion here. 
James.Bottomley> > 
James.Bottomley> > James, I understand the tpm engine as an external project, 
not part
James.Bottomley> > of the OpenSSL source proper and not intended to be. 
James.Bottomley> > 
James.Bottomley> > However, openssl-dev@openssl.org is a list focused on the 
development
James.Bottomley> > of OpenSSL proper. That makes it a bit odd to discuss the 
tpm engine
James.Bottomley> > here. Largely off topic. 
James.Bottomley> 
James.Bottomley> Fair enough.  You were cc'd since it's a modification of code 
used by
James.Bottomley> openSSL, in case there was interest.

Strictly speaking, that belongs in openssl-us...@openssl.org.

The reason I point this out is that for code that isn't meant to be
part of OpenSSL proper, the whole discussion about CLAs, licenses and
whatnot is a red herring that belongs neither here not there.  As long
as you do stuff as a separate project, YOU (collective you) decide
what license to use, let alone your contribution policy.

Of course, I do recall that there was an attempt of patches to be
applied to OpenSSL proper.  That alone is subject to our license and
our policies, if that's still interesting (I don't know if it is).  If
it is, that should be contributed as a separate patch, preferably as a
github PR (sourceforge is entirely uninteresting to us).

Me, I haven't really minded the discussion here, as long as it didn't
become confusing.  After all, it did spark some discussion around my
STORE project ;-)

Did I leave something out or is the situation clear?

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [PATCH 1/1] add TPM2 version of create_tpm2_key and libtpm2.so engine

[Richard Levitte]

⁣There seems to be some confusion here. 

James, I understand the tpm engine as an external project, not part of the 
OpenSSL source proper and not intended to be. 

However, openssl-dev@openssl.org is a list focused on the development of 
OpenSSL proper. That makes it a bit odd to discuss the tpm engine here. Largely 
off topic. 

Cheers 
Richard 

Skickat från BlueMail ​

Den 2 jan. 2017 19:22, kI 19:22, "Salz, Rich"  skrev:
>> Really, how?  By pull request, you mean one against the openssl
>github
>> account so people subscribing to that account see it, I presume?  For
>that to
>> happen, the tree the patch is against must actually exist within the
>account,
>> which this one doesn't.
>
>You clone the openssl git repo, create your own branch off master,
>apply the diffs you are mailing to the list, and commit/push and then
>make a PR.  Yes it's a bit of work for you.  But it then becomes
>near-zero work for anyone on openssl to look at it.
>
>> This patch is mostly FYI, so yes, I do given that multiple mailing
>lists have
>> some interest.
>
>It's all about trade-offs.  Multiple people have said multiple times
>that PR's are the best way to work with OpenSSL.  If those other
>groups, individually or collectively, are higher on your priority list,
>that's fine.  But do understand what's going on.
>
>> I'm still waiting on a reply ... I assume holidays are contributing
>to the delay.
>> However, openssl_tpm_engine is a DCO project, so that concern is
>irrelevant
>> here.
>
>Sorry, I'll push to get the bylaws made public, is that what you need?
>
>And no, it's not irrelevant.  If this is ever going to appear in
>OpenSSL, a CLA must be signed.
>
>-- 
>openssl-dev mailing list
>To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Linker error when adding new cipher in crypto folder

[Richard Levitte]

In message <b5ac3fbd-33f9-4377-aeb3-e2671de09...@unh.newhaven.edu> on Fri, 30 
Dec 2016 02:36:45 +, "Schmicker, Robert" <rsc...@unh.newhaven.edu> said:

rschm2> Hello,
rschm2> 
rschm2> I am attempting to add a new cipher into the crypto library. I have
rschm2> done the following so far…
rschm2> 
rschm2> 1. Added my code to the openssl/crypto folder
rschm2> 2. Created a build.info for make to compile my code (created this
rschm2> based off of openssl/crypto/dh’s build.info)
rschm2> 3. Added my cipher name to the list of ciphers to compile in Configure
rschm2> 4. Compiled and installed all code without errors and object files for
rschm2> my new cipher are created in openssl/crypto/mycipher/
rschm2> 5. Created a test.c file that verifies that the library is installed
rschm2> and working properly by generating a MD5 hash of a string
rschm2> Compiled as:
rschm2> gcc test.c -I/usr/local/openssl/include/openssl/ -o test -lcrypto -
rschm2> lssl -Wall
rschm2> 6. I am able to properly include  in my test.c
rschm2> file
rschm2> 
rschm2> However, as soon as I make a call to my cipher in test.c I get a
rschm2> linker error and gcc is unable to find any of my functions.
rschm2> 
rschm2> It seems that the header file I have in the openssl/include folder
rschm2> isn’t being linked somehow to my code in the openssl/cypto folder.
rschm2> 
rschm2> I feel like I’m missing a step here…
rschm2> 
rschm2> Any help is much appreciated!

On way is to, as already mentioned, edit util/libcrypto.num.  The
other is to edit util/mkdef.pl and then run 'make update'.  In
mkdef.pl, you'll find a bunch of lines like this:

$crypto.="include/openssl/whatever.h"

Simply add a line like that for mycipher.h.

(note: mkdef.pl might be a bit picky sometimes)

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Proposal for the ASN.1 form of TPM1.2 and TPM2 keys

[Richard Levitte]

In message <1482516363.2501.34.ca...@hansenpartnership.com> on Fri, 23 Dec 2016 
10:06:03 -0800, James Bottomley <james.bottom...@hansenpartnership.com> said:

James.Bottomley> The reason this comes about is because we already have a 
standard form
James.Bottomley> for TPM 1.2 keys here:
James.Bottomley> 
James.Bottomley> 
http://david.woodhou.se/draft-woodhouse-cert-best-practice.html#ident-tpm
James.Bottomley> 
James.Bottomley> However, since I'm working on TPM2 enabling for openssl and 
gnutls, I
James.Bottomley> need to come up with a new key format because TPM2 requires 
some extra
James.Bottomley> parameters and the original TSS KEY BLOB, being a single
James.Bottomley> ASN1_OCTET_STRING isn't expandable.
James.Bottomley> 
James.Bottomley> As a digression, the extra parameters that TPM2 needs are:
James.Bottomley> 
James.Bottomley>1. A public key blob.  In TPM12 the key complex was a joint
James.Bottomley>   public/private part.  In TPM2, the public and private 
key structures
James.Bottomley>   have variable length and are supplied separately.
James.Bottomley>2. a boolean for emptyAuth.  In TPM12 there's a way to tell 
if a
James.Bottomley>   structure has no authorization.  In TPM2 there's no such 
thing as no
James.Bottomley>   authorization, but there's a conventional empty 
authorization to
James.Bottomley>   replace it but no way of querying whether any given key 
is using it,
James.Bottomley>   so we need to know explicitly whether to prompt for a 
password or
James.Bottomley>   not.
James.Bottomley>3. There are different forms a TPM private key could be in. 
 One is
James.Bottomley>   symmetrically encrypted with a TPM private key, which 
makes it
James.Bottomley>   loadable, meaning it must be produced on the TPM itself 
and the
James.Bottomley>   other is asymmetrically encrypted meaning it can be 
produced away
James.Bottomley>   from the TPM but must be imported before being loaded.
James.Bottomley> 
James.Bottomley> I think there's value in having a universal structure for the 
key
James.Bottomley> representations, so I'm proposing an ASN1 representation that 
will work
James.Bottomley> for both TPM1.2 and TPM2 keys.  I'd also like it to be self 
describing,
James.Bottomley> so I think we should use an OID as the initial parameter of the
James.Bottomley> sequence.  With that, I think the format that works is
James.Bottomley> 
James.Bottomley> TPMKey ::= SEQUENCE {
James.Bottomley>typeOBJECT IDENTIFIER
James.Bottomley>version [0] IMPLICIT INTEGER OPTIONAL
James.Bottomley>emptyAuth   [1] IMPLICIT BOOLEAN OPTIONAL
James.Bottomley>parent  [2] IMPLICIT INTEGER OPTIONAL
James.Bottomley>publicKey   [3] IMPLICIT OCTET STRING OPTIONAL
James.Bottomley>privateKey  OCTET STRING
James.Bottomley> }
James.Bottomley> 
James.Bottomley> Where TPM12 keys would have a TPM12Key type and use no 
optional fields
James.Bottomley> (meaning only privateKey) and TPM2 keys would have type 
TPM2LoadableKey
James.Bottomley> or TPM2ImportableKey type and then make use of all the 
optional fields
James.Bottomley> (except version).
James.Bottomley> 
James.Bottomley> Version is there for future expansion, but is unused in the 
initial
James.Bottomley> incarnation.
James.Bottomley> 
James.Bottomley> I'm torn on where to get the OIDs from.  Since this is a TPM 
key, it
James.Bottomley> might make sense to use the TCG OID (2.23.133) and just add 
something
James.Bottomley> they haven't already used, like 10 for key formats, or we 
could go with
James.Bottomley> a pkcs OID (1.2.840.113549.1)
James.Bottomley> 
James.Bottomley> If we can agree on this, we can update David's document and 
make it a
James.Bottomley> formal RFC.
James.Bottomley> 
James.Bottomley> Thoughts?

First reaction is +1, at least for actually making a universally
parsable description.

One detail that escapes me, though, is why you don't use version for,
well, TPM versions?  So, something like this?

TCG OBJECT IDENTIFIER ::=
{ joint-iso-itu-t(2) international-organizations(23) TCG(133)  }

TPMKey ::= SEQUENCE {
   typeOBJECT IDENTIFIER-- always TCG
   version [0] INTEGER { v1_2(0), v2(1) } DEFAULT v1_2
   emptyAuth   [1] IMPLICIT BOOLEAN OPTIONAL-- v2 only
   parent      [2] IMPLICIT INTEGER OPTIONAL-- v2 only
   publicKey   [3] IMPLICIT OCTET STRING OPTIONAL   -- v2 only
   privateKey  OCTET STRING
}

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Add a new algorithm in "crypto" dir, how to add the source code into the build system

[Richard Levitte]

In message <20161222.225335.92995302056231655.levi...@openssl.org> on Thu, 22 
Dec 2016 22:53:35 +0100 (CET), Richard Levitte <levi...@openssl.org> said:

levitte> In message <e6400041-6133-8b74-2ff9-043ec6dcb...@gmail.com> on Thu, 22 
Dec 2016 13:33:16 -0800, Joey Yandle <xol...@gmail.com> said:
levitte> 
levitte> xoloki> > May I suggest you have a look at the GOST engine?  It does 
implement
levitte> xoloki> > the algorithm entirely in the engine.  The only things added 
in the
levitte> xoloki> > OpenSSL code are the OIDs (not strictly necessary) and the 
TLS
levitte> xoloki> > ciphersuites (I don't think that can be done dynamically at 
all, at
levitte> xoloki> > least yet).
levitte> xoloki> 
levitte> xoloki> How are the OIDs not necessary?  What about the NIDs?
levitte> 
levitte> It's not stricly necessary to add them statically in the libcrypto
levitte> code.  They can be added dynamically by the engine by calling
levitte> OBJ_create() with the correct arguments.

Applications will then have to find out the nid by calling
OBJ_txt2nid, OBJ_sn2nid or OBJ_ln2nid, depending on the data they
have.  Note: this can already be done for the built in OIDs.

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Add a new algorithm in "crypto" dir, how to add the source code into the build system

[Richard Levitte]

In message <e6400041-6133-8b74-2ff9-043ec6dcb...@gmail.com> on Thu, 22 Dec 2016 
13:33:16 -0800, Joey Yandle <xol...@gmail.com> said:

xoloki> > May I suggest you have a look at the GOST engine?  It does implement
xoloki> > the algorithm entirely in the engine.  The only things added in the
xoloki> > OpenSSL code are the OIDs (not strictly necessary) and the TLS
xoloki> > ciphersuites (I don't think that can be done dynamically at all, at
xoloki> > least yet).
xoloki> 
xoloki> How are the OIDs not necessary?  What about the NIDs?

It's not stricly necessary to add them statically in the libcrypto
code.  They can be added dynamically by the engine by calling
OBJ_create() with the correct arguments.

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Add a new algorithm in "crypto" dir, how to add the source code into the build system

[Richard Levitte]

In message 
<936be946f26e274a8595dbf91a05bd767c4fa...@shsmsx101.ccr.corp.intel.com> on Thu, 
22 Dec 2016 13:12:36 +, "Wei, Changzheng" <changzheng@intel.com> said:

changzheng.wei> Hi,
changzheng.wei> 
changzheng.wei> I want to implement some new algorithm. To make my future work
changzheng.wei> smoothly, I want to add a new algorithm method like 
“RSA_METHOD” in
changzheng.wei> OpenSSL framework so as to I can use an “engine” to support such
changzheng.wei> algorithm.
changzheng.wei> 
changzheng.wei> So I add a new subdir in “crypto” and implement the code and
changzheng.wei> build.info refer to “crypto/rsa”.
changzheng.wei> 
changzheng.wei> My question is how to add my new source code into the build 
system?

A note here: if you're implementing a new algo, OpenSSL doesn't need
to know anything at all, everything can be added dynamically from the
engine and be reachable through the EVP API.

The only exception for now is the code in libssl.

May I suggest you have a look at the GOST engine?  It does implement
the algorithm entirely in the engine.  The only things added in the
OpenSSL code are the OIDs (not strictly necessary) and the TLS
ciphersuites (I don't think that can be done dynamically at all, at
least yet).

https://github.com/gost-engine/engine

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Cross compiling openssl for an old ARM environment - howto?

[Richard Levitte]

We are saying that we don't support 16-bit platforms and haven't
really done so for a long time.  The C-only version is no exception.

However, the response was that 32- and 64-bit operations and numbers
are supported on that device, so it should still be possible to play
with.

Cheers,
Richard

In message 

Re: [openssl-dev] Cross compiling openssl for an old ARM environment - howto?

[Richard Levitte]

In message 
<72e690f1b12147588b1dc3e7ee93c...@usma1ex-dag1mb1.msg.corp.akamai.com> on Mon, 
19 Dec 2016 18:22:30 +, "Salz, Rich" <rs...@akamai.com> said:

rsalz> 
rsalz> > I have an embedded device which runs an ARM7T armv4t 16-bit thumb 
platform.
rsalz> 
rsalz> I don't think openssl ever really ran on 16bit.

It certainly doesn't any more.  It did, though, a long time ago.

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys

[Richard Levitte]

In message <584d7f4e.8090...@roumenpetrov.info> on Sun, 11 Dec 2016 18:31:10 
+0200, Roumen Petrov <open...@roumenpetrov.info> said:

openssl> One remark for store load function api - in most cases (load from
openssl> file) it is password callback but is other cases it could be PIN or
openssl> something different.
openssl> Please use more generic description.
openssl> For instance engine callback is defined in generic way - ui_method and
openssl> its callback_data.

I see what you mean.  Just did the improvement.

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

[openssl-dev] STORE [was: [RFC v2 2/2] pem: load engine keys]

[Richard Levitte]

In message <584d77cb.7090...@roumenpetrov.info> on Sun, 11 Dec 2016 17:59:07 
+0200, Roumen Petrov <open...@roumenpetrov.info> said:

openssl> HI Richard,
openssl> 
openssl> Richard Levitte wrote:
openssl> > In message<58472e4f.3010...@roumenpetrov.info> on Tue, 06 Dec 2016
openssl> > 23:31:59 +0200, Roumen Petrov<open...@roumenpetrov.info> said:
openssl> >
openssl> > openssl> Hi Richard,
openssl> > openssl>
openssl> > [SNIP]
openssl> > openssl> > Check.  My STORE branch is made to support that.
openssl> > openssl> One URI could represent more then one item.
openssl> > openssl> STORE_INFO_types is enumerate but URI could be associated to
openssl> > custom
openssl> > openssl> data (handle) and this data could be used to get other
openssl> > data(handles).
openssl> > openssl>
openssl> > openssl> See capi engine CAPI_KEY *capi_find_key(CAPI_CTX * ctx, 
const
openssl> > char
openssl> > openssl> *id)
openssl> > [SNIP]
openssl> > openssl> Is above case PKEY is loaded only if CERT is located(found).
openssl> >
openssl> > I'm trying to understand but am failing.  Looking at your example,
openssl> > it's quite clear that what you want to retrieve is a key, even though
openssl> > you have to go through the corresponding certificate to get to it.
openssl> After first review of API delared in openssl/store.h I misunderstand
openssl> goal of load method.

Actually, it did change at one point.  Oh, and just so we're all on
the same page, I started over a few weeks ago, when Rich Salz
expressed some concerns with the previous "grab everything" load
call.  The active branch is "store2", available as this pull request:

https://github.com/openssl/openssl/pull/2011

(I can't remember if I said that already)

openssl> I think that code of capi engine could be considered as sample what is
openssl> need for an loadable module (engine) to use "OpenSSL Store API". I
openssl> post above code just to get idea where currently is used an "external
openssl> store api".
openssl> Just imagine how existing capi code could be changed to use store-API
openssl> and to implement loader(scheme?).

I don't know CAPI at all, really, so whatever scheme is used will
depend on what CAPI provides.  From a quick read of the docs, it looks
like simple files, and it's possible that a few extra file handlers
(which are another set of dynamically definable functions within the
STORE 'file' scheme) will suffice.

openssl> I'm asking as currently there is no interface (API) that could
openssl> associate key (private) and X.509 certificate. Currently engines
openssl> implement custom command as work-around. For instance LOAD_CERT_CTRL
openssl> (pkcs11 and e_nss) and LOAD_CERT_EVP(e_nss).
openssl> 
openssl> This one of areas where applications could benefit from "Store API".

That's exactly the sort of thing I'm aiming for :-)

openssl> I post a sniped from CAPI code because it is part of OpenSSL, but king
openssl> of "external store api" is used by other engines.
openssl> 
openssl> 
openssl> > However,*nothing*  stops anyone from making a loader for the "capi"
openssl> > scheme (if there is such a thing) that has a load method that will
openssl> > return the certificate (STORE_INFO_CERT) on the first call and the
openssl> > associated key (STORE_INFO_PKEY) on the second for the same URI.  
It's
openssl> > all about caching information, and there is a context variable (type
openssl> > STORE_LOADER_CTX, which is just a template type for loader defined
openssl> > 'struct store_loader_ctx_st') to be used exactly for that kind of
openssl> > purpose.
openssl> 
openssl> I guess that "load" method is supposed to return all data at once.
openssl> 
openssl> Actually it is an iterator!

Yes.  That was a change that happened a few weeks ago

openssl> Please update comments before method and if possible to change name of
openssl> method.

I don't know how up to date you are.  This is the current comment for
STORE_load() in store.h:

/*
 * Read one data item (a key, a cert, a CRL) that is supported by the STORE
 * functionality, given a context.
 * Returns a STORE_INFO pointer, from which OpenSSL typed data can be 
extracted
 * with STORE_INFO_get0_PKEY(), STORE_INFO_get0_CERT(), ...
 * NULL is returned on error, which may include that the data found at the 
URI
 * can't be figured out for certain or is ambiguous.
 */
STORE_INFO *STORE_load(STORE_CTX *ctx);

If STORE_load() is a bad name, what would you suggest that makes it
better?

openssl> > [SNIP]
openssl> > In your example above, I fail to see where the custom data would

Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys

[Richard Levitte]

Roumen Petrov <open...@roumenpetrov.info> skrev: (11 december 2016 17:31:10 CET)
>Hi Richard,
>
>Richard Levitte wrote:
>> In message<20161206.223057.237264374331072901.levi...@openssl.org> 
>on Tue, 06 Dec 2016 22:30:57 +0100 (CET), Richard
>Levitte<levi...@openssl.org>  said:
>>
>> levitte> [SNIP]
>>
>> The easiest was actually to rewrite PEM_read_bio_PrivateKey()
>> entirely, so it solely uses the internal store_file functions I've
>> provided.
>> I wonder what kind of impact this would have on the community at
>> large.
>
>PEM_read_bio_PrivateKey use custom password callback . You propose
>"Store-API" with UI_METHOD  as password callback.
>Rewrite of pem_read... method obsoletes pem_password_cb.
>
>What about to ensure a transition period?
>For instance in openssl 1.1 to provide new functions based on UI_METHOD
>and to mark existing as deprecated.
>
>
>One remark for store load function api - in most cases (load from file)
>it is password callback but is other cases it could be PIN or something
>different.
>Please use more generic description.
>For instance engine callback is defined in generic way -  ui_method and
>its callback_data.

Earlier, I mentioned an experimental branch, 
https://github.com/levitte/openssl/tree/tpm_engine-support?files=1

If you have a look, you'll find an added UI utility function to wrap a pem 
password callback in a UI_METHOD. 

>
> 
>
>> Cheers,
>> Richard
>
>Regards,
>Roumen

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys

[Richard Levitte]

In message <20161206.223057.237264374331072901.levi...@openssl.org> on Tue, 06 
Dec 2016 22:30:57 +0100 (CET), Richard Levitte <levi...@openssl.org> said:

levitte> That being said, it should certainly be easy enough to change the
levitte> appropriate places to make sure headers are available as well, and I
levitte> have zero issues adding a header parameter to the try_decode
levitte> prototype and associated functions.

Done.

levitte> One thing I didn't think of earlier is that PEM_bytes_read_bio()
levitte> checks the pem name against a known set, *or* in the private key case,
levitte> that the pem name ends with " PRIVATE KEY" (which "TSS KEY BLOB" does
levitte> not), so some kind of refactoring is needed to accomodate the
levitte> store_file_load() call either way.
levitte> (quite frankly, I'm slowly realising that the STORE_FILE_HANDLER code
levitte> can replace quite a lot of the discovery code in the PEM module, so
levitte> refactoring could be in order either way)

The easiest was actually to rewrite PEM_read_bio_PrivateKey()
entirely, so it solely uses the internal store_file functions I've
provided.
I wonder what kind of impact this would have on the community at
large.

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys

[Richard Levitte]

In message <58472e4f.3010...@roumenpetrov.info> on Tue, 06 Dec 2016 23:31:59 
+0200, Roumen Petrov <open...@roumenpetrov.info> said:

openssl> Hi Richard,
openssl> 
openssl> Richard Levitte wrote:
openssl> > [SNIP]
openssl> > James.Bottomley> 1. We agreed that usability is greatly enhanced if
openssl> > openssl simply loads
openssl> > James.Bottomley> a key when presented with the file/uri etc. without
openssl> > the user having
openssl> > James.Bottomley>   to specify what the format of a key is
openssl> >
openssl> > Check.  My STORE branch is made to support that.
openssl> One URI could represent more then one item.
openssl> STORE_INFO_types is enumerate but URI could be associated to custom
openssl> data (handle) and this data could be used to get other data(handles).
openssl> 
openssl> See capi engine CAPI_KEY *capi_find_key(CAPI_CTX * ctx, const char
openssl> *id)
openssl> ..
openssl> hstore = capi_open_store(ctx, NULL);
openssl> if (!hstore)
openssl> return NULL;
openssl> cert = capi_find_cert(ctx, id, hstore);
openssl> if (cert) {
openssl> key = capi_get_cert_key(ctx, cert);
openssl> CertFreeCertificateContext(cert);
openssl> }
openssl> CertCloseStore(hstore, 0);
openssl> ..
openssl> Is above case PKEY is loaded only if CERT is located(found).

I'm trying to understand but am failing.  Looking at your example,
it's quite clear that what you want to retrieve is a key, even though
you have to go through the corresponding certificate to get to it.

However, *nothing* stops anyone from making a loader for the "capi"
scheme (if there is such a thing) that has a load method that will
return the certificate (STORE_INFO_CERT) on the first call and the
associated key (STORE_INFO_PKEY) on the second for the same URI.  It's
all about caching information, and there is a context variable (type
STORE_LOADER_CTX, which is just a template type for loader defined
'struct store_loader_ctx_st') to be used exactly for that kind of
purpose.

In your example above, I fail to see where the custom data would be
needed...  And frankly, STORE is first of all meant to handle types
that can be used with the rest of OpenSSL.  That being said, adding a
"whatever" STORE_INFO type isn't very hard either.  I'm just not
terribly convinced yet, but let's keep talking, I'll probably
understand sooner or later what you're actually after.

Cheers,
Richard ( oh, and if example code is needed, I can provide )

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys

[Richard Levitte]

In message <1481043672.4406.22.ca...@hansenpartnership.com> on Tue, 06 Dec 2016 
09:01:12 -0800, James Bottomley <james.bottom...@hansenpartnership.com> said:

James.Bottomley> On Tue, 2016-12-06 at 17:47 +0100, Richard Levitte wrote:
James.Bottomley> > In message <1481042048.4406.14.ca...@hansenpartnership.com> 
on Tue,
James.Bottomley> > 06 Dec 2016 08:34:08 -0800, James Bottomley <
James.Bottomley> > james.bottom...@hansenpartnership.com> said:
James.Bottomley> > 
James.Bottomley> > James.Bottomley> On Tue, 2016-12-06 at 15:12 +0100, Richard 
Levitte
James.Bottomley> > wrote:
James.Bottomley> > James.Bottomley> > In message <
James.Bottomley> > 1480697558.2410.33.ca...@hansenpartnership.com> on Fri,
James.Bottomley> > James.Bottomley> > 02 Dec 2016 08:52:38 -0800, James 
Bottomley <
James.Bottomley> > James.Bottomley> > james.bottom...@hansenpartnership.com> 
said:
James.Bottomley> > James.Bottomley> > 
James.Bottomley> > James.Bottomley> > When I made that argument, I hadn't 
thought and
James.Bottomley> > felt things through
James.Bottomley> > James.Bottomley> > entirely.  Truth be told, I'm feeling 
very uneasy
James.Bottomley> > handing over the
James.Bottomley> > James.Bottomley> > reading and parsing of the PEM file to an 
engine. 
James.Bottomley> > However, handing
James.Bottomley> > James.Bottomley> > over the decoded data and leaving it to 
the engine
James.Bottomley> > to do something
James.Bottomley> > James.Bottomley> > sensible with it, no issues at all.
James.Bottomley> > James.Bottomley> 
James.Bottomley> > James.Bottomley> OK, can I pick on this a bit (I'll look at 
the store
James.Bottomley> > stuff later and
James.Bottomley> > James.Bottomley> reply after I've played with it)?  What is 
it that
James.Bottomley> > you imagine handing
James.Bottomley> > James.Bottomley> to the engine?  Some type of buffer that 
contains
James.Bottomley> > the full PEM
James.Bottomley> > James.Bottomley> representation?
James.Bottomley> > 
James.Bottomley> > In my STORE branch, you will find this in
James.Bottomley> > include/openssl/store_file.h:
James.Bottomley> > 
James.Bottomley> > /*
James.Bottomley> >  * The try_decode function is called to check if the 
blob of data
James.Bottomley> > can
James.Bottomley> >  * be used by this handler, and if it can, decodes it 
into a
James.Bottomley> > supported
James.Bottomley> >  * OpenSSL and returns a STORE_INFO with the recorded 
data.
James.Bottomley> >  * Input:
James.Bottomley> >  *pem_name: If this blob comes from a PEM file, 
this
James.Bottomley> > holds
James.Bottomley> >  *  the PEM name.  If it comes from 
another type
James.Bottomley> > of
James.Bottomley> >  *  file, this is NULL.
James.Bottomley> >  *blob: The blob of data to match with what 
this
James.Bottomley> > handler
James.Bottomley> >  *  can use.
James.Bottomley> >  *len:  The length of the blob.
James.Bottomley> >  *handler_ctx:  For a handler marked repeatable, 
this pointer
James.Bottomley> > can
James.Bottomley> >  *  be used to create a context for the 
handler. 
James.Bottomley> > IT IS
James.Bottomley> >  *  THE HANDLER'S RESPONSIBILITY TO 
CREATE AND
James.Bottomley> > DESTROY
James.Bottomley> >  *  THIS CONTEXT APPROPRIATELY, i.e. 
create on
James.Bottomley> > first call
James.Bottomley> >  *  and destroy when about to return 
NULL.
James.Bottomley> >  *password_ui:  Application UI method for getting a 
password.
James.Bottomley> >  *password_ui_data:
James.Bottomley> >  *  Application data to be passed to 
password_ui
James.Bottomley> > when
James.Bottomley> >  *  it's called.
James.Bottomley> >  * Output:
James.Bottomley> >  *a STORE_INFO
James.Bottomley> >  */
James.Bottomley> > typedef STORE_INFO *(*STORE_FILE_try_decode_fn)(const 
char
James.Bottomley> > *pem_name,
James.Bottomley> > const 
unsigned
James.Bottomley> > char *blob,
James.Bottomley> > size_t 
len,
James.Bottomley> > void
James.Bottomley> > **handler_ctx,
James.Bottomley> > const 
UI_

Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys

[Richard Levitte]

In message <1481042048.4406.14.ca...@hansenpartnership.com> on Tue, 06 Dec 2016 
08:34:08 -0800, James Bottomley <james.bottom...@hansenpartnership.com> said:

James.Bottomley> On Tue, 2016-12-06 at 15:12 +0100, Richard Levitte wrote:
James.Bottomley> > In message <1480697558.2410.33.ca...@hansenpartnership.com> 
on Fri,
James.Bottomley> > 02 Dec 2016 08:52:38 -0800, James Bottomley <
James.Bottomley> > james.bottom...@hansenpartnership.com> said:
James.Bottomley> > 
James.Bottomley> > When I made that argument, I hadn't thought and felt things 
through
James.Bottomley> > entirely.  Truth be told, I'm feeling very uneasy handing 
over the
James.Bottomley> > reading and parsing of the PEM file to an engine.  However, 
handing
James.Bottomley> > over the decoded data and leaving it to the engine to do 
something
James.Bottomley> > sensible with it, no issues at all.
James.Bottomley> 
James.Bottomley> OK, can I pick on this a bit (I'll look at the store stuff 
later and
James.Bottomley> reply after I've played with it)?  What is it that you imagine 
handing
James.Bottomley> to the engine?  Some type of buffer that contains the full PEM
James.Bottomley> representation?

In my STORE branch, you will find this in include/openssl/store_file.h:

/*
 * The try_decode function is called to check if the blob of data can
 * be used by this handler, and if it can, decodes it into a supported
 * OpenSSL and returns a STORE_INFO with the recorded data.
 * Input:
 *pem_name: If this blob comes from a PEM file, this holds
 *  the PEM name.  If it comes from another type of
 *  file, this is NULL.
 *blob: The blob of data to match with what this handler
 *  can use.
 *len:  The length of the blob.
 *handler_ctx:  For a handler marked repeatable, this pointer can
 *  be used to create a context for the handler.  IT IS
 *  THE HANDLER'S RESPONSIBILITY TO CREATE AND DESTROY
 *  THIS CONTEXT APPROPRIATELY, i.e. create on first call
 *  and destroy when about to return NULL.
 *password_ui:  Application UI method for getting a password.
 *password_ui_data:
 *  Application data to be passed to password_ui when
 *  it's called.
 * Output:
 *a STORE_INFO
 */
typedef STORE_INFO *(*STORE_FILE_try_decode_fn)(const char *pem_name,
const unsigned char *blob,
size_t len,
void **handler_ctx,
const UI_METHOD 
*password_ui,
void *password_ui_data);

This is the central function that tries to see if it can decode
whatever's thrown at it.  As you can see, it gets handed the PEM name
if it's called as part of a PEM read.  The |blob| is the decoded *and
decrypted* data.

James.Bottomley> The reason I chose a BIO is that it's the basic abstract data 
handler
James.Bottomley> for openssl. I can hand a buffer to the engine, sure, but I'd 
need to
James.Bottomley> transform it to a BIO again (because it would need PEM parsing 
and all
James.Bottomley> the PEM parsers speak BIOs), so it feels suboptimal.

With a try_decode function, I really do not see why there's a need to
deal with the BIO...

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys

[Richard Levitte]

In message <1480697558.2410.33.ca...@hansenpartnership.com> on Fri, 02 Dec 2016 
08:52:38 -0800, James Bottomley <james.bottom...@hansenpartnership.com> said:

James.Bottomley> On Thu, 2016-12-01 at 09:30 +0100, Richard Levitte wrote:
James.Bottomley> > 
James.Bottomley> > James Bottomley <james.bottom...@hansenpartnership.com> 
skrev: (1
James.Bottomley> > december 2016 07:36:26 CET)
James.Bottomley> > > On Thu, 2016-12-01 at 01:38 +0100, Richard Levitte wrote:
James.Bottomley> > > > 
James.Bottomley> > > > James Bottomley <james.bottom...@hansenpartnership.com> 
skrev: (1
James.Bottomley> > > > december 2016 00:42:09 CET)
James.Bottomley> [...]
James.Bottomley> > > > > On Thu, 2016-12-01 at 00:22 +0100, Richard Levitte 
wrote:
James.Bottomley> > > > > > Generally speaking, I am unsure about your solution. 
It seems
James.Bottomley> > > > > > like hack to fit a specific case where something 
more general
James.Bottomley> > > > > > could be of greater service to others as well.
James.Bottomley> > > > > 
James.Bottomley> > > > > Well, the more adaptable patch set was the previous 
one that 
James.Bottomley> > > > > overloaded the meaning of key_id.  This one has a 
specific bio 
James.Bottomley> > > > > mechanism for loading PEM files, so it only really 
works for 
James.Bottomley> > > > > engines that have PEM representable unloaded keys 
(which, to be
James.Bottomley> > > > > fair, is almost all of them, since even the USB crypto 
keys 
James.Bottomley> > > > > have a wrapped format).
James.Bottomley> > > > > 
James.Bottomley> > > > > I've tried to make it as generic as possible, but I am 
James.Bottomley> > > > > conditioned to think to my use case: TPM keys.  If you 
give an 
James.Bottomley> > > > > example of another use case, it will help me see where 
it 
James.Bottomley> > > > > should be more generic.
James.Bottomley> > > > 
James.Bottomley> > > > Among other things, I'd rather not encourage an API that 
James.Bottomley> > > > inherently makes the engine<->libcrypto tie even 
tighter. Also, 
James.Bottomley> > > > it puts a requirement on the engine to parse PEM, which 
is
James.Bottomley> > > > unnecessary.
James.Bottomley> 
James.Bottomley> Actually, I missed this initially.  This is definitely not a
James.Bottomley> requirement: engines that wish to parse PEM keys may do so, 
but an
James.Bottomley> engine that doesn't simply doesn't implement the callback.  
Only
James.Bottomley> engines that are loaded from the configuration file actually 
get asked
James.Bottomley> if they understand the key, and then only if they supply the 
callback,
James.Bottomley> so this is a decision on which engines to is made by the admin 
or
James.Bottomley> packager (and the engine builder, who decides whether to 
implement or
James.Bottomley> not).

When I made that argument, I hadn't thought and felt things through
entirely.  Truth be told, I'm feeling very uneasy handing over the
reading and parsing of the PEM file to an engine.  However, handing
over the decoded data and leaving it to the engine to do something
sensible with it, no issues at all.

James.Bottomley> If I summarise the arguments about self identifying files from 
the v1
James.Bottomley> thread:
James.Bottomley> 
James.Bottomley>1. We agreed that usability is greatly enhanced if openssl 
simply loads
James.Bottomley>   a key when presented with the file/uri etc. without the 
user having
James.Bottomley>   to specify what the format of a key is

Check.  My STORE branch is made to support that.

James.Bottomley>2. For PEM files, we believe this is easy because the 
guards uniquely
James.Bottomley>   identify the file format, so whatever key the 
application needs, we
James.Bottomley>   can verify is contained in the thing presented

Check.

James.Bottomley>3. THere was more debate on the files that aren't fully self
James.Bottomley>   describing, like DER.  The argument was that the DER 
structure is
James.Bottomley>   usually unique enough, so we could do this, but there 
were
James.Bottomley>   reservations.

Yup...  The current TSS KEY BLOB is among those reservations ;-)

James.Bottomley>4. I thought there was agreement that we could move 
forwards with the
James.Bottomley>   PEM bit because it was uncontroversial

We're not quite on the same page regarding the exact implementation.
However, I think I have a solution...

James.Bottomley> The current PEM key loading code is already hooked for PKCS8, 
to make
Ja

[openssl-dev] STORE (was: [RFC 0/2] Proposal for seamless handling of TPM based RSA keys in openssl)

[Richard Levitte]

In message <1479823032.8937.37.ca...@infradead.org> on Tue, 22 Nov 2016 
13:57:12 +, David Woodhouse <dw...@infradead.org> said:

dwmw2> On Tue, 2016-11-22 at 14:18 +0100, Richard Levitte wrote:
dwmw2> > 
dwmw2> > Just let me shamelessly mention my STORE effort again ;-)
dwmw2> > Among others, it does attempt to solve that very problem (in the
dwmw2> > 'file' scheme handler).
dwmw2> 
dwmw2> Neat. Note that I have a ready-made test suite for you in OpenConnect:
dwmw2> 
http://git.infradead.org/users/dwmw2/openconnect.git/blob/HEAD:/tests/Makefile.am
dwmw2> 
dwmw2> For every one of the key files therein, does your current
dwmw2> implementation work? :)

It does now for everything but PKCS#12 files.  Those are trickier, and
I'm working on it.

Speaking of...  do you mind if I pilfer from that Makefile.am for our
tests?

dwmw2> (Yeah, I need to sort out the tpm emulator in the test environment,
dwmw2> then add some -BEGIN TSS KEY BLOB- files too :)

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys

[Richard Levitte]

James Bottomley <james.bottom...@hansenpartnership.com> skrev: (1 december 2016 
07:36:26 CET)
>On Thu, 2016-12-01 at 01:38 +0100, Richard Levitte wrote:
>> 
>> James Bottomley <james.bottom...@hansenpartnership.com> skrev: (1
>> december 2016 00:42:09 CET)
>> > On Thu, 2016-12-01 at 00:22 +0100, Richard Levitte wrote:
>> > > This patch doesn't fit the rest... 
>> > 
>> > I'm not quite sure I follow why.
>> 
>> It casts bp to const char *. That was for your earlier
>> implementation, wasn't it? It doesn't fit the latest incarnation of
>> your API. 
>
>Good catch: a leftover I missed the warning for in the spray of other
>compile information.  Apparently the debug-linux-x86_64 target doesn't
>have a -Werror ... I've added it so I should pick a problem like this
>up easily.

The debug config targets create binaries that can easily be run through a 
debugger, nothing more. What you're looking for is the configuration option 
--strict-warnings. 

>> > > Generally speaking, I am unsure about your solution. It seems 
>> > > like hack to fit a specific case where something more general 
>> > > could be of greater service to others as well.
>> > 
>> > Well, the more adaptable patch set was the previous one that 
>> > overloaded the meaning of key_id.  This one has a specific bio 
>> > mechanism for loading PEM files, so it only really works for 
>> > engines that have PEM representable unloaded keys (which, to be 
>> > fair, is almost all of them, since even the USB crypto keys have a
>> > wrapped format).
>> > 
>> > I've tried to make it as generic as possible, but I am conditioned 
>> > to think to my use case: TPM keys.  If you give an example of 
>> > another use case, it will help me see where it should be more
>> > generic.
>> 
>> Among other things, I'd rather not encourage an API that inherently
>> makes the engine<->libcrypto tie even tighter. Also, it puts a
>> requirement on the engine to parse PEM, which is unnecessary. 
>> 
>> Also, we already have thoughts on loading keys by uri references, and
>> there may be new ideas and formats coming in the future. All this is
>> tied together and solving it one small hack at a time is sub-optimal
>> in the long run. 
>> 
>> I'll repeat myself again, please have a look at the STORE stuff I'm
>> working on. TPM files could very well serve as a first use case. 
>
>That's this new pull request:
>
>https://github.com/openssl/openssl/pull/2011/files

Yes. 

>Just so I understand you, you mean register a store handler from the
>engine for the TPM BLOB PEM files so that a reference to the file via
>the store can use the PEM file as an EVP_PKEY?

Essentially, yes. 

>That will work, I'm just not clear from the current pull request how
>the store would be integrated with the existing PEM file handler ...
>Will every application have to be modified to use the new store, or
>will you hook it like I did for the engine keys?

Generally speaking, I was thinking that applications would move to use the 
STORE API. 

>The think I'm looking for is to have a TPM PEM key file just work in
>place of a standard RSA private key file.

I get your point. That could be med as a call after the PEM_bytes_read_bio 
call. At that point, the necessary data for an internal store loading are 
available. It's still a hack, but if you must... 

>
>James
>
>
>> > James
>> > 
>> > 
>> > > Cheers 
>> > > Richard 
>> > > 
>> > > On November 30, 2016 4:27:49 PM GMT+01:00, James Bottomley <
>> > > james.bottom...@hansenpartnership.com> wrote:
>> > > > Before trying to process the PEM file, hand it to each of the
>> > > > loaded
>> > > > engines to see if they recognise the PEM guards.  This uses the
>> > > > new
>> > > > bio based load key callback, so the engine must be loaded and
>> > > > implement this callback to be considered.
>> > > > 
>> > > > Signed-off-by: James Bottomley <j...@linux.vnet.ibm.com>
>> > > > ---
>> > > > crypto/pem/pem_pkey.c | 4 
>> > > > 1 file changed, 4 insertions(+)
>> > > > 
>> > > > diff --git a/crypto/pem/pem_pkey.c b/crypto/pem/pem_pkey.c
>> > > > index 04d6319..e3737f0 100644
>> > > > --- a/crypto/pem/pem_pkey.c
>> > > > +++ b/crypto/pem/pem_pkey.c
>> > > > @@ -85,6 +85,10 @@ EVP_PKEY *PEM_read_bio_PrivateKey(BIO *bp,
>> &g

Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys

[Richard Levitte]

James Bottomley <james.bottom...@hansenpartnership.com> skrev: (1 december 2016 
00:42:09 CET)
>On Thu, 2016-12-01 at 00:22 +0100, Richard Levitte wrote:
>> This patch doesn't fit the rest... 
>
>I'm not quite sure I follow why.

It casts bp to const char *. That was for your earlier implementation, wasn't 
it? It doesn't fit the latest incarnation of your API. 

>> Generally speaking, I am unsure about your solution. It seems like
>> hack to fit a specific case where something more general could be of
>> greater service to others as well. 
>
>Well, the more adaptable patch set was the previous one that overloaded
>the meaning of key_id.  This one has a specific bio mechanism for
>loading PEM files, so it only really works for engines that have PEM
>representable unloaded keys (which, to be fair, is almost all of them,
>since even the USB crypto keys have a wrapped format).
>
>I've tried to make it as generic as possible, but I am conditioned to
>think to my use case: TPM keys.  If you give an example of another use
>case, it will help me see where it should be more generic.

Among other things, I'd rather not encourage an API that inherently makes the 
engine<->libcrypto tie even tighter. Also, it puts a requirement on the engine 
to parse PEM, which is unnecessary. 

Also, we already have thoughts on loading keys by uri references, and there may 
be new ideas and formats coming in the future. All this is tied together and 
solving it one small hack at a time is sub-optimal in the long run. 

I'll repeat myself again, please have a look at the STORE stuff I'm working on. 
TPM files could very well serve as a first use case. 

>James
>
>
>> Cheers 
>> Richard 
>> 
>> On November 30, 2016 4:27:49 PM GMT+01:00, James Bottomley <
>> james.bottom...@hansenpartnership.com> wrote:
>> > Before trying to process the PEM file, hand it to each of the
>> > loaded
>> > engines to see if they recognise the PEM guards.  This uses the new
>> > bio based load key callback, so the engine must be loaded and
>> > implement this callback to be considered.
>> > 
>> > Signed-off-by: James Bottomley <j...@linux.vnet.ibm.com>
>> > ---
>> > crypto/pem/pem_pkey.c | 4 
>> > 1 file changed, 4 insertions(+)
>> > 
>> > diff --git a/crypto/pem/pem_pkey.c b/crypto/pem/pem_pkey.c
>> > index 04d6319..e3737f0 100644
>> > --- a/crypto/pem/pem_pkey.c
>> > +++ b/crypto/pem/pem_pkey.c
>> > @@ -85,6 +85,10 @@ EVP_PKEY *PEM_read_bio_PrivateKey(BIO *bp,
>> > EVP_PKEY
>> > **x, pem_password_cb *cb,
>> > int slen;
>> > EVP_PKEY *ret = NULL;
>> > 
>> > +/* first check to see if an engine can load the PEM */
>> > +if (ENGINE_find_engine_load_key(NULL, , (const char *)bp,
>> > cb,
>> > u) == 1)
>> > +return ret;
>> > +
>> > if (!PEM_bytes_read_bio(, , , PEM_STRING_EVP_PKEY, bp,
>> > cb,
>> > u))
>> > return NULL;
>> > p = data;
>> 
>> -- 
>> levi...@openssl.org 

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [RFC v2 2/2] pem: load engine keys

[Richard Levitte]

This patch doesn't fit the rest... 

Generally speaking, I am unsure about your solution. It seems like hack to fit 
a specific case where something more general could be of greater service to 
others as well. 

Cheers 
Richard 

On November 30, 2016 4:27:49 PM GMT+01:00, James Bottomley 
 wrote:
>Before trying to process the PEM file, hand it to each of the loaded
>engines to see if they recognise the PEM guards.  This uses the new
>bio based load key callback, so the engine must be loaded and
>implement this callback to be considered.
>
>Signed-off-by: James Bottomley 
>---
> crypto/pem/pem_pkey.c | 4 
> 1 file changed, 4 insertions(+)
>
>diff --git a/crypto/pem/pem_pkey.c b/crypto/pem/pem_pkey.c
>index 04d6319..e3737f0 100644
>--- a/crypto/pem/pem_pkey.c
>+++ b/crypto/pem/pem_pkey.c
>@@ -85,6 +85,10 @@ EVP_PKEY *PEM_read_bio_PrivateKey(BIO *bp, EVP_PKEY
>**x, pem_password_cb *cb,
> int slen;
> EVP_PKEY *ret = NULL;
> 
>+/* first check to see if an engine can load the PEM */
>+if (ENGINE_find_engine_load_key(NULL, , (const char *)bp, cb,
>u) == 1)
>+return ret;
>+
>if (!PEM_bytes_read_bio(, , , PEM_STRING_EVP_PKEY, bp, cb,
>u))
> return NULL;
> p = data;

-- 
levi...@openssl.org 
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [RFC 0/2] Proposal for seamless handling of TPM based RSA keys in openssl

[Richard Levitte]

In message <1479993631.8937.91.ca...@infradead.org> on Thu, 24 Nov 2016 
13:20:31 +, David Woodhouse <dw...@infradead.org> said:

dwmw2> On Wed, 2016-11-23 at 22:33 +0100, Richard Levitte wrote:
dwmw2> > That being said, though, your recommendation should probably specify
dwmw2> > (after discussing it) exactly what keys, certs and so on should be
dwmw2> > supported. Otherwise, everyone will have a slightly different idea of
dwmw2> > what's reasonable and you will end up in the same space as today... 
dwmw2> 
dwmw2> Oh $DEITY yes, that's the whole point. And I don't think I've left much
dwmw2> ambiguity there. As ever, suggestions for improvement would be most
dwmw2> welcome...

dwmw2> http://david.woodhou.se/draft-woodhouse-cert-best-practice.html#formats
dwmw2> 
dwmw2> 5.  File formats
dwmw2> 
dwmw2>    ...
...

D'oh, I feel silly now.

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [RFC 0/2] Proposal for seamless handling of TPM based RSA keys in openssl

[Richard Levitte]

Richard Levitte <levi...@openssl.org> skrev: (23 november 2016 22:23:18 CET)
>
>
>David Woodhouse <dw...@infradead.org> skrev: (23 november 2016 19:42:29
>CET)
>>On Wed, 2016-11-23 at 17:00 +, Salz, Rich wrote:
>>> 
>>> > FWIW I am perfectly content for applications *not* to
>automatically
>>work
>>> > with such keys. Making the user jump through extra hoops to use
>>them
>>> > would be perfectly fine in my book.
>>> 
>>> oh I see.  "Users shouldn't care, it should just work"  But only for
>>some keys.
>>> 
>>> Part of my I am opposed to guessing.
>>
>>For me it's the other way round. Magically detecting *that* particular
>>perfectly valid PKCS#1 RSA key is actually intended for the gem engine
>>would indeed be guessing. It's a bizarre abuse of PKCS#1 and it
>doesn't
>>seem reasonable for anyone to "guess" that without explicit direction.
>>
>>But for the sane and common cases of PKCS#1, PKCS#8, PKCS#12 and
>>similar files in both DER and PEM forms, for *those* it makes sense
>for
>>applications to Just Work. And it shouldn't really involve "guessing".
>
>I take that as "recognizing what we decide to support". And as has
>already been mentioned, we already do that with d2i_AutoPrivatekey. 

That being said, though, your recommendation should probably specify (after 
discussing it) exactly what keys, certs and so on should be supported. 
Otherwise, everyone will have a slightly different idea of what's reasonable 
and you will end up in the same space as today... 

Cheers 
Richard 

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [RFC 0/2] Proposal for seamless handling of TPM based RSA keys in openssl

[Richard Levitte]

David Woodhouse  skrev: (23 november 2016 19:42:29 CET)
>On Wed, 2016-11-23 at 17:00 +, Salz, Rich wrote:
>> 
>> > FWIW I am perfectly content for applications *not* to automatically
>work
>> > with such keys. Making the user jump through extra hoops to use
>them
>> > would be perfectly fine in my book.
>> 
>> oh I see.  "Users shouldn't care, it should just work"  But only for
>some keys.
>> 
>> Part of my I am opposed to guessing.
>
>For me it's the other way round. Magically detecting *that* particular
>perfectly valid PKCS#1 RSA key is actually intended for the gem engine
>would indeed be guessing. It's a bizarre abuse of PKCS#1 and it doesn't
>seem reasonable for anyone to "guess" that without explicit direction.
>
>But for the sane and common cases of PKCS#1, PKCS#8, PKCS#12 and
>similar files in both DER and PEM forms, for *those* it makes sense for
>applications to Just Work. And it shouldn't really involve "guessing".

I take that as "recognizing what we decide to support". And as has already been 
mentioned, we already do that with d2i_AutoPrivatekey. 

Cheers 
Richard 

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] STORE (was: [RFC 0/2] Proposal for seamless handling of TPM based RSA keys in openssl)

[Richard Levitte]

[subject change]

In message 
<3d837eb338bb47a68938676967ed1...@usma1ex-dag1mb1.msg.corp.akamai.com> on Wed, 
23 Nov 2016 14:41:14 +, "Salz, Rich" <rs...@akamai.com> said:

rsalz> 
rsalz> > Essentially, you're suggesting that we split out the matching part of 
the d2i
rsalz> > functions and put that to good use.  Or do you have some other idea, 
along
rsalz> > the lines if magic?
rsalz> 
rsalz> NO.  I am suggesting add one new routine that tries varies
rsalz> "convert to native" and returns which conversion worked.  And
rsalz> then another new routine that takes that return value and calls
rsalz> that conversion routine directly.  The cost of doing this is
rsalz> one extra d2i.  By the application.  And that first routine
rsalz> should ideally return a bitmask of all functions that succeeded
rsalz> so that handling ambiguities are built-in to the API.

Ok, I hear you, and this can be done.  However, since this is in STORE
terms, it will have to be a STORE API thing.  Not all URIs will give
you a DER blob to fiddle with at your heart's content, some of them
will just give you a key referense (which is best stored in a
EVP_PKEY).  Engine keys stored in the engine device are the prime
example.

rsalz> > rsalz> Security libraries *should not guess.*
rsalz> > 
rsalz> > Isn't telling the application "we think it's a FOO" guessing?  What's 
the
rsalz> > application going to do, go "nh, methinks it's a BAR" and try to 
decode
rsalz> > the blob as that (and most probably fail) rather than FOO?
rsalz> 
rsalz> It might.  Or it might throw up its hands and give up.  Or it
rsalz> might check to see if the file is ambiguous and do something.
rsalz> The point is, it is not openssl that is doing that.

Speaking of ambiguity, I was thinking of having my 'file' scheme
loader try all d2i's and having it "throw up its hands" if it found
more than one matching.  STOREerr(..., STORE_R_MABIGUOUS_CONTENT) type
of thing...  The application or users would be left to give some extra
information in the URI.

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [RFC 0/2] Proposal for seamless handling of TPM based RSA keys in openssl

[Richard Levitte]

In message 
<2360f57bb7504a328e5517ac92e19...@usma1ex-dag1mb1.msg.corp.akamai.com> on Wed, 
23 Nov 2016 13:51:03 +, "Salz, Rich" <rs...@akamai.com> said:

rsalz> 
rsalz> > Why is it different if we do exactly that in libcrypto?
rsalz> 
rsalz> Because *we* are not guessing.  We are telling the application
rsalz> "we think it's a FOO" and then letting the application decide
rsalz> what to do.

We don't have the functionality to do it that way, at all.  All we
have are the d2i functions, which will either return with an error
indication or return the fully parsed and decoded structure.

Essentially, you're suggesting that we split out the matching part of
the d2i functions and put that to good use.  Or do you have some other
idea, along the lines if magic?

rsalz> Security libraries *should not guess.*

Isn't telling the application "we think it's a FOO" guessing?  What's
the application going to do, go "nh, methinks it's a BAR" and try
to decode the blob as that (and most probably fail) rather than FOO?

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [RFC 0/2] Proposal for seamless handling of TPM based RSA keys in openssl

[Richard Levitte]

In message <1479908025.8937.74.ca...@infradead.org> on Wed, 23 Nov 2016 
13:33:45 +, David Woodhouse <dw...@infradead.org> said:

dwmw2> On Wed, 2016-11-23 at 13:13 +, Salz, Rich wrote:
dwmw2> > > But, what I get from you is "what if a octet stream matches two 
different
dwmw2> > > ASN.1 types?  Is that it?
dwmw2> > 
dwmw2> > Yes among others.  How do you know it will *never* happen?
dwmw2> 
dwmw2> Because if anyone tries to invent yet *another* ASN.1 form for storing
dwmw2> keys, I am going to personally visit them in the small hours and stick
dwmw2> a bat up their nightshirt?

(let's keep the heat down, shall we?)

dwmw2> Hopefully we don't need to add completely new ones; we can use the
dwmw2> existing PKCS#8 and PKCS#12 containers for new things.
dwmw2> 
dwmw2> But even if a new form is invented which is ambiguous with existing
dwmw2> forms, that's OK too. We don't support 'detection' of that new format
dwmw2> by its ASN.1 structure. It'll be PEM-only like the TSS blobs are unless
dwmw2> the type is explicitly specified.

Errr...  Now I'm confused.  Wasn't that (explicit type spec) exactly
what you didn't want to see, no matter if the file was PEM or raw DER?

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

[openssl-dev] STORE (was: [RFC 0/2] Proposal for seamless handling of TPM based RSA keys in openssl)

[Richard Levitte]

Change of subject, this part of the thread isn't so much TPM any more...

In message <1479823032.8937.37.ca...@infradead.org> on Tue, 22 Nov 2016 
13:57:12 +, David Woodhouse <dw...@infradead.org> said:

dwmw2> On Tue, 2016-11-22 at 14:18 +0100, Richard Levitte wrote:
dwmw2> > 
dwmw2> > Just let me shamelessly mention my STORE effort again ;-)
dwmw2> > Among others, it does attempt to solve that very problem (in the
dwmw2> > 'file' scheme handler).
dwmw2> 
dwmw2> Neat. Note that I have a ready-made test suite for you in OpenConnect:
dwmw2> 
http://git.infradead.org/users/dwmw2/openconnect.git/blob/HEAD:/tests/Makefile.am
dwmw2> 
dwmw2> For every one of the key files therein, does your current
dwmw2> implementation work? :)

Nope, not even for all the PEM files...  I'm seeing a number of them
(I noticed the PKCS#8 ones in particular) that don't return any data
at all.

I haven't look that deeply into PEM_X509_INFO_read_bio before, and it
seems to only cover a subset of all the types we recognise.
Considering it's undocumented, I'm wondering if that's the right
function to pursue.  The other option is to create a function in the
'file' scheme loader that does the same thing but with a table of
handlers.  I'm quite fine with that idea...

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [RFC 0/2] Proposal for seamless handling of TPM based RSA keys in openssl

[Richard Levitte]

In message 
<c5e2f2ebdd9c42be840c34cdbb7bf...@usma1ex-dag1mb1.msg.corp.akamai.com> on Wed, 
23 Nov 2016 13:13:05 +, "Salz, Rich" <rs...@akamai.com> said:

rsalz> 
rsalz> > Uh...  the d2i functions are already both in one.  Are you saying 
they
rsalz> > should be split in two, one part that does all the checking and the 
other that
rsalz> > just decodes, trusting that all checks are already done?  What you're 
gonna
rsalz> > do there is double part of the work.
rsalz> 
rsalz> Well, not double, but first do the cascade then return an
rsalz> indicator of which specific one worked.  Then the application
rsalz> can call a routine to again do the decode.

Why is it different if we do exactly that in libcrypto?

rsalz> If it bothers you, return the size as an output parameter.
rsalz> That fits with our i2d model.

Dunno about you, but I'm talking d2i, not i2d.  Different beast.

rsalz> > But, what I get from you is "what if a octet stream matches two 
different
rsalz> > ASN.1 types?  Is that it?
rsalz> 
rsalz> Yes among others.  How do you know it will *never* happen?

I don't, and in some cases (such as the TSS KEY BLOB), there will most
likely ONLY be a PEM decoder, which is a much easier ballgame.

Quite frankly, that's a though that should go back to David and his
demand of automatic detection.  If anyone would *ever* create a raw
DER file holding a tss OCTET STRING, then the file spec *will* have to
have an indication of what type of content it is.
If we're thinking in URI terms, I could think of a contraption like
file:whatever.pem?t=tsskeyblob ...  or dare I say it,
tpmkey:file=whatever.pem  (David is so going to hate me ;-) ...)


Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [RFC 0/2] Proposal for seamless handling of TPM based RSA keys in openssl

[Richard Levitte]

In message <1479894913.8937.58.ca...@infradead.org> on Wed, 23 Nov 2016 
09:55:13 +, David Woodhouse <dw...@infradead.org> said:

dwmw2> On Wed, 2016-11-23 at 09:56 +0100, Richard Levitte wrote:
dwmw2> > 
dwmw2> > 
dwmw2> > dwmw2> So maybe it's just "content types" that we have handlers for, 
each with
dwmw2> > dwmw2> an optional PEM tag for matching, *and* an optional match 
function
dwmw2> > dwmw2> which is given the parsed ASN.1 and checks if it's a match.
dwmw2> > 
dwmw2> > I'm not sure what you mean with a match function...  but going off on
dwmw2> > a limb, how about a reference to an OpenSSL style ASN1 description?
dwmw2> > So basically, for an imaginary TSS KEY BLOB (one that actually would
dwmw2> > use that TssBlob definition we talked about earlier), these three
dwmw2> > items would be specified:
dwmw2> > 
dwmw2> >     "TSS KEY BLOB",
dwmw2> >     ASN1_ITEM_rptr(TSS_BLOB),   /* TSS_BLOB ASN1 stuff defined in 
engine */
dwmw2> >     handler /* Essentially a d2i function */
dwmw2> > 
dwmw2> > Or did you mean that the matching would also involve checking the
dwmw2> > contents of the OCTET_STRING that TSS KEY BLOBs currently are, to see
dwmw2> > if they correspond to your structures?  If that's what you mean, my
dwmw2> > gut feeling tells me - and I haven't had my morning coffee yet - we're
dwmw2> > now moving into a territory where I fully expect dragons...  not to
dwmw2> > mention the worries Rich expressed.
dwmw2> 
dwmw2> Oh $DEITY no,  not looking in the contents of the OCTET_STRING.
dwmw2> Basically I'm thinking of doing precisely what d2i_AutoPrivateKey()
dwmw2> already does, just with expanded coverage and slightly less hard-coded
dwmw2> stuff.



dwmw2> Doing it like that would prevent any implementations (like the one in
dwmw2> the TPM engine) from being *tempted* to go prodding around in the
dwmw2> contents of OCTET STRINGs. Which is probably a feature :)

Right...

But then, embedding everything in an OCTET STRING isn't exactly a
novel idea either.  How do we discern a DER encoded TSS KEY BLOB from
whatever else that had the same "novel" idea? An OCTET STRING is an
OCTET STRING is an OCTET STRING...  See the dragons hovering over
there? ;-)

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [RFC 0/2] Proposal for seamless handling of TPM based RSA keys in openssl

[Richard Levitte]

In message <1479889418.8937.54.ca...@infradead.org> on Wed, 23 Nov 2016 
08:23:38 +, David Woodhouse <dw...@infradead.org> said:

dwmw2> On Tue, 2016-11-22 at 18:06 +0100, Richard Levitte wrote:
dwmw2> > 
dwmw2> > Actually, I agree with this, and that goes along with how our PEM
dwmw2> > routines work (specifically, PEM_X509_INFO_read_bio()), it just
dwmw2> > treats the data as an octet string and hands it down to a d2i routine
dwmw2> > of choice, with a pointer to where to place the result.
dwmw2> > 
dwmw2> > It's not very hard to imagine adding the possibility for external
dwmw2> > handlers for specific PEM content types, which could be handed an
dwmw2> > octet string and a pointer to a X509_INFO (which is the structure used
dwmw2> > to collect the data in), or something like that (I can also imagine
dwmw2> > having one separate handler for each type of data that can be
dwmw2> > returned, so one for a EVP_KEY, one for a X509, one for a X509_CRL,
dwmw2> > and so on and so forth).  That would make it possible for an engine to
dwmw2> > declare its own handler during the binding call, along with all other
dwmw2> > information it feeds back to libcrypto.
dwmw2> 
dwmw2> Yeah, something like that.
dwmw2> 
dwmw2> It's worth noting that the 'PEM content types' are just the same as the
dwmw2> 'DER content types'. It's just that if you have a PEM header, you know
dwmw2> precisely *which* DER content type you have after base64-decoding (and
dwmw2> decryption in some cases), and you don't have to do what
dwmw2> d2i_AutoPrivateKey() does to infer it from the ASN.1 structure.

True.

dwmw2> So maybe it's just "content types" that we have handlers for, each with
dwmw2> an optional PEM tag for matching, *and* an optional match function
dwmw2> which is given the parsed ASN.1 and checks if it's a match.

I'm not sure what you mean with a match function...  but going off on
a limb, how about a reference to an OpenSSL style ASN1 description?
So basically, for an imaginary TSS KEY BLOB (one that actually would
use that TssBlob definition we talked about earlier), these three
items would be specified:

"TSS KEY BLOB",
ASN1_ITEM_rptr(TSS_BLOB),   /* TSS_BLOB ASN1 stuff defined in engine */
handler /* Essentially a d2i function */

Or did you mean that the matching would also involve checking the
contents of the OCTET_STRING that TSS KEY BLOBs currently are, to see
if they correspond to your structures?  If that's what you mean, my
gut feeling tells me - and I haven't had my morning coffee yet - we're
now moving into a territory where I fully expect dragons...  not to
mention the worries Rich expressed.

Cheers,
Richard ( coffee.  Now! )

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [RFC 0/2] Proposal for seamless handling of TPM based RSA keys in openssl

[Richard Levitte]

In message <1479839148.2376.31.ca...@hansenpartnership.com> on Tue, 22 Nov 2016 
10:25:48 -0800, James Bottomley <james.bottom...@hansenpartnership.com> said:

James.Bottomley> On Tue, 2016-11-22 at 18:03 +, Salz, Rich wrote:
James.Bottomley> > > > It does this by trying to interpret the blob against 
known ASN.1
James.Bottomley> > > > definitions, and will only succeed when there's a 
complete match.
James.Bottomley> > > >   I'm
James.Bottomley> > > > not terribly worried...
James.Bottomley> > 
James.Bottomley> > I am.  With locales and UTF8, the old simple days of 
text/binary are
James.Bottomley> > probably long gone.  And if any ASN.1 definition has 
extensibility in
James.Bottomley> > it, then we have to be concerned about things being wrapped,
James.Bottomley> > something like prefix attacks, and so on.  
James.Bottomley> >  
James.Bottomley> > > And even if you were, you should be *more* worried about 
making
James.Bottomley> > > *applications* do it for themselves :)
James.Bottomley> > 
James.Bottomley> > I cannot control what an application does, and I am not 
responsible
James.Bottomley> > for any other application's reputation.  I do have a 
strongly vested
James.Bottomley> > stake in OpenSSL's. 
James.Bottomley> > 
James.Bottomley> > It is already possible to write a utility library that tries 
James.Bottomley> > everything in turn, and returns an enumeration that says 
"seems to be 
James.Bottomley> > an X509 certificate" etc.  And then another routine that 
takes that 
James.Bottomley> > enumeration and the blob and calls the right decoder.  I 
would be 
James.Bottomley> > okay with that, even if it were part of OpenSSL.  I am 
opposed to 
James.Bottomley> > guessing and parsing in one step, and would -1 any PR for 
that, 
James.Bottomley> > forcing a team discussion.
James.Bottomley> 
James.Bottomley> That's not the proposal.  The proposal is to use PEM form 
because we
James.Bottomley> can make it uniquely self describing using the guard tags which
James.Bottomley> obviates the problem above.

This is a side thread that discusses the 'file' scheme loader in my
STORE effort.  So, uhmmm, we're a bit away from just PEM here.
However, if we go back to the discussion about TSS KEY BLOBs, yeah,
I've only seen a PEM proposal, and that's a mch easier case.

James.Bottomley> On the larger issue of non-self describing formats like ASN.1: 
if your
James.Bottomley> theory that there's a security hole by allowing opportunistic 
format
James.Bottomley> detection is correct, simply making the user specify is 
palming our bug
James.Bottomley> off on to the user and abdicating responsibility because now 
when
James.Bottomley> they're tricked into an exploit they can be blamed not 
openssl.  If
James.Bottomley> such a bug exists, doing opportunistic format detection the 
better
James.Bottomley> guarantor of overall system security because if such a bug is 
found, it
James.Bottomley> would have to be fixed within openssl to everyone's benefit.

I agree with that sentiment.

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [RFC 0/2] Proposal for seamless handling of TPM based RSA keys in openssl

[Richard Levitte]

In message 
<021a5d5b885845f5ab79c4420232e...@usma1ex-dag1mb1.msg.corp.akamai.com> on Tue, 
22 Nov 2016 18:03:31 +, "Salz, Rich" <rs...@akamai.com> said:

rsalz> It is already possible to write a utility library that tries
rsalz> everything in turn, and returns an enumeration that says "seems
rsalz> to be an X509 certificate" etc.  And then another routine that
rsalz> takes that enumeration and the blob and calls the right
rsalz> decoder.  I would be okay with that, even if it were part of
rsalz> OpenSSL.  I am opposed to guessing and parsing in one step, and
rsalz> would -1 any PR for that, forcing a team discussion.

Uh...  the d2i functions are already both in one.  Are you saying
they should be split in two, one part that does all the checking and
the other that just decodes, trusting that all checks are already
done?  What you're gonna do there is double part of the work.

But, what I get from you is "what if a octet stream matches two
different ASN.1 types?  Is that it?

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [RFC 0/2] Proposal for seamless handling of TPM based RSA keys in openssl

[Richard Levitte]

In message 
<489af892b16b43ee9a7009ffe52db...@usma1ex-dag1mb1.msg.corp.akamai.com> on Tue, 
22 Nov 2016 17:40:54 +, "Salz, Rich" <rs...@akamai.com> said:

rsalz> > The more interesting part is when it tries to load files it guesses 
are raw DER.
rsalz> 
rsalz> And this part worries me.  I do not think a "security library" should be 
guessing.

It does this by trying to interpret the blob against known ASN.1
definitions, and will only succeed when there's a complete match.  I'm
not terribly worried...

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [RFC 0/2] Proposal for seamless handling of TPM based RSA keys in openssl

[Richard Levitte]

In message <1479833048.2376.21.ca...@hansenpartnership.com> on Tue, 22 Nov 2016 
08:44:08 -0800, James Bottomley <james.bottom...@hansenpartnership.com> said:

James.Bottomley> On Tue, 2016-11-22 at 16:28 +, David Woodhouse wrote:
James.Bottomley> > On Tue, 2016-11-22 at 17:21 +0100, Richard Levitte wrote:
James.Bottomley> > > 
James.Bottomley> > > dwmw2> It is *only* the OCTET STRING of the blob itself. 
Everything
James.Bottomley> > > else is
James.Bottomley> > > dwmw2> redundant anyway.
James.Bottomley> > > 
James.Bottomley> > > Oh!  Ok, that makes things much simpler (at least in a way)
James.Bottomley> > 
James.Bottomley> > Kind of. But then again, there's an argument that it was 
none of your
James.Bottomley> > business anyway. If it says "BEGIN TSS KEY BLOB" you hand it 
off to
James.Bottomley> > the
James.Bottomley> > TPM engine and after that you really don't care about what's 
in it.
James.Bottomley> > 
James.Bottomley> > Once upon a time, the TPM engine wrote those TPM_KEY blobs 
to binary
James.Bottomley> > files (no ASN.1 at all). For some reason it didn't use the 
TssBlob
James.Bottomley> > object type, although perhaps it should.
James.Bottomley> > 
James.Bottomley> > When I started looking at it, I used the -BEGIN TSS KEY 
BLOB-
James.Bottomley> > for an OCTET STRING containing *just* that the code had 
previously
James.Bottomley> > been
James.Bottomley> > writing into its binary files.
James.Bottomley> > 
James.Bottomley> > If I'd been aware of the TssBlob definition at that time, I 
suppose I
James.Bottomley> > would have used it instead of just the OCTET STRING. But I 
didn't.
James.Bottomley> > 
James.Bottomley> > If we write an I-D covering the TPM keys, perhaps the PEM 
contents
James.Bottomley> > should be permitted to be *either* a raw OCTET STRING with 
the key
James.Bottomley> > blob, OR a TssBlob object. Or maybe we should add a
James.Bottomley> > BEGIN TSS BLOB- (without 'KEY' in it) instead?
James.Bottomley> 
James.Bottomley> Before we rathole on this: if we use the current code to just 
hand off
James.Bottomley> to the engine, openssl never needs to know the format of the 
key files
James.Bottomley> or even what they mean.  If we hard code recognising TPM keys 
into
James.Bottomley> openssl, we are going to have to agree (and stick to) a key 
format. 
James.Bottomley>  This is one of the decisions that needs making to transform 
the RFC
James.Bottomley> into a real patch.
James.Bottomley> 
James.Bottomley> I will note that gnutls does hard code recognising TPM keys so 
there's
James.Bottomley> precedent for doing it that way.  However, I have a marginal 
preference
James.Bottomley> for letting the loaded engines do it because that's the way 
that gives
James.Bottomley> most flexibility with regard to new engines as they come 
along.  This
James.Bottomley> problem isn't theoretical: TPM 2.0 keys are very different 
from TPM 1.2
James.Bottomley> ones, so they'll likely have a new engine to handle them and a 
new file
James.Bottomley> format.

Actually, I agree with this, and that goes along with how our PEM
routines work (specifically, PEM_X509_INFO_read_bio()), it just
treats the data as an octet string and hands it down to a d2i routine
of choice, with a pointer to where to place the result.

It's not very hard to imagine adding the possibility for external
handlers for specific PEM content types, which could be handed an
octet string and a pointer to a X509_INFO (which is the structure used
to collect the data in), or something like that (I can also imagine
having one separate handler for each type of data that can be
returned, so one for a EVP_KEY, one for a X509, one for a X509_CRL,
and so on and so forth).  That would make it possible for an engine to
declare its own handler during the binding call, along with all other
information it feeds back to libcrypto.

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [RFC 0/2] Proposal for seamless handling of TPM based RSA keys in openssl

[Richard Levitte]

In message <1479830167.8937.43.ca...@infradead.org> on Tue, 22 Nov 2016 
15:56:07 +, David Woodhouse <dw...@infradead.org> said:

dwmw2> On Tue, 2016-11-22 at 16:32 +0100, Richard Levitte wrote:
dwmw2> > In message <1479815862.8937.22.ca...@infradead.org> on Tue, 22 Nov 
2016 11:57:42 +, David Woodhouse <dw...@infradead.org> said:
dwmw2> > 
dwmw2> > dwmw2> Besides, it requires files in the form described by the 
Portable Data
dwmw2> > dwmw2> section of the TSS (1.2) spec. That's a SEQUENCE with a blob 
type
dwmw2> > dwmw2> (which is mostly redundant as in this case we're always talking 
about
dwmw2> > dwmw2> key blobs), the blob length (which is entirely redundant) and 
then the
dwmw2> > dwmw2> actual blob as an OCTET STRING. I don't know of any tool which 
actually
dwmw2> > dwmw2> creates such files.
dwmw2> > 
dwmw2> > I'm just having a look at the spec (page 151 in
dwmw2> > 
http://www.trustedcomputinggroup.org/wp-content/uploads/TSS_1_2_Errata_A-final.pdf),
dwmw2> > and am a bit confused by the TssBlobType type.  Which is it in
dwmw2> > practice, an ENUMERATED or an INTEGER?
dwmw2> 
dwmw2> In practice, it doesn't get used at all. The object encoded with
dwmw2> -BEGIN TSS KEY BLOB- and used by both the OpenSSL TPM ENGINE
dwmw2> and by GnuTLS is not the TssBlob object that you're looking at.
dwmw2> 
dwmw2> It is *only* the OCTET STRING of the blob itself. Everything else is
dwmw2> redundant anyway.

Oh!  Ok, that makes things much simpler (at least in a way)

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [RFC 0/2] Proposal for seamless handling of TPM based RSA keys in openssl

[Richard Levitte]

In message <1479829450.2376.10.ca...@hansenpartnership.com> on Tue, 22 Nov 2016 
07:44:10 -0800, James Bottomley <james.bottom...@hansenpartnership.com> said:

James.Bottomley> On Tue, 2016-11-22 at 16:32 +0100, Richard Levitte wrote:
James.Bottomley> > In message <1479815862.8937.22.ca...@infradead.org> on Tue, 
22 Nov
James.Bottomley> > 2016 11:57:42 +, David Woodhouse <dw...@infradead.org> 
said:
James.Bottomley> > 
James.Bottomley> > dwmw2> Besides, it requires files in the form described by 
the
James.Bottomley> > Portable Data
James.Bottomley> > dwmw2> section of the TSS (1.2) spec. That's a SEQUENCE with 
a blob
James.Bottomley> > type
James.Bottomley> > dwmw2> (which is mostly redundant as in this case we're 
always
James.Bottomley> > talking about
James.Bottomley> > dwmw2> key blobs), the blob length (which is entirely 
redundant) and
James.Bottomley> > then the
James.Bottomley> > dwmw2> actual blob as an OCTET STRING. I don't know of any 
tool which
James.Bottomley> > actually
James.Bottomley> > dwmw2> creates such files.
James.Bottomley> > 
James.Bottomley> > I'm just having a look at the spec (page 151 in
James.Bottomley> > 
http://www.trustedcomputinggroup.org/wp-content/uploads/TSS_1_2_Errat
James.Bottomley> > a_A-final.pdf), and am a bit confused by the TssBlobType 
type.  Which 
James.Bottomley> > is it in practice, an ENUMERATED or an INTEGER?
James.Bottomley> 
James.Bottomley> It's actually here:
James.Bottomley> 
James.Bottomley> 
http://trustedcomputinggroup.org/wp-content/uploads/TPM-Main-Part-2-TPM-Structures_v1.2_rev116_01032011.pdf
James.Bottomley> 
James.Bottomley> It's around page 101, section 10.3 the TPM_KEY12 structure.  
That tells
James.Bottomley> you what to encrypt and how to construct the encrypted part of 
the
James.Bottomley> blob.  It refers to other structures, so you end up doing a 
bit of a
James.Bottomley> pointer chase through the document.

I'm sorry, I have obviously had you a bit confused.  What I'm
currently interested in is decoding the content of a 'TSS KEY BLOB'
PEM file, and I assume that's a TssBlob type, yeah?

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [RFC 0/2] Proposal for seamless handling of TPM based RSA keys in openssl

[Richard Levitte]

In message <1479815862.8937.22.ca...@infradead.org> on Tue, 22 Nov 2016 
11:57:42 +, David Woodhouse <dw...@infradead.org> said:

dwmw2> Besides, it requires files in the form described by the Portable Data
dwmw2> section of the TSS (1.2) spec. That's a SEQUENCE with a blob type
dwmw2> (which is mostly redundant as in this case we're always talking about
dwmw2> key blobs), the blob length (which is entirely redundant) and then the
dwmw2> actual blob as an OCTET STRING. I don't know of any tool which actually
dwmw2> creates such files.

I'm just having a look at the spec (page 151 in
http://www.trustedcomputinggroup.org/wp-content/uploads/TSS_1_2_Errata_A-final.pdf),
and am a bit confused by the TssBlobType type.  Which is it in
practice, an ENUMERATED or an INTEGER?

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [RFC 0/2] Proposal for seamless handling of TPM based RSA keys in openssl

[Richard Levitte]

In message <1479823032.8937.37.ca...@infradead.org> on Tue, 22 Nov 2016 
13:57:12 +, David Woodhouse <dw...@infradead.org> said:

dwmw2> On Tue, 2016-11-22 at 14:18 +0100, Richard Levitte wrote:
dwmw2> > 
dwmw2> > Just let me shamelessly mention my STORE effort again ;-)
dwmw2> > Among others, it does attempt to solve that very problem (in the
dwmw2> > 'file' scheme handler).
dwmw2> 
dwmw2> Neat. Note that I have a ready-made test suite for you in OpenConnect:
dwmw2> 
http://git.infradead.org/users/dwmw2/openconnect.git/blob/HEAD:/tests/Makefile.am
dwmw2> 
dwmw2> For every one of the key files therein, does your current
dwmw2> implementation work? :)
dwmw2> 
dwmw2> (Yeah, I need to sort out the tpm emulator in the test environment,
dwmw2> then add some -BEGIN TSS KEY BLOB- files too :)

All I can see is PEM files...  For any PEM content that OpenSSL
supports, the STORE 'file' scheme loader does as well.  That's just a
one liner, quite precisely this one:

https://github.com/openssl/openssl/pull/1962/files#diff-34958ca30387ac75fa5011f98c18b3baR69

The more interesting part is when it tries to load files it guesses
are raw DER.  It's currently only trying a few chosen content types,
I'm happy to add more as time goes.  However, I suspect that nothing
in your test suite will trigger that part.

TSS KEY BLOBs is a different thing.  That needs added PEM support, and
the STORE 'file' scheme loader will not have to be changed at all.

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [RFC 0/2] Proposal for seamless handling of TPM based RSA keys in openssl

[Richard Levitte]

In message 
<da958b9e865a4268b95fd3e0b0774...@usma1ex-dag1mb1.msg.corp.akamai.com> on Tue, 
22 Nov 2016 14:42:35 +, "Salz, Rich" <rs...@akamai.com> said:

rsalz> > dwmw2> It should work out what the contents are for *itself*. Whether
rsalz> > dwmw2> they be PEM, DER, PKCS#n, TPM-wrapped blobs, or anything else.
rsalz> 
rsalz> I disagree with this approach, but that's just my opinion.  I am worried 
about "keep trying something until it works" because you'll get strange errors 
you can't decode, 'only allow N tries' devices will lock you out, and the order 
in which you try things could result in needless long delays.
rsalz> 
rsalz> But don't let that stop you.

I *think* the guessing part is just about the step of loading the file
content and transparently understanding what type of content it is.
That's basically looking at a bunch of bytes and recognising it for
what it is.  When that's done, the trial and error phase is over, and
for stuff that libcrypto has support for, libcrypto will be able to
act, deterministically.

>From the application point of view, this would be just one call, but
we are talking OpenSSL internals now, aren't we?

David, correct me if I got you wrong.

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [RFC 0/2] Proposal for seamless handling of TPM based RSA keys in openssl

[Richard Levitte]

In message <1479820334.8937.31.ca...@infradead.org> on Tue, 22 Nov 2016 
13:12:14 +, David Woodhouse <dw...@infradead.org> said:

dwmw2> On Tue, 2016-11-22 at 14:06 +0100, Richard Levitte wrote:
dwmw2> > 
dwmw2> > Not sure I follow...  'file=/foo/bar/key.pem' is just a path /
dwmw2> > parameter that the 'tpmkey' handler is free to interpret in whatever
dwmw2> > way it sees fit.  For me as a user, it's just a string.  For all I
dwmw2> > care, the URI could just as well be 
'tpmkey:id=L2Zvby9iYXIva2V5LnBlbQ=='
dwmw2> > That doesn't say anything about the contents of /foo/bar/key.pem, not
dwmw2> > more than file:/foo/bar/key.pem does, or even if there actually is a
dwmw2> > file /foo/bar/key.pem.  Maybe I misunderstand what you're after...
dwmw2> 
dwmw2> Where files are involved, I do not want the application to be told:
dwmw2>  pkcs8:/foo/bar/key
dwmw2>  pkcs1:/foo/bar/key
dwmw2>  pkcs12:/foo/bar/key or
dwmw2>  tpmkey:/foo/bar/key
dwmw2> 
dwmw2> I only want the application to be told "/foo/bar/key"

Ah, yeah, ok, so basically have OpenSSL support the "TSS KEY BLOB" PEM
type would be a way to go, wouldn't you say?  That, or add functionality
to have PEM content handlers added dynamically, one for each PEM
content type.
Just please, that "pass the BIO" hack...  sorry, I'm not a supporter.

dwmw2> It should work out what the contents are for *itself*. Whether they be
dwmw2> PEM, DER, PKCS#n, TPM-wrapped blobs, or anything else.

Yeah, got it...  my thinking was on a tachnical level, that
'whatever.pem' would have to be handled by OpenSSL itself (or in URI
terms, by the 'file' scheme handler), while 'tpmkey:file=whatever.pem'
would be handled by the 'tpmkey' scheme handler, which is a different
story to me.

I dunno about you, but to me, the URI scheme is not the same as an
indication of what contents I'll get.  But i guess that's a matter of
interpretation.

dwmw2> And if the string it's given *isn't* a filename but is instead a
dwmw2> PKCS#11 URI or a TPM URI according to Nikos's spec, that should Just
dwmw2> Work too.

You *do* indicate those with a URI scheme, though ;-)

dwmw2> User pass string identifying key. Application Just Work™. dwmw2 happy.

:-)

Cheers,
Richard ( who'd be *much* happier if his fingers didn't constantly
  want to typ tmpkey ;-) )

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [RFC 0/2] Proposal for seamless handling of TPM based RSA keys in openssl

[Richard Levitte]

In message <1479820158.8937.29.ca...@infradead.org> on Tue, 22 Nov 2016 
13:09:18 +, David Woodhouse <dw...@infradead.org> said:

dwmw2> On Tue, 2016-11-22 at 12:54 +, Salz, Rich wrote:
dwmw2> > > would much rather have seen a patch where OpenSSL's PEM module is
dwmw2> > > tought to recognise 'BEGIN TSS KEY BLOB', pull out the blob from it, 
securing
dwmw2> > 
dwmw2> > Yes, that would be much more consistent with the existing OpenSSL
dwmw2> > code which -- like it or not -- works that way.
dwmw2> 
dwmw2> Yeah. Although I'd note that the OpenSSL code only works that way for
dwmw2> PEM files. I really want to make it work the same way for DER files
dwmw2> too. There's an *attempt* in d2i_AutoPrivateKey() but that doesn't
dwmw2> handle encrypted PKCS#8 IIRC. Or PKCS#12. And the app still shouldn't
dwmw2> have to call different functions for PEM vs. DER files anyway.

Just let me shamelessly mention my STORE effort again ;-)
Among others, it does attempt to solve that very problem (in the
'file' scheme handler).

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [RFC 0/2] Proposal for seamless handling of TPM based RSA keys in openssl

[Richard Levitte]

In message <1479815862.8937.22.ca...@infradead.org> on Tue, 22 Nov 2016 
11:57:42 +, David Woodhouse <dw...@infradead.org> said:

dwmw2> On Mon, 2016-11-21 at 13:50 +, Blumenthal, Uri - 0553 - MITLL
dwmw2> wrote:
dwmw2> > Frankly, I think this approach of specially-encoded PEM or DER files
dwmw2> > telling the app what key to ask from the engine is madness, compared
dwmw2> > to the straightforward URI approach (no pun intended :).
dwmw2> 
dwmw2> Right. There are two separate things that the TPM can do for keys.
dwmw2> 
dwmw2> There is storage in the TPM itself, and you can reference a key therein
dwmw2> by its UUID. In Nikos's draft, and in GnuTLS, you end up with something
dwmw2> like tpmkey:uuid=7f468c16-cb7f-11e1-824d-b3a4f4b20343;storage=user
dwmw2> 
dwmw2> To use a PEM file for that does seem like madness; I agree.
dwmw2> 
dwmw2> However, Nikos's draft also supports a URI of the form:
dwmw2>  tpmkey:file=/foo/bar/key.pem
dwmw2> 
dwmw2> This, I do not like. It runs entirely contrary to my assertion in
dwmw2> http://david.woodhou.se/draft-woodhouse-cert-best-practice.html that
dwmw2> applications should Just Bloody Work with whatever file they're handed,
dwmw2> without needing to be *told* what the file contains.

Not sure I follow...  'file=/foo/bar/key.pem' is just a path /
parameter that the 'tpmkey' handler is free to interpret in whatever
way it sees fit.  For me as a user, it's just a string.  For all I
care, the URI could just as well be 'tpmkey:id=L2Zvby9iYXIva2V5LnBlbQ=='
That doesn't say anything about the contents of /foo/bar/key.pem, not
more than file:/foo/bar/key.pem does, or even if there actually is a
file /foo/bar/key.pem.  Maybe I misunderstand what you're after...

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [RFC 0/2] Proposal for seamless handling of TPM based RSA keys in openssl

[Richard Levitte]

In message <1479744347.2309.21.ca...@hansenpartnership.com> on Mon, 21 Nov 2016 
08:05:47 -0800, James Bottomley <james.bottom...@hansenpartnership.com> said:

James.Bottomley> On Mon, 2016-11-21 at 13:42 +, David Woodhouse wrote:
James.Bottomley> > On Wed, 2016-11-16 at 19:07 +0100, Richard Levitte wrote:
James.Bottomley> > > 
James.Bottomley> > > Many years ago, I was thinking of something along the same 
lines, 
James.Bottomley> > > but with a .pem file that would just have a few headers, 
holding 
James.Bottomley> > > the name of the intended engine and the key identity, 
something
James.Bottomley> > > like this:
James.Bottomley> > > 
James.Bottomley> > > -BEGIN PRIVATE KEY-
James.Bottomley> > > X-key-id: flarflarflar
James.Bottomley> > > X-key-engine: foo
James.Bottomley> > > -END PRIVATE KEY-
James.Bottomley> > > 
James.Bottomley> > > The intent was that the PEM code would be massaged to 
recognise 
James.Bottomley> > > these headers and would then use ENGINE_by_id() /
James.Bottomley> > > ENGINE_load_private_key() with those data and that would 
be it.
James.Bottomley> > > 
James.Bottomley> > > James, did I catch your intention about right?  I think 
that's
James.Bottomley> > > essentially what e_tpm.c does for loading keys, right?
James.Bottomley> 
James.Bottomley> Yes, that's right.  When any SSL program sees a TPM wrapped 
key, it
James.Bottomley> should just do the right thing if it has the engine capability 
without
James.Bottomley> needing the user to add any options to the command line.

Mm...  I'm not sure I agree with the method, passing a BIO for the
key_id.  I would much rather have seen a patch where OpenSSL's PEM
module is tought to recognise 'BEGIN TSS KEY BLOB', pull out the blob
from it, securing it somehow (since key_id is expected to be be NUL
terminated) and pass that to the engine.

James.Bottomley> > Once the dust settles on TPMv2.0 we should probably put 
together an I
James.Bottomley> > -D for the TPM-wrapped blob PEM files. And I should 
definitely add
James.Bottomley> > something about them to ¹.
James.Bottomley> 
James.Bottomley> Once we agree, I'll be happy to write up something.  We can 
use the pem
James.Bottomley> header concept to extend this format if it becomes necessary.

My vote goes to a URI based spec rather than bastardising PEM files.
I understand this kinda throws years of developmemt out the window,
but there you have it.

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [RFC 0/2] Proposal for seamless handling of TPM based RSA keys in openssl

[Richard Levitte]

I'm leaning in that direction as well.

Speaking of URIs, you might be interested in some work I did last
week, which would do good to get a bit of external scrutiny.

https://github.com/openssl/openssl/pull/1961 (for URI decoding)
https://github.com/openssl/openssl/pull/1962 (a STORE module that
essentially uses a URI and tries to fetch certs, keys, crls, ... from
it)

Please have a look.

Cheers,
Richard

In message <20161121135052.18280523.78584.102...@ll.mit.edu> on Mon, 21 Nov 
2016 13:50:43 +, "Blumenthal, Uri - 0553 - MITLL" <u...@ll.mit.edu> said:

uri> Frankly, I think this approach of specially-encoded PEM or DER files 
telling the app what key to ask from the engine is madness, compared to the 
straightforward URI approach (no pun intended :).
uri> 
uri> Sent from my BlackBerry 10 smartphone on the Verizon Wireless 4G LTE 
network.
uri>   Original Message  
uri> From: David Woodhouse
uri> Sent: Monday, November 21, 2016 08:43
uri> To: Richard Levitte; openssl-dev@openssl.org
uri> Reply To: openssl-dev@openssl.org
uri> Cc: James Bottomley
uri> Subject: Re: [openssl-dev] [RFC 0/2] Proposal for seamless handling of TPM 
based RSA keys in openssl
uri> 
uri> On Wed, 2016-11-16 at 19:07 +0100, Richard Levitte wrote:
uri> > 
uri> > Many years ago, I was thinking of something along the same lines, but
uri> > with a .pem file that would just have a few headers, holding the name
uri> > of the intended engine and the key identity, something like this:
uri> > 
uri> >     -BEGIN PRIVATE KEY-
uri> >     X-key-id: flarflarflar
uri> >     X-key-engine: foo
uri> >     -END PRIVATE KEY-
uri> > 
uri> > The intent was that the PEM code would be massaged to recognise these
uri> > headers and would then use ENGINE_by_id() / ENGINE_load_private_key()
uri> > with those data and that would be it.
uri> > 
uri> > James, did I catch your intention about right?  I think that's
uri> > essentially what e_tpm.c does for loading keys, right?
uri> ‎
uri> Right. The TPM engine currently uses BEGIN TSS KEY BLOB-; I
uri> added that a few years back (it used to just dump the binary blob
uri> instead). Both the TPM ENGINE and GnuTLS will load those files, as
uri> noted at http://www.infradead.org/openconnect/tpm.html
uri> 
uri> The problem is that applications have to jump through special hoops to
uri> recognise the files and invoke the engine (and there's a special API in
uri> GnuTLS too). It would be good if the appropriate engine could be
uri> invoked *automatically*, so the crypto library just does the right
uri> thing without all the applications even having to *know* about it.
uri> (Just like GnuTLS will automatically Just Work in many situations when
uri> presented with a PKCS#11 URI instead a filename, as OpenSSL also
uri> should, but doesn't yet.)
uri> 
uri> However, the contents of the PEM file should *not* be OpenSSL-specific
uri> and have engine names; I objected to James's original incarnation of
uri> this, which had something like -BEGIN tpm ENGINE PRIVATE KEY-
uri> and had the "tpm" engine automatically loaded on demand. It needs to be
uri> something generic. Which means engines need to indicate *which* PEM
uri> headers they can grok. And maybe the solution to this will tie in with
uri> the general fixes we need for "normal" key files, so that applications
uri> can Just Work with all of those too (qv¹).
uri> 
uri> Once the dust settles on TPMv2.0 we should probably put together an I-D 
uri> for the TPM-wrapped blob PEM files. And I should definitely add
uri> something about them to ¹.
uri> 
uri> -- 
uri> dwmw2
uri> 
uri> ¹ http://david.woodhou.se/draft-woodhouse-cert-best-practice.html
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] [RFC 0/2] Proposal for seamless handling of TPM based RSA keys in openssl

[Richard Levitte]

If I understand correctly, the intention is to avoid having to use
ENGINE_load_private_key() directly or having to say '-keyform ENGINE'
to the openssl commands, and to avoid having to remember some cryptic
key identity to give with '-key'.  Instead of all that, just give the
name of a .pem file with '-key' and if that file contains some kind of
magic information that the engine can understand, it will dig out a
reference to the hw protected key.

Many years ago, I was thinking of something along the same lines, but
with a .pem file that would just have a few headers, holding the name
of the intended engine and the key identity, something like this:

-BEGIN PRIVATE KEY-
X-key-id: flarflarflar
X-key-engine: foo
-END PRIVATE KEY-

The intent was that the PEM code would be massaged to recognise these
headers and would then use ENGINE_by_id() / ENGINE_load_private_key()
with those data and that would be it.

James, did I catch your intention about right?  I think that's
essentially what e_tpm.c does for loading keys, right?

Cheers,
Richard ( gotta love to see someone use "flarflarflar" as a key id ;-) )

In message <60f14e07-d0dc-486f-aff7-c74f5929b...@ll.mit.edu> on Wed, 16 Nov 
2016 15:56:05 +, "Blumenthal, Uri - 0553 - MITLL"  said:

uri> My apologies – I don’t fully understand “file based engine keys”. I 
thought the keys were either on a hardware device (a TPM, a PKCS#11-accessible 
HSM or smartcard, etc), or in a file. If a key is in a file – it’s not an 
“engine key”.
uri> 
uri> What am I missing, and what’s your use case(s)?
uri> — 
uri> Regards,
uri> Uri
uri> 
uri> 
uri> On 11/16/16, 10:46 AM, "openssl-dev on behalf of James Bottomley" 
 wrote:
uri> 
uri> [David Woodhouse told me that openssl-dev is a closed list, so the
uri> original messages got trashed.  This is a resend with apologies to
uri> David and Peter]
uri> 
uri> One of the principle problems of using TPM based keys is that there's
uri> no easy way of integrating them with standard file based keys.  This
uri> proposal adds a generic method for handling file based engine keys that
uri> can be loaded as PEM files.  Integration into the PEM loader requires a
uri> BIO based engine API callback which the first patch adds.  The second
uri> patch checks to see if the key can be loaded by any of the present
uri> engines.  Note that this requires that any engine which is to be used
uri> must be present and initialised via openssl.cnf.
uri> 
uri> I'll also post to this list the patch to openssl_tpm_engine that makes
uri> use if this infrastructure so the integration of the whole can be seen.
uri>  It should also be noted that gnutls has had this functionality since
uri> 2012.
uri> 
uri> The patch was done against 1.0.2h for easier testing and you can try it
uri> and the openssl_tpm_engine out (if you run openSUSE) here:
uri> 
uri> https://build.opensuse.org/project/show/home:jejb1:Tumbleweed
uri> 
uri> James
uri> 
uri> ---
uri> 
uri> James Bottomley (2):
uri>   engine: add new flag based method for loading engine keys
uri>   pem: load engine keys
uri> 
uri>  crypto/engine/eng_int.h  |  1 +
uri>  crypto/engine/eng_pkey.c | 38 ++
uri>  crypto/engine/engine.h   | 26 ++
uri>  crypto/pem/pem_pkey.c|  5 +
uri>  4 files changed, 70 insertions(+)
uri> 
uri> -- 
uri> openssl-dev mailing list
uri> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
uri> 
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] openssl enc changed behaviour between 1.1.0 and earlear

[Richard Levitte]

In message <20161104205933.gw7pyvclnmdkv...@breakpoint.cc> on Fri, 4 Nov 2016 
21:59:33 +0100, Sebastian Andrzej Siewior <openssl-...@ml.breakpoint.cc> said:

openssl-dev> On 2016-11-03 22:12:44 [+0100], Richard Levitte wrote:
openssl-dev> > 
openssl-dev> > That would be quite a job.  The correctness of the key can't be
openssl-dev> > discovered before the last encrypted block, where the decrypted
openssl-dev> > padding will either be correct (because it was the right key) or 
not
openssl-dev> > (because it was the wrong key).  Take into account a pipe with a 
10MB
openssl-dev> > file, I'm sure you see where that takes us.
openssl-dev> > 
openssl-dev> > The solution in that bug report seems sane, even though 
unfortunate.
openssl-dev> okay. And since the encrypted file has no header there is nothing 
we
openssl-dev> could hide. And if we add one now then it won't work with older 
openssl.
openssl-dev> 
openssl-dev> So I will try to put this in the release notes for the Debian 
package.
openssl-dev> Do you have an idea where this would fit best in the Wiki? A new 
page
openssl-dev> with one entry does not make sense and it does not look like it 
belongs
openssl-dev> to
openssl-dev>https://wiki.openssl.org/index.php/1.1_API_Changes

Actually, I would think that a parallell page for the openssl app
(program?) would be the perfect place.  It shouldn't matter if it
starts with just one item, it has to start somewhere (if you look at
the history of 1.1_API_Changes, you'll notice that it started small as
well).

Other things I can think of putting on such a page is the that the
1.1.0 'openssl' app takes all options before all non-option arguments,
there's no mixing them like there was in versions before 1.1.0.  I.e.,
this doesn't work any more:

openssl ciphers AES -V

while this does:

openssl ciphers -V AES

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev