Re: Basic SHA3 support (cryptographic discussion)

[Alexander Bluhm]

On Wed, Jan 17, 2018 at 10:20:50PM +0100, Christian Weisgerber wrote:
> What do you want to USE your SHA-3 implementation for?

I would like to have a sha3 command line tool.  Just to have it
there and start using it.  For example adding it to ports distfiles
would be trivial.

Yes, general protocol transition will be hard.  And it will not
happen before sha2 is broken.  And we will live with broken hashes
for a long time.  But I think this is not an argument against sha3.

Of course there is the question whether this should be in libc and
/bin.  But this is independent of adding sha3 and even more relevant
for broken md5 and sha1.  So I would add one sha3 tool there for
consistency.  If we feel it does not belong there and it hurts, all
the hashes could be moved.

bluhm

Re: Basic SHA3 support (cryptographic discussion)

[Christian Weisgerber]

Daniel Loebenberger:

> - The construction of SHA3 differs considerably from the SHA2
> constructions
> - SHA3's design principles are far better understood than the ones of
> SHA2.

I hear you, but you are missing the point.

> - A possible migration away from SHA2 will be
> faster when including SHA3 in OpenBSD now if it should happen that major
> cryptanalytic advances attacking SHA2 pop up in the future.

You are arguing for cryptographic algorithm agility.  That is a
concept the OpenBSD project has become increasingly critical of,
because it adds complexity and code size for questionable gain.

SHA-2 is baked into numerous protocols.  Off the top of my head:
* signify(1)
* all non-legacy SSH key exchange and authentication methods
* all non-legacy TLS cipher suites and certificates

For all of those, a switchover to SHA-3 would require defining new
protocol variants and then deploying them throughout the ecosystem.
Having a SHA-3 implementation in libc is a rather small part of the
overall effort.  And there is no practical algorithm agility until
you get to the point where you already HAVE deployed the new protocol
variants.

SHA-3 may be better, but so far SHA-2 is good enough.
Algorithm agility is a questionable goal.

So let me repeat the question:
What do you want to USE your SHA-3 implementation for?

-- 
Christian "naddy" Weisgerber  na...@mips.inka.de

Re: Basic SHA3 support

[bytevolcano]

On Wed, 10 Jan 2018 20:32:30 -0700
"Theo de Raadt"  wrote:

> > I also use the sha512 command on a regular basis, and notice a
> > performance improvement (compared to sha256) with amd64 systems.  
> 
> But is /bin the place to put a performance improving command?
> 
> No.  That isn't the place for putting strange performance commands.
> It is a place for putting *interoperating* commands.

It isn't really a "performance improving" command, it's a different
hash. I just mentioned the performance improvement in passing.

> 
> I don't think Todd's original question was answered, except along
> the lines of "i love features".
> 

His question was "who uses the sha512 link", so I answered. I don't see
where the issue is here.

Though if the link were to go, I would be able to adapt to "cksum -a
sha512 ..."

Re: Basic SHA3 support (cryptographic discussion)

[Daniel Loebenberger]

Hi,

concerning the question who needs SHA3, we do not agree that SHA3 should
be skipped as a standard. As cryptographers we sincerely believe that
the SHA3 design is superior to the one of SHA256 also due to the process
it was created. We believe that an efficient implementation will trigger
increased use of the standard and hope that its user base will grow
quickly once the commands are simply there.

For the discussion, find here some additional arguments in favor of
SHA3:

- The construction of SHA3 differs considerably from the SHA2
constructions (which covers all variants). Cryptanalytic progress for
SHA2 can destroy all variants at once, but will probably not affect
SHA3.

- SHA3's design principles are far better understood than the ones of
SHA2. The invention of sponge functions is in our opinion one of the
greatest inventions in hash-function design over the past few years. It
is simple and brilliant, and the generic properties of the construction
have appealing properties. - A possible migration away from SHA2 will be
faster when including SHA3 in OpenBSD now if it should happen that major
cryptanalytic advances attacking SHA2 pop up in the future. - The claim
that we now "know" how to build secure hash functions in general seems
problematic. To break a function, substantial cryptanalytic effort must
be made. It is not clear how much of this effort was put in the
presumably secure "newer" functions. - The argument that SHA3 is slow
does at least not apply to the reference code we used: The current
implementation of sha3-256 is indeed [slightly] faster on our machine
than the one of sha256.

Best regards,
Daniel, Stefan and Alexander

Re: Basic SHA3 support

[Daniel Loebenberger]

Hi,

we thank you for all your comments!

The following contains our answers specific to your technical remarks
on the patch itself. Another email on the cryptographic discussion
will follow.

1.) We agree that six new commands are not a good idea. The minimal
variant would be to include sha3-256. If there is a sha512 command, one
could think of adding -- for the sake of consistency -- sha3-512 as
well.

2.) The issue with the SHA2_ONLY macro is fixed now, we also
crosschecked that in distrib/special/md5 no additional code is included
by our patch.

3.) Concerning the question on whether one should also include the
implementation in LibreSSL, we did not think of such an application in
the first place (and also think that we cannot answer it competently).
This is also the reason for the wording "basic" in our submission.

Attached you find the updated patch.

Best regards,
Daniel, Stefan and Alexander

Index: bin/md5/Makefile
===
RCS file: /cvs/src/bin/md5/Makefile,v
retrieving revision 1.15
diff -u -p -u -p -r1.15 Makefile
--- bin/md5/Makefile30 Mar 2016 06:38:40 -  1.15
+++ bin/md5/Makefile11 Jan 2018 09:04:04 -
@@ -6,6 +6,8 @@ MAN=cksum.1 md5.1
 LINKS= ${BINDIR}/md5 ${BINDIR}/sha1 \
${BINDIR}/md5 ${BINDIR}/sha256 \
${BINDIR}/md5 ${BINDIR}/sha512 \
+   ${BINDIR}/md5 ${BINDIR}/sha3-256 \
+   ${BINDIR}/md5 ${BINDIR}/sha3-512 \
${BINDIR}/md5 ${BINDIR}/cksum
 
 CPPFLAGS+= -I${.CURDIR}
Index: bin/md5/md5.1
===
RCS file: /cvs/src/bin/md5/md5.1,v
retrieving revision 1.47
diff -u -p -u -p -r1.47 md5.1
--- bin/md5/md5.1   23 Feb 2017 20:46:08 -  1.47
+++ bin/md5/md5.1   11 Jan 2018 09:04:04 -
@@ -18,14 +18,18 @@
 .\" Agency (DARPA) and Air Force Research Laboratory, Air Force
 .\" Materiel Command, USAF, under agreement number F39502-99-1-0512.
 .\"
-.Dd $Mdocdate: February 23 2017 $
+.Dd $Mdocdate: January 9 2018 $
 .Dt MD5 1
 .Os
 .Sh NAME
 .Nm md5 ,
 .Nm sha1 ,
 .Nm sha256 ,
-.Nm sha512
+.Nm sha512 ,
+.Nm sha3-224 ,
+.Nm sha3-256 ,
+.Nm sha3-384 ,
+.Nm sha3-512
 .Nd calculate a message digest (checksum) for a file
 .Sh SYNOPSIS
 .Nm md5
@@ -52,6 +56,30 @@
 .Op Fl h Ar hashfile
 .Op Fl s Ar string
 .Op Ar
+.Nm sha3-224
+.Op Fl bcpqrtx
+.Op Fl C Ar checklist
+.Op Fl h Ar hashfile
+.Op Fl s Ar string
+.Op Ar
+.Nm sha3-256
+.Op Fl bcpqrtx
+.Op Fl C Ar checklist
+.Op Fl h Ar hashfile
+.Op Fl s Ar string
+.Op Ar
+.Nm sha3-384
+.Op Fl bcpqrtx
+.Op Fl C Ar checklist
+.Op Fl h Ar hashfile
+.Op Fl s Ar string
+.Op Ar
+.Nm sha3-512
+.Op Fl bcpqrtx
+.Op Fl C Ar checklist
+.Op Fl h Ar hashfile
+.Op Fl s Ar string
+.Op Ar
 .Sh DESCRIPTION
 These utilities take as input a message of arbitrary length and produce
 as output a message digest (checksum) of the input.
@@ -136,7 +164,13 @@ and \*(Gt0 if an error occurs.
 .%R RFC 3174
 .%T US Secure Hash Algorithm 1 (SHA1)
 .Re
+.Pp
 .Rs
 .%T Secure Hash Standard
 .%O FIPS PUB 180-2
+.Re
+.Pp
+.Rs
+.%T SHA-3 Standard: Permutation-Based Hash and Extendable-Output Functions
+.%O FIPS PUB 202
 .Re
Index: bin/md5/md5.c
===
RCS file: /cvs/src/bin/md5/md5.c,v
retrieving revision 1.92
diff -u -p -u -p -r1.92 md5.c
--- bin/md5/md5.c   11 Sep 2017 16:35:38 -  1.92
+++ bin/md5/md5.c   11 Jan 2018 09:04:04 -
@@ -42,6 +42,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 
 #define STYLE_MD5  0
@@ -59,6 +60,7 @@ union ANY_CTX {
MD5_CTX md5;
RMD160_CTX rmd160;
SHA1_CTX sha1;
+   SHA3_CTX sha3;
 #endif /* !defined(SHA2_ONLY) */
SHA2_CTX sha2;
 };
@@ -178,6 +180,52 @@ struct hash_function {
(void (*)(unsigned char *, void *))SHA512Final,
(char *(*)(void *, char *))SHA512End
},
+#if !defined(SHA2_ONLY)
+   {
+   "SHA3-224",
+   SHA3_224_DIGEST_LENGTH,
+   STYLE_MD5,
+   0,
+   NULL,
+   (void (*)(void *))SHA3_224Init,
+   (void (*)(void *, const unsigned char *, size_t))SHA3_Update,
+   (void (*)(unsigned char *, void *))SHA3_Final,
+   (char *(*)(void *, char *))SHA3_224End
+   },
+   {
+   "SHA3-256",
+   SHA3_256_DIGEST_LENGTH,
+   STYLE_MD5,
+   0,
+   NULL,
+   (void (*)(void *))SHA3_256Init,
+   (void (*)(void *, const unsigned char *, size_t))SHA3_Update,
+   (void (*)(unsigned char *, void *))SHA3_Final,
+   (char *(*)(void *, char *))SHA3_256End
+   },
+   {
+   "SHA3-384",
+   SHA3_384_DIGEST_LENGTH,
+   STYLE_MD5,
+   0,
+   NULL,
+   (void (*)(void *))SHA3_384Init,
+   (void (*)(void *, const unsigned 

Re: Basic SHA3 support

[Todd C. Miller]

On Wed, 10 Jan 2018 22:06:37 +, Christian Weisgerber wrote:

> I've had something like this on my to-do list for some time and
> eventually dropped it because I couldn't think of a compelling use
> case.
>
> "Maybe Skip SHA-3"
> https://www.imperialviolet.org/2017/05/31/skipsha3.html

That makes a pretty convincing case for *not* inluding sha-3.

I think at this point there needs to be a compelling argument to
include it in libc.  As you say, it is not even present in LibreSSL,
though OpenSSL did eventually add it.

 - todd

Re: Basic SHA3 support

[lists]

Wed, 10 Jan 2018 20:34:44 -0700 "Theo de Raadt" 
> > Another question raised by a SHA-3 import is whether this should
> > also be added to LibreSSL and if we want to continue with having
> > duplicate implementations of hash algorithms in libc and libcrypto.  
> 
> very good question.
> 
> Who *crucially* needs this
> 
> OpenBSD is intended to be a small operating system.  I don't want
> to compile SHA3 stuff every single build unless there is a substantial
> and growing userbase for it
> 
> Is that the case, or is this fad of the month?
> 

Hi tech@,

I won't cry if this became a port instead of importing 'it' in base:

NIST https://en.wikipedia.org/wiki/The_Mist_(film)
MIST https://en.wikipedia.org/wiki/SHA-3#Capacity_change_controversy

I won't have a use for it even in that case, given the above detail.

Kind regards,
Anton Lazarov

Re: Basic SHA3 support

[Theo de Raadt]

> Another question raised by a SHA-3 import is whether this should
> also be added to LibreSSL and if we want to continue with having
> duplicate implementations of hash algorithms in libc and libcrypto.

very good question.

Who *crucially* needs this

OpenBSD is intended to be a small operating system.  I don't want
to compile SHA3 stuff every single build unless there is a substantial
and growing userbase for it

Is that the case, or is this fad of the month?

Re: Basic SHA3 support

[Theo de Raadt]

> I also use the sha512 command on a regular basis, and notice a
> performance improvement (compared to sha256) with amd64 systems.

But is /bin the place to put a performance improving command?

No.  That isn't the place for putting strange performance commands.
It is a place for putting *interoperating* commands.

I don't think Todd's original question was answered, except along
the lines of "i love features".

Re: Basic SHA3 support

[bytevolcano]

I also use the sha512 command on a regular basis, and notice a
performance improvement (compared to sha256) with amd64 systems.

On Wed, 10 Jan 2018 17:10:40 + (GMT)
Tom Cosgrove  wrote:

> >>> "Todd C. Miller" 10-Jan-18 16:28 >>>  
> >
> > On Wed, 10 Jan 2018 15:59:30 +0100, Jeremie Courreges-Anglas wrote:
> >  
> > > Do we really want these?  For SHA-2, sha256 and sha512 are enough IMHO.  
> >
> > Does anyone actually use the sha512 command?  I'd be fine with
> > retiring the sha512 link and adding a sha3-256 one.
> >
> >  - todd  
> 
> I'd like to keep the sha512 link - I do use it.
> 
> Tom
> 

Re: Basic SHA3 support

[Christian Weisgerber]

On 2018-01-09, Daniel Loebenberger  wrote:

> enclosed you find a patch to add basic SHA3-/Keccak support to OpenBSD.

What do you want to use it for?

I've had something like this on my to-do list for some time and
eventually dropped it because I couldn't think of a compelling use
case.

"Maybe Skip SHA-3"
https://www.imperialviolet.org/2017/05/31/skipsha3.html

Another question raised by a SHA-3 import is whether this should
also be added to LibreSSL and if we want to continue with having
duplicate implementations of hash algorithms in libc and libcrypto.

-- 
Christian "naddy" Weisgerber  na...@mips.inka.de

Re: Basic SHA3 support

[Tom Cosgrove]

>>> "Todd C. Miller" 10-Jan-18 16:28 >>>
>
> On Wed, 10 Jan 2018 15:59:30 +0100, Jeremie Courreges-Anglas wrote:
>
> > Do we really want these?  For SHA-2, sha256 and sha512 are enough IMHO.
>
> Does anyone actually use the sha512 command?  I'd be fine with
> retiring the sha512 link and adding a sha3-256 one.
>
>  - todd

I'd like to keep the sha512 link - I do use it.

Tom

Re: Basic SHA3 support

[Todd C. Miller]

On Wed, 10 Jan 2018 15:59:30 +0100, Jeremie Courreges-Anglas wrote:

> Do we really want these?  For SHA-2, sha256 and sha512 are enough IMHO.

Does anyone actually use the sha512 command?  I'd be fine with
retiring the sha512 link and adding a sha3-256 one.

 - todd

Re: Basic SHA3 support

[Theo de Raadt]

> >  LINKS= ${BINDIR}/md5 ${BINDIR}/sha1 \
> > +   ${BINDIR}/md5 ${BINDIR}/sha224 \
> > ${BINDIR}/md5 ${BINDIR}/sha256 \
> > +   ${BINDIR}/md5 ${BINDIR}/sha384 \
> 
> Do we really want these?  For SHA-2, sha256 and sha512 are enough IMHO.
> 
> > ${BINDIR}/md5 ${BINDIR}/sha512 \
> > +   ${BINDIR}/md5 ${BINDIR}/sha3-224 \
> > +   ${BINDIR}/md5 ${BINDIR}/sha3-256 \
> > +   ${BINDIR}/md5 ${BINDIR}/sha3-384 \
> > +   ${BINDIR}/md5 ${BINDIR}/sha3-512 \
> 
> Same question but for the SHA-3 version.

I think this is also a waste of time.

Look, it is the /bin directory.  Look at what is in it now.

Adding 6 commands?  How many people will use them?  shells and
other things cache the existance of these things.

Re: Basic SHA3 support

[Jeremie Courreges-Anglas]

Hi,

On Wed, Jan 10 2018, Daniel Loebenberger  wrote:
> Hi Todd,
>
> thank you for your comments!
>
> We have included all the necessary changes in the patch attached.

One thing caught my eye:

> Best regards,
> Daniel, Stefan and Alexander
>
> Index: bin/md5/Makefile
> ===
> RCS file: /cvs/src/bin/md5/Makefile,v
> retrieving revision 1.15
> diff -u -p -u -p -r1.15 Makefile
> --- bin/md5/Makefile  30 Mar 2016 06:38:40 -  1.15
> +++ bin/md5/Makefile  10 Jan 2018 13:43:55 -
> @@ -3,9 +3,16 @@
>  PROG=md5
>  SRCS=crc.c md5.c
>  MAN= cksum.1 md5.1
> +
>  LINKS=   ${BINDIR}/md5 ${BINDIR}/sha1 \
> + ${BINDIR}/md5 ${BINDIR}/sha224 \
>   ${BINDIR}/md5 ${BINDIR}/sha256 \
> + ${BINDIR}/md5 ${BINDIR}/sha384 \

Do we really want these?  For SHA-2, sha256 and sha512 are enough IMHO.

>   ${BINDIR}/md5 ${BINDIR}/sha512 \
> + ${BINDIR}/md5 ${BINDIR}/sha3-224 \
> + ${BINDIR}/md5 ${BINDIR}/sha3-256 \
> + ${BINDIR}/md5 ${BINDIR}/sha3-384 \
> + ${BINDIR}/md5 ${BINDIR}/sha3-512 \

Same question but for the SHA-3 version.

[...]

-- 
jca | PGP : 0x1524E7EE / 5135 92C1 AD36 5293 2BDF  DDCC 0DFA 74AE 1524 E7EE

Re: Basic SHA3 support

[Theo de Raadt]

I don't think you compiled a ramdisk.

The ramdisk users distrib/special/md5 to build a binary with
fewer included components, otherwise it won't fit on the media.

At least this isn't right:

 #endif /* !defined(SHA2_ONLY) */
SHA2_CTX sha2;
+   SHA3_CTX sha3;

You can inspect the Makefile there and make the SHA2_ONLY #ifdefs
work right.

Re: Basic SHA3 support

[Daniel Loebenberger]

Hi Todd,

thank you for your comments!

We have included all the necessary changes in the patch attached.

Best regards,
Daniel, Stefan and Alexander

Index: bin/md5/Makefile
===
RCS file: /cvs/src/bin/md5/Makefile,v
retrieving revision 1.15
diff -u -p -u -p -r1.15 Makefile
--- bin/md5/Makefile30 Mar 2016 06:38:40 -  1.15
+++ bin/md5/Makefile10 Jan 2018 13:43:55 -
@@ -3,9 +3,16 @@
 PROG=  md5
 SRCS=  crc.c md5.c
 MAN=   cksum.1 md5.1
+
 LINKS= ${BINDIR}/md5 ${BINDIR}/sha1 \
+   ${BINDIR}/md5 ${BINDIR}/sha224 \
${BINDIR}/md5 ${BINDIR}/sha256 \
+   ${BINDIR}/md5 ${BINDIR}/sha384 \
${BINDIR}/md5 ${BINDIR}/sha512 \
+   ${BINDIR}/md5 ${BINDIR}/sha3-224 \
+   ${BINDIR}/md5 ${BINDIR}/sha3-256 \
+   ${BINDIR}/md5 ${BINDIR}/sha3-384 \
+   ${BINDIR}/md5 ${BINDIR}/sha3-512 \
${BINDIR}/md5 ${BINDIR}/cksum
 
 CPPFLAGS+= -I${.CURDIR}
Index: bin/md5/md5.1
===
RCS file: /cvs/src/bin/md5/md5.1,v
retrieving revision 1.47
diff -u -p -u -p -r1.47 md5.1
--- bin/md5/md5.1   23 Feb 2017 20:46:08 -  1.47
+++ bin/md5/md5.1   10 Jan 2018 13:43:55 -
@@ -18,14 +18,18 @@
 .\" Agency (DARPA) and Air Force Research Laboratory, Air Force
 .\" Materiel Command, USAF, under agreement number F39502-99-1-0512.
 .\"
-.Dd $Mdocdate: February 23 2017 $
+.Dd $Mdocdate: January 9 2018 $
 .Dt MD5 1
 .Os
 .Sh NAME
 .Nm md5 ,
 .Nm sha1 ,
 .Nm sha256 ,
-.Nm sha512
+.Nm sha512 ,
+.Nm sha3-224 ,
+.Nm sha3-256 ,
+.Nm sha3-384 ,
+.Nm sha3-512
 .Nd calculate a message digest (checksum) for a file
 .Sh SYNOPSIS
 .Nm md5
@@ -52,6 +56,30 @@
 .Op Fl h Ar hashfile
 .Op Fl s Ar string
 .Op Ar
+.Nm sha3-224
+.Op Fl bcpqrtx
+.Op Fl C Ar checklist
+.Op Fl h Ar hashfile
+.Op Fl s Ar string
+.Op Ar
+.Nm sha3-256
+.Op Fl bcpqrtx
+.Op Fl C Ar checklist
+.Op Fl h Ar hashfile
+.Op Fl s Ar string
+.Op Ar
+.Nm sha3-384
+.Op Fl bcpqrtx
+.Op Fl C Ar checklist
+.Op Fl h Ar hashfile
+.Op Fl s Ar string
+.Op Ar
+.Nm sha3-512
+.Op Fl bcpqrtx
+.Op Fl C Ar checklist
+.Op Fl h Ar hashfile
+.Op Fl s Ar string
+.Op Ar
 .Sh DESCRIPTION
 These utilities take as input a message of arbitrary length and produce
 as output a message digest (checksum) of the input.
@@ -136,7 +164,13 @@ and \*(Gt0 if an error occurs.
 .%R RFC 3174
 .%T US Secure Hash Algorithm 1 (SHA1)
 .Re
+.Pp
 .Rs
 .%T Secure Hash Standard
 .%O FIPS PUB 180-2
+.Re
+.Pp
+.Rs
+.%T SHA-3 Standard: Permutation-Based Hash and Extendable-Output Functions
+.%O FIPS PUB 202
 .Re
Index: bin/md5/md5.c
===
RCS file: /cvs/src/bin/md5/md5.c,v
retrieving revision 1.92
diff -u -p -u -p -r1.92 md5.c
--- bin/md5/md5.c   11 Sep 2017 16:35:38 -  1.92
+++ bin/md5/md5.c   10 Jan 2018 13:43:55 -
@@ -42,6 +42,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 
 #define STYLE_MD5  0
@@ -61,6 +62,7 @@ union ANY_CTX {
SHA1_CTX sha1;
 #endif /* !defined(SHA2_ONLY) */
SHA2_CTX sha2;
+   SHA3_CTX sha3;
 };
 
 struct hash_function {
@@ -177,6 +179,50 @@ struct hash_function {
(void (*)(void *, const unsigned char *, size_t))SHA512Update,
(void (*)(unsigned char *, void *))SHA512Final,
(char *(*)(void *, char *))SHA512End
+   },
+   {
+   "SHA3-224",
+   SHA3_224_DIGEST_LENGTH,
+   STYLE_MD5,
+   0,
+   NULL,
+   (void (*)(void *))SHA3_224Init,
+   (void (*)(void *, const unsigned char *, size_t))SHA3_Update,
+   (void (*)(unsigned char *, void *))SHA3_Final,
+   (char *(*)(void *, char *))SHA3_224End
+   },
+   {
+   "SHA3-256",
+   SHA3_256_DIGEST_LENGTH,
+   STYLE_MD5,
+   0,
+   NULL,
+   (void (*)(void *))SHA3_256Init,
+   (void (*)(void *, const unsigned char *, size_t))SHA3_Update,
+   (void (*)(unsigned char *, void *))SHA3_Final,
+   (char *(*)(void *, char *))SHA3_256End
+   },
+   {
+   "SHA3-384",
+   SHA3_384_DIGEST_LENGTH,
+   STYLE_MD5,
+   0,
+   NULL,
+   (void (*)(void *))SHA3_384Init,
+   (void (*)(void *, const unsigned char *, size_t))SHA3_Update,
+   (void (*)(unsigned char *, void *))SHA3_Final,
+   (char *(*)(void *, char *))SHA3_384End
+   },
+   {
+   "SHA3-512",
+   SHA3_512_DIGEST_LENGTH,
+   STYLE_MD5,
+   0,
+   NULL,
+   (void (*)(void *))SHA3_512Init,
+   (void (*)(void *, const unsigned char *, size_t))SHA3_Update,
+   (void (*)(unsigned char *, void *))SHA3_Final,
+   (char *(*)(void *, 

Re: Basic SHA3 support

[Todd C. Miller]

On Tue, 09 Jan 2018 16:47:18 +0100, Daniel Loebenberger wrote:

> Changes have been made to libc, and a suite of sha3 checksum tools
> were added (sha3-224, sha3-256, sha3-384, sha-512), extending the
> existing md5(1) checksum tool.
>
> The SHA3 implementation itself was taken from the reference code written
> by the Keccak Team (https://keccak.team/) which is available under public
> domain (CC0).
>
> We'd be happy to see this in OpenBSD and appreciate any comments.

Looks good in general, some comments inline.

 - todd

> Index: include/sha3.h
> ===
> RCS file: include/sha3.h
> diff -N include/sha3.h
> --- /dev/null 1 Jan 1970 00:00:00 -
> +++ include/sha3.h9 Jan 2018 15:17:42 -
> @@ -0,0 +1,76 @@
> +#ifndef _SHA3_H
> +#define _SHA3_H
> +
> +#define SHA3_224_DIGEST_LENGTH   28
> +#define SHA3_224_DIGEST_STRING_LENGTH(SHA3_224_DIGEST_LENGTH * 2 + 1
> )
> +
> +#define SHA3_256_DIGEST_LENGTH   32
> +#define SHA3_256_DIGEST_STRING_LENGTH(SHA3_256_DIGEST_LENGTH * 2 + 1
> )
> +
> +#define SHA3_384_DIGEST_LENGTH   48
> +#define SHA3_384_DIGEST_STRING_LENGTH(SHA3_384_DIGEST_LENGTH * 2 + 1
> )
> +
> +#define SHA3_512_DIGEST_LENGTH   64
> +#define SHA3_512_DIGEST_STRING_LENGTH(SHA3_512_DIGEST_LENGTH * 2 + 1
> )
> +
> +#define SHA3_STATE_SIZE  1600 / 8

This should be parenthesized, i.e. (1600 / 8)

> +
> +
> +typedef struct _SHA3_CTX {
> + unsigned char state[SHA3_STATE_SIZE];
> + unsigned int rate;
> + unsigned int byteIOIndex;
> + unsigned int fixedOutputLength;
> +} SHA3_CTX;
> +
> +
> +void SHA3_224Init(SHA3_CTX *);
> +char *SHA3_224End(SHA3_CTX *, char *)
> + __attribute__((__bounded__(__minbytes__,2,SHA3_224_DIGEST_STRING_LENGTH
> )));
> +char *SHA3_224File(const char *, char *)
> + __attribute__((__bounded__(__minbytes__,2,SHA3_224_DIGEST_STRING_LENGTH
> )));
> +char *SHA3_224FileChunk(const char *, char *, off_t, off_t)
> + __attribute__((__bounded__(__minbytes__,2,SHA3_224_DIGEST_STRING_LENGTH
> )));
> +char *SHA3_224Data(const u_int8_t *, size_t, char *)
> + __attribute__((__bounded__(__string__,1,2)))
> + __attribute__((__bounded__(__minbytes__,3,SHA3_224_DIGEST_STRING_LENGTH
> )));
> +
> +void SHA3_256Init(SHA3_CTX *);
> +char *SHA3_256End(SHA3_CTX *, char *)
> + __attribute__((__bounded__(__minbytes__,2,SHA3_256_DIGEST_STRING_LENGTH
> )));
> +char *SHA3_256File(const char *, char *)
> + __attribute__((__bounded__(__minbytes__,2,SHA3_256_DIGEST_STRING_LENGTH
> )));
> +char *SHA3_256FileChunk(const char *, char *, off_t, off_t)
> + __attribute__((__bounded__(__minbytes__,2,SHA3_256_DIGEST_STRING_LENGTH
> )));
> +char *SHA3_256Data(const u_int8_t *, size_t, char *)
> + __attribute__((__bounded__(__string__,1,2)))
> + __attribute__((__bounded__(__minbytes__,3,SHA3_256_DIGEST_STRING_LENGTH
> )));
> +
> +void SHA3_384Init(SHA3_CTX *);
> +char *SHA3_384End(SHA3_CTX *, char *)
> + __attribute__((__bounded__(__minbytes__,2,SHA3_384_DIGEST_STRING_LENGTH
> )));
> +char *SHA3_384File(const char *, char *)
> + __attribute__((__bounded__(__minbytes__,2,SHA3_384_DIGEST_STRING_LENGTH
> )));
> +char *SHA3_384FileChunk(const char *, char *, off_t, off_t)
> + __attribute__((__bounded__(__minbytes__,2,SHA3_384_DIGEST_STRING_LENGTH
> )));
> +char *SHA3_384Data(const u_int8_t *, size_t, char *)
> + __attribute__((__bounded__(__string__,1,2)))
> + __attribute__((__bounded__(__minbytes__,3,SHA3_384_DIGEST_STRING_LENGTH
> )));
> +
> +void SHA3_512Init(SHA3_CTX *);
> +char *SHA3_512End(SHA3_CTX *, char *)
> + __attribute__((__bounded__(__minbytes__,2,SHA3_512_DIGEST_STRING_LENGTH
> )));
> +char *SHA3_512File(const char *, char *)
> + __attribute__((__bounded__(__minbytes__,2,SHA3_512_DIGEST_STRING_LENGTH
> )));
> +char *SHA3_512FileChunk(const char *, char *, off_t, off_t)
> + __attribute__((__bounded__(__minbytes__,2,SHA3_512_DIGEST_STRING_LENGTH
> )));
> +char *SHA3_512Data(const u_int8_t *, size_t, char *)
> + __attribute__((__bounded__(__string__,1,2)))
> + __attribute__((__bounded__(__minbytes__,3,SHA3_512_DIGEST_STRING_LENGTH
> )));
> +
> +void SHA3_Update(SHA3_CTX *, const u_int8_t *, size_t)
> + __attribute__((__bounded__(__string__,2,3)));
> +void SHA3_Final(u_int8_t [SHA3_224_DIGEST_LENGTH], SHA3_CTX *)
> + __attribute__((__bounded__(__minbytes__,1,SHA3_224_DIGEST_LENGTH)));
> +
> +#endif /* _SHA3_H */
> Index: lib/libc/hash/sha3.c
> ===
> RCS file: lib/libc/hash/sha3.c
> diff -N lib/libc/hash/sha3.c
> --- /dev/null 1 Jan 1970 00:00:00 -
> +++ lib/libc/hash/sha3.c  9 Jan 2018 15:17:42 -
> @@ -0,0 +1,756 @@
> +/*
> + * Implementation by the Keccak, Keyak and Ketje Teams, namely, Guido
> Bertoni,
> + * Joan Daemen, Micha�l Peeters, Gilles Van Assche and Ronny Van Keer,
> 

Basic SHA3 support

[Daniel Loebenberger]

Hi everyone,

enclosed you find a patch to add basic SHA3-/Keccak support to OpenBSD.

Changes have been made to libc, and a suite of sha3 checksum tools
were added (sha3-224, sha3-256, sha3-384, sha-512), extending the 
existing md5(1) checksum tool.

The SHA3 implementation itself was taken from the reference code written
by the Keccak Team (https://keccak.team/) which is available under public
domain (CC0).

We'd be happy to see this in OpenBSD and appreciate any comments.

Best regards,
Daniel, Stefan and Alexander

--
Dr. Daniel Loebenberger
Evaluation & Research

genua GmbH
Domagkstrasse 7, 85551 Kirchheim bei München
Tel. +49 89 991950-0, Fax -999, www.genua.de
Geschäftsführer: Matthias Ochs, Marc Tesch. Amtsgericht München 
HRB 98238.
Die genua GmbH ist ein Unternehmen der Bundesdruckerei-Gruppe.


Index: bin/md5/Makefile
===
RCS file: /cvs/src/bin/md5/Makefile,v
retrieving revision 1.15
diff -u -p -u -p -r1.15 Makefile
--- bin/md5/Makefile30 Mar 2016 06:38:40 -  1.15
+++ bin/md5/Makefile9 Jan 2018 15:17:20 -
@@ -2,13 +2,20 @@
 
 PROG=  md5
 SRCS=  crc.c md5.c
-MAN=   cksum.1 md5.1
+
 LINKS= ${BINDIR}/md5 ${BINDIR}/sha1 \
+   ${BINDIR}/md5 ${BINDIR}/sha224 \
${BINDIR}/md5 ${BINDIR}/sha256 \
+   ${BINDIR}/md5 ${BINDIR}/sha384 \
${BINDIR}/md5 ${BINDIR}/sha512 \
+   ${BINDIR}/md5 ${BINDIR}/sha3-224 \
+   ${BINDIR}/md5 ${BINDIR}/sha3-256 \
+   ${BINDIR}/md5 ${BINDIR}/sha3-384 \
+   ${BINDIR}/md5 ${BINDIR}/sha3-512 \
${BINDIR}/md5 ${BINDIR}/cksum
 
-CPPFLAGS+= -I${.CURDIR}
+CPPFLAGS+=-I${.CURDIR}
+
 COPTS+=-Wall -Wconversion -Wmissing-prototypes
 
 .include 
Index: bin/md5/md5.1
===
RCS file: /cvs/src/bin/md5/md5.1,v
retrieving revision 1.47
diff -u -p -u -p -r1.47 md5.1
--- bin/md5/md5.1   23 Feb 2017 20:46:08 -  1.47
+++ bin/md5/md5.1   9 Jan 2018 15:17:20 -
@@ -18,14 +18,18 @@
 .\" Agency (DARPA) and Air Force Research Laboratory, Air Force
 .\" Materiel Command, USAF, under agreement number F39502-99-1-0512.
 .\"
-.Dd $Mdocdate: February 23 2017 $
+.Dd $Mdocdate: January 9 2018 $
 .Dt MD5 1
 .Os
 .Sh NAME
 .Nm md5 ,
 .Nm sha1 ,
 .Nm sha256 ,
-.Nm sha512
+.Nm sha512 ,
+.Nm sha3-224 ,
+.Nm sha3-256 ,
+.Nm sha3-384 ,
+.Nm sha3-512
 .Nd calculate a message digest (checksum) for a file
 .Sh SYNOPSIS
 .Nm md5
@@ -52,6 +56,30 @@
 .Op Fl h Ar hashfile
 .Op Fl s Ar string
 .Op Ar
+.Nm sha3-224
+.Op Fl bcpqrtx
+.Op Fl C Ar checklist
+.Op Fl h Ar hashfile
+.Op Fl s Ar string
+.Op Ar
+.Nm sha3-256
+.Op Fl bcpqrtx
+.Op Fl C Ar checklist
+.Op Fl h Ar hashfile
+.Op Fl s Ar string
+.Op Ar
+.Nm sha3-384
+.Op Fl bcpqrtx
+.Op Fl C Ar checklist
+.Op Fl h Ar hashfile
+.Op Fl s Ar string
+.Op Ar
+.Nm sha3-512
+.Op Fl bcpqrtx
+.Op Fl C Ar checklist
+.Op Fl h Ar hashfile
+.Op Fl s Ar string
+.Op Ar
 .Sh DESCRIPTION
 These utilities take as input a message of arbitrary length and produce
 as output a message digest (checksum) of the input.
@@ -136,7 +164,13 @@ and \*(Gt0 if an error occurs.
 .%R RFC 3174
 .%T US Secure Hash Algorithm 1 (SHA1)
 .Re
+.Pp
 .Rs
 .%T Secure Hash Standard
 .%O FIPS PUB 180-2
+.Re
+.Pp
+.Rs
+.%T SHA-3 Standard: Permutation-Based Hash and Extendable-Output Functions
+.%O FIPS PUB 202
 .Re
Index: bin/md5/md5.c
===
RCS file: /cvs/src/bin/md5/md5.c,v
retrieving revision 1.92
diff -u -p -u -p -r1.92 md5.c
--- bin/md5/md5.c   11 Sep 2017 16:35:38 -  1.92
+++ bin/md5/md5.c   9 Jan 2018 15:17:20 -
@@ -42,6 +42,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 
 #define STYLE_MD5  0
@@ -61,6 +62,7 @@ union ANY_CTX {
SHA1_CTX sha1;
 #endif /* !defined(SHA2_ONLY) */
SHA2_CTX sha2;
+   SHA3_CTX sha3;
 };
 
 struct hash_function {
@@ -177,6 +179,50 @@ struct hash_function {
(void (*)(void *, const unsigned char *, size_t))SHA512Update,
(void (*)(unsigned char *, void *))SHA512Final,
(char *(*)(void *, char *))SHA512End
+   },
+   {
+   "SHA3-224",
+   SHA3_224_DIGEST_LENGTH,
+   STYLE_MD5,
+   0,
+   NULL,
+   (void (*)(void *))SHA3_224Init,
+   (void (*)(void *, const unsigned char *, size_t))SHA3_Update,
+   (void (*)(unsigned char *, void *))SHA3_Final,
+   (char *(*)(void *, char *))SHA3_224End
+   },
+   {
+   "SHA3-256",
+   SHA3_256_DIGEST_LENGTH,
+   STYLE_MD5,
+   0,
+   NULL,
+   (void (*)(void *))SHA3_256Init,
+   (void (*)(void *, const unsigned char *, size_t))SHA3_Update,
+   (void (*)(unsigned char *, void *))SHA3_Final,
+   (char *(*)(void *, char *))SHA3_256End
+   },
+   {