Re: [tor-bugs] #17799 [Tor]: Hash All PRNG output before use

[Tor Bug Tracker & Wiki]

#17799: Hash All PRNG output before use
---+
 Reporter:  teor   |  Owner:  nickm
 Type:  defect | Status:  needs_revision
 Priority:  Medium |  Milestone:  Tor: 0.2.9.x-final
Component:  Tor|Version:  Tor: unspecified
 Severity:  Normal | Resolution:
 Keywords:  TorCoreTeam201604  |  Actual Points:
Parent ID: | Points:  small/medium-remaining
 Reviewer:  asn|Sponsor:
---+
Changes (by nickm):

 * status:  needs_review => needs_revision


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #17799 [Tor]: Hash All PRNG output before use

[Tor Bug Tracker & Wiki]

#17799: Hash All PRNG output before use
---+
 Reporter:  teor   |  Owner:  nickm
 Type:  defect | Status:  needs_review
 Priority:  Medium |  Milestone:  Tor: 0.2.9.x-final
Component:  Tor|Version:  Tor: unspecified
 Severity:  Normal | Resolution:
 Keywords:  TorCoreTeam201604  |  Actual Points:
Parent ID: | Points:  small/medium-remaining
 Reviewer:  asn|Sponsor:
---+

Comment (by yawning):

 Casual review/thoughts:

  * `REPLACE_OPENSSL_RAND` probably should be enabled by default once we
 actually deploy this, because this isn't any worse than OpenSSL's current
 RNG, and is likely better, with a compile time flag to not do this.

  * (Abstraction) Should we have a generic ability to allocate `mlock()`ed
 pages useable by other places of code? (I think yes, because common rlimit
 configs allow for up to 64 KiB worth of locked pages, and we can keep long
 term keying material in such for example).

  * (Paranoia) The code should call `getrlimit()` to ensure that the
 `mlock()` has a chance of succeeding.  Failure in either case should be
 loud, hard, and always fatal on platforms where locked memory is
 supported.  A config option to allow using a non-mlocked() backing store
 perhaps.

  * (Paranoia) Is there any reason (that isn't performance) to not hit up
 the syscall entropy source on supported platforms on each refill call?

  * (Paranoia) The `memset()` call in `shake_prng_getbytes_raw()` should be
 a `memwipe()`.  Since this routine is called with the mutex held, wiping
 the consumed portion of the buffer probably can be moved out of the
 while() loop to above the invariant check (Tiny optimization).

  * (Bug) Child side of a `fork()` does not inherit `mlock()` status.  Need
 to re-`mlock()` in the post fork handler.

  * We should run this through dieharder/testU01/the NIST suite or similar,
 just to say we did.  Most CSPRNGs (even broken/horribad ones) will pass
 both tests, but it's better than nothing.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #17799 [Tor]: Hash All PRNG output before use

[Tor Bug Tracker & Wiki]

#17799: Hash All PRNG output before use
---+
 Reporter:  teor   |  Owner:  nickm
 Type:  defect | Status:  needs_review
 Priority:  Medium |  Milestone:  Tor: 0.2.9.x-final
Component:  Tor|Version:  Tor: unspecified
 Severity:  Normal | Resolution:
 Keywords:  TorCoreTeam201604  |  Actual Points:
Parent ID: | Points:  small/medium-remaining
 Reviewer:  asn|Sponsor:
---+
Changes (by asn):

 * reviewer:   => asn


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #17799 [Tor]: Hash All PRNG output before use

[Tor Bug Tracker & Wiki]

#17799: Hash All PRNG output before use
+
 Reporter:  teor|  Owner:  nickm
 Type:  defect  | Status:  needs_review
 Priority:  Medium  |  Milestone:  Tor: 0.2.9.x-final
Component:  Tor |Version:  Tor: unspecified
 Severity:  Normal  | Resolution:
 Keywords:  |  Actual Points:
Parent ID:  | Points:  small/medium-remaining
 Reviewer:  |Sponsor:
+
Changes (by nickm):

 * points:   => small/medium-remaining


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #17799 [Tor]: Hash All PRNG output before use

[Tor Bug Tracker & Wiki]

#17799: Hash All PRNG output before use
+
 Reporter:  teor|  Owner:  nickm
 Type:  defect  | Status:  needs_review
 Priority:  Medium  |  Milestone:  Tor: 0.2.9.x-final
Component:  Tor |Version:  Tor: unspecified
 Severity:  Normal  | Resolution:
 Keywords:  |  Actual Points:
Parent ID:  | Points:
  Sponsor:  |
+
Changes (by nickm):

 * milestone:  Tor: 0.2.8.x-final => Tor: 0.2.9.x-final


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #17799 [Tor]: Hash All PRNG output before use

[Tor Bug Tracker & Wiki]

#17799: Hash All PRNG output before use
+
 Reporter:  teor|  Owner:
 Type:  defect  | Status:  needs_review
 Priority:  Medium  |  Milestone:  Tor: 0.2.8.x-final
Component:  Tor |Version:  Tor: unspecified
 Severity:  Normal  | Resolution:
 Keywords:  |  Actual Points:
Parent ID:  | Points:
  Sponsor:  |
+
Changes (by nickm):

 * status:  needs_revision => needs_review


Comment:

 I made a couple more changes to shake_prng_v2.  Notably the pointer
 arithmetic is much simpler.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #17799 [Tor]: Hash All PRNG output before use

[Tor Bug Tracker & Wiki]

#17799: Hash All PRNG output before use
+
 Reporter:  teor|  Owner:
 Type:  defect  | Status:  needs_review
 Priority:  Medium  |  Milestone:  Tor: 0.2.8.x-final
Component:  Tor |Version:  Tor: unspecified
 Severity:  Normal  | Resolution:
 Keywords:  |  Actual Points:
Parent ID:  | Points:
  Sponsor:  |
+

Comment (by nickm):

 Rebased and squashed as shake_prng_v3.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #17799 [Tor]: Hash All PRNG output before use

[Tor Bug Tracker & Wiki]

#17799: Hash All PRNG output before use
+
 Reporter:  teor|  Owner:
 Type:  defect  | Status:  needs_revision
 Priority:  Medium  |  Milestone:  Tor: 0.2.8.x-final
Component:  Tor |Version:  Tor: unspecified
 Severity:  Normal  | Resolution:
 Keywords:  |  Actual Points:
Parent ID:  | Points:
  Sponsor:  |
+

Comment (by nickm):

 (I just did a force-update on it to clean up a couple of things; please
 don't hate me.)

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #17799 [Tor]: Hash All PRNG output before use

[Tor Bug Tracker & Wiki]

#17799: Hash All PRNG output before use
+
 Reporter:  teor|  Owner:
 Type:  defect  | Status:  needs_revision
 Priority:  Medium  |  Milestone:  Tor: 0.2.8.x-final
Component:  Tor |Version:  Tor: unspecified
 Severity:  Normal  | Resolution:
 Keywords:  |  Actual Points:
Parent ID:  | Points:
  Sponsor:  |
+

Comment (by nickm):

 I think their performance is likely to be worse, but their space usage is
 likely to be better.  If I'm doing the math right, and we're using
 SHAKE-128 or equivalent, and we assume that we want to have 2^256^
 security against backtracking attacks, I get 166.7 bytes per Keccak-f call
 with 4K memory usage, and they get 152 bytes,with somewhere between 200
 and 400 bytes of memory usage.

 (I'm also assuming that reseed events are comparatively uncommon.  If
 that's wrong, oops!)

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #17799 [Tor]: Hash All PRNG output before use

[Tor Bug Tracker & Wiki]

#17799: Hash All PRNG output before use
+
 Reporter:  teor|  Owner:
 Type:  defect  | Status:  needs_revision
 Priority:  Medium  |  Milestone:  Tor: 0.2.8.x-final
Component:  Tor |Version:  Tor: unspecified
 Severity:  Normal  | Resolution:
 Keywords:  |  Actual Points:
Parent ID:  | Points:
  Sponsor:  |
+

Comment (by yawning):

 The Keccak authors have a paper regarding how to construct a sponge based
 PRNG. http://sponge.noekeon.org/SpongePRNG.pdf

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #17799 [Tor]: Hash All PRNG output before use

[Tor Bug Tracker & Wiki]

#17799: Hash All PRNG output before use
+
 Reporter:  teor|  Owner:
 Type:  defect  | Status:  needs_revision
 Priority:  Medium  |  Milestone:  Tor: 0.2.8.x-final
Component:  Tor |Version:  Tor: unspecified
 Severity:  Normal  | Resolution:
 Keywords:  |  Actual Points:
Parent ID:  | Points:
  Sponsor:  |
+

Comment (by nickm):

 The Keccak sponge function is an invertible permutation, right?  If so,
 unless I'm crazy or missing something,  I don't think their design would
 provide backtracking resistance except when new entropy is added.  Looks a
 teeny bit faster though, but probably not a huge amount.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #17799 [Tor]: Hash All PRNG output before use

[Tor Bug Tracker & Wiki]

#17799: Hash All PRNG output before use
+
 Reporter:  teor|  Owner:
 Type:  defect  | Status:  needs_revision
 Priority:  Medium  |  Milestone:  Tor: 0.2.8.x-final
Component:  Tor |Version:  Tor: unspecified
 Severity:  Normal  | Resolution:
 Keywords:  |  Actual Points:
Parent ID:  | Points:
  Sponsor:  |
+

Comment (by nickm):

 Shake_prng_v2 is rebased and mostly squashed.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #17799 [Tor]: Hash All PRNG output before use

[Tor Bug Tracker & Wiki]

#17799: Hash All PRNG output before use
+
 Reporter:  teor|  Owner:
 Type:  defect  | Status:  needs_revision
 Priority:  Medium  |  Milestone:  Tor: 0.2.8.x-final
Component:  Tor |Version:  Tor: unspecified
 Severity:  Normal  | Resolution:
 Keywords:  |  Actual Points:
Parent ID:  | Points:
  Sponsor:  |
+

Comment (by nickm):

 I added a whole bunch of fixes and changes in shake_prng, quite likely
 breaking something.

 Benchmark says it's 6.6x faster than openssl's PRNG for short-ish outputs.
 IMO this means there's no need to jump into the wacky world of libottery's
 additional hacks.

 The main change still needed would be getting #17783 merged, and then
 rebasing this and cleanup up the branch.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #17799 [Tor]: Hash All PRNG output before use

[Tor Bug Tracker & Wiki]

#17799: Hash All PRNG output before use
+
 Reporter:  teor|  Owner:
 Type:  defect  | Status:  needs_revision
 Priority:  Medium  |  Milestone:  Tor: 0.2.8.x-final
Component:  Tor |Version:  Tor: unspecified
 Severity:  Normal  | Resolution:
 Keywords:  |  Actual Points:
Parent ID:  | Points:
  Sponsor:  |
+

Comment (by yawning):

 I don't mind this in principle.

 But if we're *that* scared of OpenSSL's RAND_bytes(), we may as well wrap
 our newfangled PRNG construct in the appropriate OpenSSL struct and set it
 as the engine so that TLS and all the OpenSSL internal entropy requests
 use our thing as well.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #17799 [Tor]: Hash All PRNG output before use

[Tor Bug Tracker & Wiki]

#17799: Hash All PRNG output before use
+
 Reporter:  teor|  Owner:
 Type:  defect  | Status:  needs_revision
 Priority:  Medium  |  Milestone:  Tor: 0.2.8.x-final
Component:  Tor |Version:  Tor: unspecified
 Severity:  Normal  | Resolution:
 Keywords:  |  Actual Points:
Parent ID:  | Points:
  Sponsor:  |
+

Comment (by nickm):

 See the second patch on my branch. :)

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #17799 [Tor]: Hash All PRNG output before use

[Tor Bug Tracker & Wiki]

#17799: Hash All PRNG output before use
+
 Reporter:  teor|  Owner:
 Type:  defect  | Status:  needs_revision
 Priority:  Medium  |  Milestone:  Tor: 0.2.8.x-final
Component:  Tor |Version:  Tor: unspecified
 Severity:  Normal  | Resolution:
 Keywords:  |  Actual Points:
Parent ID:  | Points:
  Sponsor:  |
+

Comment (by yawning):

 Replying to [comment:4 nickm]:
 > See the second patch on my branch. :)

 Spiffy.  Minor quibble with the code, don't use KECCAK_MAX_RATE like that
 (Yeah, I should have renamed it/hid it).  Since you're instantiating
 SHAKE128, `KECCAK_TARGET_TO_RATE(128)` is what you want (Or 168, which is
 the SHAKE128 rate in bytes).

 It'll still work, but you're lowering your security level.  My improved
 version of the branch will be more opaque to avoid this confusion/pitfall
 and provide shake128_init/absorb/squeeze/free functions.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #17799 [Tor]: Hash All PRNG output before use

[Tor Bug Tracker & Wiki]

#17799: Hash All PRNG output before use
+
 Reporter:  teor|  Owner:
 Type:  defect  | Status:  needs_revision
 Priority:  Medium  |  Milestone:  Tor: 0.2.8.x-final
Component:  Tor |Version:  Tor: unspecified
 Severity:  Normal  | Resolution:
 Keywords:  |  Actual Points:
Parent ID:  | Points:
  Sponsor:  |
+
Changes (by nickm):

 * status:  new => needs_revision


Comment:

 I accidentally another prng.

 Branch is `shake_prng` based on Yawning's branch for #17783.

 Ready for quick looks, but not for serious review.  It needs more
 documentation, real testing, free-on-exit, windows testing,  etc.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #17799 [Tor]: Hash All PRNG output before use

[Tor Bug Tracker & Wiki]

#17799: Hash All PRNG output before use
+
 Reporter:  teor|  Owner:
 Type:  defect  | Status:  new
 Priority:  Medium  |  Milestone:  Tor: 0.2.8.x-final
Component:  Tor |Version:  Tor: unspecified
 Severity:  Normal  | Resolution:
 Keywords:  |  Actual Points:
Parent ID:  | Points:
  Sponsor:  |
+

Comment (by nickm):

 Here is the design I would suggest for such a thing.

 Replace crypto_rand() with a construction that fills an internal buffer by
 taking bytes from RAND_byes() and then passing them through SHAKE128 to
 expand them.  Then yield bytes from that buffer as required.  As yielding
 them, clear the buffer.  When the buffer is exhausted, refill it.

 If we want, we can seed the initial buffer with crypto_strongest_rand(),
 and seed subsequent buffers with a mixture of the tail-end of the previous
 buffer and RAND_bytes().

 This should meet the design criteria described above, and provide better
 performance and security than the current openssl nonsense.

 I can take care of this once the SHA3 branch (#17783) is merged.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs