Re: [zapps-wg] Version of Rust
On Tue, Nov 21, 2017 at 05:34:04AM +, Devrandom wrote: > On Mon, Nov 20, 2017 at 9:30 PM Peter Todd wrote: > > > On Mon, Nov 20, 2017 at 10:11:00PM +, Devrandom wrote: > > > Update on boostrapping via mrustc - I'm waiting on the author to resolve > > > https://github.com/thepowersgang/mrustc/issues/33. > > > > FWIW, I'm pretty sure you don't actually need cargo to compile the rust > > compiler; have you gotten mrustc to compile Rust itself? > > > > The actual failure is in the rustc that was built by mrustc - see last line > of pasted blob. Cargo is just used for convenience to trigger the build > steps (and will be replaced by a script once this issue fixed). Ah I see - you're using the Original Rust Cargo binary, telling it to call mrustc's rustc instead of the Original Rust rustsc, with the aim of building the original Rust compiler with mrustc, and the error was in mrustc's rustc. Yes, I think that's a pretty reasonable first step! Just a confusing mishmash of names. :) -- https://petertodd.org 'peter'[:-1]@petertodd.org signature.asc Description: Digital signature
Re: [zapps-wg] Version of Rust
On Mon, Nov 20, 2017 at 9:30 PM Peter Todd wrote: > On Mon, Nov 20, 2017 at 10:11:00PM +, Devrandom wrote: > > Update on boostrapping via mrustc - I'm waiting on the author to resolve > > https://github.com/thepowersgang/mrustc/issues/33. > > FWIW, I'm pretty sure you don't actually need cargo to compile the rust > compiler; have you gotten mrustc to compile Rust itself? > The actual failure is in the rustc that was built by mrustc - see last line of pasted blob. Cargo is just used for convenience to trigger the build steps (and will be replaced by a script once this issue fixed).
Re: [zapps-wg] Version of Rust
On Mon, Nov 20, 2017 at 9:27 PM Peter Todd wrote: > On Sat, Nov 18, 2017 at 01:46:10AM +, Devrandom wrote: > > I made some progress with walking back the Rust version, getting to > > 2016-04-01 nightly (pre-1.8.0). A rough cut of what I did is here: > > > > > https://github.com/devrandom/powersoftau/compare/experiment-devrandom-vendor...devrandom:experiment-devrandom-old-compile > > > > This doesn't quite get us to when zcash was announced in January of 2016. > > To make further progress, I will look into backporting the question-mark > > operator from > > > https://github.com/rust-lang/rust/commit/210dd611aa1bd80ed2f4e663b3c4b87b3cea069a > > to > > an earlier nightly. > > Not sure how familiar you are with Rust, but FWIW it might be easier to > just > mechanically replace the question mark operator with the try! macro. For > example, this line: > > let alpha_g1_s = read_uncompressed(reader)?; > > simply becomes: > > let alpha_g1_s = try!(read_uncompressed(reader)); > > This has the advantage of having compatibility with potentially quite old > versions of Rust. > Will try that if the mrustc path continues to be blocked for longer than a few days.
Re: [zapps-wg] Version of Rust
On Mon, Nov 20, 2017 at 10:11:00PM +, Devrandom wrote: > Update on boostrapping via mrustc - I'm waiting on the author to resolve > https://github.com/thepowersgang/mrustc/issues/33. FWIW, I'm pretty sure you don't actually need cargo to compile the rust compiler; have you gotten mrustc to compile Rust itself? -- https://petertodd.org 'peter'[:-1]@petertodd.org signature.asc Description: Digital signature
Re: [zapps-wg] Version of Rust
On Sat, Nov 18, 2017 at 01:46:10AM +, Devrandom wrote: > I made some progress with walking back the Rust version, getting to > 2016-04-01 nightly (pre-1.8.0). A rough cut of what I did is here: > > https://github.com/devrandom/powersoftau/compare/experiment-devrandom-vendor...devrandom:experiment-devrandom-old-compile > > This doesn't quite get us to when zcash was announced in January of 2016. > To make further progress, I will look into backporting the question-mark > operator from > https://github.com/rust-lang/rust/commit/210dd611aa1bd80ed2f4e663b3c4b87b3cea069a > to > an earlier nightly. Not sure how familiar you are with Rust, but FWIW it might be easier to just mechanically replace the question mark operator with the try! macro. For example, this line: let alpha_g1_s = read_uncompressed(reader)?; simply becomes: let alpha_g1_s = try!(read_uncompressed(reader)); This has the advantage of having compatibility with potentially quite old versions of Rust. -- https://petertodd.org 'peter'[:-1]@petertodd.org signature.asc Description: Digital signature
Re: [zapps-wg] Version of Rust
Update on boostrapping via mrustc - I'm waiting on the author to resolve https://github.com/thepowersgang/mrustc/issues/33. On Fri, Nov 17, 2017 at 6:54 PM Devrandom wrote: > The mrustc author confirms that bootstrapping rustc is feasible. Will > explore that next. > > > On Fri, Nov 17, 2017 at 5:46 PM Devrandom > wrote: > >> I made some progress with walking back the Rust version, getting to >> 2016-04-01 nightly (pre-1.8.0). A rough cut of what I did is here: >> >> >> https://github.com/devrandom/powersoftau/compare/experiment-devrandom-vendor...devrandom:experiment-devrandom-old-compile >> >> This doesn't quite get us to when zcash was announced in January of >> 2016. To make further progress, I will look into backporting the >> question-mark operator from >> https://github.com/rust-lang/rust/commit/210dd611aa1bd80ed2f4e663b3c4b87b3cea069a >> to >> an earlier nightly. >> >> Will consider Peter's points and respond later. >> >>
Re: [zapps-wg] Version of Rust
The mrustc author confirms that bootstrapping rustc is feasible. Will explore that next. On Fri, Nov 17, 2017 at 5:46 PM Devrandom wrote: > I made some progress with walking back the Rust version, getting to > 2016-04-01 nightly (pre-1.8.0). A rough cut of what I did is here: > > > https://github.com/devrandom/powersoftau/compare/experiment-devrandom-vendor...devrandom:experiment-devrandom-old-compile > > This doesn't quite get us to when zcash was announced in January of 2016. > To make further progress, I will look into backporting the question-mark > operator from > https://github.com/rust-lang/rust/commit/210dd611aa1bd80ed2f4e663b3c4b87b3cea069a > to > an earlier nightly. > > Will consider Peter's points and respond later. > >
Re: [zapps-wg] Version of Rust
I made some progress with walking back the Rust version, getting to 2016-04-01 nightly (pre-1.8.0). A rough cut of what I did is here: https://github.com/devrandom/powersoftau/compare/experiment-devrandom-vendor...devrandom:experiment-devrandom-old-compile This doesn't quite get us to when zcash was announced in January of 2016. To make further progress, I will look into backporting the question-mark operator from https://github.com/rust-lang/rust/commit/210dd611aa1bd80ed2f4e663b3c4b87b3cea069a to an earlier nightly. Will consider Peter's points and respond later.
Re: [zapps-wg] Version of Rust
On Fri, Nov 17, 2017 at 05:08:02PM +, Devrandom wrote:
> I'm going to try to compile with older versions, will let you know how it
> goes. Ideally we could get to versions before zcash was announced.
>
> This will be fun, have not done Rust before. But here are some thoughts
> about the Rust compiler after trying to compile it from source:
>
> - the Rust compiler can only be built with itself, which means that you
> have to start from a binary, so you might as well trust the binary
Nope! There's an alternate Rust compiler written in C++ called mrustc:
https://github.com/thepowersgang/mrustc
It's not suitable for general use as it lacks a borrow checker, but Rust only
needs the borrow checker to catch mistakes; if your code is perfect alredy you
don't need it. Thus mrustc is suitable for compiling the Rust compiler itself,
something I hear it is able to do. Assuming you can in fact get that to work,
that's sufficient to do diverse double-compilation.
Secondly, if you're feeling adventerous, you could try re-bootstrapping it from
the original OCaml implementation. :)
FWIW I have a timestamp on mrustc from a few days ago:
https://github.com/petertodd/mrustc/commit/6b30d486dc3b653302fd4bc4887b92e5feb1044c
> - if we start with a really old one, the chances of a targeted attack are
> pretty slim
Agreed, but note that a targeted attack isn't our *only* threat.
When you think about it, compiler backdoors are basically a class of exploit
where the attack causes source code to be miscompiled into machine code that
the auditors looking at the source code didn't expect. This means that a
compiler backdoor can be generic, and can be prepared in advance.
Basically, in this scenario the Rust compiler would have a subtle compilation
flaw that could be maliciously triggered by the powersoftau authors to produce
machine code that did something other than what you'd expect it to do, e.g.
causing a PRNG to zero out part of it's entropy pool. Starting with a really
old Rust compiler doesn't necessarily protect us from this attack; this type of
attack may also be done with an *accidental* compiler bug too (although it'd be
harder to find a suitable one).
A cheap way to defeat this intentional miscompilation attack is to simply
modify the powersoftau codebase as much as possible so that any hypothetical
miscompilations are no longer triggered. For example, even if you didn't have
the skillset to understand the code, you could mechanically go through the
codebase (and dependencies) and replace all integers with a hexadecimal string
type, and reimplement the necessary operator traits for that new type.
Sure, that'd be slow as heck, but it doesn't have to be fast: by using a
deterministic secret you can compare your ridiculous hex string implementation
with the reference implementation offline. If both implementations match,
you've just ruled out a whole class of intentional miscompilation attacks.
> - a general trojan that detects access to /dev/random and reduces entropy
> could still be lurking
> - the mixing of entropy from the user should mitigate, unless the compiler
> can be smart enough to detect even that
A trojan that can detect access to /dev/random would be in a position to do
whatever it wanted to the powersoftau binary; this is basically the same
problem as getting a clean Rust compiler.
> BTW, what does the timestamp on crates.io buy us? Looking at the repo, it
> looks like they just commit version numbers, but not git hashes of the
> source code.
They do. For example, powersoftau depends on the blake2 crate, v0.6.1
The relevant crates.io-index entry is at
https://github.com/rust-lang/crates.io-index/blob/master/bl/ak/blake2,
specifically:
{"name":"blake2","vers":"0.6.1","deps":[{"name":"digest","req":"^0.6","features":[],"optional":false,"default_features":true,"target":null,"kind":"normal"},{"name":"crypto-mac","req":"^0.4","features":[],"optional":false,"default_features":true,"target":null,"kind":"normal"},{"name":"generic-array","req":"^0.8","features":[],"optional":false,"default_features":true,"target":null,"kind":"normal"},{"name":"byte-tools","req":"^0.2","features":[],"optional":false,"default_features":true,"target":null,"kind":"normal"},{"name":"crypto-tests","req":"^0.5","features":[],"optional":false,"default_features":true,"target":null,"kind":"dev"}],"cksum":"53bf612c0f2839b7e764ebac65d6cb985f7c6812de399d0728038f4b1da141bc","features":{"simd_asm":["simd_opt"],"simd_opt":["simd"],"simd":[]},"yanked":false}
The "chsum" field is actually the SHA256 hash of the crate file for that
specific version, and the crate file is simply a tarball of the sourcecode.
You'll also note how the Cargo.lock file in the powersoftau repo specifies the
exact hash digest in the [metadata] section at the end:
"checksum blake2 0.6.1
(registry+https://github.com/rust-lang/crates.io-index)" =
"53bf612c0f2839b7e764ebac65d6cb985f7c6812de399d0728038f4b1da141bc"
--
https://petertod
Re: [zapps-wg] Version of Rust
I'm going to try to compile with older versions, will let you know how it goes. Ideally we could get to versions before zcash was announced. This will be fun, have not done Rust before. But here are some thoughts about the Rust compiler after trying to compile it from source: - the Rust compiler can only be built with itself, which means that you have to start from a binary, so you might as well trust the binary - if we start with a really old one, the chances of a targeted attack are pretty slim - a general trojan that detects access to /dev/random and reduces entropy could still be lurking - the mixing of entropy from the user should mitigate, unless the compiler can be smart enough to detect even that BTW, what does the timestamp on crates.io buy us? Looking at the repo, it looks like they just commit version numbers, but not git hashes of the source code. On Thu, Nov 16, 2017 at 4:45 PM Peter Todd wrote: > On Thu, Nov 16, 2017 at 01:59:53PM -0700, Sean Bowe via zapps-wg wrote: > > I think it is the current version (1.21). I imagine it would be > > possible to modify the code (and many of the dependencies) so that it > > could compile on a really old version too. > > Also, if someone does manage to do this, I have a OpenTimestamps Git > timestamp(1) > on Rust crates.io crate registry: > > > https://github.com/petertodd/crates.io-index/commit/763a730f2275d69eb13ee8b212fc9aa0d6fe92b5 > > Secondly the Internet Archive contains quite a bit of uploaded software, > such > as Debian install images, and via my Internet Archive timestamp project we > have > timestamps from May this year for most of that: > > > https://petertodd.org/2017/carbon-dating-the-internet-archive-with-opentimestamps > > While these timestamps would be only one part of an argument as to why a > given > compile wasn't backdoored, I think it's worth using cryptographically > timestamped dependencies over non-timestamped ones when possible. > > > 1) https://petertodd.org/2016/opentimestamps-git-integration > > -- > https://petertodd.org 'peter'[:-1]@petertodd.org >
Re: [zapps-wg] Version of Rust
On Thu, Nov 16, 2017 at 01:59:53PM -0700, Sean Bowe via zapps-wg wrote: > I think it is the current version (1.21). I imagine it would be > possible to modify the code (and many of the dependencies) so that it > could compile on a really old version too. Also, if someone does manage to do this, I have a OpenTimestamps Git timestamp(1) on Rust crates.io crate registry: https://github.com/petertodd/crates.io-index/commit/763a730f2275d69eb13ee8b212fc9aa0d6fe92b5 Secondly the Internet Archive contains quite a bit of uploaded software, such as Debian install images, and via my Internet Archive timestamp project we have timestamps from May this year for most of that: https://petertodd.org/2017/carbon-dating-the-internet-archive-with-opentimestamps While these timestamps would be only one part of an argument as to why a given compile wasn't backdoored, I think it's worth using cryptographically timestamped dependencies over non-timestamped ones when possible. 1) https://petertodd.org/2016/opentimestamps-git-integration -- https://petertodd.org 'peter'[:-1]@petertodd.org signature.asc Description: Digital signature
Re: [zapps-wg] Version of Rust
I think it is the current version (1.21). I imagine it would be possible to modify the code (and many of the dependencies) so that it could compile on a really old version too. Sean On Thu, Nov 16, 2017 at 1:35 PM, Devrandom via zapps-wg wrote: > Hi Sean, > > Do you know what is the oldest version of Rust that will successfully > compile the Tau software? >
[zapps-wg] Version of Rust
Hi Sean, Do you know what is the oldest version of Rust that will successfully compile the Tau software?
