Re: [bitcoin-dev] Segregated witnesses and validationless mining

2015-12-31 Thread Peter Todd via bitcoin-dev
On Tue, Dec 22, 2015 at 05:31:19PM -0800, Peter Todd via bitcoin-dev wrote:
> # Summary

Updates from IRC discussion:

1) There was some debate about what exactly should be required from the
current block to calculate the previous block posession proof. For
instance, requiring the coinbase outputs potentially restricts some
mining setups; requiring a commitment to the current block's
(non-coinbase) transaction outputs restricts tx selection outsourcing

However, it appears that we can allow the nonce to be picked
arbitrarily. Equally, if the nonce is arbitrary, then a future soft-fork
can be add commitments to current block contents. Thus the previous
block proof can be simple H( + )

2) Pieter Wuille brought up fraud proofs in relation to previous block
content proofs - specifically how the simplest H( +
) construction requires a large fraud proof to
prove incorrect. This followed a bunch of debate over what exactly fraud
proofs would be - a proof that some data is fraudulent, or a unmet
challenge that some data is correct?

Regardless, if the posession proof is structured as a merkle tree, then
fraud can be easily proven with a merkle path. In that model we'd take
the previous block contents and rehash it in its entirety with the
nonce. The fraud proof then becomes two merkle paths - one in the
original block with the original hash, and the second with the same
data, and same structure, but with the nonce mixed into the hashing

Todo: writeup the difference between the fraud proof model, and the
validity challenge model, to provide background to making this decision.

Incidentally, based the positive response to fixing this issue w/
segregated witnesses - my main objection to the plan - I've signed the
Bitcoin Core capacity increases statement:


Description: Digital signature
bitcoin-dev mailing list

Re: [bitcoin-dev] Segregated witnesses and validationless mining

2015-12-23 Thread Peter Todd via bitcoin-dev
On Tue, Dec 22, 2015 at 05:31:19PM -0800, Peter Todd via bitcoin-dev wrote:
> # Easy solution: previous witness data proof
> To return segregated witnesses to the status quo, we need to at least
> make having the previous block's witness data be a precondition to
> creating a block with transactions; ideally we would make it a
> precondition to making any valid block, although going this far may
> receive pushback from miners who are currently using validationless
> mining techniques.
> We can require blocks to include the previous witness data, hashed with
> a different hash function that the commitment in the previous block.
> With witness data W, and H(W) the witness commitment in the previous
> block, require the current block to include H'(W)
> A possible concrete implementation would be to compute the hash of the
> current block's coinbase txouts (unique per miner for obvious reasons!)
> as well as the previous block hash. Then recompute the previous block's
> witness data merkle tree (and optionally, transaction data merkle tree)
> with that hash prepended to the serialized data for each witness.
> This calculation can only be done by a trusted entity with access to all
> witness data from the previous block, forcing miners to both publish
> their witness data promptly, as well as at least obtain witness data
> from other miners. (if not actually validate it!) This returns us to at
> least the status quo, if not slightly better.
> This solution is a soft-fork. As the calculation is only done once per
> block, it is *not* a change to the PoW algorithm and is thus compatible
> with existing miner/hasher setups. (modulo validationless mining
> optimizations, which are no longer possible)

Note that this fix can be designed to retain the possibility of
validationless mining, by allowing empty blocks to be created if the
previous witness data proof is omitted. This would achieve the same goal
as Gregory Maxwell's blockchain verification flag(1) but with
significantly less ability/reason to lie about the status of that flag.

1) [bitcoin-dev] Blockchain verification flag (BIP draft),
   Gregory Maxwell, Dec 4th 2015,


Description: Digital signature
bitcoin-dev mailing list

[bitcoin-dev] Segregated witnesses and validationless mining

2015-12-22 Thread Peter Todd via bitcoin-dev
# Summary

1) Segregated witnesses separates transaction information about what
coins were transferred from the information proving those transfers were

2) In its current form, segregated witnesses makes validationless mining
easier and more profitable than the status quo, particularly as
transaction fees increase in relevance.

3) This can be easily fixed by changing the protocol to make having a
copy of the previous block's (witness) data a precondition to creating a

# Background

## Why should a miner publish the blocks they find?

Suppose Alice has negligible hashing power. She finds a block. Should
she publish that block to the rest of the hashing power? Yes! If she
doesn't publish, the rest of the hashing power will build a longer chain
than her chain, and she won't be rewarded. Right?

Well, can other miners build on top of Alice's block? If she publishes
nothing at all, the answer is certainely no - block headers commit to
the previous block's hash, so without knowing at least the hash of
Alice's block other miners can't build upon it.

## Validationless mining

Suppose Bob knows the hash of Alice's new block, as well as the height
of it. This is sufficient information for Bob to create a new, valid,
block building upon Alice's block. The hash is needed because of the
prevhash field in the block header; the height is needed because the
coinbase has to contain the block height. (technically he needs to know
nTime as well to be 100% sure he's satisfying the median time rule) What
Bob is doing is validationless mining: he hasn't validated Alice's
block, and is assuming it is valid.

If Alice runs a pool her stratum or getblocktemplate interfaces give
sufficient information for Bob to figure all this out. Miners today take
advantage of this to reduce their orphan rates - the sooner you can
start mining on top of the most recently found block the more money you
earn. Pools have strong incentives to only publish work that's valid to
their hashers, so as long as the target pool doesn't know who you are,
you have high assurance that the block hash you're building upon is

Of course, when this goes wrong it goes very wrong, greatly amplifying
the effect of 51% attacks and technical screwups, as seen by the July
4th 2015 chain fork, where a majority of hashing power was building on
top of an invalid block.

## Transactions

However other than coinbase transactions, validationless mined blocks
are nearly always empty: if Bob doesn't know what transactions Alice
included in her block, he doesn't know what transaction outputs are
still unspent and can't safely include transactions in his block. In
short, Bob doesn't know what the current state of the UTXO set is. This
helps limit the danger of validationless mining by making it visible to
everyone, as well as making it not as profitable due to the inability to
collect transaction fees. (among other reasons)

# Segregated witnesses and validationless mining

With segregated witnesses the information required to update the UTXO
set state is now separate from the information required to prove that
the new state is valid. We can fully expect miners to take advantage of
this to reduce latency and thus improve their profitability.

We can expect block relaying with segregated witnesses to separate block
propagation into four different parts, from fastest to propagate to

1) Stratum/getblocktemplate - status quo between semi-trusting miners

2) Block header - bare minimum information needed to build upon a block.
Not much trust required as creating an invalid header is expensive.

3) Block w/o witness data - significant bandwidth savings, (~75%) and
allows next miner to include transactions as normal. Again, not much
trust required as creating an invalid header is expensive.

4) Witness data - proves that block is actually valid.

The problem is #4 is optional: the only case where not having the
witness data matters is when an invalid block is created, which is a
very rare event. It's also difficult to test in production, as creating
invalid blocks is extremely expensive - it would be surprising if an
anyone had ever deliberately created an invalid block meeting the
current difficulty target in the past year or two.

# The nightmare scenario - never tested code ~never works

The obvious implementation of highly optimised mining with segregated
witnesses will have the main codepath that creates blocks do no
validation at all; if the current ecosystem's validationless mining is
any indication the actual code doing this will be proprietary codebases
written on a budget with little testing, and lots of bugs. At best the
codepaths that actually do validation will be rarely, if ever, tested in

Secondly, as the UTXO set can be updated without the witness data, it
would not be surprising if at least some of the wallet ecosystem skips
witness validation.

With that in mind, what happens in the event of a validation