Thanks a lot for taking the time to brush up the BIP. For what it's worth, I am all for these changes, and I see them as clear improvements all around.
IIRC Pieter was the one who originally suggested the two-checks approach (invalid, inconclusive, valid) which is being modified here, so would be good if you chimed in (or not -- which I'll assume means you don't mind). On Sat, Dec 19, 2020 at 12:27 AM Andrew Poelstra via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote: > > I have gone over BIP-0322 and substantially rewritten the text. > Everything I did is (I think) simply clarifying the existing > protocol, which felt like it was written by committee and wasn't > easy to follow, EXCEPT: > > 1. I rewrote the motivation section, which I believe originally > was a paraphrase of Luke-jr's general objections to having any > signmessage functionality. I hope Luke in particular can take > a look at what I wrote under "Motivation" and see if it > captures his concerns. > > 2. I merged the "consensus" and "upgradeable" rules to simply be > one set of rules, consisting of consensus checks plus additional > restrictions, all of which must be included. The new "Extensions" > section allows validators to output the state "consensus-valid" > if they really don't want to check the additional restrictions. > > 3. The "inconclusive" state, which was originally used for what I've > called "consensus-valid", now indicates that a validator does not > understand the script that it is checking (also described in the > new "Extensions" section). The goal is that implementors are able > to be meaningfully BIP-0322 while only supporting a subset of > Script, e.g. the templates that their own software supports, or > Miniscript, or the non-raw non-address set of output descriptors, > or whatever. > > We have seen opposition to supporting BIP-322, e.g. [1] because > of the requirement that you either have a full script interpreter > (plus an open-ended list of Core's standardness flags, which is > not even available through libbitcoinconsensus) or nothing. On > the other hand, the vast majority of outputs are single-key p2pkh, > p2pkwh or p2sh-wpkh. > > The new text is here (and for posterity I will also include it > inline below, though unless Github deletes it it will be easier > to read in rendered form): > > https://github.com/apoelstra/bips/blob/2020-12--bip322-overhaul/bip-0322.mediawiki > > I'll also PR this to the BIPs repo in the next day or two, and > comments on Github are then welcome. > > > [1] https://bitcointalk.org/index.php?topic=5261605.0 > > > > * * * * * Full text of the above link * * * * * > > <pre> > BIP: 322 > Layer: Applications > Title: Generic Signed Message Format > Author: Karl-Johan Alm <karljohan-...@garage.co.jp> > Comments-Summary: No comments yet. > Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0322 > Status: Draft > Type: Standards Track > Created: 2018-09-10 > License: CC0-1.0 > </pre> > > == Abstract == > > A standard for interoperable signed messages based on the Bitcoin Script > format, either for proving fund availability, or committing to a message as > the intended recipient of funds sent to the invoice address. > > == Motivation == > > The current message signing standard only works for P2PKH (1...) invoice > addresses. We propose to extend and generalize the standard by using a > Bitcoin Script based approach. This ensures that any coins, no matter what > script they are controlled by, can in-principle be signed for. For easy > interoperability with existing signing hardware, we also define a signature > message format which resembles a Bitcoin transaction (except that it contains > an invalid input, so it cannot be spent on any real network). > > Additionally, the current message signature format uses ECDSA signatures > which do not commit to the public key, meaning that they do not actually > prove knowledge of any secret keys. (Indeed, valid signatures can be tweaked > by 3rd parties to become valid signatures on certain related keys.) > > Ultimately no message signing protocol can actually prove control of funds, > both because a signature is obsolete as soon as it is created, and because > the possessor of a secret key may be willing to sign messages on others' > behalf even if it would not sign actual transactions. No signmessage protocol > can fix these limitations. > > == Types of Signatures == > > This BIP specifies three formats for signing messages: ''legacy'', ''simple'' > and ''full''. Additionally, a variant of the ''full'' format can be used to > demonstrate control over a set of UTXOs. > > === Legacy === > > New proofs should use the new format for all invoice address formats, > including P2PKH. > > The legacy format MAY be used, but must be restricted to the legacy P2PKH > invoice address format. > > === Simple === > > A ''simple'' signature consists of a witness stack, consensus encoded as a > vector of vectors of bytes, and base64-encoded. Validators should construct > <code>to_spend</code> and <code>to_sign</code> as defined below, with default > values for all fields except that > > * <code>message_hash</code> is a BIP340-tagged hash of the message, as > specified below > * <code>message_challenge</code> in <code>to_spend</code> is set to the > scriptPubKey being signed with > * <code>message_signature</code> in <code>to_sign</code> is set to the > provided simple signature. > > and then proceed as they would for a full signature. > > === Full === > > Full signatures follow an analogous specification to the BIP-325 challenges > and solutions used by Signet. > > Let there be two virtual transactions to_spend and to_sign. > > The "to_spend" transaction is: > > nVersion = 0 > nLockTime = 0 > vin[0].prevout.hash = 0000...000 > vin[0].prevout.n = 0xFFFFFFFF > vin[0].nSequence = 0 > vin[0].scriptSig = OP_0 PUSH32[ message_hash ] > vin[0].scriptWitness = [] > vout[0].nValue = 0 > vout[0].scriptPubKey = message_challenge > > where <code>message_hash</code> is a BIP340-tagged hash of the message, i.e. > sha256_tag(m), where tag = <code>BIP0322-signed-message</code>, and > <code>message_challenge</code> is the to be proven (public) key script. > > The "to_sign" transaction is: > > nVersion = 0 or as appropriate (e.g. 2, for time locks) > nLockTime = 0 or as appropriate (for time locks) > vin[0].prevout.hash = to_spend.txid > vin[0].prevout.n = 0 > vin[0].nSequence = 0 or as appropriate (for time locks) > vin[0].scriptWitness = message_signature > vout[0].nValue = 0 > vout[0].scriptPubKey = OP_RETURN > > A full signature consists of the base64-encoding of the to_spend and to_sign > transactions concatenated in standard network serialisation. > > === Full (Proof of Funds) === > > A signer may construct a proof of funds, demonstrating control of a set of > UTXOs, by constructing a full signature as above, with the following > modifications. > > * <code>message_challenge</code> is unused and shall be set to > <code>OP_TRUE</code> > * Similarly, <code>message_signature</code> is then empty. > * All outputs that the signer wishes to demonstrate control of are included > as additional outputs to <code>to_sign</code>, and their witness and > scriptSig data should be set as though these outputs were actually being > spent. > > Unlike an ordinary signature, validators of a proof of funds need access to > the current UTXO set, to learn that the claimed inputs exist on the > blockchain, and to learn their scriptPubKeys. > > == Detailed Specification == > > For all signature types, except legacy, the <code>to_spend</code> and > <code>to_sign</code> transactions must be valid transactions which pass all > consensus checks, except of course that the output with prevout > <code>000...000:FFFFFFFF</code> does not exist. > > We additionally require the following restrictions be met. > > * All signatures must use the SIGHASH_ALL flag. > * The use of <code>CODESEPARATOR</code> or <code>FindAndDelete</code> is > forbidden. > * The use of NOPs reserved for upgrades is forbidden. > * The use of segwit versions greater than 1 are forbidden. > * <code>LOW_S</code>, <code>STRICTENC</code> and <code>NULLFAIL</code>: valid > ECDSA signatures must be strictly DER-encoded and have a low-S value; invalid > ECDSA signature must be the empty push > * <code>MINIMALDATA</code>: all pushes must be minimally encoded > * <code>CLEANSTACK</code>: require that only a single stack element remains > after evaluation > * <code>MINIMALIF</code>: the argument of <code>IF</code>/<code>NOTIF</code> > must be exactly 0x01 or empty push > > Future versions of this BIP may relax these rules, in particular those around > NOPs and future Segwit versions, as they are deployed on Bitcoin. > > === Verification === > > Validation consists of the following steps. A validator is given as input an > address ''A'' (which may be omitted in a proof-of-funds), signature ''s'' and > message ''m'', and outputs one of four states (although validators are only > required to be able to output the first and last): > * ''valid'' indicates that the signature passed all checks described below > * ''valid at time t and age s'' indicates that the signature has set > timelocks but is otherwise valid (see "Extensions" below) > * ''consensus-valid'' indicates that the signature passed validation except > for the additonal restrictions in the above section (see "Extensions" below) > * ''inconclusive'' means the validator was unable to check the scripts (see > "Extensions" below) > * ''invalid'' means none of the other states > > # Decode ''s'' as the transactions <code>to_sign</code> and > <code>to_spend</code> > # Confirm that <code>message_hash</code> is the correct hash of ''m'' > # Confirm that <code>message_challenge</code> is the scriptPubKey > corresponding to ''A'' if ''A'' is present, and otherwise must be > <code>OP_TRUE</code> > # Confirm that all other fields are set as specified above; in particular that > ** <code>to_spend</code> has exactly one input and one output > ** <code>to_sign</code> has at least one input and its first input spends the > output of </code>to_spend</code> > ** <code>to_sign</code> has exactly one output, as specified above > # Confirm that the two transactions together satisfy all consensus rules, > except for <code>to_spend</code>'s missing input, and except that > ''nSequence'' of <code>to_sign</code>'s first input and ''nLockTime'' of > <code>to_sign</code> are not checked. > # Confirm that all of the above restrictions are met. > > If the above conditions are met, the signature is considered ''valid''. > Otherwise the signature is ''invalid''. > > === Signing === > > Signers who control an address ''A'' who wish to sign a message ''m'' act as > follows: > > # They construct <code>to_spend</code> and <code>to_sign</code> as specified > above, using the scriptPubKey of ''A'' for <code>message_challenge</code> and > tagged hash of ''m'' as <code>message_hash</code>. > # Optionally, they may set nLockTime of <code>to_sign</code> or nSequence of > its first input. > # Optionally, they may add any additional outputs to <code>to_sign</code> > that they wish to prove control of. > # They satisfy <code>to_sign</code> as they would any other transaction. > > They then encode their signature, choosing either ''simple'' or ''full'' as > follows: > > * If they added no inputs to <code>to_sign</code>, left nSequence and > nLockTime at 0, and ''A'' is a Segwit address (either pure or P2SH-wrapped), > then they may base64-encode <code>message_signature</code> > * Otherwise they must base64-encode the concatenation of > <code>to_spend</code> followed by <code>to_sign</code>. > > == Extensions == > > To ease implementation, we allow some additional states to be output rather > than ''valid'' or ''invalid''. Users who do not understand or who do not wish > to deal with these states may treat them as ''invalid''. > > === Timelocks === > > If the nLockTime of <code>to_sign</code> is set to ''t'', and the nSequence > of the first input of <code>to_sign</code> is set to ''s'', the validator may > output the state ''valid at time t and age s''. > > If both ''t'' and ''s'' are 0, the validator must instead output ''valid''. > > Users may then wish to interpret this state as ''valid'' or ''invalid'' > relative to the state of the current blockchain, but the rules for doing so > are out of scope of this BIP. > > === Incomplete Validation === > > Some validators may not wish to implement a full script interpreter, choosing > instead to support only specific script templates, or only Miniscript, for > example. In this case, if they are unable to execute the scripts used by > <code>to_sign</code>, they should output the state ''inconclusive''. > > Users should interpret this state as the same thing as ''invalid'', although > take it as a sign that they should find more capable software. > > === Consensus-Only Validation === > > Validators which are only able to check consensus-correctness of witnesses, > but not the additional restrictions imposed by this BIP, may output the state > ''consensus-valid'' to indicate that a signature has passed all consensus and > structural checks. > > Users should interpret this state as the same thing as ''valid'' but be aware > that other software may fail to validate the same signature. > > == Compatibility == > > This specification is backwards compatible with the legacy > signmessage/verifymessage specification through the special case as described > above. > > == Reference implementation == > > TODO > > == Acknowledgements == > > Thanks to David Harding, Jim Posen, Kalle Rosenbaum, Pieter Wuille, Andrew > Poelstra, and many others for their feedback on the specification. > > == References == > > # Original mailing list thread: > https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-March/015818.html > > == Copyright == > > This document is licensed under the Creative Commons CC0 1.0 Universal > license. > > == Test vectors == > > TODO > > * * * * * End full text * * * * * > > -- > Andrew Poelstra > Director of Research, Blockstream > Email: apoelstra at wpsoftware.net > Web: https://www.wpsoftware.net/andrew > > The sun is always shining in space > -Justin Lewis-Webster > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev _______________________________________________ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
