Re: [bitcoin-dev] BIP-0322 (generic signmessage) improvements
On Tue, Dec 22, 2020 at 12:22:37AM +, Pieter Wuille via bitcoin-dev wrote: > > Re-reading your proposed text, I'm wondering if the "consensus-only > validation" extension is intended to replace the > inconclusive-due-to-consensus-and-standardness-differ state. If so, I don't > think it does, and regardless it doesn't seem very useful. > > What I'm suggestion could be specified this way: > * If validator understands the script: > * If signature is consensus valid (as far as the validator knows): > * If signature is not known to trigger standardness rules intended for > future extension (well-defined set of rules listed in BIP, and revisable): > return valid > * Otherwise: return inconclusive > * Otherwise: return invalid > * Otherwise: return inconclusive > > Or in other words: every signature has a well-defined result (valid, invalid, > inconclusive) + validators may choose to report inconclusive for anything > they don't understand. > > This has the property that as long as new consensus rules only change things > that were covered under for-future-extension standardness rules, no two > validators will ever claim valid and invalid for the same signature. Only > valid+inconclusive or invalid+inconclusive. > I've updated my PR at https://github.com/bitcoin/bips/pull/1048 Differences: 1. I compacted all the validation states into three: valid at time/age T/S, invalid, and inconclusive. 2. "Inconclusive" means either an "upgradeable rule" failed, e.g. use of a NOP or a bad network version, or the validator just didn't understand the scripts. 3. I removed the "Extensions" sections now everything is in the main protocol. 4. I removed the "to_sign" transaction from the wire serialization, since after all this, it can always be inferred from the message and address. (This does mean, however, that there is no way to sign for scriptPubKeys that don't have addresses, e.g. bare public keys or multisigs. I don't think it's worth complicated the protocol for such obscure things.) -- Andrew Poelstra Director of Research, Blockstream Email: apoelstra at wpsoftware.net Web: https://www.wpsoftware.net/andrew The sun is always shining in space -Justin Lewis-Webster signature.asc Description: PGP signature ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] BIP-0322 (generic signmessage) improvements
On Tue, Dec 22, 2020 at 12:22:37AM +, Pieter Wuille via bitcoin-dev wrote: > > Re-reading your proposed text, I'm wondering if the "consensus-only > validation" extension is intended to replace the > inconclusive-due-to-consensus-and-standardness-differ state. If so, I don't > think it does, and regardless it doesn't seem very useful. > > What I'm suggestion could be specified this way: > * If validator understands the script: > * If signature is consensus valid (as far as the validator knows): > * If signature is not known to trigger standardness rules intended for > future extension (well-defined set of rules listed in BIP, and revisable): > return valid > * Otherwise: return inconclusive > * Otherwise: return invalid > * Otherwise: return inconclusive > > Or in other words: every signature has a well-defined result (valid, invalid, > inconclusive) + validators may choose to report inconclusive for anything > they don't understand. > > This has the property that as long as new consensus rules only change things > that were covered under for-future-extension standardness rules, no two > validators will ever claim valid and invalid for the same signature. Only > valid+inconclusive or invalid+inconclusive. > I like it! My thinking regarding standardness vs consensus rules was essentially that I wanted to enforce the included standardness rules for anti-malleability reasons, i.e. the hope that for "normal scripts" we would get strong signatures, which may be important for anti-DoS reasons. (What I mean by this is that if you can easily create mutations of signatures, it may confuse software in similar ways to the Gox-era malleability attacks on wallet software of the time.) But conversely, it is hard to enforce these rules as an implementor, because libbitcoinconsensus does not expose them. So allowing both forms of validation, to me, was an attempt to encourage adoption rather than anything principled. I didn't even consider the idea that validators should be able to signal "this signature appears to use future consensus rules", although I should have been clued in by your "upgradeable rules" language that this was your goal. Now that you say this, it's obvious that this is desireable, and also obvious that using the "inconclusive" state is an elegant way to achieve this. I also agree that "confirming validators should never disagree on valid vs invalid" is a good design goal and we should make that explicit. I'll add a commit to my PR at https://github.com/bitcoin/bips/pull/1048 which adds these thoughts. -- Andrew Poelstra Director of Research, Blockstream Email: apoelstra at wpsoftware.net Web: https://www.wpsoftware.net/andrew The sun is always shining in space -Justin Lewis-Webster signature.asc Description: PGP signature ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] BIP-0322 (generic signmessage) improvements
On Monday, December 21, 2020 2:57 PM, Pieter Wuille via bitcoin-dev wrote: > On Sunday, December 20, 2020 9:37 PM, Karl-Johan Alm via bitcoin-dev > bitcoin-dev@lists.linuxfoundation.org wrote: > > > Thanks a lot for taking the time to brush up the BIP. For what it's > > worth, I am all for these changes, and I see them as clear > > improvements all around. > > IIRC Pieter was the one who originally suggested the two-checks > > approach (invalid, inconclusive, valid) which is being modified here, > > so would be good if you chimed in (or not -- which I'll assume means > > you don't mind). > > I agree with the idea of permitting incomplete validators to return > inconclusive as well. That doesn't really reduce the functionality (given > that "inconclusive" was already a potential result), and it obviously makes > it much more accessible to a variety of software. > > This suggestion breaks the original use of inconclusive though: the ability > to detect that future features are used in the signature. The idea was to use > divergence between "consensus valid" and "standardness valid" as a proxy for > future extensions to be detected (e.g. OP_NOPn, future witness versions, > ...). I think it's undesirable that these things now become unconditionally > invalid (until the BIP is updated, but once that happens old validators will > give a different result than new ones). > > Since the BIP no longer relies on a nebulous concept of standardness, and > instead specifically defines which standardness features are to be > considered, this seems easy to fix: whenever validation fails due to any of > those, require reporting inconclusive instead of invalid (unless of course > something actually invalid also happens). In practice I guess you'd implement > that (in capable validators) by still doing validation twice, once with all > features enabled that distinguish between valid/invalid, and if valid, again > but now with the features enabled that distinguish between valid and (invalid > or inconclusive). Re-reading your proposed text, I'm wondering if the "consensus-only validation" extension is intended to replace the inconclusive-due-to-consensus-and-standardness-differ state. If so, I don't think it does, and regardless it doesn't seem very useful. What I'm suggestion could be specified this way: * If validator understands the script: * If signature is consensus valid (as far as the validator knows): * If signature is not known to trigger standardness rules intended for future extension (well-defined set of rules listed in BIP, and revisable): return valid * Otherwise: return inconclusive * Otherwise: return invalid * Otherwise: return inconclusive Or in other words: every signature has a well-defined result (valid, invalid, inconclusive) + validators may choose to report inconclusive for anything they don't understand. This has the property that as long as new consensus rules only change things that were covered under for-future-extension standardness rules, no two validators will ever claim valid and invalid for the same signature. Only valid+inconclusive or invalid+inconclusive. Cheers, -- Pieter ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] BIP-0322 (generic signmessage) improvements
On Sunday, December 20, 2020 9:37 PM, Karl-Johan Alm via bitcoin-dev wrote: > Thanks a lot for taking the time to brush up the BIP. For what it's > worth, I am all for these changes, and I see them as clear > improvements all around. > > IIRC Pieter was the one who originally suggested the two-checks > approach (invalid, inconclusive, valid) which is being modified here, > so would be good if you chimed in (or not -- which I'll assume means > you don't mind). I agree with the idea of permitting incomplete validators to return inconclusive as well. That doesn't really reduce the functionality (given that "inconclusive" was already a potential result), and it obviously makes it much more accessible to a variety of software. This suggestion breaks the original use of inconclusive though: the ability to detect that future features are used in the signature. The idea was to use divergence between "consensus valid" and "standardness valid" as a proxy for future extensions to be detected (e.g. OP_NOPn, future witness versions, ...). I think it's undesirable that these things now become unconditionally invalid (until the BIP is updated, but once that happens old validators will give a different result than new ones). Since the BIP no longer relies on a nebulous concept of standardness, and instead specifically defines which standardness features are to be considered, this seems easy to fix: whenever validation fails due to any of those, require reporting inconclusive instead of invalid (unless of course something actually invalid also happens). In practice I guess you'd implement that (in capable validators) by still doing validation twice, once with all features enabled that distinguish between valid/invalid, and if valid, again but now with the features enabled that distinguish between valid and (invalid or inconclusive). Cheers, -- Pieter ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] BIP-0322 (generic signmessage) improvements
Thanks a lot for taking the time to brush up the BIP. For what it's worth, I am all for these changes, and I see them as clear improvements all around. IIRC Pieter was the one who originally suggested the two-checks approach (invalid, inconclusive, valid) which is being modified here, so would be good if you chimed in (or not -- which I'll assume means you don't mind). On Sat, Dec 19, 2020 at 12:27 AM Andrew Poelstra via bitcoin-dev wrote: > > I have gone over BIP-0322 and substantially rewritten the text. > Everything I did is (I think) simply clarifying the existing > protocol, which felt like it was written by committee and wasn't > easy to follow, EXCEPT: > > 1. I rewrote the motivation section, which I believe originally >was a paraphrase of Luke-jr's general objections to having any >signmessage functionality. I hope Luke in particular can take >a look at what I wrote under "Motivation" and see if it >captures his concerns. > > 2. I merged the "consensus" and "upgradeable" rules to simply be >one set of rules, consisting of consensus checks plus additional >restrictions, all of which must be included. The new "Extensions" >section allows validators to output the state "consensus-valid" >if they really don't want to check the additional restrictions. > > 3. The "inconclusive" state, which was originally used for what I've >called "consensus-valid", now indicates that a validator does not >understand the script that it is checking (also described in the >new "Extensions" section). The goal is that implementors are able >to be meaningfully BIP-0322 while only supporting a subset of >Script, e.g. the templates that their own software supports, or >Miniscript, or the non-raw non-address set of output descriptors, >or whatever. > >We have seen opposition to supporting BIP-322, e.g. [1] because >of the requirement that you either have a full script interpreter >(plus an open-ended list of Core's standardness flags, which is >not even available through libbitcoinconsensus) or nothing. On >the other hand, the vast majority of outputs are single-key p2pkh, >p2pkwh or p2sh-wpkh. > > The new text is here (and for posterity I will also include it > inline below, though unless Github deletes it it will be easier > to read in rendered form): > > https://github.com/apoelstra/bips/blob/2020-12--bip322-overhaul/bip-0322.mediawiki > > I'll also PR this to the BIPs repo in the next day or two, and > comments on Github are then welcome. > > > [1] https://bitcointalk.org/index.php?topic=5261605.0 > > > > * * * * * Full text of the above link * * * * * > > > BIP: 322 > Layer: Applications > Title: Generic Signed Message Format > Author: Karl-Johan Alm > Comments-Summary: No comments yet. > Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0322 > Status: Draft > Type: Standards Track > Created: 2018-09-10 > License: CC0-1.0 > > > == Abstract == > > A standard for interoperable signed messages based on the Bitcoin Script > format, either for proving fund availability, or committing to a message as > the intended recipient of funds sent to the invoice address. > > == Motivation == > > The current message signing standard only works for P2PKH (1...) invoice > addresses. We propose to extend and generalize the standard by using a > Bitcoin Script based approach. This ensures that any coins, no matter what > script they are controlled by, can in-principle be signed for. For easy > interoperability with existing signing hardware, we also define a signature > message format which resembles a Bitcoin transaction (except that it contains > an invalid input, so it cannot be spent on any real network). > > Additionally, the current message signature format uses ECDSA signatures > which do not commit to the public key, meaning that they do not actually > prove knowledge of any secret keys. (Indeed, valid signatures can be tweaked > by 3rd parties to become valid signatures on certain related keys.) > > Ultimately no message signing protocol can actually prove control of funds, > both because a signature is obsolete as soon as it is created, and because > the possessor of a secret key may be willing to sign messages on others' > behalf even if it would not sign actual transactions. No signmessage protocol > can fix these limitations. > > == Types of Signatures == > > This BIP specifies three formats for signing messages: ''legacy'', ''simple'' > and ''full''. Additionally, a variant of the ''full'' format can be used to > demonstrate control over a set of UTXOs. > > === Legacy === > > New proofs should use the new format for all invoice address formats, > including P2PKH. > > The legacy format MAY be used, but must be restricted to the legacy P2PKH > invoice address format. > > === Simple === > > A ''simple'' signature consists of a witness stack, consensus encoded as a > vector of vectors of
[bitcoin-dev] BIP-0322 (generic signmessage) improvements
I have gone over BIP-0322 and substantially rewritten the text. Everything I did is (I think) simply clarifying the existing protocol, which felt like it was written by committee and wasn't easy to follow, EXCEPT: 1. I rewrote the motivation section, which I believe originally was a paraphrase of Luke-jr's general objections to having any signmessage functionality. I hope Luke in particular can take a look at what I wrote under "Motivation" and see if it captures his concerns. 2. I merged the "consensus" and "upgradeable" rules to simply be one set of rules, consisting of consensus checks plus additional restrictions, all of which must be included. The new "Extensions" section allows validators to output the state "consensus-valid" if they really don't want to check the additional restrictions. 3. The "inconclusive" state, which was originally used for what I've called "consensus-valid", now indicates that a validator does not understand the script that it is checking (also described in the new "Extensions" section). The goal is that implementors are able to be meaningfully BIP-0322 while only supporting a subset of Script, e.g. the templates that their own software supports, or Miniscript, or the non-raw non-address set of output descriptors, or whatever. We have seen opposition to supporting BIP-322, e.g. [1] because of the requirement that you either have a full script interpreter (plus an open-ended list of Core's standardness flags, which is not even available through libbitcoinconsensus) or nothing. On the other hand, the vast majority of outputs are single-key p2pkh, p2pkwh or p2sh-wpkh. The new text is here (and for posterity I will also include it inline below, though unless Github deletes it it will be easier to read in rendered form): https://github.com/apoelstra/bips/blob/2020-12--bip322-overhaul/bip-0322.mediawiki I'll also PR this to the BIPs repo in the next day or two, and comments on Github are then welcome. [1] https://bitcointalk.org/index.php?topic=5261605.0 * * * * * Full text of the above link * * * * * BIP: 322 Layer: Applications Title: Generic Signed Message Format Author: Karl-Johan Alm Comments-Summary: No comments yet. Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0322 Status: Draft Type: Standards Track Created: 2018-09-10 License: CC0-1.0 == Abstract == A standard for interoperable signed messages based on the Bitcoin Script format, either for proving fund availability, or committing to a message as the intended recipient of funds sent to the invoice address. == Motivation == The current message signing standard only works for P2PKH (1...) invoice addresses. We propose to extend and generalize the standard by using a Bitcoin Script based approach. This ensures that any coins, no matter what script they are controlled by, can in-principle be signed for. For easy interoperability with existing signing hardware, we also define a signature message format which resembles a Bitcoin transaction (except that it contains an invalid input, so it cannot be spent on any real network). Additionally, the current message signature format uses ECDSA signatures which do not commit to the public key, meaning that they do not actually prove knowledge of any secret keys. (Indeed, valid signatures can be tweaked by 3rd parties to become valid signatures on certain related keys.) Ultimately no message signing protocol can actually prove control of funds, both because a signature is obsolete as soon as it is created, and because the possessor of a secret key may be willing to sign messages on others' behalf even if it would not sign actual transactions. No signmessage protocol can fix these limitations. == Types of Signatures == This BIP specifies three formats for signing messages: ''legacy'', ''simple'' and ''full''. Additionally, a variant of the ''full'' format can be used to demonstrate control over a set of UTXOs. === Legacy === New proofs should use the new format for all invoice address formats, including P2PKH. The legacy format MAY be used, but must be restricted to the legacy P2PKH invoice address format. === Simple === A ''simple'' signature consists of a witness stack, consensus encoded as a vector of vectors of bytes, and base64-encoded. Validators should construct to_spend and to_sign as defined below, with default values for all fields except that * message_hash is a BIP340-tagged hash of the message, as specified below * message_challenge in to_spend is set to the scriptPubKey being signed with * message_signature in to_sign is set to the provided simple signature. and then proceed as they would for a full signature. === Full === Full signatures follow an analogous specification to the BIP-325 challenges and solutions used by Signet. Let there be two virtual transactions to_spend and to_sign. The "to_spend" transaction is:
