Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
Jean-Pierre Rupp from Haskoin here. I support a hard fork to fix consensus bugs. The Bitcoin protocol should eventually get to a state where it is documented in a clear and understandable fashion. Bugs are bugs, and are the enemy. We should not attempt to live with them. We should be opening a process of thoroughly documenting and reparing consensus bugs on a separate branch, and eventually schedule a hard fork. There are two good things that will come out of that: 1. Known bugs will be gone, and 2. We will have a process in place to get rid of future bugs in eventual future hard forks. We do not need to become paranoid about the ramifications of a hard fork, or how it will open the door for unwanted changes in the protocol. We are discussing about removing bugs, and bugs that could be used to exploit the network in ways that may not be immediately obvious. There are 144 blocks generated per day by groups of miners that are mostly identified. It is not going to be a titanic task to get consensus from the main mining pools on fixing this at the mining level. We must address how the fixes for some of these bugs affect other types of software such as wallets. I can think that fixing the bug where OP_CHECKMULTISIG pops an extra value from the stash could be more traumatic, since it requires anything that creates and validates multi-signature transactions to change the way it works. Hardware wallets could be impacted. But most of the consensus bugs would not affect the way the vast majority of bitcoin transactions that are currently created. Therefore it should not be traumatic at all for users, but only really affect mining pools, who would only need to be convinced to upgrade their bitcoind well in advance, which seems to me that it is not an issue at all. We should not compare doing a Bitcoin hard-fork with doing something like deploying IPv6 world-wide or enforcing TLS and SPF on every SMTP connection. We should not conflate Bitcoin with other network protocols. The Bitcoin protocol is actually relatively easy to upgrade at this point. Let's take advantage of this fact. On 06/11/14 15:36, Justus Ranvier wrote: > Because Bitcoin has a extra consensus requirements, requirements which > are really rare in engineering, the necessity of fixing bugs is even > greater. -- Be Happy :) 0x310A8A5B.asc Description: application/pgp-keys signature.asc Description: OpenPGP digital signature -- Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for $9/Month. Get alerted through email, SMS, voice calls or mobile push notifications. Take corrective actions from your mobile device. http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
> > > Who benefits from not fixing bugs in Bitcoin? > > We can bring up politics if you want. No, please don't. That question was rhetorical, not an invitation for you to try and convince bystanders that anyone who disagrees with you is a shadowy Agent Of Centralisation or an idiot. You use that tactic way too much: it's obnoxious and you need to stop it. Hard forks vs soft forks are *purely* about whether you drag along old nodes in a quasi-broken state. They do not reduce total work needed by the community one iota. Non-miners who wish to reject a soft fork can easily run a node that does so, if they wanted to - the voting mechanism still boils down to "which side of the fork do I accept in my economic activity". It's certainly garbage to claim that the reason to want to avoid soft forks is being an Evil Centralised Foundation: this is about a set of engineering tradeoffs only. -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
On Fri, Nov 7, 2014 at 12:30 PM, Clément Elbaz wrote: > Thinking out loud here : would it make sense to separate the consensus code > into some kind of "Bitcoin Kernel" (similar to the Linux Kernel) project > that could be used by anyone ? Yes, we're moving in that direction. First with a script verification library in 0.10, which will be extended to other parts of the consensus by 0.11 and after that. Wladimir -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
On Fri, Nov 07, 2014 at 11:30:22AM +, Clément Elbaz wrote:
> Thinking out loud here : would it make sense to separate the consensus code
> into some kind of "Bitcoin Kernel" (similar to the Linux Kernel) project
> that could be used by anyone ?
That's a pretty old idea, and we're working on it. First step is a
stand-alone script verification library:
https://github.com/theuni/bitcoin/blob/da18a0266c4de76c2a461cc2984aa2fa066c42f5/src/script/bitcoinconsensus.h
#ifndef H_BITCOIN_BITCOINCONSENSUS
#define H_BITCOIN_BITCOINCONSENSUS
#if defined(BUILD_BITCOIN_INTERNAL) && defined(HAVE_CONFIG_H)
#include "config/bitcoin-config.h"
#if defined(_WIN32)
#if defined(DLL_EXPORT)
#if defined(HAVE_FUNC_ATTRIBUTE_DLLEXPORT)
#define EXPORT_SYMBOL __declspec(dllexport)
#else
#define EXPORT_SYMBOL
#endif
#endif
#elif defined(HAVE_FUNC_ATTRIBUTE_VISIBILITY)
#define EXPORT_SYMBOL __attribute__ ((visibility ("default")))
#endif
#elif defined(MSC_VER) && !defined(STATIC_LIBBITCOINCONSENSUS)
#define EXPORT_SYMBOL __declspec(dllimport)
#endif
#ifndef EXPORT_SYMBOL
#define EXPORT_SYMBOL
#endif
#ifdef __cplusplus
extern "C" {
#else
#include
#endif
/** Script verification flags */
enum
{
bitcoinconsensus_SCRIPT_FLAGS_VERIFY_NONE = 0,
bitcoinconsensus_SCRIPT_FLAGS_VERIFY_P2SH = (1U << 0), // evaluate
P2SH (BIP16) subscripts
};
/// Returns true if the input nIn of the serialized transaction pointed to by
/// txTo correctly spends the scriptPubKey pointed to by scriptPubKey under
/// the additional constraints specified by flags.
EXPORT_SYMBOL bool bitcoinconsensus_verify_script(const unsigned char
*scriptPubKey, const unsigned int scriptPubKeyLen,
const unsigned char *txTo, const
unsigned int txToLen,
const unsigned int nIn, const unsigned int
flags);
EXPORT_SYMBOL unsigned int bitcoinconsensus_version();
#ifdef __cplusplus
} // extern "C"
#endif
#undef EXPORT_SYMBOL
#endif // H_BITCOIN_BITCOINCONSENSUS
--
'peter'[:-1]@petertodd.org
01208038fd7130083ff118147890dbb37913ffa83c1f48cd
signature.asc
Description: Digital signature
--
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
Thinking out loud here : would it make sense to separate the consensus code into some kind of "Bitcoin Kernel" (similar to the Linux Kernel) project that could be used by anyone ? Bitcoin Core (and any other application wishing to do so) could be based on it. The kernel would just contain the absolute minimum code for reaching consensus, leaving every other aspects of the implementation to the applications built with it. It would be stateless : it would provide an interface to submit a block/transaction to be validated, including the context needed to validate it (the previously validated blocks referenced by this block/transaction). What do you think ? Clément Le Fri Nov 07 2014 at 9:49:05 AM, Peter Todd a écrit : On Fri, Nov 07, 2014 at 09:07:47AM +0100, Tamas Blummer wrote: > > Peter, > > > > forking would work best with a freeze of the consensus code. Do you see > any chance for that? > > To a first approximation the consensus code *is* frozen; if we introduce > any consensus changes into it at this point it's due to a mistake, not > intentionally. > > Of course, that's not including the two serious soft-fork proposals in > the air right now, Pieter Wuille's BIP62 and my CHECKLOCKTIMEVERIFY. > However dealing with proposed changes like those in an environment where > the competing implementations all use essentially the same > consensus-critical code is much easier than in an environment where they > don't; I say this on both a technical and political level. > > -- > 'peter'[:-1]@petertodd.org > 0c901eb1b6b765519b99c3afd7a9062ff4cfa29666ce140d > > -- > ___ > Bitcoin-development mailing list > Bitcoin-development@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
On Fri, Nov 07, 2014 at 09:07:47AM +0100, Tamas Blummer wrote: > Peter, > > forking would work best with a freeze of the consensus code. Do you see any > chance for that? To a first approximation the consensus code *is* frozen; if we introduce any consensus changes into it at this point it's due to a mistake, not intentionally. Of course, that's not including the two serious soft-fork proposals in the air right now, Pieter Wuille's BIP62 and my CHECKLOCKTIMEVERIFY. However dealing with proposed changes like those in an environment where the competing implementations all use essentially the same consensus-critical code is much easier than in an environment where they don't; I say this on both a technical and political level. -- 'peter'[:-1]@petertodd.org 0c901eb1b6b765519b99c3afd7a9062ff4cfa29666ce140d signature.asc Description: Digital signature -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
Peter, forking would work best with a freeze of the consensus code. Do you see any chance for that? Tamas Blummer On Nov 7, 2014, at 1:03 AM, Peter Todd wrote: > Forking the codebase, rather than rewriting it, best > ensures that your code actually implements the protocol properly, is > safe to use for mining, and actually gets used. > > Rewriting Bitcoin Core is a fun project, but it's terrible politics. signature.asc Description: Message signed with OpenPGP using GPGMail -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
On Thu, Nov 6, 2014 at 11:12 PM, Peter Todd wrote: > BIP62 does make life easier for wallet authors as they don't have to > deal with malleability - maybe! - Yes, I agree for most contract purposes CTLV is what you want to be using, instead of refund transactions beyond being more clearly correct, it shrinks the protocol state machine by one step. Though BIP62 also achieves the secondary goal of making required implementation behaviour more explicit (e.g. the parts enforced in all transactions), and that shouldn't be discounted. They're somewhat orthogonal, somwhat complementary things. -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
On Thu, Nov 06, 2014 at 05:36:55PM -0600, Justus Ranvier wrote: > This explanation is completely incoherent. > > Because Bitcoin has a extra consensus requirements, requirements which > are really rare in engineering, the necessity of fixing bugs is even > greater. > > There are two general ways to fix bugs: either as part of a > controlled, planned, and managed process, or as a response to an > immediate disaster. > > The alternative to scheduling and planning the upgrades which are > necessary to fix the bugs in the protocol, where such fixes can be > written, tested, and documented at leisure, is to wait for some crisis > and slap on another bandaid when the network breaks again (like it did > March of last year). The protocol is what the protocol is; the bugs are when you don't match the protocol. > Who benefits from not fixing bugs in Bitcoin? We can bring up politics if you want. In the current model, the specification *is* the protocol, and the Bitcoin Core team is scared to death of changing anything; they've got very little real power. Soft-forks are the minimum-viable way of making changes to the protocol, and it's very clear how they get adopted: minerr consensus. They're also a fundemental way of changing the protocol that is impossible to prevent, so you might as well use it. Hard-forks require political consensus to achieve, and the way you create that political consensus is by creating committes, groups, associations... Foundations. Every last one of those things requires centralization and political power. You know, the smartest thing the Bitcoin Foundation could do if they wanted to cement their place in the Bitcoin ecosystem as a power broker would be to setup a program of periodic hard-forks, say every year or two, and then manage the committees that decide what goes into those hard-forks. That they haven't suggested that yet is a sign that they're either not evil, or they don't understand Bitcoin very well. I think programmers find this reality hard to accept, because they're mostly interested in writing code that'll get widely used. To them it's hard to accept that the Bitcoin protocol *is* a few thousand lines of C++ code, and they're not good enough to write their own implementation and make it match; if we replaced programmers with writers we might get the equally bizzare and pointless situation of people taking perfectly good RFCs and rewriting them in their own words. If you do care about keeping the politics of Bitcoin development free from centralized control you should do what I advised the Dark Wallet team to do a year ago: fork Bitcoin Core and change the non-consensus-critical code that implements policy. I've done this myself in a minor way with my replace-by-fee(1) version. Luke-Jr has also done this with his Eligius branch, a fork that something like 30% of the Bitcoin hashing power appear to run. (Discus Fish has been mining non-standard transactions(2) lately) Multiple *forks* of the Bitcoin Core reference client that are actually getting used by miners and other users ensures that no one group maintaining such a fork has the ability to change anything without strong consensus. Forking the codebase, rather than rewriting it, best ensures that your code actually implements the protocol properly, is safe to use for mining, and actually gets used. Rewriting Bitcoin Core is a fun project, but it's terrible politics. 1) https://github.com/petertodd/bitcoin/tree/replace-by-fee-v0.9.3 2) https://blockchain.info/tx/e24a4085c54a6362e615f8eab758c12d80e488b73757e6d2b8ab6bfc8be7007e -- 'peter'[:-1]@petertodd.org 08f2290924a6882928d4566f487f33cc57203a6535795201 signature.asc Description: Digital signature -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11/06/2014 05:26 PM, Peter Todd wrote:> For the same reason we don't do hard-forking upgrades of basically every > protocol on the planet on a regular basis, even when we don't have > consensus problems to worry about. > > Flag days are really rare in engineering, and for good reason. This explanation is completely incoherent. Because Bitcoin has a extra consensus requirements, requirements which are really rare in engineering, the necessity of fixing bugs is even greater. There are two general ways to fix bugs: either as part of a controlled, planned, and managed process, or as a response to an immediate disaster. The alternative to scheduling and planning the upgrades which are necessary to fix the bugs in the protocol, where such fixes can be written, tested, and documented at leisure, is to wait for some crisis and slap on another bandaid when the network breaks again (like it did March of last year). Who benefits from not fixing bugs in Bitcoin? - -- Support online privacy by using email encryption whenever possible. Learn how here: http://www.youtube.com/watch?v=bakOKJFtB-k -BEGIN PGP SIGNATURE- iQEcBAEBAgAGBQJUXAYVAAoJEMP3uyY4RQ21YxMH/3O+vFK2jDXV5V8IIsJnU/u1 D4gYyVm89E0zmGTyLAUYCJGj0eg5tMyUUzu62hOECOeQuKdVi+mbkLi4TiF0sHIb 8k25MgqJgzH/021eoI2g2w1FrDlZut02LNX/V09+owd1yhp+SEXZ3/+HlqsZXhsv /u9u5OayzhGlzS6apQtrosl5P+KIquHqIbtwBtPOb2rvlL0miJ6sRcAH2JCXCBDT HMcswMtIGZbgqL/K7e/6vH7dUWdp0866RZXvt7aWGNUgvxHbGMs+zsnxp4nNslxM wqL71gTmtMnLcw0GtqmXPjcjo+adrPnqp45xc9lSt23PGjxxfR0FKYIPb2uZdq8= =9GOY -END PGP SIGNATURE- 0x38450DB5.asc Description: application/pgp-keys -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
On Thu, Nov 06, 2014 at 04:48:54PM -0600, Justus Ranvier wrote: > Why not schedule protocol upgrades every two years for the foreseeable > future? For the same reason we don't do hard-forking upgrades of basically every protocol on the planet on a regular basis, even when we don't have consensus problems to worry about. Flag days are really rare in engineering, and for good reason. -- 'peter'[:-1]@petertodd.org 08f2290924a6882928d4566f487f33cc57203a6535795201 signature.asc Description: Digital signature -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
On Thu, Nov 06, 2014 at 11:11:52PM +0100, Jeff Garzik wrote: > IMO, CHECKLOCKTIMEVERIFY should be included in that list, too. > > RE soft fork vs. hard fork: It's about this time at Mike Hearn will > chime in, on the side of hard forks. Hard forks are in a sense much > cleaner, and permit solving problems not otherwise solvable with a > hard fork. However, hard forks clearly have risks, notably the Big > Risk akin to a US Constitutional Convention: once you open the door, > anything can happen, any rule no matter how "sacred" can be changed. I think people in this community often miss the serious political and legal ramifications of hard-forks. Being in the social position of being able to succesfully pull off hard-forks, particularly for new features, is clear evidence that you have de-facto control over the system. Regulators around the world appear to be going in directions that would make that control subject to regulation and licensing, e.g. the European Banking Association proposals, and initial Bitlicense proposals. Equally, look how hard-forks - known as flag days elsewhere - are generally considered to be dangerous and worth avoiding in other contexts due to simple engineering reasons. It's just easier to upgrade systems in backward compatible ways, especially when they incorporate features specifically to make that possible. (as does bitcoin!) > Soft forks are not without their own risks, e.g. reducing some things > to SPV levels of security. This is a misconception; you can't prevent soft-forks from happening, so you always have an SPV level of security by that standard. People put *way* too much trust in small numbers of confirmations... -- 'peter'[:-1]@petertodd.org 0094d543907eaf0f94f4ff5f4c760b3552d84ff811cd9053 signature.asc Description: Digital signature -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
On Thu, Nov 06, 2014 at 10:05:54PM +, Matt Corallo wrote: > Depends, without BIP62 a /lot/ of the even basic contracts that people > want to use today (or wanted to use 18 months ago) are unusable, in > fact, without BIP62, the atomic swaps suggested as important for > sidechains are not secure. While redoing Bitcoin in a hardfork is nice, > its a very long-term thing, so I'm not sure about making people wait for > a large hardfork just to use payment channels. BIP62 is a less-than-ideal way of making contracts secure against malleability as it relies on a "whack-a-mole" approach to security that is insecure if any flaw is missed. If you only wanted to make contracts secure, you'd either implement a new SignatureHash() that could leave out the prevout field in favor of hashing the previous input's CTxOut() structure, and/or implement the significantly more limited CHECKLOCKTIMEVERIFY. Equally BIP62 fails at making more complex types of contracts secure. For instance suppose I had a multi-step protocol that required more than two transactions: tx1: Alice -> (Alice, Bob) tx1_refund: (Alice, Bob) -> Alice tx2: (Alice, Bob) -> Charlie tx2_refund: (Alice, Bob) -> Bob tx1 can only be modified by Alice, so tx1_refund is secure. However the second stage, where the output of tx1 is spent by tx2, with a refund transaction giving the funds back to Bob, can't be made secure as BIP62 can't prevent Alice from changing her signature, getting tx2' mined instead, and making tx2_refund invalid. OTOH a new form of signature hash that was a signature on tx2.vout structure rather than it's txid would be secure, as tx2_refund would be valid regardless of tx2's actual txid. Obviously there are good reasons to not use such signature hashes in the general case, as they imply you can't reuse scriptPubKeys securely, but that's a minor problem for purpose-built contract protocols. It's certainly a much more minor problem then the huge number of holes possible with BIP62. BIP62 does make life easier for wallet authors as they don't have to deal with malleability - maybe! - but for contracts it's a bad design. > Also, I echo the difficulty of writing consensus-compatible code and > highly suggest anyone with money behind an implementation that is doing > script verification in code that isnt Bitcoin Core rethink that decision. FWIW I've done due-dilligence reviews for investors on projects and companies that have re-implemented Bitcoin Core consensus-critical code, and every time my review lists doing so as a major red flag. -- 'peter'[:-1]@petertodd.org 166801ed3959dde6b7d979735c290e7c4271ae3cf75ced63 signature.asc Description: Digital signature -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
On 11/06/2014 04:11 PM, Jeff Garzik wrote: > RE soft fork vs. hard fork: It's about this time at Mike Hearn will > chime in, on the side of hard forks. Hard forks are in a sense much > cleaner, and permit solving problems not otherwise solvable with a > hard fork. However, hard forks clearly have risks, notably the Big > Risk akin to a US Constitutional Convention: once you open the door, > anything can happen, any rule no matter how "sacred" can be changed. Yes, there are risks, but those risks could be managed with appropriate effort. Major players could publicly commit to a set of ground rules vis a vis what categories of changes are and are not acceptable. Maybe at some point there could even be something that resembles project management for the Bitcoin protocol. Why not schedule protocol upgrades every two years for the foreseeable future? Spend one year achieving broad consensus regarding what changes to make in the next upgrade, then spend one year in feature freeze (all future proposals postponed for the next cycle) then execute the upgrade. The top priority should be fixing bugs that make specifying and re-implementing the protocol nearly impossible. Those kinds of changes should have little difficulty achieving near-unanimous consensus. There shouldn't be any problems separating obviously-needed changes from the ones that let third parties blacklist coins, or a majority of miners vote to confiscate block rewards from minority, tamper with the issuance schedule, etc. -- Support online privacy by using email encryption whenever possible. Learn how here: http://www.youtube.com/watch?v=bakOKJFtB-k 0x38450DB5.asc Description: application/pgp-keys signature.asc Description: OpenPGP digital signature -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
IMO, CHECKLOCKTIMEVERIFY should be included in that list, too. RE soft fork vs. hard fork: It's about this time at Mike Hearn will chime in, on the side of hard forks. Hard forks are in a sense much cleaner, and permit solving problems not otherwise solvable with a hard fork. However, hard forks clearly have risks, notably the Big Risk akin to a US Constitutional Convention: once you open the door, anything can happen, any rule no matter how "sacred" can be changed. Soft forks are not without their own risks, e.g. reducing some things to SPV levels of security. Leaning towards soft fork, but it is a good discussion to have. A poorly implemented soft fork may potentially require a hard fork to fix rollout bugs. On Thu, Nov 6, 2014 at 11:05 PM, Matt Corallo wrote: > Depends, without BIP62 a /lot/ of the even basic contracts that people > want to use today (or wanted to use 18 months ago) are unusable, in > fact, without BIP62, the atomic swaps suggested as important for > sidechains are not secure. While redoing Bitcoin in a hardfork is nice, > its a very long-term thing, so I'm not sure about making people wait for > a large hardfork just to use payment channels. > > Also, I echo the difficulty of writing consensus-compatible code and > highly suggest anyone with money behind an implementation that is doing > script verification in code that isnt Bitcoin Core rethink that decision. > > Matt > > On 11/06/14 21:58, Tamas Blummer wrote: >> Thanks Peter, >> >> Having tried to write a bug-for-bug compatible code with Satoshi, I can only >> second that it is rather close to impossible. >> >> The aim of BIP62 is noble, still it does not feel right for me to increase >> the complexity of the code with e.g. soft-fork-ready versioning. >> Freezing the consensus code, studying its bugs appears more appropriate to >> me. What we learn could define a hard fork or a better >> chain we migrate to as discussed by blockstream. >> >> Tamas Blummer > > -- > ___ > Bitcoin-development mailing list > Bitcoin-development@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Jeff Garzik Bitcoin core developer and open source evangelist BitPay, Inc. https://bitpay.com/ -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
Depends, without BIP62 a /lot/ of the even basic contracts that people want to use today (or wanted to use 18 months ago) are unusable, in fact, without BIP62, the atomic swaps suggested as important for sidechains are not secure. While redoing Bitcoin in a hardfork is nice, its a very long-term thing, so I'm not sure about making people wait for a large hardfork just to use payment channels. Also, I echo the difficulty of writing consensus-compatible code and highly suggest anyone with money behind an implementation that is doing script verification in code that isnt Bitcoin Core rethink that decision. Matt On 11/06/14 21:58, Tamas Blummer wrote: > Thanks Peter, > > Having tried to write a bug-for-bug compatible code with Satoshi, I can only > second that it is rather close to impossible. > > The aim of BIP62 is noble, still it does not feel right for me to increase > the complexity of the code with e.g. soft-fork-ready versioning. > Freezing the consensus code, studying its bugs appears more appropriate to > me. What we learn could define a hard fork or a better > chain we migrate to as discussed by blockstream. > > Tamas Blummer -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
Thanks Peter, Having tried to write a bug-for-bug compatible code with Satoshi, I can only second that it is rather close to impossible. The aim of BIP62 is noble, still it does not feel right for me to increase the complexity of the code with e.g. soft-fork-ready versioning. Freezing the consensus code, studying its bugs appears more appropriate to me. What we learn could define a hard fork or a better chain we migrate to as discussed by blockstream. Tamas Blummer signature.asc Description: Message signed with OpenPGP using GPGMail -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
Recently wrote the following for a friend and thought others might learn
from it.
> Nope, never heard that term. By "bug-for-bug" compatibility, do you mean
> that, for each version which has a bug, each bug must behave in the *same*
> buggy way?
Exactly. tl;dr: if you accept a block as valid due to a bug that others reject,
you're forked and the world ends.
Long answer... well you reminded me I've never actually written up a good
example for others, and a few people have asked me for one. A great example of
this is the SIGHASH_SINGLE bug in the SignatureHash() function:
uint256 SignatureHash(CScript scriptCode, const CTransaction& txTo,
unsigned int nIn, int nHashType)
{
else if ((nHashType & 0x1f) == SIGHASH_SINGLE)
{
// Only lock-in the txout payee at same index as txin
unsigned int nOut = nIn;
if (nOut >= txTmp.vout.size())
{
printf("ERROR: SignatureHash() : nOut=%d out of range\n", nOut);
return 1;
}
}
// Serialize and hash
CHashWriter ss(SER_GETHASH, 0);
ss << txTmp << nHashType;
return ss.GetHash();
}
So that error condition results in SignatureHash() returning 1 rather than the
actual hash. But the consensus-critical code that implements the CHECKSIG
operators doesn't check for that condition! Thus as long as you use the
SIGHASH_SINGLE hashtype and the txin index is >= the number of txouts any valid
signature for the hash of the number 1 is considered valid!
When I found this bug¹ I used it to fork bitcoin-ruby, among others.
(I'm not the first; I found it independently after Matt Corallo) Those
alt-implementations handled this edge-case as an exception, which in
turn caused the script to fail. Thus they'd reject blocks containing
transactions using such scripts, and be forked off the network.
You can also use this bug for something even more subtle. So the
CHECKSIG* opcode evaluation does this:
// Drop the signature, since there's no way for a signature to sign itself
scriptCode.FindAndDelete(CScript(vchSig));
and CHECKMULTISIG* opcode:
// Drop the signatures, since there's no way for a signature to sign itself
for (int k = 0; k < nSigsCount; k++)
{
valtype& vchSig = stacktop(-isig-k);
scriptCode.FindAndDelete(CScript(vchSig));
}
We used to think that code could never be triggered by a scriptPubKey or
redeemScript, basically because there was no way to get a signature into a
transaction in the right place without the signature depending on the txid of
the transaction it was to be included in. (long story) But SIGHASH_SINGLE makes
that a non-issue, as you can now calculate the signature that signs '1' ahead
of time! In a CHECKMULTISIG that signature is valid, so is included in the list
of signatures being dropped, and thus the other signatures must take that
removal into account or they're invalid. Again, you've got a fork.
However this isn't the end of it! So the way FindAndDelete() works is as
follows:
int CScript::FindAndDelete(const CScript& b)
{
int nFound = 0;
if (b.empty())
return nFound;
iterator pc = begin();
opcodetype opcode;
do
{
while (end() - pc >= (long)b.size() && memcmp(&pc[0], &b[0],
b.size()) == 0)
{
pc = erase(pc, pc + b.size());
++nFound;
}
}
while (GetOp(pc, opcode));
return nFound;
}
So that's pretty ugly, but basically what's happening is the loop iterates
though all the opcodes in the script. Every opcode is compared at the *byte*
level to the bytes in the argument. If they match those bytes are removed from
the script and iteration continues. The resulting script, with chunks sliced
out of it at the byte level, is what gets hashed as part of the signature
checking algorithm.
As FindAndDelete() is always called with CScript(vchSig) the signature
being found and deleted is reserialized. Serialization of bytes isn't
unique; there are multiple valid encodings for PUSHDATA operations. The
way CScript() is called the most compact encoding is used, however this
means that if the script being hashed used a different encoding those
bytes are *not* removed and thus the signature is different.
Again, if you don't get every last one of those details exactly right, you'll
get forked.
...and I'm still not done! So when you call CScript(vchSig) the relevant code
is the following:
class CScript : public std::vector
{
explicit CScript(const CScriptNum& b) { operator<<(b); }
CScript& operator<<(const std::vector& b)
{
if (b.size() < OP_PUSHDATA1)
{
insert(end(), (unsigned char)b.size());
}
else if (b.size() <= 0xff)
{
insert(end(), OP_PUSHDATA1);
insert(end(),
