[Bitcoin-development] Malleability and MtGox's announcement
Hi all, I was a bit surprised to see MtGox's announcement. The malleability of transactions was known for years already (see for example the wiki article on it, https://en.bitcoin.it/wiki/Transaction_Malleability it, or mails on this list from 2012 and 2013). I don't consider it a very big problem, but it does make it harder for infrastructure to interact with Bitcoin. If we'd design Bitcoin today, I'm sure we would try to avoid it altogether to make life easier for everyone. But we can't just change all infrastructure that exists today. We're slowly working towards making malleability harder (and hopefully impossible someday), but this will take a long time. For example, 0.8 not supporting non-DER encoded signatures was a step in that direction (and ironically, the trigger that caused MtGox's initial problems here). In any case, this will take years, and nobody should wait for this. There seem to be two more direct problems here. * Wallets which deal badly with modified txids. * Services that use the transaction id to detect unconfirming transactions. The first is something that needs to be done correctly in software - it just needs to be aware of malleability. The second is something I was unaware of and would have advised against. If you plan on reissuing a transaction because on old version doesn't confirm, make sure to make it a double spend of the first one - so that not both can confirm. I certainly don't like press making this sound like a problem in the Bitcoin protocol or clients. I think this is an issue that needs to be solved at the layer above - the infrastructure building on the Bitcoin system. Despite that, I do think that we (as a community, not just developers) can benefit from defining a standard way to identify transactions unambiguously. This is something Mark Karpeles suggested a few days ago, and my proposal is this: We define the normalized transaction id as SHA256^2(normalized_tx + 0x0100), where normalized_tx is the transaction with all input scripts replaced by empty scripts. This is exactly what would be signed inside transaction signatures using SIGHASH_ALL (except not substituting the previous scriptPubKey to be signed, and not dealing with the input being signed specially). An implementation is here: https://github.com/sipa/bitcoin/commits/normtxid. Note that this is not a solution for all problems related to malleability, but maybe it can make people more aware of it, in tangible way. -- Pieter -- Managing the Performance of Cloud-Based Applications Take advantage of what the Cloud has to offer - Avoid Common Pitfalls. Read the Whitepaper. http://pubads.g.doubleclick.net/gampad/clk?id=121051231iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Malleability and MtGox's announcement
Okay, why the everloving FUCK is there not someone on this list with a @mtgox.com address talking about this? I started using bitcoin because I could audit the code, and when the developer cabal does stuff 'off-list' what you do is hand over market manipulation power to the selected cabal of company insiders who are discussing things 'off-list'. The people having a 'private' discussion about how to solve this are TAKING MONEY from everyone else, by having access to insider information. I don't think any of the developers actually have a clue this is the result, because a good chunk of them are employed by for-profit companies funded by venture capital, and VC lawyers are very good at writing employment contracts that provide plausible deniability of insider trading. The press MAKES MONEY (okay, takes money) by manipulating markets, and venture capitalists pay lots of money to ensure the market is manipulated in ways they can profit from. Private market manipulation is one of the costs of anonymity and privacy, and I don't really like paying for some off-list discussion of what appears to be a serious scalability and usability problem. Bitcoin is such a powerful tool because it broadcasts transactions to the network for everyone to see. Can we please broadcast some more technical details to this mailing list, including exactly what MtGox is doing, and how they wish to resolve it? If you gave me the entire code stack that MtGox runs on under an AGPLv3 license, I'm pretty sure I, along with everyone else here could come up with a workable solution. I think a code release would be a huge win for MtGox as well, and would cement their position as market leader in transparent cryptocurrency trading. Otherwise we are just a bunch of dinghys getting capsized one by one in a sea of market-manipulating white whales. Isn't the closed door market manipulation of the big banks one of the reasons we all started using Bitcoin in the first place? Why do revolutions always put the same old bullshit back in power? What we need is some transparent code evolution. On Mon, Feb 10, 2014 at 01:28:42PM +0100, Pieter Wuille wrote: Hi all, I was a bit surprised to see MtGox's announcement. The malleability of transactions was known for years already (see for example the wiki article on it, https://en.bitcoin.it/wiki/Transaction_Malleability it, or mails on this list from 2012 and 2013). I don't consider it a very big problem, but it does make it harder for infrastructure to interact with Bitcoin. If we'd design Bitcoin today, I'm sure we would try to avoid it altogether to make life easier for everyone. But we can't just change all infrastructure that exists today. We're slowly working towards making malleability harder (and hopefully impossible someday), but this will take a long time. For example, 0.8 not supporting non-DER encoded signatures was a step in that direction (and ironically, the trigger that caused MtGox's initial problems here). In any case, this will take years, and nobody should wait for this. There seem to be two more direct problems here. * Wallets which deal badly with modified txids. * Services that use the transaction id to detect unconfirming transactions. The first is something that needs to be done correctly in software - it just needs to be aware of malleability. The second is something I was unaware of and would have advised against. If you plan on reissuing a transaction because on old version doesn't confirm, make sure to make it a double spend of the first one - so that not both can confirm. I certainly don't like press making this sound like a problem in the Bitcoin protocol or clients. I think this is an issue that needs to be solved at the layer above - the infrastructure building on the Bitcoin system. Despite that, I do think that we (as a community, not just developers) can benefit from defining a standard way to identify transactions unambiguously. This is something Mark Karpeles suggested a few days ago, and my proposal is this: We define the normalized transaction id as SHA256^2(normalized_tx + 0x0100), where normalized_tx is the transaction with all input scripts replaced by empty scripts. This is exactly what would be signed inside transaction signatures using SIGHASH_ALL (except not substituting the previous scriptPubKey to be signed, and not dealing with the input being signed specially). An implementation is here: https://github.com/sipa/bitcoin/commits/normtxid. Note that this is not a solution for all problems related to malleability, but maybe it can make people more aware of it, in tangible way. -- Pieter -- Managing the Performance of Cloud-Based Applications Take advantage of what the Cloud has to offer - Avoid Common Pitfalls. Read the Whitepaper.
Re: [Bitcoin-development] Malleability and MtGox's announcement
You must be new here. MtGox very rarely comments on things like this publicly, outside of irc or their website. Second, MtGox problem is a MtGox problem. You have no right to demand access to their private code. If you feel wronged as a customer, sue them. Otherwise, they have no obligation to you. I believe you are barking up the wrong tree. Respectfully, Nick On February 10, 2014 10:14:02 AM CST, Troy Benjegerdes ho...@hozed.org wrote: Okay, why the everloving FUCK is there not someone on this list with a @mtgox.com address talking about this? I started using bitcoin because I could audit the code, and when the developer cabal does stuff 'off-list' what you do is hand over market manipulation power to the selected cabal of company insiders who are discussing things 'off-list'. The people having a 'private' discussion about how to solve this are TAKING MONEY from everyone else, by having access to insider information. I don't think any of the developers actually have a clue this is the result, because a good chunk of them are employed by for-profit companies funded by venture capital, and VC lawyers are very good at writing employment contracts that provide plausible deniability of insider trading. The press MAKES MONEY (okay, takes money) by manipulating markets, and venture capitalists pay lots of money to ensure the market is manipulated in ways they can profit from. Private market manipulation is one of the costs of anonymity and privacy, and I don't really like paying for some off-list discussion of what appears to be a serious scalability and usability problem. Bitcoin is such a powerful tool because it broadcasts transactions to the network for everyone to see. Can we please broadcast some more technical details to this mailing list, including exactly what MtGox is doing, and how they wish to resolve it? If you gave me the entire code stack that MtGox runs on under an AGPLv3 license, I'm pretty sure I, along with everyone else here could come up with a workable solution. I think a code release would be a huge win for MtGox as well, and would cement their position as market leader in transparent cryptocurrency trading. Otherwise we are just a bunch of dinghys getting capsized one by one in a sea of market-manipulating white whales. Isn't the closed door market manipulation of the big banks one of the reasons we all started using Bitcoin in the first place? Why do revolutions always put the same old bullshit back in power? What we need is some transparent code evolution. On Mon, Feb 10, 2014 at 01:28:42PM +0100, Pieter Wuille wrote: Hi all, I was a bit surprised to see MtGox's announcement. The malleability of transactions was known for years already (see for example the wiki article on it, https://en.bitcoin.it/wiki/Transaction_Malleability it, or mails on this list from 2012 and 2013). I don't consider it a very big problem, but it does make it harder for infrastructure to interact with Bitcoin. If we'd design Bitcoin today, I'm sure we would try to avoid it altogether to make life easier for everyone. But we can't just change all infrastructure that exists today. We're slowly working towards making malleability harder (and hopefully impossible someday), but this will take a long time. For example, 0.8 not supporting non-DER encoded signatures was a step in that direction (and ironically, the trigger that caused MtGox's initial problems here). In any case, this will take years, and nobody should wait for this. There seem to be two more direct problems here. * Wallets which deal badly with modified txids. * Services that use the transaction id to detect unconfirming transactions. The first is something that needs to be done correctly in software - it just needs to be aware of malleability. The second is something I was unaware of and would have advised against. If you plan on reissuing a transaction because on old version doesn't confirm, make sure to make it a double spend of the first one - so that not both can confirm. I certainly don't like press making this sound like a problem in the Bitcoin protocol or clients. I think this is an issue that needs to be solved at the layer above - the infrastructure building on the Bitcoin system. Despite that, I do think that we (as a community, not just developers) can benefit from defining a standard way to identify transactions unambiguously. This is something Mark Karpeles suggested a few days ago, and my proposal is this: We define the normalized transaction id as SHA256^2(normalized_tx + 0x0100), where normalized_tx is the transaction with all input scripts replaced by empty scripts. This is exactly what would be signed inside transaction signatures using SIGHASH_ALL (except not substituting the previous scriptPubKey to be signed, and not dealing with the input being signed specially). An implementation is here: https://github.com/sipa/bitcoin/commits/normtxid. Note
Re: [Bitcoin-development] Malleability and MtGox's announcement
I cannot judge the competence of code I've never seen, so I have no position on that. What I HAVE seen quite clearly is both overt and covert market manipulation under cover of blame for 'the developers', or 'the exchange' You're doing it yourself with the phrase 'incompetence'. When you run an exchange of that volume, then maybe you might be in a position to say so, but if you were *invested* in a competitor to MtGox you'd make a lot of money calling them incompetent, wouldn't you? I'm looking to drum up some consulting business by making my observations about market manipulation public, and if I can't drum up any business, at least I can speak my mind free of any non-disclsosure agreements. What do you stand to gain from your statements on this list? On another note, is there any third-party archive of bitcointalk.org? I much prefer mailing lists because *I* have an archive. On Mon, Feb 10, 2014 at 11:39:19AM -0500, Christophe Biocca wrote: The bug MtGox is blaming has been documented on the wiki for years. Mark Karpeles was on IRC publicly discussing the topic https://bitcointalk.org/index.php?topic=458076.msg5052255#msg5052255 MtGox's incompetence has been on public display since day 1. I'm not sure what critical information you think secret cabals are keeping from you. On Mon, Feb 10, 2014 at 11:14 AM, Troy Benjegerdes ho...@hozed.org wrote: Okay, why the everloving FUCK is there not someone on this list with a @mtgox.com address talking about this? I started using bitcoin because I could audit the code, and when the developer cabal does stuff 'off-list' what you do is hand over market manipulation power to the selected cabal of company insiders who are discussing things 'off-list'. The people having a 'private' discussion about how to solve this are TAKING MONEY from everyone else, by having access to insider information. I don't think any of the developers actually have a clue this is the result, because a good chunk of them are employed by for-profit companies funded by venture capital, and VC lawyers are very good at writing employment contracts that provide plausible deniability of insider trading. The press MAKES MONEY (okay, takes money) by manipulating markets, and venture capitalists pay lots of money to ensure the market is manipulated in ways they can profit from. Private market manipulation is one of the costs of anonymity and privacy, and I don't really like paying for some off-list discussion of what appears to be a serious scalability and usability problem. Bitcoin is such a powerful tool because it broadcasts transactions to the network for everyone to see. Can we please broadcast some more technical details to this mailing list, including exactly what MtGox is doing, and how they wish to resolve it? If you gave me the entire code stack that MtGox runs on under an AGPLv3 license, I'm pretty sure I, along with everyone else here could come up with a workable solution. I think a code release would be a huge win for MtGox as well, and would cement their position as market leader in transparent cryptocurrency trading. Otherwise we are just a bunch of dinghys getting capsized one by one in a sea of market-manipulating white whales. Isn't the closed door market manipulation of the big banks one of the reasons we all started using Bitcoin in the first place? Why do revolutions always put the same old bullshit back in power? What we need is some transparent code evolution. On Mon, Feb 10, 2014 at 01:28:42PM +0100, Pieter Wuille wrote: Hi all, I was a bit surprised to see MtGox's announcement. The malleability of transactions was known for years already (see for example the wiki article on it, https://en.bitcoin.it/wiki/Transaction_Malleability it, or mails on this list from 2012 and 2013). I don't consider it a very big problem, but it does make it harder for infrastructure to interact with Bitcoin. If we'd design Bitcoin today, I'm sure we would try to avoid it altogether to make life easier for everyone. But we can't just change all infrastructure that exists today. We're slowly working towards making malleability harder (and hopefully impossible someday), but this will take a long time. For example, 0.8 not supporting non-DER encoded signatures was a step in that direction (and ironically, the trigger that caused MtGox's initial problems here). In any case, this will take years, and nobody should wait for this. There seem to be two more direct problems here. * Wallets which deal badly with modified txids. * Services that use the transaction id to detect unconfirming transactions. The first is something that needs to be done correctly in software - it just needs to be aware of malleability. The second is something I was unaware of and would have advised against. If you plan on reissuing a transaction because on old version
Re: [Bitcoin-development] Malleability and MtGox's announcement
A bitcoin problem is not really my problem, and if MtGox's investors can't seem to understand the value of publishing their code, I'll be happy to take their money as it leaves bitcoin for more distributed and transparent cryptocurrency ecosystems. I feel some sort of moral obligation to point out to this community when something stupid is going on, and if you think a MtGox problem is not a Bitcoin problem then I can't really help you, all I can do is point out my observations and facts as I see them, and then execute trades to relieve those who choose to ignore these facts of their money. Happy trading On Mon, Feb 10, 2014 at 10:57:03AM -0600, Nick Simpson wrote: You must be new here. MtGox very rarely comments on things like this publicly, outside of irc or their website. Second, MtGox problem is a MtGox problem. You have no right to demand access to their private code. If you feel wronged as a customer, sue them. Otherwise, they have no obligation to you. I believe you are barking up the wrong tree. Respectfully, Nick On February 10, 2014 10:14:02 AM CST, Troy Benjegerdes ho...@hozed.org wrote: Okay, why the everloving FUCK is there not someone on this list with a @mtgox.com address talking about this? I started using bitcoin because I could audit the code, and when the developer cabal does stuff 'off-list' what you do is hand over market manipulation power to the selected cabal of company insiders who are discussing things 'off-list'. The people having a 'private' discussion about how to solve this are TAKING MONEY from everyone else, by having access to insider information. I don't think any of the developers actually have a clue this is the result, because a good chunk of them are employed by for-profit companies funded by venture capital, and VC lawyers are very good at writing employment contracts that provide plausible deniability of insider trading. The press MAKES MONEY (okay, takes money) by manipulating markets, and venture capitalists pay lots of money to ensure the market is manipulated in ways they can profit from. Private market manipulation is one of the costs of anonymity and privacy, and I don't really like paying for some off-list discussion of what appears to be a serious scalability and usability problem. Bitcoin is such a powerful tool because it broadcasts transactions to the network for everyone to see. Can we please broadcast some more technical details to this mailing list, including exactly what MtGox is doing, and how they wish to resolve it? If you gave me the entire code stack that MtGox runs on under an AGPLv3 license, I'm pretty sure I, along with everyone else here could come up with a workable solution. I think a code release would be a huge win for MtGox as well, and would cement their position as market leader in transparent cryptocurrency trading. Otherwise we are just a bunch of dinghys getting capsized one by one in a sea of market-manipulating white whales. Isn't the closed door market manipulation of the big banks one of the reasons we all started using Bitcoin in the first place? Why do revolutions always put the same old bullshit back in power? What we need is some transparent code evolution. On Mon, Feb 10, 2014 at 01:28:42PM +0100, Pieter Wuille wrote: Hi all, I was a bit surprised to see MtGox's announcement. The malleability of transactions was known for years already (see for example the wiki article on it, https://en.bitcoin.it/wiki/Transaction_Malleability it, or mails on this list from 2012 and 2013). I don't consider it a very big problem, but it does make it harder for infrastructure to interact with Bitcoin. If we'd design Bitcoin today, I'm sure we would try to avoid it altogether to make life easier for everyone. But we can't just change all infrastructure that exists today. We're slowly working towards making malleability harder (and hopefully impossible someday), but this will take a long time. For example, 0.8 not supporting non-DER encoded signatures was a step in that direction (and ironically, the trigger that caused MtGox's initial problems here). In any case, this will take years, and nobody should wait for this. There seem to be two more direct problems here. * Wallets which deal badly with modified txids. * Services that use the transaction id to detect unconfirming transactions. The first is something that needs to be done correctly in software - it just needs to be aware of malleability. The second is something I was unaware of and would have advised against. If you plan on reissuing a transaction because on old version doesn't confirm, make sure to make it a double spend of the first one - so that not both can confirm. I certainly don't like press making this sound like a problem in the Bitcoin protocol or clients. I think this is an issue that needs to be
Re: [Bitcoin-development] Malleability and MtGox's announcement
Am 10.02.2014 13:28, schrieb Pieter Wuille: Hi all, I was a bit surprised to see MtGox's announcement. The malleability of transactions was known for years already (see for example the wiki article on it, https://en.bitcoin.it/wiki/Transaction_Malleability it, or mails on this list from 2012 and 2013). I don't consider it a very big problem, but it does make it harder for infrastructure to interact with Bitcoin. If we'd design Bitcoin today, I'm sure we would try to avoid it altogether to make life easier for everyone. Sorry, I'm not a developer, but I have got a question. It's a little bit off-topic and can't maybe answered easy. As I understand this attack someone renames the transaction ID before being confirmed in the blockchain. Not easy but if he is fast enough it should be possible. With a bit of luck for the attacker the new transaction is added to the block chain and the original transaction is discarded as double-spend. Right? Up to this point the attacker has nothing gained. But next the attacker stressed the Gox support and refers to the original transaction ID. Gox was then probably fooled in such cases and has refunded already paid Bitcoins to the attackers (virtual) Gox-wallet. So far everything is clear. But what I do not understand: Why apparently had so many customers of Gox payment defaults or severely delayed payments? I would imagine that the attacker may have doubled not only his own transaction (maybe for obfuscating the fraud). But then all transfers would still have go through anyway. And a normal customers would have been satisfied. Most people observe only their wallets, I think. What am I missing here? Sorry, is perhaps a silly question. But maybe you can put me on the right track. regards Oliver -- Androi apps run on BlackBerry 10 Introducing the new BlackBerry 10.2.1 Runtime for Android apps. Now with support for Jelly Bean, Bluetooth, Mapview and more. Get your Android app in front of a whole new audience. Start now. http://pubads.g.doubleclick.net/gampad/clk?id=124407151iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Malleability and MtGox's announcement
On Mon, Feb 10, 2014 at 11:47 AM, Oliver Egginger bitc...@olivere.de wrote: As I understand this attack someone renames the transaction ID before being confirmed in the blockchain. Not easy but if he is fast enough it should be possible. With a bit of luck for the attacker the new transaction is added to the block chain and the original transaction is discarded as double-spend. Right? Up to this point the attacker has nothing gained. But next the attacker stressed the Gox support and refers to the original transaction ID. Gox was then probably fooled in such cases and has refunded already paid Bitcoins to the attackers (virtual) Gox-wallet. At this point the attack should fail. Before crediting the funds back Gox should form a new transaction moving at least one of the coins the prior transaction was spending and wait for that transaction to confirm. Without performing this step— even if there were no malleability at all you'd have the risk that someone would go resurrect the old transaction and get a miner to mine it. Then it goes through. With performing it, even if they missed the mutated transaction in the chain their cancellation transaction could not confirm (because its a double spend). So far everything is clear. But what I do not understand: Why apparently had so many customers of Gox payment defaults or severely delayed payments? Back in September a lot of people were having stuck transactions and when I looked it was because Mtgox was spending immature coins: newly generated coins which cannot be spent for 100 blocks since their creation. (A rule since Bitcoin's started) Then later it was noticed that they were producing transactions with invalid DER encodings (excessively padded signatures). The Bitcoin network used to accept these transactions, but in order to more towards eliminating malleability Bitcoin 0.8 and later will not relay and mine them. Then after people started using mutation to fix those excessively padded transactions and/or someone was exploiting Gox's behavior— it seems that Gox's wallet may have been in a state where it thought a lot of coins weren't spent that were and was reusing them in new transansactions... this one is harder to tell externally— I saw it appeared to be producing a LOT of double spends with different destinations, but I'm guessing as to the exact cause. -- Android apps run on BlackBerry 10 Introducing the new BlackBerry 10.2.1 Runtime for Android apps. Now with support for Jelly Bean, Bluetooth, Mapview and more. Get your Android app in front of a whole new audience. Start now. http://pubads.g.doubleclick.net/gampad/clk?id=124407151iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Malleability and MtGox's announcement
On Mon, Feb 10, 2014 at 7:47 PM, Oliver Egginger bitc...@olivere.de wrote: As I understand this attack someone renames the transaction ID before being confirmed in the blockchain. Not easy but if he is fast enough it should be possible. With a bit of luck for the attacker the new transaction is added to the block chain and the original transaction is discarded as double-spend. Right? No, the problem was that the transaction MtGox produced was poorly formatted. It wouldn't cause a block containing the transaction to be rejected, but the default client wouldn't relay the transaction or add it into a block. This means that transaction stalls. If the attacker has a direct connection to MtGox, they can receive the transaction directly. The attacker would fix the formatting (which changes the transaction id, but doesn't change the signature) and then forward it to the network, as normal. The old transaction never propagates correctly. Up to this point the attacker has nothing gained. But next the attacker stressed the Gox support and refers to the original transaction ID. Gox was then probably fooled in such cases and has refunded already paid Bitcoins to the attackers (virtual) Gox-wallet. They sent out the transaction a second time. The right solution is that the new transaction should re-spend at least one of the coins that the first transaction spent. That way only one can possibly be accepted. -- Android apps run on BlackBerry 10 Introducing the new BlackBerry 10.2.1 Runtime for Android apps. Now with support for Jelly Bean, Bluetooth, Mapview and more. Get your Android app in front of a whole new audience. Start now. http://pubads.g.doubleclick.net/gampad/clk?id=124407151iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Malleability and MtGox's announcement
On Mon, Feb 10, 2014 at 12:47 PM, Tier Nolan tier.no...@gmail.com wrote: If the attacker has a direct connection to MtGox, they can receive the transaction directly. MtGox had a php script that returned base64 data for all their stalled transactions. Not just attackers used that, some people trying to unstick their transactions tried manually fixing them with honest intent and no idea it would potentially confuse mtgox's software. -- Android apps run on BlackBerry 10 Introducing the new BlackBerry 10.2.1 Runtime for Android apps. Now with support for Jelly Bean, Bluetooth, Mapview and more. Get your Android app in front of a whole new audience. Start now. http://pubads.g.doubleclick.net/gampad/clk?id=124407151iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
