Re: [Bitcoin-development] 0.8.1 ideas
On Mar 13, 2013, at 8:18 PM, Cameron Garnham da2...@gmail.com wrote: For me, everyone signed up to bitcoin thinking that there was a 1MB / block limit. The lock limits were unexpected, and could be considered extremely uncontroversial to remove. This. Software behavior which is not described by the source code should not be considered an integral part of the rule set. Any influence of external libraries on the consensus mechanism is unacceptable. -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] 0.8.1 ideas
On Friday, March 15, 2013 5:06:20 PM Benjamin Lindner wrote: On Mar 13, 2013, at 8:18 PM, Cameron Garnham da2...@gmail.com wrote: For me, everyone signed up to bitcoin thinking that there was a 1MB / block limit. The lock limits were unexpected, and could be considered extremely uncontroversial to remove. This. Software behavior which is not described by the source code should not be considered an integral part of the rule set. Any influence of external libraries on the consensus mechanism is unacceptable. Note that the lock limits were explicitly set in the bitcoind source code. -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] 0.8.1 ideas
On Fri, Mar 15, 2013 at 10:06 AM, Benjamin Lindner b...@benlabs.net wrote: This. Software behavior which is not described by the source code should not be considered an integral part of the rule set. Any influence of external libraries on the consensus mechanism is unacceptable. No one thinks its controversial to remove it or that it's a good thing to have— only that its technically somewhat complicated and risky to remove it. -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] 0.8.1 ideas
On Wed, Mar 13, 2013 at 12:56:29PM +, Luke-Jr wrote: Here's a simple proposal to start discussion from... BEFORE block 262144: - Never make a block that, combined with the previous 4 blocks, results in over 4500 transaction modifications. - Reject any block that includes more than 4500 transaction modifications on its own (slight soft-fork) - (these rules should make older clients safe under most circumstances) FROM block 262144 to block 393216 (hard fork #1): - Never make, and reject any block that includes more than 24391 transaction modifications on its own (this *should* be equivalent to 1 MB) - (this rules can make older client backports safe unless a reorg is more than 6 blocks deep) FROM block 393216 onward (hard fork #2): - Never make, and reject any block that includes more than 48781 transaction modifications on its own (this *should* be equivalent to 2 MB) - Accept blocks up to 2 MB in data size If we're going to consider doing this, at minimum we need to also include a separate limit for how much the UTXO set can be grown by each block, calculated as the size of the scriptPubKey + constant metadata. (tx hash, index #, nValue, nVersion, nHeight should cover it) A P2SH transaction txout would measure 71bytes under that model. Given that we haven't even shown we can limit the creation of txouts that can not be spent economically caution would dictate setting the UTXO growth limit fairly low, say 1/4th of the block limit. -- 'peter'[:-1]@petertodd.org signature.asc Description: Digital signature -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] 0.8.1 ideas
On Wed, Mar 13, 2013 at 8:05 AM, Peter Todd p...@petertodd.org wrote: If we're going to consider doing this, at minimum we need to also I beg people to not derail discussion about fixing things with discussion of other controversial changes. Luke-jr, any chance in getting you to revise your proposal to narrow the scope to things that don't need serious debate? -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] 0.8.1 ideas
On Wednesday, March 13, 2013 3:18:36 PM Gregory Maxwell wrote: On Wed, Mar 13, 2013 at 8:05 AM, Peter Todd p...@petertodd.org wrote: If we're going to consider doing this, at minimum we need to also I beg people to not derail discussion about fixing things with discussion of other controversial changes. I figured 2 MB in 2-3 years was fairly uncontroversial. If not, let's scrap that idea for now. Luke-jr, any chance in getting you to revise your proposal to narrow the scope to things that don't need serious debate? It was a one-time start the conversation proposal. I expect what we end up going with may be substantially different. Luke -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] 0.8.1 ideas
On Wed, Mar 13, 2013 at 03:26:14PM +, Luke-Jr wrote: On Wednesday, March 13, 2013 3:18:36 PM Gregory Maxwell wrote: On Wed, Mar 13, 2013 at 8:05 AM, Peter Todd p...@petertodd.org wrote: If we're going to consider doing this, at minimum we need to also I beg people to not derail discussion about fixing things with discussion of other controversial changes. I figured 2 MB in 2-3 years was fairly uncontroversial. If not, let's scrap that idea for now. The very statement that we're willing to increase the blocksize as our solution to increased transaction volume rather go down the path of off-chain transactions is incredibly controversial. Fuck it, I'll make this public: I've had at least one person who went to the trouble of finding my personal phone number just so they could leave a few text messages saying I was going to do serious harm to Bitcoin. At the same time I've also had a few people asking questions along the line of had started and/or was considering starting a formal group opposing the blocksize increase. I even got a significant anonymous donation a few weeks ago. (rather fittingly this was done by emailing me an easywallet URL from a throwaway account) It's not just forum trolls who care about the issue, even if they make the most noise about it. -- 'peter'[:-1]@petertodd.org signature.asc Description: Digital signature -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] 0.8.1 ideas
I'm not sure I understand the need for hard forks. We can get through this crisis by mining pool collusion to prevent forking blocks until there is widespread adoption of patched clients. Proposal: 1) Patch the pre-0.8 branches to support an increased lock count, whatever number is required to make sure that this problem never shows up again at the current block size (I defer to Luke-Jr and gmaxwell's numbers on this). 2) Patch all branches to not *generate* blocks which trigger the lock count limit. A larger block would still be accepted as valid, however, if it is on the longest chain. 3) Simultaneously, provide an additional non-standard patch to mining pool operators (50% network hash) *rejecting* blocks that trigger the lock count limit. This keeps miners in collusion with each other to stay on a 'compatibility fork'. 4) At some point in the future once we've crossed an acceptable adoption threshold, the miners remove the above patch in a coordinated way. Does that not get us past this crisis without a hard-fork? Mark (Aside: I'm for BOTH raising the block-size limit and off-chain transactions, but like it or not there are political sides to that debate and we should keep politics out of crisis management.) On Wed, Mar 13, 2013 at 5:56 AM, Luke-Jr l...@dashjr.org wrote: Here's a simple proposal to start discussion from... BEFORE block 262144: - Never make a block that, combined with the previous 4 blocks, results in over 4500 transaction modifications. - Reject any block that includes more than 4500 transaction modifications on its own (slight soft-fork) - (these rules should make older clients safe under most circumstances) FROM block 262144 to block 393216 (hard fork #1): - Never make, and reject any block that includes more than 24391 transaction modifications on its own (this *should* be equivalent to 1 MB) - (this rules can make older client backports safe unless a reorg is more than 6 blocks deep) FROM block 393216 onward (hard fork #2): - Never make, and reject any block that includes more than 48781 transaction modifications on its own (this *should* be equivalent to 2 MB) - Accept blocks up to 2 MB in data size - Discontinue support for clients prior to 0.8.1 I intentionally set the block numbers conservatively to try to account for the yet-unseen ASIC upgrade. Thoughts? -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] 0.8.1 ideas
On Wednesday, March 13, 2013 5:41:29 PM you wrote: I'm not sure I understand the need for hard forks. We can get through this crisis by mining pool collusion to prevent forking blocks until there is widespread adoption of patched clients. Anything requiring widespread adoption of patched clients *is by definition* a hard fork. Proposal: 1) Patch the pre-0.8 branches to support an increased lock count, whatever number is required to make sure that this problem never shows up again at the current block size (I defer to Luke-Jr and gmaxwell's numbers on this). This is a hard fork. The only way to avoid a hard fork is to apply the existing lock limit to all clients forever. That would be fine, except that pre-0.8 clients cannot reorg N blocks without dividing that limit by (N * 2) + 1; that leaves us with the limit of around 1000 locks per block on average. Each transaction uses at least 3 locks on average (many times more). So about 300 transactions per block. This is a much smaller limit than the 1 MB we've been assuming is the bottleneck so far, and the need to increase it is much more urgent - as Pieter noted on IRC, we are probably already using more than that even ignoring DP spam. The only reason pre-0.8 clients have survived as well as they have thus far is because the blockchain has managed to avoid very deep reorgs. Luke -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] 0.8.1 ideas
This may be a semantic issue. I meant that it's not a hard-fork of the bitcoin protocol, which I'm taking to mean the way in which we all *expected* every version of the Satoshi client to behave: the rules which we have documented informally on the wiki, this mailing list, and in code comments, etc. I'm just trying to prevent protocol-creep. Luke-Jr is suggesting that we add-to/modify the bitcoin protocol rules which all verifying implementations must adhere to. I'm suggesting that we instead change the old codebase to do what we expected it to do all along (what 0.8 does and what every other verifying implementation does), and through miner collusion buy ourselves enough time for people to update their own installations. I know there's people here who will jump in saying that the bitcoin protocol is the behavior of the Satoshi client, period. But which Satoshi client? 0.7 or 0.8? How do you resolve that without being arbitrary? And regardless, we are moving very quickly towards a multi-client future. This problem is very clearly a *bug* in the old codebase. So let's be forward thinking and do what we would do in any other situation: fix the bug, responsibly notify people and give them time to react, then move on. Let's not codify the bug in the protocol. Mark On Wed, Mar 13, 2013 at 10:58 AM, Pieter Wuille pieter.wui...@gmail.comwrote: On Wed, Mar 13, 2013 at 10:41:29AM -0700, Mark Friedenbach wrote: 4) At some point in the future once we've crossed an acceptable adoption threshold, the miners remove the above patch in a coordinated way. Does that not get us past this crisis without a hard-fork? This is a hardfork: it means some nodes will have to accept blocks they formerly considered invalid. -- Pieter -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] 0.8.1 ideas
On Wed, Mar 13, 2013 at 11:27:13AM -0700, Mark Friedenbach wrote: I know there's people here who will jump in saying that the bitcoin protocol is the behavior of the Satoshi client, period. But which Satoshi client? 0.7 or 0.8? The protocol is whatever the network enforces - and that is some mix of versions of the reference client right now, but doesn't need to remain that way. I would very much like to have a text book of rules that is authorative, and every client that follows it would be correct. Unfortunately, that is not how a consensus system works. All (full) clients validate all rules, and all must independently come to the same solution. Consensus is of utmost importance, more than some theoretical correctness. If we'd have a specification document, and it was discovered that a lot of nodes on the network were doing something different than the document, those nodes would be buggy, but it would be the specification that is wrong. That is what happened: 0.7 and before had a bug, but 0.8 was wrong for not following the rules of the network (which I hate to say, as I'm responsible for many changes in 0.8). As said in another thread, the problem in the old versions needs fixing (this would even be the case if no 0.8 existed at all, and no fork risk existed at all). But let's please do it in a way we can all agree about, in a controlled fashion. -- Pieter -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] 0.8.1 ideas
Agree. I quite like Mark's proposal. Yes, formally it is hard fork. But the step 4) can come very far in the future, when the penetration of 0.8 clients will be mininimal. slush On Wed, Mar 13, 2013 at 7:27 PM, Mark Friedenbach m...@monetize.io wrote: This problem is very clearly a *bug* in the old codebase. So let's be forward thinking and do what we would do in any other situation: fix the bug, responsibly notify people and give them time to react, then move on. Let's not codify the bug in the protocol. -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] 0.8.1 ideas
On Wednesday, March 13, 2013 6:27:13 PM Mark Friedenbach wrote: Luke-Jr is suggesting that we add-to/modify the bitcoin protocol rules which all verifying implementations must adhere to. I'm suggesting that we instead change the old codebase to do what we expected it to do all along (what 0.8 does and what every other verifying implementation does), and through miner collusion buy ourselves enough time for people to update their own installations. Curiously enough, at least MtGox's custom implementation stuck with the canonical blockchain despite 0.8's accidental rule change. I know there's people here who will jump in saying that the bitcoin protocol is the behavior of the Satoshi client, period. But which Satoshi client? 0.7 or 0.8? How do you resolve that without being arbitrary? And regardless, we are moving very quickly towards a multi-client future. This problem is very clearly a *bug* in the old codebase. So let's be forward thinking and do what we would do in any other situation: fix the bug, responsibly notify people and give them time to react, then move on. Let's not codify the bug in the protocol. No, if any other client released diverged from the consensus of all past/existing clients, we would do the same thing: call it a formerly unknown protocol rule, that this new client has a bug implementing, and be done with it. The only reason this particular issue needs special treatment is because the implications of the new rule mean that we're up against a hard limit in the protocol today rather than 2 years from now. Luke -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] 0.8.1 ideas
On Wed, Mar 13, 2013 at 1:10 PM, Matthew Mitchell matthewmitch...@thelibertyportal.com wrote: Why would it be a difficulty in getting people to update away from 0.7 and earlier? How long would that roughly take? If people are hesitant to update, imagine if a more serious vulnerability is found. It could be disastrous. The development community backports critical fixes which makes updating instead of upgrading possible, but that still is not free. Many people are carrying patches against Bitcoin which require integration and time for testing— even if its just an update. Small behavior changes can still break things for the users. For example, a major mining pool lost well over 1000 BTC when upgrading to 0.8 because the reindex interacted poorly with their pool server software and caused them to pay people 25 BTC per share, an update or upgrade is just a risky even whos risk can be minimized if its done at your own pace. Sometimes when there is a vulnerability what people will do is isolate their production nodes from the internet using upgraded nodes, so they avoid touching the production systems. Other times the vulnerability is only a DOS attack so they ignore it unless the attack happens, or only applies to something else they don't care about. Another point is that if everyone instantly upgrades in response to developers claim that an urgent is needed (as opposed to implementing other workarounds) then the security of the system much more obviously reduces to the ability to compromise a developer— something no one should want. When roll outs take time there is more time for review to catch things, fewer nodes harmed by an introduced flaw, etc. -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] 0.8.1 ideas
On Wed, Mar 13, 2013 at 09:14:03PM +, Luke-Jr wrote: On Wednesday, March 13, 2013 9:06:44 PM Andy Parkins wrote: On Wednesday 13 Mar 2013 12:56:29 Luke-Jr wrote: Here's a simple proposal to start discussion from... It seems to me that the biggest failure was not the development of two chains, but the assurance to users (by the client) that their transactions were confirmed. These are both the same thing. The idea of the client detecting/warning about not-trivial forking seems worthwhile too, though, assuming it doesn't already (AIUI it doesn't). I don't know if there's any automatic monitoring for forks, but if not I would assume that the core devs and/or Bitcoin Foundation would be planning to put some in place. But there's no reason I can see why end users clients should't be warning of such situations, too, when they can (obviously they won't always be aware of the fork). roy -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] 0.8.1 ideas
On Wed, Mar 13, 2013 at 2:22 PM, Roy Badami r...@gnomon.org.uk wrote: The idea of the client detecting/warning about not-trivial forking seems worthwhile too, though, assuming it doesn't already (AIUI it doesn't). It does warn— if its heard the fork and its on the lower difficulty side. Extending that to also alert if its on the winning side and the fork is long enough might be wise, though I have a little concern that it'll be mistaken to be more dependable than it would be. -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] 0.8.1 ideas
On Wed, Mar 13, 2013 at 02:27:01PM -0700, Gregory Maxwell wrote: On Wed, Mar 13, 2013 at 2:22 PM, Roy Badami r...@gnomon.org.uk wrote: The idea of the client detecting/warning about not-trivial forking seems worthwhile too, though, assuming it doesn't already (AIUI it doesn't). It does warn??? if its heard the fork and its on the lower difficulty side. Extending that to also alert if its on the winning side and the fork is long enough might be wise, though I have a little concern that it'll be mistaken to be more dependable than it would be. Still, it would have meant that all 0.8 users would have immediatley been told that something was wrong. I don't know to what extent it was luck that this was dealt with as promptly and efficiently as it was, but to the extent that luck was involved, a slew of 0.8 users shouting in various places wtf is going on couldn't but help in reducing the element of luck if something similar were to happen again. roy -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] 0.8.1 ideas
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 I think that the course of action is quite simple: 1. Upgrade all the clients to implement the lock limits. (in code, not at the DB exception layer). A bit of research is needed to work out exactly what these limits are so we can maximise the number of transactions. 2. Fix the DB layer, and test that all the clients can support 1MB blocks. 3. Once we are confident that the network supports 1MB blocks, set a date where the lock limits are removed. For me, everyone signed up to bitcoin thinking that there was a 1MB / block limit. The lock limits were unexpected, and could be considered extremely uncontroversial to remove. The discussion of larger blocks (i.e. 1MB ), that I happen to disagree with, is not relevant to the discussion of the removal of the lock limits. -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.19 (MingW32) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iF4EAREIAAYFAlFBF0QACgkQBJ8cMDO159aWbwEAs8Ldt8hRpzjS4HdrH3U9Jnaq MWhifXqkJuVC0TVCz3EBAOAfSogdSS7rJvtfV8FqTIox1ek/xJxuHvZdonUnQN1K =I5Cf -END PGP SIGNATURE- -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
