Re: [Bitcoin-development] Concerns Regarding Threats by a Developer to Remove Commit Access from Other Developers
Not that I know how to do this, but would you be willing to attempt some other method of measuring just how much of a super-majority we have before deploying code? Maybe that information would be helpful for everyone. Obviously such a poll couldn't be perfect, but maybe better than the information we have now. A) I don't believe we should consider changing the 1 MB limit now B) I conceptually believe in increasing block size, but would like to follow a more conservative process and wait to see if a stronger technical consensus on a plan to do so can develop. C) I'd like to go along with Gavin and Mike's 8MB proposal (maybe we wait til this is fully specified, but again not deployed) Perhaps there can even be 4 polls: Miners can vote in coinbases Known corporate entities can announce their vote Does the Bitcoin Foundation infrastructure still exist to represent some authenticated (I think) set of individuals A reddit poll I don't even know if I think this is a good idea, but just trying to find a way to move forward where more of us are on the same page. On Thu, Jun 18, 2015 at 2:23 PM, Gavin Andresen gavinandre...@gmail.com wrote: On Thu, Jun 18, 2015 at 1:42 PM, Alex Morcos mor...@gmail.com wrote: Let me take a pass at explaining how I see this. 1) Code changes to Bitcoin Core that don't change consensus: Wladimir is the decider but he works under a process that is well understood by developers on the project in which he takes under reasonable consideration other technical opinions and prefers to have clear agreement among them. Yes. 2) Changes to the consensus rules: As others have said, this isn't anyone's decision for anyone else. Yes. It's up to each individual user as to what code they run and what rules they enforce. So then why is everyone so up in arms about what Mike and Gavin are proposing if everyone is free to decide for themselves? I believe that each individual user should adhere to the principle that there should be no changes to the consensus rules unless there is near complete agreement among the entire community, users, developers, businesses miners etc. It is not necessary to define complete agreement exactly because every individual person decides for themselves. I believe that this is what gives Bitcoin, or really any money, its value and what makes it work, that we all agree on exactly what it is. So I believe that it is misleading and bad for Bitcoin to tell users and business that you can just choose without concern for everyone else which code you'll run and we'll see which one wins out. No. You should run the old consensus rules (on any codebase you want) until you believe that pretty much everyone has consented to a change in the rules. It is your choice, but I think a lot of people that have spent time thinking about the philosophy of consensus systems believe that when the users of the system have this principle in mind, it's what will make the system work best. I don't think I agree with pretty much everybody, because status-quo bias is a very powerful thing. Any change that disrupts the way they've been doing things will generate significant resistance -- there will be 10 or 20% of any population that will take a position of too busy to think about this, everything seems to be working great, I don't like change, NO to any change. For example, I think some of the resistance for bigger blocks is coming from contributors who are worried they, personally, won't be able to keep up with a bigger blockchain. They might not be able to run full nodes from their home network connections (or might not be able to run a full node AND stream Game of Thrones), on their old raspberry pi machines. The criteria for me is clear super-majority of the people and businesses who are using Bitcoin the most, and I think that criteria is met. 3) Code changes to Core that do change consensus: I think that Wladimir, all the other committers besides Gavin, and almost all of the other developers on Core would defer to #2 above and wait for its outcome to be clear before considering such a code change. Yes, that's the way it has mostly been working. But even before stepping down as Lead I was starting to wonder if there are ANY successful open source projects that didn't have either a Benevolent Dictator or some clear voting process to resolve disputes that cannot be settled with rough consensus. -- -- Gavin Andresen -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Concerns Regarding Threats by a Developer to Remove Commit Access from Other Developers
Let me take a pass at explaining how I see this. 1) Code changes to Bitcoin Core that don't change consensus: Wladimir is the decider but he works under a process that is well understood by developers on the project in which he takes under reasonable consideration other technical opinions and prefers to have clear agreement among them. 2) Changes to the consensus rules: As others have said, this isn't anyone's decision for anyone else. It's up to each individual user as to what code they run and what rules they enforce. So then why is everyone so up in arms about what Mike and Gavin are proposing if everyone is free to decide for themselves? I believe that each individual user should adhere to the principle that there should be no changes to the consensus rules unless there is near complete agreement among the entire community, users, developers, businesses miners etc. It is not necessary to define complete agreement exactly because every individual person decides for themselves. I believe that this is what gives Bitcoin, or really any money, its value and what makes it work, that we all agree on exactly what it is. So I believe that it is misleading and bad for Bitcoin to tell users and business that you can just choose without concern for everyone else which code you'll run and we'll see which one wins out. No. You should run the old consensus rules (on any codebase you want) until you believe that pretty much everyone has consented to a change in the rules. It is your choice, but I think a lot of people that have spent time thinking about the philosophy of consensus systems believe that when the users of the system have this principle in mind, it's what will make the system work best. 3) Code changes to Core that do change consensus: I think that Wladimir, all the other committers besides Gavin, and almost all of the other developers on Core would defer to #2 above and wait for its outcome to be clear before considering such a code change. I'm sure my description of point 2 is not the most eloquent or clear, but maybe someone else can try to elucidate this principle if they've grasped what I'm trying to say. On Thu, Jun 18, 2015 at 1:04 PM, justusranv...@riseup.net wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 2015-06-18 16:28, Jeff Garzik wrote: This is an engineering list. The quote precisely describes how the bitcoin consensus system functions. Users' choice is largely binary: Follow the rules, or bitcoin software ignores you. Software engineers should understand that they have a binary choice: produce the software that your customers want, or the world will ignore your software. There is *no inherent value* to Bitcoin's software rules. The only value that is exists is that produced by the individuals who voluntarily choose to run the software. Failing to account for all design requirements is bad engineering. Nobody cares about the design features of a bridge to nowhere. -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQIcBAEBCgAGBQJVgvoDAAoJECpf2nDq2eYj0h4P/0YaTsS963qpb63zvB6WlIPS 2lhCJ9FtAd3II5Et+5c/cisfJ9YI2OnM0y8nQpyB9NEOeueN1L1sLFcayE5aHASd EgF7F81AhQD2iSIVwQNs2qAzrZNC2/Nx+nBzBDcrgZ6gRiPpQdsNLy2p0OuZdOgX yG4xl6tKADB2kNi6tVPtZqUC300uQHvggtm+pexYilT0ojEbeVHCoDV40MNDZC2h 1kcdTnGU2SHJJqeZN2vChJCOMfhmK4JwKgoz7JRXe/GHkUUJKriE6Kb7SVczii9e 9qfcosbnR3gjATMoHFYuJX/nsUx52Q1LM9eQgvE8Ml+6Mim5bj2KCJFh7YISxSq9 FhDujfZFCRRQLPJCSkEUePxU/LS7lmoTZXYl3Zz1j9zbq4ncpRHpIFy9QX6iIqK6 Dursnge9ELQwB+H6HoosWRzxOZyo+oiGj17OngJvZYcvzrc2wjHbpZfVqSkmZepU SfJZ64O7yjjXjITwhOc4XF2drzvhsjTsHH5BIwdbCn82SoCkJIwXraj7sxIundli LUJBPiAE0csdmsvW/2kkxLsd9JwTw9lJ9Pf8fiqH3itgrkPkO5mf10DPPnay1SNk Wnm1bAJ05WnKXSo0m0SzaFgZkdfFuhWR4fieSzhLpa+s/HHj18NZvJCmCBR6ic9G 0A+51wwSZnAdMIw7lwIb =r4Co -END PGP SIGNATURE- -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] questions about bitcoin-XT code fork non-consensus hard-fork
Aaron, My understanding is that Gavin and Mike are proceeding with the XT fork, I hope that understanding is wrong. As for improving the non-consensus code to handle full blocks more gracefully. This is something I'm very interested in, block size increase or not. Perhaps I shouldn't hijack this thread, but maybe there are others who also believe this would ameliorate some of the time pressure for deciding on a block size increase. What is it that you would like to see improved? The fee estimation code that is included for 0.11 will give much more accurate fee estimates, which should allow adding the correct fee to a transaction to see it likely to be confirmed in a reasonable time. For further improvements: - There has recently been attention to overhauling the block creation and mempool limiting code in such a way that actual outstanding queues to be included in a block could also be incorporated in fee estimation. See https://github.com/bitcoin/bitcoin/pull/6281. - CPFP and RBF are candidates for inclusion in core soon, both of which could be integrated into transaction processing to handle the edge cases where a priori fee estimation fails. See https://github.com/bitcoin/bitcoin/pull/1647 and https://github.com/bitcoin/bitcoin/pull/6176 I know there has been much discussion of fee estimation not working for SPV clients, but I believe several independent servers which were serving the estimates from full nodes would go a long way towards allowing that information to be used by SPV clients even if its not a completely decentralized solution. See for example http://core2.bitcoincore.org/smartfee/latest.json On Mon, Jun 15, 2015 at 8:08 PM, Aaron Voisine vois...@gmail.com wrote: Wasn't the XT hard fork proposed as a last resort, should the bitcoin-core maintainers simply refuse to lift the 1Mb limit? No one wants to go that route. An alternate hard-fork proposal like BIP100 that gets consensus, or a modified version of gavin's that ups the limit to 8Mb instead of 20Mb, or hell even some major changes to the non-consunsus code to make it adequately handle the situation when blocks fill up, and allow wallet software to continue working with a send-and-forget use pattern, any of these would be enough to avoid the need for an XT only hard-fork. So far BIP100 is the only one that seems to actually be getting any sort of momentum toward consensus, and it was proposed... 2 days ago? When the XT fork was proposed as a last resort, it was when the opponents were (to my understanding) suggesting we just let blocks fill up, and hopefully things would just work out on their own. Aaron Voisine co-founder and CEO breadwallet.com On Mon, Jun 15, 2015 at 3:56 PM, Brian Hoffman brianchoff...@gmail.com wrote: Who is actually planning to move to Bitcoin-XT if this happens? Just Gavin and Mike? [image: image1.JPG] On Jun 15, 2015, at 6:17 PM, Faiz Khan faizkha...@gmail.com wrote: I'm quite puzzled by the response myself, it doesn't seem to address some of the (more serious) concerns that Adam put out, the most important question that was asked being the one regarding personal ownership of the proposed fork: How do you plan to deal with security incident response for the duration you describe where you will have control while you are deploying the unilateral hard-fork and being in sole maintainership control? I do genuinely hope that whomever (now and future) wishes to fork the protocol reconsider first whether they are truly ready to test/flex their reputation/skills/resources in this way... Intuitively, to me it seems counterproductive, and I don't fully believe it is within a single developer's talents to manage the process start-to-finish (as it is non-trivial to hard-fork successfully, others have rehashed this in other threads)... That being said I think it appropriate if Adam's questions were responded in-line when Mike is feeling up to it. I think that the answers are important for the community to hear when such a drastic change is being espoused. Faiz On Mon, Jun 15, 2015 at 4:56 PM, Bryan Bishop kanz...@gmail.com wrote: On Mon, Jun 15, 2015 at 3:55 PM, Mike Hearn m...@plan99.net wrote: Re: anyone who agrees with noted non-programmers MikeGavin must be non-technical, stupid, uninformed, etc OK, go ahead and show them the error of their ways. Anyone can write blogs. I worry that if this is the level of care you take with reading and (mis)interpreting Adam's messages, that you might not be taking extreme care with evaluating consensus changes, even while tired or sleeping. I encourage you to evaluate both messages and source code more carefully, especially in the world of bitcoin. However, this goes for everyone and not just you. Specifically, when Adam mentioned your conversations with non-technical people, he did not mean Mike has talked with people who have possibly not made pull requests to Bitcoin Core, so therefore Mike is
Re: [Bitcoin-development] Block Size Increase
That strikes me as a dangerous path forward. I don't actually think there is anything wrong with this: everybody eventually gets tired of arguing angels-dancing-on-the-head-of-a-pin, and we're left with the status quo What gives Bitcoin value aren't its technical merits but the fact that people believe in it. The biggest risk here isn't that 20MB blocks will be bad or that 1MB blocks will be bad, but that by forcing a hard fork that isn't nearly universally agreed upon, we will be damaging that belief. If I strongly believed some hard fork would be better for Bitcoin, say permanent inflation of 1% a year to fund mining, and I managed to convince 80% of users, miners, businesses and developers to go along with me, I would still vote against doing it. Because that's not nearly universal agreement, and it changes what people chose to believe in without their consent. Forks should be hard, very hard. And both sides should recognize that belief in the value of Bitcoin might be a fragile thing. I'd argue that if we didn't force through a 20MB fork now, and we ran into major network difficulties a year from now and had no other technical solutions, that maybe we would get nearly universal agreement, and the businesses and users that were driven away by the unusable system would be a short term loss in value considerably smaller than the impairment we risk by forcing a change. On Thu, May 7, 2015 at 10:52 AM, Gavin Andresen gavinandre...@gmail.com wrote: For reference: the blog post that (re)-started this debate, and which links to individual issues, is here: http://gavinandresen.ninja/time-to-roll-out-bigger-blocks In it, I asked people to email me objections I might have missed. I would still appreciate it if people do that; it is impossible to keep up with this mailing list, /r/bitcoin posts and comments, and #bitcoin-wizards and also have time to respond thoughtfully to the objections raised. I would very much like to find some concrete course of action that we can come to consensus on. Some compromise so we can tell entrepreneurs THIS is how much transaction volume the main Bitcoin blockchain will be able to support over the next eleven years. I've been pretty clear on what I think is a reasonable compromise (a one-time increase scheduled for early next year), and I have tried to explain why I think it it is the right set of tradeoffs. There ARE tradeoffs here, and the hard question is what process do we use to decide those tradeoffs? How do we come to consensus? Is it worth my time to spend hours responding thoughtfully to every new objection raised here, or will the same thing happen that happened last year and the year before-- everybody eventually gets tired of arguing angels-dancing-on-the-head-of-a-pin, and we're left with the status quo? I AM considering contributing some version of the bigger blocksize-limit hard-fork patch to the Bitcoin-Xt fork (probably target a hobbyist with a fast Internet connection, and assume Nelson's law to increase over time), and then encouraging merchants and exchanges and web wallets and individuals who think it strikes a reasonable balance to run it. And then, assuming it became a super-majority of nodes on the network, encourage miners to roll out a soft-fork to start producing bigger blocks and eventually trigger the hard fork. Because ultimately consensus comes down to what software people choose to run. -- -- Gavin Andresen -- One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight. http://ad.doubleclick.net/ddm/clk/290420510;117567292;y ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight. http://ad.doubleclick.net/ddm/clk/290420510;117567292;y___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] [softfork proposal] Strict DER signatures
Could we see a PR that adds it to BIP 66? Perhaps we'd all agree quickly that its so simple we can just add it... In either case it doesn't seem strictly necessary to me that it was non-standard before it becomes a soft-fork... On Tue, Feb 3, 2015 at 7:00 AM, Wladimir laa...@gmail.com wrote: One way to do that is to just - right now - add a patch to 0.10 to make those non-standard. This requires another validation flag, with a bunch of switching logic. The much simpler alternative is just adding this to BIP66's DERSIG right now, which is a one-line change that's obviously softforking. Is anyone opposed to doing so at this stage? Not opposed, but is kind of late for 0.10, I had hoped to tag rc4 today. Wladimir -- Dive into the World of Parallel Programming. The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Dive into the World of Parallel Programming. The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Reworking the policy estimation code (fee estimates)
Oh in just a couple of blocks, it'll give you a somewhat reasonable estimate for asking about every confirmation count other than 1, but it could take several hours for it to have enough data points to give you a good estimate for getting confirmed in one block (because the prevalent feerate is not always confirmed in 1 block 80% of the time) Essentially what it does is just combine buckets until it has enough data points, so after the first block it might be treating all of the txs as belonging to the same feerate bucket, but since the answer it returns is the median* fee rate for that bucket, its a reasonable answer right off the get go. Do you think it would make sense to make that 90% number an argument to rpc call? For instance there could be a default (I would use 80%) but then you could specify if you required a different certainty. It wouldn't require any code changes and might make it easier for people to build more complicated logic on top of it. *It can't actually track the median, but it identifies which of the smaller actual buckets the median would have fallen into and returns the average feerate for that median bucket. On Tue, Oct 28, 2014 at 9:59 AM, Gavin Andresen gavinandre...@gmail.com wrote: I think Alex's approach is better; I don't think we can know how much better until we have a functioning fee market. We don't have a functioning fee market now, because fees are hard-coded. So we get pay the hard-coded fee and you'll get confirmed in one or two or three blocks, depending on which miners mine the next three blocks and what time of day it is. git HEAD code says you need a fee of 10, satoshis/kb to be pretty sure you'll get confirmed in the next block. That looks about right with Alex's real-world data (if we take 90% chance as 'pretty sure you'll get confirmed'): Fee rate 10 Avg blocks to confirm 1.09 NumBlocks:% confirmed 1: 0.901 2: 1.0 3: 1.0 My only concern with Alex's code is that it takes much longer to get 'primed' -- Alex, if I started with no data about fees, how long would it take to be able to get enough data for a reasonable estimate of what is the least I can pay and still be 90% sure I get confirmed in 20 blocks ? Hours? Days? Weeks? -- -- Gavin Andresen -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Reworking the policy estimation code (fee estimates)
Sorry, perhaps I misinterpreted that question. The estimates will be dominated by the prevailing transaction rates initially, so the estimates you get for something like what is the least I can pay and still be 90% sure I get confirmed in 20 blocks won't be insane, but they will still be way too conservative. I'm not sure what you meant by reasonable. You won't get the correct answer of something significantly less than 40k sat/kB for quite some time. Given that the half-life of the decay is 2.5 days, then within a couple of days. And in fact even in the steady state, the new code will still return a much higher rate than the existing code, say 10k sat/kB instead of 1k sat/kB, but that's just a result of the sorting the existing code does and the fact that no one places transactions with that small fee. To correctly give such low answers, the new code will require that those super low feerate transactions are occurring frequently enough, but the bar for enough datapoints in a feerate bucket is pretty low, an average of 1 tx per block. The bar can be made lower at the expense of a bit of noisiness in the answers, for instance for priorities I had to make the bar significantly lower because there are so many fewer transactions confirmed because of priorities. I'm certainly open to tuning some of these variables. On Tue, Oct 28, 2014 at 10:30 AM, Alex Morcos mor...@gmail.com wrote: Oh in just a couple of blocks, it'll give you a somewhat reasonable estimate for asking about every confirmation count other than 1, but it could take several hours for it to have enough data points to give you a good estimate for getting confirmed in one block (because the prevalent feerate is not always confirmed in 1 block 80% of the time) Essentially what it does is just combine buckets until it has enough data points, so after the first block it might be treating all of the txs as belonging to the same feerate bucket, but since the answer it returns is the median* fee rate for that bucket, its a reasonable answer right off the get go. Do you think it would make sense to make that 90% number an argument to rpc call? For instance there could be a default (I would use 80%) but then you could specify if you required a different certainty. It wouldn't require any code changes and might make it easier for people to build more complicated logic on top of it. *It can't actually track the median, but it identifies which of the smaller actual buckets the median would have fallen into and returns the average feerate for that median bucket. On Tue, Oct 28, 2014 at 9:59 AM, Gavin Andresen gavinandre...@gmail.com wrote: I think Alex's approach is better; I don't think we can know how much better until we have a functioning fee market. We don't have a functioning fee market now, because fees are hard-coded. So we get pay the hard-coded fee and you'll get confirmed in one or two or three blocks, depending on which miners mine the next three blocks and what time of day it is. git HEAD code says you need a fee of 10, satoshis/kb to be pretty sure you'll get confirmed in the next block. That looks about right with Alex's real-world data (if we take 90% chance as 'pretty sure you'll get confirmed'): Fee rate 10 Avg blocks to confirm 1.09 NumBlocks:% confirmed 1: 0.901 2: 1.0 3: 1.0 My only concern with Alex's code is that it takes much longer to get 'primed' -- Alex, if I started with no data about fees, how long would it take to be able to get enough data for a reasonable estimate of what is the least I can pay and still be 90% sure I get confirmed in 20 blocks ? Hours? Days? Weeks? -- -- Gavin Andresen -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Reworking the policy estimation code (fee estimates)
RE: 90% : I think it's fine to use 90% for anything other than 1 confirmation, but if you look at the real world data test I did, or the raw data from this new code, you'll see that even the highest fee rate transactions only get confirmed at about a 90% rate in 1 block, so that if you use that as your cut-off you will sometimes get no answer and sometimes get a very high fee rate and sometimes get a reasonable fee rate, it just depends because the data is too noisy. I think thats just because there is no good answer to that question. There is no fee you can put on your transaction to guarantee greater than 90% chance of getting confirmed in one block. I think 85% might be safe? RE: tunable as command-line/bitcoin.conf: sounds good! OK, sorry to have all this conversation on the dev list, maybe i'll turn this into an actual PR if we want to comment on the code? I just wanted to see if it even made sense to make a PR for this or this isn't the way we wanted to go about it. On Tue, Oct 28, 2014 at 10:58 AM, Gavin Andresen gavinandre...@gmail.com wrote: On Tue, Oct 28, 2014 at 10:30 AM, Alex Morcos mor...@gmail.com wrote: Do you think it would make sense to make that 90% number an argument to rpc call? For instance there could be a default (I would use 80%) but then you could specify if you required a different certainty. It wouldn't require any code changes and might make it easier for people to build more complicated logic on top of it. RE: 80% versus 90% : I think a default of 80% will get us a lot of the fee estimation logic is broken, I want my transactions to confirm quick and a lot of them aren't confirming for 2 or 3 blocks. RE: RPC argument: I'm reluctant to give too many 'knobs' for the RPC interface. I think the default percentage makes sense as a command-line/bitcoin.conf option; I can imagine services that want to save on fees running with -estimatefeethreshold=0.5 (or -estimatefeethreshold=0.95 if as-fast-as-possible confirmations are needed). Setting both the number of confirmations and the estimation threshold on a transaction-by-transaction basis seems like overkill to me. -- -- Gavin Andresen -- ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] Reworking the policy estimation code (fee estimates)
I've been playing around with the code for estimating fees and found a few issues with the existing code. I think this will address several observations that the estimates returned by the existing code appear to be too high. For instance see @cozz in Issue 4866 https://github.com/bitcoin/bitcoin/issues/4866. Here's what I found: 1) We're trying to answer the question of what fee X you need in order to be confirmed within Y blocks. The existing code tries to do that by calculating the median fee for each possible Y instead of gathering statistics for each possible X. That approach is statistically incorrect. In fact since certain X's appear so frequently, they tend to dominate the statistics at all possible Y's (a fee rate of about 40k satoshis) 2) The existing code then sorts all of the data points in all of the buckets together by fee rate and then reassigns buckets before calculating the medians for each confirmation bucket. The sorting forces a relationship where there might not be one. Imagine some other variable, such as first 2 bytes of the transaction hash. If we sorted these and then used them to give estimates, we'd see a clear but false relationship where transactions with low starting bytes in their hashes took longer to confirm. 3) Transactions which don't have all their inputs available (because they depend on other transactions in the mempool) aren't excluded from the calculations. This skews the results. I rewrote the code to follow a different approach. I divided all possible fee rates up into fee rate buckets (I spaced these logarithmically). For each transaction that was confirmed, I updated the appropriate fee rate bucket with how many blocks it took to confirm that transaction. The hardest part of doing this fee estimation is to decide what the question really is that we're trying to answer. I took the approach that if you are asking what fee rate I need to be confirmed within Y blocks, then what you would like to know is the lowest fee rate such that a relatively high percentage of transactions of that fee rate are confirmed within Y blocks. Since even the highest fee transactions are confirmed within the first block only 90-93% of the time, I decided to use 80% as my cutoff. So now to answer estimatefee Y, I scan through all of the fee buckets from the most expensive down until I find the last bucket with 80% of the transactions confirmed within Y blocks. Unfortunately we still have the problem of not having enough data points for non-typical fee rates, and so it requires gathering a lot of data to give reasonable answers. To keep all of these data points in a circular buffer and then sort them for every analysis (or after every new block) is expensive. So instead I adopted the approach of keeping an exponentially decaying moving average for each bucket. I used a decay of .998 which represents a half life of 374 blocks or about 2.5 days. Also if a bucket doesn't have very many transactions, I combine it with the next bucket. Here is a link https://github.com/morcos/bitcoin/pull/3 to the code. I can create an actual pull request if there is consensus that it makes sense to do so. I've attached a graph comparing the estimates produced for 1-3 confirmations by the new code and the old code. I did apply the patch to fix issue 3 above to the old code first. The new code is in green and the fixed code is in purple. The Y axis is a log scale of feerate in satoshis per KB and the X axis is chain height. The new code produces the same estimates for 2 and 3 confirmations (the answers are effectively quantized by bucket). I've also completely reworked smartfees.py. It turns out to require many many more transactions are put through in order to have statistically significant results, so the test is quite slow to run (about 3 mins on my machine). I've also been running a real world test, sending transactions of various fee rates and seeing how long they took to get confirmed. After almost 200 tx's at each fee rate, here are the results so far: Fee rate 1100 Avg blocks to confirm 2.30 NumBlocks:% confirmed 1: 0.528 2: 0.751 3: 0.870 Fee rate 2500 Avg blocks to confirm 2.22 NumBlocks:% confirmed 1: 0.528 2: 0.766 3: 0.880 Fee rate 5000 Avg blocks to confirm 1.93 NumBlocks:% confirmed 1: 0.528 2: 0.782 3: 0.891 Fee rate 1 Avg blocks to confirm 1.67 NumBlocks:% confirmed 1: 0.569 2: 0.844 3: 0.943 Fee rate 2 Avg blocks to confirm 1.33 NumBlocks:% confirmed 1: 0.715 2: 0.963 3: 0.989 Fee rate 3 Avg blocks to confirm 1.27 NumBlocks:% confirmed 1: 0.751 2: 0.974 3: 1.0 Fee rate 4 Avg blocks to confirm 1.25 NumBlocks:% confirmed 1: 0.792 2: 0.953 3: 0.994 Fee rate 6 Avg blocks to confirm 1.12 NumBlocks:% confirmed 1: 0.875 2: 1.0 3: 1.0 Fee rate 10 Avg blocks to confirm 1.09 NumBlocks:% confirmed 1: 0.901 2: 1.0 3: 1.0 Fee rate 30 Avg blocks to confirm 1.12 NumBlocks:% confirmed 1: 0.886 2: 0.989 3: 1.0 Alex
Re: [Bitcoin-development] moving the default display to mbtc
I think Mark makes some good arguments. I realize this would only add to the confusion, but... What if we did relabel 100 satoshis to be some new kind of unit (bit or whatever else), with a proper 3 letter code, and then from a user standpoint, where people are using mBTC, they could switch to using Kbits (ok thats obviously bad, but you get the idea) at the same nominal price. But accounting backends and so forth would operate in the bit base unit with 2 decimals of precision. On Fri, Mar 14, 2014 at 12:01 PM, Mark Friedenbach m...@monetize.io wrote: A cup of coffee in Tokyo costs about 55 yen. You see similar magnitude numbers in both Chinas, Thailand, and other economically important East Asian countries. Expect to pay hundreds of rupees in India, or thousands of rupees in Indonesia. This concept that money should have low, single digits for everyday prices is not just Western-centric, it's English-centric. An expresso in Rome would have cost you a few (tens of?) thousand lira in recent memory. It was pegging of the Euro to the U.S. dollar that brought European states in line with the English-speaking world (who themselves trace lineage to the pound sterling). No, there is no culturally-neutral common standards for currency and pricing. But there are ill-advised, ill-informed standards in accounting software that we nevertheless must live with. These software packages do not handle more than two decimal places gracefully. That gives technical justifications for moving to either uBTC or accounting in Satoshis directly. An argument for uBTC is that it retains alignment with the existing kBTC/BTC/mBTC/uBTC conventions. However another limitation of these accounting software practices is that they do not always handle SI notation very well, particularly sub-unit prefixes. By relabeling uBTC to be a new three-digit symbol (XBT, XBC, IBT, NBC, or whatever--I really don't care), we are now fully compliant with any software accounting package out there. We are still very, very early in the adoption period. These are changes that could be made now simply by a few big players and/or the bitcoin foundation changing their practice and their users following suit. On 03/14/2014 07:49 AM, Andreas Schildbach wrote: How much do you pay for an Espresso in your local currency? At least for the Euro and the Dollar, mBTC 3.56 is very close to what people would expect. Certainly more familiar than µBTC 3558 or BTC 0.003578. Anyway, I was just sharing real-world experience: nobody is confused. On 03/14/2014 03:14 PM, Tamas Blummer wrote: You give them a hard to interpret thing like mBTC and then wonder why they rather look at local currency. Because the choices you gave them are bad. I think Bitcoin would have a better chance to be percieved as a currency of its own if it had prices and fractions like currencies do. 3.558 mBTC or 0.003578 BTC will never be as accepted as 3558 bits would be. Tamas Blummer Bits of Proof On 14.03.2014, at 15:05, Andreas Schildbach andr...@schildbach.de wrote: btw. None of Bitcoin Wallet's users complained about confusion because of the mBTC switch. In contrast, I get many mails and questions if exchange rates happen to differ by 10%. I suspect nobody looks at the Bitcoin price. It's the amount in local currency that matters to the users. On 03/13/2014 02:40 PM, Andreas Schildbach wrote: Indeed. And users were crying for mBTC. Nobody was asking for µBTC. I must admit I was not aware if this thread. I just watched other wallets and at some point decided its time to switch to mBTC. On 03/13/2014 02:31 PM, Mike Hearn wrote: The standard has become mBTC and that's what was adopted. It's too late to try and sway this on a mailing list thread now. On Thu, Mar 13, 2014 at 2:29 PM, Gary Rowe g.r...@froot.co.uk mailto:g.r...@froot.co.uk wrote: The MultiBit HD view is that this is a locale-sensitive presentation issue. As a result we offer a simple configuration panel giving pretty much every possible combination: icon, m+icon, μ+icon, BTC, mBTC, μBTC, XBT, mXBT, μXBT, sat along with settings for leading/trailing symbol, commas, spaces and points. This allows anyone to customise to meet their own needs beyond the offered default. We apply the NIST guidelines for representation of SI unit symbols (i.e no conversion to native language, no RTL giving icon+m etc). Right now MultiBit HD is configured to use m+icon taken from the Font Awesome icon set. However reading earlier posts it seems that μ+icon is more sensible. Let us know what you'd like. Links: m+icon screenshot: http://imgur.com/a/WCDoG Font Awesome icon: http://fortawesome.github.io/Font-Awesome/icon/btc/ NIST SI guidelines: http://physics.nist.gov/Pubs/SP811/sec07.html On 13 March 2014 12:56, Jeff Garzik jgar...@bitpay.com mailto:jgar...@bitpay.com
Re: [Bitcoin-development] [RFC] [BIP proposal] Dealing with malleability
I apologize if this has been discussed many times before. As a long term solution to malleable transactions, wouldn't it be possible to modify the signatures to be of the entire transaction. Why do you have to zero out the inputs? I can see that this would be a hard fork, and maybe it would be somewhat tricky to extract signatures first (since you can sign everything except the signatures), but it would seem to me that this is an important enough change to consider making. On Wed, Feb 12, 2014 at 5:52 PM, Luke-Jr l...@dashjr.org wrote: On Wednesday, February 12, 2014 8:27:52 PM Mark Friedenbach wrote: On 02/12/2014 08:44 AM, Alan Reiner wrote: Changing the protocol to use these static IDs is a pretty fundamental change that would never happen in Bitcoin. But they can still be useful at the application level to mitigate these issues. Not to mention that it would be potentially very insecure to have consensus depend on data (scriptSigs) which are not hashed in the Merkle structure of a block. Not that anyone on this list has suggested such a change, but I've seen it raised multiple times on the forum This would be a problem if it was used in the merkle tree, but I'm pretty sure using it for input selection would be pretty safe. One could even avoid the index by simply using the hashScript as the sole input value; then even CoinJoins would be safe without breaking chains of transactions (although this would break address reuse entirely - but I don't see that as a problem in a theoretical world). One of those things that an altcoin could improve upon Bitcoin with... ;) -- Android apps run on BlackBerry 10 Introducing the new BlackBerry 10.2.1 Runtime for Android apps. Now with support for Jelly Bean, Bluetooth, Mapview and more. Get your Android app in front of a whole new audience. Start now. http://pubads.g.doubleclick.net/gampad/clk?id=124407151iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Android apps run on BlackBerry 10 Introducing the new BlackBerry 10.2.1 Runtime for Android apps. Now with support for Jelly Bean, Bluetooth, Mapview and more. Get your Android app in front of a whole new audience. Start now. http://pubads.g.doubleclick.net/gampad/clk?id=124407151iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
