Re: [Bitcoin-development] Proposal to replace BIP0039
slush wrote : Two years ago I proposed exactly this and you refused to add extra information to mnemonic, because it isn't necessary and it makes it longer to mnemonization. What changed since then? I was wrong, and I fully acknowledge it. My concern was that adding extra information would make the mnemonic longer than 12 words. In addition, you proposed to allocate these extra bits for a checksum, not for metadata. However, a checksum does not really add any information, because Electrum checks the existence of a wallet directly from the blockchain. So, my feeling at that time was that adding extra bits would increase the risks (a longer seed is harder to memorize, increases the probability of mistakes, etc), and did not bring any real benefit. However, you showed since then how to solve this by using a slightly longer dictionary, and I do like your solution, I find it absolutely brilliant. In addition, I realize now that metadata (ie a version number) is crucially needed, for the reasons mentioned in my previous post. Hm, what exactly do you need to store about wallet structure? I lived in opinion that everything is able to recover using CKD function to generate new addresses and blockchain lookups for their balances. BIP32 gives a lot of freedom to wallet developers: it does not specify which branches of the HD tree shall be used for which purpose. However, if you want to recover a wallet from its mnemonic (a requirement for Electrum), then you need to know which branches to explore. In Electrum 1.9 I had to make some choices about branch allocation. However, the decisions that I made are certainly not final, so it is important to be able to change them in the future. Thus, this metadata needs to be added to the mnemonic. Yes, that's true. It isn't possible to make everybody 100% happy. At least I wanted to be constructive and asked you to replace the most problematic words. No pull request from you so far. The solution I propose is very different from BIP39, and it does not require to predefine a dictionary. My proposal is actually somewhat similar to Pieter Wuille's proposal, which I discovered after his recent post. ( https://bitcointalk.org/index.php?topic=102349.0 ) Yes, it was original idea. So far I don't think this is a problem. Of course some words may have some meaning across languages, but it should be easy to avoid them. There are tens of thousands words in every language and we need to pick only 2048 words to wordlist. ... Are your worries about overlapping words across languages a real issue? No, there are not so many words that are frequent enough. Overlapping will be an issue, especially if we go for a 4096 words dictionary. If I understand this well, it is basically one-way algorithm mnemonic - seed, right? Seed cannot be printed out as mnemonic, because there's hashing involved, but the bi-directionality has been the original requirement for such algorithm (at least in Electrum and bip39). You are right, this encoding is not symmetric. Bi-directionality has never been a requirement for Electrum. May I ask why you need bi-directionality in Trezor? (the only reason I can think of is if you want to export a bip32 branch into another wallet, but this would create a very long mnemonic string) Then, how is this different to picking 12 random words from dictionary and hashing them together? I don't see any benefit in that mining part of the proposal (except that it is lowering the entropy for given length of mnemonic). it makes it possible to hash a utf8 string, and to retrieve the metadata from the hash. Thus we don't need to spend ages arguing about the best choice of a dictionary, and to set it in stone. -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Making fee estimation better
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 There's no reason the signing can't be done all at once. The wallet app would create and sign three transactions, paying avg-std.D, avg, and avg+std.D fee. It just waits to broadcast the latter two until it has to. On 10/25/13 5:02 AM, Andreas Petersson wrote: Worth thinking about the whole ecosystem of wallets involved; they all have to handle double-spends gracefully to make tx replacement of any kind user friendly. We should try to give people a heads up that this is coming soon if that's your thinking. If there is a situation where wallets are supposed to constantly monitor the tx propagation and recreate their transactions with different fees, this would make a lot of usecases inconvenient. half-offline bluetooth transactions, users with unstable connections, battery power lost, etc, etc. - and last but not least power concerns on hardware wallets on the bitcoincard (tx signing drains a significant amount of power and should therefore only be done once) -Andreas -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.19 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJSanJVAAoJEAdzVfsmodw4RHYQAKBrku4S80GXtbt4wBgkRMgx EQuobBrwtknxHOhKyYuBeAJ+h8ao1zSSNeqLvS5fJShH7vwBD2UOePLw4Nsy5p9U pe56c07pRmgi+EWdq/3o1tggp9HN0FR3HDRwt03U4qrPTx449kHb11aOw5KZH7VS ZiG09gKxkMPOtUy9dmVukjkG3zQ1AWjax+aOoseCnkU8u1I4kfhOyWLIjD7ciMm4 07gD8MzBLHTfJ6/pwUczQCby76Xdg51G/5d/toT3EnXyEOC7tCbI4xunAn1eIyg3 eCUNYaOQ7WYV9tjBUDGFwjVkGDJ8KdzEUqMPEK5nAWF29vmrwBSGJ4H2C47OkTQA 58Ie0hEYc5FMNuUCUWz3IGt2zoQ/8YENtNUDKG8oVoNhAIp5zkLK8wsMAJjZP6WM z56JUl8NZ2Ka5U1OelImGGVZIx4NXrXlccyxemAn3/c+krkpNv0CHAeMCeNbPG8i e4l2vQandiBW4NBGVYcm5A/EO6VJHAJhLEPT0pjmbuq4qTACo4Fgeb0LpOnWb/1a 6b1SdGGhMMrXeR2IaIbnx0+0WArixsOPl9w+R9WbrMh8g7hYBLH8EpGrRj0omim7 OoJb+W599HU37XZyWtuov+8Ouh5DpnP9l4hvNxHmro77uPPq10i/ibMd0Bnm4zZd ALtIYpYYgUCN1D9lQwPQ =BjIH -END PGP SIGNATURE- -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Making fee estimation better
There's no reason the signing can't be done all at once. The wallet app would create and sign three transactions, paying avg-std.D, avg, and avg+std.D fee. It just waits to broadcast the latter two until it has to. i see several reasons why this is problematic. So how would that work in a setting where the user signs a transaction created offline, transmitted via Bluetooth via a one-way broadcast? does it transmit all 3 tx to the receiver and just hopes they he will do the right thing? On 10/25/13 5:02 AM, Andreas Petersson wrote: Worth thinking about the whole ecosystem of wallets involved; they all have to handle double-spends gracefully to make tx replacement of any kind user friendly. We should try to give people a heads up that this is coming soon if that's your thinking. If there is a situation where wallets are supposed to constantly monitor the tx propagation and recreate their transactions with different fees, this would make a lot of usecases inconvenient. half-offline bluetooth transactions, users with unstable connections, battery power lost, etc, etc. - and last but not least power concerns on hardware wallets on the bitcoincard (tx signing drains a significant amount of power and should therefore only be done once) -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Making fee estimation better
On Fri, Oct 25, 2013 at 02:02:35PM +0200, Andreas Petersson wrote: Worth thinking about the whole ecosystem of wallets involved; they all have to handle double-spends gracefully to make tx replacement of any kind user friendly. We should try to give people a heads up that this is coming soon if that's your thinking. If there is a situation where wallets are supposed to constantly monitor the tx propagation and recreate their transactions with different fees, this would make a lot of usecases inconvenient. half-offline bluetooth transactions, users with unstable connections, battery power lost, etc, etc. - and last but not least power concerns on hardware wallets on the bitcoincard (tx signing drains a significant amount of power and should therefore only be done once) Anyway, as I've said repeatedly my problem with fee estimation is that it needs to be combined with some form of transaction replacement to give users a way to recover from bad estimates, not that I think the idea shouldn't be implemented at all. After all, we alrady have fee estimation: wallet authors and users manully estimate fees! This particular case is a nasty one re: recovering from a bad estimate, and it's exactly why the payment protocol is designed for the sender to give the receiver a copy of every transaction they make so the receiver can be held responsible for getting them mined, eg. with child-pays-for-parent, out-of-band fee payment, or maybe even by adding inputs to the transaction. (SIGHASH_ANYONECANPAY) -- 'peter'[:-1]@petertodd.org 0001231d6e04b4b18f85fa0ad00e837727e7141eaa8cfecc734b signature.asc Description: Digital signature -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Making fee estimation better
Two thoughts: 1. Please keep it simple, miner will override it either. 2. If block construction algorithm compares alternate chains and not individual transactions, then receiver can bump up the fee by spending the unconfirmed output again with higher fee, no need for replacement in the mempool. Tamas Blummer -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] BIP 38
Hey everyone, I have noticed that there was a recent change to BIP 0038 (Password-Protected Private Key) on the Wiki, which is a proposal I wrote in late 2012. Gregory, it looks to me as though you have made this change, and I'm hoping for your help here. The change suggests that the number was never assigned, and that there has been no discussion regarding the proposal on this list. I had this number assigned by Amir Taaki in November of 2012, consistent with what I understood the procedure to be at the time by reading BIP 0001 on the Wiki. First off, I want to confirm that when I send to the list, that there isn't a technical reason it's not getting to everybody. I believe I most recently mentioned BIP 38 to this list on August 17, 2013. (EDIT: seems my prior messages, including an earlier revision of this message, have not made it to the list) Secondly, in the case that it is deemed that this has never been properly submitted, discussed, or pushed forward, I'd like to propose that this happen, and request help with the formalities where I'm lacking. I believe BIP 38 is a valuable proposal that is seeing real-world use. BIP 38 allows people to create private keys (including paper wallets) protected by a password, and also allows one party to select the password for paper wallets to be created by another party. Real-world use includes a working implementation at BitAddress.org, one at Bit2Factor.org, implementation by Mycelium, and others. Also, others are informally using it as a sort of abbreviated escrow scheme where a buyer and seller agree on the buyer maintaining control over the release of funds. In short, it would be terribly confusing to reassign the number BIP 38 after already having had an established meaning for the better part of the year, particularly on what appears to be procedural grounds. Mike -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Making fee estimation better
Do you think we're at the point where wallets have to be able to actively bid the fee using replacement due to block contention? I think a fee estimation API is just a data point. Depending on the properties of the estimator, and how that's presented in the UI, it could serve to either increase or decrease the need for recovery. Like you said, we already have fee estimation in the form of user, please estimate the fee! Now we want to make fee estimation better, and one key aspect of better fee estimation is decreasing the need for recovery. Techniques like signing multiple transactions with different fee levels should become less useful the better you are at estimating the fee. What I find interesting is that fee estimation can look at the size and type of the transaction, the age of the inputs, the number of inputs versus outputs, amount of the outputs, factor in [assumptions about] what fee policies miners are actually using, and after all that, look at the actual competing transactions on the blockchain and try to figure out how many of those are even real. For example, if you just look at fee-per-KB of mempool versus fee-per-KB of recently mined transactions, without taking into account input age, number of inputs vs outputs, output amounts... all the other things miner might have used to discriminate between transactions, then I don't think you'll end up with a better fee estimator. Contention might bump you out of a few blocks, but if the basis for calculating the fee is fundamentally compatible with the relay policies and the transaction-inclusion policies being run by large mining pools, the transaction isn't dead, it's just pending. On Fri, 25 Oct 2013 09:13:23 -0700, Peter Todd p...@petertodd.org wrote: On Fri, Oct 25, 2013 at 02:02:35PM +0200, Andreas Petersson wrote: Worth thinking about the whole ecosystem of wallets involved; they all have to handle double-spends gracefully to make tx replacement of any kind user friendly. We should try to give people a heads up that this is coming soon if that's your thinking. If there is a situation where wallets are supposed to constantly monitor the tx propagation and recreate their transactions with different fees, this would make a lot of usecases inconvenient. half-offline bluetooth transactions, users with unstable connections, battery power lost, etc, etc. - and last but not least power concerns on hardware wallets on the bitcoincard (tx signing drains a significant amount of power and should therefore only be done once) Anyway, as I've said repeatedly my problem with fee estimation is that it needs to be combined with some form of transaction replacement to give users a way to recover from bad estimates, not that I think the idea shouldn't be implemented at all. After all, we alrady have fee estimation: wallet authors and users manully estimate fees! This particular case is a nasty one re: recovering from a bad estimate, and it's exactly why the payment protocol is designed for the sender to give the receiver a copy of every transaction they make so the receiver can be held responsible for getting them mined, eg. with child-pays-for-parent, out-of-band fee payment, or maybe even by adding inputs to the transaction. (SIGHASH_ANYONECANPAY) -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] BIP 38
On Fri, Oct 25, 2013 at 11:50 AM, Mike Caldwell mcaldw...@swipeclock.com wrote: I have noticed that there was a recent change to BIP 0038 (Password-Protected Private Key) on the Wiki, which is a proposal I wrote in late 2012. Gregory, it looks to me as though you have made this change, and I’m hoping for your help here. The change suggests that the number was never assigned, and that there has been no discussion regarding the proposal on this list. Greetings, (repeating from our discussion on IRC) No prior messages about your proposal have made it to the list, and no mention of the assignment had been made in the wiki. The first I ever heard of this scheme was long after you'd written the document when I attempted to assign the number to something else then noticed something existed at that name. Since you had previously created BIP documents without public discussion (e.g. BIP 22 https://en.bitcoin.it/wiki/OP_CHECKSIGEX_DRAFT_BIP [...] Or, I wonder did your emails just get eaten that time too?), I'd just assumed something similar had happened here. I didn't take any action at the time I first noticed it, but after someone complained about bitcoin-qt not confirming with BIP38 to me today it was clear to me that people were confusing this with something that was officially (as much as anything is) supported, so I moved the document out. (I've since moved it back, having heard from you that you thought that it had actually been assigned/announced). With respect to moving it forward: Having a wallet which can only a single address is poor form. Jean-Paul Kogelman has a draft proposal which is based on your BIP38 work though the encoding scheme is different, having been revised in response to public discussion. Perhaps efforts here can be combined? -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] BIP 38
Gregory, No problem, thanks for providing the IRC recap, and glad I've finally made radio contact with the list. Perhaps there can be some long overdue discussion on the topic. I see Kogelman's improvements to my proposal as being of merit and may very well be sufficient to supersede what I've originally proposed. I suppose the main thing I'm wanting to ensure is that the identity of my original proposal is maintained. Regardless of whether a paper wallet or physical bitcoin with a single address is poor form or whether my proposal is rejected or superseded, I hope there can be a consensus that BIP38 can continue to be understood to mean Password-protected private key proposal by Mike Caldwell, and that it can appear in the lists of BIPs alongside others. Regarding BIP 22... I in fact did not originally attempt to post to the list over what I had created and called BIP 22 once upon a time, I literally just created a wiki entry contrary to advice in BIP 1 that I had not read at the time. I recognize it's totally legitimate to feel and act upon the appearance that BIP 38 was created in a similar shortcut fashion. Certainly, the next thing I propose will be in the form of a draft outside the BIP numberspace and I won't solicit a BIP number without an established consensus in the future. That said, I'm asking for BIP 38 to stand and be recognized as in existence, so as to not confuse those who call it by that name and who have already chosen to do something with it (whether that's to implement it, or to draft improvements to it like Kogelman). If I did BIP 38 over again, there's a couple shortcomings of my own that I wouldn't mind seeing addressed in another iteration, and the right venue for that may very well be to contribute to Kogelman's work. My particular improvements might include wanting the ability to outsource the computationally expensive step to another service at a minimized risk to the user, potentially the ability to have special-purpose encrypted minikeys (sort of how ARM has Thumb for places where the tradeoff makes sense), and a typo check with better privacy (I currently use sha256(address)[0...3] which may unintentionally reveal the bitcoin address, if it's funded, to someone who has the encrypted key but doesn't know the password). mike -Original Message- From: Gregory Maxwell [mailto:gmaxw...@gmail.com] Sent: Friday, October 25, 2013 2:05 PM To: Mike Caldwell Cc: bitcoin-development@lists.sourceforge.net Subject: Re: [Bitcoin-development] BIP 38 On Fri, Oct 25, 2013 at 11:50 AM, Mike Caldwell mcaldw...@swipeclock.com wrote: I have noticed that there was a recent change to BIP 0038 (Password-Protected Private Key) on the Wiki, which is a proposal I wrote in late 2012. Gregory, it looks to me as though you have made this change, and I’m hoping for your help here. The change suggests that the number was never assigned, and that there has been no discussion regarding the proposal on this list. Greetings, (repeating from our discussion on IRC) No prior messages about your proposal have made it to the list, and no mention of the assignment had been made in the wiki. The first I ever heard of this scheme was long after you'd written the document when I attempted to assign the number to something else then noticed something existed at that name. Since you had previously created BIP documents without public discussion (e.g. BIP 22 https://en.bitcoin.it/wiki/OP_CHECKSIGEX_DRAFT_BIP [...] Or, I wonder did your emails just get eaten that time too?), I'd just assumed something similar had happened here. I didn't take any action at the time I first noticed it, but after someone complained about bitcoin-qt not confirming with BIP38 to me today it was clear to me that people were confusing this with something that was officially (as much as anything is) supported, so I moved the document out. (I've since moved it back, having heard from you that you thought that it had actually been assigned/announced). With respect to moving it forward: Having a wallet which can only a single address is poor form. Jean-Paul Kogelman has a draft proposal which is based on your BIP38 work though the encoding scheme is different, having been revised in response to public discussion. Perhaps efforts here can be combined? -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Making fee estimation better
On Fri, Oct 25, 2013 at 12:35:34PM -0700, Jeremy Spilman wrote: Do you think we're at the point where wallets have to be able to actively bid the fee using replacement due to block contention? If Bitcoin continues to grow we probably will be at some as-yet-unknown point in the future. I think a fee estimation API is just a data point. Depending on the properties of the estimator, and how that's presented in the UI, it could serve to either increase or decrease the need for recovery. Like you said, we already have fee estimation in the form of user, please estimate the fee! Now we want to make fee estimation better, and one key aspect of better fee estimation is decreasing the need for recovery. Techniques like signing multiple transactions with different fee levels should become less useful the better you are at estimating the fee. Yes, but equally all estimates are imperfect, and you can trade-off risk that your transaction will not go through initially for lower fees. Estimates can be made sufficiently conservative that they are rarely wrong - this is basically the strategy of the current system. Given that demand for blockchain space isn't saturated it works reasonably well for now. But without a good mechanism to recover from an initial bad estimate you have to be more conservative than is efficient. What I find interesting is that fee estimation can look at the size and type of the transaction, the age of the inputs, the number of inputs versus outputs, amount of the outputs, factor in [assumptions about] what fee policies miners are actually using, and after all that, look at the actual competing transactions on the blockchain and try to figure out how many of those are even real. For example, if you just look at fee-per-KB of mempool versus fee-per-KB of recently mined transactions, without taking into account input age, number of inputs vs outputs, output amounts... all the other things miner might have used to discriminate between transactions, then I don't think you'll end up with a better fee estimator. To a first approximation there's not much reason for miners to take anything other than fee-per-KB into account when determining what transactions to mine; you want to stuff your 1MB block full of high paying transactions. That a child tx may make a parent more profitable to mine complicates things - Gavin's current fee estimator also makes too-low-estimates in that case - and not all algorithms to do so will come to the same conclusion. (doing it perfectly is something like O(n^2), and imperfectly is O(1) but doesn't handle multiple children well) There are some second-order effects, a block is less likely to be orphaned if all transactions in it have propagated sufficiently, thus a miner should penalize very recently broadcast transactions. In addition because miners never orphan themselves large miners have a significant advantage regarding orphan-inducing effects. However those effects all tend to be miner specific, and/or only temporary. FWIW the logic behind orphans is currently rather frightening: a rational miner will, the moment they learn that a block exists via the quickly propagating block header, start working to extend that block with one that either doesn't contain any transactions, or only contains transactions they can be reasonably sure another miner didn't mine. (e.g. via exclusive tx mining contracts) This boosts their profit because they aren't wasting their effort while the rest of the block propagates, removes much of the incentive have any limit on block size, and incentivizes miners to extend chains they haven't actually validated yet. (relying on the other miners incentive not to produce an invalid block) Contention might bump you out of a few blocks, but if the basis for calculating the fee is fundamentally compatible with the relay policies and the transaction-inclusion policies being run by large mining pools, the transaction isn't dead, it's just pending. With a size-limited blocks inclusion is more a matter of supply and demand than policy. -- 'peter'[:-1]@petertodd.org 00066c29f3319f83f1c6e912b5add5534da1b938c4c65a07b02a signature.asc Description: Digital signature -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Making fee estimation better
On Fri, Oct 25, 2013 at 5:51 PM, Jeremy Spilman jer...@taplink.co wrote: ** Gavin, can you confirm the best place to read up on the discuss fee estimation changes for v0.9? The blog post is the best place for high-level overview. The (closed for now, but it will come back) pull request is the best place for low-level details and nit-picking discussion: https://github.com/bitcoin/bitcoin/pull/3024 I think fee estimation at its core is about providing a data point, or even call it an API, which can be used however you see fit. What parameters do I want to see in a 'fee estimation' API? - 30 minutes vs 24 hours processing time - Confidence Levels (50%/90%) The pull request adds an 'estimatefees' JSON-RPC api call: estimatefees [prioritymedian=0.1] [feemedian=0.5] Estimates the priority or fee a transaction needs to be relayed across the network and included in the block chain. prioritymedian and feemedian are values from 0.0 to 1.0, where 0.0 will return the smallest recently-included-in-a-block priority (or fee) seen, 1.0 the largest, and 0.5 the median priority (or fee) for transactions that were broadcast on the network and included in a block. The default value for prioritymedian (0.1) is chosen to return a priority for free transactions that will eventually be confirmed, but might take several hours. The default value for feemedian (0.5) returns how much fee you should include to have your transactions confirmed in an average amount of time. Values returned are: freepriority : priority needed to out-compete a prioritymedian fraction of free transactions to be relayed and included in blocks. feeperbyte : fee, in satoshis/byte, needed to out-compete a feemedian fraction of fee-paying transactions. Values of -1.0 are returned if not enough transactions have been seen to make a good estimate. That API doesn't give 30 minute versus 24 hour confirmation time or confidence intervals. I've always regretted not taking a statistics class; if you want to help write code that estimates confidence intervals send me an email. The API certainly isn't set in stone. - Is it globally consistent? Ummm roughly, yes, it will be. Nodes that have just joined the network and haven't seen enough transactions enter and leave the memory pool will have a different estimate than long-running nodes, but in my testing the estimate narrows down very quickly (with three or four blocks enough fee-paying transactions have been seen to make a reasonable estimate; it takes longer to see enough free transactions to get a good estimate of the priority needed to get into the free space of a block). RE: lots of other comments: I feel like there is a lot of in the weeds discussion here about theoretical, what-if-this-and-that-happens-in-the-future scenarios. I would just like to point out (again) that this is not intended to be The One True Solution For Transaction Fees And Transaction Prioritization. If you've got a better mechanism for estimating fees, fantastic! If it turns out estimates are often-enough wrong to be a problem and you've got a solution for that, fantastic! RE: are we already seeing pressure on transaction fees: I believe we are, yes. As part of the prep work for the smart fee work I spent some time plotting priority (for zero-fee transactions) and transaction fee (for zero-priority transactions) versus confirmation time, and it looks to me like people/services are starting to include more than the hard-coded fees in the reference implementation-- I assume because they want their transactions to be confirmed more quickly. There is definitely already competition among zero-fee transactions for the free block space. One of the reasons I'm comfortable with the fee changes I'm proposing is if the estimation code gets it very wrong we'll see that first as free transactions taking too long to confirm, but they'll confirm eventually because priority increases over time. -- -- Gavin Andresen -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] Feedback requested: reject p2p message
Mike Hearn has been lobbying for an error message in the Bitcoin p2p protocol for years (at least since the ban peers if they send us garbage denial-of-service mitigation code was pull-requested). This came up again with my proposed smartfee changes, which would drop low-priority or low-fee transactions. In short, giving peers feedback about why their blocks or transactions are dropped or why they are being banned should help interoperability between different implementations, and will give SPV (simplified payment verification) clients feedback when their transactions are rejected due to insufficient priority or fees. See the gist for details, I'm looking for feedback and planning on implementing this before circling back to finish the 'smart fee' work: https://gist.github.com/gavinandresen/7079034 -- -- Gavin Andresen -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Feedback requested: reject p2p message
Would it make sense to use either fixed length strings or maybe even enums?On Oct 25, 2013, at 05:34 PM, Gavin Andresen gavinandre...@gmail.com wrote:Mike Hearn has been lobbying for an "error" message in the Bitcoin p2p protocol for years (at least since the "ban peers if they send us garbage" denial-of-service mitigation code was pull-requested). This came up again with my proposed "smartfee" changes, which would drop low-priority or low-fee transactions. -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Feedback requested: reject p2p message
On Oct 26, 2013, at 11:01 AM, Jean-Paul Kogelman jeanpaulkogel...@me.com wrote: Would it make sense to use either fixed length strings or maybe even enums? No. Enums or fixed length strings just make it harder to extend, for no benefit (bandwidth of 'reject' messages doesn't matter, they will be rare and are not relayed). -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] Payment protocol for onion URLs.
One limitation of the payment protocol as speced is that there is no way for a hidden service site to make use of its full authentication capability because they are unable to get SSL certificates issued to them. A tor hidden service (onion site) is controlled by an RSA key. It would be trivial to pack a tor HS pubkey into a self-signed x509 certificate with the cn set to f.onion. If we specified in the payment protocol an additional validation procedure for [base32].onion hosts that just has it hash and base32 encode the pubkey (as tor does) then the payment protocol could work seamlessly with tor hosts. (Displaying that the payment request came from f.onion). I believe that the additional code for this would be trivial (and I'll write it if there is support for making this a standard feature). This would give us an fully supported option which is completely CA free... it would only work for tor sites, but the people concerned about CA trechery are likely to want to use tor in any case. Thoughts? -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Payment protocol for onion URLs.
On Saturday, October 26, 2013 3:31:05 AM Gregory Maxwell wrote: One limitation of the payment protocol as speced is that there is no way for a hidden service site to make use of its full authentication capability because they are unable to get SSL certificates issued to them. A tor hidden service (onion site) is controlled by an RSA key. It would be trivial to pack a tor HS pubkey into a self-signed x509 certificate with the cn set to f.onion. ... Thoughts? Is there any point to additional encryption over tor (which afaik is already encrypted end-to-end)? Is there a safe way to make this work through tor entry nodes/gateways? It'd be nice to have a way to support namecoin-provided keys too... Luke -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Payment protocol for onion URLs.
On Sat, Oct 26, 2013 at 1:31 PM, Gregory Maxwell gmaxw...@gmail.com wrote: This would give us an fully supported option which is completely CA free... it would only work for tor sites, but the people concerned about CA trechery are likely to want to use tor in any case. Thoughts? I think a tiny number of people would use it, so from a purely engineering priority perspective my initial reaction is not worth it. However, as a demonstration of the flexibility of the payment protocol and because it is a really nifty idea that will give lots of people warm fuzzies I think you should do it and we should pull it. -- -- Gavin Andresen -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Payment protocol for onion URLs.
On Fri, Oct 25, 2013 at 8:41 PM, Luke-Jr l...@dashjr.org wrote: Is there any point to additional encryption over tor (which afaik is already encrypted end-to-end)? Is there a safe way to make this work through tor entry nodes/gateways? The x.509 in the payment protocol itself is for authentication and non-repudiation, not confidentiality. It's used to sign the payment request so that later there is cryptographic evidence in the event of a dispute: He didn't send me my alpaca socks! Thats not the address I told you to pay! He told me he'd send my 99 red-balloons, not just one! No way, that was the price for 1 red-balloon! Just using SSL or .onion (or whatever else) gets you confidentiality and authentication. Neither of these things get you non-repudiation. It'd be nice to have a way to support namecoin-provided keys too... The payment protocol is extensible, so I hope that someday someone will support namecoin authenticated messages (but note: this requires namecoin to support trust-free SPV resolvers, otherwise there is no way to extract a compact proof that can be stuck into a payment request) and GPG authenticated messages. But those things will require a fair amount of code (even fixing the namecoin protocol in the nmc case), and GPG could be done just by externally signing the actual payment request like you'd sign any file... and considering the sorry state of their _practical_ usability, I don't think they're worth doing at this time. By contrast, I _think_ the tor onion support would require only a relatively few lines of code since it could just be the existing x.509 mechanism with just a simple special validation rule for .onion, plus a little tool to repack the keys. I think it would easily be more widely used than namecoin (though probably both would not really be used, as gavin notes). w/ Gavin's comments I'll go check in with the tor folks and see if anyone has ever though of doing this before and if there is already a canonical structure for the x.509 certs used in this way. -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Payment protocol for onion URLs.
On Fri, Oct 25, 2013 at 08:31:05PM -0700, Gregory Maxwell wrote: One limitation of the payment protocol as speced is that there is no way for a hidden service site to make use of its full authentication capability because they are unable to get SSL certificates issued to them. A tor hidden service (onion site) is controlled by an RSA key. It would be trivial to pack a tor HS pubkey into a self-signed x509 certificate with the cn set to f.onion. If we specified in the payment protocol an additional validation procedure for [base32].onion hosts that just has it hash and base32 encode the pubkey (as tor does) then the payment protocol could work seamlessly with tor hosts. (Displaying that the payment request came from f.onion). I believe that the additional code for this would be trivial (and I'll write it if there is support for making this a standard feature). This would give us an fully supported option which is completely CA free... it would only work for tor sites, but the people concerned about CA trechery are likely to want to use tor in any case. Thoughts? Strong ACK on the basis of responding for forum trolls alone. It's easy enough to make it a genuinely useful tool for multisig wallets too: keep a copy of your Tor URL bookmarks on your second signing computer. So long as either computer has the correct URL you're safe. -- 'peter'[:-1]@petertodd.org 0006fbd917e8b4770c566dbc8ed4bedd00f441286ffb6e7f73ac signature.asc Description: Digital signature -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Feedback requested: reject p2p message
The HTTP status code system seems to work well enough, and seems to give the best of both worlds. A 3 digit numeric code that is machine-readable, and a freeform text note for humans. The clever part about that system was in realizing that the numeric codes didn't need to account for every possible error. They just need to give the other node the most useful information, like try that again later, I'm having a temporary problem vs. That is just plain wrong and it will still be wrong next time too, so don't bother to retry. We can leave it to the humans to puzzle out the meaning of 403: values of txid gives rise to dom! Gavin wrote: On Oct 26, 2013, at 11:01 AM, Jean-Paul Kogelman jeanpaulkogel...@me.com wrote: Would it make sense to use either fixed length strings or maybe even enums? No. Enums or fixed length strings just make it harder to extend, for no benefit (bandwidth of 'reject' messages doesn't matter, they will be rare and are not relayed). -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
